Be sure to use paging in membership providers for the MOSS Profile Import Tool...

Just a quick lesson learned on the MOSS Profile Import Tool to save you import time and resources.

The Profile Import Tool's current release has a hard-coded paging algorithm that attempts to page through any membership provider's GetAllUsers method by retrieving 1000 users at a time. If you DON'T support paging your GetAllUsers method - meaning you try to retrieve all users at once - then the import algorithm will retrieve and import all users N times, where N = (# users / 1000) . This becomes totally unmanageable - but a really good test for memory leaks in your provider - if you have many thousands of users... :)

The best practice is (if able) to add paging to your membership provider, since that will be useful in all calls to the GetAllUsers, FindUsersByName, and FindUsersByEmail methods, all of which will be used by SharePoint.

If you feel the need to alter this aspect of the Profile Import Tool's behavior - i.e. you have 100K users and don't want to take the time to build paging logic into your provider, or don't have access to the membership provider's source code - you'll find the paging logic in the ImportProvider.cs class. All you have to do is change the value of the pageSize constant to a sufficiently large value (i.e. 101000 if you have 100000 users), and you'll bypass the paging:

privateconstint pageSize = 1000;

If you choose to increase the pageSize value, make sure you have enough RAM to hold all those user records... and don't run the profile import application on a server that's hosting a RAM-intensive role like Query, Web App, or SQL.