How it works: MOSS 2007 automatic user profile removal

I have seen many posts filled with myths and legends how this actually works in MOSS 2007. This post is based on information received from SharePoint product group, my own research, debugging and testing to clarify the internals of this feature.

In MOSS 2007 the inactive user profiles are deleted by a timer job called “My Site Cleanup Job”.

This new job was the product group’s answer for customer feedback about the problems with SPS 2003 user profile removal to make it more robust.

The job runs once every hour which confused many people who thought that 3 full imports will delete users in MOSS 2007 as it was in SPS 2003. It is not the case anymore. You can do as many full imports as you like, if you disable this job, no user will be removed from the inactive user list. Since it runs hourly and full import can be long, 3 runs can take about 1 hour and it seems the full import did the trick, but in fact it did not.

To understand how this new feature works let’s start from the basics.

During the user import process (crawl) if MOSS cannot find a user in AD/LDAP directory it marks the user deleted in the SSP user profile store without removing it.

You can check these users in the SSP administration site under user profiles and properties on the View user profiles page selecting the “Profiles Missing From Import” view. You can delete the profiles here manually.

This list is the input for the “My Site Cleanup Job”.

Let’s dive into the details.

The job in fact does two things every hour:

1. Updates all personal sites and sets the mysite host's portal url as portal url on all mysites. This way if you defined your company intranet as portalurl on the host, users’s mysites will have a top breadcrumb pointing to the company intranet.
2. Processes pending user profile deletes using the “profiles missing from import” list

The following steps happen during user profile removal:

1. Using the account name of the to be deleted user the job fetches the user profile

Checks if the user is active using all defined import connections defined in this SSP

1. LDAP connection is created to search for the user
   1. using it’s Username (just the user without the domain\) for Active directory connections. The filter is samaccountname=user and the filter which was defined in the connection.
   2. using the user portion of the AccountName , for eg user is used for an AccountName of “membershipprovider:user”. The filter is uid=user and the filter which was defined in the connection.
2. After MOSS 2007 Sp2 if the user’s domain cannot be contacted the user is assumed inactive and user profile is removed.
   1. UPDATE: In 2009 December MOSS Cumulative update there is a change in behavior in the previous step. Now it is possible to control how aggressive this job will be on user profile removal with the stsadm -o sync command. From 2009 December CU by default if the domain controller cannot be contacted, the user is not considered missing. It is only considered missing if the domain controller can be contacted and the DC says that the user does not exist. To turn back the SP2 behavior of aggressive delete in case DC cannot be contacted, you must run stsadm -o sync -AggressiveMySiteCleanup 1 . Thanks for my colleague Jose Vigenor to draw my attention to this recent change.
3. If all connections return zero results then the user is assumed inactive and the profile is removed.
4. Just before the actual profile delete happens, the profile delete event handler is called. The event handler can cancel the deletion if it returns false in its PreProfileDeleted method implementation.
   1. The out-of-the-box event handler takes the manager of the to-be-deleted user and sets that user (if found) as the owner of the user’s my site.
   2. The manager gets an email with a subject of “The My Site of username is scheduled for deletion” and the url of the mysite.
   3. The event handler returns true for all users, there is no filtering.
5. If the user is found in any of the import connections its deleted status is removed and the user is set active in the SSP.

Troubleshooting

To troubleshoot this feature you need to increase the trace level of “User Profiles” ULS category in central administration / operations / diagnostic logging.

Alternatively you can use stsadm to set it:

stsadm -o setlogginglevel -category "User Profiles" -tracelevel Verbose

Then verify all lines with “MySiteCleanup:” to follow what the job is doing.

I have to mention a special case of problem which is difficult to figure out. When an admin defines an import connection which uses a custom account, MOSS stores this setting in two locations. When you save the setting, a crawl rule is created for the Profile import project in the registry – since the user profile import is in fact a crawl, this is somehow expected. Furthermore the regularly called Synchronize method stores/updates this account information in the configuration database as well which is used by the “My Site Cleanup Job”. Sometimes these accounts gets out of sync and the “My Site Cleanup Job” tries to validate a user with invalid connection credentials. In this case usually the user profiles are not deleted automatically. To solve the problem, first of all resolve any exceptions which happen during the Synchronize method which is synchronizing the search settings on all SharePoint machines. Once the errors are gone, you need to delete the recreate the user profile import connections to ensure that the credentials are ready to be created again in the configuration database.

Known issue as of 5/31/2011:

If there are two import connections to two different forests and the same username is used in both, deleting the user from the second forest will be picked up by the profile import correctly but the my site cleanup timerjob will issue an AD query against the first forest as samaccountname=user without the domain part and will find this user active and will restore the marked user from second forest  as active while it does not exists in that AD anymore.

Notes

Automatic profile removal only works for MOSS imported profiles which can be marked as deleted during an import. If you manually add any profiles, those will never will be automatically deleted. Same applies if you add profiles using Object Model. You need to delete these users manually or with Object Model.

UPDATE: I received a lot of questions around actual my site deletion. I would like to emphasize that the  “My Site Cleanup Job” - although its name might suggest it - does not delete actual my sites. It only removes the user profile from the SSP profile store and changes the my site owner to the user’s manager if there is one. The my site site collection will not get deleted by this job. In order to get to a my site which belongs to a deleted user, you have to type the actual my site url directly, since the user profile has been deleted, you cannot get there using person.aspx?accountname=domain\user – it will display user not found as expected. You have to know the direct url or check the my site naming convention on the SSP admin page and figure out the url yourself.

There is an independent feature for automatic site deletion which can be enabled for a web application which is not discussed in this post. It is called “SIte Use Confirmation and Deletion” and can be found under Application management in Central administration. That feature will apply to any site collection in the web application which is idle, not necessarily to my sites which belong to a removed user profile.