Sharepoint 2016 - MysiteCleanup Job functionality changes

Here is a Quick look at what's new  with the  Mysite Cleanup job in Sharepoint On-Prem  2016 &  how it impacts the  functionality of  profile deletion & cleanup .

How this worked in Sharepoint 2010 / Sharepoint 2013 ?

The user profile in the profile store can be marked for deletion (bdeleted=1) when the actual user is deleted or it meets a User filter in FIM (eg:UserAccountControl bitonequal 2). When the mysite clean up job is executed, such profiles are immediately deleted and corresponding mysites are kept for 14 days for data recovery.

More Info : https://blogs.msdn.microsoft.com/kaevans/2012/06/25/inside-the-sharepoint-2010-my-site-cleanup-timer-job/

Note : While AD import is configured, you will need to execute a set of commands to get the disabled / deleted users marked for deletion

Ref: https://blogs.msdn.microsoft.com/spses/2015/03/04/sharepoint-2013-active-directory-import-and-known-behaviors/ and https://blogs.msdn.microsoft.com/spses/2014/04/13/sharepoint-2013-adimport-is-not-cleaning-up-user-profiles-in-sharepoint-whose-ad-accounts-are-disabled/

Whats new in Sharepoint 2016 ?

The SharePoint 2016 works similar to Sharepoint 2013 , however the profiles marked for deletion (bdeleted=1) will not be immediately deleted, such profiles will be preserved for 30 days.

This is how a profile look like when its active.

 

 

 

 

When the user is marked for deletion , the bdeleted and NTName values are updated. The NTName column value will be appended with the UserID.

 

 

 

 

To calculate the 30 days retention , we have introduced a new table (upa.userprofilescheduledforremoval) and the value will be populated when the user is marked for deletion. When the mysite Clean up job runs , it queries this table and picks up the profiles that are more than 30 days and process them.

 

 

 

 

Note : It is not recommended to Query any Sharepoint databases manually or to make changes other than ones described at https://support.microsoft.com/kb/841057 .

I see the Profiles in my farm are never  Deleted , Even after 30 days & Successful Run of Mysite Cleanup Job ?

We  at Microsoft are already aware of this issue  &  working  to  get this addressed in one our upcoming updates .  Such profiles  can be removed manually, using Central Admin  or   PowerShell script as needed .  We will update this post once we have more info on the fix .

 

Update [Aug 9, 2017] : The above mentioned issue with MysiteCleanup job on Sharepoint 2016 has been resolved in Aug 2017 CU.

Download Sharepoint 2016 updates : https://technet.microsoft.com/en-us/library/mt715807(v=office.16).aspx\#BKMK\_2016

Please test , plan and review , back up the data and then install the CU on production farms.

 

POST By : Manjesh Menon [MSFT]

Comments

• Anonymous
  May 23, 2017
  Thanks for the update . We have large no of disabled User profiles not getting deleted even after Running Mysite cleanup job in our SharePoint 2016 environment running MIM . Do you have PowerShell script somewhere to delete those till Microsoft releases the fix?
  • Anonymous
    May 24, 2017
    Hi MP,Below is the sample script that can be used to remove the profiles only. You may need to modify it to remove the Mysites if you wish to do so. You may need to prepare an input CSV file (Profiles_tobe_deleted.csv) copying the result of SQL query towards the profile DB "select ntname from upa.userprofile_full where bdeleted=1 (nolock)". Its pretty important to have proper back up of the databases before you proceed.Disclaimer: Sample Code is provided for the purpose of illustration only and is not intended to be used in a production environment. THIS SAMPLE CODE AND ANY RELATED INFORMATION ARE PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR PURPOSE. We grant You a nonexclusive, royalty-free right to use and modify the Sample Code and to reproduce and distribute the object code form of the Sample Code, provided that. You agree: (i) to not use Our name, logo, or trademarks to market Your software product in which the Sample Code is embedded; (ii) to include a valid copyright notice on Your software product in which the Sample Code is embedded; and (iii) to indemnify, hold harmless, and defend Us and Our suppliers from and against any claims or lawsuits, including attorneys’ fees, that arise or result from the use or distribution of the Sample Code.# Adding PS Snapin SharePoint$snapin = Get-PSSnapin | Where-Object {$.Name -eq 'Microsoft.SharePoint.Powershell'}if ($snapin -eq $null){ Write-Host "Loading SharePoint Powershell Snapin" -ForegroundColor Green Add-PSSnapin "Microsoft.SharePoint.Powershell"}# Importing the set of users. $fileName = "C:\Profiles_tobe_deleted.csv"; $inputFile = "";try { $inputFile = Import-CSV $fileName -Delimiter ',' -Header NTName} catch { Write-Error "Could not open file $fileName" exit;} # Delcaring the Sharepoint Variables. $site = new-object Microsoft.SharePoint.SPSite("http://sps16app"); $ServiceContext = [Microsoft.SharePoint.SPServiceContext]::GetContext($site); $pm = new-object Microsoft.Office.Server.UserProfiles.UserProfileManager($ServiceContext) # Declaring and creating the log files. Each time the script is executed, a new file will be created with the current time in the filename.$dateTime=Get-Date -format "dd-MMM-yyyy HH-mm-ss"$UPLogFile="UserProfiles_Remove_bdeleted"+""+ $dateTime + ".log"$inputFile | Foreach-Object($){$User=$.ntname try { $profile = $pm.GetUserProfile($User) $DisplayName = $profile.DisplayName Write-host "Current User:" $DisplayName $messageDisplayname = "Current User:" + $DisplayName Add-Content -Path $UPLogFile -Value $messageDisplayname $AccountName = $profile[[Microsoft.Office.Server.UserProfiles.PropertyConstants]::AccountName].Value $id=$profile.ID Write-host "ID for the user :" $DisplayName "is " $id $messageid = "ID for the user :" + $DisplayName + "is " + $id Add-Content -Path $UPLogFile -Value $messageid Write-host "Removing the Profile.." $messageremove = "Removing the Profile.." Add-Content -Path $UPLogFile -Value $messageremove try { $pm.RemoveUserProfile($id) Write-host "Successfully Removed the Profile" $AccountName $messagesuccess = "Successfully Removed the Profile" + $AccountName Add-Content -Path $UPLogFile -Value $messagesuccess Add-Content -Path $UPLogFile -Value " " } catch { Write-host "Failed to remove the profile " $AccountName $messagefail = "Failed to remove the profile " + $AccountName Add-Content -Path $UPLogFile -Value $messagefail Add-Content -Path $UPLogFile -Value " " } } catch { Write-host "Exception when handling the user " $User $messageexcp = "Exception when handling the user " + $User Add-Content -Path $UPLogFile -Value $messageexcp Add-Content -Path $UPLogFile -Value " " }}
    • Anonymous
      September 21, 2017
      Thanks for the script . I installed feature Pack2 in our Test environment and it seems to have fixed Mysitecleanup job issue . I also changed MIM filter ( to bit on equals 546) so it can properly sync disabled accounts in AD. I can now see mysite deletion emails being sent to Managers . The problem is if we run this in production it will send hundreds of emails to managers for employees that left the company long time back . Is there a way to change the timer job to send email to specific account spadmin@abc.com rather than the manager ?
      • Anonymous
        September 24, 2017
        Hello MP, Unfortunately there is no way we can control this - we can completely disable the outgoing emails so that the managers will NOT be updated about the deletion, however there will be a reminder email 3 days prior to the deletion as well. Keeping the outgoing email settings for longer days may impact the "alert" features too. Hence you can modify the script in such a way that , once the profile is removed, delete the corresponding mysite as well. In the same script , you may add the following lines right before $pm.RemoveUserProfile($id)$acctprofile = $profileManager.ResolveProfile($AccountName)$acctprofile.PersonalSite.Delete()Further checks for the personal site existence before deleting will help the Script run better.
• Anonymous
  June 05, 2017
  Hello Manjesh,Do the entries in the userprofilescheduledforremoval table get cleared when the user profile is re-imported? Does it prevent the deletion of mysites in such cases unlike in previous SharePoint versions when once marked for deletion, the mysite will remain queued for deletion and will be deleted no matter what.Regards,Aneesh
  • Anonymous
    June 07, 2017
    Hi Aneesh, As of today, our tests shows that the entry in the userprofilescheduledforremoval table will be removed or the value for the schedulestate will be updated once the user is imported back, This will prevent the mysite being deleted. We will further test a couple of scenarios once the above mentioned issue is completely addressed.
• Anonymous
  August 09, 2017
  Issue was resolved in August CU for SP 2016 released today : https://support.microsoft.com/en-us/help/4011049/august-8-2017-update-for-sharepoint-server-2016-kb4011049
• Anonymous
  August 25, 2017
  The comment has been removed
  • Anonymous
    August 25, 2017
    Just an update on the my question above , after I remove the filter query , save connection , import profiles, reenter the same filter to exclude disabled and executing MySiteCleanup job , i can see the same behavior as deleted users. Now question is do i really have to wait for 30 days for these to be gone or any safe way to remove them for further testing ?
    • Anonymous
      August 30, 2017
      Yes Umr. As of now (Aug 2017) its by design that the property is NOT manageable and its hard coded to 30 days. However you may use PowerShell / OM codes to remove such profiles before the wait period if you wish so. Please note that a profile that is marked as Bdeleted=1 will not be crawled, will not be picked up for Org chart etc.
      • Anonymous
        September 05, 2017
        Thanks for your reply . I tried -PurgeNonImportedObjects , hoping it will remove users from "UserProfilesScheduledForRemoval" table but that did not do any thing. Is there any other PS command you are referring to in your response above or same command ? Thanks again.
        • Anonymous
          September 05, 2017
          The comment has been removed