PowerShell Module for Working with AD SID History Updated

TechReady

At Microsoft we have an internal event called TechReady that happens twice each year.  It is similar to our public TechEd event.  Approximately 5,000 Microsoft technical field people from all over the world gather to learn the details of the latest product releases and dialog with the product groups.  It is a fantastic event, and this July I'll be presenting a session called "Active Directory PowerShell Notes From the Field".  I'll be covering three case studies related to schema updates, DNS migrations, and Active Directory SID history.  I will conclude the session with a quick look at the future of DCPROMO, REPADMIN, and DNSCMD in Windows Server 2012, comparing the new PowerShell ways to do Active Directory administration in the next release.  If you are going to TechReady then look me up.

More SID History

I get emails frequently from folks who have read my series of articles on Active Directory SID history, so I'm guessing that is a good theme to continue.  Working with a customer recently I was able to enhance the functionality in the Active Directory SID history PowerShell module that I posted last December.  Today's post will publish and discuss those improvements.

Version 1.4
June, 2012

Functions added in this release:
Export-SIDHistoryShare
Merge-CSV

Functions modified in this release:
Convert-SIDHistoryNTFS
Export-SIDMapping
Update-SIDMapping
Get-SIDHistory

Fixes:
Removed Test-Path validation on NewReport parameter of Update-SIDMapping.
Added file validation for DomainFile parameter of Get-SIDHistory.

It's All About The Customer

Imagine if…

• your company acquired an average of 12 other companies each year.
• your forest had over 35 domains (consolidated from 50+).
• your forest had over 80 trusts to other domains you still need to migrate.
• your forest had over 172,000 instances of SID history.
• you lived in the ADMT from sun up to sun down.
• token bloat plagued your support engineers with mystery troubleshooting.

Obviously I can't make this stuff up.  Those points describe a customer who recently invited me to help them develop a process to identify and remediate SID history in their forest.  This also gave them a process going forward that would help them do future domain migrations in a way that would minimize the impact of SID history.

Enter PowerShell

I knew the SID history PowerShell module could help them, but I also knew that some improvements would be necessary to help them scale to the size of data collection they would need.  Here's what I did:

• I added a function called Export-SIDHistoryShare that will collect SID history information from share permissions.  (Note that it will not convert SID history as the NTFS function does.)
• I modified the NTFS SID translation report CSV generated by Convert-SIDHistoryNTFS so that the columns would match the output from the new share CSV output.  This involved adding ACLType and DateTimeStamp.
• Both NTFS and Share reports now include a date time stamp so that the CSV data can be imported into a larger database multiple times.  Different scans of the same data will be distinguished by the date time stamp.
• I modified the Export-SIDMapping and Update-SIDMapping to include a date time stamp field as well for the same reasons.
• I created a new function called Merge-CSV that will take all of the NTFS and Share SID history reports and roll them into a single, large CSV file for importing to a database (like Microsoft Access or Microsoft SQL).
• This is huge! I created a quick proof-of-concept Access database that would relate the forest object SID history data against the SID history discovered in shares and NTFS permissions on servers. This is super powerful for analyzing the impact of SID history. Now you can produce reports that tell you:
  • which former domain has the most SID history
  • which groups with SID history are most used on your file servers
  • which file servers need ACL translation with the ADMT
  • where to target your remediation efforts for the largest impact
  • where to begin the cleanup with the users and groups whose SID history is not in use
  • etc.
• I also resolved one issue with the NewReport parameter of the Update-SIDMapping function and resolved one issue with the DomainFile parameter of the Get-SIDHistory function. (Special thanks to Andrew Hill and Greg Jaworski for their feedback.)

These changes added the scalability and flexibility for the customer to begin inventorying hundreds for servers for SID history and manage all the data with a full database they would implement later on their own.  I love the simplicity of CSV in PowerShell!

The New Process

My last post on SID history has the overall steps for installing the module and completing the SID history remediation so I will not repeat that content here.  But I do want to list out the steps for you to automate the NTFS and share SID history data collection:

• Inventory SID history in share and NTFS ACLs:
  • Run these commands against the servers and share paths where you want to check for
    SID history:
    • Convert-SIDHistoryNTFS \\servername\share\path –WhatIf
    • Export-SIDHistoryShare servername
    • NOTE: You will need to run this under an administrative account that has permissions to view all of the recursive subfolders.
  • Repeat these steps for all servers where you want to collect data.  See the example code below for another way to automate mass data collection.
  • Now merge all of the output into a single CSV file for analysis:
    • Put all of the Share and NTFS CSV output files into a new working folder.
    • Merge-CSV –Path "C:\Temp\WorkingFolder"
    • This creates a merged CSV called ACL_SID_History_xxxxxxxxxxxxxxxx.csv.
• Copy these two files to the folder with the provided Access database:
  • SIDReportUpdated.csv
  • ACL_SID_History_xxxxxxxxxxxxxxxx.csv
• Rename the ACL_SID_History_xxxxxxxxxxxxxxxx.csv file to ACL_SID_History.csv.
• Open the Access database.
• In order for the linked tables in Access to see these new files you must repair the links and point them to the files in your working folder.
  • Right click the table ACL_SID_History on the left.
  • Choose “Linked Table Manager”.
  • Check the box at the bottom “Always prompt for new location”.
  • Check the box beside each of the two tables.
  • Click OK.
  • Browse to the each of the two CSV files that you just copied into the database folder. Pay special attention to the title bar of the File browser dialog box. Make sure that you choose the file that matches the name in the title bar.
  • Close the Linked Table Manager dialog box.
• Now you can double click any of the example queries on the left to see the data analysis.  You can also create your own queries and custom reports.

Automating the Automation

It would be quite time consuming to run the NTFS and share scan commands one at a time against all of your servers.  Instead try these handy PowerShell routines to make the data collection go faster.  As these processes usually take hours to scan large file shares it would be a good idea to let them run over a night or weekend.

 # Sample code to scan a group of Windows servers in a text file.
$Servers = Get-Content ".\servers.txt"
ForEach ($ComputerName in $Servers) {
    Export-SIDHistoryShare $ComputerName
    # Grab only file shares that were manually created
    $Shares = Get-WmiObject Win32_Share -ComputerName $ComputerName -Filter "Type=0"
    ForEach ($Share in $Shares) {
        Convert-SIDHistoryNTFS "\\$ComputerName\$($Share.Name)" -WhatIf
    }
}

# Sample code to scan a list of share paths in a text file.
# This can work for Windows or NAS servers.
$Shares = Get-Content ".\shares.txt"
ForEach ($Share in $Shares) {
    Convert-SIDHistoryNTFS $Share -WhatIf
}

Where can I get all this PowerShell goodness?

I have updated this code at the TechNet Script Gallery.  Attached to that entry you'll find the following in a single compressed file:

• Updated PowerShell module for AD SID history
• Sample output data files
• Sample Access database (You must update the linked tables to point to the CSV files.  See instructions above.)
• Sample script file for using the module

Extract all of the files into a working folder.  Install the module.  The usual disclaimers apply: this is sample code for use at your own risk.  Enjoy!

But Wait!  There's More…

If you would like me or another Microsoft PFE to visit your company and help you with the ideas discussed on this blog, please contact your Premier Technical Account Manager (TAM).  We would love to come see you.

Follow me on Twitter: @GoateePFE.

Comments

• Anonymous
  June 15, 2012
  super awesome!

• Anonymous
  August 19, 2014
  I'm using your PS routine in this post to run this against multiple shares. Removing the - WhatIf, the script then prompts for confirmation of each share. I need to suppress that message so it'll run through the shares.txt without expecting a response. I've tried -Confirm:$false when calling Convert-SIDHistoryNTFS, but no luck.
  

• Anonymous
  August 20, 2014
  Hi Rich,
  Can you paste in an example of the console output you are getting? I need to see what action is requesting the confirmation. Is it Set-ACL? That would help me troubleshoot. You can also use the "Email blog author" link (top right) if you want to discuss it privately.
  Ashley @GoateePFE

• Anonymous
  August 21, 2014
  I'm using this code for the .ps1 from your blog
  
  $Shares = Get-Content ".shares.txt"
  ForEach ($Share in $Shares) {
  Convert-SIDHistoryNTFS $Share
  }
  
  "Note I removed the -WhatIf"
  
  It's the same confirmation prompt you get when you run Convert-SIDHistoryNTFS from the PS prompt without the -WhatIf.
  So for each share the above code reads from the shares.txt file, it prompts me for confirmation.
  
  I ended up modifying the code in your module, and just commented out that confirmation check, but if there's a better solution I'd rather use that. I'm guessing you would need to add a check for a -Y switch for the Convert-SIDHistoryNTFS command that could be used when using that command in batch.
  
  Thanks

• Anonymous
  August 21, 2014
  The comment has been removed