The case of the big AD LDS database

recently we were working with customers, who did run into issues where the AD LDS database grew very large and they ended up with a full disc on System Drive.

The AD LDS database files can be found in %windir%\ADAM. The file which did grow quite big is the adamntds.dit.

The AD LDS on TMG is used to save and distribute the TMG configuration. Back in ISA days, you could only find it for Enterprise Edition Servers, in TMG every server has AD LDS installed, but only the Master Node or the EMS has a running and active AD LDS instance running. The AD LDS instance is called ISASTGCTRL.

When analyzing the customers environment, we found, that they were applying multiple configuration changes every hour, by running a custom script which modified the configuration in a way, that some elements were deleted and added.

By design AD LDS keeps track on all deleted records though. This lead to the scenario where the database file grew more and more, as all deleted records were not permanently deleted by only flagged as “to be deleted”.

When we did run the command

Ldifde –m –f exportfpc2.ldf –s localhost –t 2171 –d “CN=FPC2”

we did see a quite small export file of the configuration, which can be some MB depending on the configuration.

When you execute ldifde and add the –X parameter to also show deleted or tombstone objects like this:

Ldifde.exe –x -m -f exportfpc2.ldf -s localhost -t 2171 -d "CN=FPC2" you might see that the output file can grow very large indeed.

 

The reason for this behavior is, that the AD LDS is configured with a default Tombstone lifetime of 180 days. Therefore all objects remain in the database file until this tombstone lifetime had been exceeded.

Please be aware, that the Tombstone lifetime also controls the amount of days, which can be between a sync of the AD LDS for the case you have multiple AD LDS Server, e.g. in a EMS scenario.

Please visit these sites or some useful information about the tombstone lifetime:

Scenario Overview for Restoring Deleted Active Directory Objects - http://technet.microsoft.com/en-us/library/dd379542(v=WS.10).aspx

Appendix A: Additional Active Directory Recycle Bin Tasks - http://technet.microsoft.com/en-us/library/dcf3431a-c562-447f-a591-4742d2bcdbfa(v=ws.10)#BKMK_1

Depending on your backup strategy for AD LDS you might want to decrease this value if you face the issue, that the AD LDS is using all your Systemdrive disc space. As a rule of thumb the Tombstone Lifetime should not be smaller than the period of days between every full backup of AD LDS. E.g. Full Backup every 7 days -> Tombstone should be at least 8+ days. The minimum value you can define is 3 days.

You can modify the Tombstone lifetime like this:

On TMG where AD LDS is running, open ADSI Edit and connect to the local AD LDS -> localhost:2171 in Well-known-Context: “Configuration”

In Services->Windows NT -> Directory Services – right-click Directory Services and search for tombstoneLifetime. By default this is set to 180, depending on the customers backup strategy they can decrease this value. As a rule of thumb the Tombstone Lifetime should not be smaller than the period of days between every full backup of AD LDS. E.g. Full Backup every 7 days -> Tombstone should be 8+ days.

After the change, you need to restart the ISASTGCTRL to apply the change.

To manually trigger a garbage collection, you need to open ldp.exe on the host and connect to localhost:2171 again. Please make sure to connect using a local admin or AD LDS admin.

Next choose Browse -> Modify.

In Attribute add dogarbagecollection with value „1“. Next click Enter and afterwards run.

Depending on the size of the DB, the garbage collection might not be able to clean all records at once / it might take some days to completely clean up the records.

To monitor the garbage collection, you can increase the tracing using by changing the registry.

Important This section, method, or task contains steps that tell you how to modify the registry. However, serious problems might occur if you modify the registry incorrectly. Therefore, make sure that you follow these steps carefully. For added protection, back up the registry before you modify it. Then, you can restore the registry if a problem occurs. For more information about how to back up and restore the registry, click the following article number to view the article in the Microsoft Knowledge Base:

How to back up and restore the registry in Windows 322756 (http://support.microsoft.com/kb/322756/)

The RegKey you need to look for is HKLM\SYSTEM\CCS\services\ADAM_ISASTGCTRL\Diagnostics you can change the value of the key „6 Garbage collection” to 5 for purposes of monitoring/troubleshooting.

Afterwards you can monitor the events in Application and Services -> ADAM (ISASTGCTRL).

Please be aware the garbage collection is usually triggered once a day, therefore you do not need to trigger the garbage collection manually.

Afterwards you can check the size of the database files on your disc again, and you will realize, that the size did not change… at least it should not grow any more now

In order to free up some disc space, you will need to perform an offline defragmentation of the AD LDS as described here: http://support.microsoft.com/kb/232122 or http://support.microsoft.com/kb/930832

If you decide to use esentutl, the command looks like this: esentutl.exe /d adamntds.dit

Hope that helps!

Author:

Philipp Sand

Microsoft CSS Forefront Security Edge Team

Comments

• Anonymous
  January 01, 2003
  The case of the big AD LDS database
  thank you
• Anonymous
  January 01, 2003
  thanks
• Anonymous
  December 15, 2013
  Pingback from NeWay Technologies – Weekly Newsletter #73 – December 12, 2013 | NeWay
• Anonymous
  December 15, 2013
  Pingback from NeWay Technologies – Weekly Newsletter #73 – December 13, 2013 | NeWay
• Anonymous
  December 17, 2013
  226 Microsoft Team blogs searched, 52 blogs have new articles. 163 new articles found searching from
• Anonymous
  September 19, 2014
  This was a very good and informative article.
• Anonymous
  November 25, 2014
  Thanks a lot! This article is really practical and useful. Really appreciate it!