An upgrade that saves space

It’s always great to show all of the new, innovative things a product can do. But sometimes it’s great to show how you are doing what you did before, but now you can do it better, faster and more efficiently than you did in the previous version. There’s something to be said for that.

 

A good example of this is the DIT shrinkage people observe after a 2000->2003 in-place upgrade of a domain controller. In 2003, we began to single instance store (SIS) our security descriptors. In a nutshell, we took what were a series of large elements (security descriptors) and no longer store them with the objects themselves. Rather, we now store them in a special table in the DIT, and each object has a pointer to the appropriate element in that table.

 

Why does this save space? Well, in many cases, people do not have a unique security descriptor for every object in their forest. If you did, you probably have a lot of management work to do, and probably don’t have time to read this post. :) From what we’ve seen, people tend to have security descriptors on containers/OUs that propagate down to elements within those containers. This tells us that many objects have the same security descriptor as many other objects. Since SDs are large and a pointer to an SD can be made quite small, there is an efficiency to be had here.

 

How do I sign up for this wonderful feature? How do I get a copy of my very own? Well, the beauty of the system is that you get it automatically post-upgrade of the domain controller. Note that this is a per-dc feature and is not dependent upon any domain or forest functional levels.

 

After the upgrade and final reboot of the DC, keep your eye on the DS event log. After some period of time (depending upon DC load, the hardware specs on the machine and the size of the DIT; typically in the order of minutes), you’ll notice this event:

Event Type: Information
Event Source: NTDS SDPROP
Event Category: Internal Processing
Event ID: 1966
Date: MM/DD/YYYY
Time: HH:MM:SS AM|PM
User: NT AUTHORITY\ANONYMOUS LOGON
Computer: <computername>
Description: The security descriptor propagator has completed a full propagation pass.
Allocated space (MB):
XX Free space (MB): XX

This may have increased free space in the Active Directory database.
User Action: Consider defragmenting the database offline to reclaim the free space that may be available in the Active Directory database. For more information, see Help and Support Center at https://go.microsoft.com/fwlink/events.asp.

This is the magical moment! This means that SDProp has finished the SIS work needed for this optimization. It’s party time.

 

At this point, are you done? If you want to be, sure. You now have a lot of extra space in the DIT, and over time as new data is added to the directory the DIT will not grow (online defrag will move things around and mark this new space white space, and it will then be reused going forward). The directory will reuse this white space and you should observe little to no growth until you have used it all up.

 

If you want to reclaim the space, you certainly can do so. Given the way AD/ESE is implemented today, online defragmentation will not actually shrink the database, only mark it as available such that it can be reused. To actually reclaim the disk space and cause the DIT to shrink, reboot the DC to DS repair mode and perform an offline defrag. Information on how to do that can be found in the usual places (KB, MSDN, etc.).

 

How much space do you save? Well that does depend upon your particular environment and how you use the product. Mileage may vary, but reports have been anywhere from 15% to 45% space savings.

 

There’s definitely a Seinfeld reference to be made here. I’ll leave that to the reader.

Comments

• Anonymous
  October 02, 2004
  The comment has been removed
• Anonymous
  October 02, 2004
  Great post ~Eric. Keep them coming. Heres a topic idea... Discussion around maximum record sizes and administrative limits on objects. And aother... Can indexing hurt you? And... Is there any benefit to indexing objectclass and why does it help? That should keep you going for a bit. :o)
• Anonymous
  October 02, 2004
  Hi Eric, great info - I wasn't aware of this but greatly apprechiate it. Curious to see more.