Using the DirSync control

 

Hello world, This is Michael. I’m a member of the Israeli platform team and I’m here to write about some attempts and eventually the success (wohooooo!) of implementing and using the DirSync control from S.DS.P (System.DirectoryServices.Protocols) namespace.

This post will be around developing in C# for AD… I guess it’s also possible in powershell, so if someone will re-write it for PShell let me know and I’ll post it.

So… first of all, what’s the DirSync control and why do we want to use it??

Well, you maybe came across this KB - http://support.microsoft.com/kb/891995 (How to poll for object attribute changes in Active Directory on Windows 2000 and Windows Server 2003) – it will also work for Windows 2008 and R2 BTW.

The DirSync control is an LDAP Control (OID 1.2.840.113556.1.4.841 ) which gives us the option to query our DC only for objects which were changed since the last time we queried it.. pretty cool ha?

Another cool thing about the DirSync control is that you can also query incremental values for multi-valued attributes – meaning, if you add a user to a group you’ll be able to get only that specific change.

A good link explaining a bit more in-depth about the control is here: http://msdn.microsoft.com/en-us/library/ms677626(VS.85).aspx

OK… Now let’s get to the interesting part. First of all the requirements in order to be able to execute a search is the permissions on the partition. Obviously the Domain Admins wouldn’t want everyone that comes into the organization to perform a sync of the Active Directory environment so the exact rights you need is the replicate directory changes:

Now after you gave yourself enough rights you can go into the coding section.

When you perform a DirSync search, you pass in a provider-specific data element (cookie) that identifies the directory state at the time of the previous DirSync search. For the first search, you pass in a null cookie, and the search returns all objects that match the filter. The search also returns a valid cookie. Store the cookie in the same storage that you are synchronizing with the Active Directory server. On subsequent searches, get the cookie from storage and pass it with the search request. The search results now include only the objects and attributes that have changed since the previous state identified by the cookie. The search also returns a new cookie to store for the next search.

In the first part we’ll be using the System.DirectoryServices.Protocols in order to construct the search request:

 string str_dcName = System.DirectoryServices.ActiveDirectory.Domain.GetCurrentDomain().FindDomainController().Name;
System.DirectoryServices.DirectoryEntry rootDSE = new System.DirectoryServices.DirectoryEntry("LDAP://rootDSE");
LdapConnection connection = new LdapConnection(str_dcName);

Note on selecting the DC - For incremental searches, the best practice is to bind to the same domain controller (DC) used in the previous search, that is, the DC that generated the cookie. If the same DC is unavailable, either wait until it is, or bind to a new DC and perform a full synchronization. Store the DNS name of the DC in the secondary storage with the cookie.

You can pass a cookie generated by one DC to a different DC hosting a replica of the same directory partition. There is no chance that a client will miss changes by using a cookie from one DC on another DC. However, it is possible that the search results from the new DC may include reported changes by the old DC; in some cases, the new DC may return all objects and attributes, as with a full synchronization. The client should just make its database consistent with reported search results for any given DirSync call, that is, handle all incremental results as if they were the latest state. It does not matter whether you have seen the change before or are even going back to a previous state because repeated incremental synchronizations will converge on consistency.

Now, After we have connected to the DC we’ll create the first empty cookie

 byte[] cookie = null;
SearchRequest request = new SearchRequest(rootDSE.Properties["defaultNamingContext"].Value.ToString(), "(objectClass=*)", SearchScope.Subtree, null);
DirSyncRequestControl dirSyncRC = new DirSyncRequestControl(cookie, DirectorySynchronizationOptions.IncrementalValues, Int32.MaxValue);
request.Controls.Add(dirSyncRC);

The object we have used is the DirSyncRequestControl (http://msdn.microsoft.com/en-us/library/ms141757.aspx). Regarding the different values – The different DirectorySynchronizationOptions can be found here http://msdn.microsoft.com/en-us/library/system.directoryservices.protocols.directorysynchronizationoptions.aspx.

A note on the last option of the constructor (the attributeCount) – since we cannot combine DirSync with paged searches we’ll basically get a maximum of 1000 objects anyway, so using the MaxValue isn’t as bad idea as it sounds (otherwise you’re using the default which is 1048576 and get unbelievable variety of objects returned in each search).

After we have added the control to the request we can send the request

 SearchResponse searchResponse = (SearchResponse)connection.SendRequest(request);

Now we’re dealing with our first response..obviously you can do whatever you want with the results – in my sample I just write them to the console:

 foreach (SearchResultEntry entry in searchResponse.Entries)
{
   Console.WriteLine("{0}:{1}",
   searchResponse.Entries.IndexOf(entry),
   entry.DistinguishedName);
}

Now the interesting part is how we handle the cookie. Well, first of all as I mentioned we cannot combine paged search and dirsync which means we might have more data coming as part of the same sync request (if I have 1001 objects in my AD i’ll get the first 1000 in the first request and the last object in my second request), thank god the AD guys and the framework guys were smart enough to include the following attribute – MoreData (http://msdn.microsoft.com/en-us/library/system.directoryservices.protocols.dirsyncresponsecontrol.moredata.aspx). Guess what it means (do I need to mentioned it can be true or false??).

So that’s the way I handled the response of the RequestControl:

 foreach (DirectoryControl control in searchResponse.Controls)
{
 if (control is DirSyncResponseControl)
  {
    DirSyncResponseControl dsrc = control as DirSyncResponseControl;
    cookie = dsrc.Cookie;
    bMoreData = dsrc.MoreData;
    break;
  }
}

Now we can just use the bMoreData to see whether we need to run again or not. When we construct the next request it is very important to use the cookie which we got in our last response for the Domain Controller to know exactly where we stopped and at what state we are in synchronization (I’m pretty sure it’s mentioned somewhere… oh wait.. it’s just at line #11 at this blog… if you missed start over!).

 dirSyncRC.Cookie = cookie;
searchResponse = (SearchResponse)connection.SendRequest(request);

And eventually I serialize the cookie for next searches to get only the changes from my last search request (And that’s what the DirSync control is all about).

 using (FileStream fsStream = new FileStream(strFileName, FileMode.Create))
 {
    //Serialize the data to the steam. To get the data for 
    //the cookie, call the GetDirectorySynchronizationCookie method.
    bFormat.Serialize(fsStream, cookie);
  }

So… that’s how I implemented the DirSync control sample for a customer of mine… Took me a while as there’s not a lot of complete stuff on DirSync available so I guess here’s the first.

The full source code is available for Download from here:

Download

Wish you all pleasant syncing :-).

-Michael.

Comments

• Anonymous
  January 01, 2003
  The Download-link at the end is not working :(

• Anonymous
  January 01, 2003
  Thanks Michael. 1 question : Suppose I don't want to perform initial search with empty cookie. I already have all AD data. So can we get cookie with respect to current state of AD directly without performing dirSync search. Is it stored somewhere in registry ? I just want to avoid get same data which I have already for just getting cookies . In case of uSNChanged attribute we directly get uSNChanged number,so is it possible in case of dirSync ?

• Anonymous
  January 01, 2003
  Thanks for the roadmap.  A PowerShell translation is now available at dloder.blogspot.com/.../powershell-dirsync-sample.html.

• Anonymous
  January 01, 2003
  Michael, great gudiance of how to solve real world problem. I like it - keep sharing your wisdom. thank you

• Anonymous
  January 01, 2003
  Hi Michael, how to get group membership changes ?Thanks,Sagar

• Anonymous
  September 21, 2010
  Thanks a lot. This was very helpful. Also there is very less documentation on this feature