Partilhar via

Metadata #0 - Metadata, what is it and why do we care?

  • Artigo

You are on site, audit wasn't properly configured and yet you need to understand what has changed. If Active Directory is not your specialty, it's hard to catch what exactly happened. You will probably open a case with GBS but you also know that every second you wait, you risk to lose information. In this session, we will quickly review what the customers could have done proactively, but also what they can do after a modification to track what has changed in the directory. With simple tools and command lines out of the box and a more elaborated PowerShell code, we can easily keep track of every single change in the directory reading metadata and log files. It's maybe not too late to understand what happened!

Of course, when I write metadata I mean Active Directory Replication Metadata. The Metadata series will describe various scenarios where the use of the metadata can help understanding what happened in an Active Directory forest.

Metadata #1 - When did the delegation change? How to track security descriptor modifications Metadata #2 - The ephemeral admin, or how to track the group membership Metadata #3 - Spot who is cheating with the password policy
Metadata #4 - How to track the password policy changes
Metadata #5 - The locked-out built-in administrator
Metadata #6 - Track the recovery operations
Metadata #7 - The adminSDHolder strikes again!
Metadata #8 - Get-ADHistory or how to list the environment milestones

But before digging into those scenarios, let's go through a quick intro to make sure we're all on the same page.

Why do we care?

We care because metadata gives you a ton of information. When something changed, from where, how many times it changed... And everything without having the audit configured! Well it will not tell you who did a change, but sometimes it will give us just enough information to understand what's going on in the forest.

What is it?

Fundamentally, it is information that the Active Directory replication engine is using to track the changes and help the replication partners to know what to replicate.

Some basics...

Active Directory is a multi master environment. You can basically modify an object on any domain controller and the changes will be propagated everywhere on the forest (off course there are some exceptions but in general that's the drift). What do we replicate? Objects and attributes. The replication is actually attribute based. So if you modify the password of a user on a domain controllers in France and at the same time some provisioning tool is changing the phone number of the same user on a domain controller in Canada both changes are effective and there is no replication conflict.

Double click on the metadata

What if I change the telephone number again from Paris? Well it will be replicated all the way to Canada. To keep track of the changes, each attribute has a set of metadata which helps domain controllers to know what the most recent change is. As long as an attribute needs to be replicated, Active Directory will need replication metadata. The reason why I mentioned this is because some attributes are not being replicated. Either because it is irrelevant to replicate them or because replicating them will cause to much replication traffic on the network. What if an attribute changes every two minutes? The domain controllers will spend their life replicating it and making sure it is the right value everywhere. So it better be for a good reason. It is actually for that reason that the good old attribute lastLogon is not replicated. Every time a user logs on, interactively or on the network it updates the attribute on the domain controllers where the authentication is performed but it does not get replicated (see this article for more details about the lastLogon attribute and its friend the lastLogonTimestamp attribute: “The LastLogonTimeStamp Attribute” – “What it was designed for and how it works"). How do we know which attributes are getting replicated and which are not? It is defined in the schema with the flag FLAG_ATTR_NOT_REPLICATED. So a simple LDAP query will give us the list of all attributes which do not replicate in our forest. The LDAP filter would be (&(objectClass=attributeSchema)(systemFlags:1.2.840.113556.1.4.803:=1)) and of course scoping the naming context of the schema for example with ADSIEDIT:

Note that the attribute whenChanged is not replicated. Therefore, you cannot fully trust this one if you want to know when was the last time an attribute has been changed for a given object.

So let's focus on the attribute which do replicate. Can I see their metadata? Sure anytime, as long as you have the right to read an object you can read its metadata. They are readable from the attribute msDS-ReplAttributeMetaData. This attribute is a bit special because it is in fact a constructed attribute, meaning that if you want to see the value of this one, you need to explicitly request for it. Example:

Get-ADObject `

  -SearchBase "CN=Administrator,CN=Users,DC=contoso,DC=com" `

  -Filter * -SearchScope Subtree `

  -Properties *  | Select-Object msDS-ReplAttributeMetaData                        

 

msDS-ReplAttributeMetaData

--------------------------

 {} 

In the query above, I do ask for all the attributes of the object Administrator (properties = *). Yet the value of msDS-ReplAttributeMetaData is empty. Bellow is the same query but this time I explicitly ask for the attribute containing the metadata:

Get-ADObject `

  -SearchBase "CN=Administrator,CN=Users,DC=contoso,DC=com" `

  -Filter * -SearchScope Subtree `

  -Properties msDS-ReplAttributeMetaData | Select-Object msDS-ReplAttributeMetaData

 

msDS-ReplAttributeMetaData 

--------------------------

{<DS_REPL_ATTR_META_DATA>...

This time I do have something because I asked for it (properties = msDS-ReplAttributeMetaData). The value is structured in a very specific way: DS_REPL_ATTR_META_DATA (well to be exact, it is in fact the structure documented here: DS_REPL_ATTR_META_DATA_2). When using the usual graphical tools, you have to play with the filter button:


This is what it looks like:

<DS_REPL_ATTR_META_DATA>   <pszAttributeName>mail</pszAttributeName>   <dwVersion>3</dwVersion>   <ftimeLastOriginatingChange>2014-07-18T19:15:03Z</ftimeLastOriginatingChange>   <uuidLastOriginatingDsaInvocationID>c17b3678-c060-4e9f-b0c1-d471e7139fef</uuidLastOriginatingDsaInvocationID>   <usnOriginatingChange>20574</usnOriginatingChange>   <usnLocalChange>19365</usnLocalChange>   <pszLastOriginatingDsaDN>CN=NTDS Settings,CN=DC2012,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=contoso,DC=com</pszLastOriginatingDsaDN></DS_REPL_ATTR_META_DATA><DS_REPL_ATTR_META_DATA>   <pszAttributeName>lastLogonTimestamp</pszAttributeName>   <dwVersion>7</dwVersion>   <ftimeLastOriginatingChange>2014-08-05T02:02:23Z</ftimeLastOriginatingChange>   <uuidLastOriginatingDsaInvocationID>90d9dcd9-9da9-49ff-8040-896227e1487f</uuidLastOriginatingDsaInvocationID>   <usnOriginatingChange>20539</usnOriginatingChange>   <usnLocalChange>20539</usnLocalChange>   <pszLastOriginatingDsaDN>CN=NTDS Settings,CN=DC2008R2,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=contoso,DC=com</pszLastOriginatingDsaDN> </DS_REPL_ATTR_META_DATA>...

This attribute contains all the metadata for all the replicated attributed set on the object (technically if the attribute has never been set, there is no metadata associated to it yet but sometimes the console you are using to create an object is setting things that you would not suspect). Note that it does not contain the value of the attribute, just the metadata which is the following:

  • pszAttributeName is the ldapDisplayName of the attribute, well it is the name of the attribute like you see it in the majority of the tools you are using.

  • dwVersion the version number of the attribute. Is incremented each time the attribute is modified. So it basically relflect how many times the value attribute changed. Setting the attribute back to "not set" just increments the version number like any other modification.

  • ftimeLastOriginatingChange is the timestamp of the last time the attribute has been modified. So you can easily know what are the attributes which have been changed when you are seeing that an object has changed.

  • uuidLastOriginatingDsaInvocationID is the domain controller from which the modification is from. In the replication world, domain controllers are identified by a unique identifier called invocation ID, it is basically the ID of the database. You can see them in the output of a repadmin /showrepl

    C:\>repadmin /showrepl dc2008r2Default-First-Site-Name\DC2008R2DSA Options: IS_GCSite Options: (none)DSA object GUID: e5622c77-09c3-419e-8db2-c93ae981eedcDSA invocationID: 90d9dcd9-9da9-49ff-8040-896227e1487f==== INBOUND NEIGHBORS ======================================DC=contoso,DC=com    Default-First-Site-Name\DC2012 via RPC        DSA object GUID: 9d26e124-6b7d-491f-a0a1-ac037b5b8447        Last attempt @ 2014-08-04 18:58:09 failed, result 1908 (0x774):            Could not find the domain controller for this domain.        1 consecutive failure(s).        Last success @ 2014-07-18 12:56:49....

    or just with any LDAP tools

  • pszLastOriginatingDsaDN represents the DN of the NTDS object from which the modification is from. Well, it sounds complicated but that's just the DN of the DC in the configuration naming context or where you can find it in the Sites and Services console. It is more "readable" than the invocation ID isn't it?

  • usnOriginatingChange and usnLocalChange tell us what the Update Sequence Number of the DC was, when the modification happened and what the USN of the domain controller is, when the attribute has been replicated to the DC we are looking up. USN are just local counters, each domain controller has a USN number which is incremented every time they write something in their database. This is a very powerful information to track changes locally and we'll discuss it in another article. In the mean time, you can have a look at this article to refresh your skills on the AD replication engine: How the Active Directory Replication Model Works.

How to read this information?

Depending on what you are trying to achieve by reading the information (simple check or complex report) you might want to use simple out of the box tools or more complicated PowerShell code. As an admin (if you are a developer you can use whatever language you want as long as you use the right classes etc. see MSDN), this is what we have to read this info...

  • repadmin is the Active Directory admin ultimate tool. It has a parameter to query metadata:

    C:\>repadmin /showobjmeta DC2008R2 "CN=Administrator,CN=Users,DC=contoso,DC=com"37 entries.Loc.USN                           Originating DSA  Org.USN  Org.Time/Date        Ver Attribute=======                           =============== ========= =============        === =========   7697      a6768194-9d5a-43ab-9139-97da4b7fbca1      1410 2000-07-02 19:59:15    1 objectClass   7697          Default-First-Site-Name\DC2008R2      7697 2010-07-10 12:54:43    1 cn  19350            Default-First-Site-Name\DC2012     20559 2014-07-18 12:13:13    2 sn  19344            Default-First-Site-Name\DC2012     20549 2014-07-18 12:10:09    1 l   7697      a6768194-9d5a-43ab-9139-97da4b7fbca1      1410 2000-07-02 19:59:15    1 description   7697      a6768194-9d5a-43ab-9139-97da4b7fbca1      1410 2000-07-02 19:59:15    1 instanceType   7697      a6768194-9d5a-43ab-9139-97da4b7fbca1      1410 2000-07-02 19:59:15    1 whenCreated   7697      a6768194-9d5a-43ab-9139-97da4b7fbca1      1410 2000-07-02 19:59:15    1 displayName   7697      ecc31b35-05d1-406a-941c-581804e9ad3c     18511 2008-07-21 17:15:53    4 nTSecurityDescriptor  19355            Default-First-Site-Name\DC2012     20560 2014-07-18 12:13:32    1 wWWHomePage   7697      a6768194-9d5a-43ab-9139-97da4b7fbca1      1410 2000-07-02 19:59:15    1 name...

    Where Org.Time/Date is the time of the last modification of the Attribute and Ver the number of time the value changed. Originating DSA tells you on which domain controller the modification has been done. If the DC does not exist anymore, you'll just see the invocation ID.

  • LDP can also retrieve this information but it is harder to read:

    As you can see it is not the display name of the attribute that we're seeing but an ID.

  • Classic LDAP tools, as long as you explicitly ask for the metadata attributes. For example, still using LDP but this time performing a simple LDAP query:

  • Scripting using VBScript and of course PowerShell (Iwill actually not even discuss about VBScript). It is super easy if you have Windows 8 or Windows Server 2012 in your environment and the Active Directory Web Service. If you have only Windows Server 2003 domain controllers, you can install the Active Directory Gateway Service, or you can ping me and I send you a code which works without the web service. An example from a Windows 8 machine (against a Windows Server 2008 R2 domain controller):

    Get-ADReplicationAttributeMetadata `

      -Object "CN=Guest,CN=Users,DC=contoso,DC=com" `

      -Server DC2008R2 | Select-Object `

              AttributeName, `

              AttributeValue, `

              LastOriginatingChangeTime, `

              LastOriginatingChangeDirectoryServerInvocationId | Out-GridView

     

    And you will have the following output (the Out-GridView requires PowerShell_ISE installed):

Now that you have the basics, it is time to browse the different scenario listed at the beginning of the post!

Wait a minute... What about the msDS-ReplValueMetaData attribute?

There are actually two attributes which can be used to retrieve the replication metadata. The one we discussed so far, the msDS-ReplAttributeMetaData, stores the replication metadata for the "regular" attributes. But there is another one, the msDS-ReplValueMetaData, which store the metadata for the linked value attributes (information such as group membership, direct reports...). In PowerShell, all of them do not show up by default unless you request for them:

Get-ADReplicationAttributeMetadata `

  -Object "CN=Administrators,CN=Builtin,DC=contoso,DC=com" `

  -ShowAllLinkedValues `

  -Server DC2008R2

This one will be seen in details in the post describing how to track group membership. So have a look there :)

Comments

  • Anonymous
    August 25, 2014
    This post is a part of the Metadata series. Have a look at the intro to have more information about it
  • Anonymous
    October 18, 2014
    Metadata #0 - Metadata, what is it and why do we care? - Once upon a case... - Site Home - TechNet Blogs
  • Anonymous
    October 24, 2014
    Metadata #0 - Metadata, what is it and why do we care? - Once upon a case... - Site Home - TechNet Blogs
  • Anonymous
    October 24, 2014
    Metadata #0 - Metadata, what is it and why do we care? - Once upon a case... - Site Home - TechNet Blogs
  • Anonymous
    October 24, 2014
    Metadata #0 - Metadata, what is it and why do we care? - Once upon a case... - Site Home - TechNet Blogs