Friday Mail Sack: Carl Sandburg Edition

Hi folks, Jonathan again. Ned is taking some time off visiting his old stomping grounds – the land of Mother-in-Laws and heart-breaking baseball. Or, as Sandburg put it:

“Hog Butcher for the World,
Tool Maker, Stacker of Wheat,
Player with Railroads and the Nation's Freight Handler;
Stormy, husky, brawling,
City of the Big Shoulders”

Cool, huh?

Anyway, today we talk about:

• DC utilization
• DFS Namespace interoperability
• DFSN and NTFS mount points
• Backing up Themes with USMT
• Automatic LinkID creation
• Delegating update of the servicePrincipalName attribute
• Enabled property on PowerShell computer objects
• Other stuff

And awayyy we go!

Question

When thousands of clients are rebooted for Windows Update or other scheduled tasks, my domain controllers log many KDC 7 System event errors:

Log Name: System
Source: Microsoft-Windows-Kerberos-Key-Distribution-Center
Event ID: 7
Level: Error
Description:

The Security Account Manager failed a KDC request in an unexpected way. The error is in the data field.

Error 170000C0

I’m trying to figure out if this is a performance issue, if the mass reboots are related, if my DCs are over-utilized, or something else.

Answer

That extended error is:

C0000017 = STATUS_NO_MEMORY - {Not Enough Quota} - Not enough virtual memory or paging file quota is available to complete the specified operation.

The DCs are being pressured with so many requests that they are running out of Kernel memory. We see this very occasionally with applications that make heavy use of the older SAMR protocol for lookups (instead of say, LDAP). In some cases we could change the client application's behavior. In others, the customer just had to add more capacity. The mass reboots alone are not the problem here - it's the software that runs at boot up on each client that is then creating what amounts to a denial of service attack against the domain controllers.

Examine one of the client computers mentioned in the event for all non-Windows-provided services, scheduled tasks that run at startup, SCCM/SMS at boot jobs, computer startup scripts, or anything else that runs when the computer is restarted. Then get promiscuous network captures of that computer starting (any time, not en masse) while also running Process Monitor in boot mode, and you'll probably see some very likely candidates. You can also use SPA or AD Data Collector sets (https://blogs.technet.com/b/askds/archive/2010/06/08/son-of-spa-ad-data-collector-sets-in-win2008-and-beyond.aspx) in combination with network captures to see exactly what protocol is being used to overwhelm the DC, if you want to troubleshoot the issue as it happens. Probably at 3AM, that sounds sucky.

Ultimately, the application causing the issue must be stopped, reconfigured, or removed - the only alternative is to add more DCs as a capacity Band-Aid or stagger your mass reboots.

Question

Is it possible to have 2003 and 2008 servers co-exist in the same DFS namespace? I don’t see it documented either “for” or “against” on the blog anywhere.

Answer

It's totally ok to mix OSes in the DFSN namespace, as long as you don't use Windows Server 2008 ("V2 mode") namespaces, which won't allow any Win2003 servers. If you are using DFSR to replicate the data, make sure all server have the latest DFSR hotfixes (here and here), as there areincompatibilities in DFSR that these hotfixes resolve.

Question

Should I create DFS namespace folders (used by the DFS service itself) under NTFS mount points? Is there any advantage to this?

Answer

DFSN management tools do not allow you to create DFSN roots and links under mount points ordinarily, and once you do through alternate hax0r means, they are hard to remove (you have to use FSUTIL). Ergo, do not do it – the management tools blocking you means that it is not supported.

There is no real value in placing the DFSN special folders under mount points - the DFSN special folders consume no space, do not contain files, and exist only to provide reparse point tags to the DFSN service and its file IO driver goo. By default, they are configured on the root of the C: drive in a folder called c:\dfsroots. That ensures that they are available when the OS boots. If clustering you'd create them on one of your drive-lettered shared disks.

Question

How do you back up the Themes folder using USMT4 in Windows 7?

Answer

The built-in USMT migration code copies the settings but not the files, as it knows the files will exist somewhere on the user’s source profile and that those are being copied by the migdocs.xml/miguser.xml. It also knows that the Themes system will take care of the rest after migration; the Themes system creates the transcoded image files using the theme settings and copies the image files itself.

Note here how after scanstate, my USMT store’s Themes folder is empty:

After I loadstate that user, the Themes system fixed it all up in that user’s real profile when the user logged on:

However, if you still specifically need to copy the Themes folder intact for some reason, here’s a sample custom XML file:

<?xml version="1.0" encoding="UTF-8"?>

<migration urlid="https://www.microsoft.com/migration/1.0/migxmlext/migratethemefolder">

<component type="Documents" context="User">

<!-- sample theme folder migrator -->

<displayName>ThemeFolderMigSample</displayName>

 <role role="Data">

  <rules>

   <include filter='MigXmlHelper.IgnoreIrrelevantLinks()'>

   <objectSet>

    <pattern type="File">%CSIDL_APPDATA%\Microsoft\Windows\Themes\* [*]</pattern>

   </objectSet>

  </include>

 </rules>

 </role>

And here it is in action:

Question

I've recently been working on extending my AD schema with a new back-linked attribute pair, and I used the instructions on this blog to auto-generate the linkIDs for my new attributes. Confusingly, the resulting linkIDs are negative values (-912314983 and -912314984). The attributes and backlinks seem to work as expected, but when looking at the MSDN definition of the linkID attribute, it specifically states that the linkID should be a positive value. Do you know why I'm getting a negative value, and if I should be concerned?

Answer

The only hard and fast rule is that the forward link (flink) be an even number and the backward link (blink) be the flink's ID plus one. In your case, the flink is -912314984 then the blink had better be -912314983, which I assume is the case since things are working. But, we were curious when you posted the linkID documentation from MSDN so we dug a little deeper.

The fact that your linkIDs are negative numbers is correct and expected, and is the result of a feature called AutoLinkID. Automatically generated linkIDs are in the range of 0xC0000000-0xFFFFFFFC (-1,073,741,824 to -4). This means that it is a good idea to use positive numbers if you are going to set the linkID manually. That way you are guaranteed not to conflict with automatically generated linkIDs.

The bottom line is, you're all good.

Question

I am trying to delegate permissions to the DBA team to create, modify, and delete SPNs since they're the team that swaps out the local accounts SQL is installed under to the domain service accounts we create to run SQL.

Documentation on the Internet has led me down the rabbit hole to no end.  Can you tell me how this is done in a W2K8 R2 domain and a W2K3 domain?

Answer

So you will want to delegate a specific group of users -- your DBA team -- permissions to modify the SPN attribute of a specific set of objects -- computer accounts for servers running SQL server and user accounts used as service accounts under which SQL Server can run.

The easiest way to accomplish this is to put all such accounts in one OU, ie OU=SQL Server Accounts, and run the following commands:

Dsacls "OU=SQL Server Accounts,DC=corp,DC=contoso,DC=com" /I:S /G "CORP\DBA Team":WPRP;servicePrincipalName;user
Dsacls "OU=SQL Server Accounts,DC=corp,DC=contoso,DC=com" /I:S /G "CORP\DBA Team":WPRP;servicePrincipalName;computer

These two commands will grant the DBA Team group permission to read and write the servicePrincipalName attribute on user and computer objects in the SQL Server Accounts OU.

Your admins should then be able to use setspn.exe to modify that property on the designated accounts.

But…what if you have a large number of accounts spread across multiple OUs? The above solution only works well if all of your accounts are concentrated in a few (preferably one) OUs. In this case, you basically have two options:

1. You can run the two commands specifying the root of the domain as the object, but you would be delegating permissions for EVERY user and computer in the domain. Do you want your DBA team to be able to modify accounts for which they have no legitimate purpose?
2. Compile a list of specific accounts the DBA team can manage and modify each of them individually. That can be done with a single command line. Create a text file that contains the DNs of each account for which you want to delegate permissions and then use the following command:for /f "tokens=*" %i in (object-list.txt) do dsacls "%i" /G "CORP\DBA Team":WPRP;servicePrincipalName

None of these are really great options, however, because you’re essentially giving a group of non-AD Administrators the ability to screw up authentication to what are perhaps critical business resources. You might actually be better off creating an expedited process whereby these DBAs can submit a request to a real Administrator who already has permissions to make the required changes, as well as the experience to verify such a change won’t cause any problems.

Author’s Note: This gentleman pointed out in a reply that these DBAs wouldn’t want him messing with tables, rows and the SA account, so he doesn’t want them touching AD. I thought that was sort of amusing.

Question

What is Powershell checking when your run get-adcomputer -properties * -filter * | format-table Name,Enabled?  Is Enabled an attribute, a flag, a bit, a setting?  What, if at all, would that setting show up as in something like ADSIEdit.msc?

I get that stuff like samAccountName, sn, telephonenumber, etc.  are attributes but what the heck is enabled?

Answer

All objects in PowerShell are PSObjects, which essentially wrap the underlying .NET or COM objects and expose some or all of the methods and properties of the wrapped object. In this case, Enabled is an attribute ultimately inherited from the System.DirectoryServices.AccountManagement.AuthenticablePrincipal .NET class. This answer isn’t very helpful, however, as it just moves your search for answers from PowerShell to the .NET Framework, right? Ultimately, you want to know how a computer’s or user’s account state (enabled or disabled) is stored in Active Directory.

Whether or not an account is disabled is reflected in the appropriate bit being set on the object’s userAccountControl attribute. Check out the following KB: How to use the UserAccountControl flags to manipulate user account properties. You’ll find that the penultimate least significant bit of the userAccountControl bitmask is called ACCOUNTDISABLE, and reflects the appropriate state; 1 is disabled and 0 is enabled.

If you find that you need to use an actual LDAP query to search for disabled accounts, then you can use a bitwise filter. The appropriate LDAP filter would be:

 (UserAccountControl:1.2.840.113556.1.4.803:=2)

Other stuff

I watched this and, despite the lack of lots of moving arms and tools, had sort of a Count Zero moment:

And just for Ned (because he REALLY loves this stuff!): Kittens!

No need to rush back, dude.

Jonathan “Payback is a %#*@&!” Stephens

Comments

• Anonymous
  January 28, 2012
  The whole DBA discussion is far not as amusing as <strike>kittens</strike> having two completely unrelated things called "User Account Control" in the same OS.

• Anonymous
  January 28, 2012
  Artem - We just overloaded the term. --Jonathan

• Anonymous
  January 30, 2012
  Thanks for including the LDAP filter syntax and the link.  That is an obscure bit of LDAP...trivia(?).

• Anonymous
  January 30, 2012
  You're welcome, LA.

• Anonymous
  January 30, 2012
  This query will give you enabled accounts: (&(objectclass=computer)(!(UserAccountControl:1.2.840.113556.1.4.803:=2))) Is there a query statement you can use, that will give you only enabled accounts, without using the '!'?

• Anonymous
  January 30, 2012
  LA - Not with the two filters available to us, no. The bitwise AND returns true of the bit matching 0x2 is also turned on in the userAccountControl value. You would need a bitwise XOR to determine if the bit matching 0x2 were turned off -- indicating that the account was enabled. Active Directory doesn't support an XOR bitwise query filter. AD doesn't need to, though. Your NOT (!) expression accomplishes the same goal. --Jonathan

• Anonymous
  January 31, 2012
  Nice work on the mail sack, Jonathan. I guess I won't need this cudgel after all. Too bad, it was a real trick getting it past TSA...