Finding Pesky Stale DNS SRV Records
Good day to you all - Dougga here with a simple post today using tools you know.
Stale DNS SRV records are common due to no scavenging on DNS zones and each zone has to be setup correctly to have this happen. So, I have often found the "contoso.com" setup correctly, but the "_msdcs.contoso.com" is not. So this leads to stale DNS SRV records in DNS from failed domain controllers or due to de register failure during a successful demotion. This is something I have found probably hundreds of times over the years. Managing the DNS is obviously important and a thorough post from Hilde, Brent, and Bryan around the DNS topic can be found here.
That said, I have now been tasked upgrading or replacing a domain controller using the same name and IP address. This could be done with an in-place upgrade, but none of us reading this are likely to be a fan of the in-place upgrade. Nor are we a fan of building a new domain controller with the same name and IP address - but here we are.
There are plenty of blog posts to emphasize potential issues of doing this task. However, we have to do it anyway, and I have worked with many customers that have done the same. So this post is addressing these two topics: Stale DNS SRV records, and replacing a DC with the same name an IP address.
You can look at this as an upgrade to 2012 R2 or simply a replacement of hardware and using the same operating system. Either way, the steps are the same not counting domain and forest preps and other testing if you are upgrading.
Short description of the process to replace DC1 with a "new" DC1.
1. Build new box and name it DC1-replacement.
2. Join DC1-Replacement to the domain and reboot it.
3. Demote DC1.
4. Force replication and make sure the demotion was heard around the world (assuming your environment expands the globe).
5. Verify DNS de-registration was successful (this what got this blog started).
6. Power off DC1.
7. Re-IP DC1-replacement to DC1's IP address.
8. Rename DC1-replacement to DC1.
9. Reboot for the rename operation to be completed.
10. Promote the new DC1.
11. Verify msDS-AdditionalDnsNames does not contain the DC1-replacement hostname - if it does the SPNs for that host name will remain in the ServicePrincipalName attribute.
a. Delete them if they exist.
You may be thinking: So what? Who cares about the DNS records - once the new DC is renamed and has the original IP won't it just work if the original records from the retired DC1 were not de-registered? If you have secure updates on DNS, then the ownership will belong to the retired domain controller even though it is the same name. SIDs and all that.
As I mentioned above, these stale DNS records typically are not discovered because scavenging is disabled for the zone, no one complains, no one looks for the stale records, and the issue does not surface.
In a large (heck, even a small) environment I would challenge any engineer to know all the DNS records registered by a domain controller. It is not impossible, but browse the _msdcs DNS zone on a small lab and you will see what I mean. So, the challenge here is to ensure all those records for a demoted domain controller were successfully un-registered. Or if a domain controller failed and is not returning, then this process may be useful as well.
We can use the netlogon.dns file found in C:\windows\system32\config folder to find out what records should be present if the DC is operational. Before the demotion of the domain controller targeted to be replaced, I copied this file and made two copies - one for SRV records and one for non-SRV records and created a batch file to run three commands.
Setup ready to go, I have 4 files in my folder:
1. FindNonSrv.txt - This is a copy of netlogon.dns minus SRV records
2. FindSrv.txt - This is a copy of netlogon.dns minus Non-SRV records
3. Netlogon.dns - This is a copy from c:\windows\system32\config so I have an original copy available for my mistakes.
4. RunFindRecords.cmd - This file I created with the Nslookup commands to find the records
This is the content of the RunFindRecords.cmd:
=======================================
for /f %%a in (c:\nslookup\FindSrv.txt) do nslookup -querytype=srv %%a >>xFindSrv.txt
for /f %%a in (c:\nslookup\FindNonSrv.txt) do nslookup %%a >>xFindNonSrv.txt
nslookup -querytype=ns contoso.com >>xFinsNS.txt
======================================
After the script runs we have 3 files added to the folder (NOTE: for sorting purposes the output files start with an x):
1. xFindNonSrv.txt
2. xFindSrv.txt
3. xFindNS.txt
Then the process is rather simple: Use your favorite text editor and search for the existence of the demoted or failed domain controller using both the NetBIOS name and IP address of the demoted DC in the output xfiles.
Doug "use what you know" Gabbard - PFE