Today I follow your advices check both servers again. I recorded a event file.
dc1server: 172.31.15.250
PS C:\Users\Administrator> netdom query fsmo
Schema master DC1SERVER.Qingdao.DALLAS.usagroup.com
Domain naming master DC1SERVER.Qingdao.DALLAS.usagroup.com
PDC DC1SERVER.Qingdao.DALLAS.usagroup.com
RID pool manager DC1SERVER.Qingdao.DALLAS.usagroup.com
Infrastructure master DC1SERVER.Qingdao.DALLAS.usagroup.com
The command completed successfully.
PS C:\Users\Administrator> dcdiag /test:dns
Directory Server Diagnosis
Performing initial setup:
Trying to find home server...
Home Server = DC1SERVER
* Identified AD Forest.
Done gathering initial info.
Doing initial required tests
Testing server: STAGING\DC1SERVER
Starting test: Connectivity
The host 588b7734-c9e2-46ef-8cf0-92b9d9c148c2._msdcs.DALLAS.usagroup.com could not be resolved to an IP
address. Check the DNS server, DHCP, server name, etc.
Got error while checking LDAP and RPC connectivity. Please check your firewall settings.
......................... DC1SERVER failed test Connectivity
Doing primary tests
Testing server: STAGING\DC1SERVER
Starting test: DNS
DNS Tests are running and not hung. Please wait a few minutes...
......................... DC1SERVER passed test DNS
Running partition tests on : Qingdao
Running partition tests on : ForestDnsZones
Running partition tests on : Schema
Running partition tests on : Configuration
Running partition tests on : DomainDnsZones
Running enterprise tests on : DALLAS.usagroup.com
Starting test: DNS
Test results for domain controllers:
DC: DC1SERVER.Qingdao.DALLAS.usagroup.com
Domain: Qingdao.DALLAS.usagroup.com
TEST: Basic (Basc)
Error: No LDAP connectivity
Warning: adapter [00000010] Broadcom NetXtreme Gigabit Ethernet has invalid DNS server: 172.31.15.250
(DC1SERVER)
Error: all DNS servers are invalid
No host records (A or AAAA) were found for this DC
TEST: Forwarders/Root hints (Forw)
Error: All forwarders in the forwarder list are invalid.
TEST: Records registration (RReg)
Error: Record registrations cannot be found for all the network adapters
Summary of test results for DNS servers used by the above domain controllers:
DNS server: 10.0.0.10 (
1 test failure on this DNS server
PTR record query for the 1.0.0.127.in-addr.arpa. failed on the DNS server 10.0.0.10 Name re
solution is not functional. _ldap._tcp.DALLAS.usagroup.com. failed on the DNS server 10.0.0.10
DNS server: 114.114.114.114 (
1 test failure on this DNS server
Name resolution is not functional. _ldap._tcp.DALLAS.usagroup.com. failed on the DNS server 114.114.114.
114
DNS server: 172.31.15.250 (DC1SERVER)
1 test failure on this DNS server
Name resolution is not functional. _ldap._tcp.DALLAS.usagroup.com. failed on the DNS server 172.31.15.25
0
DNS server: 202.102.128.68 (
1 test failure on this DNS server
PTR record query for the 1.0.0.127.in-addr.arpa. failed on the DNS server 202.102.128.68 Na
me resolution is not functional. _ldap._tcp.DALLAS.usagroup.com. failed on the DNS server 202.102.128.68
DNS server: 223.6.6.6 (
1 test failure on this DNS server
Name resolution is not functional. _ldap._tcp.DALLAS.usagroup.com. failed on the DNS server 223.6.6.6
Summary of DNS test results:
Auth Basc Forw Del Dyn RReg Ext
_________________________________________________________________
Domain: Qingdao.DALLAS.usagroup.com
DC1SERVER PASS FAIL FAIL PASS PASS FAIL n/a
......................... DALLAS.usagroup.com failed test DNS
PS C:\Users\Administrator>
PS C:\Users\Administrator> repadmin /showrepl
Repadmin: running command /showrepl against full DC localhost
STAGING\DC1SERVER
DSA Options: IS_GC
Site Options: (none)
DSA object GUID: 588b7734-c9e2-46ef-8cf0-92b9d9c148c2
DSA invocationID: 444e0925-efa3-485b-b253-075e209e82e9
Source: MADISON\TDGAD
******* 31 CONSECUTIVE FAILURES since 2025-02-27 01:13:02
Last error: 8524 (0x214c):
The DSA operation is unable to proceed because of a DNS lookup failure.
Source: WHITEHOUSE\NJ-DC-02
******* 47 CONSECUTIVE FAILURES since 2025-02-26 21:01:35
Last error: 1722 (0x6ba):
The RPC server is unavailable.
Naming Context: CN=Configuration,DC=DALLAS,DC=usagroup,DC=com
Source: WHITEHOUSE\NJ-DC-02
******* WARNING: KCC could not add this REPLICA LINK due to error.
Naming Context: DC=ForestDnsZones,DC=DALLAS,DC=usagroup,DC=com
Source: WHITEHOUSE\NJ-DC-02
******* WARNING: KCC could not add this REPLICA LINK due to error.
Naming Context: DC=DALLAS,DC=usagroup,DC=com
Source: WHITEHOUSE\NJ-DC-02
******* WARNING: KCC could not add this REPLICA LINK due to error.
Naming Context: DC=texascity,DC=DALLAS,DC=usagroup,DC=com
Source: WHITEHOUSE\NJ-DC-02
******* WARNING: KCC could not add this REPLICA LINK due to error.
Source: WHITEHOUSE\NJ-DC-01
******* 3 CONSECUTIVE FAILURES since 2025-02-27 09:03:54
Last error: 1722 (0x6ba):
The RPC server is unavailable.
Naming Context: CN=Configuration,DC=DALLAS,DC=usagroup,DC=com
Source: WHITEHOUSE\NJ-DC-01
******* WARNING: KCC could not add this REPLICA LINK due to error.
Naming Context: DC=ForestDnsZones,DC=DALLAS,DC=usagroup,DC=com
Source: WHITEHOUSE\NJ-DC-01
******* WARNING: KCC could not add this REPLICA LINK due to error.
Naming Context: DC=DALLAS,DC=usagroup,DC=com
Source: WHITEHOUSE\NJ-DC-01
******* WARNING: KCC could not add this REPLICA LINK due to error.
Source: STAGING\DALLAS-EXTRADC
******* 81 CONSECUTIVE FAILURES since 2025-02-26 12:09:53
Last error: 1722 (0x6ba):
The RPC server is unavailable.
Naming Context: CN=Configuration,DC=DALLAS,DC=usagroup,DC=com
Source: STAGING\DALLAS-EXTRADC
******* WARNING: KCC could not add this REPLICA LINK due to error.
Naming Context: CN=Schema,CN=Configuration,DC=DALLAS,DC=usagroup,DC=com
Source: STAGING\DALLAS-EXTRADC
******* WARNING: KCC could not add this REPLICA LINK due to error.
Naming Context: DC=Qingdao,DC=DALLAS,DC=usagroup,DC=com
Source: STAGING\DALLAS-EXTRADC
******* WARNING: KCC could not add this REPLICA LINK due to error.
PS C:\Users\Administrator> dfsrdiag replicationstate
Summary
Active inbound connections: 0
Updates received: 0
Active outbound connections: 0
Updates sent out: 0
Operation Succeeded
PS C:\Users\Administrator> dcdiag /a /v
Directory Server Diagnosis
Performing initial setup:
Trying to find home server...
* Verifying that the local machine DC1SERVER, is a Directory Server.
Home Server = DC1SERVER
* Connecting to directory service on server DC1SERVER.
* Identified AD Forest.
Collecting AD specific global data
* Collecting site info.
Calling ldap_search_init_page(hld,CN=Sites,CN=Configuration,DC=DALLAS,DC=usagroup,DC=com,LDAP_SCOPE_SUBTREE,(objectC
ategory=ntDSSiteSettings),.......
The previous call succeeded
Iterating through the sites
Looking at base site object: CN=NTDS Site Settings,CN=STAGING,CN=Sites,CN=Configuration,DC=DALLAS,DC=usagroup,DC=com
Getting ISTG and options for the site
Looking at base site object: CN=NTDS Site Settings,CN=JEFFERSONVILLE,CN=Sites,CN=Configuration,DC=DALLAS,DC=usagroup
,DC=com
Getting ISTG and options for the site
Looking at base site object: CN=NTDS Site Settings,CN=TEXASCITY,CN=Sites,CN=Configuration,DC=DALLAS,DC=usagroup,DC=c
om
Getting ISTG and options for the site
Looking at base site object: CN=NTDS Site Settings,CN=MADISON,CN=Sites,CN=Configuration,DC=DALLAS,DC=usagroup,DC=com
Getting ISTG and options for the site
Looking at base site object: CN=NTDS Site Settings,CN=WHITEHOUSE,CN=Sites,CN=Configuration,DC=DALLAS,DC=usagroup,DC=
com
Getting ISTG and options for the site
* Identifying all servers.
Calling ldap_search_init_page(hld,CN=Sites,CN=Configuration,DC=DALLAS,DC=usagroup,DC=com,LDAP_SCOPE_SUBTREE,(objectC
lass=ntDSDsa),.......
The previous call succeeded....
The previous call succeeded
Iterating through the list of servers
Getting information for the server CN=NTDS Settings,CN=NJ-DC-02,CN=Servers,CN=WHITEHOUSE,CN=Sites,CN=Configuration,DC
=DALLAS,DC=usagroup,DC=com
objectGuid obtained
InvocationID obtained
dnsHostname obtained
site info obtained
All the info for the server collected
Getting information for the server CN=NTDS Settings,CN=NJ-DC-01,CN=Servers,CN=WHITEHOUSE,CN=Sites,CN=Configuration,DC
=DALLAS,DC=usagroup,DC=com
objectGuid obtained
InvocationID obtained
dnsHostname obtained
site info obtained
All the info for the server collected
Getting information for the server CN=NTDS Settings,CN=TX-DC-01,CN=Servers,CN=TEXASCITY,CN=Sites,CN=Configuration,DC=
DALLAS,DC=usagroup,DC=com
objectGuid obtained
InvocationID obtained
dnsHostname obtained
site info obtained
All the info for the server collected
Getting information for the server CN=NTDS Settings,CN=DLLASGRP-DC3,CN=Servers,CN=JEFFERSONVILLE,CN=Sites,CN=Configur
ation,DC=DALLAS,DC=usagroup,DC=com
objectGuid obtained
InvocationID obtained
dnsHostname obtained
site info obtained
All the info for the server collected
Getting information for the server CN=NTDS Settings,CN=TDGAD,CN=Servers,CN=MADISON,CN=Sites,CN=Configuration,DC=DALLA
S,DC=usagroup,DC=com
objectGuid obtained
InvocationID obtained
dnsHostname obtained
site info obtained
All the info for the server collected
Getting information for the server CN=NTDS Settings,CN=DC1SERVER,CN=Servers,CN=STAGING,CN=Sites,CN=Configuration,DC=DA
LLAS,DC=usagroup,DC=com
objectGuid obtained
InvocationID obtained
dnsHostname obtained
site info obtained
All the info for the server collected
Getting information for the server CN=NTDS Settings,CN=DALLAS-EXTRADC,CN=Servers,CN=STAGING,CN=Sites,CN=Configuration
,DC=DALLAS,DC=usagroup,DC=com
objectGuid obtained
InvocationID obtained
site info obtained
All the info for the server collected
* Identifying all NC cross-refs.
Ldap search capability attribute search failed on server DALLAS-EXTRADC, return value = 81
Got error while checking if the DC is using FRS or DFSR. Error: Win32 Error 81The VerifyReferences, FrsEvent and
DfsrEvent tests might fail because of this error.
* Found 7 DC(s). Testing 2 of them.
Done gathering initial info.
Doing initial required tests
Testing server: STAGING\DC1SERVER
Starting test: Connectivity
* Active Directory LDAP Services Check
The host 588b7734-c9e2-46ef-8cf0-92b9d9c148c2._msdcs.DALLAS.usagroup.com could not be resolved to an IP
address. Check the DNS server, DHCP, server name, etc.
Got error while checking LDAP and RPC connectivity. Please check your firewall settings.
......................... DC1SERVER failed test Connectivity
Testing server: STAGING\DALLAS-EXTRADC
Starting test: Connectivity
* Active Directory LDAP Services Check
The host 6fbc7b6c-e934-44c3-9db1-5f209f481fa2._msdcs.DALLAS.usagroup.com could not be resolved to an IP
address. Check the DNS server, DHCP, server name, etc.
Neither the the server name (DALLAS-EXTRADC.@missing_dnsHostName@) nor the Guid DNS name
(6fbc7b6c-e934-44c3-9db1-5f209f481fa2._msdcs.DALLAS.usagroup.com) could be resolved by DNS. Check that the
server is up and is registered correctly with the DNS server.
Got error while checking LDAP and RPC connectivity. Please check your firewall settings.
......................... DALLAS-EXTRADC failed test Connectivity
Doing primary tests
Testing server: STAGING\DC1SERVER
Skipping all tests, because server DC1SERVER is not responding to directory service requests.
Test omitted by user request: Advertising
Test omitted by user request: CheckSecurityError
Test omitted by user request: CutoffServers
Test omitted by user request: FrsEvent
Test omitted by user request: DFSREvent
Test omitted by user request: SysVolCheck
Test omitted by user request: KccEvent
Test omitted by user request: KnowsOfRoleHolders
Test omitted by user request: MachineAccount
Test omitted by user request: NCSecDesc
Test omitted by user request: NetLogons
Test omitted by user request: ObjectsReplicated
Test omitted by user request: OutboundSecureChannels
Test omitted by user request: Replications
Test omitted by user request: RidManager
Test omitted by user request: Services
Test omitted by user request: SystemLog
Test omitted by user request: Topology
Test omitted by user request: VerifyEnterpriseReferences
Test omitted by user request: VerifyReferences
Test omitted by user request: VerifyReplicas
Testing server: STAGING\DALLAS-EXTRADC
Skipping all tests, because server DALLAS-EXTRADC is not responding to directory service requests.
Test omitted by user request: Advertising
Test omitted by user request: CheckSecurityError
Test omitted by user request: CutoffServers
Test omitted by user request: FrsEvent
Test omitted by user request: DFSREvent
Test omitted by user request: SysVolCheck
Test omitted by user request: KccEvent
Test omitted by user request: KnowsOfRoleHolders
Test omitted by user request: MachineAccount
Test omitted by user request: NCSecDesc
Test omitted by user request: NetLogons
Test omitted by user request: ObjectsReplicated
Test omitted by user request: OutboundSecureChannels
Test omitted by user request: Replications
Test omitted by user request: RidManager
Test omitted by user request: Services
Test omitted by user request: SystemLog
Test omitted by user request: Topology
Test omitted by user request: VerifyEnterpriseReferences
Test omitted by user request: VerifyReferences
Test omitted by user request: VerifyReplicas
Test omitted by user request: DNS
Test omitted by user request: DNS
Test omitted by user request: DNS
Test omitted by user request: DNS
Running partition tests on : Qingdao
Starting test: CheckSDRefDom
......................... Qingdao passed test CheckSDRefDom
Starting test: CrossRefValidation
For the partition (DC=Qingdao,DC=DALLAS,DC=usagroup,DC=com) we encountered the following error retrieving
the cross-ref's (CN=QINGDAO,CN=Partitions,CN=Configuration,DC=DALLAS,DC=usagroup,DC=com) information:
LDAP Error 0x3a (58).
......................... Qingdao failed test CrossRefValidation
Running partition tests on : ForestDnsZones
Starting test: CheckSDRefDom
For the partition (DC=ForestDnsZones,DC=DALLAS,DC=usagroup,DC=com) we encountered the following error
retrieving the cross-ref's
(CN=7398587e-b7e5-44b2-989b-7f3884511790,CN=Partitions,CN=Configuration,DC=DALLAS,DC=usagroup,DC=com)
information:
LDAP Error 0x3a (58).
......................... ForestDnsZones failed test CheckSDRefDom
Starting test: CrossRefValidation
For the partition (DC=ForestDnsZones,DC=DALLAS,DC=usagroup,DC=com) we encountered the following error
retrieving the cross-ref's
(CN=7398587e-b7e5-44b2-989b-7f3884511790,CN=Partitions,CN=Configuration,DC=DALLAS,DC=usagroup,DC=com)
information:
LDAP Error 0x3a (58).
......................... ForestDnsZones failed test CrossRefValidation
Running partition tests on : Schema
Starting test: CheckSDRefDom
......................... Schema passed test CheckSDRefDom
Starting test: CrossRefValidation
For the partition (CN=Schema,CN=Configuration,DC=DALLAS,DC=usagroup,DC=com) we encountered the following
error retrieving the cross-ref's
(CN=Enterprise Schema,CN=Partitions,CN=Configuration,DC=DALLAS,DC=usagroup,DC=com) information:
LDAP Error 0x3a (58).
......................... Schema failed test CrossRefValidation
Running partition tests on : Configuration
Starting test: CheckSDRefDom
......................... Configuration passed test CheckSDRefDom
Starting test: CrossRefValidation
For the partition (CN=Configuration,DC=DALLAS,DC=usagroup,DC=com) we encountered the following error
retrieving the cross-ref's
(CN=Enterprise Configuration,CN=Partitions,CN=Configuration,DC=DALLAS,DC=usagroup,DC=com) information:
LDAP Error 0x3a (58).
......................... Configuration failed test CrossRefValidation
Running partition tests on : DomainDnsZones
Starting test: CheckSDRefDom
......................... DomainDnsZones passed test CheckSDRefDom
Starting test: CrossRefValidation
For the partition (DC=DomainDnsZones,DC=Qingdao,DC=DALLAS,DC=usagroup,DC=com) we encountered the following
error retrieving the cross-ref's
(CN=8ebf1ff0-36d8-4f81-9380-ce4b462180f4,CN=Partitions,CN=Configuration,DC=DALLAS,DC=usagroup,DC=com)
information:
LDAP Error 0x3a (58).
......................... DomainDnsZones failed test CrossRefValidation
Running partition tests on : DomainDnsZones
Starting test: CheckSDRefDom
......................... DomainDnsZones passed test CheckSDRefDom
Starting test: CrossRefValidation
For the partition (DC=DomainDnsZones,DC=Qingdao,DC=DALLAS,DC=usagroup,DC=com) we encountered the following
error retrieving the cross-ref's
(CN=8ebf1ff0-36d8-4f81-9380-ce4b462180f4,CN=Partitions,CN=Configuration,DC=DALLAS,DC=usagroup,DC=com)
information:
LDAP Error 0x3a (58).
......................... DomainDnsZones failed test CrossRefValidation
Running enterprise tests on : DALLAS.usagroup.com
Test omitted by user request: DNS
Test omitted by user request: DNS
Starting test: LocatorCheck
GC Name: \DC1SERVER.Qingdao.DALLAS.usagroup.com
Locator Flags: 0xe00073fd
PDC Name: \DC1SERVER.Qingdao.DALLAS.usagroup.com
Locator Flags: 0xe00073fd
Time Server Name: \DC1SERVER.Qingdao.DALLAS.usagroup.com
Locator Flags: 0xe00073fd
Preferred Time Server Name: \DC1SERVER.Qingdao.DALLAS.usagroup.com
Locator Flags: 0xe00073fd
KDC Name: \DC1SERVER.Qingdao.DALLAS.usagroup.com
Locator Flags: 0xe00073fd
......................... DALLAS.usagroup.com passed test LocatorCheck
Starting test: Intersite
Doing intersite inbound replication test on site STAGING:
Locating & Contacting Intersite Topology Generator (ISTG) ...
The ISTG for site STAGING is: DC1SERVER.
Checking for down bridgeheads ...
*Warning: Remote bridgehead MADISON\TDGAD is not eligible as a bridgehead due to too many failures.
Replication may be disrupted into the local site STAGING.
Bridghead STAGING\DC1SERVER is up and replicating fine.
Remote bridgehead STAGING\DC1SERVER also couldn't be contacted by dcdiag. Check this server.
*Warning: Remote bridgehead WHITEHOUSE\NJ-DC-02 is not eligible as a bridgehead due to too many
failures. Replication may be disrupted into the local site STAGING.
*Warning: Remote bridgehead WHITEHOUSE\NJ-DC-01 has some replication syncs failing. It will be 1 hours
3 minutes before the bridgehead is considered ineligible to be a bridgehead.
*Warning: Remote bridgehead STAGING\DALLAS-EXTRADC is not eligible as a bridgehead due to too many
failures. Replication may be disrupted into the local site STAGING.
Remote bridgehead STAGING\DALLAS-EXTRADC also couldn't be contacted by dcdiag. Check this server.
Doing in depth site analysis ...
Remote site MADISON is replicating to the local site STAGING the writeable NC ForestDnsZones correctly.
Remote site MADISON is replicating to the local site STAGING the writeable NC Schema correctly.
Remote site MADISON is replicating to the local site STAGING the writeable NC Configuration correctly.
Remote site MADISON is replicating to the local site STAGING the writeable NC Qingdao correctly.
Remote site WHITEHOUSE is replicating to the local site STAGING the writeable NC ForestDnsZones
correctly.
Remote site WHITEHOUSE is replicating to the local site STAGING the writeable NC Schema correctly.
Remote site WHITEHOUSE is replicating to the local site STAGING the writeable NC Configuration
correctly.
Remote site WHITEHOUSE is replicating to the local site STAGING the writeable NC Qingdao correctly.
Skipping site JEFFERSONVILLE, this site is outside the scope provided by the command line arguments provided.
Skipping site TEXASCITY, this site is outside the scope provided by the command line arguments provided.
Skipping site MADISON, this site is outside the scope provided by the command line arguments provided.
Skipping site WHITEHOUSE, this site is outside the scope provided by the command line arguments provided.
......................... DALLAS.usagroup.com passed test Intersite
PS C:\Users\Administrator>
dallas-extradc: 172.31.15.104
repadmin /showrepl
PS C:\Users\administrator.QINGDAO> repadmin /showrepl
Repadmin: running command /showrepl against full DC localhost
STAGING\DALLAS-EXTRADC
DSA Options: IS_GC
WARNING: Not advertising as a global catalog.
Site Options: (none)
DSA object GUID: 6fbc7b6c-e934-44c3-9db1-5f209f481fa2
DSA invocationID: 8126e391-9613-4320-9a4c-57b45c88f379
Source: STAGING\DC1SERVER
******* 82 CONSECUTIVE FAILURES since 2025-02-26 13:18:00
Last error: 8464 (0x2110):
Synchronization attempt failed because the destination DC is currently waiting to synchronize new partial at
tributes from source. This condition is normal if a recent schema change modified the partial attribute set. The destina
tion partial attribute set is not a subset of source partial attribute set.