Create Listener Fails with Message 'The WSFC cluster could not bring the Network Name resource online'
One of the most common configuration issues customers encounter is availability group listener creation. When creating an availability group listener in SQL Server, you might encounter the following messages:
Msg 19471, Level 16, State 0, Line 2
The WSFC cluster could not bring the Network Name resource with DNS name '<DNS name>' online. The DNS name may have been taken or have a conflict with existing name services, or the WSFC cluster service may not be running or may be inaccessible. Use a different DNS name to resolve name conflicts, or check the WSFC cluster log for more information.
Msg 19476, Level 16, State 4, Line 2
The attempt to create the network name and IP address for the listener failed. The WSFC service may not be running or may be inaccessible in its current state, or the values provided for the network name and IP address may be incorrect. Check the state of the WSFC cluster and validate the network name and IP address with the network administrator.
When using SQL Server Management Studio to add the listener to an existing availability group, these errors may appear in a dialog box:
Diagnosing Listener Creation Failure
The majority of time, listener creation failure resulting in the messages above are due to a lack of permissions for the Cluster Name Object (CNO) in Active Directory to create and read the listener computer object.
The CNO is the Windows Cluster computer object itself. If you created your Windows Cluster and named it AGCluster - it would appear in Failover Cluster Manager. This is your CNO:
In order to confirm that a lack of CNO permissions is responsible for listener creation failure, launch Powershell with elevated privileges and generate the cluster log on the node hosting the availability group primary replica, where you failed to create the listener:
Confirm the problem is CNO permissions
Open the cluster log using Notepad. Here are examples of errors raised due to the CNO lack of permissions, when attempting to create the listener. Some important information these errors tell us:
1 The domain controller where those permissions are being checked. In the below examples, it is in domain 'AGDC.' Therefore, when modifying the permissions in Active Directory be sure it is done in that DC.
2 The container that the listener is being created in. In the below examples, it is the Computers container. This is important - you need to know the container in Active Directory to assign the CNO permissions on.
Example
00000be0.00001a4c::2014/03/18-15:32:06.068 INFO [RES] Network Name <ag_aglisten>: Using domain controller\DC.AGDC.COM.
00000be0.00001a4c::2014/03/18-15:32:06.237 INFO [RES] Network Name <ag_aglisten>: Failed to find a computer account for aglisten. Attempting to create one on DC \DC.AGDC.COM.
00000be0.00001a4c::2014/03/18-15:32:06.237 INFO [RES] Network Name <ag_aglisten>: Trying NetUserAdd() to create computer account aglisten on DC \DC.AGDC.COM in default Computers container
00000be0.00001a4c::2014/03/18-15:32:06.295 ERR [RES] Network Name <ag_aglisten>: Unable to create computer account aglisten on DC \\DC.AGDC.COM, in default Computers container, status 5
000004f8.00002154::2014/03/18-15:32:06.491 INFO [RCM] rcm::RcmGum::SetDependencies(ag_aglisten)
00000be0.00001a4c::2014/03/18-15:32:06.492 ERR [RHS] Online for resource ag_aglisten failed.
Example
0000125c.00000f0c::2014/01/14-21:48:31.533 ERR [RES] Network Name: [NNLIB] Binding to DC \\DC.AGDC.COM (domain AGDC.COM) failed, status 5.
0000125c.00000f0c::2014/01/14-21:48:31.533 INFO [RES] Network Name <ag_aglisten>: AccountAD: End of Slow Operation, state: Initializing/Writing, prevWorkState: Writing
0000125c.00000f0c::2014/01/14-21:48:31.533 WARN [RES] Network Name <ag_aglisten>: AccountAD: Slow operation has exception ERROR_ACCESS_DENIED(5)' because of 'status'
0000125c.00000f0c::2014/01/14-21:48:31.533 INFO [RES] Network Name: Agent: OnInitializeReply, Failure on (2a487cec-66af-4ea6-92d4-95bc70f082a4,AccountAD): 5
Example
000007c8.00001720::2014/03/27-15:48:21.804 INFO [RES] Network Name <ag_aglisten>: AccountAD: OU name for VCO is CN=Computers,DC=AGDC,DC=COM
...
000007c8.00001720::2014/03/27-15:48:21.845 INFO [RES] Network Name: [NNLIB] Using legacy API to create object in default container CN=Computers,DC=AGDC,DC=COM
000007c8.00001720::2014/03/27-15:48:22.165 INFO [RES] Network Name: [NNLIB] NetUserAdd object aglisten on DC: \\DC.AGDC.COM , result: 0
...
000007c8.00001720::2014/03/27-15:48:22.387 INFO [RES] Network Name <ag_aglisten>: AccountAD: Giving write permissions for OS and OS version attributes for the core netname
000007c8.00001720::2014/03/27-15:48:22.450 INFO [RES] Network Name <ag_aglisten>: AccountAD: Setting OS and OS version in AD
00001298.000015f8::2014/03/27-15:48:22.450 INFO [NM] Received request from client address SQLNODE1.
000007c8.00001720::2014/03/27-15:48:22.498 WARN [RES] Network Name <ag_aglisten>: AccountAD: Setting OS attributes failed with error 5
Resolve CNO Permissions or Provision the Listener Computer Object
Option 1 Grant CNO necessary permissions
Grant the permissions "Read all properties" and "Create Computer objects" to the CNO via the container. Here's an example of granting the required permissions for demonstration purposes:
1. Open the Active Directory Users and Computers Snap-in (dsa.msc).
2. Locate the “Computers” container or the container that the listener is being created in, this is assuming that this is the container where the listener computer object creation is being attempted.
3. Right-click View and select "Advanced Features."
4. Right-click the Computers container and choose Properties.
5. Click the Add button and enter the cluster named object (CNO). In this example, it is agcluster$. Click the Object Types button, check 'Computers' and click Ok and then Ok again.
6. Back in the Properties dialog, click the Advanced button and the "Advanced Security Settings for SQL" dialog should appear.
7. Click the Add button. Select "Read all properties" and "Create Computer objects." Click OK until you're back to the Active Directory Users and Computer window.
8. Attempt to recreate the listener.
Option # 2 Pre-Stage the VCO
This option is useful in situations where the domain administrator does not allow the CNO “Read All Properties” and “Create computer Objects” permissions:
1. Ensure that you are logged in as a user that has permissions to create computer objects in the domain.
2. Open the Active Directory Users and Computers Snap-in (dsa.msc).
3. Right-click View and select "Advanced Features."
4. Right click the OU/Container you want the VCO to reside in and click “New” -> “Computer.” In the example below, we are creating the listener object in the Computers container.
5. Provide a name for the object (this will be your listener name) and click “OK."
6. Right click the VCO you just created and select “Properties”. Click the Security tab.
7. Under Security tab, click the Add button. Enter the cluster named object (CNO). In this example, it is agcluster$. Click the Object Types button. Select Computers and click Ok.
8. Highlight the CNO, check the following permissions, and click “OK” (alternatively, choose Full Control)
Read
Allowed To Authenticate
Change Password
Receive As
Reset Password
Send As
Validate write To DNS Host Name
Validate Write To Service Principle Name
Read Account Restrictions
Write Account Restrictions
Read DNS Host Name Attributes
Read MS-TS-GatewayAccess
Read Personal Information
Read Public Information
9. Attempt to create the availability group listener.
Comments
Anonymous
June 26, 2014
Wonderful. Thanks this resolved my issue!!Anonymous
October 14, 2014
Great Job thank you !Anonymous
October 24, 2014
Thanks a million. Issue resolvedAnonymous
November 10, 2014
Thank you sir, saved my bacon. :)Anonymous
May 26, 2015
Thanks so much for this post, Do you know how this can be done in powershell?Anonymous
August 27, 2015
To answer to Moh Kokoy's question, through powershell it can be done like : New-Cluster -Name CN=MyCluster,OU=Cluster,DC=Contoso,DC=com -Node node1,node2Anonymous
November 24, 2015
The comment has been removedAnonymous
January 28, 2016
Great article, thankyou very much - saved a world of painAnonymous
February 03, 2016
Thanks! It helps a lot!Anonymous
April 19, 2016
Great Information it worked fine, Thank you very much savedAnonymous
April 27, 2016
Amazing article. Thanks so much.Anonymous
June 08, 2016
Thanx! Saved my time!Anonymous
June 16, 2016
Thanks for sharing such a nice article. We are able to resolve the permissions issue.Anonymous
September 21, 2016
Thanks, saved my day :)Anonymous
September 21, 2016
The comment has been removedAnonymous
October 18, 2016
Doesnt work for me... i don't have clustername$ in my AD at all ??Anonymous
December 19, 2016
Great resolution. Thanks for the informationAnonymous
January 06, 2017
Excelente solución, problema resuelto- Anonymous
January 06, 2017
Very helpful - thank you!
- Anonymous
Anonymous
January 24, 2017
This didn't work for me, but in reviewing the logs, I found a similar issue with the cluster CNO account showing login failed to the domain. There is a way to reset it in the Failover Cluster Manager by right clicking on the Cluster Core Resource - Name, taking it offline, and then clicking repair. After that, I was able to create the listener. The blog post below talks about this process.https://blogs.msdn.microsoft.com/clustering/2013/12/13/understanding-the-repair-active-directory-object-recovery-action/Anonymous
June 05, 2017
Tks so much！！Anonymous
July 26, 2017
Has anyone done this is Azure Active Directory Domain Services?Anonymous
December 22, 2017
Resolved,Great Information.Thank you very muchAnonymous
January 09, 2019
In large and well managed enterprise environment recommend to go with option #2