SQL Server troubleshooting: SQL Server Cluster Resources Failed to Come Online after removal and adding Node back to cluster.
Problem:
On a 3 Node Cluster NODE B/NODE C SQL Server Cluster Resources Failed to Come Online on a NODE A after removal and adding Node back to cluster.
Event ID 1069: Cluster resource 'SQL Server (SQLPC)' in clustered service or application 'SQL (C)' failed.
Event ID 1230 : Cluster resource 'SQL Server (SQLPC)' (resource type '', DLL 'sqsrvres.dll') either crashed or deadlocked. The Resource Hosting Subsystem (RHS) process will now attempt to terminate, and the resource will be marked to run in a separate monitor.
In this picture we can note that except SQL Server Resources all other resources are coming online.
Issue Caused:
SQL Server Services are missing in services.msc for the rest of the nodes hence resource is not coming online.
But SQL Services of NODE A will come online as services are present for this instance.
Resolution:
- S1: NODE A: Move the instance to C.
- S2: NODE B: Move the instance to C.
- S3: Make sure NODE A and NODE B are passive.
- S4: Now on NODE A, Start add node from CMD/GUI.
- E:\SQL 2012 Ent Edition with SP2 64bit\Setup.exe /SkipRules=Cluster_VerifyForErrors /Action=AddNode
- S5: Now select NODE B, i.e SQLPB and click install and follow on screen guidelines (this will install services)
- S6: Install SP and CU applicable on NODE A only for SQLPB (NODE B Instance) to bring instance to the same level .
- S7: Verify the SQL Server Services are installed.
- S8: Now move Node A-A, B-B verify
- S9: Now move Node B- A.
- S10: All resources will be online.
Repeat above steps for other node (move C-B, B-B, Node A and Node C is passive) too.