Every 20 min an Event 1069 and 1558?
DISCLAIMER: Event 1069 is an generic event therefore you should check and decide by your own if this blog post is an possible solution for your scenario and environment. This can be verified in the cluster logs when you identify " 'Failed to create cluster directory on witness” ! which is generated by Quorum Agent.
Problem:
In a customer scenario we had identified every 20 min the events 1069 and 1558. They had pointed to quorum disk issues which we could not confirm in the 1st step as the disk was online and could be failed over to other nodes without any issues.
Event ID 1069 — Clustered Service or Application Availability
https://technet.microsoft.com/en-us/library/cc756225(WS.10).aspx
Event ID 1558 — Cluster Witness Functionality
https://technet.microsoft.com/en-us/library/dd353960(WS.10).aspx
After digging deeper and doing an cluster log analyses I had found an very interesting pointer there:
ERR mscs::QuorumAgent::PostOnline: ERROR_PATH_NOT_FOUND(3)' because of 'Failed to create cluster directory on witness, path \\?\Volume{5c65f7b0-15e4-11e0-b316-002655db949a}\Cluster'
This tells me, that the cluster services has issues when trying to access the quorum disk when “he” want to write his cluster hive (=configuration) to the quorum disk.
As the cluster hive is “redundantly” available on each in the cluster and can also be manually created anytime when changing the cluster quorum configuration in your cluster, I used this scenario for troubleshooting my issue here.
Solution:
1. Changing the Quorum Modell temporary to “Node Majority” so that I can remove the Quorum Disk “Q:” from the cluster (Note: When changing quorum model, be aware of the available “votes” (keep majority) in your cluster)
2. Remove Quorum Disk from Cluster , Re-Format with NTFS and back to Cluster
3. Restore Quorum model – in my case “Node und Disk Majority” and point to new-formatted Quorum disk Q:
Result:: Die Cluster Hive is newly created on Quorum Disk Q:\
After the cluster hive is successfully created from cluster service, all entries in cluster logs and also events 1069 and 1558 are gone
Information:
General Information’s around Cluster Logs can be found here:
How to create the cluster.log in Windows Server 2008 Failover Clustering
https://blogs.msdn.com/b/clustering/archive/2008/09/24/8962934.aspx
Troubleshooting Cluster Logs 101 - Why did the resources failover to the other node?
https://blogs.technet.com/b/askcore/archive/2008/02/06/troubleshooting-cluster-logs-101-why-did-the-resources-failover-to-the-other-node.aspx
Introduction to Cluster Diagnostics and Verification Tool for Exchange Administrators
https://technet.microsoft.com/en-us/library/aa996161(EXCHG.65).aspx
….wish you good luck with “troubleshooting”
Best Regards
Ramazan
Comments
- Anonymous
August 23, 2011
The comment has been removed - Anonymous
December 05, 2011
worked for me! thanks for the good info. - Anonymous
January 22, 2014
Thanks a lot. Worked like a charm!