Working with File Shares in Windows Server 2008 (R2) Failover Clusters
I know what you are thinking, “How hard can it be to work with cluster file shares?”. I would be willing to bet a lot of you have been working with File Server clusters since NT 4.0 days. If you are still working with them today in Windows Server 2008 R2, you know things have changed. In this blog, I hope to give you some insight into a piece of functionality both within Failover Cluster and Explorer that may alter the way you work with file shares in your organization. It may even help finally solve a mystery that has been plaguing some of you for a while now.
I will be working with a 2-Node Windows Server 2008 R2 Failover Cluster (Figure 1).
Figure 1
In the cluster, I created a highly available File Server (CONTOSO-FS1). I created a series of folders, using the Explorer interface, on the storage in the File Server resource group (Figure 2).
Figure 2
I use the folders to make shares highly available in the CONTOSO-FS1 File Server resource group.
There are three main ways to provision shares in a Failover Cluster using built-in GUI tools.
1. Failover Cluster Management snap-in
2. Share and Storage Manager snap-in
3. Explorer interface
In the Failover Cluster Management interface, the Add a shared folder function is available in the Actions pane (Figure 3).
Figure 3
In the Share and Storage Management interface, the Provision Share function is available in the Actions pane (Figure 4).
Figure 4
In Explorer, you simply Right-Click on the folder and Share with users (or nobody to stop sharing) (Figure 5).
Figure 5
The end result using any of these three methodologies is shared folders appearing in the Failover Cluster Manager snap-in in the CONTOSO-FS1 resource group (Figure 6).
Figure 6
A similar display can be seen in Share and Storage Manager (Figure 7).
Figure 7
Inspecting the cluster registry hive, we can see the shares defined under the appropriate File Server Resource (FileServer-(CONTOSO-FS1)(Contoso-FS1 (Disk)) (Figure 8).
Figure 8
At this point you may be thinking, “So what Chuck. This isn’t rocket science. We know all this stuff.” And, you may be right. Setting up the shares is the easy part, and we provide you with several methods with which to accomplish this, but what happens when you no longer want to share ‘stuff’ anymore? This is where it could get a little interesting.
If you do not want to share a folder anymore, there are three correct ways to do this.
Option 1: In the Failover Cluster Management interface, Right-Click on the shared folder and select Stop Sharing (Figure 9).
Figure 9
Option 2: In the Share and Storage Manager interface. Right-Click on the share and select Stop Sharing (Figure 10).
Figure 10
Option 3: In Windows Explorer, Right-Click on the folder and select Share with Nobody (Figure 11).
Figure 11
The unexpected behavior occurs in the Explorer interface if instead of choosing to stop sharing by executing the process in Figure 11, the user chooses to Delete the folder (Figure 12). There could be unintended consequences for that action.
Figure 12
In Explorer, when the folder is selected for deletion, a pop-up Confirmation window is displayed. An example of one is shown in Figure 13.
Figure 13
If Yes is selected, the folder is deleted. In the Failover Cluster Management interface, however, the shared folder that was just deleted in Explorer is still displayed and appears to be Online (Figure 14).
Figure 14
Even the cluster registry hive will show the share present under the File Server resource (Figure 15).
Figure 15
Note: In previous versions of clustering, the cluster service maintained cluster file share information in the registry key HKLM\System\CurrrentControlSet\Services\LanmanServer\Shares.
Here is the punch line – the next time the File Server Resource is cycled Offline and then back Online again (like during a Failover of the resource group to another node in the cluster), an Error (Event ID 1588) will be registered in the System Event Log (Figure 16). The error indicates that the share that cannot be found also cannot be brought Online by the File Server resource.
Figure 16
The cluster log reports a problem as well but it is only a Warning (Figure 17).
00000944.00000688::2010/08/07-18:05:31.183 WARN [RES] File Server <FileServer-(CONTOSO-FS1)(Contoso-FS1 (Disk))>: Failed in NetShareGetInfo(CONTOSO-FS1, Pictures), status 2310. Tolerating...
00000944.00000b04::2010/08/07-18:06:31.185 WARN [RES] File Server <FileServer-(CONTOSO-FS1)(Contoso-FS1 (Disk))>: Failed in NetShareGetInfo(CONTOSO-FS1, Pictures), status 2310. Tolerating...
00000944.00000590::2010/08/07-18:07:31.190 WARN [RES] File Server <FileServer-(CONTOSO-FS1)(Contoso-FS1 (Disk))>: Failed in NetShareGetInfo(CONTOSO-FS1, Pictures), status 2310. Tolerating...
00000944.00000830::2010/08/07-18:08:31.194 WARN [RES] File Server <FileServer-(CONTOSO-FS1)(Contoso-FS1 (Disk))>: Failed in NetShareGetInfo(CONTOSO-FS1, Pictures), status 2310. Tolerating...
00000944.00000b48::2010/08/07-18:09:31.197 WARN [RES] File Server <FileServer-(CONTOSO-FS1)(Contoso-FS1 (Disk))>: Failed in NetShareGetInfo(CONTOSO-FS1, Pictures), status 2310. Tolerating...
Figure 17
Decoding Status 2310 (Figure 18)
Figure 18
These errors in the System Event Log do not prevent the File Server resource from coming Online and bringing all the other valid shared folders Online (except if it were the last shared folder associated with the File Server resource. See the ‘bonus material’ at the end of the blog). However, I think you can quickly see that the process of deleting shared folders instead of just stopping them from being shared can, over time, accumulate orphaned entries in the cluster registry hive and the Event ID 1588 Error messages will continue to be registered for each of the ‘orphaned’ shares.
One way this behavior manifests itself is if a shared folder is created in Failover Cluster Manager or Share and Storage Manager, and is then deleted in Explorer. The Event ID 1588 is registered because the cluster registry hive is not ‘cleaned’ up properly. If the folder is shared in Explorer and then subsequently deleted in Explorer, a different pop-up Warning is displayed (Figure 19).
Figure 19
If folders are not deleted but instead are just stopped from being shared, then the cluster is cleaned up properly and the error should not be registered. If the pop-up in Figure 19 is displayed (as opposed to the pop-up shown in Figure 13), then the share will be properly removed from the Failover Cluster and the cluster registry hive will be properly cleaned up.
Another scenario where we could see an Event ID 1588 registered, but not be the result of the cluster registry hive not being cleaned up properly, would be where the System account had been removed from the default security setting for a folder that was shared in a Failover Cluster.
Bonus Material:
What happens if the final shared folder that is associated with a File Server Resource is deleted? At the first LooksAlive\IsAlive check, the File Server resource will fail. A failover will be initiated, but in the end, the File Server Resource will remain in a Failed state. An Event ID 1587 (Figure 20) could be registered along with the customary Event ID 1069 reporting a cluster resource failure.
Figure 20
The cluster log entry will be different from the previous entry (Figure 17) as shown in the highlighted section below (Figure 21). This time it is not a Warning but an Error ([ERR]) that is seen in the cluster log.
00000720.00000a70::2010/08/10-22:25:13.616 INFO [RES] File Server <FileServer-(CONTOSO-FS1)(Contoso-FS1 (Disk))>: Shares 'are being scoped to virtual name CONTOSO-FS1
00000720.00000a70::2010/08/10-22:25:13.616 DBG [RHS] Resource FileServer-(CONTOSO-FS1)(Contoso-FS1 (Disk)) called SetResourceStatus: checkpoint 2. Old state OnlinePending, new state OnlinePending
00000720.00000a70::2010/08/10-22:25:13.616 WARN [RES] File Server <FileServer-(CONTOSO-FS1)(Contoso-FS1 (Disk))>: Failed to open path e:\Documents. Error: 2. Maybe a reparse point...
00000720.00000a70::2010/08/10-22:25:13.616 ERR [RES] File Server <FileServer-(CONTOSO-FS1)(Contoso-FS1 (Disk))>: Failed to open path e:\Documents with reparse flag. Error: 2.
00000720.00000a70::2010/08/10-22:25:13.616 ERR [RES] File Server <FileServer-(CONTOSO-FS1)(Contoso-FS1 (Disk))>: Failed to online a single share among 1 shares.
00000720.00000a70::2010/08/10-22:25:13.616 DBG [RHS] Resource FileServer-(CONTOSO-FS1)(Contoso-FS1 (Disk)) called SetResourceStatus: checkpoint 2. Old state OnlinePending, new state Failed
00000720.00000a70::2010/08/10-22:25:13.616 ERR [RHS] Online for resource FileServer-(CONTOSO-FS1)(Contoso-FS1 (Disk)) failed.
Bonus Material [Resolution]: If you have intentionally deleted the last shared folder via Explorer and found yourself in the above described state, you can safely delete the File Server resource. If you need to share out folders on that same drive, the first time you do so, the File Server resource will automatically recreate.
I hope this information has been helpful and perhaps solved a few mysteries out there.
Thanks for your attention and come back.
Chuck Timon
Senior Support Escalation Engineer
Microsoft Enterprise Platforms Support
Comments
- Anonymous
January 01, 2003
@Mohamed B
If you bring up the properties of the File Server resource and go to the Policies tab, you can control what you want it to do. If you want it to just fail and not do anything else, you can mark it as "If resource fails, do not restart". Then it will just sit there in a failed state and not move the group anywhere. - Anonymous
August 20, 2010
Nice post - wish there'd been a post as clear as this when I first encountered a w2k8 cluster. I shall add this to my store of essential documents. Thanks. - Anonymous
December 09, 2010
Great post! Adding to our archive as well. - Anonymous
February 22, 2011
Great post, another very good document to add to my library. I appreciate the time you've taken to outline this so clearly. I do have one question though, what happens if I have a file server share resource where I have disabled Offline File Caching on the share and I add several folders and files to the shares in windows explorer (less than 2 TB), would it produce an issue such as the Event ID 1587 or 1588? Thanks for your reply - Anonymous
February 16, 2012
The comment has been removed - Anonymous
May 30, 2012
I think MS totally miffed clustering up by taking away the ability to take a share offline without deleting it. In my large enterprise, that feature is very missed and makes 2k8 R2 even more annoying. - Anonymous
September 20, 2012
Wonderful resource! Thank you. - Anonymous
October 30, 2012
gud 1. - Anonymous
February 22, 2013
Awsmmmm!! - Anonymous
March 14, 2013
Hello,I created shared folder in Folder Shared Drive and shared it within explorer.Not used Cluster Manager to sharing.And then i deleted that folder :((( Now File server disk failed in cluster manager.What should i do? :( - Anonymous
May 17, 2013
amazing thing will you give in this passage to take knowledge to learn this passage . you will take some words ,,match in the colummn to internal knowledge will be devlope. - Anonymous
July 11, 2013
I have a cluster however can not map network drives via IP only via hostname. What can be? - Anonymous
July 18, 2013
Great Post! Information about event 1588 when System Account permission is removed has saved my day :)Dan Pinheiro. - Anonymous
September 10, 2013
Can anyone point me to some documentation on how to do this with powershell 3, in particular figure 11?Thanks. - Anonymous
August 08, 2014
so by deleting the registry value for the deleted folder will solve this 1588 error issue?? - Anonymous
August 11, 2014
I like to know if deleting the registry entries for the deleted folders will help solve the issue with error 1588 referencing to the missing folder?? - Anonymous
September 22, 2014
nice... - Anonymous
January 31, 2015
So axing the registry entry will not solve problem because the cluster will still remain in a failed state. For me I had to manually recreate the folder that was missing, then reload failover cluster manager, then the drive went green and the actual share re-appaeared under share and storage management - Anonymous
February 19, 2015
“What happens if the final shared folder that is associated with a File Server Resource is deleted? At the first LooksAliveIsAlive check, the File Server resource will fail. A failover will be initiated”
This is a very very strange design and behavior ! No sysadmin or DBA on the earth want the failover occur if by mistake someone drop a shared folder, is there any setting to prevent the failover of the group occur? I believe Microsoft should fix this and just put the resource in failed state. - Anonymous
May 31, 2015
currently going through this issue badly, Please provide me the full resolution steps, - Anonymous
July 16, 2015
How can we cleanup the the orphaned Cluster shares? - Anonymous
January 19, 2016
Nice post - I came across this issue today with one of our cluster. Someone deleted the shared folder from the volume and this article was very useful. Thanks for posting.