Help! I Lost a Disk From My Storage Spaces Mirror, How Do I Recover?
I love Server 2012 Storage Spaces because it provides me the opportunity to consolidate a bunch of separate drives into one big contiguous space. One of the options on storage spaces is mirroring, which I used to create a storage pool using two 1TB drives. My goal in doing this was to leverage better read performance from mirroring and fault tolerance should one of the drives fail. The two 1TB drives give me a mirrored 1TB storage space that I use for virtual machine storage.
Recently, I indeed experienced a drive failure. The storage space tolerated this fine as the remaining drive continued to serve the data. Of course, I quickly swapped out the failed drive with a new one in order to bring my system back to full health. Once the new drive was installed and the system powered up again, I discovered getting the mirrored space back to full health was not as obvious as I had thought. My intent here is to show you as the reader how I recovered should you experience the same problem.
Below is a picture of my storage space virtual disk. PhysicalDisk4 is the newly installed disk that I intended to replace the failed (and now physically removed) disk. I added PhysicalDisk4 to the storage pool. The storage space still contained a reference to the dead disk, which you can see below as a GUID with a warning icon next to it.
The problem I had was that I could not get the mirror space to rebuild and could not remove the old disk (GUID with warning). When I right clicked on the old disk and clicked remove, I received an error stating that it could not be removed and when I chose rebuild from the context menu
on VM_Disk (to the left), it read ‘Rebuiding 0%’ for a split second, then did nothing further.
I did some searching for instructions here because what seemed obvious to me wasn't working. After not finding what I was looking for, I started bugging colleagues, who outlined the steps for me. As an overview, I needed to use the Powershell cmdlets to retire the dead disk, and then repair the mirror. Below is exactly what I did to accomplish this:
First, I started a Powershell administrator prompt.
Next, I executed get-physicaldisk to get a list of disks so I could retire the bad one:
PS C:\> get-physicaldisk
| FriendlyName | CanPool | OperationalStatus | HealthStatus | Usage | Size |
| ------------ | ------- | ----------------- | ------------ | ----- | ---- |
| PhysicalDisk3 | FALSE | OK | Healthy | Auto-Select | 111.79 GB |
| FALSE | Lost Communication | Warning | Auto-Select | 930.75 GB | |
| PhysicalDisk4 | FALSE | OK | Healthy | Auto-Select | 930.75 GB |
| PhysicalDisk1 | FALSE | OK | Healthy | Auto-Select | 930.75 GB |
| PhysicalDisk2 | FALSE | OK | Healthy | Auto-Select | 1.36 TB |
| PhysicalDisk0 | FALSE | OK | Healthy | Auto-Select | 1.36 TB |
I could now see the dead disk in the list (the second one listed). The disk had no FriendlyName and the GUID didn't work as a UniqueId (-FrendlyName and -UniqueId are 2 ways to tell get-physicaldisk what disk you're referring to), so I was not sure how to refer to it. However, Powershell is a rich environment that returns objects, not just text, so the second item in the list was actually a reference to the disk I wanted to retire. I captured the list to a variable called $results:
PS C:\> $results = get-physicaldisk
The $results variable is a zero based array, so our disk of interest is $results[1]:
PS C:\> $results[1]
| FriendlyName | CanPool | OperationalStatus | HealthStatus | Usage | Size |
| ------------ | ------- | ----------------- | ------------ | ----- | ---- |
| FALSE | Lost Communication | Warning | Auto-Select | 930.75 GB |
Since I had a reference to the disk, I could pipe it to the set-physicaldisk cmdlet and retire the dead disk:
PS C:\> $results[1] | set-physicaldisk -usage retired
PS C:\> get-physicaldisk
| FriendlyName | CanPool | OperationalStatus | HealthStatus | Usage | Size |
| ------------ | ------- | ----------------- | ------------ | ----- | ---- |
| PhysicalDisk3 | FALSE | OK | Healthy | Auto-Select | 111.79 GB |
| FALSE | Lost Communication | Warning | Retired | 930.75 GB | |
| PhysicalDisk4 | FALSE | OK | Healthy | Auto-Select | 930.75 GB |
| PhysicalDisk1 | FALSE | OK | Healthy | Auto-Select | 930.75 GB |
| PhysicalDisk2 | FALSE | OK | Healthy | Auto-Select | 1.36 TB |
| PhysicalDisk0 | FALSE | OK | Healthy | Auto-Select | 1.36 TB |
With the bad disk retired, I could then kick off a repair of my mirrored virtual disk (note that its status below is InService now, meaning it is being repaired):
PS C:\> repair-virtualdisk vm_vdisk
PS C:\> get-virtualdisk
| FriendlyName | ResiliencySettingName | OperationalStatus | HealthStatus | IsManualAttach | Size |
| ------------ | ------------------- | ----------------- | ------------ | -------------- | ---- |
| VM_VDisk | Mirror | InService | Warning | FALSE | 930 GB |
| Shares_VDisk | Mirror | OK | Healthy | FALSE | 1.36 TB |
With the repair underway on the new disk and the bad disk retired, I could now remove the bad disk (note that I'm referring to it with my $results variable):
PS C:\> remove-physicaldisk -physicaldisk $results[1] -storagepoolfriendlyname vm_pool
Confirm
Are you sure you want to perform this action?
Removing a Physical Disk will cause problems with the fault tolerence capabilities of StoragePool "VM_Pool".
[Y] Yes [A] Yes to All [N] No
[L] No to All [S] Suspend [?] Help (default is "Y"): Y
I confirmed (said Yes to) the warning because the retired dead disk was not contributing to the size or health of the pool anymore, so it is good to go.
After a few hours of rebuild (InService), the mirror was healthy and as good as new.
Comments
Anonymous
July 23, 2013
Good to know, thank you for sharing experience!Anonymous
June 16, 2014
Good man, much good. It work. Thanks.Anonymous
January 09, 2016
Great write up! The only thing I would have done differently is retire and remove the bad disk from the start, then put my new disk in and start the rebuild. That way, the new disk is on the same port at the dead disk was (and often the bays on the server are full, so there isn't room to stuff a spare drive in without removing the dead one first). I'd make sure there was a backup first, too - just in case a typo removed the wrong disk, or something grin.Anonymous
January 19, 2016
Hi, I tried this but when I hit 'Remove-PhysicalDisk' I get the following Error (after hitting Y) Remove-PhysicalDisk : The storage pool could not complete the operation because the health or operational status of one or more virtual disks in it does not permit it. Any idea?