Where did my volume label go?
I recently worked an interesting case where 3rd party application had a dependency on the label name for a volume. The customer reported that the label name was actually missing and therefore the application would not start. Further, the label name was present on Windows 2003 servers and not on Window 2008 R2. Finally, the last piece of information shared on scoping call was application server mapped a logical drive letter, where the label name was required, using a 3rd party storage filer. I had never heard of an application having a dependency on a “label name” and further was concerned that the interoperability between Windows and 3rd party storage appliance could be a factor.
What is an example of a label name and what is it? From a Windows internals perspective, the label name is stored per volume in a NTFS metafile called $Volume. A file record within exists called $Volume_Name that contains the label name. Example from UI where ‘Demo’ is the volume label assigned:
Based on the definition above, when accessing a share remotely, you do not have block level access, the data is being processed through a network file redirector on your behalf. In this case, the Windows member server running the client application is merely the client receiving this information over the network.
For me, the main questions were:
1. Does this work as expected taking 3rd party appliance out of the equation?
2. Is the Windows 2008 R2 server receiving the expected data but not rendering it in the UI or is the Windows 2008 R2 server not receiving expected data from 3rd party filer at all?
In my lab I quickly mocked up 3 scenarios as to compare some of the legacy operating systems to current operating system for any differences simply using Windows to Windows. Can we reproduce the issue without 3rd party?
No. When using Windows as the server sharing the volume, the volume label was always present on the client. From the customer perspective, this was a Windows problem for sure. How could it not be since the Windows 2003 server displayed the label name and Windows 2008 R2 did not?
My next step was to collect a network trace of the working and nonworking scenario with hopes this would provide a clue on root cause. I collected a ‘working’ trace using a new protocol analyzer called (MMA)Microsoft Message Analyzer which has replaced Netmon.
Alternatively you can collect a trace with Netsh and ETW provider in the event you can’t or don’t want to install MMA on the system you’re investigating.
ipconfig /flushdns
klist purge
netsh trace start capture = yes
<repro>
netsh trace stop
This will create a ETL file that you can consume with MMA. You can also export the trace into a .cap or .pcap in case you want to use Netmon or WireShark. Current versions of the OS like 2012 R2 allow remote captures with MMA but that’s for another blog
Now that we have the trace to review, leverage a feature in MMA I really like is called protocol dashboard which shows a summary of all protocols in the trace. Since we are mapping a drive we know this should be a SMB or Server Message Block conversation. Our working baseline looks like this:
The primary protocol used for the working scenario was SMB. You can access this feature “Protocol Dashboard” by right clicking on the trace and selecting “New viewer”…
We should document was version of SMB works and verify that label name is received. Using a filter (SMB.Summary contains "Negotiate") I’m able to quickly see the following:
What does NTLM0.12 mean? Fortunately reference material Definition of SMB version numbers defines it to mean SMBv1 for Windows XP\2003. This must mean that “SMB” in the protocol summary with MMA is really SMBv1. Since our working scenario is on a Windows 2003 server this makes sense. Finally, where is the label name in the working trace? Open the trace with MMA
*IPs are from private nonroutable network
* Due to real estate and formatting you may need to click on the picture to see image with best quality
Now we need to add a filter to locate the Label name. I’ve found that by typing in SMB then DOT pops the intellisense feature.. SMB.V will pop a list of methods to choose from, one of which is SMB.VolumeLabel. Inserting that into filter, click apply makes the trace much easier to review.
Let’s click on message 539 and see the request\response.
Expand the response message until you see the label name. Demo is the string Windows received from the target hosting the share. Now we have the signature of what to look for. If we look at the broken trace and see the VolumeLabel populated then this is a Windows bug. If we don’t receive the value then it’s a bug with the 3rd party appliance. One additional tip, if you right click on the VolumeLabel item a fly out window pops up, select “Add as Analysis Grid Column” which will add a column which makes it easier to see.
Let’s open the broken trace and see what’s different including the new column…
No messages are shown. Why? Let’s take a look at the protocol dashboard for the broken trace to understand why. Working is on the left and broken on the right. This will lead to the first break in the case. The working trace is using SMBv1 and broken is SMBv2 which also lines up with the Operating System. Windows 2003 SMBv1 and 2008 R2 SMBv2.
Let’s confirm by comparing the dialect negotiation as we did earlier. Confirmed SMB2, earlier it was NTLM0.12.
Let’s change our filter from SMB.VolumeLabel to SMB2.VolumeLabel. This is the customer trace so I’ve remove everything except the string we are interested in.
Our answer is in the absence of data. The fact is the 3rd party appliance is not providing the VolumeLabel to Windows and therefore we are not displaying it. Our options…
1. Fix the application to remove dependency on label name
2. File (DCR) Design Change Request with 3rd party appliance to include label name over SMB2 just like they did in SMBv1
3. (Not recommended) Follow the following KB and force SMBv1 on Windows 2008 R2
How to enable and disable SMBv1, SMBv2, and SMBv3 in Windows Vista, Windows Server 2008, Windows 7, Windows Server 2008 R2, Windows 8, and Windows Server 2012
https://support.microsoft.com/kb/2696547
I hope you found the blog interesting and hope you give Message Analyzer a try for future protocol analysis.