SC 2012 VMM: Collecting storage related traces

SC 2012 VMM ships with a new service called the Storage Management Service which communicates with SMI-S based providers from vendors like NetApp, EMC, HP, and IBM. To help you troubleshoot problems, SCVMM include lots of storage related tracing information in its own logs and whenever possible, the CIMXML output from the providers. In some cases however, you will need to get CIMXML output and trace output from the Storage Management service directly to help you troubleshoot further.

 

So there are three levels of tracing that you need to keep in mind: 

Enable VMM tracing

SCVMM traces will give you the error/exception information. Make sure to collect traces on the VMM server. Hyper-V host traces are only necessary if the failure us on the Hyper-V side (e.g. volume mount issues)

Refer to this KB article for instructions on how to setup SCVMM tracing: https://support.microsoft.com/kb/970066

Enable SCX CIM-XML command tracing

Storage Management Service communicates with SMI providers using CIMXML. This output is the raw call and response interaction between the service and the provider. This information is very verbose so collecting this information only when you reproduce the issue will help minimize the noise.

  1. Add registry subkey 
    1. W2K8R2/SP1: HKLM\Software\Microsoft\Storage Management\CIMXMLLog
    2. W2K12: HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Storage Management\CIMXMLLog
  2. Add registry DWORD value LogLevel, value 4
  3. Add registry String value LogFileName and assign it to the full path and file name to use for logging. Make sure the directory exists and that the Network Service account has read/write access
  4. Stop and start “Microsoft Storage Management Service”

Here is a sample of the information contained in the trace:

Traceview ETL

Finally, the Storage Management Service has its own trace output which you collect using Traceview.

  1. Download WDK 7.1 https://www.microsoft.com/download/en/details.aspx?displaylang=en&id=11800 and install on VMM server
  2. Unzip traceview.zip (attached to this blog post) to a local folder where VMM is installed
  3. Copy  traceview.exe from WDK folder  to this folder
  4. Run traceview.exe as administrator
  5. File->Open Workspace, select scx and click ok

With this tracing will start but you might not see anything in the UI until SCVMM communicates with the Storage Management Service. Refresh a provider to ma

The information in the UI also gets logged to storageservice.etl file in same folder as TraceView.  

 

TraceView.zip

Comments

  • Anonymous
    April 27, 2014
    Is it possible the TMF files in the traceview are old (eg. 1b3da1d4-9dfb-00d7-1710-f5aab27cdfb9.tmf). They have a date of 9-2011 are the TMF files still appropriate for SCVMM 2012r2 (
    version 3.2.7510.0) Reason I ask is I get the error,
    No Format Information found. listed in link as
    http://msdn.microsoft.com/en-us/library/windows/hardware/ff553881(v=vs.85).aspx failing to get the proper tmf file.
    If so where do I get the scvmm 2012r2 provider tmf files. Unknown 24 10FC 112C 0 1 00�0 0-00:00:00 Unknown( 24): GUID=6c793b74-4825-f279-2f63-9b6e89b34fac (No Format Information found). I do see the provider listed when I create a session so it seems to be finding the tmf files.