WMI: REFERENCES OF query failure during Provider startup could be disastrous
Hello, my name is Venkatesh Ganga. I’m an Escalation Engineer on the Microsoft Platform Global Escalation Services team. In this blog I would like to talk about an interesting WMI issue I worked few months ago. The idea behind this blog is to discuss how WMI works under the hood. In this article I show a few call stacks but the focus of the article is really to point out WMI concepts.
We had a customer who was getting "Provider cannot perform the requested operation" error for all WMI queries to any provider on some of his Windows XP machines.
Also the WMI Control (wmimgmt.msc) properties show the following:-
Figure 1:
Moreover nothing happens when he clicks on any other tabs like Logging, Backup/Restore etc. Running WMIDIAG showed “24 error(s) 0x80041024 - (WBEM_E_PROVIDER_NOT_CAPABLE) Provider cannot perform the requested operation“.
All the standard WMI troubleshooting was done before I was engaged like rebuilding the WMI repository, checking DCOM permissions etc. But nothing resolved the issue. We got the image of one of the problem machines to reproduce the problem locally. Initially I didn’t know where to start to debug the issue as there were multiple symptoms. I decided to trace the execution of the inbox provider CIMWin32.dll by issuing the WMI query “Select * from Win32_Processor” and find where it fails and to go from there.
WMI maintains each provider’s physical implementation information such as the CLSID of the provider’s COM object, the hosting model, etc in an instance of the __Win32Provider system class. Each provider can support one or more types mentioned below. Based on the type(s) the provider supports, WMI will create an instance of each supported type.
· __ClassProviderRegistration
· __EventConsumerProviderRegistration
· __EventProvideRegistration
· __InstanceProviderRegistration
· __MethodProviderRegistration
· __ObjectProviderRegistration
· __PropertyProviderRegisration
For example, looking at the .mof file for the Cimwin32 provider we can quickly determine that the provider registers for 2 types, the Instance Provider and the Method Provider as Figure 2 displays.
Figure 2
Since our test query “select * from Win32_Processor” retrieves the instances of Win32_Processor class, WMI checks the CIMWin32’s __InstanceProviderRegistration object to see if the SupportsEnumeration property is set. In debugging the customer’s image, I found that the runtime object WMI maintains for CIMWin32 provider stated “False” for the SupportsEnumeration property.
WMI maintains an in-memory structure for each provider and fills that structure out with information from the repository. The CreateInstanceEnumAsync checks the provider’s in-memory structure to determine whether it supports enumeration. From my debug on the customer’s system the provider did not support enumeration.
In Memory at Runtime
0:003> kL
ChildEBP RetAddr
0091f560 77e7a1ac wmiprvse!CInterceptor_IWbemSyncProvider::CreateInstanceEnumAsync+0x21
0091f588 77ef421a rpcrt4!Invoke+0x30
0091f998 77ef4bf3 rpcrt4!NdrStubCall2+0x297
0091f9f0 756bd7fe rpcrt4!CStdStubBuffer_Invoke+0xc6
0091fa04 77600c31 fastprox!CBaseStublet::Invoke+0x22
0091fa44 77600bdb ole32!SyncStubInvoke+0x33
0091fa8c 7750f237 ole32!StubInvoke+0xa7
0091fb64 7750f15c ole32!CCtxComChnl::ContextInvoke+0xe3
0091fb80 77600b11 ole32!MTAInvoke+0x1a
0091fbb0 776009bc ole32!AppInvoke+0x9c
0091fc84 77600715 ole32!ComInvokeWithLockAndIPID+0x2e0
0091fcd0 77e79c75 ole32!ThreadInvoke+0x1cd
0091fd04 77e79bda rpcrt4!DispatchToStubInC+0x38
0091fd58 77e79b06 rpcrt4!RPC_INTERFACE::DispatchToStubWorker+0x113
0091fd7c 77e89f9c rpcrt4!RPC_INTERFACE::DispatchToStub+0x84
0091fdbc 77e89fdd rpcrt4!RPC_INTERFACE::DispatchToStubWithObject+0xc0
0091fdfc 77e7be65 rpcrt4!LRPC_SCALL::DealWithRequestMessage+0x2cd
0091fe20 77e76794 rpcrt4!LRPC_ADDRESS::DealWithLRPCRequest+0x16d
0091ff80 00000000 rpcrt4!LRPC_ADDRESS::ReceiveLotsaCalls+0x28f
Repository
In contrast when we checked the customer’s system, we determined that the repository showed that the provider’s “SupportEnumeration” property was set to true. Verification steps below:
1. Run wbemtest and connect to root\cimv2
2. Enter below query
select * from __InstanceProviderRegistration where provider="__Win32Provider.Name=\"CIMWin32\""
3. You should get 1 object in the Query Result window. Double click it and look at the properties. On a working system you should notice that the SupportEnumeration property is set to “True” for the CimWin32 provider as Figure 3 shows.
Figure 3.
Discrepancy Between the In-Memory and Repository
Now we know WMI is not initializing the provider correctly as there is a discrepancy between the in-memory objects it maintains and the repository.
So I started debugging the CIMWin32 provider initialization where WMI fills the in-memory structure. During the provider initialization WMI makes a REFERENCES OF query to get all the types CIMWin32 supports. On our customer’s machine this query yields 0 objects whereas on a working system it returns 2 objects (one for __InstanceProviderRegistration and other for __MethodProviderRegistration)
"references of {__Win32Provider.Name="CimWin32"}"
Figure 4.
Here is the debug of how we found the exact “REFERENCES OF“query being issued.
0:007> kL
ChildEBP RetAddr
0164fa38 59841a6c wbemcore!CWbemNamespace::ExecQuery
0164fa88 59841c41 wmiprvsd!CServerObject_ProviderRegistrationV1::QueryRepositoryUsingQuery+0x47
0164faac 59841d31 wmiprvsd!CServerObject_ProviderRegistrationV1::QueryRepository+0x42
0164fac8 5980f056 wmiprvsd!CServerObject_ProviderRegistrationV1::Load+0x33
0164fba8 762e9cfd wmiprvsd!CServerObject_BindingFactory::GetProvider+0x1f5
0164fc24 76301d1e wbemcore!CWbemNamespace::DynAux_ExecQueryExtendedAsync+0x81
0164fce0 76302912 wbemcore!CQueryEngine::ExecComplexQuery+0x1f4
0164fdec 763032bd wbemcore!CQueryEngine::ExecQlQuery+0x37
0164fe90 762fc769 wbemcore!CQueryEngine::ExecQuery+0x228
0164feac 762cef24 wbemcore!CAsyncReq_ExecQueryAsync::Execute+0x19
0164fed8 762ced4e wbemcore!CCoreQueue::pExecute+0x3c
0164ff08 762f25cb wbemcore!CCoreQueue::Execute+0x18
0164ff50 762cee89 wbemcore!CWbemQueue::Execute+0xf6
0164ff84 762cf055 wbemcore!CCoreQueue::ThreadMain+0x111
0164ffb4 7c80b683 wbemcore!CCoreQueue::_ThreadEntry+0x45
0164ffec 00000000 kernel32!BaseThreadStart+0x37
0:044> du 04f57424
04f57424 "references of {__Win32Provider.N"
04f57464 "ame="CIMWin32"}"
Repository driver (repdrvfs.dll) works on the above references query. The references query failed with error 80041017 (WBEM_E_INVALID_QUERY). It failed due to the apostrophe in the computer name. While parsing the query the repository driver considers anything between the apostrophes or quotes as a name. This apostrophe in the computer name misleads WMI to look for a closing apostrophe which is not there and hence the references query fails with error.
0:005> kL
ChildEBP RetAddr
05cbfec8 75214a74 repdrvfs!CNamespaceHandle::ExecReferencesQuery+0x1dd
05cbff88 75215518 repdrvfs!CNamespaceHandle::ExecQuerySink+0xc2
05cbffac 7520e11c repdrvfs!CExecQueryObject::Execute+0x28
05cbffb4 7c80b683 repdrvfs!A51FiberBase+0xd
05cbffec 7c82ffa9 kernel32!BaseThreadStart+0x37
05cbffec 00000000 kernel32!BaseFiberStart+0x17
0:005> du 00b56218
00b56218 "references of {\\COMPUTERNAME'User1"
00b56258 "\ROOT\cimv2:__Win32Provider.Name"
00b56298 "="CIMWin32"}"
0:005>r
WMI makes similar REFERENCES OF query for all the providers to get the types they support. But they all fail for same reason. Hence WMI queries to all the providers fail. We renamed the computer name without apostrophe and rebooted the box. Now WMI initializes properly and the WMI queries start working again. An apostrophe is not a valid character for the computer name. We never found out how the computer name got set with a apostrophe. If we try to set the computer name through GUI it throws error saying the computer name has invalid characters. However, hopefully through this experience, you have a better understanding of how WMI implements providers to help you better troubleshoot your next WMI issue.
So in review here are the main takeaways from this blog:
1. WMI maintains an instance of __Win32Provider system class for each provider. Based on the provider’s registration (see .mof file above) it maintains one or more registration system classes such as __InstanceProviderRegistration.
2. WMI uses the REFERENCES OF query to map the instance of __Win32Provider to the providers registration types (e.g. __InstanceProviderRegistration).
3. To view the various providers registered on your system, connect to root\Cimv2 and issue the following query: “Select * from __Win32Provider”.
4. And to get the types that each provider supports, issue the following query:
a. references of {__Win32Provider.Name="<PROVIDER_NAME>"}