KB969269 - VMM Admin Console crashes with errors 19999 and 1 as logged in VM Manager
Anyone ever had problems with the admin console crashing? How about the crash happening each time the admin console refreshes? Or perhaps troubleshooting has identified that the problem only occurs when a particular host is refreshed? Ever wonder why this is happening? While there are many potential causes of this problem, I want to take a few minutes and discuss one potential cause and provide you with some possible workarounds.
The error messages displayed in the console are usually the typical 1612 (lost connectivity) or 1700 (during a refresh) and are only a symptom and not the underlying issue. You’ll need to open the VMManager log and look for the following errors which contain NullReferenceException:
Error 1/18/2009 3:21:27 PM Virtual Machine Manager 19999 None
Description: Virtual Machine Manager (vmmservice:6452) has encountered an error and needed to exit the process. Windows generated an error report with the following parameters:
Event:VMM20
P1(appName):vmmservice
P2(appVersion):2.0.3444.0
P3(assemblyName):E.Adhc.Operations
P4(assemblyVer):2.0.3444.0
P5(methodName):M.V.E.A.VirHostDatabaseUpdater.PostVirtualNetworkUpdate
P6(exceptionType):System.NullReferenceException
P7(callstackHash):d1e7
Error 1/18/2009 3:21:27 PM Virtual Machine Manager 1 None
Description: Virtual Machine Manager (vmmservice:6452) has encountered an error and needed to exit the process. Windows generated an error report with the following parameters:
Event:VMM20
P1(appName):vmmservice
P2(appVersion):2.0.3444.0
P3(assemblyName):E.Adhc.Operations
P4(assemblyVer):2.0.3444.0
P5(methodName):M.V.E.A.VirHostDatabaseUpdater.PostVirtualNetworkUpdate
P6(exceptionType):System.NullReferenceException
P7(callstackHash):d1e7
The cause is that Windows is reporting 2 NICs with the same name on the Host computer. This issue is a known bug and will be corrected in the next version.
The workaround options at this point can be:
a) Update drivers in the hope that that resets names
b) Avoid binding VNs to either of the duplicate NICs
c) Disable all but one of the duplicate NICs on the host
d) Disable host refresh and only do on-demand refresh (for all hosts but the affected one)
e) Don’t manage the host with VMM
Note: Other causes have been reported that if the Host has NIC teaming enabled or has any ghosted NICs…
So, now we know what the problem is but how do we resolve it? Here are the steps that I use when confronted with this issue. First, determine which Host is experiencing the problem. Either of the following methods will work.
1. Use the devcon utility which can be downloaded from KB311272. After installing devcon, open a command prompt and navigate to the directory where it’s located. Then run the following commands.
devcon listclass net
devcon findall =net
The output should be identical. If not, then any additional devices will have to be manually deleted from the Host.
2. Open a command prompt and type the following command:
systeminfo (if local)
systeminfo /s <machine_name> (if remote)
The output will display all NICs for a given machine.
Now the final step is to actually remove the ghosted NIC from the Host. This can often be tricky if the managed host computer is running the Core flavor of Windows Server 2008. If so, we'll have to use the DEVCON utility to remove the ghosted NIC. However if you're running the full version of Windows you might want to consider skipping straight to #3 below and simply resolving this from the UI instead of the command line.
1. From the output from the previous devcon commands, locate the ghosted NIC. The line will look similar to the following:
PCI\VEN_14E4&DEV_1677&SUBSYS_01A81028&REV_01\4&22443A69&0&00E5: Broadcom NetXtreme 57xx Gigabit Controller
2. Remove the ghosted NIC by typing the following command:
devcon remove @“PCI\VEN_14E4&DEV_1677&SUBSYS_01A81028&REV_01\4&22443A69&0&00E5”
Another alternative would be something like this:
devcon remove @“*DEV_1677&SUBSYS_01A81028&REV_01\4&22443A69&0&00E5”
3. An alternative solution to remove the ghosted device is as follows:
a. Click Start, click Run, type cmd.exe, and then press ENTER.
b. Type set devmgr_show_nonpresent_devices=1, and then press ENTER.
c. Type Start DEVMGMT.MSC, and then press ENTER.
d. Click View, and then click Show Hidden Devices.
e. Expand the Network Adapters tree.
f. Right-click the dimmed network adapter, and then click Uninstall.
For additional information on this process please refer to KB269155.