Tomb stoned or Stale Objects not removable in Virtual Machine Manager 2008

For users of System Center Virtual Machine Manager 2008 (or R2), you might put yourself in a position where you can’t seem to get rid of a virtual machine and every attempt fails.  I recently ran into this situation and got so frustrated that I actually reached out to some SCVMM product group folks for some assistance…

Turns out, giving server and database access to the developer was a bit trickier than just popping open the “hood” of SCVMM and taking a peak.  What I’m about to discuss isn’t supported yet is realistically a real situation that you might find yourself in so I share with that in mind.

NOTE: Editing the SQL Server Database, VirtualMachineDB, directly isn’t a recommended approach.

Scenario & Issue

You have, in some way or another, caused a virtual machine object in the SCVMM 2008 database to not get correctly deleted though the object doesn’t exist on the Hyper-V host.  You locate the server, right-click on the object, select Delete and the removal job is kicked off.  Within seconds, the job fails and you check the Jobs menu and you see something similar to the following:

Frustrated as I was, I check the object again in the console and validate that the delete option is no longer available but we now have the option to Repair.  You select repair but there are no options so you are stuck. 

In the description of the job that failed, you get something similar to the following:

This situation was caused as the object in question was deleted as a physical object from the library share.  However, the object couldn’t get restored to the library share but made no difference.

Thus, the object is neither capable of being deleted nor is it capable of getting put back into place.  The red X object in the console isn’t overly annoying (depending on who you are and your tolerance level) though if you are using Systems Center Operations Manager to monitor your VMM environment then the alerts will begin to get very annoying.  Moving forward…

Finding the Object in the Database

The next step has you accessing the database for the Virtual Machine Manager.  By default, this database is named VirtualManagerDB and is either a SQL Server Express database or an instance on one of your SQL Servers.

NOTE: These steps are potentially dangerous. It is highly recommended that you backup your Virtual Machine database prior to doing these steps. These steps could render your Virtual Machine Manager infrastructure in a state that is unrecoverable.

These steps outlined below will backup your Virtual Machine database:

1. Open the VMM Administrators Console
2. Click the Administration tab
3. In Administration menu, click the General tab
4. Click the Backup Virtual Machine Manager in the Action pane
5. Enter the share to save the Database backup

The steps outlined below will help you locate he object if you are in the state which we found ourselves in:

1. Login to the SQL Server housing the Virtual Machine DB using SQL Management Studio (user must be a member of the sysadmins group or have direct access to the DB VirtualManagerDB
2. Locate the table dbo.tbl_WLC_VObject
3. Right-click on the table and select Open Table
4. In the database results field, locate the object that is stale using the name of the Virtual Machine as reference
5. Delete the row that has the data of this object
6. Close the database results
7. Close SQL Management Studio
8. Open an Elevated Command prompt, and enter the following:

After the restart of the service, open the VMM Administrators console and verify that the object is no longer in the database.

Summary

In this post, I cheated and shared with you something that you shouldn’t do on a regular basis unless you absolutely know what you are doing.  I did it after realizing that I asked for help without first doing what comes as an instinct and that instinct was right.  In this post, you effectively can remove metadata that is stale and not desired from your Virtual Machine Manager database.

Enjoy!

-Chris