SCVMM: Delete the duplicate vHBA and vPorts
How Do I Know If I Am Experiencing This Issue?
First, your VM must have vHBAs configured in it. If it doesn’t have them, this document is not related to your issue.
Second, from the Failover Cluster Manager (FCM), when attempting to power on a VM it may go into a failed/error state. If you are attempting to Live Migrate a VM, it will run through a full migration of the VM’s RAM data, up to the point of cutting over to the new host, and then the Live Migration will fail, and the VM will go back to the original host, after which the VM may remain online on the original host, or it could enter into a failed/error state and go offline.
Right-click a VM role in FCM, and hit ‘Show Critical Events’ to view the errors logged when attempting to power on the VM. You should see errors listed similar to the following:
Cluster resource 'SCVMM XXXXXXXXXX' of type 'Virtual Machine' in clustered role 'SCVMM XXXXXXXXXX Resources' failed. The error code was '0x5aa' ('Insufficient system resources exist to complete the requested service.').
Based on the failure policies for the resource and role, the cluster service may try to bring the resource online on this node or move the group to another node of the cluster and then restart it. Check the resource and group state using Failover Cluster Manager or the Get-ClusterResource Windows PowerShell cmdlet.
'SCVMM XXXXXXXXXX' failed to start.
'XXXXXXXXXX' failed to start. (Virtual machine ID 0A63A9D8-5F0A-4EF5-BA77-A34E2E89F235)
'XXXXXXXXXX' Synthetic FibreChannel Port: Failed to start reserving resources with Error 'Insufficient system resources exist to complete the requested service.' (0x800705AA). (Virtual machine ID 0A63A9D8-5F0A-4EF5-BA77-A34E2E89F235)
'XXXXXXXXXX': Virtual port (C003FF6E7C7D0054) creation failed with a NPIV error(Virtual machine ID 0A63A9D8-5F0A-4EF5-BA77-A34E2E89F235).
'XXXXXXXXXX': NPIV virtual port operation on virtual port (C003FF6E7C7D0054) failed with an error: The world wide port name already exists on the fabric. (Virtual machine ID 0A63A9D8-5F0A-4EF5-BA77-A34E2E89F235)
When issue comes up, first check using the ping test and open SCVMM to check the status of VM. If it is in stopped state and when we try to start the VM then below error will appears.
Resolving the Error
At a high level, what we need to do is manually delete the old vPorts for this VM from any host that might still have them active. There are different steps, depending on the HBA card installed, as each manufacturer has a different toolset to be able to interface to the physical HBA and delete the vPorts.
A quick note on what to delete – if your VM is currently down/offline, then it is safe to delete any and all vPorts with WWPNs belonging to your VM. However, if your VM is online on a host, but just not able to successfully live migrate to another host, you need to be careful as to which vPorts you delete – you don’t what to delete the WWPNs that the VM is currently using while online, as it will cut off all SAN storage to the VM. Thus, you will want to look for leftover vPorts on Hosts that your VM is NOT currently living on, and delete those. If in doubt, or you are unable to be sure which ones you want to delete, get emergency downtime for the VM, shut it off, and then you will be safe to delete any/all vPorts belonging to your VM.
Gather the WWPNs for your VM
- Log into the host where your VM lives.
- Open Failover Cluster Manager (FCM) and find your VM in the ‘Roles’ section of FCM
- Right-Click your VM and go to ‘Settings’
- Make note of the 2x WWPN addresses for EACH of the vHBAs on the VM. Each VM will have 2x vHBAs with 2x WWPN addresses each, so you should have a total of 4x WWPNs that belong to the VM:
We can find out WWNN and WWPN here also for future reference.
Deleting vPorts
The next steps to follow will depend on if the host cluster is utilizing Emulex or Brocade HBAs. Click the appropriate link for your situation and continue on.
vPort Cleanup on Servers with Brocade HBAs
- Log into the host machine where your VM lives.
- Open the PowerShell console with Administrative privileges (right-click ‘Run as administrator’)
- Run set-executionpolicy unrestricted and choose yes at the prompt
(No Need to execute this commend it already changed. Some time we are unable to execute the Vports.ps1 script due to some exceptional policy error that time we need to execute this commend again in which host you are unable to execute the script)
- Change your local directory to “C:\Program Files\Brocade\Adapter\Driver\util” and run vports.ps1 from that directory
- Press ‘4’ to delete ports
- The script will spit out a list of all of the vPorts it sees on the card currently, broken up by each physical host HBA (each host will have 2x HBAs). Look through the list for any WWPNs belonging to your VM, which you noted above. Type in the WWPN address you wish to delete, and hit ENTER.
- The virtual port will be deleted after pressing ENTER. Press ENTER 2 more times to return to the script menu.
- We will repeat these steps to delete the virtual port on the second HBA on this server as well. Note that the one we deleted previously is now no longer listed, as well:
- Log into the other hosts in the cluster, and perform the same steps, to verify that there are no virtual ports left over on those hosts for this same VM – any of the 4 addresses for the VM could be left over on any of the hosts, so it’s important to clean them all up to prevent a future migration issue.
- Repeat steps 6-10 for any additional VMs/virtual ports that need to be deleted.
For vPort cleanup on HyperV server(s) with Emulex HBAs follow below steps:
- Start OneCommand Manager from Programs (shortcut = OCManager) or use the executable path: "C:\Program Files\Emulex\Util\OCManager\OCManager.exe"
- In OneCommand Manager Console, select "View" then select "Group Adapters by Virtual Port"
- The console will now display Virtual Port WWPNs for the VMs under the physical port name like Port 0, Port 1. Select the exact WWPN that was reported as duplicate (already exists) as mentioned in the issue description above.
- On the right pane of the console select "Remove Virtual Port"
- It will show you a warning message - read it carefully, select the check box for "It is OK to delete the virtual port." and click OK. This will delete the Virtual Port that was conflicting while the HyperV tried to start the VM.
- Now try to power on the VM and it will successfully boot IA.
(Note: Make sure you delete the correct WWPN else it will impact other running VM and may result in data loss.)