Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause
In a physical only world, you don’t usually have to worry about MAC addresses that much as each NIC vendor carves off a MAC address from their ranges which have been allocated to them. However, in a virtual environment, you have to be a little more careful, particularly if you are using dynamic MAC address assignment. This post looks at how Hyper-V allocates dynamic MAC addresses and some potential problems you can face. So often it can be the last thing people think to check, but can be the root cause of otherwise unexplained network oddities.
Here’s a screenshot of a typical MAC collision problem – pings sometimes work, sometimes fail – and this is all on a local isolated network.
To start the walkthrough, I have a base install of Windows Server 2008 on a server with a single physical NIC – against best practice, but it serves fine for demonstration. I have already installed the RTM update (KB950050) to the server, but have not yet added the Hyper-V role. Let’s look at an output of “ipconfig /all”. You can see that the MAC address of the physical NIC is 00-13-20-F5-F8-7D and I’m obtaining an IP address from a DHCP server on the private test network I’m using.
Now let’s use Server Manager to enable the Hyper-V role. Note that Server Manager allows you to create an external virtual network switch during role enabling, but I am choosing not to do this. Let’s see what has happened in the registry after the Hyper-V role is enabled. Specifically, I’m looking at two keys which have been created under HKLM\Software\Microsoft\Windows\NT\CurrentVersion\Virtualization, as-yet unpopulated: MinimumMacAddress and MaximumMacAddress, plus another key in the worker node, CurrentMacAddress – again as-yet unpopulated. (The astute walking through this in front of a machine will notice that CurrentMacAddress also appears in the Virtualization node. That key is not used though.)
Next, I’m going to create my first virtual machine. As I haven’t created any virtual network switches yet, I’ll leave the network disconnected. I don’t need a hard disk. Also, I’m deliberately choosing not to start it. Let’s see what’s happened in the registry. MinimumMacAddress and MaximumMacAddress have been populated with 00-15-5d-c8-6a-00 and 00-15-5d-c8-6a-ff respectively – a range of 256 possible MAC addresses.
So where did this range come from? The first three bytes are the Microsoft IEEE Organizationally Unique Identifier, 00-15-5D which we use in Hyper-V. The next two bytes, C8-6A are derived from the lowest two octects of an IPv4 address on the server (the first IP address as NICs are enumerated). If you look at the second screenshot in this post, the IPv4 address on the only NIC on this server was 192.168.200.106. In Hex, this is “C0.A8.C8.6A”. The last two octets or bytes are C8 and 6A. The last byte of the address range is automatically generated with a minimum 00 and maximum FF.
You can probably now realize, that while this algorithm will work for many people, it may not necessarily be perfect and cause MAC address range clashes. To cope with multiple Hyper-V enabled servers, you would need to ensure address ranges are managed at a higher level across those servers, such as the use of SCVMM.
Let’s go back to the virtual machine I created. By default, when a virtual machine is created, it is allocated a dynamic MAC address. This can of course be changed in the settings for the virtual machine. Here’s the setting for the blank virtual machine. Notice that it’s set to Dynamic and the MAC address in the “Static” boxes show 00-00-00-00-00-00
Now I’m going to start the Virtual Machine and open the settings. Although some settings cannot be changed while a virtual machine is running (including changing static/dynamic MAC, or the static MAC itself), notice that the boxes under the static MAC address radio button are now populated with the first MAC address in the range defined in the registry: 00-15-5D-C8-6A-00.
Now for a bit of fun (and to make the walkthrough a bit simpler), let’s change the registry so that the maximum MAC address is 00-15-5D-C8-6A-02. (I’ll also do a reboot just to make sure the change takes effect) This change means that we are limited to three possible dynamically assigned MAC addresses, the last octet being 00 (in use by the “Blank” VM), 01 or 02.
Now, I’m going to create another virtual machine named 6A-01 and power it on, then create a third virtual machine named 6A-02 and power that on too. Let’s look at the settings for each of these while all three virtual machines are running. As expected 6A-01 has a MAC address ending 6A-01 and 6A-02 has a MAC address ending 6A-02. That’s why we have the “CurrentMacAddress” registry key to track what MAC address to assign to VMs in turn.
Can you guess though at this point what would happen though if I create another virtual machine and power it on? I don’t have any MAC addresses left in my available range and all MAC addresses are currently in use.
Did you guess correctly? Let’s now power off the very first virtual machine (“Blank”) I created with MAC address 6A-00, and then try to run through the New Virtual Machine Wizard again with my “No MAC Addresses Available In Range” virtual machine. Try to guess what will happen at the end.
The virtual machine starts successfully and now has a duplicate MAC address to the first virtual machine I created, ‘Blank’:
Last quiz question: What would happen then if I tried to start “Blank” – will it start or not? After all, it has already been allocated a MAC address ending 6A-00.
Actually, we will detect this as you can see above and stop the virtual machine from powering on. So in some ways, on a single Hyper-V enabled server, we’re relatively immune to duplicate MAC addresses across virtual machines running on a single server. However, due to the algorithm for choosing the ranges of MAC addresses, while relatively safe, there is no guarantee of being unique across an entire network. And of course, chances are that you will want packets from or to virtual machines on a Hyper-V server to “hit” the physical network.
So hopefully that gives you a better idea why it is important to manage MAC addresses across multiple servers in a virtual machine environment. While the walkthrough above was specific to Hyper-V, the same types of issues could arise in Virtual Server.
Cheers,
John.
Comments
Anonymous
January 01, 2003
Umair - it depends. If you have a single parent vNIC on an external switch, by default this inherits the MAC address of the physical adapter itself when the switch is created. VMNICs take their MAC address from the pool. So it's the physical for the parent vNIC, and virtual MAC addresses for guests which are presented to the physical network switch.Anonymous
January 01, 2003
Chris - yes, that would be correct. We don't have a sysprep provider implemented in Hyper-V v1 (2008), so you would have clashing ranges in each of the clones. Thanks, John.Anonymous
January 01, 2003
John Howard: Our Virtual Switch got smarter in Windows Server 2008 R2. In Windows Server 2008, VMs are susceptible to MAC spoofing. MAC spoofing is where a (generally) malicious machine pretends to be another machine on a network (there are legitimateAnonymous
January 01, 2003
Omar - no, the dynamic MAC address once allocated will not change, so if all VMs have unique MAC addresses, I wouldn't expect to see what you are seeing. Is it possible that there are other Hyper-V boxes elsewhere on the network which are conflicting? Thanks, John.Anonymous
January 01, 2003
Bryan - No, this is is not possible. Thanks, John.Anonymous
January 01, 2003
Polk - yes, good question and one which I didn't make clear above. I was using a second server running the Hyper-V role with a virtual machine on it configured with a duplicate MAC. Thanks, John.Anonymous
January 01, 2003
Sachin - take a look at the WMI documentation for msvm_virtualswitchmanagementservice http://msdn.microsoft.com/en-us/library/cc136938(VS.85).aspx. Specifically the CreateInternalEthernetPort* methods. Thanks, John.Anonymous
January 01, 2003
Our Virtual Switch got smarter in Windows Server 2008 R2. In Windows Server 2008, VMs are susceptibleAnonymous
January 01, 2003
Our Virtual Switch got smarter in Windows Server 2008 R2. In Windows Server 2008, VMs are susceptibleAnonymous
January 01, 2003
The comment has been removedAnonymous
January 01, 2003
Isambert - I would not expect two physical computers to have the same MAC address - very unlikely. I can think of one situation where it is possible due to a bug in Hyper-V for the virtual NIC in the parent partition to take on a MAC address from the internal pool MACs for VMs that we maintain. Can you confirm the MAC addresses which are clashing start 00-15-5D (the default range for our pool); that these are virtual NICs rather than physical nics in the parent (ipconfig /all will list the adapter as Microsoft Virtual Network Switch Adapter #n); the steps in Hyper-V Manager you went through to create the external virtual network switch (I suspect you created an internal virtual network, then changed it to external). Thanks, John.Anonymous
July 16, 2008
John, so if Hyper-V is immune to MAC duplication, how did you get the timeouts on MAC collision ping in the very first screenshot?Anonymous
July 16, 2008
The comment has been removedAnonymous
July 18, 2008
Here is any idea, MS design a product that doesn't have these limitations. You don't have to worry about MAC duplication in a VMWare ESX environment. This is just typical MS "features" that are only half baked, but the MS bigots can see no faults.Anonymous
July 25, 2008
The comment has been removedAnonymous
February 04, 2009
Thanks for the article John, I now understand what has been happening with the MAC address generation on my sysprep cloned hyper-v servers. It seems that these registry entries were just copied from the template system, the MAC addresses as a result were all based on the template systems original IP address and were starting to come out with the same MAC addresses on different clones. It looks like hyper-v doesn't currently check dynamically whether these max and min entries still match the current machine address.Anonymous
February 07, 2009
When connecting two physical machines with hyper-V for a demo (ie one network card shared for physical and virtual machine), ipconfig/all say that the two notebooks have the same MAC Address for physical computer. To be sure, we have made a reservation in DHCP. When the first boot, it takes the reserved IP address. When the second boot, it take the same ip address.... Do you know this problem ? How can we fix it ?Anonymous
March 18, 2009
I am experiencing this issue within our Hyper-V Cluster (4 servers) running about 25 VM's. When we had the network down for maint, the cluster shut down and saved the VM's state. When the network was restored, we experienced intermittent issues with access to VM's and even the Hyper-V Parent Partition. We have MAC management set to automatic. The VM's primarily use the same subnet. It appears that based on the above comments, we should be manually managing the MACs (BUMMER! )Anonymous
April 06, 2009
John- Thanks for the article. Also, can you let me know or point to some article which talk about where are these MAC address for VMs stored in WMI of Hyper V server.Anonymous
October 07, 2010
Hi John, I just set up 2 virtual machines on the Hyper-V host. The 2 NICs on the host are teamed to have better reliability and speed. The MAC address settings for the 2 VM are both dynamic MAC. Now this VMSMP event id 28 keeps filling up my host server's log every second. Any idea why? Here's the detailed log: Port '00E81214-A1C2-422E-A8BD-2F784E959359' was prevented from using MAC address '00-15-5D-01-EB-05' because it is pinned to port 'BC514DC5-757E-42D6-BCAD-0A1AB62739F1'. Why is it keeping trying to use that MAC when there are totally 255 MAC addresses available?Anonymous
June 30, 2011
@JasonL - This is far from ideal, but I ultimately solved the event id 28 problem by manually setting the mac address of the virtual network adapter. I changed the last digit so that it was not identical to the mac address of the physical network adapter.Anonymous
July 08, 2012
Great, Simply put a very beautiful and clear explanation.Anonymous
May 20, 2013
Great article. With a virtual switch, is it the physical or virtual MAC address that is presented to the physical network switch ?Anonymous
May 23, 2013
This doesn't seem to be followed when creating vHBA World-wide names (the fibre-channel equivalent of MAC addresses) - can you describe that process too ?Anonymous
January 20, 2014
I have a completely different issue. I have moved a harddisk of my Windows 8 with Hyper-V enabled from the old to a new computer in my company. And now, the old computer with a brand new harddisk can't start -- it registers a MAC conflict. Apparently my new computer stilll retained the same MAC it had in previous computer. It doesn't take the new MAC. How can that be, and how to fix it?Anonymous
February 10, 2014
I am facing and issue with cluster NLB after we had deployed to hard node into our Hyperv cluster .the cluster NLB composed of two virtual servers shows a mac address conflict with one the mac adress of one of the two new hard nodes .When we stop those two nodes there is no issue .Do you have any idea how to fix it ?Anonymous
April 25, 2014
Thank you ALOT for writing this article.
The fact that this article 6 YEARS old - and I experienced this -->BUG
But besides that, hyper-v rocks.Anonymous
April 25, 2014
Oh, forgot to mention; experienced this with two hosts with vNIC's - the physical hosts collided.Anonymous
August 19, 2014
The comment has been removedAnonymous
October 28, 2014
Hyper-V: MAC Address allocation and apparent network issues MAC collisions can cause - John Howard - Senior Program Manager in the Hyper-V team at Microsoft - Site Home - TechNet Blogs