How fast is fast? Virtual Machine Live Migration Improvements
When you’re doing a Live Migration** of a virtual machine between hyper-v hosts, you want it to go quickly. You may be doing the migration of one or several or dozens of virtual machines all at once, and the performance of the network and the network paths you choose are going to determine how quickly you can get the job done. Yes, sure, in one sense it doesn’t matter how long it takes if the VMs will continue to run and provide service during the migration. But if I’m doing, say, an automated update of all of the hosts in my cluster, and allowing it to drive the live migrations of machines among hosts, the speed with which those migrations complete will ultimately determine how long it takes to complete the updates of all of those hosts. If I’m really maxing out the capabilities of Hyper-V in Server 2012 R2 or Hyper-V Server 2012 R2, that could mean as many as 8,000 virtual machines moving around and among 64 clustered hypervisor nodes. So, speed is still important.
In the past, memory of a running virtual machine was just sent over the wire (TCP/IP) as it was. Nothing special was done to it. But as hardware costs have improved to support larger and larger scale, and as we’re afforded the ability to run more virtual machines with more and more memory, we certainly want to do everything we can to make that transfer of memory and configuration data go as quickly as possible. So to address this and improve things, we’ve added two new technologies to hyper-v in Windows Server 2012 R2 and Hyper-V Server 2012 R2:
- Live Migration Compression, and
- Live Migration via SMB Direct (RDMA)
Let’s talk about those, shall we?
Live Migration With Compression
Did you know that your hypervisor host isn’t typically suffering much when it comes to processor capacity?
“I didn’t know that.”
It’s true. So, what we’re going to is borrow some extra CPU cycles while we’re doing a live migration, and actually compress the migration data before it goes over the wire, and decompress at the destination.
If it sounds just that simple, well, it is. And it’s just a simple choice in the Live Migrations –> Advanced Features settings on your Hyper-V hosts:
And as if that wasn’t good enough…
Live Migration via SMB Direct (RDMA)
In Windows Server 2012 we introduced a new version of SMB – SMB 3. Among other things, this version of the protocol greatly improves performance; even to the extent that we can trust a basic file share to be the location for live data such as a virtual machine’s hard disks and data disks, or a SQL Server database. (Click here for a good summary of what SMB 3 provides.)
SMB Direct (SMB over Remote Direct Memory Access, or RDMA) is technology that, given hardware (the NICs) supporting it, can establish an efficient memory-to-memory transfer of data. In Server 2012 the main beneficiary of this was faster file services. But in R2 we’re using this to send live migration data between the Hyper-V hosts.
So now instead of just sending the memory and configuration of a VM over the wire using TCP/IP, or compressing it first, we’ll use a direct memory-to-memory channel.
Can you say “FAST”?
“Fast!”
I knew you could.
“But, can you give me an example? Can you show me how they compare?”
The best example I can give you is Jeff Woolsey’s demonstration he did for the TechEd 2013 North America keynote this past June.
Click this link to watch his demo (at 1:56:15) : TechEd 2013 North America Keynote Video – Jeff Woolsey’s Live Migration Demo
And for a more detailed description of Live Migration and the improvements made, check out this page: Virtual Machine Live Migration Overview
Questions? Comments? Make sure you add them to the comments at the bottom of this post! And try it out yourself by downloading the evaluations of either Windows Server 2012 R2 or Hyper-V Server 2012 R2 .
---
**That’s a ‘ vMotion ’ for those of you who are more familiar with the VMware terminology.
Comments
Anonymous
October 31, 2013
Two days of in depth talks on Hyper-V in San Francisco November 14-15 by the IT Pro community. More info at www.techdays.orgAnonymous
October 31, 2013
Thanks, Doug. Is Jennelle involved in this? I should hope so. :)Anonymous
October 31, 2013
The comment has been removedAnonymous
October 31, 2013
Fantastic! I wish I could join you guys there again. Say 'hi' for me. :)