Increased Network Performance using SR-IOV in Windows Server 2012
In this blog entry, Bob Combs from the Windows Networking team discusses a technology called Single Root I/O Virtualization (SR-IOV). SR-IOV is helpful when virtualizing demanding workloads that require high network performance and is one of the ways we deliver on the promise of being able to virtualize any workload. 
Note, however, that because this technology effectively bypasses the Hyper-V Switch, SR-IOV is not something you would use in all cases and for all workloads. Specifically, when building general-purpose and flexible clouds that expose a common resource pool of compute, storage and network resources and that run a range of different workloads, you would probably prefer to have the ability to enforce QoS policies, security policies, or network virtualization over the ability to provide high network performance to the virtual machines.
When used appropriately, however, SR-IOV is a key technology that will allow you to virtualize your most complicated and demanding workloads.
Yigal
The Holy Grail of performance with virtualization is for virtual machines to perform as if they were running on stand-alone physical machines. For networking this goal means reducing CPU utilization, reducing latency and jitter, and achieving native I/O throughput as if the VM was talking directly to the physical network adapter.
Single Root I/O Virtualization (SR-IOV) in Windows Server 2012 realizes this goal by enabling virtual machines to perform I/O directly to the physical network adapter, bypassing the root partition. SR-IOV is ideal for high I/O workloads that do not require port policies, QoS, or network virtualization enforced at the end host virtual switch. In Windows Server 2012, SR-IOV can be deployed in conjunction with key capabilities such as Live Migration. In addition, SR-IOV can be deployed on existing servers, because it is compatible with most current-generation 10 Gbps NICS, and several SR-IOV drivers come inbox with the operating system.
How SR-IOV works
The diagram below illustrates how SR-IOV allows virtual machines to directly address the physical NIC. Traditionally (the configuration on the left), data from the physical NIC traverses the Hyper-V switch and then is routed on VMBUS to the destination VM; traffic generated by the VM follows the opposite path, through VMBUS, Hyper-V switch, and physical NIC. In the SR-IOV case (the configuration on the right), data from the SR-IOV-capable NIC is transferred from the hardware directly to the VM.
Figure 1. Non-SR-IOV configuration on the left, versus a SR-IOV configuration on the right.
SR-IOV can be deployed without losing flexibility. For example, Live Migration is fully supported for SR-IOV. You can Live Migrate a VM using SR-IOV to another host that either does or does not support SR-IOV, and back again. The VM will use SR-IOV if it is available on the target host, and if SR-IOV is unavailable, it will use the traditional software network path.
Performance Benefits of SR-IOV
In our own testing with Windows Server 2012 Beta, we have seen the following performance improvements with SR-IOV:
- Lower CPU utilization (by up to 50%)
- Lower network latency (by up to 50%)
- Higher network throughput (by up to 30%)
These results can translate to supporting more VMs per host, delivering increased network bandwidth utilization on the host, and providing greater performance predictability to guest VMs. For example, even a medium load SQL Server can be difficult to virtualize because of the intense I/O load it generates. With its better I/O performance, SR-IOV can help SQL Server achieve the higher network performance it needs to run in a VM. At the same time, financial applications and interactive applications (such as video streaming or VDI) benefit from SR-IOV for its lower latency.
Deploying SR-IOV
Like other Windows Server 2012 features, SR-IOV can be managed using the Hyper-V Manager or PowerShell. SR-IOV deployment involves two steps: Creating an SR-IOV capable virtual switch and enabling SR-IOV in the guest VM. An SR-IOV capable virtual switch can be created using the following PowerShell cmdlet:
PS C:> New-VMSwitch “IovSwitch” –EnableIov $True –NetAdapterName “Ethernet 2”
Then, a Windows Server 2012 or Windows 8 x64 VM on that switch can be enabled for SR-IOV by setting the SR-IOV weight to 1 (a weight of 0 disables SR-IOV):
PS C:> Get-VM “VM1” | Set-VMNetworkAdapter –IovWeight 1
Deploying SR-IOV requires that the server and network adapter are both SR-IOV capable. Servers need to have BIOS support for SR-IOV; the Windows release notes provide more details on compatible servers. Network adapters need to have the necessary hardware and driver support. We have been working closely with the networking hardware community to implement and test SR-IOV, and, in fact, many existing NICs already support SR-IOV. Windows Server 2012 provides inbox SR-IOV drivers for
- Broadcom 10GbE 57712 controller
- Emulex 10GbE OneConnect controller
- Intel 10GbE X520, 1GbE 82576, and 1GbE I350 controllers
Other partners are working to bring SR-IOV capable products to Windows Server 2012. For example, Cisco’s UCS (Unified Computing System) with a Cisco Palo driver supports SR-IOV using the Cisco VM-FEX extension in the Hyper-V Extensible Switch. Look for other SR-IOV announcements soon.
Summary
In summary, SR-IOV is an exciting performance enhancement for Windows Server 2012 Hyper-V that will allow virtual machines to match “bare metal” network performance, while also reducing CPU load. Most high performance workloads will benefit from this feature. PowerShell and Server Manager provide a rich model for both local and remote management. Finally, SR-IOV is available on existing servers and network adapters. We are working closely with the networking hardware community to give you the flexibility and choice in building your systems for high performance. We look forward to hearing about your deployment experience!
Additional SR-IOV information can be found in John Howards’ blogs at https://blogs.technet.com/b/jhoward/
Bob Combs
Senior Program Manager, Windows Networking
Comments
Anonymous
January 02, 2013
Hi Bob, I just wonder if you can tell me any free test suit I can use to test CPU performance? I used NTTTCP to test SR-IOV, and found that the network through put is improved with SR-IOV enabled, but the average CPU usage (for both physical host and VMs) is not.Anonymous
January 04, 2013
Ntttcp is a reasonable tool to use; it is the utility that I used in my internal comparisons. I presume you are using 10GbE NICs for SR-IOV. It’s much harder to measure performance differences on a 1GbE NIC. Make sure that you are using the latest NIC driver. For example, I know the retail version of the Intel Niantic driver has better performance than the inbox driver that ships with Windows Server 2012. The latest driver can be downloaded from Windows Update of from the NIC manufacturer’s website. The reduction in CPU is diminished or hidden when there are additional cores averaged into the performance measurement. I found that results were best viewed when I reduced the number of available cores to 4. You can reduce the number of cores the machine uses by running msconfig.exe, selecting the “Boot” tab, click on “Advanced options”, and setting the “Number of processors” to a lower number. As I said, I set it to 4. I also used an isolated network to ensure I was only measuring machine to machine for each SR-IOV or non-SR-IOV run.Anonymous
March 04, 2013
A bit confused, if sr-iov bypasses det Virtual Switch and talks directly to The nic, why do we create a Virtual sr-iov enabled Switch?Anonymous
June 22, 2014
can i enable the SR-IVO in virtual switch via GUI?Anonymous
December 02, 2014
皆さん、こんにちは。 SQL Server/Microsoft Azure SQL Database サポートチーム です。
今回は、SQL Server 2008 R2 と SQL ServerAnonymous
January 13, 2015
Nice Blog! tnx