Windows Server 2012 File Server Tip: Make sure your network interfaces are RSS-capable
The new SMB Multichannel feature improves performance for network interfaces by using multiple TCP connections for a single network interface automatically. SMB will only do this if your network interface reports itself as RSS-capable, which means it can use Receive Side Scaling. You can check that with the Get-SmbServerNetworkInterface or the Get-SmbClientNetworkInterface PowerShell cmdlets. See below:
Without this capability being present on the NIC, using additional TCP connections is not helpful, so SMB will only use a single connection, as it did in previous versions. Virtually all server-class network interfaces should report themselves as RSS-capable these days, but some desktop-class NICs might still say they don’t support RSS. I’ve also seen a few cases where an old driver or a misconfiguration might lead even a 10GbE NIC to report itself as non-RSS-capable. Another possibility is that the NIC reports itself as RSS-capable but with only one queue, which effective means that using multiple connections will not help. Those NICs are treated by SMB as non-RSS capable, even if they are reported as RSS-capable by the networking stack. You can verify that using the Get-NetAdapterRSS cmdlet. See below:
In the example above, we have four adapters. Two RDMA adapters that are also RDMA capable, one 1Gbps adapter and a 100Mbps. One of the two RDMA adapters have no cable connected (see status showing as “Not Present”). The 100Mbps adapter is an old one that does not support RSS. The 1GbE adapter supports RSS, but only offers 1 queue, so SMB treats it as non-RSS.
SMB will always prefer RDMA-capable NICs, then RSS-capable NICs and then all other NICs. In general that works fine, but in certain configuration it can lead to unexpected behavior. For instance, SMB might prefer a 1GbE NIC over a 10GbE NIC if the 10GbE is reported as non-RSS and the 1GbE NIC reports RSS support. In this case, first confirm that the 10GbE has the latest drivers (you might want to check Windows Update or the manufacturer’s web site). Also, you want to make sure that the NIC configuration wasn’t changed to disable RSS. This can be done using PowerShell cmdlets (Enable-NetAdapterRss and Disable-NetAdapterRss) and is also commonly found under the advanced properties pages for the NIC. See below:
Getting the right type of NIC with the right RSS-capable driver is always your best choice. However, if you really can’t make your preferred NIC show up as RSS-capable, you might need to take some more drastic measures like configuring SMB Multichannel Constraints or disabling RSS on the other NICs.
For more information about SMB Multichannel, see The basics of SMB Multichannel, a feature of Windows Server 2012 and SMB 3.0.
Comments
- Anonymous
February 12, 2016
Is there a list of RDMA capable Nic's in the market? - Anonymous
March 03, 2016
This was very helpful, but on my NICs they showed they were "false" until they were plugged into a working connection. When plugged in their capabilities changed to show RSS is "True". I don't know if that is with every NIC, but I have Broadcom 5709 and an Intel i350 and they both behaved that same way. Also, some older network cards like the Intel PRO/1000 Series technically have RSS capability, but since Intel no longer offers updated drivers they are not RSS capable in Windows Server 2012. My servers are the same way, some drivers won't support RSS under Windows 2012 even if the card technically has that feature. - Anonymous
March 04, 2016
This is valuable information, thank you. For my nics they didn't show as RSS "True" until they were plugged in (Intel i350, Broadcom 5609, HP NC382i). As well the Max Processors Table is effected by the RSS Starting CPU number, if I choose any higher starting CPU than 0 it will reduce the Max Processors in PowerShell to 4, 5 or 6 if I have the RSS Number of CPU and Que set. So for example, if I choose the starting CPU at 6 in the driver settings, I only gives me 4 processors available. Strange.