HyperV Failover Cluster Validation complains about RDMA

Question

Hi,

I've just upgraded our HyperV cluster with 3 new machines running Windows Server 2022.
We first ran Windows Server 2016 so I had to upgrade to 2019 first before I could move completely to 2022. Before each upgrade of the cluster I ran a validation just to see if everything was OK.

When I upgraded the cluster to 2022 the validation started to complain about my virtual NICs, something about RDMA which I've never heard before and it never complained about before either.

What it says is:

Unable to verify that all all adapters associated with SET Switch 'Converged-vSwitch01' on Node 'HYPERV-04.contoso.com' are of the same RDMA Technology type, some adapters are reporting 'Device Default' or not reporting data at all.

This is how my network is installed. I've made a SET team with three of my NICs and then I've created three separate vNICs used for Heartbeat, Live Migration and CSV.

New-VMSwitch -Name "Converged-vSwitch01" -NetAdapterName "Ethernet 1 VM Switch", "Ethernet 2 VM Switch", "Ethernet 3 VM Switch" -AllowManagementOS $false  
Add-VMNetworkAdapter -VMNetworkAdapterName Hyperv-04-Heartbeat01 -ManagementOS -SwitchName Converged-vSwitch01  
Add-VMNetworkAdapter -VMNetworkAdapterName Hyperv-04-LiveMig01 -ManagementOS -SwitchName Converged-vSwitch01  
Add-VMNetworkAdapter -VMNetworkAdapterName Hyperv-04-CSV01 -ManagementOS -SwitchName Converged-vSwitch01  

Here is a picture of the validation:

If I run Get-NetAdapterRdma I get the following:

It only shows the vNICs which makes me think that I need to create them with another flag or something?

Does anyone know how I can get rid of this?

The adapters are 1Gbit if that is important.

Answer 1

I have the same problem. Did you ever figure this out? I'm not using RDMA capable adapters, but it insists on testing them even though they don't report support for it. I've gone so far as to:

Disable-NetAdapterRDMA

Doing so does disable it per Get-NetAdapterRdma results. I then tried unbinding it which also did not help:

Get-NetAdapterBinding -IncludeHidden -AllBindings | where DisplayName -like "rdma"

Thoughts?

Answer 2

Hello @Patric F

You are getting this validation error because it seems some of your adapters doesn't support the RDMA technology.

Remote Direct Memory Access (RDMA) is an extension of the Direct Memory Access (DMA) technology, which is the ability to access host memory directly without CPU intervention.

In order to use RDMA, you need a network adapter that has RDMA capability (such as Mellanox's Connect-X family of adapters). The link layer protocol of the network can be either Ethernet or InfiniBand -- both can transfer the RDMA based applications.

Here is a thread as well that discusses the same issue and you can try out some troubleshooting steps from this and see if that helps you to sort the Issue.

Server 2022 RDMA Mellanox ConnectX-4
https://learn.microsoft.com/en-us/answers/questions/620793/server-2022-rdma-mellanox-connectx-4.html

Hope this resolves your Query!!

--
--If the reply is helpful, please Upvote and Accept it as an answer–

Answer 3

Additionally:

Get-NetAdapterAdvancedProperty | where displayname -like "rdma"

Name DisplayName DisplayValue RegistryKeyword RegistryValue
---- ----------- ------------ --------------- -------------
vEthernet (Public) Network Direct (RDMA) Disabled *NetworkDirect {0}
vEthernet (Private) Network Direct (RDMA) Disabled *NetworkDirect {0}