Share via

Understanding HighMemoryMappedIoSpace Allocation in Hyper-V with NVIDIA GPUs Using DDA

Anonymous
2025-01-29T19:52:23+00:00

Hi Everyone,

I'm currently managing a Hyper-V environment and I am using the NVIDIA GPUs with Hyper-V VM using the Discrete Device Assignment (DDA) method.

I need some guidance on a memory allocation issue I've encountered. Specifically, I'm trying to understand the practical implications of following command 

`Set-VM -HighMemoryMappedIoSpace  -VMName `

Example : 

`Set-VM -HighMemoryMappedIoSpace 128 -VMName Ubuntu24.04`

Here are the details:

  • I allocated 128GB of HighMemoryMappedIoSpace to a virtual machine.
  • The host machine only has 64GB of physical RAM available.

My Current Understanding:

Memory allocation for HighMemoryMappedIoSpace in Hyper-V involves setting aside a specific amount of memory on the host system to be mapped for I/O operations, which can be used by the virtual machine for accessing devices like GPUs. The allocated memory is taken from the host's total available RAM.

- On Allocation

When we allocate 128Gb of memory mapped I/O space, this amount is reserved from the total RAM available on the host system. This can impact the overall available memory for the physical host and other VMs running on that host.

- Availability:

The allocated memory is available to the virtual machine in the sense that it's mapped for its exclusive use for I/O operations. The VM can access this memory-mapped I/O space to efficiently handle I/O transactions.

- Relation to Windows Host:

The Windows Hyper-V host acts as a hypervisor that manages physical resources and assigns them to virtual machines. By allocating HighMemoryMappedIoSpace, you're instructing Hyper-V to set aside a portion of the host's physical memory specifically for memory-mapped I/O operations for the VM you specified.

Clarification Needed:

Given this configuration:

  1. How does Hyper-V handle the overcommitment of memory in this scenario?
  2. What impact does this have on the performance of the host machine and VM?
  3. Are there any best practices for managing memory overcommitment to avoid potential issues?
  4. Does setting the HighMemoryMappedIoSpace affect the overall system stability and, if so, how can I mitigate these risks?

Additional Context:

I've set 128GB of HighMemoryMappedIoSpace because of the requirements for the NVIDIA L4 GPUs. But on same enviornment NVIDIA RTX series GPUs generally need between 16GB to 32GB allocation Only. However, for an NVIDIA L4 single GPU, 45GB is required, and for two GPUs, more than 100GB is needed. To accommodate these needs, I have set 128GB of HighMemoryMappedIoSpace.

I appreciate any insights, advice, or experiences you can share.

Thank you in advance for your help!

Windows Server High availability Virtualization and Hyper-V

Locked Question. This question was migrated from the Microsoft Support Community. You can vote on whether it's helpful, but you can't add comments or replies or follow the question. To protect privacy, user profiles for migrated questions are anonymized.

0 comments

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. Anonymous
    2025-01-30T13:24:36+00:00

    Hello,

    The easiest way to determine how much MMIO space to allocate is to use the Machine Profile Script. Due to the nature of how Discrete Device Assignment is implemented, some features of a VM are restricted while a device is attached. The following features aren't available:

    • VM Save/Restore
    • Live migration of a VM
    • The use of dynamic memory
    • Adding the VM to a high availability (HA) cluster

    Plan for deploying devices by using Discrete Device Assignment | Microsoft Learn

    Best Regards

    ZUnhui

    0 comments