Windows 10/11 VPN From RDP Session Kills Remote Desktop

Question

I'm hoping someone else out there has seen this and may have some insight regarding a fix.
In my environment, we have certain networks that only specific machines are permitted to VPN to. With so many people working from home, some WFH users will VPN to network 1 so that they can RDP to their onsite machine, then VPN from that onsite machine to network 2 to access resources available only on network 2.

With other AV products in place, this works no problem, but for whatever reason Defender takes issue with this and kills the user's RDP session as soon as that VPN connection is made from the machine they're RDP'd to. It disconnects that new VPN session on the onsite machine, kills their RDP session for a couple seconds, and then reestablishes the RDP session.

At one point I saw records of Defender's firewall blocking the VPN connections and figured maybe that was it. Kept tweaking rules without any luck until I finally just deleted all of the firewall rules and created one allow all inbound and one allow all outbound rule to at least confirm the firewall was the culprit. However, even that didn't resolve the issue... the ONLY way I've been able to get it to work is to disable all three firewall profiles. If I leave even one of the firewall profiles (Domain, Private, or Public) turned on, the issue comes right back.

Any help/ideas anyone could provide would be extremely helpful. Really trying hard to push a successful transition to Defender and this issue is making it really tough. ☹

Answer 1

Hi. Thank you for your question and reaching out. I’d be more than happy to help you with your query

It sounds like there could be a conflict between the VPN connection and the Microsoft Defender firewall. Here are a few things you can try to resolve the issue:

    Disable the Firewall temporarily: As you mentioned, disabling the firewall profiles seems to resolve the issue, so you could try disabling the firewall temporarily while the VPN connection is being established and then re-enable it once the connection has been established.

    Create a firewall rule for the VPN connection: Another option would be to create a firewall rule specifically for the VPN connection. You could do this by creating an inbound rule that allows traffic from the VPN connection and specifies the relevant ports and protocols used by the VPN connection.

    Disable Network Protection: If the issue persists, you could try disabling Network Protection temporarily. This feature is designed to protect against network-based threats, and it's possible that it's conflicting with the VPN connection.

    Disable SmartScreen: If the issue still persists, you could try disabling SmartScreen. This feature is designed to protect against malicious websites and downloads, and it's possible that it's conflicting with the VPN connection.

If the reply was helpful, please don’t forget to upvote or accept as answer, thank you.

Answer 2

Resolving RDP Unresponsiveness Over OpenVPN Due to MTU Issues

Note:  Issue usually occurs from home computers connecting OPENVPN, then using remote desktop client to connect to work Windows 11 computer.  System may appear unresponsive after setting credentials.

Issue is caused by packet fragmenting via bad MTU value on the OpenVPN computer

Users experience RDP (Remote Desktop Protocol) becoming unresponsive when connecting to a Windows 11 machine over OpenVPN, while it works fine on the local network.

Cause

The issue is caused by incorrect MTU (Maximum Transmission Unit) settings, leading to packet fragmentation, network instability, and RDP freezing or disconnections.

Solution: Determine and Set the Correct MTU

Step 1: Identify the Maximum MTU

Open Command Prompt as Administrator on the OpenVPN client machine.

Run a ping test to determine the highest MTU without fragmentation:

ping <Windows11_IP> -f -l 1472

If you get the message "Packet needs to be fragmented but DF set", lower the MTU in increments of 10 and test again:

   ping <Windows11_IP> -f -l 1460
ping <Windows11_IP> -f -l 1450

  Repeat until the ping succeeds **without fragmentation errors**.
  
  **Once a successful ping is found**, add **28 bytes** to the value to account for ICMP and IP headers.
  
     Example: If `ping -f -l 1390` works, the correct MTU is **1418**.
     

Step 2: Apply the Correct MTU to OpenVPN Client

Since OpenVPN uses a profile and server settings cannot be modified, you must configure MTU at the client level.

A. Set MTU for the VPN Adapter

Find the OpenVPN Adapter Name

Run:

   netsh interface ipv4 show interfaces

  Look for the **VPN adapter name** (e.g., "TAP-Windows Adapter V9").
  
  **Set the Correct MTU** (Replace `<Index>` with the interface index number found earlier):
  
  ```sql
  netsh interface ipv4 set subinterface <Index> mtu=1418 store=persistent
  ```
  
  Example:
  
  ```sql
  netsh interface ipv4 set subinterface 22 mtu=1418 store=persistent
  ```
  
  **Restart the VPN Adapter**
  
  ```yaml
  netsh interface set interface "<Adapter Name>" disable

netsh interface set interface "<Adapter Name>" enable ```

  Alternatively, disconnect and reconnect to OpenVPN.
  

B. Verify the MTU Setting

Run:

netsh interface ipv4 show interfaces  

Ensure the MTU for the VPN adapter matches the calculated MTU (e.g., 1418).

Step 3: Test RDP Stability

Reconnect to OpenVPN.

Run a new ping test to ensure packets are not fragmented:

ping <Windows11_IP> -f -l 1380

Try RDP Again:

mstsc /v:<Windows11_IP>

Step 4: Backup Registry and Force RDP to Use TCP (Disable UDP) [Optional]

If RDP still lags or disconnects, disable UDP mode to force TCP-only communication:

Backup the Registry Before Making Changes:

Open Command Prompt as Administrator and run:

   reg export "HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT\Terminal Services\Client" "%USERPROFILE%\Desktop\RDP_Backup.reg"

  This will create a backup file on the desktop that can be restored if needed.
  
  **On the OpenVPN Client Machine, Disable UDP:**
  
  ```json
  reg add "HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT\Terminal Services\Client" /v fClientDisableUDP /t REG_DWORD /d 1 /f
  ```
  
  **Restart RDP Service:**
  
  ```yaml
  net stop termservice && net start termservice
  ```
  
  **Reconnect and test RDP.**
  
  1. **Reboot the computer hosting OpenVPN (your home computer for example)**
  

Conclusion

 

By correctly identifying and setting the MTU, the OpenVPN tunnel becomes stable, eliminating RDP freezes and improving connectivity. These steps ensure packet fragmentation is avoided, leading to a smooth remote desktop experience over OpenVPN.

Resolving RDP Unresponsiveness Over OpenVPN Due to MTU Issues

Note:  Issue usually occurs from home computers connecting OPENVPN, then using remote desktop client to connect to work Windows 11 computer.  System may appear unresponsive after setting credentials.

Issue is caused by fragmenting MTU value on the OpenVPN computer

Issue

Users experience RDP (Remote Desktop Protocol) becoming unresponsive when connecting to a Windows 11 machine over OpenVPN, while it works fine on the local network.

Cause

The issue is caused by incorrect MTU (Maximum Transmission Unit) settings, leading to packet fragmentation, network instability, and RDP freezing or disconnections.

Solution: Determine and Set the Correct MTU

Step 1: Identify the Maximum MTU

Open Command Prompt as Administrator on the OpenVPN client machine.

Run a ping test to determine the highest MTU without fragmentation:

ping <Windows11_IP> -f -l 1472

If you get the message "Packet needs to be fragmented but DF set", lower the MTU in increments of 10 and test again:

   ping <Windows11_IP> -f -l 1460
ping <Windows11_IP> -f -l 1450

  Repeat until the ping succeeds **without fragmentation errors**.
  
  **Once a successful ping is found**, add **28 bytes** to the value to account for ICMP and IP headers.
  
     Example: If `ping -f -l 1390` works, the correct MTU is **1418**.
     

Step 2: Apply the Correct MTU to OpenVPN Client

Since OpenVPN uses a profile and server settings cannot be modified, you must configure MTU at the client level.

A. Set MTU for the VPN Adapter

Find the OpenVPN Adapter Name

Run:

   netsh interface ipv4 show interfaces

  Look for the **VPN adapter name** (e.g., "TAP-Windows Adapter V9").
  
  **Set the Correct MTU** (Replace `<Index>` with the interface index number found earlier):
  
  ```sql
  netsh interface ipv4 set subinterface <Index> mtu=1418 store=persistent
  ```
  
  Example:
  
  ```sql
  netsh interface ipv4 set subinterface 22 mtu=1418 store=persistent
  ```
  
  **Restart the VPN Adapter**
  
  ```yaml
  netsh interface set interface "<Adapter Name>" disable

netsh interface set interface "<Adapter Name>" enable ```

  Alternatively, disconnect and reconnect to OpenVPN.
  

B. Verify the MTU Setting

Run:

netsh interface ipv4 show interfaces  

Ensure the MTU for the VPN adapter matches the calculated MTU (e.g., 1418).

Step 3: Test RDP Stability

Reconnect to OpenVPN.

Run a new ping test to ensure packets are not fragmented:

ping <Windows11_IP> -f -l 1380

Try RDP Again:

mstsc /v:<Windows11_IP>

Step 4: Backup Registry and Force RDP to Use TCP (Disable UDP) [Optional]

If RDP still lags or disconnects, disable UDP mode to force TCP-only communication:

Backup the Registry Before Making Changes:

Open Command Prompt as Administrator and run:

   reg export "HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT\Terminal Services\Client" "%USERPROFILE%\Desktop\RDP_Backup.reg"

  This will create a backup file on the desktop that can be restored if needed.
  
  **On the OpenVPN Client Machine, Disable UDP:**
  
  ```json
  reg add "HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows NT\Terminal Services\Client" /v fClientDisableUDP /t REG_DWORD /d 1 /f
  ```
  
  **Restart RDP Service:**
  
  ```yaml
  net stop termservice && net start termservice
  ```
  
  **Reconnect and test RDP.**
  
  1. **Reboot the computer hosting OpenVPN (your home computer for example)**
  

Conclusion

 

By correctly identifying and setting the MTU, the OpenVPN tunnel becomes stable, eliminating RDP freezes and improving connectivity. These steps ensure packet fragmentation is avoided, leading to a smooth remote desktop experience over OpenVPN.