Got it, solved the Problem with this:
UPDATE:
I managed to successfully boot the VM with the graphics card by adding the following to the registery and typing [bcdedit /set hypervisoriommupolicy enable] in Powershell.
Registry key path (create it if not present): HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\HyperV
Registry values:
RequireSecureDeviceAssignment = 0 (REG_DWORD)
RequireSupportedDeviceAssignment = 0 (REG_DWORD)
Source: https://github.com/MicrosoftDocs/windowsserverdocs/issues/3083#issuecomment-578502515
However, I get a code 43 after installing the latest driver for the RX 590. I have never seen this happen on an AMD card. I tried installing an older driver and the Code 43 still persists.
https://www.reddit.com/r/HyperV/comments/elxkiv/hyper_v_dda_windows_server_2019/