Hi @Zuuber
Welcome to Microsoft Q&A Platform. Thank you for reaching out & hope you are doing well.
I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to "Accept " the answer.
Issue: RDP via Azure VPN through Azure FW.
Solution: The FW network rule had the incorrect source IP, I needed to add the 'VPN IP address' as source into the firewall and not the GW subnet range.
Please don’t forget to close the thread by clicking "Accept answer" so that other community members facing the same issues can find the right answers.
Your Contribution is highly appreciated.
Thanks,
Sai.