Hello @Lewis Zwart
You are right, HTTPS Only option set to "On" would automatically redirect HTTP requests to HTTPS
https://learn.microsoft.com/en-us/azure/app-service/configure-ssl-bindings#enforce-https
You should also enable HTTPS Only option in App Service - Custom Domains tab to enable HTTPS redirection for your domain
https://learn.microsoft.com/en-us/azure/app-service/app-service-web-tutorial-custom-domain