@Sébastien Hernandez Thanks for reaching out, the error message, "DR config cannot be deleted because replication is in progress," indicates that there is an ongoing replication process that needs to be resolved before you can delete the Disaster Recovery (DR) configuration.
do you have DR configured in your premium servicebus namespace? If yes you need to either failover or break the pairing of the DR configuration. This can be done through the Azure portal.
Once the pairing is broken, you should be able to delete the Service Bus namespace without encountering the 409 error.
I have sent you a private message asking for a few details about the issue. Please check and reply once you get a chance.