Issue with logging MLflow run in Databricks - BAD_REQUEST: Unable to connect to the linked AzureML workspace

Question

I am encountering an issue while trying to log MLflow runs to an experiment and register a model in Azure Databricks. While executing the mlflow.start_run(), I continually receive a RestException: BAD_REQUEST error with the message "Unable to connect to the linked AzureML workspace. Check that the workspace exists."

This issue persists even though the same code operated without errors previously. The last successful run was on October 17, 2023. Despite attempts to troubleshoot the problem, the error persists, leading me to believe the issue may reside on the Azure server side rather than the client side. Here is a simplified version of the code I'm trying to execute:

with mlflow.start_run():
    mlflow.log_param("param1", 5)
    mlflow.log_metric("foo", 1)
    mlflow.log_metric("foo", 2)
    mlflow.log_metric("foo", 3)

After running the code, the error message is shown and I have checked that notebook experiment is although created in Databricks, there are no logs and runs in it. I have attempted to rectify the issue through various methods, including checking the network connectivity, verifying the Databricks workspace URL and experiment name, and ensuring that I have the necessary permissions to access the Databricks workspace and write to the experiment. I have also verified that the versions of Databricks and MLflow I'm using are compatible. However, none of these measures have resolved the problem.

Answer 1

@Joe I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others (Opens in new window or tab)", I'll repost your solution in case you'd like to accept the answer (Opens in new window or tab).

Issue:

• Encountering an issue while trying to log MLflow runs to an experiment and register a model in Azure Databricks. While executing the mlflow.start_run(), user continually receive a RestException: BAD_REQUEST error with the message "Unable to connect to the linked AzureML workspace. Check that the workspace exists."

Solution:

• Issue has been resolved by unlinking the Databricks workspace from AzureML. This ensures that the tracking_uri isn't linked to the AzureML workspace during MLflow runs.

If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

I hope this helps!

If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.

---

Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.

Answer 2

@KranthiPakala-MSFT Please let us know how to unlink the workspace? don't see an option to do so.