Azure Machine Learning - SSH Error When Attaching Virtual Machine Compute

chistian.b 0

Hello everyone,

I'm experiencing an issue with the "Attach Virtual Machine Compute" function in Azure Machine Learning. I created a virtual machine and used "User Authentication" as the authentication type. The authentication worked fine, and the machine appeared under compute resources. However, when I tried to run a simple job (Execute Python Script), I received an "Unable to establish SSH connection" error.

Run a job on a VM (attached with password authentication) I attempted to connect manually via SSH, and everything seemed to work fine. Then, I tried attaching the machine again using SSH key authentication, but this time, I couldn't attach the machine and I got this error.

failed_ssh_vm02 Any suggestions on how to resolve this issue would be greatly appreciated. Thank you!

Regards,

Chris

kothapally Snigdha 490 Reputation points Microsoft Vendor

2024-11-12T23:21:59.3066667+00:00

Hi chistian.b

Thanks for Reaching the Microsoft Q&A Forum

Sometimes, resetting the SSH configuration can resolve connection issues. You can do this through the Azure portal or using the Azure CLI.

When you switch to using SSH key authentication, make sure the key pair is configured correctly on both Azure and the VM.

Ensure that public key is correctly placed in the ~/.ssh/authorized keys file on the VM for the user you're trying to authenticate as.

Ensure the private key you're using matches the public key stored in the VM and is specified correctly when attaching the compute resource in AML. When switching authentication methods, make sure that the Azure Machine Learning workspace has the correct permissions to interact with the VM.

Ensure that the Azure ML Compute resource is configured with the correct service principal or user permissions. Sometimes, access permissions might be misaligned when switching from user authentication to SSH key authentication.

Sometimes, simply restarting or redeploying the VM can resolve connection issues.

Thank you!
chistian.b 0 Reputation points

2024-11-13T14:10:39.1633333+00:00

Hi kothapally,

thank you very much for the reply. I checked the SSH configuration, and it seems fine. I also deployed a new machine from scratch. For now, I'm using password authentication because, at least with this, I can connect the machine to Azure ML. I have some questions about permissions. I assigned the RBAC Contributor role for the Workspace to the VM, is this correct? Does the user on the Ubuntu machine need any special permissions? Currently, it has passwordless sudo access.

Thank you

Chris.
kothapally Snigdha 490 Reputation points Microsoft Vendor

2024-11-13T20:35:45.8866667+00:00

Hi chistian.b

Assigning the RBAC Contributor role to the VM for the Azure ML Workspace is generally correct, as this role allows the VM to manage resources within that workspa

The user on your Ubuntu machine does not require any special permissions beyond what you have already set up. Since it has pass wordless Sudo access, this should be sufficient for most administrative tasks. Just ensure that the user has access to any necessary files or directories required for your Azure ML operations.

Thank you!
vishnu 0 Reputation points

2024-11-14T09:52:28.79+00:00

Hi, I had the same issue
With password authentication, I'm able to attach VM.

But getting "Unable to establish SSH connection" while executing the pipeline.
kothapally Snigdha 490 Reputation points Microsoft Vendor

2024-11-14T19:37:12.5966667+00:00

Hi chistian.b

Following up to see if the above response was helpful.
kothapally Snigdha 490 Reputation points Microsoft Vendor

2024-11-15T22:27:43.2433333+00:00

Hi chistian.b

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution, please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
chistian.b 0 Reputation points

2024-11-16T03:51:05.3833333+00:00

Hi @kothapally Snigdha

Apologies for the delay, I wanted to try a few things before reaching out. Unfortunately, I wasn’t able to solve the issue. I experimented with RBAC settings to see if the problem might be related, and I also attempted to create a new workspace from scratch, but neither approach worked.

What's interesting is that I can successfully attach a new VM using a password, and when I tested with incorrect credentials, I received an error, confirming that Azure ML is indeed attempting to connect. This suggests that Azure ML can reach the VM, but the issue arises specifically when running a job. This may be worth investigating further, as it appears others are experiencing the same issue.

Regards,

Chris
kothapally Snigdha 490 Reputation points Microsoft Vendor

2024-11-20T22:29:06.82+00:00

Hi chistian.b

Ensure that your VM is in a running state and that all necessary services are up and running. Verify that the VM's configuration matches the requirements for running Azure ML jobs.

The NSG rules associated with your VM to ensure that they allow the necessary inbound and outbound traffic for Azure ML jobs. Make sure that ports required for job execution are open.

Since you mentioned that you can attach a VM using a password, ensure that the SSH configuration is correct. Verify that the SSH service is running on the VM and that the SSH keys are properly configured if you're using key-based authentication.

Review the Role-Based Access Control (RBAC) settings to ensure that the Azure ML workspace and the VM have the appropriate permissions to interact with each other. Sometimes, misaligned permissions can cause issues during job execution.

Check the logs and diagnostic information for both the VM and the Azure ML workspace. Look for any error messages or warnings that might provide more insight into the issue.

Thank you!
chistian.b 0 Reputation points

2024-11-26T01:19:59.7633333+00:00

Hi @kothapally Snigdha

I apologize for the delay in my response; I was too busy last week.

I have a question regarding the statement, "Verify that the VM's configuration matches the requirements for running Azure ML jobs." Do I need to install any services on the VM? I couldn't find much information in the Microsoft documentation; it seems quite limited.

I think checking the logs on the VM is a great idea. I will definitely take a look at them. As for the other aspects like ports, RBAC, and SSH, everything is in order.

Thanks.

Share via

Azure Machine Learning - SSH Error When Attaching Virtual Machine Compute

Your answer