Using a private RSA key stored in key vault to ssh within python code

Question

I am confused about how to use my keys stored in key vault.

My scenario:

• I have an Azure function, which needs to ssh into a virtual machine
• I use Python's paramiko library to manage ssh access to this VM
• Basically, I need to mimic the operation in my python code: ssh -i my_key.pem user@azure.vm.address

In order to do this safely, I thought that I just upload my private key my_key.pem to the key vault, and then use it in my code. But this does not work.

Below is my python code:

def main(request: func.HttpRequest) -> func.HttpResponse:
    if request.method == 'GET':
        # for checking job status only
        key = get_private_key_pem(
            vault_url,
            key_name
        )
        ip_address = ...
        user = ...

        private_key = paramiko.RSAKey(file_obj=io.BytesIO(key))
        ssh_client = paramiko.SSHClient()
        ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
        ssh_client.connect(ip_address, username=user, pkey=private_key)

        stdin, stdout, stderr = ssh_client.exec_command('pwd')
        stdout = stdout.read().decode()
        stderr = stderr.read().decode()
        msg = "stdout={}; stderr={}".format(stdout, stderr)
        logging.info(msg)
        return func.HttpResponse(msg, status_code=200)

    else:
        return func.HttpResponse("Method not allowed", status_code=405)


def get_private_key_pem(vault_url, key_name):
    """
    Retrieve a secret from Azure Key Vault.
    """
    credential = DefaultAzureCredential()
    client = KeyClient(vault_url=vault_url, credential=credential)

    # Get the secret value
    key = client.get_key(key_name)

    private_key_pem = key.key.n

    return private_key_pem

I tried many ways but it seems that the object returned by

client.get_key(key_name)

is not quite what paramiko.RSAKey is expecting.

I can confirm that the key retrieved is an RSA key and it works if I just do 'ssh -i ...' in a shell.

Have I misunderstood how key vault works? Should I use secrets instead but isn't that less safe?

Thanks