How to run docker without root user in ML Batch Endpoint

Tran Hong Thu (DPS.VI.DTS) 40

I have a project to deploy on an Azure ML batch endpoint. I need to run code in Docker without a root user (in the Batch Endpoint job). In the Dockerfile, I have to create a non-root user and grant permissions for it in the system temp folder (/tmp). However, a permission error occurs at runtime when the service creates a JSON file in a subfolder of '/tmp'. I have searched for the error and tried to fix it, but it still does not work for me. Please help me detect the root cause and suggest a solution. Below are my Dockerfile and the error message: User's image

User's image

more info, with this dockerfile i successfully ran with the ML Online Endpoint.

Arko 335 Reputation points Microsoft External Staff

2025-03-03T05:05:54.06+00:00

Your dockeruser is a non root user and it does not have write permission on /home/dockeruser/floor. Try editing your CMD ["bash", "-c", "chown -R dockeruser:dockeruser /home/dockeruser/floor && chmod -R 777 /home/dockeruser/floor && bash"] or instead of switching to dockeruser in the Dockerfile, run as root and manually switch to dockeruser at execution.

CMD ["su", "-c", "python3 main.py", "dockeruser"]

Also need these details-
az ml job create --file job.yml --debug

Once inside, manually check permissions

ls -ld /home/dockeruser/floor

whoami

touch /home/dockeruser/floor/test.json

Try this and let me know the outcome.
JAYA SHANKAR G S 900 Reputation points Microsoft External Staff

2025-03-03T05:54:21.05+00:00

Hi Tran Hong Thu (DPS.VI.DTS),

The error clearly says does not have write permission on /home/dockeruser/floor. So, try giving permission with the command given by Arko in above comments. Also please add you batch scoring script and deployment configuration.
Tran Hong Thu (DPS.VI.DTS) 40 Reputation points

2025-03-03T07:37:56.18+00:00

Hi Arko, JAYA SHANKAR G S,

I have just tried the suggestion from Arko to change CMD ["bash", "-c", "chown -R dockeruser:dockeruser /home/dockeruser/floor && chmod -R 777 /home/dockeruser/floor && bash"], but i still have an error

the suggestion is to change CMD ["su", "-c", "python3 main.py", "dockeruser"], I have not tried it because file "python3 main.py" will be designated by batch endpoint when the job starts.

Accepted answer

JAYA SHANKAR G S 900 Reputation points Microsoft External Staff

2025-03-04T06:59:46.2666667+00:00

Hi @Tran Hong Thu (DPS.VI.DTS) ,

when you run the job, it might take different user because the Dockerfile runs only once when you are building the environment later when you invoke it executes your scoring script with default user. You try using os.environ["AZUREML_BI_OUTPUT_PATH"] as output path instead of using /tmp or /home . you can refer this sample one where writing predictions to output path.

Thank you
Please sign in to rate this answer.
JAYA SHANKAR G S 900 Reputation points Microsoft External Staff

2025-03-05T04:41:52.9733333+00:00

Hi @Tran Hong Thu (DPS.VI.DTS) ,

Did you get the chance to work on above given suggestion?

Do let me know if you any further query.

Thank you

Tran Hong Thu (DPS.VI.DTS) 40 Reputation points

2025-03-05T09:30:06.0533333+00:00

Hi JAYA SHANKAR G S,

I am temporarily accepting your suggestion, but I don't have the output for you yet because my company's IT team removed my test environment.

Thank you.

JAYA SHANKAR G S 900 Reputation points Microsoft External Staff

2025-03-05T09:33:02.4266667+00:00

Hi @Tran Hong Thu (DPS.VI.DTS) ,

Let me know if you have query when you done with testing above suggestions.

Thank you.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

1 additional answer

Tran Hong Thu (DPS.VI.DTS) 40 Reputation points

2025-03-03T07:44:07.5466667+00:00

Hi Arko, JAYA SHANKAR G S,

I have just tried the suggestion to change CMD ["bash", "-c", "chown -R dockeruser:dockeruser /home/dockeruser/floor && chmod -R 777 /home/dockeruser/floor && bash"], but i have still an error

the suggestion to change CMD ["su", "-c", "python3 main.py", "dockeruser"], i have not tried it because the Scoring script will be designated by the batch endpoint when a job starts
Please sign in to rate this answer.
JAYA SHANKAR G S 900 Reputation points Microsoft External Staff

2025-03-03T08:54:41.0866667+00:00

Hi Tran Hong Thu (DPS.VI.DTS),

Please add the scoring script, so that we can try to reproduce the issue from our end.

Tran Hong Thu (DPS.VI.DTS) 40 Reputation points

2025-03-03T09:23:02.2+00:00

Hi JAYA SHANKAR G S,

Due to my company's security policies, I can't share the full of my code with you. my code is so simple, I create a JSON file in the run function (in the same thread) as the code below:

def write_json(data, json_path): # Write data to json file with open(json_path, "w", encoding="utf-8") as json_file: json.dump( obj=data, fp=json_file, indent=4, ensure_ascii=False )

JAYA SHANKAR G S 900 Reputation points Microsoft External Staff

2025-03-03T09:32:04.86+00:00

@Tran Hong Thu (DPS.VI.DTS) ok and what is the json path is it /tmp/dockeruser/floor/train.json? If yes please give permission to your tmp folder like you given for /home/

Tran Hong Thu (DPS.VI.DTS) 40 Reputation points

2025-03-03T09:41:17.88+00:00

Hi JAYA SHANKAR G S, I have tried two folders, '/home/dockeruser' and '/tmp/dockeruser', with the same permissions granted in the Dockerfile, and both have the same error.

JAYA SHANKAR G S 900 Reputation points Microsoft External Staff

2025-03-04T04:40:20.5966667+00:00

Hi @Tran Hong Thu (DPS.VI.DTS) ,

The error occurs while building the environment or when you invoke the batch endpoint? also try running as root user, if it works then it's the permission issue.

Thank you

Tran Hong Thu (DPS.VI.DTS) 40 Reputation points

2025-03-04T06:19:14.25+00:00

Hi JAYA SHANKAR G S, thank you for your feedback,

The error occurred when the batch endpoint was invoked. It ran successfully with the root user, but I need to fix Sonar's security issue, as it must run with a non-root user.

I know it's a permission issue and I'm trying to resolve it by creating a temp folder and granting permission for a non-root user in Dockerfile, However, I don't understand why it runs successfully in the ML Online Endpoint but fails in the Batch Endpoint job.

Tran Hong Thu (DPS.VI.DTS) 40 Reputation points

2025-03-04T07:46:36.1966667+00:00

Thank JAYA SHANKAR G S, Thank you! Let me try it based on your suggestion.

JAYA SHANKAR G S 900 Reputation points Microsoft External Staff

2025-03-04T09:08:41.9666667+00:00

Hi @Tran Hong Thu (DPS.VI.DTS) ,

Did you try using os.environ["AZUREML_BI_OUTPUT_PATH"] as output path, please let me know if you have any query.

Thank you

Tran Hong Thu (DPS.VI.DTS) 40 Reputation points

2025-03-04T09:12:59.15+00:00

Hi JAYA SHANKAR G S,

I can't try it now, so I'll send you the output later.

Thank you so much.

JAYA SHANKAR G S 900 Reputation points Microsoft External Staff

2025-03-04T09:18:31.8466667+00:00

@Tran Hong Thu (DPS.VI.DTS) ok waiting for your output, when you write to os.environ["AZUREML_BI_OUTPUT_PATH"] you will get an output folder after job completion where you can download it to local path. So it's recommended to use the above path only.

Thank you
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

How to run docker without root user in ML Batch Endpoint

1 additional answer

Your answer