Parameterizing Azure ML linked service in ADF

Adhisivan Ragunathan 40

I am trying to dynamically configure ML Execute activity in ADF. It allowed me to create dynamic content with parameters for ML linked service for subscription ID, workspace name etc. However when integrating with ADF ML Execute activity, I don't see any option to pass on these parameters. Generally I get to see linked service parameters showing up under settings section, in this case it is not. Is it something not supported and only works with hardcode configuration or Am I missing something?

Chandra Boorla 9,120 Reputation points Microsoft Vendor

2025-02-13T07:29:49.2066667+00:00

Hi @Adhisivan Ragunathan

Greetings & Welcome to Microsoft Q&A forum! Thanks for posting your query!

In Azure Data Factory, when you are working with linked services, you can often parameterize certain properties to make your pipelines more dynamic and reusable. However, the behavior can vary depending on the type of linked service and the activities you are using.

Key Points

Current Limitation - The ML Execute activity in ADF does not support dynamic parameter passing for linked service parameters at runtime. This means that you cannot pass dynamic values for parameters like subscription ID or workspace name directly through the ML Execute activity's settings.

Hardcoded Configuration - As a result of this limitation, you are effectively required to use hardcoded values for the linked service parameters when setting up the ML Execute activity. This restricts the flexibility you might expect from a fully parameterized and dynamic setup.

Not Missing Anything - Given the current state of ADF's integration with Azure ML, you are not missing any hidden configuration or feature. This behavior is consistent with the current design and limitations of the platform.

If dynamic parameterization is crucial for your workflow, consider the following alternatives:

Use a Web Activity to call Azure ML REST API - You can construct the API request dynamically using pipeline parameters to trigger an ML pipeline or model.

Use an Azure Function or Logic App - If you need more flexibility, you can create an Azure Function that takes parameters from ADF and triggers the ML pipeline.

The best approach depends on your flexibility and whether you need full control over ML execution.

Appreciate if you could share the feedback on our feedback channel. Which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.

I hope this information helps. Please do let us know if you have any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Chandra Boorla 9,120 Reputation points Microsoft Vendor

2025-02-14T07:29:41.48+00:00

@Adhisivan Ragunathan

Just checking in to see if the below suggestion helped. If this answers your query, do click Accept Answer and Yes for was this answer helpful. This will help other community members who are experiencing similar issues. If you have any further questions, please let us know.
Chandra Boorla 9,120 Reputation points Microsoft Vendor

2025-02-17T01:13:31.39+00:00

@Adhisivan Ragunathan

Following up to see if the above suggestion was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Adhisivan Ragunathan 40 Reputation points

2025-02-19T05:45:22.37+00:00

Hi Chandra,

Thank you for your response and for highlighting the current limitations of ADF's dynamic configuration for ML activities. It would be great to see this feature included as a standard capability in the future.

I understand that this cannot be achieved through the UI settings. However, is there a way to modify the ADF YAML/JSON configurations at the code level to parameterize the ML activity configuration?

Regards
Chandra Boorla 9,120 Reputation points Microsoft Vendor

2025-02-19T07:13:20.6133333+00:00

@Adhisivan Ragunathan

is there a way to modify the ADF YAML/JSON configurations at the code level to parameterize the ML activity configuration?

Currently, modifying the ADF JSON/YAML configurations at the code level will not allow you to dynamically parameterize the ML Execute activity’s linked service properties (such as subscription ID and workspace name). The ML Execute activity requires a predefined linked service, and ADF does not support dynamically changing linked service parameters at runtime, even through JSON modifications.

I hope this information helps.

Thank you.

Accepted answer

Chandra Boorla 9,120 Reputation points Microsoft Vendor

2025-02-27T14:00:46.82+00:00

@Adhisivan Ragunathan

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer.

Issue:

I am trying to dynamically configure ML Execute activity in ADF. It allowed me to create dynamic content with parameters for ML linked service for subscription ID, workspace name etc. However when integrating with ADF ML Execute activity, I don't see any option to pass on these parameters. Generally I get to see linked service parameters showing up under settings section, in this case it is not. Is it something not supported and only works with hardcode configuration or Am I missing something?

Solution:

"The issue has been resolved, by handling it by overwriting the ADF ARM template parameters in the Azure DevOps deployment pipeline, allowing for dynamic configuration of the ML Execute activity."

If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

Hope this helps. Do let us know if you have any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Please sign in to rate this answer.

0 comments No comments
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

1 additional answer

Adhisivan Ragunathan 40 Reputation points

2025-02-27T08:55:56.47+00:00

We solved this problem by overwriting the ADF ARM template parameters through Azure DevOps Deployment pipeline. Thank you all for the prompt response.
Please sign in to rate this answer.
Chandra Boorla 9,120 Reputation points Microsoft Vendor

2025-02-27T13:56:23.4466667+00:00

@Adhisivan Ragunathan

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Parameterizing Azure ML linked service in ADF

1 additional answer

Your answer