Azure Data Factory Web activity calling child pipeline across ADF instance

Pankaj Joshi 371

I have child pipeline in ADF Instance 1 which invoke rest api POST method through web activity

Parent pipeline is in ADF instance 2 which call above child pipeline also through web activity.

I am able to successfully call child pipeline from parent pipeline.

But I want to capture response(output) of child pipeline in Parent pipeline.

Please advise how it can done .

Vinodh247 28,211 Reputation points MVP

2025-02-19T16:33:02.69+00:00

In ADF, when a parent pipeline (Instance 2) invokes a child pipeline (Instance 1) via a Web Activity, you cannot directly capture the response of the child pipeline execution in the parent pipeline because ADF Web Activity does not wait for the child pipeline execution to complete -it triggers the execution and returns an HTTP response immediately.
Pankaj Joshi 371 Reputation points

2025-02-19T17:38:35.5933333+00:00

Is there any other way we can achieve that in ADF? Can be do it through using some wait activity or webhook
phemanth 13,900 Reputation points Microsoft Vendor

2025-02-20T19:51:46.9766667+00:00

@Pankaj Joshi

Thank you for posting your query!

Yes, you can achieve this using a Webhook activity in Azure Data Factory (ADF). The Webhook activity allows you to call an endpoint and wait for a callback, which can be useful for capturing the response of a child pipeline. Here’s how you can do it:

Configure the Child Pipeline:

In the child pipeline, use a Webhook activity to call an endpoint that can handle the callback. This endpoint should be able to process the data and return the necessary response.

Set Up the Webhook Activity:

In the Webhook activity, specify the URL of the endpoint and configure the request payload. Ensure that the endpoint can handle the callback URL provided by the Webhook activity1.

Wait for the Callback:

The Webhook activity will wait for the callback invocation before proceeding to the next activity. This allows you to capture the response from the child pipeline and use it in the parent pipeline1.

Use the Response in the Parent Pipeline:

Once the Webhook activity completes, you can use the output of the Webhook activity in subsequent activities within the parent pipeline. This allows you to process the response data as needed1.

Here’s a brief example:

Child Pipeline:

Add a Webhook activity and configure it to call an endpoint that can handle the callback. Ensure the endpoint processes the data and returns the necessary response.

Parent Pipeline:

Add a Webhook activity to call the child pipeline’s endpoint and wait for the callback. Use the output of the Webhook activity to access the response data.

This approach allows you to capture and use the response of the child pipeline in the parent pipeline effectively.

:Webhook activity - Azure Data Factory & Azure Synapse

Share via

Azure Data Factory Web activity calling child pipeline across ADF instance

Your answer