Starting runbook with webhook through data factory

Lucia Zatkova (NZ) 0

Hi,

I've recently created a pipeline in ADF that is using webhook to start a runbook in the Azure automation. When I run the webhook in ADF it stays in progress until it times out (even though when I look at the status of the runbook it clearly was triggered and then completed.) I can see the expected outputs of the runbook as well.

So the runbook gets triggered and completed successfully however the webhook activity just stays in progress until it times out.

The runbook is just a python script that calls API and saved json response in the blob storage.

Any ideas why the webhook activity remains in progress until it times out?

Harishga 5,995 Reputation points Microsoft Vendor

2024-03-13T09:36:26.9133333+00:00

Hi @Lucia Zaťková

Welcome to Microsoft Q&A platform and thanks for posting your question here.

When the runbook triggered by the webhook completes successfully, but the webhook activity in the Azure Data Factory pipeline times out instead of completing, it could be due to several reasons.

One of the reasons is the default timeout value for a webhook activity being 7 days. To resolve this, the timeout value for the webhook activity can be increased by setting the "timeout" property in the webhook activity JSON definition.

Another possible reason for the issue could be network-related, which may prevent the webhook activity from receiving the response from the webhook URL. To check for any errors or warnings related to the runbook execution, review the Azure Automation job logs.

If the webhook URL requires authentication, it is necessary to provide the necessary credentials in the webhook activity definition. It's important to verify that the webhook activity is configured correctly by checking the correctness and accessibility of the webhook URL, the HTTP method used in the webhook activity, and the formatting of the headers and body of the webhook activity. It's also important to verify that the webhook URL returns a response with the correct status code (e.g., 200 OK) and body.

By verifying these details, the webhook activity can be correctly configured, and any issues with the activity can be resolved. If there are any further questions, please let me know.

Reference
https://learn.microsoft.com/en-us/azure/data-factory/control-flow-webhook-activity

I hope this information helps you. Let me know if you have any further questions or concerns.
Harishga 5,995 Reputation points Microsoft Vendor

2024-03-14T08:31:52.6533333+00:00

@Lucia Zaťková
We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution, please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Harishga 5,995 Reputation points Microsoft Vendor

2024-03-15T12:22:35.4066667+00:00

Hi @Lucia Zaťková
If the webhook URL is returning a 202-success message and the runbook is completing successfully in Azure, the issue may be related to the webhook activity configuration in Azure Data Factory. You can check the webhook activity timeout value and retry policy in the pipeline definition, adjust them if necessary, and enable diagnostic logging to get more information about the issue. Checking the Azure Data Factory pipeline run logs can also help identify any errors or warnings related to the webhook activity.
Davis Pfaff 0 Reputation points

2025-01-20T15:54:47.7633333+00:00

@Harishga I have run into this problem as well. The jobs run successfully when looking at through the Automation Account Runbook side. It just never sends something back to ADF to say it was successful. Always times out.

Currently I have: URL

Method: Post

Headers: Content-Type - application/json

Body: { "key": "value" }

Timeout: 2:00

The job is successful around 1:10 of the run. So I think that is plenty of time for it to turn successful, but just doesn't receive a thing and always times out.

Share via

Starting runbook with webhook through data factory

Your answer