Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,886 questions
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(112.00000000000001, 86%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Was this ever resolved? I face the same issue?
How to pass outputs to another activity from a Custom Activity in an Azure Data Factory pipeline?
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 6%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECM%3C/text%3E%3C/svg%3E)
Calabria Montero, Salvador (SGRE SE D FP&DC WEF)
81
Reputation points
Hello,
I need to capture output values from a "Custom Activity" in an Azure Data Factory pipeline to be used, afterwards, by another activity.
I have followed the guidelines in https://learn.microsoft.com/en-us/azure/data-factory/transform-data-using-custom-activity
I have written a outputs.json file with the output and I have tried to get the Activity Output as the value of the customOutput property, but there is not such property in the Activity Output object. I have used the expression "@activity('
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 9%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
KranthiPakala-MSFT 46,512 Reputation points Microsoft Employee2023-11-07T02:18:51.69+00:00 @Calabria Montero, Salvador (SGRE SE D FP&DC WEF) Welcome to Microsoft Q&A forum and thanks for reaching out here.
To better assist on this issue, could you please help share your custom activity output and what specific data from your output you would like to access?
Also please share your dynamic expression that you have used which isn't working.
Have you got a chance to use the expression similar to below:
-
Calabria Montero, Salvador (SGRE SE D FP&DC WEF) 81 Reputation points
2023-11-07T09:27:37.12+00:00 Hello,
Thank you for the response.
My goal was to use the functionality described in the documentation: Use custom activities in an Azure Data Factory or Azure Synapse Analytics pipeline in URL https://learn.microsoft.com/en-us/azure/data-factory/transform-data-using-custom-activity that it seems that provides the option described bellow (read the screenshot included below)
What I did was run a Custom Activity with a python script. The python script creates a json file called "outputs.json" in the batch node located in the same path that the executed python script, and the aim was to use such file in the next activities using the expression @activity('<MyCustomActivity>').output.customOutput.
But the file has been properly created at the Azure Batch node, but I have not been able to get the json file contents in the expression @activity('<MyCustomActivity>').output.customOutput as stated in the MS documentation.
Below the error I get when trying to get the customOutput property
The expression 'activity('fcst-data-retrieval').output.customOutput' cannot be evaluated because property 'customOutput' doesn't exist, available properties are 'exitcode, outputs, computeInformation, effectiveIntegrationRuntime, executionDuration, durationInQueue, billingReference'.
Regards.
-
Calabria Montero, Salvador (SGRE SE D FP&DC WEF) 81 Reputation points
2023-11-07T09:30:39.0866667+00:00 The question that arise is if the functionality described in the official documentacion works and, if so, how it he proper way to consume the information from the json file "outputs.json" that, regarding the doc description, "the service copies the content of the outputs.json and appends into the Activity Output as the value of the customOutput property".
Where is the correct location of the outputs.json file to be properly parsed by the activity.
Thanks in advance.
-
Calabria Montero, Salvador (SGRE SE D FP&DC WEF) 81 Reputation points
2023-11-08T09:20:15.5+00:00 Hello,
Thank you for the response.
My goal was to use the functionality described in the documentation: Use custom activities in an Azure Data Factory or Azure Synapse Analytics pipeline in URL https://learn.microsoft.com/en-us/azure/data-factory/transform-data-using-custom-activity that it seems that provides the option described bellow (read the screenshot included below)
What I did was run a Custom Activity with a python script. The python script creates a json file called "outputs.json" in the batch node located in the same path that the executed python script, and the aim was to use such file in the next activities using the expression @activity('<MyCustomActivity>').output.customOutput.
But the file has been properly created at the Azure Batch node, but I have not been able to get the json file contents in the expression @activity('<MyCustomActivity>').output.customOutput as stated in the MS documentation.
Below the error I get when trying to get the customOutput property
The expression 'activity('fcst-data-retrieval').output.customOutput' cannot be evaluated because property 'customOutput' doesn't exist, available properties are 'exitcode, outputs, computeInformation, effectiveIntegrationRuntime, executionDuration, durationInQueue, billingReference'.
The question that arise is if the functionality described in the official documentacion works and, if so, how it he proper way to consume the information from the json file "outputs.json" that, regarding the doc description, "the service copies the content of the outputs.json and appends into the Activity Output as the value of the customOutput property".
Where is the correct location of the outputs.json file to be properly parsed by the activity.
Thanks in advance.
-
KranthiPakala-MSFT 46,512 Reputation points Microsoft Employee
2023-11-17T01:19:38.01+00:00 @Calabria Montero, Salvador (SGRE SE D FP&DC WEF) Thanks for the detailed info. Could you please confirm if you have already tried the below expression? If so, kindly share the JSON payload of it by masking any sensitive information.
-
Calabria Montero, Salvador (SGRE SE D FP&DC WEF) 81 Reputation points
2023-11-17T07:40:11.43+00:00 Hello, bellow I provide attached screenshot with the json structure f the output of the Custom Activity after run at the Azure Batch node. As I mentioned, the property customOutput is missed and the outputs.json file was created at the the same local folder where the script executed at the az batch node.
Hope this helps.
-
KranthiPakala-MSFT 46,512 Reputation points Microsoft Employee
2023-11-27T23:40:45.2333333+00:00 @Calabria Montero, Salvador (SGRE SE D FP&DC WEF) This looks strange. If the issue still persist, please file a support ticket for deeper investigation. In case if you don't have a support plan, please let me know.
Thank you
Sign in to comment