ADF multiple copy acticity thorugh Control Flow Foreach activity billing meter is costing for individual copy activity instead of overall pipeline execution

Question

Hi Team,

I have question regarding copy activity billing meter based which is based on single copy activity run on per minute basis.

We have some 100 table sql server in source which we copy into ADLS and we copy it in a single pipeline run using control flow through ForEach activity and copy all 100 tables into ADLS Gen2 and we observe for each copy activity run it is taking less than 30 sec however, we noticed billing is calculated based on per min on each copy activity which is calculating the billing of 100 copy activities to 100 min even though the pipeline execution is happening with in 5 min duration.

Do you have any solution approach to reduce billing meter for copy activity when we bulk table copy requirement.

Thank you in anticipation!!

Best Regards,

Mahammad Khan

mahammedkhan@Stuff .com

+91-9885360726

Answer 1

Hi @Abdulla Mahammad Khan Dawood ,

Welcome to Microsoft Q&A forum and thanks for your query.

In Azure Data Factory, you pay for data pipeline orchestration by activity run and activity execution by integration runtime hours.

According to Azure Data Factory pricing document, each data movement activity will round to minute and multiply the used data integration unit, so the cost for copy activity is :

  Data movement activity duration(round to minute) * 1(min)/60 * DIU (default 4) * $0.25/hour  

As per above information, though each of your copy activity runs for 30sec, it will round up to a minute. And since you are having 100 tables = 100 copy activity runs = 100 min of billing for copy activity.

As this is the standard billing procedure for ADF data movement activities, I don't see other alternatives to reduce the cost.

If you have any feedback regarding ADF product, I would recommend you to please share your feedback in ADF user voice channel: https://feedback.azure.com/forums/270578-data-factory

All the feedback shared in this forum are actively monitored and reviewed by ADF engineering team. Also please do share the feedback link once it is posted in user voice channel so that others with similar idea can upvote and/or comment on your suggestion/feedback.

Hope this information helps.

Additional info:

• Understanding Data Factory pricing through examples

----------

Thank you
Please do consider to click on "Accept Answer" and "Upvote" on the post that helps you, as it can be beneficial to other community members.