Data flow failed due to transient error

Ananya patil 20

The ADF data flow failed due to a transient error, and during the retry, some data was already loaded into the sink, leading to duplicate records in the insert scenario. Since I didn't implement CDC at the source level and there are no sink checkpoint options to prevent duplicates, is there a way to restart the job from the point of failure on the second attempt?

Your support would be appreciated.

Thanks,

Ananya

Ganesh Gurram 2,550 Reputation points Microsoft Vendor

2025-01-02T07:30:01.5266667+00:00

@Ananya patil - Just checking to see if the below answer provided by @Nandan Hegde helped. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

1 answer

Nandan Hegde 33,001 Reputation points MVP

2024-12-31T04:13:42.5433333+00:00

Unfortunately, based on my understanding;

ADF/Synapse dataflows/copy activity are not transaction bound. And there is no way you can rerun from the point of failure w.r.t number of records and it would be from the beginning only.

So you would have to revert your sink into original state prior to the execution (by deleting/updating the records) and rerun the job to overcome the issue
Please sign in to rate this answer.
Ganesh Gurram 2,550 Reputation points Microsoft Vendor

2025-01-02T13:31:47.6366667+00:00

Hi @Ananya patil

Thanks for the question and using MS Q&A forum.

Similar threads in MS Q&A forum for reference:

https://learn.microsoft.com/en-us/answers/questions/1666422/transient-error

https://learn.microsoft.com/en-us/answers/questions/1627629/how-to-implement-check-points-in-my-existing-desig

Hope this helps. Do let us know if you have any further queries.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Data flow failed due to transient error

1 answer

Your answer