issues in synapse pipelines which accessing the checkpoints key with input parameters

ANKIT GHENGE 40

hello team,

i am getting synapse analytics pipelines failures with error referring the checkpoint mistaching with the checkpoint in input parameters.

tried to check the input as the respective dataflow is fetching data from cosmosDb with CDC approach and seems to be issue with CDC checkpoint.

please assist me on this.

Ganesh Gurram 2,305 Reputation points Microsoft Vendor

2024-12-20T18:32:21.4666667+00:00

Hello @ANKIT GHENGE

Thanks for the question and using MS Q&A platform.

To better assist you with the issue, could you please provide the detailed error message you're seeing? If possible, provide screenshot of the error message.

Looking forward to your response!
Ganesh Gurram 2,305 Reputation points Microsoft Vendor

2024-12-23T03:24:16.7966667+00:00

@ANKIT GHENGE - We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
ANKIT GHENGE 40 Reputation points

2024-12-23T17:06:38.9233333+00:00

hello @Ganesh Gurram , please refer the error details below.
operation on target dataflow failed:{"statuscod:"DFExecutorUserError","message:"job failed due to reason: at source dataflowfeed': an error occured during snapshot metadata read phase - container IDs in checkpoint dont match container IDs in input parameters" , "details": ""}

no any additional details mentioned.
Ganesh Gurram 2,305 Reputation points Microsoft Vendor

2024-12-23T18:50:28.94+00:00

@ANKIT GHENGE - The error you're encountering indicates a mismatch between the container IDs in the checkpoint and those in the input parameters of your dataflow. This can occur when the checkpoint is reset or if there are changes in the pipeline or activity names, which can lead to the checkpoint being invalidated.

To resolve this issue, you should ensure that the checkpoint key used in your dataflow activity is consistent with the input parameters. If you have changed the pipeline or activity names, consider using a custom checkpoint key to maintain the correct reference to the previous state of your data.

Additionally, if you are using the Change Data Capture (CDC) approach, make sure that the source data is consistent and that the checkpoint is properly configured to track changes since the last successful run.

References: Change data capture in Azure Data Factory and Azure Synapse Analytics

Hope this helps. Do let us know if you have any further queries.
Ganesh Gurram 2,305 Reputation points Microsoft Vendor

2024-12-24T12:20:26.9966667+00:00

@ANKIT GHENGE - Following up to see if the above suggestion was helpful. And, if you have any further query do let us know.

Share via

issues in synapse pipelines which accessing the checkpoints key with input parameters

Your answer