issues in synapse pipelines which accessing the checkpoints key with input parameters

Question

hello team,

i am getting synapse analytics pipelines failures with error referring the checkpoint mistaching with the checkpoint in input parameters.

tried to check the input as the respective dataflow is fetching data from cosmosDb with CDC approach and seems to be issue with CDC checkpoint.

please assist me on this.

Answer 1

@ANKIT GHENGE - The error you're encountering indicates a mismatch between the container IDs in the checkpoint and those in the input parameters of your dataflow. This can occur when the checkpoint is reset or if there are changes in the pipeline or activity names, which can lead to the checkpoint being invalidated.

To resolve this issue, you should ensure that the checkpoint key used in your dataflow activity is consistent with the input parameters. If you have changed the pipeline or activity names, consider using a custom checkpoint key to maintain the correct reference to the previous state of your data.

Additionally, if you are using the Change Data Capture (CDC) approach, make sure that the source data is consistent and that the checkpoint is properly configured to track changes since the last successful run.

---

References: Change data capture in Azure Data Factory and Azure Synapse Analytics

Hope this helps. Do let us know if you have any further queries.