In ADF cannot skip columns in Copy activity Upsert

Question

I have generic pipeline to upsert data in target synapse tables. Column mapping is also dynamic(using json). I want to skip few column in mapping but I am getting error of column count mismatch.

Answer 1

HI @Yogesh Nikam ,

Welcome to Microsoft Q&A platform and thanks for posting your question here.

As I understand your ask, you want to perform upsert using copy activity in Azure data factory where the mapping is dynamic, and you want to skip few columns from the mapping. Please let me know if my understanding about your query is incorrect.

Kindly make sure you are removing the unwanted columns properties of both Source and sink json in the dynamic column mapping .

I tried to replicate your case and it worked well for me. I removed 'Name' property from the Mapping. From the source, I updated three columns : 'Name','Country' and 'Contact' . However, since name column was not mapped , after the copy activity execution, the other two columns got updated but name column value remained unchanged.

Kindly refer to the below video:

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you.
  Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Answer 2

Sorry, I missed to mention the column which I want to skip is Identity column(Id IDENTITY(1,1) INT) as it is auto increment. But ADF copy UPSERT also looking for it.