Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,918 questions
Copy Activity readig extra records from SAP BW (as a source) In ADF.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 45%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESB%3C/text%3E%3C/svg%3E)
Sujithra Baskaran
1
Reputation point
Source: SAP BW (using open hub connector)
Sink : ADLS Gen 2(Blob) - Parquet format.
While using copy activity, the copy activity reading extra records (duplicate recods) from the table and writing into Sink.
SAP Source :
ADF Copy activity:
Target table (found duplications)
Usually the copy activity fetches the data as data packets from SAP Table and each data packet holds only 50k records from the source, but copy activity duplicating the records in some of the data packets for few tables..
Note: I have 45 tables to be loaded in parallel and 45 copy activity are parallely executed and SHIR is used here. This issue occuring once in 2 months. we faced this in PROD system.
Do any one came across this?, Please suggest how to resolve it.
Sign in to answer