Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,274 questions
Based on wildcard (part-*.csv) I want to pick multiple part files and insert into SQL Server
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 93%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDS%3C/text%3E%3C/svg%3E)
Deepak Singh
0
Reputation points
I have a blob path where spark is writing multiple part files . Now I want to load all these part file data to SQL Server . I'm trying using wildcard character *.csv in ADF but it's not working.
I tried using listing the files via GetMetaData but still no luck .
Can you please assit on this .?
{count} votes
-
Nandan Hegde 34,336 Reputation points MVP2025-02-20T14:13:49.7966667+00:00 Can you please share an image of your blob path that contains all your files?
Also share the image of the copy data source config. And when you say the wildcard aspect is not working, do you mean it is not picking up some files or not picking up any files -
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth 13,905 Reputation points Microsoft Vendor2025-02-20T19:33:35.12+00:00 @Deepak Singh We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
-
Deepak Singh 0 Reputation points
2025-02-21T14:31:48.87+00:00 @Nandan Hegde phemanth
Below is the error for your reference .Operation on target Insert Position data failed: ErrorCode=UserErrorSourceBlobNotExist,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The required Blob is missing. Please contact Blob Storage team if you need further support. ContainerName: https://xxxxxxxxx.blob.core.windows.net/dats, path: output/XXX/XXX/normalized/Positions/*.csv.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.WindowsAzure.Storage.StorageException,Message=The remote server returned an error: (404) Not Found.,Source=Microsoft.WindowsAzure.Storage,StorageExtendedMessage=RequestId:ba54b3cf-801e-0035-237e-83a48b000000 Time:Thu, 20 Feb 2025 10:04:53 GMT,,''Type=System.Net.WebException,Message=The remote server returned an error: (404) Not Found.,Source=Microsoft.WindowsAzure.Storage,'Copy Source.
Part Files
Sign in to comment
Sign in to answer