Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,290 questions
1 answer
Unable to use linked service to blob storage account with Test Connection as "To File Path"
I am trying to create a linked service to Azure blob storage account. I need to use System Assigned Managed Identity for Authentication. No restriction on account selection method. I tried using "From Azure Subscription" (which gets modified to…
asked 2025-02-25T14:40:14.99+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 43%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAA%3C/text%3E%3C/svg%3E)
Aqshata Akshay Ajnadkar
0
Reputation points
answered 2025-02-28T07:01:35.8066667+00:00
Rachid Labiëd
15
Reputation points
1 answer
Merge around 3000 parquet files to a single parquet file in ADF in gen2 storage
Hi I want to merge like 3000 parquet files to a single parquet file in ADF. The files are in gen2 storage account. I did try the copy activity and merge into single file but its very slow. How can I do this in a fast manner. Can we use ADF data flow if…
asked 2025-02-28T00:30:05.58+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(70.4, 76%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
Shivshankar Kanawade
0
Reputation points Microsoft Employee
answered 2025-02-28T05:53:24.0166667+00:00
Vinodh247
28,386
Reputation points MVP
1 answer
facing this below error how can i solve this
ErrorCode=UserErrorColumnMappingNotAllowed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Column mapping is not allowed for copy between 'AzureBlobStorage(DelimitedTextFormat)' and…
asked 2025-02-22T08:07:46.3166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 68%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EV%3C/text%3E%3C/svg%3E)
varalakshmi
0
Reputation points
commented 2025-02-28T05:40:01.81+00:00
Vinodh247
28,386
Reputation points MVP
1 answer
Extract Unix timestamp from json and converting to human readable datetime
Hi I have a json file in datalake. It has createtime and updatetime fields in Unix timeformat. I am unable to transform using Derived column. I am getting null in the result. "createTime": 1716327033431, "updateTime":…
asked 2025-02-28T00:29:21.3633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(185.6, 38%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAA%3C/text%3E%3C/svg%3E)
Ali, Ayesha
0
Reputation points
answered 2025-02-28T04:38:29.97+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 99%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJN%3C/text%3E%3C/svg%3E)
J N S S Kasyap
265
Reputation points Microsoft Vendor
0 answers
Unable to connect to Azure Data Factory with Azure DevOps server 2022
Trying to Setup the git repository for the Azure Data factory to the On-premise Azure DevOps Server 2020 version. we follow the steps in the Microsoft documentation. But we are getting the below…
asked 2025-02-27T12:08:27.3+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(233.6, 38%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECV%3C/text%3E%3C/svg%3E)
Chunduru, Vamsee Krishna
0
Reputation points
commented 2025-02-28T04:20:37.11+00:00
Chunduru, Vamsee Krishna
0
Reputation points
1 answer
Connect to Onprem SQL Database from Azure Data Factory
Hi Team, I am trying to create a linked Service in the Azure Data Factory for setting up a connection to SQL Database on - prem server using Windows Authentication in ADF Linked Service to copy data to Cloud and my Test Connection Failed with below Error…
asked 2025-02-27T21:56:18.79+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(131.20000000000002, 67%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENB%3C/text%3E%3C/svg%3E)
Nalini Bhavaraju
40
Reputation points
edited an answer 2025-02-28T03:02:16.32+00:00
Nandan Hegde
34,341
Reputation points MVP
0 answers
ADF Dataflow issue with source API query
am trying to setup an Azure Data Factory Dataflow. The source is a REST API from Monday.com which uses GraphQL. The API requires POST method, with 3 headers Content-Type "application/json", API-Version "2025-01 " and Authorization:…
asked 2025-02-22T03:15:01.8166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 69%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDB%3C/text%3E%3C/svg%3E)
Daniel Baird
0
Reputation points
commented 2025-02-28T00:30:42.15+00:00
J N S S Kasyap
265
Reputation points Microsoft Vendor
1 answer
DateTime source values showing as NULL in Azure Data Factory - Data Flows
Hi, Is there a reason why my data flow cannot read/treat the DateTime input column? In my source file, I have a column named 'Date' with DateTime values of format 'yyyy-MM-dd HH:mm:ss', but the data preview always shows NULL values. Hence, to fix this…
asked 2023-03-22T23:15:04.3233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 53%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVK%3C/text%3E%3C/svg%3E)
Vivek Komarla Bhaskar
936
Reputation points
commented 2025-02-28T00:01:49.24+00:00
Ali, Ayesha
0
Reputation points
0 answers
Trying to read a file in Azure DevOps from ADF pipeline using a web activity
Web activity URL - https://dev.azure.com/MB/sql1-mb/_apis/git/repositories/sql1-mb/items?path=/sql-code/readme.md&download=true&api-version=7.1 Azure access Azure DevOps Response received - I think the error is ' Enhanced Security…
asked 2025-02-25T15:09:31.8733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 28.000000000000004%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMB%3C/text%3E%3C/svg%3E)
Michael Berry
40
Reputation points
commented 2025-02-27T18:15:47.8733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
14,220
Reputation points Microsoft Vendor
0 answers
How to fetch the child items within a folder located on azure blob
I am trying to fetch child items of a folder for recursive deletion, so my expectation is to delete the folders within the folder, like if folder A has subfolder B with file b and another subfolder AB has subfolder D, then deleting all items from…
asked 2025-02-26T18:58:40.5766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(144, 0%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAC%3C/text%3E%3C/svg%3E)
Ajay Chowdary Kandula
0
Reputation points
edited a comment 2025-02-27T18:03:50.1133333+00:00
Ajay Chowdary Kandula
0
Reputation points
1 answer
One of the answers was accepted by the question author.
NAT Gateway for Managed VNet IR
How can I make Azure Managed VNet IR to make requests through a static public I.P? I want to pull data from 'xyz.com/books' api. This API provider doesn't allow requests from everyone. They only allow requests from IPs which are whitelisted. I have…
asked 2025-02-10T22:46:31.83+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 49%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Amrale, Siddhesh
20
Reputation points
accepted 2025-02-27T17:22:22.29+00:00
Amrale, Siddhesh
20
Reputation points
3 answers
One of the answers was accepted by the question author.
Parameterizing Azure ML linked service in ADF
I am trying to dynamically configure ML Execute activity in ADF. It allowed me to create dynamic content with parameters for ML linked service for subscription ID, workspace name etc. However when integrating with ADF ML Execute activity, I don't see…
asked 2025-02-13T02:00:02.9566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 14.000000000000002%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAR%3C/text%3E%3C/svg%3E)
Adhisivan Ragunathan
40
Reputation points
accepted 2025-02-27T14:23:46.22+00:00
Adhisivan Ragunathan
40
Reputation points
0 answers
Azure Data Factory trigger event not working while uploading using .NET client library
I want to trigger an ADF pipeline on blob creation, file based trigger. The manual upload to blob storage triggers the ADF pipeline correctly. But when i am uploading the same via AzureDataLakeStore .Net libraries from a web app, specifically package - …
asked 2025-02-25T22:26:52.95+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 54%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENT%3C/text%3E%3C/svg%3E)
NATASHA TAHARIYA
0
Reputation points
commented 2025-02-27T12:44:45.03+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
9,120
Reputation points Microsoft Vendor
1 answer
cosmosDB Azure DataFactor DataFlow configure sink without a throughput value
I have a simple dataflow in a pipeline that reads a tab delimited file with 5 columns. The last column is valid json. I want the data flow to read the file and insert each json in each row in column 5 to cosmosDB. All of the connections pass test…
asked 2025-02-24T21:06:28.45+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(272, 63%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBC%3C/text%3E%3C/svg%3E)
Brady Chambers
0
Reputation points
commented 2025-02-27T11:48:13.4166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 57.00000000000001%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERG%3C/text%3E%3C/svg%3E)
Rakesh Govindula
10
Reputation points Microsoft Vendor
2 answers
Azure Data Factory - Copy data from AWS Redshift to Azure Synapse - Failed to connect error
Hi, I am trying to copy a table of data from AWS Redshift into Azure Synapse, below is the JSON code from the data factory: { "name": "pipeline1", "properties": { "activities": [ { …
asked 2025-02-18T22:50:02.35+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 60%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVP%3C/text%3E%3C/svg%3E)
Vinny Prajapati
0
Reputation points
commented 2025-02-27T09:58:04.4066667+00:00
Vinny Prajapati
0
Reputation points
1 answer
One of the answers was accepted by the question author.
Unable to integrate SAP Datasphere with ADF using OData
I am trying to integrate SAP Datasphere with Azure Data Factory using OData but it failed with the below exception. I also created SFRI but did not work. Please fix this. Thank you for any help in advance.
asked 2024-09-08T19:00:34.0366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(316.8, 54%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGS%3C/text%3E%3C/svg%3E)
G, Shruti
20
Reputation points
commented 2025-02-27T09:28:36.1366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(83.2, 11%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPP%3C/text%3E%3C/svg%3E)
Preethi Pugazhendi
0
Reputation points
2 answers
ADF Power Query Sink Settings UI Missing Options
Why would my data factory not have the same UI as I see in training videos and QA answers on the Power Query data wrangling part of ADF? I can only see the delete button next to the dataset in the Power Query activity Sink…
asked 2025-01-23T15:50:49.8533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(272, 24%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EES%3C/text%3E%3C/svg%3E)
Elisa Schrader
20
Reputation points
commented 2025-02-27T07:24:21.95+00:00
AnnuKumari-MSFT
34,106
Reputation points Microsoft Employee
1 answer
How to Register an on-prem Self-Hosted Integration Runtime for Azure Data Factory that uses a Private Endpoint
I have configured an Azure Data Factory using a private endpoint w/ private DNS for access. I am wanting to be able to connect to local resources within Data Factory. I have created a self-hosted integration runtime within DF Studio and downloaded the…
asked 2023-12-05T20:57:55.4833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 94%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAH%3C/text%3E%3C/svg%3E)
Aaron Hurst
5
Reputation points
commented 2025-02-27T06:25:46.3166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(0, 96%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBR%3C/text%3E%3C/svg%3E)
Ben Roberts
0
Reputation points
2 answers
One of the answers was accepted by the question author.
Synapse Pipeline reading from Excel -- how to generate output table Schema
I'm putting together my first pipeline to capture data in an Azure SQL Database table from an incoming Excel Spreadsheet. I've figured out how to load the spreadsheet to blob storage and to link to it. It's ready to go and I can see it and preview it…
asked 2024-01-05T19:40:36.25+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(275.2, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJE%3C/text%3E%3C/svg%3E)
John Edwards
35
Reputation points
commented 2025-02-27T02:56:38.9433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(121.6, 89%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENS%3C/text%3E%3C/svg%3E)
Nicholas Smith
0
Reputation points
0 answers
Issue with Linked Service Configuration in Azure Data Factory for Cross-Tenant Microsoft Fabric SQL Database
I have two Azure tenants: Tenant ABC and Tenant XYZ. I am using Azure Data Factory (ADF) in Tenant ABC and Microsoft Fabric in Tenant XYZ. To establish connectivity between them, I have created a Linked Service in ADF (Tenant ABC) to connect to the SQL…
asked 2025-02-24T09:38:40.16+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(211.20000000000002, 42%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EID%3C/text%3E%3C/svg%3E)
Isha Dhameliya
0
Reputation points
commented 2025-02-26T18:33:34.48+00:00
phemanth
14,220
Reputation points Microsoft Vendor