Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,211 questions
0 answers
In express route setup ADF can connect on-premise without SHIR
I have on-premises Oracle databases and have established an ExpressRoute connection between my on-premises network and Azure. I want to use Azure Data Factory to migrate these databases. Do I need a Self-Hosted Integration Runtime (SHIR)? Can ADF connect…
asked 2025-02-09T18:38:08.5466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(19.2, 51%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBB%3C/text%3E%3C/svg%3E)
Brijesh Bajpai
0
Reputation points
asked 2025-02-09T18:38:08.5466667+00:00
Brijesh Bajpai
0
Reputation points
0 answers
How to access parameters passed from ADF within Azure ML Execute Python Script Module
I am passing few parameters from ADF pipeline to Execute ML Pipeline Activity. I couldn't find a way to access those parameters inside Azure ML - Execute Python Script Module. I came across post in 2021 stating "this was not supported", since…
asked 2025-02-09T14:20:20.38+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 14.000000000000002%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAR%3C/text%3E%3C/svg%3E)
Adhisivan Ragunathan
0
Reputation points
asked 2025-02-09T14:20:20.38+00:00
Adhisivan Ragunathan
0
Reputation points
0 answers
Azure data factory calling restapi hosted on app service
I am calling restapi POST method hosted on app service from azure data factory(web activity) using VNet integration runtime, I am using authentication type as NONE. but it is giving error as Error "403 Forbidden 'the web app you have attempted to…
asked 2025-02-08T05:15:33.36+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 35%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPJ%3C/text%3E%3C/svg%3E)
Pankaj Joshi
351
Reputation points
edited the question 2025-02-08T17:10:14.7+00:00
Pankaj Joshi
351
Reputation points
0 answers
Importing ARM Template into a git configured ADF Workspace
Hi All, I have a situation here and I am new to Azure Data Factory & DevOps. I have multiple ADF Workspaces of which Workspace_A is git enabled but it does not have any resources. Workspace_B is in Live mode but has all the pipelines, dataflows,…
asked 2025-02-08T01:03:08.5133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(67.2, 53%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECL%3C/text%3E%3C/svg%3E)
Chenduri, Lakshminarasu (Cognizant)
0
Reputation points
asked 2025-02-08T01:03:08.5133333+00:00
Chenduri, Lakshminarasu (Cognizant)
0
Reputation points
0 answers
How to dynamically pass variable based on the branch the ADF pipeline is triggering from
I am looking for a way to dynamically pass a variable based on what git branch the pipeline is running on. Alternatively, if within ADF I can extract the contents of the linked service JSON that should be sufficient for my task. Is there a way to specify…
asked 2025-02-08T00:27:56.4233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(102.4, 39%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDY%3C/text%3E%3C/svg%3E)
Daniel Yang
0
Reputation points
asked 2025-02-08T00:27:56.4233333+00:00
Daniel Yang
0
Reputation points
3 answers
Adf error 2108 - Accessing web activity post call
Hello, I am in the process of designing a pipeline of writing back to api using endpoint of SAP. I was able to do a post call and get the needed resultset from postman. While the same end point is giving unauthorised error in adf. I am not sure if…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 50%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EN%3C/text%3E%3C/svg%3E)
0 answers
Azure cost variance
Hi friends, our dashboard retrieved the data from the Azure cost management tables but the metrics such as "Total Cost" are lesser than "Total Effective Price" and "Total PayGPrice" which is not practically correct. I have…
asked 2025-02-06T07:57:41.74+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 88%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
azure_learner
525
Reputation points
commented 2025-02-07T21:49:49.42+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 20%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPR%3C/text%3E%3C/svg%3E)
Pranay Reddy Madireddy
1,690
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Rest call failed with client error in Source ADF , status code 401 Unauthorized with Service bus in Azure Cloud
Hi, Trying to post a JSON message to Service bus queue via ADF REST API call but ended up the following error message. The ADF has the service bus owner and sender roles. ADF and service bus are in different virtual networks (West US and West US2) in the…
asked 2025-02-06T21:18:10.75+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(131.20000000000002, 27%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECK%3C/text%3E%3C/svg%3E)
chaitanya.K
40
Reputation points
accepted 2025-02-07T19:56:27.2633333+00:00
chaitanya.K
40
Reputation points
1 answer
Error in publishing in ADF
Hi, I had some redundant databases in my Azure subscription that also had redundant pipelines in my Dev Data Factory. I deleted these databases, deleted the linked services and the activities that were referencing them in my factory, however I am unable…
asked 2025-02-06T15:37:15.2966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 90%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDR%3C/text%3E%3C/svg%3E)
Diana Rus
10
Reputation points
commented 2025-02-07T14:32:04.0333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
Ganesh Gurram
3,765
Reputation points Microsoft Vendor
1 answer
Linked Service - Read / Write to ADLS Storage using logged in user account
Hi Linked Service connection to a ADLS storage account looks to be restricted to the following (screen shot). Is there any way of changing this so the authentication type is the user who has logged into ADF? Essentially we want to ensure only the user…
asked 2025-02-07T11:30:32.8466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 20%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
SM
0
Reputation points
commented 2025-02-07T14:27:00.61+00:00
Nandan Hegde
34,021
Reputation points MVP
0 answers
Managed private endpoint -The approval state may be stale by a minute
Hi created Managed private end point in Azure data factory using RestAPI and approved. But it is showing The approval state may be stale by a minute. This is happening with multiple MPEs and did not approve after many hours as well. Fyi - Few MPEs…
asked 2025-02-07T13:18:43.16+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(121.6, 87%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELN%3C/text%3E%3C/svg%3E)
Lakshmi Narayana Tenka
5
Reputation points
edited the question 2025-02-07T13:19:40.2766667+00:00
Lakshmi Narayana Tenka
5
Reputation points
0 answers
i am facing azure sign up issues, it's failed to verify my mobile number
i am facing azure sign up issues, it's failed to verify my mobile number, i have tried it with other mobile number as well.
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,211 questions
Not Monitored
Not Monitored
Tag not monitored by Microsoft.
41,893 questions
Azure ISV (Independent Software Vendor)
Azure ISV (Independent Software Vendor)
Azure: A cloud computing platform and infrastructure for building, deploying and managing applications and services through a worldwide network of Microsoft-managed datacenters.ISV (Independent Software Vendor): A Microsoft program that helps customers adopt Microsoft Cloud solutions and drive user adoption.
1 question
asked 2025-02-07T09:50:46.6766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(211.20000000000002, 63%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMI%3C/text%3E%3C/svg%3E)
Muhammad Irfan Khalid
0
Reputation points
edited the question 2025-02-07T13:04:14.48+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 40%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKA%3C/text%3E%3C/svg%3E)
Khadeer Ali
2,945
Reputation points Microsoft Vendor
1 answer
Azure data factory DB2 connector is not converting EBCDIC encoded string
Currently, I am facing a problem with the DB2 connector, DB2 connector is not supporting EBCDIC encoding. When I see the preview data, it's showing Chinese characters. I am connecting AS-400 Series DB and getting the data using the DB2 connector (not…
asked 2021-12-09T12:24:36.727+00:00
Sabyasachi Maity
1
Reputation point
commented 2025-02-07T12:30:29.5166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 69%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Caspar Boersma
0
Reputation points
1 answer
How to write query for fetching count of entity from Dataverse
Hi All, I have a linked service connecting to Dataverse and i am trying to fetch count of an entity/table via lookup activity. What query should i write to fetch only table count of an entity?
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(291.2, 27%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPJ%3C/text%3E%3C/svg%3E)
1 answer
GetMetadat Activity returns False even if folder exists
I have enabled GetMetadata Activity to check any folder/files existing in a Blob Storage. The Exists property which was working earlier isn't functioning as expected as observed this week and returns False even if a folder exists in the path.
asked 2025-01-22T12:01:59.2466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(19.2, 83%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESS%3C/text%3E%3C/svg%3E)
Sambit Sourav
0
Reputation points
edited a comment 2025-02-07T10:03:31.5533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
7,960
Reputation points Microsoft Vendor
2 answers
Zoho Authentication: Successful in Fabric Data Factory, Fails in Azure Data Factory"
I am trying to access https://accounts.zoho.com/oauth/v2/token to get the token for my zoho endpoint.. When using Web activity on fabric it is a cakewalk and I am getting token but when I am trying same in azure data factory I am getting an error as…
asked 2025-01-31T08:55:06.53+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(233.6, 50%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAM%3C/text%3E%3C/svg%3E)
Amit Mohanty
0
Reputation points
commented 2025-02-07T09:53:13.7866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 57.00000000000001%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERG%3C/text%3E%3C/svg%3E)
Rakesh Govindula
0
Reputation points Microsoft Vendor
1 answer
How to connect Dataverse tables from ADF?
We have a Dataverse table that we need to access via ADF. Following the process of user access amnaged identity we have created the managed identity, registered the app in Azure, Gave the access in Dataverse and then used the same in ADF Linked service.…
asked 2025-02-05T11:24:23.9+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 80%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESP%3C/text%3E%3C/svg%3E)
Sowmya Pantula
0
Reputation points
commented 2025-02-07T09:33:31.1733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
19,880
Reputation points Microsoft Vendor
0 answers
Having trouble in copy data between cosmos mongodb
Operation on target ForEach_q2g failed: Activity failed because an inner activity failed; Inner activity name: Copy_q2g, Error: Failure happened on 'Sink' side.…
asked 2024-01-29T14:48:52.7+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 91%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERV%3C/text%3E%3C/svg%3E)
Rajagopal Venkataramani
5
Reputation points
commented 2025-02-07T08:42:06.3633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(108.80000000000001, 17%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMV%3C/text%3E%3C/svg%3E)
Mamidi Vasavi
0
Reputation points
1 answer
How to Reduce Queue Time in Azure Data Factory for Self-hosted Integration Runtime?
Hello everyone, We have been experiencing significant queue times in Azure Data Factory for our self-hosted integration runtime (SHIR). Despite reaching out to Microsoft support for several months, we have not been able to resolve this issue. Our SHIR…
asked 2025-01-20T06:49:04.9366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 12%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECR%3C/text%3E%3C/svg%3E)
Contreras,Rafael,MX-Ciudad de México
0
Reputation points
commented 2025-02-07T06:00:31.0366667+00:00
AnnuKumari-MSFT
34,011
Reputation points Microsoft Employee
1 answer
Can Azure Data Factory copy data activity handle 500 GB data?
Hi Team, I have one use case where I need to copy 500GB single file from sFTP location to AWS S3 bucket using Azure iPaaS services. I know ADF doesn't support S3 bucket as sink as of now but I'm planning to put that large file in AWS sFTP which can drop…
asked 2025-01-31T09:00:29.76+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(236.8, 31%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPD%3C/text%3E%3C/svg%3E)
Pratim Das, Partha C
326
Reputation points
commented 2025-02-07T01:09:40.7466667+00:00
Chandra Boorla
7,960
Reputation points Microsoft Vendor