10,843 questions with Azure Data Factory tags

Sort by: Updated
1 answer

i cannot find add directory inside container

SharedScreenshot.jpg

Azure Container Instances
Azure Container Instances
An Azure service that provides customers with a serverless container experience.
711 questions
Azure Virtual Machines
Azure Virtual Machines
An Azure service that is used to provision Windows and Linux virtual machines.
7,994 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
Active Directory
Active Directory
A set of directory-based technologies included in Windows Server.
6,644 questions
Azure App Service
Azure App Service
Azure App Service is a service used to create and deploy scalable, mission-critical web apps.
7,913 questions
asked 2024-11-09T09:26:39.45+00:00
Aradhana Chauhan 0 Reputation points
answered 2024-11-09T21:18:23.97+00:00
akinbade abiola 18,305 Reputation points
1 answer

Getting below error while try to connect with Azure SQL Database using linked service through Azure Data Factory

Please Check Below error:- Cannot connect to SQL Database. Please contact SQL server team for further support. Server: 'cms-data-cleansing-poc.database.windows.net', Database: 'datacleansingdb', User: 'admindata@poc'. Check the linked service…

Azure SQL Database
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
asked 2024-11-09T18:10:33.86+00:00
EduTech Gyan 0 Reputation points
answered 2024-11-09T18:52:29.7833333+00:00
Iufor 0 Reputation points
0 answers

could you please let me know the use of import projection in azure data factory pipeline.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
asked 2024-11-09T16:27:35.22+00:00
Neeraja Narravula 0 Reputation points
1 answer One of the answers was accepted by the question author.

DeltaCopyDatabase

Where can we find the Delta copy from Database template in ADF referred in this article - https://learn.microsoft.com/en-us/azure/data-factory/solution-template-delta-copy-with-control-table

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
asked 2024-11-08T11:25:18.0433333+00:00
S D, Sudevika 20 Reputation points
commented 2024-11-09T12:34:19.97+00:00
Smaran Thoomu 16,890 Reputation points Microsoft Vendor
1 answer One of the answers was accepted by the question author.

ADF billing is high

Hi, I had asked about the cost of scheduled as well as manual triggers as per answer in the below thread no matter whether it is manual or scheduled triggers the cost applicable in the pipeline activities involved.…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
asked 2024-11-08T16:59:44.7433333+00:00
azure_learner 340 Reputation points
accepted 2024-11-09T07:14:31.3566667+00:00
azure_learner 340 Reputation points
0 answers

How we can extract Oracle eloqua bulk api objects using Oracle Eloqua Connector in adf

We want to ingest activities data from eloqua to snowflake using Oracle Eloqua Connector but we can't see activities table. But we are able to ingest activities data from eloqua bulk api. What is the reason?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
asked 2024-11-08T19:50:57.7+00:00
Shaikh Juned Shaikh Jamil 0 Reputation points
edited the question 2024-11-08T19:51:52.3033333+00:00
Shaikh Juned Shaikh Jamil 0 Reputation points
0 answers

Connecting to SFTP Source in App Fabric using SSH Private Key

I'm trying to establish a connection to an SFTP source in Azure App Fabric, but I'm encountering a roadblock. While Azure Data Factory offers various authentication methods like SSH and Basic, I haven't found similar options in App Fabric. Options…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
Microsoft Fabric Training
Microsoft Fabric Training
Microsoft Fabric: A Microsoft unified data platform.Training: Instruction to develop new skills.
50 questions
asked 2024-11-08T19:31:48.9066667+00:00
Abhijit Shrikhande 337 Reputation points
0 answers

How to use a private key to connect to an SFTP datasource in AppFabric?

I am creating my first data factory in app fabric. One feature I found missing is that I cannot connect to a SFTP source unless, I have basic authentication. In the previous version of Azure Data Factory, I had an option of selecting the authentication…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
Microsoft Fabric Training
Microsoft Fabric Training
Microsoft Fabric: A Microsoft unified data platform.Training: Instruction to develop new skills.
50 questions
asked 2024-11-08T19:27:39.11+00:00
Abhijit Shrikhande 337 Reputation points
0 answers

ADF Copy data activity pagination with continuation token with GET request

I am building ETL pipeline in Microsoft Fabric, ADF service to load data from API, but I am having difficulties implementing pagination rules with continuation token. I know that, according to Azure documentation, it should be possible to use this…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
Microsoft Partner Center API
Microsoft Partner Center API
Microsoft Partner Center: A Microsoft website for partners that provides access to product support, a partner community, and other partner services.API: A software intermediary that allows two applications to interact with each other.
345 questions
asked 2024-11-08T17:59:47.7966667+00:00
Darja Jafanova 0 Reputation points
commented 2024-11-08T19:25:45.7733333+00:00
phemanth 11,280 Reputation points Microsoft Vendor
1 answer

How to check text in row 1 of Excel file for about 10 column headers?

I'm using Azure Data Factory (ADF) on the portal. I use the latest Chrome browser to get to ADF. I'm not an admin and have no admin rights. I have an Excel file I want to read. I'm still fairly new to ADF. I'm guessing this could be done in a data flow.…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
asked 2024-11-05T12:36:13.51+00:00
Chuck Roberts 110 Reputation points
commented 2024-11-08T18:40:33.8033333+00:00
phemanth 11,280 Reputation points Microsoft Vendor
3 answers One of the answers was accepted by the question author.

I am getting a BadRequest error while running adf pipeline which contains mapping data flow activity

Hi, I am getting BadRequest error while running adf pipeline which contains data flow activity. It was running fine previously.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
asked 2023-04-19T11:11:45.0533333+00:00
Nikita Randive 60 Reputation points
answered 2024-11-08T17:55:57.89+00:00
Mickey Stringer 0 Reputation points
1 answer

Is it possible to use function encrypt HmacSHA256 on Azure Data Factory ?

Hi All, I have ADF Pipeline to call some API and that API required signature (encrypt HmacSHA256) for authenticate in body. Is it possible to use function encrypt HmacSHA256 on Azure Data Factory ? or Do you have any solution to encrypt HmacSHA256 on…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
asked 2024-09-24T04:26:37.8933333+00:00
Nanchai Ko 5 Reputation points
commented 2024-11-08T17:13:49.46+00:00
Moyer, Todd 80 Reputation points
2 answers

Dynamic Array Mapping in ADF Copy Activity

I'm attempting to set up dynamic mapping between blob storage in an ADF (Azure Data Factory) Copy activity. In the mapping section, I'm trying to add an additional column using dynamic content from a set variable that is an array. However, I encounter…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
asked 2024-11-07T18:39:41.93+00:00
Akhil 0 Reputation points
answered 2024-11-08T16:41:13.91+00:00
Chandra Boorla 2,915 Reputation points Microsoft Vendor
1 answer

Copy Files from sharepoint online site to azure datalake storage

Hello We are trying to setup the flow which will copy files from sharepoint online site to azure datalake storage. As per my understanding there are 2 options : Using ADF to pull the files as mentioned in the link below…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,485 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
SharePoint
SharePoint
A group of Microsoft Products and technologies used for sharing and managing content, knowledge, and applications.
10,827 questions
Microsoft Entra
Microsoft Entra ID
Microsoft Entra ID
A Microsoft Entra identity service that provides identity management and access control capabilities. Replaces Azure Active Directory.
22,116 questions
asked 2024-11-06T22:12:34.5+00:00
Sourav 105 Reputation points
commented 2024-11-08T16:25:16.5366667+00:00
Keshavulu Dasari 1,155 Reputation points Microsoft Vendor
1 answer

Transforming JSON files using data flow

Hello! I currently have about 60 json files inside a blob container which most of them have different fields and values. I have created a pipeline with a get metadata activity that points to the container, with the field list set to Child items. I have…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,485 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
asked 2024-11-07T15:23:15.93+00:00
A K 20 Reputation points
commented 2024-11-08T16:19:12.42+00:00
Keshavulu Dasari 1,155 Reputation points Microsoft Vendor
1 answer

ODATA doesn't support expand in ADF

Hi, The expand verb does not seem to work in ADF with the ODATA connector. This means that we cannot fetch the child / nested items. The select works fine but not expand. Anyone has experience or workaround with this ?

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
asked 2021-12-13T04:09:19.887+00:00
Bajaj, Smita 1 Reputation point
commented 2024-11-08T15:42:38.2333333+00:00
Wing Yee Cheung 0 Reputation points
0 answers

Authentication failure with linked service. Linked service exist in corp tenant and data factory exist in torus tenant. How do I resolve this issue, is it possible to raise exception so that authentication will be success.

Operation on target Execute dataflow failed: {"StatusCode":"DFExecutorUserError","Message":"Job failed due to reason: Client initialization failed. Check if the endpoint is reachable and if your auth token is valid.…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
asked 2024-11-08T12:04:29.7433333+00:00
Shweta Wagh 0 Reputation points Microsoft Employee
edited the question 2024-11-08T15:21:05.2333333+00:00
VarunTha 9,270 Reputation points Microsoft Vendor
2 answers

Parametrization for connection string is not working in linked service

Hi there, I have solution - data pipeline in Synapse taking data from mysql database. I need to parametrize it to be able run process across many environemnts (CI/CD). I set many linked services with code referencing on variable in yml file in style…

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,997 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
asked 2024-11-07T19:55:32.4766667+00:00
Peter Michalik 0 Reputation points
answered 2024-11-08T12:17:15.1266667+00:00
Peter Michalik 0 Reputation points
3 answers

Unable to create dynamic linked service for Microsoft Access database

Hi Team I am trying to create Linked service for Microsoft Access database using dynamic connection string where i am passing connection string value through parameter, test connection working fine but I am getting following error while saving the linked…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
asked 2024-10-28T17:43:18.2+00:00
Dinesh Kumar 15 Reputation points
answered 2024-11-08T11:31:47.3766667+00:00
Dinesh Kumar 15 Reputation points
1 answer

Cannot reference to an invalid or deleted Credential object for Fabric Linked Service

Getting this error after refreshing ADF page , credentials is getting removed from Fabric Warehouse Linked Service . Seems a similar problem like…

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,843 questions
asked 2024-11-08T05:50:40.7466667+00:00
Tirthankar Biswas 0 Reputation points
commented 2024-11-08T10:47:59.7866667+00:00
Tirthankar Biswas 0 Reputation points