Moving Data From Oracle SQL to Azure SQL Dynamically
Hi All, @{activity('GetTableColumns').output.value[0].ColumnNames} - This line reading my columns from the lookup activity successfully when I try move data from MS SQL(Source) to Azure SQL Databases(Destination). But it's reading the columns from the…
ADLS Gen2 operation failed for: Operation returned an invalid status code 'Conflict'. Account: 'abc'. ErrorCode: 'EndpointUnsupportedAccountFeatures'. Message: 'This endpoint does not support BlobStorageEvents or SoftDelete.
I am new to ADF. while creating linked service for data lake storage Gen2 I am getting a error ADLS Gen2 operation failed. ADLS Gen2 operation failed for: Operation returned an invalid status code 'Conflict'. Account: 'abc'. ErrorCode:…
Servicenow linked service used connectivity vendor
I am interested in using Servicenow Linked Service to extract data from Servicenow. There are two linked services available; the legacy version and version 2 that uses table API. The documentation mentions a built-in driver for enabling connectivity. Can…
How to create a link service in Azure Data Factory for Rest API which has OAuth grant type as password type of authentication
We need to call Rest API which has OAuth grant type as password type of authentication in a loop to store response in CSV. I know we could use Copy Activity but we are unable to create link service with mentioned authentication. We tried to do simple…
Not Able to create connection for Intranet URL to Fabric Data factory
I am working on a project where the requirement is to get content from an Intranet URL through Fabric Data Factory. I have used web Activity and tried to create the connection but getting below error- " request didn't reach the server from the…
User configuration issue: Failure happened on 'Source' side.
Hi I am trying to copy Azure SQL database Prod Environment to Azure SQL database Sandbox Environment through ADF pipeline and getting below error during execution of copy data activity inside Foreach loop. ADF Pipeline is failing during Copy data…
not being able to create azure free account or even to upgrade to pay as u go
how am i suppose to learn the certification if am not being able to create account. i am really frustrated that am not being able to create account. provide me an efficient way to create account to practice for my certification
Azure Data Factory – ForEach overwriting records instead of inserting
I have followed this guide to create a for each loop retrieving the data I need and it is working, however I would like all my JSON records in a single file, but the copy task is overwriting the previously imported record and I cannot see how to make…
Delay in Azure data factory pipeline. Why it is taking so long like 15 minutes to complete execution?
Azure data factory pipeline is calling several sub pipelines depending on the use case of the request. It is observed that when using Script activity it is taking minutes to execute and slowing down the pipeline performance. For other activities it is…
Product tag for the cost with wrong location
Hi support, I have been looking into our cost for the different resources and I see something strange for our Data factory as seen below. Our subscription is located in EU west but the Product tag mention US East. Is this an error or are we paying to…
Connect Azure Data factory to Dynamic 365 and extract data best practice help
How to connect Dynamic365 application with Azure Data Factory and extract data on daily basis incremental data loading machanism
Azure Data Factory - delete row in a csv
I am having some issues with the ADF/Data Flow/Pipeline to delete one row using alterRow. I am filtering by ID=1 and deleting ID=1 row. How to include rest of the rows in my final output. I have only 10 rows. Even if there is a x mark on the row that…
ADF failing CopyData
I have a copy data action that reads from kusto db with a query and has a cosmos db sink. I have validated that the cosmos and kusto are successfully connected, yet while debugging the pipeline I run into this error: Could this be rbac related? I have…
Azure data factory copy activity DBNull error oracle to Iceberg conversion self-hosted integration runtime
Hi, When using a copy activity in a foreach to iterate through a list of table in an on-premise Oracle database and output into Iceberg tables in Azure Blob Gen2, only 26 (or 28 on last run) tables out of 619 are successful in copying. Each failure gives…
Creating sql database table from csv schema file using Azure data factory
Hi, i have requirement, we will get schema files in csv(see below highlighted), there were around 100s of files. here is sample schema csv…
Import schema csv file into sqlserver
Hi, i have scenario , where i will get schema files in csv, there were around 100s of files. here is sample schema csv…
How to use Dynamic add content in Compression type in copy activity
Hi , I created the dynamic dataset and linker service. In copy activity i used that dynamic dataset and want to pass parameter in copy activity or use Compression type option as parameter/Dynamic when i click on Compression type there is drop down menu…
Select transformation output schema is empty after renaming columns
I've followed the example from the link: https://learn.microsoft.com/en-us/azure/data-factory/tutorial-data-flow-dynamic-columns It's exactly what I need for my project, but unfortunately, no matter what I do, the output schema of the select…
User could debug the copy data activity in pipeline successfully but sometimes fail
Problem Description: Recently, when users run the Copy Data Activity in Azure Data Factory, the debug operations are successful, but sometimes they fail. There have been no changes to the pipeline configuration. Both the source and sink used by the users…
Role required to read/write data from cosmos DB (SQL API) from ADF?
We are trying to read/write data from Azure data factory, since local authentication is disabled in cosmos we are trying to access cosmos DB using managed identity. However even with Cosmos DB Account Contributor role assigned to managed identity of…