how to connect azure cache redis to azure databricks through service principal
I need to connect azure redis to databricks using service principal . but I am getting this error Failed to connect to Redis: invalid username-password pair
copy data gives error NullReferenceException: Object reference not set to an instance of an object
I'm copying from Azure Blob Storage .csv files into Snowflake, and receiving the error ErrorCode=SessionLogFailToWriteLog,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Write log is failed with error message:…
Migrating ADFS to Entra ID question
Dear All, i'm new in Azure, i have a hybird environment with ADFS, i was planning to migrate ADFS to Entra ID, and i have already downloaded entra health connect agent and install it on ADFS and domain controller, having waited for 1 day i go…
Azure Data Factory Pipeline
Hi there, This pipeline runs Azure SQL SPs. And wanted to check with you if my connections achieve the below. SP3,4,5,6 should run only after SP1 and 2 gets completed. Is there a container so its an AND operation? SP 7 should run only when SP3,4,5,6 have…
I'm using data factory and my explorer seems to be frozen. Behind the scenes I am able to create tables but when I request to drop a table it says that I have two columns with similar names and that I need to use SQL commands to rename these columns.
I have a table InstallBase which I have created successfully using a copy data task in Data Factory. When I go to Explorer I can't see the table. When I then do to drop the table using a drop table SQL command it drops the table but reports that I have…
Having trouble in copy data between cosmos mongodb
Operation on target ForEach_q2g failed: Activity failed because an inner activity failed; Inner activity name: Copy_q2g, Error: Failure happened on 'Sink' side.…
Regarding not being able to deploy the project in Azure which is connected to Github
Hello, I have a Django project which I uploaded to GitHub first and then tried to connect to Azure. The build process has been successful as per the Action tab in GitHub.But the process gets to the Deploy part, it takes around 30 minutes and then provide…
accidentally deleted during a deployment to my Azure App Service, Unfortunately, I do not have a backup enabled for this app
I am reaching out to request your assistance in recovering files that were accidentally deleted during a deployment to my Azure App Service. Unfortunately, I do not have a backup enabled for this app, and the deployment process seems to have overwritten…
Error during batchpoint invoke: Access to this resource is denied. Please check your ACL rules on the resource
Hello, I am getting a "Access to this resource is denied. Please check your ACL rules on the resource." when attempting to invoke a batch endpoint. I have tried both with Azure CLI and Azure Data Factory to invoke a batch endpoint, with both…
Unable to write rows into sink from SAP CDC Connector
I am working on extracting data from SAP BW and writing it to Azure Blob Storage using an ETL pipeline. Initially, I experimented with delta/incremental loads, but now I need to perform a full load. However, after switching back, I am unable to get any…
Self Hosted Integration Studio Out of Memory errors but system has plenty of memory
Greetings - I am fairly new to supporting a portion of a Azure Data Factory. One of our servers that is running the Integration Runtime has a current version of runtime and plenty of CPU and memory available but it is throwing out this error a couple of…
How to call AzureML Batch Model endpoint from Azure Data Factory (ADF Pipeline).
Uzing Azure ML I am able to create and deploy models to a batch endpoint and run them through the UI. When I try and execute them through the Machine Learning Execute Pipeline activity in ADF I get the error "Data set node xxx references parameter…
How to make sure delete rows are also synced with Incremental column as watermark column
Hi Team We have setup sync using ADF CDC between 2 azure SQL database. We have one table "X" having a watermark column "UpdatedAt". We have created CDC for table X with incremental column "UpdatedAt" and Keys" as…
Zoho Authentication: Successful in Fabric Data Factory, Fails in Azure Data Factory"
I am trying to access https://accounts.zoho.com/oauth/v2/token to get the token for my zoho endpoint.. When using Web activity on fabric it is a cakewalk and I am getting token but when I am trying same in azure data factory I am getting an error as…
Azure Data Factory random disconnections and pipeline experienced a "ghost run" with no trigger debug/history present.
Hello, I'm experiencing a few issues with Azure Data Factory. Our team has Azure Data Factory integrated with GIT, but at random times it disconnects. It may show as connected for one team member but appear disconnected for another. This also…
How to stop ForEach activity if previous itertion fails in ADF/Synapse pipeline
I am trying to pull multiple tables from a Azure SQL Database . When data is loaded to its destination I am calling an SP that will update the status of a table to Success. Using ForEach activity multiple tables are puling. But, While pulling one table…
Lookup activity issue from Snowflake connector in Azure data factory
Hello Team, I am facing an issue from lookup activity, I have a pipeline which is having a snowflake connector in the lookup activity and trying to fetch the records from the table. but lookup activity not getting any output when we check in the…
Data factory copy operation in immutable storage fails
I have data factory with one pipeline which do copy operation from one container to another within same storage account. I have two containers "daily" and "weekly" and my pipeline copy files from daily to weekly. When new files are…
Need to extract zreports from SAP HANA
Hi, I have a use-case where I want to extract data from SAP Hana. The use-case is as follows: I have SAP Hana deployment from where I need to extract data. The data is stored in zreports which are extracted using T-Codes. Now, I want to extract the data…
Changing ADF Trigger changes Advanced Filters in Container Blob EventGrid
Hi, I am using SFTP in a Storage Container set up with an EventGrid that triggers an ADF pipeline when a new file is uploaded to the container. My event grid subscription is set to the filters values you can see in the first attached image. With the…