Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,541 questions
2 answers
Logic app to retrieve the latest file from a blob folder
How can I create a logic app that retrieves the latest file from a blob folder when a Http request is received, where there are multiple files, and sends it as an attachment? Are there any specific steps or configurations required for this process?
asked 2024-04-01T14:00:18+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 69%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Aditya Singh
160
Reputation points
commented 2025-02-19T23:07:49.69+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 49%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMB%3C/text%3E%3C/svg%3E)
Michael Brunak
0
Reputation points Microsoft Employee
0 answers
Azure Data Factory - Copy data from AWS Redshift to Azure Synapse - Failed to connect error
Hi, I am trying to copy a table of data from AWS Redshift into Azure Synapse, below is the JSON code from the data factory: { "name": "pipeline1", "properties": { "activities": [ { …
asked 2025-02-18T22:50:02.35+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(89.60000000000001, 60%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVP%3C/text%3E%3C/svg%3E)
Vinny Prajapati
0
Reputation points
commented 2025-02-19T19:52:33.1+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 61%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
phemanth
13,810
Reputation points Microsoft Vendor
0 answers
delta file Vaccuum Immediately dataflow
Hi , I am using Azure data factory Data Flow source is parquet file sink is Delta file . I wanted run the vacuum immediately keeping no history . do we have any option for that please . Sink is Delta File in the Dataflow
asked 2025-02-19T17:04:39.2466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 54%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVR%3C/text%3E%3C/svg%3E)
Vignesh Rajendran
1
Reputation point
commented 2025-02-19T19:20:29.1733333+00:00
phemanth
13,810
Reputation points Microsoft Vendor
0 answers
Azure Devops error invalid parameters
Hi, Invalid parameters error in ADF azure devops armtemplate deployment what can be issue in armtemplate { "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#", "contentVersion":…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 13%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVS%3C/text%3E%3C/svg%3E)
2 answers
One of the answers was accepted by the question author.
What is the maximum throughput speed achievable on the Azure Data factory
Hi All, I am performing the data migration from oracle (on prem) to Azure sql database using the Self hosted integration runtime. The throughput speed -2 MB/s on the Azure data factory. 135 GB takes approx. -10 hr. What is the maximum achievable…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(3.2, 68%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ER%3C/text%3E%3C/svg%3E)
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(316.8, 70%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELZ%3C/text%3E%3C/svg%3E)
1 answer
NAT Gateway for Managed VNet IR
How can I make Azure Managed VNet IR to make requests through a static public I.P? I want to pull data from 'xyz.com/books' api. This API provider doesn't allow requests from everyone. They only allow requests from IPs which are whitelisted. I have…
asked 2025-02-10T22:46:31.83+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(54.400000000000006, 49%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Amrale, Siddhesh
0
Reputation points
commented 2025-02-19T18:51:00.9366667+00:00
AnnuKumari-MSFT
34,011
Reputation points Microsoft Employee
2 answers
Azure Data Factory pipeline to extract data from Snowflake and load it into Azure Blob Storage
Hi Team, Azure Data Factory pipeline to extract data from Snowflake and load it into Azure Blob Storage. However, I’m facing some challenges with the data copy process and could use some assistance. I have several Tables in my snowflake schema. Need…
asked 2025-02-19T16:44:10.35+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(131.20000000000002, 67%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENB%3C/text%3E%3C/svg%3E)
Nalini Bhavaraju
20
Reputation points
answered 2025-02-19T18:06:08.06+00:00
Nalini Bhavaraju
20
Reputation points
0 answers
Azure Data Factory Web activity calling child pipeline across ADF instance
I have child pipeline in ADF Instance 1 which invoke rest api POST method through web activity Parent pipeline is in ADF instance 2 which call above child pipeline also through web activity. I am able to successfully call child pipeline from parent…
asked 2025-02-19T15:19:04.2833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 35%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPJ%3C/text%3E%3C/svg%3E)
Pankaj Joshi
371
Reputation points
commented 2025-02-19T17:38:35.5933333+00:00
Pankaj Joshi
371
Reputation points
0 answers
error with ssis package in azure Failed get parameter info of parameter sql password
There is an SSIS package running on a schedule in Azure. The SQL database is also in Azure. The SSIS package moves data from SQL to D365. I am getting this error when the package starts to run: error with ssis package in azure Failed get parameter…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 52%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAL%3C/text%3E%3C/svg%3E)
2 answers
One of the answers was accepted by the question author.
Need help to copy Data from Snowflake to Azure Blob Storage via ADF Pipeline
I want to create an Azure Data Factory pipeline to perform a bulk data copy from Snowflake tables to Azure Blob Storage. Since I have large tables to handle, I aim to organize the data into folders named after each table. Inside each table’s folder, the…
asked 2025-02-14T17:26:39.2166667+00:00
Nalini Bhavaraju
20
Reputation points
answered 2025-02-19T17:06:56.1766667+00:00
Nalini Bhavaraju
20
Reputation points
1 answer
Why does ADF copy activity to Oracle RDBMS configured with concurrent connections and degree of parallelism always fail?
Configuring ADF copy activity to read from Azure Blob Storage and concurrently write to Oracle database (to speed up the copy activity) fails always with ORA-00054 "Resource busy and acquire with no wait error.....". DBA says ADF had X active…
asked 2025-02-18T16:49:47.3166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 76%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAD%3C/text%3E%3C/svg%3E)
Ajay Devulapalli
0
Reputation points
commented 2025-02-19T17:06:08.8466667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
Ganesh Gurram
4,170
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
check for pipe characters in a field and use if condition on this field
In the dataflow, I am checking for data in a field that could have Pipe (|) characters or alphanumeric characters with Pipe(|) between them. If there is only pipe (|)characters, then use iif condition and mark it as Y . If there are alphanumeric…
asked 2025-02-18T17:14:18.44+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 23%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPS%3C/text%3E%3C/svg%3E)
Peruka, Sikander reddy
90
Reputation points
accepted 2025-02-19T16:47:58.78+00:00
Peruka, Sikander reddy
90
Reputation points
1 answer
Within Azure Data Factory Studio, I get a Failed Connection to my SQL Database
Below is the error I keep getting, I've read all documentation and tried all steps to ensure identification is all there. I'm stuck and have a client deliverable. I would happily pay extra to get over this hurdle as I need to use the data warehouse…
asked 2025-02-14T00:58:25.4166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(307.2, 28.000000000000004%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EDL%3C/text%3E%3C/svg%3E)
Derek Lilley
20
Reputation points
commented 2025-02-19T16:00:43.1733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 65%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECB%3C/text%3E%3C/svg%3E)
Chandra Boorla
8,470
Reputation points Microsoft Vendor
0 answers
Removed trigger still triggers
Hi! I had a trigger in Azure Data Factory with a related pipeline. The trigger has been deleted (and the change published) but it is still triggering over 12 hours later. The trigger is not in the GUI, not connected to the pipeline and not in Git.…
asked 2025-02-19T09:54:08.86+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 73%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
JE
0
Reputation points
commented 2025-02-19T15:59:46.3033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
20,295
Reputation points Microsoft Vendor
1 answer
Running MongoDB Aggregation Operation in Azure Data Factory.
Hi, I want to query my MongoDB Atlas (Not Cosmos DB) to fetch data and process it in Azure Data Factory. I have successfully created a connection to MongoDB and can run filter operations. I want to perform aggregation operations like $lookup with the…
asked 2025-02-18T05:05:42.0433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(137.6, 68%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Anish Sapkota
5
Reputation points
commented 2025-02-19T15:14:38.93+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 99%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJN%3C/text%3E%3C/svg%3E)
J N S S Kasyap
0
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
Azure data factory calling restapi hosted on app service
I am calling restapi POST method hosted on app service from azure data factory(web activity) using VNet integration runtime, I am using authentication type as NONE. but it is giving error as Error "403 Forbidden 'the web app you have attempted to…
asked 2025-02-08T05:15:33.36+00:00
Pankaj Joshi
371
Reputation points
accepted 2025-02-19T15:11:24.02+00:00
Pankaj Joshi
371
Reputation points
2 answers
ADF Power Query Sink Settings UI Missing Options
Why would my data factory not have the same UI as I see in training videos and QA answers on the Power Query data wrangling part of ADF? I can only see the delete button next to the dataset in the Power Query activity Sink…
asked 2025-01-23T15:50:49.8533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(272, 24%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EES%3C/text%3E%3C/svg%3E)
Elisa Schrader
20
Reputation points
commented 2025-02-19T14:24:07.06+00:00
AnnuKumari-MSFT
34,011
Reputation points Microsoft Employee
2 answers
One of the answers was accepted by the question author.
How can i create a linked service in ADF for Sharepoint online?
I want to extract files from sharepoint to ADL by only using ADF. I followed few steps Step1: Azure Active Directory -> Registered new app -> created new secret key I have the Tenant ID, ClientID(App ID), Secret Key Step2: Sharepoint online ->…
asked 2024-04-27T21:58:38.2233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 73%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
Siddartha Reddy Jammula
20
Reputation points
answered 2025-02-19T14:22:07.2933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 21%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELJ%3C/text%3E%3C/svg%3E)
Luis Javier Quecedo Iglesias
0
Reputation points
0 answers
Azure Data factory: Linked service parameterization default values and CI/CD pipelines
I am struggling to understand how ADF parameterization functions when deploying from one environment to another when using DevOps pipelines. Our linked services may be set up with parameters such as "name":…
asked 2025-02-18T15:07:37.1933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 84%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELA%3C/text%3E%3C/svg%3E)
Lockyer, Aaron (SGRE COG B MS SYS BSPM)
0
Reputation points
edited a comment 2025-02-19T12:46:40.8633333+00:00
Lockyer, Aaron (SGRE COG B MS SYS BSPM)
0
Reputation points
1 answer
All accountid's in the contact-table are NULL when using Azure Synapse Link or a Synapse pipeline to parquet
I've tried two ways to copy data from our Dynamics 365 to a Storage Account on Azure: Azure Synapse Analytics: creating a pipeline that connects to Dynamics 365 and copies the data the a parquet file on a storage account. I tried fetching the data in…
asked 2025-02-17T20:05:44.3333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 97%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETD%3C/text%3E%3C/svg%3E)
Tim Derdelinckx
0
Reputation points
commented 2025-02-19T12:42:08.14+00:00
Chandra Boorla
8,470
Reputation points Microsoft Vendor