Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,466 questions
1 answer
Issue with copy data from Azure Synapse Link for Dataverse using ADF Dataflow
Hi there, We use Export to Data Lake option to copy the data from D365 and want to transform to Azure Synapse Link for Dataverse because Export to Data Lake option is obsoleting from 1st Nov 2024. We have configured Azure Synapse link for Dataverse in…
asked 2024-07-01T11:07:06.5533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 96%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERI%3C/text%3E%3C/svg%3E)
Raheel Islam
21
Reputation points
commented 2024-08-13T06:03:59.09+00:00
AnnuKumari-MSFT
32,821
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
ADLS storage cost for premium & standard tier
Currently we save about 400GB of data (expect to grow to 2TB on production) in ADLS and select the standard hot tier service. Our monthly bill shows that most of the storage cost (96%) are for the data transaction (iterative read, regular read/write,…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(262.40000000000003, 27%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPJ%3C/text%3E%3C/svg%3E)
2 answers
How to connect Business Central to Azure Data Factory
I posted this question on Re: [MicrosoftDocs/azure-docs] Business Central Connector (Issue #88574) But that didn’t help. Is it actually possible to connect Business Central to ADF?
asked 2022-02-26T12:37:04.963+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(128, 8%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Anonymous
commented 2024-08-12T13:44:57.09+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(41.6, 18%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAH%3C/text%3E%3C/svg%3E)
Andrew Hairston Cloud
0
Reputation points
1 answer
One of the answers was accepted by the question author.
Ingress and Egress cost in azure
I have gone through below thread: https://learn.microsoft.com/en-us/answers/questions/892660/will-the-ingress-egress-costs-of-a-azure-storage-a but still confused and clear about the ingress and egress costs of ADLS, and what parameters it takes to…
asked 2024-08-08T15:31:30.9766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 88%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
azure_learner
240
Reputation points
accepted 2024-08-10T13:02:20.5066667+00:00
azure_learner
240
Reputation points
1 answer
Why the size of Data Read in ADF copy activity is actually much bigger than the original size of source data?
I have initiated a data transfer operation from Google Cloud Storage to Azure Data Lake Storage within Azure Data Factory. The objective was to transfer approximately 3,000 files in snappy.parquet format, with a combined size of approximately 30 GB. Upon…
asked 2023-11-01T11:02:04.88+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(265.6, 56.00000000000001%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETT%3C/text%3E%3C/svg%3E)
Thatchapoom Thumsawatra
10
Reputation points
commented 2024-08-09T21:34:18.1+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 43%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBR%3C/text%3E%3C/svg%3E)
Brandon Roberts
0
Reputation points
7 answers
Error trying to validate storage account name while creating a new synapse workspace
I am unable to create a new data lake storage (gen2) account name in the basic tab for create Azure Synapse Workspace. The error I get is - " There was an error trying to validate storage account name. please try again ". The error message is…
asked 2022-08-03T07:01:54.587+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(252.8, 2%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
Jozure
526
Reputation points
commented 2024-08-09T17:46:54.03+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(249.60000000000002, 94%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVY%3C/text%3E%3C/svg%3E)
Varun Yadav
0
Reputation points
2 answers
Missing Override ARM Template paramater for secret name to connect to ADLS Stoarge
Hi, I have a linked service to connect to development environment of azure data lake storage account. the connection to the storage account is using Azure key vault. while doing the deployment from DEV to UAT, the ARMTemplate has the URL string…
asked 2022-06-27T02:24:02.07+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(265.6, 0%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESG%3C/text%3E%3C/svg%3E)
sachin gupta
376
Reputation points
commented 2024-08-08T18:51:45.9533333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 40%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECG%3C/text%3E%3C/svg%3E)
Charlie Gordon
5
Reputation points
1 answer
Unable to rename Blob object without adding my public ip
I have a storage account adls gen2 for my Synapse workspace, It's public access is disabled and can be accessible from only selected virtual networks and IP addresses. I have configured a private endpoint. Using VPN I'm able to access all blobs, but I'm…
asked 2023-09-20T16:21:12.0033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(288, 36%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESB%3C/text%3E%3C/svg%3E)
Sudarshan Bhamare
60
Reputation points
commented 2024-08-08T17:21:37.3833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(115.19999999999999, 6%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJM%3C/text%3E%3C/svg%3E)
Jason McReynolds
20
Reputation points
2 answers
One of the answers was accepted by the question author.
Blob to storage account
Hi , I have to copy 550 gb of data from blob storage to ADLS gen2 . I have three option CLI , AZcopy and ADF. But over 500 gb of data copy by ADF has performance issues. Please share your suggestion.
asked 2024-08-07T08:11:52.7733333+00:00
azure_learner
240
Reputation points
accepted 2024-08-08T08:53:47.0866667+00:00
azure_learner
240
Reputation points
2 answers
SAS token generation by Databricks to access CSV files from ADLS container folder
Hi Team, There are some csv files zips inside the ADLS container folder. These zip files need to be downloaded for data correction. Downloading the file requires SAS token embedded with zip file path. Databricks has been used to generate the token and…
asked 2024-07-17T05:08:24.3033333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(300.8, 91%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
Subhadip Roy
26
Reputation points
commented 2024-08-08T07:45:26.89+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 64%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENR%3C/text%3E%3C/svg%3E)
Nehruji R
7,556
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
ADF pipeline to read the data from UC table to adls gen2 account
Hello Team, We have a requirement to create Azure Datafactory pipeline to read the data from UC table, access on the table is provided ( to Azure Datafactory Managed Identity) and copy the data into adls gen2. Is there a way or article to implement this?…
asked 2024-04-11T19:05:05.9733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(51.2, 77%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAG%3C/text%3E%3C/svg%3E)
Ashwini Gaikwad
130
Reputation points
commented 2024-08-07T12:35:06.2766667+00:00
Kuldeep Singh
16
Reputation points
1 answer
One of the answers was accepted by the question author.
Inquiry Regarding the Future of Azure Data Lake Gen2
Hello, For nearly a year, we have been utilizing Azure Data Lake Gen2 to extract data into SQL using Python code. I would like to inquire about the future of Azure Data Lake Storage Gen2: Will Azure Data Lake Storage Gen2 be retired in the near…
asked 2024-08-07T06:27:48.6633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 33%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMA%3C/text%3E%3C/svg%3E)
Ömer Şahin
20
Reputation points
commented 2024-08-07T10:34:09.6966667+00:00
TP
90,636
Reputation points
2 answers
One of the answers was accepted by the question author.
DataSourceError from PowerBI when trying to access Synapse Serverless View on ADLS2
We have an issue with a particular user connecting to our Serverless SQL pool via PowerBI. I created an Azure Synapse workspace, which contains the built in serverless SQL pool which has some Views set up to pull data from some Delta tables. As the…
asked 2024-07-31T07:58:19.4166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(176, 15%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHC%3C/text%3E%3C/svg%3E)
Hobday, Colin
40
Reputation points
commented 2024-08-06T10:35:46.3933333+00:00
PRADEEPCHEEKATLA-MSFT
88,876
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
How to fix the problem : 'EndpointUnsupportedAccountFeatures'. Message: 'This endpoint does not support BlobStorageEvents or SoftDelete
I am getting this error while using the sink option in data factory The folder I am trying in path is from storage account.
asked 2024-08-02T18:58:13.3733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(9.6, 47%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKT%3C/text%3E%3C/svg%3E)
Karan Tuteja
40
Reputation points
accepted 2024-08-04T16:16:47.5733333+00:00
Karan Tuteja
40
Reputation points
1 answer
HttpRequestFailedWithUnauthorizedError output after Copy Data function when extract data from Sharepoint to adl using data factory
I followed the tutorial from "https://www.youtube.com/watch?v=FFfNu3cI-uw&pp=ygUjZXh0cmFjdCBkYXRhIGZyb20gc2hhcmVwb2ludCB0byBhZGw%3D" but this error occurred. "errors": [ { "Code": 22757, "Message":…
asked 2024-07-29T06:13:18.0233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 74%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EXL%3C/text%3E%3C/svg%3E)
Xing Le Lee
0
Reputation points
commented 2024-07-31T15:30:15.59+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 8%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EST%3C/text%3E%3C/svg%3E)
Smaran Thoomu
15,040
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Connecting PowerApps to Data Lake Storage tables
Is it possible to connect PowerApps to Data Lake Storage tables in order to create, edit, and delete data within the tables? If so, what are the prerequisites and how can it be achieved? Any guidance or resources would be greatly appreciated.
asked 2024-07-29T07:36:27.9233333+00:00
paromita das
20
Reputation points
commented 2024-07-30T10:24:35.8233333+00:00
Nehruji R
7,556
Reputation points Microsoft Vendor
1 answer
Creating Items in Datalake creates a additional 0kb file with folder name or filename
We have created two Gen 2 storage account. Creating an item in one data lake container creates a 0 KB file with the same name as that of the file or folder. Attached the screen shot of the container items with the additional file that is created and…
asked 2021-03-23T05:43:06.1+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(240, 96%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVN%3C/text%3E%3C/svg%3E)
Varun Nair
1
Reputation point
commented 2024-07-29T03:07:10.07+00:00
Ramon Eduardo Aldana Borjes
0
Reputation points
2 answers
Azure datalake creation methods
Hi friends, we need to create a datalake (ADLS) with bronze, silver, and gold layers. One of our colleagues suggested it is a better idea to create ADLS through programming, but we plan to create it manually. Are there any advantages of each approach …
asked 2024-07-24T10:24:14.9566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 15%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Anshal
2,246
Reputation points
answered 2024-07-26T18:48:18.1633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(118.4, 98%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKM%3C/text%3E%3C/svg%3E)
KarishmaTiwari-MSFT
19,952
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
ADLS Gen2 Query Acceleration
Hello, As a follow up to this thread: Query acceleration using parquet does not work with double fields - Microsoft Q&A I would like to know if and when Microsoft plans to enable the query acceleration feature for Parquet files as well for ADLS Gen2…
asked 2024-07-17T05:15:46.93+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(76.8, 99%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ED7%3C/text%3E%3C/svg%3E)
DanielMeierotto-7637
20
Reputation points
accepted 2024-07-24T09:27:59.3333333+00:00
DanielMeierotto-7637
20
Reputation points
2 answers
Getting the size of parquet files from azure blob storage
I have a blob container abcd The folder structure is like below: abcd/Folder1/Folder a, Folder b…..Folder z Inside a particular Folder a/v1/full/20230505/part12344.parquet Similarly Folder b/v1/full/20230505/part9385795.parquet Scenario is I need to get…
asked 2024-07-09T11:57:50.94+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(22.400000000000002, 89%, 12%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKJ%3C/text%3E%3C/svg%3E)
KEERTHANA JAYADEVAN
66
Reputation points
commented 2024-07-24T08:31:27.1433333+00:00
Nehruji R
7,556
Reputation points Microsoft Vendor