Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,466 questions
1 answer
One of the answers was accepted by the question author.
SQL Server Integration Services and Azure Data Lake Store Connection Manager
Is it possible a SQL Server Integration Services (SSIS) package can use the Azure Data Lake Store Connection Manager to connect to an Azure Data Lake Storage Gen2 ?? From what I read the article below refers only for Gen 1 but and about the Gen 2? Link…
asked 2024-09-18T14:52:20.88+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(0, 74%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELB%3C/text%3E%3C/svg%3E)
Leonardo Bruno Pereira de Araujo
0
Reputation points
commented 2024-09-19T11:29:24.41+00:00
Leonardo Bruno Pereira de Araujo
0
Reputation points
0 answers
Azure data lake folder structure
Hi, the link provided in the below thread is not working, any way to find the information or a new URL/link: FAQs About Organizing a Data…
asked 2024-09-19T08:43:49.7+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 88%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
azure_learner
240
Reputation points
commented 2024-09-19T09:49:49.6333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 64%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENR%3C/text%3E%3C/svg%3E)
Nehruji R
7,556
Reputation points Microsoft Vendor
2 answers
Unable to connect the ADLS to snowflake
I am not able to connect the ADLS to snowflake. I requested azure to create the connection but I don't see the user that starts with Snowflake... when I create a role in IAM.
asked 2024-09-05T18:47:27.9966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(198.4, 13%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPS%3C/text%3E%3C/svg%3E)
Purnima Sharma
0
Reputation points
commented 2024-09-19T08:09:02.9566667+00:00
AnnuKumari-MSFT
32,821
Reputation points Microsoft Employee
2 answers
Copy Activity very slow for Azure data factory
Hi All, we are having issue where copy activity from Azure Table storage to data factory is taking over 50mins on daily basis. it used to take 15-20mins until a week back. What's the source of the slow copy activity? Azure Table storage What's the sink…
asked 2024-09-13T03:50:38.0833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(310.4, 74%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMS%3C/text%3E%3C/svg%3E)
Mayuri Shah
0
Reputation points
commented 2024-09-19T08:03:54.6633333+00:00
AnnuKumari-MSFT
32,821
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
How to overwrite a table from SQL Server to SQL Dedicated Pool using Copy Data Activity in Azure Synapse Analytics
I am trying to use the Copy Data Activity in Azure Synapse Analytics to copy data from SQL Server to SQL Dedicated Pool. However, I want to ensure that every time I run the Copy Data Activity, the table in SQL Dedicated Pool is overwritten. How can I…
asked 2024-09-18T08:58:39.26+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 69%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Aditya Singh
135
Reputation points
accepted 2024-09-19T05:17:18.4233333+00:00
Aditya Singh
135
Reputation points
2 answers
can abac be applied to a data lake?
Can abac be applied to a data lake storage account or does it have to be rbac with potential ACL?
asked 2024-09-13T04:26:59.0133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 22%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EML%3C/text%3E%3C/svg%3E)
Michael Liveri
0
Reputation points
commented 2024-09-18T06:57:44.8566667+00:00
Nehruji R
7,556
Reputation points Microsoft Vendor
0 answers
Slow performance reading parquet files from Data Lake Gen2
I'm using Azure Data Lake Gen2 to store data in parquet file format. I have split the data using partitions by year, month and day to benefit from the filtering functionality. I'm reading the data from python directly (no Spark or Synapse here) using the…
asked 2024-02-15T14:36:23.51+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(256, 30%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJL%3C/text%3E%3C/svg%3E)
Jorge Lopez
36
Reputation points
edited the question 2024-09-17T17:55:48.4433333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 76%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EV%3C/text%3E%3C/svg%3E)
VarunTha
7,810
Reputation points Microsoft Vendor
0 answers
I got this error when debug my copy data on azure df 7cf2da46-b65b-4098-b8ec-d9c7858c190e
after connecting my Azure data factory to MsSqL, I tried to copy my data into azure data factory and I got this error 7cf2da46-b65b-4098-b8ec-d9c7858c190e
asked 2024-09-13T14:44:21.2833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 72%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECO%3C/text%3E%3C/svg%3E)
Charles Onyebuchi
20
Reputation points
commented 2024-09-17T15:42:37.1166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(227.2, 46%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENN%3C/text%3E%3C/svg%3E)
NIKHILA NETHIKUNTA
1,920
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Need to use service principal in datalake connection string instead of storage key
Hello Team, I hope you are doing well. Currently in function app customer using connection string which is consist of storage key but now he wants to use service principal in connection string please let me know how to format connection string: current…
asked 2024-09-14T07:13:44.0966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 72%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERB%3C/text%3E%3C/svg%3E)
Rahul Balayya (LTIMINDTREE LIMITED)
60
Reputation points Microsoft Vendor
commented 2024-09-17T05:23:51.04+00:00
Vinodh247
18,906
Reputation points
2 answers
One of the answers was accepted by the question author.
Can't create a datalake storage account Gen2 when creating an azure synapse resource
When trying to create a new data lake gen2 storage account (while creating a azure Synapse resource), I keep on getting an error message that the account name can't be verified and that I need to try again. I've waited for a couple of hours and have…
asked 2023-11-16T17:15:29.1633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(118.4, 46%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EUF%3C/text%3E%3C/svg%3E)
u512353 (Fuentes Alfaro, Patricia)
25
Reputation points
commented 2024-09-16T21:17:31.1166667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(249.60000000000002, 64%, 34%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECC%3C/text%3E%3C/svg%3E)
Charles Chinedu Dike
0
Reputation points
2 answers
One of the answers was accepted by the question author.
How to improve speed of data transfer of RO_MOUNT Data asset in Azure ML Job?
Is there a reason for this slow transfer speed during job run? This slowdown is causing GPU utilization to tank from 85-90% (if files are on disk during test runs) to 12-15% during actual job training with complete data being transferred through…
asked 2024-09-03T11:52:26.11+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(70.4, 20%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERK%3C/text%3E%3C/svg%3E)
Rahul Kurian Jacob
20
Reputation points
edited an answer 2024-09-16T11:50:14.7866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 61%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
santoshkc
8,100
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Dealing with sensitive data in azure
I am ingesting data RaaS and APIs for other source systems, which must be accessed through API and HTTPS or HTTP. ADF does not encrypt the data, and data masking using a dataflow for huge amounts of data is hard. We are dealing with a lot of sensitive…
asked 2024-09-04T13:31:44.7666667+00:00
azure_learner
240
Reputation points
accepted 2024-09-13T16:42:55.7966667+00:00
azure_learner
240
Reputation points
1 answer
One of the answers was accepted by the question author.
Lost access to data storage containers
I created a storage account last week verticaltbr. I have created 2 storage containers and uploaded the contents. But today I can't access these contents on Azure portal and Storage Explorer. I can, on the other hand, create new containers but can't…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(307.2, 10%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAY%3C/text%3E%3C/svg%3E)
1 answer
Azure storage Adls blob container set read only
I have storage account with hierarchical namespace enabled. Now I have container "test" where I have uploaded one file, I need this file to be read only so that no one can update it. Now problem is "test" container inherit rbac role…
asked 2024-09-04T14:58:15.73+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(92.8, 35%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPJ%3C/text%3E%3C/svg%3E)
Pankaj Joshi
331
Reputation points
commented 2024-09-11T07:17:54.84+00:00
Pankaj Joshi
331
Reputation points
2 answers
Azure data factory and secure access
What are the secure options available to prevent ADF and ADLS from cyber threats and other security vulnerabilities besides accessing the services through private link and express routes and using vnet and private endpoints? Please suggest a holistic…
asked 2024-08-30T10:58:08.7566667+00:00
azure_learner
240
Reputation points
commented 2024-09-09T07:34:03.2166667+00:00
Nehruji R
7,556
Reputation points Microsoft Vendor
1 answer
Got error code 403 when try to access my Synapse workspace created within a virtual network
I got a 403 error when access my Synapse Workspace created in a managed virtual network within a jumphost inside my virtual network. This is the network settings of my Synapse Analytics workspace My bicep files have help me set up some private…
asked 2024-08-23T02:49:17.8333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 12%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKW%3C/text%3E%3C/svg%3E)
Kenny Wong (HK)
60
Reputation points
commented 2024-09-09T06:40:12.4466667+00:00
Nehruji R
7,556
Reputation points Microsoft Vendor
1 answer
Cost of ADLS Gen2 while enabling Az synpase link for Cosmos database
Hi Team, I am exploring the Azure synapse link for Azure cosmos database, but when I enable the Azure synapse link then it require the azure synapse workspace to be created along with Azure data lake storage Gen2. So, below are my queries around it. …
asked 2024-08-26T06:43:39.3933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(3.2, 77%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJS%3C/text%3E%3C/svg%3E)
Jaydeep Suryawanshi
26
Reputation points
edited a comment 2024-09-04T10:34:21.6466667+00:00
Nehruji R
7,556
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Data is missing from management API using service principle. ex: Advisor Recommendations
I have a requirement to load datasets from management.Azure.com (Management API), for example: Advisor Recommendations, Resources I have created a service principal in app registration and have been able to obtain the access token. However, when loading…
asked 2024-09-02T13:26:14.2+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 35%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMR%3C/text%3E%3C/svg%3E)
Mahesh Reddy
20
Reputation points
commented 2024-09-03T09:54:06.36+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 31%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVK%3C/text%3E%3C/svg%3E)
Vinod Kumar Reddy Chilupuri
155
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Restore ADLS Gen2 (HNS) Account / Containers
Hello, I am using an ADLS Gen2 / HNS storage account or data lake. I have enabled blob versioning and soft delete. That is helpful to recover deleted blobs. However I want to selectively restore my storage account as well as container to a point in…
asked 2024-08-29T11:43:56.1+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(118.4, 63%, 21%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP3%3C/text%3E%3C/svg%3E)
ParthibanSekarUK-3136
20
Reputation points
accepted 2024-09-02T08:19:17.0133333+00:00
ParthibanSekarUK-3136
20
Reputation points
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(73.60000000000001, 43%, 17%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EG%3C/text%3E%3C/svg%3E)