Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,466 questions
1 answer
One of the answers was accepted by the question author.
How can i create a linked service in ADF for Sharepoint online?
I want to extract files from sharepoint to ADL by only using ADF. I followed few steps Step1: Azure Active Directory -> Registered new app -> created new secret key I have the Tenant ID, ClientID(App ID), Secret Key Step2: Sharepoint online ->…
asked 2024-04-27T21:58:38.2233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 73%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESR%3C/text%3E%3C/svg%3E)
Siddartha Reddy Jammula
20
Reputation points
commented 2024-08-30T14:11:19.83+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(0, 57.00000000000001%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJW%3C/text%3E%3C/svg%3E)
Jan Wojciechowski
20
Reputation points
2 answers
One of the answers was accepted by the question author.
US government regulations with azure
I am looking for detailed documentation on ( Family Educational Rights and Privacy Act) FERPA and I am looking for useful information. Still, I could not get a detailed implementation of FERPA with various Azure services such as ADLS, Azure Databricks,…
asked 2024-08-27T16:08:32.4133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 88%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
azure_learner
240
Reputation points
commented 2024-08-29T11:14:12.34+00:00
azure_learner
240
Reputation points
2 answers
One of the answers was accepted by the question author.
Terabytes of data to Azure
I am required to ingest 50+TB of data from a learning system with all types of structured, unstructured, and semi-structured data. Since the data volume is large, I am not sure the methods below are the right tools to ingest 50+TB of data into ADLS …
asked 2024-08-08T09:08:13.68+00:00
azure_learner
240
Reputation points
answered 2024-08-27T18:59:38.8933333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(310.4, 92%, 40%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Alex Stevens
0
Reputation points
1 answer
One of the answers was accepted by the question author.
API source data and intermediatory storage
I am moving data from a highly secure system with compliance as report as service (RaaS) as I discussed in the questions below.…
asked 2024-08-22T04:04:42.67+00:00
azure_learner
240
Reputation points
accepted 2024-08-27T16:02:40.9733333+00:00
azure_learner
240
Reputation points
1 answer
One of the answers was accepted by the question author.
can we add multiple containers in Microsoft Fabric Data Lake house?
I want to add one more container in Microsoft fabric lake house, as per my knowledge we can add only one container but i am curious to know can we add multiple containers in Microsoft fabric lake house
asked 2024-08-26T09:48:53.9066667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(227.2, 92%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGS%3C/text%3E%3C/svg%3E)
Gnana Sai krishna
80
Reputation points
accepted 2024-08-27T16:00:34.8633333+00:00
Gnana Sai krishna
80
Reputation points
2 answers
Setting different permissions for SAS Token on ADLS Gen2
Hello, I'm interested to know if it's possible to set two different sets of permissions for an access policy on ADLS Gen2 Storage - one at the container level, and another at the folder level. I'm looking for a setup like this: Access Policy setting…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(156.8, 74%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EP%3C/text%3E%3C/svg%3E)
3 answers
One of the answers was accepted by the question author.
Multiple Storage Accounts for a Data lake
All, We are thinking of implementing the Zones using Storage Accounts. Which means, each of the 3 Zones (RAW, Staged, Curated) will have one Storage Account each. Are there any best practice document? Is it normal to have more than one Storage…
asked 2022-02-08T02:56:18.287+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(0, 24%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGR%3C/text%3E%3C/svg%3E)
Gopinath Rajee
651
Reputation points
answered 2024-08-26T09:26:23.97+00:00
Gnana Sai krishna
80
Reputation points
1 answer
How to get parquet file data from data lake gen2 in tabular form using postman or web api.
i have a file in my blob storage gen2 container i want to fetch this file data by using api or postman in the tabular form by using query without azure synapse, azure sql and databricks.
asked 2024-08-15T17:34:42.1366667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(19.2, 23%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERG%3C/text%3E%3C/svg%3E)
randeep gurjar
0
Reputation points
commented 2024-08-26T04:43:51.6633333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(150.4, 64%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENR%3C/text%3E%3C/svg%3E)
Nehruji R
7,556
Reputation points Microsoft Vendor
1 answer
Can't connect to Microsoft Search Graph Connector - ADLS Gen2
We are trying to add a new “ADLS Gen2” Connector in Microsoft Search and we are receiving this error below when testing the connection. We are sure that we provided the correct Connection String from access keys within our storage account in Azure. We’ve…
asked 2021-07-01T04:59:47.21+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(208, 38%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
AJGT
21
Reputation points
commented 2024-08-25T15:16:52.49+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(60.8, 69%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJC%3C/text%3E%3C/svg%3E)
John Chang
0
Reputation points
1 answer
One of the answers was accepted by the question author.
Read Delta Tables from ADLS Gen2 in Power BI: Exploring Direct Access Methods
As part of our data pipeline, I'm using Azure Blob Storage as the source, Azure Data Factory (ADF) for processing, and Azure Data Lake Storage Gen2 (ADLS Gen2) as the destination. Here's the process I've followed: Step 1: I read a CSV file from Azure…
asked 2024-08-23T13:46:12.77+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 16%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAP%3C/text%3E%3C/svg%3E)
Akshay Patel
45
Reputation points
accepted 2024-08-25T00:51:02.23+00:00
Akshay Patel
45
Reputation points
2 answers
One of the answers was accepted by the question author.
Data size of databricks delta tables
It has been observed that the size of delta tables are much less as compared to when checked the underlying delta files in the storage account. Suppose a databricks delta table raw.deltaTableA has size of 2MB if we check the size of underlying delta…
asked 2024-05-02T09:39:01.4133333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(230.39999999999998, 23%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENK%3C/text%3E%3C/svg%3E)
NIKHIL KUMAR
101
Reputation points
answered 2024-08-23T07:05:10.3+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(140.8, 64%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAC%3C/text%3E%3C/svg%3E)
Anupam Chand／ＳＰ／JRI (anupam chand)
0
Reputation points
1 answer
Replicating a Pipeline Multiple Times in Azure Synapse Analytics
How can I replicate a pipeline multiple times in Azure Synapse Analytics? I need to replicate my pipeline 10 times with only a change to the table name which is passed as a parameter. Thanks in advance!
asked 2024-08-16T09:26:31.9833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 69%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAS%3C/text%3E%3C/svg%3E)
Aditya Singh
135
Reputation points
commented 2024-08-21T16:35:32.7966667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(227.2, 46%, 32%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENN%3C/text%3E%3C/svg%3E)
NIKHILA NETHIKUNTA
1,920
Reputation points Microsoft Vendor
1 answer
Storage Datalake allows upload, but a read causes a 401
I'm following the documentation here: https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-directory-file-acl-javascript Have verified, triple verified the account name and the account key. Uploading a file works: async function…
asked 2024-08-15T17:02:01.67+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 54%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EEI%3C/text%3E%3C/svg%3E)
Eugen Istoc
0
Reputation points
commented 2024-08-20T09:31:22.07+00:00
Nehruji R
7,556
Reputation points Microsoft Vendor
1 answer
How to access ADLS Gen2 storage with restricted network access from PySpark notebook
I'm trying to access ADLS Gen2 storage from PySpark notebook in my Synapse workspace. The storage account has public network access enabled from selected virtual networks and IP addresses, and my IP address is added to the firewall rules, trusted Azure…
asked 2024-08-15T14:06:01.63+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(6.4, 32%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EUS%3C/text%3E%3C/svg%3E)
Uros Stojiljkovic
0
Reputation points Microsoft Employee
commented 2024-08-20T09:30:51.1266667+00:00
Nehruji R
7,556
Reputation points Microsoft Vendor
3 answers
How to send a mail notification for a failed pipeline in Azure Synapse Analytics?
How can I send a notification email to a specific email address without using a logic app when one of my Synapse Analytics pipelines fails? I would like to include the error message in the email notification.
asked 2024-03-20T15:48:58.53+00:00
Aditya Singh
135
Reputation points
edited a comment 2024-08-19T15:43:05.72+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(131.20000000000002, 68%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAB%3C/text%3E%3C/svg%3E)
Asger Biehl
0
Reputation points
1 answer
Business Central Connection with Azure Data Factory
Hello, is there a way to connect Business Central to Azure Data Lake? I am only seeing one for NAV/AX and D365 CRM Apps.
asked 2024-08-07T19:38:39.42+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(41.6, 18%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAH%3C/text%3E%3C/svg%3E)
Andrew Hairston Cloud
0
Reputation points
commented 2024-08-16T00:02:21.19+00:00
NIKHILA NETHIKUNTA
1,920
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
Cannot turn on Updata and Process on Premium storage SAS
I create a premium storage account and try to add Shared access signature. In the allowed permissions, I cannot turn on "Update" and "Process" selection. See Screenshot 2024-08-13 at 2.48.42 PM.jpg. But I have no problem to turn…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(262.40000000000003, 27%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPJ%3C/text%3E%3C/svg%3E)
2 answers
One of the answers was accepted by the question author.
pre-data validation in Azure
When the data is CSV or database we could do pre-data load validation such as number of rows, size of the tables and no of the table, etc but when the source is given as report as data and consumed by RaaS through API, how the pre-data load validation…
asked 2024-08-09T06:57:26.85+00:00
azure_learner
240
Reputation points
accepted 2024-08-14T11:54:09.8933333+00:00
azure_learner
240
Reputation points
2 answers
One of the answers was accepted by the question author.
Datalake Gen2 Design and understanding
I need lots of clarity on the design aspects of Azure Data Lake Gen2. I will be grateful for your help. 1. How to determine latency requirements, and where and what I should look at? 2. How to know the transaction patterns on analytics…
asked 2024-08-12T06:49:48.4266667+00:00
azure_learner
240
Reputation points
accepted 2024-08-14T11:53:06.6633333+00:00
azure_learner
240
Reputation points
3 answers
One of the answers was accepted by the question author.
ADLS Gen2 Access Logs via Diagnostic Settings
Hi, I was wondering if there is any way to enable access logs like read, write, delete on Azure Data Lake Storage Gen2 (ADLS Gen2). For Azure Blob Storage we achieved this via Diagnostic Settings, but to me it looks like this doesn't cover any operations…
asked 2024-07-17T15:08:17.9566667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(163.2, 89%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECA%3C/text%3E%3C/svg%3E)
Cedric Ahlers
35
Reputation points
accepted 2024-08-13T07:44:07.5466667+00:00
Cedric Ahlers
35
Reputation points