Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,527 questions
1 answer
Stream RealTime data from an On Prem Microsoft Server to a Azure Data Lake.
There is a need for Real Time data to be collected from a On Prem SQL Server. This means ADF is not suitable because it only does batch. Azure Streaming Analytics only connects to Azure SQL Server and not on prem. Has anybody else encountered this and…
asked 2023-03-16T02:02:54.12+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 50%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESW%3C/text%3E%3C/svg%3E)
Sallie Worsley
5
Reputation points
commented 2025-01-17T23:10:26.6+00:00
Sander van de Velde | MVP
34,281
Reputation points MVP
0 answers
Unable to create a directory named space character using ADLS Gen2 REST API
I am unable to create a directory in "Azure Data Lake Storage Gen2" that is named as just a single space character, despite reviewing the documentation and there being no indication this is a disallowed name. My primary access to "Azure…
asked 2025-01-16T15:07:47.1866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 19%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPR%3C/text%3E%3C/svg%3E)
Phil R
0
Reputation points
commented 2025-01-17T19:09:01.51+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(304, 6%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKD%3C/text%3E%3C/svg%3E)
Keshavulu Dasari
2,910
Reputation points Microsoft Vendor
2 answers
ADF/ Synapse Activity times out after 12 hours
Hello, I have a Notebook activity in one of my pipelines that is expected to run longer than 12 hours. Below we configured the timeout to the max value, 7 days. But the activity would still timeout after 12 hours. Is there a way for the activity…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(67.2, 28.000000000000004%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
2 answers
Why are access tier options greyed out for Azure Data Lake Gen 2 storage?
When attempting to change the access tier of a file stored in Azure Data Lake Gen 2, all the access tier options are greyed out. What could be causing this issue and what are the possible solutions?
asked 2025-01-13T16:45:58.49+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 61%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESK%3C/text%3E%3C/svg%3E)
Samuel Kwesiga Kisitu
0
Reputation points
commented 2025-01-17T06:26:52.91+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(265.6, 2%, 35%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHB%3C/text%3E%3C/svg%3E)
Hari Babu Vattepally
1,450
Reputation points Microsoft Vendor
1 answer
generate SAS tokens from serviceprincipal credentials
I am working on to create java client that generates sas tokens for the given service principal credentials. I am taking a reference from…
asked 2025-01-08T12:00:29.36+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(169.60000000000002, 78%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESG%3C/text%3E%3C/svg%3E)
SHYAMALA GOWRI
90
Reputation points
commented 2025-01-17T06:25:42.78+00:00
Hari Babu Vattepally
1,450
Reputation points Microsoft Vendor
1 answer
D365 Outbound Marketing Data sync to AdLS Gen2 cannot be synced inside a folder structure but only on conatiner.
I wanted to copy Market Outbound Data from CRM like email sent, delivered, Email clicked, Email Opened , etc from the marketing campaign to ADLS Gen 2 using Data Factory but seems like I cannot use ADF to do that as they are calculated live. Also, I…
asked 2025-01-13T03:18:27.1333333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(32, 35%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERC%3C/text%3E%3C/svg%3E)
Rabin Chaudhary
1
Reputation point
commented 2025-01-17T02:21:53.41+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(57.599999999999994, 31%, 15%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EVK%3C/text%3E%3C/svg%3E)
Vinod Kumar Reddy Chilupuri
2,305
Reputation points Microsoft Vendor
1 answer
How to fix nested split issue in Azure data factory Pipeline expression builder?
Hi there, I'm trying to make use of hierarchical namespaces for Azure data lake storage gen2. When I'm dynamically setting the file structure in the Pipeline expression builder in the ADF, the following error is returned: Cannot fit unknown into the…
asked 2025-01-07T23:56:02.7266667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(112.00000000000001, 30%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EZS%3C/text%3E%3C/svg%3E)
Zohaib Sheraz Mughal
15
Reputation points
commented 2025-01-16T18:29:02.8+00:00
Keshavulu Dasari
2,910
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
How to Automate Large .SAV File to Parquet Conversion in Azure?
I'm using Azure ADLS as our primary storage, Azure Data Factory (ADF) for data transformations, and Power BI for reporting and visualization. I have a large .SAV file (200-300 MB, containing 2-4 million rows) stored in Azure Data Lake Storage (ADLS). To…
asked 2024-12-27T10:31:15.69+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(204.8, 16%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAP%3C/text%3E%3C/svg%3E)
Akshay Patel
90
Reputation points
accepted 2025-01-16T10:02:14.32+00:00
Akshay Patel
90
Reputation points
0 answers
How to write the query on Azure to get the information of particular SPN linked with all the folders present in containers..
Hi, As there are multiple folders are preset into Azure storage containers, each folders of containers linked with some SPN. I am looking for the script to create to find, which SPN associate with which all folders?
asked 2025-01-13T14:36:48.55+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(105.60000000000001, 92%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESP%3C/text%3E%3C/svg%3E)
Saurabh Parmar
0
Reputation points
commented 2025-01-16T07:04:52.8+00:00
Nandamuri Pranay Teja
325
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Read AppServiceConsoleLogs and push to Blob Storage container/table
Hello Team, we have a webapp python Django based. We are generating custom logs for user actions using code below. Settings.py: LOGGING = { "version": 1, "disable_existing_loggers": False, "formatters": { …
asked 2024-11-20T09:31:14.9233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(272, 35%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
BalaKrishna Mahamkali
20
Reputation points
accepted 2025-01-16T04:32:25.14+00:00
BalaKrishna Mahamkali
20
Reputation points
1 answer
One of the answers was accepted by the question author.
Change the Azure synapse workspace default storage account.
Hi Team, We created an Azure Synapse workspace with a default storage account associated with it. However, due to various reasons, we are not using the storage account tied to the Synapse workspace. Instead, we have created another ADLS Gen2 storage…
asked 2025-01-13T22:36:47.74+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 97%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESP%3C/text%3E%3C/svg%3E)
Sai Pranavi Unnam
20
Reputation points Microsoft Employee
accepted 2025-01-14T20:47:55.6766667+00:00
Sai Pranavi Unnam
20
Reputation points Microsoft Employee
2 answers
how to fix this error The request content was invalid and could not be deserialized: Required property 'name' not found in 'sku'
getting error while synapse workspace deployment
asked 2024-12-09T14:46:47.8+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(294.40000000000003, 53%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ENS%3C/text%3E%3C/svg%3E)
Narendra Singh
15
Reputation points
commented 2025-01-11T03:44:00.18+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(83.2, 83%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMB%3C/text%3E%3C/svg%3E)
milan barot
0
Reputation points
1 answer
One of the answers was accepted by the question author.
Preview features like AllowDataSharingInHeroRegion and AllowDataSharing are in pending status since a month
Hi team, Azure subscription preview features are like AllowDataSharingInHeroRegion and AllowDataSharing are in pending status since a month. We tried to unregister and re-register but still no luck. Could you please help us on this? Thanks, and…
asked 2025-01-03T19:40:21.0733333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(131.20000000000002, 27%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECK%3C/text%3E%3C/svg%3E)
chaitanya.K
20
Reputation points
accepted 2025-01-11T00:30:14.16+00:00
chaitanya.K
20
Reputation points
1 answer
How to extract ProcessInstanceHistory object records using new Salesforce connector from ADF
We are trying to extract the data from Salesforce for the ProcessInstanceHistory object through the new Salesforce connector. There is a limitation for the new connector as it doesn't allow inner SOQL query for Bulk API. We have tried using REST API but…
asked 2025-01-07T12:19:33.4766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(268.8, 47%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESD%3C/text%3E%3C/svg%3E)
S D, Sudevika
40
Reputation points
commented 2025-01-10T09:19:03.98+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(201.6, 18%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EGG%3C/text%3E%3C/svg%3E)
Ganesh Gurram
3,110
Reputation points Microsoft Vendor
2 answers
One of the answers was accepted by the question author.
Can i recover blob file from Azure Storage blob in container, if soft delete is not is turned on
ive accidentaly deleted nessecery file (.xlsx extension), is there any way to recover it?
asked 2025-01-08T14:57:17.2866667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(38.4, 15%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBW%3C/text%3E%3C/svg%3E)
Bibel, Wojciech (BSS)
20
Reputation points
accepted 2025-01-08T15:56:52.2033333+00:00
Bibel, Wojciech (BSS)
20
Reputation points
1 answer
One of the answers was accepted by the question author.
Ingress and Egress cost in azure
I have gone through below thread: https://learn.microsoft.com/en-us/answers/questions/892660/will-the-ingress-egress-costs-of-a-azure-storage-a but still confused and clear about the ingress and egress costs of ADLS, and what parameters it takes to…
asked 2024-08-08T15:31:30.9766667+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(243.2, 88%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
azure_learner
525
Reputation points
commented 2025-01-07T14:53:01.79+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(278.4, 46%, 36%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EJ%3C/text%3E%3C/svg%3E)
jhaydt
0
Reputation points
3 answers
Troubleshooting Power BI and Azure Storage Explorer Connection Issues to Data Lake Gen2 from Azure VM
I rented a virtual machine (VM) and had no issues using it until I needed to integrate it with the Data Lake to optimize my processes. I subscribed to the Data Lake, but I cannot connect to it through Power BI when using the VM. My account is a Gmail…
asked 2024-12-10T11:54:22.8833333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(124.80000000000001, 73%, 22%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERC%3C/text%3E%3C/svg%3E)
Rafael CST
80
Reputation points
edited the question 2025-01-07T13:24:27.49+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(105.60000000000001, 79%, 20%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3END%3C/text%3E%3C/svg%3E)
Nikhil Duserla
3,960
Reputation points Microsoft Vendor
1 answer
One of the answers was accepted by the question author.
Constantly Running Interactive Clusters Best Practices
Hello there, I’ve been creating an ETL/ELT Pipeline with Azure Databricks Workflows, Spark and Azure Data Lake. It should process in Near Real Time changes (A Change Data Capture process) from an Azure SQL Database. For that purpose, I will have…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(224.00000000000003, 36%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
1 answer
One of the answers was accepted by the question author.
issues in synapse pipelines which accessing the checkpoints key with input parameters
hello team, i am getting synapse analytics pipelines failures with error referring the checkpoint mistaching with the checkpoint in input parameters. tried to check the input as the respective dataflow is fetching data from cosmosDb with CDC approach…
asked 2024-12-20T14:24:05.0233333+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(3.2, 98%, 10%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAG%3C/text%3E%3C/svg%3E)
ANKIT GHENGE
65
Reputation points
accepted 2025-01-06T14:41:28.34+00:00
ANKIT GHENGE
65
Reputation points
1 answer
I am getting the below error when I try to insert into a azure data lake table. "Type=System.IO.FileNotFoundException,Message=Could not load file or assembly 'System.Text.Json"
I am getting the below error when I try to insert into a azure data lake table from adf pipeline . "The type initializer for 'Azure.Core.Serialization.JsonObjectSerializer' threw an…
asked 2024-12-27T08:56:47.04+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(102.4, 87%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBT%3C/text%3E%3C/svg%3E)
Beebi Thasleem M
20
Reputation points
commented 2025-01-06T07:39:25.1733333+00:00
AnnuKumari-MSFT
33,986
Reputation points Microsoft Employee