1,531 questions with Azure Data Lake Storage tags

Sort by: Updated
2 answers

How to fix nested split issue in Azure data factory Pipeline expression builder?

Hi there, I'm trying to make use of hierarchical namespaces for Azure data lake storage gen2. When I'm dynamically setting the file structure in the Pipeline expression builder in the ADF, the following error is returned: Cannot fit unknown into the…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,159 questions
asked 2025-01-07T23:56:02.7266667+00:00
Zohaib Sheraz Mughal 15 Reputation points
edited an answer 2025-01-26T13:20:13.0033333+00:00
Keshavulu Dasari 3,090 Reputation points Microsoft Vendor
1 answer

ErrorCode=AdlsGen2OperationFailed...Failure happened on 'Sink' side.

Getting the below error while copying the data to ADLS from SAP Hana, however this error is random to tables failing daily. When I try load each of the failed tables from the same source and with the same file name then there is no issue and it works as…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,159 questions
asked 2025-01-25T11:59:38.9966667+00:00
Tyagi, Amit 5 Reputation points
answered 2025-01-26T07:28:24.8066667+00:00
Vinodh247 27,281 Reputation points MVP
1 answer

Monitor AutoResolve Integration Runtime in Azure Data Factory

Hi all I need to monitor the Integration Runtime performance for a pipeline that uses stored procedure activity to load data from ADLS to Snowflake. The stored procedure in this case resides in snowflake itself and it’s called from Snowflake Tenant to…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,159 questions
asked 2025-01-24T06:12:53.87+00:00
AyusmanBasu-6332 6 Reputation points
answered 2025-01-25T11:11:19.7833333+00:00
Vinodh247 27,281 Reputation points MVP
0 answers

I am integrating the dataflow activity which removes duplicates from the csv file to the existing pipeline i have. I need a help to configure the parameters.

I created a dataflow activity which removes duplicates from the source files( csv formatted files). So now i integrated this dataflow activity to existing pipeline i have. but i am not able to parametrize the source files. It would be helpful if i get…

Azure SQL Database
Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
3,340 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
3,044 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,159 questions
asked 2025-01-24T08:02:14.5133333+00:00
Lakshmi Moulya Nerella 0 Reputation points
commented 2025-01-24T09:50:21.43+00:00
Pinaki Ghatak 5,575 Reputation points Microsoft Employee
1 answer

How to write the query on Azure to get the information of particular SPN linked with all the folders present in containers..

Hi, As there are multiple folders are preset into Azure storage containers, each folders of containers linked with some SPN. I am looking for the script to create to find, which SPN associate with which all folders?

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
Azure Container Apps
Azure Container Apps
An Azure service that provides a general-purpose, serverless container platform.
511 questions
asked 2025-01-13T14:36:48.55+00:00
Saurabh Parmar 0 Reputation points
commented 2025-01-24T05:26:09.4433333+00:00
Nandamuri Pranay Teja 330 Reputation points Microsoft Vendor
1 answer

Azure Data Factory: No data when trying to preview data source

Azure Data Factory: No data when trying to preview data source. Data is present at source and even when we minimize the browser to 30% we are able to see data in data preview but when we are doing normal data preview we are unable to see any data in all…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,159 questions
asked 2025-01-23T08:21:02.1733333+00:00
KOLI, SONIA 0 Reputation points
commented 2025-01-24T04:47:50.9266667+00:00
Ganesh Gurram 3,270 Reputation points Microsoft Vendor
2 answers

ADF/ Synapse Activity times out after 12 hours

Hello, I have a Notebook activity in one of my pipelines that is expected to run longer than 12 hours. Below we configured the timeout to the max value, 7 days. But the activity would still timeout after 12 hours. Is there a way for the activity…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
Azure Storage Accounts
Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
3,340 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,149 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,159 questions
asked 2025-01-13T23:02:58.05+00:00
Marc 0 Reputation points
commented 2025-01-22T07:11:03.34+00:00
phemanth 13,150 Reputation points Microsoft Vendor
1 answer

D365 Outbound Marketing Data sync to AdLS Gen2 cannot be synced inside a folder structure but only on conatiner.

I wanted to copy Market Outbound Data from CRM like email sent, delivered, Email clicked, Email Opened , etc from the marketing campaign to ADLS Gen 2 using Data Factory but seems like I cannot use ADF to do that as they are calculated live. Also, I…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,159 questions
asked 2025-01-13T03:18:27.1333333+00:00
Rabin Chaudhary 1 Reputation point
commented 2025-01-20T07:47:35.8766667+00:00
Vinod Kumar Reddy Chilupuri 2,310 Reputation points Microsoft Vendor
1 answer

Stream RealTime data from an On Prem Microsoft Server to a Azure Data Lake.

There is a need for Real Time data to be collected from a On Prem SQL Server. This means ADF is not suitable because it only does batch. Azure Streaming Analytics only connects to Azure SQL Server and not on prem. Has anybody else encountered this and…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
Azure Event Hubs
Azure Event Hubs
An Azure real-time data ingestion service.
676 questions
Azure Stream Analytics
Azure Stream Analytics
An Azure real-time analytics service designed for mission-critical workloads.
373 questions
asked 2023-03-16T02:02:54.12+00:00
Sallie Worsley 5 Reputation points
commented 2025-01-17T23:10:26.6+00:00
Sander van de Velde | MVP 34,386 Reputation points MVP
2 answers

Why are access tier options greyed out for Azure Data Lake Gen 2 storage?

When attempting to change the access tier of a file stored in Azure Data Lake Gen 2, all the access tier options are greyed out. What could be causing this issue and what are the possible solutions?

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
asked 2025-01-13T16:45:58.49+00:00
Samuel Kwesiga Kisitu 0 Reputation points
commented 2025-01-17T06:26:52.91+00:00
Hari Babu Vattepally 1,475 Reputation points Microsoft Vendor
1 answer

generate SAS tokens from serviceprincipal credentials

I am working on to create java client that generates sas tokens for the given service principal credentials. I am taking a reference from…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
Microsoft Entra ID
Microsoft Entra ID
A Microsoft Entra identity service that provides identity management and access control capabilities. Replaces Azure Active Directory.
22,996 questions
asked 2025-01-08T12:00:29.36+00:00
SHYAMALA GOWRI 90 Reputation points
commented 2025-01-17T06:25:42.78+00:00
Hari Babu Vattepally 1,475 Reputation points Microsoft Vendor
1 answer

Unable to create a directory named space character using ADLS Gen2 REST API

I am unable to create a directory in "Azure Data Lake Storage Gen2" that is named as just a single space character, despite reviewing the documentation and there being no indication this is a disallowed name. My primary access to "Azure…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
asked 2025-01-16T15:07:47.1866667+00:00
Phil R 0 Reputation points
answered 2025-01-16T18:08:44.9833333+00:00
Keshavulu Dasari 3,090 Reputation points Microsoft Vendor
1 answer One of the answers was accepted by the question author.

How to Automate Large .SAV File to Parquet Conversion in Azure?

I'm using Azure ADLS as our primary storage, Azure Data Factory (ADF) for data transformations, and Power BI for reporting and visualization. I have a large .SAV file (200-300 MB, containing 2-4 million rows) stored in Azure Data Lake Storage (ADLS). To…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
Azure Functions
Azure Functions
An Azure service that provides an event-driven serverless compute platform.
5,352 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,159 questions
asked 2024-12-27T10:31:15.69+00:00
Akshay Patel 90 Reputation points
accepted 2025-01-16T10:02:14.32+00:00
Akshay Patel 90 Reputation points
1 answer One of the answers was accepted by the question author.

Read AppServiceConsoleLogs and push to Blob Storage container/table

Hello Team, we have a webapp python Django based. We are generating custom logs for user actions using code below. Settings.py: LOGGING = { "version": 1, "disable_existing_loggers": False, "formatters": { …

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
3,044 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,312 questions
Azure App Service
Azure App Service
Azure App Service is a service used to create and deploy scalable, mission-critical web apps.
8,214 questions
Azure Static Web Apps
Azure Static Web Apps
An Azure service that provides streamlined full-stack web app development.
1,055 questions
asked 2024-11-20T09:31:14.9233333+00:00
BalaKrishna Mahamkali 20 Reputation points
accepted 2025-01-16T04:32:25.14+00:00
BalaKrishna Mahamkali 20 Reputation points
1 answer One of the answers was accepted by the question author.

Change the Azure synapse workspace default storage account.

Hi Team, We created an Azure Synapse workspace with a default storage account associated with it. However, due to various reasons, we are not using the storage account tied to the Synapse workspace. Instead, we have created another ADLS Gen2 storage…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,149 questions
asked 2025-01-13T22:36:47.74+00:00
Sai Pranavi Unnam 20 Reputation points Microsoft Employee
accepted 2025-01-14T20:47:55.6766667+00:00
Sai Pranavi Unnam 20 Reputation points Microsoft Employee
2 answers

how to fix this error The request content was invalid and could not be deserialized: Required property 'name' not found in 'sku'

getting error while synapse workspace deployment

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
5,149 questions
asked 2024-12-09T14:46:47.8+00:00
Narendra Singh 15 Reputation points
commented 2025-01-11T03:44:00.18+00:00
milan barot 0 Reputation points
1 answer One of the answers was accepted by the question author.

Preview features like AllowDataSharingInHeroRegion and AllowDataSharing are in pending status since a month

Hi team, Azure subscription preview features are like AllowDataSharingInHeroRegion and AllowDataSharing are in pending status since a month. We tried to unregister and re-register but still no luck. Could you please help us on this? Thanks, and…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
Microsoft Purview
Microsoft Purview
A Microsoft data governance service that helps manage and govern on-premises, multicloud, and software-as-a-service data. Previously known as Azure Purview.
1,355 questions
asked 2025-01-03T19:40:21.0733333+00:00
chaitanya.K 20 Reputation points
accepted 2025-01-11T00:30:14.16+00:00
chaitanya.K 20 Reputation points
1 answer

How to extract ProcessInstanceHistory object records using new Salesforce connector from ADF

We are trying to extract the data from Salesforce for the ProcessInstanceHistory object through the new Salesforce connector. There is a limitation for the new connector as it doesn't allow inner SOQL query for Bulk API. We have tried using REST API but…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
2,312 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,159 questions
asked 2025-01-07T12:19:33.4766667+00:00
S D, Sudevika 40 Reputation points
commented 2025-01-10T09:19:03.98+00:00
Ganesh Gurram 3,270 Reputation points Microsoft Vendor
2 answers One of the answers was accepted by the question author.

Can i recover blob file from Azure Storage blob in container, if soft delete is not is turned on

ive accidentaly deleted nessecery file (.xlsx extension), is there any way to recover it?

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
Azure Blob Storage
Azure Blob Storage
An Azure service that stores unstructured data in the cloud as blobs.
3,044 questions
Azure Disk Storage
Azure Disk Storage
A high-performance, durable block storage designed to be used with Azure Virtual Machines and Azure VMware Solution.
659 questions
asked 2025-01-08T14:57:17.2866667+00:00
Bibel, Wojciech (BSS) 20 Reputation points
accepted 2025-01-08T15:56:52.2033333+00:00
Bibel, Wojciech (BSS) 20 Reputation points
1 answer One of the answers was accepted by the question author.

Ingress and Egress cost in azure

I have gone through below thread: https://learn.microsoft.com/en-us/answers/questions/892660/will-the-ingress-egress-costs-of-a-azure-storage-a but still confused and clear about the ingress and egress costs of ADLS, and what parameters it takes to…

Azure Data Lake Storage
Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,531 questions
Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,159 questions
asked 2024-08-08T15:31:30.9766667+00:00
azure_learner 525 Reputation points
commented 2025-01-07T14:53:01.79+00:00
jhaydt 0 Reputation points