Azure Data Lake Storage

2 answers

how to fix this error The request content was invalid and could not be deserialized: Required property 'name' not found in 'sku'

getting error while synapse workspace deployment

asked

Narendra Singh 15

commented

milan barot 0

1 answer

Preview features like AllowDataSharingInHeroRegion and AllowDataSharing are in pending status since a month

Hi team, Azure subscription preview features are like AllowDataSharingInHeroRegion and AllowDataSharing are in pending status since a month. We tried to unregister and re-register but still no luck. Could you please help us on this? Thanks, and…

asked

chaitanya.K 20

accepted

chaitanya.K 20

0 answers

How to Automate Large .SAV File to Parquet Conversion in Azure?

I'm using Azure ADLS as our primary storage, Azure Data Factory (ADF) for data transformations, and Power BI for reporting and visualization. I have a large .SAV file (200-300 MB, containing 2-4 million rows) stored in Azure Data Lake Storage (ADLS). To…

asked

Akshay Patel 70

commented

Hari Babu Vattepally 1,365 Microsoft Vendor

1 answer

How to extract ProcessInstanceHistory object records using new Salesforce connector from ADF

We are trying to extract the data from Salesforce for the ProcessInstanceHistory object through the new Salesforce connector. There is a limitation for the new connector as it doesn't allow inner SOQL query for Bulk API. We have tried using REST API but…

asked

S D, Sudevika 40

commented

Ganesh Gurram 3,025 Microsoft Vendor

1 answer

How to fix nested split issue in Azure data factory Pipeline expression builder?

Hi there, I'm trying to make use of hierarchical namespaces for Azure data lake storage gen2. When I'm dynamically setting the file structure in the Pipeline expression builder in the ADF, the following error is returned: Cannot fit unknown into the…

asked

Zohaib Sheraz Mughal 15

commented

Keshavulu Dasari 2,835 Microsoft Vendor

0 answers

generate SAS tokens from serviceprincipal credentials

I am working on to create java client that generates sas tokens for the given service principal credentials. I am taking a reference from…

asked

SHYAMALA GOWRI 90

commented

Hari Babu Vattepally 1,365 Microsoft Vendor

2 answers

Can i recover blob file from Azure Storage blob in container, if soft delete is not is turned on

ive accidentaly deleted nessecery file (.xlsx extension), is there any way to recover it?

asked

Bibel, Wojciech (BSS) 20

accepted

Bibel, Wojciech (BSS) 20

1 answer

Ingress and Egress cost in azure

I have gone through below thread: https://learn.microsoft.com/en-us/answers/questions/892660/will-the-ingress-egress-costs-of-a-azure-storage-a but still confused and clear about the ingress and egress costs of ADLS, and what parameters it takes to…

asked

azure_learner 525

commented

jhaydt 0

3 answers

Troubleshooting Power BI and Azure Storage Explorer Connection Issues to Data Lake Gen2 from Azure VM

I rented a virtual machine (VM) and had no issues using it until I needed to integrate it with the Data Lake to optimize my processes. I subscribed to the Data Lake, but I cannot connect to it through Power BI when using the VM. My account is a Gmail…

asked

Rafael CST 80

edited the question

Nikhil Duserla 3,880 Microsoft Vendor

1 answer

Constantly Running Interactive Clusters Best Practices

Hello there, I’ve been creating an ETL/ELT Pipeline with Azure Databricks Workflows, Spark and Azure Data Lake. It should process in Near Real Time changes (A Change Data Capture process) from an Azure SQL Database. For that purpose, I will have…

asked

Martin 100

commented

Martin 100

1 answer

issues in synapse pipelines which accessing the checkpoints key with input parameters

hello team, i am getting synapse analytics pipelines failures with error referring the checkpoint mistaching with the checkpoint in input parameters. tried to check the input as the respective dataflow is fetching data from cosmosDb with CDC approach…

asked

ANKIT GHENGE 65

accepted

ANKIT GHENGE 65

1 answer

I am getting the below error when I try to insert into a azure data lake table. "Type=System.IO.FileNotFoundException,Message=Could not load file or assembly 'System.Text.Json"

I am getting the below error when I try to insert into a azure data lake table from adf pipeline . "The type initializer for 'Azure.Core.Serialization.JsonObjectSerializer' threw an…

asked

Beebi Thasleem M 20

commented

AnnuKumari-MSFT 33,986 Microsoft Employee

1 answer

We want to loop all folder and file recursively of BLOB Storage for given container using python in azure data factory custom activity and write data in in blob in .CSV file

Hi All, We want to loop all folder and file recursively of BLOB Storage for given container using python in azure data factory custom activity and write data in in blob in .CSV file. also when we connect blob storage using ADF custom activity using SAS…

asked

manish verma 501

commented

AnnuKumari-MSFT 33,986 Microsoft Employee

1 answer

The certificate with identifier used to sign the client assertion is expired on application

Hi Team, I am trying to create some external tables from the azure synapse on the parquet files that I have load into the Azure Data Lake storage Gen2. I also created the app in azure directly and extended the expiry date and also provided the Synapse…

asked

Prashant Meshram 46

commented

Rustin Davitt 0

2 answers

Append vs update in Azure Data Lake Storage Gen2 for a csv file.

I have a CSV file in a Data Lake Storage Gen2. I am referring to the ACL permissions below.https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-access-control#common-scenarios-related-to-acl-permissionsThe table gives different ACL…

asked

Ravikiran Srinivasulu 10

edited an answer

Keshavulu Dasari 2,835 Microsoft Vendor

2 answers

Uploaded File in ADLS Gen 2 Account is not Visible.

I have created ADLS Gen 2 account and then Container. In Container, I created 1 Directory. Now, When i am uploading CSV file or any file, i am not able to see the file in the directory. It says successfully loaded 1 blob but nothing is visible. I have…

asked

geita naarang 0

commented

Keshavulu Dasari 2,835 Microsoft Vendor

2 answers

Getting TokenNotFoundInConfigurationException error when uploading a Python wheel as workspace package in Synpase

Hello Everyone I'm using Azure Synapse to run my Spark pipelines. I've build my pipeline to use a custom library code from utils package of the same project to perform some operation. To access the utils.pipeline file, have packaged my entire project in…

asked

manigandan 0

commented

Smaran Thoomu 18,965 Microsoft Vendor

1 answer

Getting "storage_root does not specify a URI scheme." while setting ADLS path in default metastore.

Hi Team, I am facing while setting the ADLS path for the existing default metastore in account console page in azure. Below is the image showing i have already created the container and have enabled Hierarchical namespace as well. But getting below…

asked

Akancha 0

commented

Keshavulu Dasari 2,835 Microsoft Vendor

1 answer

When generating a SAS token, specifying the path causes an authorization failure.

I use following code to generate SAS token, when path is null or empty string, it's working. String sasToken = new DataLakeSasImplUtil(signatureValues, container, path, true) .generateUserDelegationSas(userDelegationKey, accountName,…

asked

Jun 0

commented

Keshavulu Dasari 2,835 Microsoft Vendor

2 answers

ADLS Gen2 operation failed for: Operation returned an invalid status code 'Conflict'. Account: 'abc'. ErrorCode: 'EndpointUnsupportedAccountFeatures'. Message: 'This endpoint does not support BlobStorageEvents or SoftDelete.

I am new to ADF. while creating linked service for data lake storage Gen2 I am getting a error ADLS Gen2 operation failed. ADLS Gen2 operation failed for: Operation returned an invalid status code 'Conflict'. Account: 'abc'. ErrorCode:…

asked

manoj reddy 20

commented

manoj reddy 20

Filter

Content

1,521 questions with Azure Data Lake Storage tags

how to fix this error The request content was invalid and could not be deserialized: Required property 'name' not found in 'sku'

Preview features like AllowDataSharingInHeroRegion and AllowDataSharing are in pending status since a month

How to Automate Large .SAV File to Parquet Conversion in Azure?

How to extract ProcessInstanceHistory object records using new Salesforce connector from ADF

How to fix nested split issue in Azure data factory Pipeline expression builder?

generate SAS tokens from serviceprincipal credentials

Can i recover blob file from Azure Storage blob in container, if soft delete is not is turned on

Ingress and Egress cost in azure

Troubleshooting Power BI and Azure Storage Explorer Connection Issues to Data Lake Gen2 from Azure VM

Constantly Running Interactive Clusters Best Practices

issues in synapse pipelines which accessing the checkpoints key with input parameters

I am getting the below error when I try to insert into a azure data lake table. "Type=System.IO.FileNotFoundException,Message=Could not load file or assembly 'System.Text.Json"

We want to loop all folder and file recursively of BLOB Storage for given container using python in azure data factory custom activity and write data in in blob in .CSV file

The certificate with identifier used to sign the client assertion is expired on application

Append vs update in Azure Data Lake Storage Gen2 for a csv file.

Uploaded File in ADLS Gen 2 Account is not Visible.

Getting TokenNotFoundInConfigurationException error when uploading a Python wheel as workspace package in Synpase

Getting "storage_root does not specify a URI scheme." while setting ADLS path in default metastore.

When generating a SAS token, specifying the path causes an authorization failure.

ADLS Gen2 operation failed for: Operation returned an invalid status code 'Conflict'. Account: 'abc'. ErrorCode: 'EndpointUnsupportedAccountFeatures'. Message: 'This endpoint does not support BlobStorageEvents or SoftDelete.