The remote server returned an error: (400) Bad

Zaeemul Haque 65

Our PIPELINE file process was running perfectly, but couple of days ago it started failing below

it is copying sql data to .parquet format on Hot Storage, please let me know what to check.

Failure happened on 'Sink' side. ErrorCode=AzureBlobWriteOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Blob write operation on path='locatecvhot/repository/cv/LiveFeed' and blobName='2024-12-13.parquet

' failed with error message: The remote server returned an error: (400) Bad Request..,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.WindowsAzure.Storage.StorageException,Message=The remote server returned an error: (400) Bad Request.,Source=Microsoft.WindowsAzure.Storage,''Type=System.Net.WebException,Message=The remote server returned an error: (400) Bad Request.,Source=Microsoft.WindowsAzure.Storage,'

Thank You,

Zaeemul

Chandra Boorla 5,755 Reputation points Microsoft Vendor

2024-12-13T03:06:26.0366667+00:00

Hi @Zaeemul Haque

Greetings & Welcome to Microsoft Q&A forum! Thanks for posting your query!

The error indicates that the Azure Data Factory pipeline is failing during the "Sink" operation, specifically when attempting to write data to Azure Blob Storage in .parquet format. A (400) Bad Request response typically signifies a client-side issue with the request sent to the Azure Storage service. It seems that there is a problem with writing data to Azure Blob Storage in your data pipeline.

Below are some troubleshooting steps that may help you in resolving your query:

Invalid File Path or Blob Name - Check if the file path (locatecvhot/repository/cv/LiveFeed) and blob name (2024-12-13.parquet) comply with Azure Storage naming conventions. Ensure the filename 2024-12-13.parquet adheres to Azure Blob naming conventions (only lowercase letters, numbers, hyphens, and underscores). If necessary, adjust the filename to comply with these rules.

Permissions Issue - The Azure Data Factory might not have sufficient permissions to write to the specified Blob Storage container. Ensure the Managed Identity or Service Principal used in the ADF pipeline has the correct Storage Blob Data Contributor role. Verify and update the IAM role assignments in Azure Blob Storage.

Storage Account Issues - There may be issues with the Azure Storage account itself, such as being disabled or having reached its quota.

Pipeline Configuration - Double-check the pipeline configuration, especially the sink settings, to ensure they are correct and point to the correct storage account, container, and blob name. Review the pipeline's error handling and logging to gain more insights into the issue.

Data Format Issues - Ensure that the data being written is in the correct format expected by Azure Blob Storage. If the data is corrupted or not in the expected format, it could lead to a bad request.

Existing File Conflicts - If a file with the same name already exists in the target location, consider enabling overwriting or ensuring unique naming for each file.

For additional information, please refer the following Microsoft documentation.

Troubleshoot the Azure Blob Storage connector in Azure Data Factory and Azure Synapse

Troubleshoot connector and format issues in mapping data flows in Azure Data Factory

If the issue still persists, please do let us know. Glad to help.

I hope this information helps.

Thank you.
Chandra Boorla 5,755 Reputation points Microsoft Vendor

2024-12-16T01:43:42.95+00:00

@Zaeemul Haque

We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution, please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Accepted answer

Amrinder Singh 5,555 Reputation points Microsoft Employee

2024-12-13T03:38:27.8033333+00:00

Hi Zaeemul Haque - Thanks for reaching over Q&A Forum.

The error code 400 bad request means that when the request is present to storage server it fails over validation and hence the failure. This is due to the request being malformed in terms of URI or some headers not being set correctly.

One of the probable reason for that is some kind of additional characters/specific language etc. The recommendation here is to enable the client side logging to log the request being formed which shall help with the investigation further.

Hope this information helps! please let us know if you have any further queries. I’m happy to assist you further.

Please "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members
Please sign in to rate this answer.
Amrinder Singh 5,555 Reputation points Microsoft Employee

2024-12-14T03:45:28.5566667+00:00

Hi Zaeemul Haque - Hope you are doing well.

A gentle follow up to check if the information helped out. Please let us know if you have any further queries. I’m happy to assist you further.

Please "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members

Chandra Boorla 5,755 Reputation points Microsoft Vendor

2024-12-17T01:04:18.4233333+00:00

@Zaeemul Haque

Just checking in to see if the above answer provided by @ Amrinder Singh helped.

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

The remote server returned an error: (400) Bad

0 additional answers

Your answer