Azure Storage Accounts
Globally unique resources that provide access to data management services and serve as the parent namespace for the services.
3,352 questions
Data factory copy operation in immutable storage fails
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 9%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EAV%3C/text%3E%3C/svg%3E)
Akshay Vithalkar
1
Reputation point
I have data factory with one pipeline which do copy operation from one container to another within same storage account. I have two containers "daily" and "weekly" and my pipeline copy files from daily to weekly.
When new files are coming into daily container, I need those files copied over to weekly. When I am running pipeline it is successfully copying files but it returing error
ErrorCode=UserErrorWriteFailedFileOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The file operation is failed, upload file failed at path: 'weekly/2912025.txt'.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.WindowsAzure.Storage.StorageException,Message=The remote server returned an error: (409) Conflict.,Source=Microsoft.WindowsAzure.Storage,StorageExtendedMessage=This operation is not permitted as the blob is immutable due to a policy. Daily has retention policy to 1 and weekly has 7. What could be the issue.
{count} votes
-
Akshay Vithalkar 1 Reputation point
2025-01-29T13:06:06.6066667+00:00 I am not understanding why this thowing errors for new files. However, its copying it to destination with no issues. It just I am getting error.
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(307.2, 85%, 39%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESA%3C/text%3E%3C/svg%3E)
Syed Aaqid Ali 5 Reputation points Microsoft Vendor2025-01-29T16:42:14.98+00:00 Greetings & Welcome to Microsoft Q&A forum! Thanks for posting your query!
Based on the error message you're seeing indicates that the blob you're trying to copy is immutable due to a policy. This means that the blob has a retention policy that prevents it from being modified or deleted until a certain period of time has passed.
For example, suppose that a user creates a time-based retention policy with a retention interval of five years. An existing blob in that container, testblob1, was created one year ago, so the effective retention period for testblob1 is four years. When a new blob, testblob2, is uploaded to the container, the effective retention period for testblob2 is five years from the time of its creation.
In your case, it seems that the blob you're trying to copy from the "daily" container to the "weekly" container has a retention policy of 1 day, which means it cannot be modified or deleted until at least 1 day has passed since it was created. This is why you're seeing the "409 Conflict" error when you try to modify the blob. To resolve this issue, you can either wait until the retention period has passed before copying the blob, or you can modify the retention policy to allow the blob to be modified or deleted sooner.
Kindly let me know if this is helpful!
Thanks.
-
Akshay Vithalkar 1 Reputation point
2025-01-30T09:18:53.18+00:00 but what about new blob,, when I am copying new file to destination its being copied successfully but pipeline showing error.
-
Syed Aaqid Ali 5 Reputation points Microsoft Vendor
2025-01-30T11:21:02.4866667+00:00 Hi Akshay,
Copy will be successful as it is a read operation at the Source container, kindly verify if you are making any other write operations after copying the blob from the source, for example delete or overwrite. I would highly appreciate you to check the ADF pipeline if its making any write operations as error message confirms that error message is for the failed write operation.
ErrorCode=UserErrorWriteFailedFileOperation,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The file operation is failed, upload file failed at path: 'weekly/2912025.txt'.,Source=Microsoft.DataTransfer.ClientLibrary,''Type=Microsoft.WindowsAzure.Storage.StorageException,Message=The remote server returned an error: (409) .
-
Akshay Vithalkar 1 Reputation point
2025-02-03T09:47:25.2933333+00:00 weekly/2912025.txt was new file copied from daily, there was no file present before in weekly container. Still same issue.
-
Syed Aaqid Ali 5 Reputation points Microsoft Vendor
2025-02-03T16:55:55.2333333+00:00 Hi Akshay,
As I previously explained the cause of the error messages, to further investigate the root cause of these errors, I highly recommend enabling diagnostic settings within the same storage account and analyzing the corresponding logs.
Logs are stored as data in your account. Log data can accumulate in your account over time which can increase the cost of storage. If you need log data for only a small period of time, you can reduce your costs by modifying the data retention policy. Stale log data (data older than your retention policy) is deleted by the system. We recommend setting a retention policy based on how long you want to retain the log data for your account. See Billing on storage metrics for more information.
Reference:
Azure Storage analytics logging
Kindly let me know if this helps or require further help!
Sign in to comment
Sign in to answer