How much time will it take to rehydrate 4.7Tb of data organized in 24 folders, each folder having 3390 blobs of around 50Mb each?

Question

Hello, I have 4.72Tb of data organised in 24 folders each having around 3390 blobs and each blob of approximative size of 50Mb on the archive tier.

Im on GPv2 with GRS replication in westeurope.

How long will it take me to rehydrate all of this data?

How many concurrent requests for setBlobTier can I send for changing the tier of this data?

Also could you please confirm the associated cost operations linked to the rehydration. My understanding is that I will be charged for the read transactions and data retrieval, as well as the listBlob operations. If i decide to use BlobBatch then I will be charged for that operation as well?

Answer 1

Hi Marija Kirandjiska

Rehydrating data from the Azure Blob Storage Archive tier to the Hot or Cool tier involves several factors, including the size of the data, the number of blobs, the performance of your Azure Storage account, and the network conditions.

The rehydration time for 4.72 TB of data organized in 24 folders with 3,390 blobs each (totalling approximately 81,360 blobs) can vary significantly based on several factors:

• Rehydration Speed: Azure typically processes rehydration requests at a rate of about 1-2 TB per day, but this can vary based on the current load on the storage account and the number of concurrent requests.
• ⁠Standard Priority: Takes up to 15 hours for objects under 10GB.
• High Priority: Takes less than one hour for objects under 10GB but costs more. https://learn.microsoft.com/en-us/azure/storage/blobs/archive-rehydrate-overview
• ⁠When you rehydrate a blob, you can specify the priority for the operation to either standard priority or high priority. A standard-priority rehydration operation may take up to 15 hours to complete. A high-priority operation is prioritized over standard-priority requests and may complete in less than one hour for objects under 10 GB in size. You can change the rehydration priority from Standard to High while the operation is pending. Estimated Rehydration Time: 2 to 5 days (varies based on conditions). https://learn.microsoft.com/en-us/azure/storage/blobs/archive-rehydrate-to-online-tier?tabs=azure-portal

Concurrent Requests for setBlobTier:

Azure Blob Storage allows you to send multiple concurrent requests to change the blob tier. While there is no hard limit on the number of concurrent requests, it is generally recommended to keep the number of concurrent requests to a reasonable level to avoid throttling.

https://learn.microsoft.com/en-us/azure/storage/blobs/concurrency-manage

Cost of Rehydration Operations:

When rehydrating data from the Archive tier, you will incur several costs:

• Data Retrieval Charges: You will be charged for the amount of data retrieved from the Archive tier. As of the last update, the cost for retrieving data from the Archive tier is higher than from the Hot or Cool tiers.
• Read Transactions: Each read operation (including the rehydration process) will incur a transaction cost. This includes the operations to read the blobs as they are being rehydrated.
• List Blob Operations: If you perform operations to list blobs (e.g., to check the status of the rehydration), you will also incur costs for these operations.
• BlobBatch Operations: If you decide to use the BlobBatch feature to change the tier of multiple blobs in a single request, you will be charged for the batch operation itself. The cost for batch operations is typically lower than making individual requests, but you should check the latest Azure pricing documentation for specifics.
• Check through Azure pricing calculator.
  https://azure.microsoft.com/en-us/pricing/calculator/
  https://azure.microsoft.com/en-us/pricing/details/storage/blobs/
  ⁠https://learn.microsoft.com/en-us/azure/storage/blobs/archive-cost-estimation

Hope the above suggestion helps! Please let us know do you have any further queries.

Please do consider Accepting the answer wherever the information provided helps you, this can be beneficial to other community members.