I have deleted lots of blobs but the metrics (and cost) didn't get updated
Hello,
Over the last 7 days, I have deleted over 787M blobs on my storage account.
But I see no change in the blob count metric or in the daily billing for data stored in that storage.
I have no soft-delete or snapshot enabled on the storage.
When should I expect the change to be taken into account, and would it be retroactive regarding the billing?
Azure Storage Accounts
-
Vinod Kumar Reddy Chilupuri 2,465 Reputation points • Microsoft Vendor
2025-02-11T10:31:03.9666667+00:00 Hi @Michael Guirao
Greetings! Welcome to Microsoft Q&A Forum and thanks for posting your query here!
When you delete blobs in Azure Blob Storage, it may take some time for the Blob Capacity metrics to show the changes. The metrics are updated hourly, and a background process updates them multiple times a day. So, if you delete a lot of data, it may take some time before you see the changes in your capacity metrics.
As for billing, while the metrics may take time to update, the exact amount of time it takes can vary depending on the size and complexity of your storage account, the billing for data stored is typically based on the actual data stored at the end of the billing cycle. If you've deleted blobs, you should see a reduction in your billing for the next cycle, but it may not be retroactive for the current billing period. However, this can also depend on the specific billing plan and terms that you have with Azure.
In the meantime, you can continue to monitor the metrics and billing information for your storage account to see if the changes are eventually reflected.
https://learn.microsoft.com/en-us/azure/storage/blobs/storage-blob-faq#metrics-and-logs
Hope the above answer helps! Please let us know do you have any further queries.
Please do consider to “up-vote” wherever the information provided helps you, this can be beneficial to other community members.
-
Michael Guirao • 0 Reputation points
2025-02-12T09:54:40.6066667+00:00 Hello @Vinod Kumar Reddy Chilupuri
Thank you for posting this, I was already aware of that, and this is because of it that I posted my message.
It looks to be anormal behavior from Azure system that my blob count metrics is not updated following my deletion of around 800M blobs after few days.
Can you help me investigate further? -
Vinod Kumar Reddy Chilupuri 2,465 Reputation points • Microsoft Vendor
2025-02-12T12:36:29.4733333+00:00 Hi @Michael Guirao
Potential Causes for Delayed Blob Count Metrics Update
- Processing Time: The Azure system may take extra time to handle a large number of deletions, especially with hundreds of millions of blobs.
- Metric Update Frequency: Azure Blob Storage metrics might not update instantly. There could be scheduled intervals for updates, meaning changes may not appear immediately.
- System Load: High system load or maintenance activities within Azure could slow down metric updates.
- Configuration Issues: Check for any misconfigurations in your Azure Blob Storage account that might affect metric reporting.
- Wait for Updates: Allow some time for the metrics to refresh. Check again after a few hours or days.
- Check Azure Status: Visit the Azure status page to see if there are any ongoing issues or maintenance affecting service performance.
Please use the below scripts manually check the size of all blobs after deleting the blobs in particular container with script:
az storage blob list --account-name (Account name) --container-name (name) --query "[].properties.contentLength" --output tsv | awk '{sum+=$1} END {print sum/1024/1024 " MB"}'
Also, this command is used to check the blob count in particular container.
az storage blob list --account-name (Account name) --container-name (name) --query "length([])"
Hope the above answer helps! Please let us know do you have any further queries.
-
Vinod Kumar Reddy Chilupuri 2,465 Reputation points • Microsoft Vendor
2025-02-13T12:16:55.5733333+00:00 Hi @Michael Guirao
Just checking to see if the above suggestion helped you to solve your issue, please let us know if you have any further queries. If the above suggestion help do Upvote for the answer that helped you, this can be beneficial to other community members.
-
Vinod Kumar Reddy Chilupuri 2,465 Reputation points • Microsoft Vendor
2025-02-14T12:26:56.11+00:00 Hi @Michael Guirao
Just checking to see if the above suggestion helped you to solve your issue, please let us know if you have any further queries. If the above suggestion help do Upvote for the answer that helped you, this can be beneficial to other community members.
-
Michael Guirao • 0 Reputation points
2025-02-17T09:35:34.8066667+00:00 Hi @Vinod Kumar Reddy Chilupuri
I have tested the 2 command but the result is irrelevant because they are limited to 5000 blobs.
As you can see on the metrics I'm talking about Billions of blobs, so even looping using the next marker would take ages to get the result.Can you please loop with other engineer to find a better solution, because in the mean time I'm paying data stored for blobs that doesn't exists...
Thank you
-
Venkatesan S • 175 Reputation points • Microsoft Vendor
2025-02-17T11:53:36.48+00:00 Hi Michael Guirao
Could you please check the particular Storage account metrics by Navigating to Storage account -> Monitoring -> Metrics -> Metric name= Blob count and check still it is showing same blob count.Also refer this MS Document for Monitoring Azure Storage cost analysis.
Please let me know do you have any further queries.
-
Michael Guirao • 0 Reputation points
2025-02-17T14:24:34.6+00:00 Hello Venkatesan S
Here's the capture from the metric view, as you can see the blob count only changed a bit during the period, with 50M reduction as opposed to the ~800M expected based on the delete transactions shown by the other metric. -
Venkatesan S • 175 Reputation points • Microsoft Vendor
2025-02-17T14:47:39.2866667+00:00 Have you verified the blobs are deleted successfully? Refer here https://learn.microsoft.com/en-us/azure/storage/blobs/soft-delete-blob-manage.
-
Venkatesan S • 175 Reputation points • Microsoft Vendor
2025-02-18T03:54:18.2833333+00:00 I verified your first picture shows you deleted 787M blobs in normal Azure blob storage and in your third picture you are saying that storage account kind is Data Lake gen 2 account.
For Azure Data Lake gen2 storage account the Metric API name is
DeleteFile
for deleted blobs.
Please check that you have mixed up with two storage account.Please let me know do you have any further queries.
Please do consider to “up-vote” wherever the information provided helps you, this can be beneficial to other community members.
-
Michael Guirao • 0 Reputation points
2025-02-18T10:37:01.09+00:00 Venkatesan S
I can confirm there is no mix up in the storage, this is a ADLS gen2 storage with hierarchical enabled as shown in the this picture of the initale message.Regarding the api name, here is a new capture from the metric pane, and the api used is definitely 'DeleteBlob'.
Details about how the deletion is done, we are using Aspire.Azure.Storage.Blobs in version 9.0.0, BlobServiceClient with DeleteIfExistsAsync.
-
Deleted
This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
-
Venkatesan S • 175 Reputation points • Microsoft Vendor
2025-02-18T10:59:49.2133333+00:00 You can see my gif here with same settings and there is no
DeleteBlob
metric in Metrics plane with Azure Data Lake Gen2 storage.
Please let me know do you have any further queries.
-
Michael Guirao • 0 Reputation points
2025-02-18T13:10:11.5266667+00:00 Venkatesan S
Well... you don't have this, ok, but i do, see my gif
Now, did i convince you that I did not mixed up the storages ?
Let me know the next steps -
Venkatesan S • 175 Reputation points • Microsoft Vendor
2025-02-18T14:08:55.9033333+00:00 @Michael Guirao Apologies, and thanks for the GIF. We will check with our internal team and keep you posted.
-
KarishmaTiwari-MSFT • 20,567 Reputation points • Microsoft Employee
2025-02-18T23:01:49.11+00:00 @Michael Guirao Thanks for sharing the detailed information.
Please create a free billing support ticket so that our team of billing support engineers can review your case. You can follow the steps described to create the support request: https://learn.microsoft.com/en-us/azure/azure-portal/supportability/how-to-create-azure-support-request#open-a-support-request-from-the-global-header
Please select 'Billing' in the Issue Type, as shown below:
Once you have created the support request, please share the support request number with me over private message. I will monitor it internally to ensure that you are receiving the support you need.
Sign in to comment