Create a Managed Application to Store Blob Digests
Prerequisites
- An Azure Storage Account
- Azure CLI (optional)
- Python version that is supported by the Azure SDK for Python (optional)
Overview
The Blob Storage Digest Backed by Confidential Ledger Managed Application can be used to guarantee that the blobs within a blob container are trusted and not tampered with. The application, once connected to a storage account, tracks all blobs being added to every container in the storage account in real time in addition to calculating and storing the digests into Azure Confidential Ledger. Audits can be performed at any time to check the validity of the blobs and to ensure that the blob container isn't tampered with.
Deploying the managed application
The Managed Application can be found in the Azure Marketplace here: Blob Storage Digests Backed by Confidential Ledger (preview).
Resources to be created
Once the required fields are filled and the application is deployed, the following resources are created under a Managed Resource Group:
- Confidential Ledger
- Service Bus Queue with Sessions enabled
- Storage Account (Publisher owned storage account used to store digest logic and audit history)
- Function App
- Application Insights
Connecting a storage account to the managed application
Once a Managed Application is created, you're able to then connect the Managed Application to your Storage Account to start processing and recording Blob Container digests to Azure Confidential Ledger.
Create a topic and event subscription for the storage account
The Managed Application uses an Azure Service Bus Queue to track and record all Create Blob events. You will use the Queue created in the Managed Resource Group by the Managed Application and add it as an Event Subscriber for any storage account that you're creating blobs for.
On the Azure portal, you can navigate to the storage account that you would like to start creating blob digests for and go to the Events
blade. There you can create an Event Subscription and connect it to the Azure Service Bus Queue Endpoint.
The queue uses sessions to maintain ordering across multiple storage accounts so you will also need to navigate to the Delivery Properties
tab and to enter a unique session ID for this event subscription.
Add required role to storage account
The Managed Application requires the Storage Blob Data Owner
role to read and create hashes for each blob and this role is required to be added in order for the digest to be calculated correctly.
Note
Multiple storage accounts can be connected to a single Managed Application instance. We currently recommend a maximum of 10 storage accounts that contain high usage blob containers.
Adding blobs and digest creation
Once the storage account is properly connected to the Managed Application, blobs can start being added to containers within the storage account. The blobs are tracked in real-time and digests are calculated and stored in Azure Confidential Ledger.
Transaction and block tables
All blob creation events are tracked in internal tables stored within the Managed Application.
The transaction table holds information about each blob and a unique hash that is generated using a combination of the blob's metadata and/ or contents.
The block table holds information related to every digest this is created for the blob container and the associated transaction ID for the digest is stored in Azure Confidential Ledger.
Digest settings
There are a few digest settings that can be selected when creating the managed application. You can choose the Hashing Algorithm
being used to create the digests whether it be MD5
or SHA256
. You are also able to choose the number of blobs that are contained within each digest or the Digest Size
. The Digest Size ranges from 1-16
and is the number of blobs that will be hashed together within each block. Lastly, you are able to select the Hash Contents
and what will be hashed when creating each digest. This can be the File Contents + Metadata
of each blob or just the File Contents
.
Viewing digest on Azure Confidential Ledger
You can view the digests being stored directly in Azure Confidential Ledger by navigating to the Ledger Explorer
blade.
Performing an audit
If you ever want to check the validity of the blobs that are added to a container to ensure that they aren't tampered with, an audit can be run at any point in time. The audit replays every blob creation event and recalculates the digests with the blobs that are stored in the container during the audit. It then compares the recalculated digests with the digests stored in Azure Confidential and provides a report displaying all digest comparisons and whether or not the blob container is tampered with.
Triggering an audit
An audit can be triggered by including the following message to the Service Bus Queue associated with your Managed Application:
{
"eventType": "PerformAudit",
"storageAccount": "<storage_account_name>",
"blobContainer": "<blob_container_name>"
}
Be sure to include a Session ID
as the queue has sessions enabled.
Viewing audit results
Once an audit is performed successfully, the results of the audit can be found under a container named <managed-application-name>-audit-records
found within the respective storage account. The results contain the recalculated digest, the digest retrieved from Azure Confidential Ledger and whether or not the blobs are tampered with.
When creating the managed application, if you opt in for email alerts you will get an email sent to your email during an Audit Failure
or an Audit Success and Failure
depending on what option is selected.
Logging and errors
Error logs can be found under a container named <managed-application-name>-error-logs
found within the respective storage account. If a blob creation event or audit process fails, the cause of the failure is recorded and stored in this container. If there are any questions about the error logs or application functionality, contact the Azure Confidential Ledger Support team provided in the Managed Application details.
Clean up managed application
You can delete the Managed Application to clean up and remove all associated resources. Deleting the Managed Application stops all blob transactions from being tracked and stop all digests from being created. Audit reports remain valid for the blobs that were added before the deletion.
More resources
For more information about managed applications and the deployed resources, see the following links: