Project FlashCast – upload and consume blobs simultaneously

An offhand comment by my business counterpart went something like this: "the partner wants to be able to upload an asset to the cloud and start processing it immediately rather than waiting for it to be fully transmitted." I had done something like this with Azure Network Watcher, NSG Flow Logs in this project: Azure Network Watcher NSG Flow Logs Connector. The use case is possible with Azure Blob Storage because of a technical detail: once the block list of an Azure Storage block blob is committed, the blob is readable. Add to this the platform's notification system, Azure Event Grid, and we have all of the required elements.

Find the POC here: Project FlashCast. It is comprised of two Windows console applications written in C# using Visual Studio 2017. "UploadFileConsole" sends a file to Azure blob storage, committing each block as it's transmitted. "DownloadBlobConsole" receives events in the form of Azure storage queue messages and then reads the uploaded blob block by block accordingly.

There are several ways to consume the Azure Event Grid notifications – Azure storage queue is only one of them. There's probably one that fits your needs.

Please add your comments outlining use cases that would be facilitated by this technique.

Cheers!