Walkthrough: Windows Azure Blob Storage (Nov 2009 and later)

Similar to the table storage walkthrough I posted last week, I updated this blog post for the Nov 2009/v1.0 and later release of the Windows Azure Tools.

This walkthrough covers what I found to be the simplest way to get a sample up and running on Windows Azure that uses the Blob Storage service. It is not trying to be comprehensive or trying to dive deep in the technology, it just serves as an introduction to how the Windows Azure Blob Storage Service works.

Please take the Quick Lap Around the Tools before doing this walkthrough.

Note: The code for this walkthrough is attached to this blog post.

After you have completed this walkthrough, you will have a Web Role that is a simple ASP.NET Web Application that shows a list of files that are stored and can be downloaded from Blob Storage. You can use the Web Role to add files to Blob storage and make them available in the list.

Blob Concepts

Each storage account has access to blob storage. For each account there can be 0..n containers. Each container contains the actual blobs, which is a raw byte array. Containers can be public or private. In public containers, the URLs to the blobs can be accessed over the internet while in a private container, only the account holder can access those blob URLs.

Each Blob can have a set of metadata set as a NameValueCollection of strings.

Creating the Cloud Service Project

1. Start Visual Studio as an administrator (Screen shots below are from VS 2008, VS 2010 is also supported)

2. Create a new project: File | New Project

3. Under the Visual C# node (VB is also supported), select the “Cloud Service” project type then select the “Windows Azure Cloud Service” project template. Set the name to be “SimpleBlobSample”. Hit OK to continue.

This will bring up a dialog to add Roles to the Cloud Service.

4. Add an ASP.NET Web Role to the Cloud Service, we’ll use the default name of “WebRole1”.  Hit OK.

Solution explorer should look as follows:

We’ll now cover the implementation, which can be broken up into 5 different parts:

1. Implementing the UI
2. Connecting to Windows Azure storage
3. Adding blobs
4. Enumerating existing blobs
5. Deleting blobs

Implementing the UI

5. Next open up Default.aspx and add the code for the UI. The UI consists of:

• GridView at the top
• Label and FileUpload control
• 2 Label and TextBox pairs (File Name and Submitter)
• Field validators to ensure that all of the fields are filled out before the file is uploaded.

Add the following between the template generated <div></div> elements:

 <asp:GridView ID="fileView" AutoGenerateColumns="false" DataKeyNames="FileUri" runat="server"
    OnRowCommand="RowCommandHandler">
    <Columns>
        <asp:ButtonField Text="Delete" CommandName="DeleteItem" />
        <asp:HyperLinkField HeaderText="Link" DataTextField="FileName" DataNavigateUrlFields="FileUri" />
        <asp:BoundField DataField="Submitter" HeaderText="Submitted by" />
    </Columns>
</asp:GridView>
<br />
<asp:Label ID="filePathLabel" Text="File Path:" AssociatedControlID="fileUploadControl"
    runat="server" />
<asp:FileUpload ID="fileUploadControl" runat="server" />
<asp:RequiredFieldValidator ID="filUploadValidator" ControlToValidate="fileUploadControl"
    ValidationGroup="fileInfoGroup" ErrorMessage="Select a File" runat="Server">
</asp:RequiredFieldValidator>
<br />
<asp:Label ID="fileNameLabel" Text="File Name:" AssociatedControlID="fileNameBox"
    runat="server" />
<asp:TextBox ID="fileNameBox" runat="server" />
<asp:RequiredFieldValidator ID="fileNameValidator" ControlToValidate="fileNameBox"
    ValidationGroup="fileInfoGroup" ErrorMessage="Enter the File Name" runat="Server">
</asp:RequiredFieldValidator>
<br />
<asp:Label ID="submitterLabel" Text="Submitter:" AssociatedControlID="submitterBox"
    runat="server" />
<asp:TextBox ID="submitterBox" runat="server" />
<asp:RequiredFieldValidator ID="submitterValidator" ControlToValidate="submitterBox"
    ValidationGroup="fileInfoGroup" ErrorMessage="Enter the Submitter Name" runat="Server">
</asp:RequiredFieldValidator>
<br />
<asp:Button ID="insertButton" Text="Submit" CausesValidation="true" ValidationGroup="fileInfoGroup"
    runat="server" OnClick="insertButton_Click" />
<br />
<br />
<asp:Label ID="statusMessage" runat="server" />

6. If you now switch to design view, you will see:

You’ll also notice from that aspx that there is an event handler for OnRowCommand on the GridView to handle the DeleteItem command, IDs for the TextBoxes and an event handler for the OnClick event on the Submit button.

The code for these will be filled out further down in the walkthrough.

Connecting to Windows Azure storage

7. Open Default.aspx.cs and add the code to connect to the Blob Storage Service to the Page_Load() method.

 using Microsoft.WindowsAzure;
using Microsoft.WindowsAzure.StorageClient;

namespace WebRole1
{
    public partial class _Default : System.Web.UI.Page
    {
        private CloudBlobClient _BlobClient = null;
        private CloudBlobContainer _BlobContainer = null;

        protected void Page_Load(object sender, EventArgs e)
        {
            // Setup the connection to Windows Azure Storage
            var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
            _BlobClient = storageAccount.CreateCloudBlobClient();

8.  Setup the “DataConnectionString” setting by opening up the configuration UI for WebRole1.  Right click on the WebRole1 node under the Roles folder in the SimpleBlobStorage cloud service project and select “Properties”.

9. Switch to the Settings tab and click “Add Setting”.  Name it DataConnectionString, set the type to ConnectionString and click on the “…” button on the far right.

Hit “OK” to set the credentials to use Development Storage.  We’ll first get this sample working on development storage then convert it to use cloud storage later.

10. If you actually tried to connect to Blob Storage at this point, you would find that the CloudStorageAccount.FromConfigurationSetting() call would fail with the following message:

ConfigurationSettingSubscriber needs to be set before FromConfigurationSetting can be used

This message is in fact incorrect – I have a bug filed to get this fixed, to say “Configuration Setting Publisher” and not “ConfigurationSettingSubscriber”.

To resolve this, we need to add a bit of template code to the WebRole.cs file in WebRole1.  Add the following to the WebRole.OnStart() method in WebRole.cs in the WebRole1 project.

 using Microsoft.WindowsAzure;

 public override bool OnStart()
{
    DiagnosticMonitor.Start("DiagnosticsConnectionString");

    #region Setup CloudStorageAccount Configuration Setting Publisher

    // This code sets up a handler to update CloudStorageAccount instances when their corresponding
    // configuration settings change in the service configuration file.
    CloudStorageAccount.SetConfigurationSettingPublisher((configName, configSetter) =>
    {
        // Provide the configSetter with the initial value
        configSetter(RoleEnvironment.GetConfigurationSettingValue(configName));

        RoleEnvironment.Changed += (sender, arg) =>
        {
            if (arg.Changes.OfType<RoleEnvironmentConfigurationSettingChange>()
                .Any((change) => (change.ConfigurationSettingName == configName)))
            {
                // The corresponding configuration setting has changed, propagate the value
                if (!configSetter(RoleEnvironment.GetConfigurationSettingValue(configName)))
                {
                    // In this case, the change to the storage account credentials in the
                    // service configuration is significant enough that the role needs to be
                    // recycled in order to use the latest settings. (for example, the 
                    // endpoint has changed)
                    RoleEnvironment.RequestRecycle();
                }
            }
        };
    });
    #endregion

    // For information on handling configuration changes
    // see the MSDN topic at https://go.microsoft.com/fwlink/?LinkId=166357.
    RoleEnvironment.Changing += RoleEnvironmentChanging;

    return base.OnStart();
}

The comments (which I wrote and is included in the samples) should explain what is going on if you care to know.  In a nutshell, this code bridges the gap between the Microsoft.WindowsAzure.StorageClient assembly and the Microsoft.WindowsAzure.ServiceRuntime library – the Storage Client library is agnostic to the Windows Azure runtime as it can be used in non Windows Azure applications.

This code essentially says how to get a setting value given a setting name and sets up an event handler to handle setting changes while running in the cloud (because the ServiceConfiguration.cscfg file was updated in the cloud).

The key point is that with this snippet of code in place, you now have everything in place to connect to Windows Azure storage and create a CloudBlobClient instance.

Adding Blobs

11. In order to add blobs to a container, you first need to setup a container.  Let’s add this code to the Page_Load() method in Default.aspx.cs.  For a production application, you will want to optimize this code to avoid doing all this work on every page load.

Page_Load() should now look as follows.

 protected void Page_Load(object sender, EventArgs e)
{
    // Setup the connection to Windows Azure Storage
    var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
    _BlobClient = storageAccount.CreateCloudBlobClient();

    // Get and create the container
    _BlobContainer = _BlobClient.GetContainerReference("publicfiles");
    _BlobContainer.CreateIfNotExist();

    // Setup the permissions on the container to be public
    var permissions = new BlobContainerPermissions();
    permissions.PublicAccess = BlobContainerPublicAccessType.Container;
    _BlobContainer.SetPermissions(permissions);

    // Show the current list.
    UpdateFileList();
}

Note: The container is named with DNS naming restrictions (i.e. all lower case) and is created if it does not exist.  Additionally, the container is set to be a public container – i.e. the URIs to the blobs are accessible by anyone over the internet. 

Had this been a private container, the blobs in that container could only be read by code that has the access key and account name.

12.  Let’s now add the code to Default.aspx.cs to add a blob when the “Submit” button on the UI is clicked (remember the event handler was defined in the aspx).

A GUID is created for the file name to ensure a unique blob name is used.  The file name and submitter are gotten from the TextBoxes in the UI.

Blob Metadata, or user defined key/value pairs, is used to store the file name and submitter along with the blob. 

 protected void insertButton_Click(object sender, EventArgs e)
{
    // Make a unique blob name
    string extension = System.IO.Path.GetExtension(fileUploadControl.FileName);

    // Create the Blob and upload the file
    var blob = _BlobContainer.GetBlobReference(Guid.NewGuid().ToString() + extension);
    blob.UploadFromStream(fileUploadControl.FileContent);

    // Set the metadata into the blob
    blob.Metadata["FileName"] = fileNameBox.Text;
    blob.Metadata["Submitter"] = submitterBox.Text;
    blob.SetMetadata();

    // Set the properties
    blob.Properties.ContentType = fileUploadControl.PostedFile.ContentType;
    blob.SetProperties();

    // Update the UI
    UpdateFileList();
    fileNameBox.Text = "";
    statusMessage.Text = "";
}

Enumerating Existing Blobs

13. In order to simplify the databinding to the UI, lets add a FileEntry class that contains the data we want to show in the UI for each blob.  One instance of a FileEntry corresponds to a blob. 

Right click on WebRole1 and select Add | Class…

Name the class FileEntry.cs and hit OK.

14. Fill out FileEntry.cs with the following code:

 public class FileEntry
{
    public Uri FileUri { get; set; }
    public string FileName { get; set; }
    public string Submitter { get; set; }
}

15. Back to Default.aspx.cs, let’s add code to populate the GridView by getting the collection of blobs from ListBlobs() and creating a FileEntry for each item.

FetchAttributes() is used to retrieve the blob metadata.

 using System.Collections.Generic;
using System.Collections.Specialized;

 private void UpdateFileList()
{
    // Get a list of the blobs
    var blobs = _BlobContainer.ListBlobs();
    var filesList = new List<FileEntry>();

    // For each item, create a FileEntry which will populate the grid
    foreach (var blobItem in blobs)
    {
        var cloudBlob = _BlobContainer.GetBlobReference(blobItem.Uri.ToString());
        cloudBlob.FetchAttributes();

        filesList.Add(new FileEntry() { 
            FileUri = blobItem.Uri,
            FileName = cloudBlob.Metadata["FileName"],
            Submitter = cloudBlob.Metadata["Submitter"]
        });    
    }
    
    // Bind the grid
    fileView.DataSource = filesList;
    fileView.DataBind();
}

Deleting Blobs

16. Add code to delete the blob in the row command handler that was setup in the aspx. This is as simple as calling CloudBlobContainer.DeleteIfExists() for the blob where the CloudBlob instance is gotten from the Guid + file extension generated during the upload.

 protected void RowCommandHandler(object sender, GridViewCommandEventArgs e)
{
    if (e.CommandName == "DeleteItem")
    {
        var index = Convert.ToInt32(e.CommandArgument);
        var blobName = (string)fileView.DataKeys[index].Value;
        var blobContainer = _BlobClient.GetContainerReference("publicFiles");
        var blob = blobContainer.GetBlobReference(blobName);
        blob.DeleteIfExists();
    }

    // Update the UI
    UpdateFileList();
}

Testing the Application

17. Build and hit F5 to run the application.

Note: The FileUpload control has a file size limit. You can modify it by changing the httpRuntime maxRequestLength attribute in web.config.

Moving from Development Storage to Cloud Storage

18. Now I want to switch this to use Windows Azure Storage, not the development storage.  The first step is to go to the Windows Azure Developer Portal and create a storage account.

Login and click on “New Service”, and select “Storage Account”:

Fill out the service name, the public name and optionally choose a region.  You will be brought to a page that contains the following (note I rubbed out the access keys):

19. You will use the first part of the endpoint, (jnakstorageaccount) and the one of the access keys to fill out your connection string.

20. Open the WebRole1 config again, bring up Settings | DataConnectionString and fill out the account name and the account key and hit OK.

21. Hit F5 to run the application again. 

This time you will be running against cloud storage – note that the data you entered when running against the development storage is no longer there.

Important Note: Before deploying to the cloud, the DiagnosticsConnectionString also needs to use storage account credentials and not the development storage account.

Please see the Deploying a Cloud Service walkthrough to learn how to deploy this to the Windows Azure cloud.

For more information, please see the Blob Service API documentation and the Programming Blob Storage white paper.

I know that there are a number of different concepts that have to be pieced together, hopefully this walkthrough has been helpful in understand how everything fits together.

SimpleBlobSample.zip

Comments

• Anonymous
  January 12, 2010
  This would've been of great help a few days ago back when I was doing my Storage Explorer. (http://storageexplorer.cloudapp.net) Great post though :)

• Anonymous
  January 12, 2010
  Great walkthrough, thanks! First one I've found that accurately applied to the latest APIs and was well written enough to follow- I'm finally creating blobs and containers, seems so easy now :)

• Anonymous
  January 13, 2010
  seab.gomez, rabelBodhi -- thanks so much, glad it was a help for you.

• Anonymous
  March 19, 2010
  Once again the best tutorial on the given subject (Azure Storage Blobs).   Good developers also make good documentation writers - since we know what is important and we care about the technology. Microsoft should have real developers in its MSDN staff. ;-)

• Anonymous
  March 19, 2010
  Thanks ehuna - much appreciated.  

• Anonymous
  April 28, 2010
  I have a question about the work-around you have regarding this error: ConfigurationSettingSubscriber needs to be set before FromConfigurationSetting can be used I used this trick to successfully make a cloud worker role project to upload/download stuff from my cloud storage blob. However, I tried to put a silverlight web service onto the cloud, using the instructions here: http://davidburela.wordpress.com/2008/12/10/silverlight-hoste-on-windows-azure/ I'm having trouble debugging this application inside the Azure sandbox environment (but with a data connection string pointing at my blob storage account in azure). I get exactly the error you mention, even though I have the code that you told us to put, inserted into the webrole.cs file of the web service that my silverlight web app calls. Any ideas on why I might still be seeing this error?

• Anonymous
  April 29, 2010
  Dipankar, Can you set a breakpoint on SetConfigurationSettingPublisher and the call to FromConfigurationSetting?   The only reason for the issue you are seeing is if that code isn't getting run before the call to FromConfigurationSetting(), so please double check that. You can also use CloudStorageAccount.Parse() which is a simpler way to get things working but you lose the ability to rotate the storage account key by updating the service configuration file.

• Anonymous
  September 07, 2010
  The comment has been removed

• Anonymous
  September 10, 2010
  Sunanda, you can explicitly set the container to be private by setting the BlobContainerPublicAccessType to Off. What this will do will make it so that you can only access blobs in that container via the API and not by URL.  

• Anonymous
  January 12, 2011
  This no longer works with SDK 1.3 and VS 2010, maybe you could reply here if you know the problem? social.msdn.microsoft.com/.../fb381f8f-19ce-4a90-8004-bc2c87c8d4c8 Thank you!

• Anonymous
  February 14, 2011
  Nice work Jnak, But instead of doing STEP 9 (9. Switch to the Settings tab and click “Add Setting”.  Name it DataConnectionString,...) and doing a work around for the webrole's onStart(), we can simple do a CloudStorageAccount myStorageAccount = CloudStorageAccount.DevelopmentStorageAccount; and continue as it is...? I use this method, what are the merits and demerits of using this over yours...?

• Anonymous
  February 14, 2011
  Naveen - the problem with using dev storage programmatically is that you'll have to change your code to run in the cloud where there is no dev storage. Also, as other people have mentioned, with the release of 1.3, it's better (i.e. it'll work) to use CloudStorageAccount.Parse(RoleEnvironment.GetConfigurationSetting(mysetting)) instead of FromConfigurationSetting.

• Anonymous
  July 07, 2011
  Hi, I'm having trouble uploading files to our website that runs on multiple instances on azure. I get file not found when the code tries to process the file i just uploaded.. any ideas to fix that ?

• Anonymous
  July 08, 2011
  SvovelSyre - uploading files where?  To storage?  You can use the azure storage viewer in VS to see the files once you upload them which will ensure that you uploaded them correctly.  If you mean you uploaded them to the temp storage space on the VM, make sure you are using the local storage APIs.

• Anonymous
  September 11, 2011
  Am new to Windows Azure Storage and am learning it through this documentation. Its My first try. But the source code is giving a runtime error which i have no idea how to fix that. I'd really appreciate any help. "ConfigurationSettingSubscriber needs to be set before FromConfigurationSetting can be used"

• Anonymous
  September 13, 2011
  For everyone having issues with the exception: "SetConfigurationSettingPublisher needs to be called before FromConfigurationSetting can be used", According to this blog post, the setting needs to be in a global.asax file for Azure SDK 1.3. This fix worked for me. blogs.msdn.com/.../how-to-resolve-setconfigurationsettingpublisher-needs-to-be-called-before-fromconfigurationsetting-can-be-used-after-moving-to-windows-azure-sdk-1-3.aspx

• Anonymous
  September 14, 2011
  It worked just fine. Thanks

• Anonymous
  December 22, 2011
  Nice explanation of blob storage with code! (http://www.arunrana.net)

• Anonymous
  February 16, 2013
  Hey! I’m just getting started with my first Azure app. Here’s something which I wanted to implement as my first application: I want to create a service on Microsoft Azure using Visual Studio 2012 which enables various users to upload and edit their documents. I’ve found some tutorials online but all of them include downloading that particular document and then uploading it back again on the cloud. I want the users to be able to edit it online without the need of downloading the document and then submitting it after the required changes are made. Please let me know if there are any useful links which I can refer to. Thanks a lot.

• Anonymous
  February 17, 2013
  Pallavi -- I'd have a look at http://www.excelmashup.com/ you can embe Excel in your app and use that to edit Excel documents.  The other thing you can do is store the documents on skydrive and enable editing that way.  If you are in a SharePoint scenario, you can also enable editing of documents online by storing the document on SharePoint and launching the web app versions Excel, Word, PowerPoint and so on.  Hope that helps.

• Anonymous
  February 17, 2013
  I’m just getting started with my first Azure app. Here’s something which I wanted to implement in my first application:

1. The first page allows various users to login, each user given their own username.
2. Once they’ve logged in, they would be able to upload their documents which will be stored in the Windows Azure Blob. Each document would have a particular id.
3. All the documents which have been uploaded on the Blob will be visible to all the users. However, if a user wishes to edit a particular document, he will need the document id to access it.
4. Once he’s given access, he’ll be able to edit the document just like Google docs and save changes after he’s done.
5. The last feature is something which will depend on how quickly I get the first 4 steps done. I was earlier thinking of collaborative editing but that would be a little out of my scope right now as I’m a beginner. An alternative to this would be to restrict access to a user if he’s trying to open a document which has already been opened by some other user. I would like to implement all of this on Microsoft Visual Studio 2012 using Visual C# and ASP.net. I’ve already created the free Azure cloud. Please let me know how to get started or if any of you know any useful links which I can refer to. Thanks a lot.

• Anonymous
  February 17, 2013
  Pallavi, the Word & Excel web apps only enable editing from Skydrive or SharePoint, not from Windows Azure Blob storage.  You could manually sync the document between skydrive and blob storage and would need the user to login to skydrive.  Not the best solution but that's all I can think of.  I'd look at the docs at msdn.microsoft.com/.../hh826521

• Anonymous
  February 20, 2013
  Hi Jim I'll be trying your tutorial on Visual Studio 2012. Is there anything I should keep in mind while following the steps mentioned above? Also, once the document has been uploaded in the Azure Blob, I would like users to edit this document on the web (without downloading it) and saving the changes once they're done. Any links to other such tutorials will be really helpful. :)

• Anonymous
  February 22, 2013
  Hi Dhvanil, There have been a bunch of changes to Azure Storage since Jim published the blog post. I assume that you are using Visual Studio 2012 and the Azure SDK 1.8, aka October 2012 release.  In this release we actually ship two versions of the storage client library. The one that is used by default in the Cloud Services templates has the version 1.7 and Azure storage access works the same way as described in the article. There is also a newer version of the Storage library available through Nuget, which can be installed through the packaging manager. This library introduces a bunch of new functionality and the code shown by Jim would need to be revised to work with the new objects. You can find more information on the new library here: blogs.msdn.com/.../introducing-windows-azure-storage-client-library-2-0-for-net-and-windows-runtime.aspx Also we have implemented improvements in the tools itself, for example the Publishing to Azure experience is way smoother now. You can read up on the new tooling features here: msdn.microsoft.com/.../jj618299.aspx Hope this helps Thanks Boris

• Anonymous
  February 24, 2013
  So I followed the tutorial and it still works without making any major changes (The link by Chris was really handy). I just wanted to add one more functionality to my application. I want the users to be able to edit the document on the webpage itself (just like Google docs) and save the changes after he’s done. It would be really helpful if you could elaborate on how to go about it or post links for the same. Thanks! :)

• Anonymous
  April 03, 2013
  hi i am getting following error at  _BlobContainer.CreateIfNotExist(); statement "storageserverexception was unhandled by user code" and then build failed. please reply ASAP thanks