Introducing Windows Azure Storage Client Library 2.0 for .NET and Windows Runtime

Today we are releasing version 2.0 of the Windows Azure Storage Client Library. This is our largest update to our .NET library to date which includes new features, broader platform compatibility, and revisions to address the great feedback you’ve given us over time. The code is available on GitHub now. The libraries are also available through NuGet, and also included in the Windows Azure SDK for .NET - October 2012; for more information and links see below. In addition to the .NET 4.0 library, we are also releasing two libraries for Windows Store apps as Community Technology Preview (CTP) that fully supports the Windows Runtime platform and can be used to build modern Windows Store apps for both Windows RT (which supports ARM based systems), and Windows 8, which runs in any of the languages supported by Windows Store apps (JavaScript, C++, C#, and Visual Basic). This blog post serves as an overview of these libraries and covers some of the implementation details that will be helpful to understand when developing cloud applications in .NET regardless of platform.

What’s New

We have introduced a number of new features in this release of the Storage Client Library including:

• Simplicity and Usability - A greatly simplified API surface which will allow developers new to storage to get up and running faster while still providing the extensibility for developers who wish to customize the behavior of their applications beyond the default implementation.
• New Table Implementation - An entirely new Table Service implementation which provides a simple interface that is optimized for low latency/high performance workloads, as well as providing a more extensible serialization model to allow developers more control over their data.
• Rich debugging and configuration capabilities – One common piece of feedback we receive is that it’s too difficult to know what happened “under the covers” when making a call to the storage service. How many retries were there? What were the error codes? The OperationContext object provides rich debugging information, real-time status events for parallel and complex actions, and extension points allowing users the ability to customize requests or enable end to end client tracing
• Windows Runtime Support - A Windows Runtime component with support for developing Windows Store apps using JavaScript, C++,C#, and Visual Basic; as well as a Strong Type Tables Extension library for C++, C#, and Visual Basic
• Complete Sync and Asynchronous Programming Model (APM) implementation - A complete Synchronous API for .Net 4.0. Previous releases of the client implemented synchronous methods by simply surrounding the corresponding APM methods with a ManualResetEvent, this was not ideal as extra threads remained blocked during execution. In this release all synchronous methods will complete work on the thread in which they are called with the notable exceptions of the stream implementations available via Cloud[Page|Block]Blob.Open[Read|Write] due to parallelism.
• Simplified RetryPolicies - Easy and reusable RetryPolicies
• .NET Client Profile – The library now supports the .NET Client Profile. For more on the .Net Client Profile see here.
• Streamlined Authentication Model - There is now a single StorageCredentials type that supports Anonymous, Shared Access Signature, and Account and Key authentication schemes
• Consistent Exception Handling - The library immediately will throw any exception encountered prior to making the request to the server. Any exception that occurs during the execution of the request will subsequently be wrapped inside a single StorageException type that wraps all other exceptions as well as providing rich information regarding the execution of the request.
• API Clarity - All methods that make requests to the server are clearly marked with the [DoesServiceRequest] attribute
• Expanded Blob API - Blob DownloadRange allows user to specify a given range of bytes to download rather than rely on a stream implementation
• Blob download resume - A feature that will issue a subsequent range request(s) to download only the bytes not received in the event of a loss of connectivity
• Improved MD5 - Simplified MD5 behavior that is consistent across all client APIs
• Updated Page Blob Implementation - Full Page Blob implementation including read and write streams
• Cancellation - Support for Asynchronous Cancellation via the ICancellableAsyncResult. Note, this can be used with .NET CancellationTokens via the CancellationToken.Register() method.
• Timeouts - Separate client and server timeouts which support end to end timeout scenarios
• Expanded Azure Storage Feature Support – It supports the 2012-02-12 REST API version with implementation for for Blob & Container Leases, Blob, Table, and Queue Shared Access Signatures, and Asynchronous Cross-Account Copy Blob

Design

When designing the new Storage Client for .NET and Windows Runtime, we set up a series of design guidelines to follow throughout the development process. In addition to these guidelines, there are some unique requirements when developing for Windows Runtime, and specifically when projecting into JavaScript, that has driven some key architectural decisions.

For example, our previous RetryPolicy was based on a delegate that the user could configure; however as this cannot be supported on all platforms we have redesigned the RetryPolicy to be a simple and consistent implementation everywhere. This change has also allowed us to simplify the interface in order to address user feedback regarding the complexity of the previous implementation. Now a user who constructs a custom RetryPolicy can re-use that same implementation across platforms.

Windows Runtime

A key driver in this release was expanding platform support, specifically targeting the upcoming releases of Windows 8, Windows RT, and Windows Server 2012. As such, we are releasing the following two Windows Runtime components to support Windows Runtime as Community Technology Preview (CTP):

• Microsoft.WindowsAzure.Storage.winmd - A fully projectable storage client that supports JavaScript, C++, C#, and VB. This library contains all core objects as well as support for Blobs, Queues, and a base Tables Implementation consumable by JavaScript
• Microsoft.WindowsAzure.Storage.Table.dll – A table extension library that provides generic query support and strong type entities. This is used by non-JavaScript applications to provide strong type entities as well as reflection based serialization of POCO objects

Breaking Changes

With the introduction of Windows 8, Windows RT, and Windows Server 2012 we needed to broaden the platform support of our current libraries. To meet this requirement we have invested significant effort in reworking the existing Storage Client codebase to broaden platform support, while also delivering new features and significant performance improvements (more details below). One of the primary goals in this version of the client libraries was to maintain a consistent API across platforms so that developer’s knowledge and code could transfer naturally from one platform to another. As such, we have introduced some breaking changes from the previous version of the library to support this common interface. We have also used this opportunity to act on user feedback we have received via the forums and elsewhere regarding both the .Net library as well as the recently released Windows Azure Storage Client Library for Java. For existing users we will be posting an upgrade guide for breaking changes to this blog that describes each change in more detail.

Please note the new client is published under the same NuGet package as previous 1.x releases. As such, please check any existing projects as an automatic upgrade will introduce breaking changes.

Additional Dependencies

The new table implementation depends on three libraries (collectively referred to as ODataLib), which are resolved through the ODataLib (version 5.0.2) packages available through NuGet and not the WCF Data Services installer which currently contains 5.0.0 versions.  The ODataLib libraries can be downloaded directly or referenced by your code project through NuGet.  The specific ODataLib packages are:

https://nuget.org/packages/Microsoft.Data.OData/5.0.2

https://nuget.org/packages/Microsoft.Data.Edm/5.0.2

https://nuget.org/packages/System.Spatial/5.0.2

Namespaces

One particular breaking change of note is that the name of the assembly and root namespace has moved to Microsoft.WindowsAzure.Storage instead of Microsoft.WindowsAzure.StorageClient.In addition to aligning better with other Windows Azure service libraries this change allows developers to use the legacy 1.X versions of the library and the 2.0 release side-by-side as they migrate their applications. Additionally, each Storage Abstraction (Blob, Table, and Queue) has now been moved to its own sub-namespace to provide a more targeted developer experience and cleaner IntelliSense experience. For example the Blob implementation is located in Microsoft.WindowsAzure.Storage.Blob , and all relevant protocol constructs are located in Microsoft.WindowsAzure.Storage.Blob.Protocol.

Testing, stability, and engaging the open source community

We are committed to providing a rock solid API that is consistent, stable, and reliable. In this release we have made significant progress in increasing test coverage as well as breaking apart large test scenarios into more targeted ones that are more consumable by the public.

Microsoft and Windows Azure are making great efforts to be as open and transparent as possible regarding the client libraries for our services. The source code for all the libraries can be downloaded via GitHub under the Apache 2.0 license. In addition we have provided over 450 new Unit Tests for the .Net 4.0 library alone. Now users who wish to modify the codebase have a simple and light weight way to validate their changes. It is also important to note that most of these tests run against the Storage Emulator that ships via the Windows Azure SDK for .NET allowing users to execute tests without incurring any usage on their storage accounts. We will also be providing a series of higher level scenarios and How-To’s to get users up and running both simple and advanced topics relating to using Windows Azure Storage.

Summary

We have put a lot of work into providing a truly first class development experience for the .NET community to work with Windows Azure Storage. In addition to the content provided in these blog posts we will continue to release a series of additional blog posts which will target various features and scenarios in more detail, so check back soon. Hopefully you can see your past feedback reflected in this new library. We really do appreciate the feedback we have gotten from the community, so please keep it coming by leaving a comment below or participating on our forums.

Joe Giardino
Serdar Ozler
Justin Yu
Veena Udayabhanu

Windows Azure Storage

Resources

Get the Windows Azure SDK for .Net

• Download the Windows Azure SDK
• Windows Azure SDK for .Net source code on GitHub
• Nuget Package

Comments

• Anonymous
  November 03, 2012
  The comment has been removed

• Anonymous
  December 11, 2012
  Thank you for your release. I am getting the following errors when submitting my WINRT app. to the Windows 8 store: ◦Type Microsoft.WindowsAzure.Storage.Core.NullType in file Microsoft.WindowsAzure.Storage.winmd is not sealed and does not have the ComposableAttribute. Unsealed types must have ComposableAttribute. ◦Type Microsoft.WindowsAzure.Storage.Queue.Protocol.QueueMessage in file Microsoft.WindowsAzure.Storage.winmd is not sealed and does not have the ComposableAttribute. Unsealed types must have ComposableAttribute.

• Anonymous
  December 11, 2012
  @David - Thank you for reporting this issue. We apologize for the inconvenience. This is a known issue and the fix is already in testing. Storage Client Library 2.0.3 release that includes the fix will be released soon. For more information on known issues, please refer to blogs.msdn.com/.../updated-known-issues-for-windows-azure-storage-client-library-2-0-for-net-and-windows-runtime.aspx

• Anonymous
  January 13, 2013
  I am getting error The remote server returned an error: (400) Bad Request Here is my code CloudTableClient tableClient;        protected void Unnamed_Click(object sender, EventArgs e)        {            var storageAccount = CloudStorageAccount.DevelopmentStorageAccount;            tableClient = storageAccount.CreateCloudTableClient();            CloudTable myTable = tableClient.GetTableReference("MyTable");            myTable.CreateIfNotExists();            TableOperation insertHuman = TableOperation.Insert(new Human() { Name="ABC",Age=23});            myTable.Execute(insertHuman);            TableQuery<TableEntity> query = new TableQuery<TableEntity>().Where(TableQuery.GenerateFilterCondition("RowKey", QueryComparisons.GreaterThanOrEqual, "1")).Take(5);        } and This is entity public class Human:TableEntity    {        public string Name { get; set; }        public int Age { get; set; }    }

• Anonymous
  January 14, 2013
  Hi Omair, you need to set the PartitionKey and RowKey properties of the entity before inserting it into a table. This is why you are seeing the 400 response.

• Anonymous
  March 16, 2013
  I m using Microsoft.WindowsAzure.Storage.dll, version 2.0 , runtime version v4.0.30319 We get Serialization Error ========================== <Error> <Message>An error has occurred.</Message> <ExceptionMessage> The 'ObjectContent`1' type failed to serialize the response body for content type 'application/xml; charset=utf-8'. </ExceptionMessage> <ExceptionType>System.InvalidOperationException</ExceptionType> <StackTrace/> <InnerException> <Message>An error has occurred.</Message> <ExceptionMessage> Type 'Microsoft.WindowsAzure.Storage.Table.EntityProperty' cannot be serialized. Consider marking it with the DataContractAttribute attribute, and marking all of its members you want serialized with the DataMemberAttribute attribute. If the type is a collection, consider marking it with the CollectionDataContractAttribute. See the Microsoft .NET Framework documentation for other supported types. </ExceptionMessage> <ExceptionType> System.Runtime.Serialization.InvalidDataContractException </ExceptionType> </InnerException> </Error> and here is Our MOdel ======================== public  class DomainEntity:TableEntity    {        private DateTime _CreatedOn;        public DateTime CreatedOn        {            get { return _CreatedOn; }            set { _CreatedOn = value; }        }        public DomainEntity() {_CreatedOn = DateTime.Now;}        public string CreatedBy { get; set; }        public DateTime? UpdatedOn { get; set; }        public string UpdatedBy { get; set; }        private int _RowSate;        public int RowState{get { return _RowSate; }set { _RowSate = value; }}    } public class Group :DomainEntity    {        public Group(){}        public Group(int groupId)        {          this.PartitionKey="GroupName";          this.RowKey = groupId.ToString();        }        public string GroupName { get; set; }    } And I USE ASP.net mvc 4 web api as web role =========================================== public IQueryable<DynamicTableEntity> Get() {            TableQuery query = new TableQuery().Select(new List<string>() { "CreatedBy", "GroupName" });            var myTable = GetTableReference("Group");            List<DynamicTableEntity> L = new List<DynamicTableEntity>();              return myTable.ExecuteQuery(query).Select(x => x).AsQueryable<DynamicTableEntity>(); }

• Anonymous
  March 19, 2013
  The comment has been removed

• Anonymous
  March 19, 2013
  FYI, a heterogeneous query is when you are dealing with different types being stored in the same table and returned as the query result.

• Anonymous
  June 30, 2013
  Hi there.  Any tips on how to see if DevelopmentStorage is running (Windows Store App)?  Any way for my App to start it?

• Anonymous
  July 15, 2013
  Hi Larry, the best way to see if the Storage Emulator is running is to try making an API call such as List Containers. To start the Storage Emulator, you’ll need to invoke the following command:    csrun.exe /devstore:start Csrun.exe is distributed with the Windows Azure SDK (normally C:Program FilesMicrosoft SDKsWindows AzureEmulatorcsrun.exe). Note that the command for starting the emulator may change in future SDK releases.

• Anonymous
  September 13, 2013
  I like the new client library API, and I don't mind break changes on API signature. Just it won't work with local Windows Azure Simulator anymore. I get "No valid combination of account information found" when try to parse StorageConnectionString with usedevelopmentstorage=true.

• Anonymous
  September 16, 2013
  Hey Ryan, Thanks for the feedback. The string is case sensitive currently and hence you should use "UseDevelopmentStorage=true". Sorry for the inconvenience. We will take it as a feature request to relax this restriction in a future release.