Freigeben über

Windows Azure Storage Client Library for C++ Preview

  • Artikel

We are excited to announce the availability of our new Windows Azure Storage Client Library for C++. This is currently a Preview release, which means the library should not be used in your production code just yet. Instead, please kick the tires and give us any feedback you might have for us to improve/change the interface based on feedback for the GA release. This blog post serves as an overview of the library.

Please refer to SOSP Paper - Windows Azure Storage: A Highly Available Cloud Storage Service with Strong Consistency for more information on Windows Azure Storage.

Emulator Guidance

Please note, that the 2013-08-15 REST version, which this library utilizes, is currently unsupported by the storage emulator. An updated Windows Azure Storage Emulator is expected to ship with full support of these new features in the next month. Users attempting to develop against the current version of the Storage emulator will receive Bad Request errors in the interim.  Until then, users wanting to use the new features would need to develop and test against a Windows Azure Storage Account to leverage the 2013-08-15 REST version.

Supported Platforms

In this release, we provide x64 and x86 versions of the library for both Visual Studio 2012 (v110) and Visual Studio 2013 (v120) platform toolsets. Therefore you will find 8 build flavors in the package:

  1. Release, x64, v120
  2. Debug, x64, v120
  3. Release, Win32, v120
  4. Debug, Win32, v120
  5. Release, x64, v110
  6. Debug, x64, v110
  7. Release, Win32, v110
  8. Debug, Win32, v110

Where is it?

The library can be downloaded from NuGet and full source code is available on GitHub. NuGet packages are created using the CoApp tools and therefore consist of 3 separate packages:

  • wastorage.0.2.0-preview.nupkg: This package contains header and LIB files required to develop your application. This is the package you need to install, which has a dependency on the redist package and thus will force NuGet to install that one automatically.
  • wastorage.redist.0.2.0-preview.nupkg: This package contains DLL files required to run and redistribute your application.
  • wastorage.symbols.0.2.0-preview.nupkg: This package contains symbols for the respective DLL files and therefore is an optional package.

The package also contains a dependency on C++ REST SDK, which will also be automatically installed by NuGet. The C++ REST SDK (codename "Casablanca") is a Microsoft project for cloud-based client-server communication in native code and provides support for accessing REST services from native code on multiple platforms by providing asynchronous C++ bindings to HTTP, JSON, and URIs. Windows Azure Storage Client Library uses it to communicate with the Windows Azure Storage Blob, Queue, and Table services.

What do you get?

Here is a summary of the functionality you will get by using Windows Azure Storage Client Library instead of directly talking to the REST API:

  • Easy-to-use implementations of the entire Windows Azure Storage REST API version 2013-08-15
  • Retry policies that retry certain failed requests using an exponential or linear back off algorithm
  • Streamlined authentication model that supports both Shared Keys and Shared Authentication Signatures
  • Ability to dive into the request details and results using an operation context and ETW logging
  • Blob uploads regardless of size and blob type, and parallel block/page uploads configurable by the user
  • Blob streams that allow reading from or writing to a blob without having to deal with specific upload/download APIs
  • Full MD5 support across all blob upload and download APIs
  • Table layer that uses the new JSON support on Windows Azure Storage announced in November 2013
  • Entity Group Transaction support for Table service that enables multiple operations within a single transaction

Support for Read Access Geo Redundant Storage

This release has full support for Read Access to the storage account data in the secondary region. This functionality needs to be enabled via the portal for a given storage account, you can read more about RA-GRS here.

How to use it?

After installing the NuGet package, all header files you need to use will be located in a folder named “was” (stands for Windows Azure Storage). Under this directory, the following header files are critical:

  • blob.h: Declares all types related to Blob service
  • queue.h: Declares all types related to Queue service
  • table.h: Declares all types related to Table service
  • storage_account.h: Declares the cloud_storage_account type that can be used to easily create service client objects using an account name/key or a connection string
  • retry_policies.h: Declares different retry policies available to use with all operations

So, you can start with including the headers you are going to use:

 #include "was/storage_account.h"
#include "was/queue.h"
#include "was/table.h"
#include "was/blob.h"

Then we will create a cloud_storage_account object, which enables us to create service client objects later in the code. Please note that we are using https below for a secure connection, but http is very useful when you are debugging your application.

 wa::storage::cloud_storage_account storage_account = wa::storage::cloud_storage_account::parse(U("AccountName=<account_name>;AccountKey=<account_key>;DefaultEndpointsProtocol=https"));

Blobs

Here we create a blob container, a blob with “some text” in it, download it, and then list all the blobs in our container:

 // Create a blob container
wa::storage::cloud_blob_client blob_client = storage_account.create_cloud_blob_client();
wa::storage::cloud_blob_container container = blob_client.get_container_reference(U("mycontainer"));
container.create_if_not_exists();

// Upload a blob
wa::storage::cloud_block_blob blob1 = container.get_block_blob_reference(U("myblob"));
blob1.upload_text(U("some text"));

// Download a blob
wa::storage::cloud_block_blob blob2 = container.get_block_blob_reference(U("myblob"));
utility::string_t text = blob2.download_text();

// List blobs
wa::storage::blob_result_segment blobs = container.list_blobs_segmented(wa::storage::blob_continuation_token());

Tables

The sample below creates a table, inserts an entity with couple properties of different types, and finally retrieves that specific entity. In the first retrieve operation, we do a point query and retrieve the specific entity. In the query operation, on the other hand, we query all entities with PartitionKey is equal to “partition” and RowKey is greater than or equal to “m”, which will eventually get us the original entity we inserted.

For more information on Windows Azure Tables, please refer to the Understanding the Table Service Data Model article and the How to get most out of Windows Azure Tables blog post.

 // Create a table
wa::storage::cloud_table_client table_client = storage_account.create_cloud_table_client();
wa::storage::cloud_table table = table_client.get_table_reference(U("mytable"));
table.create_if_not_exists();

// Insert a table entity
wa::storage::table_entity entity(U("partition"), U("row"));
entity.properties().insert(wa::storage::table_entity::property_type(U("PropertyA"), wa::storage::table_entity_property(U("some string"))));
entity.properties().insert(wa::storage::table_entity::property_type(U("PropertyB"), wa::storage::table_entity_property(utility::datetime::utc_now())));
entity.properties().insert(wa::storage::table_entity::property_type(U("PropertyC"), wa::storage::table_entity_property(utility::new_uuid())));
wa::storage::table_operation operation1 = wa::storage::table_operation::insert_or_replace_entity(entity);
wa::storage::table_result table_result = table.execute(operation1);

// Retrieve a table entity
wa::storage::table_operation operation2 = wa::storage::table_operation::retrieve_entity(U("partition"), U("row"));
wa::storage::table_result result = table.execute(operation2);

// Query table entities
wa::storage::table_query query;
query.set_filter_string(wa::storage::table_query::combine_filter_conditions(
    wa::storage::table_query::generate_filter_condition(U("PartitionKey"), wa::storage::query_comparison_operator::equal, U("partition")), 
    wa::storage::query_logical_operator::and, 
    wa::storage::table_query::generate_filter_condition(U("RowKey"), wa::storage::query_comparison_operator::greater_than_or_equal, U("m"))));
std::vector<wa::storage::table_entity> results = table.execute_query(query);

Queues

In our final example, we will create a queue, add a message to it, retrieve the same message, and finally update it:

 // Create a queue
wa::storage::cloud_queue_client queue_client = storage_account.create_cloud_queue_client();
wa::storage::cloud_queue queue = queue_client.get_queue_reference(U("myqueue"));
queue.create_if_not_exists();

// Add a queue message
wa::storage::cloud_queue_message message1(U("mymessage"));
queue.add_message(message1);

// Get a queue message
wa::storage::cloud_queue_message message2 = queue.get_message();

// Update a queue message
message2.set_content(U("changedmessage"));
queue.update_message(message2, std::chrono::seconds(30), true);

How to debug it?

When things go wrong, you might get an exception from one of your calls. This exception will be of type wa::storage::storage_exception and contain detailed information about what went wrong. Consider the following code:

 try
{
    blob1.download_attributes();
}
catch (const wa::storage::storage_exception& e)
{
    std::cout << "Exception: " << e.what() << std::endl;
    ucout << U("The request that started at ") << e.result().start_time().to_string() << U(" and ended at ") << e.result().end_time().to_string() << U(" resulted in HTTP status code ") << e.result().http_status_code() << U(" and the request ID reported by the server was ") << e.result().service_request_id() << std::endl;
}

When run on a non-existing blob, this code will print out:

Exception: The specified blob does not exist.

The request that started at Fri, 13 Dec 2013 18:31:11 GMT and ended at Fri, 13 Dec 2013 18:31:11 GMT resulted in HTTP status code 404 and the request ID reported by the server was 5de65ae4-9a71-4b1d-9c99-cc4225e714c6

The library also provides the type wa::storage::operation_context, which is supported by all APIs, to obtain more information about what is being done during an operation. Now consider the following code:

wa::storage::operation_context context;

context.set_sending_request([] (web::http::http_request& request, wa::storage::operation_context)

{

ucout << U("The request is being sent to ") << request.request_uri().to_string() << std::endl;

});

context.set_response_received([] (web::http::http_request&, const web::http::http_response& response, wa::storage::operation_context)

{

ucout << U("The reason phrase is ") << response.reason_phrase() << std::endl;

});

try

{

blob1.download_attributes(wa::storage::access_condition(), wa::storage::blob_request_options(), context);

}

catch (const wa::storage::storage_exception& e)

{

std::cout << "Exception: " << e.what() << std::endl;

}

ucout << U("Executed ") << context.request_results().size() << U(" request(s) to perform this operation and the last request's status code was ") << context.request_results().back().http_status_code() << std::endl;

Again, when run on a non-existing blob, this code will print out:

The request is being sent to https://myaccount.blob.core.windows.net/mycontainer/myblob?timeout=90

The reason phrase is The specified blob does not exist.

Exception: The specified blob does not exist.

Executed 1 request(s) to perform this operation and the last request's status code was 404

Samples

We have provided sample projects on GitHub to help get you up and running with each storage abstraction and to illustrate some additional key scenarios. All sample projects are found under the folder named “samples”.

Open the samples solution file named “Microsoft.WindowsAzure.Storage.Samples.sln” in Visual Studio. Update your storage credentials in the samples_common.h file under the Microsoft.WindowsAzure.Storage.SamplesCommon project. Go to the Solution Explorer window and select the sample project you want to run (for example, Microsoft.WindowsAzure.Storage.BlobsGettingStarted) and choose “Set as StartUp Project” from the Project menu (or, alternatively, right-click the project, then choose the same option from the context menu).

Summary

We welcome any feedback you may have in the comments section below, the forums, or GitHub. If you hit any bugs, filing them on GitHub will also allow you to track the resolution.

Serdar Ozler, Mike Fisher, and Joe Giardino

Resources

Source (GitHub)

Binaries (NuGet)

Windows Azure Storage Release - Introducing CORS, JSON, Minute Metrics, and More

Windows Azure Tables: Introducing JSON

Comments

  • Anonymous
    February 20, 2014
    Does the Windows Azure Emulator work for the Windows Azure Storage Client Library for C++ Preview?

  • Anonymous
    February 20, 2014
    "Please note, that the 2013-08-15 REST version, which this library utilizes, is currently unsupported by the storage emulator. An updated Windows Azure Storage Emulator is expected to ship with full support of these new features in the next month." Missed this! Feel free to delete these comments.

  • Anonymous
    February 21, 2014
    @Sean, please see our Preview release of Windows Azure Storage Emulator with 2013-08-15 REST version support at blogs.msdn.com/.../windows-azure-storage-emulator-2-2-1-preview-release-with-support-for-2013-08-15-version.aspx This version should work with the C++ library. Please let us know if you hit any issues. You can also use github.com/.../issues to file any feedback you might have.

  • Anonymous
    May 01, 2014
    I notice that the nuget dependency is set to CPPREST SDK >= 1.3.1 on the redist package, however it needs to be an == since the DLL versions are named by version.

  • Anonymous
    October 15, 2014
    Any chance of getting this library to compile on Linux? I am using casablanca on Ubuntu successfully and would like to make it easier to communicate with storage using this library

  • Anonymous
    October 15, 2014
    @James, we are actively looking at adding Linux support for our client library, though no timelines to share yet. You can monitor this open issue on github (github.com/.../12) which we will update as we have more details around this work.

  • Anonymous
    November 10, 2014
    Hei, I get a error while  create a cloud_storage_account objec: Error C2653:'wa' is not a class or namespace name! Do you know why?

  • Anonymous
    November 12, 2014
    I just wonder if it's possible to replace "mycontainer" with a string like: CString myString = "Some text"; wa::storage::cloud_blob_container container = blob_client.get_container_reference(U(myString ));  ??

  • Anonymous
    November 13, 2014
    @Tuan Le - For your first comment, I assume you're using version 0.3.0?  The namespace was changed from wa::storage::... to azure::storage::... since this blog post was written.  For a full list of breaking changes, see github.com/.../BreakingChanges.txt. For your second comment, do you mean "is it possible to use CStrings to call into the storage client?"  A utility::string_t type is used throughout, which is a typedef of std::wstring.  If you have a CString, you'll need to convert it.

  • Anonymous
    November 17, 2014
    Thank you for your feedback. Everything seems to be fine. But when I check with the Denpendency Walker I found some dll missing: API-MS-WIN.... Do you know what could be wrong? Thank you very much.

  • Anonymous
    February 16, 2015
    I have a memory leak problem using Azure Storage Client Library for C++ 0.5.0 Preview. Running a very small code below, watching task manager to check the memory usage, then found memory consumption growing rapidly. Any help?

int _tmain(int argc, _TCHAR* argv[]) { azure::storage::cloud_storage_account storage_account = azure::storage::cloud_storage_account::parse(CONNECTION_STRING); azure::storage::cloud_blob_client blob_client = storage_account.create_cloud_blob_client(); azure::storage::cloud_blob_container container = blob_client.get_container_reference(U("test1")); azure::storage::cloud_block_blob blob = container.get_blob_reference(U("test.txt")); std::wstring text = U("text"); while (1){ blob.upload_text(text); } }

  • Anonymous
    March 19, 2015
    @Tomoyuki Ohya- We are investigating. By trying out the test code you provide, we do observe a very slow trend of memory usage increase after a long running (with 1+M blobs uploaded), but we do not see a rapid memory growth .  Could you provide more information about how you reproduce the issue, for example,
  • how long does it take before you see memory leak happening?
  • how fast does the memory consumption grow?
  • do you ever try to run your program with any memory leak detecting tools, like Visual Leak Detector? How is it going?