Supporting Billions of entities/rows for Mobile – Android Series - Part 6–Reading and Writing to Windows Azure (Cloud-based) Tables using standard HTTP and Fiddler

Past Posts in this series

Post Topic Link Part 1 - Why Scale Matters https://blogs.msdn.com/b/brunoterkaly/archive/2011/09/27/supporting-billions-of-entities-rows-for-mobile-android-series-part-1-why-scale-matters.aspx Part 2 - What are some high level cloud offerings? https://blogs.msdn.com/b/brunoterkaly/archive/2011/09/27/supporting-billions-of-entities-rows-for-mobile-android-series-part-2-what-are-some-high-level-cloud-offerings.aspx Part 3–Architecture and Data Options https://blogs.msdn.com/b/brunoterkaly/archive/2011/09/28/supporting-billions-of-entities-rows-for-mobile-android-series-part-3-architecture-and-data-options.aspx Part 4–Building a Cloud-based RESTful service for our Android, iOS, and Windows Phone 7 Clients https://blogs.msdn.com/b/brunoterkaly/archive/2011/09/28/supporting-billions-of-entities-rows-for-mobile-android-series-part-4-building-a-cloud-based-restful-service-for-our-android-ios-and-windows-phone-7-clients.aspx Part 5–Using the Portal and Setting up your Azure Account (Microsoft Cloud) https://blogs.msdn.com/b/brunoterkaly/archive/2011/10/05/supporting-billions-of-entities-rows-for-mobile-android-series-part-5-using-the-portal-and-setting-up-your-azure-account-microsoft-cloud.aspx You will need to download the free Azure trial.

The last post was about …

In the last post I illustrated how to create a RESTful service using .NET technologies. Here is the diagram from the last post:

This post is to illustrate that you can go directly to the Storage Service with HTTP bypassing .NET

Although it is not recommended for this particular application, it is possible communicate directly with the Storage Service. Direct communication with Azure tables requires some advanced techniques, working with request headers and request bodies. Later in this post you will learn what makes this technique somewhat challenging. We need to able to: 1. Create a table 2. Insert into a table 3. Query a table We need do this using all standard http without any .NET whatsoever. We will use Fiddler to get this done. We can also use the Azure Storage Explorer.

Azure Storage Explorer

Azure Storage Explorer is a useful GUI tool for inspecting and altering the data in your Windows Azure Storage storage projects including the logs of your cloud-hosted applications. All 3 types of cloud storage can be viewed and edited: blobs, queues, and tables. Description Link Home page on CodePlex for Azure Storage Explorer https://azurestorageexplorer.codeplex.com/     Login to https://windows.azure.portal and choose Hosted Services, Storage Accounts & CDN. Now you need your storage access keys. Copy the storage access keys to the clipboard. Now are are ready to configure the Azure Storage Explorer. As expected, there are no tables and no data. That is the next step.

Adding a Table Object with Storage Explorer

Follow these steps to add a table object: 1. Click Tables 2. Click New 3. Enter a table name

Adding Entities

Each table can have many Entities. You can think of an entity as a row in a table. But in reality an entity is a collection of name- value pairs.  Notice under the Storage Type panel you can see the Tables button. Click tables before following the rest of the steps. Notice that a PartitionKey and RowKey are required columns.

Querying the table

You can also hit the query button to see the entities in the table. Later, when we create the Android client, you will be able to use LINQ to query the entities.

Using http to interact with Azure Tables

Http is the most generic way to interact with Azure tables. I will be using Fiddler add new records and to query the data in Azure tables. You can download Fiddler here: Description Link Download Fiddler https://www.fiddler2.com/fiddler2/     Authentication The Table service requires that each request be authenticated. Both Shared Key and Shared Key Lite authentication are supported. More information can be found here Description Link Authentication Information at Microsoft https://msdn.microsoft.com/en-us/library/windowsazure/dd179428.aspx       Entering an HTTP Request in Fiddler   Step 1 -  Click on Request Builder Step 2 -  Enter the URL and select GET Step 3 -  Enter the Request Header (you cannot do this without my tool explained below. See “Generating the Request Header” section below). Step 4 -  Hit Execute   The Request Header is going to require you to generate a Shared Key Lite.

Viewing the Result in Fiddler

After hitting Execute in Step 4 above, results will appear.   Viewing the results (see above)   Step 1 – Double click the result item in the upper left window. This brings up results window in the lower right pane of Fiddler Step 2 – Click on TextView to see the results in AtomPub format. Step 3 – View the data. Notice you can see Bruno and Haybusa that we added previously with the Azure Storage Explorer.

Generating the Request Header

If you look carefully, you will notice the SharedKeyLite authorization signature. This isn’t trivial to generate, since you need to compute a Hash-based Message Authentication Code (HMAC) using the SHA256 hash function. Luckily I’ve done that for you.

A Simple Windows Forms Application

You are looking at a custom application that I wrote. I am providing the code below in case you want to do this yourself. You typically would use the ADO.NET Data Services library in C# or VB. But if you want to interact with Azure Tables from non-.NET clients, you will need to master the techniques I am illustrating here. You can learn more at the link below. Description Link Azure REST-API https://msdn.microsoft.com/en-us/library/windowsazure/dd179423.aspx     The trick is understanding the code behind the Make Request button, which fills in all the boxes you see above. The ones to paste into fiddler are: 1. URI 2. HttpHeader

public partial class Form1 : Form{ const string uriTemplate = "https://{0}.table.core.windows.net/{1}"; const string uriTemplate2 = "https://{0}.table.core.windows.net/{1}/{2}"; string _sharedKey = "== your shared key from the portal goes here=="; string _accountName = "fastmotorcycle"; string _tableName = "fastbikes"; public Form1() { InitializeComponent(); } private void cmdMakeHeader_Click(object sender, EventArgs e) { var account = _accountName; var sharedKey = Convert.FromBase64String(_sharedKey); this.txtAccountName.Text = account; this.txtSharedKey.Text = _sharedKey; this.txtTableName.Text = _tableName; // Fill datetime field string datetime = DateTime.UtcNow.ToString("R", CultureInfo.InvariantCulture); this.txtDate.Text = datetime; // Fill uri field string uri = string.Format(uriTemplate, _accountName, _tableName); this.txtURI.Text = uri; // Create request object WebRequest request = WebRequest.Create(uri); // Add date header request.ContentLength = 0; request.Headers.Add("x-ms-date", datetime); // Get resource var resource = request.RequestUri.PathAndQuery; // Sign date, account and uri string stringToSign = string.Format("{0}\n/{1}{2}", request.Headers["x-ms-date"], account, resource ); this.txtMessageToSign.Text = stringToSign; // Sign the string using shared key var hasher = new HMACSHA256(sharedKey); string signedSignature = Convert.ToBase64String(hasher.ComputeHash(Encoding.UTF8.GetBytes(stringToSign))); this.txtSignedMessage.Text = signedSignature; // Add authorization header string authorizationHeader = string.Format("{0} {1}:{2}", "SharedKeyLite", account, signedSignature); request.Headers.Add("Authorization", authorizationHeader); // Show the signed message this.txtSignedMessage.Text = request.Headers.Get(0).ToString() + "\r\n"; this.txtSignedMessage.Text += request.Headers.Get(1).ToString() + "\r\n"; // Show the entire header this.txtHttpHeader.Text = "x-ms-date:" + request.Headers.Get(0).ToString() + "\r\n"; this.txtHttpHeader.Text += "Authorization:" + request.Headers.Get(1).ToString() + "\r\n"; this.txtHttpHeader.Text += "Content-Type:" + "application/atom+xml" + "\r\n"; if (this.txtPartitionKey.Text == String.Empty || this.txtRowKey.Text == String.Empty) return; this.txtResponseBody.Text = string.Format(Strings.response_header, this.txtPartitionKey.Text, this.txtRowKey.Text); } private void cmdClip2_Click(object sender, EventArgs e) { Clipboard.SetData(DataFormats.Text, (Object)this.txtResponseBody.Text); } private void cmdClip1_Click(object sender, EventArgs e) { Clipboard.SetData(DataFormats.Text, (Object)this.txtHttpHeader.Text); }}

You can even add data using this technique

Notice that this time I added a PartitionKey and RowKey in Steps 1 and 2 below. Now we need to copy the Response Body in addition to the previous fields. The XML below was generated by my tool. Notice the partitionkey and rowkey from the form above.   <?xml version="1.0" encoding="utf-8"?> <entry xmlns:d="https://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="https://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns="https://www.w3.org/2005/Atom">   <title />   <author>     <name />   </author>   <updated>2011-10-04T15:28:55.7259928Z</updated>   <id />   <content type="application/xml">     <m:properties>       <d:PartitionKey>Bruno</d:PartitionKey>       <d:RowKey>GSXR1000</d:RowKey>       <d:Timestamp m:type="Edm.DateTime">0001-01-01T00:00:00</d:Timestamp>     </m:properties>   </content> </entry> Notice the Make Header tool creates the 3 core pieces of information you will need to paste into Fiddler’s Request Builder: 1. URI 2. Request Header 3. Request Body When adding date, you will need to do a POST. Here is a table that explains the HTTP verbs and how they relate to operations: HTTP Verb Operation GET Query data POST Insert data DELETE Delete data PUT Update data MERGE Merge data   In Fiddler do the following:   Step 1 – Change from GET to POST Step 2 – Paste in the new Request headers from the tool above Step 3 – Paste in the Request Body from the tool above Step 4 – Hit execute Step 5 – Realize that you just added data to your table using a PURE http protocol Notice that we have added the data. You can see the GSXR1000 data in Azure Storage Explorer.

Conclusions

This post was important for several reasons. First, it introduced the Windows Azure Portal and addressed the Hosted Service and Storage Account and how they related to your RESTful service and to your infinitely scaled Azure Table Service Storage. Windows Azure Portal Section Description Hosted Service Where your RESTful service will reside and be available to mobile clients. This is the next blog post. We will migrate our RESTful project in Visual Studio to the Microsoft Cloud. Remember, not only is our data scalable, but so is our RESTful service. Building the RESTful Service The blog link Part 4–Building a Cloud-based RESTful service for our Android, iOS, and Windows Phone 7 Clients https://blogs.msdn.com/b/brunoterkaly/archive/2011/09/28/supporting-billions-of-entities-rows-for-mobile-android-series-part-4-building-a-cloud-based-restful-service-for-our-android-ios-and-windows-phone-7-clients.aspx     Storage Account Where our data is stored. Explored deeply in this post, using the Azure Storage Explorer and Fiddler with standard HTTP requests. In the next post we will show how this data is consumed by the RESTful service. Mobile clients will not access Azure tables directly. They will go through the RESTful Hosted Service.     Typically, you will access Azure Table Storage through the ADO.NET Data Services API, found inside the Azure SDK Download. But there may be situations where you want to access Azure table storage outside of a Windows environment, such as from Java or PHP or any other non-MS platform. Remember, Azure tables are incredibly scalable, cost-effective, and performant. And Azure tables are also available from a purely http world. What Where Free download link for the 90-day Azure trial. Click the photo to the right.

Comments

• Anonymous
  January 08, 2012
  Thanks for all the articles. I am new to Win Azure and Mobile domains. This was very helpful. Is there a link to download the Make Header win form app? Please share. I can be reached at prashant_ nerkar@hotmail.com