Supporting Billions of entities/rows for Mobile – Android Series - Part 3–Architecture and Data Options

Previous Posts to this post

Previous Blog Posts Link Supporting Billions of entities/rows for Mobile – Android Series - Part 1 - Why Scale Matters https://blogs.msdn.com/b/brunoterkaly/archive/2011/09/27/supporting-billions-of-entities-rows-for-mobile-android-series-part-1-why-scale-matters.aspx Supporting Billions of entities/rows for Mobile – Android Series - Part 2 - What are some high level cloud offerings? https://blogs.msdn.com/b/brunoterkaly/archive/2011/09/27/supporting-billions-of-entities-rows-for-mobile-android-series-part-2-what-are-some-high-level-cloud-offerings.aspx Free download link for Azure

Let’s talk architecture

At this point we are ready to write some code. Let’s just show a couple of diagrams to kind of give you the picture about what we are trying to do. What we are talking about is the “Service Provider” section. We are about to create a web service in the “Windows Azure” section of the cloud diagram below. This diagram depicts an easy way to think about the style of architecture for today’s applications. You have “consumers” as well as “providers” of data. The “middleware” glues these two worlds together and makes it possible to add basic services as the data spans both worlds.

Accessing Table data from the cloud

This diagram depicts how we we implement our system. Notice the “Solution Explorer” for Visual Studio. It represents our project and code that is running the Azure Data Center (The Microsoft Cloud)

Storage options

Here is a breakdown of the options: Blobs File system in the cloud Tables Massively scalable structured storage (key / value pairs) Queues Reliable storage and delivery of messages Drives Durable NTFS volumes for Windows Azure applications SQL Azure A relational database hosted in the cloud that can hold up to 50GB (9/27/2011)

Which data store is appropriate for you?

Here is a summary of their uses: Binary Large Object (BLOB) Service The simplest way to store text or binary data with Windows Azure. Think about large videos, images, pdfs, zip files. Table Service Is better for large amounts of data that need additional structure, which works exceptionally well with applications that need to work with data in a very detailed manner via queries. This is the data store that I will demonstrate in this post. The Table service offers structured storage in the form of tables. The Table service API is a REST API for working with tables and the data that they contain. The API is a Windows-based framework that you can access from most languages that run under Windows. Queue Service Is for reliable, persistent messaging between Web and Worker role instances. Windows Azure Drive Allows Windows Azure applications to mount a Page Blob, which is a single volume NTFS VHD. This allows applications to upload/download VHDs via blob

Performance Tips for Azure tables

Click here to get Azure performance tips

Additional resources for storage

Description Link Summary of Table Service Functionality https://msdn.microsoft.com/en-us/library/dd135720.aspx Overview of Windows Azure Storage https://www.microsoft.com/windowsazure/features/storage/ Table Service REST API https://msdn.microsoft.com/en-us/library/dd179423.aspx Security Resources for Windows Azure https://msdn.microsoft.com/en-us/library/ff934690.aspx

Your account drives the URL of the various storage abstractions

You can see how the RESTful interfaces map to the specific storage types.

Our RESTful service will leverage Tables

Scalable Structured Storage Store Tables with billions of entities and TBs of data. Provides flexible schema (NoSQL). Data Model A table is a set of entities (rows). An entity is a set of properties (columns) Familiar and Easy to use API OData Protocol. WCF Data Services - .NET classes and LINQ

RESTful data

Description Link Windows Azure Storage Services REST API Reference https://msdn.microsoft.com/en-us/library/windowsazure/dd179355.aspx# Table Service REST API https://msdn.microsoft.com/en-us/library/windowsazure/dd179423.aspx# Blob Service REST API https://msdn.microsoft.com/en-us/library/windowsazure/dd135733.aspx# Queue Service REST API https://msdn.microsoft.com/en-us/library/windowsazure/dd179363.aspx#

Key Whitepapers

Here is an excellent link to read more about Windows Azure. Important Link Address Windows Azure Platform Whitepaper Downloads https://www.microsoft.com/windowsazure/whitepaperdownload/

Next steps

Step Description Part 4 Building a cloud-based, RESTful service Part 5 Building an Android client that consumes the RESTful data