Share via

Windows Azure’s Flat Network Storage to Enable Higher Scalability Targets

  • Article

Brad Calder, a Distinguished Engineer at Microsoft,  recently blogged about higher scalability targets for Azure Storage enabled by Flat Network architecture: Windows Azure’s Flat Network Storage and 2012 Scalability Targets. Windows Azure’s implementation of Flat Network toplogy, referred to as “Quantum 10” (Q10) network architecture,  is influenced by the Microsoft Research paper: VL2: A Scalable and Flexible Data Center Network presented at SIGCOMM’09.

Once software improvements are fully rolled out for flat network implementation, which will happen per Brad by the end of 2012, will provide the following scalability targets for each Azure Storage account created after June 7th, 2012:

Storage Account Scalability Targets

  • Capacity – Up to 200 TBs
  • Transactions – Up to 20,000 entities/messages/blobs per second
  • Bandwidth for a Geo Redundant storage account
    • Ingress - up to 5 gigabits per second
    • Egress - up to 10 gigabits per second
  • Bandwidth for a Locally Redundant storage account
    • Ingress - up to 10 gigabits per second
    • Egress - up to 15 gigabits per second

Please keep in mind that any storage account created before June 7th, 2012 need to be migrated to newly created storage account to take advantage of the above higher scalability targets. 

Since each storage account is composed of many partitions, it is important to understand the scalability targets (achieved with 1kb object size) for a single partition which are given below:

Partition Scalability Targets

  • Single Queue (all the messages in a queue are accessed through a single partition)
    • Up to 2,000 messages/second
  • Single Table Partition (all the entities in a table with the same partition key)
    • Up to 2,000 entities/second (with efficient partitioning scheme one should be able to achieve 20,000 entities/second)
  • Single Blob (each blob will be in its own partition as the partition key for blog is “container name + blob name”)
    • Up to 60 MBytes/sec

The above are the high end numbers and as Brad suggests, your mileage will vary depending on the object sizes and access patterns. So, establishing application specific scalability numbers before production deployment helps with the scale out planning of your Azure infrastructure as application load ramps up.

Read Brad’s excellent paper titled Windows Azure Storage: A Highly Available Cloud Storage Service with Strong Consistency for the internals of Azure Storage architecture presented at SIGOPS’11.

Experience Microsoft cloud platform with Azure 90-day Free Trial.

-Hanu

Technorati Tags: Technorati Tags: Brad Calder,Azure Storage,Azure Flat Network,Azure Storage Scalability Targets