Supporting Billions of entities/rows for Mobile – Android Series - Part 1 - Why Scale Matters
| It is the trend that matters |
| As you read this post, keep in mind that I’d like you to focus the trend. Some of the numbers presented my be dated or slightly off, but keep one thing clear – mobile is a huge game changer and the trend of growth is staggering. |
| Why you should care about scale? |
| One thing is certain – the world of mobile is putting major pressure on the world of cloud. When you think of the 4.5+ billion mobile users out there, it starts to be obvious. The amount of data consumed by mobile devices is about to catch up to fixed traffic – terabytes of data every day. Remember one thing when you read this post. It will get a lot more technical. There will be lots of code presented very soon. From my experience being in the field very few companies are ready to meet scale, especially when you consider all the potential mobile apps hitting these 4.5 billion devices. I’ve been a field engineer. You’d be amazed at how many companies don’t have a plan in place to meet potentially explosive demand and use of their mobile applications. It should come as no surprise that a $500 million dollar data center **is not** within reach of most American businesses. Fortunately, with some careful planning, a business today can start modestly and keep expenses low, while maintaining the potential to dial up any number of servers to meet customer demand. It simply doesn’t make sense to always build a data center. There are many scenarios where the customer need for scale is temporary, so why should a business invest in dozens of servers “just in case you need them?” Well, that’s the way it used to be. You necessarily needed overcapacity for most of the time, because demand could spike and your lack of scale would give them a terrible user experience. When I did a startup with a bunch partners back in 1999/2000, we all got stuck with a bunch of worthless hardware. The economic meltdown drove us to shutdown and give away our rack of servers co-located in some obscure data center in San Francisco. |
| Today that would never happen. |
| You would never get funding today for your startup if you requested budget for a scalable web farm. You’d be laughed out of the office of the venture firm. It is assumed that you have a “cloud component” to your pitch should your idea require scalable resources. |
| One reason for the mobile explosion |
Simply put, today’s mobile devices are fantastically powerful.  Think about the Apollo spacecraft having 4,000 bytes of RAM. Think about today’s phone device having 256,000,000 bytes of RAM. That is 64,000 times more RAM. It is the stuff astronauts would envy. |
| Is mobile done growing? |
| Far from it. There are 4.5 billion devices in the wild. The average human uses an average of 4.3 internet connected devices. The projected smartphone growth is expected to increase over 80% over the next 3 years. There could be over a trillion internet aware devices by 2015. A typical smart phone consumes 100's of megabytes of data per month. Mobile devices are expected to overtake fixed devices in terms of consuming internet traffic. |
| More shipments than PCs |
Some amazing statistics. Vendors shipped 100.9 million smartphones during the fourth quarter, according to Monday data, while IDC logged 92.1 million PC shipments during the same time period, according to IDC numbers from January.
|
| Browser based queries are less important |
The way every interacts with the internet is changing rapidly. It used to be if you wanted to catch a taxi you'd do a search on Bing or Google. Today you are downloading a specialized application, specializing in taxi services, providing you the ability to leverage GPS, text messaging, taxi routing. The purchase of specialized applications from internet-based marketplaces is expected to increase dramatically.
|
| It is about mobile applications in the US - study from the Pew Internet Project |
| 82% of U.S. adults who are now active cellphone users, 43% now have apps on their phones, and more than two-thirds of them use those apps regularly. In other words, 24% of the U.S. adult population actively uses apps. |
| Why the cloud is needed |
If you look at the way phone applications are being used, it is clear they need a back end data store that is scalable.
|
| How often are apps used? |
| Nearly 2 in 3 said they use their apps daily, and 1 in 4 use their apps for more than 30 minutes per day. Most (71%) use their apps alone, while roughly half use them while waiting for someone or something, or while at work. Another 36% use their apps while commuting (which happens to be when I use apps most heavily). |
| Next blog post |
| I’m going to start to talk about the different cloud offerings. There are differing levels of sophistication and customizability among the different vendors. Some cloud offerings do not require any programming, while the others allow you to do highly sophisticated server side applications, with custom languages, frameworks, and tools. I might be talking about everyone from Simple Geo to Urban Airship up to Open Saint. I will also talk about Amazon and Microsoft Windows Azure. |
 |
 |