Data Access for Highly-Scalable Solutions: Using SQL, NoSQL, and Polyglot Persistence

September 2013

Summary

The business requirements of many modern business applications often involve processing large amounts of data. Additionally, the performance requirements typically make it imperative that the application uses the most efficient means to store and retrieve this data. In the past, most application designers would have simply chosen to save this data in a SQL database, but the recent advances in NoSQL technology have brought about highly-scalable, non-relational databases that can handle vast amounts of information very efficiently. Each type of NoSQL database is often optimized for a particular pattern of data access, and application designers are now increasingly looking to integrate multiple NoSQL and SQL databases into their solutions.

This guide describes how to design and build applications and services that can take best advantage of SQL and NoSQL databases by combining them into a polyglot solution. It provides an end to end walkthrough of a business application that uses SQL Server in conjunction with a variety of NoSQL databases, showing how the designers selected the databases to closely match the various business requirements.

This guide describes:

• The different types of NoSQL databases currently available, the features that they commonly provide, and how they are more suited to some common data access scenarios that relational databases.
• The principles of designing a relational database to support fast queries and transaction processing.
• How using a key/value store can optimize operations that identify data by using key values only.
• When to use a document database to store and retrieve structured information quickly and efficiently.
• The features of a column-family database that enable applications to store, retrieve, and analyze data held in very large datasets.
• How to use a graph database to store information about the complex relationships between data items, and how to query these items based on these relationships very efficiently.
• How to combine different SQL and NoSQL databases together into a cohesive solution that meets the scalability, availability, and consistency requirements of your business applications.

In addition to describing the sample application, its integration with the web services, and the decisions made during its design and implementation, this book discusses related factors, such as the design patterns used, and good practice for separating the business logic of an application from the data access logic to avoid introducing dependencies between the two.

The result is that, after reading this book, you will be familiar with the basic principles that underpin most SQL and NoSQL databases, and you will be able to design and implement business applications that can take best advantage of the different models of data access that these databases provide.

Download Code	Data Access Guidance – Reference Implementation
Readme and Release Notes	Readme for the Reference Implementation Release Notes for the Examples
Download Book	PDF and epub: Data Access for Highly-Scalable Solutions: Using SQL, NoSQL, and Polyglot Persistence Kindle (Mobi): Data Access for Highly-Scalable Solutions: Using SQL, NoSQL, and Polyglot Persistence
Order Paperback	Available on Amazon.com: Data Access for Highly-Scalable Solutions: Using SQL, NoSQL, and Polyglot Persistence
Chapters	Preface 1- Data Storage for Modern High-Performance Business Applications Issues with Implementing a Relational Database The Advent of NoSQL Common Features of NoSQL Databases Integrating NoSQL Databases into a Polyglot Solution 2- The Adventure Works Scenario How a Customer Uses the Shopping Application How the Shopping Application Stores Information The Architecture of the Shopping Application 3- Implementing a Relational Database Designing a Relational Database to Support Fast Transactions Designing a Relational Database to Optimize Queries Implementing a Relational Database to Maximize Concurrency, Scalability, and Availability Accessing Data in a Relational Database from an Application 4- Implementing a Key/Value Store Designing a Key/Value Store Implementing a Key/Value Store to Maximize Scalability, Availability, and Consistency 5- Implementing a Document Database What is a Document? Designing a Document Database Implementing a Document Database to Maximize Scalability, Availability, and Consistency 6- Implementing a Column-Family Database What is a Column Family? Designing a Column-Family Database Implementing a Column-Family Database to Maximize Scalability, Availability, and Consistency 7- Implementing a Graph Database What is a Graph? Designing a Graph Database Implementing a Graph Database to Maximize Scalability, Availability, and Consistency 8- Building a Polyglot Solution Partitioning Data Across Different Types of Database Managing Cross-Database Consistency Appendix A- How the MvcWebApi Web Service Works Understanding the Flow of Control Through the Web Service Routing Incoming Requests to a Controller Transmitting Data Between a Controller and a Client Using the Repository Pattern to Access Data Decoupling Entities from the Data Access Technology Instantiating Repository Objects Appendix B- How the Developers Selected the NoSQL Databases Bibliography
Community	http://dataguidance.codeplex.com/
License	Microsoft patterns & practices License

Authors and Contributors

This guide was produced by the following individuals:

• Program and Product Management: Andrew Oakley (Microsoft Corporation)
• Development: Hanz Zhang (Microsoft Corporation), Douglas McMurtry (Ciber), Mauro Krikorian (Southworks SRL)
• Test team: Mani Subramanian and Carlos Farre (Microsoft Corporation), Partha Krishnasamy (Infosys Ltd), Nicolas Botto (Digit Factory), Vivek Patel (Infosys Ltd)
• Documentation: John Sharp (Content Master Ltd)
• Editor: RoAnn Corbisier (Microsoft Corporation)
• Illustrations and book layout: Chris Burns (Linda Werner & Associates Inc)
• Release Management: Nelly Delgado (Microsoft Corporation)

Feedback and Support

Questions? Comments? Suggestions? This content is a guidance offering, designed to be reused, customized, and extended. It is not a Microsoft product. Code-based guidance is shipped "as is" and without warranties. Customers can obtain support through Microsoft Support Services for a fee, but the code is considered user-written by Microsoft support staff.