Polyglot Compute - A Pattern for Running Modern Applications

One thing that has always amazed me is how people tend to latch on to the latest trend, even if that trend offers limited value. Maybe it's the desire to keep current and not look silly or maybe it's our built-in nature to change and adapt and not stay stuck in our old ways and habits. Don't get me wrong I love trying new things and especially when it comes to technology for building applications, we all know there are loads of options out there to help us get the job done. Different programming languages, frameworks, design patterns, database technologies and now with the cloud, different ways of running your application. The key, however, is choosing the right mix of all of this for the application being architected. I have seen this many times over the years when developers (and I include myself in that) start using a new framework or language because it's becoming popular and solves a real problem, only to disregard what they have used in the past. An example of this was when Microsoft released the MVC framework for building web sites. Developers jumped on this and decided that this was the way forward and God forbid you went in to an interview and spoke about your knowledge of ASP.NET forms, this was now just old hat, right? Of course, asp.net forms is not hold hat and still solves a really important problem ( and I would argue is based on the MVC pattern too ), namely how to build event driven form based applications. The MVC framework can do this too, but it takes longer to develop and is more suited to developing websites that can surface well on any device or in any browser. Microsoft have even recognised this lately by allowing developers to choose both within a single Visual Studio project. So a responsive website can now have forms too :-)

Polyglot Programming

Programming languages are no different. Most of us tend to stick to the languages we know best and master them, but are those languages appropriate for everything we do? Why can't we build a solution that uses F# for certain functional/mathematical routines but continue to use c# for backend/frontend capabilities? This concept of Polyglot Programming has been around a long time (thanks to Neal Ford in 2006), yet I am not certain we have really adopted it. It's one of the main reasons I love the .NET architecture. The idea that a developer can choose the language they want, even mix them together, and produce an application appeals to me.

Polyglot Persistance

This concept was then brought forward in 2008 by Scott Leberknight when he coined the term 'Polyglot Persistence' and described how developers should use database technologies appropriate for the data they are storing. This is something I feel quite passionate about. If we think about the data we store, it's not just relational, structured and transactional data. It's unstructured data or structured data that has no fixed schema or relationships. Yet, developers still tend to store their data in a relational database store like SQL Server, Oracle or MySQL. Of course, this works fine (well kind of ugly though) until we start to look at the cost of doing this. When it comes to clustering relational databases don't scale out too well, simply because replicating transactional changes across more than two servers synchronously starts to add too much overhead on the cluster. Of course, nowadays we have NoSQL options like MongoDB and DocumentDB, that allow us to store our data in a more de-normalised way, meaning we are not so much at the mercy of replication issues and we don't care too much about relationships and defined schemas. But of course some developers will just do everything in a NoSQL database and this is not the answer either. Understanding your data and knowing how to spread it across relational systems and non-relational systems is what Polyglot Persistence is all about. When we then think about unstructured data even more technologies come in to play like Hadoop and HDInsight which allow us to gain insights from data we may never have looked at before. We now have so much data to analyse it makes sense for us to consider different technologies for different data types and consistency requirements.

 

Polyglot Compute

Now if we think about how developers run their applications there is more choice than ever. Cloud has revolutionised how developers can deploy their applications, either to dev, test, staging or production environments. Developers are looking to extend their agile capabilities out to their hosting platform and continuously integrate their code with a running application. Yet, IT hasn't caught up. The IT department still, in a lot of cases, just manages infrastructure so will simply provision virtual machines for the developers. As developers, we have even created tools like Chef and Puppet so development teams can orchestrate these virtual environments quickly as part of the development life cycle. This is a move towards developers creating their own compute platform for running their applications. 

But of course, spinning up Virtual Machines is not the only option, albeit the default. We now have tools like Docker so developers can containerise their applications so they can run anywhere, we have platform as a service offerings like Azure Websites and Azure Cloud Services. We also have the choice of running parts of our applications by simply consuming services delivered by others, like Media Services for encoding, indexing and streaming media, service bus for creating queues or push notification hubs or event hubs for IoT devices. We can even now think about how our applications will run from on-premise to the cloud and from one cloud provider to another.

So as we think about how we architect our applications right now to achieve the best user experiences we not only think about the best programming languages to get the job done as well as the most appropriate database technologies to manage the data, we will also consider how to integrate different compute resources to maximize the end user experience and cost effectiveness of the overall solution.

I often find myself talking to customers about the benefits of PaaS over IaaS, or the benefits of moving workloads to the cloud or keeping them on-premise, the fact is applications should be architected as a composition of multiple loosely coupled components and we should consider the best compute mechanism for each of these components, this will give us the best balance between performance, scalability, speed, agility, security and control and allow us to focus on what we do best, build great applications. 

It should be noted that eventually the evolution towards Micro-services, Containerization and Cloud will mean developers will no longer need to concern themselves with an OS, never mind hardware. We are approaching a fundamental shift in how we use technology to deliver applications.

.