Introducing Mastermind: Powerful and Simple Deployment, Scaling, Load Balancing and Testing of Applications in the Cloud

Guest post by University College London IXN Project Group COMP214P Team 22 on Project Mastermind.

Mastermind is an extensible CLI application that allows easy deployment, updating, scaling and testing of applications using only a Git repository as a parameter. All application-specific configuration and deployment options are specified in a Kubernetes-compatible “mastermind.yaml” file within the Git repository itself that specifies how the application should be deployed, scaled, exposed to the internet and provisioned with database backends and testing suites.

DevOps Challenges

Mastermind was designed to solve one of the fundamental problems that DevOps Engineers and System Administrators face on a daily basis, which is the complexity of publishing and updating applications to staging, testing and production environments in a standards-compliant way that causes minimal disruption and adheres to modern best practices, such as the following:

· Speed of delivery: it shouldn’t take long to publish or update an application. This should take seconds, not hours.

· No downtime: we need to be able to update live applications without causing user disruption. We shouldn’t have to bring down our entire application in order to perform an upgrade.

· Fault tolerance: we want our applications to remain available in the case of failures in other systems. We don’t want a single failing server to bring down our entire application.

· Horizontal scalability: we want to be able to enable our application to handle more load by spinning up multiple copies of it, and scaling it in this way automatically, instead of needing to purchase increasingly expensive servers. Even with the best single server, a high-traffic application would crash.

· Load balancing: Once we have multiple copies of our application, we need a way to spread traffic over all of them, instead of just one. For this, we need a gateway to effectively proxy traffic to different containers of the application based on their current load and other parameters.

· Principle of least privilege: We want our applications to only have the minimum access necessary to do their jobs. We don’t want a single compromised application to be able to take over our network of servers.

· Secret sharing: We don’t want to have to hard-code secrets, such as database passwords, directly in the source code of our applications. We need a way to store them in a central, safe place and make them available to the applications as they need them.

Previously, the luxuries required tooling available only to biggest and/or most technically adept companies in the world or required hundreds of man-hours of work to build the necessary infrastructure. Prior to Mastermind, no software package or toolkit, to my knowledge, enables more than half of the goals above to be realized in an out-of-the-box fashion. Even when some of the ideas above were possible, they required extensive configuration or took a long time to deploy, or took a long time to deploy because of the extensive configuration required on the part of the developers.

Introducing Project Mastermind

Mastermind solves all of these issues, and democratizes access to high-availability, fault-tolerant application distribution in public Clouds and, through extension, private and on premise Clouds. This will allow lone developers, freelances and small companies to enjoy the same luxuries as Google in making their applications available to the world.

The key feature of Mastermind is its ease of use, it’s made available as a POSIX-compliant system-wide command-line utility, think “ls” or “cat”: it takes a mere 3 commands in a terminal to install it on any operating system, 2 commands to prepare to use a cloud provider like Azure, 1 command to deploy an entire Mastermind cluster, 1 command to deploy an application to said cluster, and 1 command to revert any of the aforementioned. This will all become apparent because we will proceed to explain how to use it below. For complete documentation, you can visit the public GitHub page for Mastermind: https://github.com/bsc-comp214p-team22/bsc-comp214p-mastermind.

Configuring a Provider

Once Mastermind is installed, we have to instruct it which service it should use as underlying infrastructure, we can run “mastermind config set provider GCLOUD” for Google Cloud or “mastermind config set provider AZURE” for Microsoft Azure. Mastermind will intelligently inherit settings and credentials from the “gcloud” and “az” utilities on the system if they are installed, so there is no further configuration needed to connect to each service. It also includes a pluggable provider interface, meaning that other public Cloud, private Cloud or on premise services can be added to it easily.

Creating a Cluster

Once we have configured a provider, we need to initialize a Mastermind cluster on the service provider we chose. To do that, we need only run “mastermind cluster create”. This is one command, but the difficulty it abstracts away is enormous. Mastermind deploys a fully functional Kubernetes cluster, meaning that in the background it will provision compute resources such as VMs, generate CA and TLS certificates, generate configuration files, generate encryption keys, deploy an etcd cluster to store configuration data, bootstrap a control node and worker nodes, configure Kubernetes for remote access, provision pod network routes, among other things. To grasp the difficulty of setting up a full Kubernetes cluster you can read “Kubernetes the Hard Way” at https://github.com/kelseyhightower/kubernetes-the-hard-way. But for users of Mastermind, this all becomes one command.

Deploying an App

Now that we have our cluster up and running, deploying an app on Mastermind is equally simple. We need a Git repository with a valid Dockerfile and a “mastermind.yaml” file. The former is a file used to build a Docker image from the code in the Git repository, and the latter is a Kubernetes deployment file with the addition that a user can substitute the Docker image name in the deployment file with the keyword “THIS”. Mastermind will proceed to build a Docker image from the current repository, upload it to the provider’s container image repository, and substitute it in the deployment file when it’s time to push the application to the Cloud. Effectively this means that a developer using Mastermind can use the full power of the world-class Kubernetes framework, without needing to perform the usual configuration steps such as building a Docker image, publishing it to a container repository, etc. All of this is done for them in the command “mastermind app create <git-url-here>”. In addition, the developer can have multiple entries in their deployment file, enabling the ability to deploy databases alongside the application, and sharing configuration options, hostnames and other information.

Updating an App

When the developer makes a change to their application, all they have to do to update the live version of the application is: “mastermind app update <git-url-here>”. Mastermind will intelligently find the previous deployment and update it in a way that leads to no downtime: by bringing down one container of the live application, bringing up a single container of the updated application and repeating this process until only new versions of the applications are live. This leads to no downtime because there are multiple containers running per application, and the deletion of one simply load-balances traffic among the remaining.

Updating with Visual Studio Code

The process of updating an app can be simplified further in Visual Studio Code, where a user can install the Mastermind extension (available at https://marketplace.visualstudio.com/items?itemName=bsc-comp214p-team22.mastermind-update) and run the “Mastermind Update” internal command through the editor, thereby making Mastermind application updates a shortcut away.

Testing

Finally, Mastermind can deploy a testing suite alongside any deployed application to perform Performance, Load and Functionality testing in a real Cloud environment, ending the guessing game of whether an application will hold up under traffic. This testing suite is a web application that can simulate any number of users performing any number of tests on an application remotely, without direct intervention in the application’s code. It plots graphs, shows tables of statistics and allows export via CSV and other formats.

Conclusion

No longer do developers have to jump through hoops in order to deploy, test, load balance and scale their applications. Most of the work is done for them. All they have to do is write the code, and run one command, quite literally.