Implementing Rolling Updates in Azure Container Service (ACS) with Docker Swarm using VSTS
Docker 1.13 was released recently. 1 of the important feature that came out of this release was simplified deployment using docker-compose file. Also debuted is new compose file v3. As of now(Feb-17), it is most current and recommended compose file format. To understand what new features this enables, refer to this link. A new configuration option, deploy , brings a lot of goodness to overall deployment(including rolling updates) process in swarm mode. In this post, I'll discuss this new configuration option and use it to implement rolling updates.
Deploy configuration option is available ONLY in docker compose v3 file format. It has following sub-configuration options -
mode: This option is used to determine how containers gets deployed on swarm cluster. There are 2 possible values: global and replicated. global value makes a instance of container to be deployed on every swarm node. replicated value makes swarm to ensure at least X instances (where X is value provided by replicas value) of containers are running at all times.
replicas: It determines the number of containers running at any point in time. This setting works together with mode value of replicated.
placement: It defines the placement constraints. They are useful in specifying conditions for containers such as OS requirements or node type.
update_config: This setting determines how upgrade to a running service are applied. This is the core setting that drives rolling updates in Swarm mode. This option has following sub-options -
- parallelism: This value determines how many containers are updated during upgrade.
- delay: Amount of time to wait between last and next container upgrade.
- failure_action: Action to take in case container upgrade fails. It can take any one of the 2 values (continue or pause). Default is pause.
- monitor: Amount of time for which a container is monitored for failure.
- max_failure_ratio: Percentage value between o.o and 1.0 to determine tolerance of upgrade faulure before entire deploy process is declated a failure.
resources: This value is used to set the constraints on resources a container is allowed to use. It is combination of reservation as well as limits.
restart_policy: This value determines a service's restart behavior.
labels: They are used to label the service.
Now that we have looked at deploy configuration option, let's switch to Azure Container Service (ACS) and Docker Swarm.
Up until now, docker-compose file is used with docker-compose command (docker-compose -up) as below.
New guideline however is to use it with deploy command. This command is available as part of docker daemon (server) running in experimental mode. ACS, by default, doesn't deploy docker daemon in experimental mode. So first step is to spin up ACS with server in experimental mode. With Azure Container Service now open sourced, it is very easy to take a look at code, clone/fork it and use it to run the customized ACS engine in Azure.
Instructions to run a customized ACS engine locally are specified here. After the source code is cloned locally, simplest option is to use "Docker for Windows" for ACS customization. There are few pre-requisites that needs to be in place first. These pre-requisites are discussed below. It is required to have local drive "Shared". This setting needs to be set as shown below.
After clicking "Apply" button, copy the powershell command listed in the prompt.
Once the drive is shared, execute the devenv.ps1 script. This script is available inside /scripts folder that comes along with ACS source code.
Once the script execution is finished, a container is spun up and a console logged into it is available to execute further commands. Any IDE/Editor can now be used to change code as required.
To enable ACS engine run with docker server in experimental mode, change \acs-engine\parts\configure-swarmmode-cluster.sh file as shown below.
After code change, build the codebase again by executing following command.
When build finishes, a customized ACS engine is locally available.
Run acs-engine command on the prompt available to understand usage of the command.
To get this local ACS engine deployed to Azure, use instructions available here. The process in nutshell has following key steps.
- Install acs-engine locally. This step is already discussed above.
- Generate SSH key
- Edit the Swarm Mode example and fill in the blank strings
- Generate the template
- Deploy the output azuredeploy.json and azuredeploy.parameters.json
A swarm mode file (referred in step 2 above) is available at /acs-engine/examples folder in the source code. It typically looks as below and can be changed to suit requirements. Ensure that dnsPrefix value is different for maser and agent. It is also possible to change other values such as agent and master counts, VM sizes for both, SSH public key data, etc.
This file is to be used to generate azure deploy template. Use following command to generate the template.
Note that acs-engine simply takes the path of swarmmode.json file. After command successfully executes, it generates a folder called as _output. One more sub-folder (with name as hash of the SSH key is also created). This new sub-folder has following 3 files.
azuredeploy.parameters file is to be modified to include an additional nameSuffix node as shown below.
"nameSuffix": {
"value": "<sub-folder name inside _output folder>"
},
After changes, file looks something like below.
To deploy to Azure, use 1 of the options as described here. Note that a resource group should be created before executing the command.
Once the deployment to Azure finishes, verify that there are no failures in deploying any component inside resource group.
Now connect to ACS cluster as described here and verify that server has experimental flag set as True.
Now it is possible to use docker deploy command to deploy multi-container application. There is a very good guidance available on setting up CI/CD pipeline in VSTS (Visual Studio Team Services) for ACS with Docker Swarm. I followed instructions as given in that guidance except only 1 change. "Run shell commands on remote machine" task in the release tasks needs to be adjusted as below.
Once the shell script in the release task is successfully executed, it is reflected in the release log as shown below.
An important point to note here is that unlike docker-compose up command execution, which needs a docker-compose down command execution, which necessitates downtime, docker deploy command updates services (group of containers) without needing any downtime. Deploy configuration policy specified in the docker-compose file gives developers complete control over the update process.
It is now possible to implement rolling updates using docker deploy and docker-compose file on Azure Container Service (ACS) with Docker Swarm. Use this github repository for accessing code/docker files used for implementing rolling updates.