Get Started with Kubernetes in Azure (Government)
Azure offers many different options for deploying cloud workloads. You can deploy Virtual Machines or use Azure App Service. You can even use App Service in a virtual network by deploying Azure App Service Environment (ASE), which is also available in the US Government Cloud. In this blog post, I will explore another deployment option - Kubernetes. Specifically, I will discuss how to deploy a Kubernetes cluster in Azure Government and host a web application in it.
There is already some documentation on deploying Kubernetes to Azure Government. You can also watch this video on how to all of this works. The main purpose of this blog post is to provide a bit more detail on the instructions and provide some tools and examples to get you started.
Kubernetes is an open source orchestration engine for automated deployment, scaling, and management of containerized applications. If you are not familiar with containers, I recommend running through an introduction to containers before proceeding. It is easy to host Kubernetes in Azure. There is a tool called ACS Engine (Azure Container Service Engine), which you can use to generate templates for deploying clusters into Azure. ACS Engine actually supports several different orchestrators (Kubernetes, OpenShift, DC/OS, etc.), but we will focus on Kubernetes here.
In Azure Commercial, there is now a fully managed offering for Kubernetes called Azure Kubernetes Service (AKS). If you are using AKS, the platform will handle several operations and management tasks for you, e.g. patching of the OS on the nodes, etc. AKS is a great option for many types of workloads, but it is not yet available in Azure Government and it does not offer the configuration flexibility that ACS Engine does. Consequently, we will be taking a closer look at deploying with ACS Engine. By the end of this blog post, I will have demonstrated:
- How to deploy a Kubernetes cluster to Azure Government.
- How to interact with the cluster (using
kubectl). - How to view the web dashboard for the cluster.
- How to deploy a web application in the cluster.
Let's get started. I have created a GitHub repository with some notes and utilities for ACS Engine. All the code samples, etc. that we will be using can be found there. I am assuming that you have the following tools installed:
- Bash shell (Linux or Linux subsystem in Windows).
- Azure CLI 2.0.
- jq.
- Git.
Install Tools
You need a few additional tools to be able to deploy and manage Kubernetes clusters.
Install ACS Engine:
[shell]
#Download binary, replace with latest version
wget https://github.com/Azure/acs-engine/releases/download/v0.17.0/acs-engine-v0.17.0-linux-amd64.zip
#Unzip and copy to location in path
unzip acs-engine-v0.17.0-linux-amd64.zip
mkdir -p ~/bin
cp acs-engine-v0.17.0-linux-amd64/acs-engine ~/bin/
[/shell]
To be able to interact with your cluster, you will need kubectl:
[shell]
#Download the latest version
curl -LO https://storage.googleapis.com/kubernetes-release/release/$(curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt)/bin/linux/amd64/kubectl
#Make it executable:
chmod +x ./kubectl
#Copy to bin, you could also move it to /usr/bin
mv kubectl ~/bin/
[/shell]
Deploying a Kubernetes Cluster in Azure Government
You are now ready to follow the instructions on the ACS Engine site, and I will give a few specific hints to make it a smooth operation in Azure Government.
Clone the scripts and notes to have a local copy:
[shell]
git clone https://github.com/hansenms/acs-utils.git
[/shell]
Make sure you are logged into Azure Government using the Azure CLI:
[shell]
az cloud set --name AzureUSGovernment
az login
[/shell]
This will take you through the device login procedure. If you have multiple subscriptions, make sure you pick the right one with:
[shell]
az account set -s SUBSCRIPTION-ID-OR-NAME
[/shell]
Then we can create a resource group and a service principal for the Kubernetes cluster. We will grant the service principal Contributor rights on the resource group to allow it to create and manage resources in the cluster:
[shell]
subscription=$(az account show | jq -r .id)
#Name and Location
rgname=mykubernetes
loc=usgovvirginia
#Create Group
rg=$(az group create --name $rgname --location $loc)
rgid=$(echo $rg | jq -r .id)
#Create the Service Principal and assign as contributor on group
sp=$(az ad sp create-for-rbac --role contributor --scopes $rgid)
[/shell]
Next step is to define the cluster. ACS Engine uses an cluster definition, a.k.a. api model, file (json format) to define the cluster options. There are a number of example definitions (api models) available in the ACS Engine examples, and there are a few included in the acs-utils GitHub project. The api model file needs to be populated with the DNS name for the cluster and the details for the service principal created above. It is pretty easy to do that manually with some copy and paste, but I have also created a utility script (convert-api.sh) that will take an api model file and populate it with the details. You can use the script like this:
[shell]
./scripts/convert-api.sh -c $(echo $sp | jq -r .appId) \
-s $(echo $sp | jq -r .password) -f kubernetes.json \
-d myClusterDns | jq -M . > converted.json
[/shell]
After this, the cluster definition is in converted.json and ready to be deployed:
[shell]
acs-engine deploy --api-model converted.json \
--subscription-id $subscription --resource-group $rgname \
--location $loc --azure-env AzureUSGovernmentCloud
[/shell]
You may be prompted to go through the device login procedure. After deployment (which will take 10 minutes or so), you will find a folder called _output/myClusterDns (replace your DNS name). This folder will contain a bunch of files including the actual Resource Manager template, that was used for deployment. For the purposes of this demo, we will need the _output/myClusterDns/kubeconfig/kubeconfig.REGIONNAME.json file, which contains your credentials, etc. for interacting with the cluster.
Your available Kubernetes configurations are usually stored in ~/.kube/config. You can also specify a particular kubeconfig by setting the KUBECONFIG environment variable. If you would like to merge the configuration of your new cluster into your existing ~/.kube/config, you could do something like this:
[shell]
#Merge configuration with your current kube config
KUBECONFIG=~/.kube/config:_output/myClusterDns/kubeconfig/kubeconfig.usgovvirginia.json \
kubectl config view --flatten > new-config
#Always good to back up
cp ~/.kube/config ~/config-backup
#Put the new merged config in place
cp new-config ~/.kube/config
[/shell]
Now you should be able to list your "contexts" and see that your new cluster is in the list:
[shell]
kubectl config get-contexts
[/shell]
You can select the new cluster with:
[shell]
kubectl config use-context CONTEXT-NAME
[/shell]
Finally validate that you can interact with the cluster by sending a command, e.g.:
[shell]
kubectl get nodes
[/shell]
which will give you a list of the current nodes in the cluster (there should be a master and 2 agents if you are following the tutorial here):
[shell]
NAME STATUS ROLES AGE VERSION
k8s-agentpool1-15861492-0 Ready agent 12m v1.10.2
k8s-agentpool1-15861492-1 Ready agent 13m v1.10.2
k8s-master-15861492-0 Ready master 13m v1.10.2
[/shell]
Accessing the Web Dashboard
Kubernetes has a nice web dashboard for monitoring and managing the cluster. You can access it by opening up a proxy:
[shell]
kubectl proxy
[/shell]
Then point your browser to https://localhost:PROXY-PORT/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/. You will need a token to sign in. I have made a convenience script:
[shell]
./scripts/get-token.sh
[/shell]
This will display a token that you can copy and paste to log in. You should see the dashboard:
Deploying a Workload
Workloads are deployed in Kubernetes clusters using manifest files. If you are completely new to Kubernetes, have a look at one of the introductory tutorials. I have included some simple manifest files in the GitHub repository. As an example, you can deploy an ASP.NET Core Web App with the web-service.yaml manifest:
[shell]
kubectl apply -f manifests/web-service.yaml
[/shell]
This will deploy the web application itself in a pod. You can see the list of pods running with:
[shell]
kubectl get pods
[/shell]
You should see something like:
[shell]
NAME READY STATUS RESTARTS AGE
web-service-674f64575b-xzflq 1/1 Running 0 1m
[/shell]
There is also a frontend service (a load balancer) that exposes the web application outside the cluster. You can list the services:
[shell]
kubectl get svc
[/shell]
And you should see the running services:
[shell]
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
frontend-service LoadBalancer 10.0.189.218 13.XX.XX.X 80:30353/TCP 3m
kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 1h
[/shell]
Copy the EXTERNAL-IP of the frontend-service and point your browser to https://EXTERNAL-IP and you should see the web application:
Conclusions and Next Steps
In this blog post, we have looked at how easy it is to set up a Kubernetes cluster in Azure Government and deploy the first workload to it.
There is still work to be done. If you try to access the web application with HTTPS, you will notice that we need a valid certificate. To make that work well, we will need to set up an Ingress Controller and Ingress rules. I cover that in a this blog post on Ingress in Kubernetes. It may also be a requirement to keep Government workloads on internal/private IP space. I will also cover that scenario in a later blog post.
For now, have some fun setting up Kubernetes in Azure Government (or Commercial) and let me know if you have questions/comments/suggestions.