This chapter covers

• Using containers to package a microservice into a deployable artifact
• How to run a microservice on Kubernetes, a container scheduler
• Core Kubernetes concepts, including pods, services, and replica sets
• Performing canary deployments and rollbacks on Kubernetes

Containers are an elegant abstraction for deploying and running microservices, offering consistent cross-language packaging, application-level isolation, and rapid startup time.

In turn, container schedulers provide a higher level deployment platform for containers by orchestrating and managing the execution of different workloads across a pool of underlying infrastructure resources. Schedulers also provide (or tightly integrate with) other tools — such as networking, service discovery, load balancing, and configuration management — to deliver a holistic environment for running service-based applications.

Containers aren’t a requirement for working with microservices. You can deploy services using many methods such as using the single service per VM model we outlined in the previous chapter. But together with a scheduler, containers provide a particularly elegant and flexible approach that meets our two deployment goals: speed and automation.

Docker is the most commonly used container tool, although other container runtimes are available, such as CoreOS’s rkt. An active group — the Open Container Initiative — is also working to standardize container specifications.

Some of the popular container schedulers available are Docker Swarm, Kubernetes, and Apache Mesos; different tools and distributions are built on top of those platforms. Of these, Kubernetes, Google’s open source container scheduler, has the widest mindshare and has garnered significant implementation support from other organizations, such as Microsoft, and the open source community. Because of this popularity and the ease of setting up a local installation, we’ll use Kubernetes in this book.

We significantly increased deployment velocity at our own company using Kubernetes. Whereas our previous approach could take several days to get a new service deployment working smoothly, with Kubernetes, any engineer can now deploy a new service in a few hours.

In this chapter, you’ll get your hands dirty with Docker and Kubernetes. You’ll use Docker to build, store, and run a container for a new service at SimpleBank. And you’ll take that service to production using Kubernetes. Along with these examples, we’ll illustrate how a scheduler executes and manages different types of workloads and how familiar production concepts map to a scheduler platform. We’ll also examine the high-level architecture of Kubernetes.

9.1 Containerizing a service

Let’s jump right in! Over the course of this chapter, your goal will be to take one of SimpleBank’s Python services — market-data — and get it running in production. You can find a starting point for this service in the book’s repository on Github (http://mng.bz/7eN9). Figure 9.1 illustrates the process that will occur. Docker packages service code into a container image, which is stored in a repository. You'll use deploy instructions to tell a scheduler to deploy and operate the packaged service on a cluster of underlying hosts.

As you know, a successful deployment is about more than running a single instance. For each new version, you want to build an artifact that you can deploy multiple times for redundancy, reliability, and horizontal scaling. In this section, you’ll learn how to do the following:

• Build an image for a service
• Run multiple instances — or containers — of your image
• Push your image to a shared repository, or registry

First things first: if you’re going to ship this, you need to figure out how to put it in a box. For this section, you’ll need to have Docker installed. You can find up-to-date instructions online at https://docs.docker.com/install.

9.1.1 Working with images

To package an application into a container, you need to build an image. The image will include the file system that your application needs to run — code and dependencies — and other metadata, such as the command that starts your application. When you run your application, you’ll start multiple instances of this image.

Most powerfully, images can inherit from other images. That means your application images can inherit from public, canonical images for different technology stacks, or you can build your own base images to encapsulate standards and tools you use across multiple services.

To get a feel for working with images, fire up the command line and try to pull a publicly available Docker image:

$ docker pull python:3.6
3.6: Pulling from library/python
ef0380f84d05: Pull complete
24c170465c65: Pull complete
4f38f9d5c3c0: Pull complete
4125326b53d8: Pull complete
35de80d77198: Pull complete
ea2eeab506f8: Pull complete
1c7da8f3172e: Pull complete
e30a226be67a: Pull complete
Digest: sha256:210d29a06581e5cd9da346e99ee53419910ec8071d166ad499a909c49705ba9b
Status: Downloaded newer image for python:3.6

Pulling an image downloads it to your local machine, ready for you to run. In this case, you pulled a Python image from Docker Hub, the default public registry (or repository) for Docker images. Running the following command will start an instance of that image, placing you at a Python interactive shell inside your new container:

$ docker run --interactive --tty python:3.6
Python 3.6.1 (default, Jun 17 2017, 06:29:46)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

You should note a few things here. The –-interactive (or –i) flag indicates that the container should be interactive, accepting input from STDIN, whereas the –-tty (or –t) flag connects a terminal for user input to the Docker container. When you started the container, it executed the default command set within the image. You can check what that is by inspecting the image metadata:

$ docker image inspect python:3.6 --format="{{.Config
➥.Cmd}}"   ①  
[python3]

You can instruct Docker to execute other commands inside your container; for example, to enter the container at an OS shell, rather than Python, you could suffix the command you used to start the image instance with bash.

When you watched the output of your earlier pull command, you might’ve noticed that Docker downloaded multiple items, each identified by a hash — these are layers. An image is a union of multiple layers; when you build an image, each command you run (apt-get update, pip install, apt-get install –y, and so on) creates a new layer. You can list the commands that went into building the python:3.6 image:

$ docker image history python:3.6

Each line that this script returns represents a different command used to construct the python:3.6 image. In turn, some of those layers were inherited from another base image. Commands defined in a Dockerfile specify the layers in an image using a lightweight domain-specific language (DSL). If you look at the Dockerfile for this image — you can find it on Github (http://mng.bz/JxDj) — you’ll notice the first line:

FROM buildpack-deps:jessie

This specifies that the image should inherit from the buildpack-deps:jessie image. If you follow that thread on Docker Hub, you can see that your Python container has a deep inheritance hierarchy that installs common binary dependencies and the underlying Debian operating system. This is detailed in figure 9.2.

Other container ecosystems use different mechanisms — for example, rkt uses the acbuild command-line tool — but the end outcome is similar.

As well as enabling reusability, image layers optimize launch times for containers. If a parent layer is shared between two derivative images on one machine, you only need to pull it from a registry once, not twice.

9.1.2 Building your image

This Python image is a good starting point for you to build your own application image. Let’s take a quick look at the dependencies of the market-data service:

1. It needs to run on an operating system — any distribution of Linux should do.
2. It relies on Python 3.6.x.
3. It installs several open source dependencies from PyPI using pip, a Python package manager.

In fact, this list maps quite closely to the structure of the image that you’re going to build. Figure 9.3 illustrates the relationship between your image and the Python base image you’ve worked with so far.

To build this image, first you need to create a Dockerfile in the root of the market-data service directory. This should do the trick:

Listing 9.1 Dockerfile for application container

FROM python:3.6    ①  
ADD . /app    ②  
WORKDIR /app    ③  

That’s not quite the whole picture, but try building this image and see what it looks like. You can use the docker build command to create an image from a Dockerfile:

$ docker build -t market-data:first-build .
Sending build context to Docker daemon 71.17 kB
Step 1/3 : FROM python:3.6
 ---> 74145628c331
Step 2/3 : ADD . /app
 ---> bb3608d5143f
Removing intermediate container 74c250f83f8c
Step 3/3 : WORKDIR /app
 ---> 7a595179cc39
Removing intermediate container 19d3bffa4d2a
Successfully built 7a595179cc39

This builds an image with the name market-data and the tag first-build. We’ll make more use of tagging later in this chapter. Check that you can start the container and it contains the files you expect:

$ docker run market-data:first-build bash -c 'ls'
Dockerfile
app.py
config.py
requirements.txt

The output of this command should match the contents of the market-data directory. If it did for you, that’s great! You’ve built a new container and added some files — only a few more steps until you have it running an application.

Although you’ve added your application code, you still need to pull down dependencies and start the application up. First, you can use a RUN command within your Dockerfile to execute an arbitrary shell script:

RUN pip install -r requirements.txt

If you recall, the pip tool itself was installed as part of the python base image. If you were working with Ruby or Node, at this point you might call bundle install or npm install; if you were working with a compiled language, you might use a tool like make to produce compiled artifacts.

Next, you need to set the command that’ll be used to start your application. Add another line to your Dockerfile:

CMD ["gunicorn", "-c ", "config.py", "app:app"]

And a final touch: you need to instruct Docker to expose a port to your app. In this case, your Flask app expects traffic on port 8000. Putting that together, you get your final Dockerfile, as shown in the following listing. You should build the image again, this time tagging it as latest.

Listing 9.2 Complete Dockerfile for the market-data service

FROM python:3.6
ADD . /app
WORKDIR /app
RUN pip install -r requirements.txt
CMD ["gunicorn", "-c ", "config.py", "app:app"]
EXPOSE 8000

9.1.3 Running containers

Now that you’ve built an image for the application, you can run it. Try it out:

$ docker run -d -p 8000:8000 --name market-data market-data:latest

This command should return a long hash to the terminal. That’s the ID of your container — you’ve started it in detached mode, rather than in the foreground. You’ve also used the -p flag to map the container port so it’s accessible from the Docker host. If you try and call the service — it has a health-check endpoint at /ping — you should get a successful response:

$ curl -I http://{DOCKER_HOST}:8000/ping   ①  
HTTP/1.0 200 OK
Content-Type: text/plain
Server: Werkzeug/0.12.2 Python/3.6.1

You could easily run multiple instances and balance between them. Try a basic example, using NGINX as a load balancer. Luckily you can pull an NGINX container from the public registry — no hard work to get that running. Figure 9.4 illustrates the containers you’re going to run.

First, start up three instances of the market-data service. Run the code below in your terminal:

$ docker network create market-data    ①  
$ for i in {1..3}    ②  
  do
    docker run -d 
      --name market-data-$i 
      -–network market-data 
      market-data:latest
  done

If you run docker ps -a, you’ll see three instances of the market-data service up and running.

Unlike earlier, you didn’t map each container’s port to the host machine. That’s because you’ll only access these containers through NGINX. Instead, you created a network. Running on the same network will allow the NGINX container to easily discover your market-data instances, using the container name as a host name.

Now, you can set up NGINX. Unlike before, you’re not going to build your own image; instead, you’ll pull the official NGINX image from the public Docker registry. First, you’ll need to configure NGINX to load balance between three instances. Create a file called nginx.conf using the following code.

Listing 9.3 nginx.conf

upstream app {
    server market-data-1:8000;    ①  
    server market-data-2:8000;
    server market-data-3:8000;
}

server {
    listen 80;

    location / {
        proxy_pass http://app;    ②  
    }
}

Then you can start an NGINX container. You’ll use the volume flag (or –v) to mount your new nginx.conf file into the container, sharing it with the local filesystem. This is useful for sharing secrets and configurations that aren’t — or shouldn’t be — built into a container image, such as encryption keys, SSL certificates, and environment-specific configuration files. In this case, you avoid having to build a separate container to include a single new configuration file. Start the container by entering the following:

$ docker run -d --name=nginx 
--network market-data     ①  
--volume `pwd`/nginx.conf:/etc/nginx/conf.d/
➥default.conf     ②  
-p 80:80     ③  
nginx

And that should do the trick. Curling http://localhost/ping should return the hostname — by default, the container ID — of the container instance responding to that request. NGINX will round-robin requests across the three nodes to (naively) balance load across your instances.

9.1.4 Storing an image

Good work so far — you’ve built an image and you’ve seen that it’s easy to run multiple independent instances of an application. Unfortunately, that image isn’t much use in the long run if it’s only on your machine. When it comes to deploying this image, you’ll pull it from a Docker registry. This might be Docker Hub, which you’ve already encountered; a managed registry, such as AWS ECR or Google Container Registry; or self-hosted — for example, using the Docker distribution open source project (https://github.com/docker/distribution). When you build a continuous delivery pipeline, that pipeline will push to your registry on every valid commit.

For now, you can push your image to https://hub.docker.com. First, you’ll need to create an account and choose a Docker ID. This will be the namespace you’ll use to store your containers. Once you’ve logged in, you’ll need to create a new repository — a store for multiple versions of the same image — using the web UI (figure 9.5).

To push to this repository, you need to tag your market-data image with an appropriate name. Docker image names follow the format <registry>/<repository>:<tag>. Once that’s done, a simple docker push will upload your image to the registry. Try it out:

$ docker tag market-data:latest <docker id>/market-data:latest
$ docker login
$ docker push <docker id>/market-data:latest

That’s it! You’ve successfully pushed your image to a public repository. You can double-check that through the web UI (figure 9.6) by logging into https://hub.docker.com. Other engineers (if your repository is private, you’ll need to grant them access) can pull your image using docker pull [image name].

Let’s take stock for a moment:

• You’ve learned how to package a simple application into a lightweight, cross-platform artifact — a container image.
• We’ve explored how Docker images are built from multiple layers to support inheritance from common base containers and increase startup speed.
• You’ve run multiple isolated instances of an application container.
• You’ve pushed the image you built to a Docker registry.

Using these techniques in a build pipeline will ensure greater consistency and predictability across a fleet of services, regardless of underlying programming language, as well as helping to simplify local development. Next, we’ll explore how a container scheduler works by taking your containerized application and deploying it with Kubernetes.

9.2 Deploying to a cluster

A container scheduler is a software tool that abstracts away from underlying hosts by managing the execution of atomic, containerized applications across a shared pool of resources. This is possible because containers provide strong isolation of resources and a consistent API.

Using a scheduler is a compelling deployment platform for microservices because it eases the management of scaling, health checks, and releases across, in theory, any number of independent services. And it does so while ensuring efficient utilization of underlying infrastructure. At a high level, a container scheduler workflow looks something like this:

• Developers write declarative instructions to specify which applications they want to run. These workloads might vary: you might want to run a stateless, long-running service; a one-off job; or a stateful application, like a database.
• Those instructions go to a master node.
• The master node executes those instructions, distributing the workloads to a cluster of underlying worker nodes.
• Worker nodes pull containers from an appropriate registry and run those applications as specified.

Figure 9.7 illustrates this scheduler architecture. To an engineer, where and how an application is executed is ultimately unimportant: the scheduler takes care of it. In addition to running containers, Kubernetes provides other functionality to support running applications, such as service discovery and secret management.

Many well-known cluster management tools are available, but in your case, you’re going to use Kubernetes, an open-source project that evolved from Google’s internal work on Borg and Omega (https://research.google.com/pubs/pub41684.html). It’s possible to run Kubernetes pretty much anywhere — public cloud, private data center, or as a managed service (such as Google Kubernetes Engine (GKE)).

In the next few sections, we’re going to cover a lot of ground. You’ll do the following:

• Learn about the unit of deployment used on Kubernetes — pods
• Define and deploy multiple replicas of a pod for the market-data microservice
• Route requests to your pods using services
• Deploy a new version of the market-data microservice
• Learn how to communicate between microservices on Kubernetes

We’ll start by using Minikube, which will run in a virtual machine on your local host. In a real deployment environment, the master and worker nodes would be separate virtual machines, but locally, the same machine will fulfill both roles. You can find an installation guide for Minikube on the project’s Github page (https://github.com/kubernetes/minikube).

9.2.1 Designing and running pods

The basic building block in Kubernetes is a pod: a single container or a tightly coupled group of containers that are scheduled together on the same machine. A pod is the unit of deployment and represents a single instance of a service. Because it’s the unit of deployment, it’s also the unit of horizontal scalability (or replication). When you scale capacity up or down, you add or remove pods.

You can define a set of pods for your market-data service. Create a file called market-data-replica-set.yml in your app directory. Don’t worry if it doesn’t make much sense yet. Include the following code in your file.

Listing 9.4 market-data-replica-set.yml

---
kind: ReplicaSet    ①  
apiVersion: extensions/v1beta1
metadata:
  name: market-data
spec:
  replicas: 3    ②  
  template:    ③  
    metadata:
      labels:    ④  
        app: market-data    ④  
        tier: backend    ④  
        track: stable    ④  
    spec:
      containers:
      - name: market-data
        image: <docker id>/market-data:latest    ⑤  
        ports:
        - containerPort: 8000

In Kubernetes, you typically declare instructions to the scheduler in YAML files (or JSON, but YAML’s easier on the eyes). These instructions define Kubernetes objects, and a pod is one kind of object. These configuration files represent the desiredstate of your cluster. When you apply this configuration to Kubernetes, the scheduler will continually work to maintain that ideal state. In this file, you’ve defined a ReplicaSet, which is a Kubernetes object that manages a group of pods.

To apply this to your local cluster, you can use the kubectl command-line tool. When you started Minikube, it should have automatically configured kubectl to operate on your cluster. This tool interacts with an API exposed by the cluster’s master node. Give it a try:

$ kubectl apply -f market-data-replica-set.yml
replicaset "market-data" configured

Kubernetes will asynchronously create the objects you’ve defined. You can observe the status of this operation using kubectl. Running kubectl get pods (or kubectl get pods -l app=market-data) will show you the pods that your command has created (figure 9.8). They’ll take a few minutes to start up for the first time as the node downloads your Docker image.

You saw earlier that you didn’t create individual pods. It’s unusual to create or destroy pods directly; instead, pods are managed by controllers. A controller is responsible for taking some desired state — say, always running three instances of the market-data pod — and performing actions to reach that state. This observe-diff-act loop happens continually.

You’ve just encountered the most common type of controller: the ReplicaSet. If you’ve ever encountered instance groups on AWS or GCP, you might find their behavior similar. A replica set aims to ensure a specific number of pods are running at any one time. For example, let’s say a pod dies — maybe a node in the cluster failed — the replica set will observe that the state of the cluster no longer matches the desired state and will attempt to schedule a replacement elsewhere in the cluster.

You can see this in action. Delete one of the pods you’ve just created (pods are identified by name):

$ kubectl delete pod <pod name>

The replica set will schedule a new pod to replace the one you destroyed (figure 9.9).

This matches the ideal we laid out in chapter 8: that deploying microservice instances should be built on a single primitive operation. By combining controllers and immutable containers, you can treat pods like cattle and rely on automation to maintain capacity, even when the underlying infrastructure is unreliable.

9.2.2 Load balancing

Right, so you’re running a microservice on Kubernetes. That was pretty quick. The bad news is, you can’t access those pods yet. Like you did earlier with NGINX, you need to link them to a load balancer to route requests and expose their capabilities to other collaborators, either inside or outside your cluster.

In Kubernetes, a servicedefines a set of pods and provides a method for reaching them, either by other applications in the cluster or from outside the cluster. The networking magic that achieves this feat is outside the scope of this book, but figure 9.10 illustrates how a service would connect to your existing pods.

Now, you’re currently running a replica set containing three market-data pods. If you recall from listing 9.4, your market-data pods have the labels app: market-data and tier: backend. That’s important, because a service forms a group of pods based on their labels.

To create a service, you need another YAML file, as shown in the following listing. This time, call it market-data-service.yml (great naming convention).

Listing 9.5 market-data-service.yml

---
apiVersion: v1
kind: Service
metadata:
  name: market-data
spec:
  type: NodePort
  selector:    ①  
    app: market-data    ①  
    tier: backend    ①  
  ports:
    - protocol: TCP
      port: 8000    ②  
      nodePort: 30623    ③  

Apply this configuration using the same $ kubectl apply -f command you used to create the replica set before, substituting the name of your new YAML file. This will create a service accessible on port 30623 of your cluster, which routes requests to your market-data pods on port 8000.

You should be able to curl your service and send requests to your pods. Doing so will return the name of each pod that serves the request:

$ curl http://`minikube ip`:30623/ping    ①  

Several types of services are available, and they’re outlined in table 9.1. In this case, you used a NodePort service to map your service to an externally available port on your cluster, but if only other cluster services access your microservice, it usually makes more sense to use ClusterIP to keep access local to the cluster.

The service listens for events across the cluster and will be dynamically updated if the group of pods changes. For example, if you kill a pod, it will be removed from the group, and the service will route requests to any new pod created by the replica set.

9.2.3 A quick look under the hood

So far, this has been seamless: you send an instruction, and Kubernetes executes it! Let’s take a moment to learn how Kubernetes runs your pods.

If you drill down a level, you can see that the master and worker nodes on Kubernetes run several specialized components. Figure 9.11 illustrates these components.

Components of the master node

The master node consists of four components:

• The API server — When you ran commands on kubectl, this is what it communicated with to perform operations. The API server exposes an API for both external users and other components within the cluster.
• The scheduler — This is responsible for selecting an appropriate node where a pod will run, given priority, resource needs, and other constraints.
• The controller manager — This is responsible for executing control loops: the continual observe-diff-act operation that underpins the operation of Kubernetes.
• A distributed key-value data store, etcd — This stores the underlying state of the cluster and thereby makes sure it persists when nodes fail or restarts are required.

Together, these components act as a control plane for the cluster. Picture this as something like the cockpit of an airplane. Together, these components provide the API and backend required to orchestrate operations across a cluster of nodes.

Components of a worker node

Each worker node uses the following components to run and monitor applications:

• A container runtime — In your case, this is Docker.
• The kubelet — This interacts with the Kubernetes master to start, stop, and monitor containers on the node.
• The kube-proxy — This provides a network proxy to direct requests to and between different pods across the cluster.

These components are relatively small and loosely coupled. A key design principle of Kubernetes is to separate concerns and ensure components can operate autonomously — a little like microservices!

Watches for state changes

The API server is responsible for recording the state of the cluster — and receiving instructions from clients — but it doesn’t explicitly tell other components what to do. Instead, each component works independently to orchestrate cluster behavior when some event or change occurs. To learn about state changes, each component watches the API server: a component requests to be notified by the API server when something interesting happens, so it can perform appropriate actions to attempt to match the desired state.

For example, the scheduler needs to know when it should assign new pods to nodes. Therefore, it connects to the API server to receive a continuous stream of events that relate to the pod resource. When it receives a notification about a newly created pod, it finds an appropriate node for that pod. Figure 9.12 shows this process.

In turn, your kubelets watch the API server to learn when a pod has been assigned to its node and then they start the pod appropriately. Each component watches resources and events that interest it; for example, the controller manager watches replica sets and services (among other things).

Understanding how pods are run

What happens when you create a replica set? You saw earlier that this results in the expected number of pods being run — from your perspective, it looked simple! But in reality, creating your replica set through kubectl triggers a complex chain of events across multiple components. This chain is illustrated in figure 9.13.

Let’s walk through each step:

1. You instructed the API server to create a new replica set, using kubectl. The API server stores this new resource in etcd.
2. The controller manager watches for creation and modification of replica sets. It receives a notification about the new set you created.
3. The controller manager compares the current state of the cluster to the new state, determining that it needs to create new pods. It creates these pod resources through the API server, based on the template you provided through kubectl.
4. The scheduler receives a notification about a new pod and assigns it an appropriate node, again updating the pod’s definition through the API server. At this point, you haven’t run any real application — the controllers and scheduler have only updated the state that the API server is storing.
5. Once the pod is assigned to a node, the API server notifies the appropriate kubelet, and the kubelet instructs Docker to run containers. Images are downloaded, containers are started, and the kubelet begins to monitor their operation. At this point, your pods are running!

As you can see, each component acts independently, but together, they orchestrate a complex deployment action. Hopefully this has given you a useful glance under the cover. Now, back to running your microservices.

9.2.4 Health checks

You’re missing something. Unlike a typical cloud load balancer, a Kubernetes service doesn’t itself execute health checks on your underlying application. Instead, the service checks the shared state of the cluster to determine if a pod is ready to receive requests. But how do you know if a pod is ready?

In chapter 6, we introduced two types of health check:

• Liveness — Whether an application has started correctly
• Readiness — Whether an application is ready to serve requests

These health checks are crucial to the resiliency of your service. They ensure that traffic is routed to healthy instances of your microservice and away from instances that are performing poorly (or not at all).

By default, Kubernetes executes lightweight, process-based liveness checks for every pod you run. If one of your market-data containers fails a liveness check, Kubernetes will attempt to restart that container (as long as the container’s restart policy isn’t set to Never). The kubelet process on each worker node carries out this health check. This process continually queries the container runtime (in your case, the Docker daemon) to establish whether it needs to restart a container.

This alone isn’t adequate, as your microservice may run into failure scenarios that don’t cause the container itself to fail: whether deadlocks due to request saturation, timeouts of underlying resources, or a plain old coding error. If the scheduler can’t identify this scenario, performance can deteriorate as a service routes requests to unresponsive pods, potentially leading to cascading failures.

To avoid this situation, you need the scheduler to continually check the state of the application inside your container, ensuring it’s both live and ready. With Kubernetes, you can configure probes to achieve this, which you can define as part of your pod template. Figure 9.14 illustrates how these checks, and the previous process check, will be run.

Adding probes is straightforward, although you do need to add some configuration see the next listing to the container specification in market-data-replica-set.yml. Probes can be HTTP GET requests, scripts executed inside a container, or TCP socket checks. In this case, you’ll use a GET request, as shown in the following listing.

Listing 9.6 Liveness probe in market-data-replica-set.yml

livenessProbe:    ①  
  httpGet:    ①  
    path: /ping    ①  
    port: 8000    ①  
  initialDelaySeconds: 10    ①  
  timeoutSeconds: 15    ①  
readinessProbe:    ②  
    path: /ping    ②  
    port: 8000    ②  
  initialDelaySeconds: 10    ②  
  timeoutSeconds: 15    ②  

Reapply this configuration using kubectl to update the state of the replica set. Kubernetes will, to the best of its ability, use these probes to help ensure instances of your microservice are alive and kicking. In this example, both liveness and readiness check the same endpoint, but if your microservice has external dependencies, such as a queueing service, it makes sense to make readiness dependent on connectivity from your application to those dependencies.

9.2.5 Deploying a new version

You should now understand how you use replica sets, pods, and services to run stateless microservices on Kubernetes. On top of these concepts, you can build a stable, seamless deployment process for each of your microservices. In chapter 8, you learned about canary deployments; in this section, you’ll try out the technique with Kubernetes.

Deployments

Before we get started, we should quickly introduce deployments. Kubernetes provides a higher level abstraction, the Deployment object, for orchestrating the deployment of new replica sets. Each time you update a deployment, the scheduler will orchestrate a rolling update of instances in a replica set, ensuring they’re deployed seamlessly.

You can change the original approach to use a deployment instead. First, delete your original replica set:

$ kubectl delete replicaset market-data

After that, create a new file, market-data-deployment.yml. This should be similar to the replica set you created earlier, except that the type of object should be Deployment, rather than ReplicaSet, as shown in the following listing.

Listing 9.7 market-data-deployment.yml

---
apiVersion: extensions/v1beta1
kind: Deployment    ①  
metadata:
  name: market-data
spec:    ②  
  replicas: 3    ②  
  template:    ③  
    metadata:
      labels:
        app: market-data
        tier: backend
        track: stable
    spec:
      containers:
      - name: market-data
        image: <docker id>/market-data:latest
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
        ports:
        - containerPort: 8000
        livenessProbe:
          httpGet:
            path: /ping
            port: 8000
          initialDelaySeconds: 10
          timeoutSeconds: 15
        readinessProbe:
          httpGet:
            path: /ping
            port: 8000
          initialDelaySeconds: 10
          timeoutSeconds: 15

Use kubectl to apply this file to the cluster. This will create a deployment, which will create a replica set and three instances of the market-data pod.

Canaries

In a canary deploy, you deploy a single instance of a microservice to ensure that a new build is stable when it faces real production traffic. This instance should run alongside existing production instances. A canary release has four steps:

1. You release a single instance of a new version alongside the previous version.
2. You route some proportion of traffic to the new instance.
3. You assess the health of the new version by, for example, monitoring error rates or observing behavior.
4. If the new version is healthy, you commence a full rollout to replace other instances. If not, you remove the canary instance, halting the release.

On Kubernetes, you can use labels to identify a canary pod. In the first example, you specified a label track: stable on each pod in your replica set. To deploy a canary, you’ll need to deploy a new pod that’s distinguished with track: canary. The service you created earlier only selects on two labels (app and tier), so it’ll route requests to both stable and canary pods. This is illustrated in figure 9.15.

First, you should build a new container for your new release. You’ll use tags to identify the new version, and don’t forget to substitute your own Docker ID:

$ docker build -t <docker id>/market-data:v2 .
$ docker push <docker id>/market-data:v2

This version is tagged as v2, although in practice it may not be appropriate to apply a numeric versioning scheme to your services. We’ve found tagging them with the commit ID also works well. (For Git repositories, we use git rev-parse --short HEAD.)

Once you’ve pushed that new image, create a yml file specifying your canary deployment:

• It should create one replica, instead of three.
• It should release the v2 tag of the container, rather than latest.
• It should look like the following listing.

Listing 9.8 market-data-canary-deployment.yml

---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: market-data-canary
spec:
  replicas: 1    ①  
  template:
    metadata:
      labels:
        app: market-data    ②  
        tier: backend    ②  
        track: canary    ②  
    spec:
      containers:
      - name: market-data
        image: <docker id>/market-data:v2    ③  
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
        ports:
        - containerPort: 8000
        livenessProbe:
          httpGet:
            path: /ping
            port: 8000
          initialDelaySeconds: 10
          timeoutSeconds: 15
        readinessProbe:
          httpGet:
            path: /ping
            port: 8000
          initialDelaySeconds: 10
          timeoutSeconds: 15

Use kubectl to apply this to your cluster. The deployment will create a new replica set containing a single canary pod for v2.

Let’s take a closer look at the state of your cluster. If you run minikube dashboard on the command line, it’ll open the dashboard for your cluster in a browser window (figure 9.16). In the dashboard — under Workloads — you should be able to see:

• The canary deployment you’ve just created, as well as your original deployment
• Four pods: the original three, plus a canary pod
• Two replica sets: one each for the stable and canary tracks

So far so good! At this stage, for a real microservice, you might run some automated tests, or check the monitoring output of your service to ensure it’s processing work as expected. For now, you can safely assume your canary is healthy and performing as expected, which means you can safely roll out the new version, replacing all your old instances.

Edit the market-data-deployment.yml file and make two changes:

• Change the container used to market-data:v2.
• Add a strategy field to specify how pods will be updated.

Your updated deployment file should look like the following listing.

Listing 9.9 Updated market-data-deployment.yml

---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: market-data
spec:
  replicas: 3
  strategy:    ①  
    type: RollingUpdate    ①  
    rollingUpdate:    ①  
      maxUnavailable: 50%    ①  
      maxSurge: 50%    ①  
  template:
    metadata:
      labels:
        app: market-data
        tier: backend
        track: stable
    spec:
      containers:
      - name: market-data
        image: morganjbruce/market-data:v2
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
        ports:
        - containerPort: 8000

Applying this configuration will create a new replica set, starting instances one by one while removing them from the original set. This process is illustrated in figure 9.17.

You can also observe this in the event history of the controller by running kubectl describe deployment/market-data (figure 9.18).

From this history, you can see how Kubernetes allows you to build higher level deployment operations on top of simple operations. In this case, the scheduler used your desired state of the world and a set of constraints to determine an appropriate path of deployment, but you could use replica sets and pods to build any deployment pattern that was appropriate for your service.

9.2.6 Rolling back

Well done! You’ve smoothly deployed a new version of your microservice. If something went wrong, you can also use the deployment object to undo all your hard work. First, check the rollout history:

$ kubectl rollout history deployment/market-data

This should return two revisions: your original deployment and your v2 deployment. To roll back, specify the target revision:

$ kubectl rollout undo deployment/market-data --to-revision=1

This will perform the reverse of the previous rolling update to return the underlying replica set to its original state.

9.2.7 Connecting multiple services

Lastly, your microservice isn’t going to be much use by itself, and several of SimpleBank’s services depend on the capabilities that the market-data service provides. It’d be pretty much insane to hardcode a port number or an IP address into each service to refer to the underlying endpoint of each collaborator; you shouldn’t tightly couple any service to another’s internal network location. Instead, you need some way of accessing a collaborator by a known name.

Kubernetes integrates a local DNS service to achieve this, and it runs as a pod on the Kubernetes master. When new service is created, the DNS service assigns a name in the format {my-svc}.{my-namespace}.svc.cluster.local; for example, you should be able to resolve your market-data service from any other pod using the name market-data.default.svc.cluster.local.

Give it a shot. You can use kubectl to run an arbitrary container in your cluster — try busybox, which is a great little image containing several common Linux utilities, such as nslookup. Run the following command to open a command prompt inside a container running on Minikube:

$ kubectl run -i --tty lookup --image=busybox /bin/sh

Then you can try an nslookup:

/ # nslookup market-data.default.svc.cluster.local

You should get output that looks something like this:

Server:    10.0.0.10
Address 1: 10.0.0.10 kube-dns.kube-system.svc.cluster.local

Name:      market-data.default.svc.cluster.local
Address 1: 10.0.0.156 market-data.default.svc.cluster.local

The IP address in the last entry should match the cluster IP assigned to your service. (If you don’t believe me, you can double-check by calling kubectl get services.) If so, success! You’ve covered a lot of ground: building and storing an image for a microservice, running it on Kubernetes, load-balancing multiple instances, deploying a new version (and rolling back), and connecting microservices together.

Summary

• Packaging microservices as immutable, executable artifacts allows you to orchestrate deployment through a primitive operation — adding or removing a container.
• Schedulers and containers abstract away underlying machine management for service development and deployment.
• Schedulers work by trying to match the resource needs of an application to the resource usage of a cluster of machines, while health-checking running services to ensure they’re operating correctly.
• Kubernetes provides ideal features of a microservice deployment platform, including secret management, service discovery, and horizontal scalability.
• A Kubernetes user defines the desired state (or specification) of their cluster services, and Kubernetes figures out how to achieve that state, executing a continual loop of observe-diff-act.
• The logical application unit on Kubernetes is a pod: one or more containers that execute together.
• Replica sets manage the lifecycle of groups of pods, starting new pods if existing ones fail.
• Deployments on Kubernetes are designed to maintain service availability by executing rolling updates of pods across replica sets.
• You can use service objects to group underlying pods and make them available to other applications inside and outside of the cluster.