The two main objectives of a microservices architecture is the speed to production and the capability of the application to evolve. Unlike a monolithic application, a microservices deployment includes many individual (and independent) deployments. Instead of one single deployment, now we have hundreds of deployments. Unless we have an automated build system, managing such a deployment is a nightmare. An automated build system will help to streamline the deployment process, but will not solve all the issues in a large-scale deployment. We also need to worry about making a microservice portable, along with all its dependencies, so that the environment the developer tests will not be different from the test and production environments. This helps identify any issues very early in the development cycle and the chances are quite minimal that there will be issues in production. In this chapter we talk about different microservices deployment patterns, containers, container orchestration, container native microservices frameworks, and finally continuous delivery.
Containers and Microservices
The primary goal of a container is to provide a containerized environment for the applications it runs. A containerized environment is an isolated environment. One or more containers can run on the same physical host machine. But one container does not know about the processes running in other containers. For example, if you run your microservice in a container, it has its own view of the filesystem, network interfaces, processes, and hostname. Let’s say a foo microservice, which is running in the foo container, can refer to any other service running in the same container with the hostname localhost, while a bar microservice, which is running in the bar container, can refer to any other service running in the same container with the hostname localhost, while both foo and bar containers running on the same host machine1.
The concept of containers was made popular a few years back with Docker. But it has a long history. In 1979, with chroot system call in UNIX V7, users were able to change the root directory of a running process, so it will not be able to access any part of the filesystem, beyond the root. This was added to BSD in 1982, and even today, chroot is considered a best practice among sys-admins, when you run any process that’s not in a containerized environment. A couple of decades later in 2000, FreeBSD introduced a new concept called FreeBSD Jails. With Jails, the same host environment can be partitioned into multiple isolated environments, where each environment can have its own IP address. In 2001, Linux VServer introduced a similar concept like FreeBSD Jails. With Linux VServer, the host environment can be partitioned by the filesystem, network, and the memory. Solaris came up with the Solaris Containers in 2004, which introduced another variation of process isolation. Google launched Process Containers in 2006, which was designed to build isolation over CPU, disk I/O, memory and the network. A year later this was renamed to control groups and merged into the Linux kernel 2.6.24.
Linux cgroups and namespaces are the foundation of the containers we see today. LXC (Linux Containers) in 2008 implemented the Linux Container Manager using cgroups and namespaces. CloudFoundry, in 2011, came up with a container implementation based on LXC, called Warden, but later changed it to its own implementation. Docker was born in 2013, and just like Warden, it too was built on top of LXC. Docker made container technologies much more usable and in the next sections, we’ll talk about Docker in detail.
Introduction to Docker
As we discussed, the containerization provided by Docker is built on top of Linux control groups and namespaces. The Linux namespaces build isolation in a way that each process sees its own view of the system: file, process, network interfaces, hostname, and many more. Control groups, also known as cgroups, limit the number of resources each process can consume. The combination of these two can build an isolated environment on the same host machine, sharing the same CPU, memory, network and the filesystem, with no impact on others.
Unlike a virtual machine, all the containers deployed in the same host machine share the same operating system kernel. It’s in fact an isolated process. To boot up a container, there is no overhead in booting up an operating system; it’s just the application running in the container. Also, since containers do not pack the operating system, you can run many containers in the same host machine. These are the key driving forces behind picking containers as the most popular way of deploying and distributing microservices.
Docker builds the process isolation on top of control groups and namespaces come with the Linux kernel. Even without Docker, we could still do the same with the Linux kernel. So, why has Docker become the most popular container technology? Docker adds several new features, apart from process isolation to make it more attractive to the developer community. Making containers portable, building an ecosystem around Docker Hub, exposing an API for container management and building tooling around that, and making container images reusable are some of these features. We discuss each of them in detail in this chapter.
Installing Docker
Docker Architecture
Docker Images
A Docker image is a package that includes your microservice (or your application) along with all its dependencies. Your application will see only the dependencies that you pack in your Docker image. You define the filesystem for your image, and it will not have access to the host filesystem. A Docker image is created using a Dockerfile. The Dockerfile defines all the dependencies to build a Docker image.
Note
For readers who are keen on exploring more on Docker concepts, we recommend following the Docker documentation3.
The second line says to copy sample01-1.0.0.jar from the target directory under the current location of the filesystem (this is in fact our host filesystem, which we use to create a Docker image) to the root of the filesystem of the image we want to create. The third line defines the entry point or the command to execute when someone runs this Docker image.
Docker Registry
If you are familiar with Maven, you may already know how Maven repositories work. The Docker registries follow a similar concept. A Docker registry is a repository of Docker images. They operate at different levels. The Docker Hub is a public Docker registry where anyone can push and pull images. Later in the chapter, we explain how to publish and retrieve images to and from the Docker Hub. Having a centralized registry of all the images helps you share and reuse them. It is also becoming a popular way of distributing software. If your organization needs a restricted Docker registry just for its employees, that can be done too.
Containers
Deploying Microservices with Docker
This will result in the target/sample01-1.0.0.jar file. This is our microservice, and now we need to create a Docker image with it.
Note
To run the examples in this chapter, you need Java 8 or latest, Maven 3.2 or latest, and a Git client. Once you have successfully installed those tools, you need to clone the Git repo: https://github.com/microservices-for-enterprise/samples.git . The chapter samples are in the ch08 directory.
:> git clone https://github.com/microservices-for-enterprise/samples.git
Creating a Docker Image with a Microservice
There is one important thing to learn from this output. There you can see Docker engine performs the operation in three steps. There is a step corresponding to each line in the Dockerfile. Docker builds images as layers. Each step in this operation creates a layer. These layers are read-only and reusable between multiple containers. Let’s revisit this concept, later in this chapter, once we better understand the behavior of containers.
Running a Microservice as a Docker Container
This command instructs the Docker engine to spin up a new Docker container from the sample01 image and map port 9000 of the host machine to port 9000 of the Docker container. We picked 9000 here, because our microservice in the container starts on port 9000. Unless we map the container port to a host machine port, we won’t be able to communicate with our microservice running in a container.
Hint
If we want to see all the containers running on our Docker engine, we can use the docker ps command. To stop a running container, we need to use the docker stop <container id> command. Also if you want remove a container, use the docker rm <container id> command. Once you remove the container, you can delete the corresponding Docker image from your local registry with the docker rmi <image name> command.
Publishing a Docker Image to the Docker Hub
Docker Compose
In practice, in a microservices deployment we have more than one service, where each service has its own container. For example, in Figure 8-4 the Order Processing microservice talks to the Inventory microservice. Also, there can be cases where one microservice depends on other services like a database. The database will be another container, but still part of the same application. The Docker Compose helps define and manage such multi-container environments. It’s another tool that we have to install apart from the Docker client and the Docker engine (host).
Note
Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration4. You can refer to the Docker documentation5 for more details. It’s quite straightforward.
Here we define two services, inventory and orderprocessing . The ports tag under each service defines the port forwarding rules (which we discussed). The image tag points to the Docker image. The depends_on tag defined under orderprocessing states that it depends on the inventory service. You can define multiple services under the depends_on tag. Finally, you may notice the value of the restart tag is set to always, under both the services, which instructs the Docker engine to restart the services always, in case they are down.
Building a Docker Image for Inventory Microservice
Launching the Application with Docker Compose
Since we are launching docker-compose here in the attached mode (with no –d) you will be able to see the output from both the containers on your terminal. Also you will notice that each log from both the microservices is tagged with the corresponding service name defined in the docker-compose.yml file.
Testing the Application with cURL
How Does the Communication Happen Between Containers?
Note
By default, Compose sets up a single network for your app. Each container for a service joins the default network and is both reachable by other containers on that network, as well as discoverable by them at a hostname identical to the container name6.
Container Orchestration
Containers and Docker took most of the pain out of working on microservices. If not for Docker, microservices wouldn’t be popular today. Then again, containers only solve one part of the problem in a large-scale microservices deployment. How about managing the lifecycle of a container from the point it is born to the point it is terminated? How about scheduling containers to run on different physical machines in a network, tracking their running status, and load balancing between multiple containers of a given cluster? How about autoscaling containers to meet varying requests or the load on the containers? These are the issues addressed in a container orchestration framework. Kubernetes and Apache Mesos are the most popular container orchestration frameworks. In this chapter, we only focus on Kubernetes.
Introduction to Kubernetes
In short, Docker abstracts the machine (or the computer), while Kubernetes abstracts the network. Google introduced Kubernetes to the public in 2014 as an open source project. Before Kubernetes, for many years, Google worked on a project called Borg, to help their internal developers and system administrators manage thousands of applications/services deployed over large datacenters. Kubernetes is the next phase of Borg.
Kubernetes lets you deploy and scale an application of any type, running on a container, to thousands of nodes effortlessly. For example, in a deployment descriptor (which is understood by Kubernetes), you can specify how many instances of the Order Processing microservice you need to have.
Kubernetes Architecture
The controller manager is responsible for managing and tracking all the nodes in the Kubernetes deployment. It makes sure components are replicated properly and autoscaled, the failures are handled gracefully, as well as performs many other tasks. The etcd is a highly available, consistent datastore, which stores the Kubernetes cluster configuration.
A worker node consists of three components—a kubelet, a container runtime, and a kube-proxy. The container runtime can be based on Docker or rkt. Even though the Kubernetes container runtime was initially tied to Docker and rkt, it can now be extended to support any Open Container Initiative (OCI)-based container runtimes via the Container Runtime Interface (CRI). The responsibility of the kubelet is to manage the node by communicating with the API server running on the master node. It is running on each worker node and acts as a node agent to the master node. Kube-proxy or the Kubernetes network proxy does simple TCP and UDP stream forwarding or round-robin TCP and UDP forwarding across a set of backends.
Installing Kubectl
The Kubernetes master node can run anywhere. To interact with the master node, we need to have kubectl installed locally in our machine. The kubectl installation instructions are available at https://kubernetes.io/docs/tasks/tools/install-kubectl/ . Make sure that the version of kubectl is within one minor version difference of your cluster.
Installing Minikube
Minikube is the best way to get started with Kubernetes in your local machine. Unlike a production Kubernetes cluster, Minikube only supports one-node Kubernetes cluster. In this chapter, we use Minikube to set up a Kubernetes cluster. Minikube installation details are available at https://kubernetes.io/docs/tasks/tools/install-minikube/ . You will never use Minikube in a production environment, but the Kubernetes concepts that you learn with Minikube are still valid across any type of a Kubernetes distribution.
Test the Kubernetes Setup
Kubernetes Core Concepts
In this section, we discuss fundamental concepts associated with Kubernetes. Before we get there, let’s look at how Kubernetes works with containers.
The first thing we need to do prior to a Kubernetes deployment is to identify and package our applications into container images, where at runtime, each application will have its own isolated container. Then we need to identify how we are going to group these containers. For example, there can be microservices that run always together, and only one microservice exposes the functionality to the outsiders. In all the other cases, communication between microservices is just internal. Another example is a database we have as a container, which is only used by one microservice. In that case, we can group the database container and the corresponding microservice together. Well, someone of course can argue this as an anti-pattern. We do not disagree. Let’s revisit our example, after defining the term pod, within the context of Kubernetes.
Pod
In Kubernetes, we call a group of containers a pod. A pod is the smallest deployment unit in Kubernetes. You cannot just deploy containers. First we need to group one or more containers as a pod. Since the pod is the smallest deployment unit in Kubernetes, it can scale only the pods, not the containers. In other words, all the containers grouped as a pod must have the same scalability requirements. If we revisit our previous example, you can group a set of microservices together in a pod. If all of them have the same scalability requirements. But grouping a microservice with a database in a pod is not an ideal use case. Usually a database has different scalability requirements than a microservice.
Creating a Pod
Let’s look at the following deployment descriptor, which is used to create a pod with the Order Processing microservice and the Inventory microservice. Here first we set the value of the kind attribute to Pod, and later under the spec attribute, we define the set of images need to be grouped into this pod.
Important
Here we made an assumption that both the Order Processing and Inventory microservices fit into a pod and both of them have the same scalability requirements. This is a mere assumption we made here to make the examples straightforward and explain the concepts.
Once the application developer comes up with the deployment descriptor, he or she will feed that into the Kubernetes control plane using kubectl, via the API server. Then the scheduler will schedule the pods on worker nodes, and the corresponding container runtime will pull the container images from the Docker registry.
Note
In the Kubernetes deployment descriptor we point to a slightly modified version of the Order Processing microservice (from what we discussed earlier) called sample04, instead of the sample01.
ReplicaSet
When you deploy a pod in a Kubernetes environment, you can specify how many instances of a given pod you want to keep running all the time. There can be many other scaling requirements as well, which we discuss later in the chapter with examples. It is the responsibility of the ReplicaSet7 to monitor the number of running pods and make sure to maintain the expected number. If one pod goes down for some reason, the ReplicaSet will make sure to spin up a new one. Later in the chapter, we explain how ReplicaSet works.
Service
In a way, a service8 in Kubernetes is a grouping of pods that provide the same functionality. These pods in fact are different instances of the same pod. For example, you may run five instances of the same Order Processing microservice as five pods to cater to high traffic. These pods are exposed to the outside world, via a service. A service has its own IP address and port, which will never change during its lifetime and it knows how to route traffic to the attached pods. None of the microservice client applications need worry about talking directly to a pod, but to a service. At the same time, based on the scalability requirements and other reasons, pods may come up and go down. Whenever a pod comes up and goes down, it may carry a different IP addresses and it is hard to make any connection from a client application to a pod. The service in Kubernetes solves this problem. There is an example later in the chapter that demonstrates how to create a service in Kubernetes.
Deployment
Deployment9 is a higher-level construct that can be used to deploy and manage applications in a Kubernetes environment. It uses the ReplicaSet (which is a low-level construct) to create pods. We discuss how to use the deployment construct later in this chapter.
Deploying Microservices in a Kubernetes Environment
In this section, we see how to create a pod with two microservices and invoke one microservice from the host machine using cURL while the two microservices communicate with each other, in the same pod.
Creating a Pod with a YAML File
Creating a Service with a YAML File
Here you can see the value of the kind attribute is set to Service, and under selector/app attribute, the value is set to the label of the ecomm-pod we created before. This service exposed to the outside world via HTTP port 80 and the traffic coming to it will be routed to port 9000 (targetPort), which is the port of the Order Processing microservice. Finally, another important attribute we cannot miss is the type, where the value is set to NodePort. When the value of the service type is set to NodePort, it exposes the service on each node’s IP at a static port.
Testing the Pod with cURL
How Does Communication Happen Between Containers in the Same Pod?
In addition to using HTTP over localhost, there are two other popular options for inter-container communication within a single pod: using shared volumes and using inter-process communication (IPC). We recommend interested readers to refer to Kubernetes documentation10 on those topics.
How Does Communication Happen Between Containers in Different Pods?
The containers in different pods communicate with each other using the pod IP address and the corresponding port. In a Kubernetes cluster, each pod has its own distinct IP address.
Deployments and Replica Sets
Even though we discussed creating a pod using a deployment descriptor, in practice you will never use it. You’ll use a deployment instead. A deployment is an object type in Kubernetes (just like a pod) that helps us manage pods using ReplicaSets. A ReplicaSet is an essential component in a Kubernetes cluster where you can specify how you want to scale up and down a given pod. Before we create a deployment, let’s do some clean up by deleting the pod we created and the corresponding service.
Scaling a Deployment
Autoscaling a Deployment
Helm: Package Management with Kubernetes
Helm11 is a package manager for Kubernetes, and it makes it possible to organize Kubernetes objects in a packaged application that anyone can download and install in one click or customize. Such packages are known as charts in Helm. Using Helm charts, you can define, install, and upgrade even the most complex Kubernetes applications.
A chart is a collection of files that describes a related set of Kubernetes resources. A single chart might be used to deploy something simple, like a memcached pod, or something complex, like a full web app stack with HTTP servers, databases, caches, and so on. Installing a chart is quite similar to installing a package using a package management tool such as Apt or Yum. Once you have Helm up and running, installing a package can be as simple as running helm install stable/mysql in the command line.
Helm charts describe even the most complex apps, provide repeatable application installation, and serve as a single point of authority. Also, you can build lifecycle management for your Kubernetes-based microservices as charts, which are easy to version, share, and host on public or private servers. You can also roll back to any specific version if required.
Microservices Deployment Patterns
Based on the business requirements, we find that there are multiple deployment patterns for microservices. Microservices were there a while, even before the containers became mainstream, and these deployment patterns have evolved over time. In the following sections, we weigh the pros and cons of each of these deployment patterns.
Multiple Services per Host
This model is useful when you have fewer microservices and do not expect each microservice to be isolated from the others. Here the host can be a physical machine or a virtual machine. This pattern will not scale when you have multiple microservices, and also will not help you in achieving the benefits of a microservices architecture.
Service per Host
With this model, the physical host machine isolates each microservice. This pattern will not scale when you have many microservices, even for something aroung 10. It will be a waste of resources and will be a maintenance nightmare. Also, it becomes harder and more time consuming to replicate the same operating environment across development, test, staging, and production setups.
Service per Virtual Machine
With this model, the virtual machine isolates each microservice. This pattern is better than the previous service per host model, but still will not scale when you have many microservices. A virtual machine carries a lot of overhead, and we need to have powerful hardware to run multiple virtual machines in a single physical host. Also, due to the increased size, the virtual machine images are less portable.
Service per Container
This is the most common and the recommended deployment model. Each microservice is deployed in its own container. This makes microservices more portable and scalable.
Container-Native Microservice Frameworks
Most of the microservices frameworks and programming languages that we use to build microservices are not designed to work with container management and orchestration technologies by default. Therefore, developers or DevOps have to put extra effort to create the artifacts/configurations that are required for deploying the applications as containers. There are certain technologies, such as Metaparticle.io, that try to build a uniform set of plugins to incorporate such container-native capabilities into the application’s or microservice’s code that you develop, as annotations. We discuss several of them next.
Metaparticle
Metaparticle12 is getting some traction when it comes to microservice development as it provides some interesting features to harness your applications with containers and Kubernetes.
Metaparticle is a standard library for cloud-native applications on Kubernetes. The objective of Metaparticle is to democratize the development of distributed systems by providing simple, but powerful building blocks, built on top of containers and Kubernetes.
Containerize your applications.
Deploy your applications to Kubernetes.
Quickly develop replicated, load-balanced services.
Handle synchronization like locking and master election between distributed replicas.
Easily develop cloud-native patterns like sharded systems.
Here, we use the @Package annotation to describe how to package the application and specify the Docker Hub username. Also, we need to wrap the main function in the Containerize function, which triggers the Metaparticle code when we build our microservice application.
Now we can build your application with mvn compile and, once the build is successful, we can run it with mvn exec:java -Dexec.mainClass=io.metaparticle.tutorial.Main.
After we compile and run this, we can see that there are four pod replicas running behind a Kubernetes ClusterIP service.
Containerizing a Spring Boot Service
In addition to seamless containerization, it also offers other features such as distributed synchronization, sharding, etc. In addition to Metaparticle, languages such as Ballerina.io support such capabilities via the annotations. You can find the complete example in the ch08/sample06 directory.
Spring Boot and Docker Integration
You can add the docker-file plugin13 to your pom file of your project. The complete code example is in the ch08/sample07 directory.
When it comes to Spring Boot runtime, it uses an embedded web server such as Tomcat to deploy and boot up your services. So you can easily start a microservice with its embedded Tomcat runtime. Although the startup time and memory footprint are relatively high (several seconds), still we can consider Spring Boot a container native technology.
Ballerina: Docker and Kubernetes Integration
During the Ballerina build phase, it will generate Docker images and respective Kubernetes deployment artifacts. The complete code example is in the ch08/sample08 directory. Ballerina deployment choices are so diverse so that you can deploy it in a conventional virtual machine (VM) or bare metal servers, Docker, Kubernetes and on a service mesh, if you are using a Service Mesh such as Istio.
Continuous Integration, Delivery, and Deployment
One of the key rationales behind microservice architecture is less time to production and shorter feedback cycles. One cannot meet such goals without automation. A good microservices architecture will only look good on paper (or on a whiteboard), if not for the timely advancements in DevOps and tooling around automation. Microservices came as a good idea, as they have all the tooling support at the time they started to become mainstream, in the form of Docker, Ansible, Puppet, Chef, and many more. Tooling around automation can be divided into two broader categories—continuous integration tools and continuous deployment tools.
Note
This article is a great source of information, and it includes ideas from different personalities on continuous integration, continuous deployment, and continuous delivery: https://bit.ly/2wyBLNW .
Continuous Integration
Continuous integration enables software development teams to work collaboratively, without stepping on each other's toes, by automating builds and source code integration to maintain source code integrity. It also integrates with DevOps tools to create an automated code delivery pipeline. Continuous integration helps development teams avoid integration hell where the software works on individual developers’ machines, but it fails when all developers integrate their code. Forrester, one of the top analyst firms, in its latest report14 on continuous integration tools, identified the top ten tools in the domain: Atlassian Bamboo, AWS CodeBuild, CircleCI, CloudBees Jenkins, Codeship, GitLab CI, IBM UrbanCode Build, JetBrains TeamCity, Microsoft VSTS, and Travis CI.
Continuous Delivery
The continuous delivery tools bundle applications, infrastructure, middleware, and their supporting installation processes and dependencies into release packages that transition across the lifecycle. The objective of this is to keep the code in a deployable state all the time. The latest Forrester report on continuous delivery and release automation highlighted 15 most significant vendors in the domain: Atlassian, CA Technologies, Chef Software, Clarive, CloudBees, Electric Cloud, Flexagon, Hewlett Packard Enterprise (HPE), IBM, Micro Focus, Microsoft, Puppet, Red Hat, VMware, and XebiaLabs.
However, if we look at the continuous delivery tools that are primarily supporting Kubernetes, the tools such as Weave Cloud15, Spinnaker16 (by Netflix), Codefresh17, Harness18, and GoCD19 are quite popular.
Continuous Deployment
Continuous deployment is a process that takes the artifacts produced by the continuous delivery process and deploys into a production setup, ideally every time a developer updates the code! Organizations that follow continuous deployment practices deploy code into production more than one hundred times a day. Blue-green, A/B testing, and canary releases are the three main approaches or practices people follow in continuous deployment.
Blue-Green Deployment
Blue-green deployment is a proven strategy for introducing changes into a running system. It’s been around for almost a decade now, and it’s successfully used by many large enterprises. Under the blue-green strategy we maintain two close-to-identical deployments; one is called blue and the other one is green. At a given time either blue or green takes the live traffic—let’s say blue. Now, we have the green environment to deploy our changes and test. If all works fine, we redirect the live traffic from blue to green at the load balancer. Then green becomes the environment that takes live traffic, while blue becomes available to deploy new changes. If there is an issue, we can quite easily switch the environments, and therefore can automatically roll back the new changes.
Canary Releases
The concept behind the canary comes from a tactic used by coal miners. They used to bring canaries into the mines with them to monitor the level of carbon monoxide in the air. If a canary dies, that means the level of carbon monoxide in the air is high, so they leave the coalmine. With the canary releases, a build is first made available to a selected set of the audience (maybe 5% to 10% of the entire live traffic), and if it works well (or doesn’t die), then it’s made available to everyone. This minimizes the risk of introducing new changes, as the production roll out happens slowly, in smaller chunks.
A/B Testing
A/B testing is about evaluating the impact of a new change against the old system or evaluating the impact of two different changes simultaneously. This is mostly used to track user behavior due to some changes introduced to a website. For example, you may have one version of your website having enabled social login as an option for signup, and another without social login. Another example would be having different colors or placements for important messages on a website and seeing which one is clicked more. A/B testing is used to measure different competing functionalities of an application with live traffic. After some time only the winner stays and the other competing features will be rolled back.
Summary
In this chapter, we discussed how to run microservices in a production deployment with containers. Containers and microservices are a match made in heaven and if not for containers, microservices wouldn’t be mainstream. We discussed deploying microservices with Docker and later discussed how the container orchestration works with Kubernetes. We also looked at Metaparticle, one of the prominent cloud native microservices frameworks, which helps us incorporate container-native capabilities into the applications or microservices code that we develop as annotations. Finally we discussed continuous integration/delivery and deployment.
In the next chapter, we discuss one of the trending topics in microservices architecture and also a key ingredient in any microservices deployment: the Service Mesh.