Swarm mode

Docker Swarm mode (for Docker Engines of version 1.12 or newer) imports the SwarmKit libraries in order to make distributed container orchestration over multiple hosts possible and easy to operate.

The main difference between SwarmKit and Swarm Mode is that Swarm Mode is integrated into Docker itself, starting from version 1.12. This means that Swarm Mode commands such as swarm, nodes, service, and task are available inside the Docker client, and that through the docker command it's possible to initiate and manage Swarms, as well as deploy services and tasks:

  • docker swarm init: This is to initialize a Swarm cluster
  • docker node ls: This is used to list the available nodes
  • docker service tasks: This is used to list the tasks associated to a specific service

Old versus new Swarm versus SwarmKit

At the time of writing, (August 2016), we have three Docker orchestration systems: the old one (that is) Swarm v1, SwarmKit, and the new one (that is) integrated Swarm Mode.

Old versus new Swarm versus SwarmKit

The original Swarm v1, the one we showed in Chapter 1, Welcome to Docker Swarm and that's still used around, is not yet deprecated. It's a way of using (recycling?) older infrastructures. But starting from Docker 1.12, the new Swarm Mode is the recommended way to begin a new orchestration project, especially if it will need to scale to a big size.

To make things simpler, let's summarize the differences between these projects with some tables.

First, the old Swarm v1 versus the new Swarm Mode:

Swarm standalone

Swarm Mode

This is available since Docker 1.8

This is available since Docker 1.12

This is available as a container

This is integrated into the Docker Engine

This needs an external discovery service (such as Consul, Etcd, or Zookeeper)

This doesn't need an external discovery service, Etcd integrated

This is not secure by default

This is secure by default

The replica and scaling features are not available

The replica and scaling features are available

There are no service and task concepts for modeling microservices

There are out of the box services, tasks, load balancing, and service discovery

There is no additional networking available

This has integrated VxLAN (mesh networking)

And now, to clarify ideas, let's compare SwarmKit and Swarm mode:

SwarmKit

Swarm mode

These are released as binaries (swarmd and swarmctl)--use swarmctl

These are integrated into the Docker Engine--use docker

These are generic tasks

These are container tasks

These include services and tasks

These include services and tasks

These include no service advanced features, such as load balancing and VxLAN networking

These include out of the box service advanced features, such as load balancing and VxLAN networking

Swarm Mode zoom in

As we already summarized in the preceding table in Swarm standalone versus Swarm mode comparison, the main new features available in Swarm Mode are the integration into the engine, no need for an external discovery service, and replica, scale, load balancing, and networking included.

Integration into the engine

With docker 1.12+, some new commands are added to the docker client. We now take a survey on the ones that are relevant to the matter of this book.

docker swarm command

This is the current command to manage Swarms:

docker swarm command

It accepts the following options:

  • init: This initializes a Swarm. Behind the curtain, this command creates a manager for the current Docker host and generates a secret (its password that workers will pass to the API so as to be authorized to join the cluster).
  • join: This is used by a worker to join a cluster, must specify the secret and a list of managers IP port values.
  • join-token: This is used to manage the join-tokens. join-tokens are special token secrets used to make join managers or workers (managers and workers have different token values). This command is a convenient way to make Swarm print the necessary command to join a manager or a worker:
            docker swarm join-token worker
    

    To add a worker to this swarm, run the following command:

            docker swarm join  --token SWMTKN-1-  
            36gj6glgi3ub2i28ekm1b1er8aa51vltv00760t7umh3wmo1sc- 
            aucj6a94tqhhn2k0iipnc6096  192.168.65.2:2377
            docker swarm join-token manager
    

    To add a manager to this swarm, run the following command:

            docker swarm join  --token SWMTKN-1- 
            36gj6glgi3ub2i28ekm1b1er8aa51vltv00760t7umh3wmo1sc- 
            98glton0ot50j1yn8eci48rvq  192.168.65.2:2377
    
  • update: This updates the cluster by changing some of its values, for example, you can use it to specify a new URL of the certificate endpoint
  • leave: This commands the current node to leave the cluster. If something is blocking the operation, there is a useful --force option.

docker node

This is the command to handle swarm nodes. You must launch it from a manager, so you need to be connected to a manager in order to use it.

docker node
  • demote and promote: These are commands used to manage the status of nodes. With that mechanism, you can promote a node to a manager, or demote it to a worker. In practice, Swarm will try to demote/promote. We will cover this concept just a bit later in this chapter.
  • inspect: This is the equivalent of docker info, but for a Swarm node. It prints information regarding the node/s.
  • ls: This lists the nodes connected to the cluster.
  • rm: This attempts to remove a worker. If you want to remove a manager, you have before to demote it to worker.
  • ps: This shows the list of tasks running on a specified node.
  • update: This allows you to change some configuration values for a node, namely tags.

docker service

This is the command to manage the services running on a Swarm cluster:

docker service

Apart from the expected commands such as create, inspect, ps, ls, rm, and update, there is a new interesting one: scale.

Docker Stack

Not directly necessary to Swarm operations, but introduced as experimental in Docker 1.12, there is the stack command. Stacks are now bundles of containers. For example, a nginx + php + mysql container setup can be stacked in a self-contained Docker Stack, called Distributed Application Bundle (DAB) and described by a JSON file.

The core command of docker stack will be deploy, thanks to which it will be possible to create and update DABs. We'll meet stacks later in Chapter 6, Deploy Real Applications on Swarm.

Etcd's Raft is integrated already

Docker Swarm Mode already integrates RAFT through CoreOS Etcd Raft library. There is no further need to integrate external discovery services such as Zookeeper or Consul anymore. Swarm directly takes care of essential services such as DNS and load balancing.

Installing a Swarm Mode cluster is just a matter of starting Docker hosts and running Docker commands, making it super easy to set up.

Load balancing and DNS

By design, the cluster managers assign to every service in the swarm a unique DNS name and load balances running containers, using an internal DNS to Docker. Queries and resolutions work automatically out of the box.

For each service created with a --name myservice, every container in the swarm will be able to resolve the service IP address just as they were resolving (dig myservice) internal network names, using the Docker embedded DNS server. So, if you have a nginx-service (made of nginx containers, for example), you can just ping nginx-service to reach the frontend head.

Also, in Swarm mode, operators have the possibility to publish services ports to an external load balancer. Ports are then exposed outside to a port in a range from 30000 to 32767. Internally, Swarm uses iptables and IPVS to execute packet filtering and forwarding, and load balancing respectively.

Iptables is the default packet filter firewall used by Linux while IPVS is the seasoned IP Virtual Server defined in the Linux kernel, that can be used for load balancing traffic, and that's just what Docker Swarm uses.

Ports are published either when a new service is created or updated, using the --publish-add option. With this option, an internal service is published, and gets load balanced.

For example, if we have a cluster with three workers each running nginx (on a service named nginx-service), we can expose their target-port to the load balancer with:

docker service update --port-add 80 nginx-service

This will create a mapping between the published port 30000 on any of the nodes cluster, and the nginx containers (port 80). If you connect any node to port 30000, you will be greeted by the Nginx welcome page.

Load balancing and DNS

But how does this work? As you see in the preceding screenshot, there is an associated VirtualIP (10.255.0.7/16), or VIP, and it is collocated on the overlay network 2xbr2upsr3yl, created by Swarm for ingress to the load balancer:

Load balancing and DNS

From any host, you are able to reach nginx-service, because the DNS name resolves to the VIP, here 10.255.0.7, acting as a frontend to the load balancer:

On each node of the swarm, Swarm implements load balancing in kernel, specifically inside the namespaces, by adding a MARK rule in the OUTPUT chain inside the network namespace dedicated to the network, as shown in the following screenshot:

Load balancing and DNS

We'll cover networking concepts in greater detail later, in Chapter 5, Administer a Swarm Cluster and Chapter 8, Exploring Additional features of Swarm.

Promotion and demotion

With the docker node command, the cluster operator can promote nodes from workers to managers and, vice versa, demote them from managers to workers.

Demoting a node from manager to worker is the only way to remove a manager (now worker) from the cluster.

We'll cover promotion and demotion operations in detail in Chapter 5, Administer a Swarm Cluster.

Replicas and scale

Deploying an app on a Swarm cluster means to define and configure services, start them and wait for Docker engines scattered across the cluster to launch containers. We'll deploy complete apps on Swarm in Chapter 6, Deploy Real Applications on Swarm.

Services and tasks

The core of the Swarm workload is divided into services. A service is just an abstraction to group an arbitrary number of tasks (this number is called the replica factor, or just replicas). Tasks are running containers.

docker service scale

With the docker service scale command, you order Swarm to ensure that a certain number of replicas are running at the same time on the cluster. For example, you can start with 10 containers running some task distributed over the cluster, and then when you need to scale their size to 30 you just execute:

    docker service scale myservice=30

Swarm is ordered to schedule 20 new containers, so it takes the appropriate decisions for load balancing, DNS, and networking coherence. If a container for task goes down, making the replica factor equal to 29, Swarm will reschedule another one on another cluster node (that will have a new ID) to maintain the factor equal to 30.

A note on replicas and new nodes addition. People frequently ask about Swarm automatic capabilities. If you have five workers running 30 tasks, and add five new nodes, you should not expect Swarm to balance the 30 tasks across the new nodes automatically, moving them from the original to the new nodes. The behavior of the Swarm scheduler is conservative, until some event (for example, an operator intervention) triggers a new scale command. Only in that case, the scheduler will take into account the five new nodes and possibly start new replica tasks on the 5 new workers.

We'll see how scale command works in practice in Chapter 7, Scaling Up Your Platform.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset