For the very first example, we'll run the easiest possible configuration of a Swarm v1 cluster locally to get a taste of how "old" Swarms worked (and still works). This tiny cluster will have the following features:
nodes://
mechanism will be usedOur cluster will look similar to the following diagram. Four Engines will be connected to each other through port 3376
in a mesh. Beyond the Docker engine, in fact, each of them will run a Docker container exposing port 3376
(Swarm) on host and redirecting it into itself. We, operators, will be able to connect to (any of) the hosts by setting the environment variable DOCKER_HOST
to IP:3376
. Everything will become clearer if you follow the example step-by-step.
To begin, we must create four Docker hosts with Docker Machine. Docker Machine automates these steps with one command instead of manually creating a Linux VM, generating and uploading certificates, logging into it via SSH, and installing and configuring the Docker daemon.
Machine will perform the following steps:
As a result, we'll have a VM running Docker and ready to be accessed to run containers.
Built with Tiny Core Linux, Boot2Docker is a lightweight distribution, which is especially designed for running Docker containers. It's runs completely on RAM and the boot time is extremely fast, around five seconds from start to the console. When starting the Engine, Boot2Docker starts Docker Engine at the secure port 2376 by default.
Boot2Docker is by no mean for the production workload. It's designed for development and testing purpose only. We'll start with using boot2docker then later move on to the production in subsequent chapters. At the time of writing, Boot2Docker supports Docker 1.12.3 and uses Linux Kernel 4.4. It comes with AUFS 4 as the default storage driver for Docker Engine.
If we execute:
$ docker-machine ls
on our new installation to list the available machines, we see that we have no running ones.
So, let's start by creating one, with this command:
$ docker-machine create --driver virtualbox node0
This command specifically asks to use the VirtualBox driver (-d for short) and to name the machine node0. Docker Machines can provision machines on dozens of different public and private providers, such as AWS, DigitalOcean, Azure, OpenStack, and has lots of options. For now, we go with the standard settings. The first cluster node will be ready after some time.
At this point, issue the following command to get a control of this host (so to remotely gain access):
$ docker-machine env node0
This will print some shell variables. Just copy the last row, the one with eval, paste it and issue enter. With those variables configured, you are not operating the local daemon anymore (if any), but the Docker daemon of node0
.
If you check the list of machines again, you will see a *
next to the image name, to indicate that it's the machine currently in use. Alternatively, you can type the following command to print the currently active machine:
$ docker-machine active
The daemon is running on this machine, with some standard settings (such as on port tcp/2376
enabled TLS). You can ensure that by SSHing to the node and verify the running processes:
$ docker-machine ssh node0 ps aux | grep docker 1320 root /usr/local/bin/docker daemon -D -g /var/lib/docker -H unix:// -H tcp://0.0.0.0:2376 --label provider=virtualbox -- tlsverify --tlscacert=/var/lib/boot2docker/ca.pem -- tlscert=/var/lib/boot2docker/server.pem -- tlskey=/var/lib/boot2docker/server-key.pem -s aufs
So, you can immediately this Docker daemon by, for example, starting containers and checking the Docker status:
Perfect! Now we provision the other three hosts, in the same exact way, by naming them node1
, node2
, and node3
:
$ docker-machine create --driver virtualbox node1 $ docker-machine create --driver virtualbox node2 $ docker-machine create --driver virtualbox node3
When they finish, you will have four Docker hosts available. Check with Docker machine.
We're now ready to start a Swarm cluster. But, before, for this very first example in order to keep it as simple as possible, we'll go with disabling TLS for running engines. Our plan is: Run the Docker daemon on port 2375
, without TLS.
Let's make a bit of order and explain all ports combinations in detail.
Insecure |
Secure |
Engine: 2375 |
Engine: 2376 |
Swarm: 3375 |
Swarm: 3376 |
Swarm v2 uses 2377 for node discovery among nodes |
Port 2377
is for Swarm v2 node to discover each other nodes in the cluster.
To understand where the TLS configuration is, we'll do some exercises by turning off the TLS of all our Docker hosts. Also turning it off here is intended to motivate the readers to learn how the swarm manage
command works by invoking it ourselves.
We have four hosts running Docker on port tcp/2376
and with TLS, as Docker Machine creates them by default. We must reconfigure them to change the daemon port to tls/2375
and remove TLS. So, we log in into each of them, with this command:
$ docker-machine ssh node0
Then, we gain root privileges:
$ sudo su -
And configure boot2docker
, by modifying the file /var/lib/boot2docker/profile
:
# cp /var/lib/boot2docker/profile /var/lib/boot2docker/profile-bak # vi /var/lib/boot2docker/profile
We delete the rows with CACERT, SERVERKEY, and SERVERCERT and configure the daemon port to tcp/2375
and DOCKER_TLS
to no
. In practice this will be our configuration:
After this log out from the SSH session and restart the machine:
$ docker-machine restart node0
Docker is now running on port tcp/2375
with no security. You can check this with the following command:
$ docker-machine ssh node0 ps aux | grep docker 1127 root /usr/local/bin/docker daemon -D -g /var/lib/docker -H unix:// -H tcp://0.0.0.0:2375 --label provider=virtualbox -s aufs
Finally, on your local desktop computer, unset DOCKER_TLS_VERIFY
and re-export DOCKER_HOST
in order to use the daemon listening on tcp/2375
with no TLS:
$ unset DOCKER_TLS_VERIFY $ export DOCKER_HOST="tcp://192.168.99.103:2375"
We must repeat these steps for each of our four nodes that will be part of our first Swarm.
To get started with Swarm v1 (no surprise), one must pull the swarm
image from the Docker hub. Open the four terminals, source the environment variables for each of your machines in each one in the first one, source node0 (docker-machine env node0
, and copy and paste the env
variable to the shell), in second node1
, and so on -, and after completing the steps for changing the standard port and disabling TLS described some lines above, on each of them do:
$ docker pull swarm
We'll use no discovery service for the first example, but the simplest of the mechanisms, such as the nodes://
. With nodes://
, the Swarm cluster nodes are connected manually, to form a grid of peers. What the operator has to do is simply define a list of nodes IPs and the daemon port, separated by commas, as shown:
nodes://192.168.99.101:2375,192.168.99.102:2375,192.168.99.103:2375,192.168.99.107:2375
To use Swarm, you simply run the swarm container with some arguments. To show the help online, you type:
$ docker run swarm --help
As you see, Swarm has basically four commands:
token://
For now, we'll use the manage
command. This is the command with most of the options (which you can investigate by issuing docker run swarm manage --help
). We limit now to connect nodes. The following is the strategy on each node:
daemon
(-d
) mode.tcp/3376
to the internal (on container) port tcp/2375
.nodes://
- each host has to be a pair IP:port
where the port is the Docker engine port (tcp/2375
).So, in each terminal you're connected to every machine, execute this:
$ docker run -d -p 3376:2375 swarm manage nodes://192.168.99.101:2375,192.168.99.102:2375, 192.168.99.103:2375,192.168.99.107:2375
Now, as the next step, we'll connect to it and inspect its information before starting using for running containers. For convenience, open a new terminal. We connect now not anymore to the Docker engine on one of our nodes, but to the Docker Swarm. So we will connect to tcp/3376
and not to tcp/2375
anymore. For the purpose of showing in detail what we're doing, let's start by sourcing node0
variables:
$ docker-machine env node0
Copy and paste the eval line, as you already know, and check what shell variables are exported with the following command:
$ export | grep DOCKER_
We now need to do the following:
DOCKER_HOST
to connect to Swarm port tcp/3376
instead of Engine tcp/2375
DOCKER_TLS_VERIFY
.DOCKER_CERT_PATH
.You should have a configuration similar to this:
If we now connect to the Docker swarm at 3376
, and show some info, we see that we're running Swarm:
Congratulations! You just started your first Docker cluster with Swarm. We can see that we still have no containers running on our cluster apart from the four swarms, but the Server Version is swarm/1.2.3, the scheduling strategy is spread, and, most importantly, we have four healthy nodes in our swarm (details of each Swarm node follow).
Also, you can get some extra information regarding the scheduler behavior of this Swarm cluster:
Strategy: spread Filters: health, port, containerslots, dependency, affinity, constraint
A spread scheduling strategy means that Swarm will attempt to place containers on the less utilized host and the listed filters are available when you create containers, thus allowing you to decide to manually suggest some options. For example, you might want to make your Galera cluster containers geographically near but on different hosts.
But, what's the size of this Swarm? You can see it at the very end of this output:
It means that on this tiny Swarm you have the total availability of these resources: four CPUs and 4GB of memory. That's just what we expected, by merging the computational resources of 4 VirtualBox hosts with a CPU and 1GB of memory each.