Etcd is a highly available, distributed, and consistent key-value store that is used for shared configuration and service discovery. Some notable projects that use Etcd are SwarmKit, Kubernetes, and Fleet.
Etcd can gracefully manage master elections in case of network splits and can tolerate node failure, including the master. Applications, in our case Docker containers and Swarm nodes, can read and write data into Etcd's key-value storage, for example the location of services.
We once again create an example with one manager and three nodes, this time by illustrating Etcd.
This time, we'll need a real discovery service. We can simulate a non-HA system by running the Etcd server inside Docker itself. We create a cluster made of four hosts, with the following names:
etcd-mwill be the Swarm master and will host also the Etcd serveretcd-1: The first Swarm nodeetcd-2: The second Swarm nodeetcd-3: The third Swarm node
The operator, by connecting to etcd-m:3376, will operate Swarm on the three nodes, as usual.
Let's start by creating the hosts with Machine:
for i in m `seq 1 3`; do docker-machine create -d virtualbox etcd-$i;
done
Now we will run the Etcd master on etcd-m. We use the quay.io/coreos/etcd official image from CoreOS, following the documentation available at https://github.com/coreos/etcd/blob/master/Documentation/op-guide/clustering.md.
First, in a terminal, we source the etcd-m shell variables:
term0$ eval $(docker-machine env etcd-m)
Then, we run the Etcd master in a single-host mode (that is, no fault-tolerance, and so on):
docker run -d -p 2379:2379 -p 2380:2380 -p 4001:4001 --name etcd quay.io/coreos/etcd -name etcd-m -initial-advertise-peer-urls http://$(docker-machine ip etcd-m):2380 -listen-peer-urls http://0.0.0.0:2380 -listen-client-urls http://0.0.0.0:2379,http://0.0.0.0:4001 -advertise-client-urls http://$(docker-machine ip etcd-m):2379 -initial-cluster-token etcd-cluster-1 -initial-cluster etcd-m=http://$(docker-machine ip etcd-m):2380 -initial-cluster-state new
What we do here is start the Etcd image in the daemon (-d) mode and expose ports 2379 (Etcd client communication), 2380 (Etcd server communication), 4001 (), and specify the following Etcd options:
name: The name of the node, in this case we select etcd-m, as the name of the node hosting this containerinitial-advertise-peer-urlsin this static configuration is the address:port of the clusterlisten-peer-urlslisten-client-urlsadvertise-client-urlsinitial-cluster-tokeninitial-clusterinitial-cluster-state
We can ensure that this one-node Etcd cluster is healthy, using the etcdctl cluster-health command-line utility:
term0$ docker run fsoppelsa/etcdctl -C $(dm ip etcd-m):2379
cluster-health
This indicates that Etcd is at least up and running, so we can use it to set up a Swarm v1 cluster.
We create the Swarm manager on the same etcd-m host:
term0$ docker run -d -p 3376:3376 swarm manage -H tcp://0.0.0.0:3376 ` etcd://$(docker-machine ip etcd-m)/swarm
This exposes the usual 3376 port from host to container, but this time starts the manager using the etcd:// URL for the discovery service.
We now join the nodes, etcd-1, etcd-2, and etcd-3.
As usual, we can source and command machines, one, for each terminal:
term1$ eval $(docker-machine env etcd-1) term1$ docker run -d swarm join --advertise $(docker-machine ip etcd-1):2379 etcd://$(docker-machine ip etcd-m):2379 term2$ eval $(docker-machine env etcd-2) term1$ docker run -d swarm join --advertise $(docker-machine ip etcd-2):2379 etcd://$(docker-machine ip etcd-m):2379 term3$ eval $(docker-machine env etcd-3) term3$ docker run -d swarm join --advertise $(docker-machine ip etcd-3):2379 etcd://$(docker-machine ip etcd-m):2379
With join -advertise, we order the local node to join the Swarm cluster, using the Etcd service running and exposed on etcd-m.
We now go to etcd-m and see the nodes of our cluster, by invoking the Etcd discovery service:
We have the three hosts already joined to the cluster as expected.