etcd

« back  main post  next »

Why?

etcd is a [KV][^1] store, and we need it for use with Flannel to provide a Software Defined Network layer to avoid relying on Scaleway's IP ranges and potential for collisions there.

On top of that, you can use etcd can be used by your application if you need a distributed kv store for your application configuration.

The following walkthrough should work with any version after v3.2.17 as well unless a v4 comes around. We won't be using the package manager provided etcd as this is quite old (at the time of writing, v2.2.5).

The plan

We will be using the latest version of etcd. Currently, that is v3.2.17. You can find that on the Github repo.

Doing it

So to start off, download the latest release to all the nodes:

export ETCD_VERSION=v3.2.17
wget https://github.com/coreos/etcd/releases/download/$ETCD_VERSION/etcd-$ETCD_VERSION-linux-amd64.tar.gz
wget https://github.com/coreos/etcd/releases/download/$ETCD_VERSION/etcd-$ETCD_VERSION-linux-amd64.tar.gz.asc
wget https://coreos.com/dist/pubkeys/app-signing-pubkey.gpg
# Import the CoreOS signing key
gpg --import app-signing-pubkey.gpg
# Verify the download
gpg --verify etcd-$ETCD_VERSION-linux-amd64.tar.gz.asc

The latter 2 commands should look something along the lines of:

[email protected]:~# gpg --import app-signing-pubkey.gpg  
gpg: /root/.gnupg/trustdb.gpg: trustdb created      
gpg: key FC8A365E: public key "CoreOS Application Signing Key <[email protected]>" imported            
gpg: Total number processed: 1                      
gpg:               imported: 1  (RSA: 1)            
[email protected]:~# gpg --verify etcd-$ETCD_VERSION-linux-amd64.tar.gz.asc                                                                                                                                                    
gpg: assuming signed data in `etcd-v3.2.17-linux-amd64.tar.gz'                                           
gpg: Signature made Thu 08 Mar 2018 09:56:19 PM UTC using RSA key ID 7EF48FD3
gpg: Good signature from "CoreOS Application Signing Key <[email protected]>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: 18AD 5014 C99E F7E3 BA5F  6CE9 50BD D3E0 FC8A 365E
     Subkey fingerprint: B261 4119 157B E592 32DF  D2AA F804 F413 7EF4 8FD3

Now that we have downloaded etcd and verified that it's a valid file, extraction time and add etcd's installation directory to our path

mkdir /opt/etcd
tar xzvf etcd-$ETCD_VERSION-linux-amd64.tar.gz -C /opt/etcd --strip-components=1

echo 'export PATH=$PATH:/opt/etcd' >> ~/.bashrc
. ~/.bashrc

We should now be able to verify that we got etcdctl available in our path:

[email protected]:~# etcdctl --version
etcdctl version: 3.2.17
API version: 2

We are now ready to bootstrap the etcd cluster;

etcd --name swarm1 --initial-advertise-peer-urls http://10.0.0.1:2380 \
  --listen-peer-urls http://10.0.0.1:2380 \
  --listen-client-urls http://10.0.0.1:2379,http://127.0.0.1:2379 \
  --advertise-client-urls http://10.0.0.1:2379 \
  --initial-cluster-token etcd-cluster-1 \
  --initial-cluster swarm1=http://10.0.0.1:2380,swarm2=http://10.0.0.2:2380,swarm3=http://10.0.0.3:2380 \
  --initial-cluster-state new \
  --data-dir /opt/etcd/data

Swap out the advertise-peer-urls, listen-peer-urls, listen-client-urls, advertise-client-urls for each node, and list all nodes in the initial-cluster parameter.

Then on each node, create /etc/systemd/system/etcd.service with the following contents:

[Unit]
Description=etcd
Documentation=https://github.com/coreos/etcd

[Service]
Type=notify
Restart=always
RestartSec=5s
LimitNOFILE=40000
TimeoutStartSec=0

ExecStart=/opt/etcd/etcd --name swarm1 \
    --initial-advertise-peer-urls http://10.0.0.1:2380 \
    --listen-peer-urls http://10.0.0.1:2380 \
    --listen-client-urls http://10.0.0.1:2379,http://127.0.0.1:2379 \
    --advertise-client-urls http://10.0.0.1:2379 \
    --data-dir /opt/etcd/data

[Install]
WantedBy=multi-user.target

The final step is to set up the environment so that you can use etcdctl without having to specify any parameters to point it in the right direction.

In /etc/environment add the following two lines:

ETCDCTL_ENDPOINTS=http://10.0.0.1:2379,http://10.0.0.2:2379,http://10.0.0.3:2379
ETCDCTL_API=3

then log out and back in again.

Just to verify that the cluster is functional and healthy, we can now run etcdctl endpoint health and etcdctl endpoint status.
The first command should list three hosts as healthy, and the second should list three hosts, one with true in the 5th place, this is the master in the cluster.

« back  main post  next »

[^1] Key Value database