etcd集群采用raft算法选举Leader， 最小raft集群需要3个参与者，所以一个etcd集群最少需要3台虚拟机.
1. 准备
1.1 准备环境
准备三台CentOS7.3主机，主机名分别为etcd1, etcd2, etcd3
etcd1: 10.62.232.41
etcd2: 10.62.232.42
etcd3: 10.62.232.43
1.2 环境配置
在每台主机上运行以下命令:
cat >> /etc/hosts <<EOF
# etcd hosts
10.62.232.41 etcd1
10.62.232.42 etcd2
10.62.232.43 etcd3
EOF
如果需要启动防火墙，请确保打开官方给出的etcd端口, 官方端口号查询
这里简单粗暴的禁用防火墙
systemctl stop firewalld
systemctl disable firewalld
禁用Selinux
setenforce 0
vi /etc/selinux/config
SELINUX=disabled
2. TLS 密钥和证书
使用TLS证书对通信进行加密，并开启即与CA根证书签名的双向认证
2.1 建立目录
每台主机都要确保以下目录存在
- 证书目录:
/etc/ssl/etcd/ssl/
- 数据目录:
/var/lib/etcd/
mkdir -p /etc/ssl/etcd/ssl/
mkdir -p /var/lib/etcd/
2.2 创建CA证书和私钥
有很多方式可以创建CA证书和私钥，其中比较流行的有两种
- openssl
- cfssl
这里使用openssl来生成私钥ca.key和证书ca.crt
登录etcd1主机， 运行以下命令
mkdir -p /xiak/ssl
cd /xiak/ssl
openssl genrsa -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -subj "/CN=xiak" -days 10000 -out ca.crt
2.3 创建etcd证书和私钥
接下来我们通过之前生成的CA证书和私钥生成etcd的证书和私钥，整个过程需要5步：
- 创建配置文件etcd-ca.conf
cd /xiak/ssl
cat > /xiak/ssl/etcd-ca.conf <<EOF
[ req ]
default_bits = 2048
prompt = no
default_md = sha256
req_extensions = req_ext
distinguished_name = dn
[ dn ]
C = CN
ST = Sichuan
L = Chengdu
O = etcd
OU = xiak
CN = etcd
[ req_ext ]
subjectAltName = @alt_names
[ alt_names ]
DNS.1 = localhost
DNS.2 = etcd1
DNS.3 = etcd2
DNS.4 = etcd3
IP.1 = 127.0.0.1
IP.2 = 10.62.232.41
IP.3 = 10.62.232.42
IP.4 = 10.62.232.43
[ v3_ext ]
authorityKeyIdentifier=keyid,issuer:always
basicConstraints=CA:FALSE
keyUsage=keyEncipherment,dataEncipherment
extendedKeyUsage=serverAuth,clientAuth
subjectAltName=@alt_names
EOF
- 生成密钥
openssl genrsa -out etcd.key 2048
- 生成证书签发请求(certificate signing request)
openssl req -new -key etcd.key -out etcd.csr -config etcd-ca.conf
- 生成证书
openssl x509 -req -in etcd.csr -CA ca.crt -CAkey ca.key \
-CAcreateserial -out etcd.crt -days 10000 \
-extensions v3_ext -extfile etcd-ca.conf
- 验证证书
openssl verify -CAfile ca.crt etcd.crt
3 安装etcd
3.1 安装证书
将CA证书ca.crt, etcd证书etcd.crt和秘钥etcd.key, 拷贝到各节点的/etc/ssl/etcd/ssl/目录中。
3.2 下载etcd二进制包
在每个节点上下载etcd二进制包
mkdir -p /xiak/pkg
cd /xiak/pkg
wget https://github.com/coreos/etcd/releases/download/v3.3.8/etcd-v3.3.8-linux-amd64.tar.gz
3.3 安装etcd二进制包
解压缩etcd-v3.3.5-linux-amd64.tar.gz，将其中的etcd和etcdctl两个可执行文件复制到各节点的/usr/bin目录
cd /xiak/pkg
tar -zxvf etcd-v3.3.8-linux-amd64.tar.gz
cd ./etcd-v3.3.8-linux-amd64/
cp {etcd,etcdctl} /usr/local/bin
3.4 以systemd形式管理etcd服务
在每个节点上创建etcd的systemd unit文件/usr/lib/systemd/system/etcd.service
在etcd1主机上执行以下命令集
```bash
mkdir -p /etc/etcd/
cat > /etc/etcd/etcd.conf <<EOF
# [Member Flags]
# ETCD_ELECTION_TIMEOUT=1000
# ETCD_HEARTBEAT_INTERVAL=100
# 指定etcd的数据目录
ETCD_NAME=etcd1
ETCD_DATA_DIR=/var/lib/etcd/
# [Cluster Flags]
# ETCD_AUTO_COMPACTION_RETENTIO:N=0
ETCD_INITIAL_CLUSTER_STATE=new
ETCD_ADVERTISE_CLIENT_URLS=https://10.62.232.41:2379
ETCD_INITIAL_ADVERTISE_PEER_URLS=https://10.62.232.41:2380
ETCD_LISTEN_CLIENT_URLS=https://10.62.232.41:2379,https://127.0.0.1:2379
ETCD_INITIAL_CLUSTER_TOKEN=etcd-cluster
ETCD_LISTEN_PEER_URLS=https://10.62.232.41:2380
ETCD_INITIAL_CLUSTER=etcd1=https://10.62.232.41:2380,etcd2=https://10.62.232.42:2380,etcd3=https://10.62.232.43:2380
# [Proxy Flags]
ETCD_PROXY=off
# [Security flags]
# ETCD_CLIENT_CERT_AUTH=
# ETCD_PEER_CLIENT_CERT_AUTH=
# 指定etcd的公钥证书和私钥
ETCD_TRUSTED_CA_FILE=/etc/ssl/etcd/ssl/ca.crt
ETCD_CERT_FILE=/etc/ssl/etcd/ssl/etcd.crt
ETCD_KEY_FILE=/etc/ssl/etcd/ssl/etcd.key
# 指定etcd的Peers通信的公钥证书和私钥
ETCD_PEER_TRUSTED_CA_FILE=/etc/ssl/etcd/ssl/ca.crt
ETCD_PEER_CERT_FILE=/etc/ssl/etcd/ssl/etcd.crt
ETCD_PEER_KEY_FILE=/etc/ssl/etcd/ssl/etcd.key
# [Profiling flags]
# ETCD_METRICS={{ etcd_metrics }}
EOF
cat > /usr/lib/systemd/system/etcd.service <<EOF
[Unit]
Description=etcd server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=-/etc/etcd/etcd.conf
ExecStart=/usr/local/bin/etcd
NotifyAccess=all
Restart=always
RestartSec=5s
LimitNOFILE=40000
[Install]
WantedBy=multi-user.target
EOF
详细配置见etcd configuration
etcd2和etcd3主机需要修改
ETCD_NAME=etcd(2或3)
ETCD_ADVERTISE_CLIENT_URLS=https://10.62.232.(42或43):2379
ETCD_INITIAL_ADVERTISE_PEER_URLS=https://10.62.232.(42或43):2380
ETCD_LISTEN_CLIENT_URLS=https://10.62.232.(42或43):2379,https://127.0.0.1:2379
ETCD_LISTEN_PEER_URLS=https://10.62.232.(42或43):2380
3.5 启动etcd:
在各个节点运行
systemctl daemon-reload
systemctl enable etcd
systemctl start etcd
systemctl status etcd
只有一个节点启动etcd服务后，etcd会去访问其他两个节点的2380端口，如果一定时间内无法访问，则会服务启动失败。所以应当迅速启动至少两个节点，服务才会成功启动。
3.6 检查集群是否健康:
在任意节点运行
etcdctl \
--ca-file=/etc/ssl/etcd/ssl/ca.crt \
--cert-file=/etc/ssl/etcd/ssl/etcd.crt \
--key-file=/etc/ssl/etcd/ssl/etcd.key \
--endpoints=https://10.62.232.41:2379,https://10.62.232.42:2379,https://10.62.232.43:2379 \
cluster-health
看到cluster is healthy的输出，说明etcd集群部署成功。
网友评论