centos下安装etcd集群

etcd介绍

etcd是CoreOS团队于2013年6月发起的开源项目，它的目标是构建一个高可用的分布式键值(key-value)数据库。etcd内部采用raft协议作为一致性算法，etcd基于Go语言实现。

etcd的有点有以下：

• 简单：安装配置简单，而且提供了HTTP API进行交互，使用也很简单
• 安全：支持SSL证书验证
• 快速：根据官方提供的benchmark数据，单实例支持每秒2k+读操作
• 可靠：采用raft算法，实现分布式系统数据的可用性和一致性

etcd 项目详见：https://github.com/coreos/etcd/

安装之前

一般不建议root安装业务中间件服务，故这里需要先进行环境初始化

主机IP	etcd name
192.168.2.213	etcd1
192.168.2.214	etcd2
192.168.2.215	etcd3

新增系统用户etcd

groupadd --system etcd
useradd -s /sbin/nologin -m /var/lib/etcd --system -g etcd etcd

新增配置文件路径

mkdir /etc/etcd
chown -R etcd:etcd /etc/etcd

修改 SELinux 为 disabled

setenforce 0

sed -i 's/^SELINUX=.*/SELINUX=disabled/g' /etc/selinux/config

安装

下载二进制安装包

curl -s https://api.github.com/repos/etcd-io/etcd/releases/latest | grep browser_download_url | grep linux-amd64 |cut -d '"' -f 4 | wget -qi -

或者百度网盘下载

链接: https://pan.baidu.com/s/1xOeIYWTAs0VMcLoCuT6BTg 提取码: 5kmf

如下进入安装

tar -zxvf etcd-v3.2.32-linux-amd64.tar.gz
cd etcd-v3.2.32-linux-amd64
cp etcd etcdctl /usr/local/bin
chown -R etcd:etcd /usr/local/bin/etcd*

验证安装

root@pts/1 $ etcd --version
etcd Version: 3.2.32
Git SHA: 7dc07f2a9
Go Version: go1.12.17
Go OS/Arch: linux/amd64

root@pts/1 $ etcdctl --version
etcdctl version: 3.2.32
API version: 2

配置和启动

1、三台主机分别配置/etc/hosts如下

# etcd
192.168.2.213 etc1
192.168.2.214 etc2
192.168.2.215 etc3

2、三台主机配置新增配置文件/etc/etcd/etcd.conf 这里注意 etcd1 配置中 ETCD_INITIAL_CLUSTER_STATE 为 new 其余两个为exist
另外参数ETCD_LISTEN_CLIENT_URLS、ETCD_ADVERTISE_CLIENT_URLS、ETCD_LISTEN_PEER_URLS 、ETCD_INITIAL_ADVERTISE_PEER_URLS 分别为当前主机的IP，ETCD_NAME 根据开始的定义配置

# member
ETCD_NAME=etcd1
ETCD_DATA_DIR=/var/lib/etcd
ETCD_LISTEN_CLIENT_URLS=http://192.168.2.213:2379,http://127.0.0.1:2379
ETCD_ADVERTISE_CLIENT_URLS=http://192.168.2.213:2379

# cluster
ETCD_LISTEN_PEER_URLS=http://192.168.2.213:2380
ETCD_INITIAL_ADVERTISE_PEER_URLS=http://192.168.2.213:2380
ETCD_INITIAL_CLUSTER=etcd1=http://192.168.2.213:2380,etcd2=http://192.168.2.214:2380,etcd3=http://192.168.2.215:2380
ETCD_INITIAL_CLUSTER_STATE=new
ETCD_INITIAL_CLUSTER_TOKEN=k8s_etcd

3、三台主机配置systemctl启动方式

/usr/lib/systemd/system/etcd.service

[Unit]
Description=etcd key-value store
Documentation=https://github.com/etcd-io/etcd
After=network.target

[Service]
User=etcd
Type=notify
WorkingDirectory=/var/lib/etcd/
EnvironmentFile=/etc/etcd/etcd.conf
ExecStart=/usr/local/bin/etcd
Restart=on-failure
RestartSec=10s
LimitNOFILE=40000

[Install]
WantedBy=multi-user.target

启动服务

systemctl enable etcd.service
systemctl start etcd.service
systemctl status etcd.service

测试和验收

1、检查各节点

root@pts/1 $ etcdctl member list
59b0ee3829e2b866: name=etcd2 peerURLs=http://192.168.2.214:2380 clientURLs=http://192.168.2.214:2379 isLeader=false
a8b07bac1693e30e: name=etcd1 peerURLs=http://192.168.2.213:2380 clientURLs=http://192.168.2.213:2379 isLeader=false
cf682ab5655702b8: name=etcd3 peerURLs=http://192.168.2.215:2380 clientURLs=http://192.168.2.215:2379 isLeader=true

2、检查集群状态

root@pts/1 $ etcdctl cluster-health
member 59b0ee3829e2b866 is healthy: got healthy result from http://192.168.2.214:2379
member a8b07bac1693e30e is healthy: got healthy result from http://192.168.2.213:2379
member cf682ab5655702b8 is healthy: got healthy result from http://192.168.2.215:2379
cluster is healthy

3、进行测试

root@pts/1 $ etcdctl set /demo 'hello etcd'
hello etcd

root@pts/1 $ etcdctl get /demo
hello etcd

故障演练

停掉Leader节点

root@pts/1 $ etcdctl member list
59b0ee3829e2b866: name=etcd2 peerURLs=http://192.168.2.214:2380 clientURLs=http://192.168.2.214:2379 isLeader=false
a8b07bac1693e30e: name=etcd1 peerURLs=http://192.168.2.213:2380 clientURLs=http://192.168.2.213:2379 isLeader=true
cf682ab5655702b8: name=etcd3 peerURLs=http://192.168.2.215:2380 clientURLs=http://192.168.2.215:2379 isLeader=false

看到Leader从etcd3转移到etcd1

查看集群状态

root@pts/1 $ etcdctl cluster-health
member 59b0ee3829e2b866 is healthy: got healthy result from http://192.168.2.214:2379
member a8b07bac1693e30e is healthy: got healthy result from http://192.168.2.213:2379
failed to check the health of member cf682ab5655702b8 on http://192.168.2.215:2379: Get http://192.168.2.215:2379/health: dial tcp 192.168.2.215:2379: connect: connection refused
member cf682ab5655702b8 is unreachable: [http://192.168.2.215:2379] are all unreachable
cluster is healthy

看到etcd3连接失败，这个时候在执行如下命令进行测试，发现请求正常

root@pts/1 $ etcdctl get /demo
hello etcd

然后启动故障节点之后检查，集群状态恢复

root@pts/1 $ etcdctl cluster-health
member 59b0ee3829e2b866 is healthy: got healthy result from http://192.168.2.214:2379
member a8b07bac1693e30e is healthy: got healthy result from http://192.168.2.213:2379
member cf682ab5655702b8 is healthy: got healthy result from http://192.168.2.215:2379
cluster is healthy