美文网首页
【云原生】第五章 k8s基础组件-部署Etcd集群

【云原生】第五章 k8s基础组件-部署Etcd集群

作者: 云之图 | 来源:发表于2022-12-12 20:16 被阅读0次

    Etcd 是一个分布式键值存储系统，Kubernetes使用Etcd进行数据存储，所以先准备一个Etcd数据库，为解决Etcd单点故障，应采用集群方式部署，这里使用3台组建集群，可容忍1台机器故障，当然，你也可以使用5台组建集群，可容忍2台机器故障。

    1、节点地址

    节点名称 节点地址 hostsname
    Etcd-1 172.21.209.32 k8s-master01
    Etcd-2 172.21.209.33 k8s-master02
    Etcd-3 172.21.209.34 k8s-master03

    2、etcd下载

    最新版本：https://github.com/etcd-io/etcd/releases/download/

    wget https://github.com/etcd-io/etcd/releases/download/v3.5.4/etcd-v3.5.4-linux-amd64.tar.gz

    3、分发二进制包到其他的etcd节点

    for  i in {32..34};do scp -P 22022 ./etcd-v3.5.4-linux-amd64.tar.gz 172.21.209.$i:/data/; done

说明：-P表示端口。


#或者将刚刚解压的二进制文件拷贝到其它服务器上
for i in k8s-master02 k8s-master03;do
    scp -r -P 22022 /usr/local/bin/kube* root@$i:/usr/local/bin/
    scp  -r -P 22022 /usr/local/bin/{etcd,etcdctl}   root@$i:/usr/local/bin/
done

    4、解压二进制包并完成安装

    tar -xf etcd-v3.5.4-linux-amd64.tar.gz --strip-components=1 -C /usr/local/bin etcd-v3.5.4-linux-amd64/etcd{,ctl}

查看版本，安装完成。
root@k8s-master02:/data# etcd --version
etcd Version: 3.5.4
Git SHA: 08407ff76
Go Version: go1.16.15
Go OS/Arch: linux/amd64
root@k8s-master02:/data#

    5、分发证书到其他节点上。

    1、创建证书存放目录并，将做好的证书拷贝到其他节点上
批量执行 #master的所有节点执行
for i in k8s-master02 k8s-master03;do
    ssh -p 22022  $i "mkdir  /etc/kubernetes/pki/ -p"
    scp -r -P 22022 /etc/kubernetes/pki $i:/etc/kubernetes/
done

查看节点02上的证书
root@k8s-master02:/etc/kubernetes/pki# ll
total 112
drwxr-xr-x 2 root root 4096 Dec 13 16:13 ./
drwxr-xr-x 3 root root 4096 Dec 13 16:13 ../
-rw-r--r-- 1 root root 1025 Dec 13 16:13 admin.csr
-rw------- 1 root root 1679 Dec 13 16:13 admin-key.pem
-rw-r--r-- 1 root root 1444 Dec 13 16:13 admin.pem
-rw-r--r-- 1 root root 1029 Dec 13 16:13 apiserver.csr
-rw------- 1 root root 1679 Dec 13 16:13 apiserver-key.pem
-rw-r--r-- 1 root root 1996 Dec 13 16:13 apiserver.pem
-rw-r--r-- 1 root root 1025 Dec 13 16:13 ca.csr
-rw------- 1 root root 1675 Dec 13 16:13 ca-key.pem
-rw-r--r-- 1 root root 1411 Dec 13 16:13 ca.pem
-rw-r--r-- 1 root root 1082 Dec 13 16:13 controller-manager.csr
-rw------- 1 root root 1679 Dec 13 16:13 controller-manager-key.pem
-rw-r--r-- 1 root root 1501 Dec 13 16:13 controller-manager.pem
-rw-r--r-- 1 root root  891 Dec 13 16:13 front-proxy-ca.csr
-rw------- 1 root root 1675 Dec 13 16:13 front-proxy-ca-key.pem
-rw-r--r-- 1 root root 1143 Dec 13 16:13 front-proxy-ca.pem
-rw-r--r-- 1 root root  903 Dec 13 16:13 front-proxy-client.csr
-rw------- 1 root root 1679 Dec 13 16:13 front-proxy-client-key.pem
-rw-r--r-- 1 root root 1188 Dec 13 16:13 front-proxy-client.pem
-rw-r--r-- 1 root root 1045 Dec 13 16:13 kube-proxy.csr
-rw------- 1 root root 1675 Dec 13 16:13 kube-proxy-key.pem
-rw-r--r-- 1 root root 1464 Dec 13 16:13 kube-proxy.pem
-rw------- 1 root root 1679 Dec 13 16:13 sa.key
-rw-r--r-- 1 root root  451 Dec 13 16:13 sa.pub
-rw-r--r-- 1 root root 1058 Dec 13 16:13 scheduler.csr
-rw------- 1 root root 1679 Dec 13 16:13 scheduler-key.pem
-rw-r--r-- 1 root root 1476 Dec 13 16:13 scheduler.pem
root@k8s-master02:/etc/kubernetes/pki#

    6、配置ETCD

    k8s-master01配置文件，请根据需求修改

    
# 如果要用IPv6那么把IPv4地址修改为IPv6即可
cat > /etc/etcd/etcd.config.yml << EOF 
name: 'k8s-master01'
data-dir: /var/lib/etcd
wal-dir: /var/lib/etcd/wal
snapshot-count: 5000
heartbeat-interval: 100
election-timeout: 1000
quota-backend-bytes: 0
listen-peer-urls: 'https://172.21.209.32:2380'
listen-client-urls: 'https://172.21.209.32:2379,http://127.0.0.1:2379'
max-snapshots: 3
max-wals: 5
cors:
initial-advertise-peer-urls: 'https://172.21.209.32:2380'
advertise-client-urls: 'https://172.21.209.32:2379'
discovery:
discovery-fallback: 'proxy'
discovery-proxy:
discovery-srv:
initial-cluster: 'k8s-master01=https://172.21.209.32:2380,k8s-master02=https://172.21.209.33:2380,k8s-master03=https://172.21.209.34:2380'
initial-cluster-token: 'etcd-k8s-cluster'
initial-cluster-state: 'new'
strict-reconfig-check: false
enable-v2: true
enable-pprof: true
proxy: 'off'
proxy-failure-wait: 5000
proxy-refresh-interval: 30000
proxy-dial-timeout: 1000
proxy-write-timeout: 5000
proxy-read-timeout: 0
client-transport-security:
  cert-file: '/etc/kubernetes/etcd/etcd.pem'
  key-file: '/etc/kubernetes/etcd/etcd-key.pem'
  client-cert-auth: true
  trusted-ca-file: '/etc/kubernetes/etcd/etcd-ca.pem'
  auto-tls: true
peer-transport-security:
  cert-file: '/etc/kubernetes/etcd/etcd.pem'
  key-file: '/etc/kubernetes/etcd/etcd-key.pem'
  peer-client-cert-auth: true
  trusted-ca-file: '/etc/kubernetes/etcd/etcd-ca.pem'
  auto-tls: true
debug: false
log-package-levels:
log-outputs: [default]
force-new-cluster: false
EOF

    k8s-master02配置文件，请根据需求修改

    
# 如果要用IPv6那么把IPv4地址修改为IPv6即可
cat > /etc/etcd/etcd.config.yml << EOF 
name: 'k8s-master02'
data-dir: /var/lib/etcd
wal-dir: /var/lib/etcd/wal
snapshot-count: 5000
heartbeat-interval: 100
election-timeout: 1000
quota-backend-bytes: 0
listen-peer-urls: 'https://172.21.209.33:2380'
listen-client-urls: 'https://172.21.209.33:2379,http://127.0.0.1:2379'
max-snapshots: 3
max-wals: 5
cors:
initial-advertise-peer-urls: 'https://172.21.209.33:2380'
advertise-client-urls: 'https://172.21.209.33:2379'
discovery:
discovery-fallback: 'proxy'
discovery-proxy:
discovery-srv:
initial-cluster: 'k8s-master01=https://172.21.209.32:2380,k8s-master02=https://172.21.209.33:2380,k8s-master03=https://172.21.209.34:2380'
initial-cluster-token: 'etcd-k8s-cluster'
initial-cluster-state: 'new'
strict-reconfig-check: false
enable-v2: true
enable-pprof: true
proxy: 'off'
proxy-failure-wait: 5000
proxy-refresh-interval: 30000
proxy-dial-timeout: 1000
proxy-write-timeout: 5000
proxy-read-timeout: 0
client-transport-security:
  cert-file: '/etc/kubernetes/etcd/etcd.pem'
  key-file: '/etc/kubernetes/etcd/etcd-key.pem'
  client-cert-auth: true
  trusted-ca-file: '/etc/kubernetes/etcd/etcd-ca.pem'
  auto-tls: true
peer-transport-security:
  cert-file: '/etc/kubernetes/etcd/etcd.pem'
  key-file: '/etc/kubernetes/etcd/etcd-key.pem'
  peer-client-cert-auth: true
  trusted-ca-file: '/etc/kubernetes/etcd/etcd-ca.pem'
  auto-tls: true
debug: false
log-package-levels:
log-outputs: [default]
force-new-cluster: false
EOF

    k8s-master03配置文件，请根据需求修改

    
# 如果要用IPv6那么把IPv4地址修改为IPv6即可
cat > /etc/etcd/etcd.config.yml << EOF 
name: 'k8s-master03'
data-dir: /var/lib/etcd
wal-dir: /var/lib/etcd/wal
snapshot-count: 5000
heartbeat-interval: 100
election-timeout: 1000
quota-backend-bytes: 0
listen-peer-urls: 'https://172.21.209.34:2380'
listen-client-urls: 'https://172.21.209.34:2379,http://127.0.0.1:2379'
max-snapshots: 3
max-wals: 5
cors:
initial-advertise-peer-urls: 'https://172.21.209.34:2380'
advertise-client-urls: 'https://172.21.209.34:2379'
discovery:
discovery-fallback: 'proxy'
discovery-proxy:
discovery-srv:
initial-cluster: 'k8s-master01=https://172.21.209.32:2380,k8s-master02=https://172.21.209.33:2380,k8s-master03=https://172.21.209.34:2380'
initial-cluster-token: 'etcd-k8s-cluster'
initial-cluster-state: 'new'
strict-reconfig-check: false
enable-v2: true
enable-pprof: true
proxy: 'off'
proxy-failure-wait: 5000
proxy-refresh-interval: 30000
proxy-dial-timeout: 1000
proxy-write-timeout: 5000
proxy-read-timeout: 0
client-transport-security:
  cert-file: '/etc/kubernetes/etcd/etcd.pem'
  key-file: '/etc/kubernetes/etcd/etcd-key.pem'
  client-cert-auth: true
  trusted-ca-file: '/etc/kubernetes/etcd/etcd-ca.pem'
  auto-tls: true
peer-transport-security:
  cert-file: '/etc/kubernetes/etcd/etcd.pem'
  key-file: '/etc/kubernetes/etcd/etcd-key.pem'
  peer-client-cert-auth: true
  trusted-ca-file: '/etc/kubernetes/etcd/etcd-ca.pem'
  auto-tls: true
debug: false
log-package-levels:
log-outputs: [default]
force-new-cluster: false
EOF

    7、创建etcd启动服务（需要在所有master节点操作）

    cat > /usr/lib/systemd/system/etcd.service << EOF
[Unit]
Description=Etcd Service
Documentation=https://coreos.com/etcd/docs/latest/
After=network.target
[Service]
Type=notify
ExecStart=/usr/local/bin/etcd --config-file=/etc/etcd/etcd.config.yml
Restart=on-failure
RestartSec=10
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
Alias=etcd3.service
EOF

    8、启动服务

    systemctl daemon-reload
systemctl enable --now etcd

    9、查看etcd状态

    
# 如果要用IPv6那么把IPv4地址修改为IPv6即可
root@k8s-master02:~# export ETCDCTL_API=3
root@k8s-master02:~# etcdctl --endpoints="k8s-master01:2379,k8s-master02:2379,k8s-master03:2379" --cacert=/etc/kubernetes/etcd/etcd-ca.pem --cert=/etc/kubernetes/etcd/etcd.pem --key=/etc/kubernetes/etcd/etcd-key.pem  endpoint status --write-out=table
+-------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|     ENDPOINT      |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+-------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| k8s-master01:2379 | 8d64b715c092a95e |   3.5.4 |   20 kB |     false |      false |         3 |         18 |                 18 |        |
| k8s-master02:2379 | 360eef617d4fed2c |   3.5.4 |   20 kB |      true |      false |         3 |         18 |                 18 |        |
| k8s-master03:2379 | 344830ef5ebe5c27 |   3.5.4 |   20 kB |     false |      false |         3 |         18 |                 18 |        |
+-------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
root@k8s-master02:~#

    10、etcd的常用操作

    #查看 etcd 集群成员列表
root@k8s-master02:~#  etcdctl member list
344830ef5ebe5c27, started, k8s-master03, https://172.21.209.34:2380, https://172.21.209.34:2379, false
360eef617d4fed2c, started, k8s-master02, https://172.21.209.33:2380, https://172.21.209.33:2379, false
8d64b715c092a95e, started, k8s-master01, https://172.21.209.32:2380, https://172.21.209.32:2379, false
root@k8s-master02:~# 



# 删除 etcd 集群成员 k8s-master-2-11
 etcdctl member remove 344830ef5ebe5c27

    问题：

    问题：查看etcd的状态时，只有一个在线，并且其他的节点的服务也是正常的。无法获取集群状态，但是通过地址就可以，经过排除发现证书有问题，只对k8s-master01做了授权，没有对k8s-master02和k8s-master03授权。通过IP地址可以正常获取状态。

    root@k8s-master01:~# export ETCDCTL_API=3
root@k8s-master01:~# etcdctl --endpoints="k8s-master01:2379,k8s-master02:2379,k8s-master03:2379" --cacert=/etc/kubernetes/etcd/etcd-ca.pem --cert=/etc/kubernetes/etcd/etcd.pem --key=/etc/kubernetes/etcd/etcd-key.pem  endpoint status --write-out=table

{"level":"warn","ts":"2022-12-13T19:57:47.382+0800","logger":"etcd-client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc000444380/k8s-master01:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = \"transport: authentication handshake failed: x509: certificate is valid for k8s-master01, k8s-master01, k8s-master01, not k8s-master02\""}
Failed to get the status of endpoint k8s-master02:2379 (context deadline exceeded)
{"level":"warn","ts":"2022-12-13T19:57:52.382+0800","logger":"etcd-client","caller":"v3/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"etcd-endpoints://0xc000444380/k8s-master01:2379","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = \"transport: authentication handshake failed: x509: certificate is valid for k8s-master01, k8s-master01, k8s-master01, not k8s-master03\""}
Failed to get the status of endpoint k8s-master03:2379 (context deadline exceeded)
+-------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|     ENDPOINT      |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+-------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| k8s-master01:2379 | 8d64b715c092a95e |   3.5.4 |   20 kB |     false |      false |         2 |         14 |                 14 |        |
+-------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
root@k8s-master01:~# 
root@k8s-master01:~# 


通过主机名无法获取，但是通过ip可以获取。
etcdctl --endpoints="k8s-master02:2379" --cacert=/etc/kubernetes/etcd/etcd-ca.pem --cert=/etc/kubernetes/etcd/etcd.pem --key=/etc/kubernetes/etcd/etcd-key.pem  endpoint status --write-out=table



root@k8s-master03:/etc/etcd# etcdctl --endpoints="172.21.209.34:2379" --cacert=/etc/kubernetes/etcd/etcd-ca.pem --cert=/etc/kubernetes/etcd/etcd.pem --key=/etc/kubernetes/etcd/etcd-key.pem  endpoint status --write-out=table
+--------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|      ENDPOINT      |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+--------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| 172.21.209.34:2379 | 344830ef5ebe5c27 |   3.5.4 |   20 kB |      true |      false |         2 |         14 |                 14 |        |
+--------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
root@k8s-master03:/etc/etcd# 
root@k8s-master03:/etc/etcd# 
root@k8s-master03:/etc/etcd# 
root@k8s-master03:/etc/etcd# etcdctl --endpoints="172.21.209.33:2379" --cacert=/etc/kubernetes/etcd/etcd-ca.pem --cert=/etc/kubernetes/etcd/etcd.pem --key=/etc/kubernetes/etcd/etcd-key.pem  endpoint status --write-out=table
+--------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
|      ENDPOINT      |        ID        | VERSION | DB SIZE | IS LEADER | IS LEARNER | RAFT TERM | RAFT INDEX | RAFT APPLIED INDEX | ERRORS |
+--------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
| 172.21.209.33:2379 | 360eef617d4fed2c |   3.5.4 |   20 kB |     false |      false |         2 |         14 |                 14 |        |
+--------------------+------------------+---------+---------+-----------+------------+-----------+------------+--------------------+--------+
root@k8s-master03:/etc/etcd#

    至此：ETCD集群部署完成。

    相关文章

      网友评论

          本文标题：【云原生】第五章 k8s基础组件-部署Etcd集群

          本文链接：https://www.haomeiwen.com/subject/coafqdtx.html

          延伸阅读

          深度阅读

          • 您也可以注册成为美文阅读网的作者，发表您的原创作品、分享您的心情！

          栏目导航

          热点阅读

          关于我们|服务条款|联系我们|【云原生】第五章 k8s基础组件-部署Etcd集群|投稿指南|网站地图|RSS订阅|排版工具|手机版

          提供经典美文摘抄,优美散文欣赏,现代诗歌精选,短篇小说,心情随笔,表白情书范文,故事会在线阅读欣赏

          Copyright © 2014-2023 Haomeiwen.com All Rights Reserved. 好美文阅读网 版权所有

          备案信息：桂公网安备 45052102000051号 · 桂ICP备13007215号-3

          本站所收录作品、热点评论等信息部分来源互联网，目的只是为了系统归纳学习和传递资讯

          所有作品版权归原创作者所有，与本站立场无关，如不慎侵犯了你的权益，请联系我们告知，我们将做删除处理！