K8s 证书过期更新

作者: Sven_qi | 来源:发表于2021-12-15 14:55 被阅读0次

k8s1.20.6用kubeadm安装的3master+3nod
K8s 证书过期更新
解决 k8s 环境证书过期的问题
K8S证书过期问题
k8s证书过期
Kubeadm部署集群证书过期更换
Kubernetes集群证书过期解决办法
k8sv1.15证书过期处理
K8s You must be logged in to the
k8s集群证书过期

在项目的系统中，使用了 k8s 集群容器化服务的方式承载 Java项目系统。

K8S 集群使用的机器情况：

n1628 （master）
n1542
n1509

使用的 k8s 环境：

v1.15.3

在使用的过程中，由于证书过期导致 kube-apiserver 无法通信，服务故障

发现问题
2021年7月22日发现比对系统异常，任务没有点击，无排重数据，登录到 n1628 查看容器情况，

发现 kube-apiserver 异常退出，查看日志信息

docker logs  -t --since="2021-07-20T13:23:37" --until "2021-07-23T12:23:37" 2d25ad94ef32

观察到错误信息

2021-07-22T03:18:27.477804611Z I0722 03:18:27.477643 1 controller.go:107] OpenAPI AggregationController: Processing item
2021-07-22T03:18:27.477852624Z I0722 03:18:27.477691 1 controller.go:130] OpenAPI AggregationController: action for item : Nothing (removed from the queue).
2021-07-22T03:18:27.477878308Z I0722 03:18:27.477705 1 controller.go:130] OpenAPI AggregationController: action for item k8s_internal_local_delegation_chain_0000000000: Nothing (removed from the queue).
2021-07-22T03:18:27.487305589Z I0722 03:18:27.487152 1 storage_scheduling.go:128] all system priority classes are created successfully or already exist.
2021-07-22T03:18:28.120293228Z E0722 03:18:28.120118 1 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
2021-07-22T03:18:28.126428343Z E0722 03:18:28.126310 1 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
2021-07-22T03:18:28.130438641Z E0722 03:18:28.130330 1 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
2021-07-22T03:18:28.133040542Z E0722 03:18:28.132961 1 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
2021-07-22T03:18:28.135957952Z E0722 03:18:28.135862 1 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
2021-07-22T03:18:30.234536126Z E0722 03:18:30.234254 1 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
2021-07-22T03:18:30.238039464Z E0722 03:18:30.237922 1 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
2021-07-22T03:18:30.240711863Z E0722 03:18:30.240616 1 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
2021-07-22T03:18:30.243274286Z E0722 03:18:30.243179 1 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid
2021-07-22T03:18:30.245872244Z E0722 03:18:30.245775 1 authentication.go:65] Unable to authenticate the request due to an error: x509: certificate has expired or is not yet valid

定位到证书过期，进行验证

for item in `find /etc/kubernetes/pki -maxdepth 2 -name "*.crt"`;
do openssl x509 -in $item -text -noout| grep Not;
echo ======================$item===============;
done

信息看出，2021年7月22日 03 年证书到期

Not Before: Sep 19 06:27:23 2019 GMT
Not After : Jul 22 03:16:23 2021 GMT
======================/etc/kubernetes/pki/front-proxy-client.crt===============
Not Before: Sep 19 06:27:24 2019 GMT
Not After : Jul 22 03:16:21 2021 GMT
======================/etc/kubernetes/pki/apiserver-etcd-client.crt===============
Not Before: Sep 19 06:27:25 2019 GMT
Not After : Sep 16 06:27:25 2029 GMT
======================/etc/kubernetes/pki/ca.crt===============
Not Before: Sep 19 06:27:25 2019 GMT
Not After : Jul 22 03:16:21 2021 GMT
======================/etc/kubernetes/pki/apiserver.crt===============
Not Before: Sep 19 06:27:25 2019 GMT
Not After : Jul 22 03:16:21 2021 GMT
======================/etc/kubernetes/pki/apiserver-kubelet-client.crt===============
Not Before: Sep 19 06:27:23 2019 GMT
Not After : Sep 16 06:27:23 2029 GMT
======================/etc/kubernetes/pki/front-proxy-ca.crt===============
Not Before: Sep 19 06:27:24 2019 GMT
Not After : Jul 22 03:16:22 2021 GMT
======================/etc/kubernetes/pki/etcd/server.crt===============
Not Before: Sep 19 06:27:24 2019 GMT
Not After : Sep 16 06:27:24 2029 GMT
======================/etc/kubernetes/pki/etcd/ca.crt===============
Not Before: Sep 19 06:27:24 2019 GMT
Not After : Jul 22 03:16:22 2021 GMT
======================/etc/kubernetes/pki/etcd/healthcheck-client.crt===============
Not Before: Sep 19 06:27:24 2019 GMT
Not After : Jul 22 03:16:22 2021 GMT
======================/etc/kubernetes/pki/etcd/peer.crt===============

更新证书

更新证书操作

kubeadm alpha certs renew all --config=/root/kubeadm.conf

在 master 上执行重启 kube-apiserver, kube-controller, kube-scheduler, etcd 容器，使证书生效

docker ps -a | grep -v pause | grep -E "etcd|scheduler|controller|apiserver" | awk '{print $1}' | awk '{print "docker","restart",$1}' | bash

查看证书过期时间

for item in `find /etc/kubernetes/pki -maxdepth 2 -name "*.crt"`;do openssl x509 -in $item -text -noout| grep Not;echo ======================$item===============;done

2022年7月22日过期

==========;done
Not Before: Sep 19 06:27:23 2019 GMT
Not After : Jul 22 03:16:23 2022 GMT
======================/etc/kubernetes/pki/front-proxy-client.crt===============
Not Before: Sep 19 06:27:24 2019 GMT
Not After : Jul 22 03:16:21 2022 GMT
======================/etc/kubernetes/pki/apiserver-etcd-client.crt===============
Not Before: Sep 19 06:27:25 2019 GMT
Not After : Sep 16 06:27:25 2029 GMT
======================/etc/kubernetes/pki/ca.crt===============
Not Before: Sep 19 06:27:25 2019 GMT
Not After : Jul 22 03:16:21 2022 GMT
======================/etc/kubernetes/pki/apiserver.crt===============
Not Before: Sep 19 06:27:25 2019 GMT
Not After : Jul 22 03:16:21 2022 GMT
======================/etc/kubernetes/pki/apiserver-kubelet-client.crt===============
Not Before: Sep 19 06:27:23 2019 GMT
Not After : Sep 16 06:27:23 2029 GMT
======================/etc/kubernetes/pki/front-proxy-ca.crt===============
Not Before: Sep 19 06:27:24 2019 GMT
Not After : Jul 22 03:16:22 2022 GMT
======================/etc/kubernetes/pki/etcd/server.crt===============
Not Before: Sep 19 06:27:24 2019 GMT
Not After : Sep 16 06:27:24 2029 GMT
======================/etc/kubernetes/pki/etcd/ca.crt===============
Not Before: Sep 19 06:27:24 2019 GMT
Not After : Jul 22 03:16:22 2022 GMT
======================/etc/kubernetes/pki/etcd/healthcheck-client.crt===============
Not Before: Sep 19 06:27:24 2019 GMT
Not After : Jul 22 03:16:22 2022 GMT
======================/etc/kubernetes/pki/etcd/peer.crt===============