配置
kube-controller
启用server证书颁发配置完成后，启用证书的轮转，证书过期的话，可以自动的续签。在每个master节点上添加一条配置：- --feature-gates=RotateKubeletServerCertificate=true
vim /etc/kubernetes/manifests/kube-controller-manager.yaml
……
spec:
containers:
- command:
- kube-controller-manager
- --feature-gates=RotateKubeletServerCertificate=true
- --cluster-signing-duration=87600h0m0s
……
修改可kube-controller-manager.yaml后需要重启kube-controller pod
mv /etc/kubernetes/manifests/kube-controller-manager.yaml \
/etc/kubernetes/manifests/kube-controller-manager.yaml.bk && \
sleep 30 && mv /etc/kubernetes/manifests/kube-controller-manager.yaml.bk \
/etc/kubernetes/manifests/kube-controller-manager.yaml
kubelet
检查每个节点的kubelet配置是否开启了kubelet 证书轮转rotateCertificates（默认都是开启的）
vim /var/lib/kubelet/config.yaml
rotateCertificates: true
如果修改了kubelet配置，重启重启kubelet服务
systemctl restart kubelet
检查证书过期时间
由 kubeadm 生成的客户端证书有效期为1年，所以我们需要在证书过过期之前对集群证书进行更新，在操作之前一定要先对证书目录进行备份，防止操作错误进行回滚。
首先，使用kubeadm certs check-expiration命令检查集群中的证书过期时间。
~ # kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED
admin.conf Jul 16, 2023 09:55 UTC 90d ca no
apiserver Jul 16, 2023 09:54 UTC 90d ca no
apiserver-etcd-client Jul 16, 2023 09:54 UTC 90d etcd-ca no
apiserver-kubelet-client Jul 16, 2023 09:54 UTC 90d ca no
controller-manager.conf Jul 16, 2023 09:55 UTC 90d ca no
etcd-healthcheck-client Jul 16, 2023 09:53 UTC 90d etcd-ca no
etcd-peer Jul 16, 2023 09:53 UTC 90d etcd-ca no
etcd-server Jul 16, 2023 09:53 UTC 90d etcd-ca no
front-proxy-client Jul 16, 2023 09:54 UTC 90d front-proxy-ca no
scheduler.conf Jul 16, 2023 09:55 UTC 90d ca no
CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED
ca Dec 28, 2030 09:14 UTC 7y no
etcd-ca Dec 28, 2030 09:14 UTC 7y no
front-proxy-ca Dec 28, 2030 09:14 UTC 7y no
有些低版本的k8s使用的是
kubeadm alpha certs check-expiratio
备份旧证书和配置文件等
在升级证书之前，需要备份旧证书和密钥以免更新证书的时候出错，kubeadm生成的证书一般在/etc/kubernetes/pki下
# 创建备份目录
/home # mkdir /etc/kubernetes.bak
# 备份旧证书
/home # cp -r /etc/kubernetes/pki/ /etc/kubernetes.bak
# 备份配置文件
/home # cp /etc/kubernetes/*.conf /etc/kubernetes.bak
# 备份etcd数据
/home # cp -r /var/lib/etcd /var/lib/etcd.bak
执行证书升级命令
/home # kubeadm certs renew all
[renew] Reading configuration from the cluster...
[renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healthcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed
Done renewing certificates. You must restart the kube-apiserver, kube-controller-manager, kube-scheduler and etcd, so that they can use the new certificates.
上面的列表中没有包含 kubelet.conf，因为 kubeadm 将 kubelet 配置为自动更新证书。 轮换的证书位于目录 /var/lib/kubelet/pki。 要修复过期的 kubelet 客户端证书，请参阅 kubelet 客户端证书轮换失败。
再次检查证书过期时间
/home # kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
CERTIFICATE EXPIRES RESIDUAL TIME CERTIFICATE AUTHORITY EXTERNALLY MANAGED
admin.conf Apr 16, 2024 08:14 UTC 364d ca no
apiserver Apr 16, 2024 08:14 UTC 364d ca no
apiserver-etcd-client Apr 16, 2024 08:14 UTC 364d etcd-ca no
apiserver-kubelet-client Apr 16, 2024 08:14 UTC 364d ca no
controller-manager.conf Apr 16, 2024 08:14 UTC 364d ca no
etcd-healthcheck-client Apr 16, 2024 08:14 UTC 364d etcd-ca no
etcd-peer Apr 16, 2024 08:14 UTC 364d etcd-ca no
etcd-server Apr 16, 2024 08:14 UTC 364d etcd-ca no
front-proxy-client Apr 16, 2024 08:14 UTC 364d front-proxy-ca no
scheduler.conf Apr 16, 2024 08:14 UTC 364d ca no
CERTIFICATE AUTHORITY EXPIRES RESIDUAL TIME EXTERNALLY MANAGED
ca Dec 28, 2030 09:14 UTC 7y no
etcd-ca Dec 28, 2030 09:14 UTC 7y no
front-proxy-ca Dec 28, 2030 09:14 UTC 7y no
证书过期时间已更新
重启组件
更新完证书后需要重启kube-apiserver, kube-controller-manager, kube-scheduler和etcd组件
mkdir -p /etc/kubernetes.bak/manifests/
mv /etc/kubernetes/manifests/kube-scheduler.yaml /etc/kubernetes.bak/manifests/
mv /etc/kubernetes/manifests/kube-controller-manager.yaml /etc/kubernetes.bak/manifests/
mv /etc/kubernetes/manifests/kube-apiserver.yaml /etc/kubernetes.bak/manifests/
mv /etc/kubernetes/manifests/etcd.yaml /etc/kubernetes.bak/manifests/
sleep 30
mv /etc/kubernetes.bak/manifests/kube-scheduler.yaml /etc/kubernetes/manifests/
mv /etc/kubernetes.bak/manifests/kube-controller-manager.yaml /etc/kubernetes/manifests/
mv /etc/kubernetes.bak/manifests/kube-apiserver.yaml /etc/kubernetes/manifests/
mv /etc/kubernetes.bak/manifests/etcd.yaml /etc/kubernetes/manifests/
docker runtime 可使用
docker ps |egrep "k8s_kube-apiserver|k8s_kube-scheduler|k8s_kube-controller|k8s_etcd"|awk '{print $1}'|xargs docker restart进行重启
更新 ~/.kube/config 文件
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
网友评论