美文网首页
kubeadm集群更改证书

kubeadm集群更改证书

作者: H_appiness | 来源:发表于2019-04-19 21:35 被阅读0次

    Note: 本文基于使用kubeadm搭建的kubernetes集群进行讲解

    第一步：获取证书签发信息

    • 方式一：通过原有证书进行获取相关信息

      • 获取apiserver签发信息

        $ openssl x509 -noout -text -in /etc/kubernetes/pki/apiserver.crt
......
X509v3 extensions:
    X509v3 Key Usage: critical
        Digital Signature, Key Encipherment
    X509v3 Extended Key Usage:
        TLS Web Server Authentication
    X509v3 Subject Alternative Name:
        DNS:localhost, DNS:node1, DNS:node2, DNS:node3, DNS:kubernetes, DNS:kubernetes.default, DNS:kubernetes.default.svc,   DNS:kubernetes.default.svc.cluster.local, IP Address:127.0.0.1, IP Address:10.96.0.1, IP Address:192.168.12.10, IP   Address:192.168.12.11, IP Address:192.168.12.12, IP Address:192.168.12.13
......

      • 从上面输出信息可知签发的DNS和IP详情

        DNS.1 = localhost
# 各 master 节点 nodeName
DNS.2 = node1
DNS.3 = node2
DNS.4 = node3
# kubenertes svc 地址
DNS.5 = kubernetes
DNS.6 = kubernetes.default
DNS.7 = kubernetes.default.svc
# 带 dns domain 的 kubenertes svc 地址
DNS.8 = kubernetes.default.svc.cluster.local
IP.6 = 127.0.0.1
# kubenertes svc 的 clusterip
IP.5 = 10.96.0.1
# api-server VIP 地址
IP.4 = 192.168.12.10
# 各 master 节点 IP
IP.1 = 192.168.12.11
IP.2 = 192.168.12.12
IP.3 = 192.168.12.13

    • 方式二：自行统计各项信息

      • 在创建证书之前需要获取到以下信息，在签发证书的时候会用到它：

        • 各master节点IP
        • 各master节点nodeName
        • 如果有设置apiserver负载均衡则需要VIP，否则请忽略
        • kubernetes集群dns domain
        • kubenertes.default.svc的clusterip

      • 本文以下面集群信息为例：

        • 节点信息：

          nodeName ip relo
          slb 192.168.12.10 slb
          node1 192.168.12.11 master
          node2 192.168.12.12 master
          node3 192.168.12.13 master
          node4 192.168.12.14 worker
          node5 192.168.12.15 worker

        • 获取kubernetes dns domain：

          # CoreDNS 查看方法
$ kubectl get cm -n kube-system coredns -o yaml | grep kubernetes
kubernetes cluster.local in-addr.arpa ip6.arpa {
# 由以上输出可知dns domain为 cluster.local

# kube-dns 查看方法
$ kubectl get deployment -n kube-system kube-dns -o yaml | grep domain
- --domain=cluster.local.
# 由以上输出可知dns domain为 cluster.local

        • kubernetes apiserver clusterip

          $ kubectl get svc kubernetes -n default
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1      <none>        443/TCP   280d
# 由结果可知apiserver clusterip

    同理获取etcd集群证书签发的DNS和IP详情，默认etcd证书路径为/etc/kubernetes/pki/etcd

    第二步：创建证书

    Note: 我们只需要在一个节点上进行证书生成，生成的证书分发到其他节点即可

    • 创建CA服务端证书签名请求配置文件openssl.conf，内容如下，注意替换alt_names_cluster、alt_names_etcd域中的值

      [ req ]
default_bits = 2048
default_md = sha256
distinguished_name = req_distinguished_name

[req_distinguished_name]

[ v3_ca ]
basicConstraints = critical, CA:TRUE
keyUsage = critical, digitalSignature, keyEncipherment, keyCertSign

[ v3_req_server ]
basicConstraints = CA:FALSE
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth

[ v3_req_client ]
basicConstraints = CA:FALSE
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth

[ v3_req_apiserver ]
basicConstraints = CA:FALSE
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names_cluster

[ v3_req_etcd ]
basicConstraints = CA:FALSE
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth, clientAuth
subjectAltName = @alt_names_etcd

[ alt_names_cluster ]
DNS.1 = localhost
DNS.2 = node1
DNS.3 = node2
DNS.4 = node3
DNS.5 = kubernetes
DNS.6 = kubernetes.default
DNS.7 = kubernetes.default.svc
DNS.8 = kubernetes.default.svc.cluster.local
IP.1 = 127.0.0.1
IP.2 = 10.96.0.1
IP.3 = 192.168.12.10
IP.4 = 192.168.12.11
IP.5 = 192.168.12.12
IP.6 = 192.168.12.13

[ alt_names_etcd ]
DNS.1 = localhost
DNS.2 = node1
DNS.3 = node2
DNS.4 = node3
IP.1 = 127.0.0.1
IP.2 = 0:0:0:0:0:0:0:1
IP.3 = 192.168.12.11
IP.4 = 192.168.12.12
IP.5 = 192.168.12.13

    • 创建集群 key 与 CA

      • 将要创建的 CA

        路径 Common Name 描述
        ca.crt,key kubernetes Kubernetes general CA
        etcd/ca.crt,key kubernetes For all etcd-related functions
        front-proxy-ca.crt,key kubernetes For the front-end proxy

      • 要注意 CA 中 CN(Common Name) 与 O(Organization) 等内容是会影响Kubernetes组件认证的

        • CA (Certificate Authority) 是自签名的根证书，用来签名后续创建的其它证书
        • CN (Common Name), apiserver 会从证书中提取该字段作为请求的用户名 (User Name)
        • O (Organization), apiserver 会从证书中提取该字段作为请求用户所属的组 (Group)

      一般CA根证书有效期为10年，若举例过期时间还长，可跳过本节操作，命令中的3650为3650天，即证书有效期。

      • kubernetes-ca

        openssl genrsa -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key \
  -subj "/CN=kubernetes" -config openssl.conf \
  -extensions v3_ca -out ca.crt -days 3560

      • etcd-ca

        mkdir -p etcd
openssl genrsa -out etcd/ca.key 2048
openssl req -x509 -new -nodes -key etcd/ca.key \
  -subj "/CN=kubernetes" -config openssl.conf \
  -extensions v3_ca -out etcd/ca.crt -days 3560

      • front-proxy-ca

        openssl genrsa -out front-proxy-ca.key 2048
openssl req -x509 -new -nodes -key front-proxy-ca.key \
  -subj "/CN=kubernetes" -config openssl.conf \
  -extensions v3_ca -out front-proxy-ca.crt -days 3560

    • 创建 Certificates

      • 将要创建的 Certificates
        Name Key Certificates Common Name Organization
        etcd/server etcd/server.key etcd/server.crt master
        etcd/peer etcd/peer.key etcd/peer.crt master
        etcd/healthcheck-client etcd/healthcheck-client.key etcd/healthcheck-client.crt kube-etcd-healthcheck-client system:masters
        apiserver-etcd-client apiserver-etcd-client.key apiserver-etcd-client.crt kube-apiserver-etcd-client system:masters
        apiserver apiserver.key apiserver.crt kube-apiserver
        apiserver-kubelet-client apiserver-kubelet-client.key apiserver-kubelet-client.crt kube-apiserver-kubelet-client system:masters
        front-proxy-client front-proxy-client.key front-proxy-client.crt front-proxy-client
        kube-scheduler kube-scheduler.key kube-scheduler.crt system:kube-scheduler
        sa(kube-controller-manager) sa.key(sa.pub) kube-controller-manager.crt system:kube-controller-manager
        admin(kubectl) admin.key admin.crt kubernetes-admin system:masters
        kubelet kubelet.key kubelet.crt system:node:master system:nodes

    mkdir etcd
    cp /etc/kubernetes/pki/etcd/*.key etcd/
    cp /etc/kubernetes/pki/etcd/ca.crt etcd/

    • etcd/server

      ~~openssl genrsa -out etcd/server.key 2048~~
openssl req -new -key etcd/server.key \
  -subj "/CN=master" -out etcd/server.csr
openssl x509 -in etcd/server.csr -req -CA etcd/ca.crt \
  -CAkey etcd/ca.key -CAcreateserial -extensions v3_req_etcd \
  -extfile openssl.conf -out etcd/server.crt -days 3560

    • etcd/peer

      ~~openssl genrsa -out etcd/peer.key 2048~~
openssl req -new -key etcd/peer.key \
  -subj "/CN=master" -out etcd/peer.csr
openssl x509 -in etcd/peer.csr -req -CA etcd/ca.crt \
  -CAkey etcd/ca.key -CAcreateserial -extensions v3_req_etcd \
  -extfile openssl.conf -out etcd/peer.crt -days 3560

    • etcd/healthcheck-client

      ~~openssl genrsa -out etcd/healthcheck-client.key 2048~~
openssl req -new -key etcd/healthcheck-client.key \
  -subj "/CN=kube-etcd-healthcheck-client/O=system:masters" \
  -out etcd/healthcheck-client.csr
openssl x509 -in etcd/healthcheck-client.csr -req -CA etcd/ca.crt \
  -CAkey etcd/ca.key -CAcreateserial -extensions v3_req_etcd \
  -extfile openssl.conf -out etcd/healthcheck-client.crt -days 3560

    cp /etc/kubernetes/pki/*.key .
    cp /etc/kubernetes/pki/ca.crt .
    cp /etc/kubernetes/pki/front-proxy-ca.crt .

    • apiserver-etcd-client

      ~~openssl genrsa -out apiserver-etcd-client.key 2048~~
openssl req -new -key apiserver-etcd-client.key \
  -subj "/CN=kube-apiserver-etcd-client/O=system:masters" \
  -out apiserver-etcd-client.csr
openssl x509 -in apiserver-etcd-client.csr -req -CA etcd/ca.crt \
  -CAkey etcd/ca.key -CAcreateserial -extensions v3_req_etcd \
  -extfile openssl.conf -out apiserver-etcd-client.crt -days 3560

    • apiserver

      ~~openssl genrsa -out apiserver.key 2048~~
openssl req -new -key apiserver.key \
  -subj "/CN=kube-apiserver" -config openssl.conf \
  -out apiserver.csr
openssl x509 -req -in apiserver.csr -CA ca.crt -CAkey ca.key \
  -CAcreateserial -extensions v3_req_apiserver \
  -extfile openssl.conf -out apiserver.crt -days 3560

    • apiserver-kubelet-client

      ~~openssl genrsa -out apiserver-kubelet-client.key 2048~~
openssl req -new -key apiserver-kubelet-client.key \
  -subj "/CN=kube-apiserver-kubelet-client/O=system:masters" \
  -out apiserver-kubelet-client.csr
openssl x509 -req -in apiserver-kubelet-client.csr -CA ca.crt -CAkey ca.key \
  -CAcreateserial -extensions v3_req_client \
  -extfile openssl.conf -out apiserver-kubelet-client.crt -days 3560

    • front-proxy-client

      ~~openssl genrsa -out front-proxy-client.key 2048~~
openssl req -new -key front-proxy-client.key \
  -subj "/CN=front-proxy-client" \
  -out front-proxy-client.csr
openssl x509 -req -in front-proxy-client.csr -CA front-proxy-ca.crt -CAkey front-proxy-ca.key \
  -CAcreateserial -extensions v3_req_client \
  -extfile openssl.conf -out front-proxy-client.crt -days 3560

    • kube-scheduler

      openssl genrsa -out kube-scheduler.key 2048
openssl req -new -key kube-scheduler.key \
  -subj "/CN=system:kube-scheduler" \
  -out kube-scheduler.csr
openssl x509 -req -in kube-scheduler.csr -CA ca.crt -CAkey ca.key \
  -CAcreateserial -extensions v3_req_client \
  -extfile openssl.conf -out kube-scheduler.crt -days 3560

    cp /etc/kubernetes/pki/sa.pub .

    • sa(kube-controller-manager)

      ~~openssl genrsa -out sa.key 2048~~
~~openssl rsa -in sa.key -pubout -out sa.pub~~
openssl req -new -key sa.key \
  -subj "/CN=system:kube-controller-manager" \
  -out kube-controller-manager.csr
openssl x509 -req -in kube-controller-manager.csr -CA ca.crt -CAkey ca.key \
  -CAcreateserial -extensions v3_req_client \
  -extfile openssl.conf -out kube-controller-manager.crt -days 3560

    • admin(kubectl)

      openssl genrsa -out admin.key 2048
openssl req -new -key admin.key \
  -subj "/CN=kubernetes-admin/O=system:masters" \
  -out admin.csr
openssl x509 -req -in admin.csr -CA ca.crt -CAkey ca.key \
  -CAcreateserial -extensions v3_req_client \
  -extfile openssl.conf -out admin.crt -days 3560

    • kubelet

      openssl genrsa -out kubelet.key 2048
# 此处为 master 节点 nodeName，每个 master 生成对应的证书
openssl req -new -key kubelet.key \
  -subj "/CN=system:node:node1/O=system:nodes" \
  -out kubelet.csr
openssl x509 -req -CA ca.crt -CAkey ca.key \
  -CAcreateserial -extensions v3_req_client \
  -extfile openssl.conf -days 3560 -in kubelet.csr -out kubelet.crt

    第三步：生成kubernetes各组件配置文件并应用

    • 所要生成的配置文件列表

      配置文件名称 组件证书文件名称 组件秘钥文件名称 根证书文件名称
      admin.conf(kubectl) admin.crt admin.key ca.crt
      kubelet.conf kubelet.crt kubelet.key ca.crt
      scheduler.conf kube-scheduler.crt kube-scheduler.key ca.crt
      controller-manager.conf kube-controller-manager.crt sa.key ca.crt
      • 操作前请先备份原有配置文件
      • 除了kubelet.conf文件需注意配置为对应节点的nodeName，其余配置文件可通用
      • 以下操作请先在一台 master 节点上操作确认没有问题后再进行配置其他节点
      • –certificate-authority：指定根证书
      • –client-certificate、–client-key：指定组件证书及秘钥
      • –embed-certs=true：将组件证书内容嵌入到生成的配置文件中(不加时，写入的是证书文件路径)

    • admin.conf(kubectl)

      KUBE_APISERVER="https://192.168.12.10:6443"
CLUSTER_NAME="kubernetes"
KUBE_USER="kubernetes-admin"
KUBE_CERT="admin"
KUBE_CONFIG="admin.conf"

# 设置集群参数
kubectl config set-cluster ${CLUSTER_NAME} \
  --certificate-authority=ca.crt \
  --embed-certs=true \
  --server=${KUBE_APISERVER} \
  --kubeconfig=${KUBE_CONFIG}

# 设置客户端认证参数
kubectl config set-credentials ${KUBE_USER} \
  --client-certificate=${KUBE_CERT}.crt \
  --client-key=${KUBE_CERT}.key \
  --embed-certs=true \
  --kubeconfig=${KUBE_CONFIG}

# 设置上下文参数
kubectl config set-context ${KUBE_USER}@${CLUSTER_NAME} \
  --cluster=${CLUSTER_NAME} \
  --user=${KUBE_USER} \
  --kubeconfig=${KUBE_CONFIG}

# 设置当前使用的上下文
kubectl config use-context ${KUBE_USER}@${CLUSTER_NAME} --kubeconfig=${KUBE_CONFIG}

# 查看生成的配置文件
kubectl config view --kubeconfig=${KUBE_CONFIG}

    cp /etc/kubernetes/kubelet.conf .

    • kubelet.conf（注意配置对应的nodeName）

      KUBE_APISERVER="https://192.168.12.10:6443"
CLUSTER_NAME="kubernetes"
# 此处为 master 节点 nodeName，每个 master 生成对应的 kubelet.conf
KUBE_USER="system:node1:master"
KUBE_CERT="kubelet"
KUBE_CONFIG="kubelet.conf"

# 设置集群参数
kubectl config set-cluster ${CLUSTER_NAME} \
  --certificate-authority=ca.crt \
  --embed-certs=true \
  --server=${KUBE_APISERVER} \
  --kubeconfig=${KUBE_CONFIG}

# 设置客户端认证参数
kubectl config set-credentials ${KUBE_USER} \
  --client-certificate=${KUBE_CERT}.crt \
  --client-key=kubelet.key \
  --embed-certs=true \
  --kubeconfig=${KUBE_CONFIG}

# 设置上下文参数
kubectl config set-context ${KUBE_USER}@${CLUSTER_NAME} \
  --cluster=${CLUSTER_NAME} \
  --user=${KUBE_USER} \
  --kubeconfig=${KUBE_CONFIG}

# 设置当前使用的上下文
kubectl config use-context ${KUBE_USER}@${CLUSTER_NAME} --kubeconfig=${KUBE_CONFIG}

# 查看生成的配置文件
kubectl config view --kubeconfig=${KUBE_CONFIG}

    • scheduler.conf

      KUBE_APISERVER="https://192.168.12.10:6443"
CLUSTER_NAME="kubernetes"
KUBE_USER="system:kube-scheduler"
KUBE_CERT="kube-scheduler"
KUBE_CONFIG="scheduler.conf"

# 设置集群参数
kubectl config set-cluster ${CLUSTER_NAME} \
  --certificate-authority=ca.crt \
  --embed-certs=true \
  --server=${KUBE_APISERVER} \
  --kubeconfig=${KUBE_CONFIG}

# 设置客户端认证参数
kubectl config set-credentials ${KUBE_USER} \
  --client-certificate=${KUBE_CERT}.crt \
  --client-key=${KUBE_CERT}.key \
  --embed-certs=true \
  --kubeconfig=${KUBE_CONFIG}

# 设置上下文参数
kubectl config set-context ${KUBE_USER}@${CLUSTER_NAME} \
  --cluster=${CLUSTER_NAME} \
  --user=${KUBE_USER} \
  --kubeconfig=${KUBE_CONFIG}

# 设置当前使用的上下文
kubectl config use-context ${KUBE_USER}@${CLUSTER_NAME} --kubeconfig=${KUBE_CONFIG}

# 查看生成的配置文件
kubectl config view --kubeconfig=${KUBE_CONFIG}

    • controller-manager.conf

      KUBE_APISERVER="https://192.168.12.10:6443"
CLUSTER_NAME="kubernetes"
KUBE_USER="system:kube-controller-manager"
KUBE_CERT="kube-controller-manager"
KUBE_CONFIG="controller-manager.conf"

# 设置集群参数
kubectl config set-cluster ${CLUSTER_NAME} \
  --certificate-authority=ca.crt \
  --embed-certs=true \
  --server=${KUBE_APISERVER} \
  --kubeconfig=${KUBE_CONFIG}

# 设置客户端认证参数
kubectl config set-credentials ${KUBE_USER} \
  --client-certificate=${KUBE_CERT}.crt \
  --client-key=sa.key \
  --embed-certs=true \
  --kubeconfig=${KUBE_CONFIG}

# 设置上下文参数
kubectl config set-context ${KUBE_USER}@${CLUSTER_NAME} \
  --cluster=${CLUSTER_NAME} \
  --user=${KUBE_USER} \
  --kubeconfig=${KUBE_CONFIG}

# 设置当前使用的上下文
kubectl config use-context ${KUBE_USER}@${CLUSTER_NAME} --kubeconfig=${KUBE_CONFIG}

# 查看生成的配置文件
kubectl config view --kubeconfig=${KUBE_CONFIG}

      rm -rf *.csr
  cp -r /etc/kubernetes/ /etc/kubernetes.bak
  cd /etc/kubernetes/pki
 \cp -rf /root/key/* .
  cd ..
  \cp pki/*.conf .
   rm -rf openssl.conf

    • 应用配置

      • 重启 Docker 和 Kubelet
      • 查看三个kubernetes组件（kubelet，controller-manager，scheduler）的日志，确认是否还有证书过期报错信息。

    • Worker节点证书更新操作

      • 停止docker和kubelet

        systemctl stop docker && systemctl stop kubelet

      • 删除kubelet.conf文件，文件一般在/etc/kubernetes目录下。

      • 编辑bootstrap-kubelet.conf文件（文件一般在/etc/kubernetes目录下），修改certificate-authority-data内容，与master节点中的admin.conf文件的该区域内容相同。

      • 备份后删除该目录下的文件rm -rf /var/lib/kubelet/pki

      • 重启所有节点docker和kubelet

        systemctl restart docker && systemctl restart kubelet

    重启服务

    • 将全部kube-proxy重启
    • 将全部网络插件重启（比如：flannel）

    更新权限更新token

    • cp /etc/kubernetes/admin.conf ~/.kube/config
    • kubeadm token list
    • kubeadm token create --ttl=0
    • kubeadm token list
      把获取到的token更新到node节点/etc/kubernetes/bootstrap-kubelet.conf
      重启node节点kubelet

    相关文章

      网友评论

          本文标题：kubeadm集群更改证书

          本文链接：https://www.haomeiwen.com/subject/ovkzwqtx.html

          延伸阅读

          深度阅读

          • 您也可以注册成为美文阅读网的作者，发表您的原创作品、分享您的心情！

          栏目导航

          热点阅读

          关于我们|服务条款|联系我们|kubeadm集群更改证书|投稿指南|网站地图|RSS订阅|排版工具|手机版

          提供经典美文摘抄,优美散文欣赏,现代诗歌精选,短篇小说,心情随笔,表白情书范文,故事会在线阅读欣赏

          Copyright © 2014-2023 Haomeiwen.com All Rights Reserved. 好美文阅读网 版权所有

          备案信息：桂公网安备 45052102000051号 · 桂ICP备13007215号-3

          本站所收录作品、热点评论等信息部分来源互联网，目的只是为了系统归纳学习和传递资讯

          所有作品版权归原创作者所有，与本站立场无关，如不慎侵犯了你的权益，请联系我们告知，我们将做删除处理！