kubernetes集群搭建五：配置apiserver

作者: 开始懂了90 | 来源:发表于2018-12-18 16:58 被阅读0次

kubernetes集群搭建五：配置apiserver
kubernetes安装部署-day02
Kubernetes apiserver认证
搭建高可用Kubernetes集群之kubernetes集群搭建
Kubernetes自动化部署脚本
k8s搭建集群
kubelet 事件机制
centos7.4安装kubernetes搭建集群
kubernetes(k8s)集群搭建
安装kubeadm

创建 kubernetes 证书

cd /opt/ssl

# 如果使用负载均衡器 记得写入负载均衡器的域名

cat > kubernetes-csr.json << EOF
{
  "CN": "kubernetes",
  "hosts": [
    "127.0.0.1",
    "10.39.7.51",
    "10.39.7.52",
    "10.39.7.57",
    "10.254.0.1",
    "kubernetes",
    "kubernetes.default",
    "kubernetes.default.svc",
    "kubernetes.default.svc.cluster",
    "kubernetes.default.svc.cluster.local"
  ],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Beijing",
      "L": "Beijing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}
EOF

这里 hosts 字段中三个 IP 分别为 127.0.0.1, 10.39.7.51, 10.39.7.52, 10.39.7.57 为 Master 的IP，有多个Master需要写多个masterIP
如果使用了负载均衡器也要写上域名
10.254.0.1 为 kubernetes SVC 的 IP，一般是部署网络的第一个IP , 如: 10.254.0.1 ，在启动完成后，我们使用 kubectl get svc ，就可以查看到

生成 kubernetes 证书和私钥

cfssl gencert -ca=/opt/ssl/ca.pem \
  -ca-key=/opt/ssl/ca-key.pem \
  -config=/opt/ssl/ca-config.json \
  -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes

分发kubernetes证书

cp kubernetes* /etc/kubernetes/ssl
scp kubernetes* root@10.39.7.52:/etc/kubernetes/ssl
scp kubernetes* root@10.39.7.57:/etc/kubernetes/ssl

配置 kube-apiserver

kubelet 首次启动时向 kube-apiserver 发送 TLS Bootstrapping 请求，kube-apiserver 验证 kubelet 请求中的 token 是否与它配置的 token 一致，如果一致则自动为 kubelet生成证书和秘钥

# 生成 token

[root@k8s-master-6 ssl]# head -c 16 /dev/urandom | od -An -t x | tr -d ' '
8cd1b18581291d05dd07ee6b0ae748a6

# 创建 token.csv 文件

cd /opt/ssl

cat > /etc/kubernetes/token.csv << EOF
8cd1b18581291d05dd07ee6b0ae748a6,kubelet-bootstrap,10001,"system:kubelet-bootstrap"
EOF

审计

cat > /etc/kubernetes/audit-policy.yaml<<EOF
apiVersion: audit.k8s.io/v1beta1
kind: Policy
rules:
- level: Metadata
EOF

#分发略

生成加密encryption-config.yaml文件

cat > encryption-config.yaml <<EOF
kind: EncryptionConfig
apiVersion: v1
resources:
  - resources:
      - secrets
    providers:
      - aescbc:
          keys:
            - name: key1
              secret: 8cd1b18581291d05dd07ee6b0ae748a6
      - identity: {}
EOF

分发

cp  encryption-config.yaml /etc/kubernetes/
scp  encryption-config.yaml root@10.39.7.52:/etc/kubernetes/
scp  encryption-config.yaml root@10.39.7.57:/etc/kubernetes/

创建api-server 需要的相关目录

kubelet 创建pod有三种方式: 指定目录 URl 和api-server 我们创建的kubeapi-server kube-controller-manager kube-scheduler 都已json文件的形式放在/etc/kubernetes/manifests目录下 kubelet在启动是会加载这个目录并创建此目录下的需要的pod。如果删除了这里定义的pod 如apiserver 只需要重启kubelet服务就回重新自动创建

mkdir -pv /var/log/kubernetes/
mkdir -pv /etc/kubernetes/manifests

创建 kube-apiserver.json 文件

cat  > /etc/kubernetes/manifests/kube-apiserver.json<<EOF
{
  "kind": "Pod",
  "apiVersion": "v1",
  "metadata": {
    "name": "kube-apiserver",
    "namespace": "kube-system",
    "creationTimestamp": null,
    "labels": {
      "component": "kube-apiserver",
      "tier": "control-plane"
    }
  },
  "spec": {
    "volumes": [
      {
  "name": "certs",
  "hostPath": {
    "path": "/etc/ssl/certs"
  }
      },
      {
        "name": "hosts",
        "hostPath": {
          "path": "/etc/hosts"
        }
      },
      {
  "name": "pki",
  "hostPath": {
    "path": "/etc/kubernetes"
  }
      }
    ],
    "containers": [
      {
  "name": "kube-apiserver",
  "image": "harbor.enncloud.cn/enncloud/hyperkube-amd64:v1.11.2",
  "imagePullPolicy": "Always",
  "command": [
    "/apiserver",
    "--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota",
    "--anonymous-auth=false",
    "--token-auth-file=/etc/kubernetes/token.csv",
    "--advertise-address=10.39.7.51",
    "--allow-privileged=true",
    "--apiserver-count=3",
    "--audit-policy-file=/etc/kubernetes/audit-policy.yaml",
    "--audit-log-maxage=30",
    "--audit-log-maxbackup=3",
    "--audit-log-maxsize=100",
    "--audit-log-path=/var/log/kubernetes/audit.log",
    "--authorization-mode=Node,AlwaysAllow",
    "--bind-address=10.39.7.51",
    "--secure-port=6443",
    "--client-ca-file=/etc/kubernetes/ssl/ca.pem",
    "--kubelet-client-certificate=/etc/kubernetes/ssl/kubernetes.pem",
    "--kubelet-client-key=/etc/kubernetes/ssl/kubernetes-key.pem",
    "--enable-swagger-ui=true",
    "--etcd-cafile=/etc/kubernetes/ssl/ca.pem",
    "--etcd-certfile=/etc/kubernetes/ssl/etcd.pem",
    "--etcd-keyfile=/etc/kubernetes/ssl/etcd-key.pem",
    "--etcd-servers=https://10.39.7.51:2379,https://10.39.7.52:2379,https://10.39.7.57:2379",
    "--event-ttl=1h",
    "--kubelet-https=true",
    "--insecure-bind-address=127.0.0.1",
    "--insecure-port=8080",
    "--service-account-key-file=/etc/kubernetes/ssl/ca-key.pem",
    "--service-cluster-ip-range=10.254.0.0/18",
    "--service-node-port-range=30000-32000",
    "--tls-cert-file=/etc/kubernetes/ssl/kubernetes.pem",
    "--tls-private-key-file=/etc/kubernetes/ssl/kubernetes-key.pem",
    "--enable-bootstrap-token-auth",
          "--requestheader-client-ca-file=/etc/kubernetes/ssl/ca.pem", 
          "--requestheader-allowed-names=admin", 
          "--requestheader-extra-headers-prefix=X-Remote-Extra-", 
          "--requestheader-group-headers=X-Remote-Group", 
          "--requestheader-username-headers=X-Remote-User", 
          "--proxy-client-cert-file=/etc/kubernetes/ssl/admin.pem", 
          "--proxy-client-key-file=/etc/kubernetes/ssl/admin-key.pem", 
          "--enable-aggregator-routing=true", 
    "--v=2"
  ],
  "resources": {
    "requests": {
      "cpu": "250m"
    }
  },
  "volumeMounts": [
    {
      "name": "certs",
      "mountPath": "/etc/ssl/certs"
    },
         {
            "name": "hosts",
            "mountPath": "/etc/hosts"
          },
    {
      "name": "pki",
      "readOnly": true,
      "mountPath": "/etc/kubernetes/"
    }
  ],
  "livenessProbe": {
    "httpGet": {
      "path": "/healthz",
      "port": 8080,
      "host": "127.0.0.1"
    },
    "initialDelaySeconds": 15,
    "timeoutSeconds": 15,
    "failureThreshold": 8
  }
      }
    ],
    "hostNetwork": true
  }
 }
EOF

创建kube-scheduler.json

cat > /etc/kubernetes/manifests/kube-scheduler.json <<EOF
{
  "kind": "Pod",
  "apiVersion": "v1",
  "metadata": {
    "name": "kube-scheduler",
    "namespace": "kube-system",
    "creationTimestamp": null,
    "labels": {
      "component": "kube-scheduler",
      "tier": "control-plane"
    }
  },
  "spec": {
    "containers": [
      {
  "name": "kube-scheduler",
  "image": "harbor.enncloud.cn/enncloud/hyperkube-amd64:v1.11.2",
  "imagePullPolicy": "Always",
  "command": [
    "/scheduler",
    "--address=127.0.0.1",
    "--master=http://127.0.0.1:8080",
    "--leader-elect=true",
    "--v=2"
  ],
  "resources": {
    "requests": {
      "cpu": "100m"
    }
  },
  "livenessProbe": {
    "httpGet": {
      "path": "/healthz",
      "port": 10251,
      "host": "127.0.0.1"
    },
    "initialDelaySeconds": 15,
    "timeoutSeconds": 15,
    "failureThreshold": 8
  }
      }
    ],
    "hostNetwork": true
  }
}
EOF

创建 kube-controller-manager.json

cat > /etc/kubernetes/manifests/kube-controller-manager.json <<EOF
{
  "kind": "Pod",
  "apiVersion": "v1",
  "metadata": {
    "name": "kube-controller-manager",
    "namespace": "kube-system",
    "labels": {
      "component": "kube-controller-manager",
      "tier": "control-plane"
    }
  },
  "spec": {
    "volumes": [
      {
  "name": "certs",
  "hostPath": {
    "path": "/etc/ssl/certs"
  }
      },
      {
  "name": "pki",
  "hostPath": {
    "path": "/etc/kubernetes"
  }
      },
      {
  "name": "plugin",
  "hostPath": {
    "path": "/usr/libexec/kubernetes/kubelet-plugins"
  }
      },
      {
  "name": "qingcloud",
  "hostPath": {
    "path": "/etc/qingcloud"
  }
      }
    ],
    "containers": [
      {
  "name": "kube-controller-manager",
  "image": "harbor.enncloud.cn/enncloud/hyperkube-amd64:v1.11.2",
  "imagePullPolicy": "Always",
  "command": [
    "/controller-manager",
    "--address=127.0.0.1",
    "--master=http://127.0.0.1:8080",
    "--allocate-node-cidrs=false",
    "--service-cluster-ip-range=10.254.0.0/18",
    "--cluster-cidr=10.254.64.0/18",
    "--cluster-name=kubernetes",
    "--cluster-signing-cert-file=/etc/kubernetes/ssl/ca.pem",
    "--cluster-signing-key-file=/etc/kubernetes/ssl/ca-key.pem",
    "--service-account-private-key-file=/etc/kubernetes/ssl/ca-key.pem",
    "--root-ca-file=/etc/kubernetes/ssl/ca.pem",
    "--experimental-cluster-signing-duration=86700h0m0s",
    "--leader-elect=true",
    "--controllers=*,tokencleaner,bootstrapsigner",
    "--feature-gates=RotateKubeletServerCertificate=true",
    "--horizontal-pod-autoscaler-use-rest-clients",
    "--horizontal-pod-autoscaler-sync-period=60s",
    "--node-monitor-grace-period=40s",
    " --node-monitor-period=5s",
    "--pod-eviction-timeout=5m0s",
    "--v=10"
  ],
  "resources": {
    "requests": {
      "cpu": "200m"
    }
  },
  "volumeMounts": [
    {
      "name": "certs",
      "mountPath": "/etc/ssl/certs"
    },
    {
      "name": "pki",
      "readOnly": true,
      "mountPath": "/etc/kubernetes/"
    },
                  {
      "name": "plugin",
      "mountPath": "/usr/libexec/kubernetes/kubelet-plugins"
    },
    {
      "name": "qingcloud",
      "readOnly": true,
      "mountPath": "/etc/qingcloud"
    }
  ],
  "livenessProbe": {
    "httpGet": {
      "path": "/healthz",
      "port": 10252,
      "host": "127.0.0.1"
    },
    "initialDelaySeconds": 15,
    "timeoutSeconds": 15,
    "failureThreshold": 8
  }
      }
    ],
    "hostNetwork": true
  }
}
EOF

kubernetes集群搭建五：配置apiserver
创建 kubernetes 证书这里 hosts 字段中三个 IP 分别为 127.0.0.1, 10.39....
kubernetes安装部署-day02
三、kubernetes集群部署之master部署： 3.1、kubernetes apiserver部署： K...
Kubernetes apiserver认证
kubernetes认证 Kubernetes集群的操作可以通过apiserver来进行操作，kubectl命令最...
搭建高可用Kubernetes集群之kubernetes集群搭建
在搭建kubernetes集群之前，请参考我的文章搭建高可用Kubernetes集群之etcd集群搭建篇（一）完成...
Kubernetes自动化部署脚本
使用Kubeadm搭建Kubernetes(1.13.1)集群中已经详细介绍了Kubernetes集群的搭建过程...
k8s搭建集群
使用mimikube搭建集群 minikube搭建集群: http://docs.kubernetes.org.c...
kubelet 事件机制
kubelet 事件机制对于 kubernetes 来说，apiserver 是整个集群的交互中心，客户端主要和...
centos7.4安装kubernetes搭建集群
搭建kubernetes集群在线安装离线安装
kubernetes(k8s)集群搭建
1.安装配置Kubernetes Master 1.1 启动etcd、kube-apiserver、kube-c...
安装kubeadm
概述 kubeadm 是 kubernetes 的集群安装工具，能够快速安装 kubernetes 集群。配置软...

kubernetes集群搭建五：配置apiserver

相关文章

kubernetes集群搭建五：配置apiserver

kubernetes安装部署-day02

Kubernetes apiserver认证

搭建高可用Kubernetes集群之kubernetes集群搭建

Kubernetes自动化部署脚本

k8s搭建集群

kubelet 事件机制

centos7.4安装kubernetes搭建集群

kubernetes(k8s)集群搭建

安装kubeadm

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读