原创内容，转载请注明出处
博主地址：https://aronligithub.github.io/
闲言乱语
在前段日子编写了kubernetes部署全过程之后，好友告诉我，你写得太长啦。能不能拆分章节一下。但是由于各种工作上和学习自研上的计划以及任务太多了，这个篇章的修改以及新篇章的编写给延迟了下来，但是为了更加方便各位读者们阅读，我以下对内容做了四个篇章的拆分
kubernetes v1.11 二进制部署篇章目录
前言
在经过上一篇章关于kubernetes 基本技术概述铺垫,在部署etcd集群之后，就可以开始部署kubernetes的集群服务了。
部署Node节点服务
在部署完毕上面的步骤之后，我们就可以开始部署Node的节点服务了，在部署之前，首先淡定将master部署时候创建的TLS以及相关kubeconfig文件都拷贝至各台node节点上。
Node服务器拓扑
因为上面已经写了很多内容了，相信读者还要找拓扑来看比较麻烦，那么就在这里部署Node服务之前，再次讲述一下。
1.首先我在之前的篇章已经部署好了三台etcd的集群服务
2.在server81的服务器上我部署好了Master节点的服务
3.那么下一步就是要给Server81、86、87三台服务器都部署上Node节点的服务了。
那么下面我们就开始动手部署Node节点的服务吧。
拷贝Master节点创建的TLS以及kubeconfig文件至Node节点服务
因为Server81就是Master节点服务，所以不需要拷贝证书。
而Server86、87服务器就需要拷贝了，执行命名如下：
[root@server81 etc]# scp -r kubernetes root@server86:/etc
ca.pem 100% 1135 243.9KB/s 00:00
ca.key 100% 1679 383.9KB/s 00:00
apiserver.pem 100% 1302 342.6KB/s 00:00
apiserver.key 100% 1675 378.4KB/s 00:00
admin.pem 100% 1050 250.3KB/s 00:00
admin.key 100% 1675 401.5KB/s 00:00
proxy.pem 100% 1009 253.2KB/s 00:00
proxy.key 100% 1679 74.5KB/s 00:00
token.csv 100% 84 4.5KB/s 00:00
config 100% 656 45.9KB/s 00:00
apiserver 100% 1656 484.7KB/s 00:00
controller-manager 100% 615 163.8KB/s 00:00
scheduler 100% 243 10.9KB/s 00:00
kube-proxy.kubeconfig 100% 5451 335.3KB/s 00:00
bootstrap.kubeconfig 100% 1869 468.9KB/s 00:00
[root@server81 etc]#
[root@server81 etc]# scp -r kubernetes root@server87:/etc
ca.pem 100% 1135 373.4KB/s 00:00
ca.key 100% 1679 470.8KB/s 00:00
apiserver.pem 100% 1302 511.5KB/s 00:00
apiserver.key 100% 1675 565.6KB/s 00:00
admin.pem 100% 1050 340.2KB/s 00:00
admin.key 100% 1675 468.4KB/s 00:00
proxy.pem 100% 1009 247.8KB/s 00:00
proxy.key 100% 1679 516.4KB/s 00:00
token.csv 100% 84 30.2KB/s 00:00
config 100% 656 217.0KB/s 00:00
apiserver 100% 1656 415.7KB/s 00:00
controller-manager 100% 615 240.0KB/s 00:00
scheduler 100% 243 92.1KB/s 00:00
kube-proxy.kubeconfig 100% 5451 1.3MB/s 00:00
bootstrap.kubeconfig 100% 1869 614.0KB/s 00:00
[root@server81 etc]#
查看Server86的拷贝文件情况，如下：
[root@server86 etc]# pwd
/etc
[root@server86 etc]#
[root@server86 etc]# tree kubernetes/
kubernetes/
├── apiserver
├── bootstrap.kubeconfig
├── config
├── controller-manager
├── kube-proxy.kubeconfig
├── kubernetesTLS
│ ├── admin.key
│ ├── admin.pem
│ ├── apiserver.key
│ ├── apiserver.pem
│ ├── ca.key
│ ├── ca.pem
│ ├── proxy.key
│ └── proxy.pem
├── scheduler
└── token.csv
1 directory, 15 files
[root@server86 etc]#
[root@server86 etc]# cd kubernetes/
[root@server86 kubernetes]# ls
apiserver bootstrap.kubeconfig config controller-manager kube-proxy.kubeconfig kubernetesTLS scheduler token.csv
[root@server86 kubernetes]# ls kubernetesTLS/
admin.key admin.pem apiserver.key apiserver.pem ca.key ca.pem proxy.key proxy.pem
[root@server86 kubernetes]#
查看Server87的拷贝文件情况，如下：
[root@server87 ~]# cd /etc/
[root@server87 etc]# pwd
/etc
[root@server87 etc]# tree kubernetes/
kubernetes/
├── apiserver
├── bootstrap.kubeconfig
├── config
├── controller-manager
├── kube-proxy.kubeconfig
├── kubernetesTLS
│ ├── admin.key
│ ├── admin.pem
│ ├── apiserver.key
│ ├── apiserver.pem
│ ├── ca.key
│ ├── ca.pem
│ ├── proxy.key
│ └── proxy.pem
├── scheduler
└── token.csv
1 directory, 15 files
[root@server87 etc]# cd kubernetes/
[root@server87 kubernetes]# ls
apiserver bootstrap.kubeconfig config controller-manager kube-proxy.kubeconfig kubernetesTLS scheduler token.csv
[root@server87 kubernetes]#
[root@server87 kubernetes]# ls kubernetesTLS/
admin.key admin.pem apiserver.key apiserver.pem ca.key ca.pem proxy.key proxy.pem
[root@server87 kubernetes]#
拷贝访问etcd集群的TLS证书文件
- 因为每台
Node都需要访问Etcd集群服务，在后面部署Calico或者flanneld网络的时候都是需要证书访问etcd集群的，该部分就会在后面的部署中说明了。 - 但是因为恰好
Server81、86、87服务器节点，我是用来做etcd三台服务集群的，在部署的时候已经拷贝好相关证书目录了。 - 可是，如果新增一台服务器想要加入Node的话，这时候该台服务器就需要单独将证书拷贝至相应的文件目录了。
那么这里展示一下etcd集群TLS证书文件应该放在Node节点的哪个目录文件下
其实哪个文件目录在部署etcd集群的时候我有说明过是可以自定义的，不过每个Node文件夹需要相同的服务器路径而已。
Server81存放etcd的TLS文件路径(
/etc/etcd/etcdSSL)
[root@server81 etc]# cd etcd/
[root@server81 etcd]# ls
etcd.conf etcdSSL
[root@server81 etcd]#
[root@server81 etcd]# cd etcdSSL/
[root@server81 etcdSSL]#
[root@server81 etcdSSL]# pwd
/etc/etcd/etcdSSL
[root@server81 etcdSSL]#
[root@server81 etcdSSL]# ls
ca-config.json ca.csr ca-csr.json ca-key.pem ca.pem etcd.csr etcd-csr.json etcd-key.pem etcd.pem
[root@server81 etcdSSL]#
[root@server81 etcdSSL]# ls -ll
total 36
-rw-r--r-- 1 root root 288 Aug 14 14:05 ca-config.json
-rw-r--r-- 1 root root 997 Aug 14 14:05 ca.csr
-rw-r--r-- 1 root root 205 Aug 14 14:05 ca-csr.json
-rw------- 1 root root 1675 Aug 14 14:05 ca-key.pem
-rw-r--r-- 1 root root 1350 Aug 14 14:05 ca.pem
-rw-r--r-- 1 root root 1066 Aug 14 14:05 etcd.csr
-rw-r--r-- 1 root root 296 Aug 14 14:05 etcd-csr.json
-rw------- 1 root root 1675 Aug 14 14:05 etcd-key.pem
-rw-r--r-- 1 root root 1436 Aug 14 14:05 etcd.pem
[root@server81 etcdSSL]#
Server86存在etcd的TLS文件路径(
/etc/etcd/etcdSSL)
[root@server86 etcd]# cd etcdSSL/
[root@server86 etcdSSL]#
[root@server86 etcdSSL]# pwd
/etc/etcd/etcdSSL
[root@server86 etcdSSL]# ls -ll
total 36
-rw-r--r-- 1 root root 288 Aug 14 16:42 ca-config.json
-rw-r--r-- 1 root root 997 Aug 14 16:42 ca.csr
-rw-r--r-- 1 root root 205 Aug 14 16:42 ca-csr.json
-rw------- 1 root root 1675 Aug 14 16:42 ca-key.pem
-rw-r--r-- 1 root root 1350 Aug 14 16:42 ca.pem
-rw-r--r-- 1 root root 1066 Aug 14 16:42 etcd.csr
-rw-r--r-- 1 root root 296 Aug 14 16:42 etcd-csr.json
-rw------- 1 root root 1675 Aug 14 16:42 etcd-key.pem
-rw-r--r-- 1 root root 1436 Aug 14 16:42 etcd.pem
[root@server86 etcdSSL]#
Server87存在etcd的TLS文件路径(
/etc/etcd/etcdSSL)
[root@server87 etcd]# cd etcdSSL/
[root@server87 etcdSSL]# ls
ca-config.json ca.csr ca-csr.json ca-key.pem ca.pem etcd.csr etcd-csr.json etcd-key.pem etcd.pem
[root@server87 etcdSSL]#
[root@server87 etcdSSL]# pwd
/etc/etcd/etcdSSL
[root@server87 etcdSSL]#
[root@server87 etcdSSL]# ls -ll
total 36
-rw-r--r-- 1 root root 288 Aug 14 16:52 ca-config.json
-rw-r--r-- 1 root root 997 Aug 14 16:52 ca.csr
-rw-r--r-- 1 root root 205 Aug 14 16:52 ca-csr.json
-rw------- 1 root root 1675 Aug 14 16:52 ca-key.pem
-rw-r--r-- 1 root root 1350 Aug 14 16:52 ca.pem
-rw-r--r-- 1 root root 1066 Aug 14 16:52 etcd.csr
-rw-r--r-- 1 root root 296 Aug 14 16:52 etcd-csr.json
-rw------- 1 root root 1675 Aug 14 16:52 etcd-key.pem
-rw-r--r-- 1 root root 1436 Aug 14 16:52 etcd.pem
[root@server87 etcdSSL]#
部署Node步骤说明
- 部署docker-ce （如果是直接部署docker的话，那就要启用cgroup参数了，用docker-ce则不需要）
- 部署kubelet服务
- 部署kube-proxy服务
基本上每台Node节点都需要部署这三个服务的，我就单独拿一台Server81部署进行说明先吧。其余Server86、87的部署过程都是跟Server81的Node节点部署一致的。
部署Docker-ce
如果不太懂docker安装的读者，可以访问docker官网的部署文档说明（官网需要翻墙访问比较顺畅）
1.下载docker-ce的rpm包
点击这里，下载docker-ce的rpm安装包。
2.执行安装docker-ce
yum install docker-ce-18.03.0.ce-1.el7.centos.x86_64.rpm -y
执行安装过程如下：
[root@server81 docker]# ls
certs.d docker-ce-18.03.0.ce-1.el7.centos.x86_64.rpm docker.service.simple install_docker-ce.sh set_docker_network.sh
daemon.json docker.service erase_docker-ce.sh login_registry.sh test.sh
[root@server81 docker]#
[root@server81 docker]# yum install docker-ce-18.03.0.ce-1.el7.centos.x86_64.rpm -y
Loaded plugins: fastestmirror
Examining docker-ce-18.03.0.ce-1.el7.centos.x86_64.rpm: docker-ce-18.03.0.ce-1.el7.centos.x86_64
Marking docker-ce-18.03.0.ce-1.el7.centos.x86_64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package docker-ce.x86_64 0:18.03.0.ce-1.el7.centos will be installed
--> Processing Dependency: container-selinux >= 2.9 for package: docker-ce-18.03.0.ce-1.el7.centos.x86_64
Loading mirror speeds from cached hostfile
......
Installed:
docker-ce.x86_64 0:18.03.0.ce-1.el7.centos
Dependency Installed:
audit-libs-python.x86_64 0:2.8.1-3.el7 checkpolicy.x86_64 0:2.5-6.el7 container-selinux.noarch 2:2.66-1.el7
libcgroup.x86_64 0:0.41-15.el7 libseccomp.x86_64 0:2.3.1-3.el7 libsemanage-python.x86_64 0:2.5-11.el7
pigz.x86_64 0:2.3.4-1.el7 policycoreutils-python.x86_64 0:2.5-22.el7 python-IPy.noarch 0:0.75-6.el7
setools-libs.x86_64 0:3.3.8-2.el7
Dependency Updated:
audit.x86_64 0:2.8.1-3.el7 audit-libs.x86_64 0:2.8.1-3.el7 libselinux.x86_64 0:2.5-12.el7
libselinux-python.x86_64 0:2.5-12.el7 libselinux-utils.x86_64 0:2.5-12.el7 libsemanage.x86_64 0:2.5-11.el7
libsepol.x86_64 0:2.5-8.1.el7 policycoreutils.x86_64 0:2.5-22.el7 selinux-policy.noarch 0:3.13.1-192.el7_5.4
selinux-policy-targeted.noarch 0:3.13.1-192.el7_5.4
Complete!
[root@server81 docker]#
3.启用docker-ce
systemctl daemon-reload
systemctl enable docker
systemctl restart docker
systemctl status docker
执行如下：
[root@server81 install_k8s_node]# systemctl daemon-reload
[root@server81 install_k8s_node]# systemctl enable docker
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.
[root@server81 install_k8s_node]# systemctl restart docker
[root@server81 install_k8s_node]# systemctl status docker
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2018-08-20 14:11:17 HKT; 639ms ago
Docs: https://docs.docker.com
Main PID: 3014 (dockerd)
Memory: 36.4M
CGroup: /system.slice/docker.service
├─3014 /usr/bin/dockerd
└─3021 docker-containerd --config /var/run/docker/containerd/containerd.toml
Aug 20 14:11:17 server81 dockerd[3014]: time="2018-08-20T14:11:17+08:00" level=info msg=serving... address="/var/run/docker/c...d/grpc"
Aug 20 14:11:17 server81 dockerd[3014]: time="2018-08-20T14:11:17+08:00" level=info msg="containerd successfully booted in 0....tainerd
Aug 20 14:11:17 server81 dockerd[3014]: time="2018-08-20T14:11:17.492174891+08:00" level=info msg="Graph migration to content...econds"
Aug 20 14:11:17 server81 dockerd[3014]: time="2018-08-20T14:11:17.493087053+08:00" level=info msg="Loading containers: start."
Aug 20 14:11:17 server81 dockerd[3014]: time="2018-08-20T14:11:17.608563905+08:00" level=info msg="Default bridge (docker0) i...ddress"
Aug 20 14:11:17 server81 dockerd[3014]: time="2018-08-20T14:11:17.645395453+08:00" level=info msg="Loading containers: done."
Aug 20 14:11:17 server81 dockerd[3014]: time="2018-08-20T14:11:17.659457843+08:00" level=info msg="Docker daemon" commit=0520...03.0-ce
Aug 20 14:11:17 server81 dockerd[3014]: time="2018-08-20T14:11:17.659619134+08:00" level=info msg="Daemon has completed initialization"
Aug 20 14:11:17 server81 dockerd[3014]: time="2018-08-20T14:11:17.669961967+08:00" level=info msg="API listen on /var/run/docker.sock"
Aug 20 14:11:17 server81 systemd[1]: Started Docker Application Container Engine.
Hint: Some lines were ellipsized, use -l to show in full.
[root@server81 install_k8s_node]#
[root@server81 install_k8s_node]# docker version
Client:
Version: 18.03.0-ce
API version: 1.37
Go version: go1.9.4
Git commit: 0520e24
Built: Wed Mar 21 23:09:15 2018
OS/Arch: linux/amd64
Experimental: false
Orchestrator: swarm
Server:
Engine:
Version: 18.03.0-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.9.4
Git commit: 0520e24
Built: Wed Mar 21 23:13:03 2018
OS/Arch: linux/amd64
Experimental: false
[root@server81 install_k8s_node]#
拷贝二进制可执行文件至Node服务器（
/usr/bin）
#!/bin/bash
basedir=$(cd `dirname $0`;pwd)
softwareDir=$basedir/../install_kubernetes_software
binDir=/usr/bin
## function and implments
function check_firewalld_selinux(){
systemctl status firewalld
/usr/sbin/sestatus -v
swapoff -a
}
check_firewalld_selinux
function copy_bin(){
cp -v $softwareDir/kubectl $binDir
cp -v $softwareDir/kubelet $binDir
cp -v $softwareDir/kube-proxy $binDir
}
copy_bin
执行结果如下：
[root@server81 install_k8s_node]# ./Step1_config.sh
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
Active: inactive (dead)
Docs: man:firewalld(1)
SELinux status: disabled
‘/opt/install_kubernetes/install_k8s_node/../install_kubernetes_software/kubectl’ -> ‘/usr/bin/kubectl’
‘/opt/install_kubernetes/install_k8s_node/../install_kubernetes_software/kubelet’ -> ‘/usr/bin/kubelet’
‘/opt/install_kubernetes/install_k8s_node/../install_kubernetes_software/kube-proxy’ -> ‘/usr/bin/kube-proxy’
[root@server81 install_k8s_node]#
[root@server81 install_k8s_node]# ls -ll /usr/bin/kube*
-rwxr-xr-x 1 root root 185471375 Aug 19 22:57 /usr/bin/kube-apiserver
-rwxr-xr-x 1 root root 154056749 Aug 19 22:57 /usr/bin/kube-controller-manager
-rwxr-xr-x 1 root root 55421261 Aug 20 14:14 /usr/bin/kubectl
-rwxr-xr-x 1 root root 162998216 Aug 20 14:14 /usr/bin/kubelet
-rwxr-xr-x 1 root root 52055519 Aug 20 14:14 /usr/bin/kube-proxy
-rwxr-xr-x 1 root root 55610654 Aug 19 22:57 /usr/bin/kube-scheduler
[root@server81 install_k8s_node]#
首先关闭每台Node服务器的swap分区、防火墙、selinux，然后将二进制可执行文件拷贝至/usr/bin目录下。
那么下面开始部署Node节点的kubelet和kube-proxy服务。
部署kubelet服务
1.编写kubelet.service文件（/usr/lib/systemd/system）
编写kubelet.service写入/usr/lib/systemd/system目录下：
[root@server81 install_k8s_node]# cat /usr/lib/systemd/system/kubelet.service
[Unit]
Description=Kubernetes Kubelet Server
Documentation=https://github.com/GoogleCloudPlatform/kubernetes
After=docker.service
Requires=docker.service
[Service]
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/kubelet
ExecStart=/usr/bin/kubelet \
$KUBE_LOGTOSTDERR \
$KUBE_LOG_LEVEL \
$KUBELET_CONFIG\
$KUBELET_ADDRESS \
$KUBELET_PORT \
$KUBELET_HOSTNAME \
$KUBELET_POD_INFRA_CONTAINER \
$KUBELET_ARGS
Restart=on-failure
[Install]
WantedBy=multi-user.target
[root@server81 install_k8s_node]#
kubelet.service参数说明
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/kubelet
说明：配置kubelet启用读取的两个配置文件config、kubelet，其中config在部署master服务的时候已经写好了，这是一个通用的配置文件。那么下面则单独编写kubelet的配置文件。
ExecStart=/usr/bin/kubelet \
$KUBE_LOGTOSTDERR \
$KUBE_LOG_LEVEL \
$KUBELET_CONFIG\
$KUBELET_ADDRESS \
$KUBELET_PORT \
$KUBELET_HOSTNAME \
$KUBELET_POD_INFRA_CONTAINER \
$KUBELET_ARGS
说明：定义service启用的时候运行的二进制可执行文件（/usr/bin/kubelet）以及相关服务启动所需的参数（这些参数从配置文件中读取）。
配置文件kubelet（/etc/kubernetes）
编写kubelet配置文件至/etc/kubernetes/目录下：
[root@server81 install_k8s_node]# cat /etc/kubernetes/
apiserver config kubelet kubernetesTLS/ token.csv
bootstrap.kubeconfig controller-manager kube-proxy.kubeconfig scheduler
[root@server81 install_k8s_node]# cat /etc/kubernetes/kubelet
###
## kubernetes kubelet (minion) config
#
## The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
#KUBELET_ADDRESS="--address=0.0.0.0"
#
## The port for the info server to serve on
#KUBELET_PORT="--port=10250"
#
## You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname-override=172.16.5.81"
#
## location of the api-server
KUBELET_CONFIG="--kubeconfig=/etc/kubernetes/kubelet.kubeconfig"
#
## pod infrastructure container
KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=172.16.5.81:5000/pause-amd64:3.1"
#
## Add your own!
KUBELET_ARGS="--cluster-dns=10.0.6.200 --serialize-image-pulls=false --bootstrap-kubeconfig=/etc/kubernetes/bootstrap.kubeconfig --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --cert-dir=/etc/kubernetes/kubernetesTLS --cluster-domain=cluster.local. --hairpin-mode promiscuous-bridge --network-plugin=cni"
[root@server81 install_k8s_node]#
kubelet配置文件中的相关参数说明
## You may leave this blank to use the actual hostname
KUBELET_HOSTNAME="--hostname-override=172.16.5.81"
说明：这里是写Node节点的名称，我使用该服务器的IP地址进行覆盖。如果是在Server87、Server86上部署，则修改相应的IP地址即可。
在部署完毕之后，执行kubectl get node，你就可以看到你定义的node节点名称的了。
## location of the api-server
KUBELET_CONFIG="--kubeconfig=/etc/kubernetes/kubelet.kubeconfig"
说明：定义kubelet的kubeconfig文件路径，之前在master部署的时候创建的。
## pod infrastructure container
KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=172.16.5.81:5000/pause-amd64:3.1"
说明：
- 在创建应用的时候，
kubelet是需要依赖于pause镜像的，如果没有pause镜像，那么镜像就会启用失败。 - 所以每个
Node节点上必须要有pause的镜像，但是默认pause镜像需要翻墙后再去官网下载的，这样会影响镜像启动的效率，那么我就将pause镜像下载到我的私有仓库中，方便内网启动。
这里pause镜像的私有地址：172.16.5.81:5000/pause-amd64:3.1 - 对于读者可以从以下地址地址
pause镜像，然后再搭设一个自己的私有仓库。下载地址如下：（该仓库是另一位博客作者提供的，在此感谢他）该作者写的kuberntes部署是没有启用RBAC模式的，是极简模式，有兴趣的读者也可以去看看。
docker pull mirrorgooglecontainers/pause-amd64:3.1
docker tag mirrorgooglecontainers/pause-amd64:3.1 k8s.gcr.io/pause-amd64:3.1
- 如果你可以翻墙，直接下载官网的镜像地址即可
docker pull k8s.gcr.io/pause-amd64:3.1
## Add your own!
KUBELET_ARGS="--cluster-dns=10.0.6.200 \
--serialize-image-pulls=false \
--bootstrap-kubeconfig=/etc/kubernetes/bootstrap.kubeconfig \
--kubeconfig=/etc/kubernetes/kubelet.kubeconfig \
--cert-dir=/etc/kubernetes/kubernetesTLS \
--cluster-domain=cluster.local. \
--hairpin-mode promiscuous-bridge \
--network-plugin=cni"
| 参数 | 说明 |
|---|---|
--cluster-dns=10.0.6.200 |
设置kubernetes集群网络中内部DNS的IP地址，后续用于CoreDNS
|
--serialize-image-pulls=false |
设置kubernetes集群允许使用http非安全镜像拉取 |
--bootstrap-kubeconfig=/etc/kubernetes/bootstrap.kubeconfig |
设置bootstrap.kubeconfig的文件路径 |
--cert-dir=/etc/kubernetes/kubernetesTLS |
设置kubernetes的TLS文件路径，后续kubelet服务启动之后，会在该文件夹自动创建kubelet相关公钥和私钥文件 |
--cluster-domain=cluster.local. |
设置kubernetes集群的DNS域名 |
--hairpin-mode promiscuous-bridge |
设置pod桥接网络模式 |
--network-plugin=cni |
设置启用CNI网络插件，因为后续是使用Calico网络，所以需要配置 |
如果你还想更加详细了解kubelet的参数配置，可以访问官网，点击这里。
启动kubelet服务
systemctl daemon-reload
systemctl enable kubelet
systemctl start kubelet
systemctl status kubelet
执行运行如下：
[root@server81 kubernetesTLS]# ls -ll
total 32
-rw-r--r-- 1 root root 1675 Aug 19 22:21 admin.key
-rw-r--r-- 1 root root 1050 Aug 19 22:21 admin.pem
-rw-r--r-- 1 root root 1675 Aug 19 22:21 apiserver.key
-rw-r--r-- 1 root root 1302 Aug 19 22:21 apiserver.pem
-rw-r--r-- 1 root root 1679 Aug 19 22:21 ca.key
-rw-r--r-- 1 root root 1135 Aug 19 22:21 ca.pem
-rw-r--r-- 1 root root 1679 Aug 19 22:21 proxy.key
-rw-r--r-- 1 root root 1009 Aug 19 22:21 proxy.pem
[root@server81 kubernetesTLS]#
[root@server81 kubernetesTLS]# systemctl daemon-reload
[root@server81 kubernetesTLS]# systemctl enable kubelet
[root@server81 kubernetesTLS]# systemctl start kubelet
[root@server81 kubernetesTLS]# systemctl status kubelet
● kubelet.service - Kubernetes Kubelet Server
Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2018-08-20 15:07:26 HKT; 640ms ago
Docs: https://github.com/GoogleCloudPlatform/kubernetes
Main PID: 3589 (kubelet)
Memory: 16.1M
CGroup: /system.slice/kubelet.service
└─3589 /usr/bin/kubelet --logtostderr=true --v=0 --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --hostname-override=172.16.5.81 --pod-infra-container-image=172.16.5.81:5000/...
Aug 20 15:07:26 server81 systemd[1]: Started Kubernetes Kubelet Server.
Aug 20 15:07:26 server81 systemd[1]: Starting Kubernetes Kubelet Server...
Aug 20 15:07:26 server81 kubelet[3589]: Flag --cluster-dns has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. Se...information.
Aug 20 15:07:26 server81 kubelet[3589]: Flag --serialize-image-pulls has been deprecated, This parameter should be set via the config file specified by the Kubelet's --confi...information.
Aug 20 15:07:26 server81 kubelet[3589]: Flag --cluster-domain has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag....information.
Aug 20 15:07:26 server81 kubelet[3589]: Flag --hairpin-mode has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. S...information.
Aug 20 15:07:26 server81 kubelet[3589]: I0820 15:07:26.364083 3589 feature_gate.go:230] feature gates: &{map[]}
Aug 20 15:07:26 server81 kubelet[3589]: I0820 15:07:26.364224 3589 feature_gate.go:230] feature gates: &{map[]}
Hint: Some lines were ellipsized, use -l to show in full.
[root@server81 kubernetesTLS]#
[root@server81 kubernetesTLS]# ls -ll
total 44
-rw-r--r-- 1 root root 1675 Aug 19 22:21 admin.key
-rw-r--r-- 1 root root 1050 Aug 19 22:21 admin.pem
-rw-r--r-- 1 root root 1675 Aug 19 22:21 apiserver.key
-rw-r--r-- 1 root root 1302 Aug 19 22:21 apiserver.pem
-rw-r--r-- 1 root root 1679 Aug 19 22:21 ca.key
-rw-r--r-- 1 root root 1135 Aug 19 22:21 ca.pem
-rw------- 1 root root 227 Aug 20 15:07 kubelet-client.key.tmp
-rw-r--r-- 1 root root 2177 Aug 20 15:07 kubelet.crt
-rw------- 1 root root 1679 Aug 20 15:07 kubelet.key
-rw-r--r-- 1 root root 1679 Aug 19 22:21 proxy.key
-rw-r--r-- 1 root root 1009 Aug 19 22:21 proxy.pem
[root@server81 kubernetesTLS]#
注意：
- 可以从文件夹中看出，
kubelet服务启动之后，自动响应生成了这三个文件：kubelet-client.key.tmp kubelet.crt kubelet.key。 - 如果需要重新部署
kubelet服务，那么就需要删除这三个文件即可。不然会提示过期，服务启动异常。 - 另外，可以看到
kubelet-client.key.tmp该文件还没有亮色，不可以运行起来，原因是kubelet向apiserver发出CSR认证的请求，此时apiserver还没有认证通过。 - 那么下一步就需要回到
master服务认证csr。
在master节点服务器认证通过csr
master认证通过csr脚本如下：
#!/bin/bash
basedir=$(cd `dirname $0`;pwd)
## function
function node_approve_csr(){
CSR=`kubectl get csr | grep csr | grep Pending | awk '{print $1}' | head -n 1`
kubectl certificate approve $CSR
kubectl get nodes
}
node_approve_csr
执行通过csr过程如下：
[root@server81 kubernetesTLS]# ls
admin.key admin.pem apiserver.key apiserver.pem ca.key ca.pem kubelet-client.key.tmp kubelet.crt kubelet.key proxy.key proxy.pem
[root@server81 kubernetesTLS]#
[root@server81 kubernetesTLS]# kubectl get node
No resources found.
[root@server81 kubernetesTLS]#
[root@server81 kubernetesTLS]# kubectl get csr
NAME AGE REQUESTOR CONDITION
node-csr-_xuU6rY0NNn9v2kgY58dOI86X_F1PBcbziXByJXnB7s 54m kubelet-bootstrap Pending
node-csr-fH4Ct4Fg4TgzFV0dP-SlfVCtTo9XNCJjajzPohDVxHE 6m kubelet-bootstrap Pending
[root@server81 kubernetesTLS]#
[root@server81 kubernetesTLS]# kubectl certificate approve node-csr-fH4Ct4Fg4TgzFV0dP-SlfVCtTo9XNCJjajzPohDVxHE
certificatesigningrequest.certificates.k8s.io/node-csr-fH4Ct4Fg4TgzFV0dP-SlfVCtTo9XNCJjajzPohDVxHE approved
[root@server81 kubernetesTLS]#
[root@server81 kubernetesTLS]# kubectl get node
NAME STATUS ROLES AGE VERSION
172.16.5.81 NotReady <none> 5s v1.11.0
[root@server81 kubernetesTLS]#
[root@server81 kubernetesTLS]# kubectl get csr
NAME AGE REQUESTOR CONDITION
node-csr-_xuU6rY0NNn9v2kgY58dOI86X_F1PBcbziXByJXnB7s 54m kubelet-bootstrap Pending
node-csr-fH4Ct4Fg4TgzFV0dP-SlfVCtTo9XNCJjajzPohDVxHE 7m kubelet-bootstrap Approved,Issued
[root@server81 kubernetesTLS]#
[root@server81 kubernetesTLS]# ls -ll
total 44
-rw-r--r-- 1 root root 1675 Aug 19 22:21 admin.key
-rw-r--r-- 1 root root 1050 Aug 19 22:21 admin.pem
-rw-r--r-- 1 root root 1675 Aug 19 22:21 apiserver.key
-rw-r--r-- 1 root root 1302 Aug 19 22:21 apiserver.pem
-rw-r--r-- 1 root root 1679 Aug 19 22:21 ca.key
-rw-r--r-- 1 root root 1135 Aug 19 22:21 ca.pem
-rw------- 1 root root 1183 Aug 20 15:14 kubelet-client-2018-08-20-15-14-35.pem
lrwxrwxrwx 1 root root 68 Aug 20 15:14 kubelet-client-current.pem -> /etc/kubernetes/kubernetesTLS/kubelet-client-2018-08-20-15-14-35.pem
-rw-r--r-- 1 root root 2177 Aug 20 15:07 kubelet.crt
-rw------- 1 root root 1679 Aug 20 15:07 kubelet.key
-rw-r--r-- 1 root root 1679 Aug 19 22:21 proxy.key
-rw-r--r-- 1 root root 1009 Aug 19 22:21 proxy.pem
[root@server81 kubernetesTLS]#
说明：
- 可以看到执行
kubectl certificate approve node-csr-fH4Ct4Fg4TgzFV0dP-SlfVCtTo9XNCJjajzPohDVxHE之后， - 再次执行
kubectl get csr查看csr的时候，在node-csr的状态就变成了Approved,Issued了， - 此时
kubectl get node的时候就可以看到node节点了，只是状态为NotReady而已 - 另外，查看
TLS文件夹，可以看到kubelet-client.key.tmp该临时文件在csr通过之后，变成了文件如下：
-rw------- 1 root root 1183 Aug 20 15:14 kubelet-client-2018-08-20-15-14-35.pem
lrwxrwxrwx 1 root root 68 Aug 20 15:14 kubelet-client-current.pem -> /etc/kubernetes/kubernetesTLS/kubelet-client-2018-08-20-15-14-35.pem
最后查看一下kubelet启动后的日志：
[root@server81 install_k8s_node]# journalctl -f -u kubelet
-- Logs begin at Sun 2018-08-19 21:26:42 HKT. --
Aug 20 15:20:51 server81 kubelet[3589]: W0820 15:20:51.476453 3589 cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d
Aug 20 15:20:51 server81 kubelet[3589]: E0820 15:20:51.477201 3589 kubelet.go:2112] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Aug 20 15:20:56 server81 kubelet[3589]: W0820 15:20:56.479691 3589 cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d
Aug 20 15:20:56 server81 kubelet[3589]: E0820 15:20:56.480061 3589 kubelet.go:2112] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Aug 20 15:21:01 server81 kubelet[3589]: W0820 15:21:01.483272 3589 cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d
Aug 20 15:21:01 server81 kubelet[3589]: E0820 15:21:01.484824 3589 kubelet.go:2112] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Aug 20 15:21:06 server81 kubelet[3589]: W0820 15:21:06.488203 3589 cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d
Aug 20 15:21:06 server81 kubelet[3589]: E0820 15:21:06.489788 3589 kubelet.go:2112] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Aug 20 15:21:11 server81 kubelet[3589]: W0820 15:21:11.497281 3589 cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d
Aug 20 15:21:11 server81 kubelet[3589]: E0820 15:21:11.497941 3589 kubelet.go:2112] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
Aug 20 15:21:16 server81 kubelet[3589]: W0820 15:21:16.502290 3589 cni.go:172] Unable to update cni config: No networks found in /etc/cni/net.d
Aug 20 15:21:16 server81 kubelet[3589]: E0820 15:21:16.502733 3589 kubelet.go:2112] Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
说明：此时日志提示没有cni网络，这个后续在安装Calico网络的时候说明。
部署kube-proxy服务
编写kube-proxy.service文件（
/usr/lib/systemd/system）
[root@server81 install_k8s_node]# cat /usr/lib/systemd/system/kube-proxy.service
[Unit]
Description=Kube Proxy Service
After=network.target
[Service]
Type=simple
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/proxy
ExecStart=/usr/bin/kube-proxy \
$KUBE_LOGTOSTDERR \
$KUBE_LOG_LEVEL \
$KUBE_MASTER \
$KUBE_PROXY_ARGS
Restart=always
LimitNOFILE=65536
[Install]
WantedBy=default.target
[root@server81 install_k8s_node]#
kube-proxy.service 说明：
EnvironmentFile=-/etc/kubernetes/config
EnvironmentFile=-/etc/kubernetes/proxy
配置kube-proxy启用读取的两个配置文件config、proxy，其中config在部署master服务的时候已经写好了，这是一个通用的配置文件。那么下面则单独编写proxy的配置文件。
ExecStart=/usr/bin/kube-proxy \
$KUBE_LOGTOSTDERR \
$KUBE_LOG_LEVEL \
$KUBE_MASTER \
$KUBE_PROXY_ARGS
配置kube-proxy服务启动使用的二进制可执行文件的路径（/usr/bin/kube-proxy）以及相关启动参数
配置文件proxy（
/etc/kubernetes）
[root@server81 install_k8s_node]# cat /etc/kubernetes/
apiserver config kubelet kube-proxy.kubeconfig proxy token.csv
bootstrap.kubeconfig controller-manager kubelet.kubeconfig kubernetesTLS/ scheduler
[root@server81 install_k8s_node]# cat /etc/kubernetes/proxy
###
# kubernetes proxy config
# defaults from config and proxy should be adequate
# Add your own!
KUBE_PROXY_ARGS="--kubeconfig=/etc/kubernetes/kube-proxy.kubeconfig --cluster-cidr=10.1.0.0/16"
[root@server81 install_k8s_node]#
参数说明：
-
--kubeconfig=/etc/kubernetes/kube-proxy.kubeconfig指定proxy运行的kubeconfig文件路径 -
--cluster-cidr=10.1.0.0/16指定pod在kubernetes启动的虚拟IP网段(CNI网络),提供后续calico使用参数
启动kube-proxy服务
systemctl daemon-reload
systemctl enable kube-proxy
systemctl start kube-proxy
systemctl status kube-proxy
执行如下：
[root@server81 install_k8s_node]# systemctl daemon-reload
[root@server81 install_k8s_node]# systemctl enable kube-proxy
[root@server81 install_k8s_node]# systemctl start kube-proxy
[root@server81 install_k8s_node]# systemctl status kube-proxy
● kube-proxy.service - Kube Proxy Service
Loaded: loaded (/usr/lib/systemd/system/kube-proxy.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2018-08-20 15:32:10 HKT; 11min ago
Main PID: 3988 (kube-proxy)
CGroup: /system.slice/kube-proxy.service
‣ 3988 /usr/bin/kube-proxy --logtostderr=true --v=0 --master=http://172.16.5.81:8080 --kubeconfig=/etc/kubernetes/kube-proxy.kubeconfig --cluster-cidr=10.1.0.0/16
Aug 20 15:32:10 server81 kube-proxy[3988]: I0820 15:32:10.742562 3988 conntrack.go:52] Setting nf_conntrack_max to 131072
Aug 20 15:32:10 server81 kube-proxy[3988]: I0820 15:32:10.748678 3988 conntrack.go:83] Setting conntrack hashsize to 32768
Aug 20 15:32:10 server81 kube-proxy[3988]: I0820 15:32:10.749216 3988 conntrack.go:98] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400
Aug 20 15:32:10 server81 kube-proxy[3988]: I0820 15:32:10.749266 3988 conntrack.go:98] Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600
Aug 20 15:32:10 server81 kube-proxy[3988]: I0820 15:32:10.749762 3988 config.go:102] Starting endpoints config controller
Aug 20 15:32:10 server81 kube-proxy[3988]: I0820 15:32:10.749807 3988 controller_utils.go:1025] Waiting for caches to sync for endpoints config controller
Aug 20 15:32:10 server81 kube-proxy[3988]: I0820 15:32:10.749838 3988 config.go:202] Starting service config controller
Aug 20 15:32:10 server81 kube-proxy[3988]: I0820 15:32:10.749845 3988 controller_utils.go:1025] Waiting for caches to sync for service config controller
Aug 20 15:32:10 server81 kube-proxy[3988]: I0820 15:32:10.850911 3988 controller_utils.go:1032] Caches are synced for endpoints config controller
Aug 20 15:32:10 server81 kube-proxy[3988]: I0820 15:32:10.850959 3988 controller_utils.go:1032] Caches are synced for service config controller
[root@server81 install_k8s_node]#
执行到这里，关于node的服务也已经部署好了，而其他Server86和87的服务，我这边使用脚本快速部署一下，执行过程于Server81一致。
使用脚本快速部署Server86服务器
[root@server86 kubernetesTLS]# cd /opt/
[root@server86 opt]# ls
install_etcd_cluster install_kubernetes rh
[root@server86 opt]#
[root@server86 opt]#
[root@server86 opt]# cd install_kubernetes/
[root@server86 install_kubernetes]# ls
check_etcd install_Calico install_CoreDNS install_k8s_master install_k8s_node install_kubernetes_software install_RAS_node MASTER_INFO reademe.txt
[root@server86 install_kubernetes]#
[root@server86 install_kubernetes]# cd install_k8s_node/
[root@server86 install_k8s_node]# ls
nodefile Step1_config.sh Step2_install_docker.sh Step3_install_kubelet.sh Step4_install_proxy.sh Step5_node_approve_csr.sh Step6_master_node_context.sh
[root@server86 install_k8s_node]#
[root@server86 install_k8s_node]# ./Step1_config.sh
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
Active: inactive (dead)
Docs: man:firewalld(1)
SELinux status: disabled
‘/opt/install_kubernetes/install_k8s_node/../install_kubernetes_software/kubectl’ -> ‘/usr/bin/kubectl’
‘/opt/install_kubernetes/install_k8s_node/../install_kubernetes_software/kubelet’ -> ‘/usr/bin/kubelet’
‘/opt/install_kubernetes/install_k8s_node/../install_kubernetes_software/kube-proxy’ -> ‘/usr/bin/kube-proxy’
[root@server86 install_k8s_node]#
[root@server86 install_k8s_node]# ./Step2_install_docker.sh
Loaded plugins: fastestmirror, langpacks
Examining /opt/install_kubernetes/install_k8s_node/nodefile/docker/docker-ce-18.03.0.ce-1.el7.centos.x86_64.rpm: docker-ce-18.03.0.ce-1.el7.centos.x86_64
Marking /opt/install_kubernetes/install_k8s_node/nodefile/docker/docker-ce-18.03.0.ce-1.el7.centos.x86_64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package docker-ce.x86_64 0:18.03.0.ce-1.el7.centos will be installed
--> Processing Dependency: container-selinux >= 2.9 for package: docker-ce-18.03.0.ce-1.el7.centos.x86_64
Loading mirror speeds from cached hostfile
* base: mirrors.aliyun.com
* epel: mirrors.tongji.edu.cn
* extras: mirrors.aliyun.com
* updates: mirrors.163.com
--> Processing Dependency: pigz for package: docker-ce-18.03.0.ce-1.el7.centos.x86_64
--> Running transaction check
---> Package container-selinux.noarch 2:2.66-1.el7 will be installed
--> Processing Dependency: selinux-policy-targeted >= 3.13.1-192 for package: 2:container-selinux-2.66-1.el7.noarch
--> Processing Dependency: selinux-policy-base >= 3.13.1-192 for package: 2:container-selinux-2.66-1.el7.noarch
--> Processing Dependency: selinux-policy >= 3.13.1-192 for package: 2:container-selinux-2.66-1.el7.noarch
---> Package pigz.x86_64 0:2.3.4-1.el7 will be installed
--> Running transaction check
---> Package selinux-policy.noarch 0:3.13.1-166.el7_4.5 will be updated
---> Package selinux-policy.noarch 0:3.13.1-192.el7_5.4 will be an update
--> Processing Dependency: policycoreutils >= 2.5-18 for package: selinux-policy-3.13.1-192.el7_5.4.noarch
---> Package selinux-policy-targeted.noarch 0:3.13.1-166.el7_4.5 will be updated
---> Package selinux-policy-targeted.noarch 0:3.13.1-192.el7_5.4 will be an update
--> Running transaction check
---> Package policycoreutils.x86_64 0:2.5-17.1.el7 will be updated
--> Processing Dependency: policycoreutils = 2.5-17.1.el7 for package: policycoreutils-python-2.5-17.1.el7.x86_64
---> Package policycoreutils.x86_64 0:2.5-22.el7 will be an update
--> Processing Dependency: libsepol >= 2.5-8 for package: policycoreutils-2.5-22.el7.x86_64
--> Processing Dependency: libselinux-utils >= 2.5-12 for package: policycoreutils-2.5-22.el7.x86_64
--> Running transaction check
---> Package libselinux-utils.x86_64 0:2.5-11.el7 will be updated
---> Package libselinux-utils.x86_64 0:2.5-12.el7 will be an update
--> Processing Dependency: libselinux(x86-64) = 2.5-12.el7 for package: libselinux-utils-2.5-12.el7.x86_64
---> Package libsepol.i686 0:2.5-6.el7 will be updated
---> Package libsepol.x86_64 0:2.5-6.el7 will be updated
---> Package libsepol.i686 0:2.5-8.1.el7 will be an update
---> Package libsepol.x86_64 0:2.5-8.1.el7 will be an update
---> Package policycoreutils-python.x86_64 0:2.5-17.1.el7 will be updated
---> Package policycoreutils-python.x86_64 0:2.5-22.el7 will be an update
--> Processing Dependency: setools-libs >= 3.3.8-2 for package: policycoreutils-python-2.5-22.el7.x86_64
--> Processing Dependency: libsemanage-python >= 2.5-9 for package: policycoreutils-python-2.5-22.el7.x86_64
--> Running transaction check
---> Package libselinux.i686 0:2.5-11.el7 will be updated
---> Package libselinux.x86_64 0:2.5-11.el7 will be updated
--> Processing Dependency: libselinux(x86-64) = 2.5-11.el7 for package: libselinux-python-2.5-11.el7.x86_64
---> Package libselinux.i686 0:2.5-12.el7 will be an update
---> Package libselinux.x86_64 0:2.5-12.el7 will be an update
---> Package libsemanage-python.x86_64 0:2.5-8.el7 will be updated
---> Package libsemanage-python.x86_64 0:2.5-11.el7 will be an update
--> Processing Dependency: libsemanage = 2.5-11.el7 for package: libsemanage-python-2.5-11.el7.x86_64
---> Package setools-libs.x86_64 0:3.3.8-1.1.el7 will be updated
---> Package setools-libs.x86_64 0:3.3.8-2.el7 will be an update
--> Running transaction check
---> Package libselinux-python.x86_64 0:2.5-11.el7 will be updated
---> Package libselinux-python.x86_64 0:2.5-12.el7 will be an update
---> Package libsemanage.x86_64 0:2.5-8.el7 will be updated
---> Package libsemanage.x86_64 0:2.5-11.el7 will be an update
--> Finished Dependency Resolution
Dependencies Resolved
===========================================================================================================================================================================
Package Arch Version Repository Size
===========================================================================================================================================================================
Installing:
docker-ce x86_64 18.03.0.ce-1.el7.centos /docker-ce-18.03.0.ce-1.el7.centos.x86_64 151 M
Installing for dependencies:
container-selinux noarch 2:2.66-1.el7 extras 35 k
pigz x86_64 2.3.4-1.el7 epel 81 k
Updating for dependencies:
libselinux i686 2.5-12.el7 base 166 k
libselinux x86_64 2.5-12.el7 base 162 k
libselinux-python x86_64 2.5-12.el7 base 235 k
libselinux-utils x86_64 2.5-12.el7 base 151 k
libsemanage x86_64 2.5-11.el7 base 150 k
libsemanage-python x86_64 2.5-11.el7 base 112 k
libsepol i686 2.5-8.1.el7 base 293 k
libsepol x86_64 2.5-8.1.el7 base 297 k
policycoreutils x86_64 2.5-22.el7 base 867 k
policycoreutils-python x86_64 2.5-22.el7 base 454 k
selinux-policy noarch 3.13.1-192.el7_5.4 updates 453 k
selinux-policy-targeted noarch 3.13.1-192.el7_5.4 updates 6.6 M
setools-libs x86_64 3.3.8-2.el7 base 619 k
Transaction Summary
===========================================================================================================================================================================
Install 1 Package (+ 2 Dependent packages)
Upgrade ( 13 Dependent packages)
Total size: 161 M
Total download size: 11 M
Downloading packages:
No Presto metadata available for base
updates/7/x86_64/prestodelta | 420 kB 00:00:00
(1/15): container-selinux-2.66-1.el7.noarch.rpm | 35 kB 00:00:00
(2/15): libselinux-2.5-12.el7.i686.rpm | 166 kB 00:00:00
(3/15): libsemanage-2.5-11.el7.x86_64.rpm | 150 kB 00:00:00
(4/15): libsemanage-python-2.5-11.el7.x86_64.rpm | 112 kB 00:00:00
(5/15): libselinux-utils-2.5-12.el7.x86_64.rpm | 151 kB 00:00:00
(6/15): libselinux-2.5-12.el7.x86_64.rpm | 162 kB 00:00:00
(7/15): libsepol-2.5-8.1.el7.i686.rpm | 293 kB 00:00:00
(8/15): libsepol-2.5-8.1.el7.x86_64.rpm | 297 kB 00:00:00
(9/15): selinux-policy-3.13.1-192.el7_5.4.noarch.rpm | 453 kB 00:00:00
(10/15): policycoreutils-2.5-22.el7.x86_64.rpm | 867 kB 00:00:00
(11/15): selinux-policy-targeted-3.13.1-192.el7_5.4.noarch.rpm | 6.6 MB 00:00:00
(12/15): policycoreutils-python-2.5-22.el7.x86_64.rpm | 454 kB 00:00:01
(13/15): setools-libs-3.3.8-2.el7.x86_64.rpm | 619 kB 00:00:00
(14/15): pigz-2.3.4-1.el7.x86_64.rpm | 81 kB 00:00:01
(15/15): libselinux-python-2.5-12.el7.x86_64.rpm | 235 kB 00:00:01
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 4.7 MB/s | 11 MB 00:00:02
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Updating : libsepol-2.5-8.1.el7.x86_64 1/29
Updating : libselinux-2.5-12.el7.x86_64 2/29
Updating : libsemanage-2.5-11.el7.x86_64 3/29
Updating : libselinux-utils-2.5-12.el7.x86_64 4/29
Updating : policycoreutils-2.5-22.el7.x86_64 5/29
Updating : selinux-policy-3.13.1-192.el7_5.4.noarch 6/29
Updating : selinux-policy-targeted-3.13.1-192.el7_5.4.noarch 7/29
Updating : libsemanage-python-2.5-11.el7.x86_64 8/29
Updating : libselinux-python-2.5-12.el7.x86_64 9/29
Updating : setools-libs-3.3.8-2.el7.x86_64 10/29
Updating : policycoreutils-python-2.5-22.el7.x86_64 11/29
Installing : 2:container-selinux-2.66-1.el7.noarch 12/29
setsebool: SELinux is disabled.
Installing : pigz-2.3.4-1.el7.x86_64 13/29
Updating : libsepol-2.5-8.1.el7.i686 14/29
Installing : docker-ce-18.03.0.ce-1.el7.centos.x86_64 15/29
Updating : libselinux-2.5-12.el7.i686 16/29
Cleanup : selinux-policy-targeted-3.13.1-166.el7_4.5.noarch 17/29
Cleanup : policycoreutils-python-2.5-17.1.el7.x86_64 18/29
Cleanup : selinux-policy-3.13.1-166.el7_4.5.noarch 19/29
Cleanup : libselinux-2.5-11.el7 20/29
Cleanup : policycoreutils-2.5-17.1.el7.x86_64 21/29
Cleanup : libselinux-utils-2.5-11.el7.x86_64 22/29
Cleanup : setools-libs-3.3.8-1.1.el7.x86_64 23/29
Cleanup : libselinux-python-2.5-11.el7.x86_64 24/29
Cleanup : libsemanage-python-2.5-8.el7.x86_64 25/29
Cleanup : libsepol-2.5-6.el7 26/29
Cleanup : libsemanage-2.5-8.el7.x86_64 27/29
Cleanup : libselinux-2.5-11.el7 28/29
Cleanup : libsepol-2.5-6.el7 29/29
Verifying : libselinux-python-2.5-12.el7.x86_64 1/29
Verifying : selinux-policy-3.13.1-192.el7_5.4.noarch 2/29
Verifying : setools-libs-3.3.8-2.el7.x86_64 3/29
Verifying : libsemanage-python-2.5-11.el7.x86_64 4/29
Verifying : policycoreutils-2.5-22.el7.x86_64 5/29
Verifying : libsepol-2.5-8.1.el7.i686 6/29
Verifying : libsemanage-2.5-11.el7.x86_64 7/29
Verifying : selinux-policy-targeted-3.13.1-192.el7_5.4.noarch 8/29
Verifying : pigz-2.3.4-1.el7.x86_64 9/29
Verifying : policycoreutils-python-2.5-22.el7.x86_64 10/29
Verifying : 2:container-selinux-2.66-1.el7.noarch 11/29
Verifying : libselinux-2.5-12.el7.i686 12/29
Verifying : libsepol-2.5-8.1.el7.x86_64 13/29
Verifying : libselinux-2.5-12.el7.x86_64 14/29
Verifying : docker-ce-18.03.0.ce-1.el7.centos.x86_64 15/29
Verifying : libselinux-utils-2.5-12.el7.x86_64 16/29
Verifying : libselinux-utils-2.5-11.el7.x86_64 17/29
Verifying : libsepol-2.5-6.el7.i686 18/29
Verifying : libselinux-2.5-11.el7.x86_64 19/29
Verifying : libsepol-2.5-6.el7.x86_64 20/29
Verifying : policycoreutils-python-2.5-17.1.el7.x86_64 21/29
Verifying : selinux-policy-targeted-3.13.1-166.el7_4.5.noarch 22/29
Verifying : policycoreutils-2.5-17.1.el7.x86_64 23/29
Verifying : libsemanage-python-2.5-8.el7.x86_64 24/29
Verifying : libselinux-2.5-11.el7.i686 25/29
Verifying : libsemanage-2.5-8.el7.x86_64 26/29
Verifying : selinux-policy-3.13.1-166.el7_4.5.noarch 27/29
Verifying : libselinux-python-2.5-11.el7.x86_64 28/29
Verifying : setools-libs-3.3.8-1.1.el7.x86_64 29/29
Installed:
docker-ce.x86_64 0:18.03.0.ce-1.el7.centos
Dependency Installed:
container-selinux.noarch 2:2.66-1.el7 pigz.x86_64 0:2.3.4-1.el7
Dependency Updated:
libselinux.i686 0:2.5-12.el7 libselinux.x86_64 0:2.5-12.el7 libselinux-python.x86_64 0:2.5-12.el7
libselinux-utils.x86_64 0:2.5-12.el7 libsemanage.x86_64 0:2.5-11.el7 libsemanage-python.x86_64 0:2.5-11.el7
libsepol.i686 0:2.5-8.1.el7 libsepol.x86_64 0:2.5-8.1.el7 policycoreutils.x86_64 0:2.5-22.el7
policycoreutils-python.x86_64 0:2.5-22.el7 selinux-policy.noarch 0:3.13.1-192.el7_5.4 selinux-policy-targeted.noarch 0:3.13.1-192.el7_5.4
setools-libs.x86_64 0:3.3.8-2.el7
Complete!
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2018-08-20 15:57:07 HKT; 21ms ago
Docs: https://docs.docker.com
Main PID: 2955 (dockerd)
Memory: 39.0M
CGroup: /system.slice/docker.service
├─2955 /usr/bin/dockerd
└─2964 docker-containerd --config /var/run/docker/containerd/containerd.toml
Aug 20 15:57:06 server86 dockerd[2955]: time="2018-08-20T15:57:06.737217664+08:00" level=info msg="devmapper: Creating filesystem xfs on device docker-8:3-67...8927-base]"
Aug 20 15:57:07 server86 dockerd[2955]: time="2018-08-20T15:57:07.045640563+08:00" level=info msg="devmapper: Successfully created filesystem xfs on device d...18927-base"
Aug 20 15:57:07 server86 dockerd[2955]: time="2018-08-20T15:57:07.257682803+08:00" level=info msg="Graph migration to content-addressability took 0.00 seconds"
Aug 20 15:57:07 server86 dockerd[2955]: time="2018-08-20T15:57:07.260865731+08:00" level=info msg="Loading containers: start."
Aug 20 15:57:07 server86 dockerd[2955]: time="2018-08-20T15:57:07.603658334+08:00" level=info msg="Default bridge (docker0) is assigned with an IP address 17...IP address"
Aug 20 15:57:07 server86 dockerd[2955]: time="2018-08-20T15:57:07.763307367+08:00" level=info msg="Loading containers: done."
Aug 20 15:57:07 server86 dockerd[2955]: time="2018-08-20T15:57:07.812802202+08:00" level=info msg="Docker daemon" commit=0520e24 graphdriver(s)=devicemapper ...=18.03.0-ce
Aug 20 15:57:07 server86 dockerd[2955]: time="2018-08-20T15:57:07.813732684+08:00" level=info msg="Daemon has completed initialization"
Aug 20 15:57:07 server86 dockerd[2955]: time="2018-08-20T15:57:07.866979598+08:00" level=info msg="API listen on /var/run/docker.sock"
Aug 20 15:57:07 server86 systemd[1]: Started Docker Application Container Engine.
Hint: Some lines were ellipsized, use -l to show in full.
[root@server86 install_k8s_node]#
[root@server86 install_k8s_node]# ls
nodefile Step1_config.sh Step2_install_docker.sh Step3_install_kubelet.sh Step4_install_proxy.sh Step5_node_approve_csr.sh Step6_master_node_context.sh
[root@server86 install_k8s_node]#
[root@server86 install_k8s_node]# ./Step3_install_kubelet.sh
MASTER_IP=172.16.5.81
cat: /opt/ETCD_CLUSER_INFO: No such file or directory
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
● kubelet.service - Kubernetes Kubelet Server
Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2018-08-20 15:57:15 HKT; 142ms ago
Docs: https://github.com/GoogleCloudPlatform/kubernetes
Main PID: 3195 (kubelet)
Memory: 5.8M
CGroup: /system.slice/kubelet.service
└─3195 /usr/bin/kubelet --logtostderr=true --v=0 --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --hostname-override=172.16.5.86 --pod-infra-container-image=...
Aug 20 15:57:15 server86 systemd[1]: Started Kubernetes Kubelet Server.
Aug 20 15:57:15 server86 systemd[1]: Starting Kubernetes Kubelet Server...
[root@server86 install_k8s_node]#
[root@server86 install_k8s_node]# ./Step4_install_proxy.sh
Created symlink from /etc/systemd/system/default.target.wants/kube-proxy.service to /usr/lib/systemd/system/kube-proxy.service.
● kube-proxy.service - Kube Proxy Service
Loaded: loaded (/usr/lib/systemd/system/kube-proxy.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2018-08-20 15:57:19 HKT; 97ms ago
Main PID: 3282 (kube-proxy)
Memory: 5.5M
CGroup: /system.slice/kube-proxy.service
└─3282 /usr/bin/kube-proxy --logtostderr=true --v=0 --master=http://172.16.5.81:8080 --kubeconfig=/etc/kubernetes/kube-proxy.kubeconfig --cluster-cidr=10.1.0...
Aug 20 15:57:19 server86 systemd[1]: Started Kube Proxy Service.
Aug 20 15:57:19 server86 systemd[1]: Starting Kube Proxy Service...
[root@server86 install_k8s_node]#
[root@server86 install_k8s_node]#
使用脚本快速部署Server87服务器
[root@server87 ~]# cd /opt/
[root@server87 opt]# ls
install_etcd_cluster install_kubernetes rh
[root@server87 opt]# cd install_kubernetes/
[root@server87 install_kubernetes]# ls
check_etcd install_Calico install_CoreDNS install_k8s_master install_k8s_node install_kubernetes_software install_RAS_node MASTER_INFO reademe.txt
[root@server87 install_kubernetes]# cd install_k8s_node/
[root@server87 install_k8s_node]# ls
nodefile Step1_config.sh Step2_install_docker.sh Step3_install_kubelet.sh Step4_install_proxy.sh Step5_node_approve_csr.sh Step6_master_node_context.sh
[root@server87 install_k8s_node]# ./Step1_config.sh
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
Active: inactive (dead)
SELinux status: disabled
‘/opt/install_kubernetes/install_k8s_node/../install_kubernetes_software/kubectl’ -> ‘/usr/bin/kubectl’
‘/opt/install_kubernetes/install_k8s_node/../install_kubernetes_software/kubelet’ -> ‘/usr/bin/kubelet’
‘/opt/install_kubernetes/install_k8s_node/../install_kubernetes_software/kube-proxy’ -> ‘/usr/bin/kube-proxy’
[root@server87 install_k8s_node]# ./Step2_install_docker.sh
Loaded plugins: fastestmirror, langpacks
Examining /opt/install_kubernetes/install_k8s_node/nodefile/docker/docker-ce-18.03.0.ce-1.el7.centos.x86_64.rpm: docker-ce-18.03.0.ce-1.el7.centos.x86_64
Marking /opt/install_kubernetes/install_k8s_node/nodefile/docker/docker-ce-18.03.0.ce-1.el7.centos.x86_64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package docker-ce.x86_64 0:18.03.0.ce-1.el7.centos will be installed
--> Processing Dependency: container-selinux >= 2.9 for package: docker-ce-18.03.0.ce-1.el7.centos.x86_64
Loading mirror speeds from cached hostfile
* base: mirrors.aliyun.com
* epel: mirrors.tongji.edu.cn
* extras: mirrors.aliyun.com
* updates: mirrors.163.com
--> Processing Dependency: libseccomp >= 2.3 for package: docker-ce-18.03.0.ce-1.el7.centos.x86_64
--> Processing Dependency: pigz for package: docker-ce-18.03.0.ce-1.el7.centos.x86_64
--> Running transaction check
---> Package container-selinux.noarch 2:2.66-1.el7 will be installed
--> Processing Dependency: selinux-policy-targeted >= 3.13.1-192 for package: 2:container-selinux-2.66-1.el7.noarch
--> Processing Dependency: selinux-policy-base >= 3.13.1-192 for package: 2:container-selinux-2.66-1.el7.noarch
--> Processing Dependency: selinux-policy >= 3.13.1-192 for package: 2:container-selinux-2.66-1.el7.noarch
--> Processing Dependency: policycoreutils >= 2.5-11 for package: 2:container-selinux-2.66-1.el7.noarch
---> Package libseccomp.x86_64 0:2.2.1-1.el7 will be updated
---> Package libseccomp.x86_64 0:2.3.1-3.el7 will be an update
---> Package pigz.x86_64 0:2.3.4-1.el7 will be installed
--> Running transaction check
---> Package policycoreutils.x86_64 0:2.2.5-20.el7 will be updated
--> Processing Dependency: policycoreutils = 2.2.5-20.el7 for package: policycoreutils-python-2.2.5-20.el7.x86_64
---> Package policycoreutils.x86_64 0:2.5-22.el7 will be an update
--> Processing Dependency: libsepol >= 2.5-8 for package: policycoreutils-2.5-22.el7.x86_64
--> Processing Dependency: libselinux-utils >= 2.5-12 for package: policycoreutils-2.5-22.el7.x86_64
--> Processing Dependency: libsepol.so.1(LIBSEPOL_1.1)(64bit) for package: policycoreutils-2.5-22.el7.x86_64
--> Processing Dependency: libsepol.so.1(LIBSEPOL_1.0)(64bit) for package: policycoreutils-2.5-22.el7.x86_64
--> Processing Dependency: libsemanage.so.1(LIBSEMANAGE_1.1)(64bit) for package: policycoreutils-2.5-22.el7.x86_64
---> Package selinux-policy.noarch 0:3.13.1-60.el7 will be updated
---> Package selinux-policy.noarch 0:3.13.1-192.el7_5.4 will be an update
---> Package selinux-policy-targeted.noarch 0:3.13.1-60.el7 will be updated
---> Package selinux-policy-targeted.noarch 0:3.13.1-192.el7_5.4 will be an update
--> Running transaction check
---> Package libselinux-utils.x86_64 0:2.2.2-6.el7 will be updated
---> Package libselinux-utils.x86_64 0:2.5-12.el7 will be an update
--> Processing Dependency: libselinux(x86-64) = 2.5-12.el7 for package: libselinux-utils-2.5-12.el7.x86_64
---> Package libsemanage.x86_64 0:2.1.10-18.el7 will be updated
--> Processing Dependency: libsemanage = 2.1.10-18.el7 for package: libsemanage-python-2.1.10-18.el7.x86_64
---> Package libsemanage.x86_64 0:2.5-11.el7 will be an update
---> Package libsepol.x86_64 0:2.1.9-3.el7 will be updated
---> Package libsepol.x86_64 0:2.5-8.1.el7 will be an update
---> Package policycoreutils-python.x86_64 0:2.2.5-20.el7 will be updated
---> Package policycoreutils-python.x86_64 0:2.5-22.el7 will be an update
--> Processing Dependency: setools-libs >= 3.3.8-2 for package: policycoreutils-python-2.5-22.el7.x86_64
--> Running transaction check
---> Package libselinux.x86_64 0:2.2.2-6.el7 will be updated
--> Processing Dependency: libselinux = 2.2.2-6.el7 for package: libselinux-python-2.2.2-6.el7.x86_64
---> Package libselinux.x86_64 0:2.5-12.el7 will be an update
---> Package libsemanage-python.x86_64 0:2.1.10-18.el7 will be updated
---> Package libsemanage-python.x86_64 0:2.5-11.el7 will be an update
---> Package setools-libs.x86_64 0:3.3.7-46.el7 will be updated
---> Package setools-libs.x86_64 0:3.3.8-2.el7 will be an update
--> Running transaction check
---> Package libselinux-python.x86_64 0:2.2.2-6.el7 will be updated
---> Package libselinux-python.x86_64 0:2.5-12.el7 will be an update
--> Processing Conflict: libselinux-2.5-12.el7.x86_64 conflicts systemd < 219-20
--> Restarting Dependency Resolution with new changes.
--> Running transaction check
---> Package systemd.x86_64 0:219-19.el7 will be updated
--> Processing Dependency: systemd = 219-19.el7 for package: systemd-python-219-19.el7.x86_64
--> Processing Dependency: systemd = 219-19.el7 for package: systemd-sysv-219-19.el7.x86_64
---> Package systemd.x86_64 0:219-57.el7 will be an update
--> Processing Dependency: systemd-libs = 219-57.el7 for package: systemd-219-57.el7.x86_64
--> Processing Dependency: liblz4.so.1()(64bit) for package: systemd-219-57.el7.x86_64
--> Running transaction check
---> Package lz4.x86_64 0:1.7.5-2.el7 will be installed
---> Package systemd-libs.x86_64 0:219-19.el7 will be updated
--> Processing Dependency: systemd-libs = 219-19.el7 for package: libgudev1-219-19.el7.x86_64
---> Package systemd-libs.x86_64 0:219-57.el7 will be an update
---> Package systemd-python.x86_64 0:219-19.el7 will be updated
---> Package systemd-python.x86_64 0:219-57.el7 will be an update
---> Package systemd-sysv.x86_64 0:219-19.el7 will be updated
---> Package systemd-sysv.x86_64 0:219-57.el7 will be an update
--> Running transaction check
---> Package libgudev1.x86_64 0:219-19.el7 will be updated
---> Package libgudev1.x86_64 0:219-57.el7 will be an update
--> Finished Dependency Resolution
Dependencies Resolved
===========================================================================================================================================================================
Package Arch Version Repository Size
===========================================================================================================================================================================
Installing:
docker-ce x86_64 18.03.0.ce-1.el7.centos /docker-ce-18.03.0.ce-1.el7.centos.x86_64 151 M
Updating:
systemd x86_64 219-57.el7 base 5.0 M
Installing for dependencies:
container-selinux noarch 2:2.66-1.el7 extras 35 k
lz4 x86_64 1.7.5-2.el7 base 98 k
pigz x86_64 2.3.4-1.el7 epel 81 k
Updating for dependencies:
libgudev1 x86_64 219-57.el7 base 92 k
libseccomp x86_64 2.3.1-3.el7 base 56 k
libselinux x86_64 2.5-12.el7 base 162 k
libselinux-python x86_64 2.5-12.el7 base 235 k
libselinux-utils x86_64 2.5-12.el7 base 151 k
libsemanage x86_64 2.5-11.el7 base 150 k
libsemanage-python x86_64 2.5-11.el7 base 112 k
libsepol x86_64 2.5-8.1.el7 base 297 k
policycoreutils x86_64 2.5-22.el7 base 867 k
policycoreutils-python x86_64 2.5-22.el7 base 454 k
selinux-policy noarch 3.13.1-192.el7_5.4 updates 453 k
selinux-policy-targeted noarch 3.13.1-192.el7_5.4 updates 6.6 M
setools-libs x86_64 3.3.8-2.el7 base 619 k
systemd-libs x86_64 219-57.el7 base 402 k
systemd-python x86_64 219-57.el7 base 128 k
systemd-sysv x86_64 219-57.el7 base 79 k
Transaction Summary
===========================================================================================================================================================================
Install 1 Package (+ 3 Dependent packages)
Upgrade 1 Package (+16 Dependent packages)
Total size: 166 M
Total download size: 16 M
Downloading packages:
No Presto metadata available for base
updates/7/x86_64/prestodelta | 420 kB 00:00:01
(1/19): libselinux-2.5-12.el7.x86_64.rpm | 162 kB 00:00:00
(2/19): libselinux-utils-2.5-12.el7.x86_64.rpm | 151 kB 00:00:00
(3/19): libsemanage-2.5-11.el7.x86_64.rpm | 150 kB 00:00:00
(4/19): libgudev1-219-57.el7.x86_64.rpm | 92 kB 00:00:00
(5/19): libsemanage-python-2.5-11.el7.x86_64.rpm | 112 kB 00:00:00
(6/19): libsepol-2.5-8.1.el7.x86_64.rpm | 297 kB 00:00:00
(7/19): lz4-1.7.5-2.el7.x86_64.rpm | 98 kB 00:00:00
(8/19): libselinux-python-2.5-12.el7.x86_64.rpm | 235 kB 00:00:00
(9/19): selinux-policy-3.13.1-192.el7_5.4.noarch.rpm | 453 kB 00:00:00
(10/19): policycoreutils-python-2.5-22.el7.x86_64.rpm | 454 kB 00:00:00
(11/19): setools-libs-3.3.8-2.el7.x86_64.rpm | 619 kB 00:00:00
(12/19): systemd-219-57.el7.x86_64.rpm | 5.0 MB 00:00:00
(13/19): container-selinux-2.66-1.el7.noarch.rpm | 35 kB 00:00:01
(14/19): systemd-libs-219-57.el7.x86_64.rpm | 402 kB 00:00:00
(15/19): systemd-sysv-219-57.el7.x86_64.rpm | 79 kB 00:00:00
(16/19): selinux-policy-targeted-3.13.1-192.el7_5.4.noarch.rpm | 6.6 MB 00:00:01
(17/19): systemd-python-219-57.el7.x86_64.rpm | 128 kB 00:00:00
(18/19): pigz-2.3.4-1.el7.x86_64.rpm | 81 kB 00:00:01
(19/19): policycoreutils-2.5-22.el7.x86_64.rpm | 867 kB 00:00:01
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 7.4 MB/s | 16 MB 00:00:02
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Updating : libsepol-2.5-8.1.el7.x86_64 1/38
Updating : libselinux-2.5-12.el7.x86_64 2/38
Updating : libsemanage-2.5-11.el7.x86_64 3/38
Installing : lz4-1.7.5-2.el7.x86_64 4/38
Updating : systemd-libs-219-57.el7.x86_64 5/38
Updating : systemd-219-57.el7.x86_64 6/38
Updating : libselinux-utils-2.5-12.el7.x86_64 7/38
Updating : policycoreutils-2.5-22.el7.x86_64 8/38
Updating : selinux-policy-3.13.1-192.el7_5.4.noarch 9/38
Updating : selinux-policy-targeted-3.13.1-192.el7_5.4.noarch 10/38
Updating : libsemanage-python-2.5-11.el7.x86_64 11/38
Updating : libselinux-python-2.5-12.el7.x86_64 12/38
Updating : setools-libs-3.3.8-2.el7.x86_64 13/38
Updating : policycoreutils-python-2.5-22.el7.x86_64 14/38
Installing : 2:container-selinux-2.66-1.el7.noarch 15/38
setsebool: SELinux is disabled.
Installing : pigz-2.3.4-1.el7.x86_64 16/38
Updating : libseccomp-2.3.1-3.el7.x86_64 17/38
Installing : docker-ce-18.03.0.ce-1.el7.centos.x86_64 18/38
Updating : systemd-sysv-219-57.el7.x86_64 19/38
Updating : systemd-python-219-57.el7.x86_64 20/38
Updating : libgudev1-219-57.el7.x86_64 21/38
Cleanup : selinux-policy-targeted-3.13.1-60.el7.noarch 22/38
Cleanup : policycoreutils-python-2.2.5-20.el7.x86_64 23/38
Cleanup : selinux-policy-3.13.1-60.el7.noarch 24/38
Cleanup : systemd-sysv-219-19.el7.x86_64 25/38
Cleanup : policycoreutils-2.2.5-20.el7.x86_64 26/38
Cleanup : systemd-python-219-19.el7.x86_64 27/38
Cleanup : systemd-219-19.el7.x86_64 28/38
Cleanup : setools-libs-3.3.7-46.el7.x86_64 29/38
Cleanup : libselinux-utils-2.2.2-6.el7.x86_64 30/38
Cleanup : libselinux-python-2.2.2-6.el7.x86_64 31/38
Cleanup : libsemanage-python-2.1.10-18.el7.x86_64 32/38
Cleanup : libsemanage-2.1.10-18.el7.x86_64 33/38
Cleanup : libgudev1-219-19.el7.x86_64 34/38
Cleanup : systemd-libs-219-19.el7.x86_64 35/38
Cleanup : libselinux-2.2.2-6.el7.x86_64 36/38
Cleanup : libsepol-2.1.9-3.el7.x86_64 37/38
Cleanup : libseccomp-2.2.1-1.el7.x86_64 38/38
Verifying : libsemanage-python-2.5-11.el7.x86_64 1/38
Verifying : libsemanage-2.5-11.el7.x86_64 2/38
Verifying : libselinux-python-2.5-12.el7.x86_64 3/38
Verifying : selinux-policy-3.13.1-192.el7_5.4.noarch 4/38
Verifying : setools-libs-3.3.8-2.el7.x86_64 5/38
Verifying : libseccomp-2.3.1-3.el7.x86_64 6/38
Verifying : policycoreutils-2.5-22.el7.x86_64 7/38
Verifying : selinux-policy-targeted-3.13.1-192.el7_5.4.noarch 8/38
Verifying : pigz-2.3.4-1.el7.x86_64 9/38
Verifying : policycoreutils-python-2.5-22.el7.x86_64 10/38
Verifying : libgudev1-219-57.el7.x86_64 11/38
Verifying : 2:container-selinux-2.66-1.el7.noarch 12/38
Verifying : systemd-sysv-219-57.el7.x86_64 13/38
Verifying : lz4-1.7.5-2.el7.x86_64 14/38
Verifying : systemd-219-57.el7.x86_64 15/38
Verifying : libsepol-2.5-8.1.el7.x86_64 16/38
Verifying : systemd-libs-219-57.el7.x86_64 17/38
Verifying : libselinux-2.5-12.el7.x86_64 18/38
Verifying : docker-ce-18.03.0.ce-1.el7.centos.x86_64 19/38
Verifying : libselinux-utils-2.5-12.el7.x86_64 20/38
Verifying : systemd-python-219-57.el7.x86_64 21/38
Verifying : libsemanage-python-2.1.10-18.el7.x86_64 22/38
Verifying : selinux-policy-targeted-3.13.1-60.el7.noarch 23/38
Verifying : setools-libs-3.3.7-46.el7.x86_64 24/38
Verifying : libsemanage-2.1.10-18.el7.x86_64 25/38
Verifying : systemd-sysv-219-19.el7.x86_64 26/38
Verifying : libgudev1-219-19.el7.x86_64 27/38
Verifying : systemd-219-19.el7.x86_64 28/38
Verifying : selinux-policy-3.13.1-60.el7.noarch 29/38
Verifying : systemd-libs-219-19.el7.x86_64 30/38
Verifying : libselinux-utils-2.2.2-6.el7.x86_64 31/38
Verifying : libseccomp-2.2.1-1.el7.x86_64 32/38
Verifying : libsepol-2.1.9-3.el7.x86_64 33/38
Verifying : libselinux-python-2.2.2-6.el7.x86_64 34/38
Verifying : policycoreutils-2.2.5-20.el7.x86_64 35/38
Verifying : systemd-python-219-19.el7.x86_64 36/38
Verifying : libselinux-2.2.2-6.el7.x86_64 37/38
Verifying : policycoreutils-python-2.2.5-20.el7.x86_64 38/38
Installed:
docker-ce.x86_64 0:18.03.0.ce-1.el7.centos
Dependency Installed:
container-selinux.noarch 2:2.66-1.el7 lz4.x86_64 0:1.7.5-2.el7 pigz.x86_64 0:2.3.4-1.el7
Updated:
systemd.x86_64 0:219-57.el7
Dependency Updated:
libgudev1.x86_64 0:219-57.el7 libseccomp.x86_64 0:2.3.1-3.el7 libselinux.x86_64 0:2.5-12.el7
libselinux-python.x86_64 0:2.5-12.el7 libselinux-utils.x86_64 0:2.5-12.el7 libsemanage.x86_64 0:2.5-11.el7
libsemanage-python.x86_64 0:2.5-11.el7 libsepol.x86_64 0:2.5-8.1.el7 policycoreutils.x86_64 0:2.5-22.el7
policycoreutils-python.x86_64 0:2.5-22.el7 selinux-policy.noarch 0:3.13.1-192.el7_5.4 selinux-policy-targeted.noarch 0:3.13.1-192.el7_5.4
setools-libs.x86_64 0:3.3.8-2.el7 systemd-libs.x86_64 0:219-57.el7 systemd-python.x86_64 0:219-57.el7
systemd-sysv.x86_64 0:219-57.el7
Complete!
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2018-08-20 15:51:50 HKT; 9ms ago
Docs: https://docs.docker.com
Main PID: 42077 (dockerd)
Memory: 40.8M
CGroup: /system.slice/docker.service
├─42077 /usr/bin/dockerd
└─42086 docker-containerd --config /var/run/docker/containerd/containerd.toml
Aug 20 15:51:50 server87 dockerd[42077]: time="2018-08-20T15:51:50.337814778+08:00" level=info msg="devmapper: Successfully created filesystem xfs on device d...5123-base"
Aug 20 15:51:50 server87 dockerd[42077]: time="2018-08-20T15:51:50.463516508+08:00" level=info msg="Graph migration to content-addressability took 0.00 seconds"
Aug 20 15:51:50 server87 dockerd[42077]: time="2018-08-20T15:51:50.463782799+08:00" level=warning msg="mountpoint for pids not found"
Aug 20 15:51:50 server87 dockerd[42077]: time="2018-08-20T15:51:50.464461343+08:00" level=info msg="Loading containers: start."
Aug 20 15:51:50 server87 dockerd[42077]: time="2018-08-20T15:51:50.601643093+08:00" level=info msg="Default bridge (docker0) is assigned with an IP address 17...P address"
Aug 20 15:51:50 server87 dockerd[42077]: time="2018-08-20T15:51:50.677859724+08:00" level=info msg="Loading containers: done."
Aug 20 15:51:50 server87 dockerd[42077]: time="2018-08-20T15:51:50.696315433+08:00" level=info msg="Docker daemon" commit=0520e24 graphdriver(s)=devicemapper ...18.03.0-ce
Aug 20 15:51:50 server87 dockerd[42077]: time="2018-08-20T15:51:50.696473183+08:00" level=info msg="Daemon has completed initialization"
Aug 20 15:51:50 server87 systemd[1]: Started Docker Application Container Engine.
Aug 20 15:51:50 server87 dockerd[42077]: time="2018-08-20T15:51:50.714102886+08:00" level=info msg="API listen on /var/run/docker.sock"
Hint: Some lines were ellipsized, use -l to show in full.
[root@server87 install_k8s_node]# ls
nodefile Step1_config.sh Step2_install_docker.sh Step3_install_kubelet.sh Step4_install_proxy.sh Step5_node_approve_csr.sh Step6_master_node_context.sh
[root@server87 install_k8s_node]# ./Step3_install_kubelet.sh
MASTER_IP=172.16.5.81
cat: /opt/ETCD_CLUSER_INFO: No such file or directory
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
● kubelet.service - Kubernetes Kubelet Server
Loaded: loaded (/usr/lib/systemd/system/kubelet.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2018-08-20 15:52:13 HKT; 46ms ago
Docs: https://github.com/GoogleCloudPlatform/kubernetes
Main PID: 42486 (kubelet)
Memory: 6.4M
CGroup: /system.slice/kubelet.service
└─42486 /usr/bin/kubelet --logtostderr=true --v=0 --kubeconfig=/etc/kubernetes/kubelet.kubeconfig --hostname-override=172.16.5.87 --pod-infra-container-image...
Aug 20 15:52:13 server87 systemd[1]: Started Kubernetes Kubelet Server.
Aug 20 15:52:13 server87 systemd[1]: Starting Kubernetes Kubelet Server...
[root@server87 install_k8s_node]#
[root@server87 install_k8s_node]# ./Step4_install_proxy.sh
Created symlink from /etc/systemd/system/default.target.wants/kube-proxy.service to /usr/lib/systemd/system/kube-proxy.service.
● kube-proxy.service - Kube Proxy Service
Loaded: loaded (/usr/lib/systemd/system/kube-proxy.service; enabled; vendor preset: disabled)
Active: active (running) since Mon 2018-08-20 15:52:18 HKT; 38ms ago
Main PID: 42814 (kube-proxy)
Memory: 5.8M
CGroup: /system.slice/kube-proxy.service
└─42814 /usr/bin/kube-proxy --logtostderr=true --v=0 --master=http://172.16.5.81:8080 --kubeconfig=/etc/kubernetes/kube-proxy.kubeconfig --cluster-cidr=10.1....
Aug 20 15:52:18 server87 systemd[1]: Started Kube Proxy Service.
Aug 20 15:52:18 server87 systemd[1]: Starting Kube Proxy Service...
[root@server87 install_k8s_node]#
回到Master服务器认证通过Server86、87的kubelet服务csr请求
[root@server81 opt]# kubectl get csr
NAME AGE REQUESTOR CONDITION
node-csr-1behv8FXfoDXo6SLgRlwtJ7EwOnMMqIoo7c5YI4q0Yc 1m kubelet-bootstrap Pending
node-csr-fH4Ct4Fg4TgzFV0dP-SlfVCtTo9XNCJjajzPohDVxHE 50m kubelet-bootstrap Approved,Issued
node-csr-tO2dsRk01-qNWJkeDYARuIkeV24QsX2M8txYmkXs96M 6m kubelet-bootstrap Pending
[root@server81 opt]#
[root@server81 opt]# kubectl certificate approve node-csr-1behv8FXfoDXo6SLgRlwtJ7EwOnMMqIoo7c5YI4q0Yc
certificatesigningrequest.certificates.k8s.io/node-csr-1behv8FXfoDXo6SLgRlwtJ7EwOnMMqIoo7c5YI4q0Yc approved
[root@server81 opt]#
[root@server81 opt]# kubectl certificate approve node-csr-tO2dsRk01-qNWJkeDYARuIkeV24QsX2M8txYmkXs96M
certificatesigningrequest.certificates.k8s.io/node-csr-tO2dsRk01-qNWJkeDYARuIkeV24QsX2M8txYmkXs96M approved
[root@server81 opt]#
[root@server81 opt]# kubectl get csr
NAME AGE REQUESTOR CONDITION
node-csr-1behv8FXfoDXo6SLgRlwtJ7EwOnMMqIoo7c5YI4q0Yc 1m kubelet-bootstrap Approved,Issued
node-csr-fH4Ct4Fg4TgzFV0dP-SlfVCtTo9XNCJjajzPohDVxHE 51m kubelet-bootstrap Approved,Issued
node-csr-tO2dsRk01-qNWJkeDYARuIkeV24QsX2M8txYmkXs96M 6m kubelet-bootstrap Approved,Issued
[root@server81 opt]#
[root@server81 opt]# kubectl get node
NAME STATUS ROLES AGE VERSION
172.16.5.81 NotReady <none> 44m v1.11.0
172.16.5.86 NotReady <none> 13s v1.11.0
172.16.5.87 NotReady <none> 6s v1.11.0
[root@server81 opt]#
部署到这里kubernetes的Node节点服务也部署完毕了，虽然这里是NotReady状态，但是只要部署Calico网络即可。
最后总结
综上所述，整体kubernetes启用RBAC的生成环境 二进制可执行文件的环境已部署完毕。
这里Node节点部署Calico网络的内容我就打算写在下一篇章了。
优化的方向
- 离线环境部署kubernetes环境
- 全自动部署项目
- 服务器集群外部组件的说明以及自动化部署
以上几点后续，有时间我可以陆续逐步写上来的，赞一下给我点动力吧。
kubernetes v1.11 二进制部署篇章目录
如果你想要看我写的总体系列文章目录介绍，可以点击kuberntes以及运维开发文章目录介绍
网友评论