KubeSphere安装KubeEdge

作者: beyond阿亮 | 来源:发表于2022-05-07 22:13 被阅读0次

KubeSphere安装KubeEdge

KubeSphere安装

linux安装KubeSphere

Linux安装kubeedge_亲测成功

kubeedge边缘节点安装

配置master节点

[图片上传失败...(image-292706-1651932824363)]

控制台->平台管理->集群管理->自定义CRD，搜索clusterconfiguration，查看详情，在资源列表中，点击 ks-installer 右侧的图标，然后选择编辑配置文件。

将kubeedge.enabled的false改为true

kubeedge:
  enabled: true # 将“false”更改为“true”。

将kubeedge.cloudCore.cloudHub.advertiseAddress 的值设置为集群的公共 IP 地址或边缘节点可以访问的 IP 地址。完成后，点击右下角的更新保存配置。

  kubeedge:
    cloudCore:
      cloudHub:
        advertiseAddress:
          - 172.16.0.22

查看安装过程

kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f

kubectl get node

kubectl top node

安装完成后，web页面上控制台->平台管理->集群管理->节点管理目录下，会出现边缘节点
[图片上传失败...(image-5765e6-1651932824363)]

配置边缘节点

边缘节点安装Docker-ce 19.03，镜像加速地址去自己的阿里云帐号申请

yum install docker-ce-19.03.15-3.el7  docker-ce-cli-19.03.15-3.el7 -y

cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": ["native.cgroupdriver=cgroupfs"],
  "registry-mirrors": ["https://******.mirror.aliyuncs.com"]
}
EOF

systemctl enable docker && systemctl restart docker

配置 EdgeMesh

编辑 /etc/nsswitch.conf

vi /etc/nsswitch.conf

修改hosts

hosts:          dns files mdns4_minimal [NOTFOUND=return]

运行以下命令启用 IP 转发：

echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf

检查验证

[root@edgenode-01 ~]# sysctl -p | grep ip_forward
net.ipv4.ip_forward = 1

添加边缘节点

在控制台->平台管理->集群管理->节点管理->边缘节点，点击添加节点，设置边缘节点的节点名称并输入其内网 IP 地址，添加污点一定要打勾，方便后续调度，点击验证生成命令

arch=$(uname -m); curl -LO https://kubeedge.pek3b.qingstor.com/bin/v1.6.2/$arch/keadm-v1.6.2-linux-$arch.tar.gz  && tar xvf keadm-v1.6.2-linux-$arch.tar.gz && chmod +x keadm && ./keadm join --kubeedge-version=1.6.2 --region=zh --cloudcore-ipport=192.168.0.10:10000 --quicport 10001 --certport 10002 --tunnelport 10004 --edgenode-name edgenode-vwuk --edgenode-ip 192.168.0.25 --token 805ca3c73ce17f447764d13c8bf95a73a683aa63674f760020e4da2c37a58667.eyJhbGciOinR5cCI6IkpXVCJ9.eyJleHAiOjE2MzAxMjI4NzR9.VVLBlOiSjjjLJjLSyz6dOcNl2A --with-edge-taint

如果使用的是云服务器搭建，那么添加防火墙和端口转发规则，需要将外网端口映射到相应的内网 IP 地址（主节点）和端口以及创建防火墙规则以允许流量进入这些端口（10000 至 10004）。添加完，直接将命令复制到边缘节点进行执行。

[图片上传失败...(image-433b8-1651932824363)]

因为我用的是内网搭建，所以需要将命令里的端口都改为（30000 至 30004），改完后执行

arch=$(uname -m); curl -LO https://kubeedge.pek3b.qingstor.com/bin/v1.6.2/$arch/keadm-v1.6.2-linux-$arch.tar.gz  && tar xvf keadm-v1.6.2-linux-$arch.tar.gz && chmod +x keadm && ./keadm join --kubeedge-version=1.6.2 --region=zh --cloudcore-ipport=192.168.0.100:30000 --quicport 30001 --certport 30002 --tunnelport 30004 --edgenode-name edgenode-vwuk --edgenode-ip 192.168.0.25 --token 805ca3c73ce178bf960020e4da2c37a58667.eyJhbGciOiJIUzI1pXVCJ9.eyJleHAiOjE2MzAxMjI4NzR9.VVLBlOiSeVssi5gHmiHR6dOcNl2A --with-edge-taint

成功执行，显示如下

[Run as service] service file already exisits in /etc/kubeedge//edgecore.service, skip download
kubeedge-v1.6.2-linux-amd64/
kubeedge-v1.6.2-linux-amd64/edge/
kubeedge-v1.6.2-linux-amd64/edge/edgecore
kubeedge-v1.6.2-linux-amd64/cloud/
kubeedge-v1.6.2-linux-amd64/cloud/csidriver/
kubeedge-v1.6.2-linux-amd64/cloud/csidriver/csidriver
kubeedge-v1.6.2-linux-amd64/cloud/admission/
kubeedge-v1.6.2-linux-amd64/cloud/admission/admission
kubeedge-v1.6.2-linux-amd64/cloud/cloudcore/
kubeedge-v1.6.2-linux-amd64/cloud/cloudcore/cloudcore
kubeedge-v1.6.2-linux-amd64/version

KubeEdge edgecore is running, For logs visit: journalctl -u edgecore.service -b

此时，刷新边缘节点页面，会显示新添加的边缘节点，若没有出现，查看日志找错误信息

journalctl -u edgecore.service -f

[图片上传失败...(image-b3776d-1651932824363)]

移除边缘节点

边缘节点上执行

./keadm reset
yum remove mosquitto
rm -rf /var/lib/kubeedge /var/lib/edged /etc/kubeedge/ca /etc/kubeedge/certs

然后去master节点，移除边缘节点

kubectl delete node <edgenode-name>

在边缘节点运行pod

在master编写pod的yaml，因为边缘节点上有污点node-role.kubernetes.io/edge，所以需要设置pod的容忍度

apiVersion: v1
kind: Pod
metadata:
  name: nginx-edge
spec:
  containers:
  - name: nginx
    image: nginx
    ports:
    - containerPort: 80
      hostPort: 80
  tolerations:
  - key: "node-role.kubernetes.io/edge"
    operator: "Exists"
    effect: "NoSchedule"

部署

kubectl apply -y nginx-edge.yaml

可以看到，pod已经被调度到边缘节点edgenode-jkfy上

[root@k8s-master01 yaml]# kubectl get pod nginx-edge -o wide
NAME         READY   STATUS    RESTARTS   AGE   IP           NODE            NOMINATED NODE   READINESS GATES
nginx-edge   1/1     Running   0          12m   172.17.0.2   edgenode-jkfy   <none>           <none>

云端执行了相关命令

kubeedge version
metrics-server version
kubectl -n kubeedge edit cm edge-watcher

kubectl get token
kubectl get secret -nkubeedge tokensecret -o=jsonpath='{.data.tokendata}' | base64 -d
kubectl get no
iptables -t nat -nL | grep 10350
kubectl get pod -n kubeedge -owide
kubectl -n kubeedge edit cm edge-watcher
kubectl get pod -A
kubectl describe pod cloudcore-768b88bc4-5xscz -n kubeedge
kubectl -n kubeedge edit cm cloudcore

kubectl get pods -n kubeedge -o wide
iptables -t nat -nL|grep 
iptables -t nat -nL|grep 10350
kubectl get deployment -n kubeedge

kubectl rollout restart deployment edge-watcher-controller-manager
kubectl rollout restart deployment edge-watcher-controller-manager -n kubeedge
kubectl -n kubeedge edit cm tunnelport

vi kubelet-use-node-status-port
vi metrics-server.yaml
kubectl get --raw "/apis/metrics/k8s.io/v1beta1/nodes"

curl -ik https://172.16.0.100/stats/summary
kubectl get pod -n kubeedge
kubectl get pod -A
kubectl describe pod edge-watcher-controller-manager-5bc88bcd48-z89gp -n kubeedge

iptables -t nat -nL | grep 10350
kubectl -n kubeedge edit cm edge-watcher
iptables -t nat -nL | grep 10350
kubectl -n kubeedge edit cm edge-watcher
kubectl rollout restart deployment edge-watcher-controller-manager -n kubeedge
iptables -t nat -nL | grep 10350
kubectl get pods -n kubeedge -o wide
iptables -t nat -nL | grep 10350

kubectl get node
kubectl top node

helm安装

helm uninstall kubeedge -n kubeedge
yum install helm

mkdir helm
cd helm

wegt https://get.helm.sh/helm-v3.8.1-linux-amd64.tar.gz
yum install wget
wget https://get.helm.sh/helm-v3.8.1-linux-amd64.tar.gz

tar -zxvf helm-v3.8.1-linux-amd64.tar.gz
cd linux-amd64/

chmod a+x /usr/local/bin/helm
chmod a+x /root/helm
./helm version

kubectl get pod -A

kubectl get pod -A

kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f

kubectl delete -f cluster-configuration.yaml

kubectl delete -f kubesphere-installer.yaml

kubectl get pod -A

kubeadm reset

kubeadm init --apiserver-advertise-address=172.16.0.22 --control-plane-endpoint=q-master --image-repository registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images --kubernetes-version v1.20.9 --service-cidr=10.96.0.0/16 --pod-network-cidr=192.168.0.0/16

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl apply -f calico.yaml

边缘端执行了相关命令

#安装docker
#  preinstall utils 
sudo yum install -y yum-utils   device-mapper-persistent-data   lvm2
# add repository
sudo yum-config-manager     --add-repo     https://download.docker.com/linux/centos/docker-ce.repo
# make cache
sudo yum makecache fast
# install the latest stable version of docker
sudo yum install -y docker-ce
# start deamon and enable auto start when power on
sudo systemctl start docker
sudo systemctl enable docker

# add current user 
sudo groupadd docker
sudo gpasswd -a ${USER} docker
sudo systemctl restart docker
docker version

vi /etc/nsswitch.conf
sudo echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf
sudo sysctl -p | grep ip_forward
vim /etc/sysctl.conf

arch=$(uname -m); curl -LO https://kubeedge.pek3b.qingstor.com/bin/v1.6.2/$arch/keadm-v1.6.2-linux-$arch.tar.gz  && tar xvf keadm-v1.6.2-linux-$arch.tar.gz && chmod +x keadm && ./keadm join --kubeedge-version=1.6.2 --region=zh --cloudcore-ipport=119.3.9.34:30000 --quicport 30001 --certport 30002 --tunnelport 30004 --edgenode-name edgenode-1 --edgenode-ip 172.16.0.100 --token ac18808a862915b68155b9d4876c843109190.eyJhbGciOInR5cCI6IkpXVCJ9.eyJleHAiOjE2NDk4Mzc1MDh9.Dlc2OAvJ_lSt_xAY_BgIZfx68

journalctl -u edgecore.service -b -e

systemctl stop edgecore

systemctl start edgecore

rm -rf /var/lib/kubeedge /var/lib/edged /etc/kubeedge
rm -rf /etc/systemd/system/edgecore.service
rm -rf /usr/local/bin/edgecore
systemctl restart docker

arch=$(uname -m); curl -LO https://kubeedge.pek3b.qingstor.com/bin/v1.6.2/$arch/keadm-v1.6.2-linux-$arch.tar.gz  && tar xvf keadm-v1.6.2-linux-$arch.tar.gz && chmod +x keadm && ./keadm join --kubeedge-version=1.6.2 --region=zh --cloudcore-ipport=119.3.9.34:10000 --quicport 10001 --certport 10002 --tunnelport 10004 --edgenode-name edgenode-1 --edgenode-ip 8.30.8.15 --token ac18808a862915b68e0d83f9d4876c843109190.eyJhbGciOiJIUzI1NI6IkpXVCJ9.eyJleHAiOjE2NDk4Mzc1MDh9.Dlc2OAvJ_lSt_xAY_BgIZ68

keadm join --cloudcore-ipport=119.3.9.34:10000 --token=ac18808a862915bc843109190.eyJhbGciOiI6IkpXVCJ9.eyJleHAiOjE2NDk4NDU1NTJ9.s_QkmzZhC0e2LJbdz6dOnPPbq_sw

arch=$(uname -m); curl -LO https://kubeedge.pek3b.qingstor.com/bin/v1.6.2/$arch/keadm-v1.6.2-linux-$arch.tar.gz  && tar xvf keadm-v1.6.2-linux-$arch.tar.gz && chmod +x keadm && ./keadm join --kubeedge-version=1.6.2 --region=zh --cloudcore-ipport=119.3.9.34:10000 --quicport 10001 --certport 10002 --tunnelport 10004 --edgenode-name edgenode-1 --edgenode-ip 172.16.0.100 --token ac18808a862915b68155b94f76c843109190.eyJhbGciOiJIUzkpXVCJ9.eyJleHAiOjE2NDk4NDU1NTJ9.s_QkmzZhC0e2LJbdND0OnPPbq_sw --with-edge-taint

参考链接:
https://blog.51cto.com/u_15072912/4350024

KubeSphere安装KubeEdge