使用RKE2部署集群(cilium网络插件)

Command Description
rke2 server Run the RKE2 management server, which will also launch the Kubernetes control plane components such as the API server, controller-manager, and scheduler. Only Supported on Linux.
rke2 agent Run the RKE2 node agent. This will cause RKE2 to run as a worker node, launching the Kubernetes node services kubelet and kube-proxy. Supported on Linux and Windows.

所有节点

 sysctl vm.overcommit_memory=1
 sysctl kernel.panic=10

K8S Master 节点操作

1. 安装 rke2 server，执行命令:

curl -sfL https://get.rke2.io | INSTALL_RKE2_VERSION=v1.20.9+rke2r2 sh -
systemctl enable rke2-server.service

可以从社区仓库 https://github.com/rancher/rke2/tags 来选择要安装的版本

2. 创建 rke2 server 配置

mkdir -p /etc/rancher/rke2
cat << EOF >  /etc/rancher/rke2/config.yaml
write-kubeconfig-mode: "0644"
cluster-cidr: 172.16.0.0/12
service-cidr: 192.168.0.0/16
service-node-port-range: 1-65535
selinux: false
tls-san:
  - "10.9.84.82"
cni:
  - cilium
disable:
  - rke2-canal
  - rke2-kube-proxy
  - rke2-ingress-nginx
disable-kube-proxy: true
EOF

参考 https://docs.rke2.io/install/install_options/server_config/

3. 启动 rke2-server 服务

systemctl start rke2-server.service

可以通过 journalctl -fu rke2-server.service 日志观察master节点部署状态，大
约需要3-5分钟完成初始化

5. 设置环境变量

echo 'PATH=$PATH:/var/lib/rancher/rke2/bin' >> /etc/profile
source /etc/profile 

mkdir ~/.kube
ln -s /etc/rancher/rke2/rke2.yaml ~/.kube/config
chmod 600 /root/.kube/config
ln -s /var/lib/rancher/rke2/agent/etc/crictl.yaml /etc/crictl.yaml

kubectl get node
crictl ps
crictl images

5. 安装helm软件包

wget https://mirrors.huaweicloud.com/helm/v3.5.2/helm-v3.5.2-linux-amd64.tar.gz 
tar -xf helm-v3.5.2-linux-amd64.tar.gz
mv linux-amd64/helm /usr/bin/
chmod 755 /usr/bin/helm

如果在公有云环境中部署，还需要安装 cloud_lb_provider 和 ingress

配置 Cilium Cni

kubectl -n kube-system create secret \
tls tls-ingress-hubble-ui --cert=onwalk.net.crt --key=onwalk.net.key

cat << EOF >  /var/lib/rancher/rke2/server/manifests/rke2-cilium.yaml
apiVersion: helm.cattle.io/v1
kind: HelmChartConfig
metadata:
  name: rke2-cilium
  namespace: kube-system
spec:
  valuesContent: |-
    cilium:
      k8sServiceHost: 10.0.3.107
      k8sServicePort: 6443
      operator:
        replicas: 1
      global:
        encryption:
          enabled: true
          nodeEncryption: true
      hubble:
        metrics:
          enabled:
          - dns:query;ignoreAAAA
          - drop
          - tcp
          - flow
          - icmp
          - http
        relay:
          enabled: true
        ui:
          enabled: true
          replicas: 1
          ingress:
            enabled: true
            hosts:
              - hubble.onwalk.net
            annotations:
              cert-manager.io/cluster-issuer: ca-issuer
            tls:
            - secretName: tls-ingress-hubble-ui
              hosts:
              - hubble.onwalk.net
      prometheus:
        enabled: true
        # Default port value (9090) needs to be changed since the RHEL cockpit also listens on this port.
        port: 19090
        # Configure this serviceMonitor section AFTER Rancher Monitoring is enabled!
        #serviceMonitor:
        #  enabled: true
EOF

如果开启了安全配置，需要进行如下操作，如果没有，可以略过

sudo cp -f /usr/share/rke2/rke2-cis-sysctl.conf /etc/sysctl.d/60-rke2-cis.conf
sysctl -p /etc/sysctl.d/60-rke2-cis.conf
useradd -r -c "etcd user" -s /sbin/nologin -M etcd

K8S Node 节点操作

curl -sfL https://get.rke2.io | INSTALL_RKE2_VERSION=v1.20.9+rke2r2 INSTALL_RKE2_TYPE="agent" sh -

mkdir -p /etc/rancher/rke2
cat << EOF >  /etc/rancher/rke2/config.yaml
server: https://<server_lb>:9345
token: <server_node  /var/lib/rancher/rke2/server/node-token文件的内容>
EOF

systemctl enable rke2-agent.service
systemctl start rke2-agent.service
journalctl -fu rke2-agent.service

参考

1. https://devopstales.github.io/kubernetes/rke2-cilium/
2. https://docs.rke2.io/install/install_options/server_config/
3. https://docs.rke2.io/install/install_options/install_options/
4. https://docs.cilium.io/en/v1.10/gettingstarted/k8s-install-helm/