美文网首页LinuxLinux学习之路我用 Linux
kubernetes.Etcd集群构建和flannel网络部署

kubernetes.Etcd集群构建和flannel网络部署

作者: 小短腿电工 | 来源:发表于2019-03-17 21:11 被阅读1次

    第五节.部署Etcd

    二进制包下载地址:https://github.com/coreos/etcd/releases/tag/v3.2.12

    以下部署步骤在规划的三个etcd节点操作一样,唯一不同的是etcd配置文件中的服务器IP要写当前的:
    解压二进制包:

    # mkdir /opt/etcd/{bin,cfg,ssl} -p
    # tar zxvf etcd-v3.2.12-linux-amd64.tar.gz
    # mv etcd-v3.2.12-linux-amd64/{etcd,etcdctl} /opt/etcd/bin/
    

    创建etcd配置文件:

    # cat /opt/etcd/cfg/etcd   
    #[Member]
    ETCD_NAME="etcd01"
    ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
    ETCD_LISTEN_PEER_URLS="https://192.168.31.63:2380"
    ETCD_LISTEN_CLIENT_URLS="https://192.168.31.63:2379"
    
    #[Clustering]
    ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.31.63:2380"
    ETCD_ADVERTISE_CLIENT_URLS="https://192.168.31.63:2379"
    ETCD_INITIAL_CLUSTER="etcd01=https://192.168.31.63:2380,etcd02=https://192.168.31.65:2380,etcd03=https://192.168.31.66:2380"
    ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
    ETCD_INITIAL_CLUSTER_STATE="new"
    
    • ETCD_NAME 节点名称
    • ETCD_DATA_DIR 数据目录
    • ETCD_LISTEN_PEER_URLS 集群通信监听地址
    • ETCD_LISTEN_CLIENT_URLS 客户端访问监听地址
    • ETCD_INITIAL_ADVERTISE_PEER_URLS 集群通告地址
    • ETCD_ADVERTISE_CLIENT_URLS 客户端通告地址
    • ETCD_INITIAL_CLUSTER 集群节点地址
    • ETCD_INITIAL_CLUSTER_TOKEN 集群Token
    • ETCD_INITIAL_CLUSTER_STATE 加入集群的当前状态,new是新集群,existing表示加入已有集群

    systemd管理etcd:

    # cat /usr/lib/systemd/system/etcd.service 
    [Unit]
    Description=Etcd Server
    After=network.target
    After=network-online.target
    Wants=network-online.target
    
    [Service]
    Type=notify
    EnvironmentFile=/opt/etcd/cfg/etcd
    ExecStart=/opt/etcd/bin/etcd \
    --name=${ETCD_NAME} \
    --data-dir=${ETCD_DATA_DIR} \
    --listen-peer-urls=${ETCD_LISTEN_PEER_URLS} \
    --listen-client-urls=${ETCD_LISTEN_CLIENT_URLS},http://127.0.0.1:2379 \
    --advertise-client-urls=${ETCD_ADVERTISE_CLIENT_URLS} \
    --initial-advertise-peer-urls=${ETCD_INITIAL_ADVERTISE_PEER_URLS} \
    --initial-cluster=${ETCD_INITIAL_CLUSTER} \
    --initial-cluster-token=${ETCD_INITIAL_CLUSTER_TOKEN} \
    --initial-cluster-state=new \
    --cert-file=/opt/etcd/ssl/server.pem \
    --key-file=/opt/etcd/ssl/server-key.pem \
    --peer-cert-file=/opt/etcd/ssl/server.pem \
    --peer-key-file=/opt/etcd/ssl/server-key.pem \
    --trusted-ca-file=/opt/etcd/ssl/ca.pem \
    --peer-trusted-ca-file=/opt/etcd/ssl/ca.pem
    Restart=on-failure
    LimitNOFILE=65536
    
    [Install]
    WantedBy=multi-user.target
    

    把刚才生成的证书拷贝到配置文件中的位置:

    # cp ca*pem server*pem /opt/etcd/ssl
    

    启动并设置开启启动:

    # systemctl start etcd
    # systemctl enable etcd
    

    都部署完成后,检查etcd集群状态:

    # /opt/etcd/bin/etcdctl \
    --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem \
    --endpoints="https://192.168.31.63:2379,https://192.168.31.65:2379,https://192.168.31.66:2379" \
    cluster-health
    member 18218cfabd4e0dea is healthy: got healthy result from https://192.168.31.63:2379
    member 541c1c40994c939b is healthy: got healthy result from https://192.168.31.65:2379
    member a342ea2798d20705 is healthy: got healthy result from https://192.168.31.66:2379
    cluster is healthy
    

    如果输出上面信息,就说明集群部署成功。如果有问题第一步先看日志:/var/log/message 或 journalctl -u etcd

    部署Flannel网络

    工作原理:

    图片.png

    Falnnel要用etcd存储自身一个子网信息,所以要保证能成功连接Etcd,写入预定义子网段:

    # /opt/etcd/bin/etcdctl \
    --ca-file=ca.pem --cert-file=server.pem --key-file=server-key.pem \
    --endpoints="https://192.168.31.63:2379,https://192.168.31.65:2379,https://192.168.31.66:2379" \
    set /coreos.com/network/config  '{ "Network": "172.17.0.0/16", "Backend": {"Type": "vxlan"}}'
    
    • 网络类型指定vxlan

    以下部署步骤在规划的每个node节点都操作。

    下载二进制包:

    # wget https://github.com/coreos/flannel/releases/download/v0.10.0/flannel-v0.10.0-linux-amd64.tar.gz
    # tar zxvf flannel-v0.9.1-linux-amd64.tar.gz
    # mv flanneld mk-docker-opts.sh /opt/kubernetes/bin
    

    配置Flannel:

    # cat /opt/kubernetes/cfg/flanneld
    FLANNEL_OPTIONS="--etcd-endpoints=https://192.168.31.63:2379,https://192.168.31.65:2379,https://192.168.31.66:2379 -etcd-cafile=/opt/etcd/ssl/ca.pem -etcd-certfile=/opt/etcd/ssl/server.pem -etcd-keyfile=/opt/etcd/ssl/server-key.pem"
    

    systemd管理Flannel:

    # cat /usr/lib/systemd/system/flanneld.service
    [Unit]
    Description=Flanneld overlay address etcd agent
    After=network-online.target network.target
    Before=docker.service
    
    [Service]
    Type=notify
    EnvironmentFile=/opt/kubernetes/cfg/flanneld
    ExecStart=/opt/kubernetes/bin/flanneld --ip-masq $FLANNEL_OPTIONS
    ExecStartPost=/opt/kubernetes/bin/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/subnet.env
    Restart=on-failure
    
    [Install]
    WantedBy=multi-user.target
    

    配置Docker启动指定子网段:

    # cat /usr/lib/systemd/system/docker.service 
    
    [Unit]
    Description=Docker Application Container Engine
    Documentation=https://docs.docker.com
    After=network-online.target firewalld.service
    Wants=network-online.target
    
    [Service]
    Type=notify
    EnvironmentFile=/run/flannel/subnet.env
    ExecStart=/usr/bin/dockerd $DOCKER_NETWORK_OPTIONS
    ExecReload=/bin/kill -s HUP $MAINPID
    LimitNOFILE=infinity
    LimitNPROC=infinity
    LimitCORE=infinity
    TimeoutStartSec=0
    Delegate=yes
    KillMode=process
    Restart=on-failure
    StartLimitBurst=3
    StartLimitInterval=60s
    
    [Install]
    WantedBy=multi-user.target
    

    重启flannel和docker:

    # systemctl daemon-reload
    # systemctl start flanneld
    # systemctl enable flanneld
    # systemctl restart docker
    

    检查flannel网络是否生效:

    # ps -ef |grep docker
    root     20941     1  1 Jun28 ?        09:15:34 /usr/bin/dockerd --bip=172.17.34.1/24 --ip-masq=false --mtu=1450
    # ip addr
    3607: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN 
        link/ether 8a:2e:3d:09:dd:82 brd ff:ff:ff:ff:ff:ff
        inet 172.17.34.0/32 scope global flannel.1
           valid_lft forever preferred_lft forever
    3608: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP 
        link/ether 02:42:31:8f:d3:02 brd ff:ff:ff:ff:ff:ff
        inet 172.17.34.1/24 brd 172.17.34.255 scope global docker0
           valid_lft forever preferred_lft forever
        inet6 fe80::42:31ff:fe8f:d302/64 scope link 
           valid_lft forever preferred_lft forever
    

    确保docker0与flannel.1在同一网段。
    测试不同节点互通,在当前节点访问另一个Node节点docker0 IP:

    # ping 172.17.58.1
    PING 172.17.58.1 (172.17.58.1) 56(84) bytes of data.
    64 bytes from 172.17.58.1: icmp_seq=1 ttl=64 time=0.263 ms
    64 bytes from 172.17.58.1: icmp_seq=2 ttl=64 time=0.204 ms
    如果能通说明Flannel部署成功。如果不通检查下日志:journalctl -u flannel
    
    

    相关文章

      网友评论

        本文标题:kubernetes.Etcd集群构建和flannel网络部署

        本文链接:https://www.haomeiwen.com/subject/clsumqtx.html