1、准备环境

1-1、准备host主机

• node195: 192.168.0.195
• node196: 192.168.0.196
• node197: 192.168.0.197

注意：要设置host主机hostname为不一样，因为etcd要根据hostname来识别不同的主机，如果hostname不小心设置成一样，集群中同一overlay网络不同host节点容器之间的网络通信将会失败（无法ping通）

1-2、节点信息

• etcd 服务器: node195
• swarm manage: node195
• cluster hosts（集群中的主机）: node196, node197
• registry host(局域网镜像仓库): node197

1-3、软件安装

• 在所有集群上安装docker，CentOS系统安装docker步骤可以参考官网

注意：由于后续配置集群搭建的需要，这里需要对docker的启动配置做一下调整，首先创建并编辑docker启动配置文件，具体参考下面命令:

# 创建并进入配置文件
vi /etc/default/docker
# 填写以下内容
DOCKER_OPTS="--registry-mirror=https://x5jytudq.mirror.aliyuncs.com -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --insecure-registry 192.168.0.197:5000 --cluster-advertise 192.168.0.196:2375 --cluster-store etcd://192
.168.0.195:2379"
# 打开docker的启动文件
vi /lib/systemd/system/docker.service


# 内容如下
# the default is not to use systemd for cgroups because the delegate issues still
# exists and systemd currently does not support the cgroup feature set required
# for containers run by docker
ExecStart=/usr/bin/dockerd
ExecReload=/bin/kill -s HUP $MAINPID
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
# Uncomment TasksMax if your systemd version supports it.
# Only systemd 226 and above support this version.
#TasksMax=infinity
TimeoutStartSec=0
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target


# 修改docker.service配置文件
#在ExecStart=/usr/bin/dockerd 前添加内容
EnvironmentFile=-/etc/default/docker
# 修改ExecStart=/usr/bin/dockerd
ExecStart=/usr/bin/dockerd $DOCKER_OPTS
# 重启docker服务
systemctl daemon-reload
systemctl restart docker

其中 --registry-mirror=https://x5jytudq.mirror.aliyuncs.com 表示配置阿里云私有镜像仓库；-H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock表示开启docker的远程访问；--insecure-registry 192.168.0.197:5000表示非安全（http请求，安全型是https，类似阿里云私有镜像仓库）的本地镜像仓库；--cluster-advertise 192.168.0.196:2375表示集群中host节点的服务发现IP配置；--cluster-store etcd://192.168.0.195:2379"表示服务发现的etcd数据存储服务；

• 在node195安装docker compose，具体安装步骤可参考官网
• 在node196 上执行docker pull quay.io/coreos/etcd（官网可能下载的比较慢，可以开加速器尝试加速下载）
• 在所有host上执行docker pull swarm拉取最新的swarm镜像
• 关闭局域网所有host防火墙，不然后续跨主机容器间通信可能会受到影响

2、安装etcd k-v数据库

在node195上执行以下代码，具体可以参考官网

# 设置HostIP
export HostIP=192.168.0.195
# 执行etcd安装启动命令
docker run -d -v /usr/share/ca-certificates/:/etc/ssl/certs -p 4001:4001 -p 2380:2380 -p 2379:2379 \
 --restart=always \
 --name etcd quay.io/coreos/etcd \
 /usr/local/bin/etcd \
 -name etcd0 \
 -advertise-client-urls http://${HostIP}:2379,http://${HostIP}:4001 \
 -listen-client-urls http://0.0.0.0:2379,http://0.0.0.0:4001 \
 -initial-advertise-peer-urls http://${HostIP}:2380 \
 -listen-peer-urls http://0.0.0.0:2380 \
 -initial-cluster-token etcd-cluster-1 \
 -initial-cluster etcd0=http://${HostIP}:2380 \
 -initial-cluster-state new

3、将host节点加入集群

执行以下代码，让node196和node197加入集群：

# 在node196上执行，将node196加入集群
docker run -d --name node196 --restart=always swarm join --addr=192.168.0.196:2375 etcd://192.168.0.195:2379/swarm
#在node197上执行，将node197加入集群
docker run -d --name node197 --restart=always swarm join --addr=192.168.0.197:2375 etcd://192.168.0.195:2379/swarm

4、启动swarm manage管理节点，并测试集群是否搭建成功

在node195上执行

# 启动swarm管理节点
docker run -d -p 3376:3376 -t \
--restart=always \
--name manage swarm manage \
-H 0.0.0.0:3376 \
etcd://192.168.0.195:2379/swarm
#设置DOCKER_HOST变量
export DOCKER_HOST=192.168.0.195:3376
#查看节点中的集群信息
docker info


#内容如下
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 0
Server Version: swarm/1.2.6
Role: primary
Strategy: spread
Filters: health, port, containerslots, dependency, affinity, constraint, whitelist
Nodes: 2
 node196: 192.168.0.196:2375
  └ ID: 2XXV:XLC4:6DZR:RIDS:GJI7:QIVK:ITKM:ZD2O:FX7R:J3CT:O6ST:ZU4B
  └ Status: Healthy
  └ Containers: 0 (0 Running, 0 Paused, 0 Stopped)
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 4.914 GiB
  └ Labels: kernelversion=3.10.0-327.el7.x86_64, operatingsystem=CentOS Linux 7 (Core), storagedriver=overlay
  └ UpdatedAt: 2017-03-04T02:19:23Z
  └ ServerVersion: 1.13.1
 node197: 192.168.0.197:2375
  └ ID: LMIN:53UN:MPR5:NEAQ:EQNZ:T3ZD:265Y:6CHI:JQPV:QWWV:RKK3:YTJW
  └ Status: Healthy
  └ Containers: 0 (0 Running, 0 Paused, 0 Stopped)
  └ Reserved CPUs: 0 / 2
  └ Reserved Memory: 0 B / 5.086 GiB
  └ Labels: kernelversion=3.10.0-514.6.2.el7.x86_64, operatingsystem=CentOS Linux 7 (Core), storagedriver=overlay
  └ UpdatedAt: 2017-03-04T02:19:28Z
  └ ServerVersion: 17.03.0-ce
Plugins: 
 Volume: 
 Network: 
Swarm: 
 NodeID: 
 Is Manager: false
 Node Address: 
Kernel Version: 3.10.0-514.el7.x86_64
Operating System: linux
Architecture: amd64
CPUs: 4
Total Memory: 14.91 GiB
Name: 24fb2d69de9d
Docker Root Dir: 
Debug Mode (client): false
Debug Mode (server): false
WARNING: No kernel memory limit support
Experimental: false
Live Restore Enabled: false

5、自定义overlay网络，并测试不同节点之间的网络通信

在node195 swarm manage节点上执行，关于overlay可参考官网

#创建自定义网络my-net
docker network create --driver overlay --subnet=10.0.9.0/24 my-net
#安装并启动一个nginx web容器，并约束该容器安装到node196上
docker run -itd --name=web --network=docker_default --env="constraint:node==node196" nginx
#安装并启动一个busybox容器，并约束将该容器安装到node197上，执行命令访问web服务
docker run -it --rm --network=my-net --env="constraint:node==node196" busybox wget -O- http://web

#展示内容
Unable to find image 'busybox:latest' locally
 latest: Pulling from library/busybox
 ab2b8a86ca6c: Pull complete
 2c5ac3f849df: Pull complete
 Digest: sha256:5551dbdfc48d66734d0f01cafee0952cb6e8eeecd1e2492240bf2fd9640c2279
 Status: Downloaded newer image for busybox:latest
 Connecting to web (10.0.0.2:80)
 <!DOCTYPE html>
 <html>
 <head>
 <title>Welcome to nginx!</title>
 <style>
 body {
         width: 35em;
         margin: 0 auto;
         font-family: Tahoma, Verdana, Arial, sans-serif;
 }
 </style>
 </head>
 <body>
 <h1>Welcome to nginx!</h1>
 <p>If you see this page, the nginx web server is successfully installed and
 working. Further configuration is required.</p>

 <p>For online documentation and support please refer to
 <a href="http://nginx.org/">nginx.org</a>.<br/>
 Commercial support is available at
 <a href="http://nginx.com/">nginx.com</a>.</p>

 <p><em>Thank you for using nginx.</em></p>
 </body>
 </html>
 -                    100% |*******************************|   612   0:00:00 ETA