1、环境

    IP                   服务        主机名
    192.168.3.204   Swarm管理节点   controler
    192.168.3.202   Swarm的node节点    node2
    192.168.3.201   Swarm的node节点    node1

2、配置docker yum源（三台机器）

    cd /etc/yum.repos.d/
    touch docker.repo
    vim docker.repo

添加内容如下：*

    [dockerrepo]
    name=Docker Repository
    baseurl=https://yum.dockerproject.org/repo/main/centos/$releasever/
    enabled=1
    gpgcheck=1
    gpgkey=https://yum.dockerproject.org/gpg

:wq保存退出

3、配置hosts表（三台机器）

    vim /etc/hosts

添加内容如下：

    192.168.3.204   controller
    192.168.3.201    node1
    192.168.3.202    node2

4、配置manager节点（192.168.3.204）

4.1、删除旧版本的环境

    yum remove docker docker-client docker-client-latest docker-common docker-latest docker-latest-logrotate docker-logrotate docker-selinux docker-engine-selinux docker-engine

4.2、查看可用的docker版本

    [root@controller lib]#  yum list docker-engine.x86_64  --showduplicates |sort -r
    Loading mirror speeds from cached hostfile
    Loaded plugins: fastestmirror, langpacks
    docker-engine.x86_64             1.9.1-1.el7.centos                   dockerrepo
    docker-engine.x86_64             1.9.0-1.el7.centos                   dockerrepo
    docker-engine.x86_64             1.8.3-1.el7.centos                   dockerrepo
    docker-engine.x86_64             1.8.2-1.el7.centos                   dockerrepo
    docker-engine.x86_64             1.8.1-1.el7.centos                   dockerrepo
    docker-engine.x86_64             1.8.0-1.el7.centos                   dockerrepo
    docker-engine.x86_64             1.7.1-1.el7.centos                   dockerrepo
    docker-engine.x86_64             17.05.0.ce-1.el7.centos              dockerrepo
    docker-engine.x86_64             17.04.0.ce-1.el7.centos              dockerrepo
    docker-engine.x86_64             17.03.1.ce-1.el7.centos              dockerrepo
    docker-engine.x86_64             17.03.0.ce-1.el7.centos              dockerrepo
    docker-engine.x86_64             1.7.0-1.el7.centos                   dockerrepo
    docker-engine.x86_64             1.13.1-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.13.0-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.12.6-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.12.5-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.12.4-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.12.3-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.12.2-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.12.1-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.12.0-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.11.2-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.11.1-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.11.0-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.10.3-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.10.2-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.10.1-1.el7.centos                  dockerrepo
    docker-engine.x86_64             1.10.0-1.el7.centos                  dockerrepo
    Available Packages

4.3、安装稳定的版本1.13.1

    [root@controller lib]# yum -y install docker-engine-1.13.1-1.el7.centos 

启动docker服务并设置开机自启动

    systemctl start docker
    systemctl enable docker

4.4、下载swarm镜像

    [root@controller ~]# docker pull swarm
    Using default tag: latest
    latest: Pulling from library/swarm
    Digest: sha256:406022f04a3d0c5ce4dbdb60422f24052c20ab7e6d41ebe5723aa649c3833975
    Status: Downloaded newer image for swarm:latest
    [root@controller ~]# docker images
    REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
    swarm               latest              ff454b4a0e84        7 weeks ago         12.7 MB

4.5、创建swarm

    [root@controller ~]# docker swarm init --advertise-addr 192.168.3.204
    Swarm initialized: current node (r344b4bz1ug3bnw6olsxzqtis) is now a manager.


    To add a worker to this swarm, run the following command:


        docker swarm join \
        --token SWMTKN-1-1kvrsn7j74ie7fqxzxb0wndk76gjnwdyc6jnhe4771h55edgzw-20spfbubw7ckh1w0w3plvsak3 \
        192.168.3.204:2377


    To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

注：上面的token值要保存，其他节点均通过它加入集群

4.6、查看集群信息

    [root@controller ~]# docker info
    Containers: 0
     Running: 0
     Paused: 0
     Stopped: 0
    Images: 1
    Server Version: 1.13.1
    Storage Driver: overlay
     Backing Filesystem: xfs
     Supports d_type: false
    Logging Driver: json-file
    Cgroup Driver: cgroupfs
    Plugins: 
     Volume: local
     Network: bridge host macvlan null overlay
    Swarm: active
     NodeID: r344b4bz1ug3bnw6olsxzqtis
     Is Manager: true
     ClusterID: p5r0a72jdoq9cztshyf0bvfye
     Managers: 1
     Nodes: 1
     Orchestration:
      Task History Retention Limit: 5
     Raft:
      Snapshot Interval: 10000
      Number of Old Snapshots to Retain: 0
      Heartbeat Tick: 1
      Election Tick: 3
     Dispatcher:
      Heartbeat Period: 5 seconds
     CA Configuration:
      Expiry Duration: 3 months
     Node Address: 192.168.3.204
     Manager Addresses:
      192.168.3.204:2377
    Runtimes: runc
    Default Runtime: runc
    Init Binary: docker-init
    containerd version: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1
    runc version: 9df8b306d01f59d3a8029be411de015b7304dd8f
    init version: 949e6fa
    Security Options:
     seccomp
      Profile: default
    Kernel Version: 3.10.0-327.18.2.el7.x86_64
    Operating System: CentOS Linux 7 (Core)
    OSType: linux
    Architecture: x86_64
    CPUs: 4
    Total Memory: 7.551 GiB
    Name: controller
    ID: 5E6V:BHYM:3L74:EYS3:ZUID:INQP:DINI:U7I6:QKJZ:UCYY:QZJF:GXGN
    Docker Root Dir: /var/lib/docker
    Debug Mode (client): false
    Debug Mode (server): false
    Registry: https://index.docker.io/v1/
    Experimental: false
    Insecure Registries:
     127.0.0.0/8
    Live Restore Enabled: false

4.7、查看集群节点信息

    [root@controller ~]# docker node ls
    ID                           HOSTNAME    STATUS  AVAILABILITY  MANAGER STATUS
    r344b4bz1ug3bnw6olsxzqtis *  controller  Ready   Active        Leader

5、配置节点（192.168.3.202,192.168.3.201）

5.1、安装docker

    yum -y install docker-engine-1.13.1-1.el7.centos

启动docker

    systemctl start docker

开机自启动

    systemctl enable docker

5.2、加入到集群

    docker swarm join --token SWMTKN-1-1kvrsn7j74ie7fqxzxb0wndk76gjnwdyc6jnhe4771h55edgzw-20spfbubw7ckh1w0w3plvsak3 192.168.3.204:2377

5.3、查看docker信息（两个节点一样执行，我这里只贴出在node2执行的信息）

    [root@node2 ~]# docker info
    Containers: 0
     Running: 0
     Paused: 0
     Stopped: 0
    Images: 0
    Server Version: 1.13.1
    Storage Driver: overlay
     Backing Filesystem: xfs
     Supports d_type: false
    Logging Driver: json-file
    Cgroup Driver: cgroupfs
    Plugins: 
     Volume: local
     Network: bridge host macvlan null overlay
    Swarm: active
     NodeID: pdqau3dpxtgnr6gg78zbt7nnz
     Is Manager: false
     Node Address: 192.168.3.202
     Manager Addresses:
      192.168.3.204:2377
    Runtimes: runc
    Default Runtime: runc
    Init Binary: docker-init
    containerd version: aa8187dbd3b7ad67d8e5e3a15115d3eef43a7ed1
    runc version: 9df8b306d01f59d3a8029be411de015b7304dd8f
    init version: 949e6fa
    Security Options:
     seccomp
      Profile: default
    Kernel Version: 3.10.0-229.el7.x86_64
    Operating System: CentOS Linux 7 (Core)
    OSType: linux
    Architecture: x86_64
    CPUs: 2
    Total Memory: 1.797 GiB
    Name: node2
    ID: Q7VU:TMEP:NCO6:YVXC:O6U2:2DON:5RUL:VGJ4:D2TA:345M:I4Q6:UHN5
    Docker Root Dir: /var/lib/docker
    Debug Mode (client): false
    Debug Mode (server): false
    Registry: https://index.docker.io/v1/
    WARNING: bridge-nf-call-ip6tables is disabled
    Experimental: false
    Insecure Registries:
     127.0.0.0/8
    Live Restore Enabled: false

5.4、在集群节点（192.168.3.204）查看节点信息

    [root@controller ~]# docker node ls
    ID                           HOSTNAME    STATUS  AVAILABILITY  MANAGER STATUS
    hjba4ers0ahzbdvggajc3b73s    node1       Ready   Active        
    pdqau3dpxtgnr6gg78zbt7nnz    node2       Ready   Active        
    r344b4bz1ug3bnw6olsxzqtis *  controller  Ready   Active        Leader

6、在swarm中部署nginx服务

6.1、创建覆盖网络（保障不同主机容器互通）

    [root@controller ~]# docker network create -d overlay nginx
    692ud2yyqimkuq6rufpnwnt05
    [root@controller ~]# docker network ls
    NETWORK ID          NAME                DRIVER              SCOPE
    27940e4bc00a        bridge              bridge              local
    cd675706a24e        docker_gwbridge     bridge              local
    ec66ee0e74b1        host                host                local
    l8d4j4w5o52u        ingress             overlay             swarm
    692ud2yyqimk        nginx               overlay             swarm
    3a44f24d7c1f        none                null                local

6.2、创建服务

创建：

    [root@controller ~]# docker service create --replicas 1 --network nginx --name MyNginx -p 80:80 nginx
    8sk44dipvooy4sb2e7e4l8wb1

查看服务

    [root@controller ~]# docker service ls
    ID            NAME     MODE        REPLICAS  IMAGE
    8sk44dipvooy  MyNginx  replicated  1/1       nginx:latest

查看摘要信息：

    [root@controller ~]#  docker service inspect --pretty MyNginx


    ID:        8sk44dipvooy4sb2e7e4l8wb1
    Name:        MyNginx
    Service Mode:    Replicated
     Replicas:    1
    Placement:
    UpdateConfig:
     Parallelism:    1
     On failure:    pause
     Max failure ratio: 0
    ContainerSpec:
     Image:        nginx:latest@sha256:d85914d547a6c92faa39ce7058bd7529baacab7e0cd4255442b04577c4d1f424
    Resources:
    Networks: nginx 
    Endpoint Mode:    vip
    Ports:
     PublishedPort 80
      Protocol = tcp
      TargetPort = 80 

6.3、查看服务的运行情况

    [root@controller ~]# docker service ps MyNginx
    ID            NAME       IMAGE         NODE   DESIRED STATE  CURRENT STATE               ERROR  PORTS
    khvp4r03de4f  MyNginx.1  nginx:latest  node2  Running        Running about a minute ago       

6.4、访问服务

浏览器或者curl访问服务http://192.168.3.204

    <!DOCTYPE html>
    <html>
    <head>
    <title>Welcome to nginx!</title>
    <style>
        body {
            width: 35em;
            margin: 0 auto;
            font-family: Tahoma, Verdana, Arial, sans-serif;
        }
    </style>
    </head>
    <body>
    <h1>Welcome to nginx!</h1>
    <p>If you see this page, the nginx web server is successfully installed and
    working. Further configuration is required.</p>


    <p>For online documentation and support please refer to
    <a href="http://nginx.org/">nginx.org</a>.<br/>
    Commercial support is available at
    <a href="http://nginx.com/">nginx.com</a>.</p>


    <p><em>Thank you for using nginx.</em></p>
    </body>
    </html>

7、Service的复制功能

7.1、增加

设置5个副本

    [root@controller ~]# docker service scale MyNginx=5
    MyNginx scaled to 5
    [root@controller ~]# docker service ps MyNginx
    ID            NAME       IMAGE         NODE        DESIRED STATE  CURRENT STATE            ERROR  PORTS
    khvp4r03de4f  MyNginx.1  nginx:latest  node2       Running        Running 12 minutes ago          
    wecu7jye2mg3  MyNginx.2  nginx:latest  controller  Running        Preparing 5 seconds ago         
    5a6rp8c6arvt  MyNginx.3  nginx:latest  node2       Running        Running 4 seconds ago           
    68lu0fpg75on  MyNginx.4  nginx:latest  node1       Running        Preparing 5 seconds ago         
    todza2i5lnnt  MyNginx.5  nginx:latest  controller  Running        Preparing 5 seconds ago       

7.2、减少

变更为1个副本

    [root@controller ~]# docker service scale MyNginx=1
    MyNginx scaled to 1
    [root@controller ~]# docker service ls
    ID            NAME     MODE        REPLICAS  IMAGE
    8sk44dipvooy  MyNginx  replicated  1/1       nginx:latest
    [root@controller ~]# docker service ps MyNginx
    ID            NAME           IMAGE         NODE        DESIRED STATE  CURRENT STATE           ERROR                             PORTS
    exoh860irad5  MyNginx.1      nginx:latest  node1       Running        Running 4 minutes ago         

8、使用volume

8.1、查看volume

    [root@controller ~]# docker volume ls
    DRIVER              VOLUME NAME

8.2、创建volume

    [root@controller ~]# docker volume create --name nginx
    nginx
    [root@controller ~]# docker volume ls
    DRIVER              VOLUME NAME
    local               nginx

8.3、创建服务并指定挂载

src是volume名称

dst是挂载点（也就是容器内的路径，如果不存在会自动创建）

注：若指定的是本机的路径，需要把type改成bind, src是本机的路径

    [root@controller ~]# docker service create  --replicas 2 --mount type=volume,src=nginx,dst=/farmer --name ng nginx
    r7bld5xg9cy6285z6207wlly2
    [root@controller ~]# docker service ls
    ID            NAME     MODE        REPLICAS  IMAGE
    9et4o10fn0w2  MyNginx  replicated  2/2       nginx:latest
    r7bld5xg9cy6  ng       replicated  2/2       nginx:latest
    [root@controller ~]# docker service ps ng
    ID            NAME  IMAGE         NODE   DESIRED STATE  CURRENT STATE           ERROR  PORTS
    190gm96z6brc  ng.1  nginx:latest  node2  Running        Running 25 seconds ago         
    093pql11ttdm  ng.2  nginx:latest  node1  Running        Running 25 seconds ago         

9、创建nginx服务和负载均衡的测试

9.1、创建volume名称为modules

    docker volume create --name modules

注：因为/etc/nginx/mudules是软连接到/usr/lib/nginx/modules，一旦将/etc/nginx挂载到本地，这个目录将会失效，所以需要在modules也挂到本地

    lrwxrwxrwx 1 root root   22 Jul 24 21:02 modules -> /usr/lib/nginx/modules 

9.2、创建软连接（所有节点，192.168.3.204,192.168.3.202,192.168.3.201）

    mkdir /usr/lib/nginx
    ln -s /var/lib/docker/volumes/modules/_data/modules/ /usr/lib/nginx/modules

9.3、创建test服务

    [root@controller ~]# docker service create --replicas 2 --mount type=volume,src=nginx,dst=/etc/nginx/ --mount type=volume,dst=/usr/lib/nginx/,src=modules --name test -p 8880:80 nginx
    9mt9wuabsluykwfh2j7ez9gut
    [root@controller ~]# docker service ps test
    ID            NAME    IMAGE         NODE        DESIRED STATE  CURRENT STATE          ERROR  PORTS
    hwp14enb23gz  test.1  nginx:latest  node1       Running        Running 6 seconds ago         
    t6mpq6sbongg  test.2  nginx:latest  controller  Running        Running 6 seconds ago      

9.4、配置HTML服务（三个节点，192.168.3.201,192.168.3.202,192.168.3.204）

    cd /var/lib/docker/volumes/nginx/_data
    mkdir html
    vim conf.d/default.conf

修改如下：

    location / {
            root   /etc/nginx/html;
            index  index.html index.htm;
        }

注：这里用/etc/nginx/html路径是因为在docker里面路径是这个

我把创建的新的html当做nginx的root目录

9.4.1、管理节点

    cd /var/lib/docker/volumes/nginx/_data/html
    touch index.html

添加内容：

    <h1>Controller</h1>

9.4.1、node1节点

    cd /var/lib/docker/volumes/nginx/_data/html
    touch index.html

添加内容：

    <h1>Node1</h1>

9.4.2、node2节点

    cd /var/lib/docker/volumes/nginx/_data/html
    touch index.html

添加内容：

    <h1>Node2</h1>

9.5、重启nginx服务

当前服务在node1和controller节点运行，故而只在这两个节点执行（机器环境不同，所在的节点不同，请看清自己的节点）

9.5.1、node1节点

    [root@node1 ~]# docker ps
    CONTAINER ID        IMAGE                                                                           COMMAND                  CREATED             STATUS              PORTS               NAMES
    33b4e08c57d0        nginx@sha256:d85914d547a6c92faa39ce7058bd7529baacab7e0cd4255442b04577c4d1f424   "nginx -g 'daemon ..."   4 hours ago         Up 4 hours          80/tcp              test.1.ov76pdo715cc1f24i5vfgu6si
    docker exec -ti 33b4e08c57d0 /bin/bash
    /etc/init.d/nginx reload

9.5.2、controller节点

    [root@controller _data]# docker ps
    CONTAINER ID        IMAGE                                                                           COMMAND                  CREATED             STATUS              PORTS               NAMES
    e54913d64ce9        nginx@sha256:d85914d547a6c92faa39ce7058bd7529baacab7e0cd4255442b04577c4d1f424   "nginx -g 'daemon ..."   3 days ago          Up 3 days           80/tcp              test.2.t6mpq6sbongg5rq8rd4lofp6u
    [root@controller _data]# docker exec -ti e54913d64ce9 /bin/bash
    root@e54913d64ce9:/# /etc/init.d/nginx reload

9.6、测试服务（服务是否正常，是否有负载）

注：测试过程中有时候多curl几次才会负载到另外的一个节点

    [root@mysql76 ~]# curl http://192.168.3.204:8880/
    <h1>Node1</h1>
    [root@mysql76 ~]# curl http://192.168.3.204:8880/
    <h1>Controller</h1>

9.7、测试集群

在节点1执行

    docker stop 33b4e08c57d0

查看集群服务

    [root@controller _data]# docker service ps test
    ID            NAME        IMAGE         NODE        DESIRED STATE  CURRENT STATE         ERROR  PORTS
    ov76pdo715cc  test.1      nginx:latest  node2       Running        Running 1 hours ago          
    qoyuk652tky2   \_ test.1  nginx:latest  node1       Shutdown       Complete 1 hours ago         
    t6mpq6sbongg  test.2      nginx:latest  controller  Running        Running 3 days ago

测试集群服务（服务是否正常，是否有负载）

注：测试过程中有时候多curl几次才会负载到另外的一个节点

    [root@mysql76 ~]# curl http://192.168.3.204:8880/
    <h1>Node2</h1>
    [root@mysql76 ~]# curl http://192.168.3.204:8880/
    <h1>Controller</h1>