pipework的做法是通过网桥使容器，宿主机在同一个网段中进行通信
实验环境：
一、基础环境配置，三台服务器均执行以下操作
1. 配置固定IP
[root@localhost ~]# vim /etc/sysconfig/network-scripts/ifcfg-ens32
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=yes
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=ens32
UUID=75963e3f-b289-4bbd-8489-44f6f2b8c7f0
DEVICE=ens32
ONBOOT=yes
IPADDR=192.168.0.10
PREFIX=24
GATEWAY=192.168.0.1
DNS1=114.114.114.114
[root@localhost ~]# systemctl restart network
2. 更改主机名
[root@localhost ~]# hostnamectl set-hostname docker1
[root@localhost ~]# exit //重新登陆即可
[root@docker1 ~]#
3.关闭防火墙
[root@docker1 ~]# systemctl stop firewalld
[root@docker1 ~]# systemctl disable firewalld
4. 同步系统时间
[root@docker1 ~]# yum -y install ntp
[root@docker1 ~]# systemctl enable ntpd.service
[root@docker1 ~]# ntpdate cn.pool.ntp.org
[root@docker1 ~]# hwclock -w
[root@docker1 ~]# crontab -e
0 2 * * * ntpdate ntpdate cn.pool.ntp.org && hwclock -w
5.关闭selinux
[root@docker1 ~]# vim /etc/sysconfig/selinux
SELINUX=disabled
[root@docker1 ~]# reboot
6. 安装docker-ce
1.> Docker 要求 CentOS 系统的内核版本高于 3.10 ，查看你的CentOS 版本是否支持 Docker
通过 uname -r 命令查看你当前的内核版本
[root@docker1 ~]# uname -r
3.10.0-693.2.2.el7.x86_64
2.> 使用 root 权限登录 Centos,确保 yum 包更新到最新。
[root@docker1 ~]# yum update
3.> 卸载旧版本(如果安装过旧版本的话)
[root@docker1 ~]# yum remove docker docker-common docker-selinux docker-engine
4.> 安装需要的软件包， yum-util 提供yum-config-manager功能，另外两个是devicemapper驱动依赖的
[root@docker1 ~]# yum install -y yum-utils device-mapper-persistent-data lvm2
5.> 设置yum源
[root@docker1 ~]# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
Loaded plugins: fastestmirror
adding repo from: https://download.docker.com/linux/centos/docker-ce.repo
grabbing file https://download.docker.com/linux/centos/docker-ce.repo to /etc/yum.repos.d/docker-ce.repo
repo saved to /etc/yum.repos.d/docker-ce.repo
[root@docker1 ~]# ls /etc/yum.repos.d/ |grep docker
docker-ce.repo
6.> 安装docker
[root@docker1 ~]# yum install -y docker-ce
[root@docker1 ~]# docker version
Client:
Version:18.03.0-ce
API version: 1.37
Go version: go1.9.4
Git commit: 0520e24
Built: Wed Mar 21 23:09:15 2018
OS/Arch: linux/amd64
Experimental: false
Orchestrator: swarm
Server:
Engine:
Version: 18.03.0-ce
API version: 1.37 (minimum version 1.12)
Go version: go1.9.4
Git commit: 0520e24
Built: Wed Mar 21 23:13:03 2018
OS/Arch: linux/amd64
Experimental: false
[root@docker1 ~]# systemctl start docker
[root@docker1 ~]# systemctl enable docker
docker2主机重复以上操作，注意修改对应的IP，主机名等
二、跨主机通信
1.docker1上进行如下操作
1.> 开启路由转发
[root@docker1 ~]# vim /etc/sysctl.conf
net.ipv4.ip_forward = 1
[root@docker1 ~]# sysctl -p
net.ipv4.ip_forward = 1
2.> 停止docker服务
[root@docker1 ~]# systemctl stop docker
3.> 删除docker0网卡
[root@docker1 ~]# yum -y install bridge-utils #安装网桥
[root@docker1 ~]# ifconfig docker0 down
[root@docker1 ~]# brctl delbr docker0
4.> 新建桥接物理网络虚拟网卡br0
[root@docker1 ~]# ping -c 2 www.baidu.com
PING www.a.shifen.com (61.135.169.125) 56(84) bytes of data.
64 bytes from 61.135.169.125 (61.135.169.125): icmp_seq=1 ttl=57 time=4.53 ms
64 bytes from 61.135.169.125 (61.135.169.125): icmp_seq=2 ttl=57 time=20.9 ms
[root@docker1 ~]# brctl addbr br0
[root@docker1 ~]# ip link set dev br0 up
[root@docker1 ~]# ip addr del 192.168.0.10/24 dev ens32 #删除宿主机网卡,若是使用的192.168.0.10地址连接到这台服务器，此操作会中断连接；若使用的是公网地址进行的连接，则连接不会中断。
[root@docker1 ~]# ip addr add 192.168.0.10/24 dev br0 #将宿主主机的ip设置到br0
[root@docker1 ~]# brctl addif br0 ens32 #将宿主机网卡挂到br0上
[root@docker1 ~]# brctl show
bridge name bridge id STP enabled interfaces
br0 8000.000c2921fdb4 no ens32
[root@docker1 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 br0
[root@docker1 ~]# ip route del default #删除原默认路由
RTNETLINK answers: No such process
[root@docker1 ~]# ip route add default via 192.168.0.1 dev br0 #将br0设置为默认路由[root@docker1 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 br0
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 br0
[root@docker1 ~]# ping -c 2 www.baidu.com
PING www.a.shifen.com (61.135.169.125) 56(84) bytes of data.
64 bytes from 61.135.169.125 (61.135.169.125): icmp_seq=1 ttl=57 time=4.53 ms
64 bytes from 61.135.169.125 (61.135.169.125): icmp_seq=2 ttl=57 time=20.9 ms
4.> 设置docker服务启动参数
[root@docker1 ~]# vim /etc/default/docker
DOCKER_OPTS="--registry-mirror=https://pee6w651.mirror.aliyuncs.com -b=br0"
#让docker服务启动时使用br0网卡进行桥接
5.> 启动docker服务
[root@docker1 ~]# systemctl start docker
[root@docker1 ~]# ifconfig
br0: flags=4163 mtu 1500
inet 192.168.0.10 netmask 255.255.255.0 broadcast 0.0.0.0
inet6 fe80::a8b1:2bff:fe5e:e14f prefixlen 64 scopeid 0x20
ether 00:0c:29:21:fd:b4 txqueuelen 1000 (Ethernet)
RX packets 830 bytes 85880 (83.8 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 262 bytes 33301 (32.5 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
docker0: flags=4099 mtu 1500
inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255
ether 02:42:b9:53:45:8f txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens32: flags=4163 mtu 1500
inet6 fe80::8b7b:26a1:ba3b:ec9e prefixlen 64 scopeid 0x20
ether 00:0c:29:21:fd:b4 txqueuelen 1000 (Ethernet)
RX packets 2687 bytes 252748 (246.8 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 519 bytes 66048 (64.5 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
6.> 安装pipework
[root@docker1 ~]# mkdir /root/docker
[root@docker1 ~]# cd docker
[root@docker1 docker]# yum -y install git
[root@docker1 docker]# git clonehttps://github.com/jpetazzo/pipework
[root@docker1 docker]# ls
pipework
[root@docker1 docker]# cp pipework/pipework /usr/local/bin/
7.> 启动一个手动设置网络的容器
docker启动时不让其自动获取ip，下次启动会有变化而且自动获取的ip可能会和物理网段中的ip冲突
[root@docker1 docker]# docker images
[root@docker1 docker]# docker pull centos
[root@docker1 docker]# docker network ls
NETWORK ID NAME DRIVER SCOPE
ad923d5f8619 bridge bridge local
e218382d1e2a host host local
1208a8ee5a86 none null local
[root@docker1 docker]# docker run -itd --net=none --name server01 centos
ada440283d7d9113dee4abc2e1653ac0ac28b76e92595c4bc08e58d4113247be
8.> 为server01容器设置一个与桥接物理网络同地址段的ip@网关
[root@docker1 docker]# pipework br0 server01 192.168.0.11/24@192.168.0.1
#如此容器便和宿主主机在同一个网段了
[root@docker1 docker]# docker network ls
NETWORK ID NAME DRIVER SCOPE
ad923d5f8619 bridge bridge local
e218382d1e2a host host local
1208a8ee5a86 none null local
9.> 查看server01主机运行的容器
[root@docker1 docker]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ada440283d7d centos "/bin/bash" About a minute ago Up 58 seconds server01
2.docker2上进行如下操作
1.> 开启路由转发
[root@docker2 ~]# vim /etc/sysctl.conf
net.ipv4.ip_forward = 1
[root@docker2 ~]# sysctl -p
net.ipv4.ip_forward = 1
2.> 停止docker服务
[root@docker2 ~]# systemctl stop docker
3.> 删除docker0网卡
[root@docker2 ~]# yum -y install bridge-utils #安装网桥
[root@docker2 ~]# ifconfig docker0 down
[root@docker2 ~]# brctl delbr docker0
4.> 新建桥接物理网络虚拟网卡br0
[root@docker2 ~]# ping -c 2 www.baidu.com
PING www.a.shifen.com (61.135.169.125) 56(84) bytes of data.
64 bytes from 61.135.169.125 (61.135.169.125): icmp_seq=1 ttl=57 time=4.53 ms
64 bytes from 61.135.169.125 (61.135.169.125): icmp_seq=2 ttl=57 time=20.9 ms
[root@docker2 ~]# brctl addbr br0
[root@docker2 ~]# ip link set dev br0 up
[root@docker2 ~]# ip addr del 192.168.0.20/24 dev ens32 #删除宿主机网卡,若是使用的192.168.0.20地址连接到这台服务器，此操作会中断连接；若使用的是公网地址进行的连接，则连接不会中断。
[root@docker2 ~]# ip addr add 192.168.0.20/24 dev br0 #将宿主主机的ip设置到br0
[root@docker2 ~]# brctl addif br0 ens32 #将宿主机网卡挂到br0上
[root@docker2 ~]# brctl show
bridge name bridge id STP enabled interfaces
br0 8000.000c2921fdb4 no ens32
[root@docker2 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 br0
[root@docker2 ~]# ip route del default #删除原默认路由
RTNETLINK answers: No such process
[root@docker2 ~]# ip route add default via 192.168.0.1 dev br0 #将br0设置为默认路由
[root@docker2 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 br0
192.168.0.0 0.0.0.0 255.255.255.0 U 0 0 0 br0
[root@docker2 ~]# ping -c 2 www.baidu.com
PING www.a.shifen.com (61.135.169.125) 56(84) bytes of data.
64 bytes from 61.135.169.125 (61.135.169.125): icmp_seq=1 ttl=57 time=4.53 ms
64 bytes from 61.135.169.125 (61.135.169.125): icmp_seq=2 ttl=57 time=20.9 ms
4.> 设置docker服务启动参数
[root@docker2 ~]# vim /etc/default/docker
DOCKER_OPTS="--registry-mirror=https://pee6w651.mirror.aliyuncs.com -b=br0"
#让docker服务启动时使用br0网卡进行桥接
5.> 启动docker服务
[root@docker2 ~]# systemctl start docker
[root@docker2 ~]# ifconfig
br0: flags=4163 mtu 1500
inet 192.168.0.20 netmask 255.255.255.0 broadcast 0.0.0.0
inet6 fe80::742c:abff:fe84:b7f0 prefixlen 64 scopeid 0x20
ether 00:0c:29:df:31:da txqueuelen 1000 (Ethernet)
RX packets 3935 bytes 541969 (529.2 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 728 bytes 92266 (90.1 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
docker0: flags=4099 mtu 1500
inet 172.17.0.1 netmask 255.255.0.0 broadcast 172.17.255.255
ether 02:42:b9:53:45:8f txqueuelen 0 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
ens32: flags=4163 mtu 1500
inet6 fe80::8b7b:26a1:ba3b:ec9e prefixlen 64 scopeid 0x20
ether 00:0c:29:21:fd:b4 txqueuelen 1000 (Ethernet)
RX packets 2687 bytes 252748 (246.8 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 519 bytes 66048 (64.5 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
6.> 安装pipework
[root@docker2 ~]# mkdir /root/docker
[root@docker2 ~]# cd docker
[root@docker2 docker]# yum -y install git
[root@docker2 docker]# git clone https://github.com/jpetazzo/pipework
[root@docker2 docker]# ls
pipework
[root@docker2 docker]# cp pipework/pipework /usr/local/bin/
7.> 启动一个手动设置网络的容器
docker启动时不让其自动获取ip，下次启动会有变化而且自动获取的ip可能会和物理网段中的ip冲突
[root@docker2 docker]# docker images
[root@docker2 docker]# docker pull centos
[root@docker2 docker]# docker network ls
NETWORK ID NAME DRIVER SCOPE
ad923d5f8619 bridge bridge local
e218382d1e2a host host local
1208a8ee5a86 none null local
[root@docker2 docker]# docker run -itd --net=none --name server02 centos
3ef29b11da92f9d4afa7378da0db978d60ceab9bba71273b617b60d5578d9e76
8.为server02容器设置一个与桥接物理网络同地址段的ip@网关
[root@docker2 docker]# pipework br0 server02 192.168.0.21/24@192.168.0.1
#如此容器便和宿主主机在同一个网段了
[root@docker2 docker]# docker network ls
NETWORK ID NAME DRIVER SCOPE
ad923d5f8619 bridge bridge local
e218382d1e2a host host local
1208a8ee5a86 none null local
9.> 查看docker2主机运行的容器
[root@docker2 docker]# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3ef29b11da92 centos "/bin/bash" About a minute ago Up 58 seconds server02
3. 不同宿主机内容器互通测试
1.> 查看docker1宿主机内容器
[root@docker1 docker]# docker exec -it ada440283d7d /bin/bash
[root@ada440283d7d /]# ping -c 2 www.baidu.com
PING www.a.shifen.com (61.135.169.125) 56(84) bytes of data.
64 bytes from 61.135.169.125 (61.135.169.125): icmp_seq=1 ttl=57 time=6.83 ms
64 bytes from 61.135.169.125 (61.135.169.125): icmp_seq=2 ttl=57 time=3.45 ms
--- www.a.shifen.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 3.450/5.144/6.838/1.694 ms
[root@ada440283d7d /]# yum -y install net-tools
[root@ada440283d7d /]# ifconfig
eth1: flags=4163 mtu 1500
inet 192.168.0.11 netmask 255.255.255.0 broadcast 192.168.0.255
ether ea:de:b7:c6:a1:33 txqueuelen 1000 (Ethernet)
RX packets 2265 bytes 8272860 (7.8 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1796 bytes 138497 (135.2 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73 mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
loop txqueuelen 1 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
2.> 查看docker2宿主机内容器
[root@docker2 docker]# docker exec -it 3ef29b11da92 /bin/bash
[root@3ef29b11da92 /]# ping -c 2 www.baidu.com
PING www.a.shifen.com (61.135.169.121) 56(84) bytes of data.
64 bytes from 61.135.169.121 (61.135.169.121): icmp_seq=1 ttl=57 time=7.15 ms
64 bytes from 61.135.169.121 (61.135.169.121): icmp_seq=2 ttl=57 time=4.24 ms
--- www.a.shifen.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 4.245/5.702/7.159/1.457 ms
[root@3ef29b11da92 /]# yum -y install net-tools
[root@3ef29b11da92 /]# ifconfig
eth1: flags=4163 mtu 1500
inet 192.168.0.21 netmask 255.255.255.0 broadcast 192.168.0.255
ether 2e:39:2e:c1:c9:3a txqueuelen 1000 (Ethernet)
RX packets 2538 bytes 8322272 (7.9 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1586 bytes 107200 (104.6 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo: flags=73 mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
loop txqueuelen 1 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
3.> 容器互通测试
server01 ping server02
[root@ada440283d7d /]# ping -c 2 192.168.0.21
PING 192.168.0.21 (192.168.0.21) 56(84) bytes of data.
64 bytes from 192.168.0.21: icmp_seq=1 ttl=64 time=0.451 ms
64 bytes from 192.168.0.21: icmp_seq=2 ttl=64 time=0.676 ms
--- 192.168.0.21 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.451/0.563/0.676/0.114 ms
server02 ping server01
[root@3ef29b11da92 /]# ping -c 2 192.168.0.11
PING 192.168.0.11 (192.168.0.11) 56(84) bytes of data.
64 bytes from 192.168.0.11: icmp_seq=1 ttl=64 time=0.256 ms
64 bytes from 192.168.0.11: icmp_seq=2 ttl=64 time=0.667 ms
--- 192.168.0.11 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.256/0.461/0.667/0.206 ms
此方法的缺点：
1. 此方法配置的时候有时容器之间不能访问，容器内无法ping通外网（宿主机可以ping通）。重启服务器后，同样的操作，重新配置一遍后，然后就可以了。。。
2. pipework 分配静态ip是暂时的,重启之后就会失效。并且使用pipework绑定的ip 物理机,虚拟机，docker容器的ip都在同一网段,这在实际生产环境中是很难实现的。
原创作品，允许转载，转载时请务必以超链接形式标明文章 原始出处 、作者信息和本声明。
网友评论