Ubuntu下用docker实现keepalived+lvs集群

使用docker模拟两台机器，

1、docker运行镜像

docker运行就不多提了，需要注意加上--privileged，获取真正的root权限；另外镜像里如果没有modprobe命令，apt-get install kmod

2、安装ipvsadm和keepalived

分别apt安装ipvsadm和keepalived，然后分别在两台机上编辑/etc/keepalived/keepalived.conf

global_defs {
    router_id webHA #标识
}

vrrp_instance VI_1 {    
    state MASTER    # 备用机为BACKUP
    interface eth0
    virtual_router_id 51    # 一般不用改
    priority 100   
    advert_int 1    
    nopreempt
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        172.17.0.10
    }
}

virtual_server 172.17.0.10 8080 {
    delay_loop 5
    lb_algo rr
    lb_kind DR
    persistence_timeout 10
    protocol TCP
    real_server 172.17.0.2 8080 { #和vip的端口一致
        weight 1            
        TCP_CHECK {
            connect_timeout 5
            nb_get_retry 2
            delay_before_retry 3
            connect_port 8080
        }
    }
    real_server 172.17.0.3 8080 { #和vip的端口一致
        weight 1
        TCP_CHECK {
            connect_timeout 5
            nb_get_retry 2
            delay_before_retry 3
            connect_port 8080
        }
    }
}

service keepalived restart，这个时候就可以VIP切换了
keepalived日志如果看不了，修改/etc/rsyslog.d/50-default.conf 文件，将注释行取消注释

#*.=info;*.=notice;*.=warn;\
#       auth,authpriv.none;\
#       cron,daemon.none;\
#       mail,news.none          -/var/log/messages

重启rsyslog
如果报IPVS: Can't initialize ipvs: Protocol not available，运行ipvsadm；运行ipvsadm如果报Can't initialize ipvs: No space left on device，试试用sudo

3、负载均衡

上面只是实现了VIP切换，负载均衡还没成功配置，需要配置realserver，新建/etc/init.d/realserver

#!/bin/bash
# description: Config realserver lo and apply noarp 
#Written by :NetSeek http://www.linuxtone.org

VIP=192.168.0.10

. /lib/init/vars.sh
. /lib/lsb/init-functions

case "$1" in
start)
    ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP
    /sbin/route add -host $VIP dev lo:0               
    echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
    echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
    sysctl -p >/dev/null 2>&1
    echo "RealServer Start OK"

    ;;
stop)
    ifconfig lo:0 down
    route del $VIP >/dev/null 2>&1              
    echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
    echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
    echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
    echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
    echo "RealServer Stoped"
    ;;
*)
    echo "Usage: $0 {start|stop}"
    exit 1
esac

exit 0

但是没成功:)，之后成功再更新...

4、开放端口

如果要将docker中的VIP端口暴露在宿主机上，需要做端口转发，查了下有挺多种做法：修改路由、SSH隧道、rinetd和nginx tcp代理，这里直接使用nginx。宿主机安装nginx，并在/etc/nginx/nginx.conf最后添加:

stream {
       upstream testsocket {
               server 172.17.0.10:8080;
       }
       server {
               listen 8000;
               proxy_pass testsocket;
       }
}