一、高可用集群
（一）提升系统高可用性的解决方案：冗余(redundant)
-
工作模式
- active/passive：主备
- active/active：双主
-
以心跳方式通告
- active --> HEARTBEAT --> passive
- active <--> HEARTBEAT <--> active
-
故障处理
- failover：故障切换，即某资源的主节点故障时，将资源转移至其它节点的操作
- failback：故障移回，即某资源的主节点故障后重新修改上线后，将之前已转移至其它节点的资源重新切回的过程
（二）HA Cluster实现方案
-
ais：应用接口规范完备复杂的HA集群
RHCS：Red Hat Cluster Suite红帽集群套件
heartbeat
corosync -
vrrp协议实现：虚拟路由冗余协议
keepalived
二、KeepAlived基本介绍
（一）VRRP(Virtual Router Redundancy Protocol)协议术语
-
虚拟路由器：Virtual Router，多个物理路由器对外以一个IP地址提供服务，仿佛一台路由器
- 虚拟路由器标识：VRID(0-255)，唯一标识虚拟路由器
- VIP：Virtual IP，虚拟IP
- VMAC：Virutal MAC (00-00-5e-00-01-VRID)，虚拟MAC
-
物理路由器
master：主设备
backup：备用设备
priority：优先级
（二）KeepAlived的工作特性
-
通告：心跳，优先级等；周期性
-
工作方式：抢占式，非抢占式
-
安全认证：
- 无认证
- 简单字符认证：预共享密钥
- MD5
-
工作模式：
- 主/备：单虚拟路由器
- 主/主：主/备（虚拟路由器1），备/主（虚拟路由器2）
（三）KeepAlived的功能
-
vrrp协议完成地址流动
-
为vip地址所在的节点生成ipvs规则(在配置文件中预先定义)
-
为ipvs集群的各RS做健康状态检测
-
基于脚本调用接口通过执行脚本完成脚本中定义的功能，进而影响集群事务，以此支持nginx, haproxy等服务
三、KeepAlived的配置
（一）HA Cluster配置准备：
-
各节点时间必须同步：ntp服务（CentOS 6）, chrony（CentOS 7）
// 由于ntp/chrony服务不能同步差距过大的时间，需要先使用ntpdate命令同步一次，再开启服务 ntpdate ntp_server_ip // 开启chronyd服务（CentOS 7） vim /etc/chrony.conf server 172.18.0.1 iburst systemctl enable chronyd systemctl start chronyd // 开启ntp服务（CentOS 6） vim /etc/ntp.conf server 172.18.0.1 iburst chkconfig ntpd on service ntpd start -
确保iptables及selinux不会成为阻碍
-
各节点之间可通过主机名互相通信（对KA并非必须），建议使用/etc/hosts文件实现
-
各节点之间的root用户可以基于密钥认证的ssh服务完成互相通信（对KA并非必须）
ssh-keygen ssh-copy-id destination_ip
（二）KeepAlived的程序环境
-
主配置文件：/etc/keepalived/keepalived.conf
-
主程序文件：/usr/sbin/keepalived
-
Unit File：/usr/lib/systemd/system/keepalived.service
-
Unit File的环境配置文件：/etc/sysconfig/keepalived
（三）KeepAlived的配置文件结构
-
GLOBAL CONFIGURATION：全局设置
Global definitions
Static routes/addresses -
VRRPD CONFIGURATION：VRRP设置
VRRP synchronization group(s)：vrrp同步组
VRRP instance(s)：即一个vrrp虚拟路由器 -
LVS CONFIGURATION：LVS设置
Virtual server group(s)
Virtual server(s)：ipvs集群的vs和rs
（四）配置虚拟路由器
-
语法：
vrrp_instance <STRING> { .... } -
专用参数：
- state MASTER | BACKUP
当前节点在此虚拟路由器上的初始状态；只能有一个是MASTER，余下的都应该为BACKUP - interface IFACE_NAME
绑定为当前虚拟路由器使用的物理接口 - virtual_router_id VRID
当前虚拟路由器惟一标识，范围是0-255 - priority 100
当前物理节点在此虚拟路由器中的优先级；范围1-254 - advert_int 1
vrrp通告的时间间隔，默认1s - authentication：认证机制
authentication { auth_type AH|PASS auth_pass <PASSWORD> 仅前8位有效 }- virtual_ipaddress：虚拟IP
virtual_ipaddress { <IPADDR>/<MASK> brd <IPADDR> dev <STRING> scope <SCOPE> label <LABEL> }- track_interface：配置监控网络接口，一旦出现故障，则转为FAULT状态实现地址转移
track_interface { eth0 eth1 … }- nopreempt：定义工作模式为非抢占模式
- preempt_delay 300：抢占式模式，节点上线后触发新选举操作的延迟时长，默认模式
- 定义通知脚本：
notify_master <STRING> | <QUOTED-STRING>：
当前节点成为主节点时触发的脚本
notify_backup <STRING> | <QUOTED-STRING>：
当前节点转为备节点时触发的脚本
notify_fault <STRING> | <QUOTED-STRING>：
当前节点转为“失败”状态时触发的脚本
notify <STRING> | <QUOTED-STRING>：
通用格式的通知触发机制，一个脚本可完成以上三种状态的转换时的通知
- state MASTER | BACKUP
-
实验1：实现主/备虚拟路由器
- 实验环境：
物理路由器1：ip: 192.168.136.230, 主机名: node1, MASTER
物理路由器2：ip: 192.168.136.130, 主机名: node2, BACKUP
VIP：192.168.136.100
// 配置物理路由器1 vim /etc/keepalived/keepalived.conf global_defs { notification_email { root@localhost } notification_email_from node1@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id node1 // vrrp中的路由器主机名 vrrp_mcast_group4 224.0.0.58 // 设置组播ip地址 } vrrp_instance VI_1 { state MASTER interface ens37 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass dd73f9d6 // openssl rand -hex 4 生成8位16进制密码 } virtual_ipaddress { 192.168.136.100/24 } } systemctl start keepalived // 配置物理路由器2 vim /etc/keepalived/keepalived.conf global_defs { notification_email { root@localhost } notification_email_from node2@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id node2 vrrp_mcast_group4 224.0.0.58 } vrrp_instance VI_1 { state BACKUP interface ens37 virtual_router_id 51 priority 90 //作为BACKUP优先级比MASTER要低 advert_int 1 authentication { auth_type PASS auth_pass dd73f9d6 // 密码与node1相同 } virtual_ipaddress { 192.168.136.100/24 } } systemctl start keepalived- 测试
node1的ip地址已经出现VIP
监听组播地址的tcp连接
tcpdump -i ens37 -nn host 224.0.0.58，此时关闭node1的keepalived服务systemctl stop keepalived，自动由node2接管并开始声明自身拥有虚拟路由器的IP
VIP此时已经被node2接管
- 实验环境：
- 实验2：实现keepalived日志
vim /etc/sysconfig/keepalived
KEEPALIVED_OPTIONS="-D -S 3" // -D:详细日志，-S 3: 设置日志facility为local3
vim /etc/rsyslog.conf
local3.* /var/log/keepalived.log // 设置日志存储路径
systemctl restart rsyslog
systemctl restart keepalived
tail -f /var/log/keepalived.log
-
实验3：实现主/主虚拟路由器，并且当节点发生变化时主动发送邮件
- 实验环境
物理路由器1：ip: 192.168.136.230, 主机名: node1
物理路由器2：ip: 192.168.136.130, 主机名: node2
虚拟路由器1：MASTER: node1, BACKUP: node2, VIP: 192.168.136.100
虚拟路由器2：MASTER: node2, BACKUP: node1, VIP: 192.168.136.200
// 配置物理路由器1（虚拟路由器1的MASTER，虚拟路由器2的BACKUP） vim /etc/keepalived/keepalived.conf smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id node1 vrrp_mcast_group4 224.0.0.58 } // 虚拟路由器1的设置 vrrp_instance VI_1 { state MASTER interface ens37 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass dd73f9d6 } virtual_ipaddress { 192.168.136.100/24 } notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault" } // 虚拟路由器2的设置 vrrp_instance VI_2 { state BACKUP interface ens37 virtual_router_id 61 priority 80 advert_int 1 authentication { auth_type PASS auth_pass a56c19be } virtual_ipaddress { 192.168.136.200/24 } notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault" } systemctl restart keepalived // 配置物理路由器2（虚拟路由器1的BACKUP，虚拟路由器2的MASTER） vim /etc/keepalived/keepalived.conf global_defs { notification_email { root@localhost } notification_email_from node2@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id node2 vrrp_mcast_group4 224.0.0.58 } // 虚拟路由器1的设置 vrrp_instance VI_1 { state BACKUP interface ens37 virtual_router_id 51 priority 90 advert_int 1 authentication { auth_type PASS auth_pass dd73f9d6 } virtual_ipaddress { 192.168.136.100/24 } notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault" } // 虚拟路由器2的设置 vrrp_instance VI_2 { state MASTER interface ens37 virtual_router_id 61 priority 100 advert_int 1 authentication { auth_type PASS auth_pass a56c19be } virtual_ipaddress { 192.168.136.200/24 } notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault" } // 在物理路由器1,2上添加脚本文件 vim /etc/keepalived/notify.sh #! /bin/bash contact='root@localhost' notify() { mailsubject="$(hostname) to be $1, vip floating" mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1" echo "$mailbody" | mail -s "$mailsubject" $contact } case $1 in master) notify master ;; backup) notify backup ;; fault) notify fault ;; *) echo "Usage: $(basename $0) {master|backup|fault}" exit 1 ;; esac chmod +x /etc/keepalived/notify.sh- 测试
监听组播地址的tcp连接tcpdump -i ens37 -nn host 224.0.0.58，可以看到node1, node2分别声明拥有虚拟路由器1(vrid 51)、虚拟路由器2(vrid61)的IP地址
分别查看node1和node2的网卡IP地址，进一步确认上述结果
此时，断开node1的网络连接
虚拟路由器1的VIP立即由node2的网卡接管
恢复node1的网络连接，在node1和node2上都可以看到相应的邮件通知：
node1上通知出错，很快通知自身被切换为BACKUP，恢复网络连接后通知自身重新变为MASTER；
node2上通知自身切换为MASTER，恢复网络连接后通知自身切换为BACKUP
- 实验环境
（五）Keepalived支持IPVS
- 语法：
virtual_server {IP port | fwmark int}
{
...
real_server{
...
}
...
}
-
virtual_server常用参数
- delay_loop <INT>
检查后端服务器的时间间隔 - lb_algo rr|wrr|lc|wlc|lblc|sh|dh
定义调度方法 - lb_kind NAT|DR|TUN
集群的类型 - persistence_timeout <INT>
持久连接时长 - protocol TCP
服务协议，仅支持TCP - sorry_server<IPADDR> <PORT>
所有RS故障时，备用服务器地址
- delay_loop <INT>
-
real_server <IPADDR> <PORT>常用参数
- weight <INT>
RS权重 - notify_up <STRING>|<QUOTED-STRING>
RS上线通知脚本 - notify_down <STRING>|<QUOTED-STRING>
RS下线通知脚本 - HTTP_GET|SSL_GET|TCP_CHECK|SMTP_CHECK|MISC_CHECK { ... }
定义当前主机的健康状态检测方法
- weight <INT>
-
HTTP_GET|SSL_GET：应用层健康状态检测
HTTP_GET|SSL_GET { url { path <URL_PATH> // 定义要监控的URL status_code <INT> // 判断上述检测机制为健康状态的响应码 digest <STRING> // 判断为健康状态的响应的内容的校验码 } connect_timeout <INTEGER> // 连接请求的超时时长 nb_get_retry <INT> // 重试次数 delay_before_retry <INT> // 重试之前的延迟时长 connect_ip <IP ADDRESS> // 向当前RS哪个IP地址发起健康状态检测请求 connect_port <PORT> // 向当前RS的哪个PORT发起健康状态检测请求 bindto <IP ADDRESS> // 发出健康状态检测请求时使用的源地址 bind_port <PORT> // 发出健康状态检测请求时使用的源端口 } -
TCP_CHECK参数
- connect_ip <IP ADDRESS>
向当前RS的哪个IP地址发起健康状态检测请求 - connect_port <PORT>
向当前RS的哪个PORT发起健康状态检测请求 - bindto <IP ADDRESS>
发出健康状态检测请求时使用的源地址 - bind_port <PORT>
发出健康状态检测请求时使用的源端口 - connect_timeout <INTEGER>
连接请求的超时时长
- connect_ip <IP ADDRESS>
-
实验4：实现主/备模型的IPVS集群
- 实验环境：
LB1(master)/VS：IP: 192.168.136.230
LB2(backup)/VS：IP: 192.168.136.130
VIP：192.168.136.100
RS1：IP: 192.168.136.229
RS2：IP: 192.168.136.129
// 配置LB1的keepalived设置 vim /etc/keepalived/keepalived.conf global_defs { notification_email { root@localhost } notification_email_from node1@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id node1 vrrp_mcast_group4 224.0.0.58 } vrrp_instance VI_1 { state MASTER interface ens37 virtual_router_id 51 priority 100 advert_int 1 authentication { auth_type PASS auth_pass dd73f9d6 } virtual_ipaddress { 192.168.136.100/24 } notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault" } virtual_server 192.168.136.100 80{ delay_loop 3 lb_algo wrr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.136.229 80{ weight 2 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.136.129 80{ weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } // 配置LB2的keepalived设置 vim /etc/keepalived/keepalived.conf global_defs { notification_email { root@localhost } notification_email_from node2@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id node2 vrrp_mcast_group4 224.0.0.58 } vrrp_instance VI_1 { state BACKUP interface ens37 virtual_router_id 51 priority 90 advert_int 1 authentication { auth_type PASS auth_pass dd73f9d6 } virtual_ipaddress { 192.168.136.100/24 } notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault" } virtual_server 192.168.136.100 80{ delay_loop 3 lb_algo wrr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.136.229 80{ weight 2 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.136.129 80{ weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } // 配置LB1, LB2的sorry server服务 echo sorry on LB1 > /var/www/html/index.html // LB1上操作 echo sorry on LB2 > /var/www/html/index.html // LB2上操作 systemctl start httpd // 配置RS1, RS2的Web服务 echo RS1 homepage > /var/www/html/index.html // RS1上操作 echo RS2 homepage > /var/www/html/index.html // RS2上操作 systemctl start httpd // 编辑脚本实现：禁止RS响应ARP请求，并将网卡绑定VIP vim lvs_dr_rs.sh #! /bin/bash vip='192.168.136.100' mask='255.255.255.255' dev=lo:1 rpm -q httpd &> /dev/null || yum -y install httpd &>/dev/null service httpd start &> /dev/null && echo "The httpd Server is Ready!" case $1 in start) echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce ifconfig $dev $vip netmask $mask broadcast $vip up echo "The RS Server is Ready!" ;; stop) ifconfig $dev down echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce echo "The RS Server is Canceled!" ;; *) echo "Usage: $(basename $0) start|stop" exit 1 ;; esac chmod +x lvs_dr_rs.sh bash lvs_dr_rs.sh start // LB1, LB2启动KeepAlived服务，进行测试 systemctl start keepalived访问VIP(192.168.136.100)的Web服务，正常工作
停止RS2的Web服务，自动进行健康检查，全部调度至RS1
停止RS1的Web服务，自动进行健康检查，调度至LB1的sorry server
停止LB1的KeepAlived服务，自动切换至LB2
- 实验环境：
-
实验5：实现主/主模型的IPVS集群
- 实验环境：
LB1/VS1：IP: 192.168.136.230，后端RS: RS1, RS2
LB2/VS2：IP: 192.168.136.130，后端RS: RS3, RS4
LB1 VIP：192.168.136.100
LB2 VIP：192.168.136.200
RS1：IP: 192.168.136.229
RS2：IP: 192.168.136.129
RS3：IP: 192.168.136.240
RS4：IP: 192.168.136.250
LB之间互为MASTER与BACKUP的关系
MASTER：LB1，BACKUP：LB2
MASTER：LB2，BACKUP：LB1
// 配置LB1, LB2的keepalived设置 global_defs { notification_email { root@localhost } notification_email_from node1@localhost // LB1上操作 notification_email_from node1@localhost // LB2上操作 smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id node1 // LB1上操作 router_id node2 // LB2上操作 vrrp_mcast_group4 224.0.0.58 } vrrp_instance VI_1 { state MASTER // LB1上操作 state BACKUP // LB2上操作 interface ens37 virtual_router_id 51 priority 100 // LB1上操作 priority 90 // LB2上操作 advert_int 1 authentication { auth_type PASS auth_pass dd73f9d6 } virtual_ipaddress { 192.168.136.100/24 } notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault" } vrrp_instance VI_2 { state BACKUP // LB1上操作 state MASTER // LB2上操作 interface ens37 virtual_router_id 61 priority 80 // LB1上操作 priority 100 // LB2上操作 advert_int 1 authentication { auth_type PASS auth_pass a56c19be } virtual_ipaddress { 192.168.136.200/24 } notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault" } virtual_server 192.168.136.100 80{ delay_loop 3 lb_algo wrr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.136.229 80{ weight 2 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.136.129 80{ weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } virtual_server 192.168.136.200 80{ delay_loop 3 lb_algo wrr lb_kind DR protocol TCP sorry_server 127.0.0.1 80 real_server 192.168.136.240 80{ weight 2 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } real_server 192.168.136.250 80{ weight 1 HTTP_GET { url { path / status_code 200 } connect_timeout 1 nb_get_retry 3 delay_before_retry 1 } } } // 配置LB1, LB2的sorry server服务 echo sorry on LB1 > /var/www/html/index.html // LB1上操作 echo sorry on LB2 > /var/www/html/index.html // LB2上操作 systemctl start httpd // 配置RS1, RS2, RS3, RS4的Web服务 echo RS1 homepage > /var/www/html/index.html // RS1上操作 echo RS2 homepage > /var/www/html/index.html // RS2上操作 echo RS3 homepage > /var/www/html/index.html // RS3上操作 echo RS4 homepage > /var/www/html/index.html // RS4上操作 systemctl start httpd // 编辑脚本实现：禁止RS响应ARP请求，并将网卡绑定VIP vim lvs_dr_rs.sh #! /bin/bash vip='192.168.136.100' // RS1, RS2上操作 vip='192.168.136.200' // RS3, RS4上操作 mask='255.255.255.255' dev=lo:1 rpm -q httpd &> /dev/null || yum -y install httpd &>/dev/null service httpd start &> /dev/null && echo "The httpd Server is Ready!" case $1 in start) echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce ifconfig $dev $vip netmask $mask broadcast $vip up echo "The RS Server is Ready!" ;; stop) ifconfig $dev down echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce echo "The RS Server is Canceled!" ;; *) echo "Usage: $(basename $0) start|stop" exit 1 ;; esac chmod +x lvs_dr_rs.sh bash lvs_dr_rs.sh start // LB1, LB2启动KeepAlived服务，进行测试 systemctl start keepalived - 实验环境：
使用ipvsadm -Ln命令查看ipvs调度策略，与KeepAlived的配置吻合
访问VIP1, VIP2(192.168.136.100, 192.168.136.200)的Web服务，正常工作
停止RS1的Web服务，自动进行健康检查，全部调度至RS2
停止RS2的Web服务，自动进行健康检查，调度至LB1的sorry server
停止LB1的KeepAlived服务，自动切换至LB2
停止RS3的Web服务，自动进行健康检查，全部调度至RS4
停止RS4的Web服务，自动进行健康检查，调度至LB2的sorry server
（六）Keepalived调用脚本进行资源监控
-
keepalived调用外部的辅助脚本进行资源监控，并根据监控的结果状态实现优先动态调整
-
vrrp_script：自定义资源监控脚本，vrrp实例根据脚本返回值，公共定义，可被多个实例调用，定义在vrrp实例之外
-
track_script：调用vrrp_script定义的脚本去监控资源，定义在实例之内，调用事先定义的vrrp_script
- 分两步：(1) 先定义一个脚本；(2) 调用此脚本
格式：
// 定义脚本，定义在实例外 vrrp_script <SCRIPT_NAME> { script "" // 引号内为脚本命令 interval INT weight -INT } // 调用脚本，定义在实例内 track_script { SCRIPT_NAME_1 SCRIPT_NAME_2 } - 分两步：(1) 先定义一个脚本；(2) 调用此脚本
-
实验6：实现主/主模型的高可用Nginx反向代理
- 实验环境：
LB1/VS1：IP: 192.168.136.230，后端RS: RS1, RS2
LB2/VS2：IP: 192.168.136.130，后端RS: RS3, RS4
LB1 VIP：192.168.136.100
LB2 VIP：192.168.136.200
RS1：IP: 192.168.136.229
RS2：IP: 192.168.136.129
RS3：IP: 192.168.136.240
RS4：IP: 192.168.136.250
LB之间互为MASTER与BACKUP的关系
MASTER：LB1，BACKUP：LB2
MASTER：LB2，BACKUP：LB1
// 配置LB1, LB2的KeepAlived设置 vim /etc/keepalived/keepalived.conf global_defs { notification_email { root@localhost } notification_email_from node1@localhost // LB1上操作 notification_email_from node2@localhost // LB2上操作 smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id node1 // LB1上操作 router_id node2 // LB2上操作 vrrp_mcast_group4 224.0.0.58 } vrrp_script chk_nginx { script "killall -0 nginx && exit 0 || exit 1;" interval 1 weight -20 fall 3 rise 3 } vrrp_instance VI_1 { state MASTER // LB1上操作 state BACKUP // LB2上操作 interface ens37 virtual_router_id 51 priority 100 // LB1上操作 priority 90 // LB2上操作 advert_int 1 authentication { auth_type PASS auth_pass dd73f9d6 } virtual_ipaddress { 192.168.136.100/24 } notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault" // 下面的脚本引用仅在LB1的配置文件出现 track_script { chk_nginx } } vrrp_instance VI_2 { state BACKUP // LB1上操作 state MASTER // LB2上操作 interface ens37 virtual_router_id 61 priority 90 // LB1上操作 priority 100 // LB2上操作 advert_int 1 authentication { auth_type PASS auth_pass a56c19be } virtual_ipaddress { 192.168.136.200/24 } notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault" // 下面的脚本引用仅在LB2的配置文件出现 track_script { chk_nginx } } // 配置LB1,LB2的nginx反向代理 vim /etc/nginx/nginx.conf http { upstream websrvs1 { server 192.168.136.229:80 weight=2; server 192.168.136.129:80 weight=1; } upstream websrvs2 { server 192.168.136.240:80 weight=2; server 192.168.136.250:80 weight=1; } server { listen 192.168.136.100:80; location / { proxy_pass http://websrvs1; } } server { listen 192.168.136.200:80; location / { proxy_pass http://websrvs2; } } } nginx -t systemctl start nginx // 配置RS1, RS2, RS3, RS4的Web服务 echo RS1 homepage > /var/www/html/index.html // RS1上操作 echo RS2 homepage > /var/www/html/index.html // RS2上操作 echo RS3 homepage > /var/www/html/index.html // RS3上操作 echo RS4 homepage > /var/www/html/index.html // RS4上操作 systemctl start httpd // LB1, LB2启动KeepAlived服务，进行测试 systemctl start keepalived登录192.168.136.100和192.168.136.200的web服务，确实按照设置要求调度
停止RS2的httpd服务，全部调度至RS1
停止RS3的httpd服务，全部调度至RS4
关闭LB2的nginx反向代理服务，通过
tcpdump -i ens37 -nn host 224.0.0.58查看组播情况。三个红框依次表达：
（1）未关闭nginx前的组播状态
（2）关闭nginx后，LB2的vrid 61权重减去20变作80，而LB1vrid 61的权重为90
（3）由于LB1的权重高，VIP2的所有权被LB1接管
关闭LB1的nginx反向代理服务，通过
tcpdump -i ens37 -nn host 224.0.0.58查看组播情况。三个红框依次表达：
（1）未关闭nginx前的组播状态
（2）关闭nginx后，LB1的vrid 51权重减去20变作80，而LB2的vrid 51权重为90
（3）由于LB2的权重高，VIP1的所有权被LB2接管
由于此时两个nginx反向代理均关闭，故访问192.168.136.100和192.168.136.200的web服务全部失败
打开LB2的nginx反向代理服务，通过
tcpdump -i ens37 -nn host 224.0.0.58查看组播情况。三个红框依次表达：
（1）未打开nginx前的组播状态
（2）打开nginx后，LB2的vrid 61权重增加20变作100，而LB1的vrid 61权重为90
（3）由于LB2的权重高，VIP2的所有权被LB2接管
此时VIP1和VIP2均由LB2上的nginx服务器进行反向代理，192.168.136.100和192.168.136.200的web服务全部恢复
- 实验环境：
（七）Keepalived同步组
-
LVS NAT模型VIP和DIP需要同步，需要同步组
-
格式：
vrrp_sync_group VG_1 { group { VI_1 # name of vrrp_instance(below) VI_2 # One for each moveable IP. } } vrrp_instance VI_1 { eth0 vip } vrrp_instance VI_2 { eth1 dip }
网友评论