美文网首页
NEUTRON: VROUTER HA机制的实现

NEUTRON: VROUTER HA机制的实现

作者: 圣地亚哥_SVIP | 来源:发表于2019-12-26 16:26 被阅读0次

    neutron vrouter属于单点，为了保障高可用，利用keepalived实现VPC GW及EIP的漂移。本文中，介绍如何利用keepalived实现vrouter的高可用。

    neutron-ha

    HA配置

    vrouter有四类接口：

    • loopback口
    • 网关： VPC的网关
    • EIP: floating ip的接口
    • HA: keepalived track的口，具有独立的VPC网络

    HA IP信息:

    • 169.254.192.1/18
    • 169.254.192.2/18
    • VIP: 169.254.0.1/24
    NETWORK2上添加vrouter

    在Blog已经完成一个network1上vrouter的创建和测试

    首先Down Network1上vrouter的网关及外部接口：

    # ip netns exec qrouter-123 ip link set dev qr-123 down
# ip netns exec qrouter-123 ip link set dev qg-123 down

    NETWORK2:

    添加vrouter及设置外部网络，network2复用之前COMPUTE2：

    # ovs-vsctl add-port br-int qr-123 -- set Interface qr-123 type=internal -- set port qr-123 tag=22
# ip netns add qrouter-123
# ip link set dev qr-123 netns qrouter-123
# ip netns exec qrouter-123 ip link set qr-123 up
# ip netns exec qrouter-123 ip addr add 192.168.10.1/24 dev qr-123

    外部连接的参数，参考上一篇Blog：

    • Out Vlan: 1102
    • Local Vlan: 60
    • CIDR： 18.18.18.0/24

    添加本地br-ex-biz网桥：

    # ovs-vsctl add-br br-ex-biz
# ovs-vsctl add-port br-ex-biz ex-biz--int -- set interface ex-biz--int type=patch -- set interface ex-biz--int options:peer=int--ex-biz
# ovs-vsctl add-port br-int int--ex-biz -- set interface int--ex-biz type=patch -- set interface int--ex-biz options:peer=ex-biz--int

    添加qg-xxx端口：

    # ovs-vsctl add-port br-int qg-123 -- set Interface qg-123 type=internal -- set port qg-123 tag=60
# ip link set dev qg-123 netns qrouter-123
# ip netns exec qrouter-123 ip link set qg-123 up
# ip netns exec qrouter-123 ip addr add 18.18.18.10/24 dev qg-123

    流表添加：

    # ovs-ofctl add-flow br-ex-biz 'cookie=0x79, table=0, priority=4,in_port=1,dl_vlan=60 actions=mod_vlan_vid:1102,NORMAL'
# ovs-ofctl add-flow br-ex-biz 'cookie=0x79, table=0, priority=2,in_port=1 actions=drop'
# ovs-ofctl add-flow br-ex-biz 'cookie=0x79, table=0, priority=0 actions=NORMAL'
此处in_port为：int--ex-biz
# ovs-ofctl add-flow br-int 'cookie=0x79,table=0, priority=3,in_port=5,dl_vlan=1102 actions=mod_vlan_vid:60,NORMAL'
# ovs-ofctl add-flow br-int 'cookie=0x79,table=0, priority=2,in_port=5 actions=drop'

    设置外部端口(此处主要用于测试，实际中外部网络的网关应在交换机中)：

    # ip netns add outer
# ovs-vsctl add-port br-ex-biz outer-123 -- set Interface outer-123 type=internal
  
# ip link set dev outer-123 netns outer
# ip netns exec outer ip link set outer-123 up
# ip netns exec outer ip addr add 18.18.18.1/24 dev outer-123
  
# ovs-vsctl set port outer-123 tag=1102

    NAT功能添加：

    # ip netns exec qrouter-123 iptables -t nat -A POSTROUTING  -o qg-123 -j SNAT --to-source 18.18.18.10
# ip netns exec qrouter-123 iptables -t nat -A POSTROUTING -m mark ! --mark 0x2/0xffff -m conntrack --ctstate DNAT -j SNAT --to-source 18.18.18.10
# ip netns exec qrouter-123 iptables -t mangle -I PREROUTING -i qg-123 -j MARK --set-xmark 0x2/0xffff
  
# ip netns exec qrouter-123 route add -net Default gw 18.18.18.1
# ip netns exec test route add -net 0.0.0.0 gw 192.168.10.1
# ip netns exec qrouter-123 sysctl -w net.ipv4.ip_forward=1

    测试此vrouter的功能是否正常：

    # ip netns exec test ping 18.18.18.1
PING 18.18.18.1 (18.18.18.1) 56(84) bytes of data.
64 bytes from 18.18.18.1: icmp_seq=1 ttl=63 time=0.805 ms
64 bytes from 18.18.18.1: icmp_seq=2 ttl=63 time=0.073 ms
64 bytes from 18.18.18.1: icmp_seq=3 ttl=63 time=0.064 ms
64 bytes from 18.18.18.1: icmp_seq=4 ttl=63 time=0.062 ms
^C
--- 18.18.18.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3000ms
rtt min/avg/max/mdev = 0.062/0.251/0.805/0.319 ms

    vrouter功能验证正常。

    清理vrouter中的IP信息：

    NETWORK2:

    # ip netns exec qrouter-123 ip addr del 18.18.18.10/24 dev qg-123
# ip netns exec qrouter-123 ip addr del 192.168.10.1/24 dev qr-123

    NETWORK1：

    # ip netns exec qrouter-123 ip addr del 18.18.18.10/24 dev qg-123
# ip netns exec qrouter-123 ip addr del 192.168.10.1/24 dev qr-123
# ip netns exec qrouter-123 ip link set dev qr-123 up
# ip netns exec qrouter-123 ip link set dev qg-123 up
    创建HA的网络

    参数：

    • vni: 0x43
    • network1,local vlan: 113
    • network2,local vlan: 123

    添加HA接口：

    NETWORK1：

    # ovs-vsctl add-port br-int ha-123 -- set interface ha-123 type=internal -- set port ha-123 tag=113
# ip link set dev ha-123 netns qrouter-123
# ip netns exec qrouter-123 ip link set dev ha-123 up
# ip netns exec qrouter-123 ip addr add 169.254.192.1/18 dev ha-123

    NETWORK2:

    # ovs-vsctl add-port br-int ha-123 -- set interface ha-123 type=internal -- set port ha-123 tag=123
# ip link set dev ha-123 netns qrouter-123
# ip netns exec qrouter-123 ip link set dev ha-123 up
# ip netns exec qrouter-123 ip addr add 169.254.192.2/18 dev ha-123

    添加流表

    NETWORK1：

    # ovs-ofctl add-flow br-tun 'cookie=0x79,  table=4,  priority=1,tun_id=0x43 actions=mod_vlan_vid:113,resubmit(,10)'
# ovs-ofctl add-flow br-tun 'cookie=0x79, table=22, dl_vlan=113 actions=strip_vlan,set_tunnel:0x43,output:8'

    NETWORK2：

    # ovs-ofctl add-flow br-tun 'cookie=0x79,  table=4,  priority=1,tun_id=0x43 actions=mod_vlan_vid:123,resubmit(,10)'
# ovs-ofctl add-flow br-tun 'cookie=0x79, table=22, dl_vlan=123 actions=strip_vlan,set_tunnel:0x43,output:7'

    HA网络的连通性

    Network2:

    # ip netns exec qrouter-123 ping 169.254.192.1
PING 169.254.192.1 (169.254.192.1) 56(84) bytes of data.
64 bytes from 169.254.192.1: icmp_seq=1 ttl=64 time=2.74 ms
64 bytes from 169.254.192.1: icmp_seq=2 ttl=64 time=0.488 ms
64 bytes from 169.254.192.1: icmp_seq=3 ttl=64 time=0.485 ms
64 bytes from 169.254.192.1: icmp_seq=4 ttl=64 time=0.481 ms
^C
--- 169.254.192.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3002ms
rtt min/avg/max/mdev = 0.481/1.050/2.746/0.979 ms
    1. 配置keepalived

    系统配置：

    # echo 'net.ipv4.ip_nonlocal_bind = 1' >> /etc/sysctl.conf
# sysctl -p

# iptables -I INPUT -p vrrp -j ACCEPT
# service iptables save

    配置如下，/home/keepalived/keepalived.conf：

    vrrp_instance VR_1 {
    state BACKUP
    interface ha-123
    virtual_router_id 24
    priority 50
    garp_master_delay 60
    nopreempt
    advert_int 2
    track_interface {
        ha-123
    }
    virtual_ipaddress {
        169.254.0.1/24 dev ha-123
    }
    virtual_ipaddress_excluded {
        192.168.10.1/24 dev qr-123
        18.18.18.10/24 dev qg-123
    }
    virtual_routes {
        0.0.0.0/0 via 18.18.18.1 dev qg-123
    }
}

    运行keepalived进程：

    # ip netns exec qrouter-123 keepalived -P -f /home/keepalived/keepalived.conf
    HA测试

    NETWORK1：

    #  ip netns exec qrouter-123 ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
154: qr-123: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 46:b2:c3:95:d6:20 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::44b2:c3ff:fe95:d620/64 scope link 
       valid_lft forever preferred_lft forever
156: qg-123: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether de:86:a9:7b:b0:40 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::dc86:a9ff:fe7b:b040/64 scope link 
       valid_lft forever preferred_lft forever
158: qr-124: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 66:c7:ae:94:b9:3a brd ff:ff:ff:ff:ff:ff
    inet6 fe80::64c7:aeff:fe94:b93a/64 scope link 
       valid_lft forever preferred_lft forever
161: ha-123: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 52:27:5d:9c:3c:83 brd ff:ff:ff:ff:ff:ff
    inet 169.254.192.1/18 scope global ha-123
       valid_lft forever preferred_lft forever
    inet6 fe80::5027:5dff:fe9c:3c83/64 scope link 
       valid_lft forever preferred_lft forever

    NETWORK2：

    # ip netns exec qrouter-123 ip a
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
95: qr-123: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 82:a7:97:58:91:7a brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.1/24 scope global qr-123
       valid_lft forever preferred_lft forever
    inet6 fe80::80a7:97ff:fe58:917a/64 scope link 
       valid_lft forever preferred_lft forever
97: qg-123: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 3e:44:c1:35:a7:cf brd ff:ff:ff:ff:ff:ff
    inet 18.18.18.10/24 scope global qg-123
       valid_lft forever preferred_lft forever
    inet6 fe80::3c44:c1ff:fe35:a7cf/64 scope link 
       valid_lft forever preferred_lft forever
99: ha-123: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether b2:f4:a3:4c:27:82 brd ff:ff:ff:ff:ff:ff
    inet 169.254.192.2/18 scope global ha-123
       valid_lft forever preferred_lft forever
    inet 169.254.0.1/24 scope global ha-123
       valid_lft forever preferred_lft forever
    inet6 fe80::b0f4:a3ff:fe4c:2782/64 scope link 
       valid_lft forever preferred_lft forever

    如上，我们可以看出，网关IP及外部IP都只起在其中一个节点上。

    连通性如下：

    PING 网关：
# ip netns exec test ping 192.168.10.1
PING 192.168.10.1 (192.168.10.1) 56(84) bytes of data.
64 bytes from 192.168.10.1: icmp_seq=1 ttl=64 time=0.799 ms
64 bytes from 192.168.10.1: icmp_seq=2 ttl=64 time=0.084 ms
64 bytes from 192.168.10.1: icmp_seq=3 ttl=64 time=0.091 ms
64 bytes from 192.168.10.1: icmp_seq=4 ttl=64 time=0.073 ms
^C
--- 192.168.10.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3000ms
rtt min/avg/max/mdev = 0.073/0.261/0.799/0.310 ms

PING外部网络的网关：
# ip netns exec test ping 18.18.18.1
PING 18.18.18.1 (18.18.18.1) 56(84) bytes of data.
64 bytes from 18.18.18.1: icmp_seq=1 ttl=63 time=0.860 ms
64 bytes from 18.18.18.1: icmp_seq=2 ttl=63 time=0.080 ms
64 bytes from 18.18.18.1: icmp_seq=3 ttl=63 time=0.066 ms
^C
--- 18.18.18.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.066/0.335/0.860/0.371 ms

    测试IP漂移：

    NETWORK1：

    关闭ha口
#ip netns exec qrouter-123 ip link set dev ha-123 down

    查看NETWORK2的VROUTER:

    # ip netns exec qrouter-123 ip a
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
95: qr-123: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 82:a7:97:58:91:7a brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.1/24 scope global qr-123
       valid_lft forever preferred_lft forever
    inet6 fe80::80a7:97ff:fe58:917a/64 scope link 
       valid_lft forever preferred_lft forever
97: qg-123: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 3e:44:c1:35:a7:cf brd ff:ff:ff:ff:ff:ff
    inet 18.18.18.10/24 scope global qg-123
       valid_lft forever preferred_lft forever
    inet6 fe80::3c44:c1ff:fe35:a7cf/64 scope link 
       valid_lft forever preferred_lft forever
99: ha-123: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether b2:f4:a3:4c:27:82 brd ff:ff:ff:ff:ff:ff
    inet 169.254.192.2/18 scope global ha-123
       valid_lft forever preferred_lft forever
    inet 169.254.0.1/24 scope global ha-123
       valid_lft forever preferred_lft forever
    inet6 fe80::b0f4:a3ff:fe4c:2782/64 scope link 
       valid_lft forever preferred_lft forever

    可以发现网关IP，及外部网络IP都漂移至NETWORK2上，测试一下连通性：

    # ip netns exec test ping 192.168.10.1
PING 192.168.10.1 (192.168.10.1) 56(84) bytes of data.
64 bytes from 192.168.10.1: icmp_seq=1 ttl=64 time=0.465 ms
64 bytes from 192.168.10.1: icmp_seq=2 ttl=64 time=0.062 ms
^C
--- 192.168.10.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.062/0.263/0.465/0.202 ms
[root@youngster keepalived]# ip netns exec test ping 18.18.18.1
PING 18.18.18.1 (18.18.18.1) 56(84) bytes of data.
64 bytes from 18.18.18.1: icmp_seq=1 ttl=63 time=0.428 ms
64 bytes from 18.18.18.1: icmp_seq=2 ttl=63 time=0.065 ms
64 bytes from 18.18.18.1: icmp_seq=3 ttl=63 time=0.062 ms
^C
--- 18.18.18.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.062/0.185/0.428/0.171 ms

    keepalived配置说明：

    vrrp_instance VR_1 {
    当前节点在此虚拟路由器上的初始状态
    state BACKUP
    设置实例绑定的网卡 VRRP心跳包从哪块网卡发出
    interface ha-123
    当前虚拟路由器的惟一标识，范围是0-255
    virtual_router_id 1
    当前主机在此虚拟路径器中的优先级；范围1-254
    priority 50
    当切为主状态后多久更新ARP缓存
    garp_master_delay 60
    不抢占
    nopreempt
    检查间隔，2s VRRP心跳包的发送周期
    advert_int 2
    监控网卡
    track_interface {
        ha-123
    }
    设置VIP
    virtual_ipaddress {
        169.254.0.1/24 dev ha-123
    }
    virtual_ipaddress_excluded {
        192.168.10.1/24 dev qr-123
        18.18.18.10/24 dev qg-123
    }
    默认路由
    virtual_routes {
        0.0.0.0/0 via 18.18.18.1 dev qg-123
    }
}

    相关文章

      网友评论

          本文标题：NEUTRON: VROUTER HA机制的实现

          本文链接：https://www.haomeiwen.com/subject/dhlfoctx.html

          延伸阅读

          深度阅读

          • 您也可以注册成为美文阅读网的作者，发表您的原创作品、分享您的心情！

          栏目导航

          热点阅读

          关于我们|服务条款|联系我们|NEUTRON: VROUTER HA机制的实现|投稿指南|网站地图|RSS订阅|排版工具|手机版

          提供经典美文摘抄,优美散文欣赏,现代诗歌精选,短篇小说,心情随笔,表白情书范文,故事会在线阅读欣赏

          Copyright © 2014-2023 Haomeiwen.com All Rights Reserved. 好美文阅读网 版权所有

          备案信息：桂公网安备 45052102000051号 · 桂ICP备13007215号-3

          本站所收录作品、热点评论等信息部分来源互联网，目的只是为了系统归纳学习和传递资讯

          所有作品版权归原创作者所有，与本站立场无关，如不慎侵犯了你的权益，请联系我们告知，我们将做删除处理！