neutron ovn 中的路由对照 kube-ovn

作者: cloudFans | 来源:发表于2022-11-14 16:33 被阅读0次

neutron ovn 中的路由对照 kube-ovn
kube-ovn 对照 neutron ovn
关于kube-ovn 路由的一些学习和理解
关于kube-ovn ipam性能进一步优化
kube-ovn简记
patch does not apply 的问题
关于kube-ovn NetworkAttachmentDefi
灵雀云Kube-OVN：基于OVN的开源Kubernetes网络
kube-ovn 1.7.1 ping丢包率简测
ovs/dpdk

主要记录下neutron ovn中的公网相关的路由

公共网络



(py3env) [root@control01 ~]# neutron net-show 9989ffbf-6c71-47c9-b4bf-a747bc74f734
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| availability_zone_hints   |                                      |
| availability_zones        |                                      |
| created_at                | 2020-12-01T07:16:36Z                 |
| description               |                                      |
| id                        | 9989ffbf-6c71-47c9-b4bf-a747bc74f734 |
| ipv4_address_scope        |                                      |
| ipv6_address_scope        |                                      |
| is_default                | False                                |
| mtu                       | 1442                                 |
| name                      | pubnet                               |
| port_security_enabled     | True                                 |
| project_id                | eaa5b42f109643e7b5af0f60316aeb74     |
| provider:network_type     | flat                                 |
| provider:physical_network | physnet1                             |
| provider:segmentation_id  |                                      |
| revision_number           | 11                                   |
| router:external           | True                                 |
| shared                    | True                                 |
| status                    | ACTIVE                               |
| subnets                   | e10a8459-5668-40f4-9ee7-8fb228201eb8 |
| tags                      |                                      |
| tenant_id                 | eaa5b42f109643e7b5af0f60316aeb74     |
| updated_at                | 2021-03-23T00:45:12Z                 |
+---------------------------+--------------------------------------+

(py3env) [root@control01 ~]# neutron subnet-show e10a8459-5668-40f4-9ee7-8fb228201eb8
neutron CLI is deprecated and will be removed in the future. Use openstack CLI instead.
+-------------------+----------------------------------------------------+
| Field             | Value                                              |
+-------------------+----------------------------------------------------+
| allocation_pools  | {"start": "10.120.24.100", "end": "10.120.28.255"} |
| cidr              | 10.120.24.0/21                                     |
| created_at        | 2020-12-01T07:16:38Z                               |
| description       |                                                    |
| dns_nameservers   | 10.100.1.10                                        |
|                   | 114.114.114.114                                    |
| enable_dhcp       | True                                               |
| gateway_ip        | 10.120.31.254                                      | # 公网网关
| host_routes       |                                                    |
| id                | e10a8459-5668-40f4-9ee7-8fb228201eb8               |
| ip_version        | 4                                                  |
| ipv6_address_mode |                                                    |
| ipv6_ra_mode      |                                                    |
| name              | pubsub24                                           |
| network_id        | 9989ffbf-6c71-47c9-b4bf-a747bc74f734               |
| project_id        | eaa5b42f109643e7b5af0f60316aeb74                   |
| revision_number   | 5                                                  |
| service_types     |                                                    |
| subnetpool_id     |                                                    |
| tags              |                                                    |
| tenant_id         | eaa5b42f109643e7b5af0f60316aeb74                   |
| updated_at        | 2021-03-23T00:45:12Z                               |
+-------------------+----------------------------------------------------+

静态路由


(ovn-nb-db)[root@control02 /]# ovn-nbctl lr-route-list 4fa2af30-797b-49b6-a517-1c54fe9738e7
IPv4 Routes 
# shanghai
                0.0.0.0/0             10.120.31.254 dst-ip
(ovn-nb-db)[root@control02 /]# ovn-nbctl lr-route-list 5f9c3f5e-f8b4-410f-bf5d-ca910c2b644f
IPv4 Routes
# eu
                0.0.0.0/0             10.120.31.254 dst-ip
(ovn-nb-db)[root@control02 /]# ovn-nbctl lr-policy-list 4fa2af30-797b-49b6-a517-1c54fe9738e7

(ovn-nb-db)[root@control02 /]# ovn-nbctl lr-policy-list 5f9c3f5e-f8b4-410f-bf5d-ca910c2b644f

# 可以看到各个模拟区只有一条静态路由指向运营商网络公网网关

                0.0.0.0/0             10.120.31.254 dst-ip

无论是去往任何网段，snat之后，将流量转给物理网关

扩展：

对照kube-ovn 默认vpc的路由

[root@pc-node-1 ~]# kubectl ko nbctl lr-route-list ovn-cluster
IPv4 Routes
Route Table <main>:
                0.0.0.0/0                100.64.0.1 dst-ip
[root@pc-node-1 ~]#
[root@pc-node-1 ~]# kubectl ko nbctl lr-policy-list ovn-cluster
Routing Policies
     31000                            ip4.dst == 10.16.0.0/16           allow
     31000                           ip4.dst == 100.64.0.0/16           allow
     30000                              ip4.dst == 10.5.32.51         reroute                100.64.0.2
     30000                              ip4.dst == 10.5.32.52         reroute                100.64.0.4
     30000                              ip4.dst == 10.5.32.53         reroute                100.64.0.3
     29000              ip4.src == $ovn.default.pc.node.1_ip4         reroute                100.64.0.2
     29000              ip4.src == $ovn.default.pc.node.2_ip4         reroute                100.64.0.4
     29000              ip4.src == $ovn.default.pc.node.3_ip4         reroute                100.64.0.3
[root@pc-node-1 ~]#
查看fip和 snat
#  kubectl ko nbctl show
router f76a9e70-4ae2-4614-a7ee-4f60be4a2d0d (ovn-cluster)
    port ovn-cluster-external204
        mac: "00:00:00:C3:3A:01"
        networks: ["10.5.204.101/24"]
        gateway chassis: [52c6ada2-3ca1-45ad-b682-c0164dfba354]
    port ovn-cluster-join
        mac: "00:00:00:17:EA:98"
        networks: ["100.64.0.1/16"]
    port ovn-cluster-ovn-default
        mac: "00:00:00:0E:B1:2A"
        networks: ["10.16.0.1/16"]
    nat 0cce85bd-16f0-4f52-8bbd-49b743e095a8
        external ip: "10.5.204.202"
        logical ip: "10.16.0.13"
        type: "dnat_and_snat"
    nat 855fe375-33a9-4366-be9c-d17f321e6725
        external ip: "10.5.204.200"
        logical ip: "10.16.0.4"
        type: "snat"
    nat e95235e9-c739-4de0-b310-59530e8eed92
        external ip: "10.5.204.201"
        logical ip: "10.16.0.3"
        type: "dnat_and_snat"
# 如上 当不存在面向带有eip的pod的静态路由的时候，fip是不通的

如下有两个pod 配置了eip，即fip dnat_and_snat ,同时路由表中存在两条静态路由

router f76a9e70-4ae2-4614-a7ee-4f60be4a2d0d (ovn-cluster)
    port ovn-cluster-join
        mac: "00:00:00:17:EA:98"
        networks: ["100.64.0.1/16"]
    port ovn-cluster-ovn-default
        mac: "00:00:00:0E:B1:2A"
        networks: ["10.16.0.1/16"]
    port ovn-cluster-external204
        mac: "00:00:00:C3:3A:01"
        networks: ["10.5.204.101/24"]
        gateway chassis: [52c6ada2-3ca1-45ad-b682-c0164dfba354]
    nat 512156b5-c44a-40a1-8f7f-f57d3d3db2dc
        external ip: "10.5.204.201"
        logical ip: "10.16.0.5"  # 面向pod 10.16.0.5
        type: "dnat_and_snat" # 面向pod 10.16.0.14
    nat e852f3ae-f622-4f21-b327-c309a5edb0d4
        external ip: "10.5.204.202"
        logical ip: "10.16.0.14"
        type: "dnat_and_snat"
[root@pc-node-1 01-test-old-enable-eip-snat]#
[root@pc-node-1 01-test-old-enable-eip-snat]#
[root@pc-node-1 01-test-old-enable-eip-snat]# kubectl ko nbctl lr-route-list ovn-cluster
IPv4 Routes
Route Table <main>:
                10.16.0.5              10.5.204.254 src-ip # 面向pod 10.16.0.5
               10.16.0.14              10.5.204.254 src-ip # 面向pod 10.16.0.14
                0.0.0.0/0                100.64.0.1 dst-ip
[root@pc-node-1 01-test-old-enable-eip-snat]# kubectl ko nbctl lr-policy-list ovn-cluster
Routing Policies
     31000                            ip4.dst == 10.16.0.0/16           allow
     31000                           ip4.dst == 100.64.0.0/16           allow
     30000                              ip4.dst == 10.5.32.51         reroute                100.64.0.2
     30000                              ip4.dst == 10.5.32.52         reroute                100.64.0.4
     30000                              ip4.dst == 10.5.32.53         reroute                100.64.0.3
     29000              ip4.src == $ovn.default.pc.node.1_ip4         reroute                100.64.0.2
     29000              ip4.src == $ovn.default.pc.node.2_ip4         reroute                100.64.0.4
     29000              ip4.src == $ovn.default.pc.node.3_ip4         reroute                100.64.0.3
[root@pc-node-1 01-test-old-enable-eip-snat]#


[root@pc-node-1 01-test-old-enable-eip-snat]# kubectl ko nbctl show
#...

router f76a9e70-4ae2-4614-a7ee-4f60be4a2d0d (ovn-cluster)
    port ovn-cluster-join
        mac: "00:00:00:17:EA:98"
        networks: ["100.64.0.1/16"]
    port ovn-cluster-ovn-default
        mac: "00:00:00:0E:B1:2A"
        networks: ["10.16.0.1/16"]
    port ovn-cluster-external204
        mac: "00:00:00:C3:3A:01"
        networks: ["10.5.204.101/24"]
        gateway chassis: [52c6ada2-3ca1-45ad-b682-c0164dfba354]
    nat 15cfde4d-3aca-4b58-bd62-e3787de617c2
        external ip: "10.5.204.200"
        logical ip: "10.16.0.72"
        type: "snat"


[root@pc-node-1 01-test-old-enable-eip-snat]# kubectl ko nbctl lr-route-list ovn-cluster
IPv4 Routes
Route Table <main>:
# ...
              10.16.0.72              10.5.204.254 src-ip


[root@pc-node-1 01-test-old-enable-eip-snat]# kubectl ko nbctl lr-policy-list ovn-cluster
Routing Policies
     31000                            ip4.dst == 10.16.0.0/16           allow
     31000                           ip4.dst == 100.64.0.0/16           allow
     30000                              ip4.dst == 10.5.32.51         reroute                100.64.0.2
     30000                              ip4.dst == 10.5.32.52         reroute                100.64.0.4
     30000                              ip4.dst == 10.5.32.53         reroute                100.64.0.3
     29000              ip4.src == $ovn.default.pc.node.1_ip4         reroute                100.64.0.2
     29000              ip4.src == $ovn.default.pc.node.2_ip4         reroute                100.64.0.4
     29000              ip4.src == $ovn.default.pc.node.3_ip4         reroute                100.64.0.3

分析下kube-ovn的默认vpc的路由

IPv4 Routes
Route Table <main>:
                10.16.0.5              10.5.204.254 src-ip # 面向pod 10.16.0.5
               10.16.0.14              10.5.204.254 src-ip # 面向pod 10.16.0.14
                0.0.0.0/0                100.64.0.1 dst-ip # 这条路由和neutron的路由作用一致，目主要的是为了上公网

如果已经有了fip，那么这条路由应该替换为到公网的路由，和neutron保持一致。从专用公网网络到公网。
变更（删除原来的）路由，并不会影响 node 和（默认vpc下）pod的互相访问。

可以看到fip 是需要静态路由的，只不过neutron使用的是dst-ip规则，kube-ovn 使用的是src-ip规则。
两种方式都测试下，

默认由于已经有一条面向任意地址的det-ip的静态路由了，所以只能添加一条面向整个内网的网段src路由。

目前认为面向fip的静态路由，和面向node的静态路由是互斥的，会导致配了eip的pod 无法和node直通。

自定义vpc使用dst-ip规则，和neutron一样

kubectl ko nbctl lr-route-add ovn-cluster  192.168.2.0/24 100.64.0.1

neutron ovn 中的路由对照 kube-ovn
主要记录下neutron ovn中的公网相关的路由公共网络静态路由扩展：对照kube-ovn 默认vpc的...
kube-ovn 对照 neutron ovn
关于kube-ovn 路由的一些学习和理解
最近在学习和使用ovn 默认vpc 静态路由和策略路由相关的配置：关于kube-ovn 路由的演进：一开始只有静态...
关于kube-ovn ipam性能进一步优化
kube-ovn ipam目前在内存中实现，对照另一个项目来看，内存中ipam确实是性能最好的一种实现方式。参考:...
kube-ovn简记
服务启动顺序ovn-central 要先确定运行，ovs-ovn再确定正常，然后是其他的kube-ovn的容器重...
patch does not apply 的问题
[root@control02 kube-ovn]# git apply --check /tmp/ippool...
关于kube-ovn NetworkAttachmentDefi
https://github.com/kubeovn/kube-ovn/blob/master/docs/mult...
灵雀云Kube-OVN：基于OVN的开源Kubernetes网络
近日，灵雀云发布了基于OVN的Kubernetes网络组件Kube-OVN，并正式将其在Github上开源。Kub...
kube-ovn 1.7.1 ping丢包率简测
一、 openstack linux bridge + kube-ovn underlay pod 环境 1. n...
ovs/dpdk
vm 使用 ovs/dpdk vhost-user ports Follow Kube-OVN with OVS-...

neutron ovn 中的路由对照 kube-ovn

公共网络

静态路由

分析下kube-ovn的默认vpc的路由

相关文章

neutron ovn 中的路由对照 kube-ovn

kube-ovn 对照 neutron ovn

关于kube-ovn 路由的一些学习和理解

关于kube-ovn ipam性能进一步优化

kube-ovn简记

patch does not apply 的问题

关于kube-ovn NetworkAttachmentDefi

灵雀云Kube-OVN：基于OVN的开源Kubernetes网络

kube-ovn 1.7.1 ping丢包率简测

ovs/dpdk

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读

neutron ovn 中的路由 对照 kube-ovn

公共网络

静态路由

分析下kube-ovn的默认vpc的路由

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读

neutron ovn 中的路由对照 kube-ovn