云网络内VM或者主机如果使用内网VPC做公网访问一般需要在公网出口路由器上做SNAT替换SIP，这种情况下只能由云内发起访问公网，公网是无法主动访问云内服务的。
也可以为云内VM或者主机配置EIP，实际上配置了一个内外网IP地址的一对一地址映射，提供双向访问能力。
拓扑
拓扑和配置承接上文，做增量配置。
配置
1、nat
ovn支持 snat、dnat、dnat_and_snat （实现EIP）三种 nat方式。
nat可以配置到logic switch或者logic router上，这里模拟公网访问，配置在vpc router上。
我们为子网 30.1.1.0/24所在的VPC网络配置snat。
ovn-nbctl lr-nat-add vpc-router snat 192.168.77.1 30.1.1.0/24
流表分析
### 1、路由器上，目的是192.168.77.1的包，进入ct处理，如果是公网进入云网络的包，此前已经在出方向建立了est的ct表项，这里就能够做dnat，
## 将公网地址192.168.77.1转化内网地址发往vm
cookie=0x7b8e0e0e, duration=169.183s, table=11, n_packets=0, n_bytes=0, priority=100,ip,reg14=0x3,metadata=0x7,nw_dst=192.168.77.1 actions=ct(table=12,zone=NXM_NX_REG12[0..15],nat)
cookie=0x7580a142, duration=169.183s, table=11, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x7,nw_dst=192.168.77.1 actions=load:0x1->OXM_OF_PKT_REG4[0],resubmit(,12)
### 2、router上，出公网流量，确认从公网口出后，为VPC网络 30.1.1.0/24建立 snat conntrack表项
cookie=0x1939fcab, duration=169.183s, table=41, n_packets=0, n_bytes=0, priority=25,ip,reg15=0x3,metadata=0x7,nw_src=30.1.1.0/24 actions=ct(commit,table=42,zone=NXM_NX_REG12[0..15],nat(src=192.168.77.1))
###
cookie=0xadd21e5a, duration=169.183s, table=42, n_packets=0, n_bytes=0, priority=100,ip,reg15=0x3,metadata=0x7,nw_dst=192.168.77.1 actions=clone(ct_clear,move:NXM_NX_REG15[]->NXM_NX_REG14[],load:0->NXM_NX_REG15[],load:0->NXM_NX_REG10[],load:0x1->NXM_NX_REG10[0],load:0->NXM_NX_XXREG0[96..127],load:0->NXM_NX_XXREG0[64..95],load:0->NXM_NX_XXREG0[32..63],load:0->NXM_NX_XXREG0[0..31],load:0->NXM_NX_XXREG1[96..127],load:0->NXM_NX_XXREG1[64..95],load:0->NXM_NX_XXREG1[32..63],load:0->NXM_NX_XXREG1[0..31],load:0->OXM_OF_PKT_REG4[32..63],load:0->OXM_OF_PKT_REG4[0..31],load:0x1->OXM_OF_PKT_REG4[1],resubmit(,8))
2、EIP
nat配置的命令行：
ovn-nbctl lr-nat-add ROUTER TYPE EXTERNAL_IP LOGICAL_IP [LOGICAL_PORT EXTERNAL_MAC]
通过配置 LOGICAL_PORT EXTERNAL_MAC 实现分布式EIP功能，相关流表会在LOGICAL_IP/LOGICAL_PORT 所在的计算节点下发，实现流量本地收发而不需要到集中式网关上。
如果不配置，则为集中式网关，到lrp-set-gateway-chassis所在节点公网出口。
下面分别为两个vm port配置了EIP，这两个vm分别位于Centrial节点和Node节点，他们将分别在各自节点实现EIP功能，本地转发。
ovn-nbctl lr-nat-add vpc-router dnat_and_snat 192.168.77.32 30.1.1.12 sw-300-port-vm2 0a:10:dd:1b:30:02
ovn-nbctl lr-nat-add vpc-router dnat_and_snat 192.168.77.42 40.1.1.12 sw-400-port-vm2 0a:10:dd:1b:40:02
注意，如果要实现分布式EIP的特性，需要在每个节点上都配置公网桥。
ovs-vsctl add-br br-ex
ovs-vsctl set Open_vSwitch . external-ids:ovn-bridge-mappings=dataNet:br-ex
流表分析
###### 本节点为Centrial，sw-300-port-vm2（0a:10:dd:1b:30:02）在Centrial节点，
## sw-400-port-vm2（0a:10:dd:1b:40:02） 在Node节点，重点关注nat和分布式EIP特性
#### 1、只收dmac 为 0a:10:dd:1b:30:02 的报文，0a:10:dd:1b:40:02 在Node节点接收
cookie=0x34dc7936, duration=959.144s, table=8, n_packets=0, n_bytes=0, priority=50,reg14=0x3,metadata=0x7,dl_dst=0a:10:dd:1b:30:02 actions=resubmit(,9)
#### 2、公网进入的arp报文代答流表，使用 EXTERNAL_IP的mac地址应答
#. 192.168.77.32（本机IP）的arp请求，应答来自VPC网络和公网网络的请求
#. 192.168.77.42（非本机）的arp请求，只应答来自VPC网络的请求，公网侧必须在 logic ip所在宿主机Node节点应答，
# 否则可能在公网 sw 上发生mac地址漂移，流量异常。
####
cookie=0xb1db9759, duration=1712.735s, table=9, n_packets=0, n_bytes=0, priority=90,arp,reg14=0x1,metadata=0x7,arp_tpa=192.168.77.32,arp_op=1 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],mod_dl_src:02:d4:1d:8c:30:01,load:0x2d41d8c3001->NXM_NX_ARP_SHA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xc0a84d20->NXM_OF_ARP_SPA[],load:0x1->NXM_NX_REG15[],load:0x1->NXM_NX_REG10[0],resubmit(,32)
cookie=0xb48b1ba3, duration=1712.735s, table=9, n_packets=0, n_bytes=0, priority=90,arp,reg14=0x2,metadata=0x7,arp_tpa=192.168.77.32,arp_op=1 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],mod_dl_src:02:d4:1d:8c:40:01,load:0x2d41d8c4001->NXM_NX_ARP_SHA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xc0a84d20->NXM_OF_ARP_SPA[],load:0x2->NXM_NX_REG15[],load:0x1->NXM_NX_REG10[0],resubmit(,32)
cookie=0x4b6c758e, duration=1712.735s, table=9, n_packets=1, n_bytes=42, priority=90,arp,reg14=0x3,metadata=0x7,arp_tpa=192.168.77.32,arp_op=1 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],mod_dl_src:0a:10:dd:1b:30:02,load:0xa10dd1b3002->NXM_NX_ARP_SHA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xc0a84d20->NXM_OF_ARP_SPA[],load:0x3->NXM_NX_REG15[],load:0x1->NXM_NX_REG10[0],resubmit(,32)
cookie=0xbad18a69, duration=393.564s, table=9, n_packets=0, n_bytes=0, priority=90,arp,reg14=0x2,metadata=0x7,arp_tpa=192.168.77.42,arp_op=1 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],mod_dl_src:02:d4:1d:8c:40:01,load:0x2d41d8c4001->NXM_NX_ARP_SHA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xc0a84d2a->NXM_OF_ARP_SPA[],load:0x2->NXM_NX_REG15[],load:0x1->NXM_NX_REG10[0],resubmit(,32)
cookie=0xe86209f6, duration=393.564s, table=9, n_packets=0, n_bytes=0, priority=90,arp,reg14=0x1,metadata=0x7,arp_tpa=192.168.77.42,arp_op=1 actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],load:0x2->NXM_OF_ARP_OP[],move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],mod_dl_src:02:d4:1d:8c:30:01,load:0x2d41d8c3001->NXM_NX_ARP_SHA[],move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],load:0xc0a84d2a->NXM_OF_ARP_SPA[],load:0x1->NXM_NX_REG15[],load:0x1->NXM_NX_REG10[0],resubmit(,32)
#### 3、路由器上到 EIP 的流量，进入CT模块做nat处理，如果已经存在ct表项，则做完dnat，进入table12直接next table13。否则在table12中做dnat
cookie=0xfc13ddd1, duration=959.144s, table=11, n_packets=0, n_bytes=0, priority=100,ip,reg14=0x3,metadata=0x7,nw_dst=192.168.77.32 actions=ct(table=12,zone=NXM_NX_REG12[0..15],nat)
cookie=0x341855db, duration=393.564s, table=11, n_packets=0, n_bytes=0, priority=100,ip,reg14=0x3,metadata=0x7,nw_dst=192.168.77.42 actions=ct(table=12,zone=NXM_NX_REG12[0..15],nat)
cookie=0x35164f7c, duration=959.144s, table=11, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x7,nw_dst=192.168.77.32 actions=load:0x1->OXM_OF_PKT_REG4[0],resubmit(,12)
cookie=0x1b958ca2, duration=393.564s, table=11, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x7,nw_dst=192.168.77.42 actions=load:0x1->OXM_OF_PKT_REG4[0],resubmit(,12)
#### 4、路由器上到 EIP 的流量，如果从公网来，DNAT 为VM IP，并创建CT表项。如果从内网来，为普通到EIP的报文。
cookie=0x15203869, duration=959.143s, table=12, n_packets=0, n_bytes=0, priority=100,ip,reg14=0x3,metadata=0x7,nw_dst=192.168.77.32 actions=ct(commit,table=13,zone=NXM_NX_REG11[0..15],nat(dst=30.1.1.12))
cookie=0x2c74407a, duration=393.564s, table=12, n_packets=0, n_bytes=0, priority=100,ip,reg14=0x3,metadata=0x7,nw_dst=192.168.77.42 actions=ct(commit,table=13,zone=NXM_NX_REG11[0..15],nat(dst=40.1.1.12))
cookie=0x4ee4c35c, duration=959.143s, table=12, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x7,nw_dst=192.168.77.32 actions=load:0x1->OXM_OF_PKT_REG4[0],resubmit(,13)
cookie=0xbfda9be3, duration=393.564s, table=12, n_packets=0, n_bytes=0, priority=50,ip,metadata=0x7,nw_dst=192.168.77.42 actions=load:0x1->OXM_OF_PKT_REG4[0],resubmit(,13)
cookie=0x2f100e98, duration=99026.437s, table=12, n_packets=1436, n_bytes=140236, priority=0,metadata=0x7 actions=resubmit(,13)
cookie=0xa616964f, duration=959.144s, table=17, n_packets=2, n_bytes=196, priority=100,ip,reg15=0x3,metadata=0x7,nw_src=30.1.1.12 actions=resubmit(,18)
cookie=0xcd109cd8, duration=393.564s, table=17, n_packets=0, n_bytes=0, priority=100,ip,reg15=0x3,metadata=0x7,nw_src=40.1.1.12 actions=resubmit(,18)
#### 5、VM->pub的流量，snat处理，走ct表项做nat。第二条实际上是走不到的，n_packets=0,实际流量在Node节点上
cookie=0x2345dfd5, duration=959.144s, table=40, n_packets=2, n_bytes=196, priority=100,ip,reg15=0x3,metadata=0x7,nw_src=30.1.1.12 actions=mod_dl_src:0a:10:dd:1b:30:02,ct(table=41,zone=NXM_NX_REG11[0..15],nat)
cookie=0x9fb1af16, duration=393.564s, table=40, n_packets=0, n_bytes=0, priority=100,ip,reg15=0x3,metadata=0x7,nw_src=40.1.1.12 actions=mod_dl_src:0a:10:dd:1b:40:02,ct(table=41,zone=NXM_NX_REG11[0..15],nat)
#### 6、VM->pub的流量，snat处理，nat并建立ct表项
cookie=0x25fcdcde, duration=959.143s, table=41, n_packets=2, n_bytes=196, priority=33,ip,reg15=0x3,metadata=0x7,nw_src=30.1.1.12 actions=mod_dl_src:0a:10:dd:1b:30:02,ct(commit,table=42,zone=NXM_NX_REG12[0..15],nat(src=192.168.77.32))
cookie=0x700c42e4, duration=393.564s, table=41, n_packets=0, n_bytes=0, priority=33,ip,reg15=0x3,metadata=0x7,nw_src=40.1.1.12 actions=mod_dl_src:0a:10:dd:1b:40:02,ct(commit,table=42,zone=NXM_NX_REG12[0..15],nat(src=192.168.77.42))
#### 7、举例说明，vm2 ping EIP, "ping 192.168.77.32 -I 30.1.1.12"，到这里变成：
# 0a:10:dd:1b:30:02（上面snat加工） --> 02:d4:1d:8c:30:1 , 192.168.77.32(上面snat加工） --> 192.168.77.32
# 这里先清除在上面建立的 ct表项（这种情况不需要ct），模拟从公网收包：将入接口设置为公网口（当前的出口），然后送到 table8 再走一遍
# 其结果是，经过 dnat等操作，将报文改成 0a:10:dd:1b:30:02 --> fa:10:dd:1b:30:02 , 192.168.77.32 --> 30.1.1.12，送入vm接口，数据包完全正确。
####
cookie=0x23ac3e3b, duration=959.144s, table=42, n_packets=0, n_bytes=0, priority=100,ip,reg15=0x3,metadata=0x7,nw_dst=192.168.77.32 actions=clone(ct_clear,move:NXM_NX_REG15[]->NXM_NX_REG14[],load:0->NXM_NX_REG15[],load:0->NXM_NX_REG10[],load:0x1->NXM_NX_REG10[0],load:0->NXM_NX_XXREG0[96..127],load:0->NXM_NX_XXREG0[64..95],load:0->NXM_NX_XXREG0[32..63],load:0->NXM_NX_XXREG0[0..31],load:0->NXM_NX_XXREG1[96..127],load:0->NXM_NX_XXREG1[64..95],load:0->NXM_NX_XXREG1[32..63],load:0->NXM_NX_XXREG1[0..31],load:0->OXM_OF_PKT_REG4[32..63],load:0->OXM_OF_PKT_REG4[0..31],load:0x1->OXM_OF_PKT_REG4[1],resubmit(,8))
cookie=0xe5410abf, duration=393.565s, table=42, n_packets=0, n_bytes=0, priority=100,ip,reg15=0x3,metadata=0x7,nw_dst=192.168.77.42 actions=clone(ct_clear,move:NXM_NX_REG15[]->NXM_NX_REG14[],load:0->NXM_NX_REG15[],load:0->NXM_NX_REG10[],load:0x1->NXM_NX_REG10[0],load:0->NXM_NX_XXREG0[96..127],load:0->NXM_NX_XXREG0[64..95],load:0->NXM_NX_XXREG0[32..63],load:0->NXM_NX_XXREG0[0..31],load:0->NXM_NX_XXREG1[96..127],load:0->NXM_NX_XXREG1[64..95],load:0->NXM_NX_XXREG1[32..63],load:0->NXM_NX_XXREG1[0..31],load:0->OXM_OF_PKT_REG4[32..63],load:0->OXM_OF_PKT_REG4[0..31],load:0x1->OXM_OF_PKT_REG4[1],resubmit(,8))
网友评论