从iptables理解k8s svc

准备

创建demo负载

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 3
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  selector:
    app: nginx
  ports:
  - name: http
    port: 80
    targetPort: 80
  type: NodePort

PREROUTING

查看PREROUTING chain,执行如下命令

iptables -t nat -S PREROUTING

得到如下输出

-A PREROUTING -m comment --comment "kubernetes service portals" -j KUBE-SERVICES

查看service chain,执行如下命令

iptables -t nat -S KUBE-SERVICES

得到如下输出

cluster ip
-A KUBE-SERVICES -d 10.96.240.247/32 -p tcp -m comment --comment "default/nginx:http cluster IP" -m tcp --dport 80 -j KUBE-SVC-P4Q3KNUAWJVP4ILH
nodeport
-A KUBE-SERVICES -m comment --comment "kubernetes service nodeports; NOTE: this must be the last rule in this chain" -m addrtype --dst-type LOCAL -j KUBE-NODEPORTS

针对nodeport

nodeport chain,执行如下命令

iptables -t nat -S KUBE-NODEPORTS

得到如下输出

-A KUBE-NODEPORTS -p tcp -m comment --comment "default/nginx:http" -m tcp --dport 30471 -j KUBE-EXT-P4Q3KNUAWJVP4ILH

查看nginx nodeport chain,执行如下命令

iptables -t nat -S KUBE-EXT-P4Q3KNUAWJVP4ILH

得到如下输出

-A KUBE-EXT-P4Q3KNUAWJVP4ILH -j KUBE-SVC-P4Q3KNUAWJVP4ILH

跳转到KUBE-SVC-P4Q3KNUAWJVP4ILH后同clusterip流程

针对cluster ip

查看nginx svc chain,执行如下命令

iptables -t nat -S KUBE-SVC-P4Q3KNUAWJVP4ILH

得到如下输出

node到pod命中这条
-A KUBE-SVC-P4Q3KNUAWJVP4ILH ! -s 10.244.0.0/16 -d 10.96.240.247/32 -p tcp -m comment --comment "default/nginx:http cluster IP" -m tcp --dport 80 -j KUBE-MARK-MASQ

pod到pod命中以下三条
-A KUBE-SVC-P4Q3KNUAWJVP4ILH -m comment --comment "default/nginx:http -> 10.244.1.2:80" -m statistic --mode random --probability 0.33333333349 -j KUBE-SEP-HNEN5KDVUKX4WY7U
-A KUBE-SVC-P4Q3KNUAWJVP4ILH -m comment --comment "default/nginx:http -> 10.244.2.2:80" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-RFZMCNGOXQ6LCRZG
-A KUBE-SVC-P4Q3KNUAWJVP4ILH -m comment --comment "default/nginx:http -> 10.244.2.3:80" -j KUBE-SEP-RKPJ4O2WWF2V7ERD

查看mask chain,执行如下命令

iptables -t nat -S KUBE-MARK-MASQ

得到如下输出

设置mask 0x4000/0x4000
-A KUBE-MARK-MASQ -j MARK --set-xmark 0x4000/0x4000

查看endpoint chain,执行如下命令

iptables -t nat -S KUBE-SEP-HNEN5KDVUKX4WY7U

得到如下输出

访问自身命中,跳转到mask chain
-A KUBE-SEP-HNEN5KDVUKX4WY7U -s 10.244.1.2/32 -m comment --comment "default/nginx:http" -j KUBE-MARK-MASQ
访问非自身,进行dnat
-A KUBE-SEP-HNEN5KDVUKX4WY7U -p tcp -m comment --comment "default/nginx:http" -m tcp -j DNAT --to-destination 10.244.1.2:80

POSTROUTING

查看POSTROUTING chain,执行如下命令

iptables -t nat -S POSTROUTING

得到如下输出

-A POSTROUTING -m comment --comment "kubernetes postrouting rules" -j KUBE-POSTROUTING

查看KUBE-POSTROUTING chain,执行如下命令

iptables -t nat -S KUBE-POSTROUTING

得到如下输出

pod到pod且不是访问自身命中这条
-A KUBE-POSTROUTING -m mark ! --mark 0x4000/0x4000 -j RETURN
设置mask
-A KUBE-POSTROUTING -j MARK --set-xmark 0x4000/0x0
进行snat
-A KUBE-POSTROUTING -m comment --comment "kubernetes service traffic requiring SNAT" -j MASQUERADE --random-fully