目标

• OpenshiftSDN， networkpolicy 使用说明

说明： 以下示例基于OCP4.6 进行验证

networkpolicy 说明

• networkpolicy 是一种以应用为中心的结构，允许你设置POD如何与各种网络实体进行通信的规则
• 默认情况下，POD是非隔离的，他接受任何来源的流量
• POD在被某个Networkpolicy选中时进入隔离状态，一旦有namespace下

参数说明

下面是一个NetworkPolicy的示例

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: test-network-policy
  namespace: default
spec:
  podSelector:
    matchLabels:
      role: db
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - ipBlock:
        cidr: 172.17.0.0/16
        except:
        - 172.17.1.0/24
    - namespaceSelector:
        matchLabels:
          project: myproject
    - podSelector:
        matchLabels:
          role: frontend
    ports:
    - protocol: TCP
      port: 6379
  egress:
  - to:
    - ipBlock:
        cidr: 10.0.0.0/24
    ports:
    - protocol: TCP
      port: 5978

podSelector: 选择适用于该规则的pod

podSelector: 每个Networkpolicy 都包含一个policyTypes列表，其中包含 Ingress ， Egress或者两者兼具。如果policyTypes没有配置，则默认使用Ingress，

ingress: 每个 NetworkPolicy 可包含一个 ingress 规则的白名单列表。 每个规则都允许同时匹配 from 和 ports 部分的流量

egress:: 每个 NetworkPolicy 可包含一个 egress 规则的白名单列表。 每个规则都允许匹配 to 和 port 部分的流量

选择器 to 和 from 的行为

可以在 ingress 的 from 部分或 egress 的 to 部分中指定四种选择器

• podSelector: 此选择器将在与 NetworkPolicy 相同的名字空间中选择特定的 Pod，应将其允许作为入站流量来源或出站流量目的地。

• namespaceSelector：此选择器将选择特定的名字空间，应将所有 Pod 用作其 入站流量来源或出站流量目的地。

• namespaceSelector 和 podSelector

• ipBlock: 此选择器将选择特定的 IP CIDR 范围以用作入站流量来源或出站流量目的地。 这些应该是集群外部 IP，因为 Pod IP 存在时间短暂的且随机产生

Demo

准备工作，部署测试应用

oc new-project project1
oc label namespace project1 name=project1
oc new-project project2
oc label namespace project2 name=project2
oc new-project project3
oc label namespace project3 name=project3
oc new-app -n project1 openshiftroadshow/parksmap --name=web-db

oc new-app -n project1 openshiftroadshow/parksmap --name=client
oc new-app -n project1 openshiftroadshow/parksmap --name=booksotre -l app=bookstore
oc new-app -n project2 openshiftroadshow/parksmap --name=client2
oc new-app -n project2 openshiftroadshow/parksmap --name=booksotre-ns2 -l app=bookstore

oc new-app -n project1 openshift/hello-openshift --name=web -l app=web
oc new-app -n project1 openshift/hello-openshift --name=web-db -l app=web-db
oc new-app -n project1 openshift/hello-openshift --name=redis -l app=redis

oc new-app -n project1 openshiftroadshow/parksmap --name=inventory -l app=inventory,role=web

oc new-app -n project3 openshiftroadshow/parksmap --name=client-ns3

oc new-app -n project3 openshiftroadshow/parksmap --name=inventory -l app=inventory role=web

测试方案

参考以下方式进行测试

[root@clientvm 0 ~/work/network/policy]# oc get pod
NAME                        READY   STATUS    RESTARTS   AGE
bookinfo-64f7654cf5-7ttrw   1/1     Running   0          16m
client-67b7dbd85b-5k85j     1/1     Running   0          31m
hello-c9b7d7b89-hpq4f       1/1     Running   0          31m
web-56b44b5b4d-sz2x6        1/1     Running   0          31m
web-db-6cd6bff489-m9kwk     1/1     Running   0          31m
[root@clientvm 130 ~/work/network/policy]# oc get svc
NAME       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
bookinfo   ClusterIP   172.30.86.141    <none>        8080/TCP,8888/TCP   16m
client     ClusterIP   172.30.223.91    <none>        8080/TCP            32m
hello      ClusterIP   172.30.206.159   <none>        8080/TCP,8888/TCP   32m
web        ClusterIP   172.30.152.210   <none>        8080/TCP,8888/TCP   32m
web-db     ClusterIP   172.30.217.121   <none>        8080/TCP,8888/TCP   32m
[root@clientvm 0 ~/work/network/policy]# oc rsh client-67b7dbd85b-5k85j
sh-4.2$ curl 172.30.86.141:8080
Hello OpenShift!
sh-4.2$

1. 拒绝所有请求

拒绝所有请求

# cat deny-all.yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: deny-by-default
spec:
  podSelector:
  ingress: []

2. 允许某个应用可以被访问

允许 app=bookinfo 的应用可以被访问

# cat bookinfo-allow-all.yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: bookinfo-allow-all
spec:
  podSelector:
    matchLabels:
      app: bookinfo
  ingress:
  - {}

3. 禁止某个应用被访问

禁止 app=web 应用被访问

# cat web-deny-all.yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: web-deny-all
spec:
  podSelector:
    matchLabels:
      app: web
  ingress: []

4. 限制某个应用被访问

限制 web-db 只能被label为bookstore的应用访问

# cat web-db-allow.yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: web-db-allow
spec:
  podSelector:
    matchLabels:
      app: web-db
  ingress:
  - from:
      - podSelector:
          matchLabels:
            app: bookstore

5. 拒绝其他namespace的所有请求

允许同一个namespace内的pod互相通信，拒绝其他namespace的pod请求

# cat allow-same-policy.yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  namespace: project1   
  name: allow-same-namespace
spec:
  podSelector:
  ingress:
  - from:
    - podSelector: {}

这里有两道思考题
Q1: 如果project1 同时配置了拒接所有请求和允许namespace内所有pod互相通信，那么同一个集群内的pod之间能否互相通信
Q2：如果project1 配置了拒绝所有外部namespace请求，然后配置了一个允许指定label 的pod 访问请求，此时project2 中的含有指定label的pod 能否访问该目标pod

6. 允许 指定 namespace 的所有流量

允许project3 到project1 的所有流量

# cat web-allow-project3.yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: web-allow-project3
  namespace: project1
spec:
  podSelector: {}
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          name: project3

7. 允许其他project 的指定pod访问指定的pod

其实以下内容就是上边Q2 的答案，可以在networkpolicy 中添加一个namespaceSelector 来指定允许哪些namespace访问pod

# cat web-db-allow-other-ns.yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: web-db-allow-other-ns
  namespace: project1
spec:
  podSelector:
    matchLabels:
      app: web-db
  ingress:
    - from:
      - namespaceSelector:
          matchLabels:
            name: project2
        podSelector:
          matchLabels:
            app: bookstore

8. 允许ingress controller

允许通过router 进行访问

# cat allow-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-from-openshift-ingress
spec:
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          network.openshift.io/policy-group: ingress
  podSelector: {}
  policyTypes:
  - Ingress

openshift4 里边，router 是部署在namespace openshift-ingress 中，从下面的信息可以发现，openshift-ingress 中包含label， network.openshift.io/policy-group=ingress，因此可以通过此label设置流量可以通过router进去

# oc describe namespace openshift-ingress
Name:         openshift-ingress
Labels:       name=openshift-ingress
              network.openshift.io/policy-group=ingress
              olm.operatorgroup.uid/71df423a-e1e6-4cec-9011-8f91a3e90d0e=
              openshift.io/cluster-monitoring=true
Annotations:  openshift.io/node-selector:
              openshift.io/sa.scc.mcs: s0:c23,c12
              openshift.io/sa.scc.supplemental-groups: 1000530000/10000
              openshift.io/sa.scc.uid-range: 1000530000/10000
Status:       Active

9. 限定只能访问指定端口

限定只能访问8888 端口，此时无法访问8080端口

说明：本文使用的openshift/hello-openshift， 镜像本身对外暴露了8080和8888端口，两个端口都能访问，且输出一致，可以使用该镜像验证此特性

# cat allow-port-only.yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: allow-port-only
  namespace: project1
spec:
  podSelector:
    matchLabels:
      app: web-db
  ingress:
  - ports:
    - port: 8888
    from: []

10. 多个podselector

数据库或者redis这样的公共资源可能需要为其创建一个白名单，只有白名单上的应用可以对其进行访问，基于前文中的例子，通过测试可以发现，只有bookstore 和 inventory 可以访问redis

# cat redis-allow-services.yaml
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
  name: redis-allow-services
  namespace: project1
spec:
  podSelector:
    matchLabels:
      app: redis
  ingress:
  - from:
    - podSelector:
        matchLabels:
          app: bookstore
    - podSelector:
        matchLabels:
          app: inventory
          role: web

注意：此时需要清理networkpolicy 规则，在这个例子中，需要使用以下规则，其他规则可以需要清理下

# oc get networkpolicy
NAME                   POD-SELECTOR   AGE
deny-by-default        <none>         114m
redis-allow-services   app=redis      19m

11. openshift 不支持 包含 except的 IPBlock

openshiftSDN 不支持包含except的IPBlock，因此如果配置类似以下的规则，会在SDNpod中出现warning日志，同时整条规则也会被忽略掉

# cat ipblock-except.yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: ipblock-access
  namespace: project1
spec:
  podSelector:
    matchLabels:
      role: redis
  policyTypes:
  - Ingress
  ingress:
  - from:
    - ipBlock:
        cidr: 10.131.0.0/23
        except:
        - 10.131.1.0/24

异常日志情况

# oc -n openshift-sdn logs -f sdn-d9hxt  -c sdn
...

W0215 09:43:32.291167    1844 networkpolicy.go:542] IPBlocks with except rules are not supported (NetworkPolicy [ipblock-access], Namespace [project1])
W0215 09:43:45.231881    1844 networkpolicy.go:542] IPBlocks with except rules are not supported (NetworkPolicy [ipblock-access], Namespace [project1])
W0215 09:44:15.232054    1844 networkpolicy.go:542] IPBlocks with except rules are not supported (NetworkPolicy [ipblock-access], Namespace [project1])
W0215 09:44:45.232161    1844 networkpolicy.go:542] IPBlocks with except rules are not supported (NetworkPolicy [ipblock-access], Namespace [project1])
W0215 09:45:15.232221    1844 networkpolicy.go:542] IPBlocks with except rules are not supported (NetworkPolicy [ipblock-access], Namespace [project1])

Egress

refrence

https://access.redhat.com/documentation/zh-cn/openshift_container_platform/4.6/html-single/networking/index#nw-http2-haproxy_configuring-ingress
https://kubernetes.io/zh/docs/concepts/services-networking/network-policies/
https://github.com/ahmetb/kubernetes-network-policy-recipes
https://blog.csdn.net/weixin_43902588/article/details/103536739