美文网首页
UK8S Ingress 配置

UK8S Ingress 配置

作者: 张朔源 | 来源:发表于2020-06-28 23:35 被阅读0次

UK8S Ingress 配置

参照 UK8S nginx ingress 配置

kubectl apply -f http://uk8s.cn-bj.ufileos.com/yaml/ingress/nginx/mandatory.yaml

在上述文章第二步的时候,参照 通过外网ULB访问Service,在 yaml 中添加关于 ULB 配置的声明,
具体例如如下:

apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  annotations:
    service.beta.kubernetes.io/ucloud-load-balancer-id: "ulb-xxx"
    service.beta.kubernetes.io/ucloud-load-balancer-vserver-protocol: "TCP"
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  type: LoadBalancer
  ports:
    - name: http
      port: 80
      targetPort: 80
      protocol: TCP
    - name: https
      port: 443
      targetPort: 443
      protocol: TCP
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

上面是TCP L4层的代理,没有搞L7,因为存在不同namespace下,需要用到不同 https 证书, 的场景,我们后续会使用 cert-manager 来解决 https 证书问题。

Nginx 配置

由于特定业务的特殊需求,需要对 Nginx 进行特殊配置,例如增大客户端允许上传数据的大小。可参看文档 Annotation-NGINX Ingress Controller

Multi Ingress

以上是外网 Ingress 配置,我们还需要个内网 ingress 配置,解决思路有两种,1种是切换另一种 ingress,还有一种是继续使用 nginx ingress。我们选择第二种方案,方便维护。

具体来讲,就是对 http://uk8s.cn-bj.ufileos.com/yaml/ingress/nginx/mandatory.yaml 进行定制。可参照 Google Multi Nginx-ingress 文档,具体步骤如下:

1. 修改 外网 Nignx Controller 配置

  1. 给 外网 Nignx Controller 添加 Label, 方便和内网区分开来,如: ingress-type: outer
  2. 给 外网 Ingress 添加 Selector, 内容为第一步多加的 Label

2. 修改 Role 配置

nginx-ingress-roleresourceNames 添加一行 ingress-controller-leader-internal-nginx
使得先前创建的 nginx-ingress-serviceaccount 可以操纵将要创建的 internal-nginx。

3. 部署 internal-nginx-controller

kind: ConfigMap
apiVersion: v1
metadata:
  name: internal-nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: internal-tcp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx

---
kind: ConfigMap
apiVersion: v1
metadata:
  name: internal-udp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: internal-nginx-ingress-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        ingress-type: inner
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      serviceAccountName: nginx-ingress-serviceaccount
      containers:
        - name: internal-nginx-ingress-controller
          image: uhub.service.ucloud.cn/library/nginx-ingress-controller:0.23.0
          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/internal-nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/internal-tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/internal-udp-services
            - --publish-service=$(POD_NAMESPACE)/internal-ingress-nginx
            - --ingress-class=internal-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 33
            runAsUser: 33
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: http
              containerPort: 80
            - name: https
              containerPort: 443
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
---
apiVersion: v1
kind: Service
metadata:
  name: internal-ingress-nginx
  namespace: ingress-nginx
  annotations:
    service.beta.kubernetes.io/ucloud-load-balancer-id: "ulb-xxx"
    service.beta.kubernetes.io/ucloud-load-balancer-vserver-protocol: "TCP"
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  type: LoadBalancer
  ports:
    - name: http
      port: 80
      targetPort: 80
      protocol: TCP
    - name: https
      port: 443
      targetPort: 443
      protocol: TCP
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    ingress-type: inner

这个 yaml 对比老的来说,修改点有:

  1. nginx-ingress-controller 的 args、label 和 名称 ,尤其是 --ingress-class=internal-nginx, Ingress 要用到,前面的 nginx-ingress-role 的修改也和它有关联。
  2. service 对应的内网ULB ID,以及 selector 里的 ingress-type: inner
  3. ConfigMap 的名称,如果你想内外网都用统一的配置,也可以不改。

Ingress 区分使用

以上配置默认会使用外网Ingress Controller,
内网 Ingress 的 Annotation 里面要添加 kubernetes.io/ingress.class: internal-nginx, 才能告知 内网Ingress Controller 如何 forward 流量。

获取客户端IP

参考 UCloud 文档: https://docs.ucloud.cn/compute/uk8s/service/getresourceip

相关文章

网友评论

      本文标题:UK8S Ingress 配置

      本文链接:https://www.haomeiwen.com/subject/gzfnfktx.html