1. Setup工作

Install az
改变云环境，Global Or CN

$ az cloud set -n AzureCloud
$ az cloud set -n AzureChinaCloud

本地安装kubectl

$ sudo az aks install-cli

获取aks凭证

$ az aks get-credentials --resource-group AKS-V2 --name AKS-V2

获取所有node

$ kubectl get nodes
NAME                       STATUS   ROLES   AGE   VERSION
aks-nodepool1-41549566-0   Ready    agent   5d    v1.10.8
aks-nodepool1-41549566-1   Ready    agent   5d    v1.10.8
aks-nodepool1-41549566-2   Ready    agent   5d    v1.10.8
aks-nodepool1-41549566-3   Ready    agent   5d    v1.10.8
aks-nodepool1-41549566-4   Ready    agent   5d    v1.10.8
aks-nodepool1-41549566-5   Ready    agent   5d    v1.10.8
aks-nodepool1-41549566-6   Ready    agent   5d    v1.10.8

获取所有namespace

$ kubectl get ns
NAME                       STATUS   AGE
default                    Active   10d
kube-public                Active   10d
kube-system                Active   10d

2. 创建rai-dashboard服务

2.1. 创建namespace，不同ns之间资源相互隔离，相当于子集群的概念

$ kubectl create namespace production-rai-dashboard
namespace/production-rai-dashboard created
$ kubectl get ns
NAME                       STATUS   AGE
default                    Active   10d
kube-public                Active   10d
kube-system                Active   10d
production-rai-dashboard   Active   16h

2.2. 创建secret，用于登录docker-registry下载image

$ kubectl create secret docker-registry regcred --docker-server=registry.aliyuncs.com --docker-username=****** --docker-password=****** --docker-email=****** -n production-rai-dashboard
$ kubectl create secret docker-registry regcred-cn-hangzhou --docker-server=registry.cn-hangzhou.aliyuncs.com --docker-username=****** --docker-password=****** --docker-email=****** -n production-rai-dashboard

2.3. 在production-rai-dashboard ns下，创建dashboard-api和dashboard-portal服务，dashboard-portal服务内部调用dashboard-api服务接口

$ kubectl apply -f dashboard-api.yml
$ kubectl apply -f dashboard-portal.yml

2.4. 本地aks dashboard，可用于查看各种资源、各种Controller

$ az aks browse --resource-group AKS-V2 --name AKS-V2
Merged "AKS-V2" as current context in /tmp/tmpqsd2csqs
Proxy running on http://127.0.0.1:8001/
Press CTRL+C to close the tunnel...
Forwarding from 127.0.0.1:8001 -> 9090
Forwarding from [::1]:8001 -> 9090
Handling connection for 8001
Opening in existing browser session.
Handling connection for 8001

image.png

3. 问题排查

登录到某个pod，可用于排查问题

$ kubectl exec -it rai-dashboard-portal-deployment-5749d9b944-fzvrq bash -n production-rai-dashboard

如何分析程序日志

Azure CN不支持Log Analytics，应该不可行

Enable and review Kubernetes master node logs in Azure Kubernetes Service (AKS)
https://docs.microsoft.com/en-us/azure/aks/view-master-logs

通过k8s dashboard查看每一个container日志，也可以通过cmd查看，可行
通过cmd，可以通过label筛选某一个service，对所有pod日志聚合显示，推荐使用

$ kubectl logs --tail=100 -l app=rai-dashboard-portal --all-containers=true -n production-rai-dashboard

4. 常用操作

升级镜像，修改env

在线编辑，直接apply

$ kubectl edit deployment/rai-dashboard-portal-deployment -o yaml --save-config -n production-rai-dashboard

导出yaml到本地，本地编辑后再apply

$ kubectl get deployment rai-dashboard-portal-deployment -n production-rai-dashboard -o yaml > 1.log
$ kubectl apply -f 1.log 
deployment.extensions/rai-dashboard-portal-deployment configured

5. 后续操作

CronJob测试
K8S的内部网络机制，深入了解

6. 参考

7. 附件

dashboard-portal.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: rai-dashboard-portal-deployment
  namespace: production-rai-dashboard
spec:
  replicas: 2
  selector:
    matchLabels:
      app: rai-dashboard-portal
      stage: production
  template:
    metadata:
      labels:
        app: rai-dashboard-portal
        stage: production
    spec:
      imagePullSecrets:
      - name: regcred-cn-hangzhou
      containers:
      - name: rai-dashboard-portal
        image: rai-portal-dashboard:10.10.77
        command: ["sh"]
        args: ["run_prod.sh"]
        ports:
        - containerPort: 9600
        env:
        - name: RUNENV
          value: "product"
        - name: REDIS_URI
          value: "redis://:@10.244.4.8:/11"
---
apiVersion: v1
kind: Service
metadata:
  name: rai-dashboard-portal-service
  namespace: production-rai-dashboard
spec:
  ports:
  - port: 9600
  selector:
    app: rai-dashboard-portal
    stage: production
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: rai-dashboard-portal-ingress
  namespace: production-rai-dashboard
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
#  tls:
#  - hosts:
#    - xxx.chinacloudapi.cn
#    secretName: aks-ingress-tls
  rules:
    - host: xxx.chinacloudapi.cn
      http:
        paths:
        - path: /
          backend:
            serviceName: rai-dashboard-portal-service
            servicePort: 9600
        - path: /api
          backend:
            serviceName: rai-dashboard-api-service
            servicePort: 9800