一般来说，部署无状态应用很简单，但是一旦牵涉到有状态的应用，尤其是非云原生的软件，部署就很复杂了，还好我们有Helm包管理工具，可以大大简化部署的过程

一、Helm

现在我们来安装一个k8s包管理利器Helm，目前用的比较多的是Helm3，如果用Helm2的话，很多东西用起来还会比较麻烦。我们去git的Helm主业找一个推荐的安装版本，然后解压缩放到bin目录下面。

1. Download your desired version
2. Unpack it (tar -zxvf helm-v2.0.0-linux-amd64.tgz)
3. Find the helm binary in the unpacked directory, and move it to its desired destination (mv linux-amd64/helm /usr/local/bin/helm)

二、 NFS

NFS是比较常用也比较简单的一种磁盘共享功能，它的原理是通过rpc服务记录所有需要的端口，然后通过rpc获取到的服务端口号转发到NFS服务，以此模拟磁盘共享。

首先安装需要的软件

yum install -y  nfs-utils   
yum install -y rpcbind
systemctl start rpcbind 
systemctl enable rpcbind 
systemctl start nfs-server
systemctl enable nfs-server
# 如果有防火墙
firewall-cmd --permanent --add-service=nfs
firewall-cmd  --reload 

通过df -h看看哪里可以挂载，改好k8s权限之后写到/etc/exports里面

vim /etc/exports
/mnt/resource/k8s1 *(rw,no_root_squash,sync)

#配置生效
exportfs -r
#查看生效
exportfs

#启动rpcbind、nfs服务
systemctl restart rpcbind && systemctl enable rpcbind
systemctl restart nfs && systemctl enable nfs

#查看 RPC 服务的注册状况，其中111端口是rpc的入口，从这个端口告诉客户端服务在哪
rpcinfo -p localhost

#showmount测试
showmount -e 10.0.1.7
Export list for 10.0.1.7:
/mnt/resource/k8s1 *

其它节点也安装启动nfs，作为客户端工具

yum -y install nfs-utils
systemctl start nfs && systemctl enable nfs

三、创建pv

从这里切回k8s用户，示例yaml文件如下

apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-pv001
  labels:
    pv: nfs-pv001
spec:
  capacity:
    storage: 20Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Recycle
  storageClassName: nfs-jenkins
  nfs:
    path: /mnt/resource/k8s1/jenkins
    server: 10.0.1.7

然后创建这个pv

[hanyi@ms-03 ~]$ kubectl apply -f k8script/nfs-pv001.yaml 
persistentvolume/nfs-pv001 created
[hanyi@ms-03 ~]$ kubectl apply -f k8script/nfs-pv001.yaml 
persistentvolume/nfs-pv001 created
[hanyi@ms-03 ~]$ kubectl get pv
NAME        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS   REASON   AGE
nfs-pv001   20Gi       RWO            Recycle          Available           nfs                     18s

四、Jenkins

养成良好的习惯

kubectl create namespace jenkins

开始使用Helm

[hanyi@ms-03 ~]$ helm repo add stable https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts
"stable" has been added to your repositories
[hanyi@ms-03 ~]$ helm repo list
NAME    URL                                                   
stable  https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts

[hanyi@ms-03 ~]$ helm search repo jenkins
NAME            CHART VERSION   APP VERSION DESCRIPTION                                       
stable/jenkins  0.13.5          2.73        Open source continuous integration server. It s...

以下是安装jenkins的过程

# 因为helm里面deployment的api版本错了，所以改下api版本apps/v1
[hanyi@ms-03 jenkins]$ vim templates/jenkins-master-deployment.yaml 
# 需要把Persistence的StorageClass改成'nfs'，放到jenkins目录下面
  StorageClass: "nfs-jenkins"
[hanyi@ms-03 jenkins]$ vim values.yaml 
[hanyi@ms-03 jenkins]$ helm install --name-template jenkins -f values.yaml . --namespace jenkins
NAME: jenkins
LAST DEPLOYED: Sat May 23 04:55:06 2020
NAMESPACE: jenkins
STATUS: deployed
REVISION: 1
NOTES:
1. Get your 'admin' user password by running:
  printf $(kubectl get secret --namespace jenkins jenkins -o jsonpath="{.data.jenkins-admin-password}" | base64 --decode);echo
2. Get the Jenkins URL to visit by running these commands in the same shell:
  NOTE: It may take a few minutes for the LoadBalancer IP to be available.
        You can watch the status of by running 'kubectl get svc --namespace jenkins -w jenkins'
  export SERVICE_IP=$(kubectl get svc --namespace jenkins jenkins --template "{{ range (index .status.loadBalancer.ingress 0) }}{{ . }}{{ end }}")
  echo http://$SERVICE_IP:8080/login

3. Login with the password from step 1 and the username: admin

For more information on running Jenkins on Kubernetes, visit:
https://cloud.google.com/solutions/jenkins-on-container-engine

如果想要卸载 直接用helm uninstall jenkins，然后找到访问的路径

[hanyi@ms-03 jenkins]$ kubectl get service -n jenkins
NAME            TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
jenkins         LoadBalancer   10.102.95.170   <pending>     8080:30746/TCP   2m26s
jenkins-agent   ClusterIP      10.100.155.25   <none>        50000/TCP        2m26s

五、Consul

以同样方法创建3个consul的pv，然后通过以下命令安装最新版consul

$ helm repo add hashicorp https://helm.releases.hashicorp.com
"hashicorp" has been added to your repositories
$ helm search repo hashicorp/consul
NAME                CHART VERSION   APP VERSION DESCRIPTION
hashicorp/consul    0.20.1          1.7.2       Official HashiCorp Consul Chart
$ helm install consul hashicorp/consul --set server.storage=1Gi,server.storageClass=nfs-consul
NAME: consul

需要注意的是，consul需要在三个结点运行，如果master没有去污点，那么就会部署失败。

六、Ingress

去github找找nginx的ingress，然后去官网看看哪个安装方法最简单

[hanyi@ms-03 ~]$ helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
"ingress-nginx" has been added to your repositories
[hanyi@ms-03 ~]$ helm install ingress ingress-nginx/ingress-nginx
NAME: ingress

在任意地方生成密码

$ htpasswd -c auth consuler

然后把auth文件拷到服务器上

kubectl  create secret generic basic-auth --from-file=auth

然后创建一个ingress，里面的basic-auth是你刚刚创建的名字，serviceName和Port是ui界面的service。里面的host是必须填域名的，可以用修改hosts的方式或者直接解析过去。

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-consul-with-auth-basic
  annotations:
    # type of authentication
    nginx.ingress.kubernetes.io/auth-type: basic
    # name of the secret that contains the user/password definitions
    nginx.ingress.kubernetes.io/auth-secret: basic-auth
    # message to display with an appropriate context why the authentication is required
    nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required - foo'
spec:
  rules:
  - host: consul.love.cn
    http:
      paths:
      - path: /
        backend:
          serviceName: consul-consul-ui
          servicePort: 80

apply该yaml后，查看ingress的service，找到ingress的端口号，因为我们没有指定一个，所以这里是随机生成一个端口。

ingress-ingress-nginx-controller             LoadBalancer   10.103.130.202   <pending>     80:30483/TCP,443:30639/TCP                                                48m

然后我们输入consul.love.cn:30483后提示输入账号密码，输入后就可以进入consul的ui管理界面了