k8s springcloud apollo

作者: Maple_JW | 来源:发表于2021-04-27 17:00 被阅读0次

k8s springcloud apollo
apollo 配置提前加载
Apollo 简介
springcloud通过apollo动态配置logback
springcloud in k8s springcloud项目
springcloud in k8s springcloud项目
springcloud in kubernetes 目录
kubernetes[k8s]服务间通讯[调用]
Apollo-v8.0安装踩坑记录
Apollo配置中心原理分析

springcloud + apollo的微服务化部署

创建namespace

建立属于微服务体系下的namespace,cloud.yml

apiVersion: v1
kind: Namespace
metadata:
  name: springcloud

执行 kubectl apply -f cloud.yml

eureka

由于eureka是有状态的服务，因此采用statefulset部署

首先创建镜像,编写DOCKFILE

FROM java:8
copy ./www /data/www
entrypoint java ${JVM:=-Xms1024m -Xmx1024m} -Djava.security.egd=file:/dev/./urandom -Dfile.encoding=UTF8 -jar /data/www/iggreport-registry-2.1.5.RELEASE.jar ${APP_OPT}

执行docker build -t eureka:v1 .

创建eureka.yml

apiVersion: v1
kind: Secret
metadata:
  name: mysecret
  namespace: springcloud
type: Opaque
data:
  password: WDhHbkdYcVdGUlg5V3Z3ZQ== 

---
apiVersion: v1
kind: Service
metadata:
  name: eureka
  namespace: springcloud
  labels:
    app: eureka
spec:
  clusterIP: None
  ports:
    - port: 8761
      targetPort: 8761
  selector:
    app: eureka

---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: eureka-set
  namespace: springcloud
spec:
  serviceName: eureka
  replicas: 2
  selector:
    matchLabels:
      app: eureka
  template:
    metadata:
      labels:
        app: eureka
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: eureka
          image: eureka:v4
          ports:
          - containerPort: 8761
          env:
            - name: APP_NAME
              value: "eureka"
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: SPRING_APPLICATION_NAME
              value: "igg-report-registry" 
            - name: PASSWORD
              valueFrom:
                secretKeyRef:
                  name: mysecret
                  key: password
            - name: APP_OPT
              value: "
                     --eureka.instance.hostname=${POD_NAME}.${APP_NAME}
                     --registerWithEureka=true
                     --fetchRegistry=true
                     --eureka.instance.preferIpAddress=false
                     --eureka.client.serviceUrl.defaultZone=http://${SPRING_APPLICATION_NAME}:${PASSWORD}@eureka-set-0.${APP_NAME}:8761/eureka/,http://${SPRING_APPLICATION_NAME}:${PASSWORD}@eureka-set-1.${APP_NAME}:8761/eureka/
                     "

由于我的注册中心有做安全校验，因此配置注册中心的时候需要填写密码，而密码不想明文写入到配置的话需要自己加密下,加密方法如下:

[root@master eureka]# echo -n 'X8GnGXqWFRX9Wvwe' | base64
WDhHbkdYcVdGUlg5V3Z3ZQ==

"WDhHbkdYcVdGUlg5V3Z3ZQ=="这个就是经过base64加密后的密码了。
StatefulSet 作为 Controller 为 Pod 提供唯一的标识。它可以保证部署和 scale 的顺序。
StatefulSet是为了解决有状态服务的问题（对应Deployments和ReplicaSets是为无状态服务而设计），其应用场景包括：

稳定的持久化存储，即Pod重新调度后还是能访问到相同的持久化数据，基于PVC来实现
稳定的网络标志，即Pod重新调度后其PodName和HostName不变，基于Headless Service（即没有Cluster IP的Service）来实现
有序部署，有序扩展，即Pod是有顺序的，在部署或者扩展的时候要依据定义的顺序依次依次进行（即从0到N-1，在下一个Pod运行之前所有之前的Pod必须都是Running和Ready状态），基于init containers来实现
有序收缩，有序删除（即从N-1到0）
从上面的应用场景可以发现，StatefulSet由以下几个部分组成：
用于定义网络标志（DNS domain）的Headless Service
用于创建PersistentVolumes的volumeClaimTemplates
定义具体应用的StatefulSet
StatefulSet中每个Pod的DNS格式为statefulSetName-{0..N-1}.serviceName.namespace.svc.cluster.local，其中
serviceName为Headless Service的名字
0..N-1为Pod所在的序号，从0开始到N-1
statefulSetName为StatefulSet的名字
namespace为服务所在的namespace，Headless Servic和StatefulSet必须在相同的namespace
.cluster.local为Cluster Domain

apollo配置中心

由于apollo配置中心需要mysql数据库的支持，而mysql不是部署在k8s集群内，因此需要建立一个service可以代理到外部服务去
endpoints.yml

kind: Service
apiVersion: v1
metadata:
  name: my-service
  namespace: springcloud
spec:
  clusterIP: 192.168.186.163
  ports:
    - protocol: TCP
      port: 3306
      targetPort: 3306

---
kind: Endpoints
apiVersion: v1
metadata:
  name: my-service
  namespace: springcloud
subsets:
  - addresses:
      - ip: 10.0.3.166
    ports:
      - port: 3306

这样通过访问这个serviceIp的3306就能访问到外部节点的mysql

apollo-config部署

先自建apollo-config的镜像

FROM java:8
COPY www /data/www
entrypoint java ${JVM:=-Xms1024m -Xmx1024m} -Djava.security.egd=file:/dev/./urandom -Dfile.encoding=UTF8 -jar /data/www/apollo-configservice-1.5.1.jar ${APP_OPT}

apollo-config.yml

apiVersion: v1
kind: Secret
metadata:
  name: apollo-config
  namespace: springcloud
type: Opaque
data:
  password: QXF1YXl3RGJDYVNoeEczbA== 

---
apiVersion: v1
kind: Service
metadata:
  name: apollo-config
  namespace: springcloud
  labels:
    app: apollo-config
spec:
  ports:
    - port: 8888
      targetPort: 8888
  selector:
    app: apollo-config

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: apollo-config
  namespace: springcloud
spec:
  replicas: 1
  selector:
    matchLabels:
      app: apollo-config
  template:
    metadata:
      labels:
        app: apollo-config
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: apollo-config
          image: apollo-config:v4
          ports:
          - containerPort: 8888
          env:
            - name: SPRING_APPLICATION_NAME
              value: "apollo-configservice" 
            - name: PASSWORD
              valueFrom:
                secretKeyRef:
                  name: apollo-config
                  key: password
            - name: APP_OPT
              value: "
                     --eureka.instance.hostname=apollo-config
                     --registerWithEureka=true
                     --fetchRegistry=true
                     --eureka.instance.preferIpAddress=false
                     --spring.datasource.url=jdbc:mysql://my-service:3306/apollo_config?useSSL=false&characterEncoding=utf-8
                     --eureka.client.serviceUrl.defaultZone=http://${SPRING_APPLICATION_NAME}:${PASSWORD}@eureka-set-0:8761/eureka/,http://${SPRING_APPLICATION_NAME}:${PASSWORD}@eureka-set-1:8761/eureka/
                     "

apollo-admin部署

自建apollo-admin镜像

FROM java:8
copy ./www /data/www
entrypoint java ${JVM:=-Xms1024m -Xmx1024m} -Djava.security.egd=file:/dev/./urandom -Dfile.encoding=UTF8 -jar /data/www/apollo-adminservice-1.5.1.jar ${APP_OPT}

apollo-admin.yml

apiVersion: v1
kind: Secret
metadata:
  name: apollo-admin
  namespace: springcloud
type: Opaque
data:
  password: QXF1YXl3RGJDYVNoeEczbA== 

---
apiVersion: v1
kind: Service
metadata:
  name: apollo-admin
  namespace: springcloud
  labels:
    app: apollo-admin
spec:
  ports:
    - port: 8070
      targetPort: 8070
  selector:
    app: apollo-admin

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: apollo-admin
  namespace: springcloud
spec:
  replicas: 1
  selector:
    matchLabels:
      app: apollo-admin
  template:
    metadata:
      labels:
        app: apollo-admin
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: apollo-admin
          image: apollo-admin:v1
          ports:
          - containerPort: 8070
          env:
            - name: APP_NAME
              value: apollo-admin
            - name: SPRING_APPLICATION_NAME
              value: "apollo-adminservice" 
            - name: PASSWORD
              valueFrom:
                secretKeyRef:
                  name: apollo-admin
                  key: password
            - name: APP_OPT
              value: "
                     --eureka.instance.hostname=${APP_NAME}
                     --registerWithEureka=true
                     --fetchRegistry=true
                     --spring.datasource.url=jdbc:mysql://my-service:3306/apollo_config?useSSL=false&characterEncoding=utf-8
                     --eureka.instance.preferIpAddress=false
                     --eureka.client.serviceUrl.defaultZone=http://${SPRING_APPLICATION_NAME}:${PASSWORD}@eureka-set-0.${APP_NAME}:8761/eureka/,http://${SPRING_APPLICATION_NAME}:${PASSWORD}@eureka-set-1.${APP_NAME}:8761/eureka/
                     "

apollo-portal部署

自建apollo-portal镜像

FROM java:8
copy ./www /data/www
entrypoint java ${JVM:=-Xms1024m -Xmx1024m} -Djava.security.egd=file:/dev/./urandom -Dfile.encoding=UTF8 -jar /data/www/apollo-portal-1.5.1.jar

apollo-portal.yml

apiVersion: v1
kind: Service
metadata:
  name: apollo-portal
  namespace: springcloud
  labels:
    app: apollo-portal
spec:
  ports:
    - port: 8060
      targetPort: 8060
  selector:
    app: apollo-portal

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: apollo-portal
  namespace: springcloud
spec:
  replicas: 1
  selector:
    matchLabels:
      app: apollo-portal
  template:
    metadata:
      labels:
        app: apollo-portal
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: apollo-portal
          image: apollo-portal:v1
          ports:
          - containerPort: 8060

由于apollo-portal需要web页面对外访问，因此需要使用ingress配合service，将其对外访问

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-test
  namespace: springcloud
spec:
  rules:
    #定义域名
    - host: apollo.ingress.com
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: apollo-portal
              port:
                number: 8060

api-common部署

api-common有使用到apollo的配置中心
所以配置文件如下:

app:
  id: iggreport-api-common
apollo:
  meta: http://apollo-config:8888
  bootstrap:
    enabled: true
    namespaces: application,application.yml

构建镜像Dockerfile

FROM java:8
copy ./www /data/www
entrypoint java ${JVM:=-Xms1024m -Xmx1024m} -Djava.security.egd=file:/dev/./urandom -Dfile.encoding=UTF8 -jar /data/www/iggreport-api-common-2.1.4.RELEASE.jar ${APP_OPT}

api.yml

apiVersion: v1
kind: Service
metadata:
  name: api-common-service
  namespace: springcloud
  labels:
    app: api-common-service
spec:
  ports:
    - port: 8090
      targetPort: 8090
  selector:
    app: api-common
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-common
  namespace: springcloud
spec:
  replicas: 2
  selector:
    matchLabels:
      app: api-common
  template:
    metadata:
      labels:
        app: api-common
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: api-common
          image: api:v1
          ports:
          - containerPort: 8090
          env:
            - name: APP_NAME
              value: api-common-service
            - name: APP_OPT
              value: "
                     --eureka.instance.hostname=${APP_NAME}  #这里记得要用service的名字，否则网关使用此hostname无法访问
                     "

gateway部署

由于网关需要redis,因此也需要建个service代理到外部服务的redis集群
endpoint.yml

kind: Service
apiVersion: v1
metadata:
  name: redis-service
  namespace: springcloud
spec:
  ports:
    - protocol: TCP
      port: 7000
      targetPort: 7000

---
kind: Endpoints
apiVersion: v1
metadata:
  name: redis-service
  namespace: springcloud
subsets:
  - addresses:
      - ip: 10.0.3.163
      - ip: 10.0.3.164
      - ip: 10.0.3.165
    ports:
      - port: 7000

自建镜像

FROM java:8
copy ./www /data/www
entrypoint java ${JVM:=-Xms1024m -Xmx1024m} -Djava.security.egd=file:/dev/./urandom -Dfile.encoding=UTF8 -jar /data/www/iggreport-gateway-2.2.0.RELEASE.jar ${APP_OPT}

gateway.yml

apiVersion: v1
kind: Service
metadata:
  name: gateway-service
  namespace: springcloud
  labels:
    app: gateway-service
spec:
  ports:
    - port: 8080
      targetPort: 8080
  selector:
    app: gateway
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: gateway
  namespace: springcloud
spec:
  replicas: 2
  selector:
    matchLabels:
      app: gateway
  template:
    metadata:
      labels:
        app: gateway
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: gateway
          image: gateway:v1
          ports:
          - containerPort: 8080

然后由于gateway需要对外，因此也需要加入到ingress管理

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-test
  namespace: springcloud
spec:
  rules:
    #定义域名
    - host: gateway.ingress.com
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: gateway-service
              port:
                number: 8080

这样一个完整的springcloud + apollo的统一配置中心就基于k8s搭建完毕

[root@master k8s]# kubectl get po -o wide -n springcloud
NAME                             READY   STATUS    RESTARTS   AGE    IP           NODE   NOMINATED NODE   READINESS GATES
api-common-5c59b9b649-4rkrm      1/1     Running   0          116m   10.10.2.93   cdh2   <none>           <none>
api-common-5c59b9b649-swpq8      1/1     Running   0          116m   10.10.1.88   cdh3   <none>           <none>
apollo-admin-6dfcf44f8b-flwtc    1/1     Running   0          25h    10.10.2.83   cdh2   <none>           <none>
apollo-config-5d7bddf55c-cn4ph   1/1     Running   0          24h    10.10.2.90   cdh2   <none>           <none>
apollo-portal-bb7d4685d-srw2r    1/1     Running   0          24h    10.10.2.92   cdh2   <none>           <none>
eureka-set-0                     1/1     Running   0          24h    10.10.2.91   cdh2   <none>           <none>
eureka-set-1                     1/1     Running   0          24h    10.10.1.87   cdh3   <none>           <none>
gateway-949d4f958-4ft8x          1/1     Running   0          9m7s   10.10.1.89   cdh3   <none>           <none>
gateway-949d4f958-b4wc7          1/1     Running   0          9m7s   10.10.2.94   cdh2   <none>           <none>

[root@master k8s]# kubectl get svc -o wide -n springcloud
NAME                 TYPE        CLUSTER-IP        EXTERNAL-IP   PORT(S)    AGE     SELECTOR
api-common-service   ClusterIP   192.168.144.76    <none>        8090/TCP   116m    app=api-common
apollo-admin         ClusterIP   192.168.118.145   <none>        8070/TCP   25h     app=apollo-admin
apollo-config        ClusterIP   192.168.134.51    <none>        8888/TCP   5d23h   app=apollo-config
apollo-portal        ClusterIP   192.168.31.38     <none>        8060/TCP   25h     app=apollo-portal
eureka               ClusterIP   None              <none>        8761/TCP   3d23h   app=eureka
gateway-service      ClusterIP   192.168.230.235   <none>        8080/TCP   9m27s   app=gateway
my-service           ClusterIP   192.168.186.163   <none>        3306/TCP   5h10m   <none>
redis-service        ClusterIP   192.168.143.180   <none>        7000/TCP   75m     <none>

k8s springcloud apollo
springcloud + apollo的微服务化部署创建namespace 建立属于微服务体系下的namesp...
apollo 配置提前加载
碰到的问题项目基于SpringCloud+Apollo配置中心，在开发阶段，Feign远程调用的时候url是写死...
Apollo 简介
分布式配置中心有很多，springcloud的config、百度的disconfig、携程的apollo、淘宝的d...
springcloud通过apollo动态配置logback
注：文中例子基于apollo官方提供的demo修改。1.application.properties配置通过ap...
springcloud in k8s springcloud项目
一、创建spring cloud eureka的k8s的yaml eureka-deployment.yaml h...
springcloud in k8s springcloud项目
一、创建spring cloud eureka的k8s的yaml eureka-deployment.yaml h...
springcloud in kubernetes 目录
最近工作的项目用到了springcloud和k8s，顺便总结和编写最简单的部署和开发流程。总体框架流程如下： 1...
kubernetes[k8s]服务间通讯[调用]
前言: 最近把SpringCloud 全家桶切换到k8s集群上,第一步就是把注册中心去掉. 注册中心的核心的作用:...
Apollo-v8.0安装踩坑记录
按照Apollo官网的安装说明进行apollo/apollo_software_installation_guid...
Apollo配置中心原理分析
关键字：Apollo原理、Apollo动态刷新机制什么是Apollo？ Apollo（阿波罗）是携程框架部门研...