美文网首页
2022-06-10

2022-06-10

作者: brightranger | 来源:发表于2022-06-10 16:58 被阅读0次

    Ansible playbook 自动化部署方案

    1. 方案背景

    当前Jlite非标部署比较复杂,需要部署的服务比较多,每个服务又有很多配置。一是部署时间比较长、二是部署不够标准(基础软件版本、日志文件配置等不统计)。为了使部署更加简单、方便、快捷,特提供本部署方案。

    ansible playbook自动化部署技术目前比较成熟、文档齐全,固本次选择ansible playbook为自动化部署技术方案。

    2. Ansible 简介

    Ansible是一种常用的自动运维化工具,基于python开发,分布式,无需客户端,轻量级,配置语言采用YAML。

    2.1. Ansible的特性:

    1. 模块化:调用特定的模块,完成特殊的任务。
    2. Paramiko(python对ssh的实现),PyYaml,jinja2(模块语言)三个关键模块。
    3. 支持自定义模块,可使用任何编程语言写模块。
    4. 基于python语言实现。
    5. 部署简单,基于python和SSH(默认已安装),agentless,无需代理不依赖KPI(无需SSL)。
    6. 安全,基于OpenSSH
    7. 幂等性:一个任务执行一次和执行n遍效果一样,不因重复执行带来意外情况。
    8. 支持playbook编排任务,YAML格式,编排任务,支持丰富的数据结构。
    9. 较强大的多层解决方案role。

    2.2 Ansible的作用目标:

    1. 自动化部署APP
    2. 自动化管理配置项
    3. 自动化的持续交付
    4. 自动化的云服务管理

    3. Playbook简介

    playbook是ansible用于配置,部署和管理托管主机剧本,通过playbook的详细描述,执行其中一系列tasks,可以让远程主机达到预期状态。playbook是由一个或多个"play"组成的列表。

    也可以说,playbook字面意思及剧本,现实中由演员按剧本表演,在ansible中由计算机进行安装,部署应用,提供对外服务,以及组织计算机处理各种各样的事情

    3.1. Playbook 核心元素

    • Hosts

      执行的远程主机列表(应用在哪些主机上)

    • Tasks

      任务集

    • Variables

      内置变量或自定义变量在playbook中调用

    • Templates

      可替换模板文件中的变量并实现一些简单逻辑的文件

    • Handlers/Notify

      两个为一个组合,由特定条件触发的操作,满足条件执行,不满足不执行。

    • Tags

      指定某条任务执行,用于选择运行playbook中的部署代码。

    4. 部署组件图

    结合Ansible Playbook、Docker、Docker Compose、Docker Swarm集群、Nacos功能特点。可以使用如下组件图方案对Jlite实施自动化部署。

    1654850242(1).png

    4.1. 本地nacos配置信息说明

    因为Ansible Playbook并没有提供一个界面配置,所以我们使用Nacos作为所有部署时参数的配置中心。

    • 以客户为 worksapce(工作空间),SAAS环境的话,每个SAAS环境为一个worksapce
    • 每个服务为一个group
    • 一个服务的不同配置为dataId
    4.1.1 远程服务器资源
    • 远程服务器资源配置信息 remote-hosts.yaml

      ---
      remote_hosts:
        swarm:
          # docker swarm 集群的leader服务器ip地址
          leader:
          - 10.10.205.115
        # docker swarm 集群的manager服务器ip地址
        manager:
          - 10.10.205.114
        # docker swarm 集群的worker服务器ip地址
        worker:
          - 10.10.205.95
          - 10.10.205.94
      ...
      
    4.1.2 服务部署信息
    • 需要部署的服务参数 service-docker-compose-parameter.yaml ,playbook用于生成最终的docker-compose.yaml文件。

      ---
      # 根docker-compose.yaml文件类似
      services:
        #jlite服务
        jlite:
          #版本号
          version: 3.7.11
          #外发时的镜像文件名
        image: osdfo3234234220220611
        #端口号
        ports:
            - "1081:1081"
        deploy:
          #节点数
            replicas: 2
            
        #redis服务
        redis:
          ports:
            - "6397:6397"
        password: jsicp
        
        #lucene服务
        lucene:
          ports:
            - "8108:8108"
      ...
      
    4.1.3 每个服务依赖的配置

    每个服务自已依赖的nacos配置不变,只需要要本地配置就好,自动化脚本自动下载后上传到远程部署服务器的nacos上。

    5. 服务的 docker-compose.yaml

    每个服务的docker-compose.yaml文件由研发提供,可以在里面可以使用参数占位符。参数由前面的service-docker-compose-parameter.yaml定义。

    version: '3'
    
    services:
      jlite:
        image: {{ services.jlite.image }}
        container_name: jlite_node_noe
        dns_search: .
        volumes: 
          - /etc/localtime:/etc/localtime:ro
          - /etc/timezone:/etc/timezone:ro
          - /home/jht/logs/jlite:/home/jht/logs
        environment:
          - TZ=Asia/Shanghai
        ports: {{ services.jlite.ports }}
        networks:
          - swarm_net
        deploy:
          mode: replicated
          replicas: {{ services.jlite.deploy.replicas }}
          restart_policy:
            condition: on-failure
            delay: 5s
            max_attempts: 3
            window: 120s
          update_config:
            parallelism: 2
            delay: 5s
        depends_on:
          - redis
          - lucene
          - emqx
          - nginx
      redis:
        image: redis_docker_images_name
        container_name: redis_container
        environment:
          - TZ=Asia/Shanghai
        ports: {{ services.redis.ports }}
        networks:
          - swarm_net
        deploy:
          mode: replicated
          replicas: {{ services.redis.deploy.replicas }}
          restart_policy:
            condition: on-failure
            delay: 5s
            max_attempts: 3
            window: 120s
          update_config:
            parallelism: 2
            delay: 5s
    

    6. 安装包

    安装包由OA流程走完后,运维工具跑定时任务,定时打镜像到仓库中。

    7. 日志文件

    可以在docker swarm的集群服务器上共享挂载磁盘的方式,集中在一起。如下

    services:
      jlite:
        volumes:
          - /home/docker/logs/jlite:/home/jht/logs
    

    8. 自动化部署流程图

    image.png

    9. ansible playbook 环境安装配置

    9.1 获取读取nacos配置文件 playbook脚本

    ---
    - hosts: localhost
      remote_user: jht
      vars:
        - workspace: 93549ceb-522d-4137-b02d-7aa05ab10852
      tasks:
        - name: login nacos
          uri:
            url: http://10.10.204.114:8848/nacos/v1/auth/users/login?username=nacos&password=nacos
            method: post
            return_content: yes
            status_code: 200
          register: login
        - debug:
            var: login.json.accessToken
        - name: download config
          uri:
            url: http://10.10.204.114:8848/nacos/v1/cs/configs?show=all&dataId=serviceConfiguration&group=ansible&tenant={{ workspace }}&accessToken={{ login.json.accessToken }}&namespaceId={{ workspace }}
            method: get
            return_content: yes
            status_code: 200
          register: config
        - debug:
            var: config.json.content
        - name: write file
          file:
            path: /home/jht/playbook/templates
            state: directory
        - name: create file
          file:
            path: /home/jht/playbook/templates/serviceConfiguration.yaml
            state: touch
        - name: write file content
          blockinfile:
            path: /home/jht/playbook/templates/serviceConfiguration.yaml
            block: "{{ config.json.content }}"
    
    

    9.2 安装docker

    - name: install required packages
      yum:
        name: "{{ item }}"
        state: present
      with_items:
        - yum-utils
        - device-mapper-persistent-data
        - lvm2
    - name: add docker repo to /etc/yum.repos.d
      shell: yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
      register: result
    - name: debug
      debug:
        var: not result.failed
    - name: install docker-ce
      yum:
        name: docker-ce
        state: present
      when: not result.failed
    - name: install docker-ce-cli
      yum:
        name: docker-ce-cli
        state: present
      when: not result.failed
    - name: install containerd.io
      yum:
        name: containerd.io
        state: present
      when: not result.failed
    - name: create /etc/docker
      file:
        path: "{{ docker_config_dir }}"
        state: directory
    - name: start docker service
      systemd:
        name: docker
        state: started
        enabled: true
    - name: provide docker-ce configfile
      template:
        src: daemon.json.j2
        dest: /etc/docker/daemon.json
      notify: restart docker
    

    9.3 安装docker compose

    - name: copy docker-compose-Linux-x86_64 to /usr/local/bin
      copy:
        src: ../files/docker-compose-linux-x86_64
        dest: "{{ docker_compose_dir }}/docker-compose-Linux-x86_64"
    
    - name: install docker-compose
      shell: cd "{{ docker_compose_dir }}" && mv docker-compose-Linux-x86_64 docker-compose && chmod +x docker-compose
    

    9.4 安装docker swarm leader

    ---
    - name: Create leader node of docker swarm cluster
      hosts: leader
      remote_user: jht
      gather_facts: yes
      become: yes
      become_method: sudo
      become_user: root
      vars:
        ip_addr: "{{ ansible_default_ipv4['address'] }}"
      tasks:
        - name: show ip_add
          debug:
            var: ip_addr
    
        - name: Ensure docker is running
          become: yes
          service:
            name: docker
            state: started
    
        - name: Get docker info
          shell: docker info
          register: docker_info
    
        - name: show docker_info
          debug:
            var: docker_info
    
        - name: Create docker swarm leader node
          shell: docker swarm init --advertise-addr {{ ip_addr }}
          when: "docker_info.stdout.find('Swarm: inactive') != -1"
    
        - name: Get docker swarm manager token
          shell: docker swarm join-token -q manager
          register: manager_token
    
        - name: Get docker swarm worker token
          shell: docker swarm join-token -q worker
          register: worker_token
    
        - name: Output tokens
          debug:
            msg: "{{ manager_token.stdout }} ----------------- {{ worker_token.stdout }}"
    
        - name: Add tokens to dummy host, to be shared between multiple hosts
          add_host:
            name: swarm_tokens_host
            worker_token: "{{ worker_token.stdout }}"
            manager_token: "{{ manager_token.stdout }}"
            leader_ip_addr: "{{ ip_addr }}"
    
        - name: Show nodes of docker swarm
          shell: docker node ls
    

    9.5 设置 docker swarm manager

    ---
    - name: Create manager nodes of docker swarm cluster
      hosts: managers
      remote_user: jht
      become: true
      become_method: sudo
      become_user: root
      vars:
        manager_token: "SWMTKN-1-0xwaprlwrnc1ep0839lpuwgl2muq10vxu09forqgh7ojo4hx89-2fx5aapfq3uwq65slca5ph99r"
        leader_ip_addr: "10.10.205.115"
      tasks:
        - name: Ensure docker is running
          service:
            name: docker
            state: started
    
        - name: Get docker info
          shell: docker info
          register: docker_info
    
        - name: Join as manager node
          shell: docker swarm join --token {{ manager_token }} {{ leader_ip_addr }}:2377
          when: "docker_info.stdout.find('Swarm: inactive') != -1"
          retries: 3
          delay: 20
    ...
    
    

    9.6 设置 docker swarm workers

    ---
    - name: Create docker swarm worker nodes
      gather_facts: yes
      hosts: workers
      remote_user: jht
      become: yes
      become_method: sudo
      become_user: root
      vars:
        worker_token: "SWMTKN-1-0xwaprlwrnc1ep0839lpuwgl2muq10vxu09forqgh7ojo4hx89-5yps35hmuo6eda9cncl2vra39"
        leader_ip_addr: "10.10.205.115"
      tasks:
        - name: Ensure docker is running
          become: yes
          service:
            name: docker
            state: started
    
        - name: Get docker info
          shell: docker info
          register: docker_info
    
        - name: Join as worker node
          shell: docker swarm join --token {{ worker_token }} {{ leader_ip_addr }}:2377
          when: "docker_info.stdout.find('Swarm: inactive') != -1"
          retries: 3
          delay: 20
    ...
    

    10. 部署实例

    10.1 部署 nginx

    • docker-compose.yaml

      version: '3.6'
      
      services:
        nginx:
          image: nginx:1.17.6-alpine
          ports:
            - "80:80"
          networks:
            - swarm_net
          deploy:
            mode: replicated
            replicas: 2
            restart_policy:
              condition: on-failure
              delay: 5s
              max_attempts: 3
              window: 120s
            update_config:
              parallelism: 2
              delay: 5s
      networks:
        swarm_net:
          driver: overlay
      
    • deploy-nginx-service.yml
    ---
    - name: Deploy services to docker swarm cluster
      gather_facts: no
      hosts: leader
      remote_user: jht
      become: true
      become_method: sudo
      become_user: root
      vars:
        src_docker_compose: ./docker-compose.yml
        dest_docker_compose: ~/docker-compose.yml
        stack_name: first_stack
      tasks:
        - name: Upload docker compose file
          template:
            src: "{{ src_docker_compose }}"
            dest: "{{ dest_docker_compose }}"
    
        - name: Deploy services
          shell: docker stack deploy --with-registry-auth --prune --compose-file {{ dest_docker_compose }} {{ stack_name }}
    
        - name: Pause for 10 seconds to wait for services being running
          pause:
            seconds: 10
    
        - name: Ensure services deployed
          shell: docker service ls
          register: service_output
    
        - name: Output service list
          debug:
            msg: "{{ service_output.stdout_lines }}"
    
        - name: Show stacks
          shell: docker stack ls
    ...
    

    相关文章

      网友评论

          本文标题:2022-06-10

          本文链接:https://www.haomeiwen.com/subject/hbebmrtx.html