Jenkins Shared Library - pipelin

作者: heichong | 来源:发表于2023-03-08 10:55 被阅读0次

使用Jenkins shared library构建docker
jenkins-shared-library
Unity Jenkins打包(3) Pipeline
Pipeline介绍
探究报告20170810
Xcode插件路径
安装Alcatraz插件
Jenkins系列:环境配置介绍
Jenkins pipeline自动化运行发布部署
xcode 第三方插件目录

背景

在自动化发布场景中，包括代码拉取、测试、编译、发布等阶段，而这些阶段都需要写在Jenkinsfile内。
比如下面这个：


def _dockerUrl='10.3.23.191:9902'
def _dockerNamespace='ks'
def _dockerName='operation-platform'
def _apiFolderName="${_dockerName}-api"
def _jobFolderName="${_dockerName}-job"
def _apiImageName="${_dockerUrl}/${_dockerNamespace}/${_dockerName}-api"
def _jobImageName="${_dockerUrl}/${_dockerNamespace}/${_dockerName}-job"

def createVersion() {
    // 版本号
    return new Date().format('yyyyMMddHHmmss') + "_${env.BUILD_ID}"
}
pipeline {
    options {
      timeout(time: 10, unit: 'MINUTES')
    }
    agent {
        docker {
            image '10.3.23.191:9902/devops/maven:3.8.2-openjdk-8'
            args '-v $HOME/.m2:/root/.m2 -v /root/.ssh:/root/.ssh'
        }
    }
    parameters {
        string(name: 'BRANCH', defaultValue: 'dev', description: '要部署的代码分支名称')
        choice(name: 'PROFILE',  choices:['test', 'pre', 'prod'],description: '要部署的环境')
    }

    environment {
        _DOCKER_TAG=createVersion()
        // harbor 登录凭证
        _HARBOR_CREDS=credentials('harbor-admin')
        // 服务器登录凭证
        _APPSERVER_CREDS=credentials('dev_root')
    }

    stages {
        stage('Init') {
            steps {
                echo "============================================"
                sh " echo => 构建的分支: $BRANCH"
                sh " echo => 发布的环境: $PROFILE"

                echo "============================================"
            }
        }
        stage('Git pull') {
            steps {
                sh 'date '
                sh 'ls -al '
                echo '开始拉取代码 ..'
                checkout([$class: 'GitSCM', branches: [[name: '*/$BRANCH']], extensions: [], userRemoteConfigs: [[credentialsId: '67f1f5b0-d6fb-40fa-9799-4a6dbc85a328', url: 'https://gitlab.com.cn/operation-platform/operation-platform.git']]])
                echo '代码拉取成功'
            }
        }
        stage('Test') {
            steps {
                echo 'Testing..'
            }
        }
        stage('Build Project') {
            steps {
                sh 'pwd'
                echo '开始编译代码..'
                sh "mvn --version"
                sh "mvn clean package -D maven.test.skip=true"
                echo '代码编译成功'
            }
        }
        stage('Build Docker Image') {
            steps {
                sh 'pwd'
                sh 'ls -al '
                echo "===========START,开始编译Docker Image : ${_apiImageName} ==========="
                echo "----> 删除历史镜像 ${_apiFolderName} ..."
                sh "docker rmi --force ${_apiFolderName} | true"
                //only retain last 3 images，自动删除老的容器，只保留最近2个
                sh """docker rmi \$(docker images | grep ${_apiFolderName} | sed -n  '3,\$p' | awk '{print \$3}') || true"""
                echo "----> 编译镜像 ${_apiFolderName} ..."
                sh "docker build -f ./${_apiFolderName}/Dockerfile  -t ${_apiImageName}:${_DOCKER_TAG} --build-arg JAR_NAME=./${_apiFolderName}/target/*.jar ."
                echo "----> docker push 镜像 ${_apiFolderName} ..."
                sh "echo ${_HARBOR_CREDS_PSW} | docker login -u ${_HARBOR_CREDS_USR} ${_dockerUrl} --password-stdin"
                sh "docker push ${_apiImageName}:${_DOCKER_TAG}"
                echo "===========END，编译Docker Image : ${_apiImageName} ==========="

                echo "===========START,开始编译Docker Image : ${_jobImageName} ==========="
                echo "----> 删除历史镜像 ${_jobFolderName} ..."
                sh """docker rmi \$(docker images | grep ${_jobFolderName} | sed -n  '3,\$p' | awk '{print \$3}') || true"""
                echo "----> 编译镜像 ${_jobFolderName} ..."
                sh "docker build -f ./${_jobFolderName}/Dockerfile  -t ${_jobImageName}:${_DOCKER_TAG} --build-arg JAR_NAME=./${_jobFolderName}/target/*.jar ."
                echo "----> docker push 镜像 ${_jobFolderName} ..."
                sh "docker push ${_jobImageName}:${_DOCKER_TAG}"
                echo "===========END，编译Docker Image : ${_jobImageName} ==========="
            }
        }
        stage('Deploy test') {
            when {
                expression { params.PROFILE == "test" }
            }
            steps {
                echo 'Deploying....'
                script {
                    // SSH Pipeline Steps plugin
                    def remoteServer = [:]
                    remoteServer.name = 'test'
                    remoteServer.host = '10.3.23.191'
                    remoteServer.allowAnyHosts = true
                    remoteServer.user = "${_APPSERVER_CREDS_USR}"
                    remoteServer.password = "${_APPSERVER_CREDS_PSW}"

                    //=========================== deploy api =====================================
                    sshCommand remote: remoteServer, command: """
                    pwd
                    str1=`docker ps -a | grep -w ${_apiImageName}  | awk '{print \$1}'`
                    str2=`docker images | grep -w ${_apiImageName}  | awk '{print \$3}'`
                    if [ "\$str2" !=  "" ] ; then
                        if [ "\$str1" !=  "" ] ; then
                            docker stop `docker ps -a | grep -w ${_apiImageName}  | awk '{print \$1}'`
                            docker rm `docker ps -a | grep -w ${_apiImageName}  | awk '{print \$1}'`
                            #docker rmi --force `docker images | grep -w ${_apiImageName}  | awk '{print \$3}'`
                        else
                            docker rmi --force `docker images | grep -w ${_apiImageName}  | awk '{print \$3}'`
                        fi
                    fi
                    """
                    sshCommand remote: remoteServer, command: """
                    docker run -d -p 9010:9010 \\
                        -e JAVA_OPTS="-Xmx1g -Xms1g -Djava.security.egd=file:/dev/.urandom" \\
                        -e PARAMS="--server.port=9010 --spring.profiles.active=test " \\
                        --restart always ${_apiImageName}:${_DOCKER_TAG}
                    """

                    //=========================== deploy job =====================================

                    sshCommand remote: remoteServer, command: """
                    pwd
                    str1=`docker ps -a | grep -w ${_jobImageName}  | awk '{print \$1}'`
                    str2=`docker images | grep -w ${_jobImageName}  | awk '{print \$3}'`
                    if [ "\$str2" !=  "" ] ; then
                        if [ "\$str1" !=  "" ] ; then
                            docker stop `docker ps -a | grep -w ${_jobImageName}  | awk '{print \$1}'`
                            docker rm `docker ps -a | grep -w ${_jobImageName}  | awk '{print \$1}'`
                            #docker rmi --force `docker images | grep -w ${_jobImageName}  | awk '{print \$3}'`
                        else
                            docker rmi --force `docker images | grep -w ${_jobImageName}  | awk '{print \$3}'`
                        fi
                    fi
                    """
                    sshCommand remote: remoteServer, command: """
                    docker run -d -p 9011:9011 -p 9012:9012 \\
                        -e JAVA_OPTS="-Xmx1g -Xms1g -Djava.security.egd=file:/dev/.urandom" \\
                        -e PARAMS="--server.port=9011 --spring.profiles.active=test --xxl-job.executor.ip=10.3.23.191 --xxl-job.executor.port=9012" \\
                        --restart always ${_jobImageName}:${_DOCKER_TAG}
                    """
                }
            }
        }

    }
    post {
        always {
            // 清理工作区
            cleanWs()
        }
    }
}

大量脚本堆砌在Jenkinsfile内，让用户（开发者）感觉非常乱；理论上应该只保留用户关心的东西，其他代码都对用户不可见
测试、编译、发布等过程，针对同一种环境，其代码都是一样的。我要想办法让这些共同的过程抽象出来，让所有项目的pipeline共用。这样以后有修改，也只需要修改公共的部分即可。

而Jenkins Shared Library就可以解决以上问题

安装Pipeline Groovy Libraries插件

具体过程可参考https://www.jianshu.com/p/fb12e47d7b11
其他几个插件因为在groovy脚本中使用到了，所以需要提前安装。

创建 Shared Library Git 仓库

首先创建一个groovy项目，并提交到gitlab中。
groovy项目的目录必须为以下格式：

+- src                     # Groovy source files
|   +- org
|       +- foo
|           +- Bar.groovy  # for org.foo.Bar class
+- vars
|   +- foo.groovy          # for global 'foo' variable
|   +- foo.txt             # help for 'foo' variable
+- resources               # resource files (external libraries only)
|   +- org
|       +- foo
|           +- bar.json    # static helper data for org.foo.Bar

src为源代码目录，执行Pipeline时，该目录将添加到类路径中。
vars目录托管定义可从Pipeline访问的全局脚本(一般我们可以在这里编写标准化脚本)。通常，每个.groovy文件的基本名称应使用驼峰（camelCased）模式
resources目录允许libraryResource从外部库中使用步骤来加载相关联的非Groovy文件，比如一些公共的配置文件（json/yaml等）。

以下就是我定义的全局库目录

+--- resources
|   +--- pipeline.yaml
+--- src
|   +--- com
|   |   +--- test
|   |   |   +--- devops
|   |   |   |   +--- enums
|   |   |   |   |   +--- EnvTypeEnum.groovy
|   |   |   |   |   +--- NetworkEnum.groovy
|   |   |   |   |   +--- ProjectTypeEnum.groovy
|   |   |   |   +--- utils
|   |   |   |   |   +--- tool.groovy
+--- vars
|   +--- buildImage.groovy
|   +--- deployImage.groovy
|   +--- deployImageBySSH.groovy
|   +--- paramPageRender.groovy
|   +--- paramPrint.groovy
|   +--- pipelineStart.groovy
|   +--- pipelineStartMaven.groovy

其中build.gradle内容如下：

plugins {
    id 'groovy'
}

group 'com.test.devops'
version '1.0-SNAPSHOT'

sourceSets {
    main {
        groovy {
            srcDir 'src'
            srcDir 'vars'
        }
        resources {
            srcDir 'resources'
        }
    }
    test {
        groovy {
            srcDir 'test'
        }
    }
}


repositories {
    mavenCentral()
}

dependencies {
    implementation 'org.apache.groovy:groovy:4.0.2'
    testImplementation 'org.junit.jupiter:junit-jupiter-api:5.8.1'
    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.8.1'
}

test {
    useJUnitPlatform()
}

编写好脚本以后，把脚本提交到gitlab中。

Jenkins中配置全局库

进入Manage Jenkins » Configure System » Global Pipeline Libraries

image.png
点新增：

image.png

image.png
配置完后，点击保存

Jenkinsfile

配置完全局库以后，我们就可以在Jenkinsfile中直接引用此Library

首先在Library项目定义了/var/pipelineStart.groovy
其内容如下：

#!groovy

def call(cfg) {

    pipeline {
        agent {
            docker {
                image "${cfg.global.dockerAgent}"
                args "${cfg.global.dockerAgentArgs}"
            }
        }

        options {
            skipDefaultCheckout()  //删除隐式checkout scm语句
            disableConcurrentBuilds() //禁止并行
            timeout(time: 1, unit: 'HOURS')  //流水线超时设置1h
            timestamps()
        }

        environment {
            // 镜像仓库 登录凭证
            _HARBOR_CREDS=credentials("${cfg.global.harborUserId}")
        }
        stages {

            stage('Params') {
                steps {
                    script {
                        // 渲染参数页面
                        def pageRender = paramPageRender()
                        pageRender()
                        echo '==============================参数配置start================================'
                        cfg.envType = params.envType
                        cfg.branch = params.branch

                        cfg.servers = cfg.servers[params.envType];
                        cfg.dockerEnvs = cfg.dockerEnvs[params.envType];
                        cfg.dockerPorts = cfg.dockerPorts[params.envType];
                        cfg.version = tool.createVersion("${params.envType}","${env.BUILD_ID}")
                        cfg.fullImageName = "${cfg.global.harbor}/${cfg.imageGroup}/${cfg.imageName}:${cfg.version}"
                        if(cfg.dockerBuildArg){
                            cfg.dockerBuildArg = "--build-arg ${cfg.dockerBuildArg}"
                        }
                        // 打印参数
                        paramPrint(cfg)
                        echo '==============================参数配置end================================'
                    }
                }
            }
            stage('Checkout') {
                steps {
                    script {
                        echo '==============================Checkout start================================'
                        def extensions = []
                        if(cfg.isSubmodule){
                            extensions[0] = [$class             : 'SubmoduleOption',
                                             disableSubmodules  : false,
                                             parentCredentials  : true,
                                             recursiveSubmodules: true,
                                             shallow            : false,
                                             trackingSubmodules : false]
                        }
                        checkout([$class           : 'GitSCM',
                                  branches         : [[name: "*/${cfg.branch}"]],
                                  extensions       : extensions,
                                  userRemoteConfigs: [[credentialsId: "${cfg.global.gitlabUserId}", url: "${cfg.gitUrl}"]]
                        ])
                        echo '==============================Checkout start================================'
                    }
                }
            }
            stage('Test') {
                steps {
                    echo '==============================Test start================================'
                    // skip
                    echo '==============================Test end================================'
                }
            }
            stage('Build') {
                steps {
                    echo '==============================Build start================================'
                    buildImage(cfg)
                    echo '==============================Build end================================'
                }
            }
            stage('Deploy') {
                steps {
                    echo '==============================Deploy start================================'
                    deployImage(cfg)
                    echo '==============================Deploy end================================'
                }
            }
        }

        post {
            always {
                echo '==============================Clean Workspace start================================'
                cleanWs()
                echo '==============================Clean Workspace end================================'
            }
        }
    }
}

其中buildImage、deployImage等，都是在/vars/文件夹下定义的其他脚本文件。

然后再Jenkinsfile类似这样：

#!groovy   
library 'test-library'

def cfg = [:]
// 项目类型。maven
cfg.put('projectType', 'maven')
// git地址
cfg.put('gitUrl',"https://gitlab.xxxx.com/operation-platform/operation-platform.git")
// 工作空间，根目录的相对路径。默认为.
cfg.put('workspace', './operation-platform-api')
// 镜像名称
cfg.put('imageName', 'operation-platform-api')
// 镜像组，请勿随便定义，需要提前在harbor中创建
cfg.put('imageGroup', 'ks')
// docker镜像build时的参数，格式为：'k1=v1 k2=v2 ...'，具体值请参考Dockerfile ARG
cfg.put('dockerBuildArg', 'JAR_NAME=target/*.jar')

//........其他配置...

// 启动Pipeline
pipelineStart(cfg)

Jenkinsfile通过library 'test-library'来引用共享库
Jenkinsfile中只保留用户关心的配置信息，所有的发布过程脚本全都封装在了共享库里。

使用Jenkins shared library构建docker
关于CI & CD 关于Jenkins 为什么要用shared library 示例安装Jenkinsshared...
jenkins-shared-library
前言随着pipeline交付流水线在团队中的推广，使用pipeline脚本的job也迅速增加。虽然我们已经基于公...
Unity Jenkins打包(3) Pipeline
Jenkins Pipeline 官方文档 https://jenkins.io/doc/book/pipelin...
Pipeline介绍
1.什么是pipeline？在jenkins官方文档是这样介绍pipeline的：Jenkins Pipelin...
探究报告20170810
1、指出static library 与 shared library的区别static library 与 sh...
Xcode插件路径
~/Library/Application Support/Developer/Shared/Xcode
安装Alcatraz插件
mkdir -p ~/Library/Application\ Support/Developer/Shared/...
Jenkins系列:环境配置介绍
Jenkins环境深入理解 Jenkins相关配置文件路径Jenkins工作目录：/Users/Shared/Je...
Jenkins pipeline自动化运行发布部署
JenkinsPipeline是一套插件，支持将连续输送Pipeline实施和整合到Jenkins。Pipelin...
xcode 第三方插件目录
~/Library/Application Support/Developer/Shared/Xcode/Plug...