在节点上创建并行执行的多个阶段

Question

我们的跨平台项目要求Jenkins在许多不同的平台上执行,并为每个平台进行适当的测试和打包.我能够结合使用parallel,node但只能在一个单独的,stage并得到这一点(见下文)

我希望能够将它分成多个阶段.我想创建的阶段是:

• 构建:使用工件构建库/项目
• UnitTest:构建并运行单元测试
• TestApp:使用工件构建和运行测试应用程序
• 输入阶段:想上传？
• 上传:将工件上传到外部服务器

我会做以下事情:

1. 在阶段之间复制一堆对象(使用stash)
2. 需要保持node使用的一致性.(节点上生成的对象label应相应地标记在适当的节点上label).我需要为每个节点唯一标记每个存储.

这不是更低效吗？我会人为地复制大量数据,以便我可以创建阶段.

def checkoutAndBuild(Map args) {
    node("${args.nodeName}") {

        checkout([$class: 'GitSCM', 
                    branches: scm.branches,  
                    doGenerateSubmoduleConfigurations: scm.doGenerateSubmoduleConfigurations, 
                    extensions: scm.extensions + 
                                [[$class: 'SubmoduleOption', 
                                    disableSubmodules: false, 
                                    parentCredentials: false, 
                                    recursiveSubmodules: true, 
                                    reference: '', 
                                    trackingSubmodules: false]] +
                                [[$class: 'CleanCheckout']], 
                    userRemoteConfigs: scm.userRemoteConfigs
                ])

        step([$class: 'CopyArtifact', 
            filter: "AppCommon/*/**, cmake/**/*, core/**/*, thirdparty/prebuilt/${args.prebuiltDir}/**/*, tools/**/*", 
            fingerprintArtifacts: true, 
            projectName: "${args.engineDependency_Job}", 
            selector: [$class: 'SpecificBuildSelector', buildNumber: "${args.engineDependency_BuildNo}"], 
            target: 'engine'])

        dir("build/${args.buildDir}") {
            echo 'Building..'
            if (isUnix()) {
                sh './build.sh Release'
            } else {
                bat 'build.bat Release'
            }
        }

        def extras = args.additionalArtifacts ? ", ${args.additionalArtifacts}" : ""
        archiveArtifacts artifacts: "dist/**/*${extras}", fingerprint: true

        dir("test/build") {
            echo 'Building test App'
            sh "./full.sh ${args.buildDir} Release"
        }
    }
}

pipeline {
    agent none

    stages {
        stage('Info') {
            agent any
            steps {
                echo "Running ${env.JOB_NAME} / ${env.BUILD_ID} on ${env.JENKINS_URL}"
            }
        }

        stage('Build') {
            steps {
                parallel (
                    ios: {
                        checkoutAndBuild nodeName: 'iOS', prebuiltDir: 'ios', buildDir: 'ios', engineDependency_Job: 'engine_iOS_Release', engineDependency_BuildNo: String.valueOf(engineBuild.ios)
                    },
                    tvos: {
                        checkoutAndBuild nodeName: 'tvOS', prebuiltDir: 'tvos', buildDir: 'tvos', engineDependency_Job: 'engine_tvOS_Release', engineDependency_BuildNo: String.valueOf(engineBuild.tvos)
                    },
                    android: {
                        checkoutAndBuild nodeName: 'Android', prebuiltDir: 'android', buildDir: 'AndroidNative', engineDependency_Job: 'engine_Android_Release', engineDependency_BuildNo: String.valueOf(engineBuild.android), additionalArtifacts: 'src/java/*'
                    })
            }
        }
        stage('Test Build') {
            steps {
                echo 'Testing...'
            }
        }

        stage('Deploy') {
            steps {
                echo 'Deploying...'
            }
        }

    }

    post {
        success {
            slackSend channel: '#builds',
                        color: 'good',
                        message: "${currentBuild.fullDisplayName} succeeded. (<${env.BUILD_URL}|Open>)"

        }
        failure {
            slackSend channel: '#builds',
                        color: 'danger',
                        message: "${currentBuild.fullDisplayName} failed. (<${env.BUILD_URL}|Open>)"
        }
    }
}

Answer 1

遗憾的是,声明性管道语法在您的情况下不是很灵活.在阶段之间存储对象会非常繁重,并且会导致同步阶段,其中最快的构建目标在每个阶段之前等待较慢的阶段.

如果您不需要同步运行阶段,我建议创建一个包含所有构建目标的所有阶段的大方法,即要并行运行的那些阶段.要区分阶段,您可以例如将节点名称附加到每个阶段标签.

def checkoutBuildTestDeploy(Map args) {
    node("${args.nodeName}") {

        stage("Build ${args.nodeName}") {
            checkout([$class: 'GitSCM', ... ])
            // And other build steps ...
        }
        stage("Unit test ${args.nodeName}") {
            // Unit test steps
        }
        stage("Test app ${args.nodeName}") {
            // Test steps
        }
        stage("Deploy ${args.nodeName}") {
            // Input answer and upload
        }
    }
}

pipeline {
    agent none
    stages {
        stage('Info') {
            agent any
            steps {
                echo "Running ${env.JOB_NAME} / ${env.BUILD_ID} on ${env.JENKINS_URL}"
            }
        }

        stage('Run builds parallel') {
            steps {
                parallel (
                    ios: {
                        checkoutBuildTestDeploy nodeName: 'iOS', prebuiltDir: 'ios', buildDir: 'ios', engineDependency_Job: 'engine_iOS_Release', engineDependency_BuildNo: String.valueOf(engineBuild.ios)
                    },
                    tvos: {
                        checkoutBuildTestDeploy nodeName: 'tvOS', prebuiltDir: 'tvos', buildDir: 'tvos', engineDependency_Job: 'engine_tvOS_Release', engineDependency_BuildNo: String.valueOf(engineBuild.tvos)
                    },
                    android: {
                        checkoutBuildTestDeploy nodeName: 'Android', prebuiltDir: 'android', buildDir: 'AndroidNative', engineDependency_Job: 'engine_Android_Release', engineDependency_BuildNo: String.valueOf(engineBuild.android), additionalArtifacts: 'src/java/*'
                    })
            }
        }
    }
    post {
        success {
            slackSend channel: '#builds',
                      color: 'good',
                      message: "${currentBuild.fullDisplayName} succeeded. (<${env.BUILD_URL}|Open>)"

        }
        failure {
            slackSend channel: '#builds',
                      color: 'danger',
                      message: "${currentBuild.fullDisplayName} failed. (<${env.BUILD_URL}|Open>)"
        }
    }
}