Multi-Stage Pipelines for Test Execution

A single-stage Jenkins pipeline — build, then test — is a starting point, not a finished product. Real QA pipelines have to handle multiple test types running at different speeds, deployments that only trigger on specific branches, manual parameter inputs for ad-hoc runs, and notifications that tell the right people when something breaks. This lesson builds a complete, production-quality Jenkins pipeline with all of those pieces.

Parameterised builds

Parameters turn a fixed pipeline into a flexible one. A QA engineer can trigger the pipeline with a different browser, environment, or test suite without editing the Jenkinsfile.

pipeline {
    agent any
    tools { maven 'Maven-3.9'; jdk 'JDK-21' }
 
    parameters {
        choice(name: 'BROWSER', choices: ['chrome', 'firefox', 'edge'],
               description: 'Browser to run tests on')
        choice(name: 'ENVIRONMENT', choices: ['staging', 'production'],
               description: 'Target environment')
        booleanParam(name: 'HEADLESS', defaultValue: true,
                     description: 'Run browser in headless mode')
        string(name: 'SUITE', defaultValue: 'smoke.xml',
               description: 'TestNG suite file to run')
    }
 
    stages {
        stage('Checkout') {
            steps { checkout scm }
        }
 
        stage('Build') {
            steps { sh 'mvn clean compile -B' }
        }
 
        stage('Tests') {
            steps {
                sh """
                    mvn test \
                      -DsuiteFile=${params.SUITE} \
                      -Dbrowser=${params.BROWSER} \
                      -Dheadless=${params.HEADLESS} \
                      -Denv=${params.ENVIRONMENT} \
                      -B
                """
            }
        }
    }
 
    post {
        always {
            junit 'target/surefire-reports/*.xml'
        }
    }
}

Parameters appear as input fields in the Jenkins UI when you click "Build with Parameters." They're also settable via the Jenkins API or CLI — useful for triggering parameterised builds from external tools or scripts.

Reference parameters inside the pipeline with ${params.NAME} in Groovy strings, or ${PARAM_NAME} in sh steps using triple-quoted strings.

Conditional stages with `when`

Not every stage should run on every build. Use when to control stage execution:

stage('Regression Tests') {
    when {
        expression { params.ENVIRONMENT == 'staging' }
    }
    steps {
        sh 'mvn test -DsuiteFile=regression.xml -Dheadless=true -B'
    }
}
 
stage('Deploy to Staging') {
    when {
        branch 'main'              // only run when building the main branch
    }
    steps {
        sh './scripts/deploy.sh staging'
    }
}
 
stage('Nightly Cross-Browser') {
    when {
        triggeredBy 'TimerTrigger' // only run on scheduled (cron) triggers
    }
    steps {
        sh 'mvn test -DsuiteFile=cross-browser.xml -Dbrowser=firefox -Dheadless=true -B'
    }
}

Common when conditions: branch 'main' (branch name), expression { ... } (arbitrary Groovy), triggeredBy 'TimerTrigger' (nightly cron), changeRequest() (pull request builds), environment name: 'DEPLOY', value: 'true' (environment variable check).

Parallel stages

Run independent test jobs simultaneously to cut total wall-clock time:

stage('Test') {
    parallel {
        stage('API Tests') {
            steps {
                sh 'mvn test -Dgroups=api -B'
            }
        }
        stage('UI Smoke') {
            steps {
                sh 'mvn test -Dgroups=smoke -Dheadless=true -B'
            }
        }
        stage('Database Tests') {
            steps {
                sh 'mvn test -Dgroups=db -B'
            }
        }
    }
}

Three test types run simultaneously. If your agent fleet has spare capacity, total runtime is the longest individual stage rather than the sum of all three. If all three take 10 minutes each, serial execution is 30 minutes; parallel is 10.

A complete production pipeline

pipeline {
    agent any
    tools { maven 'Maven-3.9'; jdk 'JDK-21' }
 
    parameters {
        choice(name: 'BROWSER', choices: ['chrome', 'firefox', 'edge'])
        choice(name: 'ENVIRONMENT', choices: ['staging', 'production'])
        booleanParam(name: 'HEADLESS', defaultValue: true)
    }
 
    stages {
        stage('Checkout') {
            steps { checkout scm }
        }
 
        stage('Build') {
            steps { sh 'mvn clean compile -B' }
        }
 
        stage('Smoke Tests') {
            steps {
                sh """
                    mvn test -DsuiteFile=smoke.xml \
                      -Dbrowser=${params.BROWSER} \
                      -Dheadless=${params.HEADLESS} \
                      -Denv=${params.ENVIRONMENT} -B
                """
            }
        }
 
        stage('Regression') {
            when {
                expression { params.ENVIRONMENT == 'staging' }
            }
            parallel {
                stage('UI Tests') {
                    steps {
                        sh "mvn test -Dgroups=ui -Dbrowser=${params.BROWSER} -Dheadless=true -B"
                    }
                }
                stage('API Tests') {
                    steps {
                        sh 'mvn test -Dgroups=api -B'
                    }
                }
            }
        }
 
        stage('Deploy to Staging') {
            when { branch 'main' }
            steps {
                sh './scripts/deploy.sh staging'
            }
        }
    }
 
    post {
        always {
            junit 'target/surefire-reports/*.xml'
            archiveArtifacts artifacts: 'target/screenshots/**,target/allure-results/**',
                             allowEmptyArchive: true
        }
        success {
            slackSend channel: '#qa-builds', color: 'good',
                      message: "✅ ${env.JOB_NAME} #${env.BUILD_NUMBER} passed on ${params.BROWSER}"
        }
        failure {
            slackSend channel: '#qa-builds', color: 'danger',
                      message: "❌ ${env.JOB_NAME} #${env.BUILD_NUMBER} failed — ${env.BUILD_URL}"
        }
    }
}

archiveArtifacts stores files on the Jenkins master — screenshots, Allure results, any output your tests produce. allowEmptyArchive: true prevents the step from failing if no screenshots were taken (common on a green run).

env.JOB_NAME, env.BUILD_NUMBER, and env.BUILD_URL are Jenkins built-in environment variables available in every pipeline. Use them in notifications to link directly to the failing build.

CheckoutClone repo from SCM

Buildmvn clean compile — fail fast if compila…

Smoke TestsFast gate on critical paths — every bran…

Regression (parallel)UI + API in parallel — staging only

Deploy to StagingOnly on main branch after all tests pass

post { always }Publish JUnit results + archive artifact…

Environment variables and credentials

Use withCredentials to inject secrets from Jenkins Credentials Manager into a stage:

stage('Tests') {
    steps {
        withCredentials([
            string(credentialsId: 'staging-api-key', variable: 'API_KEY'),
            usernamePassword(credentialsId: 'staging-db',
                             usernameVariable: 'DB_USER',
                             passwordVariable: 'DB_PASS')
        ]) {
            sh 'mvn test -DsuiteFile=smoke.xml -Dheadless=true -B'
            // API_KEY, DB_USER, DB_PASS are available as env vars inside this block
        }
    }
}

Credentials are stored in Jenkins Credentials Manager (Manage Jenkins → Credentials), not in the Jenkinsfile. Never hardcode passwords or API keys in a Jenkinsfile — they end up in version control and build logs.

⚠️ Common mistakes

Not using allowEmptyArchive: true on archiveArtifacts. A green run with no screenshots will fail the artifact step and mark an otherwise-passing build as failed. Always add allowEmptyArchive: true to artifact steps that might produce nothing.
Parallel stages sharing a workspace. On a single agent, parallel stages share the same workspace directory. If both stages write to target/surefire-reports/, they overwrite each other. Either write to separate output directories or use separate agents per parallel branch.
Hardcoding credentials in sh commands. sh 'mvn test -Dpassword=secret' puts the credential in the build log. Use withCredentials — Jenkins masks the value in logs automatically.

🎯 Practice task

Add parameterisation and parallel stages to your Jenkinsfile — 35 minutes.

Take the basic Jenkinsfile from the previous lesson and add a parameters block with a BROWSER choice and a HEADLESS boolean.
Update your test command to reference ${params.BROWSER} and ${params.HEADLESS}.
Add a when { branch 'main' } condition to a deploy stage (the sh step can just echo 'deploying' for practice).
Add a parallel block inside a regression stage that runs two independent test groups simultaneously.
Update post { always { } } to include archiveArtifacts pointing at your test output directory.
Stretch: add a slackSend step in post { failure { } }. Even if you don't have a Slack workspace, write the step — it documents the intent and can be connected later.

The next lesson installs and configures the Jenkins plugins that turn raw build results into rich, browsable test reports.