JMeter in Jenkins and GitHub Actions — Performance Testing with JMeter

Running JMeter in CI/CD transforms performance testing from a periodic manual activity into an automated quality gate. A smoke performance test on every pull request catches regressions before they merge. A nightly load test catches slow degradation before it reaches users. This lesson shows complete, working pipeline configurations for both Jenkins and GitHub Actions.

The CI/CD pipeline pattern

TriggerSchedule, PR, or manual dispatch

Install JMeterDownload binary or pull Docker image

Run Testjmeter -n -t test.jmx -l results.jtl

Generate Report-e -o report/ → HTML dashboard

Quality GateParse .jtl, check error rate & p95

Publish ArtefactsUpload report, notify team

Jenkins pipeline

Prerequisites: Jenkins with the Performance Plugin and HTML Publisher Plugin installed. JDK configured as a Jenkins tool.

pipeline {
    agent any
    tools { jdk 'JDK-21' }
 
    environment {
        JMETER_VERSION = '5.6.3'
        JMETER_HOME    = "${WORKSPACE}/apache-jmeter-${JMETER_VERSION}"
    }
 
    stages {
        stage('Install JMeter') {
            steps {
                sh '''
                    wget -q https://dlcdn.apache.org/jmeter/binaries/apache-jmeter-${JMETER_VERSION}.tgz
                    tar -xzf apache-jmeter-${JMETER_VERSION}.tgz
                '''
            }
        }
 
        stage('Performance Test') {
            steps {
                sh '''
                    ${JMETER_HOME}/bin/jmeter -n \
                      -t test-plans/load-test.jmx \
                      -p environments/staging.properties \
                      -JbaseUrl=${STAGING_URL} \
                      -l results/results.jtl \
                      -e -o reports/
                '''
            }
        }
 
        stage('Quality Gate') {
            steps {
                sh '''
                    python3 scripts/check_thresholds.py \
                      --jtl results/results.jtl \
                      --max-error-rate 1.0 \
                      --max-p95-ms 2000
                '''
            }
        }
    }
 
    post {
        always {
            // Track performance trends across builds
            perfReport(
                sourceDataFiles: 'results/results.jtl',
                errorFailedThreshold: 5,
                errorUnstableThreshold: 1,
                relativeFailedThresholdPositive: 20,
                relativeUnstableThresholdPositive: 10
            )
 
            // Publish HTML dashboard (CSP fix required — see tip below)
            publishHTML([
                reportDir:    'reports',
                reportFiles:  'index.html',
                reportName:   'JMeter Load Test Report',
                keepAll:      true,
                allowMissing: false,
                alwaysLinkToLastBuild: true
            ])
 
            archiveArtifacts artifacts: 'results/*.jtl, reports/**', fingerprint: true
        }
    }
}

The Jenkins Performance Plugin (perfReport) does more than just publish data — it tracks performance across builds and can fail or mark builds as unstable if:

errorFailedThreshold: 5 — fail the build if more than 5% of samples error
relativeFailedThresholdPositive: 20 — fail if response time is 20% higher than the previous build

This gives you automatic regression detection: a build that is slower than the last by more than 20% fails automatically, without manual comparison.

GitHub Actions

name: Performance Tests
 
on:
  schedule:
    - cron: '0 2 * * *'     # nightly at 02:00 UTC
  workflow_dispatch:
    inputs:
      environment:
        description: 'Target environment'
        type: choice
        options: [staging, production]
        default: staging
      users:
        description: 'Virtual users'
        default: '50'
 
jobs:
  load-test:
    runs-on: ubuntu-latest
    env:
      JMETER_VERSION: 5.6.3
 
    steps:
      - uses: actions/checkout@v4
 
      - name: Set up Java 21
        uses: actions/setup-java@v4
        with:
          java-version: '21'
          distribution: 'temurin'
 
      - name: Install JMeter
        run: |
          wget -q https://dlcdn.apache.org/jmeter/binaries/apache-jmeter-${JMETER_VERSION}.tgz
          tar -xzf apache-jmeter-${JMETER_VERSION}.tgz
          echo "$PWD/apache-jmeter-${JMETER_VERSION}/bin" >> $GITHUB_PATH
 
      - name: Run JMeter test
        env:
          TARGET_URL: ${{ secrets[format('{0}_BASE_URL', inputs.environment || 'staging')] }}
          ADMIN_PASSWORD: ${{ secrets.PERF_TEST_PASSWORD }}
        run: |
          jmeter -n \
            -t test-plans/load-test.jmx \
            -p "environments/${{ inputs.environment || 'staging' }}.properties" \
            -JbaseUrl="$TARGET_URL" \
            -JadminPassword="$ADMIN_PASSWORD" \
            -Jvusers="${{ inputs.users || '50' }}" \
            -l results.jtl \
            -e -o report/
 
      - name: Check quality gates
        run: |
          python3 - <<'EOF'
          import csv, sys
 
          jtl_file = 'results.jtl'
          max_error_rate = 1.0   # percent
          max_p95_ms     = 2000  # milliseconds
 
          with open(jtl_file) as f:
              rows = list(csv.DictReader(f))
 
          total  = len(rows)
          errors = sum(1 for r in rows if r['success'] == 'false')
          error_rate = errors / total * 100 if total > 0 else 0
 
          elapsed = sorted(int(r['elapsed']) for r in rows)
          p95_idx = int(len(elapsed) * 0.95)
          p95     = elapsed[p95_idx] if elapsed else 0
 
          print(f"Samples: {total} | Errors: {errors} ({error_rate:.2f}%) | p95: {p95}ms")
 
          failed = False
          if error_rate > max_error_rate:
              print(f"FAIL: error rate {error_rate:.2f}% > {max_error_rate}%")
              failed = True
          if p95 > max_p95_ms:
              print(f"FAIL: p95 {p95}ms > {max_p95_ms}ms")
              failed = True
 
          sys.exit(1 if failed else 0)
          EOF
 
      - name: Upload report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: jmeter-report-${{ github.run_id }}
          path: |
            results.jtl
            report/
          retention-days: 30
 
      - name: Comment on PR
        if: github.event_name == 'pull_request' && always()
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            // Parse key metrics from results.jtl and post as PR comment
            github.rest.issues.createComment({
              issue_number: context.issue.number,
              owner: context.repo.owner,
              repo: context.repo.repo,
              body: '📊 JMeter results uploaded — [view report](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})'
            });

Tiered pipeline strategy

Run different test plans at different stages:

Test type	Trigger	Users	Duration	Purpose
Smoke perf	Every PR	1	60s	Catch obvious regressions
Sanity load	Every merge to main	10	5 min	Verify under light load
Full load	Nightly	100	30 min	SLA validation
Stress	Weekly	500+	Variable	Find breaking point
Soak	Pre-release	50	8+ hours	Stability validation

The smoke perf test runs in under 2 minutes — acceptable on every PR. The full load test runs nightly and takes 30+ minutes — not appropriate for every PR. Using separate test plan files parameterised by environment and user count keeps all tests in the same repository.

Quality gate script

The inline Python quality gate in the GitHub Actions example is functional but minimal. A more robust version:

#!/usr/bin/env python3
# scripts/check_thresholds.py
import argparse, csv, sys, statistics
 
def main():
    p = argparse.ArgumentParser()
    p.add_argument('--jtl',           required=True)
    p.add_argument('--max-error-rate', type=float, default=1.0)
    p.add_argument('--max-p95-ms',     type=int,   default=2000)
    p.add_argument('--max-p99-ms',     type=int,   default=5000)
    args = p.parse_args()
 
    with open(args.jtl) as f:
        rows = list(csv.DictReader(f))
 
    total    = len(rows)
    errors   = sum(1 for r in rows if r.get('success') == 'false')
    elapsed  = sorted(int(r['elapsed']) for r in rows)
    p95      = elapsed[int(total * 0.95)] if elapsed else 0
    p99      = elapsed[int(total * 0.99)] if elapsed else 0
    err_rate = errors / total * 100 if total else 0
 
    print(f"Results: n={total}, errors={errors} ({err_rate:.2f}%), p95={p95}ms, p99={p99}ms")
 
    failures = []
    if err_rate > args.max_error_rate:
        failures.append(f"Error rate {err_rate:.2f}% > {args.max_error_rate}%")
    if p95 > args.max_p95_ms:
        failures.append(f"p95 {p95}ms > {args.max_p95_ms}ms")
    if p99 > args.max_p99_ms:
        failures.append(f"p99 {p99}ms > {args.max_p99_ms}ms")
 
    if failures:
        print("QUALITY GATE FAILED:")
        for f in failures:
            print(f"  - {f}")
        sys.exit(1)
 
    print("Quality gate passed.")
 
if __name__ == '__main__':
    main()

⚠️ Common mistakes

Not handling JMeter's exit code. JMeter exits 0 even when 30% of requests failed. Without a quality gate script that reads the .jtl file, CI/CD pipelines mark load tests as "passing" regardless of results. The quality gate is not optional — it is the mechanism that makes CI/CD performance tests meaningful.
Running the full load test on every PR. A 30-minute, 100-user load test on every pull request blocks merges for 30 minutes, exhausts CI runners, and trains developers to ignore failures. Run smoke performance tests on PRs (1 user, 60 seconds) and reserve full load tests for nightly scheduled runs.
Storing secrets in properties files committed to git. The pipeline should inject secrets via CI/CD secret variables (GitHub Actions secrets, Jenkins credentials store). Properties files in the repository should contain only non-sensitive defaults and be safe to make public.

🎯 Practice task

Build a minimal CI/CD pipeline for your JMeter test.

Create scripts/check_thresholds.py with the full quality gate script above.
Create .github/workflows/perf-test.yml with the GitHub Actions configuration. Set the trigger to workflow_dispatch for now (manual trigger only).
Add your test.jmx to the repository. Set up a GitHub secret STAGING_BASE_URL pointing to a public test endpoint.
Trigger the workflow manually from the GitHub Actions UI. Watch the run. Confirm the report artefact is attached.
Change max_error_rate in the quality gate to 0.0 (zero tolerance). Re-trigger. If any requests fail, the pipeline should now fail the quality gate step — confirm the job exits non-zero.
Add a schedule trigger to run nightly at 03:00 UTC. Commit. Verify the workflow appears in the scheduled workflows list in GitHub Actions.