Running Karate in CI/CD Pipelines

A test suite that only runs locally is a personal safety net. A test suite that runs on every pull request is a team-wide quality gate. Getting Karate into CI is straightforward — it's a standard mvn test invocation — but the details around environment variables, secrets, report artefacts, and scheduling make the difference between a CI job that's genuinely useful and one the team learns to ignore. This lesson covers both GitHub Actions and Jenkins, plus the patterns that make Karate CI configurations maintainable.

GitHub Actions — complete workflow

name: Karate API Tests
 
on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]
 
jobs:
  api-tests:
    runs-on: ubuntu-latest
 
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
 
      - name: Set up Java 21
        uses: actions/setup-java@v4
        with:
          java-version: '21'
          distribution: 'temurin'
          cache: maven
 
      - name: Run Karate tests
        run: mvn clean verify -Dtest=ParallelRunner
        env:
          KARATE_ENV: staging
          API_BASE_URL: ${{ secrets.STAGING_API_URL }}
          ADMIN_PASSWORD: ${{ secrets.STAGING_ADMIN_PASSWORD }}
 
      - name: Upload Karate reports
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: karate-reports-${{ github.run_number }}
          path: |
            target/karate-reports/
            target/cucumber-html-reports/
          retention-days: 30

Four things worth noting. cache: maven caches the local Maven repository so subsequent runs don't re-download all dependencies — a one-line win on a team that runs CI dozens of times a day. mvn clean verify runs tests and then the reporting plugin in the same command. if: always() on the upload step ensures reports are saved even when tests fail — which is exactly when you need them. retention-days: 30 keeps artefacts for a month without accumulating indefinitely.

Environment variables and secrets

karate.env drives environment switching in karate-config.js. Set it as an environment variable in the CI step:

env:
  KARATE_ENV: staging

Credentials and base URLs that differ between environments belong in GitHub Secrets, not in the YAML file:

env:
  API_BASE_URL: ${{ secrets.STAGING_API_URL }}
  ADMIN_PASSWORD: ${{ secrets.STAGING_ADMIN_PASSWORD }}

In karate-config.js, read them with java.lang.System.getenv():

function fn() {
    var env = karate.env || 'dev';
    var config = {};
 
    if (env == 'staging') {
        config.baseUrl = java.lang.System.getenv('API_BASE_URL') || 'https://api.staging.myapp.com';
    } else {
        config.baseUrl = 'https://api.dev.myapp.com';
    }
 
    config.adminPassword = java.lang.System.getenv('ADMIN_PASSWORD') || 'DevAdminPass';
    return config;
}

This pattern means the feature files contain no environment-specific values — they reference baseUrl and adminPassword from config, which are injected at runtime by the CI environment.

Jenkins — Declarative Pipeline

pipeline {
    agent any
    tools {
        maven 'Maven-3.9'
        jdk   'JDK-21'
    }
    environment {
        KARATE_ENV     = 'staging'
        API_BASE_URL   = credentials('staging-api-url')
        ADMIN_PASSWORD = credentials('staging-admin-password')
    }
    stages {
        stage('API Tests') {
            steps {
                sh 'mvn clean verify -Dtest=ParallelRunner'
            }
        }
    }
    post {
        always {
            // Parse JUnit XML — shows trend graph on Jenkins dashboard
            junit 'target/surefire-reports/*.xml'
            // Publish Cucumber HTML via the Cucumber Reports Jenkins plugin
            cucumber 'target/karate-reports/*.json'
            // Archive the built-in Karate report as a download link
            archiveArtifacts artifacts: 'target/karate-reports/**', fingerprint: true
        }
        failure {
            emailext(
                to: 'qa-team@mycompany.com',
                subject: "FAILED: Karate API Tests — ${env.JOB_NAME} #${env.BUILD_NUMBER}",
                body: "Tests failed. Download the report from: ${env.BUILD_URL}artifact/target/karate-reports/"
            )
        }
    }
}

credentials('staging-api-url') reads from Jenkins's Credentials store — the Jenkins equivalent of GitHub Secrets. The cucumber step requires the Cucumber Reports Jenkins Plugin. The junit step is built into Jenkins and parses Surefire XML automatically.

Scheduling

Run the full suite on a schedule, not just on push. A nightly regression catches degradation that incremental PR runs miss:

# GitHub Actions — add to the on: block
on:
  schedule:
    - cron: '0 2 * * 1-5'    # 2 AM UTC, Monday–Friday
  push:
    branches: [main]
  pull_request:
    branches: [main]

A common pattern: smoke-only on PR (fast feedback), full regression nightly (complete coverage). Use Karate tags to separate them:

# PR job — smoke only
- name: Smoke tests
  run: mvn test -Dtest=ParallelRunner -Dkarate.options="--tags @smoke"
 
# Nightly job — full suite
- name: Full regression
  run: mvn verify -Dtest=ParallelRunner

The CI pipeline flow

Karate + Gatling for performance testing

Karate has a unique integration with Gatling: the same .feature files that run as functional tests can be used as Gatling simulations for performance testing. Add karate-gatling to your pom.xml and write a Gatling simulation that references your feature files:

// UserLoadSimulation.java
public class UserLoadSimulation extends KarateSimulation {
    {
        setUp(
            karateFeature("classpath:users/users.feature")
                .inject(rampUsers(100).during(Duration.ofSeconds(60)))
        );
    }
}

This runs the same users.feature file under Gatling's load model — 100 virtual users over 60 seconds — and produces Gatling's HTML performance report. You write the feature once, use it twice: functional coverage and load coverage. This is a stretch capability; the functional suite comes first, but it's worth knowing the option exists.

⚠️ Common mistakes

• Using mvn test instead of mvn verify in CI when the Masterthought plugin is configured. mvn test runs the tests but skips the verify phase where the reporting plugin runs. The Karate report is there, but the Cucumber HTML report is missing. Switch to mvn verify in the CI command and the reporting plugin runs automatically.
• Not using if: always() on the upload step. When tests fail, the job's default behaviour is to skip steps that haven't been declared with if: always(). Without it, the report upload is skipped on failure — exactly when you need the report most. Always add if: always() to artefact upload steps in test jobs.
• Setting the thread count too high for the CI runner. GitHub Actions ubuntu-latest runners have 2 vCPUs. Running .parallel(8) on a 2-CPU machine doesn't give 8× speed — threads compete for the same 2 cores and the scheduler overhead can make it slower than .parallel(4). Start with .parallel(2) in CI and only increase if measurements show a benefit.

🎯 Practice task

Build a working GitHub Actions pipeline for your Karate project. 40–50 minutes.

1. Create .github/workflows/karate-tests.yml in your project root. Add the full workflow from this lesson — checkout, Java setup, mvn clean verify, artefact upload with if: always().
2. Add a GitHub repository secret named STAGING_API_URL with the value https://jsonplaceholder.typicode.com. Update karate-config.js to read java.lang.System.getenv('API_BASE_URL') when karate.env == 'staging'.
3. Push the workflow file to your repository (or a fork). Watch the Actions tab — confirm the job runs, tests execute, and the artefact appears as a downloadable zip containing the Karate report.
4. Force one test to fail by committing a wrong assertion. Confirm the Actions job turns red. Download the artefact and confirm the HTML report shows the failure detail.
5. Add a second job to the workflow that runs only on the schedule '0 2 * * 1-5'. Give it a different name (nightly-full-regression) and run without the --tags @smoke filter. Confirm the workflow file has two jobs — smoke-tests (on PR) and nightly-full-regression (on schedule).
6. Stretch: if you have a Jenkins instance available, write the Declarative Pipeline from this lesson. Wire the junit step to target/surefire-reports/*.xml and run the pipeline. Confirm the test trend graph appears on the Jenkins job page after two runs.

Next chapter: the capstone project — building a complete Karate test suite for the TeamHub User Management API.