Cucumber Reports and Third-Party Integrations

Karate's built-in HTML report is excellent for local debugging. For CI dashboards, historical trend data, and reports shared with stakeholders who don't run tests themselves, you want something richer: a report that tracks pass rates across runs, highlights new failures vs recurring ones, and presents results in a polished format. Karate generates Cucumber-compatible JSON automatically, and that JSON feeds directly into the ecosystem of third-party reporting tools that teams have built around Cucumber.

The Cucumber JSON output — the integration bridge

Every tool in this lesson consumes the same JSON file that Karate generates. Enable it with one flag in the runner:

Results results = Runner
    .path("classpath:")
    .outputCucumberJson(true)
    .parallel(4);

After the run, target/karate-reports/ contains one .json file per feature file. Each file follows the Cucumber JSON format — the same schema that tools like Cluecumber, Allure, and the Masterthought Cucumber Reports plugin have consumed for years. Add this flag once; every integration works from the same output.

maven-cucumber-reporting — rich HTML with trends

The Masterthought maven-cucumber-reporting plugin reads the Cucumber JSON and produces a polished multi-page HTML report with feature breakdowns, tag analysis, step timing charts, and a failure summary. Add it to pom.xml:

<plugin>
    <groupId>net.masterthought</groupId>
    <artifactId>maven-cucumber-reporting</artifactId>
    <version>5.8.1</version>
    <executions>
        <execution>
            <id>generate-reports</id>
            <phase>verify</phase>
            <goals><goal>generate</goal></goals>
            <configuration>
                <projectName>TeamHub API Tests</projectName>
                <outputDirectory>${project.build.directory}/cucumber-html-reports</outputDirectory>
                <jsonFiles>
                    <param>${project.build.directory}/karate-reports/*.json</param>
                </jsonFiles>
            </configuration>
        </execution>
    </executions>
</plugin>

Run mvn verify instead of mvn test — the verify phase runs after test, so the Karate JSON is ready when the plugin executes. The report appears at target/cucumber-html-reports/overview-features.html.

The Masterthought report shows scenario pass rates, feature-level breakdowns, tag-based filtering, and step-level timing. Unlike Karate's built-in report, it doesn't show the raw HTTP request/response bodies — use Karate's report for debugging, use Masterthought for stakeholder communication.

Allure integration

Allure is a widely-used reporting tool that integrates with most Java test frameworks. The cleanest Karate + Allure path is via the Cucumber JSON:

1. Run Karate with .outputCucumberJson(true).
2. Use the allure-commandline tool to generate an Allure report from the Cucumber JSON.
3. Publish the report to Allure's hosting or upload it as a CI artifact.

# Install Allure CLI (once)
brew install allure
 
# Generate Allure report from Cucumber JSON
allure generate target/karate-reports/ --clean -o target/allure-report
allure open target/allure-report

Allure's distinctive feature over Masterthought is its trend data: run it against a history of Cucumber JSON outputs and it shows pass-rate trends, flakiness scores, and test duration trends over time. This requires storing the Allure history directory between CI runs — typically done by uploading and downloading it as a CI artifact.

JUnit XML — automatic CI integration

Karate generates JUnit-compatible XML at target/surefire-reports/ alongside the HTML report. Jenkins's JUnit publisher and GitHub Actions's test reporting both parse this format automatically, without any plugin configuration:

• GitHub Actions: the junit-reporter action or the default PR check annotations pick up Surefire XML and display pass/fail counts directly on the pull request.
• Jenkins: the junit post-build step at target/surefire-reports/*.xml shows test trends on the pipeline dashboard.

No configuration in the Karate runner is needed for JUnit XML — it's generated by default whenever Karate runs through a JUnit 5 runner class.

The reporting pipeline

Choosing the right tool

Use Karate's built-in report when: debugging a specific failed scenario, reviewing the raw HTTP conversation, doing local development.

Use maven-cucumber-reporting when: generating a shareable report for stakeholders, running nightly suites where you want a clean per-run summary with feature-level breakdowns.

Use Allure when: tracking trends across many runs, identifying flaky tests over time, integrating with an existing Allure-based reporting setup.

Use JUnit XML when: you need CI-native annotations (PR pass/fail checks, Jenkins test history) without setting up an external tool.

Most teams use Karate's built-in report plus JUnit XML (automatic), and add maven-cucumber-reporting or Allure once the suite is mature enough that trend data becomes useful.

⚠️ Common mistakes

• Running mvn test instead of mvn verify with the Masterthought plugin. The plugin is bound to the verify phase, which runs after test. Running mvn test executes Karate but skips the plugin — no Masterthought report appears. Use mvn verify or add -Dskip.verify=false in CI scripts that already use mvn test.
• Pointing the plugin's <jsonFiles> at the wrong path. The glob ${project.build.directory}/karate-reports/*.json works when Karate generates files there. If you've customised the Karate output directory, update the path to match. A missing glob pattern makes the plugin generate an empty report without an error.
• Generating Allure reports without persisting history between CI runs. Allure trend data requires the history from the previous run. If each CI run starts fresh, Allure generates a single-run report with no trend charts. Store the target/allure-report/history/ directory as a CI artifact, download it before the next run, and copy it into target/allure-report/history/ before generating — this is the standard pattern for keeping Allure trends across runs.

🎯 Practice task

Wire up one third-party report tool end to end. 30–40 minutes.

1. Add .outputCucumberJson(true) to your ParallelRunner. Run mvn test. Confirm .json files appear in target/karate-reports/ alongside the .html files.
2. Add the maven-cucumber-reporting plugin to pom.xml with the configuration from this lesson. Run mvn verify. Open target/cucumber-html-reports/overview-features.html. Compare it to Karate's built-in report — note what the Masterthought report adds (tag analysis, step charts) and what it lacks (request/response bodies).
3. Install the Allure CLI (brew install allure or download from the Allure GitHub releases). Run allure generate target/karate-reports/ -o target/allure-report. Open with allure open target/allure-report. Read the Overview page.
4. Force two scenarios to fail. Run again. Compare the failure presentation in Karate's HTML, Masterthought's report, and Allure. Note which format is easiest to diagnose the root cause from.
5. In GitHub Actions (or write the YAML even if you don't have a live repo), add a step that uses actions/upload-artifact to save both target/karate-reports/ and target/cucumber-html-reports/. Use if: always() so they upload even on failure.
6. Stretch: read the Masterthought plugin's GitHub README and find the classifications configuration option. Add a classification that records the environment name and Java version in the report header. This is useful when the same report is shared across staging and production runs.

Next lesson: running Karate in CI/CD pipelines — complete GitHub Actions and Jenkins configurations, environment switching, and scheduling.