Living Documentation — Using Cucumber Reports as Specs

Every project has a documentation problem. Requirements are written once, code is written once, and then both evolve — but not together. Six months after launch, the Confluence page says the password must be 8–16 characters. The code enforces 8–64. Which one is right? Nobody knows. This is the documentation drift problem, and it's universal.

BDD's answer is living documentation: specification and verification fused into the same artifact.

What living documentation means

A Cucumber feature file is a specification. When Cucumber runs it against your application, it becomes a verified specification. The test report that comes out says: "as of this commit, these scenarios are true about the system." Not "this is what we intended" — "this is what the system actually does."

This is the fundamental difference from a wiki or a Word document:

• A wiki says: "This is how the system is supposed to work. (Last updated by someone who left the company in 2022.)"
• A feature file says: "This is how the system actually works — proven by a test run 3 minutes ago."

When requirements change, you update the feature file. The CI pipeline runs. If the step definitions and implementation don't reflect the new requirement, the test fails. The documentation cannot drift from reality, because the documentation is the test.

The anatomy of a living documentation system

Three components work together:

1. Feature files (the specification)

Written collaboratively in Three Amigos sessions. Stored in Git alongside the application code. Owned by the whole team. Updated when requirements change — not after the sprint ends, but as part of the acceptance criteria for the change.

Feature: Fund Transfer
  As a bank customer
  I want to transfer funds between my accounts
  So that I can manage my money
 
  @smoke
  Scenario: Transfer between own accounts succeeds
    Given the user has a savings account with balance £500
    And the user has a current account with balance £100
    When the user transfers £200 from savings to current
    Then the savings account balance should be £300
    And the current account balance should be £300
 
  Scenario: Transfer fails when insufficient funds
    Given the user has a savings account with balance £100
    When the user attempts to transfer £200 from savings to current
    Then the transfer should be declined
    And the user should see "Insufficient funds"

2. CI pipeline (the verification)

Every push to main (or a pull request) triggers the Cucumber suite. A failed scenario means the specification is not satisfied — either the code regressed or the requirement changed and the test needs updating. Both are intentional signals, not noise.

3. Published reports (the communication)

The Cucumber HTML report (or Allure, or cucumber-reporting) rendered and published on every build. Linked from your project wiki. Accessible to stakeholders without VPN or build tool access.

# GitHub Actions: publish living docs after every main branch build
- name: Run BDD suite
  run: mvn verify -Dheadless=true
 
- name: Publish Allure report
  if: always()
  uses: simple-elf/allure-report-action@v1
  with:
    allure_results: target/allure-results
 
- name: Deploy to GitHub Pages
  if: always()
  uses: peaceiris/actions-gh-pages@v3
  with:
    github_token: ${{ secrets.GITHUB_TOKEN }}
    publish_dir: allure-history

Now https://your-org.github.io/your-project shows the latest test run with full scenario details, step-by-step results, and embedded screenshots. Bookmark it. Link it from the product wiki. When a stakeholder asks "does this work?", point them at the green scenarios.

The living documentation workflow

When a requirement changes:

1. Update the Gherkin scenario in the feature file
2. Update the step definition if the interface changed
3. Update the implementation
4. Push — CI runs — report turns green

The documentation (the feature file), the test, and the implementation are all updated atomically. There is no "document the change later" step that gets forgotten.

Tools that extend the model

Pickles — generates a searchable HTML site from your .feature files, without running the tests. Useful for stakeholder-facing documentation portals where you want the specification readable independently of the CI run.

Serenity BDD — an alternative to plain Cucumber that generates richer living documentation: narrative reports, step-level screenshots, business-readable summaries. More setup than vanilla Cucumber, but the output reads like a product specification rather than a test report.

Allure — stores test history across builds, so the living documentation shows not just "did this pass today" but "has this been consistently passing for 30 days". Trend charts make regressions visible.

What stakeholders actually do with it

The value is specific: when a product owner or business analyst can answer "does the system handle expired password reset links?" by opening a URL and finding a green scenario, the documentation is working. When they have to ask a developer — or check a Confluence page that was last updated 18 months ago — it isn't.

Practical adoption steps for a team just starting:

1. Link the latest Cucumber HTML report from the project README
2. Share the report link in the sprint review
3. When a stakeholder asks a question about system behaviour, answer it by pointing at the feature file — not by describing it verbally
4. When the answer is "that scenario doesn't exist yet," add it as a new story backlog item

Traditional docs vs living documentation

⚠️ Common mistakes

• Publishing reports from a broken build and calling it documentation. A report with 40% failing scenarios published on a team wiki tells stakeholders the system doesn't work — which may or may not be true. Only publish reports from stable builds. Use the not @wip tag filter to exclude in-progress work.
• Feature files written after development. If you write Gherkin to describe what the code already does (rather than specifying what it should do), the living documentation is a test report in disguise — not a shared specification. The collaboration value is lost.
• No stakeholder access to the report. A CI report that only developers can access is not living documentation — it's a private test log. Publishing the report (GitHub Pages, S3, Netlify) and sharing the link is what makes it documentation.
• Treating feature files as QA-only artifacts. When product owners don't review feature files, the specification drift happens in the other direction: tests pass but the business requirement has changed and nobody updated the Gherkin. Feature files need the same change control as user stories.

🎯 Practice task

Publish a living documentation report from your project. 30–40 minutes.

1. Ensure your runner is configured with both html:target/cucumber-reports/report.html and json:target/cucumber-reports/report.json plugins.
2. Add the allure-cucumber7-jvm dependency and plugin. Run mvn verify. Then run allure serve target/allure-results and browse the generated report.
3. Find one scenario that verifies something a product owner would recognise as a requirement. Write a one-sentence description of the business value this scenario proves.
4. Stretch: set up GitHub Actions (or any CI tool you have access to) to run the suite on push. Add a step to publish the Cucumber HTML report as a CI artifact. After pushing, locate the artifact in the CI run and confirm a non-developer could browse it to understand what the system does.

Next lesson: the BDD anti-patterns that turn a powerful practice into a maintenance burden — and how to reverse them.