Karate — Contract-Style API Tests

Write Karate DSL feature files that validate a REST API contract: response schema, required fields, status codes, and data-driven scenarios using Karate's built-in fuzzy matching.

Role

API tester

Difficulty

Beginner

Time limit

60–90 min

Scenario

Your team is adopting Karate DSL for API contract testing. Your task is to write Karate feature files that validate the Reqres.in API (https://reqres.in) — a free hosted REST API designed for testing. 'Contract testing' here means: for each endpoint you define the expected response shape (field names, types, required vs optional) and assert that the API honours that shape on every call, not just a specific value. Your suite must be data-driven, reusable, and runnable by anyone with Java and Maven installed, without any IDE setup instructions.

Requirements

1.Create a karate-config.js that sets baseUrl for at least two environments: 'dev' (https://reqres.in) and 'mock' (a placeholder for a future offline stub) — use karate.env to switch between them
2.Write a feature file that validates the GET /api/users?page=2 response contract using fuzzy matchers: assert data is #array, each item has id (#number), email (#string), first_name (#string), last_name (#string), avatar (#string) — not literal values
3.Write a feature file that covers the user lifecycle: POST /api/users → assert response contains name, job, id (#notnull), createdAt (#notnull) → DELETE /api/users/{id} → assert 204; document the Reqres.in non-persistence behaviour in a comment
4.Write a login feature with a Scenario Outline covering at least 3 variants in an Examples table: valid credentials (200 + token field), missing password (400 + error field), and invalid email format (400 or 200 — assert actual behaviour and document it)
5.Extract the login POST into a reusable common/auth.feature that other features can call via `call read()` to obtain a token without duplicating HTTP setup
6.Write a negative-path feature: GET /api/users/9999 → assert 404 and assert the response body shape (Reqres.in returns an empty object {})
7.Apply @smoke tags to the two most critical scenarios (happy-path GET users list and GET single user) and @regression to all others

Starter data

›API base URL: https://reqres.in (no registration required)
›Login endpoint: POST /api/login — body: {"email": "eve.holt@reqres.in", "password": "cityslicka"} — returns {"token": "QpwL5tpe83ilfN2"}
›User list: GET /api/users?page=2 — each item in the data array has: id, email, first_name, last_name, avatar; response also has page, per_page, total, total_pages
›Single user: GET /api/users/2 — returns {"data": {id, email, first_name, last_name, avatar}, "support": {url, text}}
›Create user: POST /api/users — body: {"name": "morpheus", "job": "leader"} — returns body with id and createdAt added; status 201
›Delete: DELETE /api/users/2 — returns 204 No Content
›Reqres.in note: it is a simulation API — POSTs and DELETEs return correct shapes but mutations do NOT persist; a GET /api/users/{id} after POST will return 404 because the created user was never stored — document this in your feature as expected behaviour, not a defect

Expected deliverables

✓A runnable Maven project: pom.xml with the Karate BOM dependency and Maven Surefire configured for the JUnit Platform
✓karate-config.js at src/test/java with at least two environment configs (dev and mock)
✓KarateRunner.java (JUnit 5 @Karate.Class) that discovers all features with at least 2 parallel threads configured
✓Feature files: users-contract.feature, user-lifecycle.feature, login.feature, user-negative.feature
✓common/auth.feature callable via `call read()` that returns the auth token as a defined variable
✓@smoke and @regression tags applied consistently; running `mvn test -Dkarate.options="--tags @smoke"` executes only smoke scenarios
✓A terminal output or Karate HTML report showing all scenarios passing

Evaluation rubric

Dimension	What reviewers look for
Contract assertion depth	Does the feature use fuzzy matchers (#string, #number, #notnull, #array) to assert field types rather than exact values? Can each contract test pass on any valid response from that endpoint — not just a specific snapshot? Asserting `response.data[0].id == 7` is a value test, not a contract test.
Environment config	Does karate-config.js define at least two environments and switch baseUrl based on karate.env? Are feature files free of hardcoded URLs? Are credentials read from config rather than inlined as string literals?
Reusable feature design	Is auth.feature genuinely reusable — does it return the token as a `def` variable without making assertions that would fail when the feature is called as a setup step? Is it invoked correctly with `call read('classpath:...')` and its result consumed by the caller?
Data-driven coverage	Does the Scenario Outline cover meaningfully different cases (valid credentials, wrong password, missing field) rather than repeating the happy path with minor variations? Are expected status codes and relevant response field assertions specified per Examples row?
Negative path accuracy	Is the 404 response body shape also asserted (not just the status code)? Is the Reqres.in non-persistence behaviour documented as a comment in the relevant feature files so a future reader understands why the GET-after-POST returns 404?
Tagging and runnability	Do @smoke and @regression tags work as filters when passed via -Dkarate.options? Is the full suite runnable with `mvn test` from a fresh clone — no manual steps, no missing config? Does the Karate HTML report appear in target/karate-reports after the run?

Sample solution outline

›karate-config.js: if (karate.env == 'mock') { config.baseUrl = 'http://localhost:8080' } else { config.baseUrl = 'https://reqres.in' }
›users-contract.feature @smoke: GET /api/users?page=2 → status 200 → match response.data[0] == {id: '#number', email: '#string', first_name: '#string', last_name: '#string', avatar: '#string'}
›users-contract.feature: match response.total == '#number' and match response.data == '#array' and assert response.data.length > 0
›user-lifecycle.feature: POST /api/users {name: 'morpheus', job: 'leader'} → status 201 → def createdId = response.id → match response == {name: '#notnull', job: '#notnull', id: '#notnull', createdAt: '#notnull'}
›user-lifecycle.feature: DELETE /api/users/2 → status 204 — # Note: Reqres.in does not persist POST /api/users, so GET /api/users/{createdId} would return 404; documented as expected simulation behaviour
›login.feature Scenario Outline Examples: [eve.holt@reqres.in, cityslicka, 200, '#notnull'], [eve.holt@reqres.in, wrongpassword, 400, null], ['', cityslicka, 400, null]
›common/auth.feature: * def loginResult = call read('classpath:.../login.feature') {email: email, password: password} → * def token = loginResult.token
›user-negative.feature: GET /api/users/9999 → status 404 → match response == {}

Common mistakes

Writing `match response.data[0].id == 7` instead of `match response.data[0].id == '#number'` — this creates a brittle snapshot test rather than a contract test, and will fail whenever the API returns a different id
Hardcoding the base URL inside feature files instead of referencing the `baseUrl` variable from karate-config.js — the suite breaks immediately when the environment changes
Not returning a value from auth.feature — a feature that calls POST /api/login and asserts `status 200` is not reusable; it must `def token = response.token` so callers can consume the result
Treating Reqres.in's non-persistence as a test failure — a GET /api/users/{id} returning 404 after a POST is correct and expected for this simulation API; it must be documented as a comment, not investigated as a defect
Using Scenario Outline with Examples rows that all test the same happy path with slightly different email strings — varied Examples should cover genuinely different code paths (success, auth failure, validation failure)
Forgetting to configure Maven Surefire for the JUnit Platform — without the JUnit Platform provider, `mvn test` finds zero tests and exits 0, giving a false-green CI build
Asserting only the status code on the 404 negative-path test — Reqres.in returns an empty JSON object `{}` on 404, and verifying the body shape is part of the contract

Submission checklist

`mvn test` discovers and runs feature files — not zero tests, and Surefire does not report 'No tests were executed'
karate-config.js is at src/test/java with at least two environments; `mvn test -Dkarate.env=mock` switches the base URL without editing any feature file
users-contract.feature uses fuzzy matchers (#string, #number, #array) for all user object field assertions — no literal value comparisons for variable fields
user-lifecycle.feature covers POST (create) and DELETE, with the Reqres.in non-persistence behaviour documented in a comment
login.feature uses Scenario Outline with at least 3 Examples rows covering a success case and at least two failure cases
common/auth.feature is callable via `call read()` and returns the token as a `def` variable
@smoke and @regression tags are applied; `mvn test -Dkarate.options="--tags @smoke"` runs only the smoke scenarios
Karate HTML report is generated in target/karate-reports after `mvn test` completes

Extension ideas

+Add a performance assertion to every GET scenario using Karate's built-in `responseTime` variable: `assert responseTime < 500` to catch latency regressions without a separate load testing tool
+Replace the mock environment placeholder with a Karate MockServer or WireMock stub to enable fully offline testing and deterministic response shapes
+Extend the login Scenario Outline with an external JSON data file (`read('login-variants.json')`) containing 10+ rows to maximise input coverage without bloating the feature file itself