Q26 of 38 · Manual & exploratory
How would you test a feature that depends on a third-party service that's currently flaky?
Manual & exploratoryMidthird-partytest-doublescontract-testingintegration
Short answer
Short answer: Stub or mock the third-party in unit and integration tests so flakiness doesn't poison the suite, run a small set of contract tests against the real service to catch genuine breakage, and gate higher-level E2E tests with a smoke check on the dependency before running.
Detail
Flaky third parties are a chronic problem because their failures show up as your bugs. The architecture-level fix is layered:
- Unit and integration tests use a mock or stub. Replace the third party with a test double that returns deterministic responses (success, known error responses, timeout simulation). This isolates your code from their availability and gives fast, reliable feedback on the integration logic.
- Contract tests run against the real third party periodically (often nightly, not per-commit). A small set verifies your assumptions about its API still hold. If they change their schema, contract tests catch it. Pact, Spectral, and OpenAPI-driven validation are common tools.
- E2E tests run against a deterministic environment. If the third party offers a sandbox (Stripe test mode, Twilio test credentials), use it. Sandbox environments are themselves more reliable than production third parties.
- Flaky-dependency mitigation in test code. For tests that genuinely have to hit the third party live, build in retry-with-backoff for known-flaky operations, monitor the third-party status page, and skip-with-warning rather than fail when the dependency is down ("Stripe sandbox is degraded; payment tests skipped"). Don't let their flake be your flake.
- Push back on the dependency. If a third party is flaky enough to disrupt your tests, that's also customer-impacting. Open tickets, escalate, and consider redundancy if the business depends on it.
The senior signal: you don't try to "make the test resilient to a flaky third party" by adding retries — you remove the third party from the path that needs to be reliable.
// WHAT INTERVIEWERS LOOK FOR
Layered approach (mock → contract → live sandbox), awareness that mocks alone don't catch API drift, and recognition that retrying-the-test isn't the right fix.
// COMMON PITFALL
Saying 'I add retries' — that masks the flake and slows the suite.