Q26 of 38 · Manual & exploratory

How would you test a feature that depends on a third-party service that's currently flaky?

Manual & exploratoryMidthird-partytest-doublescontract-testingintegration

Short answer

Short answer: Stub or mock the third-party in unit and integration tests so flakiness doesn't poison the suite, run a small set of contract tests against the real service to catch genuine breakage, and gate higher-level E2E tests with a smoke check on the dependency before running.

Detail

Flaky third parties are a chronic problem because their failures show up as your bugs. The architecture-level fix is layered:

  1. Unit and integration tests use a mock or stub. Replace the third party with a test double that returns deterministic responses (success, known error responses, timeout simulation). This isolates your code from their availability and gives fast, reliable feedback on the integration logic.
  2. Contract tests run against the real third party periodically (often nightly, not per-commit). A small set verifies your assumptions about its API still hold. If they change their schema, contract tests catch it. Pact, Spectral, and OpenAPI-driven validation are common tools.
  3. E2E tests run against a deterministic environment. If the third party offers a sandbox (Stripe test mode, Twilio test credentials), use it. Sandbox environments are themselves more reliable than production third parties.
  4. Flaky-dependency mitigation in test code. For tests that genuinely have to hit the third party live, build in retry-with-backoff for known-flaky operations, monitor the third-party status page, and skip-with-warning rather than fail when the dependency is down ("Stripe sandbox is degraded; payment tests skipped"). Don't let their flake be your flake.
  5. Push back on the dependency. If a third party is flaky enough to disrupt your tests, that's also customer-impacting. Open tickets, escalate, and consider redundancy if the business depends on it.

The senior signal: you don't try to "make the test resilient to a flaky third party" by adding retries — you remove the third party from the path that needs to be reliable.

// WHAT INTERVIEWERS LOOK FOR

Layered approach (mock → contract → live sandbox), awareness that mocks alone don't catch API drift, and recognition that retrying-the-test isn't the right fix.

// COMMON PITFALL

Saying 'I add retries' — that masks the flake and slows the suite.