Flaky Test

// Definition

A test that passes and fails intermittently without any code changes, often caused by timing issues, shared state, async race conditions, or external dependencies. The single largest source of CI noise.

// Why it matters

A flaky test passes and fails without code changes, and it's corrosive: teams start ignoring red, which defeats the suite. The single largest source of CI noise. QA value is in diagnosing the flake class (timing, order, shared state, network) rather than blanket-retrying it.

// How to test

// Quarantine + repeat to confirm flakiness before fixing
// (run the single test many times; a stable test is green every time)
Cypress._.times(20, (i) => {
  it(`order list renders — run ${i}`, () => {
    cy.intercept('GET', '/api/tools').as('getTools')
    cy.visit('/tools')
    cy.wait('@getTools')               // wait on the dependency, not a sleep
    cy.get('[data-cy=tool-card]').should('have.length.greaterThan', 0)
  })
})

// Code Example

// ❌ Flaky — clicks before the list renders
cy.visit('/tools');
cy.get('.tool-card').first().click();

// ✅ Stable — waits for the API the page depends on
cy.intercept('GET', '/api/tools').as('getTools');
cy.visit('/tools');
cy.wait('@getTools');
cy.get('.tool-card').first().click();

// Related terms

• Retry Logic

  Re-running a failing test or step automatically. Useful for taming flake from infrastructure issues — but can mask real bugs if used as a band-aid for genuinely flaky tests.

• Auto-waiting

  A framework feature that pauses an action until the target element is actionable (visible, enabled, stable). Eliminates most need for explicit sleeps and reduces flake.

• Parallel Testing

  Running multiple tests concurrently to reduce wall-clock time. Requires test independence — shared state will cause non-deterministic failures.