Q11 of 38 · CI/CD & DevOps
What is a quality gate and what should it block on?
Short answer
Short answer: A quality gate is a CI step that fails the build (or blocks merge/deploy) on objective criteria. Block on: failing tests, test pass rate below threshold, coverage drop on new code, critical-severity security findings, breaking schema changes. Don't block on subjective things.
Detail
A gate has three properties:
- Objective — pass/fail driven by data, not opinion.
- Owned — someone is on the hook when it fails.
- Actionable — the failure tells the dev exactly what to do.
Things to block on:
- Failing tests — the obvious one. No retries to make green.
- Coverage drop on changed lines — not absolute coverage. "Don't decrease the coverage of code you touched." Tools: SonarQube, Codecov diff coverage.
- Critical security findings — Snyk/Dependabot/SAST flagging high-severity CVEs. Mid/low: warn, don't block.
- Breaking API changes — oasdiff or a contract test suite catching schema regressions.
- Linter errors (not warnings) — broken syntax, banned imports, unused variables. Style nits should be auto-formatted, not gated.
- Performance regression beyond threshold — k6 thresholds breaching.
- Migration safety checks — squawk, gh-ost dry-run for risky DDL.
Things NOT to block on:
- Subjective code style — auto-format with Prettier/Black, don't argue.
- Coverage absolute % (e.g. "must be above 80%") — incentivises low-quality tests written for line coverage.
- Flaky tests — quarantine and triage; don't punish PRs for unrelated flakiness.
- Warnings from new tooling rules — bulk-fix and then enable as a gate.
Granularity matters: PR gate (every change), pre-merge gate (final check), pre-deploy gate (before staging or prod). Each tier checks different things.
Override discipline: humans must be able to override in genuine emergencies (production fire). But every override is logged, reviewed weekly, and a Pareto distribution emerges — the same gate gets overridden repeatedly, signalling either it's wrong or the team needs help fixing the underlying issue.
The real test of a gate: does it block bad code without blocking good code? If devs work around it routinely, it's the wrong gate.
// WHAT INTERVIEWERS LOOK FOR
// COMMON PITFALL
// Related questions
How would you decide which test suites run on every commit versus nightly?
CI/CD & DevOps
How do you handle test data isolation when tests run in parallel?
CI/CD & DevOps
What's the difference between blue-green and canary deployments from a testing perspective?
CI/CD & DevOps
As a QA lead, how would you configure branch protection rules for `main` and `release/*` branches to enforce quality gates without slowing down the team?
Git