How do you balance fast feedback (PR-time tests) with comprehensive coverage (nightly tests)?

Two competing goods: speed (devs need feedback fast) and coverage (don't ship bugs). Treat as an ongoing optimisation, not a one-time decision.

Define a target. PR pipeline p95 of 10 minutes. Nightly pipeline can run 4 hours. Once you have targets, every new test is a budget question, not an absolute add.

Decide what's PR-worthy:

• Critical path: login, checkout, payment, signup. Always PR-tier.
• Common bug source: areas that have shipped recent bugs deserve more PR coverage.
• Cheap: unit tests are cheap; they belong on PR almost regardless.
• High-signal-per-second: a 30-second test catching 80% of regressions in its area is gold.

Keep nightly:

• Slow by nature: cross-browser, full E2E, perf, soak.
• Lower-signal: tests that catch bugs once a year.
• Expensive infra: device farms, paid SaaS scans.

The feedback loop:

1. Nightly catches a bug PR missed.
2. Triage: was the bug in critical-path code? Could a smaller test catch it?
3. Add a focused PR-tier test that catches this class.
4. Don't promote the whole nightly suite — that breaks the time budget.

The reverse direction also matters. If a PR test catches a bug 0 times in 6 months, demote or delete. The suite is finite-budget; passive tests are dead weight.

A tactic that works: tag every test with @critical, @regression, @nightly. PR runs @critical; staging deploy runs @critical || @regression; nightly runs all. Promotion/demotion is just a tag change.

Cross-cutting tests (security scans, dependency audits) can run async — PR doesn't block on them, but their failure does block the merge button. GitHub's "required checks" allows this hybrid.

Senior signal: framing this as an ongoing balance with measurable budgets and feedback loops, not a static decree.