QA Process
QA Metrics Dashboard.
A QA dashboard is only as good as the handful of metrics on it. This guide covers the metrics worth tracking across execution, defects, automation, release quality and team health — and, just as important, how to read them without being fooled.
When to use it
Use this when you're building or pruning a QA dashboard, or when leadership keeps asking "how's quality?" and you want a stable set of numbers that actually answers it rather than a wall of vanity charts.
Who uses it
QA leads and managers who own the dashboard, senior QA who feed it, and the engineering and product leaders who read it to gauge release health and where to invest.
On this page11 sections
WHAT ARE QA METRICS
QA metrics are the measurements that describe the state and trend of quality: how much has been tested, how many defects exist and where, how trustworthy your automation is, and how much escapes to production. Together they turn "I think we're okay" into evidence.
A dashboard is where the few that matter live side by side, so a glance tells you whether quality is improving, holding, or slipping. The skill isn't collecting metrics — anything can be counted — it's choosing the small set that drives decisions and ignoring the rest.
GOOD METRICS VS VANITY METRICS
A good metric changes a decision. If the number moves and nobody does anything differently, it's a vanity metric — it decorates the dashboard and dilutes the ones that matter. "Total test cases written" is the classic example: it always goes up, and it tells you nothing about quality.
Prefer metrics that are actionable (they point to a next step), hard to game (you can't improve the number without improving the thing), and trend-aware (a single value means little; the direction over releases means a lot). When in doubt, ask what you'd do if the number doubled — if the answer is "nothing", drop it.
METRICS FOR TEST EXECUTION
Execution metrics tell you where a cycle stands: how much is done, how much is passing, and what's stuck. They answer "are we on track to finish testing in time?"
Metrics to track
Test pass rate — share of executed tests that passed; a falling rate near release is a warning sign.
Test execution progress — share of planned tests run so far; shows whether you'll finish on time.
Blocked test cases — tests that can't run (environment, data, dependency); a rising count hides real coverage gaps.
METRICS FOR DEFECTS
Defect metrics describe the shape of what you're finding and how well it's being resolved. Read them by severity and by trend, never as raw totals.
Metrics to track
Defect count by severity — open defects split into Critical / High / Medium / Low, so a count is never read flat.
Reopen rate — share of fixed defects that fail re-test and reopen; high reopen rate means fixes or requirements are unclear.
Mean time to detect — how long a defect lives before it's found; shorter means testing is closer to the change.
Mean time to resolve — how long from raised to fixed-and-verified; tracks how fast the team clears risk.
METRICS FOR AUTOMATION
Automation metrics measure whether your suite is an asset or a liability. A big suite that's slow and flaky is worse than a small one people trust.
Metrics to track
Automation coverage — share of important flows covered by reliable automated tests; value matters more than raw count.
Flaky test rate — share of tests that pass and fail without code changes; above a small threshold, people stop trusting red.
Regression duration — how long a full regression run takes; if it outgrows the release window, it stops being run.
METRICS FOR RELEASE QUALITY
Release-quality metrics are the ones leadership feels most directly — they measure what got through, not just what you caught. They're the truest test of whether the rest of your process is working.
Metrics to track
Escaped defects — defects found in production after release; the clearest signal of what testing missed.
Defect leakage — share of total defects that escaped to production versus those caught before release; low and stable is the goal.
METRICS FOR TEAM HEALTH
Some of what limits quality isn't the product — it's the conditions the team works in. A couple of operational metrics keep those visible so they get fixed instead of absorbed.
Metrics to track
Environment downtime — time test environments were unavailable or unstable; lost hours here quietly become lost coverage and slipped dates.
EXAMPLE QA DASHBOARD
| Metric | Healthy looks like | Warning looks like |
|---|---|---|
| Defect leakage | Low and stable | Increasing after each release |
| Flaky test rate | Below agreed threshold | Test failures not trusted |
| Pass rate | Stable or improving | Drops near release |
| Reopen rate | Low | Requirements or fixes unclear |
| Automation coverage | Covers high-risk flows | High count but low value |
HOW TO INTERPRET METRICS
Read every metric as a trend and in context, never as a lone number. A 92% pass rate sounds fine until you see it was 99% last release and falling; a rising defect count is good news if you just doubled the testing that found them. The direction and the why are the signal.
Read metrics together, too, because they constrain each other. High automation coverage means little if the flaky rate is high enough that nobody trusts the result. Low open-defect counts look great right up until escaped defects spike — which usually means you weren't catching them, not that they weren't there. Treat the dashboard as one picture, not a row of trophies.
COMMON MISTAKES
Tracking everything you can count.
A dashboard with thirty numbers hides the five that matter. Choose the handful that change decisions and cut the rest.
Reporting raw totals with no trend.
"40 open defects" is meaningless alone. Show the trend and the severity split so the number can be read.
Rewarding test count.
Counting tests written or executed invites gaming and busywork. Measure coverage of what's risky and what escapes, not volume.
Ignoring flaky tests because the suite is "green enough".
Flakiness erodes trust in every result. Track it and keep it under a set threshold, or the whole suite loses its meaning.
Treating the dashboard as a scoreboard.
Metrics exist to prompt action, not to grade people. The moment they're used to blame, they start being gamed.