Q10 of 38 · Test design
When would you choose pairwise over full combinatorial testing? Show a worked example.
Short answer
Short answer: Use pairwise when full combinatorial is impractically large and the risk profile suggests bugs are 2-way (most are). Stick with full combinatorial when the input space is small (≤ ~16 combos) or when 3-way interactions are known to matter (compliance, security).
Detail
The choice is a cost/coverage trade-off. Full combinatorial gives complete coverage of all interactions; pairwise gives complete 2-way coverage at a fraction of the cases.
Decision factors:
- Size of full combinatorial. ≤ 16 cases? Run it — pairwise saves nothing meaningful and you get 3-way+ coverage for free. ≥ 100 and growing? Pairwise is the only practical option.
- Where do you expect bugs? NIST research suggests ~70% of defects involve 1 or 2 factors; ~95% involve ≤ 4. If your domain has known 3-way interactions (security flags + role + permission, or specific regulatory combinations), augment pairwise with explicit cases for those.
- Cost per case. If each case is fast (1s automated), running 100 isn't punitive. If each is a 30-minute manual scenario, pairwise's reduction is critical.
Worked example. A search filter with: Sort by (5 values), Result count (3), View (2), Logged in (2). Full: 5 × 3 × 2 × 2 = 60. Pairwise: ~13 cases that cover every pair (each sort with each result count, each sort with each view, each sort with each logged-in state).
When you'd run all 60: if these inputs feed a high-stakes search ranking that's contractually guaranteed (legal, financial), the cost of a 3-way bug is too high.
When pairwise is enough: a typical product UX where 60 cases is too slow for CI and there's no business-critical 3-way interaction known.
The combined approach — pairwise as the baseline, plus targeted full combinations for the inputs you know matter together. This gives reliable pair coverage with explicit attention to high-risk areas.
// EXAMPLE
pairwise-search-filters.md
Inputs and values:
sort: relevance | price-asc | price-desc | newest | popular (5)
count: 10 | 25 | 50 (3)
view: grid | list (2)
loggedIn: true | false (2)
Full combinatorial: 5 × 3 × 2 × 2 = 60
PICT-generated pairwise: 13 cases below — every pair covered
# sort count view loggedIn
1 relevance 10 grid true
2 relevance 25 list false
3 price-asc 10 list false
4 price-asc 50 grid true
5 price-desc 25 grid false
6 price-desc 50 list true
7 newest 10 list true
8 newest 25 grid true
9 newest 50 grid false
10 popular 10 grid false
11 popular 25 list true
12 popular 50 list false
13 relevance 50 grid false
Augment with explicit cases for any 3-way interaction known to matter.