Q17 of 26 · Mobile QA
How do you design a device matrix and test strategy to handle Android fragmentation?
Short answer
Short answer: Base the matrix on your actual user analytics, not on comprehensive coverage. Prioritise the top device/OS combinations by user share — typically 5-10 combinations — and tier your test effort: critical paths on real devices from that matrix, full regression on a single reliable emulator.
Detail
Android fragmentation means the same APK runs on thousands of device/OS/OEM combinations. Attempting exhaustive coverage is impossible and the wrong goal — the right goal is risk-proportionate coverage.
Step 1 — analytics-first selection: pull device and OS data from Firebase Analytics, Google Play Console, or your crash monitoring tool (Crashlytics, Sentry). The top 5 devices typically represent 60-70% of your user base. Build your real-device matrix from those.
Step 2 — risk tiers:
- Tier 1 (real devices, every build): the top 3 device/OS combos by user share. Run critical path tests — sign-in, core user journey, payment.
- Tier 2 (real devices, weekly or pre-release): the next 5-7 combos. Run the full regression suite.
- Tier 3 (emulator, every build): a standard Pixel emulator for fast feedback. Run the full suite.
OEM-specific risks: Samsung One UI changes default font sizes and may override system UI. Xiaomi MIUI restricts background processes aggressively, causing background fetch failures. Huawei devices lack Google Play Services. Each of these deserves at least one device in the matrix if you have affected users.
API level split: test at the minimum supported API level (to catch deprecated-API issues), the current stable level, and the latest beta (to catch upcoming breakage before it hits users).