Q14 of 17 · Framework design
How do you keep a test automation framework maintainable over time as the application evolves?
Short answer
Short answer: Treat the framework like production code: code reviews for test PRs, lint rules, dependency updates on a schedule, test-to-code ratio tracking, and a clear policy for updating Page Objects when the UI changes.
Detail
Practices that prevent framework decay:
1. Test code reviews — same standard as production: Test PRs reviewed for: locator strategy (prefer data-testid over CSS), assertions specificity, no hardcoded waits (sleep), no business logic in step definitions, meaningful test names.
2. Dependency updates on a schedule:
# Check outdated packages weekly (automated PR via Dependabot or Renovate)
npx npm-check-updates -u
npm test # verify nothing breaks after update
Playwright/Selenium major version upgrades often require Page Object changes. Defer them → your locators break on a day you can least afford it.
3. Flaky test policy:
Track flakiness rate per test. Tests that flake more than 2% are tagged @quarantine and moved to a separate CI job. The quarantine job runs but doesn't gate the build. Flaky tests are fixed within one sprint or deleted.
4. Page Object coverage tracking: When a dev ships a new UI page or component without a corresponding Page Object update, tests start using raw selectors. Track this via PR checklists: "Does this feature change require a PO update?"
5. Naming conventions documented and enforced:
- Page Objects:
<PageName>Page.ts/<PageName>Page.java - Test files:
<feature>.spec.ts - IDs:
login-btn, notloginBtn(kebab-case for data-testid)
6. Regular framework audits (quarterly):
- Delete tests that cover deleted features
- Remove duplicate page objects
- Update broken locators from the last UI overhaul
- Review the flaky test quarantine queue
7. Changelog:
A CHANGELOG.md for the shared framework library so teams know what changed between versions and whether they need to update their test code.