Q5 of 21 · AI for testing
How do you use AI to help write automated tests, and where does it fall short?
Short answer
Short answer: AI is useful for scaffolding test structure from acceptance criteria, generating test data, explaining failure stack traces, and drafting migrations. It falls short when it produces plausible-but-wrong assertions, doesn't know which tests are worth writing, and bakes in existing bad patterns.
Detail
Concrete uses in daily workflow: Scaffolding: paste acceptance criteria and ask for a Playwright or Cypress test skeleton. Edit the selectors and assertions to match reality. Stack trace explanation: paste a test failure output and ask what might cause it. Speeds up initial diagnosis. Assertion generation: given a known API response shape, ask AI to draft assertions covering required fields, data types, and boundary values. Boilerplate migration: ask AI to convert Selenium Java to Playwright TypeScript, then review every generated locator.
Limits that matter: AI doesn't know your app's actual behaviour, so it invents plausible assertions. It doesn't know which feature paths carry the most risk. It picks up bad patterns from your codebase or its training data. The skill now isn't getting AI to write a test — it's spotting when the test it wrote is wrong. See AI-generated automation scripts and AI PR review and test gen.