Q4 of 21 · AI for testing

What are the main risks of using AI-generated tests without reviewing them?

AI for testingJuniorai-for-testingtest-reviewtest-qualityai-limitsbest-practices

Short answer

Short answer: AI produces plausible-looking tests that may assert the wrong thing, miss real coverage, or bake in bad patterns — hard waits, brittle selectors, vacuous assertions. The test suite appears healthy but provides false confidence.

Detail

The most dangerous output is a test that runs green but proves nothing useful. An AI asked to "write a test for the login endpoint" might assert only that the response status is 200, without checking whether the returned token is valid or whether the session is actually established.

Other risks: AI doesn't know which tests are worth writing — it optimises for visible coverage, not risk. It may generate 12 tests for a trivial form while missing the integration edge case that actually fails in production. It also inherits patterns from your existing codebase — if your examples use cy.wait(2000), the generated tests will too.

The correct workflow is AI-drafts, engineer-owns. Every generated test must be read, understood, and verified against the real behaviour before it enters the suite. Unreviewed tests are worse than no tests — they erode trust in the suite without adding real coverage.

// WHAT INTERVIEWERS LOOK FOR

Vacuous assertions as the key risk. AI optimising for visible coverage not real risk. Pattern inheritance from bad examples. The principle: engineer must own and understand every test committed.