Q7 of 21 · AI for testing
How do you use AI to help triage a failing test or analyse a stack trace?
Short answer
Short answer: Paste the test output, stack trace, and relevant test code into a language model and ask it to explain the failure and suggest likely causes. Use it as a first-pass filter — it shortens the time to a hypothesis, but you still have to verify each hypothesis against the actual running system.
Detail
AI is genuinely useful here because stack trace analysis is pattern-matching at its core — and LLMs are good at pattern-matching. "Element with selector #submit not found after 10 seconds" combined with the test code usually produces a reasonable diagnosis: the selector changed, the test is hitting the wrong environment, or a preceding step is failing silently.
Workflow: paste the full test output, the failing test file, and any relevant page or component code. Ask the model to explain what it thinks happened and list 3–5 candidate causes in order of likelihood. Then verify each hypothesis against the actual environment.
Limits: AI cannot query your live DOM, check network traffic, or access your CI environment logs. It reasons from the text you provide — if you omit context (a recent code change, the environment URL), it produces plausible but wrong hypotheses. It is a fast brainstorm partner, not a debugger. See AI failure triage for triage patterns.