Exploratory charter
Mission, time-box, areas to explore, oracles, notes. Session-based testing (SBTM).
Exploratory Charter
Session ID: EXP-XXX Tester: Name Date: YYYY-MM-DD Duration: e.g. 90 minutes Feature / Area: Feature name or system area
Mission
*Complete this sentence: "Explore [area] to discover [risks / questions / unknowns]."*
*Explore [feature] to discover [what could go wrong / what is unknown / what needs verification].*
Areas to Explore
List the specific areas, flows, and scenarios within scope for this session.
- Area 1 — describe the specific angle
- Area 2 — describe the specific angle
- Area 3 — describe the specific angle
Ideas and Heuristics
List test ideas and heuristics to guide exploration. Use mnemonics like SFDPOT, CRUD, or FEW HICCUPPS.
- Test idea 1
- Test idea 2
- Test idea 3
Oracles
What will you compare results against to know if something is wrong?
- Specification / design document
- Previous version behaviour
- Related feature behaviour
- User expectation / common sense
Out of Scope
What will you NOT explore in this session (to keep focus)?
Session Notes
Time-boxed notes taken during the session. Record observations, bugs found, questions raised, and coverage achieved.
| Time | Observation / Finding |
|---|---|
| HH:MM | What happened |
| HH:MM | Bug found — link to ticket |
Debrief Summary
Complete after the session. Summarise what was covered, what was found, and what requires follow-up.
Coverage achieved: What was explored.
Bugs filed: List with ticket IDs.
Risks / open questions: Anything unresolved that needs follow-up.
Recommended next session: What would be most valuable to explore next.
Exploratory Charter
Session ID: EXP-2024-047 Tester: Sasha Koval Date: 2024-03-11 Duration: 90 minutes Feature / Area: Checkout error handling — Novu Bank v3.2 Open Banking payment consent flow
Mission
Explore the Open Banking payment consent flow to discover how the system behaves when external bank connections are slow, interrupted, or return unexpected error states.
Areas to Explore
- Happy path with each of the three supported providers (GreenPay, ClearLedger, OpenRoute) — confirming baseline before exploring edges
- Provider timeout: what happens if the redirect back from the provider takes more than 30 seconds
- User abandonment mid-consent: close the browser tab, use the back button, let the session token expire
- Multiple consent attempts: submit the consent flow twice in parallel tabs for the same user
- Partially-authorised state: provider returns success but with a reduced scope
- Error message quality: do error screens give useful guidance or expose internal stack traces
Ideas and Heuristics
- SFDPOT — Time: What happens at the 30-second provider redirect timeout boundary?
- SFDPOT — Operations: What if the user is on a slow 3G connection during consent?
- CRUD — Delete: Can a user revoke consent mid-flow and immediately re-grant it?
- Error guessing: What if the
stateparameter in the OAuth callback is tampered with? - Boundary: Authorise with exactly the minimum account permissions vs. declining one scope
- Comparison oracle: Compare error messages shown to users against the spec in Confluence/NOVU-OPENBANK-ERRORS
Oracles
- Open Banking consent flow spec: Confluence/NOVU-OPENBANK-SPEC-3.2
- Existing behaviour in v3.1 (no Open Banking, so delta is the entire feature)
- Provider sandbox documentation for error codes and expected redirect behaviours
- UX copy doc: Confluence/NOVU-OPENBANK-UX-COPY
Out of Scope
- Performance benchmarking (separate perf test scheduled for 2024-03-10)
- Security testing of the OAuth implementation (handled by security team)
- Notification behaviour after successful connection (covered by EXP-2024-045)
Session Notes
| Time | Observation / Finding |
|---|---|
| 10:05 | GreenPay happy path — connection succeeds, balance visible within 4s. ✓ |
| 10:18 | ClearLedger happy path — works. ✓ |
| 10:26 | OpenRoute happy path — works. ✓ |
| 10:35 | Simulated 35s delay on GreenPay redirect — app shows spinner indefinitely with no timeout message. Bug filed: NOVU-2041 |
| 10:52 | Pressed browser back mid-consent (GreenPay) — returned to Novu with a cryptic state_mismatch error. No user-friendly guidance. Bug filed: NOVU-2042 |
| 11:08 | Two parallel tabs, same consent flow — second tab shows "Connection already in progress" which is reasonable. No data corruption observed. ✓ |
| 11:20 | Partial scope (declined transaction history permission) — app shows "Partial connection" with a clear explanation. ✓ |
| 11:31 | Tampered state parameter — app correctly rejects with "Invalid request." No stack trace exposed. ✓ |
Debrief Summary
Coverage achieved: All three providers on happy path; timeout behaviour; mid-flow abandonment (back button and tab close); parallel sessions; partial scope; tampered OAuth state.
Bugs filed:
- NOVU-2041 — Open Banking: no timeout message when provider redirect exceeds 30s (Severity: High)
- NOVU-2042 — Open Banking: "state_mismatch" error on back-button is not user-friendly (Severity: Medium)
Risks / open questions:
- What happens when the user's bank session expires mid-redirect? No way to simulate this with current sandbox — raised with Dev (Ayo Adeyemi) to clarify.
- Tab-close abandonment leaves an orphan token in the database — confirmed with Dev as expected; token expires after 15 minutes.
Recommended next session: Explore the notifications centre push delivery flow with airline-mode device and background app state. Assign to Marcus Obi (EXP-2024-048).
// Related templates
Test case
Preconditions, steps, expected results, postconditions. Manual or automated, supports Gherkin.
Bug report
Steps to reproduce, expected vs actual behaviour, environment details, and severity. For Jira, Linear, GitHub Issues.
Sprint retrospective
What went well, what didn't, action items. QA-flavoured retro template for sprint reviews.