SaaS QA
Multi-tenant flows, subscription billing, role permissions, and feature-flag isolation.
// OVERVIEW
SaaS products run a single shared codebase for hundreds of simultaneous tenants. The most catastrophic failures are invisible to the end user — one tenant's data silently appearing in another's view, a downgrade that doesn't fully revoke access, or a flag rollout that serves different code paths to the same user on refresh.
// What makes SaaS QA different
- Cross-tenant data contamination: any missing tenant_id filter is a data breach, not just a bug
- Subscription billing is a churn trigger: a double-charge or missed downgrade directly costs retention
- Feature flags mean multiple code paths coexist in production simultaneously — each path needs its own test coverage
- RBAC must hold at the API layer, not just in the UI — UI gates are bypassable by any API client
- API rate limits enforce fair usage; a bypass lets one tenant degrade service quality for all others
// Core user journeys
| Journey | What to cover |
|---|---|
| Sign-up & onboarding | Account creation, email verification, workspace setup, first-user admin grant |
| Seat management & role assignment | Invite flow, role selection, seat count enforcement on the tier limit, role downgrade |
| Feature flag–gated feature | Flag activation per-tenant, user cohort targeting, gradual rollout, rollback to OFF |
| Plan upgrade / downgrade | Prorate credit calculation, immediate access change on upgrade, seat lock on downgrade |
| Third-party integration (OAuth/webhooks) | OAuth consent flow, webhook delivery and idempotency, token refresh and revocation |
// RISKS & TEST AREAS
// Main risk areas
| Risk | Why it matters |
|---|---|
| Tenant data leakage | A missing tenant_id clause in any DB query exposes one customer's data to another — a direct compliance breach that can trigger contract termination |
| Billing edge-case errors | Prorate miscalculation, double-charge on upgrade, or failed downgrade directly drives churn and triggers support escalations |
| RBAC gaps | A Member-role user calling an Admin-only API endpoint with a 200 response silently bypasses your entire permission model |
| Feature flag isolation failures | A flag that activates for the wrong tenant or user cohort ships unfinished code to production without warning |
| API rate-limit bypass | An uncapped tenant can exhaust shared infrastructure resources and degrade service quality for all other tenants on the platform |
// Functional areas to test
- Multi-tenant data isolation — query-level, not just UI-level filtering
- Subscription lifecycle: trial → paid → upgrade → downgrade → cancel → re-subscribe
- User and seat management: invites, seat limits, seat revocation on downgrade
- Role-based access control across Admin / Member / Viewer / Owner tiers
- Feature gating: flag activation, deactivation, per-tenant and per-user cohort targeting
- Onboarding and invite flows: email delivery, token expiry, duplicate invite handling
// API & integration areas
- Tenant-scoped auth headers required and validated on every endpoint — missing scope = 401
- Rate limit: 429 status code + correct Retry-After header returned at the configured threshold
- Webhook delivery and idempotency: duplicate events must not create duplicate state changes
- API version compatibility: older tenant integrations must not break on schema changes
- Third-party OAuth: token refresh, revocation, and re-auth prompt on expiry
// Data testing
- Seed at least two isolated tenant accounts — never share test fixtures across tenants
- Use the payment processor's sandbox mode; never run billing tests against a live gateway
- Seed per-user flag state explicitly rather than relying on default rollout percentages
- Assert audit log entries are present and complete for all billing and permission events
// CROSS-CUTTING CONCERNS
// Security & privacy
- DB query tenant isolation: confirm every query carries a tenant_id predicate at the SQL level, not as an app-layer filter
- RBAC enforcement on every HTTP route: test denied roles receive 403, not a 200 with empty data (silent permission failure)
- PII scrubbing: billing details, email addresses, and seat assignments must not appear in application logs or error payloads
- OIDC/SSO edge cases: expired SAML assertion, MFA re-prompt on new device, SSO bypass via direct credential login
// Accessibility
- WCAG AA contrast and keyboard navigation on dashboard data tables and plan comparison grids
- Screen reader compatibility on billing modals, seat-count inputs, and upgrade confirmation dialogs
- Focus management after modal open/close in seat management and role assignment flows
// Performance
- Multi-tenant load: 10+ concurrent tenants writing simultaneously — assert no cross-tenant latency bleed
- Webhook ingestion throughput: sustained event volume at peak (e.g. end-of-billing-period batch)
- Billing event processing latency: assert queue drain time stays within SLA under backlog conditions
// Mobile & responsive
- Responsive plan-upgrade and billing pages render correctly at 375 px viewport
- Mobile-first onboarding flow: invite acceptance, initial workspace setup, role confirmation
// BUGS & SCENARIOS
// Common bugs
| Bug | Scenario / repro |
|---|---|
| Tenant data bleed | Tenant A requests a resource list — records belonging to tenant B appear because the query omits a tenant_id WHERE clause |
| Phantom seat after downgrade | Seat revocation removes UI access but the user's SSO session remains active, allowing continued login until session expiry |
| Silent 200 on forbidden call | A Member-role user calls an Admin-only endpoint; the server returns HTTP 200 with an empty array instead of 403 — the client treats it as a successful empty response |
| Double-charge on plan upgrade | A race condition between the invoice.payment_succeeded webhook and the plan-change event triggers two billing cycles for the same period |
| Stale flag value after rollout | Flag cache is not invalidated when a rollout completes; users receive the pre-rollout flag value until the cache TTL expires naturally |
// Example test scenarios
- 01Authenticate as tenant A; request a resource using a known tenant B resource ID — assert 403 or 404, never a 200
- 02Downgrade plan while seats exceed the new tier limit — assert excess seats are locked within one billing cycle, not immediately upon downgrade
- 03Toggle a feature flag OFF mid-session — assert the feature disappears on next page load without requiring a full sign-out
- 04Issue one request above the API rate limit — assert 429 with a valid Retry-After header and no 500 error
- 05Complete onboarding as Admin, invite a Member, then call an Admin-only API as the Member — assert 403, not 200 or 404
// Edge cases
- Mid-cycle downgrade: prorate credit calculation when downgrading on day 15 of a 30-day billing cycle
- Cancellation with a pending payment in retry: assert no charge fires after cancellation is confirmed
- Flag rollout at exactly 50% user cohort: assert consistent assignment — the same user always gets the same value
- Concurrent role downgrade and in-flight API request: the request was authorised at send time but the role changed before server processing
- Free-trial expiry during an active authenticated session: assert the session is blocked gracefully, not with a 500
// AUTOMATION & TOOLS
// What to automate
- Generate the full RBAC matrix as data-driven parametrised tests: every role × every endpoint, asserting 200 or 403 as expected
- Automate the billing lifecycle E2E in payment sandbox: trial → paid → upgrade → downgrade → cancel
- Multi-tenant isolation contract tests: for each resource type, assert cross-tenant access returns 403 or 404
- Flag ON/OFF smoke suite: run on every deployment to catch regressions in all flag-gated features simultaneously
- Rate-limit boundary test: assert exactly 200 at limit − 1 requests and exactly 429 at limit within a single test run
// Useful tools
PlaywrightE2E for subscription flows and multi-step onboardingCypressComponent + E2E coverage for flag-gated UI featuresPostmanRBAC matrix and tenant-isolation API collectionsk6Multi-tenant load and rate-limit threshold testsFeature Flags & Experimentation toolsTesting flag lifecycle, rollout, and rollbackAPI testing checklistEnd-to-end coverage checklist for every SaaS endpoint
// SHIP & LEARN
// Release readiness checklist
- Tenant isolation regression suite is green — no cross-tenant data visible in any response
- Billing lifecycle smoke test passed in sandbox: trial → paid → cancel
- RBAC matrix complete: all roles × all sensitive endpoints asserted
- Feature flag OFF path tested for every flag-gated feature in this release
- Rate-limit 429 + Retry-After verified at the configured threshold
- Audit log entries present and complete for all billing and permission events in staging
- Staging environment uses isolated tenant accounts — no cross-contamination between test accounts
- Rollback plan confirmed with billing and infrastructure teams