Test Independence and Isolation

Run your 200-test suite in alphabetical order. Now run it in reverse. Now run only tests 50–100. If results differ between these runs, your tests have hidden dependencies — tests that only work correctly because another test ran before them and left specific state. These dependencies are invisible until the environment changes: a CI reset, a new team member running a subset, a test order randomised by a new plugin. When they surface, they produce failures that are extremely difficult to diagnose because the failing test appears correct in isolation. Test independence — the principle that every test can run correctly alone, in any order, in parallel — is not a nice-to-have quality. It is the foundational property that makes a test suite trustworthy.

What independence means

An independent test:

• Creates all the data it needs before running, and doesn't assume data was left by a previous test
• Cleans up the state it creates, so subsequent tests start from the same baseline
• Makes no assumptions about which browser page, URL, or application state it starts from
• Can run simultaneously with any other test without interference

A dependent test:

• Assumes a user created by Test A exists before Test B runs
• Assumes the browser is already on the logged-in dashboard because the previous test logged in
• Assumes a config value set by a static field in Test A is still set when Test B reads it

Detecting dependencies

The fastest way to find hidden dependencies is to randomise test order. Most frameworks support this:

# TestNG — randomise test order
<suite name="Regression" preserve-order="false">
 
# pytest — random order plugin
pip install pytest-random-order
pytest --random-order
 
# JUnit 5 — random method order
@TestMethodOrder(MethodOrderer.Random.class)
public class CheckoutTest { ... }

Run the suite in random order five times. Any test that fails inconsistently across runs has a dependency on test order. Find which test it depends on by running the failing test in isolation — if it passes alone but fails in the suite, it's consuming state it didn't create.

Achieving isolation: four patterns

Pattern 1: Each test creates and owns its data

public class OrderHistoryTest extends BaseTest {
    private User testUser;
    private List<Integer> createdOrderIds = new ArrayList<>();
 
    @BeforeMethod
    public void createTestData() {
        testUser = UserFactory.randomUser();
        userApi.create(testUser);
        // Create 3 orders for this user — unique to this test run
        for (int i = 0; i < 3; i++) {
            int orderId = orderApi.create(testUser.getId(), ProductFactory.random());
            createdOrderIds.add(orderId);
        }
    }
 
    @AfterMethod(alwaysRun = true)
    public void cleanupTestData() {
        // Clean up in reverse order — orders before user (foreign key constraint)
        createdOrderIds.forEach(id -> orderApi.delete(id));
        if (testUser != null) userApi.delete(testUser.getId());
    }
 
    @Test
    public void userSeesAllThreeOrders() {
        loginPage.loginAs(testUser);
        int displayed = orderHistoryPage.getOrderCount();
        assertEquals(3, displayed);
    }
}

The test owns its complete lifecycle. No other test can interfere because the user and orders were created fresh, with a unique email, for this specific test run.

Pattern 2: Fresh resources per test

// TypeScript/Playwright — fresh context per test via fixtures
test.beforeEach(async ({ browser }) => {
  // New browser context = new cookies, storage, auth state
  context = await browser.newContext();
  page = await context.newPage();
  loginPage = new LoginPage(page);
});
 
test.afterEach(async () => {
  await context.close();  // clears all cookies, cache, storage
});

# Python/pytest — fresh page per test via fixture scope
@pytest.fixture
def fresh_page(browser):
    context = browser.new_context()
    page = context.new_page()
    yield page
    context.close()  # teardown runs even on test failure

A new browser context per test means cookies, session storage, and local storage are completely clean. No logged-in state bleeds from one test to the next.

Pattern 3: Avoid shared mutable state

// Wrong — static counter shared across all test instances
public class CheckoutTest {
    private static int orderCount = 0;  // ❌ shared across threads
 
    @Test
    public void testSingleItemCheckout() {
        orderCount++;  // race condition in parallel execution
    }
}
 
// Right — instance variable, isolated per test
public class CheckoutTest {
    private int orderId;  // ✅ per-test instance
 
    @BeforeMethod
    public void createOrder() {
        orderId = orderApi.create(UserFactory.randomUser().getId());
    }
}

Static mutable fields are shared across all test instances and all threads. Replace them with instance variables, ThreadLocal, or test-scoped fixtures.

Pattern 4: Database transaction isolation

For test suites with direct database access, transaction-based isolation is the cleanest approach — tests run inside a transaction that's rolled back after each test:

@BeforeMethod
public void beginTransaction() {
    connection = DataSource.getConnection();
    connection.setAutoCommit(false);
}
 
@AfterMethod(alwaysRun = true)
public void rollbackTransaction() {
    connection.rollback();
    connection.close();
}

Every database change made during the test — inserts, updates, deletes — is reversed by the rollback. The database starts each test in an identical state. This only works when the application under test uses the same database connection (or when tests bypass the application and call the database directly).

Test isolation in CI vs local

Failures that appear only in CI often point to isolation problems:

• CI starts with a clean database; local doesn't. Locally, a test passes because a user created six months ago still exists. In CI, the database is freshly seeded and that user isn't there.
• CI runs tests in parallel; local runs sequentially. Isolation violations that are invisible sequentially become race conditions in parallel.
• CI uses a different time zone. Tests that hardcode "today's date" break when CI is in UTC and the developer's machine is in EST.

The solution to all three is the same: each test must create and own every piece of state it depends on, regardless of what environment it runs in.

The anti-patterns that break independence

Shared login state across tests. Ten tests that all need an authenticated user and all share a single login session — when one test logs out, the next nine fail. Each test must establish its own session, or use cookies/storage state saved before the test run (Playwright's storageState feature does this efficiently).

Relying on "seed data" that's assumed to be present. A test that navigates to /products/42 assumes product 42 exists. On a fresh CI database, it doesn't. Tests must create the product or use a factory that returns an ID that's guaranteed to exist.

@BeforeSuite data that only some tests need. Seeding a complex dataset in @BeforeSuite for all tests creates an implicit shared dependency — any test that modifies that data affects subsequent tests. Seed per-test what's needed per-test.

⚠️ Common mistakes

• @AfterMethod without alwaysRun = true. A test that fails mid-way leaves its created data in the database. The next test in a fresh run creates the same user, hits a UniqueConstraintViolation, and fails for a completely unrelated reason. Every cleanup method must run regardless of test outcome.
• Timestamps as unique identifiers without millisecond precision. email = "user-" + System.currentTimeMillis() + "@test.com" collides when two tests run within the same millisecond on fast hardware. Use UUID.randomUUID() instead — probabilistically collision-free.
• Testing framework isolation by running tests sequentially. A suite that passes sequentially may fail in parallel due to thread-level state sharing (static fields, shared WebDriver). Always validate isolation by running with thread-count > 1.

🎯 Practice task

Enforce test independence in your suite — 40 minutes.

1. Random order test. Enable preserve-order="false" (TestNG) or --random-order (pytest). Run the suite 3 times. Note which tests fail inconsistently across runs — those have dependencies.
2. Fix the first dependency. Take the first failing test from step 1. Run it in isolation. Does it fail? If not, it depends on state from a prior test. Add @BeforeMethod data creation and @AfterMethod(alwaysRun=true) cleanup to make it self-sufficient.
3. Static field audit. Search your test classes for static fields that aren't final. Each one is a potential shared mutable state violation. Convert instance-specific state to instance fields.
4. Parallel smoke test. Run just 5 tests with thread-count="5" (all 5 in parallel). Verify all pass. Then run 20 tests with thread-count="5". Failures that only appear in the parallel run are isolation violations — the tests work sequentially but collide in parallel.
5. Stretch — isolation score. Count how many of your tests have both a @BeforeMethod that creates their own data AND an @AfterMethod(alwaysRun=true) that cleans it up. Express this as a percentage of your suite. Set a goal: 100% of tests own their data within 2 sprints.

Chapter 5 is complete. Chapter 6 is the final chapter: maintenance, scaling, code reviews, and the documentation that makes a framework an asset rather than a liability when the team changes.