Test Suites, Groups, and Dependencies

A testng.xml file is what turns a folder of @Test classes into a suite — a controlled, repeatable selection of tests with the parallelism, ordering, and parameters you choose. This lesson takes the minimal testng.xml from chapter 1 and grows it into something a real project ships: multiple <test> blocks, group-based selection so the same source files run as smoke / regression / nightly suites, and dependencies that skip downstream tests when their prerequisites fail. By the end you'll know how to slice the same 200 test methods five different ways without copy-pasting a single line of Java.

The four-layer suite structure

Every testng.xml has the same shape:

<suite name="...">           <!-- one suite per file -->
    <test name="...">         <!-- one or more <test> blocks per suite -->
        <classes>             <!-- the test classes to include -->
            <class name="..."/>
        </classes>
    </test>
</suite>

Suite → Test → Class → Method — the same scopes the lifecycle annotations use. Each layer can carry its own configuration (parallelism, parameters, listeners). The scoping is what makes the file flexible.

A small but realistic example with two <test> blocks running in parallel:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Regression Suite" parallel="methods" thread-count="4">
 
    <test name="Login Tests">
        <classes>
            <class name="com.mycompany.tests.tests.LoginTest"/>
            <class name="com.mycompany.tests.tests.RegistrationTest"/>
        </classes>
    </test>
 
    <test name="Product Tests">
        <classes>
            <class name="com.mycompany.tests.tests.ProductSearchTest"/>
            <class name="com.mycompany.tests.tests.ProductDetailTest"/>
        </classes>
    </test>
 
</suite>

The <suite> attributes here matter:

• parallel="methods" — TestNG runs different @Test methods on different threads. Other valid values: classes (each class on its own thread), tests (each <test> block on its own thread), none (sequential, default).
• thread-count="4" — pool size. TestNG won't exceed four concurrent threads regardless of how many tests it has.

We'll go deeper on parallelism in chapter 8 (CI/CD). For now: know that testng.xml is where you turn it on.

Groups — slicing by tag

Groups let you tag @Test methods and run any subset in any order without changing source files:

@Test(groups = {"smoke"})
public void shouldLogIn() { ... }
 
@Test(groups = {"regression"})
public void shouldHandleEdgeCaseInLogin() { ... }
 
@Test(groups = {"smoke", "regression"})
public void shouldLoadHomePage() { ... }     // both groups

Then in your suite file, include or exclude by group:

<test name="Smoke">
    <groups>
        <run>
            <include name="smoke"/>
        </run>
    </groups>
    <packages>
        <package name="com.mycompany.tests.tests"/>
    </packages>
</test>

The same source files now run as a smoke suite (~10 fast tests) on every commit, and a full regression suite (everything) nightly. No code is duplicated.

Common group conventions on real teams:

• smoke — runs in 2–3 minutes; gates every commit.
• regression — full coverage; runs on PR merge.
• nightly — slow tests, cross-browser, edge cases.
• wip — flaky or in-development tests — excluded from CI.

You can combine includes and excludes:

<groups>
    <run>
        <include name="regression"/>
        <exclude name="wip"/>
    </run>
</groups>

That reads as "everything tagged regression, except anything tagged wip" — a useful safety net during refactors.

Dependencies — order with a fail-skip rule

Sometimes a test only makes sense if a prerequisite passed. TestNG supports two flavours:

// Method-level — depends on a specific method
@Test
public void loginTest() { /* log in */ }
 
@Test(dependsOnMethods = {"loginTest"})
public void dashboardTest() { /* runs only if loginTest passed */ }

// Group-level — depends on a whole group passing
@Test(groups = {"setup"})
public void seedTestData() { ... }
 
@Test(dependsOnGroups = {"setup"})
public void runDataDependentTests() { ... }

The crucial behaviour: if the prerequisite fails, the dependent is skipped — not failed. Skipped tests show up in the report distinctly, telling you "this test never had a chance to verify anything because its prerequisite was broken." That's a more honest failure mode than letting dashboardTest fail with ElementNotFoundException("login button") because the previous login was broken.

Use dependencies sparingly. Independent tests that each set up their own state are almost always better. Dependencies create order constraints that make parallel execution harder, and a chain of dependent tests tends to cascade-skip when the first one breaks. Reserve them for cases where setup is genuinely too expensive to repeat (a long-running data import, a payment-processor stub).

Suite structure as a concept map

Three real-world suite files

A typical project ships several testng.xml variants in src/test/resources/. Three patterns you'll write often:

smoke.xml — fastest possible loop; gates every commit:

<suite name="Smoke" parallel="methods" thread-count="4">
    <test name="Smoke">
        <groups><run><include name="smoke"/></run></groups>
        <packages><package name="com.mycompany.tests.tests"/></packages>
    </test>
</suite>

regression.xml — full coverage, excluding work-in-progress:

<suite name="Regression" parallel="methods" thread-count="4">
    <test name="Regression">
        <groups>
            <run>
                <include name="regression"/>
                <exclude name="wip"/>
            </run>
        </groups>
        <packages><package name="com.mycompany.tests.tests"/></packages>
    </test>
</suite>

cross-browser.xml — same tests, multiple browsers via parameters:

<suite name="Cross Browser" parallel="tests" thread-count="3">
    <test name="Chrome">
        <parameter name="browser" value="chrome"/>
        <packages><package name="com.mycompany.tests.tests"/></packages>
    </test>
    <test name="Firefox">
        <parameter name="browser" value="firefox"/>
        <packages><package name="com.mycompany.tests.tests"/></packages>
    </test>
    <test name="Edge">
        <parameter name="browser" value="edge"/>
        <packages><package name="com.mycompany.tests.tests"/></packages>
    </test>
</suite>

@Parameters in your BaseTest reads the browser value and creates the appropriate driver. Same Java, three browsers, no source code duplication.

Running suites from Maven

Point Surefire at the right XML in pom.xml:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <configuration>
        <suiteXmlFiles>
            <suiteXmlFile>src/test/resources/${suiteFile}</suiteXmlFile>
        </suiteXmlFiles>
    </configuration>
</plugin>

Then from the command line:

mvn clean test -DsuiteFile=smoke.xml          # smoke only
mvn clean test -DsuiteFile=regression.xml     # full regression
mvn clean test -DsuiteFile=cross-browser.xml  # all three browsers

CI configs (Jenkins, GitHub Actions) can pass different -DsuiteFile= values to different jobs — chapter 8 wires this all the way to a working pipeline.

A test demonstrating groups + dependencies

package com.mycompany.tests.tests;
 
import io.github.bonigarcia.wdm.WebDriverManager;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.annotations.*;
 
public class GroupsAndDependenciesTest {
 
    WebDriver driver;
 
    @BeforeMethod
    public void setup() {
        WebDriverManager.chromedriver().setup();
        driver = new ChromeDriver();
        driver.get("https://www.saucedemo.com");
    }
 
    @Test(groups = {"smoke", "regression"})
    public void shouldDisplayLoginForm() {
        Assert.assertTrue(driver.findElement(By.id("user-name")).isDisplayed());
    }
 
    @Test(groups = {"smoke", "regression"}, dependsOnMethods = {"shouldDisplayLoginForm"})
    public void shouldLoginSuccessfully() {
        driver.findElement(By.id("user-name")).sendKeys("standard_user");
        driver.findElement(By.id("password")).sendKeys("secret_sauce");
        driver.findElement(By.id("login-button")).click();
        Assert.assertTrue(driver.getCurrentUrl().contains("/inventory.html"));
    }
 
    @Test(groups = {"regression"}, dependsOnMethods = {"shouldLoginSuccessfully"})
    public void shouldLoadInventoryAfterLogin() {
        // Same login flow, then check inventory
        driver.findElement(By.id("user-name")).sendKeys("standard_user");
        driver.findElement(By.id("password")).sendKeys("secret_sauce");
        driver.findElement(By.id("login-button")).click();
        Assert.assertEquals(
            driver.findElements(By.cssSelector("[data-test='inventory-item']")).size(),
            6
        );
    }
 
    @AfterMethod
    public void teardown() {
        if (driver != null) driver.quit();
    }
}

The first two tests are tagged both smoke and regression; the third only runs in regression. The dependency chain ensures shouldLoadInventoryAfterLogin is skipped (not failed) if logging in is broken.

The TestNG cheat sheet lists every <suite>, <test>, and group attribute.

⚠️ Common mistakes

• Forgetting that dependsOnMethods skips downstream tests on failure. A team adds a prerequisite, the prerequisite breaks, and 40 tests show up as SKIPPED in the report. The team thinks "skipped is not failed, the suite is fine." It isn't — those 40 tests verified nothing. Always treat a wave of skips as a real failure signal.
• Listing every test class explicitly with <class name="..."/>. It works until someone adds a new test file and forgets to register it. Use <packages><package name="com.mycompany.tests.tests"/></packages> and let TestNG auto-discover. Tests added to that package are picked up on the next run with no XML change.
• Putting parallel="methods" on a suite full of tests that share state. TestNG happily runs in parallel; your tests overwrite each other's state and produce flaky failures that look random. Either move to per-method state (everything in @BeforeMethod), use ThreadLocal<WebDriver>, or stay sequential. Parallel is a powerful default — but only after the suite is properly isolated.

🎯 Practice task

Slice your suite three ways. 30–40 minutes.

1. Add GroupsAndDependenciesTest from this lesson to your project. Tag two of the tests in your existing suite with groups = {"smoke"} and the rest with groups = {"regression"}.
2. Create three suite files under src/test/resources/: smoke.xml, regression.xml, and wip.xml. The first two should run their respective groups; the third should run only tests tagged wip (you'll have none — that's fine, it'll show "Tests run: 0").
3. Update your pom.xml's Surefire config to read ${suiteFile}. Run:

   Copy

   mvn clean test -DsuiteFile=smoke.xml
   mvn clean test -DsuiteFile=regression.xml

   Confirm each runs the correct subset.
4. Force a dependency skip. Make shouldDisplayLoginForm fail intentionally (assert false). Run the suite. Confirm shouldLoginSuccessfully and shouldLoadInventoryAfterLogin are reported as SKIPPED (not FAILED). Restore the assertion.
5. Try dependsOnGroups. Tag a setup-style test with groups = {"setup"} and have a downstream test depend via dependsOnGroups = {"setup"}. Confirm the dependency holds in either order — TestNG enforces it.
6. Stretch — cross-browser suite. Build a cross-browser.xml with three <test> blocks for Chrome, Firefox, Edge, each with a <parameter name="browser" value="..."/>. Update your BaseTest.@BeforeMethod to read @Parameters({"browser"}) and create the right driver. Run the suite — same tests, three browsers, in parallel.

Next lesson: @DataProvider. The way to feed many sets of data into a single test method, turn one test into many, and stop the copy-paste-repeat that makes data-coverage tests painful.