Data Providers and Parameterised Tests

You wrote shouldLoginSuccessfully. Now you need to verify the same login flow with twenty different (email, password, expected-result) combinations: locked-out users, wrong passwords, empty fields, SQL-injection attempts, very long inputs. Copy-pasting twenty test methods is the wrong answer. @DataProvider is the right one — declare your data once, write the test logic once, and TestNG runs the test once per data row, reporting each as its own pass/fail. This lesson covers the mechanics, the three places your data can come from (inline, Excel/JSON files, dynamic methods), and the per-test method-injection trick that lets one provider feed several tests.

The basic shape

A @DataProvider is a method that returns Object[][] — an array of arrays, where each inner array is one row of arguments:

import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
 
public class LoginDataDrivenTest {
 
    @DataProvider(name = "loginData")
    public Object[][] loginTestData() {
        return new Object[][] {
            {"standard_user",    "secret_sauce",   true,  "/inventory.html"},
            {"locked_out_user",  "secret_sauce",   false, "Sorry, this user has been locked out"},
            {"problem_user",     "secret_sauce",   true,  "/inventory.html"},
            {"performance_glitch_user", "secret_sauce", true, "/inventory.html"},
            {"",                 "secret_sauce",   false, "Username is required"},
            {"standard_user",    "",               false, "Password is required"},
            {"wrong_user",       "wrong_password", false, "Username and password do not match"}
        };
    }
 
    @Test(dataProvider = "loginData")
    public void shouldHandleLoginScenario(
        String username, String password, boolean shouldSucceed, String expectedFragment
    ) {
        loginPage.login(username, password);
        if (shouldSucceed) {
            Assert.assertTrue(driver.getCurrentUrl().contains(expectedFragment));
        } else {
            Assert.assertTrue(loginPage.getErrorText().contains(expectedFragment));
        }
    }
}

Seven rows, one test method, seven separate test executions in TestNG's report. Each one passes or fails independently. Add an eighth scenario? Append a row.

This is the pattern every "verify this works for these N users" test should use. The alternative — seven hand-written test methods — duplicates the body seven times, accumulates seven copies of the same bug when you change something, and makes the report harder to read.

One test method × N rows

One @Test, seven runs, seven results

	username	password	succeed?	result
row 0	standard_user	secret_sauce	true	✅ pass
row 1	locked_out_user	secret_sauce	false	✅ pass
row 2	problem_user	secret_sauce	true	❌ fail
row 3	performance_glitch_user	secret_sauce	true	✅ pass
row 4	(empty)	secret_sauce	false	✅ pass
row 5	standard_user	(empty)	false	✅ pass
row 6	wrong_user	wrong_password	false	✅ pass

Each row in the provider is a separate row in the report — a granular pass/fail makes "row 2 (problem_user) is broken" instantly obvious, instead of "the parameterised login test is broken" which tells you nothing.

Reading data from JSON

Inline arrays are great for half a dozen rows. Once you're past twenty, externalise the data:

import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.annotation.JsonProperty;
 
import java.io.File;
import java.io.IOException;
import java.util.Arrays;
 
public class LoginDataDrivenTest {
 
    public static class LoginScenario {
        @JsonProperty public String username;
        @JsonProperty public String password;
        @JsonProperty public boolean shouldSucceed;
        @JsonProperty public String expectedFragment;
    }
 
    @DataProvider(name = "loginData")
    public Object[][] readJson() throws IOException {
        ObjectMapper mapper = new ObjectMapper();
        LoginScenario[] data = mapper.readValue(
            new File("src/test/resources/testdata/login-scenarios.json"),
            LoginScenario[].class
        );
        return Arrays.stream(data)
            .map(s -> new Object[]{ s.username, s.password, s.shouldSucceed, s.expectedFragment })
            .toArray(Object[][]::new);
    }
}

src/test/resources/testdata/login-scenarios.json:

[
  { "username": "standard_user", "password": "secret_sauce", "shouldSucceed": true, "expectedFragment": "/inventory.html" },
  { "username": "locked_out_user", "password": "secret_sauce", "shouldSucceed": false, "expectedFragment": "locked out" }
]

Test data lives in JSON; test logic lives in Java. Add scenarios by editing the JSON file — no Java change. We'll cover Excel and CSV in chapter 7's data-driven testing lesson.

@Parameters from testng.xml — the simpler cousin

For environment-shaped parameters (browser, base URL, environment) — not data rows — @Parameters is the cleaner tool:

<test name="Staging">
    <parameter name="browser" value="chrome"/>
    <parameter name="baseUrl" value="https://staging.myapp.com"/>
    <classes>
        <class name="com.mycompany.tests.tests.LoginTest"/>
    </classes>
</test>

import org.testng.annotations.Parameters;
import org.testng.annotations.BeforeMethod;
 
@Parameters({"browser", "baseUrl"})
@BeforeMethod
public void setup(String browser, String baseUrl) {
    driver = createDriver(browser);
    driver.get(baseUrl);
}

@DataProvider is for test data rows. @Parameters is for configuration values shared across an entire <test> block. Don't confuse them — they look similar and have very different ergonomics at scale.

Method injection — one provider, many tests

A @DataProvider method can accept a Method argument. TestNG injects the calling test's Method object, so you can return different data depending on which test is asking:

import java.lang.reflect.Method;
 
@DataProvider(name = "perTestData")
public Object[][] dataForCallingTest(Method method) {
    if ("shouldLoginSuccessfully".equals(method.getName())) {
        return new Object[][] {
            {"standard_user", "secret_sauce"},
            {"problem_user",  "secret_sauce"}
        };
    }
    if ("shouldFailLogin".equals(method.getName())) {
        return new Object[][] {
            {"locked_out_user", "secret_sauce"},
            {"wrong_user",      "wrong_password"}
        };
    }
    throw new IllegalArgumentException("No data set for " + method.getName());
}
 
@Test(dataProvider = "perTestData")
public void shouldLoginSuccessfully(String user, String pw) { ... }
 
@Test(dataProvider = "perTestData")
public void shouldFailLogin(String user, String pw) { ... }

Two tests, one provider — the provider routes by method name. Use this when the data shape is the same but the values differ; if the shapes diverge, write two providers.

Parallel data providers

@DataProvider rows run sequentially by default. To parallelise across rows, add parallel = true:

@DataProvider(name = "loginData", parallel = true)
public Object[][] loginTestData() {
    return new Object[][] { ... };
}

Each row now runs on its own thread — provided your tests are thread-safe (each @Test invocation needs its own driver via ThreadLocal, no shared state). Combined with parallel methods at the suite level, you can scale a 200-row data-driven test from a 30-minute serial run to a 5-minute parallel one. Chapter 8 covers the thread-safety side.

A complete, runnable example

package com.mycompany.tests.tests;
 
import io.github.bonigarcia.wdm.WebDriverManager;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.testng.Assert;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
 
public class DataDrivenLoginTest {
 
    WebDriver driver;
 
    @BeforeMethod
    public void setup() {
        WebDriverManager.chromedriver().setup();
        driver = new ChromeDriver();
        driver.get("https://www.saucedemo.com");
    }
 
    @DataProvider(name = "loginScenarios")
    public Object[][] data() {
        return new Object[][] {
            { "standard_user",    "secret_sauce",  true,  "/inventory.html" },
            { "locked_out_user",  "secret_sauce",  false, "locked out" },
            { "",                 "secret_sauce",  false, "Username is required" },
            { "standard_user",    "",              false, "Password is required" }
        };
    }
 
    @Test(dataProvider = "loginScenarios")
    public void shouldHandleLoginScenario(
        String username, String password, boolean shouldSucceed, String fragment
    ) {
        if (!username.isEmpty()) driver.findElement(By.id("user-name")).sendKeys(username);
        if (!password.isEmpty()) driver.findElement(By.id("password")).sendKeys(password);
        driver.findElement(By.id("login-button")).click();
 
        if (shouldSucceed) {
            Assert.assertTrue(driver.getCurrentUrl().contains(fragment),
                "Expected URL to contain " + fragment);
        } else {
            String error = driver.findElement(By.cssSelector("[data-test='error']")).getText();
            Assert.assertTrue(error.contains(fragment),
                "Expected error to contain " + fragment + " but was " + error);
        }
    }
 
    @AfterMethod
    public void teardown() {
        if (driver != null) driver.quit();
    }
}

Four data rows. One test method. Four reported test results. The TestNG report gives you a row each, named with the data values — you can see at a glance which scenarios pass and which fail.

Comparison with pytest and JUnit

# pytest — lightweight, in-source
@pytest.mark.parametrize("username,password,expected", [
    ("standard_user", "secret_sauce", True),
    ("locked_out_user", "secret_sauce", False),
])
def test_login(username, password, expected): ...

// JUnit 5 — ParameterizedTest with various sources
@ParameterizedTest
@CsvSource({
    "standard_user, secret_sauce, true",
    "locked_out_user, secret_sauce, false"
})
void shouldLogin(String user, String pw, boolean ok) { ... }

pytest is the most concise of the three. JUnit's @ParameterizedTest is similar to TestNG's @DataProvider but JUnit's per-source annotations (@CsvSource, @MethodSource, @ValueSource) split the responsibility differently. TestNG's single @DataProvider is the most flexible — you write Java to produce the data, so any source (DB, API, file) works without a special annotation.

The TestNG cheat sheet covers @DataProvider, @Parameters, and the related attributes.

⚠️ Common mistakes

Returning Object[] instead of Object[][]. A common typo — single brackets mean "one row." TestNG calls the test once with the array as a single argument and the type system gets confused. Always double brackets: new Object[][] { {...}, {...} }.
Hardcoding test counts in assertions. A data-driven test that expects "five product cards" because the dataset has five products will break the moment someone adds a sixth row. Read sizes from the data: Assert.assertEquals(displayedRows.size(), data.length); — the test self-adjusts.
Using @Parameters for test data rows. @Parameters reads from testng.xml; you can only specify one set of values per <test> block. Trying to express ten data rows produces ten <test> blocks — much more XML than @DataProvider's ten array rows. Use @DataProvider for data, @Parameters for environment config.

🎯 Practice task

Build a real data-driven test. 30–40 minutes.

Add DataDrivenLoginTest from this lesson to your project. Run it; you should see four test results with descriptive names.
Add four more rows with edge cases: very long username (200+ chars), SQL injection attempt (' OR '1'='1), Unicode characters (😀), trailing whitespace. Run again; observe which the app handles and which it doesn't.
Externalise to JSON. Move the four-row inline array into src/test/resources/testdata/login-scenarios.json and rewrite the data provider to read from it via Jackson. Same test results, but data lives outside Java.
Method injection. Write a single @DataProvider named perTestData that returns successful credentials for shouldLoginSuccessfully and locked-out credentials for shouldFailLogin, distinguishing via Method method. Confirm both tests get the right data.
Parallel rows. Add parallel = true to your provider. Run the suite. The four executions now run on different threads — but you'll likely see chaos because the driver field is shared. Convert WebDriver driver to private static ThreadLocal<WebDriver> driverHolder = new ThreadLocal<>(); and use driverHolder.set(...) / driverHolder.get(). Each thread now has its own driver.
Stretch — data from a CSV. Write a tiny CsvReader utility that reads src/test/resources/testdata/login-scenarios.csv and returns Object[][]. CSV is the friendliest format for non-developers (PMs, business analysts) to edit. The CSV-based provider is what most teams actually ship.

Next lesson: TestNG listeners. The hooks that turn a basic target/surefire-reports/index.html into a Slack notification, a screenshot-on-failure, and a custom HTML report — every cross-cutting concern your suite has, in one declarative class.