Logging Strategy — Log4j, SLF4J, Structured Logging

A test fails in CI at 3 AM. The team sees AssertionError: expected <'Checkout'> but was <'Error 500'>. There is no other context: no URL that was visited, no request that was sent, no element that was clicked before the assertion ran. Someone has to re-run the test — if it's flaky, the re-run may pass, and the failure is permanently mysterious. This is the cost of no logging strategy. Good logging turns every CI failure into a self-contained incident report: what test ran, what actions were taken, what the application returned, and exactly where things went wrong. This lesson covers the logging stack — SLF4J, Log4j2, structured JSON output — and the specific decisions that make logs useful versus noise.

Why System.out.println is not a logging strategy

System.out.println("Navigating to login page") works exactly once: when you're running locally and watching the console. In production CI:

• No levels — you can't turn off verbose output in normal runs and turn it on when debugging. Everything or nothing.
• No structure — parsing System.out output to find which test printed which line is string-matching archaeology.
• No redirection — you can't send it to a file, a log aggregator, or a structured sink without OS-level piping tricks.
• No thread ID — in parallel runs, output from five threads is interleaved with no way to separate them.

Replace every System.out.println in your framework with a proper logger from the first line of the first page object you write.

The Java logging stack

Java's logging ecosystem has two layers: the facade (the API your code talks to) and the implementation (the library that actually writes log output).

SLF4J is the facade. Your page objects, utilities, and base classes import org.slf4j.Logger and call log.info(...). SLF4J doesn't write logs — it delegates to whatever implementation is on the classpath.

Log4j2 (or Logback) is the implementation. It handles file sinks, rolling policies, JSON formatting, and log level filtering. Swap the implementation without changing a single line of test code.

<!-- pom.xml -->
<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-api</artifactId>
    <version>2.0.12</version>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-slf4j2-impl</artifactId>
    <version>2.23.1</version>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-core</artifactId>
    <version>2.23.1</version>
</dependency>

Using it in a page object:

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
 
public class LoginPage {
    private static final Logger log = LoggerFactory.getLogger(LoginPage.class);
 
    public void login(String email, String password) {
        log.info("Logging in as: {}", email);
        log.debug("Navigating to login form");
        find(emailInput).sendKeys(email);
        find(passwordInput).sendKeys(password);
        log.debug("Submitting credentials");
        find(submitButton).click();
        log.info("Login submitted for: {}", email);
    }
}

{} is SLF4J's placeholder — lazy string interpolation that's evaluated only if the log level is active. No string concatenation cost when DEBUG is disabled in production runs.

Log levels — what belongs where

In CI, run at INFO. When a test fails and you need to understand why, re-run at DEBUG. TRACE is for framework development — never in production CI.

Never log passwords, tokens, or PII at any level. Log password length if you need to debug auth: log.debug("Password length: {}", password.length()).

Structured logging — JSON output for CI pipelines

Plain text logs ("Login submitted for alice@test.com") are readable by humans. Structured JSON logs are readable by humans and queryable by log aggregation tools (Elasticsearch/Kibana, Splunk, Datadog, CloudWatch):

{"timestamp":"2026-05-07T10:30:45.123Z","level":"INFO","logger":"LoginPage","thread":"test-thread-2","test":"adminCanLogin","step":"login_submitted","user":"alice@test.com","duration_ms":234}

Log4j2 configuration for JSON output:

<!-- log4j2.xml -->
<Configuration status="WARN">
  <Appenders>
    <Console name="Console" target="SYSTEM_OUT">
      <JsonTemplateLayout />
    </Console>
    <RollingFile name="File" fileName="logs/test-run.log"
                 filePattern="logs/test-run-%d{yyyy-MM-dd}.log">
      <JsonTemplateLayout />
      <Policies>
        <TimeBasedTriggeringPolicy />
        <SizeBasedTriggeringPolicy size="50MB" />
      </Policies>
    </RollingFile>
  </Appenders>
  <Loggers>
    <Root level="INFO">
      <AppenderRef ref="Console"/>
      <AppenderRef ref="File"/>
    </Root>
    <!-- Debug level for framework code only -->
    <Logger name="com.mycompany.tests" level="DEBUG" additivity="false">
      <AppenderRef ref="Console"/>
    </Logger>
  </Loggers>
</Configuration>

The equivalent in Python and TypeScript:

# Python — structured logging with structlog or logging + json formatter
import logging, json, time
 
class JsonFormatter(logging.Formatter):
    def format(self, record):
        return json.dumps({
            "timestamp": self.formatTime(record),
            "level": record.levelname,
            "logger": record.name,
            "message": record.getMessage(),
        })
 
logging.basicConfig(handlers=[logging.StreamHandler()])
logging.getLogger().handlers[0].setFormatter(JsonFormatter())
log = logging.getLogger(__name__)

// TypeScript — pino (structured logging)
import pino from "pino";
 
const log = pino({
  level: process.env.LOG_LEVEL ?? "info",
  transport: process.env.CI
    ? undefined
    : { target: "pino-pretty" },  // human-readable locally, JSON in CI
});
 
// Usage
log.info({ test: "adminCanLogin", step: "navigate", url: config.baseUrl }, "Navigation started");

What to log and where

The minimum useful log set for diagnosing CI failures:

In BaseTest / test lifecycle hooks:

@BeforeMethod
public void setUp(Method method) {
    log.info("=== TEST START: {} | env: {} | browser: {} ===",
        method.getName(), Config.env(), Config.browser());
}
 
@AfterMethod
public void tearDown(ITestResult result) {
    log.info("=== TEST END: {} | status: {} | duration: {}ms ===",
        result.getName(), result.isSuccess() ? "PASS" : "FAIL",
        result.getEndMillis() - result.getStartMillis());
}

In page object actions:

public void login(String email, String password) {
    log.info("Login attempt: {}", email);
    // ...action code...
    log.debug("Login form submitted");
}

In API clients:

public Response post(String path, Object body) {
    log.debug("POST {} | body: {}", path, body);
    Response response = given().body(body).post(path);
    log.info("POST {} → {} {}ms", path, response.statusCode(), response.time());
    return response;
}

⚠️ Common mistakes

• Logging inside loops without rate limiting. A loop that logs every iteration at INFO in a suite that processes 10,000 rows produces 10,000 INFO lines — most of them identical. Log at DEBUG inside loops; log a summary at INFO after the loop completes.
• Catching exceptions silently. catch (Exception e) { log.error("Error"); } swallows the stack trace that explains exactly what went wrong. Always log the exception: log.error("Login failed for {}", email, e) — the last argument being the Throwable triggers stack trace output.
• Different log format in local vs CI. Switching from plain text locally to JSON in CI means local log analysis doesn't prepare you for what CI logs look like. Use the same structured format everywhere; use pino-pretty or similar for local human readability on top of the same underlying JSON.

🎯 Practice task

Add structured logging to your framework — 30 minutes.

1. Add the dependency. Add SLF4J + Log4j2 (or Python's structlog / Node's pino) to your project. Verify that a simple log.info("Hello") appears in the console when a test runs.
2. Replace System.out.println. Find every System.out.println or print() in your framework. Replace each with the appropriate logger call at the appropriate level.
3. Add lifecycle logging. In @BeforeMethod (or setup() fixture), log the test name, environment, and browser. In @AfterMethod, log the test result and duration. Run the suite — each test should produce a START and END log line.
4. Add action logging. In your 3 most-used page object methods, add one INFO log line per method. Run a test and trace the full sequence from the logs alone — without reading the test code.
5. Stretch — structured JSON. Configure Log4j2 to write JSON to logs/test-run.log. After a test run, open the log file. Run grep '"level":"ERROR"' logs/test-run.log to find only failures. This demonstrates the operational value of structured logging: you can query failures without a log aggregator.

Next lesson: the reporting layer — how to turn test results into HTML dashboards that stakeholders can read without a terminal.