Custom Annotations and Transformers

The previous lesson used IAnnotationTransformer for one purpose — attaching a retry analyser. The transformer can do much more: set a global timeout, disable tests that carry a @WIP annotation, or wire any framework-level behaviour declaratively. Custom annotations are the companion: you define metadata on test methods (owner, severity, category), and listeners, transformers, and reporters read that metadata to drive behaviour. IMethodInterceptor rounds out the picture — it can re-sort or filter the entire method list after discovery but before execution. Together these three extension points let you build a framework where policy (timeouts, owners, wip gates) is declared once and enforced everywhere. This lesson is aimed at framework builders — most teams use these sparingly, but knowing they exist is how you solve problems that otherwise require copy-pasting annotations on every test.

Custom annotations as metadata

Java annotations are just metadata. Define them in their own files:

package com.mycompany.tests.annotation;
 
import java.lang.annotation.*;
 
/** Marks a test as work-in-progress — will be disabled by the transformer. */
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface WIP {
    String reason() default "Work in progress";
}

package com.mycompany.tests.annotation;
 
import java.lang.annotation.*;
 
/** Declares the engineer responsible for this test. Appears in custom reports. */
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface Owner {
    String value();
}

package com.mycompany.tests.annotation;
 
import java.lang.annotation.*;
 
/** Marks the test's severity for triage priority. */
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface Severity {
    enum Level { CRITICAL, HIGH, MEDIUM, LOW }
    Level value() default Level.MEDIUM;
}

Use them on test methods:

@Test(groups = {"regression"},
      description = "Checkout flow with coupon code")
@Owner("alice")
@Severity(Severity.Level.HIGH)
public void checkoutWithCoupon() { ... }
 
@Test(groups = {"regression"})
@Owner("bob")
@WIP(reason = "Waiting for promo-engine API fix — ticket PROJ-4821")
public void checkoutWithExpiredCoupon() { ... }

IAnnotationTransformer — batch-apply framework policy

The transformer intercepts discovery before any test runs. Use it to enforce global policies without per-test annotation repetition:

package com.mycompany.tests.listener;
 
import com.mycompany.tests.annotation.WIP;
import com.mycompany.tests.retry.RetryAnalyzer;
import org.testng.IAnnotationTransformer;
import org.testng.annotations.ITestAnnotation;
import java.lang.reflect.Constructor;
import java.lang.reflect.Method;
 
public class FrameworkTransformer implements IAnnotationTransformer {
 
    @Override
    public void transform(ITestAnnotation annotation,
                          Class testClass,
                          Constructor testConstructor,
                          Method testMethod) {
 
        if (testMethod == null) return;
 
        // 1. Disable any test annotated with @WIP
        if (testMethod.isAnnotationPresent(WIP.class)) {
            WIP wip = testMethod.getAnnotation(WIP.class);
            System.out.printf("⏸ Disabling @WIP test: %s — reason: %s%n",
                testMethod.getName(), wip.reason());
            annotation.setEnabled(false);
            return;
        }
 
        // 2. Set a global default timeout (30 seconds) if none is set
        if (annotation.getTimeOut() == 0) {
            annotation.setTimeOut(30_000);
        }
 
        // 3. Attach retry analyser if not already present
        if (annotation.getRetryAnalyzer() == null) {
            annotation.setRetryAnalyzer(RetryAnalyzer.class);
        }
    }
}

<suite name="Regression">
    <listeners>
        <listener class-name="com.mycompany.tests.listener.FrameworkTransformer"/>
        <listener class-name="com.mycompany.tests.listener.TestListener"/>
    </listeners>
    <test name="All Tests">
        <packages>
            <package name="com.mycompany.tests.tests"/>
        </packages>
    </test>
</suite>

Every @Test in the suite now: disables automatically if annotated with @WIP, has a 30-second timeout, and retries up to 2 times. Zero annotation clutter in individual test classes.

Reading custom annotations in listeners

Custom annotations become useful when listeners act on them. Add annotation-aware logic to TestListener:

@Override
public void onTestStart(ITestResult result) {
    Method method = result.getMethod()
                          .getConstructorOrMethod()
                          .getMethod();
 
    Owner owner = method.getAnnotation(Owner.class);
    Severity severity = method.getAnnotation(Severity.class);
 
    System.out.printf("▶ STARTED : %s | Owner: %s | Severity: %s%n",
        result.getName(),
        owner    != null ? owner.value()        : "unassigned",
        severity != null ? severity.value()     : "MEDIUM");
}
 
@Override
public void onTestFailure(ITestResult result) {
    Method method = result.getMethod()
                          .getConstructorOrMethod()
                          .getMethod();
 
    Owner owner = method.getAnnotation(Owner.class);
    Severity sev = method.getAnnotation(Severity.class);
 
    System.out.printf("❌ FAILED : %s%n", result.getName());
    if (sev != null && sev.value() == Severity.Level.CRITICAL) {
        System.out.println("   ⚠️  CRITICAL failure — page the on-call engineer");
        // In a real framework: call a Slack webhook or PagerDuty API
    }
    if (owner != null) {
        System.out.printf("   Notify: %s%n", owner.value());
    }
    captureScreenshot(result);
}

IMethodInterceptor — filter and sort at runtime

IMethodInterceptor receives the full list of discovered test methods before any run. You can remove, reorder, or group them:

package com.mycompany.tests.listener;
 
import com.mycompany.tests.annotation.Severity;
import org.testng.IMethodInstance;
import org.testng.IMethodInterceptor;
import org.testng.ITestContext;
import java.lang.reflect.Method;
import java.util.*;
import java.util.stream.Collectors;
 
public class SeverityInterceptor implements IMethodInterceptor {
 
    @Override
    public List<IMethodInstance> intercept(List<IMethodInstance> methods,
                                            ITestContext context) {
        // Run CRITICAL tests first, then HIGH, MEDIUM, LOW
        Comparator<IMethodInstance> bySeverity = Comparator.comparingInt(m -> {
            Method method = m.getMethod().getConstructorOrMethod().getMethod();
            Severity ann = method.getAnnotation(Severity.class);
            if (ann == null) return 2; // default: MEDIUM
            return switch (ann.value()) {
                case CRITICAL -> 0;
                case HIGH     -> 1;
                case MEDIUM   -> 2;
                case LOW      -> 3;
            };
        });
 
        return methods.stream()
                      .sorted(bySeverity)
                      .collect(Collectors.toList());
    }
}

Register as a listener. TestNG runs CRITICAL tests first — if any fail, you find out immediately rather than after 20 minutes of LOW-severity tests.

TestNG extension points — the full picture

TestNG Extension Points

– Fires during test discovery
– Modify any @Test attribute
– Attach retry, set timeout globally
– Disable @WIP tests

– Fires per test lifecycle event
– Screenshots on failure
– Read custom annotations
– Clean up retried-then-passed results

– Fires once before first test
– Sort by priority or severity
– Filter by annotation at runtime
– Inject test ordering policy

Fires once after all suites finish –
Access full ISuite result tree –
Write HTML, JSON, CSV reports –
Post results to external systems –

Putting it all together

A framework that uses all the pieces from this chapter:

// testng.xml registers:
// - FrameworkTransformer  → disables @WIP, sets timeout, attaches retry
// - SeverityInterceptor   → runs CRITICAL tests first
// - TestListener          → screenshots, annotation-aware logging
// - RetryResultCleaner    → removes retried-then-passed failures
// - SummaryReporter       → generates custom HTML report
 
// Test class — clean, no infrastructure boilerplate
public class CheckoutTest extends BaseTest {
 
    @Test(groups = {"smoke", "regression"})
    @Owner("alice")
    @Severity(Severity.Level.CRITICAL)
    public void checkoutHappyPath() {
        // framework handles: timeout, retry, screenshot, logging
    }
 
    @Test(groups = {"regression"})
    @Owner("bob")
    @WIP(reason = "Pending payment-gateway contract — PROJ-5001")
    public void checkoutWithPaymentFailure() {
        // disabled automatically by FrameworkTransformer
    }
}

Each concern is declared once, in one place, and enforced everywhere.

⚠️ Common mistakes

Forgetting @Retention(RetentionPolicy.RUNTIME) on custom annotations. The default retention is CLASS — annotations are in the bytecode but are not accessible via reflection at runtime. method.getAnnotation(WIP.class) returns null and the @WIP logic silently does nothing. Always declare @Retention(RetentionPolicy.RUNTIME) on every custom test annotation.
Implementing IAnnotationTransformer but not registering it in testng.xml. The transformer only fires when TestNG knows about it. If it's only on the classpath but not in <listeners>, nothing changes. Confirm registration by adding a System.out.println at the top of transform() and running the suite — if the print doesn't appear, the listener isn't registered.
Using IMethodInterceptor to enforce test order instead of using priority. IMethodInterceptor is powerful but opaque — a new developer reading a test class cannot see why tests run in a particular order. Use priority for simple ordering; use IMethodInterceptor only for policy-driven ordering (severity-first, smoke-first) that must be applied globally.

🎯 Practice task

Build the framework extension layer. 35–45 minutes.

Create @WIP, @Owner, and @Severity annotations with the correct @Retention and @Target. Add them to three existing test methods.
Implement FrameworkTransformer that disables @WIP tests and sets a 30-second global timeout. Register it in testng.xml. Run — confirm @WIP tests are skipped, and annotation.getTimeOut() returns 30000 for all methods in a debug print.
Update TestListener to read @Owner and @Severity in onTestStart and onTestFailure. Run — the console should show owner and severity per test. Fail a CRITICAL-severity test and confirm the special alert fires.
Implement SeverityInterceptor. Add tests with three different @Severity levels. Run without the interceptor and observe the default order. Register the interceptor and run again — confirm CRITICAL tests now run first.
Test @Retention correctness. Remove @Retention(RetentionPolicy.RUNTIME) from @WIP. Run — the transformer no longer sees the annotation and the @WIP test runs. Restore the retention and reconfirm.

Stretch — IHookable. Implement IHookable and override run() to wrap every test execution with timing:

@Override
public void run(IHookCallBack callBack, ITestResult testResult) {
    long start = System.nanoTime();
    callBack.runTestMethod(testResult);
    long ms = (System.nanoTime() - start) / 1_000_000;
    System.out.printf("   Duration: %dms — %s%n", ms, testResult.getName());
}

The next chapter covers TestNG reporting in depth — the default HTML report, emailable reports, ExtentReports, Allure, and running suites in CI/CD pipelines.