Test Selection — Running Only What's Affected

Parallelism and caching make your existing pipeline faster. Test selection makes it smarter. A developer who changes one CSS file in the frontend doesn't need to wait for your backend API test suite or your mobile integration tests — those tests cannot possibly be affected by a CSS change. Running them anyway wastes time, burns CI minutes, and trains the team to tune out long pipelines. Test selection is the discipline of running the right tests, not just all the tests.

The strategies, from simple to complex

Test selection approaches range from tag-based (trivial to implement, always worth doing) to dependency-aware affected analysis (powerful but requires tooling investment). Most teams benefit enormously from the simpler approaches and never need the advanced ones.

Strategy 1: Tag-based selection (always do this first)

Tags are the foundation. Before any other test selection strategy, every test suite should have a smoke set and a full set, selectable by tag:

# Playwright
npx playwright test --grep @smoke         # smoke only
npx playwright test                        # everything
 
# TestNG
mvn test -Dgroups=smoke                   # smoke group
mvn test -Dgroups=smoke,api               # multiple groups
 
# JUnit 5
mvn test -Dgroups=smoke                   # @Tag("smoke") tests

On every PR: run @smoke. On nightly: run everything. This alone cuts PR pipeline time from 40 minutes to 5–8 minutes for most teams — without any path filtering or dependency analysis.

If your tests aren't tagged, tagging them is the first work to do before anything else in this lesson.

Strategy 2: Path filters — skip the workflow entirely

GitHub Actions can skip a workflow if only irrelevant files changed:

on:
  pull_request:
    branches: [main]
    paths:
      - 'src/**'
      - 'tests/**'
      - 'pom.xml'
      - 'package.json'
      - 'package-lock.json'
      - '.github/workflows/**'

With this configuration, a PR that only changes README.md, docs/, or CHANGELOG.md skips the workflow entirely. No runner is provisioned, no minutes are spent. For documentation-heavy repositories, this alone cuts CI cost significantly.

Be conservative about what you exclude. A change to docker-compose.yml might affect how your tests connect to services. A change to .env.example might signal a new required variable. When in doubt, include the path.

Strategy 3: Conditional jobs based on changed files

When different file changes should trigger different test subsets, use path filtering at the job level with dorny/paths-filter:

jobs:
  detect-changes:
    runs-on: ubuntu-latest
    outputs:
      api: ${{ steps.filter.outputs.api }}
      ui: ${{ steps.filter.outputs.ui }}
      db: ${{ steps.filter.outputs.db }}
    steps:
      - uses: actions/checkout@v4
      - uses: dorny/paths-filter@v3
        id: filter
        with:
          filters: |
            api:
              - 'src/api/**'
              - 'src/services/**'
            ui:
              - 'src/ui/**'
              - 'src/components/**'
            db:
              - 'src/db/**'
              - 'migrations/**'
 
  api-tests:
    needs: detect-changes
    if: needs.detect-changes.outputs.api == 'true'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-java@v4
        with: { java-version: '21', distribution: 'temurin', cache: 'maven' }
      - run: mvn test -Dgroups=api -B
 
  ui-tests:
    needs: detect-changes
    if: needs.detect-changes.outputs.ui == 'true'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '20', cache: 'npm' }
      - run: npm ci && npx playwright test --grep @smoke
 
  db-tests:
    needs: detect-changes
    if: needs.detect-changes.outputs.db == 'true'
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-java@v4
        with: { java-version: '21', distribution: 'temurin', cache: 'maven' }
      - run: mvn test -Dgroups=db -B

A PR touching only src/ui/** triggers ui-tests but skips api-tests and db-tests. A PR touching src/api/** and src/services/** triggers api-tests only. A PR touching all three triggers all three.

The detect-changes job always runs (it's cheap — no tests, just a diff analysis). The expensive test jobs are conditional.

Strategy 4: Framework-native affected test selection

Some frameworks can analyse which tests are affected by changed files without manual configuration:

Vitest (TypeScript/JavaScript):

npx vitest run --changed          # runs tests covering changed files since last commit
npx vitest run --changed HEAD~1   # since the previous commit

Nx (monorepo tooling):

npx nx affected:test              # runs tests for all affected projects in the monorepo

Jest with --changedSince:

npx jest --changedSince=main      # runs tests affected by changes since main branch

These tools use the test/source dependency graph to determine which tests to run. A change to src/checkout/discount.ts triggers tests that import it — directly or transitively. This is the most precise form of selection but requires a well-structured codebase and framework support.

For Java and traditional Selenium projects, there's no mature equivalent. The practical approach is tag-based selection combined with path filters.

The nightly safety net

Every test selection strategy accepts a risk: you might skip a test that would have caught a regression. The mitigation is a nightly run that skips no tests:

on:
  schedule:
    - cron: '0 3 * * *'   # 3:00 AM daily
 
jobs:
  full-regression:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: '20', cache: 'npm' }
      - run: npm ci
      - run: npx playwright install --with-deps chromium
      - run: npx playwright test   # no --grep, no --shard, no skipping

The nightly run is your safety net. It catches regressions that slipped through the PR-time selection. It catches flaky tests. It gives you a daily baseline. Make sure someone is notified when it fails.

The recommended layered approach

Most teams land on a combination of all the strategies above:

⚠️ Common mistakes

• Skipping the nightly safety net. Test selection on PRs is a calculated risk — you're betting that the skipped tests are genuinely unaffected. Without a nightly full run, regressions in skipped tests go undetected until production. The nightly run is not optional.
• Overly narrow path filters. A change to a shared utility that's used across the whole codebase will have a src/utils/** path but affects tests in every area. Filters that are too narrow create false confidence. When unsure, include the path.
• Using framework-native affected analysis without understanding the dependency graph. vitest --changed is powerful but requires the test files to import the source files they test. Tests that use integration-level fixtures or hit a shared API don't have the static import that the analyser follows — those tests will be skipped even when they're affected.

🎯 Practice task

Implement layered test selection for your project — 30 minutes.

1. Tag your 10 most critical tests as @smoke (Playwright) or add them to a smoke group (TestNG). Confirm npx playwright test --grep @smoke or mvn test -Dgroups=smoke runs only those tests.
2. Add path filters to your GitHub Actions workflow so it only runs on changes to src/**, tests/**, and your build file. Push a docs-only change and confirm the workflow is skipped.
3. Add a detect-changes job using dorny/paths-filter with two filters (e.g., api and ui). Make the corresponding test jobs conditional on those outputs.
4. Add a schedule: - cron: '0 3 * * *' trigger to your workflow. Make the scheduled run execute without any --grep or group filter.
5. Stretch: measure total CI time per week with the layered approach vs running everything on every PR. Estimate the saving in developer waiting time.

You've now completed Chapter 4. The next chapter shifts from optimising test execution to making its results visible: JUnit XML reports, Allure dashboards, quality gates that fail builds, coverage reporting, and Slack notifications that reach the right people when something breaks.