Scoped test selection

When the GitHub App's PR-delta orchestrator triggers a test run, the runner reads .izri/test-scope.json at the cloned head SHA and uses it to decide which Playwright specs to execute. Without scoping a 3-file PR pays the same ~10-minute ceiling as a 300-file PR; with scoping the runner can skip everything that the diff can't possibly affect.

This is a per-repo, declarative file. The izri repo ships one as a worked example — .izri/test-scope.json at the repo root.

File location

<repo-root>/.izri/test-scope.json

The runner reads it after cloning, before invoking npx playwright test. If the file is missing, malformed, or the diff doesn't match any mapping, the runner falls back to the full suite — running too many specs is a strictly safer failure mode than running too few.

Schema

{
  "version": 1,

  // Diff paths that should escalate to the full suite. Use for shared
  // infrastructure where almost anything could break (lockfiles, root
  // configs, the database schema, your test runner config).
  "fullSuiteOn": [
    "packages/database/**",
    "pnpm-lock.yaml",
    "package.json",
    "playwright.config.*"
  ],

  // Specs that always run regardless of which paths changed. Cheap
  // safety nets — error pages, regressions for previously-shipped
  // bugs, smoke tests. Keep this list small; specs here run on every
  // PR.
  "alwaysRun": [
    "errors.spec.ts",
    "meta.spec.ts"
  ],

  // The body of the mapping. Each entry says: when a changed file
  // matches `changed` (glob), include every spec in `specs`. Mappings
  // are unioned — a PR touching auth AND projects will pick up specs
  // from both.
  "mappings": [
    { "changed": "packages/auth/**", "specs": ["auth.spec.ts"] },
    {
      "changed": "packages/trpc/src/routers/projects.ts",
      "specs": ["projects.spec.ts", "settings.spec.ts"]
    }
  ]
}

Glob syntax

The runner ships a tiny in-house matcher (see apps/runner/src/specSelector.ts):

Pattern	Meaning
`*`	matches anything except `/`
`**`	matches anything including `/`; `foo/**` also matches bare `foo`
`?`	matches a single character except `/`
any other char	literal (regex metacharacters auto-escaped)

It deliberately does not pull in picomatch or minimatch — keeping the runner image lean. If you find yourself needing more expressive patterns, file an issue and we'll swap in picomatch.

Resolution order

When a job runs the selector reaches exactly one of these discrete decisions, exposed as the decision field on the log line and on test_runs.results.selection:

`decision`	When it fires	Outcome
`no_config`	`.izri/test-scope.json` missing or unreadable	Full suite
`empty_diff`	Orchestrator didn't pass any `changedFiles`	Full suite
`full_match`	A changed file matched a `fullSuiteOn` glob	Full suite
`no_mappings_matched`	No `mappings` matched AND no `alwaysRun` specs are configured	Full suite
`mapped`	`alwaysRun` ∪ specs from each matching `mappings` entry, deduped (non-empty)	Scoped subset

The decision is logged at info from apps/runner so Railway log search can answer "did this run get scoped, and if not why?" in one query:

{
  "msg": "job complete",
  "testRunId": "01KS…",
  "status": "PASSED",
  "selection": {
    "runFull": false,
    "decision": "mapped",
    "reason": "scoped to 3 spec(s) via .izri/test-scope.json",
    "specCount": 3
  }
}

Full-suite escalations carry the same decision discriminator:

{
  "msg": "job complete",
  "selection": {
    "runFull": true,
    "decision": "full_match",
    "reason": "changed file pnpm-lock.yaml matches fullSuiteOn pattern pnpm-lock.yaml"
  }
}

The same decision (with full spec list) is also persisted to test_runs.results.selection and surfaced as the first line of test_runs.logs for failure post-mortem.

One mapping table, two consumers

.github/workflows/e2e.yaml consumes the same .izri/test-scope.json through .github/scripts/select-e2e-specs.mjs (see #262). The JSON file is the single source of truth — there is no parallel mapping table to keep in sync.

Authoring tips

Map by import surface, not by feature. When module X is changed, which specs exercise code that imports X (transitively)? Those are the specs that should run. Web routes are the easy case; shared packages (packages/trpc/**, packages/shared/**) tend to escalate to full because almost anything imports them.
Lean on alwaysRun. A small set of cross-cutting specs in alwaysRun is cheaper than mapping every conceivable path → spec. Anything that catches "the build is broken" without needing feature-specific setup (404 pages, basic auth flow, key landing page) belongs there.
Prefer over-running to under-running. Selector failures default to full suite for a reason. A green CI on a half-tested PR is worse than a 12-minute CI on a 3-file PR.