izri
izri
02 / 04izri/tests

Do the tests actually pass — and cover the diff?

Not just a green run. Izri measures how much of the changed code the suite exercises, retries suspiciously fast failures, and marks flaky specs instead of letting them poison the verdict.

  • Diff coverage, not repo coverage — the lines that changed
  • Flake detection with automatic quarantine & retry
  • Score drops when new code lands untested
vitest · 214 specstests · 88
✓paginate returns last page12ms
✓cursor survives empty result9ms
⟳retry on flaky socketflaky · 2/33.1s
diff coverage91%

What it asks

Triggers a test run via the runner contract (`.izri/izri.yml`), waits for terminal state, and reads the JUnit XML the suite emits. Hard rules cover analyzer crashes and any explicit failure category the project marks blocking; the soft signal is the pass rate.

Hard rules

Auto-fail
  • •Runner crashed before reaching test execution (analyzer_error category).
  • •Any test marked critical-blocking in the suite failed.

Soft signal

hybrid

Pass rate as a percentage of tests reported, capped at 100. Composite weight in izri/quality: 0.36 — the highest of any signal because tests are the deterministic ground truth.

When it fires

Either automatically off a PR webhook (when `runtime.test` is configured in `.izri/izri.yml`) or on-demand via the dashboard "Run Tests" button. Manual triggers create a test_run row directly; webhook triggers fan out through the test-trigger service.

Implementation reference

The docs page has the full type signatures, config schema, and the on-disk shape of every analyzer output. Or grab the raw markdown via /docs/features/test-execution.md.

Read the docs
Get startedizri/qualitySee all four signals