izri/hallucination

Hallucination

Do the tests that passed actually exercise the lines that changed? Or are they passing without touching the diff?

What it asks

Intersects the per-line v8 coverage captured by the test run with the PR diff. Lines that were added but never executed are "uncovered" — by definition the tests cannot have detected a regression on them. A "change_uncovered" hard rule fires when zero added lines were exercised at all, because the test pass is then meaningless for this delta.

Hard rules

Auto-fail
  • change_uncovered — zero added lines were executed by any passing test.

Soft signal

hybrid

Coverage score 0–100 = percentage of added lines that were exercised by at least one passing test. Composite weight in izri/quality: 0.40 — the strategic differentiator, highest single weight. A partially-covered diff drags the umbrella down even when scope and tests look clean.

When it fires

Computed by the runner alongside the test suite when `runtime.coverage` is wired to a coverage file path (lcov, monocart, v8). Without coverage data the hallucination check skips and the umbrella aggregator redistributes weight to the remaining signals — no penalty for projects that haven't wired coverage yet.

Implementation reference

The docs page has the full type signatures, config schema, and the on-disk shape of every analyzer output. Or grab the raw markdown via /docs/features/test-generation.md.

Read the docs