Blueprint Risk Engine

Build tracker. Branch feat/copilot-chat at 853ee72. Updated 9 August 2026.

Verified

1220

tests passing

skipped, and reported

known gap, held open

clean

mypy strict, 37 modules

These are run results, not estimates. The six skips are the chat acceptance tests; they skip loudly and are not counted as passes. The one gap is described below and is deliberately not hidden.

Phases

Phase	State	Note
1. Deterministic engine	built	Rules, validators, ranked register, BOM, report, CLI. Golden tests compare byte for byte.
2. Document intelligence	partial	Ingestion, claim extraction, scope boundaries, the Division 8 / 28 seam.
Copilot chat lane	built	Grounded chat over one loaded package. Retrieval is code, not a model choosing what to read.
PDF ingest lane	built	A directory of package PDFs becomes a document model. Deterministic: no model call, no network, no clock.
3. Plan extraction	not started	Next buildable slice.
4. Benchmark	blocked	No dataset. See below.
5. Interface	deferred	Deliberately after the benchmark.

The first measurement against a document nobody wrote for it

The ingest lane was run against a real 316 page project manual rather than against fixtures. Until now the scope detector had only ever been tested against examples written by the same person who wrote the detector, which measures agreement with yourself.

Four of the five acceptance assertions passed: the document identity and page count match the retrieval record, all six specification sections were found, every section starts within one page of where the record says it does, and the cross-division work assignments were located in the numbers the record counts.

One failed. On one page the detector finds nothing, because that page assigns scope in ordinary English as a residual catch-all rather than using any of the standard contract markers the detector matches.

The detector was not widened to make it pass. A pattern loose enough to catch that phrasing would fire across a large fraction of any specification's body text, and a rule that fires too broadly is worse than an absent rule, because it teaches the reader to ignore the output.

The case is held open as a recorded gap that fails loudly if it is ever quietly closed. Closing it properly means a new marker family with its own positive test and its own near-miss negative test.

This is the honest shape of the result: a narrow sample, one real miss, and a note explaining why the miss is more useful known than tuned away.

What the last session actually did

Nine commits. Roughly half was building the copilot chat lane. The other half was discovering that several things reporting themselves as working did not work.

The architecture test did not enforce the directive it exists for. Writing an ordinary import into the model-authored package reached the deterministic engine with every test green. Twelve import forms evaded it, and its own tests were exercising a hand-copied duplicate of the detector rather than the detector.
The transmission consent gate collapsed on duplicate filenames. Two files with the same name in different folders became one digest entry, so replacing one wholesale left consent current. That is precisely how bid addenda are distributed.
The same gate ignored the recipient list, so adding a model vendor did not re-ask for consent.
The copy guard let pricing through. A currency symbol followed by a zero-width character renders normally and passed the check. So did three non-dollar currency signs and two characters literally named SMALL EM DASH and VERTICAL EM DASH.
Retrieval matched vocabulary as raw substrings. A question about panic hardware, this domain's life-safety case, pulled back scope markers about work assigned to others.
The page selector was wrong in both directions at once, matching page 23 against page 236 while missing a page inside a section that spans it.
Several tests could not fail. Three retrieval selectors could be deleted outright with the suite green, and two guards were mathematical tautologies.

The lesson worth keeping. Every defect above passed its own tests. They were found by mutation testing, which changes the source and checks that some test notices. A green suite is not evidence that the code works; it is evidence that the tests ran.

Blocked

What	Why
Chat acceptance against a real package	Needs recorded model responses, which do not exist yet.
Recording those responses	Requires sending a real bid package to a model vendor. That leaves the machine, so it needs a deliberate go-ahead rather than standing approval.
Phase 4 benchmark	No qualifying dataset found. Districts that publish full bid documents ran the wrong division of work; districts that ran the right work hold their documents for in-person review. This is a records-access problem, not a research problem.

Waiting on a decision

Item	Shape of the call
Committed source excerpts	Rewrite as derived annotations and leave history alone, rewrite history too, or amend the rule explicitly. Recommended: the first.
This tracker's visibility	Deployed as a private tracker. Access control is a dashboard action and has not been applied yet.

What this does and does not claim

What the project can defend today: every output line carries a rule identifier and a document citation, runs replay deterministically from the same inputs, and the system produces a flag rather than a guess where the documents do not determine an answer.

What it cannot defend yet: that the things it flags are the things that actually become change orders. That is what Phase 4 measures, and Phase 4 has no dataset. Until it does, nothing here supports a claim about accuracy, and this page is not evidence that the engine works.