What this gate measures — and what it does not¶
Read this before trusting a green build.
Name¶
This is a docs presence + link-integrity + freshness gate. It is not a documentation quality gate. Nothing here grades whether a document is clear, correct, or useful to a human.
Why it is shaped this way¶
Kerr lens, requirements altitude, 2026-07-15 (FLT-64).
- B (what we actually want): a newcomer or agent can open any fleet repo and understand what it does, how it is shaped, and why it got that way.
- A (what is easy to measure): files exist at expected paths.
A gate on A alone is worse than no gate. The cheapest way to satisfy it is also the default way an agent would: read the README and a file tree, emit plausible documents, never open a source file. The fleet goes green overnight and every repo carries confident fiction. Empty docs at least advertise their emptiness; fiction gets trusted and acted on.
So every rule below is a falsifiable claim — something a document invented from vibes will fail.
What each rule actually proves¶
| Rule | Proves | Does not prove |
|---|---|---|
| Link integrity | Every path/module/symbol named in the docs resolves in the repo. The author had the source in front of them. | That the description of those things is accurate. |
| Making-of cites SHAs | History entries reference commits that exist in this repo. The narrative is anchored to real events. | That the story told about those commits is true. |
Content binding (reviewed-at-commit) |
The doc was reviewed at a stated commit, and source has not moved far since. | That anyone actually read the diff when bumping the SHA. |
| Coverage floor | The architecture doc names every top-level module. Nothing is silently omitted. | That the modules are described correctly. |
| BOARD rows resolve | Ticket keys on the board are real issues. | That the board reflects real status. |
| Diagram presence | A file of each form exists. | Nothing else. Not clarity, not accuracy, not currency. |
Known-gameable, disclosed on purpose¶
Two holes are open and are not worth closing with machinery:
reviewed-at-commitcan be bumped without reading anything. A one-line edit resets freshness.- Diagram quality is unmeasured. An empty canvas of the right file extension passes.
Both ride on /council-code-review, where a model or a human actually looks at the content. The gate's job is to make the cheap failures impossible, not to simulate judgment.
Listing these here is the point. A gate that hides its holes teaches people to trust it more than it has earned.
The fixture that must fail¶
tests/DocsGate.Tests/StubRepoMustFailTests.cs builds a repo carrying every required file, all of them plausible-looking and all of them hollow. The suite asserts it fails, and fails for the right rules. If that fixture ever passes, this gate has become theater and the build should be treated as red regardless of what it prints.
Its mirror, HonestRepo_WithDocsBoundToSource_Passes, asserts that docs genuinely bound to the source go green. A gate that only ever reds teaches people to ignore it, which costs the same as no gate.
Calibration, and its own folly¶
Calibrated against a sweep of 22 fleet repos on 2026-07-15 (251 errors at first run, 150 after). Calibration is where a gate goes to die: "fewer errors" is trivially achieved by gutting the rules, and the number looks better either way.
Two guards against that:
- Every quieted false-positive class in
tests/DocsGate.Tests/CalibrationTests.csis paired with a real catch of the same shape that must stay an error. Loosening a rule fails its pair. - The sweep's true positives are named in tests, so a future "cleanup" that silences them fails the build.
One real trap surfaced this way. Keying link-integrity severity purely on "does the path's root exist in the repo" scored nanoclaw-extras silent — it cites src/cli/client.ts across four documents while having no src/ directory at all. A wholly fabricated layout read as an "external reference" and dropped to a warning, while a partly fabricated one stayed an error. The more invented the docs, the greener the gate. Fixed by treating conventional source roots (src/, tests/, hooks/, …) as claims about this repo whether or not the directory exists.