31 KiB
Milestone 24: skipped_unsupported is persisted, surfaced and explained
Goal: close the compile-pipeline silent drop that misleads the operator. Counts. skipped_unsupported (src/filter/compiler.zig:34) is counted on every compile (compiler.zig:92) and then discarded on the happy path — it reaches the compiled-file header and the NoValidEntries error text, but no database column, no API field and no UI cell. Its sibling skipped_regex reaches all three. A blocklist made almost entirely of cosmetic browser filters therefore compiles to almost nothing and looks, in the UI, exactly like a clean list. AGENTS.md forbids exactly this ("no silent drops"). The milestone persists the count, surfaces it everywhere skipped_regex is surfaced, and explains to the operator why the two skip counters are different facts.
Design written 2026-08-13 against HEAD 21571e4, revised after a Codex review of the first draft.
Implementation contract (read first)
- Read
AGENTS.md, then this spec whole, before session work starts. - The v1 baseline is editable and there is no migration step. As of commit
21571e4,src/storage/migrations.zigholds exactly one step andtarget_versionis 1 (migrations.zig:29-36). nxdns has zero installs; the baseline freezes at v0.1 (config_schema.zig:6-12, PLAN §3.7, §11.2). The new column is one line edited intoconfig_schema.ddl_v1plus the identical line in PLAN §11.2, which is kept byte-identical to it. A diff that adds addl_v2or a secondStepis wrong and must be reverted, not merged. - After the API shape change, regenerate the contract samples (
web/src/lib/contractSamples.gen.ts; procedure in AGENTS.md — thezig build test -Dintegration -Dcontract-samples-out=...command, never a hand edit) and updateweb/src/lib/types.tsto match. - No new
src/**.zigfiles, sosrc/tests.zigandbuild.zigare untouched.
Rulings (binding)
1. The column is skipped_unsupported_count, one edited line, no step
skipped_regex_count (config_schema.zig:63) sets the convention; the new column follows it. In config_schema.ddl_v1, directly after the skipped_regex_count line inside CREATE TABLE blocklist_sources:
skipped_unsupported_count INTEGER NOT NULL DEFAULT 0,
The identical line goes into PLAN §11.2 after PLAN.md:421 — that section is kept byte-identical to ddl_v1 and currently is (verified: the fenced SQL matches the Zig string literal line for line).
Consequence to accept, not to fix: a development database stamped version 1 before this edit will fail the first SELECT naming the column with "no such column", because the stamped version equals target_version and the step never reruns. That is the documented pre-v0.1 contract (config_schema.zig:6-12) — delete the scratch database. Do not add fallback SQL, PRAGMA table_info probing, or a migration step to paper over it.
The baseline test "a fresh database reaches the baseline with every v1 column and rule kind" (migrations.zig:266-283) gains try testing.expect(try columnExists(&database, "blocklist_sources", "skipped_unsupported_count")); beside the existing exception_count probe.
2. The repo persists it beside skipped_regex_count
src/storage/repositories/sources_repo.zig, mirroring skipped_regex_count exactly (no default on either struct field, so every construction site fails to compile until it names the count — that is the visibility this milestone is about):
SourceRow(sources_repo.zig:80-97) gainsskipped_unsupported_count: i64afterskipped_regex_count.SourceStats(sources_repo.zig:99-110) gains the same field.row_columns_sql(sources_repo.zig:112-117) appendsskipped_unsupported_countto the SELECT list (index 11);readSourceRow(sources_repo.zig:128-148) reads it withstmt.columnInt(11).update_stats_sql(sources_repo.zig:159-164) addsskipped_unsupported_count = ?8;updateSourceStats(sources_repo.zig:168-179) binds it.- The module doc comment (sources_repo.zig:3-5) adds the column to its list of server-produced facts, and so does the
src/config/model.zig:14doc comment that repeats that list.
The sum at sources_repo.zig:318 includes the new column. That sum lives in the test "insertBlocklistSource leaves the runtime columns at their defaults"; it exists to prove an insert leaves every runtime counter at 0, and the new column is a runtime counter. This is a deliberate decision, not an oversight: no production query sums these columns into a "total entries" figure, and none may start to — skipped_regex_count and skipped_unsupported_count count lines not written, unlike domain_count, wildcard_count and exception_count, so any future entries total must exclude both. The test sum asserts defaults, which is the one context where adding them is correct.
Existing SourceStats / assertion sites in this file (the literals at sources_repo.zig:386-393, 420-427, 483-493 and the zero-default loop at sources_repo.zig:365-374) carry the new field with distinct non-zero values where their siblings have them, and the round-trip assertions extend to it.
3. The manager writes it, and rehydration restores it
src/filter/manager.zig. The header already prints # skipped_unsupported {d} (manager.zig:243) and SourceStatus.counts is a full compiler.Counts (manager.zig:178), so within one process the count already reaches the status table. What is missing is the database, which is the only thing a restart reads — rehydration never reparses headers.
-
The fresh-publish path (manager.zig:854-861) passes
.skipped_unsupported_count = compiled.result.counts.skipped_unsupportedtoupdateSourceStats. -
The checksum-unchanged path (manager.zig:821-836) carries a bug this milestone fixes. It writes the stored
row.skipped_regex_countback to the database (manager.zig:830) while handing the freshcompiled.result.countsto the in-memory status (manager.zig:833). The checksum covers the.list,.wildand.allowbodies only, and a skipped line lands in none of them — so a list that changes only its regex or browser-syntax lines keeps its checksum, the running server shows the new number, the database keeps the old one, and the next restart silently reverts what the operator saw. That is milestone 21's column, wrong today.Both skip counters take
compiled.result.countson this path:.skipped_regex_count = compiled.result.counts.skipped_regexand.skipped_unsupported_count = compiled.result.counts.skipped_unsupported.Corrected post-commit: that claim was false. The unframed digest could not distinguish an entry indomain_count,wildcard_countandexception_countkeep reading fromrow— they count written entries, so an unchanged checksum does mean an unchanged value for them..listfrom the same entry in.wild, so an unchanged checksum did not vouch for the entry counts either. The addendum below frames the hash and makes all five stat fields read fresh on this path.This needs a test the author has watched fail with the fix reverted: two compiles of bodies whose written entries are identical but whose skipped lines differ, asserting the same checksum, then asserting the database holds the second compile's counts, then rehydrating through
applyLoadOutcomesto prove the restored status carries them. Report the observed failure output. -
applyLoadOutcomesrehydration (manager.zig:1545-1550) adds.skipped_unsupported = countOf(row.skipped_unsupported_count)to theCountsit rebuilds, so a restarted server reports what the last compile skipped instead of 0. -
Test literals name the field: one
SourceStats(manager.zig:1928) and twoSourceRow(manager.zig:2051, 2225). The rehydration test that feeds a row throughapplyLoadOutcomesasserts a non-zeroskipped_unsupportedlands instatus.counts. The header pin test (manager.zig:2030-2031) already covers the# skipped_unsupportedline and is extended only if its fixture counts change. -
rejectedWithoutEntries(manager.zig:1640-1643) andfailNoValidEntries(manager.zig:1127-1128) already read the count and are unchanged.
src/filter/filter_integration_test.zig names .skipped_regex_count in six SourceStats literals (:826, :858, :1075, :1145, :1257, :2158) — each gains the sibling field. The stats round-trip assertion at :915 gains a sibling assertion for the new column.
Do not add an unsupported line to the existing fixture. The fixture at filter_integration_test.zig:112 is hosts-shaped, and detectFormat (src/filter/parsers.zig:118) assigns one format to a whole source. A single ##.ad-banner or $-modifier line flips it to ABP, which re-parses every existing line: the *.wild entry becomes unsupported and address fields tokenize as domains (src/filter/parser_abp.zig:68). Every expected count in that test would move, for a reason unrelated to this milestone.
Add a separate ABP-format fixture and test carrying ##.ad-banner and ||ads.example^$third-party beside two blockable names, and assert skipped_unsupported_count = 2 through the compile-persist-read round trip there.
src/config/reconcile.zig: the runtime-column preservation test seeds stats at reconcile.zig:1096-1110 and asserts survival at reconcile.zig:1173. The seed gains .skipped_unsupported_count with a distinct value and the assertion block gains its expectation. No reconcile logic changes: the engine updates declarative columns by name and never touches runtime columns, so the new column survives with zero code change — the test is the proof.
4. The API speaks it in both shapes
Two shapes carry blocklist counters, and the new count joins both under the names its siblings set:
Blocklist(rows ofGET /api/blocklists, serialized straight fromSourceRow): the field arrives automatically onceSourceRowhas it, asskipped_unsupported_count.src/web/openapi.yaml:1877-1895adds it torequiredandproperties.SourceStatus(rows ofPOST /api/blocklists/update):StatusView(src/web/handlers/blocklists.zig:52-80) gainsskipped_unsupported: u32afterskipped_regex, mapped fromstatus.counts.skipped_unsupportedinfrom.src/web/openapi.yaml:1920-1938adds it torequiredandproperties. The handler test literals at blocklists.zig:323 (SourceStats) and :386 (counts) carry the field with non-zero values and the response assertions extend to it.
Contract fallout, all in the same session (ruling 6 explains why):
-
Regenerate
web/src/lib/contractSamples.gen.tswith the AGENTS.md command. -
web/src/lib/types.ts:Blocklistgainsskipped_unsupported_count: number(types.ts:154-166);SourceStatusgainsskipped_unsupported: number(types.ts:183-195). -
The web test mocks are not typed against these interfaces, so
tscwill not force them.BLOCKLISTSinweb/src/features/blocklists/BlocklistsPage.test.tsxis an inferred object literal handed to aRecord<string, unknown>(BlocklistsPage.test.tsx:40), and the same untyped-fetch pattern holds inweb/src/features/groups/GroupsPage.test.tsx:16andweb/src/features/settings/authority.test.tsx:64— both of which already omitexception_countwithout failing. Updating a mock here is a semantic fixture change, not a typecheck fix, and the implementer must not expect a compiler error to point at them.So: the BlocklistsPage mocks (:21, :34 blocklist rows; :104, :155, :168 status rows) gain the field because S2's rendering assertions read it. The groups and settings mocks are left alone — they render no counter column, and widening them buys nothing. S1 picks values no other cell in the same table already shows (
3and7are taken), e.g.skipped_unsupported_count: 21andskipped_unsupported: 17.
5. The UI shows it always, in both tables, and says what it means
Both tables gain a Skipped unsupported column directly after Skipped regex, rendered unconditionally:
web/src/features/blocklists/BlocklistsPage.tsx: header after :157, cell{b.skipped_unsupported_count}after :190, sameshared.td, shared.tabularNumsprops as its neighbours.web/src/features/blocklists/SourceStatusSection.tsx: header after :91, cell{source.skipped_unsupported}after :114.
Always-shown is a decision, not a default: every other counter column here is unconditional, including Skipped regex and Exceptions, which are 0 for every plain hosts list, and the tutorial already explains those zeros. A column that appears only when non-zero would make two lists' tables disagree in shape, would hide the header that gives the number its meaning, and would special-case exactly the counter this milestone exists to make visible. The zero cell is not noise; it states that nothing in this list was classified as unsupported — which is narrower than "clean", since invalid and long_lines stay unsurfaced (ruling 7).
The two counters mean different things and the page must say so once. A muted paragraph (the existing styles.empty-style muted text, matching SourceStatusSection's note treatment) rendered under the sources table in BlocklistsPage.tsx, inside the same else branch as the table — the note describes the two skip columns, so it appears exactly when they do. "Always visible" above means unconditional on values, never hidden at 0; it does not mean the empty state (blocklists.length === 0) carries a paragraph about columns that are not on screen. The empty state stays one instruction. Exact copy:
Both “Skipped” columns count lines nxdns read and did not take. Skipped regex lines are patterns nxdns accepts only from you — adopt one you trust as a regex rule. Skipped unsupported lines are syntax nxdns cannot translate into a DNS decision: cosmetic element hiding, browser-only modifiers. A skipped unsupported count that dwarfs the domain count usually means the list is written for a browser extension, and its DNS or hosts variant will block more here.
BlocklistsPage.test.tsx asserts: both new headers render, the mock values (21, and 17 after an update snapshot) render, and the note text is present. No threshold logic, no badge, no coloring by magnitude (see anti-requirements).
6. The docs explain both counters without contradicting what stands
-
docs/reference/configuration.md, section### blocklist_sources, after the exceptions paragraph (configuration.md:282-288), a new paragraph:Two counters report what a compile skipped, and they are different facts.
skipped_regexcounts regex lines: nxdns has a regex engine, but it takes patterns only from the operator, so a regex line in a downloaded list is counted, skipped and surfaced — adopt the ones you trust asregexrules.skipped_unsupportedcounts lines nxdns cannot safely translate into a DNS decision: cosmetic element hiding (##,#@#,#?#), rules carrying a$modifier (except$importanton an exception line, tolerated above), scheme anchors, non-anchored@@forms — and, in adomains-format list, a line holding more than one field before its inline comment, which usually means the list is really a hosts file that was declared asdomains. Neither is an error, and the two are never one number. A largeskipped_unsupportedbeside a smalldomain_countusually means the list is written for browser extensions, and its DNS or hosts variant will block more here. Both appear per source in the blocklists UI and on/api/blocklists. -
docs/tutorial/first-run.md: the recorded JSON at first-run.md:223 gains"skipped_unsupported":0after"skipped_regex":0(the endpoint now returns it; the recorded values themselves stand). The paragraph at first-run.md:226-231 becomes four zeros, keeps itsskipped_regexsentence as written, and drops the "parts of this list that a hosts file cannot have" framing — a deliberate small widening ruled during implementation review. The framing was false before this milestone touched it:parser_hosts.zigrecognizes regex lines (:16), reports a bare sink address as unsupported (:21), and a*.-prefixed name compiles to a wildcard in any format. Onlyexceptionsis Adblock-Plus-only. The paragraph now presents the zeros as facts about this particular download and says so. -
The
$modifier is named as skipped with its one exception:$importanton an anchored@@exception line is accepted and lands inexception_count(parser_abp.zig, PLAN §2.2). The configuration.md paragraph above, and theSourceRow.skipped_unsupported_countdoc comment insources_repo.zigthat mirrors it, both carry the qualifier — an unqualified "rules carrying a$modifier" contradicts the parser. -
PLAN.md:101 (§3.8) currently reads "regex lines are counted + skipped (counts in metadata → UI)". It becomes: regex lines and browser-syntax lines are counted and skipped, both counts in metadata → UI. One sentence; §2.2 (PLAN.md:36-37) already says unsupported forms "stay unsupported and counted" and needs no edit.
-
docs/reference/files-and-directories.mdis untouched: the compiled-file header already carried# skipped_unsupportedbefore this milestone and the file table there does not enumerate header lines.
7. The other three counters stay out, and the reasons differ per counter
The first draft claimed invalid, long_lines and duplicates were already "header-and-error-path only". That is false:
invalidreaches the compiled-file header (manager.zig:244) and theNoValidEntriestext (manager.zig:1127).long_linesreaches theNoValidEntriestext only — not the header.duplicatesreaches nothing: counted at compile, discarded on every path.
They stay out of scope, and not for one shared reason — which is why the goal above says "the silent drop that misleads the operator" rather than "the one silent drop left":
duplicateshides no failure. A deduplicated name still blocks; the count measures input redundancy, not lost coverage. Discarding it discards a curiosity, so "no silent drops" does not reach it — there is no drop.invalidandlong_linesdo measure dropped lines, and they remain only partially surfaced. The catastrophic form — a download that is all rejects — already fails loudly (rejectedWithoutEntries, stateno_valid_entries); the residual is a list that loses some lines and still loads, visible in the header file and nowhere the UI reaches. That residual is real, it is recorded here deliberately, and it is not this milestone: the two skip counters name an action the operator can take (adopt the patterns; fetch the DNS variant), while these two say only "the list is malformed", which no column makes more actionable.
The decision is reversible; a later milestone can widen the table. What this spec may not do is claim the three are visible when duplicates is not.
8. Export, import and reconcile change nothing
nxdns export / import carry configuration; the compile counters are facts a running server produces. model.BlocklistSource holds only the four configuration columns, the insert leaves runtime columns at their defaults (sources_repo.zig:1-9, :48-61), and the dump helpers used by the import and reconcile suites are SELECT * compared dump-to-dump (src/config/import.zig:167-186, src/config/reconcile.zig:988), so no golden text names columns and the new column appears in both sides of every comparison. Ruling 3's reconcile-test extension is the only touch in src/config/, plus the model.zig:14 comment from ruling 2.
Sessions
Two sessions, strictly sequential: S1 then S2. No parallelism — deliberately. The regenerated contractSamples.gen.ts typechecks only against a types.ts that already carries the new fields, and the samples are produced by a Zig integration run, so the generator and the interface it must satisfy sit on opposite sides of the language boundary and cannot land independently. S2's rendering assertions then read fields that only exist once S1 has landed both.
(The mocks are not part of this argument: they are untyped, so widening types.ts does not break them. The first draft claimed otherwise.)
Session S1: column, persistence, API, contract
Owns: src/storage/config_schema.zig, src/storage/migrations.zig (test only), src/storage/repositories/sources_repo.zig, src/filter/manager.zig, src/filter/filter_integration_test.zig, src/config/reconcile.zig (test only), src/config/model.zig (comment only), src/web/handlers/blocklists.zig, src/web/openapi.yaml, web/src/lib/types.ts, web/src/lib/contractSamples.gen.ts (regenerated, never hand-edited), web/src/features/blocklists/BlocklistsPage.test.tsx (mock fields only — no rendering assertions), PLAN.md (the §11.2 line and the §3.8 sentence).
- S1.1 rulings 1 and 2: the DDL line in both copies, the repo columns, the migrations and repo tests.
- S1.2 ruling 3: both
updateSourceStatscall sites, rehydration, test literals, the reconcile preservation assertion. - S1.3 ruling 4:
StatusView, openapi.yaml, sample regeneration,types.ts, mock fields.
Acceptance (S1):
zig build testpasses; the migrations baseline test provesblocklist_sources.skipped_unsupported_countexists.- A compile of the new ABP-format fixture carrying
##.ad-bannerand||ads.example^$third-partypersistsskipped_unsupported_count = 2throughupdateSourceStatsand reads it back throughlistSourceRows. The existing hosts-shaped fixture is unchanged, and every count it already asserts still holds. - A second compile whose written entries are byte-identical but whose skipped lines differ produces the same checksum and still updates both skip counters in the database; the test was watched failing with the manager.zig:830 fix reverted, and the failure output is recorded.
applyLoadOutcomesfed a row withskipped_unsupported_count = 5and no live refresh yieldsstatus.counts.skipped_unsupported == 5.zig build test -Dintegrationpasses, and the regeneratedcontractSamples.gen.tscarriesskipped_unsupported_countin the Blocklist sample andskipped_unsupportedin the SourceStatus sample.cd web && npm run typecheck && npm testpass with the widened types.
Session S2: UI and docs (needs S1)
Owns: web/src/features/blocklists/BlocklistsPage.tsx, web/src/features/blocklists/SourceStatusSection.tsx, web/src/features/blocklists/BlocklistsPage.test.tsx (rendering assertions), docs/reference/configuration.md, docs/tutorial/first-run.md.
- S2.1 ruling 5: both columns, the note paragraph, the rendering assertions.
- S2.2 ruling 6: the two doc edits.
Acceptance (S2):
cd web && npm run typecheck && npm test && npm run lintpass; the new tests assert bothSkipped unsupportedheaders, the mock values and the note text.npm run buildpasses (assert-css-layers.mjsruns inside it).docs/tutorial/first-run.mdno longer says "three zeros", its JSON sample carriesskipped_unsupported, and itsskipped_regexsentence is unchanged.
Orchestrator
Verify S1 acceptance before starting S2. After S2, run the full gate set (zig build test, zig build test -Dintegration, test-aarch64 if qemu is present, cd web && npm test, npm run assert-bundled), then a live smoke against a scratch server with two real sources:
https://easylist.to/easylist/easylist.txt— a browser-targeted list; expect askipped_unsupportedseveral times itsdomains(do not assert an exact number; assert the ratio and non-zero).https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts— expectskipped_unsupported0.
Verify the numbers appear in the POST /api/blocklists/update response, in GET /api/blocklists, and in both UI tables. Then restart the server and verify the counts survive into the status table without a refresh — that is ruling 3's rehydration working against the real database. Record deviations in ## Recorded (implementation).
Module layout
New files: none. Deleted surface: none.
File ownership
| File | Session |
|---|---|
src/storage/config_schema.zig |
S1 |
src/storage/migrations.zig |
S1 |
src/storage/repositories/sources_repo.zig |
S1 |
src/filter/manager.zig |
S1 |
src/filter/filter_integration_test.zig |
S1 |
src/config/reconcile.zig (test), src/config/model.zig (comment) |
S1 |
src/web/handlers/blocklists.zig, src/web/openapi.yaml |
S1 |
web/src/lib/types.ts, web/src/lib/contractSamples.gen.ts |
S1 |
PLAN.md (§11.2 line, §3.8 sentence) |
S1 |
web/src/features/blocklists/BlocklistsPage.test.tsx |
S1 (mock fields), then S2 (assertions) — sequential, never concurrent |
web/src/features/blocklists/BlocklistsPage.tsx, web/src/features/blocklists/SourceStatusSection.tsx |
S2 |
docs/reference/configuration.md, docs/tutorial/first-run.md |
S2 |
Acceptance (milestone complete)
- All session acceptance boxes above.
config_schema.ddl_v1and PLAN §11.2 are byte-identical, both carrying the new line;migrations.stepsstill holds exactly one step andtarget_versionis still 1.- The live smoke: EasyList shows a large
skipped_unsupportedand StevenBlack shows 0, in the API and in both UI tables, and both values survive a server restart. nxdns exportoutput is byte-identical before and after a refresh that wrote the new column (runtime columns stay out of exports).- The regenerated contract samples typecheck against
web/src/lib/types.ts— that is what the sample mechanism proves, and it covers server-to- TypeScript agreement only. src/web/openapi.yamlis reviewed by hand against the two changed response shapes, and the reviewer says so in## Recorded. No automated guard covers this:web_integration_test.zig:2089checks that routes and methods exist and:2575counts operations, but nothing compares a component schema to a real response. An openapi.yaml that omits the new field will pass every gate in this repo.
Anti-requirements
- No migration step, no
ddl_v2, no runtime schema probing. The baseline is edited in place per §3.7; a pre-edit scratch database is deleted, not reconciled. - No merging of the two skip counters into one number, anywhere — not in the API, not in a UI total, not in prose. They are different facts.
- No "this list targets browsers" heuristic: no threshold, badge, warning color or ratio computation in the UI. The number plus the note paragraph is the surface; a cutoff would be an invented policy.
- No conditional rendering of the new column. It shows at 0 like every other counter column.
- No change to
nxdns export/importZON: compile statistics are not configuration. - No per-line diagnostics, no sample of skipped lines in logs, API or UI — counters and health surfaces, not log spam (AGENTS.md).
- No change to
rejectedWithoutEntriesorfailNoValidEntriessemantics. - No hand edits to
contractSamples.gen.ts. - No new columns beyond the one. See ruling 7 for what that leaves open and why — the reason is a scope decision, not a claim that the other counters are already visible.
Addendum (post-1bce81e): the body checksum is framed
A filtering correctness bug found by the implementation review, predating this milestone. Fixed as a follow-up commit; this addendum is its design record — the user ruled it a follow-up, not a milestone 25.
The defect
bodyChecksum (manager.zig:1660) and the compiler's incremental hashing (compiler.zig:105-108) both digest the unframed concatenation list ++ wild ++ allow. The compiler strips *. from a wildcard candidate (compiler.zig:130-133), so upstream a.example (list a.example\n, wild empty) and upstream *.a.example (list empty, wild a.example\n) hash the same bytes. diskBodiesMatch (manager.zig:1085) recomputes with the same function, so the refresh takes the unchanged-checksum branch and a list that switches an exact block to a wildcard block never takes effect. Ruling 3's entry-count argument rested on the digest distinguishing bodies; it does not, and the strikethrough above records that.
The fix: a 0x00 separator after each body
The digest becomes SHA-256(list ‖ 00 ‖ wild ‖ 00 ‖ allow ‖ 00) — one zero byte fed to the hasher after each of the three bodies, same order as today. Soundness: a compiled body holds only validated name bytes and \n; addCandidate rejects any byte ≥ 0x80 or control byte (compiler.zig:151-155), so 0x00 cannot occur in a body and the three boundaries are unambiguous. Two distinct (list, wild, allow) triples cannot produce one digest short of SHA-256 itself.
A separator, not a length prefix, because the compiler hashes while it emits and does not know a body's length up front; a trailing byte needs no pre-pass.
Both producers move together or every refresh republishes forever: compiler.compile feeds the byte after each emit call, and manager.bodyChecksum feeds it after each body slice. Export the separator as a pub const from compiler.zig and have bodyChecksum use it — two literal 0s in two files is how the next drift starts.
Consequences, accepted
- Every stored checksum changes once. Zero installs; a development data directory fails its startup checksum verification and the startup refresh pass re-downloads and repairs it (or delete the data directory — the ruling 1 stance).
- Checksum compatibility with pre-exception digests — milestone 21 ruling 3's "empty allow body reproduces the old digest" property — is dead, and its rationale comments go with it:
Result.checksum(compiler.zig:44-53), theHeaderdoc andbodyChecksumdoc (manager.zig:218-225, 1645-1648),SourceStats.checksum(sources_repo.zig), and the PLAN §3.8 sentence "keeps the digest it had when only two existed" (PLAN.md:101). All are rewritten to state the framed digest. The body order stays list, wild, allow.
The branch reads all five fresh
On the checksum-unchanged path, all five stat fields — domain_count, wildcard_count, exception_count, skipped_regex_count, skipped_unsupported_count — now read from compiled.result.counts; checksum keeps passing stored. With framing, fresh and stored entry counts are provably equal, so this is not a correctness requirement — it removes the per-field vouching argument from the code entirely, and any future digest weakness then degrades to consistent stats rather than a split between database and status table.
Regression tests (each watched failing with the framing reverted)
- Compiler: compiling
a.exampleand compiling*.a.exampleproduce different checksums. This is the collision itself. - Manager: publish a source whose body is
a.example; refresh it with upstream bytes*.a.example. Assert the unchanged-checksum branch is not taken: the on-disk.wildstripped body isa.example\n, the.listbody is empty, and the database row readsdomain_count = 0,wildcard_count = 1. - Agreement:
bodyChecksumover the three stripped on-disk bodies equalscompile's reported checksum for a fixture where all three bodies are non-empty (extend the existing agreement coverage if it exists; the empty-allow case no longer exercises the third frame).
Report the observed failure output for each, per the ruling 3 convention.