filter: separate the compiled bodies in the checksum, a name moving between them was invisible
This commit is contained in:
+86
-2
@@ -118,9 +118,13 @@ the only thing a restart reads — rehydration never reparses headers.
|
||||
Both skip counters take `compiled.result.counts` on this path:
|
||||
`.skipped_regex_count = compiled.result.counts.skipped_regex` and
|
||||
`.skipped_unsupported_count = compiled.result.counts.skipped_unsupported`.
|
||||
`domain_count`, `wildcard_count` and `exception_count` keep reading from
|
||||
~~`domain_count`, `wildcard_count` and `exception_count` keep reading from
|
||||
`row` — they count written entries, so an unchanged checksum does mean an
|
||||
unchanged value for them.
|
||||
unchanged value for them.~~ **Corrected post-commit:** that claim was false.
|
||||
The unframed digest could not distinguish an entry in `.list` from the same
|
||||
entry in `.wild`, so an unchanged checksum did not vouch for the entry
|
||||
counts either. The addendum below frames the hash and makes all five stat
|
||||
fields read fresh on this path.
|
||||
|
||||
This needs a test the author has watched fail with the fix reverted: two
|
||||
compiles of bodies whose written entries are identical but whose skipped
|
||||
@@ -495,3 +499,83 @@ New files: none. Deleted surface: none.
|
||||
- No new columns beyond the one. See ruling 7 for what that leaves open and
|
||||
why — the reason is a scope decision, not a claim that the other counters
|
||||
are already visible.
|
||||
|
||||
## Addendum (post-`1bce81e`): the body checksum is framed
|
||||
|
||||
A filtering correctness bug found by the implementation review, predating this
|
||||
milestone. Fixed as a follow-up commit; this addendum is its design record —
|
||||
the user ruled it a follow-up, not a milestone 25.
|
||||
|
||||
### The defect
|
||||
|
||||
`bodyChecksum` (manager.zig:1660) and the compiler's incremental hashing
|
||||
(compiler.zig:105-108) both digest the unframed concatenation
|
||||
`list ++ wild ++ allow`. The compiler strips `*.` from a wildcard candidate
|
||||
(compiler.zig:130-133), so upstream `a.example` (list `a.example\n`, wild
|
||||
empty) and upstream `*.a.example` (list empty, wild `a.example\n`) hash the
|
||||
same bytes. `diskBodiesMatch` (manager.zig:1085) recomputes with the same
|
||||
function, so the refresh takes the unchanged-checksum branch and a list that
|
||||
switches an exact block to a wildcard block never takes effect. Ruling 3's
|
||||
entry-count argument rested on the digest distinguishing bodies; it does not,
|
||||
and the strikethrough above records that.
|
||||
|
||||
### The fix: a `0x00` separator after each body
|
||||
|
||||
The digest becomes `SHA-256(list ‖ 00 ‖ wild ‖ 00 ‖ allow ‖ 00)` — one zero
|
||||
byte fed to the hasher **after each of the three bodies**, same order as
|
||||
today. Soundness: a compiled body holds only validated name bytes and `\n`;
|
||||
`addCandidate` rejects any byte ≥ `0x80` or control byte
|
||||
(compiler.zig:151-155), so `0x00` cannot occur in a body and the three
|
||||
boundaries are unambiguous. Two distinct `(list, wild, allow)` triples cannot
|
||||
produce one digest short of SHA-256 itself.
|
||||
|
||||
A separator, not a length prefix, because the compiler hashes while it emits
|
||||
and does not know a body's length up front; a trailing byte needs no pre-pass.
|
||||
|
||||
Both producers move together or every refresh republishes forever:
|
||||
`compiler.compile` feeds the byte after each `emit` call, and
|
||||
`manager.bodyChecksum` feeds it after each body slice. Export the separator as
|
||||
a `pub const` from `compiler.zig` and have `bodyChecksum` use it — two literal
|
||||
`0`s in two files is how the next drift starts.
|
||||
|
||||
### Consequences, accepted
|
||||
|
||||
- Every stored checksum changes once. Zero installs; a development data
|
||||
directory fails its startup checksum verification and the startup refresh
|
||||
pass re-downloads and repairs it (or delete the data directory — the ruling
|
||||
1 stance).
|
||||
- Checksum compatibility with pre-exception digests — milestone 21 ruling 3's
|
||||
"empty allow body reproduces the old digest" property — is dead, and its
|
||||
rationale comments go with it: `Result.checksum` (compiler.zig:44-53), the
|
||||
`Header` doc and `bodyChecksum` doc (manager.zig:218-225, 1645-1648),
|
||||
`SourceStats.checksum` (sources_repo.zig), and the PLAN §3.8 sentence
|
||||
"keeps the digest it had when only two existed" (PLAN.md:101). All are
|
||||
rewritten to state the framed digest. The body order stays list, wild,
|
||||
allow.
|
||||
|
||||
### The branch reads all five fresh
|
||||
|
||||
On the checksum-unchanged path, all five stat fields —
|
||||
`domain_count`, `wildcard_count`, `exception_count`, `skipped_regex_count`,
|
||||
`skipped_unsupported_count` — now read from `compiled.result.counts`;
|
||||
`checksum` keeps passing `stored`. With framing, fresh and stored entry counts
|
||||
are provably equal, so this is not a correctness requirement — it removes the
|
||||
per-field vouching argument from the code entirely, and any future digest
|
||||
weakness then degrades to consistent stats rather than a split between
|
||||
database and status table.
|
||||
|
||||
### Regression tests (each watched failing with the framing reverted)
|
||||
|
||||
- [ ] Compiler: compiling `a.example` and compiling `*.a.example` produce
|
||||
**different** checksums. This is the collision itself.
|
||||
- [ ] Manager: publish a source whose body is `a.example`; refresh it with
|
||||
upstream bytes `*.a.example`. Assert the unchanged-checksum branch is
|
||||
**not** taken: the on-disk `.wild` stripped body is `a.example\n`, the
|
||||
`.list` body is empty, and the database row reads `domain_count = 0`,
|
||||
`wildcard_count = 1`.
|
||||
- [ ] Agreement: `bodyChecksum` over the three stripped on-disk bodies equals
|
||||
`compile`'s reported checksum for a fixture where **all three bodies are
|
||||
non-empty** (extend the existing agreement coverage if it exists; the
|
||||
empty-allow case no longer exercises the third frame).
|
||||
|
||||
Report the observed failure output for each, per the ruling 3 convention.
|
||||
|
||||
Reference in New Issue
Block a user