nxdns/specs/milestone-15.md

Milestone 15: make a green run mean a real pass

Goal: repair the verification pipeline — CI that watches the branch we push to, a test run whose output contains no false failure text, guards that make the test list and the embedded frontend complete by construction, and coverage for the seams the audit found unguarded. This is phase 1 of TECH_DEBT.md; nothing in phases 2-5 can be validated until this lands.

Origin: TECH_DEBT.md Theme 1 (lines 17-52), audited at 1ff727f, every finding adversarially verified.

Sequencing (binding)

This milestone lands before milestone 14 is implemented. Ruling 1 pulls the ci.yml branch switch forward from milestone-14 §7; when milestone 14's S4 later restructures the workflows into gates.yml/release.yml, it rebases on what this milestone leaves behind. One more overlap is binding: milestone 14's S1 rewrites build.zig (deletes cross, adds dist/verify-dist) — it must rebase on and preserve the configure-time machinery this milestone adds there (ruling 4's import guard, ruling 5's stamp check, ruling 6b's staged core module). Milestone 14's spec stands as written otherwise.

Rulings (binding)

1. CI watches master

.gitea/workflows/ci.yml:1-11 triggers on push/pull_request for main. Day-to-day pushes go to origin/master (branch.master.merge confirms), so milestone 13+ never ran through any gate. Change both branches: lists to [master]. Nothing else in the file changes — the gates.yml restructure is milestone-14 work.

Orchestrator, after merge: delete origin/main, set the Gitea default branch to master, push, and confirm a run starts. (Milestone-14 §7 already rules master canonical; this executes the branch part now.)

2. The live-network suite runs on a schedule

.gitea/workflows/live-tls.yml runs on workflow_dispatch only. Both failure modes it exists to catch have each happened once (35f2324; the milestone-4 cert drift). Add:

on:
  workflow_dispatch:
  schedule:
    - cron: "0 5 * * 1"

Weekly, Monday 05:00 UTC. It stays non-blocking — this does not conflict with the milestone-1 ruling that live tests never gate a push. Everything else in the file is untouched.

3. The teardown abort is root-caused, or documented where it lives

Reproduction, verified at HEAD: zig build test exits 0 but prints as its last line failed command: .../test ... --listen=-. The same binary exits 0 in stdio mode (1242 passed; 115 skipped; 0 failed) and exits 134 (SIGABRT) under --listen=-. The abort happens after all tests pass, in teardown, only under zig's IPC runner. No comment anywhere in the repository records this; the knowledge lives in one contributor's memory and TECH_DEBT.md:24-25.

The cost is not the abort — it is that the one string that should mean failure prints on every success, and everyone learns to ignore failure text.

The session must attempt a root cause, in this order:

1. Reproduce: run the cached test binary directly with and without --listen=-. Capture the abort backtrace (ulimit -c / gdb / the zig --debug-... runner flags — whatever this host offers).
2. Bisect the mbedTLS linkage: addMbedtlsThreadingMacros (build.zig), the shim (src/platform/mbedtls_shim.c), and the two extern fn link checks in src/tests.zig:121-123 are the suspects. A minimal reproducer (empty test file + mbedTLS link + --listen) decides whether the abort is ours or an upstream zig 0.16 runner defect.
3. Outcome A — the cause is ours: fix it. Acceptance is a zig build test run whose output contains no failed command line.
4. Outcome B — the cause is upstream: do not patch around it. Document it in two places: a comment on the test wiring in build.zig (at the b.addTest block, lines 48-69) stating the exact mechanism and the upstream reference, and a paragraph in AGENTS.md stating that failed command + exit 0 after a full pass is this known artifact and that any other failure text is real.

Either way: no check is loosened, no output is filtered, and the spec is updated afterward to record which outcome held (per the spec-update rule).

4. The test import list gets a completeness guard

src/tests.zig:3-119 is a hand-maintained comptime block of 115 _ = @import("..."); lines. Zig collects tests only from the root module; a forgotten import silently drops a file's tests (verified empirically). The list stays hand-written — generation via a staged copy of src/ would point diagnostics at cache paths — but it becomes complete by construction:

In build.zig, at configure time, walk src/ recursively. For every *.zig file except src/tests.zig itself, require that src/tests.zig contains a line which, after whitespace trim, is exactly _ = @import("<path>"); where <path> is the file's path relative to src/. A line match, not a substring search: a commented-out import (// _ = @import(...)) trims to a line that starts with // and does not match, and a path that is a prefix of another (db.zig vs db2.zig) cannot false-match because the full line is compared. Duplicate matching lines are an error too (they hide a botched merge). On the first missing file, fail the configure with an error that names it:

src/tests.zig is missing `_ = @import("filter/new_module.zig");`

No allowlist. A file with no tests still gets imported — the import is free, and the rule stays exceptionless. The walk runs on the host in every invocation, including the aarch64 configure.

5. web/dist gets a freshness stamp

A stale web/dist has already shipped a crashing settings page once (docs/explanation/performance-and-testing.md:154-163 admits the hole). Mechanism — one implementation, in JavaScript, because the frontend CI job has Node and no Zig:

• New file web/scripts/stamp-dist.mjs. Two modes:
  • default (write): hash the input set, write the hex digest to web/dist/.src-hash.
  • --check: recompute, compare with web/dist/.src-hash, exit 1 with web/dist is stale: rebuild the frontend (npm run build) on mismatch or missing stamp.
• The script resolves every path (input files and web/dist/.src-hash) from its own location via import.meta.url, never from process.cwd(): write mode runs from web/ (npm script) and check mode runs from the repo root (build.zig system command), and both must hash the same set. The acceptance run exercises both working directories.
• Input set, sorted by path, SHA-256 over path ++ "\x00" ++ contents ++ "\x00" per file: every file under web/src/ and web/public/, plus web/index.html, web/package.json, web/package-lock.json, web/vite.config.ts, web/tsconfig.json, web/tsconfig.app.json, web/tsconfig.node.json. Node's crypto module; no new dependency.
• web/package.json: "build" becomes "vite build && node scripts/stamp-dist.mjs".
• build.zig: when the resolved -Dweb-dist value is exactly web/dist, the asset pipeline (webAssetsIndex) gains a dependency on b.addSystemCommand(&.{ "node", "web/scripts/stamp-dist.mjs", "--check" }). The default web/dist-placeholder path and any other explicit path skip the check; the web-dist option description says so.

6. Three new fuzz surfaces, corpus replay only

All targets follow the existing registration shape (one test "fuzz ..." + one fn target(ctx, smith: *Smith)), attach to test_step, and run as corpus replay under plain zig build test — once per corpus entry plus once on empty input. CI never passes --fuzz (blocked by upstream zig 0.16 defects; the audit confirmed corpus replay is the sanctioned CI smoke).

6a. stripEcs — the only attacker-facing packet-rewriting entry point, absent from the fuzz targets. Add a target to tests/fuzz/dns_fuzz.zig. Derive input exactly as parseTarget (lines 57-89) already does: it ends with a validated Packet and its OptRecord, which are stripEcs's second and third parameters. Respect the three entry assertions (src/dns/edns.zig:196-204): pass the same bytes the packet was parsed from, and a separate stack out buffer — an assertion trip from a violated precondition is not a finding. Assert on .rewritten output: it re-parses, findOptRecord + parseOpt succeed, no ECS option (code 8) remains, and header counts survive. No build.zig change — the module exists.

6b. compiler.compile — the streaming takeDelimiter/StreamTooLong/discard loop (src/filter/compiler.zig:64-78) is never fuzzed, and its error.EndOfStream-during-discard arm (line 73) has no test at all. compiler.zig imports ../dns/, so a module rooted under src/filter/ fails with ImportOutsideModulePath. Use the staged-copy aggregator pattern from bench_core (build.zig:112-135) — bench_core already exposes compiler at line 124; reusing the same staged tree is allowed. New file tests/fuzz/compiler_fuzz.zig, module import name core. The target feeds compile() via std.Io.Reader.fixed over smith bytes with a deliberately small reader buffer, into discarding writers, and asserts it returns without panic and that counts are internally consistent. Corpus (inline, sliceInput idiom): a line longer than max_line_len (4096) with and without a trailing \n, so both the discard path and the EndOfStream-during-discard arm replay. Add a plain unit test for the line-73 arm alongside the existing compiler tests.

6c. http_util — the third untrusted-byte family, unfuzzed. src/web/http_util.zig imports only std, so the fuzz module roots at a new file tests/fuzz/http_util_fuzz.zig with addImport("http_util", <module rooted at src/web/http_util.zig>) — no aggregator. Targets: parsePath, decodeInPlace (both PlusRule values), queryValue. Invariants to assert, both already stated in the file: split-before-decode (a decoded segment never gains a /), and decode-only-shrinks (result length ≤ input length; result is a prefix-aliased slice of the buffer). dnsParam/decodeDnsValue stay private in doh_server.zig and stay unfuzzed — recorded, out of scope.

7. The multi-read fetch path gets a real fixture route

The exact seam of 35k-killer 35f2324 is still never driven end to end: every fixture reply is one request.respond of a ~130-byte body (src/filter/filter_integration_test.zig:414), and the post-fix unit tests pre-buffer readers. Two thresholds matter and they are different: a body over 16 KiB (fetcher.min_transfer_buf, fetcher.zig:24) forces the fetcher's pumpBody loop to iterate; a body over the fixture's 8192-byte write buffer forces the fixture to flush in parts.

• Add a route chunked to the Route enum (line 354). Its respond arm uses respondStreaming, writes a well-formed blocklist body of at least 24 KiB in at least three writes with an explicit flush() between each, then ends the stream. keep_alive = false, like every other arm (the comment at lines 409-411 is load-bearing).
• Add one -Dintegration test that drives the full refreshOnce path through this route and asserts the parsed domain counts.
• Prove it can fail: during development, reintroduce the readSliceShort(self.transfer_buf) aliasing pattern locally and confirm the new test dies where the old suite stayed green. Record the proof in the session notes; do not commit the revert.

8. The rotation failure paths get tests

src/platform/logging.zig codifies an exactly-one-sink-error contract across four functions (stated at lines 455-460) and its own test section admits the rotation failure paths are uncovered (lines 604-612). Injection seam:

var rotate_fault: enum { none, fail_delete, fail_rename } = .none;

file-private, read by deleteLocked (line 590) and renameLocked (line 597) as their first statement (if (rotate_fault == .fail_delete) return error.RotateFailed;), compiled only under @import("builtin").is_test. Two new tests, each following the established shape of the failed-open test at lines 883-909 (save and restore state.*, hold std.debug.lockStderr):

• fail_delete: prepareFileLocked returns false, sink_errors rose by exactly one, rotate_pending stays set, the file stays closed.
• fail_rename: same assertions through the rename step.

9. The CLI drift guard derives its needles

src/docs_drift_test.zig:51 hardcodes the subcommand list; its two sibling guards derive theirs (routes.table, model.toSettings). The source of truth today is three parallel copies: the if (eql(...)) parse chain (cli.zig:91-102), the main.zig:69-78 dispatch switch, and usage_text (cli.zig:334). Note the spelling trap: tags are export_/import_, argv strings are export/import.

• Add to cli.zig:

  pub const CommandName = struct { name: []const u8, tag: std.meta.Tag(Command) };
  pub const command_names = [_]CommandName{
      .{ .name = "run", .tag = .run },
      .{ .name = "check", .tag = .check },
      .{ .name = "export", .tag = .export_ },
      .{ .name = "import", .tag = .import_ },
      .{ .name = "version", .tag = .version },
      .{ .name = "help", .tag = .help },
  };
  

  plus a comptime assert that command_names.len == @typeInfo(Command).@"union".fields.len, so a new tag without a table entry fails the compile.

• parseArgs matches the command word by iterating command_names (the per-command argument parsing that follows stays as-is).

• docs_drift_test.zig iterates cli.command_names for its ## {s}`` needles; the hardcoded list is deleted.

• New test in cli.zig: every command_names[i].name appears in usage_text.

• main.zig's dispatch switch stays — it is exhaustive over the union and the compiler already guards it.

Sessions

S1-S4 run in parallel; no two sessions write the same file. The S1/S3 interface is fixed here so neither blocks: S3 writes the fuzz source files named in ruling 6; S1 wires them in build.zig with the module shapes named there (6b gets core via the staged-copy pattern; 6c gets http_util rooted at src/web/http_util.zig); both artifacts attach to test_step exactly like fuzz_tests at build.zig:92.

Session S1: build system and stamp

Owns build.zig, AGENTS.md, web/scripts/stamp-dist.mjs, web/package.json. Rulings 3, 4, 5, and the wiring half of 6.

Session S2: workflows

Owns .gitea/workflows/ci.yml, .gitea/workflows/live-tls.yml. Rulings 1 (file edit only), 2.

Session S3: fuzz targets and fixture

Owns tests/fuzz/dns_fuzz.zig, tests/fuzz/compiler_fuzz.zig (new), tests/fuzz/http_util_fuzz.zig (new), src/filter/filter_integration_test.zig, and the compiler unit-test addition in src/filter/compiler.zig. Rulings 6 (target half), 7.

Session S4: guards and seams

Owns src/cli.zig, src/docs_drift_test.zig, src/platform/logging.zig. Rulings 8, 9.

Orchestrator

Ruling 1's remote operations (delete origin/main, flip the Gitea default, confirm a run starts), the ruling-3 outcome recorded back into this spec, and striking the closed Theme-1 findings in TECH_DEBT.md.

Module layout

New files:

• web/scripts/stamp-dist.mjs — dist freshness stamp, write and --check modes.
• tests/fuzz/compiler_fuzz.zig — compiler streaming-loop target.
• tests/fuzz/http_util_fuzz.zig — HTTP parser targets.

Acceptance (milestone complete)

• ci.yml triggers on master for push and pull_request; a push to origin/master starts a CI run; origin/main is deleted; the Gitea default branch is master.
• live-tls.yml keeps workflow_dispatch and gains the weekly cron.
• Ruling 3 resolved: either zig build test prints no failed command line, or the build.zig comment and the AGENTS.md paragraph exist and name the upstream cause. The outcome is recorded in this spec.
• The import guard was proven able to fail: a temporary src/guard_probe.zig containing one test, not imported, makes zig build test fail at configure naming the file; after adding the import it runs; the probe is then deleted.
• npm run build writes web/dist/.src-hash. Appending a byte to a file under web/src/ (restored after the probe — the digest hashes paths and contents, so a bare touch cannot make it stale) then running zig build -Dweb-dist=web/dist from the repo root fails with the stale message; rebuilding the frontend clears it. (The probe is the default install step, not cross — milestone 14 deletes cross, and this check must stay valid across that rebase.) Plain zig build test (placeholder path) is unaffected.
• The three fuzz surfaces run under zig build test as corpus replay: the stripEcs target with its re-parse/no-ECS assertions, the compiler target with the long-line corpus entries, the http_util targets with both invariants. The line-73 EndOfStream arm has a unit test.
• The chunked fixture route writes ≥ 24 KiB in ≥ 3 flushed parts; the new integration test passes; the session notes record the proven-able-to-fail run against the 35f2324 aliasing pattern.
• The two rotation-failure tests pass, each asserting exactly one sink_error per the lines-455-460 contract.
• docs_drift_test.zig contains no hardcoded subcommand list; cli.command_names exists with the comptime length assert; the usage-text test passes.
• Full suite green (-Dintegration), test count strictly above 1242, and the aarch64 suite still builds and runs under qemu.

Anti-requirements

• No gates.yml, no release.yml, no dist/verify-dist steps — that is milestone 14, which rebases on this milestone.
• No --fuzz anywhere in CI.
• No fuzzing or relocation of dnsParam/decodeDnsValue.
• No generated tests.zig; the hand list stays, the guard makes it complete.
• No frontend changes beyond web/scripts/stamp-dist.mjs and the one-line build script edit.
• No filtering, wrapping, or suppression of test-runner output to hide the failed command line.
• No new dependencies, Zig or npm.