nxdns/specs/milestone-12.md

Milestone 12: performance measurement pass + aarch64 test execution

Goal: close the two gaps the post-Phase-10 completeness sweep found — PLAN §18 has no measurement infrastructure (deferred at specs/milestone-7.md:113 and never delivered), and aarch64 tests are cross-built but never executed (PLAN.md:145 promised qemu).

Rulings (binding)

1. Bench harness = tools/bench.zig + zig build bench. Not in src/ — src/ is the shipped product; tests.zig aggregates everything shippable and the bench must be a separate compilation anyway (one-file-one-module rule; matcher.zig and dns_cache.zig already belong to the test compilation). Module wiring per the fuzz pattern (build.zig:68-92): modules rooted at src/filter/matcher.zig and src/cache/dns_cache.zig — their relative import closures come along; no sqlite/mbedTLS linking needed. if (b.args) |args| run.addArgs(args).

2. What it measures (PLAN §18 targets):

   • filter: matcher.normalize + Snapshot.evaluate per op against a Snapshot built from a generated, sorted d{d:0>7}.example.com list body (~1M lines, in-memory; DomainSet.max_count 4M and compiler.max_domains 2M leave headroom) plus a small wild body. Mixed case ratios: hit, miss, parent-walk. Target p95 < 1 ms.
   • cache: buildKey + DnsCache.get + packet.setId (the handler's hit path; TTL aging happens inside get) on a ~10k-entry cache prefilled with the fuzz corpus response. Mixed hit/miss. Target p95 < 5 ms.
   • compile: compiler.compile over a generated 1M-line hosts body — wall time, informational (no §18 target; no datapoint exists today).
   • Memory: /proc/self/status VmRSS (nothing in-repo wraps it; read it directly) plus the in-repo accounting (Snapshot.memoryBytes, DnsCache.memoryBytes). Target < 100 MB RSS with ~1M domains loaded.
   • Timing: std.Io.Clock.awake.now(io) / durationTo — std.time.Timer does not exist at 0.16. Percentiles: collect per-op nanos in a preallocated []u64, sort (std.mem.sort), report p50/p95/p99/max. No percentile helper exists; write it in the tool.

3. Assertion policy. Default run is informational (prints a table). --assert exits non-zero when a §18 target is exceeded — for the Pi 5 run, NOT for CI: required CI stays deterministic (AGENTS.md) and perf assertions on shared runners are flake generators. CI does not run the bench at all this milestone.

4. Bench flags: --domains=N (default 1_000_000), --iters=N (default 200_000), --seed=N (default fixed), --assert, subcommands filter|cache|compile|all (default all). Debug-mode runs print a one-line warning recommending -Doptimize=ReleaseFast.

5. docs/performance.md: the §18 targets table; measured numbers from THIS x86_64 host (dated, hardware named honestly, marked as not-the-target-platform); the in-memory accounting numbers; the one-command Pi 5 recipe (zig build bench -Doptimize=ReleaseFast -- --assert); a sentence on why CI does not gate on perf. The Pi 5 rows stay "to be measured on hardware".

6. aarch64 execution = second test artifact, zig build test-aarch64 -fqemu. Per stdlib evidence: Run steps try qemu only when enable_qemu (the -fqemu CLI flag; a *Build field, no per-step override) and exec qemu-aarch64 bare from PATH — so CI installs qemu-user (NOT qemu-user-static, which ships the -static name Zig will not find; binfmt is unnecessary). Static musl means no sysroot/--libc-runtimes. build.zig: resolve aarch64-linux-musl, addTest with the same wiring as the native tests artifact (sqlite/mbedtls helpers are already target-parameterized), set .linkage = .static on the artifact (TestOptions has no linkage field), run.skip_foreign_checks = true, leave failing_to_execute_foreign_is_an_error true so a missing qemu is loud. Fuzz artifacts excluded.

7. qemu scope: plain suite only, blocking. No -Dintegration under qemu: the integration tests are multithreaded loopback TLS with wall-clock budgets, and qemu-user's 5-20x slowdown makes them a flake source — required CI stays deterministic. The plain suite (the aggregator's tests without the 12 fuzz-artifact tests: 1159 pass + 113 integration-gated skips; all pure DNS/filter/cache logic included) is the portable-correctness signal aarch64 needs. Recorded here as the engineering call closing PLAN.md:145's promise.

8. CI additions: one test-aarch64 job (setup-zig, apt qemu-user, zig build test-aarch64 -fqemu); plus the missing §18 size assert — the cross job gains a second build WITHOUT -Dweb-dist (placeholder dist) and asserts the stripped copies < 10 MiB per arch (the < 15 MiB with-assets assert already exists). Job graph otherwise untouched.

9. No src/ changes. matcher/dns_cache/compiler public APIs are used as-is; if the bench needs something they do not expose, report — do not extend them.

Sessions

V1 then V2 (both edit build.zig — sequenced, not parallel).

Session V1: bench harness + performance doc

Owns tools/bench.zig, build.zig (the bench step only), docs/performance.md. Rulings 1-5. Runs the bench on this host (ReleaseFast) and writes the measured numbers into the doc.

V1 As built

Delivered tools/bench.zig, the build.zig bench step, and docs/performance.md. One deviation from ruling 1's wiring: Zig 0.16 rejects separate modules rooted at matcher.zig and dns_cache.zig ("file exists in multiple modules" — their closures share model.zig/types.zig; verified with a minimal repro). Instead a b.addWriteFiles stage copies src/ plus a generated 7-line aggregator root (bench_core.zig) into one core module — the same staging trick the web assets use; a build.zig comment records why. The fuzz-corpus response is a byte-for-byte copy with provenance comment (importing corpus.zig recreates the module conflict; the corpus documents copies as house style). /proc/self/status reads use readerStreaming (procfs stats size 0, readFileAlloc returns empty). Measured on the build host (i7-14700K, ReleaseFast, defaults: 1M domains, 200k iters): filter p95 0.18 µs (target < 1 ms), cache p95 0.14 µs (target < 5 ms), VmRSS 31.8 MiB with the 1M-domain snapshot (target < 100 MiB), Snapshot.memoryBytes 28.0 MiB, compile 96 ms ≈ 10.4M lines/s (informational). --assert exit paths verified both ways (a 4M-domain run exceeds the 1M-scoped RSS target and exits 1; that was an exit-path exercise, not a target miss). No src/ changes; matcher/dns_cache/compiler APIs sufficed. Cache suite fixes entries at 10k (the handler-default Config.size); --domains shapes filter and compile only. Both suites 0 failed after the build.zig edit.

Session V2: aarch64 test execution + CI (after V1)

Owns build.zig (the test-aarch64 artifact/step only), .gitea/workflows/ci.yml (the new job + the cross-job size-assert addition). Rulings 6-8. Runs zig build test-aarch64 -fqemu locally if qemu-aarch64 is installable/present; reports honestly if not.

V2 As built

build.zig gained the test-aarch64 block after the bench block: aarch64-linux-musl addTest with wiring identical to the native tests artifact, .linkage = .static on the artifact, run.skip_foreign_checks = true, failing-to-execute left loud. Stdlib mechanics re-verified (Run.zig:71/78/226, Build.zig:73, system.zig:111 — bare qemu-aarch64 from PATH). ci.yml gained the test-aarch64 job (apt qemu-user, zig build test-aarch64 -fqemu) and the cross job builds a second no-dist pair to --prefix zig-out/nodist (existing with-assets steps byte-identical) with a < 10 MiB stripped assert; the new assert skips the static-linkage recheck (proven on the same-config with-assets build). Local verification: real qemu execution — 1159 pass / 113 skip / 0 failed (native count minus the 12 excluded fuzz tests); no-dist stripped sizes 5,880,336 (x86_64) and 5,212,640 (aarch64) bytes. No -Dintegration under qemu per ruling 7.

Review (Codex, as built)

Two rounds on one thread; round 2 returned "No findings."

Round 1 (2 important, 2 minor): bench.zig's explicit body.deinit left the earlier errdefer armed on an undefined list (double-free on any later error) → clearAndFree keeps the list valid. Ubuntu's qemu-user recommends qemu-user-binfmt, which apt installs by default — binfmt registration would bypass the -fqemu path CI intends to exercise → --no-install-recommends. performance.md's headroom claim overstated the memory margin (3x, not orders of magnitude) → corrected. Ruling 7 quoted the native aggregate test count instead of the aarch64 artifact's own (1159 + 113 skips, fuzz excluded) → corrected.

Final gates: plain 1171/1284 passed, 113 skipped (integration-gated), 0 failed; integration 1280/1284, 4 skipped (live-network by design), 0 failed; qemu aarch64 run 1159 passed, 113 skipped, 0 failed; cross ReleaseSafe with dist 18/18; no-dist stripped sizes 5.9/5.2 MB under the 10 MiB assert.

Module layout (new)

tools/bench.zig, docs/performance.md.

File ownership

V1 tools/bench.zig + docs/performance.md + build.zig(bench); V2 build.zig(aarch64) + ci.yml. Orchestrator: spec sync, review, commit.

Acceptance (milestone complete)

• zig build bench -Doptimize=ReleaseFast runs all three suites and prints p50/p95/p99/max + memory; --assert enforces the §18 targets.
• docs/performance.md holds dated x86_64 numbers + the Pi 5 recipe.
• zig build test-aarch64 -fqemu passes locally under qemu (or its unavailability is recorded with the exact CI-equivalent command).
• CI: test-aarch64 job green-by-construction (same command CI runs); cross job asserts < 10 MiB stripped without assets, < 15 MiB with (existing).
• Both existing suites 0 failed; no src/ changes.

Anti-requirements

• No perf assertions in CI; no bench in the test step.
• No qemu integration suite; no binfmt setup; no qemu-user-static.
• No new pub API on matcher/dns_cache/compiler; no synthetic-load DNS server benchmark (the ≥100 qps target is end-to-end on the Pi — the operator recipe covers it via the real binary, not a harness).