nxdns/specs/milestone-12.md

# Milestone 12: performance measurement pass + aarch64 test execution

Goal: close the two gaps the post-Phase-10 completeness sweep found — PLAN §18 has no measurement infrastructure (deferred at specs/milestone-7.md:113 and never delivered), and aarch64 tests are cross-built but never executed (PLAN.md:145 promised qemu).

## Rulings (binding)

1. **Bench harness = `tools/bench.zig` + `zig build bench`.** Not in src/ — src/ is the shipped product; tests.zig aggregates everything shippable and the bench must be a separate compilation anyway (one-file-one-module rule; matcher.zig and dns_cache.zig already belong to the test compilation). Module wiring per the fuzz pattern (build.zig:68-92): modules rooted at src/filter/matcher.zig and src/cache/dns_cache.zig — their relative import closures come along; no sqlite/mbedTLS linking needed. `if (b.args) |args| run.addArgs(args)`.

2. **What it measures** (PLAN §18 targets):
   - `filter`: `matcher.normalize` + `Snapshot.evaluate` per op against a Snapshot built from a generated, sorted `d{d:0>7}.example.com` list body (~1M lines, in-memory; DomainSet.max_count 4M and compiler.max_domains 2M leave headroom) plus a small wild body. Mixed case ratios: hit, miss, parent-walk. Target p95 < 1 ms.
   - `cache`: `buildKey` + `DnsCache.get` + `packet.setId` (the handler's hit path; TTL aging happens inside get) on a ~10k-entry cache prefilled with the fuzz corpus response. Mixed hit/miss. Target p95 < 5 ms.
   - `compile`: `compiler.compile` over a generated 1M-line hosts body — wall time, informational (no §18 target; no datapoint exists today).
   - Memory: `/proc/self/status` VmRSS (nothing in-repo wraps it; read it directly) plus the in-repo accounting (`Snapshot.memoryBytes`, `DnsCache.memoryBytes`). Target < 100 MB RSS with ~1M domains loaded.
   - Timing: `std.Io.Clock.awake.now(io)` / `durationTo` — std.time.Timer does not exist at 0.16. Percentiles: collect per-op nanos in a preallocated []u64, sort (std.mem.sort), report p50/p95/p99/max. No percentile helper exists; write it in the tool.

3. **Assertion policy.** Default run is informational (prints a table). `--assert` exits non-zero when a §18 target is exceeded — for the Pi 5 run, NOT for CI: required CI stays deterministic (AGENTS.md) and perf assertions on shared runners are flake generators. CI does not run the bench at all this milestone.

4. **Bench flags**: `--domains=N` (default 1_000_000), `--iters=N` (default 200_000), `--seed=N` (default fixed), `--assert`, subcommands `filter|cache|compile|all` (default all). Debug-mode runs print a one-line warning recommending `-Doptimize=ReleaseFast`.

5. **docs/performance.md**: the §18 targets table; measured numbers from THIS x86_64 host (dated, hardware named honestly, marked as not-the-target-platform); the in-memory accounting numbers; the one-command Pi 5 recipe (`zig build bench -Doptimize=ReleaseFast -- --assert`); a sentence on why CI does not gate on perf. The Pi 5 rows stay "to be measured on hardware".

6. **aarch64 execution = second test artifact, `zig build test-aarch64 -fqemu`.** Per stdlib evidence: Run steps try qemu only when enable_qemu (the `-fqemu` CLI flag; a *Build field, no per-step override) and exec `qemu-aarch64` bare from PATH — so CI installs `qemu-user` (NOT qemu-user-static, which ships the `-static` name Zig will not find; binfmt is unnecessary). Static musl means no sysroot/--libc-runtimes. build.zig: resolve aarch64-linux-musl, addTest with the same wiring as the native tests artifact (sqlite/mbedtls helpers are already target-parameterized), set `.linkage = .static` on the artifact (TestOptions has no linkage field), `run.skip_foreign_checks = true`, leave failing_to_execute_foreign_is_an_error true so a missing qemu is loud. Fuzz artifacts excluded.

7. **qemu scope: plain suite only, blocking.** No `-Dintegration` under qemu: the integration tests are multithreaded loopback TLS with wall-clock budgets, and qemu-user's 5-20x slowdown makes them a flake source — required CI stays deterministic. The plain suite (the aggregator's tests without the 12 fuzz-artifact tests: 1159 pass + 113 integration-gated skips; all pure DNS/filter/cache logic included) is the portable-correctness signal aarch64 needs. Recorded here as the engineering call closing PLAN.md:145's promise.

8. **CI additions**: one `test-aarch64` job (setup-zig, apt qemu-user, `zig build test-aarch64 -fqemu`); plus the missing §18 size assert — the cross job gains a second build WITHOUT `-Dweb-dist` (placeholder dist) and asserts the stripped copies < 10 MiB per arch (the < 15 MiB with-assets assert already exists). Job graph otherwise untouched.

9. **No src/ changes.** matcher/dns_cache/compiler public APIs are used as-is; if the bench needs something they do not expose, report — do not extend them.

## Sessions

V1 then V2 (both edit build.zig — sequenced, not parallel).

## Session V1: bench harness + performance doc

Owns tools/bench.zig, build.zig (the bench step only), docs/performance.md. Rulings 1-5. Runs the bench on this host (ReleaseFast) and writes the measured numbers into the doc.

### V1 As built

Delivered tools/bench.zig, the build.zig bench step, and docs/performance.md. One deviation from ruling 1's wiring: Zig 0.16 rejects separate modules rooted at matcher.zig and dns_cache.zig ("file exists in multiple modules" — their closures share model.zig/types.zig; verified with a minimal repro). Instead a b.addWriteFiles stage copies src/ plus a generated 7-line aggregator root (bench_core.zig) into one `core` module — the same staging trick the web assets use; a build.zig comment records why. The fuzz-corpus response is a byte-for-byte copy with provenance comment (importing corpus.zig recreates the module conflict; the corpus documents copies as house style). /proc/self/status reads use readerStreaming (procfs stats size 0, readFileAlloc returns empty). Measured on the build host (i7-14700K, ReleaseFast, defaults: 1M domains, 200k iters): filter p95 0.18 µs (target < 1 ms), cache p95 0.14 µs (target < 5 ms), VmRSS 31.8 MiB with the 1M-domain snapshot (target < 100 MiB), Snapshot.memoryBytes 28.0 MiB, compile 96 ms ≈ 10.4M lines/s (informational). --assert exit paths verified both ways (a 4M-domain run exceeds the 1M-scoped RSS target and exits 1; that was an exit-path exercise, not a target miss). No src/ changes; matcher/dns_cache/compiler APIs sufficed. Cache suite fixes entries at 10k (the handler-default Config.size); --domains shapes filter and compile only. Both suites 0 failed after the build.zig edit.

## Session V2: aarch64 test execution + CI (after V1)

Owns build.zig (the test-aarch64 artifact/step only), .gitea/workflows/ci.yml (the new job + the cross-job size-assert addition). Rulings 6-8. Runs `zig build test-aarch64 -fqemu` locally if qemu-aarch64 is installable/present; reports honestly if not.

### V2 As built

build.zig gained the test-aarch64 block after the bench block: aarch64-linux-musl addTest with wiring identical to the native tests artifact, `.linkage = .static` on the artifact, `run.skip_foreign_checks = true`, failing-to-execute left loud. Stdlib mechanics re-verified (Run.zig:71/78/226, Build.zig:73, system.zig:111 — bare `qemu-aarch64` from PATH). ci.yml gained the test-aarch64 job (apt qemu-user, `zig build test-aarch64 -fqemu`) and the cross job builds a second no-dist pair to `--prefix zig-out/nodist` (existing with-assets steps byte-identical) with a
< 10 MiB stripped assert; the new assert skips the static-linkage recheck (proven
on the same-config with-assets build). Local verification: real qemu execution — 1159 pass / 113 skip / 0 failed (native count minus the 12 excluded fuzz tests); no-dist stripped sizes 5,880,336 (x86_64) and 5,212,640 (aarch64) bytes. No -Dintegration under qemu per ruling 7.

## Review (Codex, as built)

Two rounds on one thread; round 2 returned "No findings."

Round 1 (2 important, 2 minor): bench.zig's explicit body.deinit left the earlier errdefer armed on an undefined list (double-free on any later error) → clearAndFree keeps the list valid. Ubuntu's qemu-user recommends qemu-user-binfmt, which apt installs by default — binfmt registration would bypass the -fqemu path CI intends to exercise → --no-install-recommends. performance.md's headroom claim overstated the memory margin (3x, not orders of magnitude) → corrected. Ruling 7 quoted the native aggregate test count instead of the aarch64 artifact's own (1159 + 113 skips, fuzz excluded) → corrected.

Final gates: plain 1171/1284 passed, 113 skipped (integration-gated), 0 failed; integration 1280/1284, 4 skipped (live-network by design), 0 failed; qemu aarch64 run 1159 passed, 113 skipped, 0 failed; cross ReleaseSafe with dist 18/18; no-dist stripped sizes 5.9/5.2 MB under the 10 MiB assert.

## Module layout (new)

tools/bench.zig, docs/performance.md.

## File ownership

V1 tools/bench.zig + docs/performance.md + build.zig(bench); V2 build.zig(aarch64) + ci.yml. Orchestrator: spec sync, review, commit.

## Acceptance (milestone complete)

- [ ] `zig build bench -Doptimize=ReleaseFast` runs all three suites and prints
      p50/p95/p99/max + memory; `--assert` enforces the §18 targets.
- [ ] docs/performance.md holds dated x86_64 numbers + the Pi 5 recipe.
- [ ] `zig build test-aarch64 -fqemu` passes locally under qemu (or its
      unavailability is recorded with the exact CI-equivalent command).
- [ ] CI: test-aarch64 job green-by-construction (same command CI runs); cross job
      asserts < 10 MiB stripped without assets, < 15 MiB with (existing).
- [ ] Both existing suites 0 failed; no src/ changes.

## Anti-requirements

- No perf assertions in CI; no bench in the test step.
- No qemu integration suite; no binfmt setup; no qemu-user-static.
- No new pub API on matcher/dns_cache/compiler; no synthetic-load DNS server benchmark (the ≥100 qps target is end-to-end on the Pi — the operator recipe covers it via the real binary, not a harness).