# Milestone 12: performance measurement pass + aarch64 test execution

Goal: close the two gaps the post-Phase-10 completeness sweep found — PLAN §18 has no
measurement infrastructure (deferred at specs/milestone-7.md:113 and never delivered),
and aarch64 tests are cross-built but never executed (PLAN.md:145 promised qemu).

## Rulings (binding)

1. **Bench harness = `tools/bench.zig` + `zig build bench`.** Not in src/ — src/ is
   the shipped product; tests.zig aggregates everything shippable and the bench must
   be a separate compilation anyway (one-file-one-module rule; matcher.zig and
   dns_cache.zig already belong to the test compilation). Module wiring per the fuzz
   pattern (build.zig:68-92): modules rooted at src/filter/matcher.zig and
   src/cache/dns_cache.zig — their relative import closures come along; no
   sqlite/mbedTLS linking needed. `if (b.args) |args| run.addArgs(args)`.

2. **What it measures** (PLAN §18 targets):
   - `filter`: `matcher.normalize` + `Snapshot.evaluate` per op against a Snapshot
     built from a generated, sorted `d{d:0>7}.example.com` list body (~1M lines,
     in-memory; DomainSet.max_count 4M and compiler.max_domains 2M leave headroom)
     plus a small wild body. Mixed case ratios: hit, miss, parent-walk. Target
     p95 < 1 ms.
   - `cache`: `buildKey` + `DnsCache.get` + `packet.setId` (the handler's hit path;
     TTL aging happens inside get) on a ~10k-entry cache prefilled with the fuzz
     corpus response. Mixed hit/miss. Target p95 < 5 ms.
   - `compile`: `compiler.compile` over a generated 1M-line hosts body — wall time,
     informational (no §18 target; no datapoint exists today).
   - Memory: `/proc/self/status` VmRSS (nothing in-repo wraps it; read it directly)
     plus the in-repo accounting (`Snapshot.memoryBytes`, `DnsCache.memoryBytes`).
     Target < 100 MB RSS with ~1M domains loaded.
   - Timing: `std.Io.Clock.awake.now(io)` / `durationTo` — std.time.Timer does not
     exist at 0.16. Percentiles: collect per-op nanos in a preallocated []u64, sort
     (std.mem.sort), report p50/p95/p99/max. No percentile helper exists; write it
     in the tool.

3. **Assertion policy.** Default run is informational (prints a table). `--assert`
   exits non-zero when a §18 target is exceeded — for the Pi 5 run, NOT for CI:
   required CI stays deterministic (AGENTS.md) and perf assertions on shared runners
   are flake generators. CI does not run the bench at all this milestone.

4. **Bench flags**: `--domains=N` (default 1_000_000), `--iters=N` (default 200_000),
   `--seed=N` (default fixed), `--assert`, subcommands `filter|cache|compile|all`
   (default all). Debug-mode runs print a one-line warning recommending
   `-Doptimize=ReleaseFast`.

5. **docs/performance.md**: the §18 targets table; measured numbers from THIS x86_64
   host (dated, hardware named honestly, marked as not-the-target-platform); the
   in-memory accounting numbers; the one-command Pi 5 recipe
   (`zig build bench -Doptimize=ReleaseFast -- --assert`); a sentence on why CI does
   not gate on perf. The Pi 5 rows stay "to be measured on hardware".

6. **aarch64 execution = second test artifact, `zig build test-aarch64 -fqemu`.**
   Per stdlib evidence: Run steps try qemu only when enable_qemu (the `-fqemu` CLI
   flag; a *Build field, no per-step override) and exec `qemu-aarch64` bare from
   PATH — so CI installs `qemu-user` (NOT qemu-user-static, which ships the
   `-static` name Zig will not find; binfmt is unnecessary). Static musl means no
   sysroot/--libc-runtimes. build.zig: resolve aarch64-linux-musl, addTest with the
   same wiring as the native tests artifact (sqlite/mbedtls helpers are already
   target-parameterized), set `.linkage = .static` on the artifact (TestOptions has
   no linkage field), `run.skip_foreign_checks = true`, leave
   failing_to_execute_foreign_is_an_error true so a missing qemu is loud. Fuzz
   artifacts excluded.

7. **qemu scope: plain suite only, blocking.** No `-Dintegration` under qemu: the
   integration tests are multithreaded loopback TLS with wall-clock budgets, and
   qemu-user's 5-20x slowdown makes them a flake source — required CI stays
   deterministic. The plain suite (the aggregator's tests without the 12
   fuzz-artifact tests: 1159 pass + 113 integration-gated skips; all pure
   DNS/filter/cache logic included)
   is the portable-correctness signal aarch64 needs. Recorded here as the
   engineering call closing PLAN.md:145's promise.

8. **CI additions**: one `test-aarch64` job (setup-zig, apt qemu-user,
   `zig build test-aarch64 -fqemu`); plus the missing §18 size assert — the cross
   job gains a second build WITHOUT `-Dweb-dist` (placeholder dist) and asserts the
   stripped copies < 10 MiB per arch (the < 15 MiB with-assets assert already
   exists). Job graph otherwise untouched.

9. **No src/ changes.** matcher/dns_cache/compiler public APIs are used as-is; if
   the bench needs something they do not expose, report — do not extend them.

## Sessions

V1 then V2 (both edit build.zig — sequenced, not parallel).

## Session V1: bench harness + performance doc

Owns tools/bench.zig, build.zig (the bench step only), docs/performance.md.
Rulings 1-5. Runs the bench on this host (ReleaseFast) and writes the measured
numbers into the doc.

### V1 As built

Delivered tools/bench.zig, the build.zig bench step, and docs/performance.md. One
deviation from ruling 1's wiring: Zig 0.16 rejects separate modules rooted at
matcher.zig and dns_cache.zig ("file exists in multiple modules" — their closures
share model.zig/types.zig; verified with a minimal repro). Instead a b.addWriteFiles
stage copies src/ plus a generated 7-line aggregator root (bench_core.zig) into one
`core` module — the same staging trick the web assets use; a build.zig comment
records why. The fuzz-corpus response is a byte-for-byte copy with provenance
comment (importing corpus.zig recreates the module conflict; the corpus documents
copies as house style). /proc/self/status reads use readerStreaming (procfs stats
size 0, readFileAlloc returns empty). Measured on the build host (i7-14700K,
ReleaseFast, defaults: 1M domains, 200k iters): filter p95 0.18 µs (target < 1 ms),
cache p95 0.14 µs (target < 5 ms), VmRSS 31.8 MiB with the 1M-domain snapshot
(target < 100 MiB), Snapshot.memoryBytes 28.0 MiB, compile 96 ms ≈ 10.4M lines/s
(informational). --assert exit paths verified both ways (a 4M-domain run exceeds
the 1M-scoped RSS target and exits 1; that was an exit-path exercise, not a target
miss). No src/ changes; matcher/dns_cache/compiler APIs sufficed. Cache suite fixes
entries at 10k (the handler-default Config.size); --domains shapes filter and
compile only. Both suites 0 failed after the build.zig edit.

## Session V2: aarch64 test execution + CI (after V1)

Owns build.zig (the test-aarch64 artifact/step only), .gitea/workflows/ci.yml
(the new job + the cross-job size-assert addition). Rulings 6-8. Runs
`zig build test-aarch64 -fqemu` locally if qemu-aarch64 is installable/present;
reports honestly if not.

### V2 As built

build.zig gained the test-aarch64 block after the bench block: aarch64-linux-musl
addTest with wiring identical to the native tests artifact, `.linkage = .static` on
the artifact, `run.skip_foreign_checks = true`, failing-to-execute left loud. Stdlib
mechanics re-verified (Run.zig:71/78/226, Build.zig:73, system.zig:111 — bare
`qemu-aarch64` from PATH). ci.yml gained the test-aarch64 job (apt qemu-user,
`zig build test-aarch64 -fqemu`) and the cross job builds a second no-dist pair to
`--prefix zig-out/nodist` (existing with-assets steps byte-identical) with a
< 10 MiB stripped assert; the new assert skips the static-linkage recheck (proven
on the same-config with-assets build). Local verification: real qemu execution —
1159 pass / 113 skip / 0 failed (native count minus the 12 excluded fuzz tests);
no-dist stripped sizes 5,880,336 (x86_64) and 5,212,640 (aarch64) bytes. No
-Dintegration under qemu per ruling 7.

## Review (Codex, as built)

Two rounds on one thread; round 2 returned "No findings."

Round 1 (2 important, 2 minor): bench.zig's explicit body.deinit left the earlier
errdefer armed on an undefined list (double-free on any later error) → clearAndFree
keeps the list valid. Ubuntu's qemu-user recommends qemu-user-binfmt, which apt
installs by default — binfmt registration would bypass the -fqemu path CI intends
to exercise → --no-install-recommends. performance.md's headroom claim overstated
the memory margin (3x, not orders of magnitude) → corrected. Ruling 7 quoted the
native aggregate test count instead of the aarch64 artifact's own (1159 + 113
skips, fuzz excluded) → corrected.

Final gates: plain 1171/1284 passed, 113 skipped (integration-gated), 0 failed;
integration 1280/1284, 4 skipped (live-network by design), 0 failed; qemu aarch64
run 1159 passed, 113 skipped, 0 failed; cross ReleaseSafe with dist 18/18; no-dist
stripped sizes 5.9/5.2 MB under the 10 MiB assert.

## Module layout (new)

tools/bench.zig, docs/performance.md.

## File ownership

V1 tools/bench.zig + docs/performance.md + build.zig(bench); V2 build.zig(aarch64) +
ci.yml. Orchestrator: spec sync, review, commit.

## Acceptance (milestone complete)

- [ ] `zig build bench -Doptimize=ReleaseFast` runs all three suites and prints
      p50/p95/p99/max + memory; `--assert` enforces the §18 targets.
- [ ] docs/performance.md holds dated x86_64 numbers + the Pi 5 recipe.
- [ ] `zig build test-aarch64 -fqemu` passes locally under qemu (or its
      unavailability is recorded with the exact CI-equivalent command).
- [ ] CI: test-aarch64 job green-by-construction (same command CI runs); cross job
      asserts < 10 MiB stripped without assets, < 15 MiB with (existing).
- [ ] Both existing suites 0 failed; no src/ changes.

## Anti-requirements

- No perf assertions in CI; no bench in the test step.
- No qemu integration suite; no binfmt setup; no qemu-user-static.
- No new pub API on matcher/dns_cache/compiler; no synthetic-load DNS server
  benchmark (the ≥100 qps target is end-to-end on the Pi — the operator recipe
  covers it via the real binary, not a harness).