milestone 13: restructure docs to diataxis, tutorial, every command executed

2026-08-02 18:02:58 +02:00
parent 35f23240e7
commit 16c9de2414
26 changed files with 3627 additions and 901 deletions
@@ -0,0 +1,163 @@
+# Measure performance
+
+`tools/bench.zig` measures the three things nxdns can measure in-process:
+blocklist lookup latency, cache-hit latency, and blocklist compile throughput.
+Sustained query rate is not one of them — that one is end-to-end and needs a
+load generator pointed at a running server.
+
+The numbers this project treats as targets, and the numbers measured so far,
+are in the [performance reference](../reference/performance.md). Why those
+targets exist and why CI does not gate on them is in
+[performance and testing](../explanation/performance-and-testing.md).
+
+## Run the whole bench
+
+```sh
+zig build bench -Doptimize=ReleaseFast
+```
+
+That runs all three suites with the defaults: 1,000,000 domains, 200,000
+iterations per suite, seed `0x5eed`. It takes minutes, most of it generating and
+loading the million-domain list.
+
+`-Doptimize=ReleaseFast` is not optional if you want the numbers to mean
+anything. A Debug build says so before it prints:
+
+```
+warning: Debug build; run with -Doptimize=ReleaseFast for meaningful numbers
+```
+
+## Run one suite, smaller
+
+Everything after `--` goes to the harness. A suite name selects one of
+`filter`, `cache`, `compile` (the default is `all`), and `--domains` /
+`--iters` shrink the load:
+
+```sh
+zig build bench -Doptimize=ReleaseFast -- filter --domains=100000 --iters=20000
+```
+
+```
+nxdns bench  suite=filter  domains=100000  iters=20000  seed=0x5eed  optimize=ReleaseFast
+
+suite           ops     p50(us)     p95(us)     p99(us)     max(us)
+filter        20000        0.14        0.25        0.27        0.51
+  blocked 6670/20000, Snapshot.memoryBytes 3.0 MiB, VmRSS 5.4 MiB
+  target p95 < 1ms: PASS
+  target VmRSS < 100 MiB: PASS
+```
+
+A reduced run is good for checking the harness works and for a rough
+regression signal. It is not a result: the memory figure scales with
+`--domains`, so 100,000 domains says nothing about the 1,000,000-domain memory
+target.
+
+The other two suites:
+
+```sh
+zig build bench -Doptimize=ReleaseFast -- cache --iters=20000
+```
+
+```
+suite           ops     p50(us)     p95(us)     p99(us)     max(us)
+cache         20000        0.12        0.21        0.23        0.54
+  hits 10000/20000, DnsCache.memoryBytes 4.3 MiB, VmRSS 6.2 MiB
+  target p95 < 5ms: PASS
+```
+
+```sh
+zig build bench -Doptimize=ReleaseFast -- compile --domains=100000
+```
+
+```
+suite           ops     p50(us)     p95(us)     p99(us)     max(us)
+compile      100000   wall 11.512ms, 8686215 lines/s, 100000 domains kept (informational)
+```
+
+`--seed=N` changes the generated domains and the query order; the default is
+`0x5eed`, so two runs on the same machine are comparable. `--domains` caps at
+4,000,000, and the `compile` suite additionally refuses more than 2,000,000 —
+the compiler's own limit.
+
+An argument the harness does not recognise stops it before any measuring:
+
+```
+error: unknown argument 'nosuch'
+usage: zig build bench -Doptimize=ReleaseFast -- [filter|cache|compile|all] [--domains=N] [--iters=N] [--seed=N] [--assert]
+```
+
+## Read the output
+
+- `p50`/`p95`/`p99`/`max` are per-operation microseconds, nearest-rank over
+  every iteration. What one operation means differs per suite: for `filter` it
+  is normalising a name plus evaluating it against the snapshot; for `cache` it
+  is building the key, getting the entry and stamping the response id.
+- `blocked N/M` and `hits N/M` are sanity counters. The harness aborts if either
+  is zero — a suite that never hits its own path measures nothing.
+- Two memory figures appear on purpose. `Snapshot.memoryBytes` and
+  `DnsCache.memoryBytes` are the in-repo accounting of those structures; `VmRSS`
+  is what the kernel holds resident for the whole process, allocator slack and
+  code included. The truth is between them, and the memory target is judged on
+  `VmRSS`.
+- `target ...: PASS` / `FAIL` lines appear for the targets a suite covers. On a
+  plain run they are informational and the exit code stays 0.
+
+## Fail the run when a target is missed
+
+`--assert` turns those lines into an exit code — 1 when any target was
+exceeded, 0 otherwise. This is meant for an acceptance run on hardware you
+control, not for CI:
+
+```sh
+zig build bench -Doptimize=ReleaseFast -- --assert
+```
+
+The full-scale form is the one worth asserting on, because the memory target
+only means something at a million domains. On this development host the reduced
+form was used to check the flag itself:
+
+```sh
+zig build bench -Doptimize=ReleaseFast -- filter --domains=100000 --iters=20000 --assert
+```
+
+```
+filter        20000        0.14        0.26        0.27        2.42
+  blocked 6670/20000, Snapshot.memoryBytes 3.0 MiB, VmRSS 5.4 MiB
+  target p95 < 1ms: PASS
+  target VmRSS < 100 MiB: PASS
+```
+
+**Not verified on this host at full scale:** the plain
+`zig build bench -Doptimize=ReleaseFast -- --assert` above was not run during
+the writing of this page — the default run takes minutes. The reduced runs
+shown were all executed as written. The full-scale numbers already recorded for
+this host are in the [performance reference](../reference/performance.md).
+
+## Measure sustained query rate
+
+The bench harness cannot do this. Query rate is a property of the whole server
+— sockets, upstreams, the query log writer — so it has to be driven from
+outside, against the real binary, on the machine you care about.
+
+Start nxdns with real blocklists configured, then drive it from another host on
+the LAN with a DNS load generator such as `dnsperf`:
+
+```sh
+dnsperf -s 192.168.1.10 -p 53 -d queries.txt -c 20 -Q 200 -l 60
+```
+
+Read the client's own rate and the server's `/metrics` together: a load
+generator that reports 200 qps while the server counts fewer has lost queries
+somewhere, and that is the interesting number.
+
+**Not verified on this host:** `dnsperf` is not installed here and the target
+platform is a Raspberry Pi 5, not this development machine. The command above
+is the shape of the measurement, not a transcript.
+
+## Where to run it
+
+The target platform is a Raspberry Pi 5. Numbers from a development x86_64 box
+do not transfer — the Pi's Cortex-A76 is far slower — so a passing run here is
+evidence the harness works and a baseline for spotting regressions on the
+machine development happens on, and nothing more. Run `--assert` on the Pi,
+where the numbers mean something.