# Measure performance

`tools/bench.zig` measures the three things nxdns can measure in-process:
blocklist lookup latency, cache-hit latency, and blocklist compile throughput.
Sustained query rate is not one of them — that one is end-to-end and needs a
load generator pointed at a running server.

The numbers this project treats as targets, and the numbers measured so far,
are in the [performance reference](../reference/performance.md). Why those
targets exist and why CI does not gate on them is in
[performance and testing](../explanation/performance-and-testing.md).

## Run the whole bench

```sh
zig build bench -Doptimize=ReleaseFast
```

That runs all three suites with the defaults: 1,000,000 domains, 200,000
iterations per suite, seed `0x5eed`. It takes minutes, most of it generating and
loading the million-domain list.

`-Doptimize=ReleaseFast` is not optional if you want the numbers to mean
anything. A Debug build says so before it prints:

```
warning: Debug build; run with -Doptimize=ReleaseFast for meaningful numbers
```

## Run one suite, smaller

Everything after `--` goes to the harness. A suite name selects one of
`filter`, `cache`, `compile` (the default is `all`), and `--domains` /
`--iters` shrink the load:

```sh
zig build bench -Doptimize=ReleaseFast -- filter --domains=100000 --iters=20000
```

```
nxdns bench  suite=filter  domains=100000  iters=20000  seed=0x5eed  optimize=ReleaseFast

suite           ops     p50(us)     p95(us)     p99(us)     max(us)
filter        20000        0.14        0.25        0.27        0.51
  blocked 6670/20000, Snapshot.memoryBytes 3.0 MiB, VmRSS 5.4 MiB
  target p95 < 1ms: PASS
  target VmRSS < 100 MiB: PASS
```

A reduced run is good for checking the harness works and for a rough
regression signal. It is not a result: the memory figure scales with
`--domains`, so 100,000 domains says nothing about the 1,000,000-domain memory
target.

The other two suites:

```sh
zig build bench -Doptimize=ReleaseFast -- cache --iters=20000
```

```
suite           ops     p50(us)     p95(us)     p99(us)     max(us)
cache         20000        0.12        0.21        0.23        0.54
  hits 10000/20000, DnsCache.memoryBytes 4.3 MiB, VmRSS 6.2 MiB
  target p95 < 5ms: PASS
```

```sh
zig build bench -Doptimize=ReleaseFast -- compile --domains=100000
```

```
suite           ops     p50(us)     p95(us)     p99(us)     max(us)
compile      100000   wall 11.512ms, 8686215 lines/s, 100000 domains kept (informational)
```

`--seed=N` changes the generated domains and the query order; the default is
`0x5eed`, so two runs on the same machine are comparable. `--domains` caps at
4,000,000, and the `compile` suite additionally refuses more than 2,000,000 —
the compiler's own limit.

An argument the harness does not recognise stops it before any measuring:

```
error: unknown argument 'nosuch'
usage: zig build bench -Doptimize=ReleaseFast -- [filter|cache|compile|all] [--domains=N] [--iters=N] [--seed=N] [--assert]
```

## Read the output

- `p50`/`p95`/`p99`/`max` are per-operation microseconds, nearest-rank over
  every iteration. What one operation means differs per suite: for `filter` it
  is normalising a name plus evaluating it against the snapshot; for `cache` it
  is building the key, getting the entry and stamping the response id.
- `blocked N/M` and `hits N/M` are sanity counters. The harness aborts if either
  is zero — a suite that never hits its own path measures nothing.
- Two memory figures appear on purpose. `Snapshot.memoryBytes` and
  `DnsCache.memoryBytes` are the in-repo accounting of those structures; `VmRSS`
  is what the kernel holds resident for the whole process, allocator slack and
  code included. The truth is between them, and the memory target is judged on
  `VmRSS`.
- `target ...: PASS` / `FAIL` lines appear for the targets a suite covers. On a
  plain run they are informational and the exit code stays 0.

## Fail the run when a target is missed

`--assert` turns those lines into an exit code — 1 when any target was
exceeded, 0 otherwise. This is meant for an acceptance run on hardware you
control, not for CI:

```sh
zig build bench -Doptimize=ReleaseFast -- --assert
```

The full-scale form is the one worth asserting on, because the memory target
only means something at a million domains. On this development host the reduced
form was used to check the flag itself:

```sh
zig build bench -Doptimize=ReleaseFast -- filter --domains=100000 --iters=20000 --assert
```

```
filter        20000        0.14        0.26        0.27        2.42
  blocked 6670/20000, Snapshot.memoryBytes 3.0 MiB, VmRSS 5.4 MiB
  target p95 < 1ms: PASS
  target VmRSS < 100 MiB: PASS
```

**Not verified on this host at full scale:** the plain
`zig build bench -Doptimize=ReleaseFast -- --assert` above was not run during
the writing of this page — the default run takes minutes. The reduced runs
shown were all executed as written. The full-scale numbers already recorded for
this host are in the [performance reference](../reference/performance.md).

## Measure sustained query rate

The bench harness cannot do this. Query rate is a property of the whole server
— sockets, upstreams, the query log writer — so it has to be driven from
outside, against the real binary, on the machine you care about.

Start nxdns with real blocklists configured, then drive it from another host on
the LAN with a DNS load generator such as `dnsperf`:

```sh
dnsperf -s 192.168.1.10 -p 53 -d queries.txt -c 20 -Q 200 -l 60
```

Read the client's own rate and the server's `/metrics` together: a load
generator that reports 200 qps while the server counts fewer has lost queries
somewhere, and that is the interesting number.

**Not verified on this host:** `dnsperf` is not installed here and the target
platform is a Raspberry Pi 5, not this development machine. The command above
is the shape of the measurement, not a transcript.

## Where to run it

The target platform is a Raspberry Pi 5. Numbers from a development x86_64 box
do not transfer — the Pi's Cortex-A76 is far slower — so a passing run here is
evidence the harness works and a baseline for spotting regressions on the
machine development happens on, and nothing more. Run `--assert` on the Pi,
where the numbers mean something.
