# Measure performance

`tools/bench.zig` measures the three things nxdns can measure in-process: blocklist lookup latency, cache-hit latency, and blocklist compile throughput. Sustained query rate is not one of them — that one is end-to-end and needs a load generator pointed at a running server.

The numbers this project treats as targets, and the numbers measured so far, are in the [performance reference](../reference/performance.md). Why those targets exist and why CI does not gate on them is in [performance and testing](../explanation/performance-and-testing.md).

## Run the whole bench

```sh
zig build bench -Doptimize=ReleaseFast
```

That runs all three suites with the defaults: 1,000,000 domains, 200,000 iterations per suite, seed `0x5eed`. It takes minutes, most of it generating and loading the million-domain list.

`-Doptimize=ReleaseFast` is not optional if you want the numbers to mean anything. A Debug build says so before it prints:

```
warning: Debug build; run with -Doptimize=ReleaseFast for meaningful numbers
```

## Run one suite, smaller

Everything after `--` goes to the harness. A suite name selects one of `filter`, `cache`, `compile` (the default is `all`), and `--domains` / `--iters` shrink the load:

```sh
zig build bench -Doptimize=ReleaseFast -- filter --domains=100000 --iters=20000
```

```
nxdns bench  suite=filter  domains=100000  iters=20000  seed=0x5eed  optimize=ReleaseFast

suite           ops     p50(us)     p95(us)     p99(us)     max(us)
filter        20000        2.38        2.76        2.88       20.32
  blocked 6670/20000, 32 regex rules, Snapshot.memoryBytes 3.0 MiB, VmRSS 5.6 MiB
  target p95 < 1ms: PASS
  target VmRSS < 100 MiB: PASS
```

A reduced run is good for checking the harness works and for a rough regression signal. It is not a result: the memory figure scales with `--domains`, so 100,000 domains says nothing about the 1,000,000-domain memory target.

The other two suites:

```sh
zig build bench -Doptimize=ReleaseFast -- cache --iters=20000
```

```
suite           ops     p50(us)     p95(us)     p99(us)     max(us)
cache         20000        0.14        0.18        0.20        5.14
  hits 10000/20000, DnsCache.memoryBytes 4.3 MiB, VmRSS 6.3 MiB
  target p95 < 5ms: PASS
```

```sh
zig build bench -Doptimize=ReleaseFast -- compile --domains=100000
```

```
suite           ops     p50(us)     p95(us)     p99(us)     max(us)
compile      100000   wall 15.623ms, 6400464 lines/s, 100000 domains kept (informational)
```

`--seed=N` changes the generated domains and the query order; the default is `0x5eed`, so two runs on the same machine are comparable. `--domains` caps at 4,000,000, and the `compile` suite additionally refuses more than 2,000,000 — the compiler's own limit.

An argument the harness does not recognise stops it before any measuring:

```
error: unknown argument 'nosuch'
usage: zig build bench -Doptimize=ReleaseFast -- [filter|cache|compile|all] [--domains=N] [--iters=N] [--seed=N] [--assert]
```

## Read the output

- `p50`/`p95`/`p99`/`max` are per-operation microseconds, nearest-rank over every iteration. What one operation means differs per suite: for `filter` it is normalising a name plus evaluating it against the snapshot; for `cache` it is building the key, getting the entry and stamping the response id.
- `blocked N/M` and `hits N/M` are sanity counters. The harness aborts if either is zero — a suite that never hits its own path measures nothing.
- `32 regex rules` on the `filter` line is the rule set the suite loads. No generated query matches any of them, so every operation runs all 32 programs to their end, which is the costly case and the one worth measuring.
- Two memory figures appear on purpose. `Snapshot.memoryBytes` and `DnsCache.memoryBytes` are the in-repo accounting of those structures; `VmRSS` is what the kernel holds resident for the whole process, allocator slack and code included. The truth is between them, and the memory target is judged on `VmRSS`.
- `target ...: PASS` / `FAIL` lines appear for the targets a suite covers. On a plain run they are informational and the exit code stays 0.

## Fail the run when a target is missed

`--assert` turns those lines into an exit code — 1 when any target was exceeded, 0 otherwise. This is meant for an acceptance run on hardware you control, not for CI:

```sh
zig build bench -Doptimize=ReleaseFast -- --assert
```

The full-scale form is the one worth asserting on, because the memory target only means something at a million domains. On this development host the reduced form was used to check the flag itself:

```sh
zig build bench -Doptimize=ReleaseFast -- filter --domains=100000 --iters=20000 --assert
```

```
filter        20000        2.34        2.71        2.85       15.06
  blocked 6670/20000, 32 regex rules, Snapshot.memoryBytes 3.0 MiB, VmRSS 5.6 MiB
  target p95 < 1ms: PASS
  target VmRSS < 100 MiB: PASS
```

**Not verified on this host at full scale:** the plain `zig build bench -Doptimize=ReleaseFast -- --assert` above was not run during the writing of this page — the default run takes minutes. The reduced runs shown were all executed as written. The full-scale numbers already recorded for this host are in the [performance reference](../reference/performance.md).

## Measure sustained query rate

The bench harness cannot do this. Query rate is a property of the whole server — sockets, upstreams, the query log writer — so it has to be driven from outside, against the real binary, on the machine you care about.

Start nxdns with real blocklists configured, then drive it from another host on the LAN with a DNS load generator such as `dnsperf`:

```sh
dnsperf -s 192.168.1.10 -p 53 -d queries.txt -c 20 -Q 200 -l 60
```

Read the client's own rate and the server's `/metrics` together: a load generator that reports 200 qps while the server counts fewer has lost queries somewhere, and that is the interesting number.

**Not verified on this host:** `dnsperf` is not installed here and the target platform is a Raspberry Pi 5, not this development machine. The command above is the shape of the measurement, not a transcript.

## Where to run it

The target platform is a Raspberry Pi 5. Numbers from a development x86_64 box do not transfer — the Pi's Cortex-A76 is far slower — so a passing run here is evidence the harness works and a baseline for spotting regressions on the machine development happens on, and nothing more. Run `--assert` on the Pi, where the numbers mean something.
