140 lines
8.0 KiB
Markdown
140 lines
8.0 KiB
Markdown
# Measure performance
|
|
|
|
`tools/bench.zig` measures the three things nxdns can measure in-process: blocklist lookup latency, cache-hit latency, and blocklist compile throughput. Sustained query rate is not one of them — that one is end-to-end and needs a load generator pointed at a running server.
|
|
|
|
The numbers this project treats as targets, and the numbers measured so far, are in the [performance reference](../reference/performance.md). Why those targets exist and why CI does not gate on them is in [performance and testing](../explanation/performance-and-testing.md).
|
|
|
|
## Run the whole bench
|
|
|
|
```sh
|
|
zig build bench -Doptimize=ReleaseFast
|
|
```
|
|
|
|
That runs all three suites with the defaults: 1,000,000 domains, 200,000 iterations per suite, seed `0x5eed`. It takes minutes, most of it generating and loading the million-domain list.
|
|
|
|
`-Doptimize=ReleaseFast` is not optional if you want the numbers to mean anything. A Debug build says so before it prints:
|
|
|
|
```
|
|
warning: Debug build; run with -Doptimize=ReleaseFast for meaningful numbers
|
|
```
|
|
|
|
## Run one suite, smaller
|
|
|
|
Everything after `--` goes to the harness. A suite name selects one of `filter`, `cache`, `compile` (the default is `all`), and `--domains` / `--iters` shrink the load:
|
|
|
|
```sh
|
|
zig build bench -Doptimize=ReleaseFast -- filter --domains=100000 --iters=20000
|
|
```
|
|
|
|
```
|
|
nxdns bench suite=filter domains=100000 iters=20000 seed=0x5eed optimize=ReleaseFast
|
|
|
|
suite ops p50(us) p95(us) p99(us) max(us)
|
|
filter 20000 2.38 2.76 2.88 20.32
|
|
blocked 6670/20000, 32 regex rules, Snapshot.memoryBytes 3.0 MiB, VmRSS 5.6 MiB
|
|
target p95 < 1ms: PASS
|
|
target VmRSS < 100 MiB: PASS
|
|
```
|
|
|
|
A reduced run is good for checking the harness works and for a rough regression signal. It is not a result: the memory figure scales with `--domains`, so 100,000 domains says nothing about the 1,000,000-domain memory target.
|
|
|
|
The other two suites:
|
|
|
|
```sh
|
|
zig build bench -Doptimize=ReleaseFast -- cache --iters=20000
|
|
```
|
|
|
|
```
|
|
suite ops p50(us) p95(us) p99(us) max(us)
|
|
cache 20000 0.14 0.18 0.20 5.14
|
|
hits 10000/20000, DnsCache.memoryBytes 4.3 MiB, VmRSS 6.3 MiB
|
|
target p95 < 5ms: PASS
|
|
```
|
|
|
|
```sh
|
|
zig build bench -Doptimize=ReleaseFast -- compile --domains=100000
|
|
```
|
|
|
|
```
|
|
suite ops p50(us) p95(us) p99(us) max(us)
|
|
compile 100000 wall 15.623ms, 6400464 lines/s, 100000 domains kept (informational)
|
|
```
|
|
|
|
`--seed=N` changes the generated domains and the query order; the default is `0x5eed`, so two runs on the same machine are comparable. `--domains` caps at 4,000,000, and the `compile` suite additionally refuses more than 2,000,000 — the compiler's own limit.
|
|
|
|
An argument the harness does not recognise stops it before any measuring:
|
|
|
|
```
|
|
error: unknown argument 'nosuch'
|
|
usage: zig build bench -Doptimize=ReleaseFast -- [filter|cache|compile|all] [--domains=N] [--iters=N] [--seed=N] [--assert]
|
|
```
|
|
|
|
## Read the output
|
|
|
|
- `p50`/`p95`/`p99`/`max` are per-operation microseconds, nearest-rank over every iteration. What one operation means differs per suite: for `filter` it is normalising a name plus evaluating it against the snapshot; for `cache` it is building the key, getting the entry and stamping the response id.
|
|
- `blocked N/M` and `hits N/M` are sanity counters. The harness aborts if either is zero — a suite that never hits its own path measures nothing.
|
|
- `32 regex rules` on the `filter` line is the rule set the suite loads. No generated query matches any of them, so every operation runs all 32 programs to their end, which is the costly case and the one worth measuring.
|
|
- Two memory figures appear on purpose. `Snapshot.memoryBytes` and `DnsCache.memoryBytes` are the in-repo accounting of those structures; `VmRSS` is what the kernel holds resident for the whole process, allocator slack and code included. The truth is between them, and the memory target is judged on `VmRSS`.
|
|
- `target ...: PASS` / `FAIL` lines appear for the targets a suite covers. On a plain run they are informational and the exit code stays 0.
|
|
|
|
## Fail the run when a target is missed
|
|
|
|
`--assert` turns those lines into an exit code — 1 when any target was exceeded, 0 otherwise. This is meant for an acceptance run on hardware you control, not for CI:
|
|
|
|
```sh
|
|
zig build bench -Doptimize=ReleaseFast -- --assert
|
|
```
|
|
|
|
The full-scale form is the one worth asserting on, because the memory target only means something at a million domains. On this development host the reduced form was used to check the flag itself:
|
|
|
|
```sh
|
|
zig build bench -Doptimize=ReleaseFast -- filter --domains=100000 --iters=20000 --assert
|
|
```
|
|
|
|
```
|
|
filter 20000 2.34 2.71 2.85 15.06
|
|
blocked 6670/20000, 32 regex rules, Snapshot.memoryBytes 3.0 MiB, VmRSS 5.6 MiB
|
|
target p95 < 1ms: PASS
|
|
target VmRSS < 100 MiB: PASS
|
|
```
|
|
|
|
**Not verified on this host at full scale:** the plain `zig build bench -Doptimize=ReleaseFast -- --assert` above was not run during the writing of this page — the default run takes minutes. The reduced runs shown were all executed as written. The full-scale numbers already recorded for this host are in the [performance reference](../reference/performance.md).
|
|
|
|
## Measure sustained query rate
|
|
|
|
The bench harness cannot do this. Query rate is a property of the whole server — sockets, upstreams, the query log writer — so it has to be driven from outside, against the real binary, on the machine you care about.
|
|
|
|
Start nxdns with real blocklists configured, then drive it from another host on the LAN with a DNS load generator such as `dnsperf`:
|
|
|
|
```sh
|
|
dnsperf -s 192.168.1.10 -p 53 -d queries.txt -c 20 -Q 200 -l 60
|
|
```
|
|
|
|
Read the client's own rate and the server's `/metrics` together: a load generator that reports 200 qps while the server counts fewer has lost queries somewhere, and that is the interesting number.
|
|
|
|
**Not verified on this host:** `dnsperf` is not installed here and the target platform is a Raspberry Pi 5, not this development machine. The command above is the shape of the measurement, not a transcript.
|
|
|
|
## Measure memory on a running server
|
|
|
|
The bench measures one structure at a time. A running server's memory also depends on the most concurrent work it has seen since it started, so read it on the server itself, after it has served a normal day:
|
|
|
|
```sh
|
|
pid=$(pidof nxdns)
|
|
grep -E 'VmRSS|VmHWM|Threads' /proc/$pid/status
|
|
curl -s http://127.0.0.1:8080/metrics | grep '^nxdns_.*_peak'
|
|
```
|
|
|
|
- `VmRSS` is what the process holds resident now. `VmHWM` is the most it has ever held, which is normally the last blocklist refresh.
|
|
- `Threads` is a high-water too: the I/O runtime adds a worker thread only when every existing worker is busy, and never retires one. The count is the largest the worker pool has been since start, plus the main thread.
|
|
- `nxdns_dns_in_flight_peak` is the most queries handled at once, every listener together.
|
|
- `nxdns_tcp_server_connections_peak`, `nxdns_dot_server_connections_peak` and `nxdns_doh_server_connections_peak` are the most connections one listener has held at once. TCP/53 has one listener per address family; the gauge is the larger of the two.
|
|
- `nxdns_querylog_queue_peak` is the most entries that have waited for the query log writer at once. It can overstate the true peak by the entries in transit at that instant, never understate it.
|
|
|
|
Each worker thread's stack is its own anonymous mapping in `/proc/$pid/smaps`, directly above a one-page `---p` guard, and every worker's mapping has the same size. Its `Rss` is the stack that thread has actually touched.
|
|
|
|
**Verified on this host** against a static x86_64 musl build under a burst of 300 UDP queries: the gauges and the stack mappings behave as described. Not yet run on the Pi.
|
|
|
|
## Where to run it
|
|
|
|
The target platform is a Raspberry Pi 5. Numbers from a development x86_64 box do not transfer — the Pi's Cortex-A76 is far slower — so a passing run here is evidence the harness works and a baseline for spotting regressions on the machine development happens on, and nothing more. Run `--assert` on the Pi, where the numbers mean something.
|