milestone 13: restructure docs to diataxis, tutorial, every command executed

This commit is contained in:
2026-08-02 18:02:58 +02:00
parent 35f23240e7
commit 16c9de2414
26 changed files with 3627 additions and 901 deletions
+264
View File
@@ -0,0 +1,264 @@
# Architecture
nxdns is a self-hosted DNS sinkhole for a household LAN: one static Zig binary
that answers DNS on UDP/TCP 53 (optionally DoH and DoT), filters against
blocklists, and serves an embedded admin SPA over HTTP. This page maps the
source tree and explains the few design rules that hold everywhere, and why
they are the rules.
For what the configuration fields, API routes and CLI flags actually are, see
[reference/configuration.md](../reference/configuration.md),
[reference/api.md](../reference/api.md) and
[reference/cli.md](../reference/cli.md). This page does not repeat them.
## Module map
Top-level files:
| File | Role |
|---|---|
| `src/main.zig` | Process shell: writers, argv, dispatch, exit code. |
| `src/cli.zig` | Every command body (`run`, `check`, `export`, `import`, `version`, `help`); takes its writers as parameters so tests capture output without a process. |
| `src/app.zig` | The composition root: everything `nxdns run` owns, built in order. Nothing else constructs a collaborator. |
| `src/version.zig` | Build-time version strings. |
| `src/tests.zig` | Test root; imports each file directly. |
| `src/docs_drift_test.zig` | Guards that keep the reference pages in step with the code. |
Directories:
| Directory | Role |
|---|---|
| `src/dns/` | Pure DNS wire format: header, names, questions, records, whole packets, EDNS(0)/ECS (`edns.zig`), enums and limits (`types.zig`). No allocation, no `std.Io` beyond writing to a caller's writer. |
| `src/filter/` | Blocklist pipeline: line parsers (hosts, domains, ABP), the compiler that turns a downloaded list into `.list`/`.wild` bodies, `domain_set.zig` (exact-match set, no Bloom filter), `matcher.zig` (the immutable snapshot every query evaluates against), per-group `rules.zig`, `wildcard.zig`, `safesearch.zig`, blocked-response synthesis (`response.zig`). Two I/O edges live here too: `fetcher.zig` (HTTP download) and `manager.zig` (files + DB + snapshot swap). |
| `src/local/` | Local DNS records and conditional forward zones: immutable lookup tables built once from DB rows (`records.zig`, `forward_zones.zig`), plus the plain UDP/TCP client for LAN resolvers (`forward_client.zig`). |
| `src/cache/` | `dns_cache.zig`: bounded in-memory TTL cache of whole response messages, keyed by the question. The clock arrives as a parameter. |
| `src/upstream/` | Upstream resolution: shared vocabulary and the `Client` interface (`transport.zig`), DoH client (RFC 8484), DoT client (RFC 7858), per-endpoint health and backoff (`health.zig`), and `pool.zig` — priority-ordered failover that is itself a `transport.Client`, so the handler sees one interface. |
| `src/server/` | The serving side: UDP/TCP/DoH/DoT listeners, `handler.zig` (the whole query pipeline), `cert_store.zig` (refcounted TLS cert holder), `rate_limiter.zig`, `pause.zig`, `clients.zig` (client auto-materialisation), `local_tables.zig` (published local-answer tables), `query_sink.zig` (log and SSE fanout), `shutdown.zig` (SIGINT/SIGTERM into one `std.Io.Event`). |
| `src/storage/` | SQLite ownership: `db.zig` is the only file that calls SQLite, `config_schema.zig` + `migrations.zig` for `config.db`, `querylog_schema.zig` (open-or-recreate), async query `logger.zig`, `retention.zig`, `disk_monitor.zig`, and one repository per table under `repositories/`. |
| `src/config/` | The one configuration model (`model.zig`), the pure validator (`validate.zig`), `import.zig`/`export.zig` (ZON to and from `config.db`, byte-stable round trip), `bootstrap.zig` (first-start seeding — a policy wrapper over import). |
| `src/web/` | The admin HTTP layer: `server.zig` (listener), `router.zig`/`routes.zig`, one file per resource under `handlers/`, `auth.zig` (sessions), `sse.zig` (live query fanout), `static.zig` (embedded SPA), `metrics.zig` (Prometheus), `openapi.zig` (served contract), `api_limiter.zig`, `http_util.zig`. |
| `src/platform/` | OS and TLS edges: IP address values, the `std.log` sink (`logging.zig`), `statfs.zig` (free-space query via libc), client TLS over `std.crypto.tls` (`tls_client.zig`), server TLS over vendored Mbed TLS (`tls_server.zig`). |
The SPA source lives in `web/` at the repo root; the build embeds its `dist/`
output as the `web_assets` module (`-Dweb-dist`).
```
main.zig ── cli.zig ── app.zig (composition root)
│ injects std.Io + collaborators
┌──────────────────────┴───────────────────────┐
│ server/ web/ upstream/ storage/ │ I/O edge
│ platform/ config/{import,export,bootstrap} │
├──────────────────────────────────────────────┤
│ dns/ filter/* local/* cache/ │ pure core:
│ config/{model,validate} │ bytes in, bytes out
└──────────────────────────────────────────────┘
* except filter/{fetcher,manager}.zig and local/forward_client.zig,
which are those directories' named I/O edges
```
## The purity rule
`dns/`, `filter/`, `local/` and `cache/` take bytes and return bytes: no
`std.Io`, no sockets, no clocks hidden inside (AGENTS.md). Anything that needs
a timestamp takes it as a parameter — the cache, the rate limiter and the
pause flag all work this way, so every decision is testable without a backend.
The point is not purity for its own sake. A decision that depends on a hidden
clock or a hidden socket can only be tested by arranging the world around it;
one that takes the clock as an argument is tested by passing a number. The
whole filtering and caching pipeline can therefore be exercised in the plain,
network-free test suite, which is what makes that suite worth gating CI on.
The exceptions are deliberate, few, and named: `filter/fetcher.zig` downloads
lists, `filter/manager.zig` owns the compiled files, the DB columns and the
snapshot swap, and `local/forward_client.zig` speaks UDP/TCP to a LAN
resolver. Those three files are the only ones under those four directories
that take a `std.Io`. The decision path a query takes through them allocates
nothing and opens nothing.
The honest cost of the exceptions shows up in
[performance-and-testing.md](performance-and-testing.md): `fetcher.zig` is
where the one production crash came from, precisely because it is the file the
pure suite cannot reach.
## std.Io injection
There is one `std.Io` in the process. `main` receives it through
`std.process.Init` — on the standard start path this is the Threaded backend
(`std.Io.Threaded`, constructed in the stdlib's start code) — and hands it to
`cli.Runner`, from which `app.zig` threads it into every collaborator as a
parameter. No module constructs its own event loop or reads an ambient clock;
tests build their own `std.Io.Threaded` instance and pass it the same way.
The one deliberate exception is `storage/db.zig`: SQLite performs its own file
I/O through its VFS, so that file takes no `std.Io` at all. Wrapping SQLite's
VFS to route through `std.Io` would be a large amount of C-boundary code to
make one dependency match a convention it does not need.
## Life of one query
The pipeline lives in `src/server/handler.zig``Handler.handle` does
validation and setup, then `Context.run` decides the answer. Its order is
PLAN §4; the stages below are the code's actual call chain:
```
UDP/53 TCP/53 DoH DoT (src/server/{udp,tcp,doh,dot}_server.zig)
└──────┴──────┴────┘
│ raw query bytes, listener-owned buffers
handler.handle
├─ header parse (too short / QR set → counted drop)
├─ rate limit (over budget → REFUSED)
├─ packet + EDNS validation (FORMERR / NOTIMP, qdcount != 1 → FORMERR)
├─ one clock read, client tracking, snapshot + local-table acquire
├─ group lookup (snapshot.groupForClient)
└─ Context.run
├─ qclass != IN ──► upstream, unfiltered and uncached
├─ local records ────────────────► authoritative answer
├─ forward zones ─► cache ─► LAN resolver ─► cache put ─► answer
└─ upstream path (filtering off while paused)
├─ filter snapshot evaluate ─► blocked? synthesized block reply
├─ safe-search rewrite (per group)
├─ cache get ─► hit? answer
├─ upstream pool: priority failover across DoH/DoT endpoints
├─ CNAME uncloak: walk the answer's chain, re-evaluate each target
└─ cache put (skipped for safe-search answers)
reply: UDP size check ─► TC=1 if over the limit ─► listener sends
└─► QuerySink ─► SSE hub (GET /api/queries/live)
└─► async logger ─► querylog.db
```
Local records win over forward zones, and both win over filtering: a name
nxdns answers itself never reaches a blocklist. Pause suspends filtering only;
local records, forward zones, cache, upstream and the query log keep running,
which is what makes pause safe to hand to a household member. A question whose
class is not IN bypasses local answers, filtering and the cache entirely and
goes straight upstream — nxdns has no opinion about CHAOS or HESIOD names and
declines to cache answers it does not model.
`handle` returns no error union. Every failure is either a DNS response the
client can act on or a counted drop, because there is no caller above it that
could do anything useful with a Zig error. Two consequences are worth knowing:
answers synthesized from a safe-search rewrite are never cached (the rewrite is
per group, and the cache is not), and a SERVFAIL reply short-circuits before the
query sink, so it appears in the counters but not in the query log.
The query path never waits on the database. `QuerySink` copies the entry, the
SSE hub gets it first, and one writer task owns the `querylog.db` handle behind
an `std.Io.Queue`. A slow disk delays logging, never resolution.
Two smaller decisions in the same spirit: the upstream pool makes a second
pass that ignores backoff, so "every endpoint is in backoff" degrades to
trying anyway rather than to a blanket SERVFAIL; and a handler that has no
filter snapshot yet answers unfiltered rather than refusing. Both prefer a
working resolver over a correct-looking failure.
## Storage
Two databases with opposite contracts, in one data directory (see
[reference/files-and-directories.md](../reference/files-and-directories.md)).
**`config.db` is the truth.** Its schema is versioned: `migrations.zig` holds
an ordered list of steps, step 1 being the verbatim DDL from
`config_schema.zig`, each applied inside one transaction. `nxdns import`
replaces the whole content atomically under `BEGIN IMMEDIATE`, so a failed
import changes nothing; `nxdns export` renders it back as canonical ZON,
byte-identical across round trips. A config file seeds this database exactly
once at first start. Why it works that way is
[configuration-model.md](configuration-model.md).
**`querylog.db` is expendable.** It is never migrated. Its schema carries a
fingerprint derived from the DDL text, and at open, a missing, corrupt,
non-database, `quick_check`-failing or fingerprint-mismatched file is moved
aside and recreated empty — the old file is kept under a new name rather than
deleted, so an operator can still look at it. Retention deletes old rows daily
and periodically rewrites the file to reclaim space.
The split exists so that the churn of the second database can never endanger
the first. Query logs are high-volume, disposable, and the thing most likely
to be corrupted by a power cut on an SD card; configuration is small,
irreplaceable, and the thing an operator would have to reconstruct by hand.
Giving them one file would force the careful contract onto the noisy data or
the loose contract onto the valuable data.
## Web stack
`web/server.zig` runs one `std.http.Server` per connection over its own accept
loop, with a fixed set of pre-allocated connection slots, optionally behind
TLS. Over capacity it answers 503 rather than queueing without bound — the
admin UI is not the product, and it must not be able to starve DNS.
The SPA is embedded at build time: `static.zig` serves the `web_assets` module
— bytes, content type, strong ETag, and a pre-compressed `.gz` sibling where
it paid off — via a linear scan with no filesystem access at runtime. (The one
exception is `nxdns run --web-dev DIR`, which serves from disk with no cache
headers, for developing the SPA against a running server.) `GET
/api/queries/live` is server-sent events over chunked transfer, fed by the
same `QuerySink` the logger reads. Routing is a flat table (`routes.zig`)
matched linearly; a few dozen routes do not justify a trie. The OpenAPI YAML
is hand-written, embedded and served at `GET /api/openapi.yaml`, kept honest
by tests that assert every served route appears in it.
Authentication (`web/auth.zig`): the operator's password is verified against
an argon2id PHC string (`web.password_hash`; the plaintext is hashed on import
and never stored). A successful login mints a 256-bit token carried in a
cookie; the in-memory session table holds only SHA-256 digests of tokens,
compared in constant time, capped at 32 sessions with LRU eviction. Nothing is
persisted, so a restart logs everyone out — for a household LAN that is a
feature, not a gap. Unauthenticated by design: the monitoring endpoints
(health, version, metrics), the served OpenAPI contract, login itself, and the
static SPA assets, which the router hands to the SPA fallback before any auth
check. Everything else requires the cookie, and the API has its own
token-bucket rate limiter. See
[how-to/set-up-admin-authentication.md](../how-to/set-up-admin-authentication.md).
## DoH, DoT and certificate hot-reload
`server/doh_server.zig` (RFC 8484 over HTTP/1.1 and TLS) and
`server/dot_server.zig` (RFC 7858) mirror the plain listeners' shape. Server
TLS terminates in Mbed TLS (`platform/tls_server.zig`), exposing plaintext as
`std.Io.Reader`/`std.Io.Writer`, so the listeners above it do not know whether
they are encrypted.
Certificates hot-reload through `server/cert_store.zig`. One refcounted
`CertStore` per endpoint owns the published TLS context generation; listeners
`acquire` it per connection and `release` it when the connection ends, so a
reload never frees a context mid-handshake. Reload publishes nothing on
failure: both PEM files are read and a whole new context is built before
anything swaps, and any failure leaves the old generation serving. A watcher
polls mtime and size of both files every 30 seconds
(`cert_store.poll_interval_s`); `POST /api/certs/reload` triggers the same
path on demand and reports the per-endpoint outcome as its payload.
The requirement driving all of this is that a certbot renewal must not need a
restart and must not be able to break DNS. A half-swapped context or a
free-while-in-use would do exactly that, so the store is built so neither is
representable. See
[how-to/enable-doh-and-dot.md](../how-to/enable-doh-and-dot.md).
## Failure visibility
Every failure mode must be visible, and the surface is counters, not log lines
(AGENTS.md). Log lines are a bad primitive for this: they are unbounded, they
are only read after someone already suspects a problem, and on an SD card they
are a way to fill a disk.
So the handler counts every outcome in atomics — drops, FORMERR, NOTIMP,
REFUSED, SERVFAIL, blocked, uncloak-blocked, truncated, cache hits, local and
forward-zone answers, safe-search rewrites, paused and unfiltered queries, and
the tracker-full condition. Listeners count dropped datagrams instead of
queueing them unboundedly. `GET /metrics` renders all of it as Prometheus text
0.0.4, and `GET /api/health` rolls it up for a monitor. Health always answers
200: "degraded" is a fact about the box, not a failed request, and a monitor
that cannot distinguish the two is worse than no monitor.
The disk monitor classifies free space against thresholds and gates
non-essential writes; the query logger holds its batches while writes are
disallowed rather than dropping them silently or writing until the filesystem
fills. `std.log` is reserved for failures nobody else records, with
upstream-error deduplication so a flapping resolver cannot fill a disk with
identical lines.
+178
View File
@@ -0,0 +1,178 @@
# The configuration model
nxdns is configured two ways — a ZON file and a web UI — and only one of them
can be the truth. This page explains which, and why that choice is the one
that leaves the fewest ways to lose an operator's work.
For the fields themselves see
[reference/configuration.md](../reference/configuration.md); for the commands
and their exit codes see [reference/cli.md](../reference/cli.md).
## The rule
The database is the truth. The file is a seed.
`config.db` in the data directory holds the running configuration. The ZON
file (default `/etc/nxdns/config.zon`, overridable with `--config`) is read on
`nxdns run` only while the database is still empty: if the file exists and the
database holds no configuration, it is imported. Once the import has put rows
in, the file is not opened again.
The condition is the state of the database, not a one-shot flag. A `run` whose
seed fails — unreadable file, parse error, failed validation, a constraint
violation inside the import transaction — leaves the database empty, so the
next `run` reads the file again. That is what makes fixing a typo and starting
again work.
`src/config/bootstrap.zig` is that policy and nothing else — a wrapper over
the same import path `nxdns import` uses. Three outcomes:
- no file: the database is used as it is;
- file present, database empty: seed it;
- file present, database already configured: skip, without reading the file.
A file that exists but is unreadable, unparseable or invalid fails the start,
with every problem printed. nxdns never falls back to silent defaults over a
file an operator wrote — a resolver that boots "successfully" with a
configuration nobody chose is the worst outcome available, because it looks
like it worked.
## Why the database wins
The alternative designs all lose data.
If the file were the truth, the admin UI could not write. Every change would
be an SSH session and a restart, which defeats the reason the UI exists: the
household member who wants to unblock one domain is not going to edit ZON.
If both were the truth, they would disagree. The UI writes a rule; the file
still says otherwise; the next restart either silently reverts the rule or
silently ignores the file. Both are silent, and both destroy work someone
intended to keep. There is no merge rule that fixes this, because the system
cannot know which of two conflicting statements is the newer intention.
So the file's authority ends the moment the database has content. Editing
`config.zon` after first boot does nothing — no partial effect, no warning
that some fields took and others did not. That is a blunt rule, and it is the
point: the failure mode is "my edit did nothing", which is visible the first
time you look at the UI, rather than "my edit was applied and then quietly
undone next Tuesday".
The emptiness check is a real query over the content tables, not a flag: the
database counts as configured when any content table has rows, or when the
default group has been altered. One consequence is worth knowing, because it
is not obvious: clients are auto-materialised when they first send a query, so
a server that has answered even one query is "configured" and will ignore a
seed file placed there afterwards.
## The round trip
Losing the file as an editing surface would be a real loss — text is
diffable, reviewable and easy to back up — so the file is kept as a
*rendering* of the database rather than a rival to it. That is what
`export`/`import` are for:
```
nxdns export → canonical ZON → edit → nxdns import --force → config.db
```
`nxdns export` renders the database as canonical ZON: a fixed two-line
header, every default emitted, deterministic ordering from the model's field
order and the repositories' `ORDER BY` clauses, and no timestamps or hostnames
anywhere. Runtime columns — first-seen, last-seen, per-source counters — are
absent from the configuration model on purpose, so two exports taken from a
live, busy server are identical. The round trip `export → import → export` is
byte-identical, and a test asserts it.
Byte-stability is not cosmetic. It is what makes an exported file usable in
version control and what makes a diff of two exports mean something: any
difference is a configuration change, never noise from when the export ran.
`nxdns import` replaces the whole database content in one transaction. Not a
merge, not a patch: delete every content table in foreign-key-safe order, then
insert what the file says. A failed import — bad syntax, failed validation, a
constraint violation halfway through — leaves the database exactly as it was,
because everything happens inside a single `BEGIN IMMEDIATE`.
Without `--force`, import refuses a database that already has content. That
check runs *inside* the transaction, after the lock is taken, so it cannot be
raced by a concurrent write. The effect is that a plain `import` can never
clobber a configured server by accident, and clobbering it deliberately takes
one visible extra word on the command line. See
[how-to/back-up-and-restore.md](../how-to/back-up-and-restore.md).
## One model, three surfaces
`src/config/model.zig` defines exactly one `Config` type. The ZON parser
produces it, the database reader produces it, the validator consumes it, the
composition root consumes it, and the settings API derives its key list and
its patch struct from its type information rather than mirroring the fields by
hand. Adding a field in one place therefore cannot leave the other surfaces
behind; the alternative — a file schema, a DB schema and an API schema kept in
step by discipline — is the standard way configuration systems rot.
The model has two shapes of field, and the difference is structural rather
than stylistic. Struct-typed fields are scalar sections (`dns`, `cache`,
`web`, …) and live in a single key/value `settings` table as
`section.field` text rows. Slice-typed fields are collections (groups,
upstreams, clients, rules, local records, forward zones, …) and each gets its
own table with foreign keys. `toSettings` and `fromSettings` are the two
halves of the scalar bridge, generated by an `inline for` over the model, so
the key list is a consequence of the type rather than a second list to
maintain.
## Unknown keys, and why the asymmetry is deliberate
An unknown key in the **database** is warned about and ignored. An unknown key
in the **file** is a hard error.
They are different situations. A settings row the running binary does not
recognise is almost always an older binary reading a database written by a
newer one — a downgrade, or a rollback after a bad upgrade. Refusing to start
there would mean a downgrade bricks the config database, and the operator
would have to hand-edit SQLite to recover. Warning and ignoring means the
downgrade works, the unknown setting sits inert, and the upgrade back picks it
up again.
A key in a file, by contrast, is something a human just typed. The likeliest
cause is a typo, and the second likeliest is a field that no longer exists.
Silently ignoring it would mean the setting the operator believes they applied
was never applied — exactly the silent-divergence failure the whole model is
built to avoid. So the ZON parser rejects unknown fields with a line and
column.
The tolerance has a boundary worth stating plainly: it covers unknown *keys*,
not unparseable *values*. A known key whose stored text does not decode into
its type is an error, not a warning.
## The password
`web.password` is a write-only input. It is never a stored value.
At import time, a non-empty `web.password` is hashed with argon2id (PHC
encoding, OWASP argon2id parameters) into `web.password_hash`, and the
plaintext field is cleared before anything is written. There is no settings
row that can hold it: the model explicitly skips `web.password` in both
directions of the settings bridge, so the plaintext has nowhere to go even by
accident. `nxdns export` always writes `.password = ""` and carries the hash
instead — which is also what makes the round trip stable, since an export that
tried to reproduce a plaintext it never had could not be byte-identical.
Setting both `password` and `password_hash` in one file is an error rather
than a precedence rule. The two say different things about what the password
is, and picking a winner would mean the operator's other statement was
silently discarded. The file has to say one thing.
The practical shape of a password change is therefore export, set `.password`
to the new value, clear `.password_hash`, import with `--force`. The procedure
is in
[how-to/set-up-admin-authentication.md](../how-to/set-up-admin-authentication.md).
## What is not configuration
Storage paths are process arguments, not configuration fields: `--data-dir`,
`--config`, `--web-dev`. They cannot live in the file, because the file is
found by way of them — a path that told you where to find the thing that told
you the path would be circular. They are also the settings a supervisor
(systemd, Docker) owns rather than the operator's policy about DNS. See
[reference/files-and-directories.md](../reference/files-and-directories.md).
+161
View File
@@ -0,0 +1,161 @@
# Performance targets and what the tests prove
Two related questions: why the performance numbers are the numbers, and why CI
does not enforce them — and then, less comfortably, what a green test suite
here does and does not tell you.
For the targets and the measured results as data, see
[reference/performance.md](../reference/performance.md); to run the bench
yourself, [how-to/measure-performance.md](../how-to/measure-performance.md).
## Where the targets come from
PLAN §18 sets five:
- sustained ≥ 100 qps on a Raspberry Pi 5;
- blocklist lookup p95 < 1 ms;
- cached response p95 < 5 ms;
- memory with ~1M blocked domains < 100 MB;
- stripped static binary < 10 MB per arch, < 15 MB with the embedded frontend.
They are household-scale numbers, and they are deliberately unambitious. 100
qps is far more than a house generates; the point of the target is not speed
but that a Pi 5 with an SD card never becomes the reason the internet feels
broken. The latency targets exist for the same reason: DNS sits in front of
every connection anyone makes, so the failure people notice is not throughput
but a stall. The memory target is what keeps a 1M-entry blocklist from
competing with everything else on a 4 GB board. The binary-size target is
about what a static single-binary deployment is for — if it does not fit on a
constrained box and copy over a slow link in one step, the packaging decision
has not paid for itself.
`tools/bench.zig` (`zig build bench`) measures the three that are measurable
in-process: `filter` (normalize plus snapshot evaluate against a ~1M-entry
snapshot), `cache` (key build plus cache get plus id patch), and `compile`
(the blocklist compiler over a 1M-line body, informational — there is no §18
target for it because no prior datapoint exists). Memory comes from
`/proc/self/status` VmRSS. The qps target is not in the harness at all: it is
end-to-end against the real binary with a DNS load generator, because a
harness number for "queries per second" would measure the harness.
The bench is `tools/`, not `src/`, on purpose: `src/` is the shipped product,
and `src/tests.zig` aggregates everything shippable.
## Why CI does not gate on performance
Required CI stays deterministic (AGENTS.md). Latency assertions on shared
runners measure the runner's noisy neighbours; the same commit passes and
fails depending on what else the host is doing. A gate that flakes does not
protect anything — it trains people to re-run the job, and once re-running is
routine, a real regression gets re-run too. The flaky gate is worse than no
gate, because it also consumes the attention a real gate would need.
So the bench defaults to informational, and `--assert` — which exits non-zero
on a missed target — exists for hardware you control. Run it on the Pi, where
the numbers describe the machine the software actually has to run on. The
x86_64 development-host numbers in
[reference/performance.md](../reference/performance.md) are a regression
baseline for the machine development happens on, not a claim about the target
platform; a Cortex-A76 is far slower and those numbers do not transfer.
CI does gate on the one performance property that *is* deterministic: binary
size. The `cross` job strips the release binaries and asserts them under the
§18 budgets. Size is a function of the input, not of the runner's mood, so it
is exactly the kind of thing a shared runner can measure honestly.
## What the test suite is
The blocking CI (Gitea Actions, `.gitea/workflows/ci.yml`) runs five jobs, all
required: the Zig suite with `-Dintegration`; the same suite cross-built for
aarch64 and executed under qemu-user; the frontend (format, lint, typecheck,
121 vitest cases, build); the cross-build with the two stripped-size asserts;
and a Docker smoke run that boots the image and polls `/api/health`.
The Zig suite has three tiers, gated by build flags:
- **plain `zig build test`** — pure logic. No sockets, no threads, no clock
budgets. This is the tier the purity rule
([architecture.md](architecture.md)) exists to make possible.
- **`-Dintegration`** — hermetic integration: loopback sockets, `:memory:`
databases, temp directories. Nothing leaves the host.
- **`-Dlive`** — the only tests that reach the public internet (DoH and DoT
handshakes against real resolvers). Four tests, and they run in a
manual-dispatch workflow, never on push or pull request.
The aarch64 job runs the plain tier only. The integration tests are
multithreaded loopback TLS with wall-clock budgets, and qemu-user's slowdown
turns those budgets into a flake source — the same reasoning that keeps the
bench out of CI. What aarch64 needs to prove is portable correctness of the
DNS, filter and cache logic, and the plain tier is exactly that.
At the time of writing, plain `zig build test` is 1175 of 1288 passing with
113 skipped and 0 failed, the skips being the integration-gated tests.
Milestone 12 recorded the other two tiers on the same tree: 1280 of 1284 with
`-Dintegration` (the 4 skips are the live-network tests) and 1159 passing
under qemu, 0 failed in each.
## What it does not prove
The suite is hermetic by design. That is the right default: it is fast, it is
deterministic, it can gate merges. But hermetic and correct are different
properties, and the gap has already cost this project twice.
**The blocklist download aborted the process on first real use.** The fetcher
constructed the HTTP response reader over `transfer_buf` and then read *into*
that same buffer. `Reader.readSliceShort` starts by `@memcpy`-ing the reader's
already-buffered bytes into the caller's destination — so source and
destination were the same allocation, and Zig's `@memcpy` requires them not to
overlap. It aborts.
The reason no test caught it is precise and instructive. The copy length is
zero whenever the reader has nothing buffered, and a zero-length `@memcpy` is
fine. Bytes only accumulate in the reader's own buffer when a read comes back
short of filling the destination and the loop goes round again — that is, when
the body arrives in more than one stream call. The loopback fixture answers
every request with one small in-memory body that lands in a single read, and
the one over-size test never streams a byte, because the fetcher refuses an
oversized `content-length` on the response head. Every test in the suite was
on the zero-length-memcpy side of the branch. The first real download — a
multi-megabyte list over TLS across the WAN, arriving in many TCP segments —
was on the other side, and took the process down. The fix (commit 35f2324)
streams the body straight into the caller's writer, so the reader's buffer is
never a destination slice, and it came with four regression tests that put a
fully-buffered reader into exactly the state the old code could not survive.
**A stale embedded SPA bundle shipped a settings page that crashed on load,
while 121 web tests passed.** `web/dist/` is gitignored and
`-Dweb-dist=web/dist` embeds whatever bytes are sitting in that directory. The
frontend tests ran against the sources, in jsdom, and were green; the binary
carried an older build. The tests were testing something the artifact did not
contain.
Note what these two have in common. Neither was a logic bug that a better unit
test would have caught. One lived in the seam between the pure core and its
one I/O edge; the other lived in the seam between two build systems. Hermetic
tests are constructed to exclude exactly those seams — that is what makes them
hermetic.
## The lesson, and where it now lives
Green hermetic tests are a floor, not a ceiling. They prove the logic is
consistent with itself. They cannot prove the program works, because the
things they deliberately exclude — real network reads, real TLS, real file
sizes, real build artifacts — are where a program meets reality.
The response is not to make CI non-deterministic. It is to require that the
real paths get exercised by a human before work is called done. That is now
ruling 3 of `specs/milestone-13.md`: every command block in the tutorial and
the how-to pages is executed verbatim, on the host, by the session that writes
it, and a command that cannot run there is marked in the page as unverified
with the reason. Documentation written from source-reading alone is how both
of these shipped; documentation that has been run is a second, independent
test suite that exercises precisely the paths the hermetic one skips.
Two honest gaps remain, stated so nobody has to rediscover them:
- Nothing in the suite drives a multi-read HTTP body through the fetcher end
to end. The regression tests cover `pumpBody` directly over a pre-buffered
reader; the loopback fixture still sends one small body per connection.
- There is no freshness check on `web/dist`. CI cannot embed a stale bundle,
because the jobs that pass `-Dweb-dist` rebuild the frontend immediately
beforehand. A local build can, and will do it without a warning.