nxdns/specs/milestone-11.md

# Milestone 11: packaging, ops and docs (PLAN Phase 10)

Goal: systemd unit, Dockerfile + compose, and the four docs (operator, architecture,
config-reference, API) — the documented deployment must work end-to-end; docs are
drift-guarded where a guard is cheap and honest.

## Rulings (binding)

1. **Layout.** `deploy/systemd/nxdns.service` + `deploy/systemd/sysusers.conf`;
   `deploy/docker/{Dockerfile,compose.yaml,.dockerignore}`; `docs/{operator.md,
   architecture.md,config-reference.md,api.md}`; `README.md` at the root (the repo has
   none; a portfolio repo needs a front door — short: what, why, quickstart, doc links).
   PLAN.md:236 sketches docs/ subdirectories; single files need no subdirectories.

2. **API docs = hand-written `docs/api.md` + drift test.** No renderer is vendored
   (redoc/scalar are exactly the dependency liability AGENTS.md refuses), and a
   build-time YAML parser for rendering is scope the yaml does not justify — the yaml
   itself is already served unauthenticated at `GET /api/openapi.yaml` (routes.zig:49)
   and is the exhaustive contract. `docs/api.md` gives human-readable orientation:
   auth model (cookie session, login flow), rate limiting, error envelope, SSE
   semantics, then one line per operation (method, path, auth, one-sentence purpose)
   and a pointer to the yaml for schemas. A drift test asserts every served route
   appears textually in api.md (mirror of openapi.zig:34's guard). This satisfies
   m8 ruling 3's deferred "docs/api rendering" as the engineering call: rendered =
   readable, guarded, in-repo; not = a vendored JS bundle.

3. **Docs drift guards.** New `src/docs_drift_test.zig` (ORCHESTRATOR-owned, written
   after the doc sessions land): (a) every route in `router.routes` appears in
   docs/api.md; (b) every settings key from `model.toSettings` (the 44 keys) appears
   in docs/config-reference.md; (c) every CLI subcommand name appears in
   docs/operator.md. Docs embedded via a `docs_files` anonymous import added in
   build.zig (test_fixtures pattern, build.zig:61). Guards are textual-containment
   only — cheap, zero false authority.

4. **systemd unit.** `Type=simple` (no forking, shutdown.zig:35 handles SIGTERM),
   `ExecStart=/usr/local/bin/nxdns run`, stderr → journald (logging.zig:301 already
   states this; `logging.output=stderr` stays the default). Static system user `nxdns`
   via `deploy/systemd/sysusers.conf` (`u nxdns - "nxdns DNS sinkhole"`), NOT
   DynamicUser — the TLS key must be chown-able to a stable uid ("TLS keys readable by
   service user only", PLAN §19). `StateDirectory=nxdns` (0700 matches cli.zig:226),
   `LogsDirectory=nxdns` (covers logging.output=file; the binary does not create the
   directory, logging.zig:502), `ConfigurationDirectory=nxdns`.
   `AmbientCapabilities=CAP_NET_BIND_SERVICE` + `CapabilityBoundingSet=` the same
   (port 53; 443/853 covered by the same cap). Hardening: `NoNewPrivileges=yes`,
   `ProtectSystem=strict`, `ProtectHome=yes`, `PrivateTmp=yes`, `PrivateDevices=yes`,
   `ProtectKernelTunables/Modules/Logs=yes`, `ProtectControlGroups=yes`,
   `ProtectClock=yes`, `ProtectHostname=yes`, `ProtectProc=invisible`,
   `RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX`, `RestrictNamespaces=yes`,
   `RestrictRealtime=yes`, `RestrictSUIDSGID=yes`, `LockPersonality=yes`,
   `MemoryDenyWriteExecute=yes` (static Zig binary, no JIT), `UMask=0077`,
   `SystemCallFilter=@system-service`, `SystemCallArchitectures=native`,
   `Restart=on-failure`, `RestartSec=2`. No `ReadWritePaths` beyond what
   StateDirectory/LogsDirectory grant. Validate with `systemd-analyze verify` if the
   build host has it; report honestly if not.

5. **Docker.** Multi-stage: builder stage only stages `ca-certificates` (upstream
   TLS verification rescans the system CA bundle, tls_client.zig:216 — a scratch
   image without a bundle breaks every DoH/DoT upstream); final `FROM scratch` with
   the static musl binary, `/etc/ssl/certs/ca-certificates.crt`, a nonroot numeric
   `USER 65532:65532`, `VOLUME /var/lib/nxdns`, `EXPOSE 53/udp 53/tcp 8080 443 853`,
   `ENTRYPOINT ["/nxdns"]`, `CMD ["run"]`. The binary is NOT built inside the
   Dockerfile (the repo builds it with zig; the Dockerfile COPYes
   `zig-out/cross/$TARGETARCH-…/nxdns` via a build arg or buildx TARGETARCH mapping —
   keep it working for both arches). compose.yaml: ports 53:53/udp+tcp and 8080:8080
   (443/853 commented), bind-mount `./etc-nxdns:/etc/nxdns:ro`, named volume for
   `/var/lib/nxdns`, `sysctls: net.ipv4.ip_unprivileged_port_start=0` so the nonroot
   user binds 53 (per-netns sysctl; documented), `restart: unless-stopped`. First
   boot needs a seeded `/etc/nxdns/config.zon` with a `default` group + one enabled
   upstream or the container exits 2 (bootstrap.zig:38, app.zig:93) — operator.md and
   a compose comment both say so. Do NOT point the host's resolv.conf at nxdns
   itself for the container's own lookups.

6. **CI.** One added job `docker` in ci.yml: after building the x86_64 exe
   (ReleaseSafe, with the SPA dist like the cross job), `docker build` the image and
   run a container smoke (seed a minimal config.zon; `nxdns version` + boot + one
   `dig`-equivalent via the test client or `curl` on 8080/api/health; SIGTERM 0).
   No registry push — no publish step exists anywhere in this repo or the infra
   repo's CI, and registry credentials are an infra decision outside this repo.
   Manual publishing to git.mial.net stays possible and is documented in operator.md
   in one paragraph.

7. **Docs content contracts.**
   - operator.md: install (systemd path and docker path, both complete), first boot +
     config.zon seeding semantics (file seeds DB once, DB is truth thereafter,
     bootstrap.zig:38), auth setup (`web.password` hashed on import, never stored,
     export writes ""), TLS cert/key provisioning + 0600 expectations + the reload
     API/watcher, backup/restore = `nxdns export`/`import --force` (+ the 0600 export
     mode and why), upgrades (schema migration = install + restart, PLAN §20.11),
     data-dir layout table, exit codes (0/1/2/64, cli.zig:35), CLI reference (all six
     subcommands + flags), troubleshooting (exit 2 causes, `nxdns check` semantics
     incl. source-selection order cli.zig:511, disk-full degradation, port 53
     conflicts with systemd-resolved — include the disable recipe).
   - architecture.md: module map (the src/ inventory), the purity rule (dns/, filter/,
     local/, cache/ take bytes, no Io — AGENTS.md), std.Io injection + Threaded
     backend, data flow for one query (listener → handler → filter/cache/local →
     upstream → sink/logger), storage split (config.db truth / querylog.db expendable),
     web stack (std.http over TLS optional, SPA embedded via web_assets, SSE), cert
     hot-reload design (refcounted CertStore), failure-visibility doctrine (counters +
     /metrics over log spam). Concise — a map, not a novel.
   - config-reference.md: complete — every section/field with type, default, unit,
     validation range, and which subsystem consumes it; collections with required
     fields; DB-vs-file truth explanation; the `web.password`/`password_hash`
     exclusivity; `logging.level=.err` serializes as "error" (model.zig:154). The
     explorer inventory in this milestone's research is the skeleton; verify against
     model.zig/validate.zig while writing, do not trust the summary blindly.
   - api.md: per ruling 2.
   - README.md: ≤120 lines; what nxdns is, feature list (honest, shipping features
     only), quickstart (docker compose path), build-from-source (zig build, node for
     the SPA), doc links, license note if a LICENSE exists (do not invent one).

8. **No new runtime code.** This milestone adds zero behavior to the binary. The only
   src/ change is the orchestrator's docs_drift_test.zig + its build wiring. If a doc
   session finds a bug while documenting, it REPORTS it (no fix); the orchestrator
   triages.

9. **Pi 5 end-to-end**: the exit criterion runs on hardware this environment does not
   have. The deliverable here is: both suites green, docker smoke green on x86_64,
   `systemd-analyze verify` clean (or honestly reported unavailable), aarch64 binary
   built and statically verified (existing cross job). The operator doc's Pi 5 recipe
   is written to be executed by the user; the spec records this boundary explicitly.

## Sessions

U1, U3, U4, U5 parallel; U2 after U1 (documents the artifacts U1 produces);
orchestrator wiring (ruling 3) after U3+U5.

## Session U1: deploy artifacts + CI

Owns `deploy/systemd/nxdns.service`, `deploy/systemd/sysusers.conf`,
`deploy/docker/{Dockerfile,compose.yaml,.dockerignore}`, `.gitea/workflows/ci.yml`
(one added job). Rulings 4, 5, 6. Verify: `systemd-analyze verify` (or report
unavailable), local `docker build` + container smoke if the docker daemon is
reachable (report honestly either way), `zig build test` untouched-green.

## Session U2: operator.md + README.md (after U1)

Owns `docs/operator.md`, `README.md`. Rulings 7 (operator + README). Reads U1's
artifacts and the runtime/CLI facts from the code (verify against src/cli.zig,
src/app.zig, src/config/bootstrap.zig — not from memory).

## Session U3: config-reference.md

Owns `docs/config-reference.md`. Ruling 7. Source of truth: src/config/model.zig +
validate.zig + import/export/bootstrap. Every field, no sampling.

## Session U4: architecture.md

Owns `docs/architecture.md`. Ruling 7. Reads module headers; no deep dives needed
beyond what the doc claims.

## Session U5: api.md

Owns `docs/api.md`. Rulings 2, 7. Source of truth: src/web/routes.zig (the served
table: method, path, auth, limiter) + openapi.yaml summaries + auth.zig/sse.zig for
the auth and SSE prose. Every route, no sampling.

## As built

**U1** delivered per rulings 4-6 with accepted deviations: `StateDirectoryMode=0700`
(systemd defaults 0755; the binary cannot tighten a pre-existing directory) and
`User=nxdns`/`Group=nxdns` added to the unit; the Dockerfile's builder stage also maps
buildx TARGETARCH → cross-target dir and pre-chowns `/var/lib/nxdns` to 65532 (a named
volume seeded from a root-owned image dir would be unwritable on first boot); compose
gained a `build:` block; `deploy/docker/.dockerignore` is documentation-grade under
BuildKit (only a root `.dockerignore` or `Dockerfile.dockerignore` is honored — the
file's header says so). Verified: `systemd-analyze verify` clean modulo the off-host
ExecStart path (an ExecStart=/bin/true copy verifies exit 0); full local docker build +
smoke passed (binds 53 as uid 65532, /api/health ok, SIGTERM exit 0). The CI docker
job probes both 127.0.0.1 and the container IP to survive either runner topology.
Compose expects the operator-created seed at `deploy/docker/etc-nxdns/config.zon`
(minimal: a `default` group + one enabled upstream), else exit 2.

**U2** delivered docs/operator.md (397 lines; systemd + docker + Pi 5 recipes, seeding
semantics, auth, TLS, backup/restore, data-dir table, full CLI reference, exit codes,
troubleshooting incl. the systemd-resolved DNSStubListener recipe) and README.md
(72 lines, no license section — no LICENSE exists). All facts source-verified.

**U3** delivered docs/config-reference.md (12 scalar sections, 9 collections, DB-vs-file
truth model, auth section, minimal + annotated examples) and surfaced five code
discrepancies during writing (see fix wave below).

**U4** delivered docs/architecture.md (module map from the //! headers, purity rule
with the honest exceptions, life-of-one-query pipeline verified against handler.zig,
storage split, web stack, CertStore design, failure-visibility doctrine). Reported one
stale comment (logging.zig:17 cited a moved cli.zig line) — orchestrator fixed the
comment to cite start.zig:724 via std.process.Init.

**U5** delivered docs/api.md: all 56 operations (matches router.routes.len), auth /
rate-limit / SSE prose, error envelope, openapi.yaml pointer. No code-vs-yaml
discrepancies found.

**Orchestrator wiring (ruling 3)**: docs/docs.zig (embeds api.md, config-reference.md,
operator.md), `docs_files` anonymous import on the test module in build.zig,
src/docs_drift_test.zig with three containment guards (routes → api.md; toSettings
keys → config-reference.md; the six subcommand names → operator.md), tests.zig import.

**Fix wave (orchestrator-triaged; ruling 8's no-runtime-code rule lifted for exactly
these)** — U3's five discrepancies, triaged with stdlib evidence:
1. `runtime.io_backend` DELETED end to end (model, settings handler + view, openapi,
   web types/SettingsPage/settingsDiff + tests, docs). Nothing consumed it — main uses
   init.io (stdlib Threaded, start.zig:724), and 0.16's std.Io.Evented has stubbed
   networking (Uring.zig netConnectIp → error.NetworkDown), so PLAN decision E's
   "io_uring via flag" is not deliverable at this tag. Re-add when std ships working
   evented net. Old DB rows warn-and-ignore via fromSettings' unknown-key path.
2. `upstream.connect_timeout_ms` DELETED. No call site; the pool races the whole
   attempt against total_timeout (app.zig sets it from totalTimeout); Threaded panics
   on IpAddress.ConnectOptions.timeout != .none; std.http.Client has no knob. The
   validate cross-check is now total >= read only.
3. `cache.size` KEPT: 0 is clean documented disabled behavior (put short-circuits;
   in-file test "a cache of zero entries stores nothing"). Doc row corrected.
4. `dns.bind_ipv6` TIGHTENED: checkBind generalized to a BindFamily enum; dns.bind_ipv6
   requires an IPv6 literal (an IPv4 wildcard there made the v4 bind AddressInUse get
   swallowed with a false "dual-stack" log — silent IPv6 loss). web/TLS binds stay .any.
5. doh/dot `readTimeout(.{})` sites KEPT: they are test fixtures; the real idle budget
   is the ruled 10s Options default. The doc claim was wrong and was removed.
Settings key counts after deletion: toSettings emits 43 (incl. web.password_hash);
the API-visible restart-required set is 42 (was 44).

## Review (Codex, as built)

Three rounds on one thread; round 3 returned "No findings."

Round 1 (5 important): stale-DB bind_ipv6 rows bypassed the new validate check at boot
→ app.zig parseBind now enforces the IP family on both dns binds (BadBindAddress,
exit 2, remedy in the message; the v4 side had the symmetric hole); app.zig's tests
were not collected by tests.zig at all — the import line was added and the new test
runs. operator.md's TLS recipe assumed the nxdns host user → Docker path now chowns
65532:65532 numerically host-side (read-only bind mount). PLAN.md still promised the
io_uring flag and connect_timeout_ms → synced (Io bullet records the drop with stdlib
evidence; decision E row; example config). Drift guards were maskable → api.md guard
anchors the full "| METHOD | `pattern` |" row per operation; operator.md guard anchors
the "### `name" reference headings.

Round 2 (1 important, 1 minor): seed-permission guidance covered only web.password →
now web.password or web.password_hash (a restored export); PLAN.md's "stubs all
networking" overstated 0.16's Uring — now names the stubbed operations precisely.

Final gates: plain 1171/1284 passed, 113 skipped (integration-gated), 0 failed;
integration 1280/1284, 4 skipped (live-network by design), 0 failed; cross ReleaseSafe
with the SPA dist 18/18; web suite 121/121 with format/lint/typecheck clean.

## Module layout (new)

deploy/systemd/{nxdns.service,sysusers.conf}, deploy/docker/{Dockerfile,compose.yaml,
.dockerignore}, docs/{operator,architecture,config-reference,api}.md, README.md,
src/docs_drift_test.zig (orchestrator).

## File ownership

U1 deploy/* + ci.yml; U2 docs/operator.md + README.md; U3 docs/config-reference.md;
U4 docs/architecture.md; U5 docs/api.md; orchestrator src/docs_drift_test.zig,
build.zig (docs_files module), src/tests.zig.

## Acceptance (milestone complete)

- [ ] All files in the module layout exist with the ruling-7 content contracts met.
- [ ] Docs drift tests pass: route coverage in api.md, settings-key coverage in
      config-reference.md, subcommand coverage in operator.md.
- [ ] CI docker job builds the image and the container smoke passes (in CI; locally
      if the daemon is available).
- [ ] `systemd-analyze verify deploy/systemd/nxdns.service` clean, or its
      unavailability recorded in the U1 report.
- [ ] Both suites 0 failed; cross ReleaseSafe with the SPA dist green.
- [ ] No runtime-behavior changes (git diff over src/ shows only docs_drift_test.zig
      + wiring).

## Anti-requirements

- No vendored API-doc renderer (redoc/scalar/swagger-ui), no YAML parser.
- No registry publish step; no k3s manifests (the infra repo owns deployment there).
- No SIGHUP/reload feature, no env-var config, no new CLI flags — document what
  exists; report gaps instead of filling them.
- No LICENSE invention; no badges or marketing prose in README.