# Milestone 11: packaging, ops and docs (PLAN Phase 10)

Goal: systemd unit, Dockerfile + compose, and the four docs (operator, architecture, config-reference, API) — the documented deployment must work end-to-end; docs are drift-guarded where a guard is cheap and honest.

## Rulings (binding)

1. **Layout.** `deploy/systemd/nxdns.service` + `deploy/systemd/sysusers.conf`; `deploy/docker/{Dockerfile,compose.yaml,.dockerignore}`; `docs/{operator.md, architecture.md,config-reference.md,api.md}`; `README.md` at the root (the repo has none; a portfolio repo needs a front door — short: what, why, quickstart, doc links). PLAN.md:236 sketches docs/ subdirectories; single files need no subdirectories.

2. **API docs = hand-written `docs/api.md` + drift test.** No renderer is vendored (redoc/scalar are exactly the dependency liability AGENTS.md refuses), and a build-time YAML parser for rendering is scope the yaml does not justify — the yaml itself is already served unauthenticated at `GET /api/openapi.yaml` (routes.zig:49) and is the exhaustive contract. `docs/api.md` gives human-readable orientation: auth model (cookie session, login flow), rate limiting, error envelope, SSE semantics, then one line per operation (method, path, auth, one-sentence purpose) and a pointer to the yaml for schemas. A drift test asserts every served route appears textually in api.md (mirror of openapi.zig:34's guard). This satisfies m8 ruling 3's deferred "docs/api rendering" as the engineering call: rendered = readable, guarded, in-repo; not = a vendored JS bundle.

3. **Docs drift guards.** New `src/docs_drift_test.zig` (ORCHESTRATOR-owned, written after the doc sessions land): (a) every route in `router.routes` appears in docs/api.md; (b) every settings key from `model.toSettings` (the 44 keys) appears in docs/config-reference.md; (c) every CLI subcommand name appears in docs/operator.md. Docs embedded via a `docs_files` anonymous import added in build.zig (test_fixtures pattern, build.zig:61). Guards are textual-containment only — cheap, zero false authority.

4. **systemd unit.** `Type=simple` (no forking, shutdown.zig:35 handles SIGTERM), `ExecStart=/usr/local/bin/nxdns run`, stderr → journald (logging.zig:301 already states this; `logging.output=stderr` stays the default). Static system user `nxdns` via `deploy/systemd/sysusers.conf` (`u nxdns - "nxdns DNS sinkhole"`), NOT DynamicUser — the TLS key must be chown-able to a stable uid ("TLS keys readable by service user only", PLAN §19). `StateDirectory=nxdns` (0700 matches cli.zig:226), `LogsDirectory=nxdns` (covers logging.output=file; the binary does not create the directory, logging.zig:502), `ConfigurationDirectory=nxdns`. `AmbientCapabilities=CAP_NET_BIND_SERVICE` + `CapabilityBoundingSet=` the same (port 53; 443/853 covered by the same cap). Hardening: `NoNewPrivileges=yes`, `ProtectSystem=strict`, `ProtectHome=yes`, `PrivateTmp=yes`, `PrivateDevices=yes`, `ProtectKernelTunables/Modules/Logs=yes`, `ProtectControlGroups=yes`, `ProtectClock=yes`, `ProtectHostname=yes`, `ProtectProc=invisible`, `RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX`, `RestrictNamespaces=yes`, `RestrictRealtime=yes`, `RestrictSUIDSGID=yes`, `LockPersonality=yes`, `MemoryDenyWriteExecute=yes` (static Zig binary, no JIT), `UMask=0077`, `SystemCallFilter=@system-service`, `SystemCallArchitectures=native`, `Restart=on-failure`, `RestartSec=2`. No `ReadWritePaths` beyond what StateDirectory/LogsDirectory grant. Validate with `systemd-analyze verify` if the build host has it; report honestly if not.

5. **Docker.** Multi-stage: builder stage only stages `ca-certificates` (upstream TLS verification rescans the system CA bundle, tls_client.zig:216 — a scratch image without a bundle breaks every DoH/DoT upstream); final `FROM scratch` with the static musl binary, `/etc/ssl/certs/ca-certificates.crt`, a nonroot numeric `USER 65532:65532`, `VOLUME /var/lib/nxdns`, `EXPOSE 53/udp 53/tcp 8080 443 853`, `ENTRYPOINT ["/nxdns"]`, `CMD ["run"]`. The binary is NOT built inside the Dockerfile (the repo builds it with zig; the Dockerfile COPYes `zig-out/cross/$TARGETARCH-…/nxdns` via a build arg or buildx TARGETARCH mapping — keep it working for both arches). compose.yaml: ports 53:53/udp+tcp and 8080:8080 (443/853 commented), bind-mount `./etc-nxdns:/etc/nxdns:ro`, named volume for `/var/lib/nxdns`, `sysctls: net.ipv4.ip_unprivileged_port_start=0` so the nonroot user binds 53 (per-netns sysctl; documented), `restart: unless-stopped`. First boot needs a seeded `/etc/nxdns/config.zon` with a `default` group + one enabled upstream or the container exits 2 (bootstrap.zig:38, app.zig:93) — operator.md and a compose comment both say so. Do NOT point the host's resolv.conf at nxdns itself for the container's own lookups.

6. **CI.** One added job `docker` in ci.yml: after building the x86_64 exe (ReleaseSafe, with the SPA dist like the cross job), `docker build` the image and run a container smoke (seed a minimal config.zon; `nxdns version` + boot + one `dig`-equivalent via the test client or `curl` on 8080/api/health; SIGTERM 0). No registry push — no publish step exists anywhere in this repo or the infra repo's CI, and registry credentials are an infra decision outside this repo. Manual publishing to git.mial.net stays possible and is documented in operator.md in one paragraph.

7. **Docs content contracts.**
   - operator.md: install (systemd path and docker path, both complete), first boot + config.zon seeding semantics (file seeds DB once, DB is truth thereafter, bootstrap.zig:38), auth setup (`web.password` hashed on import, never stored, export writes ""), TLS cert/key provisioning + 0600 expectations + the reload API/watcher, backup/restore = `nxdns export`/`import --force` (+ the 0600 export mode and why), upgrades (schema migration = install + restart, PLAN §20.11), data-dir layout table, exit codes (0/1/2/64, cli.zig:35), CLI reference (all six subcommands + flags), troubleshooting (exit 2 causes, `nxdns check` semantics incl. source-selection order cli.zig:511, disk-full degradation, port 53 conflicts with systemd-resolved — include the disable recipe).
   - architecture.md: module map (the src/ inventory), the purity rule (dns/, filter/, local/, cache/ take bytes, no Io — AGENTS.md), std.Io injection + Threaded backend, data flow for one query (listener → handler → filter/cache/local → upstream → sink/logger), storage split (config.db truth / querylog.db expendable), web stack (std.http over TLS optional, SPA embedded via web_assets, SSE), cert hot-reload design (refcounted CertStore), failure-visibility doctrine (counters + /metrics over log spam). Concise — a map, not a novel.
   - config-reference.md: complete — every section/field with type, default, unit, validation range, and which subsystem consumes it; collections with required fields; DB-vs-file truth explanation; the `web.password`/`password_hash` exclusivity; `logging.level=.err` serializes as "error" (model.zig:154). The explorer inventory in this milestone's research is the skeleton; verify against model.zig/validate.zig while writing, do not trust the summary blindly.
   - api.md: per ruling 2.
   - README.md: ≤120 lines; what nxdns is, feature list (honest, shipping features only), quickstart (docker compose path), build-from-source (zig build, node for the SPA), doc links, license note if a LICENSE exists (do not invent one).

8. **No new runtime code.** This milestone adds zero behavior to the binary. The only src/ change is the orchestrator's docs_drift_test.zig + its build wiring. If a doc session finds a bug while documenting, it REPORTS it (no fix); the orchestrator triages.

9. **Pi 5 end-to-end**: the exit criterion runs on hardware this environment does not have. The deliverable here is: both suites green, docker smoke green on x86_64, `systemd-analyze verify` clean (or honestly reported unavailable), aarch64 binary built and statically verified (existing cross job). The operator doc's Pi 5 recipe is written to be executed by the user; the spec records this boundary explicitly.

## Sessions

U1, U3, U4, U5 parallel; U2 after U1 (documents the artifacts U1 produces); orchestrator wiring (ruling 3) after U3+U5.

## Session U1: deploy artifacts + CI

Owns `deploy/systemd/nxdns.service`, `deploy/systemd/sysusers.conf`, `deploy/docker/{Dockerfile,compose.yaml,.dockerignore}`, `.gitea/workflows/ci.yml` (one added job). Rulings 4, 5, 6. Verify: `systemd-analyze verify` (or report unavailable), local `docker build` + container smoke if the docker daemon is reachable (report honestly either way), `zig build test` untouched-green.

## Session U2: operator.md + README.md (after U1)

Owns `docs/operator.md`, `README.md`. Rulings 7 (operator + README). Reads U1's artifacts and the runtime/CLI facts from the code (verify against src/cli.zig, src/app.zig, src/config/bootstrap.zig — not from memory).

## Session U3: config-reference.md

Owns `docs/config-reference.md`. Ruling 7. Source of truth: src/config/model.zig + validate.zig + import/export/bootstrap. Every field, no sampling.

## Session U4: architecture.md

Owns `docs/architecture.md`. Ruling 7. Reads module headers; no deep dives needed beyond what the doc claims.

## Session U5: api.md

Owns `docs/api.md`. Rulings 2, 7. Source of truth: src/web/routes.zig (the served table: method, path, auth, limiter) + openapi.yaml summaries + auth.zig/sse.zig for the auth and SSE prose. Every route, no sampling.

## As built

**U1** delivered per rulings 4-6 with accepted deviations: `StateDirectoryMode=0700` (systemd defaults 0755; the binary cannot tighten a pre-existing directory) and `User=nxdns`/`Group=nxdns` added to the unit; the Dockerfile's builder stage also maps buildx TARGETARCH → cross-target dir and pre-chowns `/var/lib/nxdns` to 65532 (a named volume seeded from a root-owned image dir would be unwritable on first boot); compose gained a `build:` block; `deploy/docker/.dockerignore` is documentation-grade under BuildKit (only a root `.dockerignore` or `Dockerfile.dockerignore` is honored — the file's header says so). Verified: `systemd-analyze verify` clean modulo the off-host ExecStart path (an ExecStart=/bin/true copy verifies exit 0); full local docker build + smoke passed (binds 53 as uid 65532, /api/health ok, SIGTERM exit 0). The CI docker job probes both 127.0.0.1 and the container IP to survive either runner topology. Compose expects the operator-created seed at `deploy/docker/etc-nxdns/config.zon` (minimal: a `default` group + one enabled upstream), else exit 2.

**U2** delivered docs/operator.md (397 lines; systemd + docker + Pi 5 recipes, seeding semantics, auth, TLS, backup/restore, data-dir table, full CLI reference, exit codes, troubleshooting incl. the systemd-resolved DNSStubListener recipe) and README.md (72 lines, no license section — no LICENSE exists). All facts source-verified.

**U3** delivered docs/config-reference.md (12 scalar sections, 9 collections, DB-vs-file truth model, auth section, minimal + annotated examples) and surfaced five code discrepancies during writing (see fix wave below).

**U4** delivered docs/architecture.md (module map from the //! headers, purity rule with the honest exceptions, life-of-one-query pipeline verified against handler.zig, storage split, web stack, CertStore design, failure-visibility doctrine). Reported one stale comment (logging.zig:17 cited a moved cli.zig line) — orchestrator fixed the comment to cite start.zig:724 via std.process.Init.

**U5** delivered docs/api.md: all 56 operations (matches router.routes.len), auth / rate-limit / SSE prose, error envelope, openapi.yaml pointer. No code-vs-yaml discrepancies found.

**Orchestrator wiring (ruling 3)**: docs/docs.zig (embeds api.md, config-reference.md, operator.md), `docs_files` anonymous import on the test module in build.zig, src/docs_drift_test.zig with three containment guards (routes → api.md; toSettings keys → config-reference.md; the six subcommand names → operator.md), tests.zig import.

**Fix wave (orchestrator-triaged; ruling 8's no-runtime-code rule lifted for exactly these)** — U3's five discrepancies, triaged with stdlib evidence:
1. `runtime.io_backend` DELETED end to end (model, settings handler + view, openapi, web types/SettingsPage/settingsDiff + tests, docs). Nothing consumed it — main uses init.io (stdlib Threaded, start.zig:724), and 0.16's std.Io.Evented has stubbed networking (Uring.zig netConnectIp → error.NetworkDown), so PLAN decision E's "io_uring via flag" is not deliverable at this tag. Re-add when std ships working evented net. Old DB rows warn-and-ignore via fromSettings' unknown-key path.
2. `upstream.connect_timeout_ms` DELETED. No call site; the pool races the whole attempt against total_timeout (app.zig sets it from totalTimeout); Threaded panics on IpAddress.ConnectOptions.timeout != .none; std.http.Client has no knob. The validate cross-check is now total >= read only.
3. `cache.size` KEPT: 0 is clean documented disabled behavior (put short-circuits; in-file test "a cache of zero entries stores nothing"). Doc row corrected.
4. `dns.bind_ipv6` TIGHTENED: checkBind generalized to a BindFamily enum; dns.bind_ipv6 requires an IPv6 literal (an IPv4 wildcard there made the v4 bind AddressInUse get swallowed with a false "dual-stack" log — silent IPv6 loss). web/TLS binds stay .any.
5. doh/dot `readTimeout(.{})` sites KEPT: they are test fixtures; the real idle budget is the ruled 10s Options default. The doc claim was wrong and was removed. Settings key counts after deletion: toSettings emits 43 (incl. web.password_hash); the API-visible restart-required set is 42 (was 44).

## Review (Codex, as built)

Three rounds on one thread; round 3 returned "No findings."

Round 1 (5 important): stale-DB bind_ipv6 rows bypassed the new validate check at boot → app.zig parseBind now enforces the IP family on both dns binds (BadBindAddress, exit 2, remedy in the message; the v4 side had the symmetric hole); app.zig's tests were not collected by tests.zig at all — the import line was added and the new test runs. operator.md's TLS recipe assumed the nxdns host user → Docker path now chowns 65532:65532 numerically host-side (read-only bind mount). PLAN.md still promised the io_uring flag and connect_timeout_ms → synced (Io bullet records the drop with stdlib evidence; decision E row; example config). Drift guards were maskable → api.md guard anchors the full "| METHOD | `pattern` |" row per operation; operator.md guard anchors the "### `name" reference headings.

Round 2 (1 important, 1 minor): seed-permission guidance covered only web.password → now web.password or web.password_hash (a restored export); PLAN.md's "stubs all networking" overstated 0.16's Uring — now names the stubbed operations precisely.

Final gates: plain 1171/1284 passed, 113 skipped (integration-gated), 0 failed; integration 1280/1284, 4 skipped (live-network by design), 0 failed; cross ReleaseSafe with the SPA dist 18/18; web suite 121/121 with format/lint/typecheck clean.

## Module layout (new)

deploy/systemd/{nxdns.service,sysusers.conf}, deploy/docker/{Dockerfile,compose.yaml, .dockerignore}, docs/{operator,architecture,config-reference,api}.md, README.md, src/docs_drift_test.zig (orchestrator).

## File ownership

U1 deploy/* + ci.yml; U2 docs/operator.md + README.md; U3 docs/config-reference.md; U4 docs/architecture.md; U5 docs/api.md; orchestrator src/docs_drift_test.zig, build.zig (docs_files module), src/tests.zig.

## Acceptance (milestone complete)

- [ ] All files in the module layout exist with the ruling-7 content contracts met.
- [ ] Docs drift tests pass: route coverage in api.md, settings-key coverage in
      config-reference.md, subcommand coverage in operator.md.
- [ ] CI docker job builds the image and the container smoke passes (in CI; locally
      if the daemon is available).
- [ ] `systemd-analyze verify deploy/systemd/nxdns.service` clean, or its
      unavailability recorded in the U1 report.
- [ ] Both suites 0 failed; cross ReleaseSafe with the SPA dist green.
- [ ] No runtime-behavior changes (git diff over src/ shows only docs_drift_test.zig
      + wiring).

## Anti-requirements

- No vendored API-doc renderer (redoc/scalar/swagger-ui), no YAML parser.
- No registry publish step; no k3s manifests (the infra repo owns deployment there).
- No SIGHUP/reload feature, no env-var config, no new CLI flags — document what exists; report gaps instead of filling them.
- No LICENSE invention; no badges or marketing prose in README.
