Files
nxdns/specs/milestone-10.md
T

366 lines
23 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Milestone 10: DoH/DoT server + cert watcher + certs/reload (PLAN Phase 9)
Goal: LAN clients resolve via DoH (RFC 8484 over HTTP/1.1 + TLS) and DoT (RFC 7858)
against local certs; certs hot-reload via a polling watcher and `POST /api/certs/reload`
(endpoint + openapi entry + contract test, deferred whole from milestone-8 ruling 2).
Ground truth: PLAN.md:23,155,194,232,500-503,525,547,615-616,647,664 (scope, layout,
config keys, endpoint, exit); PLAN.md:684 — HTTP/2 and DoQ are OUT; PLAN.md:297 —
"a failed reload publishes neither". Explore facts (verified this session):
- platform/tls_server.zig: ServerContext.init(gpa, cert_pem, key_pem) shared across
connections; binding is per-accept (mbedtls_ssl_setup at accept), so reload = publish
a new context pointer; the OLD context must outlive every stream set up against it.
ServerStream.reader()/writer() are real *Io.Reader/*Io.Writer; close(gpa) sends
close_notify but does NOT close the TCP stream. MBEDTLS_THREADING_C + PTHREAD are ON
(build.zig:255) — one context is safe under concurrent handshakes.
- std.http.Server.init(in: *Reader, out: *Writer) takes ARBITRARY reader/writer
(http/Server.zig:25) — web/server.zig's serveConn transfers with three lines changed.
- No usable file watching in 0.16 (inotify/fanotify are raw syscalls outside Io) —
the watcher POLLS Io.File.stat mtime+size.
- Handler.handle(self, io, which: Transport, from, query, response_buf, scratch)
(handler.zig:164); Transport {udp,tcp} drives truncation only; response_buf 512..65535;
Scratch ~7 KiB per connection. tcp_server serveConn (tcp_server.zig:265-338) is the
DoT loop verbatim modulo the stream; dns/transport.zig has prefix_len/parsePrefix/
framePrefix/max_message_len.
- doh_client.zig: media_type "application/dns-message" (:16), contentTypeOk exported
(:171). base64 url_safe_no_pad = the RFC 8484 GET codec. http_util.queryPairs for ?dns=.
- Fixtures: tests/fixtures self_signed cert/key via the test_fixtures anonymous import.
- Config: model.TlsEndpoint already exists (doh_server port 443, dot_server 853),
validated, settings-persisted, checked by `nxdns check`. app.zig ignores both today.
## Rulings (binding)
1. **Two listeners**: `src/server/doh_server.zig`, `src/server/dot_server.zig`
(PLAN:194), constructed only when their endpoint is enabled. Bind failure = warn +
continue (web precedent). Both join app.zig's group; cancel semantics mirror
tcp_server's Stop enum (.canceled cancels the connection group; listener close
drains).
2. **DoH protocol**: HTTP/1.1 only (PLAN:684 excludes h2). Path `/dns-query` exactly.
`POST` with `content-type: application/dns-message` (checked via
doh_client.contentTypeOk) and `GET ?dns=<base64url unpadded>`. Responses:
200 `application/dns-message` with the reply bytes; handler `.drop` → 502 with empty
body reason? NO — `.drop` means "no answer on purpose" (malformed/refused-silent):
close the connection for POST/GET alike via 400. Errors: 404 non-/dns-query path,
405 other methods (Allow: GET, POST), 415 wrong content-type, 400 missing/invalid
`dns` param or empty body, 413 body > dns/transport.max_message_len. No
cache-control headers (LAN, no intermediaries). Keep-alive per std.http.Server loop;
the bodyless-POST normalization from web/server.zig:443 is copied (same stdlib
assert).
3. **DoT protocol**: RFC 7858 = the tcp_server loop over ServerStream: 2-byte prefix,
parsePrefix, reject 0, read body, handle, frame + write + flush. idle_timeout race
identical (same Options shape, max_connections 64, per-conn Scratch). The TLS
handshake itself runs under the same race budget (a stalled handshake must not pin a
connection slot).
4. **Transport enum stays {udp, tcp}**: DoH/DoT call handle with `.tcp` (it only
drives truncation, which never applies to stream transports). No unused surface.
5. **ALPN, additive to platform**: ServerContext.init gains
`alpn: ?[*:null]const ?[*:0]const u8` (mbedtls_ssl_conf_alpn_protocols requires a
NULL-terminated array that OUTLIVES the config — use comptime-constant arrays).
DoH advertises ["http/1.1"], DoT ["dot"]. A client with no ALPN still connects
(mbedTLS only enforces when the client sends the extension). Existing callers pass
null; the milestone-1 spec note about "no ALPN" is superseded.
6. **Cert holder + reload**: new `src/server/cert_store.zig`. `CertStore` owns
{mutex, current: *Entry, cert_path, key_path, gpa} where Entry = {ctx: ServerContext,
refs: u32, retired: bool}. `acquire(io) *Entry` (+1 under mutex),
`release(io, entry)` (1; free when retired and 0). `reload(io) ReloadError!void`:
read both PEM files (NUL-terminated dupes, 64 KiB cap each), ServerContext.init, on
success swap + retire the old (freed when its refs drain), on ANY failure publish
nothing (PLAN:297). Listeners acquire per connection (before accept) and release
when the connection ends — so in-flight streams keep their generation alive.
File read errors and init errors map to a typed error + a human message for the API.
7. **Cert watcher**: one task per enabled endpoint inside cert_store
(`watch(store, io)` house loop: Cancelable!void, .boot clock, warn-and-continue).
Poll every 30 s: stat cert AND key (Io.File.stat), compare mtime+size against the
last LOADED pair; any change → reload(); reload failure → warn (the old cert keeps
serving) + a `reload_failures` counter; success → info-level "certificate reloaded".
mtime+size both compared (same-second atomic renames).
8. **`POST /api/certs/reload`** (PLAN:547): handler `src/web/handlers/certs.zig`
(PLAN:232). auth = .session, rate_limit = .counted. Reloads every enabled endpoint's
store; response 200 `{"doh": {"enabled": bool, "reloaded": bool, "error": <msg>|null},
"dot": {...}}` — per-endpoint outcome, 200 even when a reload fails (the outcome IS
the payload; nothing about the server's own state is exceptional). Disabled endpoint:
{enabled:false, reloaded:false, error:null}. WebState gains `doh_certs: ?*CertStore`
and `dot_certs: ?*CertStore` (null-defaulted, W3 pattern).
9. **openapi.yaml + contract test land together** (m8 ruling 2): the path entry with
the exact schema, and a contract-table entry + response-shape test in
web_integration_test.zig. Drift guard (b) count moves 55 → 56 — the test derives the
count from router.routes so only the yaml operation count assertion data changes.
10. **Observability**: each listener keeps tcp_server-shaped stats (connections,
tls_handshake_failures, idle_timeouts, connection_errors) + cert_store keeps
{reloads, reload_failures, last_reload_unix}. metrics.zig additively gains
`nxdns_doh_server_*`, `nxdns_dot_server_*`, `nxdns_cert_reloads_total`,
`nxdns_cert_reload_failures_total` families (counterGroup walks the structs —
follow its existing pattern). /api/health is unchanged (no new rollup this
milestone; the reload endpoint reports cert state on demand).
11. **app.zig wiring**: when `cfg.doh_server.enabled`, build its CertStore (initial
load happens in serve BEFORE the group — a bad cert at boot is exit 2 with the
validate-style message, matching `nxdns check`'s contract that enabled endpoints
have readable certs; the WATCHER handles later breakage gracefully), then the
listener; same for dot. Watcher tasks + listener tasks join the group. WebState
seams wired when web is also enabled. SIGTERM path unchanged.
12. **Integration tests** (both gated `-Dintegration`, loopback, fixtures):
DoT: std.crypto.tls.Client (no_verification) → framed A query → framed reply
(reuse resolver_integration_test's fake-upstream pattern or a local-records-only
handler). DoH POST + GET: raw HTTP/1.1 text over std.crypto.tls.Client (no
std.http.Client CA ceremony), assert status/content-type/bytes; 415/405/404/400
matrix; keep-alive two-requests test. Reload: swap fixture cert (write a second
self-signed pair fixture), reload, NEW connection sees the new cert
(std.crypto.tls.Client exposes the peer cert? if not: assert old connections
still serve and new handshakes succeed — the observable contract), old connection
still answers. Watcher: unit-test the decision function (stat-pair compare) pure;
do not test 30 s timing.
13. **New fixture**: `tests/fixtures/self_signed_cert2.pem` + key2 (openssl one-liner,
same profile as the existing pair; README updated) for reload tests.
14. **Docs**: NOT this milestone beyond openapi.yaml (docs/api rendering is Phase 10 —
m8 ruling 3 unchanged).
15. **Zig freeze elsewhere**: dns/, filter/, cache/, storage/, existing server files
(udp/tcp/handler) untouched except: app.zig (T6), metrics.zig + routes.zig +
openapi.yaml + web_integration_test.zig + web/server.zig(WebState fields only)
(T5). handler.zig is NOT touched (ruling 4).
## Sessions
T1+T2 parallel first; then T3+T4+T5 parallel; then T6.
---
## Session T1: platform ALPN
Owns `src/platform/tls_server.zig` (+ its tests). Add the alpn parameter per ruling 5
(extern mbedtls_ssl_conf_alpn_protocols, comptime NULL-terminated arrays, doc the
lifetime rule), update existing callers/tests with null, add one test that a client
negotiating "http/1.1" succeeds against a context advertising it (the loopback
integration test grows the assertion — std.crypto.tls.Client: check whether it can
send ALPN in 0.16; if it cannot, the unit test just proves config acceptance and the
lifetime doc stands).
Acceptance: fmt/ast clean; plain + integration suites 0 failed.
### T1 As built
`ServerContext.init(gpa, cert_pem, key_pem, alpn: ?[*:null]const ?[*:0]const u8)`.
Extern `mbedtls_ssl_conf_alpn_protocols` verified against vendored 3.6.7 ssl.h:4314
(pointer recorded, not copied — comptime-constant NULL-terminated arrays required,
documented on init). MBEDTLS_SSL_ALPN is on in the stock config. std.crypto.tls.Client
CANNOT send ALPN in 0.16 (no Options field) — negotiation untestable client-side; the
unit test proves config acceptance, the integration echo proves no-ALPN clients still
handshake against an advertising context. Servers can assert server-side via
mbedtls_ssl_get_alpn_protocol if ever needed (extern not declared — unused surface).
Mechanical ripple: tls_client_integration_test.zig:193 caller gained `, null`.
## Session T2: cert_store
Owns `src/server/cert_store.zig` (new). Ruling 6 (holder + refcount + reload) and
ruling 7 (watch loop + pure stat-compare decision fn) + ruling 10's counters +
`snapshotStats()`. Unit tests: acquire/release refcounting across a swap (old entry
freed only after last release), reload failure publishes nothing (bad path, bad PEM),
NUL-termination and size cap, stat-compare decision table. Uses fixtures via
test_fixtures import.
Acceptance: fmt/ast clean; temp-root green (needs -lc + vendored mbedtls objects).
### T2 As built
16 in-file tests; temp-root 22 passed / 0 failed. Surface:
`poll_interval_s=30`, `max_pem_bytes=64KiB`, `ReloadError` (10 cases) +
`humanMessage(err) []const u8`; `FileSig{mtime_ns: i96, size}`, `Signature{cert,key}`,
pure `changed(loaded, observed) bool`; `Entry{ctx, refs, retired}`;
`CertStore.init(gpa, io, cert_path, key_path, alpn)` (boot load; typed error for T6's
exit-2), `deinit(io)` (asserts current refs==0 — join listeners first),
`acquire(io) *Entry` / `release(io, entry)` (release safe from any task; last releaser
frees a retired entry), `reload(io) ReloadError!void`, `watch(io) Cancelable!void`
(sleeps BEFORE first poll, .boot clock), `pollOnce(io)` (testable watcher pass),
`snapshotStats()` (no io; {reloads, reload_failures, last_reload_unix}).
Notes: use `&entry.ctx` for ServerStream.accept; paths are BORROWED (config outlives
the store); `reloads` counts successes only; stat failures warn but are not
reload_failures; `last_reload_unix` set at init too.
## Session T3: dot_server (after T1+T2)
Owns `src/server/dot_server.zig` (new) + its integration test file if separate
(prefer tests inside the file, house pattern). Ruling 3: tcp_server loop shape over
ServerStream; acquire cert entry per connection, release on exit; handshake under the
race budget; stats per ruling 10; Stop enum; max_connections 64; ALPN ["dot"].
Integration tests per ruling 12 (DoT half).
Acceptance: fmt/ast clean; both suites 0 failed.
### T3 As built
`DotServer.listen(gpa, io, listen_address, h: *handler.Handler, certs: *cert_store.CertStore, options)` /
`serve(io)` / `boundAddress()` / `deinit(io)`. Options `{max_connections: u16 = 64, idle_timeout = 10s}`;
the handshake runs under the same idle budget. Deviation from tcp_server: `deinit` takes no
gpa — the allocator is stored at `listen` because `ServerStream.accept` heap-allocates one
mbedTLS ssl context per connection (documented in the file header). Stats atomics:
the four ruling-10 fields plus tcp-pattern internals `rejected_at_capacity,
rejected_at_shutdown, accept_errors`; `snapshotStats()` returns exactly the ruling-10 four.
Loop: accept → claim slot → `certs.acquire` (released on every exit) → handshake raced
against the idle budget → 2-byte-prefix loop over ServerStream reader/writer, flush per
reply, handler called with `.tcp`, `tls.close(gpa)` (close_notify) on every post-handshake
exit. A `handshook` flag ensures the ssl context closes exactly once when Select reports
expiry/cancel after a successful handshake; a timed-out handshake counts as
`tls_handshake_failures`. ALPN is set only via the store (T6). 7 unit + 4 integration
tests in-file. Teardown order: `server.deinit(io)` before `CertStore.deinit`.
## Session T4: doh_server (after T1+T2)
Owns `src/server/doh_server.zig` (new). Ruling 2 in full: std.http.Server over
ServerStream reader/writer (recv buffer 8 KiB = head cap, send 4 KiB), connection
group + 64 cap + 503-free refusal is NOT applicable here (no HTTP yet at accept
overflow: over cap → close raw, cheapest honest behavior — document); per-connection
Scratch + response_buf (65535); GET base64url decode (url_safe_no_pad, reject padding/
whitespace), POST body cap via Io.Limit; keep-alive; bodyless-POST normalization;
ALPN ["http/1.1"]; cert acquire/release per connection; stats. Integration tests per
ruling 12 (DoH half: POST, GET, 415/405/404/400, keep-alive).
Acceptance: fmt/ast clean; both suites 0 failed.
### T4 As built
`DohServer.listen(gpa, io, address, *Handler, *CertStore, Options)` / `serve(io)` /
`deinit(gpa, io)` / `boundAddress()` / `snapshotStats()`. Options
`{max_connections = 64, idle_timeout = 10s}` — the budget races the TLS handshake only;
requests have no timeout (web-listener precedent). Module-level
`serve(gpa, io, model.TlsEndpoint, *Handler, *CertStore)` is the bind-warn-and-continue
entry for T6. Exposed comptime `alpn_protocols` (["http/1.1"]) and `dns_query_path`.
Stats: the ruling-10 four + `bad_requests` + tcp-shaped accept counters
(`rejected_at_capacity, rejected_at_shutdown, accept_errors`) — over-capacity refusals
must stay visible. Deviations: POST cap via content-length precheck + fixed buffer +
one-byte probe for chunked bodies instead of Io.Limit (no allocation on the hot path);
`respond` can raise `HttpExpectationFailed` (100-continue), mapped to `connection_errors`;
no 413 integration test (ruling 12's matrix is 415/405/404/400). GET `?dns=` parsed by a
local validating decoder (duplicate param → 400, `%` and padding/whitespace rejected).
Oversize POST answers 413 with `keep_alive=false` (body never drained). Over the 64 cap
the raw TCP stream closes (no HTTP before a handshake). 8 unit + 4 integration tests.
T6 recipe: `CertStore.init(gpa, io, cert_path, key_path, doh_server.alpn_protocols)`;
run `doh_server.serve` and `store.watch` as group tasks; `DohServer.deinit` before
`CertStore.deinit`.
## Session T5: web endpoint + metrics (after T2, parallel with T3/T4)
Owns `src/web/handlers/certs.zig` (new), `src/web/routes.zig` (one entry),
`src/web/openapi.yaml` (one path), `src/web/server.zig` (two null-defaulted WebState
fields ONLY), `src/web/metrics.zig` (ruling 10 families), and the additive
web_integration_test.zig edits (contract entry + shape test + guard-b count).
Handler per ruling 8 (apply/respond split, house pattern). Unit tests for the
apply fn (both stores null; one store failing reload → error message in payload,
still 200).
Acceptance: fmt/ast clean; plain suite 0 failed; drift guards green.
### T5 As built
handlers/certs.zig: apply/respond split — `applyReload(state, io) View` with
`View{doh, dot}` and `Outcome{enabled, reloaded, @"error": ?[]const u8}` (serialized as
`error`); `post` always answers 200. Route: `POST /api/certs/reload`, `.session`,
`.counted`; table 55→56; contract entry + strict shape test + drift guards green at 56.
metrics.zig: the two ruling-10 cert counters share one family pair
(`nxdns_cert_reloads_total` / `nxdns_cert_reload_failures_total`) with an
`endpoint="doh"|"dot"` label (upstream-label pattern); unwired stores omit the family;
`last_reload_unix` not exported. WebState gains `doh_certs`/`dot_certs:
?*cert_store.CertStore = null`. Deviation (accepted): deleted the m8 tripwire test in
`src/web/openapi.zig` ("the document does not promise what phase 9 owns") — it exists to
fire when this endpoint lands. Deferred to T6: `nxdns_doh_server_*` / `nxdns_dot_server_*`
listener families (T3/T4 types did not exist at T5 build time).
## Session T6: app wiring + fixtures + smoke (after all)
Owns `src/app.zig`, `tests/fixtures/` (cert2/key2 + README + fixtures.zig), and
`src/tests.zig` is the ORCHESTRATOR's (report lines). Ruling 11 wiring; ruling 13
fixture. Smoke (report transcript): boot with doh+dot enabled on unprivileged ports
with fixture certs; DoT query via a scripted TLS client (a tiny zig run or the
integration test binary is fine — no new tooling); DoH POST via curl --insecure
--http1.1 with a binary body; certs/reload via the API twice (success, then chmod the
key unreadable → error in payload, old cert still serving a new connection); SIGTERM 0.
Both suites + cross ReleaseSafe with the SPA dist.
Acceptance: full gates green; smoke transcript.
### T6 As built
app.zig realizes ruling 11 in-frame: `openCertStore` maps non-OOM `ReloadError` to
`error.BadCertificate` (exit 2, prints both paths + `cert_store.humanMessage`);
`bindDoh`/`bindDot` warn-and-continue per ruling 1; listeners + two `store.watch` loops
join the group; stores are declared before listeners so teardown runs listener-deinit →
store-deinit and the refs==0 assert holds. `dot_alpn` (["dot"]) lives in app.zig.
Deviation: `doh_server.serve` module entry is NOT used — it constructs the DohServer in
its own task frame, so WebState could never get the pointer the `nxdns_doh_server_*`
family needs; both listeners bind in app.zig's frame instead (tcp_server style).
`doh_server.serve` remains as unused pub API. metrics.zig: `DohListenerSample`
(ruling-10 four + `bad_requests`); DoT renders `dot_server.StatsSnapshot` directly;
accept-side counters stay off the exposition; unwired listeners omit the families.
WebState gains `doh_listener`/`dot_listener` optional pointers. build.zig (out of
ownership, necessary): the exe now compiles `mbedtls_shim.c` with the threading macros —
first exe-side tls_server reference; mirrors the tests wiring. Fixtures: cert2/key2
exported as `cert2_pem`/`key2_pem` (distinct fingerprint), currently unreferenced by
tests. Config: `model.TlsEndpoint` already existed; no model/validate/schema edits.
Smoke: DoT framed query answered from local records; DoH POST 200
application/dns-message; reload success then key chmod 000 → `reloaded:false` with
"private key file is not readable" while the old cert keeps serving; SIGTERM 0; boot
with unreadable key exits 2.
---
## Review (Codex, as built)
Four rounds on one thread; round 4 returned "No findings."
Round 1 (3 important, 2 minor): doh_server leaked the TLS context when the handshake
won a race the Select reported as timeout/cancel → handshook flag (dot_server pattern).
cert_store reloads were not serialized across read→build→publish, so an older read could
publish last → `reload_mutex` held across the whole sequence (lock order: reload_mutex
before the entry mutex; accept path untouched) + a hook-pinned regression test
(`after_load_hook` seam). doh bodied-GET smuggling: std leaves unframed body bytes for
bodyless methods to be parsed as the next request → `framesUnreadableBody` guard, 400 +
close, no byte read. `.drop` now closes per ruling 2. cert2/key2 were unreferenced →
DoT reload-to-distinct-identity integration test (old connection keeps serving, new
handshake succeeds, distinct entry pointers; the 0.16 tls Client does not retain the
peer chain, so identity is proven via entry pointers).
Round 2 (2 minor): the smuggling guard ran before routing and stole 404/405 statuses →
routing decides the status, framed requests only force close. Successful POSTs ignored
the client's `Connection: close` → answer/refuse honor `head.keep_alive` in the Next
enum (the wire already did via Server.zig:628).
Round 3 (1 important, introduced by the round-2 reshape): bodied refusals (404/405/415)
kept keep-alive, so respond drained the whole body first — an unterminated chunked body
could pin all 64 slots with no response → `framesBody` helper; any pre-body-read refusal
closes when a body is framed; drain-free integration tests (missing terminal chunk →
404+EOF at once; undelivered content-length 4096 → 415+EOF); errorMatrix's 415 became a
zero-length wrong-type POST so its keep-alive reuse stays legitimate.
Final counts: doh_server.zig 19 tests, dot_server.zig 12, cert_store.zig 17. Final
gates: plain 1166/1279 passed, 113 skipped (integration-gated), 0 failed; integration
1275/1279, 4 skipped (live-network by design), 0 failed.
## Module layout (new)
src/server/{doh_server,dot_server,cert_store}.zig, src/web/handlers/certs.zig,
tests/fixtures/{self_signed_cert2.pem,self_signed_key2.pem}.
## File ownership
T1 platform/tls_server.zig; T2 server/cert_store.zig; T3 server/dot_server.zig;
T4 server/doh_server.zig; T5 web/{handlers/certs.zig,routes.zig,openapi.yaml,
server.zig(fields),metrics.zig,web_integration_test.zig}; T6 app.zig + fixtures.
Orchestrator: src/tests.zig, spec. Parallel sessions never share a file.
## Acceptance (milestone complete)
- [ ] Both suites 0 failed (plain + -Dintegration), zero err lines; cross ReleaseSafe
with SPA dist green.
- [ ] DoT and DoH loopback integration tests green (query → reply, error matrix,
keep-alive).
- [ ] certs/reload endpoint + openapi entry + contract test land together; drift
guards green at 56.
- [ ] Cert watcher reloads on file change; failed reload keeps serving the old cert.
- [ ] T6 smoke transcript incl. reload-failure path and SIGTERM 0.
- [ ] Spec As-built synced per session.
## Anti-requirements
No HTTP/2, no DoQ, no SNI/multi-cert, no client-cert auth, no session tickets, no
proxy-protocol/XFF handling, no cert generation (ACME etc.), no cache-control on DoH,
no handler.zig changes, no docs/api rendering (Phase 10), no config schema changes
(TlsEndpoint exists), no inotify.