mokhtar
5b3d1cd65c
Gates / frontend (push) Successful in 1m2s
Gates / test (push) Successful in 1m38s
Gates / package (push) Successful in 5m5s
Gates / test-aarch64 (push) Successful in 6m30s
Gates / container (push) Successful in 15s
CI / gates (push) Successful in 13m30s
302 lines
58 KiB
Markdown
302 lines
58 KiB
Markdown
# Milestone 8: web server, REST API, SSE, auth, metrics (PLAN Phase 8, Zig side)
|
||
|
||
Goal: the complete Zig web layer — HTTP server + router, every REST handler, SSE live query stream, optional argon2id auth with sessions, API token-bucket rate limiting, `/metrics`, `/api/health`, OpenAPI served + contract tests, asset embedding + dev-mode disk serving — running as one more task in app.zig's group.
|
||
|
||
Ground truth: PLAN.md:610-612 (phase text), :537-550 (endpoints), §3.11/§12.1/§19 (auth), §10:333 (API limiting), §11.4:455 (SSE precedes persistence), §13.2 (OpenAPI), scratchpad notes m8-explore-{plan,repo,zig}.md. Load-bearing Zig facts are restated inline; verify anything else against /home/mokhtar/app/zig tag 0.16.0.
|
||
|
||
## Rulings (binding)
|
||
|
||
1. **Sequencing**: this milestone is the Zig web layer end to end. The React SPA (PLAN §3.14, §14 — ten pages) is milestone 9, built against this milestone's finished, contract-tested API. Not a scope cut: the asset embedding, dev-mode disk serving, ETag and gzip machinery all ship NOW and are complete; milestone 9 only swaps the dist content. The embedded dist in this milestone is a minimal real page (see ruling 24) — machinery is not stubbed.
|
||
2. **`POST /api/certs/reload` is Phase 9's**, whole: endpoint, openapi.yaml entry and docs land together with the DoH/DoT server it reloads. No 501 stub, no dangling contract entry.
|
||
3. **`docs/api/` rendering is Phase 10** (the docs phase). Phase 8 serves the yaml.
|
||
4. **Server model**: per-connection `std.http.Server` (Server.zig:25) over our own accept loop, copied from lib/std/Build/WebServer.zig:152-185: listener task in the app group; an inner `Io.Group` of connection tasks; keep-alive loop on `receiveHead` with `error.HttpConnectionClosing => return`. Cancel semantics mirror tcp_server's S3 As-built: on cancellation the inner group is CANCELED, not awaited (a keep-alive client must not hold shutdown open); on listener close (`SocketNotListening`) it drains.
|
||
5. **Bind**: one listener socket on `web.bind:web.port` exactly as configured (default 0.0.0.0:8080). No dual-stack ceremony — milestone 7's parity ruling was about DNS client identity, not the admin UI. An operator who wants v6 sets `web.bind = "::"` (dual-stack by Linux default).
|
||
6. **`web.enabled = false`** skips the entire subsystem, `/metrics` included. No web task, no web DB connections.
|
||
7. **Limits**: recv buffer 8 KiB (this caps the request head — Server.zig:32); send buffer 4 KiB; request body cap 1 MiB (`Io.Limit`, read via `readerExpectNone`/`allocRemaining`); 64 concurrent connections (accept beyond that: respond 503 and close — never silently drop). No per-request timeout this milestone (LAN-facing; the cancel path bounds shutdown); documented in server.zig.
|
||
8. **JSON shape**: snake_case field names everywhere (matches settings keys and SQL). Error envelope `{"error":"<message>"}`. Status codes: 400 validation, 401 unauthenticated, 404 missing, 405 wrong method (with Allow), 409 conflict (duplicate key), 413 body too large, 429 rate limited (with Retry-After), 500 internal (generic message, detail to log as warn), 503 over connection cap.
|
||
9. **Resource paths** (freezing PLAN:541's ellipses): collections `GET`+`POST` and items `GET`+`PUT`+`DELETE` by numeric row id for: `/api/groups[/{id}]`, `/api/blocklists[/{id}]` (the sources table), `/api/rules[/{id}]`, `/api/local-records[/{id}]`, `/api/forward-zones[/{id}]`, `/api/upstreams[/{id}]` (PLAN's list omitted upstreams; the Settings page must edit them; new resource, same pattern). Group-source assignment: `PUT /api/groups/{id}/sources` with `{"source_ids": [..]}` — idempotent full-set replace. Clients: `GET /api/clients` (ALL rows, materialized included, each with `hand_edited`), `GET/PUT/DELETE /api/clients/{id}`; PUT sets `hand_edited=1` and may change name/group; DELETE removes the row (a live client re-materializes). Client prefixes: `GET/PUT /api/client-prefixes` as a whole-list resource (tiny table, atomic replace). No POST for clients — creation is by DNS activity or import (PLAN:540 gives clients no POST deliberately).
|
||
|
||
> **Placement deviation (milestone 17 ruling 3).** "The Settings page must
|
||
> edit them" above names the wrong page. Milestone 9 dropped the obligation
|
||
> entirely; milestone 17 restored it as a dedicated **Upstreams** page with
|
||
> its own nav entry (`web/src/features/upstreams/`, route `/upstreams`),
|
||
> matching the house resource-page pattern that every other collection
|
||
> follows. The Settings page keeps the scalar settings keys only. The API
|
||
> contract in this ruling is unchanged.
|
||
10. **Repo layer**: every list row the API serves carries its row id; each mutated resource gains `getX(db, id)`, `updateX(db, id, item)`, `deleteX(db, id)` (strict: 0 rows touched → error.NotFound), written in the house repo idiom with prepared statements. Existing import-path functions stay frozen.
|
||
11. **`GET /api/queries`**: keyset pagination `?limit` (default 100, max 1000) + `?before=<row id>`, ordered id DESC; filters `domain=` (substring), `client=` (exact), `blocked=` (bool), `since=`/`until=` (unix seconds). Response `{"queries":[...], "next_before": <id>|null}`. Row fields: id, ts, domain, client_ip, qtype, blocked, block_reason, response_time_us, cache_hit, upstream.
|
||
12. **Live vs restart**: mutations to rules, blocklists, groups, group-sources, clients and client-prefixes call `Manager.reload(io)` and take effect live (the handler's next `acquire` sees the new snapshot). Local records and forward zones ALSO apply live: a new `src/server/local_tables.zig` RCU holder (acquire/release + swap, mirroring the manager's pattern at small scale) owns `{records, zones}`; `Handler` reads through it instead of `*const` fields; the local-records/forward-zones handlers rebuild and swap. Upstream mutations and everything in `/api/settings` are restart-required. `POST /api/blocklists/update` = `Manager.refreshAll` then reload, 202 with the status snapshot.
|
||
13. **Stats grammar**: `period=1h|24h|7d|30d` (else 400). `/api/stats` returns totals {queries, blocked, cached, clients (distinct), avg_response_time_us} for the period. `/api/stats/timeseries` buckets: 1h→60×1m, 24h→48×30m, 7d→168×1h, 30d→120×6h; UTC; each bucket {ts, queries, blocked, cached}. SQL aggregates over query_log on the web task's own read connection.
|
||
14. **`GET /api/lookup?domain=&group_id=`** (group optional, default group when absent): normalizes, then reports the full pipeline view: `{domain, group_id, local_records: bool, forward_zone: <zone>|null, blocked, reason, matched, source_url|null, safe_search_rewrite: <target>|null}` via `snapshot.evaluate` + records/zones lookups. Null snapshot → 503 `{"error":"no snapshot loaded"}`.
|
||
15. **Pause API**: `GET /api/pause` → `{"paused": bool, "until": <unix s>|null}` (null while unpaused OR indefinite — disambiguated by `paused`). `POST /api/pause` body `{"paused": true, "duration_seconds": <u32, optional>}` or `{"paused": false}`.
|
||
16. **`GET/PUT /api/settings`**: the typed config sections that live in settings rows (dns, blocking, cache, edns, upstream, logging, disk, blocklist_update, safe_search flag home — read model.toSettings for the exact key list), minus `web.password*` (never serialized; a PUT carrying `web.password` re-hashes via the import path's argon2id params and stores only the hash). GET marks every key `restart_required: true` except none — ALL settings are restart-required this milestone (live behavior comes from the resource endpoints and pause; a static table in the handler carries the flag so the UI banner has transport). PUT validates the merged config through `validate.validate` before writing any row; partial updates allowed.
|
||
17. **Auth**: enabled iff `web.password_hash != ""`. Login verifies with `std.crypto.pwhash.argon2.strVerify(hash, password, .{ .allocator = gpa }, io)` (argon2.zig:619 — PHC string carries its params). Sessions: in-memory fixed table of 32 (LRU evict), storing SHA-256 of a 32-byte `io.randomSecure` token; lookup compares digests with `std.crypto.timing_safe.eql([32]u8, ...)` (slices not accepted — timing_safe.zig:12). Cookie `nxdns_session=<url_safe_no_pad base64>`; `HttpOnly; SameSite=Lax; Path=/`; `Secure` NOT set (nxdns serves plain HTTP; TLS termination is the operator's proxy — documented). Expiry `web.session_ttl_hours`. Logout deletes the session. A `PUT /api/settings` that changes `password_hash` clears every session.
|
||
18. **Auth exemptions**: `/api/health`, `/api/version`, `/metrics`, `/api/openapi.yaml`, and the static assets are always unauthenticated (monitoring endpoints; the SPA shell must load to show a login form). Everything else 401s without a valid session when auth is enabled. `/api/auth/login` is necessarily exempt; failed logins count and are rate limited like any request.
|
||
19. **API rate limiting**: token bucket per client IP (NOT the DNS fixed-window limiter — m6 ruling 8 reserved the bucket shape for the API): capacity and refill `web.api_rate_limit_per_min` per 60 s, 4096 tracked IPs (same fixed-table idiom as rate_limiter/tracker). Localhost exemption: NEW config field `web.api_localhost_exempt: bool = true` (PLAN §10 says "configurable" and §12.1 forgot the field; model + validate
|
||
+ settings key + export round trip — settings are kv rows, no migration needed). 429 + `Retry-After: <s>`. SSE: connect consumes a token AND respects `web.sse_max_connections_per_ip`; `/metrics` and `/api/health` are limiter-exempt (Prometheus must never see 429).
|
||
20. **SSE**: `GET /api/queries/live`, `text/event-stream` via `respondStreaming(&.{}, ...)` (EMPTY buffer — `BodyWriter.flush` does not flush the body writer's own buffer, http.zig:780; empty buffer makes every write go straight to the chunked drain), flush headers before the first event (test.zig:498 pattern). Fanout: milestone-6 ruling 4 concretized as `src/server/query_sink.zig`: `QuerySink { logger: *Logger, hub: ?*sse.Hub }` with `log(io, entry)` = transform once, publish to the hub, enqueue to the logger. Logger gains additive `transformed(entry) Entry` (pure) and `logTransformed(io, entry)`; existing `log` = both, frozen behavior. `Handler.logger: ?*Logger` becomes `sink: ?*QuerySink` (mechanical rename at the two call sites + app wiring). Hub: fixed 32 subscriber slots, each a 64-entry ring + `std.Io.Event`; publish never blocks (full ring → disconnect that subscriber; SSE clients auto-reconnect). Wire format: `retry: 3000` once, then `event: query` + `data: <JSON, same fields as /api/queries rows>` per entry; heartbeat comment `: ping` every 15 s from the subscriber task (Event.waitTimeout). Fanout precedes persistence (PLAN:455) — the sink publishes before enqueue.
|
||
21. **`/metrics`**: Prometheus text format 0.0.4, `nxdns_` prefix, `# HELP`/`# TYPE` lines. Counters: every `Handler.Stats` field (17, individually, `nxdns_dns_<field>_total`), logger (queries_dropped, rows_written, batches_gated), cache stats (6, under `Handler.cache_mutex`), DNS limiter stats (under `limiter_mutex`), tracker stats, retention stats, log-sink stats, `nxdns_blocklist_refreshes_gated_total`. Gauges: cache len + memory bytes, disk free/db/log bytes, tracked DNS clients, pending tracker clients, blocklist generation, `nxdns_up 1`. Per-upstream, labeled `{url="..."}`: up (available), success_rate, consecutive_failures, total_successes/_failures (copy `Pool.Snapshot` fields while holding the returned count — `last_error` is borrowed, copy immediately). No timestamps. Auth-exempt, limiter-exempt.
|
||
22. **`GET /api/health`**: `{"status":"ok"|"degraded","disk":{"state":..., "free_bytes":...,"db_bytes":...,"log_bytes":...,"sample_failures":...}, "upstreams":{"available":N,"total":M},"queries_dropped":N,"writer_failed":bool, "refreshes_gated":N,"snapshot_generation":N|null}`. Degraded iff disk state != ok, or zero upstreams available, or writer_failed. 200 either way (degraded is data, not an HTTP failure).
|
||
23. **OpenAPI + contract tests**: hand-written `src/web/openapi.yaml`, `@embedFile`d and served verbatim. No external validator (dependencies are liabilities): the contract IS a Zig table in the integration test — every route with method, auth requirement, a seeded request, expected status, and a response-shape struct that `std.json.parseFromSlice` must accept with `.ignore_unknown_fields = false`. Two drift guards: (a) the router exposes `pub const routes: []const RouteInfo`; a test asserts every entry's path+method appears textually in openapi.yaml; (b) the table covers every `routes` entry (count equality). CI gets one new job step running the existing `-Dintegration` suite (contract tests live inside it) — no new tooling.
|
||
24. **Static assets**: build.zig gains `-Dweb-dist=<path>` (default `web/dist-placeholder/`, committed, containing a real minimal `index.html` — current status via /api/health fetch, links to /metrics; plus favicon). A build step copies the dist dir via `b.addWriteFiles()` + generated `assets.zig` index (the fixtures anonymous-import pattern, per research notes — @embedFile paths must live inside the module root), exposing `pub const files: []const File{path, bytes, content_type, etag}`; etag = comptime hash. Serving: exact path match, `/` → index.html, SPA fallback (unknown non-/api path → index.html, 200 — TanStack Router needs it in M9), `ETag`/ `If-None-Match` → 304, `content-encoding: gzip` when a sibling `<name>.gz` exists in the dist AND the request accepts it. No Date/Last-Modified headers (std has no RFC 1123 formatter; ETag is strictly better for immutable embedded content). Dev mode: CLI flag `nxdns run --web-dev <dir>` serves from disk (no cache headers), bypassing the embed.
|
||
25. **Head-string trap**: `request.head.target` is invalidated by body reads (Server.zig:594/230). The router copies target into a stack buffer before any body read. Query strings: split on '?', `std.mem.splitScalar` for pairs, `Uri.percentDecodeInPlace` per value (std has no query iterator; std.Uri does not parse origin-form targets).
|
||
26. **Wiring**: one `web_server.serve` task in the app group, started last, canceled by the same group.cancel. `WebState` struct (in web/server.zig) of borrowed pointers assembled by app.serve: handler (stats + mutexes + cache/limiter), pause, tracker, manager, pool, monitor, logger, retention stats, local_tables, sink/hub, config arena view (web cfg), its OWN `config.db` + `querylog.db` connections (m7 ruling 21), version string, started timestamp. Web DB connections open only when `web.enabled`.
|
||
27. **No new schema migration**: query_log already carries what /api/queries needs; the read session verifies existing indexes and reports if a needed index is missing (then the orchestrator rules on adding migration v3 — do not add one silently).
|
||
28. **Client disconnect**: stream writes surface `error.WriteFailed` with the real cause in `stream_writer.err` (ConnectionResetByPeer / SocketUnconnected=EPIPE — MSG.NOSIGNAL is set, Threaded.zig:13071, no SIGPIPE masking needed). A failed write ends the connection task quietly (debug log at most): clients vanishing is normal.
|
||
29. **Input sanitation** (PLAN §19): every path/query/body input is length-capped and validated before SQL (prepared statements everywhere, already house rule); domain inputs run through matcher.normalize; no secrets ever logged (password, tokens, cookie values — assert by review, and the login handler logs only the client IP and outcome).
|
||
|
||
## Sessions
|
||
|
||
Wave 1 (parallel): W1 query-log reads, W2 repo CRUD, W3 web core, W4 auth+limiter, W5 sink+SSE hub+logger split. Wave 2 (after W3, parallel where files allow): W6 read handlers + metrics, W7 mutation handlers + local_tables + handler.zig sink/local_tables changes. Wave 3: W8 static assets + build.zig + openapi.yaml + router registration of everything. Wave 4: W9 app/cli wiring. Wave 5: W10 integration + contract tests + CI. The orchestrator wires src/tests.zig per wave and rules on anything a session flags.
|
||
|
||
---
|
||
|
||
## Session W1: query-log read layer
|
||
|
||
Owns: `src/storage/repositories/queries_repo.zig` (additive; BatchWriter and friends frozen).
|
||
|
||
```zig
|
||
pub const QueryRow = struct { id: i64, ts: i64, domain: []const u8, client_ip: []const u8,
|
||
qtype: ?u16, blocked: bool, block_reason: []const u8, response_time_us: ?i64,
|
||
cache_hit: ?bool, upstream: []const u8 };
|
||
pub const QueryFilter = struct { limit: u32 = 100, before: ?i64 = null,
|
||
domain_substring: ?[]const u8 = null, client: ?[]const u8 = null,
|
||
blocked: ?bool = null, since: ?i64 = null, until: ?i64 = null };
|
||
/// Rows come back newest-first. Strings are arena-allocated.
|
||
pub fn selectQueries(database: *db.Db, arena: Allocator, filter: QueryFilter)
|
||
db.Error!std.ArrayList(QueryRow)
|
||
pub const StatsTotals = struct { queries: u64, blocked: u64, cached: u64,
|
||
distinct_clients: u64, avg_response_time_us: ?i64 };
|
||
pub fn statsTotals(database: *db.Db, since: i64, until: i64) db.Error!StatsTotals
|
||
pub const Bucket = struct { ts: i64, queries: u64, blocked: u64, cached: u64 };
|
||
/// Fills `out` with fixed-width buckets covering [since, until); returns the count.
|
||
pub fn timeseries(database: *db.Db, since: i64, bucket_seconds: u32, out: []Bucket)
|
||
db.Error!usize
|
||
```
|
||
|
||
Read the real query_log schema first (querylog_schema.zig) — domain interning means a JOIN; verify what indexes exist and REPORT (ruling 27) if `ts` or the join needs one that is missing rather than adding a migration. Dynamic WHERE assembly must still use bound parameters only (build the SQL from fixed fragments, never interpolate values). Cap `limit` at 1000 in the repo too. Tests: in-memory db seeded via BatchWriter; filters, keyset paging across a boundary, bucket alignment, empty ranges, substring escaping (`%`/`_` in domain must not act as wildcards — use ESCAPE).
|
||
|
||
Acceptance: fmt/ast clean; temp-root (-lc -lsqlite3) green; frozen surface untouched.
|
||
|
||
### W1 As built
|
||
|
||
`pub const max_limit: u32 = 1000` exported. WHERE assembly: six comptime fragments copied into a comptime-sized stack buffer (overflow impossible by construction), one bare `?` per predicate, bind order = append order; `likePattern` escapes `%`, `_` and `\` with `ESCAPE '\'`. Ruling 27 discharged: EXPLAIN QUERY PLAN over all six query shapes rides idx_query_log_client / idx_query_log_ts / rowid — NO migration v3; the time-windowed page's temp B-tree re-sort and the DISTINCT count's temp B-tree are inherent and cheap at household volume. `statsTotals` computes the mean as sum/count integer division (db.Stmt has no float column reader; avg() returns REAL) — exact microseconds, null when no row recorded a time. NULL `block_reason`/`upstream` arrive as `""` (columnText convention; neither is ever written as an empty string). **W6 convention, ruled now: `""` serializes as `""`, not JSON null** — `upstream=""` already means "cache hit" per milestone-7 ruling 20 and clients branch on `blocked`/`cache_hit`, not on reason presence. `timeseries` returns `error.Misuse` for bucket_seconds == 0 or an i64 window overflow, returns 0 for an empty `out` without touching the database, and aligns buckets to `since` — the HANDLER passes a grid-aligned since per ruling 13. 14 new tests (43 in file).
|
||
|
||
---
|
||
|
||
## Session W2: repo CRUD by id
|
||
|
||
Owns every file in `src/storage/repositories/` EXCEPT queries_repo.zig (W1's), plus nothing else. Additive only; import-path functions frozen.
|
||
|
||
For groups, clients (+prefixes), upstreams, sources, rules, local records, forward zones: id-carrying list variants (or extend existing rows where the shape already has no consumers outside export — check callers first; export/import must keep compiling unchanged), `getX(db, id) db.Error!?XRow`, `insertXRow(db, item) db.Error!i64` (returns id; distinct from frozen import inserts where semantics differ — e.g. clients insert with hand_edited=1), `updateX(db, id, item) db.Error!void` (0 rows → error.NotFound), `deleteX(db, id) db.Error!void` (same), `setGroupSources(db, group_id, source_ids) db.Error!void` (transactional replace), `replaceClientPrefixes(db, items) db.Error!void`. Respect FK constraints: deleting a group with clients → error.Constraint surfaces as 409 at the handler; document each. `settings_repo`: `putSetting(db, key, value)` single-key upsert if absent. Tests per resource: round trip, NotFound on both update and delete, constraint surfacing, group-sources replace idempotence.
|
||
|
||
Acceptance: fmt/ast clean; temp-root green; `zig build test` green (export/import tests prove the frozen surface).
|
||
|
||
### W2 As built
|
||
|
||
New shared `crud.zig`: `execStrict` turns zero-rows-touched into `error.NotFound`. Accepted deviations: `getX(database, gpa, id) db.Error!?XRow` (rows hold heap strings); `insertClientRow`/`insertRuleRow` take `now_s` (NOT NULL columns, repos take no Io — the InsertContext.now mirror); client writes split `ClientInput{ip,name,group_id}` (create) vs `ClientEdit{name,group_id}` (edit) — **ip is not editable**: it is the identity upsertSeen matches, and ruling 9 promises only name/group on PUT; client_prefixes get list + replace only (whole-list resource, per-id would be unused generality); write shapes carry group_id (a missing group surfaces as the FK violation → 409, not an id-map miss); added `listGroupSourceIds` (the PUT needs a read counterpart; frozen listGroupSources speaks names). Constraint map for W7: deleting a group with clients → Constraint (clients.group_id has no ON DELETE; rules/prefixes/group-sources cascade); bad group_id on client/rule writes → Constraint; url/ip/zone/name uniques → Constraint; deletes of clients/upstreams/rules/ local-records/forward-zones can only be NotFound. `setGroupSources`: BEGIN IMMEDIATE, existence check first (empty set must not report success for a missing group), delete + distinct insert (set semantics dedupe); `replaceClientPrefixes` transactional whole-table, duplicate prefix REJECTED as Constraint (two rows for one prefix is a contradiction, not a set). Schema facts: `upstreams.tls_name` comes from migration v2 (config_schema.zig is v1 only!) — the openapi /api/upstreams entry needs the fifth field; `SourceRow` gained `is_suggested: bool = false` appended last (manager.zig literals compile unchanged, no column index moved). `putSetting` is a primary-key upsert — partial settings PUT needs no read-first and fires no constraint. 45 new tests + 3 crud.
|
||
|
||
---
|
||
|
||
## Session W3: web core — server, router, http plumbing
|
||
|
||
Owns: `src/web/server.zig`, `src/web/router.zig`, `src/web/http_util.zig`.
|
||
|
||
- server.zig: `WebState` (ruling 26 — declare the struct; fields it cannot yet point at get wired by W9), `pub fn serve(state: *WebState, io: std.Io) std.Io.Cancelable!void` — bind per ruling 5 (bind failure: warn and return — the DNS side must keep serving; app treats web bind failure as non-fatal, W9 documents), accept loop per ruling 4 with the Stop split, connection budget per ruling 7 (503 over cap), per-connection buffers (8 KiB recv / 4 KiB send), keep-alive loop, dispatch into router.
|
||
- router.zig: `RouteInfo { method: http.Method, pattern: []const u8, auth: enum {open, session}, handler: *const fn(...) }`; `pub const routes: []const RouteInfo` (ruling 23 depends on it); match = exact segments + one `{id}` numeric capture; 405 with Allow when the path matches another method; target copied before body reads (ruling 25); query-pair iterator + percent-decode helpers (in http_util, pure, tested hard: '+', '%2F', truncated %, overlong values → 400).
|
||
- http_util.zig: JSON respond helpers (`std.json.Stringify` streaming into a `Writer.Allocating` for content-length, or direct respond for small bodies), the error envelope, request-body reader with the 1 MiB cap (413), cookie parse/format, bearer of nothing else.
|
||
- The connection handler calls: limiter (W4, via a comptime-checkable interface field on WebState so W3 compiles before W4 lands — define `ApiLimiter` and `Sessions` as W3-owned INTERFACE structs? NO: simpler, W3 declares the WebState fields as `*auth.Sessions` / `*auth.ApiLimiter` types and W3 is built AFTER W4's file exists on disk in the same wave — to keep the wave parallel, W3 instead keeps auth/limiter checks behind two function pointers on WebState (`check_auth`, `check_limit`) that W9 wires; W3 tests them with test doubles. This is the one seam where indirection is warranted; document it.)
|
||
- Tests: router matching table, 405/404, query decoding, body cap, cookie round trip, keep-alive across two requests (loopback socket), 503 over cap, cancel-during-idle keep-alive returns promptly.
|
||
|
||
Acceptance: fmt/ast clean; temp-root green (may import std only + own files); no listener/handler file touched.
|
||
|
||
### W3 As built
|
||
|
||
Handler signature: `fn(state: *server.WebState, io: std.Io, request: *http_util.Request) http_util.HandlerError!void`, `HandlerError = {WriteFailed, HttpExpectationFailed, OutOfMemory}` — domain outcomes are status codes, only those three escape. `Request` folds the id capture in (`request.id: ?i64`; no separate params type) and carries a per-request arena. `RouteInfo` gained `rate_limit: enum{counted,exempt} = .counted` (ruling 19's exemptions as data, not path matching); WebState gained `fallback: ?HandlerFn` (SPA fallback is not a route entry; unmatched non-/api → fallback, unmatched /api → JSON 404) and `reload_fn` (W7's ruling-12 seam). routes.zig: W8 fills `pub const table`; router re-exports as `routes`; WebState.routes defaults to it, so the drift guards read the array the server matches. W4/W5 landed mid-session, so sessions/limiter/hub/sink are REAL typed pointers and check_auth/check_limit default to real implementations (`sessionAuth`, `bucketLimit`; `allowAll`/`neverLimit` exported as test doubles) — a half-wired server fails closed. serve returns void (cancellation is classified into the Stop enum; tcp_server precedent). Own validating percent-decoder — std.Uri's copies malformed escapes through as literal text (Uri.zig:172-192), ruling 29 wants 400; segments split before decode so %2F cannot forge a boundary. Loopback tests live in web/server_integration_test .zig (house gating pattern). Accepted tradeoff: over-capacity 503 is written and the socket closed without draining, so the client may see RST instead — draining would be an unbounded blocking read on an at-capacity accept loop (documented). RULINGS RECORDED: W7 may add the `local_tables` field + import to server.zig (two-line exception, W3 consents); Retention.stats is a cross-task data race — W6 owns storage/retention.zig additively this wave: atomic counters + `snapshotStats` (logger-counters pattern), and /metrics reads only through it. 39 tests.
|
||
|
||
**Amendment (post-W9 critical fix)**: `serveConn` normalizes the request head immediately after `receiveHead`: a body-carrying method (POST/PUT/PATCH) with neither Content-Length nor Transfer-Encoding gets `head.content_length = 0` before any dispatch or respond. RFC 9110 §8.6 defines such a request as having an empty body, but std leaves the head saying "unknown" and `respond` → `discardBody` (http/Server.zig:631) asserts one of the two is set — so a bare `curl -X POST` panicked the whole process, DNS included. Zero content-length satisfies every downstream path: `bodyReader` (http.zig:445) goes straight to `.ready`, `discardRemaining` sees EndOfStream, keep-alive survives, and `http_util.readBody` yields an empty slice. Regression test in server_integration_test.zig: raw POST without either header → well-formed response, empty body observed by the handler, fresh connection answered after, connection_errors 0.
|
||
|
||
---
|
||
|
||
## Session W4: auth + API limiter
|
||
|
||
Owns: `src/web/auth.zig`, `src/web/api_limiter.zig`, plus the `model.zig`/`validate.zig` additions for `web.api_localhost_exempt` (ruling 19) — model field, validate rule (none needed beyond type), settings key list, and the export/import round-trip expectations (check model.toSettings reflection picks it up automatically; add the settings-count test adjustments).
|
||
|
||
- auth.zig: `Sessions` fixed 32-slot store per ruling 17 (create → cookie value out; validate cookie → bool; logout; clearAll; expiry sweep on access; LRU evict), login verify via argon2 strVerify (io + allocator required — argon2.zig:600), token = 32 bytes randomSecure → url_safe_no_pad; storage = SHA-256 digest; compare timing_safe.eql([32]u8,...). `authEnabled(cfg) bool`. Pure unit tests with fixed tokens (seed io.random? no — inject the token bytes via a testable `createWithToken`).
|
||
- api_limiter.zig: token bucket per ruling 19, fixed 4096-key table (house idiom), awake clock, `check(now, key, is_localhost) Result{allowed, retry_after_s}`; localhost exempt when configured; `sse_connections` per-IP counter with `tryAcquireSse/releaseSse` against `sse_max_connections_per_ip`. Tests: refill math at boundaries, burst=capacity, exempt localhost, sse cap acquire/release, eviction.
|
||
|
||
Acceptance: fmt/ast clean; temp-root green; `zig build test` green (model/validate/export tests still pass with the new field).
|
||
|
||
### W4 As built
|
||
|
||
Sessions: digests only (SHA-256), full-slot scan on validate so work does not depend on match position; `verifyPassword` returns `Outcome{ok,denied,unavailable}` — `.unavailable` (broken PHC string, OOM) maps to 500, not 401; nothing secret is ever formatted (warn prints `@errorName` only). `createWithToken`/`validateAt` are the test seams. Cookie value is 43 chars url_safe_no_pad; `cookie_attributes = "HttpOnly; SameSite=Lax; Path=/"`. Limiter deviations, accepted: (1) `check(io, now, addr)` — the limiter holds its OWN std.Io.Mutex (W3 dispatches connections concurrently; one lock inside beats N outside) and derives loopback itself via exported `isLoopback`, so callers cannot disagree; (2) refill counts millionths of a token and advances the clock only by the span converted, so the truncated remainder survives — at 1 req/min the 60 s boundary is exact (tests at 999/1000 ms); (3) the SSE per-IP cap binds loopback clients too — `api_localhost_exempt` exempts the RATE, not the fixed hub-slot resource. Full table → unknown addresses allowed + `untracked` (DNS-limiter precedent); `sweep` drops only full, SSE-free, window-idle buckets; `retry_after_s` never 0 on refusal. Model ripple: `web.api_localhost_exempt` in Web + expected_keys + fixture; validate.zig needed no edit (bool; decodeValue already rejects non-bool text); no count assertion outside W4 files existed. 12 auth + 14 limiter tests.
|
||
|
||
---
|
||
|
||
## Session W5: query sink, SSE hub, logger split
|
||
|
||
Owns: `src/server/query_sink.zig` (new), `src/web/sse.zig` (new), `src/storage/logger.zig` (additive split ONLY: `transformed(entry) Entry` pure + `logTransformed(io, entry)`; existing `log` becomes transform+logTransformed with byte-identical behavior — the existing tests must pass unchanged), and `src/server/handler.zig` (mechanical: field `logger: ?*logger_mod.Logger` → `sink: ?*query_sink.QuerySink`, the log call site, tests updated to construct a sink around their logger).
|
||
|
||
- query_sink.zig per ruling 20: publish BEFORE enqueue (PLAN:455).
|
||
- sse.zig `Hub`: 32 slots × 64-entry Entry rings + Event per subscriber; `subscribe() ?SubscriberId` / `unsubscribe(id)`; `publish(io, entry)` copies the entry into every live ring (Entry is self-contained, ~400 B; a full ring marks the subscriber `overflowed` and sets its event — the subscriber task sees the flag and ends the response); `wait(id, timeout)` for the heartbeat loop. All under one mutex; publish is called on the DNS hot path — it must stay allocation-free and short (copy + flag set). Tests: publish/consume order, overflow disconnect flag, unsubscribe under load, heartbeat timeout returns empty.
|
||
- Handler tests: one added test proving the sink publishes and logs (tiny hub + logger).
|
||
|
||
Acceptance: fmt/ast clean; temp-root green including ALL existing handler and logger tests; `zig build test` + `-Dintegration` green (phase7 tests construct Handlers — they compile against the renamed field; update them, they are in your ownership for THIS mechanical change only — list every touched line in the report).
|
||
|
||
### W5 As built
|
||
|
||
Every Hub method takes `io` (the mutex needs it); all locking is `lockUncancelable` (milestone-7 tracker precedent: `handle` has no error union). `Hub.init(self) void` initializes IN PLACE — 32×64 rings ≈ 900 KiB, so **W9 must `gpa.create(Hub)`, never a stack local**. Read surface ruling 20 left open, as built: `next(io, id) ?Entry` (pop oldest), `overflowed(io, id) bool` (sticky; pre-overflow entries stay readable — the subscriber drains then disconnects), `wait(io, id, timeout) Cancelable!Wake` with `Wake = {ready, timeout}`; `wait` resets the event under the mutex only while the ring is empty, so a racing publish is never lost, and a spurious futex wake reports `.timeout` (costs one heartbeat). Publish-before-persist is pinned structurally (enqueue never blocks, so a clock cannot observe order): the test closes the logger queue and the entry still reaches the hub while counting `queries_dropped`. W5 also made the minimal app.zig compile fix (sink around the logger, hub = null, `.sink = &sink`) — **W9: the sink already exists in app.zig; only the hub and the rest of the wiring are missing.** 11 sse + 4 sink
|
||
+ 1 logger-split + 1 handler test; all pre-existing logger/handler tests unchanged.
|
||
|
||
---
|
||
|
||
## Session W6: read handlers + metrics + health + version + openapi route
|
||
|
||
Owns: `src/web/handlers/stats.zig`, `queries.zig`, `lookup.zig`, `upstream_health.zig`, `health.zig`, `version.zig`, `metrics.zig` (in web/, not handlers/ — PLAN layout :227).
|
||
|
||
Implement rulings 11, 13, 14, 21, 22; `GET /api/version` = version.zig string + git commit build option. Every handler is `fn(state, io, request-ish, params) !void` matching W3's handler signature; JSON via http_util. Metrics: mind the mutex discipline (cache/limiter stats under the handler's mutexes; pool snapshot copies borrowed strings immediately; atomics via .monotonic loads). Handlers read the querylog via state's own connection. Tests: pure formatting tests per handler where the data is injectable (metrics text golden test, health degraded matrix, stats bucket math via W1 fixtures on an in-memory db).
|
||
|
||
Acceptance: fmt/ast clean; temp-root green.
|
||
|
||
### W6 As built
|
||
|
||
Every handler is split into a pure core plus a thin `handle`, so the decisions are tested without an `http.Server.Request`: `metrics.collect`/`metrics.render`, `health.collect`/ `rollup`, `stats.window`/`periodParam`, `queries.parseFilter`/`page`, `lookup.evaluate`/ `body`, `upstream_health.collect`, `version.body`. Entry points for W8's table: `metrics.handle`, `stats.totals`, `stats.timeseries`, `queries.list`, `lookup.handle`, `upstream_health.handle`, `health.handle`, `version.handle` — all match `router.HandlerFn` (proven by a temp-root assignment). `/metrics` and `/api/health` are `auth = .open, rate_limit = .exempt`; `/api/version` is `.open, .counted`; the other four are `.session, .counted`. **W8 must add `_ = @import("web/metrics.zig")` and the six handler files to src/tests.zig** — this session did not wire the orchestrator's file. Retention (ruled in W3's As built): `Stats` is now the plain snapshot type and the live counters moved to a private `Counters` of `std.atomic.Value(u64)` in `Retention.counters`; `snapshotStats()` reads them `.monotonic`; `runOnce` derives its pass number from the `fetchAdd` result rather than re-reading. Test assertions changed mechanically at retention.zig lines 192-195, 215, 220, 234, 238-240, 243-244, 258-260, 283-286, 312-313 and at phase6_integration_test.zig lines 365-368 (`x.stats.f` → `x.snapshotStats().f`) — that second file is the only consumer outside retention.zig, and it compiles solely under `-Dintegration`, so the plain suite does not catch a break there. No behavior changed. Decisions and deviations, all accepted:
|
||
- `/metrics` carries the DNS counters as an ARRAY keyed by `Handler.Stats`'s comptime field list, so a new counter in the handler appears in the exposition with no edit here; the same `counterGroup` helper walks every plain stats struct. Missing collaborator = the whole family is omitted (a zero would read as health, an absent series reads as a gap).
|
||
- Added beyond ruling 21's list: `nxdns_disk_sample_failures_total` (a failure mode that ruling 22 already surfaces) and `nxdns_upstream_enabled` (an upstream that is down and one that is switched off are different operator problems). Logger `writer_failed` is deliberately NOT a metric: it is in `/api/health` and would need a gauge family of one.
|
||
- `metrics.poolSnapshot` is the one place that copies pool health, shared with the health rollup: cancel protection around `Pool.snapshot` (server.zig's precedent), then every borrowed string duped into the request arena immediately. Cache/limiter stats are read under `Handler.cache_mutex`/`limiter_mutex` with `lockUncancelable` — `HandlerError` has no `Canceled`, so the W5 precedent applies.
|
||
- Ruling 13's window: `until` = end of the bucket `now` falls in, `since = until - width × count`, both on the absolute epoch grid (every width divides 86 400, asserted at comptime), so `/api/stats` and `/api/stats/timeseries` cover the identical span — tested by summing the buckets against the totals. Both bodies also carry `period`, `since` and `until` (a chart cannot label an axis without them); the timeseries body adds `bucket_seconds`. **openapi.yaml must document those fields.**
|
||
- `/api/queries`: `limit` outside 1..1000 is a 400 rather than a silent clamp, `before` must be positive, and each bad parameter has its own message. `next_before` is the last row's id only on a full page. `domain=`/`client=` empty read as absent.
|
||
- `/api/lookup` resolves `group_id` through `groupIndexById`; an id no snapshot group has is a 400 (`unknown group_id`), a missing manager or snapshot is 503 `no snapshot loaded`, and the domain runs through `name.fromText` + `matcher.normalize` (ruling 29). `source_url` comes from `sources_repo.getSource` on the config connection keyed by the snapshot's source row id — `SourceSets` carries the display name, not the URL; an unreadable row leaves the field null rather than failing the lookup. Records and zones are read through `state.handler.?.local_tables` (acquire/release bracketed by one defer), NOT a WebState field, because that pointer already exists and is the same table W7 swaps.
|
||
- `/api/upstream/health` reports url, enabled, available, consecutive_failures, total_successes, total_failures, success_rate, last_error, plus `available`/`total` rollup counts. No timestamps: they are `awake`-clock stamps and mean nothing to a client.
|
||
- `/api/version` adds `zig_version` and `uptime_seconds` (from `WebState.started_unix`) to the version and commit strings.
|
||
- The only logging in this session is one `warn` per failed database read or pool/source fault, on the 500 path; nothing is logged for a normal request, and no `std.log.err`. 40 new tests (metrics 5, stats 8, queries 9, lookup 7, health 4, upstream_health 3, version 4); retention's 6 tests unchanged in number and green.
|
||
|
||
---
|
||
|
||
## Session W7: mutation handlers + local_tables + pause + settings + auth handlers
|
||
|
||
Owns: `src/web/handlers/` `groups.zig`, `blocklists.zig`, `rules.zig`, `local.zig`, `clients.zig`, `settings.zig`, `pause.zig`, `auth.zig` (login/logout), plus `src/server/local_tables.zig` (new, ruling 12) and the `src/server/handler.zig` edit to read records/zones through it (second mechanical handler change; coordinate: W5 already edits handler.zig — W7 runs AFTER W5 lands; sequence inside wave 2).
|
||
|
||
- local_tables.zig: `LocalTables { lock: std.Io.RwLock, records, zones }` with `acquire/release` (shared) and `swap(io, new_records, new_zones)` (exclusive; frees the old under the lock after swap — no reader can hold across queries since acquire/release brackets each query). Handler: acquire in handle, release on every exit (same single defer discipline as the snapshot).
|
||
- Handlers: rulings 9, 12, 15, 16, 17-18 behaviors; every mutation validates (shared validators — reuse validate.zig section rules by constructing the candidate row set; where validate.zig only does whole-config, extract the per-section check it already has — validate.zig is NOT in your ownership: if extraction is needed, report it and use a local check this milestone), writes via W2 repos on state's config connection, then reload/rebuild per ruling 12. blocklists/update → refreshAll + reload, 202 + status. settings PUT: merge, validate whole config, write keys, clear sessions on hash change.
|
||
- Tests: per handler with in-memory dbs and a real Manager where cheap, else assert repo effects + reload-called flag via a seam (a `reload_fn` pointer on WebState set by W9; tests inject a counter).
|
||
|
||
Acceptance: fmt/ast clean; temp-root green; phase7 integration still green.
|
||
|
||
### W7 As built
|
||
|
||
Eleven files, 4199 lines, 89 tests (local_tables 5, mutations 6, groups 13, blocklists 8, rules 7, local 12, clients 9, upstreams 6, pause 7, settings 10, auth 6).
|
||
- **The apply/respond split (house pattern for later waves).** `http_util`'s response helpers need a live `*http.Server.Request`, which a unit test cannot construct. Every route splits into an `apply*` decision function driven against an in-memory database and a thin HTTP wrapper.
|
||
- **`mutations.zig` is a new shared file, owned by W7.** It holds the `Failure` union, the constraint map (NotFound → 404, Constraint → 409, validation → 400), the `reload` and `swapLocalTables` seams, and the `Bench` test fixture.
|
||
- **`config_lock` lives on WebState** (ruled post-report): connection tasks share one config database connection, and `changes()`/`lastInsertRowid()` are connection state that `execStrict` reads, so writes serialize through `state.config_lock`.
|
||
- **Per-row validation**: a minimal valid skeleton config plus the one candidate row runs the real `validate.validate`; no validator duplicated. Duplicate keys and foreign-key violations surface from the database as 409, not 400.
|
||
- **`upstreams.zig` is W7's** (ruling 9 listed the resource; neither ownership list named it). It never calls the reload seam, refuses to delete the last enabled upstream (409), and reports `restart_required: true`.
|
||
- **Ruled additions**: `GET /api/settings` returns a derived `web.auth_enabled` flag; the default group cannot be renamed or deleted (409); a group that still holds clients cannot be deleted (409).
|
||
- **handler.zig**: `records`/`zones` replaced by `local_tables: ?*LocalTables = null`; `handle` acquires once per query, releases on defer — one query reads one generation of records, zones and the filter snapshot together. `Context` gained `records`/`zones`.
|
||
- **Ruled mechanical ripples**: web/server.zig two-line exception (import + WebState field, plus the config_lock field above); app.zig minimal fix (`var tables: LocalTables = .empty` + build calls — **W9 must review**); udp/tcp/resolver integration tests lost their `empty_records`/`empty_zones` consts and two `bareHandler` initialisers; phase7 test converted to real tables where needed.
|
||
- **Routes for W8** (all match W3's signature; id routes read `request.id.?`): groups `list get create update remove getSources putSources`; blocklists `list get create update remove refresh` (refresh = POST /api/blocklists/update, 202 + status snapshot); rules `list get create update remove`; local `listRecords getRecord createRecord updateRecord removeRecord listZones getZone createZone updateZone removeZone`; clients `list get update remove listPrefixes putPrefixes` (no create); upstreams `list get create update remove`; pause `get post`; settings `get put`; auth `login logout`.
|
||
|
||
---
|
||
|
||
## Session W8: static assets, build pipeline, openapi.yaml, route table completion
|
||
|
||
Owns: `src/web/static.zig`, `src/web/openapi.zig`, `src/web/openapi.yaml`, `web/dist-placeholder/` (index.html + favicon.svg), `build.zig` (the -Dweb-dist option + asset module generation), and the final `router.zig` route-table registration IF W3 left registration data-driven (coordinate: W3 owns router.zig — W8 only ADDS the route array entries file `src/web/routes.zig` if W3 designed it that way; otherwise W8 hands W3's owner a patch — resolve by making W3 expose `routes` as a separate file owned by W8 from the start; W3 must read this paragraph).
|
||
|
||
Ruling 24 in full: asset module via addWriteFiles + generated index; etag comptime hash; gzip sibling serving; SPA fallback; --web-dev disk serving (cli flag lands in W9 — W8 exposes `serveFromDisk(dir, ...)`). openapi.yaml documents EVERY route in the table with schemas matching the handlers' JSON (snake_case, error envelope, auth markers, 429/401 responses). Tests: static matching, 304 flow, gzip negotiation, SPA fallback vs /api 404, placeholder embeds and serves.
|
||
|
||
Acceptance: fmt/ast clean; `zig build` with default dist green; temp-root green.
|
||
|
||
### W8 As built
|
||
|
||
Seven files: static.zig 362, openapi.zig 66, routes.zig 194 (55 routes), openapi.yaml 2255, handlers/live.zig 184 (ruled mid-session — no session owned the SSE HTTP handler), tools/gen_web_assets.zig 201 (new build tool, W8's), web/dist-placeholder (real page: fetches /api/health + /api/version, renders status/upstreams/disk). 21 new tests (static 9, openapi 3, routes 6, live 3).
|
||
- **Asset pipeline**: gzip cannot run in the build system itself, so `tools/gen_web_assets.zig` runs as a build step: dist staged through WriteFiles (Run hashes only a directory arg's path string; staging bakes the content hash into the path, so edits invalidate — proven), tool output merged with generated `.gz` siblings and `assets.zig` into the `web_assets` anonymous import (exe, cross exes, tests).
|
||
- **`web_assets` module surface**: `File{path, bytes, content_type, etag}`, `files: []const File`; root is the generated assets.zig.
|
||
- **ETag** computed in the tool, not comptime (same build-time constant; avoids the comptime interpreter hashing large M9 bundles). SHA-256/128-bit, quoted, strong. `.gz` siblings are their own `files` entries with their own ETag; direct `*.gz` paths are not addressable. The tool skips gz when it does not shrink or below 128 bytes.
|
||
- **static.zig surface for W9**: `state.fallback = static.fallback` (embedded) or wrap `static.serveFromDisk(dir, io, request)` in a HandlerFn for `--web-dev` (app.zig owns the dir storage; no cache headers in dev mode). Traversal guard in `diskRelativePath`.
|
||
- **live.zig** (SSE, ruling 20): respondStreaming with empty buffer, `retry: 3000`, `event: query` frames, 15 s heartbeat (awake clock) via `Hub.wait`, per-IP cap via `tryAcquireSse`/`releaseSse`, ring overflow ends the stream with a clean chunked terminator so EventSource reconnects. RULED: route is `.session, .exempt` (overrides ruling 19's connect-token line; the per-IP SSE cap is the guard, loopback included). RULED: the SSE payload omits `id` (a live entry precedes persistence); a comptime test pins the remaining nine field names to `QueryRow`.
|
||
- **Route policy**: open = {/metrics, /api/health, /api/version, /api/openapi.yaml, /api/auth/login}; limiter-exempt = {/metrics, /api/health, /api/queries/live}; logout is `.session` (ruling 18).
|
||
- **openapi.yaml** documents all 55 routes, cookie security scheme, 401 on every session route, 429 + Retry-After on every counted route, the W6 stats fields (period, since, until, bucket_seconds), SSE as text/event-stream; no certs/reload. openapi.zig's test enforces route ⊆ yaml at unit level; W10 still owns the count-equality drift guard (b).
|
||
|
||
---
|
||
|
||
## Session W9: app + cli wiring
|
||
|
||
Owns: `src/app.zig`, `src/cli.zig`, `src/main.zig` (if forced).
|
||
|
||
WebState assembly (ruling 26) gated on `web.enabled`; two extra DB connections; hub + sink construction (sink wraps logger always — hub only when web enabled; DNS path cost without web = one null check); local_tables construction replacing the direct records/zones locals; web serve task in the group (started last); reload_fn/check seams wired; `--web-dev <dir>` CLI flag (parseArgs + help text + runCheck untouched); web bind failure = warn + continue serving DNS (ruling in W3 — confirm and document in app). Shutdown order unchanged (web task dies with group.cancel; its connection group cancels per ruling 4). Smoke test yours: boot with web enabled, curl /api/health, /api/version, /metrics, /, login flow with a password set (import a config with web.password, verify cookie + 401 without), SIGTERM exit 0. Report the transcript.
|
||
|
||
Acceptance: fmt/ast clean; zig build; both test suites green; smoke transcript.
|
||
|
||
### W9 As built
|
||
|
||
- `cli.Command.run` payload is now `RunArgs { paths: Paths, web_dev: ?[]const u8 }`; `parseRunArgs` handles `--web-dev DIR` (both spellings); `check` rejects it; `runCheck` untouched. Anything spawning `app.run` passes `RunArgs`.
|
||
- app.zig serve order: zero-guard (`web.api_rate_limit_per_min`/`session_ttl_hours` zero on an enabled web → `error.BadRateLimit`, exit 2 — collaborators assert nonzero and a hand-edited database bypasses validate); heap `sse.Hub` via `gpa.create` + in-place init (W5 rule), gated on `web.enabled`; sink wraps `(&logger, hub)` ALWAYS (hub null without web = one branch); two web DB connections (openConfigDb + reopenQuerylogDb, after openQuerylogDb establishes the file), gated; Sessions + ApiLimiter, gated; WebState assembled after the DNS handler (`reload_fn = app.reloadManager`, `fallback = static.fallback` or the dev wrapper, version.string, started_unix); web task added to the group LAST; bind failure = warn + return, DNS keeps serving. Shutdown order unchanged. `web.enabled = false` → no hub, no web DBs, no sessions/limiter, no web task.
|
||
- `--web-dev` dir lives in a file-scope var in app.zig (`WebState.fallback` is a bare fn pointer, no closure; one composition root; written once before the task starts).
|
||
- W7's `var tables: LocalTables` holder confirmed correct: `WebState.local_tables` points at the same holder the DNS handler reads, so `swapLocalTables` swaps both.
|
||
- Mechanical ripple: phase7 test line 967 `cli.Paths{...}` → `RunArgs{ .paths = ... }`.
|
||
- Smoke test passed end to end: import with web.password → boot → 200 on /api/health, /api/version, /metrics, /, /api/openapi.yaml → 401 without cookie → login sets HttpOnly/SameSite=Lax cookie → authed read → logout kills the cookie → wrong password 401 → SIGTERM exit 0. No secrets in logs. `--web-dev` smoked: disk edits appear without restart.
|
||
- **Critical finding (fixed post-session in W3's files)**: a bodyless POST/PUT — no Content-Length, no Transfer-Encoding, exactly `curl -X POST` — hit the `discardBody` assert in std (Server.zig:631) and panicked the process, DNS included. RFC 9110 gives such a request an empty body, so the fix normalizes in the request path before any respond call (see W3 As-built amendment); W10 pins it with a contract test.
|
||
|
||
---
|
||
|
||
## Session W10: integration + contract tests + CI
|
||
|
||
Owns: `src/web/web_integration_test.zig` (new), `.gitea/workflows/ci.yml` (additive job steps only).
|
||
|
||
Contract table per ruling 23 covering every route; auth on/off matrix (seeded password_hash vs empty); SSE test (connect, cause one query via a real Handler or direct sink.publish, assert `event: query` frame + heartbeat + cap enforcement via sse_max_connections_per_ip=1); rate-limit 429 + Retry-After; pagination walk; mutation → reload observed (snapshot generation bump); pause via API affects a real handler decision; settings PUT round trip; static + ETag 304; drift guards (a) and (b). CI: ensure the integration job still covers everything (contract tests ride -Dintegration; add a `zig build test -Dintegration` step only if missing — read the current yml first).
|
||
|
||
Acceptance: `zig build test -Dintegration` green with zero err lines; plain suite green.
|
||
|
||
### W10 As built
|
||
|
||
web_integration_test.zig, 1406 lines, 14 tests: 11 integration-gated (contract walk over all 55 routes with strict `ignore_unknown_fields = false` response shapes, auth on, auth off, 429 + Retry-After, SSE, pagination walk, mutation → reload, pause, settings round trip, static/ETag 304, bodyless POST) + 3 ungated pure guards (contract-covers-routes, drift guard a route ⊆ yaml, drift guard b: 55 yaml operations == router.routes.len). ci.yml UNTOUCHED — its test job already runs `zig build test -Dintegration` (line 25).
|
||
- Environment fully real except two seams: real Manager/reload over an in-memory config DB, real Sessions/ApiLimiter/Hub (heap Hub per W5 rule), seeded querylog via BatchWriter. The pool transport and fetcher are never invoked — POST /api/blocklists/update runs in the walk BEFORE any source row exists (hermetic).
|
||
- SSE reader de-frames chunked transfer coding (the unbuffered SSE writer emits one frame as many chunks; std's chunked writer emits each chunk's closing CRLF lazily — reading separators eagerly deadlocks until the next heartbeat).
|
||
- Heartbeat asserted against the real 15 s interval (`live.heartbeat_interval` is not injectable): the SSE test runs ~15-20 s in a 40 s budget; the integration suite gained roughly 45 s of wall clock. Accepted rather than weakening the assertion.
|
||
- Pause test drives Handler.handle directly with real DNS packets (phase7 queryFor pattern): blocked → API pause → forwarded (paused_queries bumps) → unpause → blocked.
|
||
- Mutation-reload observed as generation 1 → 2 plus the rule live via GET /api/lookup.
|
||
- Contract markers asserted by lookup into router.routes (method+pattern → auth and rate_limit equality, exact one-to-one coverage); response shapes reference the handlers' pub types where they exist; the strict settings parse also proves `web.password*` never serializes.
|
||
- One expected `warn` line ("web login refused") from the wrong-password case; zero err lines suite-wide; no secrets logged.
|
||
|
||
---
|
||
|
||
## Module layout (new)
|
||
|
||
src/web/{server,router,http_util,auth,api_limiter,sse,static,openapi,metrics}.zig, src/web/routes.zig (W8), src/web/handlers/{auth,stats,queries,clients,groups,blocklists, rules,local,lookup,pause,settings,upstream_health,health,version}.zig, src/server/{query_sink,local_tables}.zig, src/web/openapi.yaml, web/dist-placeholder/, src/web/web_integration_test.zig.
|
||
|
||
## File ownership
|
||
|
||
W1 queries_repo; W2 other repositories/*; W3 web/{server,router,http_util}; W4 web/{auth,api_limiter} + model/validate additions; W5 server/query_sink, web/sse, storage/logger (additive), server/handler (sink rename) + phase7 test compile fixes; W6 web/handlers read set + web/metrics; W7 web/handlers mutation set (incl. mutations.zig, upstreams.zig) + server/local_tables + server/handler (local_tables read path; AFTER W5); W8 web/{static,openapi,routes} + openapi.yaml + dist-placeholder + build.zig; W9 app/cli/main; W10 its test file + ci.yml. Orchestrator: src/tests.zig, spec. Within-wave parallel sessions never share a file; handler.zig is touched by W5 then W7, strictly sequenced.
|
||
|
||
## Acceptance (milestone complete)
|
||
|
||
- [ ] Both suites + cross green; fmt clean; zero err-level lines in integration output.
|
||
- [ ] Every PLAN:537-550 endpoint except certs/reload (ruling 2) implemented, documented in
|
||
openapi.yaml, and contract-tested; auth on/off matrix green.
|
||
- [ ] SSE live stream works end to end with per-IP cap; fanout precedes persistence.
|
||
- [ ] /metrics scrapes clean (golden test); /api/health rollup correct.
|
||
- [ ] W9 smoke transcript: boot, API answers, login round trip, SIGTERM 0.
|
||
- [ ] Placeholder UI loads at /; assets ETag/304/gzip machinery proven.
|
||
- [ ] Spec As-built synced per session.
|
||
|
||
## Review (Codex, As built)
|
||
|
||
Round 1: 6 important + 3 minor, all fixed.
|
||
- local.zig: all six mutation paths hold `config_lock` across write AND rebuild+swap, so LocalTables swaps publish in database-write order (deadlock-checked: no callee takes the lock).
|
||
- settings/auth/server/app: GET /api/settings reads under `config_lock`. New `auth.LiveHash` (mutex holder on `WebState.live_hash`, generation-checked — see round 2): login copies the hash out under a short lock and verifies OUTSIDE it; settings PUT dupes the new hash with gpa BEFORE the transaction, installs + clears sessions after commit; `sessionAuth` gates on `live_hash.enabled()`, so a first password set via PUT locks routes without restart. Boot hash borrows the config arena (owned=false); replacements are gpa-owned, freed on the next install or deinit (app.zig defer / Bench.deinit / Env.destroy).
|
||
- mutations.zig `checkUpstream`: candidate validated beside a companion enabled upstream (URL collision-avoided), so disabled upstreams are creatable. upstreams.zig `applyUpdate` gained the missing last-enabled guard: disabling the last enabled upstream is 409 (delete path already had it; create needs none).
|
||
- clients.zig: prefixes canonicalized via platform/address.zig (parse zeroes host bits; RFC 5952 text) before a set-level duplicate check (409, constraint-map ruling) and whole-list validation through the real validate.validate; canonical text is stored.
|
||
- api_limiter.zig: full table fails closed — `bucketLocked` first evicts a bucket that refills to capacity with sse==0 (behaviorally identical to fresh; sweep's idle window is churn hysteresis, not correctness), else `check` refuses with retry_after and `tryAcquireSse` returns false. `untracked` is now a subset of `refused`.
|
||
- static.zig: `acceptsGzip` parses every entry per the header grammar (specific gzip beats wildcard; malformed q = entry unusable). Dev mode realpath-checks containment of target AND index fallback (no-follow-on-open would miss symlinked intermediate directories).
|
||
- web_integration_test.zig: drift guard (a) requires the method key inside the specific path's yaml block; a negative test doctors the yaml (method swap between two paths) and proves the guard bites.
|
||
|
||
Round 2: 1 important — login could copy the old hash, race a password-change PUT (install + clearAll), finish argon2 verification, and mint a session with the revoked password. Fixed via a generation counter on LiveHash: applyLogin verifies the copy outside any lock, then `confirmSession` checks the generation and inserts the session digest under LiveHash.mutex (lock order: live_hash → sessions, single direction at every site); a changed generation denies the login.
|
||
|
||
Round 3: 2 minors, both fixed. (a) install-then-clearAll had a gap where a legitimate new-password cookie could be minted and then wiped — `install` is replaced by `installAndRevoke(io, gpa, sessions, new_hash)`: swap, generation bump, and nested clearAll in one LiveHash-ordered operation; a confirm either precedes it (wiped) or follows it (stale generation, denied). (b) confirmSession no longer holds LiveHash.mutex across entropy/clock work: token (randomSecure, zeroed on exit) and timestamp are generated before the lock; only the generation check + `createWithToken` digest insert run under the ordered locks. `Sessions.create` removed (createWithToken is the seam).
|
||
|
||
Round 4: no findings.
|
||
|
||
## Anti-requirements
|
||
|
||
- No React/SPA content (milestone 9). No DoH/DoT server or certs/reload (Phase 9). No docs rendering (Phase 10). No WebSocket. No HTTP/2, no TLS on the web port. No external schema validator or yaml parser. No new DB schema migration without an explicit orchestrator ruling (27). No pause persistence. No per-request timeouts. No changes to dns/, filter/matcher, cache, or the DNS pipeline order.
|