docs: unwrap hand-wrapped prose repo-wide
Gates / frontend (push) Successful in 1m2s
Gates / test (push) Successful in 1m38s
Gates / package (push) Successful in 5m5s
Gates / test-aarch64 (push) Successful in 6m30s
Gates / container (push) Successful in 15s
CI / gates (push) Successful in 13m30s
Gates / frontend (push) Successful in 1m2s
Gates / test (push) Successful in 1m38s
Gates / package (push) Successful in 5m5s
Gates / test-aarch64 (push) Successful in 6m30s
Gates / container (push) Successful in 15s
CI / gates (push) Successful in 13m30s
This commit is contained in:
+42
-276
@@ -1,122 +1,33 @@
|
||||
# Milestone 10: DoH/DoT server + cert watcher + certs/reload (PLAN Phase 9)
|
||||
|
||||
Goal: LAN clients resolve via DoH (RFC 8484 over HTTP/1.1 + TLS) and DoT (RFC 7858)
|
||||
against local certs; certs hot-reload via a polling watcher and `POST /api/certs/reload`
|
||||
(endpoint + openapi entry + contract test, deferred whole from milestone-8 ruling 2).
|
||||
Goal: LAN clients resolve via DoH (RFC 8484 over HTTP/1.1 + TLS) and DoT (RFC 7858) against local certs; certs hot-reload via a polling watcher and `POST /api/certs/reload` (endpoint + openapi entry + contract test, deferred whole from milestone-8 ruling 2).
|
||||
|
||||
Ground truth: PLAN.md:23,155,194,232,500-503,525,547,615-616,647,664 (scope, layout,
|
||||
config keys, endpoint, exit); PLAN.md:684 — HTTP/2 and DoQ are OUT; PLAN.md:297 —
|
||||
"a failed reload publishes neither". Explore facts (verified this session):
|
||||
- platform/tls_server.zig: ServerContext.init(gpa, cert_pem, key_pem) shared across
|
||||
connections; binding is per-accept (mbedtls_ssl_setup at accept), so reload = publish
|
||||
a new context pointer; the OLD context must outlive every stream set up against it.
|
||||
ServerStream.reader()/writer() are real *Io.Reader/*Io.Writer; close(gpa) sends
|
||||
close_notify but does NOT close the TCP stream. MBEDTLS_THREADING_C + PTHREAD are ON
|
||||
(build.zig:255) — one context is safe under concurrent handshakes.
|
||||
- std.http.Server.init(in: *Reader, out: *Writer) takes ARBITRARY reader/writer
|
||||
(http/Server.zig:25) — web/server.zig's serveConn transfers with three lines changed.
|
||||
- No usable file watching in 0.16 (inotify/fanotify are raw syscalls outside Io) —
|
||||
the watcher POLLS Io.File.stat mtime+size.
|
||||
- Handler.handle(self, io, which: Transport, from, query, response_buf, scratch)
|
||||
(handler.zig:164); Transport {udp,tcp} drives truncation only; response_buf 512..65535;
|
||||
Scratch ~7 KiB per connection. tcp_server serveConn (tcp_server.zig:265-338) is the
|
||||
DoT loop verbatim modulo the stream; dns/transport.zig has prefix_len/parsePrefix/
|
||||
framePrefix/max_message_len.
|
||||
- doh_client.zig: media_type "application/dns-message" (:16), contentTypeOk exported
|
||||
(:171). base64 url_safe_no_pad = the RFC 8484 GET codec. http_util.queryPairs for ?dns=.
|
||||
Ground truth: PLAN.md:23,155,194,232,500-503,525,547,615-616,647,664 (scope, layout, config keys, endpoint, exit); PLAN.md:684 — HTTP/2 and DoQ are OUT; PLAN.md:297 — "a failed reload publishes neither". Explore facts (verified this session):
|
||||
- platform/tls_server.zig: ServerContext.init(gpa, cert_pem, key_pem) shared across connections; binding is per-accept (mbedtls_ssl_setup at accept), so reload = publish a new context pointer; the OLD context must outlive every stream set up against it. ServerStream.reader()/writer() are real *Io.Reader/*Io.Writer; close(gpa) sends close_notify but does NOT close the TCP stream. MBEDTLS_THREADING_C + PTHREAD are ON (build.zig:255) — one context is safe under concurrent handshakes.
|
||||
- std.http.Server.init(in: *Reader, out: *Writer) takes ARBITRARY reader/writer (http/Server.zig:25) — web/server.zig's serveConn transfers with three lines changed.
|
||||
- No usable file watching in 0.16 (inotify/fanotify are raw syscalls outside Io) — the watcher POLLS Io.File.stat mtime+size.
|
||||
- Handler.handle(self, io, which: Transport, from, query, response_buf, scratch) (handler.zig:164); Transport {udp,tcp} drives truncation only; response_buf 512..65535; Scratch ~7 KiB per connection. tcp_server serveConn (tcp_server.zig:265-338) is the DoT loop verbatim modulo the stream; dns/transport.zig has prefix_len/parsePrefix/ framePrefix/max_message_len.
|
||||
- doh_client.zig: media_type "application/dns-message" (:16), contentTypeOk exported (:171). base64 url_safe_no_pad = the RFC 8484 GET codec. http_util.queryPairs for ?dns=.
|
||||
- Fixtures: tests/fixtures self_signed cert/key via the test_fixtures anonymous import.
|
||||
- Config: model.TlsEndpoint already exists (doh_server port 443, dot_server 853),
|
||||
validated, settings-persisted, checked by `nxdns check`. app.zig ignores both today.
|
||||
- Config: model.TlsEndpoint already exists (doh_server port 443, dot_server 853), validated, settings-persisted, checked by `nxdns check`. app.zig ignores both today.
|
||||
|
||||
## Rulings (binding)
|
||||
|
||||
1. **Two listeners**: `src/server/doh_server.zig`, `src/server/dot_server.zig`
|
||||
(PLAN:194), constructed only when their endpoint is enabled. Bind failure = warn +
|
||||
continue (web precedent). Both join app.zig's group; cancel semantics mirror
|
||||
tcp_server's Stop enum (.canceled cancels the connection group; listener close
|
||||
drains).
|
||||
2. **DoH protocol**: HTTP/1.1 only (PLAN:684 excludes h2). Path `/dns-query` exactly.
|
||||
`POST` with `content-type: application/dns-message` (checked via
|
||||
doh_client.contentTypeOk) and `GET ?dns=<base64url unpadded>`. Responses:
|
||||
200 `application/dns-message` with the reply bytes; handler `.drop` → 502 with empty
|
||||
body reason? NO — `.drop` means "no answer on purpose" (malformed/refused-silent):
|
||||
close the connection for POST/GET alike via 400. Errors: 404 non-/dns-query path,
|
||||
405 other methods (Allow: GET, POST), 415 wrong content-type, 400 missing/invalid
|
||||
`dns` param or empty body, 413 body > dns/transport.max_message_len. No
|
||||
cache-control headers (LAN, no intermediaries). Keep-alive per std.http.Server loop;
|
||||
the bodyless-POST normalization from web/server.zig:443 is copied (same stdlib
|
||||
assert).
|
||||
3. **DoT protocol**: RFC 7858 = the tcp_server loop over ServerStream: 2-byte prefix,
|
||||
parsePrefix, reject 0, read body, handle, frame + write + flush. idle_timeout race
|
||||
identical (same Options shape, max_connections 64, per-conn Scratch). The TLS
|
||||
handshake itself runs under the same race budget (a stalled handshake must not pin a
|
||||
connection slot).
|
||||
4. **Transport enum stays {udp, tcp}**: DoH/DoT call handle with `.tcp` (it only
|
||||
drives truncation, which never applies to stream transports). No unused surface.
|
||||
5. **ALPN, additive to platform**: ServerContext.init gains
|
||||
`alpn: ?[*:null]const ?[*:0]const u8` (mbedtls_ssl_conf_alpn_protocols requires a
|
||||
NULL-terminated array that OUTLIVES the config — use comptime-constant arrays).
|
||||
DoH advertises ["http/1.1"], DoT ["dot"]. A client with no ALPN still connects
|
||||
(mbedTLS only enforces when the client sends the extension). Existing callers pass
|
||||
null; the milestone-1 spec note about "no ALPN" is superseded.
|
||||
6. **Cert holder + reload**: new `src/server/cert_store.zig`. `CertStore` owns
|
||||
{mutex, current: *Entry, cert_path, key_path, gpa} where Entry = {ctx: ServerContext,
|
||||
refs: u32, retired: bool}. `acquire(io) *Entry` (+1 under mutex),
|
||||
`release(io, entry)` (−1; free when retired and 0). `reload(io) ReloadError!void`:
|
||||
read both PEM files (NUL-terminated dupes, 64 KiB cap each), ServerContext.init, on
|
||||
success swap + retire the old (freed when its refs drain), on ANY failure publish
|
||||
nothing (PLAN:297). Listeners acquire per connection (before accept) and release
|
||||
when the connection ends — so in-flight streams keep their generation alive.
|
||||
File read errors and init errors map to a typed error + a human message for the API.
|
||||
7. **Cert watcher**: one task per enabled endpoint inside cert_store
|
||||
(`watch(store, io)` house loop: Cancelable!void, .boot clock, warn-and-continue).
|
||||
Poll every 30 s: stat cert AND key (Io.File.stat), compare mtime+size against the
|
||||
last LOADED pair; any change → reload(); reload failure → warn (the old cert keeps
|
||||
serving) + a `reload_failures` counter; success → info-level "certificate reloaded".
|
||||
mtime+size both compared (same-second atomic renames).
|
||||
8. **`POST /api/certs/reload`** (PLAN:547): handler `src/web/handlers/certs.zig`
|
||||
(PLAN:232). auth = .session, rate_limit = .counted. Reloads every enabled endpoint's
|
||||
store; response 200 `{"doh": {"enabled": bool, "reloaded": bool, "error": <msg>|null},
|
||||
"dot": {...}}` — per-endpoint outcome, 200 even when a reload fails (the outcome IS
|
||||
the payload; nothing about the server's own state is exceptional). Disabled endpoint:
|
||||
{enabled:false, reloaded:false, error:null}. WebState gains `doh_certs: ?*CertStore`
|
||||
and `dot_certs: ?*CertStore` (null-defaulted, W3 pattern).
|
||||
9. **openapi.yaml + contract test land together** (m8 ruling 2): the path entry with
|
||||
the exact schema, and a contract-table entry + response-shape test in
|
||||
web_integration_test.zig. Drift guard (b) count moves 55 → 56 — the test derives the
|
||||
count from router.routes so only the yaml operation count assertion data changes.
|
||||
10. **Observability**: each listener keeps tcp_server-shaped stats (connections,
|
||||
tls_handshake_failures, idle_timeouts, connection_errors) + cert_store keeps
|
||||
{reloads, reload_failures, last_reload_unix}. metrics.zig additively gains
|
||||
`nxdns_doh_server_*`, `nxdns_dot_server_*`, `nxdns_cert_reloads_total`,
|
||||
`nxdns_cert_reload_failures_total` families (counterGroup walks the structs —
|
||||
follow its existing pattern). /api/health is unchanged (no new rollup this
|
||||
milestone; the reload endpoint reports cert state on demand).
|
||||
11. **app.zig wiring**: when `cfg.doh_server.enabled`, build its CertStore (initial
|
||||
load happens in serve BEFORE the group — a bad cert at boot is exit 2 with the
|
||||
validate-style message, matching `nxdns check`'s contract that enabled endpoints
|
||||
have readable certs; the WATCHER handles later breakage gracefully), then the
|
||||
listener; same for dot. Watcher tasks + listener tasks join the group. WebState
|
||||
seams wired when web is also enabled. SIGTERM path unchanged.
|
||||
12. **Integration tests** (both gated `-Dintegration`, loopback, fixtures):
|
||||
DoT: std.crypto.tls.Client (no_verification) → framed A query → framed reply
|
||||
(reuse resolver_integration_test's fake-upstream pattern or a local-records-only
|
||||
handler). DoH POST + GET: raw HTTP/1.1 text over std.crypto.tls.Client (no
|
||||
std.http.Client CA ceremony), assert status/content-type/bytes; 415/405/404/400
|
||||
matrix; keep-alive two-requests test. Reload: swap fixture cert (write a second
|
||||
self-signed pair fixture), reload, NEW connection sees the new cert
|
||||
(std.crypto.tls.Client exposes the peer cert? if not: assert old connections
|
||||
still serve and new handshakes succeed — the observable contract), old connection
|
||||
still answers. Watcher: unit-test the decision function (stat-pair compare) pure;
|
||||
do not test 30 s timing.
|
||||
13. **New fixture**: `tests/fixtures/self_signed_cert2.pem` + key2 (openssl one-liner,
|
||||
same profile as the existing pair; README updated) for reload tests.
|
||||
14. **Docs**: NOT this milestone beyond openapi.yaml (docs/api rendering is Phase 10 —
|
||||
m8 ruling 3 unchanged).
|
||||
15. **Zig freeze elsewhere**: dns/, filter/, cache/, storage/, existing server files
|
||||
(udp/tcp/handler) untouched except: app.zig (T6), metrics.zig + routes.zig +
|
||||
openapi.yaml + web_integration_test.zig + web/server.zig(WebState fields only)
|
||||
(T5). handler.zig is NOT touched (ruling 4).
|
||||
1. **Two listeners**: `src/server/doh_server.zig`, `src/server/dot_server.zig` (PLAN:194), constructed only when their endpoint is enabled. Bind failure = warn + continue (web precedent). Both join app.zig's group; cancel semantics mirror tcp_server's Stop enum (.canceled cancels the connection group; listener close drains).
|
||||
2. **DoH protocol**: HTTP/1.1 only (PLAN:684 excludes h2). Path `/dns-query` exactly. `POST` with `content-type: application/dns-message` (checked via doh_client.contentTypeOk) and `GET ?dns=<base64url unpadded>`. Responses: 200 `application/dns-message` with the reply bytes; handler `.drop` → 502 with empty body reason? NO — `.drop` means "no answer on purpose" (malformed/refused-silent): close the connection for POST/GET alike via 400. Errors: 404 non-/dns-query path, 405 other methods (Allow: GET, POST), 415 wrong content-type, 400 missing/invalid `dns` param or empty body, 413 body > dns/transport.max_message_len. No cache-control headers (LAN, no intermediaries). Keep-alive per std.http.Server loop; the bodyless-POST normalization from web/server.zig:443 is copied (same stdlib assert).
|
||||
3. **DoT protocol**: RFC 7858 = the tcp_server loop over ServerStream: 2-byte prefix, parsePrefix, reject 0, read body, handle, frame + write + flush. idle_timeout race identical (same Options shape, max_connections 64, per-conn Scratch). The TLS handshake itself runs under the same race budget (a stalled handshake must not pin a connection slot).
|
||||
4. **Transport enum stays {udp, tcp}**: DoH/DoT call handle with `.tcp` (it only drives truncation, which never applies to stream transports). No unused surface.
|
||||
5. **ALPN, additive to platform**: ServerContext.init gains `alpn: ?[*:null]const ?[*:0]const u8` (mbedtls_ssl_conf_alpn_protocols requires a NULL-terminated array that OUTLIVES the config — use comptime-constant arrays). DoH advertises ["http/1.1"], DoT ["dot"]. A client with no ALPN still connects (mbedTLS only enforces when the client sends the extension). Existing callers pass null; the milestone-1 spec note about "no ALPN" is superseded.
|
||||
6. **Cert holder + reload**: new `src/server/cert_store.zig`. `CertStore` owns {mutex, current: *Entry, cert_path, key_path, gpa} where Entry = {ctx: ServerContext, refs: u32, retired: bool}. `acquire(io) *Entry` (+1 under mutex), `release(io, entry)` (−1; free when retired and 0). `reload(io) ReloadError!void`: read both PEM files (NUL-terminated dupes, 64 KiB cap each), ServerContext.init, on success swap + retire the old (freed when its refs drain), on ANY failure publish nothing (PLAN:297). Listeners acquire per connection (before accept) and release when the connection ends — so in-flight streams keep their generation alive. File read errors and init errors map to a typed error + a human message for the API.
|
||||
7. **Cert watcher**: one task per enabled endpoint inside cert_store (`watch(store, io)` house loop: Cancelable!void, .boot clock, warn-and-continue). Poll every 30 s: stat cert AND key (Io.File.stat), compare mtime+size against the last LOADED pair; any change → reload(); reload failure → warn (the old cert keeps serving) + a `reload_failures` counter; success → info-level "certificate reloaded". mtime+size both compared (same-second atomic renames).
|
||||
8. **`POST /api/certs/reload`** (PLAN:547): handler `src/web/handlers/certs.zig` (PLAN:232). auth = .session, rate_limit = .counted. Reloads every enabled endpoint's store; response 200 `{"doh": {"enabled": bool, "reloaded": bool, "error": <msg>|null}, "dot": {...}}` — per-endpoint outcome, 200 even when a reload fails (the outcome IS the payload; nothing about the server's own state is exceptional). Disabled endpoint: {enabled:false, reloaded:false, error:null}. WebState gains `doh_certs: ?*CertStore` and `dot_certs: ?*CertStore` (null-defaulted, W3 pattern).
|
||||
9. **openapi.yaml + contract test land together** (m8 ruling 2): the path entry with the exact schema, and a contract-table entry + response-shape test in web_integration_test.zig. Drift guard (b) count moves 55 → 56 — the test derives the count from router.routes so only the yaml operation count assertion data changes.
|
||||
10. **Observability**: each listener keeps tcp_server-shaped stats (connections, tls_handshake_failures, idle_timeouts, connection_errors) + cert_store keeps {reloads, reload_failures, last_reload_unix}. metrics.zig additively gains `nxdns_doh_server_*`, `nxdns_dot_server_*`, `nxdns_cert_reloads_total`, `nxdns_cert_reload_failures_total` families (counterGroup walks the structs — follow its existing pattern). /api/health is unchanged (no new rollup this milestone; the reload endpoint reports cert state on demand).
|
||||
11. **app.zig wiring**: when `cfg.doh_server.enabled`, build its CertStore (initial load happens in serve BEFORE the group — a bad cert at boot is exit 2 with the validate-style message, matching `nxdns check`'s contract that enabled endpoints have readable certs; the WATCHER handles later breakage gracefully), then the listener; same for dot. Watcher tasks + listener tasks join the group. WebState seams wired when web is also enabled. SIGTERM path unchanged.
|
||||
12. **Integration tests** (both gated `-Dintegration`, loopback, fixtures): DoT: std.crypto.tls.Client (no_verification) → framed A query → framed reply (reuse resolver_integration_test's fake-upstream pattern or a local-records-only handler). DoH POST + GET: raw HTTP/1.1 text over std.crypto.tls.Client (no std.http.Client CA ceremony), assert status/content-type/bytes; 415/405/404/400 matrix; keep-alive two-requests test. Reload: swap fixture cert (write a second self-signed pair fixture), reload, NEW connection sees the new cert (std.crypto.tls.Client exposes the peer cert? if not: assert old connections still serve and new handshakes succeed — the observable contract), old connection still answers. Watcher: unit-test the decision function (stat-pair compare) pure; do not test 30 s timing.
|
||||
13. **New fixture**: `tests/fixtures/self_signed_cert2.pem` + key2 (openssl one-liner, same profile as the existing pair; README updated) for reload tests.
|
||||
14. **Docs**: NOT this milestone beyond openapi.yaml (docs/api rendering is Phase 10 — m8 ruling 3 unchanged).
|
||||
15. **Zig freeze elsewhere**: dns/, filter/, cache/, storage/, existing server files (udp/tcp/handler) untouched except: app.zig (T6), metrics.zig + routes.zig + openapi.yaml + web_integration_test.zig + web/server.zig(WebState fields only) (T5). handler.zig is NOT touched (ruling 4).
|
||||
|
||||
## Sessions
|
||||
|
||||
@@ -126,181 +37,63 @@ T1+T2 parallel first; then T3+T4+T5 parallel; then T6.
|
||||
|
||||
## Session T1: platform ALPN
|
||||
|
||||
Owns `src/platform/tls_server.zig` (+ its tests). Add the alpn parameter per ruling 5
|
||||
(extern mbedtls_ssl_conf_alpn_protocols, comptime NULL-terminated arrays, doc the
|
||||
lifetime rule), update existing callers/tests with null, add one test that a client
|
||||
negotiating "http/1.1" succeeds against a context advertising it (the loopback
|
||||
integration test grows the assertion — std.crypto.tls.Client: check whether it can
|
||||
send ALPN in 0.16; if it cannot, the unit test just proves config acceptance and the
|
||||
lifetime doc stands).
|
||||
Owns `src/platform/tls_server.zig` (+ its tests). Add the alpn parameter per ruling 5 (extern mbedtls_ssl_conf_alpn_protocols, comptime NULL-terminated arrays, doc the lifetime rule), update existing callers/tests with null, add one test that a client negotiating "http/1.1" succeeds against a context advertising it (the loopback integration test grows the assertion — std.crypto.tls.Client: check whether it can send ALPN in 0.16; if it cannot, the unit test just proves config acceptance and the lifetime doc stands).
|
||||
|
||||
Acceptance: fmt/ast clean; plain + integration suites 0 failed.
|
||||
|
||||
### T1 As built
|
||||
|
||||
`ServerContext.init(gpa, cert_pem, key_pem, alpn: ?[*:null]const ?[*:0]const u8)`.
|
||||
Extern `mbedtls_ssl_conf_alpn_protocols` verified against vendored 3.6.7 ssl.h:4314
|
||||
(pointer recorded, not copied — comptime-constant NULL-terminated arrays required,
|
||||
documented on init). MBEDTLS_SSL_ALPN is on in the stock config. std.crypto.tls.Client
|
||||
CANNOT send ALPN in 0.16 (no Options field) — negotiation untestable client-side; the
|
||||
unit test proves config acceptance, the integration echo proves no-ALPN clients still
|
||||
handshake against an advertising context. Servers can assert server-side via
|
||||
mbedtls_ssl_get_alpn_protocol if ever needed (extern not declared — unused surface).
|
||||
Mechanical ripple: tls_client_integration_test.zig:193 caller gained `, null`.
|
||||
`ServerContext.init(gpa, cert_pem, key_pem, alpn: ?[*:null]const ?[*:0]const u8)`. Extern `mbedtls_ssl_conf_alpn_protocols` verified against vendored 3.6.7 ssl.h:4314 (pointer recorded, not copied — comptime-constant NULL-terminated arrays required, documented on init). MBEDTLS_SSL_ALPN is on in the stock config. std.crypto.tls.Client CANNOT send ALPN in 0.16 (no Options field) — negotiation untestable client-side; the unit test proves config acceptance, the integration echo proves no-ALPN clients still handshake against an advertising context. Servers can assert server-side via mbedtls_ssl_get_alpn_protocol if ever needed (extern not declared — unused surface). Mechanical ripple: tls_client_integration_test.zig:193 caller gained `, null`.
|
||||
|
||||
## Session T2: cert_store
|
||||
|
||||
Owns `src/server/cert_store.zig` (new). Ruling 6 (holder + refcount + reload) and
|
||||
ruling 7 (watch loop + pure stat-compare decision fn) + ruling 10's counters +
|
||||
`snapshotStats()`. Unit tests: acquire/release refcounting across a swap (old entry
|
||||
freed only after last release), reload failure publishes nothing (bad path, bad PEM),
|
||||
NUL-termination and size cap, stat-compare decision table. Uses fixtures via
|
||||
test_fixtures import.
|
||||
Owns `src/server/cert_store.zig` (new). Ruling 6 (holder + refcount + reload) and ruling 7 (watch loop + pure stat-compare decision fn) + ruling 10's counters + `snapshotStats()`. Unit tests: acquire/release refcounting across a swap (old entry freed only after last release), reload failure publishes nothing (bad path, bad PEM), NUL-termination and size cap, stat-compare decision table. Uses fixtures via test_fixtures import.
|
||||
|
||||
Acceptance: fmt/ast clean; temp-root green (needs -lc + vendored mbedtls objects).
|
||||
|
||||
### T2 As built
|
||||
|
||||
16 in-file tests; temp-root 22 passed / 0 failed. Surface:
|
||||
`poll_interval_s=30`, `max_pem_bytes=64KiB`, `ReloadError` (10 cases) +
|
||||
`humanMessage(err) []const u8`; `FileSig{mtime_ns: i96, size}`, `Signature{cert,key}`,
|
||||
pure `changed(loaded, observed) bool`; `Entry{ctx, refs, retired}`;
|
||||
`CertStore.init(gpa, io, cert_path, key_path, alpn)` (boot load; typed error for T6's
|
||||
exit-2), `deinit(io)` (asserts current refs==0 — join listeners first),
|
||||
`acquire(io) *Entry` / `release(io, entry)` (release safe from any task; last releaser
|
||||
frees a retired entry), `reload(io) ReloadError!void`, `watch(io) Cancelable!void`
|
||||
(sleeps BEFORE first poll, .boot clock), `pollOnce(io)` (testable watcher pass),
|
||||
`snapshotStats()` (no io; {reloads, reload_failures, last_reload_unix}).
|
||||
Notes: use `&entry.ctx` for ServerStream.accept; paths are BORROWED (config outlives
|
||||
the store); `reloads` counts successes only; stat failures warn but are not
|
||||
reload_failures; `last_reload_unix` set at init too.
|
||||
16 in-file tests; temp-root 22 passed / 0 failed. Surface: `poll_interval_s=30`, `max_pem_bytes=64KiB`, `ReloadError` (10 cases) + `humanMessage(err) []const u8`; `FileSig{mtime_ns: i96, size}`, `Signature{cert,key}`, pure `changed(loaded, observed) bool`; `Entry{ctx, refs, retired}`; `CertStore.init(gpa, io, cert_path, key_path, alpn)` (boot load; typed error for T6's exit-2), `deinit(io)` (asserts current refs==0 — join listeners first), `acquire(io) *Entry` / `release(io, entry)` (release safe from any task; last releaser frees a retired entry), `reload(io) ReloadError!void`, `watch(io) Cancelable!void` (sleeps BEFORE first poll, .boot clock), `pollOnce(io)` (testable watcher pass), `snapshotStats()` (no io; {reloads, reload_failures, last_reload_unix}). Notes: use `&entry.ctx` for ServerStream.accept; paths are BORROWED (config outlives the store); `reloads` counts successes only; stat failures warn but are not reload_failures; `last_reload_unix` set at init too.
|
||||
|
||||
## Session T3: dot_server (after T1+T2)
|
||||
|
||||
Owns `src/server/dot_server.zig` (new) + its integration test file if separate
|
||||
(prefer tests inside the file, house pattern). Ruling 3: tcp_server loop shape over
|
||||
ServerStream; acquire cert entry per connection, release on exit; handshake under the
|
||||
race budget; stats per ruling 10; Stop enum; max_connections 64; ALPN ["dot"].
|
||||
Integration tests per ruling 12 (DoT half).
|
||||
Owns `src/server/dot_server.zig` (new) + its integration test file if separate (prefer tests inside the file, house pattern). Ruling 3: tcp_server loop shape over ServerStream; acquire cert entry per connection, release on exit; handshake under the race budget; stats per ruling 10; Stop enum; max_connections 64; ALPN ["dot"]. Integration tests per ruling 12 (DoT half).
|
||||
|
||||
Acceptance: fmt/ast clean; both suites 0 failed.
|
||||
|
||||
### T3 As built
|
||||
|
||||
`DotServer.listen(gpa, io, listen_address, h: *handler.Handler, certs: *cert_store.CertStore, options)` /
|
||||
`serve(io)` / `boundAddress()` / `deinit(io)`. Options `{max_connections: u16 = 64, idle_timeout = 10s}`;
|
||||
the handshake runs under the same idle budget. Deviation from tcp_server: `deinit` takes no
|
||||
gpa — the allocator is stored at `listen` because `ServerStream.accept` heap-allocates one
|
||||
mbedTLS ssl context per connection (documented in the file header). Stats atomics:
|
||||
the four ruling-10 fields plus tcp-pattern internals `rejected_at_capacity,
|
||||
rejected_at_shutdown, accept_errors`; `snapshotStats()` returns exactly the ruling-10 four.
|
||||
Loop: accept → claim slot → `certs.acquire` (released on every exit) → handshake raced
|
||||
against the idle budget → 2-byte-prefix loop over ServerStream reader/writer, flush per
|
||||
reply, handler called with `.tcp`, `tls.close(gpa)` (close_notify) on every post-handshake
|
||||
exit. A `handshook` flag ensures the ssl context closes exactly once when Select reports
|
||||
expiry/cancel after a successful handshake; a timed-out handshake counts as
|
||||
`tls_handshake_failures`. ALPN is set only via the store (T6). 7 unit + 4 integration
|
||||
tests in-file. Teardown order: `server.deinit(io)` before `CertStore.deinit`.
|
||||
`DotServer.listen(gpa, io, listen_address, h: *handler.Handler, certs: *cert_store.CertStore, options)` / `serve(io)` / `boundAddress()` / `deinit(io)`. Options `{max_connections: u16 = 64, idle_timeout = 10s}`; the handshake runs under the same idle budget. Deviation from tcp_server: `deinit` takes no gpa — the allocator is stored at `listen` because `ServerStream.accept` heap-allocates one mbedTLS ssl context per connection (documented in the file header). Stats atomics: the four ruling-10 fields plus tcp-pattern internals `rejected_at_capacity, rejected_at_shutdown, accept_errors`; `snapshotStats()` returns exactly the ruling-10 four. Loop: accept → claim slot → `certs.acquire` (released on every exit) → handshake raced against the idle budget → 2-byte-prefix loop over ServerStream reader/writer, flush per reply, handler called with `.tcp`, `tls.close(gpa)` (close_notify) on every post-handshake exit. A `handshook` flag ensures the ssl context closes exactly once when Select reports expiry/cancel after a successful handshake; a timed-out handshake counts as `tls_handshake_failures`. ALPN is set only via the store (T6). 7 unit + 4 integration tests in-file. Teardown order: `server.deinit(io)` before `CertStore.deinit`.
|
||||
|
||||
## Session T4: doh_server (after T1+T2)
|
||||
|
||||
Owns `src/server/doh_server.zig` (new). Ruling 2 in full: std.http.Server over
|
||||
ServerStream reader/writer (recv buffer 8 KiB = head cap, send 4 KiB), connection
|
||||
group + 64 cap + 503-free refusal is NOT applicable here (no HTTP yet at accept
|
||||
overflow: over cap → close raw, cheapest honest behavior — document); per-connection
|
||||
Scratch + response_buf (65535); GET base64url decode (url_safe_no_pad, reject padding/
|
||||
whitespace), POST body cap via Io.Limit; keep-alive; bodyless-POST normalization;
|
||||
ALPN ["http/1.1"]; cert acquire/release per connection; stats. Integration tests per
|
||||
ruling 12 (DoH half: POST, GET, 415/405/404/400, keep-alive).
|
||||
Owns `src/server/doh_server.zig` (new). Ruling 2 in full: std.http.Server over ServerStream reader/writer (recv buffer 8 KiB = head cap, send 4 KiB), connection group + 64 cap + 503-free refusal is NOT applicable here (no HTTP yet at accept overflow: over cap → close raw, cheapest honest behavior — document); per-connection Scratch + response_buf (65535); GET base64url decode (url_safe_no_pad, reject padding/ whitespace), POST body cap via Io.Limit; keep-alive; bodyless-POST normalization; ALPN ["http/1.1"]; cert acquire/release per connection; stats. Integration tests per ruling 12 (DoH half: POST, GET, 415/405/404/400, keep-alive).
|
||||
|
||||
Acceptance: fmt/ast clean; both suites 0 failed.
|
||||
|
||||
### T4 As built
|
||||
|
||||
`DohServer.listen(gpa, io, address, *Handler, *CertStore, Options)` / `serve(io)` /
|
||||
`deinit(gpa, io)` / `boundAddress()` / `snapshotStats()`. Options
|
||||
`{max_connections = 64, idle_timeout = 10s}` — the budget races the TLS handshake only;
|
||||
requests have no timeout (web-listener precedent). Module-level
|
||||
`serve(gpa, io, model.TlsEndpoint, *Handler, *CertStore)` is the bind-warn-and-continue
|
||||
entry for T6. Exposed comptime `alpn_protocols` (["http/1.1"]) and `dns_query_path`.
|
||||
Stats: the ruling-10 four + `bad_requests` + tcp-shaped accept counters
|
||||
(`rejected_at_capacity, rejected_at_shutdown, accept_errors`) — over-capacity refusals
|
||||
must stay visible. Deviations: POST cap via content-length precheck + fixed buffer +
|
||||
one-byte probe for chunked bodies instead of Io.Limit (no allocation on the hot path);
|
||||
`respond` can raise `HttpExpectationFailed` (100-continue), mapped to `connection_errors`;
|
||||
no 413 integration test (ruling 12's matrix is 415/405/404/400). GET `?dns=` parsed by a
|
||||
local validating decoder (duplicate param → 400, `%` and padding/whitespace rejected).
|
||||
Oversize POST answers 413 with `keep_alive=false` (body never drained). Over the 64 cap
|
||||
the raw TCP stream closes (no HTTP before a handshake). 8 unit + 4 integration tests.
|
||||
T6 recipe: `CertStore.init(gpa, io, cert_path, key_path, doh_server.alpn_protocols)`;
|
||||
run `doh_server.serve` and `store.watch` as group tasks; `DohServer.deinit` before
|
||||
`CertStore.deinit`.
|
||||
`DohServer.listen(gpa, io, address, *Handler, *CertStore, Options)` / `serve(io)` / `deinit(gpa, io)` / `boundAddress()` / `snapshotStats()`. Options `{max_connections = 64, idle_timeout = 10s}` — the budget races the TLS handshake only; requests have no timeout (web-listener precedent). Module-level `serve(gpa, io, model.TlsEndpoint, *Handler, *CertStore)` is the bind-warn-and-continue entry for T6. Exposed comptime `alpn_protocols` (["http/1.1"]) and `dns_query_path`. Stats: the ruling-10 four + `bad_requests` + tcp-shaped accept counters (`rejected_at_capacity, rejected_at_shutdown, accept_errors`) — over-capacity refusals must stay visible. Deviations: POST cap via content-length precheck + fixed buffer + one-byte probe for chunked bodies instead of Io.Limit (no allocation on the hot path); `respond` can raise `HttpExpectationFailed` (100-continue), mapped to `connection_errors`; no 413 integration test (ruling 12's matrix is 415/405/404/400). GET `?dns=` parsed by a local validating decoder (duplicate param → 400, `%` and padding/whitespace rejected). Oversize POST answers 413 with `keep_alive=false` (body never drained). Over the 64 cap the raw TCP stream closes (no HTTP before a handshake). 8 unit + 4 integration tests. T6 recipe: `CertStore.init(gpa, io, cert_path, key_path, doh_server.alpn_protocols)`; run `doh_server.serve` and `store.watch` as group tasks; `DohServer.deinit` before `CertStore.deinit`.
|
||||
|
||||
## Session T5: web endpoint + metrics (after T2, parallel with T3/T4)
|
||||
|
||||
Owns `src/web/handlers/certs.zig` (new), `src/web/routes.zig` (one entry),
|
||||
`src/web/openapi.yaml` (one path), `src/web/server.zig` (two null-defaulted WebState
|
||||
fields ONLY), `src/web/metrics.zig` (ruling 10 families), and the additive
|
||||
web_integration_test.zig edits (contract entry + shape test + guard-b count).
|
||||
Handler per ruling 8 (apply/respond split, house pattern). Unit tests for the
|
||||
apply fn (both stores null; one store failing reload → error message in payload,
|
||||
still 200).
|
||||
Owns `src/web/handlers/certs.zig` (new), `src/web/routes.zig` (one entry), `src/web/openapi.yaml` (one path), `src/web/server.zig` (two null-defaulted WebState fields ONLY), `src/web/metrics.zig` (ruling 10 families), and the additive web_integration_test.zig edits (contract entry + shape test + guard-b count). Handler per ruling 8 (apply/respond split, house pattern). Unit tests for the apply fn (both stores null; one store failing reload → error message in payload, still 200).
|
||||
|
||||
Acceptance: fmt/ast clean; plain suite 0 failed; drift guards green.
|
||||
|
||||
### T5 As built
|
||||
|
||||
handlers/certs.zig: apply/respond split — `applyReload(state, io) View` with
|
||||
`View{doh, dot}` and `Outcome{enabled, reloaded, @"error": ?[]const u8}` (serialized as
|
||||
`error`); `post` always answers 200. Route: `POST /api/certs/reload`, `.session`,
|
||||
`.counted`; table 55→56; contract entry + strict shape test + drift guards green at 56.
|
||||
metrics.zig: the two ruling-10 cert counters share one family pair
|
||||
(`nxdns_cert_reloads_total` / `nxdns_cert_reload_failures_total`) with an
|
||||
`endpoint="doh"|"dot"` label (upstream-label pattern); unwired stores omit the family;
|
||||
`last_reload_unix` not exported. WebState gains `doh_certs`/`dot_certs:
|
||||
?*cert_store.CertStore = null`. Deviation (accepted): deleted the m8 tripwire test in
|
||||
`src/web/openapi.zig` ("the document does not promise what phase 9 owns") — it exists to
|
||||
fire when this endpoint lands. Deferred to T6: `nxdns_doh_server_*` / `nxdns_dot_server_*`
|
||||
listener families (T3/T4 types did not exist at T5 build time).
|
||||
handlers/certs.zig: apply/respond split — `applyReload(state, io) View` with `View{doh, dot}` and `Outcome{enabled, reloaded, @"error": ?[]const u8}` (serialized as `error`); `post` always answers 200. Route: `POST /api/certs/reload`, `.session`, `.counted`; table 55→56; contract entry + strict shape test + drift guards green at 56. metrics.zig: the two ruling-10 cert counters share one family pair (`nxdns_cert_reloads_total` / `nxdns_cert_reload_failures_total`) with an `endpoint="doh"|"dot"` label (upstream-label pattern); unwired stores omit the family; `last_reload_unix` not exported. WebState gains `doh_certs`/`dot_certs: ?*cert_store.CertStore = null`. Deviation (accepted): deleted the m8 tripwire test in `src/web/openapi.zig` ("the document does not promise what phase 9 owns") — it exists to fire when this endpoint lands. Deferred to T6: `nxdns_doh_server_*` / `nxdns_dot_server_*` listener families (T3/T4 types did not exist at T5 build time).
|
||||
|
||||
## Session T6: app wiring + fixtures + smoke (after all)
|
||||
|
||||
Owns `src/app.zig`, `tests/fixtures/` (cert2/key2 + README + fixtures.zig), and
|
||||
`src/tests.zig` is the ORCHESTRATOR's (report lines). Ruling 11 wiring; ruling 13
|
||||
fixture. Smoke (report transcript): boot with doh+dot enabled on unprivileged ports
|
||||
with fixture certs; DoT query via a scripted TLS client (a tiny zig run or the
|
||||
integration test binary is fine — no new tooling); DoH POST via curl --insecure
|
||||
--http1.1 with a binary body; certs/reload via the API twice (success, then chmod the
|
||||
key unreadable → error in payload, old cert still serving a new connection); SIGTERM 0.
|
||||
Both suites + cross ReleaseSafe with the SPA dist.
|
||||
Owns `src/app.zig`, `tests/fixtures/` (cert2/key2 + README + fixtures.zig), and `src/tests.zig` is the ORCHESTRATOR's (report lines). Ruling 11 wiring; ruling 13 fixture. Smoke (report transcript): boot with doh+dot enabled on unprivileged ports with fixture certs; DoT query via a scripted TLS client (a tiny zig run or the integration test binary is fine — no new tooling); DoH POST via curl --insecure --http1.1 with a binary body; certs/reload via the API twice (success, then chmod the key unreadable → error in payload, old cert still serving a new connection); SIGTERM 0. Both suites + cross ReleaseSafe with the SPA dist.
|
||||
|
||||
Acceptance: full gates green; smoke transcript.
|
||||
|
||||
### T6 As built
|
||||
|
||||
app.zig realizes ruling 11 in-frame: `openCertStore` maps non-OOM `ReloadError` to
|
||||
`error.BadCertificate` (exit 2, prints both paths + `cert_store.humanMessage`);
|
||||
`bindDoh`/`bindDot` warn-and-continue per ruling 1; listeners + two `store.watch` loops
|
||||
join the group; stores are declared before listeners so teardown runs listener-deinit →
|
||||
store-deinit and the refs==0 assert holds. `dot_alpn` (["dot"]) lives in app.zig.
|
||||
Deviation: `doh_server.serve` module entry is NOT used — it constructs the DohServer in
|
||||
its own task frame, so WebState could never get the pointer the `nxdns_doh_server_*`
|
||||
family needs; both listeners bind in app.zig's frame instead (tcp_server style).
|
||||
`doh_server.serve` remains as unused pub API. **Superseded by milestone-18 ruling 1: the
|
||||
unused `doh_server.serve` is deleted. Nothing ever called it, and a dead second
|
||||
composition path over the listener core is exactly the divergence that milestone
|
||||
collapsed. app.zig's in-frame bind is now the only way DoH comes up.** metrics.zig: `DohListenerSample`
|
||||
(ruling-10 four + `bad_requests`); DoT renders `dot_server.StatsSnapshot` directly;
|
||||
accept-side counters stay off the exposition; unwired listeners omit the families.
|
||||
WebState gains `doh_listener`/`dot_listener` optional pointers. build.zig (out of
|
||||
ownership, necessary): the exe now compiles `mbedtls_shim.c` with the threading macros —
|
||||
first exe-side tls_server reference; mirrors the tests wiring. Fixtures: cert2/key2
|
||||
exported as `cert2_pem`/`key2_pem` (distinct fingerprint), currently unreferenced by
|
||||
tests. Config: `model.TlsEndpoint` already existed; no model/validate/schema edits.
|
||||
Smoke: DoT framed query answered from local records; DoH POST 200
|
||||
application/dns-message; reload success then key chmod 000 → `reloaded:false` with
|
||||
"private key file is not readable" while the old cert keeps serving; SIGTERM 0; boot
|
||||
with unreadable key exits 2.
|
||||
app.zig realizes ruling 11 in-frame: `openCertStore` maps non-OOM `ReloadError` to `error.BadCertificate` (exit 2, prints both paths + `cert_store.humanMessage`); `bindDoh`/`bindDot` warn-and-continue per ruling 1; listeners + two `store.watch` loops join the group; stores are declared before listeners so teardown runs listener-deinit → store-deinit and the refs==0 assert holds. `dot_alpn` (["dot"]) lives in app.zig. Deviation: `doh_server.serve` module entry is NOT used — it constructs the DohServer in its own task frame, so WebState could never get the pointer the `nxdns_doh_server_*` family needs; both listeners bind in app.zig's frame instead (tcp_server style). `doh_server.serve` remains as unused pub API. **Superseded by milestone-18 ruling 1: the unused `doh_server.serve` is deleted. Nothing ever called it, and a dead second composition path over the listener core is exactly the divergence that milestone collapsed. app.zig's in-frame bind is now the only way DoH comes up.** metrics.zig: `DohListenerSample` (ruling-10 four + `bad_requests`); DoT renders `dot_server.StatsSnapshot` directly; accept-side counters stay off the exposition; unwired listeners omit the families. WebState gains `doh_listener`/`dot_listener` optional pointers. build.zig (out of ownership, necessary): the exe now compiles `mbedtls_shim.c` with the threading macros — first exe-side tls_server reference; mirrors the tests wiring. Fixtures: cert2/key2 exported as `cert2_pem`/`key2_pem` (distinct fingerprint), currently unreferenced by tests. Config: `model.TlsEndpoint` already existed; no model/validate/schema edits. Smoke: DoT framed query answered from local records; DoH POST 200 application/dns-message; reload success then key chmod 000 → `reloaded:false` with "private key file is not readable" while the old cert keeps serving; SIGTERM 0; boot with unreadable key exits 2.
|
||||
|
||||
---
|
||||
|
||||
@@ -308,45 +101,21 @@ with unreadable key exits 2.
|
||||
|
||||
Four rounds on one thread; round 4 returned "No findings."
|
||||
|
||||
Round 1 (3 important, 2 minor): doh_server leaked the TLS context when the handshake
|
||||
won a race the Select reported as timeout/cancel → handshook flag (dot_server pattern).
|
||||
cert_store reloads were not serialized across read→build→publish, so an older read could
|
||||
publish last → `reload_mutex` held across the whole sequence (lock order: reload_mutex
|
||||
before the entry mutex; accept path untouched) + a hook-pinned regression test
|
||||
(`after_load_hook` seam). doh bodied-GET smuggling: std leaves unframed body bytes for
|
||||
bodyless methods to be parsed as the next request → `framesUnreadableBody` guard, 400 +
|
||||
close, no byte read. `.drop` now closes per ruling 2. cert2/key2 were unreferenced →
|
||||
DoT reload-to-distinct-identity integration test (old connection keeps serving, new
|
||||
handshake succeeds, distinct entry pointers; the 0.16 tls Client does not retain the
|
||||
peer chain, so identity is proven via entry pointers).
|
||||
Round 1 (3 important, 2 minor): doh_server leaked the TLS context when the handshake won a race the Select reported as timeout/cancel → handshook flag (dot_server pattern). cert_store reloads were not serialized across read→build→publish, so an older read could publish last → `reload_mutex` held across the whole sequence (lock order: reload_mutex before the entry mutex; accept path untouched) + a hook-pinned regression test (`after_load_hook` seam). doh bodied-GET smuggling: std leaves unframed body bytes for bodyless methods to be parsed as the next request → `framesUnreadableBody` guard, 400 + close, no byte read. `.drop` now closes per ruling 2. cert2/key2 were unreferenced → DoT reload-to-distinct-identity integration test (old connection keeps serving, new handshake succeeds, distinct entry pointers; the 0.16 tls Client does not retain the peer chain, so identity is proven via entry pointers).
|
||||
|
||||
Round 2 (2 minor): the smuggling guard ran before routing and stole 404/405 statuses →
|
||||
routing decides the status, framed requests only force close. Successful POSTs ignored
|
||||
the client's `Connection: close` → answer/refuse honor `head.keep_alive` in the Next
|
||||
enum (the wire already did via Server.zig:628).
|
||||
Round 2 (2 minor): the smuggling guard ran before routing and stole 404/405 statuses → routing decides the status, framed requests only force close. Successful POSTs ignored the client's `Connection: close` → answer/refuse honor `head.keep_alive` in the Next enum (the wire already did via Server.zig:628).
|
||||
|
||||
Round 3 (1 important, introduced by the round-2 reshape): bodied refusals (404/405/415)
|
||||
kept keep-alive, so respond drained the whole body first — an unterminated chunked body
|
||||
could pin all 64 slots with no response → `framesBody` helper; any pre-body-read refusal
|
||||
closes when a body is framed; drain-free integration tests (missing terminal chunk →
|
||||
404+EOF at once; undelivered content-length 4096 → 415+EOF); errorMatrix's 415 became a
|
||||
zero-length wrong-type POST so its keep-alive reuse stays legitimate.
|
||||
Round 3 (1 important, introduced by the round-2 reshape): bodied refusals (404/405/415) kept keep-alive, so respond drained the whole body first — an unterminated chunked body could pin all 64 slots with no response → `framesBody` helper; any pre-body-read refusal closes when a body is framed; drain-free integration tests (missing terminal chunk → 404+EOF at once; undelivered content-length 4096 → 415+EOF); errorMatrix's 415 became a zero-length wrong-type POST so its keep-alive reuse stays legitimate.
|
||||
|
||||
Final counts: doh_server.zig 19 tests, dot_server.zig 12, cert_store.zig 17. Final
|
||||
gates: plain 1166/1279 passed, 113 skipped (integration-gated), 0 failed; integration
|
||||
1275/1279, 4 skipped (live-network by design), 0 failed.
|
||||
Final counts: doh_server.zig 19 tests, dot_server.zig 12, cert_store.zig 17. Final gates: plain 1166/1279 passed, 113 skipped (integration-gated), 0 failed; integration 1275/1279, 4 skipped (live-network by design), 0 failed.
|
||||
|
||||
## Module layout (new)
|
||||
|
||||
src/server/{doh_server,dot_server,cert_store}.zig, src/web/handlers/certs.zig,
|
||||
tests/fixtures/{self_signed_cert2.pem,self_signed_key2.pem}.
|
||||
src/server/{doh_server,dot_server,cert_store}.zig, src/web/handlers/certs.zig, tests/fixtures/{self_signed_cert2.pem,self_signed_key2.pem}.
|
||||
|
||||
## File ownership
|
||||
|
||||
T1 platform/tls_server.zig; T2 server/cert_store.zig; T3 server/dot_server.zig;
|
||||
T4 server/doh_server.zig; T5 web/{handlers/certs.zig,routes.zig,openapi.yaml,
|
||||
server.zig(fields),metrics.zig,web_integration_test.zig}; T6 app.zig + fixtures.
|
||||
Orchestrator: src/tests.zig, spec. Parallel sessions never share a file.
|
||||
T1 platform/tls_server.zig; T2 server/cert_store.zig; T3 server/dot_server.zig; T4 server/doh_server.zig; T5 web/{handlers/certs.zig,routes.zig,openapi.yaml, server.zig(fields),metrics.zig,web_integration_test.zig}; T6 app.zig + fixtures. Orchestrator: src/tests.zig, spec. Parallel sessions never share a file.
|
||||
|
||||
## Acceptance (milestone complete)
|
||||
|
||||
@@ -362,7 +131,4 @@ Orchestrator: src/tests.zig, spec. Parallel sessions never share a file.
|
||||
|
||||
## Anti-requirements
|
||||
|
||||
No HTTP/2, no DoQ, no SNI/multi-cert, no client-cert auth, no session tickets, no
|
||||
proxy-protocol/XFF handling, no cert generation (ACME etc.), no cache-control on DoH,
|
||||
no handler.zig changes, no docs/api rendering (Phase 10), no config schema changes
|
||||
(TlsEndpoint exists), no inotify.
|
||||
No HTTP/2, no DoQ, no SNI/multi-cert, no client-cert auth, no session tickets, no proxy-protocol/XFF handling, no cert generation (ACME etc.), no cache-control on DoH, no handler.zig changes, no docs/api rendering (Phase 10), no config schema changes (TlsEndpoint exists), no inotify.
|
||||
|
||||
+37
-199
@@ -1,255 +1,94 @@
|
||||
# Milestone 11: packaging, ops and docs (PLAN Phase 10)
|
||||
|
||||
Goal: systemd unit, Dockerfile + compose, and the four docs (operator, architecture,
|
||||
config-reference, API) — the documented deployment must work end-to-end; docs are
|
||||
drift-guarded where a guard is cheap and honest.
|
||||
Goal: systemd unit, Dockerfile + compose, and the four docs (operator, architecture, config-reference, API) — the documented deployment must work end-to-end; docs are drift-guarded where a guard is cheap and honest.
|
||||
|
||||
## Rulings (binding)
|
||||
|
||||
1. **Layout.** `deploy/systemd/nxdns.service` + `deploy/systemd/sysusers.conf`;
|
||||
`deploy/docker/{Dockerfile,compose.yaml,.dockerignore}`; `docs/{operator.md,
|
||||
architecture.md,config-reference.md,api.md}`; `README.md` at the root (the repo has
|
||||
none; a portfolio repo needs a front door — short: what, why, quickstart, doc links).
|
||||
PLAN.md:236 sketches docs/ subdirectories; single files need no subdirectories.
|
||||
1. **Layout.** `deploy/systemd/nxdns.service` + `deploy/systemd/sysusers.conf`; `deploy/docker/{Dockerfile,compose.yaml,.dockerignore}`; `docs/{operator.md, architecture.md,config-reference.md,api.md}`; `README.md` at the root (the repo has none; a portfolio repo needs a front door — short: what, why, quickstart, doc links). PLAN.md:236 sketches docs/ subdirectories; single files need no subdirectories.
|
||||
|
||||
2. **API docs = hand-written `docs/api.md` + drift test.** No renderer is vendored
|
||||
(redoc/scalar are exactly the dependency liability AGENTS.md refuses), and a
|
||||
build-time YAML parser for rendering is scope the yaml does not justify — the yaml
|
||||
itself is already served unauthenticated at `GET /api/openapi.yaml` (routes.zig:49)
|
||||
and is the exhaustive contract. `docs/api.md` gives human-readable orientation:
|
||||
auth model (cookie session, login flow), rate limiting, error envelope, SSE
|
||||
semantics, then one line per operation (method, path, auth, one-sentence purpose)
|
||||
and a pointer to the yaml for schemas. A drift test asserts every served route
|
||||
appears textually in api.md (mirror of openapi.zig:34's guard). This satisfies
|
||||
m8 ruling 3's deferred "docs/api rendering" as the engineering call: rendered =
|
||||
readable, guarded, in-repo; not = a vendored JS bundle.
|
||||
2. **API docs = hand-written `docs/api.md` + drift test.** No renderer is vendored (redoc/scalar are exactly the dependency liability AGENTS.md refuses), and a build-time YAML parser for rendering is scope the yaml does not justify — the yaml itself is already served unauthenticated at `GET /api/openapi.yaml` (routes.zig:49) and is the exhaustive contract. `docs/api.md` gives human-readable orientation: auth model (cookie session, login flow), rate limiting, error envelope, SSE semantics, then one line per operation (method, path, auth, one-sentence purpose) and a pointer to the yaml for schemas. A drift test asserts every served route appears textually in api.md (mirror of openapi.zig:34's guard). This satisfies m8 ruling 3's deferred "docs/api rendering" as the engineering call: rendered = readable, guarded, in-repo; not = a vendored JS bundle.
|
||||
|
||||
3. **Docs drift guards.** New `src/docs_drift_test.zig` (ORCHESTRATOR-owned, written
|
||||
after the doc sessions land): (a) every route in `router.routes` appears in
|
||||
docs/api.md; (b) every settings key from `model.toSettings` (the 44 keys) appears
|
||||
in docs/config-reference.md; (c) every CLI subcommand name appears in
|
||||
docs/operator.md. Docs embedded via a `docs_files` anonymous import added in
|
||||
build.zig (test_fixtures pattern, build.zig:61). Guards are textual-containment
|
||||
only — cheap, zero false authority.
|
||||
3. **Docs drift guards.** New `src/docs_drift_test.zig` (ORCHESTRATOR-owned, written after the doc sessions land): (a) every route in `router.routes` appears in docs/api.md; (b) every settings key from `model.toSettings` (the 44 keys) appears in docs/config-reference.md; (c) every CLI subcommand name appears in docs/operator.md. Docs embedded via a `docs_files` anonymous import added in build.zig (test_fixtures pattern, build.zig:61). Guards are textual-containment only — cheap, zero false authority.
|
||||
|
||||
4. **systemd unit.** `Type=simple` (no forking, shutdown.zig:35 handles SIGTERM),
|
||||
`ExecStart=/usr/local/bin/nxdns run`, stderr → journald (logging.zig:301 already
|
||||
states this; `logging.output=stderr` stays the default). Static system user `nxdns`
|
||||
via `deploy/systemd/sysusers.conf` (`u nxdns - "nxdns DNS sinkhole"`), NOT
|
||||
DynamicUser — the TLS key must be chown-able to a stable uid ("TLS keys readable by
|
||||
service user only", PLAN §19). `StateDirectory=nxdns` (0700 matches cli.zig:226),
|
||||
`LogsDirectory=nxdns` (covers logging.output=file; the binary does not create the
|
||||
directory, logging.zig:502), `ConfigurationDirectory=nxdns`.
|
||||
`AmbientCapabilities=CAP_NET_BIND_SERVICE` + `CapabilityBoundingSet=` the same
|
||||
(port 53; 443/853 covered by the same cap). Hardening: `NoNewPrivileges=yes`,
|
||||
`ProtectSystem=strict`, `ProtectHome=yes`, `PrivateTmp=yes`, `PrivateDevices=yes`,
|
||||
`ProtectKernelTunables/Modules/Logs=yes`, `ProtectControlGroups=yes`,
|
||||
`ProtectClock=yes`, `ProtectHostname=yes`, `ProtectProc=invisible`,
|
||||
`RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX`, `RestrictNamespaces=yes`,
|
||||
`RestrictRealtime=yes`, `RestrictSUIDSGID=yes`, `LockPersonality=yes`,
|
||||
`MemoryDenyWriteExecute=yes` (static Zig binary, no JIT), `UMask=0077`,
|
||||
`SystemCallFilter=@system-service`, `SystemCallArchitectures=native`,
|
||||
`Restart=on-failure`, `RestartSec=2`. No `ReadWritePaths` beyond what
|
||||
StateDirectory/LogsDirectory grant. Validate with `systemd-analyze verify` if the
|
||||
build host has it; report honestly if not.
|
||||
4. **systemd unit.** `Type=simple` (no forking, shutdown.zig:35 handles SIGTERM), `ExecStart=/usr/local/bin/nxdns run`, stderr → journald (logging.zig:301 already states this; `logging.output=stderr` stays the default). Static system user `nxdns` via `deploy/systemd/sysusers.conf` (`u nxdns - "nxdns DNS sinkhole"`), NOT DynamicUser — the TLS key must be chown-able to a stable uid ("TLS keys readable by service user only", PLAN §19). `StateDirectory=nxdns` (0700 matches cli.zig:226), `LogsDirectory=nxdns` (covers logging.output=file; the binary does not create the directory, logging.zig:502), `ConfigurationDirectory=nxdns`. `AmbientCapabilities=CAP_NET_BIND_SERVICE` + `CapabilityBoundingSet=` the same (port 53; 443/853 covered by the same cap). Hardening: `NoNewPrivileges=yes`, `ProtectSystem=strict`, `ProtectHome=yes`, `PrivateTmp=yes`, `PrivateDevices=yes`, `ProtectKernelTunables/Modules/Logs=yes`, `ProtectControlGroups=yes`, `ProtectClock=yes`, `ProtectHostname=yes`, `ProtectProc=invisible`, `RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX`, `RestrictNamespaces=yes`, `RestrictRealtime=yes`, `RestrictSUIDSGID=yes`, `LockPersonality=yes`, `MemoryDenyWriteExecute=yes` (static Zig binary, no JIT), `UMask=0077`, `SystemCallFilter=@system-service`, `SystemCallArchitectures=native`, `Restart=on-failure`, `RestartSec=2`. No `ReadWritePaths` beyond what StateDirectory/LogsDirectory grant. Validate with `systemd-analyze verify` if the build host has it; report honestly if not.
|
||||
|
||||
5. **Docker.** Multi-stage: builder stage only stages `ca-certificates` (upstream
|
||||
TLS verification rescans the system CA bundle, tls_client.zig:216 — a scratch
|
||||
image without a bundle breaks every DoH/DoT upstream); final `FROM scratch` with
|
||||
the static musl binary, `/etc/ssl/certs/ca-certificates.crt`, a nonroot numeric
|
||||
`USER 65532:65532`, `VOLUME /var/lib/nxdns`, `EXPOSE 53/udp 53/tcp 8080 443 853`,
|
||||
`ENTRYPOINT ["/nxdns"]`, `CMD ["run"]`. The binary is NOT built inside the
|
||||
Dockerfile (the repo builds it with zig; the Dockerfile COPYes
|
||||
`zig-out/cross/$TARGETARCH-…/nxdns` via a build arg or buildx TARGETARCH mapping —
|
||||
keep it working for both arches). compose.yaml: ports 53:53/udp+tcp and 8080:8080
|
||||
(443/853 commented), bind-mount `./etc-nxdns:/etc/nxdns:ro`, named volume for
|
||||
`/var/lib/nxdns`, `sysctls: net.ipv4.ip_unprivileged_port_start=0` so the nonroot
|
||||
user binds 53 (per-netns sysctl; documented), `restart: unless-stopped`. First
|
||||
boot needs a seeded `/etc/nxdns/config.zon` with a `default` group + one enabled
|
||||
upstream or the container exits 2 (bootstrap.zig:38, app.zig:93) — operator.md and
|
||||
a compose comment both say so. Do NOT point the host's resolv.conf at nxdns
|
||||
itself for the container's own lookups.
|
||||
5. **Docker.** Multi-stage: builder stage only stages `ca-certificates` (upstream TLS verification rescans the system CA bundle, tls_client.zig:216 — a scratch image without a bundle breaks every DoH/DoT upstream); final `FROM scratch` with the static musl binary, `/etc/ssl/certs/ca-certificates.crt`, a nonroot numeric `USER 65532:65532`, `VOLUME /var/lib/nxdns`, `EXPOSE 53/udp 53/tcp 8080 443 853`, `ENTRYPOINT ["/nxdns"]`, `CMD ["run"]`. The binary is NOT built inside the Dockerfile (the repo builds it with zig; the Dockerfile COPYes `zig-out/cross/$TARGETARCH-…/nxdns` via a build arg or buildx TARGETARCH mapping — keep it working for both arches). compose.yaml: ports 53:53/udp+tcp and 8080:8080 (443/853 commented), bind-mount `./etc-nxdns:/etc/nxdns:ro`, named volume for `/var/lib/nxdns`, `sysctls: net.ipv4.ip_unprivileged_port_start=0` so the nonroot user binds 53 (per-netns sysctl; documented), `restart: unless-stopped`. First boot needs a seeded `/etc/nxdns/config.zon` with a `default` group + one enabled upstream or the container exits 2 (bootstrap.zig:38, app.zig:93) — operator.md and a compose comment both say so. Do NOT point the host's resolv.conf at nxdns itself for the container's own lookups.
|
||||
|
||||
6. **CI.** One added job `docker` in ci.yml: after building the x86_64 exe
|
||||
(ReleaseSafe, with the SPA dist like the cross job), `docker build` the image and
|
||||
run a container smoke (seed a minimal config.zon; `nxdns version` + boot + one
|
||||
`dig`-equivalent via the test client or `curl` on 8080/api/health; SIGTERM 0).
|
||||
No registry push — no publish step exists anywhere in this repo or the infra
|
||||
repo's CI, and registry credentials are an infra decision outside this repo.
|
||||
Manual publishing to git.mial.net stays possible and is documented in operator.md
|
||||
in one paragraph.
|
||||
6. **CI.** One added job `docker` in ci.yml: after building the x86_64 exe (ReleaseSafe, with the SPA dist like the cross job), `docker build` the image and run a container smoke (seed a minimal config.zon; `nxdns version` + boot + one `dig`-equivalent via the test client or `curl` on 8080/api/health; SIGTERM 0). No registry push — no publish step exists anywhere in this repo or the infra repo's CI, and registry credentials are an infra decision outside this repo. Manual publishing to git.mial.net stays possible and is documented in operator.md in one paragraph.
|
||||
|
||||
7. **Docs content contracts.**
|
||||
- operator.md: install (systemd path and docker path, both complete), first boot +
|
||||
config.zon seeding semantics (file seeds DB once, DB is truth thereafter,
|
||||
bootstrap.zig:38), auth setup (`web.password` hashed on import, never stored,
|
||||
export writes ""), TLS cert/key provisioning + 0600 expectations + the reload
|
||||
API/watcher, backup/restore = `nxdns export`/`import --force` (+ the 0600 export
|
||||
mode and why), upgrades (schema migration = install + restart, PLAN §20.11),
|
||||
data-dir layout table, exit codes (0/1/2/64, cli.zig:35), CLI reference (all six
|
||||
subcommands + flags), troubleshooting (exit 2 causes, `nxdns check` semantics
|
||||
incl. source-selection order cli.zig:511, disk-full degradation, port 53
|
||||
conflicts with systemd-resolved — include the disable recipe).
|
||||
- architecture.md: module map (the src/ inventory), the purity rule (dns/, filter/,
|
||||
local/, cache/ take bytes, no Io — AGENTS.md), std.Io injection + Threaded
|
||||
backend, data flow for one query (listener → handler → filter/cache/local →
|
||||
upstream → sink/logger), storage split (config.db truth / querylog.db expendable),
|
||||
web stack (std.http over TLS optional, SPA embedded via web_assets, SSE), cert
|
||||
hot-reload design (refcounted CertStore), failure-visibility doctrine (counters +
|
||||
/metrics over log spam). Concise — a map, not a novel.
|
||||
- config-reference.md: complete — every section/field with type, default, unit,
|
||||
validation range, and which subsystem consumes it; collections with required
|
||||
fields; DB-vs-file truth explanation; the `web.password`/`password_hash`
|
||||
exclusivity; `logging.level=.err` serializes as "error" (model.zig:154). The
|
||||
explorer inventory in this milestone's research is the skeleton; verify against
|
||||
model.zig/validate.zig while writing, do not trust the summary blindly.
|
||||
- operator.md: install (systemd path and docker path, both complete), first boot + config.zon seeding semantics (file seeds DB once, DB is truth thereafter, bootstrap.zig:38), auth setup (`web.password` hashed on import, never stored, export writes ""), TLS cert/key provisioning + 0600 expectations + the reload API/watcher, backup/restore = `nxdns export`/`import --force` (+ the 0600 export mode and why), upgrades (schema migration = install + restart, PLAN §20.11), data-dir layout table, exit codes (0/1/2/64, cli.zig:35), CLI reference (all six subcommands + flags), troubleshooting (exit 2 causes, `nxdns check` semantics incl. source-selection order cli.zig:511, disk-full degradation, port 53 conflicts with systemd-resolved — include the disable recipe).
|
||||
- architecture.md: module map (the src/ inventory), the purity rule (dns/, filter/, local/, cache/ take bytes, no Io — AGENTS.md), std.Io injection + Threaded backend, data flow for one query (listener → handler → filter/cache/local → upstream → sink/logger), storage split (config.db truth / querylog.db expendable), web stack (std.http over TLS optional, SPA embedded via web_assets, SSE), cert hot-reload design (refcounted CertStore), failure-visibility doctrine (counters + /metrics over log spam). Concise — a map, not a novel.
|
||||
- config-reference.md: complete — every section/field with type, default, unit, validation range, and which subsystem consumes it; collections with required fields; DB-vs-file truth explanation; the `web.password`/`password_hash` exclusivity; `logging.level=.err` serializes as "error" (model.zig:154). The explorer inventory in this milestone's research is the skeleton; verify against model.zig/validate.zig while writing, do not trust the summary blindly.
|
||||
- api.md: per ruling 2.
|
||||
- README.md: ≤120 lines; what nxdns is, feature list (honest, shipping features
|
||||
only), quickstart (docker compose path), build-from-source (zig build, node for
|
||||
the SPA), doc links, license note if a LICENSE exists (do not invent one).
|
||||
- README.md: ≤120 lines; what nxdns is, feature list (honest, shipping features only), quickstart (docker compose path), build-from-source (zig build, node for the SPA), doc links, license note if a LICENSE exists (do not invent one).
|
||||
|
||||
8. **No new runtime code.** This milestone adds zero behavior to the binary. The only
|
||||
src/ change is the orchestrator's docs_drift_test.zig + its build wiring. If a doc
|
||||
session finds a bug while documenting, it REPORTS it (no fix); the orchestrator
|
||||
triages.
|
||||
8. **No new runtime code.** This milestone adds zero behavior to the binary. The only src/ change is the orchestrator's docs_drift_test.zig + its build wiring. If a doc session finds a bug while documenting, it REPORTS it (no fix); the orchestrator triages.
|
||||
|
||||
9. **Pi 5 end-to-end**: the exit criterion runs on hardware this environment does not
|
||||
have. The deliverable here is: both suites green, docker smoke green on x86_64,
|
||||
`systemd-analyze verify` clean (or honestly reported unavailable), aarch64 binary
|
||||
built and statically verified (existing cross job). The operator doc's Pi 5 recipe
|
||||
is written to be executed by the user; the spec records this boundary explicitly.
|
||||
9. **Pi 5 end-to-end**: the exit criterion runs on hardware this environment does not have. The deliverable here is: both suites green, docker smoke green on x86_64, `systemd-analyze verify` clean (or honestly reported unavailable), aarch64 binary built and statically verified (existing cross job). The operator doc's Pi 5 recipe is written to be executed by the user; the spec records this boundary explicitly.
|
||||
|
||||
## Sessions
|
||||
|
||||
U1, U3, U4, U5 parallel; U2 after U1 (documents the artifacts U1 produces);
|
||||
orchestrator wiring (ruling 3) after U3+U5.
|
||||
U1, U3, U4, U5 parallel; U2 after U1 (documents the artifacts U1 produces); orchestrator wiring (ruling 3) after U3+U5.
|
||||
|
||||
## Session U1: deploy artifacts + CI
|
||||
|
||||
Owns `deploy/systemd/nxdns.service`, `deploy/systemd/sysusers.conf`,
|
||||
`deploy/docker/{Dockerfile,compose.yaml,.dockerignore}`, `.gitea/workflows/ci.yml`
|
||||
(one added job). Rulings 4, 5, 6. Verify: `systemd-analyze verify` (or report
|
||||
unavailable), local `docker build` + container smoke if the docker daemon is
|
||||
reachable (report honestly either way), `zig build test` untouched-green.
|
||||
Owns `deploy/systemd/nxdns.service`, `deploy/systemd/sysusers.conf`, `deploy/docker/{Dockerfile,compose.yaml,.dockerignore}`, `.gitea/workflows/ci.yml` (one added job). Rulings 4, 5, 6. Verify: `systemd-analyze verify` (or report unavailable), local `docker build` + container smoke if the docker daemon is reachable (report honestly either way), `zig build test` untouched-green.
|
||||
|
||||
## Session U2: operator.md + README.md (after U1)
|
||||
|
||||
Owns `docs/operator.md`, `README.md`. Rulings 7 (operator + README). Reads U1's
|
||||
artifacts and the runtime/CLI facts from the code (verify against src/cli.zig,
|
||||
src/app.zig, src/config/bootstrap.zig — not from memory).
|
||||
Owns `docs/operator.md`, `README.md`. Rulings 7 (operator + README). Reads U1's artifacts and the runtime/CLI facts from the code (verify against src/cli.zig, src/app.zig, src/config/bootstrap.zig — not from memory).
|
||||
|
||||
## Session U3: config-reference.md
|
||||
|
||||
Owns `docs/config-reference.md`. Ruling 7. Source of truth: src/config/model.zig +
|
||||
validate.zig + import/export/bootstrap. Every field, no sampling.
|
||||
Owns `docs/config-reference.md`. Ruling 7. Source of truth: src/config/model.zig + validate.zig + import/export/bootstrap. Every field, no sampling.
|
||||
|
||||
## Session U4: architecture.md
|
||||
|
||||
Owns `docs/architecture.md`. Ruling 7. Reads module headers; no deep dives needed
|
||||
beyond what the doc claims.
|
||||
Owns `docs/architecture.md`. Ruling 7. Reads module headers; no deep dives needed beyond what the doc claims.
|
||||
|
||||
## Session U5: api.md
|
||||
|
||||
Owns `docs/api.md`. Rulings 2, 7. Source of truth: src/web/routes.zig (the served
|
||||
table: method, path, auth, limiter) + openapi.yaml summaries + auth.zig/sse.zig for
|
||||
the auth and SSE prose. Every route, no sampling.
|
||||
Owns `docs/api.md`. Rulings 2, 7. Source of truth: src/web/routes.zig (the served table: method, path, auth, limiter) + openapi.yaml summaries + auth.zig/sse.zig for the auth and SSE prose. Every route, no sampling.
|
||||
|
||||
## As built
|
||||
|
||||
**U1** delivered per rulings 4-6 with accepted deviations: `StateDirectoryMode=0700`
|
||||
(systemd defaults 0755; the binary cannot tighten a pre-existing directory) and
|
||||
`User=nxdns`/`Group=nxdns` added to the unit; the Dockerfile's builder stage also maps
|
||||
buildx TARGETARCH → cross-target dir and pre-chowns `/var/lib/nxdns` to 65532 (a named
|
||||
volume seeded from a root-owned image dir would be unwritable on first boot); compose
|
||||
gained a `build:` block; `deploy/docker/.dockerignore` is documentation-grade under
|
||||
BuildKit (only a root `.dockerignore` or `Dockerfile.dockerignore` is honored — the
|
||||
file's header says so). Verified: `systemd-analyze verify` clean modulo the off-host
|
||||
ExecStart path (an ExecStart=/bin/true copy verifies exit 0); full local docker build +
|
||||
smoke passed (binds 53 as uid 65532, /api/health ok, SIGTERM exit 0). The CI docker
|
||||
job probes both 127.0.0.1 and the container IP to survive either runner topology.
|
||||
Compose expects the operator-created seed at `deploy/docker/etc-nxdns/config.zon`
|
||||
(minimal: a `default` group + one enabled upstream), else exit 2.
|
||||
**U1** delivered per rulings 4-6 with accepted deviations: `StateDirectoryMode=0700` (systemd defaults 0755; the binary cannot tighten a pre-existing directory) and `User=nxdns`/`Group=nxdns` added to the unit; the Dockerfile's builder stage also maps buildx TARGETARCH → cross-target dir and pre-chowns `/var/lib/nxdns` to 65532 (a named volume seeded from a root-owned image dir would be unwritable on first boot); compose gained a `build:` block; `deploy/docker/.dockerignore` is documentation-grade under BuildKit (only a root `.dockerignore` or `Dockerfile.dockerignore` is honored — the file's header says so). Verified: `systemd-analyze verify` clean modulo the off-host ExecStart path (an ExecStart=/bin/true copy verifies exit 0); full local docker build + smoke passed (binds 53 as uid 65532, /api/health ok, SIGTERM exit 0). The CI docker job probes both 127.0.0.1 and the container IP to survive either runner topology. Compose expects the operator-created seed at `deploy/docker/etc-nxdns/config.zon` (minimal: a `default` group + one enabled upstream), else exit 2.
|
||||
|
||||
**U2** delivered docs/operator.md (397 lines; systemd + docker + Pi 5 recipes, seeding
|
||||
semantics, auth, TLS, backup/restore, data-dir table, full CLI reference, exit codes,
|
||||
troubleshooting incl. the systemd-resolved DNSStubListener recipe) and README.md
|
||||
(72 lines, no license section — no LICENSE exists). All facts source-verified.
|
||||
**U2** delivered docs/operator.md (397 lines; systemd + docker + Pi 5 recipes, seeding semantics, auth, TLS, backup/restore, data-dir table, full CLI reference, exit codes, troubleshooting incl. the systemd-resolved DNSStubListener recipe) and README.md (72 lines, no license section — no LICENSE exists). All facts source-verified.
|
||||
|
||||
**U3** delivered docs/config-reference.md (12 scalar sections, 9 collections, DB-vs-file
|
||||
truth model, auth section, minimal + annotated examples) and surfaced five code
|
||||
discrepancies during writing (see fix wave below).
|
||||
**U3** delivered docs/config-reference.md (12 scalar sections, 9 collections, DB-vs-file truth model, auth section, minimal + annotated examples) and surfaced five code discrepancies during writing (see fix wave below).
|
||||
|
||||
**U4** delivered docs/architecture.md (module map from the //! headers, purity rule
|
||||
with the honest exceptions, life-of-one-query pipeline verified against handler.zig,
|
||||
storage split, web stack, CertStore design, failure-visibility doctrine). Reported one
|
||||
stale comment (logging.zig:17 cited a moved cli.zig line) — orchestrator fixed the
|
||||
comment to cite start.zig:724 via std.process.Init.
|
||||
**U4** delivered docs/architecture.md (module map from the //! headers, purity rule with the honest exceptions, life-of-one-query pipeline verified against handler.zig, storage split, web stack, CertStore design, failure-visibility doctrine). Reported one stale comment (logging.zig:17 cited a moved cli.zig line) — orchestrator fixed the comment to cite start.zig:724 via std.process.Init.
|
||||
|
||||
**U5** delivered docs/api.md: all 56 operations (matches router.routes.len), auth /
|
||||
rate-limit / SSE prose, error envelope, openapi.yaml pointer. No code-vs-yaml
|
||||
discrepancies found.
|
||||
**U5** delivered docs/api.md: all 56 operations (matches router.routes.len), auth / rate-limit / SSE prose, error envelope, openapi.yaml pointer. No code-vs-yaml discrepancies found.
|
||||
|
||||
**Orchestrator wiring (ruling 3)**: docs/docs.zig (embeds api.md, config-reference.md,
|
||||
operator.md), `docs_files` anonymous import on the test module in build.zig,
|
||||
src/docs_drift_test.zig with three containment guards (routes → api.md; toSettings
|
||||
keys → config-reference.md; the six subcommand names → operator.md), tests.zig import.
|
||||
**Orchestrator wiring (ruling 3)**: docs/docs.zig (embeds api.md, config-reference.md, operator.md), `docs_files` anonymous import on the test module in build.zig, src/docs_drift_test.zig with three containment guards (routes → api.md; toSettings keys → config-reference.md; the six subcommand names → operator.md), tests.zig import.
|
||||
|
||||
**Fix wave (orchestrator-triaged; ruling 8's no-runtime-code rule lifted for exactly
|
||||
these)** — U3's five discrepancies, triaged with stdlib evidence:
|
||||
1. `runtime.io_backend` DELETED end to end (model, settings handler + view, openapi,
|
||||
web types/SettingsPage/settingsDiff + tests, docs). Nothing consumed it — main uses
|
||||
init.io (stdlib Threaded, start.zig:724), and 0.16's std.Io.Evented has stubbed
|
||||
networking (Uring.zig netConnectIp → error.NetworkDown), so PLAN decision E's
|
||||
"io_uring via flag" is not deliverable at this tag. Re-add when std ships working
|
||||
evented net. Old DB rows warn-and-ignore via fromSettings' unknown-key path.
|
||||
2. `upstream.connect_timeout_ms` DELETED. No call site; the pool races the whole
|
||||
attempt against total_timeout (app.zig sets it from totalTimeout); Threaded panics
|
||||
on IpAddress.ConnectOptions.timeout != .none; std.http.Client has no knob. The
|
||||
validate cross-check is now total >= read only.
|
||||
3. `cache.size` KEPT: 0 is clean documented disabled behavior (put short-circuits;
|
||||
in-file test "a cache of zero entries stores nothing"). Doc row corrected.
|
||||
4. `dns.bind_ipv6` TIGHTENED: checkBind generalized to a BindFamily enum; dns.bind_ipv6
|
||||
requires an IPv6 literal (an IPv4 wildcard there made the v4 bind AddressInUse get
|
||||
swallowed with a false "dual-stack" log — silent IPv6 loss). web/TLS binds stay .any.
|
||||
5. doh/dot `readTimeout(.{})` sites KEPT: they are test fixtures; the real idle budget
|
||||
is the ruled 10s Options default. The doc claim was wrong and was removed.
|
||||
Settings key counts after deletion: toSettings emits 43 (incl. web.password_hash);
|
||||
the API-visible restart-required set is 42 (was 44).
|
||||
**Fix wave (orchestrator-triaged; ruling 8's no-runtime-code rule lifted for exactly these)** — U3's five discrepancies, triaged with stdlib evidence:
|
||||
1. `runtime.io_backend` DELETED end to end (model, settings handler + view, openapi, web types/SettingsPage/settingsDiff + tests, docs). Nothing consumed it — main uses init.io (stdlib Threaded, start.zig:724), and 0.16's std.Io.Evented has stubbed networking (Uring.zig netConnectIp → error.NetworkDown), so PLAN decision E's "io_uring via flag" is not deliverable at this tag. Re-add when std ships working evented net. Old DB rows warn-and-ignore via fromSettings' unknown-key path.
|
||||
2. `upstream.connect_timeout_ms` DELETED. No call site; the pool races the whole attempt against total_timeout (app.zig sets it from totalTimeout); Threaded panics on IpAddress.ConnectOptions.timeout != .none; std.http.Client has no knob. The validate cross-check is now total >= read only.
|
||||
3. `cache.size` KEPT: 0 is clean documented disabled behavior (put short-circuits; in-file test "a cache of zero entries stores nothing"). Doc row corrected.
|
||||
4. `dns.bind_ipv6` TIGHTENED: checkBind generalized to a BindFamily enum; dns.bind_ipv6 requires an IPv6 literal (an IPv4 wildcard there made the v4 bind AddressInUse get swallowed with a false "dual-stack" log — silent IPv6 loss). web/TLS binds stay .any.
|
||||
5. doh/dot `readTimeout(.{})` sites KEPT: they are test fixtures; the real idle budget is the ruled 10s Options default. The doc claim was wrong and was removed. Settings key counts after deletion: toSettings emits 43 (incl. web.password_hash); the API-visible restart-required set is 42 (was 44).
|
||||
|
||||
## Review (Codex, as built)
|
||||
|
||||
Three rounds on one thread; round 3 returned "No findings."
|
||||
|
||||
Round 1 (5 important): stale-DB bind_ipv6 rows bypassed the new validate check at boot
|
||||
→ app.zig parseBind now enforces the IP family on both dns binds (BadBindAddress,
|
||||
exit 2, remedy in the message; the v4 side had the symmetric hole); app.zig's tests
|
||||
were not collected by tests.zig at all — the import line was added and the new test
|
||||
runs. operator.md's TLS recipe assumed the nxdns host user → Docker path now chowns
|
||||
65532:65532 numerically host-side (read-only bind mount). PLAN.md still promised the
|
||||
io_uring flag and connect_timeout_ms → synced (Io bullet records the drop with stdlib
|
||||
evidence; decision E row; example config). Drift guards were maskable → api.md guard
|
||||
anchors the full "| METHOD | `pattern` |" row per operation; operator.md guard anchors
|
||||
the "### `name" reference headings.
|
||||
Round 1 (5 important): stale-DB bind_ipv6 rows bypassed the new validate check at boot → app.zig parseBind now enforces the IP family on both dns binds (BadBindAddress, exit 2, remedy in the message; the v4 side had the symmetric hole); app.zig's tests were not collected by tests.zig at all — the import line was added and the new test runs. operator.md's TLS recipe assumed the nxdns host user → Docker path now chowns 65532:65532 numerically host-side (read-only bind mount). PLAN.md still promised the io_uring flag and connect_timeout_ms → synced (Io bullet records the drop with stdlib evidence; decision E row; example config). Drift guards were maskable → api.md guard anchors the full "| METHOD | `pattern` |" row per operation; operator.md guard anchors the "### `name" reference headings.
|
||||
|
||||
Round 2 (1 important, 1 minor): seed-permission guidance covered only web.password →
|
||||
now web.password or web.password_hash (a restored export); PLAN.md's "stubs all
|
||||
networking" overstated 0.16's Uring — now names the stubbed operations precisely.
|
||||
Round 2 (1 important, 1 minor): seed-permission guidance covered only web.password → now web.password or web.password_hash (a restored export); PLAN.md's "stubs all networking" overstated 0.16's Uring — now names the stubbed operations precisely.
|
||||
|
||||
Final gates: plain 1171/1284 passed, 113 skipped (integration-gated), 0 failed;
|
||||
integration 1280/1284, 4 skipped (live-network by design), 0 failed; cross ReleaseSafe
|
||||
with the SPA dist 18/18; web suite 121/121 with format/lint/typecheck clean.
|
||||
Final gates: plain 1171/1284 passed, 113 skipped (integration-gated), 0 failed; integration 1280/1284, 4 skipped (live-network by design), 0 failed; cross ReleaseSafe with the SPA dist 18/18; web suite 121/121 with format/lint/typecheck clean.
|
||||
|
||||
## Module layout (new)
|
||||
|
||||
deploy/systemd/{nxdns.service,sysusers.conf}, deploy/docker/{Dockerfile,compose.yaml,
|
||||
.dockerignore}, docs/{operator,architecture,config-reference,api}.md, README.md,
|
||||
src/docs_drift_test.zig (orchestrator).
|
||||
deploy/systemd/{nxdns.service,sysusers.conf}, deploy/docker/{Dockerfile,compose.yaml, .dockerignore}, docs/{operator,architecture,config-reference,api}.md, README.md, src/docs_drift_test.zig (orchestrator).
|
||||
|
||||
## File ownership
|
||||
|
||||
U1 deploy/* + ci.yml; U2 docs/operator.md + README.md; U3 docs/config-reference.md;
|
||||
U4 docs/architecture.md; U5 docs/api.md; orchestrator src/docs_drift_test.zig,
|
||||
build.zig (docs_files module), src/tests.zig.
|
||||
U1 deploy/* + ci.yml; U2 docs/operator.md + README.md; U3 docs/config-reference.md; U4 docs/architecture.md; U5 docs/api.md; orchestrator src/docs_drift_test.zig, build.zig (docs_files module), src/tests.zig.
|
||||
|
||||
## Acceptance (milestone complete)
|
||||
|
||||
@@ -268,6 +107,5 @@ build.zig (docs_files module), src/tests.zig.
|
||||
|
||||
- No vendored API-doc renderer (redoc/scalar/swagger-ui), no YAML parser.
|
||||
- No registry publish step; no k3s manifests (the infra repo owns deployment there).
|
||||
- No SIGHUP/reload feature, no env-var config, no new CLI flags — document what
|
||||
exists; report gaps instead of filling them.
|
||||
- No SIGHUP/reload feature, no env-var config, no new CLI flags — document what exists; report gaps instead of filling them.
|
||||
- No LICENSE invention; no badges or marketing prose in README.
|
||||
|
||||
+23
-119
@@ -1,83 +1,31 @@
|
||||
# Milestone 12: performance measurement pass + aarch64 test execution
|
||||
|
||||
Goal: close the two gaps the post-Phase-10 completeness sweep found — PLAN §18 has no
|
||||
measurement infrastructure (deferred at specs/milestone-7.md:113 and never delivered),
|
||||
and aarch64 tests are cross-built but never executed (PLAN.md:145 promised qemu).
|
||||
Goal: close the two gaps the post-Phase-10 completeness sweep found — PLAN §18 has no measurement infrastructure (deferred at specs/milestone-7.md:113 and never delivered), and aarch64 tests are cross-built but never executed (PLAN.md:145 promised qemu).
|
||||
|
||||
## Rulings (binding)
|
||||
|
||||
1. **Bench harness = `tools/bench.zig` + `zig build bench`.** Not in src/ — src/ is
|
||||
the shipped product; tests.zig aggregates everything shippable and the bench must
|
||||
be a separate compilation anyway (one-file-one-module rule; matcher.zig and
|
||||
dns_cache.zig already belong to the test compilation). Module wiring per the fuzz
|
||||
pattern (build.zig:68-92): modules rooted at src/filter/matcher.zig and
|
||||
src/cache/dns_cache.zig — their relative import closures come along; no
|
||||
sqlite/mbedTLS linking needed. `if (b.args) |args| run.addArgs(args)`.
|
||||
1. **Bench harness = `tools/bench.zig` + `zig build bench`.** Not in src/ — src/ is the shipped product; tests.zig aggregates everything shippable and the bench must be a separate compilation anyway (one-file-one-module rule; matcher.zig and dns_cache.zig already belong to the test compilation). Module wiring per the fuzz pattern (build.zig:68-92): modules rooted at src/filter/matcher.zig and src/cache/dns_cache.zig — their relative import closures come along; no sqlite/mbedTLS linking needed. `if (b.args) |args| run.addArgs(args)`.
|
||||
|
||||
2. **What it measures** (PLAN §18 targets):
|
||||
- `filter`: `matcher.normalize` + `Snapshot.evaluate` per op against a Snapshot
|
||||
built from a generated, sorted `d{d:0>7}.example.com` list body (~1M lines,
|
||||
in-memory; DomainSet.max_count 4M and compiler.max_domains 2M leave headroom)
|
||||
plus a small wild body. Mixed case ratios: hit, miss, parent-walk. Target
|
||||
p95 < 1 ms.
|
||||
- `cache`: `buildKey` + `DnsCache.get` + `packet.setId` (the handler's hit path;
|
||||
TTL aging happens inside get) on a ~10k-entry cache prefilled with the fuzz
|
||||
corpus response. Mixed hit/miss. Target p95 < 5 ms.
|
||||
- `compile`: `compiler.compile` over a generated 1M-line hosts body — wall time,
|
||||
informational (no §18 target; no datapoint exists today).
|
||||
- Memory: `/proc/self/status` VmRSS (nothing in-repo wraps it; read it directly)
|
||||
plus the in-repo accounting (`Snapshot.memoryBytes`, `DnsCache.memoryBytes`).
|
||||
Target < 100 MB RSS with ~1M domains loaded.
|
||||
- Timing: `std.Io.Clock.awake.now(io)` / `durationTo` — std.time.Timer does not
|
||||
exist at 0.16. Percentiles: collect per-op nanos in a preallocated []u64, sort
|
||||
(std.mem.sort), report p50/p95/p99/max. No percentile helper exists; write it
|
||||
in the tool.
|
||||
- `filter`: `matcher.normalize` + `Snapshot.evaluate` per op against a Snapshot built from a generated, sorted `d{d:0>7}.example.com` list body (~1M lines, in-memory; DomainSet.max_count 4M and compiler.max_domains 2M leave headroom) plus a small wild body. Mixed case ratios: hit, miss, parent-walk. Target p95 < 1 ms.
|
||||
- `cache`: `buildKey` + `DnsCache.get` + `packet.setId` (the handler's hit path; TTL aging happens inside get) on a ~10k-entry cache prefilled with the fuzz corpus response. Mixed hit/miss. Target p95 < 5 ms.
|
||||
- `compile`: `compiler.compile` over a generated 1M-line hosts body — wall time, informational (no §18 target; no datapoint exists today).
|
||||
- Memory: `/proc/self/status` VmRSS (nothing in-repo wraps it; read it directly) plus the in-repo accounting (`Snapshot.memoryBytes`, `DnsCache.memoryBytes`). Target < 100 MB RSS with ~1M domains loaded.
|
||||
- Timing: `std.Io.Clock.awake.now(io)` / `durationTo` — std.time.Timer does not exist at 0.16. Percentiles: collect per-op nanos in a preallocated []u64, sort (std.mem.sort), report p50/p95/p99/max. No percentile helper exists; write it in the tool.
|
||||
|
||||
3. **Assertion policy.** Default run is informational (prints a table). `--assert`
|
||||
exits non-zero when a §18 target is exceeded — for the Pi 5 run, NOT for CI:
|
||||
required CI stays deterministic (AGENTS.md) and perf assertions on shared runners
|
||||
are flake generators. CI does not run the bench at all this milestone.
|
||||
3. **Assertion policy.** Default run is informational (prints a table). `--assert` exits non-zero when a §18 target is exceeded — for the Pi 5 run, NOT for CI: required CI stays deterministic (AGENTS.md) and perf assertions on shared runners are flake generators. CI does not run the bench at all this milestone.
|
||||
|
||||
4. **Bench flags**: `--domains=N` (default 1_000_000), `--iters=N` (default 200_000),
|
||||
`--seed=N` (default fixed), `--assert`, subcommands `filter|cache|compile|all`
|
||||
(default all). Debug-mode runs print a one-line warning recommending
|
||||
`-Doptimize=ReleaseFast`.
|
||||
4. **Bench flags**: `--domains=N` (default 1_000_000), `--iters=N` (default 200_000), `--seed=N` (default fixed), `--assert`, subcommands `filter|cache|compile|all` (default all). Debug-mode runs print a one-line warning recommending `-Doptimize=ReleaseFast`.
|
||||
|
||||
5. **docs/performance.md**: the §18 targets table; measured numbers from THIS x86_64
|
||||
host (dated, hardware named honestly, marked as not-the-target-platform); the
|
||||
in-memory accounting numbers; the one-command Pi 5 recipe
|
||||
(`zig build bench -Doptimize=ReleaseFast -- --assert`); a sentence on why CI does
|
||||
not gate on perf. The Pi 5 rows stay "to be measured on hardware".
|
||||
5. **docs/performance.md**: the §18 targets table; measured numbers from THIS x86_64 host (dated, hardware named honestly, marked as not-the-target-platform); the in-memory accounting numbers; the one-command Pi 5 recipe (`zig build bench -Doptimize=ReleaseFast -- --assert`); a sentence on why CI does not gate on perf. The Pi 5 rows stay "to be measured on hardware".
|
||||
|
||||
6. **aarch64 execution = second test artifact, `zig build test-aarch64 -fqemu`.**
|
||||
Per stdlib evidence: Run steps try qemu only when enable_qemu (the `-fqemu` CLI
|
||||
flag; a *Build field, no per-step override) and exec `qemu-aarch64` bare from
|
||||
PATH — so CI installs `qemu-user` (NOT qemu-user-static, which ships the
|
||||
`-static` name Zig will not find; binfmt is unnecessary). Static musl means no
|
||||
sysroot/--libc-runtimes. build.zig: resolve aarch64-linux-musl, addTest with the
|
||||
same wiring as the native tests artifact (sqlite/mbedtls helpers are already
|
||||
target-parameterized), set `.linkage = .static` on the artifact (TestOptions has
|
||||
no linkage field), `run.skip_foreign_checks = true`, leave
|
||||
failing_to_execute_foreign_is_an_error true so a missing qemu is loud. Fuzz
|
||||
artifacts excluded.
|
||||
6. **aarch64 execution = second test artifact, `zig build test-aarch64 -fqemu`.** Per stdlib evidence: Run steps try qemu only when enable_qemu (the `-fqemu` CLI flag; a *Build field, no per-step override) and exec `qemu-aarch64` bare from PATH — so CI installs `qemu-user` (NOT qemu-user-static, which ships the `-static` name Zig will not find; binfmt is unnecessary). Static musl means no sysroot/--libc-runtimes. build.zig: resolve aarch64-linux-musl, addTest with the same wiring as the native tests artifact (sqlite/mbedtls helpers are already target-parameterized), set `.linkage = .static` on the artifact (TestOptions has no linkage field), `run.skip_foreign_checks = true`, leave failing_to_execute_foreign_is_an_error true so a missing qemu is loud. Fuzz artifacts excluded.
|
||||
|
||||
7. **qemu scope: plain suite only, blocking.** No `-Dintegration` under qemu: the
|
||||
integration tests are multithreaded loopback TLS with wall-clock budgets, and
|
||||
qemu-user's 5-20x slowdown makes them a flake source — required CI stays
|
||||
deterministic. The plain suite (the aggregator's tests without the 12
|
||||
fuzz-artifact tests: 1159 pass + 113 integration-gated skips; all pure
|
||||
DNS/filter/cache logic included)
|
||||
is the portable-correctness signal aarch64 needs. Recorded here as the
|
||||
engineering call closing PLAN.md:145's promise.
|
||||
7. **qemu scope: plain suite only, blocking.** No `-Dintegration` under qemu: the integration tests are multithreaded loopback TLS with wall-clock budgets, and qemu-user's 5-20x slowdown makes them a flake source — required CI stays deterministic. The plain suite (the aggregator's tests without the 12 fuzz-artifact tests: 1159 pass + 113 integration-gated skips; all pure DNS/filter/cache logic included) is the portable-correctness signal aarch64 needs. Recorded here as the engineering call closing PLAN.md:145's promise.
|
||||
|
||||
8. **CI additions**: one `test-aarch64` job (setup-zig, apt qemu-user,
|
||||
`zig build test-aarch64 -fqemu`); plus the missing §18 size assert — the cross
|
||||
job gains a second build WITHOUT `-Dweb-dist` (placeholder dist) and asserts the
|
||||
stripped copies < 10 MiB per arch (the < 15 MiB with-assets assert already
|
||||
exists). Job graph otherwise untouched.
|
||||
8. **CI additions**: one `test-aarch64` job (setup-zig, apt qemu-user, `zig build test-aarch64 -fqemu`); plus the missing §18 size assert — the cross job gains a second build WITHOUT `-Dweb-dist` (placeholder dist) and asserts the stripped copies < 10 MiB per arch (the < 15 MiB with-assets assert already exists). Job graph otherwise untouched.
|
||||
|
||||
9. **No src/ changes.** matcher/dns_cache/compiler public APIs are used as-is; if
|
||||
the bench needs something they do not expose, report — do not extend them.
|
||||
9. **No src/ changes.** matcher/dns_cache/compiler public APIs are used as-is; if the bench needs something they do not expose, report — do not extend them.
|
||||
|
||||
## Sessions
|
||||
|
||||
@@ -85,70 +33,29 @@ V1 then V2 (both edit build.zig — sequenced, not parallel).
|
||||
|
||||
## Session V1: bench harness + performance doc
|
||||
|
||||
Owns tools/bench.zig, build.zig (the bench step only), docs/performance.md.
|
||||
Rulings 1-5. Runs the bench on this host (ReleaseFast) and writes the measured
|
||||
numbers into the doc.
|
||||
Owns tools/bench.zig, build.zig (the bench step only), docs/performance.md. Rulings 1-5. Runs the bench on this host (ReleaseFast) and writes the measured numbers into the doc.
|
||||
|
||||
### V1 As built
|
||||
|
||||
Delivered tools/bench.zig, the build.zig bench step, and docs/performance.md. One
|
||||
deviation from ruling 1's wiring: Zig 0.16 rejects separate modules rooted at
|
||||
matcher.zig and dns_cache.zig ("file exists in multiple modules" — their closures
|
||||
share model.zig/types.zig; verified with a minimal repro). Instead a b.addWriteFiles
|
||||
stage copies src/ plus a generated 7-line aggregator root (bench_core.zig) into one
|
||||
`core` module — the same staging trick the web assets use; a build.zig comment
|
||||
records why. The fuzz-corpus response is a byte-for-byte copy with provenance
|
||||
comment (importing corpus.zig recreates the module conflict; the corpus documents
|
||||
copies as house style). /proc/self/status reads use readerStreaming (procfs stats
|
||||
size 0, readFileAlloc returns empty). Measured on the build host (i7-14700K,
|
||||
ReleaseFast, defaults: 1M domains, 200k iters): filter p95 0.18 µs (target < 1 ms),
|
||||
cache p95 0.14 µs (target < 5 ms), VmRSS 31.8 MiB with the 1M-domain snapshot
|
||||
(target < 100 MiB), Snapshot.memoryBytes 28.0 MiB, compile 96 ms ≈ 10.4M lines/s
|
||||
(informational). --assert exit paths verified both ways (a 4M-domain run exceeds
|
||||
the 1M-scoped RSS target and exits 1; that was an exit-path exercise, not a target
|
||||
miss). No src/ changes; matcher/dns_cache/compiler APIs sufficed. Cache suite fixes
|
||||
entries at 10k (the handler-default Config.size); --domains shapes filter and
|
||||
compile only. Both suites 0 failed after the build.zig edit.
|
||||
Delivered tools/bench.zig, the build.zig bench step, and docs/performance.md. One deviation from ruling 1's wiring: Zig 0.16 rejects separate modules rooted at matcher.zig and dns_cache.zig ("file exists in multiple modules" — their closures share model.zig/types.zig; verified with a minimal repro). Instead a b.addWriteFiles stage copies src/ plus a generated 7-line aggregator root (bench_core.zig) into one `core` module — the same staging trick the web assets use; a build.zig comment records why. The fuzz-corpus response is a byte-for-byte copy with provenance comment (importing corpus.zig recreates the module conflict; the corpus documents copies as house style). /proc/self/status reads use readerStreaming (procfs stats size 0, readFileAlloc returns empty). Measured on the build host (i7-14700K, ReleaseFast, defaults: 1M domains, 200k iters): filter p95 0.18 µs (target < 1 ms), cache p95 0.14 µs (target < 5 ms), VmRSS 31.8 MiB with the 1M-domain snapshot (target < 100 MiB), Snapshot.memoryBytes 28.0 MiB, compile 96 ms ≈ 10.4M lines/s (informational). --assert exit paths verified both ways (a 4M-domain run exceeds the 1M-scoped RSS target and exits 1; that was an exit-path exercise, not a target miss). No src/ changes; matcher/dns_cache/compiler APIs sufficed. Cache suite fixes entries at 10k (the handler-default Config.size); --domains shapes filter and compile only. Both suites 0 failed after the build.zig edit.
|
||||
|
||||
## Session V2: aarch64 test execution + CI (after V1)
|
||||
|
||||
Owns build.zig (the test-aarch64 artifact/step only), .gitea/workflows/ci.yml
|
||||
(the new job + the cross-job size-assert addition). Rulings 6-8. Runs
|
||||
`zig build test-aarch64 -fqemu` locally if qemu-aarch64 is installable/present;
|
||||
reports honestly if not.
|
||||
Owns build.zig (the test-aarch64 artifact/step only), .gitea/workflows/ci.yml (the new job + the cross-job size-assert addition). Rulings 6-8. Runs `zig build test-aarch64 -fqemu` locally if qemu-aarch64 is installable/present; reports honestly if not.
|
||||
|
||||
### V2 As built
|
||||
|
||||
build.zig gained the test-aarch64 block after the bench block: aarch64-linux-musl
|
||||
addTest with wiring identical to the native tests artifact, `.linkage = .static` on
|
||||
the artifact, `run.skip_foreign_checks = true`, failing-to-execute left loud. Stdlib
|
||||
mechanics re-verified (Run.zig:71/78/226, Build.zig:73, system.zig:111 — bare
|
||||
`qemu-aarch64` from PATH). ci.yml gained the test-aarch64 job (apt qemu-user,
|
||||
`zig build test-aarch64 -fqemu`) and the cross job builds a second no-dist pair to
|
||||
`--prefix zig-out/nodist` (existing with-assets steps byte-identical) with a
|
||||
build.zig gained the test-aarch64 block after the bench block: aarch64-linux-musl addTest with wiring identical to the native tests artifact, `.linkage = .static` on the artifact, `run.skip_foreign_checks = true`, failing-to-execute left loud. Stdlib mechanics re-verified (Run.zig:71/78/226, Build.zig:73, system.zig:111 — bare `qemu-aarch64` from PATH). ci.yml gained the test-aarch64 job (apt qemu-user, `zig build test-aarch64 -fqemu`) and the cross job builds a second no-dist pair to `--prefix zig-out/nodist` (existing with-assets steps byte-identical) with a
|
||||
< 10 MiB stripped assert; the new assert skips the static-linkage recheck (proven
|
||||
on the same-config with-assets build). Local verification: real qemu execution —
|
||||
1159 pass / 113 skip / 0 failed (native count minus the 12 excluded fuzz tests);
|
||||
no-dist stripped sizes 5,880,336 (x86_64) and 5,212,640 (aarch64) bytes. No
|
||||
-Dintegration under qemu per ruling 7.
|
||||
on the same-config with-assets build). Local verification: real qemu execution — 1159 pass / 113 skip / 0 failed (native count minus the 12 excluded fuzz tests); no-dist stripped sizes 5,880,336 (x86_64) and 5,212,640 (aarch64) bytes. No -Dintegration under qemu per ruling 7.
|
||||
|
||||
## Review (Codex, as built)
|
||||
|
||||
Two rounds on one thread; round 2 returned "No findings."
|
||||
|
||||
Round 1 (2 important, 2 minor): bench.zig's explicit body.deinit left the earlier
|
||||
errdefer armed on an undefined list (double-free on any later error) → clearAndFree
|
||||
keeps the list valid. Ubuntu's qemu-user recommends qemu-user-binfmt, which apt
|
||||
installs by default — binfmt registration would bypass the -fqemu path CI intends
|
||||
to exercise → --no-install-recommends. performance.md's headroom claim overstated
|
||||
the memory margin (3x, not orders of magnitude) → corrected. Ruling 7 quoted the
|
||||
native aggregate test count instead of the aarch64 artifact's own (1159 + 113
|
||||
skips, fuzz excluded) → corrected.
|
||||
Round 1 (2 important, 2 minor): bench.zig's explicit body.deinit left the earlier errdefer armed on an undefined list (double-free on any later error) → clearAndFree keeps the list valid. Ubuntu's qemu-user recommends qemu-user-binfmt, which apt installs by default — binfmt registration would bypass the -fqemu path CI intends to exercise → --no-install-recommends. performance.md's headroom claim overstated the memory margin (3x, not orders of magnitude) → corrected. Ruling 7 quoted the native aggregate test count instead of the aarch64 artifact's own (1159 + 113 skips, fuzz excluded) → corrected.
|
||||
|
||||
Final gates: plain 1171/1284 passed, 113 skipped (integration-gated), 0 failed;
|
||||
integration 1280/1284, 4 skipped (live-network by design), 0 failed; qemu aarch64
|
||||
run 1159 passed, 113 skipped, 0 failed; cross ReleaseSafe with dist 18/18; no-dist
|
||||
stripped sizes 5.9/5.2 MB under the 10 MiB assert.
|
||||
Final gates: plain 1171/1284 passed, 113 skipped (integration-gated), 0 failed; integration 1280/1284, 4 skipped (live-network by design), 0 failed; qemu aarch64 run 1159 passed, 113 skipped, 0 failed; cross ReleaseSafe with dist 18/18; no-dist stripped sizes 5.9/5.2 MB under the 10 MiB assert.
|
||||
|
||||
## Module layout (new)
|
||||
|
||||
@@ -156,8 +63,7 @@ tools/bench.zig, docs/performance.md.
|
||||
|
||||
## File ownership
|
||||
|
||||
V1 tools/bench.zig + docs/performance.md + build.zig(bench); V2 build.zig(aarch64) +
|
||||
ci.yml. Orchestrator: spec sync, review, commit.
|
||||
V1 tools/bench.zig + docs/performance.md + build.zig(bench); V2 build.zig(aarch64) + ci.yml. Orchestrator: spec sync, review, commit.
|
||||
|
||||
## Acceptance (milestone complete)
|
||||
|
||||
@@ -174,6 +80,4 @@ ci.yml. Orchestrator: spec sync, review, commit.
|
||||
|
||||
- No perf assertions in CI; no bench in the test step.
|
||||
- No qemu integration suite; no binfmt setup; no qemu-user-static.
|
||||
- No new pub API on matcher/dns_cache/compiler; no synthetic-load DNS server
|
||||
benchmark (the ≥100 qps target is end-to-end on the Pi — the operator recipe
|
||||
covers it via the real binary, not a harness).
|
||||
- No new pub API on matcher/dns_cache/compiler; no synthetic-load DNS server benchmark (the ≥100 qps target is end-to-end on the Pi — the operator recipe covers it via the real binary, not a harness).
|
||||
|
||||
+89
-460
@@ -1,76 +1,30 @@
|
||||
# Milestone 13: restructure the documentation to Diátaxis
|
||||
|
||||
Goal: replace the four mixed-mode documents with the four Diátaxis modes, add the
|
||||
missing tutorial, and prove every instruction by running it.
|
||||
Goal: replace the four mixed-mode documents with the four Diátaxis modes, add the missing tutorial, and prove every instruction by running it.
|
||||
|
||||
## Rulings (binding)
|
||||
|
||||
1. **Four directories plus an index.** `docs/tutorial/`, `docs/how-to/`,
|
||||
`docs/reference/`, `docs/explanation/`, and `docs/README.md` — the index names
|
||||
the four modes, says who each is for, and links every page. The old
|
||||
`docs/{operator,architecture,config-reference,api,performance}.md` are deleted
|
||||
by the orchestrator once the new pages exist; no redirect stubs (greenfield
|
||||
repo, AGENTS.md forbids compatibility leftovers).
|
||||
1. **Four directories plus an index.** `docs/tutorial/`, `docs/how-to/`, `docs/reference/`, `docs/explanation/`, and `docs/README.md` — the index names the four modes, says who each is for, and links every page. The old `docs/{operator,architecture,config-reference,api,performance}.md` are deleted by the orchestrator once the new pages exist; no redirect stubs (greenfield repo, AGENTS.md forbids compatibility leftovers).
|
||||
|
||||
2. **A document serves one mode.** The defect being fixed is `operator.md`, which
|
||||
interleaves install procedure, CLI reference, exit-code tables and
|
||||
troubleshooting in 418 lines. Reference pages state what is; how-to pages state
|
||||
what to do; explanation states why; the tutorial teaches. When porting text,
|
||||
move a paragraph to the mode it belongs to rather than keeping it where it was.
|
||||
2. **A document serves one mode.** The defect being fixed is `operator.md`, which interleaves install procedure, CLI reference, exit-code tables and troubleshooting in 418 lines. Reference pages state what is; how-to pages state what to do; explanation states why; the tutorial teaches. When porting text, move a paragraph to the mode it belongs to rather than keeping it where it was.
|
||||
|
||||
3. **Every command block in `tutorial/` and `how-to/` is executed verbatim by the
|
||||
session that writes it**, on this host, before the session reports. A command
|
||||
that cannot run here (needs a Raspberry Pi, root, a domain, hardware TLS) is
|
||||
marked in the page itself as not verified on this host, with the reason. This
|
||||
ruling exists because milestone 11's docs were written from source-reading
|
||||
alone: the first real blocklist download aborted the process (commit 35f2324)
|
||||
and a stale SPA bundle crashed the settings page — both would have surfaced if
|
||||
the documented paths had been run. No session may report a page complete on the
|
||||
strength of having read the code. This ruling constrains **command lines**;
|
||||
a pasted **output line** may carry a placeholder where the literal text
|
||||
would go stale, as milestone-17 ruling 8 does with `nxdns <version>`.
|
||||
3. **Every command block in `tutorial/` and `how-to/` is executed verbatim by the session that writes it**, on this host, before the session reports. A command that cannot run here (needs a Raspberry Pi, root, a domain, hardware TLS) is marked in the page itself as not verified on this host, with the reason. This ruling exists because milestone 11's docs were written from source-reading alone: the first real blocklist download aborted the process (commit 35f2324) and a stale SPA bundle crashed the settings page — both would have surfaced if the documented paths had been run. No session may report a page complete on the strength of having read the code. This ruling constrains **command lines**; a pasted **output line** may carry a placeholder where the literal text would go stale, as milestone-17 ruling 8 does with `nxdns <version>`.
|
||||
|
||||
4. **Content is verified against `src/`, not copied from the old pages.** Defaults,
|
||||
flags, exit codes, paths and route names come from the code at HEAD. Report any
|
||||
discrepancy found; do not fix source in this milestone (docs-only, ruling 9).
|
||||
4. **Content is verified against `src/`, not copied from the old pages.** Defaults, flags, exit codes, paths and route names come from the code at HEAD. Report any discrepancy found; do not fix source in this milestone (docs-only, ruling 9).
|
||||
|
||||
5. **The tutorial has one guaranteed outcome.** `docs/tutorial/first-run.md` takes
|
||||
a reader from a clean checkout to a running nxdns that answers a query and
|
||||
blocks a domain from a real blocklist, on unprivileged ports in a scratch
|
||||
directory, then stops it cleanly. It teaches by doing and states what the reader
|
||||
will have at the end. No branching, no options menu, no "you may also".
|
||||
5. **The tutorial has one guaranteed outcome.** `docs/tutorial/first-run.md` takes a reader from a clean checkout to a running nxdns that answers a query and blocks a domain from a real blocklist, on unprivileged ports in a scratch directory, then stops it cleanly. It teaches by doing and states what the reader will have at the end. No branching, no options menu, no "you may also".
|
||||
|
||||
6. **File map.**
|
||||
- `docs/tutorial/first-run.md` — new.
|
||||
- `docs/how-to/`: `install-with-systemd.md` (incl. the Raspberry Pi 5 aarch64
|
||||
binary), `install-with-docker.md`, `upgrade.md`, `troubleshoot.md`,
|
||||
`enable-doh-and-dot.md`, `set-up-admin-authentication.md`,
|
||||
`back-up-and-restore.md`, `measure-performance.md`.
|
||||
- `docs/reference/`: `configuration.md` (every section, field, default, range),
|
||||
`api.md` (all 56 operations), `cli.md` (six subcommands, every flag, exit
|
||||
codes), `files-and-directories.md` (data dir layout, file modes),
|
||||
`performance.md` (targets + measured numbers).
|
||||
- `docs/explanation/`: `architecture.md`, `configuration-model.md` (the file
|
||||
seeds the database once, the database is truth, export/import round trip),
|
||||
`performance-and-testing.md` (why the targets exist, why CI does not gate on
|
||||
them, what hermetic tests do and do not prove).
|
||||
- `docs/how-to/`: `install-with-systemd.md` (incl. the Raspberry Pi 5 aarch64 binary), `install-with-docker.md`, `upgrade.md`, `troubleshoot.md`, `enable-doh-and-dot.md`, `set-up-admin-authentication.md`, `back-up-and-restore.md`, `measure-performance.md`.
|
||||
- `docs/reference/`: `configuration.md` (every section, field, default, range), `api.md` (all 56 operations), `cli.md` (six subcommands, every flag, exit codes), `files-and-directories.md` (data dir layout, file modes), `performance.md` (targets + measured numbers).
|
||||
- `docs/explanation/`: `architecture.md`, `configuration-model.md` (the file seeds the database once, the database is truth, export/import round trip), `performance-and-testing.md` (why the targets exist, why CI does not gate on them, what hermetic tests do and do not prove).
|
||||
|
||||
7. **Drift guards repointed and kept honest.** `docs/docs.zig` embeds
|
||||
`reference/api.md`, `reference/configuration.md`, `reference/cli.md`.
|
||||
`src/docs_drift_test.zig` keeps its three guards against those paths: every
|
||||
served route as a full `| METHOD | \`pattern\` |` row; every `toSettings` key
|
||||
verbatim; every subcommand as its own heading. `reference/cli.md` gives each
|
||||
subcommand a `## \`name\`` heading so the guard anchors on structure, not prose.
|
||||
7. **Drift guards repointed and kept honest.** `docs/docs.zig` embeds `reference/api.md`, `reference/configuration.md`, `reference/cli.md`. `src/docs_drift_test.zig` keeps its three guards against those paths: every served route as a full `| METHOD | \`pattern\` |` row; every `toSettings` key verbatim; every subcommand as its own heading. `reference/cli.md` gives each subcommand a `## \`name\`` heading so the guard anchors on structure, not prose.
|
||||
|
||||
8. **Style.** Plain sentences. No banned vocabulary (leverage, seamless, robust as
|
||||
filler, rule-of-three padding, "it's important to note"). No badges, no
|
||||
marketing. Code blocks are copy-pasteable and use the scratch paths the
|
||||
tutorial establishes, not `/var/lib/nxdns`, unless the page is about a real
|
||||
install.
|
||||
8. **Style.** Plain sentences. No banned vocabulary (leverage, seamless, robust as filler, rule-of-three padding, "it's important to note"). No badges, no marketing. Code blocks are copy-pasteable and use the scratch paths the tutorial establishes, not `/var/lib/nxdns`, unless the page is about a real install.
|
||||
|
||||
9. **Docs-only.** The only non-docs edits are the orchestrator's: `docs/docs.zig`,
|
||||
`src/docs_drift_test.zig`, `README.md` links, and PLAN.md's repo-layout line.
|
||||
No behavior changes.
|
||||
9. **Docs-only.** The only non-docs edits are the orchestrator's: `docs/docs.zig`, `src/docs_drift_test.zig`, `README.md` links, and PLAN.md's repo-layout line. No behavior changes.
|
||||
|
||||
## Sessions
|
||||
|
||||
@@ -78,226 +32,101 @@ X1, X2, X3, X4, X5 run in parallel — every session owns distinct files.
|
||||
|
||||
## Session X1: index + tutorial + README links
|
||||
|
||||
Owns `docs/README.md`, `docs/tutorial/first-run.md`, `README.md` (links section
|
||||
only). Rulings 1, 5, 3 (the tutorial is executed end to end).
|
||||
Owns `docs/README.md`, `docs/tutorial/first-run.md`, `README.md` (links section only). Rulings 1, 5, 3 (the tutorial is executed end to end).
|
||||
|
||||
## Session X2: how-to, install and operations
|
||||
|
||||
Owns `docs/how-to/{install-with-systemd,install-with-docker,upgrade,troubleshoot}.md`.
|
||||
Rulings 2, 3, 4.
|
||||
Owns `docs/how-to/{install-with-systemd,install-with-docker,upgrade,troubleshoot}.md`. Rulings 2, 3, 4.
|
||||
|
||||
## Session X3: how-to, security and measurement
|
||||
|
||||
Owns `docs/how-to/{enable-doh-and-dot,set-up-admin-authentication,back-up-and-restore,measure-performance}.md`.
|
||||
Rulings 2, 3, 4.
|
||||
Owns `docs/how-to/{enable-doh-and-dot,set-up-admin-authentication,back-up-and-restore,measure-performance}.md`. Rulings 2, 3, 4.
|
||||
|
||||
## Session X4: reference
|
||||
|
||||
Owns `docs/reference/{configuration,api,cli,files-and-directories,performance}.md`.
|
||||
Rulings 2, 4, 7 (heading convention).
|
||||
Owns `docs/reference/{configuration,api,cli,files-and-directories,performance}.md`. Rulings 2, 4, 7 (heading convention).
|
||||
|
||||
## Session X5: explanation
|
||||
|
||||
Owns `docs/explanation/{architecture,configuration-model,performance-and-testing}.md`.
|
||||
Rulings 2, 4.
|
||||
Owns `docs/explanation/{architecture,configuration-model,performance-and-testing}.md`. Rulings 2, 4.
|
||||
|
||||
## File ownership
|
||||
|
||||
X1 docs/README.md + docs/tutorial/* + README.md; X2 and X3 disjoint files under
|
||||
docs/how-to/; X4 docs/reference/*; X5 docs/explanation/*. Orchestrator:
|
||||
docs/docs.zig, src/docs_drift_test.zig, PLAN.md, deletion of the old pages, spec.
|
||||
X1 docs/README.md + docs/tutorial/* + README.md; X2 and X3 disjoint files under docs/how-to/; X4 docs/reference/*; X5 docs/explanation/*. Orchestrator: docs/docs.zig, src/docs_drift_test.zig, PLAN.md, deletion of the old pages, spec.
|
||||
|
||||
## Delivered
|
||||
|
||||
Seventeen pages: `docs/README.md`, one tutorial, eight how-to guides, five
|
||||
reference pages, three explanation pages. The five old documents are deleted.
|
||||
Seventeen pages: `docs/README.md`, one tutorial, eight how-to guides, five reference pages, three explanation pages. The five old documents are deleted.
|
||||
|
||||
The orchestrator repointed `docs/docs.zig` at `reference/{api,configuration,cli}.md`
|
||||
and `src/docs_drift_test.zig` at the same three, moved the subcommand anchor from
|
||||
`### \`name` to `## \`name`, fixed the README Quickstart link that pointed at the
|
||||
deleted `operator.md`, and corrected PLAN.md's repo-layout line and its §13.2
|
||||
OpenAPI paragraph (which still promised a renderer and contract tests that were
|
||||
never built).
|
||||
The orchestrator repointed `docs/docs.zig` at `reference/{api,configuration,cli}.md` and `src/docs_drift_test.zig` at the same three, moved the subcommand anchor from `### \`name` to `## \`name`, fixed the README Quickstart link that pointed at the deleted `operator.md`, and corrected PLAN.md's repo-layout line and its §13.2 OpenAPI paragraph (which still promised a renderer and contract tests that were never built).
|
||||
|
||||
All three guards were proven able to fail: deleting one API table row, one
|
||||
settings key row and one `## \`export` heading each produced a named build
|
||||
failure. The settings guard was strengthened during review — it searched for a
|
||||
bare key, which prose and the annotated example could satisfy after a field's row
|
||||
was deleted; it now anchors on `| \`key\` |`.
|
||||
All three guards were proven able to fail: deleting one API table row, one settings key row and one `## \`export` heading each produced a named build failure. The settings guard was strengthened during review — it searched for a bare key, which prose and the annotated example could satisfy after a field's row was deleted; it now anchors on `| \`key\` |`.
|
||||
|
||||
Ruling 3 held. The tutorial ran end to end twice, the second time after the review
|
||||
fixes. DoH and DoT answered real queries, `POST /api/certs/reload` returned both
|
||||
its success and its failure payload, the export/import round trip was
|
||||
byte-identical, a Docker image was built and its container served DNS, and the
|
||||
offline password change was proven (old password 401, new 200). What could not run
|
||||
here — root-only steps, a second LAN host, a Raspberry Pi, a registry push, the
|
||||
full-scale bench, `dnsperf` — is marked in the page that documents it, with the
|
||||
reason.
|
||||
Ruling 3 held. The tutorial ran end to end twice, the second time after the review fixes. DoH and DoT answered real queries, `POST /api/certs/reload` returned both its success and its failure payload, the export/import round trip was byte-identical, a Docker image was built and its container served DNS, and the offline password change was proven (old password 401, new 200). What could not run here — root-only steps, a second LAN host, a Raspberry Pi, a registry push, the full-scale bench, `dnsperf` — is marked in the page that documents it, with the reason.
|
||||
|
||||
Five review rounds: 15 findings, then 8, then 1, then 1, then clean. One round-1
|
||||
finding was rejected on evidence (`troubleshoot.md` already stated the verified
|
||||
exit codes; the contradiction was in `reference/cli.md`).
|
||||
Five review rounds: 15 findings, then 8, then 1, then 1, then clean. One round-1 finding was rejected on evidence (`troubleshoot.md` already stated the verified exit codes; the contradiction was in `reference/cli.md`).
|
||||
|
||||
## Fix wave (supersedes ruling 9)
|
||||
|
||||
Ruling 9 kept the documentation wave docs-only so that writing pages could not
|
||||
churn behaviour underneath itself. That job is done and committed at 16c9de2.
|
||||
The nine discrepancies the wave found are now fixed inside milestone 13, on the
|
||||
user's direction; milestone 14 stays packaging and publishing.
|
||||
Ruling 9 kept the documentation wave docs-only so that writing pages could not churn behaviour underneath itself. That job is done and committed at 16c9de2. The nine discrepancies the wave found are now fixed inside milestone 13, on the user's direction; milestone 14 stays packaging and publishing.
|
||||
|
||||
Binding rulings for the fix wave:
|
||||
|
||||
F-a. **One definition of "the operator's configuration is wrong."** The
|
||||
divergence in D1 exists because `app.isConfigFault` and `cli.failureExitCode`
|
||||
each carry their own list. Neither list is the fix. Add `src/config/faults.zig`
|
||||
with exactly:
|
||||
F-a. **One definition of "the operator's configuration is wrong."** The divergence in D1 exists because `app.isConfigFault` and `cli.failureExitCode` each carry their own list. Neither list is the fix. Add `src/config/faults.zig` with exactly:
|
||||
|
||||
```zig
|
||||
pub fn isConfigFault(err: anyerror) bool
|
||||
```
|
||||
|
||||
covering the seed and validation errors (`ParseZon`, `ConfigTooLarge`,
|
||||
`MissingDefaultGroup`, `NoUpstreams`, `NoUsableUpstreams`, `BadBindAddress`,
|
||||
`BadRateLimit`, `BadCertificate`, `PasswordAndHashBothSet`, and every
|
||||
`validate.ValidateError`). `app.zig` and `cli.zig` both call it and keep no
|
||||
private list. `run`, `check` and `import` then agree: a rejected configuration
|
||||
file is exit 2 from every subcommand.
|
||||
covering the seed and validation errors (`ParseZon`, `ConfigTooLarge`, `MissingDefaultGroup`, `NoUpstreams`, `NoUsableUpstreams`, `BadBindAddress`, `BadRateLimit`, `BadCertificate`, `PasswordAndHashBothSet`, and every `validate.ValidateError`). `app.zig` and `cli.zig` both call it and keep no private list. `run`, `check` and `import` then agree: a rejected configuration file is exit 2 from every subcommand.
|
||||
|
||||
F-b. **Diagnostics carry severity.** `validate.Diagnostics` gains a severity per
|
||||
item — `.fail` or `.warn`. `writeAll` prints `FAIL `/`WARN ` accordingly.
|
||||
Counting splits: failures set exit 2, warnings never change an exit code. This
|
||||
is the API D2 and D5 both need, so it is pinned here rather than invented twice.
|
||||
F-b. **Diagnostics carry severity.** `validate.Diagnostics` gains a severity per item — `.fail` or `.warn`. `writeAll` prints `FAIL `/`WARN ` accordingly. Counting splits: failures set exit 2, warnings never change an exit code. This is the API D2 and D5 both need, so it is pinned here rather than invented twice.
|
||||
|
||||
F-c. **`check` never writes.** It opens `config.db` read-only and does not
|
||||
migrate. A database behind the current schema is reported, not upgraded. If a
|
||||
read-only open is impossible for a database needing WAL recovery, report that as
|
||||
a failure naming `nxdns run` as the fix — do not silently fall back to a
|
||||
writable open.
|
||||
F-c. **`check` never writes.** It opens `config.db` read-only and does not migrate. A database behind the current schema is reported, not upgraded. If a read-only open is impossible for a database needing WAL recovery, report that as a failure naming `nxdns run` as the fix — do not silently fall back to a writable open.
|
||||
|
||||
F-d. **`check` proves what it claims.** It parses the certificate and the key and
|
||||
verifies they pair, through the same code the server uses, so a green `check`
|
||||
cannot be followed by `run` exiting 2 on `BadCertificate`.
|
||||
F-d. **`check` proves what it claims.** It parses the certificate and the key and verifies they pair, through the same code the server uses, so a green `check` cannot be followed by `run` exiting 2 on `BadCertificate`.
|
||||
|
||||
F-e. **Documentation follows behaviour in the same wave.** Every page that
|
||||
documents the old behaviour is corrected: exit codes in `reference/cli.md`,
|
||||
`how-to/troubleshoot.md`, `how-to/install-with-docker.md`, `tutorial/first-run.md`;
|
||||
the certificate gap in `how-to/troubleshoot.md` and `how-to/enable-doh-and-dot.md`;
|
||||
the "check writes" note in `reference/files-and-directories.md`. The three drift
|
||||
guards stay green. Ruling 3 still binds: a changed command is re-run here.
|
||||
F-e. **Documentation follows behaviour in the same wave.** Every page that documents the old behaviour is corrected: exit codes in `reference/cli.md`, `how-to/troubleshoot.md`, `how-to/install-with-docker.md`, `tutorial/first-run.md`; the certificate gap in `how-to/troubleshoot.md` and `how-to/enable-doh-and-dot.md`; the "check writes" note in `reference/files-and-directories.md`. The three drift guards stay green. Ruling 3 still binds: a changed command is re-run here.
|
||||
|
||||
F-f. **Every fix ships with a test that fails without it.**
|
||||
|
||||
## The nine discrepancies
|
||||
|
||||
1. `nxdns run` exits 1 for seed-file errors (`ParseZon`, `MissingDefaultGroup`,
|
||||
`NoUpstreams`) while `check` and `import` exit 2 for the same file, because
|
||||
`app.isConfigFault` lists only `NoUsableUpstreams`, `BadBindAddress`,
|
||||
`BadRateLimit`, `BadCertificate`.
|
||||
2. `nxdns check` prints `OK: no problems found` and exits 0 after emitting WARN
|
||||
lines.
|
||||
3. `nxdns check` never parses a certificate or tests that the key matches it — it
|
||||
checks path readability and key mode only. A green `check` is followed by
|
||||
`run` exiting 2 with `BadCertificate` on a mismatched pair.
|
||||
4. `check --config <missing file>` exits 1; the implicit path prints `nothing to
|
||||
check` and exits 2.
|
||||
5. A blocklist source attached to no group is silently inert, and `validate.zig`
|
||||
has no diagnostic for it.
|
||||
6. `nxdns check` says it validates without writing, but the `config.db` branch
|
||||
chmods, enables WAL and runs migrations.
|
||||
7. `pruneOrphans` matches only `.list`/`.wild`, so an orphaned `.raw.tmp` is never
|
||||
swept.
|
||||
8. `isEmpty` counts auto-materialised client rows, so a server that has answered
|
||||
one query ignores a seed file placed afterwards.
|
||||
1. `nxdns run` exits 1 for seed-file errors (`ParseZon`, `MissingDefaultGroup`, `NoUpstreams`) while `check` and `import` exit 2 for the same file, because `app.isConfigFault` lists only `NoUsableUpstreams`, `BadBindAddress`, `BadRateLimit`, `BadCertificate`.
|
||||
2. `nxdns check` prints `OK: no problems found` and exits 0 after emitting WARN lines.
|
||||
3. `nxdns check` never parses a certificate or tests that the key matches it — it checks path readability and key mode only. A green `check` is followed by `run` exiting 2 with `BadCertificate` on a mismatched pair.
|
||||
4. `check --config <missing file>` exits 1; the implicit path prints `nothing to check` and exits 2.
|
||||
5. A blocklist source attached to no group is silently inert, and `validate.zig` has no diagnostic for it.
|
||||
6. `nxdns check` says it validates without writing, but the `config.db` branch chmods, enables WAL and runs migrations.
|
||||
7. `pruneOrphans` matches only `.list`/`.wild`, so an orphaned `.raw.tmp` is never swept.
|
||||
8. `isEmpty` counts auto-materialised client rows, so a server that has answered one query ignores a seed file placed afterwards.
|
||||
9. PLAN §18 and the docs say 10 MB / 15 MB; CI asserts 10 MiB / 15 MiB.
|
||||
|
||||
## Fix wave delivered
|
||||
|
||||
All nine are closed. Each shipped with a test its author watched fail with the
|
||||
implementation reverted — ruling F-f was enforced by demanding the observed
|
||||
failure output, not an assertion that a test would fail.
|
||||
All nine are closed. Each shipped with a test its author watched fail with the implementation reverted — ruling F-f was enforced by demanding the observed failure output, not an assertion that a test would fail.
|
||||
|
||||
D1 is one `src/config/faults.zig` deriving the fault set by comptime reflection
|
||||
over `validate.ValidateError`, with no exclusion list; `run`, `check` and
|
||||
`import` all exit 2 on the same rejected file, for `MissingDefaultGroup` and for
|
||||
`ParseZon`. D2 and D5 rest on `validate.Diagnostics` gaining `.fail`/`.warn`:
|
||||
`check` prints `OK: no failures found, 1 warning` and exits 0. D3 proves the
|
||||
certificate pair through `cert_store.CertStore.init`, the same code the
|
||||
listeners use, so a green `check` cannot be followed by `run` exiting 2 on
|
||||
`BadCertificate`. D4 fixes the missing-file exit at the root read. D6 opens
|
||||
`config.db` with `OpenMode.immutable` and no migrate; a stale `-wal` is reported
|
||||
naming `nxdns run`, never read past. D7 sweeps all five suffixes. D8 counts
|
||||
`hand_edited = 1` only. D9 is MiB everywhere CI asserts MiB.
|
||||
D1 is one `src/config/faults.zig` deriving the fault set by comptime reflection over `validate.ValidateError`, with no exclusion list; `run`, `check` and `import` all exit 2 on the same rejected file, for `MissingDefaultGroup` and for `ParseZon`. D2 and D5 rest on `validate.Diagnostics` gaining `.fail`/`.warn`: `check` prints `OK: no failures found, 1 warning` and exits 0. D3 proves the certificate pair through `cert_store.CertStore.init`, the same code the listeners use, so a green `check` cannot be followed by `run` exiting 2 on `BadCertificate`. D4 fixes the missing-file exit at the root read. D6 opens `config.db` with `OpenMode.immutable` and no migrate; a stale `-wal` is reported naming `nxdns run`, never read past. D7 sweeps all five suffixes. D8 counts `hand_edited = 1` only. D9 is MiB everywhere CI asserts MiB.
|
||||
|
||||
Four defects the wave found that were not among the nine:
|
||||
|
||||
1. **`pruneOrphans` had no production caller.** D7's widened matching was
|
||||
unreachable at runtime. `Manager.sweepOrphans` now runs at startup, before
|
||||
each interval pass, and on `DELETE /api/blocklists/:id`.
|
||||
2. **An import destroyed a device's observed timestamps.** `first_seen` and
|
||||
`last_seen` are runtime state, not configuration, and the model carries no
|
||||
field for either — so a configured row was inserted with the import clock in
|
||||
both columns. They now follow the address: a device the database already knew
|
||||
keeps them, only an unseen address takes the import's clock, and a client the
|
||||
file omits is removed with its history. Verified live, not only in tests.
|
||||
3. **A blocklist or upstream url reached the log whole.** A signed url or an
|
||||
`?apikey=` query persisted in journald. `src/safe_url.zig` exports `redact`,
|
||||
which keeps scheme, host and port and drops userinfo, path, query and
|
||||
fragment. It scans rather than parses, deliberately: `error.BadUrl` is one of the
|
||||
failures these very lines report, so the inputs a parser refuses are exactly
|
||||
the ones that must still redact. Twenty-two call sites across `manager.zig`,
|
||||
`validate.zig`, `cli.zig` and `app.zig`.
|
||||
4. **`src/main.zig` built both runner writers in positional mode.** With stderr
|
||||
redirected to a regular file, runner output pwrote over what `std.log` had
|
||||
already written at offset 0, and `2>>` was silently broken because pwrite
|
||||
ignores `O_APPEND`. Both writers now use `writerStreaming`.
|
||||
1. **`pruneOrphans` had no production caller.** D7's widened matching was unreachable at runtime. `Manager.sweepOrphans` now runs at startup, before each interval pass, and on `DELETE /api/blocklists/:id`.
|
||||
2. **An import destroyed a device's observed timestamps.** `first_seen` and `last_seen` are runtime state, not configuration, and the model carries no field for either — so a configured row was inserted with the import clock in both columns. They now follow the address: a device the database already knew keeps them, only an unseen address takes the import's clock, and a client the file omits is removed with its history. Verified live, not only in tests.
|
||||
3. **A blocklist or upstream url reached the log whole.** A signed url or an `?apikey=` query persisted in journald. `src/safe_url.zig` exports `redact`, which keeps scheme, host and port and drops userinfo, path, query and fragment. It scans rather than parses, deliberately: `error.BadUrl` is one of the failures these very lines report, so the inputs a parser refuses are exactly the ones that must still redact. Twenty-two call sites across `manager.zig`, `validate.zig`, `cli.zig` and `app.zig`.
|
||||
4. **`src/main.zig` built both runner writers in positional mode.** With stderr redirected to a regular file, runner output pwrote over what `std.log` had already written at offset 0, and `2>>` was silently broken because pwrite ignores `O_APPEND`. Both writers now use `writerStreaming`.
|
||||
|
||||
Two review findings were answered against the reviewer rather than by it, both
|
||||
with evidence rather than argument:
|
||||
Two review findings were answered against the reviewer rather than by it, both with evidence rather than argument:
|
||||
|
||||
- **A failed stderr write must not stop the server.** The reviewer wanted
|
||||
`seedFromFile` to propagate an output failure as a runtime failure. Implementing
|
||||
that proposal and running the suite showed it replacing
|
||||
`error.MissingDefaultGroup` with `error.WriteFailed` — a broken stderr would
|
||||
hide why the seed file was refused. The discards stay, and the reasoning now
|
||||
sits above them, with the derived half of the claim labelled as derived.
|
||||
- **The log-injection hole was already closed for `std.log`.** `logging.zig:342`
|
||||
escapes control bytes in every log message, so the forged-line scenario the
|
||||
reviewer described could not happen through that path. The live gap was
|
||||
`cli.zig`'s stdout. Escaping stays in `safe_url` as well as the sink, because
|
||||
`validate.Diagnostics` builds `Problem.message` as an allocated string that
|
||||
`web/handlers/mutations.zig` returns as a 400 body — a channel no log sink can
|
||||
escape.
|
||||
- **A failed stderr write must not stop the server.** The reviewer wanted `seedFromFile` to propagate an output failure as a runtime failure. Implementing that proposal and running the suite showed it replacing `error.MissingDefaultGroup` with `error.WriteFailed` — a broken stderr would hide why the seed file was refused. The discards stay, and the reasoning now sits above them, with the derived half of the claim labelled as derived.
|
||||
- **The log-injection hole was already closed for `std.log`.** `logging.zig:342` escapes control bytes in every log message, so the forged-line scenario the reviewer described could not happen through that path. The live gap was `cli.zig`'s stdout. Escaping stays in `safe_url` as well as the sink, because `validate.Diagnostics` builds `Problem.message` as an allocated string that `web/handlers/mutations.zig` returns as a 400 body — a channel no log sink can escape.
|
||||
|
||||
Redaction ended stricter than it started. `redact` prints scheme, host and port
|
||||
only: a NextDNS DoH upstream is `https://dns.nextdns.io/abcd12`, where the path
|
||||
segment is the whole account identifier, so keeping the path kept the credential.
|
||||
The rule has no exemption for `nxdns check`'s stdout, which is the output an
|
||||
operator pastes into a bug report. Log lines identify a source by row id and name
|
||||
instead, and the two duplicate diagnostics now name the other entry
|
||||
(`duplicate of upstreams[0]`) rather than quoting a url that no longer shows why
|
||||
the two collide.
|
||||
Redaction ended stricter than it started. `redact` prints scheme, host and port only: a NextDNS DoH upstream is `https://dns.nextdns.io/abcd12`, where the path segment is the whole account identifier, so keeping the path kept the credential. The rule has no exemption for `nxdns check`'s stdout, which is the output an operator pastes into a bug report. Log lines identify a source by row id and name instead, and the two duplicate diagnostics now name the other entry (`duplicate of upstreams[0]`) rather than quoting a url that no longer shows why the two collide.
|
||||
|
||||
One credential redaction cannot remove, pinned as a test and documented rather
|
||||
than hidden: a NextDNS **DoT** upstream is `tls://abcd12.dns.nextdns.io`, which
|
||||
carries the same identifier in the hostname. Removing it would leave no host and
|
||||
no actionable line. A hostname is resolved publicly and offered as SNI in any
|
||||
case, so it is not private the way a query string is, and dropping every host
|
||||
would cost every operator a diagnostic to cover one vendor's choice. The
|
||||
reference page names the mitigation the operator controls instead: NextDNS also
|
||||
publishes a DoH endpoint whose identifier sits in the path and is redacted whole.
|
||||
One credential redaction cannot remove, pinned as a test and documented rather than hidden: a NextDNS **DoT** upstream is `tls://abcd12.dns.nextdns.io`, which carries the same identifier in the hostname. Removing it would leave no host and no actionable line. A hostname is resolved publicly and offered as SNI in any case, so it is not private the way a query string is, and dropping every host would cost every operator a diagnostic to cover one vendor's choice. The reference page names the mitigation the operator controls instead: NextDNS also publishes a DoH endpoint whose identifier sits in the path and is redacted whole.
|
||||
|
||||
Four rounds of review hardened `redact` and each one found what the last missed.
|
||||
Three of the four were the same root cause, which is worth naming because the
|
||||
first two fixes treated it as bad luck: **the scanner tried to identify a host
|
||||
inside text that is not a url, and guessed.** It guessed the query was safe to cut
|
||||
before the userinfo, leaving `user:pa55` as the host. It guessed a `\` was not a
|
||||
separator. And it guessed which side of an `@` was the host when a late delimiter
|
||||
made both readings available.
|
||||
Four rounds of review hardened `redact` and each one found what the last missed. Three of the four were the same root cause, which is worth naming because the first two fixes treated it as bad luck: **the scanner tried to identify a host inside text that is not a url, and guessed.** It guessed the query was safe to cut before the userinfo, leaving `user:pa55` as the host. It guessed a `\` was not a separator. And it guessed which side of an `@` was the host when a late delimiter made both readings available.
|
||||
|
||||
That third one was recorded here, in an earlier revision of this section, as an
|
||||
accepted cost: an `@` in a backslash path prints "a misleading host, never a
|
||||
credential". **That claim was false and is retracted.** Running the shipped code
|
||||
disproved it:
|
||||
That third one was recorded here, in an earlier revision of this section, as an accepted cost: an `@` in a backslash path prints "a misleading host, never a credential". **That claim was false and is retracted.** Running the shipped code disproved it:
|
||||
|
||||
```
|
||||
https://lists.example?token=prefix@hunter2 -> https://hunter2
|
||||
@@ -305,277 +134,79 @@ https://lists.example#f@hunter2 -> https://hunter2
|
||||
https:\\lists.example\p@hunter2 -> https://hunter2
|
||||
```
|
||||
|
||||
A query string is the most likely place in a url for a token, so the text the
|
||||
scanner promoted to "host" was the secret itself. The note claiming otherwise is
|
||||
why three subsequent rounds passed over it.
|
||||
A query string is the most likely place in a url for a token, so the text the scanner promoted to "host" was the secret itself. The note claiming otherwise is why three subsequent rounds passed over it.
|
||||
|
||||
The fix is a rule rather than a fourth special case: **the scan no longer guesses,
|
||||
it declines.** The `/` cut now runs first and unconditionally, which is safe for
|
||||
the reason the old ordering missed — everything after the `/` is dropped anyway,
|
||||
so an `@` there never needed to be userinfo to stay out of the log. Where two
|
||||
readings genuinely survive, the authority is omitted whole and `format` prints
|
||||
`(ambiguous authority omitted)`, which is prose rather than a placeholder host so
|
||||
an operator reads it as a statement about the line. `SafeUrl.authority` is
|
||||
therefore `?[]const u8`: empty and `null` are different answers, one saying the
|
||||
url names no authority and the other saying it names one that cannot be resolved.
|
||||
The same rule caught a case nobody had raised — `https:a@hunter2`, where RFC 3986
|
||||
reads `hunter2` as a path segment and WHATWG reads it as the host. The
|
||||
disagreement between two parsers is itself the evidence of ambiguity.
|
||||
The fix is a rule rather than a fourth special case: **the scan no longer guesses, it declines.** The `/` cut now runs first and unconditionally, which is safe for the reason the old ordering missed — everything after the `/` is dropped anyway, so an `@` there never needed to be userinfo to stay out of the log. Where two readings genuinely survive, the authority is omitted whole and `format` prints `(ambiguous authority omitted)`, which is prose rather than a placeholder host so an operator reads it as a statement about the line. `SafeUrl.authority` is therefore `?[]const u8`: empty and `null` are different answers, one saying the url names no authority and the other saying it names one that cannot be resolved. The same rule caught a case nobody had raised — `https:a@hunter2`, where RFC 3986 reads `hunter2` as a path segment and WHATWG reads it as the host. The disagreement between two parsers is itself the evidence of ambiguity.
|
||||
|
||||
A redesign was considered and rejected: parse with `std.Uri.parse` first and print
|
||||
nothing but the scheme when the parse fails. It reaches the same "do not guess"
|
||||
place, but it also discards the host for every url malformed in a harmless way,
|
||||
and `redact` exists to be callable from the `error.BadUrl` paths that report
|
||||
exactly those. The ordering fix gets the property without the cost.
|
||||
A redesign was considered and rejected: parse with `std.Uri.parse` first and print nothing but the scheme when the parse fails. It reaches the same "do not guess" place, but it also discards the host for every url malformed in a harmless way, and `redact` exists to be callable from the `error.BadUrl` paths that report exactly those. The ordering fix gets the property without the cost.
|
||||
|
||||
Escaping is owned by the type that introduces the delimiter: `quoteText` writes
|
||||
its own quotes and escapes `'` inside them, so a caller cannot reopen the hole by
|
||||
adding quotes of its own. Escaping inside a plain helper would have left the
|
||||
defect one caller away, which on a third review round is not a fix.
|
||||
Escaping is owned by the type that introduces the delimiter: `quoteText` writes its own quotes and escapes `'` inside them, so a caller cannot reopen the hole by adding quotes of its own. Escaping inside a plain helper would have left the defect one caller away, which on a third review round is not a fix.
|
||||
|
||||
`SafeUrl` was then found to break that same rule from the other side. Its `format`
|
||||
hardcoded the `none` delimiter, so it never escaped `'` — while eight call sites
|
||||
wrapped it in `'{f}'` of their own. `https://ho'st/x` redacts to `https://ho'st`,
|
||||
which inside a caller's quotes reads as `'ho'` followed by loose text. Two
|
||||
independent findings converged on it: this review, and the `/metrics` work, which
|
||||
hit the same shape with `"` instead of `'`.
|
||||
`SafeUrl` was then found to break that same rule from the other side. Its `format` hardcoded the `none` delimiter, so it never escaped `'` — while eight call sites wrapped it in `'{f}'` of their own. `https://ho'st/x` redacts to `https://ho'st`, which inside a caller's quotes reads as `'ho'` followed by loose text. Two independent findings converged on it: this review, and the `/metrics` work, which hit the same shape with `"` instead of `'`.
|
||||
|
||||
`redactQuoted` closes it, and the choice between the two is a stated rule rather
|
||||
than per-call-site judgement: **use the quoted form whenever anything follows the
|
||||
url on the line**, because a redacted authority can still hold a space, a `:` and a
|
||||
`'`, and unquoted it can impersonate whatever comes next — `upstream {f} failed:
|
||||
{t}` with an authority of `ok failed: Timeout` reports a failure that did not
|
||||
happen. Bare `redact` is for the two cases where that cannot arise: the url ends
|
||||
the line (`cli.zig`'s `OK upstreams[N]`, `context.zig`'s missing-id line), or the
|
||||
caller owns the escaping for a delimiter of its own (`metrics.zig`). Ten sites
|
||||
moved to the quoted form. The output is byte-identical for any url without a `'`,
|
||||
so no documented output changed.
|
||||
`redactQuoted` closes it, and the choice between the two is a stated rule rather than per-call-site judgement: **use the quoted form whenever anything follows the url on the line**, because a redacted authority can still hold a space, a `:` and a `'`, and unquoted it can impersonate whatever comes next — `upstream {f} failed: {t}` with an authority of `ok failed: Timeout` reports a failure that did not happen. Bare `redact` is for the two cases where that cannot arise: the url ends the line (`cli.zig`'s `OK upstreams[N]`, `context.zig`'s missing-id line), or the caller owns the escaping for a delimiter of its own (`metrics.zig`). Ten sites moved to the quoted form. The output is byte-identical for any url without a `'`, so no documented output changed.
|
||||
|
||||
One property is asserted rather than assumed, because it is what makes a `\` in the
|
||||
output always this file's and never the operator's: a `\` ends an authority, so
|
||||
unlike a source name it can never reach the value to be doubled.
|
||||
One property is asserted rather than assumed, because it is what makes a `\` in the output always this file's and never the operator's: a `\` ends an authority, so unlike a source name it can never reach the value to be doubled.
|
||||
|
||||
Round five found two more, both in the same scanner, which is now five rounds and
|
||||
five findings. Both were confirmed by running the code before being fixed.
|
||||
Round five found two more, both in the same scanner, which is now five rounds and five findings. Both were confirmed by running the code before being fixed.
|
||||
|
||||
The first is the ambiguity rule applied to only half its cases. `https:a@hunter2`
|
||||
was withheld, but `https:hunter2` — the same opaque path with no `@` in it —
|
||||
printed whole, because the check sat *after* the `@` lookup and a url with no `@`
|
||||
returned before reaching it. The reading is ambiguous either way; the `@` was never
|
||||
what made it so. The check now runs before the `@` lookup.
|
||||
The first is the ambiguity rule applied to only half its cases. `https:a@hunter2` was withheld, but `https:hunter2` — the same opaque path with no `@` in it — printed whole, because the check sat *after* the `@` lookup and a url with no `@` returned before reaching it. The reading is ambiguous either way; the `@` was never what made it so. The check now runs before the `@` lookup.
|
||||
|
||||
Moving it exposed the reason it had been placed there: `localhost` satisfies the
|
||||
scheme production, so a rule keyed on the production alone withholds
|
||||
`localhost:8080/x`, which is the shape an operator on a LAN is most likely to
|
||||
write. A port is not an opaque path, so the digits after the colon are what
|
||||
separate the two. This is checked against the trimmed authority, not the raw one,
|
||||
or `localhost:8080?x` would fail the digit test on the query.
|
||||
Moving it exposed the reason it had been placed there: `localhost` satisfies the scheme production, so a rule keyed on the production alone withholds `localhost:8080/x`, which is the shape an operator on a LAN is most likely to write. A port is not an opaque path, so the digits after the colon are what separate the two. This is checked against the trimmed authority, not the raw one, or `localhost:8080?x` would fail the digit test on the query.
|
||||
|
||||
The second: a network-path reference (`//lists.example/hosts.txt`) redacted to the
|
||||
empty string, because the unconditional `/` cut lands at byte zero. Not a leak —
|
||||
nothing was printed — but the line named no source at all, and the authority in it
|
||||
is not in doubt. A leading run of `/` is now consumed the way a scheme delimiter's
|
||||
is, so the userinfo in `//user:pa55@lists.example/x` is dropped rather than the
|
||||
whole authority withheld. An ambiguous one such as `//a?b@c` is still withheld: the
|
||||
branch settles where the authority starts, not that every reading of it is
|
||||
resolved.
|
||||
The second: a network-path reference (`//lists.example/hosts.txt`) redacted to the empty string, because the unconditional `/` cut lands at byte zero. Not a leak — nothing was printed — but the line named no source at all, and the authority in it is not in doubt. A leading run of `/` is now consumed the way a scheme delimiter's is, so the userinfo in `//user:pa55@lists.example/x` is dropped rather than the whole authority withheld. An ambiguous one such as `//a?b@c` is still withheld: the branch settles where the authority starts, not that every reading of it is resolved.
|
||||
|
||||
Round six found three more in the same function, all the same class again: a
|
||||
separator run that was read as an authority delimiter when it does not settle one.
|
||||
Round six found three more in the same function, all the same class again: a separator run that was read as an authority delimiter when it does not settle one.
|
||||
|
||||
`https:/hunter2` and `https:///hunter2` printed the path segment as the host. The
|
||||
scan accepted a delimiter of *any* number of separators, and the comment recording
|
||||
why was accurate when it was written and stale by the time it was read: the
|
||||
tolerance existed so `https:/user:pass@host/list` would find an authority instead
|
||||
of printing its userinfo. That reason expired when the `/` cut moved ahead of the
|
||||
userinfo lookup in round four — the authority of a url with no `://` now ends at
|
||||
its first `/`, so it holds no userinfo to print. Only a run of exactly two
|
||||
introduces an authority; one leaves an absolute path and three or more is an empty
|
||||
authority to RFC 3986 and a host to WHATWG. The four inputs that motivated the old
|
||||
tolerance are still safe, now by being withheld rather than resolved, and that is
|
||||
asserted where the old behaviour used to be.
|
||||
`https:/hunter2` and `https:///hunter2` printed the path segment as the host. The scan accepted a delimiter of *any* number of separators, and the comment recording why was accurate when it was written and stale by the time it was read: the tolerance existed so `https:/user:pass@host/list` would find an authority instead of printing its userinfo. That reason expired when the `/` cut moved ahead of the userinfo lookup in round four — the authority of a url with no `://` now ends at its first `/`, so it holds no userinfo to print. Only a run of exactly two introduces an authority; one leaves an absolute path and three or more is an empty authority to RFC 3986 and a host to WHATWG. The four inputs that motivated the old tolerance are still safe, now by being withheld rather than resolved, and that is asserted where the old behaviour used to be.
|
||||
|
||||
`///lists.example/x` had the same defect in the network-path branch added one
|
||||
round earlier, which consumed the whole run. Exactly two there too.
|
||||
`///lists.example/x` had the same defect in the network-path branch added one round earlier, which consumed the whole run. Exactly two there too.
|
||||
|
||||
The third retracts something this section claimed one round ago. `isPort` was
|
||||
introduced so a schemeless `localhost:8080` would keep resolving, on the reasoning
|
||||
that a port is not an opaque path. It is not that simple: `https:123456` is an
|
||||
opaque path whose digits are as much a token as any other text, and the exception
|
||||
printed it whole. The reviewer proposed excluding known schemes from the
|
||||
exception; the exception is gone instead, because a list of known schemes is the
|
||||
kind of thing that goes stale silently and this file already has one rule that
|
||||
covers it. `localhost:8080/x` is now withheld, which costs nothing an operator
|
||||
needs: a url reaching this without a scheme is one the validator is rejecting, and
|
||||
the field path beside it names which. An `IP:port` is unaffected — a leading digit
|
||||
fails the scheme production, so `10.0.0.2:8080` and `[::1]:853` still resolve.
|
||||
The third retracts something this section claimed one round ago. `isPort` was introduced so a schemeless `localhost:8080` would keep resolving, on the reasoning that a port is not an opaque path. It is not that simple: `https:123456` is an opaque path whose digits are as much a token as any other text, and the exception printed it whole. The reviewer proposed excluding known schemes from the exception; the exception is gone instead, because a list of known schemes is the kind of thing that goes stale silently and this file already has one rule that covers it. `localhost:8080/x` is now withheld, which costs nothing an operator needs: a url reaching this without a scheme is one the validator is rejecting, and the field path beside it names which. An `IP:port` is unaffected — a leading digit fails the scheme production, so `10.0.0.2:8080` and `[::1]:853` still resolve.
|
||||
|
||||
Round seven took the same rule one step further. "Exactly two separators" still
|
||||
accepted `\\`, `/\` and `\/`, and WHATWG converts a `\` to a `/` only for a
|
||||
*special* scheme — so `https:\\hunter2` has the same split reading as the cases
|
||||
above, and `tls:\\hunter2` has no reading at all under which `hunter2` is a host,
|
||||
`tls` not being special. RFC 3986 gives `\` no meaning anywhere. Only `//` opens
|
||||
an authority now. A `\` still ends one, and still anchors the scheme scan so a
|
||||
backslash-pasted url is reported by its scheme, but it opens nothing.
|
||||
Round seven took the same rule one step further. "Exactly two separators" still accepted `\\`, `/\` and `\/`, and WHATWG converts a `\` to a `/` only for a *special* scheme — so `https:\\hunter2` has the same split reading as the cases above, and `tls:\\hunter2` has no reading at all under which `hunter2` is a host, `tls` not being special. RFC 3986 gives `\` no meaning anywhere. Only `//` opens an authority now. A `\` still ends one, and still anchors the scheme scan so a backslash-pasted url is reported by its scheme, but it opens nothing.
|
||||
|
||||
Round seven also caught a stale comment, which is the second time in two rounds
|
||||
that a comment outlived the reason it recorded. Both said something true when
|
||||
written and false when read, and both were load-bearing — the first is why the
|
||||
any-length tolerance survived three rounds after its justification expired.
|
||||
Round seven also caught a stale comment, which is the second time in two rounds that a comment outlived the reason it recorded. Both said something true when written and false when read, and both were load-bearing — the first is why the any-length tolerance survived three rounds after its justification expired.
|
||||
|
||||
Seven rounds, nine findings, one function. Every one was a place where the scan
|
||||
committed to a reading the text did not support. What finally holds is not a
|
||||
sharper scan but a smaller claim: the authority is printed only where exactly one
|
||||
reading survives, and withheld everywhere else. Concretely, an authority is read
|
||||
only after a literal `//`, whether a scheme introduced it or not.
|
||||
Seven rounds, nine findings, one function. Every one was a place where the scan committed to a reading the text did not support. What finally holds is not a sharper scan but a smaller claim: the authority is printed only where exactly one reading survives, and withheld everywhere else. Concretely, an authority is read only after a literal `//`, whether a scheme introduced it or not.
|
||||
|
||||
The cost is paid by malformed input alone, and it is paid in diagnostic detail
|
||||
rather than in safety: a backslash-pasted or schemeless url now reports its scheme
|
||||
and no host. Every such url is one the validator is already rejecting, and the
|
||||
field path or row id beside it names which one. A sweep of eighteen shapes
|
||||
carrying `hunter2`, `abcd12`, `s3cr3t`, `token`, `pa55` and `123456` finds none of
|
||||
them in any output.
|
||||
The cost is paid by malformed input alone, and it is paid in diagnostic detail rather than in safety: a backslash-pasted or schemeless url now reports its scheme and no host. Every such url is one the validator is already rejecting, and the field path or row id beside it names which one. A sweep of eighteen shapes carrying `hunter2`, `abcd12`, `s3cr3t`, `token`, `pa55` and `123456` finds none of them in any output.
|
||||
|
||||
Round eight returned no critical and no important finding, and two minor ones that
|
||||
cut in opposite directions. A run of three or more slashes was reported as an
|
||||
authority the url does not have, when it is contested rather than absent — RFC
|
||||
3986 reads an empty authority, WHATWG resolving against a special-scheme base
|
||||
reads a host. `SafeUrl` exists to keep those two answers apart, so it is now
|
||||
withheld rather than reported as empty.
|
||||
Round eight returned no critical and no important finding, and two minor ones that cut in opposite directions. A run of three or more slashes was reported as an authority the url does not have, when it is contested rather than absent — RFC 3986 reads an empty authority, WHATWG resolving against a special-scheme base reads a host. `SafeUrl` exists to keep those two answers apart, so it is now withheld rather than reported as empty.
|
||||
|
||||
The other corrected a claim this file made about the very thing it was withholding.
|
||||
`localhost:8080` is **not** contested: `localhost` is not one of WHATWG's six
|
||||
special schemes, so both standards read a scheme and an opaque path, and the
|
||||
host-and-port an operator meant is a reading no parser offers. It is still
|
||||
withheld — the text after the colon is a path segment under every reading — but the
|
||||
marker calls it an authority that could not be resolved when the honest answer is
|
||||
that there is none.
|
||||
The other corrected a claim this file made about the very thing it was withholding. `localhost:8080` is **not** contested: `localhost` is not one of WHATWG's six special schemes, so both standards read a scheme and an opaque path, and the host-and-port an operator meant is a reading no parser offers. It is still withheld — the text after the colon is a path segment under every reading — but the marker calls it an authority that could not be resolved when the honest answer is that there is none.
|
||||
|
||||
That wording defect was first recorded rather than fixed, on the grounds that
|
||||
telling the two apart needs a scheme list and a scheme list is what the previous
|
||||
round had just removed. Round nine rejected that and was right to: the list round
|
||||
six removed described *what nxdns supports*, so it went stale whenever a transport
|
||||
was added. WHATWG's special schemes are a closed set fixed by the URL Standard —
|
||||
`ftp`, `file`, `http`, `https`, `ws`, `wss` — which never described nxdns and
|
||||
cannot go stale with it. Conflating the two was the error, and knowingly shipping
|
||||
a diagnostic that contradicts its own type's contract is the tech debt this
|
||||
project does not take on.
|
||||
That wording defect was first recorded rather than fixed, on the grounds that telling the two apart needs a scheme list and a scheme list is what the previous round had just removed. Round nine rejected that and was right to: the list round six removed described *what nxdns supports*, so it went stale whenever a transport was added. WHATWG's special schemes are a closed set fixed by the URL Standard — `ftp`, `file`, `http`, `https`, `ws`, `wss` — which never described nxdns and cannot go stale with it. Conflating the two was the error, and knowingly shipping a diagnostic that contradicts its own type's contract is the tech debt this project does not take on.
|
||||
|
||||
So `redact` now distinguishes them. Only a special scheme can disagree with RFC
|
||||
3986 about text no `//` introduced, so `https:hunter2` is withheld as contested
|
||||
and `localhost:8080`, `mailto:ops@example.com` and `tls:\\host` are withheld as
|
||||
naming no authority at all. Round nine's other finding was the same rule missing
|
||||
from the schemeless branch: `\\lists.example\path` and `/\lists.example/path` were
|
||||
reported as absent when they are contested. A run is settled when it is one
|
||||
separator, or when the reading that looks for a host finds none —
|
||||
`\\?\C:\lists\hosts.txt` ends its authority at the `?` under both — and contested
|
||||
otherwise.
|
||||
So `redact` now distinguishes them. Only a special scheme can disagree with RFC 3986 about text no `//` introduced, so `https:hunter2` is withheld as contested and `localhost:8080`, `mailto:ops@example.com` and `tls:\\host` are withheld as naming no authority at all. Round nine's other finding was the same rule missing from the schemeless branch: `\\lists.example\path` and `/\lists.example/path` were reported as absent when they are contested. A run is settled when it is one separator, or when the reading that looks for a host finds none — `\\?\C:\lists\hosts.txt` ends its authority at the `?` under both — and contested otherwise.
|
||||
|
||||
Round ten returned nothing critical and nothing important, four minor findings and
|
||||
a nit. Two were taken: the nit, which was a comment claiming
|
||||
`https:\\dns.nextdns.io\abcd12` "names a host" when the file's own reasoning is
|
||||
that it is contested; and the rendering question — `tls:\\host` prints `tls://`
|
||||
although the input held no `//`. That one is now contractual rather than
|
||||
accidental. `format` renders `scheme://authority` canonically and does not quote
|
||||
the input's syntax; nothing about which bytes separated the scheme survives
|
||||
redaction, and nothing should, because the input is not meant to be reconstructible
|
||||
from the output.
|
||||
Round ten returned nothing critical and nothing important, four minor findings and a nit. Two were taken: the nit, which was a comment claiming `https:\\dns.nextdns.io\abcd12` "names a host" when the file's own reasoning is that it is contested; and the rendering question — `tls:\\host` prints `tls://` although the input held no `//`. That one is now contractual rather than accidental. `format` renders `scheme://authority` canonically and does not quote the input's syntax; nothing about which bytes separated the scheme survives redaction, and nothing should, because the input is not meant to be reconstructible from the output.
|
||||
|
||||
**Three findings were declined, and the reason is checked rather than asserted.**
|
||||
`https:/user@/x`, `\path@hunter2` and `file:secret` are classified as contested
|
||||
when both readings in fact find no host. Running them shows why that is tolerable:
|
||||
each prints `(ambiguous authority omitted)`, so each says *less* than it could and
|
||||
none prints the path segment. The mechanisms are named in the file — emptiness
|
||||
tested before userinfo removal, a leading run of one reaching the late-delimiter
|
||||
rule, and `file` having its own WHATWG parsing states that `isSpecialScheme` does
|
||||
not model. Fixing them means modelling more of two standards for inputs no
|
||||
accepted configuration can hold: every url this program takes carries `http`,
|
||||
`https`, `tls`, `udp` or `tcp` and a `//`. All three are pinned by a test, because
|
||||
the failure that would matter is the opposite one — if any of them ever starts
|
||||
naming a host, that test fails.
|
||||
**Three findings were declined, and the reason is checked rather than asserted.** `https:/user@/x`, `\path@hunter2` and `file:secret` are classified as contested when both readings in fact find no host. Running them shows why that is tolerable: each prints `(ambiguous authority omitted)`, so each says *less* than it could and none prints the path segment. The mechanisms are named in the file — emptiness tested before userinfo removal, a leading run of one reaching the late-delimiter rule, and `file` having its own WHATWG parsing states that `isSpecialScheme` does not model. Fixing them means modelling more of two standards for inputs no accepted configuration can hold: every url this program takes carries `http`, `https`, `tls`, `udp` or `tcp` and a `//`. All three are pinned by a test, because the failure that would matter is the opposite one — if any of them ever starts naming a host, that test fails.
|
||||
|
||||
That is where the review loop was stopped, on a judgement rather than on an empty
|
||||
round. Rounds eight, nine and ten each returned nothing critical and nothing
|
||||
important, and the findings had moved from "this prints a secret" to "this claims
|
||||
more than it knows about a url no operator can configure". Round ten opened a new
|
||||
sub-class of its own — WHATWG's `file:` state machine — which is the signal that
|
||||
the remaining work is unbounded and no longer about safety.
|
||||
That is where the review loop was stopped, on a judgement rather than on an empty round. Rounds eight, nine and ten each returned nothing critical and nothing important, and the findings had moved from "this prints a secret" to "this claims more than it knows about a url no operator can configure". Round ten opened a new sub-class of its own — WHATWG's `file:` state machine — which is the signal that the remaining work is unbounded and no longer about safety.
|
||||
|
||||
Ten rounds, sixteen findings, one function. A sweep of twenty-five shapes carrying
|
||||
`hunter2`, `abcd12`, `s3cr3t`, `token`, `pa55`, `123456`, `hosts.txt` and
|
||||
`example.com` finds none of them in any output.
|
||||
Ten rounds, sixteen findings, one function. A sweep of twenty-five shapes carrying `hunter2`, `abcd12`, `s3cr3t`, `token`, `pa55`, `123456`, `hosts.txt` and `example.com` finds none of them in any output.
|
||||
|
||||
The last gap was found by sweeping every log site in the tree rather than
|
||||
trusting the review's file list: `src/upstream/dot_client.zig` printed the
|
||||
upstream url whole at four sites and `pool.zig` at a fifth. Three review rounds
|
||||
and four agents had redacted urls across `manager.zig`, `validate.zig`, `cli.zig`
|
||||
and `app.zig` without anyone asking which other modules logged the same values.
|
||||
The last gap was found by sweeping every log site in the tree rather than trusting the review's file list: `src/upstream/dot_client.zig` printed the upstream url whole at four sites and `pool.zig` at a fifth. Three review rounds and four agents had redacted urls across `manager.zig`, `validate.zig`, `cli.zig` and `app.zig` without anyone asking which other modules logged the same values.
|
||||
|
||||
Deliberately not changed: `POST /api/blocklists` still ignores the
|
||||
`SourceInNoGroup` warning. The web flow creates a source before any group link
|
||||
can exist, so the warning is structural at that moment and a 400 would be wrong.
|
||||
The three CLI paths carry it instead.
|
||||
Deliberately not changed: `POST /api/blocklists` still ignores the `SourceInNoGroup` warning. The web flow creates a source before any group link can exist, so the warning is structural at that moment and a 400 would be wrong. The three CLI paths carry it instead.
|
||||
|
||||
Claims in the wave that no test backs, recorded rather than buried: the two
|
||||
`app.zig` `log.warn` redactions, because a `std.log` line is not observable from
|
||||
a unit test under the default runner; `cli.zig`'s `not a usable DoH url`
|
||||
redaction, argued unreachable by any credential-carrying url because
|
||||
`Endpoint.parse` rejects `@?#` in the authority first; the two probe lines that
|
||||
need a reachable upstream; and the WAL guard's residual window, which is argued
|
||||
rather than observed.
|
||||
Claims in the wave that no test backs, recorded rather than buried: the two `app.zig` `log.warn` redactions, because a `std.log` line is not observable from a unit test under the default runner; `cli.zig`'s `not a usable DoH url` redaction, argued unreachable by any credential-carrying url because `Endpoint.parse` rejects `@?#` in the authority first; the two probe lines that need a reachable upstream; and the WAL guard's residual window, which is argued rather than observed.
|
||||
|
||||
## Round four
|
||||
|
||||
The `/metrics` family was the last place a credential still reached an
|
||||
unauthenticated reader. `GET /metrics` is `.auth = .open` in `src/web/routes.zig`
|
||||
and `web.bind` defaults to `0.0.0.0`, so an unauthenticated `curl` printed
|
||||
`nxdns_upstream_up{url="https://dns.nextdns.io/abcd12"} 1` with HTTP 200. Every
|
||||
other redaction in this wave was on a path that already required a session or a
|
||||
shell on the host; this one was not.
|
||||
The `/metrics` family was the last place a credential still reached an unauthenticated reader. `GET /metrics` is `.auth = .open` in `src/web/routes.zig` and `web.bind` defaults to `0.0.0.0`, so an unauthenticated `curl` printed `nxdns_upstream_up{url="https://dns.nextdns.io/abcd12"} 1` with HTTP 200. Every other redaction in this wave was on a path that already required a session or a shell on the host; this one was not.
|
||||
|
||||
Redacting it needed a second escaping layer rather than a call to `redact`.
|
||||
`redact` cuts the authority at `/@?#\` and escapes every control byte, so it can
|
||||
emit neither a raw newline nor a lone backslash — but `"` is in none of those cut
|
||||
sets, so `https://ho"st/x` arrives at the label as `https://ho"st` and closes the
|
||||
label value, letting the rest of the string write label pairs of its own.
|
||||
`writeLabelValue` therefore runs *over* the redacted text. The ordering also
|
||||
matters in the other direction: doubling the backslash turns the two characters
|
||||
`redact` writes for a control byte into an unambiguous `\\n`, so a parser reads a
|
||||
backslash rather than a newline.
|
||||
Redacting it needed a second escaping layer rather than a call to `redact`. `redact` cuts the authority at `/@?#\` and escapes every control byte, so it can emit neither a raw newline nor a lone backslash — but `"` is in none of those cut sets, so `https://ho"st/x` arrives at the label as `https://ho"st` and closes the label value, letting the rest of the string write label pairs of its own. `writeLabelValue` therefore runs *over* the redacted text. The ordering also matters in the other direction: doubling the backslash turns the two characters `redact` writes for a control byte into an unambiguous `\\n`, so a parser reads a backslash rather than a newline.
|
||||
|
||||
The redaction then introduced a defect of its own, caught before it shipped: two
|
||||
upstreams on one host redact to one label set, so `nxdns_upstream_up` rendered
|
||||
twice with identical labels in a single exposition. Two NextDNS profiles is a
|
||||
realistic configuration. The fix labels each sample with its array position in
|
||||
`renderUpstreams`, so uniqueness holds by construction and a hand-built `Sample`
|
||||
cannot forge a collision. The index is positional and therefore stable only while
|
||||
the pool order is — `pool.Snapshot` carries no row id, and threading one through
|
||||
the pool and the repository was judged out of scope for a review round. The
|
||||
caveat is in the file. It only bites when two upstreams share an origin; for
|
||||
distinct origins the url carries the identity and a reorder is harmless.
|
||||
The redaction then introduced a defect of its own, caught before it shipped: two upstreams on one host redact to one label set, so `nxdns_upstream_up` rendered twice with identical labels in a single exposition. Two NextDNS profiles is a realistic configuration. The fix labels each sample with its array position in `renderUpstreams`, so uniqueness holds by construction and a hand-built `Sample` cannot forge a collision. The index is positional and therefore stable only while the pool order is — `pool.Snapshot` carries no row id, and threading one through the pool and the repository was judged out of scope for a review round. The caveat is in the file. It only bites when two upstreams share an origin; for distinct origins the url carries the identity and a reorder is harmless.
|
||||
|
||||
Claim scoping on that defect, since the three statements are not one: the
|
||||
duplicate exposition text **was** reproduced, out of the same `render` function
|
||||
`/metrics` calls. It was **not** reproduced over HTTP from a pre-fix server — every
|
||||
live run had the index in place. And "a scrape carrying it is rejected" is
|
||||
**reasoned, not observed**; nothing was fed to a Prometheus. Whether a scraper
|
||||
rejects the pair or keeps the last sample, the down upstream is lost, which is the
|
||||
part the fix rests on.
|
||||
Claim scoping on that defect, since the three statements are not one: the duplicate exposition text **was** reproduced, out of the same `render` function `/metrics` calls. It was **not** reproduced over HTTP from a pre-fix server — every live run had the index in place. And "a scrape carrying it is rejected" is **reasoned, not observed**; nothing was fed to a Prometheus. Whether a scraper rejects the pair or keeps the last sample, the down upstream is lost, which is the part the fix rests on.
|
||||
|
||||
`src/storage/repositories/context.zig` printed a blocklist url whole and a group
|
||||
name raw on its two `NotFound` paths. Both are invariant-failure paths that fire
|
||||
rarely, which is why they outlived three rounds of sweeping.
|
||||
`src/storage/repositories/context.zig` printed a blocklist url whole and a group name raw on its two `NotFound` paths. Both are invariant-failure paths that fire rarely, which is why they outlived three rounds of sweeping.
|
||||
|
||||
One test file was never running. `src/config/faults.zig` was absent from
|
||||
`src/tests.zig`, and Zig collects tests only from the root module's explicit
|
||||
import list — a transitively imported file contributes none, even when its values
|
||||
are used, which was confirmed against a scratch project rather than assumed. The
|
||||
five reflection guards in that file are what catch a future `ValidateError`
|
||||
variant being added and left unclassified; they had never executed. Its runtime
|
||||
behaviour was covered — `cli.zig` has a test that classifies through it — so the
|
||||
gap was in the regression guard, not in what shipped. A sweep of the tree found no
|
||||
other unlisted file with tests; `dns/dns.zig` is a re-export barrel with none, and
|
||||
its own comment documents this rule.
|
||||
One test file was never running. `src/config/faults.zig` was absent from `src/tests.zig`, and Zig collects tests only from the root module's explicit import list — a transitively imported file contributes none, even when its values are used, which was confirmed against a scratch project rather than assumed. The five reflection guards in that file are what catch a future `ValidateError` variant being added and left unclassified; they had never executed. Its runtime behaviour was covered — `cli.zig` has a test that classifies through it — so the gap was in the regression guard, not in what shipped. A sweep of the tree found no other unlisted file with tests; `dns/dns.zig` is a re-export barrel with none, and its own comment documents this rule.
|
||||
|
||||
## Acceptance (milestone complete)
|
||||
|
||||
@@ -592,9 +223,7 @@ its own comment documents this rule.
|
||||
|
||||
## Anti-requirements
|
||||
|
||||
- No new documentation subjects (no metrics-families reference, no security-model
|
||||
page) — this milestone moves and splits existing content plus the tutorial.
|
||||
- No new documentation subjects (no metrics-families reference, no security-model page) — this milestone moves and splits existing content plus the tutorial.
|
||||
- No redirect stubs or `docs/legacy/`.
|
||||
- No source fixes; report discrepancies instead. (Held for the documentation
|
||||
wave. Superseded by the fix wave above, which closes the nine it reported.)
|
||||
- No source fixes; report discrepancies instead. (Held for the documentation wave. Superseded by the fix wave above, which closes the nine it reported.)
|
||||
- No doc generator, no site builder, no MkDocs.
|
||||
|
||||
+125
-486
@@ -1,9 +1,6 @@
|
||||
# Milestone 14: build, package and publish releases
|
||||
|
||||
Goal: turn a signed git tag into a published, verifiable release — two static
|
||||
musl tarballs and one multi-architecture container image on the self-hosted
|
||||
Gitea at `git.mial.net`, with a licence, third-party notices, a changelog, a
|
||||
detached signature, and documentation that leads with download-and-verify.
|
||||
Goal: turn a signed git tag into a published, verifiable release — two static musl tarballs and one multi-architecture container image on the self-hosted Gitea at `git.mial.net`, with a licence, third-party notices, a changelog, a detached signature, and documentation that leads with download-and-verify.
|
||||
|
||||
First tag: `v0.0.1`.
|
||||
|
||||
@@ -11,89 +8,49 @@ First tag: `v0.0.1`.
|
||||
|
||||
### 1. PLAN.md is amended first
|
||||
|
||||
`PLAN.md:41` lists "Prebuilt binaries / published Docker images / project
|
||||
website" under §2.2 *Out of Scope (permanent scope decisions, not deferrals)*.
|
||||
`AGENTS.md` makes PLAN the source of truth, so this milestone is invalid until
|
||||
that line is amended. The orchestrator edits PLAN before any session starts:
|
||||
`PLAN.md:41` lists "Prebuilt binaries / published Docker images / project website" under §2.2 *Out of Scope (permanent scope decisions, not deferrals)*. `AGENTS.md` makes PLAN the source of truth, so this milestone is invalid until that line is amended. The orchestrator edits PLAN before any session starts:
|
||||
|
||||
- §2.2: remove prebuilt binaries and published container images. **The project
|
||||
website stays out of scope.**
|
||||
- Add a §on publication: tag-triggered releases, the artifact set, the signing
|
||||
model, and the two deferrals in ruling 12.
|
||||
- `PLAN.md:571`: drop "build date" from the `nxdns version` line. The version
|
||||
string and the git commit identify a build exactly, and a date is one more
|
||||
input that a reproducible build would have to pin. `src/version.zig` is
|
||||
already correct; PLAN was wrong.
|
||||
- `PLAN.md:638`: state exact byte limits — 15,728,640 with embedded assets,
|
||||
10,485,760 without — resolving milestone-13 discrepancy 9 in favour of what
|
||||
CI already asserts.
|
||||
- §2.2: remove prebuilt binaries and published container images. **The project website stays out of scope.**
|
||||
- Add a §on publication: tag-triggered releases, the artifact set, the signing model, and the two deferrals in ruling 12.
|
||||
- `PLAN.md:571`: drop "build date" from the `nxdns version` line. The version string and the git commit identify a build exactly, and a date is one more input that a reproducible build would have to pin. `src/version.zig` is already correct; PLAN was wrong.
|
||||
- `PLAN.md:638`: state exact byte limits — 15,728,640 with embedded assets, 10,485,760 without — resolving milestone-13 discrepancy 9 in favour of what CI already asserts.
|
||||
|
||||
### 2. The version lives in the tag, and in exactly one other place
|
||||
|
||||
The tag is authoritative. `v0.0.1` means `-Dversion-string=0.0.1`;
|
||||
`-Dgit-commit` is the tag's peeled commit.
|
||||
The tag is authoritative. `v0.0.1` means `-Dversion-string=0.0.1`; `-Dgit-commit` is the tag's peeled commit.
|
||||
|
||||
`build.zig.zon:3` holds `.version`, which Zig requires and nothing reads. It is
|
||||
bumped in the commit before each tag, and `verify-dist` fails when it disagrees
|
||||
with the version under build. Two strings, one assert, no third copy anywhere.
|
||||
`build.zig.zon:3` holds `.version`, which Zig requires and nothing reads. It is bumped in the commit before each tag, and `verify-dist` fails when it disagrees with the version under build. Two strings, one assert, no third copy anywhere.
|
||||
|
||||
Tags are **never moved**. A tag that produced a bad release is abandoned; the
|
||||
fix ships as the next patch version. Only `vMAJOR.MINOR.PATCH` is accepted —
|
||||
the release workflow rejects any tag carrying a pre-release suffix.
|
||||
Tags are **never moved**. A tag that produced a bad release is abandoned; the fix ships as the next patch version. Only `vMAJOR.MINOR.PATCH` is accepted — the release workflow rejects any tag carrying a pre-release suffix.
|
||||
|
||||
### 3. Licence and third-party notices
|
||||
|
||||
`LICENSE` holds the English text of EUPL-1.2 verbatim, with
|
||||
`Copyright (c) 2026 Mokhtar Mial`. `README.md` gains a short notice naming the
|
||||
licence and the SPDX identifier `EUPL-1.2`. **No per-file SPDX headers.**
|
||||
`LICENSE` holds the English text of EUPL-1.2 verbatim, with `Copyright (c) 2026 Mokhtar Mial`. `README.md` gains a short notice naming the licence and the SPDX identifier `EUPL-1.2`. **No per-file SPDX headers.**
|
||||
|
||||
`THIRD-PARTY-NOTICES` is assembled by `zig build dist` from a committed,
|
||||
reviewed inventory under `licenses/`. It is not scraped from the dependency
|
||||
tree at build time: a generated notices file that nobody reads rots silently
|
||||
into a false statement.
|
||||
`THIRD-PARTY-NOTICES` is assembled by `zig build dist` from a committed, reviewed inventory under `licenses/`. It is not scraped from the dependency tree at build time: a generated notices file that nobody reads rots silently into a false statement.
|
||||
|
||||
The inventory must cover everything the shipped artifacts actually contain, not
|
||||
the direct dependency list:
|
||||
The inventory must cover everything the shipped artifacts actually contain, not the direct dependency list:
|
||||
|
||||
- **musl libc** (MIT) — statically linked into every binary.
|
||||
- **Zig standard library and compiler-rt** (MIT) — likewise.
|
||||
- **SQLite 3.53.4** — public domain, no obligation, listed for completeness.
|
||||
- **Mbed TLS 3.6.7** — with an explicit line recording that it is taken under
|
||||
the Apache-2.0 option of its dual `Apache-2.0 OR GPL-2.0-or-later` licence,
|
||||
followed by the **full Apache-2.0 text**. Naming the choice is good practice;
|
||||
shipping the text is the actual obligation.
|
||||
- **Project Everest and p256-m** — compiled in at `build.zig:394` even though
|
||||
the stock config leaves both drivers disabled.
|
||||
- **The web bundle's runtime closure** — the transitive set, not the four direct
|
||||
entries in `web/package.json`. Tailwind is a devDependency whose generated CSS
|
||||
ships, so "production dependencies" understates it.
|
||||
- **Mbed TLS 3.6.7** — with an explicit line recording that it is taken under the Apache-2.0 option of its dual `Apache-2.0 OR GPL-2.0-or-later` licence, followed by the **full Apache-2.0 text**. Naming the choice is good practice; shipping the text is the actual obligation.
|
||||
- **Project Everest and p256-m** — compiled in at `build.zig:394` even though the stock config leaves both drivers disabled.
|
||||
- **The web bundle's runtime closure** — the transitive set, not the four direct entries in `web/package.json`. Tailwind is a devDependency whose generated CSS ships, so "production dependencies" understates it.
|
||||
|
||||
A CI guard records the identity of the dependency sets (`build.zig.zon`
|
||||
dependencies, and the npm closure) and fails when either changes without a
|
||||
matching change under `licenses/`. It detects drift; it does not derive the
|
||||
inventory.
|
||||
A CI guard records the identity of the dependency sets (`build.zig.zon` dependencies, and the npm closure) and fails when either changes without a matching change under `licenses/`. It detects drift; it does not derive the inventory.
|
||||
|
||||
The **container image carries `/LICENSE` and `/THIRD-PARTY-NOTICES` too**.
|
||||
Distributing the image is distribution, and the obligations do not live in the
|
||||
tarball.
|
||||
The **container image carries `/LICENSE` and `/THIRD-PARTY-NOTICES` too**. Distributing the image is distribution, and the obligations do not live in the tarball.
|
||||
|
||||
### 4. `zig build dist` replaces `zig build cross`
|
||||
|
||||
The `cross` step is deleted. `dist` is the single command that produces
|
||||
everything releasable, and it runs on a laptop exactly as it runs on the runner.
|
||||
The `cross` step is deleted. `dist` is the single command that produces everything releasable, and it runs on a laptop exactly as it runs on the runner.
|
||||
|
||||
Inputs: `-Dversion-string` (required — no default), `-Dgit-commit`,
|
||||
`-Dweb-dist`, `-Doptimize=ReleaseSafe`.
|
||||
Inputs: `-Dversion-string` (required — no default), `-Dgit-commit`, `-Dweb-dist`, `-Doptimize=ReleaseSafe`.
|
||||
|
||||
**`dist` fails when `-Dweb-dist` resolves to `web/dist-placeholder`.** The
|
||||
default at `build.zig:24` is the placeholder, so a release built without the
|
||||
flag would silently ship a placeholder admin page. There is no override flag;
|
||||
an escape hatch here is a foot-gun with a safety label on it.
|
||||
**`dist` fails when `-Dweb-dist` resolves to `web/dist-placeholder`.** The default at `build.zig:24` is the placeholder, so a release built without the flag would silently ship a placeholder admin page. There is no override flag; an escape hatch here is a foot-gun with a safety label on it.
|
||||
|
||||
Per triple (`x86_64-linux-musl`, `aarch64-linux-musl`), `dist` builds
|
||||
ReleaseSafe with `.linkage = .static` and `.strip = true`. Stripping uses
|
||||
`std.Build.Module.strip` (`-fstrip`, `Module.zig:545`), which removes the
|
||||
`objcopy` and `binutils-aarch64-linux-gnu` dependency from the runner.
|
||||
Per triple (`x86_64-linux-musl`, `aarch64-linux-musl`), `dist` builds ReleaseSafe with `.linkage = .static` and `.strip = true`. Stripping uses `std.Build.Module.strip` (`-fstrip`, `Module.zig:545`), which removes the `objcopy` and `binutils-aarch64-linux-gnu` dependency from the runner.
|
||||
|
||||
It stages one directory per triple, `nxdns-<version>-<triple>/`, holding:
|
||||
|
||||
@@ -106,9 +63,7 @@ It stages one directory per triple, `nxdns-<version>-<triple>/`, holding:
|
||||
| `THIRD-PARTY-NOTICES` | 0644 |
|
||||
| `INSTALL.md` | 0644 |
|
||||
|
||||
`deploy/systemd/sysusers.conf` is renamed to `nxdns.conf` — the name it is
|
||||
installed under at `/usr/lib/sysusers.d/nxdns.conf`. A file that changes name
|
||||
during install is a step an operator can get wrong.
|
||||
`deploy/systemd/sysusers.conf` is renamed to `nxdns.conf` — the name it is installed under at `/usr/lib/sysusers.d/nxdns.conf`. A file that changes name during install is a step an operator can get wrong.
|
||||
|
||||
Archiving runs as two `b.addSystemCommand` steps, never one:
|
||||
|
||||
@@ -118,270 +73,130 @@ tar --format=gnu --sort=name --mtime=@0 --owner=0 --group=0 \
|
||||
gzip -n -9 <name>.tar
|
||||
```
|
||||
|
||||
`b.addSystemCommand` executes argv directly and does not interpret `|`, and a
|
||||
shell wrapper without `pipefail` would report only `gzip`'s exit status while a
|
||||
failed `tar` passed silently. `gzip -n` is required because `--mtime=@0`
|
||||
normalises the tar member times but not the timestamp gzip writes into its own
|
||||
header. The environment sets `LC_ALL=C` and `TZ=UTC`.
|
||||
`b.addSystemCommand` executes argv directly and does not interpret `|`, and a shell wrapper without `pipefail` would report only `gzip`'s exit status while a failed `tar` passed silently. `gzip -n` is required because `--mtime=@0` normalises the tar member times but not the timestamp gzip writes into its own header. The environment sets `LC_ALL=C` and `TZ=UTC`.
|
||||
|
||||
Both steps declare the staging directory as a build input and the archive as an
|
||||
output, so Zig's cache cannot serve a stale artifact.
|
||||
Both steps declare the staging directory as a build input and the archive as an output, so Zig's cache cannot serve a stale artifact.
|
||||
|
||||
`dist` also emits a checksum file covering **only the two tarballs**. It cannot
|
||||
cover the image: the digest does not exist until buildx has pushed, which
|
||||
happens later and elsewhere. The release job appends that line (ruling 7).
|
||||
`dist` also emits a checksum file covering **only the two tarballs**. It cannot cover the image: the digest does not exist until buildx has pushed, which happens later and elsewhere. The release job appends that line (ruling 7).
|
||||
|
||||
### 5. `zig build verify-dist` asserts what CI shell asserts today
|
||||
|
||||
The 50 lines of shell at `ci.yml:129` and `:168` move into a build step, so a
|
||||
developer can run the release checks on a laptop. Logic that only runs in CI is
|
||||
the brittleness this milestone exists to remove.
|
||||
The 50 lines of shell at `ci.yml:129` and `:168` move into a build step, so a developer can run the release checks on a laptop. Logic that only runs in CI is the brittleness this milestone exists to remove.
|
||||
|
||||
It operates on the **extracted** archive, not the staging directory:
|
||||
|
||||
- ELF header: correct `e_machine` per triple, **no `PT_INTERP`, no `DT_NEEDED`**.
|
||||
Matching the string `statically linked` from `file(1)` is not a static-linkage
|
||||
test.
|
||||
- ELF header: correct `e_machine` per triple, **no `PT_INTERP`, no `DT_NEEDED`**. Matching the string `statically linked` from `file(1)` is not a static-linkage test.
|
||||
- Stripped binary ≤ 15,728,640 bytes.
|
||||
- Archive layout: exactly one top-level directory, the exact file allowlist from
|
||||
ruling 4, expected modes, no symlinks, no path traversal.
|
||||
- `nxdns version` prints the version under build and the git commit. Native
|
||||
architecture only — the aarch64 binary needs qemu and is skipped without
|
||||
`-fqemu`.
|
||||
- Archive layout: exactly one top-level directory, the exact file allowlist from ruling 4, expected modes, no symlinks, no path traversal.
|
||||
- `nxdns version` prints the version under build and the git commit. Native architecture only — the aarch64 binary needs qemu and is skipped without `-fqemu`.
|
||||
- `build.zig.zon` `.version` equals the version under build.
|
||||
|
||||
The asset-free budget (10,485,760 bytes) gets **its own build against a
|
||||
generated empty assets directory**, not against the placeholder. Ruling 4 makes
|
||||
the placeholder unbuildable, and re-admitting it through a back door for one
|
||||
size check would defeat the point.
|
||||
The asset-free budget (10,485,760 bytes) gets **its own build against a generated empty assets directory**, not against the placeholder. Ruling 4 makes the placeholder unbuildable, and re-admitting it through a back door for one size check would defeat the point.
|
||||
|
||||
**A second `zig build` invocation does not inherit the first one's `-D` options.**
|
||||
`zig build dist -Dversion-string=X` followed by a bare `zig build verify-dist`
|
||||
verifies a *differently configured* build. Every caller — the workflows, the
|
||||
documentation, and this spec's own acceptance list — passes the same
|
||||
`-Dversion-string`, `-Dgit-commit`, `-Dweb-dist` and `-Doptimize` to both.
|
||||
**A second `zig build` invocation does not inherit the first one's `-D` options.** `zig build dist -Dversion-string=X` followed by a bare `zig build verify-dist` verifies a *differently configured* build. Every caller — the workflows, the documentation, and this spec's own acceptance list — passes the same `-Dversion-string`, `-Dgit-commit`, `-Dweb-dist` and `-Doptimize` to both.
|
||||
|
||||
### 6. Container image
|
||||
|
||||
`deploy/docker/Dockerfile`:
|
||||
|
||||
- Pin the base by digest:
|
||||
`alpine:3.22@sha256:14358309a308569c32bdc37e2e0e9694be33a9d99e68afb0f5ff33cc1f695dce`.
|
||||
- Add `--platform=$BUILDPLATFORM` to the builder stage. It only copies files, so
|
||||
pinning it to the build host means the arm64 image needs no qemu.
|
||||
- **Delete `apk add --no-cache ca-certificates` (`Dockerfile:15`).** Verified:
|
||||
the Alpine base already ships `/etc/ssl/certs/ca-certificates.crt` (179,359
|
||||
bytes) from `ca-certificates-bundle`. Removing it removes the only network
|
||||
fetch in the image build.
|
||||
- Keep the `mkdir` and `chown 65532` — `/var/lib/nxdns` must exist with that
|
||||
ownership so Docker copies it onto a fresh named volume.
|
||||
- Pin the base by digest: `alpine:3.22@sha256:14358309a308569c32bdc37e2e0e9694be33a9d99e68afb0f5ff33cc1f695dce`.
|
||||
- Add `--platform=$BUILDPLATFORM` to the builder stage. It only copies files, so pinning it to the build host means the arm64 image needs no qemu.
|
||||
- **Delete `apk add --no-cache ca-certificates` (`Dockerfile:15`).** Verified: the Alpine base already ships `/etc/ssl/certs/ca-certificates.crt` (179,359 bytes) from `ca-certificates-bundle`. Removing it removes the only network fetch in the image build.
|
||||
- Keep the `mkdir` and `chown 65532` — `/var/lib/nxdns` must exist with that ownership so Docker copies it onto a fresh named volume.
|
||||
- Add `/LICENSE` and `/THIRD-PARTY-NOTICES` (ruling 3).
|
||||
- OCI labels: `source`, `revision`, `version`, `licenses=EUPL-1.2`, `created`,
|
||||
`title`, `description`.
|
||||
- OCI labels: `source`, `revision`, `version`, `licenses=EUPL-1.2`, `created`, `title`, `description`.
|
||||
|
||||
`deploy/docker/.dockerignore` is renamed to `deploy/docker/Dockerfile.dockerignore`.
|
||||
The build context is the repository root, so Docker never reads the current
|
||||
path, and the whole worktree — `.git`, `node_modules`, `zig-cache` — is being
|
||||
sent to the daemon today.
|
||||
`deploy/docker/.dockerignore` is renamed to `deploy/docker/Dockerfile.dockerignore`. The build context is the repository root, so Docker never reads the current path, and the whole worktree — `.git`, `node_modules`, `zig-cache` — is being sent to the daemon today.
|
||||
|
||||
buildx runs with `--provenance=false --sbom=false`. Recent buildx adds
|
||||
provenance attestations by default, which create `unknown/unknown` platform
|
||||
entries and change the index digest; Gitea's OCI 1.1 support is unverified
|
||||
(go-gitea#25846) and this is not the milestone to find out. `SOURCE_DATE_EPOCH`
|
||||
comes from the tag date.
|
||||
buildx runs with `--provenance=false --sbom=false`. Recent buildx adds provenance attestations by default, which create `unknown/unknown` platform entries and change the index digest; Gitea's OCI 1.1 support is unverified (go-gitea#25846) and this is not the milestone to find out. `SOURCE_DATE_EPOCH` comes from the tag date.
|
||||
|
||||
`deploy/docker/compose.yaml` references the published image rather than
|
||||
building one.
|
||||
`deploy/docker/compose.yaml` references the published image rather than building one.
|
||||
|
||||
**Verify before publishing:** the binary inside each image is byte-identical to
|
||||
the binary in the matching tarball.
|
||||
**Verify before publishing:** the binary inside each image is byte-identical to the binary in the matching tarball.
|
||||
|
||||
### 7. Workflows
|
||||
|
||||
`.gitea/workflows/gates.yml`, `on: workflow_call`, holds every blocking check:
|
||||
`test` (with `-Dintegration`), `test-aarch64` (`-fqemu`), `frontend`, `package`
|
||||
(`dist` + `verify-dist` + the asset-free size check) and `container` (build the
|
||||
image and run the existing smoke test from `ci.yml:219`). Moving only the three
|
||||
test jobs would drop the packaging and container checks precisely when they
|
||||
matter most.
|
||||
`.gitea/workflows/gates.yml`, `on: workflow_call`, holds every blocking check: `test` (with `-Dintegration`), `test-aarch64` (`-fqemu`), `frontend`, `package` (`dist` + `verify-dist` + the asset-free size check) and `container` (build the image and run the existing smoke test from `ci.yml:219`). Moving only the three test jobs would drop the packaging and container checks precisely when they matter most.
|
||||
|
||||
`ci.yml` calls it on `push` and `pull_request` for **`master`**. `master` is
|
||||
canonical: `origin/main` is deleted, and Gitea's default branch is changed to
|
||||
match. Today `ci.yml:5` watches `main` while work happens on `master`, so
|
||||
milestone 13 has never run through CI.
|
||||
`ci.yml` calls it on `push` and `pull_request` for **`master`**. `master` is canonical: `origin/main` is deleted, and Gitea's default branch is changed to match. Today `ci.yml:5` watches `main` while work happens on `master`, so milestone 13 has never run through CI.
|
||||
|
||||
`.gitea/workflows/release.yml`, `on: push: tags: ['v*']`, in this order:
|
||||
|
||||
1. Checkout with `fetch-depth: 0` and tags. The default shallow clone breaks
|
||||
ancestry checks, previous-tag lookup and changelog generation.
|
||||
1. Checkout with `fetch-depth: 0` and tags. The default shallow clone breaks ancestry checks, previous-tag lookup and changelog generation.
|
||||
2. Reject any tag that is not exactly `vMAJOR.MINOR.PATCH`.
|
||||
3. `git verify-tag`, requiring an annotated tag and requiring the signature's
|
||||
fingerprint to equal a fingerprint pinned in the workflow. A bare
|
||||
`verify-tag` proves only that *some* imported key signed it.
|
||||
3. `git verify-tag`, requiring an annotated tag and requiring the signature's fingerprint to equal a fingerprint pinned in the workflow. A bare `verify-tag` proves only that *some* imported key signed it.
|
||||
4. Assert the tag's commit is an ancestor of `origin/master`.
|
||||
5. Assert no published release exists for this tag. Delete any leftover draft.
|
||||
6. Assert the version is greater than the highest published release version, so
|
||||
a late-finishing older tag cannot move `latest` backwards.
|
||||
6. Assert the version is greater than the highest published release version, so a late-finishing older tag cannot move `latest` backwards.
|
||||
7. Gates, blocking, via `needs:`.
|
||||
8. Build the web UI; `zig build dist`; `zig build verify-dist`.
|
||||
9. Extract the `CHANGELOG.md` section matching the version. **Fail when absent.**
|
||||
10. buildx build and push **`:<version>` only**. Capture the index digest from
|
||||
`--metadata-file`, validate it as `sha256:<64 hex>`, and confirm the pushed
|
||||
tag resolves to that digest with exactly the two intended platforms.
|
||||
11. Write the digest file, append its checksum line, verify the assembled
|
||||
checksum file.
|
||||
12. Sign the checksum file (ruling 8). Verify the signature locally before
|
||||
uploading it.
|
||||
10. buildx build and push **`:<version>` only**. Capture the index digest from `--metadata-file`, validate it as `sha256:<64 hex>`, and confirm the pushed tag resolves to that digest with exactly the two intended platforms.
|
||||
11. Write the digest file, append its checksum line, verify the assembled checksum file.
|
||||
12. Sign the checksum file (ruling 8). Verify the signature locally before uploading it.
|
||||
13. Create the release as a **draft**; upload the assets.
|
||||
14. Move `:latest` and verify it resolves to the built digest. Re-check the
|
||||
monotonic-version invariant here: step 6 ran before the gates, and proves
|
||||
nothing about which of two in-flight tags finishes last.
|
||||
15. **Publish the draft last.** Publication is the one irreversible act, so it
|
||||
goes after everything that can still fail.
|
||||
14. Move `:latest` and verify it resolves to the built digest. Re-check the monotonic-version invariant here: step 6 ran before the gates, and proves nothing about which of two in-flight tags finishes last.
|
||||
15. **Publish the draft last.** Publication is the one irreversible act, so it goes after everything that can still fail.
|
||||
|
||||
Amended after review. The original order published at 14 and moved `:latest` at
|
||||
15, which deadlocks: a failure while moving `:latest` leaves a published release,
|
||||
and ruling 9 makes a re-run refuse a published release. Nothing could repair it.
|
||||
The cost of the corrected order is a short window where `:latest` serves the new
|
||||
image before the release page is public. That is recoverable by a re-run; the
|
||||
deadlock was not.
|
||||
Amended after review. The original order published at 14 and moved `:latest` at 15, which deadlocks: a failure while moving `:latest` leaves a published release, and ruling 9 makes a re-run refuse a published release. Nothing could repair it. The cost of the corrected order is a short window where `:latest` serves the new image before the release page is public. That is recoverable by a re-run; the deadlock was not.
|
||||
|
||||
Every action in `release.yml` is pinned to a full commit SHA.
|
||||
`actions/checkout@v4` and `mlugg/setup-zig@v2` are mutable tags on another
|
||||
party's server, and a compromise upstream would run on the runner holding the
|
||||
signing subkey and the registry token. `ci.yml` may keep moving tags; it holds
|
||||
no secrets.
|
||||
Every action in `release.yml` is pinned to a full commit SHA. `actions/checkout@v4` and `mlugg/setup-zig@v2` are mutable tags on another party's server, and a compromise upstream would run on the runner holding the signing subkey and the registry token. `ci.yml` may keep moving tags; it holds no secrets.
|
||||
|
||||
### 8. Signing
|
||||
|
||||
**Two different keys are involved, and the original ruling conflated them.**
|
||||
|
||||
- The **tag-signing key** is the human's. `git tag -s` uses it, and step 3 checks
|
||||
it. What gets pinned in `release.yml` is the **primary certificate
|
||||
fingerprint**, which `git verify-tag --raw` emits as the **last** field of the
|
||||
`VALIDSIG` line. Field 3 is whichever key actually made the signature — the
|
||||
signing subkey once one exists. Pinning field 3 would mean that creating the
|
||||
release subkey below silently blocks every future release, with an error
|
||||
message that reads like a forged tag. Reproduced against a real keyring during
|
||||
review.
|
||||
- The **artifact-signing subkey** is the runner's, and it signs the checksum
|
||||
file. It is a dedicated GPG signing subkey of the author's existing key. A
|
||||
leaked subkey is revoked on its own; the identity, the commit signature history
|
||||
and everyone's existing trust survive.
|
||||
- The **tag-signing key** is the human's. `git tag -s` uses it, and step 3 checks it. What gets pinned in `release.yml` is the **primary certificate fingerprint**, which `git verify-tag --raw` emits as the **last** field of the `VALIDSIG` line. Field 3 is whichever key actually made the signature — the signing subkey once one exists. Pinning field 3 would mean that creating the release subkey below silently blocks every future release, with an error message that reads like a forged tag. Reproduced against a real keyring during review.
|
||||
- The **artifact-signing subkey** is the runner's, and it signs the checksum file. It is a dedicated GPG signing subkey of the author's existing key. A leaked subkey is revoked on its own; the identity, the commit signature history and everyone's existing trust survive.
|
||||
|
||||
Pinning the primary fingerprint means adding or rotating a signing subkey is a
|
||||
non-event for verification.
|
||||
Pinning the primary fingerprint means adding or rotating a signing subkey is a non-event for verification.
|
||||
|
||||
Every required secret is validated in the **guard job**, before the gates and
|
||||
before any registry push. Validating a fingerprint inside the signing step means
|
||||
a placeholder value burns an immutable version tag before it fails.
|
||||
Every required secret is validated in the **guard job**, before the gates and before any registry push. Validating a fingerprint inside the signing step means a placeholder value burns an immutable version tag before it fails.
|
||||
|
||||
Implementation requirements: a temporary `GNUPGHOME`; assert the imported
|
||||
material contains no primary secret key; `--local-user <subkey-fingerprint>!`
|
||||
so GPG cannot fall back to another key; batch and loopback pinentry; verify the
|
||||
produced signature before upload; scrub `GNUPGHOME` and `DOCKER_CONFIG` and kill
|
||||
the agent on every exit path. The registry token is passed by
|
||||
`--password-stdin` into a temporary `DOCKER_CONFIG`, and `persist-credentials`
|
||||
is off.
|
||||
Implementation requirements: a temporary `GNUPGHOME`; assert the imported material contains no primary secret key; `--local-user <subkey-fingerprint>!` so GPG cannot fall back to another key; batch and loopback pinentry; verify the produced signature before upload; scrub `GNUPGHOME` and `DOCKER_CONFIG` and kill the agent on every exit path. The registry token is passed by `--password-stdin` into a temporary `DOCKER_CONFIG`, and `persist-credentials` is off.
|
||||
|
||||
Secrets: `RELEASE_GPG_SUBKEY`, `RELEASE_GPG_PASSPHRASE`, `REGISTRY_TOKEN`. The
|
||||
built-in `GITEA_TOKEN` cannot publish to the package registry, which is why the
|
||||
third exists; it can still create the release and upload assets.
|
||||
Secrets: `RELEASE_GPG_SUBKEY`, `RELEASE_GPG_PASSPHRASE`, `REGISTRY_TOKEN`. The built-in `GITEA_TOKEN` cannot publish to the package registry, which is why the third exists; it can still create the release and upload assets.
|
||||
|
||||
**Stated honestly, in `docs/how-to/verify-a-release.md`:** the signature proves
|
||||
the artifact came from this pipeline and reached the operator unaltered. It does
|
||||
not prove the binary matches the source, because the machine that built it also
|
||||
held the key. Ruling 12 defers the control that closes that gap. Do not write a
|
||||
stronger claim than that.
|
||||
**Stated honestly, in `docs/how-to/verify-a-release.md`:** the signature proves the artifact came from this pipeline and reached the operator unaltered. It does not prove the binary matches the source, because the machine that built it also held the key. Ruling 12 defers the control that closes that gap. Do not write a stronger claim than that.
|
||||
|
||||
### 9. Failure and recovery are specified, not improvised
|
||||
|
||||
- The draft is the unit of work for the *release record*. The registry is not
|
||||
covered by it: the version tag becomes publicly pullable at step 10. Only the
|
||||
release page and its assets stay hidden until step 15.
|
||||
- The existence check happens **before** the push, not after. Gitea's container
|
||||
tags are mutable — immutability is a workflow invariant, not a registry
|
||||
guarantee — so a push-then-compare would already have overwritten the tag it
|
||||
claims to refuse. Read the existing digest with
|
||||
`HEAD /v2/mokhtar/nxdns/manifests/<version>`, sending an `Accept` header for
|
||||
the index media types and following the `401` bearer challenge with the PAT.
|
||||
Absent is `404`; present returns `Docker-Content-Digest`.
|
||||
- A re-run deletes an existing **draft** and repeats. It refuses to touch a
|
||||
**published** release.
|
||||
- The registry version tag is immutable, and the workflow never writes it twice.
|
||||
A re-run that finds the tag present pushes nothing: it adopts the pushed
|
||||
digest and asserts the *contents* of that image against the artifacts it just
|
||||
built, which is the check that matters and the only one available. Comparing a
|
||||
rebuilt index digest was the first design and is not implementable — buildx
|
||||
cannot report an index digest without pushing, and cross-machine
|
||||
reproducibility is deferred (ruling 12), so the rebuilt digest is expected to
|
||||
differ even when nothing changed. See recorded deviation 10.
|
||||
- `:latest` moves last, so a failure between the image push and publication
|
||||
leaves the version tag pushed and `latest` untouched. That is recoverable by
|
||||
re-running.
|
||||
- If a tag is burned — the pipeline itself is broken and the fix is on `master` —
|
||||
the release is abandoned and reissued as the next patch version. This is the
|
||||
terminal path, and it must be written down before `v0.0.1`, not discovered
|
||||
during it.
|
||||
- The draft is the unit of work for the *release record*. The registry is not covered by it: the version tag becomes publicly pullable at step 10. Only the release page and its assets stay hidden until step 15.
|
||||
- The existence check happens **before** the push, not after. Gitea's container tags are mutable — immutability is a workflow invariant, not a registry guarantee — so a push-then-compare would already have overwritten the tag it claims to refuse. Read the existing digest with `HEAD /v2/mokhtar/nxdns/manifests/<version>`, sending an `Accept` header for the index media types and following the `401` bearer challenge with the PAT. Absent is `404`; present returns `Docker-Content-Digest`.
|
||||
- A re-run deletes an existing **draft** and repeats. It refuses to touch a **published** release.
|
||||
- The registry version tag is immutable, and the workflow never writes it twice. A re-run that finds the tag present pushes nothing: it adopts the pushed digest and asserts the *contents* of that image against the artifacts it just built, which is the check that matters and the only one available. Comparing a rebuilt index digest was the first design and is not implementable — buildx cannot report an index digest without pushing, and cross-machine reproducibility is deferred (ruling 12), so the rebuilt digest is expected to differ even when nothing changed. See recorded deviation 10.
|
||||
- `:latest` moves last, so a failure between the image push and publication leaves the version tag pushed and `latest` untouched. That is recoverable by re-running.
|
||||
- If a tag is burned — the pipeline itself is broken and the fix is on `master` — the release is abandoned and reissued as the next patch version. This is the terminal path, and it must be written down before `v0.0.1`, not discovered during it.
|
||||
|
||||
### 10. Release notes
|
||||
|
||||
`CHANGELOG.md` at the repository root, Keep a Changelog format, with an
|
||||
`## [Unreleased]` section maintained as work happens. The release body is that
|
||||
section, plus a generated appendix: `git log --oneline` since the previous tag
|
||||
inside a collapsed `<details>`, a compare link, the tarball hashes and the image
|
||||
digest.
|
||||
`CHANGELOG.md` at the repository root, Keep a Changelog format, with an `## [Unreleased]` section maintained as work happens. The release body is that section, plus a generated appendix: `git log --oneline` since the previous tag inside a collapsed `<details>`, a compare link, the tarball hashes and the image digest.
|
||||
|
||||
The nullable value is "the previous reachable **published release**", never "the
|
||||
previous git tag". An abandoned tag from ruling 9 must not become the comparison
|
||||
base, and `git describe` would pick exactly that.
|
||||
The nullable value is "the previous reachable **published release**", never "the previous git tag". An abandoned tag from ruling 9 must not become the comparison base, and `git describe` would pick exactly that.
|
||||
|
||||
With no published base — the first release, and equally the case where `v0.0.1`
|
||||
was abandoned and `v0.0.2` becomes the first published one — the log is
|
||||
`git log --oneline <tag>` and the compare link is omitted. It is **not** a range
|
||||
with an empty left side: `..v0.0.1` resolves against `HEAD` and produces a wrong
|
||||
or empty appendix rather than "all history".
|
||||
With no published base — the first release, and equally the case where `v0.0.1` was abandoned and `v0.0.2` becomes the first published one — the log is `git log --oneline <tag>` and the compare link is omitted. It is **not** a range with an empty left side: `..v0.0.1` resolves against `HEAD` and produces a wrong or empty appendix rather than "all history".
|
||||
|
||||
The `v0.0.1` section is hand-written.
|
||||
|
||||
### 11. Documentation
|
||||
|
||||
`install-with-systemd.md`, `install-with-docker.md` and `upgrade.md` lead with
|
||||
download-and-verify; the existing build-from-source steps move to a later
|
||||
section of the same page. New `docs/how-to/verify-a-release.md` gives the
|
||||
verification commands and the rebuild recipe.
|
||||
`install-with-systemd.md`, `install-with-docker.md` and `upgrade.md` lead with download-and-verify; the existing build-from-source steps move to a later section of the same page. New `docs/how-to/verify-a-release.md` gives the verification commands and the rebuild recipe.
|
||||
|
||||
**Milestone 13 ruling 3 applies unchanged**: every command block in `tutorial/`
|
||||
and `how-to/` is executed on this host by the session that writes it, or marked
|
||||
in-page as unverified with the reason.
|
||||
**Milestone 13 ruling 3 applies unchanged**: every command block in `tutorial/` and `how-to/` is executed on this host by the session that writes it, or marked in-page as unverified with the reason.
|
||||
|
||||
Two rot hazards the drift test cannot see. `src/docs_drift_test.zig` guards only
|
||||
`reference/{api,configuration,cli}.md`; the how-to pages are unguarded:
|
||||
Two rot hazards the drift test cannot see. `src/docs_drift_test.zig` guards only `reference/{api,configuration,cli}.md`; the how-to pages are unguarded:
|
||||
|
||||
- The transcripts hardcode `nxdns 0.1.0-dev` in four pages. After `v0.0.1` they
|
||||
are wrong and nothing fails. Use a version-neutral placeholder, and add a
|
||||
guard that rejects a stale literal release version in the docs.
|
||||
- A download URL with the version in the path goes stale at `v0.0.2`. Use a
|
||||
`latest` download form if Gitea provides one, or a placeholder the reader
|
||||
substitutes. Probe first (ruling 13); do not guess.
|
||||
- The transcripts hardcode `nxdns 0.1.0-dev` in four pages. After `v0.0.1` they are wrong and nothing fails. Use a version-neutral placeholder, and add a guard that rejects a stale literal release version in the docs.
|
||||
- A download URL with the version in the path goes stale at `v0.0.2`. Use a `latest` download form if Gitea provides one, or a placeholder the reader substitutes. Probe first (ruling 13); do not guess.
|
||||
|
||||
Sweep the whole repository for surfaces the rewrite would otherwise miss:
|
||||
`README.md`, `Dockerfile` comments, `compose.yaml`, `explanation/performance-and-testing.md`,
|
||||
and any remaining `zig build cross` or source-only-distribution text.
|
||||
Sweep the whole repository for surfaces the rewrite would otherwise miss: `README.md`, `Dockerfile` comments, `compose.yaml`, `explanation/performance-and-testing.md`, and any remaining `zig build cross` or source-only-distribution text.
|
||||
|
||||
### 12. Deferred, deliberately
|
||||
|
||||
Recorded in PLAN so they are decisions rather than oversights:
|
||||
|
||||
- **The reproducibility gate** — build twice in two directory paths and assert
|
||||
identical hashes. Deferred until `v0.0.1` proves the pipeline. Until it exists,
|
||||
no document may describe the build as reproducible. Do the cheap parts now
|
||||
regardless: pin Node to an exact patch in `ci.yml` and `web/package.json`
|
||||
(both float at `24` today), `gzip -n`, `LC_ALL=C`, `TZ=UTC`.
|
||||
- **cosign signatures on the image** — Gitea Actions has no OIDC identity token,
|
||||
so keyless signing is impossible and only a key-based signature is available.
|
||||
Whether Gitea's registry accepts a cosign signature manifest is unverified.
|
||||
Spike it before committing to it.
|
||||
- **The reproducibility gate** — build twice in two directory paths and assert identical hashes. Deferred until `v0.0.1` proves the pipeline. Until it exists, no document may describe the build as reproducible. Do the cheap parts now regardless: pin Node to an exact patch in `ci.yml` and `web/package.json` (both float at `24` today), `gzip -n`, `LC_ALL=C`, `TZ=UTC`.
|
||||
- **cosign signatures on the image** — Gitea Actions has no OIDC identity token, so keyless signing is impossible and only a key-based signature is available. Whether Gitea's registry accepts a cosign signature manifest is unverified. Spike it before committing to it.
|
||||
|
||||
### 13. Manual prerequisites, done before the first tag
|
||||
|
||||
@@ -389,70 +204,37 @@ None of these are code, and all of them block `v0.0.1`:
|
||||
|
||||
The order matters, and the original list got parts of it wrong. Corrected:
|
||||
|
||||
1. Create the artifact-signing subkey, export it with `--export-secret-subkeys`,
|
||||
and publish the public key to `keys.openpgp.org`. Separately, identify which
|
||||
key actually signs your tags — after step 1 that is normally the new signing
|
||||
subkey, whose **primary** fingerprint is what gets pinned.
|
||||
2. Pin the primary fingerprint in `release.yml`, replacing the placeholder. The
|
||||
guard fails closed on the placeholder, so no tag can succeed before this.
|
||||
3. **Store** the secrets: `RELEASE_GPG_SUBKEY`, `RELEASE_GPG_PASSPHRASE`, and a
|
||||
registry personal access token with package write scope as `REGISTRY_TOKEN`.
|
||||
Creating the token is not storing it.
|
||||
4. `app.ini`: add `.asc` and `.sig` to `[attachment] ALLOWED_TYPES`, and raise
|
||||
`MAX_FILES` from 5. Restart Gitea and confirm the *effective* settings through
|
||||
`/api/v1/settings/attachment` before trusting them. Verified against the live
|
||||
instance on 2026-08-05: `max_size` is 100 MB (ample), `max_files` is 5 (this
|
||||
release has exactly five assets, no headroom), and `.asc` is absent, so the
|
||||
signature is rejected today. The implementation names the assets
|
||||
`SHA256SUMS.txt`, `SHA256SUMS.txt.asc` and `IMAGE-DIGEST.txt`, so `.txt` and
|
||||
`.asc` are the two extensions that must be allowed.
|
||||
5. **Probe: does the runner support `docker buildx` with the `docker-container`
|
||||
driver, and can it extract a foreign-platform image?** The old `docker` job
|
||||
only proved plain `docker build`, and the arm64 image-versus-tarball check
|
||||
needs more than that.
|
||||
6. Land the workflows on `master` **while `main` still exists**, and let the
|
||||
reusable-gates workflow run green once, so Gitea emits its actual
|
||||
status-check context names.
|
||||
7. Configure branch protection on `master` using **those observed names**. The
|
||||
reusable-workflow refactor changes them; keeping the old required checks can
|
||||
make merges impossible or leave the intended gates non-required.
|
||||
1. Create the artifact-signing subkey, export it with `--export-secret-subkeys`, and publish the public key to `keys.openpgp.org`. Separately, identify which key actually signs your tags — after step 1 that is normally the new signing subkey, whose **primary** fingerprint is what gets pinned.
|
||||
2. Pin the primary fingerprint in `release.yml`, replacing the placeholder. The guard fails closed on the placeholder, so no tag can succeed before this.
|
||||
3. **Store** the secrets: `RELEASE_GPG_SUBKEY`, `RELEASE_GPG_PASSPHRASE`, and a registry personal access token with package write scope as `REGISTRY_TOKEN`. Creating the token is not storing it.
|
||||
4. `app.ini`: add `.asc` and `.sig` to `[attachment] ALLOWED_TYPES`, and raise `MAX_FILES` from 5. Restart Gitea and confirm the *effective* settings through `/api/v1/settings/attachment` before trusting them. Verified against the live instance on 2026-08-05: `max_size` is 100 MB (ample), `max_files` is 5 (this release has exactly five assets, no headroom), and `.asc` is absent, so the signature is rejected today. The implementation names the assets `SHA256SUMS.txt`, `SHA256SUMS.txt.asc` and `IMAGE-DIGEST.txt`, so `.txt` and `.asc` are the two extensions that must be allowed.
|
||||
5. **Probe: does the runner support `docker buildx` with the `docker-container` driver, and can it extract a foreign-platform image?** The old `docker` job only proved plain `docker build`, and the arm64 image-versus-tarball check needs more than that.
|
||||
6. Land the workflows on `master` **while `main` still exists**, and let the reusable-gates workflow run green once, so Gitea emits its actual status-check context names.
|
||||
7. Configure branch protection on `master` using **those observed names**. The reusable-workflow refactor changes them; keeping the old required checks can make merges impossible or leave the intended gates non-required.
|
||||
8. Change Gitea's default branch to `master`.
|
||||
9. Delete `origin/main` **last** — Gitea refuses to delete the default branch.
|
||||
10. Merge the licence, changelog and `build.zig.zon` version commit, and let
|
||||
`master` go green.
|
||||
11. Dry-run the release path. A tag-triggered workflow cannot be exercised
|
||||
without pushing *some* tag, so use a disposable tag or a scratch repository.
|
||||
**Never use `v0.0.1` as the dry run** and then expect to reuse it: tags are
|
||||
never moved (ruling 2), so a burned dry-run tag is spent.
|
||||
10. Merge the licence, changelog and `build.zig.zon` version commit, and let `master` go green.
|
||||
11. Dry-run the release path. A tag-triggered workflow cannot be exercised without pushing *some* tag, so use a disposable tag or a scratch repository. **Never use `v0.0.1` as the dry run** and then expect to reuse it: tags are never moved (ruling 2), so a burned dry-run tag is spent.
|
||||
|
||||
## Sessions
|
||||
|
||||
S1–S5 run in parallel. S1 and S2 share one interface, fixed here so neither
|
||||
blocks: S2 owns everything under `licenses/` and the licence texts; S1 owns the
|
||||
build step that assembles them into `THIRD-PARTY-NOTICES`. The contract is
|
||||
`zig build dist -Dversion-string=X -Dgit-commit=Y -Dweb-dist=web/dist` writing
|
||||
to `zig-out/dist/`.
|
||||
S1–S5 run in parallel. S1 and S2 share one interface, fixed here so neither blocks: S2 owns everything under `licenses/` and the licence texts; S1 owns the build step that assembles them into `THIRD-PARTY-NOTICES`. The contract is `zig build dist -Dversion-string=X -Dgit-commit=Y -Dweb-dist=web/dist` writing to `zig-out/dist/`.
|
||||
|
||||
### Session S1: build system
|
||||
|
||||
Owns `build.zig`. Rulings 2, 4, 5. Deletes `cross`, adds `dist` and
|
||||
`verify-dist`, moves the CI shell asserts into the build graph.
|
||||
Owns `build.zig`. Rulings 2, 4, 5. Deletes `cross`, adds `dist` and `verify-dist`, moves the CI shell asserts into the build graph.
|
||||
|
||||
### Session S2: licence, notices, changelog
|
||||
|
||||
Owns `LICENSE`, `licenses/`, `CHANGELOG.md`, `README.md` (notice and links).
|
||||
Ruling 3, ruling 10. Compiles the inventory by auditing what the artifacts
|
||||
actually contain, and writes the CI drift guard for the dependency sets.
|
||||
Owns `LICENSE`, `licenses/`, `CHANGELOG.md`, `README.md` (notice and links). Ruling 3, ruling 10. Compiles the inventory by auditing what the artifacts actually contain, and writes the CI drift guard for the dependency sets.
|
||||
|
||||
### Session S3: container
|
||||
|
||||
Owns `deploy/docker/*`, `deploy/systemd/*` (the `sysusers.conf` rename).
|
||||
Ruling 6.
|
||||
Owns `deploy/docker/*`, `deploy/systemd/*` (the `sysusers.conf` rename). Ruling 6.
|
||||
|
||||
### Session S4: workflows
|
||||
|
||||
Owns `.gitea/workflows/*`. Rulings 7, 8, 9. Must not edit `build.zig` — it
|
||||
consumes the step contract above.
|
||||
Owns `.gitea/workflows/*`. Rulings 7, 8, 9. Must not edit `build.zig` — it consumes the step contract above.
|
||||
|
||||
### Session S5: documentation
|
||||
|
||||
@@ -460,146 +242,41 @@ Owns `docs/**`. Ruling 11, and milestone 13 ruling 3.
|
||||
|
||||
### Orchestrator
|
||||
|
||||
`PLAN.md` (ruling 1), this spec, deletion of `origin/main`, the manual
|
||||
prerequisites in ruling 13, and the `v0.0.1` tag.
|
||||
`PLAN.md` (ruling 1), this spec, deletion of `origin/main`, the manual prerequisites in ruling 13, and the `v0.0.1` tag.
|
||||
|
||||
## Recorded (implementation)
|
||||
|
||||
Accepted deviations and corrections from integration. The five rulings amended
|
||||
above (7, 8, 9, 10, 13, plus the option-inheritance note in 5 and the scoping of
|
||||
the last acceptance line) were all wrong as first written; each amendment says
|
||||
what it replaced and why.
|
||||
Accepted deviations and corrections from integration. The five rulings amended above (7, 8, 9, 10, 13, plus the option-inheritance note in 5 and the scoping of the last acceptance line) were all wrong as first written; each amendment says what it replaced and why.
|
||||
|
||||
1. **`build.zig.zon` said `0.1.0`.** It traced to the first build-baseline
|
||||
commit — a scaffold default never bumped. Ruling 2 makes `verify-dist` assert
|
||||
it against the version under build, so the CI packaging gate could never have
|
||||
passed. Set to `0.0.1`, matching the first tag.
|
||||
2. **Assets carry a `.txt` extension**: `SHA256SUMS.txt`, `SHA256SUMS.txt.asc`,
|
||||
`IMAGE-DIGEST.txt`. This resolves ruling 13's extensionless-asset probe by
|
||||
construction — `.txt` is already in the live `ALLOWED_TYPES`, so only `.asc`
|
||||
still has to be added.
|
||||
3. **`INSTALL.md` was added** at the repository root. `dist` hard-requires it in
|
||||
the staged payload and `verify-dist` asserts its mode.
|
||||
4. **The notices preamble moved to `licenses/preamble.txt`.** The tarball and the
|
||||
image ship different sets, and the hardcoded preamble scoped itself to "a
|
||||
single static executable" — which the image is not. The image additionally
|
||||
redistributes Alpine's Mozilla CA bundle (`MPL-2.0 AND MIT`, 179,359 bytes);
|
||||
the tarball does not.
|
||||
5. **Vite joined Tailwind in the inventory.** Both are devDependencies whose
|
||||
generated output ships inside the binary. The original inventory applied that
|
||||
rule to one of them and stopped.
|
||||
6. **Node is pinned to `24.19.0`** in `gates.yml`, `release.yml` and
|
||||
`web/package.json` — the exact-patch pin ruling 12 asks for.
|
||||
7. **`gates.yml` is SHA-pinned too.** Ruling 7 pinned only `release.yml`, but
|
||||
`release.yml` calls `gates.yml`, and those jobs share the runner host and
|
||||
docker daemon with the job holding the signing subkey. `live-tls.yml` keeps
|
||||
moving tags; it references no secret.
|
||||
8. **Determinism measured better than claimed.** Two `dist` runs with separate
|
||||
cache directories and separate prefixes produced byte-identical tarballs — a
|
||||
genuine recompile, not a cache replay. The documentation still claims only
|
||||
same-directory determinism, because ruling 12's gate does not exist yet and an
|
||||
unguarded property decays. The stronger result is recorded here, not promised
|
||||
to operators.
|
||||
9. **Test count moved from 1461 to 1481**: twelve licence-drift tests, then eight
|
||||
more from the review pass below. Skip counts are unchanged.
|
||||
1. **`build.zig.zon` said `0.1.0`.** It traced to the first build-baseline commit — a scaffold default never bumped. Ruling 2 makes `verify-dist` assert it against the version under build, so the CI packaging gate could never have passed. Set to `0.0.1`, matching the first tag.
|
||||
2. **Assets carry a `.txt` extension**: `SHA256SUMS.txt`, `SHA256SUMS.txt.asc`, `IMAGE-DIGEST.txt`. This resolves ruling 13's extensionless-asset probe by construction — `.txt` is already in the live `ALLOWED_TYPES`, so only `.asc` still has to be added.
|
||||
3. **`INSTALL.md` was added** at the repository root. `dist` hard-requires it in the staged payload and `verify-dist` asserts its mode.
|
||||
4. **The notices preamble moved to `licenses/preamble.txt`.** The tarball and the image ship different sets, and the hardcoded preamble scoped itself to "a single static executable" — which the image is not. The image additionally redistributes Alpine's Mozilla CA bundle (`MPL-2.0 AND MIT`, 179,359 bytes); the tarball does not.
|
||||
5. **Vite joined Tailwind in the inventory.** Both are devDependencies whose generated output ships inside the binary. The original inventory applied that rule to one of them and stopped.
|
||||
6. **Node is pinned to `24.19.0`** in `gates.yml`, `release.yml` and `web/package.json` — the exact-patch pin ruling 12 asks for.
|
||||
7. **`gates.yml` is SHA-pinned too.** Ruling 7 pinned only `release.yml`, but `release.yml` calls `gates.yml`, and those jobs share the runner host and docker daemon with the job holding the signing subkey. `live-tls.yml` keeps moving tags; it references no secret.
|
||||
8. **Determinism measured better than claimed.** Two `dist` runs with separate cache directories and separate prefixes produced byte-identical tarballs — a genuine recompile, not a cache replay. The documentation still claims only same-directory determinism, because ruling 12's gate does not exist yet and an unguarded property decays. The stronger result is recorded here, not promised to operators.
|
||||
9. **Test count moved from 1461 to 1481**: twelve licence-drift tests, then eight more from the review pass below. Skip counts are unchanged.
|
||||
|
||||
The rest came out of an adversarial review of the finished implementation. Each
|
||||
was reproduced before it was fixed.
|
||||
The rest came out of an adversarial review of the finished implementation. Each was reproduced before it was fixed.
|
||||
|
||||
10. **The version tag was pushed before the immutability check ran.** The first
|
||||
implementation pushed `:$VERSION` and then compared the resulting digest
|
||||
with the pre-push one — a check that reports a violation it just caused. On
|
||||
a re-run that produced different bytes the tag was already overwritten and
|
||||
the original image lost. Replaced by the probe-then-adopt design ruling 9
|
||||
now describes: a real `HEAD /v2/…/manifests/<version>`, and an existing tag
|
||||
is adopted rather than rebuilt. Exercised against a fake registry covering
|
||||
`404`, `200`, the `401` bearer challenge, `500`, and a `200` with no
|
||||
`Docker-Content-Digest`.
|
||||
11. **The guard proved nothing about the signing key.** It checked that
|
||||
`RELEASE_GPG_SUBKEY` and `RELEASE_GPG_PASSPHRASE` were non-empty. A
|
||||
public-only export, an export missing the pinned subkey, and a placeholder
|
||||
passphrase all passed it and failed for the first time in the signing step —
|
||||
after the image push. The guard now imports the material, asserts the pinned
|
||||
subkey is present as a secret key, and signs a throwaway file with the
|
||||
passphrase.
|
||||
12. **`jq … | grep … || true` swallowed a malformed API response.** The `|| true`
|
||||
exists so grep's no-match is not fatal; it covers the whole pipeline, so a
|
||||
`200` carrying a JSON object instead of an array read as "no published
|
||||
releases" — the one wrong answer that moves `:latest` backwards. The payload
|
||||
is now type-checked before it is read.
|
||||
13. **The monotonic re-check was not the concurrency backstop it claimed to be.**
|
||||
Two concurrent releases are both drafts while they run, so neither appears in
|
||||
the other's published list and both pass. The workflow `concurrency:` group
|
||||
is the only thing that serialises them. A second check was added that does
|
||||
close it: `:latest`'s own version label is read from the registry
|
||||
immediately before the tag moves, so the invariant is checked against the
|
||||
state being mutated.
|
||||
14. **Publication could deadlock the tag.** A `PATCH` that Gitea committed but
|
||||
whose response was lost left the release public and the step failed, after
|
||||
which the guard refused every re-run. The step now re-reads the release and
|
||||
treats an already-published one as success.
|
||||
15. **The secret-scrub backstop killed the wrong gpg-agent.** `gpgconf --kill`
|
||||
acts on the agent of the `GNUPGHOME` it is pointed at, and the bare call
|
||||
killed the runner's default agent while every leaked temporary home's agent
|
||||
kept running with the key cached. It now kills each home in its own home,
|
||||
and the guard job has the same `if: always()` backstop the publish job had.
|
||||
16. **The Zig dependency guard could be silenced by pasting.** Unlike the npm
|
||||
half, nothing tied `build.zig.zon` back to the inventory, so a Mbed TLS bump
|
||||
plus the suggested identity paste left the notices claiming the old version.
|
||||
Each dependency now has to appear in the inventory at the version its URL
|
||||
names. The Zig toolchain version and the entries vendored inside Mbed TLS are
|
||||
checked the same way.
|
||||
17. **The container base image was not an input to any guard.** It is the source
|
||||
of the CA bundle the image redistributes. `deploy/docker/Dockerfile` is now
|
||||
embedded in the `licenses_files` module, its digest-pinned `FROM` is a
|
||||
recorded identity section, and the CA bundle entry must name the Alpine
|
||||
release that `FROM` pins.
|
||||
18. **A tree-shaken package that started shipping would have gone unnoticed.**
|
||||
`cookie-es`, `isbot`, `seroval` and `seroval-plugins` are in the lockfile
|
||||
closure and in no shipped byte. If application code imported one, no
|
||||
lockfile, version or dependency set would change — only the bundle. The
|
||||
frontend gate now recomputes the set of packages in `web/dist` from a
|
||||
`--sourcemap` build and diffs it against a recorded section, and the drift
|
||||
test requires every name in that section to be inventoried and refuses one
|
||||
that is still listed as not shipped.
|
||||
19. **The recorded npm licence token was parsed and discarded**, so a package
|
||||
that relicensed passed as long as its version had not moved. Shipped
|
||||
packages must now all carry the licence the inventory's texts assume;
|
||||
`npm_not_shipped` is exempt, and `isbot` is Unlicense.
|
||||
20. **The full-text licence checks were marker probes.** They prove the right
|
||||
document is present but survive most of it being deleted. The Apache-2.0 and
|
||||
MPL-2.0 texts are now pinned by SHA-256.
|
||||
21. **Zig's compiler-rt contains code ported from LLVM's.** Zig's `LICENSE` is
|
||||
bare MIT naming only "Zig contributors". Reviewed: the ports carry
|
||||
Apache-2.0 WITH LLVM-exception, and that exception waives Apache §4(a),
|
||||
§4(b) and §4(d) for portions embedded in object form — the only form nxdns
|
||||
ships — so no further notice is owed. Recorded in the Zig inventory note
|
||||
rather than left as an unexamined gap.
|
||||
22. **`npx` was replaced by the installed binary** in the new frontend gate. `npx`
|
||||
downloads a package it cannot find locally, so a wrong working directory
|
||||
would have turned a licence check into an unpinned fetch. Reproduced: it
|
||||
fetched `vite@8.2.0` over the pinned `8.1.5`.
|
||||
10. **The version tag was pushed before the immutability check ran.** The first implementation pushed `:$VERSION` and then compared the resulting digest with the pre-push one — a check that reports a violation it just caused. On a re-run that produced different bytes the tag was already overwritten and the original image lost. Replaced by the probe-then-adopt design ruling 9 now describes: a real `HEAD /v2/…/manifests/<version>`, and an existing tag is adopted rather than rebuilt. Exercised against a fake registry covering `404`, `200`, the `401` bearer challenge, `500`, and a `200` with no `Docker-Content-Digest`.
|
||||
11. **The guard proved nothing about the signing key.** It checked that `RELEASE_GPG_SUBKEY` and `RELEASE_GPG_PASSPHRASE` were non-empty. A public-only export, an export missing the pinned subkey, and a placeholder passphrase all passed it and failed for the first time in the signing step — after the image push. The guard now imports the material, asserts the pinned subkey is present as a secret key, and signs a throwaway file with the passphrase.
|
||||
12. **`jq … | grep … || true` swallowed a malformed API response.** The `|| true` exists so grep's no-match is not fatal; it covers the whole pipeline, so a `200` carrying a JSON object instead of an array read as "no published releases" — the one wrong answer that moves `:latest` backwards. The payload is now type-checked before it is read.
|
||||
13. **The monotonic re-check was not the concurrency backstop it claimed to be.** Two concurrent releases are both drafts while they run, so neither appears in the other's published list and both pass. The workflow `concurrency:` group is the only thing that serialises them. A second check was added that does close it: `:latest`'s own version label is read from the registry immediately before the tag moves, so the invariant is checked against the state being mutated.
|
||||
14. **Publication could deadlock the tag.** A `PATCH` that Gitea committed but whose response was lost left the release public and the step failed, after which the guard refused every re-run. The step now re-reads the release and treats an already-published one as success.
|
||||
15. **The secret-scrub backstop killed the wrong gpg-agent.** `gpgconf --kill` acts on the agent of the `GNUPGHOME` it is pointed at, and the bare call killed the runner's default agent while every leaked temporary home's agent kept running with the key cached. It now kills each home in its own home, and the guard job has the same `if: always()` backstop the publish job had.
|
||||
16. **The Zig dependency guard could be silenced by pasting.** Unlike the npm half, nothing tied `build.zig.zon` back to the inventory, so a Mbed TLS bump plus the suggested identity paste left the notices claiming the old version. Each dependency now has to appear in the inventory at the version its URL names. The Zig toolchain version and the entries vendored inside Mbed TLS are checked the same way.
|
||||
17. **The container base image was not an input to any guard.** It is the source of the CA bundle the image redistributes. `deploy/docker/Dockerfile` is now embedded in the `licenses_files` module, its digest-pinned `FROM` is a recorded identity section, and the CA bundle entry must name the Alpine release that `FROM` pins.
|
||||
18. **A tree-shaken package that started shipping would have gone unnoticed.** `cookie-es`, `isbot`, `seroval` and `seroval-plugins` are in the lockfile closure and in no shipped byte. If application code imported one, no lockfile, version or dependency set would change — only the bundle. The frontend gate now recomputes the set of packages in `web/dist` from a `--sourcemap` build and diffs it against a recorded section, and the drift test requires every name in that section to be inventoried and refuses one that is still listed as not shipped.
|
||||
19. **The recorded npm licence token was parsed and discarded**, so a package that relicensed passed as long as its version had not moved. Shipped packages must now all carry the licence the inventory's texts assume; `npm_not_shipped` is exempt, and `isbot` is Unlicense.
|
||||
20. **The full-text licence checks were marker probes.** They prove the right document is present but survive most of it being deleted. The Apache-2.0 and MPL-2.0 texts are now pinned by SHA-256.
|
||||
21. **Zig's compiler-rt contains code ported from LLVM's.** Zig's `LICENSE` is bare MIT naming only "Zig contributors". Reviewed: the ports carry Apache-2.0 WITH LLVM-exception, and that exception waives Apache §4(a), §4(b) and §4(d) for portions embedded in object form — the only form nxdns ships — so no further notice is owed. Recorded in the Zig inventory note rather than left as an unexamined gap.
|
||||
22. **`npx` was replaced by the installed binary** in the new frontend gate. `npx` downloads a package it cannot find locally, so a wrong working directory would have turned a licence check into an unpinned fetch. Reproduced: it fetched `vite@8.2.0` over the pinned `8.1.5`.
|
||||
|
||||
23. **`actions/checkout` destroys the annotated tag object.** Found by the first
|
||||
live dry run, not by review: on a tag ref, checkout fetches the *commit* SHA
|
||||
into `refs/tags/<tag>`, so the signed tag reads as lightweight and the guard
|
||||
refuses it as unannotated. Both jobs that read the tag object — signature
|
||||
verification in the guard, the tagger date in the publish job — now force-
|
||||
refetch `refs/tags/$TAG` from origin first. The same run also proved the
|
||||
fail-closed secret guard for real: the first dry-run attempt ran with no
|
||||
secrets configured (they were on the wrong repository) and stopped in the
|
||||
guard with nothing built or pushed.
|
||||
23. **`actions/checkout` destroys the annotated tag object.** Found by the first live dry run, not by review: on a tag ref, checkout fetches the *commit* SHA into `refs/tags/<tag>`, so the signed tag reads as lightweight and the guard refuses it as unannotated. Both jobs that read the tag object — signature verification in the guard, the tagger date in the publish job — now force- refetch `refs/tags/$TAG` from origin first. The same run also proved the fail-closed secret guard for real: the first dry-run attempt ran with no secrets configured (they were on the wrong repository) and stopped in the guard with nothing built or pushed.
|
||||
|
||||
24. **Publication orchestration moved out of workflow shell into
|
||||
`tools/release.zig`.** Ruling 5 already moved the packaging asserts out of
|
||||
CI shell for one reason — "checks that only exist inside a workflow file are
|
||||
the brittleness this exists to remove" — and the release job was the larger
|
||||
half of the same problem, left in place. Three live failures came out of it,
|
||||
and each was found by executing the workflow, which is the most expensive
|
||||
place to find anything: `actions/checkout` replacing the annotated tag object
|
||||
(deviation 23), the refetch that fixed it having no credentials because
|
||||
`persist-credentials` is off, and the multiline armored subkey escaping the
|
||||
runner's log masker, which masks per line.
|
||||
24. **Publication orchestration moved out of workflow shell into `tools/release.zig`.** Ruling 5 already moved the packaging asserts out of CI shell for one reason — "checks that only exist inside a workflow file are the brittleness this exists to remove" — and the release job was the larger half of the same problem, left in place. Three live failures came out of it, and each was found by executing the workflow, which is the most expensive place to find anything: `actions/checkout` replacing the annotated tag object (deviation 23), the refetch that fixed it having no credentials because `persist-credentials` is off, and the multiline armored subkey escaping the runner's log masker, which masks per line.
|
||||
|
||||
Twelve subcommands, one per step group: `guard-tag`, `guard-ancestry`,
|
||||
`guard-releases`, `resolve`, `changelog`, `image`,
|
||||
@@ -627,55 +304,18 @@ was reproduced before it was fixed.
|
||||
writes it to a mode-600 file inside the temporary `GNUPGHOME`. A single-line
|
||||
secret is one the masker can actually mask.
|
||||
|
||||
25. **The first signing subkey was leaked into a job log and rotated.** Dry-run
|
||||
attempt 3 failed inside the credential-less refetch, and the runner printed
|
||||
the failing step's env block; the multiline armored `RELEASE_GPG_SUBKEY`
|
||||
escaped the per-line masker while the single-line passphrase was masked.
|
||||
Exposure: the passphrase-protected secret subkey only — the passphrase and
|
||||
the primary key were never on the runner. Response: both runs that ever saw
|
||||
the secret were deleted (verified 404 via the API and absent from
|
||||
`actions_log` on disk), subkey `B281CECC…` was revoked with the primary,
|
||||
and its replacement `019D00DF…` is the pinned `RELEASE_SIGNING_FPR`. The
|
||||
base64 contract in deviation 24 is the preventive half of this record.
|
||||
25. **The first signing subkey was leaked into a job log and rotated.** Dry-run attempt 3 failed inside the credential-less refetch, and the runner printed the failing step's env block; the multiline armored `RELEASE_GPG_SUBKEY` escaped the per-line masker while the single-line passphrase was masked. Exposure: the passphrase-protected secret subkey only — the passphrase and the primary key were never on the runner. Response: both runs that ever saw the secret were deleted (verified 404 via the API and absent from `actions_log` on disk), subkey `B281CECC…` was revoked with the primary, and its replacement `019D00DF…` is the pinned `RELEASE_SIGNING_FPR`. The base64 contract in deviation 24 is the preventive half of this record.
|
||||
|
||||
26. **The image is named by the public registry host, never the server URL.**
|
||||
Attempt 4 reached the registry and failed at `docker login gitea:3000`:
|
||||
inside the cluster `GITHUB_SERVER_URL` is `http://gitea:3000`, docker
|
||||
refuses plain-http registries, and an image named `gitea:3000/…` would be
|
||||
unpullable from anywhere that matters — a wrong name that would have been
|
||||
written into the released `IMAGE-DIGEST.txt`. `release.yml` now pins
|
||||
`REGISTRY_HOST: git.mial.net`; the tool uses it for docker and image
|
||||
naming, and keeps the internal URL for the manifest probe (same registry,
|
||||
no TLS dependency in the tool). The old shell had the identical latent bug;
|
||||
no run ever reached it.
|
||||
26. **The image is named by the public registry host, never the server URL.** Attempt 4 reached the registry and failed at `docker login gitea:3000`: inside the cluster `GITHUB_SERVER_URL` is `http://gitea:3000`, docker refuses plain-http registries, and an image named `gitea:3000/…` would be unpullable from anywhere that matters — a wrong name that would have been written into the released `IMAGE-DIGEST.txt`. `release.yml` now pins `REGISTRY_HOST: git.mial.net`; the tool uses it for docker and image naming, and keeps the internal URL for the manifest probe (same registry, no TLS dependency in the tool). The old shell had the identical latent bug; no run ever reached it.
|
||||
|
||||
### Not verified, and why
|
||||
|
||||
- **The workflows' validation history.** Before any live run, `release.yml` and
|
||||
`gates.yml` were validated by YAML parse and `bash -n`, plus two steps lifted
|
||||
out and run directly: the registry probe against a fake registry (five
|
||||
response shapes) and the bundled-package check against the real `web/` build,
|
||||
proven able to fail. The live dry run then superseded this: attempt 5
|
||||
published `v0.0.0` end to end — guard, gates, image push to both platforms,
|
||||
binary-identity assertion, signing, draft, `:latest`, publication — and the
|
||||
assets verified from a clean directory (checksums OK, signature good under
|
||||
the rotated subkey). The throwaway release, tag and registry versions were
|
||||
deleted afterwards.
|
||||
Everything else that talks to the registry or the Gitea API — `buildx build
|
||||
--push`, `imagetools`, draft creation, asset upload, publication, the
|
||||
adopt-an-existing-tag path — is unexercised.
|
||||
- **`RELEASE_SIGNING_FPR` is still the placeholder.** By design: the guard fails
|
||||
closed on it. It also means the release workflow cannot succeed as committed
|
||||
until manual prerequisite 2 is done.
|
||||
- **The aarch64 binary was never executed.** No `qemu-aarch64` on the build host,
|
||||
so `verify-dist`'s aarch64 version check legitimately skips and `test-aarch64`
|
||||
could not run. The binary is checked statically: ELF class, no `PT_INTERP`, no
|
||||
`DT_NEEDED`, size.
|
||||
- **The multi-architecture image build is unverified.** Only the native amd64
|
||||
image was built and run.
|
||||
- **The workflows' validation history.** Before any live run, `release.yml` and `gates.yml` were validated by YAML parse and `bash -n`, plus two steps lifted out and run directly: the registry probe against a fake registry (five response shapes) and the bundled-package check against the real `web/` build, proven able to fail. The live dry run then superseded this: attempt 5 published `v0.0.0` end to end — guard, gates, image push to both platforms, binary-identity assertion, signing, draft, `:latest`, publication — and the assets verified from a clean directory (checksums OK, signature good under the rotated subkey). The throwaway release, tag and registry versions were deleted afterwards. Everything else that talks to the registry or the Gitea API — `buildx build --push`, `imagetools`, draft creation, asset upload, publication, the adopt-an-existing-tag path — is unexercised.
|
||||
- **`RELEASE_SIGNING_FPR` is still the placeholder.** By design: the guard fails closed on it. It also means the release workflow cannot succeed as committed until manual prerequisite 2 is done.
|
||||
- **The aarch64 binary was never executed.** No `qemu-aarch64` on the build host, so `verify-dist`'s aarch64 version check legitimately skips and `test-aarch64` could not run. The binary is checked statically: ELF class, no `PT_INTERP`, no `DT_NEEDED`, size.
|
||||
- **The multi-architecture image build is unverified.** Only the native amd64 image was built and run.
|
||||
- **`shellcheck` was not run** — not installed on the build host.
|
||||
- **The local Node is 24.14.1, not the pinned 24.19.0.** There is no `.npmrc`, so
|
||||
`engines` does not hard-fail, and the frontend gates ran under the older patch.
|
||||
- **The local Node is 24.14.1, not the pinned 24.19.0.** There is no `.npmrc`, so `engines` does not hard-fail, and the frontend gates ran under the older patch.
|
||||
|
||||
## Acceptance (milestone complete)
|
||||
|
||||
@@ -742,8 +382,7 @@ was reproduced before it was fixed.
|
||||
|
||||
- No GoReleaser, no nfpm, no `.deb` or `.rpm`, no Homebrew, no AUR.
|
||||
- No cosign, no SBOM, no SLSA provenance, no in-toto attestations.
|
||||
- No `:edge` image, no rolling dev build, no pre-release tags, no floating
|
||||
`0.0` or `0` image tags.
|
||||
- No `:edge` image, no rolling dev build, no pre-release tags, no floating `0.0` or `0` image tags.
|
||||
- No third architecture, no non-Linux target, no glibc build.
|
||||
- No project website, no documentation site generator.
|
||||
- No changelog generation from commit messages, and no Conventional Commits.
|
||||
|
||||
+52
-237
@@ -1,44 +1,24 @@
|
||||
# Milestone 15: make a green run mean a real pass
|
||||
|
||||
Goal: repair the verification pipeline — CI that watches the branch we push to,
|
||||
a test run whose output contains no false failure text, guards that make the
|
||||
test list and the embedded frontend complete by construction, and coverage for
|
||||
the seams the audit found unguarded. This is phase 1 of `TECH_DEBT.md`; nothing
|
||||
in phases 2-5 can be validated until this lands.
|
||||
Goal: repair the verification pipeline — CI that watches the branch we push to, a test run whose output contains no false failure text, guards that make the test list and the embedded frontend complete by construction, and coverage for the seams the audit found unguarded. This is phase 1 of `TECH_DEBT.md`; nothing in phases 2-5 can be validated until this lands.
|
||||
|
||||
Origin: `TECH_DEBT.md` Theme 1 (lines 17-52), audited at `1ff727f`, every
|
||||
finding adversarially verified.
|
||||
Origin: `TECH_DEBT.md` Theme 1 (lines 17-52), audited at `1ff727f`, every finding adversarially verified.
|
||||
|
||||
## Sequencing (binding)
|
||||
|
||||
This milestone lands **before** milestone 14 is implemented. Ruling 1 pulls the
|
||||
`ci.yml` branch switch forward from milestone-14 §7; when milestone 14's S4
|
||||
later restructures the workflows into `gates.yml`/`release.yml`, it rebases on
|
||||
what this milestone leaves behind. One more overlap is binding: milestone 14's
|
||||
S1 rewrites `build.zig` (deletes `cross`, adds `dist`/`verify-dist`) — it must
|
||||
rebase on and **preserve** the configure-time machinery this milestone adds
|
||||
there (ruling 4's import guard, ruling 5's stamp check, ruling 6b's staged
|
||||
`core` module). Milestone 14's spec stands as written otherwise.
|
||||
This milestone lands **before** milestone 14 is implemented. Ruling 1 pulls the `ci.yml` branch switch forward from milestone-14 §7; when milestone 14's S4 later restructures the workflows into `gates.yml`/`release.yml`, it rebases on what this milestone leaves behind. One more overlap is binding: milestone 14's S1 rewrites `build.zig` (deletes `cross`, adds `dist`/`verify-dist`) — it must rebase on and **preserve** the configure-time machinery this milestone adds there (ruling 4's import guard, ruling 5's stamp check, ruling 6b's staged `core` module). Milestone 14's spec stands as written otherwise.
|
||||
|
||||
## Rulings (binding)
|
||||
|
||||
### 1. CI watches `master`
|
||||
|
||||
`.gitea/workflows/ci.yml:1-11` triggers on `push`/`pull_request` for `main`.
|
||||
Day-to-day pushes go to `origin/master` (`branch.master.merge` confirms), so
|
||||
milestone 13+ never ran through any gate. Change both `branches:` lists to
|
||||
`[master]`. Nothing else in the file changes — the `gates.yml` restructure is
|
||||
milestone-14 work.
|
||||
`.gitea/workflows/ci.yml:1-11` triggers on `push`/`pull_request` for `main`. Day-to-day pushes go to `origin/master` (`branch.master.merge` confirms), so milestone 13+ never ran through any gate. Change both `branches:` lists to `[master]`. Nothing else in the file changes — the `gates.yml` restructure is milestone-14 work.
|
||||
|
||||
Orchestrator, after merge: delete `origin/main`, set the Gitea default branch
|
||||
to `master`, push, and confirm a run starts. (Milestone-14 §7 already rules
|
||||
`master` canonical; this executes the branch part now.)
|
||||
Orchestrator, after merge: delete `origin/main`, set the Gitea default branch to `master`, push, and confirm a run starts. (Milestone-14 §7 already rules `master` canonical; this executes the branch part now.)
|
||||
|
||||
### 2. The live-network suite runs on a schedule
|
||||
|
||||
`.gitea/workflows/live-tls.yml` runs on `workflow_dispatch` only. Both failure
|
||||
modes it exists to catch have each happened once (35f2324; the milestone-4 cert
|
||||
drift). Add:
|
||||
`.gitea/workflows/live-tls.yml` runs on `workflow_dispatch` only. Both failure modes it exists to catch have each happened once (35f2324; the milestone-4 cert drift). Add:
|
||||
|
||||
```yaml
|
||||
on:
|
||||
@@ -47,225 +27,83 @@ on:
|
||||
- cron: "0 5 * * 1"
|
||||
```
|
||||
|
||||
Weekly, Monday 05:00 UTC. It stays non-blocking — this does not conflict with
|
||||
the milestone-1 ruling that live tests never gate a push. Everything else in
|
||||
the file is untouched.
|
||||
Weekly, Monday 05:00 UTC. It stays non-blocking — this does not conflict with the milestone-1 ruling that live tests never gate a push. Everything else in the file is untouched.
|
||||
|
||||
### 3. The teardown abort is root-caused, or documented where it lives
|
||||
|
||||
Reproduction, verified at HEAD: `zig build test` exits 0 but prints as its
|
||||
last line `failed command: .../test ... --listen=-`. The same binary exits 0
|
||||
in stdio mode (`1242 passed; 115 skipped; 0 failed`) and exits 134 (SIGABRT)
|
||||
under `--listen=-`. The abort happens after all tests pass, in teardown, only
|
||||
under zig's IPC runner. No comment anywhere in the repository records this;
|
||||
the knowledge lives in one contributor's memory and `TECH_DEBT.md:24-25`.
|
||||
Reproduction, verified at HEAD: `zig build test` exits 0 but prints as its last line `failed command: .../test ... --listen=-`. The same binary exits 0 in stdio mode (`1242 passed; 115 skipped; 0 failed`) and exits 134 (SIGABRT) under `--listen=-`. The abort happens after all tests pass, in teardown, only under zig's IPC runner. No comment anywhere in the repository records this; the knowledge lives in one contributor's memory and `TECH_DEBT.md:24-25`.
|
||||
|
||||
The cost is not the abort — it is that the one string that should mean failure
|
||||
prints on every success, and everyone learns to ignore failure text.
|
||||
The cost is not the abort — it is that the one string that should mean failure prints on every success, and everyone learns to ignore failure text.
|
||||
|
||||
The session must attempt a root cause, in this order:
|
||||
|
||||
1. Reproduce: run the cached test binary directly with and without
|
||||
`--listen=-`. Capture the abort backtrace (`ulimit -c` / gdb / the zig
|
||||
`--debug-...` runner flags — whatever this host offers).
|
||||
2. Bisect the mbedTLS linkage: `addMbedtlsThreadingMacros` (build.zig), the
|
||||
shim (`src/platform/mbedtls_shim.c`), and the two `extern fn` link checks in
|
||||
`src/tests.zig:121-123` are the suspects. A minimal reproducer (empty test
|
||||
file + mbedTLS link + `--listen`) decides whether the abort is ours or an
|
||||
upstream zig 0.16 runner defect.
|
||||
3. Outcome A — the cause is ours: fix it. Acceptance is a `zig build test` run
|
||||
whose output contains no `failed command` line.
|
||||
4. Outcome B — the cause is upstream: do not patch around it. Document it in
|
||||
two places: a comment on the test wiring in `build.zig` (at the
|
||||
`b.addTest` block, lines 48-69) stating the exact mechanism and the upstream
|
||||
reference, and a paragraph in `AGENTS.md` stating that `failed command` +
|
||||
exit 0 after a full pass is this known artifact and that any *other*
|
||||
failure text is real.
|
||||
1. Reproduce: run the cached test binary directly with and without `--listen=-`. Capture the abort backtrace (`ulimit -c` / gdb / the zig `--debug-...` runner flags — whatever this host offers).
|
||||
2. Bisect the mbedTLS linkage: `addMbedtlsThreadingMacros` (build.zig), the shim (`src/platform/mbedtls_shim.c`), and the two `extern fn` link checks in `src/tests.zig:121-123` are the suspects. A minimal reproducer (empty test file + mbedTLS link + `--listen`) decides whether the abort is ours or an upstream zig 0.16 runner defect.
|
||||
3. Outcome A — the cause is ours: fix it. Acceptance is a `zig build test` run whose output contains no `failed command` line.
|
||||
4. Outcome B — the cause is upstream: do not patch around it. Document it in two places: a comment on the test wiring in `build.zig` (at the `b.addTest` block, lines 48-69) stating the exact mechanism and the upstream reference, and a paragraph in `AGENTS.md` stating that `failed command` + exit 0 after a full pass is this known artifact and that any *other* failure text is real.
|
||||
|
||||
Either way: no check is loosened, no output is filtered, and the spec is
|
||||
updated afterward to record which outcome held (per the spec-update rule).
|
||||
Either way: no check is loosened, no output is filtered, and the spec is updated afterward to record which outcome held (per the spec-update rule).
|
||||
|
||||
**Outcome recorded (implementation): B — upstream, and this ruling's own
|
||||
diagnosis was wrong.** The label is not an abort and has nothing to do with
|
||||
mbedTLS. In Zig 0.16.0, `std/Build/Step/Run.zig:1540` sets
|
||||
`result_failed_command` unconditionally on every spawn and nothing clears it
|
||||
on success; `compiler/build_runner.zig:1381` prints a step's diagnostics
|
||||
whenever the child wrote to stderr, "no matter the result", and `:1515` then
|
||||
emits the `failed command:` label because the field is non-null. Any Run
|
||||
step that succeeds while writing one byte to stderr gets the label; our
|
||||
warning-path tests log through the real sink. Minimal reproducer: one
|
||||
passing test containing a `std.debug.print` shows the label; the same test
|
||||
without the print shows nothing. The old "exits 134 under `--listen=-`"
|
||||
observation was a manual-invocation artifact: the IPC runner panics with
|
||||
`EndOfStream` (test_runner.zig:88) reading a closed stdin when no build
|
||||
runner sits on the other end. Documented in the `b.addTest` comment in
|
||||
build.zig and in AGENTS.md ("Reading `zig build test` output"). No matching
|
||||
upstream issue found; the 0.16.0 source lines are the reference.
|
||||
**Outcome recorded (implementation): B — upstream, and this ruling's own diagnosis was wrong.** The label is not an abort and has nothing to do with mbedTLS. In Zig 0.16.0, `std/Build/Step/Run.zig:1540` sets `result_failed_command` unconditionally on every spawn and nothing clears it on success; `compiler/build_runner.zig:1381` prints a step's diagnostics whenever the child wrote to stderr, "no matter the result", and `:1515` then emits the `failed command:` label because the field is non-null. Any Run step that succeeds while writing one byte to stderr gets the label; our warning-path tests log through the real sink. Minimal reproducer: one passing test containing a `std.debug.print` shows the label; the same test without the print shows nothing. The old "exits 134 under `--listen=-`" observation was a manual-invocation artifact: the IPC runner panics with `EndOfStream` (test_runner.zig:88) reading a closed stdin when no build runner sits on the other end. Documented in the `b.addTest` comment in build.zig and in AGENTS.md ("Reading `zig build test` output"). No matching upstream issue found; the 0.16.0 source lines are the reference.
|
||||
|
||||
### 4. The test import list gets a completeness guard
|
||||
|
||||
`src/tests.zig:3-119` is a hand-maintained `comptime` block of 115
|
||||
`_ = @import("...");` lines. Zig collects tests only from the root module; a
|
||||
forgotten import silently drops a file's tests (verified empirically). The
|
||||
list stays hand-written — generation via a staged copy of `src/` would point
|
||||
diagnostics at cache paths — but it becomes complete by construction:
|
||||
`src/tests.zig:3-119` is a hand-maintained `comptime` block of 115 `_ = @import("...");` lines. Zig collects tests only from the root module; a forgotten import silently drops a file's tests (verified empirically). The list stays hand-written — generation via a staged copy of `src/` would point diagnostics at cache paths — but it becomes complete by construction:
|
||||
|
||||
In `build.zig`, at configure time, walk `src/` recursively. For every `*.zig`
|
||||
file except `src/tests.zig` itself, require that `src/tests.zig` contains a
|
||||
**line** which, after whitespace trim, is exactly `_ = @import("<path>");`
|
||||
where `<path>` is the file's path relative to `src/`. A line match, not a
|
||||
substring search: a commented-out import (`// _ = @import(...)`) trims to a
|
||||
line that starts with `//` and does not match, and a path that is a prefix of
|
||||
another (`db.zig` vs `db2.zig`) cannot false-match because the full line is
|
||||
compared. Duplicate matching lines are an error too (they hide a botched
|
||||
merge). On the first missing file, fail the configure with an error that
|
||||
names it:
|
||||
In `build.zig`, at configure time, walk `src/` recursively. For every `*.zig` file except `src/tests.zig` itself, require that `src/tests.zig` contains a **line** which, after whitespace trim, is exactly `_ = @import("<path>");` where `<path>` is the file's path relative to `src/`. A line match, not a substring search: a commented-out import (`// _ = @import(...)`) trims to a line that starts with `//` and does not match, and a path that is a prefix of another (`db.zig` vs `db2.zig`) cannot false-match because the full line is compared. Duplicate matching lines are an error too (they hide a botched merge). On the first missing file, fail the configure with an error that names it:
|
||||
|
||||
```
|
||||
src/tests.zig is missing `_ = @import("filter/new_module.zig");`
|
||||
```
|
||||
|
||||
No allowlist. A file with no tests still gets imported — the import is free,
|
||||
and the rule stays exceptionless. The walk runs on the host in every
|
||||
invocation, including the aarch64 configure.
|
||||
No allowlist. A file with no tests still gets imported — the import is free, and the rule stays exceptionless. The walk runs on the host in every invocation, including the aarch64 configure.
|
||||
|
||||
### 5. `web/dist` gets a freshness stamp
|
||||
|
||||
A stale `web/dist` has already shipped a crashing settings page once
|
||||
(`docs/explanation/performance-and-testing.md:154-163` admits the hole).
|
||||
Mechanism — one implementation, in JavaScript, because the frontend CI job has
|
||||
Node and no Zig:
|
||||
A stale `web/dist` has already shipped a crashing settings page once (`docs/explanation/performance-and-testing.md:154-163` admits the hole). Mechanism — one implementation, in JavaScript, because the frontend CI job has Node and no Zig:
|
||||
|
||||
- New file `web/scripts/stamp-dist.mjs`. Two modes:
|
||||
- default (write): hash the input set, write the hex digest to
|
||||
`web/dist/.src-hash`.
|
||||
- `--check`: recompute, compare with `web/dist/.src-hash`, exit 1 with
|
||||
`web/dist is stale: rebuild the frontend (npm run build)` on mismatch or
|
||||
missing stamp.
|
||||
- The script resolves every path (input files and `web/dist/.src-hash`) from
|
||||
its own location via `import.meta.url`, never from `process.cwd()`: write
|
||||
mode runs from `web/` (npm script) and check mode runs from the repo root
|
||||
(build.zig system command), and both must hash the same set. The
|
||||
acceptance run exercises both working directories.
|
||||
- Input set, sorted by path, SHA-256 over `path ++ "\x00" ++ contents ++
|
||||
"\x00"` per file: every file under `web/src/` and `web/public/`, plus
|
||||
`web/index.html`, `web/package.json`, `web/package-lock.json`,
|
||||
`web/vite.config.ts`, `web/tsconfig.json`, `web/tsconfig.app.json`,
|
||||
`web/tsconfig.node.json`. Node's `crypto` module; no new dependency.
|
||||
- `web/package.json`: `"build"` becomes `"vite build && node
|
||||
scripts/stamp-dist.mjs"`.
|
||||
- `build.zig`: when the resolved `-Dweb-dist` value is exactly `web/dist`, the
|
||||
asset pipeline (`webAssetsIndex`) gains a dependency on
|
||||
`b.addSystemCommand(&.{ "node", "web/scripts/stamp-dist.mjs", "--check" })`.
|
||||
The default `web/dist-placeholder` path and any other explicit path skip the
|
||||
check; the `web-dist` option description says so.
|
||||
- default (write): hash the input set, write the hex digest to `web/dist/.src-hash`.
|
||||
- `--check`: recompute, compare with `web/dist/.src-hash`, exit 1 with `web/dist is stale: rebuild the frontend (npm run build)` on mismatch or missing stamp.
|
||||
- The script resolves every path (input files and `web/dist/.src-hash`) from its own location via `import.meta.url`, never from `process.cwd()`: write mode runs from `web/` (npm script) and check mode runs from the repo root (build.zig system command), and both must hash the same set. The acceptance run exercises both working directories.
|
||||
- Input set, sorted by path, SHA-256 over `path ++ "\x00" ++ contents ++ "\x00"` per file: every file under `web/src/` and `web/public/`, plus `web/index.html`, `web/package.json`, `web/package-lock.json`, `web/vite.config.ts`, `web/tsconfig.json`, `web/tsconfig.app.json`, `web/tsconfig.node.json`. Node's `crypto` module; no new dependency.
|
||||
- `web/package.json`: `"build"` becomes `"vite build && node scripts/stamp-dist.mjs"`.
|
||||
- `build.zig`: when the resolved `-Dweb-dist` value is exactly `web/dist`, the asset pipeline (`webAssetsIndex`) gains a dependency on `b.addSystemCommand(&.{ "node", "web/scripts/stamp-dist.mjs", "--check" })`. The default `web/dist-placeholder` path and any other explicit path skip the check; the `web-dist` option description says so.
|
||||
|
||||
### 6. Three new fuzz surfaces, corpus replay only
|
||||
|
||||
All targets follow the existing registration shape (one `test "fuzz ..."` +
|
||||
one `fn target(ctx, smith: *Smith)`), attach to `test_step`, and run as corpus
|
||||
replay under plain `zig build test` — once per corpus entry plus once on empty
|
||||
input. CI never passes `--fuzz` (blocked by upstream zig 0.16 defects; the
|
||||
audit confirmed corpus replay is the sanctioned CI smoke).
|
||||
All targets follow the existing registration shape (one `test "fuzz ..."` + one `fn target(ctx, smith: *Smith)`), attach to `test_step`, and run as corpus replay under plain `zig build test` — once per corpus entry plus once on empty input. CI never passes `--fuzz` (blocked by upstream zig 0.16 defects; the audit confirmed corpus replay is the sanctioned CI smoke).
|
||||
|
||||
**6a. `stripEcs`** — the only attacker-facing packet-rewriting entry point,
|
||||
absent from the fuzz targets. Add a target to `tests/fuzz/dns_fuzz.zig`.
|
||||
Derive input exactly as `parseTarget` (lines 57-89) already does: it ends with
|
||||
a validated `Packet` and its `OptRecord`, which are `stripEcs`'s second and
|
||||
third parameters. Respect the three entry assertions
|
||||
(`src/dns/edns.zig:196-204`): pass the same `bytes` the packet was parsed
|
||||
from, and a separate stack `out` buffer — an assertion trip from a violated
|
||||
precondition is not a finding. Assert on `.rewritten` output: it re-parses,
|
||||
`findOptRecord` + `parseOpt` succeed, no ECS option (code 8) remains, and
|
||||
header counts survive. No build.zig change — the module exists.
|
||||
**6a. `stripEcs`** — the only attacker-facing packet-rewriting entry point, absent from the fuzz targets. Add a target to `tests/fuzz/dns_fuzz.zig`. Derive input exactly as `parseTarget` (lines 57-89) already does: it ends with a validated `Packet` and its `OptRecord`, which are `stripEcs`'s second and third parameters. Respect the three entry assertions (`src/dns/edns.zig:196-204`): pass the same `bytes` the packet was parsed from, and a separate stack `out` buffer — an assertion trip from a violated precondition is not a finding. Assert on `.rewritten` output: it re-parses, `findOptRecord` + `parseOpt` succeed, no ECS option (code 8) remains, and header counts survive. No build.zig change — the module exists.
|
||||
|
||||
**6b. `compiler.compile`** — the streaming
|
||||
`takeDelimiter`/`StreamTooLong`/discard loop (`src/filter/compiler.zig:64-78`)
|
||||
is never fuzzed, and its `error.EndOfStream`-during-discard arm (line 73) has
|
||||
no test at all. `compiler.zig` imports `../dns/`, so a module rooted under
|
||||
`src/filter/` fails with `ImportOutsideModulePath`. Use the staged-copy
|
||||
aggregator pattern from `bench_core` (`build.zig:112-135`) — `bench_core`
|
||||
already exposes `compiler` at line 124; reusing the same staged tree is
|
||||
allowed. New file `tests/fuzz/compiler_fuzz.zig`, module import name `core`.
|
||||
The target feeds `compile()` via `std.Io.Reader.fixed` over smith bytes with a
|
||||
deliberately small reader buffer, into discarding writers, and asserts it
|
||||
returns without panic and that `counts` are internally consistent. Corpus
|
||||
(inline, `sliceInput` idiom): a line longer than `max_line_len` (4096) with
|
||||
and without a trailing `\n`, so both the discard path and the
|
||||
EndOfStream-during-discard arm replay. Add a plain unit test for the line-73
|
||||
arm alongside the existing compiler tests.
|
||||
**6b. `compiler.compile`** — the streaming `takeDelimiter`/`StreamTooLong`/discard loop (`src/filter/compiler.zig:64-78`) is never fuzzed, and its `error.EndOfStream`-during-discard arm (line 73) has no test at all. `compiler.zig` imports `../dns/`, so a module rooted under `src/filter/` fails with `ImportOutsideModulePath`. Use the staged-copy aggregator pattern from `bench_core` (`build.zig:112-135`) — `bench_core` already exposes `compiler` at line 124; reusing the same staged tree is allowed. New file `tests/fuzz/compiler_fuzz.zig`, module import name `core`. The target feeds `compile()` via `std.Io.Reader.fixed` over smith bytes with a deliberately small reader buffer, into discarding writers, and asserts it returns without panic and that `counts` are internally consistent. Corpus (inline, `sliceInput` idiom): a line longer than `max_line_len` (4096) with and without a trailing `\n`, so both the discard path and the EndOfStream-during-discard arm replay. Add a plain unit test for the line-73 arm alongside the existing compiler tests.
|
||||
|
||||
**6c. `http_util`** — the third untrusted-byte family, unfuzzed.
|
||||
`src/web/http_util.zig` imports only `std`, so the fuzz module roots at a new
|
||||
file `tests/fuzz/http_util_fuzz.zig` with `addImport("http_util", <module
|
||||
rooted at src/web/http_util.zig>)` — no aggregator. Targets: `parsePath`,
|
||||
`decodeInPlace` (both `PlusRule` values), `queryValue`. Invariants to assert:
|
||||
split-before-decode and decode-only-shrinks (result length ≤ input length;
|
||||
result is a prefix-aliased slice of the buffer). **Corrected during
|
||||
implementation:** the original phrasing "a decoded segment never gains a `/`"
|
||||
is false — `%2F` legitimately decodes to a literal `/` inside its segment
|
||||
(http_util.zig's own tests assert it), and the fuzz target caught that on its
|
||||
first run. The property that holds is a count: segmentation is decided by the
|
||||
raw bytes, so each decoded segment pairs with its raw `/`-delimited chunk in
|
||||
order, the segment counts match, and each segment is no longer than its
|
||||
chunk. A decode-then-split implementation still fails this.
|
||||
`dnsParam`/`decodeDnsValue` stay
|
||||
private in `doh_server.zig` and stay unfuzzed — recorded, out of scope.
|
||||
**6c. `http_util`** — the third untrusted-byte family, unfuzzed. `src/web/http_util.zig` imports only `std`, so the fuzz module roots at a new file `tests/fuzz/http_util_fuzz.zig` with `addImport("http_util", <module rooted at src/web/http_util.zig>)` — no aggregator. Targets: `parsePath`, `decodeInPlace` (both `PlusRule` values), `queryValue`. Invariants to assert: split-before-decode and decode-only-shrinks (result length ≤ input length; result is a prefix-aliased slice of the buffer). **Corrected during implementation:** the original phrasing "a decoded segment never gains a `/`" is false — `%2F` legitimately decodes to a literal `/` inside its segment (http_util.zig's own tests assert it), and the fuzz target caught that on its first run. The property that holds is a count: segmentation is decided by the raw bytes, so each decoded segment pairs with its raw `/`-delimited chunk in order, the segment counts match, and each segment is no longer than its chunk. A decode-then-split implementation still fails this. `dnsParam`/`decodeDnsValue` stay private in `doh_server.zig` and stay unfuzzed — recorded, out of scope.
|
||||
|
||||
### 7. The multi-read fetch path gets a real fixture route
|
||||
|
||||
The exact seam of 35k-killer 35f2324 is still never driven end to end: every
|
||||
fixture reply is one `request.respond` of a ~130-byte body
|
||||
(`src/filter/filter_integration_test.zig:414`), and the post-fix unit tests
|
||||
pre-buffer readers. Two thresholds matter and they are different: a body over
|
||||
16 KiB (`fetcher.min_transfer_buf`, fetcher.zig:24) forces the fetcher's
|
||||
`pumpBody` loop to iterate; a body over the fixture's 8192-byte write buffer
|
||||
forces the fixture to flush in parts.
|
||||
The exact seam of 35k-killer 35f2324 is still never driven end to end: every fixture reply is one `request.respond` of a ~130-byte body (`src/filter/filter_integration_test.zig:414`), and the post-fix unit tests pre-buffer readers. Two thresholds matter and they are different: a body over 16 KiB (`fetcher.min_transfer_buf`, fetcher.zig:24) forces the fetcher's `pumpBody` loop to iterate; a body over the fixture's 8192-byte write buffer forces the fixture to flush in parts.
|
||||
|
||||
- Add a route `chunked` to the `Route` enum (line 354). Its `respond` arm uses
|
||||
`respondStreaming`, writes a well-formed blocklist body of at least 24 KiB
|
||||
in at least three writes with an explicit `flush()` between each, then ends
|
||||
the stream. `keep_alive = false`, like every other arm (the comment at lines
|
||||
409-411 is load-bearing).
|
||||
- Add one `-Dintegration` test that drives the full `refreshOnce` path through
|
||||
this route and asserts the parsed domain counts.
|
||||
- Prove it can fail: during development, reintroduce the
|
||||
`readSliceShort(self.transfer_buf)` aliasing pattern locally and confirm the
|
||||
new test dies where the old suite stayed green. Record the proof in the
|
||||
session notes; do not commit the revert. **Proof recorded
|
||||
(implementation):** with the aliasing reintroduced, the new test failed
|
||||
(`expected 1500, found 1238`) while all 1378 other tests stayed green. On
|
||||
this body the bug silently dropped 262 domains rather than crashing — the
|
||||
count assertion does the real work, not the crash.
|
||||
- Add a route `chunked` to the `Route` enum (line 354). Its `respond` arm uses `respondStreaming`, writes a well-formed blocklist body of at least 24 KiB in at least three writes with an explicit `flush()` between each, then ends the stream. `keep_alive = false`, like every other arm (the comment at lines 409-411 is load-bearing).
|
||||
- Add one `-Dintegration` test that drives the full `refreshOnce` path through this route and asserts the parsed domain counts.
|
||||
- Prove it can fail: during development, reintroduce the `readSliceShort(self.transfer_buf)` aliasing pattern locally and confirm the new test dies where the old suite stayed green. Record the proof in the session notes; do not commit the revert. **Proof recorded (implementation):** with the aliasing reintroduced, the new test failed (`expected 1500, found 1238`) while all 1378 other tests stayed green. On this body the bug silently dropped 262 domains rather than crashing — the count assertion does the real work, not the crash.
|
||||
|
||||
### 8. The rotation failure paths get tests
|
||||
|
||||
`src/platform/logging.zig` codifies an exactly-one-sink-error contract across
|
||||
four functions (stated at lines 455-460) and its own test section admits the
|
||||
rotation failure paths are uncovered (lines 604-612). Injection seam:
|
||||
`src/platform/logging.zig` codifies an exactly-one-sink-error contract across four functions (stated at lines 455-460) and its own test section admits the rotation failure paths are uncovered (lines 604-612). Injection seam:
|
||||
|
||||
```zig
|
||||
var rotate_fault: enum { none, fail_delete, fail_rename } = .none;
|
||||
```
|
||||
|
||||
file-private, read by `deleteLocked` (line 590) and `renameLocked` (line 597)
|
||||
as their first statement (`if (rotate_fault == .fail_delete) return
|
||||
error.RotateFailed;`), compiled only under `@import("builtin").is_test`. Two
|
||||
new tests, each following the established shape of the failed-open test at
|
||||
lines 883-909 (save and restore `state.*`, hold `std.debug.lockStderr`):
|
||||
file-private, read by `deleteLocked` (line 590) and `renameLocked` (line 597) as their first statement (`if (rotate_fault == .fail_delete) return error.RotateFailed;`), compiled only under `@import("builtin").is_test`. Two new tests, each following the established shape of the failed-open test at lines 883-909 (save and restore `state.*`, hold `std.debug.lockStderr`):
|
||||
|
||||
- `fail_delete`: `prepareFileLocked` returns false, `sink_errors` rose by
|
||||
exactly one, `rotate_pending` stays set, the file stays closed.
|
||||
- `fail_delete`: `prepareFileLocked` returns false, `sink_errors` rose by exactly one, `rotate_pending` stays set, the file stays closed.
|
||||
- `fail_rename`: same assertions through the rename step.
|
||||
|
||||
### 9. The CLI drift guard derives its needles
|
||||
|
||||
`src/docs_drift_test.zig:51` hardcodes the subcommand list; its two sibling
|
||||
guards derive theirs (`routes.table`, `model.toSettings`). The source of truth
|
||||
today is three parallel copies: the `if (eql(...))` parse chain
|
||||
(`cli.zig:91-102`), the `main.zig:69-78` dispatch switch, and `usage_text`
|
||||
(cli.zig:334). Note the spelling trap: tags are `export_`/`import_`, argv
|
||||
strings are `export`/`import`.
|
||||
`src/docs_drift_test.zig:51` hardcodes the subcommand list; its two sibling guards derive theirs (`routes.table`, `model.toSettings`). The source of truth today is three parallel copies: the `if (eql(...))` parse chain (`cli.zig:91-102`), the `main.zig:69-78` dispatch switch, and `usage_text` (cli.zig:334). Note the spelling trap: tags are `export_`/`import_`, argv strings are `export`/`import`.
|
||||
|
||||
- Add to `cli.zig`:
|
||||
|
||||
@@ -281,61 +119,41 @@ strings are `export`/`import`.
|
||||
};
|
||||
```
|
||||
|
||||
plus a comptime assert that `command_names.len ==
|
||||
@typeInfo(Command).@"union".fields.len`, so a new tag without a table entry
|
||||
fails the compile.
|
||||
- `parseArgs` matches the command word by iterating `command_names` (the
|
||||
per-command argument parsing that follows stays as-is).
|
||||
- `docs_drift_test.zig` iterates `cli.command_names` for its `## `{s}``
|
||||
needles; the hardcoded list is deleted.
|
||||
- New test in `cli.zig`: every `command_names[i].name` appears in
|
||||
`usage_text`.
|
||||
- `main.zig`'s dispatch switch stays — it is exhaustive over the union and the
|
||||
compiler already guards it.
|
||||
plus a comptime assert that `command_names.len == @typeInfo(Command).@"union".fields.len`, so a new tag without a table entry fails the compile.
|
||||
- `parseArgs` matches the command word by iterating `command_names` (the per-command argument parsing that follows stays as-is).
|
||||
- `docs_drift_test.zig` iterates `cli.command_names` for its `## `{s}`` needles; the hardcoded list is deleted.
|
||||
- New test in `cli.zig`: every `command_names[i].name` appears in `usage_text`.
|
||||
- `main.zig`'s dispatch switch stays — it is exhaustive over the union and the compiler already guards it.
|
||||
|
||||
## Sessions
|
||||
|
||||
S1-S4 run in parallel; no two sessions write the same file. The S1/S3
|
||||
interface is fixed here so neither blocks: S3 writes the fuzz source files
|
||||
named in ruling 6; S1 wires them in `build.zig` with the module shapes named
|
||||
there (6b gets `core` via the staged-copy pattern; 6c gets `http_util` rooted
|
||||
at `src/web/http_util.zig`); both artifacts attach to `test_step` exactly like
|
||||
`fuzz_tests` at build.zig:92.
|
||||
S1-S4 run in parallel; no two sessions write the same file. The S1/S3 interface is fixed here so neither blocks: S3 writes the fuzz source files named in ruling 6; S1 wires them in `build.zig` with the module shapes named there (6b gets `core` via the staged-copy pattern; 6c gets `http_util` rooted at `src/web/http_util.zig`); both artifacts attach to `test_step` exactly like `fuzz_tests` at build.zig:92.
|
||||
|
||||
### Session S1: build system and stamp
|
||||
|
||||
Owns `build.zig`, `AGENTS.md`, `web/scripts/stamp-dist.mjs`,
|
||||
`web/package.json`. Rulings 3, 4, 5, and the wiring half of 6.
|
||||
Owns `build.zig`, `AGENTS.md`, `web/scripts/stamp-dist.mjs`, `web/package.json`. Rulings 3, 4, 5, and the wiring half of 6.
|
||||
|
||||
### Session S2: workflows
|
||||
|
||||
Owns `.gitea/workflows/ci.yml`, `.gitea/workflows/live-tls.yml`. Rulings 1
|
||||
(file edit only), 2.
|
||||
Owns `.gitea/workflows/ci.yml`, `.gitea/workflows/live-tls.yml`. Rulings 1 (file edit only), 2.
|
||||
|
||||
### Session S3: fuzz targets and fixture
|
||||
|
||||
Owns `tests/fuzz/dns_fuzz.zig`, `tests/fuzz/compiler_fuzz.zig` (new),
|
||||
`tests/fuzz/http_util_fuzz.zig` (new),
|
||||
`src/filter/filter_integration_test.zig`, and the compiler unit-test addition
|
||||
in `src/filter/compiler.zig`. Rulings 6 (target half), 7.
|
||||
Owns `tests/fuzz/dns_fuzz.zig`, `tests/fuzz/compiler_fuzz.zig` (new), `tests/fuzz/http_util_fuzz.zig` (new), `src/filter/filter_integration_test.zig`, and the compiler unit-test addition in `src/filter/compiler.zig`. Rulings 6 (target half), 7.
|
||||
|
||||
### Session S4: guards and seams
|
||||
|
||||
Owns `src/cli.zig`, `src/docs_drift_test.zig`, `src/platform/logging.zig`.
|
||||
Rulings 8, 9.
|
||||
Owns `src/cli.zig`, `src/docs_drift_test.zig`, `src/platform/logging.zig`. Rulings 8, 9.
|
||||
|
||||
### Orchestrator
|
||||
|
||||
Ruling 1's remote operations (delete `origin/main`, flip the Gitea default,
|
||||
confirm a run starts), the ruling-3 outcome recorded back into this spec, and
|
||||
striking the closed Theme-1 findings in `TECH_DEBT.md`.
|
||||
Ruling 1's remote operations (delete `origin/main`, flip the Gitea default, confirm a run starts), the ruling-3 outcome recorded back into this spec, and striking the closed Theme-1 findings in `TECH_DEBT.md`.
|
||||
|
||||
## Module layout
|
||||
|
||||
New files:
|
||||
|
||||
- `web/scripts/stamp-dist.mjs` — dist freshness stamp, write and `--check`
|
||||
modes.
|
||||
- `web/scripts/stamp-dist.mjs` — dist freshness stamp, write and `--check` modes.
|
||||
- `tests/fuzz/compiler_fuzz.zig` — compiler streaming-loop target.
|
||||
- `tests/fuzz/http_util_fuzz.zig` — HTTP parser targets.
|
||||
|
||||
@@ -377,13 +195,10 @@ New files:
|
||||
|
||||
## Anti-requirements
|
||||
|
||||
- No `gates.yml`, no `release.yml`, no `dist`/`verify-dist` steps — that is
|
||||
milestone 14, which rebases on this milestone.
|
||||
- No `gates.yml`, no `release.yml`, no `dist`/`verify-dist` steps — that is milestone 14, which rebases on this milestone.
|
||||
- No `--fuzz` anywhere in CI.
|
||||
- No fuzzing or relocation of `dnsParam`/`decodeDnsValue`.
|
||||
- No generated `tests.zig`; the hand list stays, the guard makes it complete.
|
||||
- No frontend changes beyond `web/scripts/stamp-dist.mjs` and the one-line
|
||||
`build` script edit.
|
||||
- No filtering, wrapping, or suppression of test-runner output to hide the
|
||||
`failed command` line.
|
||||
- No frontend changes beyond `web/scripts/stamp-dist.mjs` and the one-line `build` script edit.
|
||||
- No filtering, wrapping, or suppression of test-runner output to hide the `failed command` line.
|
||||
- No new dependencies, Zig or npm.
|
||||
|
||||
+64
-299
@@ -1,402 +1,169 @@
|
||||
# Milestone 16: contained behavioral fixes
|
||||
|
||||
Goal: fix the verified bugs and silent failures from `TECH_DEBT.md` themes 3,
|
||||
5, 6 and 7 — each one localized, each with a test that fails before and
|
||||
passes after. No refactors: the duplication work is milestone 18, and a
|
||||
refactor that must simultaneously fix behavior is how mirrored copies diverge
|
||||
further.
|
||||
Goal: fix the verified bugs and silent failures from `TECH_DEBT.md` themes 3, 5, 6 and 7 — each one localized, each with a test that fails before and passes after. No refactors: the duplication work is milestone 18, and a refactor that must simultaneously fix behavior is how mirrored copies diverge further.
|
||||
|
||||
**PROVISIONAL.** Written before milestone 15 was built. m15 touches
|
||||
`build.zig`, workflows, fuzz files, `filter_integration_test.zig`, `cli.zig`,
|
||||
`docs_drift_test.zig` and `logging.zig` — re-verify line references in those
|
||||
files before starting.
|
||||
**PROVISIONAL.** Written before milestone 15 was built. m15 touches `build.zig`, workflows, fuzz files, `filter_integration_test.zig`, `cli.zig`, `docs_drift_test.zig` and `logging.zig` — re-verify line references in those files before starting.
|
||||
|
||||
## Rulings (binding)
|
||||
|
||||
### 1. `loadSource` propagates cancellation
|
||||
|
||||
`src/filter/manager.zig:541-544` and `:547-550`: two identical catch sites
|
||||
fold `error.Canceled` into `loadFailure(...)`, recording a bogus "Canceled"
|
||||
load-failed status, consuming the one-shot cancellation, and publishing a
|
||||
snapshot with the source excluded. Ten other catch sites in the file
|
||||
propagate it correctly. Add `if (err == error.Canceled) return
|
||||
error.Canceled;` beside the existing OutOfMemory arm at both sites —
|
||||
`Manager.Error` already has the member. Unit test: a reader that fails with
|
||||
Canceled during `loadSource` makes the whole call return Canceled and writes
|
||||
no status.
|
||||
`src/filter/manager.zig:541-544` and `:547-550`: two identical catch sites fold `error.Canceled` into `loadFailure(...)`, recording a bogus "Canceled" load-failed status, consuming the one-shot cancellation, and publishing a snapshot with the source excluded. Ten other catch sites in the file propagate it correctly. Add `if (err == error.Canceled) return error.Canceled;` beside the existing OutOfMemory arm at both sites — `Manager.Error` already has the member. Unit test: a reader that fails with Canceled during `loadSource` makes the whole call return Canceled and writes no status.
|
||||
|
||||
### 2. `commitStatus` never drops an outcome silently
|
||||
|
||||
`src/filter/manager.zig:1237-1250`: the id-match loop falls through with no
|
||||
log when a source has no status entry (reachable via a startupPass /
|
||||
web-insert race; milestone-5 policy says every non-ok state is recorded and
|
||||
surfaced). Add a `log.warn` after the loop naming the source id. Unit test:
|
||||
committing a status for an unknown id emits the warning and does not crash.
|
||||
`src/filter/manager.zig:1237-1250`: the id-match loop falls through with no log when a source has no status entry (reachable via a startupPass / web-insert race; milestone-5 policy says every non-ok state is recorded and surfaced). Add a `log.warn` after the loop naming the source id. Unit test: committing a status for an unknown id emits the warning and does not crash.
|
||||
|
||||
### 3. Downloads and compiles leave the writer lock
|
||||
|
||||
`src/filter/manager.zig:609-624`: `refreshAll` holds `writer_lock` across
|
||||
every download (300 s budget each, `app.zig:85`) and every compile; every
|
||||
web mutation ends in `Manager.reload` on the same lock, so an unrelated rule
|
||||
save parks uncancelably — on a request path with no timeout, occupying one
|
||||
of 64 web slots — until the pass ends. The download-to-tmp flow already
|
||||
exists (`download` writes `<id>.raw.tmp` via `compiledName`, with
|
||||
`deleteQuietly` defers at :646-648); the lock is just taken too early.
|
||||
`src/filter/manager.zig:609-624`: `refreshAll` holds `writer_lock` across every download (300 s budget each, `app.zig:85`) and every compile; every web mutation ends in `Manager.reload` on the same lock, so an unrelated rule save parks uncancelably — on a request path with no timeout, occupying one of 64 web slots — until the pass ends. The download-to-tmp flow already exists (`download` writes `<id>.raw.tmp` via `compiledName`, with `deleteQuietly` defers at :646-648); the lock is just taken too early.
|
||||
|
||||
Restructure:
|
||||
|
||||
- New `refresh_lock: std.Io.Mutex` on Manager, serializing refresh passes
|
||||
against each other only.
|
||||
- New `refresh_lock: std.Io.Mutex` on Manager, serializing refresh passes against each other only.
|
||||
- Split `refreshOne` (:626-729): the fetch/detect/compile stages (:650,
|
||||
:659, :668) run under `refresh_lock` alone; the publish stage (:711), the
|
||||
status commit and the final `reloadLocked` run under `writer_lock`.
|
||||
- `refreshAll` takes `refresh_lock` for the pass, and `writer_lock` only
|
||||
per-source around publish plus once for the final reload. `refreshSource`
|
||||
(:582-586) follows the same split.
|
||||
- The lock-ordering rule, documented on both mutex fields: `refresh_lock` is
|
||||
never acquired while holding `writer_lock`.
|
||||
- `refresh_lock` serializes blocklist-**directory maintenance** too, not
|
||||
just refresh passes: `pruneOrphans` (:1040, invariant comment at
|
||||
- `refreshAll` takes `refresh_lock` for the pass, and `writer_lock` only per-source around publish plus once for the final reload. `refreshSource` (:582-586) follows the same split.
|
||||
- The lock-ordering rule, documented on both mutex fields: `refresh_lock` is never acquired while holding `writer_lock`.
|
||||
- `refresh_lock` serializes blocklist-**directory maintenance** too, not just refresh passes: `pruneOrphans` (:1040, invariant comment at
|
||||
:1130-1136) today assumes every temp-file writer holds `writer_lock`;
|
||||
once downloads write `.raw.tmp`/`.list.tmp`/`.wild.tmp` under
|
||||
`refresh_lock` alone, a concurrent source deletion (blocklists.zig
|
||||
delete → reload → pruneFiles) could sweep an in-flight refresh's temp
|
||||
files. `pruneOrphans` therefore takes `refresh_lock` first (then
|
||||
`writer_lock` if it needs it — same order as everywhere), and its
|
||||
invariant comment is rewritten. Regression test: a source deletion
|
||||
during a stalled refresh must not delete the stalled refresh's temp
|
||||
files.
|
||||
once downloads write `.raw.tmp`/`.list.tmp`/`.wild.tmp` under `refresh_lock` alone, a concurrent source deletion (blocklists.zig delete → reload → pruneFiles) could sweep an in-flight refresh's temp files. `pruneOrphans` therefore takes `refresh_lock` first (then `writer_lock` if it needs it — same order as everywhere), and its invariant comment is rewritten. Regression test: a source deletion during a stalled refresh must not delete the stalled refresh's temp files.
|
||||
|
||||
Integration test (`filter_integration_test.zig`): with a refresh pass parked
|
||||
on a deliberately stalled fixture route, a concurrent rule mutation
|
||||
completes without waiting for the pass. DNS serving was never affected and
|
||||
stays covered by the existing tests.
|
||||
Integration test (`filter_integration_test.zig`): with a refresh pass parked on a deliberately stalled fixture route, a concurrent rule mutation completes without waiting for the pass. DNS serving was never affected and stays covered by the existing tests.
|
||||
|
||||
### 4. Truncated responses are never cached
|
||||
|
||||
`src/cache/dns_cache.zig:106-131`: `classify` reads `rcode` and `ancount`
|
||||
and never `flags.tc`, so a TC=1 answer from a misbehaving upstream is cached
|
||||
for up to `max_ttl_seconds` (86 400 s) and re-served — TC bit intact, even
|
||||
over TCP, which can loop retrying clients (RFC 2181 §9). Add `if
|
||||
(p.header.flags.tc) return null;` (`flags.tc` is `src/dns/header.zig:20`).
|
||||
`src/cache/dns_cache.zig:106-131`: `classify` reads `rcode` and `ancount` and never `flags.tc`, so a TC=1 answer from a misbehaving upstream is cached for up to `max_ttl_seconds` (86 400 s) and re-served — TC bit intact, even over TCP, which can loop retrying clients (RFC 2181 §9). Add `if (p.header.flags.tc) return null;` (`flags.tc` is `src/dns/header.zig:20`).
|
||||
|
||||
`transport.validateResponse` is deliberately untouched: rejecting TC there
|
||||
would change failover semantics for every exchange, and the forward client
|
||||
legitimately reads TC for its UDP-to-TCP retry (`forward_client.zig:178`).
|
||||
Record: considered, rejected. Unit test beside the existing classify tests:
|
||||
a TC=1 NOERROR response classifies as null.
|
||||
`transport.validateResponse` is deliberately untouched: rejecting TC there would change failover semantics for every exchange, and the forward client legitimately reads TC for its UDP-to-TCP retry (`forward_client.zig:178`). Record: considered, rejected. Unit test beside the existing classify tests: a TC=1 NOERROR response classifies as null.
|
||||
|
||||
### 5. The format sniffer stops eating hosts files with `##` banners
|
||||
|
||||
`src/filter/parsers.zig:54-73`: `hasAbpMarker` runs before `isComment`, and
|
||||
`isElementHiding` (:86-93) matches `##` (and `#@#`, `#?#`, `#$#`, `#%#`)
|
||||
unanchored with no comment guard — unlike the `$` branch two lines later,
|
||||
which is guarded. A hosts file with a `##` banner parses whole as ABP:
|
||||
`0.0.0.0` enters the domain set and hosts lines with inline URL comments are
|
||||
silently dropped (real blocking loss, reproduced with the URLhaus banner
|
||||
style). The milestone-5 as-built note (specs/milestone-5.md:1677-1678)
|
||||
already *claims* the separators are matched anchored — the code never was.
|
||||
Guarding the branch with `!isComment` is a circular no-op (`isComment`
|
||||
consults `isElementHiding`); do not attempt it.
|
||||
`src/filter/parsers.zig:54-73`: `hasAbpMarker` runs before `isComment`, and `isElementHiding` (:86-93) matches `##` (and `#@#`, `#?#`, `#$#`, `#%#`) unanchored with no comment guard — unlike the `$` branch two lines later, which is guarded. A hosts file with a `##` banner parses whole as ABP: `0.0.0.0` enters the domain set and hosts lines with inline URL comments are silently dropped (real blocking loss, reproduced with the URLhaus banner style). The milestone-5 as-built note (specs/milestone-5.md:1677-1678) already *claims* the separators are matched anchored — the code never was. Guarding the branch with `!isComment` is a circular no-op (`isComment` consults `isElementHiding`); do not attempt it.
|
||||
|
||||
Fix — a positional predicate in `isElementHiding`: a separator at offset `p`
|
||||
counts as element hiding only when
|
||||
Fix — a positional predicate in `isElementHiding`: a separator at offset `p` counts as element hiding only when
|
||||
|
||||
- `p == 0` and the character after the separator is not whitespace, `#`, or
|
||||
end of line (a generic rule `##.ad` counts; a banner `## Title`, `####`,
|
||||
or bare `##` does not), or
|
||||
- `p > 0` and `line[p - 1]` is not whitespace and not `#` (a rule
|
||||
`example.com##.ad` counts; prose `see ## below` does not).
|
||||
- `p == 0` and the character after the separator is not whitespace, `#`, or end of line (a generic rule `##.ad` counts; a banner `## Title`, `####`, or bare `##` does not), or
|
||||
- `p > 0` and `line[p - 1]` is not whitespace and not `#` (a rule `example.com##.ad` counts; prose `see ## below` does not).
|
||||
|
||||
Required test cases, in `parsers.zig`: the URLhaus banner style sniffs as
|
||||
hosts; `##.ad-banner` sniffs as ABP; `example.com##.ad` sniffs as ABP;
|
||||
`#@#exception` after a domain sniffs as ABP; a `#`-initial line containing
|
||||
`##` sniffs as a comment. The fuzz target (`blocklist_fuzz.zig`) already
|
||||
covers `detectFormat` for crashes.
|
||||
Required test cases, in `parsers.zig`: the URLhaus banner style sniffs as hosts; `##.ad-banner` sniffs as ABP; `example.com##.ad` sniffs as ABP; `#@#exception` after a domain sniffs as ABP; a `#`-initial line containing `##` sniffs as a comment. The fuzz target (`blocklist_fuzz.zig`) already covers `detectFormat` for crashes.
|
||||
|
||||
### 6. VACUUM respects the disk monitor
|
||||
|
||||
`src/storage/retention.zig`: `runOnce` (:78) does prune → checkpoint →
|
||||
(every 7th pass, :94-95) VACUUM, and takes no monitor — the word does not
|
||||
appear in the file — while its two sibling tasks gate on
|
||||
`disk_monitor.writesAllowed()` (logger.zig:386 as a parameter,
|
||||
manager.zig:1058 as a field). VACUUM is the most expensive write the
|
||||
program makes and fires unconditionally on the exact filesystem the monitor
|
||||
watches, then fails SQLITE_FULL with no retry for seven daily passes.
|
||||
`src/storage/retention.zig`: `runOnce` (:78) does prune → checkpoint → (every 7th pass, :94-95) VACUUM, and takes no monitor — the word does not appear in the file — while its two sibling tasks gate on `disk_monitor.writesAllowed()` (logger.zig:386 as a parameter, manager.zig:1058 as a field). VACUUM is the most expensive write the program makes and fires unconditionally on the exact filesystem the monitor watches, then fails SQLITE_FULL with no retry for seven daily passes.
|
||||
|
||||
`runOnce` and `run` gain `monitor: ?*disk_monitor.Monitor` (the logger's
|
||||
parameter shape). Only the VACUUM step gates: prune and checkpoint keep
|
||||
running — they free space. A skipped vacuum increments a new
|
||||
`Stats.vacuums_gated` and retries on the *next* pass (drop the modulo-only
|
||||
trigger for a `passes_since_vacuum >= vacuum_every_passes` counter that
|
||||
resets on success). `app.zig:561` passes the monitor like :560 does for the
|
||||
logger. Tests: extend the cadence tests (:223) with a gated pass — the
|
||||
vacuum is skipped, counted, and runs on the next allowed pass.
|
||||
`runOnce` and `run` gain `monitor: ?*disk_monitor.Monitor` (the logger's parameter shape). Only the VACUUM step gates: prune and checkpoint keep running — they free space. A skipped vacuum increments a new `Stats.vacuums_gated` and retries on the *next* pass (drop the modulo-only trigger for a `passes_since_vacuum >= vacuum_every_passes` counter that resets on success). `app.zig:561` passes the monitor like :560 does for the logger. Tests: extend the cadence tests (:223) with a gated pass — the vacuum is skipped, counted, and runs on the next allowed pass.
|
||||
|
||||
### 7. An oversized Cookie header degrades loudly, and the session survives
|
||||
|
||||
`src/web/server.zig:612-620` (`copyHeader`): a header value over the buffer
|
||||
(cookie buffer = `http_util.max_cookie_len` = 1024, http_util.zig:35)
|
||||
returns `""` with no log — under the documented reverse-proxy-on-shared-
|
||||
domain deployment (foreign cookies riding along), every request silently
|
||||
401s and the operator sees an unexplained login loop.
|
||||
`src/web/server.zig:612-620` (`copyHeader`): a header value over the buffer (cookie buffer = `http_util.max_cookie_len` = 1024, http_util.zig:35) returns `""` with no log — under the documented reverse-proxy-on-shared- domain deployment (foreign cookies riding along), every request silently 401s and the operator sees an unexplained login loop.
|
||||
|
||||
`copyHeader` stays generic. The cookie call site (:510) changes: when the
|
||||
raw header value exceeds the buffer, extract only the session pair by
|
||||
running `http_util.cookieValue` (:200 — it already parses pairs and returns
|
||||
a slice of the original header) with the existing session-cookie name
|
||||
constant against the full value, copy that pair into the buffer as
|
||||
`<name>=<value>`, and emit one `log.debug` with the dropped size (the
|
||||
`.web_server` scope at :55; no secret is logged). If even the pair does not
|
||||
fit, keep the empty result but still log. Integration test
|
||||
(`web_integration_test.zig`): a request with a 2 KiB cookie header whose
|
||||
session pair is valid is authenticated; the same header without the pair
|
||||
401s.
|
||||
`copyHeader` stays generic. The cookie call site (:510) changes: when the raw header value exceeds the buffer, extract only the session pair by running `http_util.cookieValue` (:200 — it already parses pairs and returns a slice of the original header) with the existing session-cookie name constant against the full value, copy that pair into the buffer as `<name>=<value>`, and emit one `log.debug` with the dropped size (the `.web_server` scope at :55; no secret is logged). If even the pair does not fit, keep the empty result but still log. Integration test (`web_integration_test.zig`): a request with a 2 KiB cookie header whose session pair is valid is authenticated; the same header without the pair 401s.
|
||||
|
||||
### 8. Live view detects fatal SSE rejections
|
||||
|
||||
`web/src/features/live/useLiveQueries.ts`: per the WHATWG spec a non-200
|
||||
response fails an EventSource permanently after one error event, so the
|
||||
3-consecutive-errors threshold (:29, :125-133) is unreachable on exactly the
|
||||
429/401 paths it was built for — the UI shows "Reconnecting…" forever and
|
||||
the capped state and session probe are dead. `FakeEventSource` has no
|
||||
readyState, so tests pass — the mocked-network class again.
|
||||
`web/src/features/live/useLiveQueries.ts`: per the WHATWG spec a non-200 response fails an EventSource permanently after one error event, so the 3-consecutive-errors threshold (:29, :125-133) is unreachable on exactly the 429/401 paths it was built for — the UI shows "Reconnecting…" forever and the capped state and session probe are dead. `FakeEventSource` has no readyState, so tests pass — the mocked-network class again.
|
||||
|
||||
- `EventSourceLike` (:9-13) gains `readyState: number`; export `const
|
||||
EVENT_SOURCE_CLOSED = 2`. The browser `EventSource` satisfies it
|
||||
structurally.
|
||||
- In the error handler: `readyState === EVENT_SOURCE_CLOSED` is a permanent
|
||||
failure — close, set `capped`, run the session probe immediately,
|
||||
bypassing the counter. Transient errors (browser auto-retry pending) keep
|
||||
the existing counter path.
|
||||
- `fakeEventSource.ts` gains `readyState` with the real lifecycle
|
||||
(CONNECTING → OPEN on `emit("open")`, CLOSED on `close()` and on
|
||||
`failFatal()`, a new test helper).
|
||||
- New tests: a fatal rejection (single error event, readyState CLOSED)
|
||||
reaches `capped` and calls `probeSession`; a transient error still takes
|
||||
three to trip.
|
||||
- `EventSourceLike` (:9-13) gains `readyState: number`; export `const EVENT_SOURCE_CLOSED = 2`. The browser `EventSource` satisfies it structurally.
|
||||
- In the error handler: `readyState === EVENT_SOURCE_CLOSED` is a permanent failure — close, set `capped`, run the session probe immediately, bypassing the counter. Transient errors (browser auto-retry pending) keep the existing counter path.
|
||||
- `fakeEventSource.ts` gains `readyState` with the real lifecycle (CONNECTING → OPEN on `emit("open")`, CLOSED on `close()` and on `failFatal()`, a new test helper).
|
||||
- New tests: a fatal rejection (single error event, readyState CLOSED) reaches `capped` and calls `probeSession`; a transient error still takes three to trip.
|
||||
|
||||
### 9. A timed-out DoH handshake is a handshake failure
|
||||
|
||||
`src/server/doh_server.zig:311-315` counts a handshake-race timeout as
|
||||
`idle_timeouts`; DoT (:313-317) counts it as `tls_handshake_failures`,
|
||||
which is the recorded spec ruling. Since DoH requests have no other timer,
|
||||
its `idle_timeouts` metric can only ever mean handshake stalls — the same
|
||||
exported name with disjoint semantics per listener. Align DoH's
|
||||
`.timed_out` arm to `tls_handshake_failures`.
|
||||
`src/server/doh_server.zig:311-315` counts a handshake-race timeout as `idle_timeouts`; DoT (:313-317) counts it as `tls_handshake_failures`, which is the recorded spec ruling. Since DoH requests have no other timer, its `idle_timeouts` metric can only ever mean handshake stalls — the same exported name with disjoint semantics per listener. Align DoH's `.timed_out` arm to `tls_handshake_failures`.
|
||||
|
||||
### 10. Idle DoH keep-alive connections are reclaimed
|
||||
|
||||
`src/server/doh_server.zig:70`: `idle_timeout` bounds only the handshake;
|
||||
its own doc comment admits requests have none. DoH stubs hold keep-alives by
|
||||
design, and with no TCP keepalive a vanished peer pins one of 64 slots until
|
||||
restart — while DoT on the same LAN reclaims after 10 s.
|
||||
`src/server/doh_server.zig:70`: `idle_timeout` bounds only the handshake; its own doc comment admits requests have none. DoH stubs hold keep-alives by design, and with no TCP keepalive a vanished peer pins one of 64 slots until restart — while DoT on the same LAN reclaims after 10 s.
|
||||
|
||||
Extend the existing race to `receiveHead` (:333), the wait for the next
|
||||
request on a keep-alive connection, using the DoT out-param precedent
|
||||
(readPrefix's `out_len`): a small wrapper writes the received `Request` (or
|
||||
its error) through a pointer so the raced function stays `anyerror!void`.
|
||||
On `.timed_out`: bump `idle_timeouts` (now truthfully named again after
|
||||
ruling 9) and close the connection. Preserve the existing error triage at
|
||||
Extend the existing race to `receiveHead` (:333), the wait for the next request on a keep-alive connection, using the DoT out-param precedent (readPrefix's `out_len`): a small wrapper writes the received `Request` (or its error) through a pointer so the raced function stays `anyerror!void`. On `.timed_out`: bump `idle_timeouts` (now truthfully named again after ruling 9) and close the connection. Preserve the existing error triage at
|
||||
:334-346 (`HttpConnectionClosing` and `ReadFailed` return uncounted; the
|
||||
three structural errors count `bad_requests`/`connection_errors` as today).
|
||||
The body read and `handleRequest` stay untimed, like the web listener.
|
||||
three structural errors count `bad_requests`/`connection_errors` as today). The body read and `handleRequest` stay untimed, like the web listener.
|
||||
|
||||
New integration-gated test mirroring the DoT idle test
|
||||
(dot_server.zig:1098): a client that completes the handshake and one request
|
||||
then goes quiet is closed after a short idle budget; `idle_timeouts == 1`,
|
||||
`tls_handshake_failures == 0`. DoH currently has no idle test at all.
|
||||
New integration-gated test mirroring the DoT idle test (dot_server.zig:1098): a client that completes the handshake and one request then goes quiet is closed after a short idle budget; `idle_timeouts == 1`, `tls_handshake_failures == 0`. DoH currently has no idle test at all.
|
||||
|
||||
### 11. SSE shutdown does not wait for a heartbeat
|
||||
|
||||
`src/web/sse.zig`: the Hub has no shutdown signal, so graceful drain parks
|
||||
up to 15 s (`heartbeat_interval`, web/handlers/live.zig:32) per idle
|
||||
subscriber — the web `beginShutdown` (web/server.zig:593-605) shuts down
|
||||
sockets but nothing wakes a task inside `Hub.wait`. Measured: the SSE
|
||||
integration test budgets 40 s for exactly this
|
||||
(web_integration_test.zig:74-75).
|
||||
`src/web/sse.zig`: the Hub has no shutdown signal, so graceful drain parks up to 15 s (`heartbeat_interval`, web/handlers/live.zig:32) per idle subscriber — the web `beginShutdown` (web/server.zig:593-605) shuts down sockets but nothing wakes a task inside `Hub.wait`. Measured: the SSE integration test budgets 40 s for exactly this (web_integration_test.zig:74-75).
|
||||
|
||||
- `Wake` (:33) gains `.closed`. Hub gains `closing: bool` and `pub fn
|
||||
close(self: *Hub, io: std.Io) void`: under the mutex, set the flag and
|
||||
`event.set` every active slot. `wait` returns `.closed` immediately when
|
||||
the flag is set.
|
||||
- `Wake` (:33) gains `.closed`. Hub gains `closing: bool` and `pub fn close(self: *Hub, io: std.Io) void`: under the mutex, set the flag and `event.set` every active slot. `wait` returns `.closed` immediately when the flag is set.
|
||||
- `live.zig:114`: `.closed => return`.
|
||||
- `web/server.zig` `deinit` (:349): call the hub's close (via the state's
|
||||
hub pointer) immediately before `beginShutdown` (:360).
|
||||
- Test: a subscriber parked in `wait` returns `.closed` promptly after
|
||||
`close`; the W10 integration teardown no longer stalls (tighten
|
||||
`sse_budget` only if the heartbeat assertion itself allows it).
|
||||
- `web/server.zig` `deinit` (:349): call the hub's close (via the state's hub pointer) immediately before `beginShutdown` (:360).
|
||||
- Test: a subscriber parked in `wait` returns `.closed` promptly after `close`; the W10 integration teardown no longer stalls (tighten `sse_budget` only if the heartbeat assertion itself allows it).
|
||||
|
||||
### 12. The API limiter's sweep runs
|
||||
|
||||
`src/web/api_limiter.zig:210`: `sweep` exists, preallocates
|
||||
`stale_keys` so it never allocates, is tested — and has no production
|
||||
caller; only the DNS limiter got scheduled. Once 4096 distinct addresses
|
||||
have been seen, the table stays full forever and every unknown-address
|
||||
request pays an O(4096) eviction scan under the limiter mutex.
|
||||
`src/web/api_limiter.zig:210`: `sweep` exists, preallocates `stale_keys` so it never allocates, is tested — and has no production caller; only the DNS limiter got scheduled. Once 4096 distinct addresses have been seen, the table stays full forever and every unknown-address request pays an O(4096) eviction scan under the limiter mutex.
|
||||
|
||||
`runMaintenance` (app.zig:709) gains an `api_limiter: ?*ApiLimiter`
|
||||
parameter, wired at :565 from the instance built at :380-388. In the loop,
|
||||
beside the two existing tasks: `_ = api.sweep(io, Clock.awake.now(io));` —
|
||||
note it locks itself, unlike the DNS limiter. Test: the maintenance-loop
|
||||
integration coverage asserts a stale bucket disappears.
|
||||
`runMaintenance` (app.zig:709) gains an `api_limiter: ?*ApiLimiter` parameter, wired at :565 from the instance built at :380-388. In the loop, beside the two existing tasks: `_ = api.sweep(io, Clock.awake.now(io));` — note it locks itself, unlike the DNS limiter. Test: the maintenance-loop integration coverage asserts a stale bucket disappears.
|
||||
|
||||
### 13. UDP/53 and TCP/53 stats reach /metrics
|
||||
|
||||
`src/server/udp_server.zig:38-49` and `tcp_server.zig:54-60`: both stats
|
||||
structs are written and read by nothing outside their own integration
|
||||
tests — `dropped_no_slot`/`dropped_oversize` have no metric, no API field,
|
||||
and errors log below the default level, while the DoT/DoH siblings export
|
||||
equivalent counters. The module doc sells "dropped and counted"; the count
|
||||
is unobservable.
|
||||
`src/server/udp_server.zig:38-49` and `tcp_server.zig:54-60`: both stats structs are written and read by nothing outside their own integration tests — `dropped_no_slot`/`dropped_oversize` have no metric, no API field, and errors log below the default level, while the DoT/DoH siblings export equivalent counters. The module doc sells "dropped and counted"; the count is unobservable.
|
||||
|
||||
- Both gain `pub const Snapshot` + `pub fn snapshotStats` in the exact DoT
|
||||
shape (dot_server.zig:61, :232). Field names stay as-is (the
|
||||
`accepted`/`connections` unification is milestone-18 work).
|
||||
- `WebState` (src/web/server.zig) gains `udp_listeners: []const
|
||||
*udp_server.UdpServer = &.{}` and `tcp_listeners: []const
|
||||
*tcp_server.TcpServer = &.{}` — S4 adds the fields with exactly these
|
||||
names and types; S2 consumes them (the app builds four listeners:
|
||||
udp6/udp4/tcp6/tcp4, app.zig:506-528).
|
||||
- `metrics.zig`: sum each family across its listeners into one
|
||||
`nxdns_udp_server_*_total` / `nxdns_tcp_server_*_total` group via the
|
||||
existing `counterGroup` derivation; extend the name-assertion test
|
||||
(:722-743).
|
||||
- Both gain `pub const Snapshot` + `pub fn snapshotStats` in the exact DoT shape (dot_server.zig:61, :232). Field names stay as-is (the `accepted`/`connections` unification is milestone-18 work).
|
||||
- `WebState` (src/web/server.zig) gains `udp_listeners: []const *udp_server.UdpServer = &.{}` and `tcp_listeners: []const *tcp_server.TcpServer = &.{}` — S4 adds the fields with exactly these names and types; S2 consumes them (the app builds four listeners: udp6/udp4/tcp6/tcp4, app.zig:506-528).
|
||||
- `metrics.zig`: sum each family across its listeners into one `nxdns_udp_server_*_total` / `nxdns_tcp_server_*_total` group via the existing `counterGroup` derivation; extend the name-assertion test (:722-743).
|
||||
- `app.zig` wires the slices.
|
||||
|
||||
### 14. Forward-client counters survive the query
|
||||
|
||||
`src/local/forward_client.zig:44-57`: `Stats` (queries, udp_truncated,
|
||||
foreign_datagrams, failures) is instrumented, documented as preventing "an
|
||||
unrecorded failure mode" — and the only production caller builds a
|
||||
stack-local client per query (handler.zig:394) and drops it, so the spoofing
|
||||
signal does not exist.
|
||||
`src/local/forward_client.zig:44-57`: `Stats` (queries, udp_truncated, foreign_datagrams, failures) is instrumented, documented as preventing "an unrecorded failure mode" — and the only production caller builds a stack-local client per query (handler.zig:394) and drops it, so the spoofing signal does not exist.
|
||||
|
||||
`Handler.Stats` (handler.zig:130-154) gains `forward_udp_truncated`,
|
||||
`forward_foreign_datagrams`, `forward_failures` (atomic, like the other 17;
|
||||
queries are already counted by `forward_zone_answers`). After the exchange
|
||||
in `viaForwardZone`, fetchAdd the client's counters into them. The metrics
|
||||
reflection (`dns_stat_fields`, metrics.zig:48) picks the new fields up
|
||||
without an edit — assert the three new `nxdns_dns_*_total` names in the
|
||||
metrics test. `ForwardClient.Stats` itself stays plain and per-instance.
|
||||
`Handler.Stats` (handler.zig:130-154) gains `forward_udp_truncated`, `forward_foreign_datagrams`, `forward_failures` (atomic, like the other 17; queries are already counted by `forward_zone_answers`). After the exchange in `viaForwardZone`, fetchAdd the client's counters into them. The metrics reflection (`dns_stat_fields`, metrics.zig:48) picks the new fields up without an edit — assert the three new `nxdns_dns_*_total` names in the metrics test. `ForwardClient.Stats` itself stays plain and per-instance.
|
||||
|
||||
### 15. The TLS server keeps the concrete transport cause
|
||||
|
||||
`src/platform/tls_server.zig:418-438`: both BIO callbacks discard the
|
||||
stashed cause — `net_reader.err` / `net_writer.err` (which hold
|
||||
Reset/Timeout/**Canceled**, std Io/net.zig:1260/:1324) are never read
|
||||
anywhere in the file — so a routine idle-budget cancel surfaces as a warn
|
||||
"mbedtls_ssl_read failed" and peer resets are indistinguishable from
|
||||
timeouts. The client side solved exactly this (dot_client.zig
|
||||
`concreteRead`/`concreteWrite`, :303-318).
|
||||
`src/platform/tls_server.zig:418-438`: both BIO callbacks discard the stashed cause — `net_reader.err` / `net_writer.err` (which hold Reset/Timeout/**Canceled**, std Io/net.zig:1260/:1324) are never read anywhere in the file — so a routine idle-budget cancel surfaces as a warn "mbedtls_ssl_read failed" and peer resets are indistinguishable from timeouts. The client side solved exactly this (dot_client.zig `concreteRead`/`concreteWrite`, :303-318).
|
||||
|
||||
- `ServerStream` gains `recv_cause: ?anyerror = null`, `send_cause:
|
||||
?anyerror = null`; `bioRecv`/`bioSend` stash
|
||||
`self.net_reader.err`/`self.net_writer.err` on failure before returning
|
||||
the mbedtls code.
|
||||
- `ReadError` (:216-223) widens with `Canceled`; the read path
|
||||
(`readIntoBuffer`, :334) maps a stashed `error.Canceled` to it instead of
|
||||
`TlsFailed`. Callers' race harnesses already treat cancellation
|
||||
separately.
|
||||
- Unit tests in the file's existing stub style: a reader whose `err` is
|
||||
Canceled yields `error.Canceled`; a Reset yields `TlsFailed` with the
|
||||
cause stashed for logging (ruling 16).
|
||||
- `ServerStream` gains `recv_cause: ?anyerror = null`, `send_cause: ?anyerror = null`; `bioRecv`/`bioSend` stash `self.net_reader.err`/`self.net_writer.err` on failure before returning the mbedtls code.
|
||||
- `ReadError` (:216-223) widens with `Canceled`; the read path (`readIntoBuffer`, :334) maps a stashed `error.Canceled` to it instead of `TlsFailed`. Callers' race harnesses already treat cancellation separately.
|
||||
- Unit tests in the file's existing stub style: a reader whose `err` is Canceled yields `error.Canceled`; a Reset yields `TlsFailed` with the cause stashed for logging (ruling 16).
|
||||
|
||||
### 16. Peer misbehavior logs at debug, like the read path already rules
|
||||
|
||||
`src/platform/tls_server.zig`: the read path deliberately logs a peer TCP
|
||||
drop at debug (:362-364, "a louder level would be a log-spam vector"), then
|
||||
`close` warns on close_notify against the same dead socket (:311) and
|
||||
`handshake` warns per probe (:287). `.tls_server` is not in the dedup scope
|
||||
set (logging.zig:103-108), so peer-driven warns can evict genuine warnings
|
||||
from the rotating log.
|
||||
`src/platform/tls_server.zig`: the read path deliberately logs a peer TCP drop at debug (:362-364, "a louder level would be a log-spam vector"), then `close` warns on close_notify against the same dead socket (:311) and `handshake` warns per probe (:287). `.tls_server` is not in the dedup scope set (logging.zig:103-108), so peer-driven warns can evict genuine warnings from the rotating log.
|
||||
|
||||
- `report` (:474-479) gains a level parameter. The close_notify site passes
|
||||
debug when `peer_closed` is set or the stashed cause (ruling 15) is a
|
||||
peer-class error; the handshake site likewise. Local/config failures keep
|
||||
warn.
|
||||
- Add `.tls_server` to `isDedupScope` (logging.zig:103-108) and to its
|
||||
membership test (:637-644).
|
||||
- `report` (:474-479) gains a level parameter. The close_notify site passes debug when `peer_closed` is set or the stashed cause (ruling 15) is a peer-class error; the handshake site likewise. Local/config failures keep warn.
|
||||
- Add `.tls_server` to `isDedupScope` (logging.zig:103-108) and to its membership test (:637-644).
|
||||
|
||||
**Scope widened during implementation (orchestrator ruling):** the level
|
||||
choice also applies to the `ssl_read` and `ssl_write` report sites, not only
|
||||
the two named ones. Ruling 15's "cause stashed for logging" wording requires
|
||||
the read site to consume the level, and an unconditional warn on the write
|
||||
half would keep the same spam vector open through `close` → `flush` against
|
||||
a dead peer. Peer-class causes are decided by pure helpers (`isPeerCause`,
|
||||
`causeLevel`); `check` still passes warn.
|
||||
**Scope widened during implementation (orchestrator ruling):** the level choice also applies to the `ssl_read` and `ssl_write` report sites, not only the two named ones. Ruling 15's "cause stashed for logging" wording requires the read site to consume the level, and an unconditional warn on the write half would keep the same spam vector open through `close` → `flush` against a dead peer. Peer-class causes are decided by pure helpers (`isPeerCause`, `causeLevel`); `check` still passes warn.
|
||||
|
||||
### 17. The query log heals its own gaps
|
||||
|
||||
`web/src/features/queries/QueryLogPage.tsx`: the hand-rolled accumulation
|
||||
(`extra`, `cursorOverride`, `generation`, :79-98) develops a silent
|
||||
mid-table row gap when the base page refetches after 30 s staleness — the
|
||||
newest-100 boundary moves up while `extra` starts below the old cursor, and
|
||||
the stale override means load-more never heals it. On a live DNS server the
|
||||
trigger is routine.
|
||||
`web/src/features/queries/QueryLogPage.tsx`: the hand-rolled accumulation (`extra`, `cursorOverride`, `generation`, :79-98) develops a silent mid-table row gap when the base page refetches after 30 s staleness — the newest-100 boundary moves up while `extra` starts below the old cursor, and the stale override means load-more never heals it. On a live DNS server the trigger is routine.
|
||||
|
||||
Migrate to `useInfiniteQuery`: the endpoint already maps onto it
|
||||
(`getNextPageParam: (last) => last.next_before ?? undefined`; keyset
|
||||
pagination confirmed server-side, handlers/queries.zig:103-114). A
|
||||
background refetch then refetches all pages in order — consistent, no gap.
|
||||
The three behaviors the old code carried by hand: staleness discard is
|
||||
handled by the query itself; keep the `isPlaceholderData` disable on the
|
||||
button; the 401 branch is already covered by the global cache-level
|
||||
`handleUnauthorized` (queryClient.ts:29-30). Delete `extra`,
|
||||
`cursorOverride`, `generation`, `loadingMore`, `moreError`. Rewrite the five
|
||||
load-more tests; add one for the gap scenario: base refetch with new rows
|
||||
between page renders leaves no discontinuity.
|
||||
Migrate to `useInfiniteQuery`: the endpoint already maps onto it (`getNextPageParam: (last) => last.next_before ?? undefined`; keyset pagination confirmed server-side, handlers/queries.zig:103-114). A background refetch then refetches all pages in order — consistent, no gap. The three behaviors the old code carried by hand: staleness discard is handled by the query itself; keep the `isPlaceholderData` disable on the button; the 401 branch is already covered by the global cache-level `handleUnauthorized` (queryClient.ts:29-30). Delete `extra`, `cursorOverride`, `generation`, `loadingMore`, `moreError`. Rewrite the five load-more tests; add one for the gap scenario: base refetch with new rows between page renders leaves no discontinuity.
|
||||
|
||||
### 18. Password hashing leaves the config lock
|
||||
|
||||
`src/web/handlers/settings.zig`: `applyPut` holds `config_lock` from :286
|
||||
to :341, and the argon2id call (t=2, m=19 MiB, :373-391) sits inside it at
|
||||
`src/web/handlers/settings.zig`: `applyPut` holds `config_lock` from :286 to :341, and the argon2id call (t=2, m=19 MiB, :373-391) sits inside it at
|
||||
:312 — every settings GET (:407-409 takes the same lock) and every mutation
|
||||
stalls for the hash duration on the Pi 5. The login path already does it
|
||||
right: copy under lock, hash unlocked, re-check generation under lock
|
||||
(auth.zig:47-71), and the LiveHash generation check (auth.zig:73-79)
|
||||
already closes the concurrent-install race.
|
||||
stalls for the hash duration on the Pi 5. The login path already does it right: copy under lock, hash unlocked, re-check generation under lock (auth.zig:47-71), and the LiveHash generation check (auth.zig:73-79) already closes the concurrent-install race.
|
||||
|
||||
Move the password-length check and the `hashPassword` call before the lock
|
||||
acquisition; everything from `loadConfig` on stays inside. The hash input
|
||||
is the parsed patch only, so nothing under the lock is needed. Tests: the
|
||||
existing applyPut suite passes unchanged; add one that actually
|
||||
distinguishes the new behavior — "both complete" already held before the
|
||||
fix, the GET merely waited out the hash. A stall seam under
|
||||
`builtin.is_test` (the milestone-15 rotation-seam shape) parks
|
||||
`hashPassword` on an event; the test starts a password PUT, waits until
|
||||
the hash is parked, completes a settings GET **while the hash is still
|
||||
parked**, then releases the seam and joins the PUT.
|
||||
Move the password-length check and the `hashPassword` call before the lock acquisition; everything from `loadConfig` on stays inside. The hash input is the parsed patch only, so nothing under the lock is needed. Tests: the existing applyPut suite passes unchanged; add one that actually distinguishes the new behavior — "both complete" already held before the fix, the GET merely waited out the hash. A stall seam under `builtin.is_test` (the milestone-15 rotation-seam shape) parks `hashPassword` on an event; the test starts a password PUT, waits until the hash is parked, completes a settings GET **while the hash is still parked**, then releases the seam and joins the PUT.
|
||||
|
||||
## Sessions
|
||||
|
||||
S1-S5 run in parallel. One cross-session interface, fixed here: S4 adds the
|
||||
`WebState.udp_listeners`/`tcp_listeners` fields exactly as ruling 13 names
|
||||
them; S2 wires and consumes them without touching `web/server.zig`.
|
||||
S1-S5 run in parallel. One cross-session interface, fixed here: S4 adds the `WebState.udp_listeners`/`tcp_listeners` fields exactly as ruling 13 names them; S2 wires and consumes them without touching `web/server.zig`.
|
||||
|
||||
### Session S1: filter
|
||||
|
||||
Owns `src/filter/manager.zig`, `src/filter/parsers.zig`,
|
||||
`src/filter/filter_integration_test.zig`. Rulings 1, 2, 3, 5.
|
||||
Owns `src/filter/manager.zig`, `src/filter/parsers.zig`, `src/filter/filter_integration_test.zig`. Rulings 1, 2, 3, 5.
|
||||
|
||||
### Session S2: cache, retention, counters, wiring
|
||||
|
||||
Owns `src/cache/dns_cache.zig`, `src/storage/retention.zig`, `src/app.zig`,
|
||||
`src/server/udp_server.zig`, `src/server/tcp_server.zig`,
|
||||
`src/local/forward_client.zig`, `src/server/handler.zig`,
|
||||
`src/web/metrics.zig`, and the storage/server integration tests those
|
||||
touch. Rulings 4, 6, 12, 13 (except the WebState fields), 14.
|
||||
Owns `src/cache/dns_cache.zig`, `src/storage/retention.zig`, `src/app.zig`, `src/server/udp_server.zig`, `src/server/tcp_server.zig`, `src/local/forward_client.zig`, `src/server/handler.zig`, `src/web/metrics.zig`, and the storage/server integration tests those touch. Rulings 4, 6, 12, 13 (except the WebState fields), 14.
|
||||
|
||||
### Session S3: TLS listeners
|
||||
|
||||
Owns `src/platform/tls_server.zig`, `src/platform/logging.zig`,
|
||||
`src/server/doh_server.zig`. Rulings 9, 10, 15, 16.
|
||||
Owns `src/platform/tls_server.zig`, `src/platform/logging.zig`, `src/server/doh_server.zig`. Rulings 9, 10, 15, 16.
|
||||
|
||||
### Session S4: web server
|
||||
|
||||
Owns `src/web/server.zig`, `src/web/sse.zig`, `src/web/handlers/live.zig`,
|
||||
`src/web/handlers/settings.zig`, `src/web/web_integration_test.zig`, plus
|
||||
the ruling-13 field additions. Rulings 7, 11, 18.
|
||||
Owns `src/web/server.zig`, `src/web/sse.zig`, `src/web/handlers/live.zig`, `src/web/handlers/settings.zig`, `src/web/web_integration_test.zig`, plus the ruling-13 field additions. Rulings 7, 11, 18.
|
||||
|
||||
### Session S5: frontend
|
||||
|
||||
@@ -405,8 +172,7 @@ Owns `web/src/features/live/*`, `web/src/features/queries/*`. Rulings 8,
|
||||
|
||||
### Orchestrator
|
||||
|
||||
Strikes the closed findings in `TECH_DEBT.md`; updates the stale
|
||||
milestone-5 as-built line if ruling 5 changed its truth value.
|
||||
Strikes the closed findings in `TECH_DEBT.md`; updates the stale milestone-5 as-built line if ruling 5 changed its truth value.
|
||||
|
||||
## Acceptance (milestone complete)
|
||||
|
||||
@@ -447,8 +213,7 @@ milestone-5 as-built line if ruling 5 changed its truth value.
|
||||
|
||||
## Anti-requirements
|
||||
|
||||
- No listener-core extraction, no shared repo helpers, no shared transport
|
||||
helpers — milestone 18.
|
||||
- No listener-core extraction, no shared repo helpers, no shared transport helpers — milestone 18.
|
||||
- No stats field renames (`accepted` vs `connections`) — milestone 18.
|
||||
- No config key renames and no trusted-proxy setting — milestone 17.
|
||||
- No `useInfiniteQuery` migration anywhere but the query log.
|
||||
|
||||
+60
-347
@@ -1,419 +1,134 @@
|
||||
# Milestone 17: contract repairs
|
||||
|
||||
Goal: make the operator-facing contract true. Four decided items — a real
|
||||
per-query deadline, the upstream editor milestone 8 promised, an opt-in
|
||||
trusted-proxy setting, and a field-level contract guard between the server
|
||||
and the frontend — plus BADVERS, the validator holes, and the doc claims
|
||||
the code contradicts. `TECH_DEBT.md` Theme 4. All four decisions are made;
|
||||
do not re-litigate them.
|
||||
Goal: make the operator-facing contract true. Four decided items — a real per-query deadline, the upstream editor milestone 8 promised, an opt-in trusted-proxy setting, and a field-level contract guard between the server and the frontend — plus BADVERS, the validator holes, and the doc claims the code contradicts. `TECH_DEBT.md` Theme 4. All four decisions are made; do not re-litigate them.
|
||||
|
||||
**PROVISIONAL.** Written before milestones 15-16 were built. m15 touches
|
||||
`docs_drift_test.zig` and `cli.zig`; m16 touches `pool.zig`'s neighbors,
|
||||
the listeners and the settings handler. Re-verify line references before
|
||||
each session starts.
|
||||
**PROVISIONAL.** Written before milestones 15-16 were built. m15 touches `docs_drift_test.zig` and `cli.zig`; m16 touches `pool.zig`'s neighbors, the listeners and the settings handler. Re-verify line references before each session starts.
|
||||
|
||||
## Rulings (binding)
|
||||
|
||||
### 1. `total_timeout_ms` becomes a real per-query deadline
|
||||
|
||||
Today the key feeds `Pool.attempt_timeout` (app.zig:283-288 →
|
||||
pool.zig:110) and bounds one attempt; with N upstreams down a query takes
|
||||
N × 5000 ms. The reference docs contradict each other (configuration.md
|
||||
says "per-query budget", cli.md says "per-attempt") and PLAN promises a
|
||||
total budget. Decision: the total budget becomes real.
|
||||
Today the key feeds `Pool.attempt_timeout` (app.zig:283-288 → pool.zig:110) and bounds one attempt; with N upstreams down a query takes N × 5000 ms. The reference docs contradict each other (configuration.md says "per-query budget", cli.md says "per-attempt") and PLAN promises a total budget. Decision: the total budget becomes real.
|
||||
|
||||
- `model.Upstream` gains `attempt_timeout_ms: u32 = 2500`;
|
||||
`total_timeout_ms` keeps its name and default (5000) and becomes the
|
||||
whole-exchange deadline. `read_timeout_ms` is untouched (it feeds the
|
||||
forward-zone client only, app.zig:412).
|
||||
- `pool.zig`: `attempt` keeps its race, now budgeted by the new key. The
|
||||
two-pass failover loop is extracted into a private
|
||||
`exchangeLoopLen(self, io, query, response_buf) transport.ExchangeError!usize`
|
||||
(the `exchangeLen` precedent, pool.zig:690-693: `Io.concurrent` stores
|
||||
the future's return value, so the raced loop returns a length and
|
||||
`exchange` rebuilds the slice). A new outer race wraps that helper with
|
||||
the total budget, using the select idiom already in the file. The outer
|
||||
cancel unwinds an in-flight attempt — the `entry.busy` lock is taken
|
||||
cancelably on purpose (:169-174) and released by defer. Outer expiry
|
||||
returns `error.Timeout`, **unconditionally**: the fast-failure paths
|
||||
(all upstreams disabled → `ConnectFailed`) finish long before the
|
||||
budget, so no attempted-marker is needed, and a test pins that an
|
||||
all-disabled pool still fails immediately rather than waiting out the
|
||||
deadline.
|
||||
- `validate.zig`: `checkTimeout` both keys; the cross-check becomes
|
||||
`attempt_timeout_ms <= total_timeout_ms` (the old `total >= read` check,
|
||||
which related knobs of different subsystems, is deleted).
|
||||
- Settings plumbing for the new key (the reflective walk carries it; the
|
||||
hand-written mirrors do not): `expected_keys` (model.zig:449, sorted),
|
||||
the round-trip test with a non-default value, the `SettingsView`
|
||||
upstream struct in web_integration_test.zig:528, `types.ts`
|
||||
`Settings.upstream`, `openapi.yaml` Settings + SettingsPatch,
|
||||
SettingsPage `SECTIONS[0]`, the configuration.md row (drift-guarded),
|
||||
and the cli.md/configuration.md prose — both now say: total is the
|
||||
per-query budget, attempt bounds one upstream try.
|
||||
- Tests: a pool test with two stalling upstreams proves the exchange
|
||||
returns within the total budget, not 2 × attempt; the existing
|
||||
per-attempt tests keep passing against the new key.
|
||||
- `model.Upstream` gains `attempt_timeout_ms: u32 = 2500`; `total_timeout_ms` keeps its name and default (5000) and becomes the whole-exchange deadline. `read_timeout_ms` is untouched (it feeds the forward-zone client only, app.zig:412).
|
||||
- `pool.zig`: `attempt` keeps its race, now budgeted by the new key. The two-pass failover loop is extracted into a private `exchangeLoopLen(self, io, query, response_buf) transport.ExchangeError!usize` (the `exchangeLen` precedent, pool.zig:690-693: `Io.concurrent` stores the future's return value, so the raced loop returns a length and `exchange` rebuilds the slice). A new outer race wraps that helper with the total budget, using the select idiom already in the file. The outer cancel unwinds an in-flight attempt — the `entry.busy` lock is taken cancelably on purpose (:169-174) and released by defer. Outer expiry returns `error.Timeout`, **unconditionally**: the fast-failure paths (all upstreams disabled → `ConnectFailed`) finish long before the budget, so no attempted-marker is needed, and a test pins that an all-disabled pool still fails immediately rather than waiting out the deadline.
|
||||
- `validate.zig`: `checkTimeout` both keys; the cross-check becomes `attempt_timeout_ms <= total_timeout_ms` (the old `total >= read` check, which related knobs of different subsystems, is deleted).
|
||||
- Settings plumbing for the new key (the reflective walk carries it; the hand-written mirrors do not): `expected_keys` (model.zig:449, sorted), the round-trip test with a non-default value, the `SettingsView` upstream struct in web_integration_test.zig:528, `types.ts` `Settings.upstream`, `openapi.yaml` Settings + SettingsPatch, SettingsPage `SECTIONS[0]`, the configuration.md row (drift-guarded), and the cli.md/configuration.md prose — both now say: total is the per-query budget, attempt bounds one upstream try.
|
||||
- Tests: a pool test with two stalling upstreams proves the exchange returns within the total budget, not 2 × attempt; the existing per-attempt tests keep passing against the new key.
|
||||
|
||||
### 2. The validator closes its two verified holes
|
||||
|
||||
- **Hostname `tls://` upstreams.** `Endpoint.parse` accepts any host
|
||||
text; the DoT client then refuses non-literals on every dial
|
||||
(`resolveAddress`, dot_client.zig:74-80), so a hostname `tls://` config
|
||||
validates clean and fails at query time as a peer fault. Milestone 3
|
||||
explicitly ordered this check carried into the validator and it was
|
||||
dropped. Correction to the audit: the function to mirror is **not**
|
||||
`parseResolver` itself (validate.zig:257 — that is the forward-zone
|
||||
resolver check); the upstream block in `checkCollections`
|
||||
(validate.zig:603-639) gains, for `.dot` endpoints,
|
||||
`net.IpAddress.parse(endpoint.host, endpoint.port) catch` → a new
|
||||
`error.UpstreamHostNotIpLiteral` diagnostic ("tls:// upstreams take an
|
||||
IP literal host"). `faults.zig` picks the variant up automatically.
|
||||
Fix the broken example at configuration.md:348 — the hostname NextDNS
|
||||
`tls://` config can never complete an exchange; show an IP-literal
|
||||
`tls://` example and move the hostname form to `https://`.
|
||||
- **Boot-allocation bounds.** `query_log_buffer_max` is floor-checked
|
||||
only (an Entry is ~420 bytes; `maxInt(u32)` asks for ~1.8 TB at boot),
|
||||
and `cache.size` is not validated at all (grep confirms). Both get the
|
||||
file's ceiling-only idiom: `1..1_000_000` with the "must be at most"
|
||||
message shape. No full memory-fit guarantee — out of reach, not needed.
|
||||
- **Hostname `tls://` upstreams.** `Endpoint.parse` accepts any host text; the DoT client then refuses non-literals on every dial (`resolveAddress`, dot_client.zig:74-80), so a hostname `tls://` config validates clean and fails at query time as a peer fault. Milestone 3 explicitly ordered this check carried into the validator and it was dropped. Correction to the audit: the function to mirror is **not** `parseResolver` itself (validate.zig:257 — that is the forward-zone resolver check); the upstream block in `checkCollections` (validate.zig:603-639) gains, for `.dot` endpoints, `net.IpAddress.parse(endpoint.host, endpoint.port) catch` → a new `error.UpstreamHostNotIpLiteral` diagnostic ("tls:// upstreams take an IP literal host"). `faults.zig` picks the variant up automatically. Fix the broken example at configuration.md:348 — the hostname NextDNS `tls://` config can never complete an exchange; show an IP-literal `tls://` example and move the hostname form to `https://`.
|
||||
- **Boot-allocation bounds.** `query_log_buffer_max` is floor-checked only (an Entry is ~420 bytes; `maxInt(u32)` asks for ~1.8 TB at boot), and `cache.size` is not validated at all (grep confirms). Both get the file's ceiling-only idiom: `1..1_000_000` with the "must be at most" message shape. No full memory-fit guarantee — out of reach, not needed.
|
||||
|
||||
**Recorded (implementation):** three consequences this ruling did not call
|
||||
out. (a) `cache.size = 0` was a documented "disables caching" mode; the
|
||||
`1..` floor removes it — configuration.md now says there is no off value
|
||||
(use 1). (b) The configuration.md NextDNS-DoT redaction passage lost its
|
||||
premise (a hostname `tls://` URL is now unconfigurable) and was rewritten.
|
||||
(c) Three config fixtures used `tls://dot.example:853` and now use an IP
|
||||
literal with `tls_name`; the back-up-and-restore.md export transcript grew
|
||||
to `head -10` with real re-captured output. Also: `Pool.init` takes a named
|
||||
`pool.Timeouts { attempt, total }` struct, not two positional durations —
|
||||
two same-typed adjacent parameters would be silently swappable.
|
||||
**Recorded (implementation):** three consequences this ruling did not call out. (a) `cache.size = 0` was a documented "disables caching" mode; the `1..` floor removes it — configuration.md now says there is no off value (use 1). (b) The configuration.md NextDNS-DoT redaction passage lost its premise (a hostname `tls://` URL is now unconfigurable) and was rewritten. (c) Three config fixtures used `tls://dot.example:853` and now use an IP literal with `tls_name`; the back-up-and-restore.md export transcript grew to `head -10` with real re-captured output. Also: `Pool.init` takes a named `pool.Timeouts { attempt, total }` struct, not two positional durations — two same-typed adjacent parameters would be silently swappable.
|
||||
|
||||
### 3. The upstream editor is built
|
||||
|
||||
Milestone 8 ruling 9 requires upstream CRUD in the UI ("the Settings page
|
||||
must edit them"); milestone 9 dropped the obligation when the SPA was
|
||||
built. The entire data layer exists with zero importers
|
||||
(`upstreamsQuery`, three mutation factories at queries.ts:251-264, four
|
||||
api.ts wrappers). Decision: build it. Placement deviates from m8's
|
||||
literal text: a dedicated **Upstreams page** (nav entry), matching the
|
||||
house resource-page pattern — record the deviation beside m8 ruling 9.
|
||||
Milestone 8 ruling 9 requires upstream CRUD in the UI ("the Settings page must edit them"); milestone 9 dropped the obligation when the SPA was built. The entire data layer exists with zero importers (`upstreamsQuery`, three mutation factories at queries.ts:251-264, four api.ts wrappers). Decision: build it. Placement deviates from m8's literal text: a dedicated **Upstreams page** (nav entry), matching the house resource-page pattern — record the deviation beside m8 ruling 9.
|
||||
|
||||
- New `web/src/features/upstreams/UpstreamsPage.tsx` +
|
||||
`UpstreamForm.tsx`, modeled on the blocklists pair (the closest
|
||||
analogue; copy its named idioms: editing-state-as-row, two mutation
|
||||
instances of one factory so row-toggle state never bleeds into the
|
||||
form, `mutateAsync` submit, full-row resend on toggle because **PUT is
|
||||
a replace**, `key={editing?.id ?? "add"}` remount, `window.confirm`
|
||||
delete, split error routing).
|
||||
- Fields: url, priority (number), enabled (toggle), tls_name. 400s carry
|
||||
the real validator's text (the handler validates via
|
||||
`mutations.checkUpstream`); surface the three 409 conflicts verbatim
|
||||
("an upstream with that url already exists", "the last enabled
|
||||
upstream cannot be disabled/removed").
|
||||
- Every mutation response carries `restart_required: true` and today
|
||||
nothing raises the banner outside SettingsPage. Generalize the restart
|
||||
banner copy to "Changes saved. Restart nxdns to apply." and raise it on
|
||||
upstream create/update/delete success.
|
||||
- Wiring is three edits: `createRoute` in routes.tsx, the `routeTree`
|
||||
entry, the AppShell nav item. The health table stays on the Dashboard
|
||||
(health rows have no `id` — url is the only join key — and reflect the
|
||||
running pool, so a new upstream appears there only after restart; the
|
||||
page states this beside the restart banner).
|
||||
- Dead-layer cleanup in the same stroke (the same finding): delete the
|
||||
seven unused single-row `get*` wrappers (correction: seven, not five —
|
||||
`getGroup` and `getUpstream` are shadowed by longer names in grep),
|
||||
plus `getMetrics`, `getOpenapiYaml`, and `ruleUpdateMutation`. The
|
||||
editor reads rows from the list query per house idiom, so none gains a
|
||||
caller. Git history keeps them.
|
||||
- Page tests in the stubbed-fetch house style: list render, add, toggle
|
||||
resends the full row, delete confirms, a 409 renders inline, the
|
||||
banner raises.
|
||||
- New `web/src/features/upstreams/UpstreamsPage.tsx` + `UpstreamForm.tsx`, modeled on the blocklists pair (the closest analogue; copy its named idioms: editing-state-as-row, two mutation instances of one factory so row-toggle state never bleeds into the form, `mutateAsync` submit, full-row resend on toggle because **PUT is a replace**, `key={editing?.id ?? "add"}` remount, `window.confirm` delete, split error routing).
|
||||
- Fields: url, priority (number), enabled (toggle), tls_name. 400s carry the real validator's text (the handler validates via `mutations.checkUpstream`); surface the three 409 conflicts verbatim ("an upstream with that url already exists", "the last enabled upstream cannot be disabled/removed").
|
||||
- Every mutation response carries `restart_required: true` and today nothing raises the banner outside SettingsPage. Generalize the restart banner copy to "Changes saved. Restart nxdns to apply." and raise it on upstream create/update/delete success.
|
||||
- Wiring is three edits: `createRoute` in routes.tsx, the `routeTree` entry, the AppShell nav item. The health table stays on the Dashboard (health rows have no `id` — url is the only join key — and reflect the running pool, so a new upstream appears there only after restart; the page states this beside the restart banner).
|
||||
- Dead-layer cleanup in the same stroke (the same finding): delete the seven unused single-row `get*` wrappers (correction: seven, not five — `getGroup` and `getUpstream` are shadowed by longer names in grep), plus `getMetrics`, `getOpenapiYaml`, and `ruleUpdateMutation`. The editor reads rows from the list query per house idiom, so none gains a caller. Git history keeps them.
|
||||
- Page tests in the stubbed-fetch house style: list render, add, toggle resends the full row, delete confirms, a 409 renders inline, the banner raises.
|
||||
|
||||
### 4. Trusted proxies restore real client identity
|
||||
|
||||
Behind the documented same-box reverse proxy every request is loopback:
|
||||
`localhost_exempt = true` (default) then disables the API limiter — the
|
||||
only brake on argon2 brute force — and all remote users share one
|
||||
address's 3-stream SSE cap. No X-Forwarded-For handling exists anywhere
|
||||
(verified repo-wide). Decision: opt-in trusted-proxy setting.
|
||||
Behind the documented same-box reverse proxy every request is loopback: `localhost_exempt = true` (default) then disables the API limiter — the only brake on argon2 brute force — and all remote users share one address's 3-stream SSE cap. No X-Forwarded-For handling exists anywhere (verified repo-wide). Decision: opt-in trusted-proxy setting.
|
||||
|
||||
- New setting `web.trusted_proxies: []const u8 = ""` — a comma-separated
|
||||
list of IP literals in one string (the settings codec supports scalars
|
||||
only; a list type is a compile error at model.zig:358). Empty means
|
||||
off: behavior today.
|
||||
- Validation: each comma-separated element must parse as an IP literal →
|
||||
new `error.BadTrustedProxy` diagnostic in the web block of
|
||||
`checkScalars`.
|
||||
- Mechanics: when the socket peer is in the trusted set, the effective
|
||||
client address is the **last** valid IP literal in the request's
|
||||
`X-Forwarded-For` (the entry our proxy appended); a **missing** header
|
||||
falls back to the socket peer. Implementation follows the
|
||||
copy-before-dispatch constraint (http_util.zig:10-14): a new
|
||||
`max_xff_len = 256` budget, a new Conn buffer, and a new `Request`
|
||||
field `client_addr: address.NetAddress` computed once in
|
||||
`handleRequest`. The copy keeps the **tail**, not the head: a new
|
||||
bounded suffix-copy helper beside `copyHeader`, because `copyHeader`
|
||||
returns empty on overflow (server.zig:608) and an empty result would
|
||||
fall back to the socket peer — loopback behind the intended same-box
|
||||
proxy, which `api_localhost_exempt = true` (model.zig:89, the default)
|
||||
then exempts. That is the exact bypass this ruling closes: a client
|
||||
ships an oversized XFF through the proxy and regains the exemption.
|
||||
The proxy appends `, <client-ip>` last, so the real entry always fits
|
||||
in the 256-byte tail. Fail closed on the residue: header present but
|
||||
no valid last IP literal in the tail means the trusted proxy violated
|
||||
its contract — respond 400, never fall back to the exemptible peer.
|
||||
Only a misconfigured proxy can trigger that arm; a spoofed prefix
|
||||
cannot, because the proxy's appended entry always terminates the
|
||||
chain. `bucketLimit` (server.zig:201-207) and the
|
||||
SSE acquire/release in `live.zig:82-87` both read `client_addr` — the
|
||||
SSE pair must use the same key or releases leak. `api_limiter.Config`
|
||||
is untouched: `WebState.web` already carries every web field
|
||||
(app.zig:472), so the trust check reads `state.web`.
|
||||
- With trust configured, a proxied remote address is not loopback, so
|
||||
the exemption no longer swallows it — the limiter and the per-address
|
||||
SSE cap bind per real client. Requests arriving at the socket from
|
||||
non-trusted, non-loopback peers are unaffected.
|
||||
- Settings plumbing checklist (same list as ruling 1, plus the
|
||||
hand-written web mirrors the fact-finding flagged): `expected_keys`,
|
||||
round-trip test, `WebView` + `view()` in handlers/settings.zig:207-248,
|
||||
`restart_required_keys` balance test, `SettingsView` web struct in the
|
||||
integration test, `types.ts` `Settings.web`, openapi web schemas
|
||||
(including the `required` array), SettingsPage web section, the
|
||||
configuration.md row, the api.md rate-limiting section, and a warning
|
||||
in the proxy/authentication how-to: when proxying without
|
||||
`trusted_proxies`, set `api_localhost_exempt = false`.
|
||||
- Tests: unit tests on the effective-address derivation (trusted peer +
|
||||
XFF chain, untrusted peer + spoofed XFF ignored, trusted peer + no
|
||||
XFF, trusted peer + an XFF over 256 bytes whose valid last entry is
|
||||
still honored from the tail, trusted peer + a present-but-invalid
|
||||
chain → 400); an integration test proving a proxied remote address is
|
||||
rate-limited while the proxy itself stays exempt.
|
||||
- New setting `web.trusted_proxies: []const u8 = ""` — a comma-separated list of IP literals in one string (the settings codec supports scalars only; a list type is a compile error at model.zig:358). Empty means off: behavior today.
|
||||
- Validation: each comma-separated element must parse as an IP literal → new `error.BadTrustedProxy` diagnostic in the web block of `checkScalars`.
|
||||
- Mechanics: when the socket peer is in the trusted set, the effective client address is the **last** valid IP literal in the request's `X-Forwarded-For` (the entry our proxy appended); a **missing** header falls back to the socket peer. Implementation follows the copy-before-dispatch constraint (http_util.zig:10-14): a new `max_xff_len = 256` budget, a new Conn buffer, and a new `Request` field `client_addr: address.NetAddress` computed once in `handleRequest`. The copy keeps the **tail**, not the head: a new bounded suffix-copy helper beside `copyHeader`, because `copyHeader` returns empty on overflow (server.zig:608) and an empty result would fall back to the socket peer — loopback behind the intended same-box proxy, which `api_localhost_exempt = true` (model.zig:89, the default) then exempts. That is the exact bypass this ruling closes: a client ships an oversized XFF through the proxy and regains the exemption. The proxy appends `, <client-ip>` last, so the real entry always fits in the 256-byte tail. Fail closed on the residue: header present but no valid last IP literal in the tail means the trusted proxy violated its contract — respond 400, never fall back to the exemptible peer. Only a misconfigured proxy can trigger that arm; a spoofed prefix cannot, because the proxy's appended entry always terminates the chain. `bucketLimit` (server.zig:201-207) and the SSE acquire/release in `live.zig:82-87` both read `client_addr` — the SSE pair must use the same key or releases leak. `api_limiter.Config` is untouched: `WebState.web` already carries every web field (app.zig:472), so the trust check reads `state.web`.
|
||||
- With trust configured, a proxied remote address is not loopback, so the exemption no longer swallows it — the limiter and the per-address SSE cap bind per real client. Requests arriving at the socket from non-trusted, non-loopback peers are unaffected.
|
||||
- Settings plumbing checklist (same list as ruling 1, plus the hand-written web mirrors the fact-finding flagged): `expected_keys`, round-trip test, `WebView` + `view()` in handlers/settings.zig:207-248, `restart_required_keys` balance test, `SettingsView` web struct in the integration test, `types.ts` `Settings.web`, openapi web schemas (including the `required` array), SettingsPage web section, the configuration.md row, the api.md rate-limiting section, and a warning in the proxy/authentication how-to: when proxying without `trusted_proxies`, set `api_localhost_exempt = false`.
|
||||
- Tests: unit tests on the effective-address derivation (trusted peer + XFF chain, untrusted peer + spoofed XFF ignored, trusted peer + no XFF, trusted peer + an XFF over 256 bytes whose valid last entry is still honored from the tail, trusted peer + a present-but-invalid chain → 400); an integration test proving a proxied remote address is rate-limited while the proxy itself stays exempt.
|
||||
|
||||
**Recorded (implementation of ruling 4):** `Request.client_addr` is
|
||||
`std.Io.net.IpAddress`, not `address.NetAddress` — `http_util.zig` is a
|
||||
module root for the fuzz target and cannot import nxdns modules; any future
|
||||
ruling that wants a domain type on `Request` hits this. `clientAddr` reads
|
||||
only the final chain entry, never scanning backwards: an earlier entry is
|
||||
client-controlled, so a fallback would hand the request an attacker-chosen
|
||||
identity. Known residual, out of this ruling's list: the login log line in
|
||||
`handlers/auth.zig` still prints the socket peer, which reads as loopback
|
||||
behind a trusted proxy — carried to milestone 19.
|
||||
**Recorded (implementation of ruling 4):** `Request.client_addr` is `std.Io.net.IpAddress`, not `address.NetAddress` — `http_util.zig` is a module root for the fuzz target and cannot import nxdns modules; any future ruling that wants a domain type on `Request` hits this. `clientAddr` reads only the final chain entry, never scanning backwards: an earlier entry is client-controlled, so a fallback would hand the request an attacker-chosen identity. Known residual, out of this ruling's list: the login log line in `handlers/auth.zig` still prints the socket peer, which reads as loopback behind a trusted proxy — carried to milestone 19.
|
||||
|
||||
### 5. A field-level contract guard between server and frontend
|
||||
|
||||
The REST contract lives in three hand-synced copies. The Zig side is
|
||||
genuinely well guarded (56-entry contract table parsing live responses
|
||||
with `.ignore_unknown_fields = false`; route-level openapi guards — which
|
||||
live in `web_integration_test.zig:1494-1590` and `openapi.zig`, not in
|
||||
`docs_drift_test.zig` as the audit said). **TypeScript is guarded by
|
||||
nothing**: every frontend test stubs fetch, and types.ts is strictly
|
||||
narrower than the wire in places (literal unions like
|
||||
`Health.status: "ok" | "degraded"`), so re-parsing into Zig structs can
|
||||
never catch an out-of-union string. Decision: one integration layer
|
||||
asserting the frontend's consumed shapes against real responses.
|
||||
The REST contract lives in three hand-synced copies. The Zig side is genuinely well guarded (56-entry contract table parsing live responses with `.ignore_unknown_fields = false`; route-level openapi guards — which live in `web_integration_test.zig:1494-1590` and `openapi.zig`, not in `docs_drift_test.zig` as the audit said). **TypeScript is guarded by nothing**: every frontend test stubs fetch, and types.ts is strictly narrower than the wire in places (literal unions like `Health.status: "ok" | "degraded"`), so re-parsing into Zig structs can never catch an out-of-union string. Decision: one integration layer asserting the frontend's consumed shapes against real responses.
|
||||
|
||||
Mechanism — captured samples validated by `tsc`, no new dependencies:
|
||||
|
||||
- A new `-Dintegration` test beside the contract walk drives the real
|
||||
`Env` server through every route whose contract-table kind is
|
||||
`.json` **and** which the frontend consumes through an `api.ts` wrapper
|
||||
— the GETs **and** the JSON-returning POST/PUT wrappers (login, the
|
||||
group/client/rule/upstream/blocklist writes, refresh, pause, the
|
||||
settings PUT), driven with deterministic seed payloads so the mutating
|
||||
responses are stable too (plus one sample per shared error response
|
||||
class). GET-only would leave every write-path response contract
|
||||
unguarded. Explicitly excluded:
|
||||
the `.raw` routes (`/metrics`, `/api/openapi.yaml` — not JSON, and
|
||||
ruling 3 deletes their unused wrappers), the `.sse` route, and the
|
||||
`.none` deletes. The endpoint-to-type mapping is derived from `api.ts`,
|
||||
not invented. Captured bodies are **canonicalized**: keys sorted, every
|
||||
number replaced with 0 (type-preserving; strings and booleans kept —
|
||||
they come from the deterministic seed). Canonicalization is what makes
|
||||
the output byte-stable across runs.
|
||||
- The canonical samples render into a committed file
|
||||
`web/src/lib/contractSamples.gen.ts`: for each endpoint,
|
||||
`export const sample_<name>: <TsType> = <json>;` with the matching
|
||||
import from `types.ts`. TypeScript object literals get excess-property
|
||||
checking, so a server field missing from types.ts, a types.ts field
|
||||
missing from the wire, and an out-of-union literal all fail
|
||||
`npm run typecheck` — which already runs in CI's frontend job.
|
||||
**Recorded (implementation):** canonicalization gained two rules beyond
|
||||
"keys sorted, numbers→0": repeated array elements dedupe by canonical text
|
||||
(60 identical zeroed timeseries buckets collapse to one; differing rows all
|
||||
stay), and two build-identity strings (`Version.git_commit`,
|
||||
`Version.zig_version`) redact to `"<build>"` so the golden is not pinned to
|
||||
one machine — the list is the named constant `volatile_string_keys`. The
|
||||
renderer emits prettier's own fixpoint output, because CI runs
|
||||
`format:check` on the tree. 43 samples: 38 endpoint + 5 error classes. The
|
||||
regen command is documented in AGENTS.md.
|
||||
- A new `-Dintegration` test beside the contract walk drives the real `Env` server through every route whose contract-table kind is `.json` **and** which the frontend consumes through an `api.ts` wrapper — the GETs **and** the JSON-returning POST/PUT wrappers (login, the group/client/rule/upstream/blocklist writes, refresh, pause, the settings PUT), driven with deterministic seed payloads so the mutating responses are stable too (plus one sample per shared error response class). GET-only would leave every write-path response contract unguarded. Explicitly excluded: the `.raw` routes (`/metrics`, `/api/openapi.yaml` — not JSON, and ruling 3 deletes their unused wrappers), the `.sse` route, and the `.none` deletes. The endpoint-to-type mapping is derived from `api.ts`, not invented. Captured bodies are **canonicalized**: keys sorted, every number replaced with 0 (type-preserving; strings and booleans kept — they come from the deterministic seed). Canonicalization is what makes the output byte-stable across runs.
|
||||
- The canonical samples render into a committed file `web/src/lib/contractSamples.gen.ts`: for each endpoint, `export const sample_<name>: <TsType> = <json>;` with the matching import from `types.ts`. TypeScript object literals get excess-property checking, so a server field missing from types.ts, a types.ts field missing from the wire, and an out-of-union literal all fail `npm run typecheck` — which already runs in CI's frontend job. **Recorded (implementation):** canonicalization gained two rules beyond "keys sorted, numbers→0": repeated array elements dedupe by canonical text (60 identical zeroed timeseries buckets collapse to one; differing rows all stay), and two build-identity strings (`Version.git_commit`, `Version.zig_version`) redact to `"<build>"` so the golden is not pinned to one machine — the list is the named constant `volatile_string_keys`. The renderer emits prettier's own fixpoint output, because CI runs `format:check` on the tree. 43 samples: 38 endpoint + 5 error classes. The regen command is documented in AGENTS.md.
|
||||
|
||||
- Embedding: an anonymous-import root must be Zig, not TypeScript, so a
|
||||
wrapper `web/src/lib/contract_samples.zig` exports
|
||||
`pub const bytes = @embedFile("contractSamples.gen.ts");` and build.zig
|
||||
registers it — the exact `docs/docs.zig` pattern. The Zig test
|
||||
byte-compares the embedded bytes against the freshly rendered content
|
||||
and fails on mismatch.
|
||||
- Regeneration is explicit, not write-beside-the-embed (embedded bytes
|
||||
carry no source-tree path): a build option `-Dcontract-samples-out`
|
||||
(absolute path, wired through `build_options`) makes the test write the
|
||||
fresh content there instead of comparing. The pinned command, stated in
|
||||
the failure message and the docs:
|
||||
`zig build test -Dintegration -Dcontract-samples-out="$PWD/web/src/lib/contractSamples.gen.ts"`.
|
||||
Golden-file discipline: the server side keeps the samples current;
|
||||
`tsc` keeps types.ts honest against them.
|
||||
- `types.ts` gains `ErrorEnvelope` matching the shared error responses
|
||||
(verify the exact envelope shape against the handlers before writing
|
||||
it) so the error samples are typed too.
|
||||
- Recorded residual risk: the server↔openapi seam stays route-level
|
||||
only. The yaml is served verbatim and never parsed; field-level yaml
|
||||
drift remains possible and is accepted — revisit only if it bites.
|
||||
- Embedding: an anonymous-import root must be Zig, not TypeScript, so a wrapper `web/src/lib/contract_samples.zig` exports `pub const bytes = @embedFile("contractSamples.gen.ts");` and build.zig registers it — the exact `docs/docs.zig` pattern. The Zig test byte-compares the embedded bytes against the freshly rendered content and fails on mismatch.
|
||||
- Regeneration is explicit, not write-beside-the-embed (embedded bytes carry no source-tree path): a build option `-Dcontract-samples-out` (absolute path, wired through `build_options`) makes the test write the fresh content there instead of comparing. The pinned command, stated in the failure message and the docs: `zig build test -Dintegration -Dcontract-samples-out="$PWD/web/src/lib/contractSamples.gen.ts"`. Golden-file discipline: the server side keeps the samples current; `tsc` keeps types.ts honest against them.
|
||||
- `types.ts` gains `ErrorEnvelope` matching the shared error responses (verify the exact envelope shape against the handlers before writing it) so the error samples are typed too.
|
||||
- Recorded residual risk: the server↔openapi seam stays route-level only. The yaml is served verbatim and never parsed; field-level yaml drift remains possible and is accepted — revisit only if it bites.
|
||||
|
||||
### 6. BADVERS becomes expressible
|
||||
|
||||
RFC 6891 §6.1.3: a version-1 EDNS query must get RCODE 16 (BADVERS) with
|
||||
version 0 in the reply OPT. Nothing checks `opt.version` anywhere
|
||||
(verified), and the write path cannot express it: `addOptEcho` hardcodes
|
||||
`extended_rcode = 0` (packet.zig:354) and `synthesize` takes a 4-bit
|
||||
`types.Rcode`. Negligible operational impact; the mechanism is the
|
||||
finding.
|
||||
RFC 6891 §6.1.3: a version-1 EDNS query must get RCODE 16 (BADVERS) with version 0 in the reply OPT. Nothing checks `opt.version` anywhere (verified), and the write path cannot express it: `addOptEcho` hardcodes `extended_rcode = 0` (packet.zig:354) and `synthesize` takes a 4-bit `types.Rcode`. Negligible operational impact; the mechanism is the finding.
|
||||
|
||||
- `edns.zig` gains the write-side splitter beside the read-side
|
||||
`extendedRcode` (:272): a 12-bit rcode splits into the header's four
|
||||
bits and the OPT's upper eight. `extendedRcode` stays — it is the
|
||||
read-side pair (milestone 19 assumes it survives).
|
||||
- `packet.zig` gains `addOptWithRcode(request_opt, do_bit,
|
||||
extended_rcode: u8)`; `addOptEcho` becomes a call with 0.
|
||||
- `handler.zig`: the version check sits directly after the successful
|
||||
`parseOpt` (:227-233), before the opcode check — `if (o.version != 0)`
|
||||
→ a BADVERS synthesis (header `.no_error` + OPT `extended_rcode = 1,
|
||||
version = 0`), counted in a new `Handler.Stats.badvers` (the metrics
|
||||
reflection exports it; extend the name test).
|
||||
- Tests: a version-1 query yields composite rcode 16 (assert via
|
||||
`edns.extendedRcode`) and version 0 in the reply OPT; a version-0
|
||||
query is unaffected.
|
||||
- `edns.zig` gains the write-side splitter beside the read-side `extendedRcode` (:272): a 12-bit rcode splits into the header's four bits and the OPT's upper eight. `extendedRcode` stays — it is the read-side pair (milestone 19 assumes it survives).
|
||||
- `packet.zig` gains `addOptWithRcode(request_opt, do_bit, extended_rcode: u8)`; `addOptEcho` becomes a call with 0.
|
||||
- `handler.zig`: the version check sits directly after the successful `parseOpt` (:227-233), before the opcode check — `if (o.version != 0)` → a BADVERS synthesis (header `.no_error` + OPT `extended_rcode = 1, version = 0`), counted in a new `Handler.Stats.badvers` (the metrics reflection exports it; extend the name test).
|
||||
- Tests: a version-1 query yields composite rcode 16 (assert via `edns.extendedRcode`) and version 0 in the reply OPT; a version-0 query is unaffected.
|
||||
|
||||
**Recorded (implementation):** the write-side splitter is
|
||||
`edns.splitRcode` with `pub const badvers: u12 = 16`; the BADVERS reply
|
||||
echoes the question when `qdcount == 1`, following the neighboring
|
||||
FORMERR sites, so a client can bind the reply to its query; a malformed
|
||||
OPT whose version byte is 1 answers FORMERR, because `parseOpt` fails
|
||||
before the version check can read the byte.
|
||||
**Recorded (implementation):** the write-side splitter is `edns.splitRcode` with `pub const badvers: u12 = 16`; the BADVERS reply echoes the question when `qdcount == 1`, following the neighboring FORMERR sites, so a client can bind the reply to its query; a malformed OPT whose version byte is 1 answers FORMERR, because `parseOpt` fails before the version check can read the byte.
|
||||
|
||||
### 7. The stale phase comments state the as-built contract
|
||||
|
||||
- `rate_limiter.zig:5-6` ("Phase 7 decides the locking"): rewrite to the
|
||||
as-built truth — the handler serializes with `Handler.limiter_mutex`
|
||||
(handler.zig:124), uncancelable on the query path, cancelable in the
|
||||
`runMaintenance` sweep (app.zig:702-731 already states the rationale
|
||||
to adopt).
|
||||
- `shutdown.zig` `trigger` doc ("what a Phase 8 restart endpoint would
|
||||
call"): no restart endpoint exists or is planned — m8 shipped
|
||||
`restart_required` echoes instead. Rewrite to "what a test uses".
|
||||
- `pool.zig:83-84`: tense only — the health endpoint exists; "Feeds
|
||||
`GET /api/upstream/health`."
|
||||
- `rate_limiter.zig:5-6` ("Phase 7 decides the locking"): rewrite to the as-built truth — the handler serializes with `Handler.limiter_mutex` (handler.zig:124), uncancelable on the query path, cancelable in the `runMaintenance` sweep (app.zig:702-731 already states the rationale to adopt).
|
||||
- `shutdown.zig` `trigger` doc ("what a Phase 8 restart endpoint would call"): no restart endpoint exists or is planned — m8 shipped `restart_required` echoes instead. Rewrite to "what a test uses".
|
||||
- `pool.zig:83-84`: tense only — the health endpoint exists; "Feeds `GET /api/upstream/health`."
|
||||
|
||||
### 8. Doc transcripts stop hardcoding the version
|
||||
|
||||
Four sites print `0.1.0-dev` (tutorial/first-run.md:108,
|
||||
install-with-docker.md:126, install-with-systemd.md:206,
|
||||
upgrade.md:142 — three how-to plus one tutorial, two different
|
||||
transcript shapes). Correct today; silently wrong at the first tag.
|
||||
Milestone 14 mandates a placeholder and a guard but names neither; the
|
||||
milestone-13 executed-transcript rule is in tension.
|
||||
Four sites print `0.1.0-dev` (tutorial/first-run.md:108, install-with-docker.md:126, install-with-systemd.md:206, upgrade.md:142 — three how-to plus one tutorial, two different transcript shapes). Correct today; silently wrong at the first tag. Milestone 14 mandates a placeholder and a guard but names neither; the milestone-13 executed-transcript rule is in tension.
|
||||
|
||||
- Ruling on the tension, explicit: milestone-13 ruling 3 constrains
|
||||
**command lines**; pasted **output lines** may carry placeholders.
|
||||
The orchestrator adds one clarifying sentence to milestone-13.md
|
||||
beside ruling 3.
|
||||
- The four output lines replace the version with `<version>` (e.g.
|
||||
`nxdns <version> serving on ...`).
|
||||
- The guard: `docs/docs.zig` additionally embeds the tutorial and
|
||||
how-to pages; a new test in `docs_drift_test.zig` asserts no embedded
|
||||
doc page contains the string `0.1.0-dev` and that the four transcript
|
||||
pages contain `nxdns <version>`. (Note m15 rewrote this file's CLI
|
||||
test — rebase, don't collide.)
|
||||
- Ruling on the tension, explicit: milestone-13 ruling 3 constrains **command lines**; pasted **output lines** may carry placeholders. The orchestrator adds one clarifying sentence to milestone-13.md beside ruling 3.
|
||||
- The four output lines replace the version with `<version>` (e.g. `nxdns <version> serving on ...`).
|
||||
- The guard: `docs/docs.zig` additionally embeds the tutorial and how-to pages; a new test in `docs_drift_test.zig` asserts no embedded doc page contains the string `0.1.0-dev` and that the four transcript pages contain `nxdns <version>`. (Note m15 rewrote this file's CLI test — rebase, don't collide.)
|
||||
|
||||
### 9. The log docs stop claiming append mode
|
||||
|
||||
`files-and-directories.md:143` claims the log file is "opened for
|
||||
append". It is not: `.mode = .write_only` plus `seekTo(state.file_pos)`
|
||||
per line, offset seeded once at open — the source comment at
|
||||
logging.zig:502 says so outright ("0.16.0 has no append mode"). Under
|
||||
external logrotate the divergence is destructive (copytruncate → sparse
|
||||
NUL-prefixed file; rename+create → unbounded writes to the renamed
|
||||
inode, `max_size_mb` unenforceable).
|
||||
`files-and-directories.md:143` claims the log file is "opened for append". It is not: `.mode = .write_only` plus `seekTo(state.file_pos)` per line, offset seeded once at open — the source comment at logging.zig:502 says so outright ("0.16.0 has no append mode"). Under external logrotate the divergence is destructive (copytruncate → sparse NUL-prefixed file; rename+create → unbounded writes to the renamed inode, `max_size_mb` unenforceable).
|
||||
|
||||
Docs only: correct the row (positional writes, offset tracked
|
||||
in-process), and add an operator warning — nxdns owns rotation for this
|
||||
file; do not point external logrotate at it. A per-line stat re-check
|
||||
was considered and rejected: a syscall per log line on the hot path to
|
||||
defend against a misconfiguration the docs now forbid. Record that
|
||||
decision here.
|
||||
Docs only: correct the row (positional writes, offset tracked in-process), and add an operator warning — nxdns owns rotation for this file; do not point external logrotate at it. A per-line stat re-check was considered and rejected: a syscall per log line on the hot path to defend against a misconfiguration the docs now forbid. Record that decision here.
|
||||
|
||||
## Sessions
|
||||
|
||||
Dependency graph: S1 → S2 → S5 (they share model.zig, validate.zig,
|
||||
types.ts, openapi.yaml, SettingsPage.tsx, web_integration_test.zig — the
|
||||
later session rebases on the earlier). S3, S4, S6 run in parallel with
|
||||
the chain and each other.
|
||||
Dependency graph: S1 → S2 → S5 (they share model.zig, validate.zig, types.ts, openapi.yaml, SettingsPage.tsx, web_integration_test.zig — the later session rebases on the earlier). S3, S4, S6 run in parallel with the chain and each other.
|
||||
|
||||
### Session S1: deadline and validator
|
||||
|
||||
Owns `src/upstream/pool.zig`, `src/config/model.zig`,
|
||||
`src/config/validate.zig`, `src/app.zig`, plus the ruling-1 plumbing
|
||||
trail (types.ts, openapi.yaml, SettingsPage SECTIONS[0], the
|
||||
integration-test SettingsView, configuration.md, cli.md). Rulings 1, 2,
|
||||
and the pool comment from ruling 7.
|
||||
Owns `src/upstream/pool.zig`, `src/config/model.zig`, `src/config/validate.zig`, `src/app.zig`, plus the ruling-1 plumbing trail (types.ts, openapi.yaml, SettingsPage SECTIONS[0], the integration-test SettingsView, configuration.md, cli.md). Rulings 1, 2, and the pool comment from ruling 7.
|
||||
|
||||
### Session S2: trusted proxy (after S1)
|
||||
|
||||
Owns `src/web/server.zig`, `src/web/http_util.zig`,
|
||||
`src/web/handlers/live.zig`, `src/web/handlers/settings.zig`, and its
|
||||
plumbing trail through the same shared files S1 touched. Ruling 4.
|
||||
Owns `src/web/server.zig`, `src/web/http_util.zig`, `src/web/handlers/live.zig`, `src/web/handlers/settings.zig`, and its plumbing trail through the same shared files S1 touched. Ruling 4.
|
||||
|
||||
### Session S3: upstream editor
|
||||
|
||||
Owns `web/src/features/upstreams/*` (new), `web/src/routes.tsx`,
|
||||
`web/src/shell/AppShell.tsx`, `web/src/lib/api.ts`,
|
||||
`web/src/lib/queries.ts`, `web/src/features/settings/RestartBanner.tsx`
|
||||
and `restartBanner.ts`, `specs/milestone-8.md` (the deviation note).
|
||||
Ruling 3.
|
||||
Owns `web/src/features/upstreams/*` (new), `web/src/routes.tsx`, `web/src/shell/AppShell.tsx`, `web/src/lib/api.ts`, `web/src/lib/queries.ts`, `web/src/features/settings/RestartBanner.tsx` and `restartBanner.ts`, `specs/milestone-8.md` (the deviation note). Ruling 3.
|
||||
|
||||
### Session S4: BADVERS
|
||||
|
||||
Owns `src/dns/packet.zig`, `src/dns/edns.zig`, `src/server/handler.zig`,
|
||||
the metrics name test in `src/web/metrics.zig`. Ruling 6.
|
||||
Owns `src/dns/packet.zig`, `src/dns/edns.zig`, `src/server/handler.zig`, the metrics name test in `src/web/metrics.zig`. Ruling 6.
|
||||
|
||||
### Session S5: contract samples (after S1 and S2)
|
||||
|
||||
Owns the new generator test in `src/web/web_integration_test.zig`,
|
||||
`build.zig` (the anonymous import), `src/tests.zig` (if a new file needs
|
||||
an entry), `web/src/lib/contractSamples.gen.ts` (committed golden),
|
||||
`web/src/lib/types.ts` (`ErrorEnvelope` only). Ruling 5.
|
||||
Owns the new generator test in `src/web/web_integration_test.zig`, `build.zig` (the anonymous import), `src/tests.zig` (if a new file needs an entry), `web/src/lib/contractSamples.gen.ts` (committed golden), `web/src/lib/types.ts` (`ErrorEnvelope` only). Ruling 5.
|
||||
|
||||
### Session S6: docs and comments
|
||||
|
||||
Owns `src/server/rate_limiter.zig` and `src/server/shutdown.zig`
|
||||
(headers only), `docs/docs.zig`, `src/docs_drift_test.zig`, the four
|
||||
transcript pages, `docs/reference/files-and-directories.md`,
|
||||
`specs/milestone-13.md` (the clarifying sentence). Rulings 7 (except the
|
||||
pool comment), 8, 9.
|
||||
Owns `src/server/rate_limiter.zig` and `src/server/shutdown.zig` (headers only), `docs/docs.zig`, `src/docs_drift_test.zig`, the four transcript pages, `docs/reference/files-and-directories.md`, `specs/milestone-13.md` (the clarifying sentence). Rulings 7 (except the pool comment), 8, 9.
|
||||
|
||||
### Orchestrator
|
||||
|
||||
Strikes the closed findings in `TECH_DEBT.md`; confirms PLAN's
|
||||
total-budget promise now matches the code; records the m8 placement
|
||||
deviation.
|
||||
Strikes the closed findings in `TECH_DEBT.md`; confirms PLAN's total-budget promise now matches the code; records the m8 placement deviation.
|
||||
|
||||
## Module layout
|
||||
|
||||
New files: `web/src/features/upstreams/UpstreamsPage.tsx`,
|
||||
`web/src/features/upstreams/UpstreamForm.tsx`,
|
||||
`web/src/lib/contractSamples.gen.ts` (generated, committed).
|
||||
New files: `web/src/features/upstreams/UpstreamsPage.tsx`, `web/src/features/upstreams/UpstreamForm.tsx`, `web/src/lib/contractSamples.gen.ts` (generated, committed).
|
||||
|
||||
Deleted surface: the seven unused `get*` wrappers, `getMetrics`,
|
||||
`getOpenapiYaml`, `ruleUpdateMutation`.
|
||||
Deleted surface: the seven unused `get*` wrappers, `getMetrics`, `getOpenapiYaml`, `ruleUpdateMutation`.
|
||||
|
||||
## Acceptance (milestone complete)
|
||||
|
||||
@@ -459,10 +174,8 @@ Deleted surface: the seven unused `get*` wrappers, `getMetrics`,
|
||||
## Anti-requirements
|
||||
|
||||
- No restart endpoint — the `restart_required` echo model stands.
|
||||
- No hostname resolution for `tls://` upstreams — the validator rejects;
|
||||
the client's own check stays as defense in depth.
|
||||
- No openapi codegen and no runtime schema validation library — the
|
||||
contract guard is captured samples + tsc, nothing more.
|
||||
- No hostname resolution for `tls://` upstreams — the validator rejects; the client's own check stays as defense in depth.
|
||||
- No openapi codegen and no runtime schema validation library — the contract guard is captured samples + tsc, nothing more.
|
||||
- No proxy protocol (PROXY/haproxy) support; X-Forwarded-For only.
|
||||
- No per-line stat re-check in the logger (ruling 9 records why).
|
||||
- No upstream health join in the editor beyond the stated restart note.
|
||||
|
||||
+71
-370
@@ -1,95 +1,35 @@
|
||||
# Milestone 18: collapse the duplicated infrastructure
|
||||
|
||||
Goal: the copy-paste infrastructure from `TECH_DEBT.md` Theme 2 becomes
|
||||
shared code — the listener core first (the audit's only high finding, four
|
||||
hand-synced copies of the most invariant-heavy concurrency code in the
|
||||
repository, drift already live), then the repository memory-safety
|
||||
choreography, the web CRUD shells, the transport scaffolding, the name
|
||||
normalization, the line iterator, and the frontend class constants. After
|
||||
this milestone, a fix in any of these families lands once.
|
||||
Goal: the copy-paste infrastructure from `TECH_DEBT.md` Theme 2 becomes shared code — the listener core first (the audit's only high finding, four hand-synced copies of the most invariant-heavy concurrency code in the repository, drift already live), then the repository memory-safety choreography, the web CRUD shells, the transport scaffolding, the name normalization, the line iterator, and the frontend class constants. After this milestone, a fix in any of these families lands once.
|
||||
|
||||
**PROVISIONAL.** Written before milestones 15-17 were built. m16 touches the
|
||||
listeners (DoH receiveHead race, stats export), the manager (refresh_lock)
|
||||
and the metrics tests; m17 touches pool.zig and the handlers. Re-verify
|
||||
every line reference and re-diff the copies before each session starts.
|
||||
**PROVISIONAL.** Written before milestones 15-17 were built. m16 touches the listeners (DoH receiveHead race, stats export), the manager (refresh_lock) and the metrics tests; m17 touches pool.zig and the handlers. Re-verify every line reference and re-diff the copies before each session starts.
|
||||
|
||||
## Rulings (binding)
|
||||
|
||||
### 1. One listener core
|
||||
|
||||
`tcp_server.zig`, `dot_server.zig`, `doh_server.zig` and `web/server.zig`
|
||||
each carry private copies of: the `State`/`ConnState`/`Stop`/`Claim` enums,
|
||||
`retry_delay`, `bump`, the slot pool with `claim`/`finish` (the same
|
||||
three-step close dance), `beginShutdown` (identical body, different log
|
||||
text), `serve` with the cancel-protection dance, `acceptLoop` with the same
|
||||
error mapping, `decideClaim` in two shapes plus `firstFree`, and — in the
|
||||
three DNS listeners — the byte-identical `Outcome`/`Result`/`race`/`expire`
|
||||
select harness. `readPrefix`/`readBody`/`writeReply` are byte-identical
|
||||
between tcp and dot. The predicted failure mode already fired once: the
|
||||
milestone-10 review hand-ported the `handshook` TLS-context-leak fix from
|
||||
dot to doh. Six unit tests are duplicated between tcp and dot, three more
|
||||
between doh and web.
|
||||
`tcp_server.zig`, `dot_server.zig`, `doh_server.zig` and `web/server.zig` each carry private copies of: the `State`/`ConnState`/`Stop`/`Claim` enums, `retry_delay`, `bump`, the slot pool with `claim`/`finish` (the same three-step close dance), `beginShutdown` (identical body, different log text), `serve` with the cancel-protection dance, `acceptLoop` with the same error mapping, `decideClaim` in two shapes plus `firstFree`, and — in the three DNS listeners — the byte-identical `Outcome`/`Result`/`race`/`expire` select harness. `readPrefix`/`readBody`/`writeReply` are byte-identical between tcp and dot. The predicted failure mode already fired once: the milestone-10 review hand-ported the `handshook` TLS-context-leak fix from dot to doh. Six unit tests are duplicated between tcp and dot, three more between doh and web.
|
||||
|
||||
New file `src/server/listener.zig`:
|
||||
|
||||
- The four enums, `retry_delay`, `bump` — plain shared declarations.
|
||||
- The race harness: `Outcome`, `Result`, `race`, `expire` — one copy,
|
||||
generic over the raced function (the existing `comptime f: anytype` shape
|
||||
is already generic; it only needs to move).
|
||||
- The framing helpers `readPrefix`, `readBody`, `writeReply` (tcp/dot
|
||||
consumers; doh speaks HTTP).
|
||||
- `pub fn Core(comptime Cfg: type) type` generating: the conns array with
|
||||
`Cfg.ConnPayload` per slot, mutex, `run_state`, `shutdown_begun`,
|
||||
`claim`/`finish`/`beginShutdown`/`decideClaim`/`firstFree`, the accept
|
||||
loop, and the `serve` skeleton with the cancel-protection dance. `Cfg`
|
||||
supplies exactly four things: the per-slot payload type (`ConnPayload` —
|
||||
the payloads genuinely differ per listener, TLS context and buffers),
|
||||
the per-connection serve function (`serveConn`), the read/write buffer
|
||||
sizes, and the at-capacity behavior (close for the DNS listeners; the
|
||||
web listener's `refuse` 503 for web).
|
||||
- The Core does **not** own the TLS lifecycle. Certificate pin and
|
||||
release, TLS-context ownership, the handshake, close_notify, and
|
||||
cleanup after a late race completion form one ordered sequence
|
||||
(doh_server.zig:294-330, dot_server.zig:288-323) and stay inside each
|
||||
listener's `serveConn`. What is shared is a helper,
|
||||
`listener.handshakeStage`, that owns the `handshook` exactly-once flag
|
||||
and its late-success cleanup contract — the exact defect the
|
||||
milestone-10 review hand-ported — with a doc comment stating the
|
||||
caller's required ordering (pin → handshake via the helper → serve →
|
||||
TLS close → release, cleanup on every early exit). Both TLS listeners
|
||||
adopt it; the leak class closes there, not in the Core.
|
||||
- The race harness: `Outcome`, `Result`, `race`, `expire` — one copy, generic over the raced function (the existing `comptime f: anytype` shape is already generic; it only needs to move).
|
||||
- The framing helpers `readPrefix`, `readBody`, `writeReply` (tcp/dot consumers; doh speaks HTTP).
|
||||
- `pub fn Core(comptime Cfg: type) type` generating: the conns array with `Cfg.ConnPayload` per slot, mutex, `run_state`, `shutdown_begun`, `claim`/`finish`/`beginShutdown`/`decideClaim`/`firstFree`, the accept loop, and the `serve` skeleton with the cancel-protection dance. `Cfg` supplies exactly four things: the per-slot payload type (`ConnPayload` — the payloads genuinely differ per listener, TLS context and buffers), the per-connection serve function (`serveConn`), the read/write buffer sizes, and the at-capacity behavior (close for the DNS listeners; the web listener's `refuse` 503 for web).
|
||||
- The Core does **not** own the TLS lifecycle. Certificate pin and release, TLS-context ownership, the handshake, close_notify, and cleanup after a late race completion form one ordered sequence (doh_server.zig:294-330, dot_server.zig:288-323) and stay inside each listener's `serveConn`. What is shared is a helper, `listener.handshakeStage`, that owns the `handshook` exactly-once flag and its late-success cleanup contract — the exact defect the milestone-10 review hand-ported — with a doc comment stating the caller's required ordering (pin → handshake via the helper → serve → TLS close → release, cleanup on every early exit). Both TLS listeners adopt it; the leak class closes there, not in the Core.
|
||||
- The shared unit tests move here; the per-file duplicates are deleted.
|
||||
|
||||
Unifications the extraction forces, all sanctioned:
|
||||
|
||||
- Counter name: `connections` everywhere. tcp and web rename `accepted`.
|
||||
This renames the m16-added metric family
|
||||
`nxdns_tcp_server_accepted_total` → `nxdns_tcp_server_connections_total`
|
||||
— greenfield, allowed; update the metrics name test and the docs
|
||||
reference. Core stats are one shared struct; `tls_handshake_failures`,
|
||||
`bad_requests`, `requests` stay listener-specific beside it, and the
|
||||
exported snapshots stay flat so /metrics output is unchanged except the
|
||||
tcp rename.
|
||||
- `deinit` shape: all four store the allocator at `listen` (dot's
|
||||
milestone-10 deviation becomes the rule) and take `(self, io)`.
|
||||
- The dead `pub fn serve` at doh_server.zig:687-707 (documented as
|
||||
intentional in specs/milestone-10.md:286-289) is deleted; note the
|
||||
update beside that spec line.
|
||||
- Counter name: `connections` everywhere. tcp and web rename `accepted`. This renames the m16-added metric family `nxdns_tcp_server_accepted_total` → `nxdns_tcp_server_connections_total` — greenfield, allowed; update the metrics name test and the docs reference. Core stats are one shared struct; `tls_handshake_failures`, `bad_requests`, `requests` stay listener-specific beside it, and the exported snapshots stay flat so /metrics output is unchanged except the tcp rename.
|
||||
- `deinit` shape: all four store the allocator at `listen` (dot's milestone-10 deviation becomes the rule) and take `(self, io)`.
|
||||
- The dead `pub fn serve` at doh_server.zig:687-707 (documented as intentional in specs/milestone-10.md:286-289) is deleted; note the update beside that spec line.
|
||||
|
||||
The per-listener files keep what genuinely differs: the connection payload
|
||||
(TLS context and buffers), the serve-one-connection logic, DoH's HTTP
|
||||
handling, and the web listener's request router. Every existing listener
|
||||
integration test must pass unchanged apart from renamed counters.
|
||||
The per-listener files keep what genuinely differs: the connection payload (TLS context and buffers), the serve-one-connection logic, DoH's HTTP handling, and the web listener's request router. Every existing listener integration test must pass unchanged apart from renamed counters.
|
||||
|
||||
### 2. The repository list choreography exists once
|
||||
|
||||
The `prepare → ArrayList → errdefer out.deinit → errdefer freeX →
|
||||
columnTextAlloc → append` shape is hand-rolled 19 times across the seven
|
||||
repository files, with the load-bearing errdefer ordering repeated 18 times
|
||||
and 26 hand-written `freeX` loops. The milestone-4 spec's own reference
|
||||
sample (specs/milestone-4.md:1263-1272) declares the two errdefers in the
|
||||
**reverse, use-after-free order** — every implementation silently corrected
|
||||
it; the next repo copied from the spec is a landmine.
|
||||
The `prepare → ArrayList → errdefer out.deinit → errdefer freeX → columnTextAlloc → append` shape is hand-rolled 19 times across the seven repository files, with the load-bearing errdefer ordering repeated 18 times and 26 hand-written `freeX` loops. The milestone-4 spec's own reference sample (specs/milestone-4.md:1263-1272) declares the two errdefers in the **reverse, use-after-free order** — every implementation silently corrected it; the next repo copied from the spec is a landmine.
|
||||
|
||||
`src/storage/repositories/crud.zig` (today: `execStrict` only) gains:
|
||||
|
||||
@@ -105,38 +45,16 @@ pub fn listRows(
|
||||
pub fn freeRows(comptime Row: type, gpa: Allocator, items: []const Row) void
|
||||
```
|
||||
|
||||
`listRows` owns the errdefer choreography — written once, with the ordering
|
||||
comment. `freeRows` frees every `[]const u8` **and every `?[]const u8`**
|
||||
field by comptime reflection — `SourceRow.checksum` is an allocated
|
||||
optional slice (sources_repo.zig:91-104) and its current destructor frees
|
||||
the non-null payload explicitly; a shallow slice-only reflection leaks it.
|
||||
Any other owning field shape is a `@compileError`, so a future row cannot
|
||||
silently leak. The allocation-failure tests must keep a case with a
|
||||
non-null checksum.
|
||||
The per-row `readRow` functions stay in the repos and keep their per-column
|
||||
errdefers. All 19 list functions become one-line delegations; the `freeX`
|
||||
wrappers stay as thin `pub` shims where callers use them, or are deleted
|
||||
where `freeRows` is called directly. Every `checkAllAllocationFailures`
|
||||
test stays and must pass — they now exercise the shared helper from 20
|
||||
angles.
|
||||
`listRows` owns the errdefer choreography — written once, with the ordering comment. `freeRows` frees every `[]const u8` **and every `?[]const u8`** field by comptime reflection — `SourceRow.checksum` is an allocated optional slice (sources_repo.zig:91-104) and its current destructor frees the non-null payload explicitly; a shallow slice-only reflection leaks it. Any other owning field shape is a `@compileError`, so a future row cannot silently leak. The allocation-failure tests must keep a case with a non-null checksum. The per-row `readRow` functions stay in the repos and keep their per-column errdefers. All 19 list functions become one-line delegations; the `freeX` wrappers stay as thin `pub` shims where callers use them, or are deleted where `freeRows` is called directly. Every `checkAllAllocationFailures` test stays and must pass — they now exercise the shared helper from 20 angles.
|
||||
|
||||
Correct the milestone-4 sample in place and note the correction in that
|
||||
spec.
|
||||
Correct the milestone-4 sample in place and note the correction in that spec.
|
||||
|
||||
### 3. The web CRUD shells are generated; the decisions stay hand-written
|
||||
|
||||
Seven handler files repeat the 4-line `configDb` switch 40 times in three
|
||||
forms, and five of them repeat identical `list`/`get`/`remove` shells. The
|
||||
four reload flavors (plain; local.zig's publish-under-lock; blocklists'
|
||||
pruneFiles; groups' read-back-under-lock) are genuinely different and are
|
||||
**not** unified.
|
||||
Seven handler files repeat the 4-line `configDb` switch 40 times in three forms, and five of them repeat identical `list`/`get`/`remove` shells. The four reload flavors (plain; local.zig's publish-under-lock; blocklists' pruneFiles; groups' read-back-under-lock) are genuinely different and are **not** unified.
|
||||
|
||||
- `mutations.zig` gains `pub fn requireConfigDb(state: *server.WebState)
|
||||
error{NoConfigDb}!*db.Db`. The 40 switch sites become one-line `catch`
|
||||
arms producing the same three response forms (the Failure payload is the
|
||||
same constant text everywhere).
|
||||
- `mutations.zig` gains a comptime resource descriptor generating only the
|
||||
identical trio:
|
||||
- `mutations.zig` gains `pub fn requireConfigDb(state: *server.WebState) error{NoConfigDb}!*db.Db`. The 40 switch sites become one-line `catch` arms producing the same three response forms (the Failure payload is the same constant text everywhere).
|
||||
- `mutations.zig` gains a comptime resource descriptor generating only the identical trio:
|
||||
|
||||
```zig
|
||||
// desc is an anonymous struct literal; `anytype` is not legal as a
|
||||
@@ -150,184 +68,67 @@ pruneFiles; groups' read-back-under-lock) are genuinely different and are
|
||||
// []const u8 ("upstreams", the JSON list key).
|
||||
```
|
||||
|
||||
yielding `list`/`get`/`remove` handlers in the exact current shape.
|
||||
Adopted by upstreams, groups, blocklists, clients, rules and the two
|
||||
local.zig sets where the shell matches; `settings.zig` (get + applyPut
|
||||
only) does not adopt. All `applyCreate`/`applyUpdate`/`applyDelete`
|
||||
bodies and the per-resource decision functions (countEnabledExcept,
|
||||
updateLocked/deleteLocked, pruneFiles, applyReplacePrefixes, publish,
|
||||
RuleView, applyPut) stay hand-written.
|
||||
yielding `list`/`get`/`remove` handlers in the exact current shape. Adopted by upstreams, groups, blocklists, clients, rules and the two local.zig sets where the shell matches; `settings.zig` (get + applyPut only) does not adopt. All `applyCreate`/`applyUpdate`/`applyDelete` bodies and the per-resource decision functions (countEnabledExcept, updateLocked/deleteLocked, pruneFiles, applyReplacePrefixes, publish, RuleView, applyPut) stay hand-written.
|
||||
|
||||
Acceptance is a grep: the 4-line switch appears zero times outside
|
||||
`mutations.zig`.
|
||||
Acceptance is a grep: the 4-line switch appears zero times outside `mutations.zig`.
|
||||
|
||||
### 4. The transport scaffolding is shared, and DoH gets its missing unwrap
|
||||
|
||||
- The byte-identical `Outcome` + `expire` + select-race body in
|
||||
`pool.zig:253-274/310-317` and `forward_client.zig:195-215/260-267`
|
||||
moves to `transport.zig` as one generic exchange-race helper (the raced
|
||||
function, the budget and one comment word are the only differences
|
||||
today). `manager.zig`'s `fetchWithin` may adopt it if it generalizes
|
||||
without contortion; not required.
|
||||
- `mapPhase` (duplicated byte-for-byte, dot_client.zig:283 /
|
||||
forward_client.zig:293) moves to `transport.zig` as `pub`.
|
||||
- The cancel-protected close helpers (four near-copies:
|
||||
forward_client.zig:281/287, dot_client.zig:271/277) become one
|
||||
`pub fn closeBlocked(io: std.Io, target: anytype) void` in transport.zig
|
||||
(`target.close(io)` under swapped protection). The inline copies in
|
||||
logging.zig and metrics.zig are out of scope — they are not transport
|
||||
code.
|
||||
- `doh_client.zig` gains the stashed-cause unwrap the other two transports
|
||||
carry (`concreteRead`/`concreteWrite` precedent, dot_client.zig:303-318):
|
||||
today its `mapError` never reads the http client's stashed cause behind
|
||||
`ReadFailed`/`WriteFailed`, so rare mid-exchange local errors (e.g.
|
||||
ENOBUFS) are recorded as peer faults against a healthy upstream — a
|
||||
stated spec-invariant violation, fixed once for DoT and left in DoH.
|
||||
Port the unwrap and the corresponding stub-based tests.
|
||||
- `sendFailure`/`receiveFailure` stay per-file: the unwrapping genuinely
|
||||
differs by stream type.
|
||||
- The byte-identical `Outcome` + `expire` + select-race body in `pool.zig:253-274/310-317` and `forward_client.zig:195-215/260-267` moves to `transport.zig` as one generic exchange-race helper (the raced function, the budget and one comment word are the only differences today). `manager.zig`'s `fetchWithin` may adopt it if it generalizes without contortion; not required.
|
||||
- `mapPhase` (duplicated byte-for-byte, dot_client.zig:283 / forward_client.zig:293) moves to `transport.zig` as `pub`.
|
||||
- The cancel-protected close helpers (four near-copies: forward_client.zig:281/287, dot_client.zig:271/277) become one `pub fn closeBlocked(io: std.Io, target: anytype) void` in transport.zig (`target.close(io)` under swapped protection). The inline copies in logging.zig and metrics.zig are out of scope — they are not transport code.
|
||||
- `doh_client.zig` gains the stashed-cause unwrap the other two transports carry (`concreteRead`/`concreteWrite` precedent, dot_client.zig:303-318): today its `mapError` never reads the http client's stashed cause behind `ReadFailed`/`WriteFailed`, so rare mid-exchange local errors (e.g. ENOBUFS) are recorded as peer faults against a healthy upstream — a stated spec-invariant violation, fixed once for DoT and left in DoH. Port the unwrap and the corresponding stub-based tests.
|
||||
- `sendFailure`/`receiveFailure` stay per-file: the unwrapping genuinely differs by stream type.
|
||||
|
||||
### 5. `normalizeName` lives beside `fromText`
|
||||
|
||||
The copies in `records.zig:197` and `forward_zones.zig:129` are
|
||||
byte-identical (only doc comments differ). Move the function to
|
||||
`src/dns/name.zig` as `pub fn normalizeText(text: []const u8, buf:
|
||||
*[types.max_name_len]u8) error{BadName}![]const u8`, next to `fromText`
|
||||
(both callers already import the module; it stays pure — no allocation, no
|
||||
Io). Both files delete their copies and their private `NameError`.
|
||||
The copies in `records.zig:197` and `forward_zones.zig:129` are byte-identical (only doc comments differ). Move the function to `src/dns/name.zig` as `pub fn normalizeText(text: []const u8, buf: *[types.max_name_len]u8) error{BadName}![]const u8`, next to `fromText` (both callers already import the module; it stays pure — no allocation, no Io). Both files delete their copies and their private `NameError`.
|
||||
|
||||
The other three variants are **not** unified — their policies differ on
|
||||
purpose (rules.zig:195 rejects controls and space but skips `fromText`
|
||||
because patterns hold `*`; compiler.zig:155 adds a two-label minimum and
|
||||
counter-based reporting; dns_cache.zig:59 validates nothing because its
|
||||
input is asserted). Each of the three gains a one-line comment pointing at
|
||||
`name.normalizeText` and naming its own policy difference, so the next
|
||||
reader knows the divergence is intentional.
|
||||
The other three variants are **not** unified — their policies differ on purpose (rules.zig:195 rejects controls and space but skips `fromText` because patterns hold `*`; compiler.zig:155 adds a two-label minimum and counter-based reporting; dns_cache.zig:59 validates nothing because its input is asserted). Each of the three gains a one-line comment pointing at `name.normalizeText` and naming its own policy difference, so the next reader knows the divergence is intentional.
|
||||
|
||||
### 6. One bounded-line iterator
|
||||
|
||||
The subtle `takeDelimiter`/`StreamTooLong`/`discardDelimiterInclusive` loop
|
||||
exists twice (`compiler.zig:64-87`, `manager.zig collectSample:1423-1441`),
|
||||
both correct, both carrying the infinite-loop-hazard comment and a
|
||||
regression test. Their seven behavioral differences (termination, counting,
|
||||
trimming, filtering, exit style, error set, arm order) all live *outside*
|
||||
the hazardous core. New shared iterator in `src/filter/parsers.zig`:
|
||||
The subtle `takeDelimiter`/`StreamTooLong`/`discardDelimiterInclusive` loop exists twice (`compiler.zig:64-87`, `manager.zig collectSample:1423-1441`), both correct, both carrying the infinite-loop-hazard comment and a regression test. Their seven behavioral differences (termination, counting, trimming, filtering, exit style, error set, arm order) all live *outside* the hazardous core. New shared iterator in `src/filter/parsers.zig`:
|
||||
|
||||
```zig
|
||||
pub const LineEvent = union(enum) { line: []const u8, long_line };
|
||||
pub fn nextBoundedLine(r: *std.Io.Reader, max_len: usize) error{ReadFailed}!?LineEvent
|
||||
```
|
||||
|
||||
encapsulating the take/discard dance (including the
|
||||
EndOfStream-during-discard arm and the large-reader-buffer length check).
|
||||
The limit is a **parameter** — `parsers.zig` must not import `compiler.zig`
|
||||
(compiler imports parsers, so that is a cycle, and parsers.zig is a
|
||||
standalone std-only fuzz-module root); both callers pass
|
||||
`compiler.max_line_len` from their side. `compile` keeps its `long_lines` counting
|
||||
and `\r` handling on top; `collectSample` keeps its trim/filter/count-limit
|
||||
on top. Both regression tests must still pass; the m15 compiler fuzz target
|
||||
now exercises the shared core.
|
||||
encapsulating the take/discard dance (including the EndOfStream-during-discard arm and the large-reader-buffer length check). The limit is a **parameter** — `parsers.zig` must not import `compiler.zig` (compiler imports parsers, so that is a cycle, and parsers.zig is a standalone std-only fuzz-module root); both callers pass `compiler.max_line_len` from their side. `compile` keeps its `long_lines` counting and `\r` handling on top; `collectSample` keeps its trim/filter/count-limit on top. Both regression tests must still pass; the m15 compiler fuzz target now exercises the shared core.
|
||||
|
||||
### 7. `cli.zig` stops re-spelling app policy
|
||||
|
||||
- The DoH buffer sizes: `app.zig:89-90` names
|
||||
`doh_request_buf_len = 1024` / `doh_transfer_buf_len = 4096` (private);
|
||||
`cli.zig:885-886` re-spells them as bare literals, so changing the
|
||||
constants would leave `nxdns check` probing different buffers than
|
||||
`nxdns run` uses — undermining the probe's stated purpose. The constants
|
||||
move to `doh_client.zig` as `pub const default_request_buf_len` /
|
||||
`default_transfer_buf_len`; both `app.zig` and `cli.zig` use them. No
|
||||
deeper unification of `probeUpstreams` vs `Upstreams.build` — the
|
||||
one-at-a-time probe and the slab build are intentionally different
|
||||
shapes.
|
||||
- The ZON failure channels: `check` prints the multi-line `zon_diag`
|
||||
rendering inline in a single `FAIL` line (cli.zig:721-729), embedding
|
||||
newlines mid-record and contradicting the one-line-per-problem promise
|
||||
(docs/reference/configuration.md:332); `import` routes through
|
||||
`reportParseFailure` (config/import.zig:167 — currently **private**).
|
||||
Make `reportParseFailure` `pub`; `check` builds a local
|
||||
`validate.Diagnostics`, feeds the parse failure through it, and renders
|
||||
each problem as its own `FAIL` line like its other diagnostics.
|
||||
Acceptance: a config with a multi-line ZON error yields one `FAIL` line
|
||||
per parser message from both `check` and `import`.
|
||||
- The DoH buffer sizes: `app.zig:89-90` names `doh_request_buf_len = 1024` / `doh_transfer_buf_len = 4096` (private); `cli.zig:885-886` re-spells them as bare literals, so changing the constants would leave `nxdns check` probing different buffers than `nxdns run` uses — undermining the probe's stated purpose. The constants move to `doh_client.zig` as `pub const default_request_buf_len` / `default_transfer_buf_len`; both `app.zig` and `cli.zig` use them. No deeper unification of `probeUpstreams` vs `Upstreams.build` — the one-at-a-time probe and the slab build are intentionally different shapes.
|
||||
- The ZON failure channels: `check` prints the multi-line `zon_diag` rendering inline in a single `FAIL` line (cli.zig:721-729), embedding newlines mid-record and contradicting the one-line-per-problem promise (docs/reference/configuration.md:332); `import` routes through `reportParseFailure` (config/import.zig:167 — currently **private**). Make `reportParseFailure` `pub`; `check` builds a local `validate.Diagnostics`, feeds the parse failure through it, and renders each problem as its own `FAIL` line like its other diagnostics. Acceptance: a config with a multi-line ZON error yields one `FAIL` line per parser message from both `check` and `import`.
|
||||
|
||||
### 8. `build.zig` wires a test suite once
|
||||
|
||||
The host (lines 47-69) and aarch64 (150-179) test suites repeat twelve
|
||||
wiring lines; the aarch64 triple is re-spelled at :152 instead of read from
|
||||
`cross_targets` (:10-13). Extract `fn addTestSuite(b, target, optimize,
|
||||
options, web_assets) *Step.Compile` mirroring the existing `addExecutable`
|
||||
helper; the aarch64 call resolves its target from `cross_targets[1]` (or a
|
||||
named constant both use). The two blocks' three real differences
|
||||
(`linkage = .static`, `skip_foreign_checks`) stay at the call sites. The
|
||||
m15 fuzz-suite wiring repeats the same shape three times — fold those calls
|
||||
into the helper too if the anonymous-import needs line up; otherwise leave
|
||||
them and say so.
|
||||
The host (lines 47-69) and aarch64 (150-179) test suites repeat twelve wiring lines; the aarch64 triple is re-spelled at :152 instead of read from `cross_targets` (:10-13). Extract `fn addTestSuite(b, target, optimize, options, web_assets) *Step.Compile` mirroring the existing `addExecutable` helper; the aarch64 call resolves its target from `cross_targets[1]` (or a named constant both use). The two blocks' three real differences (`linkage = .static`, `skip_foreign_checks`) stay at the call sites. The m15 fuzz-suite wiring repeats the same shape three times — fold those calls into the helper too if the anonymous-import needs line up; otherwise leave them and say so.
|
||||
|
||||
### 9. One Smith encoder
|
||||
|
||||
`sliceInput` is byte-identical in `blocklist_fuzz.zig:160` and
|
||||
`corpus.zig:123` (blocklist_fuzz cannot import corpus.zig — corpus imports
|
||||
the `dns` module its build target lacks; that is why it was copied). New
|
||||
dependency-free `tests/fuzz/smith_encode.zig` holding `sliceInput` and its
|
||||
length self-test; `corpus.zig` and `blocklist_fuzz.zig` import it by
|
||||
relative path (each fuzz module compiles its own copy — source-level dedup
|
||||
is the goal). `pairInput` stays in blocklist_fuzz, `sliceIntInput` stays in
|
||||
corpus — both build on the shared one. The m15 fuzz files
|
||||
(`compiler_fuzz.zig`, `http_util_fuzz.zig`) adopt it where they inlined
|
||||
the idiom.
|
||||
`sliceInput` is byte-identical in `blocklist_fuzz.zig:160` and `corpus.zig:123` (blocklist_fuzz cannot import corpus.zig — corpus imports the `dns` module its build target lacks; that is why it was copied). New dependency-free `tests/fuzz/smith_encode.zig` holding `sliceInput` and its length self-test; `corpus.zig` and `blocklist_fuzz.zig` import it by relative path (each fuzz module compiles its own copy — source-level dedup is the goal). `pairInput` stays in blocklist_fuzz, `sliceIntInput` stays in corpus — both build on the shared one. The m15 fuzz files (`compiler_fuzz.zig`, `http_util_fuzz.zig`) adopt it where they inlined the idiom.
|
||||
|
||||
### 10. Frontend: one class vocabulary, shared form plumbing
|
||||
|
||||
15 of 30 non-test `.tsx` files re-declare Tailwind class constants in two
|
||||
naming conventions; the primary-button literal appears at 7 sites, the
|
||||
input literal at 6, the focus-visible fragment 39 times across 17 files —
|
||||
and the two inputs that *dropped* the focus ring (PrefixesEditor.tsx:13,
|
||||
GroupsPage.tsx:17, the only unfocusable inputs in the app) silently violate
|
||||
the milestone-9 accessibility floor. The project's own precedent
|
||||
(InlineError was hoisted during milestone 9) is house style left
|
||||
unapplied.
|
||||
15 of 30 non-test `.tsx` files re-declare Tailwind class constants in two naming conventions; the primary-button literal appears at 7 sites, the input literal at 6, the focus-visible fragment 39 times across 17 files — and the two inputs that *dropped* the focus ring (PrefixesEditor.tsx:13, GroupsPage.tsx:17, the only unfocusable inputs in the app) silently violate the milestone-9 accessibility floor. The project's own precedent (InlineError was hoisted during milestone 9) is house style left unapplied.
|
||||
|
||||
- New `web/src/ui/classes.ts` exporting the named constants (camelCase, one
|
||||
convention): `inputClass`, `buttonClass`, `primaryButtonClass`,
|
||||
`thClass`, `tdClass`, `formCardClass`, `tableWrapClass`,
|
||||
`retryButtonClass` — taken verbatim from the current majority literals
|
||||
(all carrying the focus ring). All 15 declaring files adopt; the two
|
||||
ring-less inputs are fixed by adoption. Acceptance is a grep: the three
|
||||
top literals appear only in `ui/classes.ts`, and every
|
||||
input/button/select in `web/src` carries the focus-visible fragment (the
|
||||
one sanctioned variant is PauseWidget's negative offset).
|
||||
- New `web/src/ui/useCrudForm.ts`: the mutation trio + `FormState` +
|
||||
`openForm`/`onSubmit`/`onDelete` plumbing that RecordsTab and ZonesTab
|
||||
share byte-for-byte, parameterized over the entity and its three
|
||||
mutation factories. Both tabs adopt it; their field JSX and tables stay
|
||||
local. Other pages may adopt where the shape fits; none is forced.
|
||||
- The shadowing `InlineError` in DashboardPage.tsx:36: extend
|
||||
`lib/InlineError.tsx` with an optional `onRetry` prop rendering the
|
||||
retry button; DashboardPage imports it and the local copy is deleted.
|
||||
- New `web/src/ui/classes.ts` exporting the named constants (camelCase, one convention): `inputClass`, `buttonClass`, `primaryButtonClass`, `thClass`, `tdClass`, `formCardClass`, `tableWrapClass`, `retryButtonClass` — taken verbatim from the current majority literals (all carrying the focus ring). All 15 declaring files adopt; the two ring-less inputs are fixed by adoption. Acceptance is a grep: the three top literals appear only in `ui/classes.ts`, and every input/button/select in `web/src` carries the focus-visible fragment (the one sanctioned variant is PauseWidget's negative offset).
|
||||
- New `web/src/ui/useCrudForm.ts`: the mutation trio + `FormState` + `openForm`/`onSubmit`/`onDelete` plumbing that RecordsTab and ZonesTab share byte-for-byte, parameterized over the entity and its three mutation factories. Both tabs adopt it; their field JSX and tables stay local. Other pages may adopt where the shape fits; none is forced.
|
||||
- The shadowing `InlineError` in DashboardPage.tsx:36: extend `lib/InlineError.tsx` with an optional `onRetry` prop rendering the retry button; DashboardPage imports it and the local copy is deleted.
|
||||
|
||||
## Sessions
|
||||
|
||||
S1-S7 run in parallel; no two sessions write the same file. Interfaces
|
||||
fixed here: S4 makes the two DoH buffer constants `pub` in
|
||||
`doh_client.zig`; S5 consumes them from `cli.zig`/`app.zig` — wait for S4's
|
||||
commit of that one declaration or agree the exact names above and build
|
||||
against them.
|
||||
S1-S7 run in parallel; no two sessions write the same file. Interfaces fixed here: S4 makes the two DoH buffer constants `pub` in `doh_client.zig`; S5 consumes them from `cli.zig`/`app.zig` — wait for S4's commit of that one declaration or agree the exact names above and build against them.
|
||||
|
||||
### Session S1: listener core
|
||||
|
||||
Owns `src/server/listener.zig` (new), `src/server/tcp_server.zig`,
|
||||
`src/server/dot_server.zig`, `src/server/doh_server.zig`,
|
||||
`src/web/server.zig`, `src/web/metrics.zig`, and the listener integration
|
||||
tests (`tcp_server_integration_test.zig`,
|
||||
`udp_server_integration_test.zig`, `src/web/server_integration_test.zig`,
|
||||
`resolver_integration_test.zig`, `phase7_integration_test.zig` as needed).
|
||||
Ruling 1.
|
||||
Owns `src/server/listener.zig` (new), `src/server/tcp_server.zig`, `src/server/dot_server.zig`, `src/server/doh_server.zig`, `src/web/server.zig`, `src/web/metrics.zig`, and the listener integration tests (`tcp_server_integration_test.zig`, `udp_server_integration_test.zig`, `src/web/server_integration_test.zig`, `resolver_integration_test.zig`, `phase7_integration_test.zig` as needed). Ruling 1.
|
||||
|
||||
### Session S2: storage
|
||||
|
||||
Owns `src/storage/repositories/*` and the milestone-4 spec correction.
|
||||
Ruling 2.
|
||||
Owns `src/storage/repositories/*` and the milestone-4 spec correction. Ruling 2.
|
||||
|
||||
### Session S3: web handlers
|
||||
|
||||
@@ -335,19 +136,11 @@ Owns `src/web/handlers/*`, `src/web/web_integration_test.zig`. Ruling 3.
|
||||
|
||||
### Session S4: transport
|
||||
|
||||
Owns `src/upstream/transport.zig`, `src/upstream/pool.zig`,
|
||||
`src/upstream/dot_client.zig`, `src/upstream/doh_client.zig`,
|
||||
`src/local/forward_client.zig`. Ruling 4, and the `pub` constants half of
|
||||
ruling 7.
|
||||
Owns `src/upstream/transport.zig`, `src/upstream/pool.zig`, `src/upstream/dot_client.zig`, `src/upstream/doh_client.zig`, `src/local/forward_client.zig`. Ruling 4, and the `pub` constants half of ruling 7.
|
||||
|
||||
### Session S5: names, lines, cli
|
||||
|
||||
Owns `src/dns/name.zig`, `src/local/records.zig`,
|
||||
`src/local/forward_zones.zig`, `src/filter/rules.zig`,
|
||||
`src/filter/compiler.zig`, `src/filter/manager.zig`,
|
||||
`src/filter/parsers.zig`, `src/cache/dns_cache.zig` (comment only),
|
||||
`src/cli.zig`, `src/config/import.zig`, `src/app.zig`. Rulings 5, 6, and
|
||||
the consumer half of 7.
|
||||
Owns `src/dns/name.zig`, `src/local/records.zig`, `src/local/forward_zones.zig`, `src/filter/rules.zig`, `src/filter/compiler.zig`, `src/filter/manager.zig`, `src/filter/parsers.zig`, `src/cache/dns_cache.zig` (comment only), `src/cli.zig`, `src/config/import.zig`, `src/app.zig`. Rulings 5, 6, and the consumer half of 7.
|
||||
|
||||
### Session S6: build and fuzz
|
||||
|
||||
@@ -359,19 +152,13 @@ Owns `web/src/**`. Ruling 10.
|
||||
|
||||
### Orchestrator
|
||||
|
||||
Strikes the closed findings in `TECH_DEBT.md`; records the tcp metric
|
||||
rename in the docs reference; updates the milestone-10 deviation note
|
||||
(ruling 1) and the milestone-4 sample (ruling 2).
|
||||
Strikes the closed findings in `TECH_DEBT.md`; records the tcp metric rename in the docs reference; updates the milestone-10 deviation note (ruling 1) and the milestone-4 sample (ruling 2).
|
||||
|
||||
## Module layout
|
||||
|
||||
New files: `src/server/listener.zig`, `tests/fuzz/smith_encode.zig`,
|
||||
`web/src/ui/classes.ts`, `web/src/ui/useCrudForm.ts`.
|
||||
New files: `src/server/listener.zig`, `tests/fuzz/smith_encode.zig`, `web/src/ui/classes.ts`, `web/src/ui/useCrudForm.ts`.
|
||||
|
||||
Deleted surface: the four per-listener copies of the shared machinery, the
|
||||
dead `doh_server.serve`, the two `normalizeName` copies and their
|
||||
`NameError`s, the duplicated listener unit tests, the DashboardPage
|
||||
`InlineError`.
|
||||
Deleted surface: the four per-listener copies of the shared machinery, the dead `doh_server.serve`, the two `normalizeName` copies and their `NameError`s, the duplicated listener unit tests, the DashboardPage `InlineError`.
|
||||
|
||||
## Acceptance (milestone complete)
|
||||
|
||||
@@ -408,143 +195,57 @@ dead `doh_server.serve`, the two `normalizeName` copies and their
|
||||
|
||||
## Recorded (implementation)
|
||||
|
||||
Accepted deviations and findings from the built milestone. Each was
|
||||
reviewed and accepted at integration; the rulings above stand except as
|
||||
recorded here.
|
||||
Accepted deviations and findings from the built milestone. Each was reviewed and accepted at integration; the rulings above stand except as recorded here.
|
||||
|
||||
### Ruling 1 (S1)
|
||||
|
||||
- `Cfg` supplies more than the four listed members: `Owner`,
|
||||
`ConnPayload`, `serveConn`, `read_buffer_len`, `write_buffer_len`,
|
||||
plus `log` (the owner's `std.log` scope) and `name`, because the
|
||||
accept loop and shutdown log and the text had to stay byte-identical.
|
||||
Two optional decls: `refuse` (absent means close the stream; the web
|
||||
listener supplies its 503) and `initPayload`/`deinitPayload`, which
|
||||
exist only for the web arena's create-in-listen / destroy-in-deinit
|
||||
lifecycle.
|
||||
- The read/write staging buffers live in the core's `Conn`, not in
|
||||
`ConnPayload` — all four listeners have exactly one of each and
|
||||
differ only in size. The web listener's `recv_buf`/`send_buf` are now
|
||||
`read_buf`/`write_buf`.
|
||||
- Stats layout: core counters live at `server.core.stats.*`
|
||||
(`listener.CoreStats`); listener-specific counters stay on the owner
|
||||
at `server.stats.*`. `tcp_server.Stats` is an alias of `CoreStats`.
|
||||
Exported snapshots stay flat, so /metrics output is unchanged except
|
||||
the tcp rename. `idle_timeouts` sits in `CoreStats` per the ruling,
|
||||
which gives the web listener a counter it never bumps; it exports no
|
||||
family, so nothing is visible.
|
||||
- The docs-reference update for the rename has no target:
|
||||
`nxdns_tcp_server_accepted_total` appears in no file under docs/ or
|
||||
web/. Only the metrics name test changed.
|
||||
- 18 duplicated unit tests were deleted (6 tcp, 6 dot, 3 doh, 3 web)
|
||||
and replaced by 6 shared claim-rule tests in `listener.zig`.
|
||||
`web/server_integration_test.zig`'s `withServer` now returns a local
|
||||
plain-`u64` `Counters` struct instead of `server.Stats`.
|
||||
- File-ownership breach: S1 edited two files owned by other sessions,
|
||||
both mechanical fallout of the sanctioned `deinit` shape change —
|
||||
three call sites in `src/app.zig` (S5) and one line in
|
||||
`src/web/web_integration_test.zig` (S3). Both owners reviewed and
|
||||
kept the edits. S1 also added the required `src/tests.zig` import
|
||||
line for the new file.
|
||||
- `Cfg` supplies more than the four listed members: `Owner`, `ConnPayload`, `serveConn`, `read_buffer_len`, `write_buffer_len`, plus `log` (the owner's `std.log` scope) and `name`, because the accept loop and shutdown log and the text had to stay byte-identical. Two optional decls: `refuse` (absent means close the stream; the web listener supplies its 503) and `initPayload`/`deinitPayload`, which exist only for the web arena's create-in-listen / destroy-in-deinit lifecycle.
|
||||
- The read/write staging buffers live in the core's `Conn`, not in `ConnPayload` — all four listeners have exactly one of each and differ only in size. The web listener's `recv_buf`/`send_buf` are now `read_buf`/`write_buf`.
|
||||
- Stats layout: core counters live at `server.core.stats.*` (`listener.CoreStats`); listener-specific counters stay on the owner at `server.stats.*`. `tcp_server.Stats` is an alias of `CoreStats`. Exported snapshots stay flat, so /metrics output is unchanged except the tcp rename. `idle_timeouts` sits in `CoreStats` per the ruling, which gives the web listener a counter it never bumps; it exports no family, so nothing is visible.
|
||||
- The docs-reference update for the rename has no target: `nxdns_tcp_server_accepted_total` appears in no file under docs/ or web/. Only the metrics name test changed.
|
||||
- 18 duplicated unit tests were deleted (6 tcp, 6 dot, 3 doh, 3 web) and replaced by 6 shared claim-rule tests in `listener.zig`. `web/server_integration_test.zig`'s `withServer` now returns a local plain-`u64` `Counters` struct instead of `server.Stats`.
|
||||
- File-ownership breach: S1 edited two files owned by other sessions, both mechanical fallout of the sanctioned `deinit` shape change — three call sites in `src/app.zig` (S5) and one line in `src/web/web_integration_test.zig` (S3). Both owners reviewed and kept the edits. S1 also added the required `src/tests.zig` import line for the new file.
|
||||
|
||||
### Ruling 2 (S2)
|
||||
|
||||
- `listRowsBound(comptime Row, database, gpa, comptime sql, args:
|
||||
anytype, comptime readRow)` exists beside `listRows` for the bound
|
||||
queries; `freeRow` exists beside `freeRows`. Unknown owning field
|
||||
shapes are a `@compileError` as ruled. The milestone-4 sample
|
||||
correction was made by S2, not the orchestrator.
|
||||
- `listRowsBound(comptime Row, database, gpa, comptime sql, args: anytype, comptime readRow)` exists beside `listRows` for the bound queries; `freeRow` exists beside `freeRows`. Unknown owning field shapes are a `@compileError` as ruled. The milestone-4 sample correction was made by S2, not the orchestrator.
|
||||
|
||||
### Ruling 3 (S3)
|
||||
|
||||
- `plural` is a separate descriptor member: the "listing X" log context
|
||||
differs from the JSON envelope key for three resources
|
||||
(`local_records`, `client_prefixes`, `forward_zones`), so deriving one
|
||||
from the other would change three log strings.
|
||||
- `view` is an optional descriptor member: rules and local records map
|
||||
rows through `RuleView`/`RecordView`; without it neither could adopt
|
||||
without changing its response body. When absent, the row serializes
|
||||
as-is with no copy.
|
||||
- `remove` has two comptime-detected arities: three handlers' decision
|
||||
functions read rows and take an `Allocator` before the id, three do
|
||||
not. The generator validates the full signature of whichever shape it
|
||||
finds.
|
||||
- `plural` is a separate descriptor member: the "listing X" log context differs from the JSON envelope key for three resources (`local_records`, `client_prefixes`, `forward_zones`), so deriving one from the other would change three log strings.
|
||||
- `view` is an optional descriptor member: rules and local records map rows through `RuleView`/`RecordView`; without it neither could adopt without changing its response body. When absent, the row serializes as-is with no copy.
|
||||
- `remove` has two comptime-detected arities: three handlers' decision functions read rows and take an `Allocator` before the id, three do not. The generator validates the full signature of whichever shape it finds.
|
||||
|
||||
### Ruling 4 (S4)
|
||||
|
||||
- The ruling's line numbers were stale after m17 (pool.zig's race sites
|
||||
were at 184-206 and 306-335).
|
||||
- `closeBlocked` branches at comptime on the close method's parameter
|
||||
count: `tls_client.TlsStream.close()` takes no `Io` (it owns the one
|
||||
it was built with), while the other three closes take `io`.
|
||||
- "The stashed-cause accessor" is three accessors, one per collapse
|
||||
point: the send phase reads `req.connection.?.stream_writer.err`,
|
||||
`receiveHead` reads `Connection.getReadError()`, and the body read
|
||||
consults `Response.bodyErr()` first (HTTP framing faults) then the
|
||||
connection.
|
||||
- `Connection.getReadError` can panic (it reads `stream_reader.err.?`).
|
||||
The unwrap guards the plain-connection no-cause case; the TLS case
|
||||
relies on std's documented contract that a cause exists after
|
||||
`error.ReadFailed`.
|
||||
- `raceWithin` requires the raced function's return type to be exactly
|
||||
`ExchangeError!T` at comptime. Stricter than the ruling asked; it is
|
||||
what keeps every failure path inside the race group. Consequence:
|
||||
`manager.zig`'s `fetchWithin` (optional per the ruling) cannot adopt
|
||||
it as written — its raced function has a different error set.
|
||||
- The acceptance line "`fn expire(` production copies are gone" is
|
||||
scoped to ruling 4's transport files. `filter/manager.zig`,
|
||||
`storage/logger.zig` and `server/listener.zig` keep their own — none
|
||||
is transport code.
|
||||
- The ruling's line numbers were stale after m17 (pool.zig's race sites were at 184-206 and 306-335).
|
||||
- `closeBlocked` branches at comptime on the close method's parameter count: `tls_client.TlsStream.close()` takes no `Io` (it owns the one it was built with), while the other three closes take `io`.
|
||||
- "The stashed-cause accessor" is three accessors, one per collapse point: the send phase reads `req.connection.?.stream_writer.err`, `receiveHead` reads `Connection.getReadError()`, and the body read consults `Response.bodyErr()` first (HTTP framing faults) then the connection.
|
||||
- `Connection.getReadError` can panic (it reads `stream_reader.err.?`). The unwrap guards the plain-connection no-cause case; the TLS case relies on std's documented contract that a cause exists after `error.ReadFailed`.
|
||||
- `raceWithin` requires the raced function's return type to be exactly `ExchangeError!T` at comptime. Stricter than the ruling asked; it is what keeps every failure path inside the race group. Consequence: `manager.zig`'s `fetchWithin` (optional per the ruling) cannot adopt it as written — its raced function has a different error set.
|
||||
- The acceptance line "`fn expire(` production copies are gone" is scoped to ruling 4's transport files. `filter/manager.zig`, `storage/logger.zig` and `server/listener.zig` keep their own — none is transport code.
|
||||
|
||||
### Rulings 5, 6, 7 (S5, S4)
|
||||
|
||||
- `nextBoundedLine` returns `.long_line` from the
|
||||
EndOfStream-during-discard arm, and the next call returns `null`. The
|
||||
two originals disagreed there (`compile` counted then broke;
|
||||
`collectSample` returned without counting); this shape preserves both
|
||||
behaviors — `compile` counts exactly as before, `collectSample`
|
||||
ignores the event and sees `null`. Safe because
|
||||
`discardDelimiterInclusive` drains the stream before reporting
|
||||
`EndOfStream`.
|
||||
- The length check runs on the raw line for both callers, as the ruling
|
||||
placed it. One input changes classification: a line of exactly
|
||||
`max_line_len + 1` bytes ending in `\r` was compiled before and now
|
||||
counts as `long_lines`. Accepted as the intended reading.
|
||||
- `check`'s per-message `FAIL` lines render import's path constant
|
||||
(`FAIL config: <line:col: message>`), not the file name. `checkImpl`
|
||||
prints the file path immediately above and `check` examines one
|
||||
source per run, and this keeps `check` and `import` rendering the
|
||||
same failure identically — the acceptance criterion.
|
||||
- `nextBoundedLine` returns `.long_line` from the EndOfStream-during-discard arm, and the next call returns `null`. The two originals disagreed there (`compile` counted then broke; `collectSample` returned without counting); this shape preserves both behaviors — `compile` counts exactly as before, `collectSample` ignores the event and sees `null`. Safe because `discardDelimiterInclusive` drains the stream before reporting `EndOfStream`.
|
||||
- The length check runs on the raw line for both callers, as the ruling placed it. One input changes classification: a line of exactly `max_line_len + 1` bytes ending in `\r` was compiled before and now counts as `long_lines`. Accepted as the intended reading.
|
||||
- `check`'s per-message `FAIL` lines render import's path constant (`FAIL config: <line:col: message>`), not the file name. `checkImpl` prints the file path immediately above and `check` examines one source per run, and this keeps `check` and `import` rendering the same failure identically — the acceptance criterion.
|
||||
|
||||
### Rulings 8, 9 (S6)
|
||||
|
||||
- The fuzz-suite wiring did not fold into `addTestSuite` (the ruling's
|
||||
own fallback): it lives in a separate `addFuzzSuite` helper, with a
|
||||
shared `sourceModule` helper. The aarch64 target resolves from
|
||||
`cross_targets[1]` behind a comptime prefix guard.
|
||||
- `pairInput` moved into `smith_encode.zig` rather than staying in
|
||||
blocklist_fuzz: m15's fuzz files had made it a two-copy duplicate,
|
||||
which is the condition ruling 9 exists to remove.
|
||||
- The fuzz-suite wiring did not fold into `addTestSuite` (the ruling's own fallback): it lives in a separate `addFuzzSuite` helper, with a shared `sourceModule` helper. The aarch64 target resolves from `cross_targets[1]` behind a comptime prefix guard.
|
||||
- `pairInput` moved into `smith_encode.zig` rather than staying in blocklist_fuzz: m15's fuzz files had made it a two-copy duplicate, which is the condition ruling 9 exists to remove.
|
||||
|
||||
### Ruling 10 (S7)
|
||||
|
||||
- `ui/classes.ts` exports 17 constants, not the 8 listed — the extra
|
||||
ones are the focus-ring fragment and compositions the 15 adopting
|
||||
files needed to drop their literals without re-spelling anything.
|
||||
- Seven ring-less focusable controls were fixed by adoption, not the
|
||||
two the ruling named. The acceptance grep ("every input/button/select
|
||||
carries the focus-visible fragment") was taken as the invariant over
|
||||
the anti-requirement's count of two, which undercounted.
|
||||
- `ui/classes.ts` exports 17 constants, not the 8 listed — the extra ones are the focus-ring fragment and compositions the 15 adopting files needed to drop their literals without re-spelling anything.
|
||||
- Seven ring-less focusable controls were fixed by adoption, not the two the ruling named. The acceptance grep ("every input/button/select carries the focus-visible fragment") was taken as the invariant over the anti-requirement's count of two, which undercounted.
|
||||
|
||||
## Anti-requirements
|
||||
|
||||
- No behavioral changes: this milestone moves code. The only sanctioned
|
||||
behavior deltas are the tcp counter rename, DoH's cause unwrap, the two
|
||||
restored focus rings, and the `check` ZON rendering — each named above.
|
||||
- No behavioral changes: this milestone moves code. The only sanctioned behavior deltas are the tcp counter rename, DoH's cause unwrap, the two restored focus rings, and the `check` ZON rendering — each named above.
|
||||
- No unification of the three intentionally-divergent normalize policies.
|
||||
- No unification of the four reload flavors.
|
||||
- No `web/src/ui/` component library beyond `classes.ts` and
|
||||
`useCrudForm.ts` — no button/table/dialog components in this milestone.
|
||||
- No listener behavior additions (timeouts, keepalive) — m16 finished
|
||||
those.
|
||||
- No `web/src/ui/` component library beyond `classes.ts` and `useCrudForm.ts` — no button/table/dialog components in this milestone.
|
||||
- No listener behavior additions (timeouts, keepalive) — m16 finished those.
|
||||
- No renaming of exported metric families beyond the one tcp rename.
|
||||
|
||||
+70
-330
@@ -1,212 +1,92 @@
|
||||
# Milestone 19: hygiene sweep
|
||||
|
||||
Goal: remove the dead surface, unify the re-hardcoded constants, close the
|
||||
small silent failures, and fix the frontend state hazards — the residue of
|
||||
`TECH_DEBT.md` Theme 8 plus the remaining lows, after the behavioral fixes
|
||||
(m16), the contract work (m17) and the duplication refactors (m18) have
|
||||
landed.
|
||||
Goal: remove the dead surface, unify the re-hardcoded constants, close the small silent failures, and fix the frontend state hazards — the residue of `TECH_DEBT.md` Theme 8 plus the remaining lows, after the behavioral fixes (m16), the contract work (m17) and the duplication refactors (m18) have landed.
|
||||
|
||||
**PROVISIONAL.** This spec was written before milestones 16-18 were built. It
|
||||
assumes: m17 shipped BADVERS (so `edns.extendedRcode` has a production
|
||||
caller), m17 built the upstream editor (so the queries.ts upstream factories
|
||||
have consumers and are not deleted here), and m18 shipped the frontend `ui/`
|
||||
extraction and deleted doh_server's dead `serve()`. Before starting this
|
||||
milestone, re-verify every line reference and update this spec where the
|
||||
earlier milestones moved things.
|
||||
**PROVISIONAL.** This spec was written before milestones 16-18 were built. It assumes: m17 shipped BADVERS (so `edns.extendedRcode` has a production caller), m17 built the upstream editor (so the queries.ts upstream factories have consumers and are not deleted here), and m18 shipped the frontend `ui/` extraction and deleted doh_server's dead `serve()`. Before starting this milestone, re-verify every line reference and update this spec where the earlier milestones moved things.
|
||||
|
||||
## Rulings (binding)
|
||||
|
||||
### 1. The dead ECS parse surface is deleted
|
||||
|
||||
`src/dns/edns.zig`: delete `parseEcs` (:111), `Ecs` (:98-105), `EcsError`
|
||||
(:107), `ecs_family_ipv4`/`ecs_family_ipv6` (:95-96), and their in-file tests
|
||||
(:371-:469 block, nine tests). `stripEcs` filters by option code without
|
||||
decoding, forward mode keys on raw bytes — no production path decodes ECS.
|
||||
`src/dns/edns.zig`: delete `parseEcs` (:111), `Ecs` (:98-105), `EcsError` (:107), `ecs_family_ipv4`/`ecs_family_ipv6` (:95-96), and their in-file tests (:371-:469 block, nine tests). `stripEcs` filters by option code without decoding, forward mode keys on raw bytes — no production path decodes ECS.
|
||||
|
||||
Trap the audit missed: `tests/fuzz/dns_fuzz.zig:86` calls `edns.parseEcs`
|
||||
inside a fuzz target that the default `test` step builds. Delete that line in
|
||||
the same change or the build breaks.
|
||||
Trap the audit missed: `tests/fuzz/dns_fuzz.zig:86` calls `edns.parseEcs` inside a fuzz target that the default `test` step builds. Delete that line in the same change or the build breaks.
|
||||
|
||||
`extendedRcode` (:272) is **kept**: milestone 17 gave it the BADVERS caller.
|
||||
If m17 diverged and BADVERS was not built, delete it too and note the
|
||||
divergence here.
|
||||
`extendedRcode` (:272) is **kept**: milestone 17 gave it the BADVERS caller. If m17 diverged and BADVERS was not built, delete it too and note the divergence here.
|
||||
|
||||
Precedent: `ecsPayload` was built and removed within milestone 7 when it lost
|
||||
its only caller (specs/milestone-7.md:361). Git history keeps the code.
|
||||
Precedent: `ecsPayload` was built and removed within milestone 7 when it lost its only caller (specs/milestone-7.md:361). Git history keeps the code.
|
||||
|
||||
### 2. `writeSettings` returns an error union
|
||||
|
||||
`src/web/handlers/settings.zig:349` returns `?db.Error`, so every exit is a
|
||||
value return and the `errdefer` at :354 never fires — it implies protection
|
||||
it cannot provide. The real safety is two explicit `tx.rollback()` calls
|
||||
(:358, :363 — the audit said three; it is two).
|
||||
`src/web/handlers/settings.zig:349` returns `?db.Error`, so every exit is a value return and the `errdefer` at :354 never fires — it implies protection it cannot provide. The real safety is two explicit `tx.rollback()` calls (:358, :363 — the audit said three; it is two).
|
||||
|
||||
Change the return type to `db.Error!void`. The errdefer becomes live
|
||||
(`db.Tx.rollback` at storage/db.zig:814 is idempotent and documented safe in
|
||||
an errdefer), and the two manual rollback-then-return arms collapse into
|
||||
plain `try`/error returns. The one caller (`applyPut`, :327) switches from
|
||||
`if (writeSettings(...)) |err|` to `catch`; its hash-free-on-failure behavior
|
||||
is unchanged.
|
||||
Change the return type to `db.Error!void`. The errdefer becomes live (`db.Tx.rollback` at storage/db.zig:814 is idempotent and documented safe in an errdefer), and the two manual rollback-then-return arms collapse into plain `try`/error returns. The one caller (`applyPut`, :327) switches from `if (writeSettings(...)) |err|` to `catch`; its hash-free-on-failure behavior is unchanged.
|
||||
|
||||
### 3. The logger's field widths become the single source
|
||||
|
||||
`src/storage/logger.zig:44-48` holds four file-private constants
|
||||
(`max_domain_len = 253`, `max_client_len = 45`, `max_reason_len = 32`,
|
||||
`max_upstream_len = 64`). Make all four `pub`. Then:
|
||||
`src/storage/logger.zig:44-48` holds four file-private constants (`max_domain_len = 253`, `max_client_len = 45`, `max_reason_len = 32`, `max_upstream_len = 64`). Make all four `pub`. Then:
|
||||
|
||||
- `src/server/handler.zig:67` deletes its local `max_reason_len = 32`; the
|
||||
comptime guard at :69-75 checks `logger.max_reason_len`, so the guard
|
||||
finally proves what its comment claims.
|
||||
- `src/server/handler.zig:79` deletes its local `max_ip_text = 45` and uses
|
||||
`logger.max_client_len` (the :82 `max_resolver_text` derivation follows).
|
||||
- `src/server/handler.zig:67` deletes its local `max_reason_len = 32`; the comptime guard at :69-75 checks `logger.max_reason_len`, so the guard finally proves what its comment claims.
|
||||
- `src/server/handler.zig:79` deletes its local `max_ip_text = 45` and uses `logger.max_client_len` (the :82 `max_resolver_text` derivation follows).
|
||||
- `src/server/clients.zig:32` likewise.
|
||||
- `src/web/handlers/queries.zig:27-30` — the fifth copy the audit missed —
|
||||
adopts `logger.max_domain_len` for the domain width. Its `max_client_len`
|
||||
is **64**, not 45; the query log's client column is written by the logger
|
||||
and can never exceed 45 bytes, so it adopts `logger.max_client_len` too.
|
||||
If a test proves a wider value ever reaches that field, stop and record
|
||||
the divergence instead of forcing it.
|
||||
- `src/web/handlers/queries.zig:27-30` — the fifth copy the audit missed — adopts `logger.max_domain_len` for the domain width. Its `max_client_len` is **64**, not 45; the query log's client column is written by the logger and can never exceed 45 bytes, so it adopts `logger.max_client_len` too. If a test proves a wider value ever reaches that field, stop and record the divergence instead of forcing it.
|
||||
|
||||
Acceptance is a grep: the literals 253, 45 and 32 appear in these roles only
|
||||
in logger.zig.
|
||||
Acceptance is a grep: the literals 253, 45 and 32 appear in these roles only in logger.zig.
|
||||
|
||||
### 4. `track()` reports fullness; one mutex acquisition per query
|
||||
|
||||
`src/server/handler.zig:255-258` takes the tracker mutex twice per query:
|
||||
`tracker.track(io, from)` and then `tracker.snapshotStats(io)` — a full
|
||||
struct copy — solely to mirror `dropped_full` into the handler's atomic
|
||||
`tracker_full` (:150-153).
|
||||
`src/server/handler.zig:255-258` takes the tracker mutex twice per query: `tracker.track(io, from)` and then `tracker.snapshotStats(io)` — a full struct copy — solely to mirror `dropped_full` into the handler's atomic `tracker_full` (:150-153).
|
||||
|
||||
Change `Tracker.track`/`trackAt` (`src/server/clients.zig:89-97`) to return
|
||||
`u64`: the current `dropped_full`, read under the mutex they already hold.
|
||||
The handler stores that return value. `snapshotStats` stays for /metrics.
|
||||
Change `Tracker.track`/`trackAt` (`src/server/clients.zig:89-97`) to return `u64`: the current `dropped_full`, read under the mutex they already hold. The handler stores that return value. `snapshotStats` stays for /metrics.
|
||||
|
||||
### 5. Log truncation gets a marker and a counter
|
||||
|
||||
`src/platform/logging.zig:285-288`: `mw.print(format, args) catch {}` on a
|
||||
fixed 2048-byte writer, then the partial buffer is used with no marker — the
|
||||
module's own never-both-discarded-and-silent invariant, violated in its own
|
||||
kitchen. On `error.WriteFailed`: overwrite the final three bytes of the
|
||||
buffered message with `...` (the safe_url.zig marker, src/safe_url.zig:11-14)
|
||||
and increment a new `state.stats.lines_truncated`, sibling to
|
||||
`lines_deduped` (:295). One test: a message over 2048 bytes ends in `...`
|
||||
and bumps the counter by one.
|
||||
`src/platform/logging.zig:285-288`: `mw.print(format, args) catch {}` on a fixed 2048-byte writer, then the partial buffer is used with no marker — the module's own never-both-discarded-and-silent invariant, violated in its own kitchen. On `error.WriteFailed`: overwrite the final three bytes of the buffered message with `...` (the safe_url.zig marker, src/safe_url.zig:11-14) and increment a new `state.stats.lines_truncated`, sibling to `lines_deduped` (:295). One test: a message over 2048 bytes ends in `...` and bumps the counter by one.
|
||||
|
||||
### 6. TLS errors are classified by name, not by name prefix
|
||||
|
||||
`src/filter/fetcher.zig:169-170` and `src/upstream/doh_client.zig:160-161`
|
||||
classify TLS failures with `startsWith(@errorName(err), "Tls")` /
|
||||
`"Certificate"` — a std rename silently downgrades TLS failures into generic
|
||||
buckets. Verified reachable set from `std.http.Client` (0.16.0) **today**:
|
||||
exactly `error.TlsInitializationFailed` and
|
||||
`error.CertificateBundleLoadFailure`. But this milestone lands after
|
||||
milestone 18, whose ruling 4 makes the DoH path unwrap the client's stashed
|
||||
read cause — and that cause set includes the record-layer members
|
||||
(`TlsAlert`, `TlsBadRecordMac`, `TlsDecodeError`, ...) from
|
||||
`std.crypto.tls.Client`. A two-member switch written against today's
|
||||
surface would downgrade those to generic receive failures.
|
||||
`src/filter/fetcher.zig:169-170` and `src/upstream/doh_client.zig:160-161` classify TLS failures with `startsWith(@errorName(err), "Tls")` / `"Certificate"` — a std rename silently downgrades TLS failures into generic buckets. Verified reachable set from `std.http.Client` (0.16.0) **today**: exactly `error.TlsInitializationFailed` and `error.CertificateBundleLoadFailure`. But this milestone lands after milestone 18, whose ruling 4 makes the DoH path unwrap the client's stashed read cause — and that cause set includes the record-layer members (`TlsAlert`, `TlsBadRecordMac`, `TlsDecodeError`, ...) from `std.crypto.tls.Client`. A two-member switch written against today's surface would downgrade those to generic receive failures.
|
||||
|
||||
In both files: replace the prefix match with an exact switch → `error.TlsFailed`
|
||||
naming the two top-level members **plus**, on whichever paths m18's cause
|
||||
unwrap surfaces, the `Tls*` members of the unwrapped read-cause set —
|
||||
enumerated at implementation time from the pinned std sources against the
|
||||
m18 code as landed, not from this spec's list. The `else` falls through to
|
||||
the existing per-phase mapping. Rewrite both tests (fetcher.zig:311-313,
|
||||
doh_client.zig:261-263): they currently assert on `CertificateExpired`,
|
||||
which nothing can produce, and omit `CertificateBundleLoadFailure`; the new
|
||||
tests use only names from the verified reachable set, including a stashed
|
||||
record-layer cause on the unwrapped path.
|
||||
In both files: replace the prefix match with an exact switch → `error.TlsFailed` naming the two top-level members **plus**, on whichever paths m18's cause unwrap surfaces, the `Tls*` members of the unwrapped read-cause set — enumerated at implementation time from the pinned std sources against the m18 code as landed, not from this spec's list. The `else` falls through to the existing per-phase mapping. Rewrite both tests (fetcher.zig:311-313, doh_client.zig:261-263): they currently assert on `CertificateExpired`, which nothing can produce, and omit `CertificateBundleLoadFailure`; the new tests use only names from the verified reachable set, including a stashed record-layer cause on the unwrapped path.
|
||||
|
||||
If m18 already extracted a shared `mapError` into transport.zig, apply this
|
||||
ruling to the shared copy instead and note it here.
|
||||
If m18 already extracted a shared `mapError` into transport.zig, apply this ruling to the shared copy instead and note it here.
|
||||
|
||||
### 7. The transcribed mbedTLS values get an init-time check
|
||||
|
||||
`src/platform/tls_server.zig:501-511` hand-transcribes four config enums and
|
||||
six error codes from the pinned Mbed TLS 3.6.7 headers with no drift guard —
|
||||
unlike the sizes and alignment, which the shim reports. A renumbered
|
||||
`close_notify` after a version bump would silently invert truncation
|
||||
detection (:286, :351).
|
||||
`src/platform/tls_server.zig:501-511` hand-transcribes four config enums and six error codes from the pinned Mbed TLS 3.6.7 headers with no drift guard — unlike the sizes and alignment, which the shim reports. A renumbered `close_notify` after a version bump would silently invert truncation detection (:286, :351).
|
||||
|
||||
`src/platform/mbedtls_shim.c` gains ten getters in the existing shape
|
||||
(`int nx_const_ssl_transport_stream(void) { return MBEDTLS_SSL_TRANSPORT_STREAM; }`
|
||||
etc. — the needed headers are already included). `tls_server.zig` verifies
|
||||
all ten against the transcribed constants next to the existing alignment
|
||||
assert at :78, and the :657 test block asserts them too. Cheap insurance,
|
||||
one-time cost.
|
||||
`src/platform/mbedtls_shim.c` gains ten getters in the existing shape (`int nx_const_ssl_transport_stream(void) { return MBEDTLS_SSL_TRANSPORT_STREAM; }` etc. — the needed headers are already included). `tls_server.zig` verifies all ten against the transcribed constants next to the existing alignment assert at :78, and the :657 test block asserts them too. Cheap insurance, one-time cost.
|
||||
|
||||
### 8. The reload lock invariant is written down
|
||||
|
||||
Fourteen call sites in four handler files (groups, rules, clients,
|
||||
blocklists — the audit said five handlers; it is four files, fourteen sites)
|
||||
call `mutations.reload` *after* releasing `config_lock`. That is safe only
|
||||
because the production `reload_fn` re-reads the database under the manager's
|
||||
own writer lock — a cross-module fact stated nowhere. `local.zig` is the
|
||||
deliberate opposite (publish under the lock; the ordering rationale lives at
|
||||
local.zig:80-85).
|
||||
Fourteen call sites in four handler files (groups, rules, clients, blocklists — the audit said five handlers; it is four files, fourteen sites) call `mutations.reload` *after* releasing `config_lock`. That is safe only because the production `reload_fn` re-reads the database under the manager's own writer lock — a cross-module fact stated nowhere. `local.zig` is the deliberate opposite (publish under the lock; the ordering rationale lives at local.zig:80-85).
|
||||
|
||||
Extend the doc comment on `mutations.reload`
|
||||
(src/web/handlers/mutations.zig:122-125): the `reload_fn` contract is that it
|
||||
re-reads all state from the database itself; it must never accept pre-read
|
||||
rows (contrast `swapLocalTables`, which is the pre-read shape and is why
|
||||
local.zig holds the lock). A future signature change to `reload` that adds
|
||||
row-passing breaks fourteen call sites' correctness — the comment must say
|
||||
exactly that. Documentation only; no locking change.
|
||||
Extend the doc comment on `mutations.reload` (src/web/handlers/mutations.zig:122-125): the `reload_fn` contract is that it re-reads all state from the database itself; it must never accept pre-read rows (contrast `swapLocalTables`, which is the pre-read shape and is why local.zig holds the lock). A future signature change to `reload` that adds row-passing breaks fourteen call sites' correctness — the comment must say exactly that. Documentation only; no locking change.
|
||||
|
||||
### 9. `web_dev_dir` moves into WebState
|
||||
|
||||
`src/app.zig:605-609` holds the `--web-dev` directory in a module-level
|
||||
mutable global because the fallback handler "has no closure" — but
|
||||
`serveWebDev` (:613-620) receives `*WebState` and discards it. Add
|
||||
`dev_dir: []const u8 = ""` to `WebState` (src/web/server.zig, beside
|
||||
`version`), set it at the composition root, read it in `serveWebDev`, delete
|
||||
the global and its apologia comment.
|
||||
`src/app.zig:605-609` holds the `--web-dev` directory in a module-level mutable global because the fallback handler "has no closure" — but `serveWebDev` (:613-620) receives `*WebState` and discards it. Add `dev_dir: []const u8 = ""` to `WebState` (src/web/server.zig, beside `version`), set it at the composition root, read it in `serveWebDev`, delete the global and its apologia comment.
|
||||
|
||||
### 10. The private key PEM is wiped
|
||||
|
||||
`src/server/cert_store.zig:326` frees the key PEM without zeroization, and
|
||||
`readPem` (:345-362) uses `readFileAllocOptions`, whose grow-as-you-read can
|
||||
leave intermediate copies in freed pages — a wipe at the call site cannot
|
||||
reach them. Split the key path: a `readKeyPem` that reads through a single
|
||||
fixed `max_pem_bytes + 1` buffer (no reallocation), copies to an exact-size
|
||||
allocation, and `std.crypto.secureZero`s the big buffer before returning; the
|
||||
caller wipes the returned slice before `gpa.free`. The idiom precedent is
|
||||
auth.zig:228. The certificate PEM path stays as-is — it is public material.
|
||||
Runs at boot and on real renewals only (:311-312 stats gate the poll).
|
||||
`src/server/cert_store.zig:326` frees the key PEM without zeroization, and `readPem` (:345-362) uses `readFileAllocOptions`, whose grow-as-you-read can leave intermediate copies in freed pages — a wipe at the call site cannot reach them. Split the key path: a `readKeyPem` that reads through a single fixed `max_pem_bytes + 1` buffer (no reallocation), copies to an exact-size allocation, and `std.crypto.secureZero`s the big buffer before returning; the caller wipes the returned slice before `gpa.free`. The idiom precedent is auth.zig:228. The certificate PEM path stays as-is — it is public material. Runs at boot and on real renewals only (:311-312 stats gate the poll).
|
||||
|
||||
### 11. Header-array overflow asserts
|
||||
|
||||
`src/web/http_util.zig:335` maps a too-long `extra_headers` onto
|
||||
`error.OutOfMemory` — a future eighth header would surface as mysterious
|
||||
OOM-labeled drops. Every one of the 62 call sites passes a compile-time
|
||||
count, maximum 3 (static.zig:174, +1 for content-type = 4 of 8). Replace the
|
||||
check with `std.debug.assert(extra_headers.len + 1 <= headers.len);` so the
|
||||
mistake fails loudly in tests.
|
||||
`src/web/http_util.zig:335` maps a too-long `extra_headers` onto `error.OutOfMemory` — a future eighth header would surface as mysterious OOM-labeled drops. Every one of the 62 call sites passes a compile-time count, maximum 3 (static.zig:174, +1 for content-type = 4 of 8). Replace the check with `std.debug.assert(extra_headers.len + 1 <= headers.len);` so the mistake fails loudly in tests.
|
||||
|
||||
### 12. Shipped `.gz` siblings are verified; orphans are rejected
|
||||
|
||||
`tools/gen_web_assets.zig:71-77` embeds a dist-shipped `.gz` sibling with
|
||||
zero content checks, and an orphan `.gz` (no base file) is silently embedded
|
||||
as unreachable `application/octet-stream` bytes. Exposure is latent (no
|
||||
frontend compression plugin today) — close it while it is cheap:
|
||||
`tools/gen_web_assets.zig:71-77` embeds a dist-shipped `.gz` sibling with zero content checks, and an orphan `.gz` (no base file) is silently embedded as unreachable `application/octet-stream` bytes. Exposure is latent (no frontend compression plugin today) — close it while it is cheap:
|
||||
|
||||
- Sibling: decompress with `std.compress.flate.Decompress` (container
|
||||
`.gzip`; the tool already uses the Compress side at :133) and byte-compare
|
||||
against the base file; `std.process.fatal` on mismatch, matching the
|
||||
tool's existing error style (:73-75).
|
||||
- Sibling: decompress with `std.compress.flate.Decompress` (container `.gzip`; the tool already uses the Compress side at :133) and byte-compare against the base file; `std.process.fatal` on mismatch, matching the tool's existing error style (:73-75).
|
||||
- Orphan: `std.process.fatal` naming the path, instead of indexing it.
|
||||
|
||||
Extend the embedded-dist test (`src/web/static.zig:413-439`) to decompress
|
||||
each `.gz` entry and compare with its base — it currently checks magic bytes
|
||||
and size only.
|
||||
Extend the embedded-dist test (`src/web/static.zig:413-439`) to decompress each `.gz` entry and compare with its base — it currently checks magic bytes and size only.
|
||||
|
||||
### 13. The Dockerfile arch default comes from the host
|
||||
|
||||
`deploy/docker/Dockerfile:21`: `${TARGETARCH:-amd64}` — under the legacy
|
||||
builder TARGETARCH is empty, so a plain `docker build` on the Pi 5 packages
|
||||
the x86_64 binary and dies at `docker run` with exec-format, far from the
|
||||
mistake. Replace the default with a `uname -m` mapping resolved before the
|
||||
case:
|
||||
`deploy/docker/Dockerfile:21`: `${TARGETARCH:-amd64}` — under the legacy builder TARGETARCH is empty, so a plain `docker build` on the Pi 5 packages the x86_64 binary and dies at `docker run` with exec-format, far from the mistake. Replace the default with a `uname -m` mapping resolved before the case:
|
||||
|
||||
```dockerfile
|
||||
RUN arch="${TARGETARCH:-}"; \
|
||||
@@ -221,10 +101,7 @@ The existing unsupported-arch arm stays fail-fast.
|
||||
|
||||
### 14. Frontend: the settings registry is typed
|
||||
|
||||
`web/src/features/settings/SettingsPage.tsx` — `SectionDef` exists (:21) but
|
||||
`FieldDef.key` is `string` (:16), driving `as Record<string, unknown>` casts
|
||||
at :217, :226, :261. A typo'd key compiles and breaks the field. Make the
|
||||
pair generic:
|
||||
`web/src/features/settings/SettingsPage.tsx` — `SectionDef` exists (:21) but `FieldDef.key` is `string` (:16), driving `as Record<string, unknown>` casts at :217, :226, :261. A typo'd key compiles and breaks the field. Make the pair generic:
|
||||
|
||||
```ts
|
||||
interface FieldDef<S extends keyof Settings> {
|
||||
@@ -235,45 +112,19 @@ interface SectionDef<S extends keyof Settings> { section: S; title: string; fiel
|
||||
function defineSection<S extends keyof Settings>(def: SectionDef<S>) { return def; }
|
||||
```
|
||||
|
||||
`SECTIONS` (:35-118, 12 sections) becomes a tuple of `defineSection(...)`
|
||||
calls, `TLS_FIELDS` (:27) becomes `FieldDef<"doh_server" | "dot_server">[]`
|
||||
compatible, and the three `Record<string, unknown>` casts are deleted. The
|
||||
per-branch value casts inside `FieldRow` (:141, :160, :174, :198) may stay —
|
||||
they are narrowing on `kind`, not drift holes. Acceptance: renaming a
|
||||
Settings field makes `tsc` fail on the registry.
|
||||
`SECTIONS` (:35-118, 12 sections) becomes a tuple of `defineSection(...)` calls, `TLS_FIELDS` (:27) becomes `FieldDef<"doh_server" | "dot_server">[]` compatible, and the three `Record<string, unknown>` casts are deleted. The per-branch value casts inside `FieldRow` (:141, :160, :174, :198) may stay — they are narrowing on `kind`, not drift holes. Acceptance: renaming a Settings field makes `tsc` fail on the registry.
|
||||
|
||||
### 15. Frontend: the settings baseline is frozen
|
||||
|
||||
Same file, :220 — `buildSettingsPatch(data.settings, edited, ...)` diffs the
|
||||
mount-time clone against **live** query data, so a background refetch makes
|
||||
out-of-band changes appear as user edits and Save silently reverts them. Add
|
||||
a `baseline` state initialized with the same `structuredClone` and reset in
|
||||
the same `onSuccess` (:234-241) where `edited` resets; the patch becomes
|
||||
`buildSettingsPatch(baseline, edited, ...)`. `settingsDiff.ts` is unchanged.
|
||||
Same file, :220 — `buildSettingsPatch(data.settings, edited, ...)` diffs the mount-time clone against **live** query data, so a background refetch makes out-of-band changes appear as user edits and Save silently reverts them. Add a `baseline` state initialized with the same `structuredClone` and reset in the same `onSuccess` (:234-241) where `edited` resets; the patch becomes `buildSettingsPatch(baseline, edited, ...)`. `settingsDiff.ts` is unchanged.
|
||||
|
||||
### 16. Frontend: refresh status leaves the query cache
|
||||
|
||||
`web/src/features/blocklists/BlocklistsPage.tsx:33` reads the refresh
|
||||
snapshot with non-subscribing `getQueryData` from a cache entry that has no
|
||||
queryFn, no subscriber, and the default 5-minute gcTime — after which the
|
||||
page claims no refresh ever ran. This is client UI state. New module
|
||||
`web/src/features/blocklists/refreshStore.ts` in the exact shape of
|
||||
`settings/restartBanner.ts` (module-level value + listener Set +
|
||||
`useSyncExternalStore` hook), holding `SourceStatus[] | null`. The mutation
|
||||
(`blocklistsUpdateNowMutation`, lib/queries.ts:156) writes the store instead
|
||||
of `setQueryData`; the page subscribes via the hook; the
|
||||
`queryKeys.blocklistSources` entry and its doc comment are deleted. Note the
|
||||
prefix-invalidation side effect disappears with it (invalidateBlocklistWorld
|
||||
invalidating `["blocklists"]` used to mark the entry stale — harmless, since
|
||||
the entry could never refetch).
|
||||
`web/src/features/blocklists/BlocklistsPage.tsx:33` reads the refresh snapshot with non-subscribing `getQueryData` from a cache entry that has no queryFn, no subscriber, and the default 5-minute gcTime — after which the page claims no refresh ever ran. This is client UI state. New module `web/src/features/blocklists/refreshStore.ts` in the exact shape of `settings/restartBanner.ts` (module-level value + listener Set + `useSyncExternalStore` hook), holding `SourceStatus[] | null`. The mutation (`blocklistsUpdateNowMutation`, lib/queries.ts:156) writes the store instead of `setQueryData`; the page subscribes via the hook; the `queryKeys.blocklistSources` entry and its doc comment are deleted. Note the prefix-invalidation side effect disappears with it (invalidateBlocklistWorld invalidating `["blocklists"]` used to mark the entry stale — harmless, since the entry could never refetch).
|
||||
|
||||
### 17. Frontend: one default-group helper
|
||||
|
||||
Four encodings of "group 1", two semantics (GroupsPage.tsx:14 named
|
||||
constant; PrefixesEditor.tsx:21 and LookupPage.tsx:133 byte-identical
|
||||
prefer-id-1 expressions; RulesPage.tsx:24 first-of-list — which, because the
|
||||
API orders by name (groups_repo.zig:148), preselects the alphabetically
|
||||
first group, not the default). New module `web/src/lib/defaultGroup.ts`:
|
||||
Four encodings of "group 1", two semantics (GroupsPage.tsx:14 named constant; PrefixesEditor.tsx:21 and LookupPage.tsx:133 byte-identical prefer-id-1 expressions; RulesPage.tsx:24 first-of-list — which, because the API orders by name (groups_repo.zig:148), preselects the alphabetically first group, not the default). New module `web/src/lib/defaultGroup.ts`:
|
||||
|
||||
```ts
|
||||
export const DEFAULT_GROUP_ID = 1;
|
||||
@@ -282,59 +133,25 @@ export function defaultGroupId(groups: readonly Group[]): number {
|
||||
}
|
||||
```
|
||||
|
||||
All four sites adopt it; RulesPage's preselect bug goes away with the
|
||||
adoption.
|
||||
All four sites adopt it; RulesPage's preselect bug goes away with the adoption.
|
||||
|
||||
### 18. Frontend: the dashboard primes without throwing
|
||||
|
||||
`web/src/routes.tsx:97-102` — the dashboard loader's `Promise.all` over four
|
||||
`ensureQueryData` calls means one failing endpoint blanks the whole page with
|
||||
`RouteError` on cold navigation, defeating the page's own per-widget
|
||||
degrade (DashboardPage.tsx:68-96, which uses `useQuery` precisely to allow
|
||||
it). Change **only the dashboard loader** to `Promise.allSettled`. The five
|
||||
other `Promise.all` loaders stay: their pages use `useSuspenseQuery` and have
|
||||
no granular fallback to preserve.
|
||||
`web/src/routes.tsx:97-102` — the dashboard loader's `Promise.all` over four `ensureQueryData` calls means one failing endpoint blanks the whole page with `RouteError` on cold navigation, defeating the page's own per-widget degrade (DashboardPage.tsx:68-96, which uses `useQuery` precisely to allow it). Change **only the dashboard loader** to `Promise.allSettled`. The five other `Promise.all` loaders stay: their pages use `useSuspenseQuery` and have no granular fallback to preserve.
|
||||
|
||||
### 19. Frontend: the query-key table is complete
|
||||
|
||||
`web/src/lib/queries.ts` — eight raw literals, all in this file: seven
|
||||
`["lookup"]` prefix-invalidations (:99, :125, :132, :159, :167, :189, :211)
|
||||
and one `["groups"]` (:133). Add `lookupAll: ["lookup"] as const` to
|
||||
`queryKeys`, use it at the seven sites, and use `queryKeys.groups` at :133.
|
||||
Acceptance: no `["lookup"]`/`["groups"]` literal outside the table.
|
||||
`web/src/lib/queries.ts` — eight raw literals, all in this file: seven `["lookup"]` prefix-invalidations (:99, :125, :132, :159, :167, :189, :211) and one `["groups"]` (:133). Add `lookupAll: ["lookup"] as const` to `queryKeys`, use it at the seven sites, and use `queryKeys.groups` at :133. Acceptance: no `["lookup"]`/`["groups"]` literal outside the table.
|
||||
|
||||
### 20. Frontend: the form catch swallows only the expected class
|
||||
|
||||
`web/src/features/blocklists/BlocklistForm.tsx:21-33` — the bare `catch {}`
|
||||
wraps both the `await onSubmit` and the reset `setState` calls; anything but
|
||||
the mutation rejection dies silently with no console trace. House precedent
|
||||
is auth/store.tsx:85-89 (swallow only the 401 it expects, rethrow the rest),
|
||||
pinned by two tests. Restructure: `try { await onSubmit(...) } catch (error)
|
||||
{ if (error instanceof ApiError) return; throw error; }`, resets after the
|
||||
try. The page's `<InlineError>` keeps rendering the mutation error as today.
|
||||
`web/src/features/blocklists/BlocklistForm.tsx:21-33` — the bare `catch {}` wraps both the `await onSubmit` and the reset `setState` calls; anything but the mutation rejection dies silently with no console trace. House precedent is auth/store.tsx:85-89 (swallow only the 401 it expects, rethrow the rest), pinned by two tests. Restructure: `try { await onSubmit(...) } catch (error) { if (error instanceof ApiError) return; throw error; }`, resets after the try. The page's `<InlineError>` keeps rendering the mutation error as today.
|
||||
|
||||
### Carried in from milestone 17: the stale phase-comment sweep
|
||||
|
||||
Milestone 17 ruling 7 rewrote the "Phase 7/8" headers in three files; its
|
||||
repo-wide acceptance grep then found 24 more stale phase references in
|
||||
files its sessions did not own, recorded here for this milestone:
|
||||
safesearch.zig:58, forward_client.zig:7, dns_cache.zig:10/:42/:435,
|
||||
cli.zig:332, filter_integration_test.zig:1391, logger.zig:178/:303,
|
||||
retention.zig:7/:137/:151, disk_monitor.zig:7, querylog_schema.zig:71,
|
||||
db.zig:272, manager.zig:292/:613/:1145, and the six repository files
|
||||
(groups_repo.zig:7, rules_repo.zig:15, sources_repo.zig:9,
|
||||
upstreams_repo.zig:7, local_repo.zig:3, clients_repo.zig:12-13). Each
|
||||
comment is rewritten to state the as-built truth it gestures at (not
|
||||
deleted, unless it says nothing beyond the phase number). Line numbers
|
||||
are hints from m17-time HEAD; re-grep before acting. Acceptance: "Phase
|
||||
7" and "Phase 8" appear in no source doc comment repo-wide; spec files
|
||||
and TECH_DEBT.md are exempt. The session owning each file in the plan
|
||||
below picks up its share; files owned by no session fall to S4.
|
||||
Milestone 17 ruling 7 rewrote the "Phase 7/8" headers in three files; its repo-wide acceptance grep then found 24 more stale phase references in files its sessions did not own, recorded here for this milestone: safesearch.zig:58, forward_client.zig:7, dns_cache.zig:10/:42/:435, cli.zig:332, filter_integration_test.zig:1391, logger.zig:178/:303, retention.zig:7/:137/:151, disk_monitor.zig:7, querylog_schema.zig:71, db.zig:272, manager.zig:292/:613/:1145, and the six repository files (groups_repo.zig:7, rules_repo.zig:15, sources_repo.zig:9, upstreams_repo.zig:7, local_repo.zig:3, clients_repo.zig:12-13). Each comment is rewritten to state the as-built truth it gestures at (not deleted, unless it says nothing beyond the phase number). Line numbers are hints from m17-time HEAD; re-grep before acting. Acceptance: "Phase 7" and "Phase 8" appear in no source doc comment repo-wide; spec files and TECH_DEBT.md are exempt. The session owning each file in the plan below picks up its share; files owned by no session fall to S4.
|
||||
|
||||
Also carried from milestone 17 (ruling 4 residual): the login log line in
|
||||
`src/web/handlers/auth.zig` prints the socket peer, which reads as
|
||||
loopback for every login behind a trusted proxy; it should print
|
||||
`request.client_addr`.
|
||||
Also carried from milestone 17 (ruling 4 residual): the login log line in `src/web/handlers/auth.zig` prints the socket peer, which reads as loopback for every login behind a trusted proxy; it should print `request.client_addr`.
|
||||
|
||||
## Sessions
|
||||
|
||||
@@ -342,18 +159,11 @@ S1-S4 run in parallel; no two sessions write the same file.
|
||||
|
||||
### Session S1: backend surface
|
||||
|
||||
Owns `src/dns/edns.zig`, `tests/fuzz/dns_fuzz.zig`,
|
||||
`src/web/handlers/settings.zig`, `src/web/handlers/mutations.zig`,
|
||||
`src/web/http_util.zig`, `src/server/cert_store.zig`, `src/app.zig`,
|
||||
`src/web/server.zig`. Rulings 1, 2, 8, 9, 10, 11.
|
||||
Owns `src/dns/edns.zig`, `tests/fuzz/dns_fuzz.zig`, `src/web/handlers/settings.zig`, `src/web/handlers/mutations.zig`, `src/web/http_util.zig`, `src/server/cert_store.zig`, `src/app.zig`, `src/web/server.zig`. Rulings 1, 2, 8, 9, 10, 11.
|
||||
|
||||
### Session S2: constants, counters, classification
|
||||
|
||||
Owns `src/storage/logger.zig`, `src/server/handler.zig`,
|
||||
`src/server/clients.zig`, `src/web/handlers/queries.zig`,
|
||||
`src/platform/logging.zig`, `src/filter/fetcher.zig`,
|
||||
`src/upstream/doh_client.zig`, `src/platform/tls_server.zig`,
|
||||
`src/platform/mbedtls_shim.c`. Rulings 3, 4, 5, 6, 7.
|
||||
Owns `src/storage/logger.zig`, `src/server/handler.zig`, `src/server/clients.zig`, `src/web/handlers/queries.zig`, `src/platform/logging.zig`, `src/filter/fetcher.zig`, `src/upstream/doh_client.zig`, `src/platform/tls_server.zig`, `src/platform/mbedtls_shim.c`. Rulings 3, 4, 5, 6, 7.
|
||||
|
||||
### Session S3: frontend
|
||||
|
||||
@@ -361,13 +171,11 @@ Owns everything under `web/src/`. Rulings 14-20.
|
||||
|
||||
### Session S4: tools and deploy
|
||||
|
||||
Owns `tools/gen_web_assets.zig`, `src/web/static.zig`,
|
||||
`deploy/docker/Dockerfile`. Rulings 12, 13.
|
||||
Owns `tools/gen_web_assets.zig`, `src/web/static.zig`, `deploy/docker/Dockerfile`. Rulings 12, 13.
|
||||
|
||||
### Orchestrator
|
||||
|
||||
Re-verifies this spec's line references against post-m18 reality before S1-S4
|
||||
start; strikes the closed findings in `TECH_DEBT.md` after.
|
||||
Re-verifies this spec's line references against post-m18 reality before S1-S4 start; strikes the closed findings in `TECH_DEBT.md` after.
|
||||
|
||||
## Module layout
|
||||
|
||||
@@ -376,9 +184,7 @@ New files:
|
||||
- `web/src/features/blocklists/refreshStore.ts` — refresh snapshot store.
|
||||
- `web/src/lib/defaultGroup.ts` — default-group constant and helper.
|
||||
|
||||
Deleted surface: `parseEcs`/`Ecs`/`EcsError`/`ecs_family_*` (edns.zig), the
|
||||
`web_dev_dir` global (app.zig), the `queryKeys.blocklistSources` entry
|
||||
(queries.ts).
|
||||
Deleted surface: `parseEcs`/`Ecs`/`EcsError`/`ecs_family_*` (edns.zig), the `web_dev_dir` global (app.zig), the `queryKeys.blocklistSources` entry (queries.ts).
|
||||
|
||||
## Acceptance (milestone complete)
|
||||
|
||||
@@ -437,110 +243,44 @@ Accepted deviations and findings from the built milestone.
|
||||
|
||||
### Ruling 1 (S1)
|
||||
|
||||
- The edns.zig test block held six `parseEcs` tests, not nine. A seventh
|
||||
use lived inside "encodeOpt round-trips through record parse and
|
||||
parseOpt", which is not a parseEcs test; its last two lines now compare
|
||||
the found option's bytes directly.
|
||||
- `extendedRcode` stays, with a nuance the ruling's premise missed: the
|
||||
BADVERS production path uses the write-side `splitRcode`; extendedRcode
|
||||
is the read-side verifier whose callers are the two BADVERS wire-format
|
||||
tests (handler.zig, packet.zig). Deleting it would weaken those tests.
|
||||
- The write-fault seam for the rollback test has no exported control
|
||||
surface — the test lives in the same file and needs none.
|
||||
- The ruling-8 doc comment names `local.zig`'s `publish` function instead
|
||||
of a line number, so it cannot go stale.
|
||||
- The edns.zig test block held six `parseEcs` tests, not nine. A seventh use lived inside "encodeOpt round-trips through record parse and parseOpt", which is not a parseEcs test; its last two lines now compare the found option's bytes directly.
|
||||
- `extendedRcode` stays, with a nuance the ruling's premise missed: the BADVERS production path uses the write-side `splitRcode`; extendedRcode is the read-side verifier whose callers are the two BADVERS wire-format tests (handler.zig, packet.zig). Deleting it would weaken those tests.
|
||||
- The write-fault seam for the rollback test has no exported control surface — the test lives in the same file and needs none.
|
||||
- The ruling-8 doc comment names `local.zig`'s `publish` function instead of a line number, so it cannot go stale.
|
||||
|
||||
### Rulings 3-7 (S2)
|
||||
|
||||
- Ownership amendment: `src/web/metrics.zig` (owned by no session) was
|
||||
granted to S2 mid-flight. Ruling 5's new counter raises the /metrics
|
||||
sample count 29 → 30 (`nxdns_log_lines_truncated_total`); the
|
||||
hard-coded assertion became fully derived
|
||||
(`1 + dns_stat_fields.len + @typeInfo(LoggerCounters)... +
|
||||
@typeInfo(logging.Stats)...`), so a new counter in any of those
|
||||
structs extends the exposition and the assertion together.
|
||||
- Ruling 6 lands asymmetrically, following the ruling body over the
|
||||
stale acceptance bullet: doh_client.zig names eleven members (the two
|
||||
collapsed top-level ones plus the nine `Tls*` members of the unwrapped
|
||||
read-cause set); fetcher.zig names only the two, because it has no
|
||||
cause unwrap and no record-layer member can reach it. Both files carry
|
||||
a comment pointing at the other.
|
||||
- Addition beyond the ruling: an exact switch does not catch a std
|
||||
rename — `error.X` in an expression names a member into existence, so
|
||||
a renamed member leaves the switch compiling and matching nothing.
|
||||
Comptime guards close this for real: doh_client.zig asserts every
|
||||
member of `std.crypto.tls.Client.ReadError` maps to `TlsFailed`, and
|
||||
both files assert the two collapsed names still exist in
|
||||
`std.http.Client.RequestError`.
|
||||
- The ruling's "stashed record-layer cause through the unwrap" test is
|
||||
not constructible: the stub connection is `.plain` on purpose, and a
|
||||
plain connection's stashed error type has no `Tls*` member. The
|
||||
record-layer set is instead tested by an `inline for` over the std
|
||||
error set against `mapError` — all nine members, not one sample.
|
||||
- Ruling 3 behavioral note: queries.zig's client width dropped 64 → 45
|
||||
as directed; a `?client=` filter of 46-64 bytes now returns 400
|
||||
instead of matching nothing. `src/filter/wildcard.zig`'s
|
||||
`max_pattern_len = 253` is a rule-pattern limit, not a logger role,
|
||||
and stays.
|
||||
- Ownership amendment: `src/web/metrics.zig` (owned by no session) was granted to S2 mid-flight. Ruling 5's new counter raises the /metrics sample count 29 → 30 (`nxdns_log_lines_truncated_total`); the hard-coded assertion became fully derived (`1 + dns_stat_fields.len + @typeInfo(LoggerCounters)... + @typeInfo(logging.Stats)...`), so a new counter in any of those structs extends the exposition and the assertion together.
|
||||
- Ruling 6 lands asymmetrically, following the ruling body over the stale acceptance bullet: doh_client.zig names eleven members (the two collapsed top-level ones plus the nine `Tls*` members of the unwrapped read-cause set); fetcher.zig names only the two, because it has no cause unwrap and no record-layer member can reach it. Both files carry a comment pointing at the other.
|
||||
- Addition beyond the ruling: an exact switch does not catch a std rename — `error.X` in an expression names a member into existence, so a renamed member leaves the switch compiling and matching nothing. Comptime guards close this for real: doh_client.zig asserts every member of `std.crypto.tls.Client.ReadError` maps to `TlsFailed`, and both files assert the two collapsed names still exist in `std.http.Client.RequestError`.
|
||||
- The ruling's "stashed record-layer cause through the unwrap" test is not constructible: the stub connection is `.plain` on purpose, and a plain connection's stashed error type has no `Tls*` member. The record-layer set is instead tested by an `inline for` over the std error set against `mapError` — all nine members, not one sample.
|
||||
- Ruling 3 behavioral note: queries.zig's client width dropped 64 → 45 as directed; a `?client=` filter of 46-64 bytes now returns 400 instead of matching nothing. `src/filter/wildcard.zig`'s `max_pattern_len = 253` is a rule-pattern limit, not a logger role, and stays.
|
||||
|
||||
### Rulings 14-20 (S3)
|
||||
|
||||
- Ruling 14: the three `Record<string, unknown>` consumption casts
|
||||
collapse into one documented cast inside `sectionValues` rather than
|
||||
zero — iterating the heterogeneous registry erases the per-section
|
||||
correlation TypeScript needs, and `TlsListenerSettings` (a named
|
||||
interface, no implicit index signature) blocks the assignment route
|
||||
with types.ts off-limits. Four casts before, one after.
|
||||
- Ruling 20: the catch body lives in an exported `swallowMutationError`
|
||||
helper because jsdom never fires `unhandledrejection`, so a rethrow
|
||||
from an async submit handler cannot be asserted directly; the helper
|
||||
is tested directly, mirroring how auth/store.tsx's logout pair is
|
||||
tested.
|
||||
- The refresh store notifies synchronously inside `onSuccess`, one flush
|
||||
before react-query's success dispatch; one pre-existing assertion
|
||||
moved from `getByText` to `await findByText`. The sub-frame transient
|
||||
was judged cheaper than batching through notifyManager.
|
||||
- `lib/queries.ts` imports `features/blocklists/refreshStore.ts` (lib →
|
||||
features) because the spec directs the mutation factory to write the
|
||||
store; recorded as sanctioned layering.
|
||||
- `SourceStatusSection`'s prop is `SourceStatus[] | null` to match the
|
||||
store; BlocklistsPage tests clear the module-level store in
|
||||
`beforeEach`.
|
||||
- Ruling 18: post-m18 there are four other `Promise.all` loaders, not
|
||||
five.
|
||||
- Ruling 14: the three `Record<string, unknown>` consumption casts collapse into one documented cast inside `sectionValues` rather than zero — iterating the heterogeneous registry erases the per-section correlation TypeScript needs, and `TlsListenerSettings` (a named interface, no implicit index signature) blocks the assignment route with types.ts off-limits. Four casts before, one after.
|
||||
- Ruling 20: the catch body lives in an exported `swallowMutationError` helper because jsdom never fires `unhandledrejection`, so a rethrow from an async submit handler cannot be asserted directly; the helper is tested directly, mirroring how auth/store.tsx's logout pair is tested.
|
||||
- The refresh store notifies synchronously inside `onSuccess`, one flush before react-query's success dispatch; one pre-existing assertion moved from `getByText` to `await findByText`. The sub-frame transient was judged cheaper than batching through notifyManager.
|
||||
- `lib/queries.ts` imports `features/blocklists/refreshStore.ts` (lib → features) because the spec directs the mutation factory to write the store; recorded as sanctioned layering.
|
||||
- `SourceStatusSection`'s prop is `SourceStatus[] | null` to match the store; BlocklistsPage tests clear the module-level store in `beforeEach`.
|
||||
- Ruling 18: post-m18 there are four other `Promise.all` loaders, not five.
|
||||
|
||||
### Rulings 12, 13 and carry-ins (S4)
|
||||
|
||||
- The undecodable-sibling message reports `decompress.err orelse err`,
|
||||
because `std.Io.Reader` collapses every decode failure into
|
||||
`error.ReadFailed` and stashes the real one.
|
||||
- The Dockerfile keeps its `&&` chain instead of the spec sketch's `;`
|
||||
separators, so any failed step still aborts the layer; the case arms
|
||||
are untouched.
|
||||
- The phase-comment sweep leaves `PLAN.md:605`/`:609` alone — they are
|
||||
the section headings that define Phase 7 and Phase 8, not stale
|
||||
comments.
|
||||
- The orchestrator added the host-fallback sentence to
|
||||
docs/how-to/install-with-docker.md.
|
||||
- The undecodable-sibling message reports `decompress.err orelse err`, because `std.Io.Reader` collapses every decode failure into `error.ReadFailed` and stashes the real one.
|
||||
- The Dockerfile keeps its `&&` chain instead of the spec sketch's `;` separators, so any failed step still aborts the layer; the case arms are untouched.
|
||||
- The phase-comment sweep leaves `PLAN.md:605`/`:609` alone — they are the section headings that define Phase 7 and Phase 8, not stale comments.
|
||||
- The orchestrator added the host-fallback sentence to docs/how-to/install-with-docker.md.
|
||||
|
||||
### Residual (named, not fixed)
|
||||
|
||||
`fetcher.zig` hands the collapsed `ReadFailed`/`WriteFailed` to
|
||||
`mapError` with no cause unwrap, so a blocklist download canceled at
|
||||
shutdown returns `error.ReceiveFailed` instead of `error.Canceled` — the
|
||||
class m18 ruling 4 fixed for DoH. Consequence: a wrong source status and
|
||||
a spurious warn line at shutdown. Fixing it needs `req`/`resp` plumbing
|
||||
at four call sites; it deserves its own ruling.
|
||||
`fetcher.zig` hands the collapsed `ReadFailed`/`WriteFailed` to `mapError` with no cause unwrap, so a blocklist download canceled at shutdown returns `error.ReceiveFailed` instead of `error.Canceled` — the class m18 ruling 4 fixed for DoH. Consequence: a wrong source status and a spurious warn line at shutdown. Fixing it needs `req`/`resp` plumbing at four call sites; it deserves its own ruling.
|
||||
|
||||
## Anti-requirements
|
||||
|
||||
- No deletion of `extendedRcode` (m17 gave it a caller) and no deletion of
|
||||
the upstream mutation factories (m17 built their consumer).
|
||||
- No deletion of `extendedRcode` (m17 gave it a caller) and no deletion of the upstream mutation factories (m17 built their consumer).
|
||||
- No locking change in the reload handlers — ruling 8 is documentation.
|
||||
- No new frontend dependencies; the store is hand-rolled like restartBanner.
|
||||
- No compression plugin for the frontend build; ruling 12 guards the seam,
|
||||
it does not start using it.
|
||||
- No `useInfiniteQuery` migration and no QueryLogPage work — that was
|
||||
milestone 16.
|
||||
- No renaming of settings keys or API fields — typing the registry must not
|
||||
change the wire format.
|
||||
- No compression plugin for the frontend build; ruling 12 guards the seam, it does not start using it.
|
||||
- No `useInfiniteQuery` migration and no QueryLogPage work — that was milestone 16.
|
||||
- No renaming of settings keys or API fields — typing the registry must not change the wire format.
|
||||
|
||||
+113
-688
@@ -1,185 +1,64 @@
|
||||
# Milestone 20: declarative configuration for IaC
|
||||
|
||||
Goal: a config file an operator can keep in git and deploy with Ansible, where
|
||||
the file is the sole declarative source of truth, converged at every boot —
|
||||
without re-downloading every blocklist on every boot, and without the UI
|
||||
silently diverging from the file. Two authority modes, selected by the
|
||||
presence of one flag: bare `run` serves the DB; `run --config=<path>` makes
|
||||
the file authority.
|
||||
Goal: a config file an operator can keep in git and deploy with Ansible, where the file is the sole declarative source of truth, converged at every boot — without re-downloading every blocklist on every boot, and without the UI silently diverging from the file. Two authority modes, selected by the presence of one flag: bare `run` serves the DB; `run --config=<path>` makes the file authority.
|
||||
|
||||
Design finalized 2026-08-09 after three adversarial rounds (red team ops 1-16
|
||||
and debt 1-14; Codex cross-validation F1-F12; a Codex round on the premise
|
||||
revision). All findings are folded in below or declined with written reasons;
|
||||
all file:line anchors were re-verified at the pre-implementation HEAD
|
||||
(7039a9f). A premise revision replaced the earlier `--config-source` mode
|
||||
enum with presence-of-`--config` and deleted the persisted authority marker;
|
||||
rulings 1, 6 and 8 record the reasons — do not reintroduce either.
|
||||
Design finalized 2026-08-09 after three adversarial rounds (red team ops 1-16 and debt 1-14; Codex cross-validation F1-F12; a Codex round on the premise revision). All findings are folded in below or declined with written reasons; all file:line anchors were re-verified at the pre-implementation HEAD (7039a9f). A premise revision replaced the earlier `--config-source` mode enum with presence-of-`--config` and deleted the persisted authority marker; rulings 1, 6 and 8 record the reasons — do not reintroduce either.
|
||||
|
||||
## Implementation contract (read first)
|
||||
|
||||
- Read `AGENTS.md`, then this spec whole, before session work starts.
|
||||
- Session order: R1 → R2 sequential; R3 and R4 parallel to both (Sessions,
|
||||
below). File ownership is write-exclusivity; the interfaces between
|
||||
sessions (`reconcile.Summary`, `WebState.authority` + `reconciled_at`,
|
||||
`RouteInfo.policy`) are fixed in this spec and are not renegotiable
|
||||
mid-build.
|
||||
- Gates: `zig build test` and `zig build test -Dintegration` with 0 failed,
|
||||
plus the drift-guard regenerations the Tests section names. Skip counts are
|
||||
reported with their reasons (plain-suite skips are integration-gated; the 4
|
||||
integration skips are the live-network TLS tests excluded by milestone-1
|
||||
design). One `-Dlive` run covers the real download path (Tests).
|
||||
- No `std.log.err` in new code. No secrets in logs — url redaction goes
|
||||
through `src/safe_url.zig` as everywhere else.
|
||||
- Every fix or behaviour claim lands with a test the author watched fail
|
||||
against the reverted implementation (milestone-13 ruling F-f applies).
|
||||
- Doc pages touched by R4 follow milestone-13 ruling 3: every command block
|
||||
in `tutorial/` and `how-to/` is executed on this host by the session that
|
||||
writes it, or marked in-page as unverified with the reason.
|
||||
- The final commit is GPG-signed by the user (`git commit -S`, lowercase,
|
||||
single line); stage the work and hand the command over.
|
||||
- Session order: R1 → R2 sequential; R3 and R4 parallel to both (Sessions, below). File ownership is write-exclusivity; the interfaces between sessions (`reconcile.Summary`, `WebState.authority` + `reconciled_at`, `RouteInfo.policy`) are fixed in this spec and are not renegotiable mid-build.
|
||||
- Gates: `zig build test` and `zig build test -Dintegration` with 0 failed, plus the drift-guard regenerations the Tests section names. Skip counts are reported with their reasons (plain-suite skips are integration-gated; the 4 integration skips are the live-network TLS tests excluded by milestone-1 design). One `-Dlive` run covers the real download path (Tests).
|
||||
- No `std.log.err` in new code. No secrets in logs — url redaction goes through `src/safe_url.zig` as everywhere else.
|
||||
- Every fix or behaviour claim lands with a test the author watched fail against the reverted implementation (milestone-13 ruling F-f applies).
|
||||
- Doc pages touched by R4 follow milestone-13 ruling 3: every command block in `tutorial/` and `how-to/` is executed on this host by the session that writes it, or marked in-page as unverified with the reason.
|
||||
- The final commit is GPG-signed by the user (`git commit -S`, lowercase, single line); stage the work and hand the command over.
|
||||
|
||||
## Rulings (binding)
|
||||
|
||||
### 1. Authority is the invocation: `--config` present means the file governs
|
||||
|
||||
`nxdns run` — the DB is authority, today's appliance behaviour.
|
||||
`nxdns run --config=/etc/nxdns/config.zon` — the file is authority. The
|
||||
flag's presence selects the mode; there is no mode enum and no default path.
|
||||
`check` keeps its existing surface under the same rule: bare `check` grades
|
||||
the DB, `check --config <file>` grades the file. The DB-exists-wins heuristic
|
||||
in `checkImpl` (cli.zig:559-588) and `CheckArgs.config_explicit` are deleted.
|
||||
`nxdns run` — the DB is authority, today's appliance behaviour. `nxdns run --config=/etc/nxdns/config.zon` — the file is authority. The flag's presence selects the mode; there is no mode enum and no default path. `check` keeps its existing surface under the same rule: bare `check` grades the DB, `check --config <file>` grades the file. The DB-exists-wins heuristic in `checkImpl` (cli.zig:559-588) and `CheckArgs.config_explicit` are deleted.
|
||||
|
||||
Two principles, separated deliberately, because conflating them is what
|
||||
produced the rejected `--config-source` enum:
|
||||
Two principles, separated deliberately, because conflating them is what produced the rejected `--config-source` enum:
|
||||
|
||||
- **Authority must be explicit in the invocation.** An operator reads
|
||||
`ExecStart` and knows which authority is live. Probing `/etc/nxdns` for a
|
||||
file and switching behaviour on its existence is ambient magic — the same
|
||||
class of heuristic that made seed-once bootstrap a source of doc lies (the
|
||||
compose comment, the first-run tutorial) — and stays banned: a file on disk
|
||||
that no flag names changes nothing.
|
||||
- **A mode enum is the wrong shape for a two-state choice a path already
|
||||
expresses.** `--config-source=db` names an implementation, not an operator
|
||||
intent, and a mode flag beside a path flag manufactures invalid
|
||||
combinations (path without mode, mode without path) that then need pairing
|
||||
rules and usage errors to defend. Presence-of-path has no invalid
|
||||
combinations and nothing to defend.
|
||||
- **Authority must be explicit in the invocation.** An operator reads `ExecStart` and knows which authority is live. Probing `/etc/nxdns` for a file and switching behaviour on its existence is ambient magic — the same class of heuristic that made seed-once bootstrap a source of doc lies (the compose comment, the first-run tutorial) — and stays banned: a file on disk that no flag names changes nothing.
|
||||
- **A mode enum is the wrong shape for a two-state choice a path already expresses.** `--config-source=db` names an implementation, not an operator intent, and a mode flag beside a path flag manufactures invalid combinations (path without mode, mode without path) that then need pairing rules and usage errors to defend. Presence-of-path has no invalid combinations and nothing to defend.
|
||||
|
||||
Breaking change, named: `run --config <file>` today means seed-once; the same
|
||||
syntax now means file authority — reconcile on every boot, UI config writes
|
||||
rejected. For an operator who seeded once and then configured through the UI,
|
||||
the first post-upgrade restart converges the DB to that old seed file,
|
||||
deleting the UI edits. The upgrade doc's breaking-changes section leads with
|
||||
this and gives the two exits (ruling 9): drop the flag, or re-export to the
|
||||
file path first. `check --config` keeps its meaning exactly. Greenfield rules
|
||||
apply — the seed-once interface does not survive for compatibility — but the
|
||||
break is loud in the docs, never silent-by-omission.
|
||||
Breaking change, named: `run --config <file>` today means seed-once; the same syntax now means file authority — reconcile on every boot, UI config writes rejected. For an operator who seeded once and then configured through the UI, the first post-upgrade restart converges the DB to that old seed file, deleting the UI edits. The upgrade doc's breaking-changes section leads with this and gives the two exits (ruling 9): drop the flag, or re-export to the file path first. `check --config` keeps its meaning exactly. Greenfield rules apply — the seed-once interface does not survive for compatibility — but the break is loud in the docs, never silent-by-omission.
|
||||
|
||||
A rename (`--managed-config`) that would make the old invocation fail loudly
|
||||
was considered twice and declined: it trades a worse name and a permanent
|
||||
asymmetry with `check --config` against a one-time hazard whose exposed
|
||||
population is the pre-change install base of a project whose first release is
|
||||
days old. The hazard is real and the docs lead with it; the interface does
|
||||
not carry the scar. This is a judgment, recorded so it is revisited only with
|
||||
new facts (a real install base would be one).
|
||||
A rename (`--managed-config`) that would make the old invocation fail loudly was considered twice and declined: it trades a worse name and a permanent asymmetry with `check --config` against a one-time hazard whose exposed population is the pre-change install base of a project whose first release is days old. The hazard is real and the docs lead with it; the interface does not carry the scar. This is a judgment, recorded so it is revisited only with new facts (a real install base would be one).
|
||||
|
||||
### 2. File mode fails closed, and *declarative* failure is exit 2
|
||||
|
||||
File mode contract: the file is the sole declarative source; the DB stays the
|
||||
runtime substrate and the effective-config read path. Startup sequence,
|
||||
replacing the `seedFromFile` call at app.zig:196:
|
||||
File mode contract: the file is the sole declarative source; the DB stays the runtime substrate and the effective-config read path. Startup sequence, replacing the `seedFromFile` call at app.zig:196:
|
||||
|
||||
1. `DataDir.open` → open config DB → `migrate` (unchanged).
|
||||
2. Read the file (existing 4 MiB cap), ZON parse (arena, never freed — keep
|
||||
the import.zig discipline), `validate.validate`.
|
||||
3. Reconcile into the DB in one `BEGIN IMMEDIATE` transaction, `errdefer`
|
||||
rollback (ruling 3).
|
||||
2. Read the file (existing 4 MiB cap), ZON parse (arena, never freed — keep the import.zig discipline), `validate.validate`.
|
||||
3. Reconcile into the DB in one `BEGIN IMMEDIATE` transaction, `errdefer` rollback (ruling 3).
|
||||
4. `config_export.readConfig` (app.zig:206) → serve, unchanged from there on.
|
||||
|
||||
A missing, unreadable, or invalid file fails startup. Never fall back to the
|
||||
DB: a fallback turns a deploy typo into a silently stale config.
|
||||
A missing, unreadable, or invalid file fails startup. Never fall back to the DB: a fallback turns a deploy typo into a silently stale config.
|
||||
|
||||
Exit codes keep the 2am contract — exit 2 means "your config is wrong, run
|
||||
`nxdns check`"; exit 1 means "the box is wrong". `faults.isConfigFault`
|
||||
deliberately excludes `FileNotFound`/`AccessDenied` (faults.zig:107-108), and
|
||||
that stays true in general. The file-mode loader is the seam, and it maps
|
||||
**path-class open failures only**: `FileNotFound`, `AccessDenied`,
|
||||
`PermissionDenied`, `NotDir`, `IsDir`, `SymLinkLoop`, `NameTooLong`,
|
||||
`BadPathName` become `error.ManagedConfigUnreadable` (message includes the
|
||||
path), which joins the `ConfigFault` set beside `ParseZon`/`ConfigTooLarge`.
|
||||
Every other member of `ReadFileAllocError` — `SystemResources`,
|
||||
`ProcessFdQuotaExceeded`, `SystemFdQuotaExceeded`, I/O errors, `OutOfMemory` —
|
||||
propagates unmapped, exit 1: those are box faults a retry can clear, and once
|
||||
ruling 9 adds `RestartPreventExitStatus=2 64`, mapping them to exit 2 would
|
||||
stop the unit permanently on a transient fault. The mapping is an explicit
|
||||
named error set in the loader, switched exhaustively with
|
||||
`else => |other| return other`, so a std error added in a Zig bump defaults to
|
||||
exit 1 rather than silently to exit 2 — the same closed-set discipline
|
||||
faults.zig already documents. The mapping lives in **one shared helper** used
|
||||
by both `run` and `check`; two copies would let the two grade the same
|
||||
unreadable file differently, exactly the divergence faults.zig:1-8 exists to
|
||||
abolish.
|
||||
Exit codes keep the 2am contract — exit 2 means "your config is wrong, run `nxdns check`"; exit 1 means "the box is wrong". `faults.isConfigFault` deliberately excludes `FileNotFound`/`AccessDenied` (faults.zig:107-108), and that stays true in general. The file-mode loader is the seam, and it maps **path-class open failures only**: `FileNotFound`, `AccessDenied`, `PermissionDenied`, `NotDir`, `IsDir`, `SymLinkLoop`, `NameTooLong`, `BadPathName` become `error.ManagedConfigUnreadable` (message includes the path), which joins the `ConfigFault` set beside `ParseZon`/`ConfigTooLarge`. Every other member of `ReadFileAllocError` — `SystemResources`, `ProcessFdQuotaExceeded`, `SystemFdQuotaExceeded`, I/O errors, `OutOfMemory` — propagates unmapped, exit 1: those are box faults a retry can clear, and once ruling 9 adds `RestartPreventExitStatus=2 64`, mapping them to exit 2 would stop the unit permanently on a transient fault. The mapping is an explicit named error set in the loader, switched exhaustively with `else => |other| return other`, so a std error added in a Zig bump defaults to exit 1 rather than silently to exit 2 — the same closed-set discipline faults.zig already documents. The mapping lives in **one shared helper** used by both `run` and `check`; two copies would let the two grade the same unreadable file differently, exactly the divergence faults.zig:1-8 exists to abolish.
|
||||
|
||||
**Scope of the check/run agreement claim**: `check --config=<file>`
|
||||
grades exactly the declarative faults `run` would hit — read (path-class),
|
||||
parse, size, validate — through the same shared loader helper. Reconcile-time
|
||||
faults (FK violations, `SQLITE_BUSY` from a restart race, disk full) are
|
||||
runtime faults, exit 1, and structurally invisible to `check`, which needs no
|
||||
DB. The acceptance encodes the scoped claim, not "check passing guarantees run
|
||||
converges". The one *systematic* gap the red team found — a group removal
|
||||
tripping the un-cascaded `clients.group_id` FK against observed rows `check`
|
||||
cannot see — is eliminated in the engine itself (ruling 3's reassign rule),
|
||||
not papered over in `check`.
|
||||
**Scope of the check/run agreement claim**: `check --config=<file>` grades exactly the declarative faults `run` would hit — read (path-class), parse, size, validate — through the same shared loader helper. Reconcile-time faults (FK violations, `SQLITE_BUSY` from a restart race, disk full) are runtime faults, exit 1, and structurally invisible to `check`, which needs no DB. The acceptance encodes the scoped claim, not "check passing guarantees run converges". The one *systematic* gap the red team found — a group removal tripping the un-cascaded `clients.group_id` FK against observed rows `check` cannot see — is eliminated in the engine itself (ruling 3's reassign rule), not papered over in `check`.
|
||||
|
||||
Reconcile-time diagnostics include the SQLite error (`SQLITE_FULL` by name
|
||||
when that is the cause) so a full SD card reads as "disk", not as a bare
|
||||
exit 1. Diagnostics render to `r.err` and flush immediately, keeping the
|
||||
current `seedFromFile` discipline (app.zig:137-178) — `serve` never returns,
|
||||
so a buffered error line is a lost error line.
|
||||
Reconcile-time diagnostics include the SQLite error (`SQLITE_FULL` by name when that is the cause) so a full SD card reads as "disk", not as a bare exit 1. Diagnostics render to `r.err` and flush immediately, keeping the current `seedFromFile` discipline (app.zig:137-178) — `serve` never returns, so a buffered error line is a lost error line.
|
||||
|
||||
Because every boot revalidates the file, a latent file error is no longer a
|
||||
first-boot-only hazard: a bad push that skips its restart handler detonates at
|
||||
the next power blip. Two mitigations, both in ruling 9: the shipped unit gains
|
||||
`RestartPreventExitStatus=2 64` (retrying a config fault every 2 s is pure
|
||||
loop; the journal holds the diagnostics), and the deployment docs mandate
|
||||
`nxdns check --config=<file>` as the pre-restart gate in any Ansible
|
||||
handler.
|
||||
Because every boot revalidates the file, a latent file error is no longer a first-boot-only hazard: a bad push that skips its restart handler detonates at the next power blip. Two mitigations, both in ruling 9: the shipped unit gains `RestartPreventExitStatus=2 64` (retrying a config fault every 2 s is pure loop; the journal holds the diagnostics), and the deployment docs mandate `nxdns check --config=<file>` as the pre-restart gate in any Ansible handler.
|
||||
|
||||
Db mode: steps 2-3 are skipped entirely; the DB is truth exactly as today.
|
||||
Bare `check` on a box with no `config.db` exits 2: `no config database at <path>`
|
||||
plus the ruling-6 hint line — the deleted heuristic's "nothing to check"
|
||||
branch (cli.zig:582-587) is replaced, not dropped.
|
||||
Db mode: steps 2-3 are skipped entirely; the DB is truth exactly as today. Bare `check` on a box with no `config.db` exits 2: `no config database at <path>` plus the ruling-6 hint line — the deleted heuristic's "nothing to check" branch (cli.zig:582-587) is replaced, not dropped.
|
||||
|
||||
### 3. Reconcile engine: replace declarative state, preserve runtime state by stable identity
|
||||
|
||||
New module `src/config/reconcile.zig`, replacing `applyToDb`'s wipe+reinsert.
|
||||
The defect it exists to fix: today's import deletes and reinserts
|
||||
`blocklist_sources` including runtime columns (checksum, `last_updated`,
|
||||
counters — config_schema.zig:46-56), and compiled blocklists are keyed by
|
||||
source row id (`<id>.list`/`<id>.wild`). A naive re-import every boot would
|
||||
force a full re-download and recompile of every blocklist on every restart.
|
||||
New module `src/config/reconcile.zig`, replacing `applyToDb`'s wipe+reinsert. The defect it exists to fix: today's import deletes and reinserts `blocklist_sources` including runtime columns (checksum, `last_updated`, counters — config_schema.zig:46-56), and compiled blocklists are keyed by source row id (`<id>.list`/`<id>.wild`). A naive re-import every boot would force a full re-download and recompile of every blocklist on every restart.
|
||||
|
||||
Contract, per table: match rows by identity key ⇒ UPDATE declarative columns
|
||||
in place (row id survives); present in DB but absent from the file ⇒ DELETE;
|
||||
present in file but not DB ⇒ INSERT. Never wipe. One transaction. The
|
||||
`clients` table refines the match rule with promotion (below): an observed row
|
||||
whose IP the file declares is matched and updated, never deleted.
|
||||
Contract, per table: match rows by identity key ⇒ UPDATE declarative columns in place (row id survives); present in DB but absent from the file ⇒ DELETE; present in file but not DB ⇒ INSERT. Never wipe. One transaction. The `clients` table refines the match rule with promotion (below): an observed row whose IP the file declares is matched and updated, never deleted.
|
||||
|
||||
**Identity matching is on canonical forms.** Import already canonicalizes
|
||||
client IPs and prefixes before insert (import.zig:226-237; `FD00:0:0:0:0:0:0:1`
|
||||
stores as `fd00::1`) and the schema documents the columns as canonical text.
|
||||
The engine canonicalizes file values *before* matching; matching the raw file
|
||||
string against the canonical column would churn ids under an unchanged
|
||||
non-canonical file, violating ruling 5 in a way an exported-config test
|
||||
(export emits canonical forms) cannot catch. The idempotence test includes a
|
||||
non-canonical file.
|
||||
**Identity matching is on canonical forms.** Import already canonicalizes client IPs and prefixes before insert (import.zig:226-237; `FD00:0:0:0:0:0:0:1` stores as `fd00::1`) and the schema documents the columns as canonical text. The engine canonicalizes file values *before* matching; matching the raw file string against the canonical column would churn ids under an unchanged non-canonical file, violating ruling 5 in a way an exported-config test (export emits canonical forms) cannot catch. The idempotence test includes a non-canonical file.
|
||||
|
||||
**Writes only on difference.** A matched row whose declarative columns already
|
||||
equal the file's values gets no UPDATE. This is stronger than byte-stability:
|
||||
reconciling an unchanged file performs **zero writes**, so a no-op boot needs
|
||||
no WAL headroom and a querylog-full SD card cannot brick a file-mode restart
|
||||
that db mode would survive. Testable: the second reconcile returns an all-zero
|
||||
summary and `sqlite3_total_changes` does not move.
|
||||
**Writes only on difference.** A matched row whose declarative columns already equal the file's values gets no UPDATE. This is stronger than byte-stability: reconciling an unchanged file performs **zero writes**, so a no-op boot needs no WAL headroom and a querylog-full SD card cannot brick a file-mode restart that db mode would survive. Testable: the second reconcile returns an all-zero summary and `sqlite3_total_changes` does not move.
|
||||
|
||||
| Table | Identity key | Declarative columns | Runtime-owned (preserved on match) |
|
||||
|---|---|---|---|
|
||||
@@ -191,77 +70,21 @@ summary and `sqlite3_total_changes` does not move.
|
||||
| `group_sources` | `(group_id, source_id)`, resolved via the name/url maps | whole row | — |
|
||||
| `settings` | key | value via `settings_repo.putSetting` upsert, on difference only | delete keys not produced by `toSettings`, **except** `web.password_hash`, which the reconciler owns directly (ruling 4) |
|
||||
|
||||
**Pass order (binding).** `delete_order` alone under-specifies the engine:
|
||||
inserts need parents before children, and a group DELETE cascades through
|
||||
`rules`, `client_prefixes`, `group_sources` (config_schema.zig:35, :60, :67),
|
||||
which could silently undo child-table work and corrupt the diff counts if
|
||||
deletes ran first or interleaved. The order is:
|
||||
**Pass order (binding).** `delete_order` alone under-specifies the engine: inserts need parents before children, and a group DELETE cascades through `rules`, `client_prefixes`, `group_sources` (config_schema.zig:35, :60, :67), which could silently undo child-table work and corrupt the diff counts if deletes ran first or interleaved. The order is:
|
||||
|
||||
- **Phase A — upserts, parents first**: `groups`, `blocklist_sources`, then
|
||||
the referrers (`clients`, `client_prefixes`, `rules`, `group_sources`,
|
||||
`upstreams`, `local_records`, `forward_zones`, `settings`), with the
|
||||
name→id / url→id maps built after the parent passes. Client promotion
|
||||
happens here, in the `clients` pass.
|
||||
- **Phase B — delete-absent, child first**, in `delete_order`
|
||||
(config_schema.zig:99). Because every declarative child of a dying group is
|
||||
itself absent from the file (validate guarantees file rules/prefixes
|
||||
reference file groups), the child passes have already deleted and counted
|
||||
them by the time the parent DELETE runs; the FK cascades become a safety
|
||||
net, never the accountant.
|
||||
- Immediately before the `groups` delete pass: observed clients
|
||||
(`hand_edited=0`) whose `group_id` belongs to a dying group are reassigned
|
||||
to the default group (id 1). `clients.group_id` has **no** ON DELETE clause
|
||||
(config_schema.zig:26) — without this step, removing or renaming a group
|
||||
that observed devices had been assigned to trips the FK mid-transaction:
|
||||
exit 1, restart loop, and a `check` that said the file was fine. The
|
||||
reassignment is the semantics we want anyway: the operator un-declared the
|
||||
group, not the devices.
|
||||
- **Phase A — upserts, parents first**: `groups`, `blocklist_sources`, then the referrers (`clients`, `client_prefixes`, `rules`, `group_sources`, `upstreams`, `local_records`, `forward_zones`, `settings`), with the name→id / url→id maps built after the parent passes. Client promotion happens here, in the `clients` pass.
|
||||
- **Phase B — delete-absent, child first**, in `delete_order` (config_schema.zig:99). Because every declarative child of a dying group is itself absent from the file (validate guarantees file rules/prefixes reference file groups), the child passes have already deleted and counted them by the time the parent DELETE runs; the FK cascades become a safety net, never the accountant.
|
||||
- Immediately before the `groups` delete pass: observed clients (`hand_edited=0`) whose `group_id` belongs to a dying group are reassigned to the default group (id 1). `clients.group_id` has **no** ON DELETE clause (config_schema.zig:26) — without this step, removing or renaming a group that observed devices had been assigned to trips the FK mid-transaction: exit 1, restart loop, and a `check` that said the file was fine. The reassignment is the semantics we want anyway: the operator un-declared the group, not the devices.
|
||||
|
||||
Consequences that make the fix complete: an unchanged URL keeps id **and**
|
||||
checksum + counters + `last_updated` together, so `loadSource` never sees
|
||||
`.never_fetched`, `needsRefresh` does not fire spuriously (preserving
|
||||
`last_updated` matters — checksum alone is not enough), `sweepOrphans` never
|
||||
orphans `<id>.list`/`<id>.wild`, and a restart costs zero downloads. A changed
|
||||
URL is a new identity: new row, new id, fresh download — consistent with
|
||||
artifacts keyed by id. **Accepted trade, stated**: the old row's compiled
|
||||
`<old-id>.list`/`<old-id>.wild` are orphaned at commit and swept before the
|
||||
new source's first fetch succeeds, so a URL edit whose new host is down leaves
|
||||
that list unenforced until a fetch lands. Deferring the sweep until a
|
||||
successor fetch would need a cross-artifact lifecycle for an event that is
|
||||
rare, operator-initiated, and bounded by the scheduler's retry — not worth the
|
||||
machinery on a household box. The REST `updateSource` keeps runtime columns
|
||||
across a URL edit; in file mode that route is rejected (ruling 7), so the
|
||||
divergence is unreachable; in db mode the reconciler only runs via explicit
|
||||
`import`.
|
||||
Consequences that make the fix complete: an unchanged URL keeps id **and** checksum + counters + `last_updated` together, so `loadSource` never sees `.never_fetched`, `needsRefresh` does not fire spuriously (preserving `last_updated` matters — checksum alone is not enough), `sweepOrphans` never orphans `<id>.list`/`<id>.wild`, and a restart costs zero downloads. A changed URL is a new identity: new row, new id, fresh download — consistent with artifacts keyed by id. **Accepted trade, stated**: the old row's compiled `<old-id>.list`/`<old-id>.wild` are orphaned at commit and swept before the new source's first fetch succeeds, so a URL edit whose new host is down leaves that list unenforced until a fetch lands. Deferring the sweep until a successor fetch would need a cross-artifact lifecycle for an event that is rare, operator-initiated, and bounded by the scheduler's retry — not worth the machinery on a household box. The REST `updateSource` keeps runtime columns across a URL edit; in file mode that route is rejected (ruling 7), so the divergence is unreachable; in db mode the reconciler only runs via explicit `import`.
|
||||
|
||||
Related clock fix, same subsystem: `needsRefresh` is wall-clock arithmetic
|
||||
(`now - last >= interval`, manager.zig:1201-1202) and the Pi has no RTC. A
|
||||
fetch stamped while the clock was ahead (pre-NTP boot, restored image)
|
||||
suspends refresh until real time catches the future timestamp — and this
|
||||
design's idempotence would faithfully preserve the poison forever, having
|
||||
deleted the wipe that used to be the accidental reset lever. The engine's
|
||||
sibling fix: `needsRefresh` treats `last_updated > now` as refresh-due. One
|
||||
comparison, with a test.
|
||||
Related clock fix, same subsystem: `needsRefresh` is wall-clock arithmetic (`now - last >= interval`, manager.zig:1201-1202) and the Pi has no RTC. A fetch stamped while the clock was ahead (pre-NTP boot, restored image) suspends refresh until real time catches the future timestamp — and this design's idempotence would faithfully preserve the poison forever, having deleted the wipe that used to be the accidental reset lever. The engine's sibling fix: `needsRefresh` treats `last_updated > now` as refresh-due. One comparison, with a test.
|
||||
|
||||
Client promotion in place makes the `saved_clients` lift/merge/restore
|
||||
scaffolding (import.zig:268-361, including `merge_observed_timestamps_sql`)
|
||||
unnecessary: that machinery exists only because the wipe destroyed
|
||||
`first_seen`/`last_seen` and had to smuggle them across. With no wipe, the
|
||||
semantics it encodes — observed history survives declaration — are a plain
|
||||
UPDATE that never touches the timestamp columns. The scaffolding dies with
|
||||
the wipe (Deletions); its tests' semantics move to the promotion tests.
|
||||
Client promotion in place makes the `saved_clients` lift/merge/restore scaffolding (import.zig:268-361, including `merge_observed_timestamps_sql`) unnecessary: that machinery exists only because the wipe destroyed `first_seen`/`last_seen` and had to smuggle them across. With no wipe, the semantics it encodes — observed history survives declaration — are a plain UPDATE that never touches the timestamp columns. The scaffolding dies with the wipe (Deletions); its tests' semantics move to the promotion tests.
|
||||
|
||||
**Ordering invariant**: reconcile commits before `manager.reload`
|
||||
(app.zig:400) runs and before the scheduler's `sweepOrphans` can fire, so
|
||||
preserved ids and checksums are visible to the filter layer before any
|
||||
pruning. This holds by construction — reconcile completes inside `serve()`
|
||||
before the manager exists — and is enforced *behaviorally*, not by a
|
||||
statement-order unit test with nothing to grip: the restart-no-redownload
|
||||
integration test (Tests, below) fails if anything between reconcile and
|
||||
reload re-orders or wipes. No `serve()` restructuring is chartered for this.
|
||||
**Ordering invariant**: reconcile commits before `manager.reload` (app.zig:400) runs and before the scheduler's `sweepOrphans` can fire, so preserved ids and checksums are visible to the filter layer before any pruning. This holds by construction — reconcile completes inside `serve()` before the manager exists — and is enforced *behaviorally*, not by a statement-order unit test with nothing to grip: the restart-no-redownload integration test (Tests, below) fails if anything between reconcile and reload re-orders or wipes. No `serve()` restructuring is chartered for this.
|
||||
|
||||
The engine returns a summary (ruling 8's input and the cross-session
|
||||
contract):
|
||||
The engine returns a summary (ruling 8's input and the cross-session contract):
|
||||
|
||||
```zig
|
||||
pub const TableCounts = struct { inserted: u32, updated: u32, deleted: u32 };
|
||||
@@ -274,499 +97,138 @@ pub const Summary = struct {
|
||||
};
|
||||
```
|
||||
|
||||
`updated` counts only rows actually written (writes-on-difference);
|
||||
observed-client preservation counts nothing; promotion and reassignment count
|
||||
as `updated` on `clients`. `deleted` on `clients` means exactly one thing: a
|
||||
formerly declared (`hand_edited=1`) row absent from the file. An unchanged
|
||||
file yields an all-zero summary.
|
||||
`updated` counts only rows actually written (writes-on-difference); observed-client preservation counts nothing; promotion and reassignment count as `updated` on `clients`. `deleted` on `clients` means exactly one thing: a formerly declared (`hand_edited=1`) row absent from the file. An unchanged file yields an all-zero summary.
|
||||
|
||||
New repo verbs carry the engine (`sources_repo.upsertByUrl`,
|
||||
per-table `deleteWhereNotIn`-style helpers), not new call ordering. `applyToDb`
|
||||
and its wipe loop are deleted; import.zig keeps its parse/validate/diagnostics
|
||||
plumbing.
|
||||
New repo verbs carry the engine (`sources_repo.upsertByUrl`, per-table `deleteWhereNotIn`-style helpers), not new call ordering. `applyToDb` and its wipe loop are deleted; import.zig keeps its parse/validate/diagnostics plumbing.
|
||||
|
||||
### 4. Password: the file overrides when it speaks, and only then
|
||||
|
||||
`model.Web` changes to `password: ?[]const u8 = null` and
|
||||
`password_hash: ?[]const u8 = null`. The settings bridge splits its skip
|
||||
policy by direction — `isSkipped` becomes two functions, because encode and
|
||||
decode need different sets:
|
||||
`model.Web` changes to `password: ?[]const u8 = null` and `password_hash: ?[]const u8 = null`. The settings bridge splits its skip policy by direction — `isSkipped` becomes two functions, because encode and decode need different sets:
|
||||
|
||||
- `isEncodeSkipped` = {`web.password`, `web.password_hash`}: `toSettings`
|
||||
stops emitting `web.password_hash` (the reconciler owns that settings row
|
||||
directly) and continues to never emit `web.password`.
|
||||
- `isDecodeSkipped` = {`web.password`} only: `fromSettings` **still loads**
|
||||
`web.password_hash` from the settings table — the reconciler owning the
|
||||
write does not mean the read path stops seeing it. Skipping it on decode
|
||||
would leave `cfg.web.password_hash` null on every read path
|
||||
(`config_export.readConfig` at export.zig:52, `mutations.loadConfig`), turn
|
||||
`authEnabled` false, and silently disable auth in both modes.
|
||||
- `decodeValue` gains an `.optional => try decodeValue(child, text)` arm
|
||||
(model.zig:408-421 has none today; an unskipped optional field is currently
|
||||
a `@compileError`). Absent key ⇒ default `null`; present ⇒ non-null.
|
||||
- `isEncodeSkipped` = {`web.password`, `web.password_hash`}: `toSettings` stops emitting `web.password_hash` (the reconciler owns that settings row directly) and continues to never emit `web.password`.
|
||||
- `isDecodeSkipped` = {`web.password`} only: `fromSettings` **still loads** `web.password_hash` from the settings table — the reconciler owning the write does not mean the read path stops seeing it. Skipping it on decode would leave `cfg.web.password_hash` null on every read path (`config_export.readConfig` at export.zig:52, `mutations.loadConfig`), turn `authEnabled` false, and silently disable auth in both modes.
|
||||
- `decodeValue` gains an `.optional => try decodeValue(child, text)` arm (model.zig:408-421 has none today; an unskipped optional field is currently a `@compileError`). Absent key ⇒ default `null`; present ⇒ non-null.
|
||||
|
||||
`auth.authEnabled` becomes `(web.password_hash orelse "").len != 0`
|
||||
(auth.zig:62-63); app.zig:475's `.live_hash = .init(...)` unwraps with
|
||||
`orelse ""`. The cases:
|
||||
`auth.authEnabled` becomes `(web.password_hash orelse "").len != 0` (auth.zig:62-63); app.zig:475's `.live_hash = .init(...)` unwraps with `orelse ""`. The cases:
|
||||
|
||||
- **Both set**: existing error (`PasswordAndHashBothSet`, import.zig:250,
|
||||
validate.zig:395 — the check ports to `!= null` on both fields).
|
||||
- **`password` present and empty**: rejected by `validate` with a new
|
||||
diagnostic naming the remedy — `password_hash = ""` is how auth is disabled
|
||||
declaratively. Without this rejection, `.password = ""` would hash the
|
||||
empty string into a non-empty PHC (`authEnabled` true) while auth.zig:90
|
||||
refuses every empty-password login: auth on, unreachable. A declarative
|
||||
fault, exit 2, so ruling 2's check/run agreement holds.
|
||||
- **`password` set (non-empty plaintext)**: verify against the stored
|
||||
`web.password_hash` with argon2; on match keep the stored hash, on mismatch
|
||||
or absent stored hash, hash fresh. The justification is ruling 5 alone:
|
||||
hashing unconditionally generates a fresh salt per apply and breaks
|
||||
byte-stability. It is **not** a cost saving — argon2 verification recomputes
|
||||
the full function (same t=2, m=19 MiB) with the stored salt, so
|
||||
verify-and-keep costs exactly what hashing costs. Do not "optimize" the
|
||||
verify away with any cached-plaintext scheme; that would be a security bug.
|
||||
- **`password_hash` set**: written verbatim. An explicit `password_hash = ""`
|
||||
is the declarative way to disable auth (auth.zig's documented empty-hash
|
||||
state).
|
||||
- **Neither set**: the stored hash is untouched. This is the deliberate
|
||||
carve-out from file-as-sole-truth, because the naive alternative is a trap
|
||||
the red team walked straight into: `toSettings` today always emits
|
||||
`web.password_hash` with default `""` (model.zig:534, :93), so an operator
|
||||
who hand-trims the ugly PHC string out of an exported file — meaning "keep
|
||||
the current password" — would silently reconcile `""` over the stored hash
|
||||
and open the admin UI to the LAN (`authEnabled` is `len != 0`, auth.zig:63).
|
||||
Silence must mean "keep", and disabling auth must require the explicit
|
||||
empty string.
|
||||
- **Both set**: existing error (`PasswordAndHashBothSet`, import.zig:250, validate.zig:395 — the check ports to `!= null` on both fields).
|
||||
- **`password` present and empty**: rejected by `validate` with a new diagnostic naming the remedy — `password_hash = ""` is how auth is disabled declaratively. Without this rejection, `.password = ""` would hash the empty string into a non-empty PHC (`authEnabled` true) while auth.zig:90 refuses every empty-password login: auth on, unreachable. A declarative fault, exit 2, so ruling 2's check/run agreement holds.
|
||||
- **`password` set (non-empty plaintext)**: verify against the stored `web.password_hash` with argon2; on match keep the stored hash, on mismatch or absent stored hash, hash fresh. The justification is ruling 5 alone: hashing unconditionally generates a fresh salt per apply and breaks byte-stability. It is **not** a cost saving — argon2 verification recomputes the full function (same t=2, m=19 MiB) with the stored salt, so verify-and-keep costs exactly what hashing costs. Do not "optimize" the verify away with any cached-plaintext scheme; that would be a security bug.
|
||||
- **`password_hash` set**: written verbatim. An explicit `password_hash = ""` is the declarative way to disable auth (auth.zig's documented empty-hash state).
|
||||
- **Neither set**: the stored hash is untouched. This is the deliberate carve-out from file-as-sole-truth, because the naive alternative is a trap the red team walked straight into: `toSettings` today always emits `web.password_hash` with default `""` (model.zig:534, :93), so an operator who hand-trims the ugly PHC string out of an exported file — meaning "keep the current password" — would silently reconcile `""` over the stored hash and open the admin UI to the LAN (`authEnabled` is `len != 0`, auth.zig:63). Silence must mean "keep", and disabling auth must require the explicit empty string.
|
||||
|
||||
**Export's canonical form**: `password = null`, `password_hash = <stored
|
||||
value, including "">`. A non-null `password` is never emitted — the current
|
||||
`cfg.web.password = "";` at export.zig:71 ported literally would make every
|
||||
export carry a present-empty password beside a stored hash, tripping
|
||||
`PasswordAndHashBothSet` on re-import: export's own output failing its own
|
||||
rule, breaking ruling 9's adoption walkthrough and ruling 5's round trip. The
|
||||
comment at export.zig:67-70 and the header sample change with it. With
|
||||
`password = null`, the empty-plaintext rejection above does not fire and the
|
||||
"hash written verbatim" branch keeps idempotence.
|
||||
**Export's canonical form**: `password = null`, `password_hash = <stored value, including "">`. A non-null `password` is never emitted — the current `cfg.web.password = "";` at export.zig:71 ported literally would make every export carry a present-empty password beside a stored hash, tripping `PasswordAndHashBothSet` on re-import: export's own output failing its own rule, breaking ruling 9's adoption walkthrough and ruling 5's round trip. The comment at export.zig:67-70 and the header sample change with it. With `password = null`, the empty-plaintext rejection above does not fire and the "hash written verbatim" branch keeps idempotence.
|
||||
|
||||
Any auth transition (enabled/disabled/rotated) is reported in the summary and
|
||||
printed by ruling 8 — an auth change is never a silent line item in a count.
|
||||
Any auth transition (enabled/disabled/rotated) is reported in the summary and printed by ruling 8 — an auth change is never a silent line item in a count.
|
||||
|
||||
### 5. Idempotence is the invariant
|
||||
|
||||
Reconciling the same file twice produces a byte-identical database — ids,
|
||||
checksums, timestamps, `created_at`, password hash, the whole settings
|
||||
table — **and** the second pass performs zero writes (all-zero summary,
|
||||
`total_changes` unmoved). The unit test asserts
|
||||
byte-stability via the `dump()` helper (import.zig:469), which is rewritten to
|
||||
iterate an explicit all-tables list (`config_schema.table_names`, all ten
|
||||
content-bearing tables) because its current driver, `content_tables`, is
|
||||
deleted (ruling 6). Anything that churns under an unchanged file — including a
|
||||
*non-canonical but equivalent* file — is a bug in the engine, by definition.
|
||||
(This invariant is what forces ruling 3's rules-multiset, canonical matching,
|
||||
and writes-on-difference, and ruling 4's password handling — the places a
|
||||
naive design silently violates it. It is also half of why ruling 8 persists
|
||||
no authority state at all.)
|
||||
Reconciling the same file twice produces a byte-identical database — ids, checksums, timestamps, `created_at`, password hash, the whole settings table — **and** the second pass performs zero writes (all-zero summary, `total_changes` unmoved). The unit test asserts byte-stability via the `dump()` helper (import.zig:469), which is rewritten to iterate an explicit all-tables list (`config_schema.table_names`, all ten content-bearing tables) because its current driver, `content_tables`, is deleted (ruling 6). Anything that churns under an unchanged file — including a *non-canonical but equivalent* file — is a bug in the engine, by definition. (This invariant is what forces ruling 3's rules-multiset, canonical matching, and writes-on-difference, and ruling 4's password handling — the places a naive design silently violates it. It is also half of why ruling 8 persists no authority state at all.)
|
||||
|
||||
### 6. `import` becomes a thin wrapper over reconcile; the guard becomes diff-gated
|
||||
|
||||
`nxdns import` is db mode's one-shot apply and the restore tool, reimplemented
|
||||
on the reconcile engine. Reconcile is non-destructive of *runtime* state, but
|
||||
deletion of declarative rows absent from the file is still a first-class
|
||||
outcome (ruling 3) — a mistaken `nxdns import ./wrong.zon` against a
|
||||
configured DB would still remove every group, rule, upstream, and source not
|
||||
in that file. So the guard is **retargeted, not deleted**: emptiness-gating
|
||||
(`isEmpty`) becomes diff-gating.
|
||||
`nxdns import` is db mode's one-shot apply and the restore tool, reimplemented on the reconcile engine. Reconcile is non-destructive of *runtime* state, but deletion of declarative rows absent from the file is still a first-class outcome (ruling 3) — a mistaken `nxdns import ./wrong.zon` against a configured DB would still remove every group, rule, upstream, and source not in that file. So the guard is **retargeted, not deleted**: emptiness-gating (`isEmpty`) becomes diff-gating.
|
||||
|
||||
- After the reconcile passes, still inside the same `BEGIN IMMEDIATE` —
|
||||
preserving import.zig:199-203's deliberate check-inside-the-write-lock
|
||||
property, no TOCTOU — if any table's `deleted != 0` and no override flag,
|
||||
roll back and fail exit 2 with the per-table delete counts in the message.
|
||||
Observed-client reassignment is not declarative data and never trips the
|
||||
gate (and under ruling 3's promotion rule, observed rows are never deleted
|
||||
by reconcile at all).
|
||||
- The flag is `--allow-delete` (`Options.allow_delete`), the renamed
|
||||
`--force`; the error is `error.DestructiveImport`, the renamed
|
||||
`DatabaseNotEmpty`, exit-2-mapped where cli.zig:533 maps today. This is
|
||||
strictly better than the emptiness guard: additive and edit-only re-imports
|
||||
stop needing a flag at all, and the flag now names what it permits.
|
||||
"Edit-only" means edits to declarative columns on a matched identity; an
|
||||
edit that *changes an identity column* — a group name, a source or upstream
|
||||
url, a prefix, a zone, a rule tuple, a local-record identity — is a delete
|
||||
plus an insert to the engine (ruling 3) and needs the flag. cli.md states
|
||||
the distinction.
|
||||
- `import.isEmpty` and `content_tables` still die — the diff-gate needs no
|
||||
table list.
|
||||
- After the reconcile passes, still inside the same `BEGIN IMMEDIATE` — preserving import.zig:199-203's deliberate check-inside-the-write-lock property, no TOCTOU — if any table's `deleted != 0` and no override flag, roll back and fail exit 2 with the per-table delete counts in the message. Observed-client reassignment is not declarative data and never trips the gate (and under ruling 3's promotion rule, observed rows are never deleted by reconcile at all).
|
||||
- The flag is `--allow-delete` (`Options.allow_delete`), the renamed `--force`; the error is `error.DestructiveImport`, the renamed `DatabaseNotEmpty`, exit-2-mapped where cli.zig:533 maps today. This is strictly better than the emptiness guard: additive and edit-only re-imports stop needing a flag at all, and the flag now names what it permits. "Edit-only" means edits to declarative columns on a matched identity; an edit that *changes an identity column* — a group name, a source or upstream url, a prefix, a zone, a rule tuple, a local-record identity — is a delete plus an insert to the engine (ruling 3) and needs the flag. cli.md states the distinction.
|
||||
- `import.isEmpty` and `content_tables` still die — the diff-gate needs no table list.
|
||||
|
||||
`import` does not detect a file-managed DB, because nothing records one:
|
||||
authority lives in the invocation (ruling 1) and the DB carries no marker
|
||||
(ruling 8). On a box whose unit runs file mode, `import` behaves like any
|
||||
other import — the diff-gate guards deletion, and the next boot's reconcile
|
||||
converges the DB back to the file, its summary reporting what it corrected.
|
||||
The docs own this story plainly: `import` is a **stop-first operation**, on a
|
||||
file-mode box doubly so — against a running instance its effect is partial
|
||||
(the runtime divergence below) and lasts only until the next restart. The
|
||||
file-authority contract is stated with the same precision everywhere: the
|
||||
file is the sole declarative source, **converged at every boot** — not a
|
||||
lock on the database between boots. A detect-and-warn variant was
|
||||
designed and deleted (ruling 8) — a per-invocation warning cannot *prevent*
|
||||
db-side writes anyway, the CLI user is root on their own box, and the
|
||||
asymmetry with the web layer's 403 is intentional: the web UI has
|
||||
anonymous-ish LAN users, the CLI has the operator.
|
||||
`import` does not detect a file-managed DB, because nothing records one: authority lives in the invocation (ruling 1) and the DB carries no marker (ruling 8). On a box whose unit runs file mode, `import` behaves like any other import — the diff-gate guards deletion, and the next boot's reconcile converges the DB back to the file, its summary reporting what it corrected. The docs own this story plainly: `import` is a **stop-first operation**, on a file-mode box doubly so — against a running instance its effect is partial (the runtime divergence below) and lasts only until the next restart. The file-authority contract is stated with the same precision everywhere: the file is the sole declarative source, **converged at every boot** — not a lock on the database between boots. A detect-and-warn variant was designed and deleted (ruling 8) — a per-invocation warning cannot *prevent* db-side writes anyway, the CLI user is root on their own box, and the asymmetry with the web layer's 403 is intentional: the web UI has anonymous-ish LAN users, the CLI has the operator.
|
||||
|
||||
What a mid-run import against a *running* file-mode instance actually does —
|
||||
documented, because the divergence is partial, not merely deferred:
|
||||
`Manager.reload` re-reads `blocklist_sources`, `groups`, `group_sources`,
|
||||
`rules`, `clients`, `client_prefixes` from the DB at runtime
|
||||
(manager.zig:405-489) and the scheduler runs `sweepOrphans` before every pass
|
||||
(manager.zig:1095, :1115), so filtering follows the imported rows on the next
|
||||
reload and can unlink the compiled `<id>.list`/`<id>.wild` of sources the
|
||||
import deleted — while upstreams, listeners, and settings stay at boot
|
||||
values, and the web UI shows the imported state under the file-authority
|
||||
banner. The next restart's reconcile re-inserts deleted sources from the file
|
||||
with new ids: a full re-download, the exact cost this design exists to
|
||||
prevent. That is why the docs name the restart requirement and the
|
||||
re-download cost. The settings envelope's `reconciled_at` cannot see a CLI
|
||||
import — ruling 7's weakened claim covers exactly this.
|
||||
What a mid-run import against a *running* file-mode instance actually does — documented, because the divergence is partial, not merely deferred: `Manager.reload` re-reads `blocklist_sources`, `groups`, `group_sources`, `rules`, `clients`, `client_prefixes` from the DB at runtime (manager.zig:405-489) and the scheduler runs `sweepOrphans` before every pass (manager.zig:1095, :1115), so filtering follows the imported rows on the next reload and can unlink the compiled `<id>.list`/`<id>.wild` of sources the import deleted — while upstreams, listeners, and settings stay at boot values, and the web UI shows the imported state under the file-authority banner. The next restart's reconcile re-inserts deleted sources from the file with new ids: a full re-download, the exact cost this design exists to prevent. That is why the docs name the restart requirement and the re-download cost. The settings envelope's `reconciled_at` cannot see a CLI import — ruling 7's weakened claim covers exactly this.
|
||||
|
||||
The startup-vs-import race needs no code: both paths take `BEGIN IMMEDIATE`
|
||||
under the 5 s busy timeout (db.zig:252), so the outcome is ordering, not
|
||||
corruption — an import racing a restart may be reverted by the reconcile that
|
||||
wins the lock second. Documented, not engineered around.
|
||||
The startup-vs-import race needs no code: both paths take `BEGIN IMMEDIATE` under the 5 s busy timeout (db.zig:252), so the outcome is ordering, not corruption — an import racing a restart may be reverted by the reconcile that wins the lock second. Documented, not engineered around.
|
||||
|
||||
First-run story in db mode, after bootstrap dies: a fresh empty DB fails
|
||||
validation naturally (`NoUsableUpstreams`, exit 2). The remediation hint —
|
||||
one fixed line naming `nxdns import` and `run --config` — is owned by
|
||||
**cli.zig's db-source fault renderer** (one arm, beside the exit-code mapping;
|
||||
Zig errors carry no text and validate.zig must stay mode-blind), and the same
|
||||
line serves bare `check` with no DB (ruling 2). One location, R2's charter.
|
||||
First-run story in db mode, after bootstrap dies: a fresh empty DB fails validation naturally (`NoUsableUpstreams`, exit 2). The remediation hint — one fixed line naming `nxdns import` and `run --config` — is owned by **cli.zig's db-source fault renderer** (one arm, beside the exit-code mapping; Zig errors carry no text and validate.zig must stay mode-blind), and the same line serves bare `check` with no DB (ruling 2). One location, R2's charter.
|
||||
|
||||
`export` is unchanged in role: the diagnostic/capture tool in both modes, and
|
||||
the file-mode adoption tool (its canonical password form changes per
|
||||
ruling 4).
|
||||
`export` is unchanged in role: the diagnostic/capture tool in both modes, and the file-mode adoption tool (its canonical password form changes per ruling 4).
|
||||
|
||||
### 7. Web layer: policy as data, rejected after auth, plain 403
|
||||
|
||||
`WebState` gains `authority: union(enum) { database, managed_file: []const u8 }`
|
||||
and `reconciled_at: ?i64`, built where `WebState` is assembled (app.zig:468).
|
||||
The `managed_file` path slice is owned by `serve`'s arena, which outlives
|
||||
`WebState` — R2 provides it, R3 consumes it, neither copies. `reconciled_at`
|
||||
is stamped by `serve` immediately after the reconcile commits — same process,
|
||||
same frame, a local — and is `null` in db mode, which never reconciles.
|
||||
`WebState` gains `authority: union(enum) { database, managed_file: []const u8 }` and `reconciled_at: ?i64`, built where `WebState` is assembled (app.zig:468). The `managed_file` path slice is owned by `serve`'s arena, which outlives `WebState` — R2 provides it, R3 consumes it, neither copies. `reconciled_at` is stamped by `serve` immediately after the reconcile commits — same process, same frame, a local — and is `null` in db mode, which never reconciles.
|
||||
|
||||
`RouteInfo` gains `policy: enum { read, config_write, runtime_action }` beside
|
||||
`auth` and `rate_limit` — policy-as-data, matching the table's existing style.
|
||||
No default value: all 56 route entries state their class explicitly, and
|
||||
router.zig's `test_table` (:198) gains the field too. Classification is
|
||||
per-route, not per-prefix: `POST /api/blocklists/update` is a
|
||||
`runtime_action`; its CRUD siblings are `config_write`.
|
||||
`RouteInfo` gains `policy: enum { read, config_write, runtime_action }` beside `auth` and `rate_limit` — policy-as-data, matching the table's existing style. No default value: all 56 route entries state their class explicitly, and router.zig's `test_table` (:198) gains the field too. Classification is per-route, not per-prefix: `POST /api/blocklists/update` is a `runtime_action`; its CRUD siblings are `config_write`.
|
||||
|
||||
- Config writes, rejected in file mode: all group / blocklist / rule /
|
||||
local-record / forward-zone / upstream / client-prefix mutations,
|
||||
`PUT /api/settings` (password changes go through the file; its
|
||||
`live_hash.installAndRevoke` side effect never fires in file mode), and
|
||||
client PUT — naming or regrouping an observed client is declarative drift
|
||||
(open question 1).
|
||||
- Client DELETE is a **runtime action**: deleting an observed
|
||||
(`hand_edited=0`) row discards runtime state the file never declared —
|
||||
without this, a mis-identified or departed device's row is immortal in file
|
||||
mode, since the file can only promote IPs, never remove them. Deleting a
|
||||
*declared* client contradicts the file: the handler answers the same 403
|
||||
envelope. This is the one policy decision that needs a row read; it lives
|
||||
in the client handler, not the router. (After ruling 3's promotion, a
|
||||
declared IP's row *is* declared — DELETE answers 403, the intended
|
||||
reading.)
|
||||
- Runtime actions, always live: pause, blocklist refresh, cert reload,
|
||||
login/logout.
|
||||
- Config writes, rejected in file mode: all group / blocklist / rule / local-record / forward-zone / upstream / client-prefix mutations, `PUT /api/settings` (password changes go through the file; its `live_hash.installAndRevoke` side effect never fires in file mode), and client PUT — naming or regrouping an observed client is declarative drift (open question 1).
|
||||
- Client DELETE is a **runtime action**: deleting an observed (`hand_edited=0`) row discards runtime state the file never declared — without this, a mis-identified or departed device's row is immortal in file mode, since the file can only promote IPs, never remove them. Deleting a *declared* client contradicts the file: the handler answers the same 403 envelope. This is the one policy decision that needs a row read; it lives in the client handler, not the router. (After ruling 3's promotion, a declared IP's row *is* declared — DELETE answers 403, the intended reading.)
|
||||
- Runtime actions, always live: pause, blocklist refresh, cert reload, login/logout.
|
||||
|
||||
Enforcement lives in `router.dispatch` **after** `check_auth`, before the
|
||||
handler — match → rate limit → auth → policy. Pre-auth rejection would leak
|
||||
route existence; the codebase answers 401 first and this design keeps that.
|
||||
Enforcement lives in `router.dispatch` **after** `check_auth`, before the handler — match → rate limit → auth → policy. Pre-auth rejection would leak route existence; the codebase answers 401 first and this design keeps that.
|
||||
|
||||
Rejection is **403**, body the existing single-field envelope:
|
||||
`{"error":"configuration is managed by /etc/nxdns/config.zon; edit the file and restart"}`.
|
||||
Not 409 — that status already means constraint conflict four ways in
|
||||
openapi.yaml — and **no `code` field**: 403 is unused today, so the status
|
||||
alone is machine-readable; the UI learns authority declaratively from
|
||||
`GET /api/settings`, not by probing errors; and the single-property `Error`
|
||||
schema, golden contract samples, and `api.ts` stay untouched. An optional field
|
||||
with no consumer is machinery, not a contract.
|
||||
Rejection is **403**, body the existing single-field envelope: `{"error":"configuration is managed by /etc/nxdns/config.zon; edit the file and restart"}`. Not 409 — that status already means constraint conflict four ways in openapi.yaml — and **no `code` field**: 403 is unused today, so the status alone is machine-readable; the UI learns authority declaratively from `GET /api/settings`, not by probing errors; and the single-property `Error` schema, golden contract samples, and `api.ts` stay untouched. An optional field with no consumer is machinery, not a contract.
|
||||
|
||||
The envelope must survive its own path: `respondError` today builds into a
|
||||
fixed 512-byte buffer and **silently downgrades to `text/plain`** on overflow
|
||||
(http_util.zig:315-327) — a long managed-file path (nested bind mounts) would
|
||||
demote the documented JSON envelope. Fix at the root: `respondError` gets the
|
||||
arena treatment `respondJson` already uses (`Writer.Allocating` over
|
||||
`request.arena`, :337-339) and the `respondPlain` silent-downgrade path is
|
||||
deleted — which fixes every long error message, not just this one.
|
||||
`src/web/http_util.zig` joins R3's ownership.
|
||||
The envelope must survive its own path: `respondError` today builds into a fixed 512-byte buffer and **silently downgrades to `text/plain`** on overflow (http_util.zig:315-327) — a long managed-file path (nested bind mounts) would demote the documented JSON envelope. Fix at the root: `respondError` gets the arena treatment `respondJson` already uses (`Writer.Allocating` over `request.arena`, :337-339) and the `respondPlain` silent-downgrade path is deleted — which fixes every long error message, not just this one. `src/web/http_util.zig` joins R3's ownership.
|
||||
|
||||
Authority discovery: the `GET /api/settings` envelope gains
|
||||
`authority: {mode, path, reconciled_at}` — an authenticated route, so the
|
||||
filesystem path never leaks through open `/api/version`/`/api/health`. All
|
||||
three come from `WebState` (live truth); `path` is present in file mode only
|
||||
and `reconciled_at` is nullable — `null` in db mode. Its meaning is exactly
|
||||
"**this process loaded the file at T**" — restart-pending detection: an mtime
|
||||
newer than `reconciled_at` means the running process has not loaded the
|
||||
current file. The comparison is one-directional and non-authoritative — a
|
||||
stepped clock (pre-NTP boot stamping the future, ruling 3's clock fix
|
||||
territory) or a preserved mtime (`git checkout`, `rsync -a`) can make a newer
|
||||
file look older, and the DB can move without either timestamp moving
|
||||
(ruling 6's `import`, this ruling's `runtime_action` routes). It does not
|
||||
answer "is the file what the server uses"; answering that would take content
|
||||
hashing, which the anti-requirements refuse. The UI renders a read-only
|
||||
banner (RestartBanner slot precedent) and disables mutation controls, with
|
||||
the 403 as backstop.
|
||||
Authority discovery: the `GET /api/settings` envelope gains `authority: {mode, path, reconciled_at}` — an authenticated route, so the filesystem path never leaks through open `/api/version`/`/api/health`. All three come from `WebState` (live truth); `path` is present in file mode only and `reconciled_at` is nullable — `null` in db mode. Its meaning is exactly "**this process loaded the file at T**" — restart-pending detection: an mtime newer than `reconciled_at` means the running process has not loaded the current file. The comparison is one-directional and non-authoritative — a stepped clock (pre-NTP boot stamping the future, ruling 3's clock fix territory) or a preserved mtime (`git checkout`, `rsync -a`) can make a newer file look older, and the DB can move without either timestamp moving (ruling 6's `import`, this ruling's `runtime_action` routes). It does not answer "is the file what the server uses"; answering that would take content hashing, which the anti-requirements refuse. The UI renders a read-only banner (RestartBanner slot precedent) and disables mutation controls, with the 403 as backstop.
|
||||
|
||||
### 8. The operator can see what happened
|
||||
|
||||
- `logStartup` (after `logging.install`) logs the authority:
|
||||
`authority: database` / `authority: file (/etc/nxdns/config.zon)`.
|
||||
- In file mode, the reconcile summary (ruling 3's `Summary`) prints through
|
||||
the Runner at startup, matching `seedFromFile`'s existing output discipline:
|
||||
per-table inserted/updated/deleted counts, the changed settings **keys**
|
||||
(never values), and the auth transition when there is one. It is the answer
|
||||
to "what did that restart change" without opening sqlite.
|
||||
- **Authority is never persisted.** The DB carries no record of which mode
|
||||
wrote it: authority lives in the invocation (ruling 1), per-process state
|
||||
(`WebState.authority`, `reconciled_at`) serves the API (ruling 7), and the
|
||||
journal holds the history. A persisted marker was designed twice and
|
||||
deleted twice. A per-boot `reconciled_at` settings row breaks ruling 5
|
||||
outright — `dump()` iterates the settings table, so any row rewritten per
|
||||
reconcile forfeits byte-identity, and `putSetting` moves `total_changes`,
|
||||
forfeiting zero-writes. The write-on-difference `authority.mode`/`path`
|
||||
variant survived idempotence but cost a reserved key namespace, a
|
||||
`fromSettings` decode filter, and a reconciler sweep exemption — three
|
||||
mechanisms whose only consumer was one `import` warning (ruling 6). State
|
||||
that exists to power a courtesy message is not worth a namespace. The
|
||||
settings sweep's exemption list is therefore exactly one key:
|
||||
`web.password_hash` (ruling 4).
|
||||
- `logStartup` (after `logging.install`) logs the authority: `authority: database` / `authority: file (/etc/nxdns/config.zon)`.
|
||||
- In file mode, the reconcile summary (ruling 3's `Summary`) prints through the Runner at startup, matching `seedFromFile`'s existing output discipline: per-table inserted/updated/deleted counts, the changed settings **keys** (never values), and the auth transition when there is one. It is the answer to "what did that restart change" without opening sqlite.
|
||||
- **Authority is never persisted.** The DB carries no record of which mode wrote it: authority lives in the invocation (ruling 1), per-process state (`WebState.authority`, `reconciled_at`) serves the API (ruling 7), and the journal holds the history. A persisted marker was designed twice and deleted twice. A per-boot `reconciled_at` settings row breaks ruling 5 outright — `dump()` iterates the settings table, so any row rewritten per reconcile forfeits byte-identity, and `putSetting` moves `total_changes`, forfeiting zero-writes. The write-on-difference `authority.mode`/`path` variant survived idempotence but cost a reserved key namespace, a `fromSettings` decode filter, and a reconciler sweep exemption — three mechanisms whose only consumer was one `import` warning (ruling 6). State that exists to power a courtesy message is not worth a namespace. The settings sweep's exemption list is therefore exactly one key: `web.password_hash` (ruling 4).
|
||||
|
||||
No `std.log.err` in any new code, per the standing spec rule for new code
|
||||
(the pre-existing calls in db.zig/tls_server.zig are out of scope).
|
||||
No `std.log.err` in any new code, per the standing spec rule for new code (the pre-existing calls in db.zig/tls_server.zig are out of scope).
|
||||
|
||||
### 9. Deployment and migration
|
||||
|
||||
- **systemd**: the shipped unit stays flagless (db default) and gains two
|
||||
lines. `RestartPreventExitStatus=2 64` — a config fault or usage error must
|
||||
not restart-loop every 2 s until StartLimitBurst; the journal holds the
|
||||
diagnostics and the fix is a file edit, not a retry. (Correct in db mode
|
||||
too: exit 2 means the config is wrong in either mode. And it is why
|
||||
ruling 2 keeps exit 2 to *declarative* faults only — a transient box fault
|
||||
mapped to exit 2 would stop the unit permanently.) And
|
||||
`ReadOnlyPaths=/etc/nxdns` — `ConfigurationDirectory=nxdns` makes systemd
|
||||
create the directory owned by the service user, so without this line the
|
||||
source-of-truth file is writable by the very process whose mutation routes
|
||||
file mode exists to disable; nxdns never writes `/etc/nxdns` in either mode,
|
||||
so the base unit ships the enforcement rather than recommending it. Docs
|
||||
show a drop-in for file mode: `ExecStart=` reset plus
|
||||
`--config=/etc/nxdns/config.zon`.
|
||||
- **Ansible / config-management**: the docs' deployment guidance mandates
|
||||
`nxdns check --config=<file>` as the handler precondition — validate
|
||||
the pushed file *before* restarting, so a typo is a failed deploy at noon,
|
||||
not a dead resolver at the next 3am power blip.
|
||||
- **docker**: compose ships file mode as its example —
|
||||
`command: ["run", "--config=/etc/nxdns/config.zon"]` (resolving old open
|
||||
question 4; the `:ro` mount at compose.yaml:15 already suggests it). This keeps the
|
||||
fresh-install and volume-loss stories working after bootstrap dies: a
|
||||
recreated `nxdns-data` volume reconciles from the mounted file on next
|
||||
start, which is *better* than the old seed-once self-heal. The binary's
|
||||
default stays db. The seed-once comment (compose.yaml:9-12) is rewritten.
|
||||
For db-mode-in-docker, the docs show the recovery one-liner
|
||||
(`docker compose run --rm nxdns import /etc/nxdns/config.zon` — the file is
|
||||
positional, `ImportArgs.file` at cli.zig:61, plus `--allow-delete` when the
|
||||
diff deletes) — without it, `restart: unless-stopped` plus exit 2 is an
|
||||
infinite crash loop (docker has no start limit) with no documented way out.
|
||||
- **Adopt file mode** on a UI-configured box: **stop the service first**, then
|
||||
`nxdns export --out /etc/nxdns/config.zon` →
|
||||
`nxdns check --config=/etc/nxdns/config.zon` → add the flag → start.
|
||||
Stop-first is load-bearing twice: any UI edit landing between a live
|
||||
export and the restart would be silently reverted by the first reconcile,
|
||||
and `check` refuses to grade a live database (immutable open, `WalPending`
|
||||
against the steady-state WAL, db.zig:222-226) so the pre-flight gate only
|
||||
works stopped. (Sync note, found in R4: the spec originally claimed
|
||||
`export` itself refuses against a live instance — it does not; `export`
|
||||
opens read/write and succeeds. The claim was corrected to name `check`.)
|
||||
Stopped,
|
||||
the first reconcile's summary is all-zero and writes nothing — blocklist
|
||||
state, compiled files, and client history all survive. Every unchanged boot
|
||||
after it writes nothing either.
|
||||
- **Leave file mode**: drop the flag (remove the drop-in), restart. The DB
|
||||
already holds the last reconciled state; nothing else needed.
|
||||
- **Binary downgrade from file mode**: no unit edit is needed — the old
|
||||
binary accepts `run --config` with seed-once semantics, and against the
|
||||
already-configured DB it ignores the file and serves the last-reconciled
|
||||
state. The rollback note states the consequence: file edits stop applying
|
||||
until the binary is upgraded again.
|
||||
- **Restore from backup**: db mode is stop → restore `config.db` → start,
|
||||
with the WAL caveat db.zig itself documents: take backups only from a
|
||||
stopped instance (or use `export`), and on restore delete any stale
|
||||
`config.db-wal`/`config.db-shm` beside the target — a mismatched WAL is
|
||||
silently discarded by SQLite, which turns "restore" into "lose the tail".
|
||||
Restoring an *exported file* into a populated DB via `import` is exactly
|
||||
the deleting case: back-up-and-restore.md documents `--allow-delete` there
|
||||
(ruling 6) — the flag choreography is rewritten, not removed. In file mode
|
||||
`config.db` is not the config backup — the file is; restore = redeploy the
|
||||
file. The query-log DB restores independently in both modes.
|
||||
- **systemd**: the shipped unit stays flagless (db default) and gains two lines. `RestartPreventExitStatus=2 64` — a config fault or usage error must not restart-loop every 2 s until StartLimitBurst; the journal holds the diagnostics and the fix is a file edit, not a retry. (Correct in db mode too: exit 2 means the config is wrong in either mode. And it is why ruling 2 keeps exit 2 to *declarative* faults only — a transient box fault mapped to exit 2 would stop the unit permanently.) And `ReadOnlyPaths=/etc/nxdns` — `ConfigurationDirectory=nxdns` makes systemd create the directory owned by the service user, so without this line the source-of-truth file is writable by the very process whose mutation routes file mode exists to disable; nxdns never writes `/etc/nxdns` in either mode, so the base unit ships the enforcement rather than recommending it. Docs show a drop-in for file mode: `ExecStart=` reset plus `--config=/etc/nxdns/config.zon`.
|
||||
- **Ansible / config-management**: the docs' deployment guidance mandates `nxdns check --config=<file>` as the handler precondition — validate the pushed file *before* restarting, so a typo is a failed deploy at noon, not a dead resolver at the next 3am power blip.
|
||||
- **docker**: compose ships file mode as its example — `command: ["run", "--config=/etc/nxdns/config.zon"]` (resolving old open question 4; the `:ro` mount at compose.yaml:15 already suggests it). This keeps the fresh-install and volume-loss stories working after bootstrap dies: a recreated `nxdns-data` volume reconciles from the mounted file on next start, which is *better* than the old seed-once self-heal. The binary's default stays db. The seed-once comment (compose.yaml:9-12) is rewritten. For db-mode-in-docker, the docs show the recovery one-liner (`docker compose run --rm nxdns import /etc/nxdns/config.zon` — the file is positional, `ImportArgs.file` at cli.zig:61, plus `--allow-delete` when the diff deletes) — without it, `restart: unless-stopped` plus exit 2 is an infinite crash loop (docker has no start limit) with no documented way out.
|
||||
- **Adopt file mode** on a UI-configured box: **stop the service first**, then `nxdns export --out /etc/nxdns/config.zon` → `nxdns check --config=/etc/nxdns/config.zon` → add the flag → start. Stop-first is load-bearing twice: any UI edit landing between a live export and the restart would be silently reverted by the first reconcile, and `check` refuses to grade a live database (immutable open, `WalPending` against the steady-state WAL, db.zig:222-226) so the pre-flight gate only works stopped. (Sync note, found in R4: the spec originally claimed `export` itself refuses against a live instance — it does not; `export` opens read/write and succeeds. The claim was corrected to name `check`.) Stopped, the first reconcile's summary is all-zero and writes nothing — blocklist state, compiled files, and client history all survive. Every unchanged boot after it writes nothing either.
|
||||
- **Leave file mode**: drop the flag (remove the drop-in), restart. The DB already holds the last reconciled state; nothing else needed.
|
||||
- **Binary downgrade from file mode**: no unit edit is needed — the old binary accepts `run --config` with seed-once semantics, and against the already-configured DB it ignores the file and serves the last-reconciled state. The rollback note states the consequence: file edits stop applying until the binary is upgraded again.
|
||||
- **Restore from backup**: db mode is stop → restore `config.db` → start, with the WAL caveat db.zig itself documents: take backups only from a stopped instance (or use `export`), and on restore delete any stale `config.db-wal`/`config.db-shm` beside the target — a mismatched WAL is silently discarded by SQLite, which turns "restore" into "lose the tail". Restoring an *exported file* into a populated DB via `import` is exactly the deleting case: back-up-and-restore.md documents `--allow-delete` there (ruling 6) — the flag choreography is rewritten, not removed. In file mode `config.db` is not the config backup — the file is; restore = redeploy the file. The query-log DB restores independently in both modes.
|
||||
|
||||
Migration honesty, replacing the earlier blanket claim: a db-mode install
|
||||
that never passed `--config` needs nothing. Anyone whose unit, wrapper, or
|
||||
compose `command` carries `run --config` (the documented seed-once invocation
|
||||
in first-run.md and three how-to labs) gets file authority at the first
|
||||
post-upgrade start — the seed file becomes the config, and UI edits made
|
||||
since seeding are deleted by the first reconcile. The upgrade doc's
|
||||
breaking-changes section leads with this and gives the two exits: drop the
|
||||
flag to keep the DB, or re-export to the file path first to adopt file mode
|
||||
cleanly (the walkthrough above). `check --config` keeps its meaning. Docker
|
||||
db-mode fresh installs must use the new compose or run `import` once. No
|
||||
schema migration.
|
||||
Migration honesty, replacing the earlier blanket claim: a db-mode install that never passed `--config` needs nothing. Anyone whose unit, wrapper, or compose `command` carries `run --config` (the documented seed-once invocation in first-run.md and three how-to labs) gets file authority at the first post-upgrade start — the seed file becomes the config, and UI edits made since seeding are deleted by the first reconcile. The upgrade doc's breaking-changes section leads with this and gives the two exits: drop the flag to keep the DB, or re-export to the file path first to adopt file mode cleanly (the walkthrough above). `check --config` keeps its meaning. Docker db-mode fresh installs must use the new compose or run `import` once. No schema migration.
|
||||
|
||||
### 10. Documentation is part of the change
|
||||
|
||||
`explanation/configuration-model.md` is rewritten around the two-mode model
|
||||
(its "why the database wins" argument becomes mode-scoped, not superseded).
|
||||
Seed-once claims are corrected in: the first-run tutorial, install-with-systemd,
|
||||
install-with-docker, upgrade (breaking-changes + rollback sections,
|
||||
ruling 9), back-up-and-restore (new restore semantics, WAL sidecars,
|
||||
`--allow-delete`), set-up-admin-authentication (the ruling 4
|
||||
absence/empty-string rule), troubleshoot, cli.md
|
||||
(`run`/`check`/`import`/`export`, `--allow-delete`), configuration.md
|
||||
(optional password fields), files-and-directories.md, api.md (settings
|
||||
envelope, 403, per-route rejectability), the compose.yaml comment,
|
||||
README/INSTALL.
|
||||
`explanation/configuration-model.md` is rewritten around the two-mode model (its "why the database wins" argument becomes mode-scoped, not superseded). Seed-once claims are corrected in: the first-run tutorial, install-with-systemd, install-with-docker, upgrade (breaking-changes + rollback sections, ruling 9), back-up-and-restore (new restore semantics, WAL sidecars, `--allow-delete`), set-up-admin-authentication (the ruling 4 absence/empty-string rule), troubleshoot, cli.md (`run`/`check`/`import`/`export`, `--allow-delete`), configuration.md (optional password fields), files-and-directories.md, api.md (settings envelope, 403, per-route rejectability), the compose.yaml comment, README/INSTALL.
|
||||
|
||||
## Deletions (complete list)
|
||||
|
||||
`config/bootstrap.zig` (+2 tests, S7 cases 15-18, + app.zig import/call sites
|
||||
:35/:142/:146 and tests :1124/:1237) · `import.isEmpty` (+6 tests, + the
|
||||
storage_integration_test.zig call sites :763/:795/:894, + the clients_repo.zig
|
||||
doc comments :6/:129) · `config_schema.content_tables` (+its test; `dump()`
|
||||
re-pointed to the new `table_names` list) · `app.seedFromFile` (+its app.zig
|
||||
tests) · `CheckArgs.config_explicit` · the `checkImpl` source heuristic ·
|
||||
`applyToDb`'s wipe loop, including the `saved_clients` lift/merge/restore
|
||||
scaffolding (import.zig:268-361 — superseded by ruling 3's
|
||||
promotion-in-place) · `toSettings`'s `web.password_hash` emission (+its
|
||||
key-list test row; the model.zig:665-670 "skipped in both directions" test is
|
||||
rewritten for the encode/decode split) · `http_util.respondPlain`'s
|
||||
silent-downgrade path (`respondError` rebuilt on the request arena).
|
||||
storage_integration_test.zig call sites :763/:795/:894, + the clients_repo.zig doc comments :6/:129) · `config_schema.content_tables` (+its test; `dump()` re-pointed to the new `table_names` list) · `app.seedFromFile` (+its app.zig tests) · `CheckArgs.config_explicit` · the `checkImpl` source heuristic · `applyToDb`'s wipe loop, including the `saved_clients` lift/merge/restore scaffolding (import.zig:268-361 — superseded by ruling 3's promotion-in-place) · `toSettings`'s `web.password_hash` emission (+its key-list test row; the model.zig:665-670 "skipped in both directions" test is rewritten for the encode/decode split) · `http_util.respondPlain`'s silent-downgrade path (`respondError` rebuilt on the request arena).
|
||||
|
||||
Renamed, not deleted (ruling 6): `Options.force` → `Options.allow_delete`
|
||||
(`--force` → `--allow-delete`), `error.DatabaseNotEmpty` →
|
||||
`error.DestructiveImport` (cli.zig:533 arm retargeted, faults.zig exclusion
|
||||
and tests follow the name).
|
||||
Renamed, not deleted (ruling 6): `Options.force` → `Options.allow_delete` (`--force` → `--allow-delete`), `error.DatabaseNotEmpty` → `error.DestructiveImport` (cli.zig:533 arm retargeted, faults.zig exclusion and tests follow the name).
|
||||
|
||||
## Sessions
|
||||
|
||||
R1 lands first and R2 rewires onto it and carries every deletion — they are
|
||||
ordered, not parallel (the old plan had R1 deleting `bootstrap.zig` out from
|
||||
under R2-owned app.zig call sites). R1 is **additive engine plus one
|
||||
coordinated model change**, not purely additive: the `model.Web` optional
|
||||
fields force same-session edits at every site that reads them, or the tree
|
||||
stops building. R3 and R4 run parallel to both; their interfaces
|
||||
(`reconcile.Summary`, `WebState.authority` + `reconciled_at`,
|
||||
`RouteInfo.policy`) are fixed here.
|
||||
R1 lands first and R2 rewires onto it and carries every deletion — they are ordered, not parallel (the old plan had R1 deleting `bootstrap.zig` out from under R2-owned app.zig call sites). R1 is **additive engine plus one coordinated model change**, not purely additive: the `model.Web` optional fields force same-session edits at every site that reads them, or the tree stops building. R3 and R4 run parallel to both; their interfaces (`reconcile.Summary`, `WebState.authority` + `reconciled_at`, `RouteInfo.policy`) are fixed here.
|
||||
|
||||
### Session R1: reconcile engine + the coordinated model change
|
||||
|
||||
Owns `src/config/reconcile.zig` (engine + `Summary` as specified), the new
|
||||
repo verbs in `src/storage/repositories/`, the `needsRefresh` clock clamp in
|
||||
`src/filter/manager.zig`, registration in `src/tests.zig`. Rulings 3, 4, 5.
|
||||
Owns `src/config/reconcile.zig` (engine + `Summary` as specified), the new repo verbs in `src/storage/repositories/`, the `needsRefresh` clock clamp in `src/filter/manager.zig`, registration in `src/tests.zig`. Rulings 3, 4, 5.
|
||||
|
||||
Owns `src/config/model.zig` whole: the optional `Web.password`/`password_hash`
|
||||
fields, the `isEncodeSkipped`/`isDecodeSkipped` split, `decodeValue`'s
|
||||
`.optional` arm, and the bridge tests. And the sites the
|
||||
optional fields break, which no session previously owned: `validate.zig`
|
||||
(:395 both-set port, the new empty-password rejection, test :1966-1968),
|
||||
`auth.zig` (`authEnabled` orelse), `export.zig` (canonical `password = null`,
|
||||
comment and header sample), `src/web/handlers/settings.zig` (`newPassword`'s
|
||||
`orelse` chain at :184-186, and `FieldType` collapsing `?T` so `Partial`'s
|
||||
`?FieldType(...)` at :114 does not generate a double optional — landing this
|
||||
in R1 keeps R3's `web/` work unblocked). R1 also makes the one-line
|
||||
`orelse ""` edit at app.zig:475 so the tree builds; app.zig otherwise stays
|
||||
R2's, and R2 absorbs that line into its startup rewire.
|
||||
Owns `src/config/model.zig` whole: the optional `Web.password`/`password_hash` fields, the `isEncodeSkipped`/`isDecodeSkipped` split, `decodeValue`'s `.optional` arm, and the bridge tests. And the sites the optional fields break, which no session previously owned: `validate.zig` (:395 both-set port, the new empty-password rejection, test :1966-1968), `auth.zig` (`authEnabled` orelse), `export.zig` (canonical `password = null`, comment and header sample), `src/web/handlers/settings.zig` (`newPassword`'s `orelse` chain at :184-186, and `FieldType` collapsing `?T` so `Partial`'s `?FieldType(...)` at :114 does not generate a double optional — landing this in R1 keeps R3's `web/` work unblocked). R1 also makes the one-line `orelse ""` edit at app.zig:475 so the tree builds; app.zig otherwise stays R2's, and R2 absorbs that line into its startup rewire.
|
||||
|
||||
### Session R2: CLI + startup + deletions
|
||||
|
||||
Owns `src/cli.zig`, `src/app.zig`, `src/config/faults.zig`,
|
||||
`src/config/import.zig` (thin-wrapper rewrite, diff-gate, `--allow-delete`),
|
||||
the shared loader-fault mapping helper (ruling 2, used by `run` and `check`),
|
||||
and the entire Deletions list plus the renames. Rulings 1, 2, 6, 8. Threads authority and
|
||||
`reconciled_at` into `WebState` (arena-owned path) but does not touch the
|
||||
router. Owns the cli.zig hint-line renderer arm.
|
||||
Owns `src/cli.zig`, `src/app.zig`, `src/config/faults.zig`, `src/config/import.zig` (thin-wrapper rewrite, diff-gate, `--allow-delete`), the shared loader-fault mapping helper (ruling 2, used by `run` and `check`), and the entire Deletions list plus the renames. Rulings 1, 2, 6, 8. Threads authority and `reconciled_at` into `WebState` (arena-owned path) but does not touch the router. Owns the cli.zig hint-line renderer arm.
|
||||
|
||||
### Session R3: web enforcement + UI
|
||||
|
||||
Owns `src/web/router.zig` (dispatch policy step, `test_table` gains the
|
||||
policy column), `src/web/routes.zig` (all 56 entries state `policy`
|
||||
explicitly — no default), `src/web/http_util.zig` (`respondError` arena
|
||||
rewrite, `respondPlain` downgrade deletion), `src/web/openapi.yaml`, the
|
||||
client-DELETE observed/declared branch in the clients handler, `web/`
|
||||
frontend (banner, disabled controls, settings envelope with nullable
|
||||
`reconciled_at`). Ruling 7. Consumes `WebState.authority`/`reconciled_at` as
|
||||
fixed above.
|
||||
Owns `src/web/router.zig` (dispatch policy step, `test_table` gains the policy column), `src/web/routes.zig` (all 56 entries state `policy` explicitly — no default), `src/web/http_util.zig` (`respondError` arena rewrite, `respondPlain` downgrade deletion), `src/web/openapi.yaml`, the client-DELETE observed/declared branch in the clients handler, `web/` frontend (banner, disabled controls, settings envelope with nullable `reconciled_at`). Ruling 7. Consumes `WebState.authority`/`reconciled_at` as fixed above.
|
||||
|
||||
### Session R4: deployment + docs
|
||||
|
||||
Owns `deploy/**`, `docs/**`, `README.md`, `INSTALL`, compose file + comment,
|
||||
the systemd unit's two new lines. Rulings 9, 10.
|
||||
Owns `deploy/**`, `docs/**`, `README.md`, `INSTALL`, compose file + comment, the systemd unit's two new lines. Rulings 9, 10.
|
||||
|
||||
### Orchestrator
|
||||
|
||||
This spec, R1→R2 sequencing, cross-session integration, the drift-guard
|
||||
regenerations that span sessions (contract samples, openapi route counts,
|
||||
api.md/cli.md rows), and the `-Dlive` acceptance run.
|
||||
This spec, R1→R2 sequencing, cross-session integration, the drift-guard regenerations that span sessions (contract samples, openapi route counts, api.md/cli.md rows), and the `-Dlive` acceptance run.
|
||||
|
||||
## Tests
|
||||
|
||||
- **Reconcile unit tests** (R1, `:memory:` + migrate): idempotence via the
|
||||
rewritten `dump()` byte-stability across all tables after applying the same
|
||||
file twice — including a file carrying a plaintext password **and a
|
||||
non-canonical but equivalent file** (`FD00::1`-style) — plus the zero-writes
|
||||
assertion (all-zero `Summary`, `total_changes` unmoved) on the second pass;
|
||||
per-table preservation via `sources_repo.SourceRow` after seeding stats with
|
||||
`updateSourceStats`; URL change ⇒ new id; source removal; observed-client
|
||||
survival, **promote-on-declare (asserting `first_seen`/`last_seen` survive
|
||||
the promotion and the row id is stable)**, and **reassign-to-default when
|
||||
their group is removed or renamed** (the un-cascaded FK case);
|
||||
`safe_search` edit on an existing group converges; rules `created_at`
|
||||
stability including duplicate tuples; password verify-keeps-hash,
|
||||
mismatch-rehashes, absent-keeps-stored, explicit-empty-disables; rollback
|
||||
on mid-tx failure; default-group id 1 pin; `needsRefresh`
|
||||
future-`last_updated` clamp.
|
||||
- **Model/bridge tests** (R1, model.zig test neighbourhood at :659):
|
||||
`decodeValue` on an optional field (absent ⇒
|
||||
null, present ⇒ value); `web.password_hash` decodes from settings but is
|
||||
not encoded (the encode/decode split); `validate` rejects present-and-empty
|
||||
`password` with the `password_hash = ""` remedy in the diagnostic; export ⇒
|
||||
import round trip with the canonical `password = null` form.
|
||||
- **Import gate tests** (R2): a file whose diff deletes rows, without
|
||||
`--allow-delete` ⇒ rollback, exit 2, per-table delete counts in the
|
||||
message; with the flag ⇒ applied; additive/edit-only import needs no flag.
|
||||
- **Loader-fault mapping** (R2, on the shared helper directly): each
|
||||
path-class open error ⇒ `ManagedConfigUnreadable`; a non-path member of the
|
||||
set propagates unmapped.
|
||||
- **Restart-no-redownload** in filter_integration_test.zig: reconcile, then a
|
||||
`Manager` restart reuses `<id>.list`/`<id>.wild` with no refetch (fixtures
|
||||
already assert reuse by id) — this is also the behavioral enforcement of
|
||||
ruling 3's ordering invariant. Plus **one `-Dlive` run of the real download
|
||||
path** — hermetic suites have hidden a process-killing bug on the real
|
||||
network path before (MEMORY), so the acceptance includes the real path once.
|
||||
- **Router**: policy classification unit tests via `test_table`/`matchPath`,
|
||||
no sockets; the contract table in web_integration_test.zig gains a `policy`
|
||||
column and the existing 1:1 coverage assertion widens so classification
|
||||
cannot drift; one socketed test per class in file mode (config write → 403,
|
||||
runtime action → 2xx, read → 200) plus observed-vs-declared client DELETE,
|
||||
via `EnvOptions` authority; `respondError` with a message longer than the
|
||||
old 512-byte buffer stays `application/json`. Route count stays 56 — no new
|
||||
routes.
|
||||
- **CLI**: `parseArgs` tests: `run` without `--config` selects db authority;
|
||||
`run --config=<path>` selects file authority and the path lands in the run
|
||||
args; `--allow-delete` parsing; `usage_text` test; exit-2 for a missing
|
||||
managed file and for bare `check` with no DB (hint line present) — via the
|
||||
`Captured` runner (in-process; remember the buffered-writer caveat — flush
|
||||
through a real `File.Writer` where the test asserts delivery).
|
||||
- **Drift guards knowingly tripped and regenerated**: openapi.yaml (settings
|
||||
envelope, 403 responses), golden contract samples (`-Dcontract-samples-out`),
|
||||
api.md rows, cli.md sections, docs_drift_test, `toSettings` key-list test.
|
||||
- **Reconcile unit tests** (R1, `:memory:` + migrate): idempotence via the rewritten `dump()` byte-stability across all tables after applying the same file twice — including a file carrying a plaintext password **and a non-canonical but equivalent file** (`FD00::1`-style) — plus the zero-writes assertion (all-zero `Summary`, `total_changes` unmoved) on the second pass; per-table preservation via `sources_repo.SourceRow` after seeding stats with `updateSourceStats`; URL change ⇒ new id; source removal; observed-client survival, **promote-on-declare (asserting `first_seen`/`last_seen` survive the promotion and the row id is stable)**, and **reassign-to-default when their group is removed or renamed** (the un-cascaded FK case); `safe_search` edit on an existing group converges; rules `created_at` stability including duplicate tuples; password verify-keeps-hash, mismatch-rehashes, absent-keeps-stored, explicit-empty-disables; rollback on mid-tx failure; default-group id 1 pin; `needsRefresh` future-`last_updated` clamp.
|
||||
- **Model/bridge tests** (R1, model.zig test neighbourhood at :659): `decodeValue` on an optional field (absent ⇒ null, present ⇒ value); `web.password_hash` decodes from settings but is not encoded (the encode/decode split); `validate` rejects present-and-empty `password` with the `password_hash = ""` remedy in the diagnostic; export ⇒ import round trip with the canonical `password = null` form.
|
||||
- **Import gate tests** (R2): a file whose diff deletes rows, without `--allow-delete` ⇒ rollback, exit 2, per-table delete counts in the message; with the flag ⇒ applied; additive/edit-only import needs no flag.
|
||||
- **Loader-fault mapping** (R2, on the shared helper directly): each path-class open error ⇒ `ManagedConfigUnreadable`; a non-path member of the set propagates unmapped.
|
||||
- **Restart-no-redownload** in filter_integration_test.zig: reconcile, then a `Manager` restart reuses `<id>.list`/`<id>.wild` with no refetch (fixtures already assert reuse by id) — this is also the behavioral enforcement of ruling 3's ordering invariant. Plus **one `-Dlive` run of the real download path** — hermetic suites have hidden a process-killing bug on the real network path before (MEMORY), so the acceptance includes the real path once.
|
||||
- **Router**: policy classification unit tests via `test_table`/`matchPath`, no sockets; the contract table in web_integration_test.zig gains a `policy` column and the existing 1:1 coverage assertion widens so classification cannot drift; one socketed test per class in file mode (config write → 403, runtime action → 2xx, read → 200) plus observed-vs-declared client DELETE, via `EnvOptions` authority; `respondError` with a message longer than the old 512-byte buffer stays `application/json`. Route count stays 56 — no new routes.
|
||||
- **CLI**: `parseArgs` tests: `run` without `--config` selects db authority; `run --config=<path>` selects file authority and the path lands in the run args; `--allow-delete` parsing; `usage_text` test; exit-2 for a missing managed file and for bare `check` with no DB (hint line present) — via the `Captured` runner (in-process; remember the buffered-writer caveat — flush through a real `File.Writer` where the test asserts delivery).
|
||||
- **Drift guards knowingly tripped and regenerated**: openapi.yaml (settings envelope, 403 responses), golden contract samples (`-Dcontract-samples-out`), api.md rows, cli.md sections, docs_drift_test, `toSettings` key-list test.
|
||||
|
||||
## Acceptance (design complete when implemented)
|
||||
|
||||
@@ -854,68 +316,31 @@ api.md/cli.md rows), and the `-Dlive` acceptance run.
|
||||
## Anti-requirements
|
||||
|
||||
- No file watcher, no inotify, no SIGHUP reload — restart is the reload.
|
||||
- No UI write-back to the file, no partial/merge authority, no per-table
|
||||
hybrid modes, no multi-file config.
|
||||
- No persisted authority state, no config hashing, no content-based change
|
||||
detection — authority lives in the invocation; last-load time is
|
||||
per-process state in the settings envelope, honest about what it can and
|
||||
cannot answer (ruling 7).
|
||||
- No deferred deletion of a replaced source's compiled artifacts — the URL-edit
|
||||
gap is an accepted trade (ruling 3), bounded by the scheduler retry.
|
||||
- No UI write-back to the file, no partial/merge authority, no per-table hybrid modes, no multi-file config.
|
||||
- No persisted authority state, no config hashing, no content-based change detection — authority lives in the invocation; last-load time is per-process state in the settings envelope, honest about what it can and cannot answer (ruling 7).
|
||||
- No deferred deletion of a replaced source's compiled artifacts — the URL-edit gap is an accepted trade (ruling 3), bounded by the scheduler retry.
|
||||
- No etag/conditional GET for blocklist fetches.
|
||||
- No `code` field or richer error envelope — the status is the machine
|
||||
contract.
|
||||
- No `code` field or richer error envelope — the status is the machine contract.
|
||||
- No rule-identity schema change (no synthetic rule key column).
|
||||
- No preservation of rule `created_at` across pattern edits — an edited rule
|
||||
is a new rule.
|
||||
- No CLI-side blocking or detection of `import` against a file-mode box's
|
||||
DB — the diff-gate guards deletion; the next boot converges and its
|
||||
summary reports what it corrected.
|
||||
- No preservation of rule `created_at` across pattern edits — an edited rule is a new rule.
|
||||
- No CLI-side blocking or detection of `import` against a file-mode box's DB — the diff-gate guards deletion; the next boot converges and its summary reports what it corrected.
|
||||
- No fallback from file mode to db mode under any failure.
|
||||
|
||||
## Resolved defaults (were open questions; the user may overrule before R3/R4)
|
||||
|
||||
1. **UI naming and regrouping of observed (`hand_edited=0`) clients in file
|
||||
mode: rejected as declarative drift** (client PUT stays `config_write`,
|
||||
ruling 7). The household user adds the client to the file instead. The
|
||||
alternative — carving naming/grouping out as a runtime action —
|
||||
reintroduces two-way merge for one table, which the anti-requirements
|
||||
refuse. Affects R3.
|
||||
2. **`export` output is byte-identical in both modes** — no file-mode
|
||||
annotation. An annotation would make export → file → adopt produce a
|
||||
different file than the one checked, for a label the operator already has
|
||||
in the unit file. Affects R4's round-trip docs.
|
||||
3. **File mode ships as a documented systemd drop-in**, not a second
|
||||
commented `ExecStart` in the packaged unit. The packaged unit stays
|
||||
flagless and correct by itself; a commented alternative line in a unit
|
||||
file is a doc pretending to be config. Affects R4.
|
||||
1. **UI naming and regrouping of observed (`hand_edited=0`) clients in file mode: rejected as declarative drift** (client PUT stays `config_write`, ruling 7). The household user adds the client to the file instead. The alternative — carving naming/grouping out as a runtime action — reintroduces two-way merge for one table, which the anti-requirements refuse. Affects R3.
|
||||
2. **`export` output is byte-identical in both modes** — no file-mode annotation. An annotation would make export → file → adopt produce a different file than the one checked, for a label the operator already has in the unit file. Affects R4's round-trip docs.
|
||||
3. **File mode ships as a documented systemd drop-in**, not a second commented `ExecStart` in the packaged unit. The packaged unit stays flagless and correct by itself; a commented alternative line in a unit file is a doc pretending to be config. Affects R4.
|
||||
|
||||
## Cross-validation findings rejected
|
||||
|
||||
Round 1: none — all twelve findings (F1-F12) were confirmed against the repo
|
||||
and are folded into the rulings above. The F1 remedy has since been
|
||||
superseded: the premise revision deleted the persisted marker entirely
|
||||
(ruling 8) instead of keeping it write-on-difference.
|
||||
Round 1: none — all twelve findings (F1-F12) were confirmed against the repo and are folded into the rulings above. The F1 remedy has since been superseded: the premise revision deleted the persisted marker entirely (ruling 8) instead of keeping it write-on-difference.
|
||||
|
||||
Round 2 (on the premise revision) returned three important findings and one
|
||||
minor. Folded: the docker recovery one-liner used a `--config` flag `import`
|
||||
does not have (fixed in ruling 9, file is positional); `import` is now
|
||||
stated everywhere as a stop-first operation and the file-authority contract
|
||||
as "converged at every boot" (ruling 6); identity-column edits are named as
|
||||
delete-plus-insert needing `--allow-delete` (ruling 6). Declined: renaming
|
||||
`run --config` to `--managed-config` — reasons recorded in ruling 1.
|
||||
Round 2 (on the premise revision) returned three important findings and one minor. Folded: the docker recovery one-liner used a `--config` flag `import` does not have (fixed in ruling 9, file is positional); `import` is now stated everywhere as a stop-first operation and the file-authority contract as "converged at every boot" (ruling 6); identity-column edits are named as delete-plus-insert needing `--allow-delete` (ruling 6). Declined: renaming `run --config` to `--managed-config` — reasons recorded in ruling 1.
|
||||
|
||||
## Red-team findings rejected
|
||||
|
||||
None rejected outright — every finding checked out against the repo. Two
|
||||
proposed remedies were declined while their findings were accepted:
|
||||
None rejected outright — every finding checked out against the repo. Two proposed remedies were declined while their findings were accepted:
|
||||
|
||||
- **ops 10's mitigation** (keep the old compiled artifact until the successor
|
||||
URL's first successful fetch): declined — a cross-artifact lifecycle for a
|
||||
rare, operator-initiated event on a household box; the trade is now stated
|
||||
in ruling 3 and the anti-requirements instead.
|
||||
- **ops 14's db-mode warning** ("a config file exists but authority is
|
||||
database"): declined — db mode is given no file path, so warning would mean
|
||||
probing a well-known location, which is precisely the ambient inference
|
||||
ruling 1 bans; the accepted half (drift visibility) is served by the
|
||||
settings envelope's `authority` block and per-process `reconciled_at`.
|
||||
- **ops 10's mitigation** (keep the old compiled artifact until the successor URL's first successful fetch): declined — a cross-artifact lifecycle for a rare, operator-initiated event on a household box; the trade is now stated in ruling 3 and the anti-requirements instead.
|
||||
- **ops 14's db-mode warning** ("a config file exists but authority is database"): declined — db mode is given no file path, so warning would mean probing a well-known location, which is precisely the ambient inference ruling 1 bans; the accepted half (drift visibility) is served by the settings envelope's `authority` block and per-process `reconciled_at`.
|
||||
|
||||
+76
-475
@@ -1,129 +1,49 @@
|
||||
# Milestone 21: ABP exceptions from lists, regex rules for operators
|
||||
|
||||
Goal: honor `@@||domain^` exception lines in downloaded blocklists as allow
|
||||
entries scoped below operator rules and above list blocks (tier 1), and add a
|
||||
`regex` rule kind for operator rules backed by a homegrown linear-time engine
|
||||
(tier 2). Nothing else from the ABP syntax enters scope.
|
||||
Goal: honor `@@||domain^` exception lines in downloaded blocklists as allow entries scoped below operator rules and above list blocks (tier 1), and add a `regex` rule kind for operator rules backed by a homegrown linear-time engine (tier 2). Nothing else from the ABP syntax enters scope.
|
||||
|
||||
Design written 2026-08-09 against HEAD `ffc3ca6`; every anchor re-verified
|
||||
2026-08-11 against `a8e0fe4` after milestone 20 landed. The milestone amends
|
||||
PLAN §2.2, which currently rules regex out permanently; the amendment is part
|
||||
of session S3, not a side effect.
|
||||
Design written 2026-08-09 against HEAD `ffc3ca6`; every anchor re-verified 2026-08-11 against `a8e0fe4` after milestone 20 landed. The milestone amends PLAN §2.2, which currently rules regex out permanently; the amendment is part of session S3, not a side effect.
|
||||
|
||||
## Implementation contract (read first)
|
||||
|
||||
- Read `AGENTS.md`, then this spec whole, before session work starts.
|
||||
- Pure core stays pure: `src/filter/` files that are fuzz-module roots import
|
||||
only `std` (`src/filter/parsers.zig:1-14`). The new `src/filter/regex.zig`
|
||||
obeys the same constraint.
|
||||
- Every new `src/**.zig` file must be listed in `src/tests.zig`
|
||||
(`build.zig:494-539` fatals otherwise).
|
||||
- Frozen DDL is frozen: schema changes are new migration steps
|
||||
(`src/storage/config_schema.zig:1-6`, `src/storage/migrations.zig:21-22`).
|
||||
- After any API shape change, regenerate the contract samples
|
||||
(`web/src/lib/contractSamples.gen.ts`; procedure in AGENTS.md).
|
||||
- Pure core stays pure: `src/filter/` files that are fuzz-module roots import only `std` (`src/filter/parsers.zig:1-14`). The new `src/filter/regex.zig` obeys the same constraint.
|
||||
- Every new `src/**.zig` file must be listed in `src/tests.zig` (`build.zig:494-539` fatals otherwise).
|
||||
- Frozen DDL is frozen: schema changes are new migration steps (`src/storage/config_schema.zig:1-6`, `src/storage/migrations.zig:21-22`).
|
||||
- After any API shape change, regenerate the contract samples (`web/src/lib/contractSamples.gen.ts`; procedure in AGENTS.md).
|
||||
|
||||
## Rulings (binding)
|
||||
|
||||
### 1. Exception lines are `@@||name^` and nothing else, plus one modifier
|
||||
|
||||
`src/filter/parser_abp.zig:22` currently maps every `@@` line to
|
||||
`.unsupported`. After this milestone, a line is an exception when it is
|
||||
`@@||name^` or `@@||name` (same trailing-`^` and `rule_tokens` treatment as the
|
||||
block anchor at parser_abp.zig:26-34), optionally suffixed with the literal
|
||||
`$important` — that suffix is the common form in AdGuard-authored lists and
|
||||
changes nothing about the meaning here, because list exceptions already sit
|
||||
below every operator rule. Any other `@@` form (`@@name` without the anchor,
|
||||
any other `$` modifier, a path, a scheme) stays `.unsupported`. The
|
||||
parser-header policy paragraph (parser_abp.zig:5-6) is rewritten to state the
|
||||
new rule and its precedence justification: a list exception can cancel only
|
||||
list blocks, never an operator decision, so no downloaded list can open an
|
||||
allow hole the operator did not open.
|
||||
`src/filter/parser_abp.zig:22` currently maps every `@@` line to `.unsupported`. After this milestone, a line is an exception when it is `@@||name^` or `@@||name` (same trailing-`^` and `rule_tokens` treatment as the block anchor at parser_abp.zig:26-34), optionally suffixed with the literal `$important` — that suffix is the common form in AdGuard-authored lists and changes nothing about the meaning here, because list exceptions already sit below every operator rule. Any other `@@` form (`@@name` without the anchor, any other `$` modifier, a path, a scheme) stays `.unsupported`. The parser-header policy paragraph (parser_abp.zig:5-6) is rewritten to state the new rule and its precedence justification: a list exception can cancel only list blocks, never an operator decision, so no downloaded list can open an allow hole the operator did not open.
|
||||
|
||||
### 2. Precedence: list exceptions sit between operator rules and list blocks
|
||||
|
||||
`matcher.Snapshot.evaluate` (`src/filter/matcher.zig:286-338`) gains one level
|
||||
between the operator wildcard-block walk (level 4) and the blocklist domain
|
||||
probe (level 5): for each attached source, an exception match — full name or
|
||||
parent walk, apex covered — returns `blocked = false`, reason
|
||||
`.blocklist_exception`, `matched` = the matching entry, `source` = the source
|
||||
index. The doc comment at matcher.zig:269-285 and PLAN §3.10 (PLAN.md:108-117)
|
||||
are both updated with the new level. Regex rules (ruling 6) slot in as levels
|
||||
after the wildcard rules and before list exceptions, allow before block, so
|
||||
the full order is: exact allow, exact block, wildcard allow, wildcard block,
|
||||
regex allow, regex block, list exception, list domain, list wildcard.
|
||||
"Tie-break at same specificity: allow wins" is preserved.
|
||||
`matcher.Snapshot.evaluate` (`src/filter/matcher.zig:286-338`) gains one level between the operator wildcard-block walk (level 4) and the blocklist domain probe (level 5): for each attached source, an exception match — full name or parent walk, apex covered — returns `blocked = false`, reason `.blocklist_exception`, `matched` = the matching entry, `source` = the source index. The doc comment at matcher.zig:269-285 and PLAN §3.10 (PLAN.md:108-117) are both updated with the new level. Regex rules (ruling 6) slot in as levels after the wildcard rules and before list exceptions, allow before block, so the full order is: exact allow, exact block, wildcard allow, wildcard block, regex allow, regex block, list exception, list domain, list wildcard. "Tie-break at same specificity: allow wins" is preserved.
|
||||
|
||||
### 3. The compiled-source format grows a third body, checksum-compatibly
|
||||
|
||||
`compiler.compile` (`src/filter/compiler.zig:50-56`) takes a third writer
|
||||
(`allow_w`) and `Counts` (compiler.zig:23-35) gains `exceptions: u32 = 0`.
|
||||
Exception candidates go through the existing `addCandidate` path into a third
|
||||
`Entries` and emit as a sorted, deduplicated `.allow` body. The shared SHA-256
|
||||
covers the bodies in order list, wild, allow — because SHA-256 of
|
||||
`list ++ wild ++ ""` equals the current SHA-256 of `list ++ wild`, every
|
||||
already-published checksum stays valid, and `Manager.loadSource`
|
||||
(`src/filter/manager.zig:543-598`) treats a missing `<id>.allow` file as an
|
||||
empty body. No refetch is forced by upgrading. `bodyChecksum`
|
||||
(manager.zig:1572) follows the same order; `source_file_suffixes`
|
||||
(manager.zig:1589) gains `.allow.tmp` and `.allow` with longest-suffix-first
|
||||
order preserved; the on-disk header (manager.zig:221-241) gains
|
||||
`# exceptions {d}` after the `# wildcards` line and the pinning test at
|
||||
manager.zig:1914-1946 is extended, not weakened.
|
||||
`compiler.compile` (`src/filter/compiler.zig:50-56`) takes a third writer (`allow_w`) and `Counts` (compiler.zig:23-35) gains `exceptions: u32 = 0`. Exception candidates go through the existing `addCandidate` path into a third `Entries` and emit as a sorted, deduplicated `.allow` body. The shared SHA-256 covers the bodies in order list, wild, allow — because SHA-256 of `list ++ wild ++ ""` equals the current SHA-256 of `list ++ wild`, every already-published checksum stays valid, and `Manager.loadSource` (`src/filter/manager.zig:543-598`) treats a missing `<id>.allow` file as an empty body. No refetch is forced by upgrading. `bodyChecksum` (manager.zig:1572) follows the same order; `source_file_suffixes` (manager.zig:1589) gains `.allow.tmp` and `.allow` with longest-suffix-first order preserved; the on-disk header (manager.zig:221-241) gains `# exceptions {d}` after the `# wildcards` line and the pinning test at manager.zig:1914-1946 is extended, not weakened.
|
||||
|
||||
The "checksum over the `.list` body followed by the `.wild` body" sentence
|
||||
exists in THREE places, not the one this ruling first named: `compiler.zig:38-39`,
|
||||
`manager.zig:218` and `sources_repo.zig:100`. All three move together, or the
|
||||
next reader trusts a stale one.
|
||||
The "checksum over the `.list` body followed by the `.wild` body" sentence exists in THREE places, not the one this ruling first named: `compiler.zig:38-39`, `manager.zig:218` and `sources_repo.zig:100`. All three move together, or the next reader trusts a stale one.
|
||||
|
||||
### 4. Exception counts persist and surface
|
||||
|
||||
Migration step 3 (`ddl_v3`, appended at `src/storage/migrations.zig:23-26`):
|
||||
`ALTER TABLE blocklist_sources ADD COLUMN exception_count INTEGER NOT NULL
|
||||
DEFAULT 0;`. `sources_repo.SourceRow` and `updateSourceStats`
|
||||
(`src/storage/repositories/sources_repo.zig:80-93,159-169`) carry it, and the
|
||||
checksum doc line at sources_repo.zig:100 is updated alongside
|
||||
`bodyChecksum`'s per ruling 3;
|
||||
`SourceStatus` rehydration (`manager.zig:1458-1502`) restores it alongside the
|
||||
existing three counts; `StatusView` (`src/web/handlers/blocklists.zig:52-78`)
|
||||
gains `exceptions: u32`; the blocklists UI shows it where `skipped_regex`
|
||||
already shows.
|
||||
Migration step 3 (`ddl_v3`, appended at `src/storage/migrations.zig:23-26`): `ALTER TABLE blocklist_sources ADD COLUMN exception_count INTEGER NOT NULL DEFAULT 0;`. `sources_repo.SourceRow` and `updateSourceStats` (`src/storage/repositories/sources_repo.zig:80-93,159-169`) carry it, and the checksum doc line at sources_repo.zig:100 is updated alongside `bodyChecksum`'s per ruling 3; `SourceStatus` rehydration (`manager.zig:1458-1502`) restores it alongside the existing three counts; `StatusView` (`src/web/handlers/blocklists.zig:52-78`) gains `exceptions: u32`; the blocklists UI shows it where `skipped_regex` already shows.
|
||||
|
||||
`skipped_regex` shows in TWO tables, and `exceptions` follows it into both:
|
||||
`SourceStatus.skipped_regex` (`web/src/lib/types.ts:192`) renders at
|
||||
`SourceStatusSection.tsx:112`, and `Blocklist.skipped_regex_count`
|
||||
(`types.ts:163`) renders at `BlocklistsPage.tsx:188`. S1 therefore owns
|
||||
`BlocklistsPage.tsx` as well.
|
||||
`skipped_regex` shows in TWO tables, and `exceptions` follows it into both: `SourceStatus.skipped_regex` (`web/src/lib/types.ts:192`) renders at `SourceStatusSection.tsx:112`, and `Blocklist.skipped_regex_count` (`types.ts:163`) renders at `BlocklistsPage.tsx:188`. S1 therefore owns `BlocklistsPage.tsx` as well.
|
||||
|
||||
### 5. The regex engine is a Pike VM, linear-time by construction, `std` only
|
||||
|
||||
New file `src/filter/regex.zig`. Syntax: literal bytes, `.`, character
|
||||
classes `[...]` with ranges and leading-`^` negation, escapes
|
||||
`\. \\ \- \d \w`, repetition `* + ? {n} {n,m}`, alternation `|`,
|
||||
non-capturing grouping `(...)`, anchors `^` and `$`. No backreferences, no
|
||||
lookaround, no captures.
|
||||
New file `src/filter/regex.zig`. Syntax: literal bytes, `.`, character classes `[...]` with ranges and leading-`^` negation, escapes `\. \\ \- \d \w`, repetition `* + ? {n} {n,m}`, alternation `|`, non-capturing grouping `(...)`, anchors `^` and `$`. No backreferences, no lookaround, no captures.
|
||||
|
||||
**Two syntax amendments from S2's review**, both widening what is accepted:
|
||||
|
||||
- `{n,}` is legal. It lowers to `{n}` followed by `*`, stays linear, and an
|
||||
over-large `n` still reports `PatternTooComplex`. Operators write this form;
|
||||
rejecting it buys no safety.
|
||||
- `\` before any ASCII punctuation yields that literal, not only the five
|
||||
escapes listed above — `\*`, `\/` and `\+` occur in Pi-hole-style patterns.
|
||||
This can only narrow a pattern to a literal, never silently change its
|
||||
meaning. Escapes that WOULD change meaning stay rejected: any alphanumeric
|
||||
escape outside `\d` and `\w` (`\s`, `\b`, `\1`, `\D`, `\p{L}`) is
|
||||
`BadPattern`.
|
||||
- `{n,}` is legal. It lowers to `{n}` followed by `*`, stays linear, and an over-large `n` still reports `PatternTooComplex`. Operators write this form; rejecting it buys no safety.
|
||||
- `\` before any ASCII punctuation yields that literal, not only the five escapes listed above — `\*`, `\/` and `\+` occur in Pi-hole-style patterns. This can only narrow a pattern to a literal, never silently change its meaning. Escapes that WOULD change meaning stay rejected: any alphanumeric escape outside `\d` and `\w` (`\s`, `\b`, `\1`, `\D`, `\p{L}`) is `BadPattern`.
|
||||
|
||||
**One rejection the review added:** a quantifier applied directly to another
|
||||
quantifier is `BadPattern`. `a+?` previously compiled as `(a+)?`, which
|
||||
matches every name — a block rule written in conventional lazy syntax would
|
||||
have sinkholed the whole LAN instead of being refused. Parenthesised forms
|
||||
such as `(a+)?` stay legal and keep their meaning. Matching is unanchored unless anchors are written
|
||||
(POSIX-grep convention, matching Pi-hole user expectations). Input is the
|
||||
normalized lowercase name, ≤ `types.max_name_len` bytes. Hard limits, each a
|
||||
distinct error: pattern ≤ 256 bytes (`PatternTooLong`), compiled program
|
||||
≤ 1024 instructions (`PatternTooComplex`). Public API:
|
||||
**One rejection the review added:** a quantifier applied directly to another quantifier is `BadPattern`. `a+?` previously compiled as `(a+)?`, which matches every name — a block rule written in conventional lazy syntax would have sinkholed the whole LAN instead of being refused. Parenthesised forms such as `(a+)?` stay legal and keep their meaning. Matching is unanchored unless anchors are written (POSIX-grep convention, matching Pi-hole user expectations). Input is the normalized lowercase name, ≤ `types.max_name_len` bytes. Hard limits, each a distinct error: pattern ≤ 256 bytes (`PatternTooLong`), compiled program ≤ 1024 instructions (`PatternTooComplex`). Public API:
|
||||
|
||||
```zig
|
||||
pub const Error = error{ OutOfMemory, BadPattern, PatternTooLong, PatternTooComplex };
|
||||
@@ -132,166 +52,44 @@ pub fn compile(gpa: Allocator, pattern: []const u8) Error!Program;
|
||||
pub fn matches(prog: *const Program, input: []const u8) bool;
|
||||
```
|
||||
|
||||
`matches` is a Pike VM: two thread lists, each program counter admitted at
|
||||
most once per input position, worst case O(program × input) with zero
|
||||
allocation at match time.
|
||||
`matches` is a Pike VM: two thread lists, each program counter admitted at most once per input position, worst case O(program × input) with zero allocation at match time.
|
||||
|
||||
**Amended in S2.** This ruling first said the thread lists live in the
|
||||
`Program`, sized at compile time. They do not, and must not: the runtime is
|
||||
`std.Io.Threaded`, so several query threads evaluate one shared snapshot at
|
||||
once. Scratch inside a shared `Program` is a data race, and reaching it
|
||||
through `*const Program` would need a `@constCast` that is undefined
|
||||
behaviour on a genuinely const program. All VM scratch — both thread lists,
|
||||
the admission marks and the closure stack — is instead a fixed array on the
|
||||
caller's stack, sized by the compile-time `max_program_len` constant. Zero
|
||||
allocation at match time is preserved, the published signature is unchanged,
|
||||
and a `Program` becomes safe to share across threads, which the original
|
||||
wording would have prevented. The engine is a fuzz-module root like parsers.zig and imports only
|
||||
`std`.
|
||||
**Amended in S2.** This ruling first said the thread lists live in the `Program`, sized at compile time. They do not, and must not: the runtime is `std.Io.Threaded`, so several query threads evaluate one shared snapshot at once. Scratch inside a shared `Program` is a data race, and reaching it through `*const Program` would need a `@constCast` that is undefined behaviour on a genuinely const program. All VM scratch — both thread lists, the admission marks and the closure stack — is instead a fixed array on the caller's stack, sized by the compile-time `max_program_len` constant. Zero allocation at match time is preserved, the published signature is unchanged, and a `Program` becomes safe to share across threads, which the original wording would have prevented. The engine is a fuzz-module root like parsers.zig and imports only `std`.
|
||||
|
||||
### 6. `regex` is a third rule kind, validated at the edge, memoized by the cache
|
||||
|
||||
Migration step 4 (`ddl_v4`): the 12-step rebuild of `rules` with
|
||||
`CHECK(kind IN ('exact','wildcard','regex'))` — the frozen v1 DDL
|
||||
(`src/storage/config_schema.zig:65-72`) cannot be edited. The rebuilt table
|
||||
keeps the name `rules`: `config_schema.table_names`
|
||||
(config_schema.zig:112-119) and the invariant tests at
|
||||
config_schema.zig:120-127 and migrations.zig:356 assert the schema's table
|
||||
set, and a rename would fail both. `model.RuleKind`
|
||||
(`src/config/model.zig:259-275`) gains `.regex`; the exhaustive switches in
|
||||
`config/validate.zig:1099-1130` (compile the pattern, report
|
||||
`"... is not a valid regex pattern"` through the existing error path at
|
||||
validate.zig:891-902) and `src/filter/rules.zig:62-68` extend. `RuleSet`
|
||||
(`rules.zig:28-36`) grows `regex_allow` and `regex_block` slices holding
|
||||
compiled `Program`s plus their pattern texts (for `Decision.matched`);
|
||||
`bucketOf` (rules.zig:133-143) becomes a six-bucket layout, and the fixed
|
||||
`var spans: [4]std.ArrayList(Span)` at rules.zig:55 widens with it;
|
||||
`patternIsValid` (`validate.zig:1099-1103`) is declared
|
||||
`error{OutOfMemory}!bool`, so a regex compile's `BadPattern`,
|
||||
`PatternTooLong` and `PatternTooComplex` must either fold into `false` or
|
||||
widen that error set together with its caller at validate.zig:893 — the
|
||||
diagnostic text itself needs no edit, because validate.zig:898 already
|
||||
interpolates `rule.kind.toDb()` into "{f} is not a valid {s} pattern";
|
||||
`max_regex_per_group: usize = 256` with `TooManyRegexRules` mirroring
|
||||
`max_wildcards_per_group` (rules.zig:26). A pattern that fails to compile is
|
||||
`error.BadPattern` at snapshot build, never skipped (rules.zig:41-49 doc
|
||||
holds). Reason tags `rule_allow_regex` and `rule_block_regex` join
|
||||
`matcher.Reason` (matcher.zig:24-32); `/api/lookup` and the query log pick
|
||||
them up automatically via `@tagName` (`src/web/handlers/lookup.zig:89`;
|
||||
`max_reason_len = 32` in `src/storage/logger.zig` fits both at 16 chars).
|
||||
Regex evaluation runs only after every hash and wildcard level missed, and
|
||||
answers are memoized by the existing DNS cache like every other decision, so
|
||||
the per-query cost lands on cache misses only.
|
||||
Migration step 4 (`ddl_v4`): the 12-step rebuild of `rules` with `CHECK(kind IN ('exact','wildcard','regex'))` — the frozen v1 DDL (`src/storage/config_schema.zig:65-72`) cannot be edited. The rebuilt table keeps the name `rules`: `config_schema.table_names` (config_schema.zig:112-119) and the invariant tests at config_schema.zig:120-127 and migrations.zig:356 assert the schema's table set, and a rename would fail both. `model.RuleKind` (`src/config/model.zig:259-275`) gains `.regex`; the exhaustive switches in `config/validate.zig:1099-1130` (compile the pattern, report `"... is not a valid regex pattern"` through the existing error path at validate.zig:891-902) and `src/filter/rules.zig:62-68` extend. `RuleSet` (`rules.zig:28-36`) grows `regex_allow` and `regex_block` slices holding compiled `Program`s plus their pattern texts (for `Decision.matched`); `bucketOf` (rules.zig:133-143) becomes a six-bucket layout, and the fixed `var spans: [4]std.ArrayList(Span)` at rules.zig:55 widens with it; `patternIsValid` (`validate.zig:1099-1103`) is declared `error{OutOfMemory}!bool`, so a regex compile's `BadPattern`, `PatternTooLong` and `PatternTooComplex` must either fold into `false` or widen that error set together with its caller at validate.zig:893 — the diagnostic text itself needs no edit, because validate.zig:898 already interpolates `rule.kind.toDb()` into "{f} is not a valid {s} pattern"; `max_regex_per_group: usize = 256` with `TooManyRegexRules` mirroring `max_wildcards_per_group` (rules.zig:26). A pattern that fails to compile is `error.BadPattern` at snapshot build, never skipped (rules.zig:41-49 doc holds). Reason tags `rule_allow_regex` and `rule_block_regex` join `matcher.Reason` (matcher.zig:24-32); `/api/lookup` and the query log pick them up automatically via `@tagName` (`src/web/handlers/lookup.zig:89`; `max_reason_len = 32` in `src/storage/logger.zig` fits both at 16 chars). Regex evaluation runs only after every hash and wildcard level missed, and answers are memoized by the existing DNS cache like every other decision, so the per-query cost lands on cache misses only.
|
||||
|
||||
### 7. The web contract names the third kind everywhere it names the first two
|
||||
|
||||
`src/web/handlers/rules.zig`: `toInput` accepts `"regex"`; the 400 string at
|
||||
rules.zig:42 becomes `"kind must be 'exact', 'wildcard' or 'regex'"`.
|
||||
`src/web/openapi.yaml:1948,1962,1976`: all three `enum: [exact, wildcard]` become
|
||||
`[exact, wildcard, regex]`. `web/src/lib/types.ts:195`:
|
||||
`RuleKind = "exact" | "wildcard" | "regex"`; the rules page kind selector
|
||||
gains the option. Milestone 23 replaced the native `<select>` with a React
|
||||
Aria wrapper, so that is now a data edit, not JSX: append to `KIND_OPTIONS`
|
||||
at `RulesPage.tsx:15-18`, and extend the option-list assertion at
|
||||
`RulesPage.test.tsx:119`. Any new test that opens the selector depends on the
|
||||
`CSS.escape` polyfill in `web/vitest.setup.ts`. Contract samples are
|
||||
regenerated with the AGENTS.md command
|
||||
(`zig build test -Dintegration -Dcontract-samples-out=...`); no npm script
|
||||
generates them. `nxdns export` / `import`
|
||||
round-trip the new kind with no extra work once `RuleKind.toDb/fromDb` extend
|
||||
— the enum round-trip test at model.zig:782 is extended to prove it.
|
||||
`src/web/handlers/rules.zig`: `toInput` accepts `"regex"`; the 400 string at rules.zig:42 becomes `"kind must be 'exact', 'wildcard' or 'regex'"`. `src/web/openapi.yaml:1948,1962,1976`: all three `enum: [exact, wildcard]` become `[exact, wildcard, regex]`. `web/src/lib/types.ts:195`: `RuleKind = "exact" | "wildcard" | "regex"`; the rules page kind selector gains the option. Milestone 23 replaced the native `<select>` with a React Aria wrapper, so that is now a data edit, not JSX: append to `KIND_OPTIONS` at `RulesPage.tsx:15-18`, and extend the option-list assertion at `RulesPage.test.tsx:119`. Any new test that opens the selector depends on the `CSS.escape` polyfill in `web/vitest.setup.ts`. Contract samples are regenerated with the AGENTS.md command (`zig build test -Dintegration -Dcontract-samples-out=...`); no npm script generates them. `nxdns export` / `import` round-trip the new kind with no extra work once `RuleKind.toDb/fromDb` extend — the enum round-trip test at model.zig:782 is extended to prove it.
|
||||
|
||||
### 8. PLAN amendments land with the code, in S3
|
||||
|
||||
PLAN.md:36 (§2.2) is rewritten: operator regex rules are in scope, backed by
|
||||
the linear-time engine of ruling 5; regex lines in downloaded lists stay
|
||||
counted and skipped; `$` modifiers (except the `$important` suffix of ruling
|
||||
1), partial-segment wildcards, and browser-syntax honoring stay permanently
|
||||
out. PLAN.md:26 (§2.1 filtering sentence), PLAN.md:106 (§3.9), PLAN.md:108-117
|
||||
(§3.10 precedence) and PLAN.md:700 (decision B) are updated to match rulings
|
||||
2 and 6. PLAN.md:99 (§3.8) documents exactly two compiled bodies and names
|
||||
`<source_id>.wild` as the second; ruling 3's third body makes it stale, so S1
|
||||
updates that line when it lands the `.allow` body. PLAN.md:73 ("No
|
||||
TOML/regex/HTTP packages needed") stays true and stays as written — a
|
||||
homegrown engine adds no package. In-code echoes of the old §2.2 move with it:
|
||||
`src/filter/wildcard.zig:6-7,20-23`, `src/filter/parsers.zig:25`,
|
||||
`src/filter/parser_abp.zig:5-6`, `src/filter/compiler.zig:129-131`.
|
||||
PLAN.md:36 (§2.2) is rewritten: operator regex rules are in scope, backed by the linear-time engine of ruling 5; regex lines in downloaded lists stay counted and skipped; `$` modifiers (except the `$important` suffix of ruling 1), partial-segment wildcards, and browser-syntax honoring stay permanently out. PLAN.md:26 (§2.1 filtering sentence), PLAN.md:106 (§3.9), PLAN.md:108-117 (§3.10 precedence) and PLAN.md:700 (decision B) are updated to match rulings 2 and 6. PLAN.md:99 (§3.8) documents exactly two compiled bodies and names `<source_id>.wild` as the second; ruling 3's third body makes it stale, so S1 updates that line when it lands the `.allow` body. PLAN.md:73 ("No TOML/regex/HTTP packages needed") stays true and stays as written — a homegrown engine adds no package. In-code echoes of the old §2.2 move with it: `src/filter/wildcard.zig:6-7,20-23`, `src/filter/parsers.zig:25`, `src/filter/parser_abp.zig:5-6`, `src/filter/compiler.zig:129-131`.
|
||||
|
||||
### 9. Milestone 20's authority modes bound where rules are written
|
||||
|
||||
m20 classifies POST/PUT/DELETE `/api/rules` as `.config_write`
|
||||
(`src/web/routes.zig:94-97`), and under `.managed_file` authority the router
|
||||
answers 403 (`src/web/router.zig:173-178`). Everything in this milestone
|
||||
works in both modes, but the write path differs: in `.database` mode regex
|
||||
rules arrive through the API; in `.managed_file` mode they arrive through the
|
||||
config file and `src/config/reconcile.zig` (`reconcileRules`,
|
||||
reconcile.zig:566-598), whose enum comparison carries `.regex` with no code
|
||||
change. S3 owns a reconcile test proving a file-declared regex rule converges
|
||||
into the table. Every S3 API acceptance check runs a server in `.database`
|
||||
authority (no `--config`). The `SELECT *` dump helpers
|
||||
(`src/config/import.zig:167-186`, `src/config/reconcile.zig:981`) will emit
|
||||
the new `exception_count` column after `ddl_v3`; golden dump assertions in
|
||||
those suites are updated in S1, which owns that migration. Reconcile's
|
||||
runtime-column protection (reconcile.zig:11-15,435) covers `exception_count`
|
||||
with no change — the column survives reconciles untouched.
|
||||
m20 classifies POST/PUT/DELETE `/api/rules` as `.config_write` (`src/web/routes.zig:94-97`), and under `.managed_file` authority the router answers 403 (`src/web/router.zig:173-178`). Everything in this milestone works in both modes, but the write path differs: in `.database` mode regex rules arrive through the API; in `.managed_file` mode they arrive through the config file and `src/config/reconcile.zig` (`reconcileRules`, reconcile.zig:566-598), whose enum comparison carries `.regex` with no code change. S3 owns a reconcile test proving a file-declared regex rule converges into the table. Every S3 API acceptance check runs a server in `.database` authority (no `--config`). The `SELECT *` dump helpers (`src/config/import.zig:167-186`, `src/config/reconcile.zig:981`) will emit the new `exception_count` column after `ddl_v3`; golden dump assertions in those suites are updated in S1, which owns that migration. Reconcile's runtime-column protection (reconcile.zig:11-15,435) covers `exception_count` with no change — the column survives reconciles untouched.
|
||||
|
||||
### 10. Fuzz invariants move, never lapse
|
||||
|
||||
`tests/fuzz/blocklist_fuzz.zig` header invariant "covers_apex only on
|
||||
`.wildcard`" (its stated form at :4-28) becomes "only on `.wildcard` or
|
||||
`.exception`". `tests/fuzz/compiler_fuzz.zig:43` reads `Format` from
|
||||
`compile`'s parameter list by index — the new `allow_w` parameter appends
|
||||
after `wild_w`, leaving index 2 valid; the session touching `compile` runs the
|
||||
fuzz suite and fixes that line if the assumption fails. A new
|
||||
`tests/fuzz/regex_fuzz.zig` target asserts: `compile` on arbitrary bytes
|
||||
never crashes and either errors or produces a program within the ruling-5
|
||||
limits; `matches` terminates and its VM step count never exceeds
|
||||
program length × (input length + 1); compile-then-match is deterministic.
|
||||
`tests/fuzz/blocklist_fuzz.zig` header invariant "covers_apex only on `.wildcard`" (its stated form at :4-28) becomes "only on `.wildcard` or `.exception`". `tests/fuzz/compiler_fuzz.zig:43` reads `Format` from `compile`'s parameter list by index — the new `allow_w` parameter appends after `wild_w`, leaving index 2 valid; the session touching `compile` runs the fuzz suite and fixes that line if the assumption fails. A new `tests/fuzz/regex_fuzz.zig` target asserts: `compile` on arbitrary bytes never crashes and either errors or produces a program within the ruling-5 limits; `matches` terminates and its VM step count never exceeds program length × (input length + 1); compile-then-match is deterministic.
|
||||
|
||||
## Sessions
|
||||
|
||||
Three sessions. S1 and S2 run in parallel — they share no files. S3 starts
|
||||
after both land.
|
||||
Three sessions. S1 and S2 run in parallel — they share no files. S3 starts after both land.
|
||||
|
||||
### Session S1: list exceptions end to end (tier 1)
|
||||
|
||||
Owns: `src/filter/parsers.zig`, `src/filter/parser_abp.zig`,
|
||||
`src/filter/compiler.zig`, `src/filter/manager.zig`, `src/filter/matcher.zig`,
|
||||
`src/filter/domain_set.zig` (only if a helper is needed; expected untouched),
|
||||
`src/filter/filter_integration_test.zig`, `src/storage/migrations.zig`
|
||||
(step 3 only), `src/storage/repositories/sources_repo.zig`,
|
||||
`src/web/handlers/blocklists.zig`, `src/web/handlers/lookup.zig` (doc
|
||||
sentence only), `src/web/openapi.yaml` (StatusView shape only),
|
||||
`web/src/features/blocklists/*` (which includes `BlocklistsPage.tsx` per
|
||||
ruling 4), `web/src/lib/types.ts` (source-stat fields
|
||||
only), `web/src/lib/contractSamples.gen.ts`, `PLAN.md` (line 99 only, per
|
||||
ruling 8 — S3 owns every other PLAN edit),
|
||||
`tests/fuzz/blocklist_fuzz.zig`, `tests/fuzz/compiler_fuzz.zig`, and the
|
||||
dump-golden assertions in `src/config/import.zig` and
|
||||
`src/config/reconcile.zig` test suites only (ruling 9's `exception_count`
|
||||
fallout from `ddl_v3`; S1 touches no reconcile logic).
|
||||
Owns: `src/filter/parsers.zig`, `src/filter/parser_abp.zig`, `src/filter/compiler.zig`, `src/filter/manager.zig`, `src/filter/matcher.zig`, `src/filter/domain_set.zig` (only if a helper is needed; expected untouched), `src/filter/filter_integration_test.zig`, `src/storage/migrations.zig` (step 3 only), `src/storage/repositories/sources_repo.zig`, `src/web/handlers/blocklists.zig`, `src/web/handlers/lookup.zig` (doc sentence only), `src/web/openapi.yaml` (StatusView shape only), `web/src/features/blocklists/*` (which includes `BlocklistsPage.tsx` per ruling 4), `web/src/lib/types.ts` (source-stat fields only), `web/src/lib/contractSamples.gen.ts`, `PLAN.md` (line 99 only, per ruling 8 — S3 owns every other PLAN edit), `tests/fuzz/blocklist_fuzz.zig`, `tests/fuzz/compiler_fuzz.zig`, and the dump-golden assertions in `src/config/import.zig` and `src/config/reconcile.zig` test suites only (ruling 9's `exception_count` fallout from `ddl_v3`; S1 touches no reconcile logic).
|
||||
|
||||
- S1.1 `Kind.exception` in parsers.zig; parser_abp emits it per ruling 1;
|
||||
parser_hosts and parser_domains never emit it (no change beyond the enum).
|
||||
- S1.1 `Kind.exception` in parsers.zig; parser_abp emits it per ruling 1; parser_hosts and parser_domains never emit it (no change beyond the enum).
|
||||
- S1.2 compiler third body per ruling 3; `Counts.exceptions`.
|
||||
- S1.3 manager: suffixes, header line, `bodyChecksum`, `loadSource`
|
||||
missing-file-is-empty, `Snapshot.Compiled.allow_body`,
|
||||
`prepareRefresh`/`publishRefresh`/`applyLoadOutcomes` carry the count.
|
||||
`rejectedWithoutEntries` (manager.zig:1563-1570) treats a compile with only
|
||||
exceptions as loadable, not rejected.
|
||||
- S1.4 matcher: `SourceSets.exceptions`, `Reason.blocklist_exception`,
|
||||
the new evaluate level per ruling 2, `memoryBytes` includes the new sets.
|
||||
- S1.5 storage + API + UI per ruling 4. `migrations.zig:349` asserts
|
||||
`target_version == 2`; S1 moves it to 3 (S3 moves it to 4). The spec did
|
||||
not name that test; it fails otherwise.
|
||||
- S1.6 tests: parser cases (`@@||x^`, `@@||x`, `@@||x^$important`,
|
||||
`@@||x^$third-party` → unsupported, `@@x` → unsupported); compiler
|
||||
three-body + checksum-compat cases (empty allow body reproduces the old
|
||||
digest byte for byte); manager header pin extended; matcher precedence
|
||||
cases: operator block beats list exception, list exception beats list
|
||||
domain and list wildcard, exception parent walk covers apex and
|
||||
subdomains; an integration case through
|
||||
`filter_integration_test.zig` with a real ABP fixture carrying `@@` lines.
|
||||
- S1.3 manager: suffixes, header line, `bodyChecksum`, `loadSource` missing-file-is-empty, `Snapshot.Compiled.allow_body`, `prepareRefresh`/`publishRefresh`/`applyLoadOutcomes` carry the count. `rejectedWithoutEntries` (manager.zig:1563-1570) treats a compile with only exceptions as loadable, not rejected.
|
||||
- S1.4 matcher: `SourceSets.exceptions`, `Reason.blocklist_exception`, the new evaluate level per ruling 2, `memoryBytes` includes the new sets.
|
||||
- S1.5 storage + API + UI per ruling 4. `migrations.zig:349` asserts `target_version == 2`; S1 moves it to 3 (S3 moves it to 4). The spec did not name that test; it fails otherwise.
|
||||
- S1.6 tests: parser cases (`@@||x^`, `@@||x`, `@@||x^$important`, `@@||x^$third-party` → unsupported, `@@x` → unsupported); compiler three-body + checksum-compat cases (empty allow body reproduces the old digest byte for byte); manager header pin extended; matcher precedence cases: operator block beats list exception, list exception beats list domain and list wildcard, exception parent walk covers apex and subdomains; an integration case through `filter_integration_test.zig` with a real ABP fixture carrying `@@` lines.
|
||||
|
||||
Acceptance (S1):
|
||||
- [x] `zig build test` passes; fuzz targets build and run.
|
||||
@@ -312,15 +110,10 @@ Acceptance (S1):
|
||||
|
||||
### Session S2: the regex engine (tier 2, engine only)
|
||||
|
||||
Owns: `src/filter/regex.zig` (new), `tests/fuzz/regex_fuzz.zig` (new),
|
||||
`src/tests.zig` (one added line), `build.zig` (fuzz-suite wiring for the new
|
||||
target only).
|
||||
Owns: `src/filter/regex.zig` (new), `tests/fuzz/regex_fuzz.zig` (new), `src/tests.zig` (one added line), `build.zig` (fuzz-suite wiring for the new target only).
|
||||
|
||||
- S2.1 the engine per ruling 5: parser → AST → NFA program → Pike VM.
|
||||
- S2.2 unit tests in-file: every syntax form; anchored and unanchored
|
||||
matching; negated classes; `{n,m}` bounds; each error case; the
|
||||
pathological backtracker-killers (`(a+)+b` against `aaaaaaaaaaaaaaaaaaaaX`,
|
||||
nested alternation) complete within the step bound.
|
||||
- S2.2 unit tests in-file: every syntax form; anchored and unanchored matching; negated classes; `{n,m}` bounds; each error case; the pathological backtracker-killers (`(a+)+b` against `aaaaaaaaaaaaaaaaaaaaX`, nested alternation) complete within the step bound.
|
||||
- S2.3 the fuzz target per ruling 10.
|
||||
|
||||
Acceptance (S2):
|
||||
@@ -334,32 +127,15 @@ Acceptance (S2):
|
||||
|
||||
### Session S3: the regex rule kind, wired through (needs S1 + S2)
|
||||
|
||||
Owns: `src/storage/migrations.zig` (step 4), `src/storage/config_schema.zig`
|
||||
(comment only if needed), `src/config/model.zig`, `src/config/validate.zig`,
|
||||
`src/filter/rules.zig`, `src/filter/matcher.zig`,
|
||||
`src/storage/repositories/rules_repo.zig`, `src/web/handlers/rules.zig`,
|
||||
`src/web/handlers/mutations.zig` (only if `checkRule` needs the kind),
|
||||
`src/web/openapi.yaml`, `web/src/features/rules/*`, `web/src/lib/types.ts`,
|
||||
`web/src/lib/contractSamples.gen.ts`, `src/config/reconcile.zig` (test per
|
||||
ruling 9), `PLAN.md`, `src/filter/wildcard.zig` (comments),
|
||||
`src/filter/parsers.zig` (comment), `src/filter/parser_abp.zig` (comment),
|
||||
`src/filter/compiler.zig` (comment), `tools/bench.zig`.
|
||||
Owns: `src/storage/migrations.zig` (step 4), `src/storage/config_schema.zig` (comment only if needed), `src/config/model.zig`, `src/config/validate.zig`, `src/filter/rules.zig`, `src/filter/matcher.zig`, `src/storage/repositories/rules_repo.zig`, `src/web/handlers/rules.zig`, `src/web/handlers/mutations.zig` (only if `checkRule` needs the kind), `src/web/openapi.yaml`, `web/src/features/rules/*`, `web/src/lib/types.ts`, `web/src/lib/contractSamples.gen.ts`, `src/config/reconcile.zig` (test per ruling 9), `PLAN.md`, `src/filter/wildcard.zig` (comments), `src/filter/parsers.zig` (comment), `src/filter/parser_abp.zig` (comment), `src/filter/compiler.zig` (comment), `tools/bench.zig`.
|
||||
|
||||
- S3.1 migration step 4 per ruling 6; repo and model layers.
|
||||
- S3.2 validate at both edges (config import, API) per rulings 6 and 7.
|
||||
- S3.3 `RuleSet` six buckets; matcher levels per ruling 2; reasons.
|
||||
- S3.4 web + UI + openapi + samples per ruling 7.
|
||||
- S3.5 PLAN and comment amendments per ruling 8.
|
||||
- S3.6 bench: the filter suite in `tools/bench.zig` gains a variant with 32
|
||||
regex rules loaded; the existing p95 < 1 ms assertion covers it. The line to
|
||||
change is `tools/bench.zig:158`, which currently reads `.rules = &.{}` — the
|
||||
filter bench loads no rules at all today, so the variant is new coverage
|
||||
rather than an edit to an existing rule set. S3 also moves
|
||||
`migrations.zig:349` from 3 to 4.
|
||||
- S3.7 tests: rule CRUD with kind `regex` through the API including the 400
|
||||
for a bad pattern at insert time; precedence cases regex-allow over
|
||||
regex-block, wildcard over regex, regex over list entries; export/import
|
||||
round trip; a migration test upgrading a v3 database.
|
||||
- S3.6 bench: the filter suite in `tools/bench.zig` gains a variant with 32 regex rules loaded; the existing p95 < 1 ms assertion covers it. The line to change is `tools/bench.zig:158`, which currently reads `.rules = &.{}` — the filter bench loads no rules at all today, so the variant is new coverage rather than an edit to an existing rule set. S3 also moves `migrations.zig:349` from 3 to 4.
|
||||
- S3.7 tests: rule CRUD with kind `regex` through the API including the 400 for a bad pattern at insert time; precedence cases regex-allow over regex-block, wildcard over regex, regex over list entries; export/import round trip; a migration test upgrading a v3 database.
|
||||
|
||||
Acceptance (S3):
|
||||
- [x] `zig build test` and `cd web && npm test` pass.
|
||||
@@ -380,11 +156,7 @@ Acceptance (S3):
|
||||
|
||||
### Orchestrator
|
||||
|
||||
Verify S1 and S2 acceptance before starting S3. After S3: run the full gate
|
||||
set (`zig build test`, `test-aarch64` if qemu present, `npm test`,
|
||||
`npm run assert-bundled`), then a live smoke against a scratch server: load
|
||||
one real ABP list with `@@` lines, add one regex rule, verify both over dig
|
||||
and `/api/lookup`. Record deviations in `## Recorded (implementation)`.
|
||||
Verify S1 and S2 acceptance before starting S3. After S3: run the full gate set (`zig build test`, `test-aarch64` if qemu present, `npm test`, `npm run assert-bundled`), then a live smoke against a scratch server: load one real ABP list with `@@` lines, add one regex rule, verify both over dig and `/api/lookup`. Record deviations in `## Recorded (implementation)`.
|
||||
|
||||
## Module layout
|
||||
|
||||
@@ -419,227 +191,56 @@ Deleted surface: none.
|
||||
|
||||
## Recorded (anchor re-verification, 2026-08-12)
|
||||
|
||||
The spec was written against `ffc3ca6` and verified against `a8e0fe4`. It was
|
||||
re-verified a third time after milestones 22 and 23 landed (TypeScript 7,
|
||||
Tailwind removed, StyleX and React Aria). Findings, all folded into the
|
||||
rulings above:
|
||||
The spec was written against `ffc3ca6` and verified against `a8e0fe4`. It was re-verified a third time after milestones 22 and 23 landed (TypeScript 7, Tailwind removed, StyleX and React Aria). Findings, all folded into the rulings above:
|
||||
|
||||
- **No Zig file changed** between `a8e0fe4` and this re-verification. Every
|
||||
Zig anchor holds; six ranges are off by a line or two but still contain what
|
||||
the spec names. The schema is still at version 2, so migration steps 3 and 4
|
||||
are genuinely new.
|
||||
- The rules-page kind selector is no longer a `<select>`. It is a
|
||||
`KIND_OPTIONS` array feeding `web/src/ui/Select.tsx`, a React Aria wrapper
|
||||
(ruling 7 rewritten).
|
||||
- `migrations.zig:349` pins `target_version` and is unnamed by the original
|
||||
spec. S1 moves it to 3, S3 to 4.
|
||||
- `exception_count` has two UI sites, not one, so S1 owns `BlocklistsPage.tsx`
|
||||
(ruling 4 rewritten).
|
||||
- **No Zig file changed** between `a8e0fe4` and this re-verification. Every Zig anchor holds; six ranges are off by a line or two but still contain what the spec names. The schema is still at version 2, so migration steps 3 and 4 are genuinely new.
|
||||
- The rules-page kind selector is no longer a `<select>`. It is a `KIND_OPTIONS` array feeding `web/src/ui/Select.tsx`, a React Aria wrapper (ruling 7 rewritten).
|
||||
- `migrations.zig:349` pins `target_version` and is unnamed by the original spec. S1 moves it to 3, S3 to 4.
|
||||
- `exception_count` has two UI sites, not one, so S1 owns `BlocklistsPage.tsx` (ruling 4 rewritten).
|
||||
- The checksum sentence has three copies, not one (ruling 3 rewritten).
|
||||
- `patternIsValid`'s error set cannot carry the engine's three error tags as
|
||||
written (ruling 6 rewritten).
|
||||
- `PLAN.md:99` documents two compiled bodies and goes stale with ruling 3
|
||||
(ruling 8 rewritten).
|
||||
- `tools/bench.zig:158` reads `.rules = &.{}`, so the filter bench loads no
|
||||
rules today.
|
||||
- `web/vitest.setup.ts` polyfills `CSS.escape`; without it, a test that opens
|
||||
the React Aria selector throws under jsdom.
|
||||
- `npm run build` gained `scripts/assert-css-layers.mjs`, which fails on any
|
||||
rule outside a cascade layer. A pure StyleX change cannot trip it.
|
||||
- `patternIsValid`'s error set cannot carry the engine's three error tags as written (ruling 6 rewritten).
|
||||
- `PLAN.md:99` documents two compiled bodies and goes stale with ruling 3 (ruling 8 rewritten).
|
||||
- `tools/bench.zig:158` reads `.rules = &.{}`, so the filter bench loads no rules today.
|
||||
- `web/vitest.setup.ts` polyfills `CSS.escape`; without it, a test that opens the React Aria selector throws under jsdom.
|
||||
- `npm run build` gained `scripts/assert-css-layers.mjs`, which fails on any rule outside a cascade layer. A pure StyleX change cannot trip it.
|
||||
|
||||
## Recorded (implementation)
|
||||
|
||||
Deviations and findings from the build, the Codex review rounds and the live
|
||||
smoke. Everything here is folded into the code; nothing is outstanding.
|
||||
Deviations and findings from the build, the Codex review rounds and the live smoke. Everything here is folded into the code; nothing is outstanding.
|
||||
|
||||
- **Regex scratch is on the caller's stack, not in `Program`.** `std.Io.Threaded`
|
||||
means several query threads share one snapshot, so the Pike VM's two thread
|
||||
lists cannot live in the compiled program. `matches` takes its scratch from
|
||||
the caller's frame, which keeps `Program` immutable and shareable and keeps
|
||||
the match path allocation-free.
|
||||
- **`{n,}` is legal and `\` + any ASCII punctuation is legal.** Ruling 5 named
|
||||
neither. Quantifier-on-quantifier (`a+?` read as `(a+)?`) is `BadPattern`:
|
||||
the first Codex round found `a+?` compiling to something that matched
|
||||
everything.
|
||||
- **Embedded whitespace was accepted on the anchored ABP forms.** A line such
|
||||
as `||good.example bad.example^` reached the compiler, which lowercases and
|
||||
length-checks but does not reject a space, and wrote an entry only a query
|
||||
carrying the same space could match. `compiler.zig` passes `.wildcard` and
|
||||
`.exception` text to `addCandidate` whole, which is why the space survived
|
||||
there. `parser_abp.isNameCandidate` now refuses whitespace and control bytes
|
||||
on those two paths.
|
||||
- **Regex scratch is on the caller's stack, not in `Program`.** `std.Io.Threaded` means several query threads share one snapshot, so the Pike VM's two thread lists cannot live in the compiled program. `matches` takes its scratch from the caller's frame, which keeps `Program` immutable and shareable and keeps the match path allocation-free.
|
||||
- **`{n,}` is legal and `\` + any ASCII punctuation is legal.** Ruling 5 named neither. Quantifier-on-quantifier (`a+?` read as `(a+)?`) is `BadPattern`: the first Codex round found `a+?` compiling to something that matched everything.
|
||||
- **Embedded whitespace was accepted on the anchored ABP forms.** A line such as `||good.example bad.example^` reached the compiler, which lowercases and length-checks but does not reject a space, and wrote an entry only a query carrying the same space could match. `compiler.zig` passes `.wildcard` and `.exception` text to `addCandidate` whole, which is why the space survived there. `parser_abp.isNameCandidate` now refuses whitespace and control bytes on those two paths.
|
||||
|
||||
The bare-name path deliberately does **not** use it. `compiler.zig:93-98`
|
||||
tokenizes `.domain` text on whitespace and adds each field separately, so a
|
||||
bare line carrying a space was never broken — it produced two valid entries.
|
||||
Since `detectFormat` assigns one format per source, a mostly-ABP list that
|
||||
also carries hosts-style lines depends on exactly that tokenizer to keep
|
||||
them working. The first attempt at this fix applied the helper to all three
|
||||
paths and silently dropped that fallback; the second Codex pass caught it.
|
||||
Two tests now pin it: a parser test that the bare form stays `.domain`, and
|
||||
a compiler test that an ABP-classified list carrying `0.0.0.0 ads.example`
|
||||
still emits `ads.example`. The third pass pointed out that the parser test
|
||||
alone would pass even if the compiler stopped tokenizing, which is the
|
||||
behaviour the fallback actually depends on.
|
||||
- **The `.allow` body left stale enumerations behind it.** Adding a third
|
||||
compiled body — and with it a fourth temporary, `.allow.tmp` — updated the
|
||||
production code but not every place that lists the file names. The third review pass found four: two `manager.zig` tests that
|
||||
spell the names by hand, the orphan-sweep fixture in
|
||||
`filter_integration_test.zig`, and `PLAN.md` §3.13 and §5. The
|
||||
`manager.zig` table-driven test was worse than stale — it iterates
|
||||
`source_file_suffixes` itself, so deleting an entry changes the code and the
|
||||
test's expectations together and everything still passes. The whole repo was
|
||||
then swept for the pattern rather than the four instances patched, which
|
||||
turned up seven more — including the reload-cancellation test, whose fixture
|
||||
wrote no `.allow` file at all, so the third of `loadSource`'s three read
|
||||
sites was never exercised. A fourth pass then found test 10e comparing only
|
||||
`.list` and `.wild` across a restart, so a restart that rewrote the exception
|
||||
body alone would have stayed green. A `comptime` assertion on
|
||||
`source_file_suffixes.len` now breaks the build when a suffix is added or
|
||||
removed without updating the hand-written tests.
|
||||
- **A pre-existing flake in the required suite.** `src/cli.zig:1538` asserted
|
||||
`std.mem.count(u8, text, "OK") == 0` over output that embeds the temporary
|
||||
directory path. `std.testing.tmpDir` names that directory with base64 over
|
||||
12 random bytes, whose 64-symbol alphabet includes uppercase: 15 adjacent
|
||||
positions each carry `OK` with probability 1/4096, so about one run in 273
|
||||
fails a test that has nothing to do with naming. It surfaced during
|
||||
this milestone's watched-fail injections. The assertion now checks that no
|
||||
line *starts* with a verdict, which covers both `OK:` and
|
||||
`OK upstreams[...]` and cannot match a path segment. Reproduced and fixed
|
||||
outside the milestone's scope because a randomly failing required gate
|
||||
devalues every green run after it.
|
||||
- **`PLAN.md` still described the seed-once config model.** Seven sites said or
|
||||
implied that the first start seeds the database from `/etc/nxdns/config.zon`,
|
||||
which milestone 20 replaced with the two authority modes selected by the
|
||||
presence of `--config`. One of them listed a `config/bootstrap.zig` that does
|
||||
not exist — the module is `loader.zig` plus `reconcile.zig`. The claim was
|
||||
checked against `src/app.zig:339`, not just against the docs. This is
|
||||
milestone-20 drift found while fixing the milestone-21 echoes in the same
|
||||
document, and corrected because PLAN is the source of truth a later session
|
||||
builds from.
|
||||
- **`PLAN.md` §12.1 held a config sample nobody could load.** It described an
|
||||
`.upstream.servers` field that never existed and omitted the required
|
||||
`.groups` and `.upstreams`. The section now points at
|
||||
`docs/reference/configuration.md` and `nxdns export` and keeps only a
|
||||
skeleton: a second copy of the schema is what produced the drift, so the
|
||||
copy is gone rather than corrected.
|
||||
- **`PLAN.md` §7.1 claimed blocklist entries never parent-walk.** Two of the
|
||||
three list levels do: wildcard entries match every proper parent, and
|
||||
exception entries walk the candidate chain, which is why
|
||||
`@@||good.ads.example^` also lifts `y.good.ads.example`. Only domain entries
|
||||
match the query name alone.
|
||||
- **`src/config/model.zig`'s header described the retired bootstrap** and
|
||||
omitted `exception_count` from its list of runtime columns. The first attempt
|
||||
at correcting it introduced a new error — it called import a wholesale
|
||||
replacement set against reconciliation — which the sixth pass caught.
|
||||
`import.zig:3` is explicit that import has been a thin wrapper over
|
||||
`reconcile.zig` since milestone 20, so there is one declarative write path,
|
||||
not two, and it preserves the runtime state of every row the input still
|
||||
names (`reconcile.zig:1139`). A seventh pass then corrected two more claims
|
||||
in the same header: `Config` is the whole shape of a config file but not the
|
||||
only shape the repositories accept (the API writes through `RuleInput`,
|
||||
`ClientInput`, `ClientEdit`), and compiled-body reuse turns on the preserved
|
||||
source id and checksum rather than the counters — `loadSource` names the
|
||||
files after the id and accepts them only against the stored checksum.
|
||||
- **`PLAN.md` §11.2 presented the v1 DDL as the live schema.** It predates
|
||||
three migrations, so it lacks `upstreams.tls_name` and
|
||||
`blocklist_sources.exception_count` and its `kind` CHECK admits only
|
||||
`exact` and `wildcard` — implementing against it would produce a database
|
||||
that rejects every regex rule this milestone added. The section is now
|
||||
labelled the v1 baseline and points at `config_schema.zig` and
|
||||
`migrations.zig`, with the three steps named.
|
||||
- **`INSTALL.md` said the service reads `config.zon` "on the first start".**
|
||||
Neither authority mode behaves that way: under `run --config` the service
|
||||
reads it on every start, and under database authority `nxdns import` reads it
|
||||
while a bare `nxdns run` never does. The sentence justified a file mode, so
|
||||
an operator tightening permissions after first boot would have broken the
|
||||
next file-mode restart. More milestone-20 drift. The claim had a second copy
|
||||
in `docs/how-to/install-with-systemd.md`, found only because the seventh pass
|
||||
looked for it after the first copy was fixed.
|
||||
- **`PLAN.md` §7.1 and the Phase 5 summary missed this milestone's own
|
||||
additions.** The evaluation sequence went straight from operator rules to
|
||||
blocklist domains, omitting the exception level, and the matcher was still
|
||||
enumerated as exact/parent/wildcard with no regex. Ruling 8 required these
|
||||
echoes and S3 did not reach them.
|
||||
- **The UI trimmed regex patterns.** `RulesPage.onSubmit` applied
|
||||
`pattern.trim()` to every kind, so a regex created in the UI did not store
|
||||
the bytes an identical `POST /api/rules` would. It now trims only `exact`
|
||||
and `wildcard`, where the server normalizes anyway. No client-side
|
||||
whitespace rejection was added: `" foo|bar"` still has a live `bar` branch,
|
||||
so refusing it would over-reject, and the server stays the authority on
|
||||
pattern validity.
|
||||
- **The rule pattern field opted into mobile autocapitalization.** An
|
||||
autocapitalized regex validates and then silently never matches, because
|
||||
query names are lowercase and a regex is never normalized. The field now
|
||||
sets `autoCapitalize="none"`, `autoCorrect="off"`, `spellCheck={false}`.
|
||||
- **`RuleSet.build` received the snapshot arena for its temporaries.** An
|
||||
arena reclaims only its most recent allocation, so every `defer …deinit`
|
||||
inside `build` was a silent no-op and the scratch survived until snapshot
|
||||
teardown, uncounted by `memoryBytes()`. `build` now takes a permanent and a
|
||||
scratch allocator, and `matcher.zig` passes `arena` and `gpa` respectively.
|
||||
Regex programs compile into scratch and are copied across by a new
|
||||
`Program.clone`, which keeps `regex.zig` a single-allocator engine and
|
||||
leaves `compile`'s signature — and therefore `config/validate.zig` and the
|
||||
fuzz target — untouched.
|
||||
The bare-name path deliberately does **not** use it. `compiler.zig:93-98` tokenizes `.domain` text on whitespace and adds each field separately, so a bare line carrying a space was never broken — it produced two valid entries. Since `detectFormat` assigns one format per source, a mostly-ABP list that also carries hosts-style lines depends on exactly that tokenizer to keep them working. The first attempt at this fix applied the helper to all three paths and silently dropped that fallback; the second Codex pass caught it. Two tests now pin it: a parser test that the bare form stays `.domain`, and a compiler test that an ABP-classified list carrying `0.0.0.0 ads.example` still emits `ads.example`. The third pass pointed out that the parser test alone would pass even if the compiler stopped tokenizing, which is the behaviour the fallback actually depends on.
|
||||
- **The `.allow` body left stale enumerations behind it.** Adding a third compiled body — and with it a fourth temporary, `.allow.tmp` — updated the production code but not every place that lists the file names. The third review pass found four: two `manager.zig` tests that spell the names by hand, the orphan-sweep fixture in `filter_integration_test.zig`, and `PLAN.md` §3.13 and §5. The `manager.zig` table-driven test was worse than stale — it iterates `source_file_suffixes` itself, so deleting an entry changes the code and the test's expectations together and everything still passes. The whole repo was then swept for the pattern rather than the four instances patched, which turned up seven more — including the reload-cancellation test, whose fixture wrote no `.allow` file at all, so the third of `loadSource`'s three read sites was never exercised. A fourth pass then found test 10e comparing only `.list` and `.wild` across a restart, so a restart that rewrote the exception body alone would have stayed green. A `comptime` assertion on `source_file_suffixes.len` now breaks the build when a suffix is added or removed without updating the hand-written tests.
|
||||
- **A pre-existing flake in the required suite.** `src/cli.zig:1538` asserted `std.mem.count(u8, text, "OK") == 0` over output that embeds the temporary directory path. `std.testing.tmpDir` names that directory with base64 over 12 random bytes, whose 64-symbol alphabet includes uppercase: 15 adjacent positions each carry `OK` with probability 1/4096, so about one run in 273 fails a test that has nothing to do with naming. It surfaced during this milestone's watched-fail injections. The assertion now checks that no line *starts* with a verdict, which covers both `OK:` and `OK upstreams[...]` and cannot match a path segment. Reproduced and fixed outside the milestone's scope because a randomly failing required gate devalues every green run after it.
|
||||
- **`PLAN.md` still described the seed-once config model.** Seven sites said or implied that the first start seeds the database from `/etc/nxdns/config.zon`, which milestone 20 replaced with the two authority modes selected by the presence of `--config`. One of them listed a `config/bootstrap.zig` that does not exist — the module is `loader.zig` plus `reconcile.zig`. The claim was checked against `src/app.zig:339`, not just against the docs. This is milestone-20 drift found while fixing the milestone-21 echoes in the same document, and corrected because PLAN is the source of truth a later session builds from.
|
||||
- **`PLAN.md` §12.1 held a config sample nobody could load.** It described an `.upstream.servers` field that never existed and omitted the required `.groups` and `.upstreams`. The section now points at `docs/reference/configuration.md` and `nxdns export` and keeps only a skeleton: a second copy of the schema is what produced the drift, so the copy is gone rather than corrected.
|
||||
- **`PLAN.md` §7.1 claimed blocklist entries never parent-walk.** Two of the three list levels do: wildcard entries match every proper parent, and exception entries walk the candidate chain, which is why `@@||good.ads.example^` also lifts `y.good.ads.example`. Only domain entries match the query name alone.
|
||||
- **`src/config/model.zig`'s header described the retired bootstrap** and omitted `exception_count` from its list of runtime columns. The first attempt at correcting it introduced a new error — it called import a wholesale replacement set against reconciliation — which the sixth pass caught. `import.zig:3` is explicit that import has been a thin wrapper over `reconcile.zig` since milestone 20, so there is one declarative write path, not two, and it preserves the runtime state of every row the input still names (`reconcile.zig:1139`). A seventh pass then corrected two more claims in the same header: `Config` is the whole shape of a config file but not the only shape the repositories accept (the API writes through `RuleInput`, `ClientInput`, `ClientEdit`), and compiled-body reuse turns on the preserved source id and checksum rather than the counters — `loadSource` names the files after the id and accepts them only against the stored checksum.
|
||||
- **`PLAN.md` §11.2 presented the v1 DDL as the live schema.** It predates three migrations, so it lacks `upstreams.tls_name` and `blocklist_sources.exception_count` and its `kind` CHECK admits only `exact` and `wildcard` — implementing against it would produce a database that rejects every regex rule this milestone added. The section is now labelled the v1 baseline and points at `config_schema.zig` and `migrations.zig`, with the three steps named.
|
||||
- **`INSTALL.md` said the service reads `config.zon` "on the first start".** Neither authority mode behaves that way: under `run --config` the service reads it on every start, and under database authority `nxdns import` reads it while a bare `nxdns run` never does. The sentence justified a file mode, so an operator tightening permissions after first boot would have broken the next file-mode restart. More milestone-20 drift. The claim had a second copy in `docs/how-to/install-with-systemd.md`, found only because the seventh pass looked for it after the first copy was fixed.
|
||||
- **`PLAN.md` §7.1 and the Phase 5 summary missed this milestone's own additions.** The evaluation sequence went straight from operator rules to blocklist domains, omitting the exception level, and the matcher was still enumerated as exact/parent/wildcard with no regex. Ruling 8 required these echoes and S3 did not reach them.
|
||||
- **The UI trimmed regex patterns.** `RulesPage.onSubmit` applied `pattern.trim()` to every kind, so a regex created in the UI did not store the bytes an identical `POST /api/rules` would. It now trims only `exact` and `wildcard`, where the server normalizes anyway. No client-side whitespace rejection was added: `" foo|bar"` still has a live `bar` branch, so refusing it would over-reject, and the server stays the authority on pattern validity.
|
||||
- **The rule pattern field opted into mobile autocapitalization.** An autocapitalized regex validates and then silently never matches, because query names are lowercase and a regex is never normalized. The field now sets `autoCapitalize="none"`, `autoCorrect="off"`, `spellCheck={false}`.
|
||||
- **`RuleSet.build` received the snapshot arena for its temporaries.** An arena reclaims only its most recent allocation, so every `defer …deinit` inside `build` was a silent no-op and the scratch survived until snapshot teardown, uncounted by `memoryBytes()`. `build` now takes a permanent and a scratch allocator, and `matcher.zig` passes `arena` and `gpa` respectively. Regex programs compile into scratch and are copied across by a new `Program.clone`, which keeps `regex.zig` a single-allocator engine and leaves `compile`'s signature — and therefore `config/validate.zig` and the fuzz target — untouched.
|
||||
|
||||
Measured on 16 groups each holding 256 regex rules of 254 bytes,
|
||||
4096 wildcards and 2048 exact rules, with no blocklist sources:
|
||||
arena capacity fell from 135,662,440 to 12,686,214 bytes against an
|
||||
unchanged `memoryBytes()` of 11,058,791, so the ratio of real to reported
|
||||
went from 12.27× to 1.15×. The hidden footprint per snapshot fell from
|
||||
118.83 MiB to 1.55 MiB, so 117.28 MiB went away. A reload holds two
|
||||
snapshots, so it was carrying twice that.
|
||||
`memoryBytes()` needed no change: the formula was always right about what
|
||||
the `RuleSet` retains, and the divergence was arena capacity the formula
|
||||
does not claim to describe. The residual 1.15× is arena node headers and
|
||||
page rounding, which `memoryBytes` documents itself as excluding.
|
||||
Measured on 16 groups each holding 256 regex rules of 254 bytes, 4096 wildcards and 2048 exact rules, with no blocklist sources: arena capacity fell from 135,662,440 to 12,686,214 bytes against an unchanged `memoryBytes()` of 11,058,791, so the ratio of real to reported went from 12.27× to 1.15×. The hidden footprint per snapshot fell from 118.83 MiB to 1.55 MiB, so 117.28 MiB went away. A reload holds two snapshots, so it was carrying twice that. `memoryBytes()` needed no change: the formula was always right about what the `RuleSet` retains, and the divergence was arena capacity the formula does not claim to describe. The residual 1.15× is arena node headers and page rounding, which `memoryBytes` documents itself as excluding.
|
||||
|
||||
The guard is `the build's temporaries stay out of the permanent allocator`,
|
||||
which asserts `arena.queryCapacity() < 2 * set.memoryBytes()` and passes
|
||||
`scratch` as `testing.allocator`, so a permanent allocation wrongly taken
|
||||
from scratch also fails as a leak. It was watched failing with the split
|
||||
reverted.
|
||||
- **An interactive `--fuzz` session cannot run on zig 0.16.0.** Building any
|
||||
fuzz target with `-ffuzz` fails inside the stock
|
||||
`/usr/lib/zig/compiler/test_runner.zig:566`, which passes a
|
||||
`*builtin.StackTrace` to `debug.writeStackTrace` where a
|
||||
`*const debug.StackTrace` is wanted — two distinct struct declarations. The
|
||||
failure is entirely inside the toolchain and reproduces on `compiler-fuzz`,
|
||||
a target this milestone did not touch. Corpus replay under `zig build test`
|
||||
is unaffected and remains the evidence for the step-bound property.
|
||||
- **`api.md` gained a block-reason table.** The milestone added three reason
|
||||
tags and no doc page enumerated any of them. The table lists all nine in
|
||||
evaluation order plus `none` and the `cname:` prefix.
|
||||
- **Two comment sites still counted three temporaries.** `manager.zig` line 40
|
||||
(the `refresh_lock` invariant) and line 1293 (`pruneOrphans`) both omitted
|
||||
`.allow.tmp`. Each presents itself as exhaustive, so a maintainer could have
|
||||
added an `.allow.tmp` writer outside `refresh_lock` and let the orphan sweep
|
||||
delete it mid-compile. `sourceFileId` and `source_file_suffixes` already
|
||||
matched all four.
|
||||
- **The rule pattern placeholder named only two kinds.** Ruling 7 requires the
|
||||
web contract to name the third kind everywhere it names the first two, and
|
||||
the placeholder read `ads.example.com or *.example.com`. It now carries a
|
||||
regex example too.
|
||||
- **Two pre-existing tutorial errors surfaced during the docs sweep.**
|
||||
`tutorial/first-run.md` step 14 claimed the compiled list stays on disk when
|
||||
it is in fact pruned (real output: `pruned orphaned blocklist file 1.allow`,
|
||||
`1.wild`, `1.list`), and step 11 named `ads.doubleclick.net`, which
|
||||
StevenBlack no longer carries.
|
||||
The guard is `the build's temporaries stay out of the permanent allocator`, which asserts `arena.queryCapacity() < 2 * set.memoryBytes()` and passes `scratch` as `testing.allocator`, so a permanent allocation wrongly taken from scratch also fails as a leak. It was watched failing with the split reverted.
|
||||
- **An interactive `--fuzz` session cannot run on zig 0.16.0.** Building any fuzz target with `-ffuzz` fails inside the stock `/usr/lib/zig/compiler/test_runner.zig:566`, which passes a `*builtin.StackTrace` to `debug.writeStackTrace` where a `*const debug.StackTrace` is wanted — two distinct struct declarations. The failure is entirely inside the toolchain and reproduces on `compiler-fuzz`, a target this milestone did not touch. Corpus replay under `zig build test` is unaffected and remains the evidence for the step-bound property.
|
||||
- **`api.md` gained a block-reason table.** The milestone added three reason tags and no doc page enumerated any of them. The table lists all nine in evaluation order plus `none` and the `cname:` prefix.
|
||||
- **Two comment sites still counted three temporaries.** `manager.zig` line 40 (the `refresh_lock` invariant) and line 1293 (`pruneOrphans`) both omitted `.allow.tmp`. Each presents itself as exhaustive, so a maintainer could have added an `.allow.tmp` writer outside `refresh_lock` and let the orphan sweep delete it mid-compile. `sourceFileId` and `source_file_suffixes` already matched all four.
|
||||
- **The rule pattern placeholder named only two kinds.** Ruling 7 requires the web contract to name the third kind everywhere it names the first two, and the placeholder read `ads.example.com or *.example.com`. It now carries a regex example too.
|
||||
- **Two pre-existing tutorial errors surfaced during the docs sweep.** `tutorial/first-run.md` step 14 claimed the compiled list stays on disk when it is in fact pruned (real output: `pruned orphaned blocklist file 1.allow`, `1.wild`, `1.list`), and step 11 named `ads.doubleclick.net`, which StevenBlack no longer carries.
|
||||
|
||||
## Anti-requirements
|
||||
|
||||
- No `$` modifier support beyond tolerating `$important` on exception lines.
|
||||
`$dnstype`, `$dnsrewrite`, `$client`, `$denyallow` and every browser
|
||||
modifier stay unsupported and counted.
|
||||
- No regex from downloaded lists. `.regex` lines stay counted and skipped;
|
||||
`skipped_regex` keeps its meaning. The engine exists for operator rules
|
||||
only.
|
||||
- No partial-segment wildcards (`ads*.example.com`) — writing one as a rule
|
||||
stays rejected; the regex kind covers the need.
|
||||
- No backreferences, lookaround, captures, named groups or Unicode classes in
|
||||
the engine, ever. A pattern needing them is rejected, not approximated.
|
||||
- No `$` modifier support beyond tolerating `$important` on exception lines. `$dnstype`, `$dnsrewrite`, `$client`, `$denyallow` and every browser modifier stay unsupported and counted.
|
||||
- No regex from downloaded lists. `.regex` lines stay counted and skipped; `skipped_regex` keeps its meaning. The engine exists for operator rules only.
|
||||
- No partial-segment wildcards (`ads*.example.com`) — writing one as a rule stays rejected; the regex kind covers the need.
|
||||
- No backreferences, lookaround, captures, named groups or Unicode classes in the engine, ever. A pattern needing them is rejected, not approximated.
|
||||
- No PCRE2, RE2 or any external regex dependency.
|
||||
- No exception-rule UI editor: exceptions come from lists; operators write
|
||||
allow rules.
|
||||
- No re-download forced by the upgrade; checksum compatibility (ruling 3) is
|
||||
a requirement, not an optimization.
|
||||
- No exception-rule UI editor: exceptions come from lists; operators write allow rules.
|
||||
- No re-download forced by the upgrade; checksum compatibility (ruling 3) is a requirement, not an optimization.
|
||||
|
||||
+16
-58
@@ -1,63 +1,32 @@
|
||||
# Milestone 22: TypeScript 7
|
||||
|
||||
Goal: move the web toolchain from TypeScript 6.0.3 to TypeScript 7.0.x (the
|
||||
native compiler, GA 2026-07-08). One devDependency bump; every web gate and
|
||||
the release pipeline stay green.
|
||||
Goal: move the web toolchain from TypeScript 6.0.3 to TypeScript 7.0.x (the native compiler, GA 2026-07-08). One devDependency bump; every web gate and the release pipeline stay green.
|
||||
|
||||
Design written 2026-08-12 against HEAD `91a0aa9`. TypeScript 7.0.2 is
|
||||
`typescript@latest` on npm at spec time; the milestone pins whatever 7.0.x is
|
||||
current when the session runs.
|
||||
Design written 2026-08-12 against HEAD `91a0aa9`. TypeScript 7.0.2 is `typescript@latest` on npm at spec time; the milestone pins whatever 7.0.x is current when the session runs.
|
||||
|
||||
## Implementation contract (read first)
|
||||
|
||||
- Read `AGENTS.md`, then this spec whole, before session work starts.
|
||||
- `web/src/lib/contractSamples.gen.ts` is generated; it is type-checked by
|
||||
`tsc -b` but never hand-edited. If TS7 rejects it, the fix goes in the
|
||||
generator (`AGENTS.md` regeneration procedure), not the file.
|
||||
- No license work: `typescript` is a devDependency, absent from both the
|
||||
runtime closure and the bundled set in `licenses/dependency-identity.txt`.
|
||||
The Zig drift guard (`src/licenses_drift_test.zig`) must still pass —
|
||||
it recomputes from `web/package-lock.json` and dev-only changes are
|
||||
invisible to it.
|
||||
- `web/src/lib/contractSamples.gen.ts` is generated; it is type-checked by `tsc -b` but never hand-edited. If TS7 rejects it, the fix goes in the generator (`AGENTS.md` regeneration procedure), not the file.
|
||||
- No license work: `typescript` is a devDependency, absent from both the runtime closure and the bundled set in `licenses/dependency-identity.txt`. The Zig drift guard (`src/licenses_drift_test.zig`) must still pass — it recomputes from `web/package-lock.json` and dev-only changes are invisible to it.
|
||||
|
||||
## Rulings (binding)
|
||||
|
||||
### 1. The bump is one line plus the lockfile
|
||||
|
||||
`web/package.json:45` `"typescript": "6.0.3"` becomes the current 7.0.x,
|
||||
exact pin (repo convention: no ranges, milestone-14 ruling 12 pins by exact
|
||||
version everywhere). `npm install` updates `web/package-lock.json`. Nothing
|
||||
else in `dependencies` or `devDependencies` moves in this milestone.
|
||||
`web/package.json:45` `"typescript": "6.0.3"` becomes the current 7.0.x, exact pin (repo convention: no ranges, milestone-14 ruling 12 pins by exact version everywhere). `npm install` updates `web/package-lock.json`. Nothing else in `dependencies` or `devDependencies` moves in this milestone.
|
||||
|
||||
### 2. The tsconfigs are already TS7-clean; verify, do not churn
|
||||
|
||||
Checked against TS7's removed options (recorded here so the session does not
|
||||
re-derive it): `tsconfig.app.json` and `tsconfig.node.json` use
|
||||
`moduleResolution: "bundler"`, `module: "esnext"`, targets `es2022`/`es2023`,
|
||||
`paths` without `baseUrl`, `verbatimModuleSyntax`, `isolatedModules`,
|
||||
`noEmit`. None of TS7's removals (`target: es5`, `downlevelIteration`,
|
||||
`moduleResolution: node10`/`classic`, `module: amd/umd/system/none`,
|
||||
`baseUrl`-as-alias) are present. `tsconfig.json` is a two-entry project
|
||||
reference; TS7 supports `tsc -b` and project references. The session changes
|
||||
a tsconfig only if `tsc -b` errors demand it, and records any such change in
|
||||
`## Recorded (implementation)`.
|
||||
Checked against TS7's removed options (recorded here so the session does not re-derive it): `tsconfig.app.json` and `tsconfig.node.json` use `moduleResolution: "bundler"`, `module: "esnext"`, targets `es2022`/`es2023`, `paths` without `baseUrl`, `verbatimModuleSyntax`, `isolatedModules`, `noEmit`. None of TS7's removals (`target: es5`, `downlevelIteration`, `moduleResolution: node10`/`classic`, `module: amd/umd/system/none`, `baseUrl`-as-alias) are present. `tsconfig.json` is a two-entry project reference; TS7 supports `tsc -b` and project references. The session changes a tsconfig only if `tsc -b` errors demand it, and records any such change in `## Recorded (implementation)`.
|
||||
|
||||
### 3. Known TS7 edge: `skipLibCheck` no longer hides parse-level .d.ts errors
|
||||
|
||||
Both tsconfigs set `skipLibCheck: true`. Under TS7 that still skips type
|
||||
checking of `.d.ts` files but not parse-level errors in them. If a dependency
|
||||
`.d.ts` fails to parse, the fix is a dependency patch bump or an upstream
|
||||
issue reference recorded in the spec — never a copied-and-edited local
|
||||
`.d.ts`.
|
||||
Both tsconfigs set `skipLibCheck: true`. Under TS7 that still skips type checking of `.d.ts` files but not parse-level errors in them. If a dependency `.d.ts` fails to parse, the fix is a dependency patch bump or an upstream issue reference recorded in the spec — never a copied-and-edited local `.d.ts`.
|
||||
|
||||
### 4. Nothing else in the toolchain consumes the TS compiler
|
||||
|
||||
Verified at spec time: `oxlint` (own parser), `vite`/`vitest`
|
||||
(esbuild transpile, no type checking), `prettier`, and
|
||||
`@vitejs/plugin-react` run no `tsc` and load no `typescript` API. The only
|
||||
consumer is the `typecheck` script (`web/package.json:12`, `tsc -b`), run in
|
||||
CI at `.gitea/workflows/gates.yml:113-114`. TS7 has no stable programmatic
|
||||
API until 7.1; nothing in this repo needs one.
|
||||
Verified at spec time: `oxlint` (own parser), `vite`/`vitest` (esbuild transpile, no type checking), `prettier`, and `@vitejs/plugin-react` run no `tsc` and load no `typescript` API. The only consumer is the `typecheck` script (`web/package.json:12`, `tsc -b`), run in CI at `.gitea/workflows/gates.yml:113-114`. TS7 has no stable programmatic API until 7.1; nothing in this repo needs one.
|
||||
|
||||
## Sessions
|
||||
|
||||
@@ -65,16 +34,11 @@ One session.
|
||||
|
||||
### Session S1: the bump
|
||||
|
||||
Owns: `web/package.json`, `web/package-lock.json`, and — only if ruling 2
|
||||
forces it — `web/tsconfig.app.json`, `web/tsconfig.node.json`,
|
||||
`web/tsconfig.json`.
|
||||
Owns: `web/package.json`, `web/package-lock.json`, and — only if ruling 2 forces it — `web/tsconfig.app.json`, `web/tsconfig.node.json`, `web/tsconfig.json`.
|
||||
|
||||
- S1.1 bump per ruling 1; `npm install`.
|
||||
- S1.2 run, in order: `npm run format:check`, `npm run lint`,
|
||||
`npm run typecheck`, `npm test`, `npm run build`, `npm run assert-bundled`
|
||||
(all from `web/`).
|
||||
- S1.3 `zig build test` (the licenses drift guard and the contract-samples
|
||||
guard run inside it).
|
||||
- S1.2 run, in order: `npm run format:check`, `npm run lint`, `npm run typecheck`, `npm test`, `npm run build`, `npm run assert-bundled` (all from `web/`).
|
||||
- S1.3 `zig build test` (the licenses drift guard and the contract-samples guard run inside it).
|
||||
|
||||
Acceptance (S1):
|
||||
- [ ] `web/node_modules/.bin/tsc --version` reports 7.0.x.
|
||||
@@ -95,14 +59,8 @@ New files: none. Deleted surface: none.
|
||||
|
||||
## Anti-requirements
|
||||
|
||||
- No other dependency bumps ride along — not oxlint, not vite, not vitest,
|
||||
not `@types/*`.
|
||||
- No tsconfig flag additions or "modernization" beyond what a TS7 error
|
||||
forces.
|
||||
- No typescript-eslint, ts-jest, or any tool needing the TS programmatic API
|
||||
(unavailable until 7.1, and unwanted regardless — oxlint is the linter).
|
||||
- No `tsgo` binary references anywhere: the binary is `tsc`, scripts stay as
|
||||
they are.
|
||||
- No edits to `web/src/**` — a source change to satisfy the new compiler is a
|
||||
finding to record and fix, not silently absorb, unless `tsc -b` fails
|
||||
without it; then it is recorded in `## Recorded (implementation)`.
|
||||
- No other dependency bumps ride along — not oxlint, not vite, not vitest, not `@types/*`.
|
||||
- No tsconfig flag additions or "modernization" beyond what a TS7 error forces.
|
||||
- No typescript-eslint, ts-jest, or any tool needing the TS programmatic API (unavailable until 7.1, and unwanted regardless — oxlint is the linter).
|
||||
- No `tsgo` binary references anywhere: the binary is `tsc`, scripts stay as they are.
|
||||
- No edits to `web/src/**` — a source change to satisfy the new compiler is a finding to record and fix, not silently absorb, unless `tsc -b` fails without it; then it is recorded in `## Recorded (implementation)`.
|
||||
|
||||
+67
-345
@@ -1,153 +1,57 @@
|
||||
# Milestone 23: StyleX and React Aria replace Tailwind
|
||||
|
||||
Goal: every styled element in `web/src` renders through StyleX
|
||||
(`@stylexjs/unplugin` at build time); Dialog, AlertDialog, Tabs and Select
|
||||
come from `react-aria-components`; Tailwind is removed. TanStack Router and
|
||||
TanStack Query stay (decided 2026-08-12 after review; the router swap is out
|
||||
of scope permanently, not deferred).
|
||||
Goal: every styled element in `web/src` renders through StyleX (`@stylexjs/unplugin` at build time); Dialog, AlertDialog, Tabs and Select come from `react-aria-components`; Tailwind is removed. TanStack Router and TanStack Query stay (decided 2026-08-12 after review; the router swap is out of scope permanently, not deferred).
|
||||
|
||||
Design written 2026-08-12 against HEAD `91a0aa9`. Needs milestone 22 landed
|
||||
(the StyleX and RAC `.d.ts` surfaces are checked by the TS7 `tsc -b`).
|
||||
Design written 2026-08-12 against HEAD `91a0aa9`. Needs milestone 22 landed (the StyleX and RAC `.d.ts` surfaces are checked by the TS7 `tsc -b`).
|
||||
|
||||
Inventory this spec rests on: 522 `className=` occurrences across 31
|
||||
`web/src` files; 17 shared class constants in `web/src/ui/classes.ts`;
|
||||
`web/src/styles.css` is the single line `@import "tailwindcss"` (so dark mode
|
||||
is Tailwind 4's default `prefers-color-scheme` media strategy — no class
|
||||
toggle exists); the Tailwind plugin sits at `web/vite.config.ts:3,8`;
|
||||
`tailwindcss` and `@tailwindcss/vite` are devDependencies
|
||||
(`web/package.json:34,44`) and appear in neither the runtime closure nor the
|
||||
bundled set of `licenses/dependency-identity.txt`.
|
||||
Inventory this spec rests on: 522 `className=` occurrences across 31 `web/src` files; 17 shared class constants in `web/src/ui/classes.ts`; `web/src/styles.css` is the single line `@import "tailwindcss"` (so dark mode is Tailwind 4's default `prefers-color-scheme` media strategy — no class toggle exists); the Tailwind plugin sits at `web/vite.config.ts:3,8`; `tailwindcss` and `@tailwindcss/vite` are devDependencies (`web/package.json:34,44`) and appear in neither the runtime closure nor the bundled set of `licenses/dependency-identity.txt`.
|
||||
|
||||
## Implementation contract (read first)
|
||||
|
||||
- Read `AGENTS.md`, then this spec whole, before session work starts.
|
||||
- `web/src/lib/contractSamples.gen.ts` and `web/src/lib/types.ts` are API
|
||||
contract surfaces; this milestone is styling-only and touches neither.
|
||||
- Every session leaves all gates green: `npm run format:check`, `npm run
|
||||
lint`, `npm run typecheck`, `npm test`, `npm run build`,
|
||||
`npm run assert-bundled` (from `web/`), and `zig build test`. The release
|
||||
gate (`zig build dist` + `verify-dist`, `.gitea/workflows/gates.yml`) keeps
|
||||
its byte budgets; they are not tight — `web/dist` is 440,924 bytes raw
|
||||
against the 15 MiB binary budget — but `verify-dist` remains the arbiter.
|
||||
- `assert-bundled` fails whenever the bundled npm package set changes. That
|
||||
failure is the license workflow trigger, not an obstacle: review each new
|
||||
package into `licenses/inventory.zon`, then record the new set in
|
||||
`licenses/dependency-identity.txt` (procedure in that file's header).
|
||||
Never update the recorded list without the inventory review.
|
||||
- `web/src/lib/contractSamples.gen.ts` and `web/src/lib/types.ts` are API contract surfaces; this milestone is styling-only and touches neither.
|
||||
- Every session leaves all gates green: `npm run format:check`, `npm run lint`, `npm run typecheck`, `npm test`, `npm run build`, `npm run assert-bundled` (from `web/`), and `zig build test`. The release gate (`zig build dist` + `verify-dist`, `.gitea/workflows/gates.yml`) keeps its byte budgets; they are not tight — `web/dist` is 440,924 bytes raw against the 15 MiB binary budget — but `verify-dist` remains the arbiter.
|
||||
- `assert-bundled` fails whenever the bundled npm package set changes. That failure is the license workflow trigger, not an obstacle: review each new package into `licenses/inventory.zon`, then record the new set in `licenses/dependency-identity.txt` (procedure in that file's header). Never update the recorded list without the inventory review.
|
||||
|
||||
## Rulings (binding)
|
||||
|
||||
### 1. Build integration is `@stylexjs/unplugin`, proven before any page moves
|
||||
|
||||
`stylex.vite()` from `@stylexjs/unplugin` goes into `web/vite.config.ts`
|
||||
alongside the existing plugins (Tailwind stays until S3). Configuration per
|
||||
the official Vite guide (stylexjs.com/docs/learn/installation/vite):
|
||||
`useCSSLayers: true`, `runtimeInjection: false`, `dev` keyed off the Vite
|
||||
mode. The generated CSS injects into an existing CSS asset, so
|
||||
`web/src/styles.css` (imported by `main.tsx:8`) remains the CSS entry for the
|
||||
whole milestone; the dev/HMR virtual-module wiring from that guide is part of
|
||||
S1. The unmaintained community `vite-plugin-stylex` is forbidden. S1's
|
||||
acceptance proves dev server, `vitest run` and `vite build` all handle a
|
||||
StyleX component before S2 or S3 convert anything — this is a gate, not an
|
||||
assumption.
|
||||
`stylex.vite()` from `@stylexjs/unplugin` goes into `web/vite.config.ts` alongside the existing plugins (Tailwind stays until S3). Configuration per the official Vite guide (stylexjs.com/docs/learn/installation/vite): `useCSSLayers: true`, `runtimeInjection: false`, `dev` keyed off the Vite mode. The generated CSS injects into an existing CSS asset, so `web/src/styles.css` (imported by `main.tsx:8`) remains the CSS entry for the whole milestone; the dev/HMR virtual-module wiring from that guide is part of S1. The unmaintained community `vite-plugin-stylex` is forbidden. S1's acceptance proves dev server, `vitest run` and `vite build` all handle a StyleX component before S2 or S3 convert anything — this is a gate, not an assumption.
|
||||
|
||||
### 2. Tokens live in `web/src/ui/tokens.stylex.ts`
|
||||
|
||||
`stylex.defineVars` with role names, not raw shades: surface, surface-raised,
|
||||
border, border-strong, text, text-muted (the zinc ramp today), primary and
|
||||
primary-text (blue-600/white), danger, danger-surface, danger-border (the red
|
||||
ramp), focus (blue-600). Each var carries its dark value under
|
||||
`@media (prefers-color-scheme: dark)` inside `defineVars` — same media
|
||||
strategy the app has now. No theme toggle, no `data-theme` attribute.
|
||||
`stylex.defineVars` with role names, not raw shades: surface, surface-raised, border, border-strong, text, text-muted (the zinc ramp today), primary and primary-text (blue-600/white), danger, danger-surface, danger-border (the red ramp), focus (blue-600). Each var carries its dark value under `@media (prefers-color-scheme: dark)` inside `defineVars` — same media strategy the app has now. No theme toggle, no `data-theme` attribute.
|
||||
|
||||
### 3. `web/src/ui/styles.ts` replaces `ui/classes.ts`
|
||||
|
||||
The 17 exports of `web/src/ui/classes.ts` (focusRing, insetFocusRing, input,
|
||||
smallInput, button, smallButton, largeButton, primaryButton,
|
||||
largePrimaryButton, rowButton, linkButton, dangerLinkButton, retryButton, th,
|
||||
td, tableWrap, formCard) become `stylex.create` style objects in
|
||||
`web/src/ui/styles.ts`, composed at call sites via
|
||||
`{...stylex.props(styles.button, extra)}`. The milestone-9 accessibility
|
||||
floor restated by milestone-18 ruling 10 carries over verbatim: every
|
||||
interactive element renders the focus ring (`:focus-visible` outline, 2px,
|
||||
offset 2, focus token; the inset variant for controls flush against panel
|
||||
edges). `ui/classes.ts` is deleted in S3 when its last importer converts.
|
||||
The 17 exports of `web/src/ui/classes.ts` (focusRing, insetFocusRing, input, smallInput, button, smallButton, largeButton, primaryButton, largePrimaryButton, rowButton, linkButton, dangerLinkButton, retryButton, th, td, tableWrap, formCard) become `stylex.create` style objects in `web/src/ui/styles.ts`, composed at call sites via `{...stylex.props(styles.button, extra)}`. The milestone-9 accessibility floor restated by milestone-18 ruling 10 carries over verbatim: every interactive element renders the focus ring (`:focus-visible` outline, 2px, offset 2, focus token; the inset variant for controls flush against panel edges). `ui/classes.ts` is deleted in S3 when its last importer converts.
|
||||
|
||||
### 4. React Aria scope is exactly four components — end state, not a phase
|
||||
|
||||
From `react-aria-components`: Dialog + Modal (replacing the hand-rolled
|
||||
`role="dialog"` overlay in `web/src/features/clients/ClientEditDialog.tsx:27-28`),
|
||||
AlertDialog composition (ruling 5), Tabs (replacing the button-pair tab
|
||||
switcher in `web/src/features/local/LocalDnsPage.tsx`), Select (replacing the
|
||||
7 files carrying native `<select>`: LookupPage, QueryLogPage, RecordsTab,
|
||||
RulesPage, SettingsPage, PrefixesEditor, ClientEditDialog). Everything else —
|
||||
buttons, text inputs, tables, links, banners — stays native HTML styled with
|
||||
StyleX. A native styled `<button>` is the finished design, not an interim
|
||||
one. S2 wraps the four as shared components: `web/src/ui/Dialog.tsx`,
|
||||
`web/src/ui/ConfirmDialog.tsx`, `web/src/ui/Tabs.tsx`,
|
||||
`web/src/ui/Select.tsx`, each styled with StyleX through RAC's render-prop
|
||||
state booleans (`stylex.props(isSelected && styles.selected)`) — never
|
||||
data-attribute selectors, which StyleX cannot express.
|
||||
From `react-aria-components`: Dialog + Modal (replacing the hand-rolled `role="dialog"` overlay in `web/src/features/clients/ClientEditDialog.tsx:27-28`), AlertDialog composition (ruling 5), Tabs (replacing the button-pair tab switcher in `web/src/features/local/LocalDnsPage.tsx`), Select (replacing the 7 files carrying native `<select>`: LookupPage, QueryLogPage, RecordsTab, RulesPage, SettingsPage, PrefixesEditor, ClientEditDialog). Everything else — buttons, text inputs, tables, links, banners — stays native HTML styled with StyleX. A native styled `<button>` is the finished design, not an interim one. S2 wraps the four as shared components: `web/src/ui/Dialog.tsx`, `web/src/ui/ConfirmDialog.tsx`, `web/src/ui/Tabs.tsx`, `web/src/ui/Select.tsx`, each styled with StyleX through RAC's render-prop state booleans (`stylex.props(isSelected && styles.selected)`) — never data-attribute selectors, which StyleX cannot express.
|
||||
|
||||
### 5. `window.confirm` dies with the conversion
|
||||
|
||||
All call sites become `ConfirmDialog` (RAC AlertDialog: `role="alertdialog"`,
|
||||
focus trapped, danger-styled confirm button):
|
||||
`web/src/features/blocklists/BlocklistsPage.tsx:61`,
|
||||
`web/src/features/upstreams/UpstreamsPage.tsx:43`,
|
||||
`web/src/features/rules/RulesPage.tsx:34`, and
|
||||
`web/src/ui/useCrudForm.ts:49` (which carries the confirm for ZonesTab and
|
||||
RecordsTab). `useCrudForm`'s `CrudFormSpec.confirmDelete` keeps its signature;
|
||||
the hook returns the dialog state instead of calling `window.confirm`. The
|
||||
tests that stub confirm today
|
||||
(`web/src/features/upstreams/UpstreamsPage.test.tsx:140,144,188`,
|
||||
`web/src/ui/useCrudForm.test.tsx:102`) are rewritten to drive the real
|
||||
dialog: open, assert `role="alertdialog"` and the entity name in the message,
|
||||
confirm and cancel paths both asserted. BlocklistsPage and RulesPage gain the
|
||||
same coverage for their delete paths.
|
||||
All call sites become `ConfirmDialog` (RAC AlertDialog: `role="alertdialog"`, focus trapped, danger-styled confirm button): `web/src/features/blocklists/BlocklistsPage.tsx:61`, `web/src/features/upstreams/UpstreamsPage.tsx:43`, `web/src/features/rules/RulesPage.tsx:34`, and `web/src/ui/useCrudForm.ts:49` (which carries the confirm for ZonesTab and RecordsTab). `useCrudForm`'s `CrudFormSpec.confirmDelete` keeps its signature; the hook returns the dialog state instead of calling `window.confirm`. The tests that stub confirm today (`web/src/features/upstreams/UpstreamsPage.test.tsx:140,144,188`, `web/src/ui/useCrudForm.test.tsx:102`) are rewritten to drive the real dialog: open, assert `role="alertdialog"` and the entity name in the message, confirm and cancel paths both asserted. BlocklistsPage and RulesPage gain the same coverage for their delete paths.
|
||||
|
||||
### 6. Styling mechanics
|
||||
|
||||
`stylex.create`/`stylex.props` only. Conditions are boolean-guarded style
|
||||
objects; variants are separate named styles; responsive behavior (the
|
||||
`md:` sidebar grid in `web/src/shell/AppShell.tsx:90-98`, the mobile drawer)
|
||||
becomes `@media` conditions inside style values. No dynamic class-name
|
||||
strings, no `clsx`-style helpers, no `className` literals left anywhere in
|
||||
`web/src` after S3 — `className`/`style` attributes appear only as the spread
|
||||
of `stylex.props(...)` (RAC render-prop `className` functions included).
|
||||
TanStack `Link` `activeProps`/`inactiveProps` (`AppShell.tsx:35-42`) pass
|
||||
`stylex.props(...).className` outputs.
|
||||
`stylex.create`/`stylex.props` only. Conditions are boolean-guarded style objects; variants are separate named styles; responsive behavior (the `md:` sidebar grid in `web/src/shell/AppShell.tsx:90-98`, the mobile drawer) becomes `@media` conditions inside style values. No dynamic class-name strings, no `clsx`-style helpers, no `className` literals left anywhere in `web/src` after S3 — `className`/`style` attributes appear only as the spread of `stylex.props(...)` (RAC render-prop `className` functions included). TanStack `Link` `activeProps`/`inactiveProps` (`AppShell.tsx:35-42`) pass `stylex.props(...).className` outputs.
|
||||
|
||||
### 7. Visual contract: drift accepted, structure pinned
|
||||
|
||||
No pixel parity requirement and no visual-regression harness. Pinned instead,
|
||||
checkable by reading the converted file against the original: same layout
|
||||
structure (grid/flex relationships, table columns, heading levels), token
|
||||
roles per ruling 2 (zinc surfaces, blue primary actions, red danger), the
|
||||
focus floor of ruling 3 on every interactive element, dark mode via media
|
||||
query, `overflow-x-auto` table wrappers preserved.
|
||||
No pixel parity requirement and no visual-regression harness. Pinned instead, checkable by reading the converted file against the original: same layout structure (grid/flex relationships, table columns, heading levels), token roles per ruling 2 (zinc surfaces, blue primary actions, red danger), the focus floor of ruling 3 on every interactive element, dark mode via media query, `overflow-x-auto` table wrappers preserved.
|
||||
|
||||
### 8. License inventory, named per step
|
||||
|
||||
- S1 adds `@stylexjs/stylex` (runtime dependency; `stylex.props` ships in the
|
||||
bundle) to the runtime closure and the bundled set. `@stylexjs/unplugin` is
|
||||
a devDependency — runtime closure and bundled set untouched by it.
|
||||
- S2 adds `react-aria-components` and its transitive closure
|
||||
(`@react-aria/*`, `@react-stately/*`, `@react-types/*`,
|
||||
`@internationalized/*`, `@swc/helpers` — the exact bundled subset is
|
||||
whatever `npm run assert-bundled` reports after S2's conversions; every
|
||||
reported name gets a reviewed `licenses/inventory.zon` entry before the
|
||||
recorded list moves). This is the milestone's largest review surface.
|
||||
- S3 removes nothing from the license files: Tailwind was dev-only and never
|
||||
recorded. The `@tanstack/*` entries stay untouched.
|
||||
- S1 adds `@stylexjs/stylex` (runtime dependency; `stylex.props` ships in the bundle) to the runtime closure and the bundled set. `@stylexjs/unplugin` is a devDependency — runtime closure and bundled set untouched by it.
|
||||
- S2 adds `react-aria-components` and its transitive closure (`@react-aria/*`, `@react-stately/*`, `@react-types/*`, `@internationalized/*`, `@swc/helpers` — the exact bundled subset is whatever `npm run assert-bundled` reports after S2's conversions; every reported name gets a reviewed `licenses/inventory.zon` entry before the recorded list moves). This is the milestone's largest review surface.
|
||||
- S3 removes nothing from the license files: Tailwind was dev-only and never recorded. The `@tanstack/*` entries stay untouched.
|
||||
|
||||
### 9. PLAN.md wording moves in S3
|
||||
|
||||
`PLAN.md:138` "Vite + React + TypeScript + Tailwind; TanStack Router +
|
||||
TanStack Query" → "Vite + React + TypeScript + StyleX + React Aria; TanStack
|
||||
Router + TanStack Query". `PLAN.md:235` directory comment updates the same
|
||||
way. Line 562 names loaders and Query only — untouched. No other PLAN
|
||||
amendment.
|
||||
`PLAN.md:138` "Vite + React + TypeScript + Tailwind; TanStack Router + TanStack Query" → "Vite + React + TypeScript + StyleX + React Aria; TanStack Router + TanStack Query". `PLAN.md:235` directory comment updates the same way. Line 562 names loaders and Query only — untouched. No other PLAN amendment.
|
||||
|
||||
### 10. Conversion order is ascending `className` count, one commit per file
|
||||
|
||||
@@ -187,49 +91,27 @@ So a regression bisects to one page. Counts at spec time:
|
||||
| features/local/RecordsTab.tsx | 34 |
|
||||
| features/queries/QueryLogPage.tsx | 46 |
|
||||
|
||||
S2 takes its owned cluster out of this order (converted together because the
|
||||
RAC wrappers land there); S3 walks the remainder smallest-first.
|
||||
S2 takes its owned cluster out of this order (converted together because the RAC wrappers land there); S3 walks the remainder smallest-first.
|
||||
|
||||
### 11. Tests that must change, known in advance
|
||||
|
||||
RAC changes DOM and focus behavior; these edits are expected work, not
|
||||
regressions to paper over:
|
||||
- Dialog: `ClientsPage.test.tsx` (ClientEditDialog now portals into an RAC
|
||||
Modal; queries move to `within(screen.getByRole("dialog"))`).
|
||||
- Tabs: `LocalDnsPage.test.tsx` (tab switcher becomes `role="tablist"`/
|
||||
`role="tab"`; `getByRole("button", { name: ... })` queries move to
|
||||
`getByRole("tab", ...)`).
|
||||
RAC changes DOM and focus behavior; these edits are expected work, not regressions to paper over:
|
||||
- Dialog: `ClientsPage.test.tsx` (ClientEditDialog now portals into an RAC Modal; queries move to `within(screen.getByRole("dialog"))`).
|
||||
- Tabs: `LocalDnsPage.test.tsx` (tab switcher becomes `role="tablist"`/ `role="tab"`; `getByRole("button", { name: ... })` queries move to `getByRole("tab", ...)`).
|
||||
- AlertDialog: `UpstreamsPage.test.tsx`, `useCrudForm.test.tsx` per ruling 5.
|
||||
- Select: RAC Select renders a button + listbox, not `<select>`, so
|
||||
`getByLabelText(...)` + `fireEvent.change` patterns break in the tests for
|
||||
the 7 Select files (LookupPage, QueryLogPage, LocalDnsPage/RecordsTab,
|
||||
RulesPage, SettingsPage, ClientsPage). They move to
|
||||
`userEvent`-style open-then-select interactions.
|
||||
All other tests query by role/text and are expected to survive; a test that
|
||||
asserted a Tailwind class is rewritten against role or accessible name.
|
||||
- Select: RAC Select renders a button + listbox, not `<select>`, so `getByLabelText(...)` + `fireEvent.change` patterns break in the tests for the 7 Select files (LookupPage, QueryLogPage, LocalDnsPage/RecordsTab, RulesPage, SettingsPage, ClientsPage). They move to `userEvent`-style open-then-select interactions. All other tests query by role/text and are expected to survive; a test that asserted a Tailwind class is rewritten against role or accessible name.
|
||||
|
||||
## Sessions
|
||||
|
||||
Three sessions, strictly sequential: S1 → S2 → S3. (S2 and S3 both write
|
||||
`web/package.json` and the license files; the RAC wrappers S3 consumes are
|
||||
built in S2.)
|
||||
Three sessions, strictly sequential: S1 → S2 → S3. (S2 and S3 both write `web/package.json` and the license files; the RAC wrappers S3 consumes are built in S2.)
|
||||
|
||||
### Session S1: StyleX foundation, proven end to end
|
||||
|
||||
Owns: `web/package.json`, `web/package-lock.json`, `web/vite.config.ts`,
|
||||
`web/src/styles.css`, `web/src/ui/tokens.stylex.ts` (new),
|
||||
`web/src/ui/styles.ts` (new), `web/src/ui/styles.test.tsx` (new),
|
||||
`licenses/inventory.zon`, `licenses/dependency-identity.txt`.
|
||||
Owns: `web/package.json`, `web/package-lock.json`, `web/vite.config.ts`, `web/src/styles.css`, `web/src/ui/tokens.stylex.ts` (new), `web/src/ui/styles.ts` (new), `web/src/ui/styles.test.tsx` (new), `licenses/inventory.zon`, `licenses/dependency-identity.txt`.
|
||||
|
||||
- S1.1 add `@stylexjs/stylex` (dependency) and `@stylexjs/unplugin`
|
||||
(devDependency), exact pins; wire `stylex.vite()` per ruling 1. Tailwind
|
||||
plugin stays.
|
||||
- S1.2 tokens per ruling 2; shared styles per ruling 3. `ui/classes.ts` is
|
||||
not deleted yet — both vocabularies coexist until S3.
|
||||
- S1.3 `styles.test.tsx`: render a probe component using
|
||||
`stylex.props(styles.button)` under vitest; assert a non-empty
|
||||
`className` lands on the element (proves the transform runs in the test
|
||||
pipeline, since `runtimeInjection` is off).
|
||||
- S1.1 add `@stylexjs/stylex` (dependency) and `@stylexjs/unplugin` (devDependency), exact pins; wire `stylex.vite()` per ruling 1. Tailwind plugin stays.
|
||||
- S1.2 tokens per ruling 2; shared styles per ruling 3. `ui/classes.ts` is not deleted yet — both vocabularies coexist until S3.
|
||||
- S1.3 `styles.test.tsx`: render a probe component using `stylex.props(styles.button)` under vitest; assert a non-empty `className` lands on the element (proves the transform runs in the test pipeline, since `runtimeInjection` is off).
|
||||
- S1.4 license entries per ruling 8.
|
||||
|
||||
Acceptance (S1):
|
||||
@@ -244,22 +126,10 @@ Acceptance (S1):
|
||||
|
||||
### Session S2: React Aria primitives and their cluster
|
||||
|
||||
Owns: `web/package.json`, `web/package-lock.json`, `web/src/ui/Dialog.tsx`
|
||||
(new), `web/src/ui/ConfirmDialog.tsx` (new), `web/src/ui/Tabs.tsx` (new),
|
||||
`web/src/ui/Select.tsx` (new), `web/src/ui/useCrudForm.ts`,
|
||||
`web/src/ui/useCrudForm.test.tsx`, `web/src/features/clients/*`,
|
||||
`web/src/features/local/*`, `web/src/features/upstreams/*`,
|
||||
`web/src/features/blocklists/BlocklistsPage.tsx` and its test,
|
||||
`web/src/features/rules/RulesPage.tsx` and its test,
|
||||
`licenses/inventory.zon`, `licenses/dependency-identity.txt`.
|
||||
Owns: `web/package.json`, `web/package-lock.json`, `web/src/ui/Dialog.tsx` (new), `web/src/ui/ConfirmDialog.tsx` (new), `web/src/ui/Tabs.tsx` (new), `web/src/ui/Select.tsx` (new), `web/src/ui/useCrudForm.ts`, `web/src/ui/useCrudForm.test.tsx`, `web/src/features/clients/*`, `web/src/features/local/*`, `web/src/features/upstreams/*`, `web/src/features/blocklists/BlocklistsPage.tsx` and its test, `web/src/features/rules/RulesPage.tsx` and its test, `licenses/inventory.zon`, `licenses/dependency-identity.txt`.
|
||||
|
||||
- S2.1 add `react-aria-components`, exact pin; build the four wrappers per
|
||||
ruling 4, styled with tokens and `ui/styles.ts`.
|
||||
- S2.2 convert the owned cluster fully to StyleX while adopting the wrappers:
|
||||
ClientEditDialog + PrefixesEditor + ClientsPage (Dialog, Select);
|
||||
LocalDnsPage + ZonesTab + RecordsTab (Tabs, Select, ConfirmDialog via
|
||||
useCrudForm); UpstreamsPage + UpstreamForm, BlocklistsPage, RulesPage
|
||||
(ConfirmDialog, Select). No `className` literal survives in an owned file.
|
||||
- S2.1 add `react-aria-components`, exact pin; build the four wrappers per ruling 4, styled with tokens and `ui/styles.ts`.
|
||||
- S2.2 convert the owned cluster fully to StyleX while adopting the wrappers: ClientEditDialog + PrefixesEditor + ClientsPage (Dialog, Select); LocalDnsPage + ZonesTab + RecordsTab (Tabs, Select, ConfirmDialog via useCrudForm); UpstreamsPage + UpstreamForm, BlocklistsPage, RulesPage (ConfirmDialog, Select). No `className` literal survives in an owned file.
|
||||
- S2.3 `window.confirm` removal and test rewrites per rulings 5 and 11.
|
||||
- S2.4 license review per ruling 8 — every package `assert-bundled` names.
|
||||
|
||||
@@ -279,23 +149,11 @@ Acceptance (S2):
|
||||
|
||||
### Session S3: the sweep and the removal
|
||||
|
||||
Owns: every `web/src` file not owned by S1/S2 that carries `className=`
|
||||
(the ruling-10 remainder: InlineError, ReadOnlyConfigBanner, HealthBanners,
|
||||
RestartBanner, StatCards, routes.tsx, LoginPage, DashboardPage,
|
||||
GroupSourcesEditor, GroupsPage, PauseWidget, BlocklistForm,
|
||||
SourceStatusSection, DiskCard, AppShell, LiveLogPage, TimeseriesChart,
|
||||
UpstreamHealthTable, LookupPage, QueryLogPage, SettingsPage) plus their
|
||||
tests, `web/src/ui/classes.ts` (delete), `web/package.json`,
|
||||
`web/package-lock.json`, `web/vite.config.ts`, `web/src/styles.css`,
|
||||
`PLAN.md`.
|
||||
Owns: every `web/src` file not owned by S1/S2 that carries `className=` (the ruling-10 remainder: InlineError, ReadOnlyConfigBanner, HealthBanners, RestartBanner, StatCards, routes.tsx, LoginPage, DashboardPage, GroupSourcesEditor, GroupsPage, PauseWidget, BlocklistForm, SourceStatusSection, DiskCard, AppShell, LiveLogPage, TimeseriesChart, UpstreamHealthTable, LookupPage, QueryLogPage, SettingsPage) plus their tests, `web/src/ui/classes.ts` (delete), `web/package.json`, `web/package-lock.json`, `web/vite.config.ts`, `web/src/styles.css`, `PLAN.md`.
|
||||
|
||||
- S3.1 convert the remainder smallest-first per ruling 10, one commit per
|
||||
file, gates green at every commit.
|
||||
- S3.1 convert the remainder smallest-first per ruling 10, one commit per file, gates green at every commit.
|
||||
- S3.2 delete `ui/classes.ts` when its import count reaches zero.
|
||||
- S3.3 remove Tailwind: drop `tailwindcss` and `@tailwindcss/vite` from
|
||||
`package.json`, the plugin from `vite.config.ts:3,8`, and the
|
||||
`@import "tailwindcss"` line from `styles.css` (the file itself stays as
|
||||
the StyleX CSS entry per ruling 1).
|
||||
- S3.3 remove Tailwind: drop `tailwindcss` and `@tailwindcss/vite` from `package.json`, the plugin from `vite.config.ts:3,8`, and the `@import "tailwindcss"` line from `styles.css` (the file itself stays as the StyleX CSS entry per ruling 1).
|
||||
- S3.4 PLAN.md per ruling 9.
|
||||
|
||||
Acceptance (S3):
|
||||
@@ -316,11 +174,7 @@ Acceptance (S3):
|
||||
|
||||
### Orchestrator
|
||||
|
||||
Verify S1 acceptance (the ruling-1 gate) before S2 starts; S2 before S3.
|
||||
After S3: full gate set, then a live smoke — `zig build` a server with the
|
||||
fresh `web/dist`, click through every page in light and dark scheme, exercise
|
||||
one delete confirm and the local-dns tabs. Record deviations in
|
||||
`## Recorded (implementation)`.
|
||||
Verify S1 acceptance (the ruling-1 gate) before S2 starts; S2 before S3. After S3: full gate set, then a live smoke — `zig build` a server with the fresh `web/dist`, click through every page in light and dark scheme, exercise one delete confirm and the local-dns tabs. Record deviations in `## Recorded (implementation)`.
|
||||
|
||||
## Module layout
|
||||
|
||||
@@ -328,12 +182,9 @@ New files:
|
||||
- `web/src/ui/tokens.stylex.ts` — the design tokens (ruling 2).
|
||||
- `web/src/ui/styles.ts` — the shared style vocabulary (ruling 3).
|
||||
- `web/src/ui/styles.test.tsx` — the transform-pipeline probe (S1.3).
|
||||
- `web/src/ui/Dialog.tsx`, `web/src/ui/ConfirmDialog.tsx`,
|
||||
`web/src/ui/Tabs.tsx`, `web/src/ui/Select.tsx` — the RAC wrappers
|
||||
(ruling 4).
|
||||
- `web/src/ui/Dialog.tsx`, `web/src/ui/ConfirmDialog.tsx`, `web/src/ui/Tabs.tsx`, `web/src/ui/Select.tsx` — the RAC wrappers (ruling 4).
|
||||
|
||||
Deleted surface: `web/src/ui/classes.ts`; the `window.confirm` calls; the
|
||||
Tailwind dependency pair and its vite plugin line and CSS import.
|
||||
Deleted surface: `web/src/ui/classes.ts`; the `window.confirm` calls; the Tailwind dependency pair and its vite plugin line and CSS import.
|
||||
|
||||
## Acceptance (milestone complete)
|
||||
|
||||
@@ -353,222 +204,93 @@ Tailwind dependency pair and its vite plugin line and CSS import.
|
||||
|
||||
## Recorded (implementation)
|
||||
|
||||
Written during S1 and the S2 hand-off. Each entry is a place the spec was wrong
|
||||
or a decision the spec did not have.
|
||||
Written during S1 and the S2 hand-off. Each entry is a place the spec was wrong or a decision the spec did not have.
|
||||
|
||||
### S1 could not put StyleX in the bundle, and its acceptance says otherwise
|
||||
|
||||
S1 owns no page file, so no production module imports StyleX and rollup emits
|
||||
none of it. Two S1 boxes — "`npm run build` emits StyleX CSS into the `web/dist`
|
||||
assets" and "`assert-bundled` passes with `@stylexjs/stylex` in the recorded
|
||||
set" — are therefore unmeetable inside S1 as scoped, and the recorded bundled
|
||||
set is unchanged at the S1 commit. Both were proven instead against a temporary
|
||||
probe component, then deferred to S2, which converts real pages and does change
|
||||
the bundled set. Ruling 8's "S1 adds `@stylexjs/stylex` ... to the runtime
|
||||
closure and the bundled set" is half right: the closure moves in S1, the bundled
|
||||
set in S2. The inventory entry for StyleX is anticipatory at the S1 commit,
|
||||
which is deliberate — see the styleq entry for why it cannot wait.
|
||||
S1 owns no page file, so no production module imports StyleX and rollup emits none of it. Two S1 boxes — "`npm run build` emits StyleX CSS into the `web/dist` assets" and "`assert-bundled` passes with `@stylexjs/stylex` in the recorded set" — are therefore unmeetable inside S1 as scoped, and the recorded bundled set is unchanged at the S1 commit. Both were proven instead against a temporary probe component, then deferred to S2, which converts real pages and does change the bundled set. Ruling 8's "S1 adds `@stylexjs/stylex` ... to the runtime closure and the bundled set" is half right: the closure moves in S1, the bundled set in S2. The inventory entry for StyleX is anticipatory at the S1 commit, which is deliberate — see the styleq entry for why it cannot wait.
|
||||
|
||||
### The probe results, recorded rather than left as a claim
|
||||
|
||||
Against the temporary probe: the built CSS carries `@layer priority1..N` and the
|
||||
token variables, resolving to `#fafafa` light and `#09090b` dark with
|
||||
lightningcss `lab()` fallbacks. The dev server needs no wiring of its own — the
|
||||
plugin injects `<link rel="stylesheet" href="/virtual:stylex.css">` into
|
||||
`index.html`, so the `DevStyleXInject` component in StyleX's Vite guide is for
|
||||
frameworks that bypass Vite's HTML transform and is not used here. Editing a
|
||||
token regenerates the virtual stylesheet and the dev server issues a page
|
||||
reload.
|
||||
Against the temporary probe: the built CSS carries `@layer priority1..N` and the token variables, resolving to `#fafafa` light and `#09090b` dark with lightningcss `lab()` fallbacks. The dev server needs no wiring of its own — the plugin injects `<link rel="stylesheet" href="/virtual:stylex.css">` into `index.html`, so the `DevStyleXInject` component in StyleX's Vite guide is for frameworks that bypass Vite's HTML transform and is not used here. Editing a token regenerates the virtual stylesheet and the dev server issues a page reload.
|
||||
|
||||
### S1 wrote outside its owned files, and outside an anti-requirement
|
||||
|
||||
The licence guard forced it: `licenses/stylex-mit.txt`, `licenses/styleq-mit.txt`
|
||||
and their registration in `licenses/licenses.zig`, plus the `npm_not_shipped`
|
||||
array in `src/licenses_drift_test.zig`. The last contradicts the "no edits ...
|
||||
under `src/`" anti-requirement below. The anti-requirement is amended, not
|
||||
ignored: `src/licenses_drift_test.zig` is writable for its licence-fact arrays
|
||||
and, from S2, for the per-package licence change described next. The rest of
|
||||
`src/` stays closed.
|
||||
The licence guard forced it: `licenses/stylex-mit.txt`, `licenses/styleq-mit.txt` and their registration in `licenses/licenses.zig`, plus the `npm_not_shipped` array in `src/licenses_drift_test.zig`. The last contradicts the "no edits ... under `src/`" anti-requirement below. The anti-requirement is amended, not ignored: `src/licenses_drift_test.zig` is writable for its licence-fact arrays and, from S2, for the per-package licence change described next. The rest of `src/` stays closed.
|
||||
|
||||
### Apache-2.0 inbound: a decision, not a deduction
|
||||
|
||||
`react-aria-components` 1.20.0 brings eight Apache-2.0 packages
|
||||
(react-aria-components, react-aria, react-stately, `@react-types/shared`, three
|
||||
`@internationalized/*`, `@swc/helpers`) and `tslib` under 0BSD into an EUPL-1.2
|
||||
project. The user accepted Apache-2.0 inbound on 2026-08-12 and authorised the
|
||||
guard change it needs. This is recorded as their decision because nothing in the
|
||||
licence settles it: EUPL-1.2's appendix does not list Apache-2.0 — it names GPL,
|
||||
AGPL, OSL, EPL, CeCILL, MPL, LGPL, CC BY-SA, EUPL and LiLiQ — and that appendix
|
||||
governs Article 5 outbound relicensing, not inbound consumption. Any claim that
|
||||
the appendix permits this is false and must not be written into the inventory.
|
||||
`src/licenses_drift_test.zig:122` hardcodes a single expected licence and
|
||||
becomes per-package in S2; the guard stays strict, an unexpected licence still
|
||||
fails.
|
||||
`react-aria-components` 1.20.0 brings eight Apache-2.0 packages (react-aria-components, react-aria, react-stately, `@react-types/shared`, three `@internationalized/*`, `@swc/helpers`) and `tslib` under 0BSD into an EUPL-1.2 project. The user accepted Apache-2.0 inbound on 2026-08-12 and authorised the guard change it needs. This is recorded as their decision because nothing in the licence settles it: EUPL-1.2's appendix does not list Apache-2.0 — it names GPL, AGPL, OSL, EPL, CeCILL, MPL, LGPL, CC BY-SA, EUPL and LiLiQ — and that appendix governs Article 5 outbound relicensing, not inbound consumption. Any claim that the appendix permits this is false and must not be written into the inventory. `src/licenses_drift_test.zig:122` hardcodes a single expected licence and becomes per-package in S2; the guard stays strict, an unexpected licence still fails.
|
||||
|
||||
### Ruling 8's picture of the React Aria closure is wrong
|
||||
|
||||
There are no `@react-aria/*` or `@react-stately/*` scoped packages. RAC 1.20.0
|
||||
ships consolidated `react-aria` and `react-stately`. The runtime closure it adds
|
||||
is the eleven packages named above plus `aria-hidden`, `clsx` and `client-only`
|
||||
under MIT — not the "dozens" the ruling anticipated.
|
||||
There are no `@react-aria/*` or `@react-stately/*` scoped packages. RAC 1.20.0 ships consolidated `react-aria` and `react-stately`. The runtime closure it adds is the eleven packages named above plus `aria-hidden`, `clsx` and `client-only` under MIT — not the "dozens" the ruling anticipated.
|
||||
|
||||
### Ruling 11's claim about the tab tests is wrong
|
||||
|
||||
`LocalDnsPage.tsx` already renders `role="tablist"` and `role="tab"` by hand, so
|
||||
no test query moves from `getByRole("button")` to `getByRole("tab")`. What RAC
|
||||
adds there is keyboard arrow-key navigation, which the hand-rolled switcher
|
||||
lacks, and that is what the S2 acceptance box asserts.
|
||||
`LocalDnsPage.tsx` already renders `role="tablist"` and `role="tab"` by hand, so no test query moves from `getByRole("button")` to `getByRole("tab")`. What RAC adds there is keyboard arrow-key navigation, which the hand-rolled switcher lacks, and that is what the S2 acceptance box asserts.
|
||||
|
||||
### `formCard` is a layout change, not a port
|
||||
|
||||
`space-y-3` set sibling margins on a block container; the StyleX version is a
|
||||
flex column with `gap`. StyleX cannot express the `> * + *` selector `space-y`
|
||||
compiles to, so `gap` is the only mechanism available. Both call sites
|
||||
(`RecordsTab`, `ZonesTab`) are plain vertical form stacks, where the difference
|
||||
in child sizing and margin behaviour does not show. Recorded against ruling 7,
|
||||
which pins layout structure.
|
||||
`space-y-3` set sibling margins on a block container; the StyleX version is a flex column with `gap`. StyleX cannot express the `> * + *` selector `space-y` compiles to, so `gap` is the only mechanism available. Both call sites (`RecordsTab`, `ZonesTab`) are plain vertical form stacks, where the difference in child sizing and margin behaviour does not show. Recorded against ruling 7, which pins layout structure.
|
||||
|
||||
### S2 needed two build-config changes the spec did not anticipate
|
||||
|
||||
Both are in files S1 and S3 own, both were forced, and neither is optional.
|
||||
|
||||
`web/vitest.setup.ts` (new) plus `setupFiles` in `web/vite.config.ts`: jsdom
|
||||
implements no `CSS` interface at all, and React Aria's collection code calls
|
||||
`CSS.escape` unguarded when it looks an item up by key
|
||||
(`react-aria/dist/private/selection/utils.mjs:22`). Every RAC Select and Tabs
|
||||
test threw `Cannot read properties of undefined (reading 'escape')` before this
|
||||
landed. The setup file implements CSSOM's *serialize an identifier* algorithm
|
||||
rather than approximating it, because a wrong escape would break the key lookups
|
||||
silently instead of loudly.
|
||||
`web/vitest.setup.ts` (new) plus `setupFiles` in `web/vite.config.ts`: jsdom implements no `CSS` interface at all, and React Aria's collection code calls `CSS.escape` unguarded when it looks an item up by key (`react-aria/dist/private/selection/utils.mjs:22`). Every RAC Select and Tabs test threw `Cannot read properties of undefined (reading 'escape')` before this landed. The setup file implements CSSOM's *serialize an identifier* algorithm rather than approximating it, because a wrong escape would break the key lookups silently instead of loudly.
|
||||
|
||||
The `@` alias passed to `stylex.vite()` in `web/vite.config.ts`: StyleX resolves
|
||||
the `.stylex.ts` theme file itself and cannot see Vite's `resolve.alias`, so
|
||||
`import { colors } from "@/ui/tokens.stylex"` in a feature file failed the build
|
||||
with "Could not resolve the path to the imported file". Relative imports were the
|
||||
alternative; passing the alias keeps the repo's `@/` convention instead.
|
||||
The `@` alias passed to `stylex.vite()` in `web/vite.config.ts`: StyleX resolves the `.stylex.ts` theme file itself and cannot see Vite's `resolve.alias`, so `import { colors } from "@/ui/tokens.stylex"` in a feature file failed the build with "Could not resolve the path to the imported file". Relative imports were the alternative; passing the alias keeps the repo's `@/` convention instead.
|
||||
|
||||
### RAC Select changes the accessible name of a labelled control
|
||||
|
||||
`useSelect` labels the trigger button with `aria-labelledby="<value> <label>"`,
|
||||
so a select labelled "Type" showing "A" has the accessible name "A Type", and
|
||||
`getByLabelText("Type")` no longer finds it. Converted tests query
|
||||
`getByRole("button", { name: /Type$/ })` and read the options by opening the
|
||||
listbox. RAC also renders a hidden native `<select>` for form submission; it sits
|
||||
inside an `aria-hidden` container, so role and label queries skip it.
|
||||
`useSelect` labels the trigger button with `aria-labelledby="<value> <label>"`, so a select labelled "Type" showing "A" has the accessible name "A Type", and `getByLabelText("Type")` no longer finds it. Converted tests query `getByRole("button", { name: /Type$/ })` and read the options by opening the listbox. RAC also renders a hidden native `<select>` for form submission; it sits inside an `aria-hidden` container, so role and label queries skip it.
|
||||
|
||||
### The RAC anti-requirement forbids parts of a component it requires
|
||||
|
||||
`Select.tsx` imports RAC `Button`, `Popover`, `Label`, `ListBox`, `ListBoxItem`
|
||||
and `SelectValue`. The anti-requirement below names `Button` and `Popover`
|
||||
among the components RAC must not bring. Those two are not adopted components
|
||||
here — RAC composes a Select out of exactly these parts, and there is no RAC
|
||||
Select without them. The anti-requirement is amended: it forbids adopting a RAC
|
||||
component as an app-wide primitive, and the internal parts of the four adopted
|
||||
components are permitted where no file outside their wrapper imports them.
|
||||
That last clause is the check that keeps the scope honest, and it holds:
|
||||
`react-aria-components` is imported only by the four `ui/` wrappers. A RAC
|
||||
`Button` in a feature file would still be a breach.
|
||||
`Select.tsx` imports RAC `Button`, `Popover`, `Label`, `ListBox`, `ListBoxItem` and `SelectValue`. The anti-requirement below names `Button` and `Popover` among the components RAC must not bring. Those two are not adopted components here — RAC composes a Select out of exactly these parts, and there is no RAC Select without them. The anti-requirement is amended: it forbids adopting a RAC component as an app-wide primitive, and the internal parts of the four adopted components are permitted where no file outside their wrapper imports them. That last clause is the check that keeps the scope honest, and it holds: `react-aria-components` is imported only by the four `ui/` wrappers. A RAC `Button` in a feature file would still be a breach.
|
||||
|
||||
### The focus-ring floor was briefly amended on a false premise
|
||||
|
||||
S2 left the Select's options with their outline removed, showing keyboard focus
|
||||
as a background inversion alone. This section previously amended ruling 3 to
|
||||
excuse that, arguing RAC drives option focus through an `isFocused` render prop
|
||||
so a `:focus-visible` rule could never match. That argument is wrong.
|
||||
`useSelectableItem` gives options a roving `tabIndex` and focuses the option's
|
||||
own DOM node (`react-aria/dist/private/selection/useSelectableItem.mjs:64`),
|
||||
which `ListBoxItem` spreads onto its element — so `:focus-visible` matches and
|
||||
the ring was simply missing. The amendment is withdrawn and ruling 3 stands
|
||||
unchanged. `Select.tsx` now composes the shared inset ring onto every option:
|
||||
inset because an option flush against a scrolling popover clips an outset
|
||||
outline, and recoloured to `primaryText` on the highlighted row because the
|
||||
focus token is the same blue that row paints behind it. The inversion stays as
|
||||
the listbox convention; it is no longer the only indicator.
|
||||
S2 left the Select's options with their outline removed, showing keyboard focus as a background inversion alone. This section previously amended ruling 3 to excuse that, arguing RAC drives option focus through an `isFocused` render prop so a `:focus-visible` rule could never match. That argument is wrong. `useSelectableItem` gives options a roving `tabIndex` and focuses the option's own DOM node (`react-aria/dist/private/selection/useSelectableItem.mjs:64`), which `ListBoxItem` spreads onto its element — so `:focus-visible` matches and the ring was simply missing. The amendment is withdrawn and ruling 3 stands unchanged. `Select.tsx` now composes the shared inset ring onto every option: inset because an option flush against a scrolling popover clips an outset outline, and recoloured to `primaryText` on the highlighted row because the focus token is the same blue that row paints behind it. The inversion stays as the listbox convention; it is no longer the only indicator.
|
||||
|
||||
### Two notes for S3
|
||||
|
||||
`textMuted` resolves to zinc-400 in dark mode. Several current `text-zinc-500`
|
||||
call sites carry no dark override and stay zinc-500, so the token cannot replace
|
||||
those without a shade shift — decide per call site rather than sweeping.
|
||||
`textMuted` resolves to zinc-400 in dark mode. Several current `text-zinc-500` call sites carry no dark override and stay zinc-500, so the token cannot replace those without a shade shift — decide per call site rather than sweeping.
|
||||
|
||||
Vitest began reporting "Tests closed successfully but something prevents Vite
|
||||
server from exiting" with the StyleX plugin in the pipeline. The suite passes
|
||||
and exits 0. It should not be allowed to become the accepted baseline.
|
||||
Vitest began reporting "Tests closed successfully but something prevents Vite server from exiting" with the StyleX plugin in the pipeline. The suite passes and exits 0. It should not be allowed to become the accepted baseline.
|
||||
|
||||
### Both S3 notes are now settled
|
||||
|
||||
**The vitest warning is fixed, and it was not our code.** Bisecting a trivial
|
||||
one-assertion test against a bare config showed `react()` exits clean and
|
||||
`stylex.vite()` does not. `@stylexjs/unplugin@0.19.0`'s
|
||||
`lib/es/vite.mjs:59` starts a 150ms `setInterval` in `configureServer` to poll
|
||||
its CSS store for HMR, and clears it from the `close` event of
|
||||
`server.httpServer`. A probe plugin confirmed vitest resolves `httpServer` to
|
||||
`null`, because it runs the dev server in middleware mode — so the timer is
|
||||
never cleared and holds the event loop open until vitest kills the worker on a
|
||||
ten-second timeout. `web/vite.config.ts` now strips that one hook when
|
||||
`process.env.VITEST` is set; the transform hook, which the tests do need, is
|
||||
untouched, and `ui/styles.test.tsx` still proves it runs. `npm test` reports 31
|
||||
files and 183 tests passed, with no warning, and exits 0.
|
||||
**The vitest warning is fixed, and it was not our code.** Bisecting a trivial one-assertion test against a bare config showed `react()` exits clean and `stylex.vite()` does not. `@stylexjs/unplugin@0.19.0`'s `lib/es/vite.mjs:59` starts a 150ms `setInterval` in `configureServer` to poll its CSS store for HMR, and clears it from the `close` event of `server.httpServer`. A probe plugin confirmed vitest resolves `httpServer` to `null`, because it runs the dev server in middleware mode — so the timer is never cleared and holds the event loop open until vitest kills the worker on a ten-second timeout. `web/vite.config.ts` now strips that one hook when `process.env.VITEST` is set; the transform hook, which the tests do need, is untouched, and `ui/styles.test.tsx` still proves it runs. `npm test` reports 31 files and 183 tests passed, with no warning, and exits 0.
|
||||
|
||||
**`textMuted` was swept; here is the per-call-site judgement.** Measuring the
|
||||
two shifts against their grounds settles it in opposite directions:
|
||||
**`textMuted` was swept; here is the per-call-site judgement.** Measuring the two shifts against their grounds settles it in opposite directions:
|
||||
|
||||
- 39 call sites were a flat `text-zinc-500` with no dark override. On the dark
|
||||
ground that measures **4.12:1**, which fails WCAG AA. The token's zinc-400
|
||||
dark value measures **7.56:1**. The sweep repaired a real accessibility
|
||||
defect there, so it stands, and the token comment now records why.
|
||||
- 4 call sites were `text-zinc-600 dark:text-zinc-400` — already adaptive. The
|
||||
sweep moved their light value to zinc-500, from **7.40:1** to **4.62:1**.
|
||||
That clears AA by a hair on small text: the query-log table header, the
|
||||
dashboard period button, the inactive nav item and the chart legend. It buys
|
||||
nothing, so a second token `textSecondary` (zinc-600 light, zinc-400 dark)
|
||||
restores the original pair at exactly those four.
|
||||
- 39 call sites were a flat `text-zinc-500` with no dark override. On the dark ground that measures **4.12:1**, which fails WCAG AA. The token's zinc-400 dark value measures **7.56:1**. The sweep repaired a real accessibility defect there, so it stands, and the token comment now records why.
|
||||
- 4 call sites were `text-zinc-600 dark:text-zinc-400` — already adaptive. The sweep moved their light value to zinc-500, from **7.40:1** to **4.62:1**. That clears AA by a hair on small text: the query-log table header, the dashboard period button, the inactive nav item and the chart legend. It buys nothing, so a second token `textSecondary` (zinc-600 light, zinc-400 dark) restores the original pair at exactly those four.
|
||||
|
||||
Contrast was computed from the tokens' own oklch values through linear sRGB to
|
||||
WCAG relative luminance, not read off a chart.
|
||||
Contrast was computed from the tokens' own oklch values through linear sRGB to WCAG relative luminance, not read off a chart.
|
||||
|
||||
### The reset's font stack is not Tailwind's, deliberately
|
||||
|
||||
Tailwind is uninstalled, so its default stack cannot be reproduced from the
|
||||
tree, and ruling 7 accepts the metric drift. One part is not drift: `Select.tsx`
|
||||
renders U+25BE for its chevron, and Segoe UI does not carry that glyph, so on
|
||||
Windows the stack has to reach a font that does. `"Segoe UI Symbol"` and
|
||||
`"Noto Sans Symbols 2"` are appended for that glyph. No emoji families are
|
||||
listed, because the app renders no emoji.
|
||||
Tailwind is uninstalled, so its default stack cannot be reproduced from the tree, and ruling 7 accepts the metric drift. One part is not drift: `Select.tsx` renders U+25BE for its chevron, and Segoe UI does not carry that glyph, so on Windows the stack has to reach a font that does. `"Segoe UI Symbol"` and `"Noto Sans Symbols 2"` are appended for that glyph. No emoji families are listed, because the app renders no emoji.
|
||||
|
||||
### Two review findings closed by decision, not by a change
|
||||
|
||||
Both were put to Mokhtar Mial on 2026-08-12 and both are closed. Neither is to
|
||||
be re-opened by a later review reading the history and inferring an oversight.
|
||||
Both were put to Mokhtar Mial on 2026-08-12 and both are closed. Neither is to be re-opened by a later review reading the history and inferring an oversight.
|
||||
|
||||
`0f58966` carries a `README.md` edit that was already in the working tree and
|
||||
is not part of the `srOnly` work its message describes. The content is intact
|
||||
and only the attribution is wrong; rewriting published history to correct that
|
||||
was declined.
|
||||
`0f58966` carries a `README.md` edit that was already in the working tree and is not part of the `srOnly` work its message describes. The content is intact and only the attribution is wrong; rewriting published history to correct that was declined.
|
||||
|
||||
Ruling 10 asks for one commit per file in ascending `className` count. S3 did
|
||||
not hold to the order strictly. Bisectability does not depend on it and the
|
||||
commits were already published, so the order stands as it is.
|
||||
Ruling 10 asks for one commit per file in ascending `className` count. S3 did not hold to the order strictly. Bisectability does not depend on it and the commits were already published, so the order stands as it is.
|
||||
|
||||
## Anti-requirements
|
||||
|
||||
- No router or query changes: TanStack Router and TanStack Query stay,
|
||||
untouched. That decision is closed.
|
||||
- No RAC beyond Dialog, AlertDialog, Tabs, Select — no RAC Button, TextField,
|
||||
Table, Menu, DatePicker adopted as an app primitive. Native elements styled
|
||||
with StyleX are the end state. The internal parts of the four adopted
|
||||
components (Button, Popover, Label, ListBox, ListBoxItem, SelectValue inside
|
||||
Select) are permitted, and only inside their own wrapper file — see the
|
||||
recorded amendment above.
|
||||
- No runtime style injection (`runtimeInjection: false` stays), no CSS-in-JS
|
||||
at runtime, no styled-components-like patterns.
|
||||
- No dynamic class-name assembly, no `clsx`/`classnames` dependency, no
|
||||
Tailwind-compatibility shim.
|
||||
- No theme toggle, no `data-theme`, no stored preference — media-query dark
|
||||
mode only, as today.
|
||||
- No redesign: layout structure is pinned by ruling 7. Drift in shades and
|
||||
spacing is accepted; new page structures are not.
|
||||
- No router or query changes: TanStack Router and TanStack Query stay, untouched. That decision is closed.
|
||||
- No RAC beyond Dialog, AlertDialog, Tabs, Select — no RAC Button, TextField, Table, Menu, DatePicker adopted as an app primitive. Native elements styled with StyleX are the end state. The internal parts of the four adopted components (Button, Popover, Label, ListBox, ListBoxItem, SelectValue inside Select) are permitted, and only inside their own wrapper file — see the recorded amendment above.
|
||||
- No runtime style injection (`runtimeInjection: false` stays), no CSS-in-JS at runtime, no styled-components-like patterns.
|
||||
- No dynamic class-name assembly, no `clsx`/`classnames` dependency, no Tailwind-compatibility shim.
|
||||
- No theme toggle, no `data-theme`, no stored preference — media-query dark mode only, as today.
|
||||
- No redesign: layout structure is pinned by ruling 7. Drift in shades and spacing is accepted; new page structures are not.
|
||||
- No visual-regression harness, no screenshot tooling.
|
||||
- No edits to `web/src/lib/types.ts`, `web/src/lib/contractSamples.gen.ts`,
|
||||
`web/src/lib/api.ts`, or anything under `src/` (the Zig tree).
|
||||
- No edits to `web/src/lib/types.ts`, `web/src/lib/contractSamples.gen.ts`, `web/src/lib/api.ts`, or anything under `src/` (the Zig tree).
|
||||
- No `vite-plugin-stylex` (unmaintained community plugin).
|
||||
|
||||
+78
-362
@@ -1,247 +1,93 @@
|
||||
# Milestone 24: `skipped_unsupported` is persisted, surfaced and explained
|
||||
|
||||
Goal: close the compile-pipeline silent drop that misleads the operator. `Counts.
|
||||
skipped_unsupported` (`src/filter/compiler.zig:34`) is counted on every compile
|
||||
(compiler.zig:92) and then discarded on the happy path — it reaches the
|
||||
compiled-file header and the `NoValidEntries` error text, but no database
|
||||
column, no API field and no UI cell. Its sibling `skipped_regex` reaches all
|
||||
three. A blocklist made almost entirely of cosmetic browser filters therefore
|
||||
compiles to almost nothing and looks, in the UI, exactly like a clean list.
|
||||
AGENTS.md forbids exactly this ("no silent drops"). The milestone persists the
|
||||
count, surfaces it everywhere `skipped_regex` is surfaced, and explains to the
|
||||
operator why the two skip counters are different facts.
|
||||
Goal: close the compile-pipeline silent drop that misleads the operator. `Counts. skipped_unsupported` (`src/filter/compiler.zig:34`) is counted on every compile (compiler.zig:92) and then discarded on the happy path — it reaches the compiled-file header and the `NoValidEntries` error text, but no database column, no API field and no UI cell. Its sibling `skipped_regex` reaches all three. A blocklist made almost entirely of cosmetic browser filters therefore compiles to almost nothing and looks, in the UI, exactly like a clean list. AGENTS.md forbids exactly this ("no silent drops"). The milestone persists the count, surfaces it everywhere `skipped_regex` is surfaced, and explains to the operator why the two skip counters are different facts.
|
||||
|
||||
Design written 2026-08-13 against HEAD `21571e4`, revised after a Codex review
|
||||
of the first draft.
|
||||
Design written 2026-08-13 against HEAD `21571e4`, revised after a Codex review of the first draft.
|
||||
|
||||
## Implementation contract (read first)
|
||||
|
||||
- Read `AGENTS.md`, then this spec whole, before session work starts.
|
||||
- **The v1 baseline is editable and there is no migration step.** As of commit
|
||||
`21571e4`, `src/storage/migrations.zig` holds exactly one step and
|
||||
`target_version` is 1 (migrations.zig:29-36). nxdns has zero installs; the
|
||||
baseline freezes at v0.1 (`config_schema.zig:6-12`, PLAN §3.7, §11.2). The
|
||||
new column is one line edited into `config_schema.ddl_v1` plus the identical
|
||||
line in PLAN §11.2, which is kept byte-identical to it. A diff that adds a
|
||||
`ddl_v2` or a second `Step` is wrong and must be reverted, not merged.
|
||||
- After the API shape change, regenerate the contract samples
|
||||
(`web/src/lib/contractSamples.gen.ts`; procedure in AGENTS.md — the
|
||||
`zig build test -Dintegration -Dcontract-samples-out=...` command, never a
|
||||
hand edit) and update `web/src/lib/types.ts` to match.
|
||||
- **The v1 baseline is editable and there is no migration step.** As of commit `21571e4`, `src/storage/migrations.zig` holds exactly one step and `target_version` is 1 (migrations.zig:29-36). nxdns has zero installs; the baseline freezes at v0.1 (`config_schema.zig:6-12`, PLAN §3.7, §11.2). The new column is one line edited into `config_schema.ddl_v1` plus the identical line in PLAN §11.2, which is kept byte-identical to it. A diff that adds a `ddl_v2` or a second `Step` is wrong and must be reverted, not merged.
|
||||
- After the API shape change, regenerate the contract samples (`web/src/lib/contractSamples.gen.ts`; procedure in AGENTS.md — the `zig build test -Dintegration -Dcontract-samples-out=...` command, never a hand edit) and update `web/src/lib/types.ts` to match.
|
||||
- No new `src/**.zig` files, so `src/tests.zig` and `build.zig` are untouched.
|
||||
|
||||
## Rulings (binding)
|
||||
|
||||
### 1. The column is `skipped_unsupported_count`, one edited line, no step
|
||||
|
||||
`skipped_regex_count` (config_schema.zig:63) sets the convention; the new
|
||||
column follows it. In `config_schema.ddl_v1`, directly after the
|
||||
`skipped_regex_count` line inside `CREATE TABLE blocklist_sources`:
|
||||
`skipped_regex_count` (config_schema.zig:63) sets the convention; the new column follows it. In `config_schema.ddl_v1`, directly after the `skipped_regex_count` line inside `CREATE TABLE blocklist_sources`:
|
||||
|
||||
```sql
|
||||
skipped_unsupported_count INTEGER NOT NULL DEFAULT 0,
|
||||
```
|
||||
|
||||
The identical line goes into PLAN §11.2 after PLAN.md:421 — that section is
|
||||
kept byte-identical to `ddl_v1` and currently is (verified: the fenced SQL
|
||||
matches the Zig string literal line for line).
|
||||
The identical line goes into PLAN §11.2 after PLAN.md:421 — that section is kept byte-identical to `ddl_v1` and currently is (verified: the fenced SQL matches the Zig string literal line for line).
|
||||
|
||||
Consequence to accept, not to fix: a development database stamped version 1
|
||||
before this edit will fail the first `SELECT` naming the column with "no such
|
||||
column", because the stamped version equals `target_version` and the step never
|
||||
reruns. That is the documented pre-v0.1 contract (`config_schema.zig:6-12`) —
|
||||
delete the scratch database. Do not add fallback SQL, `PRAGMA table_info`
|
||||
probing, or a migration step to paper over it.
|
||||
Consequence to accept, not to fix: a development database stamped version 1 before this edit will fail the first `SELECT` naming the column with "no such column", because the stamped version equals `target_version` and the step never reruns. That is the documented pre-v0.1 contract (`config_schema.zig:6-12`) — delete the scratch database. Do not add fallback SQL, `PRAGMA table_info` probing, or a migration step to paper over it.
|
||||
|
||||
The baseline test "a fresh database reaches the baseline with every v1 column
|
||||
and rule kind" (migrations.zig:266-283) gains
|
||||
`try testing.expect(try columnExists(&database, "blocklist_sources", "skipped_unsupported_count"));`
|
||||
beside the existing `exception_count` probe.
|
||||
The baseline test "a fresh database reaches the baseline with every v1 column and rule kind" (migrations.zig:266-283) gains `try testing.expect(try columnExists(&database, "blocklist_sources", "skipped_unsupported_count"));` beside the existing `exception_count` probe.
|
||||
|
||||
### 2. The repo persists it beside `skipped_regex_count`
|
||||
|
||||
`src/storage/repositories/sources_repo.zig`, mirroring `skipped_regex_count`
|
||||
exactly (no default on either struct field, so every construction site fails
|
||||
to compile until it names the count — that is the visibility this milestone is
|
||||
about):
|
||||
`src/storage/repositories/sources_repo.zig`, mirroring `skipped_regex_count` exactly (no default on either struct field, so every construction site fails to compile until it names the count — that is the visibility this milestone is about):
|
||||
|
||||
- `SourceRow` (sources_repo.zig:80-97) gains `skipped_unsupported_count: i64`
|
||||
after `skipped_regex_count`.
|
||||
- `SourceRow` (sources_repo.zig:80-97) gains `skipped_unsupported_count: i64` after `skipped_regex_count`.
|
||||
- `SourceStats` (sources_repo.zig:99-110) gains the same field.
|
||||
- `row_columns_sql` (sources_repo.zig:112-117) appends
|
||||
`skipped_unsupported_count` to the SELECT list (index 11);
|
||||
`readSourceRow` (sources_repo.zig:128-148) reads it with
|
||||
`stmt.columnInt(11)`.
|
||||
- `update_stats_sql` (sources_repo.zig:159-164) adds
|
||||
`skipped_unsupported_count = ?8`; `updateSourceStats`
|
||||
(sources_repo.zig:168-179) binds it.
|
||||
- The module doc comment (sources_repo.zig:3-5) adds the column to its list of
|
||||
server-produced facts, and so does the `src/config/model.zig:14` doc comment
|
||||
that repeats that list.
|
||||
- `row_columns_sql` (sources_repo.zig:112-117) appends `skipped_unsupported_count` to the SELECT list (index 11); `readSourceRow` (sources_repo.zig:128-148) reads it with `stmt.columnInt(11)`.
|
||||
- `update_stats_sql` (sources_repo.zig:159-164) adds `skipped_unsupported_count = ?8`; `updateSourceStats` (sources_repo.zig:168-179) binds it.
|
||||
- The module doc comment (sources_repo.zig:3-5) adds the column to its list of server-produced facts, and so does the `src/config/model.zig:14` doc comment that repeats that list.
|
||||
|
||||
**The sum at sources_repo.zig:318 includes the new column.** That sum lives in
|
||||
the test "insertBlocklistSource leaves the runtime columns at their defaults";
|
||||
it exists to prove an insert leaves every runtime counter at 0, and the new
|
||||
column is a runtime counter. This is a deliberate decision, not an oversight:
|
||||
no production query sums these columns into a "total entries" figure, and none
|
||||
may start to — `skipped_regex_count` and `skipped_unsupported_count` count
|
||||
lines *not* written, unlike `domain_count`, `wildcard_count` and
|
||||
`exception_count`, so any future entries total must exclude both. The test sum
|
||||
asserts defaults, which is the one context where adding them is correct.
|
||||
**The sum at sources_repo.zig:318 includes the new column.** That sum lives in the test "insertBlocklistSource leaves the runtime columns at their defaults"; it exists to prove an insert leaves every runtime counter at 0, and the new column is a runtime counter. This is a deliberate decision, not an oversight: no production query sums these columns into a "total entries" figure, and none may start to — `skipped_regex_count` and `skipped_unsupported_count` count lines *not* written, unlike `domain_count`, `wildcard_count` and `exception_count`, so any future entries total must exclude both. The test sum asserts defaults, which is the one context where adding them is correct.
|
||||
|
||||
Existing `SourceStats` / assertion sites in this file (the literals at
|
||||
sources_repo.zig:386-393, 420-427, 483-493 and the zero-default loop at
|
||||
sources_repo.zig:365-374) carry the new field with distinct non-zero values
|
||||
where their siblings have them, and the round-trip assertions extend to it.
|
||||
Existing `SourceStats` / assertion sites in this file (the literals at sources_repo.zig:386-393, 420-427, 483-493 and the zero-default loop at sources_repo.zig:365-374) carry the new field with distinct non-zero values where their siblings have them, and the round-trip assertions extend to it.
|
||||
|
||||
### 3. The manager writes it, and rehydration restores it
|
||||
|
||||
`src/filter/manager.zig`. The header already prints
|
||||
`# skipped_unsupported {d}` (manager.zig:243) and `SourceStatus.counts` is a
|
||||
full `compiler.Counts` (manager.zig:178), so within one process the count
|
||||
already reaches the status table. What is missing is the database, which is
|
||||
the only thing a restart reads — rehydration never reparses headers.
|
||||
`src/filter/manager.zig`. The header already prints `# skipped_unsupported {d}` (manager.zig:243) and `SourceStatus.counts` is a full `compiler.Counts` (manager.zig:178), so within one process the count already reaches the status table. What is missing is the database, which is the only thing a restart reads — rehydration never reparses headers.
|
||||
|
||||
- The fresh-publish path (manager.zig:854-861) passes
|
||||
`.skipped_unsupported_count = compiled.result.counts.skipped_unsupported` to
|
||||
`updateSourceStats`.
|
||||
- **The checksum-unchanged path (manager.zig:821-836) carries a bug this
|
||||
milestone fixes.** It writes the *stored* `row.skipped_regex_count` back to
|
||||
the database (manager.zig:830) while handing the *fresh*
|
||||
`compiled.result.counts` to the in-memory status (manager.zig:833). The
|
||||
checksum covers the `.list`, `.wild` and `.allow` bodies only, and a skipped
|
||||
line lands in none of them — so a list that changes only its regex or
|
||||
browser-syntax lines keeps its checksum, the running server shows the new
|
||||
number, the database keeps the old one, and the next restart silently reverts
|
||||
what the operator saw. That is milestone 21's column, wrong today.
|
||||
- The fresh-publish path (manager.zig:854-861) passes `.skipped_unsupported_count = compiled.result.counts.skipped_unsupported` to `updateSourceStats`.
|
||||
- **The checksum-unchanged path (manager.zig:821-836) carries a bug this milestone fixes.** It writes the *stored* `row.skipped_regex_count` back to the database (manager.zig:830) while handing the *fresh* `compiled.result.counts` to the in-memory status (manager.zig:833). The checksum covers the `.list`, `.wild` and `.allow` bodies only, and a skipped line lands in none of them — so a list that changes only its regex or browser-syntax lines keeps its checksum, the running server shows the new number, the database keeps the old one, and the next restart silently reverts what the operator saw. That is milestone 21's column, wrong today.
|
||||
|
||||
Both skip counters take `compiled.result.counts` on this path:
|
||||
`.skipped_regex_count = compiled.result.counts.skipped_regex` and
|
||||
`.skipped_unsupported_count = compiled.result.counts.skipped_unsupported`.
|
||||
~~`domain_count`, `wildcard_count` and `exception_count` keep reading from
|
||||
`row` — they count written entries, so an unchanged checksum does mean an
|
||||
unchanged value for them.~~ **Corrected post-commit:** that claim was false.
|
||||
The unframed digest could not distinguish an entry in `.list` from the same
|
||||
entry in `.wild`, so an unchanged checksum did not vouch for the entry
|
||||
counts either. The addendum below frames the hash and makes all five stat
|
||||
fields read fresh on this path.
|
||||
Both skip counters take `compiled.result.counts` on this path: `.skipped_regex_count = compiled.result.counts.skipped_regex` and `.skipped_unsupported_count = compiled.result.counts.skipped_unsupported`. ~~`domain_count`, `wildcard_count` and `exception_count` keep reading from `row` — they count written entries, so an unchanged checksum does mean an unchanged value for them.~~ **Corrected post-commit:** that claim was false. The unframed digest could not distinguish an entry in `.list` from the same entry in `.wild`, so an unchanged checksum did not vouch for the entry counts either. The addendum below frames the hash and makes all five stat fields read fresh on this path.
|
||||
|
||||
This needs a test the author has watched fail with the fix reverted: two
|
||||
compiles of bodies whose written entries are identical but whose skipped
|
||||
lines differ, asserting the same checksum, then asserting the database holds
|
||||
the second compile's counts, then rehydrating through `applyLoadOutcomes` to
|
||||
prove the restored status carries them. Report the observed failure output.
|
||||
- `applyLoadOutcomes` rehydration (manager.zig:1545-1550) adds
|
||||
`.skipped_unsupported = countOf(row.skipped_unsupported_count)` to the
|
||||
`Counts` it rebuilds, so a restarted server reports what the last compile
|
||||
skipped instead of 0.
|
||||
- Test literals name the field: one `SourceStats` (manager.zig:1928) and two
|
||||
`SourceRow` (manager.zig:2051, 2225). The rehydration test that feeds a row
|
||||
through
|
||||
`applyLoadOutcomes` asserts a non-zero `skipped_unsupported` lands in
|
||||
`status.counts`. The header pin test (manager.zig:2030-2031) already covers
|
||||
the `# skipped_unsupported` line and is extended only if its fixture counts
|
||||
change.
|
||||
- `rejectedWithoutEntries` (manager.zig:1640-1643) and `failNoValidEntries`
|
||||
(manager.zig:1127-1128) already read the count and are unchanged.
|
||||
This needs a test the author has watched fail with the fix reverted: two compiles of bodies whose written entries are identical but whose skipped lines differ, asserting the same checksum, then asserting the database holds the second compile's counts, then rehydrating through `applyLoadOutcomes` to prove the restored status carries them. Report the observed failure output.
|
||||
- `applyLoadOutcomes` rehydration (manager.zig:1545-1550) adds `.skipped_unsupported = countOf(row.skipped_unsupported_count)` to the `Counts` it rebuilds, so a restarted server reports what the last compile skipped instead of 0.
|
||||
- Test literals name the field: one `SourceStats` (manager.zig:1928) and two `SourceRow` (manager.zig:2051, 2225). The rehydration test that feeds a row through `applyLoadOutcomes` asserts a non-zero `skipped_unsupported` lands in `status.counts`. The header pin test (manager.zig:2030-2031) already covers the `# skipped_unsupported` line and is extended only if its fixture counts change.
|
||||
- `rejectedWithoutEntries` (manager.zig:1640-1643) and `failNoValidEntries` (manager.zig:1127-1128) already read the count and are unchanged.
|
||||
|
||||
`src/filter/filter_integration_test.zig` names `.skipped_regex_count` in six
|
||||
`SourceStats` literals (:826, :858, :1075, :1145, :1257, :2158) — each gains
|
||||
the sibling field. The stats round-trip assertion at :915 gains a sibling
|
||||
assertion for the new column.
|
||||
`src/filter/filter_integration_test.zig` names `.skipped_regex_count` in six `SourceStats` literals (:826, :858, :1075, :1145, :1257, :2158) — each gains the sibling field. The stats round-trip assertion at :915 gains a sibling assertion for the new column.
|
||||
|
||||
**Do not add an unsupported line to the existing fixture.** The fixture at
|
||||
filter_integration_test.zig:112 is hosts-shaped, and `detectFormat`
|
||||
(`src/filter/parsers.zig:118`) assigns one format to a whole source. A single
|
||||
`##.ad-banner` or `$`-modifier line flips it to ABP, which re-parses every
|
||||
existing line: the `*.wild` entry becomes unsupported and address fields
|
||||
tokenize as domains (`src/filter/parser_abp.zig:68`). Every expected count in
|
||||
that test would move, for a reason unrelated to this milestone.
|
||||
**Do not add an unsupported line to the existing fixture.** The fixture at filter_integration_test.zig:112 is hosts-shaped, and `detectFormat` (`src/filter/parsers.zig:118`) assigns one format to a whole source. A single `##.ad-banner` or `$`-modifier line flips it to ABP, which re-parses every existing line: the `*.wild` entry becomes unsupported and address fields tokenize as domains (`src/filter/parser_abp.zig:68`). Every expected count in that test would move, for a reason unrelated to this milestone.
|
||||
|
||||
Add a **separate ABP-format fixture and test** carrying `##.ad-banner` and
|
||||
`||ads.example^$third-party` beside two blockable names, and assert
|
||||
`skipped_unsupported_count = 2` through the compile-persist-read round trip
|
||||
there.
|
||||
Add a **separate ABP-format fixture and test** carrying `##.ad-banner` and `||ads.example^$third-party` beside two blockable names, and assert `skipped_unsupported_count = 2` through the compile-persist-read round trip there.
|
||||
|
||||
`src/config/reconcile.zig`: the runtime-column preservation test seeds stats
|
||||
at reconcile.zig:1096-1110 and asserts survival at reconcile.zig:1173. The
|
||||
seed gains `.skipped_unsupported_count` with a distinct value and the
|
||||
assertion block gains its expectation. No reconcile logic changes: the engine
|
||||
updates declarative columns by name and never touches runtime columns, so the
|
||||
new column survives with zero code change — the test is the proof.
|
||||
`src/config/reconcile.zig`: the runtime-column preservation test seeds stats at reconcile.zig:1096-1110 and asserts survival at reconcile.zig:1173. The seed gains `.skipped_unsupported_count` with a distinct value and the assertion block gains its expectation. No reconcile logic changes: the engine updates declarative columns by name and never touches runtime columns, so the new column survives with zero code change — the test is the proof.
|
||||
|
||||
### 4. The API speaks it in both shapes
|
||||
|
||||
Two shapes carry blocklist counters, and the new count joins both under the
|
||||
names its siblings set:
|
||||
Two shapes carry blocklist counters, and the new count joins both under the names its siblings set:
|
||||
|
||||
- **`Blocklist`** (rows of `GET /api/blocklists`, serialized straight from
|
||||
`SourceRow`): the field arrives automatically once `SourceRow` has it, as
|
||||
`skipped_unsupported_count`. `src/web/openapi.yaml:1877-1895` adds it to
|
||||
`required` and `properties`.
|
||||
- **`SourceStatus`** (rows of `POST /api/blocklists/update`): `StatusView`
|
||||
(`src/web/handlers/blocklists.zig:52-80`) gains
|
||||
`skipped_unsupported: u32` after `skipped_regex`, mapped from
|
||||
`status.counts.skipped_unsupported` in `from`.
|
||||
`src/web/openapi.yaml:1920-1938` adds it to `required` and `properties`.
|
||||
The handler test literals at blocklists.zig:323 (`SourceStats`) and :386
|
||||
(`counts`) carry the field with non-zero values and the response assertions
|
||||
extend to it.
|
||||
- **`Blocklist`** (rows of `GET /api/blocklists`, serialized straight from `SourceRow`): the field arrives automatically once `SourceRow` has it, as `skipped_unsupported_count`. `src/web/openapi.yaml:1877-1895` adds it to `required` and `properties`.
|
||||
- **`SourceStatus`** (rows of `POST /api/blocklists/update`): `StatusView` (`src/web/handlers/blocklists.zig:52-80`) gains `skipped_unsupported: u32` after `skipped_regex`, mapped from `status.counts.skipped_unsupported` in `from`. `src/web/openapi.yaml:1920-1938` adds it to `required` and `properties`. The handler test literals at blocklists.zig:323 (`SourceStats`) and :386 (`counts`) carry the field with non-zero values and the response assertions extend to it.
|
||||
|
||||
Contract fallout, all in the same session (ruling 6 explains why):
|
||||
|
||||
- Regenerate `web/src/lib/contractSamples.gen.ts` with the AGENTS.md command.
|
||||
- `web/src/lib/types.ts`: `Blocklist` gains
|
||||
`skipped_unsupported_count: number` (types.ts:154-166); `SourceStatus`
|
||||
gains `skipped_unsupported: number` (types.ts:183-195).
|
||||
- The web test mocks are **not** typed against these interfaces, so `tsc` will
|
||||
not force them. `BLOCKLISTS` in
|
||||
`web/src/features/blocklists/BlocklistsPage.test.tsx` is an inferred object
|
||||
literal handed to a `Record<string, unknown>` (BlocklistsPage.test.tsx:40),
|
||||
and the same untyped-fetch pattern holds in
|
||||
`web/src/features/groups/GroupsPage.test.tsx:16` and
|
||||
`web/src/features/settings/authority.test.tsx:64` — both of which already
|
||||
omit `exception_count` without failing. Updating a mock here is a semantic
|
||||
fixture change, not a typecheck fix, and the implementer must not expect a
|
||||
compiler error to point at them.
|
||||
- `web/src/lib/types.ts`: `Blocklist` gains `skipped_unsupported_count: number` (types.ts:154-166); `SourceStatus` gains `skipped_unsupported: number` (types.ts:183-195).
|
||||
- The web test mocks are **not** typed against these interfaces, so `tsc` will not force them. `BLOCKLISTS` in `web/src/features/blocklists/BlocklistsPage.test.tsx` is an inferred object literal handed to a `Record<string, unknown>` (BlocklistsPage.test.tsx:40), and the same untyped-fetch pattern holds in `web/src/features/groups/GroupsPage.test.tsx:16` and `web/src/features/settings/authority.test.tsx:64` — both of which already omit `exception_count` without failing. Updating a mock here is a semantic fixture change, not a typecheck fix, and the implementer must not expect a compiler error to point at them.
|
||||
|
||||
So: the BlocklistsPage mocks (:21, :34 blocklist rows; :104, :155, :168
|
||||
status rows) gain the field because S2's rendering assertions read it. The
|
||||
groups and settings mocks are left alone — they render no counter column,
|
||||
and widening them buys nothing. S1 picks values no other cell in the same
|
||||
table already shows (`3` and `7` are taken), e.g.
|
||||
`skipped_unsupported_count: 21` and `skipped_unsupported: 17`.
|
||||
So: the BlocklistsPage mocks (:21, :34 blocklist rows; :104, :155, :168 status rows) gain the field because S2's rendering assertions read it. The groups and settings mocks are left alone — they render no counter column, and widening them buys nothing. S1 picks values no other cell in the same table already shows (`3` and `7` are taken), e.g. `skipped_unsupported_count: 21` and `skipped_unsupported: 17`.
|
||||
|
||||
### 5. The UI shows it always, in both tables, and says what it means
|
||||
|
||||
Both tables gain a `Skipped unsupported` column directly after
|
||||
`Skipped regex`, rendered unconditionally:
|
||||
Both tables gain a `Skipped unsupported` column directly after `Skipped regex`, rendered unconditionally:
|
||||
|
||||
- `web/src/features/blocklists/BlocklistsPage.tsx`: header after :157, cell
|
||||
`{b.skipped_unsupported_count}` after :190, same
|
||||
`shared.td, shared.tabularNums` props as its neighbours.
|
||||
- `web/src/features/blocklists/SourceStatusSection.tsx`: header after :91,
|
||||
cell `{source.skipped_unsupported}` after :114.
|
||||
- `web/src/features/blocklists/BlocklistsPage.tsx`: header after :157, cell `{b.skipped_unsupported_count}` after :190, same `shared.td, shared.tabularNums` props as its neighbours.
|
||||
- `web/src/features/blocklists/SourceStatusSection.tsx`: header after :91, cell `{source.skipped_unsupported}` after :114.
|
||||
|
||||
Always-shown is a decision, not a default: every other counter column here is
|
||||
unconditional, including `Skipped regex` and `Exceptions`, which are 0 for
|
||||
every plain hosts list, and the tutorial already explains those zeros. A
|
||||
column that appears only when non-zero would make two lists' tables disagree
|
||||
in shape, would hide the header that gives the number its meaning, and would
|
||||
special-case exactly the counter this milestone exists to make visible. The
|
||||
zero cell is not noise; it states that nothing in this list was classified as
|
||||
unsupported — which is narrower than "clean", since `invalid` and `long_lines`
|
||||
stay unsurfaced (ruling 7).
|
||||
Always-shown is a decision, not a default: every other counter column here is unconditional, including `Skipped regex` and `Exceptions`, which are 0 for every plain hosts list, and the tutorial already explains those zeros. A column that appears only when non-zero would make two lists' tables disagree in shape, would hide the header that gives the number its meaning, and would special-case exactly the counter this milestone exists to make visible. The zero cell is not noise; it states that nothing in this list was classified as unsupported — which is narrower than "clean", since `invalid` and `long_lines` stay unsurfaced (ruling 7).
|
||||
|
||||
The two counters mean different things and the page must say so once. A muted
|
||||
paragraph (the existing `styles.empty`-style muted text, matching
|
||||
`SourceStatusSection`'s `note` treatment) rendered under the sources table in
|
||||
`BlocklistsPage.tsx`, inside the same `else` branch as the table — the note
|
||||
describes the two skip columns, so it appears exactly when they do. "Always
|
||||
visible" above means unconditional on values, never hidden at 0; it does not
|
||||
mean the empty state (`blocklists.length === 0`) carries a paragraph about
|
||||
columns that are not on screen. The empty state stays one instruction. Exact
|
||||
copy:
|
||||
The two counters mean different things and the page must say so once. A muted paragraph (the existing `styles.empty`-style muted text, matching `SourceStatusSection`'s `note` treatment) rendered under the sources table in `BlocklistsPage.tsx`, inside the same `else` branch as the table — the note describes the two skip columns, so it appears exactly when they do. "Always visible" above means unconditional on values, never hidden at 0; it does not mean the empty state (`blocklists.length === 0`) carries a paragraph about columns that are not on screen. The empty state stays one instruction. Exact copy:
|
||||
|
||||
> Both “Skipped” columns count lines nxdns read and did not take. Skipped
|
||||
> regex lines are patterns nxdns accepts only from you — adopt one you trust
|
||||
@@ -251,15 +97,11 @@ copy:
|
||||
> list is written for a browser extension, and its DNS or hosts variant will
|
||||
> block more here.
|
||||
|
||||
`BlocklistsPage.test.tsx` asserts: both new headers render, the mock values
|
||||
(`21`, and `17` after an update snapshot) render, and the note text is
|
||||
present. No threshold logic, no badge, no coloring by magnitude (see
|
||||
anti-requirements).
|
||||
`BlocklistsPage.test.tsx` asserts: both new headers render, the mock values (`21`, and `17` after an update snapshot) render, and the note text is present. No threshold logic, no badge, no coloring by magnitude (see anti-requirements).
|
||||
|
||||
### 6. The docs explain both counters without contradicting what stands
|
||||
|
||||
- `docs/reference/configuration.md`, section `### blocklist_sources`, after
|
||||
the exceptions paragraph (configuration.md:282-288), a new paragraph:
|
||||
- `docs/reference/configuration.md`, section `### blocklist_sources`, after the exceptions paragraph (configuration.md:282-288), a new paragraph:
|
||||
|
||||
> Two counters report what a compile skipped, and they are different facts.
|
||||
> `skipped_regex` counts regex lines: nxdns has a regex engine, but it takes
|
||||
@@ -277,107 +119,43 @@ anti-requirements).
|
||||
> hosts variant will block more here. Both appear per source in the
|
||||
> blocklists UI and on `/api/blocklists`.
|
||||
|
||||
- `docs/tutorial/first-run.md`: the recorded JSON at first-run.md:223 gains
|
||||
`"skipped_unsupported":0` after `"skipped_regex":0` (the endpoint now
|
||||
returns it; the recorded values themselves stand). The paragraph at
|
||||
first-run.md:226-231 becomes four zeros, keeps its `skipped_regex` sentence
|
||||
as written, and **drops the "parts of this list that a hosts file cannot
|
||||
have" framing** — a deliberate small widening ruled during implementation
|
||||
review. The framing was false before this milestone touched it:
|
||||
`parser_hosts.zig` recognizes regex lines (:16), reports a bare sink
|
||||
address as unsupported (:21), and a `*.`-prefixed name compiles to a
|
||||
wildcard in any format. Only `exceptions` is Adblock-Plus-only. The
|
||||
paragraph now presents the zeros as facts about this particular download
|
||||
and says so.
|
||||
- The `$` modifier is named as skipped **with its one exception**: `$important`
|
||||
on an anchored `@@` exception line is accepted and lands in
|
||||
`exception_count` (`parser_abp.zig`, PLAN §2.2). The configuration.md
|
||||
paragraph above, and the `SourceRow.skipped_unsupported_count` doc comment
|
||||
in `sources_repo.zig` that mirrors it, both carry the qualifier — an
|
||||
unqualified "rules carrying a `$` modifier" contradicts the parser.
|
||||
- PLAN.md:101 (§3.8) currently reads "regex lines are counted + skipped
|
||||
(counts in metadata → UI)". It becomes: regex lines *and* browser-syntax
|
||||
lines are counted and skipped, both counts in metadata → UI. One sentence;
|
||||
§2.2 (PLAN.md:36-37) already says unsupported forms "stay unsupported and
|
||||
counted" and needs no edit.
|
||||
- `docs/reference/files-and-directories.md` is untouched: the compiled-file
|
||||
header already carried `# skipped_unsupported` before this milestone and
|
||||
the file table there does not enumerate header lines.
|
||||
- `docs/tutorial/first-run.md`: the recorded JSON at first-run.md:223 gains `"skipped_unsupported":0` after `"skipped_regex":0` (the endpoint now returns it; the recorded values themselves stand). The paragraph at first-run.md:226-231 becomes four zeros, keeps its `skipped_regex` sentence as written, and **drops the "parts of this list that a hosts file cannot have" framing** — a deliberate small widening ruled during implementation review. The framing was false before this milestone touched it: `parser_hosts.zig` recognizes regex lines (:16), reports a bare sink address as unsupported (:21), and a `*.`-prefixed name compiles to a wildcard in any format. Only `exceptions` is Adblock-Plus-only. The paragraph now presents the zeros as facts about this particular download and says so.
|
||||
- The `$` modifier is named as skipped **with its one exception**: `$important` on an anchored `@@` exception line is accepted and lands in `exception_count` (`parser_abp.zig`, PLAN §2.2). The configuration.md paragraph above, and the `SourceRow.skipped_unsupported_count` doc comment in `sources_repo.zig` that mirrors it, both carry the qualifier — an unqualified "rules carrying a `$` modifier" contradicts the parser.
|
||||
- PLAN.md:101 (§3.8) currently reads "regex lines are counted + skipped (counts in metadata → UI)". It becomes: regex lines *and* browser-syntax lines are counted and skipped, both counts in metadata → UI. One sentence; §2.2 (PLAN.md:36-37) already says unsupported forms "stay unsupported and counted" and needs no edit.
|
||||
- `docs/reference/files-and-directories.md` is untouched: the compiled-file header already carried `# skipped_unsupported` before this milestone and the file table there does not enumerate header lines.
|
||||
|
||||
### 7. The other three counters stay out, and the reasons differ per counter
|
||||
|
||||
The first draft claimed `invalid`, `long_lines` and `duplicates` were already
|
||||
"header-and-error-path only". That is false:
|
||||
The first draft claimed `invalid`, `long_lines` and `duplicates` were already "header-and-error-path only". That is false:
|
||||
|
||||
- `invalid` reaches the compiled-file header (manager.zig:244) and the
|
||||
`NoValidEntries` text (manager.zig:1127).
|
||||
- `invalid` reaches the compiled-file header (manager.zig:244) and the `NoValidEntries` text (manager.zig:1127).
|
||||
- `long_lines` reaches the `NoValidEntries` text only — not the header.
|
||||
- `duplicates` reaches **nothing**: counted at compile, discarded on every
|
||||
path.
|
||||
- `duplicates` reaches **nothing**: counted at compile, discarded on every path.
|
||||
|
||||
They stay out of scope, and not for one shared reason — which is why the goal
|
||||
above says "the silent drop that misleads the operator" rather than "the one
|
||||
silent drop left":
|
||||
They stay out of scope, and not for one shared reason — which is why the goal above says "the silent drop that misleads the operator" rather than "the one silent drop left":
|
||||
|
||||
- `duplicates` hides no failure. A deduplicated name still blocks; the count
|
||||
measures input redundancy, not lost coverage. Discarding it discards a
|
||||
curiosity, so "no silent drops" does not reach it — there is no drop.
|
||||
- `invalid` and `long_lines` do measure dropped lines, and they remain only
|
||||
partially surfaced. The catastrophic form — a download that is all rejects —
|
||||
already fails loudly (`rejectedWithoutEntries`, state `no_valid_entries`);
|
||||
the residual is a list that loses some lines and still loads, visible in
|
||||
the header file and nowhere the UI reaches. That residual is real, it is
|
||||
recorded here deliberately, and it is not this milestone: the two skip
|
||||
counters name an action the operator can take (adopt the patterns; fetch
|
||||
the DNS variant), while these two say only "the list is malformed", which
|
||||
no column makes more actionable.
|
||||
- `duplicates` hides no failure. A deduplicated name still blocks; the count measures input redundancy, not lost coverage. Discarding it discards a curiosity, so "no silent drops" does not reach it — there is no drop.
|
||||
- `invalid` and `long_lines` do measure dropped lines, and they remain only partially surfaced. The catastrophic form — a download that is all rejects — already fails loudly (`rejectedWithoutEntries`, state `no_valid_entries`); the residual is a list that loses some lines and still loads, visible in the header file and nowhere the UI reaches. That residual is real, it is recorded here deliberately, and it is not this milestone: the two skip counters name an action the operator can take (adopt the patterns; fetch the DNS variant), while these two say only "the list is malformed", which no column makes more actionable.
|
||||
|
||||
The decision is reversible; a later milestone can widen the table. What this
|
||||
spec may not do is claim the three are visible when `duplicates` is not.
|
||||
The decision is reversible; a later milestone can widen the table. What this spec may not do is claim the three are visible when `duplicates` is not.
|
||||
|
||||
### 8. Export, import and reconcile change nothing
|
||||
|
||||
`nxdns export` / `import` carry configuration; the compile counters are facts
|
||||
a running server produces. `model.BlocklistSource` holds only the four
|
||||
configuration columns, the insert leaves runtime columns at their defaults
|
||||
(sources_repo.zig:1-9, :48-61), and the dump helpers used by the import and
|
||||
reconcile suites are `SELECT *` compared dump-to-dump
|
||||
(`src/config/import.zig:167-186`, `src/config/reconcile.zig:988`), so no
|
||||
golden text names columns and the new column appears in both sides of every
|
||||
comparison. Ruling 3's reconcile-test extension is the only touch in
|
||||
`src/config/`, plus the model.zig:14 comment from ruling 2.
|
||||
`nxdns export` / `import` carry configuration; the compile counters are facts a running server produces. `model.BlocklistSource` holds only the four configuration columns, the insert leaves runtime columns at their defaults (sources_repo.zig:1-9, :48-61), and the dump helpers used by the import and reconcile suites are `SELECT *` compared dump-to-dump (`src/config/import.zig:167-186`, `src/config/reconcile.zig:988`), so no golden text names columns and the new column appears in both sides of every comparison. Ruling 3's reconcile-test extension is the only touch in `src/config/`, plus the model.zig:14 comment from ruling 2.
|
||||
|
||||
## Sessions
|
||||
|
||||
Two sessions, strictly sequential: S1 then S2. No parallelism — deliberately.
|
||||
The regenerated `contractSamples.gen.ts` typechecks only against a `types.ts`
|
||||
that already carries the new fields, and the samples are produced by a Zig
|
||||
integration run, so the generator and the interface it must satisfy sit on
|
||||
opposite sides of the language boundary and cannot land independently. S2's
|
||||
rendering assertions then read fields that only exist once S1 has landed both.
|
||||
Two sessions, strictly sequential: S1 then S2. No parallelism — deliberately. The regenerated `contractSamples.gen.ts` typechecks only against a `types.ts` that already carries the new fields, and the samples are produced by a Zig integration run, so the generator and the interface it must satisfy sit on opposite sides of the language boundary and cannot land independently. S2's rendering assertions then read fields that only exist once S1 has landed both.
|
||||
|
||||
(The mocks are *not* part of this argument: they are untyped, so widening
|
||||
`types.ts` does not break them. The first draft claimed otherwise.)
|
||||
(The mocks are *not* part of this argument: they are untyped, so widening `types.ts` does not break them. The first draft claimed otherwise.)
|
||||
|
||||
### Session S1: column, persistence, API, contract
|
||||
|
||||
Owns: `src/storage/config_schema.zig`, `src/storage/migrations.zig` (test
|
||||
only), `src/storage/repositories/sources_repo.zig`, `src/filter/manager.zig`,
|
||||
`src/filter/filter_integration_test.zig`, `src/config/reconcile.zig` (test
|
||||
only), `src/config/model.zig` (comment only),
|
||||
`src/web/handlers/blocklists.zig`, `src/web/openapi.yaml`,
|
||||
`web/src/lib/types.ts`, `web/src/lib/contractSamples.gen.ts` (regenerated,
|
||||
never hand-edited), `web/src/features/blocklists/BlocklistsPage.test.tsx`
|
||||
(mock fields only — no rendering assertions),
|
||||
`PLAN.md` (the §11.2 line and
|
||||
the §3.8 sentence).
|
||||
Owns: `src/storage/config_schema.zig`, `src/storage/migrations.zig` (test only), `src/storage/repositories/sources_repo.zig`, `src/filter/manager.zig`, `src/filter/filter_integration_test.zig`, `src/config/reconcile.zig` (test only), `src/config/model.zig` (comment only), `src/web/handlers/blocklists.zig`, `src/web/openapi.yaml`, `web/src/lib/types.ts`, `web/src/lib/contractSamples.gen.ts` (regenerated, never hand-edited), `web/src/features/blocklists/BlocklistsPage.test.tsx` (mock fields only — no rendering assertions), `PLAN.md` (the §11.2 line and the §3.8 sentence).
|
||||
|
||||
- S1.1 rulings 1 and 2: the DDL line in both copies, the repo columns, the
|
||||
migrations and repo tests.
|
||||
- S1.2 ruling 3: both `updateSourceStats` call sites, rehydration, test
|
||||
literals, the reconcile preservation assertion.
|
||||
- S1.3 ruling 4: `StatusView`, openapi.yaml, sample regeneration, `types.ts`,
|
||||
mock fields.
|
||||
- S1.1 rulings 1 and 2: the DDL line in both copies, the repo columns, the migrations and repo tests.
|
||||
- S1.2 ruling 3: both `updateSourceStats` call sites, rehydration, test literals, the reconcile preservation assertion.
|
||||
- S1.3 ruling 4: `StatusView`, openapi.yaml, sample regeneration, `types.ts`, mock fields.
|
||||
|
||||
Acceptance (S1):
|
||||
- [ ] `zig build test` passes; the migrations baseline test proves
|
||||
@@ -400,10 +178,7 @@ Acceptance (S1):
|
||||
|
||||
### Session S2: UI and docs (needs S1)
|
||||
|
||||
Owns: `web/src/features/blocklists/BlocklistsPage.tsx`,
|
||||
`web/src/features/blocklists/SourceStatusSection.tsx`,
|
||||
`web/src/features/blocklists/BlocklistsPage.test.tsx` (rendering assertions),
|
||||
`docs/reference/configuration.md`, `docs/tutorial/first-run.md`.
|
||||
Owns: `web/src/features/blocklists/BlocklistsPage.tsx`, `web/src/features/blocklists/SourceStatusSection.tsx`, `web/src/features/blocklists/BlocklistsPage.test.tsx` (rendering assertions), `docs/reference/configuration.md`, `docs/tutorial/first-run.md`.
|
||||
|
||||
- S2.1 ruling 5: both columns, the note paragraph, the rendering assertions.
|
||||
- S2.2 ruling 6: the two doc edits.
|
||||
@@ -419,22 +194,12 @@ Acceptance (S2):
|
||||
|
||||
### Orchestrator
|
||||
|
||||
Verify S1 acceptance before starting S2. After S2, run the full gate set
|
||||
(`zig build test`, `zig build test -Dintegration`, `test-aarch64` if qemu is
|
||||
present, `cd web && npm test`, `npm run assert-bundled`), then a live smoke
|
||||
against a scratch server with two real sources:
|
||||
Verify S1 acceptance before starting S2. After S2, run the full gate set (`zig build test`, `zig build test -Dintegration`, `test-aarch64` if qemu is present, `cd web && npm test`, `npm run assert-bundled`), then a live smoke against a scratch server with two real sources:
|
||||
|
||||
- `https://easylist.to/easylist/easylist.txt` — a browser-targeted list;
|
||||
expect a `skipped_unsupported` several times its `domains` (do not assert an
|
||||
exact number; assert the ratio and non-zero).
|
||||
- `https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts` — expect
|
||||
`skipped_unsupported` 0.
|
||||
- `https://easylist.to/easylist/easylist.txt` — a browser-targeted list; expect a `skipped_unsupported` several times its `domains` (do not assert an exact number; assert the ratio and non-zero).
|
||||
- `https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts` — expect `skipped_unsupported` 0.
|
||||
|
||||
Verify the numbers appear in the `POST /api/blocklists/update` response, in
|
||||
`GET /api/blocklists`, and in both UI tables. Then restart the server and
|
||||
verify the counts survive into the status table without a refresh — that is
|
||||
ruling 3's rehydration working against the real database. Record deviations
|
||||
in `## Recorded (implementation)`.
|
||||
Verify the numbers appear in the `POST /api/blocklists/update` response, in `GET /api/blocklists`, and in both UI tables. Then restart the server and verify the counts survive into the status table without a refresh — that is ruling 3's rehydration working against the real database. Record deviations in `## Recorded (implementation)`.
|
||||
|
||||
## Module layout
|
||||
|
||||
@@ -480,89 +245,40 @@ New files: none. Deleted surface: none.
|
||||
|
||||
## Anti-requirements
|
||||
|
||||
- No migration step, no `ddl_v2`, no runtime schema probing. The baseline is
|
||||
edited in place per §3.7; a pre-edit scratch database is deleted, not
|
||||
reconciled.
|
||||
- No merging of the two skip counters into one number, anywhere — not in the
|
||||
API, not in a UI total, not in prose. They are different facts.
|
||||
- No "this list targets browsers" heuristic: no threshold, badge, warning
|
||||
color or ratio computation in the UI. The number plus the note paragraph is
|
||||
the surface; a cutoff would be an invented policy.
|
||||
- No conditional rendering of the new column. It shows at 0 like every other
|
||||
counter column.
|
||||
- No change to `nxdns export` / `import` ZON: compile statistics are not
|
||||
configuration.
|
||||
- No per-line diagnostics, no sample of skipped lines in logs, API or UI —
|
||||
counters and health surfaces, not log spam (AGENTS.md).
|
||||
- No migration step, no `ddl_v2`, no runtime schema probing. The baseline is edited in place per §3.7; a pre-edit scratch database is deleted, not reconciled.
|
||||
- No merging of the two skip counters into one number, anywhere — not in the API, not in a UI total, not in prose. They are different facts.
|
||||
- No "this list targets browsers" heuristic: no threshold, badge, warning color or ratio computation in the UI. The number plus the note paragraph is the surface; a cutoff would be an invented policy.
|
||||
- No conditional rendering of the new column. It shows at 0 like every other counter column.
|
||||
- No change to `nxdns export` / `import` ZON: compile statistics are not configuration.
|
||||
- No per-line diagnostics, no sample of skipped lines in logs, API or UI — counters and health surfaces, not log spam (AGENTS.md).
|
||||
- No change to `rejectedWithoutEntries` or `failNoValidEntries` semantics.
|
||||
- No hand edits to `contractSamples.gen.ts`.
|
||||
- No new columns beyond the one. See ruling 7 for what that leaves open and
|
||||
why — the reason is a scope decision, not a claim that the other counters
|
||||
are already visible.
|
||||
- No new columns beyond the one. See ruling 7 for what that leaves open and why — the reason is a scope decision, not a claim that the other counters are already visible.
|
||||
|
||||
## Addendum (post-`1bce81e`): the body checksum is framed
|
||||
|
||||
A filtering correctness bug found by the implementation review, predating this
|
||||
milestone. Fixed as a follow-up commit; this addendum is its design record —
|
||||
the user ruled it a follow-up, not a milestone 25.
|
||||
A filtering correctness bug found by the implementation review, predating this milestone. Fixed as a follow-up commit; this addendum is its design record — the user ruled it a follow-up, not a milestone 25.
|
||||
|
||||
### The defect
|
||||
|
||||
`bodyChecksum` (manager.zig:1660) and the compiler's incremental hashing
|
||||
(compiler.zig:105-108) both digest the unframed concatenation
|
||||
`list ++ wild ++ allow`. The compiler strips `*.` from a wildcard candidate
|
||||
(compiler.zig:130-133), so upstream `a.example` (list `a.example\n`, wild
|
||||
empty) and upstream `*.a.example` (list empty, wild `a.example\n`) hash the
|
||||
same bytes. `diskBodiesMatch` (manager.zig:1085) recomputes with the same
|
||||
function, so the refresh takes the unchanged-checksum branch and a list that
|
||||
switches an exact block to a wildcard block never takes effect. Ruling 3's
|
||||
entry-count argument rested on the digest distinguishing bodies; it does not,
|
||||
and the strikethrough above records that.
|
||||
`bodyChecksum` (manager.zig:1660) and the compiler's incremental hashing (compiler.zig:105-108) both digest the unframed concatenation `list ++ wild ++ allow`. The compiler strips `*.` from a wildcard candidate (compiler.zig:130-133), so upstream `a.example` (list `a.example\n`, wild empty) and upstream `*.a.example` (list empty, wild `a.example\n`) hash the same bytes. `diskBodiesMatch` (manager.zig:1085) recomputes with the same function, so the refresh takes the unchanged-checksum branch and a list that switches an exact block to a wildcard block never takes effect. Ruling 3's entry-count argument rested on the digest distinguishing bodies; it does not, and the strikethrough above records that.
|
||||
|
||||
### The fix: a `0x00` separator after each body
|
||||
|
||||
The digest becomes `SHA-256(list ‖ 00 ‖ wild ‖ 00 ‖ allow ‖ 00)` — one zero
|
||||
byte fed to the hasher **after each of the three bodies**, same order as
|
||||
today. Soundness: a compiled body holds only validated name bytes and `\n`;
|
||||
`addCandidate` rejects any byte ≥ `0x80` or control byte
|
||||
(compiler.zig:151-155), so `0x00` cannot occur in a body and the three
|
||||
boundaries are unambiguous. Two distinct `(list, wild, allow)` triples cannot
|
||||
produce one digest short of SHA-256 itself.
|
||||
The digest becomes `SHA-256(list ‖ 00 ‖ wild ‖ 00 ‖ allow ‖ 00)` — one zero byte fed to the hasher **after each of the three bodies**, same order as today. Soundness: a compiled body holds only validated name bytes and `\n`; `addCandidate` rejects any byte ≥ `0x80` or control byte (compiler.zig:151-155), so `0x00` cannot occur in a body and the three boundaries are unambiguous. Two distinct `(list, wild, allow)` triples cannot produce one digest short of SHA-256 itself.
|
||||
|
||||
A separator, not a length prefix, because the compiler hashes while it emits
|
||||
and does not know a body's length up front; a trailing byte needs no pre-pass.
|
||||
A separator, not a length prefix, because the compiler hashes while it emits and does not know a body's length up front; a trailing byte needs no pre-pass.
|
||||
|
||||
Both producers move together or every refresh republishes forever:
|
||||
`compiler.compile` feeds the byte after each `emit` call, and
|
||||
`manager.bodyChecksum` feeds it after each body slice. Export the separator as
|
||||
a `pub const` from `compiler.zig` and have `bodyChecksum` use it — two literal
|
||||
`0`s in two files is how the next drift starts.
|
||||
Both producers move together or every refresh republishes forever: `compiler.compile` feeds the byte after each `emit` call, and `manager.bodyChecksum` feeds it after each body slice. Export the separator as a `pub const` from `compiler.zig` and have `bodyChecksum` use it — two literal `0`s in two files is how the next drift starts.
|
||||
|
||||
### Consequences, accepted
|
||||
|
||||
- Every stored checksum changes once. Zero installs; a development data
|
||||
directory fails its startup checksum verification and the startup refresh
|
||||
pass re-downloads and repairs it (or delete the data directory — the ruling
|
||||
1 stance).
|
||||
- Checksum compatibility with pre-exception digests — milestone 21 ruling 3's
|
||||
"empty allow body reproduces the old digest" property — is dead, and its
|
||||
rationale comments go with it: `Result.checksum` (compiler.zig:44-53), the
|
||||
`Header` doc and `bodyChecksum` doc (manager.zig:218-225, 1645-1648),
|
||||
`SourceStats.checksum` (sources_repo.zig), and the PLAN §3.8 sentence
|
||||
"keeps the digest it had when only two existed" (PLAN.md:101). All are
|
||||
rewritten to state the framed digest. The body order stays list, wild,
|
||||
allow.
|
||||
- Every stored checksum changes once. Zero installs; a development data directory fails its startup checksum verification and the startup refresh pass re-downloads and repairs it (or delete the data directory — the ruling 1 stance).
|
||||
- Checksum compatibility with pre-exception digests — milestone 21 ruling 3's "empty allow body reproduces the old digest" property — is dead, and its rationale comments go with it: `Result.checksum` (compiler.zig:44-53), the `Header` doc and `bodyChecksum` doc (manager.zig:218-225, 1645-1648), `SourceStats.checksum` (sources_repo.zig), and the PLAN §3.8 sentence "keeps the digest it had when only two existed" (PLAN.md:101). All are rewritten to state the framed digest. The body order stays list, wild, allow.
|
||||
|
||||
### The branch reads all five fresh
|
||||
|
||||
On the checksum-unchanged path, all five stat fields —
|
||||
`domain_count`, `wildcard_count`, `exception_count`, `skipped_regex_count`,
|
||||
`skipped_unsupported_count` — now read from `compiled.result.counts`;
|
||||
`checksum` keeps passing `stored`. With framing, fresh and stored entry counts
|
||||
are provably equal, so this is not a correctness requirement — it removes the
|
||||
per-field vouching argument from the code entirely, and any future digest
|
||||
weakness then degrades to consistent stats rather than a split between
|
||||
database and status table.
|
||||
On the checksum-unchanged path, all five stat fields — `domain_count`, `wildcard_count`, `exception_count`, `skipped_regex_count`, `skipped_unsupported_count` — now read from `compiled.result.counts`; `checksum` keeps passing `stored`. With framing, fresh and stored entry counts are provably equal, so this is not a correctness requirement — it removes the per-field vouching argument from the code entirely, and any future digest weakness then degrades to consistent stats rather than a split between database and status table.
|
||||
|
||||
### Regression tests (each watched failing with the framing reverted)
|
||||
|
||||
|
||||
+88
-452
@@ -1,240 +1,86 @@
|
||||
# Milestone 25: client names learned over reverse DNS
|
||||
|
||||
Goal: the clients table names devices by itself. The tracker already
|
||||
materialises every querying address into a `clients` row (PLAN §7.2,
|
||||
`src/server/clients.zig`), but `name` stays NULL until the operator types one —
|
||||
which defeats the point of auto-materialisation on a LAN whose router already
|
||||
knows every DHCP hostname. This milestone asks the router: for each unnamed
|
||||
client, nxdns builds the address's reverse name, matches it against the
|
||||
operator's conditional forward zones (PLAN §6.5), sends one PTR query to the
|
||||
zone's resolver, and stores the answer as a *learned* name — runtime state
|
||||
beside `last_seen`, never configuration. The dashboard shows it; a hand-typed
|
||||
name always wins; a reverse name no forward zone covers is never sent anywhere.
|
||||
Goal: the clients table names devices by itself. The tracker already materialises every querying address into a `clients` row (PLAN §7.2, `src/server/clients.zig`), but `name` stays NULL until the operator types one — which defeats the point of auto-materialisation on a LAN whose router already knows every DHCP hostname. This milestone asks the router: for each unnamed client, nxdns builds the address's reverse name, matches it against the operator's conditional forward zones (PLAN §6.5), sends one PTR query to the zone's resolver, and stores the answer as a *learned* name — runtime state beside `last_seen`, never configuration. The dashboard shows it; a hand-typed name always wins; a reverse name no forward zone covers is never sent anywhere.
|
||||
|
||||
Design written 2026-08-14 against HEAD `c428bc2`, revised after a Codex review
|
||||
of the first draft (all 19 findings accepted; the review's fingerprints are
|
||||
called out inline where a first-draft claim was wrong).
|
||||
Design written 2026-08-14 against HEAD `c428bc2`, revised after a Codex review of the first draft (all 19 findings accepted; the review's fingerprints are called out inline where a first-draft claim was wrong).
|
||||
|
||||
## Implementation contract (read first)
|
||||
|
||||
- Read `AGENTS.md`, then this spec whole, before session work starts.
|
||||
- **The v1 baseline is editable and there is no migration step** (milestone-24
|
||||
ruling 1 stands unchanged). The two new columns are lines edited into
|
||||
`config_schema.ddl_v1` plus the identical lines in PLAN §11.2, kept
|
||||
byte-identical. A `ddl_v2` or a second `Step` is wrong and must be reverted.
|
||||
A development database stamped version 1 before the edit fails its first
|
||||
`SELECT` naming the columns; delete the scratch database.
|
||||
- The new `src/local/reverse_name.zig` is pure: bytes in, bytes out, no
|
||||
`std.Io`, no clock, no sockets. (The blanket claim "src/local/ is pure" from
|
||||
the first draft was false — `forward_client.zig` does socket I/O by design;
|
||||
the purity requirement here binds the *new* module and the lookup-table
|
||||
modules it sits beside.) Everything that touches the network or the database
|
||||
lives in `src/server/`.
|
||||
- After the API shape change, regenerate
|
||||
`web/src/lib/contractSamples.gen.ts` with the AGENTS.md command (never a
|
||||
hand edit) and update `web/src/lib/types.ts` to match, in the same session.
|
||||
- Verify stdlib claims against `../zig` at tag 0.16.0. `specs/research/` holds
|
||||
verified notes. One verified here: randomness is on the `Io` interface —
|
||||
`io.random(buffer)` (`std/Io.zig:2468`); `std.crypto.random` does not exist
|
||||
in 0.16.0.
|
||||
- No `std.log.err` in new code. PTR answers are untrusted bytes: never let one
|
||||
reach a log line, a metric label, or the UI without passing `acceptHostname`
|
||||
(ruling 4).
|
||||
- **The v1 baseline is editable and there is no migration step** (milestone-24 ruling 1 stands unchanged). The two new columns are lines edited into `config_schema.ddl_v1` plus the identical lines in PLAN §11.2, kept byte-identical. A `ddl_v2` or a second `Step` is wrong and must be reverted. A development database stamped version 1 before the edit fails its first `SELECT` naming the columns; delete the scratch database.
|
||||
- The new `src/local/reverse_name.zig` is pure: bytes in, bytes out, no `std.Io`, no clock, no sockets. (The blanket claim "src/local/ is pure" from the first draft was false — `forward_client.zig` does socket I/O by design; the purity requirement here binds the *new* module and the lookup-table modules it sits beside.) Everything that touches the network or the database lives in `src/server/`.
|
||||
- After the API shape change, regenerate `web/src/lib/contractSamples.gen.ts` with the AGENTS.md command (never a hand edit) and update `web/src/lib/types.ts` to match, in the same session.
|
||||
- Verify stdlib claims against `../zig` at tag 0.16.0. `specs/research/` holds verified notes. One verified here: randomness is on the `Io` interface — `io.random(buffer)` (`std/Io.zig:2468`); `std.crypto.random` does not exist in 0.16.0.
|
||||
- No `std.log.err` in new code. PTR answers are untrusted bytes: never let one reach a log line, a metric label, or the UI without passing `acceptHostname` (ruling 4).
|
||||
|
||||
## Rulings (binding)
|
||||
|
||||
### 1. When: the tracker's flush pass resolves, bounded, serial, last
|
||||
|
||||
Resolution is a step appended to the tracker's existing flush pass
|
||||
(`Tracker.flushOnce`, `src/server/clients.zig:164`), not a new loop and not a
|
||||
query-path hook. Reasons: the pass already runs every `flush_interval_s` (60 s)
|
||||
on a dedicated task with a dedicated `config.db` connection (app.zig:523-524,
|
||||
Resolution is a step appended to the tracker's existing flush pass (`Tracker.flushOnce`, `src/server/clients.zig:164`), not a new loop and not a query-path hook. Reasons: the pass already runs every `flush_interval_s` (60 s) on a dedicated task with a dedicated `config.db` connection (app.zig:523-524,
|
||||
:754), it already respects the disk-monitor gate, and a name can only be
|
||||
learned for a row that exists — which the flush itself just wrote. A DNS query
|
||||
must never wait on naming, and with this placement it structurally cannot.
|
||||
learned for a row that exists — which the flush itself just wrote. A DNS query must never wait on naming, and with this placement it structurally cannot.
|
||||
|
||||
**Order within one pass is fixed: drain-and-write, then prune (when due), then
|
||||
resolve.** One `now_s` is read at the top of the pass and used by all three
|
||||
steps. Resolving before pruning could spend PTR queries on rows the same pass
|
||||
deletes; the fixed order makes that impossible, and a test proves a row past
|
||||
the prune cutoff on a due pass causes no exchange.
|
||||
**Order within one pass is fixed: drain-and-write, then prune (when due), then resolve.** One `now_s` is read at the top of the pass and used by all three steps. Resolving before pruning could spend PTR queries on rows the same pass deletes; the fixed order makes that impossible, and a test proves a row past the prune cutoff on a due pass causes no exchange.
|
||||
|
||||
Per pass, the resolver:
|
||||
|
||||
- selects at most `max_per_pass = 16` candidate rows (ruling 3 defines a
|
||||
candidate), ordered by `name_attempt_after` ascending then ip;
|
||||
- resolves them **serially** — in-flight is exactly 1. At 16 per 60 s pass the
|
||||
LAN resolver sees at most ~0.27 queries/second from naming.
|
||||
- selects at most `max_per_pass = 16` candidate rows (ruling 3 defines a candidate), ordered by `name_attempt_after` ascending then ip;
|
||||
- resolves them **serially** — in-flight is exactly 1. At 16 per 60 s pass the LAN resolver sees at most ~0.27 queries/second from naming.
|
||||
|
||||
**Naming has its own read timeout**, `ptr_read_timeout` = 2 s on the `.awake`
|
||||
clock, a `pub const` on the resolver — it does *not* inherit the handler's
|
||||
`forward_read_timeout`, which the operator may have set as high as the
|
||||
upstream budget. **The budget bounds the whole attempt, not one transport
|
||||
leg**: `ForwardClient` hands its duration to the UDP receive and then again
|
||||
to the TCP fallback, so a resolver that answers UDP late with TC=1 and then
|
||||
stalls TCP would spend up to 2× the budget per attempt — 64 s per pass, not
|
||||
32. The default `exchangeFn` therefore wraps the entire `ForwardClient`
|
||||
exchange in `transport.raceWithin(io, ptr_read_timeout, ...)` (the same
|
||||
mechanism `exchangeTcp` itself uses, forward_client.zig:195), so UDP,
|
||||
fallback and all share one deadline. The worst case is 16 × 2 s = 32 s of
|
||||
naming per pass, and that number is a bound, not an estimate. That is acceptable because the drain already ran first: a slow pass
|
||||
delays the *next* pass's start (the run loop sleeps after `flushOnce`
|
||||
returns), and one extra interval against a 512-slot pending table on a
|
||||
household LAN drops nothing. The acceptance test for this: with a stub client
|
||||
that times out on every exchange, the pass still drains its pending batch
|
||||
before any exchange is attempted, and attempts stop at `max_per_pass`.
|
||||
**Naming has its own read timeout**, `ptr_read_timeout` = 2 s on the `.awake` clock, a `pub const` on the resolver — it does *not* inherit the handler's `forward_read_timeout`, which the operator may have set as high as the upstream budget. **The budget bounds the whole attempt, not one transport leg**: `ForwardClient` hands its duration to the UDP receive and then again to the TCP fallback, so a resolver that answers UDP late with TC=1 and then stalls TCP would spend up to 2× the budget per attempt — 64 s per pass, not
|
||||
32. The default `exchangeFn` therefore wraps the entire `ForwardClient` exchange in `transport.raceWithin(io, ptr_read_timeout, ...)` (the same mechanism `exchangeTcp` itself uses, forward_client.zig:195), so UDP, fallback and all share one deadline. The worst case is 16 × 2 s = 32 s of naming per pass, and that number is a bound, not an estimate. That is acceptable because the drain already ran first: a slow pass delays the *next* pass's start (the run loop sleeps after `flushOnce` returns), and one extra interval against a 512-slot pending table on a household LAN drops nothing. The acceptance test for this: with a stub client that times out on every exchange, the pass still drains its pending batch before any exchange is attempted, and attempts stop at `max_per_pass`.
|
||||
|
||||
Retry cadence is split by outcome (first draft used one 24 h cadence for
|
||||
everything, which left a freshly-declared zone, a recovered router, or a fixed
|
||||
resolver unnamed for a day while forward-zone changes publish live):
|
||||
Retry cadence is split by outcome (first draft used one 24 h cadence for everything, which left a freshly-declared zone, a recovered router, or a fixed resolver unnamed for a day while forward-zone changes publish live):
|
||||
|
||||
- **definitive** outcomes (`answered`, `nxdomain`): next attempt after
|
||||
`refresh_after_s = 86_400` — the daily refresh that tracks DHCP renames;
|
||||
- **non-definitive** outcomes (`no_zone`, `failed`, `invalid`): next attempt
|
||||
after `retry_after_s = 3_600` — an operator who declares the missing zone
|
||||
or fixes the resolver sees names within the hour, and a dead resolver costs
|
||||
at most 16 timeouts per hour, not per minute.
|
||||
- **definitive** outcomes (`answered`, `nxdomain`): next attempt after `refresh_after_s = 86_400` — the daily refresh that tracks DHCP renames;
|
||||
- **non-definitive** outcomes (`no_zone`, `failed`, `invalid`): next attempt after `retry_after_s = 3_600` — an operator who declares the missing zone or fixes the resolver sees names within the hour, and a dead resolver costs at most 16 timeouts per hour, not per minute.
|
||||
|
||||
Both constants are `pub const` on the resolver; no config knob (household
|
||||
scale, AGENTS.md: no generality nobody asked for).
|
||||
Both constants are `pub const` on the resolver; no config knob (household scale, AGENTS.md: no generality nobody asked for).
|
||||
|
||||
### 2. Where: only through a declared forward zone, never the pool
|
||||
|
||||
The reverse name (`10.1.168.192.in-addr.arpa`, or the 32-nibble `ip6.arpa`
|
||||
form) is matched against the live forward-zones table — the same
|
||||
`forward_zones.Zones.match` the query path uses, bracketed by
|
||||
`LocalTables.acquire`/`release` (`src/server/local_tables.zig:48`), so naming
|
||||
always sees the generation the operator last published.
|
||||
The reverse name (`10.1.168.192.in-addr.arpa`, or the 32-nibble `ip6.arpa` form) is matched against the live forward-zones table — the same `forward_zones.Zones.match` the query path uses, bracketed by `LocalTables.acquire`/`release` (`src/server/local_tables.zig:48`), so naming always sees the generation the operator last published.
|
||||
|
||||
**The zone's `resolver` is copied by value under the shared lock, and the
|
||||
handle is released before any socket work.** `match` returns a pointer into
|
||||
the current table generation, and `swap` frees the old generation as soon as
|
||||
it holds the exclusive lock — a handle held across a PTR exchange, or a
|
||||
released handle whose `*const Zone` is still dereferenced, is a use-after-free
|
||||
against a config reload. `validate.Resolver` is a plain value; the copy is the
|
||||
whole fix. A regression test must place the `swap` **inside the hazard
|
||||
window** — after the handle release, before the copied resolver is used. A
|
||||
test that swaps before `runPass` evaluates its arguments exercises nothing
|
||||
(the first implementation's test had exactly this defect: the stub's exchange
|
||||
performed the swap, but the resolver had already been copied into the call's
|
||||
arguments, so the test passed regardless of what production retained). The
|
||||
required shape: two candidates in one pass, two generations whose zones
|
||||
cover both but carry **different resolver ports**; the stub `exchangeFn`
|
||||
performs the swap during the *first* attempt (freeing the old generation
|
||||
under `testing.allocator`) and records each attempt's resolver. Assertions:
|
||||
the first attempt's resolver still reads the old generation's port after the
|
||||
swap (the copy, not freed memory), and the second attempt's resolver reads
|
||||
the *new* generation's port (each attempt re-acquires; no table pointer or
|
||||
handle is cached across attempts). With a production `runPass` that retains
|
||||
`*const Zone` or the handle across the release, the first assertion reads
|
||||
freed memory and the second reads the stale generation — either fails.
|
||||
**The zone's `resolver` is copied by value under the shared lock, and the handle is released before any socket work.** `match` returns a pointer into the current table generation, and `swap` frees the old generation as soon as it holds the exclusive lock — a handle held across a PTR exchange, or a released handle whose `*const Zone` is still dereferenced, is a use-after-free against a config reload. `validate.Resolver` is a plain value; the copy is the whole fix. A regression test must place the `swap` **inside the hazard window** — after the handle release, before the copied resolver is used. A test that swaps before `runPass` evaluates its arguments exercises nothing (the first implementation's test had exactly this defect: the stub's exchange performed the swap, but the resolver had already been copied into the call's arguments, so the test passed regardless of what production retained). The required shape: two candidates in one pass, two generations whose zones cover both but carry **different resolver ports**; the stub `exchangeFn` performs the swap during the *first* attempt (freeing the old generation under `testing.allocator`) and records each attempt's resolver. Assertions: the first attempt's resolver still reads the old generation's port after the swap (the copy, not freed memory), and the second attempt's resolver reads the *new* generation's port (each attempt re-acquires; no table pointer or handle is cached across attempts). With a production `runPass` that retains `*const Zone` or the handle across the release, the first assertion reads freed memory and the second reads the stale generation — either fails.
|
||||
|
||||
- Match found: one PTR query to that zone's resolver through
|
||||
`forward_client.ForwardClient`, built per attempt as `Context.viaForwardZone`
|
||||
builds one (handler.zig:411), with `ptr_read_timeout` (ruling 1).
|
||||
- No match: **no query is sent to anyone** — not the pool, not any resolver.
|
||||
The attempt counts under `no_zone` and retries per ruling 1.
|
||||
- Match found: one PTR query to that zone's resolver through `forward_client.ForwardClient`, built per attempt as `Context.viaForwardZone` builds one (handler.zig:411), with `ptr_read_timeout` (ruling 1).
|
||||
- No match: **no query is sent to anyone** — not the pool, not any resolver. The attempt counts under `no_zone` and retries per ruling 1.
|
||||
|
||||
Scope of the promise, stated precisely (the first draft's "never to a public
|
||||
resolver" claimed more than the code enforces): what this milestone
|
||||
guarantees is that a reverse name reaches **only the resolver of a zone the
|
||||
operator declared**, and never the upstream pool. `parseResolver`
|
||||
(validate.zig:274) accepts any IP literal, so an operator who declares
|
||||
`168.192.in-addr.arpa → udp://8.8.8.8:53` has pointed their private reverse
|
||||
space at a public resolver — that is their declaration, and nxdns does not
|
||||
second-guess it, any more than it second-guesses the same zone for forward
|
||||
queries. Likewise nxdns cannot stop a declared LAN resolver forwarding
|
||||
onward. The docs (ruling 11) say this in one sentence.
|
||||
Scope of the promise, stated precisely (the first draft's "never to a public resolver" claimed more than the code enforces): what this milestone guarantees is that a reverse name reaches **only the resolver of a zone the operator declared**, and never the upstream pool. `parseResolver` (validate.zig:274) accepts any IP literal, so an operator who declares `168.192.in-addr.arpa → udp://8.8.8.8:53` has pointed their private reverse space at a public resolver — that is their declaration, and nxdns does not second-guess it, any more than it second-guesses the same zone for forward queries. Likewise nxdns cannot stop a declared LAN resolver forwarding onward. The docs (ruling 11) say this in one sentence.
|
||||
|
||||
The DNS cache is not consulted and not written: one PTR per client per day is
|
||||
not worth a cache entry, and bypassing the cache keeps the resolver a plain
|
||||
consumer of the exchange seam.
|
||||
The DNS cache is not consulted and not written: one PTR per client per day is not worth a cache entry, and bypassing the cache keeps the resolver a plain consumer of the exchange seam.
|
||||
|
||||
### 3. Precedence: the operator's name always wins; NXDOMAIN clears
|
||||
|
||||
Schema (ruling 5) keeps learned names in their own column, so precedence is a
|
||||
display rule, not a write conflict — `name` is never written by this feature
|
||||
and `learned_name` is never written by the operator.
|
||||
Schema (ruling 5) keeps learned names in their own column, so precedence is a display rule, not a write conflict — `name` is never written by this feature and `learned_name` is never written by the operator.
|
||||
|
||||
A **candidate** row satisfies both of:
|
||||
|
||||
- `name IS NULL OR name = ''` — a row whose displayed name would come from
|
||||
learning. `hand_edited` does not appear in the predicate (the first draft
|
||||
had it; display precedence never consults it, so candidacy must not
|
||||
either): a hand-edited row the operator grouped but did not name still
|
||||
benefits, and a named row is skipped whatever its flag — its learned name
|
||||
would never be shown.
|
||||
- `name IS NULL OR name = ''` — a row whose displayed name would come from learning. `hand_edited` does not appear in the predicate (the first draft had it; display precedence never consults it, so candidacy must not either): a hand-edited row the operator grouped but did not name still benefits, and a named row is skipped whatever its flag — its learned name would never be shown.
|
||||
- `name_attempt_after <= now_s` (ruling 6 gives the exact SQL).
|
||||
|
||||
Outcomes of an attempt, written through one repo call (ruling 6), each
|
||||
setting `name_attempt_after` per ruling 1's cadence:
|
||||
Outcomes of an attempt, written through one repo call (ruling 6), each setting `name_attempt_after` per ruling 1's cadence:
|
||||
|
||||
- **Answer with a valid PTR target**: store it in `learned_name`
|
||||
(lowercased, no trailing dot), overwriting whatever learned name was there —
|
||||
the router is the authority on its own zone, and a changed answer is a
|
||||
renamed device, not a conflict. A test covers the overwrite: two attempts,
|
||||
two different valid targets, second wins.
|
||||
- **NXDOMAIN, or NOERROR with no matching PTR record** (NODATA): clear
|
||||
`learned_name` to NULL. The router affirmatively says the address has no
|
||||
name; keeping a stale one would show a device under its previous owner's
|
||||
hostname after a lease change.
|
||||
- **Everything else** — transport error, any RCODE other than NOERROR and
|
||||
NXDOMAIN (REFUSED, FORMERR, SERVFAIL, unknown values), malformed reply,
|
||||
invalid hostname (ruling 4): keep the stored `learned_name` as it stands,
|
||||
count under `failed` (or `invalid`). An outage must not strip names from
|
||||
the whole dashboard.
|
||||
- **Answer with a valid PTR target**: store it in `learned_name` (lowercased, no trailing dot), overwriting whatever learned name was there — the router is the authority on its own zone, and a changed answer is a renamed device, not a conflict. A test covers the overwrite: two attempts, two different valid targets, second wins.
|
||||
- **NXDOMAIN, or NOERROR with no matching PTR record** (NODATA): clear `learned_name` to NULL. The router affirmatively says the address has no name; keeping a stale one would show a device under its previous owner's hostname after a lease change.
|
||||
- **Everything else** — transport error, any RCODE other than NOERROR and NXDOMAIN (REFUSED, FORMERR, SERVFAIL, unknown values), malformed reply, invalid hostname (ruling 4): keep the stored `learned_name` as it stands, count under `failed` (or `invalid`). An outage must not strip names from the whole dashboard.
|
||||
|
||||
**"RCODE" here is the full 12-bit value**: the header's 4 bits extended by
|
||||
the OPT record's upper 8 bits when the response carries one
|
||||
(`packet.findOptRecord` locates it; `dns/edns.zig` decodes it). A response
|
||||
whose header says NOERROR under a nonzero extended RCODE is a failure, not
|
||||
an answer and not NODATA — classifying on the header nibble alone would
|
||||
store or clear a name on what the resolver called an error. An OPT record
|
||||
that fails to parse makes the whole reply malformed (`failed`, keep). The
|
||||
first implementation had exactly this bug; the regression test is a
|
||||
NOERROR header + OPT with a nonzero extended RCODE, asserting `failed`
|
||||
counts and the stored name survives — it must fail with the classification
|
||||
reverted to header-only.
|
||||
**"RCODE" here is the full 12-bit value**: the header's 4 bits extended by the OPT record's upper 8 bits when the response carries one (`packet.findOptRecord` locates it; `dns/edns.zig` decodes it). A response whose header says NOERROR under a nonzero extended RCODE is a failure, not an answer and not NODATA — classifying on the header nibble alone would store or clear a name on what the resolver called an error. An OPT record that fails to parse makes the whole reply malformed (`failed`, keep). The first implementation had exactly this bug; the regression test is a NOERROR header + OPT with a nonzero extended RCODE, asserting `failed` counts and the stored name survives — it must fail with the classification reverted to header-only.
|
||||
|
||||
**Staleness bound, stated honestly:** an address reassigned to a new device
|
||||
immediately after a successful lookup shows the previous device's hostname for
|
||||
up to `refresh_after_s` (24 h) — the next refresh then overwrites or clears
|
||||
it. That bound is accepted: it matches the prune cadence's granularity, and
|
||||
tightening it means more PTR traffic for a cosmetic lag. The docs state the
|
||||
bound.
|
||||
**Staleness bound, stated honestly:** an address reassigned to a new device immediately after a successful lookup shows the previous device's hostname for up to `refresh_after_s` (24 h) — the next refresh then overwrites or clears it. That bound is accepted: it matches the prune cadence's granularity, and tightening it means more PTR traffic for a cosmetic lag. The docs state the bound.
|
||||
|
||||
`upsertSeen`, `updateClient`, `pruneStale` and the reconcile engine are
|
||||
untouched: a pruned row takes its learned name with it, a re-materialised
|
||||
device is re-learned within a pass, and `updateClient` writing a name removes
|
||||
the row from candidacy (the name predicate) without touching `learned_name`.
|
||||
`upsertSeen`, `updateClient`, `pruneStale` and the reconcile engine are untouched: a pruned row takes its learned name with it, a re-materialised device is re-learned within a pass, and `updateClient` writing a name removes the row from candidacy (the name predicate) without touching `learned_name`.
|
||||
|
||||
### 4. PTR answers are untrusted input: `acceptHostname` gates everything
|
||||
|
||||
The answer bytes come from whatever box the operator pointed a zone at, and
|
||||
they land in a UI. Validation has three layers, all required:
|
||||
The answer bytes come from whatever box the operator pointed a zone at, and they land in a UI. Validation has three layers, all required:
|
||||
|
||||
1. **Transport**: `transport.validateResponse` (ID and question echo — that is
|
||||
*all* it checks; the first draft leaned on it for more).
|
||||
2. **Record selection**: walk `packet.answers`; the accepted record is the
|
||||
**first** whose `rtype` is `.ptr`, whose class is `.in`, and whose owner
|
||||
name equals the queried reverse name case-insensitively
|
||||
(`name.eqlIgnoreCase`, name.zig:185). Records failing any of these are
|
||||
skipped, not errors; an answer section with no accepted record under
|
||||
RCODE NOERROR is NODATA (ruling 3 clears). Extra accepted records after
|
||||
the first are ignored.
|
||||
3. **Hostname shape**: the PTR target decodes via `record.rdataCname`
|
||||
(record.zig:101 handles PTR) and formats via `name.formatText`; the text is
|
||||
accepted only if every byte is in `[a-z0-9._-]` after ASCII-lowercasing
|
||||
`A-Z`, labels are 1–63 bytes, the whole name is ≤ 253 bytes, no label
|
||||
starts or ends with `-`, and there is no empty label (which also rejects a
|
||||
trailing dot — `formatText` emits none, so one appearing is malformed).
|
||||
Underscore is included because real DHCP hostnames carry it; nothing else
|
||||
is. A failing target counts under `invalid` and keeps the stored name.
|
||||
1. **Transport**: `transport.validateResponse` (ID and question echo — that is *all* it checks; the first draft leaned on it for more).
|
||||
2. **Record selection**: walk `packet.answers`; the accepted record is the **first** whose `rtype` is `.ptr`, whose class is `.in`, and whose owner name equals the queried reverse name case-insensitively (`name.eqlIgnoreCase`, name.zig:185). Records failing any of these are skipped, not errors; an answer section with no accepted record under RCODE NOERROR is NODATA (ruling 3 clears). Extra accepted records after the first are ignored.
|
||||
3. **Hostname shape**: the PTR target decodes via `record.rdataCname` (record.zig:101 handles PTR) and formats via `name.formatText`; the text is accepted only if every byte is in `[a-z0-9._-]` after ASCII-lowercasing `A-Z`, labels are 1–63 bytes, the whole name is ≤ 253 bytes, no label starts or ends with `-`, and there is no empty label (which also rejects a trailing dot — `formatText` emits none, so one appearing is malformed). Underscore is included because real DHCP hostnames carry it; nothing else is. A failing target counts under `invalid` and keeps the stored name.
|
||||
|
||||
`acceptHostname` is a pure function in `src/local/reverse_name.zig`, tested
|
||||
against: the empty string, a 254-byte name, a 64-byte label, `a b`, `a\x00b`,
|
||||
`héllo`, `-x`, `x-.y`, `a..b`, `.a`, `a.` (trailing dot), and accepting
|
||||
`nas-1.lan`, `my_printer.home`, `x`.
|
||||
`acceptHostname` is a pure function in `src/local/reverse_name.zig`, tested against: the empty string, a 254-byte name, a 64-byte label, `a b`, `a\x00b`, `héllo`, `-x`, `x-.y`, `a..b`, `.a`, `a.` (trailing dot), and accepting `nas-1.lan`, `my_printer.home`, `x`.
|
||||
|
||||
### 5. Schema: two runtime columns, not a reuse of `name`
|
||||
|
||||
@@ -245,211 +91,80 @@ against: the empty string, a 254-byte name, a 64-byte label, `a b`, `a\x00b`,
|
||||
name_attempt_after INTEGER NOT NULL DEFAULT 0,
|
||||
```
|
||||
|
||||
with the identical lines in PLAN §11.2. `name_attempt_after` is the epoch
|
||||
second before which the resolver will not attempt this row again; 0 (the
|
||||
default, and every pre-existing row) means "eligible now". Storing the *next*
|
||||
attempt rather than the *last* one keeps the candidate predicate a plain
|
||||
comparison (no addition that could overflow on a corrupt value, no special
|
||||
case for "never attempted") and lets one column carry ruling 1's two cadences.
|
||||
with the identical lines in PLAN §11.2. `name_attempt_after` is the epoch second before which the resolver will not attempt this row again; 0 (the default, and every pre-existing row) means "eligible now". Storing the *next* attempt rather than the *last* one keeps the candidate predicate a plain comparison (no addition that could overflow on a corrupt value, no special case for "never attempted") and lets one column carry ruling 1's two cadences.
|
||||
|
||||
Reusing `name` with a provenance marker was rejected: `name` feeds
|
||||
`listClients`, which feeds `nxdns export`, and a learned name in an export
|
||||
would turn runtime state into configuration — it would churn export diffs as
|
||||
leases move, and under file authority the next reconcile would fight it. With
|
||||
separate columns:
|
||||
Reusing `name` with a provenance marker was rejected: `name` feeds `listClients`, which feeds `nxdns export`, and a learned name in an export would turn runtime state into configuration — it would churn export diffs as leases move, and under file authority the next reconcile would fight it. With separate columns:
|
||||
|
||||
- **Export/import untouched.** `listClients` selects `name` only
|
||||
(clients_repo.zig:33-38); no code change, and a test proves an export before
|
||||
and after a learned name lands is byte-identical.
|
||||
- **Reconcile untouched.** The engine never touches runtime columns; the
|
||||
learned columns ride the row exactly as `last_seen` does. Promotion
|
||||
(declaring an observed address) keeps them; deletion of a declared row drops
|
||||
them with the row, correctly.
|
||||
- **File-authority mode needs no write rule.** Naming is a runtime write like
|
||||
`upsertSeen`, permitted in both authority modes for the same reason. In
|
||||
file mode the operator's *declared* name wins through the ordinary
|
||||
precedence (a non-empty `name` removes the row from candidacy after the
|
||||
reconcile writes it), and the learned name stays display-only runtime
|
||||
state.
|
||||
- **Export/import untouched.** `listClients` selects `name` only (clients_repo.zig:33-38); no code change, and a test proves an export before and after a learned name lands is byte-identical.
|
||||
- **Reconcile untouched.** The engine never touches runtime columns; the learned columns ride the row exactly as `last_seen` does. Promotion (declaring an observed address) keeps them; deletion of a declared row drops them with the row, correctly.
|
||||
- **File-authority mode needs no write rule.** Naming is a runtime write like `upsertSeen`, permitted in both authority modes for the same reason. In file mode the operator's *declared* name wins through the ordinary precedence (a non-empty `name` removes the row from candidacy after the reconcile writes it), and the learned name stays display-only runtime state.
|
||||
|
||||
The baseline test in `src/storage/migrations.zig` ("a fresh database reaches
|
||||
the baseline...") gains `columnExists` probes for both columns.
|
||||
The baseline test in `src/storage/migrations.zig` ("a fresh database reaches the baseline...") gains `columnExists` probes for both columns.
|
||||
|
||||
### 6. Repo surface: two calls, owned by the resolver
|
||||
|
||||
`src/storage/repositories/clients_repo.zig`, in the runtime section beside
|
||||
`upsertSeen` / `pruneStale`:
|
||||
`src/storage/repositories/clients_repo.zig`, in the runtime section beside `upsertSeen` / `pruneStale`:
|
||||
|
||||
- `resolveCandidates(database, out, now_s)` — fills a caller-supplied
|
||||
fixed-capacity buffer (capacity `max_per_pass`, each slot
|
||||
`logger.max_client_len` bytes — the bound every address text in this
|
||||
program is sized by) and returns the filled slice or count. **No
|
||||
allocation**: the candidate set is small by construction and the resolver
|
||||
runs on the tracker's task, which owns no allocator today. The WHERE
|
||||
clause, exactly:
|
||||
- `resolveCandidates(database, out, now_s)` — fills a caller-supplied fixed-capacity buffer (capacity `max_per_pass`, each slot `logger.max_client_len` bytes — the bound every address text in this program is sized by) and returns the filled slice or count. **No allocation**: the candidate set is small by construction and the resolver runs on the tracker's task, which owns no allocator today. The WHERE clause, exactly:
|
||||
|
||||
```sql
|
||||
(name IS NULL OR name = '') AND name_attempt_after <= ?1
|
||||
ORDER BY name_attempt_after, ip LIMIT 16
|
||||
```
|
||||
|
||||
Tests cover: `now_s` smaller than any cadence constant still selects
|
||||
never-attempted rows (`DEFAULT 0`); the boundary `name_attempt_after =
|
||||
now_s` selects; `now_s + 1` does not; an extreme stored value
|
||||
(`i64` max) never selects and never traps.
|
||||
- `noteNameOutcome(database, ip, outcome)` where the outcome carries
|
||||
`attempt_after: i64` and one of: store `text`, clear, keep. One UPDATE:
|
||||
always sets `name_attempt_after`; sets `learned_name` to the text on
|
||||
store, to NULL on clear, leaves it alone on keep. **A row deleted between
|
||||
selection and this call makes the UPDATE touch zero rows; that is a
|
||||
no-op by design, not an error and not a counter** — the device left, and
|
||||
nothing was learned about nothing.
|
||||
Tests cover: `now_s` smaller than any cadence constant still selects never-attempted rows (`DEFAULT 0`); the boundary `name_attempt_after = now_s` selects; `now_s + 1` does not; an extreme stored value (`i64` max) never selects and never traps.
|
||||
- `noteNameOutcome(database, ip, outcome)` where the outcome carries `attempt_after: i64` and one of: store `text`, clear, keep. One UPDATE: always sets `name_attempt_after`; sets `learned_name` to the text on store, to NULL on clear, leaves it alone on keep. **A row deleted between selection and this call makes the UPDATE touch zero rows; that is a no-op by design, not an error and not a counter** — the device left, and nothing was learned about nothing.
|
||||
|
||||
`ClientRow` gains `learned_name: []const u8` (NULL reads as `""`, like
|
||||
`name`); `readClientRow` and both SELECTs carry it. `name_attempt_after` is
|
||||
**not** exposed on `ClientRow` or the API — it is scheduling state with no
|
||||
operator meaning, and keeping it out keeps the contract surface one field.
|
||||
`listClients` (export) is not widened — that is the point of ruling 5.
|
||||
`ClientRow` gains `learned_name: []const u8` (NULL reads as `""`, like `name`); `readClientRow` and both SELECTs carry it. `name_attempt_after` is **not** exposed on `ClientRow` or the API — it is scheduling state with no operator meaning, and keeping it out keeps the contract surface one field. `listClients` (export) is not widened — that is the point of ruling 5.
|
||||
|
||||
### 7. The resolver module: `src/server/client_names.zig`
|
||||
|
||||
New file. A `Resolver` struct owning:
|
||||
|
||||
- `stats: Stats` with the six **outcome** counters `attempted`, `answered`,
|
||||
`nxdomain`, `no_zone`, `invalid`, `failed`, plus two counters *outside* the
|
||||
outcome sum: `read_failures` (the candidate SELECT failed; the pass skips
|
||||
naming) and `write_failures` (`noteNameOutcome` failed; the outcome was
|
||||
still counted). Guarded by the tracker's mutex pattern, with a
|
||||
`snapshotStats`. Invariant, asserted in a test: `attempted` equals
|
||||
`answered + nxdomain + no_zone + invalid + failed`. Database failures log
|
||||
at most one `warn` per pass each for reads and writes, tracker-style
|
||||
(clients.zig:181-186); network failures are the counters' job and log
|
||||
nothing new (the `forward_client` module's existing `debug`-level
|
||||
diagnostics are exempt from this milestone's no-log rule — they predate it,
|
||||
are debug-level, and suppressing them per-caller would fork the client).
|
||||
- an **exchange seam**: production code cannot inject into `ForwardClient`
|
||||
(it is a concrete struct the resolver constructs), so the seam sits above
|
||||
it — a field
|
||||
`exchangeFn: *const fn (io, validate.Resolver, query, response_buf) transport.ExchangeError![]u8`
|
||||
defaulting to a function that builds a stack `ForwardClient` with
|
||||
`ptr_read_timeout` and calls `exchange`. Tests replace the pointer; the
|
||||
production default is itself covered by one test against a loopback UDP
|
||||
socket if one already exists in the suite's patterns, otherwise by the live
|
||||
smoke. The no-zone test runs the **production** pass code with a counting
|
||||
stub and asserts the count stays 0.
|
||||
- a `runPass(io, database, tables, now_s)`: select candidates (ruling 6), and
|
||||
for each — build the reverse name (pure, ruling 8), acquire/match/**copy
|
||||
resolver**/release (ruling 2), build the PTR query: header with a random id
|
||||
filled from `io.random` (`std/Io.zig:2468` — *not* `std.crypto.random`,
|
||||
which 0.16.0 does not have), RD set, one question, qtype `.ptr`, qclass
|
||||
`.in`, encoded with `dns/header` + `dns/question.encode`; exchange through
|
||||
the seam, classify per rulings 3–4, write per ruling 6. The id is
|
||||
test-visible through the seam (the stub sees the query bytes), so no
|
||||
determinism hook is needed beyond the seam itself.
|
||||
- `stats: Stats` with the six **outcome** counters `attempted`, `answered`, `nxdomain`, `no_zone`, `invalid`, `failed`, plus two counters *outside* the outcome sum: `read_failures` (the candidate SELECT failed; the pass skips naming) and `write_failures` (`noteNameOutcome` failed; the outcome was still counted). Guarded by the tracker's mutex pattern, with a `snapshotStats`. Invariant, asserted in a test: `attempted` equals `answered + nxdomain + no_zone + invalid + failed`. Database failures log at most one `warn` per pass each for reads and writes, tracker-style (clients.zig:181-186); network failures are the counters' job and log nothing new (the `forward_client` module's existing `debug`-level diagnostics are exempt from this milestone's no-log rule — they predate it, are debug-level, and suppressing them per-caller would fork the client).
|
||||
- an **exchange seam**: production code cannot inject into `ForwardClient` (it is a concrete struct the resolver constructs), so the seam sits above it — a field `exchangeFn: *const fn (io, validate.Resolver, query, response_buf) transport.ExchangeError![]u8` defaulting to a function that builds a stack `ForwardClient` with `ptr_read_timeout` and calls `exchange`. Tests replace the pointer; the production default is itself covered by one test against a loopback UDP socket if one already exists in the suite's patterns, otherwise by the live smoke. The no-zone test runs the **production** pass code with a counting stub and asserts the count stays 0.
|
||||
- a `runPass(io, database, tables, now_s)`: select candidates (ruling 6), and for each — build the reverse name (pure, ruling 8), acquire/match/**copy resolver**/release (ruling 2), build the PTR query: header with a random id filled from `io.random` (`std/Io.zig:2468` — *not* `std.crypto.random`, which 0.16.0 does not have), RD set, one question, qtype `.ptr`, qclass `.in`, encoded with `dns/header` + `dns/question.encode`; exchange through the seam, classify per rulings 3–4, write per ruling 6. The id is test-visible through the seam (the stub sees the query bytes), so no determinism hook is needed beyond the seam itself.
|
||||
|
||||
Wiring: `Tracker.flushOnce` gains an optional `*client_names.Resolver`
|
||||
parameter, invoked **after** the drain/write and prune steps (ruling 1's
|
||||
order); `app.zig` constructs the Resolver beside the tracker (holding the
|
||||
`*LocalTables` pointer) and passes it through `Tracker.run`'s arguments
|
||||
(app.zig:754). The resolver runs on the tracker's task against `tracker_db` —
|
||||
the dedicated-connection rule (clients.zig:127-131) is satisfied because it
|
||||
is the *same* task, serial with the flush. A gated pass (disk critical) skips
|
||||
naming too: naming writes.
|
||||
Wiring: `Tracker.flushOnce` gains an optional `*client_names.Resolver` parameter, invoked **after** the drain/write and prune steps (ruling 1's order); `app.zig` constructs the Resolver beside the tracker (holding the `*LocalTables` pointer) and passes it through `Tracker.run`'s arguments (app.zig:754). The resolver runs on the tracker's task against `tracker_db` — the dedicated-connection rule (clients.zig:127-131) is satisfied because it is the *same* task, serial with the flush. A gated pass (disk critical) skips naming too: naming writes.
|
||||
|
||||
### 8. The pure part: `src/local/reverse_name.zig`
|
||||
|
||||
New file, pure (no Io, no clock). Two functions plus their tests:
|
||||
|
||||
- `reverseName(addr: address.NetAddress, buf: *[max_reverse_len]u8) []const u8`
|
||||
— `d.c.b.a.in-addr.arpa` for v4, the 32-nibble lowercase `ip6.arpa` form for
|
||||
v6. `max_reverse_len` is a comptime bound (v6 form: 32 nibbles · 2 + 8 for
|
||||
`ip6.arpa` = 72; compute, don't hand-wave). An IPv4-mapped v6 address was
|
||||
already canonicalised to `ip4` by `NetAddress` (clients.zig test at :305
|
||||
proves the tracker's addresses are canonical) — assert, don't re-handle.
|
||||
- `acceptHostname(text: []const u8) bool` — ruling 4's gate, with ruling 4's
|
||||
test table.
|
||||
- `reverseName(addr: address.NetAddress, buf: *[max_reverse_len]u8) []const u8` — `d.c.b.a.in-addr.arpa` for v4, the 32-nibble lowercase `ip6.arpa` form for v6. `max_reverse_len` is a comptime bound (v6 form: 32 nibbles · 2 + 8 for `ip6.arpa` = 72; compute, don't hand-wave). An IPv4-mapped v6 address was already canonicalised to `ip4` by `NetAddress` (clients.zig test at :305 proves the tracker's addresses are canonical) — assert, don't re-handle.
|
||||
- `acceptHostname(text: []const u8) bool` — ruling 4's gate, with ruling 4's test table.
|
||||
|
||||
`src/platform/address.zig` types are plain values; importing them into
|
||||
`local/` keeps the module pure. Register the file in `src/tests.zig` the way
|
||||
its siblings are registered.
|
||||
`src/platform/address.zig` types are plain values; importing them into `local/` keeps the module pure. Register the file in `src/tests.zig` the way its siblings are registered.
|
||||
|
||||
### 9. Failure visibility: one metrics group
|
||||
|
||||
The resolver must be reachable from `metrics.collect`, which reads
|
||||
`server.WebState` — so `src/web/server.zig`'s `WebState` gains an optional
|
||||
`client_names: ?*client_names.Resolver` collaborator, populated by `app.zig`
|
||||
beside `tracker` (the first draft wired the sample and forgot the state; S2
|
||||
owns `src/web/server.zig` for this field). `src/web/metrics.zig`: `Sample`
|
||||
gains `client_names: ?client_names.Resolver.Stats`, collected beside the
|
||||
tracker's (metrics.zig:191-193) and rendered as
|
||||
`counterGroup(w, "nxdns_client_names_", ...)` beside `nxdns_clients_`
|
||||
(metrics.zig:333-334). `counterGroup` renders every struct field, so all
|
||||
eight counters (six outcomes + `read_failures` + `write_failures`) surface
|
||||
without per-field code. The render test pins one line (e.g.
|
||||
`nxdns_client_names_no_zone_total`).
|
||||
The resolver must be reachable from `metrics.collect`, which reads `server.WebState` — so `src/web/server.zig`'s `WebState` gains an optional `client_names: ?*client_names.Resolver` collaborator, populated by `app.zig` beside `tracker` (the first draft wired the sample and forgot the state; S2 owns `src/web/server.zig` for this field). `src/web/metrics.zig`: `Sample` gains `client_names: ?client_names.Resolver.Stats`, collected beside the tracker's (metrics.zig:191-193) and rendered as `counterGroup(w, "nxdns_client_names_", ...)` beside `nxdns_clients_` (metrics.zig:333-334). `counterGroup` renders every struct field, so all eight counters (six outcomes + `read_failures` + `write_failures`) surface without per-field code. The render test pins one line (e.g. `nxdns_client_names_no_zone_total`).
|
||||
|
||||
### 10. API and UI: learned is visible and visibly different
|
||||
|
||||
- `GET /api/clients` rows carry `learned_name`
|
||||
(`src/web/openapi.yaml` `required` + `properties` for the client schema;
|
||||
contract samples regenerated; `web/src/lib/types.ts` client row type gains
|
||||
`learned_name: string`). `name_attempt_after` stays internal (ruling 6).
|
||||
- `web/src/features/clients/ClientsPage.tsx`: the name cell shows `name` when
|
||||
non-empty; otherwise `learned_name` in the page's existing muted text style
|
||||
with a visually-distinct treatment that marks it as learned (exact
|
||||
affordance is the implementer's, but: no new colour system, reuse the muted
|
||||
style the page already has, and the distinction must survive a screen
|
||||
reader — an `aria-label` or visible suffix, not colour alone). An empty
|
||||
both shows what it shows today.
|
||||
- `ClientEditDialog.tsx` may pre-fill its name field with `learned_name` when
|
||||
`name` is empty — adopting the learned name as a typed name is the natural
|
||||
gesture — but saving still goes through the ordinary PUT and sets
|
||||
`hand_edited` server-side as today. This applies in database-authority mode
|
||||
only; in file mode client edits answer 403 as they do today, and the
|
||||
file-mode story is ruling 5's (declared name wins via candidacy).
|
||||
- Mocks in `ClientsPage.test.tsx` gain the field with distinct values;
|
||||
rendering tests assert: a named row shows `name` and not `learned_name`; an
|
||||
unnamed row shows `learned_name` with the learned affordance.
|
||||
- `GET /api/clients` rows carry `learned_name` (`src/web/openapi.yaml` `required` + `properties` for the client schema; contract samples regenerated; `web/src/lib/types.ts` client row type gains `learned_name: string`). `name_attempt_after` stays internal (ruling 6).
|
||||
- `web/src/features/clients/ClientsPage.tsx`: the name cell shows `name` when non-empty; otherwise `learned_name` in the page's existing muted text style with a visually-distinct treatment that marks it as learned (exact affordance is the implementer's, but: no new colour system, reuse the muted style the page already has, and the distinction must survive a screen reader — an `aria-label` or visible suffix, not colour alone). An empty both shows what it shows today.
|
||||
- `ClientEditDialog.tsx` may pre-fill its name field with `learned_name` when `name` is empty — adopting the learned name as a typed name is the natural gesture — but saving still goes through the ordinary PUT and sets `hand_edited` server-side as today. This applies in database-authority mode only; in file mode client edits answer 403 as they do today, and the file-mode story is ruling 5's (declared name wins via candidacy).
|
||||
- Mocks in `ClientsPage.test.tsx` gain the field with distinct values; rendering tests assert: a named row shows `name` and not `learned_name`; an unnamed row shows `learned_name` with the learned affordance.
|
||||
|
||||
### 11. Docs
|
||||
|
||||
- `docs/reference/configuration.md`, clients section: a paragraph stating the
|
||||
learned-name mechanism; that it requires a conditional forward zone
|
||||
covering the LAN's reverse space (with the `168.192.in-addr.arpa` example);
|
||||
that hand-typed names win; that learned names never appear in exports; that
|
||||
a rename or lease change can display stale for up to a day (ruling 3's
|
||||
bound). The cadence must be described as it is, not rounded up: each pass
|
||||
attempts **at most 16 due rows**, and a row is due per ruling 1's two
|
||||
cadences — "every unnamed device, once a minute" overstates both coverage
|
||||
and rate and must not appear; and one sentence that PTR queries go to the declared zone's
|
||||
resolver, wherever the operator pointed it (ruling 2's scope).
|
||||
- The how-to/tutorial page that documents conditional forward zones gains the
|
||||
reverse-zone example if it lacks one (locate it; do not guess its path).
|
||||
- PLAN §7.2 gains one sentence: materialised clients are named by PTR through
|
||||
the declared forward zones; learned names are runtime state.
|
||||
- `docs/reference/configuration.md`, clients section: a paragraph stating the learned-name mechanism; that it requires a conditional forward zone covering the LAN's reverse space (with the `168.192.in-addr.arpa` example); that hand-typed names win; that learned names never appear in exports; that a rename or lease change can display stale for up to a day (ruling 3's bound). The cadence must be described as it is, not rounded up: each pass attempts **at most 16 due rows**, and a row is due per ruling 1's two cadences — "every unnamed device, once a minute" overstates both coverage and rate and must not appear; and one sentence that PTR queries go to the declared zone's resolver, wherever the operator pointed it (ruling 2's scope).
|
||||
- The how-to/tutorial page that documents conditional forward zones gains the reverse-zone example if it lacks one (locate it; do not guess its path).
|
||||
- PLAN §7.2 gains one sentence: materialised clients are named by PTR through the declared forward zones; learned names are runtime state.
|
||||
- `docs/reference/api.md` only if it enumerates client row fields (check).
|
||||
|
||||
## Sessions
|
||||
|
||||
Three, strictly sequential: S1 → S2 → S3. S2 needs S1's columns and repo
|
||||
calls; S2 owns the contract regeneration (the API shape changes when S1 widens
|
||||
`ClientRow`, but the sample regen runs the integration suite, which S2's
|
||||
wiring changes — one regen at the end of S2 avoids regenerating twice). S3
|
||||
reads the fields S2's regenerated types carry.
|
||||
Three, strictly sequential: S1 → S2 → S3. S2 needs S1's columns and repo calls; S2 owns the contract regeneration (the API shape changes when S1 widens `ClientRow`, but the sample regen runs the integration suite, which S2's wiring changes — one regen at the end of S2 avoids regenerating twice). S3 reads the fields S2's regenerated types carry.
|
||||
|
||||
### Session S1: pure module, schema, repo
|
||||
|
||||
Owns: `src/local/reverse_name.zig` (new), `src/tests.zig` (registration),
|
||||
`src/storage/config_schema.zig`, `src/storage/migrations.zig` (test),
|
||||
`src/storage/repositories/clients_repo.zig`, `PLAN.md` (§11.2 lines only).
|
||||
Owns: `src/local/reverse_name.zig` (new), `src/tests.zig` (registration), `src/storage/config_schema.zig`, `src/storage/migrations.zig` (test), `src/storage/repositories/clients_repo.zig`, `PLAN.md` (§11.2 lines only).
|
||||
|
||||
- S1.1 ruling 8: `reverse_name.zig` with exhaustive tests (v4, v6, bounds;
|
||||
`acceptHostname` accept/reject table from ruling 4).
|
||||
- S1.1 ruling 8: `reverse_name.zig` with exhaustive tests (v4, v6, bounds; `acceptHostname` accept/reject table from ruling 4).
|
||||
- S1.2 ruling 5: the two DDL lines in both copies, baseline column probes.
|
||||
- S1.3 ruling 6: repo calls, `ClientRow` widening, tests — including: the
|
||||
candidate SQL's boundary and extreme-value cases; the ordering; NULL
|
||||
round-trips as `""`; export is byte-stable across a `noteNameOutcome`;
|
||||
`updateClient` leaves `learned_name` alone; the zero-row UPDATE no-op.
|
||||
- S1.3 ruling 6: repo calls, `ClientRow` widening, tests — including: the candidate SQL's boundary and extreme-value cases; the ordering; NULL round-trips as `""`; export is byte-stable across a `noteNameOutcome`; `updateClient` leaves `learned_name` alone; the zero-row UPDATE no-op.
|
||||
|
||||
Acceptance (S1):
|
||||
- [ ] `zig build test` passes; baseline test proves both columns exist.
|
||||
@@ -465,36 +180,12 @@ Acceptance (S1):
|
||||
|
||||
### Session S2: resolver, wiring, metrics, API contract (needs S1)
|
||||
|
||||
Owns: `src/server/client_names.zig` (new), `src/tests.zig` (registration),
|
||||
`src/server/clients.zig`, `src/app.zig`, `src/web/server.zig` (the
|
||||
`client_names` field), `src/web/metrics.zig`, `src/web/openapi.yaml`,
|
||||
`web/src/lib/types.ts`, `web/src/lib/contractSamples.gen.ts` (regenerated),
|
||||
`web/src/features/clients/ClientsPage.test.tsx` (mock fields only).
|
||||
Owns: `src/server/client_names.zig` (new), `src/tests.zig` (registration), `src/server/clients.zig`, `src/app.zig`, `src/web/server.zig` (the `client_names` field), `src/web/metrics.zig`, `src/web/openapi.yaml`, `web/src/lib/types.ts`, `web/src/lib/contractSamples.gen.ts` (regenerated), `web/src/features/clients/ClientsPage.test.tsx` (mock fields only).
|
||||
|
||||
- S2.1 ruling 7: the Resolver, the exchange seam, `flushOnce` hook (ruling 1
|
||||
order), app and WebState wiring.
|
||||
- S2.2 rulings 1–4: classification tests against a stub exchange. Tests must
|
||||
cover: answered stores and lowercases; a second answered overwrites;
|
||||
NXDOMAIN clears; NODATA clears; REFUSED and an unknown RCODE keep and count
|
||||
`failed`; timeout keeps; invalid hostname keeps and counts `invalid`; a
|
||||
record with wrong type, class, or owner name is skipped; no-zone sends
|
||||
nothing (the stub's call count is the assertion) and counts;
|
||||
`name_attempt_after` lands at `now + refresh_after_s` for definitive and
|
||||
`now + retry_after_s` for non-definitive outcomes; the candidate cap holds;
|
||||
a named row is never attempted; a gated pass attempts nothing; drain
|
||||
precedes any exchange when every exchange times out (ruling 1); a row past
|
||||
the prune cutoff on a due pass causes no exchange (ruling 1); the
|
||||
two-generation swap test per ruling 2's exact shape (swap inside the
|
||||
hazard window, both assertions); a NOERROR header under a nonzero OPT
|
||||
extended RCODE counts `failed` and keeps the stored name (ruling 3); the
|
||||
default `exchangeFn` bounds UDP plus the TCP fallback under one
|
||||
`ptr_read_timeout` deadline (ruling 1) — a loopback resolver that answers
|
||||
UDP with TC=1 late in the budget and then stalls TCP must fail the attempt
|
||||
within one budget, not two (assert elapsed with margin; inject a shortened
|
||||
budget locally if the default seam needs one).
|
||||
- S2.1 ruling 7: the Resolver, the exchange seam, `flushOnce` hook (ruling 1 order), app and WebState wiring.
|
||||
- S2.2 rulings 1–4: classification tests against a stub exchange. Tests must cover: answered stores and lowercases; a second answered overwrites; NXDOMAIN clears; NODATA clears; REFUSED and an unknown RCODE keep and count `failed`; timeout keeps; invalid hostname keeps and counts `invalid`; a record with wrong type, class, or owner name is skipped; no-zone sends nothing (the stub's call count is the assertion) and counts; `name_attempt_after` lands at `now + refresh_after_s` for definitive and `now + retry_after_s` for non-definitive outcomes; the candidate cap holds; a named row is never attempted; a gated pass attempts nothing; drain precedes any exchange when every exchange times out (ruling 1); a row past the prune cutoff on a due pass causes no exchange (ruling 1); the two-generation swap test per ruling 2's exact shape (swap inside the hazard window, both assertions); a NOERROR header under a nonzero OPT extended RCODE counts `failed` and keeps the stored name (ruling 3); the default `exchangeFn` bounds UDP plus the TCP fallback under one `ptr_read_timeout` deadline (ruling 1) — a loopback resolver that answers UDP with TC=1 late in the budget and then stalls TCP must fail the attempt within one budget, not two (assert elapsed with margin; inject a shortened budget locally if the default seam needs one).
|
||||
- S2.3 ruling 9: WebState field, metrics sample, render pin.
|
||||
- S2.4 ruling 10's API half: openapi.yaml, regen samples, types.ts, mock
|
||||
fields.
|
||||
- S2.4 ruling 10's API half: openapi.yaml, regen samples, types.ts, mock fields.
|
||||
|
||||
Acceptance (S2):
|
||||
- [ ] `zig build test` and `zig build test -Dintegration` pass; the
|
||||
@@ -509,11 +200,7 @@ Acceptance (S2):
|
||||
|
||||
### Session S3: UI and docs (needs S2)
|
||||
|
||||
Owns: `web/src/features/clients/ClientsPage.tsx`,
|
||||
`web/src/features/clients/ClientEditDialog.tsx`,
|
||||
`web/src/features/clients/ClientsPage.test.tsx` (rendering assertions),
|
||||
`docs/reference/configuration.md`, `docs/reference/api.md` (if applicable),
|
||||
the conditional-forwarding doc page, `PLAN.md` (§7.2 sentence only).
|
||||
Owns: `web/src/features/clients/ClientsPage.tsx`, `web/src/features/clients/ClientEditDialog.tsx`, `web/src/features/clients/ClientsPage.test.tsx` (rendering assertions), `docs/reference/configuration.md`, `docs/reference/api.md` (if applicable), the conditional-forwarding doc page, `PLAN.md` (§7.2 sentence only).
|
||||
|
||||
- S3.1 ruling 10's UI half.
|
||||
- S3.2 ruling 11.
|
||||
@@ -527,26 +214,11 @@ Acceptance (S3):
|
||||
|
||||
### Orchestrator
|
||||
|
||||
Verify each session's acceptance before starting the next. After S3, the full
|
||||
gate set (`zig build test`, `zig build test -Dintegration`, `test-aarch64` if
|
||||
qemu is present, `cd web && npm test`, `npm run assert-bundled`), then a live
|
||||
smoke per the verify-against-the-real-network rule: a scratch server with a
|
||||
forward zone `168.192.in-addr.arpa → udp://<router>:53` (or a local stub
|
||||
resolver serving PTR), one real query from a LAN client, then wait one flush
|
||||
interval and confirm the learned name in `GET /api/clients`, in the UI, and in
|
||||
`nxdns_client_names_answered_total`. Then delete the zone and confirm the next
|
||||
attempt counts `no_zone` with no upstream traffic (tcpdump or the pool
|
||||
counters — pool queries must not move); with the hourly non-definitive
|
||||
cadence, age `name_attempt_after` by SQL in the scratch database rather than
|
||||
waiting. In database mode, hand-edit the name and confirm it wins and the row
|
||||
stops being attempted. In file mode, confirm a file-declared name wins after
|
||||
reconcile and the learned name remains display-only. Record deviations in
|
||||
`## Recorded (implementation)`.
|
||||
Verify each session's acceptance before starting the next. After S3, the full gate set (`zig build test`, `zig build test -Dintegration`, `test-aarch64` if qemu is present, `cd web && npm test`, `npm run assert-bundled`), then a live smoke per the verify-against-the-real-network rule: a scratch server with a forward zone `168.192.in-addr.arpa → udp://<router>:53` (or a local stub resolver serving PTR), one real query from a LAN client, then wait one flush interval and confirm the learned name in `GET /api/clients`, in the UI, and in `nxdns_client_names_answered_total`. Then delete the zone and confirm the next attempt counts `no_zone` with no upstream traffic (tcpdump or the pool counters — pool queries must not move); with the hourly non-definitive cadence, age `name_attempt_after` by SQL in the scratch database rather than waiting. In database mode, hand-edit the name and confirm it wins and the row stops being attempted. In file mode, confirm a file-declared name wins after reconcile and the learned name remains display-only. Record deviations in `## Recorded (implementation)`.
|
||||
|
||||
## Module layout
|
||||
|
||||
New files: `src/local/reverse_name.zig`, `src/server/client_names.zig`.
|
||||
Deleted surface: none.
|
||||
New files: `src/local/reverse_name.zig`, `src/server/client_names.zig`. Deleted surface: none.
|
||||
|
||||
## File ownership
|
||||
|
||||
@@ -585,58 +257,22 @@ Deleted surface: none.
|
||||
## Anti-requirements
|
||||
|
||||
- No migration step, no `ddl_v2`, no runtime schema probing.
|
||||
- No PTR query to the upstream pool, and none for a reverse name no declared
|
||||
forward zone covers — under any fallback, ever. (The declared resolver
|
||||
itself is the operator's choice; ruling 2 scopes the promise.)
|
||||
- No reuse of `clients.name` for learned names, and no learned data in
|
||||
`nxdns export` / `import` ZON or in reconcile semantics.
|
||||
- No mDNS, NetBIOS, DHCP-lease-file parsing, or any second naming source.
|
||||
PTR through declared zones is the mechanism; a router that serves no
|
||||
reverse zone yields `no_zone` counts and an unnamed row, visibly.
|
||||
- No PTR query to the upstream pool, and none for a reverse name no declared forward zone covers — under any fallback, ever. (The declared resolver itself is the operator's choice; ruling 2 scopes the promise.)
|
||||
- No reuse of `clients.name` for learned names, and no learned data in `nxdns export` / `import` ZON or in reconcile semantics.
|
||||
- No mDNS, NetBIOS, DHCP-lease-file parsing, or any second naming source. PTR through declared zones is the mechanism; a router that serves no reverse zone yields `no_zone` counts and an unnamed row, visibly.
|
||||
- No per-query or query-path resolution: naming rides the flush pass only.
|
||||
- No config knobs for cadence, caps, or timeouts; constants per ruling 1.
|
||||
- No DNS-cache participation for PTR probes.
|
||||
- No unbounded anything: candidates per pass, in-flight, per-attempt timeout,
|
||||
name length, and retry cadence are all bounded above by rulings 1 and 4.
|
||||
- No storing or displaying a PTR target that fails `acceptHostname` — not
|
||||
even truncated or escaped.
|
||||
- No new per-attempt log lines from this milestone's code; counters and the
|
||||
metrics group are the surface. `forward_client.zig`'s existing debug
|
||||
diagnostics stand as they are.
|
||||
- No allocation on the naming path: candidate storage is fixed-capacity
|
||||
(ruling 6).
|
||||
- No unbounded anything: candidates per pass, in-flight, per-attempt timeout, name length, and retry cadence are all bounded above by rulings 1 and 4.
|
||||
- No storing or displaying a PTR target that fails `acceptHostname` — not even truncated or escaped.
|
||||
- No new per-attempt log lines from this milestone's code; counters and the metrics group are the surface. `forward_client.zig`'s existing debug diagnostics stand as they are.
|
||||
- No allocation on the naming path: candidate storage is fixed-capacity (ruling 6).
|
||||
|
||||
## Recorded (implementation)
|
||||
|
||||
- `src/server/phase7_integration_test.zig` sits outside the ownership table
|
||||
but gained a mechanical `, null` argument at every `flushOnce`/`run` call
|
||||
site, following the new `names: ?*client_names.Resolver` parameter.
|
||||
- Review: one external round found four defects (extended RCODE ignored the
|
||||
OPT upper bits; `exchangeWithin` did not bound the UDP+TCP pair under one
|
||||
deadline; the swap test asserted nothing observable; a docs cadence
|
||||
sentence). All four were fixed; the re-review of
|
||||
`src/server/client_names.zig` returned no findings.
|
||||
- Every fix shipped with a test the author watched fail with the fix
|
||||
reverted. For the swap test the hazard was injected (a pass-level resolver
|
||||
cache that skips re-acquisition) and the second-port assertion failed; the
|
||||
one-deadline test's bound is `budget + late/2` (550 ms) because the
|
||||
unbounded path takes ~700 ms and a 2×-budget bound would pass it.
|
||||
- `src/web/openapi.yaml` was reviewed by hand against `ClientRow`
|
||||
(clients_repo.zig:312): the nine required fields match one-to-one and
|
||||
`name_attempt_after` is absent from the response, as intended.
|
||||
- Live smoke (scratch server, file mode, zone `127.in-addr.arpa` at a local
|
||||
stub PTR resolver): one real query materialised the client; the flush pass
|
||||
learned `smoke-host.lan` with no operator edit, visible in
|
||||
`GET /api/clients` and `nxdns_client_names_answered_total`. Removing the
|
||||
zone and re-arming `name_attempt_after` produced `no_zone` with zero
|
||||
packets to the stub and no pool movement. The learned name survived a
|
||||
restart's reconcile. A file-declared `name` won in the API after reconcile
|
||||
with the learned name still present as display-only state, and the named
|
||||
row left candidacy (`attempted` stayed 0 over a full flush interval). The
|
||||
database-mode hand-edit path runs the same candidacy SQL
|
||||
(`name IS NULL OR name = ''`) and is covered by the repo unit tests rather
|
||||
than a second live run. The UI half is covered by the ClientsPage
|
||||
rendering tests, not a live browser check.
|
||||
- The first smoke attempt polled a stale pre-milestone binary out of
|
||||
`zig-out/bin` — `zig build test` does not refresh the install step. Rebuild
|
||||
before any live check.
|
||||
- `src/server/phase7_integration_test.zig` sits outside the ownership table but gained a mechanical `, null` argument at every `flushOnce`/`run` call site, following the new `names: ?*client_names.Resolver` parameter.
|
||||
- Review: one external round found four defects (extended RCODE ignored the OPT upper bits; `exchangeWithin` did not bound the UDP+TCP pair under one deadline; the swap test asserted nothing observable; a docs cadence sentence). All four were fixed; the re-review of `src/server/client_names.zig` returned no findings.
|
||||
- Every fix shipped with a test the author watched fail with the fix reverted. For the swap test the hazard was injected (a pass-level resolver cache that skips re-acquisition) and the second-port assertion failed; the one-deadline test's bound is `budget + late/2` (550 ms) because the unbounded path takes ~700 ms and a 2×-budget bound would pass it.
|
||||
- `src/web/openapi.yaml` was reviewed by hand against `ClientRow` (clients_repo.zig:312): the nine required fields match one-to-one and `name_attempt_after` is absent from the response, as intended.
|
||||
- Live smoke (scratch server, file mode, zone `127.in-addr.arpa` at a local stub PTR resolver): one real query materialised the client; the flush pass learned `smoke-host.lan` with no operator edit, visible in `GET /api/clients` and `nxdns_client_names_answered_total`. Removing the zone and re-arming `name_attempt_after` produced `no_zone` with zero packets to the stub and no pool movement. The learned name survived a restart's reconcile. A file-declared `name` won in the API after reconcile with the learned name still present as display-only state, and the named row left candidacy (`attempted` stayed 0 over a full flush interval). The database-mode hand-edit path runs the same candidacy SQL (`name IS NULL OR name = ''`) and is covered by the repo unit tests rather than a second live run. The UI half is covered by the ClientsPage rendering tests, not a live browser check.
|
||||
- The first smoke attempt polled a stale pre-milestone binary out of `zig-out/bin` — `zig build test` does not refresh the install step. Rebuild before any live check.
|
||||
|
||||
+104
-315
@@ -1,26 +1,16 @@
|
||||
# Milestone 3: Resolver Transport
|
||||
|
||||
Goal (PLAN §16 Phase 3): UDP server, TCP server, DoH + DoT upstream clients, upstream pool with
|
||||
priority failover, backoff and health. Exit: A/AAAA forwarding over UDP and TCP; health populated.
|
||||
Goal (PLAN §16 Phase 3): UDP server, TCP server, DoH + DoT upstream clients, upstream pool with priority failover, backoff and health. Exit: A/AAAA forwarding over UDP and TCP; health populated.
|
||||
|
||||
Read first: `AGENTS.md` (values), `specs/research/zig-0.16-api-notes.md` (verified stdlib facts —
|
||||
pre-0.16 knowledge is stale and MUST NOT be used), `specs/milestone-1.md` and `specs/milestone-2.md`
|
||||
(module conventions and the "As built" notes). The Zig source of truth is `/home/mokhtar/app/zig` at
|
||||
tag `0.16.0`. RFCs: 1035 §4.2.2 (TCP length prefix), 7766 (DNS over TCP), 7858 (DoT), 8484 (DoH),
|
||||
6891 (EDNS(0)), 4343 (name case-insensitivity), 9619 (QDCOUNT must be 1).
|
||||
Read first: `AGENTS.md` (values), `specs/research/zig-0.16-api-notes.md` (verified stdlib facts — pre-0.16 knowledge is stale and MUST NOT be used), `specs/milestone-1.md` and `specs/milestone-2.md` (module conventions and the "As built" notes). The Zig source of truth is `/home/mokhtar/app/zig` at tag `0.16.0`. RFCs: 1035 §4.2.2 (TCP length prefix), 7766 (DNS over TCP), 7858 (DoT), 8484 (DoH), 6891 (EDNS(0)), 4343 (name case-insensitivity), 9619 (QDCOUNT must be 1).
|
||||
|
||||
## What already exists (do not respecify, import it)
|
||||
|
||||
- `src/dns/*` — pure wire format. Used here: `packet.parse`, `packet.Packet`, `packet.firstQuestion`,
|
||||
`packet.findOptRecord`, `packet.ResponseBuilder`, `packet.setId`, `header.parse`, `header.Flags`,
|
||||
`question.Question`, `name.eqlIgnoreCase`, `edns.parseOpt`, `types.Rcode`, `types.Type`.
|
||||
- `src/dns/*` — pure wire format. Used here: `packet.parse`, `packet.Packet`, `packet.firstQuestion`, `packet.findOptRecord`, `packet.ResponseBuilder`, `packet.setId`, `header.parse`, `header.Flags`, `question.Question`, `name.eqlIgnoreCase`, `edns.parseOpt`, `types.Rcode`, `types.Type`.
|
||||
- `src/platform/address.zig` — `NetAddress`, `Prefix`, `matchLongest`.
|
||||
- `src/platform/tls_client.zig` — `TlsStream` (pinned struct; `init`, `reader`, `writer`, `close`),
|
||||
`classify`, `ErrorClass`. DoT builds on this; do not open `std.crypto.tls.Client` directly.
|
||||
- `src/platform/tls_server.zig` — server-side TLS. **Not used in this milestone** (local DoH/DoT
|
||||
endpoints are Phase 9).
|
||||
- `build.zig` — `-Dintegration` (hermetic, loopback only, PR-blocking) and `-Dlive` (leaves the
|
||||
machine, manual workflow only) reach test files through `@import("build_options")`.
|
||||
- `src/platform/tls_client.zig` — `TlsStream` (pinned struct; `init`, `reader`, `writer`, `close`), `classify`, `ErrorClass`. DoT builds on this; do not open `std.crypto.tls.Client` directly.
|
||||
- `src/platform/tls_server.zig` — server-side TLS. **Not used in this milestone** (local DoH/DoT endpoints are Phase 9).
|
||||
- `build.zig` — `-Dintegration` (hermetic, loopback only, PR-blocking) and `-Dlive` (leaves the machine, manual workflow only) reach test files through `@import("build_options")`.
|
||||
|
||||
## Sessions
|
||||
|
||||
@@ -34,47 +24,27 @@ S1 (upstream/transport.zig + upstream/health.zig)
|
||||
+--> S7 (e2e integration test)
|
||||
```
|
||||
|
||||
S1 defines every type the other sessions share, so S2–S5 can be written against the spec alone.
|
||||
S6 depends on S5's `Handler` type; S7 depends on everything.
|
||||
S1 defines every type the other sessions share, so S2–S5 can be written against the spec alone. S6 depends on S5's `Handler` type; S7 depends on everything.
|
||||
|
||||
The orchestrator — not any session — wires `src/tests.zig` imports and any `build.zig` change. A
|
||||
session that needs a build change reports the exact change in its completion report.
|
||||
The orchestrator — not any session — wires `src/tests.zig` imports and any `build.zig` change. A session that needs a build change reports the exact change in its completion report.
|
||||
|
||||
## Design invariants (all sessions)
|
||||
|
||||
- The pure core stays pure. `src/dns/` gains nothing. All `std.Io` use lives in `src/server/` and
|
||||
`src/upstream/`. `upstream/health.zig` and the validation half of `upstream/transport.zig` are
|
||||
themselves pure (timestamps and randomness arrive as parameters), so they are unit-testable
|
||||
without a backend.
|
||||
- **Failures are classified into three disjoint groups**: peer fault, local resource, cancellation.
|
||||
Health and backoff count peer faults ONLY. A local `OutOfMemory` must never mark an upstream sick,
|
||||
and `error.Canceled` must never be recorded at all.
|
||||
- **QDCOUNT must be exactly 1** (RFC 9619) — for inbound client queries (else FORMERR) and for
|
||||
upstream responses (else the response is a peer fault).
|
||||
- Every response from an upstream is validated against the request it answers before it reaches a
|
||||
client: ID equal, QDCOUNT 1 on both sides, QR set, question name equal **case-insensitively**
|
||||
(RFC 4343), qtype and qclass equal.
|
||||
- The pure core stays pure. `src/dns/` gains nothing. All `std.Io` use lives in `src/server/` and `src/upstream/`. `upstream/health.zig` and the validation half of `upstream/transport.zig` are themselves pure (timestamps and randomness arrive as parameters), so they are unit-testable without a backend.
|
||||
- **Failures are classified into three disjoint groups**: peer fault, local resource, cancellation. Health and backoff count peer faults ONLY. A local `OutOfMemory` must never mark an upstream sick, and `error.Canceled` must never be recorded at all.
|
||||
- **QDCOUNT must be exactly 1** (RFC 9619) — for inbound client queries (else FORMERR) and for upstream responses (else the response is a peer fault).
|
||||
- Every response from an upstream is validated against the request it answers before it reaches a client: ID equal, QDCOUNT 1 on both sides, QR set, question name equal **case-insensitively** (RFC 4343), qtype and qclass equal.
|
||||
- TCP and DoT messages carry a 2-byte big-endian length prefix (RFC 1035 §4.2.2). DoH does not.
|
||||
- No stream read or write in 0.16.0 accepts a timeout (verified: `Io.Operation` has no net-stream
|
||||
variants). Bound them by running the work under `io.concurrent` / `std.Io.Select` and cancelling
|
||||
the loser, exactly as `src/platform/tls_client_integration_test.zig` does. UDP `receive` is the one
|
||||
exception — `Socket.receiveTimeout` is implemented on the POSIX Threaded backend (verified:
|
||||
`Io/Threaded.zig:2779` handles `net_receive` in the pollable batch path).
|
||||
- Never set `net.IpAddress.ConnectOptions.timeout` — the Threaded backend panics
|
||||
(`Io/Threaded.zig:12077`).
|
||||
- Every dropped datagram, refused connection and failed send increments a named counter. No silent
|
||||
drops (AGENTS.md).
|
||||
- Unit tests live in-file. Tests that touch loopback sockets live in a separate `*_integration_test.zig`
|
||||
file guarded by `if (!build_options.integration) return error.SkipZigTest;`. Tests that leave the
|
||||
machine are guarded by `build_options.live`. This mirrors milestone 1 — the guard is a runtime
|
||||
return so the body is always compiled and cannot rot.
|
||||
- No stream read or write in 0.16.0 accepts a timeout (verified: `Io.Operation` has no net-stream variants). Bound them by running the work under `io.concurrent` / `std.Io.Select` and cancelling the loser, exactly as `src/platform/tls_client_integration_test.zig` does. UDP `receive` is the one exception — `Socket.receiveTimeout` is implemented on the POSIX Threaded backend (verified: `Io/Threaded.zig:2779` handles `net_receive` in the pollable batch path).
|
||||
- Never set `net.IpAddress.ConnectOptions.timeout` — the Threaded backend panics (`Io/Threaded.zig:12077`).
|
||||
- Every dropped datagram, refused connection and failed send increments a named counter. No silent drops (AGENTS.md).
|
||||
- Unit tests live in-file. Tests that touch loopback sockets live in a separate `*_integration_test.zig` file guarded by `if (!build_options.integration) return error.SkipZigTest;`. Tests that leave the machine are guarded by `build_options.live`. This mirrors milestone 1 — the guard is a runtime return so the body is always compiled and cannot rot.
|
||||
|
||||
---
|
||||
|
||||
## Session S1: `src/upstream/transport.zig`, `src/upstream/health.zig`
|
||||
|
||||
Foundation. No sockets are opened in this session; `transport.zig` names the `Io`-taking interface
|
||||
that S2–S4 implement, and everything else in both files is pure.
|
||||
Foundation. No sockets are opened in this session; `transport.zig` names the `Io`-taking interface that S2–S4 implement, and everything else in both files is pure.
|
||||
|
||||
### S1.1 transport.zig — endpoint parsing
|
||||
|
||||
@@ -101,15 +71,9 @@ pub const Endpoint = struct {
|
||||
};
|
||||
```
|
||||
|
||||
Hand-written parse (a `std.Uri` round trip would hand back percent-encoded components that need
|
||||
re-decoding for one household-scale config value). Rules: reject any other scheme, reject an empty
|
||||
host, reject a port that is empty, non-numeric or > 65535, reject a `tls://` URL that carries a path
|
||||
other than `/` or nothing.
|
||||
Hand-written parse (a `std.Uri` round trip would hand back percent-encoded components that need re-decoding for one household-scale config value). Rules: reject any other scheme, reject an empty host, reject a port that is empty, non-numeric or > 65535, reject a `tls://` URL that carries a path other than `/` or nothing.
|
||||
|
||||
Tests: `https://cloudflare-dns.com/dns-query`, `https://dns.example/x` (path preserved),
|
||||
`https://dns.example` (path defaults), `tls://dns.google:853`, `tls://dns.google` (port defaults to
|
||||
853), `https://[2606:4700:4700::1111]:8443/dns-query` (host without brackets, port 8443),
|
||||
`udp://1.1.1.1:53` → `UnsupportedScheme`, `https://` → `MissingHost`, `https://h:99999/` → `BadPort`.
|
||||
Tests: `https://cloudflare-dns.com/dns-query`, `https://dns.example/x` (path preserved), `https://dns.example` (path defaults), `tls://dns.google:853`, `tls://dns.google` (port defaults to 853), `https://[2606:4700:4700::1111]:8443/dns-query` (host without brackets, port 8443), `udp://1.1.1.1:53` → `UnsupportedScheme`, `https://` → `MissingHost`, `https://h:99999/` → `BadPort`.
|
||||
|
||||
### S1.2 transport.zig — error groups
|
||||
|
||||
@@ -149,13 +113,9 @@ pub const Group = enum { peer_fault, local_resource, cancellation };
|
||||
pub fn group(err: ExchangeError) Group;
|
||||
```
|
||||
|
||||
The three sets are disjoint by construction; a test asserts it by walking
|
||||
`@typeInfo(PeerFault).error_set.?` and friends and checking no name appears twice.
|
||||
The three sets are disjoint by construction; a test asserts it by walking `@typeInfo(PeerFault).error_set.?` and friends and checking no name appears twice.
|
||||
|
||||
`pub fn mapLocal(err: anyerror) ?ExchangeError` — helper for S2/S3: returns the `LocalResource` or
|
||||
`Cancellation` member matching a foreign stdlib error by name, `null` when the caller should treat
|
||||
the error as a peer fault. Implemented with an explicit switch over the named errors listed above
|
||||
plus `error.Canceled`; this is the ONLY place a foreign error set is folded in.
|
||||
`pub fn mapLocal(err: anyerror) ?ExchangeError` — helper for S2/S3: returns the `LocalResource` or `Cancellation` member matching a foreign stdlib error by name, `null` when the caller should treat the error as a peer fault. Implemented with an explicit switch over the named errors listed above plus `error.Canceled`; this is the ONLY place a foreign error set is folded in.
|
||||
|
||||
### S1.3 transport.zig — the client interface
|
||||
|
||||
@@ -195,17 +155,9 @@ pub const ValidateError = error{ BadResponse, ResponseMismatch };
|
||||
pub fn validateResponse(query: []const u8, response: []const u8) ValidateError!void;
|
||||
```
|
||||
|
||||
Order of checks: response parses (`packet.parse` failure → `BadResponse`); response header
|
||||
`flags.qr` is true (else `BadResponse`); response `qdcount == 1` (else `BadResponse`); query parses
|
||||
and has `qdcount == 1` (else `BadResponse` — a caller bug, but never a crash); IDs equal (else
|
||||
`ResponseMismatch`); question `qtype` and `qclass` equal and `name.eqlIgnoreCase` (else
|
||||
`ResponseMismatch`).
|
||||
Order of checks: response parses (`packet.parse` failure → `BadResponse`); response header `flags.qr` is true (else `BadResponse`); response `qdcount == 1` (else `BadResponse`); query parses and has `qdcount == 1` (else `BadResponse` — a caller bug, but never a crash); IDs equal (else `ResponseMismatch`); question `qtype` and `qclass` equal and `name.eqlIgnoreCase` (else `ResponseMismatch`).
|
||||
|
||||
Tests (hand-built byte fixtures, reuse the shape of `src/dns/packet.zig`'s fixtures): matching pair
|
||||
passes; mixed-case question name passes; wrong ID → `ResponseMismatch`; different qtype →
|
||||
`ResponseMismatch`; different name → `ResponseMismatch`; QR clear → `BadResponse`; response with
|
||||
QDCOUNT 0 → `BadResponse`; response with QDCOUNT 2 → `BadResponse`; truncated garbage →
|
||||
`BadResponse`.
|
||||
Tests (hand-built byte fixtures, reuse the shape of `src/dns/packet.zig`'s fixtures): matching pair passes; mixed-case question name passes; wrong ID → `ResponseMismatch`; different qtype → `ResponseMismatch`; different name → `ResponseMismatch`; QR clear → `BadResponse`; response with QDCOUNT 0 → `BadResponse`; response with QDCOUNT 2 → `BadResponse`; truncated garbage → `BadResponse`.
|
||||
|
||||
### S1.5 health.zig — pure health and backoff state
|
||||
|
||||
@@ -253,29 +205,12 @@ pub const State = struct {
|
||||
|
||||
Rules:
|
||||
|
||||
1. `recordSuccess` clears `consecutive_failures` and `backoff_until`, pushes a 1 into the window,
|
||||
and sets `last_success_at` to `@max(existing, at)` by nanoseconds.
|
||||
2. `recordFailure` increments `consecutive_failures` and `total_failures`, pushes a 0, copies
|
||||
`err_name` (truncated to the buffer), sets `last_error_at` to `@max(existing, at)`. When
|
||||
`consecutive_failures >= cfg.failure_threshold`, it computes
|
||||
`delay_ms = min(max_backoff_ms, base_backoff_ms << shift)` where
|
||||
`shift = min(consecutive_failures - failure_threshold, 20)`, applies jitter
|
||||
`jittered = delay_ms/2 + rand % (delay_ms/2 + 1)`, and sets
|
||||
`backoff_until = @max(existing, at + jittered)`.
|
||||
3. **Out-of-order completions are the normal case**, not an edge case: two concurrent exchanges
|
||||
against the same endpoint complete in either order, so `at` can move backwards between calls.
|
||||
Every timestamp field therefore updates through `@max` on `.nanoseconds`, and `backoff_until` is
|
||||
only ever extended, never shortened. `available` must never compute a negative duration.
|
||||
1. `recordSuccess` clears `consecutive_failures` and `backoff_until`, pushes a 1 into the window, and sets `last_success_at` to `@max(existing, at)` by nanoseconds.
|
||||
2. `recordFailure` increments `consecutive_failures` and `total_failures`, pushes a 0, copies `err_name` (truncated to the buffer), sets `last_error_at` to `@max(existing, at)`. When `consecutive_failures >= cfg.failure_threshold`, it computes `delay_ms = min(max_backoff_ms, base_backoff_ms << shift)` where `shift = min(consecutive_failures - failure_threshold, 20)`, applies jitter `jittered = delay_ms/2 + rand % (delay_ms/2 + 1)`, and sets `backoff_until = @max(existing, at + jittered)`.
|
||||
3. **Out-of-order completions are the normal case**, not an edge case: two concurrent exchanges against the same endpoint complete in either order, so `at` can move backwards between calls. Every timestamp field therefore updates through `@max` on `.nanoseconds`, and `backoff_until` is only ever extended, never shortened. `available` must never compute a negative duration.
|
||||
4. `State` carries no lock. The pool owns the mutex.
|
||||
|
||||
Tests: threshold not reached → `available(now)` stays true; threshold reached → unavailable until
|
||||
`backoff_until`, available one nanosecond after; consecutive failures grow the delay and it saturates
|
||||
at `max_backoff_ms` (walk 40 failures with `rand = 0`); success resets consecutive count, window and
|
||||
backoff; jitter with `rand = 0` and `rand = maxInt(u32)` both land inside `[delay/2, delay]`; an
|
||||
out-of-order pair — `recordSuccess(t=100)` then `recordSuccess(t=50)` — leaves `last_success_at` at
|
||||
100; `recordFailure(t=50)` after `recordFailure(t=100)` does not shorten `backoff_until`;
|
||||
`successRate` over a half-success window is 0.5; `lastError` returns the last recorded name and is
|
||||
truncated, not overflowed, by a name longer than the buffer.
|
||||
Tests: threshold not reached → `available(now)` stays true; threshold reached → unavailable until `backoff_until`, available one nanosecond after; consecutive failures grow the delay and it saturates at `max_backoff_ms` (walk 40 failures with `rand = 0`); success resets consecutive count, window and backoff; jitter with `rand = 0` and `rand = maxInt(u32)` both land inside `[delay/2, delay]`; an out-of-order pair — `recordSuccess(t=100)` then `recordSuccess(t=50)` — leaves `last_success_at` at 100; `recordFailure(t=50)` after `recordFailure(t=100)` does not shorten `backoff_until`; `successRate` over a half-success window is 0.5; `lastError` returns the last recorded name and is truncated, not overflowed, by a name longer than the buffer.
|
||||
|
||||
### S1.6 Acceptance criteria
|
||||
|
||||
@@ -314,16 +249,12 @@ pub const DohClient = struct {
|
||||
};
|
||||
```
|
||||
|
||||
`self.uri` is derived from `endpoint` with `std.Uri.parse(endpoint.url)`; a parse failure is
|
||||
`error.BadUrl`. `endpoint` remains the source of truth for host/port/scheme in logs.
|
||||
`self.uri` is derived from `endpoint` with `std.Uri.parse(endpoint.url)`; a parse failure is `error.BadUrl`. `endpoint` remains the source of truth for host/port/scheme in logs.
|
||||
|
||||
### S2.2 Exchange
|
||||
|
||||
1. `if (query.len > self.request_buf.len) return error.BufferTooSmall;` then copy `query` into
|
||||
`request_buf` — `Request.sendBodyComplete` takes `[]u8`, not `[]const u8` (verified,
|
||||
`std/http/Client.zig:935`).
|
||||
2. ```zig
|
||||
var req = try self.http.request(.POST, self.uri, .{
|
||||
1. `if (query.len > self.request_buf.len) return error.BufferTooSmall;` then copy `query` into `request_buf` — `Request.sendBodyComplete` takes `[]u8`, not `[]const u8` (verified, `std/http/Client.zig:935`).
|
||||
2. ```zig var req = try self.http.request(.POST, self.uri, .{
|
||||
.keep_alive = true,
|
||||
.redirect_behavior = .not_allowed,
|
||||
.headers = .{
|
||||
@@ -333,8 +264,7 @@ pub const DohClient = struct {
|
||||
.accept_encoding = .{ .override = "identity" },
|
||||
},
|
||||
.extra_headers = &.{.{ .name = "accept", .value = "application/dns-message" }},
|
||||
});
|
||||
defer req.deinit();
|
||||
}); defer req.deinit();
|
||||
```
|
||||
`Request.Headers` has no `accept` field (verified, `std/http/Client.zig:845`), hence
|
||||
`extra_headers`.
|
||||
@@ -425,48 +355,24 @@ pub const DotClient = struct {
|
||||
|
||||
### S3.2 Exchange
|
||||
|
||||
One TCP connection and one TLS handshake per exchange, closed before returning. Connection reuse is
|
||||
not built: at household query rates the pool's per-attempt budget and the failover path stay simple
|
||||
and every failure is attributable to one exchange, which is worth more here than the saved round
|
||||
trips. Say that in a comment — it is a decision, not a stub.
|
||||
One TCP connection and one TLS handshake per exchange, closed before returning. Connection reuse is not built: at household query rates the pool's per-attempt budget and the failover path stay simple and every failure is attributable to one exchange, which is worth more here than the saved round trips. Say that in a comment — it is a decision, not a stub.
|
||||
|
||||
1. Resolve the endpoint to an `std.Io.net.IpAddress` with `net.IpAddress.parse(endpoint.host,
|
||||
endpoint.port)`. **A non-literal host is `error.ConnectFailed`** with a log line naming the host:
|
||||
name resolution for upstreams is not in this milestone's scope, and a silent fallback would hide
|
||||
it. (`tls://dns.google:853` from PLAN §12.1 therefore needs an IP literal in config; note this in
|
||||
the completion report so the orchestrator can carry it into the Phase 4 config validator.)
|
||||
2. `var stream = try address.connect(io, .{ .mode = .stream });` — no `timeout` field, ever.
|
||||
`defer` a close that runs under cancel protection (below).
|
||||
3. `var tls_stream: tls_client.TlsStream = undefined;` then `tls_stream.init(io, &stream, bundle,
|
||||
bundle_lock, gpa, .{ .host = endpoint.host, .ca = .system, … })`. `TlsStream` is pinned — do not
|
||||
copy or move it after init.
|
||||
1. Resolve the endpoint to an `std.Io.net.IpAddress` with `net.IpAddress.parse(endpoint.host, endpoint.port)`. **A non-literal host is `error.ConnectFailed`** with a log line naming the host: name resolution for upstreams is not in this milestone's scope, and a silent fallback would hide it. (`tls://dns.google:853` from PLAN §12.1 therefore needs an IP literal in config; note this in the completion report so the orchestrator can carry it into the Phase 4 config validator.)
|
||||
2. `var stream = try address.connect(io, .{ .mode = .stream });` — no `timeout` field, ever. `defer` a close that runs under cancel protection (below).
|
||||
3. `var tls_stream: tls_client.TlsStream = undefined;` then `tls_stream.init(io, &stream, bundle, bundle_lock, gpa, .{ .host = endpoint.host, .ca = .system, … })`. `TlsStream` is pinned — do not copy or move it after init.
|
||||
4. Write `[2]u8` big-endian length then `query` to `tls_stream.writer()`, then `flush()`.
|
||||
5. Read exactly 2 bytes, then that many bytes, from `tls_stream.reader()`. `len == 0` →
|
||||
`error.BadResponse`. `len > response_buf.len` → `error.ResponseTooLarge`. A short read or EOF →
|
||||
`error.ReceiveFailed`.
|
||||
5. Read exactly 2 bytes, then that many bytes, from `tls_stream.reader()`. `len == 0` → `error.BadResponse`. `len > response_buf.len` → `error.ResponseTooLarge`. A short read or EOF → `error.ReceiveFailed`.
|
||||
6. `try transport.validateResponse(query, response_buf[0..len]);`
|
||||
7. `tls_stream.close();` then `stream.close(io);`.
|
||||
|
||||
Cleanup under cancellation: the pool cancels this task on timeout, and the next cancelable `Io` call
|
||||
in the `defer` chain would return `error.Canceled` and skip the socket close. Wrap the close path in
|
||||
`const prev = io.swapCancelProtection(.blocked); defer _ = io.swapCancelProtection(prev);`
|
||||
(verified: `std/Io.zig:1342`) so the socket is always released.
|
||||
Cleanup under cancellation: the pool cancels this task on timeout, and the next cancelable `Io` call in the `defer` chain would return `error.Canceled` and skip the socket close. Wrap the close path in `const prev = io.swapCancelProtection(.blocked); defer _ = io.swapCancelProtection(prev);` (verified: `std/Io.zig:1342`) so the socket is always released.
|
||||
|
||||
Error mapping: `connect` → `error.ConnectFailed`; `TlsStream.init` → `error.TlsFailed` (log
|
||||
`tls_client.classify(err)` alongside — that is what milestone 1 built it for); write/flush →
|
||||
`error.SendFailed`; read → `error.ReceiveFailed`; `transport.mapLocal` first at every site.
|
||||
Error mapping: `connect` → `error.ConnectFailed`; `TlsStream.init` → `error.TlsFailed` (log `tls_client.classify(err)` alongside — that is what milestone 1 built it for); write/flush → `error.SendFailed`; read → `error.ReceiveFailed`; `transport.mapLocal` first at every site.
|
||||
|
||||
### S3.3 Tests
|
||||
|
||||
- In-file unit: `framePrefix(len: u16) [2]u8` and `parsePrefix(bytes: [2]u8) u16` round-trip,
|
||||
including 0 and 65535, big-endian byte order asserted explicitly; `init` asserts on a `.doh`
|
||||
endpoint; a non-literal host maps to `error.ConnectFailed` without touching the network (call the
|
||||
address-resolution helper directly).
|
||||
- `src/upstream/dot_client_live_test.zig`, guarded by `build_options.live`: exchange an A query for
|
||||
`example.com` against `tls://1.1.1.1:853` with `endpoint.host` overridden to `"1.1.1.1"` — this
|
||||
machine's IPv6 egress is dead, so the documented anycast literal is used, and Cloudflare's
|
||||
certificate carries `1.1.1.1` as an IP SAN. If the stdlib verifier rejects an IP-literal SAN, do
|
||||
NOT weaken verification: report the finding. Raced against a 10 s budget with `std.Io.Select`.
|
||||
- In-file unit: `framePrefix(len: u16) [2]u8` and `parsePrefix(bytes: [2]u8) u16` round-trip, including 0 and 65535, big-endian byte order asserted explicitly; `init` asserts on a `.doh` endpoint; a non-literal host maps to `error.ConnectFailed` without touching the network (call the address-resolution helper directly).
|
||||
- `src/upstream/dot_client_live_test.zig`, guarded by `build_options.live`: exchange an A query for `example.com` against `tls://1.1.1.1:853` with `endpoint.host` overridden to `"1.1.1.1"` — this machine's IPv6 egress is dead, so the documented anycast literal is used, and Cloudflare's certificate carries `1.1.1.1` as an IP SAN. If the stdlib verifier rejects an IP-literal SAN, do NOT weaken verification: report the finding. Raced against a 10 s budget with `std.Io.Select`.
|
||||
|
||||
### S3.4 Acceptance criteria
|
||||
|
||||
@@ -480,8 +386,7 @@ Error mapping: `connect` → `error.ConnectFailed`; `TlsStream.init` → `error.
|
||||
|
||||
## Session S4: `src/upstream/pool.zig`
|
||||
|
||||
Priority-ordered sequential failover with per-attempt deadline, health tracking and backoff
|
||||
(PLAN §9). The pool is itself a `transport.Client`, so the handler sees one interface.
|
||||
Priority-ordered sequential failover with per-attempt deadline, health tracking and backoff (PLAN §9). The pool is itself a `transport.Client`, so the handler sees one interface.
|
||||
|
||||
### S4.1 API
|
||||
|
||||
@@ -537,61 +442,36 @@ pub const Pool = struct {
|
||||
};
|
||||
```
|
||||
|
||||
`attempt_timeout` uses the `.awake` clock (`.{ .raw = .fromMilliseconds(2000), .clock = .awake }`) so
|
||||
a suspended Pi does not burn the budget.
|
||||
`attempt_timeout` uses the `.awake` clock (`.{ .raw = .fromMilliseconds(2000), .clock = .awake }`) so a suspended Pi does not burn the budget.
|
||||
|
||||
### S4.2 Failover algorithm
|
||||
|
||||
1. Take `now = std.Io.Clock.awake.now(io)`.
|
||||
2. **Pass one**: walk `entries` in order; skip `!enabled`; skip entries whose
|
||||
`health.available(now)` is false.
|
||||
3. **Pass two**, only if pass one attempted nothing: walk again, skipping only `!enabled`. Every
|
||||
endpoint being in backoff must not turn into `SERVFAIL` for every client — a probe is better than
|
||||
a guaranteed failure, and it is how backoff recovers. Test this explicitly.
|
||||
2. **Pass one**: walk `entries` in order; skip `!enabled`; skip entries whose `health.available(now)` is false.
|
||||
3. **Pass two**, only if pass one attempted nothing: walk again, skipping only `!enabled`. Every endpoint being in backoff must not turn into `SERVFAIL` for every client — a probe is better than a guaranteed failure, and it is how backoff recovers. Test this explicitly.
|
||||
4. For each candidate, run `attempt`:
|
||||
- Race `entry.client.exchange(io, query, response_buf)` against
|
||||
`attempt_timeout.sleep(io)` using `std.Io.Select` with a two-arm union, then
|
||||
`race.cancelDiscard()` on the way out (the milestone-1 pattern). The sleep arm winning yields
|
||||
`error.Timeout`.
|
||||
- On success: lock the mutex, `entry.health.recordSuccess(completed_at)`, unlock, return the
|
||||
slice.
|
||||
- Race `entry.client.exchange(io, query, response_buf)` against `attempt_timeout.sleep(io)` using `std.Io.Select` with a two-arm union, then `race.cancelDiscard()` on the way out (the milestone-1 pattern). The sleep arm winning yields `error.Timeout`.
|
||||
- On success: lock the mutex, `entry.health.recordSuccess(completed_at)`, unlock, return the slice.
|
||||
- On error, switch on `transport.group(err)`:
|
||||
- `.peer_fault` → lock, `recordFailure(completed_at, @errorName(err), self.cfg, self.rng.random().int(u32))`,
|
||||
unlock, remember it as `last_fault`, continue to the next candidate.
|
||||
- `.local_resource` → return the error immediately. Do not record. Do not try another upstream:
|
||||
the next one will hit the same wall.
|
||||
- `.peer_fault` → lock, `recordFailure(completed_at, @errorName(err), self.cfg, self.rng.random().int(u32))`, unlock, remember it as `last_fault`, continue to the next candidate.
|
||||
- `.local_resource` → return the error immediately. Do not record. Do not try another upstream: the next one will hit the same wall.
|
||||
- `.cancellation` → return `error.Canceled` immediately. Do not record.
|
||||
- `completed_at` is `std.Io.Clock.awake.now(io)` taken after the attempt returns.
|
||||
5. All candidates exhausted → return `last_fault` (guaranteed non-null: `entries.len > 0` and pass
|
||||
two attempts every enabled entry). If every entry is disabled, return `error.ConnectFailed`.
|
||||
5. All candidates exhausted → return `last_fault` (guaranteed non-null: `entries.len > 0` and pass two attempts every enabled entry). If every entry is disabled, return `error.ConnectFailed`.
|
||||
|
||||
`response_buf` is handed to each attempt in turn; a failed attempt may have written into it, so the
|
||||
returned slice is only meaningful on success. Note this in a comment.
|
||||
`response_buf` is handed to each attempt in turn; a failed attempt may have written into it, so the returned slice is only meaningful on success. Note this in a comment.
|
||||
|
||||
Concurrency: several handler tasks share one `Pool`. The mutex guards health mutation and
|
||||
`snapshot` only — never an in-flight exchange, so a slow upstream cannot block bookkeeping. Entry
|
||||
`client` values must therefore be safe for concurrent `exchange` calls; `DohClient` and `DotClient`
|
||||
own per-instance buffers, so **one `Entry` per concurrent task, or one shared pool with per-entry
|
||||
serialization, is a wiring decision for the orchestrator**. Record in the completion report which
|
||||
one this session assumed (the specified `Entry` layout assumes each entry's client is used by one
|
||||
task at a time; the e2e test in S7 exercises exactly one in-flight query at a time).
|
||||
Concurrency: several handler tasks share one `Pool`. The mutex guards health mutation and `snapshot` only — never an in-flight exchange, so a slow upstream cannot block bookkeeping. Entry `client` values must therefore be safe for concurrent `exchange` calls; `DohClient` and `DotClient` own per-instance buffers, so **one `Entry` per concurrent task, or one shared pool with per-entry serialization, is a wiring decision for the orchestrator**. Record in the completion report which one this session assumed (the specified `Entry` layout assumes each entry's client is used by one task at a time; the e2e test in S7 exercises exactly one in-flight query at a time).
|
||||
|
||||
### S4.3 Tests (in-file, no sockets — fake clients implement `transport.Client`)
|
||||
|
||||
- Priority order: entry with priority 10 is tried before priority 100.
|
||||
- Failover: first fake returns `error.Timeout`, second returns a valid response → the pool returns
|
||||
the second one's bytes; first entry's `consecutive_failures == 1`, second's `total_successes == 1`.
|
||||
- Backoff skip: drive a fake past `failure_threshold`, then assert the next `exchange` does not call
|
||||
that fake (a call counter in the fake) while another entry is available.
|
||||
- Failover: first fake returns `error.Timeout`, second returns a valid response → the pool returns the second one's bytes; first entry's `consecutive_failures == 1`, second's `total_successes == 1`.
|
||||
- Backoff skip: drive a fake past `failure_threshold`, then assert the next `exchange` does not call that fake (a call counter in the fake) while another entry is available.
|
||||
- All-backed-off probe: every entry in backoff → pass two still calls them.
|
||||
- `.local_resource` short-circuit: a fake returning `error.OutOfMemory` makes `exchange` return
|
||||
`error.OutOfMemory`, the second fake is never called, and the first entry's `total_failures`
|
||||
stays 0.
|
||||
- `.cancellation` short-circuit: a fake returning `error.Canceled` returns `error.Canceled` and
|
||||
records nothing.
|
||||
- Attempt timeout: a fake that sleeps longer than `attempt_timeout` yields a recorded
|
||||
`error.Timeout` peer fault and moves on. (Needs a live `Io` — use `std.Io.Threaded` inside the
|
||||
test; no sockets are involved, so this stays in the default `zig build test`.)
|
||||
- `.local_resource` short-circuit: a fake returning `error.OutOfMemory` makes `exchange` return `error.OutOfMemory`, the second fake is never called, and the first entry's `total_failures` stays 0.
|
||||
- `.cancellation` short-circuit: a fake returning `error.Canceled` returns `error.Canceled` and records nothing.
|
||||
- Attempt timeout: a fake that sleeps longer than `attempt_timeout` yields a recorded `error.Timeout` peer fault and moves on. (Needs a live `Io` — use `std.Io.Threaded` inside the test; no sockets are involved, so this stays in the default `zig build test`.)
|
||||
- All entries disabled → `error.ConnectFailed`.
|
||||
- `snapshot` reports the counters set by the preceding cases.
|
||||
|
||||
@@ -606,9 +486,7 @@ task at a time; the e2e test in S7 exercises exactly one in-flight query at a ti
|
||||
|
||||
## Session S5: `src/server/handler.zig`
|
||||
|
||||
The serving handler: bytes in from a listener, validated bytes out. It owns no socket and no clock
|
||||
beyond what it passes to the upstream. Phase 7 extends this file with filtering, cache, local
|
||||
records and logging — nothing of that appears here.
|
||||
The serving handler: bytes in from a listener, validated bytes out. It owns no socket and no clock beyond what it passes to the upstream. Phase 7 extends this file with filtering, cache, local records and logging — nothing of that appears here.
|
||||
|
||||
### S5.1 API
|
||||
|
||||
@@ -644,52 +522,36 @@ pub const Handler = struct {
|
||||
};
|
||||
```
|
||||
|
||||
`handle` never returns an error: every failure is either a DNS response or a counted drop. That is
|
||||
the "every failure mode visible" rule applied to the hot path.
|
||||
`handle` never returns an error: every failure is either a DNS response or a counted drop. That is the "every failure mode visible" rule applied to the hot path.
|
||||
|
||||
### S5.2 Behavior
|
||||
|
||||
1. `header.parse(query)` fails (< 12 bytes) → `dropped_malformed`, `.drop` (PLAN §6.1: severely
|
||||
truncated is a silent drop).
|
||||
1. `header.parse(query)` fails (< 12 bytes) → `dropped_malformed`, `.drop` (PLAN §6.1: severely truncated is a silent drop).
|
||||
2. `hdr.flags.qr == true` → a response arriving on a listener port; `dropped_malformed`, `.drop`.
|
||||
3. `packet.parse(query)`:
|
||||
- `error.Truncated` → `dropped_malformed`, `.drop`.
|
||||
- any other `WalkError` → FORMERR reply built from `hdr` with **no** echoed question (the
|
||||
question is what failed to parse); `formerr`.
|
||||
- any other `WalkError` → FORMERR reply built from `hdr` with **no** echoed question (the question is what failed to parse); `formerr`.
|
||||
4. `hdr.flags.opcode != .query` → NOTIMP reply echoing nothing; `notimp`.
|
||||
5. `hdr.qdcount != 1` → FORMERR (RFC 9619 makes any other value invalid, in both directions);
|
||||
`formerr`.
|
||||
5. `hdr.qdcount != 1` → FORMERR (RFC 9619 makes any other value invalid, in both directions); `formerr`.
|
||||
6. Forward: `self.upstream.exchange(io, query, response_buf)`.
|
||||
- success → `queries` incremented; then the UDP size check in §5.3; `.reply`.
|
||||
- `transport.group(err) == .cancellation` → `.drop` (the process is shutting down; nothing to
|
||||
say).
|
||||
- `transport.group(err) == .cancellation` → `.drop` (the process is shutting down; nothing to say).
|
||||
- `.peer_fault` or `.local_resource` → SERVFAIL reply echoing the question; `servfail`.
|
||||
7. Every synthesized reply preserves the request ID, opcode and RD bit, sets QR and RA, and echoes
|
||||
the question when one parsed — all of that is already `packet.ResponseBuilder.init`'s contract.
|
||||
When the query carried an OPT record (`packet.findOptRecord` + `edns.parseOpt`), the reply ends
|
||||
with `addOptEcho(opt, opt.do_bit)` so the DO bit passes through (PLAN §6.1).
|
||||
7. Every synthesized reply preserves the request ID, opcode and RD bit, sets QR and RA, and echoes the question when one parsed — all of that is already `packet.ResponseBuilder.init`'s contract. When the query carried an OPT record (`packet.findOptRecord` + `edns.parseOpt`), the reply ends with `addOptEcho(opt, opt.do_bit)` so the DO bit passes through (PLAN §6.1).
|
||||
|
||||
### S5.3 UDP size limit and truncation
|
||||
|
||||
The upstream answer can exceed what the client will accept over UDP.
|
||||
|
||||
- `pub fn udpLimit(query_packet: packet.Packet) u16` — the query's OPT `udp_payload_size` clamped to
|
||||
`[512, 4096]`, or 512 when there is no OPT record (RFC 1035 §4.2.1 / RFC 6891 §6.2.3).
|
||||
- `which == .udp` and the upstream reply is longer than `udpLimit` → build a reply with `TC = 1`,
|
||||
RCODE NOERROR, the question echoed, no answer records, OPT echoed when present; increment
|
||||
`truncated`. The client then retries over TCP, which is exactly what RFC 1035 §4.2.1 prescribes.
|
||||
- `pub fn udpLimit(query_packet: packet.Packet) u16` — the query's OPT `udp_payload_size` clamped to `[512, 4096]`, or 512 when there is no OPT record (RFC 1035 §4.2.1 / RFC 6891 §6.2.3).
|
||||
- `which == .udp` and the upstream reply is longer than `udpLimit` → build a reply with `TC = 1`, RCODE NOERROR, the question echoed, no answer records, OPT echoed when present; increment `truncated`. The client then retries over TCP, which is exactly what RFC 1035 §4.2.1 prescribes.
|
||||
- `which == .tcp` → no size limit beyond `response_buf`.
|
||||
|
||||
The upstream reply lands in `response_buf`; building a truncated reply over it needs a second
|
||||
buffer. Give `handle` no extra parameter: build the truncated reply in a stack `[512]u8` inside
|
||||
`handle` and copy it into `response_buf` before returning. 512 is enough — the message is a header,
|
||||
one question and at most one OPT record, and a question longer than ~270 bytes cannot exist
|
||||
(name ≤ 255).
|
||||
The upstream reply lands in `response_buf`; building a truncated reply over it needs a second buffer. Give `handle` no extra parameter: build the truncated reply in a stack `[512]u8` inside `handle` and copy it into `response_buf` before returning. 512 is enough — the message is a header, one question and at most one OPT record, and a question longer than ~270 bytes cannot exist (name ≤ 255).
|
||||
|
||||
### S5.4 Tests (in-file, fake upstream implementing `transport.Client`)
|
||||
|
||||
Build queries with hand-encoded bytes or `packet.ResponseBuilder`; the fake upstream answers by
|
||||
copying a fixture and calling `packet.setId`.
|
||||
Build queries with hand-encoded bytes or `packet.ResponseBuilder`; the fake upstream answers by copying a fixture and calling `packet.setId`.
|
||||
|
||||
- A query for `example.com` A over UDP returns the fake's bytes unchanged.
|
||||
- A response (QR set) sent to the handler → `.drop`, `dropped_malformed == 1`.
|
||||
@@ -702,8 +564,7 @@ copying a fixture and calling `packet.setId`.
|
||||
- Fake returns `error.OutOfMemory` → SERVFAIL (not a drop).
|
||||
- Fake returns `error.Canceled` → `.drop`.
|
||||
- `udpLimit`: no OPT → 512; OPT 1232 → 1232; OPT 200 → 512; OPT 9000 → 4096.
|
||||
- Over UDP, a fake reply of 900 bytes with no OPT in the query → TC=1 reply, ancount 0, question
|
||||
echoed, `truncated == 1`; the same reply over TCP passes through untouched.
|
||||
- Over UDP, a fake reply of 900 bytes with no OPT in the query → TC=1 reply, ancount 0, question echoed, `truncated == 1`; the same reply over TCP passes through untouched.
|
||||
- DO bit set in the query's OPT → set in the SERVFAIL reply's OPT.
|
||||
- Every synthesized reply re-parses with `packet.parse`.
|
||||
|
||||
@@ -718,8 +579,7 @@ copying a fixture and calling `packet.setId`.
|
||||
|
||||
## Session S6: `src/server/udp_server.zig`, `src/server/tcp_server.zig`
|
||||
|
||||
Listeners. Both take a `*handler.Handler`, both use a bounded pool of in-flight tasks so one slow
|
||||
upstream cannot stall the listener, and both count every failure.
|
||||
Listeners. Both take a `*handler.Handler`, both use a bounded pool of in-flight tasks so one slow upstream cannot stall the listener, and both count every failure.
|
||||
|
||||
### S6.1 udp_server.zig
|
||||
|
||||
@@ -774,22 +634,13 @@ pub const UdpServer = struct {
|
||||
```
|
||||
|
||||
Loop:
|
||||
1. `const msg = self.socket.receive(io, slot_query_buf)` into a claimed slot. `error.Canceled` →
|
||||
return. Any other error → `receive_errors`, log, continue (a per-datagram error must not kill the
|
||||
listener).
|
||||
2. `msg.flags.trunc` → `dropped_oversize`, release the slot, continue. A truncated datagram cannot
|
||||
be parsed reliably and answering it would be a guess.
|
||||
3. No free slot → `dropped_no_slot`, continue. Do not queue: an unbounded queue is the cloudflared
|
||||
failure mode (PLAN §1) in a different costume.
|
||||
4. `group.concurrent(io, respondOne, .{ self, io, slot_index })` — an `std.Io.Group` owned by
|
||||
`serve`. `respondOne` calls `self.handler.handle(io, .udp, query, reply)`, sends `.reply` with
|
||||
`self.socket.send(io, &slot.from, bytes)` (a send failure increments `send_errors` and is logged
|
||||
at debug, deduplicated by the caller later), then releases the slot.
|
||||
5. `serve` awaits the group before returning, under cancel protection so in-flight replies are not
|
||||
abandoned mid-send.
|
||||
1. `const msg = self.socket.receive(io, slot_query_buf)` into a claimed slot. `error.Canceled` → return. Any other error → `receive_errors`, log, continue (a per-datagram error must not kill the listener).
|
||||
2. `msg.flags.trunc` → `dropped_oversize`, release the slot, continue. A truncated datagram cannot be parsed reliably and answering it would be a guess.
|
||||
3. No free slot → `dropped_no_slot`, continue. Do not queue: an unbounded queue is the cloudflared failure mode (PLAN §1) in a different costume.
|
||||
4. `group.concurrent(io, respondOne, .{ self, io, slot_index })` — an `std.Io.Group` owned by `serve`. `respondOne` calls `self.handler.handle(io, .udp, query, reply)`, sends `.reply` with `self.socket.send(io, &slot.from, bytes)` (a send failure increments `send_errors` and is logged at debug, deduplicated by the caller later), then releases the slot.
|
||||
5. `serve` awaits the group before returning, under cancel protection so in-flight replies are not abandoned mid-send.
|
||||
|
||||
`deinit` closes the socket and frees the slots. Closing the socket makes a blocked `receive` fail;
|
||||
the loop treats that as a stop signal.
|
||||
`deinit` closes the socket and frees the slots. Closing the socket makes a blocked `receive` fail; the loop treats that as a stop signal.
|
||||
|
||||
### S6.2 tcp_server.zig
|
||||
|
||||
@@ -824,40 +675,25 @@ pub const TcpServer = struct {
|
||||
};
|
||||
```
|
||||
|
||||
Accept loop: `self.server.accept(io)`; `error.Canceled` or `error.SocketNotListening` → return
|
||||
(`deinit` shuts the listening socket down, which is the documented way to unblock a pending accept —
|
||||
API notes). At capacity → close the stream immediately, `rejected_at_capacity`. Otherwise a group
|
||||
task per connection.
|
||||
Accept loop: `self.server.accept(io)`; `error.Canceled` or `error.SocketNotListening` → return (`deinit` shuts the listening socket down, which is the documented way to unblock a pending accept — API notes). At capacity → close the stream immediately, `rejected_at_capacity`. Otherwise a group task per connection.
|
||||
|
||||
Per connection (RFC 7766 §6.2.1.1 — a connection may carry several queries, handled serially):
|
||||
1. Read the 2-byte length prefix. The read is raced against `options.idle_timeout` with
|
||||
`std.Io.Select`; the sleep arm winning means `idle_timeouts` and a clean close.
|
||||
1. Read the 2-byte length prefix. The read is raced against `options.idle_timeout` with `std.Io.Select`; the sleep arm winning means `idle_timeouts` and a clean close.
|
||||
2. `len == 0` → close (`connection_errors`). `len > transport.max_message_len` is impossible (u16).
|
||||
3. Read exactly `len` bytes; a short read → close, `connection_errors`.
|
||||
4. `handler.handle(io, .tcp, query, reply)`; `.drop` closes the connection (there is nothing to
|
||||
frame); `.reply` writes the 2-byte prefix plus the bytes and flushes.
|
||||
4. `handler.handle(io, .tcp, query, reply)`; `.drop` closes the connection (there is nothing to frame); `.reply` writes the 2-byte prefix plus the bytes and flushes.
|
||||
5. Loop back to step 1.
|
||||
6. Close under cancel protection and release the slot.
|
||||
|
||||
Buffers: each `Conn` owns `query: [transport.max_message_len]u8` and
|
||||
`reply: [transport.max_message_len]u8` plus the stream read/write buffers. That is 128 KiB+ per
|
||||
connection slot; with the default 64 slots that is ~8 MiB, inside the PLAN §18 budget. If a session
|
||||
finds a materially smaller layout that keeps the 65535-byte ceiling, take it and say so.
|
||||
Buffers: each `Conn` owns `query: [transport.max_message_len]u8` and `reply: [transport.max_message_len]u8` plus the stream read/write buffers. That is 128 KiB+ per connection slot; with the default 64 slots that is ~8 MiB, inside the PLAN §18 budget. If a session finds a materially smaller layout that keeps the 65535-byte ceiling, take it and say so.
|
||||
|
||||
### S6.3 Tests
|
||||
|
||||
Unit tests that need no socket go in-file (length-prefix framing helper, capacity accounting).
|
||||
The loopback tests live in `src/server/udp_server_integration_test.zig` and
|
||||
`src/server/tcp_server_integration_test.zig`, both guarded by
|
||||
`if (!build_options.integration) return error.SkipZigTest;`:
|
||||
Unit tests that need no socket go in-file (length-prefix framing helper, capacity accounting). The loopback tests live in `src/server/udp_server_integration_test.zig` and `src/server/tcp_server_integration_test.zig`, both guarded by `if (!build_options.integration) return error.SkipZigTest;`:
|
||||
|
||||
- UDP: bind `127.0.0.1:0`, run `serve` in an `Io.Group` task, send a query from a second bound
|
||||
socket with a fake-upstream handler, assert the reply's ID and question match; then send a 5-byte
|
||||
datagram and assert no reply arrives within a short budget and `dropped_malformed` moved.
|
||||
- TCP: listen on `127.0.0.1:0`, connect, send two length-prefixed queries on one connection, read
|
||||
two length-prefixed replies, assert both match, then close; assert `accepted == 1`.
|
||||
- TCP idle timeout: connect, send nothing, assert the server closes the connection and
|
||||
`idle_timeouts == 1` (use a short `idle_timeout` in the test options).
|
||||
- UDP: bind `127.0.0.1:0`, run `serve` in an `Io.Group` task, send a query from a second bound socket with a fake-upstream handler, assert the reply's ID and question match; then send a 5-byte datagram and assert no reply arrives within a short budget and `dropped_malformed` moved.
|
||||
- TCP: listen on `127.0.0.1:0`, connect, send two length-prefixed queries on one connection, read two length-prefixed replies, assert both match, then close; assert `accepted == 1`.
|
||||
- TCP idle timeout: connect, send nothing, assert the server closes the connection and `idle_timeouts == 1` (use a short `idle_timeout` in the test options).
|
||||
- Both: `deinit` while `serve` is blocked ends `serve` without a hang — the group awaits cleanly.
|
||||
|
||||
### S6.4 Acceptance criteria
|
||||
@@ -871,28 +707,21 @@ The loopback tests live in `src/server/udp_server_integration_test.zig` and
|
||||
|
||||
## Session S7: end-to-end integration test
|
||||
|
||||
`src/server/resolver_integration_test.zig`, guarded by
|
||||
`if (!build_options.integration) return error.SkipZigTest;`. Hermetic: loopback sockets and an
|
||||
in-process fake upstream, no external network.
|
||||
`src/server/resolver_integration_test.zig`, guarded by `if (!build_options.integration) return error.SkipZigTest;`. Hermetic: loopback sockets and an in-process fake upstream, no external network.
|
||||
|
||||
### S7.1 Fake upstream
|
||||
|
||||
In this file: a struct implementing `transport.Client` that parses the incoming query, echoes the
|
||||
question and appends one A record (`93.184.216.34`, TTL 300) with `packet.ResponseBuilder`, and
|
||||
counts calls. A second variant returns a configured `transport.PeerFault` on the first N calls.
|
||||
In this file: a struct implementing `transport.Client` that parses the incoming query, echoes the question and appends one A record (`93.184.216.34`, TTL 300) with `packet.ResponseBuilder`, and counts calls. A second variant returns a configured `transport.PeerFault` on the first N calls.
|
||||
|
||||
### S7.2 The test
|
||||
|
||||
1. `std.Io.Threaded` backend, `io`.
|
||||
2. Two `Pool` entries: entry 0 = always-fails fake (priority 10), entry 1 = good fake (priority 20).
|
||||
3. `Handler` over `pool.client()`.
|
||||
4. `UdpServer` bound to `127.0.0.1:0` and `TcpServer` on `127.0.0.1:0`, both served in one
|
||||
`Io.Group`.
|
||||
5. Client side: send an A query for `example.com` over UDP; assert the reply parses, ID matches,
|
||||
RCODE is NOERROR, ancount is 1, and the A record's rdata is `93.184.216.34`.
|
||||
4. `UdpServer` bound to `127.0.0.1:0` and `TcpServer` on `127.0.0.1:0`, both served in one `Io.Group`.
|
||||
5. Client side: send an A query for `example.com` over UDP; assert the reply parses, ID matches, RCODE is NOERROR, ancount is 1, and the A record's rdata is `93.184.216.34`.
|
||||
6. Send the same query length-prefixed over TCP; assert the same.
|
||||
7. Assert failover happened: entry 0's `consecutive_failures >= 1` and `backoff_until != null`;
|
||||
entry 1's `total_successes >= 2`. Read them through `pool.snapshot`.
|
||||
7. Assert failover happened: entry 0's `consecutive_failures >= 1` and `backoff_until != null`; entry 1's `total_successes >= 2`. Read them through `pool.snapshot`.
|
||||
8. Send a third query and assert entry 0's call counter did not grow — it is in backoff.
|
||||
9. `deinit` both servers, cancel and await the group, `threaded.deinit()` — the test must not hang.
|
||||
|
||||
@@ -958,66 +787,26 @@ No session touches milestone 1 or milestone 2 files. A needed change there is re
|
||||
|
||||
- No caching of any kind (Phase 6). The handler forwards every query.
|
||||
- No filtering, blocklists, rules, or blocked-response synthesis (Phase 5).
|
||||
- No query logging, no SQLite, no storage, no config file or ZON parsing (Phase 4). Timeouts,
|
||||
ports and pool entries arrive as parameters or named constants.
|
||||
- No query logging, no SQLite, no storage, no config file or ZON parsing (Phase 4). Timeouts, ports and pool entries arrive as parameters or named constants.
|
||||
- No rate limiting (Phase 6) — `src/server/rate_limiter.zig` is not created here.
|
||||
- No local DoH or DoT **server** endpoints (Phase 9). Only upstream DoH/DoT clients are in scope;
|
||||
`platform/tls_server.zig` is untouched.
|
||||
- No local DoH or DoT **server** endpoints (Phase 9). Only upstream DoH/DoT clients are in scope; `platform/tls_server.zig` is untouched.
|
||||
- No local records, no conditional forwarding, no CNAME uncloaking (Phase 5/7).
|
||||
- No web UI, no REST API, no `/metrics`. `Pool.snapshot` exists so Phase 8 has something to read;
|
||||
it is not exposed anywhere yet.
|
||||
- No signal handling, no `src/server/shutdown.zig`, no `main.zig` wiring. Servers stop through
|
||||
`deinit` plus group cancellation; process lifetime is Phase 7's problem.
|
||||
- No DNS name resolution for upstream hosts — DoT endpoints take IP literals, DoH resolution is
|
||||
`std.http.Client`'s business.
|
||||
- No web UI, no REST API, no `/metrics`. `Pool.snapshot` exists so Phase 8 has something to read; it is not exposed anywhere yet.
|
||||
- No signal handling, no `src/server/shutdown.zig`, no `main.zig` wiring. Servers stop through `deinit` plus group cancellation; process lifetime is Phase 7's problem.
|
||||
- No DNS name resolution for upstream hosts — DoT endpoints take IP literals, DoH resolution is `std.http.Client`'s business.
|
||||
- No HTTP/2, no DoQ, no connection reuse for DoT, no DNSSEC validation (DO bit passes through).
|
||||
- No 0x20 query-name randomization. Case-insensitive comparison per RFC 4343 is required; generating
|
||||
mixed-case queries is not.
|
||||
- No changes to `src/dns/`. If a helper is missing there, report it — do not add transport-aware
|
||||
code to the pure core.
|
||||
- No 0x20 query-name randomization. Case-insensitive comparison per RFC 4343 is required; generating mixed-case queries is not.
|
||||
- No changes to `src/dns/`. If a helper is missing there, report it — do not add transport-aware code to the pure core.
|
||||
|
||||
## As built
|
||||
|
||||
The implementation matches the spec with these review-driven refinements (three
|
||||
review rounds; findings went 9 → 3 → 0):
|
||||
The implementation matches the spec with these review-driven refinements (three review rounds; findings went 9 → 3 → 0):
|
||||
|
||||
- `pool.zig`: each `Entry` carries a `busy: std.Io.Mutex` held for the whole
|
||||
attempt against that entry, so one entry's client (and its buffers/TLS state)
|
||||
is never used by two tasks at once. Pass one re-checks `available` with a
|
||||
fresh `now` after acquiring `busy` (the wait can outlive the health it was
|
||||
admitted under); pass two probes without a re-check by design. Lock order is
|
||||
always `busy` then `Pool.mutex`; the pool mutex still guards health and
|
||||
`snapshot` only.
|
||||
- `health.zig`: staleness rules are symmetric. The newest outcome is the @max
|
||||
of `last_success_at`/`last_error_at`. A stale success counts into
|
||||
totals/window and @max-updates `last_success_at`, but does not clear
|
||||
`consecutive_failures`/`backoff_until`. A stale failure counts into
|
||||
totals/window and @max-updates `last_error_at`, but only increments
|
||||
`consecutive_failures`/extends backoff when no newer success exists, and
|
||||
never overwrites a newer failure's error text.
|
||||
- `udp_server.zig`: each slot's reply buffer is
|
||||
`[transport.max_message_len]u8` (not `max_datagram`) so an oversized upstream
|
||||
reply reaches the handler and becomes a TC=1 reply instead of SERVFAIL
|
||||
(64 slots ≈ 4.4 MiB, inside the PLAN §18 budget). A `dropped_handler` stat
|
||||
counts `.drop` outcomes from the handler.
|
||||
- `tcp_server.zig`: a shutdown flag, set under the slot mutex before the
|
||||
active-connection scan, closes the accept-vs-deinit race — a stream accepted
|
||||
during shutdown is closed immediately instead of claimed.
|
||||
- `handler.zig`: OPT validation walks every record section. OPT in answer or
|
||||
authority, more than one OPT, a non-root OPT owner, or an unparseable OPT →
|
||||
FORMERR (RFC 6891); only a genuinely absent OPT takes the no-OPT path.
|
||||
- `dot_client.zig`: handshake `ReadFailed`/`WriteFailed` unwrap the stream's
|
||||
stored cause through `transport.mapLocal` first, and certificate-bundle
|
||||
`OutOfMemory` is a local resource, not a peer fault.
|
||||
- `transport.zig`: `Endpoint.parse` rejects userinfo/query/fragment delimiters
|
||||
(`@`, `?`, `#`) in the authority, and `?`/`#` in a DoH path.
|
||||
- `dot_client.zig` (follow-up, tls_name commit): `DotClient.init` takes a
|
||||
`tls_name`; it is the SNI and certificate-verification name, while the dial
|
||||
target stays `endpoint.host`. Empty keeps the endpoint host, which is the
|
||||
behavior described above. The same follow-up fixed a send bug this file
|
||||
had from the start, invisible until a DoT handshake first succeeded:
|
||||
`tls.Client.flush` only encrypts into the socket writer's buffer and never
|
||||
flushes it, so the query never left the process and the peer eventually
|
||||
closed the connection (`ReceiveFailed`/`EndOfStream`). `TlsStream.flush` now
|
||||
does both flushes and `exchange` calls it; a hermetic loopback test in
|
||||
`tls_client_integration_test.zig` covers it.
|
||||
- `pool.zig`: each `Entry` carries a `busy: std.Io.Mutex` held for the whole attempt against that entry, so one entry's client (and its buffers/TLS state) is never used by two tasks at once. Pass one re-checks `available` with a fresh `now` after acquiring `busy` (the wait can outlive the health it was admitted under); pass two probes without a re-check by design. Lock order is always `busy` then `Pool.mutex`; the pool mutex still guards health and `snapshot` only.
|
||||
- `health.zig`: staleness rules are symmetric. The newest outcome is the @max of `last_success_at`/`last_error_at`. A stale success counts into totals/window and @max-updates `last_success_at`, but does not clear `consecutive_failures`/`backoff_until`. A stale failure counts into totals/window and @max-updates `last_error_at`, but only increments `consecutive_failures`/extends backoff when no newer success exists, and never overwrites a newer failure's error text.
|
||||
- `udp_server.zig`: each slot's reply buffer is `[transport.max_message_len]u8` (not `max_datagram`) so an oversized upstream reply reaches the handler and becomes a TC=1 reply instead of SERVFAIL (64 slots ≈ 4.4 MiB, inside the PLAN §18 budget). A `dropped_handler` stat counts `.drop` outcomes from the handler.
|
||||
- `tcp_server.zig`: a shutdown flag, set under the slot mutex before the active-connection scan, closes the accept-vs-deinit race — a stream accepted during shutdown is closed immediately instead of claimed.
|
||||
- `handler.zig`: OPT validation walks every record section. OPT in answer or authority, more than one OPT, a non-root OPT owner, or an unparseable OPT → FORMERR (RFC 6891); only a genuinely absent OPT takes the no-OPT path.
|
||||
- `dot_client.zig`: handshake `ReadFailed`/`WriteFailed` unwrap the stream's stored cause through `transport.mapLocal` first, and certificate-bundle `OutOfMemory` is a local resource, not a peer fault.
|
||||
- `transport.zig`: `Endpoint.parse` rejects userinfo/query/fragment delimiters (`@`, `?`, `#`) in the authority, and `?`/`#` in a DoH path.
|
||||
- `dot_client.zig` (follow-up, tls_name commit): `DotClient.init` takes a `tls_name`; it is the SNI and certificate-verification name, while the dial target stays `endpoint.host`. Empty keeps the endpoint host, which is the behavior described above. The same follow-up fixed a send bug this file had from the start, invisible until a DoT handshake first succeeded: `tls.Client.flush` only encrypts into the socket writer's buffer and never flushes it, so the query never left the process and the peer eventually closed the connection (`ReceiveFailed`/`EndOfStream`). `TlsStream.flush` now does both flushes and `exchange` calls it; a hermetic loopback test in `tls_client_integration_test.zig` covers it.
|
||||
|
||||
+204
-625
File diff suppressed because it is too large
Load Diff
+165
-586
File diff suppressed because it is too large
Load Diff
+83
-370
@@ -1,116 +1,52 @@
|
||||
# Milestone 6: Cache, Rate Limiter, Query Logger, Retention, Disk Monitor, Log Sink
|
||||
|
||||
PLAN Phase 6. Build every runtime component between the resolver core and the web layer: the TTL
|
||||
cache (§8), the per-client DNS rate limiter (§10), the async query logger with backpressure
|
||||
(§11.4), retention (§11.5), the disk monitor and the rotating log sink with upstream-error dedup
|
||||
(§11.6). Nothing is wired into the query path — composition is Phase 7. Every component is built
|
||||
complete, tested standalone, and exposed through accessors Phase 7 and Phase 8 will consume.
|
||||
PLAN Phase 6. Build every runtime component between the resolver core and the web layer: the TTL cache (§8), the per-client DNS rate limiter (§10), the async query logger with backpressure (§11.4), retention (§11.5), the disk monitor and the rotating log sink with upstream-error dedup (§11.6). Nothing is wired into the query path — composition is Phase 7. Every component is built complete, tested standalone, and exposed through accessors Phase 7 and Phase 8 will consume.
|
||||
|
||||
## Sessions
|
||||
|
||||
Eight sessions. Wave 1 starts five in parallel: S1 (cache), S2 (rate limiter), S3 (queries repo),
|
||||
S5 (statfs + disk monitor), S6 (log sink). Wave 2 starts when S3 verifies: S4 (logger) and S7
|
||||
(retention) in parallel — both consume S3's repo and nothing of each other. S8 (integration) runs
|
||||
last. The orchestrator — not any session — wires `src/tests.zig` and installs the log sink's
|
||||
`std_options` hook in `src/main.zig`. (`validate.zig:221` already rejects
|
||||
`rate_window_seconds` outside 1–`max_rate_window_seconds`; no rule is missing.)
|
||||
Eight sessions. Wave 1 starts five in parallel: S1 (cache), S2 (rate limiter), S3 (queries repo), S5 (statfs + disk monitor), S6 (log sink). Wave 2 starts when S3 verifies: S4 (logger) and S7 (retention) in parallel — both consume S3's repo and nothing of each other. S8 (integration) runs last. The orchestrator — not any session — wires `src/tests.zig` and installs the log sink's `std_options` hook in `src/main.zig`. (`validate.zig:221` already rejects `rate_window_seconds` outside 1–`max_rate_window_seconds`; no rule is missing.)
|
||||
|
||||
Every later session is written against **this spec**, not against another session's source. Where
|
||||
this spec and a built file disagree after a session verifies, the built file wins and the
|
||||
orchestrator records the difference in "## As built".
|
||||
Every later session is written against **this spec**, not against another session's source. Where this spec and a built file disagree after a session verifies, the built file wins and the orchestrator records the difference in "## As built".
|
||||
|
||||
## Design invariants
|
||||
|
||||
- **Pure core.** `src/cache/dns_cache.zig` and `src/server/rate_limiter.zig` take timestamps as
|
||||
parameters and hold no `std.Io`, no clock, no socket. `std.Io` appears only in
|
||||
`src/storage/logger.zig`, `src/storage/disk_monitor.zig`, `src/storage/retention.zig`,
|
||||
`src/platform/logging.zig`, `src/platform/statfs.zig` (its test), and the S8 test file.
|
||||
- **Neither the cache nor the limiter is thread-safe.** Phase 7 decides the locking when it wires
|
||||
them. Both files state that in their module doc comment.
|
||||
- **No allocation on the hit path.** `DnsCache.get` and `RateLimiter.check` allocate nothing.
|
||||
`DnsCache.put` allocates exactly one buffer for the stored response bytes.
|
||||
- **Every failure mode is counted.** Dropped log entries, refused queries, evictions, expired
|
||||
entries, deduplicated log lines, failed disk samples — a counter each, readable by Phase 8.
|
||||
- **No `std.log.err` in any file this milestone writes.** A condition returned as a typed error
|
||||
logs at `warn` at most. `err` stays reserved for swallowed failures, and this milestone swallows
|
||||
none.
|
||||
- **Timestamps follow the house style** (health.zig): `std.Io.Timestamp` parameters, arithmetic on
|
||||
`.nanoseconds`, wall seconds for DB rows via `Clock.real.now(io).toSeconds()`, monotonic
|
||||
decisions on `.awake`, long schedules on `.boot`.
|
||||
- **Background loops follow `manager.runScheduler`'s shape**: `Cancelable!void`, non-cancel
|
||||
failures log at `warn` and the loop continues, cancellation returns. Phase 7 starts them.
|
||||
- **Pure core.** `src/cache/dns_cache.zig` and `src/server/rate_limiter.zig` take timestamps as parameters and hold no `std.Io`, no clock, no socket. `std.Io` appears only in `src/storage/logger.zig`, `src/storage/disk_monitor.zig`, `src/storage/retention.zig`, `src/platform/logging.zig`, `src/platform/statfs.zig` (its test), and the S8 test file.
|
||||
- **Neither the cache nor the limiter is thread-safe.** Phase 7 decides the locking when it wires them. Both files state that in their module doc comment.
|
||||
- **No allocation on the hit path.** `DnsCache.get` and `RateLimiter.check` allocate nothing. `DnsCache.put` allocates exactly one buffer for the stored response bytes.
|
||||
- **Every failure mode is counted.** Dropped log entries, refused queries, evictions, expired entries, deduplicated log lines, failed disk samples — a counter each, readable by Phase 8.
|
||||
- **No `std.log.err` in any file this milestone writes.** A condition returned as a typed error logs at `warn` at most. `err` stays reserved for swallowed failures, and this milestone swallows none.
|
||||
- **Timestamps follow the house style** (health.zig): `std.Io.Timestamp` parameters, arithmetic on `.nanoseconds`, wall seconds for DB rows via `Clock.real.now(io).toSeconds()`, monotonic decisions on `.awake`, long schedules on `.boot`.
|
||||
- **Background loops follow `manager.runScheduler`'s shape**: `Cancelable!void`, non-cancel failures log at `warn` and the loop continues, cancellation returns. Phase 7 starts them.
|
||||
|
||||
## Resolved PLAN ambiguities (rulings for this milestone)
|
||||
|
||||
1. **Client-row pruning is Phase 7.** §7.2's "retention drops `hand_edited=0` clients" deletes
|
||||
from `config.db`, which §3.6 walls off from retention churn. The rows it prunes are created by
|
||||
auto-materialization, which is Phase 7; the pruning ships with it. §11.5 retention here touches
|
||||
`querylog.db` only.
|
||||
2. **Free space drives the thresholds; sizes are gauges.** The monitor samples `statvfs` free
|
||||
bytes for the warn/critical decision and records DB and log-directory sizes as gauges for
|
||||
Phase 8's health payload. Both are sampled in the same pass.
|
||||
3. **Flush tuning is comptime.** `flush_batch = 100`, `flush_interval_ms = 100` as public
|
||||
constants in `logger.zig`. §12.1 defines no config keys for them and a household deployment
|
||||
never tunes them.
|
||||
4. **The SSE seam is the transformed entry.** Privacy transforms run at enqueue, so every entry in
|
||||
the queue is already presentable. Phase 8 subscribes by wrapping `Logger.log` — no hook, no
|
||||
callback, no dead vtable.
|
||||
5. **The positive-TTL sanity ceiling is 86 400 seconds**, the same `max_ttl_seconds` constant
|
||||
`validate.zig:196` already enforces on `blocking.ttl` and `cache.negative_ttl_max`. Comptime
|
||||
constant in `dns_cache.zig` with a comment naming that origin. No new config key.
|
||||
6. **`negative_ttl_max = 0` disables negative caching** (the response is not stored), not the cap.
|
||||
`validate.zig` already caps the field at `max_ttl_seconds`, so "no cap" would be expressible as
|
||||
86 400 anyway; "0 disables" only has meaning as an off switch.
|
||||
7. **`cache.size` counts entries**, not bytes. 10 000 entries at the observed household response
|
||||
sizes is single-digit megabytes; a byte budget would add config surface for nothing.
|
||||
8. **The DNS limiter is a fixed window.** §10 reserves the token bucket for the API (Phase 8);
|
||||
"1000 per 60 s" plus "stale-key windowed sweep" describes a counter that resets each window.
|
||||
A fixed window admits at most 2× the limit across a window boundary, which is acceptable for
|
||||
abuse protection at household scale and costs half the state of a sliding window.
|
||||
9. **Upstream-error dedup lives in the log sink.** It is a property of how lines reach the
|
||||
operator, not of the upstream module: `platform/logging.zig` deduplicates warn-and-above lines
|
||||
from the upstream scopes at one per key per minute. No milestone-3 file is edited.
|
||||
10. **Schedules: retention daily, checkpoint after every prune, VACUUM every 7th pass.** §11.5
|
||||
names no interval. Daily matches the granularity of `retention_days`; `wal_checkpoint
|
||||
(TRUNCATE)` after each prune keeps the WAL from monopolizing the disk the monitor watches; a
|
||||
nightly full VACUUM would rewrite the whole file on an SD card every day, so VACUUM runs on
|
||||
every 7th retention pass.
|
||||
1. **Client-row pruning is Phase 7.** §7.2's "retention drops `hand_edited=0` clients" deletes from `config.db`, which §3.6 walls off from retention churn. The rows it prunes are created by auto-materialization, which is Phase 7; the pruning ships with it. §11.5 retention here touches `querylog.db` only.
|
||||
2. **Free space drives the thresholds; sizes are gauges.** The monitor samples `statvfs` free bytes for the warn/critical decision and records DB and log-directory sizes as gauges for Phase 8's health payload. Both are sampled in the same pass.
|
||||
3. **Flush tuning is comptime.** `flush_batch = 100`, `flush_interval_ms = 100` as public constants in `logger.zig`. §12.1 defines no config keys for them and a household deployment never tunes them.
|
||||
4. **The SSE seam is the transformed entry.** Privacy transforms run at enqueue, so every entry in the queue is already presentable. Phase 8 subscribes by wrapping `Logger.log` — no hook, no callback, no dead vtable.
|
||||
5. **The positive-TTL sanity ceiling is 86 400 seconds**, the same `max_ttl_seconds` constant `validate.zig:196` already enforces on `blocking.ttl` and `cache.negative_ttl_max`. Comptime constant in `dns_cache.zig` with a comment naming that origin. No new config key.
|
||||
6. **`negative_ttl_max = 0` disables negative caching** (the response is not stored), not the cap. `validate.zig` already caps the field at `max_ttl_seconds`, so "no cap" would be expressible as 86 400 anyway; "0 disables" only has meaning as an off switch.
|
||||
7. **`cache.size` counts entries**, not bytes. 10 000 entries at the observed household response sizes is single-digit megabytes; a byte budget would add config surface for nothing.
|
||||
8. **The DNS limiter is a fixed window.** §10 reserves the token bucket for the API (Phase 8); "1000 per 60 s" plus "stale-key windowed sweep" describes a counter that resets each window. A fixed window admits at most 2× the limit across a window boundary, which is acceptable for abuse protection at household scale and costs half the state of a sliding window.
|
||||
9. **Upstream-error dedup lives in the log sink.** It is a property of how lines reach the operator, not of the upstream module: `platform/logging.zig` deduplicates warn-and-above lines from the upstream scopes at one per key per minute. No milestone-3 file is edited.
|
||||
10. **Schedules: retention daily, checkpoint after every prune, VACUUM every 7th pass.** §11.5 names no interval. Daily matches the granularity of `retention_days`; `wal_checkpoint (TRUNCATE)` after each prune keeps the WAL from monopolizing the disk the monitor watches; a nightly full VACUUM would rewrite the whole file on an SD card every day, so VACUUM runs on every 7th retention pass.
|
||||
|
||||
## Verified 0.16.0 stdlib facts (see specs/research/zig-0.16-api-notes.md, "Phase 6 verifications")
|
||||
|
||||
- `std.Io.Queue(Elem)` (Io.zig:2184): bounded ring over a caller array. `put(q, io, elems, 0)`
|
||||
never blocks and returns 0 when full (Io.zig:2218); `getOne` blocks; `close(io)` unblocks it
|
||||
with `error.Closed`. Elements are copied raw (Io.zig:2189) — **queue elements must be
|
||||
self-contained values**, no slices into caller memory.
|
||||
- `std.Io.Condition` has **no `timedWait`**. The timed primitive is `Event.waitTimeout`
|
||||
(Io.zig:1827). The logger's flush interval uses `Select` racing `getOne` against a
|
||||
`Clock.Duration` sleep, the `fetchWithin` pattern from manager.zig:722.
|
||||
- **No statvfs in std.** `src/platform/statfs.zig` declares `extern fn statvfs` against libc
|
||||
(every build already links libc for sqlite; glibc and musl share the 64-bit POSIX layout).
|
||||
Free bytes = `f_bavail * f_frsize`.
|
||||
- **No append mode.** Log sink append: `openFile(io, path, .{ .mode = .write_only })`,
|
||||
`file.writer(io, &buf)`, `w.seekTo(try file.length(io))`. `Dir.rename` + `Dir.deleteFile`
|
||||
rotate. One owner per writer — the sink serializes behind its own mutex.
|
||||
- **No fixed-capacity hash map in std.** Bounded structures use a fixed slot array plus
|
||||
`std.HashMapUnmanaged` from key to slot index with `ensureTotalCapacity` at init; the CLOCK
|
||||
hand walks the slot array, never the map. Map modification invalidates live iterators
|
||||
(hash_map.zig:496).
|
||||
- `packet.decrementTtls(bytes, elapsed_seconds) ParseError!?u32` (packet.zig:234) ages every
|
||||
record in place, skips OPT, validates before writing, returns the smallest resulting TTL; on
|
||||
error the buffer is partly aged and must be discarded. With `elapsed = 0` it is a pure
|
||||
validate-and-min-TTL pass. `packet.setId` (packet.zig:202) rewrites the transaction ID.
|
||||
- `db.zig`: `Stmt.reset` clears bindings — rebind everything each iteration. `Db`/`Stmt` are not
|
||||
thread-safe; the logger's writer task owns its `Db` handle exclusively. A batch is one
|
||||
`Tx.begin` + reset/rebind/step loop + `commit`.
|
||||
- `address.NetAddress.Key` is `[17]u8` covering both families; `fromIp` folds IPv4-mapped IPv6 to
|
||||
`.ip4`, so one client is one key.
|
||||
- `std.Io.Queue(Elem)` (Io.zig:2184): bounded ring over a caller array. `put(q, io, elems, 0)` never blocks and returns 0 when full (Io.zig:2218); `getOne` blocks; `close(io)` unblocks it with `error.Closed`. Elements are copied raw (Io.zig:2189) — **queue elements must be self-contained values**, no slices into caller memory.
|
||||
- `std.Io.Condition` has **no `timedWait`**. The timed primitive is `Event.waitTimeout` (Io.zig:1827). The logger's flush interval uses `Select` racing `getOne` against a `Clock.Duration` sleep, the `fetchWithin` pattern from manager.zig:722.
|
||||
- **No statvfs in std.** `src/platform/statfs.zig` declares `extern fn statvfs` against libc (every build already links libc for sqlite; glibc and musl share the 64-bit POSIX layout). Free bytes = `f_bavail * f_frsize`.
|
||||
- **No append mode.** Log sink append: `openFile(io, path, .{ .mode = .write_only })`, `file.writer(io, &buf)`, `w.seekTo(try file.length(io))`. `Dir.rename` + `Dir.deleteFile` rotate. One owner per writer — the sink serializes behind its own mutex.
|
||||
- **No fixed-capacity hash map in std.** Bounded structures use a fixed slot array plus `std.HashMapUnmanaged` from key to slot index with `ensureTotalCapacity` at init; the CLOCK hand walks the slot array, never the map. Map modification invalidates live iterators (hash_map.zig:496).
|
||||
- `packet.decrementTtls(bytes, elapsed_seconds) ParseError!?u32` (packet.zig:234) ages every record in place, skips OPT, validates before writing, returns the smallest resulting TTL; on error the buffer is partly aged and must be discarded. With `elapsed = 0` it is a pure validate-and-min-TTL pass. `packet.setId` (packet.zig:202) rewrites the transaction ID.
|
||||
- `db.zig`: `Stmt.reset` clears bindings — rebind everything each iteration. `Db`/`Stmt` are not thread-safe; the logger's writer task owns its `Db` handle exclusively. A batch is one `Tx.begin` + reset/rebind/step loop + `commit`.
|
||||
- `address.NetAddress.Key` is `[17]u8` covering both families; `fromIp` folds IPv4-mapped IPv6 to `.ip4`, so one client is one key.
|
||||
|
||||
---
|
||||
|
||||
## Session S1: DNS cache — `src/cache/dns_cache.zig`
|
||||
|
||||
New directory `src/cache/`. Imports: `std`, `../dns/packet.zig`, `../dns/types.zig`,
|
||||
`../dns/record.zig`, `../config/model.zig`. No `std.Io`.
|
||||
New directory `src/cache/`. Imports: `std`, `../dns/packet.zig`, `../dns/types.zig`, `../dns/record.zig`, `../config/model.zig`. No `std.Io`.
|
||||
|
||||
### S1.1 Key
|
||||
|
||||
@@ -123,8 +59,7 @@ pub fn buildKey(buf: *[max_key_len]u8, qname: []const u8, qtype: u16, qclass: u1
|
||||
do_bit: bool, ecs: ?[]const u8) []const u8;
|
||||
```
|
||||
|
||||
`buildKey` lowercases `A`–`Z` while copying and strips one trailing dot. Asserts `qname.len <=
|
||||
255` and `ecs.?.len <= 40` — the caller has already validated both through the packet parser.
|
||||
`buildKey` lowercases `A`–`Z` while copying and strips one trailing dot. Asserts `qname.len <= 255` and `ecs.?.len <= 40` — the caller has already validated both through the packet parser.
|
||||
|
||||
### S1.2 Classification
|
||||
|
||||
@@ -134,15 +69,7 @@ pub const Class = struct { ttl_seconds: u32, negative: bool };
|
||||
pub fn classify(response: []const u8, negative_ttl_max: u32) ?Class;
|
||||
```
|
||||
|
||||
Rules, in order: parse the header — rcode NOERROR with at least one answer record is positive;
|
||||
rcode NXDOMAIN, or NOERROR with zero answers, is negative; every other rcode returns null.
|
||||
Positive: `ttl = min(packet.decrementTtls(copy, 0) result, max_ttl)` — since `decrementTtls`
|
||||
mutates, `classify` runs it on a stack copy (`[65535]u8` is too big for the stack; instead compute
|
||||
the min TTL by iterating records via the parser without mutation — `record` module iteration, skip
|
||||
OPT). A parse error returns null. Negative: min(SOA record TTL, SOA MINIMUM) from the authority section per RFC 2308 §5, capped by
|
||||
`negative_ttl_max`; no SOA → null; `negative_ttl_max == 0` → null (ruling 6). A TTL of 0 → null.
|
||||
A TTL with the top bit set is read as zero per RFC 2181 §8 (packet.zig:220 leaves that to the
|
||||
caller), on both paths — such a response is not cached.
|
||||
Rules, in order: parse the header — rcode NOERROR with at least one answer record is positive; rcode NXDOMAIN, or NOERROR with zero answers, is negative; every other rcode returns null. Positive: `ttl = min(packet.decrementTtls(copy, 0) result, max_ttl)` — since `decrementTtls` mutates, `classify` runs it on a stack copy (`[65535]u8` is too big for the stack; instead compute the min TTL by iterating records via the parser without mutation — `record` module iteration, skip OPT). A parse error returns null. Negative: min(SOA record TTL, SOA MINIMUM) from the authority section per RFC 2308 §5, capped by `negative_ttl_max`; no SOA → null; `negative_ttl_max == 0` → null (ruling 6). A TTL of 0 → null. A TTL with the top bit set is read as zero per RFC 2181 §8 (packet.zig:220 leaves that to the caller), on both paths — such a response is not cached.
|
||||
|
||||
### S1.3 Storage and API
|
||||
|
||||
@@ -166,26 +93,13 @@ pub const DnsCache = struct {
|
||||
};
|
||||
```
|
||||
|
||||
Slot: inline key `[max_key_len]u8` + `key_len` (no allocation), `bytes: []u8` (gpa-owned clone),
|
||||
`stored_at_s: i64`, `expires_at_s: i64`, `negative: bool`, `referenced: bool`, `occupied: bool`.
|
||||
Index: `HashMapUnmanaged` keyed by the slot's key slice (context hashes the slice) to `u32` slot
|
||||
index, `ensureTotalCapacity(config.size)` at init — no rehash ever. CLOCK: on `get` hit set
|
||||
`referenced`; when `put` finds no free slot, advance the hand clearing `referenced` until an
|
||||
unreferenced slot, evict it (`evictions += 1`). Expired-on-access is freed and counted under
|
||||
`expirations`, then treated as a miss/free slot.
|
||||
Slot: inline key `[max_key_len]u8` + `key_len` (no allocation), `bytes: []u8` (gpa-owned clone), `stored_at_s: i64`, `expires_at_s: i64`, `negative: bool`, `referenced: bool`, `occupied: bool`. Index: `HashMapUnmanaged` keyed by the slot's key slice (context hashes the slice) to `u32` slot index, `ensureTotalCapacity(config.size)` at init — no rehash ever. CLOCK: on `get` hit set `referenced`; when `put` finds no free slot, advance the hand clearing `referenced` until an unreferenced slot, evict it (`evictions += 1`). Expired-on-access is freed and counted under `expirations`, then treated as a miss/free slot.
|
||||
|
||||
`get` computes `elapsed = now_s - stored_at_s`, copies, then `packet.decrementTtls(copy,
|
||||
elapsed)`. An error from it removes the entry, counts `invalid_hits`, returns null — a poisoned
|
||||
entry never serves twice.
|
||||
`get` computes `elapsed = now_s - stored_at_s`, copies, then `packet.decrementTtls(copy, elapsed)`. An error from it removes the entry, counts `invalid_hits`, returns null — a poisoned entry never serves twice.
|
||||
|
||||
### S1.4 Tests (in-file, ≥ 14)
|
||||
|
||||
buildKey lowercases and strips the dot; distinct qtype/qclass/DO/ECS produce distinct keys; ECS
|
||||
absent vs present differ; classify: positive min-TTL over multiple answers; ceiling applied; NXDOMAIN
|
||||
uses SOA minimum; capped by negative_ttl_max; `negative_ttl_max = 0` returns null; SERVFAIL null;
|
||||
zero TTL null; put/get round-trip with TTL decrement asserted via re-parse; expiry is a miss and
|
||||
counted; CLOCK evicts the unreferenced entry, not the referenced one; replace-in-place does not
|
||||
grow `len`; sweep removes only expired; `checkAllAllocationFailures` on init+put.
|
||||
buildKey lowercases and strips the dot; distinct qtype/qclass/DO/ECS produce distinct keys; ECS absent vs present differ; classify: positive min-TTL over multiple answers; ceiling applied; NXDOMAIN uses SOA minimum; capped by negative_ttl_max; `negative_ttl_max = 0` returns null; SERVFAIL null; zero TTL null; put/get round-trip with TTL decrement asserted via re-parse; expiry is a miss and counted; CLOCK evicts the unreferenced entry, not the referenced one; replace-in-place does not grow `len`; sweep removes only expired; `checkAllAllocationFailures` on init+put.
|
||||
|
||||
### S1.5 Acceptance
|
||||
|
||||
@@ -215,31 +129,19 @@ pub const RateLimiter = struct {
|
||||
pub const max_clients = 4096;
|
||||
```
|
||||
|
||||
Fixed window per key: entry `{window_start_ns: i96, count: u32}` in a `HashMapUnmanaged` with
|
||||
capacity `max_clients` ensured at init. `check`: if the entry's window has ended, reset it to the
|
||||
current window. `count >= limit` → refused. Map full and key unknown: allow, count `untracked` —
|
||||
refusing unseen clients because the table is full would let 4096 attackers deny every new device,
|
||||
and a household LAN never has 4096 honest clients; the counter makes the state visible. `sweep`
|
||||
walks and removes stale entries (collect keys first — map modification invalidates iterators).
|
||||
Fixed window per key: entry `{window_start_ns: i96, count: u32}` in a `HashMapUnmanaged` with capacity `max_clients` ensured at init. `check`: if the entry's window has ended, reset it to the current window. `count >= limit` → refused. Map full and key unknown: allow, count `untracked` — refusing unseen clients because the table is full would let 4096 attackers deny every new device, and a household LAN never has 4096 honest clients; the counter makes the state visible. `sweep` walks and removes stale entries (collect keys first — map modification invalidates iterators).
|
||||
|
||||
`std.Io.Timestamp` is a plain value type (`{nanoseconds: i96}`) — using it keeps this file pure;
|
||||
callers pass `.awake` timestamps.
|
||||
`std.Io.Timestamp` is a plain value type (`{nanoseconds: i96}`) — using it keeps this file pure; callers pass `.awake` timestamps.
|
||||
|
||||
Tests (≥ 8): allows up to limit, refuses limit+1; new window resets; v4 and v6 keys independent;
|
||||
v4-mapped v6 shares the v4 bucket (via `address.fromIp` in the test); sweep removes only stale;
|
||||
map-full behavior allows and counts; stats add up; window arithmetic at i96 scale (a timestamp far
|
||||
from zero).
|
||||
Tests (≥ 8): allows up to limit, refuses limit+1; new window resets; v4 and v6 keys independent; v4-mapped v6 shares the v4 bucket (via `address.fromIp` in the test); sweep removes only stale; map-full behavior allows and counts; stats add up; window arithmetic at i96 scale (a timestamp far from zero).
|
||||
|
||||
Acceptance: fmt/ast-check clean; no allocation in `check` (signature proves it); in-file tests
|
||||
pass standalone.
|
||||
Acceptance: fmt/ast-check clean; no allocation in `check` (signature proves it); in-file tests pass standalone.
|
||||
|
||||
---
|
||||
|
||||
## Session S3: queries repo — `src/storage/repositories/queries_repo.zig`
|
||||
|
||||
Imports: `std`, `../db.zig`. Pure DB code, no `std.Io`. Follows the milestone-4 repository idiom
|
||||
(free functions over `*db.Db` — read `groups_repo.zig` first), plus one long-lived-statement
|
||||
struct the flush loop owns (`db.zig:360` names this file as the reason no statement cache exists).
|
||||
Imports: `std`, `../db.zig`. Pure DB code, no `std.Io`. Follows the milestone-4 repository idiom (free functions over `*db.Db` — read `groups_repo.zig` first), plus one long-lived-statement struct the flush loop owns (`db.zig:360` names this file as the reason no statement cache exists).
|
||||
|
||||
```zig
|
||||
pub const Row = struct {
|
||||
@@ -269,26 +171,17 @@ pub fn countRows(database: *db.Db) db.Error!i64;
|
||||
pub fn countDomains(database: *db.Db) db.Error!i64;
|
||||
```
|
||||
|
||||
`writeBatch` on an empty slice returns without opening a transaction. `pruneOlderThan` deletes
|
||||
`query_log` rows only — orphaned `domains` rows stay (they are a dimension table; re-interning is
|
||||
cheaper than referential garbage collection, and §11.3 sets no requirement). Statements: three in
|
||||
`BatchWriter` (insert-or-ignore domain, select domain id, insert row), reset+rebind per use.
|
||||
`writeBatch` on an empty slice returns without opening a transaction. `pruneOlderThan` deletes `query_log` rows only — orphaned `domains` rows stay (they are a dimension table; re-interning is cheaper than referential garbage collection, and §11.3 sets no requirement). Statements: three in `BatchWriter` (insert-or-ignore domain, select domain id, insert row), reset+rebind per use.
|
||||
|
||||
Tests (≥ 8, all against `:memory:` + `querylog_schema.ddl`): batch inserts rows and interns
|
||||
domains once; second batch reuses the domain id; nullable columns round-trip null and value;
|
||||
empty batch writes nothing; prune deletes strictly-older rows only; prune returns the count;
|
||||
checkpoint and vacuum execute without error on a file DB (use `:memory:` where legal, a tmp file
|
||||
where WAL is needed — vacuum works on both); countRows/countDomains agree with inserts.
|
||||
Tests (≥ 8, all against `:memory:` + `querylog_schema.ddl`): batch inserts rows and interns domains once; second batch reuses the domain id; nullable columns round-trip null and value; empty batch writes nothing; prune deletes strictly-older rows only; prune returns the count; checkpoint and vacuum execute without error on a file DB (use `:memory:` where legal, a tmp file where WAL is needed — vacuum works on both); countRows/countDomains agree with inserts.
|
||||
|
||||
Acceptance: fmt/ast-check clean; tests pass standalone via src/-level temp root with `-lc
|
||||
-lsqlite3`.
|
||||
Acceptance: fmt/ast-check clean; tests pass standalone via src/-level temp root with `-lc -lsqlite3`.
|
||||
|
||||
---
|
||||
|
||||
## Session S4: async query logger — `src/storage/logger.zig` (needs S3)
|
||||
|
||||
Imports: `std`, `db.zig`, `repositories/queries_repo.zig`, `../config/model.zig`,
|
||||
`disk_monitor.zig` (S5's file — wave 2 starts after wave 1 verifies, so it exists).
|
||||
Imports: `std`, `db.zig`, `repositories/queries_repo.zig`, `../config/model.zig`, `disk_monitor.zig` (S5's file — wave 2 starts after wave 1 verifies, so it exists).
|
||||
|
||||
### S4.1 Entry — self-contained (Queue copies raw bytes)
|
||||
|
||||
@@ -331,31 +224,12 @@ pub const Logger = struct {
|
||||
};
|
||||
```
|
||||
|
||||
- Privacy (§11.4): `hide_domains` replaces the domain with `hidden_marker`; `hide_client_ips`
|
||||
likewise. Applied inside `log`, so the queue never holds the real value — the transform runs
|
||||
before persistence and before any future SSE fanout by construction (ruling 4).
|
||||
- Drop-oldest: `put(io, &.{entry}, 0)`; on 0, `get` one entry nonblocking (min 0), count it
|
||||
dropped, retry the put — until the put succeeds, one drop counted per failed attempt. A fixed
|
||||
attempt cap is wrong under contention (one call would pay for two old entries plus its own).
|
||||
The only early exit is a zero-capacity queue, which has nothing to drop.
|
||||
- `runWriter` loop: `getOne` (blocking; `error.Closed` → final drain + flush + return). Then
|
||||
accumulate up to `flush_batch` entries: nonblocking `get` first; while short of the batch and
|
||||
the interval clock (started at the first entry, `.awake`) has time left, race `getOne` against
|
||||
the remaining interval via `Select` (manager's `fetchWithin` shape). Flush: if
|
||||
`monitor != null and !monitor.?.writesAllowed()`, do not write — hold the batch, sleep 1 s
|
||||
(`.awake`), re-check; count each held cycle under `batches_gated`. §11.6: log flushes stop at
|
||||
critical, the queue keeps dropping oldest behind them. On `writeBatch` error: log at `warn`,
|
||||
drop the batch (it is expendable log data; blocking would fill the queue), continue.
|
||||
- Entries convert to `queries_repo.Row` at flush (slices point into the batch array — safe, the
|
||||
batch lives across the call).
|
||||
- Privacy (§11.4): `hide_domains` replaces the domain with `hidden_marker`; `hide_client_ips` likewise. Applied inside `log`, so the queue never holds the real value — the transform runs before persistence and before any future SSE fanout by construction (ruling 4).
|
||||
- Drop-oldest: `put(io, &.{entry}, 0)`; on 0, `get` one entry nonblocking (min 0), count it dropped, retry the put — until the put succeeds, one drop counted per failed attempt. A fixed attempt cap is wrong under contention (one call would pay for two old entries plus its own). The only early exit is a zero-capacity queue, which has nothing to drop.
|
||||
- `runWriter` loop: `getOne` (blocking; `error.Closed` → final drain + flush + return). Then accumulate up to `flush_batch` entries: nonblocking `get` first; while short of the batch and the interval clock (started at the first entry, `.awake`) has time left, race `getOne` against the remaining interval via `Select` (manager's `fetchWithin` shape). Flush: if `monitor != null and !monitor.?.writesAllowed()`, do not write — hold the batch, sleep 1 s (`.awake`), re-check; count each held cycle under `batches_gated`. §11.6: log flushes stop at critical, the queue keeps dropping oldest behind them. On `writeBatch` error: log at `warn`, drop the batch (it is expendable log data; blocking would fill the queue), continue.
|
||||
- Entries convert to `queries_repo.Row` at flush (slices point into the batch array — safe, the batch lives across the call).
|
||||
|
||||
Tests (≥ 8): transforms applied at enqueue (inspect the queue copy); drop-oldest drops the oldest
|
||||
and counts; entry accessors round-trip; batch conversion maps "" to null; a `runWriter`
|
||||
smoke over `:memory:` — enqueue N, shutdown, all N in the DB (single-threaded: run the writer via
|
||||
`io.concurrent` in the test with a Threaded instance — this is the one in-file test that touches
|
||||
`std.Io`; keep it gated under a plain `test` since it needs no network); gating holds rows back
|
||||
and releases them (fake monitor state flip); writeBatch failure drops and continues (close the DB
|
||||
under the writer? if unreachable in-file, mark for S8).
|
||||
Tests (≥ 8): transforms applied at enqueue (inspect the queue copy); drop-oldest drops the oldest and counts; entry accessors round-trip; batch conversion maps "" to null; a `runWriter` smoke over `:memory:` — enqueue N, shutdown, all N in the DB (single-threaded: run the writer via `io.concurrent` in the test with a Threaded instance — this is the one in-file test that touches `std.Io`; keep it gated under a plain `test` since it needs no network); gating holds rows back and releases them (fake monitor state flip); writeBatch failure drops and continues (close the DB under the writer? if unreachable in-file, mark for S8).
|
||||
|
||||
Acceptance: fmt/ast-check clean; `grep -c "std.log.err" src/storage/logger.zig` = 0.
|
||||
|
||||
@@ -377,9 +251,7 @@ extern fn statvfs(path: [*:0]const u8, buf: *StatVfs) c_int;
|
||||
pub fn freeBytes(path: [:0]const u8) error{StatFailed}!u64; // f_bavail * f_frsize
|
||||
```
|
||||
|
||||
The 64-bit glibc and musl `struct statvfs` layouts agree field-for-field; both build targets are
|
||||
64-bit. One test: `freeBytes(".")` returns a nonzero value (gated `-Dintegration`? no — it is
|
||||
hermetic and deterministic-enough: assert `> 0`; a full disk failing CI is a real signal).
|
||||
The 64-bit glibc and musl `struct statvfs` layouts agree field-for-field; both build targets are 64-bit. One test: `freeBytes(".")` returns a nonzero value (gated `-Dintegration`? no — it is hermetic and deterministic-enough: assert `> 0`; a full disk failing CI is a real signal).
|
||||
|
||||
### S5.2 `src/storage/disk_monitor.zig`
|
||||
|
||||
@@ -405,15 +277,9 @@ pub const Monitor = struct {
|
||||
pub const sample_interval_s = 60;
|
||||
```
|
||||
|
||||
State is `std.atomic.Value(u8)`; a reader never sees a torn value. On the ok↔warn↔critical edges
|
||||
log one line at `warn` (state changes only, not every sample). The blocklist-update gate (§11.6
|
||||
"stop non-essential writes") is consumed by the logger now and by the manager in Phase 7 — no
|
||||
milestone-5 file is edited here.
|
||||
State is `std.atomic.Value(u8)`; a reader never sees a torn value. On the ok↔warn↔critical edges log one line at `warn` (state changes only, not every sample). The blocklist-update gate (§11.6 "stop non-essential writes") is consumed by the logger now and by the manager in Phase 7 — no milestone-5 file is edited here.
|
||||
|
||||
Tests: `classify` boundary cases (exactly min, exactly warn, between, above); state-change
|
||||
logging is observable via state transitions (assert states, not logs); init leaves `.ok` with
|
||||
zero gauges; gauge arithmetic (a fixture dir with files of known sizes, using `std.Io.Dir` in a
|
||||
plain test with a Threaded instance).
|
||||
Tests: `classify` boundary cases (exactly min, exactly warn, between, above); state-change logging is observable via state transitions (assert states, not logs); init leaves `.ok` with zero gauges; gauge arithmetic (a fixture dir with files of known sizes, using `std.Io.Dir` in a plain test with a Threaded instance).
|
||||
|
||||
Acceptance: fmt/ast-check clean on both files; `classify` covered exhaustively.
|
||||
|
||||
@@ -421,8 +287,7 @@ Acceptance: fmt/ast-check clean on both files; `classify` covered exhaustively.
|
||||
|
||||
## Session S6: log sink — `src/platform/logging.zig`
|
||||
|
||||
The custom `std.log` sink: level filter, stderr or rotating file output, upstream-error dedup
|
||||
(§11.6). Imports: `std`, `../config/model.zig`.
|
||||
The custom `std.log` sink: level filter, stderr or rotating file output, upstream-error dedup (§11.6). Imports: `std`, `../config/model.zig`.
|
||||
|
||||
```zig
|
||||
/// Matches std.Options.logFn. Before install(), passes through to stderr formatting.
|
||||
@@ -435,33 +300,17 @@ pub const Stats = struct { lines_written, lines_deduped, rotations, sink_errors:
|
||||
pub fn stats() Stats;
|
||||
```
|
||||
|
||||
Verify the exact `std.Options.logFn` signature against lib/std/std.zig + log.zig before writing —
|
||||
do not trust this spec's sketch.
|
||||
Verify the exact `std.Options.logFn` signature against lib/std/std.zig + log.zig before writing — do not trust this spec's sketch.
|
||||
|
||||
- Global sink state behind a `std.Thread.Mutex`? **No** — `std.Io.Mutex` needs an io; `logFn`
|
||||
receives none. Store the installed `std.Io` by value in the global state (it is a plain
|
||||
interface value); guard the whole sink with `std.debug` -style spinlock? Verify what std.log's
|
||||
default lock does in 0.16 and mirror it. If std provides no lock for custom logFn, use a
|
||||
file-scope `std.Thread.Mutex` (std.Thread still exists for this — verify) or an atomic spin
|
||||
guard; state the choice in a comment. This is the one deliberately-open point in this spec:
|
||||
resolve it against the stdlib source and record the resolution for "As built".
|
||||
- Global sink state behind a `std.Thread.Mutex`? **No** — `std.Io.Mutex` needs an io; `logFn` receives none. Store the installed `std.Io` by value in the global state (it is a plain interface value); guard the whole sink with `std.debug` -style spinlock? Verify what std.log's default lock does in 0.16 and mirror it. If std provides no lock for custom logFn, use a file-scope `std.Thread.Mutex` (std.Thread still exists for this — verify) or an atomic spin guard; state the choice in a comment. This is the one deliberately-open point in this spec: resolve it against the stdlib source and record the resolution for "As built".
|
||||
- Level: drop below `cfg.level` (mapping model's level enum to std.log.Level — write it).
|
||||
- Dedup: for `warn`+ lines whose scope is one of `.doh_client`, `.dot_client`, `.pool`,
|
||||
`.forward_client`: key = scope + formatted message truncated to 96 bytes; a key seen within the
|
||||
last 60 s (`.awake`) is dropped and counted. Fixed 64-entry table, oldest-stamp replacement.
|
||||
- File mode: append (seekTo end-of-file pattern), byte counter from `file.length` at open; a line
|
||||
that would cross `cfg.maxLogBytes()` triggers rotation first: delete `.{max_files-1}`, shift
|
||||
`.N`→`.N+1`, rename live → `.1`, reopen fresh. All under the sink lock. Rotation or write
|
||||
failure: fall back to stderr for that line, count `sink_errors`, keep trying the file next line.
|
||||
- Dedup: for `warn`+ lines whose scope is one of `.doh_client`, `.dot_client`, `.pool`, `.forward_client`: key = scope + formatted message truncated to 96 bytes; a key seen within the last 60 s (`.awake`) is dropped and counted. Fixed 64-entry table, oldest-stamp replacement.
|
||||
- File mode: append (seekTo end-of-file pattern), byte counter from `file.length` at open; a line that would cross `cfg.maxLogBytes()` triggers rotation first: delete `.{max_files-1}`, shift `.N`→`.N+1`, rename live → `.1`, reopen fresh. All under the sink lock. Rotation or write failure: fall back to stderr for that line, count `sink_errors`, keep trying the file next line.
|
||||
- stderr mode: format + single `write` per line (std.debug.lockStdErr equivalent — verify).
|
||||
|
||||
Tests: the pure pieces — dedup table admits first, drops repeat inside 60 s, admits after; level
|
||||
mapping; rotation name arithmetic (`rotatedName(buf, path, n)`). File behavior lands in S8 (needs
|
||||
a real dir). `logFn` itself is NOT exercised in tests (installing a sink under the test runner
|
||||
would eat the harness's own logs).
|
||||
Tests: the pure pieces — dedup table admits first, drops repeat inside 60 s, admits after; level mapping; rotation name arithmetic (`rotatedName(buf, path, n)`). File behavior lands in S8 (needs a real dir). `logFn` itself is NOT exercised in tests (installing a sink under the test runner would eat the harness's own logs).
|
||||
|
||||
Acceptance: fmt/ast-check clean; no `std.log` call inside the sink itself (it would recurse);
|
||||
dedup + rotation arithmetic covered.
|
||||
Acceptance: fmt/ast-check clean; no `std.log` call inside the sink itself (it would recurse); dedup + rotation arithmetic covered.
|
||||
|
||||
---
|
||||
|
||||
@@ -485,16 +334,9 @@ pub const Retention = struct {
|
||||
pub const pass_interval_s = 86_400;
|
||||
```
|
||||
|
||||
`runOnce` computes the cutoff from `Clock.real.now(io).toSeconds() - cfg.retentionSeconds()`.
|
||||
Sharing one handle between concurrent tasks is NOT safe: FULLMUTEX serializes single SQLite calls,
|
||||
but a transaction is connection state — a prune landing between another task's BEGIN and COMMIT
|
||||
joins that transaction. The contract therefore requires a dedicated connection; cross-connection
|
||||
isolation is SQLite's own (WAL + busy_timeout), and a pass losing a race sees Busy/Locked, logs at
|
||||
warn, and retries next interval.
|
||||
`runOnce` computes the cutoff from `Clock.real.now(io).toSeconds() - cfg.retentionSeconds()`. Sharing one handle between concurrent tasks is NOT safe: FULLMUTEX serializes single SQLite calls, but a transaction is connection state — a prune landing between another task's BEGIN and COMMIT joins that transaction. The contract therefore requires a dedicated connection; cross-connection isolation is SQLite's own (WAL + busy_timeout), and a pass losing a race sees Busy/Locked, logs at warn, and retries next interval.
|
||||
|
||||
Tests (`:memory:` + ddl): a pass prunes exactly the old rows; the 7th pass vacuums (assert via
|
||||
`stats.vacuums` after 7 `runOnce` calls); a pass on an empty DB is a no-op that still counts;
|
||||
cutoff arithmetic honors `retention_days`.
|
||||
Tests (`:memory:` + ddl): a pass prunes exactly the old rows; the 7th pass vacuums (assert via `stats.vacuums` after 7 `runOnce` calls); a pass on an empty DB is a no-op that still counts; cutoff arithmetic honors `retention_days`.
|
||||
|
||||
Acceptance: fmt/ast-check clean; tests standalone with `-lc -lsqlite3`.
|
||||
|
||||
@@ -502,35 +344,19 @@ Acceptance: fmt/ast-check clean; tests standalone with `-lc -lsqlite3`.
|
||||
|
||||
## Session S8: integration — `src/storage/phase6_integration_test.zig` (needs all)
|
||||
|
||||
Gated by `build_options.integration` exactly like `storage_integration_test.zig` (same Fixture
|
||||
idiom over `.zig-cache/tmp/`). Cases, named `"S8 case N: …"`:
|
||||
Gated by `build_options.integration` exactly like `storage_integration_test.zig` (same Fixture idiom over `.zig-cache/tmp/`). Cases, named `"S8 case N: …"`:
|
||||
|
||||
1. logger end-to-end: real file `querylog.db` via `querylog_schema.open`, writer task under
|
||||
`io.concurrent`, 250 entries → shutdown → 250 rows, domains interned (fewer domain rows than
|
||||
query rows), fingerprint stamped.
|
||||
1. logger end-to-end: real file `querylog.db` via `querylog_schema.open`, writer task under `io.concurrent`, 250 entries → shutdown → 250 rows, domains interned (fewer domain rows than query rows), fingerprint stamped.
|
||||
2. flush on interval: one entry, no shutdown; poll until it lands; well under 10× the interval.
|
||||
3. backpressure: queue capacity 8, enqueue 20 with the writer stalled (gated by a monitor stub at
|
||||
critical); `queries_dropped >= 12`, the survivors are the NEWEST entries, then un-gate and
|
||||
confirm the survivors landed.
|
||||
3. backpressure: queue capacity 8, enqueue 20 with the writer stalled (gated by a monitor stub at critical); `queries_dropped >= 12`, the survivors are the NEWEST entries, then un-gate and confirm the survivors landed.
|
||||
4. privacy: hide both → every row's domain and client_ip read `hidden`.
|
||||
5. retention: back-dated rows pruned, fresh rows kept, WAL truncated after checkpoint (assert
|
||||
`-wal` size 0 or absent).
|
||||
6. disk thresholds end-to-end (§17 exit criterion): monitor with `min_free_mb` far above the real
|
||||
free space → `.critical`, `writesAllowed() == false`, logger gates (case-3 machinery),
|
||||
`batches_gated > 0`; then thresholds far below → `.ok` and flushing resumes.
|
||||
7. log sink file mode: install to a fixture path with `max_size_mb` tiny (write the size gate via
|
||||
a test-only override — if `install` takes cfg, a 1 MB minimum makes this slow; add
|
||||
`pub fn installForTest(io, cfg, max_bytes_override)` if needed and mark it test-only),
|
||||
emit lines past the limit → `.1` exists, live file small, `max_files` honored, then
|
||||
`deinstall`. If overriding proves ugly, rotation is proven at the arithmetic level in S6 and
|
||||
this case shrinks to append+reopen round-trip — say which in the report.
|
||||
8. cache with real packets: build a response via `ResponseBuilder`, `classify` + `put`, advance
|
||||
`now_s`, `get` → TTLs visibly decremented (re-parse), expiry at the boundary is a miss.
|
||||
9. rate limiter + address integration: 1001 checks from one v6-mapped-v4 client inside one window
|
||||
→ exactly one refused… (limit 1000); a second distinct client unaffected.
|
||||
5. retention: back-dated rows pruned, fresh rows kept, WAL truncated after checkpoint (assert `-wal` size 0 or absent).
|
||||
6. disk thresholds end-to-end (§17 exit criterion): monitor with `min_free_mb` far above the real free space → `.critical`, `writesAllowed() == false`, logger gates (case-3 machinery), `batches_gated > 0`; then thresholds far below → `.ok` and flushing resumes.
|
||||
7. log sink file mode: install to a fixture path with `max_size_mb` tiny (write the size gate via a test-only override — if `install` takes cfg, a 1 MB minimum makes this slow; add `pub fn installForTest(io, cfg, max_bytes_override)` if needed and mark it test-only), emit lines past the limit → `.1` exists, live file small, `max_files` honored, then `deinstall`. If overriding proves ugly, rotation is proven at the arithmetic level in S6 and this case shrinks to append+reopen round-trip — say which in the report.
|
||||
8. cache with real packets: build a response via `ResponseBuilder`, `classify` + `put`, advance `now_s`, `get` → TTLs visibly decremented (re-parse), expiry at the boundary is a miss.
|
||||
9. rate limiter + address integration: 1001 checks from one v6-mapped-v4 client inside one window → exactly one refused… (limit 1000); a second distinct client unaffected.
|
||||
|
||||
Acceptance: `zig build test -Dintegration` exit 0 with all 9 passing; no case sleeps longer than
|
||||
2 s of wall time; no network.
|
||||
Acceptance: `zig build test -Dintegration` exit 0 with all 9 passing; no case sleeps longer than 2 s of wall time; no network.
|
||||
|
||||
---
|
||||
|
||||
@@ -562,9 +388,7 @@ Acceptance: `zig build test -Dintegration` exit 0 with all 9 passing; no case sl
|
||||
| `src/storage/phase6_integration_test.zig` | S8 |
|
||||
| `src/tests.zig`, `build.zig`, `src/main.zig` (std_options) | orchestrator |
|
||||
|
||||
No session touches `src/dns/`, `src/upstream/`, `src/filter/`, `src/local/`, `src/server/handler.zig`,
|
||||
`src/server/udp_server.zig`, `src/server/tcp_server.zig`, `src/storage/db.zig`, any milestone 4–5
|
||||
repository, `src/config/`, or `src/cli.zig`. A needed change there is reported, not made.
|
||||
No session touches `src/dns/`, `src/upstream/`, `src/filter/`, `src/local/`, `src/server/handler.zig`, `src/server/udp_server.zig`, `src/server/tcp_server.zig`, `src/storage/db.zig`, any milestone 4–5 repository, `src/config/`, or `src/cli.zig`. A needed change there is reported, not made.
|
||||
|
||||
## Acceptance Criteria (Milestone 6 Complete)
|
||||
|
||||
@@ -585,146 +409,35 @@ repository, `src/config/`, or `src/cli.zig`. A needed change there is reported,
|
||||
|
||||
## Anti-Requirements
|
||||
|
||||
- **No handler or server wiring.** `handler.zig` keeps its signature; nothing calls the cache,
|
||||
the limiter, or the logger from the query path. Phase 7.
|
||||
- **No handler or server wiring.** `handler.zig` keeps its signature; nothing calls the cache, the limiter, or the logger from the query path. Phase 7.
|
||||
- **No client-row pruning and no auto-materialization.** Phase 7 (ruling 1).
|
||||
- **No web/API/SSE/metrics/health payloads.** Phase 8 reads the accessors this milestone exposes.
|
||||
- **No API token bucket.** §10's API limiter is Phase 8.
|
||||
- **No cache persistence, no cache serialization.** In-memory only (§8).
|
||||
- **No ECS parsing.** The cache key accepts ECS bytes the caller extracted; extracting them from
|
||||
OPT is the handler's job in Phase 7.
|
||||
- **No ECS parsing.** The cache key accepts ECS bytes the caller extracted; extracting them from OPT is the handler's job in Phase 7.
|
||||
- **No statement cache in db.zig.** `BatchWriter` owns its statements; `db.zig` stays as is.
|
||||
- **No new config keys and no schema change.** Every knob this milestone reads exists in
|
||||
`model.zig`; flush tuning is comptime (ruling 3).
|
||||
- **No new config keys and no schema change.** Every knob this milestone reads exists in `model.zig`; flush tuning is comptime (ruling 3).
|
||||
- **No log sink installation under the test runner.** `install` runs only from `main`.
|
||||
- **No epoll/timerfd/signalfd.** Loops sleep on `Clock.Duration`; Phase 7 owns lifecycle.
|
||||
|
||||
## As built (S1–S7 and orchestrator wiring)
|
||||
|
||||
Deviations from the text above, recorded after the sessions verified. Where this section and the
|
||||
session text disagree, this section wins.
|
||||
Deviations from the text above, recorded after the sessions verified. Where this section and the session text disagree, this section wins.
|
||||
|
||||
**S1 dns_cache.** `Config` is an alias of `model.Cache`, not a duplicate struct. `classify`
|
||||
computes the positive min-TTL by iterating records (no mutation, no 64 KiB copy); NODATA (NOERROR,
|
||||
zero answers) takes the negative path with NXDOMAIN per RFC 2308 §2; a malformed SOA means "do not
|
||||
cache", not an error; `negative_ttl_max == 0` disables negatives only — positives still cache. A
|
||||
`put` on an existing key keeps the slot's CLOCK reference bit. A slot the CLOCK hand reclaims
|
||||
because it expired counts `expirations`, not `evictions`. `size == 0` turns caching off (`put`
|
||||
no-ops). A backwards clock ages by zero. `misses` covers lookup failure, expiry and a too-small
|
||||
`out`; a poisoned entry counts `invalid_hits` only. The slot's `negative` flag is stored but has
|
||||
no accessor yet — Phase 8 adds one if it wants the split. After review: the negative TTL is
|
||||
min(SOA record TTL, SOA MINIMUM) per RFC 2308 §5, and a top-bit-set TTL reads as zero per RFC 2181
|
||||
§8 on every non-OPT record in any section (positive path) and both SOA fields (negative path) — a
|
||||
zero result means the response is not cached. When `put` stores a negative response it lowers the
|
||||
clone's first authority SOA record TTL to the entry's lifetime, so a hit at elapsed 0 serves an
|
||||
SOA TTL equal to the entry TTL and downstream caches do not hold the negative answer past it
|
||||
(RFC 2308 §3); the caller's buffer is not modified. On the positive path `put` caps every non-OPT
|
||||
record TTL in the clone at `max_ttl_seconds` for the same reason — the stored bytes never
|
||||
advertise a lifetime past the entry's ceiling (the cap is our ruling 5, not an RFC rule; unbound's
|
||||
cache-max-ttl serves the capped value the same way). A negative response's non-SOA records are
|
||||
deliberately not capped — the clamped SOA is the record that governs negative caching downstream.
|
||||
The poisoned-packet ageing test's fixture keeps its TTL under `max_ttl_seconds` on purpose, so
|
||||
`put` stores it unmodified and only the ageing inside `get` breaks it.
|
||||
**S1 dns_cache.** `Config` is an alias of `model.Cache`, not a duplicate struct. `classify` computes the positive min-TTL by iterating records (no mutation, no 64 KiB copy); NODATA (NOERROR, zero answers) takes the negative path with NXDOMAIN per RFC 2308 §2; a malformed SOA means "do not cache", not an error; `negative_ttl_max == 0` disables negatives only — positives still cache. A `put` on an existing key keeps the slot's CLOCK reference bit. A slot the CLOCK hand reclaims because it expired counts `expirations`, not `evictions`. `size == 0` turns caching off (`put` no-ops). A backwards clock ages by zero. `misses` covers lookup failure, expiry and a too-small `out`; a poisoned entry counts `invalid_hits` only. The slot's `negative` flag is stored but has no accessor yet — Phase 8 adds one if it wants the split. After review: the negative TTL is min(SOA record TTL, SOA MINIMUM) per RFC 2308 §5, and a top-bit-set TTL reads as zero per RFC 2181 §8 on every non-OPT record in any section (positive path) and both SOA fields (negative path) — a zero result means the response is not cached. When `put` stores a negative response it lowers the clone's first authority SOA record TTL to the entry's lifetime, so a hit at elapsed 0 serves an SOA TTL equal to the entry TTL and downstream caches do not hold the negative answer past it (RFC 2308 §3); the caller's buffer is not modified. On the positive path `put` caps every non-OPT record TTL in the clone at `max_ttl_seconds` for the same reason — the stored bytes never advertise a lifetime past the entry's ceiling (the cap is our ruling 5, not an RFC rule; unbound's cache-max-ttl serves the capped value the same way). A negative response's non-SOA records are deliberately not capped — the clamped SOA is the record that governs negative caching downstream. The poisoned-packet ageing test's fixture keeps its TTL under `max_ttl_seconds` on purpose, so `put` stores it unmodified and only the ageing inside `get` breaks it.
|
||||
|
||||
**S2 rate_limiter.** Windows anchor at each client's first query (`.awake` origin is arbitrary).
|
||||
`sweep` drops entries older than two full windows (exactly two survives). `untracked` is a subset
|
||||
of `allowed`, so `allowed + refused` equals the number of `check` calls. `init` asserts
|
||||
`window_seconds != 0` (validate.zig:221 guards real configs). `limit == 0` refuses everything.
|
||||
Added: `trackedClients()` occupancy accessor. `sweep` collects stale keys into a buffer allocated
|
||||
at init (~68 KiB), not onto the stack.
|
||||
**S2 rate_limiter.** Windows anchor at each client's first query (`.awake` origin is arbitrary). `sweep` drops entries older than two full windows (exactly two survives). `untracked` is a subset of `allowed`, so `allowed + refused` equals the number of `check` calls. `init` asserts `window_seconds != 0` (validate.zig:221 guards real configs). `limit == 0` refuses everything. Added: `trackedClients()` occupancy accessor. `sweep` collects stale keys into a buffer allocated at init (~68 KiB), not onto the stack.
|
||||
|
||||
**S3 queries_repo.** Domain interning is INSERT OR IGNORE + SELECT (no `lastInsertRowid` — unset
|
||||
on ignore); a missing row right after the insert is `error.NotFound`. `select_domain` resets
|
||||
immediately after the id is read; `errdefer tx.rollback()` is declared before `errdefer
|
||||
resetAll()` so no cursor is open at ROLLBACK. `checkpointTruncate` treats a blocked checkpoint as
|
||||
success (SQLite reports it in a discarded row; retention retries next pass).
|
||||
**S3 queries_repo.** Domain interning is INSERT OR IGNORE + SELECT (no `lastInsertRowid` — unset on ignore); a missing row right after the insert is `error.NotFound`. `select_domain` resets immediately after the id is read; `errdefer tx.rollback()` is declared before `errdefer resetAll()` so no cursor is open at ROLLBACK. `checkpointTruncate` treats a blocked checkpoint as success (SQLite reports it in a discarded row; retention retries next pass).
|
||||
|
||||
**S4 logger.** `Entry.init(Entry.Fields)` builds an entry from borrowed slices, copying and
|
||||
truncating; `setDomain`/`setClientIp` are public for the privacy transform. **Spec correction:**
|
||||
the `fetchWithin` shape's `cancelDiscard` is wrong here — a `getOne` that loses the race has
|
||||
already removed an entry, and discarding its result loses the entry silently. `getWithin` drains
|
||||
the Select with a `while (race.cancel())` loop; an entry recovered on the cancellation path counts
|
||||
`queries_dropped`. A batch dropped by a `writeBatch` failure adds its length to `queries_dropped`.
|
||||
The interval deadline starts at the first entry of a batch. `gate_retry_s = 1` is public. Queue
|
||||
facts verified: `put` returns `Closed` even with space; `get(min=0)` on a closed non-empty queue
|
||||
returns elements and reports `Closed` only when empty — that is what makes the shutdown drain
|
||||
correct. A writer sitting in the disk gate at shutdown holds its batch until un-gated or
|
||||
canceled (documented on `shutdown`); `Logger` must not move after `init`. After review: `fill`
|
||||
reports its count through an out-parameter, so a canceled `fill` or `flush` counts every entry
|
||||
still held in the batch under `queries_dropped` (including the gate-blocked cancellation); a
|
||||
`BatchWriter.init` failure warns, sets `writer_failed: std.atomic.Value(bool)`, closes the queue
|
||||
and drains it with `Queue.getUncancelable` — uncancelable specifically so a cancellation racing
|
||||
the failure cannot leave buffered entries uncounted (no other consumer can reach them at that
|
||||
point); the writer never pretends to run. The zero-capacity early exit reads
|
||||
`queue.capacity()`. The prepare-failure path and
|
||||
the cancellation accounting are both covered in-file (schema-less `:memory:` DB; deterministic
|
||||
2-entry batch canceled in the gate).
|
||||
**S4 logger.** `Entry.init(Entry.Fields)` builds an entry from borrowed slices, copying and truncating; `setDomain`/`setClientIp` are public for the privacy transform. **Spec correction:** the `fetchWithin` shape's `cancelDiscard` is wrong here — a `getOne` that loses the race has already removed an entry, and discarding its result loses the entry silently. `getWithin` drains the Select with a `while (race.cancel())` loop; an entry recovered on the cancellation path counts `queries_dropped`. A batch dropped by a `writeBatch` failure adds its length to `queries_dropped`. The interval deadline starts at the first entry of a batch. `gate_retry_s = 1` is public. Queue facts verified: `put` returns `Closed` even with space; `get(min=0)` on a closed non-empty queue returns elements and reports `Closed` only when empty — that is what makes the shutdown drain correct. A writer sitting in the disk gate at shutdown holds its batch until un-gated or canceled (documented on `shutdown`); `Logger` must not move after `init`. After review: `fill` reports its count through an out-parameter, so a canceled `fill` or `flush` counts every entry still held in the batch under `queries_dropped` (including the gate-blocked cancellation); a `BatchWriter.init` failure warns, sets `writer_failed: std.atomic.Value(bool)`, closes the queue and drains it with `Queue.getUncancelable` — uncancelable specifically so a cancellation racing the failure cannot leave buffered entries uncounted (no other consumer can reach them at that point); the writer never pretends to run. The zero-capacity early exit reads `queue.capacity()`. The prepare-failure path and the cancellation accounting are both covered in-file (schema-less `:memory:` DB; deterministic 2-entry batch canceled in the gate).
|
||||
|
||||
**S5 statfs + disk_monitor.** The statvfs layout is pinned by `@offsetOf`/`@sizeOf` tests (112
|
||||
bytes); musl verified from the zig-shipped headers, glibc from the system's. `init` requires
|
||||
`data_dir` opened with `.iterate = true` (documented). `log_dir_path` opens via `cwd()` per
|
||||
sample; null keeps `log_bytes` at 0. A failed statvfs leaves the state untouched; a failed size
|
||||
scan keeps that gauge's prior value while the state still publishes. A per-file `statFile` failure
|
||||
inside a size scan fails that whole scan — the gauge keeps its prior value and `sample_failures`
|
||||
increments once for the scan, not per file; `error.FileNotFound` is the one exception (a file
|
||||
deleted between `iterate` and `statFile` is normal on rotation or recreate — skipped, scan
|
||||
continues). `classify` checks min before
|
||||
warn, so warn-below-min misconfiguration reports the severer state. `run` samples first, then
|
||||
sleeps. Sizes come from `Dir.statFile`.
|
||||
**S5 statfs + disk_monitor.** The statvfs layout is pinned by `@offsetOf`/`@sizeOf` tests (112 bytes); musl verified from the zig-shipped headers, glibc from the system's. `init` requires `data_dir` opened with `.iterate = true` (documented). `log_dir_path` opens via `cwd()` per sample; null keeps `log_bytes` at 0. A failed statvfs leaves the state untouched; a failed size scan keeps that gauge's prior value while the state still publishes. A per-file `statFile` failure inside a size scan fails that whole scan — the gauge keeps its prior value and `sample_failures` increments once for the scan, not per file; `error.FileNotFound` is the one exception (a file deleted between `iterate` and `statFile` is normal on rotation or recreate — skipped, scan continues). `classify` checks min before warn, so warn-below-min misconfiguration reports the severer state. `run` samples first, then sleeps. Sizes come from `Dir.statFile`.
|
||||
|
||||
**S6 logging sink.** **Spec corrections:** the `logFn` scope parameter is `comptime scope:
|
||||
@EnumLiteral()`, and `std.Thread.Mutex` does not exist in 0.16.0. The sink lock is
|
||||
`std.debug.lockStderr`/`unlockStderr` (the same lock `std.log.defaultLog` uses; documented
|
||||
recursive; needs no io). Added: `installForTest(io, cfg, max_bytes_override)`; public pure
|
||||
helpers `toStdLevel`, `enabled`, `isDedupScope`, `buildKey`, `DedupTable`, `rotatedName`.
|
||||
`max_files` counts the live file (generations `.1`…`.{max_files-1}`); `max_files < 2` truncates
|
||||
in place. `LogOutput.syslog` routes to stderr (journald captures it — the only supported
|
||||
deployment). File lines are `<unix_seconds> <level>(<scope>): <msg>\n`, per-line durable (fresh
|
||||
writer, seekTo, writeAll, flush); write failure closes the file and the next line reopens it;
|
||||
overlong paths fall back to stderr and count `sink_errors`; cancel protection wraps file work.
|
||||
After review: `<msg>` is escaped on BOTH output paths — `\\`, newline, carriage return and tab as
|
||||
two-character escapes, every other byte below 0x20 plus DEL as `\xNN` (a deliberate divergence
|
||||
from `std.log.defaultLog`, which writes raw; these lines are parsed by operators and journald);
|
||||
`max_escaped_message_bytes = max_message_bytes * 4` is public and the line buffer holds the worst
|
||||
case. A failed rotation step counts `sink_errors`, sends the line to stderr, and leaves the live
|
||||
file CLOSED until a later line completes the rotation — the oversized file is never reopened for
|
||||
append (`rotate_pending`); `rotations` counts completed rotations only. One exception: a single
|
||||
line longer than `max_bytes` is written to an empty file rather than rotated forever.
|
||||
`lines_written` counts only lines a sink accepted; each failed write attempt counts one
|
||||
`sink_errors` (file-fail + stderr-success = one of each). The scope name is truncated to
|
||||
`max_scope_name_bytes` (32) by `boundedScopeName`, applied once in `logFn` so the file and stderr
|
||||
paths render the same event identically; the public constant `max_line_header_bytes` (65) bounds
|
||||
the record header and the file line buffer is `max_escaped_message_bytes +
|
||||
max_line_header_bytes`. Record construction is `buildLine`, a pure function taking the unix
|
||||
seconds as a parameter — it returns an error instead of a partial record, and a failure counts
|
||||
one `sink_error` and sends the line to stderr, so a malformed record never reaches the file.
|
||||
Counting contract: every path where `prepareFileLocked` returns false has already counted exactly
|
||||
one `sink_error` at the failing step (the caller never counts again); `rotateLocked` counts its
|
||||
own failure.
|
||||
**S6 logging sink.** **Spec corrections:** the `logFn` scope parameter is `comptime scope: @EnumLiteral()`, and `std.Thread.Mutex` does not exist in 0.16.0. The sink lock is `std.debug.lockStderr`/`unlockStderr` (the same lock `std.log.defaultLog` uses; documented recursive; needs no io). Added: `installForTest(io, cfg, max_bytes_override)`; public pure helpers `toStdLevel`, `enabled`, `isDedupScope`, `buildKey`, `DedupTable`, `rotatedName`. `max_files` counts the live file (generations `.1`…`.{max_files-1}`); `max_files < 2` truncates in place. `LogOutput.syslog` routes to stderr (journald captures it — the only supported deployment). File lines are `<unix_seconds> <level>(<scope>): <msg>\n`, per-line durable (fresh writer, seekTo, writeAll, flush); write failure closes the file and the next line reopens it; overlong paths fall back to stderr and count `sink_errors`; cancel protection wraps file work. After review: `<msg>` is escaped on BOTH output paths — `\\`, newline, carriage return and tab as two-character escapes, every other byte below 0x20 plus DEL as `\xNN` (a deliberate divergence from `std.log.defaultLog`, which writes raw; these lines are parsed by operators and journald); `max_escaped_message_bytes = max_message_bytes * 4` is public and the line buffer holds the worst case. A failed rotation step counts `sink_errors`, sends the line to stderr, and leaves the live file CLOSED until a later line completes the rotation — the oversized file is never reopened for append (`rotate_pending`); `rotations` counts completed rotations only. One exception: a single line longer than `max_bytes` is written to an empty file rather than rotated forever. `lines_written` counts only lines a sink accepted; each failed write attempt counts one `sink_errors` (file-fail + stderr-success = one of each). The scope name is truncated to `max_scope_name_bytes` (32) by `boundedScopeName`, applied once in `logFn` so the file and stderr paths render the same event identically; the public constant `max_line_header_bytes` (65) bounds the record header and the file line buffer is `max_escaped_message_bytes + max_line_header_bytes`. Record construction is `buildLine`, a pure function taking the unix seconds as a parameter — it returns an error instead of a partial record, and a failure counts one `sink_error` and sends the line to stderr, so a malformed record never reaches the file. Counting contract: every path where `prepareFileLocked` returns false has already counted exactly one `sink_error` at the failing step (the caller never counts again); `rotateLocked` counts its own failure.
|
||||
|
||||
**S7 retention.** Prune, checkpoint and vacuum are independent within a pass — a failed prune
|
||||
does not skip the checkpoint (the WAL was filled by the logger, not this pass). `passes`
|
||||
increments at the top of `runOnce`; vacuum fires at `passes % 7 == 0`. The cutoff is
|
||||
strictly-less-than (a row exactly at the cutoff survives). `runOnce` opens no transaction; the
|
||||
contract on `run` requires a dedicated connection (concurrent sharing joins the other task's
|
||||
transaction — FULLMUTEX does not prevent that); Phase 7 opens it. Sequential use of one handle,
|
||||
as S8 case 5 does, is permitted.
|
||||
**S7 retention.** Prune, checkpoint and vacuum are independent within a pass — a failed prune does not skip the checkpoint (the WAL was filled by the logger, not this pass). `passes` increments at the top of `runOnce`; vacuum fires at `passes % 7 == 0`. The cutoff is strictly-less-than (a row exactly at the cutoff survives). `runOnce` opens no transaction; the contract on `run` requires a dedicated connection (concurrent sharing joins the other task's transaction — FULLMUTEX does not prevent that); Phase 7 opens it. Sequential use of one handle, as S8 case 5 does, is permitted.
|
||||
|
||||
**Orchestrator wiring.** All eight new files (S8's included) imported individually by
|
||||
`src/tests.zig`. `src/main.zig` gained `pub const std_options: std.Options = .{ .logFn =
|
||||
logging.logFn };` — effective in the executable build, inert under the test runner (tests.zig is
|
||||
that root); `zig build` type-checks the sink through real call sites. The spec's claim that a
|
||||
`rate_window_seconds` rule was missing from validate.zig was wrong — the rule exists at
|
||||
validate.zig:221; nothing was added.
|
||||
**Orchestrator wiring.** All eight new files (S8's included) imported individually by `src/tests.zig`. `src/main.zig` gained `pub const std_options: std.Options = .{ .logFn = logging.logFn };` — effective in the executable build, inert under the test runner (tests.zig is that root); `zig build` type-checks the sink through real call sites. The spec's claim that a `rate_window_seconds` rule was missing from validate.zig was wrong — the rule exists at validate.zig:221; nothing was added.
|
||||
|
||||
**S8 integration.** Case 7 runs the full rotation via `installForTest(io, cfg, 256)` with
|
||||
`max_files = 3`, calling `logging.logFn` directly (the test runner owns `std_options`, so a
|
||||
`std.log` call would never reach the sink) and comparing counters as before/after deltas
|
||||
(`install`/`deinstall` do not reset stats); it also proves append-and-reopen byte-identity. Case 3
|
||||
enqueues the full burst before the writer starts, making "12 dropped, newest 8 survive" exact, and
|
||||
starts the writer gated. Cases 3 and 6 un-gate before `shutdown` (the gate-hold documented in S4).
|
||||
Case 6 samples real `statvfs` against the fixture dir and flips state by swapping `monitor.cfg`
|
||||
between huge and zero thresholds. Case 5 asserts the WAL was non-empty before the checkpoint, so
|
||||
truncation is proven, not assumed. Shared-Db reads happen only while the writer sleeps in the gate
|
||||
or after its future is awaited. All cases use `testing.io`. Per-case wall time stays under 1.5 s;
|
||||
cases 3 and 6 pay `gate_retry_s`. With `integration = false`, all 9 skip.
|
||||
**S8 integration.** Case 7 runs the full rotation via `installForTest(io, cfg, 256)` with `max_files = 3`, calling `logging.logFn` directly (the test runner owns `std_options`, so a `std.log` call would never reach the sink) and comparing counters as before/after deltas (`install`/`deinstall` do not reset stats); it also proves append-and-reopen byte-identity. Case 3 enqueues the full burst before the writer starts, making "12 dropped, newest 8 survive" exact, and starts the writer gated. Cases 3 and 6 un-gate before `shutdown` (the gate-hold documented in S4). Case 6 samples real `statvfs` against the fixture dir and flips state by swapping `monitor.cfg` between huge and zero thresholds. Case 5 asserts the WAL was non-empty before the checkpoint, so truncation is proven, not assumed. Shared-Db reads happen only while the writer sleeps in the gate or after its future is awaited. All cases use `testing.io`. Per-case wall time stays under 1.5 s; cases 3 and 6 pay `gate_retry_s`. With `integration = false`, all 9 skip.
|
||||
|
||||
+79
-430
@@ -1,123 +1,40 @@
|
||||
# Milestone 7: serving pipeline composition (PLAN Phase 7)
|
||||
|
||||
Goal: `nxdns run` serves DNS end to end — rate limit → parse → group → local records →
|
||||
forward zones → filtering → safe-search → cache → upstream → CNAME uncloaking → cache store →
|
||||
async query log — with client auto-materialization, pause/resume, and a composition root that
|
||||
starts every background loop and shuts down cleanly on SIGINT/SIGTERM.
|
||||
Goal: `nxdns run` serves DNS end to end — rate limit → parse → group → local records → forward zones → filtering → safe-search → cache → upstream → CNAME uncloaking → cache store → async query log — with client auto-materialization, pause/resume, and a composition root that starts every background loop and shuts down cleanly on SIGINT/SIGTERM.
|
||||
|
||||
Ground truth: PLAN.md §4 (pipeline diagram, lines 151-173), §6 (DNS behavior), §7.2
|
||||
(auto-materialization), §8 (cache key), §10 (rate limit), §11.4/§11.6 (logging, disk), Phase 7
|
||||
text at PLAN.md:606-608. Verified Zig facts: specs/research/zig-0.16-api-notes.md plus the
|
||||
scratchpad note m7-explore-zig.md (signals, Io.Group, cancelable net ops) — the load-bearing
|
||||
facts are restated inline below where a session needs them.
|
||||
Ground truth: PLAN.md §4 (pipeline diagram, lines 151-173), §6 (DNS behavior), §7.2 (auto-materialization), §8 (cache key), §10 (rate limit), §11.4/§11.6 (logging, disk), Phase 7 text at PLAN.md:606-608. Verified Zig facts: specs/research/zig-0.16-api-notes.md plus the scratchpad note m7-explore-zig.md (signals, Io.Group, cancelable net ops) — the load-bearing facts are restated inline below where a session needs them.
|
||||
|
||||
## Rulings (engineering calls resolving PLAN ambiguities — binding)
|
||||
|
||||
1. **Scope**: Phase 7 owns `nxdns run` end to end: `src/app.zig` (composition root),
|
||||
`src/server/shutdown.zig` (signal → event), the `cli.runRun` body, and both listener edits.
|
||||
2. **Signals**: no signalfd/epoll/timerfd. `std.posix.sigaction` handler for INT and TERM that
|
||||
calls `std.Io.Event.set` on a file-scope event (`set` is async-signal-safe on the Threaded
|
||||
Linux backend: raw futex wake, no locks — Threaded.zig:1105,17235).
|
||||
3. **Locking**: `DnsCache` and `RateLimiter` each get one `std.Io.Mutex` owned by the handler.
|
||||
Household scale makes contention irrelevant; sharding is over-engineering.
|
||||
4. **Null snapshot** (`Manager.acquire` returns null before the first successful reload): fail
|
||||
open — skip group/filter/safe-search, use the default group semantics, count
|
||||
`unfiltered_queries`. DNS availability beats filtering for a household. The app attempts one
|
||||
synchronous `reload` before serving; a failure warns and serving starts anyway.
|
||||
5. **Cache scope**: one global cache, no group in the key (PLAN §8). Safe because blocked
|
||||
responses are never cached. Safe-search responses are not cached either (13 domains, rare;
|
||||
caching them under the rewritten name would force answer re-encoding on hit). Paused-mode
|
||||
upstream answers cache normally (they are ordinary upstream responses). Forward-zone
|
||||
responses cache normally (PLAN §6.5) under the same global key.
|
||||
6. **Local vs forward zone**: local records win (checked first; PLAN §6.4 puts them before
|
||||
everything but rate limit/parse).
|
||||
7. **Forward zones bypass** filtering, safe-search, and uncloaking (PLAN §6.5). They are logged
|
||||
with `upstream` = the resolver's canonical text.
|
||||
8. **Rate limit position**: after the 12-byte header read (the REFUSED reply needs the ID),
|
||||
before full parse. Refused queries are counted (`Stats.refused`), never query-logged.
|
||||
Localhost is not exempt (PLAN §10's localhost relaxation is the Phase 8 API limiter only).
|
||||
9. **Non-IN qclass**: skip filtering/cache/safe-search/uncloak, go straight upstream, log
|
||||
normally. Rare and harmless; special-casing CHAOS is scope creep.
|
||||
10. **Safe-search mechanics** (per safesearch.zig:56 doc): the outgoing query's QNAME becomes
|
||||
the target; the reply to the client keeps the original question, starts with a synthesized
|
||||
CNAME original→target, then copies only the A/AAAA answer records from the upstream
|
||||
response verbatim (their RDATA is raw bytes, so no compression-pointer re-encoding; other
|
||||
rtypes are skipped). Applies only on the upstream path — never to local records or forward
|
||||
zones.
|
||||
11. **CNAME uncloaking** (PLAN §6.3): walk the answer-section CNAME chain of the upstream
|
||||
response — no re-resolution. Depth 8 = at most 8 CNAME links followed. Each target is
|
||||
normalized and evaluated for the same group; a blocked target synthesizes a blocked
|
||||
response for the ORIGINAL qname/qtype. `block_reason` = `"cname:" ++ @tagName(reason)`
|
||||
(fits the 32-byte Entry field). Uncloaked-blocked responses are not cached. Skipped when
|
||||
paused, for forward zones, and for non-IN.
|
||||
12. **Local CNAME records** are returned as-is (writeAnswers emits the CNAME and stops,
|
||||
milestone-5 As-built). The client re-queries the target, which then flows through the full
|
||||
pipeline — that is how §6.4's "targets resolve through the normal pipeline" is satisfied.
|
||||
13. **ECS** (PLAN §6.1): `strip` (default) removes a client-sent ECS option from the outgoing
|
||||
query by rebuilding it (header + question + OPT sans option 8); other options are
|
||||
preserved. `forward` sends the query unchanged and puts the raw ECS option payload
|
||||
(≤ `dns_cache.max_ecs_len` = 40 bytes) into the cache key. If a rewritten query cannot fit
|
||||
the 512-byte build buffer (freak OPT payloads), the original is forwarded unchanged,
|
||||
`Stats.ecs_strip_failed` counts it, and the query is excluded from the cache in both
|
||||
directions (the subnet reached the resolver; the strip-mode key says nothing about it).
|
||||
14. **Client auto-materialization** (PLAN §7.2): never a hot-path DB write. The handler records
|
||||
the client in a mutex-guarded fixed-capacity pending table; a background loop flushes every
|
||||
60 s with its OWN config.db connection: `INSERT ... hand_edited=0 ON CONFLICT(ip) DO UPDATE
|
||||
last_seen`. At most one write per client per flush. Table full → drop silently (counted).
|
||||
15. **Snapshot staleness**: a materialized client reaches the matcher snapshot only on the next
|
||||
manager reload. Correct behavior does not depend on it: `groupForClient` already falls back
|
||||
to prefix/default, and materialized rows exist for UI visibility (Phase 8 reads the DB
|
||||
directly). No reload is triggered per client.
|
||||
16. **Client pruning** (PLAN §7.2 + milestone-6 ruling 1): the same background loop prunes
|
||||
daily: `DELETE FROM clients WHERE hand_edited=0 AND last_seen < now - retention_days`
|
||||
(`logging.retention_days`, same knob as the query log — documented). `retention.zig` stays
|
||||
frozen and querylog-only. `last_seen` is the only signal (§3.6 forbids cross-DB joins).
|
||||
17. **Disk gate**: the client flush/prune loop skips its pass when `writesAllowed()` is false;
|
||||
the logger holds its batches. As built: milestone-6.md:410 assigned the manager's
|
||||
consumption to Phase 7 and no seam existed, so S6 added `Manager.monitor:
|
||||
?*disk_monitor.Monitor` (set by the composition root after `init`, before `runScheduler`)
|
||||
plus a `refreshes_gated` counter read through `refreshesGated()`. Only scheduled passes are
|
||||
gated: `runScheduler`'s periodic pass and the source-refresh half of the startup pass.
|
||||
`reload` and `refreshAll` stay ungated (operator actions). The startup pass's
|
||||
`reloadLocked` runs ahead of the gate: publishing an already-compiled snapshot is a read,
|
||||
and a full disk must not cost the household its filtering as well as its downloads. A
|
||||
gated pass logs one warn line and counts.
|
||||
18. **Pause/resume** (PLAN §13.1 mechanism): suspends FILTERING only (evaluate + safe-search +
|
||||
uncloak are skipped); everything else — local records, forward zones, cache, upstream,
|
||||
logging — runs unchanged. Global, not per-group. In-memory only; a restart resumes
|
||||
filtering (safe default). State: one `std.atomic.Value(i64)` — 0 = active, -1 = paused
|
||||
indefinitely, else = paused until that unix second (auto-resume by compare, no timer task).
|
||||
Phase 8's endpoints will call `pauseFor`/`unpause` on the shared pointer.
|
||||
19. **Paused queries** log `blocked=false, block_reason=""`; `Stats.paused_queries` counts
|
||||
them. Evaluation is skipped entirely (not evaluated-then-ignored).
|
||||
20. **Log field semantics**: `response_time_us` = wall time from `handle` entry to reply ready.
|
||||
`cache_hit`: true/false on the upstream path, null for local/forward/refused-never-logged;
|
||||
`upstream`: "" on cache hit, "local" for local records, resolver text for forward zones,
|
||||
the pool endpoint's canonical text for upstream answers (pool already tracks which entry
|
||||
answered — if unavailable without new plumbing, "" and a spec note; do not rebuild the
|
||||
pool for it: `upstream` may be "pool" as the compromise, S2 decides and records).
|
||||
`block_reason` = `@tagName(matcher.Reason)` truncated to 32, `"cname:"`-prefixed for
|
||||
uncloaking. Malformed/dropped queries are never query-logged (handler stats only).
|
||||
21. **Connections**: config.db — one opened by the app (migrate/bootstrap/readConfig, then
|
||||
handed to the manager for reloads), plus one dedicated to the client loop. querylog.db —
|
||||
one for the log writer, one for retention (retention.zig:79 contract). Four total.
|
||||
22. **Shutdown order**: shutdown event fires → `logger.shutdown(io)` (closes the queue; the
|
||||
writer drains and its task returns; a writer stuck in the disk gate holds its final batch —
|
||||
known, documented) → `group.cancel(io)` (requests cancel and joins every task) → deinit
|
||||
servers/pool/manager/state → close DBs → return. Canceled UDP recv / TCP accept return
|
||||
`error.Canceled` on the Threaded backend (Threaded.zig:12439,12877).
|
||||
23. **No runtime config re-read** in Phase 7. The blocklist scheduler reloads snapshots;
|
||||
everything else is restart-required until Phase 8 mutates the DB and calls reload.
|
||||
24. **No new config fields, no schema change.** Pause is in-memory; pruning reuses
|
||||
`retention_days`; ECS uses the existing `edns.ecs_mode`.
|
||||
25. **Performance** (PLAN §18): no formal benchmark in this milestone; the integration test
|
||||
asserts correctness only. The targets stay in PLAN for a later measurement pass.
|
||||
1. **Scope**: Phase 7 owns `nxdns run` end to end: `src/app.zig` (composition root), `src/server/shutdown.zig` (signal → event), the `cli.runRun` body, and both listener edits.
|
||||
2. **Signals**: no signalfd/epoll/timerfd. `std.posix.sigaction` handler for INT and TERM that calls `std.Io.Event.set` on a file-scope event (`set` is async-signal-safe on the Threaded Linux backend: raw futex wake, no locks — Threaded.zig:1105,17235).
|
||||
3. **Locking**: `DnsCache` and `RateLimiter` each get one `std.Io.Mutex` owned by the handler. Household scale makes contention irrelevant; sharding is over-engineering.
|
||||
4. **Null snapshot** (`Manager.acquire` returns null before the first successful reload): fail open — skip group/filter/safe-search, use the default group semantics, count `unfiltered_queries`. DNS availability beats filtering for a household. The app attempts one synchronous `reload` before serving; a failure warns and serving starts anyway.
|
||||
5. **Cache scope**: one global cache, no group in the key (PLAN §8). Safe because blocked responses are never cached. Safe-search responses are not cached either (13 domains, rare; caching them under the rewritten name would force answer re-encoding on hit). Paused-mode upstream answers cache normally (they are ordinary upstream responses). Forward-zone responses cache normally (PLAN §6.5) under the same global key.
|
||||
6. **Local vs forward zone**: local records win (checked first; PLAN §6.4 puts them before everything but rate limit/parse).
|
||||
7. **Forward zones bypass** filtering, safe-search, and uncloaking (PLAN §6.5). They are logged with `upstream` = the resolver's canonical text.
|
||||
8. **Rate limit position**: after the 12-byte header read (the REFUSED reply needs the ID), before full parse. Refused queries are counted (`Stats.refused`), never query-logged. Localhost is not exempt (PLAN §10's localhost relaxation is the Phase 8 API limiter only).
|
||||
9. **Non-IN qclass**: skip filtering/cache/safe-search/uncloak, go straight upstream, log normally. Rare and harmless; special-casing CHAOS is scope creep.
|
||||
10. **Safe-search mechanics** (per safesearch.zig:56 doc): the outgoing query's QNAME becomes the target; the reply to the client keeps the original question, starts with a synthesized CNAME original→target, then copies only the A/AAAA answer records from the upstream response verbatim (their RDATA is raw bytes, so no compression-pointer re-encoding; other rtypes are skipped). Applies only on the upstream path — never to local records or forward zones.
|
||||
11. **CNAME uncloaking** (PLAN §6.3): walk the answer-section CNAME chain of the upstream response — no re-resolution. Depth 8 = at most 8 CNAME links followed. Each target is normalized and evaluated for the same group; a blocked target synthesizes a blocked response for the ORIGINAL qname/qtype. `block_reason` = `"cname:" ++ @tagName(reason)` (fits the 32-byte Entry field). Uncloaked-blocked responses are not cached. Skipped when paused, for forward zones, and for non-IN.
|
||||
12. **Local CNAME records** are returned as-is (writeAnswers emits the CNAME and stops, milestone-5 As-built). The client re-queries the target, which then flows through the full pipeline — that is how §6.4's "targets resolve through the normal pipeline" is satisfied.
|
||||
13. **ECS** (PLAN §6.1): `strip` (default) removes a client-sent ECS option from the outgoing query by rebuilding it (header + question + OPT sans option 8); other options are preserved. `forward` sends the query unchanged and puts the raw ECS option payload (≤ `dns_cache.max_ecs_len` = 40 bytes) into the cache key. If a rewritten query cannot fit the 512-byte build buffer (freak OPT payloads), the original is forwarded unchanged, `Stats.ecs_strip_failed` counts it, and the query is excluded from the cache in both directions (the subnet reached the resolver; the strip-mode key says nothing about it).
|
||||
14. **Client auto-materialization** (PLAN §7.2): never a hot-path DB write. The handler records the client in a mutex-guarded fixed-capacity pending table; a background loop flushes every 60 s with its OWN config.db connection: `INSERT ... hand_edited=0 ON CONFLICT(ip) DO UPDATE last_seen`. At most one write per client per flush. Table full → drop silently (counted).
|
||||
15. **Snapshot staleness**: a materialized client reaches the matcher snapshot only on the next manager reload. Correct behavior does not depend on it: `groupForClient` already falls back to prefix/default, and materialized rows exist for UI visibility (Phase 8 reads the DB directly). No reload is triggered per client.
|
||||
16. **Client pruning** (PLAN §7.2 + milestone-6 ruling 1): the same background loop prunes daily: `DELETE FROM clients WHERE hand_edited=0 AND last_seen < now - retention_days` (`logging.retention_days`, same knob as the query log — documented). `retention.zig` stays frozen and querylog-only. `last_seen` is the only signal (§3.6 forbids cross-DB joins).
|
||||
17. **Disk gate**: the client flush/prune loop skips its pass when `writesAllowed()` is false; the logger holds its batches. As built: milestone-6.md:410 assigned the manager's consumption to Phase 7 and no seam existed, so S6 added `Manager.monitor: ?*disk_monitor.Monitor` (set by the composition root after `init`, before `runScheduler`) plus a `refreshes_gated` counter read through `refreshesGated()`. Only scheduled passes are gated: `runScheduler`'s periodic pass and the source-refresh half of the startup pass. `reload` and `refreshAll` stay ungated (operator actions). The startup pass's `reloadLocked` runs ahead of the gate: publishing an already-compiled snapshot is a read, and a full disk must not cost the household its filtering as well as its downloads. A gated pass logs one warn line and counts.
|
||||
18. **Pause/resume** (PLAN §13.1 mechanism): suspends FILTERING only (evaluate + safe-search + uncloak are skipped); everything else — local records, forward zones, cache, upstream, logging — runs unchanged. Global, not per-group. In-memory only; a restart resumes filtering (safe default). State: one `std.atomic.Value(i64)` — 0 = active, -1 = paused indefinitely, else = paused until that unix second (auto-resume by compare, no timer task). Phase 8's endpoints will call `pauseFor`/`unpause` on the shared pointer.
|
||||
19. **Paused queries** log `blocked=false, block_reason=""`; `Stats.paused_queries` counts them. Evaluation is skipped entirely (not evaluated-then-ignored).
|
||||
20. **Log field semantics**: `response_time_us` = wall time from `handle` entry to reply ready. `cache_hit`: true/false on the upstream path, null for local/forward/refused-never-logged; `upstream`: "" on cache hit, "local" for local records, resolver text for forward zones, the pool endpoint's canonical text for upstream answers (pool already tracks which entry answered — if unavailable without new plumbing, "" and a spec note; do not rebuild the pool for it: `upstream` may be "pool" as the compromise, S2 decides and records). `block_reason` = `@tagName(matcher.Reason)` truncated to 32, `"cname:"`-prefixed for uncloaking. Malformed/dropped queries are never query-logged (handler stats only).
|
||||
21. **Connections**: config.db — one opened by the app (migrate/bootstrap/readConfig, then handed to the manager for reloads), plus one dedicated to the client loop. querylog.db — one for the log writer, one for retention (retention.zig:79 contract). Four total.
|
||||
22. **Shutdown order**: shutdown event fires → `logger.shutdown(io)` (closes the queue; the writer drains and its task returns; a writer stuck in the disk gate holds its final batch — known, documented) → `group.cancel(io)` (requests cancel and joins every task) → deinit servers/pool/manager/state → close DBs → return. Canceled UDP recv / TCP accept return `error.Canceled` on the Threaded backend (Threaded.zig:12439,12877).
|
||||
23. **No runtime config re-read** in Phase 7. The blocklist scheduler reloads snapshots; everything else is restart-required until Phase 8 mutates the DB and calls reload.
|
||||
24. **No new config fields, no schema change.** Pause is in-memory; pruning reuses `retention_days`; ECS uses the existing `edns.ecs_mode`.
|
||||
25. **Performance** (PLAN §18): no formal benchmark in this milestone; the integration test asserts correctness only. The targets stay in PLAN for a later measurement pass.
|
||||
|
||||
## Sessions
|
||||
|
||||
Dependency graph: S1 + S4 parallel (wave 1) → S2 (needs S1's edns API) → S3 (needs S2's
|
||||
signature in code) → S6 (needs S2, S3, S4) → S7 (needs S6). The orchestrator wires
|
||||
`src/tests.zig` after each wave.
|
||||
Dependency graph: S1 + S4 parallel (wave 1) → S2 (needs S1's edns API) → S3 (needs S2's signature in code) → S6 (needs S2, S3, S4) → S7 (needs S6). The orchestrator wires `src/tests.zig` after each wave.
|
||||
|
||||
---
|
||||
|
||||
@@ -142,13 +59,7 @@ pub fn stripEcs(query: []const u8, pkt: packet.Packet, opt: OptRecord, out: []u8
|
||||
error{ BadOption, Overflow }!StripResult
|
||||
```
|
||||
|
||||
Constraints: pure, allocation-free, no Io. The rebuilt query keeps the original ID, flags and
|
||||
question bytes verbatim (copy `query[0..opt_record_offset_of_question_end]` — concretely: copy
|
||||
everything up to the OPT record's start; the OPT is the sole additional record in a query per
|
||||
the existing handler validation, and `packet.parse` yields its offset). ARCOUNT stays 1 when
|
||||
the OPT is re-emitted (it always is — only the option list shrinks). Use the existing
|
||||
`encodeOpt` for the re-emission. The RDATA walk follows the same bounds checks `findOption`
|
||||
uses.
|
||||
Constraints: pure, allocation-free, no Io. The rebuilt query keeps the original ID, flags and question bytes verbatim (copy `query[0..opt_record_offset_of_question_end]` — concretely: copy everything up to the OPT record's start; the OPT is the sole additional record in a query per the existing handler validation, and `packet.parse` yields its offset). ARCOUNT stays 1 when the OPT is re-emitted (it always is — only the option list shrinks). Use the existing `encodeOpt` for the re-emission. The RDATA walk follows the same bounds checks `findOption` uses.
|
||||
|
||||
### S1.2 Acceptance
|
||||
|
||||
@@ -161,36 +72,13 @@ uses.
|
||||
|
||||
### S1 As built
|
||||
|
||||
Two spec claims were wrong and the implementation corrects them. (1) The handler does NOT
|
||||
guarantee the OPT is the sole additional record (handler.zig:105 rejects only misplaced OPTs;
|
||||
packet.parse rejects only a second OPT) — `stripEcs` therefore copies the bytes after the OPT
|
||||
verbatim and all four section counts survive. Caveat, documented on the function: a compression
|
||||
pointer in a trailing record that targets a byte inside the OPT's option list shifts after the
|
||||
rewrite — such a pointer is already malformed (RFC 1035 §4.1.4) and the consequence is an
|
||||
upstream FORMERR. (2) `packet.parse` does not yield the OPT's start offset; a private
|
||||
`optRecordStart` re-walks the sections and matches the record on its full RDATA span, so a
|
||||
hand-built `OptRecord` cannot aim it at a different record (deriving the start backwards is
|
||||
forgeable: a `0x00xx` transaction ID makes a pointer's low byte decode as a root name).
|
||||
`encodeOpt` takes one contiguous options slice, so `stripEcs` emits it with an empty list,
|
||||
streams the survivors, and patches RDLENGTH at `opt_rdlength_offset` (9); a guard test pins
|
||||
that offset against `encodeOpt`'s real output. All options with code 8 are removed (a malformed
|
||||
query can carry several). Walk failures map to `error.BadOption` (frozen error set; reachable
|
||||
only for hand-assembled Packets, documented). After review: `stripEcs` asserts both that
|
||||
`query` aliases `pkt.bytes` and that `out` is pointer-range disjoint from `query` in both
|
||||
directions; the caller owns keeping the rebuild buffer separate from the received query
|
||||
buffer (S2's `scratch.build` satisfies this). The import is aliased `packet_mod` (seven existing declarations
|
||||
named `packet`). `ecsPayload` was built here, then removed in the review round when the
|
||||
handler's `subnetForKey` single-walk replaced its only caller (see S2 As-built); the
|
||||
forward-mode clamp policy lives there — a payload longer than `dns_cache.max_ecs_len` (40)
|
||||
makes the query uncacheable (truncating the key would merge distinct subnets). 7 S1 tests
|
||||
remain, 25 in file.
|
||||
Two spec claims were wrong and the implementation corrects them. (1) The handler does NOT guarantee the OPT is the sole additional record (handler.zig:105 rejects only misplaced OPTs; packet.parse rejects only a second OPT) — `stripEcs` therefore copies the bytes after the OPT verbatim and all four section counts survive. Caveat, documented on the function: a compression pointer in a trailing record that targets a byte inside the OPT's option list shifts after the rewrite — such a pointer is already malformed (RFC 1035 §4.1.4) and the consequence is an upstream FORMERR. (2) `packet.parse` does not yield the OPT's start offset; a private `optRecordStart` re-walks the sections and matches the record on its full RDATA span, so a hand-built `OptRecord` cannot aim it at a different record (deriving the start backwards is forgeable: a `0x00xx` transaction ID makes a pointer's low byte decode as a root name). `encodeOpt` takes one contiguous options slice, so `stripEcs` emits it with an empty list, streams the survivors, and patches RDLENGTH at `opt_rdlength_offset` (9); a guard test pins that offset against `encodeOpt`'s real output. All options with code 8 are removed (a malformed query can carry several). Walk failures map to `error.BadOption` (frozen error set; reachable only for hand-assembled Packets, documented). After review: `stripEcs` asserts both that `query` aliases `pkt.bytes` and that `out` is pointer-range disjoint from `query` in both directions; the caller owns keeping the rebuild buffer separate from the received query buffer (S2's `scratch.build` satisfies this). The import is aliased `packet_mod` (seven existing declarations named `packet`). `ecsPayload` was built here, then removed in the review round when the handler's `subnetForKey` single-walk replaced its only caller (see S2 As-built); the forward-mode clamp policy lives there — a payload longer than `dns_cache.max_ecs_len` (40) makes the query uncacheable (truncating the key would merge distinct subnets). 7 S1 tests remain, 25 in file.
|
||||
|
||||
---
|
||||
|
||||
## Session S2: pipeline in server/handler.zig (+ pause.zig)
|
||||
|
||||
Owns: `src/server/handler.zig` (rewrite around the existing skeleton), `src/server/pause.zig`
|
||||
(new). The biggest session. Everything below is frozen interface — S3 and S6 build against it.
|
||||
Owns: `src/server/handler.zig` (rewrite around the existing skeleton), `src/server/pause.zig` (new). The biggest session. Everything below is frozen interface — S3 and S6 build against it.
|
||||
|
||||
### S2.1 pause.zig
|
||||
|
||||
@@ -203,9 +91,7 @@ pub const Pause = struct {
|
||||
};
|
||||
```
|
||||
|
||||
Auto-resume is the compare in `isPaused` — no timer. `.monotonic` ordering is sufficient (a
|
||||
lone i64 flag, no dependent data). Tests: the three states, expiry boundary (`now == until`
|
||||
is resumed), pauseFor overwrites a previous pause.
|
||||
Auto-resume is the compare in `isPaused` — no timer. `.monotonic` ordering is sufficient (a lone i64 flag, no dependent data). Tests: the three states, expiry boundary (`now == until` is resumed), pauseFor overwrites a previous pause.
|
||||
|
||||
### S2.2 Handler struct (frozen for S3/S6)
|
||||
|
||||
@@ -229,10 +115,7 @@ pub const Handler = struct {
|
||||
};
|
||||
```
|
||||
|
||||
Every `?*` defaults to null so existing tests and the check command construct a bare handler
|
||||
exactly as today (upstream + blocking + records/zones + timeout are the only required fields;
|
||||
`records`/`zones` may point at the shared `Records.empty`/`Zones.empty` constants — if
|
||||
`Zones.empty` does not exist yet, S2 adds it mirroring `Records.empty`).
|
||||
Every `?*` defaults to null so existing tests and the check command construct a bare handler exactly as today (upstream + blocking + records/zones + timeout are the only required fields; `records`/`zones` may point at the shared `Records.empty`/`Zones.empty` constants — if `Zones.empty` does not exist yet, S2 adds it mirroring `Records.empty`).
|
||||
|
||||
```zig
|
||||
pub const Scratch = struct {
|
||||
@@ -246,67 +129,29 @@ pub fn handle(self: *Handler, io: std.Io, which: Transport, from: address.NetAdd
|
||||
query: []const u8, response_buf: []u8, scratch: *Scratch) Outcome
|
||||
```
|
||||
|
||||
`Outcome` unchanged. Still no error union — every failure is a synthesized rcode or a counted
|
||||
drop. New Stats counters (all `std.atomic.Value(u64)`): `refused`, `blocked`,
|
||||
`uncloak_blocked`, `local_answers`, `forward_zone_answers`, `cache_hits`, `paused_queries`,
|
||||
`unfiltered_queries`, `safesearch_rewrites`, `ecs_strip_failed`, `tracker_full`.
|
||||
`Outcome` unchanged. Still no error union — every failure is a synthesized rcode or a counted drop. New Stats counters (all `std.atomic.Value(u64)`): `refused`, `blocked`, `uncloak_blocked`, `local_answers`, `forward_zone_answers`, `cache_hits`, `paused_queries`, `unfiltered_queries`, `safesearch_rewrites`, `ecs_strip_failed`, `tracker_full`.
|
||||
|
||||
### S2.3 Pipeline order inside handle (binding; rulings 4-13, 18-20 apply)
|
||||
|
||||
1. Read the 12-byte header (existing code). QR/opcode/qdcount checks as today.
|
||||
2. Rate limit: `limiter_mutex.lock(io)`, `check(now, from.key())`, unlock. Refused →
|
||||
synthesize REFUSED (reuse `synthesize`), count, return. `now` from
|
||||
`std.Io.Clock.awake.now(io)` (limiter windows are awake-anchored, milestone-6 As-built).
|
||||
3. Full parse + OPT validation (existing code). Record `start_us` once from
|
||||
`std.Io.Clock.real.now(io)` for both `response_time_us` and cache `now_s`.
|
||||
2. Rate limit: `limiter_mutex.lock(io)`, `check(now, from.key())`, unlock. Refused → synthesize REFUSED (reuse `synthesize`), count, return. `now` from `std.Io.Clock.awake.now(io)` (limiter windows are awake-anchored, milestone-6 As-built).
|
||||
3. Full parse + OPT validation (existing code). Record `start_us` once from `std.Io.Clock.real.now(io)` for both `response_time_us` and cache `now_s`.
|
||||
4. `tracker.track(from)` (nonblocking; full table counts `tracker_full`).
|
||||
5. Snapshot: `if (self.manager) |m| m.acquire(io) else null`; released once on every exit path
|
||||
(single defer). Null snapshot → `unfiltered_queries`, group features off (ruling 4).
|
||||
5. Snapshot: `if (self.manager) |m| m.acquire(io) else null`; released once on every exit path (single defer). Null snapshot → `unfiltered_queries`, group features off (ruling 4).
|
||||
6. Non-IN qclass → step 12 (upstream direct), no cache (ruling 9).
|
||||
7. Normalize qname into `scratch.normalize`.
|
||||
8. Local records: `hasName` → `lookup`; answer via `ResponseBuilder` + `writeAnswers` +
|
||||
`addOptEcho`, authoritative. A name that exists with no records of the qtype → NODATA,
|
||||
authoritative. Log (`upstream="local"`, `cache_hit=null`) and return.
|
||||
9. Forward zones: `zones.match` → construct `ForwardClient` on the stack
|
||||
(`init(zone.resolver, &scratch.frame, self.forward_read_timeout)`), cache get → miss →
|
||||
`exchange` into `response_buf` → cache put (classify), log (`upstream=resolver text`),
|
||||
return. Filtering/safe-search/uncloak bypassed (ruling 7). Exchange failure → SERVFAIL.
|
||||
8. Local records: `hasName` → `lookup`; answer via `ResponseBuilder` + `writeAnswers` + `addOptEcho`, authoritative. A name that exists with no records of the qtype → NODATA, authoritative. Log (`upstream="local"`, `cache_hit=null`) and return.
|
||||
9. Forward zones: `zones.match` → construct `ForwardClient` on the stack (`init(zone.resolver, &scratch.frame, self.forward_read_timeout)`), cache get → miss → `exchange` into `response_buf` → cache put (classify), log (`upstream=resolver text`), return. Filtering/safe-search/uncloak bypassed (ruling 7). Exchange failure → SERVFAIL.
|
||||
10. Pause check (`isPaused(now_s)`) → skip 11 and the uncloak in 14; count `paused_queries`.
|
||||
11. Filtering: `snapshot.evaluate(group, domain)`; blocked → `response.writeBlocked` into
|
||||
`response_buf`, log (`blocked=true`, reason), return. Then safe-search:
|
||||
`snapshot.safeSearch(group)` and `safesearch.rewrite(domain)` → remember the target;
|
||||
outgoing QNAME = target (query rebuilt in `scratch.build`; count `safesearch_rewrites`).
|
||||
12. ECS (ruling 13): `.strip` → `edns.stripEcs` into `scratch.build` (compose with the
|
||||
safe-search rebuild: build once with both the final QNAME and the filtered OPT — one
|
||||
rebuild, not two); `.forward` → `edns.ecsPayload` for the key. Overflow →
|
||||
`ecs_strip_failed`, forward original.
|
||||
13. Cache get (skip for safe-search, ruling 5): key = `buildKey(outgoing qname, qtype, qclass,
|
||||
do_bit, ecs_payload_or_null)` under `cache_mutex`; hit → copy is already in
|
||||
`response_buf`-sized `out` (pass `response_buf`), `packet.setId`, UDP truncation check as
|
||||
today, log (`cache_hit=true`, `upstream=""`), return.
|
||||
14. Upstream: `self.upstream.exchange(io, outgoing, response_buf)` (existing error mapping to
|
||||
SERVFAIL/drop stays). Then uncloak (ruling 11) unless paused/non-IN: iterate
|
||||
`packet.answers`, follow CNAMEs from the outgoing qname via `record.rdataCname`, depth ≤ 8,
|
||||
normalize each target, `evaluate`; blocked → `writeBlocked` for the ORIGINAL question,
|
||||
count `uncloak_blocked`, log (`blocked=true`, `"cname:"` reason), return.
|
||||
15. Safe-search synthesis (ruling 10): new `ResponseBuilder` with the original question,
|
||||
`addAnswer(original, CNAME, target)`, then each A/AAAA answer record of the upstream
|
||||
response verbatim (owner = target name), `addOptEcho`. The upstream bytes are read from
|
||||
`response_buf` while the builder writes into... **the builder needs its own buffer**: build
|
||||
into a second half — concretely `handle` requires `response_buf.len >= 1024` already via
|
||||
assert; S2 adds a `synth: [4096]u8` field to `Scratch` for the safe-search build, then
|
||||
copies the finished reply into `response_buf`. A synthesized reply that cannot fit 4096 →
|
||||
SERVFAIL (safe-search answers are tiny in practice).
|
||||
16. Cache put: upstream path only, not blocked/safe-search (ruling 5): `classify` →
|
||||
`cache_mutex` → `put`. Allocation failure inside put → skip caching (already
|
||||
`Allocator.Error`; count nothing, put's stats cover it).
|
||||
17. UDP truncation check (existing). Log the entry: `Entry.init(.{ .timestamp = now_s, .domain
|
||||
= normalized domain, .client_ip = formatted from, .qtype, .blocked=false, .response_time_us,
|
||||
.cache_hit=false, .upstream = per ruling 20 })`, `logger.log(io, entry)`. Return reply.
|
||||
11. Filtering: `snapshot.evaluate(group, domain)`; blocked → `response.writeBlocked` into `response_buf`, log (`blocked=true`, reason), return. Then safe-search: `snapshot.safeSearch(group)` and `safesearch.rewrite(domain)` → remember the target; outgoing QNAME = target (query rebuilt in `scratch.build`; count `safesearch_rewrites`).
|
||||
12. ECS (ruling 13): `.strip` → `edns.stripEcs` into `scratch.build` (compose with the safe-search rebuild: build once with both the final QNAME and the filtered OPT — one rebuild, not two); `.forward` → `edns.ecsPayload` for the key. Overflow → `ecs_strip_failed`, forward original.
|
||||
13. Cache get (skip for safe-search, ruling 5): key = `buildKey(outgoing qname, qtype, qclass, do_bit, ecs_payload_or_null)` under `cache_mutex`; hit → copy is already in `response_buf`-sized `out` (pass `response_buf`), `packet.setId`, UDP truncation check as today, log (`cache_hit=true`, `upstream=""`), return.
|
||||
14. Upstream: `self.upstream.exchange(io, outgoing, response_buf)` (existing error mapping to SERVFAIL/drop stays). Then uncloak (ruling 11) unless paused/non-IN: iterate `packet.answers`, follow CNAMEs from the outgoing qname via `record.rdataCname`, depth ≤ 8, normalize each target, `evaluate`; blocked → `writeBlocked` for the ORIGINAL question, count `uncloak_blocked`, log (`blocked=true`, `"cname:"` reason), return.
|
||||
15. Safe-search synthesis (ruling 10): new `ResponseBuilder` with the original question, `addAnswer(original, CNAME, target)`, then each A/AAAA answer record of the upstream response verbatim (owner = target name), `addOptEcho`. The upstream bytes are read from `response_buf` while the builder writes into... **the builder needs its own buffer**: build into a second half — concretely `handle` requires `response_buf.len >= 1024` already via assert; S2 adds a `synth: [4096]u8` field to `Scratch` for the safe-search build, then copies the finished reply into `response_buf`. A synthesized reply that cannot fit 4096 → SERVFAIL (safe-search answers are tiny in practice).
|
||||
16. Cache put: upstream path only, not blocked/safe-search (ruling 5): `classify` → `cache_mutex` → `put`. Allocation failure inside put → skip caching (already `Allocator.Error`; count nothing, put's stats cover it).
|
||||
17. UDP truncation check (existing). Log the entry: `Entry.init(.{ .timestamp = now_s, .domain = normalized domain, .client_ip = formatted from, .qtype, .blocked=false, .response_time_us, .cache_hit=false, .upstream = per ruling 20 })`, `logger.log(io, entry)`. Return reply.
|
||||
|
||||
The query-log write happens on every replied path (blocked, local, forward, cache, upstream,
|
||||
paused) — one call site per return is acceptable; a tiny private `logEntry` helper keeps it
|
||||
readable. Refused/malformed/dropped are never logged (rulings 8, 20).
|
||||
The query-log write happens on every replied path (blocked, local, forward, cache, upstream, paused) — one call site per return is acceptable; a tiny private `logEntry` helper keeps it readable. Refused/malformed/dropped are never logged (rulings 8, 20).
|
||||
|
||||
### S2.4 Acceptance
|
||||
|
||||
@@ -326,96 +171,22 @@ readable. Refused/malformed/dropped are never logged (rulings 8, 20).
|
||||
|
||||
### S2 As built
|
||||
|
||||
`Handler` gained one additive field: `negative_ttl_max: u32 = 0` (set from
|
||||
`cfg.cache.negative_ttl_max` — `DnsCache` keeps no copy of its config and `classify` needs the
|
||||
value; **S6 must wire it or negative caching is silently off**). `Scratch` gained `synth:
|
||||
[4096]u8` (spec-blessed) and `uncloak: [types.max_name_len]u8` (target normalization must not
|
||||
clobber `normalize`, which still holds the logged name). `Zones.empty` already existed
|
||||
(forward_zones.zig:34) — the S2.2 parenthetical was stale. Both mutexes and the tracker use
|
||||
`lockUncancelable` — `std.Io.Mutex.lock` returns `Cancelable!void` (Io.zig:1602) and `handle`
|
||||
has no error union. Safe-search + ECS strip compose via two buffers: rebuild into
|
||||
`scratch.build`, strip into `scratch.synth` (stripEcs requires non-overlapping output); `synth`
|
||||
is reused for the reply after the upstream answers. Uncloaking is also skipped after a
|
||||
safe-search rewrite (the answers belong to the provider's target, which the client never asked
|
||||
about). `safesearch_rewrites` counts at the rebuild, not the table lookup. SERVFAIL paths are
|
||||
counted but never query-logged (the Entry schema has no status field; a logged row would read
|
||||
as success — accepted ruling). Forward zones are exempt from the ECS strip (accepted ruling:
|
||||
the RFC 7871 concern is third-party upstreams, not the LAN resolver). Ruling 20's open choice:
|
||||
`upstream = "pool"` for pool answers; forward-zone rows log `cache_hit` false/true (ruling 20's
|
||||
"null" contradicted ruling 5 — null is now local records and blocked answers only). The
|
||||
safe-search CNAME TTL is the minimum TTL of the copied A/AAAA records, zero when none.
|
||||
`handle` keeps the 512-byte floor assert (the spec's "1024" was wrong); the copy out of
|
||||
`synth` is length-checked with SERVFAIL fallback. `tracker_full` mirrors
|
||||
`Tracker.snapshotStats().dropped_full` after each track (track returns nothing; an always-zero
|
||||
counter would lie). `max_cname_depth = 8` public. `queries` still means pool-answered queries.
|
||||
23 new handler tests + 25 adapted + 6 pause tests. After review, two ECS cache holes fixed at
|
||||
the root: (1) a failed ECS strip forwards the client's subnet upstream, so the answer may be
|
||||
subnet-specific while the `.strip`-mode key names no subnet — `outgoingQuery` returns
|
||||
`Outgoing { bytes, cacheable }` and a failed strip sets `cacheable = false`, skipping both the
|
||||
cache get and the put; (2) `.forward` mode forwards the whole OPT but `edns.findOption`
|
||||
reports only the first ECS option, so a query carrying several would key on one subnet while
|
||||
the resolver answered for another — a local `subnetForKey` walks the option list once and
|
||||
returns `.uncacheable` for a repeated option 8, for a payload over `dns_cache.max_ecs_len`,
|
||||
and for a list that will not walk; `cacheKey` then declines to cache. RFC 7871 §6 permits one
|
||||
ECS option, but a query is client-controlled, so a duplicate is treated as hostile. Both fixes
|
||||
are mutation-checked. `edns.ecsPayload` lost its only caller (the key needs the option count
|
||||
as well as the payload; one walk yields both) and is removed from edns.zig as dead surface. NOTE: the plain `zig build test` does not
|
||||
typecheck the listener call sites (serve is never analyzed there) — only `-Dintegration`
|
||||
validates S3's edit.
|
||||
`Handler` gained one additive field: `negative_ttl_max: u32 = 0` (set from `cfg.cache.negative_ttl_max` — `DnsCache` keeps no copy of its config and `classify` needs the value; **S6 must wire it or negative caching is silently off**). `Scratch` gained `synth: [4096]u8` (spec-blessed) and `uncloak: [types.max_name_len]u8` (target normalization must not clobber `normalize`, which still holds the logged name). `Zones.empty` already existed (forward_zones.zig:34) — the S2.2 parenthetical was stale. Both mutexes and the tracker use `lockUncancelable` — `std.Io.Mutex.lock` returns `Cancelable!void` (Io.zig:1602) and `handle` has no error union. Safe-search + ECS strip compose via two buffers: rebuild into `scratch.build`, strip into `scratch.synth` (stripEcs requires non-overlapping output); `synth` is reused for the reply after the upstream answers. Uncloaking is also skipped after a safe-search rewrite (the answers belong to the provider's target, which the client never asked about). `safesearch_rewrites` counts at the rebuild, not the table lookup. SERVFAIL paths are counted but never query-logged (the Entry schema has no status field; a logged row would read as success — accepted ruling). Forward zones are exempt from the ECS strip (accepted ruling: the RFC 7871 concern is third-party upstreams, not the LAN resolver). Ruling 20's open choice: `upstream = "pool"` for pool answers; forward-zone rows log `cache_hit` false/true (ruling 20's "null" contradicted ruling 5 — null is now local records and blocked answers only). The safe-search CNAME TTL is the minimum TTL of the copied A/AAAA records, zero when none. `handle` keeps the 512-byte floor assert (the spec's "1024" was wrong); the copy out of `synth` is length-checked with SERVFAIL fallback. `tracker_full` mirrors `Tracker.snapshotStats().dropped_full` after each track (track returns nothing; an always-zero counter would lie). `max_cname_depth = 8` public. `queries` still means pool-answered queries. 23 new handler tests + 25 adapted + 6 pause tests. After review, two ECS cache holes fixed at the root: (1) a failed ECS strip forwards the client's subnet upstream, so the answer may be subnet-specific while the `.strip`-mode key names no subnet — `outgoingQuery` returns `Outgoing { bytes, cacheable }` and a failed strip sets `cacheable = false`, skipping both the cache get and the put; (2) `.forward` mode forwards the whole OPT but `edns.findOption` reports only the first ECS option, so a query carrying several would key on one subnet while the resolver answered for another — a local `subnetForKey` walks the option list once and returns `.uncacheable` for a repeated option 8, for a payload over `dns_cache.max_ecs_len`, and for a list that will not walk; `cacheKey` then declines to cache. RFC 7871 §6 permits one ECS option, but a query is client-controlled, so a duplicate is treated as hostile. Both fixes are mutation-checked. `edns.ecsPayload` lost its only caller (the key needs the option count as well as the payload; one walk yields both) and is removed from edns.zig as dead surface. NOTE: the plain `zig build test` does not typecheck the listener call sites (serve is never analyzed there) — only `-Dintegration` validates S3's edit.
|
||||
|
||||
---
|
||||
|
||||
## Session S3: listeners pass the client address and scratch
|
||||
|
||||
Owns: `src/server/udp_server.zig`, `src/server/tcp_server.zig`, and the three integration test
|
||||
files whose `Handler{…}` literals the S2 struct change broke:
|
||||
`src/server/udp_server_integration_test.zig`, `src/server/tcp_server_integration_test.zig`,
|
||||
`src/server/resolver_integration_test.zig` (S2 measured nine stale literals missing
|
||||
`blocking`/`forward_read_timeout`/`records`/`zones` at udp:90/128/163/204, tcp:153/183/216/268,
|
||||
resolver:229, plus the handle-arity change once those are fixed).
|
||||
Owns: `src/server/udp_server.zig`, `src/server/tcp_server.zig`, and the three integration test files whose `Handler{…}` literals the S2 struct change broke: `src/server/udp_server_integration_test.zig`, `src/server/tcp_server_integration_test.zig`, `src/server/resolver_integration_test.zig` (S2 measured nine stale literals missing `blocking`/`forward_read_timeout`/`records`/`zones` at udp:90/128/163/204, tcp:153/183/216/268, resolver:229, plus the handle-arity change once those are fixed).
|
||||
|
||||
- `Slot` (udp) gains `scratch: handler.Scratch`; the call becomes
|
||||
`self.handler.handle(io, .udp, address.NetAddress.fromIp(slot.from), slot.query[0..slot.len],
|
||||
&slot.reply, &slot.scratch)`.
|
||||
- `Conn` (tcp) gains `peer: std.Io.net.IpAddress` (captured at claim from
|
||||
`stream.socket.address`) and `scratch: handler.Scratch`; call updated the same way.
|
||||
- No other behavior change. Slot/Conn are pool-allocated already, so the added ~5 KiB per slot
|
||||
changes no lifetime.
|
||||
- Acceptance: fmt/ast clean; both files' existing tests updated and passing; integration tests
|
||||
(`-Dintegration` udp/tcp/resolver) still pass unchanged in behavior.
|
||||
- `Slot` (udp) gains `scratch: handler.Scratch`; the call becomes `self.handler.handle(io, .udp, address.NetAddress.fromIp(slot.from), slot.query[0..slot.len], &slot.reply, &slot.scratch)`.
|
||||
- `Conn` (tcp) gains `peer: std.Io.net.IpAddress` (captured at claim from `stream.socket.address`) and `scratch: handler.Scratch`; call updated the same way.
|
||||
- No other behavior change. Slot/Conn are pool-allocated already, so the added ~5 KiB per slot changes no lifetime.
|
||||
- Acceptance: fmt/ast clean; both files' existing tests updated and passing; integration tests (`-Dintegration` udp/tcp/resolver) still pass unchanged in behavior.
|
||||
|
||||
### S3 As built
|
||||
|
||||
`Conn.peer` is captured in `claim` inside the same mutex-held section that stores the stream —
|
||||
written once per connection, read only by that connection's task; verified from source that
|
||||
`accept` fills `Socket.address` with the peer (net.zig:1442 → Threaded.zig:12439
|
||||
`addressFromPosix`), and a new TCP integration test asserts the captured peer is the real
|
||||
loopback client (an uninitialized field would typecheck and silently corrupt rate
|
||||
limiting/grouping/logging). Importing the `address` module forced renaming the bind/listen
|
||||
address parameters to `bind_address`/`listen_address` (Zig shadowing rule; call sites are
|
||||
positional, nothing else changed). The nine stale integration literals became a per-file
|
||||
`bareHandler` fixture deriving `blocking` from `model.Blocking{}` and `forward_read_timeout`
|
||||
from `model.readTimeout(.{})` (3000 ms, `.awake`), so the fixtures cannot drift from config
|
||||
defaults. Memory-cost comments updated for the scratch: UDP ≈74 KiB/slot (≈4.6 MiB at 64),
|
||||
TCP ≈137 KiB/slot (≈8.8 MiB at 64); the UDP <8 MiB budget assertion still holds.
|
||||
After review: `TcpServer.serve` distinguishes its two exit paths, because the composition root
|
||||
cancels its task group before any listener `deinit` runs (the deinit defers are declared
|
||||
first, so `group.cancel` executes first). `acceptLoop` returns a `Stop` enum: `.closing` when
|
||||
`deinit` stopped it (`error.SocketNotListening`, a `.shutting_down` claim, or the state flag)
|
||||
and `.canceled` when the task is being canceled (`error.Canceled` from `accept` or the retry
|
||||
sleep). On `.closing` the connection group drains under `.blocked` protection as before —
|
||||
`beginShutdown` already unblocked every live stream, so the wait is bounded by the shutdown.
|
||||
On `.canceled` the group is canceled instead: nothing has shut the streams down, `deinit`
|
||||
cannot run until `serve` returns, and RFC 7766 §6.2.1.1 lets a client hold a connection open
|
||||
indefinitely by asking again inside the idle budget — draining would let one client stall
|
||||
process shutdown. The cost is the single reply mid-write. The old await under `.blocked`
|
||||
protection was what made cancel invisible: `Syscall.start` (Threaded.zig:1349) never arms
|
||||
cancellation while protection is blocked. Per-connection cleanup is unaffected (`serveConn`'s
|
||||
`defer finish` runs on the canceled path; `finish` already used `lockUncancelable` + `.blocked`
|
||||
around the close). Pinned by "a canceled serve does not wait for a live connection" (blocked
|
||||
read, 600 s idle budget, cancel must return; reinstating drain-always hangs the suite). The
|
||||
same await shape remains in `UdpServer.serve` deliberately: a UDP task is one bounded upstream
|
||||
exchange, not a client-paced session.
|
||||
`Conn.peer` is captured in `claim` inside the same mutex-held section that stores the stream — written once per connection, read only by that connection's task; verified from source that `accept` fills `Socket.address` with the peer (net.zig:1442 → Threaded.zig:12439 `addressFromPosix`), and a new TCP integration test asserts the captured peer is the real loopback client (an uninitialized field would typecheck and silently corrupt rate limiting/grouping/logging). Importing the `address` module forced renaming the bind/listen address parameters to `bind_address`/`listen_address` (Zig shadowing rule; call sites are positional, nothing else changed). The nine stale integration literals became a per-file `bareHandler` fixture deriving `blocking` from `model.Blocking{}` and `forward_read_timeout` from `model.readTimeout(.{})` (3000 ms, `.awake`), so the fixtures cannot drift from config defaults. Memory-cost comments updated for the scratch: UDP ≈74 KiB/slot (≈4.6 MiB at 64), TCP ≈137 KiB/slot (≈8.8 MiB at 64); the UDP <8 MiB budget assertion still holds. After review: `TcpServer.serve` distinguishes its two exit paths, because the composition root cancels its task group before any listener `deinit` runs (the deinit defers are declared first, so `group.cancel` executes first). `acceptLoop` returns a `Stop` enum: `.closing` when `deinit` stopped it (`error.SocketNotListening`, a `.shutting_down` claim, or the state flag) and `.canceled` when the task is being canceled (`error.Canceled` from `accept` or the retry sleep). On `.closing` the connection group drains under `.blocked` protection as before — `beginShutdown` already unblocked every live stream, so the wait is bounded by the shutdown. On `.canceled` the group is canceled instead: nothing has shut the streams down, `deinit` cannot run until `serve` returns, and RFC 7766 §6.2.1.1 lets a client hold a connection open indefinitely by asking again inside the idle budget — draining would let one client stall process shutdown. The cost is the single reply mid-write. The old await under `.blocked` protection was what made cancel invisible: `Syscall.start` (Threaded.zig:1349) never arms cancellation while protection is blocked. Per-connection cleanup is unaffected (`serveConn`'s `defer finish` runs on the canceled path; `finish` already used `lockUncancelable` + `.blocked` around the close). Pinned by "a canceled serve does not wait for a live connection" (blocked read, 600 s idle budget, cancel must return; reinstating drain-always hangs the suite). The same await shape remains in `UdpServer.serve` deliberately: a UDP task is one bounded upstream exchange, not a client-paced session.
|
||||
|
||||
---
|
||||
|
||||
@@ -434,9 +205,7 @@ pub fn upsertSeen(database: *db.Db, ip: []const u8, now_s: i64) db.Error!void
|
||||
pub fn pruneStale(database: *db.Db, cutoff_s: i64) db.Error!u32
|
||||
```
|
||||
|
||||
`ON CONFLICT(ip) DO UPDATE SET last_seen=excluded.last_seen`. Check the actual clients schema
|
||||
(unique key on ip) in storage/config_schema.zig and use the real column names. The existing
|
||||
`insertClient` (hand_edited=1, import path) stays untouched.
|
||||
`ON CONFLICT(ip) DO UPDATE SET last_seen=excluded.last_seen`. Check the actual clients schema (unique key on ip) in storage/config_schema.zig and use the real column names. The existing `insertClient` (hand_edited=1, import path) stays untouched.
|
||||
|
||||
### S4.2 Tracker
|
||||
|
||||
@@ -455,54 +224,21 @@ pub const Tracker = struct {
|
||||
};
|
||||
```
|
||||
|
||||
`run` is the house loop shape (manager.zig:946): startup no-op, then every `flush_interval_s`
|
||||
on the `.boot` clock: skip the pass when `monitor` says writes are gated (ruling 17); else
|
||||
drain the pending table under the mutex into a local copy (release before I/O), one
|
||||
`upsertSeen` per client with `real` wall-clock seconds; every `prune_every_passes`-th pass also
|
||||
`pruneStale(now - retention_days * 86_400)`. A failed upsert warns once per pass and counts
|
||||
`flush_failures`; entries stay dropped (next query re-tracks). The database handle is the
|
||||
loop's own dedicated config.db connection (ruling 21) — document on `run` like retention does.
|
||||
IP text via `address.NetAddress.format` into a stack buffer.
|
||||
`run` is the house loop shape (manager.zig:946): startup no-op, then every `flush_interval_s` on the `.boot` clock: skip the pass when `monitor` says writes are gated (ruling 17); else drain the pending table under the mutex into a local copy (release before I/O), one `upsertSeen` per client with `real` wall-clock seconds; every `prune_every_passes`-th pass also `pruneStale(now - retention_days * 86_400)`. A failed upsert warns once per pass and counts `flush_failures`; entries stay dropped (next query re-tracks). The database handle is the loop's own dedicated config.db connection (ruling 21) — document on `run` like retention does. IP text via `address.NetAddress.format` into a stack buffer.
|
||||
|
||||
Tests: track dedupes per flush (same client twice → one row, `last_seen` = latest), table
|
||||
full drops and counts, flush inserts hand_edited=0 rows and touches existing hand-edited rows'
|
||||
last_seen only, prune removes only stale hand_edited=0 rows, gated pass writes nothing
|
||||
(in-memory db + a Monitor fixture — Monitor's classify/state accessors allow constructing a
|
||||
gated state; if not constructible without I/O, test the gate branch through a bool seam
|
||||
`writes_allowed` parameter on a private `flushOnce` and have `run` pass the monitor read).
|
||||
Tests: track dedupes per flush (same client twice → one row, `last_seen` = latest), table full drops and counts, flush inserts hand_edited=0 rows and touches existing hand-edited rows' last_seen only, prune removes only stale hand_edited=0 rows, gated pass writes nothing (in-memory db + a Monitor fixture — Monitor's classify/state accessors allow constructing a gated state; if not constructible without I/O, test the gate branch through a bool seam `writes_allowed` parameter on a private `flushOnce` and have `run` pass the monitor read).
|
||||
|
||||
Acceptance: fmt/ast clean; temp-root (with `-lc -lsqlite3`) green; migration untouched.
|
||||
|
||||
### S4 As built
|
||||
|
||||
`init(retention_days: u16)` (matches `model.Logging.retention_days`). The pending table is a
|
||||
fixed 512-entry array with a linear scan, not a hash map — the frozen `init` takes no
|
||||
allocator, and rate_limiter's actual pattern is a pre-reserved hash map, so "same house
|
||||
pattern" did not apply; a linear scan over ≤512 entries of 17-byte keys is fine at household
|
||||
scale. Added: `trackAt(io, addr, now_s)` (timestamp seam; `track` = `trackAt` with real
|
||||
wall-clock), `flushOnce(io, database, writes_allowed: bool)` public (the gate seam `run` feeds
|
||||
from `monitor.writesAllowed()`; S7 case 10 forces passes with it), `snapshotStats(io)` and
|
||||
`pendingClients(io)` — `Stats` stays plain u64 and every field plus `passes` is written under
|
||||
`mutex`, so readers go through the accessors. `last_seen` is stamped at track time (it means
|
||||
"last query", and it is what makes latest-wins dedupe observable); the flush writes each
|
||||
entry's own stamp. A gated pass does not increment `passes` (a skipped pass must not advance
|
||||
the prune schedule). `track` uses `lockUncancelable` — `handle` has no error union, so `track`
|
||||
is not a cancelation point. `upsertSeen` resolves `group_id` via
|
||||
`(SELECT id FROM groups WHERE name = 'default')` — the column is NOT NULL REFERENCES groups
|
||||
(config_schema.zig:26), which ruling 14's sketch omitted; a database with no `default` group
|
||||
fails the whole statement with `error.Constraint` and writes nothing (validate.zig:461 already
|
||||
rejects such configs, so this is a warn-and-continue path in the flush loop, never live). The
|
||||
upsert's DO UPDATE touches `last_seen` only — `name`, `group_id`, `hand_edited`, `first_seen`
|
||||
survive on every existing row, hand-edited or not, one test per column. `pruneStale` uses
|
||||
strict `<` (matches `queries_repo.pruneOlderThan`) and clamps `sqlite3_changes` to u32.
|
||||
16 tests (9 tracker, 7 repo).
|
||||
`init(retention_days: u16)` (matches `model.Logging.retention_days`). The pending table is a fixed 512-entry array with a linear scan, not a hash map — the frozen `init` takes no allocator, and rate_limiter's actual pattern is a pre-reserved hash map, so "same house pattern" did not apply; a linear scan over ≤512 entries of 17-byte keys is fine at household scale. Added: `trackAt(io, addr, now_s)` (timestamp seam; `track` = `trackAt` with real wall-clock), `flushOnce(io, database, writes_allowed: bool)` public (the gate seam `run` feeds from `monitor.writesAllowed()`; S7 case 10 forces passes with it), `snapshotStats(io)` and `pendingClients(io)` — `Stats` stays plain u64 and every field plus `passes` is written under `mutex`, so readers go through the accessors. `last_seen` is stamped at track time (it means "last query", and it is what makes latest-wins dedupe observable); the flush writes each entry's own stamp. A gated pass does not increment `passes` (a skipped pass must not advance the prune schedule). `track` uses `lockUncancelable` — `handle` has no error union, so `track` is not a cancelation point. `upsertSeen` resolves `group_id` via `(SELECT id FROM groups WHERE name = 'default')` — the column is NOT NULL REFERENCES groups (config_schema.zig:26), which ruling 14's sketch omitted; a database with no `default` group fails the whole statement with `error.Constraint` and writes nothing (validate.zig:461 already rejects such configs, so this is a warn-and-continue path in the flush loop, never live). The upsert's DO UPDATE touches `last_seen` only — `name`, `group_id`, `hand_edited`, `first_seen` survive on every existing row, hand-edited or not, one test per column. `pruneStale` uses strict `<` (matches `queries_repo.pruneOlderThan`) and clamps `sqlite3_changes` to u32. 16 tests (9 tracker, 7 repo).
|
||||
|
||||
---
|
||||
|
||||
## Session S6: composition root — app.zig, shutdown.zig, cli wiring
|
||||
|
||||
Owns: `src/app.zig` (new), `src/server/shutdown.zig` (new), `src/cli.zig` (runRun body,
|
||||
DataDir), `src/main.zig` (only if a signature forces it — expected unchanged).
|
||||
Owns: `src/app.zig` (new), `src/server/shutdown.zig` (new), `src/cli.zig` (runRun body, DataDir), `src/main.zig` (only if a signature forces it — expected unchanged).
|
||||
|
||||
### S6.1 shutdown.zig
|
||||
|
||||
@@ -514,53 +250,25 @@ pub fn wait(io: std.Io) std.Io.Cancelable!void // event.wait
|
||||
pub fn trigger(io: std.Io) void // tests and future admin use: event.set
|
||||
```
|
||||
|
||||
Handler fn: `callconv(.c)`, calls set only (async-signal-safe per ruling 2). `flags = 0`, empty
|
||||
mask (matches Threaded's own handlers, Threaded.zig:1653). Do not restore handlers on exit —
|
||||
the process is leaving anyway; `install` is once-per-process (assert).
|
||||
Handler fn: `callconv(.c)`, calls set only (async-signal-safe per ruling 2). `flags = 0`, empty mask (matches Threaded's own handlers, Threaded.zig:1653). Do not restore handlers on exit — the process is leaving anyway; `install` is once-per-process (assert).
|
||||
|
||||
### S6.2 app.zig — `pub fn run(runner: cli.Runner, paths: cli.Paths) u8`
|
||||
|
||||
Startup order (each failure prints to `runner.err` and returns `cli.exit_runtime`):
|
||||
|
||||
1. `DataDir.open` (S6 changes its `openDir` flags to `.{ .iterate = true }` — the monitor
|
||||
needs it; nothing else cares). `openConfigDb`. `migrations.migrate`.
|
||||
`bootstrap.bootstrap(...)` (first run seeds; diags printed like runCheck does).
|
||||
`export.readConfig` into an arena.
|
||||
2. `logging.install(io, cfg.logging, ...)` — the sink goes live (main.zig's std_options
|
||||
already routes).
|
||||
3. Build state, in dependency order, all heap-allocated where pinning is required (Logger,
|
||||
Manager, Pool entries and their DoH/DoT client structs must not move — same pattern as
|
||||
`cli.probeUpstreams` but with owned arrays that outlive the pool):
|
||||
records + zones (`local_repo` listers → `build`), fetcher + manager (`Manager.init`),
|
||||
pool (upstreams_repo → clients array → `Pool.init`), cache (`DnsCache.init`), limiter
|
||||
(`RateLimiter.init` from `cfg.dns`), pause, tracker, logger (`Logger.init` with a
|
||||
heap `queue_buf` of `cfg.logging.query_log_buffer_max` entries), monitor (`Monitor.init`
|
||||
with the data dir), retention (`Retention.init`), querylog connections: add
|
||||
`DataDir.openQuerylogDb(self, io) !db.Db` mirroring openConfigDb (same pragmas + 0600);
|
||||
open TWO (writer, retention) plus ONE extra config.db connection for the tracker
|
||||
(ruling 21).
|
||||
1. `DataDir.open` (S6 changes its `openDir` flags to `.{ .iterate = true }` — the monitor needs it; nothing else cares). `openConfigDb`. `migrations.migrate`. `bootstrap.bootstrap(...)` (first run seeds; diags printed like runCheck does). `export.readConfig` into an arena.
|
||||
2. `logging.install(io, cfg.logging, ...)` — the sink goes live (main.zig's std_options already routes).
|
||||
3. Build state, in dependency order, all heap-allocated where pinning is required (Logger, Manager, Pool entries and their DoH/DoT client structs must not move — same pattern as `cli.probeUpstreams` but with owned arrays that outlive the pool): records + zones (`local_repo` listers → `build`), fetcher + manager (`Manager.init`), pool (upstreams_repo → clients array → `Pool.init`), cache (`DnsCache.init`), limiter (`RateLimiter.init` from `cfg.dns`), pause, tracker, logger (`Logger.init` with a heap `queue_buf` of `cfg.logging.query_log_buffer_max` entries), monitor (`Monitor.init` with the data dir), retention (`Retention.init`), querylog connections: add `DataDir.openQuerylogDb(self, io) !db.Db` mirroring openConfigDb (same pragmas + 0600); open TWO (writer, retention) plus ONE extra config.db connection for the tracker (ruling 21).
|
||||
4. One synchronous `manager.reload(io)`; failure warns and continues (ruling 4).
|
||||
5. Handler construction (S2 struct) + `UdpServer.bind` + `TcpServer.listen` on
|
||||
`cfg.dns.bind_ipv4/bind_ipv6/port` — v4 and v6 both, matching PLAN §7.2 parity; a bind
|
||||
failure is fatal.
|
||||
6. `shutdown.install(io)`. Then one `std.Io.Group`:
|
||||
udp.serve, tcp.serve, logger.runWriter(io, &querylog_writer_db, &monitor),
|
||||
retention.run(io, &querylog_retention_db), monitor.run(io),
|
||||
manager.runScheduler(io), tracker.run(io, &tracker_db, &monitor),
|
||||
and a maintenance loop (private fn in app.zig, house shape, `.boot`, 60 s): cache sweep +
|
||||
limiter sweep under their mutexes... the mutexes live in the Handler; the maintenance loop
|
||||
takes `*Handler` and locks the same mutexes.
|
||||
7. `shutdown.wait(io)` (a Canceled result also proceeds to teardown). Then ruling 22 order:
|
||||
`logger.shutdown(io)` → `group.cancel(io)` → server deinits → pool/manager/state deinits →
|
||||
DB closes → DataDir.close → return `cli.exit_ok`.
|
||||
5. Handler construction (S2 struct) + `UdpServer.bind` + `TcpServer.listen` on `cfg.dns.bind_ipv4/bind_ipv6/port` — v4 and v6 both, matching PLAN §7.2 parity; a bind failure is fatal.
|
||||
6. `shutdown.install(io)`. Then one `std.Io.Group`: udp.serve, tcp.serve, logger.runWriter(io, &querylog_writer_db, &monitor), retention.run(io, &querylog_retention_db), monitor.run(io), manager.runScheduler(io), tracker.run(io, &tracker_db, &monitor), and a maintenance loop (private fn in app.zig, house shape, `.boot`, 60 s): cache sweep + limiter sweep under their mutexes... the mutexes live in the Handler; the maintenance loop takes `*Handler` and locks the same mutexes.
|
||||
7. `shutdown.wait(io)` (a Canceled result also proceeds to teardown). Then ruling 22 order: `logger.shutdown(io)` → `group.cancel(io)` → server deinits → pool/manager/state deinits → DB closes → DataDir.close → return `cli.exit_ok`.
|
||||
|
||||
Log a one-line startup summary (scope `.nxdns`, info): bind addresses, snapshot generation or
|
||||
"unfiltered", upstream count.
|
||||
Log a one-line startup summary (scope `.nxdns`, info): bind addresses, snapshot generation or "unfiltered", upstream count.
|
||||
|
||||
### S6.3 cli.zig
|
||||
|
||||
`runRun` body: `return app.run(r, paths);` — keep the doc comment honest. `DataDir.open` flag
|
||||
change + `openQuerylogDb` as above (both used by app; runCheck untouched).
|
||||
`runRun` body: `return app.run(r, paths);` — keep the doc comment honest. `DataDir.open` flag change + `openQuerylogDb` as above (both used by app; runCheck untouched).
|
||||
|
||||
### S6.4 Acceptance
|
||||
|
||||
@@ -573,88 +281,33 @@ change + `openQuerylogDb` as above (both used by app; runCheck untouched).
|
||||
|
||||
### S6 As built
|
||||
|
||||
Startup order as specified, with these corrections. **Binding**: IPv6 binds first; an
|
||||
`AddressInUse` from the IPv4 bind that follows a wildcard IPv6 bind is dual-stack coverage
|
||||
(info line), not a failure — on Linux `net.ipv6.bindv6only=0` makes the v6 wildcard own the v4
|
||||
port, and `BindOptions.ip6_only` cannot prevent it because Threaded.zig:12274 sets IPV6_V6ONLY
|
||||
to 0 when the option is true (inverted vs its doc); a foreign IPv4 holder would have failed
|
||||
the v6 wildcard bind too, so the inference is sound. `NetAddress.fromIp` unmaps `::ffff:`
|
||||
clients, so v4 clients keep their real identity. A kernel without IPv6
|
||||
(`AddressFamilyUnsupported` and friends) warns and serves IPv4; every other bind failure is
|
||||
fatal. **Exit codes**: configuration faults (`NoUsableUpstreams`, `BadBindAddress`,
|
||||
`BadRateLimit` — zero rate fields are rejected before RateLimiter.init asserts) exit
|
||||
`cli.exit_check` with a "run 'nxdns check'" hint; everything else exits `cli.exit_runtime`.
|
||||
**Querylog open**: `openQuerylogDb` wraps `querylog_schema.open` (creation/repair lives there;
|
||||
chmod of the file plus both WAL sidecars happens after, missing sidecars ignored);
|
||||
`reopenQuerylogDb` opens the extra connections without repeating the O(size) quick_check.
|
||||
**Two `std.http.Client`s**: blocklist downloads must not queue DoH queries behind a
|
||||
multi-megabyte stream. Teardown is defer-driven; the invariants held are: group.cancel joins
|
||||
first, config.db closes after manager.deinit, DataDir closes last. `negative_ttl_max` and
|
||||
`forward_read_timeout` are wired (the S2 trap). `shutdown.zig` has `reset()` for tests and 2
|
||||
in-file tests (orchestrator wired it into tests.zig — app.zig analysis via cli does not
|
||||
collect tests). app.zig itself has no unit test; S7 case 11 is the runtime proof. Smoke test
|
||||
passed: boot, migration 0->2, dual-stack answers on 15353/15354, SERVFAIL from the dead
|
||||
upstream, SIGTERM exit 0 in 0.007 s, DBs 0600, zero err lines. The manager disk gate was wired in a follow-up S6 edit — see ruling 17's as-built text; the
|
||||
gate test drives `Monitor.state_raw` directly (no I/O) and pins no-monitor/ok/warn pass,
|
||||
critical gates-and-counts, recovery resumes. After review: the composition root takes one
|
||||
synchronous `monitor.sample(io)` after `Monitor.init`, before the `Io.Group` spawns anything —
|
||||
`Monitor` initializes to `.ok` and `run` samples inside its own task, so without it the
|
||||
writer/tracker/scheduler would consult a gate still saying "writes allowed" during the boot
|
||||
window on a critically full disk. A failed first sample warns and leaves `.ok` (an unreadable
|
||||
filesystem is not evidence of a full disk — the monitor's own documented policy). `run`'s own
|
||||
first sample repeats it (one extra statvfs + scan at boot; `run`'s sample-then-sleep contract
|
||||
stays untouched). The ordering is not unit-testable (serve binds sockets and opens four
|
||||
databases); S7's boot case plus the statement order is the evidence.
|
||||
Startup order as specified, with these corrections. **Binding**: IPv6 binds first; an `AddressInUse` from the IPv4 bind that follows a wildcard IPv6 bind is dual-stack coverage (info line), not a failure — on Linux `net.ipv6.bindv6only=0` makes the v6 wildcard own the v4 port, and `BindOptions.ip6_only` cannot prevent it because Threaded.zig:12274 sets IPV6_V6ONLY to 0 when the option is true (inverted vs its doc); a foreign IPv4 holder would have failed the v6 wildcard bind too, so the inference is sound. `NetAddress.fromIp` unmaps `::ffff:` clients, so v4 clients keep their real identity. A kernel without IPv6 (`AddressFamilyUnsupported` and friends) warns and serves IPv4; every other bind failure is fatal. **Exit codes**: configuration faults (`NoUsableUpstreams`, `BadBindAddress`, `BadRateLimit` — zero rate fields are rejected before RateLimiter.init asserts) exit `cli.exit_check` with a "run 'nxdns check'" hint; everything else exits `cli.exit_runtime`. **Querylog open**: `openQuerylogDb` wraps `querylog_schema.open` (creation/repair lives there; chmod of the file plus both WAL sidecars happens after, missing sidecars ignored); `reopenQuerylogDb` opens the extra connections without repeating the O(size) quick_check. **Two `std.http.Client`s**: blocklist downloads must not queue DoH queries behind a multi-megabyte stream. Teardown is defer-driven; the invariants held are: group.cancel joins first, config.db closes after manager.deinit, DataDir closes last. `negative_ttl_max` and `forward_read_timeout` are wired (the S2 trap). `shutdown.zig` has `reset()` for tests and 2 in-file tests (orchestrator wired it into tests.zig — app.zig analysis via cli does not collect tests). app.zig itself has no unit test; S7 case 11 is the runtime proof. Smoke test passed: boot, migration 0->2, dual-stack answers on 15353/15354, SERVFAIL from the dead upstream, SIGTERM exit 0 in 0.007 s, DBs 0600, zero err lines. The manager disk gate was wired in a follow-up S6 edit — see ruling 17's as-built text; the gate test drives `Monitor.state_raw` directly (no I/O) and pins no-monitor/ok/warn pass, critical gates-and-counts, recovery resumes. After review: the composition root takes one synchronous `monitor.sample(io)` after `Monitor.init`, before the `Io.Group` spawns anything — `Monitor` initializes to `.ok` and `run` samples inside its own task, so without it the writer/tracker/scheduler would consult a gate still saying "writes allowed" during the boot window on a critically full disk. A failed first sample warns and leaves `.ok` (an unreadable filesystem is not evidence of a full disk — the monitor's own documented policy). `run`'s own first sample repeats it (one extra statvfs + scan at boot; `run`'s sample-then-sleep contract stays untouched). The ordering is not unit-testable (serve binds sockets and opens four databases); S7's boot case plus the statement order is the evidence.
|
||||
|
||||
---
|
||||
|
||||
## Session S7: phase 7 integration tests
|
||||
|
||||
Owns: `src/server/phase7_integration_test.zig` (new; gated on `build_options.integration` like
|
||||
phase6). Fixtures may reuse the S8/M6 helper patterns (in-memory dbs, fake upstream sockets).
|
||||
Owns: `src/server/phase7_integration_test.zig` (new; gated on `build_options.integration` like phase6). Fixtures may reuse the S8/M6 helper patterns (in-memory dbs, fake upstream sockets).
|
||||
|
||||
Cases (each end-to-end through a real `Handler` with real UDP sockets on 127.0.0.1 ephemeral
|
||||
ports; a fake upstream = a UDP socket task answering canned responses):
|
||||
Cases (each end-to-end through a real `Handler` with real UDP sockets on 127.0.0.1 ephemeral ports; a fake upstream = a UDP socket task answering canned responses):
|
||||
|
||||
1. Blocked domain answers 0.0.0.0 with blocking TTL; log entry captured with reason.
|
||||
2. Allow-rule-over-blocklist resolves via upstream.
|
||||
3. Local record answers authoritatively without touching the fake upstream.
|
||||
4. Forward zone: matching suffix goes to the zone resolver socket (not the pool fake), a
|
||||
blocklisted name inside the zone still resolves (bypass), response cached (second query
|
||||
does not hit the zone socket).
|
||||
4. Forward zone: matching suffix goes to the zone resolver socket (not the pool fake), a blocklisted name inside the zone still resolves (bypass), response cached (second query does not hit the zone socket).
|
||||
5. Cache: miss→hit, TTL aged, transaction ID fresh, `cache_hit` logged on the hit.
|
||||
6. Uncloak: fake upstream returns CNAME chain to a blocked target → zero-IP answer with
|
||||
`cname:` reason.
|
||||
7. Safe-search group: query for a safesearch.zig table domain returns CNAME + the fake
|
||||
upstream's A for the target; question is the original name.
|
||||
6. Uncloak: fake upstream returns CNAME chain to a blocked target → zero-IP answer with `cname:` reason.
|
||||
7. Safe-search group: query for a safesearch.zig table domain returns CNAME + the fake upstream's A for the target; question is the original name.
|
||||
8. Rate limit: limit=2 window=60 → third query REFUSED.
|
||||
9. Pause: pauseFor(indefinite) lets a blocked domain resolve; unpause blocks it again.
|
||||
10. Tracker: after queries + one forced flush (`flushOnce` seam or short interval), the
|
||||
clients table holds the source IP with hand_edited=0.
|
||||
11. Full app boot: `app.run` in a task against a temp data dir (bootstrap from a minimal
|
||||
config.zon on an ephemeral port), one real resolve through it, `shutdown.trigger`, task
|
||||
returns 0. This is the lifecycle proof.
|
||||
10. Tracker: after queries + one forced flush (`flushOnce` seam or short interval), the clients table holds the source IP with hand_edited=0.
|
||||
11. Full app boot: `app.run` in a task against a temp data dir (bootstrap from a minimal config.zon on an ephemeral port), one real resolve through it, `shutdown.trigger`, task returns 0. This is the lifecycle proof.
|
||||
|
||||
Acceptance: `zig build test -Dintegration` green, zero std.log.err emitted (binding logging
|
||||
policy), cases deterministic (no external network — the pool's upstream is the fake).
|
||||
Acceptance: `zig build test -Dintegration` green, zero std.log.err emitted (binding logging policy), cases deterministic (no external network — the pool's upstream is the fake).
|
||||
|
||||
### S7 As built
|
||||
|
||||
Case 11 runs against a `std.testing.tmpDir` rather than a fixed scratch path (a committed
|
||||
test cannot hold a machine-specific absolute path; the phase6 fixture already resolves
|
||||
cwd-relative paths through both `std.Io.Dir` and SQLite). Port 15455, IPv4+IPv6 binds, the
|
||||
dead `https://127.0.0.1:9/dns-query` upstream, one seeded local record; `app.serve` installs
|
||||
and deinstalls its own log sink, so `installForTest` is unnecessary and the case asserts the
|
||||
error writer stayed empty; `shutdown.reset()` brackets the case. Case 5 proves TTL ageing
|
||||
with an entry planted ten seconds in the past under the handler's own key — the
|
||||
miss-hit-fresh-ID half is a real round trip; no test can advance the clock. Case 4's zone
|
||||
resolver is a UDP socket task answering with `ResponseBuilder` against the received query;
|
||||
the second query proves the cache by leaving the resolver's datagram count at 1. The fake
|
||||
pool upstream builds each answer from the question it receives (safe search and ECS rewrite
|
||||
the outgoing question, so canned bytes would not bind). A shared `Loop` fixture binds the
|
||||
listener and client before the serve task starts, so no task holds a pointer into a value
|
||||
that later moves. 11 tests, all gated on `build_options.integration`; the integration suite
|
||||
emits zero err-level lines.
|
||||
Case 11 runs against a `std.testing.tmpDir` rather than a fixed scratch path (a committed test cannot hold a machine-specific absolute path; the phase6 fixture already resolves cwd-relative paths through both `std.Io.Dir` and SQLite). Port 15455, IPv4+IPv6 binds, the dead `https://127.0.0.1:9/dns-query` upstream, one seeded local record; `app.serve` installs and deinstalls its own log sink, so `installForTest` is unnecessary and the case asserts the error writer stayed empty; `shutdown.reset()` brackets the case. Case 5 proves TTL ageing with an entry planted ten seconds in the past under the handler's own key — the miss-hit-fresh-ID half is a real round trip; no test can advance the clock. Case 4's zone resolver is a UDP socket task answering with `ResponseBuilder` against the received query; the second query proves the cache by leaving the resolver's datagram count at 1. The fake pool upstream builds each answer from the question it receives (safe search and ECS rewrite the outgoing question, so canned bytes would not bind). A shared `Loop` fixture binds the listener and client before the serve task starts, so no task holds a pointer into a value that later moves. 11 tests, all gated on `build_options.integration`; the integration suite emits zero err-level lines.
|
||||
|
||||
---
|
||||
|
||||
@@ -675,11 +328,7 @@ emits zero err-level lines.
|
||||
|
||||
## File ownership
|
||||
|
||||
S1: dns/edns.zig. S2: server/handler.zig, server/pause.zig. S3: server/udp_server.zig,
|
||||
server/tcp_server.zig. S4: server/clients.zig, storage/repositories/clients_repo.zig.
|
||||
S6: app.zig, server/shutdown.zig, cli.zig, main.zig(if forced). S7: its test file.
|
||||
Orchestrator: src/tests.zig, specs. No parallel sessions share a file (S1∥S4; the rest are
|
||||
sequential).
|
||||
S1: dns/edns.zig. S2: server/handler.zig, server/pause.zig. S3: server/udp_server.zig, server/tcp_server.zig. S4: server/clients.zig, storage/repositories/clients_repo.zig. S6: app.zig, server/shutdown.zig, cli.zig, main.zig(if forced). S7: its test file. Orchestrator: src/tests.zig, specs. No parallel sessions share a file (S1∥S4; the rest are sequential).
|
||||
|
||||
## Acceptance (milestone complete)
|
||||
|
||||
|
||||
+119
-682
@@ -1,59 +1,20 @@
|
||||
# Milestone 8: web server, REST API, SSE, auth, metrics (PLAN Phase 8, Zig side)
|
||||
|
||||
Goal: the complete Zig web layer — HTTP server + router, every REST handler, SSE live query
|
||||
stream, optional argon2id auth with sessions, API token-bucket rate limiting, `/metrics`,
|
||||
`/api/health`, OpenAPI served + contract tests, asset embedding + dev-mode disk serving —
|
||||
running as one more task in app.zig's group.
|
||||
Goal: the complete Zig web layer — HTTP server + router, every REST handler, SSE live query stream, optional argon2id auth with sessions, API token-bucket rate limiting, `/metrics`, `/api/health`, OpenAPI served + contract tests, asset embedding + dev-mode disk serving — running as one more task in app.zig's group.
|
||||
|
||||
Ground truth: PLAN.md:610-612 (phase text), :537-550 (endpoints), §3.11/§12.1/§19 (auth),
|
||||
§10:333 (API limiting), §11.4:455 (SSE precedes persistence), §13.2 (OpenAPI), scratchpad
|
||||
notes m8-explore-{plan,repo,zig}.md. Load-bearing Zig facts are restated inline; verify
|
||||
anything else against /home/mokhtar/app/zig tag 0.16.0.
|
||||
Ground truth: PLAN.md:610-612 (phase text), :537-550 (endpoints), §3.11/§12.1/§19 (auth), §10:333 (API limiting), §11.4:455 (SSE precedes persistence), §13.2 (OpenAPI), scratchpad notes m8-explore-{plan,repo,zig}.md. Load-bearing Zig facts are restated inline; verify anything else against /home/mokhtar/app/zig tag 0.16.0.
|
||||
|
||||
## Rulings (binding)
|
||||
|
||||
1. **Sequencing**: this milestone is the Zig web layer end to end. The React SPA (PLAN §3.14,
|
||||
§14 — ten pages) is milestone 9, built against this milestone's finished, contract-tested
|
||||
API. Not a scope cut: the asset embedding, dev-mode disk serving, ETag and gzip machinery
|
||||
all ship NOW and are complete; milestone 9 only swaps the dist content. The embedded dist
|
||||
in this milestone is a minimal real page (see ruling 24) — machinery is not stubbed.
|
||||
2. **`POST /api/certs/reload` is Phase 9's**, whole: endpoint, openapi.yaml entry and docs
|
||||
land together with the DoH/DoT server it reloads. No 501 stub, no dangling contract entry.
|
||||
1. **Sequencing**: this milestone is the Zig web layer end to end. The React SPA (PLAN §3.14, §14 — ten pages) is milestone 9, built against this milestone's finished, contract-tested API. Not a scope cut: the asset embedding, dev-mode disk serving, ETag and gzip machinery all ship NOW and are complete; milestone 9 only swaps the dist content. The embedded dist in this milestone is a minimal real page (see ruling 24) — machinery is not stubbed.
|
||||
2. **`POST /api/certs/reload` is Phase 9's**, whole: endpoint, openapi.yaml entry and docs land together with the DoH/DoT server it reloads. No 501 stub, no dangling contract entry.
|
||||
3. **`docs/api/` rendering is Phase 10** (the docs phase). Phase 8 serves the yaml.
|
||||
4. **Server model**: per-connection `std.http.Server` (Server.zig:25) over our own accept
|
||||
loop, copied from lib/std/Build/WebServer.zig:152-185: listener task in the app group; an
|
||||
inner `Io.Group` of connection tasks; keep-alive loop on `receiveHead` with
|
||||
`error.HttpConnectionClosing => return`. Cancel semantics mirror tcp_server's S3 As-built:
|
||||
on cancellation the inner group is CANCELED, not awaited (a keep-alive client must not
|
||||
hold shutdown open); on listener close (`SocketNotListening`) it drains.
|
||||
5. **Bind**: one listener socket on `web.bind:web.port` exactly as configured (default
|
||||
0.0.0.0:8080). No dual-stack ceremony — milestone 7's parity ruling was about DNS client
|
||||
identity, not the admin UI. An operator who wants v6 sets `web.bind = "::"` (dual-stack by
|
||||
Linux default).
|
||||
6. **`web.enabled = false`** skips the entire subsystem, `/metrics` included. No web task, no
|
||||
web DB connections.
|
||||
7. **Limits**: recv buffer 8 KiB (this caps the request head — Server.zig:32); send buffer
|
||||
4 KiB; request body cap 1 MiB (`Io.Limit`, read via `readerExpectNone`/`allocRemaining`);
|
||||
64 concurrent connections (accept beyond that: respond 503 and close — never silently
|
||||
drop). No per-request timeout this milestone (LAN-facing; the cancel path bounds
|
||||
shutdown); documented in server.zig.
|
||||
8. **JSON shape**: snake_case field names everywhere (matches settings keys and SQL). Error
|
||||
envelope `{"error":"<message>"}`. Status codes: 400 validation, 401 unauthenticated,
|
||||
404 missing, 405 wrong method (with Allow), 409 conflict (duplicate key), 413 body too
|
||||
large, 429 rate limited (with Retry-After), 500 internal (generic message, detail to log
|
||||
as warn), 503 over connection cap.
|
||||
9. **Resource paths** (freezing PLAN:541's ellipses): collections `GET`+`POST` and items
|
||||
`GET`+`PUT`+`DELETE` by numeric row id for: `/api/groups[/{id}]`,
|
||||
`/api/blocklists[/{id}]` (the sources table), `/api/rules[/{id}]`,
|
||||
`/api/local-records[/{id}]`, `/api/forward-zones[/{id}]`, `/api/upstreams[/{id}]`
|
||||
(PLAN's list omitted upstreams; the Settings page must edit them; new resource, same
|
||||
pattern). Group-source assignment: `PUT /api/groups/{id}/sources` with `{"source_ids":
|
||||
[..]}` — idempotent full-set replace. Clients: `GET /api/clients` (ALL rows, materialized
|
||||
included, each with `hand_edited`), `GET/PUT/DELETE /api/clients/{id}`; PUT sets
|
||||
`hand_edited=1` and may change name/group; DELETE removes the row (a live client
|
||||
re-materializes). Client prefixes: `GET/PUT /api/client-prefixes` as a whole-list
|
||||
resource (tiny table, atomic replace). No POST for clients — creation is by DNS activity
|
||||
or import (PLAN:540 gives clients no POST deliberately).
|
||||
4. **Server model**: per-connection `std.http.Server` (Server.zig:25) over our own accept loop, copied from lib/std/Build/WebServer.zig:152-185: listener task in the app group; an inner `Io.Group` of connection tasks; keep-alive loop on `receiveHead` with `error.HttpConnectionClosing => return`. Cancel semantics mirror tcp_server's S3 As-built: on cancellation the inner group is CANCELED, not awaited (a keep-alive client must not hold shutdown open); on listener close (`SocketNotListening`) it drains.
|
||||
5. **Bind**: one listener socket on `web.bind:web.port` exactly as configured (default 0.0.0.0:8080). No dual-stack ceremony — milestone 7's parity ruling was about DNS client identity, not the admin UI. An operator who wants v6 sets `web.bind = "::"` (dual-stack by Linux default).
|
||||
6. **`web.enabled = false`** skips the entire subsystem, `/metrics` included. No web task, no web DB connections.
|
||||
7. **Limits**: recv buffer 8 KiB (this caps the request head — Server.zig:32); send buffer 4 KiB; request body cap 1 MiB (`Io.Limit`, read via `readerExpectNone`/`allocRemaining`); 64 concurrent connections (accept beyond that: respond 503 and close — never silently drop). No per-request timeout this milestone (LAN-facing; the cancel path bounds shutdown); documented in server.zig.
|
||||
8. **JSON shape**: snake_case field names everywhere (matches settings keys and SQL). Error envelope `{"error":"<message>"}`. Status codes: 400 validation, 401 unauthenticated, 404 missing, 405 wrong method (with Allow), 409 conflict (duplicate key), 413 body too large, 429 rate limited (with Retry-After), 500 internal (generic message, detail to log as warn), 503 over connection cap.
|
||||
9. **Resource paths** (freezing PLAN:541's ellipses): collections `GET`+`POST` and items `GET`+`PUT`+`DELETE` by numeric row id for: `/api/groups[/{id}]`, `/api/blocklists[/{id}]` (the sources table), `/api/rules[/{id}]`, `/api/local-records[/{id}]`, `/api/forward-zones[/{id}]`, `/api/upstreams[/{id}]` (PLAN's list omitted upstreams; the Settings page must edit them; new resource, same pattern). Group-source assignment: `PUT /api/groups/{id}/sources` with `{"source_ids": [..]}` — idempotent full-set replace. Clients: `GET /api/clients` (ALL rows, materialized included, each with `hand_edited`), `GET/PUT/DELETE /api/clients/{id}`; PUT sets `hand_edited=1` and may change name/group; DELETE removes the row (a live client re-materializes). Client prefixes: `GET/PUT /api/client-prefixes` as a whole-list resource (tiny table, atomic replace). No POST for clients — creation is by DNS activity or import (PLAN:540 gives clients no POST deliberately).
|
||||
|
||||
> **Placement deviation (milestone 17 ruling 3).** "The Settings page must
|
||||
> edit them" above names the wrong page. Milestone 9 dropped the obligation
|
||||
@@ -62,151 +23,31 @@ anything else against /home/mokhtar/app/zig tag 0.16.0.
|
||||
> matching the house resource-page pattern that every other collection
|
||||
> follows. The Settings page keeps the scalar settings keys only. The API
|
||||
> contract in this ruling is unchanged.
|
||||
10. **Repo layer**: every list row the API serves carries its row id; each mutated resource
|
||||
gains `getX(db, id)`, `updateX(db, id, item)`, `deleteX(db, id)` (strict: 0 rows touched
|
||||
→ error.NotFound), written in the house repo idiom with prepared statements. Existing
|
||||
import-path functions stay frozen.
|
||||
11. **`GET /api/queries`**: keyset pagination `?limit` (default 100, max 1000) +
|
||||
`?before=<row id>`, ordered id DESC; filters `domain=` (substring), `client=` (exact),
|
||||
`blocked=` (bool), `since=`/`until=` (unix seconds). Response `{"queries":[...],
|
||||
"next_before": <id>|null}`. Row fields: id, ts, domain, client_ip, qtype, blocked,
|
||||
block_reason, response_time_us, cache_hit, upstream.
|
||||
12. **Live vs restart**: mutations to rules, blocklists, groups, group-sources, clients and
|
||||
client-prefixes call `Manager.reload(io)` and take effect live (the handler's next
|
||||
`acquire` sees the new snapshot). Local records and forward zones ALSO apply live: a new
|
||||
`src/server/local_tables.zig` RCU holder (acquire/release + swap, mirroring the manager's
|
||||
pattern at small scale) owns `{records, zones}`; `Handler` reads through it instead of
|
||||
`*const` fields; the local-records/forward-zones handlers rebuild and swap. Upstream
|
||||
mutations and everything in `/api/settings` are restart-required. `POST
|
||||
/api/blocklists/update` = `Manager.refreshAll` then reload, 202 with the status snapshot.
|
||||
13. **Stats grammar**: `period=1h|24h|7d|30d` (else 400). `/api/stats` returns totals
|
||||
{queries, blocked, cached, clients (distinct), avg_response_time_us} for the period.
|
||||
`/api/stats/timeseries` buckets: 1h→60×1m, 24h→48×30m, 7d→168×1h, 30d→120×6h; UTC; each
|
||||
bucket {ts, queries, blocked, cached}. SQL aggregates over query_log on the web task's
|
||||
own read connection.
|
||||
14. **`GET /api/lookup?domain=&group_id=`** (group optional, default group when absent):
|
||||
normalizes, then reports the full pipeline view: `{domain, group_id, local_records:
|
||||
bool, forward_zone: <zone>|null, blocked, reason, matched, source_url|null,
|
||||
safe_search_rewrite: <target>|null}` via `snapshot.evaluate` + records/zones lookups.
|
||||
Null snapshot → 503 `{"error":"no snapshot loaded"}`.
|
||||
15. **Pause API**: `GET /api/pause` → `{"paused": bool, "until": <unix s>|null}` (null while
|
||||
unpaused OR indefinite — disambiguated by `paused`). `POST /api/pause` body
|
||||
`{"paused": true, "duration_seconds": <u32, optional>}` or `{"paused": false}`.
|
||||
16. **`GET/PUT /api/settings`**: the typed config sections that live in settings rows (dns,
|
||||
blocking, cache, edns, upstream, logging, disk, blocklist_update, safe_search flag home
|
||||
— read model.toSettings for the exact key list), minus `web.password*` (never
|
||||
serialized; a PUT carrying `web.password` re-hashes via the import path's argon2id
|
||||
params and stores only the hash). GET marks every key `restart_required: true` except
|
||||
none — ALL settings are restart-required this milestone (live behavior comes from the
|
||||
resource endpoints and pause; a static table in the handler carries the flag so the UI
|
||||
banner has transport). PUT validates the merged config through `validate.validate`
|
||||
before writing any row; partial updates allowed.
|
||||
17. **Auth**: enabled iff `web.password_hash != ""`. Login verifies with
|
||||
`std.crypto.pwhash.argon2.strVerify(hash, password, .{ .allocator = gpa }, io)`
|
||||
(argon2.zig:619 — PHC string carries its params). Sessions: in-memory fixed table of 32
|
||||
(LRU evict), storing SHA-256 of a 32-byte `io.randomSecure` token; lookup compares
|
||||
digests with `std.crypto.timing_safe.eql([32]u8, ...)` (slices not accepted —
|
||||
timing_safe.zig:12). Cookie `nxdns_session=<url_safe_no_pad base64>`; `HttpOnly;
|
||||
SameSite=Lax; Path=/`; `Secure` NOT set (nxdns serves plain HTTP; TLS termination is the
|
||||
operator's proxy — documented). Expiry `web.session_ttl_hours`. Logout deletes the
|
||||
session. A `PUT /api/settings` that changes `password_hash` clears every session.
|
||||
18. **Auth exemptions**: `/api/health`, `/api/version`, `/metrics`, `/api/openapi.yaml`, and
|
||||
the static assets are always unauthenticated (monitoring endpoints; the SPA shell must
|
||||
load to show a login form). Everything else 401s without a valid session when auth is
|
||||
enabled. `/api/auth/login` is necessarily exempt; failed logins count and are rate
|
||||
limited like any request.
|
||||
19. **API rate limiting**: token bucket per client IP (NOT the DNS fixed-window limiter —
|
||||
m6 ruling 8 reserved the bucket shape for the API): capacity and refill
|
||||
`web.api_rate_limit_per_min` per 60 s, 4096 tracked IPs (same fixed-table idiom as
|
||||
rate_limiter/tracker). Localhost exemption: NEW config field `web.api_localhost_exempt:
|
||||
bool = true` (PLAN §10 says "configurable" and §12.1 forgot the field; model + validate
|
||||
+ settings key + export round trip — settings are kv rows, no migration needed). 429 +
|
||||
`Retry-After: <s>`. SSE: connect consumes a token AND respects
|
||||
`web.sse_max_connections_per_ip`; `/metrics` and `/api/health` are limiter-exempt
|
||||
(Prometheus must never see 429).
|
||||
20. **SSE**: `GET /api/queries/live`, `text/event-stream` via `respondStreaming(&.{}, ...)`
|
||||
(EMPTY buffer — `BodyWriter.flush` does not flush the body writer's own buffer,
|
||||
http.zig:780; empty buffer makes every write go straight to the chunked drain), flush
|
||||
headers before the first event (test.zig:498 pattern). Fanout: milestone-6 ruling 4
|
||||
concretized as `src/server/query_sink.zig`: `QuerySink { logger: *Logger, hub: ?*sse.Hub
|
||||
}` with `log(io, entry)` = transform once, publish to the hub, enqueue to the logger.
|
||||
Logger gains additive `transformed(entry) Entry` (pure) and `logTransformed(io, entry)`;
|
||||
existing `log` = both, frozen behavior. `Handler.logger: ?*Logger` becomes `sink:
|
||||
?*QuerySink` (mechanical rename at the two call sites + app wiring). Hub: fixed 32
|
||||
subscriber slots, each a 64-entry ring + `std.Io.Event`; publish never blocks (full ring
|
||||
→ disconnect that subscriber; SSE clients auto-reconnect). Wire format: `retry: 3000`
|
||||
once, then `event: query` + `data: <JSON, same fields as /api/queries rows>` per entry;
|
||||
heartbeat comment `: ping` every 15 s from the subscriber task (Event.waitTimeout).
|
||||
Fanout precedes persistence (PLAN:455) — the sink publishes before enqueue.
|
||||
21. **`/metrics`**: Prometheus text format 0.0.4, `nxdns_` prefix, `# HELP`/`# TYPE` lines.
|
||||
Counters: every `Handler.Stats` field (17, individually, `nxdns_dns_<field>_total`),
|
||||
logger (queries_dropped, rows_written, batches_gated), cache stats (6, under
|
||||
`Handler.cache_mutex`), DNS limiter stats (under `limiter_mutex`), tracker stats,
|
||||
retention stats, log-sink stats, `nxdns_blocklist_refreshes_gated_total`. Gauges:
|
||||
cache len + memory bytes, disk free/db/log bytes, tracked DNS clients, pending tracker
|
||||
clients, blocklist generation, `nxdns_up 1`. Per-upstream, labeled `{url="..."}`:
|
||||
up (available), success_rate, consecutive_failures, total_successes/_failures (copy
|
||||
`Pool.Snapshot` fields while holding the returned count — `last_error` is borrowed,
|
||||
copy immediately). No timestamps. Auth-exempt, limiter-exempt.
|
||||
22. **`GET /api/health`**: `{"status":"ok"|"degraded","disk":{"state":...,
|
||||
"free_bytes":...,"db_bytes":...,"log_bytes":...,"sample_failures":...},
|
||||
"upstreams":{"available":N,"total":M},"queries_dropped":N,"writer_failed":bool,
|
||||
"refreshes_gated":N,"snapshot_generation":N|null}`. Degraded iff disk state != ok, or
|
||||
zero upstreams available, or writer_failed. 200 either way (degraded is data, not an
|
||||
HTTP failure).
|
||||
23. **OpenAPI + contract tests**: hand-written `src/web/openapi.yaml`, `@embedFile`d and
|
||||
served verbatim. No external validator (dependencies are liabilities): the contract IS a
|
||||
Zig table in the integration test — every route with method, auth requirement, a seeded
|
||||
request, expected status, and a response-shape struct that `std.json.parseFromSlice`
|
||||
must accept with `.ignore_unknown_fields = false`. Two drift guards: (a) the router
|
||||
exposes `pub const routes: []const RouteInfo`; a test asserts every entry's path+method
|
||||
appears textually in openapi.yaml; (b) the table covers every `routes` entry (count
|
||||
equality). CI gets one new job step running the existing `-Dintegration` suite (contract
|
||||
tests live inside it) — no new tooling.
|
||||
24. **Static assets**: build.zig gains `-Dweb-dist=<path>` (default `web/dist-placeholder/`,
|
||||
committed, containing a real minimal `index.html` — current status via /api/health
|
||||
fetch, links to /metrics; plus favicon). A build step copies the dist dir via
|
||||
`b.addWriteFiles()` + generated `assets.zig` index (the fixtures anonymous-import
|
||||
pattern, per research notes — @embedFile paths must live inside the module root),
|
||||
exposing `pub const files: []const File{path, bytes, content_type, etag}`; etag =
|
||||
comptime hash. Serving: exact path match, `/` → index.html, SPA fallback (unknown
|
||||
non-/api path → index.html, 200 — TanStack Router needs it in M9), `ETag`/
|
||||
`If-None-Match` → 304, `content-encoding: gzip` when a sibling `<name>.gz` exists in
|
||||
the dist AND the request accepts it. No Date/Last-Modified headers (std has no RFC 1123
|
||||
formatter; ETag is strictly better for immutable embedded content). Dev mode: CLI flag
|
||||
`nxdns run --web-dev <dir>` serves from disk (no cache headers), bypassing the embed.
|
||||
25. **Head-string trap**: `request.head.target` is invalidated by body reads
|
||||
(Server.zig:594/230). The router copies target into a stack buffer before any body
|
||||
read. Query strings: split on '?', `std.mem.splitScalar` for pairs,
|
||||
`Uri.percentDecodeInPlace` per value (std has no query iterator; std.Uri does not parse
|
||||
origin-form targets).
|
||||
26. **Wiring**: one `web_server.serve` task in the app group, started last, canceled by the
|
||||
same group.cancel. `WebState` struct (in web/server.zig) of borrowed pointers assembled
|
||||
by app.serve: handler (stats + mutexes + cache/limiter), pause, tracker, manager, pool,
|
||||
monitor, logger, retention stats, local_tables, sink/hub, config arena view (web cfg),
|
||||
its OWN `config.db` + `querylog.db` connections (m7 ruling 21), version string, started
|
||||
timestamp. Web DB connections open only when `web.enabled`.
|
||||
27. **No new schema migration**: query_log already carries what /api/queries needs; the
|
||||
read session verifies existing indexes and reports if a needed index is missing (then
|
||||
the orchestrator rules on adding migration v3 — do not add one silently).
|
||||
28. **Client disconnect**: stream writes surface `error.WriteFailed` with the real cause in
|
||||
`stream_writer.err` (ConnectionResetByPeer / SocketUnconnected=EPIPE — MSG.NOSIGNAL is
|
||||
set, Threaded.zig:13071, no SIGPIPE masking needed). A failed write ends the connection
|
||||
task quietly (debug log at most): clients vanishing is normal.
|
||||
29. **Input sanitation** (PLAN §19): every path/query/body input is length-capped and
|
||||
validated before SQL (prepared statements everywhere, already house rule); domain
|
||||
inputs run through matcher.normalize; no secrets ever logged (password, tokens, cookie
|
||||
values — assert by review, and the login handler logs only the client IP and outcome).
|
||||
10. **Repo layer**: every list row the API serves carries its row id; each mutated resource gains `getX(db, id)`, `updateX(db, id, item)`, `deleteX(db, id)` (strict: 0 rows touched → error.NotFound), written in the house repo idiom with prepared statements. Existing import-path functions stay frozen.
|
||||
11. **`GET /api/queries`**: keyset pagination `?limit` (default 100, max 1000) + `?before=<row id>`, ordered id DESC; filters `domain=` (substring), `client=` (exact), `blocked=` (bool), `since=`/`until=` (unix seconds). Response `{"queries":[...], "next_before": <id>|null}`. Row fields: id, ts, domain, client_ip, qtype, blocked, block_reason, response_time_us, cache_hit, upstream.
|
||||
12. **Live vs restart**: mutations to rules, blocklists, groups, group-sources, clients and client-prefixes call `Manager.reload(io)` and take effect live (the handler's next `acquire` sees the new snapshot). Local records and forward zones ALSO apply live: a new `src/server/local_tables.zig` RCU holder (acquire/release + swap, mirroring the manager's pattern at small scale) owns `{records, zones}`; `Handler` reads through it instead of `*const` fields; the local-records/forward-zones handlers rebuild and swap. Upstream mutations and everything in `/api/settings` are restart-required. `POST /api/blocklists/update` = `Manager.refreshAll` then reload, 202 with the status snapshot.
|
||||
13. **Stats grammar**: `period=1h|24h|7d|30d` (else 400). `/api/stats` returns totals {queries, blocked, cached, clients (distinct), avg_response_time_us} for the period. `/api/stats/timeseries` buckets: 1h→60×1m, 24h→48×30m, 7d→168×1h, 30d→120×6h; UTC; each bucket {ts, queries, blocked, cached}. SQL aggregates over query_log on the web task's own read connection.
|
||||
14. **`GET /api/lookup?domain=&group_id=`** (group optional, default group when absent): normalizes, then reports the full pipeline view: `{domain, group_id, local_records: bool, forward_zone: <zone>|null, blocked, reason, matched, source_url|null, safe_search_rewrite: <target>|null}` via `snapshot.evaluate` + records/zones lookups. Null snapshot → 503 `{"error":"no snapshot loaded"}`.
|
||||
15. **Pause API**: `GET /api/pause` → `{"paused": bool, "until": <unix s>|null}` (null while unpaused OR indefinite — disambiguated by `paused`). `POST /api/pause` body `{"paused": true, "duration_seconds": <u32, optional>}` or `{"paused": false}`.
|
||||
16. **`GET/PUT /api/settings`**: the typed config sections that live in settings rows (dns, blocking, cache, edns, upstream, logging, disk, blocklist_update, safe_search flag home — read model.toSettings for the exact key list), minus `web.password*` (never serialized; a PUT carrying `web.password` re-hashes via the import path's argon2id params and stores only the hash). GET marks every key `restart_required: true` except none — ALL settings are restart-required this milestone (live behavior comes from the resource endpoints and pause; a static table in the handler carries the flag so the UI banner has transport). PUT validates the merged config through `validate.validate` before writing any row; partial updates allowed.
|
||||
17. **Auth**: enabled iff `web.password_hash != ""`. Login verifies with `std.crypto.pwhash.argon2.strVerify(hash, password, .{ .allocator = gpa }, io)` (argon2.zig:619 — PHC string carries its params). Sessions: in-memory fixed table of 32 (LRU evict), storing SHA-256 of a 32-byte `io.randomSecure` token; lookup compares digests with `std.crypto.timing_safe.eql([32]u8, ...)` (slices not accepted — timing_safe.zig:12). Cookie `nxdns_session=<url_safe_no_pad base64>`; `HttpOnly; SameSite=Lax; Path=/`; `Secure` NOT set (nxdns serves plain HTTP; TLS termination is the operator's proxy — documented). Expiry `web.session_ttl_hours`. Logout deletes the session. A `PUT /api/settings` that changes `password_hash` clears every session.
|
||||
18. **Auth exemptions**: `/api/health`, `/api/version`, `/metrics`, `/api/openapi.yaml`, and the static assets are always unauthenticated (monitoring endpoints; the SPA shell must load to show a login form). Everything else 401s without a valid session when auth is enabled. `/api/auth/login` is necessarily exempt; failed logins count and are rate limited like any request.
|
||||
19. **API rate limiting**: token bucket per client IP (NOT the DNS fixed-window limiter — m6 ruling 8 reserved the bucket shape for the API): capacity and refill `web.api_rate_limit_per_min` per 60 s, 4096 tracked IPs (same fixed-table idiom as rate_limiter/tracker). Localhost exemption: NEW config field `web.api_localhost_exempt: bool = true` (PLAN §10 says "configurable" and §12.1 forgot the field; model + validate
|
||||
+ settings key + export round trip — settings are kv rows, no migration needed). 429 + `Retry-After: <s>`. SSE: connect consumes a token AND respects `web.sse_max_connections_per_ip`; `/metrics` and `/api/health` are limiter-exempt (Prometheus must never see 429).
|
||||
20. **SSE**: `GET /api/queries/live`, `text/event-stream` via `respondStreaming(&.{}, ...)` (EMPTY buffer — `BodyWriter.flush` does not flush the body writer's own buffer, http.zig:780; empty buffer makes every write go straight to the chunked drain), flush headers before the first event (test.zig:498 pattern). Fanout: milestone-6 ruling 4 concretized as `src/server/query_sink.zig`: `QuerySink { logger: *Logger, hub: ?*sse.Hub }` with `log(io, entry)` = transform once, publish to the hub, enqueue to the logger. Logger gains additive `transformed(entry) Entry` (pure) and `logTransformed(io, entry)`; existing `log` = both, frozen behavior. `Handler.logger: ?*Logger` becomes `sink: ?*QuerySink` (mechanical rename at the two call sites + app wiring). Hub: fixed 32 subscriber slots, each a 64-entry ring + `std.Io.Event`; publish never blocks (full ring → disconnect that subscriber; SSE clients auto-reconnect). Wire format: `retry: 3000` once, then `event: query` + `data: <JSON, same fields as /api/queries rows>` per entry; heartbeat comment `: ping` every 15 s from the subscriber task (Event.waitTimeout). Fanout precedes persistence (PLAN:455) — the sink publishes before enqueue.
|
||||
21. **`/metrics`**: Prometheus text format 0.0.4, `nxdns_` prefix, `# HELP`/`# TYPE` lines. Counters: every `Handler.Stats` field (17, individually, `nxdns_dns_<field>_total`), logger (queries_dropped, rows_written, batches_gated), cache stats (6, under `Handler.cache_mutex`), DNS limiter stats (under `limiter_mutex`), tracker stats, retention stats, log-sink stats, `nxdns_blocklist_refreshes_gated_total`. Gauges: cache len + memory bytes, disk free/db/log bytes, tracked DNS clients, pending tracker clients, blocklist generation, `nxdns_up 1`. Per-upstream, labeled `{url="..."}`: up (available), success_rate, consecutive_failures, total_successes/_failures (copy `Pool.Snapshot` fields while holding the returned count — `last_error` is borrowed, copy immediately). No timestamps. Auth-exempt, limiter-exempt.
|
||||
22. **`GET /api/health`**: `{"status":"ok"|"degraded","disk":{"state":..., "free_bytes":...,"db_bytes":...,"log_bytes":...,"sample_failures":...}, "upstreams":{"available":N,"total":M},"queries_dropped":N,"writer_failed":bool, "refreshes_gated":N,"snapshot_generation":N|null}`. Degraded iff disk state != ok, or zero upstreams available, or writer_failed. 200 either way (degraded is data, not an HTTP failure).
|
||||
23. **OpenAPI + contract tests**: hand-written `src/web/openapi.yaml`, `@embedFile`d and served verbatim. No external validator (dependencies are liabilities): the contract IS a Zig table in the integration test — every route with method, auth requirement, a seeded request, expected status, and a response-shape struct that `std.json.parseFromSlice` must accept with `.ignore_unknown_fields = false`. Two drift guards: (a) the router exposes `pub const routes: []const RouteInfo`; a test asserts every entry's path+method appears textually in openapi.yaml; (b) the table covers every `routes` entry (count equality). CI gets one new job step running the existing `-Dintegration` suite (contract tests live inside it) — no new tooling.
|
||||
24. **Static assets**: build.zig gains `-Dweb-dist=<path>` (default `web/dist-placeholder/`, committed, containing a real minimal `index.html` — current status via /api/health fetch, links to /metrics; plus favicon). A build step copies the dist dir via `b.addWriteFiles()` + generated `assets.zig` index (the fixtures anonymous-import pattern, per research notes — @embedFile paths must live inside the module root), exposing `pub const files: []const File{path, bytes, content_type, etag}`; etag = comptime hash. Serving: exact path match, `/` → index.html, SPA fallback (unknown non-/api path → index.html, 200 — TanStack Router needs it in M9), `ETag`/ `If-None-Match` → 304, `content-encoding: gzip` when a sibling `<name>.gz` exists in the dist AND the request accepts it. No Date/Last-Modified headers (std has no RFC 1123 formatter; ETag is strictly better for immutable embedded content). Dev mode: CLI flag `nxdns run --web-dev <dir>` serves from disk (no cache headers), bypassing the embed.
|
||||
25. **Head-string trap**: `request.head.target` is invalidated by body reads (Server.zig:594/230). The router copies target into a stack buffer before any body read. Query strings: split on '?', `std.mem.splitScalar` for pairs, `Uri.percentDecodeInPlace` per value (std has no query iterator; std.Uri does not parse origin-form targets).
|
||||
26. **Wiring**: one `web_server.serve` task in the app group, started last, canceled by the same group.cancel. `WebState` struct (in web/server.zig) of borrowed pointers assembled by app.serve: handler (stats + mutexes + cache/limiter), pause, tracker, manager, pool, monitor, logger, retention stats, local_tables, sink/hub, config arena view (web cfg), its OWN `config.db` + `querylog.db` connections (m7 ruling 21), version string, started timestamp. Web DB connections open only when `web.enabled`.
|
||||
27. **No new schema migration**: query_log already carries what /api/queries needs; the read session verifies existing indexes and reports if a needed index is missing (then the orchestrator rules on adding migration v3 — do not add one silently).
|
||||
28. **Client disconnect**: stream writes surface `error.WriteFailed` with the real cause in `stream_writer.err` (ConnectionResetByPeer / SocketUnconnected=EPIPE — MSG.NOSIGNAL is set, Threaded.zig:13071, no SIGPIPE masking needed). A failed write ends the connection task quietly (debug log at most): clients vanishing is normal.
|
||||
29. **Input sanitation** (PLAN §19): every path/query/body input is length-capped and validated before SQL (prepared statements everywhere, already house rule); domain inputs run through matcher.normalize; no secrets ever logged (password, tokens, cookie values — assert by review, and the login handler logs only the client IP and outcome).
|
||||
|
||||
## Sessions
|
||||
|
||||
Wave 1 (parallel): W1 query-log reads, W2 repo CRUD, W3 web core, W4 auth+limiter,
|
||||
W5 sink+SSE hub+logger split.
|
||||
Wave 2 (after W3, parallel where files allow): W6 read handlers + metrics, W7 mutation
|
||||
handlers + local_tables + handler.zig sink/local_tables changes.
|
||||
Wave 3: W8 static assets + build.zig + openapi.yaml + router registration of everything.
|
||||
Wave 4: W9 app/cli wiring. Wave 5: W10 integration + contract tests + CI.
|
||||
The orchestrator wires src/tests.zig per wave and rules on anything a session flags.
|
||||
Wave 1 (parallel): W1 query-log reads, W2 repo CRUD, W3 web core, W4 auth+limiter, W5 sink+SSE hub+logger split. Wave 2 (after W3, parallel where files allow): W6 read handlers + metrics, W7 mutation handlers + local_tables + handler.zig sink/local_tables changes. Wave 3: W8 static assets + build.zig + openapi.yaml + router registration of everything. Wave 4: W9 app/cli wiring. Wave 5: W10 integration + contract tests + CI. The orchestrator wires src/tests.zig per wave and rules on anything a session flags.
|
||||
|
||||
---
|
||||
|
||||
@@ -233,80 +74,27 @@ pub fn timeseries(database: *db.Db, since: i64, bucket_seconds: u32, out: []Buck
|
||||
db.Error!usize
|
||||
```
|
||||
|
||||
Read the real query_log schema first (querylog_schema.zig) — domain interning means a JOIN;
|
||||
verify what indexes exist and REPORT (ruling 27) if `ts` or the join needs one that is
|
||||
missing rather than adding a migration. Dynamic WHERE assembly must still use bound
|
||||
parameters only (build the SQL from fixed fragments, never interpolate values). Cap
|
||||
`limit` at 1000 in the repo too. Tests: in-memory db seeded via BatchWriter; filters,
|
||||
keyset paging across a boundary, bucket alignment, empty ranges, substring escaping
|
||||
(`%`/`_` in domain must not act as wildcards — use ESCAPE).
|
||||
Read the real query_log schema first (querylog_schema.zig) — domain interning means a JOIN; verify what indexes exist and REPORT (ruling 27) if `ts` or the join needs one that is missing rather than adding a migration. Dynamic WHERE assembly must still use bound parameters only (build the SQL from fixed fragments, never interpolate values). Cap `limit` at 1000 in the repo too. Tests: in-memory db seeded via BatchWriter; filters, keyset paging across a boundary, bucket alignment, empty ranges, substring escaping (`%`/`_` in domain must not act as wildcards — use ESCAPE).
|
||||
|
||||
Acceptance: fmt/ast clean; temp-root (-lc -lsqlite3) green; frozen surface untouched.
|
||||
|
||||
### W1 As built
|
||||
|
||||
`pub const max_limit: u32 = 1000` exported. WHERE assembly: six comptime fragments copied
|
||||
into a comptime-sized stack buffer (overflow impossible by construction), one bare `?` per
|
||||
predicate, bind order = append order; `likePattern` escapes `%`, `_` and `\` with
|
||||
`ESCAPE '\'`. Ruling 27 discharged: EXPLAIN QUERY PLAN over all six query shapes rides
|
||||
idx_query_log_client / idx_query_log_ts / rowid — NO migration v3; the time-windowed page's
|
||||
temp B-tree re-sort and the DISTINCT count's temp B-tree are inherent and cheap at household
|
||||
volume. `statsTotals` computes the mean as sum/count integer division (db.Stmt has no float
|
||||
column reader; avg() returns REAL) — exact microseconds, null when no row recorded a time.
|
||||
NULL `block_reason`/`upstream` arrive as `""` (columnText convention; neither is ever
|
||||
written as an empty string). **W6 convention, ruled now: `""` serializes as `""`, not JSON
|
||||
null** — `upstream=""` already means "cache hit" per milestone-7 ruling 20 and clients
|
||||
branch on `blocked`/`cache_hit`, not on reason presence. `timeseries` returns
|
||||
`error.Misuse` for bucket_seconds == 0 or an i64 window overflow, returns 0 for an empty
|
||||
`out` without touching the database, and aligns buckets to `since` — the HANDLER passes a
|
||||
grid-aligned since per ruling 13. 14 new tests (43 in file).
|
||||
`pub const max_limit: u32 = 1000` exported. WHERE assembly: six comptime fragments copied into a comptime-sized stack buffer (overflow impossible by construction), one bare `?` per predicate, bind order = append order; `likePattern` escapes `%`, `_` and `\` with `ESCAPE '\'`. Ruling 27 discharged: EXPLAIN QUERY PLAN over all six query shapes rides idx_query_log_client / idx_query_log_ts / rowid — NO migration v3; the time-windowed page's temp B-tree re-sort and the DISTINCT count's temp B-tree are inherent and cheap at household volume. `statsTotals` computes the mean as sum/count integer division (db.Stmt has no float column reader; avg() returns REAL) — exact microseconds, null when no row recorded a time. NULL `block_reason`/`upstream` arrive as `""` (columnText convention; neither is ever written as an empty string). **W6 convention, ruled now: `""` serializes as `""`, not JSON null** — `upstream=""` already means "cache hit" per milestone-7 ruling 20 and clients branch on `blocked`/`cache_hit`, not on reason presence. `timeseries` returns `error.Misuse` for bucket_seconds == 0 or an i64 window overflow, returns 0 for an empty `out` without touching the database, and aligns buckets to `since` — the HANDLER passes a grid-aligned since per ruling 13. 14 new tests (43 in file).
|
||||
|
||||
---
|
||||
|
||||
## Session W2: repo CRUD by id
|
||||
|
||||
Owns every file in `src/storage/repositories/` EXCEPT queries_repo.zig (W1's), plus nothing
|
||||
else. Additive only; import-path functions frozen.
|
||||
Owns every file in `src/storage/repositories/` EXCEPT queries_repo.zig (W1's), plus nothing else. Additive only; import-path functions frozen.
|
||||
|
||||
For groups, clients (+prefixes), upstreams, sources, rules, local records, forward zones:
|
||||
id-carrying list variants (or extend existing rows where the shape already has no
|
||||
consumers outside export — check callers first; export/import must keep compiling
|
||||
unchanged), `getX(db, id) db.Error!?XRow`, `insertXRow(db, item) db.Error!i64` (returns
|
||||
id; distinct from frozen import inserts where semantics differ — e.g. clients insert with
|
||||
hand_edited=1), `updateX(db, id, item) db.Error!void` (0 rows → error.NotFound),
|
||||
`deleteX(db, id) db.Error!void` (same), `setGroupSources(db, group_id, source_ids)
|
||||
db.Error!void` (transactional replace), `replaceClientPrefixes(db, items) db.Error!void`.
|
||||
Respect FK constraints: deleting a group with clients → error.Constraint surfaces as 409 at
|
||||
the handler; document each. `settings_repo`: `putSetting(db, key, value)` single-key upsert
|
||||
if absent. Tests per resource: round trip, NotFound on both update and delete, constraint
|
||||
surfacing, group-sources replace idempotence.
|
||||
For groups, clients (+prefixes), upstreams, sources, rules, local records, forward zones: id-carrying list variants (or extend existing rows where the shape already has no consumers outside export — check callers first; export/import must keep compiling unchanged), `getX(db, id) db.Error!?XRow`, `insertXRow(db, item) db.Error!i64` (returns id; distinct from frozen import inserts where semantics differ — e.g. clients insert with hand_edited=1), `updateX(db, id, item) db.Error!void` (0 rows → error.NotFound), `deleteX(db, id) db.Error!void` (same), `setGroupSources(db, group_id, source_ids) db.Error!void` (transactional replace), `replaceClientPrefixes(db, items) db.Error!void`. Respect FK constraints: deleting a group with clients → error.Constraint surfaces as 409 at the handler; document each. `settings_repo`: `putSetting(db, key, value)` single-key upsert if absent. Tests per resource: round trip, NotFound on both update and delete, constraint surfacing, group-sources replace idempotence.
|
||||
|
||||
Acceptance: fmt/ast clean; temp-root green; `zig build test` green (export/import tests
|
||||
prove the frozen surface).
|
||||
Acceptance: fmt/ast clean; temp-root green; `zig build test` green (export/import tests prove the frozen surface).
|
||||
|
||||
### W2 As built
|
||||
|
||||
New shared `crud.zig`: `execStrict` turns zero-rows-touched into `error.NotFound`. Accepted
|
||||
deviations: `getX(database, gpa, id) db.Error!?XRow` (rows hold heap strings);
|
||||
`insertClientRow`/`insertRuleRow` take `now_s` (NOT NULL columns, repos take no Io — the
|
||||
InsertContext.now mirror); client writes split `ClientInput{ip,name,group_id}` (create) vs
|
||||
`ClientEdit{name,group_id}` (edit) — **ip is not editable**: it is the identity upsertSeen
|
||||
matches, and ruling 9 promises only name/group on PUT; client_prefixes get list + replace
|
||||
only (whole-list resource, per-id would be unused generality); write shapes carry group_id
|
||||
(a missing group surfaces as the FK violation → 409, not an id-map miss); added
|
||||
`listGroupSourceIds` (the PUT needs a read counterpart; frozen listGroupSources speaks
|
||||
names). Constraint map for W7: deleting a group with clients → Constraint (clients.group_id
|
||||
has no ON DELETE; rules/prefixes/group-sources cascade); bad group_id on client/rule writes
|
||||
→ Constraint; url/ip/zone/name uniques → Constraint; deletes of clients/upstreams/rules/
|
||||
local-records/forward-zones can only be NotFound. `setGroupSources`: BEGIN IMMEDIATE,
|
||||
existence check first (empty set must not report success for a missing group), delete +
|
||||
distinct insert (set semantics dedupe); `replaceClientPrefixes` transactional whole-table,
|
||||
duplicate prefix REJECTED as Constraint (two rows for one prefix is a contradiction, not a
|
||||
set). Schema facts: `upstreams.tls_name` comes from migration v2 (config_schema.zig is v1
|
||||
only!) — the openapi /api/upstreams entry needs the fifth field; `SourceRow` gained
|
||||
`is_suggested: bool = false` appended last (manager.zig literals compile unchanged, no
|
||||
column index moved). `putSetting` is a primary-key upsert — partial settings PUT needs no
|
||||
read-first and fires no constraint. 45 new tests + 3 crud.
|
||||
New shared `crud.zig`: `execStrict` turns zero-rows-touched into `error.NotFound`. Accepted deviations: `getX(database, gpa, id) db.Error!?XRow` (rows hold heap strings); `insertClientRow`/`insertRuleRow` take `now_s` (NOT NULL columns, repos take no Io — the InsertContext.now mirror); client writes split `ClientInput{ip,name,group_id}` (create) vs `ClientEdit{name,group_id}` (edit) — **ip is not editable**: it is the identity upsertSeen matches, and ruling 9 promises only name/group on PUT; client_prefixes get list + replace only (whole-list resource, per-id would be unused generality); write shapes carry group_id (a missing group surfaces as the FK violation → 409, not an id-map miss); added `listGroupSourceIds` (the PUT needs a read counterpart; frozen listGroupSources speaks names). Constraint map for W7: deleting a group with clients → Constraint (clients.group_id has no ON DELETE; rules/prefixes/group-sources cascade); bad group_id on client/rule writes → Constraint; url/ip/zone/name uniques → Constraint; deletes of clients/upstreams/rules/ local-records/forward-zones can only be NotFound. `setGroupSources`: BEGIN IMMEDIATE, existence check first (empty set must not report success for a missing group), delete + distinct insert (set semantics dedupe); `replaceClientPrefixes` transactional whole-table, duplicate prefix REJECTED as Constraint (two rows for one prefix is a contradiction, not a set). Schema facts: `upstreams.tls_name` comes from migration v2 (config_schema.zig is v1 only!) — the openapi /api/upstreams entry needs the fifth field; `SourceRow` gained `is_suggested: bool = false` appended last (manager.zig literals compile unchanged, no column index moved). `putSetting` is a primary-key upsert — partial settings PUT needs no read-first and fires no constraint. 45 new tests + 3 crud.
|
||||
|
||||
---
|
||||
|
||||
@@ -314,365 +102,120 @@ read-first and fires no constraint. 45 new tests + 3 crud.
|
||||
|
||||
Owns: `src/web/server.zig`, `src/web/router.zig`, `src/web/http_util.zig`.
|
||||
|
||||
- server.zig: `WebState` (ruling 26 — declare the struct; fields it cannot yet point at get
|
||||
wired by W9), `pub fn serve(state: *WebState, io: std.Io) std.Io.Cancelable!void` — bind
|
||||
per ruling 5 (bind failure: warn and return — the DNS side must keep serving; app treats
|
||||
web bind failure as non-fatal, W9 documents), accept loop per ruling 4 with the Stop
|
||||
split, connection budget per ruling 7 (503 over cap), per-connection buffers (8 KiB recv
|
||||
/ 4 KiB send), keep-alive loop, dispatch into router.
|
||||
- router.zig: `RouteInfo { method: http.Method, pattern: []const u8, auth: enum {open,
|
||||
session}, handler: *const fn(...) }`; `pub const routes: []const RouteInfo` (ruling 23
|
||||
depends on it); match = exact segments + one `{id}` numeric capture; 405 with Allow when
|
||||
the path matches another method; target copied before body reads (ruling 25); query-pair
|
||||
iterator + percent-decode helpers (in http_util, pure, tested hard: '+', '%2F', truncated
|
||||
%, overlong values → 400).
|
||||
- http_util.zig: JSON respond helpers (`std.json.Stringify` streaming into a
|
||||
`Writer.Allocating` for content-length, or direct respond for small bodies), the error
|
||||
envelope, request-body reader with the 1 MiB cap (413), cookie parse/format, bearer of
|
||||
nothing else.
|
||||
- The connection handler calls: limiter (W4, via a comptime-checkable interface field on
|
||||
WebState so W3 compiles before W4 lands — define `ApiLimiter` and `Sessions` as W3-owned
|
||||
INTERFACE structs? NO: simpler, W3 declares the WebState fields as `*auth.Sessions` /
|
||||
`*auth.ApiLimiter` types and W3 is built AFTER W4's file exists on disk in the same wave
|
||||
— to keep the wave parallel, W3 instead keeps auth/limiter checks behind two function
|
||||
pointers on WebState (`check_auth`, `check_limit`) that W9 wires; W3 tests them with test
|
||||
doubles. This is the one seam where indirection is warranted; document it.)
|
||||
- Tests: router matching table, 405/404, query decoding, body cap, cookie round trip,
|
||||
keep-alive across two requests (loopback socket), 503 over cap, cancel-during-idle
|
||||
keep-alive returns promptly.
|
||||
- server.zig: `WebState` (ruling 26 — declare the struct; fields it cannot yet point at get wired by W9), `pub fn serve(state: *WebState, io: std.Io) std.Io.Cancelable!void` — bind per ruling 5 (bind failure: warn and return — the DNS side must keep serving; app treats web bind failure as non-fatal, W9 documents), accept loop per ruling 4 with the Stop split, connection budget per ruling 7 (503 over cap), per-connection buffers (8 KiB recv / 4 KiB send), keep-alive loop, dispatch into router.
|
||||
- router.zig: `RouteInfo { method: http.Method, pattern: []const u8, auth: enum {open, session}, handler: *const fn(...) }`; `pub const routes: []const RouteInfo` (ruling 23 depends on it); match = exact segments + one `{id}` numeric capture; 405 with Allow when the path matches another method; target copied before body reads (ruling 25); query-pair iterator + percent-decode helpers (in http_util, pure, tested hard: '+', '%2F', truncated %, overlong values → 400).
|
||||
- http_util.zig: JSON respond helpers (`std.json.Stringify` streaming into a `Writer.Allocating` for content-length, or direct respond for small bodies), the error envelope, request-body reader with the 1 MiB cap (413), cookie parse/format, bearer of nothing else.
|
||||
- The connection handler calls: limiter (W4, via a comptime-checkable interface field on WebState so W3 compiles before W4 lands — define `ApiLimiter` and `Sessions` as W3-owned INTERFACE structs? NO: simpler, W3 declares the WebState fields as `*auth.Sessions` / `*auth.ApiLimiter` types and W3 is built AFTER W4's file exists on disk in the same wave — to keep the wave parallel, W3 instead keeps auth/limiter checks behind two function pointers on WebState (`check_auth`, `check_limit`) that W9 wires; W3 tests them with test doubles. This is the one seam where indirection is warranted; document it.)
|
||||
- Tests: router matching table, 405/404, query decoding, body cap, cookie round trip, keep-alive across two requests (loopback socket), 503 over cap, cancel-during-idle keep-alive returns promptly.
|
||||
|
||||
Acceptance: fmt/ast clean; temp-root green (may import std only + own files); no
|
||||
listener/handler file touched.
|
||||
Acceptance: fmt/ast clean; temp-root green (may import std only + own files); no listener/handler file touched.
|
||||
|
||||
### W3 As built
|
||||
|
||||
Handler signature: `fn(state: *server.WebState, io: std.Io, request: *http_util.Request)
|
||||
http_util.HandlerError!void`, `HandlerError = {WriteFailed, HttpExpectationFailed,
|
||||
OutOfMemory}` — domain outcomes are status codes, only those three escape. `Request` folds
|
||||
the id capture in (`request.id: ?i64`; no separate params type) and carries a per-request
|
||||
arena. `RouteInfo` gained `rate_limit: enum{counted,exempt} = .counted` (ruling 19's
|
||||
exemptions as data, not path matching); WebState gained `fallback: ?HandlerFn` (SPA
|
||||
fallback is not a route entry; unmatched non-/api → fallback, unmatched /api → JSON 404)
|
||||
and `reload_fn` (W7's ruling-12 seam). routes.zig: W8 fills `pub const table`; router
|
||||
re-exports as `routes`; WebState.routes defaults to it, so the drift guards read the array
|
||||
the server matches. W4/W5 landed mid-session, so sessions/limiter/hub/sink are REAL typed
|
||||
pointers and check_auth/check_limit default to real implementations (`sessionAuth`,
|
||||
`bucketLimit`; `allowAll`/`neverLimit` exported as test doubles) — a half-wired server
|
||||
fails closed. serve returns void (cancellation is classified into the Stop enum;
|
||||
tcp_server precedent). Own validating percent-decoder — std.Uri's copies malformed escapes
|
||||
through as literal text (Uri.zig:172-192), ruling 29 wants 400; segments split before
|
||||
decode so %2F cannot forge a boundary. Loopback tests live in web/server_integration_test
|
||||
.zig (house gating pattern). Accepted tradeoff: over-capacity 503 is written and the
|
||||
socket closed without draining, so the client may see RST instead — draining would be an
|
||||
unbounded blocking read on an at-capacity accept loop (documented). RULINGS RECORDED:
|
||||
W7 may add the `local_tables` field + import to server.zig (two-line exception, W3
|
||||
consents); Retention.stats is a cross-task data race — W6 owns storage/retention.zig
|
||||
additively this wave: atomic counters + `snapshotStats` (logger-counters pattern), and
|
||||
/metrics reads only through it. 39 tests.
|
||||
Handler signature: `fn(state: *server.WebState, io: std.Io, request: *http_util.Request) http_util.HandlerError!void`, `HandlerError = {WriteFailed, HttpExpectationFailed, OutOfMemory}` — domain outcomes are status codes, only those three escape. `Request` folds the id capture in (`request.id: ?i64`; no separate params type) and carries a per-request arena. `RouteInfo` gained `rate_limit: enum{counted,exempt} = .counted` (ruling 19's exemptions as data, not path matching); WebState gained `fallback: ?HandlerFn` (SPA fallback is not a route entry; unmatched non-/api → fallback, unmatched /api → JSON 404) and `reload_fn` (W7's ruling-12 seam). routes.zig: W8 fills `pub const table`; router re-exports as `routes`; WebState.routes defaults to it, so the drift guards read the array the server matches. W4/W5 landed mid-session, so sessions/limiter/hub/sink are REAL typed pointers and check_auth/check_limit default to real implementations (`sessionAuth`, `bucketLimit`; `allowAll`/`neverLimit` exported as test doubles) — a half-wired server fails closed. serve returns void (cancellation is classified into the Stop enum; tcp_server precedent). Own validating percent-decoder — std.Uri's copies malformed escapes through as literal text (Uri.zig:172-192), ruling 29 wants 400; segments split before decode so %2F cannot forge a boundary. Loopback tests live in web/server_integration_test .zig (house gating pattern). Accepted tradeoff: over-capacity 503 is written and the socket closed without draining, so the client may see RST instead — draining would be an unbounded blocking read on an at-capacity accept loop (documented). RULINGS RECORDED: W7 may add the `local_tables` field + import to server.zig (two-line exception, W3 consents); Retention.stats is a cross-task data race — W6 owns storage/retention.zig additively this wave: atomic counters + `snapshotStats` (logger-counters pattern), and /metrics reads only through it. 39 tests.
|
||||
|
||||
**Amendment (post-W9 critical fix)**: `serveConn` normalizes the request head immediately
|
||||
after `receiveHead`: a body-carrying method (POST/PUT/PATCH) with neither Content-Length
|
||||
nor Transfer-Encoding gets `head.content_length = 0` before any dispatch or respond.
|
||||
RFC 9110 §8.6 defines such a request as having an empty body, but std leaves the head
|
||||
saying "unknown" and `respond` → `discardBody` (http/Server.zig:631) asserts one of the
|
||||
two is set — so a bare `curl -X POST` panicked the whole process, DNS included. Zero
|
||||
content-length satisfies every downstream path: `bodyReader` (http.zig:445) goes straight
|
||||
to `.ready`, `discardRemaining` sees EndOfStream, keep-alive survives, and
|
||||
`http_util.readBody` yields an empty slice. Regression test in
|
||||
server_integration_test.zig: raw POST without either header → well-formed response, empty
|
||||
body observed by the handler, fresh connection answered after, connection_errors 0.
|
||||
**Amendment (post-W9 critical fix)**: `serveConn` normalizes the request head immediately after `receiveHead`: a body-carrying method (POST/PUT/PATCH) with neither Content-Length nor Transfer-Encoding gets `head.content_length = 0` before any dispatch or respond. RFC 9110 §8.6 defines such a request as having an empty body, but std leaves the head saying "unknown" and `respond` → `discardBody` (http/Server.zig:631) asserts one of the two is set — so a bare `curl -X POST` panicked the whole process, DNS included. Zero content-length satisfies every downstream path: `bodyReader` (http.zig:445) goes straight to `.ready`, `discardRemaining` sees EndOfStream, keep-alive survives, and `http_util.readBody` yields an empty slice. Regression test in server_integration_test.zig: raw POST without either header → well-formed response, empty body observed by the handler, fresh connection answered after, connection_errors 0.
|
||||
|
||||
---
|
||||
|
||||
## Session W4: auth + API limiter
|
||||
|
||||
Owns: `src/web/auth.zig`, `src/web/api_limiter.zig`, plus the `model.zig`/`validate.zig`
|
||||
additions for `web.api_localhost_exempt` (ruling 19) — model field, validate rule (none
|
||||
needed beyond type), settings key list, and the export/import round-trip expectations
|
||||
(check model.toSettings reflection picks it up automatically; add the settings-count test
|
||||
adjustments).
|
||||
Owns: `src/web/auth.zig`, `src/web/api_limiter.zig`, plus the `model.zig`/`validate.zig` additions for `web.api_localhost_exempt` (ruling 19) — model field, validate rule (none needed beyond type), settings key list, and the export/import round-trip expectations (check model.toSettings reflection picks it up automatically; add the settings-count test adjustments).
|
||||
|
||||
- auth.zig: `Sessions` fixed 32-slot store per ruling 17 (create → cookie value out;
|
||||
validate cookie → bool; logout; clearAll; expiry sweep on access; LRU evict), login
|
||||
verify via argon2 strVerify (io + allocator required — argon2.zig:600), token =
|
||||
32 bytes randomSecure → url_safe_no_pad; storage = SHA-256 digest; compare
|
||||
timing_safe.eql([32]u8,...). `authEnabled(cfg) bool`. Pure unit tests with fixed tokens
|
||||
(seed io.random? no — inject the token bytes via a testable `createWithToken`).
|
||||
- api_limiter.zig: token bucket per ruling 19, fixed 4096-key table (house idiom), awake
|
||||
clock, `check(now, key, is_localhost) Result{allowed, retry_after_s}`; localhost exempt
|
||||
when configured; `sse_connections` per-IP counter with `tryAcquireSse/releaseSse` against
|
||||
`sse_max_connections_per_ip`. Tests: refill math at boundaries, burst=capacity, exempt
|
||||
localhost, sse cap acquire/release, eviction.
|
||||
- auth.zig: `Sessions` fixed 32-slot store per ruling 17 (create → cookie value out; validate cookie → bool; logout; clearAll; expiry sweep on access; LRU evict), login verify via argon2 strVerify (io + allocator required — argon2.zig:600), token = 32 bytes randomSecure → url_safe_no_pad; storage = SHA-256 digest; compare timing_safe.eql([32]u8,...). `authEnabled(cfg) bool`. Pure unit tests with fixed tokens (seed io.random? no — inject the token bytes via a testable `createWithToken`).
|
||||
- api_limiter.zig: token bucket per ruling 19, fixed 4096-key table (house idiom), awake clock, `check(now, key, is_localhost) Result{allowed, retry_after_s}`; localhost exempt when configured; `sse_connections` per-IP counter with `tryAcquireSse/releaseSse` against `sse_max_connections_per_ip`. Tests: refill math at boundaries, burst=capacity, exempt localhost, sse cap acquire/release, eviction.
|
||||
|
||||
Acceptance: fmt/ast clean; temp-root green; `zig build test` green (model/validate/export
|
||||
tests still pass with the new field).
|
||||
Acceptance: fmt/ast clean; temp-root green; `zig build test` green (model/validate/export tests still pass with the new field).
|
||||
|
||||
### W4 As built
|
||||
|
||||
Sessions: digests only (SHA-256), full-slot scan on validate so work does not depend on
|
||||
match position; `verifyPassword` returns `Outcome{ok,denied,unavailable}` — `.unavailable`
|
||||
(broken PHC string, OOM) maps to 500, not 401; nothing secret is ever formatted (warn
|
||||
prints `@errorName` only). `createWithToken`/`validateAt` are the test seams. Cookie value
|
||||
is 43 chars url_safe_no_pad; `cookie_attributes = "HttpOnly; SameSite=Lax; Path=/"`.
|
||||
Limiter deviations, accepted: (1) `check(io, now, addr)` — the limiter holds its OWN
|
||||
std.Io.Mutex (W3 dispatches connections concurrently; one lock inside beats N outside) and
|
||||
derives loopback itself via exported `isLoopback`, so callers cannot disagree; (2) refill
|
||||
counts millionths of a token and advances the clock only by the span converted, so the
|
||||
truncated remainder survives — at 1 req/min the 60 s boundary is exact (tests at 999/1000
|
||||
ms); (3) the SSE per-IP cap binds loopback clients too — `api_localhost_exempt` exempts
|
||||
the RATE, not the fixed hub-slot resource. Full table → unknown addresses allowed +
|
||||
`untracked` (DNS-limiter precedent); `sweep` drops only full, SSE-free, window-idle
|
||||
buckets; `retry_after_s` never 0 on refusal. Model ripple: `web.api_localhost_exempt` in
|
||||
Web + expected_keys + fixture; validate.zig needed no edit (bool; decodeValue already
|
||||
rejects non-bool text); no count assertion outside W4 files existed. 12 auth + 14 limiter
|
||||
tests.
|
||||
Sessions: digests only (SHA-256), full-slot scan on validate so work does not depend on match position; `verifyPassword` returns `Outcome{ok,denied,unavailable}` — `.unavailable` (broken PHC string, OOM) maps to 500, not 401; nothing secret is ever formatted (warn prints `@errorName` only). `createWithToken`/`validateAt` are the test seams. Cookie value is 43 chars url_safe_no_pad; `cookie_attributes = "HttpOnly; SameSite=Lax; Path=/"`. Limiter deviations, accepted: (1) `check(io, now, addr)` — the limiter holds its OWN std.Io.Mutex (W3 dispatches connections concurrently; one lock inside beats N outside) and derives loopback itself via exported `isLoopback`, so callers cannot disagree; (2) refill counts millionths of a token and advances the clock only by the span converted, so the truncated remainder survives — at 1 req/min the 60 s boundary is exact (tests at 999/1000 ms); (3) the SSE per-IP cap binds loopback clients too — `api_localhost_exempt` exempts the RATE, not the fixed hub-slot resource. Full table → unknown addresses allowed + `untracked` (DNS-limiter precedent); `sweep` drops only full, SSE-free, window-idle buckets; `retry_after_s` never 0 on refusal. Model ripple: `web.api_localhost_exempt` in Web + expected_keys + fixture; validate.zig needed no edit (bool; decodeValue already rejects non-bool text); no count assertion outside W4 files existed. 12 auth + 14 limiter tests.
|
||||
|
||||
---
|
||||
|
||||
## Session W5: query sink, SSE hub, logger split
|
||||
|
||||
Owns: `src/server/query_sink.zig` (new), `src/web/sse.zig` (new), `src/storage/logger.zig`
|
||||
(additive split ONLY: `transformed(entry) Entry` pure + `logTransformed(io, entry)`;
|
||||
existing `log` becomes transform+logTransformed with byte-identical behavior — the
|
||||
existing tests must pass unchanged), and `src/server/handler.zig` (mechanical: field
|
||||
`logger: ?*logger_mod.Logger` → `sink: ?*query_sink.QuerySink`, the log call site, tests
|
||||
updated to construct a sink around their logger).
|
||||
Owns: `src/server/query_sink.zig` (new), `src/web/sse.zig` (new), `src/storage/logger.zig` (additive split ONLY: `transformed(entry) Entry` pure + `logTransformed(io, entry)`; existing `log` becomes transform+logTransformed with byte-identical behavior — the existing tests must pass unchanged), and `src/server/handler.zig` (mechanical: field `logger: ?*logger_mod.Logger` → `sink: ?*query_sink.QuerySink`, the log call site, tests updated to construct a sink around their logger).
|
||||
|
||||
- query_sink.zig per ruling 20: publish BEFORE enqueue (PLAN:455).
|
||||
- sse.zig `Hub`: 32 slots × 64-entry Entry rings + Event per subscriber;
|
||||
`subscribe() ?SubscriberId` / `unsubscribe(id)`; `publish(io, entry)` copies the entry
|
||||
into every live ring (Entry is self-contained, ~400 B; a full ring marks the subscriber
|
||||
`overflowed` and sets its event — the subscriber task sees the flag and ends the
|
||||
response); `wait(id, timeout)` for the heartbeat loop. All under one mutex; publish is
|
||||
called on the DNS hot path — it must stay allocation-free and short (copy + flag set).
|
||||
Tests: publish/consume order, overflow disconnect flag, unsubscribe under load,
|
||||
heartbeat timeout returns empty.
|
||||
- sse.zig `Hub`: 32 slots × 64-entry Entry rings + Event per subscriber; `subscribe() ?SubscriberId` / `unsubscribe(id)`; `publish(io, entry)` copies the entry into every live ring (Entry is self-contained, ~400 B; a full ring marks the subscriber `overflowed` and sets its event — the subscriber task sees the flag and ends the response); `wait(id, timeout)` for the heartbeat loop. All under one mutex; publish is called on the DNS hot path — it must stay allocation-free and short (copy + flag set). Tests: publish/consume order, overflow disconnect flag, unsubscribe under load, heartbeat timeout returns empty.
|
||||
- Handler tests: one added test proving the sink publishes and logs (tiny hub + logger).
|
||||
|
||||
Acceptance: fmt/ast clean; temp-root green including ALL existing handler and logger
|
||||
tests; `zig build test` + `-Dintegration` green (phase7 tests construct Handlers — they
|
||||
compile against the renamed field; update them, they are in your ownership for THIS
|
||||
mechanical change only — list every touched line in the report).
|
||||
Acceptance: fmt/ast clean; temp-root green including ALL existing handler and logger tests; `zig build test` + `-Dintegration` green (phase7 tests construct Handlers — they compile against the renamed field; update them, they are in your ownership for THIS mechanical change only — list every touched line in the report).
|
||||
|
||||
### W5 As built
|
||||
|
||||
Every Hub method takes `io` (the mutex needs it); all locking is `lockUncancelable`
|
||||
(milestone-7 tracker precedent: `handle` has no error union). `Hub.init(self) void`
|
||||
initializes IN PLACE — 32×64 rings ≈ 900 KiB, so **W9 must `gpa.create(Hub)`, never a
|
||||
stack local**. Read surface ruling 20 left open, as built: `next(io, id) ?Entry` (pop
|
||||
oldest), `overflowed(io, id) bool` (sticky; pre-overflow entries stay readable — the
|
||||
subscriber drains then disconnects), `wait(io, id, timeout) Cancelable!Wake` with
|
||||
`Wake = {ready, timeout}`; `wait` resets the event under the mutex only while the ring is
|
||||
empty, so a racing publish is never lost, and a spurious futex wake reports `.timeout`
|
||||
(costs one heartbeat). Publish-before-persist is pinned structurally (enqueue never
|
||||
blocks, so a clock cannot observe order): the test closes the logger queue and the entry
|
||||
still reaches the hub while counting `queries_dropped`. W5 also made the minimal app.zig
|
||||
compile fix (sink around the logger, hub = null, `.sink = &sink`) — **W9: the sink already
|
||||
exists in app.zig; only the hub and the rest of the wiring are missing.** 11 sse + 4 sink
|
||||
Every Hub method takes `io` (the mutex needs it); all locking is `lockUncancelable` (milestone-7 tracker precedent: `handle` has no error union). `Hub.init(self) void` initializes IN PLACE — 32×64 rings ≈ 900 KiB, so **W9 must `gpa.create(Hub)`, never a stack local**. Read surface ruling 20 left open, as built: `next(io, id) ?Entry` (pop oldest), `overflowed(io, id) bool` (sticky; pre-overflow entries stay readable — the subscriber drains then disconnects), `wait(io, id, timeout) Cancelable!Wake` with `Wake = {ready, timeout}`; `wait` resets the event under the mutex only while the ring is empty, so a racing publish is never lost, and a spurious futex wake reports `.timeout` (costs one heartbeat). Publish-before-persist is pinned structurally (enqueue never blocks, so a clock cannot observe order): the test closes the logger queue and the entry still reaches the hub while counting `queries_dropped`. W5 also made the minimal app.zig compile fix (sink around the logger, hub = null, `.sink = &sink`) — **W9: the sink already exists in app.zig; only the hub and the rest of the wiring are missing.** 11 sse + 4 sink
|
||||
+ 1 logger-split + 1 handler test; all pre-existing logger/handler tests unchanged.
|
||||
|
||||
---
|
||||
|
||||
## Session W6: read handlers + metrics + health + version + openapi route
|
||||
|
||||
Owns: `src/web/handlers/stats.zig`, `queries.zig`, `lookup.zig`, `upstream_health.zig`,
|
||||
`health.zig`, `version.zig`, `metrics.zig` (in web/, not handlers/ — PLAN layout :227).
|
||||
Owns: `src/web/handlers/stats.zig`, `queries.zig`, `lookup.zig`, `upstream_health.zig`, `health.zig`, `version.zig`, `metrics.zig` (in web/, not handlers/ — PLAN layout :227).
|
||||
|
||||
Implement rulings 11, 13, 14, 21, 22; `GET /api/version` = version.zig string + git commit
|
||||
build option. Every handler is `fn(state, io, request-ish, params) !void` matching W3's
|
||||
handler signature; JSON via http_util. Metrics: mind the mutex discipline (cache/limiter
|
||||
stats under the handler's mutexes; pool snapshot copies borrowed strings immediately;
|
||||
atomics via .monotonic loads). Handlers read the querylog via state's own connection.
|
||||
Tests: pure formatting tests per handler where the data is injectable (metrics text
|
||||
golden test, health degraded matrix, stats bucket math via W1 fixtures on an in-memory
|
||||
db).
|
||||
Implement rulings 11, 13, 14, 21, 22; `GET /api/version` = version.zig string + git commit build option. Every handler is `fn(state, io, request-ish, params) !void` matching W3's handler signature; JSON via http_util. Metrics: mind the mutex discipline (cache/limiter stats under the handler's mutexes; pool snapshot copies borrowed strings immediately; atomics via .monotonic loads). Handlers read the querylog via state's own connection. Tests: pure formatting tests per handler where the data is injectable (metrics text golden test, health degraded matrix, stats bucket math via W1 fixtures on an in-memory db).
|
||||
|
||||
Acceptance: fmt/ast clean; temp-root green.
|
||||
|
||||
### W6 As built
|
||||
|
||||
Every handler is split into a pure core plus a thin `handle`, so the decisions are tested
|
||||
without an `http.Server.Request`: `metrics.collect`/`metrics.render`, `health.collect`/
|
||||
`rollup`, `stats.window`/`periodParam`, `queries.parseFilter`/`page`, `lookup.evaluate`/
|
||||
`body`, `upstream_health.collect`, `version.body`. Entry points for W8's table:
|
||||
`metrics.handle`, `stats.totals`, `stats.timeseries`, `queries.list`, `lookup.handle`,
|
||||
`upstream_health.handle`, `health.handle`, `version.handle` — all match `router.HandlerFn`
|
||||
(proven by a temp-root assignment). `/metrics` and `/api/health` are `auth = .open,
|
||||
rate_limit = .exempt`; `/api/version` is `.open, .counted`; the other four are `.session,
|
||||
.counted`. **W8 must add `_ = @import("web/metrics.zig")` and the six handler files to
|
||||
src/tests.zig** — this session did not wire the orchestrator's file.
|
||||
Retention (ruled in W3's As built): `Stats` is now the plain snapshot type and the live
|
||||
counters moved to a private `Counters` of `std.atomic.Value(u64)` in `Retention.counters`;
|
||||
`snapshotStats()` reads them `.monotonic`; `runOnce` derives its pass number from the
|
||||
`fetchAdd` result rather than re-reading. Test assertions changed mechanically at
|
||||
retention.zig lines 192-195, 215, 220, 234, 238-240, 243-244, 258-260, 283-286, 312-313 and
|
||||
at phase6_integration_test.zig lines 365-368 (`x.stats.f` → `x.snapshotStats().f`) — that
|
||||
second file is the only consumer outside retention.zig, and it compiles solely under
|
||||
`-Dintegration`, so the plain suite does not catch a break there. No behavior changed.
|
||||
Decisions and deviations, all accepted:
|
||||
- `/metrics` carries the DNS counters as an ARRAY keyed by `Handler.Stats`'s comptime field
|
||||
list, so a new counter in the handler appears in the exposition with no edit here; the
|
||||
same `counterGroup` helper walks every plain stats struct. Missing collaborator = the
|
||||
whole family is omitted (a zero would read as health, an absent series reads as a gap).
|
||||
- Added beyond ruling 21's list: `nxdns_disk_sample_failures_total` (a failure mode that
|
||||
ruling 22 already surfaces) and `nxdns_upstream_enabled` (an upstream that is down and
|
||||
one that is switched off are different operator problems). Logger `writer_failed` is
|
||||
deliberately NOT a metric: it is in `/api/health` and would need a gauge family of one.
|
||||
- `metrics.poolSnapshot` is the one place that copies pool health, shared with the health
|
||||
rollup: cancel protection around `Pool.snapshot` (server.zig's precedent), then every
|
||||
borrowed string duped into the request arena immediately. Cache/limiter stats are read
|
||||
under `Handler.cache_mutex`/`limiter_mutex` with `lockUncancelable` — `HandlerError` has
|
||||
no `Canceled`, so the W5 precedent applies.
|
||||
- Ruling 13's window: `until` = end of the bucket `now` falls in, `since = until - width ×
|
||||
count`, both on the absolute epoch grid (every width divides 86 400, asserted at
|
||||
comptime), so `/api/stats` and `/api/stats/timeseries` cover the identical span — tested
|
||||
by summing the buckets against the totals. Both bodies also carry `period`, `since` and
|
||||
`until` (a chart cannot label an axis without them); the timeseries body adds
|
||||
`bucket_seconds`. **openapi.yaml must document those fields.**
|
||||
- `/api/queries`: `limit` outside 1..1000 is a 400 rather than a silent clamp, `before` must
|
||||
be positive, and each bad parameter has its own message. `next_before` is the last row's
|
||||
id only on a full page. `domain=`/`client=` empty read as absent.
|
||||
- `/api/lookup` resolves `group_id` through `groupIndexById`; an id no snapshot group has is
|
||||
a 400 (`unknown group_id`), a missing manager or snapshot is 503 `no snapshot loaded`, and
|
||||
the domain runs through `name.fromText` + `matcher.normalize` (ruling 29). `source_url`
|
||||
comes from `sources_repo.getSource` on the config connection keyed by the snapshot's
|
||||
source row id — `SourceSets` carries the display name, not the URL; an unreadable row
|
||||
leaves the field null rather than failing the lookup. Records and zones are read through
|
||||
`state.handler.?.local_tables` (acquire/release bracketed by one defer), NOT a WebState
|
||||
field, because that pointer already exists and is the same table W7 swaps.
|
||||
- `/api/upstream/health` reports url, enabled, available, consecutive_failures,
|
||||
total_successes, total_failures, success_rate, last_error, plus `available`/`total`
|
||||
rollup counts. No timestamps: they are `awake`-clock stamps and mean nothing to a client.
|
||||
- `/api/version` adds `zig_version` and `uptime_seconds` (from `WebState.started_unix`) to
|
||||
the version and commit strings.
|
||||
- The only logging in this session is one `warn` per failed database read or pool/source
|
||||
fault, on the 500 path; nothing is logged for a normal request, and no `std.log.err`.
|
||||
40 new tests (metrics 5, stats 8, queries 9, lookup 7, health 4, upstream_health 3,
|
||||
version 4); retention's 6 tests unchanged in number and green.
|
||||
Every handler is split into a pure core plus a thin `handle`, so the decisions are tested without an `http.Server.Request`: `metrics.collect`/`metrics.render`, `health.collect`/ `rollup`, `stats.window`/`periodParam`, `queries.parseFilter`/`page`, `lookup.evaluate`/ `body`, `upstream_health.collect`, `version.body`. Entry points for W8's table: `metrics.handle`, `stats.totals`, `stats.timeseries`, `queries.list`, `lookup.handle`, `upstream_health.handle`, `health.handle`, `version.handle` — all match `router.HandlerFn` (proven by a temp-root assignment). `/metrics` and `/api/health` are `auth = .open, rate_limit = .exempt`; `/api/version` is `.open, .counted`; the other four are `.session, .counted`. **W8 must add `_ = @import("web/metrics.zig")` and the six handler files to src/tests.zig** — this session did not wire the orchestrator's file. Retention (ruled in W3's As built): `Stats` is now the plain snapshot type and the live counters moved to a private `Counters` of `std.atomic.Value(u64)` in `Retention.counters`; `snapshotStats()` reads them `.monotonic`; `runOnce` derives its pass number from the `fetchAdd` result rather than re-reading. Test assertions changed mechanically at retention.zig lines 192-195, 215, 220, 234, 238-240, 243-244, 258-260, 283-286, 312-313 and at phase6_integration_test.zig lines 365-368 (`x.stats.f` → `x.snapshotStats().f`) — that second file is the only consumer outside retention.zig, and it compiles solely under `-Dintegration`, so the plain suite does not catch a break there. No behavior changed. Decisions and deviations, all accepted:
|
||||
- `/metrics` carries the DNS counters as an ARRAY keyed by `Handler.Stats`'s comptime field list, so a new counter in the handler appears in the exposition with no edit here; the same `counterGroup` helper walks every plain stats struct. Missing collaborator = the whole family is omitted (a zero would read as health, an absent series reads as a gap).
|
||||
- Added beyond ruling 21's list: `nxdns_disk_sample_failures_total` (a failure mode that ruling 22 already surfaces) and `nxdns_upstream_enabled` (an upstream that is down and one that is switched off are different operator problems). Logger `writer_failed` is deliberately NOT a metric: it is in `/api/health` and would need a gauge family of one.
|
||||
- `metrics.poolSnapshot` is the one place that copies pool health, shared with the health rollup: cancel protection around `Pool.snapshot` (server.zig's precedent), then every borrowed string duped into the request arena immediately. Cache/limiter stats are read under `Handler.cache_mutex`/`limiter_mutex` with `lockUncancelable` — `HandlerError` has no `Canceled`, so the W5 precedent applies.
|
||||
- Ruling 13's window: `until` = end of the bucket `now` falls in, `since = until - width × count`, both on the absolute epoch grid (every width divides 86 400, asserted at comptime), so `/api/stats` and `/api/stats/timeseries` cover the identical span — tested by summing the buckets against the totals. Both bodies also carry `period`, `since` and `until` (a chart cannot label an axis without them); the timeseries body adds `bucket_seconds`. **openapi.yaml must document those fields.**
|
||||
- `/api/queries`: `limit` outside 1..1000 is a 400 rather than a silent clamp, `before` must be positive, and each bad parameter has its own message. `next_before` is the last row's id only on a full page. `domain=`/`client=` empty read as absent.
|
||||
- `/api/lookup` resolves `group_id` through `groupIndexById`; an id no snapshot group has is a 400 (`unknown group_id`), a missing manager or snapshot is 503 `no snapshot loaded`, and the domain runs through `name.fromText` + `matcher.normalize` (ruling 29). `source_url` comes from `sources_repo.getSource` on the config connection keyed by the snapshot's source row id — `SourceSets` carries the display name, not the URL; an unreadable row leaves the field null rather than failing the lookup. Records and zones are read through `state.handler.?.local_tables` (acquire/release bracketed by one defer), NOT a WebState field, because that pointer already exists and is the same table W7 swaps.
|
||||
- `/api/upstream/health` reports url, enabled, available, consecutive_failures, total_successes, total_failures, success_rate, last_error, plus `available`/`total` rollup counts. No timestamps: they are `awake`-clock stamps and mean nothing to a client.
|
||||
- `/api/version` adds `zig_version` and `uptime_seconds` (from `WebState.started_unix`) to the version and commit strings.
|
||||
- The only logging in this session is one `warn` per failed database read or pool/source fault, on the 500 path; nothing is logged for a normal request, and no `std.log.err`. 40 new tests (metrics 5, stats 8, queries 9, lookup 7, health 4, upstream_health 3, version 4); retention's 6 tests unchanged in number and green.
|
||||
|
||||
---
|
||||
|
||||
## Session W7: mutation handlers + local_tables + pause + settings + auth handlers
|
||||
|
||||
Owns: `src/web/handlers/` `groups.zig`, `blocklists.zig`, `rules.zig`, `local.zig`,
|
||||
`clients.zig`, `settings.zig`, `pause.zig`, `auth.zig` (login/logout), plus
|
||||
`src/server/local_tables.zig` (new, ruling 12) and the `src/server/handler.zig` edit to
|
||||
read records/zones through it (second mechanical handler change; coordinate: W5 already
|
||||
edits handler.zig — W7 runs AFTER W5 lands; sequence inside wave 2).
|
||||
Owns: `src/web/handlers/` `groups.zig`, `blocklists.zig`, `rules.zig`, `local.zig`, `clients.zig`, `settings.zig`, `pause.zig`, `auth.zig` (login/logout), plus `src/server/local_tables.zig` (new, ruling 12) and the `src/server/handler.zig` edit to read records/zones through it (second mechanical handler change; coordinate: W5 already edits handler.zig — W7 runs AFTER W5 lands; sequence inside wave 2).
|
||||
|
||||
- local_tables.zig: `LocalTables { lock: std.Io.RwLock, records, zones }` with
|
||||
`acquire/release` (shared) and `swap(io, new_records, new_zones)` (exclusive; frees the
|
||||
old under the lock after swap — no reader can hold across queries since acquire/release
|
||||
brackets each query). Handler: acquire in handle, release on every exit (same single
|
||||
defer discipline as the snapshot).
|
||||
- Handlers: rulings 9, 12, 15, 16, 17-18 behaviors; every mutation validates (shared
|
||||
validators — reuse validate.zig section rules by constructing the candidate row set;
|
||||
where validate.zig only does whole-config, extract the per-section check it already has
|
||||
— validate.zig is NOT in your ownership: if extraction is needed, report it and use a
|
||||
local check this milestone), writes via W2 repos on state's config connection, then
|
||||
reload/rebuild per ruling 12. blocklists/update → refreshAll + reload, 202 + status.
|
||||
settings PUT: merge, validate whole config, write keys, clear sessions on hash change.
|
||||
- Tests: per handler with in-memory dbs and a real Manager where cheap, else assert repo
|
||||
effects + reload-called flag via a seam (a `reload_fn` pointer on WebState set by W9;
|
||||
tests inject a counter).
|
||||
- local_tables.zig: `LocalTables { lock: std.Io.RwLock, records, zones }` with `acquire/release` (shared) and `swap(io, new_records, new_zones)` (exclusive; frees the old under the lock after swap — no reader can hold across queries since acquire/release brackets each query). Handler: acquire in handle, release on every exit (same single defer discipline as the snapshot).
|
||||
- Handlers: rulings 9, 12, 15, 16, 17-18 behaviors; every mutation validates (shared validators — reuse validate.zig section rules by constructing the candidate row set; where validate.zig only does whole-config, extract the per-section check it already has — validate.zig is NOT in your ownership: if extraction is needed, report it and use a local check this milestone), writes via W2 repos on state's config connection, then reload/rebuild per ruling 12. blocklists/update → refreshAll + reload, 202 + status. settings PUT: merge, validate whole config, write keys, clear sessions on hash change.
|
||||
- Tests: per handler with in-memory dbs and a real Manager where cheap, else assert repo effects + reload-called flag via a seam (a `reload_fn` pointer on WebState set by W9; tests inject a counter).
|
||||
|
||||
Acceptance: fmt/ast clean; temp-root green; phase7 integration still green.
|
||||
|
||||
### W7 As built
|
||||
|
||||
Eleven files, 4199 lines, 89 tests (local_tables 5, mutations 6, groups 13, blocklists 8,
|
||||
rules 7, local 12, clients 9, upstreams 6, pause 7, settings 10, auth 6).
|
||||
- **The apply/respond split (house pattern for later waves).** `http_util`'s response
|
||||
helpers need a live `*http.Server.Request`, which a unit test cannot construct. Every
|
||||
route splits into an `apply*` decision function driven against an in-memory database and
|
||||
a thin HTTP wrapper.
|
||||
- **`mutations.zig` is a new shared file, owned by W7.** It holds the `Failure` union, the
|
||||
constraint map (NotFound → 404, Constraint → 409, validation → 400), the `reload` and
|
||||
`swapLocalTables` seams, and the `Bench` test fixture.
|
||||
- **`config_lock` lives on WebState** (ruled post-report): connection tasks share one
|
||||
config database connection, and `changes()`/`lastInsertRowid()` are connection state that
|
||||
`execStrict` reads, so writes serialize through `state.config_lock`.
|
||||
- **Per-row validation**: a minimal valid skeleton config plus the one candidate row runs
|
||||
the real `validate.validate`; no validator duplicated. Duplicate keys and foreign-key
|
||||
violations surface from the database as 409, not 400.
|
||||
- **`upstreams.zig` is W7's** (ruling 9 listed the resource; neither ownership list named
|
||||
it). It never calls the reload seam, refuses to delete the last enabled upstream (409),
|
||||
and reports `restart_required: true`.
|
||||
- **Ruled additions**: `GET /api/settings` returns a derived `web.auth_enabled` flag; the
|
||||
default group cannot be renamed or deleted (409); a group that still holds clients cannot
|
||||
be deleted (409).
|
||||
- **handler.zig**: `records`/`zones` replaced by `local_tables: ?*LocalTables = null`;
|
||||
`handle` acquires once per query, releases on defer — one query reads one generation of
|
||||
records, zones and the filter snapshot together. `Context` gained `records`/`zones`.
|
||||
- **Ruled mechanical ripples**: web/server.zig two-line exception (import + WebState
|
||||
field, plus the config_lock field above); app.zig minimal fix (`var tables: LocalTables
|
||||
= .empty` + build calls — **W9 must review**); udp/tcp/resolver integration tests lost
|
||||
their `empty_records`/`empty_zones` consts and two `bareHandler` initialisers; phase7
|
||||
test converted to real tables where needed.
|
||||
- **Routes for W8** (all match W3's signature; id routes read `request.id.?`): groups
|
||||
`list get create update remove getSources putSources`; blocklists `list get create
|
||||
update remove refresh` (refresh = POST /api/blocklists/update, 202 + status snapshot);
|
||||
rules `list get create update remove`; local `listRecords getRecord createRecord
|
||||
updateRecord removeRecord listZones getZone createZone updateZone removeZone`; clients
|
||||
`list get update remove listPrefixes putPrefixes` (no create); upstreams `list get
|
||||
create update remove`; pause `get post`; settings `get put`; auth `login logout`.
|
||||
Eleven files, 4199 lines, 89 tests (local_tables 5, mutations 6, groups 13, blocklists 8, rules 7, local 12, clients 9, upstreams 6, pause 7, settings 10, auth 6).
|
||||
- **The apply/respond split (house pattern for later waves).** `http_util`'s response helpers need a live `*http.Server.Request`, which a unit test cannot construct. Every route splits into an `apply*` decision function driven against an in-memory database and a thin HTTP wrapper.
|
||||
- **`mutations.zig` is a new shared file, owned by W7.** It holds the `Failure` union, the constraint map (NotFound → 404, Constraint → 409, validation → 400), the `reload` and `swapLocalTables` seams, and the `Bench` test fixture.
|
||||
- **`config_lock` lives on WebState** (ruled post-report): connection tasks share one config database connection, and `changes()`/`lastInsertRowid()` are connection state that `execStrict` reads, so writes serialize through `state.config_lock`.
|
||||
- **Per-row validation**: a minimal valid skeleton config plus the one candidate row runs the real `validate.validate`; no validator duplicated. Duplicate keys and foreign-key violations surface from the database as 409, not 400.
|
||||
- **`upstreams.zig` is W7's** (ruling 9 listed the resource; neither ownership list named it). It never calls the reload seam, refuses to delete the last enabled upstream (409), and reports `restart_required: true`.
|
||||
- **Ruled additions**: `GET /api/settings` returns a derived `web.auth_enabled` flag; the default group cannot be renamed or deleted (409); a group that still holds clients cannot be deleted (409).
|
||||
- **handler.zig**: `records`/`zones` replaced by `local_tables: ?*LocalTables = null`; `handle` acquires once per query, releases on defer — one query reads one generation of records, zones and the filter snapshot together. `Context` gained `records`/`zones`.
|
||||
- **Ruled mechanical ripples**: web/server.zig two-line exception (import + WebState field, plus the config_lock field above); app.zig minimal fix (`var tables: LocalTables = .empty` + build calls — **W9 must review**); udp/tcp/resolver integration tests lost their `empty_records`/`empty_zones` consts and two `bareHandler` initialisers; phase7 test converted to real tables where needed.
|
||||
- **Routes for W8** (all match W3's signature; id routes read `request.id.?`): groups `list get create update remove getSources putSources`; blocklists `list get create update remove refresh` (refresh = POST /api/blocklists/update, 202 + status snapshot); rules `list get create update remove`; local `listRecords getRecord createRecord updateRecord removeRecord listZones getZone createZone updateZone removeZone`; clients `list get update remove listPrefixes putPrefixes` (no create); upstreams `list get create update remove`; pause `get post`; settings `get put`; auth `login logout`.
|
||||
|
||||
---
|
||||
|
||||
## Session W8: static assets, build pipeline, openapi.yaml, route table completion
|
||||
|
||||
Owns: `src/web/static.zig`, `src/web/openapi.zig`, `src/web/openapi.yaml`,
|
||||
`web/dist-placeholder/` (index.html + favicon.svg), `build.zig` (the -Dweb-dist option +
|
||||
asset module generation), and the final `router.zig` route-table registration IF W3 left
|
||||
registration data-driven (coordinate: W3 owns router.zig — W8 only ADDS the route array
|
||||
entries file `src/web/routes.zig` if W3 designed it that way; otherwise W8 hands W3's
|
||||
owner a patch — resolve by making W3 expose `routes` as a separate file owned by W8 from
|
||||
the start; W3 must read this paragraph).
|
||||
Owns: `src/web/static.zig`, `src/web/openapi.zig`, `src/web/openapi.yaml`, `web/dist-placeholder/` (index.html + favicon.svg), `build.zig` (the -Dweb-dist option + asset module generation), and the final `router.zig` route-table registration IF W3 left registration data-driven (coordinate: W3 owns router.zig — W8 only ADDS the route array entries file `src/web/routes.zig` if W3 designed it that way; otherwise W8 hands W3's owner a patch — resolve by making W3 expose `routes` as a separate file owned by W8 from the start; W3 must read this paragraph).
|
||||
|
||||
Ruling 24 in full: asset module via addWriteFiles + generated index; etag comptime hash;
|
||||
gzip sibling serving; SPA fallback; --web-dev disk serving (cli flag lands in W9 — W8
|
||||
exposes `serveFromDisk(dir, ...)`). openapi.yaml documents EVERY route in the table with
|
||||
schemas matching the handlers' JSON (snake_case, error envelope, auth markers, 429/401
|
||||
responses). Tests: static matching, 304 flow, gzip negotiation, SPA fallback vs /api 404,
|
||||
placeholder embeds and serves.
|
||||
Ruling 24 in full: asset module via addWriteFiles + generated index; etag comptime hash; gzip sibling serving; SPA fallback; --web-dev disk serving (cli flag lands in W9 — W8 exposes `serveFromDisk(dir, ...)`). openapi.yaml documents EVERY route in the table with schemas matching the handlers' JSON (snake_case, error envelope, auth markers, 429/401 responses). Tests: static matching, 304 flow, gzip negotiation, SPA fallback vs /api 404, placeholder embeds and serves.
|
||||
|
||||
Acceptance: fmt/ast clean; `zig build` with default dist green; temp-root green.
|
||||
|
||||
### W8 As built
|
||||
|
||||
Seven files: static.zig 362, openapi.zig 66, routes.zig 194 (55 routes), openapi.yaml
|
||||
2255, handlers/live.zig 184 (ruled mid-session — no session owned the SSE HTTP handler),
|
||||
tools/gen_web_assets.zig 201 (new build tool, W8's), web/dist-placeholder (real page:
|
||||
fetches /api/health + /api/version, renders status/upstreams/disk). 21 new tests
|
||||
(static 9, openapi 3, routes 6, live 3).
|
||||
- **Asset pipeline**: gzip cannot run in the build system itself, so
|
||||
`tools/gen_web_assets.zig` runs as a build step: dist staged through WriteFiles (Run
|
||||
hashes only a directory arg's path string; staging bakes the content hash into the
|
||||
path, so edits invalidate — proven), tool output merged with generated `.gz` siblings
|
||||
and `assets.zig` into the `web_assets` anonymous import (exe, cross exes, tests).
|
||||
- **`web_assets` module surface**: `File{path, bytes, content_type, etag}`,
|
||||
`files: []const File`; root is the generated assets.zig.
|
||||
- **ETag** computed in the tool, not comptime (same build-time constant; avoids the
|
||||
comptime interpreter hashing large M9 bundles). SHA-256/128-bit, quoted, strong.
|
||||
`.gz` siblings are their own `files` entries with their own ETag; direct `*.gz` paths
|
||||
are not addressable. The tool skips gz when it does not shrink or below 128 bytes.
|
||||
- **static.zig surface for W9**: `state.fallback = static.fallback` (embedded) or wrap
|
||||
`static.serveFromDisk(dir, io, request)` in a HandlerFn for `--web-dev` (app.zig owns
|
||||
the dir storage; no cache headers in dev mode). Traversal guard in `diskRelativePath`.
|
||||
- **live.zig** (SSE, ruling 20): respondStreaming with empty buffer, `retry: 3000`,
|
||||
`event: query` frames, 15 s heartbeat (awake clock) via `Hub.wait`, per-IP cap via
|
||||
`tryAcquireSse`/`releaseSse`, ring overflow ends the stream with a clean chunked
|
||||
terminator so EventSource reconnects. RULED: route is `.session, .exempt` (overrides
|
||||
ruling 19's connect-token line; the per-IP SSE cap is the guard, loopback included).
|
||||
RULED: the SSE payload omits `id` (a live entry precedes persistence); a comptime test
|
||||
pins the remaining nine field names to `QueryRow`.
|
||||
- **Route policy**: open = {/metrics, /api/health, /api/version, /api/openapi.yaml,
|
||||
/api/auth/login}; limiter-exempt = {/metrics, /api/health, /api/queries/live}; logout
|
||||
is `.session` (ruling 18).
|
||||
- **openapi.yaml** documents all 55 routes, cookie security scheme, 401 on every session
|
||||
route, 429 + Retry-After on every counted route, the W6 stats fields (period, since,
|
||||
until, bucket_seconds), SSE as text/event-stream; no certs/reload. openapi.zig's test
|
||||
enforces route ⊆ yaml at unit level; W10 still owns the count-equality drift guard (b).
|
||||
Seven files: static.zig 362, openapi.zig 66, routes.zig 194 (55 routes), openapi.yaml 2255, handlers/live.zig 184 (ruled mid-session — no session owned the SSE HTTP handler), tools/gen_web_assets.zig 201 (new build tool, W8's), web/dist-placeholder (real page: fetches /api/health + /api/version, renders status/upstreams/disk). 21 new tests (static 9, openapi 3, routes 6, live 3).
|
||||
- **Asset pipeline**: gzip cannot run in the build system itself, so `tools/gen_web_assets.zig` runs as a build step: dist staged through WriteFiles (Run hashes only a directory arg's path string; staging bakes the content hash into the path, so edits invalidate — proven), tool output merged with generated `.gz` siblings and `assets.zig` into the `web_assets` anonymous import (exe, cross exes, tests).
|
||||
- **`web_assets` module surface**: `File{path, bytes, content_type, etag}`, `files: []const File`; root is the generated assets.zig.
|
||||
- **ETag** computed in the tool, not comptime (same build-time constant; avoids the comptime interpreter hashing large M9 bundles). SHA-256/128-bit, quoted, strong. `.gz` siblings are their own `files` entries with their own ETag; direct `*.gz` paths are not addressable. The tool skips gz when it does not shrink or below 128 bytes.
|
||||
- **static.zig surface for W9**: `state.fallback = static.fallback` (embedded) or wrap `static.serveFromDisk(dir, io, request)` in a HandlerFn for `--web-dev` (app.zig owns the dir storage; no cache headers in dev mode). Traversal guard in `diskRelativePath`.
|
||||
- **live.zig** (SSE, ruling 20): respondStreaming with empty buffer, `retry: 3000`, `event: query` frames, 15 s heartbeat (awake clock) via `Hub.wait`, per-IP cap via `tryAcquireSse`/`releaseSse`, ring overflow ends the stream with a clean chunked terminator so EventSource reconnects. RULED: route is `.session, .exempt` (overrides ruling 19's connect-token line; the per-IP SSE cap is the guard, loopback included). RULED: the SSE payload omits `id` (a live entry precedes persistence); a comptime test pins the remaining nine field names to `QueryRow`.
|
||||
- **Route policy**: open = {/metrics, /api/health, /api/version, /api/openapi.yaml, /api/auth/login}; limiter-exempt = {/metrics, /api/health, /api/queries/live}; logout is `.session` (ruling 18).
|
||||
- **openapi.yaml** documents all 55 routes, cookie security scheme, 401 on every session route, 429 + Retry-After on every counted route, the W6 stats fields (period, since, until, bucket_seconds), SSE as text/event-stream; no certs/reload. openapi.zig's test enforces route ⊆ yaml at unit level; W10 still owns the count-equality drift guard (b).
|
||||
|
||||
---
|
||||
|
||||
@@ -680,118 +223,50 @@ fetches /api/health + /api/version, renders status/upstreams/disk). 21 new tests
|
||||
|
||||
Owns: `src/app.zig`, `src/cli.zig`, `src/main.zig` (if forced).
|
||||
|
||||
WebState assembly (ruling 26) gated on `web.enabled`; two extra DB connections; hub +
|
||||
sink construction (sink wraps logger always — hub only when web enabled; DNS path cost
|
||||
without web = one null check); local_tables construction replacing the direct
|
||||
records/zones locals; web serve task in the group (started last); reload_fn/check seams
|
||||
wired; `--web-dev <dir>` CLI flag (parseArgs + help text + runCheck untouched); web bind
|
||||
failure = warn + continue serving DNS (ruling in W3 — confirm and document in app).
|
||||
Shutdown order unchanged (web task dies with group.cancel; its connection group cancels
|
||||
per ruling 4). Smoke test yours: boot with web enabled, curl /api/health, /api/version,
|
||||
/metrics, /, login flow with a password set (import a config with web.password, verify
|
||||
cookie + 401 without), SIGTERM exit 0. Report the transcript.
|
||||
WebState assembly (ruling 26) gated on `web.enabled`; two extra DB connections; hub + sink construction (sink wraps logger always — hub only when web enabled; DNS path cost without web = one null check); local_tables construction replacing the direct records/zones locals; web serve task in the group (started last); reload_fn/check seams wired; `--web-dev <dir>` CLI flag (parseArgs + help text + runCheck untouched); web bind failure = warn + continue serving DNS (ruling in W3 — confirm and document in app). Shutdown order unchanged (web task dies with group.cancel; its connection group cancels per ruling 4). Smoke test yours: boot with web enabled, curl /api/health, /api/version, /metrics, /, login flow with a password set (import a config with web.password, verify cookie + 401 without), SIGTERM exit 0. Report the transcript.
|
||||
|
||||
Acceptance: fmt/ast clean; zig build; both test suites green; smoke transcript.
|
||||
|
||||
### W9 As built
|
||||
|
||||
- `cli.Command.run` payload is now `RunArgs { paths: Paths, web_dev: ?[]const u8 }`;
|
||||
`parseRunArgs` handles `--web-dev DIR` (both spellings); `check` rejects it;
|
||||
`runCheck` untouched. Anything spawning `app.run` passes `RunArgs`.
|
||||
- app.zig serve order: zero-guard (`web.api_rate_limit_per_min`/`session_ttl_hours` zero
|
||||
on an enabled web → `error.BadRateLimit`, exit 2 — collaborators assert nonzero and a
|
||||
hand-edited database bypasses validate); heap `sse.Hub` via `gpa.create` + in-place
|
||||
init (W5 rule), gated on `web.enabled`; sink wraps `(&logger, hub)` ALWAYS (hub null
|
||||
without web = one branch); two web DB connections (openConfigDb + reopenQuerylogDb,
|
||||
after openQuerylogDb establishes the file), gated; Sessions + ApiLimiter, gated;
|
||||
WebState assembled after the DNS handler (`reload_fn = app.reloadManager`,
|
||||
`fallback = static.fallback` or the dev wrapper, version.string, started_unix); web
|
||||
task added to the group LAST; bind failure = warn + return, DNS keeps serving.
|
||||
Shutdown order unchanged. `web.enabled = false` → no hub, no web DBs, no
|
||||
sessions/limiter, no web task.
|
||||
- `--web-dev` dir lives in a file-scope var in app.zig (`WebState.fallback` is a bare fn
|
||||
pointer, no closure; one composition root; written once before the task starts).
|
||||
- W7's `var tables: LocalTables` holder confirmed correct: `WebState.local_tables`
|
||||
points at the same holder the DNS handler reads, so `swapLocalTables` swaps both.
|
||||
- `cli.Command.run` payload is now `RunArgs { paths: Paths, web_dev: ?[]const u8 }`; `parseRunArgs` handles `--web-dev DIR` (both spellings); `check` rejects it; `runCheck` untouched. Anything spawning `app.run` passes `RunArgs`.
|
||||
- app.zig serve order: zero-guard (`web.api_rate_limit_per_min`/`session_ttl_hours` zero on an enabled web → `error.BadRateLimit`, exit 2 — collaborators assert nonzero and a hand-edited database bypasses validate); heap `sse.Hub` via `gpa.create` + in-place init (W5 rule), gated on `web.enabled`; sink wraps `(&logger, hub)` ALWAYS (hub null without web = one branch); two web DB connections (openConfigDb + reopenQuerylogDb, after openQuerylogDb establishes the file), gated; Sessions + ApiLimiter, gated; WebState assembled after the DNS handler (`reload_fn = app.reloadManager`, `fallback = static.fallback` or the dev wrapper, version.string, started_unix); web task added to the group LAST; bind failure = warn + return, DNS keeps serving. Shutdown order unchanged. `web.enabled = false` → no hub, no web DBs, no sessions/limiter, no web task.
|
||||
- `--web-dev` dir lives in a file-scope var in app.zig (`WebState.fallback` is a bare fn pointer, no closure; one composition root; written once before the task starts).
|
||||
- W7's `var tables: LocalTables` holder confirmed correct: `WebState.local_tables` points at the same holder the DNS handler reads, so `swapLocalTables` swaps both.
|
||||
- Mechanical ripple: phase7 test line 967 `cli.Paths{...}` → `RunArgs{ .paths = ... }`.
|
||||
- Smoke test passed end to end: import with web.password → boot → 200 on /api/health,
|
||||
/api/version, /metrics, /, /api/openapi.yaml → 401 without cookie → login sets
|
||||
HttpOnly/SameSite=Lax cookie → authed read → logout kills the cookie → wrong password
|
||||
401 → SIGTERM exit 0. No secrets in logs. `--web-dev` smoked: disk edits appear
|
||||
without restart.
|
||||
- **Critical finding (fixed post-session in W3's files)**: a bodyless POST/PUT — no
|
||||
Content-Length, no Transfer-Encoding, exactly `curl -X POST` — hit the
|
||||
`discardBody` assert in std (Server.zig:631) and panicked the process, DNS included.
|
||||
RFC 9110 gives such a request an empty body, so the fix normalizes in the request
|
||||
path before any respond call (see W3 As-built amendment); W10 pins it with a contract
|
||||
test.
|
||||
- Smoke test passed end to end: import with web.password → boot → 200 on /api/health, /api/version, /metrics, /, /api/openapi.yaml → 401 without cookie → login sets HttpOnly/SameSite=Lax cookie → authed read → logout kills the cookie → wrong password 401 → SIGTERM exit 0. No secrets in logs. `--web-dev` smoked: disk edits appear without restart.
|
||||
- **Critical finding (fixed post-session in W3's files)**: a bodyless POST/PUT — no Content-Length, no Transfer-Encoding, exactly `curl -X POST` — hit the `discardBody` assert in std (Server.zig:631) and panicked the process, DNS included. RFC 9110 gives such a request an empty body, so the fix normalizes in the request path before any respond call (see W3 As-built amendment); W10 pins it with a contract test.
|
||||
|
||||
---
|
||||
|
||||
## Session W10: integration + contract tests + CI
|
||||
|
||||
Owns: `src/web/web_integration_test.zig` (new), `.gitea/workflows/ci.yml` (additive job
|
||||
steps only).
|
||||
Owns: `src/web/web_integration_test.zig` (new), `.gitea/workflows/ci.yml` (additive job steps only).
|
||||
|
||||
Contract table per ruling 23 covering every route; auth on/off matrix (seeded
|
||||
password_hash vs empty); SSE test (connect, cause one query via a real Handler or direct
|
||||
sink.publish, assert `event: query` frame + heartbeat + cap enforcement via
|
||||
sse_max_connections_per_ip=1); rate-limit 429 + Retry-After; pagination walk; mutation →
|
||||
reload observed (snapshot generation bump); pause via API affects a real handler decision;
|
||||
settings PUT round trip; static + ETag 304; drift guards (a) and (b). CI: ensure the
|
||||
integration job still covers everything (contract tests ride -Dintegration; add a
|
||||
`zig build test -Dintegration` step only if missing — read the current yml first).
|
||||
Contract table per ruling 23 covering every route; auth on/off matrix (seeded password_hash vs empty); SSE test (connect, cause one query via a real Handler or direct sink.publish, assert `event: query` frame + heartbeat + cap enforcement via sse_max_connections_per_ip=1); rate-limit 429 + Retry-After; pagination walk; mutation → reload observed (snapshot generation bump); pause via API affects a real handler decision; settings PUT round trip; static + ETag 304; drift guards (a) and (b). CI: ensure the integration job still covers everything (contract tests ride -Dintegration; add a `zig build test -Dintegration` step only if missing — read the current yml first).
|
||||
|
||||
Acceptance: `zig build test -Dintegration` green with zero err lines; plain suite green.
|
||||
|
||||
### W10 As built
|
||||
|
||||
web_integration_test.zig, 1406 lines, 14 tests: 11 integration-gated (contract walk over
|
||||
all 55 routes with strict `ignore_unknown_fields = false` response shapes, auth on, auth
|
||||
off, 429 + Retry-After, SSE, pagination walk, mutation → reload, pause, settings round
|
||||
trip, static/ETag 304, bodyless POST) + 3 ungated pure guards (contract-covers-routes,
|
||||
drift guard a route ⊆ yaml, drift guard b: 55 yaml operations == router.routes.len).
|
||||
ci.yml UNTOUCHED — its test job already runs `zig build test -Dintegration` (line 25).
|
||||
- Environment fully real except two seams: real Manager/reload over an in-memory config
|
||||
DB, real Sessions/ApiLimiter/Hub (heap Hub per W5 rule), seeded querylog via
|
||||
BatchWriter. The pool transport and fetcher are never invoked —
|
||||
POST /api/blocklists/update runs in the walk BEFORE any source row exists (hermetic).
|
||||
- SSE reader de-frames chunked transfer coding (the unbuffered SSE writer emits one frame
|
||||
as many chunks; std's chunked writer emits each chunk's closing CRLF lazily — reading
|
||||
separators eagerly deadlocks until the next heartbeat).
|
||||
- Heartbeat asserted against the real 15 s interval (`live.heartbeat_interval` is not
|
||||
injectable): the SSE test runs ~15-20 s in a 40 s budget; the integration suite gained
|
||||
roughly 45 s of wall clock. Accepted rather than weakening the assertion.
|
||||
- Pause test drives Handler.handle directly with real DNS packets (phase7 queryFor
|
||||
pattern): blocked → API pause → forwarded (paused_queries bumps) → unpause → blocked.
|
||||
web_integration_test.zig, 1406 lines, 14 tests: 11 integration-gated (contract walk over all 55 routes with strict `ignore_unknown_fields = false` response shapes, auth on, auth off, 429 + Retry-After, SSE, pagination walk, mutation → reload, pause, settings round trip, static/ETag 304, bodyless POST) + 3 ungated pure guards (contract-covers-routes, drift guard a route ⊆ yaml, drift guard b: 55 yaml operations == router.routes.len). ci.yml UNTOUCHED — its test job already runs `zig build test -Dintegration` (line 25).
|
||||
- Environment fully real except two seams: real Manager/reload over an in-memory config DB, real Sessions/ApiLimiter/Hub (heap Hub per W5 rule), seeded querylog via BatchWriter. The pool transport and fetcher are never invoked — POST /api/blocklists/update runs in the walk BEFORE any source row exists (hermetic).
|
||||
- SSE reader de-frames chunked transfer coding (the unbuffered SSE writer emits one frame as many chunks; std's chunked writer emits each chunk's closing CRLF lazily — reading separators eagerly deadlocks until the next heartbeat).
|
||||
- Heartbeat asserted against the real 15 s interval (`live.heartbeat_interval` is not injectable): the SSE test runs ~15-20 s in a 40 s budget; the integration suite gained roughly 45 s of wall clock. Accepted rather than weakening the assertion.
|
||||
- Pause test drives Handler.handle directly with real DNS packets (phase7 queryFor pattern): blocked → API pause → forwarded (paused_queries bumps) → unpause → blocked.
|
||||
- Mutation-reload observed as generation 1 → 2 plus the rule live via GET /api/lookup.
|
||||
- Contract markers asserted by lookup into router.routes (method+pattern → auth and
|
||||
rate_limit equality, exact one-to-one coverage); response shapes reference the
|
||||
handlers' pub types where they exist; the strict settings parse also proves
|
||||
`web.password*` never serializes.
|
||||
- One expected `warn` line ("web login refused") from the wrong-password case; zero err
|
||||
lines suite-wide; no secrets logged.
|
||||
- Contract markers asserted by lookup into router.routes (method+pattern → auth and rate_limit equality, exact one-to-one coverage); response shapes reference the handlers' pub types where they exist; the strict settings parse also proves `web.password*` never serializes.
|
||||
- One expected `warn` line ("web login refused") from the wrong-password case; zero err lines suite-wide; no secrets logged.
|
||||
|
||||
---
|
||||
|
||||
## Module layout (new)
|
||||
|
||||
src/web/{server,router,http_util,auth,api_limiter,sse,static,openapi,metrics}.zig,
|
||||
src/web/routes.zig (W8), src/web/handlers/{auth,stats,queries,clients,groups,blocklists,
|
||||
rules,local,lookup,pause,settings,upstream_health,health,version}.zig,
|
||||
src/server/{query_sink,local_tables}.zig, src/web/openapi.yaml, web/dist-placeholder/,
|
||||
src/web/web_integration_test.zig.
|
||||
src/web/{server,router,http_util,auth,api_limiter,sse,static,openapi,metrics}.zig, src/web/routes.zig (W8), src/web/handlers/{auth,stats,queries,clients,groups,blocklists, rules,local,lookup,pause,settings,upstream_health,health,version}.zig, src/server/{query_sink,local_tables}.zig, src/web/openapi.yaml, web/dist-placeholder/, src/web/web_integration_test.zig.
|
||||
|
||||
## File ownership
|
||||
|
||||
W1 queries_repo; W2 other repositories/*; W3 web/{server,router,http_util}; W4
|
||||
web/{auth,api_limiter} + model/validate additions; W5 server/query_sink, web/sse,
|
||||
storage/logger (additive), server/handler (sink rename) + phase7 test compile fixes; W6
|
||||
web/handlers read set + web/metrics; W7 web/handlers mutation set (incl. mutations.zig,
|
||||
upstreams.zig) + server/local_tables + server/handler (local_tables read path; AFTER W5); W8 web/{static,openapi,routes} +
|
||||
openapi.yaml + dist-placeholder + build.zig; W9 app/cli/main; W10 its test file + ci.yml.
|
||||
Orchestrator: src/tests.zig, spec. Within-wave parallel sessions never share a file;
|
||||
handler.zig is touched by W5 then W7, strictly sequenced.
|
||||
W1 queries_repo; W2 other repositories/*; W3 web/{server,router,http_util}; W4 web/{auth,api_limiter} + model/validate additions; W5 server/query_sink, web/sse, storage/logger (additive), server/handler (sink rename) + phase7 test compile fixes; W6 web/handlers read set + web/metrics; W7 web/handlers mutation set (incl. mutations.zig, upstreams.zig) + server/local_tables + server/handler (local_tables read path; AFTER W5); W8 web/{static,openapi,routes} + openapi.yaml + dist-placeholder + build.zig; W9 app/cli/main; W10 its test file + ci.yml. Orchestrator: src/tests.zig, spec. Within-wave parallel sessions never share a file; handler.zig is touched by W5 then W7, strictly sequenced.
|
||||
|
||||
## Acceptance (milestone complete)
|
||||
|
||||
@@ -807,58 +282,20 @@ handler.zig is touched by W5 then W7, strictly sequenced.
|
||||
## Review (Codex, As built)
|
||||
|
||||
Round 1: 6 important + 3 minor, all fixed.
|
||||
- local.zig: all six mutation paths hold `config_lock` across write AND rebuild+swap, so
|
||||
LocalTables swaps publish in database-write order (deadlock-checked: no callee takes
|
||||
the lock).
|
||||
- settings/auth/server/app: GET /api/settings reads under `config_lock`. New
|
||||
`auth.LiveHash` (mutex holder on `WebState.live_hash`, generation-checked — see round
|
||||
2): login copies the hash out under a short lock and verifies OUTSIDE it; settings PUT
|
||||
dupes the new hash with gpa BEFORE the transaction, installs + clears sessions after
|
||||
commit; `sessionAuth` gates on `live_hash.enabled()`, so a first password set via PUT
|
||||
locks routes without restart. Boot hash borrows the config arena (owned=false);
|
||||
replacements are gpa-owned, freed on the next install or deinit (app.zig defer /
|
||||
Bench.deinit / Env.destroy).
|
||||
- mutations.zig `checkUpstream`: candidate validated beside a companion enabled upstream
|
||||
(URL collision-avoided), so disabled upstreams are creatable. upstreams.zig
|
||||
`applyUpdate` gained the missing last-enabled guard: disabling the last enabled
|
||||
upstream is 409 (delete path already had it; create needs none).
|
||||
- clients.zig: prefixes canonicalized via platform/address.zig (parse zeroes host bits;
|
||||
RFC 5952 text) before a set-level duplicate check (409, constraint-map ruling) and
|
||||
whole-list validation through the real validate.validate; canonical text is stored.
|
||||
- api_limiter.zig: full table fails closed — `bucketLocked` first evicts a bucket that
|
||||
refills to capacity with sse==0 (behaviorally identical to fresh; sweep's idle window
|
||||
is churn hysteresis, not correctness), else `check` refuses with retry_after and
|
||||
`tryAcquireSse` returns false. `untracked` is now a subset of `refused`.
|
||||
- static.zig: `acceptsGzip` parses every entry per the header grammar (specific gzip
|
||||
beats wildcard; malformed q = entry unusable). Dev mode realpath-checks containment of
|
||||
target AND index fallback (no-follow-on-open would miss symlinked intermediate
|
||||
directories).
|
||||
- web_integration_test.zig: drift guard (a) requires the method key inside the specific
|
||||
path's yaml block; a negative test doctors the yaml (method swap between two paths)
|
||||
and proves the guard bites.
|
||||
- local.zig: all six mutation paths hold `config_lock` across write AND rebuild+swap, so LocalTables swaps publish in database-write order (deadlock-checked: no callee takes the lock).
|
||||
- settings/auth/server/app: GET /api/settings reads under `config_lock`. New `auth.LiveHash` (mutex holder on `WebState.live_hash`, generation-checked — see round 2): login copies the hash out under a short lock and verifies OUTSIDE it; settings PUT dupes the new hash with gpa BEFORE the transaction, installs + clears sessions after commit; `sessionAuth` gates on `live_hash.enabled()`, so a first password set via PUT locks routes without restart. Boot hash borrows the config arena (owned=false); replacements are gpa-owned, freed on the next install or deinit (app.zig defer / Bench.deinit / Env.destroy).
|
||||
- mutations.zig `checkUpstream`: candidate validated beside a companion enabled upstream (URL collision-avoided), so disabled upstreams are creatable. upstreams.zig `applyUpdate` gained the missing last-enabled guard: disabling the last enabled upstream is 409 (delete path already had it; create needs none).
|
||||
- clients.zig: prefixes canonicalized via platform/address.zig (parse zeroes host bits; RFC 5952 text) before a set-level duplicate check (409, constraint-map ruling) and whole-list validation through the real validate.validate; canonical text is stored.
|
||||
- api_limiter.zig: full table fails closed — `bucketLocked` first evicts a bucket that refills to capacity with sse==0 (behaviorally identical to fresh; sweep's idle window is churn hysteresis, not correctness), else `check` refuses with retry_after and `tryAcquireSse` returns false. `untracked` is now a subset of `refused`.
|
||||
- static.zig: `acceptsGzip` parses every entry per the header grammar (specific gzip beats wildcard; malformed q = entry unusable). Dev mode realpath-checks containment of target AND index fallback (no-follow-on-open would miss symlinked intermediate directories).
|
||||
- web_integration_test.zig: drift guard (a) requires the method key inside the specific path's yaml block; a negative test doctors the yaml (method swap between two paths) and proves the guard bites.
|
||||
|
||||
Round 2: 1 important — login could copy the old hash, race a password-change PUT
|
||||
(install + clearAll), finish argon2 verification, and mint a session with the revoked
|
||||
password. Fixed via a generation counter on LiveHash: applyLogin verifies the copy
|
||||
outside any lock, then `confirmSession` checks the generation and inserts the session
|
||||
digest under LiveHash.mutex (lock order: live_hash → sessions, single direction at every
|
||||
site); a changed generation denies the login.
|
||||
Round 2: 1 important — login could copy the old hash, race a password-change PUT (install + clearAll), finish argon2 verification, and mint a session with the revoked password. Fixed via a generation counter on LiveHash: applyLogin verifies the copy outside any lock, then `confirmSession` checks the generation and inserts the session digest under LiveHash.mutex (lock order: live_hash → sessions, single direction at every site); a changed generation denies the login.
|
||||
|
||||
Round 3: 2 minors, both fixed. (a) install-then-clearAll had a gap where a legitimate
|
||||
new-password cookie could be minted and then wiped — `install` is replaced by
|
||||
`installAndRevoke(io, gpa, sessions, new_hash)`: swap, generation bump, and nested
|
||||
clearAll in one LiveHash-ordered operation; a confirm either precedes it (wiped) or
|
||||
follows it (stale generation, denied). (b) confirmSession no longer holds LiveHash.mutex
|
||||
across entropy/clock work: token (randomSecure, zeroed on exit) and timestamp are
|
||||
generated before the lock; only the generation check + `createWithToken` digest insert
|
||||
run under the ordered locks. `Sessions.create` removed (createWithToken is the seam).
|
||||
Round 3: 2 minors, both fixed. (a) install-then-clearAll had a gap where a legitimate new-password cookie could be minted and then wiped — `install` is replaced by `installAndRevoke(io, gpa, sessions, new_hash)`: swap, generation bump, and nested clearAll in one LiveHash-ordered operation; a confirm either precedes it (wiped) or follows it (stale generation, denied). (b) confirmSession no longer holds LiveHash.mutex across entropy/clock work: token (randomSecure, zeroed on exit) and timestamp are generated before the lock; only the generation check + `createWithToken` digest insert run under the ordered locks. `Sessions.create` removed (createWithToken is the seam).
|
||||
|
||||
Round 4: no findings.
|
||||
|
||||
## Anti-requirements
|
||||
|
||||
- No React/SPA content (milestone 9). No DoH/DoT server or certs/reload (Phase 9). No
|
||||
docs rendering (Phase 10). No WebSocket. No HTTP/2, no TLS on the web port. No external
|
||||
schema validator or yaml parser. No new DB schema migration without an explicit
|
||||
orchestrator ruling (27). No pause persistence. No per-request timeouts. No changes to
|
||||
dns/, filter/matcher, cache, or the DNS pipeline order.
|
||||
- No React/SPA content (milestone 9). No DoH/DoT server or certs/reload (Phase 9). No docs rendering (Phase 10). No WebSocket. No HTTP/2, no TLS on the web port. No external schema validator or yaml parser. No new DB schema migration without an explicit orchestrator ruling (27). No pause persistence. No per-request timeouts. No changes to dns/, filter/matcher, cache, or the DNS pipeline order.
|
||||
|
||||
+74
-318
@@ -1,365 +1,140 @@
|
||||
# Milestone 9: React SPA admin UI (PLAN §14, §3.14)
|
||||
|
||||
Goal: the complete admin UI — ten pages per PLAN.md:560, login, pause control, restart
|
||||
banner — built with Vite + React + TypeScript + Tailwind + TanStack Router/Query,
|
||||
embedded in the binary via the existing `-Dweb-dist` pipeline, with a frontend CI job.
|
||||
Goal: the complete admin UI — ten pages per PLAN.md:560, login, pause control, restart banner — built with Vite + React + TypeScript + Tailwind + TanStack Router/Query, embedded in the binary via the existing `-Dweb-dist` pipeline, with a frontend CI job.
|
||||
|
||||
Ground truth: PLAN.md:118-120 (auth UX), :138-140 (stack + embedding), :145/:579 (CI),
|
||||
:454-456 (live view), :531 (restart banner), :560-562 (pages + requirements), :639 (size
|
||||
budget), :641 (upstream health in UI); specs/milestone-8.md W8/W9 As-built (asset
|
||||
pipeline, SPA fallback, --web-dev); src/web/openapi.yaml (the API contract — authorative
|
||||
for every body shape; do NOT guess fields, read it).
|
||||
budget), :641 (upstream health in UI); specs/milestone-8.md W8/W9 As-built (asset pipeline, SPA fallback, --web-dev); src/web/openapi.yaml (the API contract — authorative for every body shape; do NOT guess fields, read it).
|
||||
|
||||
## Rulings (binding)
|
||||
|
||||
1. **Stack per PLAN.md:138, not house style**: Vite + React 19 + TypeScript + Tailwind
|
||||
v4 (CSS-first config, no tailwind.config.js) + TanStack Router + TanStack Query. The
|
||||
reference repos use StyleX and no TanStack; PLAN is the source of truth here. SPA, no
|
||||
SSR, base `/`.
|
||||
2. **Location**: everything under `web/` (PLAN.md:234): `web/package.json`, `web/src/`,
|
||||
`web/index.html`, build output `web/dist/` (gitignored). `web/dist-placeholder/`
|
||||
stays untouched and stays the `-Dweb-dist` default.
|
||||
3. **Toolchain**: node >= 24 (`engines`), npm with committed `web/package-lock.json`.
|
||||
Dependencies exact-pinned (house style). Checks: `tsc --noEmit`, Prettier (house
|
||||
config: tabs, tabWidth 4, printWidth 120, semi, double quotes, trailing commas all),
|
||||
oxlint (zero-config), vitest + @testing-library/react + jsdom for tests. No ESLint,
|
||||
no Biome, no msw — tests stub `fetch`/`EventSource` by hand.
|
||||
4. **No charting dependency.** The dashboard chart is a hand-rolled SVG component
|
||||
(stacked bars for queries/blocked/cached over the fixed zero-filled buckets, axis
|
||||
labels, hover tooltip via native `<title>` + a hover state). The data is small
|
||||
(60-168 buckets) and fixed-shape; a chart library is a liability with no payoff.
|
||||
This is complete, not a stub: axes, tooltip, empty state, responsive width.
|
||||
5. **Router**: code-based route tree (no file-based codegen plugin). Every data route
|
||||
uses a TanStack Router loader that primes TanStack Query (`ensureQueryData`) —
|
||||
PLAN.md:562 "route loaders for initial fetch". Pending/error components on every
|
||||
route; TanStack Query handles cache/retry (no retry on 4xx; retry 429 after
|
||||
Retry-After).
|
||||
6. **API layer**: one `src/lib/api.ts` typed client over `fetch` with
|
||||
`credentials: "same-origin"`. Types in `src/lib/types.ts` transcribed by hand from
|
||||
openapi.yaml (snake_case preserved; no codegen dependency). Error model:
|
||||
`ApiError{status, message, retryAfter?}` from the `{error}` envelope; 503 rendered
|
||||
distinctly from 500 ("server starting/degraded" vs "internal error").
|
||||
7. **Auth flow**: `/login` route outside the app shell. On any 401 the query layer
|
||||
redirects to `/login?redirect=<path>`. Login POST with `auth_required=false` in the
|
||||
response → auth is off → navigate straight in (the login page short-circuits by
|
||||
probing once). Logout button in the shell (hidden when auth is off). Session state
|
||||
lives in a tiny auth store fed by responses, not a poll. Works with auth on AND off
|
||||
(PLAN.md:562) — with auth off the SPA never shows login.
|
||||
8. **Pause control is global**: a shell-header widget (pause/resume with duration
|
||||
presets 60s/5m/30m/indefinite, countdown from `until`, `paused` disambiguates null
|
||||
`until`). Not a page.
|
||||
9. **Live log page**: `EventSource` on `/api/queries/live`; ring buffer of 500 rows in
|
||||
memory, newest first; rows keyed by a monotonically increasing client counter (the
|
||||
SSE payload has NO id). On `error`/close the browser auto-reconnects (server sends
|
||||
`retry: 3000`); on reconnect the page re-syncs the gap via `GET /api/queries`
|
||||
(since = last seen ts) and shows a "stream resumed, N missed" notice. Pause-stream
|
||||
button (client-side freeze) included. 429 from the SSE cap → visible "too many live
|
||||
viewers" state, retry button.
|
||||
10. **Query log page**: keyset pagination exactly per contract (`limit`, `before`,
|
||||
follow `next_before` until null; newest first). Filters: domain substring, client
|
||||
exact, blocked tri-state, since/until (datetime-local inputs → unix seconds).
|
||||
"Load more" appends; filter change resets the cursor.
|
||||
11. **Settings page**: sections rendered from the GET envelope; a diff-based PUT sends
|
||||
ONLY changed fields (partial patch per contract). Every key is restart-required
|
||||
today: after a successful PUT of anything except `web.password`, set a persistent
|
||||
"restart to apply" banner (dismiss resets on next change; PLAN.md:531).
|
||||
`web.password` is a write-only field with confirm input; on change the API kills
|
||||
all sessions — the SPA expects the next request to 401 and routes to login.
|
||||
`web.auth_enabled` is read-only derived. `web.password_hash` is never sent.
|
||||
12. **Blocklists page** shows per-source `skipped_regex_count` (PLAN.md:38) and source
|
||||
status (state, last_success, counts, last_error). "Update now" calls
|
||||
`POST /api/blocklists/update`, REPLACES list state from the 202 snapshot, disables
|
||||
the button while in flight.
|
||||
13. **Dashboard**: stats totals + timeseries chart (period picker 1h/24h/7d/30d),
|
||||
upstream health table (PLAN.md:641), disk card from `/api/health`
|
||||
(state/free/db/log bytes) with the warning banner when `disk.state != "ok"`
|
||||
(PLAN.md:465), plus queries_dropped/writer_failed indicators. Auto-refresh via
|
||||
Query `refetchInterval` 30s (health 10s).
|
||||
14. **Groups page** includes the group↔sources assignment editor
|
||||
(`PUT /api/groups/{id}/sources`, full-set checkboxes) and safe_search toggle.
|
||||
Default group (id 1) shows but blocks rename/delete client-side too (server 409s).
|
||||
15. **Clients page**: table of all clients (`hand_edited` badge), edit name/group, no
|
||||
create (rows appear from DNS activity — say so in the empty state), delete with
|
||||
"re-materializes on next query" note. Client-prefixes editor on the same page:
|
||||
whole-list editing per the PUT contract.
|
||||
16. **Local DNS page**: two tabs (records, forward zones), CRUD forms per contract.
|
||||
**Domain lookup page**: domain + group select → renders the full pipeline verdict
|
||||
(local_records, forward_zone, blocked/reason/matched/source_url,
|
||||
safe_search_rewrite).
|
||||
17. **Errors and loading**: every route has skeleton/pending UI and an error boundary
|
||||
with the ApiError message + retry (PLAN.md:562). Mutations surface 400/409 messages
|
||||
inline at the form, 429 with countdown, 401 via the global redirect.
|
||||
18. **Responsive**: sidebar nav collapses to a top bar + drawer under `md:`; tables get
|
||||
`overflow-x-auto` wrappers; the dashboard grid stacks. Desktop and mobile per
|
||||
PLAN.md:562 — no separate mobile pages.
|
||||
19. **Formatting/l10n**: timestamps rendered in the browser locale from unix seconds
|
||||
(`Intl.DateTimeFormat`), byte counts humanized (KiB/MiB/GiB), µs durations shown as
|
||||
ms with one decimal. One `src/lib/format.ts`, tested.
|
||||
20. **Size budget**: PLAN.md:639 — stripped static binary < 15 MB with assets (< 10 MB
|
||||
without → SPA budget ≈ 5 MB embedded, plain + .gz both count). React+TanStack+
|
||||
Tailwind lands far under that; CI asserts the final cross binaries < 15 MB.
|
||||
21. **Zig side is frozen.** No changes to src/, build.zig, tools/, or openapi.yaml. If
|
||||
the SPA reveals an API bug, STOP and report — do not work around silently. The only
|
||||
Zig-adjacent deliverables are ci.yml additions and `.gitignore` entries.
|
||||
22. **CI**: new `frontend` job (setup-node@v4, `NODE_VERSION: "24"` env pin,
|
||||
`cache: npm`, `cache-dependency-path: web/package-lock.json`; `npm ci` →
|
||||
`prettier --check` → oxlint → `tsc --noEmit` → `vitest run` → `vite build`),
|
||||
uploading `web/dist` as an artifact is NOT needed — instead the existing `cross`
|
||||
job gains: setup-node, `npm ci && npm run build` (prefix web), then
|
||||
`zig build cross -Dweb-dist=web/dist`, then the existing static assert plus a
|
||||
size assert (< 15 MB per exe). The `test` job stays Zig-only.
|
||||
23. **Accessibility floor**: semantic elements, labeled inputs, focus-visible styles,
|
||||
buttons not divs. No ARIA deep-dive beyond what semantics give.
|
||||
24. **No new pages, no dark-mode toggle bikeshed** (Tailwind default palette, system
|
||||
`prefers-color-scheme` via CSS only), no i18n framework, no state library beyond
|
||||
TanStack Query + two tiny stores (auth, restart-banner) in React context.
|
||||
1. **Stack per PLAN.md:138, not house style**: Vite + React 19 + TypeScript + Tailwind v4 (CSS-first config, no tailwind.config.js) + TanStack Router + TanStack Query. The reference repos use StyleX and no TanStack; PLAN is the source of truth here. SPA, no SSR, base `/`.
|
||||
2. **Location**: everything under `web/` (PLAN.md:234): `web/package.json`, `web/src/`, `web/index.html`, build output `web/dist/` (gitignored). `web/dist-placeholder/` stays untouched and stays the `-Dweb-dist` default.
|
||||
3. **Toolchain**: node >= 24 (`engines`), npm with committed `web/package-lock.json`. Dependencies exact-pinned (house style). Checks: `tsc --noEmit`, Prettier (house config: tabs, tabWidth 4, printWidth 120, semi, double quotes, trailing commas all), oxlint (zero-config), vitest + @testing-library/react + jsdom for tests. No ESLint, no Biome, no msw — tests stub `fetch`/`EventSource` by hand.
|
||||
4. **No charting dependency.** The dashboard chart is a hand-rolled SVG component (stacked bars for queries/blocked/cached over the fixed zero-filled buckets, axis labels, hover tooltip via native `<title>` + a hover state). The data is small (60-168 buckets) and fixed-shape; a chart library is a liability with no payoff. This is complete, not a stub: axes, tooltip, empty state, responsive width.
|
||||
5. **Router**: code-based route tree (no file-based codegen plugin). Every data route uses a TanStack Router loader that primes TanStack Query (`ensureQueryData`) — PLAN.md:562 "route loaders for initial fetch". Pending/error components on every route; TanStack Query handles cache/retry (no retry on 4xx; retry 429 after Retry-After).
|
||||
6. **API layer**: one `src/lib/api.ts` typed client over `fetch` with `credentials: "same-origin"`. Types in `src/lib/types.ts` transcribed by hand from openapi.yaml (snake_case preserved; no codegen dependency). Error model: `ApiError{status, message, retryAfter?}` from the `{error}` envelope; 503 rendered distinctly from 500 ("server starting/degraded" vs "internal error").
|
||||
7. **Auth flow**: `/login` route outside the app shell. On any 401 the query layer redirects to `/login?redirect=<path>`. Login POST with `auth_required=false` in the response → auth is off → navigate straight in (the login page short-circuits by probing once). Logout button in the shell (hidden when auth is off). Session state lives in a tiny auth store fed by responses, not a poll. Works with auth on AND off (PLAN.md:562) — with auth off the SPA never shows login.
|
||||
8. **Pause control is global**: a shell-header widget (pause/resume with duration presets 60s/5m/30m/indefinite, countdown from `until`, `paused` disambiguates null `until`). Not a page.
|
||||
9. **Live log page**: `EventSource` on `/api/queries/live`; ring buffer of 500 rows in memory, newest first; rows keyed by a monotonically increasing client counter (the SSE payload has NO id). On `error`/close the browser auto-reconnects (server sends `retry: 3000`); on reconnect the page re-syncs the gap via `GET /api/queries` (since = last seen ts) and shows a "stream resumed, N missed" notice. Pause-stream button (client-side freeze) included. 429 from the SSE cap → visible "too many live viewers" state, retry button.
|
||||
10. **Query log page**: keyset pagination exactly per contract (`limit`, `before`, follow `next_before` until null; newest first). Filters: domain substring, client exact, blocked tri-state, since/until (datetime-local inputs → unix seconds). "Load more" appends; filter change resets the cursor.
|
||||
11. **Settings page**: sections rendered from the GET envelope; a diff-based PUT sends ONLY changed fields (partial patch per contract). Every key is restart-required today: after a successful PUT of anything except `web.password`, set a persistent "restart to apply" banner (dismiss resets on next change; PLAN.md:531). `web.password` is a write-only field with confirm input; on change the API kills all sessions — the SPA expects the next request to 401 and routes to login. `web.auth_enabled` is read-only derived. `web.password_hash` is never sent.
|
||||
12. **Blocklists page** shows per-source `skipped_regex_count` (PLAN.md:38) and source status (state, last_success, counts, last_error). "Update now" calls `POST /api/blocklists/update`, REPLACES list state from the 202 snapshot, disables the button while in flight.
|
||||
13. **Dashboard**: stats totals + timeseries chart (period picker 1h/24h/7d/30d), upstream health table (PLAN.md:641), disk card from `/api/health` (state/free/db/log bytes) with the warning banner when `disk.state != "ok"` (PLAN.md:465), plus queries_dropped/writer_failed indicators. Auto-refresh via Query `refetchInterval` 30s (health 10s).
|
||||
14. **Groups page** includes the group↔sources assignment editor (`PUT /api/groups/{id}/sources`, full-set checkboxes) and safe_search toggle. Default group (id 1) shows but blocks rename/delete client-side too (server 409s).
|
||||
15. **Clients page**: table of all clients (`hand_edited` badge), edit name/group, no create (rows appear from DNS activity — say so in the empty state), delete with "re-materializes on next query" note. Client-prefixes editor on the same page: whole-list editing per the PUT contract.
|
||||
16. **Local DNS page**: two tabs (records, forward zones), CRUD forms per contract. **Domain lookup page**: domain + group select → renders the full pipeline verdict (local_records, forward_zone, blocked/reason/matched/source_url, safe_search_rewrite).
|
||||
17. **Errors and loading**: every route has skeleton/pending UI and an error boundary with the ApiError message + retry (PLAN.md:562). Mutations surface 400/409 messages inline at the form, 429 with countdown, 401 via the global redirect.
|
||||
18. **Responsive**: sidebar nav collapses to a top bar + drawer under `md:`; tables get `overflow-x-auto` wrappers; the dashboard grid stacks. Desktop and mobile per PLAN.md:562 — no separate mobile pages.
|
||||
19. **Formatting/l10n**: timestamps rendered in the browser locale from unix seconds (`Intl.DateTimeFormat`), byte counts humanized (KiB/MiB/GiB), µs durations shown as ms with one decimal. One `src/lib/format.ts`, tested.
|
||||
20. **Size budget**: PLAN.md:639 — stripped static binary < 15 MB with assets (< 10 MB without → SPA budget ≈ 5 MB embedded, plain + .gz both count). React+TanStack+ Tailwind lands far under that; CI asserts the final cross binaries < 15 MB.
|
||||
21. **Zig side is frozen.** No changes to src/, build.zig, tools/, or openapi.yaml. If the SPA reveals an API bug, STOP and report — do not work around silently. The only Zig-adjacent deliverables are ci.yml additions and `.gitignore` entries.
|
||||
22. **CI**: new `frontend` job (setup-node@v4, `NODE_VERSION: "24"` env pin, `cache: npm`, `cache-dependency-path: web/package-lock.json`; `npm ci` → `prettier --check` → oxlint → `tsc --noEmit` → `vitest run` → `vite build`), uploading `web/dist` as an artifact is NOT needed — instead the existing `cross` job gains: setup-node, `npm ci && npm run build` (prefix web), then `zig build cross -Dweb-dist=web/dist`, then the existing static assert plus a size assert (< 15 MB per exe). The `test` job stays Zig-only.
|
||||
23. **Accessibility floor**: semantic elements, labeled inputs, focus-visible styles, buttons not divs. No ARIA deep-dive beyond what semantics give.
|
||||
24. **No new pages, no dark-mode toggle bikeshed** (Tailwind default palette, system `prefers-color-scheme` via CSS only), no i18n framework, no state library beyond TanStack Query + two tiny stores (auth, restart-banner) in React context.
|
||||
|
||||
## Sessions
|
||||
|
||||
F1 first (scaffold), then F2 (api/lib) sequential on F1, then F3-F8 parallel (pages;
|
||||
disjoint files), then F9 (CI + embed + smoke) after all.
|
||||
F1 first (scaffold), then F2 (api/lib) sequential on F1, then F3-F8 parallel (pages; disjoint files), then F9 (CI + embed + smoke) after all.
|
||||
|
||||
---
|
||||
|
||||
## Session F1: scaffold + shell
|
||||
|
||||
Owns: `web/package.json`, `web/package-lock.json`, `web/index.html`,
|
||||
`web/vite.config.ts`, `web/tsconfig*.json`, `web/.oxlintrc.json` (only if needed),
|
||||
`web/src/main.tsx`, `web/src/routes.tsx` (route tree with lazy page imports and
|
||||
placeholder page stubs F3-F8 replace), `web/src/shell/` (layout, sidebar/topbar nav,
|
||||
`PauseWidget` SLOT — an import of `../features/pause/PauseWidget` that F8 fills; F1
|
||||
ships the real widget file with a disabled placeholder), `web/src/styles.css`
|
||||
(tailwind), `.gitignore` additions (web/dist, web/node_modules), Prettier config in
|
||||
package.json (house values), scripts: dev/build/typecheck/lint/format/test.
|
||||
Owns: `web/package.json`, `web/package-lock.json`, `web/index.html`, `web/vite.config.ts`, `web/tsconfig*.json`, `web/.oxlintrc.json` (only if needed), `web/src/main.tsx`, `web/src/routes.tsx` (route tree with lazy page imports and placeholder page stubs F3-F8 replace), `web/src/shell/` (layout, sidebar/topbar nav, `PauseWidget` SLOT — an import of `../features/pause/PauseWidget` that F8 fills; F1 ships the real widget file with a disabled placeholder), `web/src/styles.css` (tailwind), `.gitignore` additions (web/dist, web/node_modules), Prettier config in package.json (house values), scripts: dev/build/typecheck/lint/format/test.
|
||||
|
||||
Vite: `@vitejs/plugin-react`, tailwind v4 via `@tailwindcss/vite`, build target
|
||||
baseline-widely-available (vite 8 default), no proxy needed for build; dev proxy
|
||||
`/api` + `/metrics` → `http://127.0.0.1:8080` for `vite dev` against a running nxdns.
|
||||
Route tree: `/login` bare; shell routes `/`, `/queries`, `/live`, `/clients`,
|
||||
`/groups`, `/blocklists`, `/rules`, `/local-dns`, `/lookup`, `/settings`.
|
||||
Vite: `@vitejs/plugin-react`, tailwind v4 via `@tailwindcss/vite`, build target baseline-widely-available (vite 8 default), no proxy needed for build; dev proxy `/api` + `/metrics` → `http://127.0.0.1:8080` for `vite dev` against a running nxdns. Route tree: `/login` bare; shell routes `/`, `/queries`, `/live`, `/clients`, `/groups`, `/blocklists`, `/rules`, `/local-dns`, `/lookup`, `/settings`.
|
||||
|
||||
Acceptance: `npm ci && npm run build` produces `web/dist` with `/index.html`;
|
||||
typecheck/lint/format clean; placeholder pages render; nav works with keyboard.
|
||||
Acceptance: `npm ci && npm run build` produces `web/dist` with `/index.html`; typecheck/lint/format clean; placeholder pages render; nav works with keyboard.
|
||||
|
||||
### F1 As built
|
||||
|
||||
Pinned: react 19.2.8, @tanstack/react-router 1.170.18, @tanstack/react-query 5.101.4,
|
||||
vite 8.1.5, typescript 6.0.3, tailwindcss 4.3.3 (@tailwindcss/vite), vitest 4.1.10,
|
||||
@testing-library/react 16.3.2, jsdom 29.1.1, prettier 3.9.6, oxlint 1.75.0. All exact.
|
||||
~/.npmrc enforces min-release-age=7 (newest gate-clearing versions chosen) and
|
||||
ignore-scripts=true (works; native bins are optionalDependencies). TS6 deprecates
|
||||
baseUrl — tsconfig.app.json uses `paths` without it. Dist: 344K (main chunk 278.5 kB /
|
||||
88.2 kB gz, 11 lazy page chunks, CSS 8.8 kB). `zig build -Dweb-dist=web/dist` exit 0.
|
||||
Pinned: react 19.2.8, @tanstack/react-router 1.170.18, @tanstack/react-query 5.101.4, vite 8.1.5, typescript 6.0.3, tailwindcss 4.3.3 (@tailwindcss/vite), vitest 4.1.10, @testing-library/react 16.3.2, jsdom 29.1.1, prettier 3.9.6, oxlint 1.75.0. All exact. ~/.npmrc enforces min-release-age=7 (newest gate-clearing versions chosen) and ignore-scripts=true (works; native bins are optionalDependencies). TS6 deprecates baseUrl — tsconfig.app.json uses `paths` without it. Dist: 344K (main chunk 278.5 kB / 88.2 kB gz, 11 lazy page chunks, CSS 8.8 kB). `zig build -Dweb-dist=web/dist` exit 0.
|
||||
- Alias `@/*` → `web/src/*` (tsconfig paths + vite resolve.alias).
|
||||
- Router: code-based in src/routes.tsx; `createAppRouter(history?)` exported (tests
|
||||
pass createMemoryHistory); `Register` declared so `Link to` is typed; pages wired
|
||||
via lazyRouteComponent at stable paths — every page keeps a default export at its
|
||||
path; page sessions never edit routes.tsx. Shell = pathless layout route id "shell";
|
||||
/login hangs off root outside it; defaultPreload "intent".
|
||||
- Shell: AppShell renders PauseWidget (default export, no props) from
|
||||
features/pause/PauseWidget, header right; F8 replaces the file in place.
|
||||
VersionFooter inside AppShell.tsx (F2 wires /api/version). Restart banner has no
|
||||
premade slot — F8 mounts it in AppShell (sequential edit). Nav: aria-current +
|
||||
activeProps; drawer under md: with aria-expanded/controls.
|
||||
- Vitest lives in vite.config.ts (jsdom, globals: true; tsconfig types include
|
||||
vitest/globals). lint = `oxlint src vite.config.ts`; prettier ignores dist/,
|
||||
dist-placeholder/, package-lock.json (.prettierignore).
|
||||
- Page filenames: QueryLogPage/LiveLogPage per F4 naming; LoginPage placeholder in
|
||||
src/auth/ (F2 replaces).
|
||||
- Router: code-based in src/routes.tsx; `createAppRouter(history?)` exported (tests pass createMemoryHistory); `Register` declared so `Link to` is typed; pages wired via lazyRouteComponent at stable paths — every page keeps a default export at its path; page sessions never edit routes.tsx. Shell = pathless layout route id "shell"; /login hangs off root outside it; defaultPreload "intent".
|
||||
- Shell: AppShell renders PauseWidget (default export, no props) from features/pause/PauseWidget, header right; F8 replaces the file in place. VersionFooter inside AppShell.tsx (F2 wires /api/version). Restart banner has no premade slot — F8 mounts it in AppShell (sequential edit). Nav: aria-current + activeProps; drawer under md: with aria-expanded/controls.
|
||||
- Vitest lives in vite.config.ts (jsdom, globals: true; tsconfig types include vitest/globals). lint = `oxlint src vite.config.ts`; prettier ignores dist/, dist-placeholder/, package-lock.json (.prettierignore).
|
||||
- Page filenames: QueryLogPage/LiveLogPage per F4 naming; LoginPage placeholder in src/auth/ (F2 replaces).
|
||||
|
||||
---
|
||||
|
||||
## Session F2: API layer + auth + query plumbing
|
||||
|
||||
Owns: `web/src/lib/` (`api.ts`, `types.ts`, `format.ts`, `queryClient.ts`,
|
||||
`queries.ts` — queryOptions per resource, mutation helpers with invalidation),
|
||||
`web/src/auth/` (store, `LoginPage`, 401 redirect wiring), tests for api/format/
|
||||
pagination/settings-diff helpers.
|
||||
Owns: `web/src/lib/` (`api.ts`, `types.ts`, `format.ts`, `queryClient.ts`, `queries.ts` — queryOptions per resource, mutation helpers with invalidation), `web/src/auth/` (store, `LoginPage`, 401 redirect wiring), tests for api/format/ pagination/settings-diff helpers.
|
||||
|
||||
types.ts transcribed from src/web/openapi.yaml — every shape the pages consume
|
||||
(QueryRow, QueriesPage, StatsTotals, Timeseries, Lookup, UpstreamHealth, Health,
|
||||
Version, Group, Blocklist, SourceStatus, Rule, LocalRecord, ForwardZone, Client,
|
||||
ClientPrefix, Upstream, SettingsEnvelope, Pause, Login). ApiError per ruling 6;
|
||||
401 hook per ruling 7; 429 retry per ruling 5; settings diff builder per ruling 11
|
||||
(pure function, tested).
|
||||
types.ts transcribed from src/web/openapi.yaml — every shape the pages consume (QueryRow, QueriesPage, StatsTotals, Timeseries, Lookup, UpstreamHealth, Health, Version, Group, Blocklist, SourceStatus, Rule, LocalRecord, ForwardZone, Client, ClientPrefix, Upstream, SettingsEnvelope, Pause, Login). ApiError per ruling 6; 401 hook per ruling 7; 429 retry per ruling 5; settings diff builder per ruling 11 (pure function, tested).
|
||||
|
||||
Acceptance: vitest green; typecheck clean; login/logout round trip works against a
|
||||
live `nxdns run` (manual smoke; document the transcript).
|
||||
Acceptance: vitest green; typecheck clean; login/logout round trip works against a live `nxdns run` (manual smoke; document the transcript).
|
||||
|
||||
### F2 As built
|
||||
|
||||
lib/{types,api,queryClient,queries,format,settingsDiff}.ts + tests (22 green);
|
||||
auth/{store.tsx,LoginPage.tsx}; routes.tsx gained context+loaders+default
|
||||
pending/error components; main.tsx providers; AppShell gained logout button + live
|
||||
VersionFooter. Live-server smoke verified every consumed shape with auth on AND off;
|
||||
no API bugs. Main chunk 319 kB (100 kB gz).
|
||||
- api.ts: `request<T>` core (same-origin, 204→void, `{error}` envelope, Retry-After on
|
||||
429), `requestText`, functions for all 55 route-method pairs, list envelopes
|
||||
unwrapped to arrays, `liveQueriesUrl` for F4.
|
||||
- queryClient: staleTime 30s; no retry on 4xx except 429 (max 2, delay=retryAfter);
|
||||
QueryCache+MutationCache onError → `/login?redirect=<path+search>` on 401 (skipped
|
||||
on /login).
|
||||
- queries.ts factories: healthQuery (10s refetch), versionQuery, statsQuery/
|
||||
timeseriesQuery(period), upstreamHealthQuery (30s), queriesQuery(filter),
|
||||
lookupQuery(domain, groupId?), groupsQuery, groupSourcesQuery(id), blocklistsQuery,
|
||||
rulesQuery, localRecordsQuery, forwardZonesQuery, clientsQuery, clientPrefixesQuery,
|
||||
upstreamsQuery, pauseQuery, settingsQuery. Mutations: `xxxMutation(queryClient)` →
|
||||
useMutation options; invalidation map in the F2 report; NOTE
|
||||
`blocklistsUpdateNowMutation` seeds `queryKeys.blocklistSources` from the 202
|
||||
snapshot — the ONLY feed for source status (no GET exists); F6 reads that key.
|
||||
- Auth store: `useAuth()` → {authRequired: bool|null, probe(), login(password),
|
||||
logout()}. Probe = POST login with empty password (no status GET), StrictMode-deduped;
|
||||
authRequired mirrored in sessionStorage `nxdns_auth_required`.
|
||||
- Loaders prime: dashboard stats+timeseries("24h")+health+upstreamHealth; queries
|
||||
`queriesQuery({})` (F4 "load more" calls api.getQueries imperatively and appends);
|
||||
clients clients+prefixes+groups; groups groups+blocklists; rules rules+groups;
|
||||
local-dns records+zones; lookup groups; settings settings; live none. Pages use the
|
||||
same factory the loader primed (useSuspenseQuery/useQuery).
|
||||
- Router error component is ApiError-aware (503 "starting or degraded", 429 countdown,
|
||||
≥500 generic); Retry = router.invalidate().
|
||||
- Page-test pattern: wrap in AuthProvider+QueryClientProvider, pass the same qc to
|
||||
`createAppRouter(history, qc)`, stub fetch per-URL (see AppShell.test.tsx).
|
||||
- api.ts sends `{}` on bodyless POSTs (logout, blocklists/update) — belt-and-braces
|
||||
over the W9 server fix.
|
||||
lib/{types,api,queryClient,queries,format,settingsDiff}.ts + tests (22 green); auth/{store.tsx,LoginPage.tsx}; routes.tsx gained context+loaders+default pending/error components; main.tsx providers; AppShell gained logout button + live VersionFooter. Live-server smoke verified every consumed shape with auth on AND off; no API bugs. Main chunk 319 kB (100 kB gz).
|
||||
- api.ts: `request<T>` core (same-origin, 204→void, `{error}` envelope, Retry-After on 429), `requestText`, functions for all 55 route-method pairs, list envelopes unwrapped to arrays, `liveQueriesUrl` for F4.
|
||||
- queryClient: staleTime 30s; no retry on 4xx except 429 (max 2, delay=retryAfter); QueryCache+MutationCache onError → `/login?redirect=<path+search>` on 401 (skipped on /login).
|
||||
- queries.ts factories: healthQuery (10s refetch), versionQuery, statsQuery/ timeseriesQuery(period), upstreamHealthQuery (30s), queriesQuery(filter), lookupQuery(domain, groupId?), groupsQuery, groupSourcesQuery(id), blocklistsQuery, rulesQuery, localRecordsQuery, forwardZonesQuery, clientsQuery, clientPrefixesQuery, upstreamsQuery, pauseQuery, settingsQuery. Mutations: `xxxMutation(queryClient)` → useMutation options; invalidation map in the F2 report; NOTE `blocklistsUpdateNowMutation` seeds `queryKeys.blocklistSources` from the 202 snapshot — the ONLY feed for source status (no GET exists); F6 reads that key.
|
||||
- Auth store: `useAuth()` → {authRequired: bool|null, probe(), login(password), logout()}. Probe = POST login with empty password (no status GET), StrictMode-deduped; authRequired mirrored in sessionStorage `nxdns_auth_required`.
|
||||
- Loaders prime: dashboard stats+timeseries("24h")+health+upstreamHealth; queries `queriesQuery({})` (F4 "load more" calls api.getQueries imperatively and appends); clients clients+prefixes+groups; groups groups+blocklists; rules rules+groups; local-dns records+zones; lookup groups; settings settings; live none. Pages use the same factory the loader primed (useSuspenseQuery/useQuery).
|
||||
- Router error component is ApiError-aware (503 "starting or degraded", 429 countdown, ≥500 generic); Retry = router.invalidate().
|
||||
- Page-test pattern: wrap in AuthProvider+QueryClientProvider, pass the same qc to `createAppRouter(history, qc)`, stub fetch per-URL (see AppShell.test.tsx).
|
||||
- api.ts sends `{}` on bodyless POSTs (logout, blocklists/update) — belt-and-braces over the W9 server fix.
|
||||
|
||||
---
|
||||
|
||||
## Session F3: Dashboard + SVG chart
|
||||
|
||||
Owns: `web/src/features/dashboard/` (page, StatCards, TimeseriesChart (SVG, ruling 4),
|
||||
UpstreamHealthTable, DiskCard, HealthBanners) + chart unit tests (bucket→bar math,
|
||||
empty state).
|
||||
Owns: `web/src/features/dashboard/` (page, StatCards, TimeseriesChart (SVG, ruling 4), UpstreamHealthTable, DiskCard, HealthBanners) + chart unit tests (bucket→bar math, empty state).
|
||||
|
||||
## Session F4: Query log + Live log
|
||||
|
||||
Owns: `web/src/features/queries/` (QueryLogPage, filters, cursor pagination per ruling
|
||||
10) and `web/src/features/live/` (LiveLogPage per ruling 9: EventSource wrapper hook
|
||||
with injected EventSource for tests, ring buffer, gap re-sync, freeze, cap-hit state).
|
||||
Tests: ring buffer, gap-resync math, EventSource hook with a fake.
|
||||
10) and `web/src/features/live/` (LiveLogPage per ruling 9: EventSource wrapper hook with injected EventSource for tests, ring buffer, gap re-sync, freeze, cap-hit state). Tests: ring buffer, gap-resync math, EventSource hook with a fake.
|
||||
|
||||
## Session F5: Clients + Groups
|
||||
|
||||
Owns: `web/src/features/clients/` (table, edit dialog, prefixes editor per ruling 15)
|
||||
and `web/src/features/groups/` (list, create/rename/delete, safe_search, sources
|
||||
assignment per ruling 14).
|
||||
Owns: `web/src/features/clients/` (table, edit dialog, prefixes editor per ruling 15) and `web/src/features/groups/` (list, create/rename/delete, safe_search, sources assignment per ruling 14).
|
||||
|
||||
## Session F6: Blocklists + Rules
|
||||
|
||||
Owns: `web/src/features/blocklists/` (ruling 12) and `web/src/features/rules/`
|
||||
(table with group/kind/action columns, create form with pattern kind select, delete).
|
||||
Owns: `web/src/features/blocklists/` (ruling 12) and `web/src/features/rules/` (table with group/kind/action columns, create form with pattern kind select, delete).
|
||||
|
||||
## Session F7: Local DNS + Lookup
|
||||
|
||||
Owns: `web/src/features/local/` (records + zones tabs, CRUD forms) and
|
||||
`web/src/features/lookup/` (ruling 16).
|
||||
Owns: `web/src/features/local/` (records + zones tabs, CRUD forms) and `web/src/features/lookup/` (ruling 16).
|
||||
|
||||
## Session F8: Settings + Pause widget
|
||||
|
||||
Owns: `web/src/features/settings/` (ruling 11: section forms, diff PUT, restart
|
||||
banner store + banner component mounted in the shell via F1's slot, password flow)
|
||||
and `web/src/features/pause/PauseWidget.tsx` (REPLACES F1's placeholder; ruling 8).
|
||||
Tests: diff builder edge cases (nested partial, password only, no-op), banner logic.
|
||||
Owns: `web/src/features/settings/` (ruling 11: section forms, diff PUT, restart banner store + banner component mounted in the shell via F1's slot, password flow) and `web/src/features/pause/PauseWidget.tsx` (REPLACES F1's placeholder; ruling 8). Tests: diff builder edge cases (nested partial, password only, no-op), banner logic.
|
||||
|
||||
### F3-F8 As built (page wave)
|
||||
|
||||
All six sessions green; 100 web tests total after the wave; main chunk 318 kB
|
||||
(99 kB gz), pages as lazy chunks.
|
||||
- **F3 Dashboard**: chartLayout.ts pure layout (layoutTimeseries, niceTicks 1/2/5) +
|
||||
TimeseriesChart({data}) — self-measuring SVG stacked bars (blocked/cached/other,
|
||||
other = queries−blocked−cached clamped ≥0), sr-only data table, role="img".
|
||||
Axis ticks use a short Intl formatter (formatTime is tooltip/sr-only only).
|
||||
UpstreamHealthTable shows total_failures; success_rate ×100 (0..1 verified in
|
||||
pool.zig). HealthBanners: disk warn/critical (critical text per PLAN.md:466),
|
||||
writer_failed, queries_dropped>0, all role="alert". 12 tests.
|
||||
- **F4 Query/Live**: QueryLogPage exports QueryCells/QueryTableHead/BlockedCell,
|
||||
reused by LiveLogPage (both F4-owned). qtype.ts names 19 codes, TYPE<n> fallback.
|
||||
Filters in component state (not URL). ringBuffer.ts: 500 newest-first + mergeGap
|
||||
(dedup key ts|domain|client_ip|qtype|blocked|upstream — SSE rows have no id;
|
||||
same-second identical queries can over-dedup, accepted at household scale).
|
||||
useLiveQueries hook (injected EventSourceLike factory + fetchSince): states
|
||||
connecting/open/retrying/capped, gap re-sync getQueries({since, limit:500}) —
|
||||
gaps >500 replace the buffer wholesale; CAP_ERROR_THRESHOLD=3 consecutive errors
|
||||
→ capped state (EventSource cannot see 429; message says cap OR unreachable),
|
||||
open resets the counter. Freeze = display-only snapshot; the ring keeps filling;
|
||||
Resume swaps to current. 25 tests.
|
||||
- **F5 Clients/Groups**: prefixEditor.ts pure reducer (priority "" omitted → server
|
||||
default 100); baseline resets only on save/discard (dirty edits never clobbered by
|
||||
refetch). clientUpdateMutation takes {id, edit} (spec prompt said input — code
|
||||
wins). Groups: safe_search toggle PUTs with unchanged name (server 409s only on
|
||||
actual rename of default — verified in groups.zig); default group protections
|
||||
client-side + visible note; sourceSet.ts toggle/sameSet. 18 tests.
|
||||
- **F6 Blocklists/Rules**: rule enums confirmed kind=[exact,wildcard],
|
||||
action=[allow,block]. Status section reads queryKeys.blocklistSources via
|
||||
queryClient.getQueryData at render (no GET exists; mutation setQueryData
|
||||
re-renders); edit PUT preserves is_suggested. 3 tests.
|
||||
- **F7 Local/Lookup**: rtype=[A,AAAA,CNAME], ttl optional default 300. SPEC
|
||||
CORRECTION: Lookup.local_records is a BOOLEAN (ruling 16 said "list" loosely) —
|
||||
rendered Yes/No; verdict priority local > blocked > forwarded > allowed (pipeline
|
||||
order per lookup.zig), verdictOf exported. Local mutations live, no banner.
|
||||
4 tests.
|
||||
- **F8 Settings/Pause**: patchRequiresRestart(patch) exported — banner raised for
|
||||
any patch except bare web.password; restartBanner.ts is a useSyncExternalStore
|
||||
MODULE store (accepted deviation from ruling 24's "context": keeps the AppShell
|
||||
edit to import + mount). Number inputs NaN-guard blocks Save. PauseWidget:
|
||||
refetchInterval 5s only while paused, 1s countdown, formatRemaining exported.
|
||||
16 tests.
|
||||
- **Orchestrator integration**: the wave produced three near-identical inline-error
|
||||
components (clients+groups InlineError, local FormError) — hoisted post-wave to
|
||||
`src/lib/InlineError.tsx` (the richer variant: 400/409 verbatim, 429 countdown,
|
||||
503 distinct, non-Error → "could not reach"), seven imports updated, duplicates
|
||||
deleted. Dashboard's InlineError (query-level, onRetry) is a different component
|
||||
and stays local. Full chain re-verified green after the hoist.
|
||||
All six sessions green; 100 web tests total after the wave; main chunk 318 kB (99 kB gz), pages as lazy chunks.
|
||||
- **F3 Dashboard**: chartLayout.ts pure layout (layoutTimeseries, niceTicks 1/2/5) + TimeseriesChart({data}) — self-measuring SVG stacked bars (blocked/cached/other, other = queries−blocked−cached clamped ≥0), sr-only data table, role="img". Axis ticks use a short Intl formatter (formatTime is tooltip/sr-only only). UpstreamHealthTable shows total_failures; success_rate ×100 (0..1 verified in pool.zig). HealthBanners: disk warn/critical (critical text per PLAN.md:466), writer_failed, queries_dropped>0, all role="alert". 12 tests.
|
||||
- **F4 Query/Live**: QueryLogPage exports QueryCells/QueryTableHead/BlockedCell, reused by LiveLogPage (both F4-owned). qtype.ts names 19 codes, TYPE<n> fallback. Filters in component state (not URL). ringBuffer.ts: 500 newest-first + mergeGap (dedup key ts|domain|client_ip|qtype|blocked|upstream — SSE rows have no id; same-second identical queries can over-dedup, accepted at household scale). useLiveQueries hook (injected EventSourceLike factory + fetchSince): states connecting/open/retrying/capped, gap re-sync getQueries({since, limit:500}) — gaps >500 replace the buffer wholesale; CAP_ERROR_THRESHOLD=3 consecutive errors → capped state (EventSource cannot see 429; message says cap OR unreachable), open resets the counter. Freeze = display-only snapshot; the ring keeps filling; Resume swaps to current. 25 tests.
|
||||
- **F5 Clients/Groups**: prefixEditor.ts pure reducer (priority "" omitted → server default 100); baseline resets only on save/discard (dirty edits never clobbered by refetch). clientUpdateMutation takes {id, edit} (spec prompt said input — code wins). Groups: safe_search toggle PUTs with unchanged name (server 409s only on actual rename of default — verified in groups.zig); default group protections client-side + visible note; sourceSet.ts toggle/sameSet. 18 tests.
|
||||
- **F6 Blocklists/Rules**: rule enums confirmed kind=[exact,wildcard], action=[allow,block]. Status section reads queryKeys.blocklistSources via queryClient.getQueryData at render (no GET exists; mutation setQueryData re-renders); edit PUT preserves is_suggested. 3 tests.
|
||||
- **F7 Local/Lookup**: rtype=[A,AAAA,CNAME], ttl optional default 300. SPEC CORRECTION: Lookup.local_records is a BOOLEAN (ruling 16 said "list" loosely) — rendered Yes/No; verdict priority local > blocked > forwarded > allowed (pipeline order per lookup.zig), verdictOf exported. Local mutations live, no banner. 4 tests.
|
||||
- **F8 Settings/Pause**: patchRequiresRestart(patch) exported — banner raised for any patch except bare web.password; restartBanner.ts is a useSyncExternalStore MODULE store (accepted deviation from ruling 24's "context": keeps the AppShell edit to import + mount). Number inputs NaN-guard blocks Save. PauseWidget: refetchInterval 5s only while paused, 1s countdown, formatRemaining exported. 16 tests.
|
||||
- **Orchestrator integration**: the wave produced three near-identical inline-error components (clients+groups InlineError, local FormError) — hoisted post-wave to `src/lib/InlineError.tsx` (the richer variant: 400/409 verbatim, 429 countdown, 503 distinct, non-Error → "could not reach"), seven imports updated, duplicates deleted. Dashboard's InlineError (query-level, onRetry) is a different component and stays local. Full chain re-verified green after the hoist.
|
||||
|
||||
---
|
||||
|
||||
## Session F9: CI + embed + smoke (after F3-F8)
|
||||
|
||||
Owns: `.gitea/workflows/ci.yml` (ruling 22). Steps: verify `npm ci` reproducibility,
|
||||
frontend job, cross-job additions with `-Dweb-dist=web/dist` + size assert.
|
||||
Smoke (report transcript): `npm run build`; `zig build -Dweb-dist=web/dist`; boot;
|
||||
curl `/` returns the SPA index (not the placeholder); a deep link (`/settings`)
|
||||
returns 200 index; an asset serves gzip with ETag/304; login via browser-shaped curl
|
||||
flow still works; SIGTERM 0. Also `zig build test` and `-Dintegration` still green
|
||||
(the Zig tests embed the dist too — W10 static tests must still pass against the SPA
|
||||
dist; if one pins placeholder content, report it, do not edit Zig).
|
||||
Owns: `.gitea/workflows/ci.yml` (ruling 22). Steps: verify `npm ci` reproducibility, frontend job, cross-job additions with `-Dweb-dist=web/dist` + size assert. Smoke (report transcript): `npm run build`; `zig build -Dweb-dist=web/dist`; boot; curl `/` returns the SPA index (not the placeholder); a deep link (`/settings`) returns 200 index; an asset serves gzip with ETag/304; login via browser-shaped curl flow still works; SIGTERM 0. Also `zig build test` and `-Dintegration` still green (the Zig tests embed the dist too — W10 static tests must still pass against the SPA dist; if one pins placeholder content, report it, do not edit Zig).
|
||||
|
||||
### F9 As built
|
||||
|
||||
ci.yml +77/-5: top-level `NODE_VERSION: "24"`; new `frontend` job (setup-node@v4,
|
||||
cache npm on web/package-lock.json; npm ci → format:check → lint → typecheck →
|
||||
`npm test` → build); `cross` job builds the SPA then
|
||||
`zig build cross -Dweb-dist=web/dist -Doptimize=ReleaseSafe` — RULED deviation:
|
||||
build.zig has no strip option and Debug exes are 87/32 MB (25.6 MB stripped), so the
|
||||
PLAN:639 budget is only meaningful for release builds; PLAN:50 calls these release
|
||||
binaries. Size assert strips COPIES via binutils (zig objcopy --strip-all is
|
||||
unimplemented in 0.16; aarch64 needs binutils-aarch64-linux-gnu, apt-installed
|
||||
conditionally). Measured ReleaseSafe: raw ~24.2 MB each; stripped x86_64 5,501,624
|
||||
bytes (PASS < 15 MB), aarch64 PT_LOAD total 4.87 MB (~5 MB stripped, asserted in CI).
|
||||
`test` job stays Zig-only on the placeholder dist.
|
||||
Smoke (embedded SPA): / serves the vite index (not the placeholder), /settings deep
|
||||
link 200, main chunk served gzip with ETag then 304 on If-None-Match, login round
|
||||
trip (401 → cookie → 200 → logout → 401), SIGTERM exit 0.
|
||||
Finding, fixed post-session under an orchestrator ruling (narrow Zig-freeze
|
||||
exception): the W8 test at static.zig:413 pinned placeholder strings and failed with
|
||||
any real dist — rewritten dist-agnostic (structural asserts: /index.html exists,
|
||||
text/html, gz siblings have base entries, quoted etags, no duplicate paths).
|
||||
Import CLI note: the config file is positional on `nxdns import` (no --config flag).
|
||||
ci.yml +77/-5: top-level `NODE_VERSION: "24"`; new `frontend` job (setup-node@v4, cache npm on web/package-lock.json; npm ci → format:check → lint → typecheck → `npm test` → build); `cross` job builds the SPA then `zig build cross -Dweb-dist=web/dist -Doptimize=ReleaseSafe` — RULED deviation: build.zig has no strip option and Debug exes are 87/32 MB (25.6 MB stripped), so the PLAN:639 budget is only meaningful for release builds; PLAN:50 calls these release binaries. Size assert strips COPIES via binutils (zig objcopy --strip-all is unimplemented in 0.16; aarch64 needs binutils-aarch64-linux-gnu, apt-installed conditionally). Measured ReleaseSafe: raw ~24.2 MB each; stripped x86_64 5,501,624 bytes (PASS < 15 MB), aarch64 PT_LOAD total 4.87 MB (~5 MB stripped, asserted in CI). `test` job stays Zig-only on the placeholder dist. Smoke (embedded SPA): / serves the vite index (not the placeholder), /settings deep link 200, main chunk served gzip with ETag then 304 on If-None-Match, login round trip (401 → cookie → 200 → logout → 401), SIGTERM exit 0. Finding, fixed post-session under an orchestrator ruling (narrow Zig-freeze exception): the W8 test at static.zig:413 pinned placeholder strings and failed with any real dist — rewritten dist-agnostic (structural asserts: /index.html exists, text/html, gz siblings have base entries, quoted etags, no duplicate paths). Import CLI note: the config file is positional on `nxdns import` (no --config flag).
|
||||
|
||||
---
|
||||
|
||||
## Module layout (new)
|
||||
|
||||
web/{package.json, package-lock.json, index.html, vite.config.ts, tsconfig*.json},
|
||||
web/src/{main.tsx, routes.tsx, styles.css}, web/src/shell/*, web/src/lib/*,
|
||||
web/src/auth/*, web/src/features/{dashboard,queries,live,clients,groups,blocklists,
|
||||
rules,local,lookup,settings,pause}/*.
|
||||
web/{package.json, package-lock.json, index.html, vite.config.ts, tsconfig*.json}, web/src/{main.tsx, routes.tsx, styles.css}, web/src/shell/*, web/src/lib/*, web/src/auth/*, web/src/features/{dashboard,queries,live,clients,groups,blocklists, rules,local,lookup,settings,pause}/*.
|
||||
|
||||
## File ownership
|
||||
|
||||
F1 scaffold+shell+routes; F2 lib+auth; F3-F8 exactly their feature dirs (routes.tsx
|
||||
lazy imports point at stable paths F1 fixes up-front, so page sessions never edit
|
||||
routes.tsx; F8 alone replaces the PauseWidget file F1 created); F9 ci.yml. Orchestrator:
|
||||
spec, integration wiring if any. Parallel sessions never share a file.
|
||||
F1 scaffold+shell+routes; F2 lib+auth; F3-F8 exactly their feature dirs (routes.tsx lazy imports point at stable paths F1 fixes up-front, so page sessions never edit routes.tsx; F8 alone replaces the PauseWidget file F1 created); F9 ci.yml. Orchestrator: spec, integration wiring if any. Parallel sessions never share a file.
|
||||
|
||||
## Acceptance (milestone complete)
|
||||
|
||||
@@ -373,37 +148,18 @@ spec, integration wiring if any. Parallel sessions never share a file.
|
||||
## Review (Codex, As built)
|
||||
|
||||
Round 1: 7 important + 4 minor, all fixed.
|
||||
- auth: AuthProvider probes once on mount when authRequired is null (dedupe kept;
|
||||
sessionStorage fast path, probe overwrites); logout swallows ONLY 401 — other errors
|
||||
rethrow and AppShell's LogoutButton renders them via InlineError, navigating only on
|
||||
success; LoginPage 429 is a ticking lockout (submit + Enter disabled until zero).
|
||||
- queries: load-more generation counter — filter apply/clear increments; then/catch/
|
||||
finally discard stale completions (no old-filter rows, no stale cursor); imperative
|
||||
401s route through `handleUnauthorized` (newly exported from lib/queryClient).
|
||||
- live: gap-resync 401 → handleUnauthorized (never resyncFailed); entering capped
|
||||
fires a one-shot injectable probeSession (default GET /api/pause) so an expired
|
||||
session redirects to login instead of reading as "capped".
|
||||
- settings: fieldset disabled while the PUT is pending (no silently dropped edits);
|
||||
errors via InlineError.
|
||||
- pause: mutation errors rendered via InlineError in both branches (were silent);
|
||||
stale error resets on pause-state flip.
|
||||
- blocklists/rules: all mutation errors via InlineError (429 countdown); form error
|
||||
props are Error|null now.
|
||||
- Ripple: the auth mount probe legitimately fires POST /api/auth/login before page
|
||||
fetches — LocalDnsPage.test's first-POST assertion narrowed to match by URL.
|
||||
After fixes: 24 files / 120 web tests green; Zig suite with the fresh dist 0 failed.
|
||||
- auth: AuthProvider probes once on mount when authRequired is null (dedupe kept; sessionStorage fast path, probe overwrites); logout swallows ONLY 401 — other errors rethrow and AppShell's LogoutButton renders them via InlineError, navigating only on success; LoginPage 429 is a ticking lockout (submit + Enter disabled until zero).
|
||||
- queries: load-more generation counter — filter apply/clear increments; then/catch/ finally discard stale completions (no old-filter rows, no stale cursor); imperative 401s route through `handleUnauthorized` (newly exported from lib/queryClient).
|
||||
- live: gap-resync 401 → handleUnauthorized (never resyncFailed); entering capped fires a one-shot injectable probeSession (default GET /api/pause) so an expired session redirects to login instead of reading as "capped".
|
||||
- settings: fieldset disabled while the PUT is pending (no silently dropped edits); errors via InlineError.
|
||||
- pause: mutation errors rendered via InlineError in both branches (were silent); stale error resets on pause-state flip.
|
||||
- blocklists/rules: all mutation errors via InlineError (429 countdown); form error props are Error|null now.
|
||||
- Ripple: the auth mount probe legitimately fires POST /api/auth/login before page fetches — LocalDnsPage.test's first-POST assertion narrowed to match by URL. After fixes: 24 files / 120 web tests green; Zig suite with the fresh dist 0 failed.
|
||||
|
||||
Round 2: 1 important + 1 minor, both fixed. (a) keepPreviousData left the OLD
|
||||
filter's cursor clickable during the placeholder window — loadMore and the button now
|
||||
bail on base.isPlaceholderData (chosen over isFetching so background refetches of the
|
||||
current key stay usable); regression test proves no new-filter/old-cursor request.
|
||||
(b) safeRedirect accepted backslash network paths — now
|
||||
`/^\/(?![/\\])/.test(raw) && !raw.includes("\\")`, with unit cases.
|
||||
Round 2: 1 important + 1 minor, both fixed. (a) keepPreviousData left the OLD filter's cursor clickable during the placeholder window — loadMore and the button now bail on base.isPlaceholderData (chosen over isFetching so background refetches of the current key stay usable); regression test proves no new-filter/old-cursor request. (b) safeRedirect accepted backslash network paths — now `/^\/(?![/\\])/.test(raw) && !raw.includes("\\")`, with unit cases.
|
||||
|
||||
Round 3: no findings. Final: 121 web tests green.
|
||||
|
||||
## Anti-requirements
|
||||
|
||||
No SSR, no chart library, no codegen (openapi→TS), no msw, no ESLint, no i18n, no
|
||||
dark-mode toggle, no WebSocket, no service worker/PWA, no Zig changes, no docs/api
|
||||
rendering (Phase 10), no DoH/DoT UI (Phase 9 adds settings sections it already has).
|
||||
No SSR, no chart library, no codegen (openapi→TS), no msw, no ESLint, no i18n, no dark-mode toggle, no WebSocket, no service worker/PWA, no Zig changes, no docs/api rendering (Phase 10), no DoH/DoT UI (Phase 9 adds settings sections it already has).
|
||||
|
||||
@@ -1,219 +1,102 @@
|
||||
# Zig 0.16.0 stdlib API notes (verified against ../zig at tag 0.16.0)
|
||||
|
||||
Ground truth for build sub-agents. Every claim below was read from the 0.16.0 source
|
||||
(commit 24fdd5b7a4c1c8b5deb5b56756b9dbc8e08c86a8). When in doubt, re-check the source at
|
||||
`/home/mokhtar/app/zig/lib/std/` — do not trust pre-0.16 knowledge; the std.Io migration
|
||||
changed most of these APIs.
|
||||
Ground truth for build sub-agents. Every claim below was read from the 0.16.0 source (commit 24fdd5b7a4c1c8b5deb5b56756b9dbc8e08c86a8). When in doubt, re-check the source at `/home/mokhtar/app/zig/lib/std/` — do not trust pre-0.16 knowledge; the std.Io migration changed most of these APIs.
|
||||
|
||||
## std.Build (build.zig)
|
||||
|
||||
- `b.addExecutable(.{ .name, .root_module, ... })` — no target/optimize on the artifact.
|
||||
Target/optimize/link_libc go on the module: `b.createModule(.{ .root_source_file, .target,
|
||||
.optimize, .link_libc, .imports, ... })` (Module.CreateOptions, Module.zig:216).
|
||||
- `b.addExecutable(.{ .name, .root_module, ... })` — no target/optimize on the artifact. Target/optimize/link_libc go on the module: `b.createModule(.{ .root_source_file, .target, .optimize, .link_libc, .imports, ... })` (Module.CreateOptions, Module.zig:216).
|
||||
- All C configuration is **module-level** (Step.Compile has none of these methods):
|
||||
- `exe.root_module.addCSourceFile(.{ .file, .flags, .language })`
|
||||
- `exe.root_module.addCSourceFiles(.{ .root, .files, .flags })` — `files` must be
|
||||
**relative** paths; absolute paths panic.
|
||||
- `exe.root_module.addCSourceFiles(.{ .root, .files, .flags })` — `files` must be **relative** paths; absolute paths panic.
|
||||
- `exe.root_module.addIncludePath(lazy_path)`, `.addCMacro(name, value)`
|
||||
- libc: `link_libc = true` in CreateOptions or `exe.root_module.link_libc = true`.
|
||||
- Static lib: `b.addLibrary(.{ .linkage = .static, .name, .root_module })`
|
||||
(`b.addStaticLibrary` does not exist). Link: `exe.root_module.linkLibrary(lib)`.
|
||||
- Cross targets: `b.resolveTargetQuery(.{ .cpu_arch = .aarch64, .os_tag = .linux,
|
||||
.abi = .musl })` or `std.Target.Query.parse(.{ .arch_os_abi = "x86_64-linux-musl" })`.
|
||||
musl → static libc; for a fully static binary also set `exe.linkage = .static`.
|
||||
- Options: `b.option(std.Build.LazyPath, "web-dist", "...")` accepts LazyPath.
|
||||
`b.addOptions()` + `opts.addOption/addOptionPath` + `exe.root_module.addOptions("build_options", opts)`.
|
||||
- Embedding an asset directory: one WriteFile step holding both a generated `index.zig`
|
||||
and the assets: `const wf = b.addWriteFiles(); _ = wf.addCopyDirectory(dist, "assets", .{});
|
||||
const idx = wf.add("index.zig", src);` then
|
||||
`exe.root_module.addAnonymousImport("assets", .{ .root_source_file = idx })`.
|
||||
`@embedFile` paths must stay inside that module root (`error.ImportOutsideModulePath`).
|
||||
- Version pin: compare `@import("builtin").zig_version` (SemanticVersion) in comptime,
|
||||
`@compileError` on mismatch.
|
||||
- build.zig.zon: `.name` is an **enum literal** (`.nxdns`), `.fingerprint` is **required**
|
||||
(compiler suggests the value on first run), plus `.version`, `.paths`, `.dependencies`
|
||||
(`path`/`url`+`hash`/`lazy`), `.minimum_zig_version`.
|
||||
- Static lib: `b.addLibrary(.{ .linkage = .static, .name, .root_module })` (`b.addStaticLibrary` does not exist). Link: `exe.root_module.linkLibrary(lib)`.
|
||||
- Cross targets: `b.resolveTargetQuery(.{ .cpu_arch = .aarch64, .os_tag = .linux, .abi = .musl })` or `std.Target.Query.parse(.{ .arch_os_abi = "x86_64-linux-musl" })`. musl → static libc; for a fully static binary also set `exe.linkage = .static`.
|
||||
- Options: `b.option(std.Build.LazyPath, "web-dist", "...")` accepts LazyPath. `b.addOptions()` + `opts.addOption/addOptionPath` + `exe.root_module.addOptions("build_options", opts)`.
|
||||
- Embedding an asset directory: one WriteFile step holding both a generated `index.zig` and the assets: `const wf = b.addWriteFiles(); _ = wf.addCopyDirectory(dist, "assets", .{}); const idx = wf.add("index.zig", src);` then `exe.root_module.addAnonymousImport("assets", .{ .root_source_file = idx })`. `@embedFile` paths must stay inside that module root (`error.ImportOutsideModulePath`).
|
||||
- Version pin: compare `@import("builtin").zig_version` (SemanticVersion) in comptime, `@compileError` on mismatch.
|
||||
- build.zig.zon: `.name` is an **enum literal** (`.nxdns`), `.fingerprint` is **required** (compiler suggests the value on first run), plus `.version`, `.paths`, `.dependencies` (`path`/`url`+`hash`/`lazy`), `.minimum_zig_version`.
|
||||
|
||||
## std.process + stdio (found during S1)
|
||||
|
||||
- `std.process.args` / `std.process.ArgIterator` **do not exist** in 0.16. Arguments arrive
|
||||
through main's parameter: `pub fn main(init: std.process.Init) u8 { var args =
|
||||
init.minimal.args.iterate(); ... }`. `std.process.Init` carries `.io`, `.gpa`, `.arena`,
|
||||
`.environ_map`, `.preopens`, `.minimal` (`.args`, `.environ`).
|
||||
See lib/std/process.zig:30 and lib/std/process/Args.zig.
|
||||
- Console output: `std.Io.File.stdout().writer(io, &buffer)` then use `.interface`
|
||||
(lib/std/Io/File.zig:91,600). Same shape for stderr.
|
||||
- `std.process.args` / `std.process.ArgIterator` **do not exist** in 0.16. Arguments arrive through main's parameter: `pub fn main(init: std.process.Init) u8 { var args = init.minimal.args.iterate(); ... }`. `std.process.Init` carries `.io`, `.gpa`, `.arena`, `.environ_map`, `.preopens`, `.minimal` (`.args`, `.environ`). See lib/std/process.zig:30 and lib/std/process/Args.zig.
|
||||
- Console output: `std.Io.File.stdout().writer(io, &buffer)` then use `.interface` (lib/std/Io/File.zig:91,600). Same shape for stderr.
|
||||
- Package deps unpack into `zig-pkg/` inside the project root (gitignore it).
|
||||
- Entropy: `io.random(buf)` (Io.zig:2468). Realtime timestamp: `std.Io.Clock.real.now(io)`
|
||||
→ `Io.Timestamp` (Io.zig:778). There is NO `io.now(.real)`.
|
||||
- Entropy: `io.random(buf)` (Io.zig:2468). Realtime timestamp: `std.Io.Clock.real.now(io)` → `Io.Timestamp` (Io.zig:778). There is NO `io.now(.real)`.
|
||||
- `std.mem.indexOfScalar` is gone; use `std.mem.findScalar` (mem.zig:1219).
|
||||
- Conditional imports are impossible: no `@hasImport`, and `@import("root")` in a
|
||||
`zig test` build resolves to the compiler's test runner, never your tests root.
|
||||
Integration tests gated on `build_options` therefore live in a separate file with a
|
||||
runtime `if (!build_options.integration) return error.SkipZigTest;` guard (body stays
|
||||
semantically analyzed either way — code cannot rot).
|
||||
- ENVIRONMENT (this dev machine): IPv6 egress is broken — hostname-resolving tests hit
|
||||
AAAA-first timeouts. Use documented IPv4 literals in live tests, keep SNI on the name.
|
||||
- `zig fetch` accepts tar.gz/zip etc. but **not .tar.bz2** (no bzip2 decompressor —
|
||||
src/Package/Fetch.zig ~line 1337). Pin GitHub *source tag* tarballs when a release
|
||||
asset is bz2-only.
|
||||
- Conditional imports are impossible: no `@hasImport`, and `@import("root")` in a `zig test` build resolves to the compiler's test runner, never your tests root. Integration tests gated on `build_options` therefore live in a separate file with a runtime `if (!build_options.integration) return error.SkipZigTest;` guard (body stays semantically analyzed either way — code cannot rot).
|
||||
- ENVIRONMENT (this dev machine): IPv6 egress is broken — hostname-resolving tests hit AAAA-first timeouts. Use documented IPv4 literals in live tests, keep SNI on the name.
|
||||
- `zig fetch` accepts tar.gz/zip etc. but **not .tar.bz2** (no bzip2 decompressor — src/Package/Fetch.zig ~line 1337). Pin GitHub *source tag* tarballs when a release asset is bz2-only.
|
||||
|
||||
## std.Io (Threaded backend, concurrency)
|
||||
|
||||
- `var t = std.Io.Threaded.init(gpa, .{});` — returned by value, must live at a **stable
|
||||
address**. `const io = t.io();` `defer t.deinit();` (joins workers).
|
||||
InitOptions: `stack_size`, `async_limit` (default cpus-1), `concurrent_limit`.
|
||||
- `io.async(f, args) Future(R)` may run inline; `io.concurrent(f, args) !Future(R)`
|
||||
guarantees its own unit of concurrency — use for server loops.
|
||||
`future.await(io)`, `future.cancel(io)`.
|
||||
- `Io.Group`: `.init`, `g.async(io, f, args)`, `g.concurrent(io, f, args)`, `g.await(io)`,
|
||||
`g.cancel(io)`. Group task fns must return something coercible to `Cancelable!void`.
|
||||
Cancellation: next cancelable Io call returns `error.Canceled` once; `io.recancel()`
|
||||
re-arms; `io.swapCancelProtection(.blocked)` protects cleanup sections.
|
||||
- **No SIGINT/SIGTERM handling in Threaded** (only no-op SIGIO/SIGPIPE). Install our own
|
||||
`posix.sigaction`, set an atomic flag, cancel the group from the main task.
|
||||
- `var t = std.Io.Threaded.init(gpa, .{});` — returned by value, must live at a **stable address**. `const io = t.io();` `defer t.deinit();` (joins workers). InitOptions: `stack_size`, `async_limit` (default cpus-1), `concurrent_limit`.
|
||||
- `io.async(f, args) Future(R)` may run inline; `io.concurrent(f, args) !Future(R)` guarantees its own unit of concurrency — use for server loops. `future.await(io)`, `future.cancel(io)`.
|
||||
- `Io.Group`: `.init`, `g.async(io, f, args)`, `g.concurrent(io, f, args)`, `g.await(io)`, `g.cancel(io)`. Group task fns must return something coercible to `Cancelable!void`. Cancellation: next cancelable Io call returns `error.Canceled` once; `io.recancel()` re-arms; `io.swapCancelProtection(.blocked)` protects cleanup sections.
|
||||
- **No SIGINT/SIGTERM handling in Threaded** (only no-op SIGIO/SIGPIPE). Install our own `posix.sigaction`, set an atomic flag, cancel the group from the main task.
|
||||
|
||||
## std.Io.net
|
||||
|
||||
- UDP: `addr.bind(io, .{ .mode = .dgram })` → `Socket`.
|
||||
Receive: `sock.receive(io, buf) !IncomingMessage` — fields `from: IpAddress`,
|
||||
`data: []u8` (slice into buf), `flags.trunc`. Timeout variant: `receiveTimeout(io, buf,
|
||||
timeout)`. Send: `sock.send(io, &dest_addr, data)`. Close: `sock.close(io)`.
|
||||
(No recvFrom/sendTo names.)
|
||||
- TCP server: `addr.listen(io, .{ .reuse_address = true })` → `net.Server`;
|
||||
`srv.accept(io) !Stream`; `srv.deinit(io)`. `Stream.close(io)`, `.shutdown(io, how)`.
|
||||
Shutdown of the listener makes a blocked accept fail `error.SocketNotListening`.
|
||||
- TCP client: `addr.connect(io, .{ .mode = .stream })` → `Stream` —
|
||||
`ConnectOptions.mode` is REQUIRED (no default, net.zig:332).
|
||||
`ConnectOptions.timeout` is a LANDMINE in 0.16.0: the Threaded backend panics
|
||||
"TODO implement netConnectIpPosix with timeout" (Threaded.zig:12077). Never set it.
|
||||
Bound connects the same way as reads: race the task against a `Clock.Duration.sleep`
|
||||
via `std.Io.Select` and cancel the loser (see tls_client_integration_test.zig).
|
||||
- **Stream reads/writes accept no timeout** in 0.16.0 (VTable netRead/netWrite have none;
|
||||
Operation lacks net stream variants). Bound a TCP/TLS read by running it under
|
||||
`io.concurrent` and cancelling the future (or shutdown the socket).
|
||||
- Reader/Writer: `stream.reader(io, buf) Stream.Reader`; generic interface is
|
||||
`&stream_reader.interface` (`*Io.Reader`); same for writer. Caller owns buffers; `&.{}`
|
||||
legal. Concrete errors land in `.err`; interface returns ReadFailed/WriteFailed.
|
||||
`Stream.Writer.sendFile` returns `error.Unimplemented` in 0.16.0.
|
||||
- Known: net tests in stdlib are skipped (upstream issue 31388) — do not copy test.zig
|
||||
patterns blindly.
|
||||
- UDP: `addr.bind(io, .{ .mode = .dgram })` → `Socket`. Receive: `sock.receive(io, buf) !IncomingMessage` — fields `from: IpAddress`, `data: []u8` (slice into buf), `flags.trunc`. Timeout variant: `receiveTimeout(io, buf, timeout)`. Send: `sock.send(io, &dest_addr, data)`. Close: `sock.close(io)`. (No recvFrom/sendTo names.)
|
||||
- TCP server: `addr.listen(io, .{ .reuse_address = true })` → `net.Server`; `srv.accept(io) !Stream`; `srv.deinit(io)`. `Stream.close(io)`, `.shutdown(io, how)`. Shutdown of the listener makes a blocked accept fail `error.SocketNotListening`.
|
||||
- TCP client: `addr.connect(io, .{ .mode = .stream })` → `Stream` — `ConnectOptions.mode` is REQUIRED (no default, net.zig:332). `ConnectOptions.timeout` is a LANDMINE in 0.16.0: the Threaded backend panics "TODO implement netConnectIpPosix with timeout" (Threaded.zig:12077). Never set it. Bound connects the same way as reads: race the task against a `Clock.Duration.sleep` via `std.Io.Select` and cancel the loser (see tls_client_integration_test.zig).
|
||||
- **Stream reads/writes accept no timeout** in 0.16.0 (VTable netRead/netWrite have none; Operation lacks net stream variants). Bound a TCP/TLS read by running it under `io.concurrent` and cancelling the future (or shutdown the socket).
|
||||
- Reader/Writer: `stream.reader(io, buf) Stream.Reader`; generic interface is `&stream_reader.interface` (`*Io.Reader`); same for writer. Caller owns buffers; `&.{}` legal. Concrete errors land in `.err`; interface returns ReadFailed/WriteFailed. `Stream.Writer.sendFile` returns `error.Unimplemented` in 0.16.0.
|
||||
- Known: net tests in stdlib are skipped (upstream issue 31388) — do not copy test.zig patterns blindly.
|
||||
|
||||
## std.http.Client (DoH upstream)
|
||||
|
||||
- Construct by struct literal: `var client: std.http.Client = .{ .allocator = gpa, .io = io };`
|
||||
`defer client.deinit();` Fields: `ca_bundle`, `ca_bundle_lock`, `now: ?Io.Timestamp`,
|
||||
`connection_pool` (LRU, free_size 32), `read_buffer_size`, `write_buffer_size`.
|
||||
- Construct by struct literal: `var client: std.http.Client = .{ .allocator = gpa, .io = io };` `defer client.deinit();` Fields: `ca_bundle`, `ca_bundle_lock`, `now: ?Io.Timestamp`, `connection_pool` (LRU, free_size 32), `read_buffer_size`, `write_buffer_size`.
|
||||
- DoH POST flow (low-level; `fetch` hides the body—don't use it):
|
||||
1. `var req = try client.request(.POST, uri, .{ .headers = .{ .content_type =
|
||||
.{ .override = "application/dns-message" } } });`
|
||||
1. `var req = try client.request(.POST, uri, .{ .headers = .{ .content_type = .{ .override = "application/dns-message" } } });`
|
||||
2. `try req.sendBodyComplete(body_mut);` (body is `[]u8`, sets content-length + flush)
|
||||
3. `var resp = try req.receiveHead(&redirect_buf);`
|
||||
4. Check `resp.head.status`, `resp.head.content_type`.
|
||||
5. `const rdr = resp.reader(&transfer_buf);` then read (`allocRemaining`/`readSliceShort`).
|
||||
6. `req.deinit()` returns the connection to the pool.
|
||||
- TLS: client TLS uses std.crypto.tls.Client internally with the client's `ca_bundle`;
|
||||
lazily `rescan`s system roots when `client.now == null`
|
||||
(`error.CertificateBundleLoadFailure` on failure). Pre-fill `ca_bundle` + set `now` to
|
||||
skip the scan.
|
||||
- **No per-request deadline** exists. Enforce total-budget timeouts via concurrent+cancel
|
||||
(see std.Io note above).
|
||||
- TLS: client TLS uses std.crypto.tls.Client internally with the client's `ca_bundle`; lazily `rescan`s system roots when `client.now == null` (`error.CertificateBundleLoadFailure` on failure). Pre-fill `ca_bundle` + set `now` to skip the scan.
|
||||
- **No per-request deadline** exists. Enforce total-budget timeouts via concurrent+cancel (see std.Io note above).
|
||||
|
||||
## std.http.Server (web/API/DoH server)
|
||||
|
||||
- `var srv = std.http.Server.init(&reader.interface, &writer.interface);`
|
||||
Loop for keep-alive: `while (srv.reader.state == .ready)` + catch HttpConnectionClosing.
|
||||
- `var req = try srv.receiveHead();` — `req.head.method/.target/.content_type/...`,
|
||||
`req.iterateHeaders()`. **Copy `head.target` before reading the body** (body reads
|
||||
invalidate head memory). Body: `req.readerExpectContinue(buf)` (handles 100-continue)
|
||||
or `req.readerExpectNone(buf)`.
|
||||
- `var srv = std.http.Server.init(&reader.interface, &writer.interface);` Loop for keep-alive: `while (srv.reader.state == .ready)` + catch HttpConnectionClosing.
|
||||
- `var req = try srv.receiveHead();` — `req.head.method/.target/.content_type/...`, `req.iterateHeaders()`. **Copy `head.target` before reading the body** (body reads invalidate head memory). Body: `req.readerExpectContinue(buf)` (handles 100-continue) or `req.readerExpectNone(buf)`.
|
||||
- Simple response: `try req.respond(content, .{ .status = ..., .extra_headers = &.{...} });`
|
||||
- Streaming/SSE: `var body = try req.respondStreaming(&.{}, .{ .respond_options = ... });`
|
||||
With `content_length == null` → chunked. **Use an empty buffer** so each write drains
|
||||
through; then `response.flush()` after each event suffices; `response.end()` to finish.
|
||||
(With a non-empty buffer you must flush response.writer first — trap.)
|
||||
- Streaming/SSE: `var body = try req.respondStreaming(&.{}, .{ .respond_options = ... });` With `content_length == null` → chunked. **Use an empty buffer** so each write drains through; then `response.flush()` after each event suffices; `response.end()` to finish. (With a non-empty buffer you must flush response.writer first — trap.)
|
||||
- WebSocket upgrade exists (`upgradeRequested`/`respondWebSocket`) — unused by nxdns.
|
||||
- No MIME table, no etag: set `content-type`/`cache-control` manually via extra_headers.
|
||||
BodyWriter supports sendFile (contentLengthSendFile/chunkedSendFile).
|
||||
- No MIME table, no etag: set `content-type`/`cache-control` manually via extra_headers. BodyWriter supports sendFile (contentLengthSendFile/chunkedSendFile).
|
||||
|
||||
## std.testing.fuzz (0.16)
|
||||
|
||||
- `std.testing.fuzz(context, testOne, options)` where `testOne: fn(ctx, smith: *std.testing.Smith) anyerror!void` —
|
||||
the input is a `*Smith` structured generator, NOT a raw `[]const u8` slice (pre-0.16 form is gone).
|
||||
- `std.testing.fuzz(context, testOne, options)` where `testOne: fn(ctx, smith: *std.testing.Smith) anyerror!void` — the input is a `*Smith` structured generator, NOT a raw `[]const u8` slice (pre-0.16 form is gone).
|
||||
- `FuzzInputOptions{ .corpus: []const []const u8 }` seeds the corpus.
|
||||
- Smith surface (lib/std/testing/Smith.zig): `valueWithHash`, `valueRangeAtMostWithHash`,
|
||||
`bytesWithHash`, `sliceWithHash(buf, hash) u32` (fills buf, returns length), `eosWithHash`, etc.
|
||||
For raw-bytes parsers: fill a buffer via `sliceWithHash` and feed the prefix to the parser.
|
||||
- Plain `zig build test` replays each corpus entry once + one empty input (deterministic).
|
||||
`--fuzz=<n>` = iterations PER TEST (K/M/G suffixes); bare `--fuzz` = forever + webui;
|
||||
no time bound exists; `--fuzz=<n>` conflicts with `--webui`.
|
||||
- Smith surface (lib/std/testing/Smith.zig): `valueWithHash`, `valueRangeAtMostWithHash`, `bytesWithHash`, `sliceWithHash(buf, hash) u32` (fills buf, returns length), `eosWithHash`, etc. For raw-bytes parsers: fill a buffer via `sliceWithHash` and feed the prefix to the parser.
|
||||
- Plain `zig build test` replays each corpus entry once + one empty input (deterministic). `--fuzz=<n>` = iterations PER TEST (K/M/G suffixes); bare `--fuzz` = forever + webui; no time bound exists; `--fuzz=<n>` conflicts with `--webui`.
|
||||
- Corpus entries are Smith byte streams (slice = u32-LE length + bytes), not raw inputs.
|
||||
- STOCK 0.16.0 CANNOT RUN `--fuzz`: (A) fuzz-mode test_runner.zig:566 has a type error
|
||||
(needs patched --zig-lib-dir); (B) the self-hosted x86_64 backend emits no coverage
|
||||
PCs — set `.use_llvm = true` on the fuzz test artifact.
|
||||
- `zig test` collects tests ONLY from the root module — moving files into a named module
|
||||
silently drops their tests. A file belongs to exactly one module per compilation, and
|
||||
imports cannot escape the module root directory. Hence: aggregator imports files
|
||||
directly; separate artifacts import a module-root file (e.g. src/dns/dns.zig).
|
||||
- STOCK 0.16.0 CANNOT RUN `--fuzz`: (A) fuzz-mode test_runner.zig:566 has a type error (needs patched --zig-lib-dir); (B) the self-hosted x86_64 backend emits no coverage PCs — set `.use_llvm = true` on the fuzz test artifact.
|
||||
- `zig test` collects tests ONLY from the root module — moving files into a named module silently drops their tests. A file belongs to exactly one module per compilation, and imports cannot escape the module root directory. Hence: aggregator imports files directly; separate artifacts import a module-root file (e.g. src/dns/dns.zig).
|
||||
|
||||
## std.zon
|
||||
|
||||
- Parse: `std.zon.parse.fromSlice(T, gpa, src_z, ?*Diagnostics, .{})` for pointer-free T
|
||||
(no free needed); `fromSliceAlloc` + `std.zon.parse.free(gpa, v)` for slice-bearing T.
|
||||
Source must be `[:0]const u8`. No reader-based entry point.
|
||||
Options: `ignore_unknown_fields`, `free_on_error`. Diagnostics: init `.{}`,
|
||||
`deinit(gpa)`, print with `{f}`.
|
||||
- Struct default field values apply for absent fields. Tagged unions: `.foo` (void) or
|
||||
`.{ .foo = v }`. Enums, optionals (single level only — `??T` rejected), slices, arrays,
|
||||
nested structs all fine. Error sets/unions, many-pointers, comptime_int rejected.
|
||||
- Serialize: `std.zon.stringify.serialize(value, .{ .whitespace = true }, writer)`.
|
||||
Streaming: `var s: std.zon.Serializer = .{ .writer = w, .options = .{} };` with
|
||||
`beginStruct`/`field`/`end`.
|
||||
- Parse: `std.zon.parse.fromSlice(T, gpa, src_z, ?*Diagnostics, .{})` for pointer-free T (no free needed); `fromSliceAlloc` + `std.zon.parse.free(gpa, v)` for slice-bearing T. Source must be `[:0]const u8`. No reader-based entry point. Options: `ignore_unknown_fields`, `free_on_error`. Diagnostics: init `.{}`, `deinit(gpa)`, print with `{f}`.
|
||||
- Struct default field values apply for absent fields. Tagged unions: `.foo` (void) or `.{ .foo = v }`. Enums, optionals (single level only — `??T` rejected), slices, arrays, nested structs all fine. Error sets/unions, many-pointers, comptime_int rejected.
|
||||
- Serialize: `std.zon.stringify.serialize(value, .{ .whitespace = true }, writer)`. Streaming: `var s: std.zon.Serializer = .{ .writer = w, .options = .{} };` with `beginStruct`/`field`/`end`.
|
||||
|
||||
## std.crypto.tls.Client (upstream DoT)
|
||||
|
||||
- `std.crypto.tls.Client.init(input: *Io.Reader, output: *Io.Writer, options) !Client`
|
||||
Options: `.host = .{ .explicit = "dns.google" }` (or `.no_verification`),
|
||||
`.ca = .{ .bundle = .{ .gpa, .io, .lock, .bundle } }` (or `.no_verification`/`.self_signed`),
|
||||
`.write_buffer`, `.read_buffer` (input buffer ≥ `Client.min_buffer_len` =
|
||||
`tls.max_ciphertext_record_len`), `.entropy: *const [240]u8` (fill via `io.random`),
|
||||
`.realtime_now: Io.Timestamp`.
|
||||
- After init: plaintext via `&client.reader` / `&client.writer` (held **by value** — the
|
||||
Client must not move after init). `client.end()` sends close_notify. Errors in
|
||||
`client.read_err` / `client.alert`.
|
||||
- TLS 1.2/1.3. **No ALPN, no session resumption** (fine for DoT; DoH over HTTP/1.1 works
|
||||
without ALPN in practice — verify against real upstreams in Phase 3).
|
||||
- `std.crypto.tls.Client.init(input: *Io.Reader, output: *Io.Writer, options) !Client` Options: `.host = .{ .explicit = "dns.google" }` (or `.no_verification`), `.ca = .{ .bundle = .{ .gpa, .io, .lock, .bundle } }` (or `.no_verification`/`.self_signed`), `.write_buffer`, `.read_buffer` (input buffer ≥ `Client.min_buffer_len` = `tls.max_ciphertext_record_len`), `.entropy: *const [240]u8` (fill via `io.random`), `.realtime_now: Io.Timestamp`.
|
||||
- After init: plaintext via `&client.reader` / `&client.writer` (held **by value** — the Client must not move after init). `client.end()` sends close_notify. Errors in `client.read_err` / `client.alert`.
|
||||
- TLS 1.2/1.3. **No ALPN, no session resumption** (fine for DoT; DoH over HTTP/1.1 works without ALPN in practice — verify against real upstreams in Phase 3).
|
||||
- CA roots: `std.crypto.Certificate.Bundle` (note the path), `bundle.rescan(gpa, io, now)`.
|
||||
|
||||
## Certificate verification ignores IP SANs (verified 0.16.0)
|
||||
|
||||
`std.crypto.Certificate.Parsed.verifyHostName` (Certificate.zig:313) checks only
|
||||
`dNSName` general names in the SAN extension; the switch's `else => {}` skips
|
||||
`iPAddress` (tag 7) entries entirely. Consequence: a TLS connection whose
|
||||
verification name is an IP literal (DoT `tls://1.1.1.1:853`) always fails with
|
||||
`error.CertificateHostMismatch` against certificates that carry the address only
|
||||
as an iPAddress SAN — which is how Cloudflare and Quad9 issue theirs. A DoT
|
||||
upstream therefore needs a DNS `tls_name` for SNI + verification while dialing
|
||||
the IP; verifying by bare IP cannot work on stock 0.16.
|
||||
`std.crypto.Certificate.Parsed.verifyHostName` (Certificate.zig:313) checks only `dNSName` general names in the SAN extension; the switch's `else => {}` skips `iPAddress` (tag 7) entries entirely. Consequence: a TLS connection whose verification name is an IP literal (DoT `tls://1.1.1.1:853`) always fails with `error.CertificateHostMismatch` against certificates that carry the address only as an iPAddress SAN — which is how Cloudflare and Quad9 issue theirs. A DoT upstream therefore needs a DNS `tls_name` for SNI + verification while dialing the IP; verifying by bare IP cannot work on stock 0.16.
|
||||
|
||||
## Phase 6 verifications (concurrency, disk, files)
|
||||
|
||||
- `std.Io.Queue(Elem)` exists (Io.zig:2184): bounded ring buffer over a caller-supplied array.
|
||||
`put(q, io, elems, min)` with `min = 0` never blocks and returns 0 when full (Io.zig:2218);
|
||||
`getOne` blocks until an item or `error.Closed` after `close(io)`. Elements are copied as raw
|
||||
bytes (`@ptrCast`, Io.zig:2189) — an element holding a slice transfers only the pointer, so
|
||||
queue elements must be self-contained values.
|
||||
- `std.Io.Condition` has NO `timedWait` (Io.zig:1653 — wait/waitUncancelable/signal/broadcast
|
||||
only). The timed primitive is `std.Io.Event.waitTimeout` (Io.zig:1827).
|
||||
- Disk free space: std has NO statvfs/statfs wrapper (no Statfs struct, no `f_bavail` anywhere in
|
||||
lib/std). Options: `extern fn statvfs` against libc (we always link libc for sqlite) or raw
|
||||
`std.os.linux.syscall2(.statfs, ...)` with a hand-written struct.
|
||||
- File append mode does not exist on `Dir.OpenFileOptions`/`CreateFileOptions`. Append pattern:
|
||||
`openFile(io, path, .{ .mode = .write_only })`, `file.writer(io, &buf)`, then
|
||||
`w.seekTo(try file.length(io))`; the writer is positional. `File.setLength` truncates;
|
||||
`Dir.rename` + `Dir.deleteFile` rotate. Positional writes do not serialize — one task owns the
|
||||
log writer.
|
||||
- No fixed-capacity hash map in std. Bounded cache shape: fixed slot array plus
|
||||
`StringHashMapUnmanaged(u32)` from key to slot index (`ensureTotalCapacity` once at init);
|
||||
a CLOCK hand walks the stable slot array, never the map. Any map modification invalidates
|
||||
live iterators (hash_map.zig:496).
|
||||
- `std.Io.Queue(Elem)` exists (Io.zig:2184): bounded ring buffer over a caller-supplied array. `put(q, io, elems, min)` with `min = 0` never blocks and returns 0 when full (Io.zig:2218); `getOne` blocks until an item or `error.Closed` after `close(io)`. Elements are copied as raw bytes (`@ptrCast`, Io.zig:2189) — an element holding a slice transfers only the pointer, so queue elements must be self-contained values.
|
||||
- `std.Io.Condition` has NO `timedWait` (Io.zig:1653 — wait/waitUncancelable/signal/broadcast only). The timed primitive is `std.Io.Event.waitTimeout` (Io.zig:1827).
|
||||
- Disk free space: std has NO statvfs/statfs wrapper (no Statfs struct, no `f_bavail` anywhere in lib/std). Options: `extern fn statvfs` against libc (we always link libc for sqlite) or raw `std.os.linux.syscall2(.statfs, ...)` with a hand-written struct.
|
||||
- File append mode does not exist on `Dir.OpenFileOptions`/`CreateFileOptions`. Append pattern: `openFile(io, path, .{ .mode = .write_only })`, `file.writer(io, &buf)`, then `w.seekTo(try file.length(io))`; the writer is positional. `File.setLength` truncates; `Dir.rename` + `Dir.deleteFile` rotate. Positional writes do not serialize — one task owns the log writer.
|
||||
- No fixed-capacity hash map in std. Bounded cache shape: fixed slot array plus `StringHashMapUnmanaged(u32)` from key to slot index (`ensureTotalCapacity` once at init); a CLOCK hand walks the stable slot array, never the map. Any map modification invalidates live iterators (hash_map.zig:496).
|
||||
|
||||
Reference in New Issue
Block a user