# Milestone 34: hot-apply — tiers A and B DB-mode config changes apply live, in-process, for every key that does not create a socket. The comptime "every key is restart-required" blanket is replaced by an explicit per-key apply table. Design: rev 3 of the hot-reload analysis; spec hardened through two Codex review rounds (thread 01a02efc). Listener rebind and `web.enabled` lifecycle are milestone 35. ## The write contract (every session honors it) Prepare → commit → publish → retire: 1. **Prepare**: under the serialized config-mutation path, build and validate everything the change needs, from the FINAL MERGED config — ONE candidate per affected owner, never one per key. Prepared resources are heap-stable and owned (a candidate outlives the request arena). Nothing is published. Any failure: clean up every prepared resource, error to the client, NO db write. 2. **Commit**: the DB transaction, only after every prepare succeeded. Commit failure: clean up all prepared resources. 3. **Publish**: infallible, I/O-free operations — handle swaps, pointer swaps, stores under a lock. Closing files, joining tasks, freeing memory belong to retire, not publish. 4. **Retire**: old generations/handles are drained, closed, and freed after their readers release. Signatures: every operation that touches a `std.Io` primitive (Mutex, RwLock, Condition, concurrent, sleep) takes `io: std.Io`. The spec's named signatures include it; a builder adds it wherever else the primitive demands it. ## Sessions Six sessions, strictly sequential (S1 → … → S6); each session owns the tree while it runs and ends with `zig build test` green (S6 adds the admin suite). **Honesty rule**: `restart_pending`/`restart_required` keep firing exactly as today until S5 swaps the mechanism atomically. No earlier session removes a restart signal. --- ## Session S1: per-query policy snapshots (tier A) ### S1.1 DNS policy snapshot `src/server/handler.zig`: the per-query reads — blocking response + ttl, `ecs_mode`, `forward_read_timeout`, `negative_ttl_max` — move into one `Policy` struct behind `std.Io.RwLock` (the filter discipline, manager.zig:438/856). Each query copies the `Policy` ONCE at query start (shared lock only for the copy); `ecs_mode` at :782/:829/:835 reads the copy. Publish: `pub fn setPolicy(h: *Handler, io: std.Io, p: Policy)`. ### S1.2 Web trusted proxies `src/web/server.zig` ~:531: `trusted_proxies` becomes an owned immutable generation in `WebState`, pointer-replaced under an RwLock (io-threaded), old generation retired after readers release. Prepare copies out of the request arena. ### S1.3 Logger config split `src/storage/logger.zig`: `hide_domains` + `hide_client_ips` (applied on the PRODUCER path ~:449) are ONE privacy policy: both pack into a single atomic (one u8), stored together by `setPrivacy` and loaded ONCE per entry — a producer can never observe a mixed policy that redacts the domain but exposes the client, or vice versa. `query_log_flush_interval_s` (~:609) is an independent `.monotonic` atomic with `setFlushInterval`. ### S1.4 Disk thresholds `src/storage/disk_monitor.zig`: `min_free_mb`/`warn_free_mb` are one invariant pair (warn ≥ min): both u32s packed into ONE atomic u64; readers unpack a single load. ### S1 implementation notes (post-build) - `Context`'s provenance method `policy(...)` renamed `notePolicy` (collision with the new per-query field). - Pure-atomic setters (`setPrivacy`, `setFlushInterval`, `setThresholds`) take no `io` — they touch no Io primitive. - `LiveProxies.install` RETURNS the retired generation; the caller frees it in retire. - The trusted-proxies read hold ends at the client-address verdict, BEFORE router dispatch — a hold spanning dispatch would let a settings PUT deadlock on the shared lock its own request holds. - The flush-interval test covers short→long only; long→short is unobservable without waking a writer already parked on its old deadline (S5's PUT test inherits this bound). - S1 setters have no production caller until S5 wires the PUT flow — test-only until then, by design. ### S1.5 Acceptance criteria - [ ] One query is internally consistent across a concurrent `setPolicy` (both ecs/blocking reads agree). - [ ] trusted_proxies replaced under concurrent request-path reads; testing allocator clean. - [ ] Privacy flip with FORCED producer interleaving: pre-flip entries unredacted, post-flip redacted. - [ ] Threshold reader vs concurrent pair-stores: every observed pair satisfies warn ≥ min. - [ ] Flush-interval change observed on the writer's next cycle via the existing timing seam. - [ ] `zig build test` green. --- ## Session S2: upstream extraction + generation owner + lock hygiene ### S2.1 Extract the upstream composition out of app.zig `Upstreams`, `build`, `deinit`, and what they need from `ConfigLoad` (all private in app.zig ~:1158) move to `src/upstream/owner.zig`; app.zig imports it, never the reverse. `build` returns the generation plus a `BuildReport`. `ConfigLoad.note` is NOT just log output — it writes operational-event rows and retains canonical keys for `finalize` (app.zig:367). Contract: at BOOT, app.zig replays the successful `BuildReport` through the exact `ConfigLoad.note` path before `finalize`, so diagnostics are unchanged; at RUNTIME, candidate preparation is side-effect-free (no event rows) until commit, and the report is RECONCILED in RETIRE (after the pointer publish — event rows are SQLite I/O, banned from publish): retire reconciles SCOPED, not via `ConfigLoad.finalize` (finalize calls `Store.resolveExcept`, events.zig:402, which resolves EVERY active configuration.load event outside its kept set — at runtime that would falsely resolve unrelated boot warnings). Each generation OWNS its report keys (stored in the generation), and the retire payload is STABLE: prepare copies the LIVE generation's report keys (under the owner mutex) into the PUT-owned apply state, so reconciliation never depends on the old generation's lifetime and NEVER runs from a query's `release` path — it runs on the PUT task in retire. Retire: emit the new report's events, then individually resolve exactly `copied_previous_keys − new_report_keys`. S5 tests through the real PUT path: introducing a malformed upstream raises the warning; fixing it resolves it; an UNRELATED active configuration warning survives the whole introduce/fix cycle. ### S2.2 UpstreamOwner Discipline: CertStore's (cert_store.zig:199/:237) — a mutex held briefly for refcounted borrows. - `pub fn acquire(o: *Owner, io: std.Io) *Generation` — lock, ++refs, return. - `pub fn release(o: *Owner, io: std.Io, g: *Generation)` — lock, --refs; a release that drops a RETIRED generation to zero deinits it (Upstreams.deinit needs io). - `pub fn replace(o: *Owner, io: std.Io, prepared: *Generation) ?*Generation` — lock, swap the live pointer, mark old retired; if the old generation's refs are ALREADY zero, return it for the caller to retire immediately (the CertStore refs==0 pattern) — otherwise null and the last release retires it. - `pub fn deinit(o: *Owner, io: std.Io)` — shutdown teardown of the live generation, called by serve() after listeners and metrics readers have stopped. - Prepared generations are heap-allocated and OWN every configuration string (URL text — transport.Endpoint borrows it, transport.zig:51 — plus tls_name, host, path): each generation carries its own arena covering every borrowed row string until retirement. Candidates built from request-arena rows copy into that arena at prepare. - Timeouts are baked into the generation; a timeout change is a replace. - Query path: `Handler` stores `*Owner` (replacing the boot-time type-erased client, handler.zig:91). Per exchange: acquire → generation's `transport.Client` → exchange → copy the resolver identity from `selected` (borrowed from the endpoint, transport.zig:350) into the PER-QUERY `Context.upstream_buf` (handler.zig:453 — never a Handler-owned buffer; Handler is shared across concurrent queries) → release. - Metrics/health: `WebState` stores `*Owner` (replacing `*Pool`, web/server.zig:144); a scrape acquires for its duration. ### S2.3 Cache and rate-limiter lock hygiene - handler.zig:799 and app.zig:1124: cache pointer loads move inside `cache_mutex`; then `replaceCache` swaps under it (entries lost — accepted). - handler.zig:189: same for the rate limiter; `replaceRateLimiter` under its lock (windows reset — accepted). ### S2 implementation notes (post-build) - The retire-time scoped reconciler is DEFERRED TO S5 (S2 has no runtime replace caller; uncalled code fails the values). S2 delivers its precondition: generations own copyable report keys. **S5 must implement the reconciler + its tests (including unrelated-warning-survives).** - `Generation` gains a test-only `borrowing`/`borrowingPool` payload so ~90 existing test sites keep their fake clients without heap generations; production always takes the built path. - One skipped-upstream log string reworded to unify with the event detail; the event detail (what the criterion asserts) is byte-identical. - Integration-gated call sites were mechanically touched (comptime-dead without -Dintegration; would have broken that build). ### S2.4 Acceptance criteria - [ ] Concurrent exchanges vs `replace`: in-flight completes on G1; G1 deinits only after last release (or immediately when refs==0 at replace); new exchanges on G2; allocator clean. - [ ] A replace while NO reader holds G1 retires G1 via the replace return path (refs==0 branch covered). - [ ] Generation string ownership: build a candidate from a transient arena, free the arena, exchange still reads valid url/tls_name (allocator-poisoning test). - [ ] Metrics scrape concurrent with replace: clean. - [ ] Cache and rate-limiter swaps under concurrent use: clean. - [ ] Boot diagnostics unchanged: the events ConfigLoad.note wrote before the extraction are written identically (assert on the events store, not stdout). - [ ] Restart signals untouched this session. - [ ] `zig build test` green. --- ## Session S3: in-place reconfigure, retention, certs, sink, scheduler ### S3.1 Sessions TTL Slots gain `issued_at` beside `expires_at` (auth.zig:311). `setTtl(io, ttl)` recomputes each live slot: `expires_at = issued_at + ttl` under the existing mutex. Semantics are SERVER-SIDE: shortening takes effect for every session; lengthening is bounded by the browser cookie's original Max-Age (the cookie is not refreshed — sessions do not become sliding; state this in a comment). Login cookie Max-Age (handlers/auth.zig:127) reads the LIVE ttl. ### S3.2 ApiLimiter ALL config reads move under the existing mutex (`check` reads `localhost_exempt` pre-lock, api_limiter.zig:135). `setLimits(io, now, limits)`: refill each bucket THROUGH `now` at the old rate, clamp tokens to the new capacity, then install the new rate — no retroactive refill at the new rate, and never move a bucket's clock backward (a bucket already newer than `now` is clamped only). SSE per-IP counts untouched. ### S3.3 Retention days Two consumers (retention.zig:103, clients.zig:228): one shared `.monotonic` atomic u32 owned by app-level state, read per pass by both; `setRetentionDays` stores. ### S3.4 Certificate paths `doh_server.cert_path/key_path`, `dot_server.cert_path/key_path` on an ENABLED endpoint: CertStore's paths (borrowed boot slices, reread by reload at cert_store.zig:111, read outside `mutex` by reload/pollOnce at :226/:268) become owned, replaceable strings. The whole apply is serialized under the store's existing `reload_mutex` — the SAME mutex reload holds across load+publication: prepare (under reload_mutex) loads the candidate cert+key from the new paths; publish (still under reload_mutex, taking the generation `mutex` only for the swap — lock order: reload_mutex outer, mutex inner, matching reload today) swaps paths and generation together. `pollOnce` currently reads the paths BEFORE taking reload_mutex (cert_store.zig:268): it now takes reload_mutex around its path reads and calls a non-locking `reloadLocked` helper (reload becomes reload_mutex-lock + `reloadLocked`) so acquisition is never recursive. With that, no concurrent path borrow can outlive an apply. Connections pinning the old cert finish on it. `POST /api/certs/reload` rereads the owned paths. On a DISABLED endpoint no CertStore exists (handlers/certs.zig:31 — `doh_certs`/`dot_certs` are null): the change is DB-ONLY, and the key's table entry says so; the paths are validated when milestone 35 implements enable. This is stated in the table note and the API docs. ### S3.5 Log sink Three disjoint cases, decided from the merged config — no reuse marker, no lock spanning prepare and publish: - **File target changed** (the merged config has output=file AND the path differs from the current file target, or output switches TO file): prepare opens the NEW target (fallible, closing the TOCTOU in logging.zig:228's close-then-reopen) into an owned `PreparedSink` that carries the COMPLETE target state: the open file, its MEASURED length as the new `file_pos`, and `rotate_pending = false` (the handle couples to both fields, logging.zig:196 — inheriting the old position corrupts writes; inheriting a pending rotation rotates the new target spuriously). Publish installs `{file, file_pos, rotate_pending}` + config atomically under the sink's existing lock; the DETACHED old handle closes in retire. Races can only touch the OLD state, replaced wholesale. - **File target removed** (output switches FROM file to stderr/syslog): nothing to prepare; publish installs `{file = null, file_pos = 0, rotate_pending = false}` + config atomically; the detached file closes in retire. - **File target unchanged** (every remaining case: output stays stderr/syslog, or output=file with the same path — level, limits, or a file_path change while output is non-file): publish updates ONLY the config fields under the sink lock and NEVER touches the handle or its position/rotation state, which stay owned by the existing rotation/recovery machinery. A same-path config apply does not repair a broken handle (the existing per-write recovery does). No prepare-time handle inspection exists, so there is nothing to race. (A file_path change while output is stderr still updates the config so a later output=file switch — its own apply — opens the right target.) The disk monitor's measured directory derives from the final merged `output + file_path` on EVERY log_sink apply: output `file` → the file's directory (installed even if only output changed); output stderr/syslog → cleared (monitor stops measuring a log dir). `Monitor.setLogDir(io, owned_path_or_null)`; `sample` borrows `log_dir_path` across directory I/O (disk_monitor.zig:112), so the path is a pinned generation — sample acquires (refcount or lock held for the borrow), setLogDir swaps, old path freed after the borrow releases. Owned path prepared pre-commit. ### S3.6 Wakeable blocklist scheduler manager.zig:1493 restructured into a wakeable loop parked on a condition: - The startup refresh pass runs exactly as today, including when disabled. - `setSchedule(io, enabled, interval_hours)` stores under the scheduler's mutex with a version counter (no lost wakes between check and park) and signals. - Anchor rule: the anchor is the completion time of the last refresh pass that RAN — success or failure both advance it; a disk-gate skip ALSO advances it (today's semantics: the scheduled slot is skipped, not retried early). Next refresh = anchor + interval; if that is in the past at set/enable time, refresh immediately. - Disabled parks; the task exits only on shutdown, exactly as today. - Production wake primitive: `std.Io.Event.waitTimeout` (Condition has no timed wait in 0.16). The Event is STICKY after `set` — the reset sequence prevents both spinning and lost wakes: under the scheduler's mutex the loop reads the version, RESETS the event, recomputes its deadline, releases the mutex, then waits; `setSchedule` (under the same mutex) bumps the version, then sets the event. A set that lands between the loop's reset and its wait completes the wait immediately; the version recheck decides whether anything changed. - Test seam: an injectable clock/step seam — validated intervals are ≥ 1 h; acceptance tests must not sleep real time. ### S3 implementation notes (post-build) - The disabled-endpoint cert criterion (DB-only, no store call, no-restart response) is PURELY a PUT concern — **deferred to S5's obligations** beside the S2 reconciler. - `model.retentionSeconds` deleted (dead once both consumers read the shared cell); `RetentionDays.seconds()` is the single conversion point. - A disabled scheduler PARKS; the seam models shutdown (`shutdown_at_first_park` → error.Canceled) — the loop's only exit is shutdown. - `publishPathChange` reads no clock (loaded_at captured at prepare). - `Monitor.deinit(io)` added to free an installed log-dir generation (no-op at boot — arena-borrowed). - `setLimits` takes the full limiter `Config` (the three fields ARE the config; a twin struct would be invented generality). - Sink race criteria proven as the two reachable interleavings (publish shares the stderr lock with rotation/closure — no third ordering exists); the monitor criterion is a real two-task race. ### S3.7 Acceptance criteria - [ ] TTL: shortening expires an over-age live session immediately; lengthening extends server-side validity; a fresh login's cookie Max-Age reflects the live ttl. - [ ] Limiter: no pre-lock config read remains; refill-through-now at old rate proven with a controlled clock; capacity cut clamps; SSE counts survive. - [ ] Retention: both consumers observe a change on their next pass. - [ ] Certs: bad candidate refused at prepare, store untouched; good change serves the new cert on the next handshake (-Dintegration loopback); a concurrent reload during an apply is serialized (test drives both under the seam). - [ ] Disabled-endpoint cert change: DB row changes, no store call, response marks no restart. - [ ] Sink: publish does no open/close (close observed in retire); bad path refused at prepare; new file receives lines; monitor samples the new dir; no use-after-free under concurrent sample; a same-path config-only apply concurrent with a forced rotation AND with a write-failure closure changes only config fields, never the handle; a target-change apply racing rotation swaps cleanly (old handle closed in retire); switching to a PRE-EXISTING nonempty file starts at its measured length; switching away from a target with `rotate_pending` set does not rotate the new target; a `file → stderr` apply detaches the handle (closed in retire) and clears position/rotation state. - [ ] Scheduler via the seam: shortened interval → next at new cadence; disable parks; re-enable anchors per rule; startup pass runs when disabled; failed pass advances the anchor; gate-skip advances the anchor. - [ ] `zig build test` green. --- ## Session S4: logger queue controller ### S4.1 Controller scope `logging.query_log_buffer_max`. A stable controller owns the logger generation: buffer, `Logger`, writer future, and the writer's DB/gate dependencies (writer holds prepared statements for its run, logger.zig:514; future owned by serve today, app.zig:897). serve() creates the controller and delegates; shutdown ordering through the controller is EXACTLY today's safe order — quiesce producers → close queue / set draining → the writer drains the CLOSED queue → await the future (logger.zig:632, app.zig:921; never "drain then close": an empty writer blocks in getOne until close). Facade: `QuerySink` (query_sink.zig:18), metrics (web/metrics.zig:207) and health (web/handlers/health.zig:241, via web/server.zig:150) hold the CONTROLLER, not `*Logger`. The controller owns what must be continuous across swaps: counters, gate-episode state, `last_drop_s`, diagnostics references, AND the S1 privacy/flush atomics (setters address the controller and survive resize). `writer_failed` reflects the LIVE writer: a successful resize publish clears it (the new writer prepared cleanly); a retired writer's failures still land in the drop/diagnostic counters. Producer read-side protocol: a producer's `log()` acquires the live generation through the facade with a refcount (or a lock held through transform + enqueue) — a producer can never enqueue into a queue that retire has closed; retire waits for old-generation producer borrows to release before closing the old queue. This is what makes "no resize drop window" true, not an aspiration. ### S4.2 Resize — all fallibility before commit 1. **Prepare** (fallible): validate against the comptime ceiling (37449, startup's message); allocate the new buffer and Logger state; OPEN A NEW querylog DB connection for the new generation — one connection per writer for its whole life (logger.zig:514); two writers must never share `querylog_writer_db`. The connection factory: the canonical writer-connection opener (today `cli.DataDir.reopenQuerylogDb` + `db.applyPragmas`, cli.zig:350) MOVES into `src/storage` (a pub fn beside `querylog_schema.open` taking the data-dir handle + path); cli.zig delegates to it, and the controller receives the factory inputs at construction. The controller owns each generation's connection and closes it after that generation's writer is joined — and SPAWN the new writer PARKED: the spawn (`io.concurrent` can fail, Io.zig:2352) and statement preparation (runWriter can fail, logger.zig:524) happen NOW; the parked writer signals ready and waits on an activation gate. Any failure: tear down, close the new connection; nothing changed. If a PREVIOUS resize's retired generation has not finished retiring, prepare REFUSES with a distinct error ("previous resize still draining") — at most one retired generation exists, which bounds writers, connections, and buffers. 2. **Commit** the DB row. 3. **Publish** (infallible): swap the facade's live-generation pointer and open the activation gate. No waiting. 4. **Retire** (controller-owned reaper): wait for old-producer borrows to release → close the old queue WITHOUT setting the process-shutdown `draining` flag (a distinct retirement mode: with `draining` set, a gate-held writer takes the GatedAtShutdown path and DROPS its batch, logger.zig:642/:726 — retirement instead lets it flush when the gate reopens) → the old writer drains the closed queue → await its future → close its DB connection → free logger + buffer. Reaper ownership: the reaper task is spawned ONCE at controller construction (boot — fallibility there is fine) and lives for the controller's life, processing retirements signaled by publish; publish never spawns. `Future.await` is not thread-safe (Io.zig:1198), so the reaper is the SOLE joiner of retired writers. Process shutdown, in order: quiesce producers → CLOSE the live queue (a writer blocked in getOne wakes only on close, logger.zig:642) → set `draining` on EVERY outstanding generation (live and retired — turning a gate-parked retired writer onto the counted GatedAtShutdown path) → join the LIVE writer → signal and join the REAPER (which finishes joining any retired writer). The f1a85d3 ordering holds within each join. Accounting: every entry accepted into the old queue is written when the gate allows, or counted by the existing gate/shutdown machinery; entries after publish land in the new queue. No resize-specific drop path exists. ### S4 implementation notes (post-build) - The controller is a new file, `src/storage/logger_controller.zig`, not a second half of `logger.zig` (2190 lines before this session, and generations/retirement/the reaper are a separate concern from `Entry` and the writer loop). Add it to the module layout. - `Logger` keeps its exact public surface, so every f1a85d3 test passes unchanged. Continuity is achieved by SUMMING rather than by sharing cells: `Controller.sample` folds `base` (finals of joined generations) + the live generation + the outstanding retired one, under the controller mutex, and folds a retired generation's finals into `base` before freeing it. `last_drop_s` is a max, the gate episode is the worst of the outstanding generations, `writer_failed` is the live one's. - `Logger.runWriter` is split: it prepares its statements and calls the new `pub fn runPrepared(io, writer, monitor)`. A resize generation prepares separately so a statement failure is a refused settings change. `Logger.retire(io)` is the close-without-`draining` retirement mode. - The BOOT generation deliberately takes the plain `runWriter` path, not the parked one: a boot that cannot prepare must still serve DNS with `writer_failed` set, which is today's behaviour. Only a resize can afford to refuse. - The connection factory is `querylog_schema.reopen(io, dir, path)`; `cli.DataDir.reopenQuerylogDb` delegates. It takes the dir handle to make `open`'s resolution pairing explicit at every call site, and does not dereference it. - The setters are `Controller.setPrivacy(io, p)` / `setFlushInterval(io, s)` and take `io` (they lock), unlike S1's pure-atomic pair. They store to every outstanding generation; each generation still keeps ONE packed privacy word, so a producer's single load stays the mixed-policy guarantee. - `Controller.prepare` returns typed errors; `sizeMessage(err, requested, buf)` renders `config/validate.zig`'s exact wording for S5's client error. - `Controller.retirementPending(io)` is the "previous resize still draining" predicate, exposed for S5 and the tests. - Test sites keep their stack `Logger`s through `logger_controller.Borrowed` — S2's `upstream_owner.Borrowed` pattern. - **S5 obligations unchanged**: the S2 retire-time scoped reconciler and the S3 disabled-endpoint cert criterion. S4 adds none. ### S4.3 Acceptance criteria - [ ] Resize under forced concurrent producers: no deadlock; exact accounting — old-queue entries all written (or gate-counted), new-queue entries written, produced == written + counted; no resize-specific drops. - [ ] Resize with the disk gate CLOSED: publish completes immediately; old writer retires after the gate reopens; nothing lost beyond what the gate itself counts. - [ ] Prepare failure (spawn or statement prep) leaves the running logger untouched and writes no DB row. - [ ] Invalid size refused with startup's message. - [ ] Counters/gate state/`last_drop_s` continuous across a swap (metrics read before and after agree modulo new writes). - [ ] Privacy interleaving test pauses a producer between its two field decisions across a concurrent `setPrivacy`: a mixed policy (domain redacted, client exposed, or the reverse) is impossible. - [ ] Shutdown while a retirement is gate-blocked: clean join, the retired writer's held batch is counted by GatedAtShutdown, no deadlock, no double-await. - [ ] f1a85d3 deadlock regression tests pass unchanged. - [ ] `zig build test` green. --- ## Session S5: the apply table and the settings write path ### S5.1 The key table `src/web/handlers/settings.zig`: delete the comptime `restart_required_keys` generation (:68-95) and `touchesRestartRequiredKey` (:187-200). The table maps EVERY settings key to a CONCRETE operation enum — `dns_policy`, `trusted_proxies`, `logger_privacy`, `logger_flush`, `disk_thresholds`, `upstream_generation`, `cache`, `rate_limiter`, `sessions_ttl`, `api_limiter`, `retention`, `certs_doh`, `certs_dot`, `log_sink`, `scheduler`, `logger_queue`, `bind`, `web_lifecycle` — one entry per key, no second unguarded switch; the broad class (live/subsystem/bind/web_lifecycle) is DERIVED from the operation. `bind` (dns/web/doh/dot bind + port + doh/dot enabled) and `web_lifecycle` (`web.enabled`) set `restart_pending` exactly as today; milestone 35 executes them. Comptime test: walk `@typeInfo(model.Config)` as the old generator did; every scalar key appears exactly once (replaces :549). ### S5.2 The PUT flow validate → group changed keys by OPERATION → prepare one candidate per affected owner from the final merged config → any failure cleans up all prepared candidates, errors, no DB write → commit (failure: clean up) → publish each prepared candidate → retire. `web.password` keeps its existing path. Upstream create/update/delete (handlers/upstreams.zig): build the candidate generation from the HYPOTHETICAL post-mutation row set before the repository write; commit; publish. The direct `restart_pending` sets (:58/:90/:114) are deleted HERE. `restart_required` in upstream responses (upstreams.zig:156) becomes `false`; update the OpenAPI `UpstreamEcho` pin (openapi.yaml:2749), the `/api/settings` description that says every scalar setting requires restart (openapi.yaml:1699), the API reference claim that all upstream/settings mutations do (docs/reference/api.md:18), generated types, and contract samples. ### S5.3 Config status `GET /api/config/status`: `restart_pending` remains, fed only by `bind`/`web_lifecycle` operations. The settings envelope's `restart_required` list (:538) shrinks to those keys. ### S5 implementation notes (post-build) - The table and the four phases live in a new file, `src/web/handlers/apply.zig`, not in `settings.zig`: the upstream RESOURCE handlers need the same prepare/publish/retire, so putting it in the settings handler would have made one handler the other's library. `settings.zig` re-exports `restart_required_keys` and nothing else of it. Add it to the module layout. - Operations are derived from the VALUES, not from the keys the patch named (`changedOperations(before, after)`). The admin form submits every field it read, and treating that as eighteen applies would resize the query log on every save — and refuse the second save with `PreviousResizeDraining`. A key rewritten to what it already was changes nothing, applies nothing, and owes no restart. - `app.zig` now heap-allocates the DNS cache and rate limiter and frees them THROUGH the handler (`replaceCache(io, null)` at teardown): after a `cache.size` apply, what the handler holds is not what boot built. Both are built inside one block whose errdefers end with it, so a boot that fails between the two creations frees them and a boot that fails later leaves them to the handler's teardown defers. - The OpenAPI overview and the three upstream mutation descriptions said restart-required. They now say the change is live, which is what the runtime does and what the `restart_required: false` pin already promised. `docs/reference/api.md` was already correct. `contractSamples.gen.ts` is generated from live responses rather than from the document, so no regeneration followed. - `WebState` gains `upstream_build` (the shared HTTP client, certificate bundle and its lock) and `retention_days`. Without the first, an upstream mutation is a database row and nothing else, which is what a handler test's borrowed owner wants. - `Owner.published` is a plain `u64` under the owner mutex, with `publishedCount(io)`, not an atomic: it is the observable the "one candidate per owner" tests read. An atomic counter in the same struct reproducibly perturbed the S2 test "a replace with no reader holding the live generation retires it through the return path" into failing (~1 run in 1); a mutex-guarded field is both the discipline the rest of `Owner` follows and stable. **That S2 test's sensitivity to unrelated timing is worth a look on its own.** - `mutations.Resource` now accepts a `remove` that returns `error{OutOfMemory}!?Failure`, because a delete that builds a candidate allocates. Both shapes are still checked exactly. - The upstream note rendering moved to `upstream_owner.Rendered`, shared by `app.zig`'s boot replay and the runtime reconciler, so a warning raised at boot and the same warning raised by a write are byte-identical. - Reconciliation reads the new report by RE-ACQUIRING the owner rather than keeping the published pointer: a concurrent second write could retire and free that generation the moment its refs hit zero. - The S2 reconciler test drives its rebuilds through `/api/upstreams`, not `/api/settings`: a settings PUT validates the whole stored configuration and refuses a row malformed enough to produce a build finding, so the finding can never be reached from there. - `logging.file_path` must be absolute, so the integration environment resolves its tmp dir with `Dir.realPath`. - `Plan.prepare` abandons the partly built plan itself on a PROPAGATED error (`errdefer self.abandon(io)`), and only returns a `Failure` for the caller to abandon. An `OutOfMemory` out of the log-directory step used to bypass the caller's `abandon`, leaking whatever earlier owners had built and — because `CertStore` holds `reload_mutex` from its prepare to its publish — wedging every later certificate change and reload. - `Controller.prepare` takes the merged `model.Logging`, not an entry count. Seeding the candidate from the LIVE privacy and flush values was wrong for the one PUT that changes the buffer size and a privacy flag together: publish applies the privacy to the generation being retired, so the replacement went live writing what the operator had just asked to hide. - The `api_limiter` reading is taken in prepare (`Plan.limiter_now`) and used in publish. A clock is I/O, and publish is I/O-free. That reading is STALE by publish time whenever a request served in between refilled a bucket past it, so `setLimits` is monotonic per bucket: a bucket already newer than `now` is clamped to the new capacity and keeps its clock. Rewinding it would let the next request buy the same interval a second time, at the new rate. - `metrics.collect` loads `handler.cache` and `handler.limiter` INSIDE the mutexes that guard them, the same way the request path does: `Plan.retire` frees the displaced object as soon as the swap returns, so a pointer read before the lock can be freed under the reader. - `CertStore.publishPathChange` frees the displaced entry and the old paths BEFORE bumping `reloads`. A `bool` live across an atomic read-modify-write is the Debug-backend miscompile AGENTS.md documents. - Two S5.4 criteria are proven one step short of the wording. `certs_doh`/`certs_dot` assert the store reloaded and the owned paths moved, not a TLS handshake against the new certificate; `dns_policy` asserts `policySnapshot`, which is the read a query performs, not a query. Both are the collaborator's own observable, and neither gap is a claim about a path that is untested. ### S5.4 Acceptance criteria — one real-PUT test per OPERATION Every operation proven through the real route dispatch → prepare → commit → publish path: - [ ] dns_policy (blocking.ttl live on next query — integration), trusted_proxies, logger_privacy, logger_flush, disk_thresholds, cache (size change swaps, next lookup misses), rate_limiter, sessions_ttl, api_limiter, retention, certs_doh AND certs_dot separately (enabled: handshake serves new cert — integration; disabled: DB-only), log_sink (+ monitor re-point on path change AND on output change both directions), scheduler (via seam), logger_queue (accounting per S4), upstream_generation (row mutation: next exchange uses it, response `restart_required: false`; timeout change: generation counter +1). - [ ] web_lifecycle: a `web.enabled` PUT commits the DB row, sets `restart_pending`, and executes NOTHING (milestone 35 executes it). - [ ] A multi-key PUT touching one owner twice builds ONE candidate (generation counter delta == 1). - [ ] An upstream PUT while G1 is held by an in-flight exchange: publish + diagnostics reconciliation complete on the PUT task; G1 retires later via release; events correct throughout. - [ ] A PUT mixing a live key with a failing prepare writes NOTHING and changes nothing. - [ ] A `bind` key PUT still sets `restart_pending`; nothing else can. - [ ] Contract samples for `/api/config/status`, settings envelope, upstream responses regenerate; route-count pin passes. - [ ] `zig build test` and `zig build test -Dintegration` green. --- ## Session S6: admin SPA - Per-field "needs restart" tags (SettingsForm.tsx:119-123/:209-223/:267) render only for keys the API lists (bind + web_lifecycle). Banner logic untouched. Upstream flows lose restart messaging; regenerated types carry `restart_required: false`. Live fields say nothing — silence is success. Regenerate goldens (`just goldens`). - [ ] From `admin/`: `npm run typecheck && npm test && npm run lint && npm run format:check && npm run build && npm run assert-bundled` green; bundle under ceiling. - [ ] A test: live-key edit renders no restart affordance; a port edit still does. ### S6 implementation notes (post-build) - `SettingsForm` needed no change at all. It already built its mark set from `envelope.restart_required` and nothing else, so the twelve-key list arrived and the other fields went quiet on their own. The session's real work was the copy and the tests that had pinned the old answer. - The tests now take the key list from `sample_get_settings.restart_required` in the committed contract sample, re-exported as `RESTART_REQUIRED_KEYS` from the configuration fixtures, rather than from a hand-written array. A server-side change to the set now fails the tests that pin it instead of passing against a stale copy. - One test keeps a hand-written list, `["logging.level"]`: no shipped restart-required key is enum-backed, so the `Select` rendering — where the mark reaches a screen reader through `aria-describedby`, not through the label — has no key left to exercise it. The list is the server's to change and the form must render any key it names, so the branch and its test stay, with the override stated in the test. - `invalidateUpstreams` dropped its `/api/config/status` invalidation. An upstream write rebuilds the pool in the running process, so the status answer after the write is the answer before it, and re-reading it only asserted a claim that is no longer true. - `just goldens` was not run: S5 regenerated the samples and this session changed no server response. --- ## Module layout (new/changed) `src/upstream/owner.zig` (new); `src/storage/logger_controller.zig` (new); `src/storage/querylog_schema.zig`; `src/cli.zig`; `src/server/handler.zig`; `src/server/query_sink.zig`; `src/web/server.zig`; `src/storage/logger.zig`; `src/storage/disk_monitor.zig`; `src/storage/retention.zig`; `src/server/clients.zig`; `src/platform/logging.zig`; `src/filter/manager.zig`; `src/server/cert_store.zig`; `src/web/auth.zig` + `src/web/handlers/auth.zig`; `src/web/api_limiter.zig`; `src/web/handlers/apply.zig` (new); `src/web/handlers/settings.zig`; `src/web/handlers/upstreams.zig`; `src/web/handlers/certs.zig`; `src/web/metrics.zig`; `src/web/handlers/health.zig`; `src/web/openapi.yaml`; `docs/reference/api.md`; `src/app.zig`; `admin/src/features/configuration/*`. ## File ownership Strictly sequential; each session owns the tree while it runs. ## Acceptance criteria (milestone complete) - [ ] Every settings key except `bind`/`web_lifecycle` applies live through its table-declared operation, each proven by a real-PUT test (S5.4). - [ ] `restart_pending` only from `bind`/`web_lifecycle`; no API response or doc claims a restart for anything else. - [ ] No fallible or I/O work in any publish; every prepare/commit failure leaves DB and runtime unchanged. - [ ] Full gates: `zig build test`, `zig build test -Dintegration`, admin suite from `admin/` incl. `assert-bundled`, `zig fmt --check build.zig src tools`, bundle ceiling. - [ ] Querylog schema fingerprint unchanged (cut's schema-gate takes the equal branch). ## Anti-requirements - No generic hot-swap framework; one named operation per owner. - No listener rebind, no `web.enabled` execution, no `applied: false` surface, no `restart_pending` removal — milestone 35. - No config file watcher; file mode unchanged. No new config keys; no DB schema changes. Sessions do not become sliding.