nxdns/specs/milestone-27.md

Milestone 27: diagnostics vertical slice

Step 1 of the UI redesign (specs/ui-redesign.md). A curated log of operational failure episodes: one operational_events table in config.db, a serialized event store, fifteen typed event codes emitted at existing failure sites, GET /api/diagnostics, and a Diagnostics page in the admin SPA. The rest of the UI stays intact; the full navigation restructure is later steps.

Design authority: specs/ui-redesign.md §Diagnostics, as amended by the Fable review rulings recorded in that file. No api.storage code. No periodic probe. No new index beyond the two specced.

Sessions

S1 (store + API) → S2 (emitters) and S3 (SPA) in parallel → orchestrator integration (contract samples regen, final wiring check).

---

Session S1: event store, schema, API, health, metrics

S1.1 Schema — src/storage/config_schema.zig

Append to ddl_v1 (pre-v0.1: edit the baseline, no migration step; update PLAN §11.2 to match):

CREATE TABLE operational_events (
    id INTEGER PRIMARY KEY,
    code TEXT NOT NULL,
    subject_key TEXT NOT NULL,
    subject_label TEXT NOT NULL,
    severity TEXT NOT NULL CHECK (severity IN ('warning', 'error')),
    first_seen INTEGER NOT NULL,
    last_seen INTEGER NOT NULL,
    occurrences INTEGER NOT NULL CHECK (occurrences > 0),
    resolved_at INTEGER,
    detail TEXT NOT NULL DEFAULT '',
    CHECK (resolved_at IS NULL OR resolved_at >= first_seen)
);
CREATE UNIQUE INDEX idx_operational_events_active
    ON operational_events(code, subject_key) WHERE resolved_at IS NULL;
CREATE INDEX idx_operational_events_last_seen
    ON operational_events(last_seen DESC);

operational_events is runtime state, NOT configuration. It stays out of table_names and delete_order (config_schema.zig:112-129). Tests that must be updated deliberately, not silently: migrations.zig:365 asserts count(tables) == delete_order.len + 1 — becomes + 2, and its comment names operational_events as the second exclusion alongside schema_version. src/config/reconcile.zig:268 (comptime assert(delete_order.len == 10)) is untouched — the list does not change. Add a test asserting export/import round-trips leave operational_events rows intact.

Editing ddl_v1 changes nothing for existing installs pre-0.1 EXCEPT the silent-divergence hazard already on record: the Pi's config.db is stamped version 1 and will not get the new table. This milestone accepts that with a bridge placed at the END of migrations.migrate, after the version stamp: execute the three operational_events statements with IF NOT EXISTS added. It must NOT live in cli.openConfigDb — openConfigDb runs before migration everywhere (app.zig:333, cli.zig:484, cli.zig:516), and creating the table there would make a fresh database's non-IF NOT EXISTS ddl_v1 CREATE TABLE fail. The bridge is explicit, commented as removable when the 0.1 adoption gate lands, and touches no other table. Test: a version-1 database created WITHOUT the table gains exactly it after migrate, and a fresh database migrates cleanly (no double-create).

S1.2 Event codes — src/storage/events.zig (new)

pub const Code = enum {
    disk_space, disk_probe,
    blocklist_refresh, blocklist_snapshot, blocklist_storage,
    certificate_reload,
    query_log_write, query_log_maintenance, query_log_recreated,
    upstream_history_write, upstream_exchange,
    client_names_storage, clients_storage,
    listener_start, configuration_load,
};

Fifteen enum members — that count is the single cardinality every exhaustive test, route fixture and frontend copy union uses. The enum is the truth; the wire form is the dotted string (disk_space → "disk.space", blocklist_refresh → "blocklist.refresh", etc.) via pub fn wire(code: Code) []const u8 — an exhaustive switch, tested against every member. component(code) returns the prefix before the dot, also exhaustive.

Severity is fixed per emit call, not per code (a disk transition to warn is warning, to critical is error).

S1.3 Event store — src/storage/events.zig

pub const Store = struct {
    pub const max_detail_len = 512;
    pub const max_subject_key_len = 256;   // longer keys become sha256: digests — see the identity rule below
    pub const max_subject_label_len = 128;
    pub const resolved_retention_s: i64 = 90 * 86_400;
    pub const max_resolved_rows: i64 = 5_000;

    mutex: std.Io.Mutex,
    database: *db.Db,              // dedicated connection; ALL access goes through the Store's mutex
    write_failed: std.atomic.Value(bool),
    write_failures: std.atomic.Value(u64),   // feeds the metrics counter
    active: ActiveSet,             // in-memory mirror of active (code, subject_key) rows
    untracked_active_count: u32,   // active rows NOT in the mirror (overflow); loaded at init

    pub fn init(io, database, now_s) db.Error!Store   // loads ActiveSet + untracked count, prunes; failure = no store
    pub const max_kept_keys = 64;   // resolveExcept bound; over it the call is refused, counted and latched — truncating the kept list would close episodes that are still true
    pub fn report(self, io, now_s, code, subject_key, subject_label, severity, detail) void
    pub fn resolve(self, io, now_s, code, subject_key) void
    pub fn reportResolved(self, io, now_s, code, subject_key, subject_label, severity, detail) void
    pub fn resolveExcept(self, io, now_s, code, kept_keys: []const []const u8) void
    pub fn prune(self, io, now_s) void
    pub fn writeFailed(self) bool
    pub fn activeCounts(self, io) struct { warnings: u32, errors: u32 }
    pub fn selectEvents(self, io, arena, filter) db.Error!EventsPage
    pub fn selectOne(self, io, arena, id: i64) db.Error!?Event
};

Contract:

• Time is a parameter, not a stored seam. Every mutating method takes now_s: i64 from the caller, matching how history.zig:132 receives wall_s and logger.zig:72 receives entry timestamps. Production callers compute it from Clock.real; tests pass literals. No function pointer, no clock inside the store.
• resolve is hot-path safe. It checks active under the mutex and, when the key is absent AND untracked_active_count == 0, returns without any SQLite statement. pool.recordSuccess calls it on every successful exchange; steady state must cost a mutex acquire and a lookup. The no-SQL guarantee is tested through a debug-only statement counter on the Store (incremented before every repo call) — NOT via sqlite3_total_changes, which a SELECT probe would not move.
• Overflow is exact, not heuristic. ActiveSet capacity is 256. The no-SQL fast path belongs to resolve ONLY: mirror miss + untracked_active_count == 0 → return, no SQL (that is the steady-state success). report on a mirror miss always writes — when the count is zero it inserts directly (a new episode in normal state, no probe needed); when the count is nonzero it probes and upserts (touch the untracked active row if one exists — a second insert would collide with the partial unique index — else insert), incrementing untracked_active_count only when a new row was inserted AND the mirror is full. A successful slow-path resolve decrements the count. init loads the count as active rows - mirrored rows. events_repo.resolveExcept performs the bulk resolve and a count of TOTAL remaining active rows in ONE transaction, returning that total only after commit — the repo knows nothing of the mirror. The Store then removes resolved keys from its mirror and sets untracked_active_count = total_active - active.len; on any failure it changes neither.
• Key canonicalization happens at every entry point. report, resolve, reportResolved and each kept_keys element of resolveExcept all pass the caller's key through the same digest-if-over-length rule before any lookup or SQL. Tests cover a long-key report resolved with the same long key, and resolveExcept keeping a long kept key.
• All reads go through the Store. The handler calls store.selectEvents / store.selectOne, which lock the same mutex around events_repo — nothing touches store.database from outside. One connection, one owner.
• report upserts on the active row: present → last_seen = now_s, occurrences += 1, detail replaced, severity raised to the worse of the two, never lowered. Absent → insert new active row and add to active.
• resolve on an active row sets resolved_at = now_s and removes it from the mirror. A later failure inserts a NEW row (new episode) — the partial unique index enforces one active row per key.
• reportResolved inserts a row with resolved_at = first_seen = last_seen = now_s, occurrences = 1, and never touches active (one-shot events: query_log_recreated).
• resolveExcept(code, kept_keys): resolves every active row of code whose subject_key is not in kept_keys, in one serialized operation. Exists for the boot-finalized codes (S2); nothing else may use it.
• Write failures: on db.Error, set write_failed = true, increment write_failures, drop the event. Log only on the false → true transition — a broken diagnostics database plus a busy pool must not produce warnings at query rate; the counter and the health surface carry the ongoing state. The next successful write clears the latch (and that recovery may log once).
• No error propagates to a producer — mutating methods return void by design; a diagnostics failure must never break the subsystem reporting it. init is the exception: it returns db.Error, and app.zig responds by running with no store (null everywhere) and logging once — a store built on an unverified mirror would produce false no-op resolves, which is worse than no store.
• subject_key identity is exact at any length: a key at or under max_subject_key_len is stored verbatim; a longer one (operator URLs are unbounded — safe_url.zig:12 imposes no input limit, and manager.zig:179's 255 cap bounds only a display copy) is replaced by "sha256:" ++ hex(SHA-256(key)) — 71 bytes, deterministic, collision-free in practice, so distinct long URLs never merge and the same URL always maps to the same episode. Never truncate and never reject a key. subject_label truncates to max_subject_label_len, detail to max_detail_len — those are display fields. subject_key never leaves the process; subject_label is the redacted display identity (safe_url.redactQuoted where the subject is a URL).
• prune: delete resolved rows older than resolved_retention_s, then oldest resolved rows beyond max_resolved_rows. Active rows are never pruned. Called at store init and once per retention pass (S2).

S1.4 Repository — src/storage/repositories/events_repo.zig (new)

Free functions on *db.Db, matching upstream_history_repo.zig conventions (file-scope SQL constants, by-value row structs with fixed buffers, prose test names, :memory: fixtures). Functions: insertActive, touchActive, resolveActive, resolveActiveByKey, selectActiveId, countActive, resolveExcept, insertResolved, loadActive, selectEvents(filter), selectOne(id), pruneResolved, activeCounts (the three beyond the original list serve the exact-overflow contract). Only the Store calls these in production. selectEvents filter: state (active/resolved/all), severity, component (matched on code prefix), since/until with the repo's [since, until) convention: an empty range (since >= until) returns nothing, and overlap is first_seen < until AND (resolved_at IS NULL OR resolved_at > since) — strict >, an episode resolved exactly at since does not overlap (matches queries_repo.zig:211). limit, before (keyset on id descending).

S1.5 API — src/web/handlers/diagnostics.zig (new)

GET /api/diagnostics — params state (active|resolved|all, default all), severity (warning|error), component, since, until, limit (1..1000, default 100), before (positive id). 400 with a naming message on any bad param (copy queries.zig conventions: parseFilter + message(err)). 503 when the store is absent. Response:

{ "events": [ { "id": 42, "code": "blocklist.refresh", "component": "blocklist",
    "subject": "StevenBlack", "severity": "warning",
    "first_seen": 1787118000, "last_seen": 1787118300, "occurrences": 3,
    "resolved_at": null, "detail": "download failed: ConnectionTimedOut" } ],
  "next_before": null,
  "active": { "warnings": 1, "errors": 0 } }

subject serializes subject_label. subject_key has no wire form — assert that in a test. Pagination contract identical to /api/queries (full page carries cursor, short page nulls it).

GET /api/diagnostics/{id} — the same event object, 404 after retention or for an unknown id.

Routes: two entries in routes.zig, .auth = .session, .policy = .read. Update the pinned route-count test (routes.zig:145, 56 → 58) and every routing invariant test that enumerates. OpenAPI: paths + DiagnosticsPage / DiagnosticEvent schemas, following the /api/queries exemplar; the openapi drift tests must stay green.

S1.6 Health — src/web/handlers/health.zig

Input gains diagnostics_present: bool = true (benign default, matching the all-defaulted convention), diagnostics_write_failed: bool = false, diagnostics_active_warnings: u32 = 0, diagnostics_active_errors: u32 = 0. Body gains "diagnostics": { "state": "recording"|"unavailable", "active_warnings": N, "active_errors": N } — unavailable when !present or write_failed. degraded adds diagnostics_write_failed or !diagnostics_present: in a serving process the store is absent only when Store.init failed, which is a real degradation, and the health endpoint never runs in subcommands. collect must assign diagnostics_present = state.events != null explicitly — the natural if (state.events) |store| shape would leave an absent store reported as recording under the benign default. This milestone does NOT yet remove history_flush_failing or restructure health to the full redesign shape — that lands with step 4 (Overview replacement); here health only gains the diagnostics block. Update the degraded-matrix test to cover both new inputs.

S1.7 Metrics — src/web/metrics.zig

Gauges nxdns_diagnostics_active_warnings, nxdns_diagnostics_active_errors; counter nxdns_diagnostics_write_failures_total (increment in the store on each failed write). Follow the upstream-history block pattern.

S1.8 WebState — src/web/server.zig

events: ?*events.Store = null.

S1.9 Acceptance criteria (S1)

• zig build test green; new store tests cover: episode open/dedupe/severity-raise, resolve-then-new-episode, one-shot insert, no-op resolve executes no SQL, write-failure latch set and cleared (log only on transition), label/detail truncation, over-length subject_key digested to the same identity twice, prune retention + cap, overflow slow paths for report (upsert, no unique-index collision) and resolve (count decrements), resolveExcept keeping mirror and count exact.
• events_repo tests cover every function against :memory: including overlap-window selection and keyset pagination.
• Handler tests: each param rejected with 400 + naming message, 503 without store, pagination cursor behavior, active counts, 404 on {id}.
• Route-pin, openapi-drift, health-matrix, schema-count tests all updated intentionally and green.
• Export/import round-trip leaves operational_events intact (test).

---

Session S2: emitters (after S1)

Thread the store as ?*events.Store using the established gate: ?*disk_monitor.Monitor idiom (app.zig:776): per-call parameter for run-loops, optional post-init field for Manager/Pool/CertStore. Every subsystem must build and test with null (no store). app.zig opens the dedicated connection right after migration (app.zig:335), unconditionally — not gated on cfg.web.enabled; diagnostics record whether or not the UI is on:

var events_db = try data.openConfigDb(io);
defer events_db.close();
const boot_now = std.Io.Clock.real.now(io).toSeconds();
var event_store_storage: ?events.Store = events.Store.init(io, &events_db, boot_now) catch |err| blk: {
    log.warn("diagnostics store unavailable: {s}", .{@errorName(err)});
    break :blk null;
};
const event_store: ?*events.Store = if (event_store_storage) |*s| s else null;

event_store is what gets threaded; a failed init leaves it null and health reports unavailable + degraded.

Emit sites, from the verified survey. Each row: failure emit, recovery resolve, subject_key / subject_label, severity.

Code	Failure (file:line today)	Recovery	subject_key → label	Severity
disk.space	disk_monitor.zig:137-147 transition to warn/critical	same publish, transition to ok	"data" singleton	warn→warning, critical→error
disk.probe	disk_monitor.zig:94-97 (change catch {} → catch |err|), :101-105, :109-113	next successful branch of the same probe	operation name (statvfs/data_dir/log_dir)	warning
blocklist.refresh	manager.zig:1102-1138 (fetch/compile/empty), :1520-1522 (load)	SourceStatus.succeed paths :840, :870, :1552	source URL → source name	warning
blocklist.snapshot	app.zig:572-577, manager.zig:1164-1167, :1184-1187	manager.zig:539-544 (post-swap)	singleton	error
blocklist.storage	manager.zig:1200-1203, :1336, :1451, :1458, :1470	the success branch of each matching operation, NOT pass-end — deleteQuietly absorbs failures at manager.zig:1466 and pruneOrphans still returns success at :1350, so a pass-end resolve would close the very event its own pass emitted	operation name	warning
certificate.reload	cert_store.zig:256-262 (capture the discarded err), :273-279	:271-273 success reload	endpoint kind — CertStore gains a kind: enum { doh, dot } field set at openCertStore (app.zig:611/:617)	warning — a stat failure can be a transient rename window and a failed reload keeps the loaded certificate serving (cert_store.zig:250); nothing is down
query_log.write	logger.zig:252-260 (init, from the failing BatchWriter.init and its catch), :400-404 (batch)	:405 successful batch. Init failure has no recovery (writer exits) — the event stays active, which is the truth	writer/batch	error
query_log.maintenance	retention.zig:113, :127, :133, :143, :151	matching success branch same pass	operation name	warning
query_log.recreated	querylog_schema.open result. Two changes: OpenResult gains the aside name (fixed [path_buf_len]u8 + len — today it exists only in a stack buffer inside open, querylog_schema.zig:137), and cli.openQuerylogDb (cli.zig:335) stops discarding recreated and returns it to app.zig, which calls reportResolved once the store exists. Never emitted for .missing — first creation has no aside file and is logged as "created" (querylog_schema.zig:131); a fresh install must not record a warning with an impossible aside path	one-shot	reason tag; detail carries the aside filename	warning
upstream_history.write	history.zig:262-266	:257-261	singleton	warning
upstream.exchange	pool.zig:332-346 recordFailure — emit OUTSIDE the pool mutex, same placement discipline as recordHistory	pool.zig:316-330 recordSuccess → resolve (no-op hot path)	upstream URL → safe_url.redactQuoted	warning
client_names.storage	client_names.zig:125-131, :141-147	clean-pass determination client_names.zig:161-164	read/write	warning
clients.storage	clients.zig:196-202, :218-224	success branches :195-197, :214-217	materialise/prune	warning
listener.start	app.zig:896/:900 (DoH), :915/:919 (DoT)	boot-finalized: after the bind phase, one resolveExcept(.listener_start, failed_keys) closes prior episodes for endpoints that started clean this boot — including endpoints now disabled	doh/dot	error
configuration.load	app.zig:686, :1039-1045, :1056-1061 and managed-file validate.Diagnostics warnings rendered at app.zig:204	boot-finalized: after all boot findings are emitted, one resolveExcept(.configuration_load, emitted_keys)	setting/upstream identity (redacted)	warning

Rules that bind every site:

• The existing log.warn/log.err lines STAY. Events are additive; journald keeps the raw stream.
• Failure text passed as detail is @errorName(err) plus the site's existing message fragment — no new prose invented, no allocation: format into a stack buffer of Store.max_detail_len.
• Lock discipline is collect-then-flush, not "emit after the mutex closes". Several manager sites cannot simply move: publishRefresh runs under writer_lock by contract (manager.zig:792) and pruneOrphans holds both refresh_lock and writer_lock through its filesystem work (manager.zig:1299). At such sites, record outcomes while the locks are held and flush to the store after the outer locked operation returns. The collection must not lose events: per-source outcomes ride the manager's existing per-source status allocations (one outcome per source, bounded by the source count), and same-operation failures within one pass aggregate into a single report per pass (the row's occurrences then counts failing passes, and detail carries the last error plus how many failures that pass held) — never an unbounded list, never a silent drop. Pool follows the existing recordHistory placement.
• Boot-finalized codes (listener.start, configuration.load) use resolveExcept exactly once each, after their boot phase completes; restart is the recovery, matching the design table.
• retention.zig runOnce gains events: ?*events.Store and calls store.prune once per pass.

S2 acceptance criteria

Emitters fall into three classes, each with its own test obligation (the author watches each fail first, repo ruling F-f):

• Episodic (disk, blocklist ×3, certificate, query_log.write batch, query_log.maintenance, upstream_history.write, upstream.exchange, client_names, clients): force the failure (existing seams: injected db errors, missing files, dead fetch server), assert the (code, subject_key, severity) row; then force recovery and assert resolution.
• Boot-finalized (listener.start, configuration.load): assert the emit on a failing boot, and assert resolveExcept closes a pre-seeded stale episode on a clean boot.
• One-shot / permanent (query_log.recreated; query_log.write init): assert the one-shot row inserts already resolved; assert the init-failure episode exists and that no recovery path claims it.
• pool exchange tests: success with no active episode performs no store SQL; failure→success round-trip produces exactly one resolved episode with correct occurrences.
• All subsystems still pass with events = null (the existing suites running unchanged).

---

Session S3: admin SPA (after S1, parallel with S2)

• lib/types.ts: DiagnosticEvent, DiagnosticsPage, plus the health body's new diagnostics block.
• lib/api.ts: getDiagnostics(filter), getDiagnostic(id).
• lib/queries.ts: diagnosticsQuery (infinite, keyset via next_before, keepPreviousData), diagnosticQuery(id); refetch interval matching healthQuery's cadence for the active view.
• routes.tsx: /diagnostics route (loader + lazy component) and /diagnostics/$id. Nav: add Diagnostics to NAV_ITEMS between Lookup and Settings (full nav restructure is later milestones); update AppShell.test.tsx.
• features/diagnostics/DiagnosticsPage.tsx: active episodes first (severity, title from an exhaustive code→copy map, subject, "active for …" age, occurrences), then resolved history with the standard filters (state, severity, component) reflected in the URL search params. Empty state is the healthy state: one line, "No active operational issues.", quiet styling — it must read as good news, not as a broken page (Fable's caution).
• features/diagnostics/DiagnosticDetailPage.tsx: the ordered detail from the design — state, first/last seen, occurrences, resolution, impact and remediation text from the same exhaustive code map, last error detail, and links (the config surfaces linked are today's routes; they get re-pointed when later milestones move them).
• The code→copy map lives in features/diagnostics/eventCopy.ts: an exhaustive Record<Code, {title, impact, remediation, link?}> over a string-literal union of the fifteen wire codes; a test iterates the union and asserts every member has copy.
• Tests: page renders fixtures (active + resolved + empty), filters drive the URL, detail renders every code's copy, pagination fetches more.

S3 acceptance criteria

• npm run typecheck, npx vitest run, npx prettier --check, lint, build, assert-bundled all green; byte budget respected.

---

Orchestrator integration (after S2+S3)

• Regenerate contractSamples.gen.ts (zig build test -Dintegration -Dcontract-samples-out=…) — requires the integration capture in web_integration_test.zig to seed at least one active and one resolved event.
• Update docs/reference/api.md and any drift-guarded reference pages; changelog entry under Unreleased.
• Live smoke on the real binary before anything ships: force a blocklist failure (dead URL), watch the episode appear, fix the URL, watch it resolve; screenshot the page per standing rule.

Module layout (new files)

File	Purpose
src/storage/events.zig	Code, wire mapping, Store
src/storage/repositories/events_repo.zig	SQL
src/web/handlers/diagnostics.zig	list + detail handlers
admin/src/features/diagnostics/…	page, detail, copy map, tests

File ownership

S1: config_schema.zig, events.zig, events_repo.zig, diagnostics.zig (handler), routes.zig, openapi.yaml, health.zig, metrics.zig, server.zig, migrations.zig (bridge + test), PLAN §11.2. S2: every emitter file + app.zig + retention.zig + cert_store.zig + cli.zig (openQuerylogDb) + querylog_schema.zig (OpenResult aside name) and its tests — S2 starts after S1 lands, so nothing is shared in parallel. S3: admin/ only. No parallel writers on any file (S2 and S3 run concurrently and are disjoint).

Anti-requirements

• No info severity, no acknowledgement/dismissal state, no manual-resolve endpoint, no raw-log endpoint, no generic remediation action schema.
• No api.storage code. No periodic probe for write_failed. No configurable retention.
• No changes to the existing dashboard, query log, live, or lookup pages.
• No new dependency, front or back.

Acceptance criteria (milestone complete)

• All session criteria; full zig build test + -Dintegration, admin suite, byte budgets.
• Live smoke: forced failure → visible episode → recovery → resolved row; /api/health shows the diagnostics block; /metrics shows the three series.
• Screenshots of the page (active, resolved, empty states) shown before push.

Recorded (as built)

Deviations from the sections above, found during build, review and live smoke. The code is the authority; this section says where it moved.

• src/storage/events_fixture.zig exists (test-only): a shared migrated-config.db + Store fixture the storage and filter tests use.
• The threading fields through app.zig/server.zig are named .diagnostics; WebState carries .events. The live smoke caught that .events was never assigned in app.zig while every suite stayed green — the integration tests build their own WebState. One line fixed it; the lesson is the standing "verify against the real network" rule.
• runScheduler's loop body is the pub function scheduledPass, so tests drive one pass deterministically. runScheduler flushes only in its catch branch (for the snapshot note its failure path adds).
• Occurrences count failing passes, not flushes: SourceStatus carries pass_outcome/pass_failures, set in fail()/succeed(), drained by flushSourceDiagnostics. Detail format: "{state}: {error} ({N} this pass)".
• Pass accounting lives in exactly one table copy at a time: mergeStatuses zeroes the pass fields on carried entries, installStatuses folds the live table's unflushed accounting in by id under the exclusive lock, and the flush drains by claiming one outcome-bearing entry per lock round (immune to concurrent table replacement).
• A pass flushes before it releases the lock that serializes it: the flush defer registers after the unlock defer in refreshSource, refreshAll, startupPass; standalone reload takes writer_lock itself and flushes inside it. This amends the collect-then-flush rule — the store runs on its own connection and mutex, so no shared resource is held across the store call.
• Deleted sources: flushSourceDiagnostics ends in resolveDeletedSources — resolveExcept(.blocklist_refresh, all current keys), gated on generation != 0 (an empty pre-reload table means "not read yet", not "all deleted") and skipped above 64 keyed entries (max_kept_keys; the episode then lingers until the count drops). Store.resolveExcept is no longer boot-finalized-only; the rule is the kept list must be the whole current subject set.
• events_repo.resolveExcept is mark-then-unmark (sentinel maxInt(i64), one transaction): the prior resolve-all-then-revive matched the revive on resolved_at = now_s and revived an episode resolved by the drain in the same second.

Addendum: manual purge of resolved events

Resolution stays automatic; the operator decides when resolved history disappears. Two endpoints, both mutations under the usual auth, both allowed in file mode (diagnostics are runtime state, not configuration):

• DELETE /api/diagnostics/{id} — purges one resolved event. 409 { "error": ... } when the event is active; 404 when no row has that id.
• DELETE /api/diagnostics — purges every resolved event, returns { "purged": N }.

Store: purge functions under the store mutex; they touch only rows with resolved_at NOT NULL, so the ActiveSet mirror and untracked count never change. UI: a purge action on each resolved row and on the resolved detail page, plus a "Purge all resolved" control on the list when at least one resolved event shows; active events show no purge affordance. OpenAPI, routes count, contract goldens, api.md updated.

Accepted limitations (reviewed with Codex, ruled in proportion to household scale; each is bounded and self-correcting, counts stay visible in the detail text):

• A standalone web reload interleaving with a refresh pass can merge two passes' accounting into one occurrence.
• pruneOrphans flushes its storage aggregates with the writer locks released; two concurrent prunes can merge into one report.
• The deletion sweep's kept-key snapshot can go stale against a concurrently added source: its fresh episode can be resolved once and reopens on the next failing pass with first_seen/occurrences reset.