nxdns/specs/milestone-27.md

# Milestone 27: diagnostics vertical slice

Step 1 of the UI redesign (`specs/ui-redesign.md`). A curated log of operational failure episodes: one `operational_events` table in `config.db`, a serialized event store, fifteen typed event codes emitted at existing failure sites, `GET /api/diagnostics`, and a Diagnostics page in the admin SPA. The rest of the UI stays intact; the full navigation restructure is later steps.

Design authority: `specs/ui-redesign.md` §Diagnostics, as amended by the Fable review rulings recorded in that file. No `api.storage` code. No periodic probe. No new index beyond the two specced.

## Sessions

S1 (store + API) → S2 (emitters) and S3 (SPA) in parallel → orchestrator integration (contract samples regen, final wiring check).

---

## Session S1: event store, schema, API, health, metrics

### S1.1 Schema — `src/storage/config_schema.zig`

Append to `ddl_v1` (pre-v0.1: edit the baseline, no migration step; update PLAN §11.2 to match):

```sql
CREATE TABLE operational_events (
    id INTEGER PRIMARY KEY,
    code TEXT NOT NULL,
    subject_key TEXT NOT NULL,
    subject_label TEXT NOT NULL,
    severity TEXT NOT NULL CHECK (severity IN ('warning', 'error')),
    first_seen INTEGER NOT NULL,
    last_seen INTEGER NOT NULL,
    occurrences INTEGER NOT NULL CHECK (occurrences > 0),
    resolved_at INTEGER,
    detail TEXT NOT NULL DEFAULT '',
    CHECK (resolved_at IS NULL OR resolved_at >= first_seen)
);
CREATE UNIQUE INDEX idx_operational_events_active
    ON operational_events(code, subject_key) WHERE resolved_at IS NULL;
CREATE INDEX idx_operational_events_last_seen
    ON operational_events(last_seen DESC);
```

`operational_events` is runtime state, NOT configuration. It stays out of `table_names` and `delete_order` (`config_schema.zig:112-129`). Tests that must be updated deliberately, not silently: `migrations.zig:365` asserts `count(tables) == delete_order.len + 1` — becomes `+ 2`, and its comment names `operational_events` as the second exclusion alongside `schema_version`. `src/config/reconcile.zig:268` (`comptime assert(delete_order.len == 10)`) is untouched — the list does not change. Add a test asserting export/import round-trips leave `operational_events` rows intact.

Editing `ddl_v1` changes nothing for existing installs pre-0.1 EXCEPT the silent-divergence hazard already on record: the Pi's `config.db` is stamped version 1 and will not get the new table. This milestone accepts that with a bridge placed at the END of `migrations.migrate`, after the version stamp: execute the three `operational_events` statements with `IF NOT EXISTS` added. It must NOT live in `cli.openConfigDb` — `openConfigDb` runs before migration everywhere (`app.zig:333`, `cli.zig:484`, `cli.zig:516`), and creating the table there would make a fresh database's non-`IF NOT EXISTS` `ddl_v1` `CREATE TABLE` fail. The bridge is explicit, commented as removable when the 0.1 adoption gate lands, and touches no other table. Test: a version-1 database created WITHOUT the table gains exactly it after `migrate`, and a fresh database migrates cleanly (no double-create).

### S1.2 Event codes — `src/storage/events.zig` (new)

```zig
pub const Code = enum {
    disk_space, disk_probe,
    blocklist_refresh, blocklist_snapshot, blocklist_storage,
    certificate_reload,
    query_log_write, query_log_maintenance, query_log_recreated,
    upstream_history_write, upstream_exchange,
    client_names_storage, clients_storage,
    listener_start, configuration_load,
};
```

Fifteen enum members — that count is the single cardinality every exhaustive test, route fixture and frontend copy union uses. The enum is the truth; the wire form is the dotted string (`disk_space` → `"disk.space"`, `blocklist_refresh` → `"blocklist.refresh"`, etc.) via `pub fn wire(code: Code) []const u8` — an exhaustive switch, tested against every member. `component(code)` returns the prefix before the dot, also exhaustive.

Severity is fixed per emit call, not per code (a disk transition to `warn` is warning, to `critical` is error).

### S1.3 Event store — `src/storage/events.zig`

```zig
pub const Store = struct {
    pub const max_detail_len = 512;
    pub const max_subject_key_len = 256;   // longer keys become sha256: digests — see the identity rule below
    pub const max_subject_label_len = 128;
    pub const resolved_retention_s: i64 = 90 * 86_400;
    pub const max_resolved_rows: i64 = 5_000;

    mutex: std.Io.Mutex,
    database: *db.Db,              // dedicated connection; ALL access goes through the Store's mutex
    write_failed: std.atomic.Value(bool),
    write_failures: std.atomic.Value(u64),   // feeds the metrics counter
    active: ActiveSet,             // in-memory mirror of active (code, subject_key) rows
    untracked_active_count: u32,   // active rows NOT in the mirror (overflow); loaded at init

    pub fn init(io, database, now_s) db.Error!Store   // loads ActiveSet + untracked count, prunes; failure = no store
    pub const max_kept_keys = 64;   // resolveExcept bound; over it the call is refused, counted and latched — truncating the kept list would close episodes that are still true
    pub fn report(self, io, now_s, code, subject_key, subject_label, severity, detail) void
    pub fn resolve(self, io, now_s, code, subject_key) void
    pub fn reportResolved(self, io, now_s, code, subject_key, subject_label, severity, detail) void
    pub fn resolveExcept(self, io, now_s, code, kept_keys: []const []const u8) void
    pub fn prune(self, io, now_s) void
    pub fn writeFailed(self) bool
    pub fn activeCounts(self, io) struct { warnings: u32, errors: u32 }
    pub fn selectEvents(self, io, arena, filter) db.Error!EventsPage
    pub fn selectOne(self, io, arena, id: i64) db.Error!?Event
};
```

Contract:

- **Time is a parameter, not a stored seam.** Every mutating method takes `now_s: i64` from the caller, matching how `history.zig:132` receives `wall_s` and `logger.zig:72` receives entry timestamps. Production callers compute it from `Clock.real`; tests pass literals. No function pointer, no clock inside the store.
- **`resolve` is hot-path safe.** It checks `active` under the mutex and, when the key is absent AND `untracked_active_count == 0`, returns without any SQLite statement. `pool.recordSuccess` calls it on every successful exchange; steady state must cost a mutex acquire and a lookup. The no-SQL guarantee is tested through a debug-only statement counter on the Store (incremented before every repo call) — NOT via `sqlite3_total_changes`, which a `SELECT` probe would not move.
- **Overflow is exact, not heuristic.** `ActiveSet` capacity is 256. The no-SQL fast path belongs to `resolve` ONLY: mirror miss + `untracked_active_count == 0` → return, no SQL (that is the steady-state success). `report` on a mirror miss always writes — when the count is zero it inserts directly (a new episode in normal state, no probe needed); when the count is nonzero it probes and upserts (touch the untracked active row if one exists — a second insert would collide with the partial unique index — else insert), incrementing `untracked_active_count` only when a new row was inserted AND the mirror is full. A successful slow-path `resolve` decrements the count. `init` loads the count as `active rows - mirrored rows`. `events_repo.resolveExcept` performs the bulk resolve and a count of TOTAL remaining active rows in ONE transaction, returning that total only after commit — the repo knows nothing of the mirror. The Store then removes resolved keys from its mirror and sets `untracked_active_count = total_active - active.len`; on any failure it changes neither.
- **Key canonicalization happens at every entry point.** `report`, `resolve`, `reportResolved` and each `kept_keys` element of `resolveExcept` all pass the caller's key through the same digest-if-over-length rule before any lookup or SQL. Tests cover a long-key report resolved with the same long key, and `resolveExcept` keeping a long kept key.
- **All reads go through the Store.** The handler calls `store.selectEvents` / `store.selectOne`, which lock the same mutex around `events_repo` — nothing touches `store.database` from outside. One connection, one owner.
- `report` upserts on the active row: present → `last_seen = now_s`, `occurrences += 1`, `detail` replaced, severity raised to the worse of the two, never lowered. Absent → insert new active row and add to `active`.
- `resolve` on an active row sets `resolved_at = now_s` and removes it from the mirror. A later failure inserts a NEW row (new episode) — the partial unique index enforces one active row per key.
- `reportResolved` inserts a row with `resolved_at = first_seen = last_seen = now_s`, `occurrences = 1`, and never touches `active` (one-shot events: `query_log_recreated`).
- `resolveExcept(code, kept_keys)`: resolves every active row of `code` whose `subject_key` is not in `kept_keys`, in one serialized operation. Exists for the boot-finalized codes (S2); nothing else may use it.
- Write failures: on `db.Error`, set `write_failed = true`, increment `write_failures`, drop the event. **Log only on the `false → true` transition** — a broken diagnostics database plus a busy pool must not produce warnings at query rate; the counter and the health surface carry the ongoing state. The next successful write clears the latch (and that recovery may log once).
- **No error propagates to a producer** — mutating methods return `void` by design; a diagnostics failure must never break the subsystem reporting it. `init` is the exception: it returns `db.Error`, and `app.zig` responds by running with no store (`null` everywhere) and logging once — a store built on an unverified mirror would produce false no-op resolves, which is worse than no store.
- `subject_key` identity is exact at any length: a key at or under `max_subject_key_len` is stored verbatim; a longer one (operator URLs are unbounded — `safe_url.zig:12` imposes no input limit, and `manager.zig:179`'s 255 cap bounds only a display copy) is replaced by `"sha256:" ++ hex(SHA-256(key))` — 71 bytes, deterministic, collision-free in practice, so distinct long URLs never merge and the same URL always maps to the same episode. Never truncate and never reject a key. `subject_label` truncates to `max_subject_label_len`, `detail` to `max_detail_len` — those are display fields. `subject_key` never leaves the process; `subject_label` is the redacted display identity (`safe_url.redactQuoted` where the subject is a URL).
- `prune`: delete resolved rows older than `resolved_retention_s`, then oldest resolved rows beyond `max_resolved_rows`. Active rows are never pruned. Called at store init and once per retention pass (S2).

### S1.4 Repository — `src/storage/repositories/events_repo.zig` (new)

Free functions on `*db.Db`, matching `upstream_history_repo.zig` conventions (file-scope SQL constants, by-value row structs with fixed buffers, prose test names, `:memory:` fixtures). Functions: `insertActive`, `touchActive`, `resolveActive`, `resolveActiveByKey`, `selectActiveId`, `countActive`, `resolveExcept`, `insertResolved`, `loadActive`, `selectEvents(filter)`, `selectOne(id)`, `pruneResolved`, `activeCounts` (the three beyond the original list serve the exact-overflow contract). Only the `Store` calls these in production. `selectEvents` filter: `state` (active/resolved/all), `severity`, `component` (matched on `code` prefix), `since`/`until` with the repo's `[since, until)` convention: an empty range (`since >= until`) returns nothing, and overlap is `first_seen < until AND (resolved_at IS NULL OR resolved_at > since)` — strict `>`, an episode resolved exactly at `since` does not overlap (matches `queries_repo.zig:211`). `limit`, `before` (keyset on id descending).

### S1.5 API — `src/web/handlers/diagnostics.zig` (new)

`GET /api/diagnostics` — params `state` (`active`|`resolved`|`all`, default `all`), `severity` (`warning`|`error`), `component`, `since`, `until`, `limit` (1..1000, default 100), `before` (positive id). 400 with a naming message on any bad param (copy `queries.zig` conventions: `parseFilter` + `message(err)`). 503 when the store is absent. Response:

```json
{ "events": [ { "id": 42, "code": "blocklist.refresh", "component": "blocklist",
    "subject": "StevenBlack", "severity": "warning",
    "first_seen": 1787118000, "last_seen": 1787118300, "occurrences": 3,
    "resolved_at": null, "detail": "download failed: ConnectionTimedOut" } ],
  "next_before": null,
  "active": { "warnings": 1, "errors": 0 } }
```

`subject` serializes `subject_label`. `subject_key` has no wire form — assert that in a test. Pagination contract identical to `/api/queries` (full page carries cursor, short page nulls it).

`GET /api/diagnostics/{id}` — the same event object, 404 after retention or for an unknown id.

Routes: two entries in `routes.zig`, `.auth = .session`, `.policy = .read`. Update the pinned route-count test (`routes.zig:145`, 56 → 58) and every routing invariant test that enumerates. OpenAPI: paths + `DiagnosticsPage` / `DiagnosticEvent` schemas, following the `/api/queries` exemplar; the openapi drift tests must stay green.

### S1.6 Health — `src/web/handlers/health.zig`

`Input` gains `diagnostics_present: bool = true` (benign default, matching the all-defaulted convention), `diagnostics_write_failed: bool = false`, `diagnostics_active_warnings: u32 = 0`, `diagnostics_active_errors: u32 = 0`. Body gains `"diagnostics": { "state": "recording"|"unavailable", "active_warnings": N, "active_errors": N }` — `unavailable` when `!present` or `write_failed`. `degraded` adds `diagnostics_write_failed or !diagnostics_present`: in a serving process the store is absent only when `Store.init` failed, which is a real degradation, and the health endpoint never runs in subcommands. `collect` must assign `diagnostics_present = state.events != null` explicitly — the natural `if (state.events) |store|` shape would leave an absent store reported as recording under the benign default. **This milestone does NOT yet remove `history_flush_failing` or restructure health to the full redesign shape** — that lands with step 4 (Overview replacement); here health only gains the diagnostics block. Update the degraded-matrix test to cover both new inputs.

### S1.7 Metrics — `src/web/metrics.zig`

Gauges `nxdns_diagnostics_active_warnings`, `nxdns_diagnostics_active_errors`; counter `nxdns_diagnostics_write_failures_total` (increment in the store on each failed write). Follow the upstream-history block pattern.

### S1.8 WebState — `src/web/server.zig`

`events: ?*events.Store = null`.

### S1.9 Acceptance criteria (S1)

- [ ] `zig build test` green; new store tests cover: episode open/dedupe/severity-raise, resolve-then-new-episode, one-shot insert, no-op resolve executes no SQL, write-failure latch set and cleared (log only on transition), label/detail truncation, over-length subject_key digested to the same identity twice, prune retention + cap, overflow slow paths for report (upsert, no unique-index collision) and resolve (count decrements), resolveExcept keeping mirror and count exact.
- [ ] `events_repo` tests cover every function against `:memory:` including overlap-window selection and keyset pagination.
- [ ] Handler tests: each param rejected with 400 + naming message, 503 without store, pagination cursor behavior, active counts, 404 on `{id}`.
- [ ] Route-pin, openapi-drift, health-matrix, schema-count tests all updated intentionally and green.
- [ ] Export/import round-trip leaves `operational_events` intact (test).

---

## Session S2: emitters (after S1)

Thread the store as `?*events.Store` using the established `gate: ?*disk_monitor.Monitor` idiom (`app.zig:776`): per-call parameter for run-loops, optional post-init field for `Manager`/`Pool`/`CertStore`. Every subsystem must build and test with `null` (no store). app.zig opens the dedicated connection right after migration (`app.zig:335`), unconditionally — not gated on `cfg.web.enabled`; diagnostics record whether or not the UI is on:

```zig
var events_db = try data.openConfigDb(io);
defer events_db.close();
const boot_now = std.Io.Clock.real.now(io).toSeconds();
var event_store_storage: ?events.Store = events.Store.init(io, &events_db, boot_now) catch |err| blk: {
    log.warn("diagnostics store unavailable: {s}", .{@errorName(err)});
    break :blk null;
};
const event_store: ?*events.Store = if (event_store_storage) |*s| s else null;
```

`event_store` is what gets threaded; a failed init leaves it null and health reports `unavailable` + degraded.

Emit sites, from the verified survey. Each row: failure emit, recovery resolve, subject_key / subject_label, severity.

| Code | Failure (file:line today) | Recovery | subject_key → label | Severity |
| --- | --- | --- | --- | --- |
| `disk.space` | `disk_monitor.zig:137-147` transition to warn/critical | same `publish`, transition to ok | `"data"` singleton | warn→warning, critical→error |
| `disk.probe` | `disk_monitor.zig:94-97` (change `catch {}` → `catch \|err\|`), `:101-105`, `:109-113` | next successful branch of the same probe | operation name (`statvfs`/`data_dir`/`log_dir`) | warning |
| `blocklist.refresh` | `manager.zig:1102-1138` (fetch/compile/empty), `:1520-1522` (load) | `SourceStatus.succeed` paths `:840`, `:870`, `:1552` | source URL → source name | warning |
| `blocklist.snapshot` | `app.zig:572-577`, `manager.zig:1164-1167`, `:1184-1187` | `manager.zig:539-544` (post-swap) | singleton | error |
| `blocklist.storage` | `manager.zig:1200-1203`, `:1336`, `:1451`, `:1458`, `:1470` | **the success branch of each matching operation, NOT pass-end** — `deleteQuietly` absorbs failures at `manager.zig:1466` and `pruneOrphans` still returns success at `:1350`, so a pass-end resolve would close the very event its own pass emitted | operation name | warning |
| `certificate.reload` | `cert_store.zig:256-262` (capture the discarded err), `:273-279` | `:271-273` success reload | endpoint kind — **`CertStore` gains a `kind: enum { doh, dot }` field set at `openCertStore` (`app.zig:611/:617`)** | warning — a stat failure can be a transient rename window and a failed reload keeps the loaded certificate serving (`cert_store.zig:250`); nothing is down |
| `query_log.write` | `logger.zig:252-260` (init, from the failing `BatchWriter.init` and its catch), `:400-404` (batch) | `:405` successful batch. Init failure has no recovery (writer exits) — the event stays active, which is the truth | `writer`/`batch` | error |
| `query_log.maintenance` | `retention.zig:113`, `:127`, `:133`, `:143`, `:151` | matching success branch same pass | operation name | warning |
| `query_log.recreated` | `querylog_schema.open` result. **Two changes: `OpenResult` gains the aside name (fixed `[path_buf_len]u8` + len — today it exists only in a stack buffer inside `open`, `querylog_schema.zig:137`), and `cli.openQuerylogDb` (`cli.zig:335`) stops discarding `recreated` and returns it to `app.zig`**, which calls `reportResolved` once the store exists. **Never emitted for `.missing`** — first creation has no aside file and is logged as "created" (`querylog_schema.zig:131`); a fresh install must not record a warning with an impossible aside path | one-shot | reason tag; detail carries the aside filename | warning |
| `upstream_history.write` | `history.zig:262-266` | `:257-261` | singleton | warning |
| `upstream.exchange` | `pool.zig:332-346` `recordFailure` — emit OUTSIDE the pool mutex, same placement discipline as `recordHistory` | `pool.zig:316-330` `recordSuccess` → `resolve` (no-op hot path) | upstream URL → `safe_url.redactQuoted` | warning |
| `client_names.storage` | `client_names.zig:125-131`, `:141-147` | clean-pass determination `client_names.zig:161-164` | `read`/`write` | warning |
| `clients.storage` | `clients.zig:196-202`, `:218-224` | success branches `:195-197`, `:214-217` | `materialise`/`prune` | warning |
| `listener.start` | `app.zig:896/:900` (DoH), `:915/:919` (DoT) | boot-finalized: after the bind phase, one `resolveExcept(.listener_start, failed_keys)` closes prior episodes for endpoints that started clean this boot — including endpoints now disabled | `doh`/`dot` | error |
| `configuration.load` | `app.zig:686`, `:1039-1045`, `:1056-1061` and managed-file `validate.Diagnostics` warnings rendered at `app.zig:204` | boot-finalized: after all boot findings are emitted, one `resolveExcept(.configuration_load, emitted_keys)` | setting/upstream identity (redacted) | warning |

Rules that bind every site:

- The existing `log.warn`/`log.err` lines STAY. Events are additive; journald keeps the raw stream.
- Failure text passed as `detail` is `@errorName(err)` plus the site's existing message fragment — no new prose invented, no allocation: format into a stack buffer of `Store.max_detail_len`.
- **Lock discipline is collect-then-flush, not "emit after the mutex closes".** Several manager sites cannot simply move: `publishRefresh` runs under `writer_lock` by contract (`manager.zig:792`) and `pruneOrphans` holds both `refresh_lock` and `writer_lock` through its filesystem work (`manager.zig:1299`). At such sites, record outcomes while the locks are held and flush to the store after the outer locked operation returns. The collection must not lose events: per-source outcomes ride the manager's existing per-source status allocations (one outcome per source, bounded by the source count), and same-operation failures within one pass aggregate into a single `report` per pass (the row's `occurrences` then counts failing passes, and `detail` carries the last error plus how many failures that pass held) — never an unbounded list, never a silent drop. Pool follows the existing `recordHistory` placement.
- Boot-finalized codes (`listener.start`, `configuration.load`) use `resolveExcept` exactly once each, after their boot phase completes; restart is the recovery, matching the design table.
- `retention.zig` `runOnce` gains `events: ?*events.Store` and calls `store.prune` once per pass.

### S2 acceptance criteria

Emitters fall into three classes, each with its own test obligation (the author watches each fail first, repo ruling F-f):

- **Episodic** (disk, blocklist ×3, certificate, query_log.write batch, query_log.maintenance, upstream_history.write, upstream.exchange, client_names, clients): force the failure (existing seams: injected db errors, missing files, dead fetch server), assert the `(code, subject_key, severity)` row; then force recovery and assert resolution.
- **Boot-finalized** (`listener.start`, `configuration.load`): assert the emit on a failing boot, and assert `resolveExcept` closes a pre-seeded stale episode on a clean boot.
- **One-shot / permanent** (`query_log.recreated`; `query_log.write` init): assert the one-shot row inserts already resolved; assert the init-failure episode exists and that no recovery path claims it.
- [ ] `pool` exchange tests: success with no active episode performs no store SQL; failure→success round-trip produces exactly one resolved episode with correct occurrences.
- [ ] All subsystems still pass with `events = null` (the existing suites running unchanged).

---

## Session S3: admin SPA (after S1, parallel with S2)

- `lib/types.ts`: `DiagnosticEvent`, `DiagnosticsPage`, plus the health body's new `diagnostics` block.
- `lib/api.ts`: `getDiagnostics(filter)`, `getDiagnostic(id)`.
- `lib/queries.ts`: `diagnosticsQuery` (infinite, keyset via `next_before`, `keepPreviousData`), `diagnosticQuery(id)`; refetch interval matching healthQuery's cadence for the active view.
- `routes.tsx`: `/diagnostics` route (loader + lazy component) and `/diagnostics/$id`. Nav: add `Diagnostics` to `NAV_ITEMS` between Lookup and Settings (full nav restructure is later milestones); update `AppShell.test.tsx`.
- `features/diagnostics/DiagnosticsPage.tsx`: active episodes first (severity, title from an exhaustive code→copy map, subject, "active for …" age, occurrences), then resolved history with the standard filters (state, severity, component) reflected in the URL search params. **Empty state is the healthy state**: one line, "No active operational issues.", quiet styling — it must read as good news, not as a broken page (Fable's caution).
- `features/diagnostics/DiagnosticDetailPage.tsx`: the ordered detail from the design — state, first/last seen, occurrences, resolution, impact and remediation text from the same exhaustive code map, last error detail, and links (the config surfaces linked are today's routes; they get re-pointed when later milestones move them).
- The code→copy map lives in `features/diagnostics/eventCopy.ts`: an exhaustive `Record<Code, {title, impact, remediation, link?}>` over a string-literal union of the fifteen wire codes; a test iterates the union and asserts every member has copy.
- Tests: page renders fixtures (active + resolved + empty), filters drive the URL, detail renders every code's copy, pagination fetches more.

### S3 acceptance criteria

- [ ] `npm run typecheck`, `npx vitest run`, `npx prettier --check`, lint, build, assert-bundled all green; byte budget respected.

---

## Orchestrator integration (after S2+S3)

- Regenerate `contractSamples.gen.ts` (`zig build test -Dintegration -Dcontract-samples-out=…`) — requires the integration capture in `web_integration_test.zig` to seed at least one active and one resolved event.
- Update `docs/reference/api.md` and any drift-guarded reference pages; changelog entry under Unreleased.
- Live smoke on the real binary before anything ships: force a blocklist failure (dead URL), watch the episode appear, fix the URL, watch it resolve; screenshot the page per standing rule.

## Module layout (new files)

| File | Purpose |
| --- | --- |
| `src/storage/events.zig` | `Code`, wire mapping, `Store` |
| `src/storage/repositories/events_repo.zig` | SQL |
| `src/web/handlers/diagnostics.zig` | list + detail handlers |
| `admin/src/features/diagnostics/…` | page, detail, copy map, tests |

## File ownership

S1: `config_schema.zig`, `events.zig`, `events_repo.zig`, `diagnostics.zig` (handler), `routes.zig`, `openapi.yaml`, `health.zig`, `metrics.zig`, `server.zig`, `migrations.zig` (bridge + test), PLAN §11.2. S2: every emitter file + `app.zig` + `retention.zig` + `cert_store.zig` + `cli.zig` (openQuerylogDb) + `querylog_schema.zig` (OpenResult aside name) and its tests — S2 starts after S1 lands, so nothing is shared in parallel. S3: `admin/` only. No parallel writers on any file (S2 and S3 run concurrently and are disjoint).

## Anti-requirements

- No `info` severity, no acknowledgement/dismissal state, no manual-resolve endpoint, no raw-log endpoint, no generic remediation action schema.
- No `api.storage` code. No periodic probe for `write_failed`. No configurable retention.
- No changes to the existing dashboard, query log, live, or lookup pages.
- No new dependency, front or back.

## Acceptance criteria (milestone complete)

- [ ] All session criteria; full `zig build test` + `-Dintegration`, admin suite, byte budgets.
- [ ] Live smoke: forced failure → visible episode → recovery → resolved row; `/api/health` shows the diagnostics block; `/metrics` shows the three series.
- [ ] Screenshots of the page (active, resolved, empty states) shown before push.

## Recorded (as built)

Deviations from the sections above, found during build, review and live smoke. The code is the authority; this section says where it moved.

- `src/storage/events_fixture.zig` exists (test-only): a shared migrated-config.db + Store fixture the storage and filter tests use.
- The threading fields through `app.zig`/`server.zig` are named `.diagnostics`; `WebState` carries `.events`. The live smoke caught that `.events` was never assigned in `app.zig` while every suite stayed green — the integration tests build their own `WebState`. One line fixed it; the lesson is the standing "verify against the real network" rule.
- `runScheduler`'s loop body is the pub function `scheduledPass`, so tests drive one pass deterministically. `runScheduler` flushes only in its catch branch (for the snapshot note its failure path adds).
- Occurrences count failing passes, not flushes: `SourceStatus` carries `pass_outcome`/`pass_failures`, set in `fail()`/`succeed()`, drained by `flushSourceDiagnostics`. Detail format: `"{state}: {error} ({N} this pass)"`.
- Pass accounting lives in exactly one table copy at a time: `mergeStatuses` zeroes the pass fields on carried entries, `installStatuses` folds the live table's unflushed accounting in by id under the exclusive lock, and the flush drains by claiming one outcome-bearing entry per lock round (immune to concurrent table replacement).
- A pass flushes before it releases the lock that serializes it: the flush defer registers after the unlock defer in `refreshSource`, `refreshAll`, `startupPass`; standalone `reload` takes `writer_lock` itself and flushes inside it. This amends the collect-then-flush rule — the store runs on its own connection and mutex, so no shared resource is held across the store call.
- Deleted sources: `flushSourceDiagnostics` ends in `resolveDeletedSources` — `resolveExcept(.blocklist_refresh, all current keys)`, gated on `generation != 0` (an empty pre-reload table means "not read yet", not "all deleted") and skipped above 64 keyed entries (`max_kept_keys`; the episode then lingers until the count drops). `Store.resolveExcept` is no longer boot-finalized-only; the rule is the kept list must be the whole current subject set.
- `events_repo.resolveExcept` is mark-then-unmark (sentinel `maxInt(i64)`, one transaction): the prior resolve-all-then-revive matched the revive on `resolved_at = now_s` and revived an episode resolved by the drain in the same second.

### Addendum: manual purge of resolved events

Resolution stays automatic; the operator decides when resolved history disappears. Two endpoints, both mutations under the usual auth, both allowed in file mode (diagnostics are runtime state, not configuration):

- `DELETE /api/diagnostics/{id}` — purges one resolved event. 409 `{ "error": ... }` when the event is active; 404 when no row has that id.
- `DELETE /api/diagnostics` — purges every resolved event, returns `{ "purged": N }`.

Store: purge functions under the store mutex; they touch only rows with `resolved_at NOT NULL`, so the ActiveSet mirror and untracked count never change. UI: a purge action on each resolved row and on the resolved detail page, plus a "Purge all resolved" control on the list when at least one resolved event shows; active events show no purge affordance. OpenAPI, routes count, contract goldens, api.md updated.

Accepted limitations (reviewed with Codex, ruled in proportion to household scale; each is bounded and self-correcting, counts stay visible in the detail text):

- A standalone web reload interleaving with a refresh pass can merge two passes' accounting into one occurrence.
- `pruneOrphans` flushes its storage aggregates with the writer locks released; two concurrent prunes can merge into one report.
- The deletion sweep's kept-key snapshot can go stale against a concurrently added source: its fresh episode can be resolved once and reopens on the next failing pass with `first_seen`/`occurrences` reset.