Files
nxdns/specs/ui-redesign.md
T
mokhtar 037f209179
Gates / frontend (push) Successful in 1m33s
Gates / test (push) Successful in 1m48s
Gates / test-aarch64 (push) Successful in 7m10s
Gates / package (push) Successful in 5m31s
Gates / container (push) Successful in 15s
CI / gates (push) Successful in 14m51s
milestone 27: diagnostics — operational failures land in one curated log, resolved history purgeable
2026-08-20 20:05:59 +02:00

27 KiB

UI redesign proposal

Author: Codex (gpt-5.6-sol, extra-high effort), 2026-08-19. Not accepted yet. Untracked on purpose until Mokhtar rules on the open questions at the end.

Answers that shaped it: the server and API may change; the surface-ownership split is right; file-mode configuration pages are read-only; diagnostics are curated structured events in the vein of Pi-hole's; time scoping is per workflow; a past query must be explainable exactly; Query Log and Live merge.

Navigation

Five primary items. Configuration expands to three task-shaped subpages and holds no landing route of its own.

Navigation Route Operator question Replaces
Overview /overview Is DNS healthy and protecting the household now, and what happened in this period? Dashboard
Activity /activity What requests are happening or happened, and why did nxdns handle them that way? Query Log, Live, Lookup
Clients /clients Who is this address, which policy applies, and what has it been querying? Clients
Diagnostics /diagnostics What is failing or has failed, what is affected, what should I do? new
Protection /configuration/protection What policy governs each group, and which rules and lists produce it? Groups, Blocklists, Rules
Resolution /configuration/resolution Where does nxdns answer or forward permitted names? Local DNS, Upstreams
System /configuration/system What service, storage, logging, TLS and web settings is this process running with? Settings

Secondary routes, reached from those surfaces rather than the nav: /activity/queries/:id, /activity/test, /clients/:id, /diagnostics/:id.

No current page survives unchanged. Login, logout and pause survive functionally, restyled into the new shell.

Activity

Two modes over the same columns and filters. History is persisted queries with keyset pagination and an absolute range. Live is follow-by-default with Freeze/Resume over the existing bounded 500-row buffer.

Columns: Time, Domain, Client, Type, Result, Route, Duration. Rule matches, source URLs and upstream errors live in the detail view, never on every row.

Domain testing stays as an Activity action labelled "Current policy simulation". It must never read as an explanation of a historical query.

Protection

Group-centred: group list, selected group detail, effective safe-search setting, assigned blocklist sources, rules scoped to the group, client count linking to matching clients. A Sources tab holds the shared blocklist catalogue and the "Update now" runtime action.

Resolution

Three tabs: upstream pool, local records, forward zones.

Clients

Keeps primary navigation because identifying and naming unknown devices is an operational job, not configuration. The learned marker appears only here, beside the name. Prefix assignments live here as "Network assignments".

Overview

Three sections, nothing else.

1. Current status. Five current facts, each conveyed by text and icon as well as colour. Healthy rows stay quiet; degraded rows link to the diagnostic or configuration surface that explains them.

Status Shows Why it belongs
Protection Active, paused until a timestamp, or unavailable, with Pause/Resume Confirms filtering is in force, and carries the valid runtime action
Upstreams available / configured, now Confirms DNS can leave the network
Query history Recording, losing rows, or writer failed Says whether Activity can be trusted
Diagnostics Recording or unavailable A failure reporter that cannot record failures must itself be visible
Storage ok / low / critical, free bytes Says whether writes are safe, and explains write gating

The shell carries a small global "Protection active/paused" indicator linking back to Overview. The controls themselves stay on Overview and beside blocked-query details.

2. Active issues. Severity, short title, affected object, how long it has been active, link to the detail. When none exist, one restrained line: "No active operational issues." Resolved failures never appear here, and healthy subsystems never get permanent green cards.

3. Activity over a period. The existing 1h / 24h / 7d / 30d control. Every value uses exactly the returned [since, until) window: queries, blocked count and rate, distinct clients, average response time, one query-volume timeline split blocked/cached/other, and "Open activity for this period" carrying the exact bounds. The timeline stays the existing lightweight SVG; no charting dependency.

If the selected period predates available data, the section says "Query history is available from …" rather than charting the missing span as zero.

Removed from Overview: the historical upstream table, the "last failure" text, per-upstream period rates, the database and log byte breakdown, the standalone cache card.

Diagnostics

Not a journald viewer, and it does not subscribe to std.log. Producers emit a finite set of typed events at the failure boundary.

Event model

One row is one failure episode.

Field Meaning
id Durable identifier
code Fixed machine-readable kind
subject_key Internal stable identity, may hold a full URL, never serialized
subject_label Bounded, redacted, operator-facing identity
severity warning or error
first_seen / last_seen Episode bounds
occurrences Deduplicated report count
resolved_at Null while active
detail Bounded last error or current condition

No info severity. Normal starts, refreshes and reloads do not become entries; a success resolves its prior failure.

Active dedup key is (code, subject_key). A repeat updates last_seen, occurrences, severity and detail. A success resolves the row. A later failure opens a new episode rather than reopening the old one. Severity records the worst state reached. There is no acknowledgement or manual dismissal: active means the component has not demonstrated recovery. One-shot material events, such as a query-log recreation, are inserted already resolved.

Storage

Events live in config.db as runtime state, excluded from export, import and file reconciliation. Active rows are never pruned. Resolved rows keep 90 days, with a hard cap of the newest 5,000. detail caps at 512 bytes. Pruning runs at startup and from existing maintenance; no new scheduler.

One table, one repository, one fixed event-code enum. A small serialized event store owns a dedicated config.db connection; background producers report synchronously through its mutex. This is the existing pattern, not a new one: app.zig:333 opens a connection for migration and reconciliation and app.zig:544 opens a separate web_config_db, and the background producers (fetcher, disk monitor, logger writer) have no other safe path into config.db.

If the store itself cannot write, an atomic event_store_failed state appears in /api/health and journald. It clears on the next successful write — no periodic probe. The store receives a write whenever anything fails or recovers, and a flag left set while nothing needs writing costs nothing.

Event sources

Source Event identity Recovery
Disk warn/critical transitions disk.space, singleton next ok sample
Failed statvfs or directory sizing disk.probe, keyed by operation/path next successful probe
Per-source download, HTTP, parse, compile or file-read failure blocklist.refresh, keyed by source URL that source refreshes
Initial snapshot or whole-pass failure blocklist.snapshot, singleton a snapshot publishes
Blocklist file cleanup failure blocklist.storage, keyed by operation that operation succeeds
Certificate stat or reload failure certificate.reload, keyed by doh/dot files readable and reload succeeds
Query writer init or batch failure query_log.write, keyed by writer/batch/queue writer starts, or a batch succeeds without drops
Query retention prune/checkpoint/vacuum failure query_log.maintenance, keyed by operation that operation succeeds
Upstream-history flush failure upstream_history.write, singleton next flush succeeds
Client-name selection or persistence failure client_names.storage, keyed by operation next pass succeeds
Client materialisation or pruning failure clients.storage, keyed by operation next pass succeeds
Upstream exchange failure upstream.exchange, keyed by upstream URL next successful exchange
Enabled DoH/DoT listener that cannot start listener.start, keyed by endpoint successful start after restart
Query-log recreation query_log.recreated, one-shot inserted resolved
Configuration warning leaving a capability skipped configuration.load, keyed by setting clean load after restart

Sixteen codes. Codex proposed a seventeenth, api.storage, for a storage failure that produced an HTTP 500 — cut on review, because it breaks this section's own exclusion rule: a 500 already answered its caller. Do not merge the remaining codes to shrink the count either. A merged code forces subject_key to carry what the code no longer says.

An upstream event describes a consecutive failure episode, not one row per retry. One timeout followed by success is one resolved episode.

Outside Diagnostics: invalid requests, conflicts, rate limits and failed logins already answered to their caller; expected reverse-DNS outcomes; individual TLS handshake failures already counted; development asset-server warnings; fatal startup failures that stop the UI existing. Disk-gated skips do not duplicate — the active disk event explains the cause and counters keep the totals.

From event to remediation

The detail shows current or resolved state; first seen, last seen, occurrences, resolution time; impact in operator language; the last bounded underlying error; the exact next action; how nxdns will verify recovery; links to the relevant configuration and activity window.

The frontend uses one exhaustive switch over the fixed codes for titles, impact, remediation and routes. No generic action schema, no plugin mechanism.

Query provenance

Stays in the expendable querylog.db. Does not go into the operational-events table.

block_reason is replaced by the fuller policy_reason. New columns:

Column Type Purpose
qclass INTEGER NOT NULL explains filtering bypass for non-IN questions
rcode INTEGER NOT NULL the client-visible result, including SERVFAIL
group_id INTEGER group at query time, not a foreign key
group_name TEXT historical label, survives a rename
policy_action TEXT NOT NULL not_evaluated, allow, block
policy_reason TEXT NOT NULL pipeline or matcher reason
matched TEXT exact rule pattern or list entry
source_id INTEGER blocklist source at query time
source_name TEXT historical source label
cname_target TEXT target that caused an uncloaked block
safe_search_target TEXT name used for the rewrite
route_kind TEXT NOT NULL blocked, local, forward_zone, upstream
forward_zone TEXT exact matched zone

upstream changes from the unhelpful "pool" marker to the actual configured upstream or forward resolver on the exchange that actually happened. On a cache hit route_kind is cache and upstream is NULL. Credentials are redacted at the serialization boundary.

Codex wanted the upstream recorded on cache hits too. Cut on review: it would widen every src/cache/dns_cache.zig entry to carry an upstream label, and it states a half-truth, because on a cache hit no upstream answered. The interface change it does need is real and worth doing — handler.zig:79-83 records ruling 20, which deliberately keeps the pool's answering endpoint out of transport.Client's reach. Exposing it touches src/upstream/, not the pure core. Do not fall back to "pool": the exact upstream on a SERVFAIL row is the single most useful correlation this redesign adds.

policy_reason is a closed enum: local_record, forward_zone, non_in_class, paused, snapshot_unavailable, no_match, plus the existing rule allow/block and blocklist exception/domain/wildcard reasons.

For a CNAME-uncloaked block, the policy fields describe the target's decision and cname_target preserves the target responsible.

Every syntactically parsed request that receives a response is logged, including synthesized SERVFAIL. Requests too malformed to identify a question stay counters, not fabricated rows.

No response payloads, answer RR sets, EDNS data or packet bytes are stored. The record explains nxdns's own decision, not the external resolver's answer.

Privacy transforms apply to every new domain-bearing field, not only domain: with hide_domains on, matched names, CNAME targets and safe-search targets hide consistently.

A one-row querylog_meta (created_at INTEGER NOT NULL) table lets the stats and query APIs return a conservative available_since, which distinguishes "zero queries" from "history does not exist".

Historical query detail

Ordered explanation: request (time, domain, client, type, class); group (historical id and name); policy (evaluated or not, allow/block, exact matched rule or list candidate, historical source, whether filtering was paused or unavailable); rewrites (safe-search target, CNAME target); route (local, forward zone, cache, or selected upstream); response (rcode, duration); related actions (test the domain against current policy, view the client, view activity for the same domain or client, view diagnostics in a five-minute window around the query).

Historical and current facts are visually separated. A rule, source or group that no longer exists stays visible as a historical value and is not linked to a different current object.

Live SSE events carry the same provenance shape without a persisted id. A frozen live row shows its in-memory detail; no correlation id is invented to link it to a row SQLite has not written.

Time scoping

One contract: unix seconds UTC, since inclusive, until exclusive, point data qualifies on since <= ts < until, diagnostic episodes qualify when their active interval overlaps the range, current state is labelled "Now" and no historical selector touches it.

URLs: /overview?period=24h with the server returning the exact aligned bounds; /activity?mode=history&since=…&until=… with domain, client, blocked and the other filters in the URL; /diagnostics?since=…&until=…&severity=…&component=…; /activity?mode=live with Follow/Freeze as ephemeral UI state. Investigation links always carry absolute bounds, so a viewed incident does not drift as time passes. Timestamps display in the browser's timezone; URLs and APIs stay timezone-independent.

File mode

One persistent authority line in the shell:

File-managed · /etc/nxdns/config.zon · loaded 19 Aug 2026, 08:42

It sits in the Configuration sub-navigation and appears elsewhere as a compact lock indicator. No full-width banner on every route. The wording is "running configuration loaded from", not "file contents" — the server cannot prove a since-edited file still matches the running process.

A file-managed page uses definition lists for scalars and tables or cards for collections, with human labels and the exact ZON key shown secondarily (logging.retention_days). No text inputs, no checkboxes, no Add/Edit/Delete, no disabled form shells, no simulated Save. A short page note says where edits happen and that a restart may be needed.

Runtime actions stay ordinary enabled buttons: pause/resume, update blocklists now, reload certificates, delete an observed undeclared client, login/logout.

Database mode uses the same information architecture with real edit actions, plus a server-owned restart_pending boolean. The current client-only restart banner and its local store are removed, so a browser refresh cannot erase the warning.

API changes

GET /api/config/status{authority, path, reconciled_at, restart_pending}. restart_pending is process state: database-mode mutations that need a restart set it, a successful restart clears it.

GET /api/health becomes explicit about every condition that contributes to degradation — protection, upstreams, query_history, upstream_history, diagnostics, disk, each an object with its own state. The current hidden history_flush_failing contribution is eliminated: nothing may degrade the rollup without appearing in the response.

GET /api/diagnostics?state=&severity=&component=&since=&until=&limit=&before= returns {events[], next_before, active:{warnings, errors}}. GET /api/diagnostics/{id} returns one event or 404 after retention. No acknowledgement, dismissal, generic-action or raw-log endpoints.

GET /api/queries keeps keyset pagination and its filters; rows gain rcode, route_kind, policy_action and the short policy reason the table needs, and the body gains coverage: {complete, available_since}. GET /api/queries/{id} returns nested request / policy / route / response provenance. GET /api/queries/live sends the same object without id.

GET /api/stats and /api/stats/timeseries add complete and available_since.

Existing mutation endpoints stay specific. Diagnostics introduces no generic "perform remediation" endpoint; it invokes the existing blocklist-refresh and certificate-reload operations.

Schema changes

config.db:

CREATE TABLE operational_events (
    id INTEGER PRIMARY KEY,
    code TEXT NOT NULL,
    subject_key TEXT NOT NULL,
    subject_label TEXT NOT NULL,
    severity TEXT NOT NULL CHECK (severity IN ('warning', 'error')),
    first_seen INTEGER NOT NULL,
    last_seen INTEGER NOT NULL,
    occurrences INTEGER NOT NULL CHECK (occurrences > 0),
    resolved_at INTEGER,
    detail TEXT NOT NULL DEFAULT '',
    CHECK (resolved_at IS NULL OR resolved_at >= first_seen)
);

CREATE UNIQUE INDEX idx_operational_events_active
    ON operational_events(code, subject_key)
    WHERE resolved_at IS NULL;

CREATE INDEX idx_operational_events_last_seen
    ON operational_events(last_seen DESC);

Absent from the configuration table lists and the reconciliation delete order.

querylog.db: querylog_meta and the provenance columns above. No new index — Codex proposed idx_query_log_rcode, cut on review, because every rcode question the UI asks is time-scoped and idx_query_log_ts already bounds the scan. 700k rows on a Pi 5 do not need a second index for a rare filter, and every index taxes the hot insert path in logger.zig.

No new provenance table, no key/value store — household retention makes nullable columns cheaper than a normalized graph of decision objects.

Entry buffer widths

logger.zig's Entry uses fixed buffers sized by max_reason_len = 32 and max_upstream_len = 64, and travels through the Io.Queue by value. The new fields roughly triple it: matched holds a full pattern, and cname_target, safe_search_target and forward_zone each hold up to 253 bytes. That is fine at flush_batch = 100, but the widths are part of this contract and must be set explicitly, not left to whatever the first implementation picks.

transformed() must hide matched, cname_target and safe_search_target under hide_domains, not only domain.

On recreation: query rows, provenance and upstream-minute history reset together as today; the old file stays aside; config.db diagnostics survive; a resolved query_log.recreated event records the reason, the aside path and the new coverage start; Overview and Activity report the incomplete range instead of charting zero.

Deletions and their cost

Deleted Lost
Separate Query Log and Live pages separate bookmarks; both modes remain in Activity
Standalone Lookup page a top-level bookmark; testing remains under Activity
Top-level Groups, Blocklists, Rules, Local DNS, Upstreams, Settings direct resource navigation; all capabilities remain under task-shaped configuration
Historical upstream table on Overview at-a-glance period rates; availability stays, failures move to Diagnostics
"last failure · 9h ago" text nothing actionable; the episode becomes a diagnostic
Detailed DB/log byte gauges exact component sizes stay in Prometheus; free space stays on Overview
Standalone cache card one prominent number; cache stays in the timeline and metrics
Ephemeral blocklist SourceStatusSection transient success detail after navigation; durable counters and diagnostics remain
Disabled configuration forms in file mode the illusion that fields can be edited
Global read-only banner repeated warning text; authority stays visible once in the shell
Client-only restart banner state nothing reliable; server-owned restart_pending replaces it
Old route aliases and redirects existing bookmarks break; no permanent duplicate routing layer

Build sequence

Each step leaves the app working and shippable, and updates its OpenAPI contract, generated sample, TypeScript types and deterministic tests before landing.

  1. Diagnostics vertical slice. operational_events schema, repository, serialized store, retention, health state, every typed emitter, API, UI, navigation. Recovery paths and deterministic failure-injection tests per event code. The rest of the UI stays intact.

  2. Query provenance vertical slice. Query-log fingerprint and schema, metadata table, provenance capture in the handler and logger, parsed-SERVFAIL logging, detail and coverage APIs, the historical detail route, and the query_log.recreated emission. Existing list summary fields stay so the current pages keep working.

    This step destroys the existing query history. The provenance DDL edit changes the fingerprint, so querylog_schema.open recreates the file and keeps the old one aside as querylog.db.schema-changed-<unix seconds>. Acceptable pre-v0.1, and available_since carries the story in the UI, but it is a consequence of this step and must be stated in its spec and its changelog entry. Its acceptance tests cover the recreate, the aside name and the coverage sequence — which is also the natural first query_log.recreated emission.

    Parsed-SERVFAIL logging reverses ruling 20: handler.zig:507 counts today rather than logging. The Context exists at every servFail site that follows question parsing. Pre-parse failures correctly stay counters.

  3. Activity consolidation. The unified History/Live surface, URL filters, freeze/follow, live detail, current-policy test, historical detail links. Query Log, Live and Lookup routes and code are removed in the same change. Route, SSE, accessibility, reconnect and bounded-buffer tests.

  4. Overview replacement. Current status, active diagnostics, one coherent activity section. New health contract and completeness states. The upstream-history table, stale-failure text, detailed DiskCard and cache card go.

  5. Task-shaped configuration and file mode. /api/config/status and server-owned restart_pending. Protection, Resolution and System in both read-only and editable forms. Clients and its detail route redesigned. Old configuration routes replaced atomically; global banner and disabled forms removed.

  6. Contract closure. Remove obsolete queries, types, stores, CSS, tests and route fixtures. Regenerate contract samples, update OpenAPI and reference docs, add cross-surface acceptance tests for investigation links, file authority, query-log recreation, active-event recovery and time bounds. Zig, frontend, integration, accessibility and byte-budget checks; no new dependency.

Codex's least-certain calls

  • Clients in primary navigation. It earns the slot if identifying unknown devices and checking their group is routine. If Mokhtar almost always reaches a client from a query, Clients moves under Protection and leaves the nav.
  • Recording the exact selected upstream per query. It materially improves correlating SERVFAIL queries with upstream events, but needs the pool exchange result to expose the selected target. Drop back to "pool" only if that interface change proves invasive and exact resolver identity never changes an action.
  • 90-day / 5,000-event retention. Conservative fixed bounds, not settings. Change only after measuring real row size and event rate on the Pi; do not add configurable retention pre-emptively.

Rulings (Fable review, 2026-08-20)

Verdict: build it, with the four cuts folded in above and the rulings below.

Diagnostics live in config.db. Accepted. The querylog.db alternative is self-refuting: that file is recreated on any schema edit or corruption, so the recreation event dies with the thing it describes. A third database needs either its own migration discipline or a recreate policy that loses the events — the same problem with more files. The write-traffic objection is overstated twice: event volume is failure-rate volume, deduplicated, with no info severity, so it is near zero in steady state; and config.db already takes operational writes, because client rows are materialized from traffic (PLAN §3.5, clients.last_seen, learned_name). Pre-v0.1 the table is one edit to ddl_v1 with no migration step. Keeping it out of delete_order and table_names is enforceable — config_schema.zig has tests pinning those lists.

group_name, source_name and matched stay TEXT. Accepted, and my inconsistency objection was weaker than I put it. The domains table exists because a domain appears on every row, runs to 253 bytes, and feeds GROUP BY stats. None of that holds here. The closer precedent is client_ip, which is TEXT with the comment "not a FK: log rows are immutable facts" (querylog_schema.zig:37). group_name is short and mostly default; source_name and matched are non-NULL only on blocked rows; matched has cardinality high enough that normalizing buys nothing. Keep both halves of each id/name pair: the id links to the same object across a rename, the name survives a delete, and the detail view needs both.

Provenance capture does not violate the pure core. Verified against the code. The policy decision is made in src/server/handler.zig, which is already impure and already holds Io, the clock and the log call. matcher.Decision (matcher.zig:39-51) already returns reason, matched and source; the handler throws matched and source away at handler.zig:501. Capture is mostly widening logger.zig's Entry and LogFields, not threading state through dns/, filter/, local/ or cache/.

Six milestones under the harness, one per step. Do not fold step 6 into step 5: the closure sweep regenerates the contract samples and adds the cross-surface acceptance tests, and it deserves its own verify gate. If anything needs splitting it is step 1, which touches about ten subsystems — store, API and UI first, then the emitters. An event store with three emitters is already shippable and honest.

One caution, not a blocker. Diagnostics will be an empty page most of the year. The "No active operational issues" line has to make empty read as healthy, not broken.