16 Commits
Author SHA1 Message Date
mokhtar fa323c7ed4 milestone 29: activity — history, live and policy simulation on one surface
Gates / test (push) Successful in 1m40s
Gates / package (push) Successful in 3m58s
Gates / container (push) Successful in 14s
CI / gates (push) Successful in 12m51s
Gates / frontend (push) Successful in 1m18s
Gates / test-aarch64 (push) Successful in 6m57s
query log, live and lookup merge into /activity. history filters live
in the url, so a pasted link or back/forward reproduces the exact
view; the result column separates servfail and nxdomain from success
in the list. live is follow-by-default with freeze, and a streamed
row opens its in-memory provenance detail — no correlation invented
for rows sqlite has not written. lookup survives as the current
policy simulation under /activity/test. investigation links carry
absolute bounds, and the diagnostics page now honors since/until
instead of ignoring them. the old routes are gone without aliases.
2026-08-22 10:52:56 +02:00
mokhtar 0fd6bbd312 milestone 28: query provenance — every logged query is exactly explainable
Gates / frontend (push) Successful in 1m36s
Gates / test (push) Successful in 1m56s
Gates / test-aarch64 (push) Successful in 7m37s
Gates / package (push) Successful in 9m12s
Gates / container (push) Successful in 13s
CI / gates (push) Successful in 19m4s
query rows gain qclass, rcode, group, policy action and reason, the
matched rule or list entry with its source, cname and safe-search
targets, route kind, forward zone, and the resolver that actually
answered — the pool and local markers die. servfails are logged and
name the resolver that lost; post-parse protocol refusals become rows.
a detail page at /queries/:id renders the ordered explanation, and
coverage watermarks distinguish an empty history from a missing one.

the schema fingerprint changes: existing query history is recreated
with the old file kept aside and the reset filed as a resolved
diagnostic. fixes an oversized udp reply being rebuilt as noerror,
which handed clients a truncated nxdomain as success.
2026-08-22 09:16:40 +02:00
mokhtar 7e6cb507d2 release cut: bump-kind justfile recipe and a compiled, tested cut tool
Gates / frontend (push) Successful in 1m11s
Gates / test (push) Successful in 1m38s
Gates / test-aarch64 (push) Successful in 6m31s
Gates / container (push) Successful in 9s
CI / gates (push) Successful in 26m52s
Gates / package (push) Successful in 5m27s
2026-08-21 23:34:13 +02:00
mokhtar 8a17e9ed21 admin: fix dashboard phantom scroll, drop last-failure column from upstream table
Gates / frontend (push) Successful in 1m11s
Gates / test (push) Successful in 1m40s
Gates / test-aarch64 (push) Successful in 6m38s
Gates / package (push) Successful in 5m42s
Gates / container (push) Successful in 17s
CI / gates (push) Successful in 27m54s
the chart's screen-reader table wore srOnly directly; overflow and
height do not apply to a table box, so it laid out 1200px tall below
the page while clip-path hid the paint. wrap it in a hidden div, which
clips properly and keeps the table role. failure detail is the
diagnostics page's job since milestone 27; the column and the now
dead formatAge go.
2026-08-21 23:33:32 +02:00
mokhtar 0107df5f99 spec: measurement window is the deployment side's call
Gates / frontend (push) Successful in 1m13s
Gates / test (push) Successful in 1m37s
Gates / test-aarch64 (push) Successful in 6m35s
Gates / package (push) Successful in 5m40s
Gates / container (push) Successful in 9s
CI / gates (push) Successful in 22m27s
2026-08-21 18:49:05 +02:00
mokhtar b6cea3f539 build: bump version to 0.0.8
Gates / frontend (push) Successful in 1m34s
Gates / test (push) Successful in 1m53s
Gates / test-aarch64 (push) Successful in 6m34s
Gates / package (push) Successful in 43s
Gates / test-aarch64 (push) Successful in 7m23s
Gates / package (push) Successful in 4m11s
Gates / container (push) Successful in 18s
CI / gates (push) Successful in 27m37s
Release / guard (push) Successful in 18s
Gates / frontend (push) Successful in 1m9s
Gates / test (push) Successful in 1m36s
Gates / container (push) Successful in 10s
Release / gates (push) Successful in 9m6s
Release / publish (push) Successful in 4m33s
2026-08-21 18:43:49 +02:00
mokhtar 9b0b7c19f4 querylog: checkpoint every 8192 wal pages instead of 1000 2026-08-21 18:43:45 +02:00
mokhtar 324704b53f build: bump version to 0.0.7
Gates / package (push) Successful in 41s
Gates / test (push) Successful in 1m35s
Gates / test-aarch64 (push) Successful in 6m48s
Gates / frontend (push) Successful in 1m22s
Gates / container (push) Successful in 9s
CI / gates (push) Successful in 23m18s
Gates / frontend (push) Successful in 1m8s
Gates / package (push) Successful in 40s
Release / guard (push) Successful in 17s
Gates / test (push) Successful in 1m35s
Gates / test-aarch64 (push) Successful in 6m55s
Gates / container (push) Successful in 9s
Release / gates (push) Successful in 9m23s
Release / publish (push) Successful in 4m26s
2026-08-20 22:34:13 +02:00
mokhtar addf24f92c query log batching: one transaction per flush interval, not per query
Gates / frontend (push) Successful in 1m18s
Gates / test (push) Successful in 2m46s
Gates / test-aarch64 (push) Successful in 7m33s
Gates / package (push) Successful in 5m34s
Gates / container (push) Successful in 17s
CI / gates (push) Successful in 16m16s
Gates / frontend (push) Successful in 1m8s
Gates / container (push) Successful in 9s
Release / gates (push) Successful in 9m15s
Release / guard (push) Successful in 19s
Gates / test (push) Successful in 1m34s
Gates / test-aarch64 (push) Successful in 6m46s
Gates / package (push) Successful in 39s
Release / publish (push) Failing after 4m7s
2026-08-20 20:57:11 +02:00
mokhtar 037f209179 milestone 27: diagnostics — operational failures land in one curated log, resolved history purgeable
Gates / frontend (push) Successful in 1m33s
Gates / test (push) Successful in 1m48s
Gates / test-aarch64 (push) Successful in 7m10s
Gates / package (push) Successful in 5m31s
Gates / container (push) Successful in 15s
CI / gates (push) Successful in 14m51s
2026-08-20 20:05:59 +02:00
mokhtar 3dd8214ef2 dashboard: a success rate never rounds up to 100.0% while failures stand
Gates / frontend (push) Successful in 1m27s
Gates / test (push) Successful in 1m53s
Gates / test-aarch64 (push) Successful in 7m7s
Gates / package (push) Successful in 3m57s
Gates / container (push) Successful in 15s
CI / gates (push) Successful in 13m15s
2026-08-19 17:54:02 +02:00
mokhtar 64c0d723a6 querylog: the aside file's name says why, so a schema change is not called corrupt 2026-08-17 20:24:21 +02:00
mokhtar 377f00a35f changelog: 0.0.6 releases today
Release / guard (push) Successful in 22s
Gates / frontend (push) Successful in 1m30s
Gates / test (push) Successful in 1m50s
Gates / frontend (push) Successful in 1m9s
Gates / test-aarch64 (push) Successful in 6m55s
Gates / test (push) Successful in 1m34s
Gates / test-aarch64 (push) Successful in 6m9s
Gates / package (push) Successful in 3m49s
Gates / package (push) Successful in 32s
Gates / container (push) Successful in 15s
CI / gates (push) Successful in 21m28s
Gates / container (push) Successful in 9s
Release / gates (push) Successful in 20m7s
Release / publish (push) Successful in 4m26s
2026-08-17 18:22:54 +02:00
mokhtar e0a7cd8a6b milestone 26: upstream health answers for the selected period 2026-08-17 18:21:56 +02:00
mokhtar 3ed9a57822 changelog: 0.0.5 releases today
Gates / test (push) Successful in 1m44s
Gates / frontend (push) Successful in 1m2s
Gates / test-aarch64 (push) Successful in 5m55s
Gates / package (push) Successful in 28s
Gates / container (push) Successful in 16s
CI / gates (push) Successful in 20m25s
Release / guard (push) Successful in 18s
Gates / frontend (push) Successful in 1m5s
Gates / test-aarch64 (push) Successful in 6m33s
Gates / test (push) Successful in 1m31s
Gates / package (push) Successful in 3m41s
Gates / container (push) Successful in 9s
Release / gates (push) Successful in 19m27s
Release / publish (push) Successful in 5m43s
2026-08-16 23:55:19 +02:00
mokhtar 0601098ab0 admin: drop the learned tag from the query tables 2026-08-16 23:55:10 +02:00
144 changed files with 24503 additions and 2127 deletions
+73
View File
@@ -6,6 +6,79 @@ Sections are written by hand. Nothing here is generated from commit messages: th
## [Unreleased]
Query provenance: every logged query becomes exactly explainable — what the policy decided, what matched, where the answer came from and what the client saw. The handler records all of it as the reply goes out, `query_log` stores it, and a detail page reads one query back in the order the pipeline decided it. Read the upgrade note below first: it resets your query history.
### Added
- **Every logged query has a detail page.** A row in Activity now links to `/activity/queries/{id}`, which explains that one query in the order it was decided: the request, the group it was matched under, the policy verdict with the rule that produced it and the blocklist source that rule came from, any CNAME uncloaking or safe-search rewrite, the route the answer took — blocked, local, forward zone, upstream or cache — and what the client got back, RCODE and duration included. `GET /api/queries/{id}` serves the same object; an id that retention has already deleted is a 404. The live view carries the same provenance for the queries it streams, so a query is explainable as it happens as well as afterwards.
- **Query Log, Live and Lookup are one Activity page.** `/activity` is the single surface for what nxdns answered: History reads the stored log, Live reads the stream, and both show the same seven columns — Time, Domain, Client, Type, Result, Route, Duration. The mode and every filter live in the URL, so an investigation is one link that shows the recipient exactly what you were looking at, and an absolute time range stays that range instead of drifting as the day goes on. A new **Result** column says what the client actually got — `Blocked`, `NOERROR`, `SERVFAIL` and the rest — with the **Route** column beside it saying how the answer was produced, which is the pair the old Status column could not show: a blocked name is answered with NOERROR, and reading only the code made a block look like a success. Both unhappy cases are marked by weight and shape as well as colour. Switching between History and Live keeps your filters, and leaving Live closes the stream instead of holding a viewer slot open. A live row that the log has not written yet opens its own provenance in place — no invented row id — and the open detail stays put while the 500-row buffer scrolls past underneath it. Domain testing moves to `/activity/test` as **Current policy simulation**, worded so it can never be misread as an account of a query that already happened.
- **Diagnostics can be scoped to an absolute window.** `/diagnostics?since=…&until=…` now validates and applies both bounds to the active and resolved lists, and the page states the window it is showing with a way to clear it. A query's detail page links here with the five minutes either side of that query, which is where the underlying failure text for a SERVFAIL lives.
- **The query log and the dashboard say how far back the history goes.** `GET /api/queries`, `/api/stats` and `/api/stats/timeseries` each carry a `coverage` object: `available_since`, the first second the file can answer for, and `complete`, whether the window you asked for begins inside it. A period that starts before the query log does now says so instead of charting the missing part as zero — which is what a recreate, a retention pass or a fresh install would otherwise look like.
### Removed
- **`/queries`, `/queries/{id}`, `/live` and `/lookup` are gone, and bookmarks to them break.** There is no redirect and no alias: the paths simply stop resolving, and the app shows its not-found page. Everything those pages did is on `/activity`, `/activity/queries/{id}` and `/activity/test`. Three navigation entries collapse into one, "Activity". The API is untouched — `/api/queries`, `/api/queries/{id}`, `/api/queries/live` and `/api/lookup` all answer exactly as before.
- **The Status column, and the block reason on every row.** The reason a query was blocked was repeated on each of a hundred rows and pushed the answer the client saw off the table. Result and Route replace it; the exact rule, the blocklist source and the historical group stay one click away on the query's detail page, which is the only place they were ever readable.
### Changed
- **`GET /api/queries` rows changed shape.** Each row gains `qclass`, `rcode`, `policy_action`, `policy_reason` and `route_kind`, and `block_reason` is gone: the reason a query was blocked is now one of a closed set of values rather than a formatted string. No table column shows it — the reason is read on the query's detail page, and by an API client from `policy_reason` on the row. `blocked`, `cache_hit`, `upstream` and every other existing field are unchanged.
- **Upgrading resets your query history.** The `query_log` table gains the provenance columns below, and `querylog.db` is never migrated (it holds expendable log rows, so a schema change replaces the file instead of upgrading it). On the first start after the upgrade the old file is set aside as `querylog.db.schema-changed-<unix seconds>` and a fresh one is created. Nothing else is touched: `config.db` keeps your configuration and your diagnostics history. The recreate files a resolved `query_log.recreated` diagnostics entry naming the file that was kept and the timestamp the new history begins at, and a new `querylog_meta` table records that coverage start, so the dashboard can say "history is available from ..." instead of charting an empty range as zero. The set-aside file is a working SQLite database and can be deleted once you have decided you do not want it.
- **`logging.query_log_buffer_max` now accepts 1 to 37449, down from 1 to 1000000.** The queued entry carries every new provenance field by value and is about four times as wide as before — 1792 bytes against 432 — so the meaningful bound is bytes rather than entries. The ceiling is computed at compile time from the width of the entry so that the queue's worst case stays within 64 MiB, and it moves whenever that width does. The default of 10000 is unchanged and costs about 17 MiB. A configuration above the new ceiling is rejected at startup with the ceiling in the message.
- **Group and blocklist source names are now capped at 64 bytes.** Both are copied into every query-log row that mentions them, so an unbounded name was an unbounded cost per row. A longer name is rejected as `GroupNameTooLong` or `SourceNameTooLong`.
### Fixed
- **A UDP reply that has to be truncated keeps the answer's RCODE.** When an answer does not fit the client's UDP buffer, nxdns replaces it with an empty reply carrying the TC bit, which tells the client to retry over TCP. That replacement was always built as NOERROR, whatever the answer said — so an oversized NXDOMAIN reached the client as a success, and an EDNS extended RCODE above 15 lost the eight upper bits it needs an OPT record to carry. The truncated reply now carries the full twelve-bit code the answer had, split across the header and the reply's OPT record where the code needs it, and the query-log row records the code the client actually saw. The retry over TCP always returned the right RCODE; this was the UDP answer that preceded it.
## [0.0.8] - 2026-08-21
One constant, chosen from the 0.0.7 field numbers: the checkpoint cadence was the last first-order write cost on the Pi's SD card.
### Changed
- **The query log checkpoints its write-ahead log every 32 MiB instead of every 4 MiB.** Batching the writer in 0.0.7 took the deployed Pi from about 0.5 to 0.281 GiB of writes a day, and about 130 MiB of what is left is checkpoint writeback: SQLite's 1000-page default trips roughly every 40 minutes and rewrites the same hot index and interior pages into `querylog.db` each time. Every read-write connection to `querylog.db` now sets `wal_autocheckpoint` to 8192 pages, which stretches that to roughly five hours and cuts those in-place rewrites about eightfold, for an expected total near 190 MiB a day. The price is durability under power loss or a kernel panic. At `synchronous = NORMAL` a commit does not fsync, so the checkpoint is the only guaranteed durability boundary, and it now sits about five hours of query rows and upstream-history minutes back rather than 40 minutes. Kernel writeback normally makes the real loss far smaller than that, but nothing guarantees it. A process crash or a clean stop still loses nothing that was committed, and the database is never left inconsistent: recovery replays the longest valid prefix of the log. The `querylog.db-wal` file is expected to sit near 32 MiB rather than capped there, since a long-running reader can hold a checkpoint off and let it overshoot, and the daily retention pass still truncates it. `config.db` is unchanged.
## [0.0.7] - 2026-08-20
Operational failures get a page of their own, and the query log stops wearing out the disk it lives on: the deployed Pi was writing half a gigabyte a day to store two megabytes of query rows, one transaction per query. Both came out of running 0.0.6 on real hardware.
### Added
- **A diagnostics page.** Operational failures now land in one curated log instead of only journald: blocklist download failures, certificate reload failures, disk pressure, query-log writer and maintenance failures, upstream exchange and history failures, client tracking failures, listener and configuration problems at boot, and the query-log recreation an upgrade causes. One entry per failing subject — an entry opens on the first failure, counts repeats, and closes itself when the subject recovers; nothing needs dismissing. Each entry says what it means for the service and what to do about it. `GET /api/diagnostics` serves the log, `GET /api/health` reports the active counts and degrades while the diagnostics store itself cannot write, and `/metrics` gains `nxdns_diagnostics_active_warnings`, `nxdns_diagnostics_active_errors` and `nxdns_diagnostics_write_failures_total`. Resolved entries can be purged when you decide the history has served its purpose — one entry from its row or its detail page, or the whole resolved history at once with "Purge all resolved" (`DELETE /api/diagnostics/{id}` and `DELETE /api/diagnostics`). An entry that is still failing is the current state of the box, not history, so it has no purge action and the API answers 409.
### Changed
- **The query log commits once a minute instead of once a query.** The writer batched for 100 milliseconds, which at a household's query rate means almost every query got a transaction of its own — and a transaction costs the disk far more than the row it carries. On the deployed Pi that came to roughly 0.5 GiB of writes a day to store 2.3 MB of query rows, the kind of write volume that kills an SD card. The batch window is now `logging.query_log_flush_interval_s`: 60 seconds by default (the same minute Pi-hole's `DBinterval` defaults to, for the same reason), anything from 0 to 3600, editable on the settings page. Batches are still capped at 100 rows, so a burst is committed as soon as it fills one rather than waiting out the window, and the in-memory queue, its drop-oldest backpressure and retention are untouched. The price is two kinds of lag: a crash costs about one interval of query history — more if the writer was held back by a full disk or a slow write — and every query-log-backed view — the query-log page, the dashboard totals, the timeseries — is about one interval behind. The live page is not affected; it is fed before the queue. Set the key to `0` for the old write-immediately behavior.
### Fixed
- **Shutdown no longer races the last query rows to the disk.** The query-log writer was stopped by the same cancellation that stopped the DNS listeners, so whether the batch it was holding reached the database depended on which happened to land first, the cancellation or the queue closing. Shutdown now stops and joins the listeners and every other query producer first, then closes the queue, then waits for the writer to finish emptying it — the held batch and everything still queued get written. If free space is below the critical threshold and the disk monitor will not let that final write through, the rows are counted as dropped instead of holding the exit open indefinitely.
- **An upstream success rate no longer rounds up to 100.0% while failures stand.** One decimal place cannot hold 12,696 successes out of 12,698 attempts: it rounded to `100.0%`, so the row claimed perfect reliability next to a failure count of 2. Neither end of the scale is reachable by rounding any more — `100.0%` needs an actual absence of failures and `0.0%` an actual absence of successes, and a rate a hair off either end shows `99.9%` or `0.1%` instead.
- **A query log set aside by a schema change is no longer named `corrupt`.** Every recreate wrote the old file to `querylog.db.corrupt-<unix seconds>`, whatever sent it there — including the fingerprint mismatch an upgrade causes, where the file is a healthy database this build simply cannot read. The name is the only account of the reason that outlives the log line, so it read as an accusation and invited operators to delete an intact file. The name now says which of the four cases it hit: `querylog.db.corrupt-…`, `.not-a-database-…`, `.quick-check-failed-…` or `.schema-changed-…`. The 0.0.6 upgrade produces `schema-changed`. Nothing else about the recreate changed, and no existing aside file is renamed.
## [0.0.6] - 2026-08-17
The period picker now scopes the whole dashboard. The upstream table was the last widget that ignored it, and fixing that meant recording upstream outcomes over time instead of counting them since boot. Read the query-log note below before you upgrade.
### Added
- Four metrics for the new upstream-history recorder: `nxdns_upstream_history_flushes_total`, `nxdns_upstream_history_flush_failures_total`, `nxdns_upstream_history_rows_dropped_total` and the `nxdns_upstream_history_pending` gauge. While a flush to the database keeps failing, `GET /api/health` reports `degraded`; it recovers on the next flush that succeeds.
### Changed
- **Upstream health answers for the selected period.** The dashboard's upstream table used to print counters accumulated since process start beside a success rate taken over the last 32 exchanges, which is how "63 failures" and "100.0% success rate" ended up in the same row under a period picker that scoped nothing there. Every upstream outcome is now aggregated into its wall-clock minute and written to `querylog.db`, and `GET /api/upstream/health?period=…` serves the selected window: attempts, failures, success rate, and the last failure with its error name, all inside the period, from 31 days of history. A window with no attempts reports no success rate at all instead of a perfect one, and the table shows an em-dash. The in-memory health state that drives failover and backoff is unchanged, as are its `/metrics` series.
- **`GET /api/upstream/health` changed shape.** Gone from each upstream: `consecutive_failures`, `total_successes`, `total_failures`, the last-32 `success_rate`, `last_error` and `last_error_age_s`. Each upstream keeps `url`, `enabled` and `available` and gains a `period` object with the ranged numbers; the body gains `period`, `since`, `until` and a `complete` flag that says whether any outcome was known to be dropped inside the window. The removed counters are still exported by `/metrics` under their existing names. On the dashboard the "Right now" section is gone with them: the upstream table rejoined the ranged part of the page, and the disk card, the one live widget left, is titled "Storage now".
- **The query log is recreated on upgrade.** Recording upstream history added two tables to the `querylog.db` schema, and its fingerprint check refuses a database that does not match the shipped definition. On first start this version renames the existing `querylog.db` aside as `querylog.db.corrupt-<unix seconds>` in the data directory and creates a fresh one, so query history and stats restart empty. The renamed file is left in place rather than deleted, so removing it is your call. `config.db` is untouched: no configuration is lost.
## [0.0.5] - 2026-08-16
One rendering fix on the 0.0.4 feature, caught the day it shipped.
### Changed
- The query tables no longer repeat the *learned* tag on every row: in the live page and the query log a learned name is just muted, with the address still in the row's tooltip. The clients page keeps the tag, where it appears once per client and says something.
## [0.0.4] - 2026-08-16
The names learned in 0.0.3 now show up where queries do: the live page and the query log name each client instead of printing its address.
+41 -3
View File
@@ -325,7 +325,7 @@ Per-group boolean. Rewrites known engine domains to their safe-search CNAME targ
- Schemes: `https://…` → DoH, `tls://host:853` → DoT.
- Ordered by priority; sequential attempt; per-upstream failure counters; exponential backoff with jitter; success resets.
- `UpstreamHealth` per upstream: last_success_at, last_error_at, last_error_message, rolling success rate, consecutive failures, backoff-until. Exposed via `GET /api/upstream/health`, dashboard, `/metrics`, and `nxdns check`.
- `UpstreamHealth` per upstream: last_success_at, last_error_at, last_error_message, rolling success rate, consecutive failures, backoff-until. This is routing state: it drives failover and backoff, and is exposed through `/metrics` and `nxdns check`. `GET /api/upstream/health?period=…` exposes none of it except the live `enabled`/`available` pair; its counts, success rate and last failure are ranged aggregates read from the per-minute upstream history in `querylog.db`, so the dashboard's period scopes them like every other number on the page.
- DoH client: `std.http.Client` with `content-type/accept: application/dns-message`; strict status + payload checks.
- `platform/tls_client.zig` enforces per-connection read/write deadlines, classifies TLS errors explicitly, retries with backoff. Integration tests cover timeout/hang scenarios so compiler upgrades can't silently regress them.
- Connect, read, and total-budget timeouts each configurable.
@@ -435,8 +435,28 @@ CREATE TABLE forward_zones (
);
CREATE TABLE settings (key TEXT PRIMARY KEY, value TEXT NOT NULL);
CREATE TABLE operational_events (
id INTEGER PRIMARY KEY,
code TEXT NOT NULL,
subject_key TEXT NOT NULL,
subject_label TEXT NOT NULL,
severity TEXT NOT NULL CHECK (severity IN ('warning', 'error')),
first_seen INTEGER NOT NULL,
last_seen INTEGER NOT NULL,
occurrences INTEGER NOT NULL CHECK (occurrences > 0),
resolved_at INTEGER,
detail TEXT NOT NULL DEFAULT '',
CHECK (resolved_at IS NULL OR resolved_at >= first_seen)
);
CREATE UNIQUE INDEX idx_operational_events_active
ON operational_events(code, subject_key) WHERE resolved_at IS NULL;
CREATE INDEX idx_operational_events_last_seen
ON operational_events(last_seen DESC);
```
`operational_events` is the one table here that is **not** configuration. It is the diagnostics log of `src/storage/events.zig`: one row per failure episode, opened on the first failure and resolved when the same subject succeeds again. It is deliberately absent from `config_schema.table_names` and `config_schema.delete_order`, so `nxdns export` never emits it and `nxdns import` never wipes it.
### 11.3 querylog.db Schema
```sql
@@ -460,12 +480,30 @@ CREATE TABLE query_log (
CREATE INDEX idx_query_log_ts ON query_log(timestamp);
CREATE INDEX idx_query_log_client ON query_log(client_ip);
CREATE INDEX idx_query_log_domain ON query_log(domain_id);
CREATE TABLE upstream_targets (
id INTEGER PRIMARY KEY,
url TEXT NOT NULL UNIQUE -- the historical identity: config.db ids cannot cross database files
);
CREATE TABLE upstream_minute (
upstream_id INTEGER NOT NULL REFERENCES upstream_targets(id),
minute_ts INTEGER NOT NULL,
successes INTEGER NOT NULL,
failures INTEGER NOT NULL,
last_failure_ts INTEGER,
last_error TEXT,
PRIMARY KEY (upstream_id, minute_ts),
CHECK (successes >= 0),
CHECK (failures >= 0)
) WITHOUT ROWID;
CREATE INDEX idx_upstream_minute_ts ON upstream_minute(minute_ts);
```
### 11.4 Query Logger
- In-memory buffer, mutex guarded, hard cap `query_log_buffer_max` (default 10000).
- Flush: batch size (default 100) or max interval (default 100ms).
- Flush: batch size (100, comptime) or max interval `query_log_flush_interval_s` (default 60s, 03600, 0 = do not wait). One transaction per interval: at household query rates a per-query commit costs orders of magnitude more disk writes than the rows are worth. The interval is also roughly what a crash costs, while the writer is healthy and the disk gate is open — a gated or lock-delayed batch is older, so it is a normal case, not a bound.
- Privacy transforms (hide_domains / hide_client_ips) applied before persist + SSE fanout.
- Backpressure: buffer full → drop oldest unflushed entry, increment monotonic `queries_dropped` (exposed in `/api/health` + `/metrics`). SSE fanout precedes buffer insert, so live viewers still see dropped-from-persistence entries.
@@ -522,7 +560,7 @@ Scalars in `settings(key, value)`; ordered/structured items in dedicated tables.
- `GET /api/lookup?domain=…&group_id=…`
- `GET/POST /api/pause`
- `GET/PUT /api/settings`
- `GET /api/upstream/health`
- `GET /api/upstream/health?period=…`
- `POST /api/certs/reload`
- `GET /api/health` — overall + disk + upstream + queries_dropped rollup
- `GET /metrics` — Prometheus text exposition: query counters (total/blocked/cached), per-upstream health, cache stats, queries_dropped, disk gauges
+2 -2
View File
@@ -87,8 +87,8 @@ test("429 login shows a ticking countdown and keeps submit disabled until it end
test("safeRedirect only allows same-origin absolute paths", () => {
expect(safeRedirect(undefined)).toBe("/");
expect(safeRedirect("/queries")).toBe("/queries");
expect(safeRedirect("/queries?x=1")).toBe("/queries?x=1");
expect(safeRedirect("/activity")).toBe("/activity");
expect(safeRedirect("/activity?x=1")).toBe("/activity?x=1");
expect(safeRedirect("//evil.example")).toBe("/");
expect(safeRedirect("https://evil.example")).toBe("/");
expect(safeRedirect("/\\evil.example")).toBe("/");
@@ -0,0 +1,312 @@
import { render, screen, waitFor, within } from "@testing-library/react";
import { QueryClientProvider } from "@tanstack/react-query";
import { RouterProvider, createMemoryHistory } from "@tanstack/react-router";
import { AuthProvider } from "@/auth/store";
import { createQueryClient } from "@/lib/queryClient";
import { createAppRouter } from "@/routes";
import type { QueryDetail } from "@/lib/types";
import { provenance } from "@/features/queries/provenanceFixture";
function detail(id: number, sections: Parameters<typeof provenance>[0] = {}): QueryDetail {
return { id, ...provenance(sections) };
}
let responses: Record<string, unknown>;
beforeEach(() => {
responses = {
"/api/version": { version: "0.0.0-test", git_commit: "0000000", zig_version: "0.16.0", uptime_seconds: 1 },
};
vi.stubGlobal(
"fetch",
vi.fn(async (input: RequestInfo | URL) => {
const payload = responses[String(input)];
if (payload === undefined) {
return new Response(JSON.stringify({ error: "no such query" }), {
status: 404,
headers: { "content-type": "application/json" },
});
}
return new Response(JSON.stringify(payload), {
status: 200,
headers: { "content-type": "application/json" },
});
}),
);
});
afterEach(() => vi.unstubAllGlobals());
function renderDetail(id: number, search = "") {
const queryClient = createQueryClient();
const router = createAppRouter(
createMemoryHistory({ initialEntries: [`/activity/queries/${id}${search}`] }),
queryClient,
);
render(
<AuthProvider>
<QueryClientProvider client={queryClient}>
<RouterProvider router={router} />
</QueryClientProvider>
</AuthProvider>,
);
return router;
}
/** The search parameters a link carries, so an assertion states them by name. */
function hrefSearch(link: HTMLElement): Record<string, string> {
const query = link.getAttribute("href")?.split("?")[1] ?? "";
return Object.fromEntries(new URLSearchParams(query));
}
/** The value beside a term, so a section's facts are read as pairs. */
function factValue(label: string): string {
const term = screen.getByText(label);
const value = term.nextElementSibling;
return value?.textContent ?? "";
}
/** The line under the domain, which is what a page is read as at a glance. */
function subtitle(domain: string): string {
const heading = screen.getByRole("heading", { name: domain });
return heading.nextElementSibling?.textContent ?? "";
}
test("a blocked query explains itself in the six sections, in pipeline order", async () => {
responses["/api/queries/42"] = detail(42, {
request: { time: 1_700_000_000, domain: "ads.example", client: "192.0.2.11", qtype: 28, qclass: 1 },
group: { id: 3, name: "kids" },
policy: {
action: "block",
reason: "blocklist_wildcard",
matched: "||tracker.example^",
source_id: 5,
source_name: "StevenBlack",
},
rewrites: { cname_target: "cdn.tracker.example" },
route: { kind: "blocked", upstream: "" },
response: { rcode: 0, duration_us: 1234 },
});
renderDetail(42);
await screen.findByRole("heading", { name: "ads.example" });
const headings = screen.getAllByRole("heading", { level: 2 }).map((node) => node.textContent);
expect(headings).toEqual(["Request", "Group", "Policy", "Rewrites", "Route", "Response", "Related"]);
expect(factValue("Client")).toBe("192.0.2.11");
expect(factValue("Type")).toBe("AAAA");
expect(factValue("Class")).toBe("IN (1)");
expect(factValue("Name")).toBe("kids");
expect(factValue("Id")).toBe("3");
expect(factValue("Decision")).toBe("Blocked");
expect(factValue("Reason")).toBe("Blocklist (wildcard)");
expect(factValue("Matched")).toBe("||tracker.example^");
expect(factValue("Blocklist")).toBe("StevenBlack (#5)");
expect(factValue("CNAME target")).toBe("cdn.tracker.example");
expect(factValue("Answered by")).toBe("Blocked locally");
expect(factValue("Upstream")).toBe("No upstream exchange");
expect(factValue("Result")).toBe("NOERROR (0)");
expect(factValue("Took")).toBe("1.2 ms");
// NOERROR is the ordinary case and adds nothing to the verdict.
expect(subtitle("ads.example")).toMatch(/ — Blocked$/);
});
test("an upstream SERVFAIL names the resolver that failed and the code the client saw", async () => {
responses["/api/queries/7"] = detail(7, {
request: { domain: "news.example" },
route: { kind: "upstream", upstream: "https://dns.example/dns-query" },
response: { rcode: 2, duration_us: null },
});
renderDetail(7);
await screen.findByRole("heading", { name: "news.example" });
expect(factValue("Answered by")).toBe("Upstream resolver");
expect(factValue("Upstream")).toBe("https://dns.example/dns-query");
expect(factValue("Result")).toBe("SERVFAIL (2)");
expect(factValue("Took")).toBe("Not measured");
// The policy allowed the query; the client still got nothing, and the
// headline has to say so rather than reading as a success.
expect(subtitle("news.example")).toMatch(/ — Allowed — SERVFAIL \(2\)$/);
});
test("a forward-zone answer says the matcher never ran, not that nothing matched", async () => {
responses["/api/queries/14"] = detail(14, {
request: { domain: "nas.lan.home" },
policy: { action: "allow", reason: "forward_zone", matched: "" },
route: { kind: "forward_zone", forward_zone: "lan.home", upstream: "udp://192.168.1.1:53" },
});
renderDetail(14);
await screen.findByRole("heading", { name: "nas.lan.home" });
expect(factValue("Reason")).toBe("Forward zone");
expect(factValue("Matched")).toBe("The matcher never ran");
});
test("a query the matcher did evaluate keeps the honest empty verdict", async () => {
responses["/api/queries/15"] = detail(15, {
policy: { action: "allow", reason: "no_match", matched: "" },
});
renderDetail(15);
await screen.findByRole("heading", { name: "example.com" });
expect(factValue("Reason")).toBe("No match");
expect(factValue("Matched")).toBe("Nothing matched");
});
test("empty text fields read as absent facts, never as blank values", async () => {
responses["/api/queries/8"] = detail(8, {
group: { id: null, name: "" },
policy: { action: "not_evaluated", reason: "paused", matched: "", source_id: null, source_name: "" },
route: { kind: "upstream", forward_zone: "", upstream: "udp://9.9.9.9:53" },
});
renderDetail(8);
await screen.findByRole("heading", { name: "example.com" });
expect(factValue("Name")).toBe("No group recorded");
expect(factValue("Matched")).toBe("The matcher never ran");
expect(factValue("Blocklist")).toBe("Not a blocklist decision");
expect(factValue("Safe search")).toBe("No rewrite");
expect(factValue("Reason")).toBe("Filtering paused");
});
test("a log with hidden domains renders the server's marker, with nothing invented around it", async () => {
responses["/api/queries/9"] = detail(9, {
request: { domain: "hidden" },
policy: { action: "block", reason: "blocklist_domain", matched: "hidden", source_name: "StevenBlack" },
rewrites: { cname_target: "hidden", safe_search_target: "hidden" },
route: { kind: "blocked", upstream: "" },
});
renderDetail(9);
await screen.findByRole("heading", { name: "hidden" });
expect(factValue("Domain")).toBe("hidden");
expect(factValue("Matched")).toBe("hidden");
expect(factValue("CNAME target")).toBe("hidden");
expect(factValue("Safe search")).toBe("hidden");
// The client is governed by its own flag and stays visible here.
expect(factValue("Client")).toBe("192.0.2.10");
});
test("the related actions carry absolute bounds around the query, and the domain into the simulation", async () => {
responses["/api/queries/11"] = detail(11, {
request: { time: 1_700_000_000, domain: "shop.example", client: "192.0.2.12" },
});
renderDetail(11);
await screen.findByRole("heading", { name: "shop.example" });
const related = screen.getByRole("heading", { name: "Related" }).parentElement!;
expect(
within(related)
.getByRole("link", { name: /Test this domain/ })
.getAttribute("href"),
).toBe("/activity/test?domain=shop.example");
// No origin bound at all: five minutes either side of the query itself.
expect(hrefSearch(within(related).getByRole("link", { name: "All activity for this domain" }))).toEqual({
mode: "history",
domain: "shop.example",
since: "1699999700",
until: "1700000300",
});
expect(hrefSearch(within(related).getByRole("link", { name: "All activity from this client" }))).toEqual({
mode: "history",
client: "192.0.2.12",
since: "1699999700",
until: "1700000300",
});
// The diagnostics window is the query's own moment, never the origin's.
expect(hrefSearch(within(related).getByRole("link", { name: /Diagnostics around/ }))).toEqual({
since: "1699999700",
until: "1700000300",
});
});
test("an origin bound wins over the default window, one bound at a time", async () => {
responses["/api/queries/17"] = detail(17, {
request: { time: 1_700_000_000, domain: "shop.example", client: "192.0.2.12" },
});
renderDetail(17, "?mode=history&since=1600000000");
await screen.findByRole("heading", { name: "shop.example" });
const related = screen.getByRole("heading", { name: "Related" }).parentElement!;
const link = hrefSearch(within(related).getByRole("link", { name: "All activity for this domain" }));
expect(link["since"]).toBe("1600000000");
expect(link["until"]).toBe("1700000300");
});
test("both origin bounds carry through untouched", async () => {
responses["/api/queries/18"] = detail(18, { request: { time: 1_700_000_000, domain: "shop.example" } });
renderDetail(18, "?mode=history&since=1600000000&until=1600000060");
await screen.findByRole("heading", { name: "shop.example" });
const related = screen.getByRole("heading", { name: "Related" }).parentElement!;
const link = hrefSearch(within(related).getByRole("link", { name: "All activity for this domain" }));
expect(link["since"]).toBe("1600000000");
expect(link["until"]).toBe("1600000060");
});
test("the back link restores the investigation the reader came from", async () => {
responses["/api/queries/19"] = detail(19, { request: { domain: "shop.example" } });
renderDetail(19, "?mode=history&domain=shop&since=1600000000&blocked=true");
await screen.findByRole("heading", { name: "shop.example" });
expect(hrefSearch(screen.getByRole("link", { name: "← Activity" }))).toEqual({
mode: "history",
domain: "shop",
since: "1600000000",
blocked: "true",
});
});
function clientList(client: { ip: string; name: string; learned_name: string }) {
return {
clients: [
{
id: 1,
group_id: 1,
group: "default",
hand_edited: client.name !== "",
first_seen: 1_700_000_000,
last_seen: 1_700_000_100,
...client,
},
],
};
}
/** The related section, whose text is read whole because it is prose, not facts. */
async function relatedText(expected: string) {
const related = screen.getByRole("heading", { name: "Related" }).parentElement!;
await waitFor(() => expect(related.textContent?.replace(/\s+/g, " ")).toContain(expected));
}
test("the record keeps the address the query came from, and Related carries the name it has now", async () => {
responses["/api/queries/12"] = detail(12, { request: { domain: "shop.example", client: "192.0.2.12" } });
responses["/api/clients"] = clientList({ ip: "192.0.2.12", name: "Kids iPad", learned_name: "ipad.lan" });
renderDetail(12);
await screen.findByRole("heading", { name: "shop.example" });
await relatedText("The client list currently names 192.0.2.12 “Kids iPad”.");
expect(factValue("Client")).toBe("192.0.2.12");
});
test("a learned name is told as the reverse-DNS lookup it is, never as a recorded fact", async () => {
responses["/api/queries/13"] = detail(13, { request: { client: "192.0.2.13" } });
responses["/api/clients"] = clientList({ ip: "192.0.2.13", name: "", learned_name: "printer.lan" });
renderDetail(13);
await screen.findByRole("heading", { name: "example.com" });
await relatedText("Reverse DNS currently resolves 192.0.2.13 to printer.lan.");
expect(factValue("Client")).toBe("192.0.2.13");
});
test("a row retention has pruned explains the 404 and keeps the way back to the log", async () => {
renderDetail(404, "?mode=history&domain=gone");
await screen.findByRole("alert");
expect(screen.getByText(/no such query/)).toBeTruthy();
expect(hrefSearch(screen.getByRole("link", { name: "← Activity" }))).toEqual({
mode: "history",
domain: "gone",
});
});
@@ -0,0 +1,75 @@
import { useQuery } from "@tanstack/react-query";
import { Link, useParams, useSearch } from "@tanstack/react-router";
import * as stylex from "@stylexjs/stylex";
import InlineError from "@/lib/InlineError";
import { queryDetailQuery } from "@/lib/queries";
import type { QueryDetail } from "@/lib/types";
import { styles as shared } from "@/ui/styles";
import { colors } from "@/ui/tokens.stylex";
import ProvenanceDetail from "./ProvenanceDetail";
import RelatedActions from "./RelatedActions";
import type { ActivitySearch } from "./search";
const styles = stylex.create({
back: {
fontSize: "0.875rem",
lineHeight: "1.25rem",
color: colors.primaryOnSurface,
textDecorationLine: "none",
},
loading: {
marginTop: "1rem",
color: colors.textMuted,
},
});
/**
* The way back to the investigation, not to a bare list. The originating
* Activity search rides in this route's own search, so the reader returns to
* the mode, the filters and the absolute window they left — a plain `/activity`
* would silently widen the range they had chosen.
*/
function BackLink({ origin }: { origin: ActivitySearch }) {
return (
<Link to="/activity" search={origin} {...stylex.props(styles.back, shared.focusRing)}>
Activity
</Link>
);
}
export default function ActivityDetailPage() {
const { id } = useParams({ from: "/shell/activity/queries/$id" });
const origin = useSearch({ from: "/shell/activity/queries/$id" });
const rowId = Number(id);
const { data, error, isPending, refetch } = useQuery(queryDetailQuery(rowId));
if (isPending) {
return (
<p {...stylex.props(styles.loading, shared.pulse)} role="status">
Loading query
</p>
);
}
if (data === undefined) {
return (
<section>
<BackLink origin={origin} />
<InlineError error={error} onRetry={() => void refetch()} />
</section>
);
}
const detail: QueryDetail = data;
const { domain, client, time } = detail.request;
return (
<section>
<BackLink origin={origin} />
<ProvenanceDetail
provenance={detail}
persistedId={detail.id}
relatedActions={<RelatedActions domain={domain} client={client} ts={time} origin={origin} />}
/>
</section>
);
}
@@ -0,0 +1,230 @@
/**
* The filter row over the Activity table.
*
* The applied state is the URL, never this form: what the reader sees is what
* the link they can paste to a housemate will show. So this holds a draft only,
* and the page remounts it whenever the applied search changes — a back button
* or a pasted URL has to move the form with it, and a form that seeded itself
* once would keep showing the previous investigation's filters.
*
* In live mode the row stays visible and disabled rather than disappearing: the
* filters are retained in the URL and apply again the moment history comes
* back, and hiding them would read as having lost them. The stream itself is
* unfiltered — the server sends every query — so a row that looked usable here
* would promise filtering that is not happening.
*/
import { useState, type FormEvent } from "react";
import * as stylex from "@stylexjs/stylex";
import Select from "@/ui/Select";
import { styles as shared } from "@/ui/styles";
import { colors } from "@/ui/tokens.stylex";
import { datetimeField, editDatetimeField, resolveDatetimeField, type DatetimeField } from "./datetime";
import type { ActivitySearch } from "./search";
const STATUS_OPTIONS = [
{ value: "any", label: "All" },
{ value: "blocked", label: "Blocked only" },
{ value: "allowed", label: "Allowed only" },
];
const styles = stylex.create({
/** One column on a phone, two from `sm`, five from `lg`. */
grid: {
marginTop: "1rem",
display: "grid",
gap: "0.75rem",
gridTemplateColumns: {
default: "repeat(1, minmax(0, 1fr))",
"@media (min-width: 640px)": "repeat(2, minmax(0, 1fr))",
"@media (min-width: 1024px)": "repeat(5, minmax(0, 1fr))",
},
},
label: {
display: "block",
fontSize: "0.875rem",
lineHeight: "1.25rem",
},
input: {
marginTop: "0.25rem",
width: "100%",
// A disabled native input keeps its value legible but reads as inert,
// matching what RAC does to the Select trigger beside it.
cursor: { default: null, ":disabled": "not-allowed" },
opacity: { default: null, ":disabled": 0.55 },
},
buttonRow: {
display: "flex",
alignItems: "flex-end",
gap: "0.5rem",
gridColumn: {
default: null,
"@media (min-width: 640px)": "span 2 / span 2",
"@media (min-width: 1024px)": "span 5 / span 5",
},
},
toolbarButton: {
fontWeight: 500,
},
error: {
marginTop: "0.5rem",
fontSize: "0.875rem",
lineHeight: "1.25rem",
color: colors.dangerText,
},
});
/** The applied filters, with `mode` left to the page that owns the switch. */
export type AppliedFilters = Omit<ActivitySearch, "mode">;
/** Every filter off — what Clear applies, and the loader's empty-filter case. */
export const NO_FILTERS: AppliedFilters = {
domain: undefined,
client: undefined,
blocked: undefined,
since: undefined,
until: undefined,
};
function blockedOption(blocked: boolean | undefined): string {
if (blocked === undefined) return "any";
return blocked ? "blocked" : "allowed";
}
function optionBlocked(value: string): boolean | undefined {
if (value === "blocked") return true;
return value === "allowed" ? false : undefined;
}
function boundError(label: string, reason: "unparseable" | "nonexistent"): string {
return reason === "unparseable"
? `${label} is not a complete date and time.`
: `${label} names a local time that does not exist — the clock jumps over it for daylight saving.`;
}
interface Props {
applied: AppliedFilters;
isDisabled: boolean;
onApply: (filters: AppliedFilters) => void;
onClear: () => void;
}
export default function ActivityFilters({ applied, isDisabled, onApply, onClear }: Props) {
const [domain, setDomain] = useState(applied.domain ?? "");
const [client, setClient] = useState(applied.client ?? "");
const [blocked, setBlocked] = useState(blockedOption(applied.blocked));
const [since, setSince] = useState<DatetimeField>(() => datetimeField(applied.since));
const [until, setUntil] = useState<DatetimeField>(() => datetimeField(applied.until));
const [error, setError] = useState<string | null>(null);
function submit(event: FormEvent) {
event.preventDefault();
const sinceValue = resolveDatetimeField(since);
if (!sinceValue.ok) {
setError(boundError("Since", sinceValue.reason));
return;
}
const untilValue = resolveDatetimeField(until);
if (!untilValue.ok) {
setError(boundError("Until", untilValue.reason));
return;
}
setError(null);
onApply({
domain: domain.trim() === "" ? undefined : domain.trim(),
client: client.trim() === "" ? undefined : client.trim(),
blocked: optionBlocked(blocked),
since: sinceValue.value,
until: untilValue.value,
});
}
function clear() {
setDomain("");
setClient("");
setBlocked("any");
setSince(datetimeField(undefined));
setUntil(datetimeField(undefined));
setError(null);
onClear();
}
return (
<>
<form onSubmit={submit} {...stylex.props(styles.grid)}>
<label {...stylex.props(styles.label)}>
Domain contains
<input
type="text"
value={domain}
disabled={isDisabled}
onChange={(event) => setDomain(event.target.value)}
{...stylex.props(shared.smallInput, styles.input, shared.focusRing)}
/>
</label>
<label {...stylex.props(styles.label)}>
Client (exact)
<input
type="text"
value={client}
disabled={isDisabled}
onChange={(event) => setClient(event.target.value)}
{...stylex.props(shared.smallInput, styles.input, shared.focusRing)}
/>
</label>
<Select
variant="compactField"
label="Result"
value={blocked}
isDisabled={isDisabled}
onChange={setBlocked}
options={STATUS_OPTIONS}
/>
<label {...stylex.props(styles.label)}>
Since
<input
type="datetime-local"
step={1}
value={since.text}
disabled={isDisabled}
onChange={(event) => setSince(editDatetimeField(since, event.target.value))}
{...stylex.props(shared.smallInput, styles.input, shared.focusRing)}
/>
</label>
<label {...stylex.props(styles.label)}>
Until
<input
type="datetime-local"
step={1}
value={until.text}
disabled={isDisabled}
onChange={(event) => setUntil(editDatetimeField(until, event.target.value))}
{...stylex.props(shared.smallInput, styles.input, shared.focusRing)}
/>
</label>
<div {...stylex.props(styles.buttonRow)}>
<button
type="submit"
disabled={isDisabled}
{...stylex.props(shared.button, styles.toolbarButton, shared.focusRing)}
>
Apply filters
</button>
<button
type="button"
onClick={clear}
disabled={isDisabled}
{...stylex.props(shared.button, styles.toolbarButton, shared.focusRing)}
>
Clear
</button>
</div>
</form>
{error !== null && (
<p role="alert" {...stylex.props(styles.error)}>
{error}
</p>
)}
</>
);
}
@@ -0,0 +1,554 @@
/**
* Activity in history mode, through the real router: the URL is the applied
* state, so nothing here can be checked by rendering the page on its own.
*/
import { act, fireEvent, render, screen, waitFor, within } from "@testing-library/react";
import { QueryClientProvider } from "@tanstack/react-query";
import { RouterProvider, createMemoryHistory } from "@tanstack/react-router";
import { AuthProvider } from "@/auth/store";
import { createQueryClient } from "@/lib/queryClient";
import { createAppRouter } from "@/routes";
import type { Client, Coverage, QueriesPage, QueryRow } from "@/lib/types";
import { queryRow } from "@/features/queries/provenanceFixture";
function client(id: number, ip: string, name: string, learnedName: string): Client {
return {
id,
ip,
name,
learned_name: learnedName,
group_id: 1,
group: "default",
hand_edited: name !== "",
first_seen: 1_700_000_000,
last_seen: 1_700_000_100,
};
}
const CLIENTS: Client[] = [
client(1, "192.0.2.10", "Kitchen Pi", "pi.lan"),
client(2, "192.0.2.11", "", "laptop.lan"),
client(3, "192.0.2.12", "", ""),
];
function row(id: number, domain: string, overrides: Partial<QueryRow> = {}): QueryRow {
return queryRow(id, { ts: 1_700_000_000 + id, domain, upstream: "udp://9.9.9.9:53", ...overrides });
}
const COMPLETE: Coverage = { complete: true, available_since: 1_600_000_000 };
/** The blocked row every page fixture reuses. */
const BLOCKED = {
blocked: true,
policy_action: "block",
policy_reason: "blocklist_wildcard",
route_kind: "blocked",
upstream: "",
} as const satisfies Partial<QueryRow>;
const PAGES: Record<string, QueriesPage> = {
"/api/queries": {
queries: [
row(20, "first.example", { qtype: 65, cache_hit: true, route_kind: "cache" }),
row(19, "ads.example", { ...BLOCKED, response_time_us: null, cache_hit: null }),
],
next_before: 19,
coverage: COMPLETE,
},
"/api/queries?before=19": {
queries: [row(5, "older.example")],
next_before: null,
coverage: COMPLETE,
},
"/api/queries?domain=ads": {
queries: [row(19, "ads.example", BLOCKED)],
next_before: null,
coverage: COMPLETE,
},
"/api/queries?domain=ads&blocked=true": {
queries: [row(19, "ads.example", BLOCKED)],
next_before: null,
coverage: COMPLETE,
},
"/api/queries?since=1700000000": {
queries: [row(20, "first.example")],
next_before: null,
coverage: COMPLETE,
},
};
let fetchMock: ReturnType<typeof vi.fn>;
function json(payload: unknown): Response {
return new Response(JSON.stringify(payload), { status: 200, headers: { "content-type": "application/json" } });
}
const VERSION = { version: "0.0.0-test", git_commit: "0000000", zig_version: "0.16.0", uptime_seconds: 1 };
/** The shell's own requests, which every test serves the same way. */
function stubFetch(handler: (url: string) => Response | Promise<Response>) {
fetchMock = vi.fn((input: RequestInfo | URL) => {
const url = String(input);
if (url === "/api/version") return Promise.resolve(json(VERSION));
return Promise.resolve(handler(url));
});
vi.stubGlobal("fetch", fetchMock);
}
function fromPages(url: string): Response {
if (url === "/api/clients") return json({ clients: CLIENTS });
const payload = PAGES[url];
if (payload === undefined) return new Response(JSON.stringify({ error: "not stubbed" }), { status: 404 });
return json(payload);
}
beforeEach(() => {
stubFetch(fromPages);
});
afterEach(() => {
vi.unstubAllGlobals();
});
function renderPage(path = "/activity") {
const queryClient = createQueryClient();
const history = createMemoryHistory({ initialEntries: [path] });
const router = createAppRouter(history, queryClient);
render(
<AuthProvider>
<QueryClientProvider client={queryClient}>
<RouterProvider router={router} />
</QueryClientProvider>
</AuthProvider>,
);
return { queryClient, history };
}
/** Every `/api/queries` URL the run asked for, list pages only. */
function queryCalls(): string[] {
return fetchMock.mock.calls
.map((call) => String(call[0]))
.filter((url) => url === "/api/queries" || url.startsWith("/api/queries?"));
}
test("renders the first page with the seven columns filled in", async () => {
renderPage();
await screen.findByText("first.example");
expect(screen.getAllByRole("columnheader").map((header) => header.textContent)).toEqual([
"Time",
"Domain",
"Client",
"Type",
"Result",
"Route",
"Duration",
]);
const first = screen.getByText("first.example").closest("tr")!;
expect(within(first).getByText("HTTPS")).toBeTruthy();
expect(within(first).getByText("NOERROR")).toBeTruthy();
expect(within(first).getByText("Cache")).toBeTruthy();
expect(within(first).getByText("1.2 ms")).toBeTruthy();
const blocked = screen.getByText("ads.example").closest("tr")!;
// The Result cell says Blocked even though the client saw NOERROR, and the
// Route cell says how: this is the pair the old Status column could not show.
expect(within(blocked).getAllByText("Blocked")).toHaveLength(2);
expect(within(blocked).getByText("—")).toBeTruthy();
expect(screen.getByText(/Showing 2 queries/)).toBeTruthy();
});
test("resolves each row's client to its display name, keeping the IP as the tooltip", async () => {
stubFetch((url) => {
if (url === "/api/clients") return json({ clients: CLIENTS });
if (url !== "/api/queries") return new Response(JSON.stringify({ error: "not stubbed" }), { status: 404 });
return json({
queries: [
row(20, "named.example", { client_ip: "192.0.2.10" }),
row(19, "learned.example", { client_ip: "192.0.2.11" }),
row(18, "nameless.example", { client_ip: "192.0.2.12" }),
row(17, "stranger.example", { client_ip: "192.0.2.99" }),
],
next_before: null,
coverage: COMPLETE,
} satisfies QueriesPage);
});
renderPage();
// A hand-typed name wins outright; the learned name never surfaces for it.
const named = await screen.findByText("Kitchen Pi");
expect(named.getAttribute("title")).toBe("192.0.2.10");
expect(screen.queryByText("pi.lan")).toBeNull();
// A learned name reads muted and nothing more here: the "learned" tag would
// repeat on every row of the table, so the Clients page carries it instead.
const learned = screen.getByText("laptop.lan");
expect(learned.getAttribute("title")).toBe("192.0.2.11");
expect(within(learned.closest("tr")!).queryByText("learned")).toBeNull();
// A known client with neither name, and a client the loaded list has never
// seen, both fall back to the bare address with no tooltip standing in.
expect(screen.getByText("192.0.2.12").getAttribute("title")).toBeNull();
expect(screen.getByText("192.0.2.99").getAttribute("title")).toBeNull();
});
test("load more appends the next page and stops at the end of the log", async () => {
renderPage();
await screen.findByText("first.example");
fireEvent.click(screen.getByRole("button", { name: "Load more" }));
await screen.findByText("older.example");
expect(screen.getByText("first.example")).toBeTruthy();
expect(screen.getByText(/Showing 3 queries — end of log/)).toBeTruthy();
expect(screen.queryByRole("button", { name: "Load more" })).toBeNull();
});
test("applying a filter puts it in the url, refetches, and resets the accumulated list", async () => {
const { history } = renderPage();
await screen.findByText("first.example");
fireEvent.click(screen.getByRole("button", { name: "Load more" }));
await screen.findByText("older.example");
fireEvent.change(screen.getByLabelText("Domain contains"), { target: { value: "ads" } });
fireEvent.click(screen.getByRole("button", { name: "Apply filters" }));
await screen.findByText(/Showing 1 query /);
expect(history.location.search).toContain("domain=ads");
expect(screen.getByText("ads.example")).toBeTruthy();
expect(screen.queryByText("first.example")).toBeNull();
expect(screen.queryByText("older.example")).toBeNull();
});
test("a load-more that resolves after a filter change is discarded", async () => {
let releaseLoadMore: () => void = () => {};
stubFetch((url) => {
if (url === "/api/queries?before=19") {
return new Promise<Response>((resolve) => {
releaseLoadMore = () => resolve(json(PAGES["/api/queries?before=19"]));
});
}
return fromPages(url);
});
renderPage();
await screen.findByText("first.example");
fireEvent.click(screen.getByRole("button", { name: "Load more" }));
fireEvent.change(screen.getByLabelText("Domain contains"), { target: { value: "ads" } });
fireEvent.click(screen.getByRole("button", { name: "Apply filters" }));
await screen.findByText(/Showing 1 query /);
releaseLoadMore();
await act(async () => {
await new Promise((resolve) => setTimeout(resolve, 0));
});
expect(screen.queryByText("older.example")).toBeNull();
expect(screen.getByText(/Showing 1 query /)).toBeTruthy();
expect(screen.queryByRole("alert")).toBeNull();
});
test("load more is disabled while a filter change shows placeholder data, then uses the fresh cursor", async () => {
let releaseFiltered: () => void = () => {};
const filteredPage: QueriesPage = {
queries: [row(19, "ads.example", BLOCKED)],
next_before: 7,
coverage: COMPLETE,
};
const filteredOlderPage: QueriesPage = {
queries: [row(3, "ads.older.example")],
next_before: null,
coverage: COMPLETE,
};
stubFetch((url) => {
if (url === "/api/queries?domain=ads") {
return new Promise<Response>((resolve) => {
releaseFiltered = () => resolve(json(filteredPage));
});
}
if (url === "/api/queries?domain=ads&before=7") return json(filteredOlderPage);
return fromPages(url);
});
renderPage();
await screen.findByText("first.example");
fireEvent.change(screen.getByLabelText("Domain contains"), { target: { value: "ads" } });
fireEvent.click(screen.getByRole("button", { name: "Apply filters" }));
const staleButton = await screen.findByRole("button", { name: "Load more" });
expect(staleButton).toHaveProperty("disabled", true);
fireEvent.click(staleButton);
expect(queryCalls()).not.toContain("/api/queries?domain=ads&before=19");
releaseFiltered();
await waitFor(() => {
expect(screen.queryByText("first.example")).toBeNull();
});
const freshButton = screen.getByRole("button", { name: "Load more" });
expect(freshButton).toHaveProperty("disabled", false);
fireEvent.click(freshButton);
await screen.findByText("ads.older.example");
expect(queryCalls()).toContain("/api/queries?domain=ads&before=7");
expect(screen.getByText(/Showing 2 queries — end of log/)).toBeTruthy();
});
test("a background refetch after new rows arrive leaves no gap between the loaded pages", async () => {
// The newest-100 window moves up while the reader has a second page open.
// Refetching only the first page would drop n20 and n19 out of the middle
// of the table; the second page must be replayed from the fresh cursor.
const before: Record<string, QueriesPage> = {
"/api/queries": {
queries: [row(20, "n20.example"), row(19, "n19.example")],
next_before: 19,
coverage: COMPLETE,
},
"/api/queries?before=19": {
queries: [row(18, "n18.example"), row(17, "n17.example")],
next_before: null,
coverage: COMPLETE,
},
};
const after: Record<string, QueriesPage> = {
"/api/queries": {
queries: [row(22, "n22.example"), row(21, "n21.example")],
next_before: 21,
coverage: COMPLETE,
},
"/api/queries?before=21": {
queries: [row(20, "n20.example"), row(19, "n19.example"), row(18, "n18.example"), row(17, "n17.example")],
next_before: null,
coverage: COMPLETE,
},
};
let live = before;
stubFetch((url) => {
const payload = live[url];
if (payload === undefined) return new Response(JSON.stringify({ error: "not stubbed" }), { status: 404 });
return json(payload);
});
const { queryClient } = renderPage();
await screen.findByText("n20.example");
fireEvent.click(screen.getByRole("button", { name: "Load more" }));
await screen.findByText("n17.example");
live = after;
await act(async () => {
await queryClient.invalidateQueries({ queryKey: ["queries"] });
});
await screen.findByText("n22.example");
const shown = screen.getAllByText(/^n\d+\.example$/).map((cell) => cell.textContent);
expect(shown).toEqual(["n22.example", "n21.example", "n20.example", "n19.example", "n18.example", "n17.example"]);
expect(screen.getByText(/Showing 6 queries — end of log/)).toBeTruthy();
});
test("a 401 on load more routes through handleUnauthorized instead of the inline error", async () => {
const assign = vi.fn();
vi.stubGlobal("location", { pathname: "/activity", search: "", assign });
stubFetch((url) => {
if (url === "/api/queries?before=19") {
return new Response(JSON.stringify({ error: "unauthorized" }), {
status: 401,
headers: { "content-type": "application/json" },
});
}
return fromPages(url);
});
renderPage();
await screen.findByText("first.example");
fireEvent.click(screen.getByRole("button", { name: "Load more" }));
await waitFor(() => {
expect(assign).toHaveBeenCalledWith(`/login?redirect=${encodeURIComponent("/activity")}`);
});
expect(screen.queryByRole("alert")).toBeNull();
expect(screen.queryByText(/Failed to load more/)).toBeNull();
});
test("each row links into its own detail page, carrying the investigation with it", async () => {
renderPage("/activity?mode=history&since=1700000000");
await screen.findByText("first.example");
const link = screen.getByRole("link", { name: "first.example" });
expect(link.getAttribute("href")).toContain("/activity/queries/20");
expect(link.getAttribute("href")).toContain("since=1700000000");
// An <a href> is in the tab order by default; nothing here may opt it out.
expect(link.getAttribute("tabindex")).toBeNull();
});
test("a pruned window tells the reader when history starts", async () => {
stubFetch((url) => {
if (url === "/api/clients") return json({ clients: CLIENTS });
if (url !== "/api/queries") return new Response(JSON.stringify({ error: "not stubbed" }), { status: 404 });
return json({
queries: [row(20, "kept.example")],
next_before: null,
coverage: { complete: false, available_since: 1_700_000_000 },
} satisfies QueriesPage);
});
renderPage();
await screen.findByText("kept.example");
expect(screen.getByText(/Query history is available from/)).toBeTruthy();
});
test("a complete window shows no coverage notice", async () => {
renderPage();
await screen.findByText("first.example");
expect(screen.queryByText(/Query history is available from/)).toBeNull();
});
test("a ?domain= link seeds the filter form and fetches that domain on arrival", async () => {
renderPage("/activity?domain=ads");
await screen.findByText("ads.example");
expect(screen.getByLabelText("Domain contains")).toHaveProperty("value", "ads");
expect(screen.queryByText("first.example")).toBeNull();
});
test("history forwards exactly the six normalized filter fields and nothing else", async () => {
stubFetch((url) => {
if (url === "/api/clients") return json({ clients: CLIENTS });
return json({ queries: [], next_before: null, coverage: COMPLETE } satisfies QueriesPage);
});
renderPage(
"/activity?mode=history&domain=%20ads%20&client=192.0.2.5&blocked=true&since=1700000000&until=1700000600&bogus=1&limit=9999",
);
await screen.findByText("No queries match the current filters.");
expect(queryCalls()).toEqual([
"/api/queries?domain=ads&client=192.0.2.5&blocked=true&since=1700000000&until=1700000600",
]);
});
test("a rejected search parameter is dropped rather than guessed at", async () => {
renderPage("/activity?since=1.5&blocked=%22true%22&domain=%20%20");
await screen.findByText("first.example");
// Nothing survived validation, so the request is the unfiltered one.
expect(queryCalls()).toEqual(["/api/queries"]);
expect(screen.getByLabelText("Domain contains")).toHaveProperty("value", "");
});
test("the form draft follows the url back and forward, seconds included", async () => {
stubFetch((url) => {
if (url === "/api/clients") return json({ clients: CLIENTS });
return json({ queries: [], next_before: null, coverage: COMPLETE } satisfies QueriesPage);
});
// A bound with non-zero seconds: the round trip has to keep them, and the
// untouched field has to carry the original number rather than re-parse.
const seeded = 1_700_000_017;
const { history } = renderPage(`/activity?mode=history&domain=first&since=${seeded}`);
const domainInput = await screen.findByLabelText("Domain contains");
expect(domainInput).toHaveProperty("value", "first");
const sinceInput = screen.getByLabelText("Since") as HTMLInputElement;
expect(sinceInput.value).toContain(":37");
fireEvent.change(domainInput, { target: { value: "second" } });
fireEvent.click(screen.getByRole("button", { name: "Apply filters" }));
await waitFor(() => expect(history.location.search).toContain("domain=second"));
// The untouched Since bound applied as the exact second it was seeded with.
expect(queryCalls()).toContain(`/api/queries?domain=second&since=${seeded}`);
act(() => history.back());
await waitFor(() => {
expect(screen.getByLabelText("Domain contains")).toHaveProperty("value", "first");
});
act(() => history.forward());
await waitFor(() => {
expect(screen.getByLabelText("Domain contains")).toHaveProperty("value", "second");
});
});
/** 2 a.m. on the EU spring-forward date: an hour that exists in some zones and not others. */
const DST_WALL_TIME = "2026-03-29T02:30:00";
/**
* Whether that wall time names an instant in the timezone the suite runs in.
* `new Date` slides a spring-forward gap silently forward, so an hour or minute
* that comes back different from the one written *is* the gap.
*/
function wallTimeExists(text: string): boolean {
const written = /T(\d{2}):(\d{2})/.exec(text)!;
const parsed = new Date(text);
return parsed.getHours() === Number(written[1]) && parsed.getMinutes() === Number(written[2]);
}
test("a wall-clock time the daylight-saving jump skips is refused, not silently moved", async () => {
// One expected outcome per timezone, decided here rather than accepted from
// the page: in a zone with the jump the bound must be refused outright, and
// in a zone without it the same text is an ordinary instant that applies.
const inGap = !wallTimeExists(DST_WALL_TIME);
const unix = Math.floor(new Date(DST_WALL_TIME).getTime() / 1000);
stubFetch((url) => {
if (url === `/api/queries?since=${unix}`) {
return json({ queries: [], next_before: null, coverage: COMPLETE } satisfies QueriesPage);
}
return fromPages(url);
});
const { history } = renderPage();
await screen.findByText("first.example");
const callsBefore = queryCalls().length;
const searchBefore = history.location.search;
fireEvent.change(screen.getByLabelText("Since"), { target: { value: DST_WALL_TIME } });
fireEvent.click(screen.getByRole("button", { name: "Apply filters" }));
if (inGap) {
expect(screen.getByRole("alert").textContent).toContain("daylight saving");
// Refused means refused: no navigation, and no request for the hour the
// operator did not ask for.
expect(history.location.search).toBe(searchBefore);
expect(queryCalls()).toHaveLength(callsBefore);
return;
}
await waitFor(() => expect(history.location.search).toContain(`since=${unix}`));
expect(queryCalls()).toContain(`/api/queries?since=${unix}`);
expect(screen.queryByRole("alert")).toBeNull();
});
test("the policy simulation is reachable from the header, with no rows to click through", async () => {
stubFetch((url) => {
if (url === "/api/clients") return json({ clients: CLIENTS });
if (url === "/api/groups") return json({ groups: [{ id: 1, name: "default", safe_search: false }] });
return json({ queries: [], next_before: null, coverage: COMPLETE } satisfies QueriesPage);
});
const { history } = renderPage();
// An empty log is exactly the case a row-borne link cannot serve.
await screen.findByText("No queries logged yet.");
const link = screen.getByRole("link", { name: "Current policy simulation" });
expect(link.getAttribute("href")).toBe("/activity/test");
fireEvent.click(link);
await screen.findByRole("heading", { level: 1, name: "Current policy simulation" });
expect(history.location.pathname).toBe("/activity/test");
});
test("Clear empties the url as well as the form", async () => {
const { history } = renderPage("/activity?mode=history&domain=ads&blocked=true");
await screen.findByText("ads.example");
fireEvent.click(screen.getByRole("button", { name: "Clear" }));
await waitFor(() => {
expect(history.location.search).not.toContain("domain");
});
expect(history.location.search).not.toContain("blocked");
expect(screen.getByLabelText("Domain contains")).toHaveProperty("value", "");
});
@@ -0,0 +1,152 @@
/**
* Activity: one surface over the queries nxdns answered, in two modes.
*
* History reads the persisted log and Live reads the stream, but they are the
* same seven columns over the same filters, and the reader moves between them
* without losing the question they were asking. The mode lives in the URL with
* the filters, so an investigation is one link — including which half of it the
* recipient should be looking at.
*/
import { Link, useNavigate, useSearch } from "@tanstack/react-router";
import * as stylex from "@stylexjs/stylex";
import { styles as shared } from "@/ui/styles";
import { colors } from "@/ui/tokens.stylex";
import ActivityFilters, { NO_FILTERS, type AppliedFilters } from "./ActivityFilters";
import HistoryActivity from "./HistoryActivity";
import LiveActivity from "./LiveActivity";
import type { ActivityMode } from "./search";
const MODES: ReadonlyArray<{ mode: ActivityMode; label: string }> = [
{ mode: "history", label: "History" },
{ mode: "live", label: "Live" },
];
const styles = stylex.create({
header: {
display: "flex",
flexWrap: "wrap",
alignItems: "center",
gap: "0.75rem",
},
heading: {
fontSize: "1.5rem",
lineHeight: "2rem",
fontWeight: 600,
},
switch: {
display: "flex",
gap: "0.25rem",
borderRadius: "0.25rem",
borderWidth: 1,
borderStyle: "solid",
borderColor: colors.border,
padding: "0.125rem",
},
modeButton: {
borderStyle: "none",
borderRadius: "0.1875rem",
paddingInline: "0.75rem",
paddingBlock: "0.25rem",
fontSize: "0.875rem",
lineHeight: "1.25rem",
fontWeight: 500,
cursor: "pointer",
},
modeIdle: {
backgroundColor: { default: "transparent", ":hover": colors.surfaceHover },
color: { default: colors.textSecondary, ":hover": colors.text },
},
/** The selected mode reads as a filled chip, the same weight the nav uses. */
modeSelected: {
backgroundColor: colors.primary,
color: colors.primaryText,
},
/** The one way into the simulation from here, so it cannot sit behind a row. */
simulationLink: {
marginInlineStart: "auto",
fontSize: "0.875rem",
lineHeight: "1.25rem",
color: colors.primaryOnSurface,
textDecorationLine: "none",
},
liveNote: {
marginTop: "0.5rem",
fontSize: "0.875rem",
lineHeight: "1.25rem",
color: colors.textMuted,
},
});
export default function ActivityPage() {
const search = useSearch({ from: "/shell/activity" });
const navigate = useNavigate({ from: "/activity" });
const live = search.mode === "live";
// The functional form, not a replacement object: the filters are retained
// across a mode switch on purpose, and spelling out a new search here would
// drop every one of them on the way to Live and back.
function selectMode(mode: ActivityMode) {
if (mode === search.mode) return;
void navigate({ search: (prev) => ({ ...prev, mode }) });
}
function apply(filters: AppliedFilters) {
void navigate({ search: { mode: search.mode, ...filters } });
}
return (
<section>
<div {...stylex.props(styles.header)}>
<h1 {...stylex.props(styles.heading)}>Activity</h1>
<div role="group" aria-label="Activity mode" {...stylex.props(styles.switch)}>
{MODES.map((option) => {
const selected = option.mode === search.mode;
return (
<button
key={option.mode}
type="button"
aria-pressed={selected}
onClick={() => selectMode(option.mode)}
{...stylex.props(
styles.modeButton,
selected ? styles.modeSelected : styles.modeIdle,
shared.focusRing,
)}
>
{option.label}
</button>
);
})}
</div>
<Link to="/activity/test" {...stylex.props(styles.simulationLink, shared.focusRing)}>
Current policy simulation
</Link>
</div>
{/*
* Remounted whenever the applied search changes, which is what makes
* the back button work: the draft is derived state, and the browser
* moving the URL under it has to move the form with it.
*/}
<ActivityFilters
key={`${search.domain ?? ""}|${search.client ?? ""}|${String(search.blocked)}|${String(search.since)}|${String(search.until)}`}
applied={search}
isDisabled={live}
onApply={apply}
onClear={() => apply(NO_FILTERS)}
/>
{live ? (
<>
<p {...stylex.props(styles.liveNote)}>
The stream carries every query the server answers; these filters apply to history only.
</p>
<LiveActivity origin={search} />
</>
) : (
<HistoryActivity search={search} />
)}
</section>
);
}
@@ -0,0 +1,181 @@
/**
* Activity in history mode: the persisted queries the URL's filters select,
* paged by keyset cursor.
*
* The filters arrive already applied the URL is the applied state so this
* only reads them. Everything about how the reader got here lives one level up.
*/
import { useInfiniteQuery } from "@tanstack/react-query";
import { Link } from "@tanstack/react-router";
import * as stylex from "@stylexjs/stylex";
import * as api from "@/lib/api";
import CoverageNotice from "@/lib/CoverageNotice";
import InlineError from "@/lib/InlineError";
import { queriesInfiniteQuery } from "@/lib/queries";
import type { QueryRow } from "@/lib/types";
import { useClientNames } from "@/features/clients/clientNames";
import { summarizeRow } from "@/features/queries/querySummary";
import { styles as shared } from "@/ui/styles";
import { colors } from "@/ui/tokens.stylex";
import { ActivityCells, ActivityTableHead, activityDomainLink } from "./cells";
import { queriesFilterOf, type ActivitySearch } from "./search";
const styles = stylex.create({
empty: {
marginTop: "1.5rem",
color: colors.textMuted,
},
tableWrap: {
marginTop: "1rem",
overflowX: "auto",
borderRadius: "0.25rem",
borderWidth: 1,
borderStyle: "solid",
borderColor: colors.border,
},
table: {
width: "100%",
fontSize: "0.875rem",
lineHeight: "1.25rem",
},
/** `divide-y`: a hairline between rows, so the first row carries none. */
row: {
borderTopWidth: { default: 1, ":first-child": 0 },
borderTopStyle: "solid",
borderTopColor: colors.border,
},
footer: {
marginTop: "0.75rem",
display: "flex",
alignItems: "center",
gap: "0.75rem",
},
note: {
fontSize: "0.875rem",
lineHeight: "1.25rem",
color: colors.textMuted,
},
refetching: {
marginTop: "0.75rem",
},
moreButton: {
fontWeight: 500,
},
moreError: {
marginTop: "0.5rem",
fontSize: "0.875rem",
lineHeight: "1.25rem",
color: colors.dangerText,
},
});
function errorMessage(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
export default function HistoryActivity({ search }: { search: ActivitySearch }) {
const filter = queriesFilterOf(search);
const base = useInfiniteQuery(queriesInfiniteQuery(filter));
const clientNames = useClientNames();
const pages = base.data?.pages ?? [];
const rows: QueryRow[] = pages.flatMap((page) => page.queries);
const coverage = pages[0]?.coverage;
const filterActive = Object.keys(filter).length > 0;
// `base.hasNextPage` reads the query state, which is empty while placeholder
// data stands in for a filter change; derive the cursor from what is on
// screen so the button keeps its place instead of flashing "end of log".
const lastPage = pages[pages.length - 1];
const hasMore = lastPage !== undefined && lastPage.next_before !== null;
// A 401 is already redirecting via the cache-level handleUnauthorized.
const isUnauthorized = base.error instanceof api.ApiError && base.error.status === 401;
const moreError = base.isFetchNextPageError && !isUnauthorized ? errorMessage(base.error) : null;
function loadMore() {
if (!hasMore || base.isFetchingNextPage || base.isPlaceholderData) return;
void base.fetchNextPage();
}
// The loader starts this fetch but does not wait for it, so both the first
// paint and a failed first page are this component's to render.
if (base.status === "error" && base.data === undefined) {
return <InlineError error={base.error} onRetry={() => void base.refetch()} />;
}
if (base.data === undefined) {
return (
<p {...stylex.props(styles.empty, shared.pulse)} role="status">
Loading activity
</p>
);
}
return (
<>
{base.isFetching && (
<p {...stylex.props(styles.note, styles.refetching)} role="status">
Loading
</p>
)}
{coverage !== undefined && <CoverageNotice coverage={coverage} />}
{rows.length === 0 ? (
<p {...stylex.props(styles.empty)}>
{filterActive ? "No queries match the current filters." : "No queries logged yet."}
</p>
) : (
<>
<div {...stylex.props(styles.tableWrap)}>
<table {...stylex.props(styles.table)}>
<ActivityTableHead />
<tbody>
{rows.map((row) => (
<tr key={row.id} {...stylex.props(styles.row)}>
<ActivityCells
row={summarizeRow(row)}
clientNames={clientNames}
renderDomain={(id, children) =>
id === null ? (
children
) : (
<Link
to="/activity/queries/$id"
params={{ id: String(id) }}
search={search}
{...stylex.props(activityDomainLink, shared.focusRing)}
>
{children}
</Link>
)
}
/>
</tr>
))}
</tbody>
</table>
</div>
<div {...stylex.props(styles.footer)}>
<p {...stylex.props(styles.note)}>
Showing {rows.length} {rows.length === 1 ? "query" : "queries"}
{hasMore ? "" : " — end of log"}
</p>
{hasMore && (
<button
type="button"
onClick={loadMore}
disabled={base.isFetchingNextPage || base.isPlaceholderData}
{...stylex.props(shared.button, styles.moreButton, shared.focusRing)}
>
{base.isFetchingNextPage ? "Loading…" : "Load more"}
</button>
)}
</div>
{moreError !== null && (
<p role="alert" {...stylex.props(styles.moreError)}>
Failed to load more: {moreError}
</p>
)}
</>
)}
</>
);
}
@@ -0,0 +1,399 @@
/**
* Activity in live mode, through the real router.
*
* The EventSource is a global here rather than an injected factory: whether the
* connection exists at all is the thing under test, and that is decided by
* which subtree the URL mounts, not by a prop a caller could pass.
*/
import { act, fireEvent, render, screen, within } from "@testing-library/react";
import { QueryClientProvider } from "@tanstack/react-query";
import { RouterProvider, createMemoryHistory } from "@tanstack/react-router";
import { AuthProvider } from "@/auth/store";
import { createQueryClient } from "@/lib/queryClient";
import { createAppRouter } from "@/routes";
import type { Client } from "@/lib/types";
import { provenance, queryRow } from "@/features/queries/provenanceFixture";
import { FakeEventSource } from "./fakeEventSource";
function client(ip: string, name: string, learnedName: string): Client {
return {
id: Number(ip.split(".").pop()),
ip,
name,
learned_name: learnedName,
group_id: 1,
group: "default",
hand_edited: name !== "",
first_seen: 1_700_000_000,
last_seen: 1_700_000_100,
};
}
const CLIENTS: Client[] = [
client("192.0.2.10", "Kitchen Pi", "pi.lan"),
client("192.0.2.11", "", "laptop.lan"),
client("192.0.2.12", "", ""),
];
const VERSION = { version: "0.0.0-test", git_commit: "0000000", zig_version: "0.16.0", uptime_seconds: 1 };
let sources: FakeEventSource[];
let fetchMock: ReturnType<typeof vi.fn>;
function json(payload: unknown): Response {
return new Response(JSON.stringify(payload), { status: 200, headers: { "content-type": "application/json" } });
}
function stubFetch(handler: (url: string) => Response | Promise<Response> = () => json({})) {
fetchMock = vi.fn((input: RequestInfo | URL) => {
const url = String(input);
if (url === "/api/version") return Promise.resolve(json(VERSION));
if (url === "/api/clients") return Promise.resolve(json({ clients: CLIENTS }));
return Promise.resolve(handler(url));
});
vi.stubGlobal("fetch", fetchMock);
}
beforeEach(() => {
sources = [];
vi.stubGlobal(
"EventSource",
class {
constructor(url: string) {
const source = new FakeEventSource(url);
sources.push(source);
return source as unknown as EventSource;
}
},
);
stubFetch();
});
afterEach(() => {
vi.unstubAllGlobals();
});
function frame(ts: number, domain: string, sections: Parameters<typeof provenance>[0] = {}): { data: string } {
return {
data: JSON.stringify(
provenance({
...sections,
request: { time: ts, domain, ...sections.request },
route: { kind: "cache", upstream: "", ...sections.route },
}),
),
};
}
function renderPage(path = "/activity?mode=live") {
const queryClient = createQueryClient();
const history = createMemoryHistory({ initialEntries: [path] });
const router = createAppRouter(history, queryClient);
render(
<AuthProvider>
<QueryClientProvider client={queryClient}>
<RouterProvider router={router} />
</QueryClientProvider>
</AuthProvider>,
);
return { history };
}
async function openLive(path?: string) {
const rendered = renderPage(path);
await screen.findByRole("button", { name: "Freeze" });
act(() => sources[0]!.emit("open"));
return rendered;
}
function queryCalls(): string[] {
return fetchMock.mock.calls
.map((call) => String(call[0]))
.filter((url) => url === "/api/queries" || url.startsWith("/api/queries?"));
}
test("streams rows, flags blocked ones, and freezes the display", async () => {
await openLive();
expect(screen.getByRole("status", { name: "Live" })).toBeTruthy();
expect(screen.getByText("Waiting for queries…")).toBeTruthy();
act(() => {
sources[0]!.emit("query", frame(1000, "ok.example"));
sources[0]!.emit(
"query",
frame(1001, "ads.example", {
request: { qtype: 28 },
policy: { action: "block", reason: "blocklist_wildcard" },
route: { kind: "blocked" },
}),
);
});
expect(screen.getByText("ok.example")).toBeTruthy();
expect(screen.getByText("AAAA")).toBeTruthy();
const blockedRow = screen.getByText("ads.example").closest("tr")!;
expect(within(blockedRow).getAllByText("Blocked")).toHaveLength(2);
// StyleX compiles to opaque class names, so the check is structural: a blocked
// row carries every class a plain row does, plus the ones the flag adds.
const plainRow = screen.getByText("ok.example").closest("tr")!;
const blockedClasses = new Set(blockedRow.className.split(" "));
const plainClasses = plainRow.className.split(" ");
expect(plainClasses.every((name) => blockedClasses.has(name))).toBe(true);
expect(blockedClasses.size).toBeGreaterThan(plainClasses.length);
const freeze = screen.getByRole("button", { name: "Freeze" });
fireEvent.click(freeze);
expect(freeze.getAttribute("aria-pressed")).toBe("true");
act(() => sources[0]!.emit("query", frame(1002, "later.example")));
expect(screen.queryByText("later.example")).toBeNull();
expect(screen.getByText(/3 in buffer/)).toBeTruthy();
fireEvent.click(screen.getByRole("button", { name: "Resume" }));
expect(screen.getByText("later.example")).toBeTruthy();
});
test("resolves each row's client to its display name, keeping the IP as the tooltip", async () => {
await openLive();
act(() => {
sources[0]!.emit("query", frame(1000, "named.example", { request: { client: "192.0.2.10" } }));
sources[0]!.emit("query", frame(1001, "learned.example", { request: { client: "192.0.2.11" } }));
sources[0]!.emit("query", frame(1002, "nameless.example", { request: { client: "192.0.2.12" } }));
sources[0]!.emit("query", frame(1003, "stranger.example", { request: { client: "192.0.2.99" } }));
});
const named = await screen.findByText("Kitchen Pi");
expect(named.getAttribute("title")).toBe("192.0.2.10");
expect(screen.queryByText("pi.lan")).toBeNull();
const learned = screen.getByText("laptop.lan");
expect(learned.getAttribute("title")).toBe("192.0.2.11");
expect(within(learned.closest("tr")!).queryByText("learned")).toBeNull();
expect(screen.getByText("192.0.2.12").getAttribute("title")).toBeNull();
expect(screen.getByText("192.0.2.99").getAttribute("title")).toBeNull();
});
test("rows stream in as bare IPs while the client list is still loading", async () => {
let releaseClients: () => void = () => {};
fetchMock = vi.fn((input: RequestInfo | URL) => {
const url = String(input);
if (url === "/api/version") return Promise.resolve(json(VERSION));
return new Promise<Response>((resolve) => {
if (url !== "/api/clients") {
resolve(json({}));
return;
}
releaseClients = () => resolve(json({ clients: CLIENTS }));
});
});
vi.stubGlobal("fetch", fetchMock);
await openLive();
act(() => sources[0]!.emit("query", frame(1000, "named.example", { request: { client: "192.0.2.10" } })));
expect(screen.getByText("192.0.2.10")).toBeTruthy();
expect(screen.queryByText("Kitchen Pi")).toBeNull();
releaseClients();
expect(await screen.findByText("Kitchen Pi")).toBeTruthy();
});
test("repeated connection failures show the viewer-cap state with a retry button", async () => {
renderPage();
await screen.findByRole("button", { name: "Freeze" });
act(() => {
sources[0]!.emit("error");
sources[0]!.emit("error");
sources[0]!.emit("error");
});
expect(screen.getByRole("alert").textContent).toContain("too many live viewers");
fireEvent.click(screen.getByRole("button", { name: "Retry" }));
expect(sources).toHaveLength(2);
expect(screen.getByText("Connecting…")).toBeTruthy();
});
test("a recovered row links to its stored detail; a streamed one opens in place instead", async () => {
stubFetch((url) => {
if (url.startsWith("/api/queries?")) {
return json({
queries: [queryRow(88, { ts: 1001, domain: "recovered.example" })],
next_before: null,
coverage: { complete: true, available_since: 0 },
});
}
return json({});
});
await openLive();
act(() => sources[0]!.emit("query", frame(1000, "streamed.example")));
act(() => sources[0]!.emit("error"));
act(() => sources[0]!.emit("open"));
const recovered = await screen.findByRole("link", { name: "recovered.example" });
expect(recovered.getAttribute("href")).toContain("/activity/queries/88");
// The streamed frame precedes its own insert, so it has no row to link to —
// but it does carry its own provenance, so it still has a detail.
expect(screen.queryByRole("link", { name: "streamed.example" })).toBeNull();
expect(screen.getByRole("button", { name: "streamed.example" })).toBeTruthy();
});
test("a streamed row opens its own provenance, from the keyboard as well as the pointer", async () => {
await openLive();
act(() =>
sources[0]!.emit(
"query",
frame(1000, "streamed.example", {
policy: { action: "block", reason: "blocklist_domain", matched: "streamed.example" },
route: { kind: "blocked", upstream: "" },
}),
),
);
const trigger = screen.getByRole("button", { name: "streamed.example" });
// A real <button> is in the tab order and activates on Enter and Space; the
// only way to lose that is to opt out of it, which nothing here may do.
expect(trigger.tagName).toBe("BUTTON");
expect(trigger.getAttribute("tabindex")).toBeNull();
act(() => trigger.focus());
expect(document.activeElement).toBe(trigger);
fireEvent.click(trigger);
const heading = screen.getByRole("heading", { level: 1, name: "streamed.example" });
expect(heading).toBeTruthy();
const panel = heading.closest("div")!.parentElement!;
expect(within(panel).getByText("Blocked locally")).toBeTruthy();
expect(panel.textContent).toContain("the query log may not have written it yet");
fireEvent.click(screen.getByRole("button", { name: "Close" }));
expect(screen.queryByRole("heading", { level: 1, name: "streamed.example" })).toBeNull();
});
test("the detail takes focus when a row opens it and hands it back when it closes", async () => {
await openLive();
act(() => sources[0]!.emit("query", frame(1000, "streamed.example")));
const trigger = screen.getByRole("button", { name: "streamed.example" });
expect(trigger.getAttribute("aria-expanded")).toBe("false");
expect(trigger.getAttribute("aria-controls")).toBeNull();
// A native button activates on Enter and Space; jsdom does not synthesize
// the click those keys fire, so the click is the activation.
act(() => trigger.focus());
fireEvent.click(trigger);
const panel = screen.getByRole("group", { name: "Streamed query" });
expect(trigger.getAttribute("aria-expanded")).toBe("true");
expect(trigger.getAttribute("aria-controls")).toBe(panel.id);
// The panel is inserted above the table, behind the trigger in tab order, so
// the only thing that keeps a forward tab inside it is focus moving in.
expect(panel.compareDocumentPosition(trigger) & Node.DOCUMENT_POSITION_FOLLOWING).toBeTruthy();
expect(document.activeElement).toBe(panel);
expect(panel.contains(screen.getByRole("button", { name: "Close" }))).toBe(true);
fireEvent.click(screen.getByRole("button", { name: "Close" }));
expect(screen.queryByRole("group", { name: "Streamed query" })).toBeNull();
expect(document.activeElement).toBe(trigger);
expect(trigger.getAttribute("aria-expanded")).toBe("false");
expect(trigger.getAttribute("aria-controls")).toBeNull();
});
test("opening a second row moves the expanded state and the focus with it", async () => {
await openLive();
act(() => {
sources[0]!.emit("query", frame(1000, "first.example"));
sources[0]!.emit("query", frame(1001, "second.example"));
});
const first = screen.getByRole("button", { name: "first.example" });
const second = screen.getByRole("button", { name: "second.example" });
fireEvent.click(first);
fireEvent.click(second);
const panel = screen.getByRole("group", { name: "Streamed query" });
expect(within(panel).getByRole("heading", { level: 1, name: "second.example" })).toBeTruthy();
expect(document.activeElement).toBe(panel);
expect(first.getAttribute("aria-expanded")).toBe("false");
expect(second.getAttribute("aria-expanded")).toBe("true");
fireEvent.click(screen.getByRole("button", { name: "Close" }));
expect(document.activeElement).toBe(second);
});
test("an open streamed detail survives the row being evicted from the ring buffer", async () => {
await openLive();
act(() => sources[0]!.emit("query", frame(1000, "evicted.example")));
fireEvent.click(screen.getByRole("button", { name: "evicted.example" }));
expect(screen.getByRole("heading", { level: 1, name: "evicted.example" })).toBeTruthy();
// 500 more queries: the ring keeps the newest 500, so the selected row is
// gone from the table. The detail is a snapshot, not a lookup into the ring.
act(() => {
for (let index = 0; index < 500; index += 1) {
sources[0]!.emit("query", frame(2000 + index, `filler${index}.example`));
}
});
expect(screen.queryByRole("button", { name: "evicted.example" })).toBeNull();
expect(screen.getByRole("heading", { level: 1, name: "evicted.example" })).toBeTruthy();
});
test("an open streamed detail survives Freeze and Resume", async () => {
await openLive();
act(() => sources[0]!.emit("query", frame(1000, "held.example")));
fireEvent.click(screen.getByRole("button", { name: "held.example" }));
fireEvent.click(screen.getByRole("button", { name: "Freeze" }));
expect(screen.getByRole("heading", { level: 1, name: "held.example" })).toBeTruthy();
fireEvent.click(screen.getByRole("button", { name: "Resume" }));
expect(screen.getByRole("heading", { level: 1, name: "held.example" })).toBeTruthy();
});
test("the filter row stays visible, keeps its values, and is out of the tab order", async () => {
await openLive("/activity?mode=live&domain=ads&client=192.0.2.10&blocked=true");
const domain = screen.getByLabelText("Domain contains") as HTMLInputElement;
expect(domain.value).toBe("ads");
expect(domain.disabled).toBe(true);
expect((screen.getByLabelText("Client (exact)") as HTMLInputElement).disabled).toBe(true);
expect((screen.getByLabelText("Since") as HTMLInputElement).disabled).toBe(true);
expect((screen.getByLabelText("Until") as HTMLInputElement).disabled).toBe(true);
const form = domain.closest("form")!;
const controls = [...form.querySelectorAll("input, button, select, textarea, a[href], [tabindex]")];
expect(controls.length).toBeGreaterThan(0);
for (const control of controls) {
// A disabled form control is skipped by the browser's tab order, and RAC
// pins its own trigger out of it as well. Nothing in the row may
// reintroduce itself with a reachable tabindex.
expect(control.hasAttribute("disabled")).toBe(true);
const tabindex = control.getAttribute("tabindex");
expect(tabindex === null || tabindex === "-1").toBe(true);
}
});
test("live mode asks for no query pages, whatever filters the url retained", async () => {
await openLive("/activity?mode=live&domain=ads&client=192.0.2.10&blocked=true&since=1700000000&bogus=1");
act(() => sources[0]!.emit("query", frame(1000, "streamed.example")));
expect(queryCalls()).toEqual([]);
});
test("leaving live closes the stream, and coming back opens exactly one fresh one", async () => {
await openLive("/activity?mode=live&domain=ads");
fireEvent.click(screen.getByRole("button", { name: "History" }));
await screen.findByRole("button", { name: "Apply filters" });
expect(sources).toHaveLength(1);
expect(sources[0]!.closed).toBe(true);
// The filters came along, which is the point of switching rather than
// navigating: the reader keeps the question they were asking.
expect((screen.getByLabelText("Domain contains") as HTMLInputElement).value).toBe("ads");
expect((screen.getByLabelText("Domain contains") as HTMLInputElement).disabled).toBe(false);
fireEvent.click(screen.getByRole("button", { name: "Live" }));
await screen.findByRole("button", { name: "Freeze" });
expect(sources).toHaveLength(2);
expect(sources[1]!.closed).toBe(false);
});
@@ -1,25 +1,41 @@
/**
* Activity in live mode: the SSE stream, its bounded ring buffer, and the
* in-place detail a streamed row opens.
*
* This subtree is mounted only while the URL says `mode=live`, which is what
* closes the EventSource on the way back to history: the connection is a
* server-side resource capped per address, so a page that kept it open while
* showing something else would spend a viewer slot on nothing.
*
* Freeze and Follow are display state and stay out of the URL. They describe
* what the screen is doing right now, not what it is showing, so a shared link
* would carry a frozen moment the recipient never saw fill.
*/
import { useEffect, useRef, useState } from "react";
import { Link } from "@tanstack/react-router";
import * as stylex from "@stylexjs/stylex";
import { useClientNames } from "@/features/clients/clientNames";
import { QueryCells, QueryTableHead } from "@/features/queries/QueryLogPage";
import { RING_CAPACITY } from "./ringBuffer";
import { useLiveQueries, type EventSourceFactory, type StreamStatus } from "./useLiveQueries";
import { summarizeEvent } from "@/features/queries/querySummary";
import { styles as shared } from "@/ui/styles";
import { colors } from "@/ui/tokens.stylex";
import { ActivityCells, ActivityTableHead, activityDomainLink } from "./cells";
import ProvenanceDetail from "./ProvenanceDetail";
import RelatedActions from "./RelatedActions";
import { RING_CAPACITY, summaryOf, type StreamedRow } from "./ringBuffer";
import type { ActivitySearch } from "./search";
import { useLiveQueries, type StreamStatus } from "./useLiveQueries";
const DARK = "@media (prefers-color-scheme: dark)";
const styles = stylex.create({
toolbar: {
marginTop: "1rem",
display: "flex",
flexWrap: "wrap",
alignItems: "center",
gap: "0.75rem",
},
heading: {
fontSize: "1.5rem",
lineHeight: "2rem",
fontWeight: 600,
},
toolbarButton: {
fontWeight: 500,
},
@@ -134,6 +150,40 @@ const styles = stylex.create({
[DARK]: "oklch(25.8% 0.092 26.042 / 0.4)",
},
},
/** A streamed row opens its detail here rather than at a route, so it is a button that looks like the link beside it. */
domainButton: {
borderStyle: "none",
backgroundColor: "transparent",
padding: 0,
font: "inherit",
textAlign: "left",
cursor: "pointer",
textDecorationLine: "underline",
textDecorationStyle: "dotted",
},
detailPanel: {
marginTop: "1rem",
borderRadius: "0.25rem",
borderWidth: 1,
borderStyle: "solid",
borderColor: colors.borderStrong,
backgroundColor: colors.surface,
padding: "1rem",
},
detailBar: {
display: "flex",
alignItems: "baseline",
justifyContent: "space-between",
gap: "0.75rem",
},
detailLabel: {
fontSize: "0.75rem",
lineHeight: "1rem",
fontWeight: 600,
letterSpacing: "0.05em",
textTransform: "uppercase",
color: colors.textMuted,
},
footnote: {
marginTop: "0.75rem",
fontSize: "0.875rem",
@@ -142,6 +192,10 @@ const styles = stylex.create({
},
});
/** One panel at a time, so the trigger that opened it can name it in `aria-controls`. */
const DETAIL_PANEL_ID = "live-query-detail";
const DETAIL_LABEL_ID = "live-query-detail-label";
const PILL_LABELS: Record<StreamStatus, string> = {
connecting: "Connecting…",
open: "Live",
@@ -165,16 +219,85 @@ function StatusPill({ status }: { status: StreamStatus }) {
);
}
/** Freeze is display-only: the stream stays open and the 500-row ring buffer keeps
* filling; Resume shows the current buffer (anything pushed out meanwhile is gone). */
export default function LiveLogPage({ createEventSource }: { createEventSource?: EventSourceFactory } = {}) {
const live = useLiveQueries({ createEventSource });
const clientNames = useClientNames();
/**
* The open detail, held as the selected row itself rather than as a key into
* the buffer.
*
* The buffer is a 500-row ring that a gap merge also rewrites: a reference by
* key would go stale under the reader while they were still reading it, and the
* panel would blank out for no reason they could see. The snapshot is the whole
* fact a streamed frame carries its own provenance so it survives eviction,
* a merge and a Freeze/Resume, and closes only when the reader closes it or
* leaves live mode.
*/
function LiveDetail({ row, origin, onClose }: { row: StreamedRow; origin: ActivitySearch; onClose: () => void }) {
const summary = summarizeEvent(row.event);
const panel = useRef<HTMLDivElement>(null);
// The panel opens above the table, behind the trigger in tab order, so a
// forward tab from the row would walk past it. Focus moves in on open —
// keyed on the row, so choosing a second row moves it again — and the
// closer puts it back on the trigger.
useEffect(() => {
panel.current?.focus();
}, [row.key]);
return (
<section>
<div
ref={panel}
id={DETAIL_PANEL_ID}
tabIndex={-1}
role="group"
aria-labelledby={DETAIL_LABEL_ID}
{...stylex.props(styles.detailPanel)}
>
<div {...stylex.props(styles.detailBar)}>
<span id={DETAIL_LABEL_ID} {...stylex.props(styles.detailLabel)}>
Streamed query
</span>
<button type="button" onClick={onClose} {...stylex.props(shared.button, shared.focusRing)}>
Close
</button>
</div>
<ProvenanceDetail
provenance={row.event}
persistedId={null}
relatedActions={
<RelatedActions
domain={summary.domain}
client={summary.client_ip}
ts={summary.ts}
origin={origin}
/>
}
/>
</div>
);
}
export default function LiveActivity({ origin }: { origin: ActivitySearch }) {
const live = useLiveQueries();
const clientNames = useClientNames();
const [selected, setSelected] = useState<StreamedRow | null>(null);
const trigger = useRef<HTMLButtonElement | null>(null);
function open(row: StreamedRow, from: HTMLButtonElement) {
trigger.current = from;
setSelected(row);
}
// The row that opened the panel takes focus back, unless the ring has
// already evicted it: a detached button cannot be focused, and the browser
// falls back to the document, which is the best available answer.
function close() {
setSelected(null);
trigger.current?.focus();
trigger.current = null;
}
return (
<>
<div {...stylex.props(styles.toolbar)}>
<h1 {...stylex.props(styles.heading)}>Live</h1>
<StatusPill status={live.status} />
<button
type="button"
@@ -228,6 +351,8 @@ export default function LiveLogPage({ createEventSource }: { createEventSource?:
</div>
)}
{selected !== null && <LiveDetail row={selected} origin={origin} onClose={close} />}
{live.rows.length === 0 ? (
live.status !== "capped" && (
<p {...stylex.props(styles.empty)}>
@@ -238,13 +363,50 @@ export default function LiveLogPage({ createEventSource }: { createEventSource?:
<>
<div {...stylex.props(styles.tableWrap)}>
<table {...stylex.props(styles.table)}>
<QueryTableHead />
<ActivityTableHead />
<tbody>
{live.rows.map((row) => (
<tr key={row.key} {...stylex.props(styles.row, row.blocked && styles.rowBlocked)}>
<QueryCells row={row} clientNames={clientNames} />
</tr>
))}
{live.rows.map((row) => {
const summary = summaryOf(row);
return (
<tr
key={row.key}
{...stylex.props(styles.row, summary.blocked && styles.rowBlocked)}
>
<ActivityCells
row={summary}
clientNames={clientNames}
renderDomain={(_id, children) =>
row.kind === "streamed" ? (
<button
type="button"
aria-expanded={selected?.key === row.key}
aria-controls={
selected?.key === row.key ? DETAIL_PANEL_ID : undefined
}
onClick={(event) => open(row, event.currentTarget)}
{...stylex.props(
styles.domainButton,
activityDomainLink,
shared.focusRing,
)}
>
{children}
</button>
) : (
<Link
to="/activity/queries/$id"
params={{ id: String(row.row.id) }}
search={origin}
{...stylex.props(activityDomainLink, shared.focusRing)}
>
{children}
</Link>
)
}
/>
</tr>
);
})}
</tbody>
</table>
</div>
@@ -254,6 +416,6 @@ export default function LiveLogPage({ createEventSource }: { createEventSource?:
</p>
</>
)}
</section>
</>
);
}
@@ -0,0 +1,178 @@
import { fireEvent, render, screen } from "@testing-library/react";
import { QueryClientProvider } from "@tanstack/react-query";
import { RouterProvider, createMemoryHistory } from "@tanstack/react-router";
import { AuthProvider } from "@/auth/store";
import { createQueryClient } from "@/lib/queryClient";
import { createAppRouter } from "@/routes";
import type { LookupResult } from "@/lib/types";
const BLOCKED: LookupResult = {
domain: "ads.example",
group_id: 1,
local_records: false,
forward_zone: null,
blocked: true,
reason: "blocklist_domain",
matched: "ads.example",
source_url: "https://lists.test/a",
safe_search_rewrite: null,
};
let fetchMock: ReturnType<typeof createFetchMock>;
/** What `/api/lookup` answers, so a test can make it fail without rebuilding the mock. */
let lookup: (url: string) => Response;
function json(payload: unknown, status = 200, headers: Record<string, string> = {}): Response {
return new Response(JSON.stringify(payload), {
status,
headers: { "content-type": "application/json", ...headers },
});
}
function createFetchMock() {
return vi.fn(async (input: RequestInfo | URL) => {
const url = String(input);
if (url === "/api/groups") {
return json({
groups: [
{ id: 1, name: "default", safe_search: false },
{ id: 2, name: "kids", safe_search: true },
],
});
}
if (url.startsWith("/api/lookup")) return lookup(url);
return json({ error: "not stubbed" }, 404);
});
}
beforeEach(() => {
lookup = (url) =>
url === "/api/lookup?domain=ads.example&group_id=1" ? json(BLOCKED) : json({ error: "not stubbed" }, 404);
fetchMock = createFetchMock();
vi.stubGlobal("fetch", fetchMock);
});
afterEach(() => {
vi.unstubAllGlobals();
});
/**
* `retry: false` for the failure tests: the shared client retries a 5xx twice
* and a 429 after its Retry-After, so the surfaced error is what the page does
* once the client has given up, not something a test should sit out in real
* time.
*/
function renderPage(path = "/activity/test", { retry = true } = {}) {
const queryClient = createQueryClient();
if (!retry) {
const defaults = queryClient.getDefaultOptions();
queryClient.setDefaultOptions({ ...defaults, queries: { ...defaults.queries, retry: false } });
}
const router = createAppRouter(createMemoryHistory({ initialEntries: [path] }), queryClient);
render(
<AuthProvider>
<QueryClientProvider client={queryClient}>
<RouterProvider router={router} />
</QueryClientProvider>
</AuthProvider>,
);
}
function lookupCalls(): string[] {
return fetchMock.mock.calls.map(([input]) => String(input)).filter((url) => url.startsWith("/api/lookup"));
}
test("fetches nothing until submit, then renders the blocked verdict", async () => {
renderPage();
await screen.findByRole("heading", { name: "Current policy simulation" });
await screen.findByLabelText("Group");
expect(lookupCalls()).toEqual([]);
fireEvent.change(screen.getByLabelText("Domain"), { target: { value: "ads.example" } });
expect(lookupCalls()).toEqual([]);
fireEvent.click(screen.getByRole("button", { name: "Simulate" }));
await screen.findByRole("heading", { name: "Blocked" });
expect(lookupCalls()).toEqual(["/api/lookup?domain=ads.example&group_id=1"]);
expect(screen.getByText("blocklist_domain")).toBeTruthy();
const link = screen.getByRole("link", { name: "https://lists.test/a" }) as HTMLAnchorElement;
expect(link.href).toBe("https://lists.test/a");
expect(screen.getByText("Queries for this name get a blocked response.")).toBeTruthy();
});
test("a ?domain= link asks the question on arrival instead of leaving a filled-in form", async () => {
renderPage("/activity/test?domain=ads.example");
// No submit here: the link is the question, so the verdict is what arrives.
await screen.findByRole("heading", { name: "Blocked" });
expect(lookupCalls()).toEqual(["/api/lookup?domain=ads.example&group_id=1"]);
expect(screen.getByLabelText("Domain")).toHaveProperty("value", "ads.example");
});
test("no filter snapshot reads as a server that is starting, not as a verdict", async () => {
lookup = () => json({ error: "no snapshot" }, 503);
renderPage("/activity/test", { retry: false });
fireEvent.change(await screen.findByLabelText("Domain"), { target: { value: "ads.example" } });
fireEvent.click(screen.getByRole("button", { name: "Simulate" }));
const alert = await screen.findByRole("alert");
expect(alert.textContent).toContain("No filter snapshot is loaded yet");
expect(lookupCalls()).toEqual(["/api/lookup?domain=ads.example&group_id=1"]);
// Nothing may read as an answer while the lookup has none.
expect(screen.queryByRole("heading", { name: "Blocked" })).toBeNull();
expect(screen.queryByText("Simulating…")).toBeNull();
});
test("a rate limit says how long to wait, from the server's own Retry-After", async () => {
lookup = () => json({ error: "rate limited" }, 429, { "retry-after": "12" });
renderPage("/activity/test", { retry: false });
fireEvent.change(await screen.findByLabelText("Domain"), { target: { value: "ads.example" } });
fireEvent.click(screen.getByRole("button", { name: "Simulate" }));
const alert = await screen.findByRole("alert");
expect(alert.textContent).toBe("Rate limited. Try again in 12s.");
});
test("resubmitting the same domain and group refetches rather than showing a stale verdict", async () => {
renderPage();
fireEvent.change(await screen.findByLabelText("Domain"), { target: { value: "ads.example" } });
fireEvent.click(screen.getByRole("button", { name: "Simulate" }));
await screen.findByRole("heading", { name: "Blocked" });
expect(lookupCalls()).toHaveLength(1);
// The policy can change between two identical questions, so the second one
// has to reach the server even though the query key has not moved.
lookup = () => json({ ...BLOCKED, blocked: false, reason: "no_match", matched: "", source_url: null });
fireEvent.click(screen.getByRole("button", { name: "Simulate" }));
await screen.findByRole("heading", { name: "Allowed" });
expect(lookupCalls()).toEqual([
"/api/lookup?domain=ads.example&group_id=1",
"/api/lookup?domain=ads.example&group_id=1",
]);
});
test("defaults the group select to the default group (id 1)", async () => {
renderPage();
// A RAC Select names its trigger with the current value and then the label, so
// the selected group's name is the only thing the trigger shows.
const trigger = await screen.findByRole("button", { name: /Group$/ });
expect(trigger.textContent).toContain("default");
});
test("the framing is forward-tense, so it cannot be read as an account of a past query", async () => {
renderPage();
await screen.findByRole("heading", { name: "Current policy simulation" });
const intro = screen.getByRole("heading", { name: "Current policy simulation" }).nextElementSibling;
expect(intro?.textContent).toContain("would");
expect(intro?.textContent).toContain("right now");
// Nothing on the page may claim to explain a query that already happened.
expect(document.body.textContent).not.toContain("Look up");
});
@@ -1,5 +1,6 @@
import { useState, type FormEvent, type ReactNode } from "react";
import { useQuery, useSuspenseQuery } from "@tanstack/react-query";
import { useSearch } from "@tanstack/react-router";
import * as stylex from "@stylexjs/stylex";
import { ApiError } from "@/lib/api";
import { groupsQuery, lookupQuery } from "@/lib/queries";
@@ -249,13 +250,18 @@ function VerdictCard({ result, groups }: { result: LookupResult; groups: Group[]
);
}
export default function LookupPage() {
export default function PolicyTestPage() {
const groups = useSuspenseQuery(groupsQuery()).data;
const preselectedGroupId = defaultGroupId(groups);
const [domain, setDomain] = useState("");
// A `?domain=` link (from a query's detail page) arrives already asking the
// question, so it runs the simulation rather than leaving a filled-in form.
const search = useSearch({ from: "/shell/activity/test" });
const [domain, setDomain] = useState(search.domain ?? "");
const [groupId, setGroupId] = useState(preselectedGroupId);
const [submitted, setSubmitted] = useState<Submitted | null>(null);
const [submitted, setSubmitted] = useState<Submitted | null>(
search.domain === undefined ? null : { domain: search.domain, groupId: preselectedGroupId },
);
const lookup = useQuery({
...lookupQuery(submitted?.domain ?? "", submitted?.groupId),
@@ -275,17 +281,19 @@ export default function LookupPage() {
return (
<section>
<h1 {...stylex.props(styles.heading)}>Lookup</h1>
<h1 {...stylex.props(styles.heading)}>Current policy simulation</h1>
<p {...stylex.props(styles.intro)}>
What the pipeline would do with a domain: local records, forward zones, block decision, safe search.
What the pipeline <em>would</em> do with a domain right now: local records, forward zones, block
decision, safe search. This reads the configuration in force at this moment, so it explains nothing
about a query already answered a detail page does that.
</p>
<form onSubmit={onSubmit} {...stylex.props(styles.form)}>
<div {...stylex.props(styles.domainField)}>
<label htmlFor="lookup-domain" {...stylex.props(styles.fieldLabel)}>
<label htmlFor="policy-test-domain" {...stylex.props(styles.fieldLabel)}>
Domain
</label>
<input
id="lookup-domain"
id="policy-test-domain"
required
value={domain}
onChange={(event) => setDomain(event.target.value)}
@@ -306,10 +314,10 @@ export default function LookupPage() {
disabled={lookup.isFetching}
{...stylex.props(shared.largePrimaryButton, shared.focusRing)}
>
Look up
Simulate
</button>
</form>
{lookup.isFetching && <p {...stylex.props(styles.note)}>Looking up</p>}
{lookup.isFetching && <p {...stylex.props(styles.note)}>Simulating</p>}
{!lookup.isFetching && lookup.isError && (
<p role="alert" {...stylex.props(styles.error)}>
{errorMessage(lookup.error)}
@@ -0,0 +1,300 @@
/**
* One query, explained in the order it met the pipeline.
*
* This is the body of the detail surface, with no route in it, because two
* surfaces show it: the persisted detail page, which fetched the row by id, and
* a live row, whose provenance arrived in the stream frame and which SQLite may
* not have written yet. The related actions are links into routes, so the
* caller passes them in already built.
*/
import type { ReactNode } from "react";
import * as stylex from "@stylexjs/stylex";
import { formatMicros, formatTime } from "@/lib/format";
import type { PolicyReason, Provenance } from "@/lib/types";
import { clientLabel, useClientNames } from "@/features/clients/clientNames";
import {
policyActionLabel,
policyReasonLabel,
qclassName,
rcodeName,
routeKindLabel,
} from "@/features/queries/provenanceCopy";
import { qtypeName } from "@/features/queries/qtype";
import { styles as shared } from "@/ui/styles";
import { colors } from "@/ui/tokens.stylex";
const styles = stylex.create({
heading: {
marginTop: "0.5rem",
fontSize: "1.5rem",
lineHeight: "2rem",
fontWeight: 600,
wordBreak: "break-all",
},
subtitle: {
marginTop: "0.25rem",
color: colors.textSecondary,
},
/** The recorded facts, fenced off from the live links below them. */
record: {
marginTop: "1rem",
maxWidth: "48rem",
borderRadius: "0.25rem",
borderWidth: 1,
borderStyle: "solid",
borderColor: colors.border,
backgroundColor: colors.surfaceRaised,
padding: "1rem",
},
recordNote: {
fontSize: "0.8125rem",
lineHeight: "1.25rem",
color: colors.textMuted,
},
section: {
marginTop: "1rem",
borderTopWidth: { default: 1, ":first-of-type": 0 },
borderTopStyle: "solid",
borderTopColor: colors.border,
paddingTop: { default: "1rem", ":first-of-type": 0 },
},
sectionHeading: {
fontSize: "0.75rem",
lineHeight: "1rem",
fontWeight: 600,
letterSpacing: "0.05em",
textTransform: "uppercase",
color: colors.textMuted,
},
facts: {
marginTop: "0.5rem",
marginBottom: 0,
display: "grid",
gap: "0.375rem 1rem",
gridTemplateColumns: {
default: "auto",
"@media (min-width: 640px)": "max-content 1fr",
},
fontSize: "0.875rem",
lineHeight: "1.25rem",
},
term: {
color: colors.textMuted,
},
value: {
margin: 0,
wordBreak: "break-all",
},
muted: {
color: colors.textMuted,
},
related: {
marginTop: "1.5rem",
maxWidth: "48rem",
},
relatedHeading: {
fontSize: "1.125rem",
lineHeight: "1.75rem",
fontWeight: 600,
},
relatedNote: {
marginTop: "0.25rem",
fontSize: "0.875rem",
lineHeight: "1.25rem",
color: colors.textMuted,
},
relatedList: {
marginTop: "0.5rem",
display: "flex",
flexWrap: "wrap",
gap: "0.75rem",
fontSize: "0.875rem",
lineHeight: "1.25rem",
},
link: {
color: colors.primaryOnSurface,
},
});
/** The look of one related action, so every caller's links match. */
export const provenanceRelatedLink = styles.link;
function Section({ title, children }: { title: string; children: ReactNode }) {
return (
<div {...stylex.props(styles.section)}>
<h2 {...stylex.props(styles.sectionHeading)}>{title}</h2>
<dl {...stylex.props(styles.facts)}>{children}</dl>
</div>
);
}
function Fact({ label, children }: { label: string; children: ReactNode }) {
return (
<>
<dt {...stylex.props(styles.term)}>{label}</dt>
<dd {...stylex.props(styles.value)}>{children}</dd>
</>
);
}
/**
* A recorded name, in the face the rest of the interface gives to names. It
* wraps the value rather than the row, so the prose that stands in for a
* missing one is not set in the same typewriter face.
*/
function Mono({ children }: { children: string }) {
return <span {...stylex.props(shared.mono)}>{children}</span>;
}
/** An empty text field means the server recorded nothing there, never an empty value. */
function Absent({ children }: { children: string }) {
return <span {...stylex.props(styles.muted)}>{children}</span>;
}
/**
* What an empty `matched` means. `no_match` is the only reason the matcher
* itself records with nothing to show; every other empty one names a pipeline
* step that answered before filtering a local record, a forward zone, a pause
* (see `PolicyReason` in src/storage/provenance.zig) where "nothing matched"
* would claim an evaluation that never happened.
*/
function unmatchedLabel(reason: PolicyReason): string {
return reason === "no_match" ? "Nothing matched" : "The matcher never ran";
}
interface Props {
provenance: Provenance;
/**
* The log row this explains, or null for a frame read straight off the
* stream. Null is the same fact `QuerySummary.id` carries: the event
* precedes its own insert, so there is no row to name and none is invented.
*/
persistedId: number | null;
/** The related-action links, built by whichever surface owns the routes. */
relatedActions: ReactNode;
}
export default function ProvenanceDetail({ provenance, persistedId, relatedActions }: Props) {
const { request, group, policy, rewrites, route, response } = provenance;
const clientNames = useClientNames();
const currentClient = clientLabel(request.client, clientNames);
return (
<>
<h1 {...stylex.props(styles.heading, shared.mono)}>{request.domain}</h1>
<p {...stylex.props(styles.subtitle)}>
{formatTime(request.time)} {policyActionLabel(policy.action)}
{/* The verdict alone reads as a success; a non-NOERROR answer says otherwise. */}
{response.rcode !== 0 && `${rcodeName(response.rcode)}`}
</p>
<div {...stylex.props(styles.record)}>
<p {...stylex.props(styles.recordNote)}>
What was recorded when this query was answered. Group and blocklist names are the ones in force at
that moment; they may have been renamed or deleted since.
{persistedId === null &&
" This is the event as it was streamed; the query log may not have written it yet."}
</p>
<Section title="Request">
<Fact label="Time">{formatTime(request.time)}</Fact>
<Fact label="Domain">
<Mono>{request.domain}</Mono>
</Fact>
<Fact label="Client">
<Mono>{request.client}</Mono>
</Fact>
<Fact label="Type">{qtypeName(request.qtype)}</Fact>
<Fact label="Class">{qclassName(request.qclass)}</Fact>
</Section>
<Section title="Group">
<Fact label="Name">{group.name === "" ? <Absent>No group recorded</Absent> : group.name}</Fact>
<Fact label="Id">{group.id === null ? <Absent></Absent> : group.id}</Fact>
</Section>
<Section title="Policy">
<Fact label="Decision">{policyActionLabel(policy.action)}</Fact>
<Fact label="Reason">{policyReasonLabel(policy.reason)}</Fact>
<Fact label="Matched">
{policy.matched === "" ? (
<Absent>{unmatchedLabel(policy.reason)}</Absent>
) : (
<Mono>{policy.matched}</Mono>
)}
</Fact>
<Fact label="Blocklist">
{policy.source_name === "" ? (
<Absent>Not a blocklist decision</Absent>
) : policy.source_id === null ? (
policy.source_name
) : (
`${policy.source_name} (#${policy.source_id})`
)}
</Fact>
</Section>
<Section title="Rewrites">
<Fact label="CNAME target">
{rewrites.cname_target === "" ? (
<Absent>The queried name was decided directly</Absent>
) : (
<Mono>{rewrites.cname_target}</Mono>
)}
</Fact>
<Fact label="Safe search">
{rewrites.safe_search_target === "" ? (
<Absent>No rewrite</Absent>
) : (
<Mono>{rewrites.safe_search_target}</Mono>
)}
</Fact>
</Section>
<Section title="Route">
<Fact label="Answered by">{routeKindLabel(route.kind)}</Fact>
<Fact label="Forward zone">
{route.forward_zone === "" ? <Absent></Absent> : <Mono>{route.forward_zone}</Mono>}
</Fact>
<Fact label="Upstream">
{route.upstream === "" ? <Absent>No upstream exchange</Absent> : <Mono>{route.upstream}</Mono>}
</Fact>
</Section>
<Section title="Response">
<Fact label="Result">{rcodeName(response.rcode)}</Fact>
<Fact label="Took">
{response.duration_us === null ? (
<Absent>Not measured</Absent>
) : (
formatMicros(response.duration_us)
)}
</Fact>
</Section>
</div>
<div {...stylex.props(styles.related)}>
<h2 {...stylex.props(styles.relatedHeading)}>Related</h2>
<p {...stylex.props(styles.relatedNote)}>
These read the current configuration, which may no longer be the one that decided this query.
</p>
{currentClient !== null && (
<p {...stylex.props(styles.relatedNote)}>
{currentClient.learned ? (
<>
Reverse DNS currently resolves <Mono>{request.client}</Mono> to{" "}
<Mono>{currentClient.text}</Mono>.
</>
) : (
<>
The client list currently names <Mono>{request.client}</Mono> {currentClient.text}.
</>
)}
</p>
)}
<div {...stylex.props(styles.relatedList)}>{relatedActions}</div>
</div>
</>
);
}
@@ -0,0 +1,59 @@
/**
* The links out of one query's detail, shared by the persisted detail page and
* the in-place detail a streamed row opens.
*
* Both surfaces answer the same four follow-up questions, and both are read
* from an investigation that has a time range. Every link therefore carries
* absolute bounds: a link that said "recently" would show a different set of
* queries every time it was opened, which is the opposite of what linking to an
* incident is for.
*/
import { Link } from "@tanstack/react-router";
import * as stylex from "@stylexjs/stylex";
import { styles as shared } from "@/ui/styles";
import { provenanceRelatedLink } from "./ProvenanceDetail";
import { diagnosticsBounds, relatedBounds } from "./relatedLinks";
import type { ActivitySearch } from "./search";
interface Props {
domain: string;
client: string;
/** The second this query was answered, which every window is centred on. */
ts: number;
/** The Activity search the reader came from; its bounds win over the defaults. */
origin: Pick<ActivitySearch, "since" | "until">;
}
export default function RelatedActions({ domain, client, ts, origin }: Props) {
const bounds = relatedBounds(ts, origin);
const window = diagnosticsBounds(ts);
return (
<>
<Link to="/activity/test" search={{ domain }} {...stylex.props(provenanceRelatedLink, shared.focusRing)}>
Test this domain against current policy
</Link>
<Link
to="/activity"
search={{ mode: "history", domain, client: undefined, blocked: undefined, ...bounds }}
{...stylex.props(provenanceRelatedLink, shared.focusRing)}
>
All activity for this domain
</Link>
<Link
to="/activity"
search={{ mode: "history", client, domain: undefined, blocked: undefined, ...bounds }}
{...stylex.props(provenanceRelatedLink, shared.focusRing)}
>
All activity from this client
</Link>
<Link
to="/diagnostics"
search={{ since: window.since, until: window.until }}
{...stylex.props(provenanceRelatedLink, shared.focusRing)}
>
Diagnostics around this query
</Link>
</>
);
}
+114
View File
@@ -0,0 +1,114 @@
import { render, screen } from "@testing-library/react";
import type { QueryRow } from "@/lib/types";
import type { ClientNames } from "@/features/clients/clientNames";
import { queryRow } from "@/features/queries/provenanceFixture";
import { summarizeRow, type QuerySummary } from "@/features/queries/querySummary";
import { ACTIVITY_COLUMNS, ActivityCells, ActivityTableHead, resultLabel, routeLabel } from "./cells";
const noNames: ClientNames = new Map();
function renderRow(overrides: Partial<QueryRow> = {}): HTMLTableRowElement {
const row: QuerySummary = summarizeRow(queryRow(1, overrides));
render(
<table>
<ActivityTableHead />
<tbody>
<tr data-testid="row">
<ActivityCells
row={row}
clientNames={noNames}
renderDomain={(id, children) => <a href={`/activity/queries/${id}`}>{children}</a>}
/>
</tr>
</tbody>
</table>,
);
return screen.getByTestId("row") as HTMLTableRowElement;
}
/** The cell under a header, read by its column name rather than its index. */
function cell(row: HTMLTableRowElement, column: (typeof ACTIVITY_COLUMNS)[number]): string {
const index = ACTIVITY_COLUMNS.indexOf(column);
return row.cells[index]?.textContent ?? "";
}
test("the head names the seven columns in order", () => {
render(
<table>
<ActivityTableHead />
</table>,
);
const headers = screen.getAllByRole("columnheader").map((header) => header.textContent);
expect(headers).toEqual(["Time", "Domain", "Client", "Type", "Result", "Route", "Duration"]);
});
test("an allowed NOERROR row reads as the answer it got, with no badge", () => {
const row = renderRow({ blocked: false, rcode: 0, route_kind: "upstream", response_time_us: 1234 });
expect(cell(row, "Result")).toBe("NOERROR");
expect(cell(row, "Route")).toBe("Upstream");
expect(cell(row, "Duration")).toBe("1.2 ms");
expect(cell(row, "Type")).toBe("A");
});
test("a blocked row reads Blocked even though the client got NOERROR", () => {
const row = renderRow({ blocked: true, rcode: 0, route_kind: "blocked", policy_reason: "blocklist_domain" });
expect(cell(row, "Result")).toBe("Blocked");
expect(cell(row, "Route")).toBe("Blocked");
});
test("a SERVFAIL row names the code", () => {
const row = renderRow({ blocked: false, rcode: 2, route_kind: "upstream", response_time_us: null });
expect(cell(row, "Result")).toBe("SERVFAIL");
expect(cell(row, "Duration")).toBe("—");
});
test("a cache hit names the cache as the route", () => {
const row = renderRow({ blocked: false, rcode: 0, route_kind: "cache", cache_hit: true, upstream: "" });
expect(cell(row, "Route")).toBe("Cache");
expect(cell(row, "Result")).toBe("NOERROR");
});
test("an unassigned extended rcode keeps the numeric fallback, without the long form's parentheses", () => {
const row = renderRow({ blocked: false, rcode: 3841 });
expect(cell(row, "Result")).toBe("RCODE 3841");
});
test("a persisted row links its domain to the detail the caller chose", () => {
renderRow({ domain: "ads.example" });
expect(screen.getByRole("link", { name: "ads.example" }).getAttribute("href")).toBe("/activity/queries/1");
});
test("a streamed row reaches the renderer with a null id, and can render as plain text", () => {
render(
<table>
<tbody>
<tr data-testid="row">
<ActivityCells
row={{ ...summarizeRow(queryRow(1)), id: null }}
clientNames={noNames}
renderDomain={(id, children) => {
expect(id).toBeNull();
return children;
}}
/>
</tr>
</tbody>
</table>,
);
expect(screen.queryByRole("link")).toBeNull();
expect(screen.getByTestId("row").textContent).toContain("example.com");
});
test("every route kind has a compact label", () => {
expect(routeLabel("blocked")).toBe("Blocked");
expect(routeLabel("local")).toBe("Local");
expect(routeLabel("forward_zone")).toBe("Forward zone");
expect(routeLabel("upstream")).toBe("Upstream");
expect(routeLabel("cache")).toBe("Cache");
expect(routeLabel("rejected")).toBe("Rejected");
});
test("resultLabel is the pure form of the Result cell", () => {
expect(resultLabel({ blocked: true, rcode: 2 })).toBe("Blocked");
expect(resultLabel({ blocked: false, rcode: 5 })).toBe("REFUSED");
});
+171
View File
@@ -0,0 +1,171 @@
/**
* The seven columns of the Activity table: Time, Domain, Client, Type, Result,
* Route and Duration.
*
* The labels here are the compact forms a scanned table needs. The detail page
* keeps `provenanceCopy`'s long forms, which spell out the same facts with room
* for the rcode number and the "answered by" phrasing.
*
* The cells render both a stored row and a streamed event, so they know nothing
* about routes: the caller renders the Domain cell's contents and decides what,
* if anything, a row opens. A streamed row has no id the frame precedes its
* own insert and only the caller knows whether it has a surface for one.
*/
import type { ReactNode } from "react";
import * as stylex from "@stylexjs/stylex";
import { formatMicros, formatTime } from "@/lib/format";
import type { RouteKind } from "@/lib/types";
import { ClientName, type ClientNames } from "@/features/clients/clientNames";
import { rcodeShortName } from "@/features/queries/provenanceCopy";
import { qtypeName } from "@/features/queries/qtype";
import type { QuerySummary } from "@/features/queries/querySummary";
import { styles as shared } from "@/ui/styles";
import { colors } from "@/ui/tokens.stylex";
const DARK = "@media (prefers-color-scheme: dark)";
const styles = stylex.create({
head: {
backgroundColor: { default: "oklch(98.5% 0 none)", [DARK]: "oklch(21% 0.006 285.885)" },
textAlign: "left",
},
th: {
paddingInline: "0.75rem",
paddingBlock: "0.5rem",
fontWeight: 500,
color: colors.textSecondary,
},
cell: {
paddingInline: "0.75rem",
paddingBlock: "0.5rem",
},
nowrap: {
whiteSpace: "nowrap",
},
breakAll: {
wordBreak: "break-all",
},
small: {
fontSize: "0.75rem",
lineHeight: "1rem",
},
muted: {
color: colors.textMuted,
},
domainLink: {
color: colors.primaryOnSurface,
textDecorationLine: "none",
},
/**
* The badge shape and its weight are the signal; the tint only says which
* kind of unhappy answer this was. A monochrome or colour-blind reading of
* the table still separates a blocked or failed row from a plain NOERROR
* one, which a hue alone would not.
*/
badge: {
display: "inline-block",
borderRadius: "0.25rem",
paddingInline: "0.375rem",
paddingBlock: "0.125rem",
fontSize: "0.75rem",
lineHeight: "1rem",
fontWeight: 600,
},
blockedBadge: {
backgroundColor: { default: "oklch(93.6% 0.032 17.717)", [DARK]: "oklch(39.6% 0.141 25.723)" },
color: { default: "oklch(44.4% 0.177 26.899)", [DARK]: "oklch(88.5% 0.062 18.334)" },
},
faultBadge: {
backgroundColor: { default: "oklch(96.2% 0.059 95.617)", [DARK]: "oklch(41.4% 0.112 45.904)" },
color: { default: "oklch(47.3% 0.137 46.201)", [DARK]: "oklch(90.1% 0.076 70.697)" },
},
});
const ROUTE_LABELS: Record<RouteKind, string> = {
blocked: "Blocked",
local: "Local",
forward_zone: "Forward zone",
upstream: "Upstream",
cache: "Cache",
rejected: "Rejected",
};
/** The compact Route label. `Record` over the union, so a new kind fails `tsc`. */
export function routeLabel(kind: RouteKind): string {
return ROUTE_LABELS[kind];
}
/**
* The compact Result label. A block is the answer the operator asked nxdns for,
* so it wins over the rcode it was delivered as a blocked name answered with
* NOERROR and a zero address is still "Blocked". Everything else reads as the
* code the client saw.
*/
export function resultLabel(row: Pick<QuerySummary, "blocked" | "rcode">): string {
return row.blocked ? "Blocked" : rcodeShortName(row.rcode);
}
/**
* What a row's domain is wrapped in. The caller owns the routes, so it builds
* the element; the look stays here, as `activityDomainLink`, which the caller
* spreads onto the control itself a link's own colour beats one inherited
* from a wrapper. `id` is null for a streamed row, which history has no surface
* for and live opens from memory.
*/
export type DomainRenderer = (id: number | null, children: ReactNode) => ReactNode;
export const activityDomainLink = styles.domainLink;
export function ResultCellContent({ row }: { row: Pick<QuerySummary, "blocked" | "rcode"> }) {
const label = resultLabel(row);
if (row.blocked) return <span {...stylex.props(styles.badge, styles.blockedBadge)}>{label}</span>;
if (row.rcode !== 0) return <span {...stylex.props(styles.badge, styles.faultBadge)}>{label}</span>;
return <span {...stylex.props(styles.small, styles.muted)}>{label}</span>;
}
export function ActivityCells({
row,
clientNames,
renderDomain,
}: {
row: QuerySummary;
clientNames: ClientNames;
renderDomain: DomainRenderer;
}) {
return (
<>
<td {...stylex.props(styles.cell, styles.nowrap, styles.muted)}>{formatTime(row.ts)}</td>
<td {...stylex.props(styles.cell, styles.small, styles.breakAll, shared.mono)}>
{renderDomain(row.id, row.domain)}
</td>
<td {...stylex.props(styles.cell, styles.small, styles.nowrap)}>
<ClientName ip={row.client_ip} names={clientNames} />
</td>
<td {...stylex.props(styles.cell, styles.nowrap)}>{qtypeName(row.qtype)}</td>
<td {...stylex.props(styles.cell, styles.nowrap)}>
<ResultCellContent row={row} />
</td>
<td {...stylex.props(styles.cell, styles.nowrap)}>{routeLabel(row.route_kind)}</td>
<td {...stylex.props(styles.cell, styles.nowrap, shared.tabularNums)}>
{row.response_time_us === null ? "—" : formatMicros(row.response_time_us)}
</td>
</>
);
}
export const ACTIVITY_COLUMNS = ["Time", "Domain", "Client", "Type", "Result", "Route", "Duration"] as const;
export function ActivityTableHead() {
return (
<thead {...stylex.props(styles.head)}>
<tr>
{ACTIVITY_COLUMNS.map((column) => (
<th key={column} {...stylex.props(styles.th)}>
{column}
</th>
))}
</tr>
</thead>
);
}
@@ -0,0 +1,100 @@
import {
datetimeField,
datetimeLocalToUnix,
editDatetimeField,
resolveDatetimeField,
unixToDatetimeLocal,
} from "./datetime";
/**
* Every assertion here is about local time, so the zone has to be pinned. New
* York is the zone the DST cases are written for: the fold is 2024-11-03 01:30
* and the gap is 2024-03-10 02:30.
*/
// The app never reads `process`, so `src` is typed without node's globals; the
// test host is node, where assigning `TZ` re-reads the zone for `Date`.
declare const process: { env: Record<string, string | undefined> };
const originalTz = process.env["TZ"];
beforeAll(() => {
process.env["TZ"] = "America/New_York";
});
afterAll(() => {
process.env["TZ"] = originalTz;
});
/** 2024-06-01T12:34:56 EDT. */
const SUMMER = 1_717_259_696;
test("a non-zero-second instant round trips", () => {
expect(unixToDatetimeLocal(SUMMER)).toBe("2024-06-01T12:34:56");
expect(datetimeLocalToUnix("2024-06-01T12:34:56")).toBe(SUMMER);
});
test("text without seconds parses as :00", () => {
expect(datetimeLocalToUnix("2024-06-01T12:34")).toBe(datetimeLocalToUnix("2024-06-01T12:34:00"));
});
test("text that is not a datetime-local value names no instant", () => {
expect(datetimeLocalToUnix("")).toBeUndefined();
expect(datetimeLocalToUnix("yesterday")).toBeUndefined();
expect(datetimeLocalToUnix("2024-06-01")).toBeUndefined();
expect(datetimeLocalToUnix("2024-13-01T00:00:00")).toBeUndefined();
});
/** 2024-11-03 01:30 EDT and 01:30 EST: two instants, one wall clock. */
const FOLD_FIRST = 1_730_611_800;
const FOLD_SECOND = 1_730_615_400;
test("the fall-back fold gives two instants the same text", () => {
expect(unixToDatetimeLocal(FOLD_FIRST)).toBe("2024-11-03T01:30:00");
expect(unixToDatetimeLocal(FOLD_SECOND)).toBe("2024-11-03T01:30:00");
expect(datetimeLocalToUnix("2024-11-03T01:30:00")).toBe(FOLD_FIRST);
});
test("an untouched fold bound applies the instant it was seeded with, not a re-parse of its text", () => {
const field = datetimeField(FOLD_SECOND);
expect(field.text).toBe("2024-11-03T01:30:00");
expect(resolveDatetimeField(field)).toEqual({ ok: true, value: FOLD_SECOND });
});
test("an edited fold bound resolves to the first of the two instants, which is what its text says", () => {
const field = editDatetimeField(datetimeField(FOLD_SECOND), "2024-11-03T01:30:00");
expect(resolveDatetimeField(field)).toEqual({ ok: true, value: FOLD_FIRST });
});
test("a spring-forward time that exists on no clock is rejected rather than slid forward an hour", () => {
const field = editDatetimeField(datetimeField(undefined), "2024-03-10T02:30:00");
expect(datetimeLocalToUnix("2024-03-10T02:30:00")).toBe(1_710_055_800);
expect(unixToDatetimeLocal(1_710_055_800)).toBe("2024-03-10T03:30:00");
expect(resolveDatetimeField(field)).toEqual({ ok: false, reason: "nonexistent" });
});
test("an edited bound round trips with non-zero seconds", () => {
const field = editDatetimeField(datetimeField(undefined), "2024-06-01T12:34:56");
expect(resolveDatetimeField(field)).toEqual({ ok: true, value: SUMMER });
});
test("clearing an edited bound drops the filter", () => {
const field = editDatetimeField(datetimeField(SUMMER), "");
expect(resolveDatetimeField(field)).toEqual({ ok: true, value: undefined });
});
test("an unparseable edit is reported, never silently dropped", () => {
const field = editDatetimeField(datetimeField(undefined), "2024-06-32T99:99");
expect(resolveDatetimeField(field)).toEqual({ ok: false, reason: "unparseable" });
});
test("an unset bound seeds an empty field that stays unset", () => {
const field = datetimeField(undefined);
expect(field.text).toBe("");
expect(resolveDatetimeField(field)).toEqual({ ok: true, value: undefined });
});
test("a fractional part on the seconds is parsed and dropped, not rejected", () => {
// jsdom, and any engine that sanitizes to the full grammar, hands the input
// back with milliseconds attached; a bound is a whole second either way.
const field = editDatetimeField(datetimeField(undefined), "2023-11-14T23:13:37.000");
const resolved = resolveDatetimeField(field);
expect(resolved).toEqual({ ok: true, value: datetimeLocalToUnix("2023-11-14T23:13:37") });
});
+95
View File
@@ -0,0 +1,95 @@
/**
* The bridge between a `datetime-local` input and the unix seconds the URL and
* the API speak.
*
* Local wall-clock text is lossy in a way unix seconds are not. Twice a year a
* fall-back fold gives two instants the same text, and a spring-forward gap
* gives an hour of text no instant at all. So the text is never the authority:
* a bound the operator did not touch is carried through as the number it
* already was, and a bound they did edit is accepted only when it survives a
* round trip unchanged.
*/
/** Zero-padded to the width the `datetime-local` grammar requires. */
function pad(value: number, width: number): string {
return String(value).padStart(width, "0");
}
/**
* Unix seconds the local wall-clock text a `datetime-local` input holds,
* always with seconds, because the inputs run at `step={1}`.
*/
export function unixToDatetimeLocal(unix: number): string {
const date = new Date(unix * 1000);
const day = `${pad(date.getFullYear(), 4)}-${pad(date.getMonth() + 1, 2)}-${pad(date.getDate(), 2)}`;
const time = `${pad(date.getHours(), 2)}:${pad(date.getMinutes(), 2)}:${pad(date.getSeconds(), 2)}`;
return `${day}T${time}`;
}
const DATETIME_LOCAL = /^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2})(?::(\d{2})(?:\.\d{1,3})?)?$/;
/**
* The text with its seconds spelled out, or undefined when it is not a
* `datetime-local` value at all. A browser omits `:00` seconds even at
* `step={1}`, so the canonical form is what a round trip compares against.
*
* The grammar allows a fractional part after the seconds and some engines emit
* one; a bound is a whole second here and on the wire, so it is parsed and then
* dropped rather than treated as text we do not recognise.
*/
function canonicalize(value: string): string | undefined {
const match = DATETIME_LOCAL.exec(value);
if (match === null) return undefined;
return `${match[1]}-${match[2]}-${match[3]}T${match[4]}:${match[5]}:${match[6] ?? "00"}`;
}
/** Local wall-clock text → unix seconds, or undefined when it names no instant. */
export function datetimeLocalToUnix(value: string): number | undefined {
const canonical = canonicalize(value);
if (canonical === undefined) return undefined;
const ms = new Date(canonical).getTime();
return Number.isFinite(ms) ? Math.floor(ms / 1000) : undefined;
}
/**
* One bound of the filter form: what the input shows, what the applied search
* carried, and whether the operator has touched it since.
*/
export interface DatetimeField {
text: string;
/** The applied value this field was seeded from, reused while `dirty` is false. */
original: number | undefined;
dirty: boolean;
}
export type DatetimeResolution =
{ ok: true; value: number | undefined } | { ok: false; reason: "unparseable" | "nonexistent" };
export function datetimeField(original: number | undefined): DatetimeField {
return { text: original === undefined ? "" : unixToDatetimeLocal(original), original, dirty: false };
}
export function editDatetimeField(field: DatetimeField, text: string): DatetimeField {
return { ...field, text, dirty: true };
}
/**
* The unix value this bound applies.
*
* An untouched field resolves to the number it was seeded with, never to a
* re-parse of its own text: the text of a fall-back instant names two of them,
* and re-parsing would silently move a bound the operator never edited.
*
* An edited field is parsed, then formatted back. A wall-clock time inside the
* spring-forward gap exists on no clock, and `Date` quietly slides it forward
* an hour; the round trip catches that and the caller reports it instead of
* filtering on an hour nobody asked for.
*/
export function resolveDatetimeField(field: DatetimeField): DatetimeResolution {
if (!field.dirty) return { ok: true, value: field.original };
if (field.text.trim() === "") return { ok: true, value: undefined };
const unix = datetimeLocalToUnix(field.text);
if (unix === undefined) return { ok: false, reason: "unparseable" };
if (unixToDatetimeLocal(unix) !== canonicalize(field.text)) return { ok: false, reason: "nonexistent" };
return { ok: true, value: unix };
}
@@ -0,0 +1,42 @@
/**
* The absolute bounds an investigation link carries.
*
* Every link out of a query detail is time-scoped on purpose: a relative window
* would answer a different question tomorrow than it does today, and the whole
* point of linking to an episode is that the link keeps showing that episode.
*/
import type { ActivitySearch } from "./search";
/** The five-minute window the redesign puts around one query. */
export const RELATED_WINDOW_SECONDS = 300;
export interface Bounds {
since: number;
until: number;
}
/**
* The bounds for "all activity for this domain/client", decided per bound.
*
* When the reader arrived from a bounded investigation, that bound is the one
* they are working in and it carries over. A bound they never set falls back to
* the five-minute window around this query never to no bound at all, which
* would answer with the whole retained history and lose the episode in it. The
* two bounds are decided separately, so a half-bounded origin keeps its half.
*/
export function relatedBounds(ts: number, origin: Pick<ActivitySearch, "since" | "until">): Bounds {
return {
since: origin.since ?? ts - RELATED_WINDOW_SECONDS,
until: origin.until ?? ts + RELATED_WINDOW_SECONDS,
};
}
/**
* The Diagnostics window around one query. Fixed at five minutes either side of
* the query, not inherited: the reader is asking what else was failing while
* this query was answered, which is a question about the query's own moment.
*/
export function diagnosticsBounds(ts: number): Bounds {
return { since: ts - RELATED_WINDOW_SECONDS, until: ts + RELATED_WINDOW_SECONDS };
}
@@ -0,0 +1,248 @@
import type { Provenance, QueryRow } from "@/lib/types";
import { provenance, queryRow } from "@/features/queries/provenanceFixture";
import { RING_CAPACITY, mergeGap, pushRow, summaryOf, type LiveRow } from "./ringBuffer";
function streamed(key: number, ts: number, domain: string, sections: Parameters<typeof provenance>[0] = {}): LiveRow {
return {
kind: "streamed",
key,
event: provenance({
...sections,
request: { time: ts, domain, ...sections.request },
route: { upstream: "udp://9.9.9.9:53", ...sections.route },
}),
};
}
function fetchedRow(id: number, ts: number, domain: string, overrides: Partial<QueryRow> = {}): QueryRow {
return queryRow(id, { ts, domain, upstream: "udp://9.9.9.9:53", ...overrides });
}
function counter(start = 100): () => number {
let n = start;
return () => ++n;
}
function domains(rows: LiveRow[]): string[] {
return rows.map((row) => summaryOf(row).domain);
}
describe("summaryOf", () => {
test("a streamed frame projects every summary field from the provenance it carries", () => {
const event: Provenance = provenance({
request: { time: 1700, domain: "ads.example", client: "192.0.2.11", qtype: 28 },
policy: { action: "block", reason: "blocklist_wildcard" },
route: { kind: "blocked", upstream: "" },
response: { duration_us: 42 },
});
expect(summaryOf({ kind: "streamed", key: 1, event })).toEqual({
id: null,
ts: 1700,
domain: "ads.example",
client_ip: "192.0.2.11",
qtype: 28,
blocked: true,
policy_reason: "blocklist_wildcard",
rcode: 0,
route_kind: "blocked",
response_time_us: 42,
cache_hit: null,
upstream: "",
});
});
test("a recovered row projects its stored fields and keeps its id", () => {
const row = queryRow(77, { domain: "news.example", cache_hit: true, policy_reason: "rule_allow_exact" });
expect(summaryOf({ kind: "recovered", key: 2, row })).toMatchObject({
id: 77,
domain: "news.example",
cache_hit: true,
policy_reason: "rule_allow_exact",
});
});
/**
* The guard the discriminated union exists for: a field added to the wire
* DTO must be either projected into the summary or consciously left to the
* detail page. A silent addition fails here rather than going unrendered.
*/
test("every provenance field is either projected or knowingly detail-only", () => {
const projected = [
"request.time",
"request.domain",
"request.client",
"request.qtype",
"policy.action",
"policy.reason",
"route.kind",
"route.upstream",
"response.duration_us",
];
const detailOnly = [
"request.qclass",
"group.id",
"group.name",
"policy.matched",
"policy.source_id",
"policy.source_name",
"rewrites.cname_target",
"rewrites.safe_search_target",
"route.forward_zone",
"response.rcode",
];
const leaves = Object.entries(provenance()).flatMap(([section, fields]) =>
Object.keys(fields as Record<string, unknown>).map((field) => `${section}.${field}`),
);
expect(leaves.sort()).toEqual([...projected, ...detailOnly].sort());
});
});
describe("pushRow", () => {
test("prepends newest-first", () => {
let rows: LiveRow[] = [];
rows = pushRow(rows, streamed(1, 10, "a.example"));
rows = pushRow(rows, streamed(2, 11, "b.example"));
expect(domains(rows)).toEqual(["b.example", "a.example"]);
});
test("drops the oldest beyond capacity", () => {
let rows: LiveRow[] = [];
for (let i = 0; i < 5; i++) rows = pushRow(rows, streamed(i, i, `d${i}.example`), 3);
expect(rows).toHaveLength(3);
expect(rows.map((r) => r.key)).toEqual([4, 3, 2]);
});
test("default capacity is 500", () => {
let rows: LiveRow[] = [];
for (let i = 0; i < RING_CAPACITY + 10; i++) rows = pushRow(rows, streamed(i, i, "x.example"));
expect(rows).toHaveLength(RING_CAPACITY);
});
});
describe("mergeGap", () => {
test("skips rows already in the buffer and counts only new ones", () => {
const buffer = [streamed(2, 100, "seen.example"), streamed(1, 99, "old.example")];
const fetched = [
fetchedRow(30, 102, "gap2.example"),
fetchedRow(29, 101, "gap1.example"),
fetchedRow(28, 100, "seen.example"),
];
const { rows, missed } = mergeGap(buffer, fetched, counter());
expect(missed).toBe(2);
expect(domains(rows)).toEqual(["gap2.example", "gap1.example", "seen.example", "old.example"]);
});
test("no additions returns the buffer unchanged with missed 0", () => {
const buffer = [streamed(1, 100, "seen.example")];
const { rows, missed } = mergeGap(buffer, [fetchedRow(5, 100, "seen.example")], counter());
expect(missed).toBe(0);
expect(rows).toBe(buffer);
});
/**
* A household repeats itself: one client, one name, three lookups inside the
* same second. The stream delivered one of them before the connection broke,
* so the gap fetch must recover the other two rather than let the one row in
* the buffer stand for all three.
*/
test("repeated identical queries drop only as many rows as the buffer already holds", () => {
const buffer = [streamed(1, 100, "dup.example")];
const fetched = [
fetchedRow(12, 100, "dup.example"),
fetchedRow(11, 100, "dup.example"),
fetchedRow(10, 100, "dup.example"),
];
const { rows, missed } = mergeGap(buffer, fetched, counter());
expect(missed).toBe(2);
expect(domains(rows)).toEqual(["dup.example", "dup.example", "dup.example"]);
const recoveredIds = rows.flatMap((row) => (row.kind === "recovered" ? [row.row.id] : []));
expect(new Set(recoveredIds).size).toBe(2);
});
test("a gap fetch that repeats the whole buffer adds nothing", () => {
const buffer = [streamed(2, 100, "dup.example"), streamed(1, 100, "dup.example")];
const fetched = [fetchedRow(12, 100, "dup.example"), fetchedRow(11, 100, "dup.example")];
const { rows, missed } = mergeGap(buffer, fetched, counter());
expect(missed).toBe(0);
expect(rows).toBe(buffer);
});
/**
* Two queries of the same name from the same client in the same second are
* still separate facts when any stored column differs the record type or
* class, the response code, the policy that decided them, how long they took,
* the route taken. The gap fetch here returns the differing row *first* and
* the one the buffer already holds second, so an identity blind to the column
* would let the differing row consume the buffered occurrence: the buffered
* query would come back duplicated and the other would vanish, at an
* unchanged `missed`. Order is what exposes that the count alone is 1
* either way.
*
* `blocked` and `cache_hit` have no case of their own: the server derives
* them from `policy_action` and `route_kind`, so they cannot differ while
* everything else holds, and the two columns they follow are covered here.
*/
test.each([
{ column: "qtype", sections: { request: { qtype: 1 } }, held: { qtype: 1 }, differing: { qtype: 28 } },
{ column: "qclass", sections: { request: { qclass: 1 } }, held: { qclass: 1 }, differing: { qclass: 3 } },
{ column: "rcode", sections: { response: { rcode: 0 } }, held: { rcode: 0 }, differing: { rcode: 2 } },
{
column: "response_time_us",
sections: { response: { duration_us: 1234 } },
held: { response_time_us: 1234 },
differing: { response_time_us: 9999 },
},
{
column: "route_kind",
sections: { route: { kind: "upstream" } },
held: { route_kind: "upstream" },
differing: { route_kind: "forward_zone" },
},
{
column: "policy_action",
sections: { policy: { action: "allow" } },
held: { policy_action: "allow" },
differing: { policy_action: "not_evaluated" },
},
{
column: "policy_reason",
sections: { policy: { reason: "no_match" } },
held: { policy_reason: "no_match" },
differing: { policy_reason: "rule_allow_exact" },
},
] satisfies readonly {
column: string;
sections: Parameters<typeof provenance>[0];
held: Partial<QueryRow>;
differing: Partial<QueryRow>;
}[])("rows differing only in $column survive the gap merge", ({ sections, held, differing }) => {
const buffer = [streamed(1, 100, "dual.example", sections)];
const fetched = [fetchedRow(6, 100, "dual.example", differing), fetchedRow(5, 100, "dual.example", held)];
const { rows, missed } = mergeGap(buffer, fetched, counter());
expect(missed).toBe(1);
expect(rows).toEqual([{ kind: "recovered", key: expect.any(Number), row: fetched[0] }, buffer[0]]);
});
test("recovered rows keep their id and take a fresh key", () => {
const { rows } = mergeGap([], [fetchedRow(77, 100, "gap.example")], counter(200));
const recovered = rows[0];
expect(recovered?.key).toBe(201);
expect(recovered?.kind).toBe("recovered");
expect(recovered !== undefined && recovered.kind === "recovered" ? recovered.row.id : null).toBe(77);
});
test("result is capped at capacity, keeping the newest", () => {
const buffer = [streamed(3, 300, "live.example")];
const fetched = [fetchedRow(2, 302, "g2.example"), fetchedRow(1, 301, "g1.example")];
const { rows, missed } = mergeGap(buffer, fetched, counter(), 2);
expect(missed).toBe(2);
expect(domains(rows)).toEqual(["g2.example", "g1.example"]);
});
test("merged rows stay sorted newest-first by ts", () => {
const buffer = [streamed(4, 105, "after-reopen.example"), streamed(3, 100, "before.example")];
const fetched = [fetchedRow(9, 103, "gap.example")];
const { rows } = mergeGap(buffer, fetched, counter());
expect(rows.map((row) => summaryOf(row).ts)).toEqual([105, 103, 100]);
});
});
+147
View File
@@ -0,0 +1,147 @@
import type { LiveQueryEvent, QueryRow } from "@/lib/types";
import { summarizeEvent, summarizeRow, type QuerySummary } from "@/features/queries/querySummary";
/**
* A row in the live buffer. `key` is a client-side monotonic counter, because
* neither arm has a stable identity of its own on arrival.
*
* The two arms are genuinely different facts, not two encodings of one. A
* streamed frame carries the full provenance of a query the server has not
* written yet; a row recovered by the reconnect gap-fetch is the stored summary
* of a query that *was* written, and cannot fabricate the provenance it never
* received. Only the recovered arm has a row id to link to.
*/
export type LiveRow = { key: number } & (
{ kind: "streamed"; event: LiveQueryEvent } | { kind: "recovered"; row: QueryRow }
);
/** The arm that carries its own provenance, and so its own detail surface. */
export type StreamedRow = Extract<LiveRow, { kind: "streamed" }>;
/** The flat cells both arms render, and the shared identity for gap dedupe. */
export function summaryOf(row: LiveRow): QuerySummary {
return row.kind === "streamed" ? summarizeEvent(row.event) : summarizeRow(row.row);
}
export const RING_CAPACITY = 500;
/** Prepend `row` (rows are newest-first) and drop the oldest beyond `capacity`. */
export function pushRow(rows: LiveRow[], row: LiveRow, capacity: number = RING_CAPACITY): LiveRow[] {
const next = [row, ...rows];
return next.length > capacity ? next.slice(0, capacity) : next;
}
/**
* What the gap merge compares two queries by: the whole stored row bar its id.
*
* `since` on GET /api/queries is inclusive, so the re-sync fetch returns the
* last-seen row(s) again and the merge has to recognise them. The id cannot
* serve as the identity a streamed frame precedes its own insert and has none
* so the comparison is by value, and every stored column has to take part.
* Two queries alike in name, client and second but differing in class, rcode,
* the policy that decided them or the route taken are separate facts; if they
* hashed alike, the fetched row that does *not* match the buffered one would
* consume its occurrence, duplicating one query and losing the other.
*
* `QuerySummary` is the wrong basis for that: it is what the table renders, and
* it drops qclass and policy_action. `Omit<QueryRow, "id">`
* instead makes the compiler demand a derivation for every stored column, so a
* column added to the row cannot quietly fall out of the identity.
*/
type GapIdentity = Omit<QueryRow, "id">;
function identityOfRow(row: QueryRow): GapIdentity {
return {
ts: row.ts,
domain: row.domain,
client_ip: row.client_ip,
qtype: row.qtype,
qclass: row.qclass,
rcode: row.rcode,
blocked: row.blocked,
response_time_us: row.response_time_us,
cache_hit: row.cache_hit,
upstream: row.upstream,
policy_action: row.policy_action,
policy_reason: row.policy_reason,
route_kind: row.route_kind,
};
}
/**
* The same identity out of a live frame, which carries every stored column in
* its provenance. The columns the server derives rather than sends `blocked`
* and `cache_hit` come through `summarizeEvent` so that derivation keeps
* living in exactly one place.
*/
function identityOfEvent(event: LiveQueryEvent): GapIdentity {
const summary = summarizeEvent(event);
return {
ts: summary.ts,
domain: summary.domain,
client_ip: summary.client_ip,
qtype: summary.qtype,
qclass: event.request.qclass,
rcode: event.response.rcode,
blocked: summary.blocked,
response_time_us: summary.response_time_us,
cache_hit: summary.cache_hit,
upstream: summary.upstream,
policy_action: event.policy.action,
policy_reason: summary.policy_reason,
route_kind: event.route.kind,
};
}
function identityOf(row: LiveRow): GapIdentity {
return row.kind === "streamed" ? identityOfEvent(row.event) : identityOfRow(row.row);
}
/** Sorted keys so the hash cannot depend on the order the two arms happen to build their literals in. */
function signature(identity: GapIdentity): string {
return JSON.stringify(identity, Object.keys(identity).sort());
}
/**
* How many times each signature is already in the buffer. A signature is not
* unique: one client asking for one name twice within the same second is an
* ordinary household pattern, and the two queries are separate facts. Counting
* the occurrences lets the merge drop exactly as many fetched rows as the
* buffer already holds, instead of letting one buffered row hide all of them.
*/
function occurrences(rows: LiveRow[]): Map<string, number> {
const counts = new Map<string, number>();
for (const row of rows) {
const key = signature(identityOf(row));
counts.set(key, (counts.get(key) ?? 0) + 1);
}
return counts;
}
/**
* Merge rows fetched for a reconnect gap (newest-first, from GET /api/queries)
* into the buffer. Each fetched row consumes one buffered occurrence of its
* signature and is skipped; the rest are genuinely missed and `missed` counts
* them. The result stays newest-first (stable sort by ts) and capped.
*/
export function mergeGap(
rows: LiveRow[],
fetched: QueryRow[],
nextKey: () => number,
capacity: number = RING_CAPACITY,
): { rows: LiveRow[]; missed: number } {
const buffered = occurrences(rows);
const added: LiveRow[] = [];
for (const row of fetched) {
const key = signature(identityOfRow(row));
const count = buffered.get(key) ?? 0;
if (count > 0) {
buffered.set(key, count - 1);
continue;
}
added.push({ kind: "recovered", row, key: nextKey() });
}
if (added.length === 0) return { rows, missed: 0 };
const merged = [...added, ...rows].sort((a, b) => summaryOf(b).ts - summaryOf(a).ts).slice(0, capacity);
return { rows: merged, missed: added.length };
}
+113
View File
@@ -0,0 +1,113 @@
import { validateActivitySearch, validateBlocked, validateMode, validateText, validateTimestamp } from "./search";
test("mode is the two-value union, defaulting to history", () => {
expect(validateMode("live")).toBe("live");
expect(validateMode("history")).toBe("history");
expect(validateMode(undefined)).toBe("history");
expect(validateMode("Live")).toBe("history");
expect(validateMode("")).toBe("history");
expect(validateMode(0)).toBe("history");
expect(validateMode(["live"])).toBe("history");
});
test("a bound is a safe integer or nothing at all", () => {
expect(validateTimestamp(1_700_000_000)).toBe(1_700_000_000);
expect(validateTimestamp(0)).toBe(0);
expect(validateTimestamp(-1)).toBe(-1);
});
test.each([
["a fraction", 1_700_000_000.5],
["Infinity", Number.POSITIVE_INFINITY],
["-Infinity", Number.NEGATIVE_INFINITY],
["NaN", Number.NaN],
["past the safe range", Number.MAX_SAFE_INTEGER + 1],
["1e21", 1e21],
["a numeric string", "1700000000"],
["an empty string", ""],
["null", null],
["undefined", undefined],
["a boolean", true],
["an array", [1_700_000_000]],
["a bigint", 1_700_000_000n],
])("a bound rejects %s", (_name, value) => {
expect(validateTimestamp(value)).toBeUndefined();
});
test("blocked keeps false, which is the allowed-only filter", () => {
expect(validateBlocked(true)).toBe(true);
expect(validateBlocked(false)).toBe(false);
});
test.each([
["the string true", "true"],
["the string false", "false"],
["1", 1],
["0", 0],
["null", null],
["undefined", undefined],
])("blocked rejects %s", (_name, value) => {
expect(validateBlocked(value)).toBeUndefined();
});
test("a text filter is trimmed, and an empty one is no filter", () => {
expect(validateText("ads.example")).toBe("ads.example");
expect(validateText(" ads.example ")).toBe("ads.example");
expect(validateText("")).toBeUndefined();
expect(validateText(" ")).toBeUndefined();
expect(validateText("\t\n")).toBeUndefined();
expect(validateText(42)).toBeUndefined();
expect(validateText(undefined)).toBeUndefined();
});
test("a whole search normalizes every field and drops nothing else in", () => {
expect(
validateActivitySearch({
mode: "live",
since: 1_700_000_000,
until: 1_700_000_600,
domain: " ads.example ",
client: "192.0.2.10",
blocked: false,
unknown: "kept out",
}),
).toEqual({
mode: "live",
since: 1_700_000_000,
until: 1_700_000_600,
domain: "ads.example",
client: "192.0.2.10",
blocked: false,
});
});
test("an empty search is history with no filters", () => {
expect(validateActivitySearch({})).toEqual({
mode: "history",
since: undefined,
until: undefined,
domain: undefined,
client: undefined,
blocked: undefined,
});
});
test("a search of junk applies nothing", () => {
expect(
validateActivitySearch({
mode: "HISTORY ",
since: "1700000000",
until: Number.POSITIVE_INFINITY,
domain: " ",
client: null,
blocked: "true",
}),
).toEqual({
mode: "history",
since: undefined,
until: undefined,
domain: undefined,
client: undefined,
blocked: undefined,
});
});
+88
View File
@@ -0,0 +1,88 @@
/**
* The Activity search parameters, validated as pure functions so the route's
* `validateSearch` stays a one-liner and every rejection is testable without a
* router.
*
* A search value arrives from a URL, from history state, or from a hand-typed
* link, so nothing about its type is given. Anything that is not exactly the
* value the API can filter on becomes `undefined`: an unbounded page is honest,
* a page filtered on a coerced guess is not.
*/
import type { QueriesFilter } from "@/lib/types";
export const ACTIVITY_MODES = ["history", "live"] as const;
export type ActivityMode = (typeof ACTIVITY_MODES)[number];
export interface ActivitySearch {
mode: ActivityMode;
since: number | undefined;
until: number | undefined;
domain: string | undefined;
client: string | undefined;
blocked: boolean | undefined;
}
/** History is the surface a bare `/activity` should open on: it answers questions. */
export function validateMode(value: unknown): ActivityMode {
return value === "live" ? "live" : "history";
}
/**
* A unix-second bound. `Number.isSafeInteger` is the whole test: it rejects a
* fraction, an infinity, a NaN and a magnitude past 2^53 in one step, and a
* string never passes, so `?since=now` cannot reach the API as garbage.
*/
export function validateTimestamp(value: unknown): number | undefined {
return Number.isSafeInteger(value) ? (value as number) : undefined;
}
/**
* The blocked filter. `false` is a real filter "allowed only" so it must
* survive; only a genuine boolean does, because `"false"` out of a URL parser
* that did not decode JSON would otherwise read as true.
*/
export function validateBlocked(value: unknown): boolean | undefined {
return typeof value === "boolean" ? value : undefined;
}
/**
* A text filter, trimmed. An empty result becomes `undefined` rather than `""`:
* the server treats an empty filter as no filter, and a URL that showed
* `domain=` as applied state would claim a filter that is not filtering.
*/
export function validateText(value: unknown): string | undefined {
if (typeof value !== "string") return undefined;
const trimmed = value.trim();
return trimmed === "" ? undefined : trimmed;
}
/**
* The API filter for a validated search, built field by field.
*
* Only the fields that are actually set are written, so an unfiltered request
* carries no keys at all: `GET /api/queries` rejects a parameter it does not
* know, and the infinite query's cache key is the filter object, so a key
* present-but-undefined and a key absent must not be two different windows onto
* the same rows. `mode` never appears it selects the surface, not the rows.
*/
export function queriesFilterOf(search: Omit<ActivitySearch, "mode">): QueriesFilter {
const filter: QueriesFilter = {};
if (search.domain !== undefined) filter.domain = search.domain;
if (search.client !== undefined) filter.client = search.client;
if (search.blocked !== undefined) filter.blocked = search.blocked;
if (search.since !== undefined) filter.since = search.since;
if (search.until !== undefined) filter.until = search.until;
return filter;
}
export function validateActivitySearch(search: Record<string, unknown>): ActivitySearch {
return {
mode: validateMode(search["mode"]),
since: validateTimestamp(search["since"]),
until: validateTimestamp(search["until"]),
domain: validateText(search["domain"]),
client: validateText(search["client"]),
blocked: validateBlocked(search["blocked"]),
};
}
@@ -1,6 +1,8 @@
import { act, renderHook, waitFor } from "@testing-library/react";
import { ApiError } from "@/lib/api";
import type { LiveQueryEvent, QueriesPage, QueryRow } from "@/lib/types";
import type { QueriesPage, QueryRow } from "@/lib/types";
import { provenance, queryRow } from "@/features/queries/provenanceFixture";
import { summaryOf, type LiveRow } from "./ringBuffer";
import { FakeEventSource } from "./fakeEventSource";
import { CAP_ERROR_THRESHOLD, useLiveQueries } from "./useLiveQueries";
@@ -8,41 +10,28 @@ afterEach(() => vi.unstubAllGlobals());
function stubLocationAssign() {
const assign = vi.fn();
vi.stubGlobal("location", { pathname: "/live", search: "", assign });
vi.stubGlobal("location", { pathname: "/activity", search: "?mode=live", assign });
return assign;
}
function frame(ts: number, domain: string, overrides: Partial<LiveQueryEvent> = {}): { data: string } {
const payload: LiveQueryEvent = {
ts,
domain,
client_ip: "192.0.2.10",
qtype: 1,
blocked: false,
block_reason: "",
response_time_us: 500,
cache_hit: false,
upstream: "udp://9.9.9.9:53",
...overrides,
};
function frame(ts: number, domain: string): { data: string } {
const payload = provenance({
request: { time: ts, domain },
route: { upstream: "udp://9.9.9.9:53" },
});
return { data: JSON.stringify(payload) };
}
function fetchedRow(id: number, ts: number, domain: string): QueryRow {
return {
id,
ts,
domain,
client_ip: "192.0.2.10",
qtype: 1,
blocked: false,
block_reason: "",
response_time_us: 500,
cache_hit: false,
upstream: "udp://9.9.9.9:53",
};
return queryRow(id, { ts, domain, upstream: "udp://9.9.9.9:53" });
}
function domains(rows: LiveRow[]): string[] {
return rows.map((row) => summaryOf(row).domain);
}
const FULL_COVERAGE = { complete: true, available_since: 0 };
function setup(fetchSince?: (since: number) => Promise<QueriesPage>, probeSession?: () => Promise<unknown>) {
const sources: FakeEventSource[] = [];
const createEventSource = (url: string) => {
@@ -68,7 +57,7 @@ test("open then frames: rows newest-first with increasing keys", () => {
sources[0]!.emit("query", frame(1001, "b.example"));
});
const rows = hook.result.current.rows;
expect(rows.map((r) => r.domain)).toEqual(["b.example", "a.example"]);
expect(domains(rows)).toEqual(["b.example", "a.example"]);
expect(rows[0]!.key).toBeGreaterThan(rows[1]!.key);
});
@@ -87,6 +76,7 @@ test("error then reopen re-syncs the gap since the last seen ts", async () => {
return Promise.resolve({
queries: [fetchedRow(9, 1002, "gap.example"), fetchedRow(8, since, "a.example")],
next_before: null,
coverage: FULL_COVERAGE,
});
});
const { sources, hook } = setup(fetchSince);
@@ -103,7 +93,7 @@ test("error then reopen re-syncs the gap since the last seen ts", async () => {
expect(fetchSince).toHaveBeenCalledWith(1000);
await waitFor(() => expect(hook.result.current.missed).toBe(1));
expect(hook.result.current.rows.map((r) => r.domain)).toEqual(["gap.example", "a.example"]);
expect(domains(hook.result.current.rows)).toEqual(["gap.example", "a.example"]);
act(() => hook.result.current.dismissMissed());
expect(hook.result.current.missed).toBeNull();
@@ -127,7 +117,7 @@ test("a 401 gap re-sync redirects to login instead of setting resyncFailed", asy
act(() => sources[0]!.emit("query", frame(1000, "a.example")));
act(() => sources[0]!.emit("error"));
act(() => sources[0]!.emit("open"));
await waitFor(() => expect(assign).toHaveBeenCalledWith("/login?redirect=%2Flive"));
await waitFor(() => expect(assign).toHaveBeenCalledWith("/login?redirect=%2Factivity%3Fmode%3Dlive"));
expect(hook.result.current.resyncFailed).toBe(false);
});
@@ -152,7 +142,7 @@ test("cap trip with an expired session redirects to login", async () => {
act(() => {
for (let i = 0; i < CAP_ERROR_THRESHOLD; i++) sources[0]!.emit("error");
});
await waitFor(() => expect(assign).toHaveBeenCalledWith("/login?redirect=%2Flive"));
await waitFor(() => expect(assign).toHaveBeenCalledWith("/login?redirect=%2Factivity%3Fmode%3Dlive"));
expect(probeSession).toHaveBeenCalledTimes(1);
});
@@ -193,7 +183,7 @@ test("a fatal rejection with an expired session redirects to login", async () =>
act(() => sources[0]!.failFatal());
await waitFor(() => expect(assign).toHaveBeenCalledWith("/login?redirect=%2Flive"));
await waitFor(() => expect(assign).toHaveBeenCalledWith("/login?redirect=%2Factivity%3Fmode%3Dlive"));
expect(probeSession).toHaveBeenCalledTimes(1);
});
@@ -235,12 +225,12 @@ test("freeze keeps the display fixed while the buffer keeps filling", () => {
sources[0]!.emit("query", frame(1001, "b.example"));
sources[0]!.emit("query", frame(1002, "c.example"));
});
expect(hook.result.current.rows.map((r) => r.domain)).toEqual(["a.example"]);
expect(domains(hook.result.current.rows)).toEqual(["a.example"]);
expect(hook.result.current.liveCount).toBe(3);
act(() => hook.result.current.toggleFreeze());
expect(hook.result.current.frozen).toBe(false);
expect(hook.result.current.rows.map((r) => r.domain)).toEqual(["c.example", "b.example", "a.example"]);
expect(domains(hook.result.current.rows)).toEqual(["c.example", "b.example", "a.example"]);
});
test("stale sources are ignored after retry and closed on unmount", () => {
@@ -121,8 +121,12 @@ export function useLiveQueries(options?: LiveQueriesOptions): LiveQueries {
} catch {
return;
}
lastSeenTsRef.current = payload.ts;
bufferRef.current = pushRow(bufferRef.current, { ...payload, key: ++keyRef.current });
lastSeenTsRef.current = payload.request.time;
bufferRef.current = pushRow(bufferRef.current, {
kind: "streamed",
event: payload,
key: ++keyRef.current,
});
setRows(bufferRef.current);
});
+22 -8
View File
@@ -3,6 +3,11 @@
* with the same precedence the Clients page applies: a hand-typed `name` wins,
* the reverse-DNS `learned_name` stands in muted behind it, and an address with
* neither including one the loaded list has never seen stays bare.
*
* The muted colour is the whole of the affordance here. The Clients page pairs
* it with an outlined "learned" tag, and keeps it: one mention per client is
* information. Repeating that tag down every row of a query table is noise, so
* the tables carry the name alone.
*/
import { useMemo } from "react";
@@ -32,18 +37,27 @@ export function useClientNames(): ClientNames {
);
}
export function ClientName({ ip, names }: { ip: string; names: ClientNames }) {
/**
* The name the loaded list gives this address right now, or null when it gives
* none. Callers that must distinguish "named" from "bare address" rather than
* just render whichever applies read this instead of re-deriving precedence.
*/
export function clientLabel(ip: string, names: ClientNames): { text: string; learned: boolean } | null {
const client = names.get(ip);
if (client === undefined || (client.name === "" && client.learned_name === "")) {
return <span {...stylex.props(shared.mono)}>{ip}</span>;
}
if (client === undefined) return null;
if (client.name !== "") return { text: client.name, learned: false };
if (client.learned_name !== "") return { text: client.learned_name, learned: true };
return null;
}
export function ClientName({ ip, names }: { ip: string; names: ClientNames }) {
const label = clientLabel(ip, names);
if (label === null) return <span {...stylex.props(shared.mono)}>{ip}</span>;
// The name replaces the address on screen, so the address stays reachable
// as the tooltip rather than disappearing from the row entirely.
if (client.name !== "") return <span title={ip}>{client.name}</span>;
return (
<span title={ip} {...stylex.props(shared.learnedName)}>
{client.learned_name}
<span {...stylex.props(shared.learnedTag)}>learned</span>
<span title={ip} {...stylex.props(label.learned && shared.learnedName)}>
{label.text}
</span>
);
}
@@ -5,6 +5,9 @@ import { AuthProvider } from "@/auth/store";
import { createQueryClient } from "@/lib/queryClient";
import { createAppRouter } from "@/routes";
/** Wall clock at import; the upstream fixtures date their failures against it. */
const NOW_S = Math.floor(Date.now() / 1000);
const RESPONSES: Record<string, unknown> = {
"/api/stats?period=24h": {
period: "24h",
@@ -15,6 +18,7 @@ const RESPONSES: Record<string, unknown> = {
cached: 100,
clients: 7,
avg_response_time_us: 2345,
coverage: { complete: true, available_since: 0 },
},
"/api/stats/timeseries?period=24h": {
period: "24h",
@@ -26,6 +30,7 @@ const RESPONSES: Record<string, unknown> = {
{ ts: 1800, queries: 40, blocked: 0, cached: 0 },
{ ts: 3600, queries: 0, blocked: 0, cached: 0 },
],
coverage: { complete: true, available_since: 0 },
},
"/api/stats?period=1h": {
period: "1h",
@@ -36,6 +41,7 @@ const RESPONSES: Record<string, unknown> = {
cached: 0,
clients: 2,
avg_response_time_us: null,
coverage: { complete: true, available_since: 0 },
},
"/api/stats/timeseries?period=1h": {
period: "1h",
@@ -43,6 +49,7 @@ const RESPONSES: Record<string, unknown> = {
until: 3600,
bucket_seconds: 60,
buckets: [],
coverage: { complete: true, available_since: 0 },
},
"/api/health": {
status: "degraded",
@@ -58,32 +65,67 @@ const RESPONSES: Record<string, unknown> = {
writer_failed: false,
refreshes_gated: 0,
snapshot_generation: 3,
diagnostics: { state: "recording", active_warnings: 1, active_errors: 0 },
},
"/api/upstream/health": {
"/api/upstream/health?period=24h": {
period: "24h",
since: NOW_S - 86_400,
until: NOW_S,
available: 1,
total: 2,
complete: true,
upstreams: [
{
url: "https://dns.example/dns-query",
enabled: true,
available: false,
consecutive_failures: 4,
total_successes: 90,
total_failures: 10,
success_rate: 0.9,
last_error: "timeout",
period: {
attempts: 100,
successes: 90,
failures: 10,
success_rate: 0.9,
// 3h30m before the fixture's now, far from a unit boundary.
last_failure_at: NOW_S - 12_600,
last_failure_error: "timeout",
},
},
{
url: "udp://9.9.9.9:53",
enabled: true,
available: true,
consecutive_failures: 0,
total_successes: 100,
total_failures: 0,
success_rate: 1,
last_error: "",
period: {
attempts: 100,
successes: 100,
failures: 0,
success_rate: 1,
last_failure_at: null,
last_failure_error: null,
},
},
],
},
"/api/upstream/health?period=1h": {
period: "1h",
since: NOW_S - 3600,
until: NOW_S,
available: 1,
total: 2,
total: 1,
complete: true,
upstreams: [
{
url: "https://dns.example/dns-query",
enabled: true,
available: true,
period: {
attempts: 7,
successes: 6,
failures: 1,
success_rate: 6 / 7,
last_failure_at: NOW_S - 300,
last_failure_error: "timeout",
},
},
],
},
"/api/version": { version: "0.0.0-test", git_commit: "0000000", zig_version: "0.16.0", uptime_seconds: 1 },
};
@@ -142,7 +184,7 @@ test("dashboard renders stats, chart, disk card, upstream table and health banne
expect(screen.getByRole("img", { name: /queries over time/i })).toBeTruthy();
expect(screen.getByText("Blocked", { selector: "li" })).toBeTruthy();
expect(screen.getByText("Disk")).toBeTruthy();
expect(screen.getByText("Storage now")).toBeTruthy();
expect(screen.getByText("warn")).toBeTruthy();
expect(screen.getAllByText("400.0 MiB").length).toBeGreaterThan(0);
expect(screen.getByText("12.0 MiB")).toBeTruthy();
@@ -155,10 +197,31 @@ test("dashboard renders stats, chart, disk card, upstream table and health banne
expect(screen.getByText("https://dns.example/dns-query")).toBeTruthy();
expect(screen.getByText("90.0%")).toBeTruthy();
expect(screen.getByText("100.0%")).toBeTruthy();
expect(screen.getByText("timeout")).toBeTruthy();
expect(screen.getByText("1/2 available")).toBeTruthy();
});
test("live state is labeled on its own card, not by a section that disowns the picker", async () => {
renderDashboard();
await screen.findByRole("heading", { name: "Dashboard" });
expect(screen.getByText("Storage now")).toBeTruthy();
expect(screen.queryByRole("region", { name: "Right now" })).toBeNull();
expect(screen.queryByText("Right now")).toBeNull();
expect(screen.queryByText("Snapshot state; the period above does not apply.")).toBeNull();
});
test("the period picker rescopes the upstream table", async () => {
renderDashboard();
await screen.findByRole("heading", { name: "Dashboard" });
await screen.findByText("90.0%");
fireEvent.click(screen.getByRole("button", { name: "1h" }));
await screen.findByText("85.7%");
expect(screen.getByRole("columnheader", { name: "Selected period · 1h" })).toBeTruthy();
expect(screen.queryByText("90.0%")).toBeNull();
});
test("period picker refetches stats and shows the empty chart state", async () => {
renderDashboard();
await screen.findByRole("heading", { name: "Dashboard" });
@@ -173,7 +236,7 @@ test("period picker refetches stats and shows the empty chart state", async () =
});
test("one failing endpoint degrades its own widget on cold navigation", async () => {
failing.add("/api/upstream/health");
failing.add("/api/upstream/health?period=24h");
renderDashboard();
await screen.findByRole("heading", { name: "Dashboard" });
@@ -184,6 +247,6 @@ test("one failing endpoint degrades its own widget on cold navigation", async ()
expect(screen.getByText("1,000")).toBeTruthy();
expect(screen.getByRole("img", { name: /queries over time/i })).toBeTruthy();
expect(screen.getByText("Disk")).toBeTruthy();
expect(screen.getByText("Storage now")).toBeTruthy();
expect(screen.queryByText("https://dns.example/dns-query")).toBeNull();
});
@@ -3,6 +3,7 @@ import { keepPreviousData, useQuery } from "@tanstack/react-query";
import * as stylex from "@stylexjs/stylex";
import { healthQuery, statsQuery, timeseriesQuery, upstreamHealthQuery } from "@/lib/queries";
import type { Period } from "@/lib/types";
import CoverageNotice from "@/lib/CoverageNotice";
import InlineError from "@/lib/InlineError";
import { styles as shared } from "@/ui/styles";
import { colors } from "@/ui/tokens.stylex";
@@ -66,7 +67,7 @@ const styles = stylex.create({
"@media (prefers-color-scheme: dark)": "oklch(27.4% 0.006 286.033)",
},
},
/** The chart takes two thirds beside the disk card from `lg`, one column below. */
/** The chart takes two thirds beside the storage card from `lg`, one column below. */
panelGrid: {
display: "grid",
gap: "1rem",
@@ -123,7 +124,7 @@ export default function DashboardPage() {
const stats = useQuery({ ...statsQuery(period), placeholderData: keepPreviousData });
const timeseries = useQuery({ ...timeseriesQuery(period), placeholderData: keepPreviousData });
const health = useQuery(healthQuery());
const upstreamHealth = useQuery(upstreamHealthQuery());
const upstreamHealth = useQuery({ ...upstreamHealthQuery(period), placeholderData: keepPreviousData });
return (
<section {...stylex.props(styles.page)}>
@@ -142,6 +143,10 @@ export default function DashboardPage() {
<StatCards stats={stats.data} />
)}
{/* One notice for the period: the chart is judged against the same
aligned lower bound as the totals, so it would say the same thing. */}
{stats.data !== undefined && <CoverageNotice coverage={stats.data.coverage} />}
<div {...stylex.props(styles.panelGrid)}>
<section {...stylex.props(styles.panel)}>
<h2 {...stylex.props(styles.panelHeading)}>Queries over time</h2>
+3 -1
View File
@@ -74,8 +74,10 @@ function stateStyle(state: Health["disk"]["state"]) {
export default function DiskCard({ disk }: { disk: Health["disk"] }) {
return (
<section {...stylex.props(styles.card)}>
{/* Live state, unlike the ranged widgets around it; the title says so
rather than a section rule the picker would have to disown. */}
<h2 {...stylex.props(styles.heading)}>
Disk
Storage now
<span {...stylex.props(styles.badge, stateStyle(disk.state))}>{disk.state}</span>
</h2>
<dl {...stylex.props(styles.list)}>
@@ -0,0 +1,52 @@
import { render, screen, within } from "@testing-library/react";
import * as stylex from "@stylexjs/stylex";
import type { StatsTimeseries } from "@/lib/types";
import { styles as shared } from "@/ui/styles";
import TimeseriesChart from "./TimeseriesChart";
const SINCE = 1_700_000_000;
function timeseries(bucketCount: number): StatsTimeseries {
return {
period: "24h",
since: SINCE,
until: SINCE + bucketCount * 1800,
bucket_seconds: 1800,
coverage: { complete: true, available_since: SINCE },
buckets: Array.from({ length: bucketCount }, (_, i) => ({
ts: SINCE + i * 1800,
queries: i + 1,
blocked: 1,
cached: 1,
})),
};
}
/** The element wearing the shared hidden style, found by its compiled classes. */
function hiddenElement(container: HTMLElement): Element | null {
const classes = stylex.props(shared.srOnly).className?.split(" ").filter(Boolean) ?? [];
expect(classes.length).toBeGreaterThan(0);
return container.querySelector(classes.map((name) => `.${name}`).join(""));
}
test("the data table is the SVG's accessible equivalent", () => {
render(<TimeseriesChart data={timeseries(3)} />);
expect(screen.getByRole("img", { name: /queries over time/i })).toBeTruthy();
const table = screen.getByRole("table", { name: "Queries per time bucket" });
expect(within(table).getAllByRole("row").length).toBe(4);
});
/**
* `overflow` does not apply to a table box and `height` on one is a minimum, so
* the hidden style has to sit on a block container wrapping the table. Worn by
* the table itself it clips the paint but not the layout, and 48 invisible rows
* push the document's scroll height a screen past the app shell.
*/
test("the hidden data table is clipped by a block wrapper, not by the table itself", () => {
const { container } = render(<TimeseriesChart data={timeseries(48)} />);
const hidden = hiddenElement(container);
expect(hidden?.tagName).toBe("DIV");
expect(hidden?.querySelector("table")).not.toBeNull();
});
@@ -292,29 +292,31 @@ export default function TimeseriesChart({ data }: { data: StatsTimeseries }) {
</li>
))}
</ul>
<table {...stylex.props(shared.srOnly)}>
<caption>Queries per time bucket</caption>
<thead>
<tr>
<th scope="col">Time</th>
<th scope="col">Queries</th>
<th scope="col">Blocked</th>
<th scope="col">Cached</th>
<th scope="col">Other</th>
</tr>
</thead>
<tbody>
{layout.bars.map((bar) => (
<tr key={bar.bucket.ts}>
<th scope="row">{formatTime(bar.bucket.ts)}</th>
<td>{bar.bucket.queries}</td>
<td>{bar.bucket.blocked}</td>
<td>{bar.bucket.cached}</td>
<td>{bar.other}</td>
<div {...stylex.props(shared.srOnly)}>
<table>
<caption>Queries per time bucket</caption>
<thead>
<tr>
<th scope="col">Time</th>
<th scope="col">Queries</th>
<th scope="col">Blocked</th>
<th scope="col">Cached</th>
<th scope="col">Other</th>
</tr>
))}
</tbody>
</table>
</thead>
<tbody>
{layout.bars.map((bar) => (
<tr key={bar.bucket.ts}>
<th scope="row">{formatTime(bar.bucket.ts)}</th>
<td>{bar.bucket.queries}</td>
<td>{bar.bucket.blocked}</td>
<td>{bar.bucket.cached}</td>
<td>{bar.other}</td>
</tr>
))}
</tbody>
</table>
</div>
</div>
);
}
@@ -0,0 +1,165 @@
import { render, screen, within } from "@testing-library/react";
import type { UpstreamHealth, UpstreamHealthEntry, UpstreamPeriodStats } from "@/lib/types";
import UpstreamHealthTable from "./UpstreamHealthTable";
const NOW_S = 1_700_000_000;
const ZERO: UpstreamPeriodStats = {
attempts: 0,
successes: 0,
failures: 0,
success_rate: null,
last_failure_at: null,
last_failure_error: null,
};
function period(overrides: Partial<UpstreamPeriodStats> = {}): UpstreamPeriodStats {
return {
attempts: 100,
successes: 90,
failures: 10,
success_rate: 0.9,
last_failure_at: NOW_S - 12_600,
last_failure_error: "Timeout",
...overrides,
};
}
function entry(overrides: Partial<UpstreamHealthEntry> = {}): UpstreamHealthEntry {
return {
url: "https://dns.example/dns-query",
enabled: true,
available: true,
period: period(),
...overrides,
};
}
function renderTable(upstreams: UpstreamHealthEntry[], overrides: Partial<UpstreamHealth> = {}) {
const health: UpstreamHealth = {
period: "24h",
since: NOW_S - 86_400,
until: NOW_S,
available: upstreams.filter((upstream) => upstream.available).length,
total: upstreams.length,
complete: true,
upstreams,
...overrides,
};
render(<UpstreamHealthTable health={health} />);
}
function rowOf(url: string): HTMLElement {
const cell = screen.getByText(url);
const row = cell.closest("tr");
if (row === null) throw new Error(`no row for ${url}`);
return row;
}
test("the ranged columns sit under a header naming the selected period", () => {
renderTable([entry()]);
expect(screen.getByRole("columnheader", { name: "Selected period · 24h" })).toBeTruthy();
for (const name of ["Upstream", "Status now", "Attempts", "Failures", "Success rate"]) {
expect(screen.getByRole("columnheader", { name })).toBeTruthy();
}
// The unranged yes/no pair the ranged table replaced.
expect(screen.queryByRole("columnheader", { name: "Enabled" })).toBeNull();
expect(screen.queryByRole("columnheader", { name: "Available" })).toBeNull();
});
test("failure detail is the Diagnostics page's job; the card never shows it", () => {
renderTable([entry()]);
expect(screen.queryByRole("columnheader", { name: "Last failure" })).toBeNull();
expect(screen.queryByText(/Timeout/)).toBeNull();
expect(screen.queryByText(/ago$/)).toBeNull();
});
test("status now is one word from live state, not from the window", () => {
renderTable([
entry({ url: "https://a.example/dns-query" }),
entry({ url: "https://b.example/dns-query", available: false }),
entry({ url: "https://c.example/dns-query", enabled: false, available: false }),
]);
expect(within(rowOf("https://a.example/dns-query")).getByText("Available")).toBeTruthy();
expect(within(rowOf("https://b.example/dns-query")).getByText("Backing off")).toBeTruthy();
expect(within(rowOf("https://c.example/dns-query")).getByText("Disabled")).toBeTruthy();
});
test("a window with no attempts renders an em-dash and never a perfect rate", () => {
renderTable([entry({ period: ZERO })]);
const cells = within(rowOf("https://dns.example/dns-query")).getAllByRole("cell");
expect(cells.map((cell) => cell.textContent)).toEqual([
"https://dns.example/dns-query",
"Available",
"0",
"0",
"—",
]);
expect(screen.queryByText("100.0%")).toBeNull();
expect(screen.queryByText("0.0%")).toBeNull();
});
test("the card says so when every upstream was idle in the window", () => {
renderTable([entry({ url: "https://a.example/dns-query", period: ZERO }), entry({ period: ZERO })]);
expect(screen.getByText("No upstream attempts in this period.")).toBeTruthy();
});
test("one upstream with attempts keeps the idle message away", () => {
renderTable([entry({ url: "https://a.example/dns-query", period: ZERO }), entry()]);
expect(screen.queryByText("No upstream attempts in this period.")).toBeNull();
});
test("an incomplete window carries a note; a complete one claims nothing", () => {
renderTable([entry()], { complete: false });
expect(screen.getByText(/history incomplete/i)).toBeTruthy();
});
test("a complete window shows no completeness text at all", () => {
renderTable([entry()], { complete: true });
expect(screen.queryByText(/history incomplete/i)).toBeNull();
expect(screen.queryByText(/complete/i)).toBeNull();
});
test("an empty pool says so instead of drawing a table", () => {
renderTable([]);
expect(screen.getByText("No upstreams configured.")).toBeTruthy();
expect(screen.queryByRole("table")).toBeNull();
});
test("a rate a hair under perfect never rounds up to 100.0% while failures stand", () => {
// The real row that produced this: 12,698 attempts, 2 failures, 99.984%.
renderTable([
entry({
period: period({ attempts: 12_698, successes: 12_696, failures: 2, success_rate: 12_696 / 12_698 }),
}),
]);
expect(screen.queryByText("100.0%")).toBeNull();
expect(screen.getByText("99.9%")).toBeTruthy();
});
test("a rate a hair above nothing never rounds down to 0.0% while successes stand", () => {
renderTable([
entry({
period: period({ attempts: 12_698, successes: 2, failures: 12_696, success_rate: 2 / 12_698 }),
}),
]);
expect(screen.queryByText("0.0%")).toBeNull();
expect(screen.getByText("0.1%")).toBeTruthy();
});
test("a window with no failures at all still reads 100.0%", () => {
renderTable([entry({ period: period({ attempts: 500, successes: 500, failures: 0, success_rate: 1 }) })]);
expect(screen.getByText("100.0%")).toBeTruthy();
});
@@ -1,8 +1,10 @@
import * as stylex from "@stylexjs/stylex";
import type { UpstreamHealth } from "@/lib/types";
import type { UpstreamHealth, UpstreamHealthEntry, UpstreamPeriodStats } from "@/lib/types";
import { styles as shared } from "@/ui/styles";
import { colors } from "@/ui/tokens.stylex";
const numberFormat = new Intl.NumberFormat();
const styles = stylex.create({
card: {
borderRadius: "0.25rem",
@@ -34,6 +36,13 @@ const styles = stylex.create({
lineHeight: "1.25rem",
color: colors.textMuted,
},
note: {
paddingInline: "1rem",
paddingBlock: "0.5rem",
fontSize: "0.75rem",
lineHeight: "1rem",
color: colors.textMuted,
},
tableWrap: {
overflowX: "auto",
},
@@ -43,6 +52,24 @@ const styles = stylex.create({
fontSize: "0.875rem",
lineHeight: "1.25rem",
},
/**
* The two live columns are left outside the span: everything under it answers
* for the selected window, and nothing else on this card does.
*/
groupRow: {
fontSize: "0.75rem",
lineHeight: "1rem",
color: colors.textMuted,
},
groupHead: {
borderBottomWidth: 1,
borderBottomStyle: "solid",
borderBottomColor: colors.border,
paddingInline: "1rem",
paddingBottom: "0.25rem",
textAlign: "center",
fontWeight: 500,
},
headRow: {
borderBottomWidth: 1,
borderBottomStyle: "solid",
@@ -85,12 +112,37 @@ const styles = stylex.create({
},
});
function YesNo({ value, badValue }: { value: boolean; badValue: boolean }) {
const bad = value === badValue;
return <span {...stylex.props(bad && styles.bad)}>{value ? "yes" : "no"}</span>;
/** Live pool state in one word. Configuration first: a disabled upstream is not backing off. */
function statusNow(upstream: UpstreamHealthEntry): "Available" | "Backing off" | "Disabled" {
if (!upstream.enabled) return "Disabled";
return upstream.available ? "Available" : "Backing off";
}
/**
* `success_rate` is null exactly when the window holds no attempt, and that must
* not read as perfect reliability hence the em-dash rather than `100.0%`.
*
* One decimal place cannot hold 12,696 of 12,698: it rounds to `100.0%`, and the
* row then claims perfection beside a failure count of 2. Neither endpoint may
* be reached by rounding only by actually having no failure, or no success.
*/
function successRate(period: UpstreamPeriodStats): string {
if (period.success_rate === null) return "—";
const rounded = period.success_rate * 100;
if (rounded > 99.9 && period.failures > 0) return "99.9%";
if (rounded < 0.1 && period.successes > 0) return "0.1%";
return `${rounded.toFixed(1)}%`;
}
/**
* The dashboard answers availability only. Failure detail what failed, when,
* and how often is the Diagnostics page's job, so `last_failure_at` and
* `last_failure_error` are read there rather than repeated in this row.
*/
export default function UpstreamHealthTable({ health }: { health: UpstreamHealth }) {
const idle = health.upstreams.length > 0 && health.upstreams.every(({ period }) => period.attempts === 0);
return (
<section {...stylex.props(styles.card)}>
<h2 {...stylex.props(styles.heading)}>
@@ -105,15 +157,21 @@ export default function UpstreamHealthTable({ health }: { health: UpstreamHealth
<div {...stylex.props(styles.tableWrap)}>
<table {...stylex.props(styles.table)}>
<thead>
<tr {...stylex.props(styles.groupRow)}>
<td colSpan={2} />
<th scope="colgroup" colSpan={3} {...stylex.props(styles.groupHead)}>
Selected period · {health.period}
</th>
</tr>
<tr {...stylex.props(styles.headRow)}>
<th scope="col" {...stylex.props(styles.th)}>
URL
Upstream
</th>
<th scope="col" {...stylex.props(styles.th)}>
Enabled
Status now
</th>
<th scope="col" {...stylex.props(styles.th)}>
Available
<th scope="col" {...stylex.props(styles.th, styles.thRight)}>
Attempts
</th>
<th scope="col" {...stylex.props(styles.th, styles.thRight)}>
Failures
@@ -121,36 +179,48 @@ export default function UpstreamHealthTable({ health }: { health: UpstreamHealth
<th scope="col" {...stylex.props(styles.th, styles.thRight)}>
Success rate
</th>
<th scope="col" {...stylex.props(styles.th)}>
Last error
</th>
</tr>
</thead>
<tbody>
{health.upstreams.map((upstream) => (
<tr key={upstream.url} {...stylex.props(styles.row)}>
<td {...stylex.props(styles.cell, styles.small, shared.mono)}>{upstream.url}</td>
<td {...stylex.props(styles.cell)}>
<YesNo value={upstream.enabled} badValue={false} />
</td>
<td {...stylex.props(styles.cell)}>
<YesNo value={upstream.available} badValue={false} />
</td>
<td {...stylex.props(styles.cell, styles.cellRight, shared.tabularNums)}>
{upstream.total_failures}
</td>
<td {...stylex.props(styles.cell, styles.cellRight, shared.tabularNums)}>
{(upstream.success_rate * 100).toFixed(1)}%
</td>
<td {...stylex.props(styles.cell, styles.small, styles.muted)}>
{upstream.last_error || "—"}
</td>
</tr>
))}
{health.upstreams.map((upstream) => {
const status = statusNow(upstream);
return (
<tr key={upstream.url} {...stylex.props(styles.row)}>
<td {...stylex.props(styles.cell, styles.small, shared.mono)}>
{upstream.url}
</td>
<td {...stylex.props(styles.cell)}>
<span
{...stylex.props(
status === "Backing off" && styles.bad,
status === "Disabled" && styles.muted,
)}
>
{status}
</span>
</td>
<td {...stylex.props(styles.cell, styles.cellRight, shared.tabularNums)}>
{numberFormat.format(upstream.period.attempts)}
</td>
<td {...stylex.props(styles.cell, styles.cellRight, shared.tabularNums)}>
{numberFormat.format(upstream.period.failures)}
</td>
<td {...stylex.props(styles.cell, styles.cellRight, shared.tabularNums)}>
{successRate(upstream.period)}
</td>
</tr>
);
})}
</tbody>
</table>
</div>
)}
{idle && <p {...stylex.props(styles.note)}>No upstream attempts in this period.</p>}
{!health.complete && (
<p {...stylex.props(styles.note)}>
History incomplete: outcomes were dropped in this window, so these counts are a lower bound.
</p>
)}
</section>
);
}
@@ -0,0 +1,218 @@
import { cleanup, fireEvent, render, screen, waitFor, within } from "@testing-library/react";
import { QueryClientProvider } from "@tanstack/react-query";
import { RouterProvider, createMemoryHistory } from "@tanstack/react-router";
import { AuthProvider } from "@/auth/store";
import { createQueryClient } from "@/lib/queryClient";
import { createAppRouter } from "@/routes";
import { DIAGNOSTIC_CODES, type DiagnosticEvent } from "@/lib/types";
import { EVENT_COPY } from "./eventCopy";
const NOW_S = Math.floor(Date.now() / 1000);
function event(overrides: Partial<DiagnosticEvent> = {}): DiagnosticEvent {
return {
id: 42,
code: "blocklist.refresh",
component: "blocklist",
subject: "StevenBlack",
severity: "warning",
first_seen: NOW_S - 7200,
last_seen: NOW_S - 600,
occurrences: 4,
resolved_at: null,
detail: "download failed: ConnectionTimedOut",
...overrides,
};
}
/** A 204: what `DELETE /api/diagnostics/{id}` answers on a purge. */
const NO_CONTENT = Symbol("204");
let responses: Record<string, unknown>;
let requested: string[];
beforeEach(() => {
requested = [];
responses = {
"/api/version": { version: "0.0.0-test", git_commit: "0000000", zig_version: "0.16.0", uptime_seconds: 1 },
};
vi.stubGlobal(
"fetch",
vi.fn(async (input: RequestInfo | URL, init?: { method?: string }) => {
const url = String(input);
const method = init?.method ?? "GET";
const key = method === "GET" ? url : `${method} ${url}`;
requested.push(key);
const payload = responses[key];
if (payload === undefined)
return new Response(JSON.stringify({ error: "no such event" }), {
status: 404,
headers: { "content-type": "application/json" },
});
if (payload === NO_CONTENT) return new Response(null, { status: 204 });
return new Response(JSON.stringify(payload), {
status: 200,
headers: { "content-type": "application/json" },
});
}),
);
});
afterEach(() => {
vi.unstubAllGlobals();
});
/**
* `retry` is off in the failure test: the shared client backs 5xx off for
* seconds, which the render assertions would sit through for nothing.
*/
function renderDetail(id: number, { retry = true } = {}) {
const queryClient = createQueryClient();
if (!retry) {
const defaults = queryClient.getDefaultOptions();
queryClient.setDefaultOptions({ ...defaults, queries: { ...defaults.queries, retry: false } });
}
const router = createAppRouter(createMemoryHistory({ initialEntries: [`/diagnostics/${id}`] }), queryClient);
render(
<AuthProvider>
<QueryClientProvider client={queryClient}>
<RouterProvider router={router} />
</QueryClientProvider>
</AuthProvider>,
);
return router;
}
test("an open episode shows its facts, its copy and the error the server sent", async () => {
responses["/api/diagnostics/42"] = event();
renderDetail(42);
await screen.findByRole("heading", { name: "Blocklist source failed to update" });
expect(screen.getByText("Warning")).toBeTruthy();
expect(screen.getByText("StevenBlack")).toBeTruthy();
expect(screen.getByText("Active for 2h")).toBeTruthy();
expect(screen.getByText("Not yet — still failing")).toBeTruthy();
expect(screen.getByText("4")).toBeTruthy();
expect(screen.getByText("blocklist.refresh")).toBeTruthy();
expect(screen.getByText(EVENT_COPY["blocklist.refresh"].impact)).toBeTruthy();
expect(screen.getByText(EVENT_COPY["blocklist.refresh"].remediation)).toBeTruthy();
expect(screen.getByText("download failed: ConnectionTimedOut")).toBeTruthy();
expect(screen.getByRole("link", { name: "Go to Blocklists" }).getAttribute("href")).toBe("/blocklists");
});
test("a resolved episode states how long it lasted, not how long it has run", async () => {
responses["/api/diagnostics/7"] = event({ id: 7, resolved_at: NOW_S - 3600 });
renderDetail(7);
await screen.findByRole("heading", { name: "Blocklist source failed to update" });
expect(screen.getByText("Resolved after 1h")).toBeTruthy();
expect(screen.queryByText("Not yet — still failing")).toBeNull();
});
test("an open episode offers no purge", async () => {
responses["/api/diagnostics/42"] = event();
renderDetail(42);
await screen.findByRole("heading", { name: "Blocklist source failed to update" });
expect(screen.queryByRole("button", { name: "Purge" })).toBeNull();
});
test("purging a resolved episode asks first, then returns to the list", async () => {
responses["/api/diagnostics/7"] = event({ id: 7, resolved_at: NOW_S - 3600 });
responses["DELETE /api/diagnostics/7"] = NO_CONTENT;
responses["/api/diagnostics?state=active"] = { events: [], next_before: null, active: { warnings: 0, errors: 0 } };
responses["/api/diagnostics?state=resolved"] = {
events: [],
next_before: null,
active: { warnings: 0, errors: 0 },
};
const router = renderDetail(7);
await screen.findByRole("heading", { name: "Blocklist source failed to update" });
fireEvent.click(screen.getByRole("button", { name: "Purge" }));
const dialog = await screen.findByRole("alertdialog");
expect(dialog.textContent).toContain("Purge this resolved event? Its history is gone for good.");
fireEvent.click(within(dialog).getByRole("button", { name: "Cancel" }));
expect(requested).not.toContain("DELETE /api/diagnostics/7");
fireEvent.click(screen.getByRole("button", { name: "Purge" }));
fireEvent.click(within(await screen.findByRole("alertdialog")).getByRole("button", { name: "Purge" }));
await waitFor(() => expect(requested).toContain("DELETE /api/diagnostics/7"));
// The row it was showing no longer exists, so the page it navigates to is
// the list rather than a 404 of its own.
await waitFor(() => expect(router.state.location.pathname).toBe("/diagnostics"));
});
test("a refused purge stays on the event and shows why", async () => {
responses["/api/diagnostics/7"] = event({ id: 7, resolved_at: NOW_S - 3600 });
vi.stubGlobal(
"fetch",
vi.fn(async (input: RequestInfo | URL, init?: { method?: string }) => {
if (init?.method === "DELETE")
return new Response(JSON.stringify({ error: "the event is still active" }), {
status: 409,
headers: { "content-type": "application/json" },
});
return new Response(JSON.stringify(responses[String(input)] ?? {}), {
status: 200,
headers: { "content-type": "application/json" },
});
}),
);
const router = renderDetail(7, { retry: false });
await screen.findByRole("heading", { name: "Blocklist source failed to update" });
fireEvent.click(screen.getByRole("button", { name: "Purge" }));
fireEvent.click(within(await screen.findByRole("alertdialog")).getByRole("button", { name: "Purge" }));
const alert = await screen.findByRole("alert");
expect(alert.textContent).toContain("the event is still active");
expect(router.state.location.pathname).toBe("/diagnostics/7");
});
test("every code renders its own title, impact and remediation", async () => {
for (const [index, code] of DIAGNOSTIC_CODES.entries()) {
const id = 100 + index;
responses[`/api/diagnostics/${id}`] = event({ id, code, component: code.slice(0, code.indexOf(".")) });
renderDetail(id);
const copy = EVENT_COPY[code];
await screen.findByRole("heading", { name: copy.title });
expect(screen.getByText(copy.impact), code).toBeTruthy();
expect(screen.getByText(copy.remediation), code).toBeTruthy();
screen.getByText(code);
cleanup();
}
});
test("an event retention has removed shows the server's message, not an empty page", async () => {
renderDetail(999);
await screen.findByText("no such event");
expect(screen.getByRole("link", { name: "← All diagnostics" })).toBeTruthy();
});
test("an unavailable store reports the failure instead of loading forever", async () => {
responses["/api/diagnostics/42"] = event();
vi.stubGlobal(
"fetch",
vi.fn(async (input: RequestInfo | URL) =>
String(input).startsWith("/api/diagnostics/")
? new Response(JSON.stringify({ error: "store unavailable" }), {
status: 503,
headers: { "content-type": "application/json" },
})
: new Response(JSON.stringify(responses[String(input)] ?? {}), {
status: 200,
headers: { "content-type": "application/json" },
}),
),
);
renderDetail(42, { retry: false });
const alert = await screen.findByRole("alert");
expect(alert.textContent).toContain("The server is starting or degraded.");
expect(screen.queryByText("Loading event…")).toBeNull();
expect(screen.getByRole("link", { name: "← All diagnostics" })).toBeTruthy();
});
@@ -0,0 +1,226 @@
import { useState } from "react";
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
import { Link, useNavigate, useParams } from "@tanstack/react-router";
import * as stylex from "@stylexjs/stylex";
import InlineError from "@/lib/InlineError";
import { formatDuration, formatTime } from "@/lib/format";
import { diagnosticPurgeMutation, diagnosticQuery } from "@/lib/queries";
import ConfirmDialog from "@/ui/ConfirmDialog";
import { styles as shared } from "@/ui/styles";
import { colors } from "@/ui/tokens.stylex";
import SeverityBadge from "./SeverityBadge";
import { componentLabel, copyFor } from "./eventCopy";
const styles = stylex.create({
back: {
fontSize: "0.875rem",
lineHeight: "1.25rem",
color: colors.primaryOnSurface,
textDecorationLine: "none",
},
headingRow: {
marginTop: "0.5rem",
display: "flex",
alignItems: "center",
flexWrap: "wrap",
gap: "0.5rem",
},
heading: {
fontSize: "1.5rem",
lineHeight: "2rem",
fontWeight: 600,
},
purgeAction: {
marginInlineStart: "auto",
},
subject: {
marginTop: "0.25rem",
color: colors.textSecondary,
wordBreak: "break-all",
},
panel: {
marginTop: "1rem",
maxWidth: "48rem",
borderRadius: "0.25rem",
borderWidth: 1,
borderStyle: "solid",
borderColor: colors.border,
backgroundColor: colors.surfaceRaised,
padding: "1rem",
},
facts: {
display: "grid",
gap: "0.5rem 1rem",
gridTemplateColumns: {
default: "auto",
"@media (min-width: 640px)": "max-content 1fr",
},
margin: 0,
fontSize: "0.875rem",
lineHeight: "1.25rem",
},
term: {
color: colors.textMuted,
},
value: {
margin: 0,
},
sectionHeading: {
marginTop: "1.5rem",
fontSize: "1.125rem",
lineHeight: "1.75rem",
fontWeight: 600,
},
prose: {
marginTop: "0.5rem",
maxWidth: "48rem",
fontSize: "0.875rem",
lineHeight: "1.5rem",
},
detail: {
marginTop: "0.5rem",
maxWidth: "48rem",
overflowX: "auto",
borderRadius: "0.25rem",
borderWidth: 1,
borderStyle: "solid",
borderColor: colors.border,
padding: "0.75rem",
fontSize: "0.8125rem",
lineHeight: "1.25rem",
whiteSpace: "pre-wrap",
wordBreak: "break-all",
},
links: {
marginTop: "1rem",
fontSize: "0.875rem",
lineHeight: "1.25rem",
},
link: {
color: colors.primaryOnSurface,
},
loading: {
marginTop: "1rem",
color: colors.textMuted,
},
});
export default function DiagnosticDetailPage() {
const { id } = useParams({ from: "/shell/diagnostics/$id" });
const eventId = Number(id);
const { data, error, isPending, refetch } = useQuery(diagnosticQuery(eventId));
const navigate = useNavigate();
const queryClient = useQueryClient();
const purge = useMutation(diagnosticPurgeMutation(queryClient));
const [confirming, setConfirming] = useState(false);
function confirmPurge() {
setConfirming(false);
// The row this page is about is gone, so staying here would show the
// 404 the purge itself caused.
purge.mutate(eventId, { onSuccess: () => void navigate({ to: "/diagnostics" }) });
}
if (isPending) {
return (
<p {...stylex.props(styles.loading, shared.pulse)} role="status">
Loading event
</p>
);
}
if (data === undefined) {
return (
<section>
<Link to="/diagnostics" {...stylex.props(styles.back, shared.focusRing)}>
All diagnostics
</Link>
<InlineError error={error} onRetry={() => void refetch()} />
</section>
);
}
const copy = copyFor(data.code);
const resolvedAt = data.resolved_at;
const span = (resolvedAt ?? Math.floor(Date.now() / 1000)) - data.first_seen;
return (
<section>
<Link to="/diagnostics" {...stylex.props(styles.back, shared.focusRing)}>
All diagnostics
</Link>
<div {...stylex.props(styles.headingRow)}>
<h1 {...stylex.props(styles.heading)}>{copy.title}</h1>
<SeverityBadge severity={data.severity} />
{/* Only history can be purged: an open episode is the current state of the box. */}
{resolvedAt !== null && (
<button
type="button"
onClick={() => setConfirming(true)}
disabled={purge.isPending}
{...stylex.props(styles.purgeAction, shared.dangerLinkButton, shared.focusRing)}
>
Purge
</button>
)}
</div>
<p {...stylex.props(styles.subject)}>{data.subject}</p>
<InlineError error={purge.error} />
<div {...stylex.props(styles.panel)}>
<dl {...stylex.props(styles.facts)}>
<dt {...stylex.props(styles.term)}>State</dt>
<dd {...stylex.props(styles.value)}>
{resolvedAt === null
? `Active for ${formatDuration(span)}`
: `Resolved after ${formatDuration(span)}`}
</dd>
<dt {...stylex.props(styles.term)}>First seen</dt>
<dd {...stylex.props(styles.value)}>{formatTime(data.first_seen)}</dd>
<dt {...stylex.props(styles.term)}>Last seen</dt>
<dd {...stylex.props(styles.value)}>{formatTime(data.last_seen)}</dd>
<dt {...stylex.props(styles.term)}>Occurrences</dt>
<dd {...stylex.props(styles.value, shared.tabularNums)}>{data.occurrences}</dd>
<dt {...stylex.props(styles.term)}>Resolved</dt>
<dd {...stylex.props(styles.value)}>
{data.resolved_at === null ? "Not yet — still failing" : formatTime(data.resolved_at)}
</dd>
<dt {...stylex.props(styles.term)}>Component</dt>
<dd {...stylex.props(styles.value)}>{componentLabel(data.component)}</dd>
<dt {...stylex.props(styles.term)}>Code</dt>
<dd {...stylex.props(styles.value, shared.mono)}>{data.code}</dd>
</dl>
</div>
<h2 {...stylex.props(styles.sectionHeading)}>Impact</h2>
<p {...stylex.props(styles.prose)}>{copy.impact}</p>
<h2 {...stylex.props(styles.sectionHeading)}>What to do</h2>
<p {...stylex.props(styles.prose)}>{copy.remediation}</p>
<h2 {...stylex.props(styles.sectionHeading)}>Last error</h2>
{data.detail === "" ? (
<p {...stylex.props(styles.prose)}>The server recorded no error text for this event.</p>
) : (
<pre {...stylex.props(styles.detail, shared.mono)}>{data.detail}</pre>
)}
{copy.link !== undefined && (
<p {...stylex.props(styles.links)}>
<Link to={copy.link.to} {...stylex.props(styles.link, shared.focusRing)}>
Go to {copy.link.label}
</Link>
</p>
)}
<ConfirmDialog
isOpen={confirming}
title="Purge event"
message="Purge this resolved event? Its history is gone for good."
confirmLabel="Purge"
onConfirm={confirmPurge}
onCancel={() => setConfirming(false)}
/>
</section>
);
}
@@ -0,0 +1,374 @@
import { fireEvent, render, screen, waitFor, within } from "@testing-library/react";
import { QueryClientProvider } from "@tanstack/react-query";
import { RouterProvider, createMemoryHistory } from "@tanstack/react-router";
import { AuthProvider } from "@/auth/store";
import { createQueryClient } from "@/lib/queryClient";
import { createAppRouter } from "@/routes";
import type { DiagnosticEvent, DiagnosticsPage } from "@/lib/types";
// Ages are rendered against the wall clock, so the fixtures are anchored to it
// rather than to a frozen instant: faking time here would fight the query
// client's own timers for no gain.
const NOW_S = Math.floor(Date.now() / 1000);
function event(id: number, overrides: Partial<DiagnosticEvent> = {}): DiagnosticEvent {
return {
id,
code: "blocklist.refresh",
component: "blocklist",
subject: "StevenBlack",
severity: "warning",
first_seen: NOW_S - 3600,
last_seen: NOW_S - 300,
occurrences: 3,
resolved_at: null,
detail: "download failed: ConnectionTimedOut",
...overrides,
};
}
function page(events: DiagnosticEvent[], nextBefore: number | null = null): DiagnosticsPage {
return { events, next_before: nextBefore, active: { warnings: 1, errors: 1 } };
}
const ACTIVE = page([
event(42),
event(41, {
code: "upstream.exchange",
component: "upstream",
subject: "tls://dns.example:853",
severity: "error",
occurrences: 1,
}),
]);
const RESOLVED = page([
event(30, { code: "disk.space", component: "disk", subject: "data", resolved_at: NOW_S - 7200 }),
]);
/** A stubbed response that carries a non-200 status instead of a payload. */
class Failure {
constructor(
readonly status: number,
readonly body: unknown,
) {}
}
function fail(status: number, message: string): Failure {
return new Failure(status, { error: message });
}
/** A 204: what `DELETE /api/diagnostics/{id}` answers on a purge. */
const NO_CONTENT = Symbol("204");
let responses: Record<string, unknown>;
let requested: string[];
beforeEach(() => {
requested = [];
responses = {
"/api/version": { version: "0.0.0-test", git_commit: "0000000", zig_version: "0.16.0", uptime_seconds: 1 },
"/api/diagnostics?state=active": ACTIVE,
"/api/diagnostics?state=resolved": RESOLVED,
};
vi.stubGlobal(
"fetch",
vi.fn(async (input: RequestInfo | URL, init?: { method?: string }) => {
const url = String(input);
const method = init?.method ?? "GET";
// Reads stay keyed by url alone, so the assertions below read as the
// request line they are; writes carry their method.
const key = method === "GET" ? url : `${method} ${url}`;
requested.push(key);
const payload = responses[key];
if (payload === undefined) return new Response(JSON.stringify({ error: "not stubbed" }), { status: 404 });
if (payload === NO_CONTENT) return new Response(null, { status: 204 });
if (payload instanceof Failure) {
return new Response(JSON.stringify(payload.body), {
status: payload.status,
headers: { "content-type": "application/json" },
});
}
return new Response(JSON.stringify(payload), {
status: 200,
headers: { "content-type": "application/json" },
});
}),
);
});
afterEach(() => {
vi.unstubAllGlobals();
});
/**
* `retry` is off in the failure tests: the shared client backs 5xx off for
* seconds, which the render assertions would sit through for nothing.
*/
function renderRoute(path = "/diagnostics", { retry = true } = {}) {
const queryClient = createQueryClient();
if (!retry) {
const defaults = queryClient.getDefaultOptions();
queryClient.setDefaultOptions({ ...defaults, queries: { ...defaults.queries, retry: false } });
}
const router = createAppRouter(createMemoryHistory({ initialEntries: [path] }), queryClient);
render(
<AuthProvider>
<QueryClientProvider client={queryClient}>
<RouterProvider router={router} />
</QueryClientProvider>
</AuthProvider>,
);
return router;
}
/** A RAC Select names its trigger with the current value and then the label. */
function trigger(label: string): HTMLElement {
return screen.getByRole("button", { name: new RegExp(`${label}$`) });
}
async function pick(label: string, option: string) {
fireEvent.click(trigger(label));
fireEvent.click(await screen.findByRole("option", { name: option }));
await waitFor(() => expect(screen.queryByRole("listbox")).toBeNull());
}
test("active episodes come first, each with its title, subject, age and count", async () => {
renderRoute();
await screen.findByRole("heading", { name: "Diagnostics" });
const active = screen.getByText("Blocklist source failed to update").closest("li")!;
expect(within(active).getByText("Warning")).toBeTruthy();
expect(within(active).getByText("StevenBlack")).toBeTruthy();
expect(within(active).getByText(/Active for 1h · 3 occurrences/)).toBeTruthy();
const failing = screen.getByText("Upstream failing").closest("li")!;
expect(within(failing).getByText("Error")).toBeTruthy();
expect(within(failing).getByText(/1 occurrence(?!s)/)).toBeTruthy();
// The resolved history is a separate section, below the active list.
const table = within(screen.getByRole("table"));
expect(table.getByText("Disk space low")).toBeTruthy();
expect(screen.getByText(/Showing 1 resolved entry — end of history/)).toBeTruthy();
});
test("nothing open reads as good news, not as a broken page", async () => {
responses["/api/diagnostics?state=active"] = page([]);
renderRoute();
await screen.findByRole("heading", { name: "Diagnostics" });
const healthy = await screen.findByText("No active operational issues.");
expect(healthy.getAttribute("role")).toBe("status");
// Quiet: no alert anywhere on the page, and no empty table standing in.
expect(screen.queryByRole("alert")).toBeNull();
});
test("a filter lands in the url and refetches both sections through it", async () => {
responses["/api/diagnostics?severity=error&state=active"] = page([
event(41, { code: "upstream.exchange", component: "upstream", severity: "error" }),
]);
responses["/api/diagnostics?severity=error&state=resolved"] = page([]);
const router = renderRoute();
await screen.findByRole("heading", { name: "Diagnostics" });
await pick("Severity", "Errors");
await waitFor(() => expect(router.state.location.search).toEqual({ severity: "error" }));
await waitFor(() => expect(screen.queryByText("Blocklist source failed to update")).toBeNull());
expect(requested).toContain("/api/diagnostics?severity=error&state=active");
expect(requested).toContain("/api/diagnostics?severity=error&state=resolved");
});
test("the state filter hides the section it excludes", async () => {
const router = renderRoute();
await screen.findByRole("heading", { name: "Diagnostics" });
await pick("Show", "Active only");
await waitFor(() => expect(router.state.location.search).toEqual({ state: "active" }));
expect(screen.queryByRole("heading", { name: "Resolved" })).toBeNull();
expect(screen.getByRole("heading", { name: "Active" })).toBeTruthy();
});
test("a url written by hand starts on the filters it names", async () => {
responses["/api/diagnostics?component=disk&state=resolved"] = RESOLVED;
renderRoute("/diagnostics?state=resolved&component=disk");
await screen.findByRole("heading", { name: "Diagnostics" });
await screen.findByText("Disk space low");
expect(screen.queryByRole("heading", { name: "Active" })).toBeNull();
expect(requested).toContain("/api/diagnostics?component=disk&state=resolved");
});
test("load more appends the next page of resolved history", async () => {
responses["/api/diagnostics?state=resolved"] = page(
[event(30, { code: "disk.space", component: "disk", subject: "data", resolved_at: NOW_S - 7200 })],
30,
);
responses["/api/diagnostics?state=resolved&before=30"] = page([
event(12, {
code: "certificate.reload",
component: "certificate",
subject: "doh",
resolved_at: NOW_S - 90_000,
}),
]);
renderRoute();
await screen.findByText("Disk space low");
fireEvent.click(screen.getByRole("button", { name: "Load more" }));
await screen.findByText("TLS certificate reload failed");
expect(screen.getByText(/Showing 2 resolved entries — end of history/)).toBeTruthy();
});
test("an unavailable store reports the failure instead of loading forever", async () => {
responses["/api/diagnostics?state=active"] = fail(503, "store unavailable");
renderRoute("/diagnostics", { retry: false });
await screen.findByRole("heading", { name: "Diagnostics" });
const alert = await screen.findByRole("alert");
expect(alert.textContent).toContain("The server is starting or degraded.");
expect(screen.queryByText("Loading diagnostics…")).toBeNull();
// The resolved section answered, so it still renders its own history.
expect(screen.getByText("Disk space low")).toBeTruthy();
});
test("a failed history query reports the failure and retries on demand", async () => {
responses["/api/diagnostics?state=resolved"] = fail(500, "diagnostics store read failed");
renderRoute("/diagnostics", { retry: false });
await screen.findByRole("heading", { name: "Diagnostics" });
const alert = await screen.findByRole("alert");
expect(alert.textContent).toContain("diagnostics store read failed");
expect(screen.queryByText("Loading history…")).toBeNull();
responses["/api/diagnostics?state=resolved"] = RESOLVED;
fireEvent.click(screen.getByRole("button", { name: "Retry" }));
await screen.findByText("Disk space low");
expect(screen.queryByRole("alert")).toBeNull();
});
test("only the resolved history offers a purge", async () => {
renderRoute();
await screen.findByRole("heading", { name: "Diagnostics" });
// An episode still failing is the state of the box, not history: no purge
// affordance anywhere on its card.
const active = screen.getByText("Blocklist source failed to update").closest("li")!;
expect(within(active).queryByRole("button", { name: "Purge" })).toBeNull();
const row = screen.getByText("Disk space low").closest("tr")!;
expect(within(row).getByRole("button", { name: "Purge" })).toBeTruthy();
expect(screen.getByRole("button", { name: "Purge all resolved" })).toBeTruthy();
});
test("with no resolved history there is nothing to purge in bulk", async () => {
responses["/api/diagnostics?state=resolved"] = page([]);
renderRoute();
await screen.findByRole("heading", { name: "Diagnostics" });
await screen.findByText("Nothing has failed and recovered in the retained window.");
expect(screen.queryByRole("button", { name: "Purge all resolved" })).toBeNull();
});
test("purging one row asks first, then sends the DELETE and refetches the lists", async () => {
responses["DELETE /api/diagnostics/30"] = NO_CONTENT;
renderRoute();
await screen.findByText("Disk space low");
fireEvent.click(within(screen.getByText("Disk space low").closest("tr")!).getByRole("button", { name: "Purge" }));
const dialog = await screen.findByRole("alertdialog");
expect(dialog.textContent).toContain("Purge this resolved event? Its history is gone for good.");
fireEvent.click(within(dialog).getByRole("button", { name: "Cancel" }));
expect(requested).not.toContain("DELETE /api/diagnostics/30");
fireEvent.click(within(screen.getByText("Disk space low").closest("tr")!).getByRole("button", { name: "Purge" }));
fireEvent.click(within(await screen.findByRole("alertdialog")).getByRole("button", { name: "Purge" }));
await waitFor(() => expect(requested).toContain("DELETE /api/diagnostics/30"));
// The invalidation covers both sections: the page the row left and the
// active list, whose `active` counts come from the same table.
await waitFor(() =>
expect(requested.filter((url) => url === "/api/diagnostics?state=resolved").length).toBeGreaterThan(1),
);
await waitFor(() =>
expect(requested.filter((url) => url === "/api/diagnostics?state=active").length).toBeGreaterThan(1),
);
});
test("purging the whole history asks first and sends one DELETE", async () => {
responses["DELETE /api/diagnostics"] = { purged: 1 };
renderRoute();
await screen.findByText("Disk space low");
fireEvent.click(screen.getByRole("button", { name: "Purge all resolved" }));
const dialog = await screen.findByRole("alertdialog");
expect(dialog.textContent).toContain("Purge all resolved events? Active events are kept.");
// What the server will answer once the purge has landed; the refetch the
// mutation triggers is what has to pick it up.
responses["/api/diagnostics?state=resolved"] = page([]);
fireEvent.click(within(dialog).getByRole("button", { name: "Purge all" }));
await waitFor(() => expect(requested).toContain("DELETE /api/diagnostics"));
await waitFor(() => expect(screen.queryByText("Disk space low")).toBeNull());
expect(screen.getByText("Blocklist source failed to update")).toBeTruthy();
});
test("a refused purge reports the server's reason and keeps the row", async () => {
responses["DELETE /api/diagnostics/30"] = fail(409, "the event is still active; it can be purged once it resolves");
renderRoute("/diagnostics", { retry: false });
await screen.findByText("Disk space low");
fireEvent.click(within(screen.getByText("Disk space low").closest("tr")!).getByRole("button", { name: "Purge" }));
fireEvent.click(within(await screen.findByRole("alertdialog")).getByRole("button", { name: "Purge" }));
const alert = await screen.findByRole("alert");
expect(alert.textContent).toContain("the event is still active");
expect(screen.getByText("Disk space low")).toBeTruthy();
});
test("an episode links to its own detail page", async () => {
responses["/api/diagnostics/42"] = event(42);
renderRoute();
const link = await screen.findByRole("link", { name: "Blocklist source failed to update" });
expect(link.getAttribute("href")).toBe("/diagnostics/42");
});
test("an absolute window reaches both requests and is stated on the page", async () => {
const bounded = "since=1699999700&until=1700000300";
responses[`/api/diagnostics?${bounded}&state=active`] = ACTIVE;
responses[`/api/diagnostics?${bounded}&state=resolved`] = RESOLVED;
renderRoute(`/diagnostics?${bounded}`);
await screen.findByRole("heading", { name: "Active" });
expect(requested).toContain(`/api/diagnostics?${bounded}&state=active`);
expect(requested).toContain(`/api/diagnostics?${bounded}&state=resolved`);
// An empty section inside a five-minute window means something different
// from an empty section over the whole history, so the page has to say so.
expect(screen.getByText(/Showing events that overlap/)).toBeTruthy();
});
test("clearing the range drops both bounds from the url", async () => {
const bounded = "since=1699999700&until=1700000300";
responses[`/api/diagnostics?${bounded}&state=active`] = ACTIVE;
responses[`/api/diagnostics?${bounded}&state=resolved`] = RESOLVED;
const router = renderRoute(`/diagnostics?${bounded}`);
fireEvent.click(await screen.findByRole("button", { name: "Clear the time range" }));
await waitFor(() => {
expect(router.state.location.search).not.toContain("since");
});
expect(router.state.location.search).not.toContain("until");
});
test("a bound that is not a whole second is dropped, leaving the page unbounded", async () => {
renderRoute("/diagnostics?since=1.5&until=Infinity");
await screen.findByRole("heading", { name: "Active" });
expect(requested).toContain("/api/diagnostics?state=active");
expect(screen.queryByText(/Showing events that overlap/)).toBeNull();
});
@@ -0,0 +1,533 @@
import { useState } from "react";
import {
useInfiniteQuery,
useMutation,
useQueryClient,
type InfiniteData,
type UseInfiniteQueryResult,
} from "@tanstack/react-query";
import { Link, useNavigate, useSearch } from "@tanstack/react-router";
import * as stylex from "@stylexjs/stylex";
import * as api from "@/lib/api";
import InlineError from "@/lib/InlineError";
import { formatDuration, formatTime } from "@/lib/format";
import { diagnosticPurgeMutation, diagnosticsInfiniteQuery, diagnosticsPurgeResolvedMutation } from "@/lib/queries";
import type { DiagnosticEvent, DiagnosticSeverity, DiagnosticState, DiagnosticsPage as Page } from "@/lib/types";
import ConfirmDialog from "@/ui/ConfirmDialog";
import Select from "@/ui/Select";
import { styles as shared } from "@/ui/styles";
import { colors } from "@/ui/tokens.stylex";
import SeverityBadge from "./SeverityBadge";
import { diagnosticsFilterOf } from "./filter";
import { DIAGNOSTIC_COMPONENTS, componentLabel, copyFor } from "./eventCopy";
const DARK = "@media (prefers-color-scheme: dark)";
const STATE_OPTIONS = [
{ value: "all", label: "Active and resolved" },
{ value: "active", label: "Active only" },
{ value: "resolved", label: "Resolved only" },
];
const SEVERITY_OPTIONS = [
{ value: "any", label: "Any severity" },
{ value: "warning", label: "Warnings" },
{ value: "error", label: "Errors" },
];
const COMPONENT_OPTIONS = [
{ value: "any", label: "All components" },
...DIAGNOSTIC_COMPONENTS.map((component) => ({ value: component, label: componentLabel(component) })),
];
const styles = stylex.create({
heading: {
fontSize: "1.5rem",
lineHeight: "2rem",
fontWeight: 600,
},
intro: {
marginTop: "0.25rem",
fontSize: "0.875rem",
lineHeight: "1.25rem",
color: colors.textMuted,
maxWidth: "48rem",
},
filterGrid: {
marginTop: "1rem",
display: "grid",
gap: "0.75rem",
gridTemplateColumns: {
default: "repeat(1, minmax(0, 1fr))",
"@media (min-width: 640px)": "repeat(3, minmax(0, 1fr))",
},
maxWidth: "48rem",
},
sectionHeading: {
marginTop: "1.5rem",
fontSize: "1.125rem",
lineHeight: "1.75rem",
fontWeight: 600,
},
sectionHeadingRow: {
display: "flex",
alignItems: "baseline",
flexWrap: "wrap",
justifyContent: "space-between",
gap: "0.75rem",
},
/**
* Nothing open is the normal state of a working install, so it gets one
* quiet muted line no border, no icon, no alert role. A panel here would
* read as a broken page rather than as good news.
*/
healthy: {
marginTop: "0.5rem",
fontSize: "0.875rem",
lineHeight: "1.25rem",
color: colors.textSecondary,
},
empty: {
marginTop: "0.5rem",
fontSize: "0.875rem",
lineHeight: "1.25rem",
color: colors.textMuted,
},
cardList: {
marginTop: "0.75rem",
display: "flex",
flexDirection: "column",
gap: "0.5rem",
listStyleType: "none",
padding: 0,
},
card: {
borderRadius: "0.25rem",
borderWidth: 1,
borderStyle: "solid",
borderColor: colors.border,
backgroundColor: colors.surfaceRaised,
paddingInline: "0.75rem",
paddingBlock: "0.625rem",
},
cardTop: {
display: "flex",
alignItems: "baseline",
flexWrap: "wrap",
gap: "0.5rem",
},
cardTitle: {
fontWeight: 500,
color: colors.primaryOnSurface,
textDecorationLine: "none",
},
subject: {
fontSize: "0.875rem",
lineHeight: "1.25rem",
color: colors.textSecondary,
wordBreak: "break-all",
},
meta: {
marginTop: "0.25rem",
fontSize: "0.75rem",
lineHeight: "1rem",
color: colors.textMuted,
},
rangeNotice: {
marginTop: "0.75rem",
borderRadius: "0.25rem",
borderWidth: 1,
borderStyle: "solid",
borderColor: colors.border,
backgroundColor: colors.surfaceHover,
paddingInline: "0.75rem",
paddingBlock: "0.5rem",
fontSize: "0.875rem",
lineHeight: "1.25rem",
color: colors.textSecondary,
},
tableWrap: {
marginTop: "0.75rem",
overflowX: "auto",
borderRadius: "0.25rem",
borderWidth: 1,
borderStyle: "solid",
borderColor: colors.border,
},
table: {
width: "100%",
fontSize: "0.875rem",
lineHeight: "1.25rem",
},
head: {
backgroundColor: { default: "oklch(98.5% 0 none)", [DARK]: "oklch(21% 0.006 285.885)" },
textAlign: "left",
},
th: {
paddingInline: "0.75rem",
paddingBlock: "0.5rem",
fontWeight: 500,
color: colors.textSecondary,
whiteSpace: "nowrap",
},
row: {
borderTopWidth: { default: 1, ":first-child": 0 },
borderTopStyle: "solid",
borderTopColor: colors.border,
},
cell: {
paddingInline: "0.75rem",
paddingBlock: "0.5rem",
},
nowrap: {
whiteSpace: "nowrap",
},
rowLink: {
color: colors.primaryOnSurface,
textDecorationLine: "none",
},
footer: {
marginTop: "0.75rem",
display: "flex",
alignItems: "center",
gap: "0.75rem",
},
note: {
fontSize: "0.875rem",
lineHeight: "1.25rem",
color: colors.textMuted,
},
moreError: {
marginTop: "0.5rem",
fontSize: "0.875rem",
lineHeight: "1.25rem",
color: colors.dangerText,
},
});
type Section = UseInfiniteQueryResult<InfiniteData<Page, unknown>, Error>;
/** The two enum filters, narrowed from the picker's string rather than cast. */
function asState(value: string): DiagnosticState | undefined {
return value === "active" || value === "resolved" ? value : undefined;
}
function asSeverity(value: string): DiagnosticSeverity | undefined {
return value === "warning" || value === "error" ? value : undefined;
}
function errorMessage(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
function occurrenceText(count: number): string {
return `${count} ${count === 1 ? "occurrence" : "occurrences"}`;
}
function rowsOf(section: Section): DiagnosticEvent[] {
return (section.data?.pages ?? []).flatMap((page) => page.events);
}
/**
* The cursor comes from the newest page on screen, not from `hasNextPage`:
* while placeholder data stands in for a filter change the query state is
* empty, and the button would flash away and back.
*/
function hasMore(section: Section): boolean {
const pages = section.data?.pages ?? [];
const last = pages[pages.length - 1];
return last !== undefined && last.next_before !== null;
}
function MoreButton({ section }: { section: Section }) {
const more = hasMore(section);
// A 401 is already redirecting via the cache-level handleUnauthorized.
const isUnauthorized = section.error instanceof api.ApiError && section.error.status === 401;
const failed = section.isFetchNextPageError && !isUnauthorized ? errorMessage(section.error) : null;
if (!more && failed === null) return null;
return (
<>
<div {...stylex.props(styles.footer)}>
{more && (
<button
type="button"
onClick={() => {
if (section.isFetchingNextPage || section.isPlaceholderData) return;
void section.fetchNextPage();
}}
disabled={section.isFetchingNextPage || section.isPlaceholderData}
{...stylex.props(shared.button, shared.focusRing)}
>
{section.isFetchingNextPage ? "Loading…" : "Load more"}
</button>
)}
</div>
{failed !== null && (
<p role="alert" {...stylex.props(styles.moreError)}>
Failed to load more: {failed}
</p>
)}
</>
);
}
/**
* The window the page is bounded to, whenever it is bounded.
*
* A link from a query detail arrives with an absolute five-minute window, and
* an empty Active section inside it means something very different from an
* empty Active section over the whole history. The page has to say which it is
* showing, and offer the way out of it.
*/
function RangeNotice({ since, until }: { since?: number; until?: number }) {
const navigate = useNavigate({ from: "/diagnostics" });
if (since === undefined && until === undefined) return null;
const from = since === undefined ? "the start of the history" : formatTime(since);
const to = until === undefined ? "now" : formatTime(until);
return (
<p role="status" {...stylex.props(styles.rangeNotice)}>
Showing events that overlap {from} to {to}.{" "}
<button
type="button"
onClick={() => void navigate({ search: (prev) => ({ ...prev, since: undefined, until: undefined }) })}
{...stylex.props(shared.linkButton, shared.focusRing)}
>
Clear the time range
</button>
</p>
);
}
function ActiveCard({ event, now }: { event: DiagnosticEvent; now: number }) {
const copy = copyFor(event.code);
return (
<li {...stylex.props(styles.card)}>
<div {...stylex.props(styles.cardTop)}>
<SeverityBadge severity={event.severity} />
<Link
to="/diagnostics/$id"
params={{ id: String(event.id) }}
{...stylex.props(styles.cardTitle, shared.focusRing)}
>
{copy.title}
</Link>
<span {...stylex.props(styles.subject)}>{event.subject}</span>
</div>
<p {...stylex.props(styles.meta)}>
Active for {formatDuration(now - event.first_seen)} · {occurrenceText(event.occurrences)} · last failure{" "}
{formatTime(event.last_seen)}
</p>
</li>
);
}
function HistoryRow({ event, onPurge, busy }: { event: DiagnosticEvent; onPurge: () => void; busy: boolean }) {
const copy = copyFor(event.code);
return (
<tr {...stylex.props(styles.row)}>
<td {...stylex.props(styles.cell)}>
<SeverityBadge severity={event.severity} />
</td>
<td {...stylex.props(styles.cell)}>
<Link
to="/diagnostics/$id"
params={{ id: String(event.id) }}
{...stylex.props(styles.rowLink, shared.focusRing)}
>
{copy.title}
</Link>
</td>
<td {...stylex.props(styles.cell)}>{event.subject}</td>
<td {...stylex.props(styles.cell, styles.nowrap)}>{formatTime(event.first_seen)}</td>
<td {...stylex.props(styles.cell, styles.nowrap)}>
{event.resolved_at === null ? "—" : formatTime(event.resolved_at)}
</td>
<td {...stylex.props(styles.cell, styles.nowrap, shared.tabularNums)}>{event.occurrences}</td>
<td {...stylex.props(styles.cell, styles.nowrap)}>
<button
type="button"
onClick={onPurge}
disabled={busy}
{...stylex.props(shared.dangerLinkButton, shared.focusRing)}
>
Purge
</button>
</td>
</tr>
);
}
export default function DiagnosticsPage() {
const search = useSearch({ from: "/shell/diagnostics" });
const navigate = useNavigate({ from: "/diagnostics" });
const state = search.state ?? "all";
const base = diagnosticsFilterOf(search);
const active = useInfiniteQuery(diagnosticsInfiniteQuery({ ...base, state: "active" }, state !== "resolved"));
const history = useInfiniteQuery(diagnosticsInfiniteQuery({ ...base, state: "resolved" }, state !== "active"));
const queryClient = useQueryClient();
const purgeOne = useMutation(diagnosticPurgeMutation(queryClient));
const purgeAll = useMutation(diagnosticsPurgeResolvedMutation(queryClient));
// `null` is "no dialog"; the id is which row it is about, and `"all"` the
// whole history. One piece of state, so the two dialogs cannot both be open.
const [pendingPurge, setPendingPurge] = useState<number | "all" | null>(null);
const activeRows = rowsOf(active);
const historyRows = rowsOf(history);
const now = Math.floor(Date.now() / 1000);
const purging = purgeOne.isPending || purgeAll.isPending;
function setSearch(patch: Partial<typeof search>) {
void navigate({ search: (prev) => ({ ...prev, ...patch }) });
}
function confirmPurge() {
if (pendingPurge === null) return;
if (pendingPurge === "all") {
purgeAll.mutate();
} else {
purgeOne.mutate(pendingPurge);
}
setPendingPurge(null);
}
return (
<section>
<h1 {...stylex.props(styles.heading)}>Diagnostics</h1>
<p {...stylex.props(styles.intro)}>
Operational failures, one entry per subject that failed. An entry opens on the first failure, counts
repeats, and closes when the subject recovers.
</p>
<RangeNotice since={search.since} until={search.until} />
<div {...stylex.props(styles.filterGrid)}>
<Select
variant="compactField"
label="Show"
value={state}
onChange={(value) => setSearch({ state: asState(value) })}
options={STATE_OPTIONS}
/>
<Select
variant="compactField"
label="Severity"
value={search.severity ?? "any"}
onChange={(value) => setSearch({ severity: asSeverity(value) })}
options={SEVERITY_OPTIONS}
/>
<Select
variant="compactField"
label="Component"
value={search.component ?? "any"}
onChange={(value) => setSearch({ component: value === "any" ? undefined : value })}
options={COMPONENT_OPTIONS}
/>
</div>
{state !== "resolved" && (
<>
<h2 {...stylex.props(styles.sectionHeading)}>Active</h2>
{active.status === "error" ? (
<InlineError error={active.error} onRetry={() => void active.refetch()} />
) : active.data === undefined ? (
<p {...stylex.props(styles.empty, shared.pulse)} role="status">
Loading diagnostics
</p>
) : activeRows.length === 0 ? (
<p {...stylex.props(styles.healthy)} role="status">
No active operational issues.
</p>
) : (
<>
<ul {...stylex.props(styles.cardList)}>
{activeRows.map((event) => (
<ActiveCard key={event.id} event={event} now={now} />
))}
</ul>
<MoreButton section={active} />
</>
)}
</>
)}
{state !== "active" && (
<>
<div {...stylex.props(styles.sectionHeading, styles.sectionHeadingRow)}>
<h2>Resolved</h2>
{historyRows.length > 0 && (
<button
type="button"
onClick={() => setPendingPurge("all")}
disabled={purging}
{...stylex.props(shared.dangerLinkButton, shared.focusRing)}
>
Purge all resolved
</button>
)}
</div>
{history.status === "error" ? (
<InlineError error={history.error} onRetry={() => void history.refetch()} />
) : history.data === undefined ? (
<p {...stylex.props(styles.empty, shared.pulse)} role="status">
Loading history
</p>
) : historyRows.length === 0 ? (
<p {...stylex.props(styles.empty)}>Nothing has failed and recovered in the retained window.</p>
) : (
<>
<div {...stylex.props(styles.tableWrap)}>
<table {...stylex.props(styles.table)}>
<thead {...stylex.props(styles.head)}>
<tr>
<th {...stylex.props(styles.th)}>Severity</th>
<th {...stylex.props(styles.th)}>Event</th>
<th {...stylex.props(styles.th)}>Subject</th>
<th {...stylex.props(styles.th)}>Started</th>
<th {...stylex.props(styles.th)}>Resolved</th>
<th {...stylex.props(styles.th)}>Occurrences</th>
<th {...stylex.props(styles.th)}>
<span {...stylex.props(shared.srOnly)}>Actions</span>
</th>
</tr>
</thead>
<tbody>
{historyRows.map((event) => (
<HistoryRow
key={event.id}
event={event}
busy={purging}
onPurge={() => setPendingPurge(event.id)}
/>
))}
</tbody>
</table>
</div>
<p {...stylex.props(styles.footer, styles.note)}>
Showing {historyRows.length} resolved {historyRows.length === 1 ? "entry" : "entries"}
{hasMore(history) ? "" : " — end of history"}
</p>
<MoreButton section={history} />
</>
)}
<InlineError error={purgeOne.error ?? purgeAll.error} />
</>
)}
<ConfirmDialog
isOpen={pendingPurge !== null}
title={pendingPurge === "all" ? "Purge resolved history" : "Purge event"}
message={
pendingPurge === "all"
? "Purge all resolved events? Active events are kept."
: "Purge this resolved event? Its history is gone for good."
}
confirmLabel={pendingPurge === "all" ? "Purge all" : "Purge"}
onConfirm={confirmPurge}
onCancel={() => setPendingPurge(null)}
/>
</section>
);
}
@@ -0,0 +1,42 @@
/**
* The severity chip both diagnostics views carry. The word is the affordance
* colour alone would leave the severity unreadable to a screen reader and to
* anyone who does not separate the amber from the red.
*/
import * as stylex from "@stylexjs/stylex";
import type { DiagnosticSeverity } from "@/lib/types";
import { colors } from "@/ui/tokens.stylex";
const styles = stylex.create({
badge: {
display: "inline-block",
borderRadius: "0.25rem",
borderWidth: 1,
borderStyle: "solid",
paddingInline: "0.375rem",
paddingBlock: "0.125rem",
fontSize: "0.75rem",
lineHeight: "1rem",
fontWeight: 500,
whiteSpace: "nowrap",
},
warning: {
borderColor: colors.warnBorder,
backgroundColor: colors.warnSurface,
color: colors.warnText,
},
error: {
borderColor: colors.dangerBorder,
backgroundColor: colors.dangerSurface,
color: colors.dangerText,
},
});
export default function SeverityBadge({ severity }: { severity: DiagnosticSeverity }) {
return (
<span {...stylex.props(styles.badge, severity === "error" ? styles.error : styles.warning)}>
{severity === "error" ? "Error" : "Warning"}
</span>
);
}
@@ -0,0 +1,50 @@
import { DIAGNOSTIC_CODES } from "@/lib/types";
import { DIAGNOSTIC_COMPONENTS, EVENT_COPY, componentLabel, copyFor } from "./eventCopy";
test("the enum holds the fifteen codes the store defines", () => {
expect(DIAGNOSTIC_CODES).toHaveLength(15);
expect(new Set(DIAGNOSTIC_CODES).size).toBe(15);
});
test("every code has copy, and no copy belongs to a code that does not exist", () => {
for (const code of DIAGNOSTIC_CODES) {
const copy = EVENT_COPY[code];
expect(copy, code).toBeDefined();
expect(copy.title.length, code).toBeGreaterThan(0);
expect(copy.impact.length, code).toBeGreaterThan(0);
expect(copy.remediation.length, code).toBeGreaterThan(0);
}
expect(Object.keys(EVENT_COPY).sort()).toEqual([...DIAGNOSTIC_CODES].sort());
});
test("titles are distinct, so two open episodes never read as the same event", () => {
const titles = DIAGNOSTIC_CODES.map((code) => EVENT_COPY[code].title);
expect(new Set(titles).size).toBe(titles.length);
});
test("a code this build has never heard of falls back to the code itself", () => {
// The server is the authority on the enum; a newer one can send a sixteenth.
const copy = copyFor("nonsense.code" as (typeof DIAGNOSTIC_CODES)[number]);
expect(copy.title).toBe("nonsense.code");
expect(copy.remediation.length).toBeGreaterThan(0);
});
test("the component options are the code prefixes, deduplicated and in enum order", () => {
expect(DIAGNOSTIC_COMPONENTS).toEqual([
"disk",
"blocklist",
"certificate",
"query_log",
"upstream_history",
"upstream",
"client_names",
"clients",
"listener",
"configuration",
]);
});
test("component labels read as prose without inventing a name", () => {
expect(componentLabel("query_log")).toBe("Query log");
expect(componentLabel("disk")).toBe("Disk");
});
+155
View File
@@ -0,0 +1,155 @@
/**
* What each event code means to the operator, in three fixed fields: what the
* episode is (`title`), what it costs while it stays open (`impact`), and what
* to do about it (`remediation`). The server sends a code and an error string;
* every word of explanation the page shows comes from here.
*
* The record is exhaustive over `DiagnosticCode` by type, and a test walks
* `DIAGNOSTIC_CODES` to prove it at runtime too. A sixteenth code added to the
* enum fails `tsc` here before it can reach the page as a bare dotted string.
*
* `link` points at the configuration surface that governs the failure. Those
* are today's routes; the navigation restructure re-points them.
*/
import { DIAGNOSTIC_CODES, type DiagnosticCode } from "@/lib/types";
/** The literal paths keep `link.to` assignable to a typed router `Link`. */
export type CopyLinkPath = "/settings" | "/blocklists" | "/upstreams" | "/clients";
export interface EventCopy {
title: string;
impact: string;
remediation: string;
link?: { to: CopyLinkPath; label: string };
}
/**
* The copy for a code, with a floor under it. `tsc` proves the record covers
* the union, but a server one release ahead can send a code this build has
* never heard of; showing the raw code beats rendering "undefined".
*/
export function copyFor(code: DiagnosticCode): EventCopy {
return (
EVENT_COPY[code] ?? {
title: code,
impact: "This build has no description for this event code.",
remediation: "The error detail below is the whole of what the server reported.",
}
);
}
const SETTINGS = { to: "/settings", label: "Settings" } as const;
const BLOCKLISTS = { to: "/blocklists", label: "Blocklists" } as const;
const UPSTREAMS = { to: "/upstreams", label: "Upstreams" } as const;
const CLIENTS = { to: "/clients", label: "Clients" } as const;
/**
* The component filter's options, derived from the codes rather than listed
* again: the server matches `component` against the part of `code` before the
* dot, so any list written by hand here could drift from the enum.
*/
export const DIAGNOSTIC_COMPONENTS: readonly string[] = [
...new Set(DIAGNOSTIC_CODES.map((code) => code.slice(0, code.indexOf(".")))),
];
/** `query_log` → "Query log". Display only; the filter sends the raw component. */
export function componentLabel(component: string): string {
const spaced = component.replaceAll("_", " ");
return spaced.charAt(0).toUpperCase() + spaced.slice(1);
}
export const EVENT_COPY: Record<DiagnosticCode, EventCopy> = {
"disk.space": {
title: "Disk space low",
impact: "Below the critical threshold nxdns stops blocklist updates and query log flushes to protect the disk.",
remediation: "Free space on the data volume, or lower the retention window so the query log holds fewer days.",
link: SETTINGS,
},
"disk.probe": {
title: "Disk usage probe failed",
impact: "Free space is unknown, so the low-disk guard cannot act until a probe succeeds.",
remediation: "Check that the data and log directories exist and that the service user can read them.",
link: SETTINGS,
},
"blocklist.refresh": {
title: "Blocklist source failed to update",
impact: "The source keeps serving its last good snapshot, so blocking continues but the list ages.",
remediation: "Check the source url and the machine's internet access, then update the lists again.",
link: BLOCKLISTS,
},
"blocklist.snapshot": {
title: "Filter snapshot failed to publish",
impact: "The resolver keeps the snapshot it already holds; blocklist edits do not take effect until one publishes.",
remediation: "Check free disk space and the data directory's permissions, then update the lists again.",
link: BLOCKLISTS,
},
"blocklist.storage": {
title: "Blocklist storage operation failed",
impact: "Cached list files or their database rows are out of step; a later pass can redownload what is missing.",
remediation: "Check free disk space and the data directory's permissions.",
link: BLOCKLISTS,
},
"certificate.reload": {
title: "TLS certificate reload failed",
impact: "The endpoint keeps serving the certificate it already loaded, which expires on its own schedule.",
remediation:
"Check the certificate and key paths, and that renewal writes both files the service user can read.",
link: SETTINGS,
},
"query_log.write": {
title: "Query log write failed",
impact: "Queries are resolved and answered as usual, but they are not being recorded.",
remediation: "Check free disk space and the log database's permissions, then restart nxdns.",
link: SETTINGS,
},
"query_log.maintenance": {
title: "Query log maintenance failed",
impact: "Old rows are not being trimmed, so the log database grows past its retention window.",
remediation: "Check free disk space; the next maintenance pass retries on its own.",
link: SETTINGS,
},
"query_log.recreated": {
title: "Query log recreated",
impact: "The old log database was unreadable and was moved aside; the history it held is not in the new one.",
remediation: "Keep or delete the aside file named below. Nothing else is required — logging is running.",
link: SETTINGS,
},
"upstream_history.write": {
title: "Upstream history write failed",
impact: "Resolution is unaffected; the per-upstream success and failure aggregates lose the affected window.",
remediation: "Check free disk space and the configuration database's permissions.",
link: UPSTREAMS,
},
"upstream.exchange": {
title: "Upstream failing",
impact: "Queries fall through to the remaining upstreams; answers are slower while this one backs off.",
remediation: "Check the upstream's reachability and its TLS name. Remove it if it stays down.",
link: UPSTREAMS,
},
"client_names.storage": {
title: "Client name storage failed",
impact: "Learned reverse-DNS names are not persisted, so clients can show as bare addresses after a restart.",
remediation: "Check free disk space and the configuration database's permissions.",
link: CLIENTS,
},
"clients.storage": {
title: "Client record storage failed",
impact: "New clients may not appear in the list and stale ones may not be pruned.",
remediation: "Check free disk space and the configuration database's permissions.",
link: CLIENTS,
},
"listener.start": {
title: "Encrypted DNS listener failed to start",
impact: "That endpoint is not accepting queries. Plain DNS on port 53 is unaffected.",
remediation:
"Check the bind address, the port, and the certificate paths, then restart nxdns. The episode closes on a clean start.",
link: SETTINGS,
},
"configuration.load": {
title: "Configuration problem at startup",
impact: "The setting named below was rejected or replaced by its default for this run.",
remediation: "Correct the setting and restart nxdns. The episode closes on a clean start.",
link: SETTINGS,
},
};
+28
View File
@@ -0,0 +1,28 @@
/**
* The Diagnostics search parameters and the API filter they build.
*
* The route, the page and the two infinite queries all have to agree on what
* the URL asked for the page renders the same window the loader prefetched
* so the projection lives in one place rather than being spelled out at each.
*/
import type { DiagnosticSeverity, DiagnosticState, DiagnosticsFilter } from "@/lib/types";
export interface DiagnosticsSearch {
state?: DiagnosticState;
severity?: DiagnosticSeverity;
component?: string;
/** Unix seconds, inclusive. An episode qualifies when its interval overlaps. */
since?: number;
until?: number;
}
/** Field by field, so an unset filter is an absent key rather than `undefined`. */
export function diagnosticsFilterOf(search: DiagnosticsSearch): DiagnosticsFilter {
const filter: DiagnosticsFilter = {};
if (search.severity !== undefined) filter.severity = search.severity;
if (search.component !== undefined) filter.component = search.component;
if (search.since !== undefined) filter.since = search.since;
if (search.until !== undefined) filter.until = search.until;
return filter;
}
@@ -1,197 +0,0 @@
import { act, fireEvent, render, screen, within } from "@testing-library/react";
import { QueryClientProvider } from "@tanstack/react-query";
import { createQueryClient } from "@/lib/queryClient";
import type { Client, LiveQueryEvent } from "@/lib/types";
import { FakeEventSource } from "./fakeEventSource";
import LiveLogPage from "./LiveLogPage";
function client(ip: string, name: string, learnedName: string): Client {
return {
id: Number(ip.split(".").pop()),
ip,
name,
learned_name: learnedName,
group_id: 1,
group: "default",
hand_edited: name !== "",
first_seen: 1_700_000_000,
last_seen: 1_700_000_100,
};
}
const CLIENTS: Client[] = [
client("192.0.2.10", "Kitchen Pi", "pi.lan"),
client("192.0.2.11", "", "laptop.lan"),
client("192.0.2.12", "", ""),
];
beforeEach(() => {
vi.stubGlobal(
"fetch",
vi.fn(async (input: RequestInfo | URL) => {
if (String(input) !== "/api/clients") {
return new Response(JSON.stringify({ error: "not stubbed" }), { status: 404 });
}
return new Response(JSON.stringify({ clients: CLIENTS }), {
status: 200,
headers: { "content-type": "application/json" },
});
}),
);
});
afterEach(() => {
vi.unstubAllGlobals();
});
function frame(ts: number, domain: string, overrides: Partial<LiveQueryEvent> = {}): { data: string } {
const payload: LiveQueryEvent = {
ts,
domain,
client_ip: "192.0.2.10",
qtype: 1,
blocked: false,
block_reason: "",
response_time_us: 500,
cache_hit: true,
upstream: "",
...overrides,
};
return { data: JSON.stringify(payload) };
}
function renderPage() {
const sources: FakeEventSource[] = [];
const createEventSource = (url: string) => {
const es = new FakeEventSource(url);
sources.push(es);
return es;
};
render(
<QueryClientProvider client={createQueryClient()}>
<LiveLogPage createEventSource={createEventSource} />
</QueryClientProvider>,
);
return sources;
}
test("streams rows, flags blocked ones, and freezes the display", () => {
const sources = renderPage();
expect(screen.getByText("Connecting…")).toBeTruthy();
act(() => sources[0]!.emit("open"));
expect(screen.getByRole("status", { name: "Live" })).toBeTruthy();
expect(screen.getByText("Waiting for queries…")).toBeTruthy();
act(() => {
sources[0]!.emit("query", frame(1000, "ok.example"));
sources[0]!.emit(
"query",
frame(1001, "ads.example", { blocked: true, block_reason: "blocklist:stevenblack", qtype: 28 }),
);
});
expect(screen.getByText("ok.example")).toBeTruthy();
expect(screen.getByText("Blocked")).toBeTruthy();
expect(screen.getByText("blocklist:stevenblack")).toBeTruthy();
expect(screen.getByText("AAAA")).toBeTruthy();
// StyleX compiles to opaque class names, so the check is structural: a blocked
// row carries every class a plain row does, plus the ones the flag adds.
const blockedRow = screen.getByText("ads.example").closest("tr");
const plainRow = screen.getByText("ok.example").closest("tr");
const blockedClasses = new Set(blockedRow?.className.split(" "));
const plainClasses = plainRow?.className.split(" ") ?? [];
expect(plainClasses.every((name) => blockedClasses.has(name))).toBe(true);
expect(blockedClasses.size).toBeGreaterThan(plainClasses.length);
const freeze = screen.getByRole("button", { name: "Freeze" });
fireEvent.click(freeze);
expect(freeze.getAttribute("aria-pressed")).toBe("true");
act(() => sources[0]!.emit("query", frame(1002, "later.example")));
expect(screen.queryByText("later.example")).toBeNull();
expect(screen.getByText(/3 in buffer/)).toBeTruthy();
fireEvent.click(screen.getByRole("button", { name: "Resume" }));
expect(screen.getByText("later.example")).toBeTruthy();
});
test("resolves each row's client to its display name, keeping the IP as the tooltip", async () => {
const sources = renderPage();
act(() => sources[0]!.emit("open"));
act(() => {
sources[0]!.emit("query", frame(1000, "named.example", { client_ip: "192.0.2.10" }));
sources[0]!.emit("query", frame(1001, "learned.example", { client_ip: "192.0.2.11" }));
sources[0]!.emit("query", frame(1002, "nameless.example", { client_ip: "192.0.2.12" }));
sources[0]!.emit("query", frame(1003, "stranger.example", { client_ip: "192.0.2.99" }));
});
// A hand-typed name wins outright; the learned name never surfaces for it.
const named = await screen.findByText("Kitchen Pi");
expect(named.getAttribute("title")).toBe("192.0.2.10");
expect(screen.queryByText("pi.lan")).toBeNull();
// The cell holds the learned name followed by the tag, so the match is on
// the containing span rather than on a bare text node.
const learned = screen.getByText(
(content, element) => element?.tagName === "SPAN" && content.startsWith("laptop.lan"),
);
expect(learned.getAttribute("title")).toBe("192.0.2.11");
// The affordance is text, not colour, so a screen reader announces it too.
expect(within(learned).getByText("learned")).toBeTruthy();
// A known client with neither name, and a client the loaded list has never
// seen, both fall back to the bare address with no tooltip standing in.
const nameless = screen.getByText("192.0.2.12");
expect(nameless.getAttribute("title")).toBeNull();
const stranger = screen.getByText("192.0.2.99");
expect(stranger.getAttribute("title")).toBeNull();
expect(screen.getByText("stranger.example").closest("tr")?.textContent).toContain("192.0.2.99");
});
test("rows stream in as bare IPs while the client list is still loading", async () => {
let releaseClients: () => void = () => {};
vi.stubGlobal(
"fetch",
vi.fn(
(input: RequestInfo | URL) =>
new Promise<Response>((resolve) => {
if (String(input) !== "/api/clients") {
resolve(new Response(JSON.stringify({ error: "not stubbed" }), { status: 404 }));
return;
}
releaseClients = () =>
resolve(
new Response(JSON.stringify({ clients: CLIENTS }), {
status: 200,
headers: { "content-type": "application/json" },
}),
);
}),
),
);
const sources = renderPage();
act(() => sources[0]!.emit("open"));
act(() => sources[0]!.emit("query", frame(1000, "named.example", { client_ip: "192.0.2.10" })));
expect(screen.getByText("192.0.2.10")).toBeTruthy();
expect(screen.queryByText("Kitchen Pi")).toBeNull();
releaseClients();
expect(await screen.findByText("Kitchen Pi")).toBeTruthy();
});
test("repeated connection failures show the viewer-cap state with a retry button", () => {
const sources = renderPage();
act(() => {
sources[0]!.emit("error");
sources[0]!.emit("error");
sources[0]!.emit("error");
});
expect(screen.getByRole("alert").textContent).toContain("too many live viewers");
fireEvent.click(screen.getByRole("button", { name: "Retry" }));
expect(sources).toHaveLength(2);
expect(screen.getByText("Connecting…")).toBeTruthy();
});
-101
View File
@@ -1,101 +0,0 @@
import type { LiveQueryEvent, QueryRow } from "@/lib/types";
import { RING_CAPACITY, mergeGap, pushRow, type LiveRow } from "./ringBuffer";
function event(ts: number, domain: string, overrides: Partial<LiveQueryEvent> = {}): LiveQueryEvent {
return {
ts,
domain,
client_ip: "192.0.2.10",
qtype: 1,
blocked: false,
block_reason: "",
response_time_us: 500,
cache_hit: false,
upstream: "udp://9.9.9.9:53",
...overrides,
};
}
function liveRow(key: number, ts: number, domain: string, overrides: Partial<LiveQueryEvent> = {}): LiveRow {
return { ...event(ts, domain, overrides), key };
}
function fetchedRow(id: number, ts: number, domain: string, overrides: Partial<LiveQueryEvent> = {}): QueryRow {
return { id, ...event(ts, domain, overrides) };
}
function counter(start = 100): () => number {
let n = start;
return () => ++n;
}
describe("pushRow", () => {
test("prepends newest-first", () => {
let rows: LiveRow[] = [];
rows = pushRow(rows, liveRow(1, 10, "a.example"));
rows = pushRow(rows, liveRow(2, 11, "b.example"));
expect(rows.map((r) => r.domain)).toEqual(["b.example", "a.example"]);
});
test("drops the oldest beyond capacity", () => {
let rows: LiveRow[] = [];
for (let i = 0; i < 5; i++) rows = pushRow(rows, liveRow(i, i, `d${i}.example`), 3);
expect(rows).toHaveLength(3);
expect(rows.map((r) => r.key)).toEqual([4, 3, 2]);
});
test("default capacity is 500", () => {
let rows: LiveRow[] = [];
for (let i = 0; i < RING_CAPACITY + 10; i++) rows = pushRow(rows, liveRow(i, i, "x.example"));
expect(rows).toHaveLength(RING_CAPACITY);
});
});
describe("mergeGap", () => {
test("skips rows already in the buffer and counts only new ones", () => {
const buffer = [liveRow(2, 100, "seen.example"), liveRow(1, 99, "old.example")];
const fetched = [
fetchedRow(30, 102, "gap2.example"),
fetchedRow(29, 101, "gap1.example"),
fetchedRow(28, 100, "seen.example"),
];
const { rows, missed } = mergeGap(buffer, fetched, counter());
expect(missed).toBe(2);
expect(rows.map((r) => r.domain)).toEqual(["gap2.example", "gap1.example", "seen.example", "old.example"]);
});
test("no additions returns the buffer unchanged with missed 0", () => {
const buffer = [liveRow(1, 100, "seen.example")];
const { rows, missed } = mergeGap(buffer, [fetchedRow(5, 100, "seen.example")], counter());
expect(missed).toBe(0);
expect(rows).toBe(buffer);
});
test("rows differing only in qtype are not deduplicated", () => {
const buffer = [liveRow(1, 100, "dual.example", { qtype: 1 })];
const fetched = [fetchedRow(5, 100, "dual.example", { qtype: 28 })];
const { missed } = mergeGap(buffer, fetched, counter());
expect(missed).toBe(1);
});
test("assigns fresh keys from the counter and drops the id", () => {
const { rows } = mergeGap([], [fetchedRow(77, 100, "gap.example")], counter(200));
expect(rows[0]?.key).toBe(201);
expect("id" in (rows[0] ?? {})).toBe(false);
});
test("result is capped at capacity, keeping the newest", () => {
const buffer = [liveRow(3, 300, "live.example")];
const fetched = [fetchedRow(2, 302, "g2.example"), fetchedRow(1, 301, "g1.example")];
const { rows, missed } = mergeGap(buffer, fetched, counter(), 2);
expect(missed).toBe(2);
expect(rows.map((r) => r.domain)).toEqual(["g2.example", "g1.example"]);
});
test("merged rows stay sorted newest-first by ts", () => {
const buffer = [liveRow(4, 105, "after-reopen.example"), liveRow(3, 100, "before.example")];
const fetched = [fetchedRow(9, 103, "gap.example")];
const { rows } = mergeGap(buffer, fetched, counter());
expect(rows.map((r) => r.ts)).toEqual([105, 103, 100]);
});
});
-53
View File
@@ -1,53 +0,0 @@
import type { LiveQueryEvent, QueryRow } from "@/lib/types";
/** A live stream row; `key` is a client-side monotonic counter (SSE frames carry no id). */
export interface LiveRow extends LiveQueryEvent {
key: number;
}
export const RING_CAPACITY = 500;
/** Prepend `row` (rows are newest-first) and drop the oldest beyond `capacity`. */
export function pushRow(rows: LiveRow[], row: LiveRow, capacity: number = RING_CAPACITY): LiveRow[] {
const next = [row, ...rows];
return next.length > capacity ? next.slice(0, capacity) : next;
}
// `since` on GET /api/queries is inclusive, so the re-sync fetch returns the
// last-seen row(s) again; live rows have no id, so identity is this tuple.
function signature(row: LiveQueryEvent): string {
return `${row.ts}|${row.domain}|${row.client_ip}|${row.qtype ?? -1}|${row.blocked}|${row.upstream}`;
}
/**
* Merge rows fetched for a reconnect gap (newest-first, from GET /api/queries)
* into the buffer. Rows already present are skipped; `missed` counts what was
* actually added. The result stays newest-first (stable sort by ts) and capped.
*/
export function mergeGap(
rows: LiveRow[],
fetched: QueryRow[],
nextKey: () => number,
capacity: number = RING_CAPACITY,
): { rows: LiveRow[]; missed: number } {
const seen = new Set(rows.map(signature));
const added: LiveRow[] = [];
for (const row of fetched) {
const event: LiveQueryEvent = {
ts: row.ts,
domain: row.domain,
client_ip: row.client_ip,
qtype: row.qtype,
blocked: row.blocked,
block_reason: row.block_reason,
response_time_us: row.response_time_us,
cache_hit: row.cache_hit,
upstream: row.upstream,
};
if (seen.has(signature(event))) continue;
added.push({ ...event, key: nextKey() });
}
if (added.length === 0) return { rows, missed: 0 };
const merged = [...added, ...rows].sort((a, b) => b.ts - a.ts).slice(0, capacity);
return { rows: merged, missed: added.length };
}
@@ -1,95 +0,0 @@
import { fireEvent, render, screen } from "@testing-library/react";
import { QueryClientProvider } from "@tanstack/react-query";
import { RouterProvider, createMemoryHistory } from "@tanstack/react-router";
import { AuthProvider } from "@/auth/store";
import { createQueryClient } from "@/lib/queryClient";
import { createAppRouter } from "@/routes";
import type { LookupResult } from "@/lib/types";
const BLOCKED: LookupResult = {
domain: "ads.example",
group_id: 1,
local_records: false,
forward_zone: null,
blocked: true,
reason: "blocklist_domain",
matched: "ads.example",
source_url: "https://lists.test/a",
safe_search_rewrite: null,
};
let fetchMock: ReturnType<typeof createFetchMock>;
function json(payload: unknown, status = 200): Response {
return new Response(JSON.stringify(payload), { status, headers: { "content-type": "application/json" } });
}
function createFetchMock() {
return vi.fn(async (input: RequestInfo | URL) => {
const url = String(input);
if (url === "/api/groups") {
return json({
groups: [
{ id: 1, name: "default", safe_search: false },
{ id: 2, name: "kids", safe_search: true },
],
});
}
if (url === "/api/lookup?domain=ads.example&group_id=1") return json(BLOCKED);
return json({ error: "not stubbed" }, 404);
});
}
beforeEach(() => {
fetchMock = createFetchMock();
vi.stubGlobal("fetch", fetchMock);
});
afterEach(() => {
vi.unstubAllGlobals();
});
function renderPage() {
const queryClient = createQueryClient();
const router = createAppRouter(createMemoryHistory({ initialEntries: ["/lookup"] }), queryClient);
render(
<AuthProvider>
<QueryClientProvider client={queryClient}>
<RouterProvider router={router} />
</QueryClientProvider>
</AuthProvider>,
);
}
function lookupCalls(): string[] {
return fetchMock.mock.calls.map(([input]) => String(input)).filter((url) => url.startsWith("/api/lookup"));
}
test("fetches nothing until submit, then renders the blocked verdict", async () => {
renderPage();
await screen.findByRole("heading", { name: "Lookup" });
await screen.findByLabelText("Group");
expect(lookupCalls()).toEqual([]);
fireEvent.change(screen.getByLabelText("Domain"), { target: { value: "ads.example" } });
expect(lookupCalls()).toEqual([]);
fireEvent.click(screen.getByRole("button", { name: "Look up" }));
await screen.findByRole("heading", { name: "Blocked" });
expect(lookupCalls()).toEqual(["/api/lookup?domain=ads.example&group_id=1"]);
expect(screen.getByText("blocklist_domain")).toBeTruthy();
const link = screen.getByRole("link", { name: "https://lists.test/a" }) as HTMLAnchorElement;
expect(link.href).toBe("https://lists.test/a");
expect(screen.getByText("Queries for this name get a blocked response.")).toBeTruthy();
});
test("defaults the group select to the default group (id 1)", async () => {
renderPage();
// A RAC Select names its trigger with the current value and then the label, so
// the selected group's name is the only thing the trigger shows.
const trigger = await screen.findByRole("button", { name: /Group$/ });
expect(trigger.textContent).toContain("default");
});
@@ -1,366 +0,0 @@
import { act, fireEvent, render, screen, waitFor, within } from "@testing-library/react";
import { QueryClientProvider } from "@tanstack/react-query";
import { createQueryClient } from "@/lib/queryClient";
import type { Client, QueriesPage, QueryRow } from "@/lib/types";
import QueryLogPage from "./QueryLogPage";
function client(id: number, ip: string, name: string, learnedName: string): Client {
return {
id,
ip,
name,
learned_name: learnedName,
group_id: 1,
group: "default",
hand_edited: name !== "",
first_seen: 1_700_000_000,
last_seen: 1_700_000_100,
};
}
const CLIENTS: Client[] = [
client(1, "192.0.2.10", "Kitchen Pi", "pi.lan"),
client(2, "192.0.2.11", "", "laptop.lan"),
client(3, "192.0.2.12", "", ""),
];
function row(id: number, domain: string, overrides: Partial<QueryRow> = {}): QueryRow {
return {
id,
ts: 1_700_000_000 + id,
domain,
client_ip: "192.0.2.10",
qtype: 1,
blocked: false,
block_reason: "",
response_time_us: 1234,
cache_hit: false,
upstream: "udp://9.9.9.9:53",
...overrides,
};
}
const PAGES: Record<string, QueriesPage> = {
"/api/queries": {
queries: [
row(20, "first.example", { qtype: 65, cache_hit: true, upstream: "" }),
row(19, "ads.example", {
blocked: true,
block_reason: "blocklist:stevenblack",
response_time_us: null,
cache_hit: null,
}),
],
next_before: 19,
},
"/api/queries?before=19": {
queries: [row(5, "older.example")],
next_before: null,
},
"/api/queries?domain=ads": {
queries: [row(19, "ads.example", { blocked: true, block_reason: "blocklist:stevenblack" })],
next_before: null,
},
};
beforeEach(() => {
vi.stubGlobal(
"fetch",
vi.fn(async (input: RequestInfo | URL) => {
const url = String(input);
if (url === "/api/clients") return json({ clients: CLIENTS });
const payload = PAGES[url];
if (payload === undefined) return new Response(JSON.stringify({ error: "not stubbed" }), { status: 404 });
return new Response(JSON.stringify(payload), {
status: 200,
headers: { "content-type": "application/json" },
});
}),
);
});
afterEach(() => {
vi.unstubAllGlobals();
});
function renderPage() {
const client = createQueryClient();
render(
<QueryClientProvider client={client}>
<QueryLogPage />
</QueryClientProvider>,
);
return client;
}
function json(payload: unknown): Response {
return new Response(JSON.stringify(payload), { status: 200, headers: { "content-type": "application/json" } });
}
test("renders the first page with type names, blocked badge, and formatted cells", async () => {
renderPage();
await screen.findByText("first.example");
expect(screen.getByText("HTTPS")).toBeTruthy();
expect(screen.getByText("A")).toBeTruthy();
expect(screen.getByText("Blocked")).toBeTruthy();
expect(screen.getByText("blocklist:stevenblack")).toBeTruthy();
expect(screen.getByText("1.2 ms")).toBeTruthy();
expect(screen.getByText("hit")).toBeTruthy();
expect(screen.getByText("udp://9.9.9.9:53")).toBeTruthy();
expect(screen.getByText(/Showing 2 queries/)).toBeTruthy();
});
test("resolves each row's client to its display name, keeping the IP as the tooltip", async () => {
vi.stubGlobal(
"fetch",
vi.fn(async (input: RequestInfo | URL) => {
const url = String(input);
if (url === "/api/clients") return json({ clients: CLIENTS });
if (url !== "/api/queries") return new Response(JSON.stringify({ error: "not stubbed" }), { status: 404 });
return json({
queries: [
row(20, "named.example", { client_ip: "192.0.2.10" }),
row(19, "learned.example", { client_ip: "192.0.2.11" }),
row(18, "nameless.example", { client_ip: "192.0.2.12" }),
row(17, "stranger.example", { client_ip: "192.0.2.99" }),
],
next_before: null,
} satisfies QueriesPage);
}),
);
renderPage();
// A hand-typed name wins outright; the learned name never surfaces for it.
const named = await screen.findByText("Kitchen Pi");
expect(named.getAttribute("title")).toBe("192.0.2.10");
expect(screen.queryByText("pi.lan")).toBeNull();
// The cell holds the learned name followed by the tag, so the match is on
// the containing span rather than on a bare text node.
const learned = screen.getByText(
(content, element) => element?.tagName === "SPAN" && content.startsWith("laptop.lan"),
);
expect(learned.getAttribute("title")).toBe("192.0.2.11");
// The affordance is text, not colour, so a screen reader announces it too.
expect(within(learned).getByText("learned")).toBeTruthy();
// A known client with neither name, and a client the loaded list has never
// seen, both fall back to the bare address with no tooltip standing in.
expect(screen.getByText("192.0.2.12").getAttribute("title")).toBeNull();
expect(screen.getByText("192.0.2.99").getAttribute("title")).toBeNull();
});
test("load more appends the next page and stops at the end of the log", async () => {
renderPage();
await screen.findByText("first.example");
fireEvent.click(screen.getByRole("button", { name: "Load more" }));
await screen.findByText("older.example");
expect(screen.getByText("first.example")).toBeTruthy();
expect(screen.getByText(/Showing 3 queries — end of log/)).toBeTruthy();
expect(screen.queryByRole("button", { name: "Load more" })).toBeNull();
});
test("applying a filter refetches and resets the accumulated list", async () => {
renderPage();
await screen.findByText("first.example");
fireEvent.click(screen.getByRole("button", { name: "Load more" }));
await screen.findByText("older.example");
fireEvent.change(screen.getByLabelText("Domain contains"), { target: { value: "ads" } });
fireEvent.click(screen.getByRole("button", { name: "Apply filters" }));
await screen.findByText(/Showing 1 query /);
expect(screen.getByText("ads.example")).toBeTruthy();
expect(screen.queryByText("first.example")).toBeNull();
expect(screen.queryByText("older.example")).toBeNull();
});
test("a load-more that resolves after a filter change is discarded", async () => {
let releaseLoadMore: () => void = () => {};
vi.stubGlobal(
"fetch",
vi.fn((input: RequestInfo | URL) => {
const url = String(input);
if (url === "/api/queries?before=19") {
return new Promise<Response>((resolve) => {
releaseLoadMore = () => {
resolve(
new Response(JSON.stringify(PAGES["/api/queries?before=19"]), {
status: 200,
headers: { "content-type": "application/json" },
}),
);
};
});
}
const payload = PAGES[url];
if (payload === undefined)
return Promise.resolve(new Response(JSON.stringify({ error: "not stubbed" }), { status: 404 }));
return Promise.resolve(
new Response(JSON.stringify(payload), {
status: 200,
headers: { "content-type": "application/json" },
}),
);
}),
);
renderPage();
await screen.findByText("first.example");
fireEvent.click(screen.getByRole("button", { name: "Load more" }));
fireEvent.change(screen.getByLabelText("Domain contains"), { target: { value: "ads" } });
fireEvent.click(screen.getByRole("button", { name: "Apply filters" }));
await screen.findByText(/Showing 1 query /);
releaseLoadMore();
await act(async () => {
await new Promise((resolve) => setTimeout(resolve, 0));
});
expect(screen.queryByText("older.example")).toBeNull();
expect(screen.getByText(/Showing 1 query /)).toBeTruthy();
expect(screen.queryByRole("alert")).toBeNull();
});
test("load more is disabled while a filter change shows placeholder data, then uses the fresh cursor", async () => {
let releaseFiltered: () => void = () => {};
const filteredPage: QueriesPage = {
queries: [row(19, "ads.example", { blocked: true, block_reason: "blocklist:stevenblack" })],
next_before: 7,
};
const filteredOlderPage: QueriesPage = {
queries: [row(3, "ads.older.example")],
next_before: null,
};
const fetchMock = vi.fn((input: RequestInfo | URL) => {
const url = String(input);
if (url === "/api/queries?domain=ads") {
return new Promise<Response>((resolve) => {
releaseFiltered = () => {
resolve(
new Response(JSON.stringify(filteredPage), {
status: 200,
headers: { "content-type": "application/json" },
}),
);
};
});
}
const payload = url === "/api/queries?domain=ads&before=7" ? filteredOlderPage : PAGES[url];
if (payload === undefined)
return Promise.resolve(new Response(JSON.stringify({ error: "not stubbed" }), { status: 404 }));
return Promise.resolve(
new Response(JSON.stringify(payload), {
status: 200,
headers: { "content-type": "application/json" },
}),
);
});
vi.stubGlobal("fetch", fetchMock);
renderPage();
await screen.findByText("first.example");
fireEvent.change(screen.getByLabelText("Domain contains"), { target: { value: "ads" } });
fireEvent.click(screen.getByRole("button", { name: "Apply filters" }));
const staleButton = screen.getByRole("button", { name: "Load more" });
expect(staleButton).toHaveProperty("disabled", true);
fireEvent.click(staleButton);
expect(fetchMock.mock.calls.map((call) => String(call[0]))).not.toContain("/api/queries?domain=ads&before=19");
releaseFiltered();
await waitFor(() => {
expect(screen.queryByText("first.example")).toBeNull();
});
const freshButton = screen.getByRole("button", { name: "Load more" });
expect(freshButton).toHaveProperty("disabled", false);
fireEvent.click(freshButton);
await screen.findByText("ads.older.example");
expect(fetchMock.mock.calls.map((call) => String(call[0]))).toContain("/api/queries?domain=ads&before=7");
expect(screen.getByText(/Showing 2 queries — end of log/)).toBeTruthy();
});
test("a background refetch after new rows arrive leaves no gap between the loaded pages", async () => {
// The newest-100 window moves up while the reader has a second page open.
// Refetching only the first page would drop n20 and n19 out of the middle
// of the table; the second page must be replayed from the fresh cursor.
const before: Record<string, QueriesPage> = {
"/api/queries": { queries: [row(20, "n20.example"), row(19, "n19.example")], next_before: 19 },
"/api/queries?before=19": { queries: [row(18, "n18.example"), row(17, "n17.example")], next_before: null },
};
const after: Record<string, QueriesPage> = {
"/api/queries": { queries: [row(22, "n22.example"), row(21, "n21.example")], next_before: 21 },
"/api/queries?before=21": {
queries: [row(20, "n20.example"), row(19, "n19.example"), row(18, "n18.example"), row(17, "n17.example")],
next_before: null,
},
};
let live = before;
vi.stubGlobal(
"fetch",
vi.fn(async (input: RequestInfo | URL) => {
const payload = live[String(input)];
if (payload === undefined) return new Response(JSON.stringify({ error: "not stubbed" }), { status: 404 });
return json(payload);
}),
);
const client = renderPage();
await screen.findByText("n20.example");
fireEvent.click(screen.getByRole("button", { name: "Load more" }));
await screen.findByText("n17.example");
live = after;
await act(async () => {
await client.invalidateQueries({ queryKey: ["queries"] });
});
await screen.findByText("n22.example");
const shown = screen.getAllByText(/^n\d+\.example$/).map((cell) => cell.textContent);
expect(shown).toEqual(["n22.example", "n21.example", "n20.example", "n19.example", "n18.example", "n17.example"]);
expect(screen.getByText(/Showing 6 queries — end of log/)).toBeTruthy();
});
test("a 401 on load more routes through handleUnauthorized instead of the inline error", async () => {
const assign = vi.fn();
vi.stubGlobal("location", { pathname: "/queries", search: "", assign });
vi.stubGlobal(
"fetch",
vi.fn(async (input: RequestInfo | URL) => {
const url = String(input);
if (url === "/api/queries?before=19") {
return new Response(JSON.stringify({ error: "unauthorized" }), {
status: 401,
headers: { "content-type": "application/json" },
});
}
const payload = PAGES[url];
if (payload === undefined) return new Response(JSON.stringify({ error: "not stubbed" }), { status: 404 });
return new Response(JSON.stringify(payload), {
status: 200,
headers: { "content-type": "application/json" },
});
}),
);
renderPage();
await screen.findByText("first.example");
fireEvent.click(screen.getByRole("button", { name: "Load more" }));
await waitFor(() => {
expect(assign).toHaveBeenCalledWith(`/login?redirect=${encodeURIComponent("/queries")}`);
});
expect(screen.queryByRole("alert")).toBeNull();
expect(screen.queryByText(/Failed to load more/)).toBeNull();
});
-374
View File
@@ -1,374 +0,0 @@
import { useState, type FormEvent } from "react";
import { useInfiniteQuery } from "@tanstack/react-query";
import * as stylex from "@stylexjs/stylex";
import * as api from "@/lib/api";
import { formatMicros, formatTime } from "@/lib/format";
import { queriesInfiniteQuery } from "@/lib/queries";
import type { QueriesFilter, QueryRow } from "@/lib/types";
import { ClientName, useClientNames, type ClientNames } from "@/features/clients/clientNames";
import { qtypeName } from "./qtype";
import Select from "@/ui/Select";
import { styles as shared } from "@/ui/styles";
import { colors } from "@/ui/tokens.stylex";
const DARK = "@media (prefers-color-scheme: dark)";
const STATUS_OPTIONS = [
{ value: "any", label: "All" },
{ value: "blocked", label: "Blocked only" },
{ value: "allowed", label: "Allowed only" },
];
const styles = stylex.create({
heading: {
fontSize: "1.5rem",
lineHeight: "2rem",
fontWeight: 600,
},
/** One column on a phone, two from `sm`, five from `lg`, as before. */
filterGrid: {
marginTop: "1rem",
display: "grid",
gap: "0.75rem",
gridTemplateColumns: {
default: "repeat(1, minmax(0, 1fr))",
"@media (min-width: 640px)": "repeat(2, minmax(0, 1fr))",
"@media (min-width: 1024px)": "repeat(5, minmax(0, 1fr))",
},
},
filterLabel: {
display: "block",
fontSize: "0.875rem",
lineHeight: "1.25rem",
},
filterInput: {
marginTop: "0.25rem",
width: "100%",
},
buttonRow: {
display: "flex",
alignItems: "flex-end",
gap: "0.5rem",
gridColumn: {
default: null,
"@media (min-width: 640px)": "span 2 / span 2",
"@media (min-width: 1024px)": "span 5 / span 5",
},
},
toolbarButton: {
fontWeight: 500,
},
note: {
fontSize: "0.875rem",
lineHeight: "1.25rem",
color: colors.textMuted,
},
empty: {
marginTop: "1.5rem",
color: colors.textMuted,
},
tableWrap: {
marginTop: "1rem",
overflowX: "auto",
borderRadius: "0.25rem",
borderWidth: 1,
borderStyle: "solid",
borderColor: colors.border,
},
table: {
width: "100%",
fontSize: "0.875rem",
lineHeight: "1.25rem",
},
/** The header tint is a shade off the ground in each scheme, not a token role. */
head: {
backgroundColor: { default: "oklch(98.5% 0 none)", [DARK]: "oklch(21% 0.006 285.885)" },
textAlign: "left",
},
th: {
paddingInline: "0.75rem",
paddingBlock: "0.5rem",
fontWeight: 500,
color: colors.textSecondary,
},
/** `divide-y`: a hairline between rows, so the first row carries none. */
row: {
borderTopWidth: { default: 1, ":first-child": 0 },
borderTopStyle: "solid",
borderTopColor: colors.border,
},
cell: {
paddingInline: "0.75rem",
paddingBlock: "0.5rem",
},
nowrap: {
whiteSpace: "nowrap",
},
breakAll: {
wordBreak: "break-all",
},
small: {
fontSize: "0.75rem",
lineHeight: "1rem",
},
muted: {
color: colors.textMuted,
},
blockedWrap: {
display: "inline-flex",
alignItems: "center",
gap: "0.375rem",
},
blockedBadge: {
borderRadius: "0.25rem",
paddingInline: "0.375rem",
paddingBlock: "0.125rem",
fontSize: "0.75rem",
lineHeight: "1rem",
fontWeight: 500,
backgroundColor: { default: "oklch(93.6% 0.032 17.717)", [DARK]: "oklch(39.6% 0.141 25.723)" },
color: { default: "oklch(44.4% 0.177 26.899)", [DARK]: "oklch(88.5% 0.062 18.334)" },
},
footer: {
marginTop: "0.75rem",
display: "flex",
alignItems: "center",
gap: "0.75rem",
},
moreError: {
marginTop: "0.5rem",
fontSize: "0.875rem",
lineHeight: "1.25rem",
color: colors.dangerText,
},
});
function errorMessage(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
function datetimeLocalToUnix(value: string): number | undefined {
if (value === "") return undefined;
const ms = new Date(value).getTime();
return Number.isFinite(ms) ? Math.floor(ms / 1000) : undefined;
}
export function BlockedCell({ row }: { row: Pick<QueryRow, "blocked" | "block_reason"> }) {
if (!row.blocked) return <span {...stylex.props(styles.muted)}></span>;
return (
<span {...stylex.props(styles.blockedWrap)}>
<span {...stylex.props(styles.blockedBadge)}>Blocked</span>
{row.block_reason !== "" && <span {...stylex.props(styles.small, styles.muted)}>{row.block_reason}</span>}
</span>
);
}
export function QueryCells({ row, clientNames }: { row: Omit<QueryRow, "id">; clientNames: ClientNames }) {
return (
<>
<td {...stylex.props(styles.cell, styles.nowrap, styles.muted)}>{formatTime(row.ts)}</td>
<td {...stylex.props(styles.cell, styles.small, styles.breakAll, shared.mono)}>{row.domain}</td>
<td {...stylex.props(styles.cell, styles.small, styles.nowrap)}>
<ClientName ip={row.client_ip} names={clientNames} />
</td>
<td {...stylex.props(styles.cell, styles.nowrap)}>{qtypeName(row.qtype)}</td>
<td {...stylex.props(styles.cell)}>
<BlockedCell row={row} />
</td>
<td {...stylex.props(styles.cell, styles.nowrap, shared.tabularNums)}>
{row.response_time_us === null ? "—" : formatMicros(row.response_time_us)}
</td>
<td {...stylex.props(styles.cell, styles.nowrap)}>
{row.cache_hit === null ? "—" : row.cache_hit ? "hit" : "miss"}
</td>
<td {...stylex.props(styles.cell, styles.small, styles.breakAll, shared.mono)}>
{row.upstream === "" ? "—" : row.upstream}
</td>
</>
);
}
export function QueryTableHead() {
return (
<thead {...stylex.props(styles.head)}>
<tr>
<th {...stylex.props(styles.th)}>Time</th>
<th {...stylex.props(styles.th)}>Domain</th>
<th {...stylex.props(styles.th)}>Client</th>
<th {...stylex.props(styles.th)}>Type</th>
<th {...stylex.props(styles.th)}>Status</th>
<th {...stylex.props(styles.th)}>Response</th>
<th {...stylex.props(styles.th)}>Cache</th>
<th {...stylex.props(styles.th)}>Upstream</th>
</tr>
</thead>
);
}
export default function QueryLogPage() {
const [domain, setDomain] = useState("");
const [client, setClient] = useState("");
const [blocked, setBlocked] = useState("any");
const [since, setSince] = useState("");
const [until, setUntil] = useState("");
const [applied, setApplied] = useState<QueriesFilter>({});
const base = useInfiniteQuery(queriesInfiniteQuery(applied));
const clientNames = useClientNames();
const pages = base.data?.pages ?? [];
const rows: QueryRow[] = pages.flatMap((page) => page.queries);
const filterActive = Object.keys(applied).length > 0;
// `base.hasNextPage` reads the query state, which is empty while placeholder
// data stands in for a filter change; derive the cursor from what is on
// screen so the button keeps its place instead of flashing "end of log".
const lastPage = pages[pages.length - 1];
const hasMore = lastPage !== undefined && lastPage.next_before !== null;
// A 401 is already redirecting via the cache-level handleUnauthorized.
const isUnauthorized = base.error instanceof api.ApiError && base.error.status === 401;
const moreError = base.isFetchNextPageError && !isUnauthorized ? errorMessage(base.error) : null;
function applyFilters(event: FormEvent) {
event.preventDefault();
const filter: QueriesFilter = {};
if (domain.trim() !== "") filter.domain = domain.trim();
if (client.trim() !== "") filter.client = client.trim();
if (blocked === "blocked") filter.blocked = true;
if (blocked === "allowed") filter.blocked = false;
const sinceTs = datetimeLocalToUnix(since);
if (sinceTs !== undefined) filter.since = sinceTs;
const untilTs = datetimeLocalToUnix(until);
if (untilTs !== undefined) filter.until = untilTs;
setApplied(filter);
}
function clearFilters() {
setDomain("");
setClient("");
setBlocked("any");
setSince("");
setUntil("");
setApplied({});
}
function loadMore() {
if (!hasMore || base.isFetchingNextPage || base.isPlaceholderData) return;
void base.fetchNextPage();
}
return (
<section>
<h1 {...stylex.props(styles.heading)}>Query Log</h1>
<form onSubmit={applyFilters} {...stylex.props(styles.filterGrid)}>
<label {...stylex.props(styles.filterLabel)}>
Domain contains
<input
type="text"
value={domain}
onChange={(event) => setDomain(event.target.value)}
{...stylex.props(shared.smallInput, styles.filterInput, shared.focusRing)}
/>
</label>
<label {...stylex.props(styles.filterLabel)}>
Client (exact)
<input
type="text"
value={client}
onChange={(event) => setClient(event.target.value)}
{...stylex.props(shared.smallInput, styles.filterInput, shared.focusRing)}
/>
</label>
<Select
variant="compactField"
label="Status"
value={blocked}
onChange={setBlocked}
options={STATUS_OPTIONS}
/>
<label {...stylex.props(styles.filterLabel)}>
Since
<input
type="datetime-local"
value={since}
onChange={(event) => setSince(event.target.value)}
{...stylex.props(shared.smallInput, styles.filterInput, shared.focusRing)}
/>
</label>
<label {...stylex.props(styles.filterLabel)}>
Until
<input
type="datetime-local"
value={until}
onChange={(event) => setUntil(event.target.value)}
{...stylex.props(shared.smallInput, styles.filterInput, shared.focusRing)}
/>
</label>
<div {...stylex.props(styles.buttonRow)}>
<button type="submit" {...stylex.props(shared.button, styles.toolbarButton, shared.focusRing)}>
Apply filters
</button>
<button
type="button"
onClick={clearFilters}
{...stylex.props(shared.button, styles.toolbarButton, shared.focusRing)}
>
Clear
</button>
{base.isFetching && (
<span {...stylex.props(styles.note)} role="status">
Loading
</span>
)}
</div>
</form>
{base.data === undefined ? (
<p {...stylex.props(styles.empty, shared.pulse)} role="status">
Loading query log
</p>
) : rows.length === 0 ? (
<p {...stylex.props(styles.empty)}>
{filterActive ? "No queries match the current filters." : "No queries logged yet."}
</p>
) : (
<>
<div {...stylex.props(styles.tableWrap)}>
<table {...stylex.props(styles.table)}>
<QueryTableHead />
<tbody>
{rows.map((row) => (
<tr key={row.id} {...stylex.props(styles.row)}>
<QueryCells row={row} clientNames={clientNames} />
</tr>
))}
</tbody>
</table>
</div>
<div {...stylex.props(styles.footer)}>
<p {...stylex.props(styles.note)}>
Showing {rows.length} {rows.length === 1 ? "query" : "queries"}
{hasMore ? "" : " — end of log"}
</p>
{hasMore && (
<button
type="button"
onClick={loadMore}
disabled={base.isFetchingNextPage || base.isPlaceholderData}
{...stylex.props(shared.button, styles.toolbarButton, shared.focusRing)}
>
{base.isFetchingNextPage ? "Loading…" : "Load more"}
</button>
)}
</div>
{moreError !== null && (
<p role="alert" {...stylex.props(styles.moreError)}>
Failed to load more: {moreError}
</p>
)}
</>
)}
</section>
);
}
@@ -0,0 +1,39 @@
import {
ENUM_VALUES,
policyActionLabel,
policyReasonLabel,
qclassName,
rcodeName,
routeKindLabel,
} from "./provenanceCopy";
/**
* `tsc` proves the maps total over the union; this proves the union is the set
* the server actually stores, and that no entry was left as its raw tag name.
*/
test("every stored enum value has a label of its own", () => {
const labels = [
...ENUM_VALUES.policyAction.map(policyActionLabel),
...ENUM_VALUES.policyReason.map(policyReasonLabel),
...ENUM_VALUES.routeKind.map(routeKindLabel),
];
for (const label of labels) {
expect(label).not.toBe("");
expect(label).not.toMatch(/_/);
}
expect(new Set(ENUM_VALUES.policyReason.map(policyReasonLabel)).size).toBe(ENUM_VALUES.policyReason.length);
});
test("response codes read by name where one exists, by number where none does", () => {
expect(rcodeName(0)).toBe("NOERROR (0)");
expect(rcodeName(3)).toBe("NXDOMAIN (3)");
expect(rcodeName(16)).toBe("BADVERS (16)");
// The column holds the twelve-bit extended code, most of which is unassigned.
expect(rcodeName(3841)).toBe("RCODE 3841");
});
test("query classes read the same way", () => {
expect(qclassName(1)).toBe("IN (1)");
expect(qclassName(255)).toBe("ANY (255)");
expect(qclassName(42)).toBe("CLASS 42");
});
@@ -0,0 +1,103 @@
import { POLICY_ACTIONS, POLICY_REASONS, ROUTE_KINDS } from "@/lib/types";
import type { PolicyAction, PolicyReason, RouteKind } from "@/lib/types";
/**
* Display names for the three stored enums. `Record` over the union, so a value
* added to `src/storage/provenance.zig` and mirrored into `lib/types.ts` fails
* `tsc` here instead of reaching a cell as a raw tag name.
*/
const POLICY_ACTION_LABELS: Record<PolicyAction, string> = {
not_evaluated: "Not evaluated",
allow: "Allowed",
block: "Blocked",
};
const POLICY_REASON_LABELS: Record<PolicyReason, string> = {
rule_allow_exact: "Allow rule (exact)",
rule_block_exact: "Block rule (exact)",
rule_allow_wildcard: "Allow rule (wildcard)",
rule_block_wildcard: "Block rule (wildcard)",
rule_allow_regex: "Allow rule (regex)",
rule_block_regex: "Block rule (regex)",
blocklist_exception: "Blocklist exception",
blocklist_domain: "Blocklist (domain)",
blocklist_wildcard: "Blocklist (wildcard)",
local_record: "Local record",
forward_zone: "Forward zone",
non_in_class: "Not class IN",
paused: "Filtering paused",
snapshot_unavailable: "No filter snapshot",
no_match: "No match",
protocol_error: "Protocol refusal",
};
const ROUTE_KIND_LABELS: Record<RouteKind, string> = {
blocked: "Blocked locally",
local: "Local record",
forward_zone: "Forward zone",
upstream: "Upstream resolver",
cache: "Cache",
rejected: "Rejected",
};
export function policyActionLabel(action: PolicyAction): string {
return POLICY_ACTION_LABELS[action];
}
export function policyReasonLabel(reason: PolicyReason): string {
return POLICY_REASON_LABELS[reason];
}
export function routeKindLabel(kind: RouteKind): string {
return ROUTE_KIND_LABELS[kind];
}
/** The enum value sets, for tests that prove the maps exhaustive at runtime too. */
export const ENUM_VALUES = {
policyAction: POLICY_ACTIONS,
policyReason: POLICY_REASONS,
routeKind: ROUTE_KINDS,
} as const;
const RCODE_NAMES: Record<number, string> = {
0: "NOERROR",
1: "FORMERR",
2: "SERVFAIL",
3: "NXDOMAIN",
4: "NOTIMP",
5: "REFUSED",
6: "YXDOMAIN",
7: "YXRRSET",
8: "NXRRSET",
9: "NOTAUTH",
10: "NOTZONE",
16: "BADVERS",
};
/**
* The bare mnemonic, for a table cell with no room for the number. An
* unassigned code has no mnemonic to shorten, so it keeps the same `RCODE <n>`
* shape the long form falls back to.
*/
export function rcodeShortName(rcode: number): string {
return RCODE_NAMES[rcode] ?? `RCODE ${rcode}`;
}
/** The twelve-bit extended code as `NXDOMAIN (3)`; an unassigned code keeps its number. */
export function rcodeName(rcode: number): string {
const name = RCODE_NAMES[rcode];
return name === undefined ? `RCODE ${rcode}` : `${name} (${rcode})`;
}
const QCLASS_NAMES: Record<number, string> = {
1: "IN",
3: "CH",
4: "HS",
254: "NONE",
255: "ANY",
};
export function qclassName(qclass: number): string {
const name = QCLASS_NAMES[qclass];
return name === undefined ? `CLASS ${qclass}` : `${name} (${qclass})`;
}
@@ -0,0 +1,58 @@
import type { Provenance, QueryRow } from "@/lib/types";
/**
* Fixture builders for the provenance shapes, shared by the query-log, detail
* and live-stream tests the way `features/activity/fakeEventSource.ts` is shared.
*
* The defaults describe the dullest possible query an allowed name nothing
* matched, answered upstream so each test states only the fields it is about.
*/
type Sections = {
[K in keyof Provenance]?: Partial<Provenance[K]>;
};
export function provenance(sections: Sections = {}): Provenance {
return {
request: {
time: 1_700_000_000,
domain: "example.com",
client: "192.0.2.10",
qtype: 1,
qclass: 1,
...sections.request,
},
group: { id: 1, name: "default", ...sections.group },
policy: {
action: "allow",
reason: "no_match",
matched: "",
source_id: null,
source_name: "",
...sections.policy,
},
rewrites: { cname_target: "", safe_search_target: "", ...sections.rewrites },
route: { kind: "upstream", forward_zone: "", upstream: "https://dns.example/dns-query", ...sections.route },
response: { rcode: 0, duration_us: 1234, ...sections.response },
};
}
/** The flat stored row of the same dull query. */
export function queryRow(id: number, overrides: Partial<QueryRow> = {}): QueryRow {
return {
id,
ts: 1_700_000_000,
domain: "example.com",
client_ip: "192.0.2.10",
qtype: 1,
qclass: 1,
rcode: 0,
blocked: false,
response_time_us: 1234,
cache_hit: false,
upstream: "https://dns.example/dns-query",
policy_action: "allow",
policy_reason: "no_match",
route_kind: "upstream",
...overrides,
};
}
@@ -0,0 +1,88 @@
import type { LiveQueryEvent, PolicyReason, QueryRow, RouteKind } from "@/lib/types";
/**
* What the query-log table renders for one row, whichever surface it came from.
*
* The stored list row and the live stream's provenance event describe the same
* query in two different shapes flat summary against nested full detail and
* both pages share one set of cells, so both project into this.
*
* `id` is null for a streamed event: the frame precedes its own insert, so no
* row exists to link to yet.
*/
export interface QuerySummary {
id: number | null;
ts: number;
domain: string;
client_ip: string;
qtype: number | null;
blocked: boolean;
policy_reason: PolicyReason;
/** The twelve-bit extended code the client saw, including a synthesized SERVFAIL. */
rcode: number;
route_kind: RouteKind;
response_time_us: number | null;
cache_hit: boolean | null;
upstream: string;
}
/**
* Whether the cache answered, or null where it never applied. Mirrors
* `Context.cacheHit` in src/server/handler.zig, which derives the stored
* `cache_hit` column from the same route: a local record, a blocked answer and
* a protocol refusal all bypass the cache, and "miss" would claim a lookup that
* never happened.
*/
export function cacheHitFor(kind: RouteKind): boolean | null {
switch (kind) {
case "cache":
return true;
case "upstream":
case "forward_zone":
return false;
case "local":
case "blocked":
case "rejected":
return null;
}
}
export function summarizeRow(row: QueryRow): QuerySummary {
return {
id: row.id,
ts: row.ts,
domain: row.domain,
client_ip: row.client_ip,
qtype: row.qtype,
blocked: row.blocked,
policy_reason: row.policy_reason,
rcode: row.rcode,
route_kind: row.route_kind,
response_time_us: row.response_time_us,
cache_hit: row.cache_hit,
upstream: row.upstream,
};
}
/**
* The same summary out of a live frame. `blocked` and `cache_hit` are derived
* rather than sent: the server derives the stored columns from exactly these
* two fields (handler.zig's `Entry.init` call), so the projection reproduces
* them instead of the DTO carrying the same fact twice.
*/
export function summarizeEvent(event: LiveQueryEvent): QuerySummary {
return {
id: null,
ts: event.request.time,
domain: event.request.domain,
client_ip: event.request.client,
qtype: event.request.qtype,
blocked: event.policy.action === "block",
policy_reason: event.policy.reason,
rcode: event.response.rcode,
route_kind: event.route.kind,
response_time_us: event.response.duration_us,
cache_hit: cacheHitFor(event.route.kind),
upstream: event.route.upstream,
};
}
@@ -33,6 +33,7 @@ function baseSettings(): Settings {
level: "info",
retention_days: 30,
query_log_buffer_max: 10000,
query_log_flush_interval_s: 60,
hide_domains: false,
hide_client_ips: false,
output: "stderr",
@@ -116,6 +116,7 @@ const SECTIONS: readonly AnySectionDef[] = [
{ key: "level", kind: ["error", "warn", "info", "debug"] },
{ key: "retention_days", kind: "number" },
{ key: "query_log_buffer_max", kind: "number" },
{ key: "query_log_flush_interval_s", kind: "number" },
{ key: "hide_domains", kind: "boolean" },
{ key: "hide_client_ips", kind: "boolean" },
{ key: "output", kind: ["stderr", "syslog", "file"] },
@@ -38,6 +38,7 @@ function baseSettings(): Settings {
level: "info",
retention_days: 30,
query_log_buffer_max: 10000,
query_log_flush_interval_s: 60,
hide_domains: false,
hide_client_ips: false,
output: "stderr",
+31
View File
@@ -0,0 +1,31 @@
import { render, screen } from "@testing-library/react";
import CoverageNotice from "./CoverageNotice";
import { formatTime } from "./format";
const WATERMARK = 1_700_000_000;
test("a window the log covers in full says nothing", () => {
const { container } = render(<CoverageNotice coverage={{ complete: true, available_since: WATERMARK }} />);
expect(container.textContent).toBe("");
});
test("a window reaching past the watermark names the instant history starts", () => {
render(<CoverageNotice coverage={{ complete: false, available_since: WATERMARK }} />);
const notice = screen.getByRole("status");
expect(notice.textContent).toContain("Query history is available from");
expect(notice.textContent).toContain(formatTime(WATERMARK));
});
/**
* The server judges completeness, not the page: an unbounded request is
* incomplete whatever the watermark reads, and the notice must follow that
* verdict rather than compare timestamps itself.
*/
test("the server's verdict decides, not the numbers beside it", () => {
const { rerender, container } = render(
<CoverageNotice coverage={{ complete: true, available_since: WATERMARK }} />,
);
expect(container.textContent).toBe("");
rerender(<CoverageNotice coverage={{ complete: false, available_since: 0 }} />);
expect(screen.getByRole("status").textContent).toContain(formatTime(0));
});
+38
View File
@@ -0,0 +1,38 @@
import * as stylex from "@stylexjs/stylex";
import { formatTime } from "@/lib/format";
import type { Coverage } from "@/lib/types";
import { colors } from "@/ui/tokens.stylex";
const styles = stylex.create({
notice: {
marginTop: "0.75rem",
borderRadius: "0.25rem",
borderWidth: 1,
borderStyle: "solid",
borderColor: colors.border,
backgroundColor: colors.surfaceHover,
paddingInline: "0.75rem",
paddingBlock: "0.5rem",
fontSize: "0.875rem",
lineHeight: "1.25rem",
color: colors.textSecondary,
},
});
/**
* How far back the numbers on this page can reach.
*
* Rendered whenever a response says its window is incomplete, which includes
* the common case of a request with no lower bound at all. The line states the
* watermark and nothing more: the same incompleteness covers a log retention
* has pruned and one that simply has not been running long enough, and the
* response does not say which.
*/
export default function CoverageNotice({ coverage }: { coverage: Coverage }) {
if (coverage.complete) return null;
return (
<p role="status" {...stylex.props(styles.notice)}>
Query history is available from {formatTime(coverage.available_since)}.
</p>
);
}
+24 -1
View File
@@ -6,6 +6,10 @@ import type {
ClientEdit,
ClientPrefix,
ClientPrefixInput,
DiagnosticEvent,
DiagnosticsFilter,
DiagnosticsPage,
DiagnosticsPurge,
ForwardZone,
ForwardZoneInput,
Group,
@@ -22,6 +26,7 @@ import type {
Period,
QueriesFilter,
QueriesPage,
QueryDetail,
Rule,
RuleEcho,
RuleInput,
@@ -108,6 +113,9 @@ export const logout = (): Promise<LogoutResponse> => request("/api/auth/logout",
export const getQueries = (filter: QueriesFilter = {}): Promise<QueriesPage> =>
request(`/api/queries${qs({ ...filter })}`);
/** Full provenance of one logged row. 404 when retention has removed it, 503 with no query log. */
export const getQueryDetail = (id: number): Promise<QueryDetail> => request(`/api/queries/${id}`);
/** `EventSource` URL for the live stream; not a fetch route. */
export const liveQueriesUrl = "/api/queries/live";
@@ -118,7 +126,22 @@ export const getStatsTimeseries = (period?: Period): Promise<StatsTimeseries> =>
export const getLookup = (domain: string, groupId?: number): Promise<LookupResult> =>
request(`/api/lookup${qs({ domain, group_id: groupId })}`);
export const getUpstreamHealth = (): Promise<UpstreamHealth> => request("/api/upstream/health");
export const getUpstreamHealth = (period?: Period): Promise<UpstreamHealth> =>
request(`/api/upstream/health${qs({ period })}`);
// Diagnostics
export const getDiagnostics = (filter: DiagnosticsFilter = {}): Promise<DiagnosticsPage> =>
request(`/api/diagnostics${qs({ ...filter })}`);
export const getDiagnostic = (id: number): Promise<DiagnosticEvent> => request(`/api/diagnostics/${id}`);
/** Purges one resolved event. An event still active answers 409, an unknown id 404. */
export const purgeDiagnostic = (id: number): Promise<void> => request(`/api/diagnostics/${id}`, { method: "DELETE" });
/** Purges the whole resolved history; active events are never touched. */
export const purgeResolvedDiagnostics = (): Promise<DiagnosticsPurge> =>
request("/api/diagnostics", { method: "DELETE" });
// Groups
+171 -33
View File
@@ -16,6 +16,9 @@ import type {
BlocklistEcho,
Client,
ClientPrefix,
DiagnosticEvent,
DiagnosticsPage,
DiagnosticsPurge,
ErrorEnvelope,
ForwardZone,
Group,
@@ -26,6 +29,7 @@ import type {
LookupResult,
PauseState,
QueriesPage,
QueryDetail,
Rule,
RuleEcho,
SettingsEnvelope,
@@ -39,6 +43,11 @@ import type {
} from "@/lib/types";
export const sample_get_health: Health = {
diagnostics: {
active_errors: 0,
active_warnings: 0,
state: "recording",
},
disk: {
db_bytes: 0,
free_bytes: 0,
@@ -73,6 +82,57 @@ export const sample_logout: LogoutResponse = {
authenticated: false,
};
export const sample_get_diagnostics: DiagnosticsPage = {
active: {
errors: 0,
warnings: 0,
},
events: [
{
code: "upstream_history.write",
component: "upstream_history",
detail: "Busy",
first_seen: 0,
id: 0,
last_seen: 0,
occurrences: 0,
resolved_at: 0,
severity: "warning",
subject: "history",
},
{
code: "blocklist.refresh",
component: "blocklist",
detail: "download failed: ConnectionTimedOut",
first_seen: 0,
id: 0,
last_seen: 0,
occurrences: 0,
resolved_at: null,
severity: "warning",
subject: "StevenBlack",
},
],
next_before: null,
};
export const sample_get_diagnostic: DiagnosticEvent = {
code: "blocklist.refresh",
component: "blocklist",
detail: "download failed: ConnectionTimedOut",
first_seen: 0,
id: 0,
last_seen: 0,
occurrences: 0,
resolved_at: null,
severity: "warning",
subject: "StevenBlack",
};
export const sample_purge_diagnostics: DiagnosticsPurge = {
purged: 0,
};
export const sample_create_blocklist: BlocklistEcho = {
enabled: false,
id: 0,
@@ -333,92 +393,162 @@ export const sample_update_upstream: UpstreamEcho = {
export const sample_get_upstream_health: UpstreamHealth = {
available: 0,
complete: true,
period: "24h",
since: 0,
total: 0,
until: 0,
upstreams: [
{
available: true,
consecutive_failures: 0,
enabled: true,
last_error: "",
success_rate: 0,
total_failures: 0,
total_successes: 0,
period: {
attempts: 0,
failures: 0,
last_failure_at: null,
last_failure_error: null,
success_rate: null,
successes: 0,
},
url: "https://dns.example/dns-query",
},
],
};
export const sample_get_queries: QueriesPage = {
coverage: {
available_since: 0,
complete: false,
},
next_before: 0,
queries: [
{
block_reason: "",
blocked: true,
cache_hit: null,
client_ip: "192.0.2.11",
domain: "shop.example",
id: 0,
policy_action: "block",
policy_reason: "blocklist_wildcard",
qclass: 0,
qtype: 0,
rcode: 0,
response_time_us: 0,
route_kind: "blocked",
ts: 0,
upstream: "",
},
{
blocked: false,
cache_hit: false,
client_ip: "192.0.2.10",
domain: "news.example",
id: 0,
policy_action: "allow",
policy_reason: "no_match",
qclass: 0,
qtype: 0,
rcode: 0,
response_time_us: 0,
route_kind: "upstream",
ts: 0,
upstream: "https://dns.example/dns-query",
},
{
blocked: false,
cache_hit: true,
client_ip: "192.0.2.10",
domain: "d24.example",
id: 0,
policy_action: "allow",
policy_reason: "no_match",
qclass: 0,
qtype: 0,
rcode: 0,
response_time_us: 0,
route_kind: "cache",
ts: 0,
upstream: "https://dns.example/dns-query",
upstream: "",
},
{
block_reason: "",
blocked: false,
cache_hit: false,
client_ip: "192.0.2.10",
domain: "d23.example",
id: 0,
policy_action: "allow",
policy_reason: "no_match",
qclass: 0,
qtype: 0,
rcode: 0,
response_time_us: 0,
route_kind: "upstream",
ts: 0,
upstream: "https://dns.example/dns-query",
},
{
block_reason: "",
blocked: false,
cache_hit: true,
client_ip: "192.0.2.10",
domain: "d22.example",
id: 0,
policy_action: "allow",
policy_reason: "no_match",
qclass: 0,
qtype: 0,
rcode: 0,
response_time_us: 0,
ts: 0,
upstream: "https://dns.example/dns-query",
},
{
block_reason: "",
blocked: false,
cache_hit: false,
client_ip: "192.0.2.10",
domain: "d21.example",
id: 0,
qtype: 0,
response_time_us: 0,
ts: 0,
upstream: "https://dns.example/dns-query",
},
{
block_reason: "blocklist_domain",
blocked: true,
cache_hit: null,
client_ip: "192.0.2.10",
domain: "d20.example",
id: 0,
qtype: 0,
response_time_us: 0,
route_kind: "cache",
ts: 0,
upstream: "",
},
],
};
export const sample_get_query_detail: QueryDetail = {
group: {
id: 0,
name: "kids",
},
id: 0,
policy: {
action: "block",
matched: "||tracker.example^",
reason: "blocklist_wildcard",
source_id: 0,
source_name: "StevenBlack",
},
request: {
client: "192.0.2.11",
domain: "shop.example",
qclass: 0,
qtype: 0,
time: 0,
},
response: {
duration_us: 0,
rcode: 0,
},
rewrites: {
cname_target: "cdn.tracker.example",
safe_search_target: "",
},
route: {
forward_zone: "",
kind: "blocked",
upstream: "",
},
};
export const sample_get_stats: StatsTotals = {
avg_response_time_us: null,
blocked: 0,
cached: 0,
clients: 0,
coverage: {
available_since: 0,
complete: true,
},
period: "1h",
queries: 0,
since: 0,
@@ -435,6 +565,10 @@ export const sample_get_stats_timeseries: StatsTimeseries = {
ts: 0,
},
],
coverage: {
available_since: 0,
complete: true,
},
period: "1h",
since: 0,
until: 0,
@@ -491,6 +625,7 @@ export const sample_get_settings: SettingsEnvelope = {
"logging.level",
"logging.retention_days",
"logging.query_log_buffer_max",
"logging.query_log_flush_interval_s",
"logging.hide_domains",
"logging.hide_client_ips",
"logging.output",
@@ -552,6 +687,7 @@ export const sample_get_settings: SettingsEnvelope = {
max_size_mb: 0,
output: "stderr",
query_log_buffer_max: 0,
query_log_flush_interval_s: 0,
retention_days: 0,
},
upstream: {
@@ -614,6 +750,7 @@ export const sample_put_settings: SettingsEnvelope = {
"logging.level",
"logging.retention_days",
"logging.query_log_buffer_max",
"logging.query_log_flush_interval_s",
"logging.hide_domains",
"logging.hide_client_ips",
"logging.output",
@@ -675,6 +812,7 @@ export const sample_put_settings: SettingsEnvelope = {
max_size_mb: 0,
output: "stderr",
query_log_buffer_max: 0,
query_log_flush_interval_s: 0,
retention_days: 0,
},
upstream: {
+18 -1
View File
@@ -1,4 +1,4 @@
import { formatBytes, formatMicros, formatTime } from "@/lib/format";
import { formatBytes, formatDuration, formatMicros, formatTime } from "@/lib/format";
test("formatTime renders unix seconds in the given locale and zone", () => {
// 2024-01-01T00:00:00Z; ICU emits U+202F before AM/PM in recent Node.
@@ -15,6 +15,23 @@ test("formatBytes humanizes with binary units", () => {
expect(formatBytes(2 * 1024 ** 4)).toBe("2.0 TiB");
});
test("formatDuration steps up a unit at each boundary and truncates", () => {
expect(formatDuration(0)).toBe("0s");
expect(formatDuration(59)).toBe("59s");
expect(formatDuration(60)).toBe("1m");
expect(formatDuration(3599)).toBe("59m");
expect(formatDuration(3600)).toBe("1h");
expect(formatDuration(10800)).toBe("3h");
expect(formatDuration(86399)).toBe("23h");
expect(formatDuration(86400)).toBe("1d");
expect(formatDuration(400000)).toBe("4d");
});
test("formatDuration is never negative", () => {
// Clock skew between the server's timestamps and the browser's clock.
expect(formatDuration(-5)).toBe("0s");
});
test("formatMicros renders milliseconds with one decimal", () => {
expect(formatMicros(0)).toBe("0.0 ms");
expect(formatMicros(1234)).toBe("1.2 ms");
+18
View File
@@ -21,6 +21,24 @@ export function formatBytes(bytes: number): string {
return `${value.toFixed(1)} ${unit}`;
}
const AGE_UNITS = [
{ seconds: 86400, suffix: "d" },
{ seconds: 3600, suffix: "h" },
{ seconds: 60, suffix: "m" },
] as const;
/**
* Seconds of elapsed time a coarse "3h". Truncating and single-unit on
* purpose, for a span the caller labels itself, as in "active for 3h". A
* negative span reads "0s": clock skew is not a duration.
*/
export function formatDuration(seconds: number): string {
for (const unit of AGE_UNITS) {
if (seconds >= unit.seconds) return `${Math.floor(seconds / unit.seconds)}${unit.suffix}`;
}
return `${Math.max(0, Math.floor(seconds))}s`;
}
/** Microseconds → milliseconds with one decimal, e.g. 1234 → "1.2 ms". */
export function formatMicros(micros: number): string {
return `${(micros / 1000).toFixed(1)} ms`;
+55 -3
View File
@@ -5,6 +5,8 @@ import type {
BlocklistInput,
ClientEdit,
ClientPrefixInput,
DiagnosticsFilter,
DiagnosticsPage,
ForwardZoneInput,
GroupInput,
LocalRecordInput,
@@ -23,7 +25,12 @@ export const queryKeys = {
stats: (period: Period) => ["stats", period] as const,
timeseries: (period: Period) => ["stats", "timeseries", period] as const,
queriesInfinite: (filter: QueriesFilter) => ["queries", "infinite", filter] as const,
upstreamHealth: ["upstream-health"] as const,
queryDetail: (id: number) => ["queries", "detail", id] as const,
diagnosticsInfinite: (filter: DiagnosticsFilter) => ["diagnostics", "infinite", filter] as const,
diagnostic: (id: number) => ["diagnostics", "event", id] as const,
/** Prefix of every diagnostics entry, page and detail alike; the purge target. */
diagnosticsAll: ["diagnostics"] as const,
upstreamHealth: (period: Period) => ["upstream-health", period] as const,
lookup: (domain: string, groupId?: number) => ["lookup", domain, groupId ?? null] as const,
/** Prefix of every `lookup` entry; the invalidation target after any verdict input changes. */
lookupAll: ["lookup"] as const,
@@ -69,8 +76,40 @@ export const queriesInfiniteQuery = (filter: QueriesFilter = {}) =>
placeholderData: keepPreviousData,
});
export const upstreamHealthQuery = () =>
queryOptions({ queryKey: queryKeys.upstreamHealth, queryFn: api.getUpstreamHealth, refetchInterval: 30_000 });
export const queryDetailQuery = (id: number) =>
queryOptions({ queryKey: queryKeys.queryDetail(id), queryFn: () => api.getQueryDetail(id) });
// Keyset pagination on `next_before`, exactly as the query log pages
// (handlers/diagnostics.zig copies the /api/queries contract). The active view
// polls on healthQuery's cadence because an episode opening is the same news a
// health banner carries; a resolved-history page is settled and does not poll.
// `enabled` belongs to the factory rather than to a spread at the call site:
// spreading the options object loses the page-param type, and the Diagnostics
// page turns one of its two sections off whenever a filter excludes it.
export const diagnosticsInfiniteQuery = (filter: DiagnosticsFilter = {}, enabled = true) =>
infiniteQueryOptions({
enabled,
queryKey: queryKeys.diagnosticsInfinite(filter),
queryFn: ({ pageParam }: { pageParam: number | undefined }) =>
api.getDiagnostics(pageParam === undefined ? filter : { ...filter, before: pageParam }),
initialPageParam: undefined as number | undefined,
getNextPageParam: (last: DiagnosticsPage) => last.next_before ?? undefined,
placeholderData: keepPreviousData,
refetchInterval: filter.state === "active" ? 10_000 : undefined,
});
export const diagnosticQuery = (id: number) =>
queryOptions({ queryKey: queryKeys.diagnostic(id), queryFn: () => api.getDiagnostic(id) });
// The period is part of the key: the upstream aggregates are ranged like the
// stats ones, so the picker has to refetch them rather than reuse a cached
// window under a new label.
export const upstreamHealthQuery = (period: Period = "24h") =>
queryOptions({
queryKey: queryKeys.upstreamHealth(period),
queryFn: () => api.getUpstreamHealth(period),
refetchInterval: 30_000,
});
export const lookupQuery = (domain: string, groupId?: number) =>
queryOptions({ queryKey: queryKeys.lookup(domain, groupId), queryFn: () => api.getLookup(domain, groupId) });
@@ -105,6 +144,19 @@ export const settingsQuery = () => queryOptions({ queryKey: queryKeys.settings,
// Group membership and names feed lookup verdicts and the group columns on
// clients, prefixes and rules, hence the wide invalidation on group mutations.
// Both purges invalidate the whole `diagnostics` prefix rather than one page
// key: the resolved list, the active list (whose `active` counts ride along) and
// the detail query of the row just deleted all describe the table that changed.
export const diagnosticPurgeMutation = (qc: QueryClient) => ({
mutationFn: (id: number) => api.purgeDiagnostic(id),
onSuccess: () => qc.invalidateQueries({ queryKey: queryKeys.diagnosticsAll }),
});
export const diagnosticsPurgeResolvedMutation = (qc: QueryClient) => ({
mutationFn: () => api.purgeResolvedDiagnostics(),
onSuccess: () => qc.invalidateQueries({ queryKey: queryKeys.diagnosticsAll }),
});
function invalidateGroupWorld(qc: QueryClient): Promise<unknown> {
return Promise.all([
qc.invalidateQueries({ queryKey: queryKeys.groups }),
+1
View File
@@ -25,6 +25,7 @@ function baseSettings(): Settings {
level: "info",
retention_days: 30,
query_log_buffer_max: 10000,
query_log_flush_interval_s: 60,
hide_domains: false,
hide_client_ips: false,
output: "stderr",
+220 -9
View File
@@ -28,6 +28,16 @@ export interface Health {
writer_failed: boolean;
refreshes_gated: number;
snapshot_generation: number | null;
/**
* The diagnostics store's own state, not a summary of what it holds:
* `unavailable` means the store is missing or its last write failed, so the
* counts below are the last ones it managed to observe.
*/
diagnostics: {
state: "recording" | "unavailable";
active_warnings: number;
active_errors: number;
};
}
export interface Version {
@@ -50,25 +60,133 @@ export interface LogoutResponse {
authenticated: false;
}
/**
* The three closed enums `src/storage/provenance.zig` stores, as values rather
* than bare types: the copy maps in `features/queries/provenanceCopy.ts` have to
* be proven exhaustive at runtime as well as by `tsc`, exactly as
* `DIAGNOSTIC_CODES` below.
*/
export const POLICY_ACTIONS = ["not_evaluated", "allow", "block"] as const;
export type PolicyAction = (typeof POLICY_ACTIONS)[number];
export const POLICY_REASONS = [
"rule_allow_exact",
"rule_block_exact",
"rule_allow_wildcard",
"rule_block_wildcard",
"rule_allow_regex",
"rule_block_regex",
"blocklist_exception",
"blocklist_domain",
"blocklist_wildcard",
"local_record",
"forward_zone",
"non_in_class",
"paused",
"snapshot_unavailable",
"no_match",
"protocol_error",
] as const;
export type PolicyReason = (typeof POLICY_REASONS)[number];
export const ROUTE_KINDS = ["blocked", "local", "forward_zone", "upstream", "cache", "rejected"] as const;
export type RouteKind = (typeof ROUTE_KINDS)[number];
/** The summary projection the query-log table scans; full provenance is at `/api/queries/{id}`. */
export interface QueryRow {
id: number;
ts: number;
domain: string;
client_ip: string;
qtype: number | null;
qclass: number;
rcode: number;
blocked: boolean;
block_reason: string;
response_time_us: number | null;
cache_hit: boolean | null;
upstream: string;
policy_action: PolicyAction;
policy_reason: PolicyReason;
route_kind: RouteKind;
}
/** SSE `event: query` payload: a QueryRow minus `id` (precedes persistence). */
export type LiveQueryEvent = Omit<QueryRow, "id">;
export interface ProvenanceRequest {
time: number;
domain: string;
client: string;
qtype: number | null;
qclass: number;
}
/** The group as a historical fact: the id may name a group since renamed or deleted. */
export interface ProvenanceGroup {
id: number | null;
name: string;
}
export interface ProvenancePolicy {
action: PolicyAction;
reason: PolicyReason;
matched: string;
source_id: number | null;
source_name: string;
}
export interface ProvenanceRewrites {
cname_target: string;
safe_search_target: string;
}
export interface ProvenanceRoute {
kind: RouteKind;
forward_zone: string;
/** Non-empty only for an attempted upstream or forward-zone exchange; already redacted. */
upstream: string;
}
export interface ProvenanceResponse {
/** The twelve-bit EDNS extended code, not the four header bits alone. */
rcode: number;
duration_us: number | null;
}
/**
* One query, fully explained, in the order a query meets the pipeline
* (`src/web/provenance_view.zig`). Every text field follows the repository
* convention: `""` means absent.
*/
export interface Provenance {
request: ProvenanceRequest;
group: ProvenanceGroup;
policy: ProvenancePolicy;
rewrites: ProvenanceRewrites;
route: ProvenanceRoute;
response: ProvenanceResponse;
}
/** `GET /api/queries/{id}`: the same six groups plus the row id. */
export interface QueryDetail extends Provenance {
id: number;
}
/** SSE `event: query` payload: the full provenance, minus an id it cannot have yet. */
export type LiveQueryEvent = Provenance;
/**
* How much of the requested window the query log can answer for. Retention
* deletes rows and advances the watermark in one transaction, so an empty
* window is distinguishable from a pruned one.
*/
export interface Coverage {
complete: boolean;
/** The oldest instant the log is complete for, unix seconds. */
available_since: number;
}
export interface QueriesPage {
queries: QueryRow[];
next_before: number | null;
coverage: Coverage;
}
export interface QueriesFilter {
@@ -81,6 +199,77 @@ export interface QueriesFilter {
until?: number;
}
/**
* The fifteen operational event codes, in the order `src/storage/events.zig`
* declares them. A value, not only a type, because the copy map has to be
* proven exhaustive at runtime as well as by `tsc`.
*/
export const DIAGNOSTIC_CODES = [
"disk.space",
"disk.probe",
"blocklist.refresh",
"blocklist.snapshot",
"blocklist.storage",
"certificate.reload",
"query_log.write",
"query_log.maintenance",
"query_log.recreated",
"upstream_history.write",
"upstream.exchange",
"client_names.storage",
"clients.storage",
"listener.start",
"configuration.load",
] as const;
export type DiagnosticCode = (typeof DIAGNOSTIC_CODES)[number];
export type DiagnosticSeverity = "warning" | "error";
/** Which episodes a query selects. `all` is the server's default. */
export type DiagnosticState = "active" | "resolved" | "all";
export interface DiagnosticEvent {
id: number;
code: DiagnosticCode;
/** The part of `code` before the dot, repeated by the server for filtering. */
component: string;
/** The display identity of what failed; redacted where it derives from a url. */
subject: string;
severity: DiagnosticSeverity;
first_seen: number;
last_seen: number;
occurrences: number;
/** Null while the episode is still open. */
resolved_at: number | null;
detail: string;
}
export interface DiagnosticsPage {
events: DiagnosticEvent[];
next_before: number | null;
/** Episodes open right now, whatever this page filtered to. */
active: {
warnings: number;
errors: number;
};
}
/** `DELETE /api/diagnostics` — how many resolved events the purge removed. */
export interface DiagnosticsPurge {
purged: number;
}
export interface DiagnosticsFilter {
state?: DiagnosticState;
severity?: DiagnosticSeverity;
component?: string;
since?: number;
until?: number;
limit?: number;
before?: number;
}
export interface StatsTotals {
period: Period;
since: number;
@@ -90,6 +279,7 @@ export interface StatsTotals {
cached: number;
clients: number;
avg_response_time_us: number | null;
coverage: Coverage;
}
export interface Bucket {
@@ -105,6 +295,7 @@ export interface StatsTimeseries {
until: number;
bucket_seconds: number;
buckets: Bucket[];
coverage: Coverage;
}
export interface LookupResult {
@@ -119,21 +310,40 @@ export interface LookupResult {
safe_search_rewrite: string | null;
}
export interface UpstreamPeriodStats {
attempts: number;
successes: number;
failures: number;
/** successes/attempts, 0 to 1; null when attempts is 0 — no observations is not perfect reliability. */
success_rate: number | null;
/** The newest failure inside the window, unix seconds; null when the window holds none. */
last_failure_at: number | null;
/** The error name belonging to last_failure_at; null exactly when it is. */
last_failure_error: string | null;
}
export interface UpstreamHealthEntry {
url: string;
/** Live configuration, not history. */
enabled: boolean;
/** Live state; false while the upstream is backing off. */
available: boolean;
consecutive_failures: number;
total_successes: number;
total_failures: number;
success_rate: number;
last_error: string;
period: UpstreamPeriodStats;
}
export interface UpstreamHealth {
upstreams: UpstreamHealthEntry[];
period: Period;
since: number;
until: number;
available: number;
total: number;
/**
* No capacity drops known in this process within the selected window; up to about a minute of
* the newest outcomes may not have flushed yet, and outcomes lost in an unclean shutdown are
* not detectable.
*/
complete: boolean;
upstreams: UpstreamHealthEntry[];
}
export interface Group {
@@ -369,6 +579,7 @@ export interface Settings {
level: "error" | "warn" | "info" | "debug";
retention_days: number;
query_log_buffer_max: number;
query_log_flush_interval_s: number;
hide_domains: boolean;
hide_client_ips: boolean;
output: "stderr" | "syslog" | "file";
+126 -14
View File
@@ -12,15 +12,26 @@ import {
import AppShell from "@/shell/AppShell";
import { ApiError } from "@/lib/api";
import { createQueryClient } from "@/lib/queryClient";
import { diagnosticsFilterOf, type DiagnosticsSearch } from "@/features/diagnostics/filter";
import {
queriesFilterOf,
validateActivitySearch,
validateText,
validateTimestamp,
type ActivitySearch,
} from "@/features/activity/search";
import {
blocklistsQuery,
clientPrefixesQuery,
clientsQuery,
diagnosticQuery,
diagnosticsInfiniteQuery,
forwardZonesQuery,
groupsQuery,
healthQuery,
localRecordsQuery,
queriesInfiniteQuery,
queryDetailQuery,
rulesQuery,
settingsQuery,
statsQuery,
@@ -135,17 +146,72 @@ const dashboardRoute = createRoute({
component: lazyRouteComponent(() => import("@/features/dashboard/DashboardPage")),
});
const queriesRoute = createRoute({
/**
* Activity. The URL is the applied state: mode, the five filters, and nothing
* else. Everything is validated by `activity/search.ts`, so a hand-typed or
* stale parameter becomes `undefined` here rather than reaching the API as a
* value it answers 400 to.
*/
const activityRoute = createRoute({
getParentRoute: () => shellRoute,
path: "/queries",
loader: ({ context }) => context.queryClient.ensureInfiniteQueryData(queriesInfiniteQuery({})),
component: lazyRouteComponent(() => import("@/features/queries/QueryLogPage")),
path: "/activity",
validateSearch: validateActivitySearch,
// An explicit projection, not `search` itself: the router hands the loader
// whatever else the URL carried, and an unknown key would make two
// otherwise-identical loads look like different deps.
loaderDeps: ({ search }): ActivitySearch => ({
mode: search.mode,
since: search.since,
until: search.until,
domain: search.domain,
client: search.client,
blocked: search.blocked,
}),
/**
* Starts the first page in parallel with the component chunk, and does not
* wait for it. Awaiting would make every Apply a blocking navigation, which
* throws away the `keepPreviousData` placeholder the list is built on: the
* reader would lose the rows they were reading to a pending page instead of
* watching them be replaced. The page owns the loading and error surfaces,
* so the rejection is caught here only to keep it from going unhandled.
*
* Live mode reads the SSE stream and nothing else. Prefetching the log for
* it would spend a request per navigation on rows the page never renders,
* with the retained filters attached to make it look deliberate.
*/
loader: ({ context, deps }) => {
if (deps.mode !== "history") return;
void context.queryClient.ensureInfiniteQueryData(queriesInfiniteQuery(queriesFilterOf(deps))).catch(() => {});
},
component: lazyRouteComponent(() => import("@/features/activity/ActivityPage")),
});
const liveRoute = createRoute({
/**
* One logged query. Its search is the Activity search the reader arrived from,
* validated by the same functions, so the back link and every related action
* restore the exact investigation instead of a default view of it.
*/
const activityDetailRoute = createRoute({
getParentRoute: () => shellRoute,
path: "/live",
component: lazyRouteComponent(() => import("@/features/live/LiveLogPage")),
path: "/activity/queries/$id",
validateSearch: validateActivitySearch,
// Swallowed on purpose, as the diagnostics detail route does: a row
// retention has pruned is a 404 the page explains, with the way back to the
// log. The whole-page error component would call it a request failure.
loader: ({ context, params }) =>
context.queryClient.ensureQueryData(queryDetailQuery(Number(params.id))).catch(() => undefined),
component: lazyRouteComponent(() => import("@/features/activity/ActivityDetailPage")),
});
/** `domain` prefills and runs the simulation, so a query detail can link into it. */
const activityTestRoute = createRoute({
getParentRoute: () => shellRoute,
path: "/activity/test",
validateSearch: (search: Record<string, unknown>): { domain?: string } => ({
domain: validateText(search["domain"]),
}),
loader: ({ context }) => context.queryClient.ensureQueryData(groupsQuery()),
component: lazyRouteComponent(() => import("@/features/activity/PolicyTestPage")),
});
const clientsRoute = createRoute({
@@ -207,11 +273,55 @@ const upstreamsRoute = createRoute({
component: lazyRouteComponent(() => import("@/features/upstreams/UpstreamsPage")),
});
const lookupRoute = createRoute({
/**
* The filters and the window live in the url so an episode can be linked to as
* it was read a query detail links here with an absolute five-minute window
* around one query, which only means anything if the page applies it. Anything
* else in the search object is dropped: an unknown value would reach the api as
* a query parameter the handler rejects with a 400.
*/
const diagnosticsRoute = createRoute({
getParentRoute: () => shellRoute,
path: "/lookup",
loader: ({ context }) => context.queryClient.ensureQueryData(groupsQuery()),
component: lazyRouteComponent(() => import("@/features/lookup/LookupPage")),
path: "/diagnostics",
validateSearch: (search: Record<string, unknown>): DiagnosticsSearch => {
const state = search["state"];
const severity = search["severity"];
return {
state: state === "active" || state === "resolved" ? state : undefined,
severity: severity === "warning" || severity === "error" ? severity : undefined,
component: validateText(search["component"]),
since: validateTimestamp(search["since"]),
until: validateTimestamp(search["until"]),
};
},
loaderDeps: ({ search }): DiagnosticsSearch => ({
state: search.state,
severity: search.severity,
component: search.component,
since: search.since,
until: search.until,
}),
// allSettled: the two sections render their own state, and the resolved
// history failing must not replace the active list with the error page.
loader: ({ context, deps }) => {
const base = diagnosticsFilterOf(deps);
return Promise.allSettled([
context.queryClient.ensureInfiniteQueryData(diagnosticsInfiniteQuery({ ...base, state: "active" })),
context.queryClient.ensureInfiniteQueryData(diagnosticsInfiniteQuery({ ...base, state: "resolved" })),
]);
},
component: lazyRouteComponent(() => import("@/features/diagnostics/DiagnosticsPage")),
});
const diagnosticDetailRoute = createRoute({
getParentRoute: () => shellRoute,
path: "/diagnostics/$id",
// Swallowed on purpose: an event retention has removed is a 404 the page
// itself explains, with the way back to the list. The whole-page error
// component would state it as a request failure instead.
loader: ({ context, params }) =>
context.queryClient.ensureQueryData(diagnosticQuery(Number(params.id))).catch(() => undefined),
component: lazyRouteComponent(() => import("@/features/diagnostics/DiagnosticDetailPage")),
});
const settingsRoute = createRoute({
@@ -225,15 +335,17 @@ const routeTree = rootRoute.addChildren([
loginRoute,
shellRoute.addChildren([
dashboardRoute,
queriesRoute,
liveRoute,
activityRoute,
activityDetailRoute,
activityTestRoute,
clientsRoute,
groupsRoute,
blocklistsRoute,
rulesRoute,
localDnsRoute,
upstreamsRoute,
lookupRoute,
diagnosticsRoute,
diagnosticDetailRoute,
settingsRoute,
]),
]);
+12 -4
View File
@@ -7,15 +7,14 @@ import { createAppRouter } from "@/routes";
const NAV_LABELS = [
"Dashboard",
"Query Log",
"Live",
"Activity",
"Clients",
"Groups",
"Blocklists",
"Rules",
"Local DNS",
"Upstreams",
"Lookup",
"Diagnostics",
"Settings",
];
@@ -29,8 +28,16 @@ const RESPONSES: Record<string, unknown> = {
cached: 0,
clients: 0,
avg_response_time_us: null,
coverage: { complete: true, available_since: 0 },
},
"/api/stats/timeseries?period=24h": {
period: "24h",
since: 0,
until: 86400,
bucket_seconds: 1800,
buckets: [],
coverage: { complete: true, available_since: 0 },
},
"/api/stats/timeseries?period=24h": { period: "24h", since: 0, until: 86400, bucket_seconds: 1800, buckets: [] },
"/api/health": {
status: "ok",
disk: { state: "ok", free_bytes: 0, db_bytes: 0, log_bytes: 0, sample_failures: 0 },
@@ -39,6 +46,7 @@ const RESPONSES: Record<string, unknown> = {
writer_failed: false,
refreshes_gated: 0,
snapshot_generation: null,
diagnostics: { state: "recording", active_warnings: 0, active_errors: 0 },
},
"/api/upstream/health": { upstreams: [], available: 1, total: 1 },
"/api/version": { version: "0.0.0-test", git_commit: "0000000", zig_version: "0.16.0", uptime_seconds: 1 },
+2 -3
View File
@@ -17,15 +17,14 @@ const DARK = "@media (prefers-color-scheme: dark)";
const NAV_ITEMS = [
{ to: "/", label: "Dashboard" },
{ to: "/queries", label: "Query Log" },
{ to: "/live", label: "Live" },
{ to: "/activity", label: "Activity" },
{ to: "/clients", label: "Clients" },
{ to: "/groups", label: "Groups" },
{ to: "/blocklists", label: "Blocklists" },
{ to: "/rules", label: "Rules" },
{ to: "/local-dns", label: "Local DNS" },
{ to: "/upstreams", label: "Upstreams" },
{ to: "/lookup", label: "Lookup" },
{ to: "/diagnostics", label: "Diagnostics" },
{ to: "/settings", label: "Settings" },
] as const;
+34
View File
@@ -0,0 +1,34 @@
import { fireEvent, render, screen } from "@testing-library/react";
import Select from "./Select";
const OPTIONS = [
{ value: "any", label: "All" },
{ value: "blocked", label: "Blocked only" },
];
/** RAC opens a Select from the keyboard as readily as from a pointer. */
function open(trigger: HTMLElement) {
fireEvent.keyDown(trigger, { key: "Enter" });
fireEvent.keyUp(trigger, { key: "Enter" });
}
test("a disabled select keeps its value on screen but takes no input", () => {
const onChange = vi.fn();
render(<Select label="Status" options={OPTIONS} value="blocked" onChange={onChange} isDisabled />);
const trigger = screen.getByRole("button");
expect(trigger.textContent).toContain("Blocked only");
// A disabled button is out of the tab order by definition, so the filter row
// cannot be reached by keyboard while live mode owns it.
expect(trigger).toHaveProperty("disabled", true);
open(trigger);
expect(screen.queryByRole("listbox")).toBeNull();
expect(onChange).not.toHaveBeenCalled();
});
test("an enabled select still opens", () => {
render(<Select label="Status" options={OPTIONS} value="any" onChange={vi.fn()} />);
open(screen.getByRole("button"));
expect(screen.getByRole("listbox")).toBeTruthy();
});
+16 -2
View File
@@ -32,6 +32,8 @@ interface Props {
* dialog uses, `inline` a control sitting in a row of other controls.
*/
variant?: "field" | "compactField" | "inline";
/** Visible but inert, keeping its value on screen; RAC also drops it from the tab order. */
isDisabled?: boolean;
}
const styles = stylex.create({
@@ -50,7 +52,10 @@ const styles = stylex.create({
justifyContent: "space-between",
gap: "0.5rem",
textAlign: "left",
cursor: "pointer",
// RAC renders a real `<button disabled>`, so the state is reachable as a
// pseudo-class rather than needing a second style object.
cursor: { default: "pointer", ":disabled": "not-allowed" },
opacity: { default: null, ":disabled": 0.55 },
},
compact: {
marginTop: "0.25rem",
@@ -105,12 +110,21 @@ const styles = stylex.create({
},
});
export default function Select({ options, value, onChange, label, "aria-label": ariaLabel, variant = "field" }: Props) {
export default function Select({
options,
value,
onChange,
label,
"aria-label": ariaLabel,
variant = "field",
isDisabled = false,
}: Props) {
const base = variant === "field" ? shared.input : shared.smallInput;
const block = variant === "compactField" ? styles.compact : null;
return (
<AriaSelect
aria-label={ariaLabel}
isDisabled={isDisabled}
value={value}
onChange={(key) => onChange(String(key ?? ""))}
{...stylex.props(styles.root)}
+13 -3
View File
@@ -169,8 +169,10 @@ export const styles = stylex.create({
},
/**
* A learned name is runtime state, not something the operator typed, so it
* reads muted and carries an outlined "learned" tag. The tag is real text
* a screen reader announces it because colour alone is not an affordance.
* reads muted wherever it appears. The Clients page adds the outlined tag,
* where the client is named once; the query tables leave it off rather than
* repeat it on every row. The tag is real text a screen reader announces
* it because colour alone is not an affordance.
*/
learnedName: {
color: colors.textMuted,
@@ -191,7 +193,15 @@ export const styles = stylex.create({
mono: {
fontFamily: "ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, monospace",
},
/** Visible to a screen reader only; the element keeps its place in the a11y tree. */
/**
* Visible to a screen reader only; the element keeps its place in the a11y tree.
*
* Apply it to a block container. `overflow` has no effect on a table box, and
* `height` on one is a minimum, so a `<table>` wearing this still lays out at
* its full content height and pushes the page's scrollable overflow past the
* app shell invisible, because `clip-path` still hides the paint. Wrap the
* table in a hidden `<div>` instead of hiding the table itself.
*/
srOnly: {
position: "absolute",
width: 1,
+31
View File
@@ -269,6 +269,37 @@ pub fn build(b: *std.Build) void {
});
test_step.dependOn(&b.addRunArtifact(container_check_tests).step);
// The release cut (specs/release-cut.md). A host tool like the two above,
// but run from the build graph rather than installed: it takes a bump kind
// on the command line (`zig build cut -- patch`), reads its own token, and
// needs the operator's terminal so `git commit -S` can reach pinentry —
// none of which a workflow supplies and all of which a Run step passes
// through.
const cut_tool = hostTool(b, "cut");
const cut_run = b.addRunArtifact(cut_tool);
// It pushes commits and tags, so it must never be answered from the run
// cache, and it must run at the build root whatever directory `zig build`
// was invoked from.
cut_run.has_side_effects = true;
cut_run.stdio = .inherit;
cut_run.setCwd(b.path("."));
if (b.args) |args| cut_run.addArgs(args);
b.step("cut", "Cut a release: preflight, bump, push, wait for CI, signed tag, watch the run")
.dependOn(&cut_run.step);
// Its pure decisions — semver strictness, the zon rewrite, the changelog
// section check, the runs-payload read and the tea-config token lookup —
// are the reason it is a program rather than a shell script.
const cut_tests = b.addTest(.{
.name = "cut-tool",
.root_module = b.createModule(.{
.root_source_file = b.path("tools/cut.zig"),
.target = b.graph.host,
.optimize = optimize,
}),
});
test_step.dependOn(&b.addRunArtifact(cut_tests).step);
addDist(b, options, admin_assets, .{
.version = version_option,
.version_string = version_string,
+1 -1
View File
@@ -1,6 +1,6 @@
.{
.name = .nxdns,
.version = "0.0.4",
.version = "0.0.8",
.minimum_zig_version = "0.16.0",
.paths = .{""},
.fingerprint = 0x3307b311dded1d91,
+33 -6
View File
@@ -4,7 +4,7 @@ nxdns serves its admin API itself, on `web.bind:web.port` (default port 8080), a
The machine-readable contract is `src/web/openapi.yaml`, which the running server hands out unauthenticated at `GET /api/openapi.yaml`. Request and response schemas for every operation live there. When this page and the YAML disagree, the YAML wins.
The route table is `src/web/routes.zig`; the [Operations](#operations) table below carries all 56 of its entries.
The route table is `src/web/routes.zig`; the [Operations](#operations) table below carries all 61 of its entries.
## Conventions
@@ -45,7 +45,7 @@ A token bucket per client address: capacity and refill are both `web.api_rate_li
`GET /api/queries/live` is server-sent events over chunked transfer, `Content-Type: text/event-stream`, `Cache-Control: no-store`.
- The stream opens with `retry: 3000`, so a browser `EventSource` reconnects on its own after a drop.
- Each query is one frame: `event: query` and a single `data:` line of JSON. The payload carries the `GET /api/queries` row fields minus `id` (a live entry precedes persistence): `ts`, `domain`, `client_ip`, `qtype`, `blocked`, `block_reason`, `response_time_us`, `cache_hit`, `upstream`.
- Each query is one frame: `event: query` and a single `data:` line of JSON. The payload is the `Provenance` object — the body of `GET /api/queries/{id}` without its `id`, which does not exist yet because a live entry precedes its own insert. Its six groups are `request`, `group`, `policy`, `rewrites`, `route` and `response`.
- A `: ping` comment heartbeat goes out after 15 s of quiet, keeping middleboxes from reaping the idle connection.
- Each subscriber buffers up to 64 entries. A client too slow for the query rate overflows its buffer and the server ends the stream cleanly after delivering what the buffer held — queries are never held back for a slow reader. There is no gap marker: on reconnect, re-sync through `GET /api/queries`, which has the missed rows.
- Connections per client address are capped at `web.sse_max_connections_per_ip` (default 3); over the cap is a 429. The cap binds loopback too. The server holds at most 32 concurrent streams in total; when all slots are taken, the answer is a 503.
@@ -104,11 +104,16 @@ Auth `open` means no session is required; `session` means a valid session cookie
| POST | `/api/auth/login` | open | counted | runtime action | Log in |
| POST | `/api/auth/logout` | session | counted | runtime action | Log out |
| GET | `/api/queries` | session | counted | read | Query log page |
| GET | `/api/queries/{id}` | session | counted | read | One query, fully explained |
| GET | `/api/queries/live` | session | exempt | read | Live query stream (server-sent events) |
| GET | `/api/stats` | session | counted | read | Totals for a period |
| GET | `/api/stats/timeseries` | session | counted | read | Bucketed counts for a period |
| GET | `/api/lookup` | session | counted | read | Explain a domain |
| GET | `/api/upstream/health` | session | counted | read | Upstream pool health |
| GET | `/api/diagnostics` | session | counted | read | Operational event log |
| DELETE | `/api/diagnostics` | session | counted | runtime action | Purge every resolved event |
| GET | `/api/diagnostics/{id}` | session | counted | read | One operational event |
| DELETE | `/api/diagnostics/{id}` | session | counted | runtime action | Purge one resolved event |
| GET | `/api/groups` | session | counted | read | List groups |
| POST | `/api/groups` | session | counted | config write | Create a group |
| GET | `/api/groups/{id}` | session | counted | read | Read a group |
@@ -156,6 +161,8 @@ Auth `open` means no session is required; `session` means a valid session cookie
There is no `POST /api/clients`: client rows come from DNS activity or import, never from the API.
The two diagnostics deletes purge history only. `DELETE /api/diagnostics/{id}` answers 204 for a resolved event, 409 for one that is still active — an open episode is the current state of the box, not history — and 404 for an id no row holds. `DELETE /api/diagnostics` removes every resolved event and answers `{"purged": N}`, leaving the active ones. Events still resolve on their own; these only decide when the resolved rows go.
Static assets are not routes. The router sends unmatched non-`/api` paths to the embedded SPA before any auth or rate-limit check.
## Settings keys
@@ -168,9 +175,11 @@ In file mode `PUT /api/settings` is refused with the 403 above, password changes
Request and response schemas for every operation live in the OpenAPI document: `src/web/openapi.yaml` in the repository, or `GET /api/openapi.yaml` from a running server.
### Block reasons
### Policy reasons
Three places carry the same tag: `block_reason` on a `GET /api/queries` row, `block_reason` on a live-stream frame, and `reason` on a `GET /api/lookup` answer. The tag names the level that decided the query, and the levels are listed here in the order they are consulted — the first one that matches wins, so a rule always outranks a list.
Two places carry the same closed set of tags: `policy_reason` on a `GET /api/queries` row and on a `GET /api/queries/{id}` body (where it is `policy.reason`, and where the live stream sends the same field), and `reason` on a `GET /api/lookup` answer. The tag names what decided the query.
The first nine are the matcher's own verdicts, listed in the order they are consulted — the first that matches wins, so a rule always outranks a list.
| Tag | Decided by |
| --- | --- |
@@ -184,6 +193,24 @@ Three places carry the same tag: `block_reason` on a `GET /api/queries` row, `bl
| `blocklist_domain` | A plain name in a downloaded list |
| `blocklist_wildcard` | A domain anchor (`||name^`) in a downloaded list |
`/api/lookup` also answers `none` when nothing matched. A query row never carries `none`: `block_reason` is null unless the query was blocked.
The rest name a pipeline step that answered the query without consulting the matcher, and appear on a query row only.
A `cname:` prefix means the decision landed on a CNAME target rather than on the name the client asked for, so `cname:blocklist_domain` reads as "the list blocks a name this answer redirects to". Only `/api/queries` and the live stream show the prefix; `/api/lookup` does not follow CNAMEs.
| Tag | Decided by |
| --- | --- |
| `local_record` | A configured local record, answered before filtering |
| `forward_zone` | A configured forward zone, answered before filtering |
| `non_in_class` | The question was not class IN, so no rule could apply |
| `paused` | Filtering was paused |
| `snapshot_unavailable` | No filter snapshot was published yet, so the query went unfiltered |
| `no_match` | The matcher evaluated the name and nothing matched |
| `protocol_error` | A parsed request refused on protocol grounds — BADVERS, NOTIMP, a malformed EDNS OPT |
`policy_action` says which way the verdict went: `block`, `allow`, or `not_evaluated` for a query answered before any policy could apply. `/api/lookup` answers `none` when nothing matched, where a query row says `no_match`.
`route_kind` says where the answer came from: `blocked`, `local`, `forward_zone`, `upstream`, `cache` or `rejected`.
A non-empty `rewrites.cname_target` on a query detail means the decision landed on a CNAME target rather than on the name the client asked for; `policy.reason` is then the target's own reason. `/api/lookup` does not follow CNAMEs.
### Coverage
`GET /api/queries`, `GET /api/stats` and `GET /api/stats/timeseries` each answer with a `coverage` object: `available_since` is the oldest instant the query log is still complete for, and `complete` is true only when the window the request asked about starts at or after it. Retention deletes rows and advances the watermark in one transaction, so a client can tell an empty window from a pruned one instead of charting the gap as zero. A request with no lower bound at all asks about the whole of history, and is never complete.
+19 -2
View File
@@ -131,7 +131,8 @@ Process log and query log behavior.
|---|---|---|---|---|---|
| `logging.level` | enum `.err` \| `.warn` \| `.info` \| `.debug` | `.info` | — | one of the four tags; stored as `"error"` / `"warn"` / `"info"` / `"debug"` | log threshold (`src/platform/logging.zig`) |
| `logging.retention_days` | u16 | 30 | days | at least 1 | query-log pruning cutoff (`src/storage/retention.zig`) and the client tracker's last-seen cutoff (`src/server/clients.zig`) |
| `logging.query_log_buffer_max` | u32 | 10000 | entries | 11000000 | in-memory query-log ring size and backpressure cap (`src/storage/logger.zig`) |
| `logging.query_log_buffer_max` | u32 | 10000 | entries | 137449 | in-memory query-log ring size and backpressure cap (`src/storage/logger.zig`); the ceiling is derived at compile time from `@sizeOf(logger.Entry)` so the queue's worst case stays within 64 MiB, and it moves whenever the entry's width does |
| `logging.query_log_flush_interval_s` | u16 | 60 | seconds | 03600 | how long the query-log writer gathers entries before committing them in one transaction (`src/storage/logger.zig`); see the note below |
| `logging.hide_domains` | bool | false | — | — | the query log stores a hidden marker instead of the domain |
| `logging.hide_client_ips` | bool | false | — | — | the query log stores a hidden marker instead of the client address |
| `logging.output` | enum `.stderr` \| `.syslog` \| `.file` | `.stderr` | — | one of the three tags | log sink selection (`src/platform/logging.zig`); `.stderr` and `.syslog` both write to stderr (journald captures it), `.file` rotates |
@@ -139,6 +140,19 @@ Process log and query log behavior.
| `logging.max_size_mb` | u32 | 50 | MiB | at least 1 | rotation trigger for the log file |
| `logging.max_files` | u8 | 5 | files | at least 1 | log files kept in total, the live one included, so the highest rotated generation is `max_files - 1`; the default 5 keeps `nxdns.log` plus `nxdns.log.1` through `nxdns.log.4`, and a value of 1 keeps only the live file, which rotation deletes rather than renames |
#### What `query_log_flush_interval_s` costs and buys
The query-log writer commits one transaction per interval instead of one per query. At a household's query rate — a few queries a second at most, often a fraction of one — a per-query commit writes hundreds of times more bytes to the disk than the rows themselves occupy, because every commit rewrites the WAL frames, the WAL index and the page headers whatever the row size. That write amplification is what wears out an SD card, and the default of 60 seconds is what stops it. It matches Pi-hole's `DBinterval`, which defaults to the same minute for the same reason.
What it costs:
- **Crash-loss window.** A process that dies takes roughly `interval` seconds of query history with it. That is the normal case, not a guaranteed maximum: a batch the disk monitor is holding back (free space below the critical threshold) or one waiting on a database write lock can be considerably older when the process dies. Power loss can additionally lose recent committed transactions, because `querylog.db` runs with WAL and `synchronous=NORMAL` — that was already true at any interval, and setting `0` does not buy per-query durability. Query history is the least valuable data on this box: nothing else depends on it, and it is deleted by retention anyway.
- **Staleness.** Every read backed by the query log — the query-log page, the dashboard totals, the timeseries — lags about `interval` seconds behind, and further behind while writes are gated or slow. The live view does not lag: it is fed from the SSE hub before the queue, so queries appear there the moment they are answered.
`0` means "do not wait": the writer commits the entry that woke it together with whatever is already queued, up to 100 rows. Use it when you want the query-log page to be current to the second and you do not care what that costs the disk.
Two things do not change with the interval: a batch is capped at 100 rows, so a burst is committed as soon as it fills one rather than waiting out the window, and shutdown writes what the writer is holding instead of waiting for the interval to end.
### disk
Free-space thresholds for the data directory. Below them the query-log writer, the client tracker and the blocklist scheduler are throttled (`src/storage/disk_monitor.zig`); DNS resolution is never gated.
@@ -371,10 +385,12 @@ The error set is `validate.ValidateError` in `src/config/validate.zig`:
| `MissingDefaultGroup` | no group is named `default` |
| `DuplicateGroupName` | two groups share a `name` |
| `EmptyGroupName` | a group `name` is empty |
| `GroupNameTooLong` | a group `name` is longer than 64 bytes; it is copied into every logged query |
| `UnknownGroup` | a client, prefix, group source or rule names a group that is not declared |
| `BadClientIp` / `DuplicateClientIp` | a client `ip` is unparseable, or collides after canonicalization |
| `BadClientPrefix` / `DuplicateClientPrefix` | the same for a `client_prefixes.prefix` |
| `BadSourceUrl` / `DuplicateSourceUrl` / `EmptySourceName` | blocklist source fields |
| `SourceNameTooLong` | a blocklist source `name` is longer than 64 bytes; it is copied into every logged query |
| `UnknownSource` / `DuplicateGroupSource` | `group_sources` links |
| `BadRulePattern` | a rule `pattern` does not match its `kind` |
| `BadLocalRecordName` / `BadLocalRecordValue` / `DuplicateLocalRecord` | local record fields |
@@ -383,7 +399,8 @@ The error set is `validate.ValidateError` in `src/config/validate.zig`:
| `BadTimeout` | a timeout is outside 100120000 ms, or `attempt` is above `total` |
| `BadTtl` | `blocking.ttl`, `cache.negative_ttl_max`, a record `ttl`, `web.session_ttl_hours` or `blocklist_update.interval_hours` outside its range |
| `BadCacheSize` | `cache.size` outside 11000000 |
| `BadRetention` | `logging.retention_days` below 1, or `logging.query_log_buffer_max` outside 11000000 |
| `BadRetention` | `logging.retention_days` below 1, or `logging.query_log_buffer_max` outside 137449 |
| `BadFlushInterval` | `logging.query_log_flush_interval_s` above 3600 |
| `BadLogRotation` | `logging.max_size_mb` or `logging.max_files` below 1 |
| `BadDiskThresholds` | a threshold below 1, or `min_free_mb` above `warn_free_mb` |
| `BadRateLimit` | `dns.rate_limit`, `dns.rate_window_seconds`, `web.api_rate_limit_per_min` or `web.sse_max_connections_per_ip` out of range |
+16 -3
View File
@@ -18,8 +18,8 @@ Default `/var/lib/nxdns`, overridable with `--data-dir DIR`. `nxdns run` and `nx
| `config.db-wal`, `config.db-shm` | SQLite write-ahead log and shared-memory index for `config.db`. Created by `run`, `import` and `export` when WAL is enabled, inheriting the main file's permissions. `check` creates neither. | 0600 |
| `querylog.db` | The query log: every domain every client asked for. Expendable — if it is missing or unusable it is recreated empty. | 0600 |
| `querylog.db-wal`, `querylog.db-shm` | WAL sidecars for `querylog.db`. | 0600 |
| `querylog.db.corrupt-<unix-seconds>` | A `querylog.db` that could not be used, moved aside before an empty one was created in its place. Kept, never overwritten. | Whatever the renamed file had — no chmod reaches it |
| `querylog.db.corrupt-<unix-seconds>-<n>` | The same, when the plain name is taken — `<n>` counts from 1 and rises until the name is free. Two recreates within one second is the case it exists for. | The same |
| `querylog.db.<reason>-<unix-seconds>` | A `querylog.db` this build could not use, moved aside before an empty one was created in its place. Kept, never overwritten. `<reason>` is one of `corrupt`, `not-a-database`, `quick-check-failed` or `schema-changed`; see [why a query log is moved aside](#why-a-query-log-is-moved-aside). | Whatever the renamed file had — no chmod reaches it |
| `querylog.db.<reason>-<unix-seconds>-<n>` | The same, when the plain name is taken — `<n>` counts from 1 and rises until the name is free. Two recreates within one second is the case it exists for. | The same |
| `blocklists/` | Compiled blocklist snapshots, one subdirectory of the data directory. | 0700 |
| `blocklists/<id>.list` | Exact domains for blocklist source `<id>`, one per line, behind a header. | 0600 |
| `blocklists/<id>.wild` | Wildcard entries for the same source. | 0600 |
@@ -50,7 +50,20 @@ A failed sweep is a warning, not an outage — leftover bytes do not justify los
The temporaries of a source that still exists are cleaned by the refresh that owns them rather than by the sweep: each refresh deletes its own `.raw.tmp`, `.list.tmp`, `.wild.tmp` and `.allow.tmp` as it finishes, successfully or not.
A `querylog.db` is moved aside when it is missing nothing but usability: SQLite reports it corrupt or not a database, `PRAGMA quick_check` does not answer `ok`, or its `user_version` fingerprint does not match the schema. Only the main file is renamed — its `-wal` and `-shm` are deleted, because a stale WAL would be replayed into the fresh database. A missing `querylog.db` is created without any aside file. The rename happens inside `querylog_schema.open`, before the 0600 chmod, and that chmod names `querylog.db` and its two sidecars only — so an aside file keeps the mode the file had at rename time, which for a `querylog.db` nxdns itself created is 0600 and for one an operator put there is whatever they left it at. Nothing prunes the aside files; they accumulate until an operator removes them, and each one holds the same browsing history the live query log holds.
### Why a query log is moved aside
A `querylog.db` is moved aside when it is missing nothing but usability, and the name it is given says which of the four cases it hit:
| `<reason>` | What happened |
| --- | --- |
| `corrupt` | SQLite reported the file as damaged. |
| `not-a-database` | The file is not a SQLite database at all. |
| `quick-check-failed` | `PRAGMA quick_check` did not answer `ok`. |
| `schema-changed` | Nothing is wrong with the file. Its `user_version` fingerprint does not match this build's schema, so this build cannot read it. Upgrades that touch the query-log schema produce this one, and the file they set aside is a healthy database. |
Only the main file is renamed — its `-wal` and `-shm` are deleted, because a stale WAL would be replayed into the fresh database. A missing `querylog.db` is created without any aside file. The rename happens inside `querylog_schema.open`, before the 0600 chmod, and that chmod names `querylog.db` and its two sidecars only — so an aside file keeps the mode the file had at rename time, which for a `querylog.db` nxdns itself created is 0600 and for one an operator put there is whatever they left it at. Nothing prunes the aside files; they accumulate until an operator removes them, and each one holds the same browsing history the live query log holds.
### Why the databases are 0600
The 0600 modes are not cosmetic. `config.db` holds the argon2id password hash and `querylog.db` holds the browsing history of every client on the LAN, so both are as sensitive as each other, and a WAL file holds the same rows as the database it belongs to. SQLite creates the main database at `0644 & ~umask`; nxdns chmods it to 0600 before enabling WAL, so the sidecars inherit 0600 rather than being created world-readable.
+52
View File
@@ -0,0 +1,52 @@
# Repetitive workflows. Anything a release or a review runs twice belongs here,
# so a step cannot be forgotten by hand.
#
# Recipes only. Every decision the release makes lives in tools/cut.zig, which
# `zig build test` type-checks and covers; this file exists so nobody has to
# remember the invocation.
# every recipe, described
default:
@just --list
# unit suite
test:
zig build test
# unit + integration suite, which already runs the whole ordinary suite
itest:
zig build test -Dintegration
# The committed npm scripts, never npx: npx can fetch an unpinned package.
# admin: typecheck, tests, lint, formatting
admin-check:
cd admin && npm run typecheck && npm run test && npm run lint && npm run format:check
# admin/src/lib/contractSamples.gen.ts is a committed golden of live API bodies
# the admin tests assert against; this is the invocation AGENTS.md documents.
# regenerate the admin contract goldens from live responses
goldens:
zig build test -Dintegration -Dcontract-samples-out="$PWD/admin/src/lib/contractSamples.gen.ts"
# the binary with the real admin UI embedded (a plain `zig build` embeds a placeholder page)
build:
cd admin && npm run build
zig build -Dadmin-dist=admin/dist
# Not the CI gate: this skips the admin `npm run build` and `assert-bundled`,
# the cross-target builds and verify-dist.
# the fast local checks
verify: itest admin-check
zig fmt --check build.zig src tools
# kind is major, minor or patch: the version itself is derived from
# build.zig.zon, never typed, because a published tag cannot be corrected.
# Requires a clean tree, master, and a dated CHANGELOG.md section for the
# derived version. Preflight, bump, push, wait for CI, signed tag, watch the run.
# cut a release
release kind:
zig build cut -- {{kind}}
+315
View File
@@ -0,0 +1,315 @@
# Milestone 26: upstream health honors the dashboard period
The dashboard's upstream numbers must respect the selected period. Today `GET /api/upstream/health` serves process-lifetime counters and a last-32-exchanges window beside a period picker that scopes everything else on the page; "63 failures, 100.0% success rate" side by side was the result. This milestone records per-minute upstream outcome history in `querylog.db` and serves ranged aggregates from it, keeping the in-memory health state for routing, backoff and `/metrics` only.
Design authority: the Codex design review of 2026-08-17 (adopted whole). Where this spec deviates from it, the deviation is named in the ruling that makes it.
## Implementation contract (read first)
- Verify every stdlib claim against `/home/mokhtar/app/zig` at tag 0.16.0. Pre-0.16 knowledge is stale.
- m13 ruling F-f binds every session: every behavior ships with a test the author watched fail — run the assertion before the code, or with the code reverted, and say so in the report.
- No new `std.log.err`. Counters and the health rollup are the failure surface; each new failure path logs at most one `warn` per pass.
- `admin/src/lib/contractSamples.gen.ts` is regenerated with the AGENTS.md command, never hand-edited, in the same session that changes `src/web/openapi.yaml`.
- Do not commit. The orchestrator commits after review, spec sync, and the user's screenshot approval.
## Rulings (binding)
### 1. History is event aggregation, not counter sampling
Outcomes are aggregated into their wall-clock UTC minute at the moment they are recorded, and the aggregates are flushed to storage. Nothing samples `total_successes`/`total_failures` and subtracts. Consequences a test must prove:
- a restart cannot invert or reset any persisted number (rows are additive facts; a restart within the same minute adds to the same row);
- a crash loses at most the currently unflushed aggregates — bounded undercount, never a negative delta;
- the failure that happened inside a period is reportable with its error name, because the minute row carries it.
### 2. Storage: two tables in `querylog.db`, identity is the URL
Appended to `querylog_schema.ddl` (which changes the DDL fingerprint; by the established policy — querylog_schema.zig:1-7, PLAN §3.7 — every existing `querylog.db` fails the fingerprint check on upgrade and is recreated, with the previous file renamed aside as `querylog.db.corrupt-<timestamp>` rather than deleted; the changelog must state that query history restarts empty and where the old file sits):
```sql
CREATE TABLE upstream_targets (
id INTEGER PRIMARY KEY,
url TEXT NOT NULL UNIQUE
);
CREATE TABLE upstream_minute (
upstream_id INTEGER NOT NULL REFERENCES upstream_targets(id),
minute_ts INTEGER NOT NULL,
successes INTEGER NOT NULL,
failures INTEGER NOT NULL,
last_failure_ts INTEGER,
last_error TEXT,
PRIMARY KEY (upstream_id, minute_ts),
CHECK (successes >= 0),
CHECK (failures >= 0)
) WITHOUT ROWID;
CREATE INDEX idx_upstream_minute_ts ON upstream_minute(minute_ts);
```
- `minute_ts` is the UTC minute start in seconds (`@divFloor(ts, 60) * 60`), stamped from `std.Io.Clock.real` — history participates in wall-clock periods, so it uses the wall clock. Routing state stays on `.awake` untouched.
- Identity is the URL through a dimension table, not the `config.db` upstream id: ids cannot be foreign keys across database files and may be deleted or reused; the URL is the immutable historical identity the query log already uses. A URL edit deliberately starts a new history.
- Rows exist only for minutes that had at least one attempt. Bound: upstreams × 1440 rows/day.
- A deleted upstream's history stays until retention removes it and is simply not returned (ruling 6 returns current pool entries only).
### 3. Recording: accumulate in memory at the record points, never touch SQLite on the query path
New module `src/upstream/history.zig`:
```zig
pub const max_pending = 4096;
pub const Accumulator = struct {
pub const Cell = struct {
url: []const u8, // borrowed from the pool entry's endpoint; entries live for the process
minute_ts: i64,
successes: u32,
failures: u32,
last_failure_ts: ?i64,
last_error_buf: [48]u8, // the error-name capacity health.State.last_error_buf uses (health.zig:58); S1 should lift 48 into a shared pub const rather than duplicate the literal
last_error_len: u8,
};
pub const Stats = struct {
flushes: u64 = 0,
flush_failures: u64 = 0,
rows_dropped: u64 = 0,
pending: u32 = 0,
};
mutex: std.Io.Mutex = .init,
cells: [max_pending]Cell,
count: u32,
last_drop_minute: ?i64, // max minute_ts ever dropped; feeds `complete` (ruling 6)
last_flush_failed: bool, // set by a failed flush, cleared by the next successful one (ruling 7)
// counters atomic, same shape as retention.zig's Counters
/// The declared init: count 0, last_drop_minute null, last_flush_failed false,
/// every counter zero, cells undefined (a cell is written before it is read).
pub const init: Accumulator = ...;
pub fn recordSuccess(self: *Accumulator, io: std.Io, url: []const u8, wall_s: i64) void;
pub fn recordFailure(self: *Accumulator, io: std.Io, url: []const u8, wall_s: i64, error_name: []const u8) void;
/// The only read surface. Web, health and metrics consumers read through
/// this mutex-protected snapshot; the fields above are private to the module.
pub fn snapshotStats(self: *Accumulator, io: std.Io) Stats; // Stats gains last_drop_minute: ?i64, last_flush_failed: bool
};
```
- `Pool.recordSuccess` / `Pool.recordFailure` (pool.zig:309, :318) gain, after their existing health bookkeeping and **after releasing the pool mutex**, a call into an optional `history: ?*history_mod.Accumulator` field on `Pool`, passing `entry.endpoint.url` and `std.Io.Clock.real.now(io).toSeconds()`. Restructure both wrappers so the health update sits in an inner block whose close releases the pool mutex, and the history call follows the block; a comment on each states that ordering is the constraint. The accumulator has its own mutex (`lockUncancelable`, mirroring the health sections' reasoning at pool.zig:310-313); no lock is ever held while taking the other, so no ordering deadlock exists. Proven by: (1) an S1 unit test that wires an `Accumulator` into a `Pool` built from the existing `Fake` client machinery (pool.zig tests), drives one success and one failure, and asserts both outcomes landed in the accumulator's cells; (2) the S1 report citing the restructured wrapper bodies showing the call after the mutex scope closes — the earlier draft's "hook takes the pool mutex" deadlock test is withdrawn as not runnable without production callback seams.
- Lookup is linear over the live cells (url pointer equality first, then bytes; at household scale the live set is a handful). On a miss with `count == max_pending`, evict the cell with the oldest `minute_ts`, set `last_drop_minute = @max(last_drop_minute orelse evicted.minute_ts, evicted.minute_ts)` (never plain assignment — a merge-back after eviction must not move the watermark backwards), and bump `rows_dropped`. Dropping is the overflow behavior, retrying is the flush-failure behavior — the two must not be conflated.
- On failure, keep the newest `last_failure_ts` and its error name in the cell (same max-wins rule the SQL upsert applies).
### 4. Flushing: a dedicated task, one transaction per pass, additive upserts
New repo `src/storage/repositories/upstream_history_repo.zig` — S1 owns the whole repository contract including the ranged read; S2 consumes it and defines nothing of its own SQL:
```zig
pub const FlushRow = struct { url: []const u8, minute_ts: i64, successes: u32, failures: u32, last_failure_ts: ?i64, last_error: []const u8 };
pub fn flush(database: *db.Db, rows: []const FlushRow) db.Error!void; // one Tx: ensure targets, then upsert minutes
pub const WindowStats = struct {
attempts: u64,
successes: u64,
failures: u64,
last_failure_ts: ?i64,
/// The error name of the row holding the newest last_failure_ts in the
/// window; "" when the window holds no failure.
last_failure_error_buf: [48]u8,
last_failure_error_len: u8,
};
pub fn windowStats(database: *db.Db, url: []const u8, since: i64, until: i64) db.Error!WindowStats;
pub fn pruneOlderThan(database: *db.Db, cutoff_ts: i64) db.Error!i64;
```
`windowStats` is **one statement** — two statements would not share a SQLite snapshot, and the flush connection committing between them could pair `max(last_failure_ts)` from one state with an error text from another. The error lookup is a scalar subquery inside the same statement, with a deterministic tiebreak on its ORDER BY:
```sql
SELECT coalesce(sum(m.successes), 0), coalesce(sum(m.failures), 0), max(m.last_failure_ts),
(SELECT e.last_error FROM upstream_minute e
WHERE e.upstream_id = m.upstream_id AND e.minute_ts >= ?2 AND e.minute_ts < ?3
AND e.last_failure_ts IS NOT NULL
ORDER BY e.last_failure_ts DESC, e.minute_ts DESC LIMIT 1)
FROM upstream_minute m JOIN upstream_targets t ON t.id = m.upstream_id
WHERE t.url = ?1 AND m.minute_ts >= ?2 AND m.minute_ts < ?3;
```
(Adjust the correlation to the repo's statement idioms; the binding requirements are one atomic statement and the deterministic tiebreak. A bare `SELECT max(last_failure_ts), last_error` without the subquery would pair the max with an arbitrary row's error — SQLite's bare-column-with-aggregate behavior.) An unknown URL or an empty window returns zeros and the empty error. `pruneOlderThan` wraps the minute delete and the unreferenced-target cleanup in one `db.Tx`; the returned count is deleted minute rows only, and a test proves the target delete does not inflate it.
The upsert is exactly:
```sql
INSERT INTO upstream_minute (upstream_id, minute_ts, successes, failures, last_failure_ts, last_error)
VALUES (?1, ?2, ?3, ?4, ?5, ?6)
ON CONFLICT(upstream_id, minute_ts) DO UPDATE SET
successes = successes + excluded.successes,
failures = failures + excluded.failures,
last_failure_ts = coalesce(max(last_failure_ts, excluded.last_failure_ts), last_failure_ts, excluded.last_failure_ts),
last_error = CASE
WHEN excluded.last_failure_ts IS NOT NULL
AND (last_failure_ts IS NULL OR excluded.last_failure_ts >= last_failure_ts)
THEN excluded.last_error
ELSE last_error
END
```
(`max()` over a NULL is NULL in SQLite, hence the coalesce; a test proves a success-only upsert leaves an existing failure column intact.)
The flush loop lives on `Accumulator`:
- `run(self, io, database)` — every 60 s on the `.boot` clock (retention.zig:144-146's reasoning: a box that suspends must still see its interval elapse), `flushOnce` then sleep, `std.Io.Cancelable!void`.
- `flushOnce` (seam signature two bullets below) uses a **swap, never subtraction**: under the mutex, move the dirty cells into a flush-owned `[max_pending]Cell` buffer and clear the accumulator (`count = 0`); release the mutex; run `flush` on the buffer. On success the buffer is done. On failure, set `last_flush_failed`, bump `flush_failures`, log one `warn`, retake the mutex and **merge the buffer back additively** through the same cell-merge rules recording uses — cells recorded during the write keep their outcomes, a merged cell that collides sums, and a merge that overflows `max_pending` follows the normal drop policy (oldest evicted, `rows_dropped` and `last_drop_minute` updated). No subtraction exists anywhere: the earlier copy-and-subtract draft was unsound — while SQLite ran outside the mutex, a full accumulator could evict a copied cell and later recreate the same `(url, minute_ts)`, and the post-flush subtract then destroyed newly recorded outcomes. Tests must include the interleaving the swap makes possible: during a stubbed in-flight flush, recreate a swapped-out cell's `(url, minute_ts)` and fill the accumulator to overflow; fail the flush; assert the drop policy's accounting held — drops counted, `last_drop_minute` advanced via `@max`, and the flush-owned buffer unaffected by the recording that happened beside it. (Overflow loss is the specced policy, not a defect the merge-back must prevent.)
- Counter meanings, pinned by tests: `flushes` counts successful flush transactions only; `flush_failures` counts failed flush attempts; a pass with nothing pending counts neither. A successful flush clears `last_flush_failed`.
- The flush write goes through a seam so tests can fail it deterministically: `flushOnce(self, io, database, write: *const fn (*db.Db, []const upstream_history_repo.FlushRow) db.Error!void)` with the production caller passing `upstream_history_repo.flush`; `run` closes over the real function. Tests pass a failing or recording stub.
- Clean shutdown order, explicitly: drain the query logger, cancel and join the task group, then run one final `flushOnce`, then close the history connection. A crash loses at most the pending cells.
- The task owns a dedicated `querylog.db` connection from `cli.DataDir.reopenQuerylogDb` (retention.zig:143-157 explains why handles are never shared; app.zig:525 is the pattern).
- The disk-monitor gate does not block the flush (prune-sized writes, same category as the query logger's own writes, which are ungated).
### 5. Retention: the existing daily pass, a fixed window, no knob
`Retention.runOnce` gains an upstream-history prune step placed **with the prune and checkpoint steps, before the vacuum-cadence logic** — the vacuum block early-returns (`passes_since_vacuum < vacuum_every_passes`, and the gated path returns too; retention.zig:114-123), so a step after it would be skipped on most passes. The step is `upstream_history_repo.pruneOlderThan(database, now - retention_window_s)` with `pub const retention_window_s: i64 = 31 * 86_400` on the repo. On the arithmetic: `stats.window` gives `since = until - width * count` with `until > now`, so a 30-day window's `since` never precedes `now - 30d`; 31 days is a full day of slack, not a bound the windows require, chosen so a pass that runs late clips nothing (deviation from the Codex text, which suggested computing the cutoff through `stats.window(.@"30d")` — that would import `web/handlers/stats.zig` into `storage/`, and the constant dominates every aligned window anyway). `logging.retention_days` does not apply to upstream history; a test proves a 1-day query-log retention still keeps 30 days of upstream minutes. Pruned minute rows do **not** count into the existing `rows_pruned` counter (`nxdns_retention_rows_pruned_total` stays query-log-only); `Retention.Stats` gains a separate `upstream_rows_pruned` counter with its own metric line, and a test pins that a pass pruning both kinds moves each counter by its own amount.
### 6. API: `GET /api/upstream/health?period=` with a now/period split
`src/web/handlers/upstream_health.zig` is rewritten. Period handling reuses `stats.periodParam` and `stats.window` verbatim — absent defaults to `24h`, anything else unreadable is the same 400 text stats uses. The aggregation runs on the web task's own `state.querylog_db` connection over `[since, until)` by `minute_ts`.
```zig
pub const PeriodStats = struct {
attempts: u64,
successes: u64,
failures: u64,
success_rate: ?f32, // null when attempts == 0 — no observations is not perfect reliability
last_failure_at: ?i64, // newest last_failure_ts inside the window
last_failure_error: ?[]const u8,
};
pub const Upstream = struct {
url: []const u8,
enabled: bool,
available: bool,
period: PeriodStats,
};
pub const Body = struct {
period: []const u8,
since: i64,
until: i64,
available: u32,
total: u32,
complete: bool,
upstreams: []const Upstream,
};
```
- Rows come from the current pool snapshot (`metrics.poolSnapshot`), joined to history by URL through `upstream_history_repo.windowStats` — the handler owns no SQL of its own: current upstreams only, deleted URLs' history is not returned, an upstream with no rows in the window gets zeros and nulls. `WindowStats` returns `last_failure_error_buf` by value; the handler must copy the selected error text into the request arena before it builds the response entry — a slice into the loop-local `WindowStats` dangles into stack storage the next iteration reuses.
- `complete` is per-window and stateless: false iff the accumulator's `last_drop_minute` is non-null and `>= since`. The field's definition is deliberately narrow, and the openapi description carries it verbatim: "No capacity drops known in this process within the selected window; up to about a minute of the newest outcomes may not have flushed yet, and outcomes lost in an unclean shutdown are not detectable." A window that starts after the last drop is complete again, so one historical overflow does not mark every future response. The UI shows the muted incomplete note when false and asserts nothing affirmatively when true — absence of a warning, never a "complete" badge.
- Removed from the response entirely: `consecutive_failures`, `total_successes`, `total_failures`, the last-32 `success_rate`, process-lifetime `last_error`/`last_error_age_s`. They remain in `health.State` for routing and in `/metrics` unchanged. openapi.yaml: the route gains the shared `period` query-parameter reference the stats routes use and a 400 response for a bad period, and the schema is rewritten to this shape (required: all fields; nullables marked); samples regenerated and `admin/src/lib/types.ts` updated in the same session.
### 7. Failure visibility
- `metrics.zig` gains a group fed by `Accumulator.snapshotStats`: `nxdns_upstream_history_flushes_total`, `nxdns_upstream_history_flush_failures_total`, `nxdns_upstream_history_rows_dropped_total`, `nxdns_upstream_history_pending` (gauge).
- The `/api/health` rollup (handlers/health.zig) gains one boolean in its `Input`: `history_flush_failing`, fed from the accumulator's `last_flush_failed` — current state, set by a failed flush and cleared by the next successful one. The rollup degrades to `degraded` while it is true and recovers when it clears (`/api/health` has no `warn` status). Nothing in `collect`/`rollup` compares cumulative counters across samples (they are stateless, so "grows between collections" is unimplementable there), and `rows_dropped` does not feed the rollup at all — a historical overflow must not latch `/api/health` to `degraded` forever; drops surface through the metric and through ruling 6's per-window `complete`. Extend the degraded-matrix table test with the new row in both states.
### 8. UI: one ranged table, live state labeled locally
- `upstreamHealthQuery(period)` in `admin/src/lib/queries.ts` — the query key includes the period, so the picker refetches upstream health with totals and timeseries. `UpstreamHealth` types match ruling 6.
- The upstreams card rejoins the ranged content (the m26 layout deletes the "Right now" section the previous fix added; DiskCard becomes a card titled "Storage now"). Columns: Upstream · Status now · Attempts · Failures · Success rate · Last failure. A grouped header labels the last four "Selected period · {label}".
- "Status now" is one word from live state: `Disabled` when not enabled, else `Available`/`Backing off` from `available`. The yes/no Enabled and Available columns are gone.
- Zero attempts: `0`, `0`, `—`, `—`; when every upstream has zero attempts the card shows "No upstream attempts in this period." A `100.0%` must be unreachable from zero attempts (vitest proves it).
- Last failure renders `ErrorName · age` via the existing `formatAge` against `last_failure_at` and the response's `until`… no — against `Date.now()` at render, same as the m25.5 cell; `—` when null. `complete: false` renders a muted "history incomplete" note on the card.
- Numeric columns use `shared.tabularNums`.
## Sessions
Sequential: S1 → S2 → S3. No parallel sessions; each depends on the previous session's files.
### Session S1: schema, accumulator, flush, retention, metrics, health rollup
Owns: `src/storage/querylog_schema.zig`, `src/upstream/history.zig` (new), `src/storage/repositories/upstream_history_repo.zig` (new), `src/upstream/pool.zig`, `src/storage/retention.zig`, `src/app.zig`, `src/cli.zig` (its connection-count comment on the querylog reopen becomes false — the count is writer + retention + history = three background connections, plus the web connection when the web server is enabled; correct it), `src/web/metrics.zig`, `src/web/handlers/health.zig`, `src/tests.zig`.
Tests (each watched failing): additive upsert including the NULL/max interaction and success-only-preserves-error; restart-shaped double flush into one minute row sums; `windowStats` sums only the window and pairs the newest failure with its own error row, not an arbitrary one; eviction at `max_pending` drops oldest, counts, and moves `last_drop_minute` only forward; the ruling-4 eviction-and-reinsertion-during-flush interleaving (fail the flush; assert the correct drop count, the forward-only watermark, and the unaffected flush-owned buffer); flush failure sets `last_flush_failed` and the next success clears it, with the counter meanings of ruling 4 (a no-op pass counts neither); retention prunes a 32-day-old minute row and its orphaned target while keeping day-29 rows under `retention_days = 1`, inside one transaction, counting minute rows only, into `upstream_rows_pruned` and not `rows_pruned`; the pruning step runs on a pass where the vacuum logic early-returns; the pool-with-accumulator test of ruling 3 (Fake-client success and failure both land in cells); metrics render the four new series; rollup degrades on `history_flush_failing` and recovers, and does not degrade on `rows_dropped`.
Acceptance (S1):
- [ ] `zig build test` and `-Dintegration` green with explicit counts from the test binary.
- [ ] The DDL fingerprint changed and the schema test names both new tables and the index.
- [ ] No SQLite call reachable from `exchangeLoopLen` (grep-level review stated in the report).
### Session S2: the ranged endpoint and the API contract
Owns: `src/web/handlers/upstream_health.zig`, `src/web/openapi.yaml`, `admin/src/lib/types.ts`, `admin/src/lib/contractSamples.gen.ts` (regenerated).
Tests (watched failing): window aggregation sums only `[since, until)`; zero attempts → `success_rate: null` and null failure fields; last failure inside the window beats an older one outside; `complete` flips on a drop stamped inside the window and not on one before it; the 400 and default-period behavior match stats; serialization pins the exact field set (the removed fields must not appear — assert their absence); two upstreams with different last-failure errors serialize both error texts correctly (no dangling or cross-row reuse from the by-value `WindowStats` buffer).
Acceptance (S2):
- [ ] Both Zig suites green; regenerated samples show the new shape and none of the removed fields.
- [ ] `cd admin && npx tsc --noEmit` green against the new types.
### Session S3: dashboard UI
Owns: `admin/src/lib/queries.ts`, `admin/src/lib/api.ts` (`getUpstreamHealth` takes the period), `admin/src/features/dashboard/DashboardPage.tsx`, `UpstreamHealthTable.tsx`, `DiskCard.tsx`, their test files, `admin/src/ui/styles.ts` (only if a shared style is genuinely reused), `CHANGELOG.md` (the Unreleased notes of the milestone acceptance).
Tests (watched failing): period in the query key refetches on picker change; column set and grouped header; `Backing off` and `Disabled` states; zero-attempt row renders em-dashes and never `100.0%`; card-level no-attempts message; incomplete note; the m25.5 "Right now" section is gone and "Storage now" exists.
Acceptance (S3):
- [ ] `cd admin && npx tsc --noEmit && npx vitest run && npx prettier --check . && npm run lint` all green with counts.
## Module layout
New files: `src/upstream/history.zig`, `src/storage/repositories/upstream_history_repo.zig`. Deleted surface: the unranged fields of `/api/upstream/health` (breaking API change, pre-v0.1, changelog notes it), the dashboard's "Right now" section.
## File ownership
| File | Session |
| --- | --- |
| `src/storage/querylog_schema.zig`, `src/upstream/history.zig`, `src/storage/repositories/upstream_history_repo.zig` | S1 |
| `src/upstream/pool.zig`, `src/storage/retention.zig`, `src/app.zig`, `src/cli.zig`, `src/web/metrics.zig`, `src/web/handlers/health.zig`, `src/tests.zig` | S1 |
| `src/web/handlers/upstream_health.zig`, `src/web/openapi.yaml`, `admin/src/lib/types.ts`, `admin/src/lib/contractSamples.gen.ts` | S2 |
| `admin/src/lib/queries.ts`, `admin/src/lib/api.ts`, `admin/src/features/dashboard/*`, `CHANGELOG.md` | S3 |
Sessions are strictly sequential; the table exists so a fix agent knows whose file it is touching.
## Acceptance (milestone complete)
- [ ] All session boxes.
- [ ] Live smoke per the verify-against-the-real-network rule: a scratch server with one dead upstream (TEST-NET address) and one real one; real queries; confirm the 1h window shows the dead upstream's failures with `success_rate` 0% and the real one's successes; restart the server mid-window and confirm the counts continue rather than reset; wait past a flush and confirm rows in `upstream_minute` via sqlite3; switch the picker across all four periods and confirm the numbers change window, not meaning.
- [ ] UI screenshots (dashboard across at least two periods, plus the zero-attempts state) reviewed by the user in Firefox before anything is pushed — the user's standing rule; no push, no release without it.
- [ ] `src/web/openapi.yaml` reviewed by hand against the response struct; reviewer says so in `## Recorded`.
- [ ] Changelog notes: ranged upstream health, the removed API fields, and that the upgrade recreates `querylog.db` — query history restarts empty and the previous file is kept aside as `querylog.db.corrupt-<timestamp>`, per the established schema-change policy.
## Anti-requirements
- No config knobs: cadence, capacity, retention window are `pub const`s.
- No per-exchange rows anywhere; the finest grain is the minute aggregate.
- No third database file and no history in `config.db`.
- No counter-delta sampling and no code path that can produce a negative count; restarts add, never subtract.
- No `100.0%` (or `1.0`) from zero attempts — null, rendered as an em-dash.
- No SQLite work on the query path; the accumulator is memory-only under its own lock.
- No page-wide "the period does not apply" banner; live values are labeled locally (`Status now`, `Storage now`).
- No removal of `health.State` internals — the last-32 window, lifetime totals, consecutive failures and backoff stay for routing and `/metrics`.
## Recorded (implementation)
Built S1 → S2 → S3, reviewed (one Codex round, two findings, both fixed), live-smoked. Deviations and observations, per session:
S1:
- `WebState` gained a `history` field so the handler can reach the accumulator; `web/state.zig` was not in the ownership table.
- The accumulator is heap-allocated, following the `sse.Hub` pattern, so its address is stable across the task group and the web server.
- Shutdown calls `group.cancel` explicitly before the final flush; the spec's order (drain logger → cancel and join → final flush → close connection) holds, the explicitness is the deviation.
- The `/api/health` change touched only the rollup `Input`; `collect` stayed as it was.
- Honest negative from the S1 report: a bare-column mutation of `last_failure_ts` in the upsert survives the tests because SQLite's `max()` NULL rule absorbs it; the deterministic-tiebreak test covers the observable behavior instead.
Review round:
- app.zig error-path teardown now runs the ruling-4 shutdown sequence via a single `defer`; no test — the structural argument (one defer, one sequence, no early-return can skip it) is the recorded justification.
- The u32 saturation in the accumulator's cell merge got a constraint comment.
S2:
- `bad_period_message` is duplicated in `upstream_health.zig` because `stats.badPeriod` is private; follow-up noted to export it and delete the copy.
- The route-level 400 is not testable through `handle` for either period route; the behavior is covered at the `periodParam` level, shared with stats.
- A 500 response was added to the openapi entry beyond the spec, matching the other database-backed routes.
- A float-spelling correction in the contract samples (regenerated, not hand-edited).
S3:
- Changelog entries sit under Unreleased pending the release commit.
- The incomplete note reads "history incomplete for this period" — wording chosen in session, within ruling 8's intent.
Live smoke: all four periods returned window-scoped numbers; an invalid period returned 400; restart continuity held (counts continued, no reset); the schema-fingerprint aside-rename fired and produced `querylog.db.corrupt-1786956534`; `upstream_minute` rows confirmed via sqlite3 after a flush.
+285
View File
@@ -0,0 +1,285 @@
# Milestone 27: diagnostics vertical slice
Step 1 of the UI redesign (`specs/ui-redesign.md`). A curated log of operational failure episodes: one `operational_events` table in `config.db`, a serialized event store, fifteen typed event codes emitted at existing failure sites, `GET /api/diagnostics`, and a Diagnostics page in the admin SPA. The rest of the UI stays intact; the full navigation restructure is later steps.
Design authority: `specs/ui-redesign.md` §Diagnostics, as amended by the Fable review rulings recorded in that file. No `api.storage` code. No periodic probe. No new index beyond the two specced.
## Sessions
S1 (store + API) → S2 (emitters) and S3 (SPA) in parallel → orchestrator integration (contract samples regen, final wiring check).
---
## Session S1: event store, schema, API, health, metrics
### S1.1 Schema — `src/storage/config_schema.zig`
Append to `ddl_v1` (pre-v0.1: edit the baseline, no migration step; update PLAN §11.2 to match):
```sql
CREATE TABLE operational_events (
id INTEGER PRIMARY KEY,
code TEXT NOT NULL,
subject_key TEXT NOT NULL,
subject_label TEXT NOT NULL,
severity TEXT NOT NULL CHECK (severity IN ('warning', 'error')),
first_seen INTEGER NOT NULL,
last_seen INTEGER NOT NULL,
occurrences INTEGER NOT NULL CHECK (occurrences > 0),
resolved_at INTEGER,
detail TEXT NOT NULL DEFAULT '',
CHECK (resolved_at IS NULL OR resolved_at >= first_seen)
);
CREATE UNIQUE INDEX idx_operational_events_active
ON operational_events(code, subject_key) WHERE resolved_at IS NULL;
CREATE INDEX idx_operational_events_last_seen
ON operational_events(last_seen DESC);
```
`operational_events` is runtime state, NOT configuration. It stays out of `table_names` and `delete_order` (`config_schema.zig:112-129`). Tests that must be updated deliberately, not silently: `migrations.zig:365` asserts `count(tables) == delete_order.len + 1` — becomes `+ 2`, and its comment names `operational_events` as the second exclusion alongside `schema_version`. `src/config/reconcile.zig:268` (`comptime assert(delete_order.len == 10)`) is untouched — the list does not change. Add a test asserting export/import round-trips leave `operational_events` rows intact.
Editing `ddl_v1` changes nothing for existing installs pre-0.1 EXCEPT the silent-divergence hazard already on record: the Pi's `config.db` is stamped version 1 and will not get the new table. This milestone accepts that with a bridge placed at the END of `migrations.migrate`, after the version stamp: execute the three `operational_events` statements with `IF NOT EXISTS` added. It must NOT live in `cli.openConfigDb``openConfigDb` runs before migration everywhere (`app.zig:333`, `cli.zig:484`, `cli.zig:516`), and creating the table there would make a fresh database's non-`IF NOT EXISTS` `ddl_v1` `CREATE TABLE` fail. The bridge is explicit, commented as removable when the 0.1 adoption gate lands, and touches no other table. Test: a version-1 database created WITHOUT the table gains exactly it after `migrate`, and a fresh database migrates cleanly (no double-create).
### S1.2 Event codes — `src/storage/events.zig` (new)
```zig
pub const Code = enum {
disk_space, disk_probe,
blocklist_refresh, blocklist_snapshot, blocklist_storage,
certificate_reload,
query_log_write, query_log_maintenance, query_log_recreated,
upstream_history_write, upstream_exchange,
client_names_storage, clients_storage,
listener_start, configuration_load,
};
```
Fifteen enum members — that count is the single cardinality every exhaustive test, route fixture and frontend copy union uses. The enum is the truth; the wire form is the dotted string (`disk_space``"disk.space"`, `blocklist_refresh``"blocklist.refresh"`, etc.) via `pub fn wire(code: Code) []const u8` — an exhaustive switch, tested against every member. `component(code)` returns the prefix before the dot, also exhaustive.
Severity is fixed per emit call, not per code (a disk transition to `warn` is warning, to `critical` is error).
### S1.3 Event store — `src/storage/events.zig`
```zig
pub const Store = struct {
pub const max_detail_len = 512;
pub const max_subject_key_len = 256; // longer keys become sha256: digests — see the identity rule below
pub const max_subject_label_len = 128;
pub const resolved_retention_s: i64 = 90 * 86_400;
pub const max_resolved_rows: i64 = 5_000;
mutex: std.Io.Mutex,
database: *db.Db, // dedicated connection; ALL access goes through the Store's mutex
write_failed: std.atomic.Value(bool),
write_failures: std.atomic.Value(u64), // feeds the metrics counter
active: ActiveSet, // in-memory mirror of active (code, subject_key) rows
untracked_active_count: u32, // active rows NOT in the mirror (overflow); loaded at init
pub fn init(io, database, now_s) db.Error!Store // loads ActiveSet + untracked count, prunes; failure = no store
pub const max_kept_keys = 64; // resolveExcept bound; over it the call is refused, counted and latched — truncating the kept list would close episodes that are still true
pub fn report(self, io, now_s, code, subject_key, subject_label, severity, detail) void
pub fn resolve(self, io, now_s, code, subject_key) void
pub fn reportResolved(self, io, now_s, code, subject_key, subject_label, severity, detail) void
pub fn resolveExcept(self, io, now_s, code, kept_keys: []const []const u8) void
pub fn prune(self, io, now_s) void
pub fn writeFailed(self) bool
pub fn activeCounts(self, io) struct { warnings: u32, errors: u32 }
pub fn selectEvents(self, io, arena, filter) db.Error!EventsPage
pub fn selectOne(self, io, arena, id: i64) db.Error!?Event
};
```
Contract:
- **Time is a parameter, not a stored seam.** Every mutating method takes `now_s: i64` from the caller, matching how `history.zig:132` receives `wall_s` and `logger.zig:72` receives entry timestamps. Production callers compute it from `Clock.real`; tests pass literals. No function pointer, no clock inside the store.
- **`resolve` is hot-path safe.** It checks `active` under the mutex and, when the key is absent AND `untracked_active_count == 0`, returns without any SQLite statement. `pool.recordSuccess` calls it on every successful exchange; steady state must cost a mutex acquire and a lookup. The no-SQL guarantee is tested through a debug-only statement counter on the Store (incremented before every repo call) — NOT via `sqlite3_total_changes`, which a `SELECT` probe would not move.
- **Overflow is exact, not heuristic.** `ActiveSet` capacity is 256. The no-SQL fast path belongs to `resolve` ONLY: mirror miss + `untracked_active_count == 0` → return, no SQL (that is the steady-state success). `report` on a mirror miss always writes — when the count is zero it inserts directly (a new episode in normal state, no probe needed); when the count is nonzero it probes and upserts (touch the untracked active row if one exists — a second insert would collide with the partial unique index — else insert), incrementing `untracked_active_count` only when a new row was inserted AND the mirror is full. A successful slow-path `resolve` decrements the count. `init` loads the count as `active rows - mirrored rows`. `events_repo.resolveExcept` performs the bulk resolve and a count of TOTAL remaining active rows in ONE transaction, returning that total only after commit — the repo knows nothing of the mirror. The Store then removes resolved keys from its mirror and sets `untracked_active_count = total_active - active.len`; on any failure it changes neither.
- **Key canonicalization happens at every entry point.** `report`, `resolve`, `reportResolved` and each `kept_keys` element of `resolveExcept` all pass the caller's key through the same digest-if-over-length rule before any lookup or SQL. Tests cover a long-key report resolved with the same long key, and `resolveExcept` keeping a long kept key.
- **All reads go through the Store.** The handler calls `store.selectEvents` / `store.selectOne`, which lock the same mutex around `events_repo` — nothing touches `store.database` from outside. One connection, one owner.
- `report` upserts on the active row: present → `last_seen = now_s`, `occurrences += 1`, `detail` replaced, severity raised to the worse of the two, never lowered. Absent → insert new active row and add to `active`.
- `resolve` on an active row sets `resolved_at = now_s` and removes it from the mirror. A later failure inserts a NEW row (new episode) — the partial unique index enforces one active row per key.
- `reportResolved` inserts a row with `resolved_at = first_seen = last_seen = now_s`, `occurrences = 1`, and never touches `active` (one-shot events: `query_log_recreated`).
- `resolveExcept(code, kept_keys)`: resolves every active row of `code` whose `subject_key` is not in `kept_keys`, in one serialized operation. Exists for the boot-finalized codes (S2); nothing else may use it.
- Write failures: on `db.Error`, set `write_failed = true`, increment `write_failures`, drop the event. **Log only on the `false → true` transition** — a broken diagnostics database plus a busy pool must not produce warnings at query rate; the counter and the health surface carry the ongoing state. The next successful write clears the latch (and that recovery may log once).
- **No error propagates to a producer** — mutating methods return `void` by design; a diagnostics failure must never break the subsystem reporting it. `init` is the exception: it returns `db.Error`, and `app.zig` responds by running with no store (`null` everywhere) and logging once — a store built on an unverified mirror would produce false no-op resolves, which is worse than no store.
- `subject_key` identity is exact at any length: a key at or under `max_subject_key_len` is stored verbatim; a longer one (operator URLs are unbounded — `safe_url.zig:12` imposes no input limit, and `manager.zig:179`'s 255 cap bounds only a display copy) is replaced by `"sha256:" ++ hex(SHA-256(key))` — 71 bytes, deterministic, collision-free in practice, so distinct long URLs never merge and the same URL always maps to the same episode. Never truncate and never reject a key. `subject_label` truncates to `max_subject_label_len`, `detail` to `max_detail_len` — those are display fields. `subject_key` never leaves the process; `subject_label` is the redacted display identity (`safe_url.redactQuoted` where the subject is a URL).
- `prune`: delete resolved rows older than `resolved_retention_s`, then oldest resolved rows beyond `max_resolved_rows`. Active rows are never pruned. Called at store init and once per retention pass (S2).
### S1.4 Repository — `src/storage/repositories/events_repo.zig` (new)
Free functions on `*db.Db`, matching `upstream_history_repo.zig` conventions (file-scope SQL constants, by-value row structs with fixed buffers, prose test names, `:memory:` fixtures). Functions: `insertActive`, `touchActive`, `resolveActive`, `resolveActiveByKey`, `selectActiveId`, `countActive`, `resolveExcept`, `insertResolved`, `loadActive`, `selectEvents(filter)`, `selectOne(id)`, `pruneResolved`, `activeCounts` (the three beyond the original list serve the exact-overflow contract). Only the `Store` calls these in production. `selectEvents` filter: `state` (active/resolved/all), `severity`, `component` (matched on `code` prefix), `since`/`until` with the repo's `[since, until)` convention: an empty range (`since >= until`) returns nothing, and overlap is `first_seen < until AND (resolved_at IS NULL OR resolved_at > since)` — strict `>`, an episode resolved exactly at `since` does not overlap (matches `queries_repo.zig:211`). `limit`, `before` (keyset on id descending).
### S1.5 API — `src/web/handlers/diagnostics.zig` (new)
`GET /api/diagnostics` — params `state` (`active`|`resolved`|`all`, default `all`), `severity` (`warning`|`error`), `component`, `since`, `until`, `limit` (1..1000, default 100), `before` (positive id). 400 with a naming message on any bad param (copy `queries.zig` conventions: `parseFilter` + `message(err)`). 503 when the store is absent. Response:
```json
{ "events": [ { "id": 42, "code": "blocklist.refresh", "component": "blocklist",
"subject": "StevenBlack", "severity": "warning",
"first_seen": 1787118000, "last_seen": 1787118300, "occurrences": 3,
"resolved_at": null, "detail": "download failed: ConnectionTimedOut" } ],
"next_before": null,
"active": { "warnings": 1, "errors": 0 } }
```
`subject` serializes `subject_label`. `subject_key` has no wire form — assert that in a test. Pagination contract identical to `/api/queries` (full page carries cursor, short page nulls it).
`GET /api/diagnostics/{id}` — the same event object, 404 after retention or for an unknown id.
Routes: two entries in `routes.zig`, `.auth = .session`, `.policy = .read`. Update the pinned route-count test (`routes.zig:145`, 56 → 58) and every routing invariant test that enumerates. OpenAPI: paths + `DiagnosticsPage` / `DiagnosticEvent` schemas, following the `/api/queries` exemplar; the openapi drift tests must stay green.
### S1.6 Health — `src/web/handlers/health.zig`
`Input` gains `diagnostics_present: bool = true` (benign default, matching the all-defaulted convention), `diagnostics_write_failed: bool = false`, `diagnostics_active_warnings: u32 = 0`, `diagnostics_active_errors: u32 = 0`. Body gains `"diagnostics": { "state": "recording"|"unavailable", "active_warnings": N, "active_errors": N }``unavailable` when `!present` or `write_failed`. `degraded` adds `diagnostics_write_failed or !diagnostics_present`: in a serving process the store is absent only when `Store.init` failed, which is a real degradation, and the health endpoint never runs in subcommands. `collect` must assign `diagnostics_present = state.events != null` explicitly — the natural `if (state.events) |store|` shape would leave an absent store reported as recording under the benign default. **This milestone does NOT yet remove `history_flush_failing` or restructure health to the full redesign shape** — that lands with step 4 (Overview replacement); here health only gains the diagnostics block. Update the degraded-matrix test to cover both new inputs.
### S1.7 Metrics — `src/web/metrics.zig`
Gauges `nxdns_diagnostics_active_warnings`, `nxdns_diagnostics_active_errors`; counter `nxdns_diagnostics_write_failures_total` (increment in the store on each failed write). Follow the upstream-history block pattern.
### S1.8 WebState — `src/web/server.zig`
`events: ?*events.Store = null`.
### S1.9 Acceptance criteria (S1)
- [ ] `zig build test` green; new store tests cover: episode open/dedupe/severity-raise, resolve-then-new-episode, one-shot insert, no-op resolve executes no SQL, write-failure latch set and cleared (log only on transition), label/detail truncation, over-length subject_key digested to the same identity twice, prune retention + cap, overflow slow paths for report (upsert, no unique-index collision) and resolve (count decrements), resolveExcept keeping mirror and count exact.
- [ ] `events_repo` tests cover every function against `:memory:` including overlap-window selection and keyset pagination.
- [ ] Handler tests: each param rejected with 400 + naming message, 503 without store, pagination cursor behavior, active counts, 404 on `{id}`.
- [ ] Route-pin, openapi-drift, health-matrix, schema-count tests all updated intentionally and green.
- [ ] Export/import round-trip leaves `operational_events` intact (test).
---
## Session S2: emitters (after S1)
Thread the store as `?*events.Store` using the established `gate: ?*disk_monitor.Monitor` idiom (`app.zig:776`): per-call parameter for run-loops, optional post-init field for `Manager`/`Pool`/`CertStore`. Every subsystem must build and test with `null` (no store). app.zig opens the dedicated connection right after migration (`app.zig:335`), unconditionally — not gated on `cfg.web.enabled`; diagnostics record whether or not the UI is on:
```zig
var events_db = try data.openConfigDb(io);
defer events_db.close();
const boot_now = std.Io.Clock.real.now(io).toSeconds();
var event_store_storage: ?events.Store = events.Store.init(io, &events_db, boot_now) catch |err| blk: {
log.warn("diagnostics store unavailable: {s}", .{@errorName(err)});
break :blk null;
};
const event_store: ?*events.Store = if (event_store_storage) |*s| s else null;
```
`event_store` is what gets threaded; a failed init leaves it null and health reports `unavailable` + degraded.
Emit sites, from the verified survey. Each row: failure emit, recovery resolve, subject_key / subject_label, severity.
| Code | Failure (file:line today) | Recovery | subject_key → label | Severity |
| --- | --- | --- | --- | --- |
| `disk.space` | `disk_monitor.zig:137-147` transition to warn/critical | same `publish`, transition to ok | `"data"` singleton | warn→warning, critical→error |
| `disk.probe` | `disk_monitor.zig:94-97` (change `catch {}``catch \|err\|`), `:101-105`, `:109-113` | next successful branch of the same probe | operation name (`statvfs`/`data_dir`/`log_dir`) | warning |
| `blocklist.refresh` | `manager.zig:1102-1138` (fetch/compile/empty), `:1520-1522` (load) | `SourceStatus.succeed` paths `:840`, `:870`, `:1552` | source URL → source name | warning |
| `blocklist.snapshot` | `app.zig:572-577`, `manager.zig:1164-1167`, `:1184-1187` | `manager.zig:539-544` (post-swap) | singleton | error |
| `blocklist.storage` | `manager.zig:1200-1203`, `:1336`, `:1451`, `:1458`, `:1470` | **the success branch of each matching operation, NOT pass-end**`deleteQuietly` absorbs failures at `manager.zig:1466` and `pruneOrphans` still returns success at `:1350`, so a pass-end resolve would close the very event its own pass emitted | operation name | warning |
| `certificate.reload` | `cert_store.zig:256-262` (capture the discarded err), `:273-279` | `:271-273` success reload | endpoint kind — **`CertStore` gains a `kind: enum { doh, dot }` field set at `openCertStore` (`app.zig:611/:617`)** | warning — a stat failure can be a transient rename window and a failed reload keeps the loaded certificate serving (`cert_store.zig:250`); nothing is down |
| `query_log.write` | `logger.zig:252-260` (init, from the failing `BatchWriter.init` and its catch), `:400-404` (batch) | `:405` successful batch. Init failure has no recovery (writer exits) — the event stays active, which is the truth | `writer`/`batch` | error |
| `query_log.maintenance` | `retention.zig:113`, `:127`, `:133`, `:143`, `:151` | matching success branch same pass | operation name | warning |
| `query_log.recreated` | `querylog_schema.open` result. **Two changes: `OpenResult` gains the aside name (fixed `[path_buf_len]u8` + len — today it exists only in a stack buffer inside `open`, `querylog_schema.zig:137`), and `cli.openQuerylogDb` (`cli.zig:335`) stops discarding `recreated` and returns it to `app.zig`**, which calls `reportResolved` once the store exists. **Never emitted for `.missing`** — first creation has no aside file and is logged as "created" (`querylog_schema.zig:131`); a fresh install must not record a warning with an impossible aside path | one-shot | reason tag; detail carries the aside filename | warning |
| `upstream_history.write` | `history.zig:262-266` | `:257-261` | singleton | warning |
| `upstream.exchange` | `pool.zig:332-346` `recordFailure` — emit OUTSIDE the pool mutex, same placement discipline as `recordHistory` | `pool.zig:316-330` `recordSuccess``resolve` (no-op hot path) | upstream URL → `safe_url.redactQuoted` | warning |
| `client_names.storage` | `client_names.zig:125-131`, `:141-147` | clean-pass determination `client_names.zig:161-164` | `read`/`write` | warning |
| `clients.storage` | `clients.zig:196-202`, `:218-224` | success branches `:195-197`, `:214-217` | `materialise`/`prune` | warning |
| `listener.start` | `app.zig:896/:900` (DoH), `:915/:919` (DoT) | boot-finalized: after the bind phase, one `resolveExcept(.listener_start, failed_keys)` closes prior episodes for endpoints that started clean this boot — including endpoints now disabled | `doh`/`dot` | error |
| `configuration.load` | `app.zig:686`, `:1039-1045`, `:1056-1061` and managed-file `validate.Diagnostics` warnings rendered at `app.zig:204` | boot-finalized: after all boot findings are emitted, one `resolveExcept(.configuration_load, emitted_keys)` | setting/upstream identity (redacted) | warning |
Rules that bind every site:
- The existing `log.warn`/`log.err` lines STAY. Events are additive; journald keeps the raw stream.
- Failure text passed as `detail` is `@errorName(err)` plus the site's existing message fragment — no new prose invented, no allocation: format into a stack buffer of `Store.max_detail_len`.
- **Lock discipline is collect-then-flush, not "emit after the mutex closes".** Several manager sites cannot simply move: `publishRefresh` runs under `writer_lock` by contract (`manager.zig:792`) and `pruneOrphans` holds both `refresh_lock` and `writer_lock` through its filesystem work (`manager.zig:1299`). At such sites, record outcomes while the locks are held and flush to the store after the outer locked operation returns. The collection must not lose events: per-source outcomes ride the manager's existing per-source status allocations (one outcome per source, bounded by the source count), and same-operation failures within one pass aggregate into a single `report` per pass (the row's `occurrences` then counts failing passes, and `detail` carries the last error plus how many failures that pass held) — never an unbounded list, never a silent drop. Pool follows the existing `recordHistory` placement.
- Boot-finalized codes (`listener.start`, `configuration.load`) use `resolveExcept` exactly once each, after their boot phase completes; restart is the recovery, matching the design table.
- `retention.zig` `runOnce` gains `events: ?*events.Store` and calls `store.prune` once per pass.
### S2 acceptance criteria
Emitters fall into three classes, each with its own test obligation (the author watches each fail first, repo ruling F-f):
- **Episodic** (disk, blocklist ×3, certificate, query_log.write batch, query_log.maintenance, upstream_history.write, upstream.exchange, client_names, clients): force the failure (existing seams: injected db errors, missing files, dead fetch server), assert the `(code, subject_key, severity)` row; then force recovery and assert resolution.
- **Boot-finalized** (`listener.start`, `configuration.load`): assert the emit on a failing boot, and assert `resolveExcept` closes a pre-seeded stale episode on a clean boot.
- **One-shot / permanent** (`query_log.recreated`; `query_log.write` init): assert the one-shot row inserts already resolved; assert the init-failure episode exists and that no recovery path claims it.
- [ ] `pool` exchange tests: success with no active episode performs no store SQL; failure→success round-trip produces exactly one resolved episode with correct occurrences.
- [ ] All subsystems still pass with `events = null` (the existing suites running unchanged).
---
## Session S3: admin SPA (after S1, parallel with S2)
- `lib/types.ts`: `DiagnosticEvent`, `DiagnosticsPage`, plus the health body's new `diagnostics` block.
- `lib/api.ts`: `getDiagnostics(filter)`, `getDiagnostic(id)`.
- `lib/queries.ts`: `diagnosticsQuery` (infinite, keyset via `next_before`, `keepPreviousData`), `diagnosticQuery(id)`; refetch interval matching healthQuery's cadence for the active view.
- `routes.tsx`: `/diagnostics` route (loader + lazy component) and `/diagnostics/$id`. Nav: add `Diagnostics` to `NAV_ITEMS` between Lookup and Settings (full nav restructure is later milestones); update `AppShell.test.tsx`.
- `features/diagnostics/DiagnosticsPage.tsx`: active episodes first (severity, title from an exhaustive code→copy map, subject, "active for …" age, occurrences), then resolved history with the standard filters (state, severity, component) reflected in the URL search params. **Empty state is the healthy state**: one line, "No active operational issues.", quiet styling — it must read as good news, not as a broken page (Fable's caution).
- `features/diagnostics/DiagnosticDetailPage.tsx`: the ordered detail from the design — state, first/last seen, occurrences, resolution, impact and remediation text from the same exhaustive code map, last error detail, and links (the config surfaces linked are today's routes; they get re-pointed when later milestones move them).
- The code→copy map lives in `features/diagnostics/eventCopy.ts`: an exhaustive `Record<Code, {title, impact, remediation, link?}>` over a string-literal union of the fifteen wire codes; a test iterates the union and asserts every member has copy.
- Tests: page renders fixtures (active + resolved + empty), filters drive the URL, detail renders every code's copy, pagination fetches more.
### S3 acceptance criteria
- [ ] `npm run typecheck`, `npx vitest run`, `npx prettier --check`, lint, build, assert-bundled all green; byte budget respected.
---
## Orchestrator integration (after S2+S3)
- Regenerate `contractSamples.gen.ts` (`zig build test -Dintegration -Dcontract-samples-out=…`) — requires the integration capture in `web_integration_test.zig` to seed at least one active and one resolved event.
- Update `docs/reference/api.md` and any drift-guarded reference pages; changelog entry under Unreleased.
- Live smoke on the real binary before anything ships: force a blocklist failure (dead URL), watch the episode appear, fix the URL, watch it resolve; screenshot the page per standing rule.
## Module layout (new files)
| File | Purpose |
| --- | --- |
| `src/storage/events.zig` | `Code`, wire mapping, `Store` |
| `src/storage/repositories/events_repo.zig` | SQL |
| `src/web/handlers/diagnostics.zig` | list + detail handlers |
| `admin/src/features/diagnostics/…` | page, detail, copy map, tests |
## File ownership
S1: `config_schema.zig`, `events.zig`, `events_repo.zig`, `diagnostics.zig` (handler), `routes.zig`, `openapi.yaml`, `health.zig`, `metrics.zig`, `server.zig`, `migrations.zig` (bridge + test), PLAN §11.2. S2: every emitter file + `app.zig` + `retention.zig` + `cert_store.zig` + `cli.zig` (openQuerylogDb) + `querylog_schema.zig` (OpenResult aside name) and its tests — S2 starts after S1 lands, so nothing is shared in parallel. S3: `admin/` only. No parallel writers on any file (S2 and S3 run concurrently and are disjoint).
## Anti-requirements
- No `info` severity, no acknowledgement/dismissal state, no manual-resolve endpoint, no raw-log endpoint, no generic remediation action schema.
- No `api.storage` code. No periodic probe for `write_failed`. No configurable retention.
- No changes to the existing dashboard, query log, live, or lookup pages.
- No new dependency, front or back.
## Acceptance criteria (milestone complete)
- [ ] All session criteria; full `zig build test` + `-Dintegration`, admin suite, byte budgets.
- [ ] Live smoke: forced failure → visible episode → recovery → resolved row; `/api/health` shows the diagnostics block; `/metrics` shows the three series.
- [ ] Screenshots of the page (active, resolved, empty states) shown before push.
## Recorded (as built)
Deviations from the sections above, found during build, review and live smoke. The code is the authority; this section says where it moved.
- `src/storage/events_fixture.zig` exists (test-only): a shared migrated-config.db + Store fixture the storage and filter tests use.
- The threading fields through `app.zig`/`server.zig` are named `.diagnostics`; `WebState` carries `.events`. The live smoke caught that `.events` was never assigned in `app.zig` while every suite stayed green — the integration tests build their own `WebState`. One line fixed it; the lesson is the standing "verify against the real network" rule.
- `runScheduler`'s loop body is the pub function `scheduledPass`, so tests drive one pass deterministically. `runScheduler` flushes only in its catch branch (for the snapshot note its failure path adds).
- Occurrences count failing passes, not flushes: `SourceStatus` carries `pass_outcome`/`pass_failures`, set in `fail()`/`succeed()`, drained by `flushSourceDiagnostics`. Detail format: `"{state}: {error} ({N} this pass)"`.
- Pass accounting lives in exactly one table copy at a time: `mergeStatuses` zeroes the pass fields on carried entries, `installStatuses` folds the live table's unflushed accounting in by id under the exclusive lock, and the flush drains by claiming one outcome-bearing entry per lock round (immune to concurrent table replacement).
- A pass flushes before it releases the lock that serializes it: the flush defer registers after the unlock defer in `refreshSource`, `refreshAll`, `startupPass`; standalone `reload` takes `writer_lock` itself and flushes inside it. This amends the collect-then-flush rule — the store runs on its own connection and mutex, so no shared resource is held across the store call.
- Deleted sources: `flushSourceDiagnostics` ends in `resolveDeletedSources``resolveExcept(.blocklist_refresh, all current keys)`, gated on `generation != 0` (an empty pre-reload table means "not read yet", not "all deleted") and skipped above 64 keyed entries (`max_kept_keys`; the episode then lingers until the count drops). `Store.resolveExcept` is no longer boot-finalized-only; the rule is the kept list must be the whole current subject set.
- `events_repo.resolveExcept` is mark-then-unmark (sentinel `maxInt(i64)`, one transaction): the prior resolve-all-then-revive matched the revive on `resolved_at = now_s` and revived an episode resolved by the drain in the same second.
### Addendum: manual purge of resolved events
Resolution stays automatic; the operator decides when resolved history disappears. Two endpoints, both mutations under the usual auth, both allowed in file mode (diagnostics are runtime state, not configuration):
- `DELETE /api/diagnostics/{id}` — purges one resolved event. 409 `{ "error": ... }` when the event is active; 404 when no row has that id.
- `DELETE /api/diagnostics` — purges every resolved event, returns `{ "purged": N }`.
Store: purge functions under the store mutex; they touch only rows with `resolved_at NOT NULL`, so the ActiveSet mirror and untracked count never change. UI: a purge action on each resolved row and on the resolved detail page, plus a "Purge all resolved" control on the list when at least one resolved event shows; active events show no purge affordance. OpenAPI, routes count, contract goldens, api.md updated.
Accepted limitations (reviewed with Codex, ruled in proportion to household scale; each is bounded and self-correcting, counts stay visible in the detail text):
- A standalone web reload interleaving with a refresh pass can merge two passes' accounting into one occurrence.
- `pruneOrphans` flushes its storage aggregates with the writer locks released; two concurrent prunes can merge into one report.
- The deletion sweep's kept-key snapshot can go stale against a concurrently added source: its fresh episode can be resolved once and reopens on the next failing pass with `first_seen`/`occurrences` reset.
+210
View File
@@ -0,0 +1,210 @@
# Milestone 28: query provenance
Redesign step 2 of specs/ui-redesign.md ("Query provenance", "Schema changes", "Entry buffer widths", "API changes", rulings). Every logged query becomes exactly explainable: what policy decided, what matched, where the answer came from, and what the client saw. The existing Query Log/Live/Lookup pages keep working; their replacement is step 3 (milestone 29). Codex spec review folded in (thread 01a02643); its corrections are marked where they changed a ruling.
**This milestone destroys existing query history.** The DDL edit changes the CRC fingerprint (querylog_schema.zig:71-78), so `open` recreates the file and sets the old one aside as `querylog.db.schema-changed-<unix seconds>`. Acceptable pre-v0.1. The changelog entry must say so, and the recreate is the natural first `query_log.recreated` diagnostics emission.
## Sessions
S1 (storage) and S2 (upstream identity) run in parallel — disjoint files. S3 (handler capture) needs both. S4 (web API + contracts) needs S3. S5 (admin) needs S4.
---
## Session S1: querylog schema, repo, logger, coverage watermark
### S1.1 DDL (src/storage/querylog_schema.zig)
`query_log` drops `block_reason` and gains, after the existing columns:
```sql
qclass INTEGER NOT NULL,
rcode INTEGER NOT NULL,
group_id INTEGER,
group_name TEXT,
policy_action TEXT NOT NULL,
policy_reason TEXT NOT NULL,
matched TEXT,
source_id INTEGER,
source_name TEXT,
cname_target TEXT,
safe_search_target TEXT,
route_kind TEXT NOT NULL,
forward_zone TEXT
```
Existing columns (`blocked`, `cache_hit`, `upstream`, …) stay — stats and the step-3 filters still use them. No new index (ruling: `idx_query_log_ts` bounds every time-scoped question; the insert path pays for indexes).
New singleton table, in the same DDL string (Codex: enforce one row):
```sql
CREATE TABLE querylog_meta (
id INTEGER PRIMARY KEY CHECK (id = 1),
created_at INTEGER NOT NULL,
available_since INTEGER NOT NULL
);
```
`createFresh` inserts the row with `created_at = now` and `available_since = now + 1` (conservative: an old row logged in the same second as recreation must not let `since = now` claim completeness). **`available_since` is a monotonic coverage watermark, not a constant** (Codex): pruning advances it to the retention cutoff **in the same transaction as the delete** — the repo exposes one transactional `pruneOlderThan(cutoff)` that deletes and advances together; a failure of either rolls back both. It never moves backward. `queries_repo` gains `availableSince() i64`.
The object-count test (querylog_schema.zig:295-318) updates to 5 tables. TEXT id/name pairs are deliberate (ruling: `client_ip` precedent, "log rows are immutable facts"); ids are NOT foreign keys.
### S1.2 Closed enums — neutral module `src/storage/provenance.zig` (Codex: logger already imports queries_repo, so enums owned by logger would cycle)
Imported by logger, queries_repo, handler and web code. Stored as `@tagName` TEXT; the read path parses text back to the enum and treats an unknown value as a data error, not a passthrough string.
- `PolicyAction`: `not_evaluated`, `allow`, `block`.
- `PolicyReason`: the nine serializable `matcher.Reason` tags (matcher.zig:24-37 minus `none`) plus `local_record`, `forward_zone`, `non_in_class`, `paused`, `snapshot_unavailable`, `no_match`, `protocol_error`. Matcher→PolicyReason conversion is an exhaustive switch. The CNAME case is NOT a `cname:` string prefix any more: `cname_target` non-NULL carries that fact, and `policy_reason` holds the target's own reason.
- `RouteKind`: `blocked`, `local`, `forward_zone`, `upstream`, `cache`, `rejected`.
`protocol_error` + `rejected` cover post-parse protocol refusals (BADVERS, NOTIMP, EDNS FORMERR) — see S3.4; this extends the redesign's enum list, recorded here as an amendment required by its own "every syntactically parsed request that receives a response is logged" rule.
`blockReason()`, `cname_reason_prefix` and the comptime width proof (handler.zig:66-74, 816-823) die in S3.
### S1.3 Entry widening (src/storage/logger.zig)
`Entry` stays a by-value fixed-buffer struct through `Io.Queue`. New fields with explicit widths:
| field | width / type |
| --- | --- |
| `qclass` | `u16` |
| `rcode` | `u16`, validated ≤ 0xFFF (12-bit EDNS extended RCODE, edns.zig:219; Codex: u8 cannot hold it) |
| `group_id` | `?i64` |
| `group_name` | buffer sized by the new `max_group_name_len` (S1.5) |
| `policy_action` | `provenance.PolicyAction` |
| `policy_reason` | `provenance.PolicyReason` |
| `matched` | buffer sized by the rule maximum — the regex engine accepts 256-byte patterns (regex.zig:48), so 256 bytes with a **`u16` length** (the generic `copyInto` returns `u8`; widen it or add a u16 variant — 256 does not fit u8) |
| `source_id` | `?i64` |
| `source_name` | buffer sized by the new `max_source_name_len` (S1.5) |
| `cname_target` | 253-byte buffer |
| `safe_search_target` | 253-byte buffer |
| `forward_zone` | 253-byte buffer |
| `upstream` | widened: sized for the maximum redacted `scheme://host:port` form (host up to 253 bytes), `u16` length — the current 64-byte buffer silently truncates a long valid DoH hostname (Codex); the endpoint host bound gets validated where endpoints are parsed |
`Entry.Fields` gains the borrowed equivalents with `""`/null defaults. `reason_buf`/`max_reason_len` are removed with `block_reason`. The truncation test (logger.zig:610-625) updates.
`transformed()` (logger.zig:234-239) additionally rewrites `matched`, `cname_target`, `safe_search_target` to `hidden_marker` under `hide_domains`. `forward_zone`, `group_name`, `source_name` are configuration labels, not query-derived, and stay visible.
### S1.4 Entry memory budget (Codex: validation permits 1,000,000 queued entries, validate.zig:464, and sse.zig:54 embeds 2,048 entries)
The widened `@sizeOf(Entry)` gets a documented byte budget: `query_log_buffer_max`'s validation upper bound is recomputed so the queue's worst case stays ≤ 64 MiB (`max = 64 MiB / @sizeOf(Entry)`, computed at comptime, stated in the validation reference and CHANGELOG since the accepted range shrinks). Tests assert the default and the new maximum fit the budget, and the SSE hub comment states its embedded-entry cost.
### S1.5 Name caps — neutral module `src/config/limits.zig` (Codex: logger→validate for caps plus validate→logger for `@sizeOf(Entry)` is a cycle)
No length cap exists today for group or source names (validate.zig:721, :853 reject only empty). New `config/limits.zig` owns `max_group_name_len = 64` and `max_source_name_len = 64`; validate.zig enforces them (boundary tests at 64 and 65 bytes, new classification entries per the existing pattern); logger.zig sizes its buffers from them and **exports the computed `query_log_buffer_max` ceiling** (S1.4), which validate.zig imports.
### S1.6 Repo (src/storage/repositories/queries_repo.zig)
- `Row`/`insert_row_sql`/`BatchWriter` bind the new columns; `Sql.capacity` (:262-265) updated.
- `QueryRow` (read) gains `qclass: u16`, `rcode: u16`, `route_kind`, `policy_action`, `policy_reason` (parsed enums, serialized as strings); loses `block_reason`. NULL→`""` convention unchanged for text.
- New `detailById(id) ?QueryDetail`: full provenance row joined with domains, for `GET /api/queries/{id}`.
- `availableSince()` and the transactional prune-plus-advance per S1.1; retention.zig calls the combined operation.
- `stats_totals_sql`/`timeseries_sql` unchanged.
### S1.7 `query_log.recreated` emission
The existing emission site (app.zig:638) already fires on recreate; it gains the **initial coverage start** (the fresh `available_since`) in its detail alongside reason and aside filename (ui-redesign.md:252 requires the new coverage start). Verify open order (events store vs querylog open) and carry the `OpenResult` rather than reordering database opens if needed.
### S1.8 Acceptance (S1)
- [ ] Fingerprint tests updated; recreate test proves aside name, `querylog_meta` singleton row, and the recreated-event detail carrying the coverage start (end-to-end recreate→coverage test).
- [ ] Watermark tests: prune advances `available_since` to the cutoff atomically; a failed delete, a failed watermark update, and a failed commit each leave both untouched (rollback proven); it never regresses.
- [ ] Round-trip test: an `Entry` with every provenance field set survives queue → `toRow` → insert → `detailById` intact (modulo NULL mapping); includes a 256-byte `matched` boundary case.
- [ ] `transformed()` tests split by flag (Codex: the client is governed by `hide_client_ips`, not `hide_domains`): `hide_domains` hides domain, matched, cname_target, safe_search_target and preserves client_ip; `hide_client_ips` hides client_ip and preserves the rest; both flags leave group/source/zone names.
- [ ] Buffer-budget tests per S1.4; name-cap boundary tests per S1.5.
- [ ] `zig build test` 0 failed.
---
## Session S2: selected-resolver identity (every `transport.Client` implementor)
Reverses ruling 20 for the exchange that actually happened. `transport.Client` (transport.zig:325-343) gains a per-call out-parameter: `exchangeFn(ptr, io, query, response_buf, selected: *?[]const u8)`.
**Contract (Codex critical): the identity survives failure.** Each implementation sets `selected.*` to the resolver it is about to attempt, before the attempt; after an all-failed exchange it names the last attempted resolver. The handler consumes it on success AND on the error path — a SERVFAIL row carrying its resolver is the single most useful correlation this redesign adds (ui-redesign.md:157). The slice must stay valid for the query's duration: `Pool` uses `entry.endpoint.url` (Endpoint-owned, stable); the pointer is per-call, threaded into `exchangeLoopLen` (pool.zig:203-266) — never a `*Pool` field (concurrent queries race).
`Pool` reports the **raw** URL; redaction happens in the handler when formatting into the Entry (S3) — `safe_url.redact` is a formatter, so the pool test asserts the raw selected identity and the credential end-to-end test lives in S3/S4 (Codex).
Callers initialize the output to null before the call. `ForwardClient` (forward_client.zig) holds only a parsed `Resolver`, so it gains **owned identity storage** — the formatted resolver text lives in the client and the out-parameter borrows it (Codex: this is real behavior, not a mechanical discard; the "discards only" framing was wrong). Fakes set explicit stable identities.
S2 owns every implementor, fake and call site: pool.zig, forward_client.zig, transport.zig fakes (:635-660), doh_client_live_test.zig:47 and dot_client_live_test.zig:60 (imported by tests.zig — they break compilation if missed, Codex), handler.zig's inline fake (:1074) and its call site (:457, pass-and-ignore — S3 consumes it), doh_server.zig, dot_server.zig, udp/tcp/resolver/phase7 integration tests, web_integration_test.zig. This overlaps S1 on zero files; the handler/web edits are signature-level and complete before S3/S4 start (sequential).
- [ ] Pool tests: winning endpoint reported; failover reports the answerer, not the first attempt; all-failed reports the last attempted; a timeout mid-flight reports the in-flight resolver.
- [ ] `zig build test` 0 failed.
## Session S3: handler capture (src/server/handler.zig + server tests)
After S1+S2. All provenance assembled in `Context` and passed through `LogFields``Entry.Fields`:
- `qclass` from `ctx.q.qclass`; `group_id`/`group_name` from `snapshot.groups[ctx.group]` when snapshot non-null, else null/"" with `policy_reason = snapshot_unavailable` on the unfiltered path.
- Path mapping: qclass≠IN → `not_evaluated`/`non_in_class`, route `upstream`; pause → `not_evaluated`/`paused`; local → `allow`/`local_record`, route `local`; forward zone → `allow`/`forward_zone`, route `forward_zone` + zone name; cache hits → route `cache`, upstream NULL; upstream answers → route `upstream`, upstream = S2's selected identity redacted via `safe_url` into a Context-local buffer before `Entry.init``"pool"`/`pool_upstream` die; allowed matcher decisions (`rule_allow_*`, `blocklist_exception`) → `allow` with exact reason, matched and source captured (today discarded at :442, Codex); no match → `allow`/`no_match`; blocked → `block`/matcher reason, route `blocked`.
- **`upstream` is non-null only for attempted upstream or forward-zone exchanges, including their failures** (Codex). It is NULL for local, blocked, cache and rejected routes — the `"local"` marker (handler.zig:392) dies with `"pool"`. Asserted per route in the table tests.
- `matched` + `source_id`/`source_name` from `matcher.Decision` and `snapshot.sources[decision.source.?]`. **Copy `matched` and the uncloak target into Context-local buffers before the uncloak loop continues** — the scratch buffers are reused per chain step (matcher.zig:733-736).
- CNAME-uncloaked block: policy fields describe the target's decision; `cname_target` holds the target name. `Context.uncloak` (:727-742) widens its return to the target's full decision + name.
- Safe search: `safe_search_target` = the rewrite target; policy stays `allow`.
### S3.1 rcode capture and the truncation bug (Codex)
`reply`'s UDP truncation rebuild (:522-529) currently rewrites every oversized response to NOERROR — an existing defect: an oversized NXDOMAIN reaches the client as success. Fix here: parse the source rcode before rebuilding and preserve it **via `splitRcode` into both the header and the response OPT** (edns.zig:219-225; header-only preservation loses the upper 8 bits, Codex), then parse the final bytes once for logging. No per-path "known rcode" plumbing: the logged rcode is always derived from the final packet + OPT. Tests: oversized NXDOMAIN keeps NXDOMAIN+TC; an oversized extended-RCODE response keeps the full 12-bit value on the wire and in the log.
### S3.2 servFail logging
Every `servFail` site (8, all inside Context) now replies AND logs: `rcode = servfail`, upstream = the last attempted resolver when the failure came from an exchange, policy/route fields as far as the pipeline got.
### S3.3 Post-parse protocol refusals
BADVERS (:245), NOTIMP (:253) and bad-EDNS FORMERR (:231) answer an identifiable question but precede `Context`. Construct the logging context as soon as one question is parsed and log these as `not_evaluated`/`protocol_error`, route `rejected`, with the actual rcode. Pre-question failures (rate-limit REFUSED, unparseable, qdcount≠1) stay counters — unchanged.
### S3.4 Acceptance (S3)
- [ ] Table-driven provenance tests, one asserted row per path: non-IN, paused, no-snapshot, local, forward-zone, forward-zone cache hit, upstream cache hit, upstream answer (exact redacted URL asserted), rule allow, blocklist exception (source id+name), no-match, rule block, blocklist block with source id+name, CNAME-uncloaked block (cname_target + target's reason), safe-search rewrite, each servFail flavor (exchange-failure case asserts the last-attempted resolver), BADVERS, NOTIMP, bad-EDNS.
- [ ] Credential tests (Codex: `Endpoint.parse` rejects `@`, so userinfo cannot come through production config): a handler test with an injected userinfo-bearing identity proves redaction before `Entry.init`; the production-config cross-surface sweep (row, SSE, detail all secret-free, using an accepted credential-bearing DoH path shape) lives in S4.
- [ ] Truncation-rcode regression tests per S3.1.
- [ ] `pool_upstream` and the `"pool"` marker are gone from provenance producers and serializers (scoped grep — logging fixtures elsewhere are out of scope, Codex).
- [ ] `zig build test` 0 failed.
## Session S4: web API + contracts (src/web/, openapi.yaml, contract samples)
- List `QueryRow` serialization gains `qclass`, `rcode`, `route_kind`, `policy_action`, `policy_reason`; `block_reason` is gone (S5 updates the admin in the same milestone; `blocked`/`cache_hit`/`upstream` survive, satisfying "existing summary fields stay").
- New `GET /api/queries/{id}` → 200 nested `{request:{time,domain,client,qtype,qclass}, group:{id,name}, policy:{action,reason,matched,source_id,source_name}, rewrites:{cname_target,safe_search_target}, route:{kind,forward_zone,upstream}, response:{rcode,duration_us}}`; 404 unknown/pruned; 503 no querylog. Route added; the openapi path-count guard (web_integration_test.zig:~2775) and the route-table cardinality assertion (routes.zig:154) update together.
- **One shared full-provenance DTO** (Codex): define `Provenance` (the nested body above) once; `QueryDetail = {id} + Provenance` and the SSE event = `Provenance` exactly — not "QueryRow minus id" (ui-redesign.md:177 requires full live detail). The list row stays a separate summary projection. The live.zig lockstep test compares field names AND types against the shared DTO.
- `GET /api/queries` body gains `coverage: {complete: bool, available_since: i64}`; `complete = (filter.since != null and filter.since >= available_since)` with `available_since` the S1 watermark. `/api/stats` and `/api/stats/timeseries` gain the same pair, judged against the period's aligned `since`. Documented in openapi.
- openapi.yaml: all schemas updated; `/api/queries/{id}` documented; the three enums enumerated. New focused drift guards for QueryRow, QueryDetail (nested objects included), the coverage object and the three enum value sets against `provenance.zig` — comparing field names, types, nullability and requiredness, not names alone (the path-count guard protects none of that, Codex).
- Contract samples regenerated (justfile goldens recipe); add `get_query_detail` sample; drift test green.
- [ ] Web integration: detail 200/404/503; coverage fields present and correct across a since-bounded and an unbounded request; keyset walk green. `zig build test` + `-Dintegration` 0 failed.
## Session S5: admin (admin/src)
Scope deliberately thin — Activity is milestone 29:
- `lib/types.ts`: `QueryRow` updated; new `Provenance` + `QueryDetail = {id} & Provenance`; `QueriesPage` gains `coverage`; `LiveQueryEvent = Provenance`. **`LiveRow` becomes a discriminated union** (Codex: a reconnect gap-fetch returns summary `QueryRow`s, which cannot fabricate full provenance): `{kind:"streamed", event: LiveQueryEvent}` | `{kind:"recovered", row: QueryRow}` — recovered rows keep their `id` and link to `/queries/$id`; a shared summary projection feeds the flat `QueryCells` from either arm. ringBuffer.ts and useLiveQueries.ts updated; tests cover both arms and assert the streamed projection drops no field silently.
- QueryLogPage/Live status cell switches from `block_reason` to `policy_reason`; no other column changes.
- New route `/queries/$id` + detail page modeled on `diagnosticDetailRoute`: the ordered explanation (request, group, policy, rewrites, route, response), historical facts visually separated from current-state links, related actions (lookup the domain, filtered query-log links). Navigation is a real keyboard-focusable link in the row; row-wide pointer click is an enhancement only (Codex).
- Coverage: whenever a response says `complete == false`, show "Query history is available from …" against the effective lower bound (not only when the watermark postdates the window, Codex). This touches the current dashboard's stats consumers — permitted: the anti-requirement below bans an Overview redesign, not this notice.
- [ ] vitest: detail page renders each section from a fixture; hidden-domain fixture renders the marker; coverage-line tests (watermark inside the window and after it); ring-buffer projection test. Typecheck/prettier/oxlint clean.
---
## File ownership
| Files | Session |
| --- | --- |
| storage/querylog_schema.zig, storage/provenance.zig (new), config/limits.zig (new), storage/logger.zig, storage/repositories/queries_repo.zig, storage/retention.zig, config/validate.zig, storage tests, app.zig (recreated-event detail) | S1 |
| upstream/* (incl. both live tests), local/forward_client.zig, server transport fakes + signature fixes in handler.zig/doh/dot/integration/web tests | S2 |
| server/handler.zig + server tests (behavioral) | S3 |
| web/*, openapi.yaml, web_integration_test.zig, contract samples | S4 |
| admin/src/* | S5 |
S1/S2 are disjoint. S2's mechanical signature edits in S3/S4 territory land before those sessions start.
## Anti-requirements
- No response payloads, RR sets, EDNS payloads or packet bytes stored.
- No new querylog index; no normalized provenance tables.
- No upstream label on cache hits.
- No route aliases; no Activity consolidation (step 3); no Overview redesign (step 4) — the coverage notice on existing pages is in scope.
- No config knobs for any of this.
## Acceptance (milestone complete)
- [ ] `zig build test` and `-Dintegration` 0 failed; `zig fmt --check` clean; admin typecheck/vitest/oxlint/prettier clean; goldens drift test green.
- [ ] Live smoke on the real binary: a blocked, an allowed-with-match, a cached, an upstream, a forward-zone and a SERVFAIL query each produce a correct detail page; screenshots taken.
- [ ] CHANGELOG Unreleased entry states the history reset, the aside filename pattern, the truncation-rcode fix, and the shrunk `query_log_buffer_max` range.
+91
View File
@@ -0,0 +1,91 @@
# Milestone 29: Activity consolidation
Redesign step 3 of specs/ui-redesign.md ("Activity", "Time scoping", "Deletions and their cost", build-sequence step 3). Query Log, Live and Lookup merge into one Activity surface; the three old routes and their code are removed in the same change. Admin-only — no Zig, wire or openapi changes; if a session believes it needs one, that is a spec bug to report, not code to write. Codex design review folded in (thread 01a02857); its corrections are marked where they changed the shape.
## Sessions
S1: route-independent primitives only — it must not touch routes.tsx, AppShell.tsx, or move any route-bound page (Codex: the original move-then-delete split could not keep the tree shippable between sessions). S2: one atomic landable session — routes, page assembly, moves, deletion, retargeting, test migration, smoke. Sequential.
---
## Session S1: primitives (no route changes)
### S1.1 Summary projection and cells
- `QuerySummary` (querySummary.ts:13) gains `rcode` and `route_kind` only. **Result derives from the existing `blocked` projection** (which already folds live `policy.action` — Codex: the summary has no `policy_action` and does not need one), else the rcode.
- New `admin/src/features/activity/cells.tsx`: the seven-column set. The Domain cell takes a **route-neutral link-renderer prop** (Codex: S1 cannot reference the not-yet-existing typed route without failing typecheck); S2 supplies the typed `Link` to `/activity/queries/$id`. — Time, Domain, Client, Type, Result, Route, Duration. Table-only compact labels, defined and tested exactly here (the detail page keeps `provenanceCopy`'s long forms): Result → `Blocked` | `NOERROR` | `SERVFAIL` | … (bare rcode name, no numeric suffix); Route → `Blocked` | `Local` | `Forward zone` | `Upstream` | `Cache` | `Rejected`. Non-NOERROR result carries a non-color signal (weight/icon), same treatment as Blocked. Duration formats `response_time_us`, em dash when null.
- Tests: fixtures for blocked, allowed-NOERROR (**explicitly `blocked=false`** — Codex: the allowed case must be pinned, not implied), SERVFAIL, cache hit; each cell's exact text, including an unassigned extended rcode preserving provenanceCopy's `RCODE <n>` fallback shape (compact form without the parenthesized number).
### S1.2 Shared provenance-detail renderer (Codex critical: streamed live rows must keep a detail surface)
Extract the ordered-explanation body of QueryDetailPage (request / group / policy / rewrites / route / response sections, historical-vs-current separation, the honesty wordings) into `admin/src/features/activity/ProvenanceDetail.tsx` taking a `Provenance` plus optional persisted id. QueryDetailPage becomes a thin route wrapper around it (page stays at its current route in S1 — only the body moves to a route-independent component). The renderer takes related-action links as props so S1 stays route-agnostic.
### S1.3 Form/URL plumbing primitives
- `Select` (ui/Select.tsx:23) gains `isDisabled`, passed through to React Aria (Codex: the filter row cannot otherwise be disabled).
- Datetime conversion becomes bidirectional and second-exact: `datetimeLocalToUnix` gets its inverse (`unixToDatetimeLocal`, local time), inputs use `step={1}`. **DST makes local text lossy** (Codex: a fall-back fold maps two instants to one string; a spring-forward gap silently normalizes): the form keeps the original unix value plus a per-field dirty flag, reuses the original unless the operator edited that field, and rejects an edited value that does not format back identically after parsing. Tests: non-zero-second round trips, a DST-fold instant, a nonexistent spring-forward time.
- Search-param validators as pure functions in `activity/search.ts`: `since`/`until` accepted only via `Number.isSafeInteger`; `blocked` only when `typeof === "boolean"`; `mode` in the two-value union defaulting `"history"`; `domain`/`client` trimmed, with the empty string normalized to `undefined` (Codex: `domain=` must not persist as applied state the server treats as no filter). Tests: fractions, Infinity, overflow, quoted booleans, `false`, empty and whitespace strings.
### S1.4 Acceptance (S1)
- [ ] Typecheck, vitest, oxlint, format clean; existing pages still work untouched (only additive files plus the QueryDetailPage body extraction and Select prop).
## Session S2: the Activity surface, atomically
### S2.1 Routes (routes.tsx) — created and deleted in one change
- `/activity` → ActivityPage, `validateSearch` from S1's validators. `loaderDeps` returns an **explicit object** of `{mode, since, until, domain, client, blocked}` — never the whole search (Codex: unknown keys create spurious matches without `search.strict`); the history loader builds `QueriesFilter` **field-by-field** and only calls `ensureInfiniteQueryData` in history mode.
- `/activity/queries/$id` → the detail wrapper (loader unchanged, incl. the deliberate 404 swallow). The route's search carries the originating Activity search (validated by the same functions), so the back link restores the exact investigation view (Codex: a bare `/activity` back link discards context and violates absolute time scoping).
- `/activity/test` → policy simulation, search `domain`, `?domain=` auto-run preserved.
- `/queries`, `/queries/$id`, `/live`, `/lookup` deleted from the route tree in the same edit. No aliases, no redirects; unknown paths fall to the router's existing not-found handling (verify sane, build nothing).
### S2.2 ActivityPage
- **Mode switch is URL-controlled** (the uncontrolled Tabs at ui/Tabs.tsx:76 does not fit — a controlled switch or controlled-Tabs variant), updating via the functional form `search: prev => ({...prev, mode})` so filters survive (Codex: object-replacement navigate loses them). The Live subtree (`LiveActivity`) mounts **only** in live mode so no SSE connection lingers in history. Tests: switching to history closes the EventSource; switching back creates exactly one fresh source.
- History mode: URL is the applied state. The form draft resets whenever the applied search changes (back/forward, pasted URL — Codex: the current seed-once pattern fails back/forward; test both, with non-zero seconds in the bounds). Apply = navigate with the explicit search object; keyset/infinite mechanics (`next_before`, `isPlaceholderData` guard, `hasMore` derivation) carry over; CoverageNotice stays.
- Live mode: follow-by-default, Freeze/Resume ephemeral (never URL). The live stack moves as-is: useLiveQueries (freeze snapshot, gap re-sync, 401 probe, cap threshold, stale-source guards), ringBuffer (LiveRow union, occurrence-counting mergeGap, rationale comment verbatim), fakeEventSource injection, status pill, `aria-pressed` Freeze, missed-recovered note, resync-failed alert, capped alert + Retry, blocked-row tint, footnote count.
- **Live detail**: a streamed row (id null) opens `ProvenanceDetail` in-place from its in-memory `event` — no invented correlation id, no persisted fetch (ui-redesign.md:177). Recovered rows link to `/activity/queries/$id`. **Retention contract** (Codex): selection stores the selected `LiveRow` snapshot itself; the open detail survives ring eviction, gap merges, and Freeze/Resume, closing only on explicit close or mode unmount. Tests pin: opening a streamed row's detail, keyboard access, survival across capacity eviction (501+ events) and across Freeze/Resume.
- Filter row in live mode: disabled via S1's `isDisabled` (visible, value-preserving), including native inputs and Apply/Clear; keyboard/tab-order test proves nothing in the row is tabbable while disabled.
- Detail-only facts, stated per what the wire actually carries (Codex: no source URL or upstream error text exists on `QueryDetail`): matched pattern, historical source label, selected upstream, rcode. Underlying failure text lives in Diagnostics — the detail page's related actions gain the accepted absolute-window Diagnostics link: `/diagnostics?since=<ts-300>&until=<ts+300>` (the redesign's five-minute window around the query, ui-redesign.md:173).
- Related domain/client links carry absolute bounds: `/activity?mode=history&domain=…&since=…&until=…` with **`since = origin.since ?? ts - 300` and `until = origin.until ?? ts + 300`** per bound independently (Codex: the fallback is product contract, not implementation choice). Tests cover zero-, one-, and two-bound origins.
- **Diagnostics bounds must actually apply** (Codex: `/diagnostics` neither validates nor uses `since`/`until` at HEAD, so the ±300s link would render an unbounded page): the diagnostics route gains safe-integer `since`/`until` search validation and explicit loaderDeps, DiagnosticsPage applies them to both the active and resolved queries, and the page shows a visible range indication when bounded. Tests assert the outgoing `/api/diagnostics` requests carry the bounds.
### S2.3 Policy simulation
LookupPage moves to `activity/` with the "Current policy simulation" framing (forward-tense property preserved); verdict rendering, 503/429 handling, group select, same-pair refetch carry over. Reached from an Activity action, not primary nav.
### S2.4 Navigation, deletion, migration
- `NAV_ITEMS`: Query Log/Live/Lookup out, `{to: "/activity", label: "Activity"}` in their position.
- Old page files, their cells/stylex exports, and dead helpers deleted. One home per module under `activity/`; no re-export shims.
- Test migration with the no-shrink rule: every behavior pinned by the old tests is re-pinned at the new home or listed in the report as intentionally dead with its page. Files: QueryLogPage.test, LiveLogPage.test, useLiveQueries.test, ringBuffer.test, LookupPage.test, QueryDetailPage.test, AppShell.test, LoginPage safeRedirect samples (cosmetic).
- Sweep: no import, string or test references `/queries`, `/live`, `/lookup` as routes (API paths `/api/queries*` stay).
- **CHANGELOG.md is S2-owned** (Codex): retarget the still-Unreleased m28 entry's `/queries/{id}` mention to the new route, add the removal notice (bookmarks break) and the Activity surface + Result column.
### S2.5 Smoke + screenshots
Rebuild bundle (`npm run build`, `zig build -Dadmin-dist=admin/dist`), run the real binary with the smoke28 scratchpad config: URL-driven history filters (paste a full URL, screenshot the reproduced view), live streaming with freeze and an open streamed-row detail, the SERVFAIL row distinguishable in the list, detail via recovered/history row with the context-preserving back link, policy simulation, five-minute Diagnostics link landing on the bounded window, mobile-width nav drawer.
### S2.6 Acceptance (S2)
- [ ] Typecheck, vitest, oxlint, format clean; `zig build -Dadmin-dist=admin/dist` succeeds; `zig build test` untouched-green.
- [ ] The S2.2 test list green, including EventSource lifecycle, back/forward, disabled-row tab order, streamed-row detail.
- [ ] Router-level loader regression (Codex): `/activity?mode=live` — including with retained filters and unknown search keys — makes no `/api/queries` request; history mode forwards exactly the six normalized filter fields and nothing else.
- [ ] Screenshots per S2.5.
## File ownership
Sequential; S2 starts only after S1's gates are green. S1: additive files under activity/, querySummary.ts, ui/Select.tsx, the QueryDetailPage body extraction, their tests. S2: everything else listed, admin/src/* (including the diagnostics route/page bounds work) plus CHANGELOG.md.
## Anti-requirements
- No server/API/openapi changes; no golden regeneration.
- No route aliases, redirects, or bookmark shims; no custom 404.
- No filter push-down into the SSE stream.
- No Overview changes (step 4), no configuration-page changes (step 5).
- No new dependencies; no virtualization.
## Acceptance (milestone complete)
- [ ] All admin gates clean; zig suites untouched-green; bundled build succeeds; screenshots per S2.5.
- [ ] CHANGELOG updated per S2.4.
+43
View File
@@ -0,0 +1,43 @@
# querylog.db: wal_autocheckpoint = 8192
One constant. The v0.0.7 batching cut process writes from ~0.5 to 0.281 GiB/day (measured over a 10 h process lifetime on the Pi); ~130 MiB/day of the remainder is autocheckpoint writeback — SQLite's 1000-page default trips every ~40 min and rewrites the same hot index/interior pages into the main db each time. At 8192 pages (32 MiB at the 4096-byte page size) the cadence drops to ~5 h, cutting those in-place rewrites ~8x, expected total ≈190 MiB/day. The previous SD card died of write wear; the current card's endurance is unknown, which argues for cutting known writes, not against it. Codex approved the decision and this shape (thread 01a0205d).
## Decision
`PRAGMA wal_autocheckpoint = 8192` on every read-write querylog.db connection. Hardcoded constant, no config knob, no checkpoint task. `synchronous=NORMAL` and the daily retention `wal_checkpoint(TRUNCATE)` (queries_repo.zig:169, called from the retention pass) stay as they are. config.db — including the diagnostics store's connection, which `app.zig:416` opens via `openConfigDb` despite the variable name `events_db` — keeps the SQLite default. There are exactly two database files; nothing named events.db exists.
## Durability contract (goes in the constant's comment, stated precisely)
- Commit never fsyncs at `synchronous=NORMAL`; the checkpoint's fsync is the only guaranteed durability boundary. This change moves that boundary from ~40 min to ~5 h of querylog data (query rows + upstream-history minutes) under power loss or kernel panic. Typical loss stays far smaller (kernel writeback), but that is not a guarantee.
- Process crash or clean stop loses nothing committed, at any threshold. Consistency is never at risk: recovery replays the longest valid WAL prefix atomically.
- 32 MiB is an expectation, not a cap: a pinned reader snapshot stops a passive checkpoint partway and the WAL overshoots until the reader finishes; the daily TRUNCATE is the backstop that shrinks the file.
## Implementation
1. **src/storage/db.zig**`Pragmas` (:753) gains `wal_autocheckpoint_pages: ?i32 = null`. `applyPragmas` (:762), when non-null: `PRAGMA wal_autocheckpoint = N;` then read back via the pragma's own return and fail loudly on mismatch — mirror the `foreign_keys` set-and-verify at :781-783. Default null leaves every existing `.{}` caller (config.db sites, tests) untouched with zero diffs. db.zig stays generic; it must not know the word querylog.
2. **src/storage/querylog_schema.zig** — owns the constant (the module already owns querylog policy: fingerprint, DDL, recreate classification): `pub const wal_autocheckpoint_pages: i32 = 8192;` carrying the durability contract above as its comment. Passed at both production `applyPragmas` sites: the probe path (:129) and `createFresh` (:253). The third `applyPragmas` in that file (:273) is inside an in-memory test and stays `.{}` deliberately.
3. **src/cli.zig**`reopenQuerylogDb` (:354) passes the constant. These three sites are the only read-write querylog connections by construction — every open flows through `querylog_schema.open` or `DataDir.reopenQuerylogDb`.
## Tests
- Unit, db.zig, in-memory (the pragma reads back per-connection regardless of journal mode): default `Pragmas` leaves `PRAGMA wal_autocheckpoint` at 1000; a set value reads back.
- File-backed storage integration test through the real helpers: `openQuerylogDb` and `reopenQuerylogDb` connections both read back 8192; an `openConfigDb` connection reads 1000.
- The read-back inside `applyPragmas` makes misapplication loud at startup, complementing both.
## Docs
- CHANGELOG Unreleased, Changed: the checkpoint cadence change, the measured why, and the widened power-loss window stated per the durability contract (not as an unconditional bound).
- One short paragraph appended to specs/querylog-batching.md linking here.
## Rejected (do not relitigate without new facts)
- Config knob: nobody tunes this twice; scope is small on purpose.
- Periodic checkpoint task: reproduces autocheckpoint with more moving parts (cadence state, busy handling, shutdown, diagnostics).
- `wal_autocheckpoint=0` + daily TRUNCATE only: unbounded intraday WAL growth under reader pinning; strictly worse.
- `journal_size_limit`: redundant with the daily TRUNCATE, and rejecting it needs no claim about passive checkpoints never truncating (after a completed checkpoint resets the WAL, the limit does truncate on the next write — the knob is merely surplus here).
- Touching config.db policy or differentiating reader vs writer querylog connections: readers cannot trip checkpoints, the pragma is inert on them; uniformity is simpler.
## Gates
1. `zig build test` and `-Dintegration` 0 failed; fmt clean.
2. Field verification on the released build (the Pi deploys releases, not branches, so this necessarily follows the cut — owner-ordered 2026-08-21): confirm the WAL resets normally, writeback falls materially, and no batches drop. What to measure and over what window is the deployment side's call; a bad result reverts the constant in a follow-up patch release.
+29
View File
@@ -0,0 +1,29 @@
# Query-log batching: one transaction per interval, not per query
Measured on the deployed Pi: ~0.5 GiB/day of process writes to persist ~2.3 MB/day of query rows, because the writer's 100 ms batch window degenerates to one transaction per query at household rates (~0.25 qps). Write wear killed the previous SD card. Fix: widen the window, Pi-hole precedent (FTL DBinterval=60).
## Contract
- New config field `logging.query_log_flush_interval_s: u16`, default 60, valid 03600, dedicated validation classification (not folded into an existing one).
- `0` means: do not wait for more entries; immediately flush the entry plus whatever is already queued, up to `flush_batch` rows. It is NOT per-query power-loss durability — `synchronous=NORMAL` never promised that; say so in the reference.
- Writer loop (`src/storage/logger.zig`): block on `getOne`, then deadline = now + interval on the **`.boot` clock** (matching upstream history; `.awake` would stretch the window across suspend), fill until deadline or `flush_batch` (100) rows, one transaction per flush; if the queue holds more, keep flushing in `flush_batch` chunks.
- Shutdown (the Codex-found race, fix required): today `Logger.shutdown` closes the queue and the app then cancels the task group containing the writer, so cancellation can beat the close-observation and drop the held batch. New order: stop and join query producers first, close the queue, await the writer outside the cancelled group (give the writer a lifetime separate from the producers' group). If the disk gate forbids the final write, drain and count the entries as dropped — never hang shutdown, never lose them uncounted.
- Unchanged on purpose: queue cap `query_log_buffer_max` + drop-oldest backpressure, `flush_batch`, pragmas, wal_autocheckpoint, retention, domains interning. One commit per minute makes those second-order.
## Propagation (config field checklist)
Model key + round-trip drift guards, validation + validation reference, settings API view (`src/web/handlers/settings.zig`), openapi.yaml, admin types + settings control, contract samples regenerated, docs config reference, CHANGELOG.
## Documentation wording
- Crash-loss window: up to `interval` seconds of query history on process failure; power loss can additionally lose recent committed transactions (WAL + synchronous=NORMAL). Query history is the least valuable data on the box.
- Staleness: every query-log-backed read (query-log page, totals, timeseries) lags up to `interval` seconds. The live view is unaffected — it is fed from the hub before the queue.
## Follow-up
The "unchanged on purpose" line above no longer holds for `wal_autocheckpoint`. Batching landed and the Pi measured 0.281 GiB/day, of which ~130 MiB is autocheckpoint writeback — second-order next to one transaction per query, first-order next to one per minute. specs/querylog-autocheckpoint.md raises the threshold to 8192 pages on every read-write `querylog.db` connection and states the durability contract that comes with it. Nothing else in this spec changes.
## Acceptance
- [ ] Deterministic tests: interval batching (entries within the window land in one transaction), flush_batch early flush, 0-sentinel immediate flush, shutdown drains a held batch and the queue (the race sequence: producers stopped → queue closed → writer awaited), disk-gated final drain counts drops.
- [ ] Suites: `zig build test` and `-Dintegration` 0 failed; admin typecheck/vitest/prettier clean; goldens regenerated.
+56
View File
@@ -0,0 +1,56 @@
# Local release cut: justfile + tools/cut.zig
Cutting v0.0.7 by hand missed the build.zig.zon bump; verify-dist caught it one CI round late. The cut becomes a compiled, tested tool (the same ruling that moved publication out of workflow shell into tools/release.zig), invoked from a thin justfile. Design reviewed and accepted by Codex (thread 01a020f9); its findings are folded in below.
## justfile (repo root)
Recipes only — no variables, no embedded logic:
- `test``zig build test`
- `itest``zig build test -Dintegration`
- `admin-check``cd admin && npm run typecheck && npm run test && npm run lint && npm run format:check` (the committed npm scripts, never npx: the repo records that npx can fetch an unpinned package)
- `goldens` → the documented `-Dcontract-samples-out` invocation; comment says what `admin/src/lib/contractSamples.gen.ts` is: captured live API bodies the admin tests assert against
- `build``cd admin && npm run build`, then `zig build -Dadmin-dist=admin/dist` (the default build embeds a placeholder page)
- `verify``itest` + `admin-check` + `zig fmt --check build.zig src tools` — described honestly as the fast local checks, not the CI gate (it skips admin `npm run build`/`assert-bundled`, cross-targets, verify-dist)
- `release kind``zig build cut -- {{kind}}` (kind ∈ major, minor, patch)
`verify` must NOT also depend on `test`: `-Dintegration` already runs the whole ordinary suite.
## tools/cut.zig
Wired like the other host tools (`hostTool` + `addRunArtifact`, see build.zig ~230): `zig build cut -- {major|minor|patch}`. NOT installed to zig-out/bin. Its tests join `zig build test`.
Constants: one repo API base `https://git.mial.net/api/v1/repos/mokhtar/nxdns` (the tool can only ever target this repo — no configurability). The runs API needs a token (verified: anonymous GET is 401); read it from `~/.config/tea/config.yml` (logins entry for git.mial.net); a missing token is a clear error naming the file.
### Sequence
1. **Derive the version**: the argument is a bump kind — `major`, `minor` or `patch` — never a free-form number (a number validated as "greater semver" still admits every typo, and a published tag is immutable). Parse `.version` from build.zig.zon (precedent: tools/verify_dist.zig:220, container_check.zig:317). If `v<zon-version>` exists on origin, derive the next version from the bump kind (patch 0.0.7→0.0.8, minor→0.1.0, major→1.0.0; checked u32 arithmetic). If it is absent, the zon version is an in-progress cut: resume it instead of incrementing again, reported explicitly — this preserves the bump-committed-but-untagged rerun and local-tag adoption. The CHANGELOG check in preflight applies to the derived (or resumed) version, so a changelog written for the wrong bump kind fails as a mismatch.
2. **Preflight**: working tree clean; branch master; CHANGELOG.md has a `## [<v>] - YYYY-MM-DD` heading (dated, the repo's observed form; do NOT require the date be today) with a non-empty section body (the CI tool refuses a blank section — failing later just burns the tag); the tag-free check follows the plan: a derived version does its own `git ls-remote` and refuses if `v<v>` exists; a resumed version reuses the absence that selected it (asking twice invites two answers). Transport/auth failure is always distinguished from "no match" (exit code + stderr, never "nonzero means absent") and refuses — on the resume path an error read as absent would resume a released version. The local tag namespace is checked too — see resumability.
3. **Bump if needed**: rewrite build.zig.zon atomically, reparse it, assert the git diff contains exactly that one file, then `git commit -S -m "build: bump version to <v>"`. git runs with inherited stdio so pinentry can prompt; check the child's actual termination state.
4. **Push master**, capture the exact HEAD sha; all later status lookups and the tag use that sha explicitly.
5. **Wait for CI**: poll the runs API for the ci.yml run matching that sha and event push. A run-id floor captured before the push scopes the match — applied only when the push actually moved the ref (`git push --porcelain` destination flag), so a no-op push (resume case, or a concurrent identical push) adopts the existing run for that sha. Require the run to appear within a startup deadline; wait for its terminal conclusion. Refuse to tag on anything but success, naming the failing job from the commit-status contexts. Every HTTP attempt individually bounded by a monotonic deadline; transport/JSON errors are reported, never silently treated as pending.
6. **Reassert** HEAD and tree unchanged, then `git tag -s v<v> -m v<v> <sha>` and push the tag.
7. **Wait for the release run** (matched by workflow path `release.yml@refs/tags/v<v>` — Gitea reports tag pushes as event "push"): startup deadline for the run to appear; the completion clock starts at first sighting, with a ceiling derived from the workflow's sequential jobs — guard 15 + gates 60 (the tool's own CI bound) + publish 120 (release.yml:220) = 195 minutes. Before tagging, reassert HEAD, the tree, and (for an adopted tag) the tag object id; every tag pushed — created or adopted — must carry a signature whose VALIDSIG primary fingerprint equals the one release.yml pins as TAG_SIGNING_FPR. Report the terminal conclusion; on failure name the failing context.
8. **On success**: GET the release object, require it published (not draft), print the tag and asset names.
### Resumability
A failed run must not strand the operator:
- Bump pushed, then failure: rerun continues (preflight sees the version already bumped).
- Local tag exists but never reached origin: verify it is an annotated tag by this tool's convention pointing at the current HEAD — adopt it; otherwise refuse with the exact `git tag -d` to run. Never delete a tag that exists on origin.
### Tests (in-file, join `zig build test`)
Pure functions unit-tested: semver validation (accept/reject table incl. leading zeroes, `v` prefix), bump-kind parse, derivation table with the minor/major resets and overflow refusals, derive-vs-resume decision for all three kinds, zon `.version` parse + rewrite round-trip, changelog heading + non-empty body check, runs-JSON → decision (running / success / failure / no-run), tea-config token extraction. Process spawning and HTTP live behind thin call sites and are not mocked.
## Anti-requirements
- No general release framework; no shared process plumbing extracted unless a third caller appears (container_check.zig:24 rule).
- No confirmation prompts — invoking `just release <kind>` is the authorization.
- No secrets in argv, no token printed.
## Acceptance
- [ ] `just --list` shows the recipes; `just verify` passes locally.
- [ ] `zig build cut -- patch` derives the next version and refuses in preflight on a dirty tree or a missing changelog section, mutating nothing; `zig build cut -- 0.0.9` and `-- banana` refuse naming the three kinds.
- [ ] `zig build test` and `-Dintegration` 0 failed; `zig fmt --check` clean.
+304
View File
@@ -0,0 +1,304 @@
# UI redesign proposal
Author: Codex (gpt-5.6-sol, extra-high effort), 2026-08-19. **Not accepted yet.** Untracked on purpose until Mokhtar rules on the open questions at the end.
Answers that shaped it: the server and API may change; the surface-ownership split is right; file-mode configuration pages are read-only; diagnostics are curated structured events in the vein of Pi-hole's; time scoping is per workflow; a past query must be explainable exactly; Query Log and Live merge.
## Navigation
Five primary items. Configuration expands to three task-shaped subpages and holds no landing route of its own.
| Navigation | Route | Operator question | Replaces |
| --- | --- | --- | --- |
| Overview | `/overview` | Is DNS healthy and protecting the household now, and what happened in this period? | Dashboard |
| Activity | `/activity` | What requests are happening or happened, and why did nxdns handle them that way? | Query Log, Live, Lookup |
| Clients | `/clients` | Who is this address, which policy applies, and what has it been querying? | Clients |
| Diagnostics | `/diagnostics` | What is failing or has failed, what is affected, what should I do? | new |
| Protection | `/configuration/protection` | What policy governs each group, and which rules and lists produce it? | Groups, Blocklists, Rules |
| Resolution | `/configuration/resolution` | Where does nxdns answer or forward permitted names? | Local DNS, Upstreams |
| System | `/configuration/system` | What service, storage, logging, TLS and web settings is this process running with? | Settings |
Secondary routes, reached from those surfaces rather than the nav: `/activity/queries/:id`, `/activity/test`, `/clients/:id`, `/diagnostics/:id`.
No current page survives unchanged. Login, logout and pause survive functionally, restyled into the new shell.
### Activity
Two modes over the same columns and filters. History is persisted queries with keyset pagination and an absolute range. Live is follow-by-default with Freeze/Resume over the existing bounded 500-row buffer.
Columns: Time, Domain, Client, Type, Result, Route, Duration. Rule matches, source URLs and upstream errors live in the detail view, never on every row.
Domain testing stays as an Activity action labelled "Current policy simulation". It must never read as an explanation of a historical query.
### Protection
Group-centred: group list, selected group detail, effective safe-search setting, assigned blocklist sources, rules scoped to the group, client count linking to matching clients. A Sources tab holds the shared blocklist catalogue and the "Update now" runtime action.
### Resolution
Three tabs: upstream pool, local records, forward zones.
### Clients
Keeps primary navigation because identifying and naming unknown devices is an operational job, not configuration. The *learned* marker appears only here, beside the name. Prefix assignments live here as "Network assignments".
## Overview
Three sections, nothing else.
**1. Current status.** Five current facts, each conveyed by text and icon as well as colour. Healthy rows stay quiet; degraded rows link to the diagnostic or configuration surface that explains them.
| Status | Shows | Why it belongs |
| --- | --- | --- |
| Protection | Active, paused until a timestamp, or unavailable, with Pause/Resume | Confirms filtering is in force, and carries the valid runtime action |
| Upstreams | available / configured, now | Confirms DNS can leave the network |
| Query history | Recording, losing rows, or writer failed | Says whether Activity can be trusted |
| Diagnostics | Recording or unavailable | A failure reporter that cannot record failures must itself be visible |
| Storage | ok / low / critical, free bytes | Says whether writes are safe, and explains write gating |
The shell carries a small global "Protection active/paused" indicator linking back to Overview. The controls themselves stay on Overview and beside blocked-query details.
**2. Active issues.** Severity, short title, affected object, how long it has been active, link to the detail. When none exist, one restrained line: "No active operational issues." Resolved failures never appear here, and healthy subsystems never get permanent green cards.
**3. Activity over a period.** The existing 1h / 24h / 7d / 30d control. Every value uses exactly the returned `[since, until)` window: queries, blocked count and rate, distinct clients, average response time, one query-volume timeline split blocked/cached/other, and "Open activity for this period" carrying the exact bounds. The timeline stays the existing lightweight SVG; no charting dependency.
If the selected period predates available data, the section says "Query history is available from …" rather than charting the missing span as zero.
Removed from Overview: the historical upstream table, the "last failure" text, per-upstream period rates, the database and log byte breakdown, the standalone cache card.
## Diagnostics
Not a journald viewer, and it does not subscribe to `std.log`. Producers emit a finite set of typed events at the failure boundary.
### Event model
One row is one failure episode.
| Field | Meaning |
| --- | --- |
| `id` | Durable identifier |
| `code` | Fixed machine-readable kind |
| `subject_key` | Internal stable identity, may hold a full URL, never serialized |
| `subject_label` | Bounded, redacted, operator-facing identity |
| `severity` | `warning` or `error` |
| `first_seen` / `last_seen` | Episode bounds |
| `occurrences` | Deduplicated report count |
| `resolved_at` | Null while active |
| `detail` | Bounded last error or current condition |
No `info` severity. Normal starts, refreshes and reloads do not become entries; a success resolves its prior failure.
Active dedup key is `(code, subject_key)`. A repeat updates `last_seen`, `occurrences`, severity and detail. A success resolves the row. A later failure opens a new episode rather than reopening the old one. Severity records the worst state reached. There is no acknowledgement or manual dismissal: active means the component has not demonstrated recovery. One-shot material events, such as a query-log recreation, are inserted already resolved.
### Storage
Events live in `config.db` as runtime state, excluded from export, import and file reconciliation. Active rows are never pruned. Resolved rows keep 90 days, with a hard cap of the newest 5,000. `detail` caps at 512 bytes. Pruning runs at startup and from existing maintenance; no new scheduler.
One table, one repository, one fixed event-code enum. A small serialized event store owns a dedicated `config.db` connection; background producers report synchronously through its mutex. This is the existing pattern, not a new one: `app.zig:333` opens a connection for migration and reconciliation and `app.zig:544` opens a separate `web_config_db`, and the background producers (fetcher, disk monitor, logger writer) have no other safe path into `config.db`.
If the store itself cannot write, an atomic `event_store_failed` state appears in `/api/health` and journald. It clears on the next successful write — no periodic probe. The store receives a write whenever anything fails or recovers, and a flag left set while nothing needs writing costs nothing.
### Event sources
| Source | Event identity | Recovery |
| --- | --- | --- |
| Disk warn/critical transitions | `disk.space`, singleton | next `ok` sample |
| Failed `statvfs` or directory sizing | `disk.probe`, keyed by operation/path | next successful probe |
| Per-source download, HTTP, parse, compile or file-read failure | `blocklist.refresh`, keyed by source URL | that source refreshes |
| Initial snapshot or whole-pass failure | `blocklist.snapshot`, singleton | a snapshot publishes |
| Blocklist file cleanup failure | `blocklist.storage`, keyed by operation | that operation succeeds |
| Certificate stat or reload failure | `certificate.reload`, keyed by `doh`/`dot` | files readable and reload succeeds |
| Query writer init or batch failure | `query_log.write`, keyed by `writer`/`batch`/`queue` | writer starts, or a batch succeeds without drops |
| Query retention prune/checkpoint/vacuum failure | `query_log.maintenance`, keyed by operation | that operation succeeds |
| Upstream-history flush failure | `upstream_history.write`, singleton | next flush succeeds |
| Client-name selection or persistence failure | `client_names.storage`, keyed by operation | next pass succeeds |
| Client materialisation or pruning failure | `clients.storage`, keyed by operation | next pass succeeds |
| Upstream exchange failure | `upstream.exchange`, keyed by upstream URL | next successful exchange |
| Enabled DoH/DoT listener that cannot start | `listener.start`, keyed by endpoint | successful start after restart |
| Query-log recreation | `query_log.recreated`, one-shot | inserted resolved |
| Configuration warning leaving a capability skipped | `configuration.load`, keyed by setting | clean load after restart |
Sixteen codes. Codex proposed a seventeenth, `api.storage`, for a storage failure that produced an HTTP 500 — cut on review, because it breaks this section's own exclusion rule: a 500 already answered its caller. Do not merge the remaining codes to shrink the count either. A merged code forces `subject_key` to carry what the code no longer says.
An upstream event describes a consecutive failure episode, not one row per retry. One timeout followed by success is one resolved episode.
Outside Diagnostics: invalid requests, conflicts, rate limits and failed logins already answered to their caller; expected reverse-DNS outcomes; individual TLS handshake failures already counted; development asset-server warnings; fatal startup failures that stop the UI existing. Disk-gated skips do not duplicate — the active disk event explains the cause and counters keep the totals.
### From event to remediation
The detail shows current or resolved state; first seen, last seen, occurrences, resolution time; impact in operator language; the last bounded underlying error; the exact next action; how nxdns will verify recovery; links to the relevant configuration and activity window.
The frontend uses one exhaustive `switch` over the fixed codes for titles, impact, remediation and routes. No generic action schema, no plugin mechanism.
## Query provenance
Stays in the expendable `querylog.db`. Does not go into the operational-events table.
`block_reason` is replaced by the fuller `policy_reason`. New columns:
| Column | Type | Purpose |
| --- | --- | --- |
| `qclass` | INTEGER NOT NULL | explains filtering bypass for non-IN questions |
| `rcode` | INTEGER NOT NULL | the client-visible result, including SERVFAIL |
| `group_id` | INTEGER | group at query time, not a foreign key |
| `group_name` | TEXT | historical label, survives a rename |
| `policy_action` | TEXT NOT NULL | `not_evaluated`, `allow`, `block` |
| `policy_reason` | TEXT NOT NULL | pipeline or matcher reason |
| `matched` | TEXT | exact rule pattern or list entry |
| `source_id` | INTEGER | blocklist source at query time |
| `source_name` | TEXT | historical source label |
| `cname_target` | TEXT | target that caused an uncloaked block |
| `safe_search_target` | TEXT | name used for the rewrite |
| `route_kind` | TEXT NOT NULL | `blocked`, `local`, `forward_zone`, `upstream` |
| `forward_zone` | TEXT | exact matched zone |
`upstream` changes from the unhelpful `"pool"` marker to the actual configured upstream or forward resolver **on the exchange that actually happened**. On a cache hit `route_kind` is `cache` and `upstream` is NULL. Credentials are redacted at the serialization boundary.
Codex wanted the upstream recorded on cache hits too. Cut on review: it would widen every `src/cache/dns_cache.zig` entry to carry an upstream label, and it states a half-truth, because on a cache hit no upstream answered. The interface change it does need is real and worth doing — `handler.zig:79-83` records ruling 20, which deliberately keeps the pool's answering endpoint out of `transport.Client`'s reach. Exposing it touches `src/upstream/`, not the pure core. Do not fall back to `"pool"`: the exact upstream on a SERVFAIL row is the single most useful correlation this redesign adds.
`policy_reason` is a closed enum: `local_record`, `forward_zone`, `non_in_class`, `paused`, `snapshot_unavailable`, `no_match`, plus the existing rule allow/block and blocklist exception/domain/wildcard reasons.
For a CNAME-uncloaked block, the policy fields describe the target's decision and `cname_target` preserves the target responsible.
Every syntactically parsed request that receives a response is logged, including synthesized SERVFAIL. Requests too malformed to identify a question stay counters, not fabricated rows.
No response payloads, answer RR sets, EDNS data or packet bytes are stored. The record explains nxdns's own decision, not the external resolver's answer.
Privacy transforms apply to every new domain-bearing field, not only `domain`: with `hide_domains` on, matched names, CNAME targets and safe-search targets hide consistently.
A one-row `querylog_meta (created_at INTEGER NOT NULL)` table lets the stats and query APIs return a conservative `available_since`, which distinguishes "zero queries" from "history does not exist".
### Historical query detail
Ordered explanation: request (time, domain, client, type, class); group (historical id and name); policy (evaluated or not, allow/block, exact matched rule or list candidate, historical source, whether filtering was paused or unavailable); rewrites (safe-search target, CNAME target); route (local, forward zone, cache, or selected upstream); response (rcode, duration); related actions (test the domain against current policy, view the client, view activity for the same domain or client, view diagnostics in a five-minute window around the query).
Historical and current facts are visually separated. A rule, source or group that no longer exists stays visible as a historical value and is not linked to a different current object.
Live SSE events carry the same provenance shape without a persisted `id`. A frozen live row shows its in-memory detail; no correlation id is invented to link it to a row SQLite has not written.
## Time scoping
One contract: unix seconds UTC, `since` inclusive, `until` exclusive, point data qualifies on `since <= ts < until`, diagnostic episodes qualify when their active interval overlaps the range, current state is labelled "Now" and no historical selector touches it.
URLs: `/overview?period=24h` with the server returning the exact aligned bounds; `/activity?mode=history&since=…&until=…` with `domain`, `client`, `blocked` and the other filters in the URL; `/diagnostics?since=…&until=…&severity=…&component=…`; `/activity?mode=live` with Follow/Freeze as ephemeral UI state. Investigation links always carry absolute bounds, so a viewed incident does not drift as time passes. Timestamps display in the browser's timezone; URLs and APIs stay timezone-independent.
## File mode
One persistent authority line in the shell:
> File-managed · `/etc/nxdns/config.zon` · loaded 19 Aug 2026, 08:42
It sits in the Configuration sub-navigation and appears elsewhere as a compact lock indicator. No full-width banner on every route. The wording is "running configuration loaded from", not "file contents" — the server cannot prove a since-edited file still matches the running process.
A file-managed page uses definition lists for scalars and tables or cards for collections, with human labels and the exact ZON key shown secondarily (`logging.retention_days`). No text inputs, no checkboxes, no Add/Edit/Delete, no disabled form shells, no simulated Save. A short page note says where edits happen and that a restart may be needed.
Runtime actions stay ordinary enabled buttons: pause/resume, update blocklists now, reload certificates, delete an observed undeclared client, login/logout.
Database mode uses the same information architecture with real edit actions, plus a server-owned `restart_pending` boolean. The current client-only restart banner and its local store are removed, so a browser refresh cannot erase the warning.
## API changes
`GET /api/config/status``{authority, path, reconciled_at, restart_pending}`. `restart_pending` is process state: database-mode mutations that need a restart set it, a successful restart clears it.
`GET /api/health` becomes explicit about every condition that contributes to degradation — `protection`, `upstreams`, `query_history`, `upstream_history`, `diagnostics`, `disk`, each an object with its own state. The current hidden `history_flush_failing` contribution is eliminated: nothing may degrade the rollup without appearing in the response.
`GET /api/diagnostics?state=&severity=&component=&since=&until=&limit=&before=` returns `{events[], next_before, active:{warnings, errors}}`. `GET /api/diagnostics/{id}` returns one event or 404 after retention. No acknowledgement, dismissal, generic-action or raw-log endpoints.
`GET /api/queries` keeps keyset pagination and its filters; rows gain `rcode`, `route_kind`, `policy_action` and the short policy reason the table needs, and the body gains `coverage: {complete, available_since}`. `GET /api/queries/{id}` returns nested `request` / `policy` / `route` / `response` provenance. `GET /api/queries/live` sends the same object without `id`.
`GET /api/stats` and `/api/stats/timeseries` add `complete` and `available_since`.
Existing mutation endpoints stay specific. Diagnostics introduces no generic "perform remediation" endpoint; it invokes the existing blocklist-refresh and certificate-reload operations.
## Schema changes
`config.db`:
```sql
CREATE TABLE operational_events (
id INTEGER PRIMARY KEY,
code TEXT NOT NULL,
subject_key TEXT NOT NULL,
subject_label TEXT NOT NULL,
severity TEXT NOT NULL CHECK (severity IN ('warning', 'error')),
first_seen INTEGER NOT NULL,
last_seen INTEGER NOT NULL,
occurrences INTEGER NOT NULL CHECK (occurrences > 0),
resolved_at INTEGER,
detail TEXT NOT NULL DEFAULT '',
CHECK (resolved_at IS NULL OR resolved_at >= first_seen)
);
CREATE UNIQUE INDEX idx_operational_events_active
ON operational_events(code, subject_key)
WHERE resolved_at IS NULL;
CREATE INDEX idx_operational_events_last_seen
ON operational_events(last_seen DESC);
```
Absent from the configuration table lists and the reconciliation delete order.
`querylog.db`: `querylog_meta` and the provenance columns above. No new index — Codex proposed `idx_query_log_rcode`, cut on review, because every rcode question the UI asks is time-scoped and `idx_query_log_ts` already bounds the scan. 700k rows on a Pi 5 do not need a second index for a rare filter, and every index taxes the hot insert path in `logger.zig`.
No new provenance table, no key/value store — household retention makes nullable columns cheaper than a normalized graph of decision objects.
### Entry buffer widths
`logger.zig`'s `Entry` uses fixed buffers sized by `max_reason_len = 32` and `max_upstream_len = 64`, and travels through the `Io.Queue` by value. The new fields roughly triple it: `matched` holds a full pattern, and `cname_target`, `safe_search_target` and `forward_zone` each hold up to 253 bytes. That is fine at `flush_batch = 100`, but the widths are part of this contract and must be set explicitly, not left to whatever the first implementation picks.
`transformed()` must hide `matched`, `cname_target` and `safe_search_target` under `hide_domains`, not only `domain`.
On recreation: query rows, provenance and upstream-minute history reset together as today; the old file stays aside; `config.db` diagnostics survive; a resolved `query_log.recreated` event records the reason, the aside path and the new coverage start; Overview and Activity report the incomplete range instead of charting zero.
## Deletions and their cost
| Deleted | Lost |
| --- | --- |
| Separate Query Log and Live pages | separate bookmarks; both modes remain in Activity |
| Standalone Lookup page | a top-level bookmark; testing remains under Activity |
| Top-level Groups, Blocklists, Rules, Local DNS, Upstreams, Settings | direct resource navigation; all capabilities remain under task-shaped configuration |
| Historical upstream table on Overview | at-a-glance period rates; availability stays, failures move to Diagnostics |
| "last failure · 9h ago" text | nothing actionable; the episode becomes a diagnostic |
| Detailed DB/log byte gauges | exact component sizes stay in Prometheus; free space stays on Overview |
| Standalone cache card | one prominent number; cache stays in the timeline and metrics |
| Ephemeral blocklist `SourceStatusSection` | transient success detail after navigation; durable counters and diagnostics remain |
| Disabled configuration forms in file mode | the illusion that fields can be edited |
| Global read-only banner | repeated warning text; authority stays visible once in the shell |
| Client-only restart banner state | nothing reliable; server-owned `restart_pending` replaces it |
| Old route aliases and redirects | existing bookmarks break; no permanent duplicate routing layer |
## Build sequence
Each step leaves the app working and shippable, and updates its OpenAPI contract, generated sample, TypeScript types and deterministic tests before landing.
1. **Diagnostics vertical slice.** `operational_events` schema, repository, serialized store, retention, health state, every typed emitter, API, UI, navigation. Recovery paths and deterministic failure-injection tests per event code. The rest of the UI stays intact.
2. **Query provenance vertical slice.** Query-log fingerprint and schema, metadata table, provenance capture in the handler and logger, parsed-SERVFAIL logging, detail and coverage APIs, the historical detail route, and the `query_log.recreated` emission. Existing list summary fields stay so the current pages keep working.
**This step destroys the existing query history.** The provenance DDL edit changes the fingerprint, so `querylog_schema.open` recreates the file and keeps the old one aside as `querylog.db.schema-changed-<unix seconds>`. Acceptable pre-v0.1, and `available_since` carries the story in the UI, but it is a consequence of this step and must be stated in its spec and its changelog entry. Its acceptance tests cover the recreate, the aside name and the coverage sequence — which is also the natural first `query_log.recreated` emission.
Parsed-SERVFAIL logging reverses ruling 20: `handler.zig:507` counts today rather than logging. The `Context` exists at every `servFail` site that follows question parsing. Pre-parse failures correctly stay counters.
3. **Activity consolidation.** The unified History/Live surface, URL filters, freeze/follow, live detail, current-policy test, historical detail links. Query Log, Live and Lookup routes and code are removed in the same change. Route, SSE, accessibility, reconnect and bounded-buffer tests.
4. **Overview replacement.** Current status, active diagnostics, one coherent activity section. New health contract and completeness states. The upstream-history table, stale-failure text, detailed DiskCard and cache card go.
5. **Task-shaped configuration and file mode.** `/api/config/status` and server-owned `restart_pending`. Protection, Resolution and System in both read-only and editable forms. Clients and its detail route redesigned. Old configuration routes replaced atomically; global banner and disabled forms removed.
6. **Contract closure.** Remove obsolete queries, types, stores, CSS, tests and route fixtures. Regenerate contract samples, update OpenAPI and reference docs, add cross-surface acceptance tests for investigation links, file authority, query-log recreation, active-event recovery and time bounds. Zig, frontend, integration, accessibility and byte-budget checks; no new dependency.
## Codex's least-certain calls
- **Clients in primary navigation.** It earns the slot if identifying unknown devices and checking their group is routine. If Mokhtar almost always reaches a client from a query, Clients moves under Protection and leaves the nav.
- **Recording the exact selected upstream per query.** It materially improves correlating SERVFAIL queries with upstream events, but needs the pool exchange result to expose the selected target. Drop back to `"pool"` only if that interface change proves invasive and exact resolver identity never changes an action.
- **90-day / 5,000-event retention.** Conservative fixed bounds, not settings. Change only after measuring real row size and event rate on the Pi; do not add configurable retention pre-emptively.
## Rulings (Fable review, 2026-08-20)
Verdict: build it, with the four cuts folded in above and the rulings below.
**Diagnostics live in `config.db`. Accepted.** The `querylog.db` alternative is self-refuting: that file is recreated on any schema edit or corruption, so the recreation event dies with the thing it describes. A third database needs either its own migration discipline or a recreate policy that loses the events — the same problem with more files. The write-traffic objection is overstated twice: event volume is failure-rate volume, deduplicated, with no `info` severity, so it is near zero in steady state; and `config.db` already takes operational writes, because client rows are materialized from traffic (PLAN §3.5, `clients.last_seen`, `learned_name`). Pre-v0.1 the table is one edit to `ddl_v1` with no migration step. Keeping it out of `delete_order` and `table_names` is enforceable — `config_schema.zig` has tests pinning those lists.
**`group_name`, `source_name` and `matched` stay TEXT. Accepted, and my inconsistency objection was weaker than I put it.** The `domains` table exists because a domain appears on every row, runs to 253 bytes, and feeds `GROUP BY` stats. None of that holds here. The closer precedent is `client_ip`, which is TEXT with the comment "not a FK: log rows are immutable facts" (`querylog_schema.zig:37`). `group_name` is short and mostly `default`; `source_name` and `matched` are non-NULL only on blocked rows; `matched` has cardinality high enough that normalizing buys nothing. Keep both halves of each id/name pair: the id links to the same object across a rename, the name survives a delete, and the detail view needs both.
**Provenance capture does not violate the pure core.** Verified against the code. The policy decision is made in `src/server/handler.zig`, which is already impure and already holds `Io`, the clock and the log call. `matcher.Decision` (`matcher.zig:39-51`) already returns `reason`, `matched` and `source`; the handler throws `matched` and `source` away at `handler.zig:501`. Capture is mostly widening `logger.zig`'s `Entry` and `LogFields`, not threading state through `dns/`, `filter/`, `local/` or `cache/`.
**Six milestones under the harness, one per step.** Do not fold step 6 into step 5: the closure sweep regenerates the contract samples and adds the cross-surface acceptance tests, and it deserves its own verify gate. If anything needs splitting it is step 1, which touches about ten subsystems — store, API and UI first, then the emitters. An event store with three emitters is already shippable and honest.
**One caution, not a blocker.** Diagnostics will be an empty page most of the year. The "No active operational issues" line has to make empty read as healthy, not broken.
+617 -26
View File
@@ -44,10 +44,12 @@ const doh_client = @import("upstream/doh_client.zig");
const doh_server = @import("server/doh_server.zig");
const dot_client = @import("upstream/dot_client.zig");
const dot_server = @import("server/dot_server.zig");
const events = @import("storage/events.zig");
const faults = @import("config/faults.zig");
const fetcher = @import("filter/fetcher.zig");
const forward_zones = @import("local/forward_zones.zig");
const handler = @import("server/handler.zig");
const history_mod = @import("upstream/history.zig");
const http_util = @import("web/http_util.zig");
const loader = @import("config/loader.zig");
const local_records = @import("local/records.zig");
@@ -59,7 +61,9 @@ const migrations = @import("storage/migrations.zig");
const model = @import("config/model.zig");
const pause = @import("server/pause.zig");
const pool_mod = @import("upstream/pool.zig");
const queries_repo = @import("storage/repositories/queries_repo.zig");
const query_sink = @import("server/query_sink.zig");
const querylog_schema = @import("storage/querylog_schema.zig");
const rate_limiter = @import("server/rate_limiter.zig");
const reconcile = @import("config/reconcile.zig");
const retention_mod = @import("storage/retention.zig");
@@ -68,6 +72,7 @@ const shutdown = @import("server/shutdown.zig");
const sse = @import("web/sse.zig");
const static = @import("web/static.zig");
const tcp_server = @import("server/tcp_server.zig");
const upstream_history_repo = @import("storage/repositories/upstream_history_repo.zig");
const transport = @import("upstream/transport.zig");
const udp_server = @import("server/udp_server.zig");
const validate = @import("config/validate.zig");
@@ -152,6 +157,7 @@ fn reconcileFromFile(
config_db: *db.Db,
dir: std.Io.Dir,
config_path: []const u8,
config_load: ?*ConfigLoad,
) !i64 {
return reconcileFromFileAt(
r,
@@ -159,6 +165,7 @@ fn reconcileFromFile(
dir,
config_path,
std.Io.Clock.real.now(r.io).toSeconds(),
config_load,
);
}
@@ -182,6 +189,7 @@ fn reconcileFromFileAt(
dir: std.Io.Dir,
config_path: []const u8,
pass_now: i64,
config_load: ?*ConfigLoad,
) !i64 {
var arena_state: std.heap.ArenaAllocator = .init(r.gpa);
defer arena_state.deinit();
@@ -201,9 +209,32 @@ fn reconcileFromFileAt(
diags.writeAll(r.err) catch {};
r.err.flush() catch {};
// Warnings only. A `.fail` rejects the file and the process exits, so there
// is nobody left to read a diagnostics row about it; a warning is the case
// where the box serves on with a setting the operator did not mean.
if (config_load) |collector| {
for (diags.problems.items) |problem| {
if (problem.severity != .warn) continue;
collector.note(problem.path, problem.path, problem.message);
}
}
return result;
}
/// An upstream's identity is its url: the whole url is the key, and the
/// redaction is the label, because a url can carry an account token.
fn noteUpstream(config_load: *ConfigLoad, url: []const u8, message: []const u8) void {
var label_buf: [events.Store.max_subject_label_len]u8 = undefined;
const label = std.fmt.bufPrint(&label_buf, "{f}", .{safe_url.redact(url)}) catch &label_buf;
var detail_buf: [events.Store.max_detail_len]u8 = undefined;
const detail = std.fmt.bufPrint(&detail_buf, "upstream {f} {s}", .{
safe_url.redactQuoted(url),
message,
}) catch &detail_buf;
config_load.note(url, label, detail);
}
/// Returns the moment the transaction committed, which is what the settings
/// envelope reports as `reconciled_at`.
fn applyManagedFile(
@@ -316,6 +347,55 @@ fn printSummary(
try r.out.flush();
}
/// The `configuration.load` findings of one boot.
///
/// This code is boot-finalized: nothing during the run can fix a setting, so a
/// restart is its recovery. Every finding is reported as it is made and its key
/// kept, and one `resolveExcept` after the last of them closes the episodes of
/// settings that were wrong last boot and are not wrong now.
const ConfigLoad = struct {
store: ?*events.Store,
io: std.Io,
now_s: i64,
keys: [events.Store.max_kept_keys][events.Store.max_subject_key_len]u8 = undefined,
lens: [events.Store.max_kept_keys]u16 = @splat(0),
len: usize = 0,
/// Set when a boot produced more distinct findings than `resolveExcept`
/// carries. The bulk resolve is then refused rather than truncated: a stale
/// episode left open is honest, and one that is still true closed is not.
/// The refusal goes through the store, so it is counted and latched.
overflowed: bool = false,
fn note(self: *ConfigLoad, key: []const u8, label: []const u8, detail: []const u8) void {
const store = self.store orelse return;
store.report(self.io, self.now_s, .configuration_load, key, label, .warning, detail);
self.keep(key);
}
fn keep(self: *ConfigLoad, key: []const u8) void {
var canon_buf: [events.Store.max_subject_key_len]u8 = undefined;
const canon = events.canonicalKey(key, &canon_buf);
for (0..self.len) |i| {
if (std.mem.eql(u8, self.keys[i][0..self.lens[i]], canon)) return;
}
if (self.len == self.keys.len) {
self.overflowed = true;
return;
}
@memcpy(self.keys[self.len][0..canon.len], canon);
self.lens[self.len] = @intCast(canon.len);
self.len += 1;
}
fn finalize(self: *ConfigLoad) void {
const store = self.store orelse return;
if (self.overflowed) return store.refuseResolveExcept(self.io, self.now_s);
var kept: [events.Store.max_kept_keys][]const u8 = undefined;
for (0..self.len) |i| kept[i] = self.keys[i][0..self.lens[i]];
store.resolveExcept(self.io, self.now_s, .configuration_load, kept[0..self.len]);
}
};
fn serve(r: cli.Runner, args: cli.RunArgs) !u8 {
const io = r.io;
const gpa = r.gpa;
@@ -332,13 +412,29 @@ fn serve(r: cli.Runner, args: cli.RunArgs) !u8 {
defer config_db.close();
_ = try migrations.migrate(&config_db);
// Opened unconditionally, not gated on `cfg.web.enabled`: diagnostics record
// what went wrong whether or not anyone is running the UI, and the store
// owns this connection outright (`storage/events.zig`).
var events_db = try data.openConfigDb(io);
defer events_db.close();
const boot_now_s = std.Io.Clock.real.now(io).toSeconds();
var event_store_storage: ?events.Store = events.Store.init(io, &events_db, boot_now_s) catch |err| blk: {
// No store rather than a store on a mirror it could not verify: the
// latter answers `resolve` with confident no-ops. `/api/health` reports
// the absence as `unavailable` and degrades on it.
log.warn("diagnostics store unavailable: {s}", .{@errorName(err)});
break :blk null;
};
const event_store: ?*events.Store = if (event_store_storage) |*s| s else null;
var config_load: ConfigLoad = .{ .store = event_store, .io = io, .now_s = boot_now_s };
// Ruling 1: the presence of `--config` is the whole authority decision. With
// it, the file is the sole declarative source and the database is converged
// onto it here, before anything reads the database. Without it the database
// is authority and this step does not exist — a file on disk that no flag
// names changes nothing.
const reconciled_at: ?i64 = if (args.config) |config_path|
try reconcileFromFile(r, &config_db, std.Io.Dir.cwd(), config_path)
try reconcileFromFile(r, &config_db, std.Io.Dir.cwd(), config_path, &config_load)
else
null;
@@ -435,7 +531,7 @@ fn serve(r: cli.Runner, args: cli.RunArgs) !u8 {
defer bundle.deinit(gpa);
var bundle_lock: std.Io.RwLock = .init;
var upstreams = try Upstreams.build(gpa, cfg.upstreams, &dns_http, &bundle, &bundle_lock);
var upstreams = try Upstreams.build(gpa, cfg.upstreams, &dns_http, &bundle, &bundle_lock, &config_load);
defer upstreams.deinit(gpa);
var pool: pool_mod.Pool = .init(
@@ -448,6 +544,15 @@ fn serve(r: cli.Runner, args: cli.RunArgs) !u8 {
@truncate(@as(u96, @bitCast(std.Io.Clock.real.now(io).nanoseconds))),
);
// On the heap, not in this frame: the accumulator carries its pending cells
// and the flush task's buffer inline, which is about a megabyte.
const history = try gpa.create(history_mod.Accumulator);
defer gpa.destroy(history);
history.* = .init;
history.diagnostics = event_store;
pool.history = history;
pool.diagnostics = event_store;
// -----------------------------------------------------------------------
// per-query state
// -----------------------------------------------------------------------
@@ -463,15 +568,18 @@ fn serve(r: cli.Runner, args: cli.RunArgs) !u8 {
var paused: pause.Pause = .{};
var tracker: clients.Tracker = .init(cfg.logging.retention_days);
tracker.diagnostics = event_store;
// Naming rides the tracker's pass, on the tracker's task and connection
// (milestone-25 ruling 1), and reads the live forward zones.
var client_names_resolver: client_names.Resolver = .init(&tables);
client_names_resolver.diagnostics = event_store;
// The queue holds waiting tasks in intrusive lists, so neither the buffer
// nor the `Logger` may move once a task has touched either.
const queue_buf = try gpa.alloc(logger_mod.Entry, cfg.logging.query_log_buffer_max);
defer gpa.free(queue_buf);
var query_logger: logger_mod.Logger = .init(cfg.logging, queue_buf);
query_logger.diagnostics = event_store;
// Milestone 8 fans every logged query out to the SSE hub as well. The hub
// exists only when the web interface does (ruling 6) — without it the sink
@@ -501,6 +609,7 @@ fn serve(r: cli.Runner, args: cli.RunArgs) !u8 {
// Ruling 17. The scheduler consults it before every scheduled pass; the
// startup `reload` below is an operator action and stays ungated.
manager.monitor = &monitor;
manager.diagnostics = event_store;
// One synchronous sample before anything can consult the gate. `Monitor`
// initializes to `.ok`, and `Monitor.run` takes its first sample inside the
@@ -516,14 +625,19 @@ fn serve(r: cli.Runner, args: cli.RunArgs) !u8 {
// fails therefore leaves the state at `.ok` — an unreadable filesystem is
// not evidence that the disk is full, which is the monitor's documented
// policy and the right one here too.
monitor.sample(io);
monitor.sample(io, event_store, boot_now_s);
var retention: retention_mod.Retention = .init(cfg.logging);
var querylog_writer_db = try data.openQuerylogDb(io);
var querylog_opened = try data.openQuerylogDb(io);
var querylog_writer_db = querylog_opened.database;
defer querylog_writer_db.close();
reportQuerylogRecreated(event_store, io, boot_now_s, &querylog_opened, &querylog_writer_db);
var querylog_retention_db = try data.reopenQuerylogDb(io);
defer querylog_retention_db.close();
var querylog_history_db = try data.reopenQuerylogDb(io);
defer querylog_history_db.close();
var tracker_db = try data.openConfigDb(io);
defer tracker_db.close();
@@ -563,6 +677,17 @@ fn serve(r: cli.Runner, args: cli.RunArgs) !u8 {
"loading the blocklist snapshot failed ({s}); serving unfiltered until the next refresh",
.{@errorName(err)},
);
// No manager lock is held here, so this reports directly rather than
// through the manager's collector.
if (event_store) |store| {
var buf: [events.Store.max_detail_len]u8 = undefined;
const detail = std.fmt.bufPrint(
&buf,
"loading the blocklist snapshot failed: {s}",
.{@errorName(err)},
) catch buf[0..];
store.report(io, boot_now_s, .blocklist_snapshot, "snapshot", "blocklist snapshot", .@"error", detail);
}
};
// -----------------------------------------------------------------------
@@ -598,12 +723,16 @@ fn serve(r: cli.Runner, args: cli.RunArgs) !u8 {
defer if (doh_certs) |*store| store.deinit(io);
if (cfg.doh_server.enabled) {
doh_certs = try openCertStore(r, gpa, io, cfg.doh_server, "doh_server", doh_server.alpn_protocols);
doh_certs.?.kind = .doh;
doh_certs.?.diagnostics = event_store;
}
var dot_certs: ?cert_store.CertStore = null;
defer if (dot_certs) |*store| store.deinit(io);
if (cfg.dot_server.enabled) {
dot_certs = try openCertStore(r, gpa, io, cfg.dot_server, "dot_server", dot_alpn);
dot_certs.?.kind = .dot;
dot_certs.?.diagnostics = event_store;
}
// Bound here, in this frame, rather than through doh_server's module-level
@@ -611,13 +740,35 @@ fn serve(r: cli.Runner, args: cli.RunArgs) !u8 {
// only a listener that lives in this frame has an address to wire there.
// A failed bind warns and stays off (ruling 1, the web precedent): TLS DNS
// failing to come up must not stop the plain-DNS side this box exists for.
var failed_listeners: [2][]const u8 = undefined;
var failed_listener_count: usize = 0;
var doh: ?doh_server.DohServer = null;
defer if (doh) |*server| server.deinit(io);
if (doh_certs) |*store| doh = bindDoh(gpa, io, cfg.doh_server, &h, store);
if (doh_certs) |*store| {
doh = bindDoh(gpa, io, cfg.doh_server, &h, store, event_store, boot_now_s);
if (doh == null) {
failed_listeners[failed_listener_count] = "doh";
failed_listener_count += 1;
}
}
var dot: ?dot_server.DotServer = null;
defer if (dot) |*server| server.deinit(io);
if (dot_certs) |*store| dot = bindDot(gpa, io, cfg.dot_server, &h, store);
if (dot_certs) |*store| {
dot = bindDot(gpa, io, cfg.dot_server, &h, store, event_store, boot_now_s);
if (dot == null) {
failed_listeners[failed_listener_count] = "dot";
failed_listener_count += 1;
}
}
// Boot-finalized: one call closes whatever the last boot left open for an
// endpoint that started clean this time — including an endpoint now
// disabled, which contributes no key and so is not kept.
if (event_store) |store| {
store.resolveExcept(io, boot_now_s, .listener_start, failed_listeners[0..failed_listener_count]);
}
// -----------------------------------------------------------------------
// web interface (ruling 26)
@@ -646,6 +797,7 @@ fn serve(r: cli.Runner, args: cli.RunArgs) !u8 {
.local_tables = &tables,
.logger = &query_logger,
.retention = &retention,
.history = history,
.sessions = if (sessions) |*s| s else null,
.limiter = if (web_limiter) |*l| l else null,
.hub = hub,
@@ -656,6 +808,7 @@ fn serve(r: cli.Runner, args: cli.RunArgs) !u8 {
.dot_listener = if (dot) |*server| server else null,
.config_db = if (web_config_db) |*database| database else null,
.querylog_db = if (web_querylog_db) |*database| database else null,
.events = event_store,
.version = version.string,
.started_unix = std.Io.Clock.real.now(io).toSeconds(),
// Ruling 24: `--admin-dev` serves from disk with no cache headers;
@@ -672,6 +825,7 @@ fn serve(r: cli.Runner, args: cli.RunArgs) !u8 {
var udp6: ?udp_server.UdpServer = udp_server.UdpServer.bind(gpa, io, v6_bind, &h, .{}) catch |err| bound: {
if (!ipv6Unavailable(err)) return reportBind(r, "udp", v6_bind, err);
log.warn("this system has no IPv6; serving IPv4 only", .{});
config_load.note("dns.bind_ipv6", "dns.bind_ipv6", "this system has no IPv6; serving IPv4 only");
break :bound null;
};
defer if (udp6) |*s| s.deinit(gpa, io);
@@ -700,6 +854,12 @@ fn serve(r: cli.Runner, args: cli.RunArgs) !u8 {
};
defer if (tcp4) |*s| s.deinit(io);
// Every `configuration.load` finding of this boot is in by here: the managed
// file, the upstream table and the IPv6 bind above. Finalizing any earlier
// would resolve an episode this boot is about to reopen, so consecutive
// IPv6-less boots would read as a new episode each time.
config_load.finalize();
// Ruling 13: `/metrics` sums each transport's listeners into one family, so
// the web state carries pointers to whichever of the four came up. The
// arrays are declared here rather than beside `web_state` because a
@@ -735,12 +895,49 @@ fn serve(r: cli.Runner, args: cli.RunArgs) !u8 {
shutdown.install(io);
// Declared after everything it borrows, so its `cancel` — which both
// requests cancellation and joins — is the first thing that runs on the way
// out (ruling 22). Nothing below this line may be released while a task
// could still touch it.
// Declared after everything it borrows, so the teardown below — whose
// `cancel` both requests cancellation and joins — is the first thing that
// runs on the way out (ruling 22). Nothing below this line may be released
// while a task could still touch it.
var group: std.Io.Group = .init;
defer group.cancel(io);
// The gate every non-essential write consults. Reading it before the
// monitor's own task has sampled is safe: a fresh `Monitor` publishes `.ok`
// (disk_monitor.zig:63), so nothing is refused for want of a sample.
const gate: ?*disk_monitor.Monitor = &monitor;
// The query-log writer is deliberately *not* in `group`, and starts before
// every producer. Inside the group its life would end with the same
// `cancel` that stops the producers, and cancellation would race the
// queue's close: whichever landed first decided whether the batch the
// writer was holding reached the database or was counted as dropped. Given
// its own future, it outlives the producers by construction, and the
// teardown below can close the queue with nobody left to fill it and then
// wait for the writer to finish emptying it.
var writer_future = try io.concurrent(
logger_mod.Logger.runWriter,
.{ &query_logger, io, &querylog_writer_db, gate },
);
// Ruling 4's shutdown order, on the one path every exit from here takes:
// every producer stops and is joined, then the queue closes, then the
// writer is awaited — so the last batch is written rather than raced — and
// only then does the final history flush run, with no recording task left
// that could add a cell after it. A writer the disk gate will not let write
// counts its batch as dropped instead of holding the exit open
// (`logger.zig`), so this wait always ends.
//
// A `defer` and not straight-line code after `shutdown.wait`, because a
// `concurrent` spawn below can fail with the DNS listeners already
// serving; an orderly error teardown owes the operator the same drain a
// signal gets. The `querylog_history_db` this flush writes through is
// declared above, so its `close` runs after it.
defer {
group.cancel(io);
query_logger.shutdown(io);
writer_future.await(io) catch {};
history.flushOnce(io, &querylog_history_db, upstream_history_repo.flush);
}
if (udp6) |*s| try group.concurrent(io, udp_server.UdpServer.serve, .{ s, io });
if (udp4) |*s| try group.concurrent(io, udp_server.UdpServer.serve, .{ s, io });
@@ -751,10 +948,11 @@ fn serve(r: cli.Runner, args: cli.RunArgs) !u8 {
if (doh_certs) |*store| try group.concurrent(io, cert_store.CertStore.watch, .{ store, io });
if (dot_certs) |*store| try group.concurrent(io, cert_store.CertStore.watch, .{ store, io });
const gate: ?*disk_monitor.Monitor = &monitor;
try group.concurrent(io, logger_mod.Logger.runWriter, .{ &query_logger, io, &querylog_writer_db, gate });
try group.concurrent(io, retention_mod.Retention.run, .{ &retention, io, &querylog_retention_db, gate });
try group.concurrent(io, disk_monitor.Monitor.run, .{ &monitor, io });
try group.concurrent(io, retention_mod.Retention.run, .{ &retention, io, &querylog_retention_db, gate, event_store });
// Ungated: a flush writes at most one row per upstream per minute, the same
// category as the query logger's own writes, which are ungated too.
try group.concurrent(io, history_mod.Accumulator.run, .{ history, io, &querylog_history_db });
try group.concurrent(io, disk_monitor.Monitor.run, .{ &monitor, io, event_store });
try group.concurrent(io, manager_mod.Manager.runScheduler, .{ &manager, io });
try group.concurrent(io, clients.Tracker.run, .{ &tracker, io, &tracker_db, gate, &client_names_resolver });
try group.concurrent(io, runMaintenance, .{ &h, if (web_limiter) |*l| l else null, io });
@@ -774,14 +972,11 @@ fn serve(r: cli.Runner, args: cli.RunArgs) !u8 {
});
// A canceled wait is a shutdown request too: whoever canceled this task
// wants the process to stop, and the teardown below is how it stops.
// wants the process to stop, and returning into the teardown deferred above
// is how it stops.
shutdown.wait(io) catch {};
log.info("shutting down", .{});
// Before the group is canceled, so the writer sees a closed queue and
// drains what it holds rather than losing it to cancellation (ruling 22).
query_logger.shutdown(io);
return cli.exit_ok;
}
@@ -842,6 +1037,23 @@ fn openCertStore(
};
}
/// An error, not a warning: an endpoint the operator enabled is not serving.
fn reportListener(
store: ?*events.Store,
io: std.Io,
now_s: i64,
kind: []const u8,
comptime fmt: []const u8,
args: anytype,
) void {
const s = store orelse return;
var buf: [events.Store.max_detail_len]u8 = undefined;
var w: std.Io.Writer = .fixed(&buf);
w.print("{s} listener ", .{kind}) catch {};
w.print(fmt, args) catch {};
s.report(io, now_s, .listener_start, kind, kind, .@"error", w.buffered());
}
/// Ruling 1: a listener that cannot bind warns and stays off. The bind text
/// itself gets the same treatment — `validate` refuses it, but a hand-edited
/// database can still carry one, and it is not worth taking DNS down over.
@@ -851,13 +1063,17 @@ fn bindDoh(
endpoint: model.TlsEndpoint,
h: *handler.Handler,
store: *cert_store.CertStore,
diagnostics: ?*events.Store,
now_s: i64,
) ?doh_server.DohServer {
const bind_address = net.IpAddress.parse(endpoint.bind, endpoint.port) catch {
log.warn("doh_server.bind '{s}' is not an IP address; DoH is disabled", .{endpoint.bind});
reportListener(diagnostics, io, now_s, "doh", "bind '{s}' is not an IP address", .{endpoint.bind});
return null;
};
const server = doh_server.DohServer.listen(gpa, io, bind_address, h, store, .{}) catch |err| {
log.warn("doh listener cannot listen on {s}:{d}: {t}", .{ endpoint.bind, endpoint.port, err });
reportListener(diagnostics, io, now_s, "doh", "cannot listen on {s}:{d}: {t}", .{ endpoint.bind, endpoint.port, err });
return null;
};
log.info("doh listener on {f}", .{server.boundAddress()});
@@ -870,13 +1086,17 @@ fn bindDot(
endpoint: model.TlsEndpoint,
h: *handler.Handler,
store: *cert_store.CertStore,
diagnostics: ?*events.Store,
now_s: i64,
) ?dot_server.DotServer {
const bind_address = net.IpAddress.parse(endpoint.bind, endpoint.port) catch {
log.warn("dot_server.bind '{s}' is not an IP address; DoT is disabled", .{endpoint.bind});
reportListener(diagnostics, io, now_s, "dot", "bind '{s}' is not an IP address", .{endpoint.bind});
return null;
};
const server = dot_server.DotServer.listen(gpa, io, bind_address, h, store, .{}) catch |err| {
log.warn("dot listener cannot listen on {s}:{d}: {t}", .{ endpoint.bind, endpoint.port, err });
reportListener(diagnostics, io, now_s, "dot", "cannot listen on {s}:{d}: {t}", .{ endpoint.bind, endpoint.port, err });
return null;
};
log.info("dot listener on {f}", .{server.boundAddress()});
@@ -966,6 +1186,7 @@ const Upstreams = struct {
http: *std.http.Client,
bundle: *Certificate.Bundle,
bundle_lock: *std.Io.RwLock,
config_load: *ConfigLoad,
) (Allocator.Error || error{NoUsableUpstreams})!Upstreams {
var enabled: usize = 0;
for (servers) |server| {
@@ -1001,6 +1222,7 @@ const Upstreams = struct {
"upstream {f} is not an https:// or tls:// endpoint; skipped",
.{safe_url.redactQuoted(server.url)},
);
noteUpstream(config_load, server.url, "not an https:// or tls:// endpoint; skipped");
continue;
};
@@ -1018,6 +1240,7 @@ const Upstreams = struct {
"upstream {f} is not a usable DoH url; skipped",
.{safe_url.redactQuoted(server.url)},
);
noteUpstream(config_load, server.url, "not a usable DoH url; skipped");
continue;
};
doh_count += 1;
@@ -1114,6 +1337,251 @@ fn parseBind(
return addr;
}
/// Files the one-shot `query_log.recreated` event for a boot that replaced the
/// query log.
///
/// One-shot and already over: the file was recreated during this boot, and
/// there is nothing to recover from. Never emitted for `.missing` — a first
/// creation renames nothing aside, so the event would carry an aside path that
/// does not exist and would greet every fresh install with a warning.
///
/// `database` is the connection to the file that was just created; the coverage
/// start is read from it rather than recomputed, so the event states the value
/// the API will.
fn reportQuerylogRecreated(
store: ?*events.Store,
io: std.Io,
now_s: i64,
opened: *const querylog_schema.OpenResult,
database: *db.Db,
) void {
const cause = opened.recreated orelse return;
if (cause == .missing) return;
const s = store orelse return;
// The coverage start belongs in this detail: the recreate is exactly the
// moment the history the operator had stops existing, and the watermark is
// the answer to "from when can I still ask?".
const coverage_start: ?i64 = queries_repo.availableSince(database) catch null;
var detail_buf: [events.Store.max_detail_len]u8 = undefined;
const detail = recreatedDetail(&detail_buf, opened.aside(), coverage_start);
s.reportResolved(io, now_s, .query_log_recreated, @tagName(cause), @tagName(cause), .warning, detail);
}
/// The `query_log.recreated` detail line: what was kept, and from when the new
/// file can answer.
///
/// The coverage start is the operator's actual remedy information — the event
/// says "this history is gone" and this says "and here is where the new history
/// begins". Null only when the fresh file would not answer, which is already a
/// separate failure; the line still names the aside rather than saying nothing.
///
/// The aside is a full path under the data directory, which can be longer than
/// the whole detail column, so the two facts compete for the buffer. The
/// watermark always wins and the name degrades in whole steps: full path, then
/// basename — which the event's own database directory disambiguates — then no
/// name at all. Never a path cut mid-string, which names no file on disk and
/// reads as if it did.
fn recreatedDetail(
buf: *[events.Store.max_detail_len]u8,
aside: []const u8,
coverage_start: ?i64,
) []const u8 {
const names = [_][]const u8{ aside, std.fs.path.basename(aside) };
const since = coverage_start orelse {
for (names) |name| {
return std.fmt.bufPrint(buf, "previous file kept as '{s}'", .{name}) catch continue;
}
return "previous file kept aside";
};
for (names) |name| {
return std.fmt.bufPrint(
buf,
"previous file kept as '{s}'; query history is available from {d}",
.{ name, since },
) catch continue;
}
// The buffer is `max_detail_len`, which no i64 can overrun on its own.
return std.fmt.bufPrint(buf, "query history is available from {d}", .{since}) catch unreachable;
}
const events_fixture = @import("storage/events_fixture.zig");
const testing = std.testing;
test "the recreated detail names the aside and the new coverage start" {
var buf: [events.Store.max_detail_len]u8 = undefined;
try std.testing.expectEqualStrings(
"previous file kept as 'querylog.db.schema-changed-1700000000'; " ++
"query history is available from 1700000001",
recreatedDetail(&buf, "querylog.db.schema-changed-1700000000", 1700000001),
);
// A fresh file that will not answer is a separate failure; the line still
// says what was kept rather than reporting nothing.
try std.testing.expectEqualStrings(
"previous file kept as 'querylog.db.corrupt-1700000000'",
recreatedDetail(&buf, "querylog.db.corrupt-1700000000", null),
);
// A data directory deep enough that its path alone would fill the column:
// the watermark is complete and the name degrades to the basename, which
// still names a real file.
const deep = "/srv/" ++ ("d" ** 60 ++ "/") ** 8 ++ "querylog.db.corrupt-1700000000";
try std.testing.expectEqualStrings(
"previous file kept as 'querylog.db.corrupt-1700000000'; " ++
"query history is available from 1700000001",
recreatedDetail(&buf, deep, 1700000001),
);
try std.testing.expectEqualStrings(
"previous file kept as 'querylog.db.corrupt-1700000000'",
recreatedDetail(&buf, deep, null),
);
// No filesystem produces a name this long, but a truncated one would name
// nothing: the watermark survives alone rather than half-named.
const unnameable = "/srv/" ++ "n" ** 500;
try std.testing.expectEqualStrings(
"query history is available from 1700000001",
recreatedDetail(&buf, unnameable, 1700000001),
);
try std.testing.expectEqualStrings(
"previous file kept aside",
recreatedDetail(&buf, unnameable, null),
);
}
test "a fingerprint recreate files a resolved event naming the real aside and watermark" {
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
defer threaded.deinit();
const io = threaded.io();
var tmp = testing.tmpDir(.{ .iterate = true });
defer tmp.cleanup();
var path_buf: [256]u8 = undefined;
const path = try std.fmt.bufPrintZ(&path_buf, ".zig-cache/tmp/{s}/querylog.db", .{tmp.sub_path});
var fx: events_fixture.Fixture = .{};
try fx.init(io, 1000);
defer fx.deinit();
// A fresh install: the file was missing, nothing was set aside, and the
// event would name a path that does not exist.
var created = try querylog_schema.open(io, std.Io.Dir.cwd(), path);
reportQuerylogRecreated(&fx.store, io, 1000, &created, &created.database);
created.database.close();
try testing.expectEqual(@as(i64, 0), try fx.count("SELECT count(*) FROM operational_events"));
// A healthy file this build's DDL no longer matches, which is what an
// upgrade that edits the schema produces.
{
var stamped = try db.Db.open(path, .{ .mode = .read_write_existing });
defer stamped.close();
var sql_buf: [64]u8 = undefined;
try stamped.exec(try std.fmt.bufPrintZ(
&sql_buf,
"PRAGMA user_version = {d};",
.{querylog_schema.fingerprint +% 1},
));
}
var recreated = try querylog_schema.open(io, std.Io.Dir.cwd(), path);
defer recreated.database.close();
try testing.expectEqual(querylog_schema.RecreateReason.fingerprint_mismatch, recreated.recreated.?);
reportQuerylogRecreated(&fx.store, io, 2000, &recreated, &recreated.database);
try testing.expectEqualStrings("query_log.recreated", try fx.text("SELECT code FROM operational_events"));
try testing.expectEqualStrings("fingerprint_mismatch", try fx.text("SELECT subject_key FROM operational_events"));
try testing.expectEqualStrings("warning", try fx.text("SELECT severity FROM operational_events"));
// One-shot: already over when it is filed, so it never becomes an open
// episode `/api/health` counts.
try testing.expectEqual(
@as(i64, 0),
try fx.count("SELECT count(*) FROM operational_events WHERE resolved_at IS NULL"),
);
// The detail carries the path that is actually on disk and the watermark
// the API will serve, both read back from the recreate rather than from
// the arguments the event was built with.
try tmp.dir.access(io, std.fs.path.basename(recreated.aside()), .{});
const coverage = try queries_repo.availableSince(&recreated.database);
var expected_buf: [events.Store.max_detail_len]u8 = undefined;
const expected = try std.fmt.bufPrint(
&expected_buf,
"previous file kept as '{s}'; query history is available from {d}",
.{ recreated.aside(), coverage },
);
try testing.expectEqualStrings(expected, try fx.text("SELECT detail FROM operational_events"));
// A boot with no diagnostics store configured is not a failure path.
reportQuerylogRecreated(null, io, 2000, &recreated, &recreated.database);
try testing.expectEqual(@as(i64, 1), try fx.count("SELECT count(*) FROM operational_events"));
}
test "a recreate under a long data directory keeps the watermark and a usable name" {
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
defer threaded.deinit();
const io = threaded.io();
var tmp = testing.tmpDir(.{ .iterate = true });
defer tmp.cleanup();
// Deep enough that the aside outgrows the detail column, shallow enough
// that SQLite's unix VFS still opens the file: it caps a path at 512 bytes.
const nested = ("d" ** 60 ++ "/") ** 6 ++ "d" ** 60;
try tmp.dir.createDirPath(io, nested);
var path_buf: [1024]u8 = undefined;
const path = try std.fmt.bufPrintZ(
&path_buf,
".zig-cache/tmp/{s}/{s}/querylog.db",
.{ tmp.sub_path, nested },
);
var fx: events_fixture.Fixture = .{};
try fx.init(io, 1000);
defer fx.deinit();
{
var created = try querylog_schema.open(io, std.Io.Dir.cwd(), path);
defer created.database.close();
var sql_buf: [64]u8 = undefined;
try created.database.exec(try std.fmt.bufPrintZ(
&sql_buf,
"PRAGMA user_version = {d};",
.{querylog_schema.fingerprint +% 1},
));
}
var recreated = try querylog_schema.open(io, std.Io.Dir.cwd(), path);
defer recreated.database.close();
try testing.expectEqual(querylog_schema.RecreateReason.fingerprint_mismatch, recreated.recreated.?);
const line_overhead = "previous file kept as ''; query history is available from ".len;
try testing.expect(recreated.aside().len + line_overhead > events.Store.max_detail_len);
reportQuerylogRecreated(&fx.store, io, 2000, &recreated, &recreated.database);
const detail = try fx.text("SELECT detail FROM operational_events");
const coverage = try queries_repo.availableSince(&recreated.database);
const name = std.fs.path.basename(recreated.aside());
var expected_buf: [events.Store.max_detail_len]u8 = undefined;
const expected = try std.fmt.bufPrint(
&expected_buf,
"previous file kept as '{s}'; query history is available from {d}",
.{ name, coverage },
);
// The watermark is whole — the fact that would be lost to a mid-string cut
// — and the name it kept is the file's, not a prefix of its path.
try testing.expectEqualStrings(expected, detail);
try testing.expect(detail.len <= events.Store.max_detail_len);
var deep = try tmp.dir.openDir(io, nested, .{});
defer deep.close(io);
try deep.access(io, name, .{});
}
test "parseBind refuses a bind address of the wrong family" {
var out_buf: [8]u8 = undefined;
var err_buf: [256]u8 = undefined;
@@ -1285,7 +1753,7 @@ test "a start in file mode prints the warnings the file earned" {
// The file is valid, so the start succeeds and the database converges onto
// it. The returned stamp is what the settings envelope reports.
const reconciled_at = try reconcileFromFile(r, &database, tmp.dir, "config.zon");
const reconciled_at = try reconcileFromFile(r, &database, tmp.dir, "config.zon", null);
try std.testing.expect(reconciled_at > 0);
try std.testing.expectEqual(@as(i64, 1), try database.queryInt("SELECT count(*) FROM upstreams"));
@@ -1391,7 +1859,7 @@ test "reconciled_at is stamped after the commit, not from the clock the pass wro
const pass_now: i64 = 42;
const before = std.Io.Clock.real.now(io).toSeconds();
const reconciled_at = try reconcileFromFileAt(r, &database, tmp.dir, "config.zon", pass_now);
const reconciled_at = try reconcileFromFileAt(r, &database, tmp.dir, "config.zon", pass_now, null);
// The pinned clock reached the engine, so the two values really are separate
// inputs rather than the same read twice.
@@ -1434,7 +1902,7 @@ test "the startup summary reports what the reconcile changed, then that nothing
var err: Writer = .fixed(&err_buf);
const r: cli.Runner = .{ .io = io, .gpa = gpa, .out = &out_writer.interface, .err = &err };
_ = try reconcileFromFile(r, &database, tmp.dir, "config.zon");
_ = try reconcileFromFile(r, &database, tmp.dir, "config.zon", null);
{
// Read back through the file: `serve` does not return for as long as the
// service runs, so a summary still in the buffer is a summary nobody
@@ -1451,7 +1919,7 @@ test "the startup summary reports what the reconcile changed, then that nothing
}
try tmp.dir.writeFile(io, .{ .sub_path = "stdout.txt", .data = "" });
_ = try reconcileFromFile(r, &database, tmp.dir, "config.zon");
_ = try reconcileFromFile(r, &database, tmp.dir, "config.zon", null);
{
const printed = try tmp.dir.readFileAlloc(io, "stdout.txt", gpa, .limited(8192));
defer gpa.free(printed);
@@ -1508,7 +1976,7 @@ test "a broken error writer does not replace the reason a managed file was rejec
try std.testing.expectError(
error.MissingDefaultGroup,
reconcileFromFile(r, &database, tmp.dir, "config.zon"),
reconcileFromFile(r, &database, tmp.dir, "config.zon", null),
);
// Empty, so the writer did fail — without this the assertion above would
@@ -1554,7 +2022,7 @@ test "a broken error writer does not stop a start whose file applied" {
var err_writer = brokenErrWriter(io, err_file, &err_buf);
const r: cli.Runner = .{ .io = io, .gpa = gpa, .out = &out, .err = &err_writer.interface };
_ = try reconcileFromFile(r, &database, tmp.dir, "config.zon");
_ = try reconcileFromFile(r, &database, tmp.dir, "config.zon", null);
try std.testing.expectEqual(@as(i64, 1), try database.queryInt("SELECT count(*) FROM upstreams"));
// Nothing reached the file, so the flush really did fail.
@@ -1739,3 +2207,126 @@ test "one maintenance pass drops the api limiter's stale buckets" {
// A limiter the app did not build is not a reason for the pass to fail.
try maintenanceOnce(&h, null, io);
}
test "a failing listener bind opens an error episode a clean boot closes" {
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
defer threaded.deinit();
const io = threaded.io();
var fx: events_fixture.Fixture = .{};
try fx.init(io, 1000);
defer fx.deinit();
reportListener(&fx.store, io, 1000, "dot", "cannot listen on {s}:{d}: {t}", .{
"0.0.0.0",
@as(u16, 853),
error.AddressInUse,
});
try testing.expectEqualStrings("listener.start", try fx.text("SELECT code FROM operational_events"));
try testing.expectEqualStrings("dot", try fx.text("SELECT subject_key FROM operational_events"));
try testing.expectEqualStrings("error", try fx.text("SELECT severity FROM operational_events"));
// The next boot: DoH failed, DoT started clean. One `resolveExcept` over
// the failed keys closes the stale DoT episode and leaves the DoH one open.
reportListener(&fx.store, io, 1100, "doh", "cannot listen on {s}:{d}: {t}", .{
"0.0.0.0",
@as(u16, 443),
error.AddressInUse,
});
fx.store.resolveExcept(io, 1100, .listener_start, &.{"doh"});
try testing.expectEqual(
@as(i64, 1),
try fx.count("SELECT count(*) FROM operational_events WHERE resolved_at IS NULL"),
);
try testing.expectEqualStrings("doh", try fx.text(
"SELECT subject_key FROM operational_events WHERE resolved_at IS NULL",
));
}
test "a boot with no listener finding closes every listener episode" {
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
defer threaded.deinit();
const io = threaded.io();
var fx: events_fixture.Fixture = .{};
try fx.init(io, 1000);
defer fx.deinit();
fx.store.report(io, 900, .listener_start, "doh", "doh", .@"error", "stale");
fx.store.report(io, 900, .listener_start, "dot", "dot", .@"error", "stale");
fx.store.resolveExcept(io, 1000, .listener_start, &.{});
try testing.expectEqual(
@as(i64, 0),
try fx.count("SELECT count(*) FROM operational_events WHERE resolved_at IS NULL"),
);
}
test "a configuration finding is reported once and finalize closes the rest" {
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
defer threaded.deinit();
const io = threaded.io();
var fx: events_fixture.Fixture = .{};
try fx.init(io, 1000);
defer fx.deinit();
// Two stale episodes from a previous boot.
fx.store.report(io, 900, .configuration_load, "dns.bind_ipv6", "dns.bind_ipv6", .warning, "stale");
fx.store.report(io, 900, .configuration_load, "upstreams[0].url", "upstreams[0].url", .warning, "stale");
var collector: ConfigLoad = .{ .store = &fx.store, .io = io, .now_s = 1000 };
collector.note("dns.bind_ipv6", "dns.bind_ipv6", "this system has no IPv6; serving IPv4 only");
// The same finding twice is one episode and one kept key.
collector.note("dns.bind_ipv6", "dns.bind_ipv6", "this system has no IPv6; serving IPv4 only");
try testing.expectEqual(@as(usize, 1), collector.len);
collector.finalize();
try testing.expectEqual(
@as(i64, 1),
try fx.count("SELECT count(*) FROM operational_events WHERE resolved_at IS NULL"),
);
try testing.expectEqualStrings("dns.bind_ipv6", try fx.text(
"SELECT subject_key FROM operational_events WHERE resolved_at IS NULL",
));
// The stale episode is still the same one: this boot bumped it twice.
try testing.expectEqual(@as(i64, 3), try fx.count(
"SELECT occurrences FROM operational_events WHERE resolved_at IS NULL",
));
}
test "an over-long boot finding list refuses to finalize rather than truncate" {
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
defer threaded.deinit();
const io = threaded.io();
var fx: events_fixture.Fixture = .{};
try fx.init(io, 1000);
defer fx.deinit();
fx.store.report(io, 900, .configuration_load, "stale.setting", "stale.setting", .warning, "stale");
var collector: ConfigLoad = .{ .store = &fx.store, .io = io, .now_s = 1000 };
var key_buf: [32]u8 = undefined;
for (0..events.Store.max_kept_keys + 1) |i| {
const key = try std.fmt.bufPrint(&key_buf, "upstreams[{d}].url", .{i});
collector.note(key, key, "not a usable url");
}
try testing.expect(collector.overflowed);
collector.finalize();
// Closing more than the boot meant would resolve episodes that are still
// true, so the stale one stays open instead.
try testing.expectEqual(@as(i64, 1), try fx.count(
"SELECT count(*) FROM operational_events WHERE subject_key = 'stale.setting' AND resolved_at IS NULL",
));
// Refused, and refused out loud: the store counted and latched it, so
// `/api/health` reports the diagnostics log as not recording.
try testing.expect(fx.store.writeFailed());
try testing.expectEqual(@as(u64, 1), fx.store.writeFailures());
}
+16 -9
View File
@@ -332,22 +332,28 @@ pub const DataDir = struct {
/// `querylog_schema.open` resolves the path through SQLite's VFS as well as
/// through the directory handle, so it is given `cwd` and the joined path
/// rather than `self.dir` and a name (see its doc comment).
pub fn openQuerylogDb(self: *const DataDir, io: std.Io) !db.Db {
const opened = try querylog_schema.open(io, std.Io.Dir.cwd(), self.querylog_db_path);
var database = opened.database;
errdefer database.close();
/// Returns the whole `OpenResult`, recreate reason and aside name included:
/// the composition root records a recreate as a one-shot diagnostics event,
/// and the aside name is what tells an operator where the old file went.
pub fn openQuerylogDb(self: *const DataDir, io: std.Io) !querylog_schema.OpenResult {
var opened = try querylog_schema.open(io, std.Io.Dir.cwd(), self.querylog_db_path);
errdefer opened.database.close();
try self.restrictQuerylogPermissions(io);
return database;
return opened;
}
/// An additional connection to a `querylog.db` that `openQuerylogDb` has
/// already established. A running server needs two — the log writer and the
/// retention pass each own one (`retention.zig`'s contract).
/// already established. A running server needs three background ones — the
/// log writer, the retention pass and the upstream-history flush each own
/// one (`retention.zig`'s contract) — plus a fourth for the web task when
/// the web interface is enabled.
pub fn reopenQuerylogDb(self: *const DataDir, io: std.Io) !db.Db {
_ = io;
var database = try db.Db.open(self.querylog_db_path, .{ .mode = .read_write_existing });
errdefer database.close();
try db.applyPragmas(&database, .{});
try db.applyPragmas(&database, .{
.wal_autocheckpoint_pages = querylog_schema.wal_autocheckpoint_pages,
});
return database;
}
@@ -1001,7 +1007,8 @@ fn probeUpstreams(r: Runner, cfg: model.Config) !usize {
}};
var single: pool.Pool = .init(&entries, .{}, timeouts, seed);
if (single.exchange(r.io, probe_query, response_buf)) |_| {
var selected: ?[]const u8 = null;
if (single.exchange(r.io, probe_query, response_buf, &selected)) |_| {
try r.out.print("OK upstreams[{d}] {f}\n", .{ i, safe_url.redact(server.url) });
} else |_| {
// The concrete cause lives in the entry's health, which is where the
+50
View File
@@ -151,6 +151,7 @@ fn reportDeletes(diags: *validate.Diagnostics, summary: reconcile.Summary) error
const testing = std.testing;
const config_schema = @import("../storage/config_schema.zig");
const export_mod = @import("export.zig");
const migrations = @import("../storage/migrations.zig");
fn openMigrated() !db.Db {
@@ -282,6 +283,55 @@ test "importSource converges a migrated database and group 'default' keeps id 1"
);
}
test "an export and re-import leaves the operational_events log untouched" {
// `operational_events` is runtime state, not configuration: it is out of
// `table_names` and out of `delete_order`, so an export must not emit it and
// an import's wipe must not reach it. A diagnostics log destroyed by a
// routine `nxdns import` would take the record of what the box has been
// doing with it.
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
defer threaded.deinit();
const io = threaded.io();
const gpa = testing.allocator;
var database = try openMigrated();
defer database.close();
try importText(io, &database, full_source, .{});
try database.exec(
\\INSERT INTO operational_events
\\ (code, subject_key, subject_label, severity, first_seen, last_seen, occurrences, resolved_at)
\\VALUES ('disk.space', 'data', 'data', 'warning', 100, 200, 3, NULL),
\\ ('blocklist.refresh', 'https://a.example', 'A', 'warning', 100, 150, 1, 300);
);
var rendered: std.Io.Writer.Allocating = .init(gpa);
defer rendered.deinit();
try export_mod.writeToWriter(gpa, &database, &rendered.writer);
const source = try gpa.dupeZ(u8, rendered.written());
defer gpa.free(source);
// The export is the whole declared configuration and says nothing about
// the log.
try testing.expect(!std.mem.containsAtLeast(u8, source, 1, "operational_events"));
try testing.expect(!std.mem.containsAtLeast(u8, source, 1, "disk.space"));
// A round trip is by definition delete-free for the config tables, and the
// events survive it untouched, resolved and active alike.
try importText(io, &database, source, .{ .allow_delete = true });
var stmt = try database.prepare(
"SELECT code, occurrences, resolved_at FROM operational_events ORDER BY id",
);
defer stmt.deinit();
try testing.expect(try stmt.step());
try testing.expectEqualStrings("disk.space", stmt.columnText(0));
try testing.expectEqual(@as(i64, 3), stmt.columnInt(1));
try testing.expect(stmt.isNull(2));
try testing.expect(try stmt.step());
try testing.expectEqualStrings("blocklist.refresh", stmt.columnText(0));
try testing.expectEqual(@as(i64, 300), stmt.columnInt(2));
try testing.expect(!try stmt.step());
}
test "an import whose diff deletes rows is refused, names the tables, and changes nothing" {
// Ruling 6: the emptiness guard is gone, so this is what stops
// `nxdns import ./wrong.zon` from emptying a configured database.
+19
View File
@@ -0,0 +1,19 @@
//! Length caps on the configuration labels that the query log copies into every
//! row it writes.
//!
//! They live in a module of their own because two files need them and neither
//! may import the other: `config/validate.zig` rejects a name that exceeds a cap
//! and `storage/logger.zig` sizes an `Entry` buffer from it, while `validate`
//! already imports `logger` for the entry-size budget it derives. A cap owned by
//! either file would close that loop.
//!
//! The values are deliberately short. A group or a source name is a label an
//! operator reads in a table cell, and every byte of it is copied into every
//! logged row — the cap is what keeps a pasted paragraph out of an `Entry` that
//! travels by value through the queue.
/// `groups[].name`.
pub const max_group_name_len = 64;
/// `blocklist_sources[].name`.
pub const max_source_name_len = 64;
+6
View File
@@ -231,6 +231,10 @@ pub const Logging = struct {
level: LogLevel = .info,
retention_days: u16 = 30,
query_log_buffer_max: u32 = 10000,
/// How long the query-log writer gathers entries before it commits them.
/// `0` does not wait at all: it flushes the entry that woke the writer plus
/// whatever is already queued.
query_log_flush_interval_s: u16 = 60,
hide_domains: bool = false,
hide_client_ips: bool = false,
output: LogOutput = .stderr,
@@ -565,6 +569,7 @@ const expected_keys = [_][]const u8{
"logging.max_size_mb",
"logging.output",
"logging.query_log_buffer_max",
"logging.query_log_flush_interval_s",
"logging.retention_days",
"upstream.attempt_timeout_ms",
"upstream.read_timeout_ms",
@@ -657,6 +662,7 @@ test "toSettings and fromSettings round-trip a non-default config" {
.level = .err,
.retention_days = 41,
.query_log_buffer_max = 43,
.query_log_flush_interval_s = 44,
.hide_domains = true,
.hide_client_ips = true,
.output = .file,
+136 -5
View File
@@ -6,9 +6,13 @@
//! instead of a line number, so every `UNIQUE` and every foreign key in
//! PLAN §11.2 has a check here.
//!
//! Pure: no `std.Io` value is a parameter anywhere, no SQLite, no clock. The
//! only `std.Io` type used is `std.Io.Writer`, for rendering diagnostics. The
//! allocator exists for diagnostic text and scratch bookkeeping alone.
//! Pure: no `std.Io` value is a parameter anywhere, no SQLite call, no clock.
//! The only `std.Io` type used is `std.Io.Writer`, for rendering diagnostics.
//! The allocator exists for diagnostic text and scratch bookkeeping alone. The
//! `storage/logger.zig` import is a comptime one — `query_log_buffer_max` is
//! derived from `@sizeOf(logger.Entry)`, because the bound this file enforces
//! on the queue is a bound on bytes and only the logger knows how wide a queued
//! entry is. Nothing in this file calls into storage.
//!
//! Parsers are reused, never reimplemented: `transport.Endpoint.parse` for
//! upstream URLs, `NetAddress.parse` / `Prefix.parse` for addresses, and
@@ -44,6 +48,8 @@ const Writer = std.Io.Writer;
const model = @import("model.zig");
const address = @import("../platform/address.zig");
const dns_name = @import("../dns/name.zig");
const limits = @import("limits.zig");
const logger = @import("../storage/logger.zig");
const regex = @import("../filter/regex.zig");
const safe_url = @import("../safe_url.zig");
const transport = @import("../upstream/transport.zig");
@@ -72,6 +78,7 @@ pub const ValidateError = error{
DuplicateGroupName,
UnknownGroup,
EmptyGroupName,
GroupNameTooLong,
BadClientIp,
DuplicateClientIp,
BadClientPrefix,
@@ -79,6 +86,7 @@ pub const ValidateError = error{
BadSourceUrl,
DuplicateSourceUrl,
EmptySourceName,
SourceNameTooLong,
UnknownSource,
DuplicateGroupSource,
BadRulePattern,
@@ -93,6 +101,7 @@ pub const ValidateError = error{
BadTtl,
BadCacheSize,
BadRetention,
BadFlushInterval,
BadLogRotation,
BadDiskThresholds,
BadRateLimit,
@@ -335,6 +344,11 @@ const max_rate_window_seconds = 3_600;
/// the box, and nxdns does not try to know that.
const max_boot_entries = 1_000_000;
/// The ceiling on the query-log flush window. An hour of queries is already
/// more history than a crash is allowed to cost; beyond that the setting stops
/// being a batching knob and becomes a way to lose a working day of rows.
const max_flush_interval_s = 3_600;
fn checkScalars(cfg: Config, diags: *Diagnostics) error{OutOfMemory}!void {
const up = cfg.upstream;
try checkTimeout(diags, up.attempt_timeout_ms, "upstream.attempt_timeout_ms");
@@ -455,13 +469,26 @@ fn checkScalars(cfg: Config, diags: *Diagnostics) error{OutOfMemory}!void {
if (cfg.logging.query_log_buffer_max < 1) {
try diags.add(error.BadRetention, "logging.query_log_buffer_max", .{}, "must be at least 1", .{});
}
if (cfg.logging.query_log_buffer_max > max_boot_entries) {
// Its own ceiling, not `max_boot_entries`: a queued `logger.Entry` carries
// every provenance field by value, so the queue's cost is bytes rather than
// entries and the bound follows the width of the entry.
if (cfg.logging.query_log_buffer_max > logger.query_log_buffer_max) {
try diags.add(
error.BadRetention,
"logging.query_log_buffer_max",
.{},
"must be at most {d}, got {d}",
.{ max_boot_entries, cfg.logging.query_log_buffer_max },
.{ logger.query_log_buffer_max, cfg.logging.query_log_buffer_max },
);
}
// No floor: 0 is the documented "do not wait" setting, not a mistake.
if (cfg.logging.query_log_flush_interval_s > max_flush_interval_s) {
try diags.add(
error.BadFlushInterval,
"logging.query_log_flush_interval_s",
.{},
"must be at most {d}, got {d}",
.{ max_flush_interval_s, cfg.logging.query_log_flush_interval_s },
);
}
if (cfg.logging.max_size_mb < 1) {
@@ -698,6 +725,29 @@ fn checkDotHost(
};
}
/// The shared shape of the two label caps.
///
/// Both names are copied by value into every `query_log` row that mentions
/// them, so the cap is what keeps a pasted paragraph out of the fixed buffers
/// of `storage/logger.zig`. Bytes, not codepoints: the buffer counts bytes.
fn checkNameLength(
diags: *Diagnostics,
comptime fault: ValidateError,
comptime path: []const u8,
path_args: anytype,
value: []const u8,
cap: usize,
) error{OutOfMemory}!void {
if (value.len <= cap) return;
try diags.add(
fault,
path,
path_args,
"must be at most {d} bytes, got {d}; the name is copied into every logged query",
.{ cap, value.len },
);
}
fn checkCollections(cfg: Config, diags: *Diagnostics, scratch: Allocator) error{OutOfMemory}!void {
var group_names: IndexSet = .empty;
var has_default = false;
@@ -713,6 +763,16 @@ fn checkCollections(cfg: Config, diags: *Diagnostics, scratch: Allocator) error{
.{safe_url.quoteText(group.name)},
);
}
// Independent of the chain above: an over-long name is still a name,
// and a duplicate of one is still a duplicate.
try checkNameLength(
diags,
error.GroupNameTooLong,
"groups[{d}].name",
.{i},
group.name,
limits.max_group_name_len,
);
if (std.mem.eql(u8, group.name, "default")) has_default = true;
}
if (!has_default) {
@@ -843,6 +903,14 @@ fn checkCollections(cfg: Config, diags: *Diagnostics, scratch: Allocator) error{
.{},
);
}
try checkNameLength(
diags,
error.SourceNameTooLong,
"blocklist_sources[{d}].name",
.{i},
source.name,
limits.max_source_name_len,
);
}
var group_source_pairs: IndexSet = .empty;
@@ -1292,6 +1360,24 @@ test "an https:// upstream may name a host" {
try expectClean(cfg);
}
/// The longest host `transport.Endpoint.parse` accepts: four labels, 253 bytes.
const host_at_bound = ("a" ** 63 ++ ".") ** 3 ++ "a" ** 61;
test "an upstream host at the length bound validates cleanly" {
var cfg = baseConfig();
cfg.upstreams = &.{.{ .url = "https://" ++ host_at_bound ++ "/dns-query" }};
try expectClean(cfg);
}
test "error.BadUpstreamUrl on an upstream host one byte past the length bound" {
// The bound is the query log's `upstream` width and every other identity
// built from the endpoint, so an over-long host has to fail here rather
// than be shortened downstream.
var cfg = baseConfig();
cfg.upstreams = &.{.{ .url = "https://" ++ host_at_bound ++ "a/dns-query" }};
try expectProblem(cfg, error.BadUpstreamUrl, "upstreams[0].url");
}
test "an IPv6 literal tls:// upstream validates cleanly" {
// `Endpoint.parse` strips the brackets, so the host reaching the check is
// exactly what the client hands to the address parser.
@@ -1355,6 +1441,35 @@ test "error.EmptyGroupName" {
try expectProblem(cfg, error.EmptyGroupName, "groups[1].name");
}
test "error.GroupNameTooLong" {
const cap = limits.max_group_name_len;
// Exactly at the cap is accepted; one byte past it is not. The cap is what
// `storage/logger.zig` sizes its `Entry` buffer from, so a name that passes
// here is a name a logged row stores whole.
var at_cap = baseConfig();
at_cap.groups = &.{ .{ .name = "default" }, .{ .name = "g" ** cap } };
try expectClean(at_cap);
var over = baseConfig();
over.groups = &.{ .{ .name = "default" }, .{ .name = "g" ** (cap + 1) } };
try expectProblem(over, error.GroupNameTooLong, "groups[1].name");
}
test "error.SourceNameTooLong" {
const cap = limits.max_source_name_len;
const url = "https://lists.example/hosts.txt";
var at_cap = baseConfig();
at_cap.blocklist_sources = &.{.{ .url = url, .name = "s" ** cap }};
at_cap.group_sources = &.{.{ .group = "default", .source_url = url }};
try expectClean(at_cap);
var over = baseConfig();
over.blocklist_sources = &.{.{ .url = url, .name = "s" ** (cap + 1) }};
over.group_sources = &.{.{ .group = "default", .source_url = url }};
try expectProblem(over, error.SourceNameTooLong, "blocklist_sources[0].name");
}
test "error.BadClientIp" {
var cfg = baseConfig();
cfg.clients = &.{.{ .ip = "nonsense" }};
@@ -1946,6 +2061,22 @@ test "error.BadRetention" {
try expectProblem(huge_buffer, error.BadRetention, "logging.query_log_buffer_max");
}
test "error.BadFlushInterval" {
var cfg = baseConfig();
cfg.logging.query_log_flush_interval_s = 3601;
try expectProblem(cfg, error.BadFlushInterval, "logging.query_log_flush_interval_s");
// 0 is the "do not wait" setting and 3600 is the ceiling itself: both are
// legal, and a floor check would reject the first.
var immediate = baseConfig();
immediate.logging.query_log_flush_interval_s = 0;
try expectClean(immediate);
var edge = baseConfig();
edge.logging.query_log_flush_interval_s = max_flush_interval_s;
try expectClean(edge);
}
test "error.BadLogRotation" {
var cfg = baseConfig();
cfg.logging.max_files = 0;
+160 -1
View File
@@ -24,6 +24,7 @@ const net = std.Io.net;
const model = @import("../config/model.zig");
const reconcile = @import("../config/reconcile.zig");
const db = @import("../storage/db.zig");
const events_fixture = @import("../storage/events_fixture.zig");
const migrations = @import("../storage/migrations.zig");
const context = @import("../storage/repositories/context.zig");
const groups_repo = @import("../storage/repositories/groups_repo.zig");
@@ -2328,7 +2329,6 @@ test "23: a name moving from the list body to the wild body forces a republish"
try testing.expectEqual(@as(i64, 0), row.wildcard_count);
@memcpy(&first_checksum, row.checksum orelse return error.TestNoChecksum);
}
}
test "22: a list that changed only its skipped lines still updates both skip counters" {
@@ -2405,3 +2405,162 @@ test "22: a list that changed only its skipped lines still updates both skip cou
try testing.expectEqual(@as(u32, 1), restored.counts.skipped_regex);
try testing.expectEqual(@as(u32, 2), restored.counts.skipped_unsupported);
}
// ---------------------------------------------------------------------------
// diagnostics: the events the manager records (milestone 27)
// ---------------------------------------------------------------------------
test "27: a failing refresh opens a blocklist.refresh episode a good one closes" {
if (!build_options.integration) return error.SkipZigTest;
const gpa = testing.allocator;
const env = try Env.create(gpa);
defer env.destroy();
const io = env.io();
var fx: events_fixture.Fixture = .{};
try fx.init(io, 1000);
defer fx.deinit();
env.mgr.diagnostics = &fx.store;
var fixture = try HttpFixture.init(io, http_body);
defer fixture.deinit(io);
fixture.setRoute(.oversize);
var group: std.Io.Group = .init;
defer group.cancel(io);
try group.concurrent(io, HttpFixture.serve, .{ &fixture, io });
var url_buf: [64]u8 = undefined;
const url = try fixture.url(&url_buf);
_ = try seedSource(&env.database, url);
try testing.expect(!try refreshOnce(env, url));
try testing.expectEqualStrings("blocklist.refresh", try fx.text(
"SELECT code FROM operational_events WHERE resolved_at IS NULL",
));
// The whole url, not the display copy: `subject_key` is the identity.
try testing.expectEqualStrings(url, try fx.text(
"SELECT subject_key FROM operational_events WHERE resolved_at IS NULL",
));
try testing.expectEqualStrings("warning", try fx.text(
"SELECT severity FROM operational_events WHERE resolved_at IS NULL",
));
fixture.setRoute(.body);
try testing.expect(try refreshOnce(env, url));
try testing.expectEqual(
@as(i64, 0),
try fx.count("SELECT count(*) FROM operational_events WHERE code = 'blocklist.refresh' AND resolved_at IS NULL"),
);
}
test "27: one refreshAll pass records one occurrence of a failing source" {
if (!build_options.integration) return error.SkipZigTest;
const gpa = testing.allocator;
const env = try Env.create(gpa);
defer env.destroy();
const io = env.io();
var fx: events_fixture.Fixture = .{};
try fx.init(io, 1000);
defer fx.deinit();
env.mgr.diagnostics = &fx.store;
var fixture = try HttpFixture.init(io, http_body);
defer fixture.deinit(io);
fixture.setRoute(.oversize);
var group: std.Io.Group = .init;
defer group.cancel(io);
try group.concurrent(io, HttpFixture.serve, .{ &fixture, io });
var url_buf: [64]u8 = undefined;
const url = try fixture.url(&url_buf);
_ = try seedSource(&env.database, url);
// `scheduledPass` is what an elapsed interval runs, and it ends in
// `refreshAll`, which ends in a reload. One pass is one flush: a reload
// that flushed on its own, or a scheduler that flushed after a pass that
// already had, would report this failure twice.
try env.mgr.scheduledPass(io);
try testing.expectEqual(@as(i64, 1), try fx.count(
"SELECT occurrences FROM operational_events WHERE code = 'blocklist.refresh'",
));
// A second pass on a still-failing source is one more occurrence: the
// failures one pass held ride the detail, and the flushes a pass makes are
// not what `occurrences` counts.
try env.mgr.scheduledPass(io);
try testing.expectEqual(@as(i64, 2), try fx.count(
"SELECT occurrences FROM operational_events WHERE code = 'blocklist.refresh'",
));
const detail = try fx.text("SELECT detail FROM operational_events WHERE code = 'blocklist.refresh'");
try testing.expect(std.mem.indexOf(u8, detail, "this pass") != null);
}
test "27: a clean reload resolves blocklist.snapshot and records no storage failure" {
if (!build_options.integration) return error.SkipZigTest;
const gpa = testing.allocator;
const env = try Env.create(gpa);
defer env.destroy();
const io = env.io();
var fx: events_fixture.Fixture = .{};
try fx.init(io, 1000);
defer fx.deinit();
// A stale episode from a previous boot, which a published snapshot closes.
fx.store.report(io, 900, .blocklist_snapshot, "snapshot", "blocklist snapshot", .@"error", "stale");
env.mgr.diagnostics = &fx.store;
try env.mgr.reload(io);
try testing.expectEqual(
@as(i64, 0),
try fx.count("SELECT count(*) FROM operational_events WHERE resolved_at IS NULL"),
);
try testing.expectEqual(@as(i64, 900), try fx.count("SELECT first_seen FROM operational_events"));
}
test "27: an unreadable blocklist directory opens a blocklist.storage episode" {
if (!build_options.integration) return error.SkipZigTest;
// Mode bits do not apply to root, so the denial the test needs cannot happen.
if (std.c.geteuid() == 0) return error.SkipZigTest;
const gpa = testing.allocator;
const env = try Env.create(gpa);
defer env.destroy();
const io = env.io();
var fx: events_fixture.Fixture = .{};
try fx.init(io, 1000);
defer fx.deinit();
env.mgr.diagnostics = &fx.store;
var dir = try env.blocklistDir();
dir.close(io);
try env.tmp.dir.setPermissions(io, .fromMode(0o600));
const failed = env.mgr.pruneOrphans(io);
try env.tmp.dir.setPermissions(io, .fromMode(0o700));
try testing.expectError(error.FileSystem, failed);
try testing.expectEqualStrings("blocklist.storage", try fx.text(
"SELECT code FROM operational_events WHERE resolved_at IS NULL",
));
// `createDirPathStatus` is the first call to touch the unreadable parent,
// so it is the operation that fails.
try testing.expectEqualStrings("create_dir", try fx.text(
"SELECT subject_key FROM operational_events WHERE resolved_at IS NULL",
));
// A pass that can read the directory again closes it.
try env.mgr.pruneOrphans(io);
try testing.expectEqual(
@as(i64, 0),
try fx.count("SELECT count(*) FROM operational_events WHERE resolved_at IS NULL"),
);
}
+627 -25
View File
@@ -59,6 +59,7 @@ const groups_repo = @import("../storage/repositories/groups_repo.zig");
const rules_repo = @import("../storage/repositories/rules_repo.zig");
const sources_repo = @import("../storage/repositories/sources_repo.zig");
const disk_monitor = @import("../storage/disk_monitor.zig");
const events = @import("../storage/events.zig");
const compiler = @import("compiler.zig");
const fetcher = @import("fetcher.zig");
const matcher = @import("matcher.zig");
@@ -162,6 +163,51 @@ pub const State = enum {
}
};
/// The `blocklist.storage` operations, each its own episode subject.
///
/// A fixed set on purpose: several of these fail once per file in a pass, and
/// one slot per operation is what turns that into one report per pass instead
/// of an unbounded list of them.
pub const StorageOp = enum { sweep, directory_read, create_dir, open_dir, delete };
/// One operation's outcome across one pass. `detail` keeps the last failure,
/// and `failures` says how many that pass held — the row's `occurrences` counts
/// failing passes, so the count belongs in the text.
const Aggregate = struct {
failures: u32 = 0,
succeeded: bool = false,
detail: [events.Store.max_detail_len]u8 = @splat(0),
detail_len: u16 = 0,
fn detailText(self: *const Aggregate) []const u8 {
return self.detail[0..self.detail_len];
}
};
/// What a locked body observed, held until `Manager.flushDiagnostics` can
/// report it with no manager lock held.
///
/// Two of this file's operations cannot report from where they stand:
/// `publishRefresh` runs under `writer_lock` by contract and `pruneOrphans`
/// holds both writer mutexes through its filesystem work. Collect-then-flush is
/// what keeps their outcomes without holding a lock across a store call, and
/// nothing here can grow: the storage slots are an enum array and a refresh
/// outcome rides the status entry the source already has.
///
/// The flush still happens inside the lock that serializes passes — a pass
/// drains its own outcomes before it releases `refresh_lock` (or, for a
/// standalone `reload`, `writer_lock`). What collect-then-flush avoids is
/// holding a *manager* lock across a store call, not deferring the report until
/// the next pass could merge into it.
const Pending = struct {
mutex: std.Io.Mutex = .init,
storage: std.EnumArray(StorageOp, Aggregate) = .initFill(.{}),
/// Null until a pass observes a snapshot outcome at all.
snapshot_failed: ?bool = null,
snapshot_detail: [events.Store.max_detail_len]u8 = @splat(0),
snapshot_detail_len: u16 = 0,
};
/// A status is a value with no borrowed memory, so a copy handed to the API
/// outlives every reload. The url is held inline for that reason.
pub const SourceStatus = struct {
@@ -182,6 +228,32 @@ pub const SourceStatus = struct {
url_len: u8 = 0,
last_error: [max_error_len]u8 = @splat(0),
last_error_len: u8 = 0,
/// The diagnostics identity of this source, canonicalized from the WHOLE
/// url by `setUrl`. `url` above is a display copy truncated at
/// `max_url_len`, and two urls sharing a 255-byte prefix would share one
/// episode if that copy were the key.
event_key: [events.Store.max_subject_key_len]u8 = @splat(0),
event_key_len: u16 = 0,
/// Diagnostics accounting for the pass in progress, cleared by every
/// `flushDiagnostics`. `pass_outcome` says this source recorded one at all;
/// `pass_failures` counts the failing ones, which a pass can hold more than
/// one of (a refresh that failed, then the reload that could not load the
/// files it did not write). One flush reports one `blocklist.refresh`
/// occurrence per source, so `occurrences` counts failing passes rather
/// than flushes, and the detail carries how many failures the pass held.
///
/// These two fields live in exactly one copy of the status table at a time,
/// which is what makes that count right while reloads replace the table
/// underneath: a candidate built by `mergeStatuses` carries none of them,
/// `installStatuses` folds the live table's in as it swaps, and the flush
/// claims an entry by copying it and zeroing both fields in one locked
/// step. Copy them anywhere else and the outcome gets reported twice.
pass_outcome: bool = false,
pass_failures: u16 = 0,
pub fn eventKey(self: *const SourceStatus) []const u8 {
return self.event_key[0..self.event_key_len];
}
pub fn errorText(self: *const SourceStatus) []const u8 {
return self.last_error[0..self.last_error_len];
@@ -196,9 +268,12 @@ pub const SourceStatus = struct {
@memcpy(self.url[0..kept], url[0..kept]);
@memset(self.url[kept..], 0);
self.url_len = @intCast(kept);
self.event_key_len = @intCast(events.canonicalKey(url, &self.event_key).len);
}
fn fail(self: *SourceStatus, state: State, text: []const u8) void {
self.pass_failures +|= 1;
self.pass_outcome = true;
self.state = state;
const kept = @min(text.len, max_error_len);
@memcpy(self.last_error[0..kept], text[0..kept]);
@@ -207,6 +282,7 @@ pub const SourceStatus = struct {
}
fn succeed(self: *SourceStatus, at: i64, counts: compiler.Counts) void {
self.pass_outcome = true;
self.state = .ok;
self.counts = counts;
self.last_success = at;
@@ -294,6 +370,14 @@ pub const Manager = struct {
/// what every test and `nxdns check` want. Only the scheduler consults it —
/// see `refreshGated`.
monitor: ?*disk_monitor.Monitor = null,
/// The diagnostics store, wired the same way as `monitor` and null
/// everywhere else. Never touched while a manager lock is held: see
/// `flushDiagnostics`.
diagnostics: ?*events.Store = null,
/// What the locked bodies observed and could not report from where they
/// stood. Bounded by construction — one slot per storage operation, one
/// snapshot outcome — and drained by `flushDiagnostics`.
pending: Pending = .{},
/// Scheduled refresh passes skipped by the disk gate. The `/api/health`
/// rollup reads it through `refreshesGated`.
refreshes_gated: std.atomic.Value(u64) = .init(0),
@@ -388,6 +472,228 @@ pub const Manager = struct {
return kept;
}
// -----------------------------------------------------------------------
// diagnostics
// -----------------------------------------------------------------------
/// Records one storage operation's failure. Callable from anywhere,
/// including under both writer mutexes: it touches `pending` only.
fn noteStorageFailure(
self: *Manager,
io: std.Io,
op: StorageOp,
comptime fmt: []const u8,
args: anytype,
) void {
if (self.diagnostics == null) return;
self.pending.mutex.lockUncancelable(io);
defer self.pending.mutex.unlock(io);
const slot = self.pending.storage.getPtr(op);
slot.failures +|= 1;
var w: std.Io.Writer = .fixed(&slot.detail);
w.print(fmt, args) catch {};
slot.detail_len = @intCast(w.end);
}
fn noteStorageSuccess(self: *Manager, io: std.Io, op: StorageOp) void {
if (self.diagnostics == null) return;
self.pending.mutex.lockUncancelable(io);
defer self.pending.mutex.unlock(io);
self.pending.storage.getPtr(op).succeeded = true;
}
/// Records whether a snapshot was published. `reason` null is the post-swap
/// success; anything else is the pass that could not publish one.
fn noteSnapshot(self: *Manager, io: std.Io, reason: ?[]const u8) void {
if (self.diagnostics == null) return;
self.pending.mutex.lockUncancelable(io);
defer self.pending.mutex.unlock(io);
self.pending.snapshot_failed = reason != null;
const text = reason orelse "";
const kept = @min(text.len, self.pending.snapshot_detail.len);
@memcpy(self.pending.snapshot_detail[0..kept], text[0..kept]);
self.pending.snapshot_detail_len = @intCast(kept);
}
/// Drains `pending` and the status table into the store, holding no manager
/// lock across a store call.
///
/// Called by every pass that can fill either one, and *before that pass
/// releases the lock serializing it* — `refresh_lock` for a refresh pass,
/// `writer_lock` for a standalone `reload`. Draining after the release
/// would let the next pass record its own outcomes on the same entries
/// first, and two failing passes would reach the store as one occurrence.
/// The `defer` that calls this is registered after the unlock `defer` for
/// that reason; defers run last-registered-first.
///
/// It is idempotent: a drained collector reports nothing.
pub fn flushDiagnostics(self: *Manager, io: std.Io) void {
const store = self.diagnostics orelse return;
const now_s = std.Io.Clock.real.now(io).toSeconds();
var storage: std.EnumArray(StorageOp, Aggregate) = undefined;
var snapshot_failed: ?bool = null;
var snapshot_detail: [events.Store.max_detail_len]u8 = undefined;
var snapshot_detail_len: u16 = 0;
{
self.pending.mutex.lockUncancelable(io);
defer self.pending.mutex.unlock(io);
storage = self.pending.storage;
snapshot_failed = self.pending.snapshot_failed;
snapshot_detail = self.pending.snapshot_detail;
snapshot_detail_len = self.pending.snapshot_detail_len;
self.pending.storage = .initFill(.{});
self.pending.snapshot_failed = null;
self.pending.snapshot_detail_len = 0;
}
var it = storage.iterator();
while (it.next()) |kv| {
const op = @tagName(kv.key);
if (kv.value.failures != 0) {
var buf: [events.Store.max_detail_len]u8 = undefined;
const detail = std.fmt.bufPrint(&buf, "{s} ({d} this pass)", .{
kv.value.detailText(),
kv.value.failures,
}) catch buf[0..];
store.report(io, now_s, .blocklist_storage, op, op, .warning, detail);
} else if (kv.value.succeeded) {
store.resolve(io, now_s, .blocklist_storage, op);
}
}
if (snapshot_failed) |failed| {
if (failed) {
store.report(
io,
now_s,
.blocklist_snapshot,
snapshot_key,
"blocklist snapshot",
.@"error",
snapshot_detail[0..snapshot_detail_len],
);
} else {
store.resolve(io, now_s, .blocklist_snapshot, snapshot_key);
}
}
self.flushSourceDiagnostics(io, store, now_s);
}
/// One `blocklist.refresh` episode per source, from the status table.
///
/// The table IS the per-source collection the collect-then-flush rule asks
/// for: `prepareRefresh`, `publishRefresh` and the reload's load outcomes
/// all write their result into the entry, under locks this cannot take. So
/// one entry is copied out at a time under the exclusive lock and the store
/// is called with nothing held.
///
/// The walk is a drain, not an index scan: a reload can replace the whole
/// table between two iterations, and an index into the table it replaced
/// would skip or repeat entries. Each round takes the lock, claims the
/// first entry that still carries pass accounting by copying it out and
/// zeroing the two fields, and reports it with nothing held. Claiming and
/// clearing are one locked step, so an outcome is reported once: a table
/// swapped in mid-drain carries the entries this flush has not claimed yet,
/// and `installStatuses` folded them in for exactly that reason. The drain
/// ends when a scan finds nothing left to claim.
///
/// The drain reaches only the sources the table still holds, so the sweep
/// below is what closes the episode of one that is gone.
fn flushSourceDiagnostics(self: *Manager, io: std.Io, store: *events.Store, now_s: i64) void {
drain: while (true) {
var status: SourceStatus = undefined;
{
self.lock.lockUncancelable(io);
defer self.lock.unlock(io);
const claimed = for (self.statuses) |*entry| {
if (!entry.pass_outcome) continue;
status = entry.*;
entry.pass_outcome = false;
entry.pass_failures = 0;
// A source with no diagnostics identity has nothing to
// report under, but its accounting is cleared all the same:
// left set, it would make every later scan claim it and the
// drain would never end.
if (entry.event_key_len == 0) continue;
break true;
} else false;
if (!claimed) break :drain;
}
if (!status.state.isRefreshFailure() and status.state != .load_failed) {
store.resolve(io, now_s, .blocklist_refresh, status.eventKey());
continue;
}
var buf: [events.Store.max_detail_len]u8 = undefined;
const detail = std.fmt.bufPrint(&buf, "{t}: {s} ({d} this pass)", .{
status.state,
status.errorText(),
status.pass_failures,
}) catch buf[0..];
var label_buf: [events.Store.max_subject_label_len]u8 = undefined;
const label = std.fmt.bufPrint(&label_buf, "{f}", .{
safe_url.redact(status.urlText()),
}) catch &label_buf;
store.report(io, now_s, .blocklist_refresh, status.eventKey(), label, .warning, detail);
}
self.resolveDeletedSources(io, store, now_s);
}
/// Closes the `blocklist.refresh` episode of a source that no longer exists.
///
/// Nothing else can. An episode of this code is closed by its source
/// succeeding, and a source deleted through the API or dropped by a config
/// import never succeeds again: the drain above walks the status table, the
/// deleted source has no entry in it, and the resolved-row pruning never
/// touches an active row. Without this the operator keeps a warning about a
/// list they removed on purpose, and no restart clears it.
///
/// The status table holds every source at every flush site — `refreshAll`
/// syncs it before it refreshes anything and a reload rebuilds it from the
/// rows — so its keys are exactly the episodes that may stay open.
fn resolveDeletedSources(self: *Manager, io: std.Io, store: *events.Store, now_s: i64) void {
var storage: [events.Store.max_kept_keys][events.Store.max_subject_key_len]u8 = undefined;
var lens: [events.Store.max_kept_keys]u16 = undefined;
var len: usize = 0;
{
// Shared: this reads the table and changes nothing in it. The keys
// are copied out because the arena they live in is freed by the
// next `installStatuses`, and the store is called below with
// nothing held.
self.lock.lockSharedUncancelable(io);
defer self.lock.unlockShared(io);
// An empty table before the first published snapshot means "no
// source set has been read yet", not "every source was deleted".
// Sweeping on it would close every episode the last run left open,
// and the pass that follows would reopen each one as a new episode
// with its history reset.
if (self.generation == 0) return;
for (self.statuses) |*entry| {
if (entry.event_key_len == 0) continue;
// `resolveExcept` refuses a kept list longer than
// `max_kept_keys`, because it canonicalizes onto the stack.
// Over that many keyed sources the sweep is skipped whole: the
// alternative is a truncated kept list, which would close
// episodes that are still true. A source deleted while the
// household is over the cap keeps its episode until the count
// falls back under it.
if (len == storage.len) return;
const key = entry.eventKey();
@memcpy(storage[len][0..key.len], key);
lens[len] = entry.event_key_len;
len += 1;
}
}
var kept: [events.Store.max_kept_keys][]const u8 = undefined;
for (0..len) |i| kept[i] = storage[i][0..lens[i]];
store.resolveExcept(io, now_s, .blocklist_refresh, kept[0..len]);
}
// -----------------------------------------------------------------------
// reload
// -----------------------------------------------------------------------
@@ -407,12 +713,35 @@ pub const Manager = struct {
/// table keeps describing that snapshot too — the table is rebuilt off to
/// the side and the load findings are written into it there, so a reload
/// that never publishes changes neither.
///
/// A standalone reload is its own pass, and `writer_lock` is what serializes
/// it against every other writer of the status table. So it flushes inside
/// that lock: the load outcomes it wrote at the swap are drained before any
/// other pass can add its own to the same entries, which is what keeps two
/// failing passes two occurrences instead of one.
pub fn reload(self: *Manager, io: std.Io) Error!void {
// Must not be entered with `writer_lock` held.
self.writer_lock.lockUncancelable(io);
defer self.writer_lock.unlock(io);
// Registered after the unlock so it runs before it, and `defer` and not
// straight-line code after the call: a reload that fails has already
// collected the outcomes that explain why, and leaving them pending
// would hold them until some later pass flushed them under the wrong
// timestamp.
defer self.flushDiagnostics(io);
return self.reloadLocked(io);
}
/// `reload` without the flush, for a caller that is inside a pass with a
/// flush of its own. One pass flushes once: flushing here as well would
/// split the pass's outcomes across two reports.
fn reloadCollecting(self: *Manager, io: std.Io) Error!void {
// Must not be entered with `writer_lock` held.
self.writer_lock.lockUncancelable(io);
defer self.writer_lock.unlock(io);
try self.reloadLocked(io);
}
fn reloadLocked(self: *Manager, io: std.Io) Error!void {
var rows = try sources_repo.listSourceRows(self.database, self.gpa);
defer rows.deinit(self.gpa);
@@ -543,6 +872,7 @@ pub const Manager = struct {
rows.items.len,
memory_bytes,
});
self.noteSnapshot(io, null);
}
/// What one enabled source contributes to the snapshot being built. Nothing
@@ -642,6 +972,11 @@ pub const Manager = struct {
pub fn refreshSource(self: *Manager, io: std.Io, row: sources_repo.SourceRow) Error!bool {
self.refresh_lock.lockUncancelable(io);
defer self.refresh_lock.unlock(io);
// Registered after the unlock, so it runs before it: a pass drains its
// own outcomes while it still holds `refresh_lock`. `defer` at all, so
// a refresh that fails outright still reports what it collected instead
// of leaving it for an unrelated later flush.
defer self.flushDiagnostics(io);
return self.refreshSourceLocked(io, row);
}
@@ -706,6 +1041,9 @@ pub const Manager = struct {
pub fn refreshAll(self: *Manager, io: std.Io) Error!void {
self.refresh_lock.lockUncancelable(io);
defer self.refresh_lock.unlock(io);
// Inside `refresh_lock`, by being registered after the unlock: see
// `refreshSource`.
defer self.flushDiagnostics(io);
var rows = try sources_repo.listSourceRows(self.database, self.gpa);
defer rows.deinit(self.gpa);
@@ -718,8 +1056,9 @@ pub const Manager = struct {
_ = try self.refreshSourceLocked(io, row);
}
// `reload` takes `writer_lock`, which the pass has been careful not to
// hold: the order is `refresh_lock` first, always.
return self.reload(io);
// hold: the order is `refresh_lock` first, always. The collecting
// variant, because the `defer` above is this pass's one flush.
return self.reloadCollecting(io);
}
/// The three temporary files one refresh compiles into, before the header
@@ -1161,9 +1500,16 @@ pub const Manager = struct {
// ask the same filesystem for.
try self.sweepOrphans(io);
// `startupPass` flushes its own outcomes before it releases
// `refresh_lock`, so the only flush left here is the one the failure
// note below needs.
self.startupPass(io) catch |err| switch (err) {
error.Canceled => return error.Canceled,
else => log.warn("blocklist startup pass failed: {s}", .{@errorName(err)}),
else => {
log.warn("blocklist startup pass failed: {s}", .{@errorName(err)});
self.noteSnapshot(io, @errorName(err));
self.flushDiagnostics(io);
},
};
if (!self.update.enabled) return;
@@ -1175,19 +1521,33 @@ pub const Manager = struct {
};
while (true) {
try interval.sleep(io);
// Ahead of the gate as well as ahead of the pass: the sweep only
// unlinks, so it is the one thing here that can give a critically
// full disk room back, and gating it would keep the residue that
// helped fill the disk in the first place.
try self.sweepOrphans(io);
if (self.refreshGated()) continue;
self.refreshAll(io) catch |err| switch (err) {
error.Canceled => return error.Canceled,
else => log.warn("blocklist refresh pass failed: {s}", .{@errorName(err)}),
};
try self.scheduledPass(io);
}
}
/// What one elapsed interval does. Split from the loop above so a test can
/// run the pass without waiting the interval out; nothing in production
/// calls it but `runScheduler`.
pub fn scheduledPass(self: *Manager, io: std.Io) std.Io.Cancelable!void {
// Ahead of the gate as well as ahead of the refresh: the sweep only
// unlinks, so it is the one thing here that can give a critically full
// disk room back, and gating it would keep the residue that helped fill
// the disk in the first place.
try self.sweepOrphans(io);
if (self.refreshGated()) return;
// `refreshAll` flushes the pass itself, so the only flush left here is
// the one the failure note below needs: flushing unconditionally would
// report every outcome of the pass a second time.
self.refreshAll(io) catch |err| switch (err) {
error.Canceled => return error.Canceled,
else => {
log.warn("blocklist refresh pass failed: {s}", .{@errorName(err)});
self.noteSnapshot(io, @errorName(err));
self.flushDiagnostics(io);
},
};
}
/// `pruneOrphans` with its failure absorbed. Leftover bytes under
/// `<data_dir>/blocklists/` are not an outage, and a sweep that could not
/// read the directory must not cost the household the refresh pass behind
@@ -1197,10 +1557,16 @@ pub const Manager = struct {
/// Taken from outside every `*Locked` body: `pruneOrphans` takes both
/// writer mutexes itself and neither is reentrant.
fn sweepOrphans(self: *Manager, io: std.Io) std.Io.Cancelable!void {
self.pruneOrphans(io) catch |err| switch (err) {
if (self.pruneOrphans(io)) {
self.noteStorageSuccess(io, .sweep);
} else |err| switch (err) {
error.Canceled => return error.Canceled,
else => log.warn("pruning orphaned blocklist files failed: {s}", .{@errorName(err)}),
};
else => {
log.warn("pruning orphaned blocklist files failed: {s}", .{@errorName(err)});
self.noteStorageFailure(io, .sweep, "pruning orphaned blocklist files failed: {s}", .{@errorName(err)});
},
}
self.flushDiagnostics(io);
}
/// The §11.6 gate, consulted by scheduled passes only (ruling 17). A
@@ -1232,11 +1598,14 @@ pub const Manager = struct {
fn startupPass(self: *Manager, io: std.Io) Error!void {
self.refresh_lock.lockUncancelable(io);
defer self.refresh_lock.unlock(io);
// Inside `refresh_lock`, by being registered after the unlock: see
// `refreshSource`.
defer self.flushDiagnostics(io);
// Ahead of the gate on purpose: loading the compiled files that already
// exist is a read. A full disk must not cost the household its
// filtering as well as its downloads.
try self.reload(io);
try self.reloadCollecting(io);
if (self.refreshGated()) return;
@@ -1251,7 +1620,7 @@ pub const Manager = struct {
if (!self.needsRefresh(io, row, now)) continue;
if (try self.refreshSourceLocked(io, row)) refreshed = true;
}
if (refreshed) try self.reload(io);
if (refreshed) try self.reloadCollecting(io);
}
fn needsRefresh(self: *Manager, io: std.Io, row: sources_repo.SourceRow, now: i64) bool {
@@ -1297,6 +1666,13 @@ pub const Manager = struct {
/// `<data_dir>/blocklists/` if nothing has yet, and an empty directory
/// sweeps to nothing.
pub fn pruneOrphans(self: *Manager, io: std.Io) Error!void {
defer self.flushDiagnostics(io);
return self.pruneOrphansLocked(io);
}
/// Assumes nothing and takes both writer mutexes itself. Split from
/// `pruneOrphans` so the diagnostics flush above happens with neither held.
fn pruneOrphansLocked(self: *Manager, io: std.Io) Error!void {
// `refresh_lock` first, and for the reason it exists: the download and
// the compile are the only writers of `.raw.tmp`, `.list.tmp`,
// `.wild.tmp` and `.allow.tmp`, and they hold it for as long as they
@@ -1334,6 +1710,7 @@ pub const Manager = struct {
error.Canceled => return error.Canceled,
else => {
log.warn("pruning blocklists: reading the directory failed: {s}", .{@errorName(err)});
self.noteStorageFailure(io, .directory_read, "reading the blocklist directory failed: {s}", .{@errorName(err)});
return error.FileSystem;
},
} orelse break;
@@ -1343,6 +1720,8 @@ pub const Manager = struct {
try doomed.append(self.gpa, try self.gpa.dupe(u8, entry.name));
}
self.noteStorageSuccess(io, .directory_read);
for (doomed.items) |name| {
self.deleteQuietly(io, dir, name);
log.info("pruned orphaned blocklist file {s}", .{name});
@@ -1381,8 +1760,21 @@ pub const Manager = struct {
}
/// Publishes a built table and frees the one it replaces. The caller holds
/// the exclusive lock, so no reader is inside the old table.
/// the exclusive lock, so no reader is inside the old table and no flush is
/// half way through draining it.
///
/// The pass accounting the live table still holds is folded into the
/// incoming entry of the same id first. `mergeStatuses` left the candidate
/// carrying none, so an outcome recorded after the candidate was built —
/// and any a flush has not drained yet — survives the swap exactly once. An
/// outcome a flush already reported is zero in the live table, so nothing
/// here resurrects it.
fn installStatuses(self: *Manager, table: StatusTable) void {
for (table.items) |*incoming| {
const live = entryFor(self.statuses, incoming.id) orelse continue;
incoming.pass_failures +|= live.pass_failures;
incoming.pass_outcome = incoming.pass_outcome or live.pass_outcome;
}
self.status_arena.deinit();
self.status_arena = table.arena;
self.statuses = table.items;
@@ -1449,29 +1841,46 @@ pub const Manager = struct {
error.Canceled => return error.Canceled,
else => {
log.warn("creating {s} failed: {s}", .{ self.paths.subdir, @errorName(err) });
self.noteStorageFailure(io, .create_dir, "creating {s} failed: {s}", .{ self.paths.subdir, @errorName(err) });
return error.FileSystem;
},
};
return self.paths.dir.openDir(io, self.paths.subdir, options) catch |err| switch (err) {
self.noteStorageSuccess(io, .create_dir);
const dir = self.paths.dir.openDir(io, self.paths.subdir, options) catch |err| switch (err) {
error.Canceled => return error.Canceled,
else => {
log.warn("opening {s} failed: {s}", .{ self.paths.subdir, @errorName(err) });
self.noteStorageFailure(io, .open_dir, "opening {s} failed: {s}", .{ self.paths.subdir, @errorName(err) });
return error.FileSystem;
},
};
self.noteStorageSuccess(io, .open_dir);
return dir;
}
/// A temporary that cannot be removed is not a failure of the operation
/// that made it, but it is not nothing either: it is left visible.
fn deleteQuietly(self: *Manager, io: std.Io, dir: std.Io.Dir, name: []const u8) void {
_ = self;
dir.deleteFile(io, name) catch |err| switch (err) {
error.FileNotFound => {},
else => log.warn("deleting {s} failed: {s}", .{ name, @errorName(err) }),
};
if (dir.deleteFile(io, name)) {
self.noteStorageSuccess(io, .delete);
} else |err| switch (err) {
// A name that was never created is the ordinary case: the temporary
// deletes are installed before the files exist.
error.FileNotFound => self.noteStorageSuccess(io, .delete),
else => {
log.warn("deleting {s} failed: {s}", .{ name, @errorName(err) });
self.noteStorageFailure(io, .delete, "deleting {s} failed: {s}", .{ name, @errorName(err) });
},
}
}
};
/// The one subject `blocklist.snapshot` ever has: a box publishes exactly one
/// snapshot, and every source that failed to load is its own
/// `blocklist.refresh` episode.
const snapshot_key = "snapshot";
/// A status table and the arena holding it. Until `installStatuses` takes it,
/// it is a candidate nobody can see, and `deinit` frees it whole.
const StatusTable = struct {
@@ -1488,6 +1897,14 @@ const StatusTable = struct {
/// over from `previous`. A source deleted since `previous` was built is gone; a
/// source added since starts blank. `previous` is only read, so the caller's
/// published table is untouched by this.
///
/// The pass accounting is *not* carried: it lives in exactly one table copy at
/// a time. `previous` is a snapshot of the published table taken outside the
/// swap, so copying its counters here would leave the same outcomes in two
/// tables — the live one for a flush to drain, and this candidate for the
/// reload's own flush to report a second time. A candidate holds only what
/// `applyLoadOutcomes` writes into it; what the live table holds is folded in
/// by `installStatuses` under the exclusive lock.
fn mergeStatuses(
table: []SourceStatus,
rows: []const sources_repo.SourceRow,
@@ -1500,6 +1917,8 @@ fn mergeStatuses(
status.* = prior;
break;
}
status.pass_outcome = false;
status.pass_failures = 0;
// After the carry-over: a url edited on the row wins over the one the
// prior entry recorded.
status.setUrl(row.url);
@@ -1716,6 +2135,7 @@ fn containsId(rows: []const sources_repo.SourceRow, id: i64) bool {
// real swaps under load are the integration suite's (S9).
const testing = std.testing;
const events_fixture = @import("../storage/events_fixture.zig");
const migrations = @import("../storage/migrations.zig");
fn openMigrated() !db.Db {
@@ -2275,6 +2695,13 @@ test "a candidate table carries prior entries over and leaves the published one
try testing.expectEqual(State.never_fetched, candidate[1].state);
try testing.expect(!candidate[1].loaded);
// The one thing a candidate does not carry. `published[0]` is holding a
// failure no flush has drained yet; copying its accounting here would leave
// the same outcome in two tables, and the flush of each would report it.
try testing.expect(published[0].pass_outcome);
try testing.expect(!candidate[0].pass_outcome);
try testing.expectEqual(@as(u16, 0), candidate[0].pass_failures);
// The published table is untouched, so a reload that fails before the swap
// leaves it describing the snapshot that is still serving — including the
// entry of the deleted source, which that snapshot still enforces.
@@ -2284,6 +2711,181 @@ test "a candidate table carries prior entries over and leaves the published one
try testing.expectEqualStrings("https://lists.example/one.txt", published[0].urlText());
}
test "installing a table folds the live pass accounting in by id" {
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
defer threaded.deinit();
const io = threaded.io();
var database = try openMigrated();
defer database.close();
var f: fetcher.Fetcher = undefined;
var mgr = try testManager(&database, &f);
defer mgr.deinit(io);
// Source 1 recorded a failure the flush has not drained; source 2 was
// drained already; source 3 recorded one this reload knows nothing about.
var live: std.heap.ArenaAllocator = .init(testing.allocator);
const live_items = try live.allocator().alloc(SourceStatus, 3);
live_items[0] = .{ .id = 1, .pass_outcome = true, .pass_failures = 2 };
live_items[1] = .{ .id = 2 };
live_items[2] = .{ .id = 3, .pass_outcome = true, .pass_failures = 1 };
{
mgr.lock.lockUncancelable(io);
defer mgr.lock.unlock(io);
mgr.installStatuses(.{ .arena = live, .items = live_items });
}
// What a reload built beside it, carrying only its own load outcomes.
var incoming: std.heap.ArenaAllocator = .init(testing.allocator);
const incoming_items = try incoming.allocator().alloc(SourceStatus, 3);
incoming_items[0] = .{ .id = 1, .pass_outcome = true, .pass_failures = 1 };
incoming_items[1] = .{ .id = 2, .pass_outcome = true, .pass_failures = 4 };
incoming_items[2] = .{ .id = 3 };
{
mgr.lock.lockUncancelable(io);
defer mgr.lock.unlock(io);
mgr.installStatuses(.{ .arena = incoming, .items = incoming_items });
}
try testing.expectEqual(@as(u16, 3), mgr.statuses[0].pass_failures);
try testing.expect(mgr.statuses[0].pass_outcome);
// A drained entry adds nothing: what the swap publishes is the reload's own
// accounting and no resurrection of what was already reported.
try testing.expectEqual(@as(u16, 4), mgr.statuses[1].pass_failures);
try testing.expect(mgr.statuses[1].pass_outcome);
// The half the swap used to lose: an outcome the live table held and the
// candidate never saw.
try testing.expectEqual(@as(u16, 1), mgr.statuses[2].pass_failures);
try testing.expect(mgr.statuses[2].pass_outcome);
}
test "the flush claims every entry that carries pass accounting, once" {
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
defer threaded.deinit();
const io = threaded.io();
var database = try openMigrated();
defer database.close();
var fx: events_fixture.Fixture = .{};
try fx.init(io, 1_700_000_000);
defer fx.deinit();
var f: fetcher.Fetcher = undefined;
var mgr = try testManager(&database, &f);
defer mgr.deinit(io);
mgr.diagnostics = &fx.store;
var arena: std.heap.ArenaAllocator = .init(testing.allocator);
const items = try arena.allocator().alloc(SourceStatus, 3);
items[0] = .{ .id = 1 };
items[0].setUrl("https://lists.example/one.txt");
items[0].fail(.fetch_failed, "HttpStatus");
// No url, so no episode to report under. The drain has to claim it anyway:
// an entry left with `pass_outcome` set is the one every later scan finds
// first, and the entry behind it would never be reached.
items[1] = .{ .id = 2, .pass_outcome = true, .pass_failures = 1 };
items[2] = .{ .id = 3 };
items[2].setUrl("https://lists.example/three.txt");
items[2].succeed(1_700_000_000, .{ .domains = 3 });
{
mgr.lock.lockUncancelable(io);
defer mgr.lock.unlock(io);
mgr.installStatuses(.{ .arena = arena, .items = items });
}
mgr.flushDiagnostics(io);
try testing.expectEqual(@as(i64, 1), try fx.count(
"SELECT count(*) FROM operational_events WHERE code = 'blocklist.refresh'",
));
try testing.expectEqual(@as(i64, 1), try fx.count(
"SELECT occurrences FROM operational_events WHERE code = 'blocklist.refresh'",
));
for (mgr.statuses) |entry| {
try testing.expect(!entry.pass_outcome);
try testing.expectEqual(@as(u16, 0), entry.pass_failures);
}
// Drained: flushing the same table again reports nothing a second time.
mgr.flushDiagnostics(io);
try testing.expectEqual(@as(i64, 1), try fx.count(
"SELECT occurrences FROM operational_events WHERE code = 'blocklist.refresh'",
));
}
test "the flush closes the episode of a source that is no longer in the table" {
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
defer threaded.deinit();
const io = threaded.io();
var database = try openMigrated();
defer database.close();
var fx: events_fixture.Fixture = .{};
try fx.init(io, 1_700_000_000);
defer fx.deinit();
var f: fetcher.Fetcher = undefined;
var mgr = try testManager(&database, &f);
defer mgr.deinit(io);
mgr.diagnostics = &fx.store;
// What the operator deleted while it was failing. Nothing will ever record
// a success for it, so nothing but the sweep can close this.
fx.store.report(
io,
1_700_000_000,
.blocklist_refresh,
"https://lists.example/deleted.txt",
"lists.example/deleted.txt",
.warning,
"fetch_failed: HttpStatus (1 this pass)",
);
var arena: std.heap.ArenaAllocator = .init(testing.allocator);
const items = try arena.allocator().alloc(SourceStatus, 1);
items[0] = .{ .id = 1 };
items[0].setUrl("https://lists.example/one.txt");
items[0].fail(.fetch_failed, "HttpStatus");
{
mgr.lock.lockUncancelable(io);
defer mgr.lock.unlock(io);
mgr.installStatuses(.{ .arena = arena, .items = items });
}
// No snapshot published yet, so the table is not known to describe the
// source set and the sweep must not run: the drain reports the failing
// source and the deleted one's episode is left alone.
mgr.flushDiagnostics(io);
try testing.expectEqual(@as(i64, 2), try fx.count(
"SELECT count(*) FROM operational_events WHERE code = 'blocklist.refresh' AND resolved_at IS NULL",
));
mgr.generation = 1;
mgr.flushDiagnostics(io);
// One left active, and it is the source that still exists.
try testing.expectEqual(@as(i64, 1), try fx.count(
"SELECT count(*) FROM operational_events WHERE code = 'blocklist.refresh' AND resolved_at IS NULL",
));
try testing.expectEqualStrings(
"https://lists.example/one.txt",
try fx.text(
"SELECT subject_key FROM operational_events WHERE code = 'blocklist.refresh' AND resolved_at IS NULL",
),
);
try testing.expectEqualStrings(
"https://lists.example/deleted.txt",
try fx.text(
"SELECT subject_key FROM operational_events WHERE code = 'blocklist.refresh' AND resolved_at IS NOT NULL",
),
);
}
test "a disabled source stops being loaded" {
var statuses = [_]SourceStatus{.{ .id = 1 }};
statuses[0].succeed(1_700_000_000, .{ .domains = 9 });
+62 -1
View File
@@ -33,8 +33,17 @@ const log = std.log.scoped(.forward_client);
/// each half large enough to frame a query in one write, not a capacity.
pub const min_frame_buf: usize = 1024;
/// `tcp://[` + the longest IPv6 text form + `]:65535`, the widest spelling
/// `identityText` can produce.
pub const max_identity_len: usize = "tcp://[".len + 45 + "]:65535".len;
pub const ForwardClient = struct {
resolver: validate.Resolver,
/// The resolver as text, owned here so the `transport.Client` out-parameter
/// has something stable to borrow: `validate.Resolver` is a parsed address,
/// and a caller logging the exchange needs its spelling.
identity_buf: [max_identity_len]u8 = undefined,
identity_len: usize = 0,
/// Caller-owned scratch for the TCP length-prefixed path.
frame_buf: []u8,
/// On the `.awake` clock at the caller's choosing, so a suspended host does
@@ -64,11 +73,20 @@ pub const ForwardClient = struct {
read_timeout: std.Io.Clock.Duration,
) ForwardClient {
std.debug.assert(frame_buf.len >= min_frame_buf);
return .{
var self: ForwardClient = .{
.resolver = resolver,
.frame_buf = frame_buf,
.read_timeout = read_timeout,
};
self.identity_len = identityText(resolver, &self.identity_buf).len;
return self;
}
/// `udp://192.168.1.1:53`, `tcp://[fd00::1]:53` — the same spelling
/// `validate.parseResolver` accepts, so a log row names the configured
/// value. Valid for as long as this client is.
pub fn identity(self: *const ForwardClient) []const u8 {
return self.identity_buf[0..self.identity_len];
}
pub fn client(self: *ForwardClient) transport.Client {
@@ -80,8 +98,12 @@ pub const ForwardClient = struct {
io: std.Io,
query: []const u8,
response_buf: []u8,
selected: *?[]const u8,
) transport.ExchangeError![]u8 {
const self: *ForwardClient = @ptrCast(@alignCast(ptr));
// Set before the attempt: a failed forward-zone exchange still names
// the resolver it was sent to.
selected.* = self.identity();
return self.exchange(io, query, response_buf);
}
@@ -238,6 +260,24 @@ pub const ForwardClient = struct {
}
};
fn identityText(resolver: validate.Resolver, buf: *[max_identity_len]u8) []const u8 {
var w: std.Io.Writer = .fixed(buf);
w.writeAll(switch (resolver.scheme) {
.udp => "udp://",
.tcp => "tcp://",
}) catch unreachable;
const bracketed = switch (resolver.addr) {
.ip4 => false,
.ip6 => true,
};
if (bracketed) w.writeByte('[') catch unreachable;
resolver.addr.format(&w) catch unreachable;
if (bracketed) w.writeByte(']') catch unreachable;
w.print(":{d}", .{resolver.port}) catch unreachable;
return w.buffered();
}
/// The local address a datagram to `dest` is sent from: same family, port
/// chosen by the kernel.
fn wildcardFor(dest: net.IpAddress) net.IpAddress {
@@ -286,6 +326,27 @@ test "ForwardClient satisfies the Client interface" {
try testing.expectEqual(@as(u16, 53), fc.resolver.port);
}
test "the client owns its resolver identity in both address families" {
var buf = testBuf();
const v4: ForwardClient = .init(
try validate.parseResolver("udp://192.168.1.1:5300"),
&buf,
.{ .raw = .fromSeconds(1), .clock = .awake },
);
try testing.expectEqualStrings("udp://192.168.1.1:5300", v4.identity());
var buf6 = testBuf();
const v6: ForwardClient = .init(
try validate.parseResolver("tcp://[fd00::1]:5353"),
&buf6,
.{ .raw = .fromSeconds(1), .clock = .awake },
);
try testing.expectEqualStrings("tcp://[fd00::1]:5353", v6.identity());
// The borrow points into the client, not into `init`'s frame.
try testing.expect(@intFromPtr(v6.identity().ptr) >= @intFromPtr(&v6));
}
test "the stats struct starts at zero" {
const stats: ForwardClient.Stats = .{};
try testing.expectEqual(@as(u64, 0), stats.queries);
+130 -10
View File
@@ -13,6 +13,8 @@
//! through the size.
const std = @import("std");
const events = @import("../storage/events.zig");
const tls_server = @import("../platform/tls_server.zig");
const log = std.log.scoped(.cert_store);
@@ -101,6 +103,11 @@ pub const ReloadHook = struct {
};
pub const CertStore = struct {
/// Which endpoint's certificate this store holds. It is the subject of
/// every `certificate.reload` event, and a box serving both DoH and DoT
/// runs two stores over two file pairs.
pub const Kind = enum { doh, dot };
gpa: std.mem.Allocator,
/// Borrowed from the config; must outlive the store.
cert_path: []const u8,
@@ -132,6 +139,12 @@ pub const CertStore = struct {
/// `reload` synchronously (that would self-deadlock on `reload_mutex`).
after_load_hook: ?ReloadHook,
/// Set by the composition root right after `init`, with `diagnostics`.
kind: Kind = .doh,
/// Wired the same way and for the same reason as every other subsystem's:
/// the store is fully usable without it, and `nxdns check` has none.
diagnostics: ?*events.Store = null,
reloads: std.atomic.Value(u64),
reload_failures: std.atomic.Value(u64),
/// Wall-clock second of the last successful load, including the one in
@@ -243,7 +256,7 @@ pub const CertStore = struct {
};
while (true) {
try interval.sleep(io);
self.pollOnce(io);
self.pollOnce(io, std.Io.Clock.real.now(io).toSeconds());
}
}
@@ -252,13 +265,15 @@ pub const CertStore = struct {
/// changed, and the old one keeps serving either way. A failed reload
/// warns and counts (`reload_failures`); the signature stays at the loaded
/// pair, so every subsequent poll retries until the files parse.
pub fn pollOnce(self: *CertStore, io: std.Io) void {
const cert_sig = statSig(io, self.cert_path) catch {
pub fn pollOnce(self: *CertStore, io: std.Io, now_s: i64) void {
const cert_sig = statSig(io, self.cert_path) catch |err| {
log.warn("stat {s} failed; keeping the loaded certificate", .{self.cert_path});
self.reportReload(io, now_s, "stat of the certificate failed", @errorName(err));
return;
};
const key_sig = statSig(io, self.key_path) catch {
const key_sig = statSig(io, self.key_path) catch |err| {
log.warn("stat {s} failed; keeping the loaded certificate", .{self.key_path});
self.reportReload(io, now_s, "stat of the private key failed", @errorName(err));
return;
};
const observed: Signature = .{ .cert = cert_sig, .key = key_sig };
@@ -266,18 +281,36 @@ pub const CertStore = struct {
self.mutex.lockUncancelable(io);
const loaded = self.loaded;
self.mutex.unlock(io);
if (!changed(loaded, observed)) return;
if (!changed(loaded, observed)) {
// A poll that stat'ed both files and found nothing to do is a
// fully healthy pass, so it closes any episode a transient stat
// failure opened. Without this, a file that never changes again
// would leave that episode open forever.
if (self.diagnostics) |store| store.resolve(io, now_s, .certificate_reload, @tagName(self.kind));
return;
}
if (self.reload(io)) {
log.info("certificate reloaded from {s}", .{self.cert_path});
if (self.diagnostics) |store| store.resolve(io, now_s, .certificate_reload, @tagName(self.kind));
} else |err| {
log.warn("certificate reload from {s} failed ({s}); the old certificate keeps serving", .{
self.cert_path,
humanMessage(err),
});
self.reportReload(io, now_s, "certificate reload failed", humanMessage(err));
}
}
/// A warning, never an error: a stat failure can be a rename window, and a
/// failed reload leaves the loaded certificate serving. Nothing is down.
fn reportReload(self: *CertStore, io: std.Io, now_s: i64, message: []const u8, reason: []const u8) void {
const store = self.diagnostics orelse return;
var buf: [events.Store.max_detail_len]u8 = undefined;
const detail = std.fmt.bufPrint(&buf, "{s}: {s}", .{ message, reason }) catch buf[0..];
store.report(io, now_s, .certificate_reload, @tagName(self.kind), @tagName(self.kind), .warning, detail);
}
pub fn snapshotStats(self: *const CertStore) Stats {
return .{
.reloads = self.reloads.load(.monotonic),
@@ -406,6 +439,7 @@ fn statSig(io: std.Io, path: []const u8) !FileSig {
// tests
// ---------------------------------------------------------------------------
const events_fixture = @import("../storage/events_fixture.zig");
const fixtures = @import("test_fixtures");
const testing = std.testing;
@@ -699,7 +733,7 @@ test "pollOnce reloads on a changed stat pair and stays put on an unchanged one"
var store = try CertStore.init(testing.allocator, io, env.cert_path, env.key_path, null);
defer store.deinit(io);
store.pollOnce(io);
store.pollOnce(io, 1000);
try testing.expectEqual(@as(u64, 0), store.snapshotStats().reloads);
// Same certificate plus a trailing newline: the PEM still parses and the
@@ -710,14 +744,14 @@ test "pollOnce reloads on a changed stat pair and stays put on an unchanged one"
const old = store.acquire(io);
store.release(io, old);
store.pollOnce(io);
store.pollOnce(io, 1000);
try testing.expectEqual(@as(u64, 1), store.snapshotStats().reloads);
const fresh = store.acquire(io);
try testing.expect(fresh != old);
store.release(io, fresh);
store.pollOnce(io);
store.pollOnce(io, 1000);
try testing.expectEqual(@as(u64, 1), store.snapshotStats().reloads);
}
@@ -734,7 +768,7 @@ test "pollOnce warns and keeps serving when a reload fails" {
store.release(io, before);
try env.tmp.dir.writeFile(io, .{ .sub_path = "cert.pem", .data = "still not a certificate" });
store.pollOnce(io);
store.pollOnce(io, 1000);
const stats = store.snapshotStats();
try testing.expectEqual(@as(u64, 0), stats.reloads);
@@ -755,7 +789,7 @@ test "pollOnce does nothing when a file cannot be stat'ed" {
defer store.deinit(io);
try env.tmp.dir.deleteFile(io, "cert.pem");
store.pollOnce(io);
store.pollOnce(io, 1000);
const stats = store.snapshotStats();
try testing.expectEqual(@as(u64, 0), stats.reloads);
@@ -873,3 +907,89 @@ test "a reload overlapping another reload's window publishes last" {
store.mutex.unlock(io);
try testing.expectEqual(@as(u64, grown.len), final.cert.size);
}
test "a failed reload opens an episode keyed by endpoint kind and a good one closes it" {
var env: TestEnv = undefined;
try env.init();
defer env.deinit();
const io = env.io();
var fx: events_fixture.Fixture = .{};
try fx.init(io, 1000);
defer fx.deinit();
var store = try CertStore.init(testing.allocator, io, env.cert_path, env.key_path, null);
defer store.deinit(io);
store.kind = .dot;
store.diagnostics = &fx.store;
try env.tmp.dir.writeFile(io, .{ .sub_path = "cert.pem", .data = "still not a certificate" });
store.pollOnce(io, 1000);
try testing.expectEqualStrings("certificate.reload", try fx.text("SELECT code FROM operational_events"));
try testing.expectEqualStrings("dot", try fx.text("SELECT subject_key FROM operational_events"));
try testing.expectEqualStrings("warning", try fx.text("SELECT severity FROM operational_events"));
// The same certificate plus a newline: it parses, and the size differs even
// within one timestamp granule.
const grown = try std.mem.concat(testing.allocator, u8, &.{ fixtures.cert_pem, "\n" });
defer testing.allocator.free(grown);
try env.tmp.dir.writeFile(io, .{ .sub_path = "cert.pem", .data = grown });
store.pollOnce(io, 1100);
try testing.expectEqual(@as(u64, 1), store.snapshotStats().reloads);
try testing.expectEqual(
@as(i64, 0),
try fx.count("SELECT count(*) FROM operational_events WHERE resolved_at IS NULL"),
);
}
test "a failed stat opens the same episode a failed reload would" {
var env: TestEnv = undefined;
try env.init();
defer env.deinit();
const io = env.io();
var fx: events_fixture.Fixture = .{};
try fx.init(io, 1000);
defer fx.deinit();
var store = try CertStore.init(testing.allocator, io, env.cert_path, env.key_path, null);
defer store.deinit(io);
store.diagnostics = &fx.store;
try env.tmp.dir.deleteFile(io, "cert.pem");
store.pollOnce(io, 1000);
try testing.expectEqualStrings("doh", try fx.text(
"SELECT subject_key FROM operational_events WHERE resolved_at IS NULL",
));
try testing.expectEqual(@as(u64, 0), store.snapshotStats().reload_failures);
}
test "an unchanged poll closes the episode a transient stat failure opened" {
var env: TestEnv = undefined;
try env.init();
defer env.deinit();
const io = env.io();
var fx: events_fixture.Fixture = .{};
try fx.init(io, 1000);
defer fx.deinit();
var store = try CertStore.init(testing.allocator, io, env.cert_path, env.key_path, null);
defer store.deinit(io);
store.diagnostics = &fx.store;
// What a stat failure in a rename window left open. The file it named is
// back and unchanged, so no reload will ever close this episode.
fx.store.report(io, 1000, .certificate_reload, "doh", "doh", .warning, "stat of the certificate failed: FileNotFound");
store.pollOnce(io, 1100);
try testing.expectEqual(@as(u64, 0), store.snapshotStats().reloads);
try testing.expectEqual(
@as(i64, 0),
try fx.count("SELECT count(*) FROM operational_events WHERE resolved_at IS NULL"),
);
}
+105 -1
View File
@@ -30,6 +30,7 @@ const clients_repo = @import("../storage/repositories/clients_repo.zig");
const db = @import("../storage/db.zig");
const dns_header = @import("../dns/header.zig");
const edns = @import("../dns/edns.zig");
const events = @import("../storage/events.zig");
const forward_client = @import("../local/forward_client.zig");
const local_tables = @import("local_tables.zig");
const name_mod = @import("../dns/name.zig");
@@ -103,6 +104,9 @@ pub const Resolver = struct {
tables: *local_tables.LocalTables,
stats: Stats = .{},
exchange_fn: ExchangeFn = defaultExchange,
/// Wired by the composition root after `init`, following the
/// `gate: ?*disk_monitor.Monitor` idiom. Null in every unit test here.
diagnostics: ?*events.Store = null,
pub fn init(tables: *local_tables.LocalTables) Resolver {
return .{ .tables = tables };
@@ -127,10 +131,13 @@ pub const Resolver = struct {
self.mutex.lockUncancelable(io);
self.stats.read_failures += 1;
self.mutex.unlock(io);
self.reportStorage(io, now_s, "read", "selecting clients to name failed", @errorName(err), 1);
return;
};
if (self.diagnostics) |store| store.resolve(io, now_s, .client_names_storage, "read");
var write_failures: u64 = 0;
var last_write_error: []const u8 = "";
for (candidates[0..count]) |*candidate| {
var learned_buf: [types.max_name_len]u8 = undefined;
const result = self.attempt(io, candidate.ip(), &learned_buf);
@@ -144,6 +151,7 @@ pub const Resolver = struct {
log.warn("recording the name of {s} failed: {s}", .{ candidate.ip(), @errorName(err) });
}
write_failures += 1;
last_write_error = @errorName(err);
}
self.mutex.lockUncancelable(io);
@@ -158,10 +166,38 @@ pub const Resolver = struct {
self.mutex.unlock(io);
}
if (write_failures == 0) return;
// The clean-pass determination is what closes a `write` episode: one
// aggregated outcome per pass, so a row's `occurrences` counts failing
// passes rather than failing rows.
if (write_failures == 0) {
if (self.diagnostics) |store| store.resolve(io, now_s, .client_names_storage, "write");
return;
}
self.mutex.lockUncancelable(io);
self.stats.write_failures += write_failures;
self.mutex.unlock(io);
self.reportStorage(io, now_s, "write", "recording a client name failed", last_write_error, write_failures);
}
/// One aggregated report per pass. `count` is how many failures that pass
/// held; the detail carries it because the row itself counts passes.
fn reportStorage(
self: *Resolver,
io: std.Io,
now_s: i64,
operation: []const u8,
message: []const u8,
error_name: []const u8,
count: u64,
) void {
const store = self.diagnostics orelse return;
var buf: [events.Store.max_detail_len]u8 = undefined;
const detail = std.fmt.bufPrint(&buf, "{s}: {s} ({d} this pass)", .{
message,
error_name,
count,
}) catch buf[0..];
store.report(io, now_s, .client_names_storage, operation, operation, .warning, detail);
}
const Attempt = struct {
@@ -355,6 +391,7 @@ fn exchangeOnce(
const forward_zones = @import("../local/forward_zones.zig");
const migrations = @import("../storage/migrations.zig");
const events_fixture = @import("../storage/events_fixture.zig");
const testing = std.testing;
/// A stub exchange whose reply and error are set by the test. `calls` is the
@@ -1157,3 +1194,70 @@ test "an extended rcode of zero still reads as the header's rcode" {
try testing.expectEqualStrings("nas.lan", (try f.learned("192.168.1.10", &buf)).?);
try testing.expectEqual(@as(u64, 1), f.resolver.snapshotStats(f.io()).answered);
}
test "a failing write opens one episode per pass and a clean pass closes it" {
var f: Fixture = undefined;
try fixture(&f, "168.192.in-addr.arpa");
defer f.deinit();
var fx: events_fixture.Fixture = .{};
try fx.init(f.io(), 1000);
defer fx.deinit();
f.resolver.diagnostics = &fx.store;
// Two addresses outside the declared zone: both reach the write step
// without any exchange, which is not what this test is about.
try clients_repo.upsertSeen(&f.database, "10.0.0.1", 1700000000);
try clients_repo.upsertSeen(&f.database, "10.0.0.2", 1700000000);
try f.database.exec(
\\CREATE TRIGGER refuse_update BEFORE UPDATE ON clients
\\BEGIN SELECT RAISE(ABORT, 'refused'); END;
);
f.resolver.runPass(f.io(), &f.database, 1700000000);
// Two failing rows, one aggregated report: the row counts failing passes.
try testing.expectEqual(@as(i64, 1), try fx.count("SELECT count(*) FROM operational_events"));
try testing.expectEqualStrings("client_names.storage", try fx.text("SELECT code FROM operational_events"));
try testing.expectEqualStrings("write", try fx.text("SELECT subject_key FROM operational_events"));
try testing.expectEqualStrings("warning", try fx.text("SELECT severity FROM operational_events"));
try testing.expectEqual(@as(i64, 1), try fx.count("SELECT occurrences FROM operational_events"));
f.resolver.runPass(f.io(), &f.database, 1700000060);
try testing.expectEqual(@as(i64, 2), try fx.count("SELECT occurrences FROM operational_events"));
try f.database.exec("DROP TRIGGER refuse_update;");
f.resolver.runPass(f.io(), &f.database, 1700000120);
try testing.expectEqual(
@as(i64, 0),
try fx.count("SELECT count(*) FROM operational_events WHERE resolved_at IS NULL"),
);
}
test "a failing candidate select opens a read episode the next clean pass closes" {
var f: Fixture = undefined;
try fixture(&f, "168.192.in-addr.arpa");
defer f.deinit();
var fx: events_fixture.Fixture = .{};
try fx.init(f.io(), 1000);
defer fx.deinit();
f.resolver.diagnostics = &fx.store;
try clients_repo.upsertSeen(&f.database, "10.0.0.1", 1700000000);
try f.database.exec("ALTER TABLE clients RENAME TO clients_aside;");
f.resolver.runPass(f.io(), &f.database, 1700000000);
try testing.expectEqualStrings("read", try fx.text(
"SELECT subject_key FROM operational_events WHERE resolved_at IS NULL",
));
try f.database.exec("ALTER TABLE clients_aside RENAME TO clients;");
f.resolver.runPass(f.io(), &f.database, 1700000060);
try testing.expectEqual(
@as(i64, 0),
try fx.count("SELECT count(*) FROM operational_events WHERE resolved_at IS NULL"),
);
}
+118
View File
@@ -25,6 +25,7 @@ const client_names = @import("client_names.zig");
const clients_repo = @import("../storage/repositories/clients_repo.zig");
const db = @import("../storage/db.zig");
const disk_monitor = @import("../storage/disk_monitor.zig");
const events = @import("../storage/events.zig");
const logger = @import("../storage/logger.zig");
const log = std.log.scoped(.clients);
@@ -63,6 +64,9 @@ pub const Tracker = struct {
count: u32,
passes: u64,
stats: Stats,
/// Wired by the composition root after `init`, following the
/// `gate: ?*disk_monitor.Monitor` idiom. Null in every unit test here.
diagnostics: ?*events.Store = null,
/// `retention_days` is `logging.retention_days`, the same knob the query log
/// prunes by (milestone-7 ruling 16). A client silent for that long is as
@@ -185,6 +189,7 @@ pub const Tracker = struct {
var flushed: u64 = 0;
var failures: u64 = 0;
var last_failure: []const u8 = "";
for (batch) |entry| {
// `logger.max_client_len` is the RFC 5952 bound every address text
// in this program is sized by, so `format` cannot fail here.
@@ -199,9 +204,19 @@ pub const Tracker = struct {
log.warn("materialising client {s} failed: {s}", .{ w.buffered(), @errorName(err) });
}
failures += 1;
last_failure = @errorName(err);
}
}
// One aggregated outcome per pass, so the row's `occurrences` counts
// failing passes rather than failing addresses. A pass that wrote
// nothing resolves nothing: an empty batch is not evidence of success.
if (failures != 0) {
self.reportStorage(io, now_s, "materialise", "materialising a client failed", last_failure, failures);
} else if (flushed != 0) {
if (self.diagnostics) |store| store.resolve(io, now_s, .clients_storage, "materialise");
}
self.mutex.lockUncancelable(io);
self.passes += 1;
self.stats.flushed += flushed;
@@ -215,17 +230,39 @@ pub const Tracker = struct {
self.mutex.lockUncancelable(io);
self.stats.pruned += deleted;
self.mutex.unlock(io);
if (self.diagnostics) |store| store.resolve(io, now_s, .clients_storage, "prune");
} else |err| {
log.warn("pruning clients before {d} failed: {s}", .{ cutoff, @errorName(err) });
self.mutex.lockUncancelable(io);
self.stats.flush_failures += 1;
self.mutex.unlock(io);
self.reportStorage(io, now_s, "prune", "pruning stale clients failed", @errorName(err), 1);
}
}
if (names) |resolver| resolver.runPass(io, database, now_s);
}
/// One aggregated report per pass; `count` is how many failures it held.
fn reportStorage(
self: *Tracker,
io: std.Io,
now_s: i64,
operation: []const u8,
message: []const u8,
error_name: []const u8,
count: u64,
) void {
const store = self.diagnostics orelse return;
var buf: [events.Store.max_detail_len]u8 = undefined;
const detail = std.fmt.bufPrint(&buf, "{s}: {s} ({d} this pass)", .{
message,
error_name,
count,
}) catch buf[0..];
store.report(io, now_s, .clients_storage, operation, operation, .warning, detail);
}
pub fn snapshotStats(self: *Tracker, io: std.Io) Stats {
self.mutex.lockUncancelable(io);
defer self.mutex.unlock(io);
@@ -258,6 +295,7 @@ pub const Tracker = struct {
// tests
// ---------------------------------------------------------------------------
const events_fixture = @import("../storage/events_fixture.zig");
const migrations = @import("../storage/migrations.zig");
const testing = std.testing;
@@ -649,3 +687,83 @@ test "a gated pass attempts no naming either" {
try testing.expectEqual(@as(usize, 0), CountingExchange.calls);
try testing.expectEqual(@as(u64, 0), names.snapshotStats(io).attempted);
}
test "a failing materialise opens one episode per pass and a clean pass closes it" {
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
defer threaded.deinit();
const io = threaded.io();
var database = try openMigrated();
defer database.close();
var fx: events_fixture.Fixture = .{};
try fx.init(io, 1000);
defer fx.deinit();
var tracker: Tracker = .init(30);
tracker.diagnostics = &fx.store;
try database.exec(
\\CREATE TRIGGER refuse_insert BEFORE INSERT ON clients
\\BEGIN SELECT RAISE(ABORT, 'refused'); END;
);
_ = tracker.trackAt(io, parsed("192.168.1.10"), 1700000000);
_ = tracker.trackAt(io, parsed("192.168.1.11"), 1700000001);
tracker.flushOnce(io, &database, true, null);
try testing.expectEqual(@as(i64, 1), try fx.count("SELECT count(*) FROM operational_events"));
try testing.expectEqualStrings("clients.storage", try fx.text("SELECT code FROM operational_events"));
try testing.expectEqualStrings("materialise", try fx.text("SELECT subject_key FROM operational_events"));
try testing.expectEqual(@as(i64, 1), try fx.count("SELECT occurrences FROM operational_events"));
_ = tracker.trackAt(io, parsed("192.168.1.10"), 1700000060);
tracker.flushOnce(io, &database, true, null);
try testing.expectEqual(@as(i64, 2), try fx.count("SELECT occurrences FROM operational_events"));
try database.exec("DROP TRIGGER refuse_insert;");
_ = tracker.trackAt(io, parsed("192.168.1.10"), 1700000120);
tracker.flushOnce(io, &database, true, null);
try testing.expectEqual(
@as(i64, 0),
try fx.count("SELECT count(*) FROM operational_events WHERE resolved_at IS NULL"),
);
}
test "a failing prune opens its own episode the next due pass closes" {
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
defer threaded.deinit();
const io = threaded.io();
var database = try openMigrated();
defer database.close();
var fx: events_fixture.Fixture = .{};
try fx.init(io, 1000);
defer fx.deinit();
var tracker: Tracker = .init(30);
tracker.diagnostics = &fx.store;
// One pass short of due, so the pass below is the pruning one.
tracker.passes = Tracker.prune_every_passes - 1;
try database.exec(
\\CREATE TRIGGER refuse_delete BEFORE DELETE ON clients
\\BEGIN SELECT RAISE(ABORT, 'refused'); END;
);
try clients_repo.upsertSeen(&database, "192.168.1.10", 1);
tracker.flushOnce(io, &database, true, null);
try testing.expectEqualStrings("prune", try fx.text(
"SELECT subject_key FROM operational_events WHERE resolved_at IS NULL",
));
try database.exec("DROP TRIGGER refuse_delete;");
tracker.passes = Tracker.prune_every_passes - 1;
tracker.flushOnce(io, &database, true, null);
try testing.expectEqual(
@as(i64, 0),
try fx.count("SELECT count(*) FROM operational_events WHERE resolved_at IS NULL"),
);
}
+2
View File
@@ -592,11 +592,13 @@ const FailingUpstream = struct {
io: std.Io,
query: []const u8,
response_buf: []u8,
selected: *?[]const u8,
) transport.ExchangeError![]u8 {
_ = ptr;
_ = io;
_ = query;
_ = response_buf;
selected.* = "fake://failing-upstream";
return error.ConnectFailed;
}
+2
View File
@@ -351,8 +351,10 @@ const FakeUpstream = struct {
io: std.Io,
query: []const u8,
response_buf: []u8,
selected: *?[]const u8,
) transport.ExchangeError![]u8 {
_ = io;
selected.* = "fake://dot-server-upstream";
const self: *FakeUpstream = @ptrCast(@alignCast(ptr));
if (self.reply.len > response_buf.len) return error.ResponseTooLarge;
@memcpy(response_buf[0..self.reply.len], self.reply);
+1390 -183
View File
File diff suppressed because it is too large Load Diff

Some files were not shown because too many files have changed in this diff Show More