milestone 28: query provenance — every logged query is exactly explainable
Gates / frontend (push) Successful in 1m36s
Gates / test (push) Successful in 1m56s
Gates / test-aarch64 (push) Successful in 7m37s
Gates / package (push) Successful in 9m12s
Gates / container (push) Successful in 13s
CI / gates (push) Successful in 19m4s

query rows gain qclass, rcode, group, policy action and reason, the
matched rule or list entry with its source, cname and safe-search
targets, route kind, forward zone, and the resolver that actually
answered — the pool and local markers die. servfails are logged and
name the resolver that lost; post-parse protocol refusals become rows.
a detail page at /queries/:id renders the ordered explanation, and
coverage watermarks distinguish an empty history from a missing one.

the schema fingerprint changes: existing query history is recreated
with the old file kept aside and the reset filed as a resolved
diagnostic. fixes an oversized udp reply being rebuilt as noerror,
which handed clients a truncated nxdomain as success.
This commit is contained in:
2026-08-22 09:16:40 +02:00
parent 7e6cb507d2
commit 0fd6bbd312
65 changed files with 7036 additions and 685 deletions
+27 -6
View File
@@ -4,7 +4,7 @@ nxdns serves its admin API itself, on `web.bind:web.port` (default port 8080), a
The machine-readable contract is `src/web/openapi.yaml`, which the running server hands out unauthenticated at `GET /api/openapi.yaml`. Request and response schemas for every operation live there. When this page and the YAML disagree, the YAML wins.
The route table is `src/web/routes.zig`; the [Operations](#operations) table below carries all 60 of its entries.
The route table is `src/web/routes.zig`; the [Operations](#operations) table below carries all 61 of its entries.
## Conventions
@@ -45,7 +45,7 @@ A token bucket per client address: capacity and refill are both `web.api_rate_li
`GET /api/queries/live` is server-sent events over chunked transfer, `Content-Type: text/event-stream`, `Cache-Control: no-store`.
- The stream opens with `retry: 3000`, so a browser `EventSource` reconnects on its own after a drop.
- Each query is one frame: `event: query` and a single `data:` line of JSON. The payload carries the `GET /api/queries` row fields minus `id` (a live entry precedes persistence): `ts`, `domain`, `client_ip`, `qtype`, `blocked`, `block_reason`, `response_time_us`, `cache_hit`, `upstream`.
- Each query is one frame: `event: query` and a single `data:` line of JSON. The payload is the `Provenance` object — the body of `GET /api/queries/{id}` without its `id`, which does not exist yet because a live entry precedes its own insert. Its six groups are `request`, `group`, `policy`, `rewrites`, `route` and `response`.
- A `: ping` comment heartbeat goes out after 15 s of quiet, keeping middleboxes from reaping the idle connection.
- Each subscriber buffers up to 64 entries. A client too slow for the query rate overflows its buffer and the server ends the stream cleanly after delivering what the buffer held — queries are never held back for a slow reader. There is no gap marker: on reconnect, re-sync through `GET /api/queries`, which has the missed rows.
- Connections per client address are capped at `web.sse_max_connections_per_ip` (default 3); over the cap is a 429. The cap binds loopback too. The server holds at most 32 concurrent streams in total; when all slots are taken, the answer is a 503.
@@ -104,6 +104,7 @@ Auth `open` means no session is required; `session` means a valid session cookie
| POST | `/api/auth/login` | open | counted | runtime action | Log in |
| POST | `/api/auth/logout` | session | counted | runtime action | Log out |
| GET | `/api/queries` | session | counted | read | Query log page |
| GET | `/api/queries/{id}` | session | counted | read | One query, fully explained |
| GET | `/api/queries/live` | session | exempt | read | Live query stream (server-sent events) |
| GET | `/api/stats` | session | counted | read | Totals for a period |
| GET | `/api/stats/timeseries` | session | counted | read | Bucketed counts for a period |
@@ -174,9 +175,11 @@ In file mode `PUT /api/settings` is refused with the 403 above, password changes
Request and response schemas for every operation live in the OpenAPI document: `src/web/openapi.yaml` in the repository, or `GET /api/openapi.yaml` from a running server.
### Block reasons
### Policy reasons
Three places carry the same tag: `block_reason` on a `GET /api/queries` row, `block_reason` on a live-stream frame, and `reason` on a `GET /api/lookup` answer. The tag names the level that decided the query, and the levels are listed here in the order they are consulted — the first one that matches wins, so a rule always outranks a list.
Two places carry the same closed set of tags: `policy_reason` on a `GET /api/queries` row and on a `GET /api/queries/{id}` body (where it is `policy.reason`, and where the live stream sends the same field), and `reason` on a `GET /api/lookup` answer. The tag names what decided the query.
The first nine are the matcher's own verdicts, listed in the order they are consulted — the first that matches wins, so a rule always outranks a list.
| Tag | Decided by |
| --- | --- |
@@ -190,6 +193,24 @@ Three places carry the same tag: `block_reason` on a `GET /api/queries` row, `bl
| `blocklist_domain` | A plain name in a downloaded list |
| `blocklist_wildcard` | A domain anchor (`||name^`) in a downloaded list |
`/api/lookup` also answers `none` when nothing matched. A query row never carries `none`: `block_reason` is null unless the query was blocked.
The rest name a pipeline step that answered the query without consulting the matcher, and appear on a query row only.
A `cname:` prefix means the decision landed on a CNAME target rather than on the name the client asked for, so `cname:blocklist_domain` reads as "the list blocks a name this answer redirects to". Only `/api/queries` and the live stream show the prefix; `/api/lookup` does not follow CNAMEs.
| Tag | Decided by |
| --- | --- |
| `local_record` | A configured local record, answered before filtering |
| `forward_zone` | A configured forward zone, answered before filtering |
| `non_in_class` | The question was not class IN, so no rule could apply |
| `paused` | Filtering was paused |
| `snapshot_unavailable` | No filter snapshot was published yet, so the query went unfiltered |
| `no_match` | The matcher evaluated the name and nothing matched |
| `protocol_error` | A parsed request refused on protocol grounds — BADVERS, NOTIMP, a malformed EDNS OPT |
`policy_action` says which way the verdict went: `block`, `allow`, or `not_evaluated` for a query answered before any policy could apply. `/api/lookup` answers `none` when nothing matched, where a query row says `no_match`.
`route_kind` says where the answer came from: `blocked`, `local`, `forward_zone`, `upstream`, `cache` or `rejected`.
A non-empty `rewrites.cname_target` on a query detail means the decision landed on a CNAME target rather than on the name the client asked for; `policy.reason` is then the target's own reason. `/api/lookup` does not follow CNAMEs.
### Coverage
`GET /api/queries`, `GET /api/stats` and `GET /api/stats/timeseries` each answer with a `coverage` object: `available_since` is the oldest instant the query log is still complete for, and `complete` is true only when the window the request asked about starts at or after it. Retention deletes rows and advances the watermark in one transaction, so a client can tell an empty window from a pruned one instead of charting the gap as zero. A request with no lower bound at all asks about the whole of history, and is never complete.