milestone 28: query provenance — every logged query is exactly explainable
Gates / frontend (push) Successful in 1m36s
Gates / test (push) Successful in 1m56s
Gates / test-aarch64 (push) Successful in 7m37s
Gates / package (push) Successful in 9m12s
Gates / container (push) Successful in 13s
CI / gates (push) Successful in 19m4s

query rows gain qclass, rcode, group, policy action and reason, the
matched rule or list entry with its source, cname and safe-search
targets, route kind, forward zone, and the resolver that actually
answered — the pool and local markers die. servfails are logged and
name the resolver that lost; post-parse protocol refusals become rows.
a detail page at /queries/:id renders the ordered explanation, and
coverage watermarks distinguish an empty history from a missing one.

the schema fingerprint changes: existing query history is recreated
with the old file kept aside and the reset filed as a resolved
diagnostic. fixes an oversized udp reply being rebuilt as noerror,
which handed clients a truncated nxdomain as success.
This commit is contained in:
2026-08-22 09:16:40 +02:00
parent 7e6cb507d2
commit 0fd6bbd312
65 changed files with 7036 additions and 685 deletions
+218 -10
View File
@@ -226,14 +226,47 @@ paths:
"503":
$ref: "#/components/responses/Unavailable"
/api/queries/{id}:
get:
summary: One query, fully explained
description: |
The full provenance of one logged query: what was asked, which group's
policy applied, what that policy decided and matched on, what was
rewritten, where the answer came from, and what the client received.
Every field is a fact recorded when the query was answered, so a group
or blocklist renamed since keeps the name it had.
parameters:
- name: id
in: path
required: true
schema: { type: integer, minimum: 1 }
responses:
"200":
description: The query.
content:
application/json:
schema:
$ref: "#/components/schemas/QueryDetail"
"401":
$ref: "#/components/responses/Unauthorized"
"404":
$ref: "#/components/responses/NotFound"
"429":
$ref: "#/components/responses/RateLimited"
"500":
$ref: "#/components/responses/Internal"
"503":
$ref: "#/components/responses/Unavailable"
/api/queries/live:
get:
summary: Live query stream (server-sent events)
description: |
`text/event-stream`. The stream opens with `retry: 3000`, then sends
one `event: query` frame per resolved query whose `data:` line is a
JSON object with the `/api/queries` row fields minus `id` (the entry
precedes persistence). A `: ping` comment goes out every 15 seconds.
`Provenance` object the body of `/api/queries/{id}` without its `id`,
which does not exist yet because the entry precedes its own insert.
A `: ping` comment goes out every 15 seconds.
A client that falls more than 64 events behind is disconnected and
should re-sync via `/api/queries` after reconnecting. Connections
per address are capped by `web.sse_max_connections_per_ip`; the
@@ -1855,9 +1888,33 @@ components:
type: boolean
description: False when no password is configured; no cookie is set.
PolicyAction:
type: string
description: |
Whether the filtering policy reached a verdict. `not_evaluated` is the
honest answer for a query answered before filtering could apply, and is
not the same as `allow`.
enum: [not_evaluated, allow, block]
PolicyReason:
type: string
description: |
Why the policy landed where it did. The first nine are the matcher's own
verdicts; the rest name a pipeline step that decided without consulting
the matcher.
enum: [rule_allow_exact, rule_block_exact, rule_allow_wildcard, rule_block_wildcard, rule_allow_regex, rule_block_regex, blocklist_exception, blocklist_domain, blocklist_wildcard, local_record, forward_zone, non_in_class, paused, snapshot_unavailable, no_match, protocol_error]
RouteKind:
type: string
description: Where the answer the client received came from.
enum: [blocked, local, forward_zone, upstream, cache, rejected]
QueryRow:
type: object
required: [id, ts, domain, client_ip, qtype, blocked, block_reason, response_time_us, cache_hit, upstream]
description: |
The summary projection the query-log table scans. The full provenance of
a row is one request away at `/api/queries/{id}`.
required: [id, ts, domain, client_ip, qtype, qclass, rcode, blocked, response_time_us, cache_hit, upstream, policy_action, policy_reason, route_kind]
properties:
id: { type: integer }
ts:
@@ -1868,10 +1925,11 @@ components:
qtype:
type: integer
nullable: true
qclass: { type: integer }
rcode:
type: integer
description: The twelve-bit EDNS extended code, not the four header bits alone.
blocked: { type: boolean }
block_reason:
type: string
description: Empty when the query was not blocked.
response_time_us:
type: integer
nullable: true
@@ -1880,11 +1938,155 @@ components:
nullable: true
upstream:
type: string
description: Empty for cache hits and local answers.
description: Empty for cache hits, local answers and blocked queries.
policy_action:
$ref: "#/components/schemas/PolicyAction"
policy_reason:
$ref: "#/components/schemas/PolicyReason"
route_kind:
$ref: "#/components/schemas/RouteKind"
ProvenanceRequest:
type: object
required: [time, domain, client, qtype, qclass]
properties:
time:
type: integer
description: Unix seconds.
domain: { type: string }
client: { type: string }
qtype:
type: integer
nullable: true
qclass: { type: integer }
ProvenanceGroup:
type: object
description: |
The client's filtering group at the time of the query, as a historical
fact: the id may name a group since renamed or deleted.
required: [id, name]
properties:
id:
type: integer
nullable: true
name: { type: string }
ProvenancePolicy:
type: object
required: [action, reason, matched, source_id, source_name]
properties:
action:
$ref: "#/components/schemas/PolicyAction"
reason:
$ref: "#/components/schemas/PolicyReason"
matched:
type: string
description: The rule pattern or list entry that decided; empty when nothing matched.
source_id:
type: integer
nullable: true
source_name:
type: string
description: The blocklist the match came from; empty for a rule.
ProvenanceRewrites:
type: object
required: [cname_target, safe_search_target]
properties:
cname_target:
type: string
description: Set when the decision was made about a CNAME target rather than the queried name.
safe_search_target: { type: string }
ProvenanceRoute:
type: object
required: [kind, forward_zone, upstream]
properties:
kind:
$ref: "#/components/schemas/RouteKind"
forward_zone: { type: string }
upstream:
type: string
description: |
Non-empty only for an attempted upstream or forward-zone exchange,
including one that failed. Already redacted: the userinfo, path,
query and fragment of a resolver url never reach here.
ProvenanceResponse:
type: object
required: [rcode, duration_us]
properties:
rcode:
type: integer
description: The twelve-bit EDNS extended code, not the four header bits alone.
duration_us:
type: integer
nullable: true
Provenance:
type: object
description: |
One query, fully explained, in the order a query meets the pipeline. The
`data:` payload of a live-stream `event: query` frame is exactly this.
required: [request, group, policy, rewrites, route, response]
properties:
request:
$ref: "#/components/schemas/ProvenanceRequest"
group:
$ref: "#/components/schemas/ProvenanceGroup"
policy:
$ref: "#/components/schemas/ProvenancePolicy"
rewrites:
$ref: "#/components/schemas/ProvenanceRewrites"
route:
$ref: "#/components/schemas/ProvenanceRoute"
response:
$ref: "#/components/schemas/ProvenanceResponse"
QueryDetail:
type: object
description: |
`Provenance` plus the row id. Written out rather than composed with
`allOf` so the drift guard reads one property list per schema.
required: [id, request, group, policy, rewrites, route, response]
properties:
id: { type: integer }
request:
$ref: "#/components/schemas/ProvenanceRequest"
group:
$ref: "#/components/schemas/ProvenanceGroup"
policy:
$ref: "#/components/schemas/ProvenancePolicy"
rewrites:
$ref: "#/components/schemas/ProvenanceRewrites"
route:
$ref: "#/components/schemas/ProvenanceRoute"
response:
$ref: "#/components/schemas/ProvenanceResponse"
Coverage:
type: object
description: |
How much of the requested window the query log can still answer for.
Retention deletes rows and advances the watermark in one transaction, so
a client can tell an empty window from a pruned one instead of charting
the gap as zero.
required: [complete, available_since]
properties:
complete:
type: boolean
description: |
True only when the window's lower bound is at or after
`available_since`. A request with no lower bound asks about all of
history, which no file that has ever pruned can promise.
available_since:
type: integer
description: The oldest instant the file is complete for, unix seconds.
QueriesPage:
type: object
required: [queries, next_before]
required: [queries, next_before, coverage]
properties:
queries:
type: array
@@ -1894,6 +2096,8 @@ components:
type: integer
nullable: true
description: Cursor for the next page; null on the last page.
coverage:
$ref: "#/components/schemas/Coverage"
DiagnosticEvent:
type: object
@@ -1983,7 +2187,7 @@ components:
StatsTotals:
type: object
required: [period, since, until, queries, blocked, cached, clients, avg_response_time_us]
required: [period, since, until, queries, blocked, cached, clients, avg_response_time_us, coverage]
properties:
period:
type: string
@@ -2004,6 +2208,8 @@ components:
type: integer
nullable: true
description: Null when no query in the window recorded a time.
coverage:
$ref: "#/components/schemas/Coverage"
Bucket:
type: object
@@ -2018,7 +2224,7 @@ components:
StatsTimeseries:
type: object
required: [period, since, until, bucket_seconds, buckets]
required: [period, since, until, bucket_seconds, buckets, coverage]
properties:
period:
type: string
@@ -2030,6 +2236,8 @@ components:
type: array
items:
$ref: "#/components/schemas/Bucket"
coverage:
$ref: "#/components/schemas/Coverage"
Lookup:
type: object