nxdns/PLAN.md

nxdns — Implementation Plan v3.0 (Zig 0.16.0 Stable)

Source of truth for nxdns, a self-hosted DNS sinkhole written in Zig 0.16.0 stable. All stdlib claims in this document are verified against the 0.16.0 tag of the Zig repo (../zig).

There is no v1/v2 versioning. Scope is binary: a feature is in scope (and gets built) or out of scope (and does not). "Done" = everything in scope implemented, tested, documented.

1. Purpose

Fix three pain points in Pi-hole/AdGuard/Technitium:

1. Clunky, dated admin UI.
2. No config-as-code story (no way to export current state as a text artifact).
3. Silent operational failures (cloudflared proxy-dns incident: broken DoH setup + unbounded error logs filled SD card).

Serves a household LAN (≈2–20 devices). Portfolio-grade public repo with extensive docs, self-hosted on Gitea.

---

2. Scope

2.1 In Scope

• DNS server for LAN clients: UDP/53, TCP/53, DoH server, DoT server.
• Upstream resolution: DoH (HTTP/1.1), DoT.
• Local DNS records (A/AAAA/CNAME) + conditional forwarding (zone → designated resolver, plain UDP/TCP allowed).
• Domain filtering: blocklists (hosts/domains/ABP, including @@||name^ exception lines), custom rules (allow/block; exact, parent-walk, wildcard, regex), CNAME uncloaking (depth 8), per-group safe-search rewrite.
• DNS caching: positive + negative, in-memory only.
• Client/group model: IPv4 + IPv6 parity, per-client group assignment, per-group source assignments.
• Query logging + analytics: async batched writes to SQLite (WAL), retention cleanup, dashboard + time buckets, live SSE stream.
• Web app + REST API: LAN/Tailscale admin UI, optional password auth, OpenAPI schema + CI contract tests.
• Observability: upstream health API + UI, disk monitor with UI banner, bounded log rotation, Prometheus /metrics.
• Ops: config authority chosen by the invocation (database, or a file named by --config), nxdns export/import (ZON), scheduled + manual blocklist updates, TLS cert watcher + reload, auto-migration on upgrade, systemd service + Dockerfile + compose.

2.2 Out of Scope (permanent scope decisions, not deferrals)

• Regex from downloaded blocklists. A list is other people's code running on the household's DNS, and the engine exists for rules the operator wrote. Regex lines in blocklists stay counted and skipped, and the skip count stays surfaced in the UI. (Operator regex rules themselves are in scope as of milestone 21: the ReDoS objection that once ruled them out is answered by filter/regex.zig, a homegrown Pike VM whose running time is bounded by program length × name length by construction, with no dependency and no backtracking. It is reached only after every hash and wildcard level has missed, and only on a cache miss.)
• ABP syntax beyond domain anchors and @@||name^ exceptions. $dnstype, $dnsrewrite, $client, $denyallow and every browser modifier stay unsupported and counted; the one tolerated modifier is a $important suffix on an exception line, which changes nothing about where that exception lands.
• Partial-segment wildcards (ads*.example.com) as a rule kind. The regex kind covers the need without a second globbing dialect.
• DHCP server.
• DNSSEC validation (DO bit passthrough only).
• DoQ (QUIC), HTTP/2 upstream transport.
• Clustering / distributed state.
• Project website. The repository, its README and docs/ are the whole published surface.

---

3. Locked Decisions

3.1 Toolchain

• Compiler: Zig 0.16.0 stable. build.zig asserts major/minor.
• Targets: x86_64-linux-musl and aarch64-linux-musl, both first-class in CI. Release binaries are static musl — fully self-contained, no glibc version coupling on the Pi.

3.2 Concurrency: std.Io (Decision E)

Zig 0.16 stdlib is built on the std.Io interface: std.Io.net owns networking (std.net in its old form is gone) and std.http.Client requires an io: Io. Therefore:

• Io is the injected platform abstraction. Every component that does I/O takes io: Io. No project-owned wrapper interfaces around it — a second abstraction over an abstraction with one consumer is waste.
• Backend: std.Io.Threaded — the mature, debuggable path; at ≤20 devices throughput is a non-issue. The originally planned io_uring config flag was dropped in milestone 11: std.Io.Evented at Zig 0.16.0 stubs the networking a server needs — listen, accept, connect, lookup, and stream reads/writes return error.NetworkDown (Uring.zig netListenIp/netAccept/netConnectIp/netRead/netWrite) — so a selectable backend would boot a dead server. The code stays backend-agnostic by construction; revisit when std ships working evented networking.
• No hand-written thread pool. Io async/concurrent/Group covers task management.
• Core domain modules (dns, filter, cache) stay pure: no Io, no sockets — bytes in, bytes out. Only servers, upstream clients, and storage touch Io.

3.3 TLS (Decision D)

Verified: 0.16.0 ships std.crypto.tls.Client only. There is no server-side TLS in the stdlib.

• Upstream client TLS (DoH/DoT): stdlib std.crypto.tls via std.http.Client / direct. Maintained upstream, exercised by the stdlib itself.
• Local server TLS (DoH/DoT termination): vendored mbedTLS. Audited, embedded-focused, compiles cleanly with the Zig build system for both targets. We do not hand-roll a TLS server in a security-sensitive service.
• TLS glue lives in platform/tls_client.zig (stdlib wrapper: error classification, deadlines, retry policy) and platform/tls_server.zig (mbedTLS wrapper exposing std.Io.Reader/Writer so std.http.Server composes on top unchanged).

3.4 C Dependency Policy

• sqlite3 + mbedtls, both vendored and compiled from source in build.zig. Pinned versions, hermetic cross-compilation, no system libs.
• SQLite: amalgamation + our own thin wrapper (storage/db.zig: open/close, prepared-statement cache, typed row mapping, error translation). No third-party binding — the needed surface is small enough to own (Decision G).
• Zig package deps: minimized; anything adopted is pinned in build.zig.zon. No TOML/regex/HTTP packages needed under current decisions.

3.5 Config Format + Truth Model (Decision F)

• The invocation picks truth (see the next point). Config file format is ZON (std.zon parse + stringify — typed parsing into config structs, exact round-trip, stdlib-maintained, comments supported). No TOML: a third-party parser plus a hand-written serializer is two failure surfaces in the correctness-critical round-trip path, bought for syntax familiarity.
• The invocation picks the authority, and nothing else does (m20): nxdns run makes the database the configuration, nxdns run --config FILE makes the file the sole source and refuses the API routes that would edit configuration. There is no seeding and no first-start special case.
• nxdns export [--out file.zon] dumps DB state as canonical ZON. nxdns import <file.zon> validates + replaces DB contents (--force if the DB holds configuration; client rows materialised from traffic do not count, and survive the replacement. First-seen/last-seen are runtime state, not configuration: they follow the address, so an import never restamps a device the DB already knew). Export/import = backup + host migration, not upgrades (§3.7).
• No file watcher, no auto-regeneration.

3.6 Storage Layout (Decision H)

Two SQLite files with opposite write profiles, isolated from each other:

• config.db — small, precious, rarely written: groups, clients, prefixes, upstreams, blocklist source metadata, rules, local records, forward zones, settings, schema version.
• querylog.db — high-churn, large, expendable: query log + its own private domains dimension table. Client identity stored as IP text, not a FK into config — log rows are immutable facts and must not point at mutable config rows. If querylog.db is missing or corrupt at startup, rename aside, recreate, keep serving. Log loss is not an outage.
• No cross-DB references. Retention/VACUUM churn never touches config.db; config backup is a copy of a tiny file.

3.7 Upgrades: Auto-Migration (Decision J)

• config.db: numbered, sequential SQL migration steps compiled into the binary. At startup: read schema version row, apply newer steps inside a transaction, continue. Operator upgrade = install binary, restart. Before v0.1 the list holds one step — the baseline of §11.2, edited in place — because nxdns has no installs and a step exists only to reconcile a database somebody already has.
• querylog.db: no migrations. On schema mismatch: rename aside, recreate fresh.

3.8 Blocklist Storage (Decision A)

Blocklist domains are not stored in SQLite — they are a cache of re-downloadable remote artifacts, not config or state:

• Each source compiles to /var/lib/nxdns/blocklists/<source_id>.list: normalized, one domain per line, small header (source URL, fetch time, counts, checksum). Wildcard/regex/exception-flavored lines: wildcards go to <source_id>.wild, ABP exceptions (@@||name^) to <source_id>.allow; regex lines and browser-syntax lines nxdns cannot translate into a DNS decision are counted + skipped, both counts in metadata → UI. The checksum covers the three bodies in that order, each followed by a separator byte so that moving a name between bodies — an upstream switching a.example to *.a.example — changes the digest and forces a republish.
• config.db keeps source metadata only (blocklist_sources).
• Startup + post-update: parse files into the immutable in-memory matcher (RCU swap, §9.5).
• Corruption recovery is per-file: checksum mismatch → re-download one list.

3.9 Rule Model (Decision B)

Rule kinds: exact, parent-walk (implicit via candidate chain), wildcard (* segment patterns, e.g. *.doubleclick.net, ads.*.example.com), regex (the linear-time engine of filter/regex.zig, matched unanchored against the whole normalized name). Actions: allow | block. Kind is an explicit column, so a fourth kind widens one CHECK and touches no other table. A regex pattern is stored exactly as written — it is not a name, so it is never lowercased or dot-stripped — and is compiled at both edges: config/validate.zig refuses a bad one with the limit it hit, and filter/rules.zig compiles it once per snapshot.

3.10 Filtering Precedence

1. Exact/parent allow rules
2. Exact/parent block rules
3. Wildcard allow rules
4. Wildcard block rules
5. Regex allow rules
6. Regex block rules
7. Blocklist exceptions (@@||name^)
8. Blocklist domains
9. Blocklist wildcards

Tie-break at same specificity: allow wins.

Each level is checked against the whole candidate chain before the next level is checked against any, which is what makes an allow rule on a parent beat a block rule on the child.

Two positions carry an argument rather than a preference. The regex levels come last among the operator rules because they are the only ones that are not a set lookup or a label walk: a regex runs only once every cheaper level has missed. Blocklist exceptions come below every operator level because a downloaded list may cancel what another list blocked and must never cancel what the operator decided — no list can open an allow hole the operator did not open.

3.11 Network Posture

• Default web bind: LAN/Tailscale-friendly (non-loopback allowed).
• Auth optional (empty web.password disables it). Set → argon2id-hashed password, session cookie (HttpOnly, SameSite=Lax, Secure on HTTPS).
• API per-IP rate limit on by default.

3.12 Addressing

IPv4 + IPv6 full parity for: client identity, rate limiting, logging, group assignment, prefix-based group matching. Dual-stack assumed.

3.13 Filesystem Layout (FHS)

• /etc/nxdns/config.zon — the declarative source, read only when run --config names it. A file no flag names changes nothing.
• /var/lib/nxdns/config.db, /var/lib/nxdns/querylog.db
• /var/lib/nxdns/blocklists/*.list|*.wild|*.allow (plus *.raw.tmp|*.list.tmp|*.wild.tmp|*.allow.tmp during a refresh)
• /var/log/nxdns/nxdns.log — only in file output mode; default is stderr → journald.

3.14 Frontend Stack (Decision I)

• Vite + React + TypeScript + StyleX + React Aria; TanStack Router + TanStack Query. SPA, no SSR.
• Built assets embedded in the binary at compile time: single self-contained artifact, no asset-path config, no binary/UI skew. zig build accepts the dist path; the release pipeline runs npm run build first (CI always does).
• Dev-mode flag serves assets from disk so UI iteration needs no Zig rebuild.

3.15 CI

• Self-hosted Gitea + Gitea Actions runner.
• Every blocking check lives in one reusable workflow, .gitea/workflows/gates.yml. ci.yml calls it on master pushes and pull requests; release.yml calls the same file before it publishes, so a release cannot skip a check that CI runs.
• Jobs: zig build test, fuzz smoke, integration tests, OpenAPI contract tests (live server validated against openapi.yaml), frontend build and tests, dist + verify-dist for both targets, and a container job that builds the image and asserts its binaries are byte-identical to the packaged ones. aarch64 test execution via qemu-user if the runner is x86_64.

---

4. Architecture Overview

Client DNS Query
   |
   v
[UDP / TCP / DoH / DoT servers]        (std.Io; mbedTLS terminates DoH/DoT)
   |
   v
[Request Handler]
   |- rate limit check
   |- parse DNS packet
   |- group resolution (v4/v6 parity)
   |- local records check          -> answer locally
   |- forward-zone check           -> forward to designated resolver (bypasses filtering)
   |- allow/block evaluation (rules + blocklists, per group)
   |- safe-search rewrite (per group)
   |- cache lookup
   |- upstream query (DoH/DoT pool) + health tracking
   |- CNAME uncloaking check
   |- cache store
   |- async query log + SSE fanout
   v
Response to client

Cross-cutting: ConfigManager (load/reconcile/import/export/settings), BlocklistManager (fetch/compile/swap), Storage (two SQLite DBs), Cache, UpstreamHealth, DiskMonitor, Auth, Web API (REST + SSE + metrics).

---

5. Source Layout

src/
  main.zig               # Io backend instantiation, wiring
  app.zig
  version.zig

  platform/
    address.zig          # NetAddress: v4/v6 parity type + canonical key (was net_adapter)
    tls_client.zig       # stdlib TLS wrapper: deadlines, error classes, retry policy
    tls_server.zig       # mbedTLS wrapper exposing std.Io.Reader/Writer

  dns/                   # pure: bytes in, bytes out; no Io
    types.zig  header.zig  name.zig  question.zig  record.zig  edns.zig  packet.zig

  server/
    udp_server.zig  tcp_server.zig  doh_server.zig  dot_server.zig
    handler.zig  rate_limiter.zig  shutdown.zig

  upstream/
    doh_client.zig  dot_client.zig  pool.zig  health.zig

  filter/                # pure
    matcher.zig  rules.zig  wildcard.zig
    parser_hosts.zig  parser_domains.zig  parser_abp.zig
    fetcher.zig  compiler.zig            # list download -> compiled .list/.wild/.allow files
    regex.zig                            # linear-time Pike VM for operator regex rules
    safesearch.zig

  local/                 # pure
    records.zig          # local A/AAAA/CNAME answers
    forward_zones.zig    # zone -> designated resolver matching

  cache/
    dns_cache.zig

  storage/
    db.zig               # sqlite3 thin wrapper
    config_schema.zig  migrations.zig
    querylog_schema.zig
    repositories/
      clients_repo.zig  groups_repo.zig  rules_repo.zig  sources_repo.zig
      queries_repo.zig  settings_repo.zig  upstreams_repo.zig  local_repo.zig
    logger.zig  retention.zig  disk_monitor.zig

  config/
    model.zig  loader.zig  reconcile.zig  import.zig  export.zig    # all ZON via std.zon
    validate.zig  faults.zig

  web/
    server.zig  router.zig  auth.zig  sse.zig  static.zig  metrics.zig  openapi.zig
    handlers/
      auth.zig  stats.zig  queries.zig  clients.zig  groups.zig  blocklists.zig
      rules.zig  local.zig  lookup.zig  pause.zig  settings.zig
      upstream_health.zig  certs.zig  health.zig  version.zig

admin/                   # Vite + React + TS + StyleX + React Aria + TanStack
vendor/                  # sqlite3 amalgamation, mbedtls (pinned)
docs/                    # tutorial/ how-to/ reference/ explanation/ (Diátaxis)
tests/                   # dns/ integration/ fuzz/

---

6. DNS Behavior

6.1 Protocol

• RFC 1035 parse/encode. EDNS OPT passthrough. DO bit passthrough (no validation). ECS: strip (default) or forward.
• Process first question only. Preserve query ID and RD flag.
• Malformed: FORMERR when a response is possible; silent drop for severely truncated.

6.2 Blocked Response Modes

blocking.response: zero (A=0.0.0.0, AAAA=::) or nxdomain. TTL = blocking.ttl.

6.3 CNAME Uncloaking

Walk chain to depth 8; any target hitting block logic → synthesize blocked response.

6.4 Local Records

• Table-backed A/AAAA/CNAME records, group-independent. Matched before filtering.
• CNAME targets resolve through the normal pipeline (uncloaking rules apply).

6.5 Conditional Forwarding

• forward_zones: suffix match (lan.home, 10.in-addr.arpa, …) → designated resolver (udp://192.168.1.1:53 style; plain UDP/TCP permitted — these are local infra resolvers).
• Matching queries bypass filtering and blocklists; responses cached normally.

---

7. Filtering Engine

7.1 Evaluation

For {domain, group_id} (the qtype travels with the query for logging and response synthesis, not for matching):

1. Normalize: lowercase, trim trailing dot.
2. Build candidate chain (full, parent1, parent2, …).
3. Explicit rules per §3.10 precedence, allow before block at each level: exact rules against every candidate in the chain, then wildcard patterns and then regex patterns against the whole name (both kinds express their own reach, so neither walks the chain).
4. Group's blocklist exceptions (@@ entries), against every candidate in the chain. They cancel a block a list made and never one a rule made.
5. Group's blocklist domains (hash set over compiled lists), matched against the query name only.
6. Group's blocklist wildcards, matched against every proper parent of the query name.
7. No match → allow.

Blocklist domain entries do not parent-walk: they are matched against the query name alone. Wildcard entries match every proper parent, and exception entries walk the candidate chain the way rules do (§3.9), so @@||good.ads.example^ also lifts y.good.ads.example. ABP ||x.y^ emits both a domain entry x.y and a wildcard entry x.y, which together give domain-and-subdomains semantics.

7.2 Group Assignment

• Per source IP: (1) exact match in clients, (2) longest-prefix match in client_prefixes (ties: longer prefix, then priority), (3) default group.
• Auto-materialization: first query from an unseen IP inserts a clients row (hand_edited=0, first_seen=now) for UI visibility and stable group assignment. The query log does not FK to it (§3.6).
• Materialized clients name themselves: the tracker's flush pass sends one PTR query per unnamed row through the declared forward zones (§6.5), and the answer is runtime state in learned_name, never configuration.
• Retention drops hand_edited=0 clients with no queries in retention_days.

7.3 Reload

New immutable matcher built from DB + compiled list files → swap under an std.Io.RwLock with a generation counter. Readers take the shared lock for the microseconds of one evaluate; the writer takes the exclusive lock only for the swap, and the source status table is installed in the same critical section, so a failed reload publishes neither. (Deliberate deviation from "readers lock-free": freeing the old snapshot without a lock needs epoch-based reclamation, unjustifiable at household scale — see specs/milestone-5.md S8.3.)

7.4 Safe-Search

Per-group boolean. Rewrites known engine domains to their safe-search CNAME targets.

---

8. Cache

• Key: normalized qname + qtype + qclass + DO bit + forwarded ECS subnet (only when ecs_mode=forward).
• Value: full response bytes, stored_at, expires_at, positive/negative flag.
• Positive TTL: min(answer TTLs) bounded by sanity ceiling. Negative: SOA/minimum capped by cache.negative_ttl_max (0 disables).
• Hit path: clone bytes, overwrite transaction ID, decrement visible TTLs; TTL ≤ 0 → miss.
• Bounded by cache.size; CLOCK eviction; opportunistic + periodic expiry sweep.
• In-memory only, no persistence. Cold start refills within a minute; SD-corruption risk isn't worth pre-warmed entries.

---

9. Upstream Resolution

• Schemes: https://… → DoH, tls://host:853 → DoT.
• Ordered by priority; sequential attempt; per-upstream failure counters; exponential backoff with jitter; success resets.
• UpstreamHealth per upstream: last_success_at, last_error_at, last_error_message, rolling success rate, consecutive failures, backoff-until. This is routing state: it drives failover and backoff, and is exposed through /metrics and nxdns check. GET /api/upstream/health?period=… exposes none of it except the live enabled/available pair; its counts, success rate and last failure are ranged aggregates read from the per-minute upstream history in querylog.db, so the dashboard's period scopes them like every other number on the page.
• DoH client: std.http.Client with content-type/accept: application/dns-message; strict status + payload checks.
• platform/tls_client.zig enforces per-connection read/write deadlines, classifies TLS errors explicitly, retries with backoff. Integration tests cover timeout/hang scenarios so compiler upgrades can't silently regress them.
• Connect, read, and total-budget timeouts each configurable.

---

10. Rate Limiting

• DNS: default 1000 req / 60s per client IP (v4/v6 keyed alike); exceeded → REFUSED.
• API: per-IP token bucket, on by default; separate SSE connection cap per IP; localhost relaxed (configurable).
• Limiter maps bounded; stale-key windowed sweep.

---

11. Storage

11.1 Pragmas (both DBs)

journal_mode=WAL, synchronous=NORMAL, foreign_keys=ON, busy_timeout set.

11.2 config.db Schema (v1 baseline)

The DDL below is the live schema, kept byte-identical to src/storage/config_schema.zig. src/storage/migrations.zig carries it as its one and only step, so a database is at version 1 or it does not exist.

Until nxdns reaches v0.1 this baseline is editable: a schema change edits this section and config_schema.zig together and adds no migration step. nxdns has no installs, so there is no database for a step to reconcile. At v0.1 the baseline freezes and every later change becomes an append-only step.

CREATE TABLE schema_version (version INTEGER NOT NULL);

CREATE TABLE groups (
    id INTEGER PRIMARY KEY,
    name TEXT NOT NULL UNIQUE,
    safe_search INTEGER NOT NULL DEFAULT 0
);
INSERT OR IGNORE INTO groups (id, name) VALUES (1, 'default');

CREATE TABLE clients (
    id INTEGER PRIMARY KEY,
    ip TEXT NOT NULL UNIQUE,            -- canonical text form (v4 dotted / v6 RFC 5952)
    name TEXT,
    learned_name TEXT,
    name_attempt_after INTEGER NOT NULL DEFAULT 0,
    group_id INTEGER NOT NULL REFERENCES groups(id),
    hand_edited INTEGER NOT NULL DEFAULT 0,
    first_seen INTEGER NOT NULL,
    last_seen INTEGER NOT NULL
);

CREATE TABLE client_prefixes (
    id INTEGER PRIMARY KEY,
    prefix TEXT NOT NULL UNIQUE,        -- "192.168.1.0/24", "fd00:abcd::/48"
    group_id INTEGER NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
    priority INTEGER NOT NULL DEFAULT 100
);

CREATE TABLE upstreams (
    id INTEGER PRIMARY KEY,
    url TEXT NOT NULL UNIQUE,
    priority INTEGER NOT NULL DEFAULT 100,
    enabled INTEGER NOT NULL DEFAULT 1,
    tls_name TEXT NOT NULL DEFAULT ''  -- DoT verification name; empty verifies against the url host
);

CREATE TABLE blocklist_sources (
    id INTEGER PRIMARY KEY,
    url TEXT NOT NULL UNIQUE,
    name TEXT NOT NULL,
    enabled INTEGER NOT NULL DEFAULT 1,
    is_suggested INTEGER NOT NULL DEFAULT 0,
    last_updated INTEGER,
    domain_count INTEGER NOT NULL DEFAULT 0,
    wildcard_count INTEGER NOT NULL DEFAULT 0,
    exception_count INTEGER NOT NULL DEFAULT 0,
    skipped_regex_count INTEGER NOT NULL DEFAULT 0,
    skipped_unsupported_count INTEGER NOT NULL DEFAULT 0,
    checksum TEXT
);

CREATE TABLE group_sources (
    group_id INTEGER NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
    source_id INTEGER NOT NULL REFERENCES blocklist_sources(id) ON DELETE CASCADE,
    PRIMARY KEY (group_id, source_id)
);

CREATE TABLE rules (
    id INTEGER PRIMARY KEY,
    group_id INTEGER NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
    pattern TEXT NOT NULL,
    kind TEXT NOT NULL CHECK(kind IN ('exact','wildcard','regex')),
    action TEXT NOT NULL CHECK(action IN ('allow','block')),
    created_at INTEGER NOT NULL
);

CREATE TABLE local_records (
    id INTEGER PRIMARY KEY,
    name TEXT NOT NULL,
    rtype TEXT NOT NULL CHECK(rtype IN ('A','AAAA','CNAME')),
    value TEXT NOT NULL,
    ttl INTEGER NOT NULL DEFAULT 300,
    UNIQUE(name, rtype, value)
);

CREATE TABLE forward_zones (
    id INTEGER PRIMARY KEY,
    zone TEXT NOT NULL UNIQUE,
    resolver TEXT NOT NULL              -- "udp://192.168.1.1:53"
);

CREATE TABLE settings (key TEXT PRIMARY KEY, value TEXT NOT NULL);

CREATE TABLE operational_events (
    id INTEGER PRIMARY KEY,
    code TEXT NOT NULL,
    subject_key TEXT NOT NULL,
    subject_label TEXT NOT NULL,
    severity TEXT NOT NULL CHECK (severity IN ('warning', 'error')),
    first_seen INTEGER NOT NULL,
    last_seen INTEGER NOT NULL,
    occurrences INTEGER NOT NULL CHECK (occurrences > 0),
    resolved_at INTEGER,
    detail TEXT NOT NULL DEFAULT '',
    CHECK (resolved_at IS NULL OR resolved_at >= first_seen)
);
CREATE UNIQUE INDEX idx_operational_events_active
    ON operational_events(code, subject_key) WHERE resolved_at IS NULL;
CREATE INDEX idx_operational_events_last_seen
    ON operational_events(last_seen DESC);

operational_events is the one table here that is not configuration. It is the diagnostics log of src/storage/events.zig: one row per failure episode, opened on the first failure and resolved when the same subject succeeds again. It is deliberately absent from config_schema.table_names and config_schema.delete_order, so nxdns export never emits it and nxdns import never wipes it.

11.3 querylog.db Schema

CREATE TABLE domains (
    id INTEGER PRIMARY KEY,
    domain TEXT NOT NULL UNIQUE
);

CREATE TABLE query_log (
    id INTEGER PRIMARY KEY,
    timestamp INTEGER NOT NULL,
    domain_id INTEGER NOT NULL REFERENCES domains(id),
    client_ip TEXT NOT NULL,            -- text, not a FK: log rows are immutable facts
    qtype INTEGER,
    blocked INTEGER NOT NULL,
    block_reason TEXT,
    response_time_us INTEGER,
    cache_hit INTEGER,
    upstream TEXT
);
CREATE INDEX idx_query_log_ts ON query_log(timestamp);
CREATE INDEX idx_query_log_client ON query_log(client_ip);
CREATE INDEX idx_query_log_domain ON query_log(domain_id);

CREATE TABLE upstream_targets (
    id INTEGER PRIMARY KEY,
    url TEXT NOT NULL UNIQUE   -- the historical identity: config.db ids cannot cross database files
);

CREATE TABLE upstream_minute (
    upstream_id INTEGER NOT NULL REFERENCES upstream_targets(id),
    minute_ts INTEGER NOT NULL,
    successes INTEGER NOT NULL,
    failures INTEGER NOT NULL,
    last_failure_ts INTEGER,
    last_error TEXT,
    PRIMARY KEY (upstream_id, minute_ts),
    CHECK (successes >= 0),
    CHECK (failures >= 0)
) WITHOUT ROWID;
CREATE INDEX idx_upstream_minute_ts ON upstream_minute(minute_ts);

11.4 Query Logger

• In-memory buffer, mutex guarded, hard cap query_log_buffer_max (default 10000).
• Flush: batch size (100, comptime) or max interval query_log_flush_interval_s (default 60s, 0–3600, 0 = do not wait). One transaction per interval: at household query rates a per-query commit costs orders of magnitude more disk writes than the rows are worth. The interval is also roughly what a crash costs, while the writer is healthy and the disk gate is open — a gated or lock-delayed batch is older, so it is a normal case, not a bound.
• Privacy transforms (hide_domains / hide_client_ips) applied before persist + SSE fanout.
• Backpressure: buffer full → drop oldest unflushed entry, increment monotonic queries_dropped (exposed in /api/health + /metrics). SSE fanout precedes buffer insert, so live viewers still see dropped-from-persistence entries.

11.5 Retention

Periodic delete of rows older than retention_days; scheduled checkpoint/VACUUM on querylog.db only.

11.6 Disk Discipline (cloudflared lesson)

• Log output default: stderr (journald rotates). File mode: rotate at log.max_size_mb (default 50), keep log.max_files (default 5).
• DiskMonitor samples DB + log-dir sizes every 60s:
  • free < disk.warn_free_mb → UI banner + /api/health degraded.
  • free < disk.min_free_mb → stop non-essential writes (blocklist updates, log flushes); buffer continues under backpressure policy.
• Upstream error log lines deduplicated per-upstream at 1/minute.

---

12. Configuration

12.1 Config ZON Shape

The canonical shape is not duplicated here. It lives in docs/reference/configuration.md, which is handwritten against config/model.zig and only partly guarded (the drift test covers settings-key rows, not the whole shape, so a new collection can go undocumented while the guard stays green), and nxdns export emits it. A copy in this document is how §12.1 came to describe an .upstream.servers field that never existed and to omit the required .groups and .upstreams — a sample nobody could load. The skeleton, for orientation only:

.{
    .groups = .{ .{ .name = "default" } },
    .upstreams = .{ .{ .url = "https://dns.example/dns-query" } },
    .rules = .{
        .{ .group = "default", .pattern = "^ad[0-9]+-", .kind = .regex, .action = .block },
    },
}

12.2 Validation

At least one upstream; ports in range; resolver URLs parseable. nxdns check runs the validator, probes upstreams, and loads the cert+key pair of each enabled DoH/DoT server through the same CertStore.init the listeners use.

12.3 Settings Semantics

Scalars in settings(key, value); ordered/structured items in dedicated tables. Restart-required settings flagged; UI shows "restart to apply" banner.

---

13. Web / API

13.1 Endpoints

• POST /api/auth/login, POST /api/auth/logout
• GET /api/stats?period=…, GET /api/stats/timeseries?period=…
• GET /api/queries (filter + paginate), GET /api/queries/live (SSE, per-IP cap)
• GET/PUT /api/clients/{id}
• GET/POST/PUT/DELETE /api/groups…, /api/blocklists…, /api/rules…, /api/local-records…, /api/forward-zones…
• POST /api/blocklists/update
• GET /api/lookup?domain=…&group_id=…
• GET/POST /api/pause
• GET/PUT /api/settings
• GET /api/upstream/health?period=…
• POST /api/certs/reload
• GET /api/health — overall + disk + upstream + queries_dropped rollup
• GET /metrics — Prometheus text exposition: query counters (total/blocked/cached), per-upstream health, cache stats, queries_dropped, disk gauges
• GET /api/version, GET /api/openapi.yaml

13.2 OpenAPI

Hand-maintained openapi.yaml, served at GET /api/openapi.yaml and mirrored by hand in docs/reference/api.md. No renderer is vendored. Two drift guards fail the build instead: src/web/openapi.zig asserts every served route appears in the spec, and src/docs_drift_test.zig asserts every served route has its own table row in the reference page.

---

14. Frontend

Pages: Dashboard (stats + upstream health + disk), Query log, Live log, Clients, Groups, Blocklists, Rules, Local DNS (records + forward zones), Domain lookup, Settings.

Requirements: responsive desktop/mobile; route loaders for initial fetch; TanStack Query for cache/retries; error/loading states on every data view; works with auth enabled or disabled; restart-required banner.

---

15. CLI

• nxdns run — start the server.
• nxdns check — validate config, probe upstreams, load each enabled listener's certificate and verify its key pairs with it; exit 2 on failure, 0 with warnings.
• nxdns export [--out file.zon]
• nxdns import <file.zon> [--force]
• nxdns version — app version, Zig version string, git commit. No build date: the version and the commit identify a build exactly, and a date is one more input a reproducible build would have to pin.

---

16. Implementation Order

Phase 0 — Build Baseline

Scaffold tree; build.zig with 0.16 assertion, musl targets, vendored sqlite3 + mbedtls compiling; version plumbing; Gitea Actions workflows (test, integration, fuzz smoke, OpenAPI lint, frontend build). Exit: cross-compiled hello-world linking both C deps on both targets; CI green.

Phase 1 — Platform Layer

platform/address.zig (v4/v6 parity + canonical keys); platform/tls_client.zig (deadlines, error classes); platform/tls_server.zig (mbedTLS handshake → std.Io.Reader/Writer). Exit: UDP echo over std.Io; TLS client handshake against a real host; mbedTLS server terminating a loopback TLS connection.

Phase 2 — DNS Core

Types/header/name/question/record/packet; parser + encoder tests + fuzz target; EDNS + DO passthrough. Exit: unit + fuzz smoke pass.

Phase 3 — Resolver Transport

UDP server, TCP server, DoH + DoT upstream clients, pool + failover/backoff + health. Exit: A/AAAA forwarding over UDP + TCP; health populated.

Phase 4 — Storage + Config

SQLite wrapper; config.db schema + migration runner; querylog.db schema + recreate-on-mismatch; repositories; ZON loading + import/export; nxdns check. Exit: export → import round-trips byte-stable. (The ZON bootstrap this phase shipped was replaced in m20 by the two authority modes above.)

Phase 5 — Filtering + Local DNS

Rule matcher (exact/parent/wildcard; regex added in m21); blocklist parsers → compiled file format; fetcher + scheduled update; RCU swap; per-group safe-search; local records; forward zones. Exit: precedence table validated by tests; local zone answers + conditional forwards work.

Phase 6 — Cache + Rate Limit + Logging + Disk Monitor

TTL cache + reconstruction; v4/v6 rate limiter; async query logger + backpressure + retention; DiskMonitor + rotation + error-log dedup. Exit: disk thresholds trigger degradation + drop counters in integration test.

Phase 7 — Handler Integration

Full pipeline composition; CNAME uncloaking; pause/resume. Exit: end-to-end DNS flow with blocking, local records, cache, failover.

Phase 8 — Web / API / SSE / Auth / Metrics

HTTP server + router; handlers; SSE; optional auth; API rate limiting; /metrics; OpenAPI served + contract tests; embedded frontend + dev-mode disk serving. Exit: frontend fully drives config and operations; contract tests green.

Phase 9 — Local DoH/DoT Endpoints

DoH server + DoT server on platform/tls_server.zig; cert watcher + reload. Exit: LAN client resolves via DoH and DoT against local certs.

Phase 10 — Packaging + Ops + Docs

systemd unit (AmbientCapabilities=CAP_NET_BIND_SERVICE, hardened, writable /var/lib/nxdns + optional /var/log/nxdns); Dockerfile + compose (53/udp+tcp, 8080; mounts /etc/nxdns, /var/lib/nxdns); operator/architecture/config-reference/API docs. Exit: documented deployment works end-to-end on the Pi 5.

---

17. Testing Strategy

• Unit: DNS encode/decode; rule precedence + wildcard matcher; cache put/get/TTL rewrite; ZON loading + export round-trip; rate limiter; migration runner (fresh + stepwise upgrade).
• Fuzz: DNS parser malformed-packet fuzzing; blocklist parser fuzzing.
• Integration: UDP/TCP query path; blocked path; allow-over-block; wildcard precedence; CNAME uncloaking block; local records + forward zones; upstream failover/backoff/health; disk-full degradation; querylog.db corruption recovery; API CRUD; auth on/off; SSE; contract tests.
• Manual: dig @pi example.com / blocked domain / local record; DoH/DoT client checks; dashboard + live log.

---

18. Performance Targets

• Sustained ≥ 100 qps on Raspberry Pi 5.
• Blocklist lookup p95 < 1 ms.
• Cached response p95 < 5 ms.
• Memory with ~1M blocked domains < 100 MiB.
• Stripped static binary per arch: < 15,728,640 bytes with the embedded frontend assets, < 10,485,760 bytes without them. zig build verify-dist asserts both.

---

19. Security

• Sanitize all DNS and API inputs; prepared statements everywhere.
• argon2id password hashing; random session tokens with expiry.
• No secrets in logs. TLS keys readable by service user only.
• systemd hardening + least privilege.
• mbedTLS + sqlite3 versions pinned; upgrades are deliberate, reviewed bumps.

---

20. Publication

The project publishes released binaries and container images from its own Gitea instance. Building from source stays fully supported and documented; it is no longer the only path.

• Trigger. Pushing an annotated, GPG-signed tag vX.Y.Z to git.mial.net/mokhtar/nxdns. Nothing else publishes. Pre-release tags are rejected.
• Version. The tag is authoritative. build.zig.zon's .version must equal the tag, and the packaging gate asserts it. Nowhere else stores a version.
• Artifacts. Per architecture (x86_64-linux-musl, aarch64-linux-musl) a .tar.gz holding the stripped ReleaseSafe binary, LICENSE, THIRD-PARTY-NOTICES and the README. Plus one SHA256SUMS covering both tarballs and the image digest, and one detached SHA256SUMS.asc.
• Images. One multi-architecture image at git.mial.net/mokhtar/nxdns, tagged with the exact version and latest. No :edge. latest moves only forward. The builder stage runs on $BUILDPLATFORM, so no release build needs qemu.
• Signing. A GPG signing subkey held only by Gitea signs SHA256SUMS. The tag itself is signed by the primary key, and the release job pins that primary fingerprint before it does anything else. Verification is documented in docs/how-to/verify-a-release.md.
• Gates. release.yml runs gates.yml — the same file CI runs — and publishes nothing if any gate fails.
• Licensing. EUPL-1.2. THIRD-PARTY-NOTICES is generated from licenses/inventory.zon and a drift test fails when a dependency changes without a matching notice entry.
• Deferred. Bit-for-bit reproducibility across machines, and SBOM generation, are deliberate deferrals. See specs/milestone-14.md ruling 12.

---

21. Success Criteria

1. nxdns run starts cleanly on Zig 0.16.0 stable, static musl, both arches.
2. UDP + TCP resolution works; blocked domains return the configured response; precedence per §3.10.
3. Local records answer; conditional forwarding works.
4. Cache hit path returns valid ID-adjusted responses.
5. Upstream failover + backoff works; health in UI, API, /metrics.
6. Disk-fill degrades gracefully; no silent log-flood failure mode.
7. Web UI + API provide full admin functionality; OpenAPI contract tests green.
8. nxdns export round-trips via nxdns import.
9. Query logging, stats, SSE live stream work; querylog.db corruption self-heals.
10. Local DoH + DoT endpoints serve LAN clients.
11. Schema upgrade = install + restart (migration test proves it).
12. All suites green in Gitea CI for both targets.
13. Operator, architecture, config-reference, and API docs complete.

---

22. Working Notes

• dns/, filter/, local/, cache/ stay pure (no Io, no sockets). Servers, upstream clients, and storage take io: Io.
• Verify stdlib behavior against ../zig (tag 0.16.0) instead of memory — the std.Io migration invalidated older knowledge once already.
• Keep OpenAPI in sync with handlers (CI enforced).
• When a plan decision and reality diverge during build: surface 2–3 alternatives with tradeoffs, get a call, then execute.

Decision Log

#	Decision
A	Blocklists compile to flat files under /var/lib/nxdns/blocklists/; DB stores source metadata only
B	Rule kinds: exact, parent-walk, wildcard, regex (m21, own linear-time engine). Regex from downloaded lists stays out of scope
C	In scope: local DoH/DoT server, local records, conditional forwarding. Out: HTTP/2, DoQ, DHCP, DNSSEC, clustering
D	mbedTLS (vendored) terminates server TLS; stdlib TLS for upstream client
E	std.Io injected everywhere; Threaded backend (io_uring flag dropped in m11 — Evented networking is stubbed at 0.16.0); no custom thread pool
F	Config format: ZON via std.zon; DB is truth; export/import for backup + host moves
G	SQLite vendored amalgamation + own thin wrapper
H	Two DBs: config.db (precious) + querylog.db (expendable, self-contained, client IP as text)
I	Frontend embedded in binary; dev flag serves from disk; static musl release builds
J	Auto-migration for config.db at startup; querylog.db recreated on mismatch
—	Safe-search per-group; Prometheus /metrics in scope; CI on self-hosted Gitea Actions