From b609176a0b42b83c80248718ffb1a5c4c4db4b05 Mon Sep 17 00:00:00 2001 From: m5r Date: Fri, 31 Jul 2026 23:16:57 +0200 Subject: [PATCH] project plan, values, milestone 1 spec, zig 0.16 api research --- AGENTS.md | 37 ++ PLAN.md | 683 +++++++++++++++++++++++++++ specs/milestone-1.md | 290 ++++++++++++ specs/research/zig-0.16-api-notes.md | 138 ++++++ 4 files changed, 1148 insertions(+) create mode 100644 AGENTS.md create mode 100644 PLAN.md create mode 100644 specs/milestone-1.md create mode 100644 specs/research/zig-0.16-api-notes.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..34cb524 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,37 @@ +# AGENTS.md + +## Aim + +nxdns: a self-hosted DNS sinkhole for a household LAN, written in Zig 0.16.0. +Portfolio-grade public repo. PLAN.md is the source of truth for scope and design; +specs/ holds per-milestone contracts; specs/research/ holds verified stdlib facts. + +## Values + +We intentionally architect this code to be robust, maintainable, pragmatic — +good craftsmanship and good engineering. We explicitly avoid tech debt, code +smells, bad architecture decisions, and brittle implementations. + +What that means in practice: + +- This is a greenfield project. Breaking changes are allowed. Never keep a bad + interface for compatibility; fix it at the root. +- No versioning of scope. A feature is in scope (build it completely) or out of + scope (do not build it). No "v2 later", no stubs left behind. +- Fix root causes, not symptoms. Do not iterate on workarounds. +- Scope is small on purpose: household scale, two targets, few dependencies. + Do not add generality nobody asked for. +- Dependencies are liabilities: stdlib first; vendored + pinned C deps + (sqlite3, mbedTLS) only where the stdlib has nothing. +- Verify stdlib claims against ../zig at tag 0.16.0 — pre-0.16 knowledge is + stale (std.Io migration). See specs/research/zig-0.16-api-notes.md. +- Pure core: dns/, filter/, local/, cache/ take bytes and return bytes — no Io, + no sockets, no clocks hidden inside. +- Every failure mode must be visible: no silent drops, no unbounded logs, no + swallowed errors. Counters + health surfaces over log spam. +- Tests are runnable acceptance criteria, not decoration. Required CI stays + deterministic — no network-dependent tests in blocking jobs. +- Comments state constraints the code cannot show. No narration, no + commented-out code. +- Git: GPG-signed commits (`git commit -S`), simple lowercase messages, no + generated-by footers. diff --git a/PLAN.md b/PLAN.md new file mode 100644 index 0000000..10c3d83 --- /dev/null +++ b/PLAN.md @@ -0,0 +1,683 @@ +# nxdns — Implementation Plan v3.0 (Zig 0.16.0 Stable) + +Source of truth for **nxdns**, a self-hosted DNS sinkhole written in Zig 0.16.0 stable. +All stdlib claims in this document are verified against the `0.16.0` tag of the Zig repo (`../zig`). + +There is no v1/v2 versioning. Scope is binary: a feature is in scope (and gets built) or out of scope (and does not). "Done" = everything in scope implemented, tested, documented. + +## 1. Purpose + +Fix three pain points in Pi-hole/AdGuard/Technitium: +1. Clunky, dated admin UI. +2. No config-as-code story (no way to export current state as a text artifact). +3. Silent operational failures (cloudflared proxy-dns incident: broken DoH setup + unbounded error logs filled SD card). + +Serves a household LAN (≈2–20 devices). Portfolio-grade public repo with extensive docs, self-hosted on Gitea. + +--- + +## 2. Scope + +### 2.1 In Scope + +- DNS server for LAN clients: UDP/53, TCP/53, DoH server, DoT server. +- Upstream resolution: DoH (HTTP/1.1), DoT. +- Local DNS records (A/AAAA/CNAME) + conditional forwarding (zone → designated resolver, plain UDP/TCP allowed). +- Domain filtering: blocklists (hosts/domains/ABP), custom rules (allow/block; exact, parent-walk, wildcard), CNAME uncloaking (depth 8), per-group safe-search rewrite. +- DNS caching: positive + negative, in-memory only. +- Client/group model: IPv4 + IPv6 parity, per-client group assignment, per-group source assignments. +- Query logging + analytics: async batched writes to SQLite (WAL), retention cleanup, dashboard + time buckets, live SSE stream. +- Web app + REST API: LAN/Tailscale admin UI, optional password auth, OpenAPI schema + CI contract tests. +- Observability: upstream health API + UI, disk monitor with UI banner, bounded log rotation, Prometheus `/metrics`. +- Ops: DB-as-truth config, `nxdns export`/`import` (ZON), scheduled + manual blocklist updates, TLS cert watcher + reload, auto-migration on upgrade, systemd service + Dockerfile + compose. + +### 2.2 Out of Scope (permanent scope decisions, not deferrals) + +- Regex rules. Wildcards + parent-walk cover the real use cases; regex on the DNS hot path means ReDoS exposure plus an immature dependency or a homegrown engine. Regex lines in blocklists are counted, skipped, and the skip count is surfaced in the UI. +- DHCP server. +- DNSSEC validation (DO bit passthrough only). +- DoQ (QUIC), HTTP/2 upstream transport. +- Clustering / distributed state. +- Prebuilt binaries / published Docker images / project website. Repo + documented build-it-yourself path only. + +--- + +## 3. Locked Decisions + +### 3.1 Toolchain + +- Compiler: **Zig 0.16.0 stable**. `build.zig` asserts major/minor. +- Targets: **x86_64-linux-musl** and **aarch64-linux-musl**, both first-class in CI. Release binaries are **static musl** — fully self-contained, no glibc version coupling on the Pi. + +### 3.2 Concurrency: `std.Io` (Decision E) + +Zig 0.16 stdlib is built on the `std.Io` interface: `std.Io.net` owns networking (`std.net` in its old form is gone) and `std.http.Client` requires an `io: Io`. Therefore: + +- **`Io` is the injected platform abstraction.** Every component that does I/O takes `io: Io`. No project-owned wrapper interfaces around it — a second abstraction over an abstraction with one consumer is waste. +- **Backend: `std.Io.Threaded` by default** — the mature, debuggable path; at ≤20 devices throughput is a non-issue. `std.Io.Evented` (io_uring) is selectable via config flag; the code is backend-agnostic by construction, so this is a switch, not a refactor. +- No hand-written thread pool. `Io` async/concurrent/Group covers task management. +- Core domain modules (`dns`, `filter`, `cache`) stay pure: no `Io`, no sockets — bytes in, bytes out. Only servers, upstream clients, and storage touch `Io`. + +### 3.3 TLS (Decision D) + +Verified: 0.16.0 ships `std.crypto.tls.Client` only. There is no server-side TLS in the stdlib. + +- **Upstream client TLS (DoH/DoT): stdlib** `std.crypto.tls` via `std.http.Client` / direct. Maintained upstream, exercised by the stdlib itself. +- **Local server TLS (DoH/DoT termination): vendored mbedTLS.** Audited, embedded-focused, compiles cleanly with the Zig build system for both targets. We do not hand-roll a TLS server in a security-sensitive service. +- TLS glue lives in `platform/tls_client.zig` (stdlib wrapper: error classification, deadlines, retry policy) and `platform/tls_server.zig` (mbedTLS wrapper exposing `std.Io.Reader`/`Writer` so `std.http.Server` composes on top unchanged). + +### 3.4 C Dependency Policy + +- **`sqlite3` + `mbedtls`, both vendored** and compiled from source in `build.zig`. Pinned versions, hermetic cross-compilation, no system libs. +- SQLite: amalgamation + our own thin wrapper (`storage/db.zig`: open/close, prepared-statement cache, typed row mapping, error translation). No third-party binding — the needed surface is small enough to own (Decision G). +- Zig package deps: minimized; anything adopted is pinned in `build.zig.zon`. No TOML/regex/HTTP packages needed under current decisions. + +### 3.5 Config Format + Truth Model (Decision F) + +- **DB is truth. Config file format is ZON** (`std.zon` parse + stringify — typed parsing into config structs, exact round-trip, stdlib-maintained, comments supported). No TOML: a third-party parser plus a hand-written serializer is two failure surfaces in the correctness-critical bootstrap/round-trip path, bought for syntax familiarity. +- First start: if DB empty and `/etc/nxdns/config.zon` exists, validate → seed DB. Subsequent starts ignore the file. +- `nxdns export [--out file.zon]` dumps DB state as canonical ZON. `nxdns import ` validates + replaces DB contents (`--force` if DB non-empty). Export/import = backup + host migration, **not** upgrades (§3.7). +- No file watcher, no auto-regeneration. + +### 3.6 Storage Layout (Decision H) + +Two SQLite files with opposite write profiles, isolated from each other: + +- **`config.db`** — small, precious, rarely written: groups, clients, prefixes, upstreams, blocklist source metadata, rules, local records, forward zones, settings, schema version. +- **`querylog.db`** — high-churn, large, expendable: query log + its own private `domains` dimension table. Client identity stored as **IP text**, not a FK into config — log rows are immutable facts and must not point at mutable config rows. If `querylog.db` is missing or corrupt at startup, rename aside, recreate, keep serving. Log loss is not an outage. +- No cross-DB references. Retention/VACUUM churn never touches `config.db`; config backup is a copy of a tiny file. + +### 3.7 Upgrades: Auto-Migration (Decision J) + +- `config.db`: numbered, sequential SQL migration steps compiled into the binary. At startup: read schema version row, apply newer steps inside a transaction, continue. Operator upgrade = install binary, restart. +- `querylog.db`: **no migrations.** On schema mismatch: rename aside, recreate fresh. + +### 3.8 Blocklist Storage (Decision A) + +Blocklist domains are **not** stored in SQLite — they are a cache of re-downloadable remote artifacts, not config or state: + +- Each source compiles to `/var/lib/nxdns/blocklists/.list`: normalized, one domain per line, small header (source URL, fetch time, count, checksum). Wildcard/regex-flavored lines: wildcards go to `.wild`; regex lines are counted + skipped (count in metadata → UI). +- `config.db` keeps source **metadata only** (`blocklist_sources`). +- Startup + post-update: parse files into the immutable in-memory matcher (RCU swap, §9.5). +- Corruption recovery is per-file: checksum mismatch → re-download one list. + +### 3.9 Rule Model (Decision B) + +Rule kinds: `exact`, parent-walk (implicit via candidate chain), `wildcard` (`*` segment patterns, e.g. `*.doubleclick.net`, `ads.*.example.com`). Actions: `allow` | `block`. Kind is an explicit column — the model is extensible without breakage, but regex stays out of scope (§2.2). + +### 3.10 Filtering Precedence + +1. Exact/parent **allow** rules +2. Exact/parent **block** rules +3. Wildcard allow rules +4. Wildcard block rules +5. Blocklist domains +6. Blocklist wildcards + +Tie-break at same specificity: **allow wins**. + +### 3.11 Network Posture + +- Default web bind: LAN/Tailscale-friendly (non-loopback allowed). +- Auth optional (empty `web.password` disables it). Set → argon2id-hashed password, session cookie (`HttpOnly`, `SameSite=Lax`, `Secure` on HTTPS). +- API per-IP rate limit on by default. + +### 3.12 Addressing + +IPv4 + IPv6 full parity for: client identity, rate limiting, logging, group assignment, prefix-based group matching. Dual-stack assumed. + +### 3.13 Filesystem Layout (FHS) + +- `/etc/nxdns/config.zon` — bootstrap (first start only). +- `/var/lib/nxdns/config.db`, `/var/lib/nxdns/querylog.db` +- `/var/lib/nxdns/blocklists/*.list|*.wild` +- `/var/log/nxdns/nxdns.log` — only in file output mode; default is stderr → journald. + +### 3.14 Frontend Stack (Decision I) + +- Vite + React + TypeScript + Tailwind; TanStack Router + TanStack Query. SPA, no SSR. +- **Built assets embedded in the binary** at compile time: single self-contained artifact, no asset-path config, no binary/UI skew. `zig build` accepts the dist path; the release pipeline runs `npm run build` first (CI always does). +- Dev-mode flag serves assets from disk so UI iteration needs no Zig rebuild. + +### 3.15 CI + +- Self-hosted **Gitea + Gitea Actions runner**. +- Jobs: `zig build test`, fuzz smoke, integration tests, OpenAPI contract tests (live server validated against `openapi.yaml`), frontend build, cross-compile both targets. aarch64 test execution via qemu-user if the runner is x86_64. + +--- + +## 4. Architecture Overview + +``` +Client DNS Query + | + v +[UDP / TCP / DoH / DoT servers] (std.Io; mbedTLS terminates DoH/DoT) + | + v +[Request Handler] + |- rate limit check + |- parse DNS packet + |- group resolution (v4/v6 parity) + |- local records check -> answer locally + |- forward-zone check -> forward to designated resolver (bypasses filtering) + |- allow/block evaluation (rules + blocklists, per group) + |- safe-search rewrite (per group) + |- cache lookup + |- upstream query (DoH/DoT pool) + health tracking + |- CNAME uncloaking check + |- cache store + |- async query log + SSE fanout + v +Response to client +``` + +Cross-cutting: `ConfigManager` (bootstrap/import/export/settings), `BlocklistManager` (fetch/compile/swap), `Storage` (two SQLite DBs), `Cache`, `UpstreamHealth`, `DiskMonitor`, `Auth`, `Web API` (REST + SSE + metrics). + +--- + +## 5. Source Layout + +``` +src/ + main.zig # Io backend instantiation, wiring + app.zig + version.zig + + platform/ + address.zig # NetAddress: v4/v6 parity type + canonical key (was net_adapter) + tls_client.zig # stdlib TLS wrapper: deadlines, error classes, retry policy + tls_server.zig # mbedTLS wrapper exposing std.Io.Reader/Writer + + dns/ # pure: bytes in, bytes out; no Io + types.zig header.zig name.zig question.zig record.zig edns.zig packet.zig + + server/ + udp_server.zig tcp_server.zig doh_server.zig dot_server.zig + handler.zig rate_limiter.zig shutdown.zig + + upstream/ + doh_client.zig dot_client.zig pool.zig health.zig + + filter/ # pure + matcher.zig rules.zig wildcard.zig + parser_hosts.zig parser_domains.zig parser_abp.zig + fetcher.zig compiler.zig # list download -> compiled .list/.wild files + safesearch.zig + + local/ # pure + records.zig # local A/AAAA/CNAME answers + forward_zones.zig # zone -> designated resolver matching + + cache/ + dns_cache.zig + + storage/ + db.zig # sqlite3 thin wrapper + config_schema.zig migrations.zig + querylog_schema.zig + repositories/ + clients_repo.zig groups_repo.zig rules_repo.zig sources_repo.zig + queries_repo.zig settings_repo.zig upstreams_repo.zig local_repo.zig + logger.zig retention.zig disk_monitor.zig + + config/ + model.zig bootstrap.zig import.zig export.zig validate.zig # all ZON via std.zon + + web/ + server.zig router.zig auth.zig sse.zig static.zig metrics.zig openapi.zig + handlers/ + auth.zig stats.zig queries.zig clients.zig groups.zig blocklists.zig + rules.zig local.zig lookup.zig pause.zig settings.zig + upstream_health.zig certs.zig health.zig version.zig + +web/ # Vite + React + TS + Tailwind + TanStack +vendor/ # sqlite3 amalgamation, mbedtls (pinned) +docs/ # operator/ architecture/ config-reference/ api/ +tests/ # dns/ integration/ fuzz/ +``` + +--- + +## 6. DNS Behavior + +### 6.1 Protocol + +- RFC 1035 parse/encode. EDNS OPT passthrough. DO bit passthrough (no validation). ECS: `strip` (default) or `forward`. +- Process first question only. Preserve query ID and RD flag. +- Malformed: `FORMERR` when a response is possible; silent drop for severely truncated. + +### 6.2 Blocked Response Modes + +`blocking.response`: `zero` (A=`0.0.0.0`, AAAA=`::`) or `nxdomain`. TTL = `blocking.ttl`. + +### 6.3 CNAME Uncloaking + +Walk chain to depth 8; any target hitting block logic → synthesize blocked response. + +### 6.4 Local Records + +- Table-backed A/AAAA/CNAME records, group-independent. Matched before filtering. +- CNAME targets resolve through the normal pipeline (uncloaking rules apply). + +### 6.5 Conditional Forwarding + +- `forward_zones`: suffix match (`lan.home`, `10.in-addr.arpa`, …) → designated resolver (`udp://192.168.1.1:53` style; plain UDP/TCP permitted — these are local infra resolvers). +- Matching queries bypass filtering and blocklists; responses cached normally. + +--- + +## 7. Filtering Engine + +### 7.1 Evaluation + +For `{domain, qtype, group_id}`: +1. Normalize: lowercase, trim trailing dot. +2. Build candidate chain (full, parent1, parent2, …). +3. Explicit rules per §3.10 precedence. +4. Group's blocklist domains (hash set over compiled lists). +5. Group's blocklist wildcards. +6. No match → allow. + +### 7.2 Group Assignment + +- Per source IP: (1) exact match in `clients`, (2) longest-prefix match in `client_prefixes` (ties: longer prefix, then priority), (3) `default` group. +- Auto-materialization: first query from an unseen IP inserts a `clients` row (`hand_edited=0`, `first_seen=now`) for UI visibility and stable group assignment. The query log does **not** FK to it (§3.6). +- Retention drops `hand_edited=0` clients with no queries in `retention_days`. + +### 7.3 Reload + +New immutable matcher built from DB + compiled list files → atomic pointer swap (RCU, generation counter). Readers lock-free. + +### 7.4 Safe-Search + +Per-group boolean. Rewrites known engine domains to their safe-search CNAME targets. + +--- + +## 8. Cache + +- Key: normalized qname + qtype + qclass + DO bit + forwarded ECS subnet (only when `ecs_mode=forward`). +- Value: full response bytes, `stored_at`, `expires_at`, positive/negative flag. +- Positive TTL: `min(answer TTLs)` bounded by sanity ceiling. Negative: SOA/minimum capped by `cache.negative_ttl_max` (0 disables). +- Hit path: clone bytes, overwrite transaction ID, decrement visible TTLs; TTL ≤ 0 → miss. +- Bounded by `cache.size`; CLOCK eviction; opportunistic + periodic expiry sweep. +- **In-memory only, no persistence.** Cold start refills within a minute; SD-corruption risk isn't worth pre-warmed entries. + +--- + +## 9. Upstream Resolution + +- Schemes: `https://…` → DoH, `tls://host:853` → DoT. +- Ordered by priority; sequential attempt; per-upstream failure counters; exponential backoff with jitter; success resets. +- `UpstreamHealth` per upstream: last_success_at, last_error_at, last_error_message, rolling success rate, consecutive failures, backoff-until. Exposed via `GET /api/upstream/health`, dashboard, `/metrics`, and `nxdns check`. +- DoH client: `std.http.Client` with `content-type/accept: application/dns-message`; strict status + payload checks. +- `platform/tls_client.zig` enforces per-connection read/write deadlines, classifies TLS errors explicitly, retries with backoff. Integration tests cover timeout/hang scenarios so compiler upgrades can't silently regress them. +- Connect, read, and total-budget timeouts each configurable. + +--- + +## 10. Rate Limiting + +- DNS: default 1000 req / 60s per client IP (v4/v6 keyed alike); exceeded → `REFUSED`. +- API: per-IP token bucket, on by default; separate SSE connection cap per IP; localhost relaxed (configurable). +- Limiter maps bounded; stale-key windowed sweep. + +--- + +## 11. Storage + +### 11.1 Pragmas (both DBs) + +`journal_mode=WAL`, `synchronous=NORMAL`, `foreign_keys=ON`, `busy_timeout` set. + +### 11.2 config.db Schema + +```sql +CREATE TABLE schema_version (version INTEGER NOT NULL); + +CREATE TABLE groups ( + id INTEGER PRIMARY KEY, + name TEXT NOT NULL UNIQUE, + safe_search INTEGER NOT NULL DEFAULT 0 +); +INSERT OR IGNORE INTO groups (id, name) VALUES (1, 'default'); + +CREATE TABLE clients ( + id INTEGER PRIMARY KEY, + ip TEXT NOT NULL UNIQUE, -- canonical text form (v4 dotted / v6 RFC 5952) + name TEXT, + group_id INTEGER NOT NULL REFERENCES groups(id), + hand_edited INTEGER NOT NULL DEFAULT 0, + first_seen INTEGER NOT NULL, + last_seen INTEGER NOT NULL +); + +CREATE TABLE client_prefixes ( + id INTEGER PRIMARY KEY, + prefix TEXT NOT NULL UNIQUE, -- "192.168.1.0/24", "fd00:abcd::/48" + group_id INTEGER NOT NULL REFERENCES groups(id) ON DELETE CASCADE, + priority INTEGER NOT NULL DEFAULT 100 +); + +CREATE TABLE upstreams ( + id INTEGER PRIMARY KEY, + url TEXT NOT NULL UNIQUE, + priority INTEGER NOT NULL DEFAULT 100, + enabled INTEGER NOT NULL DEFAULT 1 +); + +CREATE TABLE blocklist_sources ( + id INTEGER PRIMARY KEY, + url TEXT NOT NULL UNIQUE, + name TEXT NOT NULL, + enabled INTEGER NOT NULL DEFAULT 1, + is_suggested INTEGER NOT NULL DEFAULT 0, + last_updated INTEGER, + domain_count INTEGER NOT NULL DEFAULT 0, + wildcard_count INTEGER NOT NULL DEFAULT 0, + skipped_regex_count INTEGER NOT NULL DEFAULT 0, + checksum TEXT +); + +CREATE TABLE group_sources ( + group_id INTEGER NOT NULL REFERENCES groups(id) ON DELETE CASCADE, + source_id INTEGER NOT NULL REFERENCES blocklist_sources(id) ON DELETE CASCADE, + PRIMARY KEY (group_id, source_id) +); + +CREATE TABLE rules ( + id INTEGER PRIMARY KEY, + group_id INTEGER NOT NULL REFERENCES groups(id) ON DELETE CASCADE, + pattern TEXT NOT NULL, + kind TEXT NOT NULL CHECK(kind IN ('exact','wildcard')), + action TEXT NOT NULL CHECK(action IN ('allow','block')), + created_at INTEGER NOT NULL +); + +CREATE TABLE local_records ( + id INTEGER PRIMARY KEY, + name TEXT NOT NULL, + rtype TEXT NOT NULL CHECK(rtype IN ('A','AAAA','CNAME')), + value TEXT NOT NULL, + ttl INTEGER NOT NULL DEFAULT 300, + UNIQUE(name, rtype, value) +); + +CREATE TABLE forward_zones ( + id INTEGER PRIMARY KEY, + zone TEXT NOT NULL UNIQUE, + resolver TEXT NOT NULL -- "udp://192.168.1.1:53" +); + +CREATE TABLE settings (key TEXT PRIMARY KEY, value TEXT NOT NULL); +``` + +### 11.3 querylog.db Schema + +```sql +CREATE TABLE domains ( + id INTEGER PRIMARY KEY, + domain TEXT NOT NULL UNIQUE +); + +CREATE TABLE query_log ( + id INTEGER PRIMARY KEY, + timestamp INTEGER NOT NULL, + domain_id INTEGER NOT NULL REFERENCES domains(id), + client_ip TEXT NOT NULL, -- text, not a FK: log rows are immutable facts + qtype INTEGER, + blocked INTEGER NOT NULL, + block_reason TEXT, + response_time_us INTEGER, + cache_hit INTEGER, + upstream TEXT +); +CREATE INDEX idx_query_log_ts ON query_log(timestamp); +CREATE INDEX idx_query_log_client ON query_log(client_ip); +CREATE INDEX idx_query_log_domain ON query_log(domain_id); +``` + +### 11.4 Query Logger + +- In-memory buffer, mutex guarded, hard cap `query_log_buffer_max` (default 10000). +- Flush: batch size (default 100) or max interval (default 100ms). +- Privacy transforms (hide_domains / hide_client_ips) applied before persist + SSE fanout. +- Backpressure: buffer full → drop oldest unflushed entry, increment monotonic `queries_dropped` (exposed in `/api/health` + `/metrics`). SSE fanout precedes buffer insert, so live viewers still see dropped-from-persistence entries. + +### 11.5 Retention + +Periodic delete of rows older than `retention_days`; scheduled checkpoint/VACUUM on `querylog.db` only. + +### 11.6 Disk Discipline (cloudflared lesson) + +- Log output default: **stderr** (journald rotates). File mode: rotate at `log.max_size_mb` (default 50), keep `log.max_files` (default 5). +- `DiskMonitor` samples DB + log-dir sizes every 60s: + - free < `disk.warn_free_mb` → UI banner + `/api/health` degraded. + - free < `disk.min_free_mb` → stop non-essential writes (blocklist updates, log flushes); buffer continues under backpressure policy. +- Upstream error log lines deduplicated per-upstream at 1/minute. + +--- + +## 12. Configuration + +### 12.1 Bootstrap ZON Shape + +```zon +.{ + .upstream = .{ + .servers = .{ "https://cloudflare-dns.com/dns-query", "tls://dns.google:853" }, + .connect_timeout_ms = 2000, + .read_timeout_ms = 3000, + }, + .dns = .{ + .bind_ipv4 = "0.0.0.0", + .bind_ipv6 = "::", + .port = 53, + .rate_limit = 1000, + .rate_window_seconds = 60, + }, + .blocking = .{ .response = .zero, .ttl = 5 }, // .zero | .nxdomain + .cache = .{ .size = 10000, .negative_ttl_max = 3600 }, + .web = .{ + .enabled = true, + .bind = "0.0.0.0", + .port = 8080, + .password = "", // empty => auth disabled + .session_ttl_hours = 24, + .api_rate_limit_per_min = 300, + .sse_max_connections_per_ip = 3, + }, + .doh_server = .{ .enabled = false, .bind = "0.0.0.0", .port = 443, + .cert_path = "/etc/nxdns/cert.pem", .key_path = "/etc/nxdns/key.pem" }, + .dot_server = .{ .enabled = false, .bind = "0.0.0.0", .port = 853, + .cert_path = "/etc/nxdns/cert.pem", .key_path = "/etc/nxdns/key.pem" }, + .edns = .{ .ecs_mode = .strip }, // .strip | .forward + .local_records = .{ .{ .name = "nas.lan", .rtype = .A, .value = "192.168.1.10" } }, + .forward_zones = .{ .{ .zone = "lan.home", .resolver = "udp://192.168.1.1:53" } }, + .logging = .{ + .level = .info, + .retention_days = 30, + .query_log_buffer_max = 10000, + .hide_domains = false, + .hide_client_ips = false, + .output = .stderr, // .stderr | .syslog | .file + .file_path = "/var/log/nxdns/nxdns.log", + .max_size_mb = 50, + .max_files = 5, + }, + .disk = .{ .min_free_mb = 200, .warn_free_mb = 500 }, + .blocklist_update = .{ .enabled = true, .interval_hours = 24 }, +} +``` + +### 12.2 Validation + +At least one upstream; ports in range; cert+key readable if DoH/DoT server enabled; resolver URLs parseable. `nxdns check` runs the validator + probes upstreams. + +### 12.3 Settings Semantics + +Scalars in `settings(key, value)`; ordered/structured items in dedicated tables. Restart-required settings flagged; UI shows "restart to apply" banner. + +--- + +## 13. Web / API + +### 13.1 Endpoints + +- `POST /api/auth/login`, `POST /api/auth/logout` +- `GET /api/stats?period=…`, `GET /api/stats/timeseries?period=…` +- `GET /api/queries` (filter + paginate), `GET /api/queries/live` (SSE, per-IP cap) +- `GET/PUT /api/clients/{id}` +- `GET/POST/PUT/DELETE /api/groups…`, `/api/blocklists…`, `/api/rules…`, `/api/local-records…`, `/api/forward-zones…` +- `POST /api/blocklists/update` +- `GET /api/lookup?domain=…&group_id=…` +- `GET/POST /api/pause` +- `GET/PUT /api/settings` +- `GET /api/upstream/health` +- `POST /api/certs/reload` +- `GET /api/health` — overall + disk + upstream + queries_dropped rollup +- `GET /metrics` — Prometheus text exposition: query counters (total/blocked/cached), per-upstream health, cache stats, queries_dropped, disk gauges +- `GET /api/version`, `GET /api/openapi.yaml` + +### 13.2 OpenAPI + +Hand-maintained `openapi.yaml`, served + rendered into `docs/api/`. CI contract tests spin up a seeded server, hit every documented endpoint, and validate bodies/status/auth against the schema. Drift fails CI. + +--- + +## 14. Frontend + +Pages: Dashboard (stats + upstream health + disk), Query log, Live log, Clients, Groups, Blocklists, Rules, Local DNS (records + forward zones), Domain lookup, Settings. + +Requirements: responsive desktop/mobile; route loaders for initial fetch; TanStack Query for cache/retries; error/loading states on every data view; works with auth enabled or disabled; restart-required banner. + +--- + +## 15. CLI + +- `nxdns run` — start the server. +- `nxdns check` — validate config, probe upstreams, test cert readability; nonzero exit on failure. +- `nxdns export [--out file.zon]` +- `nxdns import [--force]` +- `nxdns version` — app version, Zig version string, build date, git commit. + +--- + +## 16. Implementation Order + +### Phase 0 — Build Baseline +Scaffold tree; `build.zig` with 0.16 assertion, musl targets, vendored sqlite3 + mbedtls compiling; version plumbing; Gitea Actions workflows (test, integration, fuzz smoke, OpenAPI lint, frontend build). +Exit: cross-compiled hello-world linking both C deps on both targets; CI green. + +### Phase 1 — Platform Layer +`platform/address.zig` (v4/v6 parity + canonical keys); `platform/tls_client.zig` (deadlines, error classes); `platform/tls_server.zig` (mbedTLS handshake → `std.Io.Reader`/`Writer`). +Exit: UDP echo over `std.Io`; TLS client handshake against a real host; mbedTLS server terminating a loopback TLS connection. + +### Phase 2 — DNS Core +Types/header/name/question/record/packet; parser + encoder tests + fuzz target; EDNS + DO passthrough. +Exit: unit + fuzz smoke pass. + +### Phase 3 — Resolver Transport +UDP server, TCP server, DoH + DoT upstream clients, pool + failover/backoff + health. +Exit: A/AAAA forwarding over UDP + TCP; health populated. + +### Phase 4 — Storage + Config +SQLite wrapper; config.db schema + migration runner; querylog.db schema + recreate-on-mismatch; repositories; ZON bootstrap + import/export; `nxdns check`. +Exit: first start seeds DB from ZON; export → import round-trips byte-stable. + +### Phase 5 — Filtering + Local DNS +Rule matcher (exact/parent/wildcard); blocklist parsers → compiled file format; fetcher + scheduled update; RCU swap; per-group safe-search; local records; forward zones. +Exit: precedence table validated by tests; local zone answers + conditional forwards work. + +### Phase 6 — Cache + Rate Limit + Logging + Disk Monitor +TTL cache + reconstruction; v4/v6 rate limiter; async query logger + backpressure + retention; DiskMonitor + rotation + error-log dedup. +Exit: disk thresholds trigger degradation + drop counters in integration test. + +### Phase 7 — Handler Integration +Full pipeline composition; CNAME uncloaking; pause/resume. +Exit: end-to-end DNS flow with blocking, local records, cache, failover. + +### Phase 8 — Web / API / SSE / Auth / Metrics +HTTP server + router; handlers; SSE; optional auth; API rate limiting; `/metrics`; OpenAPI served + contract tests; embedded frontend + dev-mode disk serving. +Exit: frontend fully drives config and operations; contract tests green. + +### Phase 9 — Local DoH/DoT Endpoints +DoH server + DoT server on `platform/tls_server.zig`; cert watcher + reload. +Exit: LAN client resolves via DoH and DoT against local certs. + +### Phase 10 — Packaging + Ops + Docs +systemd unit (`AmbientCapabilities=CAP_NET_BIND_SERVICE`, hardened, writable `/var/lib/nxdns` + optional `/var/log/nxdns`); Dockerfile + compose (53/udp+tcp, 8080; mounts `/etc/nxdns`, `/var/lib/nxdns`); operator/architecture/config-reference/API docs. +Exit: documented deployment works end-to-end on the Pi 5. + +--- + +## 17. Testing Strategy + +- **Unit**: DNS encode/decode; rule precedence + wildcard matcher; cache put/get/TTL rewrite; ZON bootstrap + export round-trip; rate limiter; migration runner (fresh + stepwise upgrade). +- **Fuzz**: DNS parser malformed-packet fuzzing; blocklist parser fuzzing. +- **Integration**: UDP/TCP query path; blocked path; allow-over-block; wildcard precedence; CNAME uncloaking block; local records + forward zones; upstream failover/backoff/health; disk-full degradation; querylog.db corruption recovery; API CRUD; auth on/off; SSE; contract tests. +- **Manual**: `dig @pi example.com` / blocked domain / local record; DoH/DoT client checks; dashboard + live log. + +--- + +## 18. Performance Targets + +- Sustained ≥ 100 qps on Raspberry Pi 5. +- Blocklist lookup p95 < 1 ms. +- Cached response p95 < 5 ms. +- Memory with ~1M blocked domains < 100 MB. +- Stripped static binary < 10 MB per arch (excluding embedded frontend assets; < 15 MB with them). + +--- + +## 19. Security + +- Sanitize all DNS and API inputs; prepared statements everywhere. +- argon2id password hashing; random session tokens with expiry. +- No secrets in logs. TLS keys readable by service user only. +- systemd hardening + least privilege. +- mbedTLS + sqlite3 versions pinned; upgrades are deliberate, reviewed bumps. + +--- + +## 20. Success Criteria + +1. `nxdns run` starts cleanly on Zig 0.16.0 stable, static musl, both arches. +2. UDP + TCP resolution works; blocked domains return the configured response; precedence per §3.10. +3. Local records answer; conditional forwarding works. +4. Cache hit path returns valid ID-adjusted responses. +5. Upstream failover + backoff works; health in UI, API, `/metrics`. +6. Disk-fill degrades gracefully; no silent log-flood failure mode. +7. Web UI + API provide full admin functionality; OpenAPI contract tests green. +8. `nxdns export` round-trips via `nxdns import`. +9. Query logging, stats, SSE live stream work; querylog.db corruption self-heals. +10. Local DoH + DoT endpoints serve LAN clients. +11. Schema upgrade = install + restart (migration test proves it). +12. All suites green in Gitea CI for both targets. +13. Operator, architecture, config-reference, and API docs complete. + +--- + +## 21. Working Notes + +- `dns/`, `filter/`, `local/`, `cache/` stay pure (no `Io`, no sockets). Servers, upstream clients, and storage take `io: Io`. +- Verify stdlib behavior against `../zig` (tag 0.16.0) instead of memory — the std.Io migration invalidated older knowledge once already. +- Keep OpenAPI in sync with handlers (CI enforced). +- When a plan decision and reality diverge during build: surface 2–3 alternatives with tradeoffs, get a call, then execute. + +## Decision Log + +| # | Decision | +|---|----------| +| A | Blocklists compile to flat files under `/var/lib/nxdns/blocklists/`; DB stores source metadata only | +| B | Rule kinds: exact, parent-walk, wildcard. Regex permanently out of scope | +| C | In scope: local DoH/DoT server, local records, conditional forwarding. Out: HTTP/2, DoQ, DHCP, DNSSEC, clustering | +| D | mbedTLS (vendored) terminates server TLS; stdlib TLS for upstream client | +| E | `std.Io` injected everywhere; `Threaded` default backend, io_uring via flag; no custom thread pool | +| F | Config format: ZON via `std.zon`; DB is truth; export/import for backup + host moves | +| G | SQLite vendored amalgamation + own thin wrapper | +| H | Two DBs: `config.db` (precious) + `querylog.db` (expendable, self-contained, client IP as text) | +| I | Frontend embedded in binary; dev flag serves from disk; static musl release builds | +| J | Auto-migration for `config.db` at startup; `querylog.db` recreated on mismatch | +| — | Safe-search per-group; Prometheus `/metrics` in scope; CI on self-hosted Gitea Actions | diff --git a/specs/milestone-1.md b/specs/milestone-1.md new file mode 100644 index 0000000..8765317 --- /dev/null +++ b/specs/milestone-1.md @@ -0,0 +1,290 @@ +# Milestone 1: Build Baseline + Platform Layer + +Goal: `zig build test` green natively; static musl executables for x86_64-linux and aarch64-linux linking pinned sqlite3 + mbedTLS; platform modules (address, tls_client, tls_server) implemented and tested; Gitea CI green. No DNS logic in this milestone. + +Read first: `AGENTS.md` (values), `specs/research/zig-0.16-api-notes.md` (verified stdlib facts — pre-0.16 API knowledge is stale and MUST NOT be used). The Zig source of truth is `/home/mokhtar/app/zig` at tag `0.16.0`. + +## Sessions + +Five sessions. S1 runs first, alone. S2, S3, S4, S5 run in parallel after S1 is verified. The orchestrator (not any session) wires `src/tests.zig` imports and any `build.zig` additions afterward. + +``` +S1 ──> { S2, S3, S4, S5 } ──> orchestrator integration +``` + +--- + +## Session S1: Build Baseline + +### S1.1 build.zig.zon + +- `.name = .nxdns`, `.version = "0.1.0"`, `.minimum_zig_version = "0.16.0"`, `.paths = .{""}`, `.fingerprint` (compiler suggests the value on first build — accept it). +- Dependencies added with `zig fetch --save= ` so the content hash is pinned: + - `sqlite`: the newest stable SQLite **amalgamation zip** from sqlite.org (check https://sqlite.org/download.html for the current one; record the version in a comment). + - `mbedtls`: the newest **mbedTLS 3.6.x LTS** release tarball from the Mbed-TLS GitHub releases (3.6 line only — not 4.x). + +### S1.2 build.zig + +- Comptime guard: `@import("builtin").zig_version` major==0 and minor==16, else `@compileError`. +- Options: `-Dintegration` (bool, default false) exposed to tests via a `build_options` module (`b.addOptions()`); also embed version string + git commit (`b.option([]const u8, "version-string", ...)` defaulting to "0.1.0-dev") for `src/version.zig`. +- C static libs, one per dependency, built with `b.addLibrary(.{ .linkage = .static, ... })` (NOT addStaticLibrary — it does not exist in 0.16): + - `sqlite3`: compile `sqlite3.c` from the sqlite dependency via `lib.root_module.addCSourceFile`; `addIncludePath` the dep root. Flags: `-DSQLITE_ENABLE_FTS5`, `-DSQLITE_THREADSAFE=1`, `-DSQLITE_DEFAULT_WAL_SYNCHRONOUS=1`, `-DSQLITE_OMIT_LOAD_EXTENSION`. + - `mbedtls`: compile every `library/*.c` file via `root_module.addCSourceFiles(.{ .root = dep.path("library"), .files = ... })` (paths must be relative — absolute paths panic) plus the `3rdparty/everest` and `3rdparty/p256m` sources; `addIncludePath` for `include`, `library`, and both 3rdparty include dirs. Default `mbedtls_config.h`, no custom config in this milestone. +- Executable `nxdns`: root module `src/main.zig`, `link_libc = true`, links both libs. Native artifact + installed. +- Test step `test`: `b.addTest` on a module rooted at `src/tests.zig`, importing `build_options`, linking both C libs. +- Step `cross`: for `x86_64-linux-musl` and `aarch64-linux-musl` (via `b.resolveTargetQuery`), build the exe with `exe.linkage = .static`, install to `zig-out/cross//nxdns`. +- `.gitignore`: `.zig-cache/`, `zig-out/`. + +### S1.3 src/main.zig, src/version.zig, src/tests.zig + +- `src/version.zig`: `pub const string` / `pub const zig_version_string` pulled from `build_options`. +- `src/main.zig`: parse first CLI arg. `version` → print version + zig version, exit 0. `run`, `check`, `export`, `import` → print `not implemented`, exit 2. No arg / unknown → usage on stderr, exit 64. No allocator gymnastics — `std.process.args` is enough here. +- `src/tests.zig`: `comptime { _ = @import("main.zig"); _ = @import("version.zig"); }` plus `test` block asserting the sqlite3 and mbedTLS C headers link: call `sqlite3_libversion()` and `mbedtls_version_get_string_full()` via `@cImport`-free extern declarations (declare the two extern fns manually) and check non-empty results. + +### S1.4 Acceptance Criteria + +- [ ] `zig build test` exits 0 natively; the C-link test prints/asserts both library version strings. +- [ ] `zig build cross` exits 0; `file zig-out/cross/x86_64-linux-musl/nxdns` and the aarch64 one both report "statically linked". +- [ ] `./zig-out/bin/nxdns version` prints the version and Zig 0.16.0, exit 0. +- [ ] `./zig-out/bin/nxdns` (no args) exits 64 with usage on stderr. +- [ ] `git status` clean of build artifacts (gitignore works). + +--- + +## Session S2: platform/address.zig + +Pure module. No `std.Io` operations — only type conversions to/from `std.Io.net.IpAddress` values. Unit tests in-file. + +### S2.1 Types + API + +```zig +pub const NetAddress = union(enum) { + ip4: [4]u8, + ip6: [16]u8, + + pub const Key = [17]u8; // tag byte (4 or 6) + address bytes, zero-padded for ip4 + + pub fn parse(text: []const u8) error{InvalidAddress}!NetAddress; // "1.2.3.4", "fd00::1"; no port, no brackets + pub fn format(self: NetAddress, w: *std.Io.Writer) std.Io.Writer.Error!void; // v4 dotted; v6 RFC 5952 lowercase, :: compression + pub fn key(self: NetAddress) Key; + pub fn fromIp(addr: std.Io.net.IpAddress) NetAddress; // drops port; IPv4-mapped IPv6 (::ffff:a.b.c.d) normalizes to .ip4 + pub fn toIp(self: NetAddress, port: u16) std.Io.net.IpAddress; + pub fn eql(a: NetAddress, b: NetAddress) bool; +}; + +pub const Prefix = struct { + addr: NetAddress, // host bits zeroed on parse + bits: u8, // <= 32 for ip4, <= 128 for ip6 + + pub fn parse(text: []const u8) error{InvalidPrefix}!Prefix; // "192.168.1.0/24", "fd00:abcd::/48" + pub fn contains(self: Prefix, addr: NetAddress) bool; // family mismatch => false + pub fn format(self: Prefix, w: *std.Io.Writer) std.Io.Writer.Error!void; +}; + +/// Longest-prefix winner; ties broken by lower `priority` value. Returns null when nothing matches. +pub fn matchLongest(comptime T: type, entries: []const T, addr: NetAddress) ?*const T; +// T must have fields: prefix: Prefix, priority: i64 +``` + +Implement parse/format by hand or delegate to `std.Io.net.Ip4Address/Ip6Address` parsing where it fits — but the RFC 5952 output rules (lowercase hex, longest zero-run compressed, no compression of a single group) must hold and be tested either way. + +### S2.2 Acceptance Criteria + +- [ ] Round-trip tests: parse→format is identity for canonical inputs (`"192.168.1.1"`, `"fd00::1"`, `"::"`, `"2001:db8::8:800:200c:417a"`). +- [ ] RFC 5952 tests: `"2001:0DB8:0:0:1::1"` formats as `"2001:db8::1:0:0:0:1"`-style rules — specifically: longest run compressed, single zero group NOT compressed, lowercase. +- [ ] `fromIp` on an IPv4-mapped IPv6 address yields `.ip4`. +- [ ] `Prefix.contains`: `192.168.1.0/24` contains `.1.5`, not `.2.5`; `/0` contains everything of its family; family mismatch false. +- [ ] `matchLongest`: `/24` beats `/16`; equal bits → lower priority value wins. +- [ ] All tests pass via `zig test src/platform/address.zig` (orchestrator wires them into `zig build test` later). + +--- + +## Session S3: platform/tls_client.zig + +Wrapper over `std.crypto.tls.Client` for upstream DoT (and reused by anything needing client TLS over a `std.Io.net.Stream`). See the API notes §std.crypto.tls.Client for exact init requirements. + +### S3.1 API + +```zig +pub const ErrorClass = enum { handshake, certificate, io, protocol }; + +pub fn classify(err: anyerror) ErrorClass; + +pub const TlsStream = struct { + // all fields private in practice; struct is pinned: MUST NOT move after init + // (tls.Client holds its reader/writer by value) + + pub const Options = struct { + host: []const u8, // SNI + verification name + ca: enum { system, insecure_skip_verify }, + // buffers supplied by caller; read_buffer.len >= std.crypto.tls.Client.min_buffer_len + read_buffer: []u8, + write_buffer: []u8, + stream_read_buffer: []u8, + stream_write_buffer: []u8, + }; + + /// In-place init (pinned struct). `bundle` is scanned lazily for .system via + /// std.crypto.Certificate.Bundle.rescan when empty; caller owns bundle + lock lifetime. + pub fn init( + self: *TlsStream, + io: std.Io, + stream: *std.Io.net.Stream, + bundle: *std.crypto.Certificate.Bundle, + bundle_lock: *std.Io.RwLock, + gpa: std.mem.Allocator, + options: Options, + ) InitError!void; + + pub fn reader(self: *TlsStream) *std.Io.Reader; // plaintext + pub fn writer(self: *TlsStream) *std.Io.Writer; // plaintext + pub fn close(self: *TlsStream) void; // close_notify via Client.end, errors swallowed to log-level +}; +``` + +- Entropy: 240 bytes via `io.random`; `realtime_now` via the Io clock (`io.now(.real)` — check the exact 0.16 name in `/home/mokhtar/app/zig/lib/std/Io.zig` before use). +- `classify` maps the stdlib error sets: cert/trust errors → `.certificate`; handshake alerts/negotiation → `.handshake`; ReadFailed/WriteFailed/connection errors → `.io`; everything else → `.protocol`. Write the mapping exhaustively over `std.crypto.tls.Client.InitError` — a `switch` with explicit arms, no `else => .protocol` catch-all for that set. + +### S3.2 Tests + +- Unit: `classify` mapping table (pick 6+ representative errors across the four classes). +- Integration (compiled only when `build_options.integration`): connect to a live host (`cloudflare-dns.com:853` — DoT port, TLS without ALPN), complete the handshake with `.ca = .system`, close cleanly. This test is NOT part of the default `zig build test` run and NOT part of blocking CI. + +### S3.3 Acceptance Criteria + +- [ ] `zig test src/platform/tls_client.zig` passes (unit tests only). +- [ ] `classify` covers `InitError` exhaustively (compiles with explicit arms — adding a new stdlib error breaks the build here, by design). +- [ ] Integration test compiles under `-Dintegration` (orchestrator runs it after wiring; a live-network failure is an environment finding, not a session failure). + +--- + +## Session S4: platform/tls_server.zig + +mbedTLS-backed server-side TLS termination exposing `std.Io.Reader`/`std.Io.Writer`, so `std.http.Server` and the DoT server can sit on top of any accepted TCP stream. + +### S4.1 mbedTLS extern layer + +Declare the needed mbedTLS API as extern fns/opaque types in this file (no `@cImport` — keep translate-c out of the build). Needed surface: `mbedtls_ssl_context`, `mbedtls_ssl_config`, `mbedtls_x509_crt`, `mbedtls_pk_context`, `mbedtls_entropy_context`, `mbedtls_ctr_drbg_context` + their init/free/setup/parse functions, `mbedtls_ssl_handshake`, `mbedtls_ssl_read`, `mbedtls_ssl_write`, `mbedtls_ssl_close_notify`, `mbedtls_ssl_set_bio`, `mbedtls_strerror`. Sizes: allocate contexts with the C sizes via opaque + `extern` allocation pattern — simplest correct approach: define `extern struct` mirrors is NOT acceptable (fragile); instead heap-allocate via wrapper C-callable `malloc(sizeof)` is also not available — so: declare the context structs as `opaque` and allocate them with `gpa.alignedAlloc(u8, .of(usize), mbedtls_ssl_context_size)` where the sizes come from a tiny C shim file `src/platform/mbedtls_shim.c` exporting `size_t nx_sizeof_ssl_context(void)` etc. The shim is owned by this session and added to build by the orchestrator (S4 must NOT edit build.zig — note the shim path in the completion report). + +### S4.2 API + +```zig +pub const ServerContext = struct { + // holds parsed cert chain + key + ssl_config + drbg; one per listener, reused across connections + pub fn init(gpa: std.mem.Allocator, cert_pem: [:0]const u8, key_pem: [:0]const u8) InitError!ServerContext; + pub fn deinit(self: *ServerContext, gpa: std.mem.Allocator) void; +}; + +pub const ServerStream = struct { + // pinned after accept(); owns the ssl_context for one connection + + /// Performs the TLS handshake over an accepted TCP stream. + /// BIO callbacks bridge mbedtls_ssl_read/write to stream.reader/writer interfaces. + pub fn accept( + self: *ServerStream, + gpa: std.mem.Allocator, + ctx: *ServerContext, + io: std.Io, + stream: *std.Io.net.Stream, + read_buffer: []u8, + write_buffer: []u8, + ) AcceptError!void; + + pub fn reader(self: *ServerStream) *std.Io.Reader; // plaintext, implemented via Io.Reader vtable over mbedtls_ssl_read + pub fn writer(self: *ServerStream) *std.Io.Writer; // plaintext, via mbedtls_ssl_write + pub fn close(self: *ServerStream, gpa: std.mem.Allocator) void; // close_notify + free ssl_context +}; +``` + +- Error mapping: negative mbedTLS return codes → Zig error set with named errors for the common cases (`CertParse`, `KeyParse`, `HandshakeFailed`, `PeerClosed`, `WantReadWrite` handled internally); include the raw code in a log via `mbedtls_strerror`. +- `MBEDTLS_ERR_SSL_WANT_READ/WANT_WRITE` loop inside read/write — never surfaces to callers. + +### S4.3 Test fixture + loopback test + +- Generate once and commit: `tests/fixtures/self_signed_cert.pem` + `self_signed_key.pem` (openssl, EC P-256, CN=localhost, SAN DNS:localhost + IP:127.0.0.1, 100-year validity — a fixture, not a secret; note "test fixture, private key intentionally committed" in a `tests/fixtures/README.md`). +- Loopback test (compiled only under `build_options.integration`): thread A: `IpAddress.listen` on 127.0.0.1:0 → accept → `ServerStream.accept` with the fixture → echo one message read back to the writer. Thread B (client): `std.crypto.tls.Client` with `.host = .no_verification`, `.ca = .no_verification` → write message → read echo → assert equality → clean close both sides. Drive both with `io.concurrent` on one `Threaded` instance. + +### S4.4 Acceptance Criteria + +- [ ] `ServerContext.init` with the fixture cert+key succeeds; with truncated PEM returns `error.CertParse`/`error.KeyParse` (unit tests, no network). +- [ ] Loopback echo test passes under `-Dintegration` (orchestrator wires + runs). +- [ ] No `@cImport` anywhere; extern decls + C shim only. +- [ ] `close` sends close_notify (verified in the loopback test by the client reading EOF without error after `end`). + +--- + +## Session S5: Gitea CI + +### S5.1 .gitea/workflows/ci.yml + +House style: `runs-on: ubuntu-24.04`, `actions/checkout@v4`, top-level `env:` for versions (model: `~/app/phoenix_inertia_react_starter/.gitea/workflows/ci.yml`). Runner: x86_64, dind, full egress, `uses:` resolves against github.com. Cache is ephemeral (runner restarts daily) — `actions/cache@v4` allowed as best-effort, never load-bearing. + +Jobs: +1. `test`: checkout → `mlugg/setup-zig@v2` with `version: 0.16.0` → `zig build test` → `zig build test -Dintegration` is NOT run here (loopback integration runs are wired by the orchestrator in a later pass once tests.zig includes them; leave a commented job stub with a TODO referencing milestone-1 integration wiring). +2. `cross`: checkout → setup-zig → `zig build cross` → assert both output binaries exist and `file` reports statically linked. + +### S5.2 .gitea/workflows/live-tls.yml + +`workflow_dispatch` only. Runs `zig build test -Dintegration` (which includes the live DoT handshake test once wired). Non-blocking by construction. + +### S5.3 Acceptance Criteria + +- [ ] `yamllint`-clean (or at minimum `python3 -c "import yaml,sys; yaml.safe_load(open('.gitea/workflows/ci.yml'))"` passes for both files). +- [ ] Workflow YAML uses only actions available from github.com (`actions/checkout@v4`, `mlugg/setup-zig@v2`, `actions/cache@v4`). +- [ ] No job depends on cache hits for correctness. + +--- + +## Module Layout + +``` +AGENTS.md values + aim (exists) +PLAN.md source of truth (exists) +specs/milestone-1.md this file +specs/research/zig-0.16-api-notes.md stdlib ground truth (exists) +build.zig S1 +build.zig.zon S1 +.gitignore S1 +src/main.zig S1 CLI dispatch stub +src/version.zig S1 build_options plumbing +src/tests.zig S1 test aggregator (orchestrator extends) +src/platform/address.zig S2 NetAddress/Prefix/matchLongest +src/platform/tls_client.zig S3 stdlib TLS client wrapper +src/platform/tls_server.zig S4 mbedTLS server wrapper +src/platform/mbedtls_shim.c S4 sizeof shims +tests/fixtures/self_signed_cert.pem S4 +tests/fixtures/self_signed_key.pem S4 +tests/fixtures/README.md S4 +.gitea/workflows/ci.yml S5 +.gitea/workflows/live-tls.yml S5 +``` + +## File Ownership + +| Files | Owner | Notes | +|---|---|---| +| build.zig, build.zig.zon, .gitignore, src/main.zig, src/version.zig, src/tests.zig | S1 | frozen after S1; orchestrator edits afterward | +| src/platform/address.zig | S2 | | +| src/platform/tls_client.zig | S3 | | +| src/platform/tls_server.zig, src/platform/mbedtls_shim.c, tests/fixtures/* | S4 | shim build-wiring done by orchestrator | +| .gitea/workflows/* | S5 | | + +S2–S5 MUST NOT edit build.zig, build.zig.zon, or src/tests.zig. If a session needs a build change, it reports the exact needed change in its completion report; the orchestrator applies it. + +## Acceptance Criteria (Milestone 1 Complete) + +- [ ] `zig build test` exits 0 (aggregator includes address, tls_client unit, tls_server unit tests). +- [ ] `zig build test -Dintegration` exits 0 locally (loopback TLS echo; live DoT test may be skipped on network failure with a visible skip message). +- [ ] `zig build cross` produces two statically linked executables that print `nxdns version` output under qemu-user or on-target (checked manually for aarch64 if qemu absent). +- [ ] CI workflows valid YAML; `test` + `cross` jobs green on the Gitea runner. +- [ ] All files committed with GPG-signed, lowercase-message commits. + +## Anti-Requirements + +- No DNS packet code, no sockets beyond the tests, no SQLite usage beyond the link check — that is milestone 2+. +- No custom mbedtls_config.h, no cipher tuning, no session tickets. +- No `@cImport`/translate-c anywhere. +- No extra CLI behavior beyond the specified stubs. +- No Docker/systemd packaging yet. +- No third-party Zig packages. +- Do not "fix" or extend files another session owns — report, don't touch. diff --git a/specs/research/zig-0.16-api-notes.md b/specs/research/zig-0.16-api-notes.md new file mode 100644 index 0000000..9fb5deb --- /dev/null +++ b/specs/research/zig-0.16-api-notes.md @@ -0,0 +1,138 @@ +# Zig 0.16.0 stdlib API notes (verified against ../zig at tag 0.16.0) + +Ground truth for build sub-agents. Every claim below was read from the 0.16.0 source +(commit 24fdd5b7a4c1c8b5deb5b56756b9dbc8e08c86a8). When in doubt, re-check the source at +`/home/mokhtar/app/zig/lib/std/` — do not trust pre-0.16 knowledge; the std.Io migration +changed most of these APIs. + +## std.Build (build.zig) + +- `b.addExecutable(.{ .name, .root_module, ... })` — no target/optimize on the artifact. + Target/optimize/link_libc go on the module: `b.createModule(.{ .root_source_file, .target, + .optimize, .link_libc, .imports, ... })` (Module.CreateOptions, Module.zig:216). +- All C configuration is **module-level** (Step.Compile has none of these methods): + - `exe.root_module.addCSourceFile(.{ .file, .flags, .language })` + - `exe.root_module.addCSourceFiles(.{ .root, .files, .flags })` — `files` must be + **relative** paths; absolute paths panic. + - `exe.root_module.addIncludePath(lazy_path)`, `.addCMacro(name, value)` + - libc: `link_libc = true` in CreateOptions or `exe.root_module.link_libc = true`. +- Static lib: `b.addLibrary(.{ .linkage = .static, .name, .root_module })` + (`b.addStaticLibrary` does not exist). Link: `exe.root_module.linkLibrary(lib)`. +- Cross targets: `b.resolveTargetQuery(.{ .cpu_arch = .aarch64, .os_tag = .linux, + .abi = .musl })` or `std.Target.Query.parse(.{ .arch_os_abi = "x86_64-linux-musl" })`. + musl → static libc; for a fully static binary also set `exe.linkage = .static`. +- Options: `b.option(std.Build.LazyPath, "web-dist", "...")` accepts LazyPath. + `b.addOptions()` + `opts.addOption/addOptionPath` + `exe.root_module.addOptions("build_options", opts)`. +- Embedding an asset directory: one WriteFile step holding both a generated `index.zig` + and the assets: `const wf = b.addWriteFiles(); _ = wf.addCopyDirectory(dist, "assets", .{}); + const idx = wf.add("index.zig", src);` then + `exe.root_module.addAnonymousImport("assets", .{ .root_source_file = idx })`. + `@embedFile` paths must stay inside that module root (`error.ImportOutsideModulePath`). +- Version pin: compare `@import("builtin").zig_version` (SemanticVersion) in comptime, + `@compileError` on mismatch. +- build.zig.zon: `.name` is an **enum literal** (`.nxdns`), `.fingerprint` is **required** + (compiler suggests the value on first run), plus `.version`, `.paths`, `.dependencies` + (`path`/`url`+`hash`/`lazy`), `.minimum_zig_version`. + +## std.Io (Threaded backend, concurrency) + +- `var t = std.Io.Threaded.init(gpa, .{});` — returned by value, must live at a **stable + address**. `const io = t.io();` `defer t.deinit();` (joins workers). + InitOptions: `stack_size`, `async_limit` (default cpus-1), `concurrent_limit`. +- `io.async(f, args) Future(R)` may run inline; `io.concurrent(f, args) !Future(R)` + guarantees its own unit of concurrency — use for server loops. + `future.await(io)`, `future.cancel(io)`. +- `Io.Group`: `.init`, `g.async(io, f, args)`, `g.concurrent(io, f, args)`, `g.await(io)`, + `g.cancel(io)`. Group task fns must return something coercible to `Cancelable!void`. + Cancellation: next cancelable Io call returns `error.Canceled` once; `io.recancel()` + re-arms; `io.swapCancelProtection(.blocked)` protects cleanup sections. +- **No SIGINT/SIGTERM handling in Threaded** (only no-op SIGIO/SIGPIPE). Install our own + `posix.sigaction`, set an atomic flag, cancel the group from the main task. + +## std.Io.net + +- UDP: `addr.bind(io, .{ .mode = .dgram })` → `Socket`. + Receive: `sock.receive(io, buf) !IncomingMessage` — fields `from: IpAddress`, + `data: []u8` (slice into buf), `flags.trunc`. Timeout variant: `receiveTimeout(io, buf, + timeout)`. Send: `sock.send(io, &dest_addr, data)`. Close: `sock.close(io)`. + (No recvFrom/sendTo names.) +- TCP server: `addr.listen(io, .{ .reuse_address = true })` → `net.Server`; + `srv.accept(io) !Stream`; `srv.deinit(io)`. `Stream.close(io)`, `.shutdown(io, how)`. + Shutdown of the listener makes a blocked accept fail `error.SocketNotListening`. +- TCP client: `addr.connect(io, .{ .timeout = ... })` → `Stream`. + `Io.Timeout = union(enum){ none, duration, deadline }` — **connect only**. +- **Stream reads/writes accept no timeout** in 0.16.0 (VTable netRead/netWrite have none; + Operation lacks net stream variants). Bound a TCP/TLS read by running it under + `io.concurrent` and cancelling the future (or shutdown the socket). +- Reader/Writer: `stream.reader(io, buf) Stream.Reader`; generic interface is + `&stream_reader.interface` (`*Io.Reader`); same for writer. Caller owns buffers; `&.{}` + legal. Concrete errors land in `.err`; interface returns ReadFailed/WriteFailed. + `Stream.Writer.sendFile` returns `error.Unimplemented` in 0.16.0. +- Known: net tests in stdlib are skipped (upstream issue 31388) — do not copy test.zig + patterns blindly. + +## std.http.Client (DoH upstream) + +- Construct by struct literal: `var client: std.http.Client = .{ .allocator = gpa, .io = io };` + `defer client.deinit();` Fields: `ca_bundle`, `ca_bundle_lock`, `now: ?Io.Timestamp`, + `connection_pool` (LRU, free_size 32), `read_buffer_size`, `write_buffer_size`. +- DoH POST flow (low-level; `fetch` hides the body—don't use it): + 1. `var req = try client.request(.POST, uri, .{ .headers = .{ .content_type = + .{ .override = "application/dns-message" } } });` + 2. `try req.sendBodyComplete(body_mut);` (body is `[]u8`, sets content-length + flush) + 3. `var resp = try req.receiveHead(&redirect_buf);` + 4. Check `resp.head.status`, `resp.head.content_type`. + 5. `const rdr = resp.reader(&transfer_buf);` then read (`allocRemaining`/`readSliceShort`). + 6. `req.deinit()` returns the connection to the pool. +- TLS: client TLS uses std.crypto.tls.Client internally with the client's `ca_bundle`; + lazily `rescan`s system roots when `client.now == null` + (`error.CertificateBundleLoadFailure` on failure). Pre-fill `ca_bundle` + set `now` to + skip the scan. +- **No per-request deadline** exists. Enforce total-budget timeouts via concurrent+cancel + (see std.Io note above). + +## std.http.Server (web/API/DoH server) + +- `var srv = std.http.Server.init(&reader.interface, &writer.interface);` + Loop for keep-alive: `while (srv.reader.state == .ready)` + catch HttpConnectionClosing. +- `var req = try srv.receiveHead();` — `req.head.method/.target/.content_type/...`, + `req.iterateHeaders()`. **Copy `head.target` before reading the body** (body reads + invalidate head memory). Body: `req.readerExpectContinue(buf)` (handles 100-continue) + or `req.readerExpectNone(buf)`. +- Simple response: `try req.respond(content, .{ .status = ..., .extra_headers = &.{...} });` +- Streaming/SSE: `var body = try req.respondStreaming(&.{}, .{ .respond_options = ... });` + With `content_length == null` → chunked. **Use an empty buffer** so each write drains + through; then `response.flush()` after each event suffices; `response.end()` to finish. + (With a non-empty buffer you must flush response.writer first — trap.) +- WebSocket upgrade exists (`upgradeRequested`/`respondWebSocket`) — unused by nxdns. +- No MIME table, no etag: set `content-type`/`cache-control` manually via extra_headers. + BodyWriter supports sendFile (contentLengthSendFile/chunkedSendFile). + +## std.zon + +- Parse: `std.zon.parse.fromSlice(T, gpa, src_z, ?*Diagnostics, .{})` for pointer-free T + (no free needed); `fromSliceAlloc` + `std.zon.parse.free(gpa, v)` for slice-bearing T. + Source must be `[:0]const u8`. No reader-based entry point. + Options: `ignore_unknown_fields`, `free_on_error`. Diagnostics: init `.{}`, + `deinit(gpa)`, print with `{f}`. +- Struct default field values apply for absent fields. Tagged unions: `.foo` (void) or + `.{ .foo = v }`. Enums, optionals (single level only — `??T` rejected), slices, arrays, + nested structs all fine. Error sets/unions, many-pointers, comptime_int rejected. +- Serialize: `std.zon.stringify.serialize(value, .{ .whitespace = true }, writer)`. + Streaming: `var s: std.zon.Serializer = .{ .writer = w, .options = .{} };` with + `beginStruct`/`field`/`end`. + +## std.crypto.tls.Client (upstream DoT) + +- `std.crypto.tls.Client.init(input: *Io.Reader, output: *Io.Writer, options) !Client` + Options: `.host = .{ .explicit = "dns.google" }` (or `.no_verification`), + `.ca = .{ .bundle = .{ .gpa, .io, .lock, .bundle } }` (or `.no_verification`/`.self_signed`), + `.write_buffer`, `.read_buffer` (input buffer ≥ `Client.min_buffer_len` = + `tls.max_ciphertext_record_len`), `.entropy: *const [240]u8` (fill via `io.random`), + `.realtime_now: Io.Timestamp`. +- After init: plaintext via `&client.reader` / `&client.writer` (held **by value** — the + Client must not move after init). `client.end()` sends close_notify. Errors in + `client.read_err` / `client.alert`. +- TLS 1.2/1.3. **No ALPN, no session resumption** (fine for DoT; DoH over HTTP/1.1 works + without ALPN in practice — verify against real upstreams in Phase 3). +- CA roots: `std.crypto.Certificate.Bundle` (note the path), `bundle.rescan(gpa, io, now)`.