mokhtar/nxdns
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

project plan, values, milestone 1 spec, zig 0.16 api research

Browse Source
This commit is contained in:
mokhtar
2026-07-31 23:16:57 +02:00
commit b609176a0b
4 changed files with 1148 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+37
View File
@@ -0,0 +1,37 @@
# AGENTS.md
## Aim
nxdns: a self-hosted DNS sinkhole for a household LAN, written in Zig 0.16.0.
Portfolio-grade public repo. PLAN.md is the source of truth for scope and design;
specs/ holds per-milestone contracts; specs/research/ holds verified stdlib facts.
## Values
We intentionally architect this code to be robust, maintainable, pragmatic —
good craftsmanship and good engineering. We explicitly avoid tech debt, code
smells, bad architecture decisions, and brittle implementations.
What that means in practice:
- This is a greenfield project. Breaking changes are allowed. Never keep a bad
interface for compatibility; fix it at the root.
- No versioning of scope. A feature is in scope (build it completely) or out of
scope (do not build it). No "v2 later", no stubs left behind.
- Fix root causes, not symptoms. Do not iterate on workarounds.
- Scope is small on purpose: household scale, two targets, few dependencies.
Do not add generality nobody asked for.
- Dependencies are liabilities: stdlib first; vendored + pinned C deps
(sqlite3, mbedTLS) only where the stdlib has nothing.
- Verify stdlib claims against ../zig at tag 0.16.0 — pre-0.16 knowledge is
stale (std.Io migration). See specs/research/zig-0.16-api-notes.md.
- Pure core: dns/, filter/, local/, cache/ take bytes and return bytes — no Io,
no sockets, no clocks hidden inside.
- Every failure mode must be visible: no silent drops, no unbounded logs, no
swallowed errors. Counters + health surfaces over log spam.
- Tests are runnable acceptance criteria, not decoration. Required CI stays
deterministic — no network-dependent tests in blocking jobs.
- Comments state constraints the code cannot show. No narration, no
commented-out code.
- Git: GPG-signed commits (`git commit -S`), simple lowercase messages, no
generated-by footers.
PLAN.md
+683
View File
@@ -0,0 +1,683 @@
# nxdns — Implementation Plan v3.0 (Zig 0.16.0 Stable)
Source of truth for **nxdns**, a self-hosted DNS sinkhole written in Zig 0.16.0 stable.
All stdlib claims in this document are verified against the `0.16.0` tag of the Zig repo (`../zig`).
There is no v1/v2 versioning. Scope is binary: a feature is in scope (and gets built) or out of scope (and does not). "Done" = everything in scope implemented, tested, documented.
## 1. Purpose
Fix three pain points in Pi-hole/AdGuard/Technitium:
1. Clunky, dated admin UI.
2. No config-as-code story (no way to export current state as a text artifact).
3. Silent operational failures (cloudflared proxy-dns incident: broken DoH setup + unbounded error logs filled SD card).
Serves a household LAN (≈2–20 devices). Portfolio-grade public repo with extensive docs, self-hosted on Gitea.
---
## 2. Scope
### 2.1 In Scope
- DNS server for LAN clients: UDP/53, TCP/53, DoH server, DoT server.
- Upstream resolution: DoH (HTTP/1.1), DoT.
- Local DNS records (A/AAAA/CNAME) + conditional forwarding (zone → designated resolver, plain UDP/TCP allowed).
- Domain filtering: blocklists (hosts/domains/ABP), custom rules (allow/block; exact, parent-walk, wildcard), CNAME uncloaking (depth 8), per-group safe-search rewrite.
- DNS caching: positive + negative, in-memory only.
- Client/group model: IPv4 + IPv6 parity, per-client group assignment, per-group source assignments.
- Query logging + analytics: async batched writes to SQLite (WAL), retention cleanup, dashboard + time buckets, live SSE stream.
- Web app + REST API: LAN/Tailscale admin UI, optional password auth, OpenAPI schema + CI contract tests.
- Observability: upstream health API + UI, disk monitor with UI banner, bounded log rotation, Prometheus `/metrics`.
- Ops: DB-as-truth config, `nxdns export`/`import` (ZON), scheduled + manual blocklist updates, TLS cert watcher + reload, auto-migration on upgrade, systemd service + Dockerfile + compose.
### 2.2 Out of Scope (permanent scope decisions, not deferrals)
- Regex rules. Wildcards + parent-walk cover the real use cases; regex on the DNS hot path means ReDoS exposure plus an immature dependency or a homegrown engine. Regex lines in blocklists are counted, skipped, and the skip count is surfaced in the UI.
- DHCP server.
- DNSSEC validation (DO bit passthrough only).
- DoQ (QUIC), HTTP/2 upstream transport.
- Clustering / distributed state.
- Prebuilt binaries / published Docker images / project website. Repo + documented build-it-yourself path only.
---
## 3. Locked Decisions
### 3.1 Toolchain
- Compiler: **Zig 0.16.0 stable**. `build.zig` asserts major/minor.
- Targets: **x86_64-linux-musl** and **aarch64-linux-musl**, both first-class in CI. Release binaries are **static musl** — fully self-contained, no glibc version coupling on the Pi.
### 3.2 Concurrency: `std.Io` (Decision E)
Zig 0.16 stdlib is built on the `std.Io` interface: `std.Io.net` owns networking (`std.net` in its old form is gone) and `std.http.Client` requires an `io: Io`. Therefore:
- **`Io` is the injected platform abstraction.** Every component that does I/O takes `io: Io`. No project-owned wrapper interfaces around it — a second abstraction over an abstraction with one consumer is waste.
- **Backend: `std.Io.Threaded` by default** — the mature, debuggable path; at ≤20 devices throughput is a non-issue. `std.Io.Evented` (io_uring) is selectable via config flag; the code is backend-agnostic by construction, so this is a switch, not a refactor.
- No hand-written thread pool. `Io` async/concurrent/Group covers task management.
- Core domain modules (`dns`, `filter`, `cache`) stay pure: no `Io`, no sockets — bytes in, bytes out. Only servers, upstream clients, and storage touch `Io`.
### 3.3 TLS (Decision D)
Verified: 0.16.0 ships `std.crypto.tls.Client` only. There is no server-side TLS in the stdlib.
- **Upstream client TLS (DoH/DoT): stdlib** `std.crypto.tls` via `std.http.Client` / direct. Maintained upstream, exercised by the stdlib itself.
- **Local server TLS (DoH/DoT termination): vendored mbedTLS.** Audited, embedded-focused, compiles cleanly with the Zig build system for both targets. We do not hand-roll a TLS server in a security-sensitive service.
- TLS glue lives in `platform/tls_client.zig` (stdlib wrapper: error classification, deadlines, retry policy) and `platform/tls_server.zig` (mbedTLS wrapper exposing `std.Io.Reader`/`Writer` so `std.http.Server` composes on top unchanged).
### 3.4 C Dependency Policy
- **`sqlite3` + `mbedtls`, both vendored** and compiled from source in `build.zig`. Pinned versions, hermetic cross-compilation, no system libs.
- SQLite: amalgamation + our own thin wrapper (`storage/db.zig`: open/close, prepared-statement cache, typed row mapping, error translation). No third-party binding — the needed surface is small enough to own (Decision G).
- Zig package deps: minimized; anything adopted is pinned in `build.zig.zon`. No TOML/regex/HTTP packages needed under current decisions.
### 3.5 Config Format + Truth Model (Decision F)
- **DB is truth. Config file format is ZON** (`std.zon` parse + stringify — typed parsing into config structs, exact round-trip, stdlib-maintained, comments supported). No TOML: a third-party parser plus a hand-written serializer is two failure surfaces in the correctness-critical bootstrap/round-trip path, bought for syntax familiarity.
- First start: if DB empty and `/etc/nxdns/config.zon` exists, validate → seed DB. Subsequent starts ignore the file.
- `nxdns export [--out file.zon]` dumps DB state as canonical ZON. `nxdns import <file.zon>` validates + replaces DB contents (`--force` if DB non-empty). Export/import = backup + host migration, **not** upgrades (§3.7).
- No file watcher, no auto-regeneration.
### 3.6 Storage Layout (Decision H)
Two SQLite files with opposite write profiles, isolated from each other:
- **`config.db`** — small, precious, rarely written: groups, clients, prefixes, upstreams, blocklist source metadata, rules, local records, forward zones, settings, schema version.
- **`querylog.db`** — high-churn, large, expendable: query log + its own private `domains` dimension table. Client identity stored as **IP text**, not a FK into config — log rows are immutable facts and must not point at mutable config rows. If `querylog.db` is missing or corrupt at startup, rename aside, recreate, keep serving. Log loss is not an outage.
- No cross-DB references. Retention/VACUUM churn never touches `config.db`; config backup is a copy of a tiny file.
### 3.7 Upgrades: Auto-Migration (Decision J)
- `config.db`: numbered, sequential SQL migration steps compiled into the binary. At startup: read schema version row, apply newer steps inside a transaction, continue. Operator upgrade = install binary, restart.
- `querylog.db`: **no migrations.** On schema mismatch: rename aside, recreate fresh.
### 3.8 Blocklist Storage (Decision A)
Blocklist domains are **not** stored in SQLite — they are a cache of re-downloadable remote artifacts, not config or state:
- Each source compiles to `/var/lib/nxdns/blocklists/<source_id>.list`: normalized, one domain per line, small header (source URL, fetch time, count, checksum). Wildcard/regex-flavored lines: wildcards go to `<source_id>.wild`; regex lines are counted + skipped (count in metadata → UI).
- `config.db` keeps source **metadata only** (`blocklist_sources`).
- Startup + post-update: parse files into the immutable in-memory matcher (RCU swap, §9.5).
- Corruption recovery is per-file: checksum mismatch → re-download one list.
### 3.9 Rule Model (Decision B)
Rule kinds: `exact`, parent-walk (implicit via candidate chain), `wildcard` (`*` segment patterns, e.g. `*.doubleclick.net`, `ads.*.example.com`). Actions: `allow` | `block`. Kind is an explicit column — the model is extensible without breakage, but regex stays out of scope (§2.2).
### 3.10 Filtering Precedence
1. Exact/parent **allow** rules
2. Exact/parent **block** rules
3. Wildcard allow rules
4. Wildcard block rules
5. Blocklist domains
6. Blocklist wildcards
Tie-break at same specificity: **allow wins**.
### 3.11 Network Posture
- Default web bind: LAN/Tailscale-friendly (non-loopback allowed).
- Auth optional (empty `web.password` disables it). Set → argon2id-hashed password, session cookie (`HttpOnly`, `SameSite=Lax`, `Secure` on HTTPS).
- API per-IP rate limit on by default.
### 3.12 Addressing
IPv4 + IPv6 full parity for: client identity, rate limiting, logging, group assignment, prefix-based group matching. Dual-stack assumed.
### 3.13 Filesystem Layout (FHS)
- `/etc/nxdns/config.zon` — bootstrap (first start only).
- `/var/lib/nxdns/config.db`, `/var/lib/nxdns/querylog.db`
- `/var/lib/nxdns/blocklists/*.list|*.wild`
- `/var/log/nxdns/nxdns.log` — only in file output mode; default is stderr → journald.
### 3.14 Frontend Stack (Decision I)
- Vite + React + TypeScript + Tailwind; TanStack Router + TanStack Query. SPA, no SSR.
- **Built assets embedded in the binary** at compile time: single self-contained artifact, no asset-path config, no binary/UI skew. `zig build` accepts the dist path; the release pipeline runs `npm run build` first (CI always does).
- Dev-mode flag serves assets from disk so UI iteration needs no Zig rebuild.
### 3.15 CI
- Self-hosted **Gitea + Gitea Actions runner**.
- Jobs: `zig build test`, fuzz smoke, integration tests, OpenAPI contract tests (live server validated against `openapi.yaml`), frontend build, cross-compile both targets. aarch64 test execution via qemu-user if the runner is x86_64.
---
## 4. Architecture Overview
```
Client DNS Query
|
v
[UDP / TCP / DoH / DoT servers] (std.Io; mbedTLS terminates DoH/DoT)
|
v
[Request Handler]
|- rate limit check
|- parse DNS packet
|- group resolution (v4/v6 parity)
|- local records check -> answer locally
|- forward-zone check -> forward to designated resolver (bypasses filtering)
|- allow/block evaluation (rules + blocklists, per group)
|- safe-search rewrite (per group)
|- cache lookup
|- upstream query (DoH/DoT pool) + health tracking
|- CNAME uncloaking check
|- cache store
|- async query log + SSE fanout
v
Response to client
```
Cross-cutting: `ConfigManager` (bootstrap/import/export/settings), `BlocklistManager` (fetch/compile/swap), `Storage` (two SQLite DBs), `Cache`, `UpstreamHealth`, `DiskMonitor`, `Auth`, `Web API` (REST + SSE + metrics).
---
## 5. Source Layout
```
src/
main.zig # Io backend instantiation, wiring
app.zig
version.zig
platform/
address.zig # NetAddress: v4/v6 parity type + canonical key (was net_adapter)
tls_client.zig # stdlib TLS wrapper: deadlines, error classes, retry policy
tls_server.zig # mbedTLS wrapper exposing std.Io.Reader/Writer
dns/ # pure: bytes in, bytes out; no Io
types.zig header.zig name.zig question.zig record.zig edns.zig packet.zig
server/
udp_server.zig tcp_server.zig doh_server.zig dot_server.zig
handler.zig rate_limiter.zig shutdown.zig
upstream/
doh_client.zig dot_client.zig pool.zig health.zig
filter/ # pure
matcher.zig rules.zig wildcard.zig
parser_hosts.zig parser_domains.zig parser_abp.zig
fetcher.zig compiler.zig # list download -> compiled .list/.wild files
safesearch.zig
local/ # pure
records.zig # local A/AAAA/CNAME answers
forward_zones.zig # zone -> designated resolver matching
cache/
dns_cache.zig
storage/
db.zig # sqlite3 thin wrapper
config_schema.zig migrations.zig
querylog_schema.zig
repositories/
clients_repo.zig groups_repo.zig rules_repo.zig sources_repo.zig
queries_repo.zig settings_repo.zig upstreams_repo.zig local_repo.zig
logger.zig retention.zig disk_monitor.zig
config/
model.zig bootstrap.zig import.zig export.zig validate.zig # all ZON via std.zon
web/
server.zig router.zig auth.zig sse.zig static.zig metrics.zig openapi.zig
handlers/
auth.zig stats.zig queries.zig clients.zig groups.zig blocklists.zig
rules.zig local.zig lookup.zig pause.zig settings.zig
upstream_health.zig certs.zig health.zig version.zig
web/ # Vite + React + TS + Tailwind + TanStack
vendor/ # sqlite3 amalgamation, mbedtls (pinned)
docs/ # operator/ architecture/ config-reference/ api/
tests/ # dns/ integration/ fuzz/
```
---
## 6. DNS Behavior
### 6.1 Protocol
- RFC 1035 parse/encode. EDNS OPT passthrough. DO bit passthrough (no validation). ECS: `strip` (default) or `forward`.
- Process first question only. Preserve query ID and RD flag.
- Malformed: `FORMERR` when a response is possible; silent drop for severely truncated.
### 6.2 Blocked Response Modes
`blocking.response`: `zero` (A=`0.0.0.0`, AAAA=`::`) or `nxdomain`. TTL = `blocking.ttl`.
### 6.3 CNAME Uncloaking
Walk chain to depth 8; any target hitting block logic → synthesize blocked response.
### 6.4 Local Records
- Table-backed A/AAAA/CNAME records, group-independent. Matched before filtering.
- CNAME targets resolve through the normal pipeline (uncloaking rules apply).
### 6.5 Conditional Forwarding
- `forward_zones`: suffix match (`lan.home`, `10.in-addr.arpa`, …) → designated resolver (`udp://192.168.1.1:53` style; plain UDP/TCP permitted — these are local infra resolvers).
- Matching queries bypass filtering and blocklists; responses cached normally.
---
## 7. Filtering Engine
### 7.1 Evaluation
For `{domain, qtype, group_id}`:
1. Normalize: lowercase, trim trailing dot.
2. Build candidate chain (full, parent1, parent2, …).
3. Explicit rules per §3.10 precedence.
4. Group's blocklist domains (hash set over compiled lists).
5. Group's blocklist wildcards.
6. No match → allow.
### 7.2 Group Assignment
- Per source IP: (1) exact match in `clients`, (2) longest-prefix match in `client_prefixes` (ties: longer prefix, then priority), (3) `default` group.
- Auto-materialization: first query from an unseen IP inserts a `clients` row (`hand_edited=0`, `first_seen=now`) for UI visibility and stable group assignment. The query log does **not** FK to it (§3.6).
- Retention drops `hand_edited=0` clients with no queries in `retention_days`.
### 7.3 Reload
New immutable matcher built from DB + compiled list files → atomic pointer swap (RCU, generation counter). Readers lock-free.
### 7.4 Safe-Search
Per-group boolean. Rewrites known engine domains to their safe-search CNAME targets.
---
## 8. Cache
- Key: normalized qname + qtype + qclass + DO bit + forwarded ECS subnet (only when `ecs_mode=forward`).
- Value: full response bytes, `stored_at`, `expires_at`, positive/negative flag.
- Positive TTL: `min(answer TTLs)` bounded by sanity ceiling. Negative: SOA/minimum capped by `cache.negative_ttl_max` (0 disables).
- Hit path: clone bytes, overwrite transaction ID, decrement visible TTLs; TTL ≤ 0 → miss.
- Bounded by `cache.size`; CLOCK eviction; opportunistic + periodic expiry sweep.
- **In-memory only, no persistence.** Cold start refills within a minute; SD-corruption risk isn't worth pre-warmed entries.
---
## 9. Upstream Resolution
- Schemes: `https://…` → DoH, `tls://host:853` → DoT.
- Ordered by priority; sequential attempt; per-upstream failure counters; exponential backoff with jitter; success resets.
- `UpstreamHealth` per upstream: last_success_at, last_error_at, last_error_message, rolling success rate, consecutive failures, backoff-until. Exposed via `GET /api/upstream/health`, dashboard, `/metrics`, and `nxdns check`.
- DoH client: `std.http.Client` with `content-type/accept: application/dns-message`; strict status + payload checks.
- `platform/tls_client.zig` enforces per-connection read/write deadlines, classifies TLS errors explicitly, retries with backoff. Integration tests cover timeout/hang scenarios so compiler upgrades can't silently regress them.
- Connect, read, and total-budget timeouts each configurable.
---
## 10. Rate Limiting
- DNS: default 1000 req / 60s per client IP (v4/v6 keyed alike); exceeded → `REFUSED`.
- API: per-IP token bucket, on by default; separate SSE connection cap per IP; localhost relaxed (configurable).
- Limiter maps bounded; stale-key windowed sweep.
---
## 11. Storage
### 11.1 Pragmas (both DBs)
`journal_mode=WAL`, `synchronous=NORMAL`, `foreign_keys=ON`, `busy_timeout` set.
### 11.2 config.db Schema
```sql
CREATE TABLE schema_version (version INTEGER NOT NULL);
CREATE TABLE groups (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL UNIQUE,
safe_search INTEGER NOT NULL DEFAULT 0
);
INSERT OR IGNORE INTO groups (id, name) VALUES (1, 'default');
CREATE TABLE clients (
id INTEGER PRIMARY KEY,
ip TEXT NOT NULL UNIQUE, -- canonical text form (v4 dotted / v6 RFC 5952)
name TEXT,
group_id INTEGER NOT NULL REFERENCES groups(id),
hand_edited INTEGER NOT NULL DEFAULT 0,
first_seen INTEGER NOT NULL,
last_seen INTEGER NOT NULL
);
CREATE TABLE client_prefixes (
id INTEGER PRIMARY KEY,
prefix TEXT NOT NULL UNIQUE, -- "192.168.1.0/24", "fd00:abcd::/48"
group_id INTEGER NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
priority INTEGER NOT NULL DEFAULT 100
);
CREATE TABLE upstreams (
id INTEGER PRIMARY KEY,
url TEXT NOT NULL UNIQUE,
priority INTEGER NOT NULL DEFAULT 100,
enabled INTEGER NOT NULL DEFAULT 1
);
CREATE TABLE blocklist_sources (
id INTEGER PRIMARY KEY,
url TEXT NOT NULL UNIQUE,
name TEXT NOT NULL,
enabled INTEGER NOT NULL DEFAULT 1,
is_suggested INTEGER NOT NULL DEFAULT 0,
last_updated INTEGER,
domain_count INTEGER NOT NULL DEFAULT 0,
wildcard_count INTEGER NOT NULL DEFAULT 0,
skipped_regex_count INTEGER NOT NULL DEFAULT 0,
checksum TEXT
);
CREATE TABLE group_sources (
group_id INTEGER NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
source_id INTEGER NOT NULL REFERENCES blocklist_sources(id) ON DELETE CASCADE,
PRIMARY KEY (group_id, source_id)
);
CREATE TABLE rules (
id INTEGER PRIMARY KEY,
group_id INTEGER NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
pattern TEXT NOT NULL,
kind TEXT NOT NULL CHECK(kind IN ('exact','wildcard')),
action TEXT NOT NULL CHECK(action IN ('allow','block')),
created_at INTEGER NOT NULL
);
CREATE TABLE local_records (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL,
rtype TEXT NOT NULL CHECK(rtype IN ('A','AAAA','CNAME')),
value TEXT NOT NULL,
ttl INTEGER NOT NULL DEFAULT 300,
UNIQUE(name, rtype, value)
);
CREATE TABLE forward_zones (
id INTEGER PRIMARY KEY,
zone TEXT NOT NULL UNIQUE,
resolver TEXT NOT NULL -- "udp://192.168.1.1:53"
);
CREATE TABLE settings (key TEXT PRIMARY KEY, value TEXT NOT NULL);
```
### 11.3 querylog.db Schema
```sql
CREATE TABLE domains (
id INTEGER PRIMARY KEY,
domain TEXT NOT NULL UNIQUE
);
CREATE TABLE query_log (
id INTEGER PRIMARY KEY,
timestamp INTEGER NOT NULL,
domain_id INTEGER NOT NULL REFERENCES domains(id),
client_ip TEXT NOT NULL, -- text, not a FK: log rows are immutable facts
qtype INTEGER,
blocked INTEGER NOT NULL,
block_reason TEXT,
response_time_us INTEGER,
cache_hit INTEGER,
upstream TEXT
);
CREATE INDEX idx_query_log_ts ON query_log(timestamp);
CREATE INDEX idx_query_log_client ON query_log(client_ip);
CREATE INDEX idx_query_log_domain ON query_log(domain_id);
```
### 11.4 Query Logger
- In-memory buffer, mutex guarded, hard cap `query_log_buffer_max` (default 10000).
- Flush: batch size (default 100) or max interval (default 100ms).
- Privacy transforms (hide_domains / hide_client_ips) applied before persist + SSE fanout.
- Backpressure: buffer full → drop oldest unflushed entry, increment monotonic `queries_dropped` (exposed in `/api/health` + `/metrics`). SSE fanout precedes buffer insert, so live viewers still see dropped-from-persistence entries.
### 11.5 Retention
Periodic delete of rows older than `retention_days`; scheduled checkpoint/VACUUM on `querylog.db` only.
### 11.6 Disk Discipline (cloudflared lesson)
- Log output default: **stderr** (journald rotates). File mode: rotate at `log.max_size_mb` (default 50), keep `log.max_files` (default 5).
- `DiskMonitor` samples DB + log-dir sizes every 60s:
- free < `disk.warn_free_mb` → UI banner + `/api/health` degraded.
- free < `disk.min_free_mb` → stop non-essential writes (blocklist updates, log flushes); buffer continues under backpressure policy.
- Upstream error log lines deduplicated per-upstream at 1/minute.
---
## 12. Configuration
### 12.1 Bootstrap ZON Shape
```zon
.{
.upstream = .{
.servers = .{ "https://cloudflare-dns.com/dns-query", "tls://dns.google:853" },
.connect_timeout_ms = 2000,
.read_timeout_ms = 3000,
},
.dns = .{
.bind_ipv4 = "0.0.0.0",
.bind_ipv6 = "::",
.port = 53,
.rate_limit = 1000,
.rate_window_seconds = 60,
},
.blocking = .{ .response = .zero, .ttl = 5 }, // .zero | .nxdomain
.cache = .{ .size = 10000, .negative_ttl_max = 3600 },
.web = .{
.enabled = true,
.bind = "0.0.0.0",
.port = 8080,
.password = "", // empty => auth disabled
.session_ttl_hours = 24,
.api_rate_limit_per_min = 300,
.sse_max_connections_per_ip = 3,
},
.doh_server = .{ .enabled = false, .bind = "0.0.0.0", .port = 443,
.cert_path = "/etc/nxdns/cert.pem", .key_path = "/etc/nxdns/key.pem" },
.dot_server = .{ .enabled = false, .bind = "0.0.0.0", .port = 853,
.cert_path = "/etc/nxdns/cert.pem", .key_path = "/etc/nxdns/key.pem" },
.edns = .{ .ecs_mode = .strip }, // .strip | .forward
.local_records = .{ .{ .name = "nas.lan", .rtype = .A, .value = "192.168.1.10" } },
.forward_zones = .{ .{ .zone = "lan.home", .resolver = "udp://192.168.1.1:53" } },
.logging = .{
.level = .info,
.retention_days = 30,
.query_log_buffer_max = 10000,
.hide_domains = false,
.hide_client_ips = false,
.output = .stderr, // .stderr | .syslog | .file
.file_path = "/var/log/nxdns/nxdns.log",
.max_size_mb = 50,
.max_files = 5,
},
.disk = .{ .min_free_mb = 200, .warn_free_mb = 500 },
.blocklist_update = .{ .enabled = true, .interval_hours = 24 },
}
```
### 12.2 Validation
At least one upstream; ports in range; cert+key readable if DoH/DoT server enabled; resolver URLs parseable. `nxdns check` runs the validator + probes upstreams.
### 12.3 Settings Semantics
Scalars in `settings(key, value)`; ordered/structured items in dedicated tables. Restart-required settings flagged; UI shows "restart to apply" banner.
---
## 13. Web / API
### 13.1 Endpoints
- `POST /api/auth/login`, `POST /api/auth/logout`
- `GET /api/stats?period=…`, `GET /api/stats/timeseries?period=…`
- `GET /api/queries` (filter + paginate), `GET /api/queries/live` (SSE, per-IP cap)
- `GET/PUT /api/clients/{id}`
- `GET/POST/PUT/DELETE /api/groups…`, `/api/blocklists…`, `/api/rules…`, `/api/local-records…`, `/api/forward-zones…`
- `POST /api/blocklists/update`
- `GET /api/lookup?domain=…&group_id=…`
- `GET/POST /api/pause`
- `GET/PUT /api/settings`
- `GET /api/upstream/health`
- `POST /api/certs/reload`
- `GET /api/health` — overall + disk + upstream + queries_dropped rollup
- `GET /metrics` — Prometheus text exposition: query counters (total/blocked/cached), per-upstream health, cache stats, queries_dropped, disk gauges
- `GET /api/version`, `GET /api/openapi.yaml`
### 13.2 OpenAPI
Hand-maintained `openapi.yaml`, served + rendered into `docs/api/`. CI contract tests spin up a seeded server, hit every documented endpoint, and validate bodies/status/auth against the schema. Drift fails CI.
---
## 14. Frontend
Pages: Dashboard (stats + upstream health + disk), Query log, Live log, Clients, Groups, Blocklists, Rules, Local DNS (records + forward zones), Domain lookup, Settings.
Requirements: responsive desktop/mobile; route loaders for initial fetch; TanStack Query for cache/retries; error/loading states on every data view; works with auth enabled or disabled; restart-required banner.
---
## 15. CLI
- `nxdns run` — start the server.
- `nxdns check` — validate config, probe upstreams, test cert readability; nonzero exit on failure.
- `nxdns export [--out file.zon]`
- `nxdns import <file.zon> [--force]`
- `nxdns version` — app version, Zig version string, build date, git commit.
---
## 16. Implementation Order
### Phase 0 — Build Baseline
Scaffold tree; `build.zig` with 0.16 assertion, musl targets, vendored sqlite3 + mbedtls compiling; version plumbing; Gitea Actions workflows (test, integration, fuzz smoke, OpenAPI lint, frontend build).
Exit: cross-compiled hello-world linking both C deps on both targets; CI green.
### Phase 1 — Platform Layer
`platform/address.zig` (v4/v6 parity + canonical keys); `platform/tls_client.zig` (deadlines, error classes); `platform/tls_server.zig` (mbedTLS handshake → `std.Io.Reader`/`Writer`).
Exit: UDP echo over `std.Io`; TLS client handshake against a real host; mbedTLS server terminating a loopback TLS connection.
### Phase 2 — DNS Core
Types/header/name/question/record/packet; parser + encoder tests + fuzz target; EDNS + DO passthrough.
Exit: unit + fuzz smoke pass.
### Phase 3 — Resolver Transport
UDP server, TCP server, DoH + DoT upstream clients, pool + failover/backoff + health.
Exit: A/AAAA forwarding over UDP + TCP; health populated.
### Phase 4 — Storage + Config
SQLite wrapper; config.db schema + migration runner; querylog.db schema + recreate-on-mismatch; repositories; ZON bootstrap + import/export; `nxdns check`.
Exit: first start seeds DB from ZON; export → import round-trips byte-stable.
### Phase 5 — Filtering + Local DNS
Rule matcher (exact/parent/wildcard); blocklist parsers → compiled file format; fetcher + scheduled update; RCU swap; per-group safe-search; local records; forward zones.
Exit: precedence table validated by tests; local zone answers + conditional forwards work.
### Phase 6 — Cache + Rate Limit + Logging + Disk Monitor
TTL cache + reconstruction; v4/v6 rate limiter; async query logger + backpressure + retention; DiskMonitor + rotation + error-log dedup.
Exit: disk thresholds trigger degradation + drop counters in integration test.
### Phase 7 — Handler Integration
Full pipeline composition; CNAME uncloaking; pause/resume.
Exit: end-to-end DNS flow with blocking, local records, cache, failover.
### Phase 8 — Web / API / SSE / Auth / Metrics
HTTP server + router; handlers; SSE; optional auth; API rate limiting; `/metrics`; OpenAPI served + contract tests; embedded frontend + dev-mode disk serving.
Exit: frontend fully drives config and operations; contract tests green.
### Phase 9 — Local DoH/DoT Endpoints
DoH server + DoT server on `platform/tls_server.zig`; cert watcher + reload.
Exit: LAN client resolves via DoH and DoT against local certs.
### Phase 10 — Packaging + Ops + Docs
systemd unit (`AmbientCapabilities=CAP_NET_BIND_SERVICE`, hardened, writable `/var/lib/nxdns` + optional `/var/log/nxdns`); Dockerfile + compose (53/udp+tcp, 8080; mounts `/etc/nxdns`, `/var/lib/nxdns`); operator/architecture/config-reference/API docs.
Exit: documented deployment works end-to-end on the Pi 5.
---
## 17. Testing Strategy
- **Unit**: DNS encode/decode; rule precedence + wildcard matcher; cache put/get/TTL rewrite; ZON bootstrap + export round-trip; rate limiter; migration runner (fresh + stepwise upgrade).
- **Fuzz**: DNS parser malformed-packet fuzzing; blocklist parser fuzzing.
- **Integration**: UDP/TCP query path; blocked path; allow-over-block; wildcard precedence; CNAME uncloaking block; local records + forward zones; upstream failover/backoff/health; disk-full degradation; querylog.db corruption recovery; API CRUD; auth on/off; SSE; contract tests.
- **Manual**: `dig @pi example.com` / blocked domain / local record; DoH/DoT client checks; dashboard + live log.
---
## 18. Performance Targets
- Sustained ≥ 100 qps on Raspberry Pi 5.
- Blocklist lookup p95 < 1 ms.
- Cached response p95 < 5 ms.
- Memory with ~1M blocked domains < 100 MB.
- Stripped static binary < 10 MB per arch (excluding embedded frontend assets; < 15 MB with them).
---
## 19. Security
- Sanitize all DNS and API inputs; prepared statements everywhere.
- argon2id password hashing; random session tokens with expiry.
- No secrets in logs. TLS keys readable by service user only.
- systemd hardening + least privilege.
- mbedTLS + sqlite3 versions pinned; upgrades are deliberate, reviewed bumps.
---
## 20. Success Criteria
1. `nxdns run` starts cleanly on Zig 0.16.0 stable, static musl, both arches.
2. UDP + TCP resolution works; blocked domains return the configured response; precedence per §3.10.
3. Local records answer; conditional forwarding works.
4. Cache hit path returns valid ID-adjusted responses.
5. Upstream failover + backoff works; health in UI, API, `/metrics`.
6. Disk-fill degrades gracefully; no silent log-flood failure mode.
7. Web UI + API provide full admin functionality; OpenAPI contract tests green.
8. `nxdns export` round-trips via `nxdns import`.
9. Query logging, stats, SSE live stream work; querylog.db corruption self-heals.
10. Local DoH + DoT endpoints serve LAN clients.
11. Schema upgrade = install + restart (migration test proves it).
12. All suites green in Gitea CI for both targets.
13. Operator, architecture, config-reference, and API docs complete.
---
## 21. Working Notes
- `dns/`, `filter/`, `local/`, `cache/` stay pure (no `Io`, no sockets). Servers, upstream clients, and storage take `io: Io`.
- Verify stdlib behavior against `../zig` (tag 0.16.0) instead of memory — the std.Io migration invalidated older knowledge once already.
- Keep OpenAPI in sync with handlers (CI enforced).
- When a plan decision and reality diverge during build: surface 2–3 alternatives with tradeoffs, get a call, then execute.
## Decision Log
| # | Decision |
|---|----------|
| A | Blocklists compile to flat files under `/var/lib/nxdns/blocklists/`; DB stores source metadata only |
| B | Rule kinds: exact, parent-walk, wildcard. Regex permanently out of scope |
| C | In scope: local DoH/DoT server, local records, conditional forwarding. Out: HTTP/2, DoQ, DHCP, DNSSEC, clustering |
| D | mbedTLS (vendored) terminates server TLS; stdlib TLS for upstream client |
| E | `std.Io` injected everywhere; `Threaded` default backend, io_uring via flag; no custom thread pool |
| F | Config format: ZON via `std.zon`; DB is truth; export/import for backup + host moves |
| G | SQLite vendored amalgamation + own thin wrapper |
| H | Two DBs: `config.db` (precious) + `querylog.db` (expendable, self-contained, client IP as text) |
| I | Frontend embedded in binary; dev flag serves from disk; static musl release builds |
| J | Auto-migration for `config.db` at startup; `querylog.db` recreated on mismatch |
| — | Safe-search per-group; Prometheus `/metrics` in scope; CI on self-hosted Gitea Actions |
specs/milestone-1.md
+290
View File
@@ -0,0 +1,290 @@
# Milestone 1: Build Baseline + Platform Layer
Goal: `zig build test` green natively; static musl executables for x86_64-linux and aarch64-linux linking pinned sqlite3 + mbedTLS; platform modules (address, tls_client, tls_server) implemented and tested; Gitea CI green. No DNS logic in this milestone.
Read first: `AGENTS.md` (values), `specs/research/zig-0.16-api-notes.md` (verified stdlib facts — pre-0.16 API knowledge is stale and MUST NOT be used). The Zig source of truth is `/home/mokhtar/app/zig` at tag `0.16.0`.
## Sessions
Five sessions. S1 runs first, alone. S2, S3, S4, S5 run in parallel after S1 is verified. The orchestrator (not any session) wires `src/tests.zig` imports and any `build.zig` additions afterward.
```
S1 ──> { S2, S3, S4, S5 } ──> orchestrator integration
```
---
## Session S1: Build Baseline
### S1.1 build.zig.zon
- `.name = .nxdns`, `.version = "0.1.0"`, `.minimum_zig_version = "0.16.0"`, `.paths = .{""}`, `.fingerprint` (compiler suggests the value on first build — accept it).
- Dependencies added with `zig fetch --save=<name> <url>` so the content hash is pinned:
- `sqlite`: the newest stable SQLite **amalgamation zip** from sqlite.org (check https://sqlite.org/download.html for the current one; record the version in a comment).
- `mbedtls`: the newest **mbedTLS 3.6.x LTS** release tarball from the Mbed-TLS GitHub releases (3.6 line only — not 4.x).
### S1.2 build.zig
- Comptime guard: `@import("builtin").zig_version` major==0 and minor==16, else `@compileError`.
- Options: `-Dintegration` (bool, default false) exposed to tests via a `build_options` module (`b.addOptions()`); also embed version string + git commit (`b.option([]const u8, "version-string", ...)` defaulting to "0.1.0-dev") for `src/version.zig`.
- C static libs, one per dependency, built with `b.addLibrary(.{ .linkage = .static, ... })` (NOT addStaticLibrary — it does not exist in 0.16):
- `sqlite3`: compile `sqlite3.c` from the sqlite dependency via `lib.root_module.addCSourceFile`; `addIncludePath` the dep root. Flags: `-DSQLITE_ENABLE_FTS5`, `-DSQLITE_THREADSAFE=1`, `-DSQLITE_DEFAULT_WAL_SYNCHRONOUS=1`, `-DSQLITE_OMIT_LOAD_EXTENSION`.
- `mbedtls`: compile every `library/*.c` file via `root_module.addCSourceFiles(.{ .root = dep.path("library"), .files = ... })` (paths must be relative — absolute paths panic) plus the `3rdparty/everest` and `3rdparty/p256m` sources; `addIncludePath` for `include`, `library`, and both 3rdparty include dirs. Default `mbedtls_config.h`, no custom config in this milestone.
- Executable `nxdns`: root module `src/main.zig`, `link_libc = true`, links both libs. Native artifact + installed.
- Test step `test`: `b.addTest` on a module rooted at `src/tests.zig`, importing `build_options`, linking both C libs.
- Step `cross`: for `x86_64-linux-musl` and `aarch64-linux-musl` (via `b.resolveTargetQuery`), build the exe with `exe.linkage = .static`, install to `zig-out/cross/<triple>/nxdns`.
- `.gitignore`: `.zig-cache/`, `zig-out/`.
### S1.3 src/main.zig, src/version.zig, src/tests.zig
- `src/version.zig`: `pub const string` / `pub const zig_version_string` pulled from `build_options`.
- `src/main.zig`: parse first CLI arg. `version` → print version + zig version, exit 0. `run`, `check`, `export`, `import` → print `not implemented`, exit 2. No arg / unknown → usage on stderr, exit 64. No allocator gymnastics — `std.process.args` is enough here.
- `src/tests.zig`: `comptime { _ = @import("main.zig"); _ = @import("version.zig"); }` plus `test` block asserting the sqlite3 and mbedTLS C headers link: call `sqlite3_libversion()` and `mbedtls_version_get_string_full()` via `@cImport`-free extern declarations (declare the two extern fns manually) and check non-empty results.
### S1.4 Acceptance Criteria
- [ ] `zig build test` exits 0 natively; the C-link test prints/asserts both library version strings.
- [ ] `zig build cross` exits 0; `file zig-out/cross/x86_64-linux-musl/nxdns` and the aarch64 one both report "statically linked".
- [ ] `./zig-out/bin/nxdns version` prints the version and Zig 0.16.0, exit 0.
- [ ] `./zig-out/bin/nxdns` (no args) exits 64 with usage on stderr.
- [ ] `git status` clean of build artifacts (gitignore works).
---
## Session S2: platform/address.zig
Pure module. No `std.Io` operations — only type conversions to/from `std.Io.net.IpAddress` values. Unit tests in-file.
### S2.1 Types + API
```zig
pub const NetAddress = union(enum) {
ip4: [4]u8,
ip6: [16]u8,
pub const Key = [17]u8; // tag byte (4 or 6) + address bytes, zero-padded for ip4
pub fn parse(text: []const u8) error{InvalidAddress}!NetAddress; // "1.2.3.4", "fd00::1"; no port, no brackets
pub fn format(self: NetAddress, w: *std.Io.Writer) std.Io.Writer.Error!void; // v4 dotted; v6 RFC 5952 lowercase, :: compression
pub fn key(self: NetAddress) Key;
pub fn fromIp(addr: std.Io.net.IpAddress) NetAddress; // drops port; IPv4-mapped IPv6 (::ffff:a.b.c.d) normalizes to .ip4
pub fn toIp(self: NetAddress, port: u16) std.Io.net.IpAddress;
pub fn eql(a: NetAddress, b: NetAddress) bool;
};
pub const Prefix = struct {
addr: NetAddress, // host bits zeroed on parse
bits: u8, // <= 32 for ip4, <= 128 for ip6
pub fn parse(text: []const u8) error{InvalidPrefix}!Prefix; // "192.168.1.0/24", "fd00:abcd::/48"
pub fn contains(self: Prefix, addr: NetAddress) bool; // family mismatch => false
pub fn format(self: Prefix, w: *std.Io.Writer) std.Io.Writer.Error!void;
};
/// Longest-prefix winner; ties broken by lower `priority` value. Returns null when nothing matches.
pub fn matchLongest(comptime T: type, entries: []const T, addr: NetAddress) ?*const T;
// T must have fields: prefix: Prefix, priority: i64
```
Implement parse/format by hand or delegate to `std.Io.net.Ip4Address/Ip6Address` parsing where it fits — but the RFC 5952 output rules (lowercase hex, longest zero-run compressed, no compression of a single group) must hold and be tested either way.
### S2.2 Acceptance Criteria
- [ ] Round-trip tests: parse→format is identity for canonical inputs (`"192.168.1.1"`, `"fd00::1"`, `"::"`, `"2001:db8::8:800:200c:417a"`).
- [ ] RFC 5952 tests: `"2001:0DB8:0:0:1::1"` formats as `"2001:db8::1:0:0:0:1"`-style rules — specifically: longest run compressed, single zero group NOT compressed, lowercase.
- [ ] `fromIp` on an IPv4-mapped IPv6 address yields `.ip4`.
- [ ] `Prefix.contains`: `192.168.1.0/24` contains `.1.5`, not `.2.5`; `/0` contains everything of its family; family mismatch false.
- [ ] `matchLongest`: `/24` beats `/16`; equal bits → lower priority value wins.
- [ ] All tests pass via `zig test src/platform/address.zig` (orchestrator wires them into `zig build test` later).
---
## Session S3: platform/tls_client.zig
Wrapper over `std.crypto.tls.Client` for upstream DoT (and reused by anything needing client TLS over a `std.Io.net.Stream`). See the API notes §std.crypto.tls.Client for exact init requirements.
### S3.1 API
```zig
pub const ErrorClass = enum { handshake, certificate, io, protocol };
pub fn classify(err: anyerror) ErrorClass;
pub const TlsStream = struct {
// all fields private in practice; struct is pinned: MUST NOT move after init
// (tls.Client holds its reader/writer by value)
pub const Options = struct {
host: []const u8, // SNI + verification name
ca: enum { system, insecure_skip_verify },
// buffers supplied by caller; read_buffer.len >= std.crypto.tls.Client.min_buffer_len
read_buffer: []u8,
write_buffer: []u8,
stream_read_buffer: []u8,
stream_write_buffer: []u8,
};
/// In-place init (pinned struct). `bundle` is scanned lazily for .system via
/// std.crypto.Certificate.Bundle.rescan when empty; caller owns bundle + lock lifetime.
pub fn init(
self: *TlsStream,
io: std.Io,
stream: *std.Io.net.Stream,
bundle: *std.crypto.Certificate.Bundle,
bundle_lock: *std.Io.RwLock,
gpa: std.mem.Allocator,
options: Options,
) InitError!void;
pub fn reader(self: *TlsStream) *std.Io.Reader; // plaintext
pub fn writer(self: *TlsStream) *std.Io.Writer; // plaintext
pub fn close(self: *TlsStream) void; // close_notify via Client.end, errors swallowed to log-level
};
```
- Entropy: 240 bytes via `io.random`; `realtime_now` via the Io clock (`io.now(.real)` — check the exact 0.16 name in `/home/mokhtar/app/zig/lib/std/Io.zig` before use).
- `classify` maps the stdlib error sets: cert/trust errors → `.certificate`; handshake alerts/negotiation → `.handshake`; ReadFailed/WriteFailed/connection errors → `.io`; everything else → `.protocol`. Write the mapping exhaustively over `std.crypto.tls.Client.InitError` — a `switch` with explicit arms, no `else => .protocol` catch-all for that set.
### S3.2 Tests
- Unit: `classify` mapping table (pick 6+ representative errors across the four classes).
- Integration (compiled only when `build_options.integration`): connect to a live host (`cloudflare-dns.com:853` — DoT port, TLS without ALPN), complete the handshake with `.ca = .system`, close cleanly. This test is NOT part of the default `zig build test` run and NOT part of blocking CI.
### S3.3 Acceptance Criteria
- [ ] `zig test src/platform/tls_client.zig` passes (unit tests only).
- [ ] `classify` covers `InitError` exhaustively (compiles with explicit arms — adding a new stdlib error breaks the build here, by design).
- [ ] Integration test compiles under `-Dintegration` (orchestrator runs it after wiring; a live-network failure is an environment finding, not a session failure).
---
## Session S4: platform/tls_server.zig
mbedTLS-backed server-side TLS termination exposing `std.Io.Reader`/`std.Io.Writer`, so `std.http.Server` and the DoT server can sit on top of any accepted TCP stream.
### S4.1 mbedTLS extern layer
Declare the needed mbedTLS API as extern fns/opaque types in this file (no `@cImport` — keep translate-c out of the build). Needed surface: `mbedtls_ssl_context`, `mbedtls_ssl_config`, `mbedtls_x509_crt`, `mbedtls_pk_context`, `mbedtls_entropy_context`, `mbedtls_ctr_drbg_context` + their init/free/setup/parse functions, `mbedtls_ssl_handshake`, `mbedtls_ssl_read`, `mbedtls_ssl_write`, `mbedtls_ssl_close_notify`, `mbedtls_ssl_set_bio`, `mbedtls_strerror`. Sizes: allocate contexts with the C sizes via opaque + `extern` allocation pattern — simplest correct approach: define `extern struct` mirrors is NOT acceptable (fragile); instead heap-allocate via wrapper C-callable `malloc(sizeof)` is also not available — so: declare the context structs as `opaque` and allocate them with `gpa.alignedAlloc(u8, .of(usize), mbedtls_ssl_context_size)` where the sizes come from a tiny C shim file `src/platform/mbedtls_shim.c` exporting `size_t nx_sizeof_ssl_context(void)` etc. The shim is owned by this session and added to build by the orchestrator (S4 must NOT edit build.zig — note the shim path in the completion report).
### S4.2 API
```zig
pub const ServerContext = struct {
// holds parsed cert chain + key + ssl_config + drbg; one per listener, reused across connections
pub fn init(gpa: std.mem.Allocator, cert_pem: [:0]const u8, key_pem: [:0]const u8) InitError!ServerContext;
pub fn deinit(self: *ServerContext, gpa: std.mem.Allocator) void;
};
pub const ServerStream = struct {
// pinned after accept(); owns the ssl_context for one connection
/// Performs the TLS handshake over an accepted TCP stream.
/// BIO callbacks bridge mbedtls_ssl_read/write to stream.reader/writer interfaces.
pub fn accept(
self: *ServerStream,
gpa: std.mem.Allocator,
ctx: *ServerContext,
io: std.Io,
stream: *std.Io.net.Stream,
read_buffer: []u8,
write_buffer: []u8,
) AcceptError!void;
pub fn reader(self: *ServerStream) *std.Io.Reader; // plaintext, implemented via Io.Reader vtable over mbedtls_ssl_read
pub fn writer(self: *ServerStream) *std.Io.Writer; // plaintext, via mbedtls_ssl_write
pub fn close(self: *ServerStream, gpa: std.mem.Allocator) void; // close_notify + free ssl_context
};
```
- Error mapping: negative mbedTLS return codes → Zig error set with named errors for the common cases (`CertParse`, `KeyParse`, `HandshakeFailed`, `PeerClosed`, `WantReadWrite` handled internally); include the raw code in a log via `mbedtls_strerror`.
- `MBEDTLS_ERR_SSL_WANT_READ/WANT_WRITE` loop inside read/write — never surfaces to callers.
### S4.3 Test fixture + loopback test
- Generate once and commit: `tests/fixtures/self_signed_cert.pem` + `self_signed_key.pem` (openssl, EC P-256, CN=localhost, SAN DNS:localhost + IP:127.0.0.1, 100-year validity — a fixture, not a secret; note "test fixture, private key intentionally committed" in a `tests/fixtures/README.md`).
- Loopback test (compiled only under `build_options.integration`): thread A: `IpAddress.listen` on 127.0.0.1:0 → accept → `ServerStream.accept` with the fixture → echo one message read back to the writer. Thread B (client): `std.crypto.tls.Client` with `.host = .no_verification`, `.ca = .no_verification` → write message → read echo → assert equality → clean close both sides. Drive both with `io.concurrent` on one `Threaded` instance.
### S4.4 Acceptance Criteria
- [ ] `ServerContext.init` with the fixture cert+key succeeds; with truncated PEM returns `error.CertParse`/`error.KeyParse` (unit tests, no network).
- [ ] Loopback echo test passes under `-Dintegration` (orchestrator wires + runs).
- [ ] No `@cImport` anywhere; extern decls + C shim only.
- [ ] `close` sends close_notify (verified in the loopback test by the client reading EOF without error after `end`).
---
## Session S5: Gitea CI
### S5.1 .gitea/workflows/ci.yml
House style: `runs-on: ubuntu-24.04`, `actions/checkout@v4`, top-level `env:` for versions (model: `~/app/phoenix_inertia_react_starter/.gitea/workflows/ci.yml`). Runner: x86_64, dind, full egress, `uses:` resolves against github.com. Cache is ephemeral (runner restarts daily) — `actions/cache@v4` allowed as best-effort, never load-bearing.
Jobs:
1. `test`: checkout → `mlugg/setup-zig@v2` with `version: 0.16.0` → `zig build test` → `zig build test -Dintegration` is NOT run here (loopback integration runs are wired by the orchestrator in a later pass once tests.zig includes them; leave a commented job stub with a TODO referencing milestone-1 integration wiring).
2. `cross`: checkout → setup-zig → `zig build cross` → assert both output binaries exist and `file` reports statically linked.
### S5.2 .gitea/workflows/live-tls.yml
`workflow_dispatch` only. Runs `zig build test -Dintegration` (which includes the live DoT handshake test once wired). Non-blocking by construction.
### S5.3 Acceptance Criteria
- [ ] `yamllint`-clean (or at minimum `python3 -c "import yaml,sys; yaml.safe_load(open('.gitea/workflows/ci.yml'))"` passes for both files).
- [ ] Workflow YAML uses only actions available from github.com (`actions/checkout@v4`, `mlugg/setup-zig@v2`, `actions/cache@v4`).
- [ ] No job depends on cache hits for correctness.
---
## Module Layout
```
AGENTS.md values + aim (exists)
PLAN.md source of truth (exists)
specs/milestone-1.md this file
specs/research/zig-0.16-api-notes.md stdlib ground truth (exists)
build.zig S1
build.zig.zon S1
.gitignore S1
src/main.zig S1 CLI dispatch stub
src/version.zig S1 build_options plumbing
src/tests.zig S1 test aggregator (orchestrator extends)
src/platform/address.zig S2 NetAddress/Prefix/matchLongest
src/platform/tls_client.zig S3 stdlib TLS client wrapper
src/platform/tls_server.zig S4 mbedTLS server wrapper
src/platform/mbedtls_shim.c S4 sizeof shims
tests/fixtures/self_signed_cert.pem S4
tests/fixtures/self_signed_key.pem S4
tests/fixtures/README.md S4
.gitea/workflows/ci.yml S5
.gitea/workflows/live-tls.yml S5
```
## File Ownership
| Files | Owner | Notes |
|---|---|---|
| build.zig, build.zig.zon, .gitignore, src/main.zig, src/version.zig, src/tests.zig | S1 | frozen after S1; orchestrator edits afterward |
| src/platform/address.zig | S2 | |
| src/platform/tls_client.zig | S3 | |
| src/platform/tls_server.zig, src/platform/mbedtls_shim.c, tests/fixtures/* | S4 | shim build-wiring done by orchestrator |
| .gitea/workflows/* | S5 | |
S2–S5 MUST NOT edit build.zig, build.zig.zon, or src/tests.zig. If a session needs a build change, it reports the exact needed change in its completion report; the orchestrator applies it.
## Acceptance Criteria (Milestone 1 Complete)
- [ ] `zig build test` exits 0 (aggregator includes address, tls_client unit, tls_server unit tests).
- [ ] `zig build test -Dintegration` exits 0 locally (loopback TLS echo; live DoT test may be skipped on network failure with a visible skip message).
- [ ] `zig build cross` produces two statically linked executables that print `nxdns version` output under qemu-user or on-target (checked manually for aarch64 if qemu absent).
- [ ] CI workflows valid YAML; `test` + `cross` jobs green on the Gitea runner.
- [ ] All files committed with GPG-signed, lowercase-message commits.
## Anti-Requirements
- No DNS packet code, no sockets beyond the tests, no SQLite usage beyond the link check — that is milestone 2+.
- No custom mbedtls_config.h, no cipher tuning, no session tickets.
- No `@cImport`/translate-c anywhere.
- No extra CLI behavior beyond the specified stubs.
- No Docker/systemd packaging yet.
- No third-party Zig packages.
- Do not "fix" or extend files another session owns — report, don't touch.
specs/research/zig-0.16-api-notes.md
+138
View File
@@ -0,0 +1,138 @@
# Zig 0.16.0 stdlib API notes (verified against ../zig at tag 0.16.0)
Ground truth for build sub-agents. Every claim below was read from the 0.16.0 source
(commit 24fdd5b7a4c1c8b5deb5b56756b9dbc8e08c86a8). When in doubt, re-check the source at
`/home/mokhtar/app/zig/lib/std/` — do not trust pre-0.16 knowledge; the std.Io migration
changed most of these APIs.
## std.Build (build.zig)
- `b.addExecutable(.{ .name, .root_module, ... })` — no target/optimize on the artifact.
Target/optimize/link_libc go on the module: `b.createModule(.{ .root_source_file, .target,
.optimize, .link_libc, .imports, ... })` (Module.CreateOptions, Module.zig:216).
- All C configuration is **module-level** (Step.Compile has none of these methods):
- `exe.root_module.addCSourceFile(.{ .file, .flags, .language })`
- `exe.root_module.addCSourceFiles(.{ .root, .files, .flags })` — `files` must be
**relative** paths; absolute paths panic.
- `exe.root_module.addIncludePath(lazy_path)`, `.addCMacro(name, value)`
- libc: `link_libc = true` in CreateOptions or `exe.root_module.link_libc = true`.
- Static lib: `b.addLibrary(.{ .linkage = .static, .name, .root_module })`
(`b.addStaticLibrary` does not exist). Link: `exe.root_module.linkLibrary(lib)`.
- Cross targets: `b.resolveTargetQuery(.{ .cpu_arch = .aarch64, .os_tag = .linux,
.abi = .musl })` or `std.Target.Query.parse(.{ .arch_os_abi = "x86_64-linux-musl" })`.
musl → static libc; for a fully static binary also set `exe.linkage = .static`.
- Options: `b.option(std.Build.LazyPath, "web-dist", "...")` accepts LazyPath.
`b.addOptions()` + `opts.addOption/addOptionPath` + `exe.root_module.addOptions("build_options", opts)`.
- Embedding an asset directory: one WriteFile step holding both a generated `index.zig`
and the assets: `const wf = b.addWriteFiles(); _ = wf.addCopyDirectory(dist, "assets", .{});
const idx = wf.add("index.zig", src);` then
`exe.root_module.addAnonymousImport("assets", .{ .root_source_file = idx })`.
`@embedFile` paths must stay inside that module root (`error.ImportOutsideModulePath`).
- Version pin: compare `@import("builtin").zig_version` (SemanticVersion) in comptime,
`@compileError` on mismatch.
- build.zig.zon: `.name` is an **enum literal** (`.nxdns`), `.fingerprint` is **required**
(compiler suggests the value on first run), plus `.version`, `.paths`, `.dependencies`
(`path`/`url`+`hash`/`lazy`), `.minimum_zig_version`.
## std.Io (Threaded backend, concurrency)
- `var t = std.Io.Threaded.init(gpa, .{});` — returned by value, must live at a **stable
address**. `const io = t.io();` `defer t.deinit();` (joins workers).
InitOptions: `stack_size`, `async_limit` (default cpus-1), `concurrent_limit`.
- `io.async(f, args) Future(R)` may run inline; `io.concurrent(f, args) !Future(R)`
guarantees its own unit of concurrency — use for server loops.
`future.await(io)`, `future.cancel(io)`.
- `Io.Group`: `.init`, `g.async(io, f, args)`, `g.concurrent(io, f, args)`, `g.await(io)`,
`g.cancel(io)`. Group task fns must return something coercible to `Cancelable!void`.
Cancellation: next cancelable Io call returns `error.Canceled` once; `io.recancel()`
re-arms; `io.swapCancelProtection(.blocked)` protects cleanup sections.
- **No SIGINT/SIGTERM handling in Threaded** (only no-op SIGIO/SIGPIPE). Install our own
`posix.sigaction`, set an atomic flag, cancel the group from the main task.
## std.Io.net
- UDP: `addr.bind(io, .{ .mode = .dgram })` → `Socket`.
Receive: `sock.receive(io, buf) !IncomingMessage` — fields `from: IpAddress`,
`data: []u8` (slice into buf), `flags.trunc`. Timeout variant: `receiveTimeout(io, buf,
timeout)`. Send: `sock.send(io, &dest_addr, data)`. Close: `sock.close(io)`.
(No recvFrom/sendTo names.)
- TCP server: `addr.listen(io, .{ .reuse_address = true })` → `net.Server`;
`srv.accept(io) !Stream`; `srv.deinit(io)`. `Stream.close(io)`, `.shutdown(io, how)`.
Shutdown of the listener makes a blocked accept fail `error.SocketNotListening`.
- TCP client: `addr.connect(io, .{ .timeout = ... })` → `Stream`.
`Io.Timeout = union(enum){ none, duration, deadline }` — **connect only**.
- **Stream reads/writes accept no timeout** in 0.16.0 (VTable netRead/netWrite have none;
Operation lacks net stream variants). Bound a TCP/TLS read by running it under
`io.concurrent` and cancelling the future (or shutdown the socket).
- Reader/Writer: `stream.reader(io, buf) Stream.Reader`; generic interface is
`&stream_reader.interface` (`*Io.Reader`); same for writer. Caller owns buffers; `&.{}`
legal. Concrete errors land in `.err`; interface returns ReadFailed/WriteFailed.
`Stream.Writer.sendFile` returns `error.Unimplemented` in 0.16.0.
- Known: net tests in stdlib are skipped (upstream issue 31388) — do not copy test.zig
patterns blindly.
## std.http.Client (DoH upstream)
- Construct by struct literal: `var client: std.http.Client = .{ .allocator = gpa, .io = io };`
`defer client.deinit();` Fields: `ca_bundle`, `ca_bundle_lock`, `now: ?Io.Timestamp`,
`connection_pool` (LRU, free_size 32), `read_buffer_size`, `write_buffer_size`.
- DoH POST flow (low-level; `fetch` hides the body—don't use it):
1. `var req = try client.request(.POST, uri, .{ .headers = .{ .content_type =
.{ .override = "application/dns-message" } } });`
2. `try req.sendBodyComplete(body_mut);` (body is `[]u8`, sets content-length + flush)
3. `var resp = try req.receiveHead(&redirect_buf);`
4. Check `resp.head.status`, `resp.head.content_type`.
5. `const rdr = resp.reader(&transfer_buf);` then read (`allocRemaining`/`readSliceShort`).
6. `req.deinit()` returns the connection to the pool.
- TLS: client TLS uses std.crypto.tls.Client internally with the client's `ca_bundle`;
lazily `rescan`s system roots when `client.now == null`
(`error.CertificateBundleLoadFailure` on failure). Pre-fill `ca_bundle` + set `now` to
skip the scan.
- **No per-request deadline** exists. Enforce total-budget timeouts via concurrent+cancel
(see std.Io note above).
## std.http.Server (web/API/DoH server)
- `var srv = std.http.Server.init(&reader.interface, &writer.interface);`
Loop for keep-alive: `while (srv.reader.state == .ready)` + catch HttpConnectionClosing.
- `var req = try srv.receiveHead();` — `req.head.method/.target/.content_type/...`,
`req.iterateHeaders()`. **Copy `head.target` before reading the body** (body reads
invalidate head memory). Body: `req.readerExpectContinue(buf)` (handles 100-continue)
or `req.readerExpectNone(buf)`.
- Simple response: `try req.respond(content, .{ .status = ..., .extra_headers = &.{...} });`
- Streaming/SSE: `var body = try req.respondStreaming(&.{}, .{ .respond_options = ... });`
With `content_length == null` → chunked. **Use an empty buffer** so each write drains
through; then `response.flush()` after each event suffices; `response.end()` to finish.
(With a non-empty buffer you must flush response.writer first — trap.)
- WebSocket upgrade exists (`upgradeRequested`/`respondWebSocket`) — unused by nxdns.
- No MIME table, no etag: set `content-type`/`cache-control` manually via extra_headers.
BodyWriter supports sendFile (contentLengthSendFile/chunkedSendFile).
## std.zon
- Parse: `std.zon.parse.fromSlice(T, gpa, src_z, ?*Diagnostics, .{})` for pointer-free T
(no free needed); `fromSliceAlloc` + `std.zon.parse.free(gpa, v)` for slice-bearing T.
Source must be `[:0]const u8`. No reader-based entry point.
Options: `ignore_unknown_fields`, `free_on_error`. Diagnostics: init `.{}`,
`deinit(gpa)`, print with `{f}`.
- Struct default field values apply for absent fields. Tagged unions: `.foo` (void) or
`.{ .foo = v }`. Enums, optionals (single level only — `??T` rejected), slices, arrays,
nested structs all fine. Error sets/unions, many-pointers, comptime_int rejected.
- Serialize: `std.zon.stringify.serialize(value, .{ .whitespace = true }, writer)`.
Streaming: `var s: std.zon.Serializer = .{ .writer = w, .options = .{} };` with
`beginStruct`/`field`/`end`.
## std.crypto.tls.Client (upstream DoT)
- `std.crypto.tls.Client.init(input: *Io.Reader, output: *Io.Writer, options) !Client`
Options: `.host = .{ .explicit = "dns.google" }` (or `.no_verification`),
`.ca = .{ .bundle = .{ .gpa, .io, .lock, .bundle } }` (or `.no_verification`/`.self_signed`),
`.write_buffer`, `.read_buffer` (input buffer ≥ `Client.min_buffer_len` =
`tls.max_ciphertext_record_len`), `.entropy: *const [240]u8` (fill via `io.random`),
`.realtime_now: Io.Timestamp`.
- After init: plaintext via `&client.reader` / `&client.writer` (held **by value** — the
Client must not move after init). `client.end()` sends close_notify. Errors in
`client.read_err` / `client.alert`.
- TLS 1.2/1.3. **No ALPN, no session resumption** (fine for DoT; DoH over HTTP/1.1 works
without ALPN in practice — verify against real upstreams in Phase 3).
- CA roots: `std.crypto.Certificate.Bundle` (note the path), `bundle.rescan(gpa, io, now)`.