mokhtar/nxdns
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity
Files
59d94df7225d10fc187b76a5c31af177bcd0e46c
nxdns/PLAN.md
T

32 KiB
Raw Blame History

nxdns — Implementation Plan v3.0 (Zig 0.16.0 Stable)

Source of truth for nxdns, a self-hosted DNS sinkhole written in Zig 0.16.0 stable. All stdlib claims in this document are verified against the 0.16.0 tag of the Zig repo (../zig).

There is no v1/v2 versioning. Scope is binary: a feature is in scope (and gets built) or out of scope (and does not). "Done" = everything in scope implemented, tested, documented.

1. Purpose

Fix three pain points in Pi-hole/AdGuard/Technitium:

  1. Clunky, dated admin UI.
  2. No config-as-code story (no way to export current state as a text artifact).
  3. Silent operational failures (cloudflared proxy-dns incident: broken DoH setup + unbounded error logs filled SD card).

Serves a household LAN (≈2–20 devices). Portfolio-grade public repo with extensive docs, self-hosted on Gitea.

2. Scope

2.1 In Scope

  • DNS server for LAN clients: UDP/53, TCP/53, DoH server, DoT server.
  • Upstream resolution: DoH (HTTP/1.1), DoT.
  • Local DNS records (A/AAAA/CNAME) + conditional forwarding (zone → designated resolver, plain UDP/TCP allowed).
  • Domain filtering: blocklists (hosts/domains/ABP), custom rules (allow/block; exact, parent-walk, wildcard), CNAME uncloaking (depth 8), per-group safe-search rewrite.
  • DNS caching: positive + negative, in-memory only.
  • Client/group model: IPv4 + IPv6 parity, per-client group assignment, per-group source assignments.
  • Query logging + analytics: async batched writes to SQLite (WAL), retention cleanup, dashboard + time buckets, live SSE stream.
  • Web app + REST API: LAN/Tailscale admin UI, optional password auth, OpenAPI schema + CI contract tests.
  • Observability: upstream health API + UI, disk monitor with UI banner, bounded log rotation, Prometheus /metrics.
  • Ops: DB-as-truth config, nxdns export/import (ZON), scheduled + manual blocklist updates, TLS cert watcher + reload, auto-migration on upgrade, systemd service + Dockerfile + compose.

2.2 Out of Scope (permanent scope decisions, not deferrals)

  • Regex rules. Wildcards + parent-walk cover the real use cases; regex on the DNS hot path means ReDoS exposure plus an immature dependency or a homegrown engine. Regex lines in blocklists are counted, skipped, and the skip count is surfaced in the UI.
  • DHCP server.
  • DNSSEC validation (DO bit passthrough only).
  • DoQ (QUIC), HTTP/2 upstream transport.
  • Clustering / distributed state.
  • Prebuilt binaries / published Docker images / project website. Repo + documented build-it-yourself path only.

3. Locked Decisions

3.1 Toolchain

  • Compiler: Zig 0.16.0 stable. build.zig asserts major/minor.
  • Targets: x86_64-linux-musl and aarch64-linux-musl, both first-class in CI. Release binaries are static musl — fully self-contained, no glibc version coupling on the Pi.

3.2 Concurrency: std.Io (Decision E)

Zig 0.16 stdlib is built on the std.Io interface: std.Io.net owns networking (std.net in its old form is gone) and std.http.Client requires an io: Io. Therefore:

  • Io is the injected platform abstraction. Every component that does I/O takes io: Io. No project-owned wrapper interfaces around it — a second abstraction over an abstraction with one consumer is waste.
  • Backend: std.Io.Threaded by default — the mature, debuggable path; at ≤20 devices throughput is a non-issue. std.Io.Evented (io_uring) is selectable via config flag; the code is backend-agnostic by construction, so this is a switch, not a refactor.
  • No hand-written thread pool. Io async/concurrent/Group covers task management.
  • Core domain modules (dns, filter, cache) stay pure: no Io, no sockets — bytes in, bytes out. Only servers, upstream clients, and storage touch Io.

3.3 TLS (Decision D)

Verified: 0.16.0 ships std.crypto.tls.Client only. There is no server-side TLS in the stdlib.

  • Upstream client TLS (DoH/DoT): stdlib std.crypto.tls via std.http.Client / direct. Maintained upstream, exercised by the stdlib itself.
  • Local server TLS (DoH/DoT termination): vendored mbedTLS. Audited, embedded-focused, compiles cleanly with the Zig build system for both targets. We do not hand-roll a TLS server in a security-sensitive service.
  • TLS glue lives in platform/tls_client.zig (stdlib wrapper: error classification, deadlines, retry policy) and platform/tls_server.zig (mbedTLS wrapper exposing std.Io.Reader/Writer so std.http.Server composes on top unchanged).

3.4 C Dependency Policy

  • sqlite3 + mbedtls, both vendored and compiled from source in build.zig. Pinned versions, hermetic cross-compilation, no system libs.
  • SQLite: amalgamation + our own thin wrapper (storage/db.zig: open/close, prepared-statement cache, typed row mapping, error translation). No third-party binding — the needed surface is small enough to own (Decision G).
  • Zig package deps: minimized; anything adopted is pinned in build.zig.zon. No TOML/regex/HTTP packages needed under current decisions.

3.5 Config Format + Truth Model (Decision F)

  • DB is truth. Config file format is ZON (std.zon parse + stringify — typed parsing into config structs, exact round-trip, stdlib-maintained, comments supported). No TOML: a third-party parser plus a hand-written serializer is two failure surfaces in the correctness-critical bootstrap/round-trip path, bought for syntax familiarity.
  • First start: if DB empty and /etc/nxdns/config.zon exists, validate → seed DB. Subsequent starts ignore the file.
  • nxdns export [--out file.zon] dumps DB state as canonical ZON. nxdns import <file.zon> validates + replaces DB contents (--force if DB non-empty). Export/import = backup + host migration, not upgrades (§3.7).
  • No file watcher, no auto-regeneration.

3.6 Storage Layout (Decision H)

Two SQLite files with opposite write profiles, isolated from each other:

  • config.db — small, precious, rarely written: groups, clients, prefixes, upstreams, blocklist source metadata, rules, local records, forward zones, settings, schema version.
  • querylog.db — high-churn, large, expendable: query log + its own private domains dimension table. Client identity stored as IP text, not a FK into config — log rows are immutable facts and must not point at mutable config rows. If querylog.db is missing or corrupt at startup, rename aside, recreate, keep serving. Log loss is not an outage.
  • No cross-DB references. Retention/VACUUM churn never touches config.db; config backup is a copy of a tiny file.

3.7 Upgrades: Auto-Migration (Decision J)

  • config.db: numbered, sequential SQL migration steps compiled into the binary. At startup: read schema version row, apply newer steps inside a transaction, continue. Operator upgrade = install binary, restart.
  • querylog.db: no migrations. On schema mismatch: rename aside, recreate fresh.

3.8 Blocklist Storage (Decision A)

Blocklist domains are not stored in SQLite — they are a cache of re-downloadable remote artifacts, not config or state:

  • Each source compiles to /var/lib/nxdns/blocklists/<source_id>.list: normalized, one domain per line, small header (source URL, fetch time, count, checksum). Wildcard/regex-flavored lines: wildcards go to <source_id>.wild; regex lines are counted + skipped (count in metadata → UI).
  • config.db keeps source metadata only (blocklist_sources).
  • Startup + post-update: parse files into the immutable in-memory matcher (RCU swap, §9.5).
  • Corruption recovery is per-file: checksum mismatch → re-download one list.

3.9 Rule Model (Decision B)

Rule kinds: exact, parent-walk (implicit via candidate chain), wildcard (* segment patterns, e.g. *.doubleclick.net, ads.*.example.com). Actions: allow | block. Kind is an explicit column — the model is extensible without breakage, but regex stays out of scope (§2.2).

3.10 Filtering Precedence

  1. Exact/parent allow rules
  2. Exact/parent block rules
  3. Wildcard allow rules
  4. Wildcard block rules
  5. Blocklist domains
  6. Blocklist wildcards

Tie-break at same specificity: allow wins.

3.11 Network Posture

  • Default web bind: LAN/Tailscale-friendly (non-loopback allowed).
  • Auth optional (empty web.password disables it). Set → argon2id-hashed password, session cookie (HttpOnly, SameSite=Lax, Secure on HTTPS).
  • API per-IP rate limit on by default.

3.12 Addressing

IPv4 + IPv6 full parity for: client identity, rate limiting, logging, group assignment, prefix-based group matching. Dual-stack assumed.

3.13 Filesystem Layout (FHS)

  • /etc/nxdns/config.zon — bootstrap (first start only).
  • /var/lib/nxdns/config.db, /var/lib/nxdns/querylog.db
  • /var/lib/nxdns/blocklists/*.list|*.wild
  • /var/log/nxdns/nxdns.log — only in file output mode; default is stderr → journald.

3.14 Frontend Stack (Decision I)

  • Vite + React + TypeScript + Tailwind; TanStack Router + TanStack Query. SPA, no SSR.
  • Built assets embedded in the binary at compile time: single self-contained artifact, no asset-path config, no binary/UI skew. zig build accepts the dist path; the release pipeline runs npm run build first (CI always does).
  • Dev-mode flag serves assets from disk so UI iteration needs no Zig rebuild.

3.15 CI

  • Self-hosted Gitea + Gitea Actions runner.
  • Jobs: zig build test, fuzz smoke, integration tests, OpenAPI contract tests (live server validated against openapi.yaml), frontend build, cross-compile both targets. aarch64 test execution via qemu-user if the runner is x86_64.

4. Architecture Overview

Client DNS Query
   |
   v
[UDP / TCP / DoH / DoT servers]        (std.Io; mbedTLS terminates DoH/DoT)
   |
   v
[Request Handler]
   |- rate limit check
   |- parse DNS packet
   |- group resolution (v4/v6 parity)
   |- local records check          -> answer locally
   |- forward-zone check           -> forward to designated resolver (bypasses filtering)
   |- allow/block evaluation (rules + blocklists, per group)
   |- safe-search rewrite (per group)
   |- cache lookup
   |- upstream query (DoH/DoT pool) + health tracking
   |- CNAME uncloaking check
   |- cache store
   |- async query log + SSE fanout
   v
Response to client

Cross-cutting: ConfigManager (bootstrap/import/export/settings), BlocklistManager (fetch/compile/swap), Storage (two SQLite DBs), Cache, UpstreamHealth, DiskMonitor, Auth, Web API (REST + SSE + metrics).

5. Source Layout

src/
  main.zig               # Io backend instantiation, wiring
  app.zig
  version.zig

  platform/
    address.zig          # NetAddress: v4/v6 parity type + canonical key (was net_adapter)
    tls_client.zig       # stdlib TLS wrapper: deadlines, error classes, retry policy
    tls_server.zig       # mbedTLS wrapper exposing std.Io.Reader/Writer

  dns/                   # pure: bytes in, bytes out; no Io
    types.zig  header.zig  name.zig  question.zig  record.zig  edns.zig  packet.zig

  server/
    udp_server.zig  tcp_server.zig  doh_server.zig  dot_server.zig
    handler.zig  rate_limiter.zig  shutdown.zig

  upstream/
    doh_client.zig  dot_client.zig  pool.zig  health.zig

  filter/                # pure
    matcher.zig  rules.zig  wildcard.zig
    parser_hosts.zig  parser_domains.zig  parser_abp.zig
    fetcher.zig  compiler.zig            # list download -> compiled .list/.wild files
    safesearch.zig

  local/                 # pure
    records.zig          # local A/AAAA/CNAME answers
    forward_zones.zig    # zone -> designated resolver matching

  cache/
    dns_cache.zig

  storage/
    db.zig               # sqlite3 thin wrapper
    config_schema.zig  migrations.zig
    querylog_schema.zig
    repositories/
      clients_repo.zig  groups_repo.zig  rules_repo.zig  sources_repo.zig
      queries_repo.zig  settings_repo.zig  upstreams_repo.zig  local_repo.zig
    logger.zig  retention.zig  disk_monitor.zig

  config/
    model.zig  bootstrap.zig  import.zig  export.zig  validate.zig   # all ZON via std.zon

  web/
    server.zig  router.zig  auth.zig  sse.zig  static.zig  metrics.zig  openapi.zig
    handlers/
      auth.zig  stats.zig  queries.zig  clients.zig  groups.zig  blocklists.zig
      rules.zig  local.zig  lookup.zig  pause.zig  settings.zig
      upstream_health.zig  certs.zig  health.zig  version.zig

web/                     # Vite + React + TS + Tailwind + TanStack
vendor/                  # sqlite3 amalgamation, mbedtls (pinned)
docs/                    # operator/ architecture/ config-reference/ api/
tests/                   # dns/ integration/ fuzz/

6. DNS Behavior

6.1 Protocol

  • RFC 1035 parse/encode. EDNS OPT passthrough. DO bit passthrough (no validation). ECS: strip (default) or forward.
  • Process first question only. Preserve query ID and RD flag.
  • Malformed: FORMERR when a response is possible; silent drop for severely truncated.

6.2 Blocked Response Modes

blocking.response: zero (A=0.0.0.0, AAAA=::) or nxdomain. TTL = blocking.ttl.

6.3 CNAME Uncloaking

Walk chain to depth 8; any target hitting block logic → synthesize blocked response.

6.4 Local Records

  • Table-backed A/AAAA/CNAME records, group-independent. Matched before filtering.
  • CNAME targets resolve through the normal pipeline (uncloaking rules apply).

6.5 Conditional Forwarding

  • forward_zones: suffix match (lan.home, 10.in-addr.arpa, …) → designated resolver (udp://192.168.1.1:53 style; plain UDP/TCP permitted — these are local infra resolvers).
  • Matching queries bypass filtering and blocklists; responses cached normally.

7. Filtering Engine

7.1 Evaluation

For {domain, group_id} (the qtype travels with the query for logging and response synthesis, not for matching):

  1. Normalize: lowercase, trim trailing dot.
  2. Build candidate chain (full, parent1, parent2, …).
  3. Explicit rules per §3.10 precedence, evaluated against every candidate in the chain.
  4. Group's blocklist domains (hash set over compiled lists), matched against the query name only.
  5. Group's blocklist wildcards, matched against every proper parent of the query name.
  6. No match → allow.

Blocklist entries do not parent-walk; only rules do (§3.9). ABP ||x.y^ emits both a domain entry x.y and a wildcard entry x.y, which together give domain-and-subdomains semantics.

7.2 Group Assignment

  • Per source IP: (1) exact match in clients, (2) longest-prefix match in client_prefixes (ties: longer prefix, then priority), (3) default group.
  • Auto-materialization: first query from an unseen IP inserts a clients row (hand_edited=0, first_seen=now) for UI visibility and stable group assignment. The query log does not FK to it (§3.6).
  • Retention drops hand_edited=0 clients with no queries in retention_days.

7.3 Reload

New immutable matcher built from DB + compiled list files → swap under an std.Io.RwLock with a generation counter. Readers take the shared lock for the microseconds of one evaluate; the writer takes the exclusive lock only for the swap, and the source status table is installed in the same critical section, so a failed reload publishes neither. (Deliberate deviation from "readers lock-free": freeing the old snapshot without a lock needs epoch-based reclamation, unjustifiable at household scale — see specs/milestone-5.md S8.3.)

7.4 Safe-Search

Per-group boolean. Rewrites known engine domains to their safe-search CNAME targets.

8. Cache

  • Key: normalized qname + qtype + qclass + DO bit + forwarded ECS subnet (only when ecs_mode=forward).
  • Value: full response bytes, stored_at, expires_at, positive/negative flag.
  • Positive TTL: min(answer TTLs) bounded by sanity ceiling. Negative: SOA/minimum capped by cache.negative_ttl_max (0 disables).
  • Hit path: clone bytes, overwrite transaction ID, decrement visible TTLs; TTL ≤ 0 → miss.
  • Bounded by cache.size; CLOCK eviction; opportunistic + periodic expiry sweep.
  • In-memory only, no persistence. Cold start refills within a minute; SD-corruption risk isn't worth pre-warmed entries.

9. Upstream Resolution

  • Schemes: https://… → DoH, tls://host:853 → DoT.
  • Ordered by priority; sequential attempt; per-upstream failure counters; exponential backoff with jitter; success resets.
  • UpstreamHealth per upstream: last_success_at, last_error_at, last_error_message, rolling success rate, consecutive failures, backoff-until. Exposed via GET /api/upstream/health, dashboard, /metrics, and nxdns check.
  • DoH client: std.http.Client with content-type/accept: application/dns-message; strict status + payload checks.
  • platform/tls_client.zig enforces per-connection read/write deadlines, classifies TLS errors explicitly, retries with backoff. Integration tests cover timeout/hang scenarios so compiler upgrades can't silently regress them.
  • Connect, read, and total-budget timeouts each configurable.

10. Rate Limiting

  • DNS: default 1000 req / 60s per client IP (v4/v6 keyed alike); exceeded → REFUSED.
  • API: per-IP token bucket, on by default; separate SSE connection cap per IP; localhost relaxed (configurable).
  • Limiter maps bounded; stale-key windowed sweep.

11. Storage

11.1 Pragmas (both DBs)

journal_mode=WAL, synchronous=NORMAL, foreign_keys=ON, busy_timeout set.

11.2 config.db Schema

CREATE TABLE schema_version (version INTEGER NOT NULL);

CREATE TABLE groups (
    id INTEGER PRIMARY KEY,
    name TEXT NOT NULL UNIQUE,
    safe_search INTEGER NOT NULL DEFAULT 0
);
INSERT OR IGNORE INTO groups (id, name) VALUES (1, 'default');

CREATE TABLE clients (
    id INTEGER PRIMARY KEY,
    ip TEXT NOT NULL UNIQUE,            -- canonical text form (v4 dotted / v6 RFC 5952)
    name TEXT,
    group_id INTEGER NOT NULL REFERENCES groups(id),
    hand_edited INTEGER NOT NULL DEFAULT 0,
    first_seen INTEGER NOT NULL,
    last_seen INTEGER NOT NULL
);

CREATE TABLE client_prefixes (
    id INTEGER PRIMARY KEY,
    prefix TEXT NOT NULL UNIQUE,        -- "192.168.1.0/24", "fd00:abcd::/48"
    group_id INTEGER NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
    priority INTEGER NOT NULL DEFAULT 100
);

CREATE TABLE upstreams (
    id INTEGER PRIMARY KEY,
    url TEXT NOT NULL UNIQUE,
    priority INTEGER NOT NULL DEFAULT 100,
    enabled INTEGER NOT NULL DEFAULT 1
);

CREATE TABLE blocklist_sources (
    id INTEGER PRIMARY KEY,
    url TEXT NOT NULL UNIQUE,
    name TEXT NOT NULL,
    enabled INTEGER NOT NULL DEFAULT 1,
    is_suggested INTEGER NOT NULL DEFAULT 0,
    last_updated INTEGER,
    domain_count INTEGER NOT NULL DEFAULT 0,
    wildcard_count INTEGER NOT NULL DEFAULT 0,
    skipped_regex_count INTEGER NOT NULL DEFAULT 0,
    checksum TEXT
);

CREATE TABLE group_sources (
    group_id INTEGER NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
    source_id INTEGER NOT NULL REFERENCES blocklist_sources(id) ON DELETE CASCADE,
    PRIMARY KEY (group_id, source_id)
);

CREATE TABLE rules (
    id INTEGER PRIMARY KEY,
    group_id INTEGER NOT NULL REFERENCES groups(id) ON DELETE CASCADE,
    pattern TEXT NOT NULL,
    kind TEXT NOT NULL CHECK(kind IN ('exact','wildcard')),
    action TEXT NOT NULL CHECK(action IN ('allow','block')),
    created_at INTEGER NOT NULL
);

CREATE TABLE local_records (
    id INTEGER PRIMARY KEY,
    name TEXT NOT NULL,
    rtype TEXT NOT NULL CHECK(rtype IN ('A','AAAA','CNAME')),
    value TEXT NOT NULL,
    ttl INTEGER NOT NULL DEFAULT 300,
    UNIQUE(name, rtype, value)
);

CREATE TABLE forward_zones (
    id INTEGER PRIMARY KEY,
    zone TEXT NOT NULL UNIQUE,
    resolver TEXT NOT NULL              -- "udp://192.168.1.1:53"
);

CREATE TABLE settings (key TEXT PRIMARY KEY, value TEXT NOT NULL);

11.3 querylog.db Schema

CREATE TABLE domains (
    id INTEGER PRIMARY KEY,
    domain TEXT NOT NULL UNIQUE
);

CREATE TABLE query_log (
    id INTEGER PRIMARY KEY,
    timestamp INTEGER NOT NULL,
    domain_id INTEGER NOT NULL REFERENCES domains(id),
    client_ip TEXT NOT NULL,            -- text, not a FK: log rows are immutable facts
    qtype INTEGER,
    blocked INTEGER NOT NULL,
    block_reason TEXT,
    response_time_us INTEGER,
    cache_hit INTEGER,
    upstream TEXT
);
CREATE INDEX idx_query_log_ts ON query_log(timestamp);
CREATE INDEX idx_query_log_client ON query_log(client_ip);
CREATE INDEX idx_query_log_domain ON query_log(domain_id);

11.4 Query Logger

  • In-memory buffer, mutex guarded, hard cap query_log_buffer_max (default 10000).
  • Flush: batch size (default 100) or max interval (default 100ms).
  • Privacy transforms (hide_domains / hide_client_ips) applied before persist + SSE fanout.
  • Backpressure: buffer full → drop oldest unflushed entry, increment monotonic queries_dropped (exposed in /api/health + /metrics). SSE fanout precedes buffer insert, so live viewers still see dropped-from-persistence entries.

11.5 Retention

Periodic delete of rows older than retention_days; scheduled checkpoint/VACUUM on querylog.db only.

11.6 Disk Discipline (cloudflared lesson)

  • Log output default: stderr (journald rotates). File mode: rotate at log.max_size_mb (default 50), keep log.max_files (default 5).
  • DiskMonitor samples DB + log-dir sizes every 60s:
    • free < disk.warn_free_mb → UI banner + /api/health degraded.
    • free < disk.min_free_mb → stop non-essential writes (blocklist updates, log flushes); buffer continues under backpressure policy.
  • Upstream error log lines deduplicated per-upstream at 1/minute.

12. Configuration

12.1 Bootstrap ZON Shape

.{
    .upstream = .{
        .servers = .{ "https://cloudflare-dns.com/dns-query", "tls://dns.google:853" },
        .connect_timeout_ms = 2000,
        .read_timeout_ms = 3000,
    },
    .dns = .{
        .bind_ipv4 = "0.0.0.0",
        .bind_ipv6 = "::",
        .port = 53,
        .rate_limit = 1000,
        .rate_window_seconds = 60,
    },
    .blocking = .{ .response = .zero, .ttl = 5 },       // .zero | .nxdomain
    .cache = .{ .size = 10000, .negative_ttl_max = 3600 },
    .web = .{
        .enabled = true,
        .bind = "0.0.0.0",
        .port = 8080,
        .password = "",                                  // empty => auth disabled
        .session_ttl_hours = 24,
        .api_rate_limit_per_min = 300,
        .sse_max_connections_per_ip = 3,
    },
    .doh_server = .{ .enabled = false, .bind = "0.0.0.0", .port = 443,
                     .cert_path = "/etc/nxdns/cert.pem", .key_path = "/etc/nxdns/key.pem" },
    .dot_server = .{ .enabled = false, .bind = "0.0.0.0", .port = 853,
                     .cert_path = "/etc/nxdns/cert.pem", .key_path = "/etc/nxdns/key.pem" },
    .edns = .{ .ecs_mode = .strip },                     // .strip | .forward
    .local_records = .{ .{ .name = "nas.lan", .rtype = .A, .value = "192.168.1.10" } },
    .forward_zones = .{ .{ .zone = "lan.home", .resolver = "udp://192.168.1.1:53" } },
    .logging = .{
        .level = .info,
        .retention_days = 30,
        .query_log_buffer_max = 10000,
        .hide_domains = false,
        .hide_client_ips = false,
        .output = .stderr,                               // .stderr | .syslog | .file
        .file_path = "/var/log/nxdns/nxdns.log",
        .max_size_mb = 50,
        .max_files = 5,
    },
    .disk = .{ .min_free_mb = 200, .warn_free_mb = 500 },
    .blocklist_update = .{ .enabled = true, .interval_hours = 24 },
}

12.2 Validation

At least one upstream; ports in range; cert+key readable if DoH/DoT server enabled; resolver URLs parseable. nxdns check runs the validator + probes upstreams.

12.3 Settings Semantics

Scalars in settings(key, value); ordered/structured items in dedicated tables. Restart-required settings flagged; UI shows "restart to apply" banner.

13. Web / API

13.1 Endpoints

  • POST /api/auth/login, POST /api/auth/logout
  • GET /api/stats?period=…, GET /api/stats/timeseries?period=…
  • GET /api/queries (filter + paginate), GET /api/queries/live (SSE, per-IP cap)
  • GET/PUT /api/clients/{id}
  • GET/POST/PUT/DELETE /api/groups…, /api/blocklists…, /api/rules…, /api/local-records…, /api/forward-zones…
  • POST /api/blocklists/update
  • GET /api/lookup?domain=…&group_id=…
  • GET/POST /api/pause
  • GET/PUT /api/settings
  • GET /api/upstream/health
  • POST /api/certs/reload
  • GET /api/health — overall + disk + upstream + queries_dropped rollup
  • GET /metrics — Prometheus text exposition: query counters (total/blocked/cached), per-upstream health, cache stats, queries_dropped, disk gauges
  • GET /api/version, GET /api/openapi.yaml

13.2 OpenAPI

Hand-maintained openapi.yaml, served + rendered into docs/api/. CI contract tests spin up a seeded server, hit every documented endpoint, and validate bodies/status/auth against the schema. Drift fails CI.

14. Frontend

Pages: Dashboard (stats + upstream health + disk), Query log, Live log, Clients, Groups, Blocklists, Rules, Local DNS (records + forward zones), Domain lookup, Settings.

Requirements: responsive desktop/mobile; route loaders for initial fetch; TanStack Query for cache/retries; error/loading states on every data view; works with auth enabled or disabled; restart-required banner.

15. CLI

  • nxdns run — start the server.
  • nxdns check — validate config, probe upstreams, test cert readability; nonzero exit on failure.
  • nxdns export [--out file.zon]
  • nxdns import <file.zon> [--force]
  • nxdns version — app version, Zig version string, build date, git commit.

16. Implementation Order

Phase 0 — Build Baseline

Scaffold tree; build.zig with 0.16 assertion, musl targets, vendored sqlite3 + mbedtls compiling; version plumbing; Gitea Actions workflows (test, integration, fuzz smoke, OpenAPI lint, frontend build). Exit: cross-compiled hello-world linking both C deps on both targets; CI green.

Phase 1 — Platform Layer

platform/address.zig (v4/v6 parity + canonical keys); platform/tls_client.zig (deadlines, error classes); platform/tls_server.zig (mbedTLS handshake → std.Io.Reader/Writer). Exit: UDP echo over std.Io; TLS client handshake against a real host; mbedTLS server terminating a loopback TLS connection.

Phase 2 — DNS Core

Types/header/name/question/record/packet; parser + encoder tests + fuzz target; EDNS + DO passthrough. Exit: unit + fuzz smoke pass.

Phase 3 — Resolver Transport

UDP server, TCP server, DoH + DoT upstream clients, pool + failover/backoff + health. Exit: A/AAAA forwarding over UDP + TCP; health populated.

Phase 4 — Storage + Config

SQLite wrapper; config.db schema + migration runner; querylog.db schema + recreate-on-mismatch; repositories; ZON bootstrap + import/export; nxdns check. Exit: first start seeds DB from ZON; export → import round-trips byte-stable.

Phase 5 — Filtering + Local DNS

Rule matcher (exact/parent/wildcard); blocklist parsers → compiled file format; fetcher + scheduled update; RCU swap; per-group safe-search; local records; forward zones. Exit: precedence table validated by tests; local zone answers + conditional forwards work.

Phase 6 — Cache + Rate Limit + Logging + Disk Monitor

TTL cache + reconstruction; v4/v6 rate limiter; async query logger + backpressure + retention; DiskMonitor + rotation + error-log dedup. Exit: disk thresholds trigger degradation + drop counters in integration test.

Phase 7 — Handler Integration

Full pipeline composition; CNAME uncloaking; pause/resume. Exit: end-to-end DNS flow with blocking, local records, cache, failover.

Phase 8 — Web / API / SSE / Auth / Metrics

HTTP server + router; handlers; SSE; optional auth; API rate limiting; /metrics; OpenAPI served + contract tests; embedded frontend + dev-mode disk serving. Exit: frontend fully drives config and operations; contract tests green.

Phase 9 — Local DoH/DoT Endpoints

DoH server + DoT server on platform/tls_server.zig; cert watcher + reload. Exit: LAN client resolves via DoH and DoT against local certs.

Phase 10 — Packaging + Ops + Docs

systemd unit (AmbientCapabilities=CAP_NET_BIND_SERVICE, hardened, writable /var/lib/nxdns + optional /var/log/nxdns); Dockerfile + compose (53/udp+tcp, 8080; mounts /etc/nxdns, /var/lib/nxdns); operator/architecture/config-reference/API docs. Exit: documented deployment works end-to-end on the Pi 5.

17. Testing Strategy

  • Unit: DNS encode/decode; rule precedence + wildcard matcher; cache put/get/TTL rewrite; ZON bootstrap + export round-trip; rate limiter; migration runner (fresh + stepwise upgrade).
  • Fuzz: DNS parser malformed-packet fuzzing; blocklist parser fuzzing.
  • Integration: UDP/TCP query path; blocked path; allow-over-block; wildcard precedence; CNAME uncloaking block; local records + forward zones; upstream failover/backoff/health; disk-full degradation; querylog.db corruption recovery; API CRUD; auth on/off; SSE; contract tests.
  • Manual: dig @pi example.com / blocked domain / local record; DoH/DoT client checks; dashboard + live log.

18. Performance Targets

  • Sustained ≥ 100 qps on Raspberry Pi 5.
  • Blocklist lookup p95 < 1 ms.
  • Cached response p95 < 5 ms.
  • Memory with ~1M blocked domains < 100 MB.
  • Stripped static binary < 10 MB per arch (excluding embedded frontend assets; < 15 MB with them).

19. Security

  • Sanitize all DNS and API inputs; prepared statements everywhere.
  • argon2id password hashing; random session tokens with expiry.
  • No secrets in logs. TLS keys readable by service user only.
  • systemd hardening + least privilege.
  • mbedTLS + sqlite3 versions pinned; upgrades are deliberate, reviewed bumps.

20. Success Criteria

  1. nxdns run starts cleanly on Zig 0.16.0 stable, static musl, both arches.
  2. UDP + TCP resolution works; blocked domains return the configured response; precedence per §3.10.
  3. Local records answer; conditional forwarding works.
  4. Cache hit path returns valid ID-adjusted responses.
  5. Upstream failover + backoff works; health in UI, API, /metrics.
  6. Disk-fill degrades gracefully; no silent log-flood failure mode.
  7. Web UI + API provide full admin functionality; OpenAPI contract tests green.
  8. nxdns export round-trips via nxdns import.
  9. Query logging, stats, SSE live stream work; querylog.db corruption self-heals.
  10. Local DoH + DoT endpoints serve LAN clients.
  11. Schema upgrade = install + restart (migration test proves it).
  12. All suites green in Gitea CI for both targets.
  13. Operator, architecture, config-reference, and API docs complete.

21. Working Notes

  • dns/, filter/, local/, cache/ stay pure (no Io, no sockets). Servers, upstream clients, and storage take io: Io.
  • Verify stdlib behavior against ../zig (tag 0.16.0) instead of memory — the std.Io migration invalidated older knowledge once already.
  • Keep OpenAPI in sync with handlers (CI enforced).
  • When a plan decision and reality diverge during build: surface 2–3 alternatives with tradeoffs, get a call, then execute.

Decision Log

# Decision
A Blocklists compile to flat files under /var/lib/nxdns/blocklists/; DB stores source metadata only
B Rule kinds: exact, parent-walk, wildcard. Regex permanently out of scope
C In scope: local DoH/DoT server, local records, conditional forwarding. Out: HTTP/2, DoQ, DHCP, DNSSEC, clustering
D mbedTLS (vendored) terminates server TLS; stdlib TLS for upstream client
E std.Io injected everywhere; Threaded default backend, io_uring via flag; no custom thread pool
F Config format: ZON via std.zon; DB is truth; export/import for backup + host moves
G SQLite vendored amalgamation + own thin wrapper
H Two DBs: config.db (precious) + querylog.db (expendable, self-contained, client IP as text)
I Frontend embedded in binary; dev flag serves from disk; static musl release builds
J Auto-migration for config.db at startup; querylog.db recreated on mismatch
— Safe-search per-group; Prometheus /metrics in scope; CI on self-hosted Gitea Actions
Reference in New Issue View Git Blame Copy Permalink