project plan, values, milestone 1 spec, zig 0.16 api research

2026-07-31 23:16:57 +02:00
commit b609176a0b
4 changed files with 1148 additions and 0 deletions
@@ -0,0 +1,290 @@
+# Milestone 1: Build Baseline + Platform Layer
+
+Goal: `zig build test` green natively; static musl executables for x86_64-linux and aarch64-linux linking pinned sqlite3 + mbedTLS; platform modules (address, tls_client, tls_server) implemented and tested; Gitea CI green. No DNS logic in this milestone.
+
+Read first: `AGENTS.md` (values), `specs/research/zig-0.16-api-notes.md` (verified stdlib facts — pre-0.16 API knowledge is stale and MUST NOT be used). The Zig source of truth is `/home/mokhtar/app/zig` at tag `0.16.0`.
+
+## Sessions
+
+Five sessions. S1 runs first, alone. S2, S3, S4, S5 run in parallel after S1 is verified. The orchestrator (not any session) wires `src/tests.zig` imports and any `build.zig` additions afterward.
+
+```
+S1 ──> { S2, S3, S4, S5 } ──> orchestrator integration
+```
+
+---
+
+## Session S1: Build Baseline
+
+### S1.1 build.zig.zon
+
+- `.name = .nxdns`, `.version = "0.1.0"`, `.minimum_zig_version = "0.16.0"`, `.paths = .{""}`, `.fingerprint` (compiler suggests the value on first build — accept it).
+- Dependencies added with `zig fetch --save=<name> <url>` so the content hash is pinned:
+  - `sqlite`: the newest stable SQLite **amalgamation zip** from sqlite.org (check https://sqlite.org/download.html for the current one; record the version in a comment).
+  - `mbedtls`: the newest **mbedTLS 3.6.x LTS** release tarball from the Mbed-TLS GitHub releases (3.6 line only — not 4.x).
+
+### S1.2 build.zig
+
+- Comptime guard: `@import("builtin").zig_version` major==0 and minor==16, else `@compileError`.
+- Options: `-Dintegration` (bool, default false) exposed to tests via a `build_options` module (`b.addOptions()`); also embed version string + git commit (`b.option([]const u8, "version-string", ...)` defaulting to "0.1.0-dev") for `src/version.zig`.
+- C static libs, one per dependency, built with `b.addLibrary(.{ .linkage = .static, ... })` (NOT addStaticLibrary — it does not exist in 0.16):
+  - `sqlite3`: compile `sqlite3.c` from the sqlite dependency via `lib.root_module.addCSourceFile`; `addIncludePath` the dep root. Flags: `-DSQLITE_ENABLE_FTS5`, `-DSQLITE_THREADSAFE=1`, `-DSQLITE_DEFAULT_WAL_SYNCHRONOUS=1`, `-DSQLITE_OMIT_LOAD_EXTENSION`.
+  - `mbedtls`: compile every `library/*.c` file via `root_module.addCSourceFiles(.{ .root = dep.path("library"), .files = ... })` (paths must be relative — absolute paths panic) plus the `3rdparty/everest` and `3rdparty/p256m` sources; `addIncludePath` for `include`, `library`, and both 3rdparty include dirs. Default `mbedtls_config.h`, no custom config in this milestone.
+- Executable `nxdns`: root module `src/main.zig`, `link_libc = true`, links both libs. Native artifact + installed.
+- Test step `test`: `b.addTest` on a module rooted at `src/tests.zig`, importing `build_options`, linking both C libs.
+- Step `cross`: for `x86_64-linux-musl` and `aarch64-linux-musl` (via `b.resolveTargetQuery`), build the exe with `exe.linkage = .static`, install to `zig-out/cross/<triple>/nxdns`.
+- `.gitignore`: `.zig-cache/`, `zig-out/`.
+
+### S1.3 src/main.zig, src/version.zig, src/tests.zig
+
+- `src/version.zig`: `pub const string` / `pub const zig_version_string` pulled from `build_options`.
+- `src/main.zig`: parse first CLI arg. `version` → print version + zig version, exit 0. `run`, `check`, `export`, `import` → print `not implemented`, exit 2. No arg / unknown → usage on stderr, exit 64. No allocator gymnastics — `std.process.args` is enough here.
+- `src/tests.zig`: `comptime { _ = @import("main.zig"); _ = @import("version.zig"); }` plus `test` block asserting the sqlite3 and mbedTLS C headers link: call `sqlite3_libversion()` and `mbedtls_version_get_string_full()` via `@cImport`-free extern declarations (declare the two extern fns manually) and check non-empty results.
+
+### S1.4 Acceptance Criteria
+
+- [ ] `zig build test` exits 0 natively; the C-link test prints/asserts both library version strings.
+- [ ] `zig build cross` exits 0; `file zig-out/cross/x86_64-linux-musl/nxdns` and the aarch64 one both report "statically linked".
+- [ ] `./zig-out/bin/nxdns version` prints the version and Zig 0.16.0, exit 0.
+- [ ] `./zig-out/bin/nxdns` (no args) exits 64 with usage on stderr.
+- [ ] `git status` clean of build artifacts (gitignore works).
+
+---
+
+## Session S2: platform/address.zig
+
+Pure module. No `std.Io` operations — only type conversions to/from `std.Io.net.IpAddress` values. Unit tests in-file.
+
+### S2.1 Types + API
+
+```zig
+pub const NetAddress = union(enum) {
+    ip4: [4]u8,
+    ip6: [16]u8,
+
+    pub const Key = [17]u8; // tag byte (4 or 6) + address bytes, zero-padded for ip4
+
+    pub fn parse(text: []const u8) error{InvalidAddress}!NetAddress; // "1.2.3.4", "fd00::1"; no port, no brackets
+    pub fn format(self: NetAddress, w: *std.Io.Writer) std.Io.Writer.Error!void; // v4 dotted; v6 RFC 5952 lowercase, :: compression
+    pub fn key(self: NetAddress) Key;
+    pub fn fromIp(addr: std.Io.net.IpAddress) NetAddress; // drops port; IPv4-mapped IPv6 (::ffff:a.b.c.d) normalizes to .ip4
+    pub fn toIp(self: NetAddress, port: u16) std.Io.net.IpAddress;
+    pub fn eql(a: NetAddress, b: NetAddress) bool;
+};
+
+pub const Prefix = struct {
+    addr: NetAddress, // host bits zeroed on parse
+    bits: u8,         // <= 32 for ip4, <= 128 for ip6
+
+    pub fn parse(text: []const u8) error{InvalidPrefix}!Prefix; // "192.168.1.0/24", "fd00:abcd::/48"
+    pub fn contains(self: Prefix, addr: NetAddress) bool;       // family mismatch => false
+    pub fn format(self: Prefix, w: *std.Io.Writer) std.Io.Writer.Error!void;
+};
+
+/// Longest-prefix winner; ties broken by lower `priority` value. Returns null when nothing matches.
+pub fn matchLongest(comptime T: type, entries: []const T, addr: NetAddress) ?*const T;
+// T must have fields: prefix: Prefix, priority: i64
+```
+
+Implement parse/format by hand or delegate to `std.Io.net.Ip4Address/Ip6Address` parsing where it fits — but the RFC 5952 output rules (lowercase hex, longest zero-run compressed, no compression of a single group) must hold and be tested either way.
+
+### S2.2 Acceptance Criteria
+
+- [ ] Round-trip tests: parse→format is identity for canonical inputs (`"192.168.1.1"`, `"fd00::1"`, `"::"`, `"2001:db8::8:800:200c:417a"`).
+- [ ] RFC 5952 tests: `"2001:0DB8:0:0:1::1"` formats as `"2001:db8::1:0:0:0:1"`-style rules — specifically: longest run compressed, single zero group NOT compressed, lowercase.
+- [ ] `fromIp` on an IPv4-mapped IPv6 address yields `.ip4`.
+- [ ] `Prefix.contains`: `192.168.1.0/24` contains `.1.5`, not `.2.5`; `/0` contains everything of its family; family mismatch false.
+- [ ] `matchLongest`: `/24` beats `/16`; equal bits → lower priority value wins.
+- [ ] All tests pass via `zig test src/platform/address.zig` (orchestrator wires them into `zig build test` later).
+
+---
+
+## Session S3: platform/tls_client.zig
+
+Wrapper over `std.crypto.tls.Client` for upstream DoT (and reused by anything needing client TLS over a `std.Io.net.Stream`). See the API notes §std.crypto.tls.Client for exact init requirements.
+
+### S3.1 API
+
+```zig
+pub const ErrorClass = enum { handshake, certificate, io, protocol };
+
+pub fn classify(err: anyerror) ErrorClass;
+
+pub const TlsStream = struct {
+    // all fields private in practice; struct is pinned: MUST NOT move after init
+    // (tls.Client holds its reader/writer by value)
+
+    pub const Options = struct {
+        host: []const u8,                       // SNI + verification name
+        ca: enum { system, insecure_skip_verify },
+        // buffers supplied by caller; read_buffer.len >= std.crypto.tls.Client.min_buffer_len
+        read_buffer: []u8,
+        write_buffer: []u8,
+        stream_read_buffer: []u8,
+        stream_write_buffer: []u8,
+    };
+
+    /// In-place init (pinned struct). `bundle` is scanned lazily for .system via
+    /// std.crypto.Certificate.Bundle.rescan when empty; caller owns bundle + lock lifetime.
+    pub fn init(
+        self: *TlsStream,
+        io: std.Io,
+        stream: *std.Io.net.Stream,
+        bundle: *std.crypto.Certificate.Bundle,
+        bundle_lock: *std.Io.RwLock,
+        gpa: std.mem.Allocator,
+        options: Options,
+    ) InitError!void;
+
+    pub fn reader(self: *TlsStream) *std.Io.Reader;   // plaintext
+    pub fn writer(self: *TlsStream) *std.Io.Writer;   // plaintext
+    pub fn close(self: *TlsStream) void;              // close_notify via Client.end, errors swallowed to log-level
+};
+```
+
+- Entropy: 240 bytes via `io.random`; `realtime_now` via the Io clock (`io.now(.real)` — check the exact 0.16 name in `/home/mokhtar/app/zig/lib/std/Io.zig` before use).
+- `classify` maps the stdlib error sets: cert/trust errors → `.certificate`; handshake alerts/negotiation → `.handshake`; ReadFailed/WriteFailed/connection errors → `.io`; everything else → `.protocol`. Write the mapping exhaustively over `std.crypto.tls.Client.InitError` — a `switch` with explicit arms, no `else => .protocol` catch-all for that set.
+
+### S3.2 Tests
+
+- Unit: `classify` mapping table (pick 6+ representative errors across the four classes).
+- Integration (compiled only when `build_options.integration`): connect to a live host (`cloudflare-dns.com:853` — DoT port, TLS without ALPN), complete the handshake with `.ca = .system`, close cleanly. This test is NOT part of the default `zig build test` run and NOT part of blocking CI.
+
+### S3.3 Acceptance Criteria
+
+- [ ] `zig test src/platform/tls_client.zig` passes (unit tests only).
+- [ ] `classify` covers `InitError` exhaustively (compiles with explicit arms — adding a new stdlib error breaks the build here, by design).
+- [ ] Integration test compiles under `-Dintegration` (orchestrator runs it after wiring; a live-network failure is an environment finding, not a session failure).
+
+---
+
+## Session S4: platform/tls_server.zig
+
+mbedTLS-backed server-side TLS termination exposing `std.Io.Reader`/`std.Io.Writer`, so `std.http.Server` and the DoT server can sit on top of any accepted TCP stream.
+
+### S4.1 mbedTLS extern layer
+
+Declare the needed mbedTLS API as extern fns/opaque types in this file (no `@cImport` — keep translate-c out of the build). Needed surface: `mbedtls_ssl_context`, `mbedtls_ssl_config`, `mbedtls_x509_crt`, `mbedtls_pk_context`, `mbedtls_entropy_context`, `mbedtls_ctr_drbg_context` + their init/free/setup/parse functions, `mbedtls_ssl_handshake`, `mbedtls_ssl_read`, `mbedtls_ssl_write`, `mbedtls_ssl_close_notify`, `mbedtls_ssl_set_bio`, `mbedtls_strerror`. Sizes: allocate contexts with the C sizes via opaque + `extern` allocation pattern — simplest correct approach: define `extern struct` mirrors is NOT acceptable (fragile); instead heap-allocate via wrapper C-callable `malloc(sizeof)` is also not available — so: declare the context structs as `opaque` and allocate them with `gpa.alignedAlloc(u8, .of(usize), mbedtls_ssl_context_size)` where the sizes come from a tiny C shim file `src/platform/mbedtls_shim.c` exporting `size_t nx_sizeof_ssl_context(void)` etc. The shim is owned by this session and added to build by the orchestrator (S4 must NOT edit build.zig — note the shim path in the completion report).
+
+### S4.2 API
+
+```zig
+pub const ServerContext = struct {
+    // holds parsed cert chain + key + ssl_config + drbg; one per listener, reused across connections
+    pub fn init(gpa: std.mem.Allocator, cert_pem: [:0]const u8, key_pem: [:0]const u8) InitError!ServerContext;
+    pub fn deinit(self: *ServerContext, gpa: std.mem.Allocator) void;
+};
+
+pub const ServerStream = struct {
+    // pinned after accept(); owns the ssl_context for one connection
+
+    /// Performs the TLS handshake over an accepted TCP stream.
+    /// BIO callbacks bridge mbedtls_ssl_read/write to stream.reader/writer interfaces.
+    pub fn accept(
+        self: *ServerStream,
+        gpa: std.mem.Allocator,
+        ctx: *ServerContext,
+        io: std.Io,
+        stream: *std.Io.net.Stream,
+        read_buffer: []u8,
+        write_buffer: []u8,
+    ) AcceptError!void;
+
+    pub fn reader(self: *ServerStream) *std.Io.Reader;   // plaintext, implemented via Io.Reader vtable over mbedtls_ssl_read
+    pub fn writer(self: *ServerStream) *std.Io.Writer;   // plaintext, via mbedtls_ssl_write
+    pub fn close(self: *ServerStream, gpa: std.mem.Allocator) void; // close_notify + free ssl_context
+};
+```
+
+- Error mapping: negative mbedTLS return codes → Zig error set with named errors for the common cases (`CertParse`, `KeyParse`, `HandshakeFailed`, `PeerClosed`, `WantReadWrite` handled internally); include the raw code in a log via `mbedtls_strerror`.
+- `MBEDTLS_ERR_SSL_WANT_READ/WANT_WRITE` loop inside read/write — never surfaces to callers.
+
+### S4.3 Test fixture + loopback test
+
+- Generate once and commit: `tests/fixtures/self_signed_cert.pem` + `self_signed_key.pem` (openssl, EC P-256, CN=localhost, SAN DNS:localhost + IP:127.0.0.1, 100-year validity — a fixture, not a secret; note "test fixture, private key intentionally committed" in a `tests/fixtures/README.md`).
+- Loopback test (compiled only under `build_options.integration`): thread A: `IpAddress.listen` on 127.0.0.1:0 → accept → `ServerStream.accept` with the fixture → echo one message read back to the writer. Thread B (client): `std.crypto.tls.Client` with `.host = .no_verification`, `.ca = .no_verification` → write message → read echo → assert equality → clean close both sides. Drive both with `io.concurrent` on one `Threaded` instance.
+
+### S4.4 Acceptance Criteria
+
+- [ ] `ServerContext.init` with the fixture cert+key succeeds; with truncated PEM returns `error.CertParse`/`error.KeyParse` (unit tests, no network).
+- [ ] Loopback echo test passes under `-Dintegration` (orchestrator wires + runs).
+- [ ] No `@cImport` anywhere; extern decls + C shim only.
+- [ ] `close` sends close_notify (verified in the loopback test by the client reading EOF without error after `end`).
+
+---
+
+## Session S5: Gitea CI
+
+### S5.1 .gitea/workflows/ci.yml
+
+House style: `runs-on: ubuntu-24.04`, `actions/checkout@v4`, top-level `env:` for versions (model: `~/app/phoenix_inertia_react_starter/.gitea/workflows/ci.yml`). Runner: x86_64, dind, full egress, `uses:` resolves against github.com. Cache is ephemeral (runner restarts daily) — `actions/cache@v4` allowed as best-effort, never load-bearing.
+
+Jobs:
+1. `test`: checkout → `mlugg/setup-zig@v2` with `version: 0.16.0` → `zig build test` → `zig build test -Dintegration` is NOT run here (loopback integration runs are wired by the orchestrator in a later pass once tests.zig includes them; leave a commented job stub with a TODO referencing milestone-1 integration wiring).
+2. `cross`: checkout → setup-zig → `zig build cross` → assert both output binaries exist and `file` reports statically linked.
+
+### S5.2 .gitea/workflows/live-tls.yml
+
+`workflow_dispatch` only. Runs `zig build test -Dintegration` (which includes the live DoT handshake test once wired). Non-blocking by construction.
+
+### S5.3 Acceptance Criteria
+
+- [ ] `yamllint`-clean (or at minimum `python3 -c "import yaml,sys; yaml.safe_load(open('.gitea/workflows/ci.yml'))"` passes for both files).
+- [ ] Workflow YAML uses only actions available from github.com (`actions/checkout@v4`, `mlugg/setup-zig@v2`, `actions/cache@v4`).
+- [ ] No job depends on cache hits for correctness.
+
+---
+
+## Module Layout
+
+```
+AGENTS.md                     values + aim (exists)
+PLAN.md                       source of truth (exists)
+specs/milestone-1.md          this file
+specs/research/zig-0.16-api-notes.md   stdlib ground truth (exists)
+build.zig                     S1
+build.zig.zon                 S1
+.gitignore                    S1
+src/main.zig                  S1  CLI dispatch stub
+src/version.zig               S1  build_options plumbing
+src/tests.zig                 S1  test aggregator (orchestrator extends)
+src/platform/address.zig      S2  NetAddress/Prefix/matchLongest
+src/platform/tls_client.zig   S3  stdlib TLS client wrapper
+src/platform/tls_server.zig   S4  mbedTLS server wrapper
+src/platform/mbedtls_shim.c   S4  sizeof shims
+tests/fixtures/self_signed_cert.pem  S4
+tests/fixtures/self_signed_key.pem   S4
+tests/fixtures/README.md      S4
+.gitea/workflows/ci.yml       S5
+.gitea/workflows/live-tls.yml S5
+```
+
+## File Ownership
+
+| Files | Owner | Notes |
+|---|---|---|
+| build.zig, build.zig.zon, .gitignore, src/main.zig, src/version.zig, src/tests.zig | S1 | frozen after S1; orchestrator edits afterward |
+| src/platform/address.zig | S2 | |
+| src/platform/tls_client.zig | S3 | |
+| src/platform/tls_server.zig, src/platform/mbedtls_shim.c, tests/fixtures/* | S4 | shim build-wiring done by orchestrator |
+| .gitea/workflows/* | S5 | |
+
+S2–S5 MUST NOT edit build.zig, build.zig.zon, or src/tests.zig. If a session needs a build change, it reports the exact needed change in its completion report; the orchestrator applies it.
+
+## Acceptance Criteria (Milestone 1 Complete)
+
+- [ ] `zig build test` exits 0 (aggregator includes address, tls_client unit, tls_server unit tests).
+- [ ] `zig build test -Dintegration` exits 0 locally (loopback TLS echo; live DoT test may be skipped on network failure with a visible skip message).
+- [ ] `zig build cross` produces two statically linked executables that print `nxdns version` output under qemu-user or on-target (checked manually for aarch64 if qemu absent).
+- [ ] CI workflows valid YAML; `test` + `cross` jobs green on the Gitea runner.
+- [ ] All files committed with GPG-signed, lowercase-message commits.
+
+## Anti-Requirements
+
+- No DNS packet code, no sockets beyond the tests, no SQLite usage beyond the link check — that is milestone 2+.
+- No custom mbedtls_config.h, no cipher tuning, no session tickets.
+- No `@cImport`/translate-c anywhere.
+- No extra CLI behavior beyond the specified stubs.
+- No Docker/systemd packaging yet.
+- No third-party Zig packages.
+- Do not "fix" or extend files another session owns — report, don't touch.