docs: unwrap hand-wrapped prose repo-wide
Gates / frontend (push) Successful in 1m2s
Gates / test (push) Successful in 1m38s
Gates / package (push) Successful in 5m5s
Gates / test-aarch64 (push) Successful in 6m30s
Gates / container (push) Successful in 15s
CI / gates (push) Successful in 13m30s
Gates / frontend (push) Successful in 1m2s
Gates / test (push) Successful in 1m38s
Gates / package (push) Successful in 5m5s
Gates / test-aarch64 (push) Successful in 6m30s
Gates / container (push) Successful in 15s
CI / gates (push) Successful in 13m30s
This commit is contained in:
+104
-315
@@ -1,26 +1,16 @@
|
||||
# Milestone 3: Resolver Transport
|
||||
|
||||
Goal (PLAN §16 Phase 3): UDP server, TCP server, DoH + DoT upstream clients, upstream pool with
|
||||
priority failover, backoff and health. Exit: A/AAAA forwarding over UDP and TCP; health populated.
|
||||
Goal (PLAN §16 Phase 3): UDP server, TCP server, DoH + DoT upstream clients, upstream pool with priority failover, backoff and health. Exit: A/AAAA forwarding over UDP and TCP; health populated.
|
||||
|
||||
Read first: `AGENTS.md` (values), `specs/research/zig-0.16-api-notes.md` (verified stdlib facts —
|
||||
pre-0.16 knowledge is stale and MUST NOT be used), `specs/milestone-1.md` and `specs/milestone-2.md`
|
||||
(module conventions and the "As built" notes). The Zig source of truth is `/home/mokhtar/app/zig` at
|
||||
tag `0.16.0`. RFCs: 1035 §4.2.2 (TCP length prefix), 7766 (DNS over TCP), 7858 (DoT), 8484 (DoH),
|
||||
6891 (EDNS(0)), 4343 (name case-insensitivity), 9619 (QDCOUNT must be 1).
|
||||
Read first: `AGENTS.md` (values), `specs/research/zig-0.16-api-notes.md` (verified stdlib facts — pre-0.16 knowledge is stale and MUST NOT be used), `specs/milestone-1.md` and `specs/milestone-2.md` (module conventions and the "As built" notes). The Zig source of truth is `/home/mokhtar/app/zig` at tag `0.16.0`. RFCs: 1035 §4.2.2 (TCP length prefix), 7766 (DNS over TCP), 7858 (DoT), 8484 (DoH), 6891 (EDNS(0)), 4343 (name case-insensitivity), 9619 (QDCOUNT must be 1).
|
||||
|
||||
## What already exists (do not respecify, import it)
|
||||
|
||||
- `src/dns/*` — pure wire format. Used here: `packet.parse`, `packet.Packet`, `packet.firstQuestion`,
|
||||
`packet.findOptRecord`, `packet.ResponseBuilder`, `packet.setId`, `header.parse`, `header.Flags`,
|
||||
`question.Question`, `name.eqlIgnoreCase`, `edns.parseOpt`, `types.Rcode`, `types.Type`.
|
||||
- `src/dns/*` — pure wire format. Used here: `packet.parse`, `packet.Packet`, `packet.firstQuestion`, `packet.findOptRecord`, `packet.ResponseBuilder`, `packet.setId`, `header.parse`, `header.Flags`, `question.Question`, `name.eqlIgnoreCase`, `edns.parseOpt`, `types.Rcode`, `types.Type`.
|
||||
- `src/platform/address.zig` — `NetAddress`, `Prefix`, `matchLongest`.
|
||||
- `src/platform/tls_client.zig` — `TlsStream` (pinned struct; `init`, `reader`, `writer`, `close`),
|
||||
`classify`, `ErrorClass`. DoT builds on this; do not open `std.crypto.tls.Client` directly.
|
||||
- `src/platform/tls_server.zig` — server-side TLS. **Not used in this milestone** (local DoH/DoT
|
||||
endpoints are Phase 9).
|
||||
- `build.zig` — `-Dintegration` (hermetic, loopback only, PR-blocking) and `-Dlive` (leaves the
|
||||
machine, manual workflow only) reach test files through `@import("build_options")`.
|
||||
- `src/platform/tls_client.zig` — `TlsStream` (pinned struct; `init`, `reader`, `writer`, `close`), `classify`, `ErrorClass`. DoT builds on this; do not open `std.crypto.tls.Client` directly.
|
||||
- `src/platform/tls_server.zig` — server-side TLS. **Not used in this milestone** (local DoH/DoT endpoints are Phase 9).
|
||||
- `build.zig` — `-Dintegration` (hermetic, loopback only, PR-blocking) and `-Dlive` (leaves the machine, manual workflow only) reach test files through `@import("build_options")`.
|
||||
|
||||
## Sessions
|
||||
|
||||
@@ -34,47 +24,27 @@ S1 (upstream/transport.zig + upstream/health.zig)
|
||||
+--> S7 (e2e integration test)
|
||||
```
|
||||
|
||||
S1 defines every type the other sessions share, so S2–S5 can be written against the spec alone.
|
||||
S6 depends on S5's `Handler` type; S7 depends on everything.
|
||||
S1 defines every type the other sessions share, so S2–S5 can be written against the spec alone. S6 depends on S5's `Handler` type; S7 depends on everything.
|
||||
|
||||
The orchestrator — not any session — wires `src/tests.zig` imports and any `build.zig` change. A
|
||||
session that needs a build change reports the exact change in its completion report.
|
||||
The orchestrator — not any session — wires `src/tests.zig` imports and any `build.zig` change. A session that needs a build change reports the exact change in its completion report.
|
||||
|
||||
## Design invariants (all sessions)
|
||||
|
||||
- The pure core stays pure. `src/dns/` gains nothing. All `std.Io` use lives in `src/server/` and
|
||||
`src/upstream/`. `upstream/health.zig` and the validation half of `upstream/transport.zig` are
|
||||
themselves pure (timestamps and randomness arrive as parameters), so they are unit-testable
|
||||
without a backend.
|
||||
- **Failures are classified into three disjoint groups**: peer fault, local resource, cancellation.
|
||||
Health and backoff count peer faults ONLY. A local `OutOfMemory` must never mark an upstream sick,
|
||||
and `error.Canceled` must never be recorded at all.
|
||||
- **QDCOUNT must be exactly 1** (RFC 9619) — for inbound client queries (else FORMERR) and for
|
||||
upstream responses (else the response is a peer fault).
|
||||
- Every response from an upstream is validated against the request it answers before it reaches a
|
||||
client: ID equal, QDCOUNT 1 on both sides, QR set, question name equal **case-insensitively**
|
||||
(RFC 4343), qtype and qclass equal.
|
||||
- The pure core stays pure. `src/dns/` gains nothing. All `std.Io` use lives in `src/server/` and `src/upstream/`. `upstream/health.zig` and the validation half of `upstream/transport.zig` are themselves pure (timestamps and randomness arrive as parameters), so they are unit-testable without a backend.
|
||||
- **Failures are classified into three disjoint groups**: peer fault, local resource, cancellation. Health and backoff count peer faults ONLY. A local `OutOfMemory` must never mark an upstream sick, and `error.Canceled` must never be recorded at all.
|
||||
- **QDCOUNT must be exactly 1** (RFC 9619) — for inbound client queries (else FORMERR) and for upstream responses (else the response is a peer fault).
|
||||
- Every response from an upstream is validated against the request it answers before it reaches a client: ID equal, QDCOUNT 1 on both sides, QR set, question name equal **case-insensitively** (RFC 4343), qtype and qclass equal.
|
||||
- TCP and DoT messages carry a 2-byte big-endian length prefix (RFC 1035 §4.2.2). DoH does not.
|
||||
- No stream read or write in 0.16.0 accepts a timeout (verified: `Io.Operation` has no net-stream
|
||||
variants). Bound them by running the work under `io.concurrent` / `std.Io.Select` and cancelling
|
||||
the loser, exactly as `src/platform/tls_client_integration_test.zig` does. UDP `receive` is the one
|
||||
exception — `Socket.receiveTimeout` is implemented on the POSIX Threaded backend (verified:
|
||||
`Io/Threaded.zig:2779` handles `net_receive` in the pollable batch path).
|
||||
- Never set `net.IpAddress.ConnectOptions.timeout` — the Threaded backend panics
|
||||
(`Io/Threaded.zig:12077`).
|
||||
- Every dropped datagram, refused connection and failed send increments a named counter. No silent
|
||||
drops (AGENTS.md).
|
||||
- Unit tests live in-file. Tests that touch loopback sockets live in a separate `*_integration_test.zig`
|
||||
file guarded by `if (!build_options.integration) return error.SkipZigTest;`. Tests that leave the
|
||||
machine are guarded by `build_options.live`. This mirrors milestone 1 — the guard is a runtime
|
||||
return so the body is always compiled and cannot rot.
|
||||
- No stream read or write in 0.16.0 accepts a timeout (verified: `Io.Operation` has no net-stream variants). Bound them by running the work under `io.concurrent` / `std.Io.Select` and cancelling the loser, exactly as `src/platform/tls_client_integration_test.zig` does. UDP `receive` is the one exception — `Socket.receiveTimeout` is implemented on the POSIX Threaded backend (verified: `Io/Threaded.zig:2779` handles `net_receive` in the pollable batch path).
|
||||
- Never set `net.IpAddress.ConnectOptions.timeout` — the Threaded backend panics (`Io/Threaded.zig:12077`).
|
||||
- Every dropped datagram, refused connection and failed send increments a named counter. No silent drops (AGENTS.md).
|
||||
- Unit tests live in-file. Tests that touch loopback sockets live in a separate `*_integration_test.zig` file guarded by `if (!build_options.integration) return error.SkipZigTest;`. Tests that leave the machine are guarded by `build_options.live`. This mirrors milestone 1 — the guard is a runtime return so the body is always compiled and cannot rot.
|
||||
|
||||
---
|
||||
|
||||
## Session S1: `src/upstream/transport.zig`, `src/upstream/health.zig`
|
||||
|
||||
Foundation. No sockets are opened in this session; `transport.zig` names the `Io`-taking interface
|
||||
that S2–S4 implement, and everything else in both files is pure.
|
||||
Foundation. No sockets are opened in this session; `transport.zig` names the `Io`-taking interface that S2–S4 implement, and everything else in both files is pure.
|
||||
|
||||
### S1.1 transport.zig — endpoint parsing
|
||||
|
||||
@@ -101,15 +71,9 @@ pub const Endpoint = struct {
|
||||
};
|
||||
```
|
||||
|
||||
Hand-written parse (a `std.Uri` round trip would hand back percent-encoded components that need
|
||||
re-decoding for one household-scale config value). Rules: reject any other scheme, reject an empty
|
||||
host, reject a port that is empty, non-numeric or > 65535, reject a `tls://` URL that carries a path
|
||||
other than `/` or nothing.
|
||||
Hand-written parse (a `std.Uri` round trip would hand back percent-encoded components that need re-decoding for one household-scale config value). Rules: reject any other scheme, reject an empty host, reject a port that is empty, non-numeric or > 65535, reject a `tls://` URL that carries a path other than `/` or nothing.
|
||||
|
||||
Tests: `https://cloudflare-dns.com/dns-query`, `https://dns.example/x` (path preserved),
|
||||
`https://dns.example` (path defaults), `tls://dns.google:853`, `tls://dns.google` (port defaults to
|
||||
853), `https://[2606:4700:4700::1111]:8443/dns-query` (host without brackets, port 8443),
|
||||
`udp://1.1.1.1:53` → `UnsupportedScheme`, `https://` → `MissingHost`, `https://h:99999/` → `BadPort`.
|
||||
Tests: `https://cloudflare-dns.com/dns-query`, `https://dns.example/x` (path preserved), `https://dns.example` (path defaults), `tls://dns.google:853`, `tls://dns.google` (port defaults to 853), `https://[2606:4700:4700::1111]:8443/dns-query` (host without brackets, port 8443), `udp://1.1.1.1:53` → `UnsupportedScheme`, `https://` → `MissingHost`, `https://h:99999/` → `BadPort`.
|
||||
|
||||
### S1.2 transport.zig — error groups
|
||||
|
||||
@@ -149,13 +113,9 @@ pub const Group = enum { peer_fault, local_resource, cancellation };
|
||||
pub fn group(err: ExchangeError) Group;
|
||||
```
|
||||
|
||||
The three sets are disjoint by construction; a test asserts it by walking
|
||||
`@typeInfo(PeerFault).error_set.?` and friends and checking no name appears twice.
|
||||
The three sets are disjoint by construction; a test asserts it by walking `@typeInfo(PeerFault).error_set.?` and friends and checking no name appears twice.
|
||||
|
||||
`pub fn mapLocal(err: anyerror) ?ExchangeError` — helper for S2/S3: returns the `LocalResource` or
|
||||
`Cancellation` member matching a foreign stdlib error by name, `null` when the caller should treat
|
||||
the error as a peer fault. Implemented with an explicit switch over the named errors listed above
|
||||
plus `error.Canceled`; this is the ONLY place a foreign error set is folded in.
|
||||
`pub fn mapLocal(err: anyerror) ?ExchangeError` — helper for S2/S3: returns the `LocalResource` or `Cancellation` member matching a foreign stdlib error by name, `null` when the caller should treat the error as a peer fault. Implemented with an explicit switch over the named errors listed above plus `error.Canceled`; this is the ONLY place a foreign error set is folded in.
|
||||
|
||||
### S1.3 transport.zig — the client interface
|
||||
|
||||
@@ -195,17 +155,9 @@ pub const ValidateError = error{ BadResponse, ResponseMismatch };
|
||||
pub fn validateResponse(query: []const u8, response: []const u8) ValidateError!void;
|
||||
```
|
||||
|
||||
Order of checks: response parses (`packet.parse` failure → `BadResponse`); response header
|
||||
`flags.qr` is true (else `BadResponse`); response `qdcount == 1` (else `BadResponse`); query parses
|
||||
and has `qdcount == 1` (else `BadResponse` — a caller bug, but never a crash); IDs equal (else
|
||||
`ResponseMismatch`); question `qtype` and `qclass` equal and `name.eqlIgnoreCase` (else
|
||||
`ResponseMismatch`).
|
||||
Order of checks: response parses (`packet.parse` failure → `BadResponse`); response header `flags.qr` is true (else `BadResponse`); response `qdcount == 1` (else `BadResponse`); query parses and has `qdcount == 1` (else `BadResponse` — a caller bug, but never a crash); IDs equal (else `ResponseMismatch`); question `qtype` and `qclass` equal and `name.eqlIgnoreCase` (else `ResponseMismatch`).
|
||||
|
||||
Tests (hand-built byte fixtures, reuse the shape of `src/dns/packet.zig`'s fixtures): matching pair
|
||||
passes; mixed-case question name passes; wrong ID → `ResponseMismatch`; different qtype →
|
||||
`ResponseMismatch`; different name → `ResponseMismatch`; QR clear → `BadResponse`; response with
|
||||
QDCOUNT 0 → `BadResponse`; response with QDCOUNT 2 → `BadResponse`; truncated garbage →
|
||||
`BadResponse`.
|
||||
Tests (hand-built byte fixtures, reuse the shape of `src/dns/packet.zig`'s fixtures): matching pair passes; mixed-case question name passes; wrong ID → `ResponseMismatch`; different qtype → `ResponseMismatch`; different name → `ResponseMismatch`; QR clear → `BadResponse`; response with QDCOUNT 0 → `BadResponse`; response with QDCOUNT 2 → `BadResponse`; truncated garbage → `BadResponse`.
|
||||
|
||||
### S1.5 health.zig — pure health and backoff state
|
||||
|
||||
@@ -253,29 +205,12 @@ pub const State = struct {
|
||||
|
||||
Rules:
|
||||
|
||||
1. `recordSuccess` clears `consecutive_failures` and `backoff_until`, pushes a 1 into the window,
|
||||
and sets `last_success_at` to `@max(existing, at)` by nanoseconds.
|
||||
2. `recordFailure` increments `consecutive_failures` and `total_failures`, pushes a 0, copies
|
||||
`err_name` (truncated to the buffer), sets `last_error_at` to `@max(existing, at)`. When
|
||||
`consecutive_failures >= cfg.failure_threshold`, it computes
|
||||
`delay_ms = min(max_backoff_ms, base_backoff_ms << shift)` where
|
||||
`shift = min(consecutive_failures - failure_threshold, 20)`, applies jitter
|
||||
`jittered = delay_ms/2 + rand % (delay_ms/2 + 1)`, and sets
|
||||
`backoff_until = @max(existing, at + jittered)`.
|
||||
3. **Out-of-order completions are the normal case**, not an edge case: two concurrent exchanges
|
||||
against the same endpoint complete in either order, so `at` can move backwards between calls.
|
||||
Every timestamp field therefore updates through `@max` on `.nanoseconds`, and `backoff_until` is
|
||||
only ever extended, never shortened. `available` must never compute a negative duration.
|
||||
1. `recordSuccess` clears `consecutive_failures` and `backoff_until`, pushes a 1 into the window, and sets `last_success_at` to `@max(existing, at)` by nanoseconds.
|
||||
2. `recordFailure` increments `consecutive_failures` and `total_failures`, pushes a 0, copies `err_name` (truncated to the buffer), sets `last_error_at` to `@max(existing, at)`. When `consecutive_failures >= cfg.failure_threshold`, it computes `delay_ms = min(max_backoff_ms, base_backoff_ms << shift)` where `shift = min(consecutive_failures - failure_threshold, 20)`, applies jitter `jittered = delay_ms/2 + rand % (delay_ms/2 + 1)`, and sets `backoff_until = @max(existing, at + jittered)`.
|
||||
3. **Out-of-order completions are the normal case**, not an edge case: two concurrent exchanges against the same endpoint complete in either order, so `at` can move backwards between calls. Every timestamp field therefore updates through `@max` on `.nanoseconds`, and `backoff_until` is only ever extended, never shortened. `available` must never compute a negative duration.
|
||||
4. `State` carries no lock. The pool owns the mutex.
|
||||
|
||||
Tests: threshold not reached → `available(now)` stays true; threshold reached → unavailable until
|
||||
`backoff_until`, available one nanosecond after; consecutive failures grow the delay and it saturates
|
||||
at `max_backoff_ms` (walk 40 failures with `rand = 0`); success resets consecutive count, window and
|
||||
backoff; jitter with `rand = 0` and `rand = maxInt(u32)` both land inside `[delay/2, delay]`; an
|
||||
out-of-order pair — `recordSuccess(t=100)` then `recordSuccess(t=50)` — leaves `last_success_at` at
|
||||
100; `recordFailure(t=50)` after `recordFailure(t=100)` does not shorten `backoff_until`;
|
||||
`successRate` over a half-success window is 0.5; `lastError` returns the last recorded name and is
|
||||
truncated, not overflowed, by a name longer than the buffer.
|
||||
Tests: threshold not reached → `available(now)` stays true; threshold reached → unavailable until `backoff_until`, available one nanosecond after; consecutive failures grow the delay and it saturates at `max_backoff_ms` (walk 40 failures with `rand = 0`); success resets consecutive count, window and backoff; jitter with `rand = 0` and `rand = maxInt(u32)` both land inside `[delay/2, delay]`; an out-of-order pair — `recordSuccess(t=100)` then `recordSuccess(t=50)` — leaves `last_success_at` at 100; `recordFailure(t=50)` after `recordFailure(t=100)` does not shorten `backoff_until`; `successRate` over a half-success window is 0.5; `lastError` returns the last recorded name and is truncated, not overflowed, by a name longer than the buffer.
|
||||
|
||||
### S1.6 Acceptance criteria
|
||||
|
||||
@@ -314,16 +249,12 @@ pub const DohClient = struct {
|
||||
};
|
||||
```
|
||||
|
||||
`self.uri` is derived from `endpoint` with `std.Uri.parse(endpoint.url)`; a parse failure is
|
||||
`error.BadUrl`. `endpoint` remains the source of truth for host/port/scheme in logs.
|
||||
`self.uri` is derived from `endpoint` with `std.Uri.parse(endpoint.url)`; a parse failure is `error.BadUrl`. `endpoint` remains the source of truth for host/port/scheme in logs.
|
||||
|
||||
### S2.2 Exchange
|
||||
|
||||
1. `if (query.len > self.request_buf.len) return error.BufferTooSmall;` then copy `query` into
|
||||
`request_buf` — `Request.sendBodyComplete` takes `[]u8`, not `[]const u8` (verified,
|
||||
`std/http/Client.zig:935`).
|
||||
2. ```zig
|
||||
var req = try self.http.request(.POST, self.uri, .{
|
||||
1. `if (query.len > self.request_buf.len) return error.BufferTooSmall;` then copy `query` into `request_buf` — `Request.sendBodyComplete` takes `[]u8`, not `[]const u8` (verified, `std/http/Client.zig:935`).
|
||||
2. ```zig var req = try self.http.request(.POST, self.uri, .{
|
||||
.keep_alive = true,
|
||||
.redirect_behavior = .not_allowed,
|
||||
.headers = .{
|
||||
@@ -333,8 +264,7 @@ pub const DohClient = struct {
|
||||
.accept_encoding = .{ .override = "identity" },
|
||||
},
|
||||
.extra_headers = &.{.{ .name = "accept", .value = "application/dns-message" }},
|
||||
});
|
||||
defer req.deinit();
|
||||
}); defer req.deinit();
|
||||
```
|
||||
`Request.Headers` has no `accept` field (verified, `std/http/Client.zig:845`), hence
|
||||
`extra_headers`.
|
||||
@@ -425,48 +355,24 @@ pub const DotClient = struct {
|
||||
|
||||
### S3.2 Exchange
|
||||
|
||||
One TCP connection and one TLS handshake per exchange, closed before returning. Connection reuse is
|
||||
not built: at household query rates the pool's per-attempt budget and the failover path stay simple
|
||||
and every failure is attributable to one exchange, which is worth more here than the saved round
|
||||
trips. Say that in a comment — it is a decision, not a stub.
|
||||
One TCP connection and one TLS handshake per exchange, closed before returning. Connection reuse is not built: at household query rates the pool's per-attempt budget and the failover path stay simple and every failure is attributable to one exchange, which is worth more here than the saved round trips. Say that in a comment — it is a decision, not a stub.
|
||||
|
||||
1. Resolve the endpoint to an `std.Io.net.IpAddress` with `net.IpAddress.parse(endpoint.host,
|
||||
endpoint.port)`. **A non-literal host is `error.ConnectFailed`** with a log line naming the host:
|
||||
name resolution for upstreams is not in this milestone's scope, and a silent fallback would hide
|
||||
it. (`tls://dns.google:853` from PLAN §12.1 therefore needs an IP literal in config; note this in
|
||||
the completion report so the orchestrator can carry it into the Phase 4 config validator.)
|
||||
2. `var stream = try address.connect(io, .{ .mode = .stream });` — no `timeout` field, ever.
|
||||
`defer` a close that runs under cancel protection (below).
|
||||
3. `var tls_stream: tls_client.TlsStream = undefined;` then `tls_stream.init(io, &stream, bundle,
|
||||
bundle_lock, gpa, .{ .host = endpoint.host, .ca = .system, … })`. `TlsStream` is pinned — do not
|
||||
copy or move it after init.
|
||||
1. Resolve the endpoint to an `std.Io.net.IpAddress` with `net.IpAddress.parse(endpoint.host, endpoint.port)`. **A non-literal host is `error.ConnectFailed`** with a log line naming the host: name resolution for upstreams is not in this milestone's scope, and a silent fallback would hide it. (`tls://dns.google:853` from PLAN §12.1 therefore needs an IP literal in config; note this in the completion report so the orchestrator can carry it into the Phase 4 config validator.)
|
||||
2. `var stream = try address.connect(io, .{ .mode = .stream });` — no `timeout` field, ever. `defer` a close that runs under cancel protection (below).
|
||||
3. `var tls_stream: tls_client.TlsStream = undefined;` then `tls_stream.init(io, &stream, bundle, bundle_lock, gpa, .{ .host = endpoint.host, .ca = .system, … })`. `TlsStream` is pinned — do not copy or move it after init.
|
||||
4. Write `[2]u8` big-endian length then `query` to `tls_stream.writer()`, then `flush()`.
|
||||
5. Read exactly 2 bytes, then that many bytes, from `tls_stream.reader()`. `len == 0` →
|
||||
`error.BadResponse`. `len > response_buf.len` → `error.ResponseTooLarge`. A short read or EOF →
|
||||
`error.ReceiveFailed`.
|
||||
5. Read exactly 2 bytes, then that many bytes, from `tls_stream.reader()`. `len == 0` → `error.BadResponse`. `len > response_buf.len` → `error.ResponseTooLarge`. A short read or EOF → `error.ReceiveFailed`.
|
||||
6. `try transport.validateResponse(query, response_buf[0..len]);`
|
||||
7. `tls_stream.close();` then `stream.close(io);`.
|
||||
|
||||
Cleanup under cancellation: the pool cancels this task on timeout, and the next cancelable `Io` call
|
||||
in the `defer` chain would return `error.Canceled` and skip the socket close. Wrap the close path in
|
||||
`const prev = io.swapCancelProtection(.blocked); defer _ = io.swapCancelProtection(prev);`
|
||||
(verified: `std/Io.zig:1342`) so the socket is always released.
|
||||
Cleanup under cancellation: the pool cancels this task on timeout, and the next cancelable `Io` call in the `defer` chain would return `error.Canceled` and skip the socket close. Wrap the close path in `const prev = io.swapCancelProtection(.blocked); defer _ = io.swapCancelProtection(prev);` (verified: `std/Io.zig:1342`) so the socket is always released.
|
||||
|
||||
Error mapping: `connect` → `error.ConnectFailed`; `TlsStream.init` → `error.TlsFailed` (log
|
||||
`tls_client.classify(err)` alongside — that is what milestone 1 built it for); write/flush →
|
||||
`error.SendFailed`; read → `error.ReceiveFailed`; `transport.mapLocal` first at every site.
|
||||
Error mapping: `connect` → `error.ConnectFailed`; `TlsStream.init` → `error.TlsFailed` (log `tls_client.classify(err)` alongside — that is what milestone 1 built it for); write/flush → `error.SendFailed`; read → `error.ReceiveFailed`; `transport.mapLocal` first at every site.
|
||||
|
||||
### S3.3 Tests
|
||||
|
||||
- In-file unit: `framePrefix(len: u16) [2]u8` and `parsePrefix(bytes: [2]u8) u16` round-trip,
|
||||
including 0 and 65535, big-endian byte order asserted explicitly; `init` asserts on a `.doh`
|
||||
endpoint; a non-literal host maps to `error.ConnectFailed` without touching the network (call the
|
||||
address-resolution helper directly).
|
||||
- `src/upstream/dot_client_live_test.zig`, guarded by `build_options.live`: exchange an A query for
|
||||
`example.com` against `tls://1.1.1.1:853` with `endpoint.host` overridden to `"1.1.1.1"` — this
|
||||
machine's IPv6 egress is dead, so the documented anycast literal is used, and Cloudflare's
|
||||
certificate carries `1.1.1.1` as an IP SAN. If the stdlib verifier rejects an IP-literal SAN, do
|
||||
NOT weaken verification: report the finding. Raced against a 10 s budget with `std.Io.Select`.
|
||||
- In-file unit: `framePrefix(len: u16) [2]u8` and `parsePrefix(bytes: [2]u8) u16` round-trip, including 0 and 65535, big-endian byte order asserted explicitly; `init` asserts on a `.doh` endpoint; a non-literal host maps to `error.ConnectFailed` without touching the network (call the address-resolution helper directly).
|
||||
- `src/upstream/dot_client_live_test.zig`, guarded by `build_options.live`: exchange an A query for `example.com` against `tls://1.1.1.1:853` with `endpoint.host` overridden to `"1.1.1.1"` — this machine's IPv6 egress is dead, so the documented anycast literal is used, and Cloudflare's certificate carries `1.1.1.1` as an IP SAN. If the stdlib verifier rejects an IP-literal SAN, do NOT weaken verification: report the finding. Raced against a 10 s budget with `std.Io.Select`.
|
||||
|
||||
### S3.4 Acceptance criteria
|
||||
|
||||
@@ -480,8 +386,7 @@ Error mapping: `connect` → `error.ConnectFailed`; `TlsStream.init` → `error.
|
||||
|
||||
## Session S4: `src/upstream/pool.zig`
|
||||
|
||||
Priority-ordered sequential failover with per-attempt deadline, health tracking and backoff
|
||||
(PLAN §9). The pool is itself a `transport.Client`, so the handler sees one interface.
|
||||
Priority-ordered sequential failover with per-attempt deadline, health tracking and backoff (PLAN §9). The pool is itself a `transport.Client`, so the handler sees one interface.
|
||||
|
||||
### S4.1 API
|
||||
|
||||
@@ -537,61 +442,36 @@ pub const Pool = struct {
|
||||
};
|
||||
```
|
||||
|
||||
`attempt_timeout` uses the `.awake` clock (`.{ .raw = .fromMilliseconds(2000), .clock = .awake }`) so
|
||||
a suspended Pi does not burn the budget.
|
||||
`attempt_timeout` uses the `.awake` clock (`.{ .raw = .fromMilliseconds(2000), .clock = .awake }`) so a suspended Pi does not burn the budget.
|
||||
|
||||
### S4.2 Failover algorithm
|
||||
|
||||
1. Take `now = std.Io.Clock.awake.now(io)`.
|
||||
2. **Pass one**: walk `entries` in order; skip `!enabled`; skip entries whose
|
||||
`health.available(now)` is false.
|
||||
3. **Pass two**, only if pass one attempted nothing: walk again, skipping only `!enabled`. Every
|
||||
endpoint being in backoff must not turn into `SERVFAIL` for every client — a probe is better than
|
||||
a guaranteed failure, and it is how backoff recovers. Test this explicitly.
|
||||
2. **Pass one**: walk `entries` in order; skip `!enabled`; skip entries whose `health.available(now)` is false.
|
||||
3. **Pass two**, only if pass one attempted nothing: walk again, skipping only `!enabled`. Every endpoint being in backoff must not turn into `SERVFAIL` for every client — a probe is better than a guaranteed failure, and it is how backoff recovers. Test this explicitly.
|
||||
4. For each candidate, run `attempt`:
|
||||
- Race `entry.client.exchange(io, query, response_buf)` against
|
||||
`attempt_timeout.sleep(io)` using `std.Io.Select` with a two-arm union, then
|
||||
`race.cancelDiscard()` on the way out (the milestone-1 pattern). The sleep arm winning yields
|
||||
`error.Timeout`.
|
||||
- On success: lock the mutex, `entry.health.recordSuccess(completed_at)`, unlock, return the
|
||||
slice.
|
||||
- Race `entry.client.exchange(io, query, response_buf)` against `attempt_timeout.sleep(io)` using `std.Io.Select` with a two-arm union, then `race.cancelDiscard()` on the way out (the milestone-1 pattern). The sleep arm winning yields `error.Timeout`.
|
||||
- On success: lock the mutex, `entry.health.recordSuccess(completed_at)`, unlock, return the slice.
|
||||
- On error, switch on `transport.group(err)`:
|
||||
- `.peer_fault` → lock, `recordFailure(completed_at, @errorName(err), self.cfg, self.rng.random().int(u32))`,
|
||||
unlock, remember it as `last_fault`, continue to the next candidate.
|
||||
- `.local_resource` → return the error immediately. Do not record. Do not try another upstream:
|
||||
the next one will hit the same wall.
|
||||
- `.peer_fault` → lock, `recordFailure(completed_at, @errorName(err), self.cfg, self.rng.random().int(u32))`, unlock, remember it as `last_fault`, continue to the next candidate.
|
||||
- `.local_resource` → return the error immediately. Do not record. Do not try another upstream: the next one will hit the same wall.
|
||||
- `.cancellation` → return `error.Canceled` immediately. Do not record.
|
||||
- `completed_at` is `std.Io.Clock.awake.now(io)` taken after the attempt returns.
|
||||
5. All candidates exhausted → return `last_fault` (guaranteed non-null: `entries.len > 0` and pass
|
||||
two attempts every enabled entry). If every entry is disabled, return `error.ConnectFailed`.
|
||||
5. All candidates exhausted → return `last_fault` (guaranteed non-null: `entries.len > 0` and pass two attempts every enabled entry). If every entry is disabled, return `error.ConnectFailed`.
|
||||
|
||||
`response_buf` is handed to each attempt in turn; a failed attempt may have written into it, so the
|
||||
returned slice is only meaningful on success. Note this in a comment.
|
||||
`response_buf` is handed to each attempt in turn; a failed attempt may have written into it, so the returned slice is only meaningful on success. Note this in a comment.
|
||||
|
||||
Concurrency: several handler tasks share one `Pool`. The mutex guards health mutation and
|
||||
`snapshot` only — never an in-flight exchange, so a slow upstream cannot block bookkeeping. Entry
|
||||
`client` values must therefore be safe for concurrent `exchange` calls; `DohClient` and `DotClient`
|
||||
own per-instance buffers, so **one `Entry` per concurrent task, or one shared pool with per-entry
|
||||
serialization, is a wiring decision for the orchestrator**. Record in the completion report which
|
||||
one this session assumed (the specified `Entry` layout assumes each entry's client is used by one
|
||||
task at a time; the e2e test in S7 exercises exactly one in-flight query at a time).
|
||||
Concurrency: several handler tasks share one `Pool`. The mutex guards health mutation and `snapshot` only — never an in-flight exchange, so a slow upstream cannot block bookkeeping. Entry `client` values must therefore be safe for concurrent `exchange` calls; `DohClient` and `DotClient` own per-instance buffers, so **one `Entry` per concurrent task, or one shared pool with per-entry serialization, is a wiring decision for the orchestrator**. Record in the completion report which one this session assumed (the specified `Entry` layout assumes each entry's client is used by one task at a time; the e2e test in S7 exercises exactly one in-flight query at a time).
|
||||
|
||||
### S4.3 Tests (in-file, no sockets — fake clients implement `transport.Client`)
|
||||
|
||||
- Priority order: entry with priority 10 is tried before priority 100.
|
||||
- Failover: first fake returns `error.Timeout`, second returns a valid response → the pool returns
|
||||
the second one's bytes; first entry's `consecutive_failures == 1`, second's `total_successes == 1`.
|
||||
- Backoff skip: drive a fake past `failure_threshold`, then assert the next `exchange` does not call
|
||||
that fake (a call counter in the fake) while another entry is available.
|
||||
- Failover: first fake returns `error.Timeout`, second returns a valid response → the pool returns the second one's bytes; first entry's `consecutive_failures == 1`, second's `total_successes == 1`.
|
||||
- Backoff skip: drive a fake past `failure_threshold`, then assert the next `exchange` does not call that fake (a call counter in the fake) while another entry is available.
|
||||
- All-backed-off probe: every entry in backoff → pass two still calls them.
|
||||
- `.local_resource` short-circuit: a fake returning `error.OutOfMemory` makes `exchange` return
|
||||
`error.OutOfMemory`, the second fake is never called, and the first entry's `total_failures`
|
||||
stays 0.
|
||||
- `.cancellation` short-circuit: a fake returning `error.Canceled` returns `error.Canceled` and
|
||||
records nothing.
|
||||
- Attempt timeout: a fake that sleeps longer than `attempt_timeout` yields a recorded
|
||||
`error.Timeout` peer fault and moves on. (Needs a live `Io` — use `std.Io.Threaded` inside the
|
||||
test; no sockets are involved, so this stays in the default `zig build test`.)
|
||||
- `.local_resource` short-circuit: a fake returning `error.OutOfMemory` makes `exchange` return `error.OutOfMemory`, the second fake is never called, and the first entry's `total_failures` stays 0.
|
||||
- `.cancellation` short-circuit: a fake returning `error.Canceled` returns `error.Canceled` and records nothing.
|
||||
- Attempt timeout: a fake that sleeps longer than `attempt_timeout` yields a recorded `error.Timeout` peer fault and moves on. (Needs a live `Io` — use `std.Io.Threaded` inside the test; no sockets are involved, so this stays in the default `zig build test`.)
|
||||
- All entries disabled → `error.ConnectFailed`.
|
||||
- `snapshot` reports the counters set by the preceding cases.
|
||||
|
||||
@@ -606,9 +486,7 @@ task at a time; the e2e test in S7 exercises exactly one in-flight query at a ti
|
||||
|
||||
## Session S5: `src/server/handler.zig`
|
||||
|
||||
The serving handler: bytes in from a listener, validated bytes out. It owns no socket and no clock
|
||||
beyond what it passes to the upstream. Phase 7 extends this file with filtering, cache, local
|
||||
records and logging — nothing of that appears here.
|
||||
The serving handler: bytes in from a listener, validated bytes out. It owns no socket and no clock beyond what it passes to the upstream. Phase 7 extends this file with filtering, cache, local records and logging — nothing of that appears here.
|
||||
|
||||
### S5.1 API
|
||||
|
||||
@@ -644,52 +522,36 @@ pub const Handler = struct {
|
||||
};
|
||||
```
|
||||
|
||||
`handle` never returns an error: every failure is either a DNS response or a counted drop. That is
|
||||
the "every failure mode visible" rule applied to the hot path.
|
||||
`handle` never returns an error: every failure is either a DNS response or a counted drop. That is the "every failure mode visible" rule applied to the hot path.
|
||||
|
||||
### S5.2 Behavior
|
||||
|
||||
1. `header.parse(query)` fails (< 12 bytes) → `dropped_malformed`, `.drop` (PLAN §6.1: severely
|
||||
truncated is a silent drop).
|
||||
1. `header.parse(query)` fails (< 12 bytes) → `dropped_malformed`, `.drop` (PLAN §6.1: severely truncated is a silent drop).
|
||||
2. `hdr.flags.qr == true` → a response arriving on a listener port; `dropped_malformed`, `.drop`.
|
||||
3. `packet.parse(query)`:
|
||||
- `error.Truncated` → `dropped_malformed`, `.drop`.
|
||||
- any other `WalkError` → FORMERR reply built from `hdr` with **no** echoed question (the
|
||||
question is what failed to parse); `formerr`.
|
||||
- any other `WalkError` → FORMERR reply built from `hdr` with **no** echoed question (the question is what failed to parse); `formerr`.
|
||||
4. `hdr.flags.opcode != .query` → NOTIMP reply echoing nothing; `notimp`.
|
||||
5. `hdr.qdcount != 1` → FORMERR (RFC 9619 makes any other value invalid, in both directions);
|
||||
`formerr`.
|
||||
5. `hdr.qdcount != 1` → FORMERR (RFC 9619 makes any other value invalid, in both directions); `formerr`.
|
||||
6. Forward: `self.upstream.exchange(io, query, response_buf)`.
|
||||
- success → `queries` incremented; then the UDP size check in §5.3; `.reply`.
|
||||
- `transport.group(err) == .cancellation` → `.drop` (the process is shutting down; nothing to
|
||||
say).
|
||||
- `transport.group(err) == .cancellation` → `.drop` (the process is shutting down; nothing to say).
|
||||
- `.peer_fault` or `.local_resource` → SERVFAIL reply echoing the question; `servfail`.
|
||||
7. Every synthesized reply preserves the request ID, opcode and RD bit, sets QR and RA, and echoes
|
||||
the question when one parsed — all of that is already `packet.ResponseBuilder.init`'s contract.
|
||||
When the query carried an OPT record (`packet.findOptRecord` + `edns.parseOpt`), the reply ends
|
||||
with `addOptEcho(opt, opt.do_bit)` so the DO bit passes through (PLAN §6.1).
|
||||
7. Every synthesized reply preserves the request ID, opcode and RD bit, sets QR and RA, and echoes the question when one parsed — all of that is already `packet.ResponseBuilder.init`'s contract. When the query carried an OPT record (`packet.findOptRecord` + `edns.parseOpt`), the reply ends with `addOptEcho(opt, opt.do_bit)` so the DO bit passes through (PLAN §6.1).
|
||||
|
||||
### S5.3 UDP size limit and truncation
|
||||
|
||||
The upstream answer can exceed what the client will accept over UDP.
|
||||
|
||||
- `pub fn udpLimit(query_packet: packet.Packet) u16` — the query's OPT `udp_payload_size` clamped to
|
||||
`[512, 4096]`, or 512 when there is no OPT record (RFC 1035 §4.2.1 / RFC 6891 §6.2.3).
|
||||
- `which == .udp` and the upstream reply is longer than `udpLimit` → build a reply with `TC = 1`,
|
||||
RCODE NOERROR, the question echoed, no answer records, OPT echoed when present; increment
|
||||
`truncated`. The client then retries over TCP, which is exactly what RFC 1035 §4.2.1 prescribes.
|
||||
- `pub fn udpLimit(query_packet: packet.Packet) u16` — the query's OPT `udp_payload_size` clamped to `[512, 4096]`, or 512 when there is no OPT record (RFC 1035 §4.2.1 / RFC 6891 §6.2.3).
|
||||
- `which == .udp` and the upstream reply is longer than `udpLimit` → build a reply with `TC = 1`, RCODE NOERROR, the question echoed, no answer records, OPT echoed when present; increment `truncated`. The client then retries over TCP, which is exactly what RFC 1035 §4.2.1 prescribes.
|
||||
- `which == .tcp` → no size limit beyond `response_buf`.
|
||||
|
||||
The upstream reply lands in `response_buf`; building a truncated reply over it needs a second
|
||||
buffer. Give `handle` no extra parameter: build the truncated reply in a stack `[512]u8` inside
|
||||
`handle` and copy it into `response_buf` before returning. 512 is enough — the message is a header,
|
||||
one question and at most one OPT record, and a question longer than ~270 bytes cannot exist
|
||||
(name ≤ 255).
|
||||
The upstream reply lands in `response_buf`; building a truncated reply over it needs a second buffer. Give `handle` no extra parameter: build the truncated reply in a stack `[512]u8` inside `handle` and copy it into `response_buf` before returning. 512 is enough — the message is a header, one question and at most one OPT record, and a question longer than ~270 bytes cannot exist (name ≤ 255).
|
||||
|
||||
### S5.4 Tests (in-file, fake upstream implementing `transport.Client`)
|
||||
|
||||
Build queries with hand-encoded bytes or `packet.ResponseBuilder`; the fake upstream answers by
|
||||
copying a fixture and calling `packet.setId`.
|
||||
Build queries with hand-encoded bytes or `packet.ResponseBuilder`; the fake upstream answers by copying a fixture and calling `packet.setId`.
|
||||
|
||||
- A query for `example.com` A over UDP returns the fake's bytes unchanged.
|
||||
- A response (QR set) sent to the handler → `.drop`, `dropped_malformed == 1`.
|
||||
@@ -702,8 +564,7 @@ copying a fixture and calling `packet.setId`.
|
||||
- Fake returns `error.OutOfMemory` → SERVFAIL (not a drop).
|
||||
- Fake returns `error.Canceled` → `.drop`.
|
||||
- `udpLimit`: no OPT → 512; OPT 1232 → 1232; OPT 200 → 512; OPT 9000 → 4096.
|
||||
- Over UDP, a fake reply of 900 bytes with no OPT in the query → TC=1 reply, ancount 0, question
|
||||
echoed, `truncated == 1`; the same reply over TCP passes through untouched.
|
||||
- Over UDP, a fake reply of 900 bytes with no OPT in the query → TC=1 reply, ancount 0, question echoed, `truncated == 1`; the same reply over TCP passes through untouched.
|
||||
- DO bit set in the query's OPT → set in the SERVFAIL reply's OPT.
|
||||
- Every synthesized reply re-parses with `packet.parse`.
|
||||
|
||||
@@ -718,8 +579,7 @@ copying a fixture and calling `packet.setId`.
|
||||
|
||||
## Session S6: `src/server/udp_server.zig`, `src/server/tcp_server.zig`
|
||||
|
||||
Listeners. Both take a `*handler.Handler`, both use a bounded pool of in-flight tasks so one slow
|
||||
upstream cannot stall the listener, and both count every failure.
|
||||
Listeners. Both take a `*handler.Handler`, both use a bounded pool of in-flight tasks so one slow upstream cannot stall the listener, and both count every failure.
|
||||
|
||||
### S6.1 udp_server.zig
|
||||
|
||||
@@ -774,22 +634,13 @@ pub const UdpServer = struct {
|
||||
```
|
||||
|
||||
Loop:
|
||||
1. `const msg = self.socket.receive(io, slot_query_buf)` into a claimed slot. `error.Canceled` →
|
||||
return. Any other error → `receive_errors`, log, continue (a per-datagram error must not kill the
|
||||
listener).
|
||||
2. `msg.flags.trunc` → `dropped_oversize`, release the slot, continue. A truncated datagram cannot
|
||||
be parsed reliably and answering it would be a guess.
|
||||
3. No free slot → `dropped_no_slot`, continue. Do not queue: an unbounded queue is the cloudflared
|
||||
failure mode (PLAN §1) in a different costume.
|
||||
4. `group.concurrent(io, respondOne, .{ self, io, slot_index })` — an `std.Io.Group` owned by
|
||||
`serve`. `respondOne` calls `self.handler.handle(io, .udp, query, reply)`, sends `.reply` with
|
||||
`self.socket.send(io, &slot.from, bytes)` (a send failure increments `send_errors` and is logged
|
||||
at debug, deduplicated by the caller later), then releases the slot.
|
||||
5. `serve` awaits the group before returning, under cancel protection so in-flight replies are not
|
||||
abandoned mid-send.
|
||||
1. `const msg = self.socket.receive(io, slot_query_buf)` into a claimed slot. `error.Canceled` → return. Any other error → `receive_errors`, log, continue (a per-datagram error must not kill the listener).
|
||||
2. `msg.flags.trunc` → `dropped_oversize`, release the slot, continue. A truncated datagram cannot be parsed reliably and answering it would be a guess.
|
||||
3. No free slot → `dropped_no_slot`, continue. Do not queue: an unbounded queue is the cloudflared failure mode (PLAN §1) in a different costume.
|
||||
4. `group.concurrent(io, respondOne, .{ self, io, slot_index })` — an `std.Io.Group` owned by `serve`. `respondOne` calls `self.handler.handle(io, .udp, query, reply)`, sends `.reply` with `self.socket.send(io, &slot.from, bytes)` (a send failure increments `send_errors` and is logged at debug, deduplicated by the caller later), then releases the slot.
|
||||
5. `serve` awaits the group before returning, under cancel protection so in-flight replies are not abandoned mid-send.
|
||||
|
||||
`deinit` closes the socket and frees the slots. Closing the socket makes a blocked `receive` fail;
|
||||
the loop treats that as a stop signal.
|
||||
`deinit` closes the socket and frees the slots. Closing the socket makes a blocked `receive` fail; the loop treats that as a stop signal.
|
||||
|
||||
### S6.2 tcp_server.zig
|
||||
|
||||
@@ -824,40 +675,25 @@ pub const TcpServer = struct {
|
||||
};
|
||||
```
|
||||
|
||||
Accept loop: `self.server.accept(io)`; `error.Canceled` or `error.SocketNotListening` → return
|
||||
(`deinit` shuts the listening socket down, which is the documented way to unblock a pending accept —
|
||||
API notes). At capacity → close the stream immediately, `rejected_at_capacity`. Otherwise a group
|
||||
task per connection.
|
||||
Accept loop: `self.server.accept(io)`; `error.Canceled` or `error.SocketNotListening` → return (`deinit` shuts the listening socket down, which is the documented way to unblock a pending accept — API notes). At capacity → close the stream immediately, `rejected_at_capacity`. Otherwise a group task per connection.
|
||||
|
||||
Per connection (RFC 7766 §6.2.1.1 — a connection may carry several queries, handled serially):
|
||||
1. Read the 2-byte length prefix. The read is raced against `options.idle_timeout` with
|
||||
`std.Io.Select`; the sleep arm winning means `idle_timeouts` and a clean close.
|
||||
1. Read the 2-byte length prefix. The read is raced against `options.idle_timeout` with `std.Io.Select`; the sleep arm winning means `idle_timeouts` and a clean close.
|
||||
2. `len == 0` → close (`connection_errors`). `len > transport.max_message_len` is impossible (u16).
|
||||
3. Read exactly `len` bytes; a short read → close, `connection_errors`.
|
||||
4. `handler.handle(io, .tcp, query, reply)`; `.drop` closes the connection (there is nothing to
|
||||
frame); `.reply` writes the 2-byte prefix plus the bytes and flushes.
|
||||
4. `handler.handle(io, .tcp, query, reply)`; `.drop` closes the connection (there is nothing to frame); `.reply` writes the 2-byte prefix plus the bytes and flushes.
|
||||
5. Loop back to step 1.
|
||||
6. Close under cancel protection and release the slot.
|
||||
|
||||
Buffers: each `Conn` owns `query: [transport.max_message_len]u8` and
|
||||
`reply: [transport.max_message_len]u8` plus the stream read/write buffers. That is 128 KiB+ per
|
||||
connection slot; with the default 64 slots that is ~8 MiB, inside the PLAN §18 budget. If a session
|
||||
finds a materially smaller layout that keeps the 65535-byte ceiling, take it and say so.
|
||||
Buffers: each `Conn` owns `query: [transport.max_message_len]u8` and `reply: [transport.max_message_len]u8` plus the stream read/write buffers. That is 128 KiB+ per connection slot; with the default 64 slots that is ~8 MiB, inside the PLAN §18 budget. If a session finds a materially smaller layout that keeps the 65535-byte ceiling, take it and say so.
|
||||
|
||||
### S6.3 Tests
|
||||
|
||||
Unit tests that need no socket go in-file (length-prefix framing helper, capacity accounting).
|
||||
The loopback tests live in `src/server/udp_server_integration_test.zig` and
|
||||
`src/server/tcp_server_integration_test.zig`, both guarded by
|
||||
`if (!build_options.integration) return error.SkipZigTest;`:
|
||||
Unit tests that need no socket go in-file (length-prefix framing helper, capacity accounting). The loopback tests live in `src/server/udp_server_integration_test.zig` and `src/server/tcp_server_integration_test.zig`, both guarded by `if (!build_options.integration) return error.SkipZigTest;`:
|
||||
|
||||
- UDP: bind `127.0.0.1:0`, run `serve` in an `Io.Group` task, send a query from a second bound
|
||||
socket with a fake-upstream handler, assert the reply's ID and question match; then send a 5-byte
|
||||
datagram and assert no reply arrives within a short budget and `dropped_malformed` moved.
|
||||
- TCP: listen on `127.0.0.1:0`, connect, send two length-prefixed queries on one connection, read
|
||||
two length-prefixed replies, assert both match, then close; assert `accepted == 1`.
|
||||
- TCP idle timeout: connect, send nothing, assert the server closes the connection and
|
||||
`idle_timeouts == 1` (use a short `idle_timeout` in the test options).
|
||||
- UDP: bind `127.0.0.1:0`, run `serve` in an `Io.Group` task, send a query from a second bound socket with a fake-upstream handler, assert the reply's ID and question match; then send a 5-byte datagram and assert no reply arrives within a short budget and `dropped_malformed` moved.
|
||||
- TCP: listen on `127.0.0.1:0`, connect, send two length-prefixed queries on one connection, read two length-prefixed replies, assert both match, then close; assert `accepted == 1`.
|
||||
- TCP idle timeout: connect, send nothing, assert the server closes the connection and `idle_timeouts == 1` (use a short `idle_timeout` in the test options).
|
||||
- Both: `deinit` while `serve` is blocked ends `serve` without a hang — the group awaits cleanly.
|
||||
|
||||
### S6.4 Acceptance criteria
|
||||
@@ -871,28 +707,21 @@ The loopback tests live in `src/server/udp_server_integration_test.zig` and
|
||||
|
||||
## Session S7: end-to-end integration test
|
||||
|
||||
`src/server/resolver_integration_test.zig`, guarded by
|
||||
`if (!build_options.integration) return error.SkipZigTest;`. Hermetic: loopback sockets and an
|
||||
in-process fake upstream, no external network.
|
||||
`src/server/resolver_integration_test.zig`, guarded by `if (!build_options.integration) return error.SkipZigTest;`. Hermetic: loopback sockets and an in-process fake upstream, no external network.
|
||||
|
||||
### S7.1 Fake upstream
|
||||
|
||||
In this file: a struct implementing `transport.Client` that parses the incoming query, echoes the
|
||||
question and appends one A record (`93.184.216.34`, TTL 300) with `packet.ResponseBuilder`, and
|
||||
counts calls. A second variant returns a configured `transport.PeerFault` on the first N calls.
|
||||
In this file: a struct implementing `transport.Client` that parses the incoming query, echoes the question and appends one A record (`93.184.216.34`, TTL 300) with `packet.ResponseBuilder`, and counts calls. A second variant returns a configured `transport.PeerFault` on the first N calls.
|
||||
|
||||
### S7.2 The test
|
||||
|
||||
1. `std.Io.Threaded` backend, `io`.
|
||||
2. Two `Pool` entries: entry 0 = always-fails fake (priority 10), entry 1 = good fake (priority 20).
|
||||
3. `Handler` over `pool.client()`.
|
||||
4. `UdpServer` bound to `127.0.0.1:0` and `TcpServer` on `127.0.0.1:0`, both served in one
|
||||
`Io.Group`.
|
||||
5. Client side: send an A query for `example.com` over UDP; assert the reply parses, ID matches,
|
||||
RCODE is NOERROR, ancount is 1, and the A record's rdata is `93.184.216.34`.
|
||||
4. `UdpServer` bound to `127.0.0.1:0` and `TcpServer` on `127.0.0.1:0`, both served in one `Io.Group`.
|
||||
5. Client side: send an A query for `example.com` over UDP; assert the reply parses, ID matches, RCODE is NOERROR, ancount is 1, and the A record's rdata is `93.184.216.34`.
|
||||
6. Send the same query length-prefixed over TCP; assert the same.
|
||||
7. Assert failover happened: entry 0's `consecutive_failures >= 1` and `backoff_until != null`;
|
||||
entry 1's `total_successes >= 2`. Read them through `pool.snapshot`.
|
||||
7. Assert failover happened: entry 0's `consecutive_failures >= 1` and `backoff_until != null`; entry 1's `total_successes >= 2`. Read them through `pool.snapshot`.
|
||||
8. Send a third query and assert entry 0's call counter did not grow — it is in backoff.
|
||||
9. `deinit` both servers, cancel and await the group, `threaded.deinit()` — the test must not hang.
|
||||
|
||||
@@ -958,66 +787,26 @@ No session touches milestone 1 or milestone 2 files. A needed change there is re
|
||||
|
||||
- No caching of any kind (Phase 6). The handler forwards every query.
|
||||
- No filtering, blocklists, rules, or blocked-response synthesis (Phase 5).
|
||||
- No query logging, no SQLite, no storage, no config file or ZON parsing (Phase 4). Timeouts,
|
||||
ports and pool entries arrive as parameters or named constants.
|
||||
- No query logging, no SQLite, no storage, no config file or ZON parsing (Phase 4). Timeouts, ports and pool entries arrive as parameters or named constants.
|
||||
- No rate limiting (Phase 6) — `src/server/rate_limiter.zig` is not created here.
|
||||
- No local DoH or DoT **server** endpoints (Phase 9). Only upstream DoH/DoT clients are in scope;
|
||||
`platform/tls_server.zig` is untouched.
|
||||
- No local DoH or DoT **server** endpoints (Phase 9). Only upstream DoH/DoT clients are in scope; `platform/tls_server.zig` is untouched.
|
||||
- No local records, no conditional forwarding, no CNAME uncloaking (Phase 5/7).
|
||||
- No web UI, no REST API, no `/metrics`. `Pool.snapshot` exists so Phase 8 has something to read;
|
||||
it is not exposed anywhere yet.
|
||||
- No signal handling, no `src/server/shutdown.zig`, no `main.zig` wiring. Servers stop through
|
||||
`deinit` plus group cancellation; process lifetime is Phase 7's problem.
|
||||
- No DNS name resolution for upstream hosts — DoT endpoints take IP literals, DoH resolution is
|
||||
`std.http.Client`'s business.
|
||||
- No web UI, no REST API, no `/metrics`. `Pool.snapshot` exists so Phase 8 has something to read; it is not exposed anywhere yet.
|
||||
- No signal handling, no `src/server/shutdown.zig`, no `main.zig` wiring. Servers stop through `deinit` plus group cancellation; process lifetime is Phase 7's problem.
|
||||
- No DNS name resolution for upstream hosts — DoT endpoints take IP literals, DoH resolution is `std.http.Client`'s business.
|
||||
- No HTTP/2, no DoQ, no connection reuse for DoT, no DNSSEC validation (DO bit passes through).
|
||||
- No 0x20 query-name randomization. Case-insensitive comparison per RFC 4343 is required; generating
|
||||
mixed-case queries is not.
|
||||
- No changes to `src/dns/`. If a helper is missing there, report it — do not add transport-aware
|
||||
code to the pure core.
|
||||
- No 0x20 query-name randomization. Case-insensitive comparison per RFC 4343 is required; generating mixed-case queries is not.
|
||||
- No changes to `src/dns/`. If a helper is missing there, report it — do not add transport-aware code to the pure core.
|
||||
|
||||
## As built
|
||||
|
||||
The implementation matches the spec with these review-driven refinements (three
|
||||
review rounds; findings went 9 → 3 → 0):
|
||||
The implementation matches the spec with these review-driven refinements (three review rounds; findings went 9 → 3 → 0):
|
||||
|
||||
- `pool.zig`: each `Entry` carries a `busy: std.Io.Mutex` held for the whole
|
||||
attempt against that entry, so one entry's client (and its buffers/TLS state)
|
||||
is never used by two tasks at once. Pass one re-checks `available` with a
|
||||
fresh `now` after acquiring `busy` (the wait can outlive the health it was
|
||||
admitted under); pass two probes without a re-check by design. Lock order is
|
||||
always `busy` then `Pool.mutex`; the pool mutex still guards health and
|
||||
`snapshot` only.
|
||||
- `health.zig`: staleness rules are symmetric. The newest outcome is the @max
|
||||
of `last_success_at`/`last_error_at`. A stale success counts into
|
||||
totals/window and @max-updates `last_success_at`, but does not clear
|
||||
`consecutive_failures`/`backoff_until`. A stale failure counts into
|
||||
totals/window and @max-updates `last_error_at`, but only increments
|
||||
`consecutive_failures`/extends backoff when no newer success exists, and
|
||||
never overwrites a newer failure's error text.
|
||||
- `udp_server.zig`: each slot's reply buffer is
|
||||
`[transport.max_message_len]u8` (not `max_datagram`) so an oversized upstream
|
||||
reply reaches the handler and becomes a TC=1 reply instead of SERVFAIL
|
||||
(64 slots ≈ 4.4 MiB, inside the PLAN §18 budget). A `dropped_handler` stat
|
||||
counts `.drop` outcomes from the handler.
|
||||
- `tcp_server.zig`: a shutdown flag, set under the slot mutex before the
|
||||
active-connection scan, closes the accept-vs-deinit race — a stream accepted
|
||||
during shutdown is closed immediately instead of claimed.
|
||||
- `handler.zig`: OPT validation walks every record section. OPT in answer or
|
||||
authority, more than one OPT, a non-root OPT owner, or an unparseable OPT →
|
||||
FORMERR (RFC 6891); only a genuinely absent OPT takes the no-OPT path.
|
||||
- `dot_client.zig`: handshake `ReadFailed`/`WriteFailed` unwrap the stream's
|
||||
stored cause through `transport.mapLocal` first, and certificate-bundle
|
||||
`OutOfMemory` is a local resource, not a peer fault.
|
||||
- `transport.zig`: `Endpoint.parse` rejects userinfo/query/fragment delimiters
|
||||
(`@`, `?`, `#`) in the authority, and `?`/`#` in a DoH path.
|
||||
- `dot_client.zig` (follow-up, tls_name commit): `DotClient.init` takes a
|
||||
`tls_name`; it is the SNI and certificate-verification name, while the dial
|
||||
target stays `endpoint.host`. Empty keeps the endpoint host, which is the
|
||||
behavior described above. The same follow-up fixed a send bug this file
|
||||
had from the start, invisible until a DoT handshake first succeeded:
|
||||
`tls.Client.flush` only encrypts into the socket writer's buffer and never
|
||||
flushes it, so the query never left the process and the peer eventually
|
||||
closed the connection (`ReceiveFailed`/`EndOfStream`). `TlsStream.flush` now
|
||||
does both flushes and `exchange` calls it; a hermetic loopback test in
|
||||
`tls_client_integration_test.zig` covers it.
|
||||
- `pool.zig`: each `Entry` carries a `busy: std.Io.Mutex` held for the whole attempt against that entry, so one entry's client (and its buffers/TLS state) is never used by two tasks at once. Pass one re-checks `available` with a fresh `now` after acquiring `busy` (the wait can outlive the health it was admitted under); pass two probes without a re-check by design. Lock order is always `busy` then `Pool.mutex`; the pool mutex still guards health and `snapshot` only.
|
||||
- `health.zig`: staleness rules are symmetric. The newest outcome is the @max of `last_success_at`/`last_error_at`. A stale success counts into totals/window and @max-updates `last_success_at`, but does not clear `consecutive_failures`/`backoff_until`. A stale failure counts into totals/window and @max-updates `last_error_at`, but only increments `consecutive_failures`/extends backoff when no newer success exists, and never overwrites a newer failure's error text.
|
||||
- `udp_server.zig`: each slot's reply buffer is `[transport.max_message_len]u8` (not `max_datagram`) so an oversized upstream reply reaches the handler and becomes a TC=1 reply instead of SERVFAIL (64 slots ≈ 4.4 MiB, inside the PLAN §18 budget). A `dropped_handler` stat counts `.drop` outcomes from the handler.
|
||||
- `tcp_server.zig`: a shutdown flag, set under the slot mutex before the active-connection scan, closes the accept-vs-deinit race — a stream accepted during shutdown is closed immediately instead of claimed.
|
||||
- `handler.zig`: OPT validation walks every record section. OPT in answer or authority, more than one OPT, a non-root OPT owner, or an unparseable OPT → FORMERR (RFC 6891); only a genuinely absent OPT takes the no-OPT path.
|
||||
- `dot_client.zig`: handshake `ReadFailed`/`WriteFailed` unwrap the stream's stored cause through `transport.mapLocal` first, and certificate-bundle `OutOfMemory` is a local resource, not a peer fault.
|
||||
- `transport.zig`: `Endpoint.parse` rejects userinfo/query/fragment delimiters (`@`, `?`, `#`) in the authority, and `?`/`#` in a DoH path.
|
||||
- `dot_client.zig` (follow-up, tls_name commit): `DotClient.init` takes a `tls_name`; it is the SNI and certificate-verification name, while the dial target stays `endpoint.host`. Empty keeps the endpoint host, which is the behavior described above. The same follow-up fixed a send bug this file had from the start, invisible until a DoT handshake first succeeded: `tls.Client.flush` only encrypts into the socket writer's buffer and never flushes it, so the query never left the process and the peer eventually closed the connection (`ReceiveFailed`/`EndOfStream`). `TlsStream.flush` now does both flushes and `exchange` calls it; a hermetic loopback test in `tls_client_integration_test.zig` covers it.
|
||||
|
||||
Reference in New Issue
Block a user