diff --git a/specs/milestone-7.md b/specs/milestone-7.md new file mode 100644 index 0000000..6fee907 --- /dev/null +++ b/specs/milestone-7.md @@ -0,0 +1,700 @@ +# Milestone 7: serving pipeline composition (PLAN Phase 7) + +Goal: `nxdns run` serves DNS end to end — rate limit → parse → group → local records → +forward zones → filtering → safe-search → cache → upstream → CNAME uncloaking → cache store → +async query log — with client auto-materialization, pause/resume, and a composition root that +starts every background loop and shuts down cleanly on SIGINT/SIGTERM. + +Ground truth: PLAN.md §4 (pipeline diagram, lines 151-173), §6 (DNS behavior), §7.2 +(auto-materialization), §8 (cache key), §10 (rate limit), §11.4/§11.6 (logging, disk), Phase 7 +text at PLAN.md:606-608. Verified Zig facts: specs/research/zig-0.16-api-notes.md plus the +scratchpad note m7-explore-zig.md (signals, Io.Group, cancelable net ops) — the load-bearing +facts are restated inline below where a session needs them. + +## Rulings (engineering calls resolving PLAN ambiguities — binding) + +1. **Scope**: Phase 7 owns `nxdns run` end to end: `src/app.zig` (composition root), + `src/server/shutdown.zig` (signal → event), the `cli.runRun` body, and both listener edits. +2. **Signals**: no signalfd/epoll/timerfd. `std.posix.sigaction` handler for INT and TERM that + calls `std.Io.Event.set` on a file-scope event (`set` is async-signal-safe on the Threaded + Linux backend: raw futex wake, no locks — Threaded.zig:1105,17235). +3. **Locking**: `DnsCache` and `RateLimiter` each get one `std.Io.Mutex` owned by the handler. + Household scale makes contention irrelevant; sharding is over-engineering. +4. **Null snapshot** (`Manager.acquire` returns null before the first successful reload): fail + open — skip group/filter/safe-search, use the default group semantics, count + `unfiltered_queries`. DNS availability beats filtering for a household. The app attempts one + synchronous `reload` before serving; a failure warns and serving starts anyway. +5. **Cache scope**: one global cache, no group in the key (PLAN §8). Safe because blocked + responses are never cached. Safe-search responses are not cached either (13 domains, rare; + caching them under the rewritten name would force answer re-encoding on hit). Paused-mode + upstream answers cache normally (they are ordinary upstream responses). Forward-zone + responses cache normally (PLAN §6.5) under the same global key. +6. **Local vs forward zone**: local records win (checked first; PLAN §6.4 puts them before + everything but rate limit/parse). +7. **Forward zones bypass** filtering, safe-search, and uncloaking (PLAN §6.5). They are logged + with `upstream` = the resolver's canonical text. +8. **Rate limit position**: after the 12-byte header read (the REFUSED reply needs the ID), + before full parse. Refused queries are counted (`Stats.refused`), never query-logged. + Localhost is not exempt (PLAN §10's localhost relaxation is the Phase 8 API limiter only). +9. **Non-IN qclass**: skip filtering/cache/safe-search/uncloak, go straight upstream, log + normally. Rare and harmless; special-casing CHAOS is scope creep. +10. **Safe-search mechanics** (per safesearch.zig:56 doc): the outgoing query's QNAME becomes + the target; the reply to the client keeps the original question, starts with a synthesized + CNAME original→target, then copies only the A/AAAA answer records from the upstream + response verbatim (their RDATA is raw bytes, so no compression-pointer re-encoding; other + rtypes are skipped). Applies only on the upstream path — never to local records or forward + zones. +11. **CNAME uncloaking** (PLAN §6.3): walk the answer-section CNAME chain of the upstream + response — no re-resolution. Depth 8 = at most 8 CNAME links followed. Each target is + normalized and evaluated for the same group; a blocked target synthesizes a blocked + response for the ORIGINAL qname/qtype. `block_reason` = `"cname:" ++ @tagName(reason)` + (fits the 32-byte Entry field). Uncloaked-blocked responses are not cached. Skipped when + paused, for forward zones, and for non-IN. +12. **Local CNAME records** are returned as-is (writeAnswers emits the CNAME and stops, + milestone-5 As-built). The client re-queries the target, which then flows through the full + pipeline — that is how §6.4's "targets resolve through the normal pipeline" is satisfied. +13. **ECS** (PLAN §6.1): `strip` (default) removes a client-sent ECS option from the outgoing + query by rebuilding it (header + question + OPT sans option 8); other options are + preserved. `forward` sends the query unchanged and puts the raw ECS option payload + (≤ `dns_cache.max_ecs_len` = 40 bytes) into the cache key. If a rewritten query cannot fit + the 512-byte build buffer (freak OPT payloads), the original is forwarded unchanged, + `Stats.ecs_strip_failed` counts it, and the query is excluded from the cache in both + directions (the subnet reached the resolver; the strip-mode key says nothing about it). +14. **Client auto-materialization** (PLAN §7.2): never a hot-path DB write. The handler records + the client in a mutex-guarded fixed-capacity pending table; a background loop flushes every + 60 s with its OWN config.db connection: `INSERT ... hand_edited=0 ON CONFLICT(ip) DO UPDATE + last_seen`. At most one write per client per flush. Table full → drop silently (counted). +15. **Snapshot staleness**: a materialized client reaches the matcher snapshot only on the next + manager reload. Correct behavior does not depend on it: `groupForClient` already falls back + to prefix/default, and materialized rows exist for UI visibility (Phase 8 reads the DB + directly). No reload is triggered per client. +16. **Client pruning** (PLAN §7.2 + milestone-6 ruling 1): the same background loop prunes + daily: `DELETE FROM clients WHERE hand_edited=0 AND last_seen < now - retention_days` + (`logging.retention_days`, same knob as the query log — documented). `retention.zig` stays + frozen and querylog-only. `last_seen` is the only signal (§3.6 forbids cross-DB joins). +17. **Disk gate**: the client flush/prune loop skips its pass when `writesAllowed()` is false; + the logger holds its batches. As built: milestone-6.md:410 assigned the manager's + consumption to Phase 7 and no seam existed, so S6 added `Manager.monitor: + ?*disk_monitor.Monitor` (set by the composition root after `init`, before `runScheduler`) + plus a `refreshes_gated` counter read through `refreshesGated()`. Only scheduled passes are + gated: `runScheduler`'s periodic pass and the source-refresh half of the startup pass. + `reload` and `refreshAll` stay ungated (operator actions). The startup pass's + `reloadLocked` runs ahead of the gate: publishing an already-compiled snapshot is a read, + and a full disk must not cost the household its filtering as well as its downloads. A + gated pass logs one warn line and counts. +18. **Pause/resume** (PLAN §13.1 mechanism): suspends FILTERING only (evaluate + safe-search + + uncloak are skipped); everything else — local records, forward zones, cache, upstream, + logging — runs unchanged. Global, not per-group. In-memory only; a restart resumes + filtering (safe default). State: one `std.atomic.Value(i64)` — 0 = active, -1 = paused + indefinitely, else = paused until that unix second (auto-resume by compare, no timer task). + Phase 8's endpoints will call `pauseFor`/`unpause` on the shared pointer. +19. **Paused queries** log `blocked=false, block_reason=""`; `Stats.paused_queries` counts + them. Evaluation is skipped entirely (not evaluated-then-ignored). +20. **Log field semantics**: `response_time_us` = wall time from `handle` entry to reply ready. + `cache_hit`: true/false on the upstream path, null for local/forward/refused-never-logged; + `upstream`: "" on cache hit, "local" for local records, resolver text for forward zones, + the pool endpoint's canonical text for upstream answers (pool already tracks which entry + answered — if unavailable without new plumbing, "" and a spec note; do not rebuild the + pool for it: `upstream` may be "pool" as the compromise, S2 decides and records). + `block_reason` = `@tagName(matcher.Reason)` truncated to 32, `"cname:"`-prefixed for + uncloaking. Malformed/dropped queries are never query-logged (handler stats only). +21. **Connections**: config.db — one opened by the app (migrate/bootstrap/readConfig, then + handed to the manager for reloads), plus one dedicated to the client loop. querylog.db — + one for the log writer, one for retention (retention.zig:79 contract). Four total. +22. **Shutdown order**: shutdown event fires → `logger.shutdown(io)` (closes the queue; the + writer drains and its task returns; a writer stuck in the disk gate holds its final batch — + known, documented) → `group.cancel(io)` (requests cancel and joins every task) → deinit + servers/pool/manager/state → close DBs → return. Canceled UDP recv / TCP accept return + `error.Canceled` on the Threaded backend (Threaded.zig:12439,12877). +23. **No runtime config re-read** in Phase 7. The blocklist scheduler reloads snapshots; + everything else is restart-required until Phase 8 mutates the DB and calls reload. +24. **No new config fields, no schema change.** Pause is in-memory; pruning reuses + `retention_days`; ECS uses the existing `edns.ecs_mode`. +25. **Performance** (PLAN §18): no formal benchmark in this milestone; the integration test + asserts correctness only. The targets stay in PLAN for a later measurement pass. + +## Sessions + +Dependency graph: S1 + S4 parallel (wave 1) → S2 (needs S1's edns API) → S3 (needs S2's +signature in code) → S6 (needs S2, S3, S4) → S7 (needs S6). The orchestrator wires +`src/tests.zig` after each wave. + +--- + +## Session S1: ECS helpers in dns/edns.zig + +Owns: `src/dns/edns.zig` (extend in place; existing API is frozen — additive only). + +### S1.1 API + +```zig +/// The raw payload of the ECS option (code 8) in `opt`, or null when absent. +/// `pkt` is the packet `opt` was parsed from. error.BadOption on malformed option lengths. +pub fn ecsPayload(pkt: packet.Packet, opt: OptRecord) error{BadOption}!?[]const u8 + +/// Rebuilds `query` without its ECS option into `out`: header + question copied verbatim, +/// then the OPT re-encoded with every option except code 8. Returns `.unchanged` when the +/// query has no OPT or no ECS option (nothing written), else `.rewritten` with the slice +/// into `out`. `pkt`/`opt` are the caller's already-parsed views of `query`. +/// error.Overflow when `out` is too small. +pub const StripResult = union(enum) { unchanged, rewritten: []u8 }; +pub fn stripEcs(query: []const u8, pkt: packet.Packet, opt: OptRecord, out: []u8) + error{ BadOption, Overflow }!StripResult +``` + +Constraints: pure, allocation-free, no Io. The rebuilt query keeps the original ID, flags and +question bytes verbatim (copy `query[0..opt_record_offset_of_question_end]` — concretely: copy +everything up to the OPT record's start; the OPT is the sole additional record in a query per +the existing handler validation, and `packet.parse` yields its offset). ARCOUNT stays 1 when +the OPT is re-emitted (it always is — only the option list shrinks). Use the existing +`encodeOpt` for the re-emission. The RDATA walk follows the same bounds checks `findOption` +uses. + +### S1.2 Acceptance + +- [ ] `zig fmt --check` + `zig ast-check` clean. +- [ ] Tests in-file: ECS present → payload returned, stripped query parses cleanly and + re-parse finds no option 8, other options survive byte-exact, DO bit and payload size + survive; no OPT → `.unchanged`; OPT without ECS → `.unchanged`; malformed option length + → `error.BadOption`; `out` one byte short → `error.Overflow`. +- [ ] Temp-root run of the file's tests passes. + +### S1 As built + +Two spec claims were wrong and the implementation corrects them. (1) The handler does NOT +guarantee the OPT is the sole additional record (handler.zig:105 rejects only misplaced OPTs; +packet.parse rejects only a second OPT) — `stripEcs` therefore copies the bytes after the OPT +verbatim and all four section counts survive. Caveat, documented on the function: a compression +pointer in a trailing record that targets a byte inside the OPT's option list shifts after the +rewrite — such a pointer is already malformed (RFC 1035 §4.1.4) and the consequence is an +upstream FORMERR. (2) `packet.parse` does not yield the OPT's start offset; a private +`optRecordStart` re-walks the sections and matches the record on its full RDATA span, so a +hand-built `OptRecord` cannot aim it at a different record (deriving the start backwards is +forgeable: a `0x00xx` transaction ID makes a pointer's low byte decode as a root name). +`encodeOpt` takes one contiguous options slice, so `stripEcs` emits it with an empty list, +streams the survivors, and patches RDLENGTH at `opt_rdlength_offset` (9); a guard test pins +that offset against `encodeOpt`'s real output. All options with code 8 are removed (a malformed +query can carry several). Walk failures map to `error.BadOption` (frozen error set; reachable +only for hand-assembled Packets, documented). After review: `stripEcs` asserts both that +`query` aliases `pkt.bytes` and that `out` is pointer-range disjoint from `query` in both +directions; the caller owns keeping the rebuild buffer separate from the received query +buffer (S2's `scratch.build` satisfies this). The import is aliased `packet_mod` (seven existing declarations +named `packet`). `ecsPayload` was built here, then removed in the review round when the +handler's `subnetForKey` single-walk replaced its only caller (see S2 As-built); the +forward-mode clamp policy lives there — a payload longer than `dns_cache.max_ecs_len` (40) +makes the query uncacheable (truncating the key would merge distinct subnets). 7 S1 tests +remain, 25 in file. + +--- + +## Session S2: pipeline in server/handler.zig (+ pause.zig) + +Owns: `src/server/handler.zig` (rewrite around the existing skeleton), `src/server/pause.zig` +(new). The biggest session. Everything below is frozen interface — S3 and S6 build against it. + +### S2.1 pause.zig + +```zig +pub const Pause = struct { + until: std.atomic.Value(i64) = .init(0), + pub fn isPaused(self: *const Pause, now_s: i64) bool // 0→false, -1→true, else now_s < until + pub fn pauseFor(self: *Pause, now_s: i64, duration_s: ?u32) void // null → -1 (indefinite) + pub fn unpause(self: *Pause) void // store 0 +}; +``` + +Auto-resume is the compare in `isPaused` — no timer. `.monotonic` ordering is sufficient (a +lone i64 flag, no dependent data). Tests: the three states, expiry boundary (`now == until` +is resumed), pauseFor overwrites a previous pause. + +### S2.2 Handler struct (frozen for S3/S6) + +```zig +pub const Handler = struct { + upstream: transport.Client, + blocking: response.Options, + ecs_mode: model.EcsMode = .strip, + forward_read_timeout: std.Io.Clock.Duration, + manager: ?*manager.Manager = null, + records: *const records.Records, + zones: *const forward_zones.Zones, + cache: ?*dns_cache.DnsCache = null, + cache_mutex: std.Io.Mutex = .init, + limiter: ?*rate_limiter.RateLimiter = null, + limiter_mutex: std.Io.Mutex = .init, + logger: ?*logger_mod.Logger = null, + pause: ?*pause.Pause = null, + tracker: ?*clients.Tracker = null, + stats: Stats = .{}, +}; +``` + +Every `?*` defaults to null so existing tests and the check command construct a bare handler +exactly as today (upstream + blocking + records/zones + timeout are the only required fields; +`records`/`zones` may point at the shared `Records.empty`/`Zones.empty` constants — if +`Zones.empty` does not exist yet, S2 adds it mirroring `Records.empty`). + +```zig +pub const Scratch = struct { + normalize: [types.max_name_len]u8, + key: [dns_cache.max_key_len]u8, + build: [512]u8, // rewritten outgoing query (ECS strip / safe-search) + frame: [forward_client.min_frame_buf]u8, // forward-zone client frame buffer +}; + +pub fn handle(self: *Handler, io: std.Io, which: Transport, from: address.NetAddress, + query: []const u8, response_buf: []u8, scratch: *Scratch) Outcome +``` + +`Outcome` unchanged. Still no error union — every failure is a synthesized rcode or a counted +drop. New Stats counters (all `std.atomic.Value(u64)`): `refused`, `blocked`, +`uncloak_blocked`, `local_answers`, `forward_zone_answers`, `cache_hits`, `paused_queries`, +`unfiltered_queries`, `safesearch_rewrites`, `ecs_strip_failed`, `tracker_full`. + +### S2.3 Pipeline order inside handle (binding; rulings 4-13, 18-20 apply) + +1. Read the 12-byte header (existing code). QR/opcode/qdcount checks as today. +2. Rate limit: `limiter_mutex.lock(io)`, `check(now, from.key())`, unlock. Refused → + synthesize REFUSED (reuse `synthesize`), count, return. `now` from + `std.Io.Clock.awake.now(io)` (limiter windows are awake-anchored, milestone-6 As-built). +3. Full parse + OPT validation (existing code). Record `start_us` once from + `std.Io.Clock.real.now(io)` for both `response_time_us` and cache `now_s`. +4. `tracker.track(from)` (nonblocking; full table counts `tracker_full`). +5. Snapshot: `if (self.manager) |m| m.acquire(io) else null`; released once on every exit path + (single defer). Null snapshot → `unfiltered_queries`, group features off (ruling 4). +6. Non-IN qclass → step 12 (upstream direct), no cache (ruling 9). +7. Normalize qname into `scratch.normalize`. +8. Local records: `hasName` → `lookup`; answer via `ResponseBuilder` + `writeAnswers` + + `addOptEcho`, authoritative. A name that exists with no records of the qtype → NODATA, + authoritative. Log (`upstream="local"`, `cache_hit=null`) and return. +9. Forward zones: `zones.match` → construct `ForwardClient` on the stack + (`init(zone.resolver, &scratch.frame, self.forward_read_timeout)`), cache get → miss → + `exchange` into `response_buf` → cache put (classify), log (`upstream=resolver text`), + return. Filtering/safe-search/uncloak bypassed (ruling 7). Exchange failure → SERVFAIL. +10. Pause check (`isPaused(now_s)`) → skip 11 and the uncloak in 14; count `paused_queries`. +11. Filtering: `snapshot.evaluate(group, domain)`; blocked → `response.writeBlocked` into + `response_buf`, log (`blocked=true`, reason), return. Then safe-search: + `snapshot.safeSearch(group)` and `safesearch.rewrite(domain)` → remember the target; + outgoing QNAME = target (query rebuilt in `scratch.build`; count `safesearch_rewrites`). +12. ECS (ruling 13): `.strip` → `edns.stripEcs` into `scratch.build` (compose with the + safe-search rebuild: build once with both the final QNAME and the filtered OPT — one + rebuild, not two); `.forward` → `edns.ecsPayload` for the key. Overflow → + `ecs_strip_failed`, forward original. +13. Cache get (skip for safe-search, ruling 5): key = `buildKey(outgoing qname, qtype, qclass, + do_bit, ecs_payload_or_null)` under `cache_mutex`; hit → copy is already in + `response_buf`-sized `out` (pass `response_buf`), `packet.setId`, UDP truncation check as + today, log (`cache_hit=true`, `upstream=""`), return. +14. Upstream: `self.upstream.exchange(io, outgoing, response_buf)` (existing error mapping to + SERVFAIL/drop stays). Then uncloak (ruling 11) unless paused/non-IN: iterate + `packet.answers`, follow CNAMEs from the outgoing qname via `record.rdataCname`, depth ≤ 8, + normalize each target, `evaluate`; blocked → `writeBlocked` for the ORIGINAL question, + count `uncloak_blocked`, log (`blocked=true`, `"cname:"` reason), return. +15. Safe-search synthesis (ruling 10): new `ResponseBuilder` with the original question, + `addAnswer(original, CNAME, target)`, then each A/AAAA answer record of the upstream + response verbatim (owner = target name), `addOptEcho`. The upstream bytes are read from + `response_buf` while the builder writes into... **the builder needs its own buffer**: build + into a second half — concretely `handle` requires `response_buf.len >= 1024` already via + assert; S2 adds a `synth: [4096]u8` field to `Scratch` for the safe-search build, then + copies the finished reply into `response_buf`. A synthesized reply that cannot fit 4096 → + SERVFAIL (safe-search answers are tiny in practice). +16. Cache put: upstream path only, not blocked/safe-search (ruling 5): `classify` → + `cache_mutex` → `put`. Allocation failure inside put → skip caching (already + `Allocator.Error`; count nothing, put's stats cover it). +17. UDP truncation check (existing). Log the entry: `Entry.init(.{ .timestamp = now_s, .domain + = normalized domain, .client_ip = formatted from, .qtype, .blocked=false, .response_time_us, + .cache_hit=false, .upstream = per ruling 20 })`, `logger.log(io, entry)`. Return reply. + +The query-log write happens on every replied path (blocked, local, forward, cache, upstream, +paused) — one call site per return is acceptable; a tiny private `logEntry` helper keeps it +readable. Refused/malformed/dropped are never logged (rulings 8, 20). + +### S2.4 Acceptance + +- [ ] `zig fmt --check` + `zig ast-check` clean; temp-root tests pass. +- [ ] Existing handler tests still pass (bare handler, null optionals). +- [ ] New in-file tests (fake `transport.Client` fixtures, as the existing tests do): REFUSED + on limit exhaustion; local A answer + NODATA; forward-zone match exchanges via the zone + resolver fake and bypasses a blocklisted name; blocked domain → zero-IP answer with + blocking TTL; allow-over-block passes; cache miss→put→hit with aged TTL + fresh ID; + blocked responses not cached; uncloak: CNAME chain to a blocked target at depth 1 and at + depth 8 blocks, depth 9 does not, `cname:` reason logged; safe-search: rewritten QNAME + goes upstream, reply carries original question + CNAME + A records only; ECS strip: + option 8 gone upstream, others preserved; ECS forward: payload lands in the cache key + (two subnets → two entries); paused: blocklisted name answers normally and counts; + non-IN bypasses filtering; null manager → unfiltered count; every path's log entry + fields per ruling 20 (capture via a Logger with a small queue, drain and assert). + +### S2 As built + +`Handler` gained one additive field: `negative_ttl_max: u32 = 0` (set from +`cfg.cache.negative_ttl_max` — `DnsCache` keeps no copy of its config and `classify` needs the +value; **S6 must wire it or negative caching is silently off**). `Scratch` gained `synth: +[4096]u8` (spec-blessed) and `uncloak: [types.max_name_len]u8` (target normalization must not +clobber `normalize`, which still holds the logged name). `Zones.empty` already existed +(forward_zones.zig:34) — the S2.2 parenthetical was stale. Both mutexes and the tracker use +`lockUncancelable` — `std.Io.Mutex.lock` returns `Cancelable!void` (Io.zig:1602) and `handle` +has no error union. Safe-search + ECS strip compose via two buffers: rebuild into +`scratch.build`, strip into `scratch.synth` (stripEcs requires non-overlapping output); `synth` +is reused for the reply after the upstream answers. Uncloaking is also skipped after a +safe-search rewrite (the answers belong to the provider's target, which the client never asked +about). `safesearch_rewrites` counts at the rebuild, not the table lookup. SERVFAIL paths are +counted but never query-logged (the Entry schema has no status field; a logged row would read +as success — accepted ruling). Forward zones are exempt from the ECS strip (accepted ruling: +the RFC 7871 concern is third-party upstreams, not the LAN resolver). Ruling 20's open choice: +`upstream = "pool"` for pool answers; forward-zone rows log `cache_hit` false/true (ruling 20's +"null" contradicted ruling 5 — null is now local records and blocked answers only). The +safe-search CNAME TTL is the minimum TTL of the copied A/AAAA records, zero when none. +`handle` keeps the 512-byte floor assert (the spec's "1024" was wrong); the copy out of +`synth` is length-checked with SERVFAIL fallback. `tracker_full` mirrors +`Tracker.snapshotStats().dropped_full` after each track (track returns nothing; an always-zero +counter would lie). `max_cname_depth = 8` public. `queries` still means pool-answered queries. +23 new handler tests + 25 adapted + 6 pause tests. After review, two ECS cache holes fixed at +the root: (1) a failed ECS strip forwards the client's subnet upstream, so the answer may be +subnet-specific while the `.strip`-mode key names no subnet — `outgoingQuery` returns +`Outgoing { bytes, cacheable }` and a failed strip sets `cacheable = false`, skipping both the +cache get and the put; (2) `.forward` mode forwards the whole OPT but `edns.findOption` +reports only the first ECS option, so a query carrying several would key on one subnet while +the resolver answered for another — a local `subnetForKey` walks the option list once and +returns `.uncacheable` for a repeated option 8, for a payload over `dns_cache.max_ecs_len`, +and for a list that will not walk; `cacheKey` then declines to cache. RFC 7871 §6 permits one +ECS option, but a query is client-controlled, so a duplicate is treated as hostile. Both fixes +are mutation-checked. `edns.ecsPayload` lost its only caller (the key needs the option count +as well as the payload; one walk yields both) and is removed from edns.zig as dead surface. NOTE: the plain `zig build test` does not +typecheck the listener call sites (serve is never analyzed there) — only `-Dintegration` +validates S3's edit. + +--- + +## Session S3: listeners pass the client address and scratch + +Owns: `src/server/udp_server.zig`, `src/server/tcp_server.zig`, and the three integration test +files whose `Handler{…}` literals the S2 struct change broke: +`src/server/udp_server_integration_test.zig`, `src/server/tcp_server_integration_test.zig`, +`src/server/resolver_integration_test.zig` (S2 measured nine stale literals missing +`blocking`/`forward_read_timeout`/`records`/`zones` at udp:90/128/163/204, tcp:153/183/216/268, +resolver:229, plus the handle-arity change once those are fixed). + +- `Slot` (udp) gains `scratch: handler.Scratch`; the call becomes + `self.handler.handle(io, .udp, address.NetAddress.fromIp(slot.from), slot.query[0..slot.len], + &slot.reply, &slot.scratch)`. +- `Conn` (tcp) gains `peer: std.Io.net.IpAddress` (captured at claim from + `stream.socket.address`) and `scratch: handler.Scratch`; call updated the same way. +- No other behavior change. Slot/Conn are pool-allocated already, so the added ~5 KiB per slot + changes no lifetime. +- Acceptance: fmt/ast clean; both files' existing tests updated and passing; integration tests + (`-Dintegration` udp/tcp/resolver) still pass unchanged in behavior. + +### S3 As built + +`Conn.peer` is captured in `claim` inside the same mutex-held section that stores the stream — +written once per connection, read only by that connection's task; verified from source that +`accept` fills `Socket.address` with the peer (net.zig:1442 → Threaded.zig:12439 +`addressFromPosix`), and a new TCP integration test asserts the captured peer is the real +loopback client (an uninitialized field would typecheck and silently corrupt rate +limiting/grouping/logging). Importing the `address` module forced renaming the bind/listen +address parameters to `bind_address`/`listen_address` (Zig shadowing rule; call sites are +positional, nothing else changed). The nine stale integration literals became a per-file +`bareHandler` fixture deriving `blocking` from `model.Blocking{}` and `forward_read_timeout` +from `model.readTimeout(.{})` (3000 ms, `.awake`), so the fixtures cannot drift from config +defaults. Memory-cost comments updated for the scratch: UDP ≈74 KiB/slot (≈4.6 MiB at 64), +TCP ≈137 KiB/slot (≈8.8 MiB at 64); the UDP <8 MiB budget assertion still holds. +After review: `TcpServer.serve` distinguishes its two exit paths, because the composition root +cancels its task group before any listener `deinit` runs (the deinit defers are declared +first, so `group.cancel` executes first). `acceptLoop` returns a `Stop` enum: `.closing` when +`deinit` stopped it (`error.SocketNotListening`, a `.shutting_down` claim, or the state flag) +and `.canceled` when the task is being canceled (`error.Canceled` from `accept` or the retry +sleep). On `.closing` the connection group drains under `.blocked` protection as before — +`beginShutdown` already unblocked every live stream, so the wait is bounded by the shutdown. +On `.canceled` the group is canceled instead: nothing has shut the streams down, `deinit` +cannot run until `serve` returns, and RFC 7766 §6.2.1.1 lets a client hold a connection open +indefinitely by asking again inside the idle budget — draining would let one client stall +process shutdown. The cost is the single reply mid-write. The old await under `.blocked` +protection was what made cancel invisible: `Syscall.start` (Threaded.zig:1349) never arms +cancellation while protection is blocked. Per-connection cleanup is unaffected (`serveConn`'s +`defer finish` runs on the canceled path; `finish` already used `lockUncancelable` + `.blocked` +around the close). Pinned by "a canceled serve does not wait for a live connection" (blocked +read, 600 s idle budget, cancel must return; reinstating drain-always hangs the suite). The +same await shape remains in `UdpServer.serve` deliberately: a UDP task is one bounded upstream +exchange, not a client-paced session. + +--- + +## Session S4: client tracker + repo additions + +Owns: `src/server/clients.zig` (new), `src/storage/repositories/clients_repo.zig` (extend). + +### S4.1 clients_repo additions (SQL; existing functions frozen) + +```zig +/// INSERT OR CONFLICT: materialize an unseen client (hand_edited=0, first_seen=last_seen=now) +/// or touch last_seen on an existing row (hand-edited rows get last_seen touched too — the +/// operator sees liveness; group/name/hand_edited never change here). +pub fn upsertSeen(database: *db.Db, ip: []const u8, now_s: i64) db.Error!void +/// DELETE hand_edited=0 AND last_seen < cutoff. Returns rows deleted. +pub fn pruneStale(database: *db.Db, cutoff_s: i64) db.Error!u32 +``` + +`ON CONFLICT(ip) DO UPDATE SET last_seen=excluded.last_seen`. Check the actual clients schema +(unique key on ip) in storage/config_schema.zig and use the real column names. The existing +`insertClient` (hand_edited=1, import path) stays untouched. + +### S4.2 Tracker + +```zig +pub const Tracker = struct { + pub const max_pending = 512; + pub const flush_interval_s = 60; + pub const prune_every_passes = 1440; // daily at 60 s per pass + mutex: std.Io.Mutex, // guards the fixed table below + // fixed-capacity map, same house pattern as rate_limiter: slot array + index + pub fn init(retention_days: u32) Tracker + pub fn track(self: *Tracker, io: std.Io, addr: address.NetAddress) void // full → drop, count stats.dropped_full + pub fn run(self: *Tracker, io: std.Io, database: *db.Db, monitor: ?*disk_monitor.Monitor) + std.Io.Cancelable!void + pub const Stats = struct { tracked: u64, flushed: u64, dropped_full: u64, pruned: u64, flush_failures: u64 }; +}; +``` + +`run` is the house loop shape (manager.zig:946): startup no-op, then every `flush_interval_s` +on the `.boot` clock: skip the pass when `monitor` says writes are gated (ruling 17); else +drain the pending table under the mutex into a local copy (release before I/O), one +`upsertSeen` per client with `real` wall-clock seconds; every `prune_every_passes`-th pass also +`pruneStale(now - retention_days * 86_400)`. A failed upsert warns once per pass and counts +`flush_failures`; entries stay dropped (next query re-tracks). The database handle is the +loop's own dedicated config.db connection (ruling 21) — document on `run` like retention does. +IP text via `address.NetAddress.format` into a stack buffer. + +Tests: track dedupes per flush (same client twice → one row, `last_seen` = latest), table +full drops and counts, flush inserts hand_edited=0 rows and touches existing hand-edited rows' +last_seen only, prune removes only stale hand_edited=0 rows, gated pass writes nothing +(in-memory db + a Monitor fixture — Monitor's classify/state accessors allow constructing a +gated state; if not constructible without I/O, test the gate branch through a bool seam +`writes_allowed` parameter on a private `flushOnce` and have `run` pass the monitor read). + +Acceptance: fmt/ast clean; temp-root (with `-lc -lsqlite3`) green; migration untouched. + +### S4 As built + +`init(retention_days: u16)` (matches `model.Logging.retention_days`). The pending table is a +fixed 512-entry array with a linear scan, not a hash map — the frozen `init` takes no +allocator, and rate_limiter's actual pattern is a pre-reserved hash map, so "same house +pattern" did not apply; a linear scan over ≤512 entries of 17-byte keys is fine at household +scale. Added: `trackAt(io, addr, now_s)` (timestamp seam; `track` = `trackAt` with real +wall-clock), `flushOnce(io, database, writes_allowed: bool)` public (the gate seam `run` feeds +from `monitor.writesAllowed()`; S7 case 10 forces passes with it), `snapshotStats(io)` and +`pendingClients(io)` — `Stats` stays plain u64 and every field plus `passes` is written under +`mutex`, so readers go through the accessors. `last_seen` is stamped at track time (it means +"last query", and it is what makes latest-wins dedupe observable); the flush writes each +entry's own stamp. A gated pass does not increment `passes` (a skipped pass must not advance +the prune schedule). `track` uses `lockUncancelable` — `handle` has no error union, so `track` +is not a cancelation point. `upsertSeen` resolves `group_id` via +`(SELECT id FROM groups WHERE name = 'default')` — the column is NOT NULL REFERENCES groups +(config_schema.zig:26), which ruling 14's sketch omitted; a database with no `default` group +fails the whole statement with `error.Constraint` and writes nothing (validate.zig:461 already +rejects such configs, so this is a warn-and-continue path in the flush loop, never live). The +upsert's DO UPDATE touches `last_seen` only — `name`, `group_id`, `hand_edited`, `first_seen` +survive on every existing row, hand-edited or not, one test per column. `pruneStale` uses +strict `<` (matches `queries_repo.pruneOlderThan`) and clamps `sqlite3_changes` to u32. +16 tests (9 tracker, 7 repo). + +--- + +## Session S6: composition root — app.zig, shutdown.zig, cli wiring + +Owns: `src/app.zig` (new), `src/server/shutdown.zig` (new), `src/cli.zig` (runRun body, +DataDir), `src/main.zig` (only if a signature forces it — expected unchanged). + +### S6.1 shutdown.zig + +```zig +var event: std.Io.Event = .unset; +var event_io: ?std.Io = null; // set once by install before handlers exist +pub fn install(io: std.Io) void // sigaction INT+TERM → handler calls event.set(event_io.?) +pub fn wait(io: std.Io) std.Io.Cancelable!void // event.wait +pub fn trigger(io: std.Io) void // tests and future admin use: event.set +``` + +Handler fn: `callconv(.c)`, calls set only (async-signal-safe per ruling 2). `flags = 0`, empty +mask (matches Threaded's own handlers, Threaded.zig:1653). Do not restore handlers on exit — +the process is leaving anyway; `install` is once-per-process (assert). + +### S6.2 app.zig — `pub fn run(runner: cli.Runner, paths: cli.Paths) u8` + +Startup order (each failure prints to `runner.err` and returns `cli.exit_runtime`): + +1. `DataDir.open` (S6 changes its `openDir` flags to `.{ .iterate = true }` — the monitor + needs it; nothing else cares). `openConfigDb`. `migrations.migrate`. + `bootstrap.bootstrap(...)` (first run seeds; diags printed like runCheck does). + `export.readConfig` into an arena. +2. `logging.install(io, cfg.logging, ...)` — the sink goes live (main.zig's std_options + already routes). +3. Build state, in dependency order, all heap-allocated where pinning is required (Logger, + Manager, Pool entries and their DoH/DoT client structs must not move — same pattern as + `cli.probeUpstreams` but with owned arrays that outlive the pool): + records + zones (`local_repo` listers → `build`), fetcher + manager (`Manager.init`), + pool (upstreams_repo → clients array → `Pool.init`), cache (`DnsCache.init`), limiter + (`RateLimiter.init` from `cfg.dns`), pause, tracker, logger (`Logger.init` with a + heap `queue_buf` of `cfg.logging.query_log_buffer_max` entries), monitor (`Monitor.init` + with the data dir), retention (`Retention.init`), querylog connections: add + `DataDir.openQuerylogDb(self, io) !db.Db` mirroring openConfigDb (same pragmas + 0600); + open TWO (writer, retention) plus ONE extra config.db connection for the tracker + (ruling 21). +4. One synchronous `manager.reload(io)`; failure warns and continues (ruling 4). +5. Handler construction (S2 struct) + `UdpServer.bind` + `TcpServer.listen` on + `cfg.dns.bind_ipv4/bind_ipv6/port` — v4 and v6 both, matching PLAN §7.2 parity; a bind + failure is fatal. +6. `shutdown.install(io)`. Then one `std.Io.Group`: + udp.serve, tcp.serve, logger.runWriter(io, &querylog_writer_db, &monitor), + retention.run(io, &querylog_retention_db), monitor.run(io), + manager.runScheduler(io), tracker.run(io, &tracker_db, &monitor), + and a maintenance loop (private fn in app.zig, house shape, `.boot`, 60 s): cache sweep + + limiter sweep under their mutexes... the mutexes live in the Handler; the maintenance loop + takes `*Handler` and locks the same mutexes. +7. `shutdown.wait(io)` (a Canceled result also proceeds to teardown). Then ruling 22 order: + `logger.shutdown(io)` → `group.cancel(io)` → server deinits → pool/manager/state deinits → + DB closes → DataDir.close → return `cli.exit_ok`. + +Log a one-line startup summary (scope `.nxdns`, info): bind addresses, snapshot generation or +"unfiltered", upstream count. + +### S6.3 cli.zig + +`runRun` body: `return app.run(r, paths);` — keep the doc comment honest. `DataDir.open` flag +change + `openQuerylogDb` as above (both used by app; runCheck untouched). + +### S6.4 Acceptance + +- [ ] fmt/ast clean on all owned files; `zig build` (exe) succeeds — app.zig is reached from + main, so full semantic analysis covers it. +- [ ] `zig build test` green (existing cli tests untouched; app.zig gets a smoke test only if + expressible without sockets — otherwise S7 owns runtime coverage; state that choice). +- [ ] Manual smoke (orchestrator runs it): `nxdns run` against a scratch data dir starts, + answers `dig @127.0.0.1 -p `, and exits 0 on SIGTERM promptly. + +### S6 As built + +Startup order as specified, with these corrections. **Binding**: IPv6 binds first; an +`AddressInUse` from the IPv4 bind that follows a wildcard IPv6 bind is dual-stack coverage +(info line), not a failure — on Linux `net.ipv6.bindv6only=0` makes the v6 wildcard own the v4 +port, and `BindOptions.ip6_only` cannot prevent it because Threaded.zig:12274 sets IPV6_V6ONLY +to 0 when the option is true (inverted vs its doc); a foreign IPv4 holder would have failed +the v6 wildcard bind too, so the inference is sound. `NetAddress.fromIp` unmaps `::ffff:` +clients, so v4 clients keep their real identity. A kernel without IPv6 +(`AddressFamilyUnsupported` and friends) warns and serves IPv4; every other bind failure is +fatal. **Exit codes**: configuration faults (`NoUsableUpstreams`, `BadBindAddress`, +`BadRateLimit` — zero rate fields are rejected before RateLimiter.init asserts) exit +`cli.exit_check` with a "run 'nxdns check'" hint; everything else exits `cli.exit_runtime`. +**Querylog open**: `openQuerylogDb` wraps `querylog_schema.open` (creation/repair lives there; +chmod of the file plus both WAL sidecars happens after, missing sidecars ignored); +`reopenQuerylogDb` opens the extra connections without repeating the O(size) quick_check. +**Two `std.http.Client`s**: blocklist downloads must not queue DoH queries behind a +multi-megabyte stream. Teardown is defer-driven; the invariants held are: group.cancel joins +first, config.db closes after manager.deinit, DataDir closes last. `negative_ttl_max` and +`forward_read_timeout` are wired (the S2 trap). `shutdown.zig` has `reset()` for tests and 2 +in-file tests (orchestrator wired it into tests.zig — app.zig analysis via cli does not +collect tests). app.zig itself has no unit test; S7 case 11 is the runtime proof. Smoke test +passed: boot, migration 0->2, dual-stack answers on 15353/15354, SERVFAIL from the dead +upstream, SIGTERM exit 0 in 0.007 s, DBs 0600, zero err lines. The manager disk gate was wired in a follow-up S6 edit — see ruling 17's as-built text; the +gate test drives `Monitor.state_raw` directly (no I/O) and pins no-monitor/ok/warn pass, +critical gates-and-counts, recovery resumes. After review: the composition root takes one +synchronous `monitor.sample(io)` after `Monitor.init`, before the `Io.Group` spawns anything — +`Monitor` initializes to `.ok` and `run` samples inside its own task, so without it the +writer/tracker/scheduler would consult a gate still saying "writes allowed" during the boot +window on a critically full disk. A failed first sample warns and leaves `.ok` (an unreadable +filesystem is not evidence of a full disk — the monitor's own documented policy). `run`'s own +first sample repeats it (one extra statvfs + scan at boot; `run`'s sample-then-sleep contract +stays untouched). The ordering is not unit-testable (serve binds sockets and opens four +databases); S7's boot case plus the statement order is the evidence. + +--- + +## Session S7: phase 7 integration tests + +Owns: `src/server/phase7_integration_test.zig` (new; gated on `build_options.integration` like +phase6). Fixtures may reuse the S8/M6 helper patterns (in-memory dbs, fake upstream sockets). + +Cases (each end-to-end through a real `Handler` with real UDP sockets on 127.0.0.1 ephemeral +ports; a fake upstream = a UDP socket task answering canned responses): + +1. Blocked domain answers 0.0.0.0 with blocking TTL; log entry captured with reason. +2. Allow-rule-over-blocklist resolves via upstream. +3. Local record answers authoritatively without touching the fake upstream. +4. Forward zone: matching suffix goes to the zone resolver socket (not the pool fake), a + blocklisted name inside the zone still resolves (bypass), response cached (second query + does not hit the zone socket). +5. Cache: miss→hit, TTL aged, transaction ID fresh, `cache_hit` logged on the hit. +6. Uncloak: fake upstream returns CNAME chain to a blocked target → zero-IP answer with + `cname:` reason. +7. Safe-search group: query for a safesearch.zig table domain returns CNAME + the fake + upstream's A for the target; question is the original name. +8. Rate limit: limit=2 window=60 → third query REFUSED. +9. Pause: pauseFor(indefinite) lets a blocked domain resolve; unpause blocks it again. +10. Tracker: after queries + one forced flush (`flushOnce` seam or short interval), the + clients table holds the source IP with hand_edited=0. +11. Full app boot: `app.run` in a task against a temp data dir (bootstrap from a minimal + config.zon on an ephemeral port), one real resolve through it, `shutdown.trigger`, task + returns 0. This is the lifecycle proof. + +Acceptance: `zig build test -Dintegration` green, zero std.log.err emitted (binding logging +policy), cases deterministic (no external network — the pool's upstream is the fake). + +### S7 As built + +Case 11 runs against a `std.testing.tmpDir` rather than a fixed scratch path (a committed +test cannot hold a machine-specific absolute path; the phase6 fixture already resolves +cwd-relative paths through both `std.Io.Dir` and SQLite). Port 15455, IPv4+IPv6 binds, the +dead `https://127.0.0.1:9/dns-query` upstream, one seeded local record; `app.serve` installs +and deinstalls its own log sink, so `installForTest` is unnecessary and the case asserts the +error writer stayed empty; `shutdown.reset()` brackets the case. Case 5 proves TTL ageing +with an entry planted ten seconds in the past under the handler's own key — the +miss-hit-fresh-ID half is a real round trip; no test can advance the clock. Case 4's zone +resolver is a UDP socket task answering with `ResponseBuilder` against the received query; +the second query proves the cache by leaving the resolver's datagram count at 1. The fake +pool upstream builds each answer from the question it receives (safe search and ECS rewrite +the outgoing question, so canned bytes would not bind). A shared `Loop` fixture binds the +listener and client before the serve task starts, so no task holds a pointer into a value +that later moves. 11 tests, all gated on `build_options.integration`; the integration suite +emits zero err-level lines. + +--- + +## Module layout (new/changed) + +| file | change | +|---|---| +| src/dns/edns.zig | +stripEcs (S1; ecsPayload added then removed in review) | +| src/server/handler.zig | pipeline rewrite (S2) | +| src/server/pause.zig | new (S2) | +| src/server/udp_server.zig, tcp_server.zig | address + scratch plumbing (S3) | +| src/server/clients.zig | new tracker (S4) | +| src/storage/repositories/clients_repo.zig | +upsertSeen, +pruneStale (S4) | +| src/app.zig | new composition root (S6) | +| src/server/shutdown.zig | new (S6) | +| src/cli.zig | runRun body, DataDir.iterate, openQuerylogDb (S6) | +| src/server/phase7_integration_test.zig | new (S7) | + +## File ownership + +S1: dns/edns.zig. S2: server/handler.zig, server/pause.zig. S3: server/udp_server.zig, +server/tcp_server.zig. S4: server/clients.zig, storage/repositories/clients_repo.zig. +S6: app.zig, server/shutdown.zig, cli.zig, main.zig(if forced). S7: its test file. +Orchestrator: src/tests.zig, specs. No parallel sessions share a file (S1∥S4; the rest are +sequential). + +## Acceptance (milestone complete) + +- [ ] `zig build test` and `zig build test -Dintegration` green; `zig fmt --check` clean; + `zig build cross` produces both static exes. +- [ ] All S7 cases pass; the app boots, serves, and shuts down cleanly (case 11). +- [ ] No std.log.err in any new code path exercised by tests (binding logging policy: + err is reserved for swallowed failures). +- [ ] Spec As-built sections synced per session. + +## Anti-requirements + +- No HTTP/SSE/metrics/auth (Phase 8). No DoH/DoT server (Phase 9). No packaging (Phase 10). +- No regex rules, DHCP, DNSSEC validation, DoQ, clustering (PLAN §2.2). +- No per-group cache, no cache persistence, no query re-resolution in uncloaking. +- No new config fields, no schema migration, no pause persistence. +- No benchmark harness this milestone. +- dns/ stays pure (S1's additions are pure); filter/local/cache modules are not edited. diff --git a/src/app.zig b/src/app.zig new file mode 100644 index 0000000..37f5976 --- /dev/null +++ b/src/app.zig @@ -0,0 +1,642 @@ +//! The composition root: everything `nxdns run` owns, in the order it is built. +//! +//! Nothing else in the program constructs a collaborator. Each module takes its +//! dependencies as parameters and knows nothing about who supplies them, so +//! this file is the only place where the object graph exists — and the only +//! place a lifetime spans the whole process. +//! +//! Two rules shape it: +//! +//! 1. **Nothing that is pointed at may move.** `Pool.Entry.client` erases a +//! pointer into a DoH or DoT client, the `Logger`'s queue holds waiting +//! tasks in intrusive lists, and the `Manager` is addressed through `self`. +//! Every such value therefore lives in `serve`'s frame or in an owned heap +//! allocation, never in a temporary that is copied afterwards. +//! 2. **Every background loop is a task in one `std.Io.Group`.** Shutdown is +//! `group.cancel`, which requests cancellation and joins, so no loop can +//! still be running when the resources it borrows are released. +//! +//! Failure policy: a startup failure prints one line to the runner's error +//! writer and exits. A configuration the operator can fix exits 2, so that +//! `nxdns check` is the obvious next step; anything else exits 1. Once serving +//! starts, nothing is fatal — the pipeline answers queries with a snapshot or +//! without one (ruling 4), and every loop logs its own failures and keeps +//! going. + +const std = @import("std"); +const Allocator = std.mem.Allocator; +const Certificate = std.crypto.Certificate; +const Writer = std.Io.Writer; +const net = std.Io.net; +const tls = std.crypto.tls; + +const bootstrap = @import("config/bootstrap.zig"); +const cli = @import("cli.zig"); +const clients = @import("server/clients.zig"); +const config_export = @import("config/export.zig"); +const disk_monitor = @import("storage/disk_monitor.zig"); +const dns_cache = @import("cache/dns_cache.zig"); +const doh_client = @import("upstream/doh_client.zig"); +const dot_client = @import("upstream/dot_client.zig"); +const fetcher = @import("filter/fetcher.zig"); +const forward_zones = @import("local/forward_zones.zig"); +const handler = @import("server/handler.zig"); +const local_records = @import("local/records.zig"); +const logger_mod = @import("storage/logger.zig"); +const logging = @import("platform/logging.zig"); +const manager_mod = @import("filter/manager.zig"); +const migrations = @import("storage/migrations.zig"); +const model = @import("config/model.zig"); +const pause = @import("server/pause.zig"); +const pool_mod = @import("upstream/pool.zig"); +const rate_limiter = @import("server/rate_limiter.zig"); +const retention_mod = @import("storage/retention.zig"); +const shutdown = @import("server/shutdown.zig"); +const tcp_server = @import("server/tcp_server.zig"); +const transport = @import("upstream/transport.zig"); +const udp_server = @import("server/udp_server.zig"); +const validate = @import("config/validate.zig"); +const version = @import("version.zig"); + +const log = std.log.scoped(.nxdns); + +/// How long the cache and rate-limiter sweeps wait between passes. Both tables +/// evict lazily on lookup as well; this is what keeps memory bounded for a +/// client that asked once and never came back. +const maintenance_interval_s = 60; + +/// Bounds one blocklist download (`Manager.total_budget`). Generous, because a +/// megabyte-scale list over a household uplink is normal, and a download that +/// takes longer than this is not going to finish at all. +const download_budget_s = 300; + +/// Per DoH upstream. `min_request_buf` is 512; the extra room costs nothing and +/// keeps a maximum-length name with a large OPT record comfortable. +const doh_request_buf_len = 1024; +const doh_transfer_buf_len = 4096; + +/// A configuration fault the operator can fix, as opposed to a runtime one. +/// These are the only failures this file raises itself; everything else comes +/// out of a collaborator. +const ConfigError = error{ + NoUsableUpstreams, + BadBindAddress, + BadRateLimit, +}; + +pub fn run(runner: cli.Runner, paths: cli.Paths) u8 { + const code = serve(runner, paths) catch |err| code: { + runner.err.print("nxdns run failed: {s}\n", .{@errorName(err)}) catch {}; + if (isConfigFault(err)) { + runner.err.writeAll("run `nxdns check` to see the configuration in full\n") catch {}; + break :code cli.exit_check; + } + break :code cli.exit_runtime; + }; + + // Output the operator never received is not output, so a failed flush + // outranks a successful run — the same rule `cli.finish` applies. + runner.out.flush() catch return cli.exit_runtime; + runner.err.flush() catch return cli.exit_runtime; + return code; +} + +fn isConfigFault(err: anyerror) bool { + return switch (err) { + error.NoUsableUpstreams, error.BadBindAddress, error.BadRateLimit => true, + else => false, + }; +} + +fn serve(r: cli.Runner, paths: cli.Paths) !u8 { + const io = r.io; + const gpa = r.gpa; + + // ----------------------------------------------------------------------- + // storage and configuration + // ----------------------------------------------------------------------- + + var data = try cli.DataDir.open(io, gpa, paths.data_dir, true); + defer data.close(io, gpa); + + var config_db = try data.openConfigDb(io); + defer config_db.close(); + _ = try migrations.migrate(&config_db); + + { + // First run only: the file seeds an empty database and is ignored + // forever after. Its diagnostics are the operator's one chance to see + // why a config file was rejected, so they are printed like `check` + // prints them. + var diags: validate.Diagnostics = .init(gpa); + defer diags.deinit(); + _ = bootstrap.bootstrap(io, gpa, &config_db, std.Io.Dir.cwd(), paths.config, &diags) catch |err| { + diags.writeAll(r.err) catch {}; + return err; + }; + } + + // Every string in `cfg` points into this arena, and the pool's endpoints, + // the handler's records and the monitor's paths all keep such strings. It + // therefore outlives all of them, which is why it is declared here and not + // inside a narrower scope. + var arena_state: std.heap.ArenaAllocator = .init(gpa); + defer arena_state.deinit(); + const arena = arena_state.allocator(); + + const cfg = try config_export.readConfig(&config_db, arena); + + // From here on `std.log` goes wherever the operator asked. Before this call + // the sink passes through to stderr, which is where a failure above needs + // to appear anyway. + logging.install(io, cfg.logging); + defer logging.deinstall(); + + if (cfg.dns.rate_limit == 0 or cfg.dns.rate_window_seconds == 0) return error.BadRateLimit; + + // ----------------------------------------------------------------------- + // local answers + // ----------------------------------------------------------------------- + + var records = try local_records.Records.build(gpa, cfg.local_records); + defer records.deinit(gpa); + + var zones = try forward_zones.Zones.build(gpa, cfg.forward_zones); + defer zones.deinit(gpa); + + // ----------------------------------------------------------------------- + // blocklists + // ----------------------------------------------------------------------- + + // Two HTTP clients on purpose. A blocklist download streams tens of + // megabytes and holds its connection for the whole of it; DoH queries must + // not queue behind that, and the two have nothing to share but a type. + var fetch_http: std.http.Client = .{ .allocator = gpa, .io = io }; + defer fetch_http.deinit(); + var dns_http: std.http.Client = .{ .allocator = gpa, .io = io }; + defer dns_http.deinit(); + + const fetch_transfer = try gpa.alloc(u8, fetcher.min_transfer_buf); + defer gpa.free(fetch_transfer); + const fetch_redirect = try gpa.alloc(u8, fetcher.redirect_buffer_len); + defer gpa.free(fetch_redirect); + var fetch: fetcher.Fetcher = .{ + .http = &fetch_http, + .transfer_buf = fetch_transfer, + .redirect_buf = fetch_redirect, + }; + + var manager: manager_mod.Manager = try .init( + gpa, + &config_db, + .{ .dir = data.dir }, + &fetch, + cfg.blocklist_update, + .{ .raw = .fromSeconds(download_budget_s), .clock = .awake }, + ); + defer manager.deinit(io); + + // ----------------------------------------------------------------------- + // upstream pool + // ----------------------------------------------------------------------- + + // Shared by every DoT client: the trust store does not vary per upstream + // and the scan is expensive. Both must outlive the clients that hold them. + var bundle: Certificate.Bundle = .empty; + defer bundle.deinit(gpa); + var bundle_lock: std.Io.RwLock = .init; + + var upstreams = try Upstreams.build(gpa, cfg.upstreams, &dns_http, &bundle, &bundle_lock); + defer upstreams.deinit(gpa); + + var pool: pool_mod.Pool = .init( + upstreams.active(), + .{}, + .{ .raw = model.totalTimeout(cfg.upstream), .clock = .awake }, + @truncate(@as(u96, @bitCast(std.Io.Clock.real.now(io).nanoseconds))), + ); + + // ----------------------------------------------------------------------- + // per-query state + // ----------------------------------------------------------------------- + + var cache: dns_cache.DnsCache = try .init(gpa, cfg.cache); + defer cache.deinit(); + + var limiter: rate_limiter.RateLimiter = try .init(gpa, .{ + .limit = cfg.dns.rate_limit, + .window_seconds = cfg.dns.rate_window_seconds, + }); + defer limiter.deinit(); + + var paused: pause.Pause = .{}; + var tracker: clients.Tracker = .init(cfg.logging.retention_days); + + // The queue holds waiting tasks in intrusive lists, so neither the buffer + // nor the `Logger` may move once a task has touched either. + const queue_buf = try gpa.alloc(logger_mod.Entry, cfg.logging.query_log_buffer_max); + defer gpa.free(queue_buf); + var query_logger: logger_mod.Logger = .init(cfg.logging, queue_buf); + + // ----------------------------------------------------------------------- + // disk, retention and the remaining connections (ruling 21) + // ----------------------------------------------------------------------- + + const data_path = try arena.dupeZ(u8, paths.data_dir); + const log_dir_path: ?[:0]const u8 = if (cfg.logging.output == .file) + try arena.dupeZ(u8, std.fs.path.dirname(cfg.logging.file_path) orelse ".") + else + null; + var monitor: disk_monitor.Monitor = .init(cfg.disk, data.dir, data_path, log_dir_path); + + // Ruling 17. The scheduler consults it before every scheduled pass; the + // startup `reload` below is an operator action and stays ungated. + manager.monitor = &monitor; + + // One synchronous sample before anything can consult the gate. `Monitor` + // initializes to `.ok`, and `Monitor.run` takes its first sample inside the + // task — so without this call the log writer, the tracker and the blocklist + // scheduler would all see "writes allowed" during the boot window and could + // write to a critically full disk before the gate ever reflected it. + // `run`'s own first sample repeats this one, which costs one extra statvfs + // and directory scan at startup and keeps `run`'s "sample first, then + // sleep" contract intact. + // + // `sample` returns nothing: every failure warns and counts inside the + // monitor, and leaves the previous reading in place. A first sample that + // fails therefore leaves the state at `.ok` — an unreadable filesystem is + // not evidence that the disk is full, which is the monitor's documented + // policy and the right one here too. + monitor.sample(io); + + var retention: retention_mod.Retention = .init(cfg.logging); + + var querylog_writer_db = try data.openQuerylogDb(io); + defer querylog_writer_db.close(); + var querylog_retention_db = try data.reopenQuerylogDb(io); + defer querylog_retention_db.close(); + var tracker_db = try data.openConfigDb(io); + defer tracker_db.close(); + + // ----------------------------------------------------------------------- + // first snapshot + // ----------------------------------------------------------------------- + + // Ruling 4: serving starts either way. A household loses more from a DNS + // server that refuses to start than from one that answers unfiltered until + // the scheduler's first pass succeeds. + manager.reload(io) catch |err| { + log.warn( + "loading the blocklist snapshot failed ({s}); serving unfiltered until the next refresh", + .{@errorName(err)}, + ); + }; + + // ----------------------------------------------------------------------- + // listeners + // ----------------------------------------------------------------------- + + var h: handler.Handler = .{ + .upstream = pool.client(), + .blocking = .{ .mode = cfg.blocking.response, .ttl = cfg.blocking.ttl }, + .ecs_mode = cfg.edns.ecs_mode, + .forward_read_timeout = .{ .raw = model.readTimeout(cfg.upstream), .clock = .awake }, + .manager = &manager, + .records = &records, + .zones = &zones, + .cache = &cache, + .negative_ttl_max = cfg.cache.negative_ttl_max, + .limiter = &limiter, + .logger = &query_logger, + .pause = &paused, + .tracker = &tracker, + }; + + const v6_bind = parseBind(r, cfg.dns.bind_ipv6, cfg.dns.port, "dns.bind_ipv6") catch |err| return err; + const v4_bind = parseBind(r, cfg.dns.bind_ipv4, cfg.dns.port, "dns.bind_ipv4") catch |err| return err; + + // IPv6 first, and the order is load-bearing — see `Listeners`. + var udp6: ?udp_server.UdpServer = udp_server.UdpServer.bind(gpa, io, v6_bind, &h, .{}) catch |err| bound: { + if (!ipv6Unavailable(err)) return reportBind(r, "udp", v6_bind, err); + log.warn("this system has no IPv6; serving IPv4 only", .{}); + break :bound null; + }; + defer if (udp6) |*s| s.deinit(gpa, io); + + const v6_is_wildcard = udp6 != null and isWildcard(v6_bind); + + var udp4: ?udp_server.UdpServer = udp_server.UdpServer.bind(gpa, io, v4_bind, &h, .{}) catch |err| bound: { + if (err != error.AddressInUse or !v6_is_wildcard) return reportBind(r, "udp", v4_bind, err); + log.info("the IPv6 UDP listener is dual-stack and already serves IPv4", .{}); + break :bound null; + }; + defer if (udp4) |*s| s.deinit(gpa, io); + + var tcp6: ?tcp_server.TcpServer = tcp_server.TcpServer.listen(gpa, io, v6_bind, &h, .{}) catch |err| bound: { + if (!ipv6Unavailable(err)) return reportBind(r, "tcp", v6_bind, err); + break :bound null; + }; + defer if (tcp6) |*s| s.deinit(gpa, io); + + var tcp4: ?tcp_server.TcpServer = tcp_server.TcpServer.listen(gpa, io, v4_bind, &h, .{}) catch |err| bound: { + if (err != error.AddressInUse or !(tcp6 != null and isWildcard(v6_bind))) { + return reportBind(r, "tcp", v4_bind, err); + } + log.info("the IPv6 TCP listener is dual-stack and already serves IPv4", .{}); + break :bound null; + }; + defer if (tcp4) |*s| s.deinit(gpa, io); + + // ----------------------------------------------------------------------- + // run + // ----------------------------------------------------------------------- + + shutdown.install(io); + + // Declared after everything it borrows, so its `cancel` — which both + // requests cancellation and joins — is the first thing that runs on the way + // out (ruling 22). Nothing below this line may be released while a task + // could still touch it. + var group: std.Io.Group = .init; + defer group.cancel(io); + + if (udp6) |*s| try group.concurrent(io, udp_server.UdpServer.serve, .{ s, io }); + if (udp4) |*s| try group.concurrent(io, udp_server.UdpServer.serve, .{ s, io }); + if (tcp6) |*s| try group.concurrent(io, tcp_server.TcpServer.serve, .{ s, io }); + if (tcp4) |*s| try group.concurrent(io, tcp_server.TcpServer.serve, .{ s, io }); + + const gate: ?*disk_monitor.Monitor = &monitor; + try group.concurrent(io, logger_mod.Logger.runWriter, .{ &query_logger, io, &querylog_writer_db, gate }); + try group.concurrent(io, retention_mod.Retention.run, .{ &retention, io, &querylog_retention_db }); + try group.concurrent(io, disk_monitor.Monitor.run, .{ &monitor, io }); + try group.concurrent(io, manager_mod.Manager.runScheduler, .{ &manager, io }); + try group.concurrent(io, clients.Tracker.run, .{ &tracker, io, &tracker_db, gate }); + try group.concurrent(io, runMaintenance, .{ &h, io }); + + logStartup(io, &manager, upstreams.active().len, .{ + .udp6 = if (udp6) |*s| s.boundAddress() else null, + .udp4 = if (udp4) |*s| s.boundAddress() else null, + .tcp6 = if (tcp6) |*s| s.boundAddress() else null, + .tcp4 = if (tcp4) |*s| s.boundAddress() else null, + }); + + // A canceled wait is a shutdown request too: whoever canceled this task + // wants the process to stop, and the teardown below is how it stops. + shutdown.wait(io) catch {}; + log.info("shutting down", .{}); + + // Before the group is canceled, so the writer sees a closed queue and + // drains what it holds rather than losing it to cancellation (ruling 22). + query_logger.shutdown(io); + + return cli.exit_ok; +} + +// --------------------------------------------------------------------------- +// background maintenance +// --------------------------------------------------------------------------- + +/// Sweeps the cache and the rate-limiter table. Both are guarded by mutexes the +/// handler owns, because the handler is what contends for them; the sweeps live +/// here because walking a whole table is not work a query should pay for. +/// +/// The locks are taken cancelably: unlike `handle`, this loop has an error +/// union to carry `error.Canceled` out of, and a shutdown that arrives while +/// the query path holds a lock should not wait for it. +fn runMaintenance(h: *handler.Handler, io: std.Io) std.Io.Cancelable!void { + const interval: std.Io.Clock.Duration = .{ + .raw = .fromSeconds(maintenance_interval_s), + .clock = .boot, + }; + while (true) { + try interval.sleep(io); + + if (h.cache) |cache| { + const now_s = std.Io.Clock.real.now(io).toSeconds(); + try h.cache_mutex.lock(io); + _ = cache.sweep(now_s); + h.cache_mutex.unlock(io); + } + + if (h.limiter) |limiter| { + const now = std.Io.Clock.awake.now(io); + try h.limiter_mutex.lock(io); + _ = limiter.sweep(now); + h.limiter_mutex.unlock(io); + } + } +} + +// --------------------------------------------------------------------------- +// upstreams +// --------------------------------------------------------------------------- + +/// The pool's entries and everything they point into. +/// +/// `Pool.Entry.client` is a type-erased pointer into `doh` or `dot`, and each +/// client borrows a slice of `doh_buf`/`dot_buf`, so all five allocations live +/// exactly as long as the pool does. One entry is used by one task at a time +/// (`Entry.busy`), which is why the buffers are per client and not shared the +/// way `cli.probeUpstreams` shares them. +const Upstreams = struct { + entries: []pool_mod.Entry, + used: usize, + doh: []doh_client.DohClient, + dot: []dot_client.DotClient, + doh_buf: []u8, + dot_buf: []u8, + + /// A disabled upstream is left out entirely; a malformed one warns and is + /// skipped, because one bad row in a table of four must not take DNS down. + /// No usable row at all is a configuration fault. + fn build( + gpa: Allocator, + servers: []const model.UpstreamServer, + http: *std.http.Client, + bundle: *Certificate.Bundle, + bundle_lock: *std.Io.RwLock, + ) (Allocator.Error || ConfigError)!Upstreams { + var enabled: usize = 0; + for (servers) |server| { + if (server.enabled) enabled += 1; + } + if (enabled == 0) return error.NoUsableUpstreams; + + const chunk = tls.Client.min_buffer_len; + + var self: Upstreams = .{ + .entries = try gpa.alloc(pool_mod.Entry, enabled), + .used = 0, + .doh = &.{}, + .dot = &.{}, + .doh_buf = &.{}, + .dot_buf = &.{}, + }; + errdefer self.deinit(gpa); + + self.doh = try gpa.alloc(doh_client.DohClient, enabled); + self.dot = try gpa.alloc(dot_client.DotClient, enabled); + self.doh_buf = try gpa.alloc(u8, enabled * (doh_request_buf_len + doh_transfer_buf_len)); + self.dot_buf = try gpa.alloc(u8, enabled * 4 * chunk); + + var doh_count: usize = 0; + var dot_count: usize = 0; + + for (servers) |server| { + if (!server.enabled) continue; + + const endpoint = transport.Endpoint.parse(server.url) catch { + log.warn("upstream '{s}' is not an https:// or tls:// endpoint; skipped", .{server.url}); + continue; + }; + + const client: transport.Client = switch (endpoint.scheme) { + .doh => doh: { + const base = doh_count * (doh_request_buf_len + doh_transfer_buf_len); + const slot = &self.doh[doh_count]; + slot.* = doh_client.DohClient.init( + http, + endpoint, + self.doh_buf[base..][0..doh_request_buf_len], + self.doh_buf[base + doh_request_buf_len ..][0..doh_transfer_buf_len], + ) catch { + log.warn("upstream '{s}' is not a usable DoH url; skipped", .{server.url}); + continue; + }; + doh_count += 1; + break :doh slot.client(); + }, + .dot => dot: { + const base = dot_count * 4 * chunk; + const slot = &self.dot[dot_count]; + slot.* = dot_client.DotClient.init(endpoint, server.tls_name, gpa, bundle, bundle_lock, .{ + .tls_read = self.dot_buf[base..][0..chunk], + .tls_write = self.dot_buf[base + chunk ..][0..chunk], + .stream_read = self.dot_buf[base + 2 * chunk ..][0..chunk], + .stream_write = self.dot_buf[base + 3 * chunk ..][0..chunk], + }); + dot_count += 1; + break :dot slot.client(); + }, + }; + + self.entries[self.used] = .{ + .endpoint = endpoint, + .client = client, + .priority = server.priority, + .enabled = true, + .health = .init, + }; + self.used += 1; + } + + if (self.used == 0) return error.NoUsableUpstreams; + return self; + } + + /// The prefix `Pool.init` is given. The rest of `entries` is allocated but + /// never filled, which is what keeps `deinit` able to free the whole block. + fn active(self: *Upstreams) []pool_mod.Entry { + return self.entries[0..self.used]; + } + + fn deinit(self: *Upstreams, gpa: Allocator) void { + gpa.free(self.dot_buf); + gpa.free(self.doh_buf); + gpa.free(self.dot); + gpa.free(self.doh); + gpa.free(self.entries); + self.* = undefined; + } +}; + +// --------------------------------------------------------------------------- +// listeners +// --------------------------------------------------------------------------- + +/// The addresses actually bound, for the startup line. A null is a family this +/// process does not listen on separately — either because the system has no +/// IPv6, or because the IPv6 socket is dual-stack and already serves IPv4. +const Listeners = struct { + udp6: ?net.IpAddress, + udp4: ?net.IpAddress, + tcp6: ?net.IpAddress, + tcp4: ?net.IpAddress, +}; + +fn parseBind(r: cli.Runner, text: []const u8, port: u16, field: []const u8) !net.IpAddress { + return net.IpAddress.parse(text, port) catch { + r.err.print("{s}: '{s}' is not an IP address\n", .{ field, text }) catch {}; + return error.BadBindAddress; + }; +} + +fn reportBind(r: cli.Runner, which: []const u8, addr: net.IpAddress, err: anyerror) anyerror { + r.err.print("cannot bind {s} {f}: {s}\n", .{ which, addr, @errorName(err) }) catch {}; + return err; +} + +/// The kernel has no IPv6 at all. Refusing to start would be the wrong answer +/// for the default configuration, which asks for both families. +fn ipv6Unavailable(err: anyerror) bool { + return switch (err) { + error.AddressFamilyUnsupported, + error.ProtocolUnsupportedBySystem, + error.ProtocolUnsupportedByAddressFamily, + => true, + else => false, + }; +} + +fn isWildcard(addr: net.IpAddress) bool { + switch (addr) { + .ip4 => |a| { + for (a.bytes) |b| if (b != 0) return false; + }, + .ip6 => |a| { + for (a.bytes) |b| if (b != 0) return false; + }, + } + return true; +} + +// --------------------------------------------------------------------------- +// startup line +// --------------------------------------------------------------------------- + +/// One line, at info, naming what an operator needs to see in `journalctl` +/// right after a restart: where it listens, how many upstreams it has, and +/// whether filtering is live. +fn logStartup(io: std.Io, manager: *manager_mod.Manager, upstream_count: usize, bound: Listeners) void { + var buf: [256]u8 = undefined; + var w: Writer = .fixed(&buf); + appendBind(&w, "udp", bound.udp6); + appendBind(&w, "udp", bound.udp4); + appendBind(&w, "tcp", bound.tcp6); + appendBind(&w, "tcp", bound.tcp4); + + // The generation is read under the snapshot handle's shared lock, which is + // the same lock the reload took to publish it. + if (manager.acquire(io)) |snapshot| { + defer snapshot.release(io); + log.info("nxdns {s} serving on{s}; {d} upstream(s); blocklist generation {d}", .{ + version.string, + w.buffered(), + upstream_count, + manager.generation, + }); + } else { + log.info("nxdns {s} serving on{s}; {d} upstream(s); unfiltered (no blocklist snapshot)", .{ + version.string, + w.buffered(), + upstream_count, + }); + } +} + +/// Silent on overflow: a truncated startup line is not worth a failure path, +/// and 256 bytes hold four addresses. +fn appendBind(w: *Writer, which: []const u8, addr: ?net.IpAddress) void { + const value = addr orelse return; + w.print(" {s} {f}", .{ which, value }) catch {}; +} diff --git a/src/cli.zig b/src/cli.zig index c186e6b..6db7cef 100644 --- a/src/cli.zig +++ b/src/cli.zig @@ -18,12 +18,14 @@ const Writer = std.Io.Writer; const Certificate = std.crypto.Certificate; const tls = std.crypto.tls; +const app = @import("app.zig"); const config_export = @import("config/export.zig"); const import = @import("config/import.zig"); const model = @import("config/model.zig"); const validate = @import("config/validate.zig"); const db = @import("storage/db.zig"); const migrations = @import("storage/migrations.zig"); +const querylog_schema = @import("storage/querylog_schema.zig"); const doh_client = @import("upstream/doh_client.zig"); const dot_client = @import("upstream/dot_client.zig"); const pool = @import("upstream/pool.zig"); @@ -204,7 +206,9 @@ pub const DataDir = struct { _ = try std.Io.Dir.cwd().createDirPathStatus(io, data_dir, .fromMode(0o700)); } - var dir = try std.Io.Dir.cwd().openDir(io, data_dir, .{}); + // `.iterate`: the disk monitor sizes the databases by scanning this + // directory, and an fd opened without it cannot be read as a directory. + var dir = try std.Io.Dir.cwd().openDir(io, data_dir, .{ .iterate = true }); errdefer dir.close(io); // SQLite opens by path, not by directory handle, so both paths are @@ -243,6 +247,54 @@ pub const DataDir = struct { try db.applyPragmas(&database, .{}); return database; } + + /// The first connection to `querylog.db`: `querylog_schema.open` creates the + /// file when it is missing and recreates it when it is unusable, so this is + /// the call that establishes the schema. `reopenQuerylogDb` is for the + /// connections that follow. + /// + /// The 0600 chmod cannot come first the way `openConfigDb` does it — the + /// file may not exist yet, and a recreate replaces it — so the main file and + /// both WAL sidecars are locked down afterwards instead. A query log holds + /// every domain every client asked for, which is as sensitive as anything in + /// `config.db`. + /// + /// `querylog_schema.open` resolves the path through SQLite's VFS as well as + /// through the directory handle, so it is given `cwd` and the joined path + /// rather than `self.dir` and a name (see its doc comment). + pub fn openQuerylogDb(self: *const DataDir, io: std.Io) !db.Db { + const opened = try querylog_schema.open(io, std.Io.Dir.cwd(), self.querylog_db_path); + var database = opened.database; + errdefer database.close(); + try self.restrictQuerylogPermissions(io); + return database; + } + + /// An additional connection to a `querylog.db` that `openQuerylogDb` has + /// already established. Phase 7 needs two — the log writer and the retention + /// pass each own one (`retention.zig`'s contract). + pub fn reopenQuerylogDb(self: *const DataDir, io: std.Io) !db.Db { + _ = io; + var database = try db.Db.open(self.querylog_db_path, .{ .mode = .read_write_existing }); + errdefer database.close(); + try db.applyPragmas(&database, .{}); + return database; + } + + /// A sidecar that does not exist yet is not a failure: `-wal` and `-shm` + /// appear when SQLite first writes, and the next call catches them. + fn restrictQuerylogPermissions(self: *const DataDir, io: std.Io) !void { + for ([_][]const u8{ + querylog_db_name, + querylog_db_name ++ "-wal", + querylog_db_name ++ "-shm", + }) |entry| { + self.dir.setFilePermissions(io, entry, .fromMode(0o600), .{}) catch |e| switch (e) { + error.FileNotFound => {}, + else => |other| return other, + }; + } + } }; // --------------------------------------------------------------------------- @@ -320,12 +372,11 @@ pub fn runVersion(r: Runner) u8 { return finish(r, exit_ok); } -/// Unchanged from milestone 1. Wiring the configuration into the servers is -/// Phase 7; there is deliberately no half-built serving path here. +/// Serves DNS until SIGINT or SIGTERM. The whole of it lives in `app.zig`, +/// which is where the composition root belongs; this stays the entry point so +/// that `main` dispatches every command the same way. pub fn runRun(r: Runner, paths: Paths) u8 { - _ = paths; - r.out.writeAll("not implemented\n") catch return finish(r, exit_runtime); - return finish(r, exit_check); + return app.run(r, paths); } pub fn runExport(r: Runner, args: ExportArgs) u8 { @@ -879,13 +930,8 @@ test "runVersion prints the milestone-1 version lines and exits 0" { try testing.expectEqual(@as(usize, 2), countLines(captured.out.written())); } -test "runRun still reports that serving is not implemented" { - var captured: Captured = .init(testing.allocator); - defer captured.deinit(); - - try testing.expectEqual(exit_check, runRun(captured.runner(), .{})); - try testing.expectEqualStrings("not implemented\n", captured.out.written()); -} +// `runRun` now binds sockets and serves until a signal arrives, so it has no +// unit test: booting it is `src/server/phase7_integration_test.zig`'s case 11. test "runUsageError names the fault and prints the usage text" { var captured: Captured = .init(testing.allocator); diff --git a/src/dns/edns.zig b/src/dns/edns.zig index ca968ab..8ff95f8 100644 --- a/src/dns/edns.zig +++ b/src/dns/edns.zig @@ -10,6 +10,8 @@ const std = @import("std"); const types = @import("types.zig"); const record = @import("record.zig"); +const question = @import("question.zig"); +const packet_mod = @import("packet.zig"); const Writer = std.Io.Writer; /// The EDNS Client Subnet option code (RFC 7871 §6). @@ -159,6 +161,111 @@ fn ttlFrom(opt: OptRecord) u32 { (@as(u32, @intFromBool(opt.do_bit)) << 15); } +pub const StripResult = union(enum) { + /// The query needs no rewrite and nothing was written to `out`. + unchanged, + /// The rewritten query, a prefix of `out`. + rewritten: []u8, +}; + +/// Byte offset of RDLENGTH inside an OPT record as `encodeOpt` writes it: a +/// one-byte root owner name, TYPE, CLASS and TTL come first. +const opt_rdlength_offset = 1 + 2 + 2 + 4; + +/// Rebuilds `query` into `out` without its ECS option, so that a client's +/// subnet never reaches the upstream resolver (PLAN §6.1). Every byte outside +/// the OPT record is copied verbatim — ID, flags, all four section counts, the +/// question and any record beside the OPT — and the OPT keeps its payload size, +/// its flags and every option other than code 8. +/// +/// `pkt` and `opt` are the caller's already-parsed views of `query`, and `out` +/// must not overlap `query`. +/// +/// One byte pattern does not survive the rewrite: a compression pointer in a +/// record that follows the OPT and targets a byte inside the OPT's option list. +/// Removing an option shifts those bytes, so the pointer then decodes to +/// something else. RFC 1035 §4.1.4 only ever points a name at an earlier *name*, +/// so such a pointer is malformed to begin with, and the upstream resolver +/// answers the rewritten query with FORMERR instead of the original's answer. +pub fn stripEcs( + query: []const u8, + pkt: packet_mod.Packet, + opt: OptRecord, + out: []u8, +) error{ BadOption, Overflow }!StripResult { + std.debug.assert(query.ptr == pkt.bytes.ptr); + std.debug.assert(query.len == pkt.bytes.len); + + // The rebuild reads `query` while it writes `out`, so an overlap would let + // it consume bytes it has already overwritten. The two buffers must be + // disjoint, and the caller finds that out here rather than in the packet it + // sends upstream. + std.debug.assert(@intFromPtr(out.ptr) + out.len <= @intFromPtr(query.ptr) or + @intFromPtr(query.ptr) + query.len <= @intFromPtr(out.ptr)); + + const opt_start = (try optRecordStart(pkt, opt)) orelse return .unchanged; + if ((try findOption(query, opt, ecs_option_code)) == null) return .unchanged; + + var w = Writer.fixed(out); + w.writeAll(query[0..opt_start]) catch return error.Overflow; + encodeOpt(opt, &.{}, &w) catch |err| switch (err) { + error.OptionsTooLong => unreachable, // the list written here is empty + error.WriteFailed => return error.Overflow, + }; + + // RDLENGTH is only known once the list has been filtered, and the surviving + // options are not contiguous in `query`, so `encodeOpt` writes a placeholder + // and the options stream in behind it. + var kept_len: usize = 0; + var it = options(query, opt); + while (try it.next()) |o| { + if (o.code == ecs_option_code) continue; + var option_header: [4]u8 = undefined; + std.mem.writeInt(u16, option_header[0..2], o.code, .big); + std.mem.writeInt(u16, option_header[2..4], @intCast(o.data.len), .big); + w.writeAll(&option_header) catch return error.Overflow; + w.writeAll(o.data) catch return error.Overflow; + kept_len += option_header.len + o.data.len; + } + + w.writeAll(query[opt.options.offset + opt.options.len ..]) catch return error.Overflow; + + const message = w.buffered(); + std.mem.writeInt(u16, message[opt_start + opt_rdlength_offset ..][0..2], @intCast(kept_len), .big); + return .{ .rewritten = message }; +} + +/// Where the record holding `opt` begins, or null when `pkt` has no such +/// record. A record's start offset is not part of a parsed view — only a walk +/// of every section before it reaches that byte — so this repeats the walk +/// `packet.parse` already did. +/// +/// The match is on the whole RDATA span, which keeps a hand-built `OptRecord` +/// from pointing this function at a different record than the one it describes. +/// A section that will not walk lands on `error.BadOption` for the same reason: +/// a `Packet` from `packet.parse` always walks, so only a `Packet` assembled by +/// hand around unvalidated bytes gets here. +fn optRecordStart(pkt: packet_mod.Packet, opt: OptRecord) error{BadOption}!?usize { + var pos: usize = types.header_len; + var q: u16 = 0; + while (q < pkt.header.qdcount) : (q += 1) { + const parsed = question.parse(pkt.bytes, pos) catch return error.BadOption; + pos = parsed.end; + } + + const record_count = @as(u32, pkt.header.ancount) + + @as(u32, pkt.header.nscount) + @as(u32, pkt.header.arcount); + var r: u32 = 0; + while (r < record_count) : (r += 1) { + const parsed = record.parse(pkt.bytes, pos) catch return error.BadOption; + if (parsed.record.rtype == .opt and + parsed.record.rdata.offset == opt.options.offset and + parsed.record.rdata.len == opt.options.len) return pos; + pos = parsed.end; + } + return null; +} + /// The full 12-bit RCODE (RFC 6891 §6.1.3): the OPT record supplies the upper /// eight bits, the header the lower four. Without an OPT record the value is /// just the header's four bits. @@ -406,6 +513,172 @@ test "encodeOpt reports a short buffer" { try testing.expectError(error.WriteFailed, encodeOpt(opt, "", &w)); } +/// A query for example.com A whose OPT record carries an ECS option for +/// 192.0.2.0/24 followed by a two-byte padding option, with DO set and a +/// 4096-byte payload size. +const ecs_query = + "\x12\x34\x01\x00\x00\x01\x00\x00\x00\x00\x00\x01" ++ // header + "\x07example\x03com\x00\x00\x01\x00\x01" ++ // question + "\x00\x00\x29\x10\x00\x00\x00\x80\x00\x00\x11" ++ // OPT, 17 rdata bytes + "\x00\x08\x00\x07\x00\x01\x18\x00\xc0\x00\x02" ++ // ECS + "\x00\x0c\x00\x02\x00\x00"; // padding + +/// `ecs_query` as `stripEcs` must rebuild it. +const ecs_query_stripped = + "\x12\x34\x01\x00\x00\x01\x00\x00\x00\x00\x00\x01" ++ + "\x07example\x03com\x00\x00\x01\x00\x01" ++ + "\x00\x00\x29\x10\x00\x00\x00\x80\x00\x00\x06" ++ + "\x00\x0c\x00\x02\x00\x00"; + +const ParsedQuery = struct { pkt: packet_mod.Packet, opt: OptRecord }; + +fn parseQuery(bytes: []const u8) !ParsedQuery { + const p = try packet_mod.parse(bytes); + return .{ .pkt = p, .opt = try parseOpt(bytes, packet_mod.findOptRecord(p).?) }; +} + +fn expectUnchanged(result: StripResult) !void { + switch (result) { + .unchanged => {}, + .rewritten => return error.TestExpectedUnchanged, + } +} + +test "stripEcs rebuilds the query without the ECS option" { + const parsed = try parseQuery(ecs_query); + var out: [512]u8 = undefined; + const result = try stripEcs(ecs_query, parsed.pkt, parsed.opt, &out); + const rewritten = result.rewritten; + try testing.expectEqualSlices(u8, ecs_query_stripped, rewritten); + + const p = try packet_mod.parse(rewritten); + try testing.expectEqual(@as(u16, 0x1234), p.header.id); + try testing.expectEqual(@as(u16, 1), p.header.qdcount); + try testing.expectEqual(@as(u16, 1), p.header.arcount); + + const opt = try parseOpt(rewritten, packet_mod.findOptRecord(p).?); + try testing.expectEqual(@as(u16, 4096), opt.udp_payload_size); + try testing.expectEqual(true, opt.do_bit); + try testing.expectEqual(@as(?Option, null), try findOption(rewritten, opt, ecs_option_code)); + try testing.expectEqualSlices(u8, "\x00\x00", (try findOption(rewritten, opt, 12)).?.data); +} + +test "stripEcs reports a query with nothing to strip as unchanged" { + const parsed = try parseQuery(ecs_query_stripped); + var out: [512]u8 = undefined; + try expectUnchanged(try stripEcs(ecs_query_stripped, parsed.pkt, parsed.opt, &out)); + + // No OPT record at all: the caller still holds an `OptRecord`, so the + // absence has to be found by the walk rather than assumed. + const no_opt = "\x12\x34\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00" ++ + "\x07example\x03com\x00\x00\x01\x00\x01"; + const p = try packet_mod.parse(no_opt); + try testing.expectEqual(@as(?record.Record, null), packet_mod.findOptRecord(p)); + const absent: OptRecord = .{ + .udp_payload_size = 4096, + .extended_rcode = 0, + .version = 0, + .do_bit = false, + .options = .{ .offset = 0, .len = 0 }, + }; + try expectUnchanged(try stripEcs(no_opt, p, absent, &out)); +} + +test "stripEcs keeps a record that follows the OPT" { + const head = "\x12\x34\x01\x00\x00\x01\x00\x00\x00\x00\x00\x02" ++ + "\x07example\x03com\x00\x00\x01\x00\x01"; + const trailing = "\x00\x00\x01\x00\x01\x00\x00\x00\x0a\x00\x04\x01\x02\x03\x04"; + const query = head ++ + "\x00\x00\x29\x10\x00\x00\x00\x80\x00\x00\x0b" ++ + "\x00\x08\x00\x07\x00\x01\x18\x00\xc0\x00\x02" ++ + trailing; + + const parsed = try parseQuery(query); + var out: [512]u8 = undefined; + const rewritten = (try stripEcs(query, parsed.pkt, parsed.opt, &out)).rewritten; + try testing.expectEqualSlices( + u8, + head ++ "\x00\x00\x29\x10\x00\x00\x00\x80\x00\x00\x00" ++ trailing, + rewritten, + ); + try testing.expectEqual(@as(u16, 2), (try packet_mod.parse(rewritten)).header.arcount); +} + +test "stripEcs removes a repeated ECS option" { + const ecs = "\x00\x08\x00\x07\x00\x01\x18\x00\xc0\x00\x02"; + const query = "\x12\x34\x01\x00\x00\x01\x00\x00\x00\x00\x00\x01" ++ + "\x07example\x03com\x00\x00\x01\x00\x01" ++ + "\x00\x00\x29\x10\x00\x00\x00\x80\x00\x00\x1c" ++ + ecs ++ "\x00\x0c\x00\x02\x00\x00" ++ ecs; + + const parsed = try parseQuery(query); + var out: [512]u8 = undefined; + const rewritten = (try stripEcs(query, parsed.pkt, parsed.opt, &out)).rewritten; + try testing.expectEqualSlices(u8, ecs_query_stripped, rewritten); +} + +test "stripEcs reports an output buffer one byte short" { + const parsed = try parseQuery(ecs_query); + + var exact: [ecs_query_stripped.len]u8 = undefined; + _ = (try stripEcs(ecs_query, parsed.pkt, parsed.opt, &exact)).rewritten; + + var short: [ecs_query_stripped.len - 1]u8 = undefined; + try testing.expectError(error.Overflow, stripEcs(ecs_query, parsed.pkt, parsed.opt, &short)); + + // Every shorter buffer fails the same way, including one that cannot even + // hold the copied header. + var i: usize = 0; + while (i < short.len) : (i += 1) { + try testing.expectError( + error.Overflow, + stripEcs(ecs_query, parsed.pkt, parsed.opt, short[0..i]), + ); + } +} + +test "stripEcs rejects a malformed option list" { + // The record is well-formed; only its option list overruns, so `parseOpt` + // is the one thing that rejects this packet. + const query = "\x12\x34\x01\x00\x00\x01\x00\x00\x00\x00\x00\x01" ++ + "\x07example\x03com\x00\x00\x01\x00\x01" ++ + "\x00\x00\x29\x10\x00\x00\x00\x00\x00\x00\x08" ++ + "\x00\x08\x00\x08\x00\x01\x18\x00"; + + const p = try packet_mod.parse(query); + const rec = packet_mod.findOptRecord(p).?; + try testing.expectError(error.BadOption, parseOpt(query, rec)); + + const hand_built: OptRecord = .{ + .udp_payload_size = rec.class, + .extended_rcode = 0, + .version = 0, + .do_bit = false, + .options = rec.rdata, + }; + + var out: [512]u8 = undefined; + try testing.expectError(error.BadOption, stripEcs(query, p, hand_built, &out)); +} + +test "encodeOpt writes RDLENGTH where stripEcs patches it" { + const opt: OptRecord = .{ + .udp_payload_size = 4096, + .extended_rcode = 0, + .version = 0, + .do_bit = true, + .options = .{ .offset = 0, .len = 0 }, + }; + var buf: [32]u8 = undefined; + var w = Writer.fixed(&buf); + try encodeOpt(opt, "\x00\x0c\x00\x02\x00\x00", &w); + const bytes = w.buffered(); + try testing.expectEqual( + @as(u16, 6), + std.mem.readInt(u16, bytes[opt_rdlength_offset..][0..2], .big), + ); +} + test "extendedRcode composes the twelve bits" { try testing.expectEqual(@as(u12, 3), extendedRcode(.nx_domain, null)); diff --git a/src/filter/manager.zig b/src/filter/manager.zig index da8681a..7983c5a 100644 --- a/src/filter/manager.zig +++ b/src/filter/manager.zig @@ -44,6 +44,7 @@ const clients_repo = @import("../storage/repositories/clients_repo.zig"); const groups_repo = @import("../storage/repositories/groups_repo.zig"); const rules_repo = @import("../storage/repositories/rules_repo.zig"); const sources_repo = @import("../storage/repositories/sources_repo.zig"); +const disk_monitor = @import("../storage/disk_monitor.zig"); const compiler = @import("compiler.zig"); const fetcher = @import("fetcher.zig"); const matcher = @import("matcher.zig"); @@ -221,6 +222,15 @@ pub const Manager = struct { /// Owns the `statuses` table. The entries themselves borrow nothing. status_arena: std.heap.ArenaAllocator, + /// The §11.6 disk gate (ruling 17). Set by the composition root after + /// `init` and before `runScheduler` starts; null disables gating, which is + /// what every test and `nxdns check` want. Only the scheduler consults it — + /// see `refreshGated`. + monitor: ?*disk_monitor.Monitor = null, + /// Scheduled refresh passes skipped by the disk gate. Phase 8's health + /// rollup reads it through `refreshesGated`. + refreshes_gated: std.atomic.Value(u64) = .init(0), + pub const Error = error{ OutOfMemory, Canceled, @@ -958,6 +968,7 @@ pub const Manager = struct { }; while (true) { try interval.sleep(io); + if (self.refreshGated()) continue; self.refreshAll(io) catch |err| switch (err) { error.Canceled => return error.Canceled, else => log.warn("blocklist refresh pass failed: {s}", .{@errorName(err)}), @@ -965,12 +976,42 @@ pub const Manager = struct { } } + /// The §11.6 gate, consulted by scheduled passes only (ruling 17). A + /// download writes tens of megabytes into the blocklist directory and the + /// compile writes as much again, which is exactly the "non-essential write" + /// a critically full disk must not take. + /// + /// `reload` and `refreshAll` are deliberately not gated: both are operator + /// actions (the composition root's startup load, Phase 8's manual refresh), + /// and an operator who asks for a refresh on a full disk has asked for it. + /// + /// Counting happens here, so a caller cannot skip a pass without recording + /// it. One `warn` line per skipped pass — at a 24-hour interval that is one + /// line a day, and the disk monitor already logs the state change itself. + fn refreshGated(self: *Manager) bool { + const monitor = self.monitor orelse return false; + if (monitor.writesAllowed()) return false; + _ = self.refreshes_gated.fetchAdd(1, .monotonic); + log.warn("free space is critical; skipping the scheduled blocklist refresh", .{}); + return true; + } + + /// Scheduled refresh passes the disk gate has skipped. + pub fn refreshesGated(self: *const Manager) u64 { + return self.refreshes_gated.load(.monotonic); + } + fn startupPass(self: *Manager, io: std.Io) Error!void { self.writer_lock.lockUncancelable(io); defer self.writer_lock.unlock(io); + // Ahead of the gate on purpose: loading the compiled files that already + // exist is a read. A full disk must not cost the household its + // filtering as well as its downloads. try self.reloadLocked(io); + if (self.refreshGated()) return; + var rows = try sources_repo.listSourceRows(self.database, self.gpa); defer rows.deinit(self.gpa); defer sources_repo.freeSourceRows(self.gpa, rows.items); @@ -1439,6 +1480,42 @@ test "acquire before any reload returns null and holds no lock" { manager.lock.unlock(io); } +test "the disk gate skips a scheduled refresh only while writes are critical" { + var threaded: std.Io.Threaded = .init(testing.allocator, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var database = try openMigrated(); + defer database.close(); + + var f: fetcher.Fetcher = undefined; + var manager = try testManager(&database, &f); + defer manager.deinit(io); + + // No monitor: every pass runs, which is what the tests and `check` rely on. + try testing.expect(!manager.refreshGated()); + try testing.expectEqual(@as(u64, 0), manager.refreshesGated()); + + var monitor: disk_monitor.Monitor = .init(.{}, std.Io.Dir.cwd(), ".", null); + manager.monitor = &monitor; + + // `.ok` and `.warn` both allow writes: only `critical` stops them. + try testing.expect(!manager.refreshGated()); + monitor.state_raw.store(@intFromEnum(disk_monitor.State.warn), .monotonic); + try testing.expect(!manager.refreshGated()); + try testing.expectEqual(@as(u64, 0), manager.refreshesGated()); + + monitor.state_raw.store(@intFromEnum(disk_monitor.State.critical), .monotonic); + try testing.expect(manager.refreshGated()); + try testing.expect(manager.refreshGated()); + try testing.expectEqual(@as(u64, 2), manager.refreshesGated()); + + // Free space recovers and the schedule resumes; the counter keeps its total. + monitor.state_raw.store(@intFromEnum(disk_monitor.State.ok), .monotonic); + try testing.expect(!manager.refreshGated()); + try testing.expectEqual(@as(u64, 2), manager.refreshesGated()); +} + test "statusSnapshot on an empty manager copies nothing" { var threaded: std.Io.Threaded = .init(testing.allocator, .{}); defer threaded.deinit(); diff --git a/src/server/clients.zig b/src/server/clients.zig new file mode 100644 index 0000000..d8125a7 --- /dev/null +++ b/src/server/clients.zig @@ -0,0 +1,502 @@ +//! Client auto-materialisation (PLAN §7.2): every address that asks a question +//! ends up as a row in `clients`, so the operator can name it and assign it a +//! group without typing an address by hand. +//! +//! A DNS query must never wait on a database write, so `track` only records the +//! address in a fixed-capacity table in memory. A background loop drains that +//! table every `flush_interval_s` and writes one row per distinct client, and +//! prunes the rows of devices that went quiet on every +//! `prune_every_passes`-th pass. +//! +//! The table is bounded at `max_pending`. A full table drops the address and +//! counts it under `dropped_full`: a burst of spoofed source addresses must not +//! be able to grow this allocation, and a dropped address costs nothing, since +//! the next query from that client tracks it again. +//! +//! A materialised row does not reach the matcher until the next manager reload. +//! Nothing depends on it: `groupForClient` already falls back to the prefix +//! rules and then to the default group, so the row exists for the operator's +//! benefit, not for resolution. + +const std = @import("std"); + +const address = @import("../platform/address.zig"); +const clients_repo = @import("../storage/repositories/clients_repo.zig"); +const db = @import("../storage/db.zig"); +const disk_monitor = @import("../storage/disk_monitor.zig"); + +const log = std.log.scoped(.clients); + +/// RFC 5952 text of any IPv6 address. `format` never writes more than this, so +/// the formatting in `flushOnce` cannot fail. +const max_ip_text = 45; + +/// One address waiting for its row, with the wall-clock second of its most +/// recent query. +const Pending = struct { + addr: address.NetAddress, + last_seen: i64, +}; + +pub const Tracker = struct { + /// Distinct clients one flush interval can carry. A household LAN holds two + /// orders of magnitude fewer; the headroom is for the spoofing case. + pub const max_pending = 512; + pub const flush_interval_s = 60; + /// One day at `flush_interval_s` seconds per pass. + pub const prune_every_passes = 1440; + + /// `tracked` counts the `track` calls that landed in the table, whether + /// they created an entry or refreshed one, so `tracked + dropped_full` is + /// the number of `track` calls. + pub const Stats = struct { + tracked: u64 = 0, + flushed: u64 = 0, + dropped_full: u64 = 0, + pruned: u64 = 0, + flush_failures: u64 = 0, + }; + + /// Guards `pending`, `count`, `passes` and `stats`. Every field below is + /// written under it, so a reader takes it too; see `snapshotStats`. + mutex: std.Io.Mutex, + retention_days: u16, + pending: [max_pending]Pending, + count: u32, + passes: u64, + stats: Stats, + + /// `retention_days` is `logging.retention_days`, the same knob the query log + /// prunes by (milestone-7 ruling 16). A client silent for that long is as + /// uninteresting as a query that old. + pub fn init(retention_days: u16) Tracker { + return .{ + .mutex = .init, + .retention_days = retention_days, + .pending = undefined, + .count = 0, + .passes = 0, + .stats = .{}, + }; + } + + /// Records `addr` as seen now. Called from the query path, so it writes no + /// database and returns no error: a full table drops the address. + /// + /// `lockUncancelable` rather than `lock`: the caller is `Handler.handle`, + /// which has no error union to carry `error.Canceled` out of. The critical + /// section is a scan of at most `max_pending` addresses and holds no I/O. + pub fn track(self: *Tracker, io: std.Io, addr: address.NetAddress) void { + self.trackAt(io, addr, std.Io.Clock.real.now(io).toSeconds()); + } + + /// `track` with the timestamp supplied, so a test does not depend on the + /// wall clock. + pub fn trackAt(self: *Tracker, io: std.Io, addr: address.NetAddress, now_s: i64) void { + self.mutex.lockUncancelable(io); + defer self.mutex.unlock(io); + + for (self.pending[0..self.count]) |*entry| { + if (!entry.addr.eql(addr)) continue; + entry.last_seen = now_s; + self.stats.tracked += 1; + return; + } + if (self.count == max_pending) { + self.stats.dropped_full += 1; + return; + } + self.pending[self.count] = .{ .addr = addr, .last_seen = now_s }; + self.count += 1; + self.stats.tracked += 1; + } + + /// Flush loop, first flush one interval in: an empty table at startup has + /// nothing to write. + /// + /// `boot` rather than `awake`, so a box that suspends still sees its day + /// elapse and prunes on schedule. + /// + /// `database` must be a connection dedicated to this loop: no other task may + /// use the same handle while it runs. `FULLMUTEX` (`db.zig:218`) serializes + /// one SQLite call against another, but a transaction is connection state, + /// not call state, so a flush that lands between another writer's BEGIN and + /// COMMIT would commit or roll back with that writer's batch. Milestone-7 + /// ruling 21 gives this loop its own `config.db` connection; the tracker + /// opens nothing itself. + /// + /// `monitor` gates the pass: while free space is critical the tracker writes + /// nothing, and the addresses it would have written stay dropped. + pub fn run( + self: *Tracker, + io: std.Io, + database: *db.Db, + monitor: ?*disk_monitor.Monitor, + ) std.Io.Cancelable!void { + const interval: std.Io.Clock.Duration = .{ + .raw = .fromSeconds(flush_interval_s), + .clock = .boot, + }; + while (true) { + try interval.sleep(io); + const writes_allowed = if (monitor) |m| m.writesAllowed() else true; + self.flushOnce(io, database, writes_allowed); + } + } + + /// One pass: drain the table, write a row per client, and prune on every + /// `prune_every_passes`-th pass. + /// + /// A gated pass does nothing at all, not even count: the work it skipped is + /// still owed, and the pending addresses it leaves behind are re-tracked by + /// the next query from each client. + /// + /// Every database failure logs one line at `warn` and counts. Nothing + /// retries within a pass: the dropped addresses come back on their own, and + /// a failed prune repeats the same work a day later against the same rows. + /// + /// Only `run` may call this concurrently with itself — the drain buffer is + /// this call's stack, but the pass counter and the prune schedule assume a + /// single caller. + pub fn flushOnce(self: *Tracker, io: std.Io, database: *db.Db, writes_allowed: bool) void { + if (!writes_allowed) return; + + var drained: [max_pending]Pending = undefined; + const batch = self.drain(io, &drained); + + var flushed: u64 = 0; + var failures: u64 = 0; + for (batch) |entry| { + var buf: [max_ip_text]u8 = undefined; + var w: std.Io.Writer = .fixed(&buf); + entry.addr.format(&w) catch unreachable; + + if (clients_repo.upsertSeen(database, w.buffered(), entry.last_seen)) { + flushed += 1; + } else |err| { + if (failures == 0) { + log.warn("materialising client {s} failed: {s}", .{ w.buffered(), @errorName(err) }); + } + failures += 1; + } + } + + self.mutex.lockUncancelable(io); + self.passes += 1; + self.stats.flushed += flushed; + self.stats.flush_failures += failures; + const due = self.passes % prune_every_passes == 0; + self.mutex.unlock(io); + + if (!due) return; + const cutoff = std.Io.Clock.real.now(io).toSeconds() - @as(i64, self.retention_days) * 86_400; + if (clients_repo.pruneStale(database, cutoff)) |deleted| { + self.mutex.lockUncancelable(io); + self.stats.pruned += deleted; + self.mutex.unlock(io); + } else |err| { + log.warn("pruning clients before {d} failed: {s}", .{ cutoff, @errorName(err) }); + self.mutex.lockUncancelable(io); + self.stats.flush_failures += 1; + self.mutex.unlock(io); + } + } + + pub fn snapshotStats(self: *Tracker, io: std.Io) Stats { + self.mutex.lockUncancelable(io); + defer self.mutex.unlock(io); + return self.stats; + } + + /// Clients waiting for their row. Reaching `max_pending` is what turns + /// further addresses into `dropped_full`. + pub fn pendingClients(self: *Tracker, io: std.Io) u32 { + self.mutex.lockUncancelable(io); + defer self.mutex.unlock(io); + return self.count; + } + + /// Empties the table into `out` and returns what it copied. The lock is + /// released before any database call, so the query path never waits on + /// SQLite. + fn drain(self: *Tracker, io: std.Io, out: *[max_pending]Pending) []const Pending { + self.mutex.lockUncancelable(io); + defer self.mutex.unlock(io); + + const n = self.count; + @memcpy(out[0..n], self.pending[0..n]); + self.count = 0; + return out[0..n]; + } +}; + +// --------------------------------------------------------------------------- +// tests +// --------------------------------------------------------------------------- + +const migrations = @import("../storage/migrations.zig"); + +const testing = std.testing; + +fn openMigrated() !db.Db { + var database = try db.Db.open(":memory:", .{ .mode = .memory }); + errdefer database.close(); + try db.applyPragmas(&database, .{}); + _ = try migrations.migrate(&database); + return database; +} + +fn lastSeen(database: *db.Db, ip: []const u8) !i64 { + var stmt = try database.prepare("SELECT last_seen FROM clients WHERE ip = ?1"); + defer stmt.deinit(); + try stmt.bindText(1, ip); + try testing.expect(try stmt.step()); + return stmt.columnInt(0); +} + +fn parsed(text: []const u8) address.NetAddress { + return address.NetAddress.parse(text) catch unreachable; +} + +test "a client tracked twice before a flush yields one row at the later time" { + var threaded: std.Io.Threaded = .init(testing.allocator, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var database = try openMigrated(); + defer database.close(); + + var tracker: Tracker = .init(30); + tracker.trackAt(io, parsed("192.168.1.10"), 1700000000); + tracker.trackAt(io, parsed("192.168.1.10"), 1700000030); + try testing.expectEqual(@as(u32, 1), tracker.pendingClients(io)); + + tracker.flushOnce(io, &database, true); + + try testing.expectEqual(@as(i64, 1), try clients_repo.countClients(&database)); + try testing.expectEqual(@as(i64, 1700000030), try lastSeen(&database, "192.168.1.10")); + + const stats = tracker.snapshotStats(io); + try testing.expectEqual(@as(u64, 2), stats.tracked); + try testing.expectEqual(@as(u64, 1), stats.flushed); + try testing.expectEqual(@as(u64, 0), stats.flush_failures); + try testing.expectEqual(@as(u32, 0), tracker.pendingClients(io)); +} + +test "distinct clients each get a row and ipv6 text is canonical" { + var threaded: std.Io.Threaded = .init(testing.allocator, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var database = try openMigrated(); + defer database.close(); + + var tracker: Tracker = .init(30); + tracker.trackAt(io, parsed("192.168.1.10"), 1700000000); + tracker.trackAt(io, parsed("192.168.1.11"), 1700000001); + tracker.trackAt(io, parsed("fd00:0:0:0:0:0:0:1"), 1700000002); + // An IPv4-mapped literal is the same client as its plain form. + tracker.trackAt(io, address.NetAddress.fromIp(try std.Io.net.IpAddress.parse("::ffff:192.168.1.10", 53)), 1700000003); + try testing.expectEqual(@as(u32, 3), tracker.pendingClients(io)); + + tracker.flushOnce(io, &database, true); + + try testing.expectEqual(@as(i64, 3), try clients_repo.countClients(&database)); + try testing.expectEqual(@as(i64, 1700000003), try lastSeen(&database, "192.168.1.10")); + try testing.expectEqual(@as(i64, 1700000002), try lastSeen(&database, "fd00::1")); + try testing.expectEqual(@as(u64, 3), tracker.snapshotStats(io).flushed); +} + +test "a full table drops further clients and counts them" { + var threaded: std.Io.Threaded = .init(testing.allocator, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var database = try openMigrated(); + defer database.close(); + + var tracker: Tracker = .init(30); + for (0..Tracker.max_pending) |i| { + var octets: [4]u8 = undefined; + std.mem.writeInt(u32, &octets, @intCast(i), .big); + tracker.trackAt(io, .{ .ip4 = octets }, 1700000000); + } + try testing.expectEqual(@as(u32, Tracker.max_pending), tracker.pendingClients(io)); + try testing.expectEqual(@as(u64, 0), tracker.snapshotStats(io).dropped_full); + + tracker.trackAt(io, parsed("203.0.113.7"), 1700000000); + tracker.trackAt(io, parsed("203.0.113.8"), 1700000000); + try testing.expectEqual(@as(u32, Tracker.max_pending), tracker.pendingClients(io)); + + const stats = tracker.snapshotStats(io); + try testing.expectEqual(@as(u64, 2), stats.dropped_full); + try testing.expectEqual(@as(u64, Tracker.max_pending), stats.tracked); + + // A tracked client still refreshes while the table is full, and the flush + // makes room for the next newcomer. + tracker.trackAt(io, .{ .ip4 = .{ 0, 0, 0, 0 } }, 1700000060); + tracker.flushOnce(io, &database, true); + try testing.expectEqual(@as(i64, Tracker.max_pending), try clients_repo.countClients(&database)); + try testing.expectEqual(@as(i64, 1700000060), try lastSeen(&database, "0.0.0.0")); + + tracker.trackAt(io, parsed("203.0.113.7"), 1700000060); + try testing.expectEqual(@as(u32, 1), tracker.pendingClients(io)); +} + +test "a flush touches a hand-edited row without changing what the operator set" { + var threaded: std.Io.Threaded = .init(testing.allocator, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var database = try openMigrated(); + defer database.close(); + try database.exec("INSERT INTO groups (id, name) VALUES (2, 'kids');"); + try database.exec( + \\INSERT INTO clients (ip, name, group_id, hand_edited, first_seen, last_seen) + \\VALUES ('192.168.1.10', 'laptop', 2, 1, 1690000000, 1690000000); + ); + + var tracker: Tracker = .init(30); + tracker.trackAt(io, parsed("192.168.1.10"), 1700000000); + tracker.flushOnce(io, &database, true); + + try testing.expectEqual(@as(i64, 1), try clients_repo.countClients(&database)); + try testing.expectEqual(@as(i64, 1700000000), try lastSeen(&database, "192.168.1.10")); + try testing.expectEqual( + @as(i64, 1), + try database.queryInt( + \\SELECT count(*) FROM clients + \\ WHERE ip = '192.168.1.10' AND name = 'laptop' AND group_id = 2 + \\ AND hand_edited = 1 AND first_seen = 1690000000 + ), + ); +} + +test "a gated pass writes nothing and keeps the pending clients" { + var threaded: std.Io.Threaded = .init(testing.allocator, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var database = try openMigrated(); + defer database.close(); + + var monitor: disk_monitor.Monitor = .init(.{}, std.Io.Dir.cwd(), ".", null); + monitor.state_raw.store(@intFromEnum(disk_monitor.State.critical), .monotonic); + try testing.expect(!monitor.writesAllowed()); + + var tracker: Tracker = .init(30); + tracker.trackAt(io, parsed("192.168.1.10"), 1700000000); + tracker.flushOnce(io, &database, monitor.writesAllowed()); + + try testing.expectEqual(@as(i64, 0), try clients_repo.countClients(&database)); + try testing.expectEqual(@as(u32, 1), tracker.pendingClients(io)); + try testing.expectEqual(@as(u64, 0), tracker.snapshotStats(io).flushed); + try testing.expectEqual(@as(u64, 0), tracker.passes); + + // Free space recovers and the same pending client lands. + monitor.state_raw.store(@intFromEnum(disk_monitor.State.warn), .monotonic); + tracker.flushOnce(io, &database, monitor.writesAllowed()); + try testing.expectEqual(@as(i64, 1), try clients_repo.countClients(&database)); + try testing.expectEqual(@as(u64, 1), tracker.passes); +} + +test "a failing upsert counts and leaves the client to be tracked again" { + var threaded: std.Io.Threaded = .init(testing.allocator, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var database = try openMigrated(); + defer database.close(); + try database.exec( + \\CREATE TRIGGER refuse_insert BEFORE INSERT ON clients + \\BEGIN SELECT RAISE(ABORT, 'refused'); END; + ); + + var tracker: Tracker = .init(30); + tracker.trackAt(io, parsed("192.168.1.10"), 1700000000); + tracker.trackAt(io, parsed("192.168.1.11"), 1700000000); + tracker.flushOnce(io, &database, true); + + try testing.expectEqual(@as(i64, 0), try clients_repo.countClients(&database)); + const stats = tracker.snapshotStats(io); + try testing.expectEqual(@as(u64, 0), stats.flushed); + try testing.expectEqual(@as(u64, 2), stats.flush_failures); + try testing.expectEqual(@as(u32, 0), tracker.pendingClients(io)); + + try database.exec("DROP TRIGGER refuse_insert;"); + tracker.trackAt(io, parsed("192.168.1.10"), 1700000060); + tracker.flushOnce(io, &database, true); + try testing.expectEqual(@as(i64, 1), try clients_repo.countClients(&database)); +} + +test "the pass that comes due prunes the clients that went quiet" { + var threaded: std.Io.Threaded = .init(testing.allocator, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var database = try openMigrated(); + defer database.close(); + + const now = std.Io.Clock.real.now(io).toSeconds(); + const day = 86_400; + try clients_repo.upsertSeen(&database, "10.0.0.1", now - 40 * day); + try clients_repo.upsertSeen(&database, "10.0.0.2", now - 29 * day); + + var tracker: Tracker = .init(30); + // Every pass before the due one leaves both rows alone. + for (0..Tracker.prune_every_passes - 1) |_| { + tracker.flushOnce(io, &database, true); + } + try testing.expectEqual(@as(i64, 2), try clients_repo.countClients(&database)); + try testing.expectEqual(@as(u64, 0), tracker.snapshotStats(io).pruned); + + tracker.flushOnce(io, &database, true); + + try testing.expectEqual(@as(i64, 1), try clients_repo.countClients(&database)); + try testing.expectEqual(@as(i64, now - 29 * day), try lastSeen(&database, "10.0.0.2")); + try testing.expectEqual(@as(u64, 1), tracker.snapshotStats(io).pruned); + try testing.expectEqual(@as(u64, Tracker.prune_every_passes), tracker.passes); +} + +test "a shorter retention prunes what the default keeps" { + var threaded: std.Io.Threaded = .init(testing.allocator, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var database = try openMigrated(); + defer database.close(); + + const now = std.Io.Clock.real.now(io).toSeconds(); + try clients_repo.upsertSeen(&database, "10.0.0.1", now - 3 * 86_400); + + var tracker: Tracker = .init(1); + tracker.passes = Tracker.prune_every_passes - 1; + tracker.flushOnce(io, &database, true); + + try testing.expectEqual(@as(i64, 0), try clients_repo.countClients(&database)); + try testing.expectEqual(@as(u64, 1), tracker.snapshotStats(io).pruned); +} + +test "the run loop flushes on its interval and returns on cancel" { + var threaded: std.Io.Threaded = .init(testing.allocator, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var database = try openMigrated(); + defer database.close(); + + var tracker: Tracker = .init(30); + tracker.trackAt(io, parsed("192.168.1.10"), 1700000000); + + var future = try io.concurrent(Tracker.run, .{ + &tracker, + io, + &database, + @as(?*disk_monitor.Monitor, null), + }); + // The first flush is one interval away, so cancelling immediately proves the + // loop starts by sleeping rather than by writing. + try testing.expectError(error.Canceled, future.cancel(io)); + try testing.expectEqual(@as(i64, 0), try clients_repo.countClients(&database)); + try testing.expectEqual(@as(u32, 1), tracker.pendingClients(io)); +} diff --git a/src/server/handler.zig b/src/server/handler.zig index 3fd221a..707f5e9 100644 --- a/src/server/handler.zig +++ b/src/server/handler.zig @@ -1,23 +1,47 @@ //! The serving handler: raw bytes from a listener in, bytes to send back out. -//! It owns no socket, no clock and no allocator — the listener supplies the -//! buffers and the upstream supplies the answer. +//! It owns no socket and no allocator — the listener supplies the buffers, the +//! upstream supplies the answer, and every collaborator is an optional pointer +//! the composition root fills in. //! //! `handle` returns no error union. Every failure is either a DNS response the //! client can act on or a counted drop, because a listener has nothing useful //! to do with an error value: it cannot retry and it must not die. That is the //! "every failure mode is visible" rule applied to the hot path — the counters -//! are the failure surface. +//! are the failure surface. The same rule is why every mutex here is taken with +//! `lockUncancelable`: `handle` has no error union to carry `error.Canceled` +//! out of, and each critical section is a bounded, I/O-free table access. //! -//! Phase 7 extends this file with filtering, cache, local records and query -//! logging. None of that is here: this handler forwards every query. +//! The pipeline is PLAN §4: rate limit → parse → group → local records → +//! forward zones → filtering → safe search → cache → upstream → CNAME +//! uncloaking → cache store → query log. It logs nothing per query: the query +//! log is a database row, and `std.log` is reserved for failures nobody else +//! records. const std = @import("std"); + +const address = @import("../platform/address.zig"); +const clients = @import("clients.zig"); +const dns_cache = @import("../cache/dns_cache.zig"); const edns = @import("../dns/edns.zig"); +const forward_client = @import("../local/forward_client.zig"); +const forward_zones = @import("../local/forward_zones.zig"); const header = @import("../dns/header.zig"); +const logger_mod = @import("../storage/logger.zig"); +const manager = @import("../filter/manager.zig"); +const matcher = @import("../filter/matcher.zig"); +const model = @import("../config/model.zig"); +const name = @import("../dns/name.zig"); const packet = @import("../dns/packet.zig"); +const pause = @import("pause.zig"); const question = @import("../dns/question.zig"); -const types = @import("../dns/types.zig"); +const rate_limiter = @import("rate_limiter.zig"); +const record = @import("../dns/record.zig"); +const records = @import("../local/records.zig"); +const response = @import("../filter/response.zig"); +const safesearch = @import("../filter/safesearch.zig"); const transport = @import("../upstream/transport.zig"); +const types = @import("../dns/types.zig"); +const validate = @import("../config/validate.zig"); /// Which listener a query arrived on. Only the UDP size limit depends on it. pub const Transport = enum { udp, tcp }; @@ -31,6 +55,36 @@ pub const udp_limit_min: u16 = types.max_udp_payload; /// reflection amplification cost more than a TCP retry. pub const udp_limit_max: u16 = 4096; +/// CNAME links followed when uncloaking (PLAN §6.3). A chain longer than this +/// is either a loop or an attempt to outrun the walk. +pub const max_cname_depth = 8; + +/// `"cname:"` plus the longest `matcher.Reason` tag, which the comptime block +/// below proves fits the 32 bytes `logger.Entry` stores. +const cname_reason_prefix = "cname:"; +const max_reason_len = 32; + +comptime { + for (std.enums.values(matcher.Reason)) |reason| { + if (cname_reason_prefix.len + @tagName(reason).len > max_reason_len) { + @compileError("a matcher.Reason tag no longer fits the query log's reason field"); + } + } +} + +/// RFC 5952 text of any IPv6 address. `logger.Entry` truncates at the same +/// width, so nothing an address formats to is ever cut. +const max_ip_text = 45; + +/// `udp://` or `tcp://`, an IPv6 literal in brackets, and a port. +const max_resolver_text = "tcp://[".len + max_ip_text + "]:65535".len; + +/// Ruling 20 leaves the pool's answering endpoint out of reach: the pool tracks +/// it, but reading it back would mean new plumbing through `transport.Client` +/// for one log field. Every pool answer is therefore logged as coming from +/// "pool"; forward zones name their resolver and local records say "local". +const pool_upstream = "pool"; + /// A synthesized reply is a header, at most one question and at most one OPT /// record: 12 + (255 + 4) + 11 = 282 bytes worst case. Every buffer this file /// builds into is at least `udp_limit_min`, so the builder cannot run out of @@ -44,15 +98,49 @@ comptime { pub const Handler = struct { /// In production this is `pool.client()`. upstream: transport.Client, + blocking: response.Options, + ecs_mode: model.EcsMode = .strip, + forward_read_timeout: std.Io.Clock.Duration, + manager: ?*manager.Manager = null, + records: *const records.Records, + zones: *const forward_zones.Zones, + cache: ?*dns_cache.DnsCache = null, + cache_mutex: std.Io.Mutex = .init, + /// `cfg.cache.negative_ttl_max`. `DnsCache` keeps no copy of its config and + /// `dns_cache.classify` needs the value, so the handler carries it. Zero + /// disables negative caching, which is what a cache-less handler wants. + negative_ttl_max: u32 = 0, + limiter: ?*rate_limiter.RateLimiter = null, + limiter_mutex: std.Io.Mutex = .init, + logger: ?*logger_mod.Logger = null, + pause: ?*pause.Pause = null, + tracker: ?*clients.Tracker = null, stats: Stats = .{}, pub const Stats = struct { + /// Queries answered by the upstream pool. queries: std.atomic.Value(u64) = .init(0), dropped_malformed: std.atomic.Value(u64) = .init(0), formerr: std.atomic.Value(u64) = .init(0), notimp: std.atomic.Value(u64) = .init(0), servfail: std.atomic.Value(u64) = .init(0), truncated: std.atomic.Value(u64) = .init(0), + refused: std.atomic.Value(u64) = .init(0), + blocked: std.atomic.Value(u64) = .init(0), + uncloak_blocked: std.atomic.Value(u64) = .init(0), + local_answers: std.atomic.Value(u64) = .init(0), + forward_zone_answers: std.atomic.Value(u64) = .init(0), + cache_hits: std.atomic.Value(u64) = .init(0), + paused_queries: std.atomic.Value(u64) = .init(0), + /// Queries answered before the first snapshot existed, so no group and + /// no filtering applied (ruling 4). + unfiltered_queries: std.atomic.Value(u64) = .init(0), + safesearch_rewrites: std.atomic.Value(u64) = .init(0), + ecs_strip_failed: std.atomic.Value(u64) = .init(0), + /// Mirrors the tracker's own `dropped_full`, refreshed on every query: + /// `Tracker.track` reports nothing back, and a counter that only the + /// tracker holds would not appear beside the rest of these. + tracker_full: std.atomic.Value(u64) = .init(0), }; pub const Outcome = union(enum) { @@ -67,8 +155,10 @@ pub const Handler = struct { self: *Handler, io: std.Io, which: Transport, + from: address.NetAddress, query: []const u8, response_buf: []u8, + scratch: *Scratch, ) Outcome { std.debug.assert(response_buf.len >= udp_limit_min); std.debug.assert(response_buf.len <= transport.max_message_len); @@ -87,6 +177,19 @@ pub const Handler = struct { return .drop; } + // Ruling 8: the limit is charged before the parse, so a flood of + // malformed queries costs one header read each. The REFUSED reply needs + // the ID, which is why the header is read first. Localhost is not + // exempt. + if (self.limiter) |limiter| { + self.limiter_mutex.lockUncancelable(io); + const allowed = limiter.check(std.Io.Clock.awake.now(io), from.key()); + self.limiter_mutex.unlock(io); + if (!allowed) { + return synthesize(hdr, null, null, .refused, &self.stats.refused, response_buf); + } + } + const p = packet.parse(query) catch |err| switch (err) { error.Truncated => { bump(&self.stats.dropped_malformed); @@ -135,37 +238,565 @@ pub const Handler = struct { const q = packet.firstQuestion(p) orelse return synthesize(hdr, null, opt, .form_err, &self.stats.formerr, response_buf); - const reply = self.upstream.exchange(io, query, response_buf) catch |err| switch (transport.group(err)) { - // The process is shutting down. There is nothing to say, and the - // client is about to lose the socket anyway; the listener's own - // counters record the abandoned datagram. - .cancellation => return .drop, - .peer_fault, .local_resource => return synthesize( - hdr, - q, - opt, - .serv_fail, - &self.stats.servfail, - response_buf, - ), - }; + // One clock read serves the log's elapsed time and every cache + // timestamp, so nothing inside one query disagrees about when it ran. + const started = std.Io.Clock.real.now(io); - bump(&self.stats.queries); - - if (which == .udp and reply.len > udpLimit(p)) { - // The upstream answer occupies `response_buf`, so the replacement - // is built beside it and copied over. - bump(&self.stats.truncated); - var scratch: [udp_limit_min]u8 = undefined; - const message = build(hdr, q, opt, .no_error, true, &scratch); - @memcpy(response_buf[0..message.len], message); - return .{ .reply = response_buf[0..message.len] }; + if (self.tracker) |tracker| { + tracker.track(io, from); + self.stats.tracker_full.store(tracker.snapshotStats(io).dropped_full, .monotonic); } - return .{ .reply = reply }; + // Ruling 4: no snapshot means no group and no filtering, and the query + // is answered anyway — DNS availability beats filtering for a + // household. + const acquired = if (self.manager) |m| m.acquire(io) else null; + defer if (acquired) |snapshot| snapshot.release(io); + const snapshot: ?*const matcher.Snapshot = if (acquired) |a| a.snapshot else null; + if (snapshot == null) bump(&self.stats.unfiltered_queries); + + var ctx: Context = .{ + .handler = self, + .io = io, + .which = which, + .from = from, + .query = query, + .response_buf = response_buf, + .scratch = scratch, + .hdr = hdr, + .p = p, + .q = q, + .opt = opt, + .do_bit = if (opt) |o| o.do_bit else false, + .started = started, + .now_s = started.toSeconds(), + .snapshot = snapshot, + .group = if (snapshot) |s| s.groupForClient(from) else 0, + .domain = matcher.normalize(q.name, &scratch.normalize), + }; + return ctx.run(); } }; +/// Per-query working memory, owned by the listener's slot so that no query path +/// allocates. Every buffer is used by one query at a time. +pub const Scratch = struct { + normalize: [types.max_name_len]u8, + key: [dns_cache.max_key_len]u8, + /// The rewritten outgoing query. + build: [512]u8, + /// Forward-zone client frame buffer. + frame: [forward_client.min_frame_buf]u8, + /// The safe-search reply, and before that the ECS-stripped form of a + /// safe-search query — the two never overlap in time, because the outgoing + /// query is spent once the upstream has answered. + synth: [4096]u8, + /// Uncloaking normalizes each CNAME target while `normalize` still holds + /// the queried name the log entry needs. + uncloak: [types.max_name_len]u8, +}; + +/// What the pipeline is allowed to do on the upstream path. Filtering is off +/// while paused (ruling 18) and for a non-IN class (ruling 9), which also skips +/// the cache. +const Mode = struct { filter: bool, cache: bool }; + +/// The fields every stage of one query shares. It exists so that the stages can +/// be separate functions without threading a dozen parameters through each. +const Context = struct { + handler: *Handler, + io: std.Io, + which: Transport, + from: address.NetAddress, + query: []const u8, + response_buf: []u8, + scratch: *Scratch, + hdr: header.Header, + p: packet.Packet, + q: question.Question, + opt: ?edns.OptRecord, + do_bit: bool, + started: std.Io.Timestamp, + now_s: i64, + snapshot: ?*const matcher.Snapshot, + group: u32, + /// The queried name, normalized into `scratch.normalize`. + domain: []const u8, + + /// PLAN §6 in order. Local records win over forward zones (ruling 6), and + /// both win over filtering: a name nxdns answers itself never reaches a + /// blocklist. + fn run(ctx: *Context) Handler.Outcome { + if (ctx.q.qclass != .in) return ctx.viaUpstream(.{ .filter = false, .cache = false }); + if (ctx.handler.records.hasName(ctx.domain)) return ctx.viaLocal(); + if (ctx.handler.zones.match(ctx.domain)) |zone| return ctx.viaForwardZone(zone); + + const paused = if (ctx.handler.pause) |p| p.isPaused(ctx.now_s) else false; + if (paused) bump(&ctx.handler.stats.paused_queries); + return ctx.viaUpstream(.{ .filter = !paused, .cache = true }); + } + + /// PLAN §6.4. A name that exists with no record of the queried type is + /// NODATA, not a forward: nxdns owns the name either way. + /// + /// A local CNAME is returned as it stands (ruling 12). The client re-queries + /// the target, and that query runs the whole pipeline. + fn viaLocal(ctx: *Context) Handler.Outcome { + const found = ctx.handler.records.lookup(ctx.domain, ctx.q.qtype); + + var b = packet.ResponseBuilder.init(ctx.response_buf, ctx.hdr, ctx.q) catch + return ctx.servFail(); + b.setAuthoritative(true); + records.writeAnswers(&b, ctx.q.name, found) catch return ctx.servFail(); + if (ctx.opt) |o| b.addOptEcho(o, ctx.do_bit) catch return ctx.servFail(); + + bump(&ctx.handler.stats.local_answers); + return ctx.reply(b.finish(), .{ .upstream = "local" }); + } + + /// PLAN §6.5. The zone's resolver answers, and filtering, safe search and + /// uncloaking are all bypassed (ruling 7): a conditional forward exists to + /// reach a box on the LAN, and a blocklist entry must not stand between the + /// two. + fn viaForwardZone(ctx: *Context, zone: *const forward_zones.Zone) Handler.Outcome { + const key = dns_cache.buildKey( + &ctx.scratch.key, + ctx.domain, + @intFromEnum(ctx.q.qtype), + @intFromEnum(ctx.q.qclass), + ctx.do_bit, + null, + ); + if (ctx.cacheGet(key)) |hit| return ctx.reply(hit, .{ .cache_hit = true }); + + var client: forward_client.ForwardClient = .init( + zone.resolver, + &ctx.scratch.frame, + ctx.handler.forward_read_timeout, + ); + const answer = client.exchange(ctx.io, ctx.query, ctx.response_buf) catch |err| { + return switch (transport.group(err)) { + .cancellation => .drop, + .peer_fault, .local_resource => ctx.servFail(), + }; + }; + + bump(&ctx.handler.stats.forward_zone_answers); + ctx.cachePut(key, answer); + + var text: [max_resolver_text]u8 = undefined; + return ctx.reply(answer, .{ + .cache_hit = false, + .upstream = resolverText(zone.resolver, &text), + }); + } + + /// Filtering, safe search, the cache and the upstream pool. + fn viaUpstream(ctx: *Context, mode: Mode) Handler.Outcome { + var target: ?name.Name = null; + if (mode.filter) if (ctx.snapshot) |s| { + const decision = s.evaluate(ctx.group, ctx.domain); + if (decision.blocked) return ctx.viaBlocked(decision.reason, false); + if (s.safeSearch(ctx.group)) target = safesearch.rewrite(ctx.domain); + }; + + const outgoing = ctx.outgoingQuery(target) orelse return ctx.servFail(); + if (target != null) bump(&ctx.handler.stats.safesearch_rewrites); + + // Ruling 5: a safe-search answer is not cached. It would have to be + // stored under the rewritten name, and every hit would then have to + // re-encode the answer records under the original one. + const cacheable = mode.cache and target == null and outgoing.cacheable; + const key: ?[]const u8 = if (cacheable) ctx.cacheKey() else null; + if (key) |k| if (ctx.cacheGet(k)) |hit| return ctx.reply(hit, .{ .cache_hit = true }); + + const answer = ctx.handler.upstream.exchange(ctx.io, outgoing.bytes, ctx.response_buf) catch |err| { + return switch (transport.group(err)) { + // The process is shutting down. There is nothing to say, and + // the client is about to lose the socket anyway; the listener's + // own counters record the abandoned datagram. + .cancellation => .drop, + .peer_fault, .local_resource => ctx.servFail(), + }; + }; + bump(&ctx.handler.stats.queries); + + // Ruling 11. A safe-search answer is exempt: its records belong to the + // provider's target name, which the client never asked about. + if (mode.filter and target == null) if (ctx.snapshot) |s| { + if (ctx.uncloak(s, answer)) |reason| return ctx.viaBlocked(reason, true); + }; + + const final = if (target) |t| + (ctx.safeSearchReply(t, answer) orelse return ctx.servFail()) + else + answer; + + if (key) |k| ctx.cachePut(k, final); + return ctx.reply(final, .{ .cache_hit = false, .upstream = pool_upstream }); + } + + /// PLAN §6.2. `uncloaked` distinguishes a name blocked in its own right + /// from one blocked through its CNAME chain; either way the reply answers + /// the question the client asked. + fn viaBlocked(ctx: *Context, reason: matcher.Reason, uncloaked: bool) Handler.Outcome { + const bytes = response.writeBlocked( + ctx.response_buf, + ctx.hdr, + ctx.q, + ctx.opt, + ctx.do_bit, + ctx.handler.blocking, + ) catch return ctx.servFail(); + + bump(if (uncloaked) &ctx.handler.stats.uncloak_blocked else &ctx.handler.stats.blocked); + + var reason_buf: [max_reason_len]u8 = undefined; + return ctx.reply(bytes, .{ + .blocked = true, + .block_reason = blockReason(&reason_buf, reason, uncloaked), + }); + } + + /// Ruling 20: a failure the client can see is still a failure nobody + /// resolved, so it is counted and not query-logged. + fn servFail(ctx: *Context) Handler.Outcome { + return synthesize( + ctx.hdr, + ctx.q, + ctx.opt, + .serv_fail, + &ctx.handler.stats.servfail, + ctx.response_buf, + ); + } + + /// The last step of every answered path: the UDP size check, then the query + /// log row. `bytes` is always a prefix of `response_buf`. + fn reply(ctx: *Context, bytes: []u8, fields: LogFields) Handler.Outcome { + var message = bytes; + if (ctx.which == .udp and message.len > udpLimit(ctx.p)) { + // The answer occupies `response_buf`, so the replacement is built + // beside it and copied over. + bump(&ctx.handler.stats.truncated); + var scratch: [udp_limit_min]u8 = undefined; + const truncated = build(ctx.hdr, ctx.q, ctx.opt, .no_error, true, &scratch); + @memcpy(ctx.response_buf[0..truncated.len], truncated); + message = ctx.response_buf[0..truncated.len]; + } + ctx.log(fields); + return .{ .reply = message }; + } + + fn log(ctx: *Context, fields: LogFields) void { + const logger = ctx.handler.logger orelse return; + + var ip_buf: [max_ip_text]u8 = undefined; + var w: std.Io.Writer = .fixed(&ip_buf); + ctx.from.format(&w) catch unreachable; + + const now = std.Io.Clock.real.now(ctx.io); + logger.log(ctx.io, logger_mod.Entry.init(.{ + .timestamp = ctx.now_s, + .domain = ctx.domain, + .client_ip = w.buffered(), + .qtype = @intFromEnum(ctx.q.qtype), + .blocked = fields.blocked, + .block_reason = fields.block_reason, + .response_time_us = now.toMicroseconds() - ctx.started.toMicroseconds(), + .cache_hit = fields.cache_hit, + .upstream = fields.upstream, + })); + } + + /// The cache key for this query, or null when it must not be cached. + fn cacheKey(ctx: *Context) ?[]const u8 { + var ecs: ?[]const u8 = null; + if (ctx.handler.ecs_mode == .forward) { + if (ctx.opt) |o| switch (subnetForKey(ctx.query, o)) { + .none => {}, + .subnet => |payload| ecs = payload, + .uncacheable => return null, + }; + } + return dns_cache.buildKey( + &ctx.scratch.key, + ctx.domain, + @intFromEnum(ctx.q.qtype), + @intFromEnum(ctx.q.qclass), + ctx.do_bit, + ecs, + ); + } + + fn cacheGet(ctx: *Context, key: []const u8) ?[]u8 { + const cache = ctx.handler.cache orelse return null; + + ctx.handler.cache_mutex.lockUncancelable(ctx.io); + const hit = cache.get(ctx.now_s, key, ctx.response_buf); + ctx.handler.cache_mutex.unlock(ctx.io); + + const bytes = hit orelse return null; + bump(&ctx.handler.stats.cache_hits); + packet.setId(bytes, ctx.hdr.id); + return bytes; + } + + /// A response the cache declines to hold, or an allocation it cannot make, + /// is not a query failure: the answer already went out. `DnsCache.stats` + /// carries what happened. + fn cachePut(ctx: *Context, key: []const u8, message: []const u8) void { + const cache = ctx.handler.cache orelse return; + const class = dns_cache.classify(message, ctx.handler.negative_ttl_max) orelse return; + + ctx.handler.cache_mutex.lockUncancelable(ctx.io); + defer ctx.handler.cache_mutex.unlock(ctx.io); + cache.put(ctx.now_s, key, message, class) catch {}; + } + + /// The bytes to send upstream. Null means the rewrite did not fit and the + /// caller answers SERVFAIL — which only a safe-search rewrite can hit, + /// because ruling 13 sends the original query when an ECS strip overflows. + fn outgoingQuery(ctx: *Context, target: ?name.Name) ?Outgoing { + const t = target orelse { + if (ctx.handler.ecs_mode != .strip) return .{ .bytes = ctx.query }; + const opt = ctx.opt orelse return .{ .bytes = ctx.query }; + return ctx.stripEcs(ctx.query, ctx.p, opt, &ctx.scratch.build); + }; + + const rewritten = ctx.rewriteQuestion(t) orelse return null; + if (ctx.handler.ecs_mode != .strip) return .{ .bytes = rewritten }; + + // The question rewrite moved the OPT record, so the ECS option comes + // out of the rebuilt query rather than the client's. + const p = packet.parse(rewritten) catch return null; + const rec = packet.findOptRecord(p) orelse return .{ .bytes = rewritten }; + const opt = edns.parseOpt(rewritten, rec) catch return .{ .bytes = rewritten }; + return ctx.stripEcs(rewritten, p, opt, &ctx.scratch.synth); + } + + /// Ruling 13: a query whose stripped form does not fit is forwarded as it + /// arrived, with the subnet still in it, and the counter says so. Dropping + /// the query instead would take the name off the air over a header nobody + /// asked for. + fn stripEcs( + ctx: *Context, + query: []const u8, + p: packet.Packet, + opt: edns.OptRecord, + out: []u8, + ) Outgoing { + const stripped = edns.stripEcs(query, p, opt, out) catch { + bump(&ctx.handler.stats.ecs_strip_failed); + // The subnet reached the resolver after all, so the answer may be + // specific to it while the key — built for `.strip` mode — says + // nothing about a subnet. Caching it would file a subnet-specific + // answer under the name every other client asks for, and reading + // the cache would answer this query from an entry that ignored the + // subnet. Neither direction is safe, so this query skips the cache. + return .{ .bytes = query, .cacheable = false }; + }; + return switch (stripped) { + .unchanged => .{ .bytes = query }, + .rewritten => |bytes| .{ .bytes = bytes }, + }; + } + + /// The client's query with the safe-search target as its question name. + /// + /// The header counts are written fresh, so no record besides the OPT + /// survives: a name in one of them may be a compression pointer into the + /// question, and the question just changed length. A query carrying such a + /// record is already outside RFC 1035 §4.1.2, and the alternative is + /// forwarding a name that no longer decodes. + fn rewriteQuestion(ctx: *Context, target: name.Name) ?[]const u8 { + var w: std.Io.Writer = .fixed(&ctx.scratch.build); + + var hdr = ctx.hdr; + hdr.qdcount = 1; + hdr.ancount = 0; + hdr.nscount = 0; + hdr.arcount = if (ctx.opt == null) 0 else 1; + var encoded: [types.header_len]u8 = undefined; + header.encode(hdr, &encoded); + + w.writeAll(&encoded) catch return null; + question.encode( + .{ .name = target, .qtype = ctx.q.qtype, .qclass = ctx.q.qclass }, + &w, + ) catch return null; + if (ctx.opt) |o| edns.encodeOpt(o, o.options.slice(ctx.query), &w) catch return null; + return w.buffered(); + } + + /// Ruling 10: the client asked about the original name, so the reply keeps + /// the original question, states the CNAME to the target, and then carries + /// the target's address records verbatim. Every other rtype is dropped — + /// their RDATA can hold compression pointers into the upstream message, + /// which would not decode inside this one. + fn safeSearchReply(ctx: *Context, target: name.Name, answer: []const u8) ?[]u8 { + const p = packet.parse(answer) catch return null; + + // The synthesized CNAME lives exactly as long as the addresses it + // points at. With no address record there is nothing to hold, and a + // zero TTL says so. + var ttl: u32 = 0; + var measure = packet.answers(p); + var seen = false; + while (measure.next() catch return null) |rec| { + if (rec.rtype != .a and rec.rtype != .aaaa) continue; + ttl = if (seen) @min(ttl, rec.ttl) else rec.ttl; + seen = true; + } + + var b = packet.ResponseBuilder.init(&ctx.scratch.synth, ctx.hdr, ctx.q) catch return null; + b.setRcode(p.header.flags.rcode); + b.addAnswer(ctx.q.name, .cname, .in, ttl, target.wire()) catch return null; + + var it = packet.answers(p); + while (it.next() catch return null) |rec| { + switch (rec.rtype) { + .a, .aaaa => b.addAnswer( + target, + rec.rtype, + .in, + rec.ttl, + rec.rdata.slice(answer), + ) catch return null, + else => {}, + } + } + if (ctx.opt) |o| b.addOptEcho(o, ctx.do_bit) catch return null; + + const message = b.finish(); + if (message.len > ctx.response_buf.len) return null; + @memcpy(ctx.response_buf[0..message.len], message); + return ctx.response_buf[0..message.len]; + } + + /// PLAN §6.3: walk the answer section's CNAME chain and evaluate every + /// target for the same group. Nothing is re-resolved — the chain is read out + /// of the answer that already arrived. The reason of the first blocked + /// target is returned; null means the chain ended, looped past + /// `max_cname_depth`, or held nothing blocked. + fn uncloak(ctx: *Context, snapshot: *const matcher.Snapshot, answer: []const u8) ?matcher.Reason { + const p = packet.parse(answer) catch return null; + + var current = ctx.q.name; + var depth: usize = 0; + while (depth < max_cname_depth) : (depth += 1) { + const next = cnameTarget(p, current) orelse return null; + const decision = snapshot.evaluate( + ctx.group, + matcher.normalize(next, &ctx.scratch.uncloak), + ); + if (decision.blocked) return decision.reason; + current = next; + } + return null; + } +}; + +/// The query as it goes upstream, and whether the cache may describe it. +const Outgoing = struct { + bytes: []const u8, + /// False when the subnet reaching the resolver is not the subnet the key + /// would state. The cache is then skipped in both directions: a hit would + /// answer from an entry built for a different subnet, and a store would + /// hand this client's answer to every other client. + cacheable: bool = true, +}; + +/// What the cache key says about the client's subnet in `.forward` mode. +const SubnetKey = union(enum) { + /// The query carries no subnet, so the key states none. + none, + subnet: []const u8, + uncacheable, +}; + +/// The subnet for the cache key, or `.uncacheable` when no single value +/// describes the query. +/// +/// RFC 7871 §6 allows one ECS option, but a query is client-controlled and can +/// carry several. The whole OPT record is forwarded in `.forward` mode, so a +/// resolver honouring the second option would answer for a subnet the key never +/// mentioned — and the entry would then serve every client whose first option +/// matched. Keying on the first option is what makes that reachable, so a +/// repeated option is not cached at all. +/// +/// A payload longer than the key holds is refused for the same reason: +/// truncating it would merge two subnets into one entry (PLAN §8). +fn subnetForKey(query: []const u8, opt: edns.OptRecord) SubnetKey { + var found: ?[]const u8 = null; + var it = edns.options(query, opt); + // `parseOpt` already walked this list, so the error is unreachable for an + // `OptRecord` this file produced; refusing the cache is the safe direction + // for one assembled by hand. + while (it.next() catch return .uncacheable) |option| { + if (option.code != edns.ecs_option_code) continue; + if (found != null) return .uncacheable; + found = option.data; + } + + const payload = found orelse return .none; + if (payload.len > dns_cache.max_ecs_len) return .uncacheable; + return .{ .subnet = payload }; +} + +/// What a query log row records beyond what the context already knows. +const LogFields = struct { + blocked: bool = false, + block_reason: []const u8 = "", + /// Null on the paths where the cache never applied: local records, forward + /// zones bypassing it, and blocked answers (ruling 20). + cache_hit: ?bool = null, + upstream: []const u8 = "", +}; + +/// The target of the answer-section CNAME owned by `owner`, or null when the +/// chain stops there. +fn cnameTarget(p: packet.Packet, owner: name.Name) ?name.Name { + var it = packet.answers(p); + while (it.next() catch return null) |rec| { + if (rec.rtype != .cname) continue; + if (!name.eqlIgnoreCase(rec.name, owner)) continue; + return record.rdataCname(p.bytes, rec) catch null; + } + return null; +} + +/// The `block_reason` column. An uncloaked block names the level that matched +/// the CNAME target, prefixed so that it is not mistaken for a decision about +/// the name the client asked for. +fn blockReason(buf: *[max_reason_len]u8, reason: matcher.Reason, uncloaked: bool) []const u8 { + const tag = @tagName(reason); + if (!uncloaked) return tag; + @memcpy(buf[0..cname_reason_prefix.len], cname_reason_prefix); + @memcpy(buf[cname_reason_prefix.len..][0..tag.len], tag); + return buf[0 .. cname_reason_prefix.len + tag.len]; +} + +/// `udp://192.168.1.1:53`, `tcp://[fd00::1]:53` — the same spelling +/// `validate.parseResolver` accepts, so a log row names the configured value. +fn resolverText(resolver: validate.Resolver, buf: *[max_resolver_text]u8) []const u8 { + var w: std.Io.Writer = .fixed(buf); + w.writeAll(switch (resolver.scheme) { + .udp => "udp://", + .tcp => "tcp://", + }) catch unreachable; + + const bracketed = switch (resolver.addr) { + .ip4 => false, + .ip6 => true, + }; + if (bracketed) w.writeByte('[') catch unreachable; + resolver.addr.format(&w) catch unreachable; + if (bracketed) w.writeByte(']') catch unreachable; + w.print(":{d}", .{resolver.port}) catch unreachable; + return w.buffered(); +} + /// Counts one synthesized reply and encodes it. The counter is passed in /// because the counter *is* the record that this failure mode happened. fn synthesize( @@ -226,8 +857,66 @@ fn bump(counter: *std.atomic.Value(u64)) void { _ = counter.fetchAdd(1, .monotonic); } +// --------------------------------------------------------------------------- +// Tests +// --------------------------------------------------------------------------- + const testing = std.testing; +/// Every test needs a real `std.Io`: the handler reads the clock on every query +/// and takes a mutex for every collaborator that is wired in. +const TestIo = struct { + threaded: std.Io.Threaded, + + fn init() TestIo { + return .{ .threaded = .init(testing.allocator, .{}) }; + } + + fn io(self: *TestIo) std.Io { + return self.threaded.io(); + } + + fn deinit(self: *TestIo) void { + self.threaded.deinit(); + } +}; + +const empty_records: records.Records = .empty; +const empty_zones: forward_zones.Zones = .empty; +const blocking: response.Options = .{ .mode = .zero, .ttl = 5 }; +const forward_timeout: std.Io.Clock.Duration = .{ .raw = .fromMilliseconds(50), .clock = .awake }; +const client_ip: address.NetAddress = .{ .ip4 = .{ 192, 168, 1, 50 } }; + +/// The handler the check command and the pre-Phase-7 listeners build: an +/// upstream and nothing else optional. +fn bare(client: transport.Client) Handler { + return .{ + .upstream = client, + .blocking = blocking, + .forward_read_timeout = forward_timeout, + .records = &empty_records, + .zones = &empty_zones, + }; +} + +/// One query with a fresh scratch. The reply is always a prefix of `buf`, so it +/// outlives the scratch this frame owns. +fn handleQuery( + h: *Handler, + io: std.Io, + which: Transport, + from: address.NetAddress, + query: []const u8, + buf: []u8, +) Handler.Outcome { + var scratch: Scratch = undefined; + return h.handle(io, which, from, query, buf, &scratch); +} + +fn udp(h: *Handler, io: std.Io, query: []const u8, buf: []u8) Handler.Outcome { + return handleQuery(h, io, .udp, client_ip, query, buf); +} + /// A query for example.com A: id 0x1234, RD set, one question, no OPT. const query_bytes = "\x12\x34\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00" ++ @@ -295,6 +984,10 @@ const FakeUpstream = struct { reply: []const u8 = &.{}, err: ?transport.ExchangeError = null, calls: usize = 0, + /// The last query the handler sent, so a test can assert what the rewrite + /// stages produced. + seen: [1024]u8 = undefined, + seen_len: usize = 0, fn exchangeFn( ptr: *anyopaque, @@ -305,6 +998,8 @@ const FakeUpstream = struct { _ = io; const self: *FakeUpstream = @ptrCast(@alignCast(ptr)); self.calls += 1; + self.seen_len = @min(query.len, self.seen.len); + @memcpy(self.seen[0..self.seen_len], query[0..self.seen_len]); if (self.err) |e| return e; if (self.reply.len > response_buf.len) return error.ResponseTooLarge; @memcpy(response_buf[0..self.reply.len], self.reply); @@ -318,6 +1013,10 @@ const FakeUpstream = struct { fn client(self: *FakeUpstream) transport.Client { return .{ .ptr = self, .exchangeFn = exchangeFn }; } + + fn sent(self: *const FakeUpstream) []const u8 { + return self.seen[0..self.seen_len]; + } }; fn expectReply(outcome: Handler.Outcome) ![]u8 { @@ -328,11 +1027,14 @@ fn expectReply(outcome: Handler.Outcome) ![]u8 { } test "a udp query is forwarded and the upstream reply is returned unchanged" { + var t: TestIo = .init(); + defer t.deinit(); + var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); var buf: [udp_limit_min]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, query_bytes, &buf)); + const reply = try expectReply(udp(&h, t.io(), query_bytes, &buf)); try testing.expectEqualSlices(u8, response_bytes, reply); try testing.expectEqual(@as(usize, 1), fake.calls); @@ -343,48 +1045,59 @@ test "a udp query is forwarded and the upstream reply is returned unchanged" { } test "a response arriving on the listener port is dropped" { + var t: TestIo = .init(); + defer t.deinit(); + var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); var buf: [udp_limit_min]u8 = undefined; - try testing.expectEqual(Handler.Outcome.drop, h.handle(undefined, .udp, response_bytes, &buf)); + try testing.expectEqual(Handler.Outcome.drop, udp(&h, t.io(), response_bytes, &buf)); try testing.expectEqual(@as(u64, 1), h.stats.dropped_malformed.load(.monotonic)); try testing.expectEqual(@as(usize, 0), fake.calls); } test "a query shorter than a header is dropped" { + var t: TestIo = .init(); + defer t.deinit(); + var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); var buf: [udp_limit_min]u8 = undefined; - const short = query_bytes[0..8]; - try testing.expectEqual(Handler.Outcome.drop, h.handle(undefined, .udp, short, &buf)); + try testing.expectEqual(Handler.Outcome.drop, udp(&h, t.io(), query_bytes[0..8], &buf)); try testing.expectEqual(@as(u64, 1), h.stats.dropped_malformed.load(.monotonic)); try testing.expectEqual(@as(usize, 0), fake.calls); } test "a query whose question runs off the end gets FORMERR" { + var t: TestIo = .init(); + defer t.deinit(); + var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); // The header promises a question; the packet ends after 12 bytes plus a // partial name, which `packet.parse` reports as a section overrun. var buf: [udp_limit_min]u8 = undefined; const overrun = query_bytes[0 .. query_bytes.len - 3]; - const outcome = h.handle(undefined, .udp, overrun, &buf); - const reply = try expectReply(outcome); + const reply = try expectReply(udp(&h, t.io(), overrun, &buf)); + const p = try packet.parse(reply); try testing.expectEqual(types.Rcode.form_err, p.header.flags.rcode); try testing.expectEqual(@as(u64, 1), h.stats.formerr.load(.monotonic)); } test "a query with QDCOUNT 0 gets FORMERR with no question" { + var t: TestIo = .init(); + defer t.deinit(); + var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); const no_question = "\x12\x34\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00"; var buf: [udp_limit_min]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, no_question, &buf)); + const reply = try expectReply(udp(&h, t.io(), no_question, &buf)); const p = try packet.parse(reply); try testing.expectEqual(types.Rcode.form_err, p.header.flags.rcode); @@ -397,15 +1110,18 @@ test "a query with QDCOUNT 0 gets FORMERR with no question" { } test "a query with QDCOUNT 2 gets FORMERR" { + var t: TestIo = .init(); + defer t.deinit(); + var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); const two = "\x12\x34\x01\x00\x00\x02\x00\x00\x00\x00\x00\x00" ++ "\x07example\x03com\x00\x00\x01\x00\x01" ++ "\x07example\x03com\x00\x00\x1c\x00\x01"; var buf: [udp_limit_min]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, two, &buf)); + const reply = try expectReply(udp(&h, t.io(), two, &buf)); const p = try packet.parse(reply); try testing.expectEqual(types.Rcode.form_err, p.header.flags.rcode); @@ -415,14 +1131,17 @@ test "a query with QDCOUNT 2 gets FORMERR" { } test "a structurally broken question gets FORMERR" { + var t: TestIo = .init(); + defer t.deinit(); + var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); // A compression pointer to itself: the name never terminates. const loop = "\x12\x34\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00" ++ "\xc0\x0c\x00\x01\x00\x01"; var buf: [udp_limit_min]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, loop, &buf)); + const reply = try expectReply(udp(&h, t.io(), loop, &buf)); const p = try packet.parse(reply); try testing.expectEqual(types.Rcode.form_err, p.header.flags.rcode); @@ -433,14 +1152,17 @@ test "a structurally broken question gets FORMERR" { } test "an unimplemented opcode gets NOTIMP" { + var t: TestIo = .init(); + defer t.deinit(); + var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); var update: [query_bytes.len]u8 = query_bytes.*; std.mem.writeInt(u16, update[2..4], 0x2900, .big); // opcode update, RD set var buf: [udp_limit_min]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, &update, &buf)); + const reply = try expectReply(udp(&h, t.io(), &update, &buf)); const p = try packet.parse(reply); try testing.expectEqual(types.Rcode.not_imp, p.header.flags.rcode); @@ -452,11 +1174,14 @@ test "an unimplemented opcode gets NOTIMP" { } test "a peer fault becomes SERVFAIL with the question echoed" { + var t: TestIo = .init(); + defer t.deinit(); + var fake: FakeUpstream = .{ .err = error.Timeout }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); var buf: [udp_limit_min]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, query_bytes, &buf)); + const reply = try expectReply(udp(&h, t.io(), query_bytes, &buf)); const p = try packet.parse(reply); try testing.expectEqual(types.Rcode.serv_fail, p.header.flags.rcode); @@ -476,11 +1201,14 @@ test "a peer fault becomes SERVFAIL with the question echoed" { } test "a local resource failure becomes SERVFAIL, not a drop" { + var t: TestIo = .init(); + defer t.deinit(); + var fake: FakeUpstream = .{ .err = error.OutOfMemory }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); var buf: [udp_limit_min]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, query_bytes, &buf)); + const reply = try expectReply(udp(&h, t.io(), query_bytes, &buf)); const p = try packet.parse(reply); try testing.expectEqual(types.Rcode.serv_fail, p.header.flags.rcode); @@ -489,11 +1217,14 @@ test "a local resource failure becomes SERVFAIL, not a drop" { } test "a canceled exchange is dropped" { + var t: TestIo = .init(); + defer t.deinit(); + var fake: FakeUpstream = .{ .err = error.Canceled }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); var buf: [udp_limit_min]u8 = undefined; - try testing.expectEqual(Handler.Outcome.drop, h.handle(undefined, .udp, query_bytes, &buf)); + try testing.expectEqual(Handler.Outcome.drop, udp(&h, t.io(), query_bytes, &buf)); try testing.expectEqual(@as(u64, 0), h.stats.servfail.load(.monotonic)); try testing.expectEqual(@as(u64, 0), h.stats.queries.load(.monotonic)); try testing.expectEqual(@as(u64, 0), h.stats.dropped_malformed.load(.monotonic)); @@ -529,15 +1260,18 @@ fn oversizeResponse(buf: []u8) []u8 { } test "an oversize udp reply is replaced by a truncated one" { + var t: TestIo = .init(); + defer t.deinit(); + var reply_buf: [2048]u8 = undefined; const oversize = oversizeResponse(&reply_buf); try testing.expect(oversize.len > udp_limit_min); var fake: FakeUpstream = .{ .reply = oversize }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); var buf: [4096]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, query_bytes, &buf)); + const reply = try expectReply(udp(&h, t.io(), query_bytes, &buf)); const p = try packet.parse(reply); try testing.expectEqual(true, p.header.flags.tc); @@ -556,14 +1290,17 @@ test "an oversize udp reply is replaced by a truncated one" { } test "the same oversize reply passes through untouched over tcp" { + var t: TestIo = .init(); + defer t.deinit(); + var reply_buf: [2048]u8 = undefined; const oversize = oversizeResponse(&reply_buf); var fake: FakeUpstream = .{ .reply = oversize }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); var buf: [4096]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .tcp, query_bytes, &buf)); + const reply = try expectReply(handleQuery(&h, t.io(), .tcp, client_ip, query_bytes, &buf)); try testing.expectEqualSlices(u8, oversize, reply); try testing.expectEqual(@as(u64, 0), h.stats.truncated.load(.monotonic)); @@ -571,31 +1308,37 @@ test "the same oversize reply passes through untouched over tcp" { } test "a reply within the advertised EDNS limit is not truncated" { + var t: TestIo = .init(); + defer t.deinit(); + var reply_buf: [2048]u8 = undefined; const oversize = oversizeResponse(&reply_buf); var fake: FakeUpstream = .{ .reply = oversize }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); var query_buf: [query_with_opt_len]u8 = undefined; const query = queryWithOpt(&query_buf, 4096, false); var buf: [4096]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, query, &buf)); + const reply = try expectReply(udp(&h, t.io(), query, &buf)); try testing.expectEqual(oversize.len, reply.len); try testing.expectEqual(@as(u64, 0), h.stats.truncated.load(.monotonic)); } test "the DO bit passes through into a synthesized reply" { + var t: TestIo = .init(); + defer t.deinit(); + for ([_]bool{ false, true }) |do_bit| { var fake: FakeUpstream = .{ .err = error.Timeout }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); var query_buf: [query_with_opt_len]u8 = undefined; const query = queryWithOpt(&query_buf, 1232, do_bit); var buf: [udp_limit_min]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, query, &buf)); + const reply = try expectReply(udp(&h, t.io(), query, &buf)); const p = try packet.parse(reply); try testing.expectEqual(types.Rcode.serv_fail, p.header.flags.rcode); @@ -606,18 +1349,21 @@ test "the DO bit passes through into a synthesized reply" { } test "the truncated reply echoes the OPT record" { + var t: TestIo = .init(); + defer t.deinit(); + var reply_buf: [2048]u8 = undefined; const oversize = oversizeResponse(&reply_buf); var fake: FakeUpstream = .{ .reply = oversize }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); // 512 is the advertised size, so the oversize answer still does not fit. var query_buf: [query_with_opt_len]u8 = undefined; const query = queryWithOpt(&query_buf, 512, true); var buf: [4096]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, query, &buf)); + const reply = try expectReply(udp(&h, t.io(), query, &buf)); const p = try packet.parse(reply); try testing.expectEqual(true, p.header.flags.tc); @@ -628,11 +1374,14 @@ test "the truncated reply echoes the OPT record" { } test "an OPT record with a non-root owner name gets FORMERR" { + var t: TestIo = .init(); + defer t.deinit(); + var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); var buf: [udp_limit_min]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, query_with_named_opt, &buf)); + const reply = try expectReply(udp(&h, t.io(), query_with_named_opt, &buf)); const p = try packet.parse(reply); try testing.expectEqual(types.Rcode.form_err, p.header.flags.rcode); @@ -652,8 +1401,11 @@ test "an OPT record with a non-root owner name gets FORMERR" { } test "an OPT record with a malformed option list gets FORMERR" { + var t: TestIo = .init(); + defer t.deinit(); + var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); // The packet as a whole is well-formed: only `parseOpt` rejects it. const p_query = try packet.parse(query_with_bad_option); @@ -663,7 +1415,7 @@ test "an OPT record with a malformed option list gets FORMERR" { ); var buf: [udp_limit_min]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, query_with_bad_option, &buf)); + const reply = try expectReply(udp(&h, t.io(), query_with_bad_option, &buf)); const p = try packet.parse(reply); try testing.expectEqual(types.Rcode.form_err, p.header.flags.rcode); @@ -674,14 +1426,17 @@ test "an OPT record with a malformed option list gets FORMERR" { } test "an OPT record whose rdata runs off the end gets FORMERR" { + var t: TestIo = .init(); + defer t.deinit(); + var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); // RDLENGTH claims three bytes and one follows, which `packet.parse` reports // as a section overrun before the OPT is ever read. const truncated = query_with_bad_option[0 .. query_with_bad_option.len - 2]; var buf: [udp_limit_min]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, truncated, &buf)); + const reply = try expectReply(udp(&h, t.io(), truncated, &buf)); const p = try packet.parse(reply); try testing.expectEqual(types.Rcode.form_err, p.header.flags.rcode); @@ -690,8 +1445,11 @@ test "an OPT record whose rdata runs off the end gets FORMERR" { } test "an OPT record in the answer section gets FORMERR" { + var t: TestIo = .init(); + defer t.deinit(); + var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); // The message walks: only its placement of the OPT is illegal, so the // handler's own section walk is what has to reject it. @@ -699,7 +1457,7 @@ test "an OPT record in the answer section gets FORMERR" { try testing.expect(packet.findOptRecord(p_query) == null); var buf: [udp_limit_min]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, query_with_opt_in_answer, &buf)); + const reply = try expectReply(udp(&h, t.io(), query_with_opt_in_answer, &buf)); const p = try packet.parse(reply); try testing.expectEqual(types.Rcode.form_err, p.header.flags.rcode); @@ -718,14 +1476,17 @@ test "an OPT record in the answer section gets FORMERR" { } test "an OPT record in the authority section gets FORMERR" { + var t: TestIo = .init(); + defer t.deinit(); + var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); const p_query = try packet.parse(query_with_opt_in_authority); try testing.expect(packet.findOptRecord(p_query) == null); var buf: [udp_limit_min]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, query_with_opt_in_authority, &buf)); + const reply = try expectReply(udp(&h, t.io(), query_with_opt_in_authority, &buf)); const p = try packet.parse(reply); try testing.expectEqual(types.Rcode.form_err, p.header.flags.rcode); @@ -738,14 +1499,17 @@ test "an OPT record in the authority section gets FORMERR" { } test "two OPT records in the additional section get FORMERR" { + var t: TestIo = .init(); + defer t.deinit(); + var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); // Here the rejection comes from `packet.parse`, one section walk earlier. try testing.expectError(error.MultipleOptRecords, packet.parse(query_with_two_opts)); var buf: [udp_limit_min]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, query_with_two_opts, &buf)); + const reply = try expectReply(udp(&h, t.io(), query_with_two_opts, &buf)); const p = try packet.parse(reply); try testing.expectEqual(types.Rcode.form_err, p.header.flags.rcode); @@ -757,14 +1521,17 @@ test "two OPT records in the additional section get FORMERR" { } test "a query with a valid OPT record is still forwarded" { + var t: TestIo = .init(); + defer t.deinit(); + var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); var query_buf: [query_with_opt_len]u8 = undefined; const query = queryWithOpt(&query_buf, 1232, true); var buf: [udp_limit_min]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, query, &buf)); + const reply = try expectReply(udp(&h, t.io(), query, &buf)); try testing.expectEqualSlices(u8, response_bytes, reply); try testing.expectEqual(@as(usize, 1), fake.calls); @@ -773,8 +1540,8 @@ test "a query with a valid OPT record is still forwarded" { // The same query, answered from the SERVFAIL path, still echoes the OPT. var fail: FakeUpstream = .{ .err = error.Timeout }; - var h2: Handler = .{ .upstream = fail.client() }; - const synthesized = try expectReply(h2.handle(undefined, .udp, query, &buf)); + var h2 = bare(fail.client()); + const synthesized = try expectReply(udp(&h2, t.io(), query, &buf)); const p = try packet.parse(synthesized); const opt = try edns.parseOpt(synthesized, packet.findOptRecord(p).?); try testing.expectEqual(true, opt.do_bit); @@ -783,6 +1550,9 @@ test "a query with a valid OPT record is still forwarded" { } test "every synthesized reply re-parses and binds to its request" { + var t: TestIo = .init(); + defer t.deinit(); + const Case = struct { query: []const u8, err: ?transport.ExchangeError, @@ -809,10 +1579,10 @@ test "every synthesized reply re-parses and binds to its request" { for (cases) |c| { var fake: FakeUpstream = .{ .reply = response_bytes, .err = c.err }; - var h: Handler = .{ .upstream = fake.client() }; + var h = bare(fake.client()); var buf: [udp_limit_min]u8 = undefined; - const reply = try expectReply(h.handle(undefined, .udp, c.query, &buf)); + const reply = try expectReply(udp(&h, t.io(), c.query, &buf)); const p = try packet.parse(reply); try testing.expectEqual(c.rcode, p.header.flags.rcode); @@ -823,3 +1593,1101 @@ test "every synthesized reply re-parses and binds to its request" { try testing.expectEqual(@as(u16, 0), p.header.nscount); } } + +// --- pipeline fixtures ----------------------------------------------------- + +const Allocator = std.mem.Allocator; + +/// A query with an arbitrary name, type and class, and no OPT record. +fn queryFor( + buf: []u8, + id: u16, + domain: []const u8, + qtype: types.Type, + qclass: types.Class, +) []const u8 { + var w: std.Io.Writer = .fixed(buf); + var encoded: [types.header_len]u8 = undefined; + header.encode(.{ + .id = id, + .flags = .{ + .rcode = .no_error, + .z = 0, + .ra = false, + .rd = true, + .tc = false, + .aa = false, + .opcode = .query, + .qr = false, + }, + .qdcount = 1, + .ancount = 0, + .nscount = 0, + .arcount = 0, + }, &encoded); + w.writeAll(&encoded) catch unreachable; + question.encode(.{ + .name = name.fromText(domain) catch unreachable, + .qtype = qtype, + .qclass = qclass, + }, &w) catch unreachable; + return w.buffered(); +} + +/// The same query with an OPT record carrying `options_bytes` verbatim. +fn queryWithOptions(buf: []u8, domain: []const u8, options_bytes: []const u8) []const u8 { + var w: std.Io.Writer = .fixed(buf); + var encoded: [types.header_len]u8 = undefined; + header.encode(.{ + .id = 0x1234, + .flags = .{ + .rcode = .no_error, + .z = 0, + .ra = false, + .rd = true, + .tc = false, + .aa = false, + .opcode = .query, + .qr = false, + }, + .qdcount = 1, + .ancount = 0, + .nscount = 0, + .arcount = 1, + }, &encoded); + w.writeAll(&encoded) catch unreachable; + question.encode(.{ + .name = name.fromText(domain) catch unreachable, + .qtype = .a, + .qclass = .in, + }, &w) catch unreachable; + edns.encodeOpt(.{ + .udp_payload_size = 1232, + .extended_rcode = 0, + .version = 0, + .do_bit = false, + .options = .{ .offset = 0, .len = 0 }, + }, options_bytes, &w) catch unreachable; + return w.buffered(); +} + +/// An EDNS Client Subnet option for `a.b.c.0/24`. +fn ecsOption(buf: *[11]u8, third: u8) []const u8 { + const bytes = [_]u8{ + 0x00, 0x08, // code 8 + 0x00, 0x07, // length + 0x00, 0x01, // family: IPv4 + 24, 0, // source prefix, scope prefix + 192, 168, + third, + }; + @memcpy(buf, &bytes); + return buf; +} + +/// A DNS cookie option (code 10), which a strip must leave alone. +const cookie_option = "\x00\x0a\x00\x08\x01\x02\x03\x04\x05\x06\x07\x08"; + +/// A response to `query` whose answer section is the CNAME chain +/// `names[0] → names[1] → …`. `names[0]` is the queried name. +fn cnameChain(buf: []u8, query: []const u8, names: []const []const u8) []u8 { + const p = packet.parse(query) catch unreachable; + const q = packet.firstQuestion(p).?; + var b = packet.ResponseBuilder.init(buf, p.header, q) catch unreachable; + var i: usize = 0; + while (i + 1 < names.len) : (i += 1) { + const owner = name.fromText(names[i]) catch unreachable; + const target = name.fromText(names[i + 1]) catch unreachable; + b.addAnswer(owner, .cname, .in, 300, target.wire()) catch unreachable; + } + return b.finish(); +} + +const SnapshotFixture = struct { + groups: []const model.Group = &.{.{ .name = "default" }}, + group_ids: []const i64 = &.{1}, + rules: []const model.Rule = &.{}, +}; + +fn buildSnapshot(gpa: Allocator, fixture: SnapshotFixture) !matcher.Snapshot { + return matcher.Snapshot.build(gpa, .{ + .groups = fixture.groups, + .group_ids = fixture.group_ids, + .group_sources = &.{}, + .sources = &.{}, + .source_ids = &.{}, + .rules = fixture.rules, + .clients = &.{}, + .prefixes = &.{}, + .compiled = &.{}, + .seed = 0x5eed, + .generation = 1, + }); +} + +/// `Manager.acquire` reads the manager's lock and its current snapshot and +/// nothing else, so a manager that publishes one hand-built snapshot needs +/// none of the database, fetcher or blocklist directory the real one owns. +fn fixtureManager(m: *manager.Manager, snapshot: *matcher.Snapshot) void { + m.* = .{ + .gpa = testing.allocator, + .database = undefined, + .paths = undefined, + .fetcher = undefined, + .update = .{}, + .total_budget = forward_timeout, + .lock = .init, + .writer_lock = .init, + .current = snapshot, + .generation = 1, + .statuses = &.{}, + .status_arena = .init(testing.allocator), + }; +} + +fn blockRule(pattern: []const u8) model.Rule { + return .{ .group = "default", .pattern = pattern, .kind = .exact, .action = .block }; +} + +fn allowRule(pattern: []const u8) model.Rule { + return .{ .group = "default", .pattern = pattern, .kind = .exact, .action = .allow }; +} + +/// The A record of the first answer, which the blocking mode `.zero` sets to +/// 0.0.0.0. +fn firstAnswer(p: packet.Packet) !record.Record { + var it = packet.answers(p); + return (try it.next()) orelse error.TestExpectedAnswer; +} + +fn drainLog(lg: *logger_mod.Logger, io: std.Io, out: []logger_mod.Entry) []logger_mod.Entry { + const n = lg.queue.getUncancelable(io, out, 0) catch 0; + return out[0..n]; +} + +// --- rate limit ------------------------------------------------------------ + +test "a client over its rate limit gets REFUSED before the query is parsed" { + var t: TestIo = .init(); + defer t.deinit(); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + + var limiter: rate_limiter.RateLimiter = try .init( + testing.allocator, + .{ .limit = 2, .window_seconds = 60 }, + ); + defer limiter.deinit(); + h.limiter = &limiter; + + var buf: [udp_limit_min]u8 = undefined; + for (0..2) |_| { + const reply = try expectReply(udp(&h, t.io(), query_bytes, &buf)); + try testing.expectEqual(types.Rcode.no_error, (try packet.parse(reply)).header.flags.rcode); + } + + const refused = try expectReply(udp(&h, t.io(), query_bytes, &buf)); + const p = try packet.parse(refused); + try testing.expectEqual(types.Rcode.refused, p.header.flags.rcode); + try testing.expectEqual(@as(u16, 0x1234), p.header.id); + try testing.expectEqual(@as(u64, 1), h.stats.refused.load(.monotonic)); + try testing.expectEqual(@as(usize, 2), fake.calls); + + // A different client has its own window. + const other: address.NetAddress = .{ .ip4 = .{ 192, 168, 1, 51 } }; + _ = try expectReply(handleQuery(&h, t.io(), .udp, other, query_bytes, &buf)); + try testing.expectEqual(@as(usize, 3), fake.calls); +} + +// --- local records --------------------------------------------------------- + +test "a local record answers authoritatively without reaching the upstream" { + var t: TestIo = .init(); + defer t.deinit(); + + var table = try records.Records.build(testing.allocator, &.{ + .{ .name = "nas.lan", .rtype = .a, .value = "192.168.1.10", .ttl = 60 }, + }); + defer table.deinit(testing.allocator); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + h.records = &table; + + var query_buf: [512]u8 = undefined; + const query = queryFor(&query_buf, 0x1234, "nas.lan", .a, .in); + + var buf: [udp_limit_min]u8 = undefined; + const reply = try expectReply(udp(&h, t.io(), query, &buf)); + + const p = try packet.parse(reply); + try testing.expectEqual(types.Rcode.no_error, p.header.flags.rcode); + try testing.expectEqual(true, p.header.flags.aa); + try testing.expectEqual(@as(u16, 1), p.header.ancount); + + const answer = try firstAnswer(p); + try testing.expectEqual(types.Type.a, answer.rtype); + try testing.expectEqual(@as(u32, 60), answer.ttl); + try testing.expectEqual([4]u8{ 192, 168, 1, 10 }, try record.rdataA(p.bytes, answer)); + + try testing.expectEqual(@as(usize, 0), fake.calls); + try testing.expectEqual(@as(u64, 1), h.stats.local_answers.load(.monotonic)); +} + +test "a local name with no record of the queried type is authoritative NODATA" { + var t: TestIo = .init(); + defer t.deinit(); + + var table = try records.Records.build(testing.allocator, &.{ + .{ .name = "nas.lan", .rtype = .a, .value = "192.168.1.10", .ttl = 60 }, + }); + defer table.deinit(testing.allocator); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + h.records = &table; + + var query_buf: [512]u8 = undefined; + const query = queryFor(&query_buf, 0x1234, "nas.lan", .aaaa, .in); + + var buf: [udp_limit_min]u8 = undefined; + const p = try packet.parse(try expectReply(udp(&h, t.io(), query, &buf))); + + try testing.expectEqual(types.Rcode.no_error, p.header.flags.rcode); + try testing.expectEqual(true, p.header.flags.aa); + try testing.expectEqual(@as(u16, 0), p.header.ancount); + try testing.expectEqual(@as(usize, 0), fake.calls); +} + +// --- forward zones --------------------------------------------------------- + +test "a forward zone answers from the cache and never reaches the pool" { + var t: TestIo = .init(); + defer t.deinit(); + + var zones = try forward_zones.Zones.build(testing.allocator, &.{ + .{ .zone = "lan.home", .resolver = "udp://127.0.0.1:1" }, + }); + defer zones.deinit(testing.allocator); + + var cache: dns_cache.DnsCache = try .init(testing.allocator, .{ .size = 8, .negative_ttl_max = 3600 }); + defer cache.deinit(); + + // The name is blocklisted, so only the zone path can answer it from the + // cache: the upstream path would have blocked it before the lookup. + var snapshot = try buildSnapshot(testing.allocator, .{ .rules = &.{blockRule("nas.lan.home")} }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + h.zones = &zones; + h.cache = &cache; + h.negative_ttl_max = 3600; + h.manager = &mgr; + + var query_buf: [512]u8 = undefined; + const query = queryFor(&query_buf, 0x1234, "nas.lan.home", .a, .in); + + // The zone resolver is unreachable, so the answer can only come from the + // entry planted under the key the handler builds. + var zone_reply_buf: [512]u8 = undefined; + const zone_p = try packet.parse(query); + var b = try packet.ResponseBuilder.init(&zone_reply_buf, zone_p.header, packet.firstQuestion(zone_p).?); + try b.addAnswer(try name.fromText("nas.lan.home"), .a, .in, 300, "\xc0\xa8\x01\x0a"); + const zone_reply = b.finish(); + + var key_buf: [dns_cache.max_key_len]u8 = undefined; + const key = dns_cache.buildKey( + &key_buf, + "nas.lan.home", + @intFromEnum(types.Type.a), + @intFromEnum(types.Class.in), + false, + null, + ); + try cache.put(std.Io.Clock.real.now(t.io()).toSeconds(), key, zone_reply, .{ .ttl_seconds = 300, .negative = false }); + + var buf: [udp_limit_min]u8 = undefined; + const p = try packet.parse(try expectReply(udp(&h, t.io(), query, &buf))); + + try testing.expectEqual(@as(u16, 1), p.header.ancount); + try testing.expectEqual([4]u8{ 192, 168, 1, 10 }, try record.rdataA(p.bytes, try firstAnswer(p))); + try testing.expectEqual(@as(usize, 0), fake.calls); + try testing.expectEqual(@as(u64, 1), h.stats.cache_hits.load(.monotonic)); + try testing.expectEqual(@as(u64, 0), h.stats.blocked.load(.monotonic)); +} + +test "a forward zone bypasses the blocklist and fails on its own resolver" { + var t: TestIo = .init(); + defer t.deinit(); + + var snapshot = try buildSnapshot(testing.allocator, .{ + .rules = &.{blockRule("ads.lan.home")}, + }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + var zones = try forward_zones.Zones.build(testing.allocator, &.{ + .{ .zone = "lan.home", .resolver = "udp://127.0.0.1:1" }, + }); + defer zones.deinit(testing.allocator); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + h.zones = &zones; + h.manager = &mgr; + + var query_buf: [512]u8 = undefined; + const query = queryFor(&query_buf, 0x1234, "ads.lan.home", .a, .in); + + var buf: [udp_limit_min]u8 = undefined; + const p = try packet.parse(try expectReply(udp(&h, t.io(), query, &buf))); + + // A blocked answer would be NOERROR with an address, so SERVFAIL is proof + // the query went to the zone resolver instead of the blocklist. + try testing.expectEqual(types.Rcode.serv_fail, p.header.flags.rcode); + try testing.expectEqual(@as(u64, 0), h.stats.blocked.load(.monotonic)); + try testing.expectEqual(@as(usize, 0), fake.calls); +} + +// --- filtering ------------------------------------------------------------- + +test "a blocked domain gets the zero address with the blocking ttl" { + var t: TestIo = .init(); + defer t.deinit(); + + var snapshot = try buildSnapshot(testing.allocator, .{ .rules = &.{blockRule("example.com")} }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + h.manager = &mgr; + + var buf: [udp_limit_min]u8 = undefined; + const p = try packet.parse(try expectReply(udp(&h, t.io(), query_bytes, &buf))); + + try testing.expectEqual(types.Rcode.no_error, p.header.flags.rcode); + try testing.expectEqual(@as(u16, 1), p.header.ancount); + + const answer = try firstAnswer(p); + try testing.expectEqual(@as(u32, blocking.ttl), answer.ttl); + try testing.expectEqual([4]u8{ 0, 0, 0, 0 }, try record.rdataA(p.bytes, answer)); + + try testing.expectEqual(@as(usize, 0), fake.calls); + try testing.expectEqual(@as(u64, 1), h.stats.blocked.load(.monotonic)); + try testing.expectEqual(@as(u64, 0), h.stats.unfiltered_queries.load(.monotonic)); +} + +test "an allow rule beats the block rule and the query reaches the upstream" { + var t: TestIo = .init(); + defer t.deinit(); + + var snapshot = try buildSnapshot(testing.allocator, .{ + .rules = &.{ blockRule("com"), allowRule("example.com") }, + }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + h.manager = &mgr; + + var buf: [udp_limit_min]u8 = undefined; + const reply = try expectReply(udp(&h, t.io(), query_bytes, &buf)); + + try testing.expectEqualSlices(u8, response_bytes, reply); + try testing.expectEqual(@as(usize, 1), fake.calls); + try testing.expectEqual(@as(u64, 0), h.stats.blocked.load(.monotonic)); +} + +test "a handler with no snapshot answers unfiltered and counts it" { + var t: TestIo = .init(); + defer t.deinit(); + + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, undefined); + mgr.current = null; + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + h.manager = &mgr; + + var buf: [udp_limit_min]u8 = undefined; + const reply = try expectReply(udp(&h, t.io(), query_bytes, &buf)); + + try testing.expectEqualSlices(u8, response_bytes, reply); + try testing.expectEqual(@as(u64, 1), h.stats.unfiltered_queries.load(.monotonic)); +} + +// --- cache ----------------------------------------------------------------- + +test "a miss stores the answer and the next query is a hit with a fresh id" { + var t: TestIo = .init(); + defer t.deinit(); + + var cache: dns_cache.DnsCache = try .init(testing.allocator, .{ .size = 8, .negative_ttl_max = 3600 }); + defer cache.deinit(); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + h.cache = &cache; + h.negative_ttl_max = 3600; + + var buf: [udp_limit_min]u8 = undefined; + _ = try expectReply(udp(&h, t.io(), query_bytes, &buf)); + try testing.expectEqual(@as(u32, 1), cache.len()); + try testing.expectEqual(@as(usize, 1), fake.calls); + + var query_buf: [512]u8 = undefined; + const again = queryFor(&query_buf, 0x5678, "example.com", .a, .in); + const p = try packet.parse(try expectReply(udp(&h, t.io(), again, &buf))); + + try testing.expectEqual(@as(u16, 0x5678), p.header.id); + try testing.expectEqual(@as(u16, 1), p.header.ancount); + try testing.expectEqual(@as(usize, 1), fake.calls); + try testing.expectEqual(@as(u64, 1), h.stats.cache_hits.load(.monotonic)); +} + +test "a blocked answer is never cached" { + var t: TestIo = .init(); + defer t.deinit(); + + var snapshot = try buildSnapshot(testing.allocator, .{ .rules = &.{blockRule("example.com")} }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + var cache: dns_cache.DnsCache = try .init(testing.allocator, .{ .size = 8, .negative_ttl_max = 3600 }); + defer cache.deinit(); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + h.manager = &mgr; + h.cache = &cache; + h.negative_ttl_max = 3600; + + var buf: [udp_limit_min]u8 = undefined; + _ = try expectReply(udp(&h, t.io(), query_bytes, &buf)); + + try testing.expectEqual(@as(u32, 0), cache.len()); + try testing.expectEqual(@as(u64, 1), h.stats.blocked.load(.monotonic)); +} + +// --- CNAME uncloaking ------------------------------------------------------ + +test "a CNAME chain into a blocked target blocks the original question" { + var t: TestIo = .init(); + defer t.deinit(); + + var snapshot = try buildSnapshot(testing.allocator, .{ + .rules = &.{blockRule("bad.example.org")}, + }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + var chain_buf: [512]u8 = undefined; + const chain = cnameChain(&chain_buf, query_bytes, &.{ "example.com", "bad.example.org" }); + + var fake: FakeUpstream = .{ .reply = chain }; + var h = bare(fake.client()); + h.manager = &mgr; + + var buf: [udp_limit_min]u8 = undefined; + const p = try packet.parse(try expectReply(udp(&h, t.io(), query_bytes, &buf))); + + try testing.expectEqual(types.Rcode.no_error, p.header.flags.rcode); + try testing.expectEqual(@as(u16, 1), p.header.ancount); + + // The answer is about the name the client asked for, not the target. + const answer = try firstAnswer(p); + try testing.expectEqual(types.Type.a, answer.rtype); + try testing.expectEqualSlices( + u8, + (try name.fromText("example.com")).wire(), + answer.name.wire(), + ); + try testing.expectEqual([4]u8{ 0, 0, 0, 0 }, try record.rdataA(p.bytes, answer)); + try testing.expectEqual(@as(u64, 1), h.stats.uncloak_blocked.load(.monotonic)); + try testing.expectEqual(@as(u64, 0), h.stats.blocked.load(.monotonic)); +} + +test "uncloaking follows eight links and stops" { + var t: TestIo = .init(); + defer t.deinit(); + + var snapshot = try buildSnapshot(testing.allocator, .{ + .rules = &.{blockRule("bad.example.org")}, + }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + const hops = [_][]const u8{ + "h1.example.net", "h2.example.net", "h3.example.net", "h4.example.net", + "h5.example.net", "h6.example.net", "h7.example.net", "h8.example.net", + }; + + // `links` CNAME records, the last of which points at the blocked name. + for ([_]usize{ 8, 9 }) |links| { + var names: [11][]const u8 = undefined; + names[0] = "example.com"; + for (0..links - 1) |i| names[i + 1] = hops[i]; + names[links] = "bad.example.org"; + + var chain_buf: [1024]u8 = undefined; + const chain = cnameChain(&chain_buf, query_bytes, names[0 .. links + 1]); + + var fake: FakeUpstream = .{ .reply = chain }; + var h = bare(fake.client()); + h.manager = &mgr; + + var buf: [udp_limit_min]u8 = undefined; + const reply = try expectReply(udp(&h, t.io(), query_bytes, &buf)); + + if (links == max_cname_depth) { + try testing.expectEqual(@as(u64, 1), h.stats.uncloak_blocked.load(.monotonic)); + } else { + try testing.expectEqual(@as(u64, 0), h.stats.uncloak_blocked.load(.monotonic)); + try testing.expectEqualSlices(u8, chain, reply); + } + } +} + +// --- safe search ----------------------------------------------------------- + +test "safe search sends the target upstream and answers the original question" { + var t: TestIo = .init(); + defer t.deinit(); + + var snapshot = try buildSnapshot(testing.allocator, .{ + .groups = &.{.{ .name = "default", .safe_search = true }}, + }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + // The upstream answers the rewritten question with one address and one + // record of another type, which must not be copied over. + var target_buf: [512]u8 = undefined; + const target_query = queryFor(&target_buf, 0x1234, "forcesafesearch.google.com", .a, .in); + var upstream_buf: [512]u8 = undefined; + const target_p = try packet.parse(target_query); + var b = try packet.ResponseBuilder.init( + &upstream_buf, + target_p.header, + packet.firstQuestion(target_p).?, + ); + const target_name = try name.fromText("forcesafesearch.google.com"); + try b.addAnswer(target_name, .a, .in, 120, "\x08\x08\x08\x08"); + try b.addAnswer(target_name, .txt, .in, 60, "\x03abc"); + + var fake: FakeUpstream = .{ .reply = b.finish() }; + var h = bare(fake.client()); + h.manager = &mgr; + + var query_buf: [512]u8 = undefined; + const query = queryFor(&query_buf, 0x1234, "www.google.com", .a, .in); + + var buf: [udp_limit_min]u8 = undefined; + const p = try packet.parse(try expectReply(udp(&h, t.io(), query, &buf))); + + // What went out carried the target name. + const sent = try packet.parse(fake.sent()); + try testing.expectEqualSlices( + u8, + target_name.wire(), + packet.firstQuestion(sent).?.name.wire(), + ); + + // What came back keeps the original question. + try testing.expectEqualSlices( + u8, + (try name.fromText("www.google.com")).wire(), + packet.firstQuestion(p).?.name.wire(), + ); + try testing.expectEqual(@as(u16, 2), p.header.ancount); + + var it = packet.answers(p); + const cname = (try it.next()).?; + try testing.expectEqual(types.Type.cname, cname.rtype); + try testing.expectEqual(@as(u32, 120), cname.ttl); + try testing.expectEqualSlices(u8, target_name.wire(), (try record.rdataCname(p.bytes, cname)).wire()); + + const a = (try it.next()).?; + try testing.expectEqual(types.Type.a, a.rtype); + try testing.expectEqualSlices(u8, target_name.wire(), a.name.wire()); + try testing.expectEqual([4]u8{ 8, 8, 8, 8 }, try record.rdataA(p.bytes, a)); + try testing.expect((try it.next()) == null); + + try testing.expectEqual(@as(u64, 1), h.stats.safesearch_rewrites.load(.monotonic)); +} + +// --- ECS ------------------------------------------------------------------- + +test "ecs strip removes option 8 from the outgoing query and keeps the rest" { + var t: TestIo = .init(); + defer t.deinit(); + + var ecs_buf: [11]u8 = undefined; + var options_buf: [64]u8 = undefined; + const ecs = ecsOption(&ecs_buf, 1); + @memcpy(options_buf[0..cookie_option.len], cookie_option); + @memcpy(options_buf[cookie_option.len..][0..ecs.len], ecs); + const options_bytes = options_buf[0 .. cookie_option.len + ecs.len]; + + var query_buf: [512]u8 = undefined; + const query = queryWithOptions(&query_buf, "example.com", options_bytes); + + // The query really carries a subnet, so the assertions below measure a + // strip rather than an absence. + const asked = try packet.parse(query); + const asked_opt = try edns.parseOpt(query, packet.findOptRecord(asked).?); + try testing.expect((try edns.findOption(query, asked_opt, edns.ecs_option_code)) != null); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + + var buf: [udp_limit_min]u8 = undefined; + _ = try expectReply(udp(&h, t.io(), query, &buf)); + + const sent = try packet.parse(fake.sent()); + const opt = try edns.parseOpt(fake.sent(), packet.findOptRecord(sent).?); + try testing.expect((try edns.findOption(fake.sent(), opt, edns.ecs_option_code)) == null); + + const cookie = (try edns.findOption(fake.sent(), opt, 10)).?; + try testing.expectEqualSlices(u8, cookie_option[4..], cookie.data); + try testing.expectEqual(@as(u64, 0), h.stats.ecs_strip_failed.load(.monotonic)); +} + +test "ecs forward keeps the subnet in the query and splits the cache key" { + var t: TestIo = .init(); + defer t.deinit(); + + var cache: dns_cache.DnsCache = try .init(testing.allocator, .{ .size = 8, .negative_ttl_max = 3600 }); + defer cache.deinit(); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + h.ecs_mode = .forward; + h.cache = &cache; + h.negative_ttl_max = 3600; + + var buf: [udp_limit_min]u8 = undefined; + for ([_]u8{ 1, 2 }) |third| { + var ecs_buf: [11]u8 = undefined; + var query_buf: [512]u8 = undefined; + const query = queryWithOptions(&query_buf, "example.com", ecsOption(&ecs_buf, third)); + _ = try expectReply(udp(&h, t.io(), query, &buf)); + + // `.forward` sends the query unchanged, subnet included. + const sent = try packet.parse(fake.sent()); + const opt = try edns.parseOpt(fake.sent(), packet.findOptRecord(sent).?); + try testing.expect((try edns.findOption(fake.sent(), opt, edns.ecs_option_code)) != null); + } + + // Two subnets, two entries, two upstream exchanges. + try testing.expectEqual(@as(u32, 2), cache.len()); + try testing.expectEqual(@as(usize, 2), fake.calls); + + var ecs_buf: [11]u8 = undefined; + var query_buf: [512]u8 = undefined; + const repeat = queryWithOptions(&query_buf, "example.com", ecsOption(&ecs_buf, 1)); + _ = try expectReply(udp(&h, t.io(), repeat, &buf)); + try testing.expectEqual(@as(usize, 2), fake.calls); + try testing.expectEqual(@as(u64, 1), h.stats.cache_hits.load(.monotonic)); +} + +// --- pause, class and tracking --------------------------------------------- + +test "a paused handler answers a blocked domain from the upstream" { + var t: TestIo = .init(); + defer t.deinit(); + + var snapshot = try buildSnapshot(testing.allocator, .{ .rules = &.{blockRule("example.com")} }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + var paused: pause.Pause = .{}; + paused.pauseFor(0, null); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + h.manager = &mgr; + h.pause = &paused; + + var buf: [udp_limit_min]u8 = undefined; + try testing.expectEqualSlices(u8, response_bytes, try expectReply(udp(&h, t.io(), query_bytes, &buf))); + try testing.expectEqual(@as(u64, 1), h.stats.paused_queries.load(.monotonic)); + try testing.expectEqual(@as(u64, 0), h.stats.blocked.load(.monotonic)); + + // Resuming puts the block back without a restart. + paused.unpause(); + const p = try packet.parse(try expectReply(udp(&h, t.io(), query_bytes, &buf))); + try testing.expectEqual([4]u8{ 0, 0, 0, 0 }, try record.rdataA(p.bytes, try firstAnswer(p))); + try testing.expectEqual(@as(u64, 1), h.stats.blocked.load(.monotonic)); +} + +test "a non-IN class query skips filtering and the cache" { + var t: TestIo = .init(); + defer t.deinit(); + + var snapshot = try buildSnapshot(testing.allocator, .{ .rules = &.{blockRule("example.com")} }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + var cache: dns_cache.DnsCache = try .init(testing.allocator, .{ .size = 8, .negative_ttl_max = 3600 }); + defer cache.deinit(); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + h.manager = &mgr; + h.cache = &cache; + h.negative_ttl_max = 3600; + + var query_buf: [512]u8 = undefined; + const query = queryFor(&query_buf, 0x1234, "example.com", .txt, .ch); + + var buf: [udp_limit_min]u8 = undefined; + _ = try expectReply(udp(&h, t.io(), query, &buf)); + + try testing.expectEqual(@as(usize, 1), fake.calls); + try testing.expectEqual(@as(u64, 0), h.stats.blocked.load(.monotonic)); + try testing.expectEqual(@as(u32, 0), cache.len()); +} + +test "the tracker sees the client address of every parsed query" { + var t: TestIo = .init(); + defer t.deinit(); + + var tracker: clients.Tracker = .init(30); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + h.tracker = &tracker; + + var buf: [udp_limit_min]u8 = undefined; + _ = try expectReply(udp(&h, t.io(), query_bytes, &buf)); + _ = try expectReply(udp(&h, t.io(), query_bytes, &buf)); + + try testing.expectEqual(@as(u32, 1), tracker.pendingClients(t.io())); + try testing.expectEqual(@as(u64, 2), tracker.snapshotStats(t.io()).tracked); + try testing.expectEqual(@as(u64, 0), h.stats.tracker_full.load(.monotonic)); +} + +// --- query log ------------------------------------------------------------- + +test "every answered path logs the fields ruling 20 defines" { + var t: TestIo = .init(); + defer t.deinit(); + const io = t.io(); + + var snapshot = try buildSnapshot(testing.allocator, .{ .rules = &.{blockRule("blocked.example.com")} }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + var table = try records.Records.build(testing.allocator, &.{ + .{ .name = "nas.lan", .rtype = .a, .value = "192.168.1.10", .ttl = 60 }, + }); + defer table.deinit(testing.allocator); + + var cache: dns_cache.DnsCache = try .init(testing.allocator, .{ .size = 8, .negative_ttl_max = 3600 }); + defer cache.deinit(); + + var queue_buf: [8]logger_mod.Entry = undefined; + var lg: logger_mod.Logger = .init(.{}, &queue_buf); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + h.manager = &mgr; + h.records = &table; + h.cache = &cache; + h.negative_ttl_max = 3600; + h.logger = ≶ + + var buf: [udp_limit_min]u8 = undefined; + var query_buf: [512]u8 = undefined; + + // Local, blocked, upstream, then the same name again for a cache hit. + _ = try expectReply(udp(&h, io, queryFor(&query_buf, 1, "nas.lan", .a, .in), &buf)); + var blocked_buf: [512]u8 = undefined; + _ = try expectReply(udp(&h, io, queryFor(&blocked_buf, 2, "blocked.example.com", .a, .in), &buf)); + _ = try expectReply(udp(&h, io, query_bytes, &buf)); + _ = try expectReply(udp(&h, io, query_bytes, &buf)); + + var entries: [8]logger_mod.Entry = undefined; + const logged = drainLog(&lg, io, &entries); + try testing.expectEqual(@as(usize, 4), logged.len); + + try testing.expectEqualStrings("nas.lan", logged[0].domain()); + try testing.expectEqualStrings("192.168.1.50", logged[0].clientIp()); + try testing.expectEqualStrings("local", logged[0].upstream()); + try testing.expectEqual(@as(?bool, null), logged[0].cache_hit); + try testing.expectEqual(false, logged[0].blocked); + try testing.expectEqual(@as(?u16, @intFromEnum(types.Type.a)), logged[0].qtype); + try testing.expect(logged[0].response_time_us.? >= 0); + + try testing.expectEqualStrings("blocked.example.com", logged[1].domain()); + try testing.expectEqual(true, logged[1].blocked); + try testing.expectEqualStrings("rule_block_exact", logged[1].blockReason()); + try testing.expectEqualStrings("", logged[1].upstream()); + try testing.expectEqual(@as(?bool, null), logged[1].cache_hit); + + try testing.expectEqualStrings("example.com", logged[2].domain()); + try testing.expectEqualStrings("pool", logged[2].upstream()); + try testing.expectEqual(@as(?bool, false), logged[2].cache_hit); + + try testing.expectEqual(@as(?bool, true), logged[3].cache_hit); + try testing.expectEqualStrings("", logged[3].upstream()); +} + +test "an uncloaked block logs the cname-prefixed reason" { + var t: TestIo = .init(); + defer t.deinit(); + + var snapshot = try buildSnapshot(testing.allocator, .{ .rules = &.{blockRule("bad.example.org")} }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + var chain_buf: [512]u8 = undefined; + const chain = cnameChain(&chain_buf, query_bytes, &.{ "example.com", "bad.example.org" }); + + var queue_buf: [4]logger_mod.Entry = undefined; + var lg: logger_mod.Logger = .init(.{}, &queue_buf); + + var fake: FakeUpstream = .{ .reply = chain }; + var h = bare(fake.client()); + h.manager = &mgr; + h.logger = ≶ + + var buf: [udp_limit_min]u8 = undefined; + _ = try expectReply(udp(&h, t.io(), query_bytes, &buf)); + + var entries: [4]logger_mod.Entry = undefined; + const logged = drainLog(&lg, t.io(), &entries); + try testing.expectEqual(@as(usize, 1), logged.len); + try testing.expectEqual(true, logged[0].blocked); + try testing.expectEqualStrings("cname:rule_block_exact", logged[0].blockReason()); + // The row names the question the client asked, not the target. + try testing.expectEqualStrings("example.com", logged[0].domain()); +} + +test "a refused query is counted and never logged" { + var t: TestIo = .init(); + defer t.deinit(); + + var limiter: rate_limiter.RateLimiter = try .init( + testing.allocator, + .{ .limit = 1, .window_seconds = 60 }, + ); + defer limiter.deinit(); + + var queue_buf: [4]logger_mod.Entry = undefined; + var lg: logger_mod.Logger = .init(.{}, &queue_buf); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + h.limiter = &limiter; + h.logger = ≶ + + var buf: [udp_limit_min]u8 = undefined; + _ = try expectReply(udp(&h, t.io(), query_bytes, &buf)); + _ = try expectReply(udp(&h, t.io(), query_bytes, &buf)); + + var entries: [4]logger_mod.Entry = undefined; + const logged = drainLog(&lg, t.io(), &entries); + try testing.expectEqual(@as(usize, 1), logged.len); + try testing.expectEqual(@as(u64, 1), h.stats.refused.load(.monotonic)); +} + +test "safe search and an ecs strip compose into one outgoing query" { + var t: TestIo = .init(); + defer t.deinit(); + + var snapshot = try buildSnapshot(testing.allocator, .{ + .groups = &.{.{ .name = "default", .safe_search = true }}, + }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + var ecs_buf: [11]u8 = undefined; + var options_buf: [64]u8 = undefined; + const ecs = ecsOption(&ecs_buf, 1); + @memcpy(options_buf[0..cookie_option.len], cookie_option); + @memcpy(options_buf[cookie_option.len..][0..ecs.len], ecs); + + var query_buf: [512]u8 = undefined; + const query = queryWithOptions( + &query_buf, + "www.google.com", + options_buf[0 .. cookie_option.len + ecs.len], + ); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + h.manager = &mgr; + + var buf: [udp_limit_min]u8 = undefined; + _ = try expectReply(udp(&h, t.io(), query, &buf)); + + const sent = try packet.parse(fake.sent()); + try testing.expectEqualSlices( + u8, + (try name.fromText("forcesafesearch.google.com")).wire(), + packet.firstQuestion(sent).?.name.wire(), + ); + + const opt = try edns.parseOpt(fake.sent(), packet.findOptRecord(sent).?); + try testing.expect((try edns.findOption(fake.sent(), opt, edns.ecs_option_code)) == null); + try testing.expect((try edns.findOption(fake.sent(), opt, 10)) != null); + try testing.expectEqual(@as(u64, 1), h.stats.safesearch_rewrites.load(.monotonic)); +} + +/// A query whose OPT record is too large for the 512-byte build buffer to +/// rebuild, carrying a subnet behind an RFC 7830 padding option. +fn paddedEcsQuery(options_buf: *[1024]u8, query_buf: *[1024]u8) []const u8 { + const padding_len = 600; + std.mem.writeInt(u16, options_buf[0..2], 12, .big); + std.mem.writeInt(u16, options_buf[2..4], padding_len, .big); + @memset(options_buf[4..][0..padding_len], 0); + + var ecs_buf: [11]u8 = undefined; + const ecs = ecsOption(&ecs_buf, 1); + @memcpy(options_buf[4 + padding_len ..][0..ecs.len], ecs); + return queryWithOptions(query_buf, "example.com", options_buf[0 .. 4 + padding_len + ecs.len]); +} + +test "a query too large to rebuild keeps its subnet and is counted" { + var t: TestIo = .init(); + defer t.deinit(); + + var options_buf: [1024]u8 = undefined; + var query_buf: [1024]u8 = undefined; + const query = paddedEcsQuery(&options_buf, &query_buf); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + + var buf: [udp_limit_min]u8 = undefined; + _ = try expectReply(udp(&h, t.io(), query, &buf)); + + // Ruling 13: the original goes out unchanged rather than the query failing. + try testing.expectEqualSlices(u8, query, fake.sent()); + try testing.expectEqual(@as(u64, 1), h.stats.ecs_strip_failed.load(.monotonic)); +} + +test "a failed ecs strip keeps the query out of the cache in both directions" { + var t: TestIo = .init(); + defer t.deinit(); + + var cache: dns_cache.DnsCache = try .init(testing.allocator, .{ .size = 8, .negative_ttl_max = 3600 }); + defer cache.deinit(); + + // An entry stored under the subnet-free key every other client shares. + var planted_buf: [512]u8 = undefined; + const planted_p = try packet.parse(query_bytes); + var b = try packet.ResponseBuilder.init( + &planted_buf, + planted_p.header, + packet.firstQuestion(planted_p).?, + ); + try b.addAnswer(try name.fromText("example.com"), .a, .in, 300, "\x09\x09\x09\x09"); + const planted = b.finish(); + + var key_buf: [dns_cache.max_key_len]u8 = undefined; + const global_key = dns_cache.buildKey( + &key_buf, + "example.com", + @intFromEnum(types.Type.a), + @intFromEnum(types.Class.in), + false, + null, + ); + try cache.put( + std.Io.Clock.real.now(t.io()).toSeconds(), + global_key, + planted, + .{ .ttl_seconds = 300, .negative = false }, + ); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + h.cache = &cache; + h.negative_ttl_max = 3600; + + var options_buf: [1024]u8 = undefined; + var query_buf: [1024]u8 = undefined; + const query = paddedEcsQuery(&options_buf, &query_buf); + + var buf: [udp_limit_min]u8 = undefined; + const p = try packet.parse(try expectReply(udp(&h, t.io(), query, &buf))); + + // The subnet-free entry must not answer a query whose subnet went upstream. + try testing.expectEqual(@as(u64, 1), h.stats.ecs_strip_failed.load(.monotonic)); + try testing.expectEqual(@as(u64, 0), h.stats.cache_hits.load(.monotonic)); + try testing.expectEqual(@as(usize, 1), fake.calls); + try testing.expectEqual([4]u8{ 93, 184, 216, 34 }, try record.rdataA(p.bytes, try firstAnswer(p))); + + // Nor may the subnet-specific answer be stored where they would find it. + try testing.expectEqual(@as(u32, 1), cache.len()); + _ = try expectReply(udp(&h, t.io(), query, &buf)); + try testing.expectEqual(@as(usize, 2), fake.calls); + try testing.expectEqual(@as(u32, 1), cache.len()); +} + +test "forward mode refuses to cache a query carrying two subnets" { + var t: TestIo = .init(); + defer t.deinit(); + + var cache: dns_cache.DnsCache = try .init(testing.allocator, .{ .size = 8, .negative_ttl_max = 3600 }); + defer cache.deinit(); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bare(fake.client()); + h.ecs_mode = .forward; + h.cache = &cache; + h.negative_ttl_max = 3600; + + // Two ECS options: the key could only name one of the two subnets, and the + // resolver is free to honour the other. + var first_buf: [11]u8 = undefined; + var second_buf: [11]u8 = undefined; + var options_buf: [64]u8 = undefined; + const first = ecsOption(&first_buf, 1); + const second = ecsOption(&second_buf, 2); + @memcpy(options_buf[0..first.len], first); + @memcpy(options_buf[first.len..][0..second.len], second); + + var query_buf: [512]u8 = undefined; + const query = queryWithOptions(&query_buf, "example.com", options_buf[0 .. first.len + second.len]); + + var buf: [udp_limit_min]u8 = undefined; + _ = try expectReply(udp(&h, t.io(), query, &buf)); + try testing.expectEqual(@as(u32, 0), cache.len()); + try testing.expectEqual(@as(usize, 1), fake.calls); + + // Repeating it finds nothing to hit, in either subnet's key or the global one. + _ = try expectReply(udp(&h, t.io(), query, &buf)); + try testing.expectEqual(@as(usize, 2), fake.calls); + try testing.expectEqual(@as(u64, 0), h.stats.cache_hits.load(.monotonic)); + + // One subnet is still cached normally. + var single_buf: [512]u8 = undefined; + const single = queryWithOptions(&single_buf, "example.com", first); + _ = try expectReply(udp(&h, t.io(), single, &buf)); + try testing.expectEqual(@as(u32, 1), cache.len()); + _ = try expectReply(udp(&h, t.io(), single, &buf)); + try testing.expectEqual(@as(u64, 1), h.stats.cache_hits.load(.monotonic)); +} diff --git a/src/server/pause.zig b/src/server/pause.zig new file mode 100644 index 0000000..8a18364 --- /dev/null +++ b/src/server/pause.zig @@ -0,0 +1,93 @@ +//! Global pause of filtering (PLAN §13.1). One 64-bit flag the query path +//! reads and the API writes. Pure: no allocation, no `std.Io`, no clock — the +//! caller supplies the time, because the handler already has it. +//! +//! Pausing suspends FILTERING only: local records, forward zones, the cache, +//! the upstream and the query log all keep running (milestone-7 ruling 18). +//! +//! The state is in memory and is deliberately not persisted: a restart resumes +//! filtering, which is the safe default for a household. + +const std = @import("std"); + +pub const Pause = struct { + /// 0 = filtering active. -1 = paused until someone resumes. Any other value + /// is the unix second filtering resumes at, so the expiry is a comparison + /// on the query path rather than a timer task. + until: std.atomic.Value(i64) = .init(0), + + /// `.monotonic` throughout: the flag guards no other data, so nothing has + /// to be ordered against it. + pub fn isPaused(self: *const Pause, now_s: i64) bool { + const until = self.until.load(.monotonic); + if (until == 0) return false; + if (until < 0) return true; + return now_s < until; + } + + /// `null` pauses until `unpause`. A pause already in force is replaced, so + /// the last call wins whether it lengthens or shortens the pause. + pub fn pauseFor(self: *Pause, now_s: i64, duration_s: ?u32) void { + const value: i64 = if (duration_s) |seconds| now_s +| @as(i64, seconds) else -1; + self.until.store(value, .monotonic); + } + + pub fn unpause(self: *Pause) void { + self.until.store(0, .monotonic); + } +}; + +const testing = std.testing; + +test "a fresh pause is not paused" { + const p: Pause = .{}; + try testing.expect(!p.isPaused(0)); + try testing.expect(!p.isPaused(1_700_000_000)); +} + +test "an indefinite pause holds at every time" { + var p: Pause = .{}; + p.pauseFor(1_700_000_000, null); + try testing.expect(p.isPaused(1_700_000_000)); + try testing.expect(p.isPaused(1_700_000_000 + 86_400 * 365)); + try testing.expectEqual(@as(i64, -1), p.until.load(.monotonic)); +} + +test "a timed pause expires at its own second" { + var p: Pause = .{}; + p.pauseFor(1_000, 60); + try testing.expect(p.isPaused(1_000)); + try testing.expect(p.isPaused(1_059)); + // The stored second is when filtering is back on, so it is not paused. + try testing.expect(!p.isPaused(1_060)); + try testing.expect(!p.isPaused(1_061)); +} + +test "unpause resumes both kinds of pause" { + var p: Pause = .{}; + p.pauseFor(1_000, null); + p.unpause(); + try testing.expect(!p.isPaused(1_000)); + + p.pauseFor(1_000, 60); + p.unpause(); + try testing.expect(!p.isPaused(1_000)); +} + +test "pauseFor overwrites a pause already in force" { + var p: Pause = .{}; + p.pauseFor(1_000, 3_600); + p.pauseFor(1_000, 10); + try testing.expect(!p.isPaused(1_010)); + + // And in the other direction: indefinite replaces a timed pause. + p.pauseFor(1_000, 10); + p.pauseFor(1_000, null); + try testing.expect(p.isPaused(1_010)); +} + +test "a duration that would overflow saturates instead of wrapping" { + var p: Pause = .{}; + p.pauseFor(std.math.maxInt(i64), std.math.maxInt(u32)); + try testing.expect(p.isPaused(std.math.maxInt(i64) - 1)); +} diff --git a/src/server/phase7_integration_test.zig b/src/server/phase7_integration_test.zig new file mode 100644 index 0000000..b4fe2a8 --- /dev/null +++ b/src/server/phase7_integration_test.zig @@ -0,0 +1,996 @@ +//! Milestone-7 integration tests (spec S7): the serving pipeline end to end, +//! over real sockets. +//! +//! This lives in its own file because it needs `@import("build_options")`, which +//! only exists when the compilation is driven by `build.zig`. The body compiles +//! on every `zig build test` run, so it cannot rot, and every case skips at run +//! time unless `-Dintegration` is passed. +//! +//! What separates these cases from `handler.zig`'s own tests is the socket. The +//! handler tests call `handle` directly; here every query travels through a real +//! `UdpServer` on 127.0.0.1, through the real handler with its real cache, +//! limiter, tracker and query log, and the reply is read back off the wire. The +//! upstream is a `transport.Client` fixture, except in the forward-zone case, +//! where the zone resolver has to be a real UDP socket because `ForwardClient` +//! speaks wire DNS to an address. +//! +//! Hermetic: every socket is bound to 127.0.0.1, every database is in memory or +//! inside a `std.testing.tmpDir`, and every wait carries a budget. + +const std = @import("std"); +const build_options = @import("build_options"); +const net = std.Io.net; +const Allocator = std.mem.Allocator; + +const app = @import("../app.zig"); +const cli = @import("../cli.zig"); +const clients = @import("clients.zig"); +const clients_repo = @import("../storage/repositories/clients_repo.zig"); +const db = @import("../storage/db.zig"); +const dns_cache = @import("../cache/dns_cache.zig"); +const forward_zones = @import("../local/forward_zones.zig"); +const handler = @import("handler.zig"); +const header = @import("../dns/header.zig"); +const logger_mod = @import("../storage/logger.zig"); +const manager = @import("../filter/manager.zig"); +const matcher = @import("../filter/matcher.zig"); +const migrations = @import("../storage/migrations.zig"); +const model = @import("../config/model.zig"); +const name = @import("../dns/name.zig"); +const packet = @import("../dns/packet.zig"); +const pause = @import("pause.zig"); +const question = @import("../dns/question.zig"); +const rate_limiter = @import("rate_limiter.zig"); +const record = @import("../dns/record.zig"); +const records = @import("../local/records.zig"); +const response = @import("../filter/response.zig"); +const shutdown = @import("shutdown.zig"); +const transport = @import("../upstream/transport.zig"); +const types = @import("../dns/types.zig"); +const udp_server = @import("udp_server.zig"); + +const testing = std.testing; + +// --------------------------------------------------------------------------- +// shared fixtures +// --------------------------------------------------------------------------- + +/// Long enough that a loopback round trip cannot lose to scheduling, short +/// enough that a broken server fails the run instead of hanging it. +const budget: std.Io.Timeout = .{ .duration = .{ .raw = .fromSeconds(5), .clock = .awake } }; + +const empty_records: records.Records = .empty; +const empty_zones: forward_zones.Zones = .empty; + +/// A five-second TTL makes the blocking answer's TTL unmistakable next to the +/// upstream's 300. +const blocking: response.Options = .{ .mode = .zero, .ttl = 5 }; + +const forward_timeout: std.Io.Clock.Duration = .{ .raw = .fromSeconds(2), .clock = .awake }; + +/// What every fake upstream answers with, and the TTL it carries. +const upstream_rdata = [4]u8{ 93, 184, 216, 34 }; +const upstream_ttl: u32 = 300; + +/// The zone resolver's answer, distinct from the pool's so a case can tell +/// which of the two replied. +const zone_rdata = [4]u8{ 10, 0, 0, 7 }; +const zone_ttl: u32 = 120; + +/// The handler every case starts from: an upstream, the blocking options and +/// the empty local tables. Each case wires in the collaborators it exercises. +fn baseHandler(client: transport.Client) handler.Handler { + return .{ + .upstream = client, + .blocking = blocking, + .forward_read_timeout = forward_timeout, + .records = &empty_records, + .zones = &empty_zones, + }; +} + +/// A real listener, a real client socket and the task that serves them. +/// +/// Two phases: `bind` produces the value, `start` spawns the serve task against +/// its final address. Nothing may copy a `Loop` after `start`, because the task +/// holds a pointer into it. +const Loop = struct { + server: udp_server.UdpServer, + group: std.Io.Group, + client: net.Socket, + server_address: net.IpAddress, + + fn bind(gpa: Allocator, io: std.Io, h: *handler.Handler) !Loop { + const listen_address: net.IpAddress = try .parse("127.0.0.1", 0); + var server = try udp_server.UdpServer.bind(gpa, io, listen_address, h, .{ .max_in_flight = 4 }); + errdefer server.deinit(gpa, io); + + const client_address: net.IpAddress = try .parse("127.0.0.1", 0); + const client = try client_address.bind(io, .{ .mode = .dgram }); + + return .{ + .server = server, + .group = .init, + .client = client, + .server_address = server.boundAddress(), + }; + } + + fn start(self: *Loop, io: std.Io) !void { + try self.group.concurrent(io, udp_server.UdpServer.serve, .{ &self.server, io }); + } + + /// One query, one reply. The reply is a prefix of `buf`. + fn ask(self: *Loop, io: std.Io, query: []const u8, buf: []u8) ![]u8 { + try self.client.send(io, &self.server_address, query); + const msg = try self.client.receiveTimeout(io, buf, budget); + return msg.data; + } + + fn stop(self: *Loop, gpa: Allocator, io: std.Io) void { + self.server.deinit(gpa, io); + self.group.cancel(io); + self.client.close(io); + } +}; + +/// A query for `domain`, RD set, one question, no OPT. +fn queryFor(buf: []u8, id: u16, domain: []const u8, qtype: types.Type) []const u8 { + var w: std.Io.Writer = .fixed(buf); + var encoded: [types.header_len]u8 = undefined; + header.encode(.{ + .id = id, + .flags = .{ + .rcode = .no_error, + .z = 0, + .ra = false, + .rd = true, + .tc = false, + .aa = false, + .opcode = .query, + .qr = false, + }, + .qdcount = 1, + .ancount = 0, + .nscount = 0, + .arcount = 0, + }, &encoded); + w.writeAll(&encoded) catch unreachable; + question.encode(.{ + .name = name.fromText(domain) catch unreachable, + .qtype = qtype, + .qclass = .in, + }, &w) catch unreachable; + return w.buffered(); +} + +/// The pool stand-in. It answers the question it is given rather than a fixed +/// byte string, because the safe-search and uncloaking cases both change the +/// question on the way out. +/// +/// `calls` is atomic: the listener task runs on another thread than the one +/// asserting. +const FakeUpstream = struct { + reply: Reply, + calls: std.atomic.Value(u64) = .init(0), + + const Reply = union(enum) { + /// One A record for the queried name. + a, + /// One CNAME record for the queried name, pointing at this target. + cname: []const u8, + }; + + fn exchangeFn( + ptr: *anyopaque, + io: std.Io, + query: []const u8, + response_buf: []u8, + ) transport.ExchangeError![]u8 { + _ = io; + const self: *FakeUpstream = @ptrCast(@alignCast(ptr)); + _ = self.calls.fetchAdd(1, .monotonic); + + const request = packet.parse(query) catch return error.BadResponse; + const q = packet.firstQuestion(request) orelse return error.BadResponse; + + var b = packet.ResponseBuilder.init(response_buf, request.header, q) catch + return error.ResponseTooLarge; + switch (self.reply) { + .a => b.addAnswer(q.name, .a, .in, upstream_ttl, &upstream_rdata) catch + return error.ResponseTooLarge, + .cname => |target| { + const t = name.fromText(target) catch return error.BadResponse; + b.addAnswer(q.name, .cname, .in, upstream_ttl, t.wire()) catch + return error.ResponseTooLarge; + }, + } + return b.finish(); + } + + fn client(self: *FakeUpstream) transport.Client { + return .{ .ptr = self, .exchangeFn = exchangeFn }; + } +}; + +/// The forward zone's resolver: a real UDP socket, because `ForwardClient` +/// speaks wire DNS to an address and nothing smaller would prove it did. +/// +/// The loop ends when the receive is canceled, which is what `group.cancel` +/// does at the end of the case. +fn zoneResolver(io: std.Io, socket: *const net.Socket, calls: *std.atomic.Value(u64)) void { + var buf: [udp_server.max_datagram]u8 = undefined; + while (true) { + const msg = socket.receive(io, &buf) catch return; + _ = calls.fetchAdd(1, .monotonic); + + const request = packet.parse(msg.data) catch continue; + const q = packet.firstQuestion(request) orelse continue; + + var reply_buf: [512]u8 = undefined; + var b = packet.ResponseBuilder.init(&reply_buf, request.header, q) catch continue; + b.addAnswer(q.name, .a, .in, zone_ttl, &zone_rdata) catch continue; + socket.send(io, &msg.from, b.finish()) catch return; + } +} + +const SnapshotFixture = struct { + groups: []const model.Group = &.{.{ .name = "default" }}, + rules: []const model.Rule = &.{}, +}; + +fn buildSnapshot(gpa: Allocator, fixture: SnapshotFixture) !matcher.Snapshot { + return matcher.Snapshot.build(gpa, .{ + .groups = fixture.groups, + .group_ids = &.{1}, + .group_sources = &.{}, + .sources = &.{}, + .source_ids = &.{}, + .rules = fixture.rules, + .clients = &.{}, + .prefixes = &.{}, + .compiled = &.{}, + .seed = 0x5eed, + .generation = 1, + }); +} + +/// `Manager.acquire` reads the manager's lock and its current snapshot and +/// nothing else, so a manager that publishes one hand-built snapshot needs +/// none of the database, fetcher or blocklist directory the real one owns. +fn fixtureManager(m: *manager.Manager, snapshot: *matcher.Snapshot) void { + m.* = .{ + .gpa = testing.allocator, + .database = undefined, + .paths = undefined, + .fetcher = undefined, + .update = .{}, + .total_budget = forward_timeout, + .lock = .init, + .writer_lock = .init, + .current = snapshot, + .generation = 1, + .statuses = &.{}, + .status_arena = .init(testing.allocator), + }; +} + +fn blockRule(pattern: []const u8) model.Rule { + return .{ .group = "default", .pattern = pattern, .kind = .exact, .action = .block }; +} + +fn allowRule(pattern: []const u8) model.Rule { + return .{ .group = "default", .pattern = pattern, .kind = .exact, .action = .allow }; +} + +fn firstAnswer(p: packet.Packet) !record.Record { + var it = packet.answers(p); + return (try it.next()) orelse error.TestExpectedAnswer; +} + +fn drainLog(lg: *logger_mod.Logger, io: std.Io, out: []logger_mod.Entry) []logger_mod.Entry { + const n = lg.queue.getUncancelable(io, out, 0) catch 0; + return out[0..n]; +} + +/// Every case that asserts on the query log wants the same shape: a queue big +/// enough to hold the whole case, drained once at the end. +const log_queue_len = 8; + +// --------------------------------------------------------------------------- +// case 1: blocked domain +// --------------------------------------------------------------------------- + +test "S7 case 1: a blocked domain is answered with the zero address and logged" { + if (!build_options.integration) return error.SkipZigTest; + + const gpa = testing.allocator; + var threaded: std.Io.Threaded = .init(gpa, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var snapshot = try buildSnapshot(gpa, .{ .rules = &.{blockRule("ads.example.com")} }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + var queue_buf: [log_queue_len]logger_mod.Entry = undefined; + var lg: logger_mod.Logger = .init(.{}, &queue_buf); + + var fake: FakeUpstream = .{ .reply = .a }; + var h = baseHandler(fake.client()); + h.manager = &mgr; + h.logger = ≶ + + var loop = try Loop.bind(gpa, io, &h); + defer loop.stop(gpa, io); + try loop.start(io); + + var query_buf: [512]u8 = undefined; + var reply_buf: [udp_server.max_datagram]u8 = undefined; + const reply = try loop.ask(io, queryFor(&query_buf, 0x1111, "ads.example.com", .a), &reply_buf); + + const p = try packet.parse(reply); + try testing.expectEqual(@as(u16, 0x1111), p.header.id); + try testing.expectEqual(types.Rcode.no_error, p.header.flags.rcode); + try testing.expectEqual(@as(u16, 1), p.header.ancount); + + const answer = try firstAnswer(p); + try testing.expectEqual(@as(u32, blocking.ttl), answer.ttl); + try testing.expectEqual([4]u8{ 0, 0, 0, 0 }, try record.rdataA(p.bytes, answer)); + + try testing.expectEqual(@as(u64, 0), fake.calls.load(.monotonic)); + try testing.expectEqual(@as(u64, 1), h.stats.blocked.load(.monotonic)); + + var entries: [log_queue_len]logger_mod.Entry = undefined; + const logged = drainLog(&lg, io, &entries); + try testing.expectEqual(@as(usize, 1), logged.len); + try testing.expectEqual(true, logged[0].blocked); + try testing.expectEqualStrings("ads.example.com", logged[0].domain()); + try testing.expectEqualStrings("rule_block_exact", logged[0].blockReason()); + try testing.expectEqualStrings("127.0.0.1", logged[0].clientIp()); +} + +// --------------------------------------------------------------------------- +// case 2: allow over block +// --------------------------------------------------------------------------- + +test "S7 case 2: an allow rule beats the blocklist and the upstream answers" { + if (!build_options.integration) return error.SkipZigTest; + + const gpa = testing.allocator; + var threaded: std.Io.Threaded = .init(gpa, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var snapshot = try buildSnapshot(gpa, .{ + .rules = &.{ blockRule("com"), allowRule("example.com") }, + }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + var fake: FakeUpstream = .{ .reply = .a }; + var h = baseHandler(fake.client()); + h.manager = &mgr; + + var loop = try Loop.bind(gpa, io, &h); + defer loop.stop(gpa, io); + try loop.start(io); + + var query_buf: [512]u8 = undefined; + var reply_buf: [udp_server.max_datagram]u8 = undefined; + const reply = try loop.ask(io, queryFor(&query_buf, 0x2222, "example.com", .a), &reply_buf); + + const p = try packet.parse(reply); + const answer = try firstAnswer(p); + try testing.expectEqual(upstream_rdata, try record.rdataA(p.bytes, answer)); + try testing.expectEqual(upstream_ttl, answer.ttl); + try testing.expectEqual(@as(u64, 1), fake.calls.load(.monotonic)); + try testing.expectEqual(@as(u64, 0), h.stats.blocked.load(.monotonic)); +} + +// --------------------------------------------------------------------------- +// case 3: local records +// --------------------------------------------------------------------------- + +test "S7 case 3: a local record answers authoritatively without an upstream" { + if (!build_options.integration) return error.SkipZigTest; + + const gpa = testing.allocator; + var threaded: std.Io.Threaded = .init(gpa, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var table = try records.Records.build(gpa, &.{ + .{ .name = "nas.lan", .rtype = .a, .value = "192.168.1.10", .ttl = 60 }, + }); + defer table.deinit(gpa); + + var fake: FakeUpstream = .{ .reply = .a }; + var h = baseHandler(fake.client()); + h.records = &table; + + var loop = try Loop.bind(gpa, io, &h); + defer loop.stop(gpa, io); + try loop.start(io); + + var query_buf: [512]u8 = undefined; + var reply_buf: [udp_server.max_datagram]u8 = undefined; + const reply = try loop.ask(io, queryFor(&query_buf, 0x3333, "nas.lan", .a), &reply_buf); + + const p = try packet.parse(reply); + try testing.expectEqual(true, p.header.flags.aa); + try testing.expectEqual(@as(u16, 1), p.header.ancount); + + const answer = try firstAnswer(p); + try testing.expectEqual(@as(u32, 60), answer.ttl); + try testing.expectEqual([4]u8{ 192, 168, 1, 10 }, try record.rdataA(p.bytes, answer)); + + try testing.expectEqual(@as(u64, 0), fake.calls.load(.monotonic)); + try testing.expectEqual(@as(u64, 1), h.stats.local_answers.load(.monotonic)); +} + +// --------------------------------------------------------------------------- +// case 4: forward zones +// --------------------------------------------------------------------------- + +test "S7 case 4: a forward zone reaches its resolver, bypasses the blocklist and caches" { + if (!build_options.integration) return error.SkipZigTest; + + const gpa = testing.allocator; + var threaded: std.Io.Threaded = .init(gpa, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + // The zone resolver is a socket of its own, so the case can tell a query + // that reached it from one the pool answered. + const resolver_bind: net.IpAddress = try .parse("127.0.0.1", 0); + const resolver_socket = try resolver_bind.bind(io, .{ .mode = .dgram }); + defer resolver_socket.close(io); + + var resolver_calls: std.atomic.Value(u64) = .init(0); + var resolver_group: std.Io.Group = .init; + defer resolver_group.cancel(io); + try resolver_group.concurrent(io, zoneResolver, .{ io, &resolver_socket, &resolver_calls }); + + var resolver_text: [64]u8 = undefined; + const resolver_url = try std.fmt.bufPrint(&resolver_text, "udp://127.0.0.1:{d}", .{ + resolver_socket.address.ip4.port, + }); + + var zones = try forward_zones.Zones.build(gpa, &.{ + .{ .zone = "lan.home", .resolver = resolver_url }, + }); + defer zones.deinit(gpa); + + // The name is blocklisted, so an answer from the zone resolver is proof the + // bypass (ruling 7) holds over the wire. + var snapshot = try buildSnapshot(gpa, .{ .rules = &.{blockRule("nas.lan.home")} }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + var cache: dns_cache.DnsCache = try .init(gpa, .{ .size = 8, .negative_ttl_max = 3600 }); + defer cache.deinit(); + + var fake: FakeUpstream = .{ .reply = .a }; + var h = baseHandler(fake.client()); + h.manager = &mgr; + h.zones = &zones; + h.cache = &cache; + h.negative_ttl_max = 3600; + + var loop = try Loop.bind(gpa, io, &h); + defer loop.stop(gpa, io); + try loop.start(io); + + var query_buf: [512]u8 = undefined; + var reply_buf: [udp_server.max_datagram]u8 = undefined; + const query = queryFor(&query_buf, 0x4444, "nas.lan.home", .a); + + const first = try loop.ask(io, query, &reply_buf); + const p = try packet.parse(first); + try testing.expectEqual(types.Rcode.no_error, p.header.flags.rcode); + try testing.expectEqual(zone_rdata, try record.rdataA(p.bytes, try firstAnswer(p))); + try testing.expectEqual(@as(u64, 1), resolver_calls.load(.monotonic)); + try testing.expectEqual(@as(u64, 0), fake.calls.load(.monotonic)); + try testing.expectEqual(@as(u64, 0), h.stats.blocked.load(.monotonic)); + try testing.expectEqual(@as(u32, 1), cache.len()); + + // The second query is answered from the cache: the resolver socket sees + // nothing more (PLAN §6.5). + var second_buf: [512]u8 = undefined; + const second_query = queryFor(&second_buf, 0x4455, "nas.lan.home", .a); + const second = try loop.ask(io, second_query, &reply_buf); + + const second_p = try packet.parse(second); + try testing.expectEqual(@as(u16, 0x4455), second_p.header.id); + try testing.expectEqual(zone_rdata, try record.rdataA(second_p.bytes, try firstAnswer(second_p))); + try testing.expectEqual(@as(u64, 1), resolver_calls.load(.monotonic)); + try testing.expectEqual(@as(u64, 1), h.stats.cache_hits.load(.monotonic)); +} + +// --------------------------------------------------------------------------- +// case 5: cache +// --------------------------------------------------------------------------- + +test "S7 case 5: a cached answer comes back with a fresh id, an aged ttl and a logged hit" { + if (!build_options.integration) return error.SkipZigTest; + + const gpa = testing.allocator; + var threaded: std.Io.Threaded = .init(gpa, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var cache: dns_cache.DnsCache = try .init(gpa, .{ .size = 8, .negative_ttl_max = 3600 }); + defer cache.deinit(); + + var queue_buf: [log_queue_len]logger_mod.Entry = undefined; + var lg: logger_mod.Logger = .init(.{}, &queue_buf); + + var fake: FakeUpstream = .{ .reply = .a }; + var h = baseHandler(fake.client()); + h.cache = &cache; + h.negative_ttl_max = 3600; + h.logger = ≶ + + var loop = try Loop.bind(gpa, io, &h); + defer loop.stop(gpa, io); + try loop.start(io); + + var reply_buf: [udp_server.max_datagram]u8 = undefined; + + // Miss, then hit under a different transaction ID. + var miss_buf: [512]u8 = undefined; + _ = try loop.ask(io, queryFor(&miss_buf, 0x5501, "example.com", .a), &reply_buf); + try testing.expectEqual(@as(u32, 1), cache.len()); + try testing.expectEqual(@as(u64, 1), fake.calls.load(.monotonic)); + + var hit_buf: [512]u8 = undefined; + const hit = try loop.ask(io, queryFor(&hit_buf, 0x5502, "example.com", .a), &reply_buf); + const p = try packet.parse(hit); + try testing.expectEqual(@as(u16, 0x5502), p.header.id); + try testing.expectEqual(upstream_rdata, try record.rdataA(p.bytes, try firstAnswer(p))); + try testing.expectEqual(@as(u64, 1), fake.calls.load(.monotonic)); + try testing.expectEqual(@as(u64, 1), h.stats.cache_hits.load(.monotonic)); + + // Ageing needs elapsed time, and a test cannot wait 10 seconds for it. The + // entry is therefore planted with a stored-at stamp 10 seconds in the past, + // under exactly the key the handler builds for this query. + var aged_query_buf: [512]u8 = undefined; + const aged_query = queryFor(&aged_query_buf, 0x5503, "aged.example.com", .a); + var stored_buf: [512]u8 = undefined; + const aged_p = try packet.parse(aged_query); + var b = try packet.ResponseBuilder.init(&stored_buf, aged_p.header, packet.firstQuestion(aged_p).?); + try b.addAnswer(try name.fromText("aged.example.com"), .a, .in, upstream_ttl, &upstream_rdata); + + var key_buf: [dns_cache.max_key_len]u8 = undefined; + const key = dns_cache.buildKey( + &key_buf, + "aged.example.com", + @intFromEnum(types.Type.a), + @intFromEnum(types.Class.in), + false, + null, + ); + const aged_by = 10; + try cache.put( + std.Io.Clock.real.now(io).toSeconds() - aged_by, + key, + b.finish(), + .{ .ttl_seconds = upstream_ttl, .negative = false }, + ); + + const aged = try loop.ask(io, aged_query, &reply_buf); + const aged_reply = try packet.parse(aged); + try testing.expectEqual(upstream_ttl - aged_by, (try firstAnswer(aged_reply)).ttl); + try testing.expectEqual(@as(u64, 1), fake.calls.load(.monotonic)); + + var entries: [log_queue_len]logger_mod.Entry = undefined; + const logged = drainLog(&lg, io, &entries); + try testing.expectEqual(@as(usize, 3), logged.len); + try testing.expectEqual(@as(?bool, false), logged[0].cache_hit); + try testing.expectEqualStrings("pool", logged[0].upstream()); + try testing.expectEqual(@as(?bool, true), logged[1].cache_hit); + try testing.expectEqualStrings("", logged[1].upstream()); + try testing.expectEqual(@as(?bool, true), logged[2].cache_hit); +} + +// --------------------------------------------------------------------------- +// case 6: CNAME uncloaking +// --------------------------------------------------------------------------- + +test "S7 case 6: a cname into a blocked target blocks the original question" { + if (!build_options.integration) return error.SkipZigTest; + + const gpa = testing.allocator; + var threaded: std.Io.Threaded = .init(gpa, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var snapshot = try buildSnapshot(gpa, .{ .rules = &.{blockRule("tracker.example.org")} }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + var queue_buf: [log_queue_len]logger_mod.Entry = undefined; + var lg: logger_mod.Logger = .init(.{}, &queue_buf); + + var fake: FakeUpstream = .{ .reply = .{ .cname = "tracker.example.org" } }; + var h = baseHandler(fake.client()); + h.manager = &mgr; + h.logger = ≶ + + var loop = try Loop.bind(gpa, io, &h); + defer loop.stop(gpa, io); + try loop.start(io); + + var query_buf: [512]u8 = undefined; + var reply_buf: [udp_server.max_datagram]u8 = undefined; + const reply = try loop.ask(io, queryFor(&query_buf, 0x6666, "cdn.example.com", .a), &reply_buf); + + const p = try packet.parse(reply); + try testing.expectEqual(@as(u16, 1), p.header.ancount); + + // The answer is about the name the client asked for, not the target. + const answer = try firstAnswer(p); + try testing.expectEqual(types.Type.a, answer.rtype); + try testing.expectEqualSlices( + u8, + (try name.fromText("cdn.example.com")).wire(), + answer.name.wire(), + ); + try testing.expectEqual([4]u8{ 0, 0, 0, 0 }, try record.rdataA(p.bytes, answer)); + try testing.expectEqual(@as(u64, 1), h.stats.uncloak_blocked.load(.monotonic)); + + var entries: [log_queue_len]logger_mod.Entry = undefined; + const logged = drainLog(&lg, io, &entries); + try testing.expectEqual(@as(usize, 1), logged.len); + try testing.expectEqual(true, logged[0].blocked); + try testing.expectEqualStrings("cname:rule_block_exact", logged[0].blockReason()); + try testing.expectEqualStrings("cdn.example.com", logged[0].domain()); +} + +// --------------------------------------------------------------------------- +// case 7: safe search +// --------------------------------------------------------------------------- + +test "S7 case 7: safe search answers the original question with a cname to the target" { + if (!build_options.integration) return error.SkipZigTest; + + const gpa = testing.allocator; + var threaded: std.Io.Threaded = .init(gpa, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var snapshot = try buildSnapshot(gpa, .{ + .groups = &.{.{ .name = "default", .safe_search = true }}, + }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + var fake: FakeUpstream = .{ .reply = .a }; + var h = baseHandler(fake.client()); + h.manager = &mgr; + + var loop = try Loop.bind(gpa, io, &h); + defer loop.stop(gpa, io); + try loop.start(io); + + var query_buf: [512]u8 = undefined; + var reply_buf: [udp_server.max_datagram]u8 = undefined; + const reply = try loop.ask(io, queryFor(&query_buf, 0x7777, "www.google.com", .a), &reply_buf); + + const p = try packet.parse(reply); + const target = try name.fromText("forcesafesearch.google.com"); + + // The reply keeps the question the client asked. + try testing.expectEqualSlices( + u8, + (try name.fromText("www.google.com")).wire(), + packet.firstQuestion(p).?.name.wire(), + ); + try testing.expectEqual(@as(u16, 2), p.header.ancount); + + var it = packet.answers(p); + const cname = (try it.next()).?; + try testing.expectEqual(types.Type.cname, cname.rtype); + try testing.expectEqualSlices(u8, target.wire(), (try record.rdataCname(p.bytes, cname)).wire()); + + const a = (try it.next()).?; + try testing.expectEqual(types.Type.a, a.rtype); + try testing.expectEqualSlices(u8, target.wire(), a.name.wire()); + try testing.expectEqual(upstream_rdata, try record.rdataA(p.bytes, a)); + + try testing.expectEqual(@as(u64, 1), h.stats.safesearch_rewrites.load(.monotonic)); +} + +// --------------------------------------------------------------------------- +// case 8: rate limit +// --------------------------------------------------------------------------- + +test "S7 case 8: the third query inside the window is refused" { + if (!build_options.integration) return error.SkipZigTest; + + const gpa = testing.allocator; + var threaded: std.Io.Threaded = .init(gpa, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var limiter: rate_limiter.RateLimiter = try .init(gpa, .{ .limit = 2, .window_seconds = 60 }); + defer limiter.deinit(); + + var queue_buf: [log_queue_len]logger_mod.Entry = undefined; + var lg: logger_mod.Logger = .init(.{}, &queue_buf); + + var fake: FakeUpstream = .{ .reply = .a }; + var h = baseHandler(fake.client()); + h.limiter = &limiter; + h.logger = ≶ + + var loop = try Loop.bind(gpa, io, &h); + defer loop.stop(gpa, io); + try loop.start(io); + + var reply_buf: [udp_server.max_datagram]u8 = undefined; + for ([_]u16{ 0x8801, 0x8802 }) |id| { + var query_buf: [512]u8 = undefined; + const reply = try loop.ask(io, queryFor(&query_buf, id, "example.com", .a), &reply_buf); + try testing.expectEqual(types.Rcode.no_error, (try packet.parse(reply)).header.flags.rcode); + } + + var third_buf: [512]u8 = undefined; + const refused = try loop.ask(io, queryFor(&third_buf, 0x8803, "example.com", .a), &reply_buf); + const p = try packet.parse(refused); + try testing.expectEqual(types.Rcode.refused, p.header.flags.rcode); + try testing.expectEqual(@as(u16, 0x8803), p.header.id); + try testing.expectEqual(@as(u64, 1), h.stats.refused.load(.monotonic)); + try testing.expectEqual(@as(u64, 2), fake.calls.load(.monotonic)); + + // Ruling 8: a refused query is never query-logged. + var entries: [log_queue_len]logger_mod.Entry = undefined; + try testing.expectEqual(@as(usize, 2), drainLog(&lg, io, &entries).len); +} + +// --------------------------------------------------------------------------- +// case 9: pause +// --------------------------------------------------------------------------- + +test "S7 case 9: pause lifts filtering and unpause restores it" { + if (!build_options.integration) return error.SkipZigTest; + + const gpa = testing.allocator; + var threaded: std.Io.Threaded = .init(gpa, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var snapshot = try buildSnapshot(gpa, .{ .rules = &.{blockRule("ads.example.com")} }); + defer snapshot.deinit(); + var mgr: manager.Manager = undefined; + fixtureManager(&mgr, &snapshot); + + var paused: pause.Pause = .{}; + paused.pauseFor(std.Io.Clock.real.now(io).toSeconds(), null); + + var fake: FakeUpstream = .{ .reply = .a }; + var h = baseHandler(fake.client()); + h.manager = &mgr; + h.pause = &paused; + + var loop = try Loop.bind(gpa, io, &h); + defer loop.stop(gpa, io); + try loop.start(io); + + var reply_buf: [udp_server.max_datagram]u8 = undefined; + + var paused_buf: [512]u8 = undefined; + const while_paused = try loop.ask(io, queryFor(&paused_buf, 0x9901, "ads.example.com", .a), &reply_buf); + const p = try packet.parse(while_paused); + try testing.expectEqual(upstream_rdata, try record.rdataA(p.bytes, try firstAnswer(p))); + try testing.expectEqual(@as(u64, 1), h.stats.paused_queries.load(.monotonic)); + try testing.expectEqual(@as(u64, 0), h.stats.blocked.load(.monotonic)); + + // Resuming puts the block back without a restart (ruling 18). + paused.unpause(); + + var resumed_buf: [512]u8 = undefined; + const after = try loop.ask(io, queryFor(&resumed_buf, 0x9902, "ads.example.com", .a), &reply_buf); + const after_p = try packet.parse(after); + try testing.expectEqual([4]u8{ 0, 0, 0, 0 }, try record.rdataA(after_p.bytes, try firstAnswer(after_p))); + try testing.expectEqual(@as(u64, 1), h.stats.blocked.load(.monotonic)); +} + +// --------------------------------------------------------------------------- +// case 10: client tracking +// --------------------------------------------------------------------------- + +test "S7 case 10: the querying client is materialised as a row" { + if (!build_options.integration) return error.SkipZigTest; + + const gpa = testing.allocator; + var threaded: std.Io.Threaded = .init(gpa, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + // The tracker's own connection (ruling 21), in memory here: the flush is + // what this case asserts on, not where the file lives. + var database = try db.Db.open(":memory:", .{ .mode = .memory }); + defer database.close(); + try db.applyPragmas(&database, .{}); + _ = try migrations.migrate(&database); + + var tracker: clients.Tracker = .init(30); + + var fake: FakeUpstream = .{ .reply = .a }; + var h = baseHandler(fake.client()); + h.tracker = &tracker; + + var loop = try Loop.bind(gpa, io, &h); + defer loop.stop(gpa, io); + try loop.start(io); + + var reply_buf: [udp_server.max_datagram]u8 = undefined; + for ([_]u16{ 0xa001, 0xa002 }) |id| { + var query_buf: [512]u8 = undefined; + _ = try loop.ask(io, queryFor(&query_buf, id, "example.com", .a), &reply_buf); + } + + // Two queries from one client are one pending entry, and the forced pass + // stands in for the 60-second flush interval (S4 As-built seam). + try testing.expectEqual(@as(u32, 1), tracker.pendingClients(io)); + tracker.flushOnce(io, &database, true); + + try testing.expectEqual(@as(i64, 1), try clients_repo.countClients(&database)); + try testing.expectEqual(@as(u32, 0), tracker.pendingClients(io)); + try testing.expectEqual(@as(u64, 1), tracker.snapshotStats(io).flushed); + + var stmt = try database.prepare("SELECT ip, hand_edited FROM clients"); + defer stmt.deinit(); + try testing.expect(try stmt.step()); + try testing.expectEqualStrings("127.0.0.1", stmt.columnText(0)); + try testing.expectEqual(@as(i64, 0), stmt.columnInt(1)); +} + +// --------------------------------------------------------------------------- +// case 11: the whole application +// --------------------------------------------------------------------------- + +/// `std.testing.tmpDir` creates its directory against `std.testing.io`, so the +/// application under test runs on the same `Io` instance the fixture used. The +/// other cases build an `Io.Threaded` of their own, the way the listener tests +/// do; this one cannot, because the temporary directory is already bound to +/// this instance. +const test_io = testing.io; + +/// Where `std.testing.tmpDir` puts its directories (`lib/std/testing.zig:634`). +const tmp_prefix = ".zig-cache/tmp/"; +const sub_path_len = @typeInfo(@FieldType(testing.TmpDir, "sub_path")).array.len; + +/// High enough to need no privilege, and not the 15353/15354 pair the milestone +/// smoke test used, so a stray smoke process cannot make this case pass. +const app_port = 15455; + +/// The unreachable upstream the seed configuration names. Nothing in this case +/// needs it: the query it resolves is a local record, and a dead upstream is +/// what proves the fail-open design still serves. +const dead_upstream = "https://127.0.0.1:9/dns-query"; + +const app_config = + \\.{ + \\ .dns = .{ + \\ .bind_ipv4 = "127.0.0.1", + \\ .bind_ipv6 = "::1", + \\ .port = 15455, + \\ .rate_limit = 1000, + \\ .rate_window_seconds = 60, + \\ }, + \\ .logging = .{ .level = .info, .output = .stderr }, + \\ .web = .{ .enabled = false }, + \\ .groups = .{ .{ .name = "default" } }, + \\ .upstreams = .{ .{ .url = "https://127.0.0.1:9/dns-query" } }, + \\ .local_records = .{ + \\ .{ .name = "boot.test", .rtype = .a, .value = "10.9.8.7", .ttl = 60 }, + \\ }, + \\} + \\ +; + +comptime { + // The port and the upstream appear in the configuration text as literals, + // because a `.zon` file is data and not a format string. + std.debug.assert(std.mem.containsAtLeast(u8, app_config, 1, std.fmt.comptimePrint("{d}", .{app_port}))); + std.debug.assert(std.mem.containsAtLeast(u8, app_config, 1, dead_upstream)); +} + +/// How long one attempt at reaching the booting server waits, and how many +/// attempts it gets. The product is the time the application has to bind. +const boot_attempt: std.Io.Timeout = .{ .duration = .{ .raw = .fromMilliseconds(200), .clock = .awake } }; +const boot_attempts = 100; + +/// Queries the booting server until it answers. A server that has not bound yet +/// either swallows the datagram or answers it with an ICMP rejection, and both +/// arrive here as an error worth retrying. +fn askUntilAnswered( + socket: *const net.Socket, + dest: net.IpAddress, + query: []const u8, + buf: []u8, +) ![]u8 { + var attempt: usize = 0; + while (attempt < boot_attempts) : (attempt += 1) { + socket.send(test_io, &dest, query) catch continue; + const msg = socket.receiveTimeout(test_io, buf, boot_attempt) catch continue; + return msg.data; + } + return error.TestAppNeverAnswered; +} + +test "S7 case 11: the app boots, serves a query and exits zero on shutdown" { + if (!build_options.integration) return error.SkipZigTest; + + const gpa = testing.allocator; + + var tmp = testing.tmpDir(.{ .iterate = true }); + defer tmp.cleanup(); + try tmp.dir.writeFile(test_io, .{ .sub_path = "config.zon", .data = app_config }); + + var root_buf: [tmp_prefix.len + sub_path_len]u8 = undefined; + @memcpy(root_buf[0..tmp_prefix.len], tmp_prefix); + @memcpy(root_buf[tmp_prefix.len..], &tmp.sub_path); + const root: []const u8 = &root_buf; + + var config_buf: [root_buf.len + "/config.zon".len]u8 = undefined; + const config_path = try std.fmt.bufPrint(&config_buf, "{s}/config.zon", .{root}); + + var out: std.Io.Writer.Allocating = .init(gpa); + defer out.deinit(); + var err: std.Io.Writer.Allocating = .init(gpa); + defer err.deinit(); + + const runner: cli.Runner = .{ + .io = test_io, + .gpa = gpa, + .out = &out.writer, + .err = &err.writer, + }; + + // The shutdown event is process-global, and another case in this binary may + // have left it set. + shutdown.reset(); + defer shutdown.reset(); + + var future = try test_io.concurrent(app.run, .{ runner, cli.Paths{ + .data_dir = root, + .config = config_path, + } }); + + const client_address: net.IpAddress = try .parse("127.0.0.1", 0); + const client = try client_address.bind(test_io, .{ .mode = .dgram }); + defer client.close(test_io); + + const server_address: net.IpAddress = try .parse("127.0.0.1", app_port); + + var query_buf: [512]u8 = undefined; + const query = queryFor(&query_buf, 0xb001, "boot.test", .a); + var reply_buf: [udp_server.max_datagram]u8 = undefined; + + const reply = askUntilAnswered(&client, server_address, query, &reply_buf) catch |e| { + shutdown.trigger(test_io); + _ = future.await(test_io); + return e; + }; + + const p = try packet.parse(reply); + try testing.expectEqual(@as(u16, 0xb001), p.header.id); + try testing.expectEqual(types.Rcode.no_error, p.header.flags.rcode); + try testing.expectEqual(true, p.header.flags.aa); + try testing.expectEqual([4]u8{ 10, 9, 8, 7 }, try record.rdataA(p.bytes, try firstAnswer(p))); + + shutdown.trigger(test_io); + try testing.expectEqual(cli.exit_ok, future.await(test_io)); + + // The lifecycle proof is the exit code, and a clean exit prints nothing. + try testing.expectEqualStrings("", err.written()); +} diff --git a/src/server/resolver_integration_test.zig b/src/server/resolver_integration_test.zig index 316f556..22a2908 100644 --- a/src/server/resolver_integration_test.zig +++ b/src/server/resolver_integration_test.zig @@ -18,6 +18,10 @@ const net = std.Io.net; const handler = @import("handler.zig"); const tcp_server = @import("tcp_server.zig"); const udp_server = @import("udp_server.zig"); +const model = @import("../config/model.zig"); +const response = @import("../filter/response.zig"); +const forward_zones = @import("../local/forward_zones.zig"); +const records = @import("../local/records.zig"); const packet = @import("../dns/packet.zig"); const record = @import("../dns/record.zig"); const types = @import("../dns/types.zig"); @@ -27,6 +31,31 @@ const transport = @import("../upstream/transport.zig"); const testing = std.testing; +const empty_records: records.Records = .empty; +const empty_zones: forward_zones.Zones = .empty; +const blocking_defaults: model.Blocking = .{}; +const blocking: response.Options = .{ + .mode = blocking_defaults.response, + .ttl = blocking_defaults.ttl, +}; +const forward_timeout: std.Io.Clock.Duration = .{ + .raw = model.readTimeout(.{}), + .clock = .awake, +}; + +/// An upstream and nothing else optional: no filtering, no cache, no log. The +/// listeners and the pool are what this test exercises, so the handler is the +/// same bare one its own tests use. +fn bareHandler(client: transport.Client) handler.Handler { + return .{ + .upstream = client, + .blocking = blocking, + .forward_read_timeout = forward_timeout, + .records = &empty_records, + .zones = &empty_zones, + }; +} + /// Long enough that a loopback round trip cannot lose to scheduling, short /// enough that a broken server fails the run instead of hanging it. const budget: std.Io.Timeout = .{ .duration = .{ .raw = .fromSeconds(5), .clock = .awake } }; @@ -226,7 +255,7 @@ test "the whole resolver answers over udp and tcp and fails over to a healthy up }; var upstreams: pool.Pool = .init(&entries, test_cfg, attempt_timeout, 1); - var h: handler.Handler = .{ .upstream = upstreams.client() }; + var h = bareHandler(upstreams.client()); const listen_address: net.IpAddress = try .parse("127.0.0.1", 0); var udp = try udp_server.UdpServer.bind(gpa, io, listen_address, &h, .{ .max_in_flight = 4 }); diff --git a/src/server/shutdown.zig b/src/server/shutdown.zig new file mode 100644 index 0000000..9110199 --- /dev/null +++ b/src/server/shutdown.zig @@ -0,0 +1,118 @@ +//! SIGINT and SIGTERM, turned into one `std.Io.Event`. +//! +//! No signalfd, no self-pipe, no epoll: the handler does exactly one thing, and +//! `std.Io.Event.set` is async-signal-safe on the Threaded Linux backend — a +//! raw `futex` wake with no allocation and no lock (`Io.zig:1855` → +//! `Threaded.futexWake`). The `.mask`/`.flags` shape is the one Threaded uses +//! for its own `SIG.IO`/`SIG.PIPE` handlers (`Threaded.zig:1653`): an empty +//! mask and no `SA_RESTART`, so a blocking syscall returns `EINTR` and the +//! backend's retry loop re-reads the cancellation state. +//! +//! Everything a shutdown actually has to do — drain the query log, cancel the +//! task group, close the databases — happens on the task blocked in `wait`. +//! +//! The previous handlers are not restored. The process is leaving, and a +//! second SIGTERM during teardown should still terminate it the default way +//! only if the operator sends it before this module is armed. + +const std = @import("std"); +const posix = std.posix; + +var event: std.Io.Event = .unset; + +/// Read by the signal handler, written by `install` before the handler exists. +/// A `std.Io` is two pointers and cannot be stored atomically, so ordering is +/// what makes the read safe: the store precedes the `sigaction` syscall that +/// arms the handler, and no signal can reach the handler before that call +/// returns. +var handler_io: ?std.Io = null; + +var installed: bool = false; + +/// Arms the handlers for INT and TERM. Calling it again is a no-op: the process +/// has one event and one pair of handlers, and a second boot inside one process +/// (which only a test does) must not re-arm anything. +pub fn install(io: std.Io) void { + if (installed) return; + handler_io = io; + installed = true; + + const act: posix.Sigaction = .{ + .handler = .{ .handler = onSignal }, + .mask = posix.sigemptyset(), + .flags = 0, + }; + posix.sigaction(.INT, &act, null); + posix.sigaction(.TERM, &act, null); +} + +fn onSignal(_: posix.SIG) callconv(.c) void { + const io = handler_io orelse return; + event.set(io); +} + +/// Blocks until a shutdown is requested. A canceled wait is the caller's cue to +/// tear down as well, which is why `app.run` treats both results the same. +pub fn wait(io: std.Io) std.Io.Cancelable!void { + return event.wait(io); +} + +/// The programmatic equivalent of the signal: what a test uses to shut the app +/// down, and what a Phase 8 restart endpoint would call. +pub fn trigger(io: std.Io) void { + event.set(io); +} + +pub fn isRequested() bool { + return event.isSet(); +} + +/// Clears the request so the next `wait` blocks again. Only a test that boots +/// the app more than once in one process needs this; a served process shuts +/// down once. +pub fn reset() void { + event.reset(); +} + +const testing = std.testing; + +test "trigger releases a waiter and isRequested reports it" { + var threaded: std.Io.Threaded = .init(testing.allocator, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + reset(); + try testing.expect(!isRequested()); + + trigger(io); + try testing.expect(isRequested()); + // Already set, so this returns without blocking. + try wait(io); + + reset(); + try testing.expect(!isRequested()); +} + +test "a waiting task is released by a later trigger" { + var threaded: std.Io.Threaded = .init(testing.allocator, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + reset(); + + var group: std.Io.Group = .init; + try group.concurrent(io, waitThenSet, .{ io, &done }); + trigger(io); + try group.await(io); + + try testing.expect(done.isSet()); + reset(); + done.reset(); +} + +var done: std.Io.Event = .unset; + +fn waitThenSet(io: std.Io, flag: *std.Io.Event) std.Io.Cancelable!void { + try wait(io); + flag.set(io); +} diff --git a/src/server/tcp_server.zig b/src/server/tcp_server.zig index 73f0a1b..4f5a21a 100644 --- a/src/server/tcp_server.zig +++ b/src/server/tcp_server.zig @@ -12,8 +12,25 @@ //! No stream read or write in 0.16.0 accepts a timeout, so every per-connection //! operation is raced against `Options.idle_timeout` through `std.Io.Select` and //! the loser is canceled. +//! +//! Shutdown takes one of two paths, and they end the live connections +//! differently on purpose: +//! +//! - `deinit` shuts every active stream down first, so the connections unblock +//! and finish by themselves. `serve` then drains them, and a reply that was +//! half written still goes out whole. +//! - A canceled `serve` cannot drain. `deinit` is what would shut the streams +//! down, and it cannot run until `serve` returns — the composition root +//! cancels its task group before it releases anything (app.zig). Meanwhile +//! RFC 7766 §6.2.1.1 lets a client hold a connection open indefinitely by +//! asking again inside the idle budget, so draining would let one chatty +//! client stall the whole process's shutdown. The connections are canceled +//! instead, at the cost of the one reply that was mid-write. +//! +//! Either way `serve` returns only once no task can still touch a slot. const std = @import("std"); +const address = @import("../platform/address.zig"); const handler = @import("handler.zig"); const transport = @import("../upstream/transport.zig"); @@ -53,6 +70,16 @@ const State = enum(u32) { idle, serving, closing }; /// before the close, and `deinit` only touches `.active` slots. const ConnState = enum { free, active, closing }; +/// Why the accept loop stopped, which decides what happens to the connections +/// still in flight. +const Stop = enum { + /// `deinit` published `.closing`. It has already shut every live connection + /// down, so each one is unblocked and finishing on its own. + closing, + /// This task is being canceled. Nothing has touched the connections. + canceled, +}; + /// What the accept loop does with a stream it has just accepted. const Claim = union(enum) { /// The stream owns `conns[index]`. @@ -77,7 +104,7 @@ pub const TcpServer = struct { state: std.atomic.Value(State), stopped: std.Io.Event, - /// One slot is ~131 KiB, so the default 64 connections cost ~8.4 MiB, which + /// One slot is ~137 KiB, so the default 64 connections cost ~8.8 MiB, which /// is inside the PLAN §18 budget. The two message buffers cannot be shared /// or shrunk: the handler holds the query while the reply is built, and /// both ceilings are the 65535 bytes the length prefix can express. @@ -86,7 +113,15 @@ pub const TcpServer = struct { reply: [transport.max_message_len]u8, read_buf: [stream_buffer_len]u8, write_buf: [stream_buffer_len]u8, + /// The handler's per-query working memory. It belongs to the slot so + /// that answering a message allocates nothing, and a connection is + /// answered serially, so one query uses it at a time. + scratch: handler.Scratch, stream: std.Io.net.Stream, + /// The client, read off the accepted socket once at claim time: every + /// message on this connection comes from the same peer, and the handler + /// needs it for rate limiting, groups and the query log. + peer: std.Io.net.IpAddress, /// Guarded by `TcpServer.mutex`. state: ConnState, }; @@ -96,7 +131,7 @@ pub const TcpServer = struct { pub fn listen( gpa: std.mem.Allocator, io: std.Io, - address: std.Io.net.IpAddress, + listen_address: std.Io.net.IpAddress, h: *handler.Handler, options: Options, ) ListenError!TcpServer { @@ -106,7 +141,7 @@ pub const TcpServer = struct { errdefer gpa.free(conns); for (conns) |*conn| conn.state = .free; - const local = address; + const local = listen_address; const server = try local.listen(io, .{ .reuse_address = true }); return .{ @@ -132,15 +167,28 @@ pub const TcpServer = struct { if (self.state.cmpxchgStrong(.idle, .serving, .acq_rel, .acquire) != null) return; var group: std.Io.Group = .init; - self.acceptLoop(io, &group); - - // A reply that is half written is worse than no reply, so the live - // connections are awaited even when this task is being canceled. - const prev = io.swapCancelProtection(.blocked); - group.await(io) catch |err| switch (err) { - error.Canceled => unreachable, - }; - _ = io.swapCancelProtection(prev); + switch (self.acceptLoop(io, &group)) { + // `deinit` shut every live connection down before it published + // `.closing`, so each one is already unblocked and ending on its + // own. Awaiting them means a half-written reply still goes out + // whole, and the wait is bounded by the shutdown, not the client. + .closing => { + const prev = io.swapCancelProtection(.blocked); + group.await(io) catch |err| switch (err) { + error.Canceled => unreachable, + }; + _ = io.swapCancelProtection(prev); + }, + // Nothing has shut these connections down: `deinit` cannot run + // until this task returns, and RFC 7766 lets a client hold a + // connection open forever by asking again inside the idle budget. + // Draining here would therefore let one client stall the whole + // process's shutdown for as long as it likes. `cancel` requests + // cancellation and joins, so the slots are still quiet — and the + // buffers still unreferenced — by the time `serve` returns; the + // price is the one reply that was mid-write. + .canceled => group.cancel(io), + } self.stopped.set(io); } @@ -167,14 +215,17 @@ pub const TcpServer = struct { self.* = undefined; } - fn acceptLoop(self: *TcpServer, io: std.Io, group: *std.Io.Group) void { + fn acceptLoop(self: *TcpServer, io: std.Io, group: *std.Io.Group) Stop { while (self.state.load(.acquire) == .serving) { const stream = self.server.accept(io) catch |err| switch (err) { - error.Canceled, error.SocketNotListening => return, + error.Canceled => return .canceled, + // `deinit` shuts the listening socket down to unblock exactly + // this call, so it is the shutdown path arriving early. + error.SocketNotListening => return .closing, else => { bump(&self.stats.accept_errors); log.debug("tcp accept failed: {t}", .{err}); - retry_delay.sleep(io) catch return; + retry_delay.sleep(io) catch return .canceled; continue; }, }; @@ -192,7 +243,7 @@ pub const TcpServer = struct { .shutting_down => { bump(&self.stats.rejected_at_shutdown); stream.close(io); - return; + return .closing; }, }; @@ -206,6 +257,9 @@ pub const TcpServer = struct { bump(&self.stats.accepted); } + + // The loop condition failed, which only `deinit` can cause. + return .closing; } fn serveConn(self: *TcpServer, io: std.Io, index: usize) void { @@ -258,7 +312,15 @@ pub const TcpServer = struct { }, } - const bytes = switch (self.handler.handle(io, .tcp, conn.query[0..len], &conn.reply)) { + const outcome = self.handler.handle( + io, + .tcp, + address.NetAddress.fromIp(conn.peer), + conn.query[0..len], + &conn.reply, + &conn.scratch, + ); + const bytes = switch (outcome) { // There is no framing for "no answer", so the connection ends. .drop => return, .reply => |b| b, @@ -286,6 +348,7 @@ pub const TcpServer = struct { switch (outcome) { .slot => |index| { self.conns[index].stream = stream; + self.conns[index].peer = stream.socket.address; self.conns[index].state = .active; }, .at_capacity, .shutting_down => {}, diff --git a/src/server/tcp_server_integration_test.zig b/src/server/tcp_server_integration_test.zig index aad8784..63d1369 100644 --- a/src/server/tcp_server_integration_test.zig +++ b/src/server/tcp_server_integration_test.zig @@ -15,6 +15,10 @@ const net = std.Io.net; const handler = @import("handler.zig"); const tcp_server = @import("tcp_server.zig"); +const model = @import("../config/model.zig"); +const response = @import("../filter/response.zig"); +const forward_zones = @import("../local/forward_zones.zig"); +const records = @import("../local/records.zig"); const header = @import("../dns/header.zig"); const packet = @import("../dns/packet.zig"); const types = @import("../dns/types.zig"); @@ -22,6 +26,31 @@ const transport = @import("../upstream/transport.zig"); const testing = std.testing; +const empty_records: records.Records = .empty; +const empty_zones: forward_zones.Zones = .empty; +const blocking_defaults: model.Blocking = .{}; +const blocking: response.Options = .{ + .mode = blocking_defaults.response, + .ttl = blocking_defaults.ttl, +}; +const forward_timeout: std.Io.Clock.Duration = .{ + .raw = model.readTimeout(.{}), + .clock = .awake, +}; + +/// An upstream and nothing else optional: no filtering, no cache, no log. The +/// listener is what these tests exercise, so the handler is the same bare one +/// its own tests use. +fn bareHandler(client: transport.Client) handler.Handler { + return .{ + .upstream = client, + .blocking = blocking, + .forward_read_timeout = forward_timeout, + .records = &empty_records, + .zones = &empty_zones, + }; +} + const budget: std.Io.Clock.Duration = .{ .raw = .fromSeconds(5), .clock = .awake }; /// Short enough to keep the idle-timeout test quick, long enough that a @@ -150,7 +179,7 @@ test "two length-prefixed queries share one connection" { const io = threaded.io(); var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: handler.Handler = .{ .upstream = fake.client() }; + var h = bareHandler(fake.client()); const listen_address: net.IpAddress = try .parse("127.0.0.1", 0); var server = try tcp_server.TcpServer.listen(gpa, io, listen_address, &h, .{ .max_connections = 2 }); @@ -171,6 +200,145 @@ test "two length-prefixed queries share one connection" { }; } +test "the claimed slot records the connecting client" { + if (!build_options.integration) return error.SkipZigTest; + + const gpa = testing.allocator; + var threaded: std.Io.Threaded = .init(gpa, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bareHandler(fake.client()); + + const listen_address: net.IpAddress = try .parse("127.0.0.1", 0); + var server = try tcp_server.TcpServer.listen(gpa, io, listen_address, &h, .{ .max_connections = 2 }); + const server_address = server.boundAddress(); + + var group: std.Io.Group = .init; + try group.concurrent(io, tcp_server.TcpServer.serve, .{ &server, io }); + + try bounded(io, twoQueriesOnOneConnection, .{ io, server_address }); + + // The only connection took slot 0, and the reply the client already read + // was written after `claim` filled the slot in, so this read races nothing. + // Without a real peer the handler would rate-limit, group and log every TCP + // client under whatever the uninitialized slot happened to hold. + const peer = server.conns[0].peer; + try testing.expectEqual(net.IpAddress.ip4, std.meta.activeTag(peer)); + try testing.expectEqualSlices(u8, &[_]u8{ 127, 0, 0, 1 }, &peer.ip4.bytes); + try testing.expect(peer.ip4.port != 0); + + server.deinit(gpa, io); + group.await(io) catch |err| switch (err) { + error.Canceled => unreachable, + }; +} + +/// `std.Io.Group` takes only `Cancelable!void`, so the result is reported out +/// of band. `answered` is set either way — a client that fails early must not +/// leave the test blocked waiting for a reply that will never come, and `set` +/// is documented to have no effect the second time. +fn holdConnection( + io: std.Io, + address: net.IpAddress, + answered: *std.Io.Event, + result: *anyerror!void, +) void { + result.* = holdConnectionOpen(io, address, answered); + answered.set(io); +} + +/// Holds a connection open the way RFC 7766 lets a real client hold one: one +/// query answered, then nothing, so the server sits blocked on the read for the +/// next message. Returns once the server ends the connection, however it ends +/// it — a canceled server closes the socket, which the client sees either as +/// end of stream or as a reset. +fn holdConnectionOpen(io: std.Io, address: net.IpAddress, answered: *std.Io.Event) anyerror!void { + const remote = address; + var stream = try remote.connect(io, .{ .mode = .stream }); + defer stream.close(io); + + var read_buf: [1024]u8 = undefined; + var write_buf: [1024]u8 = undefined; + var reader = stream.reader(io, &read_buf); + var writer = stream.writer(io, &write_buf); + + try writer.interface.writeAll(&transport.framePrefix(@intCast(query_bytes.len))); + try writer.interface.writeAll(query_bytes); + try writer.interface.flush(); + + const len = transport.parsePrefix((try reader.interface.takeArray(transport.prefix_len)).*); + try expectAnswersQuery(try reader.interface.take(len)); + + answered.set(io); + + var sink: [64]u8 = undefined; + _ = reader.interface.readSliceShort(&sink) catch {}; +} + +test "a canceled serve does not wait for a live connection" { + if (!build_options.integration) return error.SkipZigTest; + + const gpa = testing.allocator; + var threaded: std.Io.Threaded = .init(gpa, .{}); + defer threaded.deinit(); + const io = threaded.io(); + + var fake: FakeUpstream = .{ .reply = response_bytes }; + var h = bareHandler(fake.client()); + + const listen_address: net.IpAddress = try .parse("127.0.0.1", 0); + // The idle budget is the whole time a drain would have to wait out, so it + // is set far beyond any patience this test run has: if `serve` waits for + // the connection instead of canceling it, the wait never ends. + var server = try tcp_server.TcpServer.listen(gpa, io, listen_address, &h, .{ + .max_connections = 2, + .idle_timeout = .{ .raw = .fromSeconds(600), .clock = .awake }, + }); + const server_address = server.boundAddress(); + + var serving: std.Io.Group = .init; + try serving.concurrent(io, tcp_server.TcpServer.serve, .{ &server, io }); + + var answered: std.Io.Event = .unset; + var client_result: anyerror!void = {}; + var client_group: std.Io.Group = .init; + try client_group.concurrent(io, holdConnection, .{ io, server_address, &answered, &client_result }); + + // The reply proves the connection is claimed and served, so the connection + // task is now blocked reading the message that never comes. That is the + // state a drain hangs in. + answered.waitUncancelable(io); + + // This is the composition root's shutdown: cancel the task group before + // anything it borrows is released, so no `deinit` has shut the connection + // down. It must still return. A regression here does not fail the test, it + // hangs the run — there is no way to bound a join that does not finish. + serving.cancel(io); + + // Cancellation must not skip the per-connection cleanup: the slot is + // released and the socket closed by `serveConn`'s defer, which runs on the + // canceled path like any other. + try testing.expectEqual(@as(?usize, 0), firstFreeSlot(&server)); + + client_group.cancel(io); + server.deinit(gpa, io); + + // Checked last: the connection had to be answered for the test to mean + // anything, and the server is torn down before a failure is reported. + try client_result; +} + +/// The first slot the server would hand out, read after `serve` has returned so +/// nothing can be writing it. +fn firstFreeSlot(server: *const tcp_server.TcpServer) ?usize { + for (server.conns, 0..) |*conn, index| { + if (conn.state == .free) return index; + } + return null; +} + test "an idle connection is closed and counted" { if (!build_options.integration) return error.SkipZigTest; @@ -180,7 +348,7 @@ test "an idle connection is closed and counted" { const io = threaded.io(); var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: handler.Handler = .{ .upstream = fake.client() }; + var h = bareHandler(fake.client()); const listen_address: net.IpAddress = try .parse("127.0.0.1", 0); var server = try tcp_server.TcpServer.listen(gpa, io, listen_address, &h, .{ @@ -213,7 +381,7 @@ test "a zero-length message is a connection error" { const io = threaded.io(); var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: handler.Handler = .{ .upstream = fake.client() }; + var h = bareHandler(fake.client()); const listen_address: net.IpAddress = try .parse("127.0.0.1", 0); var server = try tcp_server.TcpServer.listen(gpa, io, listen_address, &h, .{ @@ -265,7 +433,7 @@ test "deinit ends a serve loop that is blocked on accept" { const io = threaded.io(); var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: handler.Handler = .{ .upstream = fake.client() }; + var h = bareHandler(fake.client()); const listen_address: net.IpAddress = try .parse("127.0.0.1", 0); var server = try tcp_server.TcpServer.listen(gpa, io, listen_address, &h, .{ .max_connections = 2 }); diff --git a/src/server/udp_server.zig b/src/server/udp_server.zig index 225f4a3..ef78930 100644 --- a/src/server/udp_server.zig +++ b/src/server/udp_server.zig @@ -9,6 +9,7 @@ //! allocates nothing. The pool is sized once in `bind` and never grows. const std = @import("std"); +const address = @import("../platform/address.zig"); const handler = @import("handler.zig"); const transport = @import("../upstream/transport.zig"); @@ -75,10 +76,14 @@ pub const UdpServer = struct { /// The datagram actually sent stays bounded by `udpLimit` inside the /// handler, so nothing larger than 4096 bytes leaves this socket. /// - /// Cost: 4096 + 65535 ≈ 68 KiB per slot, so the default 64 slots hold - /// ≈ 4.3 MiB. The PLAN §18 budget is 100 MB with ~1M blocked domains, - /// so this pool takes about 4% of it. + /// Cost: 4096 + 65535 + the scratch below ≈ 74 KiB per slot, so the + /// default 64 slots hold ≈ 4.6 MiB. The PLAN §18 budget is 100 MB with + /// ~1M blocked domains, so this pool takes about 5% of it. reply: [transport.max_message_len]u8, + /// The handler's per-query working memory. It belongs to the slot so + /// that answering a datagram still allocates nothing, and one slot + /// serves one query at a time. + scratch: handler.Scratch, from: std.Io.net.IpAddress, len: usize, /// Guarded by `UdpServer.mutex`. @@ -90,7 +95,7 @@ pub const UdpServer = struct { pub fn bind( gpa: std.mem.Allocator, io: std.Io, - address: std.Io.net.IpAddress, + bind_address: std.Io.net.IpAddress, h: *handler.Handler, options: Options, ) BindError!UdpServer { @@ -100,7 +105,7 @@ pub const UdpServer = struct { errdefer gpa.free(slots); for (slots) |*slot| slot.in_use = false; - const local = address; + const local = bind_address; const socket = try local.bind(io, .{ .mode = .dgram }); return .{ @@ -204,7 +209,15 @@ pub const UdpServer = struct { const slot = &self.slots[index]; defer self.release(io, index); - switch (self.handler.handle(io, .udp, slot.query[0..slot.len], &slot.reply)) { + const outcome = self.handler.handle( + io, + .udp, + address.NetAddress.fromIp(slot.from), + slot.query[0..slot.len], + &slot.reply, + &slot.scratch, + ); + switch (outcome) { .drop => bump(&self.stats.dropped_handler), .reply => |bytes| self.socket.send(io, &slot.from, bytes) catch |err| { bump(&self.stats.send_errors); @@ -283,8 +296,8 @@ test "a slot's reply buffer holds a whole DNS message" { } test "the default slot pool stays inside the memory budget" { - // 4096 + 65535 ≈ 68 KiB per slot; 64 slots ≈ 4.3 MiB, against the 100 MB - // of PLAN §18. + // 4096 + 65535 + scratch ≈ 74 KiB per slot; 64 slots ≈ 4.6 MiB, against the + // 100 MB of PLAN §18. const options: Options = .{}; const pool_bytes = @sizeOf(UdpServer.Slot) * @as(usize, options.max_in_flight); try testing.expect(pool_bytes < 8 * 1024 * 1024); diff --git a/src/server/udp_server_integration_test.zig b/src/server/udp_server_integration_test.zig index 27b50c6..d5ab52d 100644 --- a/src/server/udp_server_integration_test.zig +++ b/src/server/udp_server_integration_test.zig @@ -14,6 +14,10 @@ const net = std.Io.net; const handler = @import("handler.zig"); const udp_server = @import("udp_server.zig"); +const model = @import("../config/model.zig"); +const response = @import("../filter/response.zig"); +const forward_zones = @import("../local/forward_zones.zig"); +const records = @import("../local/records.zig"); const header = @import("../dns/header.zig"); const packet = @import("../dns/packet.zig"); const types = @import("../dns/types.zig"); @@ -21,6 +25,31 @@ const transport = @import("../upstream/transport.zig"); const testing = std.testing; +const empty_records: records.Records = .empty; +const empty_zones: forward_zones.Zones = .empty; +const blocking_defaults: model.Blocking = .{}; +const blocking: response.Options = .{ + .mode = blocking_defaults.response, + .ttl = blocking_defaults.ttl, +}; +const forward_timeout: std.Io.Clock.Duration = .{ + .raw = model.readTimeout(.{}), + .clock = .awake, +}; + +/// An upstream and nothing else optional: no filtering, no cache, no log. The +/// listener is what these tests exercise, so the handler is the same bare one +/// its own tests use. +fn bareHandler(client: transport.Client) handler.Handler { + return .{ + .upstream = client, + .blocking = blocking, + .forward_read_timeout = forward_timeout, + .records = &empty_records, + .zones = &empty_zones, + }; +} + /// Long enough that a loopback round trip cannot lose to scheduling, short /// enough that a broken server fails the run instead of hanging it. const budget: std.Io.Timeout = .{ .duration = .{ .raw = .fromSeconds(5), .clock = .awake } }; @@ -87,7 +116,7 @@ test "a udp query is answered on the loopback" { const io = threaded.io(); var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: handler.Handler = .{ .upstream = fake.client() }; + var h = bareHandler(fake.client()); const listen_address: net.IpAddress = try .parse("127.0.0.1", 0); var server = try udp_server.UdpServer.bind(gpa, io, listen_address, &h, .{ .max_in_flight = 4 }); @@ -125,7 +154,7 @@ test "a runt datagram is dropped and no reply is sent" { const io = threaded.io(); var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: handler.Handler = .{ .upstream = fake.client() }; + var h = bareHandler(fake.client()); const listen_address: net.IpAddress = try .parse("127.0.0.1", 0); var server = try udp_server.UdpServer.bind(gpa, io, listen_address, &h, .{ .max_in_flight = 4 }); @@ -160,7 +189,7 @@ test "an oversize datagram arrives truncated and is dropped" { const io = threaded.io(); var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: handler.Handler = .{ .upstream = fake.client() }; + var h = bareHandler(fake.client()); const listen_address: net.IpAddress = try .parse("127.0.0.1", 0); var server = try udp_server.UdpServer.bind(gpa, io, listen_address, &h, .{ .max_in_flight = 4 }); @@ -201,7 +230,7 @@ test "deinit ends a serve loop that is blocked on receive" { const io = threaded.io(); var fake: FakeUpstream = .{ .reply = response_bytes }; - var h: handler.Handler = .{ .upstream = fake.client() }; + var h = bareHandler(fake.client()); const listen_address: net.IpAddress = try .parse("127.0.0.1", 0); var server = try udp_server.UdpServer.bind(gpa, io, listen_address, &h, .{ .max_in_flight = 4 }); diff --git a/src/storage/repositories/clients_repo.zig b/src/storage/repositories/clients_repo.zig index 2e583cf..6903a9b 100644 --- a/src/storage/repositories/clients_repo.zig +++ b/src/storage/repositories/clients_repo.zig @@ -5,7 +5,8 @@ //! not appear in an export. `countClients` counts **all** rows, because S5's //! "has this database ever been configured" predicate needs the true count. //! -//! Only list / insert / deleteAll / count exist. +//! Only list / insert / deleteAll / count, plus the two runtime calls +//! `upsertSeen` and `pruneStale` that the Phase 7 client tracker owns. const std = @import("std"); const Allocator = std.mem.Allocator; @@ -81,6 +82,51 @@ pub fn insertClient(database: *db.Db, item: model.Client, ctx: InsertContext) db try stmt.exec(); } +const upsert_seen_sql = + \\INSERT INTO clients (ip, name, group_id, hand_edited, first_seen, last_seen) + \\VALUES (?1, NULL, (SELECT id FROM groups WHERE name = 'default'), 0, ?2, ?2) + \\ON CONFLICT(ip) DO UPDATE SET last_seen = excluded.last_seen +; + +/// Materialises a client seen in live traffic (PLAN §7.2), or touches +/// `last_seen` on the row that already holds `ip`. +/// +/// The conflict target is `clients.ip`, which the schema declares UNIQUE. Only +/// `last_seen` is updated: `name`, `group_id` and `hand_edited` are the +/// operator's, and a device that keeps querying must not overwrite them. A +/// hand-edited row is touched too, so the operator sees liveness for the +/// clients they named. +/// +/// A new row lands in the `default` group. `groupForClient` resolves the real +/// group from the prefix rules at query time, so the column here only decides +/// what the operator sees before they assign the device themselves. +/// +/// A database with no group named `default` fails the insert with +/// `error.Constraint` rather than writing a dangling row. `validate.zig` +/// rejects such a configuration long before a server serves from it. +pub fn upsertSeen(database: *db.Db, ip: []const u8, now_s: i64) db.Error!void { + var stmt = try database.prepare(upsert_seen_sql); + defer stmt.deinit(); + try stmt.bindText(1, ip); + try stmt.bindInt(2, now_s); + try stmt.exec(); +} + +/// Deletes the auto-materialised clients whose last query predates `cutoff_s`, +/// and returns how many rows went. `hand_edited = 1` rows are configuration and +/// survive any silence. +/// +/// `last_seen` is the only signal, because §3.6 forbids joining `config.db` +/// against the query log. +pub fn pruneStale(database: *db.Db, cutoff_s: i64) db.Error!u32 { + var stmt = try database.prepare("DELETE FROM clients WHERE hand_edited = 0 AND last_seen < ?1"); + defer stmt.deinit(); + try stmt.bindInt(1, cutoff_s); + try stmt.exec(); + const deleted = database.changes(); + return @intCast(@min(deleted, std.math.maxInt(u32))); +} + pub fn deleteAllClients(database: *db.Db) db.Error!void { return database.exec("DELETE FROM clients;"); } @@ -281,6 +327,124 @@ test "listClients is leak-safe under allocation failure" { try testing.checkAllAllocationFailures(testing.allocator, listClientsUnderFailure, .{&ids}); } +fn seenRow(database: *db.Db, ip: []const u8) !struct { hand_edited: i64, first_seen: i64, last_seen: i64, group_id: i64 } { + var stmt = try database.prepare( + "SELECT hand_edited, first_seen, last_seen, group_id FROM clients WHERE ip = ?1", + ); + defer stmt.deinit(); + try stmt.bindText(1, ip); + try testing.expect(try stmt.step()); + return .{ + .hand_edited = stmt.columnInt(0), + .first_seen = stmt.columnInt(1), + .last_seen = stmt.columnInt(2), + .group_id = stmt.columnInt(3), + }; +} + +test "upsertSeen materialises an unseen client in the default group" { + var database = try openMigrated(); + defer database.close(); + + try upsertSeen(&database, "192.168.1.50", 1700000000); + + try testing.expectEqual(@as(i64, 1), try countClients(&database)); + const row = try seenRow(&database, "192.168.1.50"); + try testing.expectEqual(@as(i64, 0), row.hand_edited); + try testing.expectEqual(@as(i64, 1700000000), row.first_seen); + try testing.expectEqual(@as(i64, 1700000000), row.last_seen); + try testing.expectEqual(@as(i64, 1), row.group_id); + + // Materialised clients are runtime state, so an export must not see them. + var items = try listClients(&database, testing.allocator); + defer items.deinit(testing.allocator); + defer freeClients(testing.allocator, items.items); + try testing.expectEqual(@as(usize, 0), items.items.len); +} + +test "upsertSeen touches last_seen and leaves first_seen alone" { + var database = try openMigrated(); + defer database.close(); + + try upsertSeen(&database, "192.168.1.50", 1700000000); + try upsertSeen(&database, "192.168.1.50", 1700000600); + + try testing.expectEqual(@as(i64, 1), try countClients(&database)); + const row = try seenRow(&database, "192.168.1.50"); + try testing.expectEqual(@as(i64, 1700000000), row.first_seen); + try testing.expectEqual(@as(i64, 1700000600), row.last_seen); +} + +test "upsertSeen keeps a hand-edited row's name, group and flag" { + var database = try openMigrated(); + defer database.close(); + var ids = try seedGroups(&database); + defer ids.deinit(testing.allocator); + try seedClients(&database, &ids); + + try upsertSeen(&database, "192.168.1.20", 1700009999); + + try testing.expectEqual(@as(i64, 3), try countClients(&database)); + const row = try seenRow(&database, "192.168.1.20"); + try testing.expectEqual(@as(i64, 1), row.hand_edited); + try testing.expectEqual(@as(i64, 2), row.group_id); + try testing.expectEqual(@as(i64, 1700000000), row.first_seen); + try testing.expectEqual(@as(i64, 1700009999), row.last_seen); + + var items = try listClients(&database, testing.allocator); + defer items.deinit(testing.allocator); + defer freeClients(testing.allocator, items.items); + try testing.expectEqual(@as(usize, 3), items.items.len); + try testing.expectEqualStrings("laptop", items.items[1].name); + try testing.expectEqualStrings("kids", items.items[1].group); +} + +test "upsertSeen reports a database with no default group" { + var database = try openMigrated(); + defer database.close(); + try database.exec("UPDATE groups SET name = 'renamed' WHERE id = 1;"); + + try testing.expectError(error.Constraint, upsertSeen(&database, "192.168.1.50", 1700000000)); + try testing.expectEqual(@as(i64, 0), try countClients(&database)); +} + +test "pruneStale removes only stale auto-materialised rows" { + var database = try openMigrated(); + defer database.close(); + var ids = try seedGroups(&database); + defer ids.deinit(testing.allocator); + // A hand-edited row far older than the cutoff. + try seedClients(&database, &ids); + + try upsertSeen(&database, "10.0.0.1", 1700000000); + try upsertSeen(&database, "10.0.0.2", 1700000199); + // Exactly at the cutoff: the comparison is strict, so it stays. + try upsertSeen(&database, "10.0.0.3", 1700000200); + try upsertSeen(&database, "10.0.0.4", 1700000300); + + try testing.expectEqual(@as(u32, 2), try pruneStale(&database, 1700000200)); + try testing.expectEqual(@as(i64, 5), try countClients(&database)); + try testing.expectEqual( + @as(i64, 0), + try database.queryInt("SELECT count(*) FROM clients WHERE ip IN ('10.0.0.1', '10.0.0.2')"), + ); + try testing.expectEqual(@as(i64, 3), try database.queryInt("SELECT count(*) FROM clients WHERE hand_edited = 1")); + + // A second pass over the same cutoff finds nothing left to do. + try testing.expectEqual(@as(u32, 0), try pruneStale(&database, 1700000200)); +} + +test "pruneStale spares hand-edited rows however stale" { + var database = try openMigrated(); + defer database.close(); + var ids = try seedGroups(&database); + defer ids.deinit(testing.allocator); + try seedClients(&database, &ids); + + try testing.expectEqual(@as(u32, 0), try pruneStale(&database, 1800000000)); + try testing.expectEqual(@as(i64, 3), try countClients(&database)); +} + test "client_prefixes round-trip in prefix order with group names resolved" { var database = try openMigrated(); defer database.close(); diff --git a/src/tests.zig b/src/tests.zig index b88365c..a4858a4 100644 --- a/src/tests.zig +++ b/src/tests.zig @@ -72,6 +72,10 @@ comptime { _ = @import("platform/logging.zig"); _ = @import("storage/retention.zig"); _ = @import("storage/phase6_integration_test.zig"); + _ = @import("server/pause.zig"); + _ = @import("server/clients.zig"); + _ = @import("server/shutdown.zig"); + _ = @import("server/phase7_integration_test.zig"); } extern fn sqlite3_libversion() [*:0]const u8;