milestone 7: serving pipeline, client tracking, pause and lifecycle
This commit is contained in:
@@ -0,0 +1,700 @@
|
||||
# Milestone 7: serving pipeline composition (PLAN Phase 7)
|
||||
|
||||
Goal: `nxdns run` serves DNS end to end — rate limit → parse → group → local records →
|
||||
forward zones → filtering → safe-search → cache → upstream → CNAME uncloaking → cache store →
|
||||
async query log — with client auto-materialization, pause/resume, and a composition root that
|
||||
starts every background loop and shuts down cleanly on SIGINT/SIGTERM.
|
||||
|
||||
Ground truth: PLAN.md §4 (pipeline diagram, lines 151-173), §6 (DNS behavior), §7.2
|
||||
(auto-materialization), §8 (cache key), §10 (rate limit), §11.4/§11.6 (logging, disk), Phase 7
|
||||
text at PLAN.md:606-608. Verified Zig facts: specs/research/zig-0.16-api-notes.md plus the
|
||||
scratchpad note m7-explore-zig.md (signals, Io.Group, cancelable net ops) — the load-bearing
|
||||
facts are restated inline below where a session needs them.
|
||||
|
||||
## Rulings (engineering calls resolving PLAN ambiguities — binding)
|
||||
|
||||
1. **Scope**: Phase 7 owns `nxdns run` end to end: `src/app.zig` (composition root),
|
||||
`src/server/shutdown.zig` (signal → event), the `cli.runRun` body, and both listener edits.
|
||||
2. **Signals**: no signalfd/epoll/timerfd. `std.posix.sigaction` handler for INT and TERM that
|
||||
calls `std.Io.Event.set` on a file-scope event (`set` is async-signal-safe on the Threaded
|
||||
Linux backend: raw futex wake, no locks — Threaded.zig:1105,17235).
|
||||
3. **Locking**: `DnsCache` and `RateLimiter` each get one `std.Io.Mutex` owned by the handler.
|
||||
Household scale makes contention irrelevant; sharding is over-engineering.
|
||||
4. **Null snapshot** (`Manager.acquire` returns null before the first successful reload): fail
|
||||
open — skip group/filter/safe-search, use the default group semantics, count
|
||||
`unfiltered_queries`. DNS availability beats filtering for a household. The app attempts one
|
||||
synchronous `reload` before serving; a failure warns and serving starts anyway.
|
||||
5. **Cache scope**: one global cache, no group in the key (PLAN §8). Safe because blocked
|
||||
responses are never cached. Safe-search responses are not cached either (13 domains, rare;
|
||||
caching them under the rewritten name would force answer re-encoding on hit). Paused-mode
|
||||
upstream answers cache normally (they are ordinary upstream responses). Forward-zone
|
||||
responses cache normally (PLAN §6.5) under the same global key.
|
||||
6. **Local vs forward zone**: local records win (checked first; PLAN §6.4 puts them before
|
||||
everything but rate limit/parse).
|
||||
7. **Forward zones bypass** filtering, safe-search, and uncloaking (PLAN §6.5). They are logged
|
||||
with `upstream` = the resolver's canonical text.
|
||||
8. **Rate limit position**: after the 12-byte header read (the REFUSED reply needs the ID),
|
||||
before full parse. Refused queries are counted (`Stats.refused`), never query-logged.
|
||||
Localhost is not exempt (PLAN §10's localhost relaxation is the Phase 8 API limiter only).
|
||||
9. **Non-IN qclass**: skip filtering/cache/safe-search/uncloak, go straight upstream, log
|
||||
normally. Rare and harmless; special-casing CHAOS is scope creep.
|
||||
10. **Safe-search mechanics** (per safesearch.zig:56 doc): the outgoing query's QNAME becomes
|
||||
the target; the reply to the client keeps the original question, starts with a synthesized
|
||||
CNAME original→target, then copies only the A/AAAA answer records from the upstream
|
||||
response verbatim (their RDATA is raw bytes, so no compression-pointer re-encoding; other
|
||||
rtypes are skipped). Applies only on the upstream path — never to local records or forward
|
||||
zones.
|
||||
11. **CNAME uncloaking** (PLAN §6.3): walk the answer-section CNAME chain of the upstream
|
||||
response — no re-resolution. Depth 8 = at most 8 CNAME links followed. Each target is
|
||||
normalized and evaluated for the same group; a blocked target synthesizes a blocked
|
||||
response for the ORIGINAL qname/qtype. `block_reason` = `"cname:" ++ @tagName(reason)`
|
||||
(fits the 32-byte Entry field). Uncloaked-blocked responses are not cached. Skipped when
|
||||
paused, for forward zones, and for non-IN.
|
||||
12. **Local CNAME records** are returned as-is (writeAnswers emits the CNAME and stops,
|
||||
milestone-5 As-built). The client re-queries the target, which then flows through the full
|
||||
pipeline — that is how §6.4's "targets resolve through the normal pipeline" is satisfied.
|
||||
13. **ECS** (PLAN §6.1): `strip` (default) removes a client-sent ECS option from the outgoing
|
||||
query by rebuilding it (header + question + OPT sans option 8); other options are
|
||||
preserved. `forward` sends the query unchanged and puts the raw ECS option payload
|
||||
(≤ `dns_cache.max_ecs_len` = 40 bytes) into the cache key. If a rewritten query cannot fit
|
||||
the 512-byte build buffer (freak OPT payloads), the original is forwarded unchanged,
|
||||
`Stats.ecs_strip_failed` counts it, and the query is excluded from the cache in both
|
||||
directions (the subnet reached the resolver; the strip-mode key says nothing about it).
|
||||
14. **Client auto-materialization** (PLAN §7.2): never a hot-path DB write. The handler records
|
||||
the client in a mutex-guarded fixed-capacity pending table; a background loop flushes every
|
||||
60 s with its OWN config.db connection: `INSERT ... hand_edited=0 ON CONFLICT(ip) DO UPDATE
|
||||
last_seen`. At most one write per client per flush. Table full → drop silently (counted).
|
||||
15. **Snapshot staleness**: a materialized client reaches the matcher snapshot only on the next
|
||||
manager reload. Correct behavior does not depend on it: `groupForClient` already falls back
|
||||
to prefix/default, and materialized rows exist for UI visibility (Phase 8 reads the DB
|
||||
directly). No reload is triggered per client.
|
||||
16. **Client pruning** (PLAN §7.2 + milestone-6 ruling 1): the same background loop prunes
|
||||
daily: `DELETE FROM clients WHERE hand_edited=0 AND last_seen < now - retention_days`
|
||||
(`logging.retention_days`, same knob as the query log — documented). `retention.zig` stays
|
||||
frozen and querylog-only. `last_seen` is the only signal (§3.6 forbids cross-DB joins).
|
||||
17. **Disk gate**: the client flush/prune loop skips its pass when `writesAllowed()` is false;
|
||||
the logger holds its batches. As built: milestone-6.md:410 assigned the manager's
|
||||
consumption to Phase 7 and no seam existed, so S6 added `Manager.monitor:
|
||||
?*disk_monitor.Monitor` (set by the composition root after `init`, before `runScheduler`)
|
||||
plus a `refreshes_gated` counter read through `refreshesGated()`. Only scheduled passes are
|
||||
gated: `runScheduler`'s periodic pass and the source-refresh half of the startup pass.
|
||||
`reload` and `refreshAll` stay ungated (operator actions). The startup pass's
|
||||
`reloadLocked` runs ahead of the gate: publishing an already-compiled snapshot is a read,
|
||||
and a full disk must not cost the household its filtering as well as its downloads. A
|
||||
gated pass logs one warn line and counts.
|
||||
18. **Pause/resume** (PLAN §13.1 mechanism): suspends FILTERING only (evaluate + safe-search +
|
||||
uncloak are skipped); everything else — local records, forward zones, cache, upstream,
|
||||
logging — runs unchanged. Global, not per-group. In-memory only; a restart resumes
|
||||
filtering (safe default). State: one `std.atomic.Value(i64)` — 0 = active, -1 = paused
|
||||
indefinitely, else = paused until that unix second (auto-resume by compare, no timer task).
|
||||
Phase 8's endpoints will call `pauseFor`/`unpause` on the shared pointer.
|
||||
19. **Paused queries** log `blocked=false, block_reason=""`; `Stats.paused_queries` counts
|
||||
them. Evaluation is skipped entirely (not evaluated-then-ignored).
|
||||
20. **Log field semantics**: `response_time_us` = wall time from `handle` entry to reply ready.
|
||||
`cache_hit`: true/false on the upstream path, null for local/forward/refused-never-logged;
|
||||
`upstream`: "" on cache hit, "local" for local records, resolver text for forward zones,
|
||||
the pool endpoint's canonical text for upstream answers (pool already tracks which entry
|
||||
answered — if unavailable without new plumbing, "" and a spec note; do not rebuild the
|
||||
pool for it: `upstream` may be "pool" as the compromise, S2 decides and records).
|
||||
`block_reason` = `@tagName(matcher.Reason)` truncated to 32, `"cname:"`-prefixed for
|
||||
uncloaking. Malformed/dropped queries are never query-logged (handler stats only).
|
||||
21. **Connections**: config.db — one opened by the app (migrate/bootstrap/readConfig, then
|
||||
handed to the manager for reloads), plus one dedicated to the client loop. querylog.db —
|
||||
one for the log writer, one for retention (retention.zig:79 contract). Four total.
|
||||
22. **Shutdown order**: shutdown event fires → `logger.shutdown(io)` (closes the queue; the
|
||||
writer drains and its task returns; a writer stuck in the disk gate holds its final batch —
|
||||
known, documented) → `group.cancel(io)` (requests cancel and joins every task) → deinit
|
||||
servers/pool/manager/state → close DBs → return. Canceled UDP recv / TCP accept return
|
||||
`error.Canceled` on the Threaded backend (Threaded.zig:12439,12877).
|
||||
23. **No runtime config re-read** in Phase 7. The blocklist scheduler reloads snapshots;
|
||||
everything else is restart-required until Phase 8 mutates the DB and calls reload.
|
||||
24. **No new config fields, no schema change.** Pause is in-memory; pruning reuses
|
||||
`retention_days`; ECS uses the existing `edns.ecs_mode`.
|
||||
25. **Performance** (PLAN §18): no formal benchmark in this milestone; the integration test
|
||||
asserts correctness only. The targets stay in PLAN for a later measurement pass.
|
||||
|
||||
## Sessions
|
||||
|
||||
Dependency graph: S1 + S4 parallel (wave 1) → S2 (needs S1's edns API) → S3 (needs S2's
|
||||
signature in code) → S6 (needs S2, S3, S4) → S7 (needs S6). The orchestrator wires
|
||||
`src/tests.zig` after each wave.
|
||||
|
||||
---
|
||||
|
||||
## Session S1: ECS helpers in dns/edns.zig
|
||||
|
||||
Owns: `src/dns/edns.zig` (extend in place; existing API is frozen — additive only).
|
||||
|
||||
### S1.1 API
|
||||
|
||||
```zig
|
||||
/// The raw payload of the ECS option (code 8) in `opt`, or null when absent.
|
||||
/// `pkt` is the packet `opt` was parsed from. error.BadOption on malformed option lengths.
|
||||
pub fn ecsPayload(pkt: packet.Packet, opt: OptRecord) error{BadOption}!?[]const u8
|
||||
|
||||
/// Rebuilds `query` without its ECS option into `out`: header + question copied verbatim,
|
||||
/// then the OPT re-encoded with every option except code 8. Returns `.unchanged` when the
|
||||
/// query has no OPT or no ECS option (nothing written), else `.rewritten` with the slice
|
||||
/// into `out`. `pkt`/`opt` are the caller's already-parsed views of `query`.
|
||||
/// error.Overflow when `out` is too small.
|
||||
pub const StripResult = union(enum) { unchanged, rewritten: []u8 };
|
||||
pub fn stripEcs(query: []const u8, pkt: packet.Packet, opt: OptRecord, out: []u8)
|
||||
error{ BadOption, Overflow }!StripResult
|
||||
```
|
||||
|
||||
Constraints: pure, allocation-free, no Io. The rebuilt query keeps the original ID, flags and
|
||||
question bytes verbatim (copy `query[0..opt_record_offset_of_question_end]` — concretely: copy
|
||||
everything up to the OPT record's start; the OPT is the sole additional record in a query per
|
||||
the existing handler validation, and `packet.parse` yields its offset). ARCOUNT stays 1 when
|
||||
the OPT is re-emitted (it always is — only the option list shrinks). Use the existing
|
||||
`encodeOpt` for the re-emission. The RDATA walk follows the same bounds checks `findOption`
|
||||
uses.
|
||||
|
||||
### S1.2 Acceptance
|
||||
|
||||
- [ ] `zig fmt --check` + `zig ast-check` clean.
|
||||
- [ ] Tests in-file: ECS present → payload returned, stripped query parses cleanly and
|
||||
re-parse finds no option 8, other options survive byte-exact, DO bit and payload size
|
||||
survive; no OPT → `.unchanged`; OPT without ECS → `.unchanged`; malformed option length
|
||||
→ `error.BadOption`; `out` one byte short → `error.Overflow`.
|
||||
- [ ] Temp-root run of the file's tests passes.
|
||||
|
||||
### S1 As built
|
||||
|
||||
Two spec claims were wrong and the implementation corrects them. (1) The handler does NOT
|
||||
guarantee the OPT is the sole additional record (handler.zig:105 rejects only misplaced OPTs;
|
||||
packet.parse rejects only a second OPT) — `stripEcs` therefore copies the bytes after the OPT
|
||||
verbatim and all four section counts survive. Caveat, documented on the function: a compression
|
||||
pointer in a trailing record that targets a byte inside the OPT's option list shifts after the
|
||||
rewrite — such a pointer is already malformed (RFC 1035 §4.1.4) and the consequence is an
|
||||
upstream FORMERR. (2) `packet.parse` does not yield the OPT's start offset; a private
|
||||
`optRecordStart` re-walks the sections and matches the record on its full RDATA span, so a
|
||||
hand-built `OptRecord` cannot aim it at a different record (deriving the start backwards is
|
||||
forgeable: a `0x00xx` transaction ID makes a pointer's low byte decode as a root name).
|
||||
`encodeOpt` takes one contiguous options slice, so `stripEcs` emits it with an empty list,
|
||||
streams the survivors, and patches RDLENGTH at `opt_rdlength_offset` (9); a guard test pins
|
||||
that offset against `encodeOpt`'s real output. All options with code 8 are removed (a malformed
|
||||
query can carry several). Walk failures map to `error.BadOption` (frozen error set; reachable
|
||||
only for hand-assembled Packets, documented). After review: `stripEcs` asserts both that
|
||||
`query` aliases `pkt.bytes` and that `out` is pointer-range disjoint from `query` in both
|
||||
directions; the caller owns keeping the rebuild buffer separate from the received query
|
||||
buffer (S2's `scratch.build` satisfies this). The import is aliased `packet_mod` (seven existing declarations
|
||||
named `packet`). `ecsPayload` was built here, then removed in the review round when the
|
||||
handler's `subnetForKey` single-walk replaced its only caller (see S2 As-built); the
|
||||
forward-mode clamp policy lives there — a payload longer than `dns_cache.max_ecs_len` (40)
|
||||
makes the query uncacheable (truncating the key would merge distinct subnets). 7 S1 tests
|
||||
remain, 25 in file.
|
||||
|
||||
---
|
||||
|
||||
## Session S2: pipeline in server/handler.zig (+ pause.zig)
|
||||
|
||||
Owns: `src/server/handler.zig` (rewrite around the existing skeleton), `src/server/pause.zig`
|
||||
(new). The biggest session. Everything below is frozen interface — S3 and S6 build against it.
|
||||
|
||||
### S2.1 pause.zig
|
||||
|
||||
```zig
|
||||
pub const Pause = struct {
|
||||
until: std.atomic.Value(i64) = .init(0),
|
||||
pub fn isPaused(self: *const Pause, now_s: i64) bool // 0→false, -1→true, else now_s < until
|
||||
pub fn pauseFor(self: *Pause, now_s: i64, duration_s: ?u32) void // null → -1 (indefinite)
|
||||
pub fn unpause(self: *Pause) void // store 0
|
||||
};
|
||||
```
|
||||
|
||||
Auto-resume is the compare in `isPaused` — no timer. `.monotonic` ordering is sufficient (a
|
||||
lone i64 flag, no dependent data). Tests: the three states, expiry boundary (`now == until`
|
||||
is resumed), pauseFor overwrites a previous pause.
|
||||
|
||||
### S2.2 Handler struct (frozen for S3/S6)
|
||||
|
||||
```zig
|
||||
pub const Handler = struct {
|
||||
upstream: transport.Client,
|
||||
blocking: response.Options,
|
||||
ecs_mode: model.EcsMode = .strip,
|
||||
forward_read_timeout: std.Io.Clock.Duration,
|
||||
manager: ?*manager.Manager = null,
|
||||
records: *const records.Records,
|
||||
zones: *const forward_zones.Zones,
|
||||
cache: ?*dns_cache.DnsCache = null,
|
||||
cache_mutex: std.Io.Mutex = .init,
|
||||
limiter: ?*rate_limiter.RateLimiter = null,
|
||||
limiter_mutex: std.Io.Mutex = .init,
|
||||
logger: ?*logger_mod.Logger = null,
|
||||
pause: ?*pause.Pause = null,
|
||||
tracker: ?*clients.Tracker = null,
|
||||
stats: Stats = .{},
|
||||
};
|
||||
```
|
||||
|
||||
Every `?*` defaults to null so existing tests and the check command construct a bare handler
|
||||
exactly as today (upstream + blocking + records/zones + timeout are the only required fields;
|
||||
`records`/`zones` may point at the shared `Records.empty`/`Zones.empty` constants — if
|
||||
`Zones.empty` does not exist yet, S2 adds it mirroring `Records.empty`).
|
||||
|
||||
```zig
|
||||
pub const Scratch = struct {
|
||||
normalize: [types.max_name_len]u8,
|
||||
key: [dns_cache.max_key_len]u8,
|
||||
build: [512]u8, // rewritten outgoing query (ECS strip / safe-search)
|
||||
frame: [forward_client.min_frame_buf]u8, // forward-zone client frame buffer
|
||||
};
|
||||
|
||||
pub fn handle(self: *Handler, io: std.Io, which: Transport, from: address.NetAddress,
|
||||
query: []const u8, response_buf: []u8, scratch: *Scratch) Outcome
|
||||
```
|
||||
|
||||
`Outcome` unchanged. Still no error union — every failure is a synthesized rcode or a counted
|
||||
drop. New Stats counters (all `std.atomic.Value(u64)`): `refused`, `blocked`,
|
||||
`uncloak_blocked`, `local_answers`, `forward_zone_answers`, `cache_hits`, `paused_queries`,
|
||||
`unfiltered_queries`, `safesearch_rewrites`, `ecs_strip_failed`, `tracker_full`.
|
||||
|
||||
### S2.3 Pipeline order inside handle (binding; rulings 4-13, 18-20 apply)
|
||||
|
||||
1. Read the 12-byte header (existing code). QR/opcode/qdcount checks as today.
|
||||
2. Rate limit: `limiter_mutex.lock(io)`, `check(now, from.key())`, unlock. Refused →
|
||||
synthesize REFUSED (reuse `synthesize`), count, return. `now` from
|
||||
`std.Io.Clock.awake.now(io)` (limiter windows are awake-anchored, milestone-6 As-built).
|
||||
3. Full parse + OPT validation (existing code). Record `start_us` once from
|
||||
`std.Io.Clock.real.now(io)` for both `response_time_us` and cache `now_s`.
|
||||
4. `tracker.track(from)` (nonblocking; full table counts `tracker_full`).
|
||||
5. Snapshot: `if (self.manager) |m| m.acquire(io) else null`; released once on every exit path
|
||||
(single defer). Null snapshot → `unfiltered_queries`, group features off (ruling 4).
|
||||
6. Non-IN qclass → step 12 (upstream direct), no cache (ruling 9).
|
||||
7. Normalize qname into `scratch.normalize`.
|
||||
8. Local records: `hasName` → `lookup`; answer via `ResponseBuilder` + `writeAnswers` +
|
||||
`addOptEcho`, authoritative. A name that exists with no records of the qtype → NODATA,
|
||||
authoritative. Log (`upstream="local"`, `cache_hit=null`) and return.
|
||||
9. Forward zones: `zones.match` → construct `ForwardClient` on the stack
|
||||
(`init(zone.resolver, &scratch.frame, self.forward_read_timeout)`), cache get → miss →
|
||||
`exchange` into `response_buf` → cache put (classify), log (`upstream=resolver text`),
|
||||
return. Filtering/safe-search/uncloak bypassed (ruling 7). Exchange failure → SERVFAIL.
|
||||
10. Pause check (`isPaused(now_s)`) → skip 11 and the uncloak in 14; count `paused_queries`.
|
||||
11. Filtering: `snapshot.evaluate(group, domain)`; blocked → `response.writeBlocked` into
|
||||
`response_buf`, log (`blocked=true`, reason), return. Then safe-search:
|
||||
`snapshot.safeSearch(group)` and `safesearch.rewrite(domain)` → remember the target;
|
||||
outgoing QNAME = target (query rebuilt in `scratch.build`; count `safesearch_rewrites`).
|
||||
12. ECS (ruling 13): `.strip` → `edns.stripEcs` into `scratch.build` (compose with the
|
||||
safe-search rebuild: build once with both the final QNAME and the filtered OPT — one
|
||||
rebuild, not two); `.forward` → `edns.ecsPayload` for the key. Overflow →
|
||||
`ecs_strip_failed`, forward original.
|
||||
13. Cache get (skip for safe-search, ruling 5): key = `buildKey(outgoing qname, qtype, qclass,
|
||||
do_bit, ecs_payload_or_null)` under `cache_mutex`; hit → copy is already in
|
||||
`response_buf`-sized `out` (pass `response_buf`), `packet.setId`, UDP truncation check as
|
||||
today, log (`cache_hit=true`, `upstream=""`), return.
|
||||
14. Upstream: `self.upstream.exchange(io, outgoing, response_buf)` (existing error mapping to
|
||||
SERVFAIL/drop stays). Then uncloak (ruling 11) unless paused/non-IN: iterate
|
||||
`packet.answers`, follow CNAMEs from the outgoing qname via `record.rdataCname`, depth ≤ 8,
|
||||
normalize each target, `evaluate`; blocked → `writeBlocked` for the ORIGINAL question,
|
||||
count `uncloak_blocked`, log (`blocked=true`, `"cname:"` reason), return.
|
||||
15. Safe-search synthesis (ruling 10): new `ResponseBuilder` with the original question,
|
||||
`addAnswer(original, CNAME, target)`, then each A/AAAA answer record of the upstream
|
||||
response verbatim (owner = target name), `addOptEcho`. The upstream bytes are read from
|
||||
`response_buf` while the builder writes into... **the builder needs its own buffer**: build
|
||||
into a second half — concretely `handle` requires `response_buf.len >= 1024` already via
|
||||
assert; S2 adds a `synth: [4096]u8` field to `Scratch` for the safe-search build, then
|
||||
copies the finished reply into `response_buf`. A synthesized reply that cannot fit 4096 →
|
||||
SERVFAIL (safe-search answers are tiny in practice).
|
||||
16. Cache put: upstream path only, not blocked/safe-search (ruling 5): `classify` →
|
||||
`cache_mutex` → `put`. Allocation failure inside put → skip caching (already
|
||||
`Allocator.Error`; count nothing, put's stats cover it).
|
||||
17. UDP truncation check (existing). Log the entry: `Entry.init(.{ .timestamp = now_s, .domain
|
||||
= normalized domain, .client_ip = formatted from, .qtype, .blocked=false, .response_time_us,
|
||||
.cache_hit=false, .upstream = per ruling 20 })`, `logger.log(io, entry)`. Return reply.
|
||||
|
||||
The query-log write happens on every replied path (blocked, local, forward, cache, upstream,
|
||||
paused) — one call site per return is acceptable; a tiny private `logEntry` helper keeps it
|
||||
readable. Refused/malformed/dropped are never logged (rulings 8, 20).
|
||||
|
||||
### S2.4 Acceptance
|
||||
|
||||
- [ ] `zig fmt --check` + `zig ast-check` clean; temp-root tests pass.
|
||||
- [ ] Existing handler tests still pass (bare handler, null optionals).
|
||||
- [ ] New in-file tests (fake `transport.Client` fixtures, as the existing tests do): REFUSED
|
||||
on limit exhaustion; local A answer + NODATA; forward-zone match exchanges via the zone
|
||||
resolver fake and bypasses a blocklisted name; blocked domain → zero-IP answer with
|
||||
blocking TTL; allow-over-block passes; cache miss→put→hit with aged TTL + fresh ID;
|
||||
blocked responses not cached; uncloak: CNAME chain to a blocked target at depth 1 and at
|
||||
depth 8 blocks, depth 9 does not, `cname:` reason logged; safe-search: rewritten QNAME
|
||||
goes upstream, reply carries original question + CNAME + A records only; ECS strip:
|
||||
option 8 gone upstream, others preserved; ECS forward: payload lands in the cache key
|
||||
(two subnets → two entries); paused: blocklisted name answers normally and counts;
|
||||
non-IN bypasses filtering; null manager → unfiltered count; every path's log entry
|
||||
fields per ruling 20 (capture via a Logger with a small queue, drain and assert).
|
||||
|
||||
### S2 As built
|
||||
|
||||
`Handler` gained one additive field: `negative_ttl_max: u32 = 0` (set from
|
||||
`cfg.cache.negative_ttl_max` — `DnsCache` keeps no copy of its config and `classify` needs the
|
||||
value; **S6 must wire it or negative caching is silently off**). `Scratch` gained `synth:
|
||||
[4096]u8` (spec-blessed) and `uncloak: [types.max_name_len]u8` (target normalization must not
|
||||
clobber `normalize`, which still holds the logged name). `Zones.empty` already existed
|
||||
(forward_zones.zig:34) — the S2.2 parenthetical was stale. Both mutexes and the tracker use
|
||||
`lockUncancelable` — `std.Io.Mutex.lock` returns `Cancelable!void` (Io.zig:1602) and `handle`
|
||||
has no error union. Safe-search + ECS strip compose via two buffers: rebuild into
|
||||
`scratch.build`, strip into `scratch.synth` (stripEcs requires non-overlapping output); `synth`
|
||||
is reused for the reply after the upstream answers. Uncloaking is also skipped after a
|
||||
safe-search rewrite (the answers belong to the provider's target, which the client never asked
|
||||
about). `safesearch_rewrites` counts at the rebuild, not the table lookup. SERVFAIL paths are
|
||||
counted but never query-logged (the Entry schema has no status field; a logged row would read
|
||||
as success — accepted ruling). Forward zones are exempt from the ECS strip (accepted ruling:
|
||||
the RFC 7871 concern is third-party upstreams, not the LAN resolver). Ruling 20's open choice:
|
||||
`upstream = "pool"` for pool answers; forward-zone rows log `cache_hit` false/true (ruling 20's
|
||||
"null" contradicted ruling 5 — null is now local records and blocked answers only). The
|
||||
safe-search CNAME TTL is the minimum TTL of the copied A/AAAA records, zero when none.
|
||||
`handle` keeps the 512-byte floor assert (the spec's "1024" was wrong); the copy out of
|
||||
`synth` is length-checked with SERVFAIL fallback. `tracker_full` mirrors
|
||||
`Tracker.snapshotStats().dropped_full` after each track (track returns nothing; an always-zero
|
||||
counter would lie). `max_cname_depth = 8` public. `queries` still means pool-answered queries.
|
||||
23 new handler tests + 25 adapted + 6 pause tests. After review, two ECS cache holes fixed at
|
||||
the root: (1) a failed ECS strip forwards the client's subnet upstream, so the answer may be
|
||||
subnet-specific while the `.strip`-mode key names no subnet — `outgoingQuery` returns
|
||||
`Outgoing { bytes, cacheable }` and a failed strip sets `cacheable = false`, skipping both the
|
||||
cache get and the put; (2) `.forward` mode forwards the whole OPT but `edns.findOption`
|
||||
reports only the first ECS option, so a query carrying several would key on one subnet while
|
||||
the resolver answered for another — a local `subnetForKey` walks the option list once and
|
||||
returns `.uncacheable` for a repeated option 8, for a payload over `dns_cache.max_ecs_len`,
|
||||
and for a list that will not walk; `cacheKey` then declines to cache. RFC 7871 §6 permits one
|
||||
ECS option, but a query is client-controlled, so a duplicate is treated as hostile. Both fixes
|
||||
are mutation-checked. `edns.ecsPayload` lost its only caller (the key needs the option count
|
||||
as well as the payload; one walk yields both) and is removed from edns.zig as dead surface. NOTE: the plain `zig build test` does not
|
||||
typecheck the listener call sites (serve is never analyzed there) — only `-Dintegration`
|
||||
validates S3's edit.
|
||||
|
||||
---
|
||||
|
||||
## Session S3: listeners pass the client address and scratch
|
||||
|
||||
Owns: `src/server/udp_server.zig`, `src/server/tcp_server.zig`, and the three integration test
|
||||
files whose `Handler{…}` literals the S2 struct change broke:
|
||||
`src/server/udp_server_integration_test.zig`, `src/server/tcp_server_integration_test.zig`,
|
||||
`src/server/resolver_integration_test.zig` (S2 measured nine stale literals missing
|
||||
`blocking`/`forward_read_timeout`/`records`/`zones` at udp:90/128/163/204, tcp:153/183/216/268,
|
||||
resolver:229, plus the handle-arity change once those are fixed).
|
||||
|
||||
- `Slot` (udp) gains `scratch: handler.Scratch`; the call becomes
|
||||
`self.handler.handle(io, .udp, address.NetAddress.fromIp(slot.from), slot.query[0..slot.len],
|
||||
&slot.reply, &slot.scratch)`.
|
||||
- `Conn` (tcp) gains `peer: std.Io.net.IpAddress` (captured at claim from
|
||||
`stream.socket.address`) and `scratch: handler.Scratch`; call updated the same way.
|
||||
- No other behavior change. Slot/Conn are pool-allocated already, so the added ~5 KiB per slot
|
||||
changes no lifetime.
|
||||
- Acceptance: fmt/ast clean; both files' existing tests updated and passing; integration tests
|
||||
(`-Dintegration` udp/tcp/resolver) still pass unchanged in behavior.
|
||||
|
||||
### S3 As built
|
||||
|
||||
`Conn.peer` is captured in `claim` inside the same mutex-held section that stores the stream —
|
||||
written once per connection, read only by that connection's task; verified from source that
|
||||
`accept` fills `Socket.address` with the peer (net.zig:1442 → Threaded.zig:12439
|
||||
`addressFromPosix`), and a new TCP integration test asserts the captured peer is the real
|
||||
loopback client (an uninitialized field would typecheck and silently corrupt rate
|
||||
limiting/grouping/logging). Importing the `address` module forced renaming the bind/listen
|
||||
address parameters to `bind_address`/`listen_address` (Zig shadowing rule; call sites are
|
||||
positional, nothing else changed). The nine stale integration literals became a per-file
|
||||
`bareHandler` fixture deriving `blocking` from `model.Blocking{}` and `forward_read_timeout`
|
||||
from `model.readTimeout(.{})` (3000 ms, `.awake`), so the fixtures cannot drift from config
|
||||
defaults. Memory-cost comments updated for the scratch: UDP ≈74 KiB/slot (≈4.6 MiB at 64),
|
||||
TCP ≈137 KiB/slot (≈8.8 MiB at 64); the UDP <8 MiB budget assertion still holds.
|
||||
After review: `TcpServer.serve` distinguishes its two exit paths, because the composition root
|
||||
cancels its task group before any listener `deinit` runs (the deinit defers are declared
|
||||
first, so `group.cancel` executes first). `acceptLoop` returns a `Stop` enum: `.closing` when
|
||||
`deinit` stopped it (`error.SocketNotListening`, a `.shutting_down` claim, or the state flag)
|
||||
and `.canceled` when the task is being canceled (`error.Canceled` from `accept` or the retry
|
||||
sleep). On `.closing` the connection group drains under `.blocked` protection as before —
|
||||
`beginShutdown` already unblocked every live stream, so the wait is bounded by the shutdown.
|
||||
On `.canceled` the group is canceled instead: nothing has shut the streams down, `deinit`
|
||||
cannot run until `serve` returns, and RFC 7766 §6.2.1.1 lets a client hold a connection open
|
||||
indefinitely by asking again inside the idle budget — draining would let one client stall
|
||||
process shutdown. The cost is the single reply mid-write. The old await under `.blocked`
|
||||
protection was what made cancel invisible: `Syscall.start` (Threaded.zig:1349) never arms
|
||||
cancellation while protection is blocked. Per-connection cleanup is unaffected (`serveConn`'s
|
||||
`defer finish` runs on the canceled path; `finish` already used `lockUncancelable` + `.blocked`
|
||||
around the close). Pinned by "a canceled serve does not wait for a live connection" (blocked
|
||||
read, 600 s idle budget, cancel must return; reinstating drain-always hangs the suite). The
|
||||
same await shape remains in `UdpServer.serve` deliberately: a UDP task is one bounded upstream
|
||||
exchange, not a client-paced session.
|
||||
|
||||
---
|
||||
|
||||
## Session S4: client tracker + repo additions
|
||||
|
||||
Owns: `src/server/clients.zig` (new), `src/storage/repositories/clients_repo.zig` (extend).
|
||||
|
||||
### S4.1 clients_repo additions (SQL; existing functions frozen)
|
||||
|
||||
```zig
|
||||
/// INSERT OR CONFLICT: materialize an unseen client (hand_edited=0, first_seen=last_seen=now)
|
||||
/// or touch last_seen on an existing row (hand-edited rows get last_seen touched too — the
|
||||
/// operator sees liveness; group/name/hand_edited never change here).
|
||||
pub fn upsertSeen(database: *db.Db, ip: []const u8, now_s: i64) db.Error!void
|
||||
/// DELETE hand_edited=0 AND last_seen < cutoff. Returns rows deleted.
|
||||
pub fn pruneStale(database: *db.Db, cutoff_s: i64) db.Error!u32
|
||||
```
|
||||
|
||||
`ON CONFLICT(ip) DO UPDATE SET last_seen=excluded.last_seen`. Check the actual clients schema
|
||||
(unique key on ip) in storage/config_schema.zig and use the real column names. The existing
|
||||
`insertClient` (hand_edited=1, import path) stays untouched.
|
||||
|
||||
### S4.2 Tracker
|
||||
|
||||
```zig
|
||||
pub const Tracker = struct {
|
||||
pub const max_pending = 512;
|
||||
pub const flush_interval_s = 60;
|
||||
pub const prune_every_passes = 1440; // daily at 60 s per pass
|
||||
mutex: std.Io.Mutex, // guards the fixed table below
|
||||
// fixed-capacity map, same house pattern as rate_limiter: slot array + index
|
||||
pub fn init(retention_days: u32) Tracker
|
||||
pub fn track(self: *Tracker, io: std.Io, addr: address.NetAddress) void // full → drop, count stats.dropped_full
|
||||
pub fn run(self: *Tracker, io: std.Io, database: *db.Db, monitor: ?*disk_monitor.Monitor)
|
||||
std.Io.Cancelable!void
|
||||
pub const Stats = struct { tracked: u64, flushed: u64, dropped_full: u64, pruned: u64, flush_failures: u64 };
|
||||
};
|
||||
```
|
||||
|
||||
`run` is the house loop shape (manager.zig:946): startup no-op, then every `flush_interval_s`
|
||||
on the `.boot` clock: skip the pass when `monitor` says writes are gated (ruling 17); else
|
||||
drain the pending table under the mutex into a local copy (release before I/O), one
|
||||
`upsertSeen` per client with `real` wall-clock seconds; every `prune_every_passes`-th pass also
|
||||
`pruneStale(now - retention_days * 86_400)`. A failed upsert warns once per pass and counts
|
||||
`flush_failures`; entries stay dropped (next query re-tracks). The database handle is the
|
||||
loop's own dedicated config.db connection (ruling 21) — document on `run` like retention does.
|
||||
IP text via `address.NetAddress.format` into a stack buffer.
|
||||
|
||||
Tests: track dedupes per flush (same client twice → one row, `last_seen` = latest), table
|
||||
full drops and counts, flush inserts hand_edited=0 rows and touches existing hand-edited rows'
|
||||
last_seen only, prune removes only stale hand_edited=0 rows, gated pass writes nothing
|
||||
(in-memory db + a Monitor fixture — Monitor's classify/state accessors allow constructing a
|
||||
gated state; if not constructible without I/O, test the gate branch through a bool seam
|
||||
`writes_allowed` parameter on a private `flushOnce` and have `run` pass the monitor read).
|
||||
|
||||
Acceptance: fmt/ast clean; temp-root (with `-lc -lsqlite3`) green; migration untouched.
|
||||
|
||||
### S4 As built
|
||||
|
||||
`init(retention_days: u16)` (matches `model.Logging.retention_days`). The pending table is a
|
||||
fixed 512-entry array with a linear scan, not a hash map — the frozen `init` takes no
|
||||
allocator, and rate_limiter's actual pattern is a pre-reserved hash map, so "same house
|
||||
pattern" did not apply; a linear scan over ≤512 entries of 17-byte keys is fine at household
|
||||
scale. Added: `trackAt(io, addr, now_s)` (timestamp seam; `track` = `trackAt` with real
|
||||
wall-clock), `flushOnce(io, database, writes_allowed: bool)` public (the gate seam `run` feeds
|
||||
from `monitor.writesAllowed()`; S7 case 10 forces passes with it), `snapshotStats(io)` and
|
||||
`pendingClients(io)` — `Stats` stays plain u64 and every field plus `passes` is written under
|
||||
`mutex`, so readers go through the accessors. `last_seen` is stamped at track time (it means
|
||||
"last query", and it is what makes latest-wins dedupe observable); the flush writes each
|
||||
entry's own stamp. A gated pass does not increment `passes` (a skipped pass must not advance
|
||||
the prune schedule). `track` uses `lockUncancelable` — `handle` has no error union, so `track`
|
||||
is not a cancelation point. `upsertSeen` resolves `group_id` via
|
||||
`(SELECT id FROM groups WHERE name = 'default')` — the column is NOT NULL REFERENCES groups
|
||||
(config_schema.zig:26), which ruling 14's sketch omitted; a database with no `default` group
|
||||
fails the whole statement with `error.Constraint` and writes nothing (validate.zig:461 already
|
||||
rejects such configs, so this is a warn-and-continue path in the flush loop, never live). The
|
||||
upsert's DO UPDATE touches `last_seen` only — `name`, `group_id`, `hand_edited`, `first_seen`
|
||||
survive on every existing row, hand-edited or not, one test per column. `pruneStale` uses
|
||||
strict `<` (matches `queries_repo.pruneOlderThan`) and clamps `sqlite3_changes` to u32.
|
||||
16 tests (9 tracker, 7 repo).
|
||||
|
||||
---
|
||||
|
||||
## Session S6: composition root — app.zig, shutdown.zig, cli wiring
|
||||
|
||||
Owns: `src/app.zig` (new), `src/server/shutdown.zig` (new), `src/cli.zig` (runRun body,
|
||||
DataDir), `src/main.zig` (only if a signature forces it — expected unchanged).
|
||||
|
||||
### S6.1 shutdown.zig
|
||||
|
||||
```zig
|
||||
var event: std.Io.Event = .unset;
|
||||
var event_io: ?std.Io = null; // set once by install before handlers exist
|
||||
pub fn install(io: std.Io) void // sigaction INT+TERM → handler calls event.set(event_io.?)
|
||||
pub fn wait(io: std.Io) std.Io.Cancelable!void // event.wait
|
||||
pub fn trigger(io: std.Io) void // tests and future admin use: event.set
|
||||
```
|
||||
|
||||
Handler fn: `callconv(.c)`, calls set only (async-signal-safe per ruling 2). `flags = 0`, empty
|
||||
mask (matches Threaded's own handlers, Threaded.zig:1653). Do not restore handlers on exit —
|
||||
the process is leaving anyway; `install` is once-per-process (assert).
|
||||
|
||||
### S6.2 app.zig — `pub fn run(runner: cli.Runner, paths: cli.Paths) u8`
|
||||
|
||||
Startup order (each failure prints to `runner.err` and returns `cli.exit_runtime`):
|
||||
|
||||
1. `DataDir.open` (S6 changes its `openDir` flags to `.{ .iterate = true }` — the monitor
|
||||
needs it; nothing else cares). `openConfigDb`. `migrations.migrate`.
|
||||
`bootstrap.bootstrap(...)` (first run seeds; diags printed like runCheck does).
|
||||
`export.readConfig` into an arena.
|
||||
2. `logging.install(io, cfg.logging, ...)` — the sink goes live (main.zig's std_options
|
||||
already routes).
|
||||
3. Build state, in dependency order, all heap-allocated where pinning is required (Logger,
|
||||
Manager, Pool entries and their DoH/DoT client structs must not move — same pattern as
|
||||
`cli.probeUpstreams` but with owned arrays that outlive the pool):
|
||||
records + zones (`local_repo` listers → `build`), fetcher + manager (`Manager.init`),
|
||||
pool (upstreams_repo → clients array → `Pool.init`), cache (`DnsCache.init`), limiter
|
||||
(`RateLimiter.init` from `cfg.dns`), pause, tracker, logger (`Logger.init` with a
|
||||
heap `queue_buf` of `cfg.logging.query_log_buffer_max` entries), monitor (`Monitor.init`
|
||||
with the data dir), retention (`Retention.init`), querylog connections: add
|
||||
`DataDir.openQuerylogDb(self, io) !db.Db` mirroring openConfigDb (same pragmas + 0600);
|
||||
open TWO (writer, retention) plus ONE extra config.db connection for the tracker
|
||||
(ruling 21).
|
||||
4. One synchronous `manager.reload(io)`; failure warns and continues (ruling 4).
|
||||
5. Handler construction (S2 struct) + `UdpServer.bind` + `TcpServer.listen` on
|
||||
`cfg.dns.bind_ipv4/bind_ipv6/port` — v4 and v6 both, matching PLAN §7.2 parity; a bind
|
||||
failure is fatal.
|
||||
6. `shutdown.install(io)`. Then one `std.Io.Group`:
|
||||
udp.serve, tcp.serve, logger.runWriter(io, &querylog_writer_db, &monitor),
|
||||
retention.run(io, &querylog_retention_db), monitor.run(io),
|
||||
manager.runScheduler(io), tracker.run(io, &tracker_db, &monitor),
|
||||
and a maintenance loop (private fn in app.zig, house shape, `.boot`, 60 s): cache sweep +
|
||||
limiter sweep under their mutexes... the mutexes live in the Handler; the maintenance loop
|
||||
takes `*Handler` and locks the same mutexes.
|
||||
7. `shutdown.wait(io)` (a Canceled result also proceeds to teardown). Then ruling 22 order:
|
||||
`logger.shutdown(io)` → `group.cancel(io)` → server deinits → pool/manager/state deinits →
|
||||
DB closes → DataDir.close → return `cli.exit_ok`.
|
||||
|
||||
Log a one-line startup summary (scope `.nxdns`, info): bind addresses, snapshot generation or
|
||||
"unfiltered", upstream count.
|
||||
|
||||
### S6.3 cli.zig
|
||||
|
||||
`runRun` body: `return app.run(r, paths);` — keep the doc comment honest. `DataDir.open` flag
|
||||
change + `openQuerylogDb` as above (both used by app; runCheck untouched).
|
||||
|
||||
### S6.4 Acceptance
|
||||
|
||||
- [ ] fmt/ast clean on all owned files; `zig build` (exe) succeeds — app.zig is reached from
|
||||
main, so full semantic analysis covers it.
|
||||
- [ ] `zig build test` green (existing cli tests untouched; app.zig gets a smoke test only if
|
||||
expressible without sockets — otherwise S7 owns runtime coverage; state that choice).
|
||||
- [ ] Manual smoke (orchestrator runs it): `nxdns run` against a scratch data dir starts,
|
||||
answers `dig @127.0.0.1 -p <port>`, and exits 0 on SIGTERM promptly.
|
||||
|
||||
### S6 As built
|
||||
|
||||
Startup order as specified, with these corrections. **Binding**: IPv6 binds first; an
|
||||
`AddressInUse` from the IPv4 bind that follows a wildcard IPv6 bind is dual-stack coverage
|
||||
(info line), not a failure — on Linux `net.ipv6.bindv6only=0` makes the v6 wildcard own the v4
|
||||
port, and `BindOptions.ip6_only` cannot prevent it because Threaded.zig:12274 sets IPV6_V6ONLY
|
||||
to 0 when the option is true (inverted vs its doc); a foreign IPv4 holder would have failed
|
||||
the v6 wildcard bind too, so the inference is sound. `NetAddress.fromIp` unmaps `::ffff:`
|
||||
clients, so v4 clients keep their real identity. A kernel without IPv6
|
||||
(`AddressFamilyUnsupported` and friends) warns and serves IPv4; every other bind failure is
|
||||
fatal. **Exit codes**: configuration faults (`NoUsableUpstreams`, `BadBindAddress`,
|
||||
`BadRateLimit` — zero rate fields are rejected before RateLimiter.init asserts) exit
|
||||
`cli.exit_check` with a "run 'nxdns check'" hint; everything else exits `cli.exit_runtime`.
|
||||
**Querylog open**: `openQuerylogDb` wraps `querylog_schema.open` (creation/repair lives there;
|
||||
chmod of the file plus both WAL sidecars happens after, missing sidecars ignored);
|
||||
`reopenQuerylogDb` opens the extra connections without repeating the O(size) quick_check.
|
||||
**Two `std.http.Client`s**: blocklist downloads must not queue DoH queries behind a
|
||||
multi-megabyte stream. Teardown is defer-driven; the invariants held are: group.cancel joins
|
||||
first, config.db closes after manager.deinit, DataDir closes last. `negative_ttl_max` and
|
||||
`forward_read_timeout` are wired (the S2 trap). `shutdown.zig` has `reset()` for tests and 2
|
||||
in-file tests (orchestrator wired it into tests.zig — app.zig analysis via cli does not
|
||||
collect tests). app.zig itself has no unit test; S7 case 11 is the runtime proof. Smoke test
|
||||
passed: boot, migration 0->2, dual-stack answers on 15353/15354, SERVFAIL from the dead
|
||||
upstream, SIGTERM exit 0 in 0.007 s, DBs 0600, zero err lines. The manager disk gate was wired in a follow-up S6 edit — see ruling 17's as-built text; the
|
||||
gate test drives `Monitor.state_raw` directly (no I/O) and pins no-monitor/ok/warn pass,
|
||||
critical gates-and-counts, recovery resumes. After review: the composition root takes one
|
||||
synchronous `monitor.sample(io)` after `Monitor.init`, before the `Io.Group` spawns anything —
|
||||
`Monitor` initializes to `.ok` and `run` samples inside its own task, so without it the
|
||||
writer/tracker/scheduler would consult a gate still saying "writes allowed" during the boot
|
||||
window on a critically full disk. A failed first sample warns and leaves `.ok` (an unreadable
|
||||
filesystem is not evidence of a full disk — the monitor's own documented policy). `run`'s own
|
||||
first sample repeats it (one extra statvfs + scan at boot; `run`'s sample-then-sleep contract
|
||||
stays untouched). The ordering is not unit-testable (serve binds sockets and opens four
|
||||
databases); S7's boot case plus the statement order is the evidence.
|
||||
|
||||
---
|
||||
|
||||
## Session S7: phase 7 integration tests
|
||||
|
||||
Owns: `src/server/phase7_integration_test.zig` (new; gated on `build_options.integration` like
|
||||
phase6). Fixtures may reuse the S8/M6 helper patterns (in-memory dbs, fake upstream sockets).
|
||||
|
||||
Cases (each end-to-end through a real `Handler` with real UDP sockets on 127.0.0.1 ephemeral
|
||||
ports; a fake upstream = a UDP socket task answering canned responses):
|
||||
|
||||
1. Blocked domain answers 0.0.0.0 with blocking TTL; log entry captured with reason.
|
||||
2. Allow-rule-over-blocklist resolves via upstream.
|
||||
3. Local record answers authoritatively without touching the fake upstream.
|
||||
4. Forward zone: matching suffix goes to the zone resolver socket (not the pool fake), a
|
||||
blocklisted name inside the zone still resolves (bypass), response cached (second query
|
||||
does not hit the zone socket).
|
||||
5. Cache: miss→hit, TTL aged, transaction ID fresh, `cache_hit` logged on the hit.
|
||||
6. Uncloak: fake upstream returns CNAME chain to a blocked target → zero-IP answer with
|
||||
`cname:` reason.
|
||||
7. Safe-search group: query for a safesearch.zig table domain returns CNAME + the fake
|
||||
upstream's A for the target; question is the original name.
|
||||
8. Rate limit: limit=2 window=60 → third query REFUSED.
|
||||
9. Pause: pauseFor(indefinite) lets a blocked domain resolve; unpause blocks it again.
|
||||
10. Tracker: after queries + one forced flush (`flushOnce` seam or short interval), the
|
||||
clients table holds the source IP with hand_edited=0.
|
||||
11. Full app boot: `app.run` in a task against a temp data dir (bootstrap from a minimal
|
||||
config.zon on an ephemeral port), one real resolve through it, `shutdown.trigger`, task
|
||||
returns 0. This is the lifecycle proof.
|
||||
|
||||
Acceptance: `zig build test -Dintegration` green, zero std.log.err emitted (binding logging
|
||||
policy), cases deterministic (no external network — the pool's upstream is the fake).
|
||||
|
||||
### S7 As built
|
||||
|
||||
Case 11 runs against a `std.testing.tmpDir` rather than a fixed scratch path (a committed
|
||||
test cannot hold a machine-specific absolute path; the phase6 fixture already resolves
|
||||
cwd-relative paths through both `std.Io.Dir` and SQLite). Port 15455, IPv4+IPv6 binds, the
|
||||
dead `https://127.0.0.1:9/dns-query` upstream, one seeded local record; `app.serve` installs
|
||||
and deinstalls its own log sink, so `installForTest` is unnecessary and the case asserts the
|
||||
error writer stayed empty; `shutdown.reset()` brackets the case. Case 5 proves TTL ageing
|
||||
with an entry planted ten seconds in the past under the handler's own key — the
|
||||
miss-hit-fresh-ID half is a real round trip; no test can advance the clock. Case 4's zone
|
||||
resolver is a UDP socket task answering with `ResponseBuilder` against the received query;
|
||||
the second query proves the cache by leaving the resolver's datagram count at 1. The fake
|
||||
pool upstream builds each answer from the question it receives (safe search and ECS rewrite
|
||||
the outgoing question, so canned bytes would not bind). A shared `Loop` fixture binds the
|
||||
listener and client before the serve task starts, so no task holds a pointer into a value
|
||||
that later moves. 11 tests, all gated on `build_options.integration`; the integration suite
|
||||
emits zero err-level lines.
|
||||
|
||||
---
|
||||
|
||||
## Module layout (new/changed)
|
||||
|
||||
| file | change |
|
||||
|---|---|
|
||||
| src/dns/edns.zig | +stripEcs (S1; ecsPayload added then removed in review) |
|
||||
| src/server/handler.zig | pipeline rewrite (S2) |
|
||||
| src/server/pause.zig | new (S2) |
|
||||
| src/server/udp_server.zig, tcp_server.zig | address + scratch plumbing (S3) |
|
||||
| src/server/clients.zig | new tracker (S4) |
|
||||
| src/storage/repositories/clients_repo.zig | +upsertSeen, +pruneStale (S4) |
|
||||
| src/app.zig | new composition root (S6) |
|
||||
| src/server/shutdown.zig | new (S6) |
|
||||
| src/cli.zig | runRun body, DataDir.iterate, openQuerylogDb (S6) |
|
||||
| src/server/phase7_integration_test.zig | new (S7) |
|
||||
|
||||
## File ownership
|
||||
|
||||
S1: dns/edns.zig. S2: server/handler.zig, server/pause.zig. S3: server/udp_server.zig,
|
||||
server/tcp_server.zig. S4: server/clients.zig, storage/repositories/clients_repo.zig.
|
||||
S6: app.zig, server/shutdown.zig, cli.zig, main.zig(if forced). S7: its test file.
|
||||
Orchestrator: src/tests.zig, specs. No parallel sessions share a file (S1∥S4; the rest are
|
||||
sequential).
|
||||
|
||||
## Acceptance (milestone complete)
|
||||
|
||||
- [ ] `zig build test` and `zig build test -Dintegration` green; `zig fmt --check` clean;
|
||||
`zig build cross` produces both static exes.
|
||||
- [ ] All S7 cases pass; the app boots, serves, and shuts down cleanly (case 11).
|
||||
- [ ] No std.log.err in any new code path exercised by tests (binding logging policy:
|
||||
err is reserved for swallowed failures).
|
||||
- [ ] Spec As-built sections synced per session.
|
||||
|
||||
## Anti-requirements
|
||||
|
||||
- No HTTP/SSE/metrics/auth (Phase 8). No DoH/DoT server (Phase 9). No packaging (Phase 10).
|
||||
- No regex rules, DHCP, DNSSEC validation, DoQ, clustering (PLAN §2.2).
|
||||
- No per-group cache, no cache persistence, no query re-resolution in uncloaking.
|
||||
- No new config fields, no schema migration, no pause persistence.
|
||||
- No benchmark harness this milestone.
|
||||
- dns/ stays pure (S1's additions are pure); filter/local/cache modules are not edited.
|
||||
+642
@@ -0,0 +1,642 @@
|
||||
//! The composition root: everything `nxdns run` owns, in the order it is built.
|
||||
//!
|
||||
//! Nothing else in the program constructs a collaborator. Each module takes its
|
||||
//! dependencies as parameters and knows nothing about who supplies them, so
|
||||
//! this file is the only place where the object graph exists — and the only
|
||||
//! place a lifetime spans the whole process.
|
||||
//!
|
||||
//! Two rules shape it:
|
||||
//!
|
||||
//! 1. **Nothing that is pointed at may move.** `Pool.Entry.client` erases a
|
||||
//! pointer into a DoH or DoT client, the `Logger`'s queue holds waiting
|
||||
//! tasks in intrusive lists, and the `Manager` is addressed through `self`.
|
||||
//! Every such value therefore lives in `serve`'s frame or in an owned heap
|
||||
//! allocation, never in a temporary that is copied afterwards.
|
||||
//! 2. **Every background loop is a task in one `std.Io.Group`.** Shutdown is
|
||||
//! `group.cancel`, which requests cancellation and joins, so no loop can
|
||||
//! still be running when the resources it borrows are released.
|
||||
//!
|
||||
//! Failure policy: a startup failure prints one line to the runner's error
|
||||
//! writer and exits. A configuration the operator can fix exits 2, so that
|
||||
//! `nxdns check` is the obvious next step; anything else exits 1. Once serving
|
||||
//! starts, nothing is fatal — the pipeline answers queries with a snapshot or
|
||||
//! without one (ruling 4), and every loop logs its own failures and keeps
|
||||
//! going.
|
||||
|
||||
const std = @import("std");
|
||||
const Allocator = std.mem.Allocator;
|
||||
const Certificate = std.crypto.Certificate;
|
||||
const Writer = std.Io.Writer;
|
||||
const net = std.Io.net;
|
||||
const tls = std.crypto.tls;
|
||||
|
||||
const bootstrap = @import("config/bootstrap.zig");
|
||||
const cli = @import("cli.zig");
|
||||
const clients = @import("server/clients.zig");
|
||||
const config_export = @import("config/export.zig");
|
||||
const disk_monitor = @import("storage/disk_monitor.zig");
|
||||
const dns_cache = @import("cache/dns_cache.zig");
|
||||
const doh_client = @import("upstream/doh_client.zig");
|
||||
const dot_client = @import("upstream/dot_client.zig");
|
||||
const fetcher = @import("filter/fetcher.zig");
|
||||
const forward_zones = @import("local/forward_zones.zig");
|
||||
const handler = @import("server/handler.zig");
|
||||
const local_records = @import("local/records.zig");
|
||||
const logger_mod = @import("storage/logger.zig");
|
||||
const logging = @import("platform/logging.zig");
|
||||
const manager_mod = @import("filter/manager.zig");
|
||||
const migrations = @import("storage/migrations.zig");
|
||||
const model = @import("config/model.zig");
|
||||
const pause = @import("server/pause.zig");
|
||||
const pool_mod = @import("upstream/pool.zig");
|
||||
const rate_limiter = @import("server/rate_limiter.zig");
|
||||
const retention_mod = @import("storage/retention.zig");
|
||||
const shutdown = @import("server/shutdown.zig");
|
||||
const tcp_server = @import("server/tcp_server.zig");
|
||||
const transport = @import("upstream/transport.zig");
|
||||
const udp_server = @import("server/udp_server.zig");
|
||||
const validate = @import("config/validate.zig");
|
||||
const version = @import("version.zig");
|
||||
|
||||
const log = std.log.scoped(.nxdns);
|
||||
|
||||
/// How long the cache and rate-limiter sweeps wait between passes. Both tables
|
||||
/// evict lazily on lookup as well; this is what keeps memory bounded for a
|
||||
/// client that asked once and never came back.
|
||||
const maintenance_interval_s = 60;
|
||||
|
||||
/// Bounds one blocklist download (`Manager.total_budget`). Generous, because a
|
||||
/// megabyte-scale list over a household uplink is normal, and a download that
|
||||
/// takes longer than this is not going to finish at all.
|
||||
const download_budget_s = 300;
|
||||
|
||||
/// Per DoH upstream. `min_request_buf` is 512; the extra room costs nothing and
|
||||
/// keeps a maximum-length name with a large OPT record comfortable.
|
||||
const doh_request_buf_len = 1024;
|
||||
const doh_transfer_buf_len = 4096;
|
||||
|
||||
/// A configuration fault the operator can fix, as opposed to a runtime one.
|
||||
/// These are the only failures this file raises itself; everything else comes
|
||||
/// out of a collaborator.
|
||||
const ConfigError = error{
|
||||
NoUsableUpstreams,
|
||||
BadBindAddress,
|
||||
BadRateLimit,
|
||||
};
|
||||
|
||||
pub fn run(runner: cli.Runner, paths: cli.Paths) u8 {
|
||||
const code = serve(runner, paths) catch |err| code: {
|
||||
runner.err.print("nxdns run failed: {s}\n", .{@errorName(err)}) catch {};
|
||||
if (isConfigFault(err)) {
|
||||
runner.err.writeAll("run `nxdns check` to see the configuration in full\n") catch {};
|
||||
break :code cli.exit_check;
|
||||
}
|
||||
break :code cli.exit_runtime;
|
||||
};
|
||||
|
||||
// Output the operator never received is not output, so a failed flush
|
||||
// outranks a successful run — the same rule `cli.finish` applies.
|
||||
runner.out.flush() catch return cli.exit_runtime;
|
||||
runner.err.flush() catch return cli.exit_runtime;
|
||||
return code;
|
||||
}
|
||||
|
||||
fn isConfigFault(err: anyerror) bool {
|
||||
return switch (err) {
|
||||
error.NoUsableUpstreams, error.BadBindAddress, error.BadRateLimit => true,
|
||||
else => false,
|
||||
};
|
||||
}
|
||||
|
||||
fn serve(r: cli.Runner, paths: cli.Paths) !u8 {
|
||||
const io = r.io;
|
||||
const gpa = r.gpa;
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// storage and configuration
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
var data = try cli.DataDir.open(io, gpa, paths.data_dir, true);
|
||||
defer data.close(io, gpa);
|
||||
|
||||
var config_db = try data.openConfigDb(io);
|
||||
defer config_db.close();
|
||||
_ = try migrations.migrate(&config_db);
|
||||
|
||||
{
|
||||
// First run only: the file seeds an empty database and is ignored
|
||||
// forever after. Its diagnostics are the operator's one chance to see
|
||||
// why a config file was rejected, so they are printed like `check`
|
||||
// prints them.
|
||||
var diags: validate.Diagnostics = .init(gpa);
|
||||
defer diags.deinit();
|
||||
_ = bootstrap.bootstrap(io, gpa, &config_db, std.Io.Dir.cwd(), paths.config, &diags) catch |err| {
|
||||
diags.writeAll(r.err) catch {};
|
||||
return err;
|
||||
};
|
||||
}
|
||||
|
||||
// Every string in `cfg` points into this arena, and the pool's endpoints,
|
||||
// the handler's records and the monitor's paths all keep such strings. It
|
||||
// therefore outlives all of them, which is why it is declared here and not
|
||||
// inside a narrower scope.
|
||||
var arena_state: std.heap.ArenaAllocator = .init(gpa);
|
||||
defer arena_state.deinit();
|
||||
const arena = arena_state.allocator();
|
||||
|
||||
const cfg = try config_export.readConfig(&config_db, arena);
|
||||
|
||||
// From here on `std.log` goes wherever the operator asked. Before this call
|
||||
// the sink passes through to stderr, which is where a failure above needs
|
||||
// to appear anyway.
|
||||
logging.install(io, cfg.logging);
|
||||
defer logging.deinstall();
|
||||
|
||||
if (cfg.dns.rate_limit == 0 or cfg.dns.rate_window_seconds == 0) return error.BadRateLimit;
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// local answers
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
var records = try local_records.Records.build(gpa, cfg.local_records);
|
||||
defer records.deinit(gpa);
|
||||
|
||||
var zones = try forward_zones.Zones.build(gpa, cfg.forward_zones);
|
||||
defer zones.deinit(gpa);
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// blocklists
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
// Two HTTP clients on purpose. A blocklist download streams tens of
|
||||
// megabytes and holds its connection for the whole of it; DoH queries must
|
||||
// not queue behind that, and the two have nothing to share but a type.
|
||||
var fetch_http: std.http.Client = .{ .allocator = gpa, .io = io };
|
||||
defer fetch_http.deinit();
|
||||
var dns_http: std.http.Client = .{ .allocator = gpa, .io = io };
|
||||
defer dns_http.deinit();
|
||||
|
||||
const fetch_transfer = try gpa.alloc(u8, fetcher.min_transfer_buf);
|
||||
defer gpa.free(fetch_transfer);
|
||||
const fetch_redirect = try gpa.alloc(u8, fetcher.redirect_buffer_len);
|
||||
defer gpa.free(fetch_redirect);
|
||||
var fetch: fetcher.Fetcher = .{
|
||||
.http = &fetch_http,
|
||||
.transfer_buf = fetch_transfer,
|
||||
.redirect_buf = fetch_redirect,
|
||||
};
|
||||
|
||||
var manager: manager_mod.Manager = try .init(
|
||||
gpa,
|
||||
&config_db,
|
||||
.{ .dir = data.dir },
|
||||
&fetch,
|
||||
cfg.blocklist_update,
|
||||
.{ .raw = .fromSeconds(download_budget_s), .clock = .awake },
|
||||
);
|
||||
defer manager.deinit(io);
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// upstream pool
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
// Shared by every DoT client: the trust store does not vary per upstream
|
||||
// and the scan is expensive. Both must outlive the clients that hold them.
|
||||
var bundle: Certificate.Bundle = .empty;
|
||||
defer bundle.deinit(gpa);
|
||||
var bundle_lock: std.Io.RwLock = .init;
|
||||
|
||||
var upstreams = try Upstreams.build(gpa, cfg.upstreams, &dns_http, &bundle, &bundle_lock);
|
||||
defer upstreams.deinit(gpa);
|
||||
|
||||
var pool: pool_mod.Pool = .init(
|
||||
upstreams.active(),
|
||||
.{},
|
||||
.{ .raw = model.totalTimeout(cfg.upstream), .clock = .awake },
|
||||
@truncate(@as(u96, @bitCast(std.Io.Clock.real.now(io).nanoseconds))),
|
||||
);
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// per-query state
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
var cache: dns_cache.DnsCache = try .init(gpa, cfg.cache);
|
||||
defer cache.deinit();
|
||||
|
||||
var limiter: rate_limiter.RateLimiter = try .init(gpa, .{
|
||||
.limit = cfg.dns.rate_limit,
|
||||
.window_seconds = cfg.dns.rate_window_seconds,
|
||||
});
|
||||
defer limiter.deinit();
|
||||
|
||||
var paused: pause.Pause = .{};
|
||||
var tracker: clients.Tracker = .init(cfg.logging.retention_days);
|
||||
|
||||
// The queue holds waiting tasks in intrusive lists, so neither the buffer
|
||||
// nor the `Logger` may move once a task has touched either.
|
||||
const queue_buf = try gpa.alloc(logger_mod.Entry, cfg.logging.query_log_buffer_max);
|
||||
defer gpa.free(queue_buf);
|
||||
var query_logger: logger_mod.Logger = .init(cfg.logging, queue_buf);
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// disk, retention and the remaining connections (ruling 21)
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
const data_path = try arena.dupeZ(u8, paths.data_dir);
|
||||
const log_dir_path: ?[:0]const u8 = if (cfg.logging.output == .file)
|
||||
try arena.dupeZ(u8, std.fs.path.dirname(cfg.logging.file_path) orelse ".")
|
||||
else
|
||||
null;
|
||||
var monitor: disk_monitor.Monitor = .init(cfg.disk, data.dir, data_path, log_dir_path);
|
||||
|
||||
// Ruling 17. The scheduler consults it before every scheduled pass; the
|
||||
// startup `reload` below is an operator action and stays ungated.
|
||||
manager.monitor = &monitor;
|
||||
|
||||
// One synchronous sample before anything can consult the gate. `Monitor`
|
||||
// initializes to `.ok`, and `Monitor.run` takes its first sample inside the
|
||||
// task — so without this call the log writer, the tracker and the blocklist
|
||||
// scheduler would all see "writes allowed" during the boot window and could
|
||||
// write to a critically full disk before the gate ever reflected it.
|
||||
// `run`'s own first sample repeats this one, which costs one extra statvfs
|
||||
// and directory scan at startup and keeps `run`'s "sample first, then
|
||||
// sleep" contract intact.
|
||||
//
|
||||
// `sample` returns nothing: every failure warns and counts inside the
|
||||
// monitor, and leaves the previous reading in place. A first sample that
|
||||
// fails therefore leaves the state at `.ok` — an unreadable filesystem is
|
||||
// not evidence that the disk is full, which is the monitor's documented
|
||||
// policy and the right one here too.
|
||||
monitor.sample(io);
|
||||
|
||||
var retention: retention_mod.Retention = .init(cfg.logging);
|
||||
|
||||
var querylog_writer_db = try data.openQuerylogDb(io);
|
||||
defer querylog_writer_db.close();
|
||||
var querylog_retention_db = try data.reopenQuerylogDb(io);
|
||||
defer querylog_retention_db.close();
|
||||
var tracker_db = try data.openConfigDb(io);
|
||||
defer tracker_db.close();
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// first snapshot
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
// Ruling 4: serving starts either way. A household loses more from a DNS
|
||||
// server that refuses to start than from one that answers unfiltered until
|
||||
// the scheduler's first pass succeeds.
|
||||
manager.reload(io) catch |err| {
|
||||
log.warn(
|
||||
"loading the blocklist snapshot failed ({s}); serving unfiltered until the next refresh",
|
||||
.{@errorName(err)},
|
||||
);
|
||||
};
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// listeners
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
var h: handler.Handler = .{
|
||||
.upstream = pool.client(),
|
||||
.blocking = .{ .mode = cfg.blocking.response, .ttl = cfg.blocking.ttl },
|
||||
.ecs_mode = cfg.edns.ecs_mode,
|
||||
.forward_read_timeout = .{ .raw = model.readTimeout(cfg.upstream), .clock = .awake },
|
||||
.manager = &manager,
|
||||
.records = &records,
|
||||
.zones = &zones,
|
||||
.cache = &cache,
|
||||
.negative_ttl_max = cfg.cache.negative_ttl_max,
|
||||
.limiter = &limiter,
|
||||
.logger = &query_logger,
|
||||
.pause = &paused,
|
||||
.tracker = &tracker,
|
||||
};
|
||||
|
||||
const v6_bind = parseBind(r, cfg.dns.bind_ipv6, cfg.dns.port, "dns.bind_ipv6") catch |err| return err;
|
||||
const v4_bind = parseBind(r, cfg.dns.bind_ipv4, cfg.dns.port, "dns.bind_ipv4") catch |err| return err;
|
||||
|
||||
// IPv6 first, and the order is load-bearing — see `Listeners`.
|
||||
var udp6: ?udp_server.UdpServer = udp_server.UdpServer.bind(gpa, io, v6_bind, &h, .{}) catch |err| bound: {
|
||||
if (!ipv6Unavailable(err)) return reportBind(r, "udp", v6_bind, err);
|
||||
log.warn("this system has no IPv6; serving IPv4 only", .{});
|
||||
break :bound null;
|
||||
};
|
||||
defer if (udp6) |*s| s.deinit(gpa, io);
|
||||
|
||||
const v6_is_wildcard = udp6 != null and isWildcard(v6_bind);
|
||||
|
||||
var udp4: ?udp_server.UdpServer = udp_server.UdpServer.bind(gpa, io, v4_bind, &h, .{}) catch |err| bound: {
|
||||
if (err != error.AddressInUse or !v6_is_wildcard) return reportBind(r, "udp", v4_bind, err);
|
||||
log.info("the IPv6 UDP listener is dual-stack and already serves IPv4", .{});
|
||||
break :bound null;
|
||||
};
|
||||
defer if (udp4) |*s| s.deinit(gpa, io);
|
||||
|
||||
var tcp6: ?tcp_server.TcpServer = tcp_server.TcpServer.listen(gpa, io, v6_bind, &h, .{}) catch |err| bound: {
|
||||
if (!ipv6Unavailable(err)) return reportBind(r, "tcp", v6_bind, err);
|
||||
break :bound null;
|
||||
};
|
||||
defer if (tcp6) |*s| s.deinit(gpa, io);
|
||||
|
||||
var tcp4: ?tcp_server.TcpServer = tcp_server.TcpServer.listen(gpa, io, v4_bind, &h, .{}) catch |err| bound: {
|
||||
if (err != error.AddressInUse or !(tcp6 != null and isWildcard(v6_bind))) {
|
||||
return reportBind(r, "tcp", v4_bind, err);
|
||||
}
|
||||
log.info("the IPv6 TCP listener is dual-stack and already serves IPv4", .{});
|
||||
break :bound null;
|
||||
};
|
||||
defer if (tcp4) |*s| s.deinit(gpa, io);
|
||||
|
||||
// -----------------------------------------------------------------------
|
||||
// run
|
||||
// -----------------------------------------------------------------------
|
||||
|
||||
shutdown.install(io);
|
||||
|
||||
// Declared after everything it borrows, so its `cancel` — which both
|
||||
// requests cancellation and joins — is the first thing that runs on the way
|
||||
// out (ruling 22). Nothing below this line may be released while a task
|
||||
// could still touch it.
|
||||
var group: std.Io.Group = .init;
|
||||
defer group.cancel(io);
|
||||
|
||||
if (udp6) |*s| try group.concurrent(io, udp_server.UdpServer.serve, .{ s, io });
|
||||
if (udp4) |*s| try group.concurrent(io, udp_server.UdpServer.serve, .{ s, io });
|
||||
if (tcp6) |*s| try group.concurrent(io, tcp_server.TcpServer.serve, .{ s, io });
|
||||
if (tcp4) |*s| try group.concurrent(io, tcp_server.TcpServer.serve, .{ s, io });
|
||||
|
||||
const gate: ?*disk_monitor.Monitor = &monitor;
|
||||
try group.concurrent(io, logger_mod.Logger.runWriter, .{ &query_logger, io, &querylog_writer_db, gate });
|
||||
try group.concurrent(io, retention_mod.Retention.run, .{ &retention, io, &querylog_retention_db });
|
||||
try group.concurrent(io, disk_monitor.Monitor.run, .{ &monitor, io });
|
||||
try group.concurrent(io, manager_mod.Manager.runScheduler, .{ &manager, io });
|
||||
try group.concurrent(io, clients.Tracker.run, .{ &tracker, io, &tracker_db, gate });
|
||||
try group.concurrent(io, runMaintenance, .{ &h, io });
|
||||
|
||||
logStartup(io, &manager, upstreams.active().len, .{
|
||||
.udp6 = if (udp6) |*s| s.boundAddress() else null,
|
||||
.udp4 = if (udp4) |*s| s.boundAddress() else null,
|
||||
.tcp6 = if (tcp6) |*s| s.boundAddress() else null,
|
||||
.tcp4 = if (tcp4) |*s| s.boundAddress() else null,
|
||||
});
|
||||
|
||||
// A canceled wait is a shutdown request too: whoever canceled this task
|
||||
// wants the process to stop, and the teardown below is how it stops.
|
||||
shutdown.wait(io) catch {};
|
||||
log.info("shutting down", .{});
|
||||
|
||||
// Before the group is canceled, so the writer sees a closed queue and
|
||||
// drains what it holds rather than losing it to cancellation (ruling 22).
|
||||
query_logger.shutdown(io);
|
||||
|
||||
return cli.exit_ok;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// background maintenance
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Sweeps the cache and the rate-limiter table. Both are guarded by mutexes the
|
||||
/// handler owns, because the handler is what contends for them; the sweeps live
|
||||
/// here because walking a whole table is not work a query should pay for.
|
||||
///
|
||||
/// The locks are taken cancelably: unlike `handle`, this loop has an error
|
||||
/// union to carry `error.Canceled` out of, and a shutdown that arrives while
|
||||
/// the query path holds a lock should not wait for it.
|
||||
fn runMaintenance(h: *handler.Handler, io: std.Io) std.Io.Cancelable!void {
|
||||
const interval: std.Io.Clock.Duration = .{
|
||||
.raw = .fromSeconds(maintenance_interval_s),
|
||||
.clock = .boot,
|
||||
};
|
||||
while (true) {
|
||||
try interval.sleep(io);
|
||||
|
||||
if (h.cache) |cache| {
|
||||
const now_s = std.Io.Clock.real.now(io).toSeconds();
|
||||
try h.cache_mutex.lock(io);
|
||||
_ = cache.sweep(now_s);
|
||||
h.cache_mutex.unlock(io);
|
||||
}
|
||||
|
||||
if (h.limiter) |limiter| {
|
||||
const now = std.Io.Clock.awake.now(io);
|
||||
try h.limiter_mutex.lock(io);
|
||||
_ = limiter.sweep(now);
|
||||
h.limiter_mutex.unlock(io);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// upstreams
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// The pool's entries and everything they point into.
|
||||
///
|
||||
/// `Pool.Entry.client` is a type-erased pointer into `doh` or `dot`, and each
|
||||
/// client borrows a slice of `doh_buf`/`dot_buf`, so all five allocations live
|
||||
/// exactly as long as the pool does. One entry is used by one task at a time
|
||||
/// (`Entry.busy`), which is why the buffers are per client and not shared the
|
||||
/// way `cli.probeUpstreams` shares them.
|
||||
const Upstreams = struct {
|
||||
entries: []pool_mod.Entry,
|
||||
used: usize,
|
||||
doh: []doh_client.DohClient,
|
||||
dot: []dot_client.DotClient,
|
||||
doh_buf: []u8,
|
||||
dot_buf: []u8,
|
||||
|
||||
/// A disabled upstream is left out entirely; a malformed one warns and is
|
||||
/// skipped, because one bad row in a table of four must not take DNS down.
|
||||
/// No usable row at all is a configuration fault.
|
||||
fn build(
|
||||
gpa: Allocator,
|
||||
servers: []const model.UpstreamServer,
|
||||
http: *std.http.Client,
|
||||
bundle: *Certificate.Bundle,
|
||||
bundle_lock: *std.Io.RwLock,
|
||||
) (Allocator.Error || ConfigError)!Upstreams {
|
||||
var enabled: usize = 0;
|
||||
for (servers) |server| {
|
||||
if (server.enabled) enabled += 1;
|
||||
}
|
||||
if (enabled == 0) return error.NoUsableUpstreams;
|
||||
|
||||
const chunk = tls.Client.min_buffer_len;
|
||||
|
||||
var self: Upstreams = .{
|
||||
.entries = try gpa.alloc(pool_mod.Entry, enabled),
|
||||
.used = 0,
|
||||
.doh = &.{},
|
||||
.dot = &.{},
|
||||
.doh_buf = &.{},
|
||||
.dot_buf = &.{},
|
||||
};
|
||||
errdefer self.deinit(gpa);
|
||||
|
||||
self.doh = try gpa.alloc(doh_client.DohClient, enabled);
|
||||
self.dot = try gpa.alloc(dot_client.DotClient, enabled);
|
||||
self.doh_buf = try gpa.alloc(u8, enabled * (doh_request_buf_len + doh_transfer_buf_len));
|
||||
self.dot_buf = try gpa.alloc(u8, enabled * 4 * chunk);
|
||||
|
||||
var doh_count: usize = 0;
|
||||
var dot_count: usize = 0;
|
||||
|
||||
for (servers) |server| {
|
||||
if (!server.enabled) continue;
|
||||
|
||||
const endpoint = transport.Endpoint.parse(server.url) catch {
|
||||
log.warn("upstream '{s}' is not an https:// or tls:// endpoint; skipped", .{server.url});
|
||||
continue;
|
||||
};
|
||||
|
||||
const client: transport.Client = switch (endpoint.scheme) {
|
||||
.doh => doh: {
|
||||
const base = doh_count * (doh_request_buf_len + doh_transfer_buf_len);
|
||||
const slot = &self.doh[doh_count];
|
||||
slot.* = doh_client.DohClient.init(
|
||||
http,
|
||||
endpoint,
|
||||
self.doh_buf[base..][0..doh_request_buf_len],
|
||||
self.doh_buf[base + doh_request_buf_len ..][0..doh_transfer_buf_len],
|
||||
) catch {
|
||||
log.warn("upstream '{s}' is not a usable DoH url; skipped", .{server.url});
|
||||
continue;
|
||||
};
|
||||
doh_count += 1;
|
||||
break :doh slot.client();
|
||||
},
|
||||
.dot => dot: {
|
||||
const base = dot_count * 4 * chunk;
|
||||
const slot = &self.dot[dot_count];
|
||||
slot.* = dot_client.DotClient.init(endpoint, server.tls_name, gpa, bundle, bundle_lock, .{
|
||||
.tls_read = self.dot_buf[base..][0..chunk],
|
||||
.tls_write = self.dot_buf[base + chunk ..][0..chunk],
|
||||
.stream_read = self.dot_buf[base + 2 * chunk ..][0..chunk],
|
||||
.stream_write = self.dot_buf[base + 3 * chunk ..][0..chunk],
|
||||
});
|
||||
dot_count += 1;
|
||||
break :dot slot.client();
|
||||
},
|
||||
};
|
||||
|
||||
self.entries[self.used] = .{
|
||||
.endpoint = endpoint,
|
||||
.client = client,
|
||||
.priority = server.priority,
|
||||
.enabled = true,
|
||||
.health = .init,
|
||||
};
|
||||
self.used += 1;
|
||||
}
|
||||
|
||||
if (self.used == 0) return error.NoUsableUpstreams;
|
||||
return self;
|
||||
}
|
||||
|
||||
/// The prefix `Pool.init` is given. The rest of `entries` is allocated but
|
||||
/// never filled, which is what keeps `deinit` able to free the whole block.
|
||||
fn active(self: *Upstreams) []pool_mod.Entry {
|
||||
return self.entries[0..self.used];
|
||||
}
|
||||
|
||||
fn deinit(self: *Upstreams, gpa: Allocator) void {
|
||||
gpa.free(self.dot_buf);
|
||||
gpa.free(self.doh_buf);
|
||||
gpa.free(self.dot);
|
||||
gpa.free(self.doh);
|
||||
gpa.free(self.entries);
|
||||
self.* = undefined;
|
||||
}
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// listeners
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// The addresses actually bound, for the startup line. A null is a family this
|
||||
/// process does not listen on separately — either because the system has no
|
||||
/// IPv6, or because the IPv6 socket is dual-stack and already serves IPv4.
|
||||
const Listeners = struct {
|
||||
udp6: ?net.IpAddress,
|
||||
udp4: ?net.IpAddress,
|
||||
tcp6: ?net.IpAddress,
|
||||
tcp4: ?net.IpAddress,
|
||||
};
|
||||
|
||||
fn parseBind(r: cli.Runner, text: []const u8, port: u16, field: []const u8) !net.IpAddress {
|
||||
return net.IpAddress.parse(text, port) catch {
|
||||
r.err.print("{s}: '{s}' is not an IP address\n", .{ field, text }) catch {};
|
||||
return error.BadBindAddress;
|
||||
};
|
||||
}
|
||||
|
||||
fn reportBind(r: cli.Runner, which: []const u8, addr: net.IpAddress, err: anyerror) anyerror {
|
||||
r.err.print("cannot bind {s} {f}: {s}\n", .{ which, addr, @errorName(err) }) catch {};
|
||||
return err;
|
||||
}
|
||||
|
||||
/// The kernel has no IPv6 at all. Refusing to start would be the wrong answer
|
||||
/// for the default configuration, which asks for both families.
|
||||
fn ipv6Unavailable(err: anyerror) bool {
|
||||
return switch (err) {
|
||||
error.AddressFamilyUnsupported,
|
||||
error.ProtocolUnsupportedBySystem,
|
||||
error.ProtocolUnsupportedByAddressFamily,
|
||||
=> true,
|
||||
else => false,
|
||||
};
|
||||
}
|
||||
|
||||
fn isWildcard(addr: net.IpAddress) bool {
|
||||
switch (addr) {
|
||||
.ip4 => |a| {
|
||||
for (a.bytes) |b| if (b != 0) return false;
|
||||
},
|
||||
.ip6 => |a| {
|
||||
for (a.bytes) |b| if (b != 0) return false;
|
||||
},
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// startup line
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// One line, at info, naming what an operator needs to see in `journalctl`
|
||||
/// right after a restart: where it listens, how many upstreams it has, and
|
||||
/// whether filtering is live.
|
||||
fn logStartup(io: std.Io, manager: *manager_mod.Manager, upstream_count: usize, bound: Listeners) void {
|
||||
var buf: [256]u8 = undefined;
|
||||
var w: Writer = .fixed(&buf);
|
||||
appendBind(&w, "udp", bound.udp6);
|
||||
appendBind(&w, "udp", bound.udp4);
|
||||
appendBind(&w, "tcp", bound.tcp6);
|
||||
appendBind(&w, "tcp", bound.tcp4);
|
||||
|
||||
// The generation is read under the snapshot handle's shared lock, which is
|
||||
// the same lock the reload took to publish it.
|
||||
if (manager.acquire(io)) |snapshot| {
|
||||
defer snapshot.release(io);
|
||||
log.info("nxdns {s} serving on{s}; {d} upstream(s); blocklist generation {d}", .{
|
||||
version.string,
|
||||
w.buffered(),
|
||||
upstream_count,
|
||||
manager.generation,
|
||||
});
|
||||
} else {
|
||||
log.info("nxdns {s} serving on{s}; {d} upstream(s); unfiltered (no blocklist snapshot)", .{
|
||||
version.string,
|
||||
w.buffered(),
|
||||
upstream_count,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/// Silent on overflow: a truncated startup line is not worth a failure path,
|
||||
/// and 256 bytes hold four addresses.
|
||||
fn appendBind(w: *Writer, which: []const u8, addr: ?net.IpAddress) void {
|
||||
const value = addr orelse return;
|
||||
w.print(" {s} {f}", .{ which, value }) catch {};
|
||||
}
|
||||
+59
-13
@@ -18,12 +18,14 @@ const Writer = std.Io.Writer;
|
||||
const Certificate = std.crypto.Certificate;
|
||||
const tls = std.crypto.tls;
|
||||
|
||||
const app = @import("app.zig");
|
||||
const config_export = @import("config/export.zig");
|
||||
const import = @import("config/import.zig");
|
||||
const model = @import("config/model.zig");
|
||||
const validate = @import("config/validate.zig");
|
||||
const db = @import("storage/db.zig");
|
||||
const migrations = @import("storage/migrations.zig");
|
||||
const querylog_schema = @import("storage/querylog_schema.zig");
|
||||
const doh_client = @import("upstream/doh_client.zig");
|
||||
const dot_client = @import("upstream/dot_client.zig");
|
||||
const pool = @import("upstream/pool.zig");
|
||||
@@ -204,7 +206,9 @@ pub const DataDir = struct {
|
||||
_ = try std.Io.Dir.cwd().createDirPathStatus(io, data_dir, .fromMode(0o700));
|
||||
}
|
||||
|
||||
var dir = try std.Io.Dir.cwd().openDir(io, data_dir, .{});
|
||||
// `.iterate`: the disk monitor sizes the databases by scanning this
|
||||
// directory, and an fd opened without it cannot be read as a directory.
|
||||
var dir = try std.Io.Dir.cwd().openDir(io, data_dir, .{ .iterate = true });
|
||||
errdefer dir.close(io);
|
||||
|
||||
// SQLite opens by path, not by directory handle, so both paths are
|
||||
@@ -243,6 +247,54 @@ pub const DataDir = struct {
|
||||
try db.applyPragmas(&database, .{});
|
||||
return database;
|
||||
}
|
||||
|
||||
/// The first connection to `querylog.db`: `querylog_schema.open` creates the
|
||||
/// file when it is missing and recreates it when it is unusable, so this is
|
||||
/// the call that establishes the schema. `reopenQuerylogDb` is for the
|
||||
/// connections that follow.
|
||||
///
|
||||
/// The 0600 chmod cannot come first the way `openConfigDb` does it — the
|
||||
/// file may not exist yet, and a recreate replaces it — so the main file and
|
||||
/// both WAL sidecars are locked down afterwards instead. A query log holds
|
||||
/// every domain every client asked for, which is as sensitive as anything in
|
||||
/// `config.db`.
|
||||
///
|
||||
/// `querylog_schema.open` resolves the path through SQLite's VFS as well as
|
||||
/// through the directory handle, so it is given `cwd` and the joined path
|
||||
/// rather than `self.dir` and a name (see its doc comment).
|
||||
pub fn openQuerylogDb(self: *const DataDir, io: std.Io) !db.Db {
|
||||
const opened = try querylog_schema.open(io, std.Io.Dir.cwd(), self.querylog_db_path);
|
||||
var database = opened.database;
|
||||
errdefer database.close();
|
||||
try self.restrictQuerylogPermissions(io);
|
||||
return database;
|
||||
}
|
||||
|
||||
/// An additional connection to a `querylog.db` that `openQuerylogDb` has
|
||||
/// already established. Phase 7 needs two — the log writer and the retention
|
||||
/// pass each own one (`retention.zig`'s contract).
|
||||
pub fn reopenQuerylogDb(self: *const DataDir, io: std.Io) !db.Db {
|
||||
_ = io;
|
||||
var database = try db.Db.open(self.querylog_db_path, .{ .mode = .read_write_existing });
|
||||
errdefer database.close();
|
||||
try db.applyPragmas(&database, .{});
|
||||
return database;
|
||||
}
|
||||
|
||||
/// A sidecar that does not exist yet is not a failure: `-wal` and `-shm`
|
||||
/// appear when SQLite first writes, and the next call catches them.
|
||||
fn restrictQuerylogPermissions(self: *const DataDir, io: std.Io) !void {
|
||||
for ([_][]const u8{
|
||||
querylog_db_name,
|
||||
querylog_db_name ++ "-wal",
|
||||
querylog_db_name ++ "-shm",
|
||||
}) |entry| {
|
||||
self.dir.setFilePermissions(io, entry, .fromMode(0o600), .{}) catch |e| switch (e) {
|
||||
error.FileNotFound => {},
|
||||
else => |other| return other,
|
||||
};
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -320,12 +372,11 @@ pub fn runVersion(r: Runner) u8 {
|
||||
return finish(r, exit_ok);
|
||||
}
|
||||
|
||||
/// Unchanged from milestone 1. Wiring the configuration into the servers is
|
||||
/// Phase 7; there is deliberately no half-built serving path here.
|
||||
/// Serves DNS until SIGINT or SIGTERM. The whole of it lives in `app.zig`,
|
||||
/// which is where the composition root belongs; this stays the entry point so
|
||||
/// that `main` dispatches every command the same way.
|
||||
pub fn runRun(r: Runner, paths: Paths) u8 {
|
||||
_ = paths;
|
||||
r.out.writeAll("not implemented\n") catch return finish(r, exit_runtime);
|
||||
return finish(r, exit_check);
|
||||
return app.run(r, paths);
|
||||
}
|
||||
|
||||
pub fn runExport(r: Runner, args: ExportArgs) u8 {
|
||||
@@ -879,13 +930,8 @@ test "runVersion prints the milestone-1 version lines and exits 0" {
|
||||
try testing.expectEqual(@as(usize, 2), countLines(captured.out.written()));
|
||||
}
|
||||
|
||||
test "runRun still reports that serving is not implemented" {
|
||||
var captured: Captured = .init(testing.allocator);
|
||||
defer captured.deinit();
|
||||
|
||||
try testing.expectEqual(exit_check, runRun(captured.runner(), .{}));
|
||||
try testing.expectEqualStrings("not implemented\n", captured.out.written());
|
||||
}
|
||||
// `runRun` now binds sockets and serves until a signal arrives, so it has no
|
||||
// unit test: booting it is `src/server/phase7_integration_test.zig`'s case 11.
|
||||
|
||||
test "runUsageError names the fault and prints the usage text" {
|
||||
var captured: Captured = .init(testing.allocator);
|
||||
|
||||
@@ -10,6 +10,8 @@
|
||||
const std = @import("std");
|
||||
const types = @import("types.zig");
|
||||
const record = @import("record.zig");
|
||||
const question = @import("question.zig");
|
||||
const packet_mod = @import("packet.zig");
|
||||
const Writer = std.Io.Writer;
|
||||
|
||||
/// The EDNS Client Subnet option code (RFC 7871 §6).
|
||||
@@ -159,6 +161,111 @@ fn ttlFrom(opt: OptRecord) u32 {
|
||||
(@as(u32, @intFromBool(opt.do_bit)) << 15);
|
||||
}
|
||||
|
||||
pub const StripResult = union(enum) {
|
||||
/// The query needs no rewrite and nothing was written to `out`.
|
||||
unchanged,
|
||||
/// The rewritten query, a prefix of `out`.
|
||||
rewritten: []u8,
|
||||
};
|
||||
|
||||
/// Byte offset of RDLENGTH inside an OPT record as `encodeOpt` writes it: a
|
||||
/// one-byte root owner name, TYPE, CLASS and TTL come first.
|
||||
const opt_rdlength_offset = 1 + 2 + 2 + 4;
|
||||
|
||||
/// Rebuilds `query` into `out` without its ECS option, so that a client's
|
||||
/// subnet never reaches the upstream resolver (PLAN §6.1). Every byte outside
|
||||
/// the OPT record is copied verbatim — ID, flags, all four section counts, the
|
||||
/// question and any record beside the OPT — and the OPT keeps its payload size,
|
||||
/// its flags and every option other than code 8.
|
||||
///
|
||||
/// `pkt` and `opt` are the caller's already-parsed views of `query`, and `out`
|
||||
/// must not overlap `query`.
|
||||
///
|
||||
/// One byte pattern does not survive the rewrite: a compression pointer in a
|
||||
/// record that follows the OPT and targets a byte inside the OPT's option list.
|
||||
/// Removing an option shifts those bytes, so the pointer then decodes to
|
||||
/// something else. RFC 1035 §4.1.4 only ever points a name at an earlier *name*,
|
||||
/// so such a pointer is malformed to begin with, and the upstream resolver
|
||||
/// answers the rewritten query with FORMERR instead of the original's answer.
|
||||
pub fn stripEcs(
|
||||
query: []const u8,
|
||||
pkt: packet_mod.Packet,
|
||||
opt: OptRecord,
|
||||
out: []u8,
|
||||
) error{ BadOption, Overflow }!StripResult {
|
||||
std.debug.assert(query.ptr == pkt.bytes.ptr);
|
||||
std.debug.assert(query.len == pkt.bytes.len);
|
||||
|
||||
// The rebuild reads `query` while it writes `out`, so an overlap would let
|
||||
// it consume bytes it has already overwritten. The two buffers must be
|
||||
// disjoint, and the caller finds that out here rather than in the packet it
|
||||
// sends upstream.
|
||||
std.debug.assert(@intFromPtr(out.ptr) + out.len <= @intFromPtr(query.ptr) or
|
||||
@intFromPtr(query.ptr) + query.len <= @intFromPtr(out.ptr));
|
||||
|
||||
const opt_start = (try optRecordStart(pkt, opt)) orelse return .unchanged;
|
||||
if ((try findOption(query, opt, ecs_option_code)) == null) return .unchanged;
|
||||
|
||||
var w = Writer.fixed(out);
|
||||
w.writeAll(query[0..opt_start]) catch return error.Overflow;
|
||||
encodeOpt(opt, &.{}, &w) catch |err| switch (err) {
|
||||
error.OptionsTooLong => unreachable, // the list written here is empty
|
||||
error.WriteFailed => return error.Overflow,
|
||||
};
|
||||
|
||||
// RDLENGTH is only known once the list has been filtered, and the surviving
|
||||
// options are not contiguous in `query`, so `encodeOpt` writes a placeholder
|
||||
// and the options stream in behind it.
|
||||
var kept_len: usize = 0;
|
||||
var it = options(query, opt);
|
||||
while (try it.next()) |o| {
|
||||
if (o.code == ecs_option_code) continue;
|
||||
var option_header: [4]u8 = undefined;
|
||||
std.mem.writeInt(u16, option_header[0..2], o.code, .big);
|
||||
std.mem.writeInt(u16, option_header[2..4], @intCast(o.data.len), .big);
|
||||
w.writeAll(&option_header) catch return error.Overflow;
|
||||
w.writeAll(o.data) catch return error.Overflow;
|
||||
kept_len += option_header.len + o.data.len;
|
||||
}
|
||||
|
||||
w.writeAll(query[opt.options.offset + opt.options.len ..]) catch return error.Overflow;
|
||||
|
||||
const message = w.buffered();
|
||||
std.mem.writeInt(u16, message[opt_start + opt_rdlength_offset ..][0..2], @intCast(kept_len), .big);
|
||||
return .{ .rewritten = message };
|
||||
}
|
||||
|
||||
/// Where the record holding `opt` begins, or null when `pkt` has no such
|
||||
/// record. A record's start offset is not part of a parsed view — only a walk
|
||||
/// of every section before it reaches that byte — so this repeats the walk
|
||||
/// `packet.parse` already did.
|
||||
///
|
||||
/// The match is on the whole RDATA span, which keeps a hand-built `OptRecord`
|
||||
/// from pointing this function at a different record than the one it describes.
|
||||
/// A section that will not walk lands on `error.BadOption` for the same reason:
|
||||
/// a `Packet` from `packet.parse` always walks, so only a `Packet` assembled by
|
||||
/// hand around unvalidated bytes gets here.
|
||||
fn optRecordStart(pkt: packet_mod.Packet, opt: OptRecord) error{BadOption}!?usize {
|
||||
var pos: usize = types.header_len;
|
||||
var q: u16 = 0;
|
||||
while (q < pkt.header.qdcount) : (q += 1) {
|
||||
const parsed = question.parse(pkt.bytes, pos) catch return error.BadOption;
|
||||
pos = parsed.end;
|
||||
}
|
||||
|
||||
const record_count = @as(u32, pkt.header.ancount) +
|
||||
@as(u32, pkt.header.nscount) + @as(u32, pkt.header.arcount);
|
||||
var r: u32 = 0;
|
||||
while (r < record_count) : (r += 1) {
|
||||
const parsed = record.parse(pkt.bytes, pos) catch return error.BadOption;
|
||||
if (parsed.record.rtype == .opt and
|
||||
parsed.record.rdata.offset == opt.options.offset and
|
||||
parsed.record.rdata.len == opt.options.len) return pos;
|
||||
pos = parsed.end;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/// The full 12-bit RCODE (RFC 6891 §6.1.3): the OPT record supplies the upper
|
||||
/// eight bits, the header the lower four. Without an OPT record the value is
|
||||
/// just the header's four bits.
|
||||
@@ -406,6 +513,172 @@ test "encodeOpt reports a short buffer" {
|
||||
try testing.expectError(error.WriteFailed, encodeOpt(opt, "", &w));
|
||||
}
|
||||
|
||||
/// A query for example.com A whose OPT record carries an ECS option for
|
||||
/// 192.0.2.0/24 followed by a two-byte padding option, with DO set and a
|
||||
/// 4096-byte payload size.
|
||||
const ecs_query =
|
||||
"\x12\x34\x01\x00\x00\x01\x00\x00\x00\x00\x00\x01" ++ // header
|
||||
"\x07example\x03com\x00\x00\x01\x00\x01" ++ // question
|
||||
"\x00\x00\x29\x10\x00\x00\x00\x80\x00\x00\x11" ++ // OPT, 17 rdata bytes
|
||||
"\x00\x08\x00\x07\x00\x01\x18\x00\xc0\x00\x02" ++ // ECS
|
||||
"\x00\x0c\x00\x02\x00\x00"; // padding
|
||||
|
||||
/// `ecs_query` as `stripEcs` must rebuild it.
|
||||
const ecs_query_stripped =
|
||||
"\x12\x34\x01\x00\x00\x01\x00\x00\x00\x00\x00\x01" ++
|
||||
"\x07example\x03com\x00\x00\x01\x00\x01" ++
|
||||
"\x00\x00\x29\x10\x00\x00\x00\x80\x00\x00\x06" ++
|
||||
"\x00\x0c\x00\x02\x00\x00";
|
||||
|
||||
const ParsedQuery = struct { pkt: packet_mod.Packet, opt: OptRecord };
|
||||
|
||||
fn parseQuery(bytes: []const u8) !ParsedQuery {
|
||||
const p = try packet_mod.parse(bytes);
|
||||
return .{ .pkt = p, .opt = try parseOpt(bytes, packet_mod.findOptRecord(p).?) };
|
||||
}
|
||||
|
||||
fn expectUnchanged(result: StripResult) !void {
|
||||
switch (result) {
|
||||
.unchanged => {},
|
||||
.rewritten => return error.TestExpectedUnchanged,
|
||||
}
|
||||
}
|
||||
|
||||
test "stripEcs rebuilds the query without the ECS option" {
|
||||
const parsed = try parseQuery(ecs_query);
|
||||
var out: [512]u8 = undefined;
|
||||
const result = try stripEcs(ecs_query, parsed.pkt, parsed.opt, &out);
|
||||
const rewritten = result.rewritten;
|
||||
try testing.expectEqualSlices(u8, ecs_query_stripped, rewritten);
|
||||
|
||||
const p = try packet_mod.parse(rewritten);
|
||||
try testing.expectEqual(@as(u16, 0x1234), p.header.id);
|
||||
try testing.expectEqual(@as(u16, 1), p.header.qdcount);
|
||||
try testing.expectEqual(@as(u16, 1), p.header.arcount);
|
||||
|
||||
const opt = try parseOpt(rewritten, packet_mod.findOptRecord(p).?);
|
||||
try testing.expectEqual(@as(u16, 4096), opt.udp_payload_size);
|
||||
try testing.expectEqual(true, opt.do_bit);
|
||||
try testing.expectEqual(@as(?Option, null), try findOption(rewritten, opt, ecs_option_code));
|
||||
try testing.expectEqualSlices(u8, "\x00\x00", (try findOption(rewritten, opt, 12)).?.data);
|
||||
}
|
||||
|
||||
test "stripEcs reports a query with nothing to strip as unchanged" {
|
||||
const parsed = try parseQuery(ecs_query_stripped);
|
||||
var out: [512]u8 = undefined;
|
||||
try expectUnchanged(try stripEcs(ecs_query_stripped, parsed.pkt, parsed.opt, &out));
|
||||
|
||||
// No OPT record at all: the caller still holds an `OptRecord`, so the
|
||||
// absence has to be found by the walk rather than assumed.
|
||||
const no_opt = "\x12\x34\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00" ++
|
||||
"\x07example\x03com\x00\x00\x01\x00\x01";
|
||||
const p = try packet_mod.parse(no_opt);
|
||||
try testing.expectEqual(@as(?record.Record, null), packet_mod.findOptRecord(p));
|
||||
const absent: OptRecord = .{
|
||||
.udp_payload_size = 4096,
|
||||
.extended_rcode = 0,
|
||||
.version = 0,
|
||||
.do_bit = false,
|
||||
.options = .{ .offset = 0, .len = 0 },
|
||||
};
|
||||
try expectUnchanged(try stripEcs(no_opt, p, absent, &out));
|
||||
}
|
||||
|
||||
test "stripEcs keeps a record that follows the OPT" {
|
||||
const head = "\x12\x34\x01\x00\x00\x01\x00\x00\x00\x00\x00\x02" ++
|
||||
"\x07example\x03com\x00\x00\x01\x00\x01";
|
||||
const trailing = "\x00\x00\x01\x00\x01\x00\x00\x00\x0a\x00\x04\x01\x02\x03\x04";
|
||||
const query = head ++
|
||||
"\x00\x00\x29\x10\x00\x00\x00\x80\x00\x00\x0b" ++
|
||||
"\x00\x08\x00\x07\x00\x01\x18\x00\xc0\x00\x02" ++
|
||||
trailing;
|
||||
|
||||
const parsed = try parseQuery(query);
|
||||
var out: [512]u8 = undefined;
|
||||
const rewritten = (try stripEcs(query, parsed.pkt, parsed.opt, &out)).rewritten;
|
||||
try testing.expectEqualSlices(
|
||||
u8,
|
||||
head ++ "\x00\x00\x29\x10\x00\x00\x00\x80\x00\x00\x00" ++ trailing,
|
||||
rewritten,
|
||||
);
|
||||
try testing.expectEqual(@as(u16, 2), (try packet_mod.parse(rewritten)).header.arcount);
|
||||
}
|
||||
|
||||
test "stripEcs removes a repeated ECS option" {
|
||||
const ecs = "\x00\x08\x00\x07\x00\x01\x18\x00\xc0\x00\x02";
|
||||
const query = "\x12\x34\x01\x00\x00\x01\x00\x00\x00\x00\x00\x01" ++
|
||||
"\x07example\x03com\x00\x00\x01\x00\x01" ++
|
||||
"\x00\x00\x29\x10\x00\x00\x00\x80\x00\x00\x1c" ++
|
||||
ecs ++ "\x00\x0c\x00\x02\x00\x00" ++ ecs;
|
||||
|
||||
const parsed = try parseQuery(query);
|
||||
var out: [512]u8 = undefined;
|
||||
const rewritten = (try stripEcs(query, parsed.pkt, parsed.opt, &out)).rewritten;
|
||||
try testing.expectEqualSlices(u8, ecs_query_stripped, rewritten);
|
||||
}
|
||||
|
||||
test "stripEcs reports an output buffer one byte short" {
|
||||
const parsed = try parseQuery(ecs_query);
|
||||
|
||||
var exact: [ecs_query_stripped.len]u8 = undefined;
|
||||
_ = (try stripEcs(ecs_query, parsed.pkt, parsed.opt, &exact)).rewritten;
|
||||
|
||||
var short: [ecs_query_stripped.len - 1]u8 = undefined;
|
||||
try testing.expectError(error.Overflow, stripEcs(ecs_query, parsed.pkt, parsed.opt, &short));
|
||||
|
||||
// Every shorter buffer fails the same way, including one that cannot even
|
||||
// hold the copied header.
|
||||
var i: usize = 0;
|
||||
while (i < short.len) : (i += 1) {
|
||||
try testing.expectError(
|
||||
error.Overflow,
|
||||
stripEcs(ecs_query, parsed.pkt, parsed.opt, short[0..i]),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
test "stripEcs rejects a malformed option list" {
|
||||
// The record is well-formed; only its option list overruns, so `parseOpt`
|
||||
// is the one thing that rejects this packet.
|
||||
const query = "\x12\x34\x01\x00\x00\x01\x00\x00\x00\x00\x00\x01" ++
|
||||
"\x07example\x03com\x00\x00\x01\x00\x01" ++
|
||||
"\x00\x00\x29\x10\x00\x00\x00\x00\x00\x00\x08" ++
|
||||
"\x00\x08\x00\x08\x00\x01\x18\x00";
|
||||
|
||||
const p = try packet_mod.parse(query);
|
||||
const rec = packet_mod.findOptRecord(p).?;
|
||||
try testing.expectError(error.BadOption, parseOpt(query, rec));
|
||||
|
||||
const hand_built: OptRecord = .{
|
||||
.udp_payload_size = rec.class,
|
||||
.extended_rcode = 0,
|
||||
.version = 0,
|
||||
.do_bit = false,
|
||||
.options = rec.rdata,
|
||||
};
|
||||
|
||||
var out: [512]u8 = undefined;
|
||||
try testing.expectError(error.BadOption, stripEcs(query, p, hand_built, &out));
|
||||
}
|
||||
|
||||
test "encodeOpt writes RDLENGTH where stripEcs patches it" {
|
||||
const opt: OptRecord = .{
|
||||
.udp_payload_size = 4096,
|
||||
.extended_rcode = 0,
|
||||
.version = 0,
|
||||
.do_bit = true,
|
||||
.options = .{ .offset = 0, .len = 0 },
|
||||
};
|
||||
var buf: [32]u8 = undefined;
|
||||
var w = Writer.fixed(&buf);
|
||||
try encodeOpt(opt, "\x00\x0c\x00\x02\x00\x00", &w);
|
||||
const bytes = w.buffered();
|
||||
try testing.expectEqual(
|
||||
@as(u16, 6),
|
||||
std.mem.readInt(u16, bytes[opt_rdlength_offset..][0..2], .big),
|
||||
);
|
||||
}
|
||||
|
||||
test "extendedRcode composes the twelve bits" {
|
||||
try testing.expectEqual(@as(u12, 3), extendedRcode(.nx_domain, null));
|
||||
|
||||
|
||||
@@ -44,6 +44,7 @@ const clients_repo = @import("../storage/repositories/clients_repo.zig");
|
||||
const groups_repo = @import("../storage/repositories/groups_repo.zig");
|
||||
const rules_repo = @import("../storage/repositories/rules_repo.zig");
|
||||
const sources_repo = @import("../storage/repositories/sources_repo.zig");
|
||||
const disk_monitor = @import("../storage/disk_monitor.zig");
|
||||
const compiler = @import("compiler.zig");
|
||||
const fetcher = @import("fetcher.zig");
|
||||
const matcher = @import("matcher.zig");
|
||||
@@ -221,6 +222,15 @@ pub const Manager = struct {
|
||||
/// Owns the `statuses` table. The entries themselves borrow nothing.
|
||||
status_arena: std.heap.ArenaAllocator,
|
||||
|
||||
/// The §11.6 disk gate (ruling 17). Set by the composition root after
|
||||
/// `init` and before `runScheduler` starts; null disables gating, which is
|
||||
/// what every test and `nxdns check` want. Only the scheduler consults it —
|
||||
/// see `refreshGated`.
|
||||
monitor: ?*disk_monitor.Monitor = null,
|
||||
/// Scheduled refresh passes skipped by the disk gate. Phase 8's health
|
||||
/// rollup reads it through `refreshesGated`.
|
||||
refreshes_gated: std.atomic.Value(u64) = .init(0),
|
||||
|
||||
pub const Error = error{
|
||||
OutOfMemory,
|
||||
Canceled,
|
||||
@@ -958,6 +968,7 @@ pub const Manager = struct {
|
||||
};
|
||||
while (true) {
|
||||
try interval.sleep(io);
|
||||
if (self.refreshGated()) continue;
|
||||
self.refreshAll(io) catch |err| switch (err) {
|
||||
error.Canceled => return error.Canceled,
|
||||
else => log.warn("blocklist refresh pass failed: {s}", .{@errorName(err)}),
|
||||
@@ -965,12 +976,42 @@ pub const Manager = struct {
|
||||
}
|
||||
}
|
||||
|
||||
/// The §11.6 gate, consulted by scheduled passes only (ruling 17). A
|
||||
/// download writes tens of megabytes into the blocklist directory and the
|
||||
/// compile writes as much again, which is exactly the "non-essential write"
|
||||
/// a critically full disk must not take.
|
||||
///
|
||||
/// `reload` and `refreshAll` are deliberately not gated: both are operator
|
||||
/// actions (the composition root's startup load, Phase 8's manual refresh),
|
||||
/// and an operator who asks for a refresh on a full disk has asked for it.
|
||||
///
|
||||
/// Counting happens here, so a caller cannot skip a pass without recording
|
||||
/// it. One `warn` line per skipped pass — at a 24-hour interval that is one
|
||||
/// line a day, and the disk monitor already logs the state change itself.
|
||||
fn refreshGated(self: *Manager) bool {
|
||||
const monitor = self.monitor orelse return false;
|
||||
if (monitor.writesAllowed()) return false;
|
||||
_ = self.refreshes_gated.fetchAdd(1, .monotonic);
|
||||
log.warn("free space is critical; skipping the scheduled blocklist refresh", .{});
|
||||
return true;
|
||||
}
|
||||
|
||||
/// Scheduled refresh passes the disk gate has skipped.
|
||||
pub fn refreshesGated(self: *const Manager) u64 {
|
||||
return self.refreshes_gated.load(.monotonic);
|
||||
}
|
||||
|
||||
fn startupPass(self: *Manager, io: std.Io) Error!void {
|
||||
self.writer_lock.lockUncancelable(io);
|
||||
defer self.writer_lock.unlock(io);
|
||||
|
||||
// Ahead of the gate on purpose: loading the compiled files that already
|
||||
// exist is a read. A full disk must not cost the household its
|
||||
// filtering as well as its downloads.
|
||||
try self.reloadLocked(io);
|
||||
|
||||
if (self.refreshGated()) return;
|
||||
|
||||
var rows = try sources_repo.listSourceRows(self.database, self.gpa);
|
||||
defer rows.deinit(self.gpa);
|
||||
defer sources_repo.freeSourceRows(self.gpa, rows.items);
|
||||
@@ -1439,6 +1480,42 @@ test "acquire before any reload returns null and holds no lock" {
|
||||
manager.lock.unlock(io);
|
||||
}
|
||||
|
||||
test "the disk gate skips a scheduled refresh only while writes are critical" {
|
||||
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var database = try openMigrated();
|
||||
defer database.close();
|
||||
|
||||
var f: fetcher.Fetcher = undefined;
|
||||
var manager = try testManager(&database, &f);
|
||||
defer manager.deinit(io);
|
||||
|
||||
// No monitor: every pass runs, which is what the tests and `check` rely on.
|
||||
try testing.expect(!manager.refreshGated());
|
||||
try testing.expectEqual(@as(u64, 0), manager.refreshesGated());
|
||||
|
||||
var monitor: disk_monitor.Monitor = .init(.{}, std.Io.Dir.cwd(), ".", null);
|
||||
manager.monitor = &monitor;
|
||||
|
||||
// `.ok` and `.warn` both allow writes: only `critical` stops them.
|
||||
try testing.expect(!manager.refreshGated());
|
||||
monitor.state_raw.store(@intFromEnum(disk_monitor.State.warn), .monotonic);
|
||||
try testing.expect(!manager.refreshGated());
|
||||
try testing.expectEqual(@as(u64, 0), manager.refreshesGated());
|
||||
|
||||
monitor.state_raw.store(@intFromEnum(disk_monitor.State.critical), .monotonic);
|
||||
try testing.expect(manager.refreshGated());
|
||||
try testing.expect(manager.refreshGated());
|
||||
try testing.expectEqual(@as(u64, 2), manager.refreshesGated());
|
||||
|
||||
// Free space recovers and the schedule resumes; the counter keeps its total.
|
||||
monitor.state_raw.store(@intFromEnum(disk_monitor.State.ok), .monotonic);
|
||||
try testing.expect(!manager.refreshGated());
|
||||
try testing.expectEqual(@as(u64, 2), manager.refreshesGated());
|
||||
}
|
||||
|
||||
test "statusSnapshot on an empty manager copies nothing" {
|
||||
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
|
||||
defer threaded.deinit();
|
||||
|
||||
@@ -0,0 +1,502 @@
|
||||
//! Client auto-materialisation (PLAN §7.2): every address that asks a question
|
||||
//! ends up as a row in `clients`, so the operator can name it and assign it a
|
||||
//! group without typing an address by hand.
|
||||
//!
|
||||
//! A DNS query must never wait on a database write, so `track` only records the
|
||||
//! address in a fixed-capacity table in memory. A background loop drains that
|
||||
//! table every `flush_interval_s` and writes one row per distinct client, and
|
||||
//! prunes the rows of devices that went quiet on every
|
||||
//! `prune_every_passes`-th pass.
|
||||
//!
|
||||
//! The table is bounded at `max_pending`. A full table drops the address and
|
||||
//! counts it under `dropped_full`: a burst of spoofed source addresses must not
|
||||
//! be able to grow this allocation, and a dropped address costs nothing, since
|
||||
//! the next query from that client tracks it again.
|
||||
//!
|
||||
//! A materialised row does not reach the matcher until the next manager reload.
|
||||
//! Nothing depends on it: `groupForClient` already falls back to the prefix
|
||||
//! rules and then to the default group, so the row exists for the operator's
|
||||
//! benefit, not for resolution.
|
||||
|
||||
const std = @import("std");
|
||||
|
||||
const address = @import("../platform/address.zig");
|
||||
const clients_repo = @import("../storage/repositories/clients_repo.zig");
|
||||
const db = @import("../storage/db.zig");
|
||||
const disk_monitor = @import("../storage/disk_monitor.zig");
|
||||
|
||||
const log = std.log.scoped(.clients);
|
||||
|
||||
/// RFC 5952 text of any IPv6 address. `format` never writes more than this, so
|
||||
/// the formatting in `flushOnce` cannot fail.
|
||||
const max_ip_text = 45;
|
||||
|
||||
/// One address waiting for its row, with the wall-clock second of its most
|
||||
/// recent query.
|
||||
const Pending = struct {
|
||||
addr: address.NetAddress,
|
||||
last_seen: i64,
|
||||
};
|
||||
|
||||
pub const Tracker = struct {
|
||||
/// Distinct clients one flush interval can carry. A household LAN holds two
|
||||
/// orders of magnitude fewer; the headroom is for the spoofing case.
|
||||
pub const max_pending = 512;
|
||||
pub const flush_interval_s = 60;
|
||||
/// One day at `flush_interval_s` seconds per pass.
|
||||
pub const prune_every_passes = 1440;
|
||||
|
||||
/// `tracked` counts the `track` calls that landed in the table, whether
|
||||
/// they created an entry or refreshed one, so `tracked + dropped_full` is
|
||||
/// the number of `track` calls.
|
||||
pub const Stats = struct {
|
||||
tracked: u64 = 0,
|
||||
flushed: u64 = 0,
|
||||
dropped_full: u64 = 0,
|
||||
pruned: u64 = 0,
|
||||
flush_failures: u64 = 0,
|
||||
};
|
||||
|
||||
/// Guards `pending`, `count`, `passes` and `stats`. Every field below is
|
||||
/// written under it, so a reader takes it too; see `snapshotStats`.
|
||||
mutex: std.Io.Mutex,
|
||||
retention_days: u16,
|
||||
pending: [max_pending]Pending,
|
||||
count: u32,
|
||||
passes: u64,
|
||||
stats: Stats,
|
||||
|
||||
/// `retention_days` is `logging.retention_days`, the same knob the query log
|
||||
/// prunes by (milestone-7 ruling 16). A client silent for that long is as
|
||||
/// uninteresting as a query that old.
|
||||
pub fn init(retention_days: u16) Tracker {
|
||||
return .{
|
||||
.mutex = .init,
|
||||
.retention_days = retention_days,
|
||||
.pending = undefined,
|
||||
.count = 0,
|
||||
.passes = 0,
|
||||
.stats = .{},
|
||||
};
|
||||
}
|
||||
|
||||
/// Records `addr` as seen now. Called from the query path, so it writes no
|
||||
/// database and returns no error: a full table drops the address.
|
||||
///
|
||||
/// `lockUncancelable` rather than `lock`: the caller is `Handler.handle`,
|
||||
/// which has no error union to carry `error.Canceled` out of. The critical
|
||||
/// section is a scan of at most `max_pending` addresses and holds no I/O.
|
||||
pub fn track(self: *Tracker, io: std.Io, addr: address.NetAddress) void {
|
||||
self.trackAt(io, addr, std.Io.Clock.real.now(io).toSeconds());
|
||||
}
|
||||
|
||||
/// `track` with the timestamp supplied, so a test does not depend on the
|
||||
/// wall clock.
|
||||
pub fn trackAt(self: *Tracker, io: std.Io, addr: address.NetAddress, now_s: i64) void {
|
||||
self.mutex.lockUncancelable(io);
|
||||
defer self.mutex.unlock(io);
|
||||
|
||||
for (self.pending[0..self.count]) |*entry| {
|
||||
if (!entry.addr.eql(addr)) continue;
|
||||
entry.last_seen = now_s;
|
||||
self.stats.tracked += 1;
|
||||
return;
|
||||
}
|
||||
if (self.count == max_pending) {
|
||||
self.stats.dropped_full += 1;
|
||||
return;
|
||||
}
|
||||
self.pending[self.count] = .{ .addr = addr, .last_seen = now_s };
|
||||
self.count += 1;
|
||||
self.stats.tracked += 1;
|
||||
}
|
||||
|
||||
/// Flush loop, first flush one interval in: an empty table at startup has
|
||||
/// nothing to write.
|
||||
///
|
||||
/// `boot` rather than `awake`, so a box that suspends still sees its day
|
||||
/// elapse and prunes on schedule.
|
||||
///
|
||||
/// `database` must be a connection dedicated to this loop: no other task may
|
||||
/// use the same handle while it runs. `FULLMUTEX` (`db.zig:218`) serializes
|
||||
/// one SQLite call against another, but a transaction is connection state,
|
||||
/// not call state, so a flush that lands between another writer's BEGIN and
|
||||
/// COMMIT would commit or roll back with that writer's batch. Milestone-7
|
||||
/// ruling 21 gives this loop its own `config.db` connection; the tracker
|
||||
/// opens nothing itself.
|
||||
///
|
||||
/// `monitor` gates the pass: while free space is critical the tracker writes
|
||||
/// nothing, and the addresses it would have written stay dropped.
|
||||
pub fn run(
|
||||
self: *Tracker,
|
||||
io: std.Io,
|
||||
database: *db.Db,
|
||||
monitor: ?*disk_monitor.Monitor,
|
||||
) std.Io.Cancelable!void {
|
||||
const interval: std.Io.Clock.Duration = .{
|
||||
.raw = .fromSeconds(flush_interval_s),
|
||||
.clock = .boot,
|
||||
};
|
||||
while (true) {
|
||||
try interval.sleep(io);
|
||||
const writes_allowed = if (monitor) |m| m.writesAllowed() else true;
|
||||
self.flushOnce(io, database, writes_allowed);
|
||||
}
|
||||
}
|
||||
|
||||
/// One pass: drain the table, write a row per client, and prune on every
|
||||
/// `prune_every_passes`-th pass.
|
||||
///
|
||||
/// A gated pass does nothing at all, not even count: the work it skipped is
|
||||
/// still owed, and the pending addresses it leaves behind are re-tracked by
|
||||
/// the next query from each client.
|
||||
///
|
||||
/// Every database failure logs one line at `warn` and counts. Nothing
|
||||
/// retries within a pass: the dropped addresses come back on their own, and
|
||||
/// a failed prune repeats the same work a day later against the same rows.
|
||||
///
|
||||
/// Only `run` may call this concurrently with itself — the drain buffer is
|
||||
/// this call's stack, but the pass counter and the prune schedule assume a
|
||||
/// single caller.
|
||||
pub fn flushOnce(self: *Tracker, io: std.Io, database: *db.Db, writes_allowed: bool) void {
|
||||
if (!writes_allowed) return;
|
||||
|
||||
var drained: [max_pending]Pending = undefined;
|
||||
const batch = self.drain(io, &drained);
|
||||
|
||||
var flushed: u64 = 0;
|
||||
var failures: u64 = 0;
|
||||
for (batch) |entry| {
|
||||
var buf: [max_ip_text]u8 = undefined;
|
||||
var w: std.Io.Writer = .fixed(&buf);
|
||||
entry.addr.format(&w) catch unreachable;
|
||||
|
||||
if (clients_repo.upsertSeen(database, w.buffered(), entry.last_seen)) {
|
||||
flushed += 1;
|
||||
} else |err| {
|
||||
if (failures == 0) {
|
||||
log.warn("materialising client {s} failed: {s}", .{ w.buffered(), @errorName(err) });
|
||||
}
|
||||
failures += 1;
|
||||
}
|
||||
}
|
||||
|
||||
self.mutex.lockUncancelable(io);
|
||||
self.passes += 1;
|
||||
self.stats.flushed += flushed;
|
||||
self.stats.flush_failures += failures;
|
||||
const due = self.passes % prune_every_passes == 0;
|
||||
self.mutex.unlock(io);
|
||||
|
||||
if (!due) return;
|
||||
const cutoff = std.Io.Clock.real.now(io).toSeconds() - @as(i64, self.retention_days) * 86_400;
|
||||
if (clients_repo.pruneStale(database, cutoff)) |deleted| {
|
||||
self.mutex.lockUncancelable(io);
|
||||
self.stats.pruned += deleted;
|
||||
self.mutex.unlock(io);
|
||||
} else |err| {
|
||||
log.warn("pruning clients before {d} failed: {s}", .{ cutoff, @errorName(err) });
|
||||
self.mutex.lockUncancelable(io);
|
||||
self.stats.flush_failures += 1;
|
||||
self.mutex.unlock(io);
|
||||
}
|
||||
}
|
||||
|
||||
pub fn snapshotStats(self: *Tracker, io: std.Io) Stats {
|
||||
self.mutex.lockUncancelable(io);
|
||||
defer self.mutex.unlock(io);
|
||||
return self.stats;
|
||||
}
|
||||
|
||||
/// Clients waiting for their row. Reaching `max_pending` is what turns
|
||||
/// further addresses into `dropped_full`.
|
||||
pub fn pendingClients(self: *Tracker, io: std.Io) u32 {
|
||||
self.mutex.lockUncancelable(io);
|
||||
defer self.mutex.unlock(io);
|
||||
return self.count;
|
||||
}
|
||||
|
||||
/// Empties the table into `out` and returns what it copied. The lock is
|
||||
/// released before any database call, so the query path never waits on
|
||||
/// SQLite.
|
||||
fn drain(self: *Tracker, io: std.Io, out: *[max_pending]Pending) []const Pending {
|
||||
self.mutex.lockUncancelable(io);
|
||||
defer self.mutex.unlock(io);
|
||||
|
||||
const n = self.count;
|
||||
@memcpy(out[0..n], self.pending[0..n]);
|
||||
self.count = 0;
|
||||
return out[0..n];
|
||||
}
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// tests
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const migrations = @import("../storage/migrations.zig");
|
||||
|
||||
const testing = std.testing;
|
||||
|
||||
fn openMigrated() !db.Db {
|
||||
var database = try db.Db.open(":memory:", .{ .mode = .memory });
|
||||
errdefer database.close();
|
||||
try db.applyPragmas(&database, .{});
|
||||
_ = try migrations.migrate(&database);
|
||||
return database;
|
||||
}
|
||||
|
||||
fn lastSeen(database: *db.Db, ip: []const u8) !i64 {
|
||||
var stmt = try database.prepare("SELECT last_seen FROM clients WHERE ip = ?1");
|
||||
defer stmt.deinit();
|
||||
try stmt.bindText(1, ip);
|
||||
try testing.expect(try stmt.step());
|
||||
return stmt.columnInt(0);
|
||||
}
|
||||
|
||||
fn parsed(text: []const u8) address.NetAddress {
|
||||
return address.NetAddress.parse(text) catch unreachable;
|
||||
}
|
||||
|
||||
test "a client tracked twice before a flush yields one row at the later time" {
|
||||
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var database = try openMigrated();
|
||||
defer database.close();
|
||||
|
||||
var tracker: Tracker = .init(30);
|
||||
tracker.trackAt(io, parsed("192.168.1.10"), 1700000000);
|
||||
tracker.trackAt(io, parsed("192.168.1.10"), 1700000030);
|
||||
try testing.expectEqual(@as(u32, 1), tracker.pendingClients(io));
|
||||
|
||||
tracker.flushOnce(io, &database, true);
|
||||
|
||||
try testing.expectEqual(@as(i64, 1), try clients_repo.countClients(&database));
|
||||
try testing.expectEqual(@as(i64, 1700000030), try lastSeen(&database, "192.168.1.10"));
|
||||
|
||||
const stats = tracker.snapshotStats(io);
|
||||
try testing.expectEqual(@as(u64, 2), stats.tracked);
|
||||
try testing.expectEqual(@as(u64, 1), stats.flushed);
|
||||
try testing.expectEqual(@as(u64, 0), stats.flush_failures);
|
||||
try testing.expectEqual(@as(u32, 0), tracker.pendingClients(io));
|
||||
}
|
||||
|
||||
test "distinct clients each get a row and ipv6 text is canonical" {
|
||||
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var database = try openMigrated();
|
||||
defer database.close();
|
||||
|
||||
var tracker: Tracker = .init(30);
|
||||
tracker.trackAt(io, parsed("192.168.1.10"), 1700000000);
|
||||
tracker.trackAt(io, parsed("192.168.1.11"), 1700000001);
|
||||
tracker.trackAt(io, parsed("fd00:0:0:0:0:0:0:1"), 1700000002);
|
||||
// An IPv4-mapped literal is the same client as its plain form.
|
||||
tracker.trackAt(io, address.NetAddress.fromIp(try std.Io.net.IpAddress.parse("::ffff:192.168.1.10", 53)), 1700000003);
|
||||
try testing.expectEqual(@as(u32, 3), tracker.pendingClients(io));
|
||||
|
||||
tracker.flushOnce(io, &database, true);
|
||||
|
||||
try testing.expectEqual(@as(i64, 3), try clients_repo.countClients(&database));
|
||||
try testing.expectEqual(@as(i64, 1700000003), try lastSeen(&database, "192.168.1.10"));
|
||||
try testing.expectEqual(@as(i64, 1700000002), try lastSeen(&database, "fd00::1"));
|
||||
try testing.expectEqual(@as(u64, 3), tracker.snapshotStats(io).flushed);
|
||||
}
|
||||
|
||||
test "a full table drops further clients and counts them" {
|
||||
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var database = try openMigrated();
|
||||
defer database.close();
|
||||
|
||||
var tracker: Tracker = .init(30);
|
||||
for (0..Tracker.max_pending) |i| {
|
||||
var octets: [4]u8 = undefined;
|
||||
std.mem.writeInt(u32, &octets, @intCast(i), .big);
|
||||
tracker.trackAt(io, .{ .ip4 = octets }, 1700000000);
|
||||
}
|
||||
try testing.expectEqual(@as(u32, Tracker.max_pending), tracker.pendingClients(io));
|
||||
try testing.expectEqual(@as(u64, 0), tracker.snapshotStats(io).dropped_full);
|
||||
|
||||
tracker.trackAt(io, parsed("203.0.113.7"), 1700000000);
|
||||
tracker.trackAt(io, parsed("203.0.113.8"), 1700000000);
|
||||
try testing.expectEqual(@as(u32, Tracker.max_pending), tracker.pendingClients(io));
|
||||
|
||||
const stats = tracker.snapshotStats(io);
|
||||
try testing.expectEqual(@as(u64, 2), stats.dropped_full);
|
||||
try testing.expectEqual(@as(u64, Tracker.max_pending), stats.tracked);
|
||||
|
||||
// A tracked client still refreshes while the table is full, and the flush
|
||||
// makes room for the next newcomer.
|
||||
tracker.trackAt(io, .{ .ip4 = .{ 0, 0, 0, 0 } }, 1700000060);
|
||||
tracker.flushOnce(io, &database, true);
|
||||
try testing.expectEqual(@as(i64, Tracker.max_pending), try clients_repo.countClients(&database));
|
||||
try testing.expectEqual(@as(i64, 1700000060), try lastSeen(&database, "0.0.0.0"));
|
||||
|
||||
tracker.trackAt(io, parsed("203.0.113.7"), 1700000060);
|
||||
try testing.expectEqual(@as(u32, 1), tracker.pendingClients(io));
|
||||
}
|
||||
|
||||
test "a flush touches a hand-edited row without changing what the operator set" {
|
||||
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var database = try openMigrated();
|
||||
defer database.close();
|
||||
try database.exec("INSERT INTO groups (id, name) VALUES (2, 'kids');");
|
||||
try database.exec(
|
||||
\\INSERT INTO clients (ip, name, group_id, hand_edited, first_seen, last_seen)
|
||||
\\VALUES ('192.168.1.10', 'laptop', 2, 1, 1690000000, 1690000000);
|
||||
);
|
||||
|
||||
var tracker: Tracker = .init(30);
|
||||
tracker.trackAt(io, parsed("192.168.1.10"), 1700000000);
|
||||
tracker.flushOnce(io, &database, true);
|
||||
|
||||
try testing.expectEqual(@as(i64, 1), try clients_repo.countClients(&database));
|
||||
try testing.expectEqual(@as(i64, 1700000000), try lastSeen(&database, "192.168.1.10"));
|
||||
try testing.expectEqual(
|
||||
@as(i64, 1),
|
||||
try database.queryInt(
|
||||
\\SELECT count(*) FROM clients
|
||||
\\ WHERE ip = '192.168.1.10' AND name = 'laptop' AND group_id = 2
|
||||
\\ AND hand_edited = 1 AND first_seen = 1690000000
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
test "a gated pass writes nothing and keeps the pending clients" {
|
||||
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var database = try openMigrated();
|
||||
defer database.close();
|
||||
|
||||
var monitor: disk_monitor.Monitor = .init(.{}, std.Io.Dir.cwd(), ".", null);
|
||||
monitor.state_raw.store(@intFromEnum(disk_monitor.State.critical), .monotonic);
|
||||
try testing.expect(!monitor.writesAllowed());
|
||||
|
||||
var tracker: Tracker = .init(30);
|
||||
tracker.trackAt(io, parsed("192.168.1.10"), 1700000000);
|
||||
tracker.flushOnce(io, &database, monitor.writesAllowed());
|
||||
|
||||
try testing.expectEqual(@as(i64, 0), try clients_repo.countClients(&database));
|
||||
try testing.expectEqual(@as(u32, 1), tracker.pendingClients(io));
|
||||
try testing.expectEqual(@as(u64, 0), tracker.snapshotStats(io).flushed);
|
||||
try testing.expectEqual(@as(u64, 0), tracker.passes);
|
||||
|
||||
// Free space recovers and the same pending client lands.
|
||||
monitor.state_raw.store(@intFromEnum(disk_monitor.State.warn), .monotonic);
|
||||
tracker.flushOnce(io, &database, monitor.writesAllowed());
|
||||
try testing.expectEqual(@as(i64, 1), try clients_repo.countClients(&database));
|
||||
try testing.expectEqual(@as(u64, 1), tracker.passes);
|
||||
}
|
||||
|
||||
test "a failing upsert counts and leaves the client to be tracked again" {
|
||||
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var database = try openMigrated();
|
||||
defer database.close();
|
||||
try database.exec(
|
||||
\\CREATE TRIGGER refuse_insert BEFORE INSERT ON clients
|
||||
\\BEGIN SELECT RAISE(ABORT, 'refused'); END;
|
||||
);
|
||||
|
||||
var tracker: Tracker = .init(30);
|
||||
tracker.trackAt(io, parsed("192.168.1.10"), 1700000000);
|
||||
tracker.trackAt(io, parsed("192.168.1.11"), 1700000000);
|
||||
tracker.flushOnce(io, &database, true);
|
||||
|
||||
try testing.expectEqual(@as(i64, 0), try clients_repo.countClients(&database));
|
||||
const stats = tracker.snapshotStats(io);
|
||||
try testing.expectEqual(@as(u64, 0), stats.flushed);
|
||||
try testing.expectEqual(@as(u64, 2), stats.flush_failures);
|
||||
try testing.expectEqual(@as(u32, 0), tracker.pendingClients(io));
|
||||
|
||||
try database.exec("DROP TRIGGER refuse_insert;");
|
||||
tracker.trackAt(io, parsed("192.168.1.10"), 1700000060);
|
||||
tracker.flushOnce(io, &database, true);
|
||||
try testing.expectEqual(@as(i64, 1), try clients_repo.countClients(&database));
|
||||
}
|
||||
|
||||
test "the pass that comes due prunes the clients that went quiet" {
|
||||
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var database = try openMigrated();
|
||||
defer database.close();
|
||||
|
||||
const now = std.Io.Clock.real.now(io).toSeconds();
|
||||
const day = 86_400;
|
||||
try clients_repo.upsertSeen(&database, "10.0.0.1", now - 40 * day);
|
||||
try clients_repo.upsertSeen(&database, "10.0.0.2", now - 29 * day);
|
||||
|
||||
var tracker: Tracker = .init(30);
|
||||
// Every pass before the due one leaves both rows alone.
|
||||
for (0..Tracker.prune_every_passes - 1) |_| {
|
||||
tracker.flushOnce(io, &database, true);
|
||||
}
|
||||
try testing.expectEqual(@as(i64, 2), try clients_repo.countClients(&database));
|
||||
try testing.expectEqual(@as(u64, 0), tracker.snapshotStats(io).pruned);
|
||||
|
||||
tracker.flushOnce(io, &database, true);
|
||||
|
||||
try testing.expectEqual(@as(i64, 1), try clients_repo.countClients(&database));
|
||||
try testing.expectEqual(@as(i64, now - 29 * day), try lastSeen(&database, "10.0.0.2"));
|
||||
try testing.expectEqual(@as(u64, 1), tracker.snapshotStats(io).pruned);
|
||||
try testing.expectEqual(@as(u64, Tracker.prune_every_passes), tracker.passes);
|
||||
}
|
||||
|
||||
test "a shorter retention prunes what the default keeps" {
|
||||
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var database = try openMigrated();
|
||||
defer database.close();
|
||||
|
||||
const now = std.Io.Clock.real.now(io).toSeconds();
|
||||
try clients_repo.upsertSeen(&database, "10.0.0.1", now - 3 * 86_400);
|
||||
|
||||
var tracker: Tracker = .init(1);
|
||||
tracker.passes = Tracker.prune_every_passes - 1;
|
||||
tracker.flushOnce(io, &database, true);
|
||||
|
||||
try testing.expectEqual(@as(i64, 0), try clients_repo.countClients(&database));
|
||||
try testing.expectEqual(@as(u64, 1), tracker.snapshotStats(io).pruned);
|
||||
}
|
||||
|
||||
test "the run loop flushes on its interval and returns on cancel" {
|
||||
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var database = try openMigrated();
|
||||
defer database.close();
|
||||
|
||||
var tracker: Tracker = .init(30);
|
||||
tracker.trackAt(io, parsed("192.168.1.10"), 1700000000);
|
||||
|
||||
var future = try io.concurrent(Tracker.run, .{
|
||||
&tracker,
|
||||
io,
|
||||
&database,
|
||||
@as(?*disk_monitor.Monitor, null),
|
||||
});
|
||||
// The first flush is one interval away, so cancelling immediately proves the
|
||||
// loop starts by sleeping rather than by writing.
|
||||
try testing.expectError(error.Canceled, future.cancel(io));
|
||||
try testing.expectEqual(@as(i64, 0), try clients_repo.countClients(&database));
|
||||
try testing.expectEqual(@as(u32, 1), tracker.pendingClients(io));
|
||||
}
|
||||
+1951
-83
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,93 @@
|
||||
//! Global pause of filtering (PLAN §13.1). One 64-bit flag the query path
|
||||
//! reads and the API writes. Pure: no allocation, no `std.Io`, no clock — the
|
||||
//! caller supplies the time, because the handler already has it.
|
||||
//!
|
||||
//! Pausing suspends FILTERING only: local records, forward zones, the cache,
|
||||
//! the upstream and the query log all keep running (milestone-7 ruling 18).
|
||||
//!
|
||||
//! The state is in memory and is deliberately not persisted: a restart resumes
|
||||
//! filtering, which is the safe default for a household.
|
||||
|
||||
const std = @import("std");
|
||||
|
||||
pub const Pause = struct {
|
||||
/// 0 = filtering active. -1 = paused until someone resumes. Any other value
|
||||
/// is the unix second filtering resumes at, so the expiry is a comparison
|
||||
/// on the query path rather than a timer task.
|
||||
until: std.atomic.Value(i64) = .init(0),
|
||||
|
||||
/// `.monotonic` throughout: the flag guards no other data, so nothing has
|
||||
/// to be ordered against it.
|
||||
pub fn isPaused(self: *const Pause, now_s: i64) bool {
|
||||
const until = self.until.load(.monotonic);
|
||||
if (until == 0) return false;
|
||||
if (until < 0) return true;
|
||||
return now_s < until;
|
||||
}
|
||||
|
||||
/// `null` pauses until `unpause`. A pause already in force is replaced, so
|
||||
/// the last call wins whether it lengthens or shortens the pause.
|
||||
pub fn pauseFor(self: *Pause, now_s: i64, duration_s: ?u32) void {
|
||||
const value: i64 = if (duration_s) |seconds| now_s +| @as(i64, seconds) else -1;
|
||||
self.until.store(value, .monotonic);
|
||||
}
|
||||
|
||||
pub fn unpause(self: *Pause) void {
|
||||
self.until.store(0, .monotonic);
|
||||
}
|
||||
};
|
||||
|
||||
const testing = std.testing;
|
||||
|
||||
test "a fresh pause is not paused" {
|
||||
const p: Pause = .{};
|
||||
try testing.expect(!p.isPaused(0));
|
||||
try testing.expect(!p.isPaused(1_700_000_000));
|
||||
}
|
||||
|
||||
test "an indefinite pause holds at every time" {
|
||||
var p: Pause = .{};
|
||||
p.pauseFor(1_700_000_000, null);
|
||||
try testing.expect(p.isPaused(1_700_000_000));
|
||||
try testing.expect(p.isPaused(1_700_000_000 + 86_400 * 365));
|
||||
try testing.expectEqual(@as(i64, -1), p.until.load(.monotonic));
|
||||
}
|
||||
|
||||
test "a timed pause expires at its own second" {
|
||||
var p: Pause = .{};
|
||||
p.pauseFor(1_000, 60);
|
||||
try testing.expect(p.isPaused(1_000));
|
||||
try testing.expect(p.isPaused(1_059));
|
||||
// The stored second is when filtering is back on, so it is not paused.
|
||||
try testing.expect(!p.isPaused(1_060));
|
||||
try testing.expect(!p.isPaused(1_061));
|
||||
}
|
||||
|
||||
test "unpause resumes both kinds of pause" {
|
||||
var p: Pause = .{};
|
||||
p.pauseFor(1_000, null);
|
||||
p.unpause();
|
||||
try testing.expect(!p.isPaused(1_000));
|
||||
|
||||
p.pauseFor(1_000, 60);
|
||||
p.unpause();
|
||||
try testing.expect(!p.isPaused(1_000));
|
||||
}
|
||||
|
||||
test "pauseFor overwrites a pause already in force" {
|
||||
var p: Pause = .{};
|
||||
p.pauseFor(1_000, 3_600);
|
||||
p.pauseFor(1_000, 10);
|
||||
try testing.expect(!p.isPaused(1_010));
|
||||
|
||||
// And in the other direction: indefinite replaces a timed pause.
|
||||
p.pauseFor(1_000, 10);
|
||||
p.pauseFor(1_000, null);
|
||||
try testing.expect(p.isPaused(1_010));
|
||||
}
|
||||
|
||||
test "a duration that would overflow saturates instead of wrapping" {
|
||||
var p: Pause = .{};
|
||||
p.pauseFor(std.math.maxInt(i64), std.math.maxInt(u32));
|
||||
try testing.expect(p.isPaused(std.math.maxInt(i64) - 1));
|
||||
}
|
||||
@@ -0,0 +1,996 @@
|
||||
//! Milestone-7 integration tests (spec S7): the serving pipeline end to end,
|
||||
//! over real sockets.
|
||||
//!
|
||||
//! This lives in its own file because it needs `@import("build_options")`, which
|
||||
//! only exists when the compilation is driven by `build.zig`. The body compiles
|
||||
//! on every `zig build test` run, so it cannot rot, and every case skips at run
|
||||
//! time unless `-Dintegration` is passed.
|
||||
//!
|
||||
//! What separates these cases from `handler.zig`'s own tests is the socket. The
|
||||
//! handler tests call `handle` directly; here every query travels through a real
|
||||
//! `UdpServer` on 127.0.0.1, through the real handler with its real cache,
|
||||
//! limiter, tracker and query log, and the reply is read back off the wire. The
|
||||
//! upstream is a `transport.Client` fixture, except in the forward-zone case,
|
||||
//! where the zone resolver has to be a real UDP socket because `ForwardClient`
|
||||
//! speaks wire DNS to an address.
|
||||
//!
|
||||
//! Hermetic: every socket is bound to 127.0.0.1, every database is in memory or
|
||||
//! inside a `std.testing.tmpDir`, and every wait carries a budget.
|
||||
|
||||
const std = @import("std");
|
||||
const build_options = @import("build_options");
|
||||
const net = std.Io.net;
|
||||
const Allocator = std.mem.Allocator;
|
||||
|
||||
const app = @import("../app.zig");
|
||||
const cli = @import("../cli.zig");
|
||||
const clients = @import("clients.zig");
|
||||
const clients_repo = @import("../storage/repositories/clients_repo.zig");
|
||||
const db = @import("../storage/db.zig");
|
||||
const dns_cache = @import("../cache/dns_cache.zig");
|
||||
const forward_zones = @import("../local/forward_zones.zig");
|
||||
const handler = @import("handler.zig");
|
||||
const header = @import("../dns/header.zig");
|
||||
const logger_mod = @import("../storage/logger.zig");
|
||||
const manager = @import("../filter/manager.zig");
|
||||
const matcher = @import("../filter/matcher.zig");
|
||||
const migrations = @import("../storage/migrations.zig");
|
||||
const model = @import("../config/model.zig");
|
||||
const name = @import("../dns/name.zig");
|
||||
const packet = @import("../dns/packet.zig");
|
||||
const pause = @import("pause.zig");
|
||||
const question = @import("../dns/question.zig");
|
||||
const rate_limiter = @import("rate_limiter.zig");
|
||||
const record = @import("../dns/record.zig");
|
||||
const records = @import("../local/records.zig");
|
||||
const response = @import("../filter/response.zig");
|
||||
const shutdown = @import("shutdown.zig");
|
||||
const transport = @import("../upstream/transport.zig");
|
||||
const types = @import("../dns/types.zig");
|
||||
const udp_server = @import("udp_server.zig");
|
||||
|
||||
const testing = std.testing;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// shared fixtures
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// Long enough that a loopback round trip cannot lose to scheduling, short
|
||||
/// enough that a broken server fails the run instead of hanging it.
|
||||
const budget: std.Io.Timeout = .{ .duration = .{ .raw = .fromSeconds(5), .clock = .awake } };
|
||||
|
||||
const empty_records: records.Records = .empty;
|
||||
const empty_zones: forward_zones.Zones = .empty;
|
||||
|
||||
/// A five-second TTL makes the blocking answer's TTL unmistakable next to the
|
||||
/// upstream's 300.
|
||||
const blocking: response.Options = .{ .mode = .zero, .ttl = 5 };
|
||||
|
||||
const forward_timeout: std.Io.Clock.Duration = .{ .raw = .fromSeconds(2), .clock = .awake };
|
||||
|
||||
/// What every fake upstream answers with, and the TTL it carries.
|
||||
const upstream_rdata = [4]u8{ 93, 184, 216, 34 };
|
||||
const upstream_ttl: u32 = 300;
|
||||
|
||||
/// The zone resolver's answer, distinct from the pool's so a case can tell
|
||||
/// which of the two replied.
|
||||
const zone_rdata = [4]u8{ 10, 0, 0, 7 };
|
||||
const zone_ttl: u32 = 120;
|
||||
|
||||
/// The handler every case starts from: an upstream, the blocking options and
|
||||
/// the empty local tables. Each case wires in the collaborators it exercises.
|
||||
fn baseHandler(client: transport.Client) handler.Handler {
|
||||
return .{
|
||||
.upstream = client,
|
||||
.blocking = blocking,
|
||||
.forward_read_timeout = forward_timeout,
|
||||
.records = &empty_records,
|
||||
.zones = &empty_zones,
|
||||
};
|
||||
}
|
||||
|
||||
/// A real listener, a real client socket and the task that serves them.
|
||||
///
|
||||
/// Two phases: `bind` produces the value, `start` spawns the serve task against
|
||||
/// its final address. Nothing may copy a `Loop` after `start`, because the task
|
||||
/// holds a pointer into it.
|
||||
const Loop = struct {
|
||||
server: udp_server.UdpServer,
|
||||
group: std.Io.Group,
|
||||
client: net.Socket,
|
||||
server_address: net.IpAddress,
|
||||
|
||||
fn bind(gpa: Allocator, io: std.Io, h: *handler.Handler) !Loop {
|
||||
const listen_address: net.IpAddress = try .parse("127.0.0.1", 0);
|
||||
var server = try udp_server.UdpServer.bind(gpa, io, listen_address, h, .{ .max_in_flight = 4 });
|
||||
errdefer server.deinit(gpa, io);
|
||||
|
||||
const client_address: net.IpAddress = try .parse("127.0.0.1", 0);
|
||||
const client = try client_address.bind(io, .{ .mode = .dgram });
|
||||
|
||||
return .{
|
||||
.server = server,
|
||||
.group = .init,
|
||||
.client = client,
|
||||
.server_address = server.boundAddress(),
|
||||
};
|
||||
}
|
||||
|
||||
fn start(self: *Loop, io: std.Io) !void {
|
||||
try self.group.concurrent(io, udp_server.UdpServer.serve, .{ &self.server, io });
|
||||
}
|
||||
|
||||
/// One query, one reply. The reply is a prefix of `buf`.
|
||||
fn ask(self: *Loop, io: std.Io, query: []const u8, buf: []u8) ![]u8 {
|
||||
try self.client.send(io, &self.server_address, query);
|
||||
const msg = try self.client.receiveTimeout(io, buf, budget);
|
||||
return msg.data;
|
||||
}
|
||||
|
||||
fn stop(self: *Loop, gpa: Allocator, io: std.Io) void {
|
||||
self.server.deinit(gpa, io);
|
||||
self.group.cancel(io);
|
||||
self.client.close(io);
|
||||
}
|
||||
};
|
||||
|
||||
/// A query for `domain`, RD set, one question, no OPT.
|
||||
fn queryFor(buf: []u8, id: u16, domain: []const u8, qtype: types.Type) []const u8 {
|
||||
var w: std.Io.Writer = .fixed(buf);
|
||||
var encoded: [types.header_len]u8 = undefined;
|
||||
header.encode(.{
|
||||
.id = id,
|
||||
.flags = .{
|
||||
.rcode = .no_error,
|
||||
.z = 0,
|
||||
.ra = false,
|
||||
.rd = true,
|
||||
.tc = false,
|
||||
.aa = false,
|
||||
.opcode = .query,
|
||||
.qr = false,
|
||||
},
|
||||
.qdcount = 1,
|
||||
.ancount = 0,
|
||||
.nscount = 0,
|
||||
.arcount = 0,
|
||||
}, &encoded);
|
||||
w.writeAll(&encoded) catch unreachable;
|
||||
question.encode(.{
|
||||
.name = name.fromText(domain) catch unreachable,
|
||||
.qtype = qtype,
|
||||
.qclass = .in,
|
||||
}, &w) catch unreachable;
|
||||
return w.buffered();
|
||||
}
|
||||
|
||||
/// The pool stand-in. It answers the question it is given rather than a fixed
|
||||
/// byte string, because the safe-search and uncloaking cases both change the
|
||||
/// question on the way out.
|
||||
///
|
||||
/// `calls` is atomic: the listener task runs on another thread than the one
|
||||
/// asserting.
|
||||
const FakeUpstream = struct {
|
||||
reply: Reply,
|
||||
calls: std.atomic.Value(u64) = .init(0),
|
||||
|
||||
const Reply = union(enum) {
|
||||
/// One A record for the queried name.
|
||||
a,
|
||||
/// One CNAME record for the queried name, pointing at this target.
|
||||
cname: []const u8,
|
||||
};
|
||||
|
||||
fn exchangeFn(
|
||||
ptr: *anyopaque,
|
||||
io: std.Io,
|
||||
query: []const u8,
|
||||
response_buf: []u8,
|
||||
) transport.ExchangeError![]u8 {
|
||||
_ = io;
|
||||
const self: *FakeUpstream = @ptrCast(@alignCast(ptr));
|
||||
_ = self.calls.fetchAdd(1, .monotonic);
|
||||
|
||||
const request = packet.parse(query) catch return error.BadResponse;
|
||||
const q = packet.firstQuestion(request) orelse return error.BadResponse;
|
||||
|
||||
var b = packet.ResponseBuilder.init(response_buf, request.header, q) catch
|
||||
return error.ResponseTooLarge;
|
||||
switch (self.reply) {
|
||||
.a => b.addAnswer(q.name, .a, .in, upstream_ttl, &upstream_rdata) catch
|
||||
return error.ResponseTooLarge,
|
||||
.cname => |target| {
|
||||
const t = name.fromText(target) catch return error.BadResponse;
|
||||
b.addAnswer(q.name, .cname, .in, upstream_ttl, t.wire()) catch
|
||||
return error.ResponseTooLarge;
|
||||
},
|
||||
}
|
||||
return b.finish();
|
||||
}
|
||||
|
||||
fn client(self: *FakeUpstream) transport.Client {
|
||||
return .{ .ptr = self, .exchangeFn = exchangeFn };
|
||||
}
|
||||
};
|
||||
|
||||
/// The forward zone's resolver: a real UDP socket, because `ForwardClient`
|
||||
/// speaks wire DNS to an address and nothing smaller would prove it did.
|
||||
///
|
||||
/// The loop ends when the receive is canceled, which is what `group.cancel`
|
||||
/// does at the end of the case.
|
||||
fn zoneResolver(io: std.Io, socket: *const net.Socket, calls: *std.atomic.Value(u64)) void {
|
||||
var buf: [udp_server.max_datagram]u8 = undefined;
|
||||
while (true) {
|
||||
const msg = socket.receive(io, &buf) catch return;
|
||||
_ = calls.fetchAdd(1, .monotonic);
|
||||
|
||||
const request = packet.parse(msg.data) catch continue;
|
||||
const q = packet.firstQuestion(request) orelse continue;
|
||||
|
||||
var reply_buf: [512]u8 = undefined;
|
||||
var b = packet.ResponseBuilder.init(&reply_buf, request.header, q) catch continue;
|
||||
b.addAnswer(q.name, .a, .in, zone_ttl, &zone_rdata) catch continue;
|
||||
socket.send(io, &msg.from, b.finish()) catch return;
|
||||
}
|
||||
}
|
||||
|
||||
const SnapshotFixture = struct {
|
||||
groups: []const model.Group = &.{.{ .name = "default" }},
|
||||
rules: []const model.Rule = &.{},
|
||||
};
|
||||
|
||||
fn buildSnapshot(gpa: Allocator, fixture: SnapshotFixture) !matcher.Snapshot {
|
||||
return matcher.Snapshot.build(gpa, .{
|
||||
.groups = fixture.groups,
|
||||
.group_ids = &.{1},
|
||||
.group_sources = &.{},
|
||||
.sources = &.{},
|
||||
.source_ids = &.{},
|
||||
.rules = fixture.rules,
|
||||
.clients = &.{},
|
||||
.prefixes = &.{},
|
||||
.compiled = &.{},
|
||||
.seed = 0x5eed,
|
||||
.generation = 1,
|
||||
});
|
||||
}
|
||||
|
||||
/// `Manager.acquire` reads the manager's lock and its current snapshot and
|
||||
/// nothing else, so a manager that publishes one hand-built snapshot needs
|
||||
/// none of the database, fetcher or blocklist directory the real one owns.
|
||||
fn fixtureManager(m: *manager.Manager, snapshot: *matcher.Snapshot) void {
|
||||
m.* = .{
|
||||
.gpa = testing.allocator,
|
||||
.database = undefined,
|
||||
.paths = undefined,
|
||||
.fetcher = undefined,
|
||||
.update = .{},
|
||||
.total_budget = forward_timeout,
|
||||
.lock = .init,
|
||||
.writer_lock = .init,
|
||||
.current = snapshot,
|
||||
.generation = 1,
|
||||
.statuses = &.{},
|
||||
.status_arena = .init(testing.allocator),
|
||||
};
|
||||
}
|
||||
|
||||
fn blockRule(pattern: []const u8) model.Rule {
|
||||
return .{ .group = "default", .pattern = pattern, .kind = .exact, .action = .block };
|
||||
}
|
||||
|
||||
fn allowRule(pattern: []const u8) model.Rule {
|
||||
return .{ .group = "default", .pattern = pattern, .kind = .exact, .action = .allow };
|
||||
}
|
||||
|
||||
fn firstAnswer(p: packet.Packet) !record.Record {
|
||||
var it = packet.answers(p);
|
||||
return (try it.next()) orelse error.TestExpectedAnswer;
|
||||
}
|
||||
|
||||
fn drainLog(lg: *logger_mod.Logger, io: std.Io, out: []logger_mod.Entry) []logger_mod.Entry {
|
||||
const n = lg.queue.getUncancelable(io, out, 0) catch 0;
|
||||
return out[0..n];
|
||||
}
|
||||
|
||||
/// Every case that asserts on the query log wants the same shape: a queue big
|
||||
/// enough to hold the whole case, drained once at the end.
|
||||
const log_queue_len = 8;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// case 1: blocked domain
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test "S7 case 1: a blocked domain is answered with the zero address and logged" {
|
||||
if (!build_options.integration) return error.SkipZigTest;
|
||||
|
||||
const gpa = testing.allocator;
|
||||
var threaded: std.Io.Threaded = .init(gpa, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var snapshot = try buildSnapshot(gpa, .{ .rules = &.{blockRule("ads.example.com")} });
|
||||
defer snapshot.deinit();
|
||||
var mgr: manager.Manager = undefined;
|
||||
fixtureManager(&mgr, &snapshot);
|
||||
|
||||
var queue_buf: [log_queue_len]logger_mod.Entry = undefined;
|
||||
var lg: logger_mod.Logger = .init(.{}, &queue_buf);
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = .a };
|
||||
var h = baseHandler(fake.client());
|
||||
h.manager = &mgr;
|
||||
h.logger = ≶
|
||||
|
||||
var loop = try Loop.bind(gpa, io, &h);
|
||||
defer loop.stop(gpa, io);
|
||||
try loop.start(io);
|
||||
|
||||
var query_buf: [512]u8 = undefined;
|
||||
var reply_buf: [udp_server.max_datagram]u8 = undefined;
|
||||
const reply = try loop.ask(io, queryFor(&query_buf, 0x1111, "ads.example.com", .a), &reply_buf);
|
||||
|
||||
const p = try packet.parse(reply);
|
||||
try testing.expectEqual(@as(u16, 0x1111), p.header.id);
|
||||
try testing.expectEqual(types.Rcode.no_error, p.header.flags.rcode);
|
||||
try testing.expectEqual(@as(u16, 1), p.header.ancount);
|
||||
|
||||
const answer = try firstAnswer(p);
|
||||
try testing.expectEqual(@as(u32, blocking.ttl), answer.ttl);
|
||||
try testing.expectEqual([4]u8{ 0, 0, 0, 0 }, try record.rdataA(p.bytes, answer));
|
||||
|
||||
try testing.expectEqual(@as(u64, 0), fake.calls.load(.monotonic));
|
||||
try testing.expectEqual(@as(u64, 1), h.stats.blocked.load(.monotonic));
|
||||
|
||||
var entries: [log_queue_len]logger_mod.Entry = undefined;
|
||||
const logged = drainLog(&lg, io, &entries);
|
||||
try testing.expectEqual(@as(usize, 1), logged.len);
|
||||
try testing.expectEqual(true, logged[0].blocked);
|
||||
try testing.expectEqualStrings("ads.example.com", logged[0].domain());
|
||||
try testing.expectEqualStrings("rule_block_exact", logged[0].blockReason());
|
||||
try testing.expectEqualStrings("127.0.0.1", logged[0].clientIp());
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// case 2: allow over block
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test "S7 case 2: an allow rule beats the blocklist and the upstream answers" {
|
||||
if (!build_options.integration) return error.SkipZigTest;
|
||||
|
||||
const gpa = testing.allocator;
|
||||
var threaded: std.Io.Threaded = .init(gpa, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var snapshot = try buildSnapshot(gpa, .{
|
||||
.rules = &.{ blockRule("com"), allowRule("example.com") },
|
||||
});
|
||||
defer snapshot.deinit();
|
||||
var mgr: manager.Manager = undefined;
|
||||
fixtureManager(&mgr, &snapshot);
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = .a };
|
||||
var h = baseHandler(fake.client());
|
||||
h.manager = &mgr;
|
||||
|
||||
var loop = try Loop.bind(gpa, io, &h);
|
||||
defer loop.stop(gpa, io);
|
||||
try loop.start(io);
|
||||
|
||||
var query_buf: [512]u8 = undefined;
|
||||
var reply_buf: [udp_server.max_datagram]u8 = undefined;
|
||||
const reply = try loop.ask(io, queryFor(&query_buf, 0x2222, "example.com", .a), &reply_buf);
|
||||
|
||||
const p = try packet.parse(reply);
|
||||
const answer = try firstAnswer(p);
|
||||
try testing.expectEqual(upstream_rdata, try record.rdataA(p.bytes, answer));
|
||||
try testing.expectEqual(upstream_ttl, answer.ttl);
|
||||
try testing.expectEqual(@as(u64, 1), fake.calls.load(.monotonic));
|
||||
try testing.expectEqual(@as(u64, 0), h.stats.blocked.load(.monotonic));
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// case 3: local records
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test "S7 case 3: a local record answers authoritatively without an upstream" {
|
||||
if (!build_options.integration) return error.SkipZigTest;
|
||||
|
||||
const gpa = testing.allocator;
|
||||
var threaded: std.Io.Threaded = .init(gpa, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var table = try records.Records.build(gpa, &.{
|
||||
.{ .name = "nas.lan", .rtype = .a, .value = "192.168.1.10", .ttl = 60 },
|
||||
});
|
||||
defer table.deinit(gpa);
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = .a };
|
||||
var h = baseHandler(fake.client());
|
||||
h.records = &table;
|
||||
|
||||
var loop = try Loop.bind(gpa, io, &h);
|
||||
defer loop.stop(gpa, io);
|
||||
try loop.start(io);
|
||||
|
||||
var query_buf: [512]u8 = undefined;
|
||||
var reply_buf: [udp_server.max_datagram]u8 = undefined;
|
||||
const reply = try loop.ask(io, queryFor(&query_buf, 0x3333, "nas.lan", .a), &reply_buf);
|
||||
|
||||
const p = try packet.parse(reply);
|
||||
try testing.expectEqual(true, p.header.flags.aa);
|
||||
try testing.expectEqual(@as(u16, 1), p.header.ancount);
|
||||
|
||||
const answer = try firstAnswer(p);
|
||||
try testing.expectEqual(@as(u32, 60), answer.ttl);
|
||||
try testing.expectEqual([4]u8{ 192, 168, 1, 10 }, try record.rdataA(p.bytes, answer));
|
||||
|
||||
try testing.expectEqual(@as(u64, 0), fake.calls.load(.monotonic));
|
||||
try testing.expectEqual(@as(u64, 1), h.stats.local_answers.load(.monotonic));
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// case 4: forward zones
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test "S7 case 4: a forward zone reaches its resolver, bypasses the blocklist and caches" {
|
||||
if (!build_options.integration) return error.SkipZigTest;
|
||||
|
||||
const gpa = testing.allocator;
|
||||
var threaded: std.Io.Threaded = .init(gpa, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
// The zone resolver is a socket of its own, so the case can tell a query
|
||||
// that reached it from one the pool answered.
|
||||
const resolver_bind: net.IpAddress = try .parse("127.0.0.1", 0);
|
||||
const resolver_socket = try resolver_bind.bind(io, .{ .mode = .dgram });
|
||||
defer resolver_socket.close(io);
|
||||
|
||||
var resolver_calls: std.atomic.Value(u64) = .init(0);
|
||||
var resolver_group: std.Io.Group = .init;
|
||||
defer resolver_group.cancel(io);
|
||||
try resolver_group.concurrent(io, zoneResolver, .{ io, &resolver_socket, &resolver_calls });
|
||||
|
||||
var resolver_text: [64]u8 = undefined;
|
||||
const resolver_url = try std.fmt.bufPrint(&resolver_text, "udp://127.0.0.1:{d}", .{
|
||||
resolver_socket.address.ip4.port,
|
||||
});
|
||||
|
||||
var zones = try forward_zones.Zones.build(gpa, &.{
|
||||
.{ .zone = "lan.home", .resolver = resolver_url },
|
||||
});
|
||||
defer zones.deinit(gpa);
|
||||
|
||||
// The name is blocklisted, so an answer from the zone resolver is proof the
|
||||
// bypass (ruling 7) holds over the wire.
|
||||
var snapshot = try buildSnapshot(gpa, .{ .rules = &.{blockRule("nas.lan.home")} });
|
||||
defer snapshot.deinit();
|
||||
var mgr: manager.Manager = undefined;
|
||||
fixtureManager(&mgr, &snapshot);
|
||||
|
||||
var cache: dns_cache.DnsCache = try .init(gpa, .{ .size = 8, .negative_ttl_max = 3600 });
|
||||
defer cache.deinit();
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = .a };
|
||||
var h = baseHandler(fake.client());
|
||||
h.manager = &mgr;
|
||||
h.zones = &zones;
|
||||
h.cache = &cache;
|
||||
h.negative_ttl_max = 3600;
|
||||
|
||||
var loop = try Loop.bind(gpa, io, &h);
|
||||
defer loop.stop(gpa, io);
|
||||
try loop.start(io);
|
||||
|
||||
var query_buf: [512]u8 = undefined;
|
||||
var reply_buf: [udp_server.max_datagram]u8 = undefined;
|
||||
const query = queryFor(&query_buf, 0x4444, "nas.lan.home", .a);
|
||||
|
||||
const first = try loop.ask(io, query, &reply_buf);
|
||||
const p = try packet.parse(first);
|
||||
try testing.expectEqual(types.Rcode.no_error, p.header.flags.rcode);
|
||||
try testing.expectEqual(zone_rdata, try record.rdataA(p.bytes, try firstAnswer(p)));
|
||||
try testing.expectEqual(@as(u64, 1), resolver_calls.load(.monotonic));
|
||||
try testing.expectEqual(@as(u64, 0), fake.calls.load(.monotonic));
|
||||
try testing.expectEqual(@as(u64, 0), h.stats.blocked.load(.monotonic));
|
||||
try testing.expectEqual(@as(u32, 1), cache.len());
|
||||
|
||||
// The second query is answered from the cache: the resolver socket sees
|
||||
// nothing more (PLAN §6.5).
|
||||
var second_buf: [512]u8 = undefined;
|
||||
const second_query = queryFor(&second_buf, 0x4455, "nas.lan.home", .a);
|
||||
const second = try loop.ask(io, second_query, &reply_buf);
|
||||
|
||||
const second_p = try packet.parse(second);
|
||||
try testing.expectEqual(@as(u16, 0x4455), second_p.header.id);
|
||||
try testing.expectEqual(zone_rdata, try record.rdataA(second_p.bytes, try firstAnswer(second_p)));
|
||||
try testing.expectEqual(@as(u64, 1), resolver_calls.load(.monotonic));
|
||||
try testing.expectEqual(@as(u64, 1), h.stats.cache_hits.load(.monotonic));
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// case 5: cache
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test "S7 case 5: a cached answer comes back with a fresh id, an aged ttl and a logged hit" {
|
||||
if (!build_options.integration) return error.SkipZigTest;
|
||||
|
||||
const gpa = testing.allocator;
|
||||
var threaded: std.Io.Threaded = .init(gpa, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var cache: dns_cache.DnsCache = try .init(gpa, .{ .size = 8, .negative_ttl_max = 3600 });
|
||||
defer cache.deinit();
|
||||
|
||||
var queue_buf: [log_queue_len]logger_mod.Entry = undefined;
|
||||
var lg: logger_mod.Logger = .init(.{}, &queue_buf);
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = .a };
|
||||
var h = baseHandler(fake.client());
|
||||
h.cache = &cache;
|
||||
h.negative_ttl_max = 3600;
|
||||
h.logger = ≶
|
||||
|
||||
var loop = try Loop.bind(gpa, io, &h);
|
||||
defer loop.stop(gpa, io);
|
||||
try loop.start(io);
|
||||
|
||||
var reply_buf: [udp_server.max_datagram]u8 = undefined;
|
||||
|
||||
// Miss, then hit under a different transaction ID.
|
||||
var miss_buf: [512]u8 = undefined;
|
||||
_ = try loop.ask(io, queryFor(&miss_buf, 0x5501, "example.com", .a), &reply_buf);
|
||||
try testing.expectEqual(@as(u32, 1), cache.len());
|
||||
try testing.expectEqual(@as(u64, 1), fake.calls.load(.monotonic));
|
||||
|
||||
var hit_buf: [512]u8 = undefined;
|
||||
const hit = try loop.ask(io, queryFor(&hit_buf, 0x5502, "example.com", .a), &reply_buf);
|
||||
const p = try packet.parse(hit);
|
||||
try testing.expectEqual(@as(u16, 0x5502), p.header.id);
|
||||
try testing.expectEqual(upstream_rdata, try record.rdataA(p.bytes, try firstAnswer(p)));
|
||||
try testing.expectEqual(@as(u64, 1), fake.calls.load(.monotonic));
|
||||
try testing.expectEqual(@as(u64, 1), h.stats.cache_hits.load(.monotonic));
|
||||
|
||||
// Ageing needs elapsed time, and a test cannot wait 10 seconds for it. The
|
||||
// entry is therefore planted with a stored-at stamp 10 seconds in the past,
|
||||
// under exactly the key the handler builds for this query.
|
||||
var aged_query_buf: [512]u8 = undefined;
|
||||
const aged_query = queryFor(&aged_query_buf, 0x5503, "aged.example.com", .a);
|
||||
var stored_buf: [512]u8 = undefined;
|
||||
const aged_p = try packet.parse(aged_query);
|
||||
var b = try packet.ResponseBuilder.init(&stored_buf, aged_p.header, packet.firstQuestion(aged_p).?);
|
||||
try b.addAnswer(try name.fromText("aged.example.com"), .a, .in, upstream_ttl, &upstream_rdata);
|
||||
|
||||
var key_buf: [dns_cache.max_key_len]u8 = undefined;
|
||||
const key = dns_cache.buildKey(
|
||||
&key_buf,
|
||||
"aged.example.com",
|
||||
@intFromEnum(types.Type.a),
|
||||
@intFromEnum(types.Class.in),
|
||||
false,
|
||||
null,
|
||||
);
|
||||
const aged_by = 10;
|
||||
try cache.put(
|
||||
std.Io.Clock.real.now(io).toSeconds() - aged_by,
|
||||
key,
|
||||
b.finish(),
|
||||
.{ .ttl_seconds = upstream_ttl, .negative = false },
|
||||
);
|
||||
|
||||
const aged = try loop.ask(io, aged_query, &reply_buf);
|
||||
const aged_reply = try packet.parse(aged);
|
||||
try testing.expectEqual(upstream_ttl - aged_by, (try firstAnswer(aged_reply)).ttl);
|
||||
try testing.expectEqual(@as(u64, 1), fake.calls.load(.monotonic));
|
||||
|
||||
var entries: [log_queue_len]logger_mod.Entry = undefined;
|
||||
const logged = drainLog(&lg, io, &entries);
|
||||
try testing.expectEqual(@as(usize, 3), logged.len);
|
||||
try testing.expectEqual(@as(?bool, false), logged[0].cache_hit);
|
||||
try testing.expectEqualStrings("pool", logged[0].upstream());
|
||||
try testing.expectEqual(@as(?bool, true), logged[1].cache_hit);
|
||||
try testing.expectEqualStrings("", logged[1].upstream());
|
||||
try testing.expectEqual(@as(?bool, true), logged[2].cache_hit);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// case 6: CNAME uncloaking
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test "S7 case 6: a cname into a blocked target blocks the original question" {
|
||||
if (!build_options.integration) return error.SkipZigTest;
|
||||
|
||||
const gpa = testing.allocator;
|
||||
var threaded: std.Io.Threaded = .init(gpa, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var snapshot = try buildSnapshot(gpa, .{ .rules = &.{blockRule("tracker.example.org")} });
|
||||
defer snapshot.deinit();
|
||||
var mgr: manager.Manager = undefined;
|
||||
fixtureManager(&mgr, &snapshot);
|
||||
|
||||
var queue_buf: [log_queue_len]logger_mod.Entry = undefined;
|
||||
var lg: logger_mod.Logger = .init(.{}, &queue_buf);
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = .{ .cname = "tracker.example.org" } };
|
||||
var h = baseHandler(fake.client());
|
||||
h.manager = &mgr;
|
||||
h.logger = ≶
|
||||
|
||||
var loop = try Loop.bind(gpa, io, &h);
|
||||
defer loop.stop(gpa, io);
|
||||
try loop.start(io);
|
||||
|
||||
var query_buf: [512]u8 = undefined;
|
||||
var reply_buf: [udp_server.max_datagram]u8 = undefined;
|
||||
const reply = try loop.ask(io, queryFor(&query_buf, 0x6666, "cdn.example.com", .a), &reply_buf);
|
||||
|
||||
const p = try packet.parse(reply);
|
||||
try testing.expectEqual(@as(u16, 1), p.header.ancount);
|
||||
|
||||
// The answer is about the name the client asked for, not the target.
|
||||
const answer = try firstAnswer(p);
|
||||
try testing.expectEqual(types.Type.a, answer.rtype);
|
||||
try testing.expectEqualSlices(
|
||||
u8,
|
||||
(try name.fromText("cdn.example.com")).wire(),
|
||||
answer.name.wire(),
|
||||
);
|
||||
try testing.expectEqual([4]u8{ 0, 0, 0, 0 }, try record.rdataA(p.bytes, answer));
|
||||
try testing.expectEqual(@as(u64, 1), h.stats.uncloak_blocked.load(.monotonic));
|
||||
|
||||
var entries: [log_queue_len]logger_mod.Entry = undefined;
|
||||
const logged = drainLog(&lg, io, &entries);
|
||||
try testing.expectEqual(@as(usize, 1), logged.len);
|
||||
try testing.expectEqual(true, logged[0].blocked);
|
||||
try testing.expectEqualStrings("cname:rule_block_exact", logged[0].blockReason());
|
||||
try testing.expectEqualStrings("cdn.example.com", logged[0].domain());
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// case 7: safe search
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test "S7 case 7: safe search answers the original question with a cname to the target" {
|
||||
if (!build_options.integration) return error.SkipZigTest;
|
||||
|
||||
const gpa = testing.allocator;
|
||||
var threaded: std.Io.Threaded = .init(gpa, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var snapshot = try buildSnapshot(gpa, .{
|
||||
.groups = &.{.{ .name = "default", .safe_search = true }},
|
||||
});
|
||||
defer snapshot.deinit();
|
||||
var mgr: manager.Manager = undefined;
|
||||
fixtureManager(&mgr, &snapshot);
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = .a };
|
||||
var h = baseHandler(fake.client());
|
||||
h.manager = &mgr;
|
||||
|
||||
var loop = try Loop.bind(gpa, io, &h);
|
||||
defer loop.stop(gpa, io);
|
||||
try loop.start(io);
|
||||
|
||||
var query_buf: [512]u8 = undefined;
|
||||
var reply_buf: [udp_server.max_datagram]u8 = undefined;
|
||||
const reply = try loop.ask(io, queryFor(&query_buf, 0x7777, "www.google.com", .a), &reply_buf);
|
||||
|
||||
const p = try packet.parse(reply);
|
||||
const target = try name.fromText("forcesafesearch.google.com");
|
||||
|
||||
// The reply keeps the question the client asked.
|
||||
try testing.expectEqualSlices(
|
||||
u8,
|
||||
(try name.fromText("www.google.com")).wire(),
|
||||
packet.firstQuestion(p).?.name.wire(),
|
||||
);
|
||||
try testing.expectEqual(@as(u16, 2), p.header.ancount);
|
||||
|
||||
var it = packet.answers(p);
|
||||
const cname = (try it.next()).?;
|
||||
try testing.expectEqual(types.Type.cname, cname.rtype);
|
||||
try testing.expectEqualSlices(u8, target.wire(), (try record.rdataCname(p.bytes, cname)).wire());
|
||||
|
||||
const a = (try it.next()).?;
|
||||
try testing.expectEqual(types.Type.a, a.rtype);
|
||||
try testing.expectEqualSlices(u8, target.wire(), a.name.wire());
|
||||
try testing.expectEqual(upstream_rdata, try record.rdataA(p.bytes, a));
|
||||
|
||||
try testing.expectEqual(@as(u64, 1), h.stats.safesearch_rewrites.load(.monotonic));
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// case 8: rate limit
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test "S7 case 8: the third query inside the window is refused" {
|
||||
if (!build_options.integration) return error.SkipZigTest;
|
||||
|
||||
const gpa = testing.allocator;
|
||||
var threaded: std.Io.Threaded = .init(gpa, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var limiter: rate_limiter.RateLimiter = try .init(gpa, .{ .limit = 2, .window_seconds = 60 });
|
||||
defer limiter.deinit();
|
||||
|
||||
var queue_buf: [log_queue_len]logger_mod.Entry = undefined;
|
||||
var lg: logger_mod.Logger = .init(.{}, &queue_buf);
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = .a };
|
||||
var h = baseHandler(fake.client());
|
||||
h.limiter = &limiter;
|
||||
h.logger = ≶
|
||||
|
||||
var loop = try Loop.bind(gpa, io, &h);
|
||||
defer loop.stop(gpa, io);
|
||||
try loop.start(io);
|
||||
|
||||
var reply_buf: [udp_server.max_datagram]u8 = undefined;
|
||||
for ([_]u16{ 0x8801, 0x8802 }) |id| {
|
||||
var query_buf: [512]u8 = undefined;
|
||||
const reply = try loop.ask(io, queryFor(&query_buf, id, "example.com", .a), &reply_buf);
|
||||
try testing.expectEqual(types.Rcode.no_error, (try packet.parse(reply)).header.flags.rcode);
|
||||
}
|
||||
|
||||
var third_buf: [512]u8 = undefined;
|
||||
const refused = try loop.ask(io, queryFor(&third_buf, 0x8803, "example.com", .a), &reply_buf);
|
||||
const p = try packet.parse(refused);
|
||||
try testing.expectEqual(types.Rcode.refused, p.header.flags.rcode);
|
||||
try testing.expectEqual(@as(u16, 0x8803), p.header.id);
|
||||
try testing.expectEqual(@as(u64, 1), h.stats.refused.load(.monotonic));
|
||||
try testing.expectEqual(@as(u64, 2), fake.calls.load(.monotonic));
|
||||
|
||||
// Ruling 8: a refused query is never query-logged.
|
||||
var entries: [log_queue_len]logger_mod.Entry = undefined;
|
||||
try testing.expectEqual(@as(usize, 2), drainLog(&lg, io, &entries).len);
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// case 9: pause
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test "S7 case 9: pause lifts filtering and unpause restores it" {
|
||||
if (!build_options.integration) return error.SkipZigTest;
|
||||
|
||||
const gpa = testing.allocator;
|
||||
var threaded: std.Io.Threaded = .init(gpa, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var snapshot = try buildSnapshot(gpa, .{ .rules = &.{blockRule("ads.example.com")} });
|
||||
defer snapshot.deinit();
|
||||
var mgr: manager.Manager = undefined;
|
||||
fixtureManager(&mgr, &snapshot);
|
||||
|
||||
var paused: pause.Pause = .{};
|
||||
paused.pauseFor(std.Io.Clock.real.now(io).toSeconds(), null);
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = .a };
|
||||
var h = baseHandler(fake.client());
|
||||
h.manager = &mgr;
|
||||
h.pause = &paused;
|
||||
|
||||
var loop = try Loop.bind(gpa, io, &h);
|
||||
defer loop.stop(gpa, io);
|
||||
try loop.start(io);
|
||||
|
||||
var reply_buf: [udp_server.max_datagram]u8 = undefined;
|
||||
|
||||
var paused_buf: [512]u8 = undefined;
|
||||
const while_paused = try loop.ask(io, queryFor(&paused_buf, 0x9901, "ads.example.com", .a), &reply_buf);
|
||||
const p = try packet.parse(while_paused);
|
||||
try testing.expectEqual(upstream_rdata, try record.rdataA(p.bytes, try firstAnswer(p)));
|
||||
try testing.expectEqual(@as(u64, 1), h.stats.paused_queries.load(.monotonic));
|
||||
try testing.expectEqual(@as(u64, 0), h.stats.blocked.load(.monotonic));
|
||||
|
||||
// Resuming puts the block back without a restart (ruling 18).
|
||||
paused.unpause();
|
||||
|
||||
var resumed_buf: [512]u8 = undefined;
|
||||
const after = try loop.ask(io, queryFor(&resumed_buf, 0x9902, "ads.example.com", .a), &reply_buf);
|
||||
const after_p = try packet.parse(after);
|
||||
try testing.expectEqual([4]u8{ 0, 0, 0, 0 }, try record.rdataA(after_p.bytes, try firstAnswer(after_p)));
|
||||
try testing.expectEqual(@as(u64, 1), h.stats.blocked.load(.monotonic));
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// case 10: client tracking
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
test "S7 case 10: the querying client is materialised as a row" {
|
||||
if (!build_options.integration) return error.SkipZigTest;
|
||||
|
||||
const gpa = testing.allocator;
|
||||
var threaded: std.Io.Threaded = .init(gpa, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
// The tracker's own connection (ruling 21), in memory here: the flush is
|
||||
// what this case asserts on, not where the file lives.
|
||||
var database = try db.Db.open(":memory:", .{ .mode = .memory });
|
||||
defer database.close();
|
||||
try db.applyPragmas(&database, .{});
|
||||
_ = try migrations.migrate(&database);
|
||||
|
||||
var tracker: clients.Tracker = .init(30);
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = .a };
|
||||
var h = baseHandler(fake.client());
|
||||
h.tracker = &tracker;
|
||||
|
||||
var loop = try Loop.bind(gpa, io, &h);
|
||||
defer loop.stop(gpa, io);
|
||||
try loop.start(io);
|
||||
|
||||
var reply_buf: [udp_server.max_datagram]u8 = undefined;
|
||||
for ([_]u16{ 0xa001, 0xa002 }) |id| {
|
||||
var query_buf: [512]u8 = undefined;
|
||||
_ = try loop.ask(io, queryFor(&query_buf, id, "example.com", .a), &reply_buf);
|
||||
}
|
||||
|
||||
// Two queries from one client are one pending entry, and the forced pass
|
||||
// stands in for the 60-second flush interval (S4 As-built seam).
|
||||
try testing.expectEqual(@as(u32, 1), tracker.pendingClients(io));
|
||||
tracker.flushOnce(io, &database, true);
|
||||
|
||||
try testing.expectEqual(@as(i64, 1), try clients_repo.countClients(&database));
|
||||
try testing.expectEqual(@as(u32, 0), tracker.pendingClients(io));
|
||||
try testing.expectEqual(@as(u64, 1), tracker.snapshotStats(io).flushed);
|
||||
|
||||
var stmt = try database.prepare("SELECT ip, hand_edited FROM clients");
|
||||
defer stmt.deinit();
|
||||
try testing.expect(try stmt.step());
|
||||
try testing.expectEqualStrings("127.0.0.1", stmt.columnText(0));
|
||||
try testing.expectEqual(@as(i64, 0), stmt.columnInt(1));
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// case 11: the whole application
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/// `std.testing.tmpDir` creates its directory against `std.testing.io`, so the
|
||||
/// application under test runs on the same `Io` instance the fixture used. The
|
||||
/// other cases build an `Io.Threaded` of their own, the way the listener tests
|
||||
/// do; this one cannot, because the temporary directory is already bound to
|
||||
/// this instance.
|
||||
const test_io = testing.io;
|
||||
|
||||
/// Where `std.testing.tmpDir` puts its directories (`lib/std/testing.zig:634`).
|
||||
const tmp_prefix = ".zig-cache/tmp/";
|
||||
const sub_path_len = @typeInfo(@FieldType(testing.TmpDir, "sub_path")).array.len;
|
||||
|
||||
/// High enough to need no privilege, and not the 15353/15354 pair the milestone
|
||||
/// smoke test used, so a stray smoke process cannot make this case pass.
|
||||
const app_port = 15455;
|
||||
|
||||
/// The unreachable upstream the seed configuration names. Nothing in this case
|
||||
/// needs it: the query it resolves is a local record, and a dead upstream is
|
||||
/// what proves the fail-open design still serves.
|
||||
const dead_upstream = "https://127.0.0.1:9/dns-query";
|
||||
|
||||
const app_config =
|
||||
\\.{
|
||||
\\ .dns = .{
|
||||
\\ .bind_ipv4 = "127.0.0.1",
|
||||
\\ .bind_ipv6 = "::1",
|
||||
\\ .port = 15455,
|
||||
\\ .rate_limit = 1000,
|
||||
\\ .rate_window_seconds = 60,
|
||||
\\ },
|
||||
\\ .logging = .{ .level = .info, .output = .stderr },
|
||||
\\ .web = .{ .enabled = false },
|
||||
\\ .groups = .{ .{ .name = "default" } },
|
||||
\\ .upstreams = .{ .{ .url = "https://127.0.0.1:9/dns-query" } },
|
||||
\\ .local_records = .{
|
||||
\\ .{ .name = "boot.test", .rtype = .a, .value = "10.9.8.7", .ttl = 60 },
|
||||
\\ },
|
||||
\\}
|
||||
\\
|
||||
;
|
||||
|
||||
comptime {
|
||||
// The port and the upstream appear in the configuration text as literals,
|
||||
// because a `.zon` file is data and not a format string.
|
||||
std.debug.assert(std.mem.containsAtLeast(u8, app_config, 1, std.fmt.comptimePrint("{d}", .{app_port})));
|
||||
std.debug.assert(std.mem.containsAtLeast(u8, app_config, 1, dead_upstream));
|
||||
}
|
||||
|
||||
/// How long one attempt at reaching the booting server waits, and how many
|
||||
/// attempts it gets. The product is the time the application has to bind.
|
||||
const boot_attempt: std.Io.Timeout = .{ .duration = .{ .raw = .fromMilliseconds(200), .clock = .awake } };
|
||||
const boot_attempts = 100;
|
||||
|
||||
/// Queries the booting server until it answers. A server that has not bound yet
|
||||
/// either swallows the datagram or answers it with an ICMP rejection, and both
|
||||
/// arrive here as an error worth retrying.
|
||||
fn askUntilAnswered(
|
||||
socket: *const net.Socket,
|
||||
dest: net.IpAddress,
|
||||
query: []const u8,
|
||||
buf: []u8,
|
||||
) ![]u8 {
|
||||
var attempt: usize = 0;
|
||||
while (attempt < boot_attempts) : (attempt += 1) {
|
||||
socket.send(test_io, &dest, query) catch continue;
|
||||
const msg = socket.receiveTimeout(test_io, buf, boot_attempt) catch continue;
|
||||
return msg.data;
|
||||
}
|
||||
return error.TestAppNeverAnswered;
|
||||
}
|
||||
|
||||
test "S7 case 11: the app boots, serves a query and exits zero on shutdown" {
|
||||
if (!build_options.integration) return error.SkipZigTest;
|
||||
|
||||
const gpa = testing.allocator;
|
||||
|
||||
var tmp = testing.tmpDir(.{ .iterate = true });
|
||||
defer tmp.cleanup();
|
||||
try tmp.dir.writeFile(test_io, .{ .sub_path = "config.zon", .data = app_config });
|
||||
|
||||
var root_buf: [tmp_prefix.len + sub_path_len]u8 = undefined;
|
||||
@memcpy(root_buf[0..tmp_prefix.len], tmp_prefix);
|
||||
@memcpy(root_buf[tmp_prefix.len..], &tmp.sub_path);
|
||||
const root: []const u8 = &root_buf;
|
||||
|
||||
var config_buf: [root_buf.len + "/config.zon".len]u8 = undefined;
|
||||
const config_path = try std.fmt.bufPrint(&config_buf, "{s}/config.zon", .{root});
|
||||
|
||||
var out: std.Io.Writer.Allocating = .init(gpa);
|
||||
defer out.deinit();
|
||||
var err: std.Io.Writer.Allocating = .init(gpa);
|
||||
defer err.deinit();
|
||||
|
||||
const runner: cli.Runner = .{
|
||||
.io = test_io,
|
||||
.gpa = gpa,
|
||||
.out = &out.writer,
|
||||
.err = &err.writer,
|
||||
};
|
||||
|
||||
// The shutdown event is process-global, and another case in this binary may
|
||||
// have left it set.
|
||||
shutdown.reset();
|
||||
defer shutdown.reset();
|
||||
|
||||
var future = try test_io.concurrent(app.run, .{ runner, cli.Paths{
|
||||
.data_dir = root,
|
||||
.config = config_path,
|
||||
} });
|
||||
|
||||
const client_address: net.IpAddress = try .parse("127.0.0.1", 0);
|
||||
const client = try client_address.bind(test_io, .{ .mode = .dgram });
|
||||
defer client.close(test_io);
|
||||
|
||||
const server_address: net.IpAddress = try .parse("127.0.0.1", app_port);
|
||||
|
||||
var query_buf: [512]u8 = undefined;
|
||||
const query = queryFor(&query_buf, 0xb001, "boot.test", .a);
|
||||
var reply_buf: [udp_server.max_datagram]u8 = undefined;
|
||||
|
||||
const reply = askUntilAnswered(&client, server_address, query, &reply_buf) catch |e| {
|
||||
shutdown.trigger(test_io);
|
||||
_ = future.await(test_io);
|
||||
return e;
|
||||
};
|
||||
|
||||
const p = try packet.parse(reply);
|
||||
try testing.expectEqual(@as(u16, 0xb001), p.header.id);
|
||||
try testing.expectEqual(types.Rcode.no_error, p.header.flags.rcode);
|
||||
try testing.expectEqual(true, p.header.flags.aa);
|
||||
try testing.expectEqual([4]u8{ 10, 9, 8, 7 }, try record.rdataA(p.bytes, try firstAnswer(p)));
|
||||
|
||||
shutdown.trigger(test_io);
|
||||
try testing.expectEqual(cli.exit_ok, future.await(test_io));
|
||||
|
||||
// The lifecycle proof is the exit code, and a clean exit prints nothing.
|
||||
try testing.expectEqualStrings("", err.written());
|
||||
}
|
||||
@@ -18,6 +18,10 @@ const net = std.Io.net;
|
||||
const handler = @import("handler.zig");
|
||||
const tcp_server = @import("tcp_server.zig");
|
||||
const udp_server = @import("udp_server.zig");
|
||||
const model = @import("../config/model.zig");
|
||||
const response = @import("../filter/response.zig");
|
||||
const forward_zones = @import("../local/forward_zones.zig");
|
||||
const records = @import("../local/records.zig");
|
||||
const packet = @import("../dns/packet.zig");
|
||||
const record = @import("../dns/record.zig");
|
||||
const types = @import("../dns/types.zig");
|
||||
@@ -27,6 +31,31 @@ const transport = @import("../upstream/transport.zig");
|
||||
|
||||
const testing = std.testing;
|
||||
|
||||
const empty_records: records.Records = .empty;
|
||||
const empty_zones: forward_zones.Zones = .empty;
|
||||
const blocking_defaults: model.Blocking = .{};
|
||||
const blocking: response.Options = .{
|
||||
.mode = blocking_defaults.response,
|
||||
.ttl = blocking_defaults.ttl,
|
||||
};
|
||||
const forward_timeout: std.Io.Clock.Duration = .{
|
||||
.raw = model.readTimeout(.{}),
|
||||
.clock = .awake,
|
||||
};
|
||||
|
||||
/// An upstream and nothing else optional: no filtering, no cache, no log. The
|
||||
/// listeners and the pool are what this test exercises, so the handler is the
|
||||
/// same bare one its own tests use.
|
||||
fn bareHandler(client: transport.Client) handler.Handler {
|
||||
return .{
|
||||
.upstream = client,
|
||||
.blocking = blocking,
|
||||
.forward_read_timeout = forward_timeout,
|
||||
.records = &empty_records,
|
||||
.zones = &empty_zones,
|
||||
};
|
||||
}
|
||||
|
||||
/// Long enough that a loopback round trip cannot lose to scheduling, short
|
||||
/// enough that a broken server fails the run instead of hanging it.
|
||||
const budget: std.Io.Timeout = .{ .duration = .{ .raw = .fromSeconds(5), .clock = .awake } };
|
||||
@@ -226,7 +255,7 @@ test "the whole resolver answers over udp and tcp and fails over to a healthy up
|
||||
};
|
||||
var upstreams: pool.Pool = .init(&entries, test_cfg, attempt_timeout, 1);
|
||||
|
||||
var h: handler.Handler = .{ .upstream = upstreams.client() };
|
||||
var h = bareHandler(upstreams.client());
|
||||
|
||||
const listen_address: net.IpAddress = try .parse("127.0.0.1", 0);
|
||||
var udp = try udp_server.UdpServer.bind(gpa, io, listen_address, &h, .{ .max_in_flight = 4 });
|
||||
|
||||
@@ -0,0 +1,118 @@
|
||||
//! SIGINT and SIGTERM, turned into one `std.Io.Event`.
|
||||
//!
|
||||
//! No signalfd, no self-pipe, no epoll: the handler does exactly one thing, and
|
||||
//! `std.Io.Event.set` is async-signal-safe on the Threaded Linux backend — a
|
||||
//! raw `futex` wake with no allocation and no lock (`Io.zig:1855` →
|
||||
//! `Threaded.futexWake`). The `.mask`/`.flags` shape is the one Threaded uses
|
||||
//! for its own `SIG.IO`/`SIG.PIPE` handlers (`Threaded.zig:1653`): an empty
|
||||
//! mask and no `SA_RESTART`, so a blocking syscall returns `EINTR` and the
|
||||
//! backend's retry loop re-reads the cancellation state.
|
||||
//!
|
||||
//! Everything a shutdown actually has to do — drain the query log, cancel the
|
||||
//! task group, close the databases — happens on the task blocked in `wait`.
|
||||
//!
|
||||
//! The previous handlers are not restored. The process is leaving, and a
|
||||
//! second SIGTERM during teardown should still terminate it the default way
|
||||
//! only if the operator sends it before this module is armed.
|
||||
|
||||
const std = @import("std");
|
||||
const posix = std.posix;
|
||||
|
||||
var event: std.Io.Event = .unset;
|
||||
|
||||
/// Read by the signal handler, written by `install` before the handler exists.
|
||||
/// A `std.Io` is two pointers and cannot be stored atomically, so ordering is
|
||||
/// what makes the read safe: the store precedes the `sigaction` syscall that
|
||||
/// arms the handler, and no signal can reach the handler before that call
|
||||
/// returns.
|
||||
var handler_io: ?std.Io = null;
|
||||
|
||||
var installed: bool = false;
|
||||
|
||||
/// Arms the handlers for INT and TERM. Calling it again is a no-op: the process
|
||||
/// has one event and one pair of handlers, and a second boot inside one process
|
||||
/// (which only a test does) must not re-arm anything.
|
||||
pub fn install(io: std.Io) void {
|
||||
if (installed) return;
|
||||
handler_io = io;
|
||||
installed = true;
|
||||
|
||||
const act: posix.Sigaction = .{
|
||||
.handler = .{ .handler = onSignal },
|
||||
.mask = posix.sigemptyset(),
|
||||
.flags = 0,
|
||||
};
|
||||
posix.sigaction(.INT, &act, null);
|
||||
posix.sigaction(.TERM, &act, null);
|
||||
}
|
||||
|
||||
fn onSignal(_: posix.SIG) callconv(.c) void {
|
||||
const io = handler_io orelse return;
|
||||
event.set(io);
|
||||
}
|
||||
|
||||
/// Blocks until a shutdown is requested. A canceled wait is the caller's cue to
|
||||
/// tear down as well, which is why `app.run` treats both results the same.
|
||||
pub fn wait(io: std.Io) std.Io.Cancelable!void {
|
||||
return event.wait(io);
|
||||
}
|
||||
|
||||
/// The programmatic equivalent of the signal: what a test uses to shut the app
|
||||
/// down, and what a Phase 8 restart endpoint would call.
|
||||
pub fn trigger(io: std.Io) void {
|
||||
event.set(io);
|
||||
}
|
||||
|
||||
pub fn isRequested() bool {
|
||||
return event.isSet();
|
||||
}
|
||||
|
||||
/// Clears the request so the next `wait` blocks again. Only a test that boots
|
||||
/// the app more than once in one process needs this; a served process shuts
|
||||
/// down once.
|
||||
pub fn reset() void {
|
||||
event.reset();
|
||||
}
|
||||
|
||||
const testing = std.testing;
|
||||
|
||||
test "trigger releases a waiter and isRequested reports it" {
|
||||
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
reset();
|
||||
try testing.expect(!isRequested());
|
||||
|
||||
trigger(io);
|
||||
try testing.expect(isRequested());
|
||||
// Already set, so this returns without blocking.
|
||||
try wait(io);
|
||||
|
||||
reset();
|
||||
try testing.expect(!isRequested());
|
||||
}
|
||||
|
||||
test "a waiting task is released by a later trigger" {
|
||||
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
reset();
|
||||
|
||||
var group: std.Io.Group = .init;
|
||||
try group.concurrent(io, waitThenSet, .{ io, &done });
|
||||
trigger(io);
|
||||
try group.await(io);
|
||||
|
||||
try testing.expect(done.isSet());
|
||||
reset();
|
||||
done.reset();
|
||||
}
|
||||
|
||||
var done: std.Io.Event = .unset;
|
||||
|
||||
fn waitThenSet(io: std.Io, flag: *std.Io.Event) std.Io.Cancelable!void {
|
||||
try wait(io);
|
||||
flag.set(io);
|
||||
}
|
||||
+80
-17
@@ -12,8 +12,25 @@
|
||||
//! No stream read or write in 0.16.0 accepts a timeout, so every per-connection
|
||||
//! operation is raced against `Options.idle_timeout` through `std.Io.Select` and
|
||||
//! the loser is canceled.
|
||||
//!
|
||||
//! Shutdown takes one of two paths, and they end the live connections
|
||||
//! differently on purpose:
|
||||
//!
|
||||
//! - `deinit` shuts every active stream down first, so the connections unblock
|
||||
//! and finish by themselves. `serve` then drains them, and a reply that was
|
||||
//! half written still goes out whole.
|
||||
//! - A canceled `serve` cannot drain. `deinit` is what would shut the streams
|
||||
//! down, and it cannot run until `serve` returns — the composition root
|
||||
//! cancels its task group before it releases anything (app.zig). Meanwhile
|
||||
//! RFC 7766 §6.2.1.1 lets a client hold a connection open indefinitely by
|
||||
//! asking again inside the idle budget, so draining would let one chatty
|
||||
//! client stall the whole process's shutdown. The connections are canceled
|
||||
//! instead, at the cost of the one reply that was mid-write.
|
||||
//!
|
||||
//! Either way `serve` returns only once no task can still touch a slot.
|
||||
|
||||
const std = @import("std");
|
||||
const address = @import("../platform/address.zig");
|
||||
const handler = @import("handler.zig");
|
||||
const transport = @import("../upstream/transport.zig");
|
||||
|
||||
@@ -53,6 +70,16 @@ const State = enum(u32) { idle, serving, closing };
|
||||
/// before the close, and `deinit` only touches `.active` slots.
|
||||
const ConnState = enum { free, active, closing };
|
||||
|
||||
/// Why the accept loop stopped, which decides what happens to the connections
|
||||
/// still in flight.
|
||||
const Stop = enum {
|
||||
/// `deinit` published `.closing`. It has already shut every live connection
|
||||
/// down, so each one is unblocked and finishing on its own.
|
||||
closing,
|
||||
/// This task is being canceled. Nothing has touched the connections.
|
||||
canceled,
|
||||
};
|
||||
|
||||
/// What the accept loop does with a stream it has just accepted.
|
||||
const Claim = union(enum) {
|
||||
/// The stream owns `conns[index]`.
|
||||
@@ -77,7 +104,7 @@ pub const TcpServer = struct {
|
||||
state: std.atomic.Value(State),
|
||||
stopped: std.Io.Event,
|
||||
|
||||
/// One slot is ~131 KiB, so the default 64 connections cost ~8.4 MiB, which
|
||||
/// One slot is ~137 KiB, so the default 64 connections cost ~8.8 MiB, which
|
||||
/// is inside the PLAN §18 budget. The two message buffers cannot be shared
|
||||
/// or shrunk: the handler holds the query while the reply is built, and
|
||||
/// both ceilings are the 65535 bytes the length prefix can express.
|
||||
@@ -86,7 +113,15 @@ pub const TcpServer = struct {
|
||||
reply: [transport.max_message_len]u8,
|
||||
read_buf: [stream_buffer_len]u8,
|
||||
write_buf: [stream_buffer_len]u8,
|
||||
/// The handler's per-query working memory. It belongs to the slot so
|
||||
/// that answering a message allocates nothing, and a connection is
|
||||
/// answered serially, so one query uses it at a time.
|
||||
scratch: handler.Scratch,
|
||||
stream: std.Io.net.Stream,
|
||||
/// The client, read off the accepted socket once at claim time: every
|
||||
/// message on this connection comes from the same peer, and the handler
|
||||
/// needs it for rate limiting, groups and the query log.
|
||||
peer: std.Io.net.IpAddress,
|
||||
/// Guarded by `TcpServer.mutex`.
|
||||
state: ConnState,
|
||||
};
|
||||
@@ -96,7 +131,7 @@ pub const TcpServer = struct {
|
||||
pub fn listen(
|
||||
gpa: std.mem.Allocator,
|
||||
io: std.Io,
|
||||
address: std.Io.net.IpAddress,
|
||||
listen_address: std.Io.net.IpAddress,
|
||||
h: *handler.Handler,
|
||||
options: Options,
|
||||
) ListenError!TcpServer {
|
||||
@@ -106,7 +141,7 @@ pub const TcpServer = struct {
|
||||
errdefer gpa.free(conns);
|
||||
for (conns) |*conn| conn.state = .free;
|
||||
|
||||
const local = address;
|
||||
const local = listen_address;
|
||||
const server = try local.listen(io, .{ .reuse_address = true });
|
||||
|
||||
return .{
|
||||
@@ -132,15 +167,28 @@ pub const TcpServer = struct {
|
||||
if (self.state.cmpxchgStrong(.idle, .serving, .acq_rel, .acquire) != null) return;
|
||||
|
||||
var group: std.Io.Group = .init;
|
||||
self.acceptLoop(io, &group);
|
||||
|
||||
// A reply that is half written is worse than no reply, so the live
|
||||
// connections are awaited even when this task is being canceled.
|
||||
const prev = io.swapCancelProtection(.blocked);
|
||||
group.await(io) catch |err| switch (err) {
|
||||
error.Canceled => unreachable,
|
||||
};
|
||||
_ = io.swapCancelProtection(prev);
|
||||
switch (self.acceptLoop(io, &group)) {
|
||||
// `deinit` shut every live connection down before it published
|
||||
// `.closing`, so each one is already unblocked and ending on its
|
||||
// own. Awaiting them means a half-written reply still goes out
|
||||
// whole, and the wait is bounded by the shutdown, not the client.
|
||||
.closing => {
|
||||
const prev = io.swapCancelProtection(.blocked);
|
||||
group.await(io) catch |err| switch (err) {
|
||||
error.Canceled => unreachable,
|
||||
};
|
||||
_ = io.swapCancelProtection(prev);
|
||||
},
|
||||
// Nothing has shut these connections down: `deinit` cannot run
|
||||
// until this task returns, and RFC 7766 lets a client hold a
|
||||
// connection open forever by asking again inside the idle budget.
|
||||
// Draining here would therefore let one client stall the whole
|
||||
// process's shutdown for as long as it likes. `cancel` requests
|
||||
// cancellation and joins, so the slots are still quiet — and the
|
||||
// buffers still unreferenced — by the time `serve` returns; the
|
||||
// price is the one reply that was mid-write.
|
||||
.canceled => group.cancel(io),
|
||||
}
|
||||
|
||||
self.stopped.set(io);
|
||||
}
|
||||
@@ -167,14 +215,17 @@ pub const TcpServer = struct {
|
||||
self.* = undefined;
|
||||
}
|
||||
|
||||
fn acceptLoop(self: *TcpServer, io: std.Io, group: *std.Io.Group) void {
|
||||
fn acceptLoop(self: *TcpServer, io: std.Io, group: *std.Io.Group) Stop {
|
||||
while (self.state.load(.acquire) == .serving) {
|
||||
const stream = self.server.accept(io) catch |err| switch (err) {
|
||||
error.Canceled, error.SocketNotListening => return,
|
||||
error.Canceled => return .canceled,
|
||||
// `deinit` shuts the listening socket down to unblock exactly
|
||||
// this call, so it is the shutdown path arriving early.
|
||||
error.SocketNotListening => return .closing,
|
||||
else => {
|
||||
bump(&self.stats.accept_errors);
|
||||
log.debug("tcp accept failed: {t}", .{err});
|
||||
retry_delay.sleep(io) catch return;
|
||||
retry_delay.sleep(io) catch return .canceled;
|
||||
continue;
|
||||
},
|
||||
};
|
||||
@@ -192,7 +243,7 @@ pub const TcpServer = struct {
|
||||
.shutting_down => {
|
||||
bump(&self.stats.rejected_at_shutdown);
|
||||
stream.close(io);
|
||||
return;
|
||||
return .closing;
|
||||
},
|
||||
};
|
||||
|
||||
@@ -206,6 +257,9 @@ pub const TcpServer = struct {
|
||||
|
||||
bump(&self.stats.accepted);
|
||||
}
|
||||
|
||||
// The loop condition failed, which only `deinit` can cause.
|
||||
return .closing;
|
||||
}
|
||||
|
||||
fn serveConn(self: *TcpServer, io: std.Io, index: usize) void {
|
||||
@@ -258,7 +312,15 @@ pub const TcpServer = struct {
|
||||
},
|
||||
}
|
||||
|
||||
const bytes = switch (self.handler.handle(io, .tcp, conn.query[0..len], &conn.reply)) {
|
||||
const outcome = self.handler.handle(
|
||||
io,
|
||||
.tcp,
|
||||
address.NetAddress.fromIp(conn.peer),
|
||||
conn.query[0..len],
|
||||
&conn.reply,
|
||||
&conn.scratch,
|
||||
);
|
||||
const bytes = switch (outcome) {
|
||||
// There is no framing for "no answer", so the connection ends.
|
||||
.drop => return,
|
||||
.reply => |b| b,
|
||||
@@ -286,6 +348,7 @@ pub const TcpServer = struct {
|
||||
switch (outcome) {
|
||||
.slot => |index| {
|
||||
self.conns[index].stream = stream;
|
||||
self.conns[index].peer = stream.socket.address;
|
||||
self.conns[index].state = .active;
|
||||
},
|
||||
.at_capacity, .shutting_down => {},
|
||||
|
||||
@@ -15,6 +15,10 @@ const net = std.Io.net;
|
||||
|
||||
const handler = @import("handler.zig");
|
||||
const tcp_server = @import("tcp_server.zig");
|
||||
const model = @import("../config/model.zig");
|
||||
const response = @import("../filter/response.zig");
|
||||
const forward_zones = @import("../local/forward_zones.zig");
|
||||
const records = @import("../local/records.zig");
|
||||
const header = @import("../dns/header.zig");
|
||||
const packet = @import("../dns/packet.zig");
|
||||
const types = @import("../dns/types.zig");
|
||||
@@ -22,6 +26,31 @@ const transport = @import("../upstream/transport.zig");
|
||||
|
||||
const testing = std.testing;
|
||||
|
||||
const empty_records: records.Records = .empty;
|
||||
const empty_zones: forward_zones.Zones = .empty;
|
||||
const blocking_defaults: model.Blocking = .{};
|
||||
const blocking: response.Options = .{
|
||||
.mode = blocking_defaults.response,
|
||||
.ttl = blocking_defaults.ttl,
|
||||
};
|
||||
const forward_timeout: std.Io.Clock.Duration = .{
|
||||
.raw = model.readTimeout(.{}),
|
||||
.clock = .awake,
|
||||
};
|
||||
|
||||
/// An upstream and nothing else optional: no filtering, no cache, no log. The
|
||||
/// listener is what these tests exercise, so the handler is the same bare one
|
||||
/// its own tests use.
|
||||
fn bareHandler(client: transport.Client) handler.Handler {
|
||||
return .{
|
||||
.upstream = client,
|
||||
.blocking = blocking,
|
||||
.forward_read_timeout = forward_timeout,
|
||||
.records = &empty_records,
|
||||
.zones = &empty_zones,
|
||||
};
|
||||
}
|
||||
|
||||
const budget: std.Io.Clock.Duration = .{ .raw = .fromSeconds(5), .clock = .awake };
|
||||
|
||||
/// Short enough to keep the idle-timeout test quick, long enough that a
|
||||
@@ -150,7 +179,7 @@ test "two length-prefixed queries share one connection" {
|
||||
const io = threaded.io();
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = response_bytes };
|
||||
var h: handler.Handler = .{ .upstream = fake.client() };
|
||||
var h = bareHandler(fake.client());
|
||||
|
||||
const listen_address: net.IpAddress = try .parse("127.0.0.1", 0);
|
||||
var server = try tcp_server.TcpServer.listen(gpa, io, listen_address, &h, .{ .max_connections = 2 });
|
||||
@@ -171,6 +200,145 @@ test "two length-prefixed queries share one connection" {
|
||||
};
|
||||
}
|
||||
|
||||
test "the claimed slot records the connecting client" {
|
||||
if (!build_options.integration) return error.SkipZigTest;
|
||||
|
||||
const gpa = testing.allocator;
|
||||
var threaded: std.Io.Threaded = .init(gpa, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = response_bytes };
|
||||
var h = bareHandler(fake.client());
|
||||
|
||||
const listen_address: net.IpAddress = try .parse("127.0.0.1", 0);
|
||||
var server = try tcp_server.TcpServer.listen(gpa, io, listen_address, &h, .{ .max_connections = 2 });
|
||||
const server_address = server.boundAddress();
|
||||
|
||||
var group: std.Io.Group = .init;
|
||||
try group.concurrent(io, tcp_server.TcpServer.serve, .{ &server, io });
|
||||
|
||||
try bounded(io, twoQueriesOnOneConnection, .{ io, server_address });
|
||||
|
||||
// The only connection took slot 0, and the reply the client already read
|
||||
// was written after `claim` filled the slot in, so this read races nothing.
|
||||
// Without a real peer the handler would rate-limit, group and log every TCP
|
||||
// client under whatever the uninitialized slot happened to hold.
|
||||
const peer = server.conns[0].peer;
|
||||
try testing.expectEqual(net.IpAddress.ip4, std.meta.activeTag(peer));
|
||||
try testing.expectEqualSlices(u8, &[_]u8{ 127, 0, 0, 1 }, &peer.ip4.bytes);
|
||||
try testing.expect(peer.ip4.port != 0);
|
||||
|
||||
server.deinit(gpa, io);
|
||||
group.await(io) catch |err| switch (err) {
|
||||
error.Canceled => unreachable,
|
||||
};
|
||||
}
|
||||
|
||||
/// `std.Io.Group` takes only `Cancelable!void`, so the result is reported out
|
||||
/// of band. `answered` is set either way — a client that fails early must not
|
||||
/// leave the test blocked waiting for a reply that will never come, and `set`
|
||||
/// is documented to have no effect the second time.
|
||||
fn holdConnection(
|
||||
io: std.Io,
|
||||
address: net.IpAddress,
|
||||
answered: *std.Io.Event,
|
||||
result: *anyerror!void,
|
||||
) void {
|
||||
result.* = holdConnectionOpen(io, address, answered);
|
||||
answered.set(io);
|
||||
}
|
||||
|
||||
/// Holds a connection open the way RFC 7766 lets a real client hold one: one
|
||||
/// query answered, then nothing, so the server sits blocked on the read for the
|
||||
/// next message. Returns once the server ends the connection, however it ends
|
||||
/// it — a canceled server closes the socket, which the client sees either as
|
||||
/// end of stream or as a reset.
|
||||
fn holdConnectionOpen(io: std.Io, address: net.IpAddress, answered: *std.Io.Event) anyerror!void {
|
||||
const remote = address;
|
||||
var stream = try remote.connect(io, .{ .mode = .stream });
|
||||
defer stream.close(io);
|
||||
|
||||
var read_buf: [1024]u8 = undefined;
|
||||
var write_buf: [1024]u8 = undefined;
|
||||
var reader = stream.reader(io, &read_buf);
|
||||
var writer = stream.writer(io, &write_buf);
|
||||
|
||||
try writer.interface.writeAll(&transport.framePrefix(@intCast(query_bytes.len)));
|
||||
try writer.interface.writeAll(query_bytes);
|
||||
try writer.interface.flush();
|
||||
|
||||
const len = transport.parsePrefix((try reader.interface.takeArray(transport.prefix_len)).*);
|
||||
try expectAnswersQuery(try reader.interface.take(len));
|
||||
|
||||
answered.set(io);
|
||||
|
||||
var sink: [64]u8 = undefined;
|
||||
_ = reader.interface.readSliceShort(&sink) catch {};
|
||||
}
|
||||
|
||||
test "a canceled serve does not wait for a live connection" {
|
||||
if (!build_options.integration) return error.SkipZigTest;
|
||||
|
||||
const gpa = testing.allocator;
|
||||
var threaded: std.Io.Threaded = .init(gpa, .{});
|
||||
defer threaded.deinit();
|
||||
const io = threaded.io();
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = response_bytes };
|
||||
var h = bareHandler(fake.client());
|
||||
|
||||
const listen_address: net.IpAddress = try .parse("127.0.0.1", 0);
|
||||
// The idle budget is the whole time a drain would have to wait out, so it
|
||||
// is set far beyond any patience this test run has: if `serve` waits for
|
||||
// the connection instead of canceling it, the wait never ends.
|
||||
var server = try tcp_server.TcpServer.listen(gpa, io, listen_address, &h, .{
|
||||
.max_connections = 2,
|
||||
.idle_timeout = .{ .raw = .fromSeconds(600), .clock = .awake },
|
||||
});
|
||||
const server_address = server.boundAddress();
|
||||
|
||||
var serving: std.Io.Group = .init;
|
||||
try serving.concurrent(io, tcp_server.TcpServer.serve, .{ &server, io });
|
||||
|
||||
var answered: std.Io.Event = .unset;
|
||||
var client_result: anyerror!void = {};
|
||||
var client_group: std.Io.Group = .init;
|
||||
try client_group.concurrent(io, holdConnection, .{ io, server_address, &answered, &client_result });
|
||||
|
||||
// The reply proves the connection is claimed and served, so the connection
|
||||
// task is now blocked reading the message that never comes. That is the
|
||||
// state a drain hangs in.
|
||||
answered.waitUncancelable(io);
|
||||
|
||||
// This is the composition root's shutdown: cancel the task group before
|
||||
// anything it borrows is released, so no `deinit` has shut the connection
|
||||
// down. It must still return. A regression here does not fail the test, it
|
||||
// hangs the run — there is no way to bound a join that does not finish.
|
||||
serving.cancel(io);
|
||||
|
||||
// Cancellation must not skip the per-connection cleanup: the slot is
|
||||
// released and the socket closed by `serveConn`'s defer, which runs on the
|
||||
// canceled path like any other.
|
||||
try testing.expectEqual(@as(?usize, 0), firstFreeSlot(&server));
|
||||
|
||||
client_group.cancel(io);
|
||||
server.deinit(gpa, io);
|
||||
|
||||
// Checked last: the connection had to be answered for the test to mean
|
||||
// anything, and the server is torn down before a failure is reported.
|
||||
try client_result;
|
||||
}
|
||||
|
||||
/// The first slot the server would hand out, read after `serve` has returned so
|
||||
/// nothing can be writing it.
|
||||
fn firstFreeSlot(server: *const tcp_server.TcpServer) ?usize {
|
||||
for (server.conns, 0..) |*conn, index| {
|
||||
if (conn.state == .free) return index;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
test "an idle connection is closed and counted" {
|
||||
if (!build_options.integration) return error.SkipZigTest;
|
||||
|
||||
@@ -180,7 +348,7 @@ test "an idle connection is closed and counted" {
|
||||
const io = threaded.io();
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = response_bytes };
|
||||
var h: handler.Handler = .{ .upstream = fake.client() };
|
||||
var h = bareHandler(fake.client());
|
||||
|
||||
const listen_address: net.IpAddress = try .parse("127.0.0.1", 0);
|
||||
var server = try tcp_server.TcpServer.listen(gpa, io, listen_address, &h, .{
|
||||
@@ -213,7 +381,7 @@ test "a zero-length message is a connection error" {
|
||||
const io = threaded.io();
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = response_bytes };
|
||||
var h: handler.Handler = .{ .upstream = fake.client() };
|
||||
var h = bareHandler(fake.client());
|
||||
|
||||
const listen_address: net.IpAddress = try .parse("127.0.0.1", 0);
|
||||
var server = try tcp_server.TcpServer.listen(gpa, io, listen_address, &h, .{
|
||||
@@ -265,7 +433,7 @@ test "deinit ends a serve loop that is blocked on accept" {
|
||||
const io = threaded.io();
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = response_bytes };
|
||||
var h: handler.Handler = .{ .upstream = fake.client() };
|
||||
var h = bareHandler(fake.client());
|
||||
|
||||
const listen_address: net.IpAddress = try .parse("127.0.0.1", 0);
|
||||
var server = try tcp_server.TcpServer.listen(gpa, io, listen_address, &h, .{ .max_connections = 2 });
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
//! allocates nothing. The pool is sized once in `bind` and never grows.
|
||||
|
||||
const std = @import("std");
|
||||
const address = @import("../platform/address.zig");
|
||||
const handler = @import("handler.zig");
|
||||
const transport = @import("../upstream/transport.zig");
|
||||
|
||||
@@ -75,10 +76,14 @@ pub const UdpServer = struct {
|
||||
/// The datagram actually sent stays bounded by `udpLimit` inside the
|
||||
/// handler, so nothing larger than 4096 bytes leaves this socket.
|
||||
///
|
||||
/// Cost: 4096 + 65535 ≈ 68 KiB per slot, so the default 64 slots hold
|
||||
/// ≈ 4.3 MiB. The PLAN §18 budget is 100 MB with ~1M blocked domains,
|
||||
/// so this pool takes about 4% of it.
|
||||
/// Cost: 4096 + 65535 + the scratch below ≈ 74 KiB per slot, so the
|
||||
/// default 64 slots hold ≈ 4.6 MiB. The PLAN §18 budget is 100 MB with
|
||||
/// ~1M blocked domains, so this pool takes about 5% of it.
|
||||
reply: [transport.max_message_len]u8,
|
||||
/// The handler's per-query working memory. It belongs to the slot so
|
||||
/// that answering a datagram still allocates nothing, and one slot
|
||||
/// serves one query at a time.
|
||||
scratch: handler.Scratch,
|
||||
from: std.Io.net.IpAddress,
|
||||
len: usize,
|
||||
/// Guarded by `UdpServer.mutex`.
|
||||
@@ -90,7 +95,7 @@ pub const UdpServer = struct {
|
||||
pub fn bind(
|
||||
gpa: std.mem.Allocator,
|
||||
io: std.Io,
|
||||
address: std.Io.net.IpAddress,
|
||||
bind_address: std.Io.net.IpAddress,
|
||||
h: *handler.Handler,
|
||||
options: Options,
|
||||
) BindError!UdpServer {
|
||||
@@ -100,7 +105,7 @@ pub const UdpServer = struct {
|
||||
errdefer gpa.free(slots);
|
||||
for (slots) |*slot| slot.in_use = false;
|
||||
|
||||
const local = address;
|
||||
const local = bind_address;
|
||||
const socket = try local.bind(io, .{ .mode = .dgram });
|
||||
|
||||
return .{
|
||||
@@ -204,7 +209,15 @@ pub const UdpServer = struct {
|
||||
const slot = &self.slots[index];
|
||||
defer self.release(io, index);
|
||||
|
||||
switch (self.handler.handle(io, .udp, slot.query[0..slot.len], &slot.reply)) {
|
||||
const outcome = self.handler.handle(
|
||||
io,
|
||||
.udp,
|
||||
address.NetAddress.fromIp(slot.from),
|
||||
slot.query[0..slot.len],
|
||||
&slot.reply,
|
||||
&slot.scratch,
|
||||
);
|
||||
switch (outcome) {
|
||||
.drop => bump(&self.stats.dropped_handler),
|
||||
.reply => |bytes| self.socket.send(io, &slot.from, bytes) catch |err| {
|
||||
bump(&self.stats.send_errors);
|
||||
@@ -283,8 +296,8 @@ test "a slot's reply buffer holds a whole DNS message" {
|
||||
}
|
||||
|
||||
test "the default slot pool stays inside the memory budget" {
|
||||
// 4096 + 65535 ≈ 68 KiB per slot; 64 slots ≈ 4.3 MiB, against the 100 MB
|
||||
// of PLAN §18.
|
||||
// 4096 + 65535 + scratch ≈ 74 KiB per slot; 64 slots ≈ 4.6 MiB, against the
|
||||
// 100 MB of PLAN §18.
|
||||
const options: Options = .{};
|
||||
const pool_bytes = @sizeOf(UdpServer.Slot) * @as(usize, options.max_in_flight);
|
||||
try testing.expect(pool_bytes < 8 * 1024 * 1024);
|
||||
|
||||
@@ -14,6 +14,10 @@ const net = std.Io.net;
|
||||
|
||||
const handler = @import("handler.zig");
|
||||
const udp_server = @import("udp_server.zig");
|
||||
const model = @import("../config/model.zig");
|
||||
const response = @import("../filter/response.zig");
|
||||
const forward_zones = @import("../local/forward_zones.zig");
|
||||
const records = @import("../local/records.zig");
|
||||
const header = @import("../dns/header.zig");
|
||||
const packet = @import("../dns/packet.zig");
|
||||
const types = @import("../dns/types.zig");
|
||||
@@ -21,6 +25,31 @@ const transport = @import("../upstream/transport.zig");
|
||||
|
||||
const testing = std.testing;
|
||||
|
||||
const empty_records: records.Records = .empty;
|
||||
const empty_zones: forward_zones.Zones = .empty;
|
||||
const blocking_defaults: model.Blocking = .{};
|
||||
const blocking: response.Options = .{
|
||||
.mode = blocking_defaults.response,
|
||||
.ttl = blocking_defaults.ttl,
|
||||
};
|
||||
const forward_timeout: std.Io.Clock.Duration = .{
|
||||
.raw = model.readTimeout(.{}),
|
||||
.clock = .awake,
|
||||
};
|
||||
|
||||
/// An upstream and nothing else optional: no filtering, no cache, no log. The
|
||||
/// listener is what these tests exercise, so the handler is the same bare one
|
||||
/// its own tests use.
|
||||
fn bareHandler(client: transport.Client) handler.Handler {
|
||||
return .{
|
||||
.upstream = client,
|
||||
.blocking = blocking,
|
||||
.forward_read_timeout = forward_timeout,
|
||||
.records = &empty_records,
|
||||
.zones = &empty_zones,
|
||||
};
|
||||
}
|
||||
|
||||
/// Long enough that a loopback round trip cannot lose to scheduling, short
|
||||
/// enough that a broken server fails the run instead of hanging it.
|
||||
const budget: std.Io.Timeout = .{ .duration = .{ .raw = .fromSeconds(5), .clock = .awake } };
|
||||
@@ -87,7 +116,7 @@ test "a udp query is answered on the loopback" {
|
||||
const io = threaded.io();
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = response_bytes };
|
||||
var h: handler.Handler = .{ .upstream = fake.client() };
|
||||
var h = bareHandler(fake.client());
|
||||
|
||||
const listen_address: net.IpAddress = try .parse("127.0.0.1", 0);
|
||||
var server = try udp_server.UdpServer.bind(gpa, io, listen_address, &h, .{ .max_in_flight = 4 });
|
||||
@@ -125,7 +154,7 @@ test "a runt datagram is dropped and no reply is sent" {
|
||||
const io = threaded.io();
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = response_bytes };
|
||||
var h: handler.Handler = .{ .upstream = fake.client() };
|
||||
var h = bareHandler(fake.client());
|
||||
|
||||
const listen_address: net.IpAddress = try .parse("127.0.0.1", 0);
|
||||
var server = try udp_server.UdpServer.bind(gpa, io, listen_address, &h, .{ .max_in_flight = 4 });
|
||||
@@ -160,7 +189,7 @@ test "an oversize datagram arrives truncated and is dropped" {
|
||||
const io = threaded.io();
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = response_bytes };
|
||||
var h: handler.Handler = .{ .upstream = fake.client() };
|
||||
var h = bareHandler(fake.client());
|
||||
|
||||
const listen_address: net.IpAddress = try .parse("127.0.0.1", 0);
|
||||
var server = try udp_server.UdpServer.bind(gpa, io, listen_address, &h, .{ .max_in_flight = 4 });
|
||||
@@ -201,7 +230,7 @@ test "deinit ends a serve loop that is blocked on receive" {
|
||||
const io = threaded.io();
|
||||
|
||||
var fake: FakeUpstream = .{ .reply = response_bytes };
|
||||
var h: handler.Handler = .{ .upstream = fake.client() };
|
||||
var h = bareHandler(fake.client());
|
||||
|
||||
const listen_address: net.IpAddress = try .parse("127.0.0.1", 0);
|
||||
var server = try udp_server.UdpServer.bind(gpa, io, listen_address, &h, .{ .max_in_flight = 4 });
|
||||
|
||||
@@ -5,7 +5,8 @@
|
||||
//! not appear in an export. `countClients` counts **all** rows, because S5's
|
||||
//! "has this database ever been configured" predicate needs the true count.
|
||||
//!
|
||||
//! Only list / insert / deleteAll / count exist.
|
||||
//! Only list / insert / deleteAll / count, plus the two runtime calls
|
||||
//! `upsertSeen` and `pruneStale` that the Phase 7 client tracker owns.
|
||||
|
||||
const std = @import("std");
|
||||
const Allocator = std.mem.Allocator;
|
||||
@@ -81,6 +82,51 @@ pub fn insertClient(database: *db.Db, item: model.Client, ctx: InsertContext) db
|
||||
try stmt.exec();
|
||||
}
|
||||
|
||||
const upsert_seen_sql =
|
||||
\\INSERT INTO clients (ip, name, group_id, hand_edited, first_seen, last_seen)
|
||||
\\VALUES (?1, NULL, (SELECT id FROM groups WHERE name = 'default'), 0, ?2, ?2)
|
||||
\\ON CONFLICT(ip) DO UPDATE SET last_seen = excluded.last_seen
|
||||
;
|
||||
|
||||
/// Materialises a client seen in live traffic (PLAN §7.2), or touches
|
||||
/// `last_seen` on the row that already holds `ip`.
|
||||
///
|
||||
/// The conflict target is `clients.ip`, which the schema declares UNIQUE. Only
|
||||
/// `last_seen` is updated: `name`, `group_id` and `hand_edited` are the
|
||||
/// operator's, and a device that keeps querying must not overwrite them. A
|
||||
/// hand-edited row is touched too, so the operator sees liveness for the
|
||||
/// clients they named.
|
||||
///
|
||||
/// A new row lands in the `default` group. `groupForClient` resolves the real
|
||||
/// group from the prefix rules at query time, so the column here only decides
|
||||
/// what the operator sees before they assign the device themselves.
|
||||
///
|
||||
/// A database with no group named `default` fails the insert with
|
||||
/// `error.Constraint` rather than writing a dangling row. `validate.zig`
|
||||
/// rejects such a configuration long before a server serves from it.
|
||||
pub fn upsertSeen(database: *db.Db, ip: []const u8, now_s: i64) db.Error!void {
|
||||
var stmt = try database.prepare(upsert_seen_sql);
|
||||
defer stmt.deinit();
|
||||
try stmt.bindText(1, ip);
|
||||
try stmt.bindInt(2, now_s);
|
||||
try stmt.exec();
|
||||
}
|
||||
|
||||
/// Deletes the auto-materialised clients whose last query predates `cutoff_s`,
|
||||
/// and returns how many rows went. `hand_edited = 1` rows are configuration and
|
||||
/// survive any silence.
|
||||
///
|
||||
/// `last_seen` is the only signal, because §3.6 forbids joining `config.db`
|
||||
/// against the query log.
|
||||
pub fn pruneStale(database: *db.Db, cutoff_s: i64) db.Error!u32 {
|
||||
var stmt = try database.prepare("DELETE FROM clients WHERE hand_edited = 0 AND last_seen < ?1");
|
||||
defer stmt.deinit();
|
||||
try stmt.bindInt(1, cutoff_s);
|
||||
try stmt.exec();
|
||||
const deleted = database.changes();
|
||||
return @intCast(@min(deleted, std.math.maxInt(u32)));
|
||||
}
|
||||
|
||||
pub fn deleteAllClients(database: *db.Db) db.Error!void {
|
||||
return database.exec("DELETE FROM clients;");
|
||||
}
|
||||
@@ -281,6 +327,124 @@ test "listClients is leak-safe under allocation failure" {
|
||||
try testing.checkAllAllocationFailures(testing.allocator, listClientsUnderFailure, .{&ids});
|
||||
}
|
||||
|
||||
fn seenRow(database: *db.Db, ip: []const u8) !struct { hand_edited: i64, first_seen: i64, last_seen: i64, group_id: i64 } {
|
||||
var stmt = try database.prepare(
|
||||
"SELECT hand_edited, first_seen, last_seen, group_id FROM clients WHERE ip = ?1",
|
||||
);
|
||||
defer stmt.deinit();
|
||||
try stmt.bindText(1, ip);
|
||||
try testing.expect(try stmt.step());
|
||||
return .{
|
||||
.hand_edited = stmt.columnInt(0),
|
||||
.first_seen = stmt.columnInt(1),
|
||||
.last_seen = stmt.columnInt(2),
|
||||
.group_id = stmt.columnInt(3),
|
||||
};
|
||||
}
|
||||
|
||||
test "upsertSeen materialises an unseen client in the default group" {
|
||||
var database = try openMigrated();
|
||||
defer database.close();
|
||||
|
||||
try upsertSeen(&database, "192.168.1.50", 1700000000);
|
||||
|
||||
try testing.expectEqual(@as(i64, 1), try countClients(&database));
|
||||
const row = try seenRow(&database, "192.168.1.50");
|
||||
try testing.expectEqual(@as(i64, 0), row.hand_edited);
|
||||
try testing.expectEqual(@as(i64, 1700000000), row.first_seen);
|
||||
try testing.expectEqual(@as(i64, 1700000000), row.last_seen);
|
||||
try testing.expectEqual(@as(i64, 1), row.group_id);
|
||||
|
||||
// Materialised clients are runtime state, so an export must not see them.
|
||||
var items = try listClients(&database, testing.allocator);
|
||||
defer items.deinit(testing.allocator);
|
||||
defer freeClients(testing.allocator, items.items);
|
||||
try testing.expectEqual(@as(usize, 0), items.items.len);
|
||||
}
|
||||
|
||||
test "upsertSeen touches last_seen and leaves first_seen alone" {
|
||||
var database = try openMigrated();
|
||||
defer database.close();
|
||||
|
||||
try upsertSeen(&database, "192.168.1.50", 1700000000);
|
||||
try upsertSeen(&database, "192.168.1.50", 1700000600);
|
||||
|
||||
try testing.expectEqual(@as(i64, 1), try countClients(&database));
|
||||
const row = try seenRow(&database, "192.168.1.50");
|
||||
try testing.expectEqual(@as(i64, 1700000000), row.first_seen);
|
||||
try testing.expectEqual(@as(i64, 1700000600), row.last_seen);
|
||||
}
|
||||
|
||||
test "upsertSeen keeps a hand-edited row's name, group and flag" {
|
||||
var database = try openMigrated();
|
||||
defer database.close();
|
||||
var ids = try seedGroups(&database);
|
||||
defer ids.deinit(testing.allocator);
|
||||
try seedClients(&database, &ids);
|
||||
|
||||
try upsertSeen(&database, "192.168.1.20", 1700009999);
|
||||
|
||||
try testing.expectEqual(@as(i64, 3), try countClients(&database));
|
||||
const row = try seenRow(&database, "192.168.1.20");
|
||||
try testing.expectEqual(@as(i64, 1), row.hand_edited);
|
||||
try testing.expectEqual(@as(i64, 2), row.group_id);
|
||||
try testing.expectEqual(@as(i64, 1700000000), row.first_seen);
|
||||
try testing.expectEqual(@as(i64, 1700009999), row.last_seen);
|
||||
|
||||
var items = try listClients(&database, testing.allocator);
|
||||
defer items.deinit(testing.allocator);
|
||||
defer freeClients(testing.allocator, items.items);
|
||||
try testing.expectEqual(@as(usize, 3), items.items.len);
|
||||
try testing.expectEqualStrings("laptop", items.items[1].name);
|
||||
try testing.expectEqualStrings("kids", items.items[1].group);
|
||||
}
|
||||
|
||||
test "upsertSeen reports a database with no default group" {
|
||||
var database = try openMigrated();
|
||||
defer database.close();
|
||||
try database.exec("UPDATE groups SET name = 'renamed' WHERE id = 1;");
|
||||
|
||||
try testing.expectError(error.Constraint, upsertSeen(&database, "192.168.1.50", 1700000000));
|
||||
try testing.expectEqual(@as(i64, 0), try countClients(&database));
|
||||
}
|
||||
|
||||
test "pruneStale removes only stale auto-materialised rows" {
|
||||
var database = try openMigrated();
|
||||
defer database.close();
|
||||
var ids = try seedGroups(&database);
|
||||
defer ids.deinit(testing.allocator);
|
||||
// A hand-edited row far older than the cutoff.
|
||||
try seedClients(&database, &ids);
|
||||
|
||||
try upsertSeen(&database, "10.0.0.1", 1700000000);
|
||||
try upsertSeen(&database, "10.0.0.2", 1700000199);
|
||||
// Exactly at the cutoff: the comparison is strict, so it stays.
|
||||
try upsertSeen(&database, "10.0.0.3", 1700000200);
|
||||
try upsertSeen(&database, "10.0.0.4", 1700000300);
|
||||
|
||||
try testing.expectEqual(@as(u32, 2), try pruneStale(&database, 1700000200));
|
||||
try testing.expectEqual(@as(i64, 5), try countClients(&database));
|
||||
try testing.expectEqual(
|
||||
@as(i64, 0),
|
||||
try database.queryInt("SELECT count(*) FROM clients WHERE ip IN ('10.0.0.1', '10.0.0.2')"),
|
||||
);
|
||||
try testing.expectEqual(@as(i64, 3), try database.queryInt("SELECT count(*) FROM clients WHERE hand_edited = 1"));
|
||||
|
||||
// A second pass over the same cutoff finds nothing left to do.
|
||||
try testing.expectEqual(@as(u32, 0), try pruneStale(&database, 1700000200));
|
||||
}
|
||||
|
||||
test "pruneStale spares hand-edited rows however stale" {
|
||||
var database = try openMigrated();
|
||||
defer database.close();
|
||||
var ids = try seedGroups(&database);
|
||||
defer ids.deinit(testing.allocator);
|
||||
try seedClients(&database, &ids);
|
||||
|
||||
try testing.expectEqual(@as(u32, 0), try pruneStale(&database, 1800000000));
|
||||
try testing.expectEqual(@as(i64, 3), try countClients(&database));
|
||||
}
|
||||
|
||||
test "client_prefixes round-trip in prefix order with group names resolved" {
|
||||
var database = try openMigrated();
|
||||
defer database.close();
|
||||
|
||||
@@ -72,6 +72,10 @@ comptime {
|
||||
_ = @import("platform/logging.zig");
|
||||
_ = @import("storage/retention.zig");
|
||||
_ = @import("storage/phase6_integration_test.zig");
|
||||
_ = @import("server/pause.zig");
|
||||
_ = @import("server/clients.zig");
|
||||
_ = @import("server/shutdown.zig");
|
||||
_ = @import("server/phase7_integration_test.zig");
|
||||
}
|
||||
|
||||
extern fn sqlite3_libversion() [*:0]const u8;
|
||||
|
||||
Reference in New Issue
Block a user