specs: memory budget drops the removed performance page and keeps stack_size, a thp artifact explained the 2 mib stacks
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# Memory budget: pay for what the household uses
|
||||
|
||||
Measured on the deployed Pi (0.0.21, aarch64, 2.85M blocklist entries, 18 days up): VmRSS 238 MiB, VmHWM 358 MiB, 207 threads, almost all private anonymous memory. `docs/explanation/performance-and-testing.md` promises "memory with ~1M blocked domains < 100 MiB". Nothing leaks. Every MiB below has a measured owner, and each owner pays for a worst case the household never reaches, or pays twice.
|
||||
Measured on the deployed Pi (0.0.21, aarch64, 2.85M blocklist entries, 18 days up): VmRSS 238 MiB, VmHWM 358 MiB, 207 threads, almost all private anonymous memory. The docs promised "memory with ~1M blocked domains < 100 MiB" (`docs/explanation/performance-and-testing.md`, removed in the user-docs rewrite); the Pi never met it. Nothing leaks. Every MiB below has a measured owner, and each owner pays for a worst case the household never reaches, or pays twice.
|
||||
|
||||
| Owner | Resident | How it was measured |
|
||||
|---|---|---|
|
||||
@@ -29,17 +29,17 @@ Why freed small blocks stay resident: with libc linked, `malloc` is zig's `lib/c
|
||||
|
||||
### 0. Make the budget measurable first
|
||||
|
||||
The admission budget, the listener caps, the stack size and the queue size are chosen from the Pi's real peaks, so those peaks become measurements before anything is resized:
|
||||
The admission budget, the listener caps and the queue size are chosen from the Pi's real peaks, so those peaks become measurements before anything is resized:
|
||||
|
||||
- high-water of in-flight DNS queries: `nxdns_dns_in_flight_peak`, counted around `Handler.handle`, every listener together;
|
||||
- high-water of active stream connections, per listener (today `connections` counts accepts over time, not concurrency): `nxdns_tcp_server_connections_peak` (the larger of the two address families), `nxdns_dot_server_connections_peak`, `nxdns_doh_server_connections_peak`;
|
||||
- high-water of the `std.Io.Threaded` worker pool: the process thread count. `Threaded` spawns a worker only when every worker is busy and never retires one (`Threaded.zig:2110`), so the count is that high-water already and needs no code. It counts workers, not tasks: `async` work can run inline or wait in the run queue;
|
||||
- high-water of the query-log writer queue's occupancy (not arrivals per interval: a full batch flushes early, and a disk gate or a slow write lets backlog build past one interval): `nxdns_querylog_queue_peak`. `std.Io.Queue` exposes no length, so a count beside it is raised before each insert and lowered after each removal; it can overstate the peak by the entries in transit, never understate it;
|
||||
- worker call-stack use, read from `/proc/<pid>/smaps`: each worker stack is an anonymous mapping directly above its own guard page, all of one size, and its `Rss` is what the thread has touched and still holds resident. That finds the expensive paths; it is not a depth bound. Before item 3 lowers `stack_size`, each expensive path gets a measured stack high-water, and the size keeps headroom above the deepest one. The signal stack (`std_options.signal_stack_size`) is a separate allocation for signal handling and stack traces and keeps its stdlib default unless its own use is measured.
|
||||
- worker call-stack use, read from `/proc/<pid>/smaps`: each worker stack is an anonymous mapping directly above its own guard page, all of one size, and its `Rss` is what the thread has touched and still holds resident. It is a record, not an input: a worker stack costs only the pages it touches, so `stack_size` is address space and lowering it saves no resident memory. `stack_size` keeps its default. The signal stack (`std_options.signal_stack_size`) is a separate allocation for signal handling and stack traces and keeps its stdlib default unless its own use is measured.
|
||||
|
||||
The five gauges are in `/metrics` beside the existing per-reason drop counters. One release carrying only these measurements runs on the Pi before the budget values are fixed.
|
||||
|
||||
First reading, static x86_64 musl build, one burst of 300 UDP queries: the in-flight peak reached 64 (the UDP slot pool), threads went from 14 to 205, and of 201 worker stacks 175 touched 32 kB or less while 26 touched 2 MiB each — the same 2 MiB with only forward-zone traffic and no DoT exchange, so the TLS client is not the owner. Item 3 identifies that owner before it lowers `stack_size`: a stack smaller than that path needs is a crash, not a saving.
|
||||
First reading, static x86_64 musl build, one burst of 300 UDP queries: the in-flight peak reached 64 (the UDP slot pool), threads went from 14 to 205, and of 201 worker stacks 175 touched 32 kB or less while 26 touched 2 MiB each — the same 2 MiB with only forward-zone traffic and no DoT exchange, so the TLS client is not the owner. That 2 MiB was transparent huge pages, not a code path: the build host runs THP in `always` mode and backs a touched 2 MiB-aligned stack range with one huge page. The Pi runs `madvise`. Pi reading on 0.0.22 after 31 h (16 KiB pages): of 91 worker stacks, 6 hold 32 kB, 2 hold 80–96 kB, 73 hold 192 kB and 10 hold 208–256 kB, none with a huge page. The resident cost of the pool is therefore about 192 kB of stack per worker, which item 3 reduces by bounding the worker count.
|
||||
|
||||
### 1. Store each blocklist name once in memory
|
||||
|
||||
@@ -56,7 +56,7 @@ Within one source, as compiled entries, an exact `x` in `.list` together with `x
|
||||
|
||||
### 3. Bound concurrent DNS work
|
||||
|
||||
- `main` takes `std.process.Init.Minimal` and owns its `std.Io.Threaded`, with `stack_size` set from the item-0 stack measurement and a `concurrent_limit` derived from the chosen query and connection caps, the tasks each can nest, and the fixed services. The measured task high-water already counts every caller, so it is the cross-check, not a term in the sum.
|
||||
- `main` takes `std.process.Init.Minimal` and owns its `std.Io.Threaded`, with a `concurrent_limit` derived from the chosen query and connection caps, the tasks each can nest, and the fixed services. The measured task high-water already counts every caller, so it is the cross-check, not a term in the sum.
|
||||
- `ConcurrencyUnavailable` is counted at every boundary where it can surface. The spec work enumerates each `io.concurrent` and `Group.concurrent` call; today the logger turns the error into an early flush without counting it (`src/storage/logger.zig:748`) and the listener races fold it into a generic failure (`src/server/listener.zig:102`). Each gets its own counter or a reason on an existing one.
|
||||
- Two kinds of bound: one cap on active DNS queries, shared by every listener (UDP and TCP, both families, DoT/DoH when enabled), and a slot cap per stream listener on accepted connections, each from the item-0 high-waters. A query takes a permit only while it does work; a keep-alive connection releases its permit between requests.
|
||||
- UDP keeps today's counted drop when no permit is free (`dropped_no_slot`). The kernel buffer does not absorb overload invisibly.
|
||||
@@ -81,7 +81,7 @@ Within one source, as compiled entries, an exact `x` in `.list` together with `x
|
||||
|
||||
- Config: `cache.max_bytes`; the changed default of `logging.query_log_buffer_max`. Config field checklist for the new key: model key + round-trip drift guards, validation + reference, settings API view, openapi.yaml, admin types + settings control, contract samples regenerated, docs config reference, CHANGELOG.
|
||||
- Diagnostics: the item-0 counters, `ConcurrencyUnavailable`, oversized cache responses, cache live bytes and budget.
|
||||
- Docs: `performance-and-testing.md` states the measured RSS, refresh peak and thread count at the Pi's blocklist size, replacing the unmeasured target.
|
||||
- Docs: the user docs make no memory claim. The measured RSS, refresh peak and thread count at the Pi's blocklist size go into the measurement record below and the release's CHANGELOG entry.
|
||||
|
||||
## Acceptance
|
||||
|
||||
@@ -94,4 +94,4 @@ Within one source, as compiled entries, an exact `x` in `.list` together with `x
|
||||
- [ ] SQLite: each runtime handle reports its configured `cache_size`; migrations open without it.
|
||||
- [ ] DNS cache: puts beyond the byte budget evict; live bytes never exceed it; an oversized response is skipped and counted; a failed replacement keeps the old entry; a failed copy allocation is counted and the answer is still sent.
|
||||
- [ ] Suites: `zig build test` and `-Dintegration` 0 failed; admin typecheck/vitest/prettier clean; contract samples regenerated.
|
||||
- [ ] Measurement record, on the Pi with the same blocklists: steady VmRSS after 24 h, VmHWM across one scheduled refresh (including its concurrent download and compile), thread count, and a decoded heap dump per size class, each beside the baseline in the table above. The owner chose a record, not a pass threshold: each owner must be reduced as specified, and the numbers go into `performance-and-testing.md`. If small-class slabs remain a top owner, the allocator question reopens with that data.
|
||||
- [ ] Measurement record, on the Pi with the same blocklists: steady VmRSS after 24 h, VmHWM across one scheduled refresh (including its concurrent download and compile), thread count, and a decoded heap dump per size class, each beside the baseline in the table above. The owner chose a record, not a pass threshold: each owner must be reduced as specified, and the numbers go into this spec beside the baseline and into the release's CHANGELOG entry. If small-class slabs remain a top owner, the allocator question reopens with that data.
|
||||
|
||||
Reference in New Issue
Block a user