Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0385c34ca3
|
||
|
|
36e11ca71d
|
||
|
|
9d2511ee58
|
||
|
|
cab79c75e2
|
||
|
|
5acfec2bff
|
||
|
|
534c620695
|
@@ -4,6 +4,12 @@ All notable changes to nxdns are recorded here. The format follows [Keep a Chang
|
|||||||
|
|
||||||
Sections are written by hand. Nothing here is generated from commit messages: the point of the file is to say what changed for an operator, which a commit subject rarely does.
|
Sections are written by hand. Nothing here is generated from commit messages: the point of the file is to say what changed for an operator, which a commit subject rarely does.
|
||||||
|
|
||||||
|
## [0.0.22] - 2026-10-19
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **`/metrics` reports the most concurrent work the server has seen.** Five gauges hold peaks since start: `nxdns_dns_in_flight_peak` (queries handled at once, every listener together), `nxdns_tcp_server_connections_peak`, `nxdns_dot_server_connections_peak` and `nxdns_doh_server_connections_peak` (connections one listener held at once), and `nxdns_querylog_queue_peak` (entries waiting for the query log writer at once). The memory budget work sizes its limits from them; "Measure performance" shows how to read them beside the process's resident memory, thread count and worker stacks.
|
||||||
|
|
||||||
## [0.0.21] - 2026-09-12
|
## [0.0.21] - 2026-09-12
|
||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
|
|||||||
@@ -37,9 +37,10 @@ Serves a household LAN (≈2–20 devices). Portfolio-grade public repo with ext
|
|||||||
- Partial-segment wildcards (`ads*.example.com`) as a rule kind. The regex kind covers the need without a second globbing dialect.
|
- Partial-segment wildcards (`ads*.example.com`) as a rule kind. The regex kind covers the need without a second globbing dialect.
|
||||||
- DHCP server.
|
- DHCP server.
|
||||||
- DNSSEC validation (DO bit passthrough only).
|
- DNSSEC validation (DO bit passthrough only).
|
||||||
- DoQ (QUIC), HTTP/2 upstream transport.
|
- DoQ (DNS over QUIC). Out of scope until the Zig stdlib, or an equivalent dependency, provides QUIC; neither the stdlib nor mbedTLS does at 0.16.0. This is the one conditional entry in this list.
|
||||||
|
- HTTP/2 upstream transport.
|
||||||
|
- DNSCrypt. It is older than DoH and DoT and not an IETF standard, and DoH and DoT already cover encrypted DNS.
|
||||||
- Clustering / distributed state.
|
- Clustering / distributed state.
|
||||||
- Project website. The repository, its README and `docs/` are the whole published surface.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
@@ -28,13 +28,11 @@ One static Zig binary. SQLite holds the state and the query log, which the web U
|
|||||||
|
|
||||||
## Install
|
## Install
|
||||||
|
|
||||||
**No release exists yet.** This repository has no tags, nothing has been published to <https://git.mial.net/mokhtar/nxdns/releases>, and no container image has been pushed. Every release URL on this page and in the how-to guides is a 404 today, and `docker pull` finds nothing. Until the first tag ships, building from source is the only way to get nxdns.
|
Releases are published at <https://git.mial.net/mokhtar/nxdns/releases>, and the container image at `git.mial.net/mokhtar/nxdns`, which pulls without an account. Each tag publishes five assets — two static musl tarballs (`nxdns-<version>-x86_64-linux-musl.tar.gz`, `nxdns-<version>-aarch64-linux-musl.tar.gz`), `IMAGE-DIGEST.txt` naming the multi-architecture container image by digest, `SHA256SUMS.txt` covering those three files, and `SHA256SUMS.txt.asc`, a detached OpenPGP signature over the checksum file. Verify what you downloaded before you run it: [docs/how-to/verify-a-release.md](docs/how-to/verify-a-release.md), which also says what that signature does and does not prove.
|
||||||
|
|
||||||
What a tag will publish, once one exists: five assets — two static musl tarballs (`nxdns-<version>-x86_64-linux-musl.tar.gz`, `nxdns-<version>-aarch64-linux-musl.tar.gz`), `IMAGE-DIGEST.txt` naming the multi-architecture container image by digest, `SHA256SUMS.txt` covering those three files, and `SHA256SUMS.txt.asc`, a detached OpenPGP signature over the checksum file. Verify what you downloaded before you run it: [docs/how-to/verify-a-release.md](docs/how-to/verify-a-release.md), which also says what that signature does and does not prove.
|
|
||||||
|
|
||||||
## Quickstart (docker compose)
|
## Quickstart (docker compose)
|
||||||
|
|
||||||
Write a minimal configuration and start the published image. This is what the first release will make possible; it does not work today, because there is no image in the registry to pull:
|
Write a minimal configuration and start the published image:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
cd deploy/docker
|
cd deploy/docker
|
||||||
@@ -49,7 +47,7 @@ EOF
|
|||||||
NXDNS_VERSION=<version> docker compose up -d
|
NXDNS_VERSION=<version> docker compose up -d
|
||||||
```
|
```
|
||||||
|
|
||||||
The compose file defaults to `:latest`; pin a version for anything you intend to keep running. To run it before a release exists, build the image yourself and name it — `NXDNS_IMAGE=nxdns docker compose up -d` — as [docs/how-to/install-with-docker.md](docs/how-to/install-with-docker.md) describes. DNS is on port 53, the web UI on <http://localhost:8080>.
|
The compose file defaults to `:latest`; pin a version for anything you intend to keep running. To run an image you built yourself, name it — `NXDNS_IMAGE=nxdns docker compose up -d` — as [docs/how-to/install-with-docker.md](docs/how-to/install-with-docker.md) describes. DNS is on port 53, the web UI on <http://localhost:8080>.
|
||||||
|
|
||||||
The compose file runs `nxdns run --config=/etc/nxdns/config.zon`, which makes that file the configuration: every start reconciles the database onto it, and the UI refuses configuration edits. Edit the file and restart to change anything. Drop the `command:` line to run bare `nxdns run` instead, where the database is the configuration and changes go through the UI, the API, or `nxdns export` / `nxdns import` — the packaged systemd unit does that. Which mode is live is printed at every start (`authority: database` / `authority: file (<path>)`); see [docs/explanation/configuration-model.md](docs/explanation/configuration-model.md).
|
The compose file runs `nxdns run --config=/etc/nxdns/config.zon`, which makes that file the configuration: every start reconciles the database onto it, and the UI refuses configuration edits. Edit the file and restart to change anything. Drop the `command:` line to run bare `nxdns run` instead, where the database is the configuration and changes go through the UI, the API, or `nxdns export` / `nxdns import` — the packaged systemd unit does that. Which mode is live is printed at every start (`authority: database` / `authority: file (<path>)`); see [docs/explanation/configuration-model.md](docs/explanation/configuration-model.md).
|
||||||
|
|
||||||
|
|||||||
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
.{
|
.{
|
||||||
.name = .nxdns,
|
.name = .nxdns,
|
||||||
.version = "0.0.21",
|
.version = "0.0.22",
|
||||||
.minimum_zig_version = "0.16.0",
|
.minimum_zig_version = "0.16.0",
|
||||||
.paths = .{""},
|
.paths = .{""},
|
||||||
.fingerprint = 0x3307b311dded1d91,
|
.fingerprint = 0x3307b311dded1d91,
|
||||||
|
|||||||
@@ -79,8 +79,6 @@ Two honest gaps remain, stated so nobody has to rediscover them:
|
|||||||
|
|
||||||
The same distinction applies one level out, to the artifacts. A release is signed, and the signature is worth having: it says the artifact came from this project's pipeline and reached you unaltered. It does not say the binary was built from the source in this repository, because the machine that ran the build also held the signing key. An attacker with that machine produces something that verifies cleanly and contains whatever they put in it.
|
The same distinction applies one level out, to the artifacts. A release is signed, and the signature is worth having: it says the artifact came from this project's pipeline and reached you unaltered. It does not say the binary was built from the source in this repository, because the machine that ran the build also held the signing key. An attacker with that machine produces something that verifies cleanly and contains whatever they put in it.
|
||||||
|
|
||||||
The control that closes that gap is a reproducibility gate — an independent build, in a different directory on a different machine, landing on the same bytes. It does not exist. It is a recorded deferral (`specs/milestone-14.md` ruling 12), not something nobody thought of, and until it exists no document here describes the build as reproducible: nobody has measured whether it is. The cheap inputs to reproducibility are already in place — `gzip -n`, `--mtime=@0`, `LC_ALL=C`, `TZ=UTC`, exact Zig and Node pins — which makes the gate cheap to add later and proves nothing on its own.
|
The control that closes that gap is a reproducibility gate — an independent build, in a different directory on a different machine, landing on the same bytes. Every release from 0.0.17 on passes one (milestone 40, which closed the deferral in `specs/milestone-14.md` ruling 12). The cut builds the tarballs on the maintainer's machine and writes their hashes into `flake.nix`; the release job rebuilds them on the CI runner from the tagged tree, and `zig build verify-pins` refuses to let anything be uploaded unless the two builds landed on the same bytes. What makes that hold: the binary embeds no commit sha, the archive is written by the project's own tool with fixed modes, zero timestamps and sorted entries, and Zig, Node and npm are pinned to exact versions and run in a normalized environment.
|
||||||
|
|
||||||
What the release pipeline is required to hold to is narrower: two runs of `zig build dist` on the same commit **in the same directory** produce byte-identical tarballs. Same-directory determinism is a much weaker property than reproducibility, and conflating the two is exactly the kind of claim this page exists to refuse.
|
What the gate does not show: both builds run on machines the maintainer controls, so it proves the pipeline is deterministic across two hosts, not that someone independent of the project has rebuilt it. That is what the rebuild recipe in [Verify a release](../how-to/verify-a-release.md) is for, and anyone can run it. It reproduces the release only on x86_64 Linux with glibc, because the admin bundle is built with host-native Rolldown and Lightning CSS bindings.
|
||||||
|
|
||||||
[Verify a release](../how-to/verify-a-release.md) states the same limits where an operator will actually meet them, and gives the rebuild-and-compare recipe with the caveat that a differing hash is not evidence of tampering while this gap is open.
|
|
||||||
|
|||||||
@@ -6,18 +6,9 @@ The image is multi-architecture — `linux/amd64` and `linux/arm64` — so the s
|
|||||||
|
|
||||||
For what each configuration field means, see [the configuration reference](../reference/configuration.md).
|
For what each configuration field means, see [the configuration reference](../reference/configuration.md).
|
||||||
|
|
||||||
> Verification: the failure modes, the run and the two checks in step 3 were run
|
> Verification: on 2026-10-02, steps 1 to 3 were run against the published `v0.0.21` release with an image pulled anonymously from the registry, on the host ports printed below; the notes in those steps carry the detail. The failure modes in step 2 were run on the machine that wrote an earlier revision of this page, against an image built from this checkout. The `chown` to uid 65532 needs root and was not run.
|
||||||
> on the machine that wrote an earlier revision of this page, against an image
|
|
||||||
> built from this checkout rather than pulled from the registry — no release is
|
|
||||||
> published yet, so nothing on this page could be run against a pulled image,
|
|
||||||
> and step 1 could not be run at all. One command was run in altered form: host
|
|
||||||
> port 8080 was occupied there, so the run and the verification commands in
|
|
||||||
> step 3 were executed with the host side of the port mappings moved to 25353
|
|
||||||
> and 28088 rather than the 53 and 8080 printed below. The container side was
|
|
||||||
> unchanged. See the note in step 3. The `chown` to uid 65532 needs root and was
|
|
||||||
> not run.
|
|
||||||
>
|
>
|
||||||
> **Not re-run for the file-mode revision.** The compose file now ships
|
> **Earlier, for the file-mode revision.** The compose file now ships
|
||||||
> `command: ["run", "--config=/etc/nxdns/config.zon"]`, and no container was
|
> `command: ["run", "--config=/etc/nxdns/config.zon"]`, and no container was
|
||||||
> started against that command on this host: staging a release image needs
|
> started against that command on this host: staging a release image needs
|
||||||
> `zig build dist`, which refuses to run while `admin/dist` is stale, and the web
|
> `zig build dist`, which refuses to run while `admin/dist` is stale, and the web
|
||||||
@@ -56,12 +47,7 @@ cut -d@ -f2 IMAGE-DIGEST.txt
|
|||||||
|
|
||||||
The last two have to print the same string — `IMAGE-DIGEST.txt` holds a whole pinned reference, `name:tag@sha256:…`, so the `cut` is what reduces it to the digest `imagetools` prints. [Verify a release](verify-a-release.md) covers the key, the fingerprint, every failure message, and what the signature does and does not prove.
|
The last two have to print the same string — `IMAGE-DIGEST.txt` holds a whole pinned reference, `name:tag@sha256:…`, so the `cut` is what reduces it to the digest `imagetools` prints. [Verify a release](verify-a-release.md) covers the key, the fingerprint, every failure message, and what the signature does and does not prove.
|
||||||
|
|
||||||
> Not verified on this host: no image and no release are published yet, so
|
> Verified on 2026-10-02 against `v0.0.21`, with an empty `DOCKER_CONFIG`, so no registry credentials: the lookup printed `0.0.21`, `docker pull` succeeded without a login, `gpg --verify` printed `Good signature` with the primary fingerprint above, `sha256sum -c` printed `OK` for all three files, and `imagetools inspect` and the `cut` both printed `sha256:ce1fef32c7ee6c23656abd94e53e4d4d6b1d34fe88a2c842b82c7202d8b904c3`.
|
||||||
> `docker pull` and every URL here fail today, and the `releases/latest` lookup
|
|
||||||
> returns 404 and leaves `VERSION` empty. That lookup was run against
|
|
||||||
> `gitea.com/gitea/tea` on Gitea `1.27.0+dev` and printed `0.15.1`. The
|
|
||||||
> `docker buildx imagetools inspect --format` shape was run here against
|
|
||||||
> `alpine:3.22` on Docker Hub and printed that image's index digest.
|
|
||||||
|
|
||||||
## 2. Get the compose file and write the configuration
|
## 2. Get the compose file and write the configuration
|
||||||
|
|
||||||
@@ -72,9 +58,7 @@ mkdir -p ~/nxdns && cd ~/nxdns
|
|||||||
curl -fLO "$BASE/raw/tag/v$VERSION/deploy/docker/compose.yaml"
|
curl -fLO "$BASE/raw/tag/v$VERSION/deploy/docker/compose.yaml"
|
||||||
```
|
```
|
||||||
|
|
||||||
> Not verified against nxdns: there is no tag yet. The Gitea raw-file URL shape
|
> Verified on 2026-10-02: the `curl` returned `compose.yaml` from the `v0.0.21` tag with a 200.
|
||||||
> `<repo>/raw/tag/<tag>/<path>` was run here against `gitea.com/gitea/tea` on
|
|
||||||
> Gitea `1.27.0+dev` and returned the file with a 200.
|
|
||||||
|
|
||||||
Compose bind-mounts `deploy/docker/etc-nxdns` read-only at `/etc/nxdns`. Create it and put the configuration in it:
|
Compose bind-mounts `deploy/docker/etc-nxdns` read-only at `/etc/nxdns`. Create it and put the configuration in it:
|
||||||
|
|
||||||
@@ -166,7 +150,7 @@ A healthy first start logs the reconcile, the authority and the bound sockets:
|
|||||||
|
|
||||||
```
|
```
|
||||||
info(migrations): config.db migrated from schema version 0 to 1
|
info(migrations): config.db migrated from schema version 0 to 1
|
||||||
reconciled '/etc/nxdns/config.zon': upstreams +1 ~0 -0; settings +45 ~0 -0;
|
reconciled '/etc/nxdns/config.zon': upstreams +1 ~0 -0; settings +46 ~0 -0;
|
||||||
settings keys changed: dns.bind_ipv4 dns.bind_ipv6 dns.port web.bind web.port …
|
settings keys changed: dns.bind_ipv4 dns.bind_ipv6 dns.port web.bind web.port …
|
||||||
web authentication is now enabled
|
web authentication is now enabled
|
||||||
info(nxdns): authority: file (/etc/nxdns/config.zon)
|
info(nxdns): authority: file (/etc/nxdns/config.zon)
|
||||||
@@ -183,15 +167,7 @@ dig @127.0.0.1 example.com A +short
|
|||||||
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8080/
|
curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8080/
|
||||||
```
|
```
|
||||||
|
|
||||||
> On the machine that wrote this page, host port 8080 was already taken by an
|
> Verified on 2026-10-02 with the pulled `v0.0.21` image, the configuration above and the unchanged `compose.yaml`, on host ports 53 and 8080: the first start logged the lines above, `dig` returned the A records for `example.com`, `curl` returned 200, and `/api/health` reported `"status":"ok"` with one of one upstreams available. `docker compose up -d` fails with `failed to bind host port 0.0.0.0:8080/tcp: address already in use` when a host port is occupied; free the port or edit the `ports:` list.
|
||||||
> unrelated process, so the container was verified with the host side of the
|
|
||||||
> port mappings moved to 25353 and 28088 — everything inside the container was
|
|
||||||
> unchanged, and the log still reads `serving on udp [::]:53`. Against those
|
|
||||||
> ports `dig` returned the A records for `example.com` and `curl` returned 200.
|
|
||||||
> `docker compose up -d` fails with
|
|
||||||
> `failed to bind host port 0.0.0.0:8080/tcp: address already in use` when a
|
|
||||||
> host port is occupied; free the port or edit the `ports:` list. The image
|
|
||||||
> under test was built locally, not pulled: there is nothing published to pull.
|
|
||||||
|
|
||||||
The compose file publishes 53/udp, 53/tcp and 8080, keeps `/var/lib/nxdns` in the named volume `nxdns-data`, and sets the per-namespace sysctl `net.ipv4.ip_unprivileged_port_start=0` so uid 65532 can bind port 53 without any capability. Uncomment the 443 and 853 mappings when you enable the DoH or DoT listener; see [Enable DoH and DoT](enable-doh-and-dot.md).
|
The compose file publishes 53/udp, 53/tcp and 8080, keeps `/var/lib/nxdns` in the named volume `nxdns-data`, and sets the per-namespace sysctl `net.ipv4.ip_unprivileged_port_start=0` so uid 65532 can bind port 53 without any capability. Uncomment the 443 and 853 mappings when you enable the DoH or DoT listener; see [Enable DoH and DoT](enable-doh-and-dot.md).
|
||||||
|
|
||||||
@@ -209,9 +185,7 @@ To pull the arm64 image from an x86_64 machine — to inspect it, or to save and
|
|||||||
docker pull --platform linux/arm64 git.mial.net/mokhtar/nxdns:$VERSION
|
docker pull --platform linux/arm64 git.mial.net/mokhtar/nxdns:$VERSION
|
||||||
```
|
```
|
||||||
|
|
||||||
> Not verified on this host: nothing is published to pull, and this host is
|
> Verified on 2026-10-02 against `v0.0.21`: `imagetools inspect` lists `linux/amd64` and `linux/arm64`, and the pull above fetched an image whose `docker image inspect` reports `linux/arm64`. It was not started: this host is x86_64 with no emulation.
|
||||||
> x86_64 with no emulation, so an arm64 image could not be started here even
|
|
||||||
> if it were.
|
|
||||||
|
|
||||||
## Build the image from source instead
|
## Build the image from source instead
|
||||||
|
|
||||||
|
|||||||
@@ -14,12 +14,7 @@ For what each flag does, see [the CLI reference](../reference/cli.md); for what
|
|||||||
> target host — `install`, `systemd-sysusers`, `systemctl` — were not run; they
|
> target host — `install`, `systemd-sysusers`, `systemctl` — were not run; they
|
||||||
> are marked where they appear.
|
> are marked where they appear.
|
||||||
>
|
>
|
||||||
> The download in step 1 could not be run at all: this repository has no tags
|
> The download in step 1 was run on 2026-10-02 against the published `v0.0.21` release; its note carries the detail.
|
||||||
> and no published release, so every release URL on this page is a 404 today.
|
|
||||||
> Its commands are the ones [Verify a release](verify-a-release.md) covers in
|
|
||||||
> full, and the URL shapes — including the `releases/latest` redirect the
|
|
||||||
> version is read from — were probed there against `gitea.com`, a public
|
|
||||||
> instance running the same Gitea series.
|
|
||||||
>
|
>
|
||||||
> The `zig build dist` and `zig build verify-dist` blocks in the last section
|
> The `zig build dist` and `zig build verify-dist` blocks in the last section
|
||||||
> were run here, both to completion and both exiting 0; that section carries
|
> were run here, both to completion and both exiting 0; that section carries
|
||||||
@@ -62,10 +57,7 @@ The extracted directory `nxdns-$VERSION-x86_64-linux-musl/` holds everything thi
|
|||||||
| `THIRD-PARTY-NOTICES` | Licences of everything compiled or bundled in |
|
| `THIRD-PARTY-NOTICES` | Licences of everything compiled or bundled in |
|
||||||
| `INSTALL.md` | A short version of this page |
|
| `INSTALL.md` | A short version of this page |
|
||||||
|
|
||||||
> Not verified on this host: no release exists yet, so none of these commands
|
> Verified on 2026-10-02 against `v0.0.21`, from a clean directory with a clean `GNUPGHOME` holding only the key from keys.openpgp.org: the lookup printed `0.0.21`, `gpg --verify` printed `Good signature` over primary fingerprint `A206 1F6A B24D F2C0 E923 46FD 1509 B549 46D0 8A95`, `sha256sum -c` printed `OK` for both tarballs and `IMAGE-DIGEST.txt`, and the extracted directory held the six files in the table below at the modes given, with `./nxdns version` printing `nxdns 0.0.21`.
|
||||||
> could be run against one — the `releases/latest` lookup returns 404 for this
|
|
||||||
> repository and leaves `VERSION` empty. The same lookup was run against
|
|
||||||
> `gitea.com/gitea/tea` on Gitea `1.27.0+dev` and printed `0.15.1`.
|
|
||||||
|
|
||||||
## 2. Copy the files to the target
|
## 2. Copy the files to the target
|
||||||
|
|
||||||
@@ -364,9 +356,7 @@ scp nxdns nxdns.service nxdns.conf pi:/tmp/
|
|||||||
|
|
||||||
Then follow steps 3 to 6 on the Pi.
|
Then follow steps 3 to 6 on the Pi.
|
||||||
|
|
||||||
> Not verified on this host: no release exists to download, `pi` is a
|
> Partly verified on 2026-10-02: the `v0.0.21` aarch64 tarball downloaded and `sha256sum -c` printed `OK` for it. The `scp` and steps 3 to 6 were not run: `pi` is a placeholder for your Pi's hostname, and this page was written on an x86_64 machine with no Pi attached.
|
||||||
> placeholder for your Pi's hostname, and this page was written on an x86_64
|
|
||||||
> machine with no Pi attached.
|
|
||||||
|
|
||||||
## Build from source instead
|
## Build from source instead
|
||||||
|
|
||||||
|
|||||||
@@ -114,6 +114,26 @@ Read the client's own rate and the server's `/metrics` together: a load generato
|
|||||||
|
|
||||||
**Not verified on this host:** `dnsperf` is not installed here and the target platform is a Raspberry Pi 5, not this development machine. The command above is the shape of the measurement, not a transcript.
|
**Not verified on this host:** `dnsperf` is not installed here and the target platform is a Raspberry Pi 5, not this development machine. The command above is the shape of the measurement, not a transcript.
|
||||||
|
|
||||||
|
## Measure memory on a running server
|
||||||
|
|
||||||
|
The bench measures one structure at a time. A running server's memory also depends on the most concurrent work it has seen since it started, so read it on the server itself, after it has served a normal day:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
pid=$(pidof nxdns)
|
||||||
|
grep -E 'VmRSS|VmHWM|Threads' /proc/$pid/status
|
||||||
|
curl -s http://127.0.0.1:8080/metrics | grep '^nxdns_.*_peak'
|
||||||
|
```
|
||||||
|
|
||||||
|
- `VmRSS` is what the process holds resident now. `VmHWM` is the most it has ever held, which is normally the last blocklist refresh.
|
||||||
|
- `Threads` is a high-water too: the I/O runtime adds a worker thread only when every existing worker is busy, and never retires one. The count is the largest the worker pool has been since start, plus the main thread.
|
||||||
|
- `nxdns_dns_in_flight_peak` is the most queries handled at once, every listener together.
|
||||||
|
- `nxdns_tcp_server_connections_peak`, `nxdns_dot_server_connections_peak` and `nxdns_doh_server_connections_peak` are the most connections one listener has held at once. TCP/53 has one listener per address family; the gauge is the larger of the two.
|
||||||
|
- `nxdns_querylog_queue_peak` is the most entries that have waited for the query log writer at once. It can overstate the true peak by the entries in transit at that instant, never understate it.
|
||||||
|
|
||||||
|
Each worker thread's stack is its own anonymous mapping in `/proc/$pid/smaps`, directly above a one-page `---p` guard, and every worker's mapping has the same size. Its `Rss` is the stack that thread has actually touched.
|
||||||
|
|
||||||
|
**Verified on this host** against a static x86_64 musl build under a burst of 300 UDP queries: the gauges and the stack mappings behave as described. Not yet run on the Pi.
|
||||||
|
|
||||||
## Where to run it
|
## Where to run it
|
||||||
|
|
||||||
The target platform is a Raspberry Pi 5. Numbers from a development x86_64 box do not transfer — the Pi's Cortex-A76 is far slower — so a passing run here is evidence the harness works and a baseline for spotting regressions on the machine development happens on, and nothing more. Run `--assert` on the Pi, where the numbers mean something.
|
The target platform is a Raspberry Pi 5. Numbers from a development x86_64 box do not transfer — the Pi's Cortex-A76 is far slower — so a passing run here is evidence the harness works and a baseline for spotting regressions on the machine development happens on, and nothing more. Run `--assert` on the Pi, where the numbers mean something.
|
||||||
|
|||||||
@@ -13,10 +13,7 @@ The normal path is to download the new release, verify it, and swap the binary.
|
|||||||
> where the substitution matters, or which was not run at all, carries its own
|
> where the substitution matters, or which was not run at all, carries its own
|
||||||
> note. Nothing here was verified except where a note says so.
|
> note. Nothing here was verified except where a note says so.
|
||||||
>
|
>
|
||||||
> Step 2 could not be run at all: no nxdns release is published yet, so every
|
> The download in step 2 was run on 2026-10-02 against the published `v0.0.21` release; its note carries the detail.
|
||||||
> release URL and the `docker compose pull` on this page fail today. The URL
|
|
||||||
> shapes and the verification commands are covered by
|
|
||||||
> [Verify a release](verify-a-release.md), which says what was probed and how.
|
|
||||||
|
|
||||||
## Breaking change: `run --config` now means file authority
|
## Breaking change: `run --config` now means file authority
|
||||||
|
|
||||||
@@ -137,10 +134,7 @@ Take `aarch64-linux-musl` for a Raspberry Pi 5. Verify every time, not only on t
|
|||||||
|
|
||||||
Under Docker there is nothing to download: step 3 pulls the image, and the `IMAGE-DIGEST.txt` asset is what you verify instead.
|
Under Docker there is nothing to download: step 3 pulls the image, and the `IMAGE-DIGEST.txt` asset is what you verify instead.
|
||||||
|
|
||||||
> Not verified on this host: no release exists yet, so the `releases/latest`
|
> Verified on 2026-10-02 against `v0.0.21`: the lookup printed `0.0.21`, every `curl` succeeded, `gpg --verify` printed `Good signature`, `sha256sum -c` printed `OK`, and the tarball extracted. [Install with systemd](install-with-systemd.md) has the detail.
|
||||||
> lookup returns 404 and leaves `VERSION` empty, and every `curl` below it is a
|
|
||||||
> 404 too. The lookup form was run against `gitea.com/gitea/tea` on Gitea
|
|
||||||
> `1.27.0+dev` and printed `0.15.1`.
|
|
||||||
|
|
||||||
## 3. Replace the binary
|
## 3. Replace the binary
|
||||||
|
|
||||||
|
|||||||
@@ -309,7 +309,7 @@ Compare your two tarball hashes against two things: the published `SHA256SUMS.tx
|
|||||||
|
|
||||||
A hash that differs is a signal to check the toolchain and the environment first. A distribution's Zig package instead of the official tarball is the first thing to rule out; an unpinned Node or npm version is the next ordinary explanation, and a build run outside the normalized environment above is the next one. Rule both out before you conclude anything about the release itself.
|
A hash that differs is a signal to check the toolchain and the environment first. A distribution's Zig package instead of the official tarball is the first thing to rule out; an unpinned Node or npm version is the next ordinary explanation, and a build run outside the normalized environment above is the next one. Rule both out before you conclude anything about the release itself.
|
||||||
|
|
||||||
> Verified against `v0.0.1`, before the build was reproducible: the recipe ran from a fresh clone, `git verify-tag v0.0.1` printed `Good signature` under the release subkey, and the rebuilt tarball hashes did not match the published `SHA256SUMS.txt` (Node 24.14.1 against the pinned 24.19.0, a different build path, and an archive written by the host's `tar`). Releases from 0.0.17 on are built and checked by the pinned pipeline this page describes, and the cut records the local hashes in `flake.nix` before CI rebuilds them.
|
> Verified on 2026-10-19 against `v0.0.21`, on x86_64 Linux with glibc, Node 24.19.0 and npm 11.17.0: from a fresh clone, `git verify-tag v0.0.21` printed `Good signature`, the freshly downloaded Zig tarball passed its `sha256sum -c`, and both rebuilt tarball hashes matched the published `SHA256SUMS.txt` and the `flake.nix` pins at the tag (`26615c4b…b3b4ce` for x86_64, `29746c9a…7f19af` for aarch64). The two `sh -c` wrappers were replaced by the same nine-variable `env -i` and a `022` umask set in the calling shell. The host was the maintainer's own machine, in a fresh directory with a private `--cache-dir`, so this run shows the recipe works; it is not a third-party reproduction. An earlier run against `v0.0.1`, before the build was reproducible, did not match.
|
||||||
|
|
||||||
## If a check fails
|
## If a check fails
|
||||||
|
|
||||||
|
|||||||
@@ -8,10 +8,10 @@
|
|||||||
let
|
let
|
||||||
# `zig build cut` rewrites the lines between the delimiters by text replacement.
|
# `zig build cut` rewrites the lines between the delimiters by text replacement.
|
||||||
# BEGIN GENERATED BY zig build cut
|
# BEGIN GENERATED BY zig build cut
|
||||||
version = "0.0.21";
|
version = "0.0.22";
|
||||||
hashes = {
|
hashes = {
|
||||||
"aarch64-linux" = "sha256-KXRsmmi3wHwfib4ha2ulmnUPfWd2d4FIKxAFWFZ/Ga8=";
|
"aarch64-linux" = "sha256-XzhqMazdf4YZyTSb6KSRuJZxLZUGsq/7Z9ckez7ZxcA=";
|
||||||
"x86_64-linux" = "sha256-JmFcS1LLweYEaLxAhJvgJbQhTKpaK33N4Khtp8qztM4=";
|
"x86_64-linux" = "sha256-i5rvzs7Q5vs2HkpnC6H7el1FJ7APHwIS7wbW/5DCC80=";
|
||||||
};
|
};
|
||||||
# END GENERATED BY zig build cut
|
# END GENERATED BY zig build cut
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,97 @@
|
|||||||
|
# Memory budget: pay for what the household uses
|
||||||
|
|
||||||
|
Measured on the deployed Pi (0.0.21, aarch64, 2.85M blocklist entries, 18 days up): VmRSS 238 MiB, VmHWM 358 MiB, 207 threads, almost all private anonymous memory. `docs/explanation/performance-and-testing.md` promises "memory with ~1M blocked domains < 100 MiB". Nothing leaks. Every MiB below has a measured owner, and each owner pays for a worst case the household never reaches, or pays twice.
|
||||||
|
|
||||||
|
| Owner | Resident | How it was measured |
|
||||||
|
|---|---|---|
|
||||||
|
| Blocklist snapshot | 71 MiB, of which ~32 MiB is ABP domains stored twice | Pi smaps; `.list` and `.wild` bodies of the ABP sources are byte-identical (md5) |
|
||||||
|
| Fixed buffers | ~52 MiB: query-log writer queue 17.5, two TCP listeners 2 × 8.7, two UDP listeners 2 × 4.8, SSE hub 3.6, cache slots 3.3 | Sizes from source; resident at once because safe builds fill new allocations with 0xaa |
|
||||||
|
| Idle worker threads | ~41 MiB for ~200 parked workers | `std.Io.Threaded` never retires a worker; one burst of 64 in-flight UDP queries × 3 tasks each grows the pool for good (replica: 14 → 206 threads) |
|
||||||
|
| Heap slabs kept after peaks | ~61 MiB free + ~8 MiB live | Decoded `SmpAllocator` free lists from two `/proc/<pid>/mem` dumps; owners from each slot's size header and a local tracing build |
|
||||||
|
| Peak during refresh | +120 MiB over steady state | Old snapshot + new snapshot + every compiled body read at once; replica reproduced 350 MiB |
|
||||||
|
|
||||||
|
Heap owners, from the dumps and the trace:
|
||||||
|
|
||||||
|
- 8 KiB class, 25 MiB free: SQLite page-cache blocks (4,376 B requests in 8 KiB slots). Seven connections, no `cache_size` pragma, so each keeps the 2 MiB default, which costs ~3.7 MiB of slabs once filled.
|
||||||
|
- 128 B + 256 B classes, ~21 MiB free: `std.Io.Threaded` `Group.Task` blocks — one per UDP query (`UdpServer.receiveLoop`), two per upstream exchange (`transport.raceUntilTagged`), two per query-log wait (`Logger.getWithin`).
|
||||||
|
- 512 B class, 7.7 MiB free: `DnsCache.put` copies of cached responses (`src/cache/dns_cache.zig:356`).
|
||||||
|
|
||||||
|
Why freed small blocks stay resident: with libc linked, `malloc` is zig's `lib/c/malloc.zig` on `std.heap.SmpAllocator`, which never unmaps a 64 KiB slab. `free` pushes onto the freeing thread's slot and `alloc` searches one other slot before it maps a new slab (`max_alloc_search = 1`), so producer/consumer pairs map more slabs than their live peak needs. Upstream: ziglang/zig#23592. Fewer tasks and smaller peaks shrink this retained memory; nothing in this spec frees it after the fact.
|
||||||
|
|
||||||
|
## Decisions
|
||||||
|
|
||||||
|
- Fix each owner at its root. Do not bypass the safe-build 0xaa fill: it only makes the oversizing visible.
|
||||||
|
- Do not change the allocator in this work. Replacing `init.gpa` does not move SQLite, which calls C `malloc`. Revisit only if the measurement gate still names small-class slabs as a top owner.
|
||||||
|
- No compiled-format change, no config-key migration, no new deadline mechanism. Each of those was considered and rejected in review: the format change makes the upgrade depend on the network, packed query-log rings cost more complexity than sizing the queue, and a socket-shutdown deadline cannot bound `connect` (`std.Io.Threaded` has no connect timeout) or the pool admission wait.
|
||||||
|
- Not `SQLITE_CONFIG_PAGECACHE`, not a database page-size migration.
|
||||||
|
|
||||||
|
## Contract
|
||||||
|
|
||||||
|
### 0. Make the budget measurable first
|
||||||
|
|
||||||
|
The admission budget, the listener caps, the stack size and the queue size are chosen from the Pi's real peaks, so those peaks become measurements before anything is resized:
|
||||||
|
|
||||||
|
- high-water of in-flight DNS queries: `nxdns_dns_in_flight_peak`, counted around `Handler.handle`, every listener together;
|
||||||
|
- high-water of active stream connections, per listener (today `connections` counts accepts over time, not concurrency): `nxdns_tcp_server_connections_peak` (the larger of the two address families), `nxdns_dot_server_connections_peak`, `nxdns_doh_server_connections_peak`;
|
||||||
|
- high-water of the `std.Io.Threaded` worker pool: the process thread count. `Threaded` spawns a worker only when every worker is busy and never retires one (`Threaded.zig:2110`), so the count is that high-water already and needs no code. It counts workers, not tasks: `async` work can run inline or wait in the run queue;
|
||||||
|
- high-water of the query-log writer queue's occupancy (not arrivals per interval: a full batch flushes early, and a disk gate or a slow write lets backlog build past one interval): `nxdns_querylog_queue_peak`. `std.Io.Queue` exposes no length, so a count beside it is raised before each insert and lowered after each removal; it can overstate the peak by the entries in transit, never understate it;
|
||||||
|
- worker call-stack use, read from `/proc/<pid>/smaps`: each worker stack is an anonymous mapping directly above its own guard page, all of one size, and its `Rss` is what the thread has touched and still holds resident. That finds the expensive paths; it is not a depth bound. Before item 3 lowers `stack_size`, each expensive path gets a measured stack high-water, and the size keeps headroom above the deepest one. The signal stack (`std_options.signal_stack_size`) is a separate allocation for signal handling and stack traces and keeps its stdlib default unless its own use is measured.
|
||||||
|
|
||||||
|
The five gauges are in `/metrics` beside the existing per-reason drop counters. One release carrying only these measurements runs on the Pi before the budget values are fixed.
|
||||||
|
|
||||||
|
First reading, static x86_64 musl build, one burst of 300 UDP queries: the in-flight peak reached 64 (the UDP slot pool), threads went from 14 to 205, and of 201 worker stacks 175 touched 32 kB or less while 26 touched 2 MiB each — the same 2 MiB with only forward-zone traffic and no DoT exchange, so the TLS client is not the owner. Item 3 identifies that owner before it lowers `stack_size`: a stack smaller than that path needs is a crash, not a saving.
|
||||||
|
|
||||||
|
### 1. Store each blocklist name once in memory
|
||||||
|
|
||||||
|
Within one source, as compiled entries, an exact `x` in `.list` together with `x` in `.wild` means exactly what `||x^` compiles to. The compiled files, their checksum and their counts stay as they are.
|
||||||
|
|
||||||
|
- The snapshot builder merges one source's sorted `.list` and `.wild` bodies into one name set with two flags per name: `exact` and `subdomains`. A name in both bodies is stored once. Sources are never merged with each other, so group eligibility and the reported source do not change.
|
||||||
|
- `.allow` stays a separate set, and level 7 (exceptions) still runs before both flag checks.
|
||||||
|
- Level 8 tests the full name for `exact`; level 9 tests proper parents for `subdomains`. Sources still walk in ascending index order. The reported result stays what it is today: when the `x` entry wins, the apex reports `blocklist_domain` with matched `x` and a proper subdomain reports `blocklist_wildcard` with matched parent `x`. An exception, an exact entry for the queried name or a nearer matching parent still wins first.
|
||||||
|
|
||||||
|
### 2. Lower the refresh and boot peak
|
||||||
|
|
||||||
|
- Boot keeps one synchronous initial load before the listeners start (`src/app.zig:692`). The startup pass uses that result for its refresh decisions and reloads only after a source actually refreshed, instead of rebuilding unconditionally (`src/filter/manager.zig:1823`).
|
||||||
|
- `Snapshot` gets a pure `Builder`: add one source's bodies, then free them before reading the next. The old snapshot is still freed only after the swap, so a reload holds at most old snapshot + new snapshot + one source's bodies. That is the reload bound, not the process peak: refresh preparation (download, compile) can run beside a reload and holds its own allocations, and the Pi peak measurement includes it.
|
||||||
|
|
||||||
|
### 3. Bound concurrent DNS work
|
||||||
|
|
||||||
|
- `main` takes `std.process.Init.Minimal` and owns its `std.Io.Threaded`, with `stack_size` set from the item-0 stack measurement and a `concurrent_limit` derived from the chosen query and connection caps, the tasks each can nest, and the fixed services. The measured task high-water already counts every caller, so it is the cross-check, not a term in the sum.
|
||||||
|
- `ConcurrencyUnavailable` is counted at every boundary where it can surface. The spec work enumerates each `io.concurrent` and `Group.concurrent` call; today the logger turns the error into an early flush without counting it (`src/storage/logger.zig:748`) and the listener races fold it into a generic failure (`src/server/listener.zig:102`). Each gets its own counter or a reason on an existing one.
|
||||||
|
- Two kinds of bound: one cap on active DNS queries, shared by every listener (UDP and TCP, both families, DoT/DoH when enabled), and a slot cap per stream listener on accepted connections, each from the item-0 high-waters. A query takes a permit only while it does work; a keep-alive connection releases its permit between requests.
|
||||||
|
- UDP keeps today's counted drop when no permit is free (`dropped_no_slot`). The kernel buffer does not absorb overload invisibly.
|
||||||
|
- The upstream exchange keeps its whole-operation `Select` deadline and its distinction between peer result, budget expiry and cancellation. The thread high-water falls because the query budget is smaller, not because the race changes.
|
||||||
|
- UDP slots and TCP connection buffers are sized from these bounds, not 64 per listener.
|
||||||
|
|
||||||
|
### 4. Size the query-log buffers by use
|
||||||
|
|
||||||
|
- The writer queue default (`logging.query_log_buffer_max`) comes from the item-0 occupancy high-water, with headroom stated in the reference. Drop-oldest backpressure and its counter stay.
|
||||||
|
- The SSE hub allocates a subscriber's ring on subscribe and frees it on unsubscribe; `max_subscribers` stays a cap, not a reservation. `subscribe` becomes fallible: an allocation failure is distinct from "all slots occupied", is counted, answers with its own error rather than "live stream is full" (`src/web/handlers/live.zig:75`), and leaves the slot reusable. Overflow keeps today's contract: the hub ends that subscriber's stream so `EventSource` reconnects (`src/web/sse.zig:8`).
|
||||||
|
|
||||||
|
### 5. SQLite caches chosen per connection
|
||||||
|
|
||||||
|
- An explicit runtime-connection initialization step sets `cache_size` from a constant per role (query-log writer, retention, readers, config). `Db.open` stays free of pragmas, because migration callers control when pragmas apply (`src/storage/db.zig:302`). Every runtime handle reads its value back.
|
||||||
|
|
||||||
|
### 6. DNS cache bounded in bytes
|
||||||
|
|
||||||
|
- `DnsCache` keeps its entry count and adds a budget on live response bytes, tracked separately from `memoryBytes` (which also counts slots and index). `put` evicts by CLOCK until the copy fits. A response larger than the whole budget is not cached and is counted. A replacement keeps the old entry if allocating the new copy fails. A failed copy allocation is counted; today `handler.zig:918` discards it with `catch {}` and `DnsCache.Stats` has no field for it. The answer already obtained is still sent. The budget, the live bytes and both counters appear in cache diagnostics.
|
||||||
|
- New config key `cache.max_bytes`, through the config field checklist. Default from the Pi's measured live response bytes.
|
||||||
|
|
||||||
|
## Propagation
|
||||||
|
|
||||||
|
- Config: `cache.max_bytes`; the changed default of `logging.query_log_buffer_max`. Config field checklist for the new key: model key + round-trip drift guards, validation + reference, settings API view, openapi.yaml, admin types + settings control, contract samples regenerated, docs config reference, CHANGELOG.
|
||||||
|
- Diagnostics: the item-0 counters, `ConcurrencyUnavailable`, oversized cache responses, cache live bytes and budget.
|
||||||
|
- Docs: `performance-and-testing.md` states the measured RSS, refresh peak and thread count at the Pi's blocklist size, replacing the unmeasured target.
|
||||||
|
|
||||||
|
## Acceptance
|
||||||
|
|
||||||
|
- [ ] Matcher equivalence: a deterministic corpus (hosts, domains, ABP with `||x^`, `*.x` alone, `x` alone, `x` with `*.x`, `@@`, an exact entry for a subdomain of a wildcard, nested wildcards `*.x` and `*.y.x`, overlapping sources) gives the same `Decision` for every probe name before and after item 1.
|
||||||
|
- [ ] Builder: a reload holds at most one source's bodies (tested through an allocator that records peak bytes).
|
||||||
|
- [ ] Boot: one snapshot build when no source refreshes; listeners never start without the initial load having run.
|
||||||
|
- [ ] Measurements: each of the five item-0 gauges rises under a scripted load and is visible in `/metrics`; the thread count and the worker stack residency are recorded from `/proc` beside them.
|
||||||
|
- [ ] Bounds: with the query budget full, a further UDP query is dropped and counted; a stream listener at its slot cap refuses and counts; a keep-alive connection between requests holds no query permit; `ConcurrencyUnavailable` is counted at each enumerated boundary, including the logger wait and the listener races.
|
||||||
|
- [ ] SSE: no ring is allocated without a subscriber; an allocation failure is counted, reported as its own error and leaves the slot reusable; overflow still ends the stream.
|
||||||
|
- [ ] SQLite: each runtime handle reports its configured `cache_size`; migrations open without it.
|
||||||
|
- [ ] DNS cache: puts beyond the byte budget evict; live bytes never exceed it; an oversized response is skipped and counted; a failed replacement keeps the old entry; a failed copy allocation is counted and the answer is still sent.
|
||||||
|
- [ ] Suites: `zig build test` and `-Dintegration` 0 failed; admin typecheck/vitest/prettier clean; contract samples regenerated.
|
||||||
|
- [ ] Measurement record, on the Pi with the same blocklists: steady VmRSS after 24 h, VmHWM across one scheduled refresh (including its concurrent download and compile), thread count, and a decoded heap dump per size class, each beside the baseline in the table above. The owner chose a record, not a pass threshold: each owner must be reduced as specified, and the numbers go into `performance-and-testing.md`. If small-class slabs remain a top owner, the allocator question reopens with that data.
|
||||||
@@ -148,6 +148,10 @@ pub const DohServer = struct {
|
|||||||
return self.core.boundAddress();
|
return self.core.boundAddress();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub fn connectionsPeak(self: *const DohServer) u64 {
|
||||||
|
return self.core.connectionsPeak();
|
||||||
|
}
|
||||||
|
|
||||||
pub fn snapshotStats(self: *const DohServer) Snapshot {
|
pub fn snapshotStats(self: *const DohServer) Snapshot {
|
||||||
const core = &self.core.stats;
|
const core = &self.core.stats;
|
||||||
return .{
|
return .{
|
||||||
|
|||||||
@@ -125,6 +125,10 @@ pub const DotServer = struct {
|
|||||||
self.* = undefined;
|
self.* = undefined;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub fn connectionsPeak(self: *const DotServer) u64 {
|
||||||
|
return self.core.connectionsPeak();
|
||||||
|
}
|
||||||
|
|
||||||
pub fn snapshotStats(self: *const DotServer) StatsSnapshot {
|
pub fn snapshotStats(self: *const DotServer) StatsSnapshot {
|
||||||
const core = &self.core.stats;
|
const core = &self.core.stats;
|
||||||
return .{
|
return .{
|
||||||
|
|||||||
@@ -130,6 +130,11 @@ pub const Handler = struct {
|
|||||||
pause: ?*pause.Pause = null,
|
pause: ?*pause.Pause = null,
|
||||||
tracker: ?*clients.Tracker = null,
|
tracker: ?*clients.Tracker = null,
|
||||||
stats: Stats = .{},
|
stats: Stats = .{},
|
||||||
|
/// Queries inside `handle` right now, across every listener, and the most
|
||||||
|
/// there have been at once (specs/memory-budget.md item 0). Gauges, so they
|
||||||
|
/// stay out of `Stats`, which `/metrics` exports field by field as counters.
|
||||||
|
in_flight: std.atomic.Value(u64) = .init(0),
|
||||||
|
in_flight_peak: std.atomic.Value(u64) = .init(0),
|
||||||
|
|
||||||
pub const Stats = struct {
|
pub const Stats = struct {
|
||||||
/// Queries answered by the upstream pool.
|
/// Queries answered by the upstream pool.
|
||||||
@@ -229,6 +234,21 @@ pub const Handler = struct {
|
|||||||
query: []const u8,
|
query: []const u8,
|
||||||
response_buf: []u8,
|
response_buf: []u8,
|
||||||
scratch: *Scratch,
|
scratch: *Scratch,
|
||||||
|
) Outcome {
|
||||||
|
const now_in_flight = self.in_flight.fetchAdd(1, .monotonic) + 1;
|
||||||
|
_ = self.in_flight_peak.fetchMax(now_in_flight, .monotonic);
|
||||||
|
defer _ = self.in_flight.fetchSub(1, .monotonic);
|
||||||
|
return self.answer(io, which, from, query, response_buf, scratch);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn answer(
|
||||||
|
self: *Handler,
|
||||||
|
io: std.Io,
|
||||||
|
which: Transport,
|
||||||
|
from: address.NetAddress,
|
||||||
|
query: []const u8,
|
||||||
|
response_buf: []u8,
|
||||||
|
scratch: *Scratch,
|
||||||
) Outcome {
|
) Outcome {
|
||||||
std.debug.assert(response_buf.len >= udp_limit_min);
|
std.debug.assert(response_buf.len >= udp_limit_min);
|
||||||
std.debug.assert(response_buf.len <= transport.max_message_len);
|
std.debug.assert(response_buf.len <= transport.max_message_len);
|
||||||
@@ -1459,6 +1479,24 @@ test "a response arriving on the listener port is dropped" {
|
|||||||
try testing.expectEqual(@as(usize, 0), fake.calls);
|
try testing.expectEqual(@as(usize, 0), fake.calls);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
test "every query counts in flight for exactly its own handling" {
|
||||||
|
var t: TestIo = .init();
|
||||||
|
defer t.deinit();
|
||||||
|
|
||||||
|
var fake: FakeUpstream = .{ .reply = response_bytes };
|
||||||
|
var h_owner: upstream_owner.Borrowed = .{};
|
||||||
|
var h = bare(h_owner.client(fake.client()));
|
||||||
|
// Two queries other listeners are still handling.
|
||||||
|
h.in_flight.store(2, .monotonic);
|
||||||
|
|
||||||
|
var buf: [udp_limit_min]u8 = undefined;
|
||||||
|
try testing.expect(udp(&h, t.io(), query_bytes, &buf) == .reply);
|
||||||
|
try testing.expectEqual(Handler.Outcome.drop, udp(&h, t.io(), response_bytes, &buf));
|
||||||
|
|
||||||
|
try testing.expectEqual(@as(u64, 2), h.in_flight.load(.monotonic));
|
||||||
|
try testing.expectEqual(@as(u64, 3), h.in_flight_peak.load(.monotonic));
|
||||||
|
}
|
||||||
|
|
||||||
test "a query shorter than a header is dropped" {
|
test "a query shorter than a header is dropped" {
|
||||||
var t: TestIo = .init();
|
var t: TestIo = .init();
|
||||||
defer t.deinit();
|
defer t.deinit();
|
||||||
|
|||||||
@@ -83,6 +83,10 @@ pub const CoreStats = struct {
|
|||||||
accept_errors: std.atomic.Value(u64) = .init(0),
|
accept_errors: std.atomic.Value(u64) = .init(0),
|
||||||
connection_errors: std.atomic.Value(u64) = .init(0),
|
connection_errors: std.atomic.Value(u64) = .init(0),
|
||||||
idle_timeouts: std.atomic.Value(u64) = .init(0),
|
idle_timeouts: std.atomic.Value(u64) = .init(0),
|
||||||
|
/// The most slots that have been taken at once (specs/memory-budget.md
|
||||||
|
/// item 0). A gauge, so no listener's `Snapshot` carries it: those export
|
||||||
|
/// every field as a counter. Written only under `Core.mutex`.
|
||||||
|
connections_peak: std.atomic.Value(u64) = .init(0),
|
||||||
};
|
};
|
||||||
|
|
||||||
// ---------------------------------------------------------------------------
|
// ---------------------------------------------------------------------------
|
||||||
@@ -255,6 +259,9 @@ pub fn Core(comptime Cfg: type) type {
|
|||||||
/// that shuts the active connections down, so a stream that arrives
|
/// that shuts the active connections down, so a stream that arrives
|
||||||
/// after that scan can never claim a slot the scan will not visit again.
|
/// after that scan can never claim a slot the scan will not visit again.
|
||||||
shutdown_begun: bool,
|
shutdown_begun: bool,
|
||||||
|
/// Guarded by `mutex`. Slots that are not `.free`: a slot is taken from
|
||||||
|
/// `claim` until `finish` frees it, closing included.
|
||||||
|
taken: u64,
|
||||||
stats: CoreStats,
|
stats: CoreStats,
|
||||||
run_state: std.atomic.Value(State),
|
run_state: std.atomic.Value(State),
|
||||||
stopped: std.Io.Event,
|
stopped: std.Io.Event,
|
||||||
@@ -299,6 +306,7 @@ pub fn Core(comptime Cfg: type) type {
|
|||||||
.conns = conns,
|
.conns = conns,
|
||||||
.mutex = .init,
|
.mutex = .init,
|
||||||
.shutdown_begun = false,
|
.shutdown_begun = false,
|
||||||
|
.taken = 0,
|
||||||
.stats = .{},
|
.stats = .{},
|
||||||
.run_state = .init(.idle),
|
.run_state = .init(.idle),
|
||||||
.stopped = .unset,
|
.stopped = .unset,
|
||||||
@@ -336,6 +344,10 @@ pub fn Core(comptime Cfg: type) type {
|
|||||||
return self.listener.socket.address;
|
return self.listener.socket.address;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub fn connectionsPeak(self: *const Self) u64 {
|
||||||
|
return self.stats.connections_peak.load(.monotonic);
|
||||||
|
}
|
||||||
|
|
||||||
/// Accept loop. Returns when the task is canceled or `deinit` stops it.
|
/// Accept loop. Returns when the task is canceled or `deinit` stops it.
|
||||||
pub fn serve(self: *Self, io: std.Io) void {
|
pub fn serve(self: *Self, io: std.Io) void {
|
||||||
if (self.run_state.cmpxchgStrong(.idle, .serving, .acq_rel, .acquire) != null) return;
|
if (self.run_state.cmpxchgStrong(.idle, .serving, .acq_rel, .acquire) != null) return;
|
||||||
@@ -443,6 +455,10 @@ pub fn Core(comptime Cfg: type) type {
|
|||||||
self.conns[index].stream = stream;
|
self.conns[index].stream = stream;
|
||||||
self.conns[index].peer = stream.socket.address;
|
self.conns[index].peer = stream.socket.address;
|
||||||
self.conns[index].state = .active;
|
self.conns[index].state = .active;
|
||||||
|
self.taken += 1;
|
||||||
|
if (self.taken > self.stats.connections_peak.load(.monotonic)) {
|
||||||
|
self.stats.connections_peak.store(self.taken, .monotonic);
|
||||||
|
}
|
||||||
},
|
},
|
||||||
.at_capacity, .shutting_down => {},
|
.at_capacity, .shutting_down => {},
|
||||||
}
|
}
|
||||||
@@ -464,6 +480,7 @@ pub fn Core(comptime Cfg: type) type {
|
|||||||
|
|
||||||
self.mutex.lockUncancelable(io);
|
self.mutex.lockUncancelable(io);
|
||||||
conn.state = .free;
|
conn.state = .free;
|
||||||
|
self.taken -= 1;
|
||||||
self.mutex.unlock(io);
|
self.mutex.unlock(io);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -99,6 +99,10 @@ pub const TcpServer = struct {
|
|||||||
return self.core.boundAddress();
|
return self.core.boundAddress();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub fn connectionsPeak(self: *const TcpServer) u64 {
|
||||||
|
return self.core.connectionsPeak();
|
||||||
|
}
|
||||||
|
|
||||||
/// The counters, read one at a time. A scrape that lands mid-accept can see
|
/// The counters, read one at a time. A scrape that lands mid-accept can see
|
||||||
/// a connection counted before its outcome is; a lock would buy a
|
/// a connection counted before its outcome is; a lock would buy a
|
||||||
/// consistency no consumer needs.
|
/// consistency no consumer needs.
|
||||||
|
|||||||
@@ -329,6 +329,52 @@ test "a canceled serve does not wait for a live connection" {
|
|||||||
try client_result;
|
try client_result;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
test "the connections peak is the most slots held at once" {
|
||||||
|
if (!build_options.integration) return error.SkipZigTest;
|
||||||
|
|
||||||
|
const gpa = testing.allocator;
|
||||||
|
var threaded: std.Io.Threaded = .init(gpa, .{});
|
||||||
|
defer threaded.deinit();
|
||||||
|
const io = threaded.io();
|
||||||
|
|
||||||
|
var fake: FakeUpstream = .{ .reply = response_bytes };
|
||||||
|
var h_owner: upstream_owner.Borrowed = .{};
|
||||||
|
var h = bareHandler(h_owner.client(fake.client()));
|
||||||
|
|
||||||
|
const listen_address: net.IpAddress = try .parse("127.0.0.1", 0);
|
||||||
|
var server = try tcp_server.TcpServer.listen(gpa, io, listen_address, &h, .{
|
||||||
|
.max_connections = 3,
|
||||||
|
.idle_timeout = .{ .raw = .fromSeconds(600), .clock = .awake },
|
||||||
|
});
|
||||||
|
const server_address = server.boundAddress();
|
||||||
|
|
||||||
|
var serving: std.Io.Group = .init;
|
||||||
|
try serving.concurrent(io, tcp_server.TcpServer.serve, .{ &server, io });
|
||||||
|
|
||||||
|
var answered: [2]std.Io.Event = @splat(.unset);
|
||||||
|
var client_results: [2]anyerror!void = @splat({});
|
||||||
|
var clients_group: std.Io.Group = .init;
|
||||||
|
for (&answered, &client_results) |*event, *result| {
|
||||||
|
try clients_group.concurrent(io, holdConnection, .{ io, server_address, event, result });
|
||||||
|
}
|
||||||
|
for (&answered) |*event| event.waitUncancelable(io);
|
||||||
|
|
||||||
|
// Both replies prove both slots are claimed, and the listener has room for
|
||||||
|
// a third that never arrives.
|
||||||
|
const held_open = server.connectionsPeak();
|
||||||
|
|
||||||
|
server.deinit(io);
|
||||||
|
serving.await(io) catch |err| switch (err) {
|
||||||
|
error.Canceled => unreachable,
|
||||||
|
};
|
||||||
|
clients_group.await(io) catch |err| switch (err) {
|
||||||
|
error.Canceled => unreachable,
|
||||||
|
};
|
||||||
|
|
||||||
|
for (client_results) |result| try result;
|
||||||
|
try testing.expectEqual(@as(u64, 2), held_open);
|
||||||
|
}
|
||||||
|
|
||||||
/// The first slot the server would hand out, read after `serve` has returned so
|
/// The first slot the server would hand out, read after `serve` has returned so
|
||||||
/// nothing can be writing it.
|
/// nothing can be writing it.
|
||||||
fn firstFreeSlot(server: *const tcp_server.TcpServer) ?usize {
|
fn firstFreeSlot(server: *const tcp_server.TcpServer) ?usize {
|
||||||
|
|||||||
+91
-3
@@ -320,7 +320,68 @@ fn emptyAsNull(value: []const u8) ?[]const u8 {
|
|||||||
return if (value.len == 0) null else value;
|
return if (value.len == 0) null else value;
|
||||||
}
|
}
|
||||||
|
|
||||||
const EntryQueue = std.Io.Queue(Entry);
|
/// `std.Io.Queue` plus the occupancy high-water that sizes
|
||||||
|
/// `logging.query_log_buffer_max` (specs/memory-budget.md item 0).
|
||||||
|
///
|
||||||
|
/// The queue keeps its length behind its own mutex with no accessor, so the
|
||||||
|
/// count lives beside it. Raised before an insert and lowered after a removal,
|
||||||
|
/// it never reads below the true occupancy; it overstates it by at most the
|
||||||
|
/// producers inside `tryPut` plus the entries a getter has taken and not yet
|
||||||
|
/// subtracted. An upper bound is the safe side for sizing a buffer.
|
||||||
|
const EntryQueue = struct {
|
||||||
|
inner: std.Io.Queue(Entry),
|
||||||
|
held: std.atomic.Value(u64) = .init(0),
|
||||||
|
peak: std.atomic.Value(u64) = .init(0),
|
||||||
|
|
||||||
|
pub fn init(buffer: []Entry) EntryQueue {
|
||||||
|
return .{ .inner = .init(buffer) };
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn capacity(q: *const EntryQueue) usize {
|
||||||
|
return q.inner.capacity();
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn close(q: *EntryQueue, io: std.Io) void {
|
||||||
|
q.inner.close(io);
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn peakOccupancy(q: *const EntryQueue) u64 {
|
||||||
|
return q.peak.load(.monotonic);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Takes `entry` if there is room or a waiting getter, without blocking.
|
||||||
|
pub fn tryPut(q: *EntryQueue, io: std.Io, entry: Entry) (std.Io.QueueClosedError || std.Io.Cancelable)!bool {
|
||||||
|
const held = q.held.fetchAdd(1, .monotonic) + 1;
|
||||||
|
const put = q.inner.put(io, &.{entry}, 0) catch |err| {
|
||||||
|
_ = q.held.fetchSub(1, .monotonic);
|
||||||
|
return err;
|
||||||
|
};
|
||||||
|
if (put == 0) {
|
||||||
|
_ = q.held.fetchSub(1, .monotonic);
|
||||||
|
return false;
|
||||||
|
}
|
||||||
|
_ = q.peak.fetchMax(held, .monotonic);
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn get(q: *EntryQueue, io: std.Io, buffer: []Entry, min: usize) (std.Io.QueueClosedError || std.Io.Cancelable)!usize {
|
||||||
|
const n = try q.inner.get(io, buffer, min);
|
||||||
|
_ = q.held.fetchSub(n, .monotonic);
|
||||||
|
return n;
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn getUncancelable(q: *EntryQueue, io: std.Io, buffer: []Entry, min: usize) std.Io.QueueClosedError!usize {
|
||||||
|
const n = try q.inner.getUncancelable(io, buffer, min);
|
||||||
|
_ = q.held.fetchSub(n, .monotonic);
|
||||||
|
return n;
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn getOne(q: *EntryQueue, io: std.Io) (std.Io.QueueClosedError || std.Io.Cancelable)!Entry {
|
||||||
|
const entry = try q.inner.getOne(io);
|
||||||
|
_ = q.held.fetchSub(1, .monotonic);
|
||||||
|
return entry;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
/// A database failure is not in here: a batch the database refuses is dropped,
|
/// A database failure is not in here: a batch the database refuses is dropped,
|
||||||
/// counted and reported where it happens, and the writer carries on.
|
/// counted and reported where it happens, and the writer carries on.
|
||||||
@@ -525,8 +586,7 @@ pub const Logger = struct {
|
|||||||
while (true) {
|
while (true) {
|
||||||
// A closed queue or a canceled task means shutdown is underway;
|
// A closed queue or a canceled task means shutdown is underway;
|
||||||
// both leave this entry unwritten, which is what the counter says.
|
// both leave this entry unwritten, which is what the counter says.
|
||||||
const put = self.queue.put(io, &.{entry}, 0) catch break;
|
if (self.queue.tryPut(io, entry) catch break) return;
|
||||||
if (put == 1) return;
|
|
||||||
|
|
||||||
// A zero-capacity queue holds nothing to drop: the put above was
|
// A zero-capacity queue holds nothing to drop: the put above was
|
||||||
// this entry's one chance at a waiting getter.
|
// this entry's one chance at a waiting getter.
|
||||||
@@ -1442,6 +1502,34 @@ test "a zero-capacity queue drops every entry exactly once" {
|
|||||||
try testing.expectEqual(@as(u64, 5), logger.queries_dropped.load(.monotonic));
|
try testing.expectEqual(@as(u64, 5), logger.queries_dropped.load(.monotonic));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
test "the queue peak is the most entries held at once, never the arrivals" {
|
||||||
|
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
|
||||||
|
defer threaded.deinit();
|
||||||
|
const io = threaded.io();
|
||||||
|
|
||||||
|
var buf: [4]Entry = undefined;
|
||||||
|
var logger: Logger = .init(.{}, &buf);
|
||||||
|
try testing.expectEqual(@as(u64, 0), logger.queue.peakOccupancy());
|
||||||
|
|
||||||
|
logger.log(io, sampleEntry(1, "one.example"));
|
||||||
|
logger.log(io, sampleEntry(2, "two.example"));
|
||||||
|
logger.log(io, sampleEntry(3, "three.example"));
|
||||||
|
_ = try logger.queue.getOne(io);
|
||||||
|
_ = try logger.queue.getOne(io);
|
||||||
|
logger.log(io, sampleEntry(4, "four.example"));
|
||||||
|
try testing.expectEqual(@as(u64, 3), logger.queue.peakOccupancy());
|
||||||
|
|
||||||
|
// A full queue evicts to make room, so the peak stops at the capacity.
|
||||||
|
for (5..10) |i| logger.log(io, sampleEntry(@intCast(i), "more.example"));
|
||||||
|
try testing.expectEqual(@as(u64, 4), logger.queue.peakOccupancy());
|
||||||
|
try testing.expectEqual(@as(u64, 4), logger.queue.held.load(.monotonic));
|
||||||
|
|
||||||
|
var drained: [4]Entry = undefined;
|
||||||
|
try testing.expectEqual(@as(usize, 4), try logger.queue.get(io, &drained, 0));
|
||||||
|
try testing.expectEqual(@as(u64, 0), logger.queue.held.load(.monotonic));
|
||||||
|
try testing.expectEqual(@as(u64, 4), logger.queue.peakOccupancy());
|
||||||
|
}
|
||||||
|
|
||||||
test "log after shutdown drops instead of blocking" {
|
test "log after shutdown drops instead of blocking" {
|
||||||
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
|
var threaded: std.Io.Threaded = .init(testing.allocator, .{});
|
||||||
defer threaded.deinit();
|
defer threaded.deinit();
|
||||||
|
|||||||
@@ -63,12 +63,14 @@ const Totals = struct {
|
|||||||
rows_written: u64 = 0,
|
rows_written: u64 = 0,
|
||||||
batches_gated: u64 = 0,
|
batches_gated: u64 = 0,
|
||||||
last_drop_s: i64 = 0,
|
last_drop_s: i64 = 0,
|
||||||
|
queue_peak: u64 = 0,
|
||||||
|
|
||||||
fn fold(self: *Totals, lg: *const logger.Logger) void {
|
fn fold(self: *Totals, lg: *const logger.Logger) void {
|
||||||
self.queries_dropped += lg.queries_dropped.load(.monotonic);
|
self.queries_dropped += lg.queries_dropped.load(.monotonic);
|
||||||
self.rows_written += lg.rows_written.load(.monotonic);
|
self.rows_written += lg.rows_written.load(.monotonic);
|
||||||
self.batches_gated += lg.batches_gated.load(.monotonic);
|
self.batches_gated += lg.batches_gated.load(.monotonic);
|
||||||
self.last_drop_s = @max(self.last_drop_s, lg.last_drop_s.load(.monotonic));
|
self.last_drop_s = @max(self.last_drop_s, lg.last_drop_s.load(.monotonic));
|
||||||
|
self.queue_peak = @max(self.queue_peak, lg.queue.peakOccupancy());
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -80,6 +82,8 @@ pub const Sample = struct {
|
|||||||
batches_gated: u64,
|
batches_gated: u64,
|
||||||
/// Null while nothing has been dropped, exactly as `Logger.lastDropSeconds`.
|
/// Null while nothing has been dropped, exactly as `Logger.lastDropSeconds`.
|
||||||
last_drop_s: ?i64,
|
last_drop_s: ?i64,
|
||||||
|
/// The most entries any generation's queue has held at once.
|
||||||
|
queue_peak: u64,
|
||||||
/// The LIVE writer's state. A retired writer's failures are in the counters.
|
/// The LIVE writer's state. A retired writer's failures are in the counters.
|
||||||
writer_failed: bool,
|
writer_failed: bool,
|
||||||
/// The worst episode any outstanding generation is in. A retired writer
|
/// The worst episode any outstanding generation is in. A retired writer
|
||||||
@@ -376,6 +380,7 @@ pub const Controller = struct {
|
|||||||
.rows_written = totals.rows_written,
|
.rows_written = totals.rows_written,
|
||||||
.batches_gated = totals.batches_gated,
|
.batches_gated = totals.batches_gated,
|
||||||
.last_drop_s = if (totals.last_drop_s == 0) null else totals.last_drop_s,
|
.last_drop_s = if (totals.last_drop_s == 0) null else totals.last_drop_s,
|
||||||
|
.queue_peak = totals.queue_peak,
|
||||||
.writer_failed = self.live.logger.writer_failed.load(.acquire),
|
.writer_failed = self.live.logger.writer_failed.load(.acquire),
|
||||||
.gate_episode = episode,
|
.gate_episode = episode,
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -130,6 +130,20 @@ pub const DiagnosticsSample = struct {
|
|||||||
write_failures: u64,
|
write_failures: u64,
|
||||||
};
|
};
|
||||||
|
|
||||||
|
/// The most of each kind of concurrent work there has been at once since start
|
||||||
|
/// (specs/memory-budget.md item 0): what the admission budget, the listener
|
||||||
|
/// caps and the query-log queue are sized from. Gauges, so none of them rides
|
||||||
|
/// in a counter group. A null field is a collaborator the state does not have.
|
||||||
|
pub const PeakSample = struct {
|
||||||
|
dns_in_flight: ?u64 = null,
|
||||||
|
querylog_queue: ?u64 = null,
|
||||||
|
/// The larger of the two address families' peaks. Each family has its own
|
||||||
|
/// slot pool, so a cap is sized per listener, never from their sum.
|
||||||
|
tcp_connections: ?u64 = null,
|
||||||
|
dot_connections: ?u64 = null,
|
||||||
|
doh_connections: ?u64 = null,
|
||||||
|
};
|
||||||
|
|
||||||
/// Everything one scrape reports. A null section is a collaborator the state
|
/// Everything one scrape reports. A null section is a collaborator the state
|
||||||
/// does not have.
|
/// does not have.
|
||||||
pub const Sample = struct {
|
pub const Sample = struct {
|
||||||
@@ -172,6 +186,7 @@ pub const Sample = struct {
|
|||||||
/// label and cannot live in `UpstreamSample`. Absent while no pool is
|
/// label and cannot live in `UpstreamSample`. Absent while no pool is
|
||||||
/// wired, like every other collaborator.
|
/// wired, like every other collaborator.
|
||||||
upstream_budget_exhausted_total: ?u64 = null,
|
upstream_budget_exhausted_total: ?u64 = null,
|
||||||
|
peaks: PeakSample = .{},
|
||||||
};
|
};
|
||||||
|
|
||||||
pub fn handle(
|
pub fn handle(
|
||||||
@@ -193,6 +208,7 @@ pub fn collect(state: *server.WebState, io: std.Io, arena: Allocator) Allocator.
|
|||||||
|
|
||||||
if (state.handler) |handler| {
|
if (state.handler) |handler| {
|
||||||
sample.dns = dnsCounters(&handler.stats);
|
sample.dns = dnsCounters(&handler.stats);
|
||||||
|
sample.peaks.dns_in_flight = handler.in_flight_peak.load(.monotonic);
|
||||||
|
|
||||||
// The pointer is loaded under the same mutex a query loads it under,
|
// The pointer is loaded under the same mutex a query loads it under,
|
||||||
// never before it: a live `cache.size` or `dns.rate_limit` change
|
// never before it: a live `cache.size` or `dns.rate_limit` change
|
||||||
@@ -223,6 +239,7 @@ pub fn collect(state: *server.WebState, io: std.Io, arena: Allocator) Allocator.
|
|||||||
.rows_written = reading.rows_written,
|
.rows_written = reading.rows_written,
|
||||||
.batches_gated = reading.batches_gated,
|
.batches_gated = reading.batches_gated,
|
||||||
};
|
};
|
||||||
|
sample.peaks.querylog_queue = reading.queue_peak;
|
||||||
}
|
}
|
||||||
|
|
||||||
if (state.tracker) |tracker| sample.tracker = .{
|
if (state.tracker) |tracker| sample.tracker = .{
|
||||||
@@ -270,9 +287,12 @@ pub fn collect(state: *server.WebState, io: std.Io, arena: Allocator) Allocator.
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
if (state.dot_listener) |listener| sample.dot_listener = listener.snapshotStats();
|
if (state.dot_listener) |listener| sample.dot_listener = listener.snapshotStats();
|
||||||
|
if (state.doh_listener) |listener| sample.peaks.doh_connections = listener.connectionsPeak();
|
||||||
|
if (state.dot_listener) |listener| sample.peaks.dot_connections = listener.connectionsPeak();
|
||||||
|
|
||||||
sample.udp_listener = sumListeners(udp_server.Snapshot, udp_server.UdpServer, state.udp_listeners);
|
sample.udp_listener = sumListeners(udp_server.Snapshot, udp_server.UdpServer, state.udp_listeners);
|
||||||
sample.tcp_listener = sumListeners(tcp_server.Snapshot, tcp_server.TcpServer, state.tcp_listeners);
|
sample.tcp_listener = sumListeners(tcp_server.Snapshot, tcp_server.TcpServer, state.tcp_listeners);
|
||||||
|
sample.peaks.tcp_connections = largestConnectionsPeak(state.tcp_listeners);
|
||||||
|
|
||||||
if (state.upstreams) |owner| {
|
if (state.upstreams) |owner| {
|
||||||
const generation = owner.acquire(io);
|
const generation = owner.acquire(io);
|
||||||
@@ -308,6 +328,13 @@ fn sumListeners(comptime Snapshot: type, comptime Server: type, listeners: []con
|
|||||||
return total;
|
return total;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Null for an empty slice, like `sumListeners`.
|
||||||
|
fn largestConnectionsPeak(listeners: []const *tcp_server.TcpServer) ?u64 {
|
||||||
|
var largest: ?u64 = null;
|
||||||
|
for (listeners) |listener| largest = @max(largest orelse 0, listener.connectionsPeak());
|
||||||
|
return largest;
|
||||||
|
}
|
||||||
|
|
||||||
fn dnsCounters(stats: *const dns_handler.Handler.Stats) DnsCounters {
|
fn dnsCounters(stats: *const dns_handler.Handler.Stats) DnsCounters {
|
||||||
var out: DnsCounters = undefined;
|
var out: DnsCounters = undefined;
|
||||||
inline for (dns_stat_fields, 0..) |field, i| {
|
inline for (dns_stat_fields, 0..) |field, i| {
|
||||||
@@ -471,6 +498,8 @@ pub fn render(w: *std.Io.Writer, sample: Sample) std.Io.Writer.Error!void {
|
|||||||
try counterGroup(w, "nxdns_dot_server_", "DoT listener counter", listener);
|
try counterGroup(w, "nxdns_dot_server_", "DoT listener counter", listener);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
try renderPeaks(w, sample.peaks);
|
||||||
|
|
||||||
if (sample.doh_certs != null or sample.dot_certs != null) try renderCerts(w, sample);
|
if (sample.doh_certs != null or sample.dot_certs != null) try renderCerts(w, sample);
|
||||||
|
|
||||||
if (sample.upstreams.len != 0) try renderUpstreams(w, sample.upstreams);
|
if (sample.upstreams.len != 0) try renderUpstreams(w, sample.upstreams);
|
||||||
@@ -485,6 +514,39 @@ pub fn render(w: *std.Io.Writer, sample: Sample) std.Io.Writer.Error!void {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
fn renderPeaks(w: *std.Io.Writer, peaks: PeakSample) std.Io.Writer.Error!void {
|
||||||
|
if (peaks.dns_in_flight) |value| try gauge(
|
||||||
|
w,
|
||||||
|
"nxdns_dns_in_flight_peak",
|
||||||
|
"Most DNS queries handled at once since start, across every listener.",
|
||||||
|
value,
|
||||||
|
);
|
||||||
|
if (peaks.querylog_queue) |value| try gauge(
|
||||||
|
w,
|
||||||
|
"nxdns_querylog_queue_peak",
|
||||||
|
"Upper bound on the most entries the query log queue has held at once since start.",
|
||||||
|
value,
|
||||||
|
);
|
||||||
|
if (peaks.tcp_connections) |value| try gauge(
|
||||||
|
w,
|
||||||
|
"nxdns_tcp_server_connections_peak",
|
||||||
|
"Most connections one TCP/53 listener has held at once since start.",
|
||||||
|
value,
|
||||||
|
);
|
||||||
|
if (peaks.dot_connections) |value| try gauge(
|
||||||
|
w,
|
||||||
|
"nxdns_dot_server_connections_peak",
|
||||||
|
"Most connections the DoT listener has held at once since start.",
|
||||||
|
value,
|
||||||
|
);
|
||||||
|
if (peaks.doh_connections) |value| try gauge(
|
||||||
|
w,
|
||||||
|
"nxdns_doh_server_connections_peak",
|
||||||
|
"Most connections the DoH listener has held at once since start.",
|
||||||
|
value,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
fn renderCerts(w: *std.Io.Writer, sample: Sample) std.Io.Writer.Error!void {
|
fn renderCerts(w: *std.Io.Writer, sample: Sample) std.Io.Writer.Error!void {
|
||||||
try labeledHead(w, "nxdns_cert_reloads_total", "Certificate reloads that published a new context.", "counter");
|
try labeledHead(w, "nxdns_cert_reloads_total", "Certificate reloads that published a new context.", "counter");
|
||||||
if (sample.doh_certs) |stats| try endpointValue(w, "nxdns_cert_reloads_total", "doh", stats.reloads);
|
if (sample.doh_certs) |stats| try endpointValue(w, "nxdns_cert_reloads_total", "doh", stats.reloads);
|
||||||
@@ -979,6 +1041,35 @@ test "an unwired collaborator omits its family rather than reporting zeros" {
|
|||||||
try testing.expect(!std.mem.containsAtLeast(u8, text, 1, "nxdns_dot_server_"));
|
try testing.expect(!std.mem.containsAtLeast(u8, text, 1, "nxdns_dot_server_"));
|
||||||
try testing.expect(!std.mem.containsAtLeast(u8, text, 1, "nxdns_udp_server_"));
|
try testing.expect(!std.mem.containsAtLeast(u8, text, 1, "nxdns_udp_server_"));
|
||||||
try testing.expect(!std.mem.containsAtLeast(u8, text, 1, "nxdns_tcp_server_"));
|
try testing.expect(!std.mem.containsAtLeast(u8, text, 1, "nxdns_tcp_server_"));
|
||||||
|
try testing.expect(!std.mem.containsAtLeast(u8, text, 1, "_peak"));
|
||||||
|
}
|
||||||
|
|
||||||
|
test "each peak is a gauge, and an unwired one is absent" {
|
||||||
|
const text = try renderToString(testing.allocator, .{ .peaks = .{
|
||||||
|
.dns_in_flight = 7,
|
||||||
|
.querylog_queue = 120,
|
||||||
|
.tcp_connections = 3,
|
||||||
|
} });
|
||||||
|
defer testing.allocator.free(text);
|
||||||
|
|
||||||
|
try testing.expect(std.mem.containsAtLeast(u8, text, 1, "# TYPE nxdns_dns_in_flight_peak gauge\nnxdns_dns_in_flight_peak 7\n"));
|
||||||
|
try testing.expect(std.mem.containsAtLeast(u8, text, 1, "# TYPE nxdns_querylog_queue_peak gauge\nnxdns_querylog_queue_peak 120\n"));
|
||||||
|
try testing.expect(std.mem.containsAtLeast(u8, text, 1, "# TYPE nxdns_tcp_server_connections_peak gauge\nnxdns_tcp_server_connections_peak 3\n"));
|
||||||
|
try testing.expect(!std.mem.containsAtLeast(u8, text, 1, "nxdns_dot_server_connections_peak"));
|
||||||
|
try testing.expect(!std.mem.containsAtLeast(u8, text, 1, "nxdns_doh_server_connections_peak"));
|
||||||
|
}
|
||||||
|
|
||||||
|
test "the TCP peak is the larger listener's, never the sum" {
|
||||||
|
var tcp6: tcp_server.TcpServer = undefined;
|
||||||
|
tcp6.core.stats = .{};
|
||||||
|
tcp6.core.stats.connections_peak.store(4, .monotonic);
|
||||||
|
|
||||||
|
var tcp4: tcp_server.TcpServer = undefined;
|
||||||
|
tcp4.core.stats = .{};
|
||||||
|
tcp4.core.stats.connections_peak.store(6, .monotonic);
|
||||||
|
|
||||||
|
try testing.expectEqual(@as(?u64, 6), largestConnectionsPeak(&.{ &tcp6, &tcp4 }));
|
||||||
|
try testing.expectEqual(@as(?u64, null), largestConnectionsPeak(&.{}));
|
||||||
}
|
}
|
||||||
|
|
||||||
test "the plain-DNS listener families carry every counter of both listeners" {
|
test "the plain-DNS listener families carry every counter of both listeners" {
|
||||||
|
|||||||
+9
-9
@@ -1,6 +1,6 @@
|
|||||||
# nxdns.sh — marketing and docs website plan
|
# nxdns.sh — marketing and docs website plan
|
||||||
|
|
||||||
Agreed in an interview on 2026-08-15. This directory will hold the site's source.
|
Agreed in an interview on 2026-08-15, revised on 2026-10-02 and 2026-10-19 (screenshots in, section order set, the personal story moved to the owner's blog). This directory will hold the site's source.
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
@@ -19,13 +19,14 @@ Convince, then teach. A marketing front page sells the project; the docs sit one
|
|||||||
|
|
||||||
## Landing page
|
## Landing page
|
||||||
|
|
||||||
Three sections:
|
Four sections, in this order:
|
||||||
|
|
||||||
1. Hero: one-line pitch and a copyable install command. Releases 0.0.1–0.0.3 exist, so the command is real.
|
1. Hero: the README pitch — "A DNS sinkhole for your LAN. It blocks the names you do not want, and forwards the rest over an encrypted connection." — and a copyable install command in two tabs: Docker compose first, the signed tarball second. The registry allows anonymous pulls (verified 2026-10-02 against v0.0.21), so the Docker tab works without an account.
|
||||||
2. Feature grid: filtering, config-as-code, DoH/DoT, per-client groups, metrics, query log.
|
2. Screenshots of the web UI, taken from a running instance with realistic data: Overview, Activity (with a query's provenance), Clients, Diagnostics.
|
||||||
3. Comparison table vs Pi-hole, AdGuard Home, Technitium — short, only to help people choose.
|
3. Feature grid: filtering, config-as-code, DoH/DoT, per-client groups, metrics, query log.
|
||||||
|
4. Comparison table vs Pi-hole, AdGuard Home, Technitium — only to help people choose. Seven rows: how it ships; config as a file in git that the tool converges onto; encrypted upstreams built in; serves DoH/DoT to clients; signed, reproducible releases; DHCP server; DNSSEC validation. The last two are rows nxdns loses, kept because a table that only lists wins does not help anyone choose. Competitor cells were checked against Pi-hole 6.5 (FTL 6.8), AdGuard Home 0.107.79 and Technitium 15.6.2 on 2026-10-19.
|
||||||
|
|
||||||
The pitch leads with what nxdns does, not with attacks on the incumbents. No screenshots.
|
The pitch leads with what nxdns does, not with attacks on the incumbents. The page is about the tool: why the owner built it is a blog article, not a landing-page section.
|
||||||
|
|
||||||
## Docs section
|
## Docs section
|
||||||
|
|
||||||
@@ -38,7 +39,7 @@ The pitch leads with what nxdns does, not with attacks on the incumbents. No scr
|
|||||||
- Clean technical. Light default with a dark toggle.
|
- Clean technical. Light default with a dark toggle.
|
||||||
- Orange/amber accent.
|
- Orange/amber accent.
|
||||||
- Brand is the lowercase `nxdns` wordmark in a distinctive typeface. No drawn logo.
|
- Brand is the lowercase `nxdns` wordmark in a distinctive typeface. No drawn logo.
|
||||||
- Copy tone: personal / indie, first person — why I built it and what it fixes — in the honest voice the docs already use.
|
- Copy tone: the honest, direct voice the docs already use, about the tool. The personal story is the blog's.
|
||||||
|
|
||||||
## Links
|
## Links
|
||||||
|
|
||||||
@@ -47,5 +48,4 @@ Source links point to `git.mial.net/mokhtar/nxdns` only. No GitHub mirror.
|
|||||||
## Open questions
|
## Open questions
|
||||||
|
|
||||||
1. Wordmark typeface — resolve in the design pass.
|
1. Wordmark typeface — resolve in the design pass.
|
||||||
2. Does the Gitea container registry allow anonymous pulls? That decides whether the hero shows `docker run` or the binary install from INSTALL.md.
|
2. Re-check the comparison cells against the then-current competitor versions before publication. AdGuard Home had a 1.0 beta on 2026-09-30, so its cells are the most likely to move.
|
||||||
3. Fact-check the comparison table against current competitor versions before publication.
|
|
||||||
|
|||||||
Reference in New Issue
Block a user