release: nix flake with tag-pinned hashes, reproducible tarballs (milestone 40)
flake.nix fetches the release tarballs and carries their SRI hashes in a generated block. The cut tool builds the release locally with the toolchain gates.yml pins, in a normalized nine-variable environment, writes the hashes into flake.nix, and commits it with build.zig.zon as the single bump commit. The package job verifies the pins on the bump commit and the publish job verifies them again on the tag, before anything is uploaded. The tarballs are written by dist_stage (std.tar.Writer, flate gzip) instead of the runner's tar and gzip, and -ffile-prefix-map keeps checkout paths out of the C objects; two checkouts at different absolute paths produce byte-identical archives. nxdns version, /api/version and the admin footer report the version only: the bump commit cannot know its own sha.
This commit is contained in:
@@ -22,6 +22,7 @@ Steps for a goal you already have. They assume you know what nxdns is.
|
||||
- [how-to/verify-a-release.md](how-to/verify-a-release.md) — check the signature and the checksums before you run anything, and what they prove.
|
||||
- [how-to/install-with-systemd.md](how-to/install-with-systemd.md) — a real install as a system service, including the Raspberry Pi 5 aarch64 binary.
|
||||
- [how-to/install-with-docker.md](how-to/install-with-docker.md) — the published container image and the compose file.
|
||||
- [how-to/install-with-nix.md](how-to/install-with-nix.md) — the flake input pinned to a release tag, and Renovate for tag bumps.
|
||||
- [how-to/upgrade.md](how-to/upgrade.md) — move to a new release without losing state.
|
||||
- [how-to/troubleshoot.md](how-to/troubleshoot.md) — what to do when it does not answer, does not block, or will not start.
|
||||
- [how-to/enable-doh-and-dot.md](how-to/enable-doh-and-dot.md) — serve encrypted DNS with certificates.
|
||||
|
||||
@@ -11,6 +11,7 @@ pub const tutorial_first_run_md = @embedFile("tutorial/first-run.md");
|
||||
pub const howto_back_up_and_restore_md = @embedFile("how-to/back-up-and-restore.md");
|
||||
pub const howto_enable_doh_and_dot_md = @embedFile("how-to/enable-doh-and-dot.md");
|
||||
pub const howto_install_with_docker_md = @embedFile("how-to/install-with-docker.md");
|
||||
pub const howto_install_with_nix_md = @embedFile("how-to/install-with-nix.md");
|
||||
pub const howto_install_with_systemd_md = @embedFile("how-to/install-with-systemd.md");
|
||||
pub const howto_measure_performance_md = @embedFile("how-to/measure-performance.md");
|
||||
pub const howto_set_up_admin_authentication_md = @embedFile("how-to/set-up-admin-authentication.md");
|
||||
@@ -34,6 +35,7 @@ pub const pages: []const Page = &.{
|
||||
.{ .path = "docs/how-to/back-up-and-restore.md", .text = howto_back_up_and_restore_md },
|
||||
.{ .path = "docs/how-to/enable-doh-and-dot.md", .text = howto_enable_doh_and_dot_md },
|
||||
.{ .path = "docs/how-to/install-with-docker.md", .text = howto_install_with_docker_md },
|
||||
.{ .path = "docs/how-to/install-with-nix.md", .text = howto_install_with_nix_md },
|
||||
.{ .path = "docs/how-to/install-with-systemd.md", .text = howto_install_with_systemd_md },
|
||||
.{ .path = "docs/how-to/measure-performance.md", .text = howto_measure_performance_md },
|
||||
.{ .path = "docs/how-to/set-up-admin-authentication.md", .text = howto_set_up_admin_authentication_md },
|
||||
|
||||
@@ -220,7 +220,7 @@ The Dockerfile does not compile anything. It assembles a filesystem around binar
|
||||
```sh
|
||||
(cd admin && npm ci && npm run build)
|
||||
VERSION=$(sed -n 's/^[[:space:]]*\.version[[:space:]]*=[[:space:]]*"\([^"]*\)".*/\1/p' build.zig.zon)
|
||||
zig build dist -Dversion-string="$VERSION" -Dgit-commit="$(git rev-parse HEAD)" \
|
||||
zig build dist -Dversion-string="$VERSION" \
|
||||
-Dadmin-dist=admin/dist -Doptimize=ReleaseSafe
|
||||
DOCKER_BUILDKIT=1 docker build -t nxdns -f deploy/docker/Dockerfile .
|
||||
```
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
# Install nxdns with Nix
|
||||
|
||||
Adds nxdns to a NixOS machine as a flake input pinned to a release tag. At the end `pkgs`-style references to `inputs.nxdns.packages.${system}.default` resolve to the published release binary, and Renovate opens a pull request when a new tag appears.
|
||||
|
||||
nxdns publishes its own `flake.nix`. Its packages do not build nxdns from source: each one fetches the release tarball for the target and pins its SHA-256 hash, so a changed byte fails the build. The two supported systems are `aarch64-linux` and `x86_64-linux`, both static musl builds that need nothing on the host.
|
||||
|
||||
For the signature and checksum checks a human does once, see [verify a release](verify-a-release.md). For what each configuration field means, see [the configuration reference](../reference/configuration.md).
|
||||
|
||||
> Verification: the commands in step 1 and step 2 were run on the machine that wrote this page, against the flake at nxdns 0.0.16. `nix flake check --no-build` passed and `nix build` produced a binary that printed `nxdns 0.0.16`. The consumer snippets in step 3 and step 4 are copied from the shape `rpi.mial.net` already uses for another flake input of the same author; they were not evaluated from this checkout, which is not a NixOS configuration.
|
||||
|
||||
## 1. Add the input, pinned to a tag
|
||||
|
||||
In the consuming flake:
|
||||
|
||||
```nix
|
||||
inputs = {
|
||||
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
|
||||
|
||||
nxdns = {
|
||||
url = "git+https://git.mial.net/mokhtar/nxdns.git?ref=refs/tags/v0.0.16";
|
||||
inputs.nixpkgs.follows = "nixpkgs";
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
Pin a tag, not a branch. A branch pin moves the version under you at the next `nix flake update`, and the hashes in the flake belong to whatever release that branch last cut.
|
||||
|
||||
The `follows` line is not cosmetic. Without it Nix fetches and evaluates nxdns's own nixpkgs as a second nixpkgs, which costs a download and an evaluation for a package that only needs `stdenv`, `fetchurl`, and `lib`.
|
||||
|
||||
Write the lock entry:
|
||||
|
||||
```sh
|
||||
nix flake lock
|
||||
```
|
||||
|
||||
## 2. Check what you pinned
|
||||
|
||||
```sh
|
||||
nix flake check --no-build
|
||||
nix eval .#packages.x86_64-linux.default.outPath
|
||||
```
|
||||
|
||||
Evaluation does not fetch the tarball. The download happens at build time, and the hash in nxdns's flake is what the build asserts the bytes against.
|
||||
|
||||
## 3. Use the package in a NixOS configuration
|
||||
|
||||
Pass the flake inputs to the module system, then reference the package:
|
||||
|
||||
```nix
|
||||
{ inputs, pkgs, ... }:
|
||||
{
|
||||
environment.systemPackages = [ inputs.nxdns.packages.${pkgs.stdenv.hostPlatform.system}.default ];
|
||||
}
|
||||
```
|
||||
|
||||
The derivation installs `bin/nxdns`, plus `LICENSE` and `THIRD-PARTY-NOTICES` under `share/doc/nxdns`. It sets `meta.mainProgram`, so `lib.getExe` resolves to the binary. Run it as a service with the unit from [install with systemd](install-with-systemd.md), or write your own module around it.
|
||||
|
||||
## 4. Let Renovate bump the tag
|
||||
|
||||
Enable the Nix manager in the consuming repository's `renovate.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
|
||||
"extends": ["config:recommended"],
|
||||
"nix": {
|
||||
"enabled": true
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Renovate reads the flake inputs and the lock file, sees the `refs/tags/v0.0.16` ref, and opens a pull request when a newer tag exists. It proposes only tags that already exist, so it never pins a release that has not been cut.
|
||||
|
||||
Between the tag push and the asset upload there is a window in which the tag exists and the release tarballs do not. A pin written by hand during that window evaluates, then fails at build time with a 404 from the release download URL. Wait for the release to be published, or re-run the build once it is. Renovate's own pull requests are not affected by the window in practice, because it runs on a schedule rather than on the tag push.
|
||||
|
||||
## Upgrading
|
||||
|
||||
Change the `?ref=refs/tags/vX.Y.Z` in the input, run `nix flake lock --update-input nxdns`, and rebuild. Read the release notes first: nxdns is pre-0.1 and breaks on purpose, and [upgrade](upgrade.md) lists what state a version change touches.
|
||||
@@ -377,13 +377,13 @@ Requires Zig 0.16.0 and Node.js. From the repository root:
|
||||
```sh
|
||||
(cd admin && npm ci && npm run build)
|
||||
VERSION=$(sed -n 's/^[[:space:]]*\.version[[:space:]]*=[[:space:]]*"\([^"]*\)".*/\1/p' build.zig.zon)
|
||||
zig build dist -Dversion-string="$VERSION" -Dgit-commit="$(git rev-parse HEAD)" \
|
||||
zig build dist -Dversion-string="$VERSION" \
|
||||
-Dadmin-dist=admin/dist -Doptimize=ReleaseSafe
|
||||
```
|
||||
|
||||
The first command builds the admin interface into `admin/dist`; the last one embeds that directory in the binary. Build the interface every time, before the binary: a stale `admin/dist` ships an admin UI that does not match the API it talks to. `dist` refuses to run against the `admin/dist-placeholder` default for exactly that reason, so there is no way to skip it by accident.
|
||||
|
||||
`-Dversion-string` is required and has no default. It is what `nxdns version` prints. Take it from `build.zig.zon` rather than inventing one: `verify-dist` asserts that the version under build equals `.version` there, so a made-up string like `0.0.0-local` builds but then fails verification. `-Dgit-commit` is what distinguishes your build from the published one of the same version.
|
||||
`-Dversion-string` is required and has no default. It is what `nxdns version` prints. Take it from `build.zig.zon` rather than inventing one: `verify-dist` asserts that the version under build equals `.version` there, so a made-up string like `0.0.0-local` builds but then fails verification.
|
||||
|
||||
What comes out under `zig-out/dist/` is the same set a release publishes, minus the signature and the image digest:
|
||||
|
||||
@@ -397,7 +397,7 @@ The two targets are `x86_64-linux-musl` and `aarch64-linux-musl`. Both binaries
|
||||
Check the result the same way the release pipeline does:
|
||||
|
||||
```sh
|
||||
zig build verify-dist -Dversion-string="$VERSION" -Dgit-commit="$(git rev-parse HEAD)" \
|
||||
zig build verify-dist -Dversion-string="$VERSION" \
|
||||
-Dadmin-dist=admin/dist -Doptimize=ReleaseSafe
|
||||
```
|
||||
|
||||
|
||||
@@ -218,7 +218,7 @@ OK: no problems found
|
||||
>
|
||||
> ```
|
||||
> $ nxdns version
|
||||
> nxdns <version> (unknown)
|
||||
> nxdns <version>
|
||||
> zig 0.16.0
|
||||
> ```
|
||||
>
|
||||
@@ -338,13 +338,13 @@ If you are running something you built rather than a release, step 2 is a build
|
||||
```sh
|
||||
(cd admin && npm ci && npm run build)
|
||||
VERSION=$(sed -n 's/^[[:space:]]*\.version[[:space:]]*=[[:space:]]*"\([^"]*\)".*/\1/p' build.zig.zon)
|
||||
zig build dist -Dversion-string="$VERSION" -Dgit-commit="$(git rev-parse HEAD)" \
|
||||
zig build dist -Dversion-string="$VERSION" \
|
||||
-Dadmin-dist=admin/dist -Doptimize=ReleaseSafe
|
||||
```
|
||||
|
||||
Rebuild `admin/dist` before the binary on every upgrade. The admin interface is embedded at build time, and an old bundle against a new API is a broken System page. `dist` refuses the `admin/dist-placeholder` default outright, so the only way to ship a stale bundle is to leave an old `admin/dist` in place.
|
||||
|
||||
The staged payload for each target is under `zig-out/dist/stage/nxdns-<version>-<triple>/`, and step 3 continues from there with that path in place of the extracted one. The version string has to equal `.version` in `build.zig.zon` — `verify-dist` asserts it, so a made-up one builds and then fails verification. What tells your build apart from the published release of the same version is `-Dgit-commit`, which `nxdns version` prints beside the version.
|
||||
The staged payload for each target is under `zig-out/dist/stage/nxdns-<version>-<triple>/`, and step 3 continues from there with that path in place of the extracted one. The version string has to equal `.version` in `build.zig.zon` — `verify-dist` asserts it, so a made-up one builds and then fails verification. Your build reports the same two lines as the published release of that version, so compare the tarball hashes rather than the `nxdns version` output when you need to tell them apart.
|
||||
|
||||
Under Docker, build the image and name it instead of pulling:
|
||||
|
||||
|
||||
@@ -197,13 +197,11 @@ tar -xzf nxdns-$VERSION-x86_64-linux-musl.tar.gz
|
||||
./nxdns-$VERSION-x86_64-linux-musl/nxdns version
|
||||
```
|
||||
|
||||
`version` prints the version and the git commit it was built from, then the Zig version. The version has to match the tag you downloaded, and the commit has to match the commit the tag points at.
|
||||
`version` prints the version, then the Zig version the binary was built with. The version has to match the tag you downloaded. The binary carries no commit sha, so the check that ties a release to its source is the rebuild below rather than a string in this output.
|
||||
|
||||
> Verified against `v0.0.1`: both tarballs listed exactly the one directory and
|
||||
> six files with the stated modes, no symlinks and no absolute or `..` paths,
|
||||
> and the extracted binary printed `nxdns 0.0.1
|
||||
> (3c2d0d41f04570038e805b759da4541e198eae17)` — the commit `v0.0.1` points at —
|
||||
> then `zig 0.16.0`.
|
||||
> and the extracted binary printed `nxdns 0.0.1` then `zig 0.16.0`.
|
||||
|
||||
## 6. Verify the container image
|
||||
|
||||
@@ -262,22 +260,31 @@ It proves two things:
|
||||
|
||||
It does not prove that the binary in the tarball was built from the source in this repository. The machine that ran the build also held the signing key, so a compromise of that machine produces an artifact that is signed, verifies cleanly, and contains whatever the attacker put in it. The signature is a statement about origin and integrity in transit. It is not a statement about provenance from source.
|
||||
|
||||
Closing that gap needs a reproducibility gate — an independent build, run somewhere else, that lands on the same bytes — and this project does not have one. It is a recorded deferral, not an oversight: see `specs/milestone-14.md` ruling 12. Until it exists, nothing here claims the build is reproducible, because nobody has measured whether it is.
|
||||
Closing that gap needs a reproducible build — an independent build, run somewhere else, that lands on the same bytes. The release tarballs are reproducible: the binary carries no commit sha, the archive is written by the project's own tool with fixed modes, zero timestamps and sorted entries, and CI rebuilds the same bytes and compares them against the pinned hashes before it uploads anything. The rebuild below is how you check that for yourself.
|
||||
|
||||
The signing key is a subkey rather than the primary key, which limits the damage of the case above: a leaked release subkey is revoked on its own and the identity, the commit signatures and everyone's existing trust in the key survive.
|
||||
|
||||
## Rebuild it yourself
|
||||
|
||||
You can still build the same version from source and compare. That gets you a binary whose provenance you know, and the comparison is worth making — read the paragraph after the recipe before you draw a conclusion from it.
|
||||
Build the same version from source and compare the hashes. Match the toolchain first: `ZIG_VERSION`, `NODE_VERSION` and `NPM_VERSION` at the top of `.gitea/workflows/gates.yml` are the exact versions the release used, and a different patch release of any of them changes the bytes.
|
||||
|
||||
The host matters too: the admin bundle is built with host-native Rolldown and Lightning CSS bindings, which the lockfile ships per platform and libc, so the recipe reproduces the release only on x86_64 Linux with glibc, the runner CI uses. The two `npm_config_*` paths must not exist on your machine; the recipe relies on npm finding no config file there. The environment matters as much as the toolchain. The release cut and CI both build the bundle under exactly nine variables and a `022` umask, so the recipe below does the same: a locale, a time zone, a build timestamp or a file mode picked up from your shell each move the bytes. The Zig build gets a private `--cache-dir` so a stale local cache cannot leak into them; the global Zig cache stays shared, because it is content-addressed and a fresh one refetches every dependency (see `specs/release-cut.md`).
|
||||
|
||||
```sh
|
||||
git clone https://git.mial.net/mokhtar/nxdns
|
||||
cd nxdns
|
||||
git checkout "v$VERSION"
|
||||
git verify-tag "v$VERSION"
|
||||
(cd admin && npm ci && npm run build)
|
||||
zig build dist -Dversion-string="$VERSION" -Dgit-commit="$(git rev-parse HEAD)" \
|
||||
-Dadmin-dist=admin/dist -Doptimize=ReleaseSafe
|
||||
|
||||
env -i PATH="$PATH" HOME="$HOME" LC_ALL=C LANG=C TZ=UTC SOURCE_DATE_EPOCH=0 CI=true npm_config_userconfig=/nonexistent/npmrc-user npm_config_globalconfig=/nonexistent/npmrc-global \
|
||||
sh -c 'cd admin && umask 022 && npm ci && npm run build'
|
||||
|
||||
env -i PATH="$PATH" HOME="$HOME" LC_ALL=C LANG=C TZ=UTC SOURCE_DATE_EPOCH=0 CI=true npm_config_userconfig=/nonexistent/npmrc-user npm_config_globalconfig=/nonexistent/npmrc-global \
|
||||
sh -c 'umask 022 && exec zig build dist \
|
||||
-Dversion-string="'"$VERSION"'" \
|
||||
-Dadmin-dist=admin/dist -Doptimize=ReleaseSafe \
|
||||
--cache-dir "$(mktemp -d)"'
|
||||
|
||||
sha256sum zig-out/dist/nxdns-"$VERSION"-*.tar.gz
|
||||
```
|
||||
|
||||
@@ -285,20 +292,13 @@ sha256sum zig-out/dist/nxdns-"$VERSION"-*.tar.gz
|
||||
|
||||
`zig build dist` writes `zig-out/dist/`: the two tarballs, a staging directory per target under `stage/`, the stripped binaries under `bin/<triple>/`, and a `SHA256SUMS` covering the two tarballs. The published `SHA256SUMS.txt` is that file with a third line for `IMAGE-DIGEST.txt` appended by the release job, so the two tarball lines should match and the local file has no third line to compare.
|
||||
|
||||
Now the caveat, and it is the whole reason this section is last. **A hash that differs does not mean the release was tampered with.** Nothing in this project measures whether two builds of the same commit on two different machines produce the same bytes, and there are several ordinary reasons they would not: a different Zig patch release, a different Node version, a different path to the build directory, a different npm lockfile resolution. A hash that matches is real evidence. A hash that does not match tells you only that something about the two builds differed, and finding out what is on you.
|
||||
The tarball is written by `zig build dist` itself rather than by the host's `tar` and `gzip`, so its layout is fixed and a rebuild on a matching toolchain reproduces it byte for byte. Entries are sorted by their full path as bytes, with the payload directory first. The payload directory and `nxdns` carry mode `0755`; every other file carries `0644`. Every entry has a zero modification time, uid 0, gid 0, and no user or group name. The gzip wrapper carries no original filename and a zero header timestamp, which is what makes two archives of the same tree compare equal.
|
||||
|
||||
If you want the comparison to mean as much as it can, match the toolchain the release used. The Zig version is the second line of `nxdns version`, and both it and the Node version are pinned to exact patch releases at the top of `.gitea/workflows/gates.yml`, which is the workflow the release runs.
|
||||
Compare your two tarball hashes against two things: the published `SHA256SUMS.txt`, and the `hashes` block of `flake.nix` at the tag, which carries the same digests in SRI form. All three agree on a matching toolchain, and the release pipeline fails before it uploads anything if they do not.
|
||||
|
||||
> Verified against `v0.0.1`, and the result is the caveat above in action. The
|
||||
> whole recipe ran from a fresh clone: `git verify-tag v0.0.1` printed
|
||||
> `Good signature` under the same signing subkey as the release, and
|
||||
> `zig build dist` produced both tarballs. The hashes did **not** match the
|
||||
> published `SHA256SUMS.txt` — the binaries themselves already differ. The Zig
|
||||
> version matched the pin exactly; the Node version did not (24.14.1 against
|
||||
> the pinned 24.19.0) and the build path differed, two of the ordinary causes
|
||||
> listed above. That is a measurement of what an unpinned rebuild gives you,
|
||||
> not evidence of tampering: the signature, checksum and image checks earlier
|
||||
> on this page all passed against the same release.
|
||||
A hash that differs is a signal to check the toolchain and the environment first. An unpinned Zig, Node or npm version is the ordinary explanation, and a build run outside the normalized environment above is the next one. Rule both out before you conclude anything about the release itself.
|
||||
|
||||
> Verified against `v0.0.1`, before the build was reproducible: the recipe ran from a fresh clone, `git verify-tag v0.0.1` printed `Good signature` under the release subkey, and the rebuilt tarball hashes did not match the published `SHA256SUMS.txt` (Node 24.14.1 against the pinned 24.19.0, a different build path, and an archive written by the host's `tar`). Releases from 0.0.17 on are built and checked by the pinned pipeline this page describes, and the cut records the local hashes in `flake.nix` before CI rebuilds them.
|
||||
|
||||
## If a check fails
|
||||
|
||||
|
||||
@@ -202,7 +202,12 @@ If a restart and an import race for the write lock, one of them simply wins: bot
|
||||
|
||||
## `version`
|
||||
|
||||
Prints two lines: the nxdns version with the git commit, then the Zig version the binary was built with. Takes no flags and no arguments.
|
||||
Prints two lines: the nxdns version, then the Zig version the binary was built with. Takes no flags and no arguments.
|
||||
|
||||
```
|
||||
nxdns <version>
|
||||
zig 0.16.0
|
||||
```
|
||||
|
||||
## `help`
|
||||
|
||||
|
||||
Reference in New Issue
Block a user