milestone 14: build, package, sign and publish releases

This commit is contained in:
2026-08-08 12:38:29 +02:00
parent 6c507992e4
commit cdacc560b7
48 changed files with 7272 additions and 543 deletions
+468
View File
@@ -0,0 +1,468 @@
# Verify a release
Checks that a downloaded nxdns release is the one the project published and
that it arrived intact. It also gives the recipe for rebuilding the same
version from source, and says plainly what that does and does not settle.
Do this before you run the binary, not after. The whole point of the checksum
file is that it is signed, so a tampered mirror cannot hand you a matching
tarball and a matching checksum at the same time.
> Verification: no nxdns release exists yet. The repository has no tags, no
> release page and no pushed image, so nothing on this page could be run against
> a real release asset and no command here was pointed at
> `git.mial.net/mokhtar/nxdns` with any expectation of success. Substitutes were
> used, and every block says which one applies to it.
>
> The URL shapes were probed against `gitea.com`, a public instance of the same
> Gitea series running `1.27.0+dev-652-g0571722545`, using `gitea/tea`, which
> does have releases. `git.mial.net` reports `1.27.1`, and its
> `/mokhtar/nxdns/releases/latest` answers 404 — no release to redirect to. On
> `gitea/tea`, `releases/latest` answered 303 to the tag page of `v0.15.1`;
> `releases/download/v0.15.1/checksums.txt` and
> `releases/download/latest/checksums.txt` both answered 303 to the same stored
> object and delivered the same 1,842-byte file under `-L`;
> `releases/latest/download/checksums.txt` — GitHub's spelling — answered 404.
>
> The `gpg --verify` and `sha256sum -c` blocks were run on this host against
> stand-in files: two random-byte files named like the release tarballs, an
> `IMAGE-DIGEST.txt` holding one image reference, and a `SHA256SUMS.txt`
> computed over the three, signed by a **throwaway demonstration key generated
> for this page**. That key has the shape the real one will have — an ed25519
> primary key plus a separate ed25519 signing subkey, with the signature made by
> the subkey — so the `gpg --verify` output on this page has the two-fingerprint
> structure a subkey-signed release produces. The fingerprints printed in those
> transcripts are the throwaway key's, they are not the project's, and they will
> not match anything you download. The only edit to that run's output is the
> version in every filename, which became `<version>`.
>
> The container blocks were not run against nxdns — there is no published image.
> The two `docker buildx imagetools inspect --format` shapes were run here
> against `alpine:3.22` on Docker Hub, the base this project's builder stage
> pins; the digest form printed
> `sha256:14358309a308569c32bdc37e2e0e9694be33a9d99e68afb0f5ff33cc1f695dce` and
> the platform form printed a list. The `docker create`/`docker cp` comparison
> was run against an image built from this checkout rather than a pulled one.
## What a release contains
Five assets, on the release page at
`https://git.mial.net/mokhtar/nxdns/releases`:
| Asset | What it is |
| --- | --- |
| `nxdns-<version>-x86_64-linux-musl.tar.gz` | The x86_64 tarball |
| `nxdns-<version>-aarch64-linux-musl.tar.gz` | The aarch64 tarball, for a Raspberry Pi 5 |
| `SHA256SUMS.txt` | One `sha256sum` line each for the two tarballs and for `IMAGE-DIGEST.txt` |
| `SHA256SUMS.txt.asc` | A detached OpenPGP signature over `SHA256SUMS.txt` |
| `IMAGE-DIGEST.txt` | The container image reference this version pushed, pinned by digest |
Each tarball holds one top-level directory, `nxdns-<version>-<triple>/`, with
six files in it: the `nxdns` binary at mode 0755, and `nxdns.service`,
`nxdns.conf`, `LICENSE`, `THIRD-PARTY-NOTICES` and `INSTALL.md` at 0644.
`SHA256SUMS.txt` covers `IMAGE-DIGEST.txt` rather than the image, because the
image digest does not exist until the push has happened and cannot be computed
by the build. Signing the file that names the digest gets you the same
guarantee in one signature.
The `.txt` on three of the five names is not decoration. Gitea decides what an
attachment may be by its file extension, and whether it accepts an
extensionless upload at all is untested against this instance, so the release
uses names it is known to accept. On disk, `zig build dist` still writes a file
called `SHA256SUMS`; the release job copies it to `SHA256SUMS.txt` and appends
the image line before signing.
## 1. Pick a version
Every URL below takes the version from one shell variable. Ask the server
rather than typing a number that goes stale: Gitea redirects `releases/latest`
to the tag page of the newest published release — newest by publication time,
and drafts and pre-releases are excluded.
```sh
BASE=https://git.mial.net/mokhtar/nxdns
VERSION=$(curl -fsS -o /dev/null -w '%{redirect_url}' "$BASE/releases/latest" |
sed 's#.*/releases/tag/v##')
echo "$VERSION"
```
To take a particular version instead, set it yourself — substitute the one you
want for the placeholder:
```sh
VERSION=<version>
```
> Not verified against nxdns: there is no release to redirect to, so the first
> block prints an empty line here and every URL built from it is a 404. The
> exact two-command form was run against `gitea.com/gitea/tea`, a public
> repository on Gitea `1.27.0+dev` that does have releases, and printed
> `0.15.1`.
Pin the version in anything you script or automate. `latest` is convenient for
a person at a terminal and a liability in a machine that upgrades itself.
## 2. Download
```sh
mkdir -p ~/nxdns-release && cd ~/nxdns-release
curl -fLO "$BASE/releases/download/v$VERSION/nxdns-$VERSION-x86_64-linux-musl.tar.gz"
curl -fLO "$BASE/releases/download/v$VERSION/SHA256SUMS.txt"
curl -fLO "$BASE/releases/download/v$VERSION/SHA256SUMS.txt.asc"
curl -fLO "$BASE/releases/download/v$VERSION/IMAGE-DIGEST.txt"
```
`-L` is not optional: Gitea answers an asset URL with a 303 to wherever the
attachment is actually stored.
For the aarch64 tarball, or for both, swap or add the filename:
```sh
curl -fLO "$BASE/releases/download/v$VERSION/nxdns-$VERSION-aarch64-linux-musl.tar.gz"
```
Gitea also accepts the literal word `latest` in place of the tag, so the three
assets whose names carry no version can be fetched without one:
```sh
curl -fLO "$BASE/releases/download/latest/SHA256SUMS.txt"
curl -fLO "$BASE/releases/download/latest/SHA256SUMS.txt.asc"
curl -fLO "$BASE/releases/download/latest/IMAGE-DIGEST.txt"
```
That is the Gitea spelling, and it is not GitHub's. `releases/latest/download/`
— the form GitHub uses — is a 404 on Gitea; the alias goes in the tag
position, as `releases/download/latest/`. The tarball filenames contain the
version, so this alias never saves you from knowing it for those two.
> Not verified against nxdns: no release, so every URL above is a 404 today.
> Both URL forms, including the 404 for GitHub's spelling, were exercised
> against `gitea.com/gitea/tea` on Gitea `1.27.0+dev`; the versioned path and
> the `latest` alias each answered 303 to the same stored object and delivered
> the same 1,842-byte `checksums.txt` when the redirect was followed.
## 3. Check the signature over `SHA256SUMS.txt`
Get the public key first. It is a signing subkey of the key that signs every
commit in this repository, so you can confirm the fingerprint against a clone
you already have with `git log --show-signature` or `git verify-tag v$VERSION`:
```
A2061F6AB24DF2C0E92346FD1509B54946D08A95
```
```sh
curl -fsSL https://keys.openpgp.org/vks/v1/by-fingerprint/A2061F6AB24DF2C0E92346FD1509B54946D08A95 |
gpg --import
```
> Not verified: the key is not published yet. Run on this host, that URL
> returned 404, and so did the `by-email` lookup for the same address. The
> endpoint itself is live: the same `by-fingerprint` path returned 200 for an
> unrelated key that is on keys.openpgp.org. Until this key is published there,
> get it from a source you can check some other way.
Then verify:
```sh
gpg --verify SHA256SUMS.txt.asc SHA256SUMS.txt
```
```
gpg: Signature made Fri 07 Aug 2026 10:11:12 PM CEST
gpg: using EDDSA key 9D1EA241DAEA89E09381A21BDC27E8A3D53C32D6
gpg: Good signature from "nxdns release signing (throwaway demonstration key) <demo@example.invalid>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 6643 13AA F527 DDAE 1C1E 516C A36F F8DA 4E6C 1C07
Subkey fingerprint: 9D1E A241 DAEA 89E0 9381 A21B DC27 E8A3 D53C 32D6
```
**Those two fingerprints and that user id belong to a throwaway key generated
to produce this transcript.** They are not the project's, and what you see will
carry the project's uid and the fingerprint in this page instead. The
*structure* is what to read: three lines, not one. `using EDDSA key` and
`Subkey fingerprint` name the signing subkey that actually made the signature;
`Primary key fingerprint` names the certificate it hangs off, and that is the
one published above.
Exit status 0, and `Good signature`. That warning is normal and is not a
failure: it says you have not told GnuPG you believe the key belongs to the
person it claims to.
Now compare the `Primary key fingerprint` line with the fingerprint in this
page. GnuPG prints it as ten space-separated groups of four hex digits, with a
double space in the middle, while the fingerprint above is the same 40
characters unspaced — so compare the hex digits in order and ignore the
spacing, or strip it and let the shell do it:
```sh
gpg --verify SHA256SUMS.txt.asc SHA256SUMS.txt 2>&1 |
sed -n 's/^Primary key fingerprint: //p' | tr -d ' '
```
That prints the 40-character form, ready to compare with
`A2061F6AB24DF2C0E92346FD1509B54946D08A95`. Do not skip the comparison —
`gpg --verify` exits 0 for a good signature from *any* key in your keyring,
including one an attacker talked you into importing.
A tampered `SHA256SUMS.txt` looks like this, and exits 1:
```
gpg: Signature made Fri 07 Aug 2026 10:11:12 PM CEST
gpg: using EDDSA key 9D1EA241DAEA89E09381A21BDC27E8A3D53C32D6
gpg: BAD signature from "nxdns release signing (throwaway demonstration key) <demo@example.invalid>" [unknown]
```
> Verified on this host. A throwaway ed25519 primary key was generated into a
> temporary `GNUPGHOME`, an ed25519 **signing subkey** was added to it, and the
> stand-in `SHA256SUMS.txt` was signed with `--local-user <subkey-fingerprint>!`
> — the same construction the release workflow uses — so the transcripts above
> are what a subkey-signed release actually prints, rather than what a key
> signing with its primary would. The verification ran from a second
> `GNUPGHOME` holding only that key's public half, which is why the `[unknown]`
> trust marker and the warning are there rather than being written in by hand.
> The second transcript is the same command after one newline was appended to
> `SHA256SUMS.txt`. The `sed`/`tr` pipeline was run against that same output and
> printed `664313AAF527DDAE1C1E516CA36FF8DA4E6C1C07`, the throwaway primary.
## 4. Check the hashes
```sh
sha256sum -c --ignore-missing SHA256SUMS.txt
```
```
nxdns-<version>-x86_64-linux-musl.tar.gz: OK
IMAGE-DIGEST.txt: OK
```
`--ignore-missing` is what makes this work when you downloaded one tarball out
of the two. Without it, `sha256sum` treats every line it cannot read as a
failure and exits 1:
```
nxdns-<version>-x86_64-linux-musl.tar.gz: OK
sha256sum: nxdns-<version>-aarch64-linux-musl.tar.gz: No such file or directory
nxdns-<version>-aarch64-linux-musl.tar.gz: FAILED open or read
IMAGE-DIGEST.txt: OK
sha256sum: WARNING: 1 listed file could not be read
```
A file that is present but does not match is the case that matters, and it
says `FAILED` with no `open or read`:
```
nxdns-<version>-x86_64-linux-musl.tar.gz: FAILED
IMAGE-DIGEST.txt: OK
sha256sum: WARNING: 1 computed checksum did NOT match
```
Check the signature before the hashes, not after. An attacker who can replace
the tarball can replace `SHA256SUMS.txt` next to it; the signature is the only
thing in the set they cannot forge.
> Verified on this host against the stand-in files: all three transcripts are
> real `sha256sum` output over two random-byte files named like the release
> tarballs plus an `IMAGE-DIGEST.txt` holding one image reference, with one
> tarball deleted for the first two blocks and one byte appended to the other
> for the third. Only the version in the filenames was replaced with
> `<version>`.
## 5. Look inside before extracting
```sh
tar -tvzf nxdns-$VERSION-x86_64-linux-musl.tar.gz
```
Expect exactly one top-level directory and the six files listed above, with
mode `-rwxr-xr-x` on `nxdns` and `-rw-r--r--` on the rest, no symlinks, and no
path that begins with `/` or contains `..`. `zig build verify-dist` asserts all
of that on the extracted archive before a release is ever published, so this is
a second opinion rather than the only check — but it costs nothing and it is
the step that catches a tarball that is not the one you think it is.
Then extract:
```sh
tar -xzf nxdns-$VERSION-x86_64-linux-musl.tar.gz
./nxdns-$VERSION-x86_64-linux-musl/nxdns version
```
`version` prints the version and the git commit it was built from, then the
Zig version. The version has to match the tag you downloaded, and the commit
has to match the commit the tag points at.
> Not verified on this host: there is no release tarball to list or extract.
## 6. Verify the container image
`IMAGE-DIGEST.txt` holds one line: the image reference this version pushed,
pinned by the digest of its index, in the form
```
git.mial.net/mokhtar/nxdns:<version>@sha256:<64 hex digits>
```
`SHA256SUMS.txt` covers `IMAGE-DIGEST.txt`, so the signature you already
checked covers that line too. Confirm the tag in the registry still resolves to
that digest:
```sh
cut -d@ -f2 IMAGE-DIGEST.txt
docker buildx imagetools inspect git.mial.net/mokhtar/nxdns:$VERSION \
--format '{{.Manifest.Digest}}'
```
The two have to be the same string. A registry tag is mutable; the digest is
not, so pull the whole pinned reference rather than the tag when you care:
```sh
docker pull "$(cat IMAGE-DIGEST.txt)"
```
The index should carry exactly two platforms and nothing else:
```sh
docker buildx imagetools inspect git.mial.net/mokhtar/nxdns:$VERSION \
--format '{{range .Manifest.Manifests}}{{.Platform.OS}}/{{.Platform.Architecture}} {{end}}'
```
`linux/amd64 linux/arm64`. The build passes `--provenance=false --sbom=false`,
so there are no `unknown/unknown` attestation entries in the list; seeing any
means the image did not come from this pipeline.
The binary inside the image is the same file as the one in the matching
tarball, and the release checks that before publishing. To check it yourself:
```sh
docker create --name nxdns-verify git.mial.net/mokhtar/nxdns:$VERSION
docker cp nxdns-verify:/nxdns ./nxdns-from-image
docker rm nxdns-verify
sha256sum ./nxdns-from-image ./nxdns-$VERSION-x86_64-linux-musl/nxdns
```
> Not verified against nxdns: no image is published, so no command here was run
> against `git.mial.net/mokhtar/nxdns`. The two
> `docker buildx imagetools inspect --format` shapes were run on this host
> against `alpine:3.22` on Docker Hub — the digest form printed
> `sha256:14358309a308569c32bdc37e2e0e9694be33a9d99e68afb0f5ff33cc1f695dce`,
> which is the digest this project's builder stage pins, and the platform form
> printed `linux/amd64 unknown/unknown linux/arm unknown/unknown ...`. That
> `unknown/unknown` is exactly what the paragraph above says nxdns's own index
> must not contain: Alpine's index carries attestation entries, and nxdns's
> build turns them off. Nothing was checked about how nxdns's index will
> actually look.
>
> The `docker create` / `docker cp` / `sha256sum` comparison at the end was run
> here against an image built from this checkout rather than a pulled one, and
> the two hashes matched: the binary copied out of the image and
> `zig-out/dist/stage/nxdns-<version>-x86_64-linux-musl/nxdns` were the same
> file.
## What the signature proves, and what it does not
It proves two things:
- The release was produced by this project's release pipeline, using a key
only that pipeline holds.
- What you have on disk is byte for byte what that pipeline uploaded. A
corrupted download, a modified mirror or a tampered proxy all break the
check.
It does not prove that the binary in the tarball was built from the source in
this repository. The machine that ran the build also held the signing key, so
a compromise of that machine produces an artifact that is signed, verifies
cleanly, and contains whatever the attacker put in it. The signature is a
statement about origin and integrity in transit. It is not a statement about
provenance from source.
Closing that gap needs a reproducibility gate — an independent build, run
somewhere else, that lands on the same bytes — and this project does not have
one. It is a recorded deferral, not an oversight: see `specs/milestone-14.md`
ruling 12. Until it exists, nothing here claims the build is reproducible,
because nobody has measured whether it is.
The signing key is a subkey rather than the primary key, which limits the
damage of the case above: a leaked release subkey is revoked on its own and
the identity, the commit signatures and everyone's existing trust in the key
survive.
## Rebuild it yourself
You can still build the same version from source and compare. That gets you a
binary whose provenance you know, and the comparison is worth making — read
the paragraph after the recipe before you draw a conclusion from it.
```sh
git clone https://git.mial.net/mokhtar/nxdns
cd nxdns
git checkout "v$VERSION"
git verify-tag "v$VERSION"
(cd web && npm ci && npm run build)
zig build dist -Dversion-string="$VERSION" -Dgit-commit="$(git rev-parse HEAD)" \
-Dweb-dist=web/dist -Doptimize=ReleaseSafe
sha256sum zig-out/dist/nxdns-"$VERSION"-*.tar.gz
```
`git verify-tag` is the check that the tag itself is signed by the key from
step 3, and it is the one part of this section that stands on its own: it ties
the source you just checked out to the same identity that signed the release.
`zig build dist` writes `zig-out/dist/`: the two tarballs, a staging directory
per target under `stage/`, the stripped binaries under `bin/<triple>/`, and a
`SHA256SUMS` covering the two tarballs. The published `SHA256SUMS.txt` is that
file with a third line for `IMAGE-DIGEST.txt` appended by the release job, so
the two tarball lines should match and the local file has no third line to
compare.
Now the caveat, and it is the whole reason this section is last. **A hash that
differs does not mean the release was tampered with.** Nothing in this project
measures whether two builds of the same commit on two different machines
produce the same bytes, and there are several ordinary reasons they would not:
a different Zig patch release, a different Node version, a different path to
the build directory, a different npm lockfile resolution. A hash that matches
is real evidence. A hash that does not match tells you only that something
about the two builds differed, and finding out what is on you.
If you want the comparison to mean as much as it can, match the toolchain the
release used. The Zig version is the second line of `nxdns version`, and both
it and the Node version are pinned to exact patch releases at the top of
`.gitea/workflows/gates.yml`, which is the workflow the release runs.
> Partly verified on this host. `zig build dist` and `sha256sum` on its output
> were run to completion, with the version read out of `build.zig.zon`: `dist`
> exited 0 and wrote the two tarballs, `SHA256SUMS` and the staged payloads
> described above. `zig build verify-dist` was run on the result too and exited
> 0. What could not be run is everything that needs a release: the clone, the
> checkout and `git verify-tag` need a tag that does not exist, and there is no
> published `SHA256SUMS.txt` to compare a local build against, so the comparison
> this section is about has never been performed.
## If a check fails
Stop and do not run the binary.
- `sha256sum` says `FAILED` but the signature was good — you have a damaged or
substituted download. Delete it and fetch it again over a different network
before assuming anything worse.
- `gpg` says `BAD signature``SHA256SUMS.txt` and `SHA256SUMS.txt.asc` do not
belong together. Re-download both from the release page; a stale
`SHA256SUMS.txt.asc` left over from a previous version is the boring
explanation.
- `gpg` says `Can't check signature: No public key` — you have not imported the
key, or you imported a different one.
- The fingerprint does not match the one in step 3 — that is the case to take
seriously. Do not extract the tarball, and do not import more keys trying to
make it pass.
## Related
- [Install with systemd](install-with-systemd.md) — where the verified tarball
goes next.
- [Install with Docker](install-with-docker.md) — the published image.
- [Upgrade nxdns](upgrade.md) — the same verification, on the way to a newer
version.
- [Performance targets and what the tests prove](../explanation/performance-and-testing.md)
— the other place this project writes down what its checks do not cover.