nxdns/specs/milestone-14.md

# Milestone 14: build, package and publish releases

Goal: turn a signed git tag into a published, verifiable release — two static
musl tarballs and one multi-architecture container image on the self-hosted
Gitea at `git.mial.net`, with a licence, third-party notices, a changelog, a
detached signature, and documentation that leads with download-and-verify.

First tag: `v0.0.1`.

## Rulings (binding)

### 1. PLAN.md is amended first

`PLAN.md:41` lists "Prebuilt binaries / published Docker images / project
website" under §2.2 *Out of Scope (permanent scope decisions, not deferrals)*.
`AGENTS.md` makes PLAN the source of truth, so this milestone is invalid until
that line is amended. The orchestrator edits PLAN before any session starts:

- §2.2: remove prebuilt binaries and published container images. **The project
  website stays out of scope.**
- Add a §on publication: tag-triggered releases, the artifact set, the signing
  model, and the two deferrals in ruling 12.
- `PLAN.md:571`: drop "build date" from the `nxdns version` line. The version
  string and the git commit identify a build exactly, and a date is one more
  input that a reproducible build would have to pin. `src/version.zig` is
  already correct; PLAN was wrong.
- `PLAN.md:638`: state exact byte limits — 15,728,640 with embedded assets,
  10,485,760 without — resolving milestone-13 discrepancy 9 in favour of what
  CI already asserts.

### 2. The version lives in the tag, and in exactly one other place

The tag is authoritative. `v0.0.1` means `-Dversion-string=0.0.1`;
`-Dgit-commit` is the tag's peeled commit.

`build.zig.zon:3` holds `.version`, which Zig requires and nothing reads. It is
bumped in the commit before each tag, and `verify-dist` fails when it disagrees
with the version under build. Two strings, one assert, no third copy anywhere.

Tags are **never moved**. A tag that produced a bad release is abandoned; the
fix ships as the next patch version. Only `vMAJOR.MINOR.PATCH` is accepted —
the release workflow rejects any tag carrying a pre-release suffix.

### 3. Licence and third-party notices

`LICENSE` holds the English text of EUPL-1.2 verbatim, with
`Copyright (c) 2026 Mokhtar Mial`. `README.md` gains a short notice naming the
licence and the SPDX identifier `EUPL-1.2`. **No per-file SPDX headers.**

`THIRD-PARTY-NOTICES` is assembled by `zig build dist` from a committed,
reviewed inventory under `licenses/`. It is not scraped from the dependency
tree at build time: a generated notices file that nobody reads rots silently
into a false statement.

The inventory must cover everything the shipped artifacts actually contain, not
the direct dependency list:

- **musl libc** (MIT) — statically linked into every binary.
- **Zig standard library and compiler-rt** (MIT) — likewise.
- **SQLite 3.53.4** — public domain, no obligation, listed for completeness.
- **Mbed TLS 3.6.7** — with an explicit line recording that it is taken under
  the Apache-2.0 option of its dual `Apache-2.0 OR GPL-2.0-or-later` licence,
  followed by the **full Apache-2.0 text**. Naming the choice is good practice;
  shipping the text is the actual obligation.
- **Project Everest and p256-m** — compiled in at `build.zig:394` even though
  the stock config leaves both drivers disabled.
- **The web bundle's runtime closure** — the transitive set, not the four direct
  entries in `web/package.json`. Tailwind is a devDependency whose generated CSS
  ships, so "production dependencies" understates it.

A CI guard records the identity of the dependency sets (`build.zig.zon`
dependencies, and the npm closure) and fails when either changes without a
matching change under `licenses/`. It detects drift; it does not derive the
inventory.

The **container image carries `/LICENSE` and `/THIRD-PARTY-NOTICES` too**.
Distributing the image is distribution, and the obligations do not live in the
tarball.

### 4. `zig build dist` replaces `zig build cross`

The `cross` step is deleted. `dist` is the single command that produces
everything releasable, and it runs on a laptop exactly as it runs on the runner.

Inputs: `-Dversion-string` (required — no default), `-Dgit-commit`,
`-Dweb-dist`, `-Doptimize=ReleaseSafe`.

**`dist` fails when `-Dweb-dist` resolves to `web/dist-placeholder`.** The
default at `build.zig:24` is the placeholder, so a release built without the
flag would silently ship a placeholder admin page. There is no override flag;
an escape hatch here is a foot-gun with a safety label on it.

Per triple (`x86_64-linux-musl`, `aarch64-linux-musl`), `dist` builds
ReleaseSafe with `.linkage = .static` and `.strip = true`. Stripping uses
`std.Build.Module.strip` (`-fstrip`, `Module.zig:545`), which removes the
`objcopy` and `binutils-aarch64-linux-gnu` dependency from the runner.

It stages one directory per triple, `nxdns-<version>-<triple>/`, holding:

| File | Mode |
|---|---|
| `nxdns` | 0755 |
| `nxdns.service` | 0644 |
| `nxdns.conf` | 0644 |
| `LICENSE` | 0644 |
| `THIRD-PARTY-NOTICES` | 0644 |
| `INSTALL.md` | 0644 |

`deploy/systemd/sysusers.conf` is renamed to `nxdns.conf` — the name it is
installed under at `/usr/lib/sysusers.d/nxdns.conf`. A file that changes name
during install is a step an operator can get wrong.

Archiving runs as two `b.addSystemCommand` steps, never one:

```
tar --format=gnu --sort=name --mtime=@0 --owner=0 --group=0 \
    --numeric-owner -cf <name>.tar <dir>
gzip -n -9 <name>.tar
```

`b.addSystemCommand` executes argv directly and does not interpret `|`, and a
shell wrapper without `pipefail` would report only `gzip`'s exit status while a
failed `tar` passed silently. `gzip -n` is required because `--mtime=@0`
normalises the tar member times but not the timestamp gzip writes into its own
header. The environment sets `LC_ALL=C` and `TZ=UTC`.

Both steps declare the staging directory as a build input and the archive as an
output, so Zig's cache cannot serve a stale artifact.

`dist` also emits a checksum file covering **only the two tarballs**. It cannot
cover the image: the digest does not exist until buildx has pushed, which
happens later and elsewhere. The release job appends that line (ruling 7).

### 5. `zig build verify-dist` asserts what CI shell asserts today

The 50 lines of shell at `ci.yml:129` and `:168` move into a build step, so a
developer can run the release checks on a laptop. Logic that only runs in CI is
the brittleness this milestone exists to remove.

It operates on the **extracted** archive, not the staging directory:

- ELF header: correct `e_machine` per triple, **no `PT_INTERP`, no `DT_NEEDED`**.
  Matching the string `statically linked` from `file(1)` is not a static-linkage
  test.
- Stripped binary ≤ 15,728,640 bytes.
- Archive layout: exactly one top-level directory, the exact file allowlist from
  ruling 4, expected modes, no symlinks, no path traversal.
- `nxdns version` prints the version under build and the git commit. Native
  architecture only — the aarch64 binary needs qemu and is skipped without
  `-fqemu`.
- `build.zig.zon` `.version` equals the version under build.

The asset-free budget (10,485,760 bytes) gets **its own build against a
generated empty assets directory**, not against the placeholder. Ruling 4 makes
the placeholder unbuildable, and re-admitting it through a back door for one
size check would defeat the point.

**A second `zig build` invocation does not inherit the first one's `-D` options.**
`zig build dist -Dversion-string=X` followed by a bare `zig build verify-dist`
verifies a *differently configured* build. Every caller — the workflows, the
documentation, and this spec's own acceptance list — passes the same
`-Dversion-string`, `-Dgit-commit`, `-Dweb-dist` and `-Doptimize` to both.

### 6. Container image

`deploy/docker/Dockerfile`:

- Pin the base by digest:
  `alpine:3.22@sha256:14358309a308569c32bdc37e2e0e9694be33a9d99e68afb0f5ff33cc1f695dce`.
- Add `--platform=$BUILDPLATFORM` to the builder stage. It only copies files, so
  pinning it to the build host means the arm64 image needs no qemu.
- **Delete `apk add --no-cache ca-certificates` (`Dockerfile:15`).** Verified:
  the Alpine base already ships `/etc/ssl/certs/ca-certificates.crt` (179,359
  bytes) from `ca-certificates-bundle`. Removing it removes the only network
  fetch in the image build.
- Keep the `mkdir` and `chown 65532` — `/var/lib/nxdns` must exist with that
  ownership so Docker copies it onto a fresh named volume.
- Add `/LICENSE` and `/THIRD-PARTY-NOTICES` (ruling 3).
- OCI labels: `source`, `revision`, `version`, `licenses=EUPL-1.2`, `created`,
  `title`, `description`.

`deploy/docker/.dockerignore` is renamed to `deploy/docker/Dockerfile.dockerignore`.
The build context is the repository root, so Docker never reads the current
path, and the whole worktree — `.git`, `node_modules`, `zig-cache` — is being
sent to the daemon today.

buildx runs with `--provenance=false --sbom=false`. Recent buildx adds
provenance attestations by default, which create `unknown/unknown` platform
entries and change the index digest; Gitea's OCI 1.1 support is unverified
(go-gitea#25846) and this is not the milestone to find out. `SOURCE_DATE_EPOCH`
comes from the tag date.

`deploy/docker/compose.yaml` references the published image rather than
building one.

**Verify before publishing:** the binary inside each image is byte-identical to
the binary in the matching tarball.

### 7. Workflows

`.gitea/workflows/gates.yml`, `on: workflow_call`, holds every blocking check:
`test` (with `-Dintegration`), `test-aarch64` (`-fqemu`), `frontend`, `package`
(`dist` + `verify-dist` + the asset-free size check) and `container` (build the
image and run the existing smoke test from `ci.yml:219`). Moving only the three
test jobs would drop the packaging and container checks precisely when they
matter most.

`ci.yml` calls it on `push` and `pull_request` for **`master`**. `master` is
canonical: `origin/main` is deleted, and Gitea's default branch is changed to
match. Today `ci.yml:5` watches `main` while work happens on `master`, so
milestone 13 has never run through CI.

`.gitea/workflows/release.yml`, `on: push: tags: ['v*']`, in this order:

1. Checkout with `fetch-depth: 0` and tags. The default shallow clone breaks
   ancestry checks, previous-tag lookup and changelog generation.
2. Reject any tag that is not exactly `vMAJOR.MINOR.PATCH`.
3. `git verify-tag`, requiring an annotated tag and requiring the signature's
   fingerprint to equal a fingerprint pinned in the workflow. A bare
   `verify-tag` proves only that *some* imported key signed it.
4. Assert the tag's commit is an ancestor of `origin/master`.
5. Assert no published release exists for this tag. Delete any leftover draft.
6. Assert the version is greater than the highest published release version, so
   a late-finishing older tag cannot move `latest` backwards.
7. Gates, blocking, via `needs:`.
8. Build the web UI; `zig build dist`; `zig build verify-dist`.
9. Extract the `CHANGELOG.md` section matching the version. **Fail when absent.**
10. buildx build and push **`:<version>` only**. Capture the index digest from
    `--metadata-file`, validate it as `sha256:<64 hex>`, and confirm the pushed
    tag resolves to that digest with exactly the two intended platforms.
11. Write the digest file, append its checksum line, verify the assembled
    checksum file.
12. Sign the checksum file (ruling 8). Verify the signature locally before
    uploading it.
13. Create the release as a **draft**; upload the assets.
14. Move `:latest` and verify it resolves to the built digest. Re-check the
    monotonic-version invariant here: step 6 ran before the gates, and proves
    nothing about which of two in-flight tags finishes last.
15. **Publish the draft last.** Publication is the one irreversible act, so it
    goes after everything that can still fail.

Amended after review. The original order published at 14 and moved `:latest` at
15, which deadlocks: a failure while moving `:latest` leaves a published release,
and ruling 9 makes a re-run refuse a published release. Nothing could repair it.
The cost of the corrected order is a short window where `:latest` serves the new
image before the release page is public. That is recoverable by a re-run; the
deadlock was not.

Every action in `release.yml` is pinned to a full commit SHA.
`actions/checkout@v4` and `mlugg/setup-zig@v2` are mutable tags on another
party's server, and a compromise upstream would run on the runner holding the
signing subkey and the registry token. `ci.yml` may keep moving tags; it holds
no secrets.

### 8. Signing

**Two different keys are involved, and the original ruling conflated them.**

- The **tag-signing key** is the human's. `git tag -s` uses it, and step 3 checks
  it. What gets pinned in `release.yml` is the **primary certificate
  fingerprint**, which `git verify-tag --raw` emits as the **last** field of the
  `VALIDSIG` line. Field 3 is whichever key actually made the signature — the
  signing subkey once one exists. Pinning field 3 would mean that creating the
  release subkey below silently blocks every future release, with an error
  message that reads like a forged tag. Reproduced against a real keyring during
  review.
- The **artifact-signing subkey** is the runner's, and it signs the checksum
  file. It is a dedicated GPG signing subkey of the author's existing key. A
  leaked subkey is revoked on its own; the identity, the commit signature history
  and everyone's existing trust survive.

Pinning the primary fingerprint means adding or rotating a signing subkey is a
non-event for verification.

Every required secret is validated in the **guard job**, before the gates and
before any registry push. Validating a fingerprint inside the signing step means
a placeholder value burns an immutable version tag before it fails.

Implementation requirements: a temporary `GNUPGHOME`; assert the imported
material contains no primary secret key; `--local-user <subkey-fingerprint>!`
so GPG cannot fall back to another key; batch and loopback pinentry; verify the
produced signature before upload; scrub `GNUPGHOME` and `DOCKER_CONFIG` and kill
the agent on every exit path. The registry token is passed by
`--password-stdin` into a temporary `DOCKER_CONFIG`, and `persist-credentials`
is off.

Secrets: `RELEASE_GPG_SUBKEY`, `RELEASE_GPG_PASSPHRASE`, `REGISTRY_TOKEN`. The
built-in `GITEA_TOKEN` cannot publish to the package registry, which is why the
third exists; it can still create the release and upload assets.

**Stated honestly, in `docs/how-to/verify-a-release.md`:** the signature proves
the artifact came from this pipeline and reached the operator unaltered. It does
not prove the binary matches the source, because the machine that built it also
held the key. Ruling 12 defers the control that closes that gap. Do not write a
stronger claim than that.

### 9. Failure and recovery are specified, not improvised

- The draft is the unit of work for the *release record*. The registry is not
  covered by it: the version tag becomes publicly pullable at step 10. Only the
  release page and its assets stay hidden until step 15.
- The existence check happens **before** the push, not after. Gitea's container
  tags are mutable — immutability is a workflow invariant, not a registry
  guarantee — so a push-then-compare would already have overwritten the tag it
  claims to refuse. Read the existing digest with
  `HEAD /v2/mokhtar/nxdns/manifests/<version>`, sending an `Accept` header for
  the index media types and following the `401` bearer challenge with the PAT.
  Absent is `404`; present returns `Docker-Content-Digest`.
- A re-run deletes an existing **draft** and repeats. It refuses to touch a
  **published** release.
- The registry version tag is immutable, and the workflow never writes it twice.
  A re-run that finds the tag present pushes nothing: it adopts the pushed
  digest and asserts the *contents* of that image against the artifacts it just
  built, which is the check that matters and the only one available. Comparing a
  rebuilt index digest was the first design and is not implementable — buildx
  cannot report an index digest without pushing, and cross-machine
  reproducibility is deferred (ruling 12), so the rebuilt digest is expected to
  differ even when nothing changed. See recorded deviation 10.
- `:latest` moves last, so a failure between the image push and publication
  leaves the version tag pushed and `latest` untouched. That is recoverable by
  re-running.
- If a tag is burned — the pipeline itself is broken and the fix is on `master` —
  the release is abandoned and reissued as the next patch version. This is the
  terminal path, and it must be written down before `v0.0.1`, not discovered
  during it.

### 10. Release notes

`CHANGELOG.md` at the repository root, Keep a Changelog format, with an
`## [Unreleased]` section maintained as work happens. The release body is that
section, plus a generated appendix: `git log --oneline` since the previous tag
inside a collapsed `<details>`, a compare link, the tarball hashes and the image
digest.

The nullable value is "the previous reachable **published release**", never "the
previous git tag". An abandoned tag from ruling 9 must not become the comparison
base, and `git describe` would pick exactly that.

With no published base — the first release, and equally the case where `v0.0.1`
was abandoned and `v0.0.2` becomes the first published one — the log is
`git log --oneline <tag>` and the compare link is omitted. It is **not** a range
with an empty left side: `..v0.0.1` resolves against `HEAD` and produces a wrong
or empty appendix rather than "all history".

The `v0.0.1` section is hand-written.

### 11. Documentation

`install-with-systemd.md`, `install-with-docker.md` and `upgrade.md` lead with
download-and-verify; the existing build-from-source steps move to a later
section of the same page. New `docs/how-to/verify-a-release.md` gives the
verification commands and the rebuild recipe.

**Milestone 13 ruling 3 applies unchanged**: every command block in `tutorial/`
and `how-to/` is executed on this host by the session that writes it, or marked
in-page as unverified with the reason.

Two rot hazards the drift test cannot see. `src/docs_drift_test.zig` guards only
`reference/{api,configuration,cli}.md`; the how-to pages are unguarded:

- The transcripts hardcode `nxdns 0.1.0-dev` in four pages. After `v0.0.1` they
  are wrong and nothing fails. Use a version-neutral placeholder, and add a
  guard that rejects a stale literal release version in the docs.
- A download URL with the version in the path goes stale at `v0.0.2`. Use a
  `latest` download form if Gitea provides one, or a placeholder the reader
  substitutes. Probe first (ruling 13); do not guess.

Sweep the whole repository for surfaces the rewrite would otherwise miss:
`README.md`, `Dockerfile` comments, `compose.yaml`, `explanation/performance-and-testing.md`,
and any remaining `zig build cross` or source-only-distribution text.

### 12. Deferred, deliberately

Recorded in PLAN so they are decisions rather than oversights:

- **The reproducibility gate** — build twice in two directory paths and assert
  identical hashes. Deferred until `v0.0.1` proves the pipeline. Until it exists,
  no document may describe the build as reproducible. Do the cheap parts now
  regardless: pin Node to an exact patch in `ci.yml` and `web/package.json`
  (both float at `24` today), `gzip -n`, `LC_ALL=C`, `TZ=UTC`.
- **cosign signatures on the image** — Gitea Actions has no OIDC identity token,
  so keyless signing is impossible and only a key-based signature is available.
  Whether Gitea's registry accepts a cosign signature manifest is unverified.
  Spike it before committing to it.

### 13. Manual prerequisites, done before the first tag

None of these are code, and all of them block `v0.0.1`:

The order matters, and the original list got parts of it wrong. Corrected:

1. Create the artifact-signing subkey, export it with `--export-secret-subkeys`,
   and publish the public key to `keys.openpgp.org`. Separately, identify which
   key actually signs your tags — after step 1 that is normally the new signing
   subkey, whose **primary** fingerprint is what gets pinned.
2. Pin the primary fingerprint in `release.yml`, replacing the placeholder. The
   guard fails closed on the placeholder, so no tag can succeed before this.
3. **Store** the secrets: `RELEASE_GPG_SUBKEY`, `RELEASE_GPG_PASSPHRASE`, and a
   registry personal access token with package write scope as `REGISTRY_TOKEN`.
   Creating the token is not storing it.
4. `app.ini`: add `.asc` and `.sig` to `[attachment] ALLOWED_TYPES`, and raise
   `MAX_FILES` from 5. Restart Gitea and confirm the *effective* settings through
   `/api/v1/settings/attachment` before trusting them. Verified against the live
   instance on 2026-08-05: `max_size` is 100 MB (ample), `max_files` is 5 (this
   release has exactly five assets, no headroom), and `.asc` is absent, so the
   signature is rejected today. The implementation names the assets
   `SHA256SUMS.txt`, `SHA256SUMS.txt.asc` and `IMAGE-DIGEST.txt`, so `.txt` and
   `.asc` are the two extensions that must be allowed.
5. **Probe: does the runner support `docker buildx` with the `docker-container`
   driver, and can it extract a foreign-platform image?** The old `docker` job
   only proved plain `docker build`, and the arm64 image-versus-tarball check
   needs more than that.
6. Land the workflows on `master` **while `main` still exists**, and let the
   reusable-gates workflow run green once, so Gitea emits its actual
   status-check context names.
7. Configure branch protection on `master` using **those observed names**. The
   reusable-workflow refactor changes them; keeping the old required checks can
   make merges impossible or leave the intended gates non-required.
8. Change Gitea's default branch to `master`.
9. Delete `origin/main` **last** — Gitea refuses to delete the default branch.
10. Merge the licence, changelog and `build.zig.zon` version commit, and let
    `master` go green.
11. Dry-run the release path. A tag-triggered workflow cannot be exercised
    without pushing *some* tag, so use a disposable tag or a scratch repository.
    **Never use `v0.0.1` as the dry run** and then expect to reuse it: tags are
    never moved (ruling 2), so a burned dry-run tag is spent.

## Sessions

S1–S5 run in parallel. S1 and S2 share one interface, fixed here so neither
blocks: S2 owns everything under `licenses/` and the licence texts; S1 owns the
build step that assembles them into `THIRD-PARTY-NOTICES`. The contract is
`zig build dist -Dversion-string=X -Dgit-commit=Y -Dweb-dist=web/dist` writing
to `zig-out/dist/`.

### Session S1: build system

Owns `build.zig`. Rulings 2, 4, 5. Deletes `cross`, adds `dist` and
`verify-dist`, moves the CI shell asserts into the build graph.

### Session S2: licence, notices, changelog

Owns `LICENSE`, `licenses/`, `CHANGELOG.md`, `README.md` (notice and links).
Ruling 3, ruling 10. Compiles the inventory by auditing what the artifacts
actually contain, and writes the CI drift guard for the dependency sets.

### Session S3: container

Owns `deploy/docker/*`, `deploy/systemd/*` (the `sysusers.conf` rename).
Ruling 6.

### Session S4: workflows

Owns `.gitea/workflows/*`. Rulings 7, 8, 9. Must not edit `build.zig` — it
consumes the step contract above.

### Session S5: documentation

Owns `docs/**`. Ruling 11, and milestone 13 ruling 3.

### Orchestrator

`PLAN.md` (ruling 1), this spec, deletion of `origin/main`, the manual
prerequisites in ruling 13, and the `v0.0.1` tag.

## Recorded (implementation)

Accepted deviations and corrections from integration. The five rulings amended
above (7, 8, 9, 10, 13, plus the option-inheritance note in 5 and the scoping of
the last acceptance line) were all wrong as first written; each amendment says
what it replaced and why.

1. **`build.zig.zon` said `0.1.0`.** It traced to the first build-baseline
   commit — a scaffold default never bumped. Ruling 2 makes `verify-dist` assert
   it against the version under build, so the CI packaging gate could never have
   passed. Set to `0.0.1`, matching the first tag.
2. **Assets carry a `.txt` extension**: `SHA256SUMS.txt`, `SHA256SUMS.txt.asc`,
   `IMAGE-DIGEST.txt`. This resolves ruling 13's extensionless-asset probe by
   construction — `.txt` is already in the live `ALLOWED_TYPES`, so only `.asc`
   still has to be added.
3. **`INSTALL.md` was added** at the repository root. `dist` hard-requires it in
   the staged payload and `verify-dist` asserts its mode.
4. **The notices preamble moved to `licenses/preamble.txt`.** The tarball and the
   image ship different sets, and the hardcoded preamble scoped itself to "a
   single static executable" — which the image is not. The image additionally
   redistributes Alpine's Mozilla CA bundle (`MPL-2.0 AND MIT`, 179,359 bytes);
   the tarball does not.
5. **Vite joined Tailwind in the inventory.** Both are devDependencies whose
   generated output ships inside the binary. The original inventory applied that
   rule to one of them and stopped.
6. **Node is pinned to `24.19.0`** in `gates.yml`, `release.yml` and
   `web/package.json` — the exact-patch pin ruling 12 asks for.
7. **`gates.yml` is SHA-pinned too.** Ruling 7 pinned only `release.yml`, but
   `release.yml` calls `gates.yml`, and those jobs share the runner host and
   docker daemon with the job holding the signing subkey. `live-tls.yml` keeps
   moving tags; it references no secret.
8. **Determinism measured better than claimed.** Two `dist` runs with separate
   cache directories and separate prefixes produced byte-identical tarballs — a
   genuine recompile, not a cache replay. The documentation still claims only
   same-directory determinism, because ruling 12's gate does not exist yet and an
   unguarded property decays. The stronger result is recorded here, not promised
   to operators.
9. **Test count moved from 1461 to 1481**: twelve licence-drift tests, then eight
   more from the review pass below. Skip counts are unchanged.

The rest came out of an adversarial review of the finished implementation. Each
was reproduced before it was fixed.

10. **The version tag was pushed before the immutability check ran.** The first
    implementation pushed `:$VERSION` and then compared the resulting digest
    with the pre-push one — a check that reports a violation it just caused. On
    a re-run that produced different bytes the tag was already overwritten and
    the original image lost. Replaced by the probe-then-adopt design ruling 9
    now describes: a real `HEAD /v2/…/manifests/<version>`, and an existing tag
    is adopted rather than rebuilt. Exercised against a fake registry covering
    `404`, `200`, the `401` bearer challenge, `500`, and a `200` with no
    `Docker-Content-Digest`.
11. **The guard proved nothing about the signing key.** It checked that
    `RELEASE_GPG_SUBKEY` and `RELEASE_GPG_PASSPHRASE` were non-empty. A
    public-only export, an export missing the pinned subkey, and a placeholder
    passphrase all passed it and failed for the first time in the signing step —
    after the image push. The guard now imports the material, asserts the pinned
    subkey is present as a secret key, and signs a throwaway file with the
    passphrase.
12. **`jq … | grep … || true` swallowed a malformed API response.** The `|| true`
    exists so grep's no-match is not fatal; it covers the whole pipeline, so a
    `200` carrying a JSON object instead of an array read as "no published
    releases" — the one wrong answer that moves `:latest` backwards. The payload
    is now type-checked before it is read.
13. **The monotonic re-check was not the concurrency backstop it claimed to be.**
    Two concurrent releases are both drafts while they run, so neither appears in
    the other's published list and both pass. The workflow `concurrency:` group
    is the only thing that serialises them. A second check was added that does
    close it: `:latest`'s own version label is read from the registry
    immediately before the tag moves, so the invariant is checked against the
    state being mutated.
14. **Publication could deadlock the tag.** A `PATCH` that Gitea committed but
    whose response was lost left the release public and the step failed, after
    which the guard refused every re-run. The step now re-reads the release and
    treats an already-published one as success.
15. **The secret-scrub backstop killed the wrong gpg-agent.** `gpgconf --kill`
    acts on the agent of the `GNUPGHOME` it is pointed at, and the bare call
    killed the runner's default agent while every leaked temporary home's agent
    kept running with the key cached. It now kills each home in its own home,
    and the guard job has the same `if: always()` backstop the publish job had.
16. **The Zig dependency guard could be silenced by pasting.** Unlike the npm
    half, nothing tied `build.zig.zon` back to the inventory, so a Mbed TLS bump
    plus the suggested identity paste left the notices claiming the old version.
    Each dependency now has to appear in the inventory at the version its URL
    names. The Zig toolchain version and the entries vendored inside Mbed TLS are
    checked the same way.
17. **The container base image was not an input to any guard.** It is the source
    of the CA bundle the image redistributes. `deploy/docker/Dockerfile` is now
    embedded in the `licenses_files` module, its digest-pinned `FROM` is a
    recorded identity section, and the CA bundle entry must name the Alpine
    release that `FROM` pins.
18. **A tree-shaken package that started shipping would have gone unnoticed.**
    `cookie-es`, `isbot`, `seroval` and `seroval-plugins` are in the lockfile
    closure and in no shipped byte. If application code imported one, no
    lockfile, version or dependency set would change — only the bundle. The
    frontend gate now recomputes the set of packages in `web/dist` from a
    `--sourcemap` build and diffs it against a recorded section, and the drift
    test requires every name in that section to be inventoried and refuses one
    that is still listed as not shipped.
19. **The recorded npm licence token was parsed and discarded**, so a package
    that relicensed passed as long as its version had not moved. Shipped
    packages must now all carry the licence the inventory's texts assume;
    `npm_not_shipped` is exempt, and `isbot` is Unlicense.
20. **The full-text licence checks were marker probes.** They prove the right
    document is present but survive most of it being deleted. The Apache-2.0 and
    MPL-2.0 texts are now pinned by SHA-256.
21. **Zig's compiler-rt contains code ported from LLVM's.** Zig's `LICENSE` is
    bare MIT naming only "Zig contributors". Reviewed: the ports carry
    Apache-2.0 WITH LLVM-exception, and that exception waives Apache §4(a),
    §4(b) and §4(d) for portions embedded in object form — the only form nxdns
    ships — so no further notice is owed. Recorded in the Zig inventory note
    rather than left as an unexamined gap.
22. **`npx` was replaced by the installed binary** in the new frontend gate. `npx`
    downloads a package it cannot find locally, so a wrong working directory
    would have turned a licence check into an unpinned fetch. Reproduced: it
    fetched `vite@8.2.0` over the pinned `8.1.5`.

23. **`actions/checkout` destroys the annotated tag object.** Found by the first
    live dry run, not by review: on a tag ref, checkout fetches the *commit* SHA
    into `refs/tags/<tag>`, so the signed tag reads as lightweight and the guard
    refuses it as unannotated. Both jobs that read the tag object — signature
    verification in the guard, the tagger date in the publish job — now force-
    refetch `refs/tags/$TAG` from origin first. The same run also proved the
    fail-closed secret guard for real: the first dry-run attempt ran with no
    secrets configured (they were on the wrong repository) and stopped in the
    guard with nothing built or pushed.

24. **Publication orchestration moved out of workflow shell into
    `tools/release.zig`.** Ruling 5 already moved the packaging asserts out of
    CI shell for one reason — "checks that only exist inside a workflow file are
    the brittleness this exists to remove" — and the release job was the larger
    half of the same problem, left in place. Three live failures came out of it,
    and each was found by executing the workflow, which is the most expensive
    place to find anything: `actions/checkout` replacing the annotated tag object
    (deviation 23), the refetch that fixed it having no credentials because
    `persist-credentials` is off, and the multiline armored subkey escaping the
    runner's log masker, which masks per line.

    Twelve subcommands, one per step group: `guard-tag`, `guard-ancestry`,
    `guard-releases`, `resolve`, `changelog`, `image`,
    `verify-image-binaries`, `sign`, `draft`, `latest`, `publish`, `scrub`. Every
    behaviour recorded in deviations 10 to 15 and 23 is carried over unchanged —
    probe-adopt, the VALIDSIG last field, the subkey-only import and signing
    probe, the array-shape guard on the releases payload, the `:latest` label
    read, the publish re-read, the per-home `gpgconf --kill`, the tag refetch.
    What is new is that the semver ordering, VALIDSIG field selection, challenge
    parsing, changelog extraction, checksum-line parsing, colon-format parsing
    and payload-shape guard are 25 unit tests in `zig build test` rather than
    shell that only ever runs on a tag push. `release.yml` keeps the triggers,
    the concurrency group, the job graph, the SHA pins, the two pinned
    fingerprints and the fail-closed secret presence check — which stays as
    shell, deliberately, so that it runs before the tool is even compiled.

    The same reasoning applies to the `jq` pipeline of the bundled-package gate,
    which moved to `web/scripts/bundledPackages.mjs` with its own vitest
    coverage and an `npm run assert-bundled` entry point.

    **Secret contract change:** `RELEASE_GPG_SUBKEY` keeps its name but now
    holds `base64 -w0` of the armored
    `--export-secret-subkeys` output rather than the armored text. Manual
    prerequisite 1 and 3 change accordingly. The tool decodes it in memory and
    writes it to a mode-600 file inside the temporary `GNUPGHOME`. A single-line
    secret is one the masker can actually mask.

25. **The first signing subkey was leaked into a job log and rotated.** Dry-run
    attempt 3 failed inside the credential-less refetch, and the runner printed
    the failing step's env block; the multiline armored `RELEASE_GPG_SUBKEY`
    escaped the per-line masker while the single-line passphrase was masked.
    Exposure: the passphrase-protected secret subkey only — the passphrase and
    the primary key were never on the runner. Response: both runs that ever saw
    the secret were deleted (verified 404 via the API and absent from
    `actions_log` on disk), subkey `B281CECC…` was revoked with the primary,
    and its replacement `019D00DF…` is the pinned `RELEASE_SIGNING_FPR`. The
    base64 contract in deviation 24 is the preventive half of this record.

26. **The image is named by the public registry host, never the server URL.**
    Attempt 4 reached the registry and failed at `docker login gitea:3000`:
    inside the cluster `GITHUB_SERVER_URL` is `http://gitea:3000`, docker
    refuses plain-http registries, and an image named `gitea:3000/…` would be
    unpullable from anywhere that matters — a wrong name that would have been
    written into the released `IMAGE-DIGEST.txt`. `release.yml` now pins
    `REGISTRY_HOST: git.mial.net`; the tool uses it for docker and image
    naming, and keeps the internal URL for the manifest probe (same registry,
    no TLS dependency in the tool). The old shell had the identical latent bug;
    no run ever reached it.

### Not verified, and why

- **The workflows' validation history.** Before any live run, `release.yml` and
  `gates.yml` were validated by YAML parse and `bash -n`, plus two steps lifted
  out and run directly: the registry probe against a fake registry (five
  response shapes) and the bundled-package check against the real `web/` build,
  proven able to fail. The live dry run then superseded this: attempt 5
  published `v0.0.0` end to end — guard, gates, image push to both platforms,
  binary-identity assertion, signing, draft, `:latest`, publication — and the
  assets verified from a clean directory (checksums OK, signature good under
  the rotated subkey). The throwaway release, tag and registry versions were
  deleted afterwards.
  Everything else that talks to the registry or the Gitea API — `buildx build
  --push`, `imagetools`, draft creation, asset upload, publication, the
  adopt-an-existing-tag path — is unexercised.
- **`RELEASE_SIGNING_FPR` is still the placeholder.** By design: the guard fails
  closed on it. It also means the release workflow cannot succeed as committed
  until manual prerequisite 2 is done.
- **The aarch64 binary was never executed.** No `qemu-aarch64` on the build host,
  so `verify-dist`'s aarch64 version check legitimately skips and `test-aarch64`
  could not run. The binary is checked statically: ELF class, no `PT_INTERP`, no
  `DT_NEEDED`, size.
- **The multi-architecture image build is unverified.** Only the native amd64
  image was built and run.
- **`shellcheck` was not run** — not installed on the build host.
- **The local Node is 24.14.1, not the pinned 24.19.0.** There is no `.npmrc`, so
  `engines` does not hard-fail, and the frontend gates ran under the older patch.

## Acceptance (milestone complete)

- [x] `PLAN.md` §2.2 amended; the build-date line and the byte limits corrected.
      §3.15 now describes the single gate set, and a new §20 records the
      publication model.
- [x] `zig build dist -Dversion-string=0.0.1 -Dweb-dist=web/dist` produces two
      tarballs and a checksum file, and fails without `-Dweb-dist`.
- [x] `zig build verify-dist`, **given the same options as `dist`**, passes and
      was proven able to fail: an oversized
      binary, a dynamically linked binary, a mismatched `build.zig.zon` version
      and a wrong archive mode each produce a named failure. Proven by repacking
      tarballs with each defect: `binary-size`, `elf`, `zon-version` and
      `archive-mode` each fired and the run exited 1. The oversize proof used a
      genuinely oversized binary for `binary-size` and a lowered budget for
      `asset-free-size`.
- [x] Two runs of `zig build dist` on the same commit produce byte-identical
      tarballs **in the same directory**. (Cross-directory reproducibility is
      ruling 12 and is not claimed here.)
- [x] The image builds for both platforms with no qemu, carries `/LICENSE` and
      `/THIRD-PARTY-NOTICES` and the OCI labels, and its binaries are
      byte-identical to the tarball binaries. The v0.0.1 run built and pushed
      both platforms on the runner; the published index lists exactly
      `linux/amd64 linux/arm64`, and `release verify-image-binaries` compared
      both binaries against the tarballs before publication.
- [x] `gates.yml` runs from both `ci.yml` and `release.yml`; `ci.yml` triggers
      on `master`; `origin/main` is gone. Proven live: pushes to `master` run
      the gates through `ci.yml`, and release runs 484-493 ran them through
      `release.yml`.
- [x] `THIRD-PARTY-NOTICES` covers musl, the Zig runtime, SQLite, Mbed TLS with
      its Apache-2.0 selection line and full text, Everest, p256-m and the web
      runtime closure. The dependency drift guard was proven able to fail:
      removing an inventory entry, staling a dependency version, staling the Zig
      version, changing the base image digest, editing a pinned licence text and
      dropping a package from the recorded bundle each produce a named failure.
- [x] A dry run of `release.yml` completes with publication disabled. Done with
      a disposable published tag instead: publication cannot be disabled without
      forking the flow it is supposed to prove, so `v0.0.0` ran the real path
      end to end — five assets, verifying checksums and signature, a
      multi-architecture image — and was then deleted (release, git tag, both
      registry versions). Five attempts; the failures and their fixes are
      deviations 23-26.
- [x] `v0.0.1` is published: five assets, a verifying signature, and an image at
      `git.mial.net/mokhtar/nxdns:0.0.1` and `:latest`. Run 493, all jobs green
      on the first attempt after the dry-run fixes.
- [x] `docs/how-to/verify-a-release.md` was followed end to end against the
      published release, from a clean directory, on this host, with a clean
      `GNUPGHOME` holding only the key fetched from keys.openpgp.org. Every
      command on the page passed: the `releases/latest` redirect printed
      `0.0.1`, both tarball downloads and the `latest` alias worked (and
      GitHub's spelling answered 404 as documented), the signature verified
      with matching primary and subkey fingerprints, `sha256sum -c` said OK for
      all three files, the tarball layout and modes matched, `nxdns version`
      printed the tag's commit, the tag digest equalled `IMAGE-DIGEST.txt`, the
      platform list was exactly `linux/amd64 linux/arm64`, and the binary
      copied out of the pulled-by-digest image hashed identical to the tarball
      binary.
- [x] No `zig build cross` or source-only-distribution text remains on any
      **active** surface: `build.zig`, the workflows, `deploy/`, `README.md` and
      `docs/`. Historical milestone specs and `TECH_DEBT.md` keep their text —
      this spec contains the string itself, so "anywhere" was never satisfiable.

## Anti-requirements

- No GoReleaser, no nfpm, no `.deb` or `.rpm`, no Homebrew, no AUR.
- No cosign, no SBOM, no SLSA provenance, no in-toto attestations.
- No `:edge` image, no rolling dev build, no pre-release tags, no floating
  `0.0` or `0` image tags.
- No third architecture, no non-Linux target, no glibc build.
- No project website, no documentation site generator.
- No changelog generation from commit messages, and no Conventional Commits.
- No release automation that cannot be run from a laptop.