# Milestone 14: build, package and publish releases

Goal: turn a signed git tag into a published, verifiable release — two static
musl tarballs and one multi-architecture container image on the self-hosted
Gitea at `git.mial.net`, with a licence, third-party notices, a changelog, a
detached signature, and documentation that leads with download-and-verify.

First tag: `v0.0.1`.

## Rulings (binding)

### 1. PLAN.md is amended first

`PLAN.md:41` lists "Prebuilt binaries / published Docker images / project
website" under §2.2 *Out of Scope (permanent scope decisions, not deferrals)*.
`AGENTS.md` makes PLAN the source of truth, so this milestone is invalid until
that line is amended. The orchestrator edits PLAN before any session starts:

- §2.2: remove prebuilt binaries and published container images. **The project
  website stays out of scope.**
- Add a §on publication: tag-triggered releases, the artifact set, the signing
  model, and the two deferrals in ruling 12.
- `PLAN.md:571`: drop "build date" from the `nxdns version` line. The version
  string and the git commit identify a build exactly, and a date is one more
  input that a reproducible build would have to pin. `src/version.zig` is
  already correct; PLAN was wrong.
- `PLAN.md:638`: state exact byte limits — 15,728,640 with embedded assets,
  10,485,760 without — resolving milestone-13 discrepancy 9 in favour of what
  CI already asserts.

### 2. The version lives in the tag, and in exactly one other place

The tag is authoritative. `v0.0.1` means `-Dversion-string=0.0.1`;
`-Dgit-commit` is the tag's peeled commit.

`build.zig.zon:3` holds `.version`, which Zig requires and nothing reads. It is
bumped in the commit before each tag, and `verify-dist` fails when it disagrees
with the version under build. Two strings, one assert, no third copy anywhere.

Tags are **never moved**. A tag that produced a bad release is abandoned; the
fix ships as the next patch version. Only `vMAJOR.MINOR.PATCH` is accepted —
the release workflow rejects any tag carrying a pre-release suffix.

### 3. Licence and third-party notices

`LICENSE` holds the English text of EUPL-1.2 verbatim, with
`Copyright (c) 2026 Mokhtar Mial`. `README.md` gains a short notice naming the
licence and the SPDX identifier `EUPL-1.2`. **No per-file SPDX headers.**

`THIRD-PARTY-NOTICES` is assembled by `zig build dist` from a committed,
reviewed inventory under `licenses/`. It is not scraped from the dependency
tree at build time: a generated notices file that nobody reads rots silently
into a false statement.

The inventory must cover everything the shipped artifacts actually contain, not
the direct dependency list:

- **musl libc** (MIT) — statically linked into every binary.
- **Zig standard library and compiler-rt** (MIT) — likewise.
- **SQLite 3.53.4** — public domain, no obligation, listed for completeness.
- **Mbed TLS 3.6.7** — with an explicit line recording that it is taken under
  the Apache-2.0 option of its dual `Apache-2.0 OR GPL-2.0-or-later` licence,
  followed by the **full Apache-2.0 text**. Naming the choice is good practice;
  shipping the text is the actual obligation.
- **Project Everest and p256-m** — compiled in at `build.zig:394` even though
  the stock config leaves both drivers disabled.
- **The web bundle's runtime closure** — the transitive set, not the four direct
  entries in `web/package.json`. Tailwind is a devDependency whose generated CSS
  ships, so "production dependencies" understates it.

A CI guard records the identity of the dependency sets (`build.zig.zon`
dependencies, and the npm closure) and fails when either changes without a
matching change under `licenses/`. It detects drift; it does not derive the
inventory.

The **container image carries `/LICENSE` and `/THIRD-PARTY-NOTICES` too**.
Distributing the image is distribution, and the obligations do not live in the
tarball.

### 4. `zig build dist` replaces `zig build cross`

The `cross` step is deleted. `dist` is the single command that produces
everything releasable, and it runs on a laptop exactly as it runs on the runner.

Inputs: `-Dversion-string` (required — no default), `-Dgit-commit`,
`-Dweb-dist`, `-Doptimize=ReleaseSafe`.

**`dist` fails when `-Dweb-dist` resolves to `web/dist-placeholder`.** The
default at `build.zig:24` is the placeholder, so a release built without the
flag would silently ship a placeholder admin page. There is no override flag;
an escape hatch here is a foot-gun with a safety label on it.

Per triple (`x86_64-linux-musl`, `aarch64-linux-musl`), `dist` builds
ReleaseSafe with `.linkage = .static` and `.strip = true`. Stripping uses
`std.Build.Module.strip` (`-fstrip`, `Module.zig:545`), which removes the
`objcopy` and `binutils-aarch64-linux-gnu` dependency from the runner.

It stages one directory per triple, `nxdns-<version>-<triple>/`, holding:

| File | Mode |
|---|---|
| `nxdns` | 0755 |
| `nxdns.service` | 0644 |
| `nxdns.conf` | 0644 |
| `LICENSE` | 0644 |
| `THIRD-PARTY-NOTICES` | 0644 |
| `INSTALL.md` | 0644 |

`deploy/systemd/sysusers.conf` is renamed to `nxdns.conf` — the name it is
installed under at `/usr/lib/sysusers.d/nxdns.conf`. A file that changes name
during install is a step an operator can get wrong.

Archiving runs as two `b.addSystemCommand` steps, never one:

```
tar --format=gnu --sort=name --mtime=@0 --owner=0 --group=0 \
    --numeric-owner -cf <name>.tar <dir>
gzip -n -9 <name>.tar
```

`b.addSystemCommand` executes argv directly and does not interpret `|`, and a
shell wrapper without `pipefail` would report only `gzip`'s exit status while a
failed `tar` passed silently. `gzip -n` is required because `--mtime=@0`
normalises the tar member times but not the timestamp gzip writes into its own
header. The environment sets `LC_ALL=C` and `TZ=UTC`.

Both steps declare the staging directory as a build input and the archive as an
output, so Zig's cache cannot serve a stale artifact.

`dist` also emits a checksum file covering **only the two tarballs**. It cannot
cover the image: the digest does not exist until buildx has pushed, which
happens later and elsewhere. The release job appends that line (ruling 7).

### 5. `zig build verify-dist` asserts what CI shell asserts today

The 50 lines of shell at `ci.yml:129` and `:168` move into a build step, so a
developer can run the release checks on a laptop. Logic that only runs in CI is
the brittleness this milestone exists to remove.

It operates on the **extracted** archive, not the staging directory:

- ELF header: correct `e_machine` per triple, **no `PT_INTERP`, no `DT_NEEDED`**.
  Matching the string `statically linked` from `file(1)` is not a static-linkage
  test.
- Stripped binary ≤ 15,728,640 bytes.
- Archive layout: exactly one top-level directory, the exact file allowlist from
  ruling 4, expected modes, no symlinks, no path traversal.
- `nxdns version` prints the version under build and the git commit. Native
  architecture only — the aarch64 binary needs qemu and is skipped without
  `-fqemu`.
- `build.zig.zon` `.version` equals the version under build.

The asset-free budget (10,485,760 bytes) gets **its own build against a
generated empty assets directory**, not against the placeholder. Ruling 4 makes
the placeholder unbuildable, and re-admitting it through a back door for one
size check would defeat the point.

### 6. Container image

`deploy/docker/Dockerfile`:

- Pin the base by digest:
  `alpine:3.22@sha256:14358309a308569c32bdc37e2e0e9694be33a9d99e68afb0f5ff33cc1f695dce`.
- Add `--platform=$BUILDPLATFORM` to the builder stage. It only copies files, so
  pinning it to the build host means the arm64 image needs no qemu.
- **Delete `apk add --no-cache ca-certificates` (`Dockerfile:15`).** Verified:
  the Alpine base already ships `/etc/ssl/certs/ca-certificates.crt` (179,359
  bytes) from `ca-certificates-bundle`. Removing it removes the only network
  fetch in the image build.
- Keep the `mkdir` and `chown 65532` — `/var/lib/nxdns` must exist with that
  ownership so Docker copies it onto a fresh named volume.
- Add `/LICENSE` and `/THIRD-PARTY-NOTICES` (ruling 3).
- OCI labels: `source`, `revision`, `version`, `licenses=EUPL-1.2`, `created`,
  `title`, `description`.

`deploy/docker/.dockerignore` is renamed to `deploy/docker/Dockerfile.dockerignore`.
The build context is the repository root, so Docker never reads the current
path, and the whole worktree — `.git`, `node_modules`, `zig-cache` — is being
sent to the daemon today.

buildx runs with `--provenance=false --sbom=false`. Recent buildx adds
provenance attestations by default, which create `unknown/unknown` platform
entries and change the index digest; Gitea's OCI 1.1 support is unverified
(go-gitea#25846) and this is not the milestone to find out. `SOURCE_DATE_EPOCH`
comes from the tag date.

`deploy/docker/compose.yaml` references the published image rather than
building one.

**Verify before publishing:** the binary inside each image is byte-identical to
the binary in the matching tarball.

### 7. Workflows

`.gitea/workflows/gates.yml`, `on: workflow_call`, holds every blocking check:
`test` (with `-Dintegration`), `test-aarch64` (`-fqemu`), `frontend`, `package`
(`dist` + `verify-dist` + the asset-free size check) and `container` (build the
image and run the existing smoke test from `ci.yml:219`). Moving only the three
test jobs would drop the packaging and container checks precisely when they
matter most.

`ci.yml` calls it on `push` and `pull_request` for **`master`**. `master` is
canonical: `origin/main` is deleted, and Gitea's default branch is changed to
match. Today `ci.yml:5` watches `main` while work happens on `master`, so
milestone 13 has never run through CI.

`.gitea/workflows/release.yml`, `on: push: tags: ['v*']`, in this order:

1. Checkout with `fetch-depth: 0` and tags. The default shallow clone breaks
   ancestry checks, previous-tag lookup and changelog generation.
2. Reject any tag that is not exactly `vMAJOR.MINOR.PATCH`.
3. `git verify-tag`, requiring an annotated tag and requiring the signature's
   fingerprint to equal a fingerprint pinned in the workflow. A bare
   `verify-tag` proves only that *some* imported key signed it.
4. Assert the tag's commit is an ancestor of `origin/master`.
5. Assert no published release exists for this tag. Delete any leftover draft.
6. Assert the version is greater than the highest published release version, so
   a late-finishing older tag cannot move `latest` backwards.
7. Gates, blocking, via `needs:`.
8. Build the web UI; `zig build dist`; `zig build verify-dist`.
9. Extract the `CHANGELOG.md` section matching the version. **Fail when absent.**
10. buildx build and push **`:<version>` only**. Capture the index digest from
    `--metadata-file`, validate it as `sha256:<64 hex>`, and confirm the pushed
    tag resolves to that digest with exactly the two intended platforms.
11. Write the digest file, append its checksum line, verify the assembled
    checksum file.
12. Sign the checksum file (ruling 8). Verify the signature locally before
    uploading it.
13. Create the release as a **draft**; upload the assets.
14. Publish the draft.
15. **Move `:latest` last.**

Every action in `release.yml` is pinned to a full commit SHA.
`actions/checkout@v4` and `mlugg/setup-zig@v2` are mutable tags on another
party's server, and a compromise upstream would run on the runner holding the
signing subkey and the registry token. `ci.yml` may keep moving tags; it holds
no secrets.

### 8. Signing

Automated on the runner, with a dedicated GPG **signing subkey** of the author's
existing commit-signing key. A leaked subkey is revoked on its own; the
identity, the commit signature history and everyone's existing trust survive.

Implementation requirements: a temporary `GNUPGHOME`; assert the imported
material contains no primary secret key; `--local-user <subkey-fingerprint>!`
so GPG cannot fall back to another key; batch and loopback pinentry; verify the
produced signature before upload; scrub `GNUPGHOME` and `DOCKER_CONFIG` and kill
the agent on every exit path. The registry token is passed by
`--password-stdin` into a temporary `DOCKER_CONFIG`, and `persist-credentials`
is off.

Secrets: `RELEASE_GPG_SUBKEY`, `RELEASE_GPG_PASSPHRASE`, `REGISTRY_TOKEN`. The
built-in `GITEA_TOKEN` cannot publish to the package registry, which is why the
third exists; it can still create the release and upload assets.

**Stated honestly, in `docs/how-to/verify-a-release.md`:** the signature proves
the artifact came from this pipeline and reached the operator unaltered. It does
not prove the binary matches the source, because the machine that built it also
held the key. Ruling 12 defers the control that closes that gap. Do not write a
stronger claim than that.

### 9. Failure and recovery are specified, not improvised

- The draft is the unit of work. Nothing is visible until step 14.
- A re-run deletes an existing **draft** and repeats. It refuses to touch a
  **published** release.
- The registry version tag is immutable. A re-run accepts an existing tag whose
  digest matches exactly what it just built, and refuses a differing one.
- `:latest` moves last, so a failure between the image push and publication
  leaves the version tag pushed and `latest` untouched. That is recoverable by
  re-running.
- If a tag is burned — the pipeline itself is broken and the fix is on `master` —
  the release is abandoned and reissued as the next patch version. This is the
  terminal path, and it must be written down before `v0.0.1`, not discovered
  during it.

### 10. Release notes

`CHANGELOG.md` at the repository root, Keep a Changelog format, with an
`## [Unreleased]` section maintained as work happens. The release body is that
section, plus a generated appendix: `git log --oneline` since the previous tag
inside a collapsed `<details>`, a compare link, the tarball hashes and the image
digest.

First release: no previous tag exists, `git describe` fails, and the compare
link is omitted. Later releases select the highest reachable **published** plain
release, not merely the nearest git tag — an abandoned tag from ruling 9 must
not become the comparison base.

The `v0.0.1` section is hand-written.

### 11. Documentation

`install-with-systemd.md`, `install-with-docker.md` and `upgrade.md` lead with
download-and-verify; the existing build-from-source steps move to a later
section of the same page. New `docs/how-to/verify-a-release.md` gives the
verification commands and the rebuild recipe.

**Milestone 13 ruling 3 applies unchanged**: every command block in `tutorial/`
and `how-to/` is executed on this host by the session that writes it, or marked
in-page as unverified with the reason.

Two rot hazards the drift test cannot see. `src/docs_drift_test.zig` guards only
`reference/{api,configuration,cli}.md`; the how-to pages are unguarded:

- The transcripts hardcode `nxdns 0.1.0-dev` in four pages. After `v0.0.1` they
  are wrong and nothing fails. Use a version-neutral placeholder, and add a
  guard that rejects a stale literal release version in the docs.
- A download URL with the version in the path goes stale at `v0.0.2`. Use a
  `latest` download form if Gitea provides one, or a placeholder the reader
  substitutes. Probe first (ruling 13); do not guess.

Sweep the whole repository for surfaces the rewrite would otherwise miss:
`README.md`, `Dockerfile` comments, `compose.yaml`, `explanation/performance-and-testing.md`,
and any remaining `zig build cross` or source-only-distribution text.

### 12. Deferred, deliberately

Recorded in PLAN so they are decisions rather than oversights:

- **The reproducibility gate** — build twice in two directory paths and assert
  identical hashes. Deferred until `v0.0.1` proves the pipeline. Until it exists,
  no document may describe the build as reproducible. Do the cheap parts now
  regardless: pin Node to an exact patch in `ci.yml` and `web/package.json`
  (both float at `24` today), `gzip -n`, `LC_ALL=C`, `TZ=UTC`.
- **cosign signatures on the image** — Gitea Actions has no OIDC identity token,
  so keyless signing is impossible and only a key-based signature is available.
  Whether Gitea's registry accepts a cosign signature manifest is unverified.
  Spike it before committing to it.

### 13. Manual prerequisites, done before the first tag

None of these are code, and all of them block `v0.0.1`:

1. Create the GPG signing subkey, export it with `--export-secret-subkeys`,
   publish the public key to `keys.openpgp.org`, and store the armored subkey
   and its passphrase as secrets.
2. Create the registry personal access token with package write scope.
3. `app.ini`: add `.asc` and `.sig` to `[attachment] ALLOWED_TYPES`, and raise
   `MAX_FILES` from 5. Verified against the live instance: `max_size` is 100 MB
   (ample), `max_files` is 5 (this release has exactly five assets, no
   headroom), and `.asc` is absent, so the signature is rejected today.
4. **Probe: can an extensionless asset be uploaded?** `ALLOWED_TYPES` is
   extension-based, and `SHA256SUMS` and `IMAGE-DIGEST` have none. If the API
   rejects them, they become `SHA256SUMS.txt`, `SHA256SUMS.txt.asc` and
   `IMAGE-DIGEST.txt`. Resolve by trying it against a scratch repository, not by
   assuming.
5. **Probe: does the runner support `docker buildx` with the `docker-container`
   driver?** The current `docker` job only proves plain `docker build`.
6. Push milestone 13 to `master`, set Gitea's default branch to `master`, then
   delete `origin/main` last.
7. Merge the licence, changelog and `build.zig.zon` version commit, and let it
   pass on `master`, before creating the tag.
8. Dry-run the whole release workflow with publication disabled.

## Sessions

S1–S5 run in parallel. S1 and S2 share one interface, fixed here so neither
blocks: S2 owns everything under `licenses/` and the licence texts; S1 owns the
build step that assembles them into `THIRD-PARTY-NOTICES`. The contract is
`zig build dist -Dversion-string=X -Dgit-commit=Y -Dweb-dist=web/dist` writing
to `zig-out/dist/`.

### Session S1: build system

Owns `build.zig`. Rulings 2, 4, 5. Deletes `cross`, adds `dist` and
`verify-dist`, moves the CI shell asserts into the build graph.

### Session S2: licence, notices, changelog

Owns `LICENSE`, `licenses/`, `CHANGELOG.md`, `README.md` (notice and links).
Ruling 3, ruling 10. Compiles the inventory by auditing what the artifacts
actually contain, and writes the CI drift guard for the dependency sets.

### Session S3: container

Owns `deploy/docker/*`, `deploy/systemd/*` (the `sysusers.conf` rename).
Ruling 6.

### Session S4: workflows

Owns `.gitea/workflows/*`. Rulings 7, 8, 9. Must not edit `build.zig` — it
consumes the step contract above.

### Session S5: documentation

Owns `docs/**`. Ruling 11, and milestone 13 ruling 3.

### Orchestrator

`PLAN.md` (ruling 1), this spec, deletion of `origin/main`, the manual
prerequisites in ruling 13, and the `v0.0.1` tag.

## Acceptance (milestone complete)

- [ ] `PLAN.md` §2.2 amended; the build-date line and the byte limits corrected.
- [ ] `zig build dist -Dversion-string=0.0.1 -Dweb-dist=web/dist` produces two
      tarballs and a checksum file, and fails without `-Dweb-dist`.
- [ ] `zig build verify-dist` passes, and was proven able to fail: an oversized
      binary, a dynamically linked binary, a mismatched `build.zig.zon` version
      and a wrong archive mode each produce a named failure.
- [ ] Two runs of `zig build dist` on the same commit produce byte-identical
      tarballs **in the same directory**. (Cross-directory reproducibility is
      ruling 12 and is not claimed here.)
- [ ] The image builds for both platforms with no qemu, carries `/LICENSE` and
      `/THIRD-PARTY-NOTICES` and the OCI labels, and its binaries are
      byte-identical to the tarball binaries.
- [ ] `gates.yml` runs from both `ci.yml` and `release.yml`; `ci.yml` triggers
      on `master`; `origin/main` is gone.
- [ ] `THIRD-PARTY-NOTICES` covers musl, the Zig runtime, SQLite, Mbed TLS with
      its Apache-2.0 selection line and full text, Everest, p256-m and the web
      runtime closure. The dependency drift guard was proven able to fail.
- [ ] A dry run of `release.yml` completes with publication disabled.
- [ ] `v0.0.1` is published: five assets, a verifying signature, and an image at
      `git.mial.net/mokhtar/nxdns:0.0.1` and `:latest`.
- [ ] `docs/how-to/verify-a-release.md` was followed end to end against the
      published release, from a clean directory, on this host.
- [ ] No `zig build cross` or source-only-distribution text remains anywhere.

## Anti-requirements

- No GoReleaser, no nfpm, no `.deb` or `.rpm`, no Homebrew, no AUR.
- No cosign, no SBOM, no SLSA provenance, no in-toto attestations.
- No `:edge` image, no rolling dev build, no pre-release tags, no floating
  `0.0` or `0` image tags.
- No third architecture, no non-Linux target, no glibc build.
- No project website, no documentation site generator.
- No changelog generation from commit messages, and no Conventional Commits.
- No release automation that cannot be run from a laptop.