From 1ff727feb87fb770c21960a7722afcfd2deac598 Mon Sep 17 00:00:00 2001 From: m5r Date: Wed, 5 Aug 2026 21:40:16 +0200 Subject: [PATCH] milestone 14 spec: build, package and publish releases --- specs/milestone-14.md | 430 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 430 insertions(+) create mode 100644 specs/milestone-14.md diff --git a/specs/milestone-14.md b/specs/milestone-14.md new file mode 100644 index 0000000..fb0165f --- /dev/null +++ b/specs/milestone-14.md @@ -0,0 +1,430 @@ +# Milestone 14: build, package and publish releases + +Goal: turn a signed git tag into a published, verifiable release — two static +musl tarballs and one multi-architecture container image on the self-hosted +Gitea at `git.mial.net`, with a licence, third-party notices, a changelog, a +detached signature, and documentation that leads with download-and-verify. + +First tag: `v0.0.1`. + +## Rulings (binding) + +### 1. PLAN.md is amended first + +`PLAN.md:41` lists "Prebuilt binaries / published Docker images / project +website" under §2.2 *Out of Scope (permanent scope decisions, not deferrals)*. +`AGENTS.md` makes PLAN the source of truth, so this milestone is invalid until +that line is amended. The orchestrator edits PLAN before any session starts: + +- §2.2: remove prebuilt binaries and published container images. **The project + website stays out of scope.** +- Add a §on publication: tag-triggered releases, the artifact set, the signing + model, and the two deferrals in ruling 12. +- `PLAN.md:571`: drop "build date" from the `nxdns version` line. The version + string and the git commit identify a build exactly, and a date is one more + input that a reproducible build would have to pin. `src/version.zig` is + already correct; PLAN was wrong. +- `PLAN.md:638`: state exact byte limits — 15,728,640 with embedded assets, + 10,485,760 without — resolving milestone-13 discrepancy 9 in favour of what + CI already asserts. + +### 2. The version lives in the tag, and in exactly one other place + +The tag is authoritative. `v0.0.1` means `-Dversion-string=0.0.1`; +`-Dgit-commit` is the tag's peeled commit. + +`build.zig.zon:3` holds `.version`, which Zig requires and nothing reads. It is +bumped in the commit before each tag, and `verify-dist` fails when it disagrees +with the version under build. Two strings, one assert, no third copy anywhere. + +Tags are **never moved**. A tag that produced a bad release is abandoned; the +fix ships as the next patch version. Only `vMAJOR.MINOR.PATCH` is accepted — +the release workflow rejects any tag carrying a pre-release suffix. + +### 3. Licence and third-party notices + +`LICENSE` holds the English text of EUPL-1.2 verbatim, with +`Copyright (c) 2026 Mokhtar Mial`. `README.md` gains a short notice naming the +licence and the SPDX identifier `EUPL-1.2`. **No per-file SPDX headers.** + +`THIRD-PARTY-NOTICES` is assembled by `zig build dist` from a committed, +reviewed inventory under `licenses/`. It is not scraped from the dependency +tree at build time: a generated notices file that nobody reads rots silently +into a false statement. + +The inventory must cover everything the shipped artifacts actually contain, not +the direct dependency list: + +- **musl libc** (MIT) — statically linked into every binary. +- **Zig standard library and compiler-rt** (MIT) — likewise. +- **SQLite 3.53.4** — public domain, no obligation, listed for completeness. +- **Mbed TLS 3.6.7** — with an explicit line recording that it is taken under + the Apache-2.0 option of its dual `Apache-2.0 OR GPL-2.0-or-later` licence, + followed by the **full Apache-2.0 text**. Naming the choice is good practice; + shipping the text is the actual obligation. +- **Project Everest and p256-m** — compiled in at `build.zig:394` even though + the stock config leaves both drivers disabled. +- **The web bundle's runtime closure** — the transitive set, not the four direct + entries in `web/package.json`. Tailwind is a devDependency whose generated CSS + ships, so "production dependencies" understates it. + +A CI guard records the identity of the dependency sets (`build.zig.zon` +dependencies, and the npm closure) and fails when either changes without a +matching change under `licenses/`. It detects drift; it does not derive the +inventory. + +The **container image carries `/LICENSE` and `/THIRD-PARTY-NOTICES` too**. +Distributing the image is distribution, and the obligations do not live in the +tarball. + +### 4. `zig build dist` replaces `zig build cross` + +The `cross` step is deleted. `dist` is the single command that produces +everything releasable, and it runs on a laptop exactly as it runs on the runner. + +Inputs: `-Dversion-string` (required — no default), `-Dgit-commit`, +`-Dweb-dist`, `-Doptimize=ReleaseSafe`. + +**`dist` fails when `-Dweb-dist` resolves to `web/dist-placeholder`.** The +default at `build.zig:24` is the placeholder, so a release built without the +flag would silently ship a placeholder admin page. There is no override flag; +an escape hatch here is a foot-gun with a safety label on it. + +Per triple (`x86_64-linux-musl`, `aarch64-linux-musl`), `dist` builds +ReleaseSafe with `.linkage = .static` and `.strip = true`. Stripping uses +`std.Build.Module.strip` (`-fstrip`, `Module.zig:545`), which removes the +`objcopy` and `binutils-aarch64-linux-gnu` dependency from the runner. + +It stages one directory per triple, `nxdns--/`, holding: + +| File | Mode | +|---|---| +| `nxdns` | 0755 | +| `nxdns.service` | 0644 | +| `nxdns.conf` | 0644 | +| `LICENSE` | 0644 | +| `THIRD-PARTY-NOTICES` | 0644 | +| `INSTALL.md` | 0644 | + +`deploy/systemd/sysusers.conf` is renamed to `nxdns.conf` — the name it is +installed under at `/usr/lib/sysusers.d/nxdns.conf`. A file that changes name +during install is a step an operator can get wrong. + +Archiving runs as two `b.addSystemCommand` steps, never one: + +``` +tar --format=gnu --sort=name --mtime=@0 --owner=0 --group=0 \ + --numeric-owner -cf .tar +gzip -n -9 .tar +``` + +`b.addSystemCommand` executes argv directly and does not interpret `|`, and a +shell wrapper without `pipefail` would report only `gzip`'s exit status while a +failed `tar` passed silently. `gzip -n` is required because `--mtime=@0` +normalises the tar member times but not the timestamp gzip writes into its own +header. The environment sets `LC_ALL=C` and `TZ=UTC`. + +Both steps declare the staging directory as a build input and the archive as an +output, so Zig's cache cannot serve a stale artifact. + +`dist` also emits a checksum file covering **only the two tarballs**. It cannot +cover the image: the digest does not exist until buildx has pushed, which +happens later and elsewhere. The release job appends that line (ruling 7). + +### 5. `zig build verify-dist` asserts what CI shell asserts today + +The 50 lines of shell at `ci.yml:129` and `:168` move into a build step, so a +developer can run the release checks on a laptop. Logic that only runs in CI is +the brittleness this milestone exists to remove. + +It operates on the **extracted** archive, not the staging directory: + +- ELF header: correct `e_machine` per triple, **no `PT_INTERP`, no `DT_NEEDED`**. + Matching the string `statically linked` from `file(1)` is not a static-linkage + test. +- Stripped binary ≤ 15,728,640 bytes. +- Archive layout: exactly one top-level directory, the exact file allowlist from + ruling 4, expected modes, no symlinks, no path traversal. +- `nxdns version` prints the version under build and the git commit. Native + architecture only — the aarch64 binary needs qemu and is skipped without + `-fqemu`. +- `build.zig.zon` `.version` equals the version under build. + +The asset-free budget (10,485,760 bytes) gets **its own build against a +generated empty assets directory**, not against the placeholder. Ruling 4 makes +the placeholder unbuildable, and re-admitting it through a back door for one +size check would defeat the point. + +### 6. Container image + +`deploy/docker/Dockerfile`: + +- Pin the base by digest: + `alpine:3.22@sha256:14358309a308569c32bdc37e2e0e9694be33a9d99e68afb0f5ff33cc1f695dce`. +- Add `--platform=$BUILDPLATFORM` to the builder stage. It only copies files, so + pinning it to the build host means the arm64 image needs no qemu. +- **Delete `apk add --no-cache ca-certificates` (`Dockerfile:15`).** Verified: + the Alpine base already ships `/etc/ssl/certs/ca-certificates.crt` (179,359 + bytes) from `ca-certificates-bundle`. Removing it removes the only network + fetch in the image build. +- Keep the `mkdir` and `chown 65532` — `/var/lib/nxdns` must exist with that + ownership so Docker copies it onto a fresh named volume. +- Add `/LICENSE` and `/THIRD-PARTY-NOTICES` (ruling 3). +- OCI labels: `source`, `revision`, `version`, `licenses=EUPL-1.2`, `created`, + `title`, `description`. + +`deploy/docker/.dockerignore` is renamed to `deploy/docker/Dockerfile.dockerignore`. +The build context is the repository root, so Docker never reads the current +path, and the whole worktree — `.git`, `node_modules`, `zig-cache` — is being +sent to the daemon today. + +buildx runs with `--provenance=false --sbom=false`. Recent buildx adds +provenance attestations by default, which create `unknown/unknown` platform +entries and change the index digest; Gitea's OCI 1.1 support is unverified +(go-gitea#25846) and this is not the milestone to find out. `SOURCE_DATE_EPOCH` +comes from the tag date. + +`deploy/docker/compose.yaml` references the published image rather than +building one. + +**Verify before publishing:** the binary inside each image is byte-identical to +the binary in the matching tarball. + +### 7. Workflows + +`.gitea/workflows/gates.yml`, `on: workflow_call`, holds every blocking check: +`test` (with `-Dintegration`), `test-aarch64` (`-fqemu`), `frontend`, `package` +(`dist` + `verify-dist` + the asset-free size check) and `container` (build the +image and run the existing smoke test from `ci.yml:219`). Moving only the three +test jobs would drop the packaging and container checks precisely when they +matter most. + +`ci.yml` calls it on `push` and `pull_request` for **`master`**. `master` is +canonical: `origin/main` is deleted, and Gitea's default branch is changed to +match. Today `ci.yml:5` watches `main` while work happens on `master`, so +milestone 13 has never run through CI. + +`.gitea/workflows/release.yml`, `on: push: tags: ['v*']`, in this order: + +1. Checkout with `fetch-depth: 0` and tags. The default shallow clone breaks + ancestry checks, previous-tag lookup and changelog generation. +2. Reject any tag that is not exactly `vMAJOR.MINOR.PATCH`. +3. `git verify-tag`, requiring an annotated tag and requiring the signature's + fingerprint to equal a fingerprint pinned in the workflow. A bare + `verify-tag` proves only that *some* imported key signed it. +4. Assert the tag's commit is an ancestor of `origin/master`. +5. Assert no published release exists for this tag. Delete any leftover draft. +6. Assert the version is greater than the highest published release version, so + a late-finishing older tag cannot move `latest` backwards. +7. Gates, blocking, via `needs:`. +8. Build the web UI; `zig build dist`; `zig build verify-dist`. +9. Extract the `CHANGELOG.md` section matching the version. **Fail when absent.** +10. buildx build and push **`:` only**. Capture the index digest from + `--metadata-file`, validate it as `sha256:<64 hex>`, and confirm the pushed + tag resolves to that digest with exactly the two intended platforms. +11. Write the digest file, append its checksum line, verify the assembled + checksum file. +12. Sign the checksum file (ruling 8). Verify the signature locally before + uploading it. +13. Create the release as a **draft**; upload the assets. +14. Publish the draft. +15. **Move `:latest` last.** + +Every action in `release.yml` is pinned to a full commit SHA. +`actions/checkout@v4` and `mlugg/setup-zig@v2` are mutable tags on another +party's server, and a compromise upstream would run on the runner holding the +signing subkey and the registry token. `ci.yml` may keep moving tags; it holds +no secrets. + +### 8. Signing + +Automated on the runner, with a dedicated GPG **signing subkey** of the author's +existing commit-signing key. A leaked subkey is revoked on its own; the +identity, the commit signature history and everyone's existing trust survive. + +Implementation requirements: a temporary `GNUPGHOME`; assert the imported +material contains no primary secret key; `--local-user !` +so GPG cannot fall back to another key; batch and loopback pinentry; verify the +produced signature before upload; scrub `GNUPGHOME` and `DOCKER_CONFIG` and kill +the agent on every exit path. The registry token is passed by +`--password-stdin` into a temporary `DOCKER_CONFIG`, and `persist-credentials` +is off. + +Secrets: `RELEASE_GPG_SUBKEY`, `RELEASE_GPG_PASSPHRASE`, `REGISTRY_TOKEN`. The +built-in `GITEA_TOKEN` cannot publish to the package registry, which is why the +third exists; it can still create the release and upload assets. + +**Stated honestly, in `docs/how-to/verify-a-release.md`:** the signature proves +the artifact came from this pipeline and reached the operator unaltered. It does +not prove the binary matches the source, because the machine that built it also +held the key. Ruling 12 defers the control that closes that gap. Do not write a +stronger claim than that. + +### 9. Failure and recovery are specified, not improvised + +- The draft is the unit of work. Nothing is visible until step 14. +- A re-run deletes an existing **draft** and repeats. It refuses to touch a + **published** release. +- The registry version tag is immutable. A re-run accepts an existing tag whose + digest matches exactly what it just built, and refuses a differing one. +- `:latest` moves last, so a failure between the image push and publication + leaves the version tag pushed and `latest` untouched. That is recoverable by + re-running. +- If a tag is burned — the pipeline itself is broken and the fix is on `master` — + the release is abandoned and reissued as the next patch version. This is the + terminal path, and it must be written down before `v0.0.1`, not discovered + during it. + +### 10. Release notes + +`CHANGELOG.md` at the repository root, Keep a Changelog format, with an +`## [Unreleased]` section maintained as work happens. The release body is that +section, plus a generated appendix: `git log --oneline` since the previous tag +inside a collapsed `
`, a compare link, the tarball hashes and the image +digest. + +First release: no previous tag exists, `git describe` fails, and the compare +link is omitted. Later releases select the highest reachable **published** plain +release, not merely the nearest git tag — an abandoned tag from ruling 9 must +not become the comparison base. + +The `v0.0.1` section is hand-written. + +### 11. Documentation + +`install-with-systemd.md`, `install-with-docker.md` and `upgrade.md` lead with +download-and-verify; the existing build-from-source steps move to a later +section of the same page. New `docs/how-to/verify-a-release.md` gives the +verification commands and the rebuild recipe. + +**Milestone 13 ruling 3 applies unchanged**: every command block in `tutorial/` +and `how-to/` is executed on this host by the session that writes it, or marked +in-page as unverified with the reason. + +Two rot hazards the drift test cannot see. `src/docs_drift_test.zig` guards only +`reference/{api,configuration,cli}.md`; the how-to pages are unguarded: + +- The transcripts hardcode `nxdns 0.1.0-dev` in four pages. After `v0.0.1` they + are wrong and nothing fails. Use a version-neutral placeholder, and add a + guard that rejects a stale literal release version in the docs. +- A download URL with the version in the path goes stale at `v0.0.2`. Use a + `latest` download form if Gitea provides one, or a placeholder the reader + substitutes. Probe first (ruling 13); do not guess. + +Sweep the whole repository for surfaces the rewrite would otherwise miss: +`README.md`, `Dockerfile` comments, `compose.yaml`, `explanation/performance-and-testing.md`, +and any remaining `zig build cross` or source-only-distribution text. + +### 12. Deferred, deliberately + +Recorded in PLAN so they are decisions rather than oversights: + +- **The reproducibility gate** — build twice in two directory paths and assert + identical hashes. Deferred until `v0.0.1` proves the pipeline. Until it exists, + no document may describe the build as reproducible. Do the cheap parts now + regardless: pin Node to an exact patch in `ci.yml` and `web/package.json` + (both float at `24` today), `gzip -n`, `LC_ALL=C`, `TZ=UTC`. +- **cosign signatures on the image** — Gitea Actions has no OIDC identity token, + so keyless signing is impossible and only a key-based signature is available. + Whether Gitea's registry accepts a cosign signature manifest is unverified. + Spike it before committing to it. + +### 13. Manual prerequisites, done before the first tag + +None of these are code, and all of them block `v0.0.1`: + +1. Create the GPG signing subkey, export it with `--export-secret-subkeys`, + publish the public key to `keys.openpgp.org`, and store the armored subkey + and its passphrase as secrets. +2. Create the registry personal access token with package write scope. +3. `app.ini`: add `.asc` and `.sig` to `[attachment] ALLOWED_TYPES`, and raise + `MAX_FILES` from 5. Verified against the live instance: `max_size` is 100 MB + (ample), `max_files` is 5 (this release has exactly five assets, no + headroom), and `.asc` is absent, so the signature is rejected today. +4. **Probe: can an extensionless asset be uploaded?** `ALLOWED_TYPES` is + extension-based, and `SHA256SUMS` and `IMAGE-DIGEST` have none. If the API + rejects them, they become `SHA256SUMS.txt`, `SHA256SUMS.txt.asc` and + `IMAGE-DIGEST.txt`. Resolve by trying it against a scratch repository, not by + assuming. +5. **Probe: does the runner support `docker buildx` with the `docker-container` + driver?** The current `docker` job only proves plain `docker build`. +6. Push milestone 13 to `master`, set Gitea's default branch to `master`, then + delete `origin/main` last. +7. Merge the licence, changelog and `build.zig.zon` version commit, and let it + pass on `master`, before creating the tag. +8. Dry-run the whole release workflow with publication disabled. + +## Sessions + +S1–S5 run in parallel. S1 and S2 share one interface, fixed here so neither +blocks: S2 owns everything under `licenses/` and the licence texts; S1 owns the +build step that assembles them into `THIRD-PARTY-NOTICES`. The contract is +`zig build dist -Dversion-string=X -Dgit-commit=Y -Dweb-dist=web/dist` writing +to `zig-out/dist/`. + +### Session S1: build system + +Owns `build.zig`. Rulings 2, 4, 5. Deletes `cross`, adds `dist` and +`verify-dist`, moves the CI shell asserts into the build graph. + +### Session S2: licence, notices, changelog + +Owns `LICENSE`, `licenses/`, `CHANGELOG.md`, `README.md` (notice and links). +Ruling 3, ruling 10. Compiles the inventory by auditing what the artifacts +actually contain, and writes the CI drift guard for the dependency sets. + +### Session S3: container + +Owns `deploy/docker/*`, `deploy/systemd/*` (the `sysusers.conf` rename). +Ruling 6. + +### Session S4: workflows + +Owns `.gitea/workflows/*`. Rulings 7, 8, 9. Must not edit `build.zig` — it +consumes the step contract above. + +### Session S5: documentation + +Owns `docs/**`. Ruling 11, and milestone 13 ruling 3. + +### Orchestrator + +`PLAN.md` (ruling 1), this spec, deletion of `origin/main`, the manual +prerequisites in ruling 13, and the `v0.0.1` tag. + +## Acceptance (milestone complete) + +- [ ] `PLAN.md` §2.2 amended; the build-date line and the byte limits corrected. +- [ ] `zig build dist -Dversion-string=0.0.1 -Dweb-dist=web/dist` produces two + tarballs and a checksum file, and fails without `-Dweb-dist`. +- [ ] `zig build verify-dist` passes, and was proven able to fail: an oversized + binary, a dynamically linked binary, a mismatched `build.zig.zon` version + and a wrong archive mode each produce a named failure. +- [ ] Two runs of `zig build dist` on the same commit produce byte-identical + tarballs **in the same directory**. (Cross-directory reproducibility is + ruling 12 and is not claimed here.) +- [ ] The image builds for both platforms with no qemu, carries `/LICENSE` and + `/THIRD-PARTY-NOTICES` and the OCI labels, and its binaries are + byte-identical to the tarball binaries. +- [ ] `gates.yml` runs from both `ci.yml` and `release.yml`; `ci.yml` triggers + on `master`; `origin/main` is gone. +- [ ] `THIRD-PARTY-NOTICES` covers musl, the Zig runtime, SQLite, Mbed TLS with + its Apache-2.0 selection line and full text, Everest, p256-m and the web + runtime closure. The dependency drift guard was proven able to fail. +- [ ] A dry run of `release.yml` completes with publication disabled. +- [ ] `v0.0.1` is published: five assets, a verifying signature, and an image at + `git.mial.net/mokhtar/nxdns:0.0.1` and `:latest`. +- [ ] `docs/how-to/verify-a-release.md` was followed end to end against the + published release, from a clean directory, on this host. +- [ ] No `zig build cross` or source-only-distribution text remains anywhere. + +## Anti-requirements + +- No GoReleaser, no nfpm, no `.deb` or `.rpm`, no Homebrew, no AUR. +- No cosign, no SBOM, no SLSA provenance, no in-toto attestations. +- No `:edge` image, no rolling dev build, no pre-release tags, no floating + `0.0` or `0` image tags. +- No third architecture, no non-Linux target, no glibc build. +- No project website, no documentation site generator. +- No changelog generation from commit messages, and no Conventional Commits. +- No release automation that cannot be run from a laptop.