mokhtar/nxdns
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

milestone 14: build, package, sign and publish releases

Browse Source
This commit is contained in:
mokhtar
2026-08-08 12:38:29 +02:00
parent 6c507992e4
commit cdacc560b7
48 changed files with 7272 additions and 543 deletions
Download Patch File Download Diff File Expand all files Collapse all files
specs/milestone-14.md
+266 -41
View File
@@ -155,6 +155,12 @@ generated empty assets directory**, not against the placeholder. Ruling 4 makes
the placeholder unbuildable, and re-admitting it through a back door for one
size check would defeat the point.
**A second `zig build` invocation does not inherit the first one's `-D` options.**
`zig build dist -Dversion-string=X` followed by a bare `zig build verify-dist`
verifies a *differently configured* build. Every caller — the workflows, the
documentation, and this spec's own acceptance list — passes the same
`-Dversion-string`, `-Dgit-commit`, `-Dweb-dist` and `-Doptimize` to both.
### 6. Container image
`deploy/docker/Dockerfile`:
@@ -227,8 +233,18 @@ milestone 13 has never run through CI.
12. Sign the checksum file (ruling 8). Verify the signature locally before
uploading it.
13. Create the release as a **draft**; upload the assets.
14. Publish the draft.
15. **Move `:latest` last.**
14. Move `:latest` and verify it resolves to the built digest. Re-check the
monotonic-version invariant here: step 6 ran before the gates, and proves
nothing about which of two in-flight tags finishes last.
15. **Publish the draft last.** Publication is the one irreversible act, so it
goes after everything that can still fail.
Amended after review. The original order published at 14 and moved `:latest` at
15, which deadlocks: a failure while moving `:latest` leaves a published release,
and ruling 9 makes a re-run refuse a published release. Nothing could repair it.
The cost of the corrected order is a short window where `:latest` serves the new
image before the release page is public. That is recoverable by a re-run; the
deadlock was not.
Every action in `release.yml` is pinned to a full commit SHA.
`actions/checkout@v4` and `mlugg/setup-zig@v2` are mutable tags on another
@@ -238,9 +254,27 @@ no secrets.
### 8. Signing
Automated on the runner, with a dedicated GPG **signing subkey** of the author's
existing commit-signing key. A leaked subkey is revoked on its own; the
identity, the commit signature history and everyone's existing trust survive.
**Two different keys are involved, and the original ruling conflated them.**
- The **tag-signing key** is the human's. `git tag -s` uses it, and step 3 checks
it. What gets pinned in `release.yml` is the **primary certificate
fingerprint**, which `git verify-tag --raw` emits as the **last** field of the
`VALIDSIG` line. Field 3 is whichever key actually made the signature — the
signing subkey once one exists. Pinning field 3 would mean that creating the
release subkey below silently blocks every future release, with an error
message that reads like a forged tag. Reproduced against a real keyring during
review.
- The **artifact-signing subkey** is the runner's, and it signs the checksum
file. It is a dedicated GPG signing subkey of the author's existing key. A
leaked subkey is revoked on its own; the identity, the commit signature history
and everyone's existing trust survive.
Pinning the primary fingerprint means adding or rotating a signing subkey is a
non-event for verification.
Every required secret is validated in the **guard job**, before the gates and
before any registry push. Validating a fingerprint inside the signing step means
a placeholder value burns an immutable version tag before it fails.
Implementation requirements: a temporary `GNUPGHOME`; assert the imported
material contains no primary secret key; `--local-user <subkey-fingerprint>!`
@@ -262,11 +296,26 @@ stronger claim than that.
### 9. Failure and recovery are specified, not improvised
- The draft is the unit of work. Nothing is visible until step 14.
- The draft is the unit of work for the *release record*. The registry is not
covered by it: the version tag becomes publicly pullable at step 10. Only the
release page and its assets stay hidden until step 15.
- The existence check happens **before** the push, not after. Gitea's container
tags are mutable — immutability is a workflow invariant, not a registry
guarantee — so a push-then-compare would already have overwritten the tag it
claims to refuse. Read the existing digest with
`HEAD /v2/mokhtar/nxdns/manifests/<version>`, sending an `Accept` header for
the index media types and following the `401` bearer challenge with the PAT.
Absent is `404`; present returns `Docker-Content-Digest`.
- A re-run deletes an existing **draft** and repeats. It refuses to touch a
**published** release.
- The registry version tag is immutable. A re-run accepts an existing tag whose
digest matches exactly what it just built, and refuses a differing one.
- The registry version tag is immutable, and the workflow never writes it twice.
A re-run that finds the tag present pushes nothing: it adopts the pushed
digest and asserts the *contents* of that image against the artifacts it just
built, which is the check that matters and the only one available. Comparing a
rebuilt index digest was the first design and is not implementable — buildx
cannot report an index digest without pushing, and cross-machine
reproducibility is deferred (ruling 12), so the rebuilt digest is expected to
differ even when nothing changed. See recorded deviation 10.
- `:latest` moves last, so a failure between the image push and publication
leaves the version tag pushed and `latest` untouched. That is recoverable by
re-running.
@@ -283,10 +332,15 @@ section, plus a generated appendix: `git log --oneline` since the previous tag
inside a collapsed `<details>`, a compare link, the tarball hashes and the image
digest.
First release: no previous tag exists, `git describe` fails, and the compare
link is omitted. Later releases select the highest reachable **published** plain
release, not merely the nearest git tag — an abandoned tag from ruling 9 must
not become the comparison base.
The nullable value is "the previous reachable **published release**", never "the
previous git tag". An abandoned tag from ruling 9 must not become the comparison
base, and `git describe` would pick exactly that.
With no published base — the first release, and equally the case where `v0.0.1`
was abandoned and `v0.0.2` becomes the first published one — the log is
`git log --oneline <tag>` and the compare link is omitted. It is **not** a range
with an empty left side: `..v0.0.1` resolves against `HEAD` and produces a wrong
or empty appendix rather than "all history".
The `v0.0.1` section is hand-written.
@@ -333,26 +387,43 @@ Recorded in PLAN so they are decisions rather than oversights:
None of these are code, and all of them block `v0.0.1`:
1. Create the GPG signing subkey, export it with `--export-secret-subkeys`,
publish the public key to `keys.openpgp.org`, and store the armored subkey
and its passphrase as secrets.
2. Create the registry personal access token with package write scope.
3. `app.ini`: add `.asc` and `.sig` to `[attachment] ALLOWED_TYPES`, and raise
`MAX_FILES` from 5. Verified against the live instance: `max_size` is 100 MB
(ample), `max_files` is 5 (this release has exactly five assets, no
headroom), and `.asc` is absent, so the signature is rejected today.
4. **Probe: can an extensionless asset be uploaded?** `ALLOWED_TYPES` is
extension-based, and `SHA256SUMS` and `IMAGE-DIGEST` have none. If the API
rejects them, they become `SHA256SUMS.txt`, `SHA256SUMS.txt.asc` and
`IMAGE-DIGEST.txt`. Resolve by trying it against a scratch repository, not by
assuming.
The order matters, and the original list got parts of it wrong. Corrected:
1. Create the artifact-signing subkey, export it with `--export-secret-subkeys`,
and publish the public key to `keys.openpgp.org`. Separately, identify which
key actually signs your tags — after step 1 that is normally the new signing
subkey, whose **primary** fingerprint is what gets pinned.
2. Pin the primary fingerprint in `release.yml`, replacing the placeholder. The
guard fails closed on the placeholder, so no tag can succeed before this.
3. **Store** the secrets: `RELEASE_GPG_SUBKEY`, `RELEASE_GPG_PASSPHRASE`, and a
registry personal access token with package write scope as `REGISTRY_TOKEN`.
Creating the token is not storing it.
4. `app.ini`: add `.asc` and `.sig` to `[attachment] ALLOWED_TYPES`, and raise
`MAX_FILES` from 5. Restart Gitea and confirm the *effective* settings through
`/api/v1/settings/attachment` before trusting them. Verified against the live
instance on 2026-08-05: `max_size` is 100 MB (ample), `max_files` is 5 (this
release has exactly five assets, no headroom), and `.asc` is absent, so the
signature is rejected today. The implementation names the assets
`SHA256SUMS.txt`, `SHA256SUMS.txt.asc` and `IMAGE-DIGEST.txt`, so `.txt` and
`.asc` are the two extensions that must be allowed.
5. **Probe: does the runner support `docker buildx` with the `docker-container`
driver?** The current `docker` job only proves plain `docker build`.
6. Push milestone 13 to `master`, set Gitea's default branch to `master`, then
delete `origin/main` last.
7. Merge the licence, changelog and `build.zig.zon` version commit, and let it
pass on `master`, before creating the tag.
8. Dry-run the whole release workflow with publication disabled.
driver, and can it extract a foreign-platform image?** The old `docker` job
only proved plain `docker build`, and the arm64 image-versus-tarball check
needs more than that.
6. Land the workflows on `master` **while `main` still exists**, and let the
reusable-gates workflow run green once, so Gitea emits its actual
status-check context names.
7. Configure branch protection on `master` using **those observed names**. The
reusable-workflow refactor changes them; keeping the old required checks can
make merges impossible or leave the intended gates non-required.
8. Change Gitea's default branch to `master`.
9. Delete `origin/main` **last** — Gitea refuses to delete the default branch.
10. Merge the licence, changelog and `build.zig.zon` version commit, and let
`master` go green.
11. Dry-run the release path. A tag-triggered workflow cannot be exercised
without pushing *some* tag, so use a disposable tag or a scratch repository.
**Never use `v0.0.1` as the dry run** and then expect to reuse it: tags are
never moved (ruling 2), so a burned dry-run tag is spent.
## Sessions
@@ -392,31 +463,185 @@ Owns `docs/**`. Ruling 11, and milestone 13 ruling 3.
`PLAN.md` (ruling 1), this spec, deletion of `origin/main`, the manual
prerequisites in ruling 13, and the `v0.0.1` tag.
## Recorded (implementation)
Accepted deviations and corrections from integration. The five rulings amended
above (7, 8, 9, 10, 13, plus the option-inheritance note in 5 and the scoping of
the last acceptance line) were all wrong as first written; each amendment says
what it replaced and why.
1. **`build.zig.zon` said `0.1.0`.** It traced to the first build-baseline
commit — a scaffold default never bumped. Ruling 2 makes `verify-dist` assert
it against the version under build, so the CI packaging gate could never have
passed. Set to `0.0.1`, matching the first tag.
2. **Assets carry a `.txt` extension**: `SHA256SUMS.txt`, `SHA256SUMS.txt.asc`,
`IMAGE-DIGEST.txt`. This resolves ruling 13's extensionless-asset probe by
construction — `.txt` is already in the live `ALLOWED_TYPES`, so only `.asc`
still has to be added.
3. **`INSTALL.md` was added** at the repository root. `dist` hard-requires it in
the staged payload and `verify-dist` asserts its mode.
4. **The notices preamble moved to `licenses/preamble.txt`.** The tarball and the
image ship different sets, and the hardcoded preamble scoped itself to "a
single static executable" — which the image is not. The image additionally
redistributes Alpine's Mozilla CA bundle (`MPL-2.0 AND MIT`, 179,359 bytes);
the tarball does not.
5. **Vite joined Tailwind in the inventory.** Both are devDependencies whose
generated output ships inside the binary. The original inventory applied that
rule to one of them and stopped.
6. **Node is pinned to `24.19.0`** in `gates.yml`, `release.yml` and
`web/package.json` — the exact-patch pin ruling 12 asks for.
7. **`gates.yml` is SHA-pinned too.** Ruling 7 pinned only `release.yml`, but
`release.yml` calls `gates.yml`, and those jobs share the runner host and
docker daemon with the job holding the signing subkey. `live-tls.yml` keeps
moving tags; it references no secret.
8. **Determinism measured better than claimed.** Two `dist` runs with separate
cache directories and separate prefixes produced byte-identical tarballs — a
genuine recompile, not a cache replay. The documentation still claims only
same-directory determinism, because ruling 12's gate does not exist yet and an
unguarded property decays. The stronger result is recorded here, not promised
to operators.
9. **Test count moved from 1461 to 1481**: twelve licence-drift tests, then eight
more from the review pass below. Skip counts are unchanged.
The rest came out of an adversarial review of the finished implementation. Each
was reproduced before it was fixed.
10. **The version tag was pushed before the immutability check ran.** The first
implementation pushed `:$VERSION` and then compared the resulting digest
with the pre-push one — a check that reports a violation it just caused. On
a re-run that produced different bytes the tag was already overwritten and
the original image lost. Replaced by the probe-then-adopt design ruling 9
now describes: a real `HEAD /v2/…/manifests/<version>`, and an existing tag
is adopted rather than rebuilt. Exercised against a fake registry covering
`404`, `200`, the `401` bearer challenge, `500`, and a `200` with no
`Docker-Content-Digest`.
11. **The guard proved nothing about the signing key.** It checked that
`RELEASE_GPG_SUBKEY` and `RELEASE_GPG_PASSPHRASE` were non-empty. A
public-only export, an export missing the pinned subkey, and a placeholder
passphrase all passed it and failed for the first time in the signing step —
after the image push. The guard now imports the material, asserts the pinned
subkey is present as a secret key, and signs a throwaway file with the
passphrase.
12. **`jq … | grep … || true` swallowed a malformed API response.** The `|| true`
exists so grep's no-match is not fatal; it covers the whole pipeline, so a
`200` carrying a JSON object instead of an array read as "no published
releases" — the one wrong answer that moves `:latest` backwards. The payload
is now type-checked before it is read.
13. **The monotonic re-check was not the concurrency backstop it claimed to be.**
Two concurrent releases are both drafts while they run, so neither appears in
the other's published list and both pass. The workflow `concurrency:` group
is the only thing that serialises them. A second check was added that does
close it: `:latest`'s own version label is read from the registry
immediately before the tag moves, so the invariant is checked against the
state being mutated.
14. **Publication could deadlock the tag.** A `PATCH` that Gitea committed but
whose response was lost left the release public and the step failed, after
which the guard refused every re-run. The step now re-reads the release and
treats an already-published one as success.
15. **The secret-scrub backstop killed the wrong gpg-agent.** `gpgconf --kill`
acts on the agent of the `GNUPGHOME` it is pointed at, and the bare call
killed the runner's default agent while every leaked temporary home's agent
kept running with the key cached. It now kills each home in its own home,
and the guard job has the same `if: always()` backstop the publish job had.
16. **The Zig dependency guard could be silenced by pasting.** Unlike the npm
half, nothing tied `build.zig.zon` back to the inventory, so a Mbed TLS bump
plus the suggested identity paste left the notices claiming the old version.
Each dependency now has to appear in the inventory at the version its URL
names. The Zig toolchain version and the entries vendored inside Mbed TLS are
checked the same way.
17. **The container base image was not an input to any guard.** It is the source
of the CA bundle the image redistributes. `deploy/docker/Dockerfile` is now
embedded in the `licenses_files` module, its digest-pinned `FROM` is a
recorded identity section, and the CA bundle entry must name the Alpine
release that `FROM` pins.
18. **A tree-shaken package that started shipping would have gone unnoticed.**
`cookie-es`, `isbot`, `seroval` and `seroval-plugins` are in the lockfile
closure and in no shipped byte. If application code imported one, no
lockfile, version or dependency set would change — only the bundle. The
frontend gate now recomputes the set of packages in `web/dist` from a
`--sourcemap` build and diffs it against a recorded section, and the drift
test requires every name in that section to be inventoried and refuses one
that is still listed as not shipped.
19. **The recorded npm licence token was parsed and discarded**, so a package
that relicensed passed as long as its version had not moved. Shipped
packages must now all carry the licence the inventory's texts assume;
`npm_not_shipped` is exempt, and `isbot` is Unlicense.
20. **The full-text licence checks were marker probes.** They prove the right
document is present but survive most of it being deleted. The Apache-2.0 and
MPL-2.0 texts are now pinned by SHA-256.
21. **Zig's compiler-rt contains code ported from LLVM's.** Zig's `LICENSE` is
bare MIT naming only "Zig contributors". Reviewed: the ports carry
Apache-2.0 WITH LLVM-exception, and that exception waives Apache §4(a),
§4(b) and §4(d) for portions embedded in object form — the only form nxdns
ships — so no further notice is owed. Recorded in the Zig inventory note
rather than left as an unexamined gap.
22. **`npx` was replaced by the installed binary** in the new frontend gate. `npx`
downloads a package it cannot find locally, so a wrong working directory
would have turned a licence check into an unpinned fetch. Reproduced: it
fetched `vite@8.2.0` over the pinned `8.1.5`.
### Not verified, and why
- **No workflow has ever executed.** `release.yml` and `gates.yml` were validated
by YAML parse and `bash -n`, plus two steps lifted out and run directly: the
registry probe against a fake registry (five response shapes) and the whole
bundled-package check against the real `web/` build, proven able to fail.
Everything else that talks to the registry or the Gitea API — `buildx build
--push`, `imagetools`, draft creation, asset upload, publication, the
adopt-an-existing-tag path — is unexercised.
- **`RELEASE_SIGNING_FPR` is still the placeholder.** By design: the guard fails
closed on it. It also means the release workflow cannot succeed as committed
until manual prerequisite 2 is done.
- **The aarch64 binary was never executed.** No `qemu-aarch64` on the build host,
so `verify-dist`'s aarch64 version check legitimately skips and `test-aarch64`
could not run. The binary is checked statically: ELF class, no `PT_INTERP`, no
`DT_NEEDED`, size.
- **The multi-architecture image build is unverified.** Only the native amd64
image was built and run.
- **`shellcheck` was not run** — not installed on the build host.
- **The local Node is 24.14.1, not the pinned 24.19.0.** There is no `.npmrc`, so
`engines` does not hard-fail, and the frontend gates ran under the older patch.
## Acceptance (milestone complete)
- [ ] `PLAN.md` §2.2 amended; the build-date line and the byte limits corrected.
- [ ] `zig build dist -Dversion-string=0.0.1 -Dweb-dist=web/dist` produces two
- [x] `PLAN.md` §2.2 amended; the build-date line and the byte limits corrected.
§3.15 now describes the single gate set, and a new §20 records the
publication model.
- [x] `zig build dist -Dversion-string=0.0.1 -Dweb-dist=web/dist` produces two
tarballs and a checksum file, and fails without `-Dweb-dist`.
- [ ] `zig build verify-dist` passes, and was proven able to fail: an oversized
- [x] `zig build verify-dist`, **given the same options as `dist`**, passes and
was proven able to fail: an oversized
binary, a dynamically linked binary, a mismatched `build.zig.zon` version
and a wrong archive mode each produce a named failure.
- [ ] Two runs of `zig build dist` on the same commit produce byte-identical
and a wrong archive mode each produce a named failure. Proven by repacking
tarballs with each defect: `binary-size`, `elf`, `zon-version` and
`archive-mode` each fired and the run exited 1. The oversize proof used a
genuinely oversized binary for `binary-size` and a lowered budget for
`asset-free-size`.
- [x] Two runs of `zig build dist` on the same commit produce byte-identical
tarballs **in the same directory**. (Cross-directory reproducibility is
ruling 12 and is not claimed here.)
- [ ] The image builds for both platforms with no qemu, carries `/LICENSE` and
`/THIRD-PARTY-NOTICES` and the OCI labels, and its binaries are
byte-identical to the tarball binaries.
byte-identical to the tarball binaries. Verified for the native amd64
image only; the arm64 half needs a runner with buildx.
- [ ] `gates.yml` runs from both `ci.yml` and `release.yml`; `ci.yml` triggers
on `master`; `origin/main` is gone.
- [ ] `THIRD-PARTY-NOTICES` covers musl, the Zig runtime, SQLite, Mbed TLS with
on `master`; `origin/main` is gone. The first two are in the files; no
workflow has run and `origin/main` still exists (manual prerequisite).
- [x] `THIRD-PARTY-NOTICES` covers musl, the Zig runtime, SQLite, Mbed TLS with
its Apache-2.0 selection line and full text, Everest, p256-m and the web
runtime closure. The dependency drift guard was proven able to fail.
runtime closure. The dependency drift guard was proven able to fail:
removing an inventory entry, staling a dependency version, staling the Zig
version, changing the base image digest, editing a pinned licence text and
dropping a package from the recorded bundle each produce a named failure.
- [ ] A dry run of `release.yml` completes with publication disabled.
- [ ] `v0.0.1` is published: five assets, a verifying signature, and an image at
`git.mial.net/mokhtar/nxdns:0.0.1` and `:latest`.
- [ ] `docs/how-to/verify-a-release.md` was followed end to end against the
published release, from a clean directory, on this host.
- [ ] No `zig build cross` or source-only-distribution text remains anywhere.
- [x] No `zig build cross` or source-only-distribution text remains on any
**active** surface: `build.zig`, the workflows, `deploy/`, `README.md` and
`docs/`. Historical milestone specs and `TECH_DEBT.md` keep their text —
this spec contains the string itself, so "anywhere" was never satisfiable.
## Anti-requirements