Files
nxdns/.gitea/workflows/gates.yml
T
mokhtar 22abcd9b7b release: nix flake with tag-pinned hashes, reproducible tarballs (milestone 40)
flake.nix fetches the release tarballs and carries their SRI hashes in a generated block. The cut tool builds the release locally with the toolchain gates.yml pins, in a normalized nine-variable environment, writes the hashes into flake.nix, and commits it with build.zig.zon as the single bump commit. The package job verifies the pins on the bump commit and the publish job verifies them again on the tag, before anything is uploaded.

The tarballs are written by dist_stage (std.tar.Writer, flate gzip) instead of the runner's tar and gzip, and -ffile-prefix-map keeps checkout paths out of the C objects; two checkouts at different absolute paths produce byte-identical archives. nxdns version, /api/version and the admin footer report the version only: the bump commit cannot know its own sha.
2026-09-08 21:45:22 +02:00

438 lines
21 KiB
YAML

name: Gates
# Every blocking check lives here, and nowhere else. ci.yml calls it on push
# and pull_request; release.yml calls it before it builds anything publishable
# (milestone-14 ruling 7). Keeping the packaging and container checks in this
# file — not only the three test jobs — is the point: a release must not be
# able to skip the checks that guard the artifacts it publishes.
#
# Every action here is pinned to a full commit SHA, on the same grounds as
# release.yml (ruling 7). The ruling's carve-out — "ci.yml may keep moving
# tags; it holds no secrets" — does not reach this file: release.yml *calls*
# it, so these jobs run on the same self-hosted runner host and against the
# same docker daemon as the job holding the signing subkey and the registry
# token. A moved tag upstream would execute there. ci.yml itself still uses no
# third-party action; its only `uses:` is this workflow.
#
# Resolved with `git ls-remote <repo> refs/tags/vN 'refs/tags/vN^{}'` — the
# three originals on 2026-08-07, the two artifact actions on 2026-08-12. Every
# tag pinned here is lightweight — no `^{}` peel row — so the listed SHA is the
# commit. download-artifact's `v3-node20` is the exception that proves the
# rule: it is annotated, and its peel row names a different commit than the tag
# object. This file pins `v3.1.0-node20`, which is lightweight.
#
# The artifact actions are pinned to v3, and to the node20 rebuilds of v3, for
# two separate reasons.
#
# v4 and v5 abort in the *client*, not the server. @actions/artifact asks
# isGhes(), Gitea answers the way GitHub Enterprise Server does, and the action
# refuses before it uploads anything. The server side is present: Gitea 1.27.1
# routes the whole v4 twirp service — CreateArtifact, FinalizeArtifact,
# ListArtifacts, GetSignedArtifactURL and DeleteArtifact all answer 401, where
# an unrouted path answers 404. The forks that patch isGhes() out are
# third-party code, which is what the pinning rule above exists to keep away
# from the job that holds the signing subkey.
#
# The stock v3 tags — upload ff15f030 (v3.2.1), download 9bc31d5c (v3.0.2) —
# declare `using: node16`, and a runner is not required to still ship a node16.
# The `-node20` rebuilds are that same action code on a runtime that exists.
on:
workflow_call:
env:
ZIG_VERSION: "0.16.0"
# Exact patch, not a floating "24" (milestone-14 ruling 12): the bundled npm
# and the emitted bundle change under a floating major.
NODE_VERSION: "24.19.0"
# The npm the cut runs and the npm CI runs must be one version: npm writes the
# admin bundle whose bytes the release hashes are pinned to before CI ever
# rebuilds them. setup-node installs the npm that ships with the node above,
# so this pin is asserted, not installed.
NPM_VERSION: "11.17.0"
# There is deliberately no CI_VERSION literal here. Ruling 2 allows the
# version to exist in the tag and in build.zig.zon and nowhere else, and
# ruling 5 makes verify-dist fail when the version under build disagrees with
# build.zig.zon. An invented CI string such as "0.0.0-ci" therefore cannot
# pass this file's own packaging gate. The package job reads the version out
# of build.zig.zon instead, and the container job takes it from that job's
# output.
jobs:
test:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- name: Set up Zig
uses: mlugg/setup-zig@d1434d08867e3ee9daa34448df10607b98908d29 # v2.2.1
with:
version: ${{ env.ZIG_VERSION }}
# zig 0.16.0's package fetcher creates tmp/<hex>.zip inside the global
# cache without creating tmp/ first (src/Package/Fetch.zig:1499), and
# setup-zig's restored cache never contains tmp/. Without this, every
# dependency fetch dies with "failed to create temporary zip file:
# FileNotFound" before any network I/O.
- name: Create the fetch temp dir zig assumes
run: mkdir -p "${ZIG_GLOBAL_CACHE_DIR:?}/tmp"
- name: Run test suite (unit + hermetic loopback integration)
run: zig build test -Dintegration
test-aarch64:
runs-on: ubuntu-24.04
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- name: Set up Zig
uses: mlugg/setup-zig@d1434d08867e3ee9daa34448df10607b98908d29 # v2.2.1
with:
version: ${{ env.ZIG_VERSION }}
# zig 0.16.0's package fetcher creates tmp/<hex>.zip inside the global
# cache without creating tmp/ first (src/Package/Fetch.zig:1499), and
# setup-zig's restored cache never contains tmp/. Without this, every
# dependency fetch dies with "failed to create temporary zip file:
# FileNotFound" before any network I/O.
- name: Create the fetch temp dir zig assumes
run: mkdir -p "${ZIG_GLOBAL_CACHE_DIR:?}/tmp"
# qemu-user, not qemu-user-static: Zig execs the bare `qemu-aarch64`
# name, and the -static package only ships `qemu-aarch64-static`.
- name: Install qemu-user
run: |
sudo apt-get update -qq
sudo apt-get install -qq -y --no-install-recommends qemu-user
# qemu 11.1.0 has a TCG regression: its translation-block optimization
# takes the panic branch of Zig's UBSan pointer-overflow check on a
# valid in-bounds pointer (11.0.3 and earlier are clean; bisected via
# the Arch archive). This image's qemu 8.2 is not affected — do not
# upgrade the emulator past 11.0.x until qemu fixes it.
- name: Run test suite under qemu (plain suite, no -Dintegration)
run: zig build test-aarch64 -fqemu
frontend:
# The light runner, a second act_runner at capacity 1 that advertises only
# this label. This job peaks around 355 MB (tsc), well inside that runner's
# 1536Mi dind limit, and it runs no docker command — so it overlaps the
# heavy runner's zig and image work instead of queueing behind it.
runs-on: ubuntu-24.04-light
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
- name: Set up Node
uses: actions/setup-node@49933ea5288caeca8642d1e84afbd3f7d6820020 # v4.4.0
with:
node-version: ${{ env.NODE_VERSION }}
cache: npm
cache-dependency-path: admin/package-lock.json
# The bundle is a release input whose hash is pinned before this run
# exists, so a runner on a different node or npm must fail here rather
# than emit different bytes further down.
- name: Assert the pinned Node and npm
run: |
test "$(node --version)" = "v${NODE_VERSION:?}"
test "$(npm --version)" = "${NPM_VERSION:?}"
test ! -e /nonexistent/npmrc-user
test ! -e /nonexistent/npmrc-global
# `npm ci` and `npm run build` run under the same normalized environment
# the cut builds the bundle in: exactly nine variables, a file mode from
# the umask, a C locale, UTC, and a zero build timestamp. Each of those
# can move the bytes the release hashes cover. The format, lint,
# typecheck and test steps below stay ambient on purpose: only the
# bundle's bytes are pinned, and those checks emit nothing that ships.
- name: Install dependencies
working-directory: admin
run: env -i PATH="$PATH" HOME="$HOME" LC_ALL=C LANG=C TZ=UTC SOURCE_DATE_EPOCH=0 CI=true npm_config_userconfig=/nonexistent/npmrc-user npm_config_globalconfig=/nonexistent/npmrc-global sh -c 'umask 022 && npm ci'
- name: Check formatting
working-directory: admin
run: npm run format:check
- name: Lint
working-directory: admin
run: npm run lint
- name: Typecheck
working-directory: admin
run: npm run typecheck
- name: Run tests
working-directory: admin
run: npm test
- name: Build
working-directory: admin
run: env -i PATH="$PATH" HOME="$HOME" LC_ALL=C LANG=C TZ=UTC SOURCE_DATE_EPOCH=0 CI=true npm_config_userconfig=/nonexistent/npmrc-user npm_config_globalconfig=/nonexistent/npmrc-global sh -c 'umask 022 && npm run build'
# The licence inventory has to cover every package whose bytes ship, and
# the lockfile does not answer that question: it lists what could be
# reached, not what rollup kept. The bundle is what this reads. The logic
# lives in admin/scripts/, unit-tested by `npm test`, so it runs on a laptop
# exactly as it runs here (milestone-14 deviation 24).
- name: Assert the packages bundled into admin/dist are the recorded ones
working-directory: admin
run: npm run assert-bundled
# The package and container jobs consume this bundle instead of building
# their own. The upload is last on purpose: a bundle that failed the
# licence inventory or any check above must not reach the jobs that
# package it into a tarball and an image.
#
# Artifact names are scoped to the run, so the ci.yml and release.yml
# runs that share this runner cannot read each other's.
#
# A later move to v4 has to add `include-hidden-files: true` here.
# `npm run build` writes the freshness stamp to the hidden file
# admin/dist/.src-hash (milestone-15 ruling 5), and v4.4.0 and later drop
# dotfiles by default. It is inactive today — v3 keeps them, and the
# package job's target path skips the stamp check regardless — but a move
# to v4 that also pointed the download back at admin/dist would fail with
# "admin/dist is stale".
- name: Upload the built web UI
uses: actions/upload-artifact@c24449f33cd45d4826c6702db7e49f7cdb9b551d # v3.2.1-node20
with:
name: admin-dist
path: admin/dist
if-no-files-found: error
package:
needs: [frontend]
runs-on: ubuntu-24.04
# The container job needs the version and installs no Zig, so it cannot read
# build.zig.zon the way this job does.
outputs:
version: ${{ steps.zon-version.outputs.version }}
steps:
# Depth 2, not the default 1: the pin check at the foot of this job runs
# only on the commit that changed build.zig.zon, and answering that
# question needs HEAD's first parent. No other job here reads a parent.
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
with:
fetch-depth: 2
- name: Set up Zig
uses: mlugg/setup-zig@d1434d08867e3ee9daa34448df10607b98908d29 # v2.2.1
with:
version: ${{ env.ZIG_VERSION }}
# zig 0.16.0's package fetcher creates tmp/<hex>.zip inside the global
# cache without creating tmp/ first (src/Package/Fetch.zig:1499), and
# setup-zig's restored cache never contains tmp/. Without this, every
# dependency fetch dies with "failed to create temporary zip file:
# FileNotFound" before any network I/O.
- name: Create the fetch temp dir zig assumes
run: mkdir -p "${ZIG_GLOBAL_CACHE_DIR:?}/tmp"
# `dist` refuses admin/dist-placeholder (ruling 4), so a real bundle has to
# exist before the packaging gate runs. It arrives from the frontend job,
# already formatted, linted, typechecked, tested and licence-checked.
#
# The target is deliberately not `admin/dist`: build.zig runs the freshness
# stamp check for that exact path and no other (milestone-15 ruling 5),
# and the check shells out to `node`. Here it would buy nothing — the
# stamp hashes the admin/ sources, not the bundle, so against a checkout of
# the same commit that built the bundle it can only agree. An explicit
# path is the case build.zig documents for a bundle built elsewhere, and
# taking it keeps node out of this job entirely.
- name: Download the web UI built by the frontend job
uses: actions/download-artifact@ad191675b41f6a5b46da9a048cb6893812da158b # v3.1.0-node20
with:
name: admin-dist
path: admin-dist-ci
# Ruling 2: build.zig.zon is the only place besides the tag that carries
# the version, and ruling 5 makes verify-dist assert the two agree. The
# gate builds the version the repository declares.
#
# This is the only job that reads it. The container job used to run its own
# `sed` over the same file; it now receives this step's output, so the two
# jobs cannot disagree about what the repository declares. The parse itself
# matches verify-dist's, through the zon grammar rather than a regex.
- name: Build the container gate tool
run: zig build container-check-tool
- name: Read the version from build.zig.zon
id: zon-version
run: ./zig-out/bin/container_check version
- name: Build the release artifacts
run: |
set -euo pipefail
zig build dist \
-Dversion-string="$CI_VERSION" \
-Dadmin-dist=admin-dist-ci \
-Doptimize=ReleaseSafe
# verify-dist owns every assert the CI shell used to make: ELF static
# linkage, the 15,728,640-byte stripped budget, the archive layout and
# modes, `nxdns version`, the build.zig.zon version, and the separate
# 10,485,760-byte asset-free build (ruling 5). That is why the objcopy
# and binutils-aarch64-linux-gnu install and the two inline size-assert
# shell blocks are gone from this file: the checks live in the build
# graph now and run identically on a laptop.
- name: Verify the release artifacts
run: |
set -euo pipefail
zig build verify-dist \
-Dversion-string="$CI_VERSION" \
-Dadmin-dist=admin-dist-ci \
-Doptimize=ReleaseSafe
# The cut writes the release hashes into flake.nix before it makes the
# bump commit, so the bump commit is the one commit whose pins nothing has
# verified yet — the tag's own run (release.yml) is the next chance, and by
# then the tag is public. This step is that first chance.
#
# It is deliberately NOT part of verify-dist. Every other commit on master
# builds the same build.zig.zon version from a different tree, so its bytes
# legitimately differ from the pins and a check there would fail the whole
# branch. The commit is identified by the version it declares, not by its
# message: a message is a string anyone can write, and the pins follow
# the manifest.
#
# A root commit has no first parent. That is an error rather than a skip:
# this repository has history, so `HEAD^` failing means the checkout is
# shallower than the depth 2 declared above and the question went
# unanswered, which must never read as "nothing to check".
#
# The predicate is the declared VERSION, not the file: build.zig.zon also
# carries the dependency pins, and updating a sqlite or mbedTLS hash
# changes the file without cutting a release. Such a commit builds the
# same version from a different tree, so its bytes are not the pinned
# ones and this check would fail it.
#
# HEAD's version is CI_VERSION, parsed out of the working tree by the
# container gate tool. The parent's is read with `sed`, because that tool
# reads `build.zig.zon` at a fixed path and has no mode for a blob out of
# history. An empty parse is a failure, not a bump: it means the manifest
# moved and the question went unanswered.
- name: Verify the flake pins on a version bump
run: |
set -euo pipefail
parent="$(git rev-parse --verify HEAD^)"
parent_version="$(git show "$parent":build.zig.zon | sed -n 's/^[[:space:]]*\.version = "\([^"]*\)".*/\1/p')"
if [ -z "$parent_version" ]; then
echo "cannot read .version out of $parent:build.zig.zon" >&2
exit 1
fi
if [ "$parent_version" != "$CI_VERSION" ]; then
zig build verify-pins \
-Dversion-string="$CI_VERSION" \
-Dadmin-dist=admin-dist-ci \
-Doptimize=ReleaseSafe
else
echo "skipped: $GITHUB_SHA declares version $CI_VERSION and $parent already declared $parent_version, so it is not a version bump and its bytes are not the ones flake.nix pins"
fi
# deploy/docker/Dockerfile copies both of these trees and nothing else
# out of zig-out/dist: the binary comes from dist/bin/<triple>/, and
# /LICENSE and /THIRD-PARTY-NOTICES come from the matching dist/stage/
# directory, which the container job then hashes against the image. The
# tarballs and SHA256SUMS stay behind — no later job reads them, and
# verify-dist has already checked them here.
#
# The zip round-trip drops the executable bit. That is survivable only
# because the Dockerfile chmods the binary itself and the contents
# assertion compares sha256 of file contents, never modes. The archive
# modes are asserted by verify-dist, above, on the originals. The gate tool
# rides along in the same artifact and the container job chmods it back.
#
# Adding zig-out/bin/container_check moves the artifact's common root from
# zig-out/dist up to zig-out, which is why the container job restores into
# zig-out rather than zig-out/dist. The Dockerfile's COPY paths still
# resolve; they are relative to the repository root either way.
- name: Upload the staged payload for the container job
uses: actions/upload-artifact@c24449f33cd45d4826c6702db7e49f7cdb9b551d # v3.2.1-node20
with:
name: dist-payload
path: |
zig-out/dist/bin
zig-out/dist/stage
zig-out/bin/container_check
if-no-files-found: error
container:
needs: [package]
runs-on: ubuntu-24.04
env:
# Ruling 2 and ruling 5: one parse of build.zig.zon per run, done in the
# package job. This job installs no Zig and cannot repeat it.
CI_VERSION: ${{ needs.package.outputs.version }}
steps:
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
# Restored under zig-out because that is where the Dockerfile's COPY lines
# look for dist/, with the repository root as the build context, and
# because the payload also carries bin/container_check. This job compiles
# nothing and bundles nothing: the payload is the one the package job
# already built and verify-dist already checked, which is also the point —
# an image built from a second, independent `dist` run would prove nothing
# about the artifacts the release publishes.
- name: Download the staged payload built by the package job
uses: actions/download-artifact@ad191675b41f6a5b46da9a048cb6893812da158b # v3.1.0-node20
with:
name: dist-payload
path: zig-out
# The artifact zip carries no modes.
- name: Restore the gate tool's executable bit
run: chmod +x zig-out/bin/container_check
# Everything this gate decides — this run's docker object names, the
# ownership label, the image build, the contents assertion against the
# staged payload, and the smoke test with its topology detection and its
# absolute probe deadline — lives in tools/container_check.zig, along with
# the rationale comments that used to sit in this file. That is the same
# move release.yml made (milestone-14 deviation 24): logic in a `run:`
# block cannot be type-checked, run on a laptop, or covered by a test.
- name: Run the container acceptance gate
run: ./zig-out/bin/container_check gate
# The gate removes its own container and image on every exit path it
# survives. This step is the exit path for a cancelled job and for a step
# that died before the gate's own defers ran — on a long-lived self-hosted
# daemon those accumulate one layer set per run. The name pair only exists
# if the gate got as far as writing $GITHUB_ENV, so the label sweep covers
# the rest of this run.
#
# It covers no more than that. `always()` does not run when the runner or
# the pod itself dies, and the filter below names THIS attempt's label
# value exactly — deliberately, since a concurrent run of another
# repository must not be swept, but that also means a later attempt cannot
# collect an earlier one's leak. What the label buys for those cases is
# discovery, not recovery: `docker ps -a --filter
# label=net.mial.nxdns.ci` and the matching `docker images` list every
# object this workflow has ever left behind, with the repository, run and
# attempt that owns each one. Reclaiming them is a manual sweep today, and
# the hook a janitor job would use later.
- name: Remove this run's docker objects
if: always()
env:
CI_LABEL: net.mial.nxdns.ci=${{ github.repository }}/${{ github.run_id }}/${{ github.run_attempt }}
run: |
set -uo pipefail
if [ -n "${SMOKE_NAME:-}" ]; then
docker rm -f "$SMOKE_NAME" >/dev/null 2>&1 || true
fi
if [ -n "${CI_IMAGE:-}" ]; then
docker image rm -f "$CI_IMAGE" >/dev/null 2>&1 || true
fi
for cid in $(docker ps -aq --filter "label=$CI_LABEL"); do
docker rm -f "$cid" >/dev/null 2>&1 || true
done
for iid in $(docker images -q --filter "label=$CI_LABEL"); do
docker image rm -f "$iid" >/dev/null 2>&1 || true
done
exit 0