milestone 15: make a green run mean a real pass
CI / test (push) Failing after 1m12s
CI / test-aarch64 (push) Failing after 2m27s
CI / frontend (push) Successful in 1m28s
CI / cross (push) Failing after 27s
CI / docker (push) Failing after 24s

This commit is contained in:
2026-08-07 01:28:13 +02:00
parent f2898479d4
commit 9f8a5cd753
16 changed files with 1070 additions and 31 deletions
+29
View File
@@ -35,3 +35,32 @@ What that means in practice:
commented-out code.
- Git: GPG-signed commits (`git commit -S`), simple lowercase messages, no
generated-by footers.
## Reading `zig build test` output
A fully passing `zig build test` still prints a line like `failed command:
.../test --cache-dir=... --seed=... --listen=-`, and still exits 0. That line
is a known upstream zig 0.16.0 labelling defect. It does not mean a test
failed, and no test binary crashed.
The build runner sets a step's `result_failed_command` on every spawn
(`std/Build/Step/Run.zig:1540`) and never clears it on success. It then prints
a step's diagnostics whenever the step wrote anything to stderr, explicitly "no
matter the result" (`compiler/build_runner.zig:1381`), and that printer emits
the `failed command: ` label unconditionally when the field is set
(`compiler/build_runner.zig:1515`). Our suite writes to stderr on every run,
because the tests that cover the warning paths log through the real sink. A
minimal reproducer with no mbedTLS and no C — one passing test whose body is a
`std.debug.print` — prints the same label and reports "3/3 steps succeeded;
1/1 tests passed"; deleting the print removes the label. No upstream issue
matched a search, so the reference is the 0.16.0 source lines above.
Any *other* failure text is real. Trust the summary line: `zig build test`
exiting non-zero, a `N failed` count, or a panic backtrace all mean a genuine
failure. Do not filter, wrap, or suppress the runner's output to hide the
label — that would hide real failures with it.
One trap: running a cached test binary by hand with `--listen=-` aborts with
`internal test runner failure: EndOfStream`. That is not a teardown bug; the
IPC runner is talking to a closed stdin because no build runner is on the other
end. Run the binary with no arguments to get the plain stdio report.