milestone 15: make a green run mean a real pass
This commit is contained in:
@@ -35,3 +35,32 @@ What that means in practice:
|
||||
commented-out code.
|
||||
- Git: GPG-signed commits (`git commit -S`), simple lowercase messages, no
|
||||
generated-by footers.
|
||||
|
||||
## Reading `zig build test` output
|
||||
|
||||
A fully passing `zig build test` still prints a line like `failed command:
|
||||
.../test --cache-dir=... --seed=... --listen=-`, and still exits 0. That line
|
||||
is a known upstream zig 0.16.0 labelling defect. It does not mean a test
|
||||
failed, and no test binary crashed.
|
||||
|
||||
The build runner sets a step's `result_failed_command` on every spawn
|
||||
(`std/Build/Step/Run.zig:1540`) and never clears it on success. It then prints
|
||||
a step's diagnostics whenever the step wrote anything to stderr, explicitly "no
|
||||
matter the result" (`compiler/build_runner.zig:1381`), and that printer emits
|
||||
the `failed command: ` label unconditionally when the field is set
|
||||
(`compiler/build_runner.zig:1515`). Our suite writes to stderr on every run,
|
||||
because the tests that cover the warning paths log through the real sink. A
|
||||
minimal reproducer with no mbedTLS and no C — one passing test whose body is a
|
||||
`std.debug.print` — prints the same label and reports "3/3 steps succeeded;
|
||||
1/1 tests passed"; deleting the print removes the label. No upstream issue
|
||||
matched a search, so the reference is the 0.16.0 source lines above.
|
||||
|
||||
Any *other* failure text is real. Trust the summary line: `zig build test`
|
||||
exiting non-zero, a `N failed` count, or a panic backtrace all mean a genuine
|
||||
failure. Do not filter, wrap, or suppress the runner's output to hide the
|
||||
label — that would hide real failures with it.
|
||||
|
||||
One trap: running a cached test binary by hand with `--listen=-` aborts with
|
||||
`internal test runner failure: EndOfStream`. That is not a teardown bug; the
|
||||
IPC runner is talking to a closed stdin because no build runner is on the other
|
||||
end. Run the binary with no arguments to get the plain stdio report.
|
||||
|
||||
Reference in New Issue
Block a user