220 lines
14 KiB
Markdown
220 lines
14 KiB
Markdown
# Zig 0.16.0 stdlib API notes (verified against ../zig at tag 0.16.0)
|
|
|
|
Ground truth for build sub-agents. Every claim below was read from the 0.16.0 source
|
|
(commit 24fdd5b7a4c1c8b5deb5b56756b9dbc8e08c86a8). When in doubt, re-check the source at
|
|
`/home/mokhtar/app/zig/lib/std/` — do not trust pre-0.16 knowledge; the std.Io migration
|
|
changed most of these APIs.
|
|
|
|
## std.Build (build.zig)
|
|
|
|
- `b.addExecutable(.{ .name, .root_module, ... })` — no target/optimize on the artifact.
|
|
Target/optimize/link_libc go on the module: `b.createModule(.{ .root_source_file, .target,
|
|
.optimize, .link_libc, .imports, ... })` (Module.CreateOptions, Module.zig:216).
|
|
- All C configuration is **module-level** (Step.Compile has none of these methods):
|
|
- `exe.root_module.addCSourceFile(.{ .file, .flags, .language })`
|
|
- `exe.root_module.addCSourceFiles(.{ .root, .files, .flags })` — `files` must be
|
|
**relative** paths; absolute paths panic.
|
|
- `exe.root_module.addIncludePath(lazy_path)`, `.addCMacro(name, value)`
|
|
- libc: `link_libc = true` in CreateOptions or `exe.root_module.link_libc = true`.
|
|
- Static lib: `b.addLibrary(.{ .linkage = .static, .name, .root_module })`
|
|
(`b.addStaticLibrary` does not exist). Link: `exe.root_module.linkLibrary(lib)`.
|
|
- Cross targets: `b.resolveTargetQuery(.{ .cpu_arch = .aarch64, .os_tag = .linux,
|
|
.abi = .musl })` or `std.Target.Query.parse(.{ .arch_os_abi = "x86_64-linux-musl" })`.
|
|
musl → static libc; for a fully static binary also set `exe.linkage = .static`.
|
|
- Options: `b.option(std.Build.LazyPath, "web-dist", "...")` accepts LazyPath.
|
|
`b.addOptions()` + `opts.addOption/addOptionPath` + `exe.root_module.addOptions("build_options", opts)`.
|
|
- Embedding an asset directory: one WriteFile step holding both a generated `index.zig`
|
|
and the assets: `const wf = b.addWriteFiles(); _ = wf.addCopyDirectory(dist, "assets", .{});
|
|
const idx = wf.add("index.zig", src);` then
|
|
`exe.root_module.addAnonymousImport("assets", .{ .root_source_file = idx })`.
|
|
`@embedFile` paths must stay inside that module root (`error.ImportOutsideModulePath`).
|
|
- Version pin: compare `@import("builtin").zig_version` (SemanticVersion) in comptime,
|
|
`@compileError` on mismatch.
|
|
- build.zig.zon: `.name` is an **enum literal** (`.nxdns`), `.fingerprint` is **required**
|
|
(compiler suggests the value on first run), plus `.version`, `.paths`, `.dependencies`
|
|
(`path`/`url`+`hash`/`lazy`), `.minimum_zig_version`.
|
|
|
|
## std.process + stdio (found during S1)
|
|
|
|
- `std.process.args` / `std.process.ArgIterator` **do not exist** in 0.16. Arguments arrive
|
|
through main's parameter: `pub fn main(init: std.process.Init) u8 { var args =
|
|
init.minimal.args.iterate(); ... }`. `std.process.Init` carries `.io`, `.gpa`, `.arena`,
|
|
`.environ_map`, `.preopens`, `.minimal` (`.args`, `.environ`).
|
|
See lib/std/process.zig:30 and lib/std/process/Args.zig.
|
|
- Console output: `std.Io.File.stdout().writer(io, &buffer)` then use `.interface`
|
|
(lib/std/Io/File.zig:91,600). Same shape for stderr.
|
|
- Package deps unpack into `zig-pkg/` inside the project root (gitignore it).
|
|
- Entropy: `io.random(buf)` (Io.zig:2468). Realtime timestamp: `std.Io.Clock.real.now(io)`
|
|
→ `Io.Timestamp` (Io.zig:778). There is NO `io.now(.real)`.
|
|
- `std.mem.indexOfScalar` is gone; use `std.mem.findScalar` (mem.zig:1219).
|
|
- Conditional imports are impossible: no `@hasImport`, and `@import("root")` in a
|
|
`zig test` build resolves to the compiler's test runner, never your tests root.
|
|
Integration tests gated on `build_options` therefore live in a separate file with a
|
|
runtime `if (!build_options.integration) return error.SkipZigTest;` guard (body stays
|
|
semantically analyzed either way — code cannot rot).
|
|
- ENVIRONMENT (this dev machine): IPv6 egress is broken — hostname-resolving tests hit
|
|
AAAA-first timeouts. Use documented IPv4 literals in live tests, keep SNI on the name.
|
|
- `zig fetch` accepts tar.gz/zip etc. but **not .tar.bz2** (no bzip2 decompressor —
|
|
src/Package/Fetch.zig ~line 1337). Pin GitHub *source tag* tarballs when a release
|
|
asset is bz2-only.
|
|
|
|
## std.Io (Threaded backend, concurrency)
|
|
|
|
- `var t = std.Io.Threaded.init(gpa, .{});` — returned by value, must live at a **stable
|
|
address**. `const io = t.io();` `defer t.deinit();` (joins workers).
|
|
InitOptions: `stack_size`, `async_limit` (default cpus-1), `concurrent_limit`.
|
|
- `io.async(f, args) Future(R)` may run inline; `io.concurrent(f, args) !Future(R)`
|
|
guarantees its own unit of concurrency — use for server loops.
|
|
`future.await(io)`, `future.cancel(io)`.
|
|
- `Io.Group`: `.init`, `g.async(io, f, args)`, `g.concurrent(io, f, args)`, `g.await(io)`,
|
|
`g.cancel(io)`. Group task fns must return something coercible to `Cancelable!void`.
|
|
Cancellation: next cancelable Io call returns `error.Canceled` once; `io.recancel()`
|
|
re-arms; `io.swapCancelProtection(.blocked)` protects cleanup sections.
|
|
- **No SIGINT/SIGTERM handling in Threaded** (only no-op SIGIO/SIGPIPE). Install our own
|
|
`posix.sigaction`, set an atomic flag, cancel the group from the main task.
|
|
|
|
## std.Io.net
|
|
|
|
- UDP: `addr.bind(io, .{ .mode = .dgram })` → `Socket`.
|
|
Receive: `sock.receive(io, buf) !IncomingMessage` — fields `from: IpAddress`,
|
|
`data: []u8` (slice into buf), `flags.trunc`. Timeout variant: `receiveTimeout(io, buf,
|
|
timeout)`. Send: `sock.send(io, &dest_addr, data)`. Close: `sock.close(io)`.
|
|
(No recvFrom/sendTo names.)
|
|
- TCP server: `addr.listen(io, .{ .reuse_address = true })` → `net.Server`;
|
|
`srv.accept(io) !Stream`; `srv.deinit(io)`. `Stream.close(io)`, `.shutdown(io, how)`.
|
|
Shutdown of the listener makes a blocked accept fail `error.SocketNotListening`.
|
|
- TCP client: `addr.connect(io, .{ .mode = .stream })` → `Stream` —
|
|
`ConnectOptions.mode` is REQUIRED (no default, net.zig:332).
|
|
`ConnectOptions.timeout` is a LANDMINE in 0.16.0: the Threaded backend panics
|
|
"TODO implement netConnectIpPosix with timeout" (Threaded.zig:12077). Never set it.
|
|
Bound connects the same way as reads: race the task against a `Clock.Duration.sleep`
|
|
via `std.Io.Select` and cancel the loser (see tls_client_integration_test.zig).
|
|
- **Stream reads/writes accept no timeout** in 0.16.0 (VTable netRead/netWrite have none;
|
|
Operation lacks net stream variants). Bound a TCP/TLS read by running it under
|
|
`io.concurrent` and cancelling the future (or shutdown the socket).
|
|
- Reader/Writer: `stream.reader(io, buf) Stream.Reader`; generic interface is
|
|
`&stream_reader.interface` (`*Io.Reader`); same for writer. Caller owns buffers; `&.{}`
|
|
legal. Concrete errors land in `.err`; interface returns ReadFailed/WriteFailed.
|
|
`Stream.Writer.sendFile` returns `error.Unimplemented` in 0.16.0.
|
|
- Known: net tests in stdlib are skipped (upstream issue 31388) — do not copy test.zig
|
|
patterns blindly.
|
|
|
|
## std.http.Client (DoH upstream)
|
|
|
|
- Construct by struct literal: `var client: std.http.Client = .{ .allocator = gpa, .io = io };`
|
|
`defer client.deinit();` Fields: `ca_bundle`, `ca_bundle_lock`, `now: ?Io.Timestamp`,
|
|
`connection_pool` (LRU, free_size 32), `read_buffer_size`, `write_buffer_size`.
|
|
- DoH POST flow (low-level; `fetch` hides the body—don't use it):
|
|
1. `var req = try client.request(.POST, uri, .{ .headers = .{ .content_type =
|
|
.{ .override = "application/dns-message" } } });`
|
|
2. `try req.sendBodyComplete(body_mut);` (body is `[]u8`, sets content-length + flush)
|
|
3. `var resp = try req.receiveHead(&redirect_buf);`
|
|
4. Check `resp.head.status`, `resp.head.content_type`.
|
|
5. `const rdr = resp.reader(&transfer_buf);` then read (`allocRemaining`/`readSliceShort`).
|
|
6. `req.deinit()` returns the connection to the pool.
|
|
- TLS: client TLS uses std.crypto.tls.Client internally with the client's `ca_bundle`;
|
|
lazily `rescan`s system roots when `client.now == null`
|
|
(`error.CertificateBundleLoadFailure` on failure). Pre-fill `ca_bundle` + set `now` to
|
|
skip the scan.
|
|
- **No per-request deadline** exists. Enforce total-budget timeouts via concurrent+cancel
|
|
(see std.Io note above).
|
|
|
|
## std.http.Server (web/API/DoH server)
|
|
|
|
- `var srv = std.http.Server.init(&reader.interface, &writer.interface);`
|
|
Loop for keep-alive: `while (srv.reader.state == .ready)` + catch HttpConnectionClosing.
|
|
- `var req = try srv.receiveHead();` — `req.head.method/.target/.content_type/...`,
|
|
`req.iterateHeaders()`. **Copy `head.target` before reading the body** (body reads
|
|
invalidate head memory). Body: `req.readerExpectContinue(buf)` (handles 100-continue)
|
|
or `req.readerExpectNone(buf)`.
|
|
- Simple response: `try req.respond(content, .{ .status = ..., .extra_headers = &.{...} });`
|
|
- Streaming/SSE: `var body = try req.respondStreaming(&.{}, .{ .respond_options = ... });`
|
|
With `content_length == null` → chunked. **Use an empty buffer** so each write drains
|
|
through; then `response.flush()` after each event suffices; `response.end()` to finish.
|
|
(With a non-empty buffer you must flush response.writer first — trap.)
|
|
- WebSocket upgrade exists (`upgradeRequested`/`respondWebSocket`) — unused by nxdns.
|
|
- No MIME table, no etag: set `content-type`/`cache-control` manually via extra_headers.
|
|
BodyWriter supports sendFile (contentLengthSendFile/chunkedSendFile).
|
|
|
|
## std.testing.fuzz (0.16)
|
|
|
|
- `std.testing.fuzz(context, testOne, options)` where `testOne: fn(ctx, smith: *std.testing.Smith) anyerror!void` —
|
|
the input is a `*Smith` structured generator, NOT a raw `[]const u8` slice (pre-0.16 form is gone).
|
|
- `FuzzInputOptions{ .corpus: []const []const u8 }` seeds the corpus.
|
|
- Smith surface (lib/std/testing/Smith.zig): `valueWithHash`, `valueRangeAtMostWithHash`,
|
|
`bytesWithHash`, `sliceWithHash(buf, hash) u32` (fills buf, returns length), `eosWithHash`, etc.
|
|
For raw-bytes parsers: fill a buffer via `sliceWithHash` and feed the prefix to the parser.
|
|
- Plain `zig build test` replays each corpus entry once + one empty input (deterministic).
|
|
`--fuzz=<n>` = iterations PER TEST (K/M/G suffixes); bare `--fuzz` = forever + webui;
|
|
no time bound exists; `--fuzz=<n>` conflicts with `--webui`.
|
|
- Corpus entries are Smith byte streams (slice = u32-LE length + bytes), not raw inputs.
|
|
- STOCK 0.16.0 CANNOT RUN `--fuzz`: (A) fuzz-mode test_runner.zig:566 has a type error
|
|
(needs patched --zig-lib-dir); (B) the self-hosted x86_64 backend emits no coverage
|
|
PCs — set `.use_llvm = true` on the fuzz test artifact.
|
|
- `zig test` collects tests ONLY from the root module — moving files into a named module
|
|
silently drops their tests. A file belongs to exactly one module per compilation, and
|
|
imports cannot escape the module root directory. Hence: aggregator imports files
|
|
directly; separate artifacts import a module-root file (e.g. src/dns/dns.zig).
|
|
|
|
## std.zon
|
|
|
|
- Parse: `std.zon.parse.fromSlice(T, gpa, src_z, ?*Diagnostics, .{})` for pointer-free T
|
|
(no free needed); `fromSliceAlloc` + `std.zon.parse.free(gpa, v)` for slice-bearing T.
|
|
Source must be `[:0]const u8`. No reader-based entry point.
|
|
Options: `ignore_unknown_fields`, `free_on_error`. Diagnostics: init `.{}`,
|
|
`deinit(gpa)`, print with `{f}`.
|
|
- Struct default field values apply for absent fields. Tagged unions: `.foo` (void) or
|
|
`.{ .foo = v }`. Enums, optionals (single level only — `??T` rejected), slices, arrays,
|
|
nested structs all fine. Error sets/unions, many-pointers, comptime_int rejected.
|
|
- Serialize: `std.zon.stringify.serialize(value, .{ .whitespace = true }, writer)`.
|
|
Streaming: `var s: std.zon.Serializer = .{ .writer = w, .options = .{} };` with
|
|
`beginStruct`/`field`/`end`.
|
|
|
|
## std.crypto.tls.Client (upstream DoT)
|
|
|
|
- `std.crypto.tls.Client.init(input: *Io.Reader, output: *Io.Writer, options) !Client`
|
|
Options: `.host = .{ .explicit = "dns.google" }` (or `.no_verification`),
|
|
`.ca = .{ .bundle = .{ .gpa, .io, .lock, .bundle } }` (or `.no_verification`/`.self_signed`),
|
|
`.write_buffer`, `.read_buffer` (input buffer ≥ `Client.min_buffer_len` =
|
|
`tls.max_ciphertext_record_len`), `.entropy: *const [240]u8` (fill via `io.random`),
|
|
`.realtime_now: Io.Timestamp`.
|
|
- After init: plaintext via `&client.reader` / `&client.writer` (held **by value** — the
|
|
Client must not move after init). `client.end()` sends close_notify. Errors in
|
|
`client.read_err` / `client.alert`.
|
|
- TLS 1.2/1.3. **No ALPN, no session resumption** (fine for DoT; DoH over HTTP/1.1 works
|
|
without ALPN in practice — verify against real upstreams in Phase 3).
|
|
- CA roots: `std.crypto.Certificate.Bundle` (note the path), `bundle.rescan(gpa, io, now)`.
|
|
|
|
## Certificate verification ignores IP SANs (verified 0.16.0)
|
|
|
|
`std.crypto.Certificate.Parsed.verifyHostName` (Certificate.zig:313) checks only
|
|
`dNSName` general names in the SAN extension; the switch's `else => {}` skips
|
|
`iPAddress` (tag 7) entries entirely. Consequence: a TLS connection whose
|
|
verification name is an IP literal (DoT `tls://1.1.1.1:853`) always fails with
|
|
`error.CertificateHostMismatch` against certificates that carry the address only
|
|
as an iPAddress SAN — which is how Cloudflare and Quad9 issue theirs. A DoT
|
|
upstream therefore needs a DNS `tls_name` for SNI + verification while dialing
|
|
the IP; verifying by bare IP cannot work on stock 0.16.
|
|
|
|
## Phase 6 verifications (concurrency, disk, files)
|
|
|
|
- `std.Io.Queue(Elem)` exists (Io.zig:2184): bounded ring buffer over a caller-supplied array.
|
|
`put(q, io, elems, min)` with `min = 0` never blocks and returns 0 when full (Io.zig:2218);
|
|
`getOne` blocks until an item or `error.Closed` after `close(io)`. Elements are copied as raw
|
|
bytes (`@ptrCast`, Io.zig:2189) — an element holding a slice transfers only the pointer, so
|
|
queue elements must be self-contained values.
|
|
- `std.Io.Condition` has NO `timedWait` (Io.zig:1653 — wait/waitUncancelable/signal/broadcast
|
|
only). The timed primitive is `std.Io.Event.waitTimeout` (Io.zig:1827).
|
|
- Disk free space: std has NO statvfs/statfs wrapper (no Statfs struct, no `f_bavail` anywhere in
|
|
lib/std). Options: `extern fn statvfs` against libc (we always link libc for sqlite) or raw
|
|
`std.os.linux.syscall2(.statfs, ...)` with a hand-written struct.
|
|
- File append mode does not exist on `Dir.OpenFileOptions`/`CreateFileOptions`. Append pattern:
|
|
`openFile(io, path, .{ .mode = .write_only })`, `file.writer(io, &buf)`, then
|
|
`w.seekTo(try file.length(io))`; the writer is positional. `File.setLength` truncates;
|
|
`Dir.rename` + `Dir.deleteFile` rotate. Positional writes do not serialize — one task owns the
|
|
log writer.
|
|
- No fixed-capacity hash map in std. Bounded cache shape: fixed slot array plus
|
|
`StringHashMapUnmanaged(u32)` from key to slot index (`ensureTotalCapacity` once at init);
|
|
a CLOCK hand walks the stable slot array, never the map. Any map modification invalidates
|
|
live iterators (hash_map.zig:496).
|