//! The one redaction an operator-supplied string passes through before it //! reaches a log line or an operator-facing diagnostic. //! //! It lives at the root rather than under `filter/` because both the blocklist //! manager and the configuration validator print urls, and a `config/` module //! importing `filter/` would be the wrong dependency direction. It takes bytes //! and returns bytes: no `Io`, no allocator, no failure mode. const std = @import("std"); /// The longest text one `SafeUrl` or one `QuotedText` prints, ahead of the `...` /// that marks a truncation. An operator-supplied url or name has no length /// limit and a log line must have one. A `QuotedText`'s two quotes fall outside /// the count: they are the delimiter rather than the text, and a truncated name /// still closes the one it opened. /// /// The count is of printed characters, not of source bytes: an escape sequence /// costs the two or four characters it prints, not the one byte it stands for. /// A string of control characters therefore truncates early instead of printing /// several times its length. /// /// This is deliberately a second number rather than a reuse of the blocklist /// manager's `max_url_len`, which sizes the inline url copy inside /// `SourceStatus`. The two answer different questions — how wide one log line /// may be, and how large a status value a caller may keep is — and they agree /// on 255 only because that width suits both. Neither may be changed on the /// other's account. pub const max_len: usize = 255; /// The only form of a url that may be written to a log. `format` prints the /// scheme, the host and the port; it drops the userinfo, the path, the query /// and the fragment; it escapes every control character; and it bounds the /// result at `max_len`. /// /// Where the line falls, and why it falls there. A blocklist source url is /// operator-supplied and nothing on the way in restricts it to a bare public /// path: `?apikey=…`, a signed url whose signature is a query parameter, /// `https://user:pass@host/list.txt` and `https://host/d//hosts.txt` all /// validate, import and store. Userinfo, path, query and fragment are the four /// components where a credential can legally live, and a log line outlives the /// process — journald and the configured log file keep it, and neither is as /// protected as the database row the url came from. The rule against writing a /// secret to a log is absolute, so all four go. /// /// Dropping the path costs the url the one job it used to do here: telling two /// sources on one host apart. That job was never the url's. A caller names the /// source it reports on from the source's own identity — `filter/manager.zig` /// prints the row id and the name beside this — and an identity read from the /// row cannot leak what the url holds. /// /// What it still prints, deliberately: the scheme, the host and the port, /// because an operator reading a failure has to know where the source points; /// and, for a url that names no scheme at all, whatever precedes the first `/` /// once the userinfo is removed, because nothing in such a string distinguishes /// a host from anything else and that text is the closest thing to one. A /// string that names a scheme without a `//` after it gets no such treatment, /// whether or not it holds an `@` — see `redact`, where the text after such a /// scheme is a path segment under at least one reading and is never printed. /// /// The host is therefore the one place a credential still survives this, and it /// is not hypothetical: NextDNS identifies an account by a profile id, which its /// DoH url carries in the path — `https://dns.nextdns.io/abcd12`, dropped here — /// and its DoT url carries in the hostname — `tls://abcd12.dns.nextdns.io`, /// kept. Nothing can be done about the second without printing no host at all, /// which would leave every line unactionable. An operator whose provider puts a /// secret in the hostname has published it to every resolver on the way to it /// long before it reaches this function. /// /// It scans and never parses, so it cannot fail. `error.BadUrl` — the error /// `std.Uri.parse` raises — is one of the failures reported through here, so /// the inputs a parser refuses are exactly the inputs this must still redact. /// Only `://` introduces an authority, though: a run of one separator, of three /// or more, or of two that are not both `/`, leaves text that RFC 3986 reads as a /// path and WHATWG may read as a host, and `redact` prints no host it cannot /// settle. A `\` still ends an authority and still anchors the scheme scan — see /// `isSeparator` — it just does not open one. /// /// Scanning a string a parser refuses means some of those strings have more /// than one reading, and the readings disagree about which side of an `@` the /// host is on. `https://lists.example?token=prefix@hunter2` is one: RFC 3986 /// ends the authority at the `?`, so the host is `lists.example` and `hunter2` /// is part of an api key; read the `@` as a userinfo delimiter instead and the /// host is `hunter2`. No scan resolves that, and for a while this one chose the /// second reading and printed `https://hunter2` — a query-string secret printed /// as a host, which is the exact leak this file exists to stop. `authority` is /// therefore `null` on such a url and `format` prints /// `(ambiguous authority omitted)` in place of a host. Withholding the /// authority is always available and never wrong; choosing a side is wrong half /// the time, and the half it is wrong in is the half that leaks. pub const SafeUrl = struct { /// The scheme without its `:`, or empty when the url carries no scheme /// delimiter. scheme: []const u8, /// The authority without its userinfo — the host and, when present, the /// port — or `null` when the url admits two readings of where the host is. /// Empty and `null` are different answers: empty says the url names no /// authority, `null` says it names one this cannot resolve. authority: ?[]const u8, /// Unquoted. A caller that prints this inside a delimiter of its own must /// use `redactQuoted` instead, or handle the delimiter itself the way /// `web/metrics.zig` does for a Prometheus label value. /// /// This renders `scheme://authority` canonically; it does not quote the /// input's own syntax. A scheme is always followed by `://`, so /// `tls:\\host` prints `tls://` and `https://?apikey=…` prints `https://`, /// though only the second contained a `//`. The `://` marks where a host /// would go, and the operator's remedy is the same either way: the url names /// no host this can print. Nothing about which bytes separated the scheme /// from the rest survives redaction, and nothing should — the input is not /// reproducible from this and is not meant to be. pub fn format(self: SafeUrl, w: *std.Io.Writer) std.Io.Writer.Error!void { return self.write(w, .none); } /// A truncation ends the value and returns, so a caller that has written an /// opening delimiter still gets to write the closing one. fn write(self: SafeUrl, w: *std.Io.Writer, delimiter: Delimiter) std.Io.Writer.Error!void { var budget: usize = max_len; if (!try writeEscaped(w, self.scheme, &budget, delimiter)) return w.writeAll("..."); if (self.scheme.len != 0 and !try writeEscaped(w, "://", &budget, delimiter)) return w.writeAll("..."); const authority = self.authority orelse ambiguous_authority; if (!try writeEscaped(w, authority, &budget, delimiter)) return w.writeAll("..."); } }; /// A redacted url in the form a caller may print inside a log line or a /// diagnostic sentence, where it needs a delimiter to keep a host from running /// into the words around it. /// /// **`format` writes the quotes**, for the reason `QuotedText` states: a /// delimiter a caller adds is a delimiter the value can close. Redaction does /// not make that go away. A `'` is neither a component separator nor a control /// character, so it survives the scan into the authority — `https://ho'st/x` /// redacts to `https://ho'st`, and inside a caller's own `'…'` that reads as /// `'ho'` followed by loose text. The quote and its escape therefore live here, /// together, and a caller adds none of its own. /// /// A caller sizing a fixed buffer needs `max_len + 3` for the value, as /// `SafeUrl` does, plus the two quotes. pub const QuotedUrl = struct { url: SafeUrl, pub fn format(self: QuotedUrl, w: *std.Io.Writer) std.Io.Writer.Error!void { try w.writeByte('\''); try self.url.write(w, .single_quote); try w.writeByte('\''); } }; /// What `format` prints where a host would go when it cannot say which text is /// the host. It is prose rather than a placeholder host because an operator has /// to read it as a statement about the line and not as an address: `https://` /// alone already means "this url names no authority", and the two call for /// different actions. /// /// It spends the same printing budget the authority would have, so the bound /// `web/metrics.zig` sizes its buffer against still holds. /// /// An operator can write a source url whose redaction is this same text, since /// a url that is not a url prints as itself. That collision costs nothing: it /// makes one line say less about a source than it could, and it cannot make a /// credential read as a host, which is the direction that matters. const ambiguous_authority = "(ambiguous authority omitted)"; /// An operator-supplied string that is not a url — a blocklist source name — in /// the only form it may be written to a log. It holds no credential by design, /// so nothing is dropped from it; it comes out of a database row the same way a /// url does, so a control character in it can forge a log line the same way, /// and `format` escapes and bounds it for that reason alone. /// /// **`format` writes the quotes.** A caller printing a name inside a log line /// has to delimit it, or a name with a space in it runs into the words around /// it; and a delimiter a caller adds is a delimiter the name can close. `ads' /// (https://decoy.example) --` inside a caller's quotes produces a line naming a /// url no source has. So the quote and its escape live in one place, here, /// where they cannot drift apart. A caller adds none of its own. pub const QuotedText = struct { text: []const u8, pub fn format(self: QuotedText, w: *std.Io.Writer) std.Io.Writer.Error!void { var budget: usize = max_len; try w.writeByte('\''); if (!try writeEscaped(w, self.text, &budget, .single_quote)) try w.writeAll("..."); try w.writeByte('\''); } }; /// The `SafeUrl` of `url`. Both fields borrow from `url`, which every caller /// holds for the length of the call it prints in. pub fn redact(url: []const u8) SafeUrl { const first_sep = std.mem.indexOfAny(u8, url, separators) orelse url.len; const first_colon = std.mem.indexOfScalar(u8, url, ':') orelse url.len; // A scheme delimiter is a colon immediately before the first separator of // the whole string. Anchoring on the first separator is what keeps `a/b:/c` // from reading as a scheme. It anchors on a `\` too, so a url pasted with // backslashes still has its scheme recognised and echoed, even though a `\` // no longer opens an authority. var scheme: []const u8 = ""; var rest = url; var delimited = false; if (first_colon + 1 == first_sep and isScheme(url[0..first_colon])) { scheme = url[0..first_colon]; var after = first_sep; while (after < url.len and isSeparator(url[after])) after += 1; // Exactly `//` introduces an authority, and nothing else does. One // separator leaves an absolute path — RFC 3986 reads `https:/hunter2` as // the path `/hunter2`, WHATWG reads `hunter2` as the host — and three or // more is an empty authority to the first and a host to the second. // // A run of two that is not two slashes depends on the scheme. WHATWG // converts a `\` to a `/` only for a *special* scheme, so `https:\\host` // is contested the same way, while `tls:\\host` — `tls` is not special — // has no reading at all under which `host` is a host. RFC 3986 gives `\` // no meaning anywhere. // // So the two answers are different answers, and `authority` carries the // difference: `null` where the readings disagree, empty where they agree // the url names no authority. Either way the path segment stays out of // the log, which is the property that matters; this decides only what the // line then claims about it. // // An earlier revision accepted any number of separators, to keep // `https:/user:pass@host/list` from printing its userinfo. That reason // expired when the `/` cut moved ahead of the userinfo lookup: the // authority of a url with no `://` now ends at its first `/`, so it holds // no userinfo to print. Withholding it is both safe and the honest // answer. if (after - first_sep != 2 or url[first_sep] != '/' or url[first_sep + 1] != '/') return .{ .scheme = scheme, .authority = contestedOrAbsent(scheme, url[after..]) }; rest = url[after..]; delimited = true; } // A network-path reference names an authority and no scheme (RFC 3986 // §4.2). Without this the `/` cut below lands at byte zero, the authority is // empty, and the line reports nothing at all — including for // `//user:pa55@lists.example/x`, where the host is not in doubt. // Exactly `//` here too, and for the same reason. RFC 3986 reads an authority // after `//` and nothing else — `///hunter2/x` is an empty authority and the // path `/hunter2/x`, and `\\hunter2\x` is a path outright. WHATWG resolves a // reference against a base, and against a special-scheme base its // ignore-slashes state reads `hunter2` as the host in both. A run that is not // exactly `//` is therefore contested, not settled, and it is withheld rather // than reported as an authority the url does not have. // // A run of one is settled: both readings make it a path. So is any run whose // candidate authority is empty — `\\?\C:\lists\hosts.txt` ends the authority // at its `?` under the reading that looks for one, so neither finds a host // and there is nothing to contest. if (!delimited) { var run: usize = 0; while (run < rest.len and isSeparator(rest[run])) run += 1; if (run >= 2) { if (candidateAuthority(rest[run..]).len == 0) return .{ .scheme = scheme, .authority = "" }; if (run != 2 or rest[0] != '/' or rest[1] != '/') return .{ .scheme = scheme, .authority = null }; rest = rest[2..]; delimited = true; } } // The authority ends at the first `/`. Everything from there on is path, // query or fragment, and none of the three is printed — so an `@` after // that slash is not userinfo by any reading, and no longer needs to be one // to stay out of the log. const authority = rest[0 .. std.mem.indexOfScalar(u8, rest, '/') orelse rest.len]; // A scheme with no separator after it is an opaque path under RFC 3986 and, // for a special scheme, a host under WHATWG — so `https:hunter2` is either a // path segment, which is where a token lives, or a host. The disagreement is // the whole of the evidence, exactly as it is for `https:a@hunter2`, and this // check runs before the `@` lookup so both readings reach it. // // No exception is made for a suffix that looks like a port. An earlier // revision took the digits in `localhost:8080` for one, which also let // `https:123456` through, whose digits are an opaque path and as much a token // as any other text. Neither is resolved now. // // Note what the two cases are not. `https:` is a WHATWG special scheme, so // `https:hunter2` is contested — an opaque path to RFC 3986, a host to // WHATWG. `localhost:` is not special, so `localhost:8080` is a scheme and an // opaque path to *both*, and what an operator meant by it — a host and a // port — is a reading no parser offers. It is withheld all the same, because // the text after the colon is a path segment under every reading and a path // segment is never printed. // // So the two get the same treatment and different answers: `https:hunter2` is // withheld as contested, `localhost:8080` as naming no authority at all. // // An `IP:port` is unaffected: a leading digit fails the scheme production, so // `10.0.0.2:8080` and `[::1]:853` resolve. if (!delimited) { const trimmed = cut(authority); const colon = std.mem.indexOfScalar(u8, trimmed, ':') orelse trimmed.len; if (colon != trimmed.len and isScheme(trimmed[0..colon])) return .{ .scheme = scheme, .authority = contestedOrAbsent(trimmed[0..colon], trimmed[colon + 1 ..]), }; } // With no `@` there is no userinfo to remove and no side to choose. A `?`, // a `#` or a `\` ends the authority; each of the three ends it under every // reading of the text before it. const at = std.mem.lastIndexOfScalar(u8, authority, '@') orelse return .{ .scheme = scheme, .authority = cut(authority) }; // A `?`, a `#` or a `\` in front of that `@` makes the authority ambiguous, // and the two readings put the host on opposite sides of the `@`. On // `https://lists.example?token=prefix@hunter2` the text after it is an api // key; on `https://user:pa55?@host` the text before it is a password. Both // readings are available on both urls and nothing in either string tells // them apart, so neither side may be printed. if (std.mem.indexOfAny(u8, authority[0..at], "?#\\") != null) return .{ .scheme = scheme, .authority = null }; // An `@` is a userinfo delimiter only inside an authority. The scan knows one // is there in exactly three cases: a scheme delimiter introduced it, a `//` // did, or the string names no scheme at all, where this file's rule is that // the text before the first `/` is the closest thing to a host. The remaining // case — a scheme with no separator after it — was withheld above, before the // `@` was looked for, because it is ambiguous whether or not an `@` is in it. return .{ .scheme = scheme, .authority = cut(authority[at + 1 ..]) }; } /// `text` up to the first `?`, `#` or `\`, each of which ends an authority. A /// `\` is here rather than in `redact`'s `/` cut because the `/` cut runs before /// the userinfo is located and a `\` before an `@` is not a separator under /// every reading — `https://user:pa55\@host` is ambiguous, not hierarchical. fn cut(text: []const u8) []const u8 { return text[0 .. std.mem.indexOfAny(u8, text, "?#\\") orelse text.len]; } /// The `QuotedText` of `text`, which it borrows for the length of the call that /// prints it. The result prints its own quotes; see `QuotedText`. pub fn quoteText(text: []const u8) QuotedText { return .{ .text = text }; } /// The `redact` of `url`, quoted. /// /// The rule for choosing between the two, so it does not have to be rediscovered /// per call site: **use this whenever anything follows the url on the line.** A /// redacted authority can still hold a space, a `:` and a `'`, so unquoted it can /// impersonate whatever comes next — `upstream {f} failed: {t}` with a url whose /// authority is `ok failed: Timeout` reports a failure that did not happen. /// /// `redact` is for the two cases where that cannot arise: the url ends the line, /// or the caller owns the escaping for a delimiter of its own, as /// `web/metrics.zig` does for a Prometheus label value. /// /// The result prints its own quotes; see `QuotedUrl`. pub fn redactQuoted(url: []const u8) QuotedUrl { return .{ .url = redact(url) }; } /// What ends a url's components. `/` is RFC 3986's. `\` is here because the /// strings this scan exists for are the ones a parser refuses: /// `https:\\dns.nextdns.io\abcd12` carries an account identifier after a `\`, /// and a scan that read only `/` printed the whole of it. /// /// It ends a component; it does not open an authority. What that url *names* is /// contested — path text to RFC 3986, which gives `\` no meaning, and a host to /// WHATWG, which reads `\` as `/` for a special scheme — so `redact` withholds /// it. The `\` still matters here because both readings agree the account /// identifier after it is not part of any host. /// `null` when the readings disagree about whether `after` holds a host, empty /// when they agree it holds none. Both withhold; they differ in what the line /// claims, and `SafeUrl.authority` documents that as a real distinction. /// /// Known imprecision, in the safe direction. Three shapes return `null` where /// both readings in fact find no host, so the line says "could not resolve" /// where "names none" is the truth: /// /// - `https:/user@/x` — emptiness is decided before the userinfo is removed, /// so `user@` counts as a candidate host when the host after it is empty. /// - `\path@hunter2` — a leading run of one is settled as a path under both /// readings, but reaches the late-delimiter rule instead of returning here. /// - `file:secret` — `file` is in `special_schemes`, but WHATWG gives it its /// own parsing states in which that input is a local path with no host. /// /// Each prints *less* than it could, never more; none prints the path segment. /// Fixing them means modelling more of two standards for inputs no accepted /// configuration can hold: every url this program takes carries `http`, `https`, /// `tls`, `udp` or `tcp` and a `//`, so all three shapes reach a line only as /// something the validator is already rejecting by field path. /// /// Only a WHATWG special scheme reads an authority out of text a `//` did not /// introduce, so only a special scheme can disagree with RFC 3986 here. And /// nothing is contested when the reading that looks for a host finds none — /// `https:` alone names no authority under either. fn contestedOrAbsent(scheme: []const u8, after: []const u8) ?[]const u8 { if (candidateAuthority(after).len == 0) return ""; return if (isSpecialScheme(scheme)) null else ""; } /// The host a WHATWG-style reading would take out of `after`, used only to tell /// an empty one from a non-empty one. fn candidateAuthority(after: []const u8) []const u8 { return cut(after[0 .. std.mem.indexOfScalar(u8, after, '/') orelse after.len]); } /// WHATWG's special schemes: the ones whose urls it reads an authority into /// without a `//`, and whose backslashes it converts to slashes. /// /// This is a closed set fixed by the URL Standard, not a list of what this /// program supports. That is the difference between it and the known-scheme list /// an earlier revision removed: this one cannot go stale when nxdns learns a new /// transport, because it never described nxdns in the first place. const special_schemes = [_][]const u8{ "ftp", "file", "http", "https", "ws", "wss" }; fn isSpecialScheme(scheme: []const u8) bool { for (special_schemes) |special| { if (std.ascii.eqlIgnoreCase(scheme, special)) return true; } return false; } const separators = "/\\"; fn isSeparator(c: u8) bool { return std.mem.indexOfScalar(u8, separators, c) != null; } /// Whether `text` is a url scheme: an ASCII letter followed by letters, digits, /// `+`, `-` and `.` — RFC 3986's production, which is what a scheme delimiter /// has to look like before the text in front of it may be dropped as one. fn isScheme(text: []const u8) bool { if (text.len == 0) return false; if (!std.ascii.isAlphabetic(text[0])) return false; for (text[1..]) |c| { if (std.ascii.isAlphanumeric(c)) continue; if (c == '+' or c == '-' or c == '.') continue; return false; } return true; } /// Writes `text` with every control character escaped, spending at most /// `budget` printed characters and never splitting an escape sequence. Returns /// whether the whole of `text` was written. /// /// The control escapes are the ones `platform/logging.zig` uses on a whole log /// message, byte for byte, and that is deliberate. `delimiter` adds the one /// escape that sink has no reason to make. /// /// Two layers escape the same bytes here and neither is redundant. Do not /// delete this one on the grounds that the log sink already covers it: that /// sink covers every `std.log` line and nothing else, and a redacted url does /// not only reach a sink. `config/validate.zig` builds `Problem.message` as an /// allocated string; `web/handlers/mutations.zig` prints that message into the /// body of the 400 from `POST /api/blocklists`, and `cli.zig` prints it to the /// stdout of an interactive `nxdns check`. Control bytes baked into that string /// reach an HTTP response and an operator's terminal, neither of which any log /// sink is in a position to escape. A url has to arrive safe rather than be /// made safe by whatever it is written to. /// /// The cost is that a url inside a log line is escaped twice: `\n` in the row /// prints as `\\n` in journald and as `\n` on stdout. One notation across both /// is what keeps that legible. /// /// A `\` is escaped for the same reason it is there: `\n` in the output then /// means the byte this function replaced and `\\n` means two characters an /// operator typed. It is what makes `\'` unambiguous as well. fn writeEscaped( w: *std.Io.Writer, text: []const u8, budget: *usize, delimiter: Delimiter, ) std.Io.Writer.Error!bool { const hex = "0123456789abcdef"; for (text) |byte| { var hex_buf: [4]u8 = undefined; const escape: ?[]const u8 = switch (byte) { '\\' => "\\\\", '\n' => "\\n", '\r' => "\\r", '\t' => "\\t", '\'' => if (delimiter == .single_quote) "\\'" else null, 0x00...0x08, 0x0b, 0x0c, 0x0e...0x1f, 0x7f => blk: { hex_buf = .{ '\\', 'x', hex[byte >> 4], hex[byte & 0x0f] }; break :blk &hex_buf; }, else => null, }; if (escape) |seq| { if (budget.* < seq.len) return false; try w.writeAll(seq); budget.* -= seq.len; } else { if (budget.* < 1) return false; try w.writeByte(byte); budget.* -= 1; } } return true; } /// The character the caller of `writeEscaped` wraps the escaped text in, which /// is therefore the one character beyond the control set that has to be escaped /// inside it. `.none` is a value nothing wraps: a `SafeUrl` is printed bare, and /// escaping a `'` in a host would say a `'` there means something it does not. const Delimiter = enum { none, single_quote }; // --------------------------------------------------------------------------- // tests // --------------------------------------------------------------------------- const testing = std.testing; fn expectRedacted(expected: []const u8, url: []const u8) !void { var buf: [8 * max_len]u8 = undefined; try testing.expectEqualStrings(expected, try std.fmt.bufPrint(&buf, "{f}", .{redact(url)})); } test "redact drops the query, the fragment and the userinfo" { // The credential shapes a source url can hold: an api key in the query, a // signed url whose signature is a query parameter, and userinfo. try expectRedacted( "https://lists.example", "https://lists.example/hosts.txt?apikey=s3cr3t", ); try expectRedacted( "https://cdn.example", "https://cdn.example/l/hosts.txt?Expires=1700000000&Signature=abc123&Key-Pair-Id=K2", ); try expectRedacted( "https://lists.example", "https://user:pa55@lists.example/hosts.txt", ); try expectRedacted( "https://lists.example:8443", "https://token@lists.example:8443/hosts.txt?t=1#frag", ); } test "redact drops a credential carried in a path segment" { // The path is a place a token lives — a per-subscriber download url is the // common shape — and the rule against writing a secret to a log does not // bend for the component it sits in. try expectRedacted( "https://lists.example", "https://lists.example/download/token/hunter2/hosts.txt", ); // NextDNS: the path segment is the account identifier, so this is the shape // the rule exists for rather than an invented one. try expectRedacted("https://dns.nextdns.io", "https://dns.nextdns.io/abcd12"); try expectRedacted("https://dns.nextdns.io", "https://dns.nextdns.io/abcd12/mydevice"); // Its DoT form puts the same id in the hostname, where redaction cannot // reach it. Pinned so the limit stays visible rather than being discovered. try expectRedacted("tls://abcd12.dns.nextdns.io", "tls://abcd12.dns.nextdns.io"); try expectRedacted("https://lists.example", "https://lists.example/hunter2"); // An `@` in the path is not userinfo, and no longer has to be told apart // from it: the path goes either way. try expectRedacted("https://lists.example", "https://lists.example/@who/hosts.txt"); } test "redact keeps everything an operator needs to know where a source points" { try expectRedacted("https://lists.example", "https://lists.example/hosts.txt"); try expectRedacted("http://10.0.0.2:8080", "http://10.0.0.2:8080/a/b.txt"); try expectRedacted("https://lists.example", "https://lists.example?apikey=s3cr3t"); // No host: what is left still names the scheme. try expectRedacted("https://", "https://?apikey=s3cr3t"); } test "redact holds on the malformed urls a parser refuses" { // These reach the log through `error.BadUrl`, so scanning has to hold where // `std.Uri.parse` gives up. A scheme delimiter of one separator, of three, // and of none at all: each once carried the userinfo into the log, because // the scan looked for `://` and found no authority without it. // // Each is now withheld rather than resolved. An earlier revision read the // text after a run of any length as the authority, which drops the userinfo // on these four but prints the path segment of `https:/hunter2` as a host. // Only a run of exactly two says an authority is there; what these hold after // one, or after three, is a path to RFC 3986 and a host to WHATWG. The // property the line asserts is unchanged — no userinfo reaches the log — and // it now holds by withholding rather than by resolving. try expectRedacted("https://(ambiguous authority omitted)", "https:/user:hunter2@host/list"); try expectRedacted("https://(ambiguous authority omitted)", "https:///user:hunter2@host/list"); try expectRedacted("HTTPS://(ambiguous authority omitted)", "HTTPS:/user:hunter2@host/list"); try expectRedacted("https://(ambiguous authority omitted)", "https:\\user:hunter2@host\\list"); // A separator of none at all is withheld for the same reason. try expectRedacted("(ambiguous authority omitted)", "https:user:hunter2@host/list"); try expectRedacted("", "?apikey=s3cr3t"); try expectRedacted("not a url", "not a url"); try expectRedacted("", ""); // A colon that is not a scheme delimiter does not make one. try expectRedacted("lists.example", "lists.example/a:/b"); try expectRedacted("", "/download/token/hunter2/hosts.txt"); } test "redact treats a backslash as a hierarchical separator" { // A url typed or pasted with backslashes reaches these lines through // `error.BadUrl`, so the scan has to cut on one. NextDNS again, because the // path segment it carries is the whole account identifier. // A `\` ends an authority but never opens one. WHATWG converts it to a `/` // only for a special scheme, and RFC 3986 gives it no meaning at all, so none // of these names a host that both readings agree on — and `tls:\\host` has no // reading at all that makes it one. The account identifier stays out of the // line either way, which is the property this test is for. try expectRedacted("https://(ambiguous authority omitted)", "https:\\\\dns.nextdns.io\\abcd12"); try expectRedacted("https://(ambiguous authority omitted)", "https:/\\dns.nextdns.io\\abcd12"); try expectRedacted("https://(ambiguous authority omitted)", "https:\\/dns.nextdns.io\\abcd12"); try expectRedacted("https://(ambiguous authority omitted)", "https:\\dns.nextdns.io\\abcd12"); try expectRedacted("tls://", "tls:\\\\abcd12.dns.nextdns.io"); // A token in a backslash path goes the way a token in a `/` path goes. try expectRedacted( "https://lists.example", "https://lists.example\\download\\token\\hunter2\\hosts.txt", ); // The `\\` does not open an authority, so this is reported by its scheme // alone. What the assertion is really for is that the token in front of the // `@` does not reach the line, and withholding delivers that at least as // well as resolving did. try expectRedacted( "https://(ambiguous authority omitted)", "https:\\\\token@lists.example:8443\\hosts.txt?t=1#frag", ); // A backslash separator does not make a scheme out of a colon that is not // one, exactly as a `/` does not. try expectRedacted("lists.example", "lists.example\\a:\\b"); try expectRedacted("", "\\download\\token\\hunter2\\hosts.txt"); } test "redact resolves a backslash authority only where one reading survives" { // A `\` after the userinfo ends the authority under the WHATWG reading and // is an illegal host byte under RFC 3986's, so both readings agree that // nothing after it is a host. Cutting there prints less than either. try expectRedacted("https://host", "https://user@host\\list"); // But only once a `//` has established that an authority is there at all. A // leading `\\` does not, so the userinfo is withheld with everything else // rather than cut out of a host that was never settled. try expectRedacted( "https://(ambiguous authority omitted)", "https:\\\\user:hunter2@host\\list", ); // A `\` in front of the `@` is the ambiguous shape instead, and used to // print the text on one side of it: `https:\\lists.example\path@evil` gave // `https://evil`. See "redact omits an authority it cannot resolve". try expectRedacted( "https://(ambiguous authority omitted)", "https:\\\\lists.example\\path@evil", ); // A Windows path is not a url and leaves nothing that names a host. The // line it appears on still carries the source's row id and name. try expectRedacted("", "\\\\?\\C:\\lists\\hosts.txt"); } test "redact omits an authority it cannot resolve" { // The shape this exists for: a query parameter whose value holds an `@`. // The text after that `@` is the query, which is where an api key lives, and // a scan that read it as the end of a userinfo printed the key where the // host goes — having already dropped the real host. try expectRedacted( "https://(ambiguous authority omitted)", "https://lists.example?token=prefix@hunter2", ); try expectRedacted( "https://(ambiguous authority omitted)", "https://lists.example?user=a@b.example&key=hunter2", ); try expectRedacted( "https://(ambiguous authority omitted)", "https://lists.example#f@hunter2", ); try expectRedacted( "https://(ambiguous authority omitted)", "https:\\\\lists.example\\p@hunter2", ); // The mirror image, which the previous ordering fixed and this keeps fixed: // the text before the `@` is a credential just as often, so neither side may // be printed. Both of these once printed `user:pa55` as the host. try expectRedacted( "https://(ambiguous authority omitted)", "https://user:pa55?@host/x", ); try expectRedacted( "https://(ambiguous authority omitted)", "https://user:pa55#@host/x", ); try expectRedacted( "https://(ambiguous authority omitted)", "https://user:pa55\\@host/x", ); try expectRedacted( "https://(ambiguous authority omitted)", "https:\\\\user:pa55?@host\\x", ); // A url naming a scheme without a separator after it has no authority by // RFC 3986 and one by the WHATWG parser, so the `@` in it is a path // separator under the first reading and a userinfo delimiter under the // second. `hunter2` is a path segment or a host and no scan can say which. try expectRedacted("(ambiguous authority omitted)", "https:a@hunter2"); try expectRedacted("(ambiguous authority omitted)", "https:user:hunter2@host/list"); // `user` is not one of WHATWG's special schemes, so both readings make this a // scheme and an opaque path and neither finds a host. It names no authority // rather than one that cannot be resolved, and the password is withheld by // the same rule either way. try expectRedacted("", "user:pa55@lists.example/hosts.txt?apikey=s3cr3t"); // A url with no scheme at all prints the marker on its own, which is still // not a url and still not a host. try expectRedacted("(ambiguous authority omitted)", "lists.example?token=a@hunter2"); } test "redact tells an omitted authority apart from an absent one" { // Three outcomes an operator has to be able to tell apart, because the // action each calls for differs: a host, no host, and a host this cannot // name. Only the third withholds anything. try expectRedacted("https://lists.example", "https://lists.example/hosts.txt"); try expectRedacted("https://", "https://?apikey=s3cr3t"); try expectRedacted("https://(ambiguous authority omitted)", "https://?u=a@hunter2"); try expectRedacted("lists.example", "lists.example/hosts.txt"); try expectRedacted("", "?apikey=s3cr3t"); try expectRedacted("(ambiguous authority omitted)", "?u=a@hunter2"); } test "redact still resolves an authority whose delimiters follow the userinfo" { // The ambiguity is an `@` after a `?`, a `#` or a `\`, not an `@` at all. // Where the delimiters fall the way a url puts them, the host is not in // doubt and withholding it would cost an operator the line's whole point. try expectRedacted("https://lists.example", "https://user@lists.example?apikey=s3cr3t"); try expectRedacted("https://lists.example", "https://user:pa55@lists.example#frag"); try expectRedacted("https://lists.example:8443", "https://token@lists.example:8443\\hosts.txt"); try expectRedacted("https://lists.example", "https://user@lists.example/x?u=a@b"); // No `@` in the authority: a `?` still ends it, and nothing is ambiguous. try expectRedacted("https://lists.example", "https://lists.example?apikey=s3cr3t"); } test "redact escapes the control characters that would forge a log line" { // A row can be written by a path that does not validate as strictly as the // config validator, so the manager prints whatever the column holds. A // newline in it would end the line and start one of the operator's // choosing. try expectRedacted( "https://lists.example\\n2026-01-01 ERROR forged", "https://lists.example\n2026-01-01 ERROR forged/hosts.txt", ); try expectRedacted("https://a\\rb", "https://a\rb/x"); try expectRedacted("https://a\\tb", "https://a\tb/x"); try expectRedacted("https://a\\x00b", "https://a\x00b/x"); try expectRedacted("https://a\\x7fb", "https://a\x7fb/x"); // An ESC would reach a terminal as a control sequence on the one path the // log sink does not cover: `cli.zig` prints a diagnostic to stdout. try expectRedacted("https://a\\x1bb", "https://a\x1bb/x"); // A literal backslash ends the authority, so `redact` cannot print one at // all. The doubling that keeps an escape sequence unambiguous is exercised // where a backslash does survive: `quoteText`, below. try expectRedacted("https://a", "https://a\\nb/x"); } test "redact bounds the line it prints at max_len" { const long_host = "h" ** (2 * max_len); var buf: [8 * max_len]u8 = undefined; const printed = try std.fmt.bufPrint(&buf, "{f}", .{redact("https://" ++ long_host ++ "/x")}); try testing.expectEqualStrings(("https://" ++ long_host)[0..max_len] ++ "...", printed); // The bound counts what is printed, so an escape cannot spend four // characters of a log line per byte of url. const control_host = "\n" ** max_len; const escaped = try std.fmt.bufPrint(&buf, "{f}", .{redact("https://" ++ control_host ++ "/x")}); try testing.expect(escaped.len <= max_len + 3); try testing.expect(std.mem.endsWith(u8, escaped, "...")); try testing.expect(!std.mem.containsAtLeast(u8, escaped, 1, "\n")); // A scheme long enough on its own truncates inside the scheme rather than // printing it whole and starting on the host. const long_scheme = "s" ** (2 * max_len); const truncated = try std.fmt.bufPrint(&buf, "{f}", .{redact(long_scheme ++ ":/host")}); try testing.expectEqualStrings(long_scheme[0..max_len] ++ "...", truncated); } test "quoteText escapes and bounds an operator-supplied name" { var buf: [8 * max_len]u8 = undefined; try testing.expectEqualStrings( "'ads and trackers'", try std.fmt.bufPrint(&buf, "{f}", .{quoteText("ads and trackers")}), ); try testing.expectEqualStrings( "'ads\\n2026-01-01 ERROR forged'", try std.fmt.bufPrint(&buf, "{f}", .{quoteText("ads\n2026-01-01 ERROR forged")}), ); // A name cannot close the quote around it and write what follows as though // it were another field of the line. try testing.expectEqualStrings( "'ads\\' (https://decoy.example) --'", try std.fmt.bufPrint(&buf, "{f}", .{quoteText("ads' (https://decoy.example) --")}), ); // Nor by escaping the escape: a `\` before the quote is doubled first, so // `\'` in the output is this function's and never the operator's. try testing.expectEqualStrings( "'ads\\\\\\' (https://decoy.example) --'", try std.fmt.bufPrint(&buf, "{f}", .{quoteText("ads\\' (https://decoy.example) --")}), ); // Nor by running past `max_len`: the truncation closes the quote too. const long_name = "n" ** (2 * max_len); try testing.expectEqualStrings( "'" ++ long_name[0..max_len] ++ "...'", try std.fmt.bufPrint(&buf, "{f}", .{quoteText(long_name)}), ); try testing.expectEqualStrings( "'" ++ "\\'" ** (max_len / 2) ++ "...'", try std.fmt.bufPrint(&buf, "{f}", .{quoteText("'" ** max_len)}), ); } test "redact withholds a scheme whose separator is missing, with or without an @" { // RFC 3986 reads `hunter2` as an opaque path and a path segment is where a // token lives; WHATWG inserts the missing `//` for a special scheme and reads // it as the host. An earlier revision withheld this only when an `@` was // present, so the plainer shape printed the path whole. try expectRedacted("(ambiguous authority omitted)", "https:hunter2"); try expectRedacted("(ambiguous authority omitted)", "https:a@hunter2"); try expectRedacted("(ambiguous authority omitted)", "https:123456"); // Only a WHATWG special scheme can disagree with RFC 3986 here, because only // a special scheme reads an authority out of text no `//` introduced. For // every other scheme both readings say the same thing — a scheme and an // opaque path — so these name no authority rather than an unresolved one. // Withheld either way; what differs is what the line then claims. try expectRedacted("", "mailto:ops@example.com"); try expectRedacted("", "localhost:8080/hosts.txt"); try expectRedacted("", "localhost:8080@evil/x"); // An `IP:port` is unaffected, because a leading digit fails the scheme // production and there is nothing to disagree about. try expectRedacted("10.0.0.2:8080", "10.0.0.2:8080/hosts.txt"); try expectRedacted("[::1]:853", "[::1]:853/x"); try expectRedacted("lists.example", "lists.example/hosts.txt"); } test "redact takes exactly two separators as an authority delimiter" { // One separator leaves an absolute path: RFC 3986 reads `https:/hunter2` as // the path `/hunter2`, WHATWG reads `hunter2` as the host. Three or more is // an empty authority to the first and a host to the second. An earlier // revision accepted any run and printed the path segment as the host. try expectRedacted("https://(ambiguous authority omitted)", "https:/hunter2"); try expectRedacted("https://(ambiguous authority omitted)", "https:///hunter2"); try expectRedacted("localhost://", "localhost:/hunter2"); try expectRedacted("https://lists.example", "https://lists.example/hosts.txt"); // The userinfo this tolerance was introduced to protect is protected by // withholding instead. The `/` cut runs ahead of the userinfo lookup now, so // the authority of a url with no `://` ends before any `@` it holds. try expectRedacted("https://(ambiguous authority omitted)", "https:/user:pass@host/list"); } test "redact resolves a network-path reference instead of reporting nothing" { // The `/` cut lands at byte zero here, so without the `//` branch every one // of these redacted to the empty string and the line named no source at all. try expectRedacted("lists.example", "//lists.example/hosts.txt"); // Exactly two. A longer run is contested — an empty authority to RFC 3986, // and `lists.example` as the host to WHATWG resolving against a // special-scheme base — so it is withheld rather than reported as a url that // names no authority. Those are different answers and this type keeps them // apart. try expectRedacted("(ambiguous authority omitted)", "///lists.example/x"); // The authority is not in doubt, so the userinfo is dropped rather than the // whole of it withheld. try expectRedacted("lists.example", "//user:pa55@lists.example/hosts.txt"); // An ambiguous one is still withheld: the branch says where the authority // starts, not that every reading of it is settled. try expectRedacted("(ambiguous authority omitted)", "//a?b@c"); // A single leading `/` is a path, not an authority, and still names none. try expectRedacted("", "/path/only"); } test "the shapes redact over-withholds on print less, never more" { // The three imprecisions `contestedOrAbsent` documents. Each says "could not // resolve" where both readings in fact find no host, so each prints less than // it could. Pinned because the failure that matters is the other direction: // if one of these ever starts naming a host, this test says so. try expectRedacted("https://(ambiguous authority omitted)", "https:/user@/x"); try expectRedacted("https://(ambiguous authority omitted)", "https:/user:hunter2@/x"); try expectRedacted("(ambiguous authority omitted)", "\\path@hunter2"); try expectRedacted("(ambiguous authority omitted)", "file:secret"); try expectRedacted("file://(ambiguous authority omitted)", "file:\\secret"); // `file` still resolves where a `//` settles it, which is why the imprecision // is in the classification and not in the scan. try expectRedacted("file://host", "file://host/secret"); } test "redactQuoted writes its own quotes and closes them in every exit" { var buf: [8 * max_len]u8 = undefined; try testing.expectEqualStrings( "'https://lists.example'", try std.fmt.bufPrint(&buf, "{f}", .{redactQuoted("https://lists.example/hosts.txt")}), ); // The omitted-authority value is prose with spaces in it, which is the case // the quotes exist for: unquoted it runs into the words of the sentence // around it. try testing.expectEqualStrings( "'https://(ambiguous authority omitted)'", try std.fmt.bufPrint(&buf, "{f}", .{redactQuoted("https://lists.example?token=prefix@hunter2")}), ); // A truncation returns early, and the closing quote still has to be written // or the rest of the line reads as part of the value. const long_host = "h" ** (2 * max_len); try testing.expectEqualStrings( "'https://" ++ ("h" ** (max_len - "https://".len)) ++ "...'", try std.fmt.bufPrint(&buf, "{f}", .{redactQuoted("https://" ++ long_host ++ "/x")}), ); } test "a redacted authority cannot close the quote a caller would have added" { var buf: [8 * max_len]u8 = undefined; // A `'` is neither a component separator nor a control character, so it // survives redaction into the authority. `redact` leaves it, which is why a // caller may not supply the quotes itself. try testing.expectEqualStrings( "https://ho'st", try std.fmt.bufPrint(&buf, "{f}", .{redact("https://ho'st/x")}), ); try testing.expectEqualStrings( "'https://ho\\'st'", try std.fmt.bufPrint(&buf, "{f}", .{redactQuoted("https://ho'st/x")}), ); // The forging shape, whole: an operator-supplied url that ends the value and // writes what follows as though the line had said it. try testing.expectEqualStrings( "'https://ho\\' is fine; upstreams[9] '", try std.fmt.bufPrint(&buf, "{f}", .{redactQuoted("https://ho' is fine; upstreams[9] /x")}), ); // The escape-the-escape shape that `QuotedText` has to defend against does // not arise here, and not because the escaper is different — it is the same // one. A `\` ends an authority, so it never reaches the value to be doubled. // This is asserted rather than assumed: it is the property that makes a // single `\` in the output always this file's and never the operator's. try testing.expectEqualStrings( "'https://ho'", try std.fmt.bufPrint(&buf, "{f}", .{redactQuoted("https://ho\\'st/x")}), ); }