milestone 13 discrepancies: redact credentials from urls in logs, metrics and cli output

docs/reference/files-and-directories.md

@@ -16,19 +16,31 @@ never looks at the directory, and without one `check` falls back to the default
 configuration file, or prints "nothing to check" and exits 2 when neither source
 exists.
 
-No subcommand opens the directory read-only. `run`, `import`, `export` and a
-`check` that resolved to the database all go through the same
-`DataDir.openConfigDb`, which opens `config.db` read/write, chmods it to 0600,
-enables WAL — creating `config.db-wal` and `config.db-shm` — and then runs any
-pending schema migrations. So `nxdns export` and `nxdns check` write to the data
-directory, and on a database one schema version behind they migrate it. `export`
-additionally creates an empty `config.db` if the directory exists without one;
-`check` reaches the database branch only when `config.db` is already there.
+`run`, `import` and `export` go through `DataDir.openConfigDb`, which opens
+`config.db` read/write, chmods it to 0600, enables WAL — creating
+`config.db-wal` and `config.db-shm` — and then runs any pending schema
+migrations. So `nxdns export` writes to the data directory, and on a database
+one schema version behind it migrates it. `export` also creates an empty
+`config.db` if the directory exists without one.
+
+`nxdns check` is the exception: it does not use that path at all. It opens
+`config.db` immutable, which is `SQLITE_OPEN_READONLY` plus `immutable=1`, so
+SQLite refuses every statement that would write and builds no wal-index. No
+`config.db-wal` and no `config.db-shm` appear beside the file, the mode is left
+alone, and no migration runs — a database behind this binary's schema is
+reported as a failure naming `nxdns run` as the fix. A `check` against a data
+directory leaves it byte-identical, and `check` reaches the database branch only
+when `config.db` is already there.
+
+`immutable=1` ignores any `-wal` file, so it is refused rather than used when
+one holds bytes: the newest settings would be invisible and `check` would grade
+older ones from the main file. That is the "uncheckpointed changes" failure in
+[the CLI reference](cli.md#what-check-does-not-do).
 
 | Path | What it is | Mode |
 | --- | --- | --- |
 | `config.db` | The configuration database — the single source of truth, including `web.password_hash`. | 0600 |
-| `config.db-wal`, `config.db-shm` | SQLite write-ahead log and shared-memory index for `config.db`. Created when WAL is enabled, inheriting the main file's permissions. | 0600 |
+| `config.db-wal`, `config.db-shm` | SQLite write-ahead log and shared-memory index for `config.db`. Created by `run`, `import` and `export` when WAL is enabled, inheriting the main file's permissions. `check` creates neither. | 0600 |
 | `querylog.db` | The query log: every domain every client asked for. Expendable — if it is missing or unusable it is recreated empty. | 0600 |
 | `querylog.db-wal`, `querylog.db-shm` | WAL sidecars for `querylog.db`. | 0600 |
 | `querylog.db.corrupt-<unix-seconds>` | A `querylog.db` that could not be used, moved aside before an empty one was created in its place. Kept, never overwritten. | Whatever the renamed file had — no chmod reaches it |
@@ -38,11 +50,45 @@ additionally creates an empty `config.db` if the directory exists without one;
 | `blocklists/<id>.wild` | Wildcard entries for the same source. | 0600 |
 | `blocklists/<id>.raw.tmp`, `<id>.list.tmp`, `<id>.wild.tmp` | Transient refresh state: the downloaded body and the two compile outputs before they are published by rename. | 0600 |
 
-`<id>` is the `blocklist_sources` row id. A `.list` or `.wild` file whose id is
-no longer a `blocklist_sources` row is deleted by the orphan sweep; files of a
-live source are left alone whatever their state, and the sweep matches only the
-two published suffixes, so a `.tmp` file left by an interrupted refresh is not
-swept — the next refresh of that source overwrites it.
+`<id>` is the `blocklist_sources` row id.
+
+### The orphan sweep
+
+The sweep decides by id, not by suffix. It matches all five names above and
+deletes those whose `<id>` is no longer a `blocklist_sources` row, so the
+compiled `.list` and `.wild` of a removed source go, and so do a `.raw.tmp`,
+`.list.tmp` or `.wild.tmp` left behind by a refresh that was killed before it
+could clean up. Files belonging to a source that still has a row are never
+touched, whatever state they are in: the sweep holds the same lock every refresh
+takes, so it never reads the directory while a refresh is part-way through.
+
+It runs at three moments:
+
+- at startup, before the first refresh pass — this is what collects what a
+  killed process left behind, and the compiled files of a source deleted while
+  the server was down;
+- before each scheduled update pass, ahead of the disk-space gate: the sweep
+  only unlinks, so it is the one step here that can give a critically full disk
+  room back, and gating it would keep the residue that helped fill the disk;
+- immediately after `DELETE /api/blocklists/{id}`, which is when an orphan is
+  actually created in normal operation. Without it a deleted list would keep its
+  megabytes until the next scheduled pass.
+
+With `blocklist_update.enabled = false` there are no scheduled passes, so only
+the first and the last of those three happen.
+
+Each deletion is logged:
+
+```
+info(blocklist_manager): pruned orphaned blocklist file 9999.list
+```
+
+A failed sweep is a warning, not an outage — leftover bytes do not justify
+losing the refresh pass behind them, let alone the server.
+
+The temporaries of a source that still exists are cleaned by the refresh that
+owns them rather than by the sweep: each refresh deletes its own `.raw.tmp`,
+`.list.tmp` and `.wild.tmp` as it finishes, successfully or not.
 
 A `querylog.db` is moved aside when it is missing nothing but usability:
 SQLite reports it corrupt or not a database, `PRAGMA quick_check` does not
@@ -111,7 +157,9 @@ no file; under systemd the journal captures it.
 
 `doh_server.cert_path` / `key_path` and `dot_server.cert_path` / `key_path`,
 conventionally under `/etc/nxdns`. nxdns reads them, never writes or creates
-them. Both must be readable by the user nxdns runs as; the key should be
-readable by its owner only, which `nxdns check` warns about when it is not. A
+them. Both must be readable by the user nxdns runs as, and the key must belong
+to the certificate: `nxdns check` loads the pair and fails when it does not. The
+key should also be readable by its owner only, which `check` warns about when it
+is not. A
 watcher polls both files and swaps a renewed pair in without a restart. See
 [enable DoH and DoT](../how-to/enable-doh-and-dot.md).