Files
nxdns/docs/reference/query-log-lifecycle.md
T
mokhtar 0f01c2fbd7
Gates / frontend (push) Successful in 2m8s
Gates / test (push) Successful in 2m46s
Gates / test-aarch64 (push) Successful in 8m38s
Gates / package (push) Successful in 4m39s
Gates / container (push) Successful in 15s
CI / gates (push) Successful in 31m58s
storage: version querylog.db and migrate it in place, never reset a healthy file
querylog.db carries a schema version; migrations run at startup as one transaction after a vacuumed 0600 backup, and every failure refuses startup (exit 2, no systemd restart loop) instead of starting empty. corruption is the only automatic recreate left. the cut gate now requires a fixture-proven migration or an explicit versioned break with restore instructions, and locks shipped migration files and fixtures byte-for-byte.
2026-08-28 17:56:19 +02:00

6.5 KiB

The query log's lifecycle

What happens to querylog.db when nxdns opens it: how the file is versioned, when it is migrated, when the server refuses to start over it, and the one case in which it is still replaced. Source of truth: src/storage/querylog_versions.zig (the version metadata), src/storage/querylog_schema.zig (the open path) and src/storage/querylog_migrations.zig (the migration runner).

The rule this page exists to state: a healthy querylog.db is never replaced and never moved aside. A schema this build cannot use refuses the startup instead. Your query history is not the server's to discard.

The version stamp

Every querylog.db carries a logical schema version in SQLite's PRAGMA user_version. It is a small counter — 1 in this release — and not a hash of anything. A file created by this build is stamped as it is created.

Two other values matter, both in querylog_versions.zig:

Constant Today What it means
current_version 1 The version this build creates and reads.
minimum_supported_version 1 The oldest stamped version this build can migrate up to current_version.
legacy_fingerprint 1975011655 The user_version the 0.0.12 and 0.0.13 binaries wrote: a CRC32 of their schema text, under the older policy where a mismatch meant "replace the file".

legacy_fingerprint is frozen forever. Those two releases stamped a hash rather than a version, so this build recognises that one literal number as "version 1" and restamps the file as 1 on the first open. The restamp runs in its own transaction; if it fails, the old stamp and every row stay exactly as they were and the startup refuses.

What an open does

nxdns opens querylog.db once at startup, before it serves anything, and no second process shares a data directory. On a file that is readable and passes PRAGMA quick_check, the stamp decides:

Stamp What happens
current_version Opens. Nothing is migrated.
legacy_fingerprint Read as version 1: restamped to 1, then treated as version 1 by the rows above and below.
Between minimum_supported_version and current_version Migrated in place, then opens.
Above current_version, up to 1000000 REFUSE: SchemaTooNew.
Anything else — 0, a negative, another fingerprint, a version below the minimum REFUSE: SchemaUnsupported.

A refusal changes nothing. The schema, the rows, the coverage watermark and the stamp are all left as they are, no file is set aside, no new file is created, and nxdns run exits. The log line names the path, the stamp it found, the range this build supports and the troubleshooting section.

Migrating in place

A migration is one backup and one transaction.

  1. Back up. VACUUM INTO writes a complete copy — including anything still only in the write-ahead log — to querylog.db.pre-migrate-<unix-seconds> beside the database. If that name is taken, -2, -3 and so on are tried. A backup that cannot be written is MigrationBackupFailed, and the partial copy is deleted; an older backup beside it survives.
  2. Migrate. BEGIN IMMEDIATE, re-read the stamp under the lock, run every step, run PRAGMA foreign_key_check, stamp the new version, COMMIT. One transaction covers the whole chain, so the file is either at the old version or at the new one and never in between.
  3. Clean up. Every other querylog.db.pre-migrate-* beside the file is deleted. One backup is kept: the one this migration just took. A later successful start retries that cleanup if it failed.

If a step fails before the commit, the transaction rolls back, this run's backup is deleted, and the startup refuses with MigrationFailed. The database keeps the version and the rows it had.

If the commit succeeds and something after it fails, the log says so plainly — the migration DID complete and the file IS at the new version. The backup is kept, the startup still refuses with MigrationFailed, and the next start opens the migrated file normally.

Corruption is the only automatic recreate

Four conditions still create a fresh, empty querylog.db: the file is missing, SQLite reports it as corrupt, it is not a SQLite database at all, or PRAGMA quick_check does not answer ok. Except for the missing case, the unusable file is renamed to querylog.db.<reason>-<unix-seconds> and kept. See why a query log is moved aside.

Every other failure — a lock held elsewhere, a permission problem, a full disk, a version this build cannot reach — propagates and leaves the file alone.

Downgrading

Downgrading to 0.0.13 or older resets your query log. Those binaries predate this contract: they compare user_version against a hash of their own schema text, find this build's version stamp instead, and treat that as a mismatch — so they rename querylog.db to querylog.db.schema-changed-<unix-seconds> and start an empty log. Nothing is destroyed, but the live log is empty until you put the aside file back, and the restamp that provoked it takes no backup of its own.

To recover, go back to a migration-aware release, stop the server, then, in the data directory:

  1. Move the empty querylog.db the old binary created out of the way.
  2. Delete its querylog.db-wal and querylog.db-shm. This is not optional: replaying the empty file's write-ahead log into the restored history would corrupt it.
  3. Rename querylog.db.schema-changed-<unix-seconds> back to querylog.db.
  4. Start the server.

Downgrading between two migration-aware releases is safe in the sense that matters: a build that finds a stamp above its own current_version refuses to start with SchemaTooNew and touches nothing. Go forward again, or restore the pre-migrate backup the upgrade left.

Breaking the schema on purpose

A release may still break the query-log schema outright rather than migrate it. That is allowed, and it is never silent. Such a release raises current_version, sets minimum_supported_version to the same value, and ships no migration step — so files from before the break classify as below the minimum and open refuses them with SchemaUnsupported rather than replacing them. The release notes carry the phrase resets your query history and a Restoring your query history section, and the cut gate refuses to build the release without both.

So the contract is: a break is always versioned, always refused at startup with the file intact, and always disclosed in the changelog.