mokhtar/nxdns
1
0
Fork 0
Code Pull Requests Actions Packages Releases 13 Activity

milestone 20: declarative configuration for iac

Browse Source
This commit is contained in:
mokhtar
2026-08-11 23:31:40 +02:00
parent 2f29121e27
commit d76afc147a
74 changed files with 6722 additions and 1949 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/explanation/configuration-model.md
+183 -146
View File
@@ -1,8 +1,9 @@
# The configuration model
nxdns is configured two ways — a ZON file and a web UI — and only one of them
can be the truth. This page explains which, and why that choice is the one
that leaves the fewest ways to lose an operator's work.
can be the truth at a time. This page explains how that choice is made, what
each mode is for, and why the design leaves the fewest ways to lose an
operator's work.
For the fields themselves see
[reference/configuration.md](../reference/configuration.md); for the commands
@@ -10,146 +11,169 @@ and their exit codes see [reference/cli.md](../reference/cli.md).
## The rule
The database is the truth. The file is a seed.
**Authority is the invocation.**
`config.db` in the data directory holds the running configuration. The ZON
file (default `/etc/nxdns/config.zon`, overridable with `--config`) is read on
`nxdns run` only while the database is still empty: if the file exists and the
database holds no configuration, it is imported. Once the import has put rows
in, the file is not opened again.
```
nxdns run the database is the truth
nxdns run --config /etc/nxdns/config.zon the file is the truth
```
The condition is the state of the database, not a one-shot flag. A `run` whose
seed fails — unreadable file, parse error, failed validation, a constraint
violation inside the import transaction — leaves the database empty, so the
next `run` reads the file again. That is what makes fixing a typo and starting
again work.
That is the entire selection mechanism. There is no mode setting, no default
file path, and nothing recorded in the database about which mode last wrote it.
A `config.zon` that exists but that no invocation names changes nothing at all.
`src/config/bootstrap.zig` is that policy and nothing else — a wrapper over
the same import path `nxdns import` uses. Three outcomes:
Two properties fall out of that, and both were chosen on purpose.
- no file: the database is used as it is;
- file present, database empty: seed it;
- file present, database already configured: skip, without reading the file.
**An operator can read `ExecStart` and know which authority is live.** The
alternative — probe a well-known path, and behave differently depending on
whether a file happens to be there — is ambient magic. It is also the exact
class of rule that produced years of documentation lies in this project: the
old design read the file only while the database was empty, which meant the same
command did two different things depending on state nobody could see from the
command line, and every page that described it eventually described it wrongly.
A file that exists but is unreadable, unparseable or invalid fails the start,
with every problem printed. nxdns never falls back to silent defaults over a
file an operator wrote — a resolver that boots "successfully" with a
configuration nobody chose is the worst outcome available, because it looks
like it worked.
**A path already expresses a two-state choice, so a mode flag beside it would
be redundant and worse.** An earlier draft had `--config-source=db|file`. A mode
flag next to a path flag manufactures combinations that cannot mean anything —
a path with no mode, a mode with no path — and each one then needs a pairing
rule and a usage error to defend it. Presence-of-path has no invalid
combinations, so there is nothing to defend.
## Why the database wins
## Database mode
The alternative designs all lose data.
`nxdns run`. `config.db` holds the configuration; the UI, the API and `nxdns
import` write to it; nothing reads a file. This is the appliance: someone sets
the box up once, and afterwards the household member who wants to unblock one
domain clicks a button.
If the file were the truth, the admin UI could not write. Every change would
be an SSH session and a restart, which defeats the reason the UI exists: the
household member who wants to unblock one domain is not going to edit ZON.
A fresh install in this mode starts from an empty database, which fails
validation on its own terms — there is nowhere to forward a query to — and says
what to do about it:
If both were the truth, they would disagree. The UI writes a rule; the file
still says otherwise; the next restart either silently reverts the rule or
silently ignores the file. Both are silent, and both destroy work someone
intended to keep. There is no merge rule that fixes this, because the system
cannot know which of two conflicting statements is the newer intention.
```
nxdns run failed: NoUsableUpstreams
run `nxdns check` to see the configuration in full
load one with `nxdns import <file>`, or make a file the source of truth with `nxdns run --config <file>`
```
So the file's authority ends the moment the database has content. Editing
`config.zon` after first boot does nothing — no partial effect, no warning
that some fields took and others did not. That is a blunt rule, and it is the
point: the failure mode is "my edit did nothing", which is visible the first
time you look at the UI, rather than "my edit was applied and then quietly
undone next Tuesday".
## File mode
The emptiness check is a real query over the content tables, not a flag: the
database counts as configured when a content table holds rows, or when the
default group has been altered. What it measures is intent, not activity, and
the client table is where those two come apart. Clients are auto-materialised
when they first send a query — the DNS path writes a row per device it sees —
and such a row records what the network did, not what an operator decided. It
carries `hand_edited = 0`, the emptiness check counts only the `hand_edited = 1`
rows, and `export` omits them. So answering queries never turns an unconfigured
database into a configured one; naming a device does.
`nxdns run --config FILE`. The file is the sole declarative source, and the
database becomes the runtime substrate: every start reads the file, validates
it, converges the database onto it, and serves from there. Configuration writes
through the API are refused with a 403.
Counting traffic here would have been a quiet trap: a server that resolved one
name would have declared itself configured and ignored a seed file placed
afterwards, and the operator would have had no line of output saying why.
This is the mode for a file kept in git and pushed by Ansible. What it buys is
that the deployed file is what is running — not "was imported once", not
"was imported unless someone clicked something since".
The same distinction survives an import. `import` replaces the content in one
transaction, which empties `clients` along with every other table, so every
client row is saved before the wipe. The materialised ones are put back after it
unchanged, and restoring a backup does not make the server forget the devices it
has met.
Three properties make it usable rather than merely correct.
An address the imported file names belongs to the file — the operator's
statement wins over the discovered row — with one carve-out. First-seen and
last-seen are not configuration: they record when a device was heard from, the
configuration model has no field for either, and an import is not a query. So
they follow the address rather than the row. If the database already knew that
address, its two timestamps are carried onto the new row; only an address the
database has never seen takes the import's clock. Without that, re-importing a
backup would stamp every device the operator had bothered to name as though it
had just arrived — and those are exactly the devices whose history is worth
something.
**It fails closed.** A file that is missing, unreadable, unparseable, oversized
or invalid stops the start. nxdns never falls back to the database, because a
fallback turns a deploy typo into a configuration that is silently months old
and looks fine. That failure is exit 2, so `nxdns check --config FILE` is a real
pre-restart gate: validate the pushed file in the handler, and a typo is a
failed deploy at noon rather than a dead resolver at the next power cut.
**It converges rather than replaces.** Reconciling matches rows by identity and
writes only what differs. A source whose URL has not changed keeps its row id,
its checksum, its counters and its compiled blocklist files — so a restart in
file mode downloads nothing, which is the difference between a design that is
tolerable to restart and one that costs three minutes and 100 MB every time.
**An unchanged file writes nothing at all.** Not "writes the same bytes" —
performs zero write statements, and reports it:
```
reconciled '/etc/nxdns/config.zon': no changes
```
That matters beyond elegance. A box whose SD card is full of query log can still
restart in file mode, because a no-op reconcile needs no write-ahead-log
headroom.
**Converged at every boot is not a lock between boots.** Nothing stops `nxdns
import` or a `runtime action` route from moving the database while the server
runs. The contract is that the next start puts it back, and says what it
corrected.
## What the file cannot take away
The file is authoritative over configuration. It is not authoritative over
things it has no vocabulary for, and reconciling has to preserve those or the
mode is unusable.
- **Blocklist download state.** Checksums, fetch timestamps and domain counts
belong to the network, not the operator. They survive on every matched row.
- **Client history.** Devices nxdns saw on the wire are kept whole. Naming one
in the file promotes that row in place — it keeps its first-seen and
last-seen and its row id, and counts as an update rather than a delete and an
insert.
- **Devices whose group is un-declared.** Remove a group from the file and the
observed clients assigned to it move to `default`. The operator un-declared
the group, not the devices.
- **The password, when the file does not mention it.** See
[the password](#the-password).
The one thing identity cannot survive is a change to identity itself. Edit a
source's URL and the engine sees one row gone and one row arrived: new id, fresh
download, and the old compiled files swept. That is consistent — artifacts are
keyed by row id — and it is why the CLI asks for `--allow-delete` when a diff
deletes anything.
## Why the database is the substrate in both modes
Even in file mode the database is where the server reads its effective
configuration from. That is not a leftover; it is what lets one read path serve
both modes, and it is what makes the runtime state above have somewhere to live.
If the file were read directly on every query path there would be no place to
keep a checksum, and no way for the UI to show anything. If both file and
database were authoritative there would be a two-way merge, and a merge cannot
work here: when the UI writes a rule and the file still says otherwise, nothing
in the system knows which of two statements is the newer intention. Every
resolution silently destroys work someone meant to keep.
So the modes are exclusive, and the failure mode of each is loud. In database
mode, editing the file does nothing, which you find out the first time you look
at the UI. In file mode, the UI refuses the edit to your face with a message
naming the file to edit instead.
## The round trip
Losing the file as an editing surface would be a real loss — text is
diffable, reviewable and easy to back up — so the file is kept as a
*rendering* of the database rather than a rival to it. That is what
`export`/`import` are for:
The file stays useful as an editing surface in both modes — text is diffable,
reviewable and easy to back up — because `export` renders the database into the
same shape `import` and file mode read:
```
nxdns export → canonical ZON → edit → nxdns import --force → config.db
nxdns export → canonical ZON → edit → nxdns import → config.db
```
`nxdns export` renders the database as canonical ZON: a fixed two-line
header, every default emitted, deterministic ordering from the model's field
order and the repositories' `ORDER BY` clauses, and no timestamps or hostnames
anywhere. Runtime columns — first-seen, last-seen, per-source counters — are
absent from the configuration model on purpose, so two exports taken from a
live, busy server are identical. The round trip `export → import → export` is
byte-identical, and a test asserts it.
`nxdns export` writes canonical ZON: a fixed two-line header, every default
emitted, deterministic ordering from the model's field order and the
repositories' `ORDER BY` clauses, and no timestamps or hostnames anywhere.
Runtime columns are absent from the configuration model on purpose, so two
exports taken from a live, busy server are identical. `export → import →
export` is byte-identical, and a test asserts it.
Byte-stability is not cosmetic. It is what makes an exported file usable in
version control and what makes a diff of two exports mean something: any
difference is a configuration change, never noise from when the export ran.
`nxdns import` replaces the whole database content in one transaction. Not a
merge, not a patch: delete every content table in foreign-key-safe order, then
insert what the file says — with the one exception described above. Every client
row is lifted out first. The materialised ones are put back rather than
recreated from a file that never held them, and the observed timestamps of an
address the file *does* name are merged onto its new row. A client the file
leaves out is gone, history included: nothing puts a `hand_edited = 1` row back.
A failed import — bad syntax, failed
validation, a constraint violation halfway through — leaves the database exactly
as it was, because everything happens inside a single `BEGIN IMMEDIATE`.
The output is also identical in both modes — nothing marks a file as coming
from a file-mode box. That is deliberate, because it is what makes export the
adoption tool: the file you check is the file you deploy, byte for byte. The
label an operator wants is in the unit file, where they put it.
Without `--force`, import refuses a database that already holds configuration.
That
check runs *inside* the transaction, after the lock is taken, so it cannot be
raced by a concurrent write. The effect is that a plain `import` can never
clobber a configured server by accident, and clobbering it deliberately takes
one visible extra word on the command line. See
[how-to/back-up-and-restore.md](../how-to/back-up-and-restore.md).
## One model, three surfaces
`src/config/model.zig` defines exactly one `Config` type. The ZON parser
produces it, the database reader produces it, the validator consumes it, the
composition root consumes it, and the settings API derives its key list and
its patch struct from its type information rather than mirroring the fields by
hand. Adding a field in one place therefore cannot leave the other surfaces
behind; the alternative — a file schema, a DB schema and an API schema kept in
step by discipline — is the standard way configuration systems rot.
The model has two shapes of field, and the difference is structural rather
than stylistic. Struct-typed fields are scalar sections (`dns`, `cache`,
`web`, …) and live in a single key/value `settings` table as
`section.field` text rows. Slice-typed fields are collections (groups,
upstreams, clients, rules, local records, forward zones, …) and each gets its
own table with foreign keys. `toSettings` and `fromSettings` are the two
halves of the scalar bridge, generated by an `inline for` over the model, so
the key list is a consequence of the type rather than a second list to
maintain.
Reconciling the same file twice produces a byte-identical database — ids,
checksums, `created_at`, the password hash, the whole settings table — including
when the file is written in a non-canonical but equivalent form, such as
`FD00:0:0:0:0:0:0:1` for an address stored as `fd00::1`. Anything that churns
under an unchanged file is a bug in the engine by definition. That single
invariant is what forces most of the design above: matching on canonical forms,
writing only on difference, treating duplicate rule tuples as a multiset, and
verifying a password rather than re-hashing it.
## Unknown keys, and why the asymmetry is deliberate
@@ -157,52 +181,65 @@ An unknown key in the **database** is warned about and ignored. An unknown key
in the **file** is a hard error.
They are different situations. A settings row the running binary does not
recognise is almost always an older binary reading a database written by a
newer one — a downgrade, or a rollback after a bad upgrade. Refusing to start
there would mean a downgrade bricks the config database, and the operator
would have to hand-edit SQLite to recover. Warning and ignoring means the
downgrade works, the unknown setting sits inert, and the upgrade back picks it
up again.
recognise is almost always an older binary reading a database written by a newer
one — a downgrade, or a rollback after a bad upgrade. Refusing to start there
would mean a downgrade bricks the config database, and the operator would have
to hand-edit SQLite to recover. Warning and ignoring means the downgrade works,
the unknown setting sits inert, and the upgrade back picks it up again.
A key in a file, by contrast, is something a human just typed. The likeliest
cause is a typo, and the second likeliest is a field that no longer exists.
Silently ignoring it would mean the setting the operator believes they applied
was never applied — exactly the silent-divergence failure the whole model is
built to avoid. So the ZON parser rejects unknown fields with a line and
column.
built to avoid. So the ZON parser rejects unknown fields with a line and column.
The tolerance has a boundary worth stating plainly: it covers unknown *keys*,
not unparseable *values*. A known key whose stored text does not decode into
its type is an error, not a warning.
not unparseable *values*. A known key whose stored text does not decode into its
type is an error, not a warning.
## The password
`web.password` is a write-only input. It is never a stored value.
`web.password` is a write-only input. It is never a stored value. A non-empty
one is hashed with argon2id (PHC encoding, OWASP argon2id parameters) into
`web.password_hash`, and the plaintext is cleared before anything is written.
There is no settings row that can hold it: the model skips `web.password` in
both directions of the settings bridge, so the plaintext has nowhere to go even
by accident.
At import time, a non-empty `web.password` is hashed with argon2id (PHC
encoding, OWASP argon2id parameters) into `web.password_hash`, and the
plaintext field is cleared before anything is written. There is no settings
row that can hold it: the model explicitly skips `web.password` in both
directions of the settings bridge, so the plaintext has nowhere to go even by
accident. `nxdns export` always writes `.password = ""` and carries the hash
instead — which is also what makes the round trip stable, since an export that
tried to reproduce a plaintext it never had could not be byte-identical.
Both fields are optional, and this is the one deliberate carve-out from
file-as-sole-truth: **a file that mentions neither leaves the stored hash
alone.**
Setting both `password` and `password_hash` in one file is an error rather
than a precedence rule. The two say different things about what the password
is, and picking a winner would mean the operator's other statement was
silently discarded. The file has to say one thing.
The reason is a trap the design walked into once. An export carries the full PHC
string, which is long and ugly, and an operator committing that file to git will
sooner or later delete the line — meaning "keep the current password". If
absence meant "no password", that edit would reconcile an empty hash over the
stored one and open the admin UI to the entire LAN, silently, because
authentication is on exactly when the hash is non-empty. Silence has to mean
keep. Disabling authentication takes the explicit `password_hash = ""`.
The practical shape of a password change is therefore export, set `.password`
to the new value, clear `.password_hash`, import with `--force`. The procedure
is in
[how-to/set-up-admin-authentication.md](../how-to/set-up-admin-authentication.md).
The other end of the same problem is `password = ""`. Hashing the empty string
produces a perfectly valid hash, so authentication would be *on* — while the
login handler refuses every empty password, so it could never be satisfied.
Auth on and unreachable is worse than either alternative, so that file is
refused at validation with a diagnostic naming the remedy.
A plaintext password that has not changed is verified against the stored hash
and kept rather than re-hashed. That is byte-stability, not a saving:
verification recomputes the same argon2id function with the stored salt and
costs exactly what hashing costs. Hashing unconditionally would generate a fresh
salt on every start and break the invariant above.
Export's canonical form is therefore `password = null` beside the stored
`password_hash`. Writing an empty *string* there instead would make every export
carry a present-but-empty password next to a hash — tripping the both-set rule
on re-import, so export's own output would fail export's own contract.
## What is not configuration
Storage paths are process arguments, not configuration fields: `--data-dir`,
`--config`, `--web-dev`. They cannot live in the file, because the file is
found by way of them — a path that told you where to find the thing that told
you the path would be circular. They are also the settings a supervisor
(systemd, Docker) owns rather than the operator's policy about DNS. See
`--config`, `--web-dev`. They cannot live in the file, because the file is found
by way of them — a path that told you where to find the thing that told you the
path would be circular. They are also the settings a supervisor (systemd,
Docker) owns rather than the operator's policy about DNS. See
[reference/files-and-directories.md](../reference/files-and-directories.md).