mokhtar/nxdns
1
0
Fork 0
Code Pull Requests Actions Packages Releases 13 Activity

querylog: checkpoint every 8192 wal pages instead of 1000

Browse Source
This commit is contained in:
mokhtar
2026-08-21 18:43:45 +02:00
parent 324704b53f
commit 9b0b7c19f4
7 changed files with 140 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+8
View File
@@ -4,6 +4,14 @@ All notable changes to nxdns are recorded here. The format follows [Keep a Chang
Sections are written by hand. Nothing here is generated from commit messages: the point of the file is to say what changed for an operator, which a commit subject rarely does.
## [0.0.8] - 2026-08-21
One constant, chosen from the 0.0.7 field numbers: the checkpoint cadence was the last first-order write cost on the Pi's SD card.
### Changed
- **The query log checkpoints its write-ahead log every 32 MiB instead of every 4 MiB.** Batching the writer in 0.0.7 took the deployed Pi from about 0.5 to 0.281 GiB of writes a day, and about 130 MiB of what is left is checkpoint writeback: SQLite's 1000-page default trips roughly every 40 minutes and rewrites the same hot index and interior pages into `querylog.db` each time. Every read-write connection to `querylog.db` now sets `wal_autocheckpoint` to 8192 pages, which stretches that to roughly five hours and cuts those in-place rewrites about eightfold, for an expected total near 190 MiB a day. The price is durability under power loss or a kernel panic. At `synchronous = NORMAL` a commit does not fsync, so the checkpoint is the only guaranteed durability boundary, and it now sits about five hours of query rows and upstream-history minutes back rather than 40 minutes. Kernel writeback normally makes the real loss far smaller than that, but nothing guarantees it. A process crash or a clean stop still loses nothing that was committed, and the database is never left inconsistent: recovery replays the longest valid prefix of the log. The `querylog.db-wal` file is expected to sit near 32 MiB rather than capped there, since a long-running reader can hold a checkpoint off and let it overshoot, and the daily retention pass still truncates it. `config.db` is unchanged.
## [0.0.7] - 2026-08-20
Operational failures get a page of their own, and the query log stops wearing out the disk it lives on: the deployed Pi was writing half a gigabyte a day to store two megabytes of query rows, one transaction per query. Both came out of running 0.0.6 on real hardware.