milestone 30: overview as a dashboard, explicit health contract, period aggregations

2026-08-22 16:45:15 +02:00
parent 17422fac21
commit 648d9b4496
89 changed files with 7222 additions and 4239 deletions
@@ -271,15 +271,17 @@ warning(disk_monitor): disk state ok -> critical: 33349095424 bytes free on /var
 curl -s http://127.0.0.1:8080/api/health
 ```

-`/api/health` needs no login and reports the state and what has been gated:
+`/api/health` needs no login. It answers with five condition objects, and `status` is `degraded` when any one of them is in a degrading state:

 ```json
-{"status":"degraded","disk":{"state":"critical","free_bytes":33349079040,"db_bytes":180224,"log_bytes":0,"sample_failures":0},"upstreams":{"available":1,"total":1},"queries_dropped":0,"writer_failed":false,"refreshes_gated":1,"snapshot_generation":2}
+{"status":"degraded","protection":{"state":"active","until":null},"upstreams":{"state":"ok","available":1,"total":1},"query_history":{"state":"recording","dropped_total":0,"last_drop_s":null},"diagnostics":{"state":"recording","active_warnings":1,"active_errors":0},"disk":{"state":"critical","free_bytes":33349079040}}
 ```

-`/metrics` carries the same free, database and log byte gauges as `nxdns_disk_free_bytes`, `nxdns_disk_db_bytes` and `nxdns_disk_log_bytes`; the state itself is on `/api/health`, not in the metrics output.
+The disk monitor's `warn` state is reported here as `low`, because `warn` reads as a log level rather than as a quantity of disk.

-**What the state means.** The monitor samples free space and database sizes once a minute. Below `disk.warn_free_mb` it logs the transition. Below `disk.min_free_mb` it gates every non-essential write: the query logger holds its batches, the client tracker stops persisting, and blocklist refreshes are skipped and counted in `refreshes_gated`. Resolution never degrades because the disk is full — this was verified by setting the thresholds above the free space on the volume: the state went critical, a refresh was gated, and queries kept being answered.
+`/metrics` carries the free, database and log byte gauges as `nxdns_disk_free_bytes`, `nxdns_disk_db_bytes` and `nxdns_disk_log_bytes`, and counts gated refreshes as `nxdns_blocklist_refreshes_gated_total`. The disk state itself is on `/api/health`, not in the metrics output.
+
+**What the state means.** The monitor samples free space and database sizes once a minute. Below `disk.warn_free_mb` it logs the transition. Below `disk.min_free_mb` it gates every non-essential write: the query logger holds its batches, the client tracker stops persisting, and blocklist refreshes are skipped and counted in `nxdns_blocklist_refreshes_gated_total`. A gate that has already cost query rows shows as `query_history.state = "losing"`. Resolution never degrades because the disk is full — this was verified by setting the thresholds above the free space on the volume: the state went critical, a refresh was gated, and queries kept being answered.

 **Fix.** Recover space — lower `logging.retention_days`, or stop the service and delete `querylog.db` — and writes resume on the next sample.