rewrite as a daemon with shutdown on low battery
the oneshot+timer produced 8367 journal lines a day of systemd start/stop noise. now Type=notify with an internal ticker, logging transitions only. hardware sits behind sensor.Sensor; internal/monitor is pure, so the shutdown state machine is tested without a pi. four guards, each tested, because a false poweroff of a box with no physical access is worse than a missed one: - a failed read resets the arming window rather than pausing it - voltage must actually fall across the window, else it is a stuck sensor. this is the only defence against the ac line reading 0 while mains is connected, so it is load-bearing - a settle period stops a restart loop acting on early readings - flapping ac cannot accumulate verified on hardware 2026-08-09: after poweroff on battery the x1208 starts the pi again when mains returns, with no button press. that was the risk that could have made this feature unsafe. a failed tick rewrites the previous sample with sensor_healthy 0 and the failure counter, so last_update still freezes for the staleness alert while the failure stays visible. module path moved to git.mial.net. debian units and the compose overlay deleted; deployment lives in the infra repo.
This commit is contained in:
@@ -0,0 +1,91 @@
|
||||
# x1208-exporter — daemon rewrite plan
|
||||
|
||||
Oneshot+timer → long-running daemon. Why: the 30 s oneshot generates 8,367 journal lines/day (systemd Starting/Finished noise), ~33 KB SD writeback per sparse journal entry on the old box. Target: `Type=notify`, `Restart=always`, internal ticker, near-zero steady-state logging. Greenfield rules apply: breaking changes allowed, no debt carried.
|
||||
|
||||
Consumer: `rpi.mial.net` (NixOS, `buildGoModule`, aarch64). Metric names are LOAD-BEARING (Grafana alert `RpiUpsExporterStale` + dashboard): `rpi_ups_last_update_seconds`, `rpi_ups_ac_power`, `rpi_ups_voltage_volts`, `rpi_ups_battery_percent` — never rename.
|
||||
|
||||
## Decisions
|
||||
|
||||
| # | Question | Call | Why |
|
||||
|---|---|---|---|
|
||||
| 1 | Textfile vs HTTP `/metrics` | **Keep the textfile.** | Zero blast radius: alert, dashboard, Alloy config, node-exporter scrape all unchanged (`no_data_state=Alerting` + `time()-last_update` verified correct as-is). HTTP would add `prometheus/client_golang` (heavy dep for 4 gauges + go runtime noise), a listener + port, a new Alloy scrape job in the other repo, and change alert semantics (frozen-file staleness → absent-series) for no functional gain. The atomic write already exists and is correct. Daemon rewrites the file per tick; textfile-as-IPC ambiguity is resolved by decision 4 (no second reader anymore). |
|
||||
| 2 | Resource lifecycle | **Held open, reopen-on-error.** i2c fd + GPIO line acquired at startup, held across ticks. Any read error after the existing retry (3×20 ms): close BOTH handles, next tick reacquires from scratch. | Held-open = cheap ticks, no per-tick open/close syscall churn. Reopen-on-error gives per-read robustness where it matters: a wedged i2c bus or vanished gpiocdev line heals on the next tick instead of poisoning the fd forever. Failure to reacquire = failed tick, covered by decision 3; daemon never exits over hardware errors (`Restart=always` is the backstop for panics, not for a flaky bus). |
|
||||
| 3 | Failure + logging policy | **Log transitions only. Failed tick = no write.** | Logs exactly: one startup line, AC state change (info), enter error state (warn, first failed tick only), exit error state (info, with failure count). NOTHING on a successful tick. Metric policy: the file is written only from a COMPLETE successful sample (all three readings); partial or failed tick leaves the previous file untouched, so `last_update` freezes and the existing staleness alert fires after its designed 3 m + 3 m — no new alert needed, no partial-file states to reason about. Additive: `rpi_ups_read_failures_total` counter (resets on restart; `rate()` handles that). |
|
||||
| 4 | Shutdown responsibility | **In this daemon, flag-gated, default OFF.** Supersedes the separate `ups-shutdown.nix` file-reader in rpi.mial.net (that module shrinks to flags + polkit rule; flagged to the other repo, not edited here). | The daemon owns the readings; a second process re-parsing a textfile the first just wrote re-implements sampling, staleness guarding and hysteresis against a lossy serialization of state it could have had directly — that is the brittle architecture. Failure coupling is identical either way (dead exporter = stale file = separate reader must no-op too). Privilege solved without root: daemon execs `systemctl poweroff` (no D-Bus dep), authorized by a polkit rule matching a static group the unit joins. Trigger: `ac==0 AND (voltage < 3.5 OR soc < 15)` sustained N consecutive ticks (default N=4 → 2 min), thresholds ABOVE the 3.3 V `RpiUpsBatteryCritical` alert so the page fires before the poweroff. Staleness guard inherent: only fresh successful samples feed the state machine. One warn log when armed, one when disarmed, one at poweroff. |
|
||||
| 5 | Testability | Interface seam + pure cores, below. | Threshold/transition/render logic unit-testable on any machine; hardware isolated to two leaf packages. |
|
||||
| 6 | Packaging | **Flake in this repo** exporting the package; rpi.mial.net consumes it as a flake input (nixpkgs `follows`). Tagged releases `vX.Y.Z`; `ldflags "-X main.version="`. | vendorHash + version live NEXT to the code — one commit bumps both, Renovate bumps the consumer's flake.lock (nix manager already enabled there). `fetchFromGitea` + rev/hash pairs in the consumer would smear this repo's build metadata across two repos. Repo goes public (decided), so the build-time-fetch-needs-no-auth constraint is satisfied either way; flake is the cleaner shape. NO nixosModule in this repo: the systemd unit is infra policy and lives in `rpi.mial.net/nixos/modules/x1208-exporter.nix` — two sources of unit truth is exactly the split-brain to avoid. |
|
||||
| 7 | Repo hygiene | Below. | — |
|
||||
| 8 | Unit shape | Below (reference for the infra module). | — |
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
main.go flags, wiring, signal handling — no logic
|
||||
internal/max17040/ fuel gauge: open/ioctl/read-word/retry/scale (from current readBattery)
|
||||
internal/pld/ GPIO 6 AC-present line via gpiocdev (from current readACPower)
|
||||
internal/sensor/ type Reading { AC bool; Volts, SOC float64; At time.Time }
|
||||
type Sensor interface { Read() (Reading, error); Close() error }
|
||||
real: composes max17040+pld, owns held-open handles + reopen-on-error
|
||||
fake: scripted readings/errors for tests
|
||||
internal/monitor/ the daemon core, PURE: ticker loop driven by injected clock,
|
||||
transition detection (AC change, error enter/exit),
|
||||
shutdown state machine (N-sample hysteresis) → Action enum,
|
||||
emits Reading → sink; no I/O, no syscalls
|
||||
internal/textfile/ Render(Reading, now) string (golden-tested, byte-identical
|
||||
metric names/HELP to today) + WriteAtomic (moved, unchanged)
|
||||
internal/sdnotify/ READY=1 / WATCHDOG=1 datagram to $NOTIFY_SOCKET (~30 lines, no dep)
|
||||
```
|
||||
|
||||
- `monitor` takes `Sensor`, clock, and callbacks (write file, log, act); unit tests cover: transition-only logging, failed-tick-no-write, N-sample shutdown hysteresis incl. flapping AC, staleness never triggers shutdown, counter increments.
|
||||
- Keep: gpiochip0 by name (no Debian symlink dependency), validation ranges (2.0–5.0 V, ≤110 %), 3×20 ms retry, big-endian word read, atomic rename.
|
||||
- Interval: `-interval 30s` flag, same cadence as today. `-out` flag kept. Shutdown: `-shutdown`, `-shutdown-voltage 3.5`, `-shutdown-soc 15`, `-shutdown-samples 4`, `-shutdown-cmd "systemctl poweroff"` (injectable for tests).
|
||||
|
||||
## Systemd unit (reference — authoritative copy goes in rpi.mial.net)
|
||||
|
||||
```
|
||||
Type=notify WatchdogSec=90s # 3 missed ticks → systemd restarts a hung daemon
|
||||
Restart=always RestartSec=5s
|
||||
DynamicUser=yes
|
||||
SupplementaryGroups=i2c ups # i2c: /dev/i2c-1; ups: static group — owns the
|
||||
# textfile dir (g+w) AND matches the polkit rule
|
||||
DeviceAllow=/dev/i2c-1 rw
|
||||
DeviceAllow=/dev/gpiochip0 rw
|
||||
ReadWritePaths=/var/lib/node-exporter/textfiles
|
||||
Nice=10 MemoryMax=32M
|
||||
NoNewPrivileges / ProtectSystem=strict / ProtectHome / PrivateTmp /
|
||||
ProtectKernel{Tunables,Modules,Logs} / ProtectControlGroups / ProtectClock /
|
||||
RestrictAddressFamilies=AF_UNIX / RestrictNamespaces / RestrictRealtime /
|
||||
LockPersonality / MemoryDenyWriteExecute / SystemCallFilter=@system-service /
|
||||
SystemCallArchitectures=native / UMask=0022
|
||||
```
|
||||
|
||||
Timer: deleted — no timer exists anymore.
|
||||
Root dropped entirely: device access via groups, poweroff via polkit (`org.freedesktop.login1.power-off` allowed for `unix-group:ups`). `MemoryDenyWriteExecute` is safe for pure Go. `/dev/gpiochip0` group access needs a udev rule on NixOS (infra repo's job — flagged, not assumed).
|
||||
|
||||
## Repo hygiene
|
||||
|
||||
| Item | Action |
|
||||
|---|---|
|
||||
| `build/x1208-exporter` committed binary | Remove. Repo has ONE commit and is about to go public: rewrite history (fresh init or amend) so the binary is not in it forever. Cheap now, impossible later. |
|
||||
| `.gitignore` | `/build/`, `/result`, `/result-*`, `*.prom` |
|
||||
| Module path `github.com/mokhtar/x1208-exporter` | Wrong host — repo lives at git.mial.net. Rename to `git.mial.net/mokhtar/x1208-exporter`. Breaking, allowed, correct. |
|
||||
| `go.mod` Go 1.23 | Bump to current stable at rewrite time. Deps: keep `go-gpiocdev`; DROP `golang.org/x/sys` from direct use only if max17040 no longer needs it (it does — ioctl; keep). |
|
||||
| `systemd/` (Debian units) + justfile scp-deploy recipes | Delete. NixOS module in rpi.mial.net is the only deployment; dead Debian paths are debt. justfile keeps: `build`, `test`, `lint`, `run-fake` (daemon against fake sensor, prints file to stdout). |
|
||||
| README | Rewrite for daemon + flake usage; drop compose/Debian instructions. |
|
||||
|
||||
## Order of work
|
||||
|
||||
1. History rewrite + hygiene (binary out, gitignore, module path). Push before publicizing.
|
||||
2. Package split with tests, behavior-preserving (still produces identical .prom bytes — golden test locks this).
|
||||
3. Daemon loop: ticker, held-open sensor, transition logging, sdnotify + watchdog.
|
||||
4. Shutdown state machine, flag-gated off.
|
||||
5. Flake (package + devShell), tag `v1.0.0`.
|
||||
6. Consumption + unit + polkit + udev + `ups` group land in rpi.mial.net (other repo — out of scope here; includes retiring `ups-shutdown.nix` as a separate reader).
|
||||
|
||||
## Unresolved questions
|
||||
|
||||
1. Shutdown thresholds sign-off: `3.5 V / 15 % / N=4` (2 min sustained) — above the 3.3 V page, below nuisance range. Confirm numbers.
|
||||
2. Static group name for textfile-dir write + polkit match (`ups`?) — infra repo decides, this plan assumes one group serves both.
|
||||
3. Polkit rule exact shape on NixOS (`security.polkit.extraConfig` matching `org.freedesktop.login1.power-off` for the group) — verify action id covers `systemctl poweroff` non-interactively before relying on it; fallback is a root-owned wrapper with a narrow sudoers line (worse; avoid if polkit works).
|
||||
4. Does anything else read the textfile once ups-shutdown-as-reader dies? (Believed: only node-exporter. Confirm before changing write policy to no-partial-updates.)
|
||||
5. `rpi_ups_read_failures_total`: worth a Grafana panel, or leave unalerted as diagnostic? (No alert planned — staleness alert remains the pager.)
|
||||
Reference in New Issue
Block a user