Nothing checks that a depended-on unit is still running: a lock stayed dead for 8 days while every check said green #50

Open
opened 2026-10-20 17:55:30 -07:00 by dank · 0 comments
Owner

Nothing on any box checks that a depended-on unit is still running

Found while fixing #47. swayidle-lock.service sat enabled, listed in
default.target.wants/, and dead for eight days while every check answered green:
is-enabled → enabled, deploy.sh --check → actions=0. The deployment tool only ever
enables; no code path anywhere asks "is the thing this machine depends on actually
running?" A security posture lapsed for a week and the tooling reported success the entire
time.

The same class produced the second miss in that incident: auditing "what locks this screen"
with pgrep returned nothing, and that read as "no lock path" — while systemd-inhibit --list showed noctalia … sleep "Lock before sleep", a locker that is a hook inside a
running shell and never appears on the process list.

Shape of the fix

scripts/live-sync.sh already runs hourly, already commits machine state, and already alerts
through ntfy on failure — the plumbing for a posture check exists. What it does not do is
look at service state. A check that would have caught October 12:

  • for each unit in dotfiles/systemd-user/ (and the zz-$HOST unit set) whose ExecStart
    resolves here and which systemd reports as enabled: assert is-active is active, and
    alert once per state change when it is not;
  • report the inhibitor table alongside, because an idle inhibitor legitimately disarms
    swayidle's timeout (Not enabling timeouts: idle inhibitor found) and that is a different
    finding from a dead unit — without it, the alert would cry wolf during every agent session;
  • stay alert-only: the heartbeat must not start services, which is the rule #45 established
    (deploy.sh starts nothing a live session may already run).

Law and measurements: tools/dotfiles-deploy — Law: never
start a unit a session may already be running
, Law: a watcher must restart on a clean exit
too
, and the note on reading systemd-inhibit --list before pgrep.


cwd: /home/blank/.omp · session: 01a155bb-6474-7366-be9d-629f74af0e05

# Nothing on any box checks that a depended-on unit is still running Found while fixing #47. `swayidle-lock.service` sat `enabled`, listed in `default.target.wants/`, and **dead for eight days** while every check answered green: `is-enabled` → `enabled`, `deploy.sh --check` → `actions=0`. The deployment tool only ever *enables*; no code path anywhere asks "is the thing this machine depends on actually running?" A security posture lapsed for a week and the tooling reported success the entire time. The same class produced the second miss in that incident: auditing "what locks this screen" with `pgrep` returned nothing, and that read as "no lock path" — while `systemd-inhibit --list` showed `noctalia … sleep "Lock before sleep"`, a locker that is a hook inside a running shell and never appears on the process list. ## Shape of the fix `scripts/live-sync.sh` already runs hourly, already commits machine state, and already alerts through ntfy on failure — the plumbing for a posture check exists. What it does not do is look at service state. A check that would have caught October 12: - for each unit in `dotfiles/systemd-user/` (and the `zz-$HOST` unit set) whose `ExecStart` resolves here and which systemd reports as enabled: assert `is-active` is `active`, and alert once per state change when it is not; - report the inhibitor table alongside, because an `idle` inhibitor legitimately disarms swayidle's timeout (`Not enabling timeouts: idle inhibitor found`) and that is a different finding from a dead unit — without it, the alert would cry wolf during every agent session; - stay alert-only: the heartbeat must not start services, which is the rule #45 established (`deploy.sh` starts nothing a live session may already run). Law and measurements: [tools/dotfiles-deploy](omp-docs/tools/dotfiles-deploy.md) — *Law: never start a unit a session may already be running*, *Law: a watcher must restart on a clean exit too*, and the note on reading `systemd-inhibit --list` before `pgrep`. --- cwd: /home/blank/.omp · session: 01a155bb-6474-7366-be9d-629f74af0e05
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dank/command-center#50