fix(fleet): refuse a container start until the mount it binds has answered
Dankserver rebooted with the array's mount and the media containers racing. A rootless quadlet cannot order itself against the system's mnt-entropy.mount, so `RequiresMountsFor=/mnt/entropy` is inert in the user manager: podman resolved the bind against the pre-mount directory, Jellyfin and Plex came up blind, and Jellyfin read an empty library and began `Removing item, Type: "Season"` by the thousand. Copyparty had no guard at all. `scripts/wait-for-mount.sh` is the gate: it waits for every named mountpoint, then refuses by name. `ConditionPathIsMountPoint=` is NOT the fix — measured on the host, an unmet condition logs `skipped, unmet condition check` with NRestarts=0 and no start attempt ever follows, which is fail-dead, not fail-closed. Jellyfin and Plex swap the condition for the gate; Copyparty gets one; the law is recorded in AGENTS.md and the view page. Verified on the host: the 5.7.0 quadlet generator accepts ExecStartPre in [Service] (exit 0, all three units generated, gate present), the gate waits for a mount that appears mid-window and proceeds, and it refuses a plain directory after its window naming the path. tests/media_overlay.bats pinned the old mechanism by name; its intent — both media servers refuse to start without the view — now rides the gate.
This commit is contained in:
1 parent
75b04dc313
commit
e82c8b2d06
8 files changed
+217
-16
No files matched your search
Executable
+74
@@ -0,0 +1,74 @@
|
||||
#!/usr/bin/env bash
|
||||
# wait-for-mount.sh — refuse a container start until what it binds is mounted.
|
||||
#
|
||||
# Why this exists (live 2026-10-31, Dankserver): a rootless quadlet cannot order
|
||||
# itself against a SYSTEM mount unit. `RequiresMountsFor=/mnt/entropy` in a user
|
||||
# unit resolves to nothing and is inert, so when the array's mount finished a
|
||||
# second after podman resolved the bind, Jellyfin and Plex started against the
|
||||
# pre-mount directory. Their library root looked EMPTY and Jellyfin began
|
||||
# deleting metadata by the thousand (`Removing item, Type: "Season"`). A blind
|
||||
# bind is not a cosmetic fault; it is a metadata-deleting machine.
|
||||
#
|
||||
# A `ConditionPathIsMountPoint=` is NOT the fix, measured on the same host: an
|
||||
# unmet condition logs "skipped, unmet condition check", NRestarts=0, and no
|
||||
# start attempt ever follows — the unit stays dead until a human starts it.
|
||||
# That is fail-dead. This script WAITS (the mount usually lands within seconds)
|
||||
# and only then refuses, so `Restart=always` carries the unit across the race.
|
||||
#
|
||||
# usage: wait-for-mount.sh [-t SECONDS] MOUNTPOINT [MOUNTPOINT...]
|
||||
# exit 0 every path is a mountpoint
|
||||
# exit 1 a path is still not a mountpoint after the window (named on stderr)
|
||||
# exit 2 usage error
|
||||
set -euo pipefail
|
||||
|
||||
usage() { printf 'usage: %s [-t SECONDS] MOUNTPOINT [MOUNTPOINT...]\n' "$0"; }
|
||||
|
||||
window=90
|
||||
while getopts ":t:h" opt; do
|
||||
case "$opt" in
|
||||
t) window=$OPTARG ;;
|
||||
h) usage; exit 0 ;;
|
||||
*) usage >&2; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
shift $((OPTIND - 1))
|
||||
|
||||
[ "$#" -ge 1 ] || { usage >&2; exit 2; }
|
||||
case "$window" in
|
||||
'' | *[!0-9]*) printf 'refused: -t wants whole seconds, got %s\n' "$window" >&2; exit 2 ;;
|
||||
esac
|
||||
|
||||
is_mount() { findmnt -M "$1" -rno TARGET >/dev/null 2>&1; }
|
||||
|
||||
report() {
|
||||
local why=$1; shift
|
||||
printf '%s:' "$why"
|
||||
for p in "$@"; do printf ' %s' "$p"; done
|
||||
printf '\n'
|
||||
}
|
||||
|
||||
deadline=$(( $(date +%s) + window ))
|
||||
warned=0
|
||||
while :; do
|
||||
missing=()
|
||||
for path in "$@"; do
|
||||
is_mount "$path" || missing+=("$path")
|
||||
done
|
||||
[ "${#missing[@]}" -eq 0 ] && break
|
||||
|
||||
if [ "$(date +%s)" -ge "$deadline" ]; then
|
||||
report "refused: still not mounted after ${window}s" "${missing[@]}" >&2
|
||||
printf 'refusing to start: a container bind-mounted here would see the empty\n' >&2
|
||||
printf 'directory underneath and conclude the library is empty — which is how a\n' >&2
|
||||
printf 'scan begins deleting metadata it cannot find.\n' >&2
|
||||
exit 1
|
||||
fi
|
||||
if [ "$warned" -eq 0 ]; then
|
||||
report "waiting for mount(s) this unit depends on" "${missing[@]}"
|
||||
warned=1
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
|
||||
[ "$warned" -eq 0 ] || printf 'mounted; proceeding.\n'
|
||||
exit 0
|
||||
Reference in new issue
Block a user