dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(scripts): the two media scan doors become commands

Browse Source 
Both media servers' scan procedure lived only as a hand-written curl block in
the vault, and that block has bitten this fleet twice: the escaped-quote form
that reads exactly like a dead API key, and a config `url` line meeting a
command-line URL so curl quietly performs a second transfer and dumps the whole
task table on your stdout. Jellyfin's door is now
`scripts/media-refresh-jellyfin.py` (trigger / status / wait) and Plex's is
`scripts/media-refresh-plex.sh` (trigger / status);
`tests/media_refresh.bats` pins the contracts an operator observes, against a
stub and with no live server.

The Jellyfin `wait` encodes the settle law instead of a poll count: the Scan
Media Library task must be seen Running — or finish while watching, or already
pass the `--after` it is given — before any quiet read counts toward settled.
A watcher that saw no Running otherwise reports the pre-scan library as the
fresh one, which is how a rename looks applied while nothing has rescanned.
Plex's `status` needs no token at all: `section_locations.scanned_at` carries
the same epoch the API prints as `scannedAt` (measured on section 1 today).

Neither driver lets a secret reach argv or a URL: the Jellyfin key rides a
mode-600 curl config as the quoted MediaBrowser form, and the Plex token is
read inside the container, where the HTTPD actually answers, into a config file
that is shredded there.
This commit is contained in:
Frank committed 2026-11-04 11:29:03 -08:00
1 parent e1bf1d6e80
commit e7d7c6eea2
3 files changed
+710
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

scripts/media-refresh-jellyfin.py
Executable
+222
View File
@@ -0,0 +1,222 @@
#!/usr/bin/env python3
"""Trigger and poll Jellyfin's library scan — the step that makes a media move visible.
media-refresh-jellyfin.py trigger [--force]
media-refresh-jellyfin.py status [--after EPOCH]
media-refresh-jellyfin.py wait [--after EPOCH] [--interval S] [--require-quiet N]
`status` answers {"scan": true, "taskState": "Running", "recentMedia": 17}: the
Scan Media Library row plus the rows the database has taken in recently; with
`--after` (a journal mtime) it also answers `scannedAfter` and exits non-zero
while the last finished scan predates it. `wait` calls the library settled only
on the page's law: a sighting of the task Running — or a run that finished while
it watched, or a `--after` the task already passes — and THEN `--require-quiet`
consecutive reads with no scan and no writes. Both the scan and the metadata
fetch must be quiet, and the Running sighting is what stops a watcher from
declaring a pre-scan library settled (Dankserver/Jellyfin.md).
Laws this encodes:
* Auth is `Authorization: MediaBrowser Token="…"`. A bare X-Emby-Token answers
401 (measured on franksdankcloud 12.4.0), which reads exactly like a dead
key. The key comes from a mode-600 file and never rides argv or a URL:
/proc/<pid>/cmdline is world-readable.
* `POST /Library/Refresh` (204) is the ONLY door that queues work on 12.4, and
it walks every library. `POST /Items/{id}/Refresh` answers 204 and queues
NOTHING (measured twice, 2026-11-02) — so there is no --library here.
* A SECOND accepted POST cancels and restarts the scan already running, so
progress looks frozen while you hammer it. `trigger` therefore refuses while
the Scan Media Library task is not Idle; --force says you want the restart.
* 429 means busy. Waiting is the only response that helps, and it is bounded
so a wedged server ends the run with a named failure instead of a loop.
* `RefreshStatus` on /Library/VirtualFolders stays Idle through a live scan;
never poll it. The task table is the only honest read.
Defaults are this fleet's layout: http://127.0.0.1:8096, the key at
~/.config/secrets/jellyfin, and the database found by trying, in order, $JELLYFIN_DB,
~/.local/share/jellyfin/jellyfin.db and the quadlet's config bind.
"""
import argparse
import json
import os
import pathlib
import sqlite3
import sys
import time
import urllib.error
import urllib.request
from datetime import datetime, timezone
SCAN_TASK = "Scan Media Library"
IDLE = ("Idle",) # Running, Cancelling and Waiting all mean: not settled
DB_CANDIDATES = ("~/.local/share/jellyfin/jellyfin.db",
"/home/dank/Containers/jellyfin/config/data/jellyfin.db")
def _epoch(iso: str | None) -> float | None:
"""Jellyfin's `2026-11-04T18:31:33Z` EndTimeUtc as a unix epoch, or None."""
if not iso:
return None
for fmt in ("%Y-%m-%dT%H:%M:%S.%fZ", "%Y-%m-%dT%H:%M:%SZ"):
try:
return datetime.strptime(iso, fmt).replace(tzinfo=timezone.utc).timestamp()
except ValueError:
continue
sys.exit(f"the server sent an unparsable EndTimeUtc: {iso!r}")
def read_key(value: str) -> str:
path = pathlib.Path(value).expanduser()
try:
token = path.read_text().strip()
except OSError as exc:
sys.exit(f"no Jellyfin API key at {path} ({exc.strerror}); "
"that file is the credential this driver authenticates with")
if not token:
sys.exit(f"Jellyfin API key file {path} is empty")
return token
def request(base: str, token: str, method: str, path: str,
timeout: float = 30.0) -> tuple[int, bytes]:
req = urllib.request.Request(
base.rstrip("/") + path, method=method,
headers={"Authorization": f'MediaBrowser Token="{token}"'})
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
return resp.status, resp.read()
except urllib.error.HTTPError as exc:
return exc.code, exc.read()
def scan_task(base: str, token: str) -> dict:
"""The one task row that means something, and when its last run finished."""
code, body = request(base, token, "GET", "/ScheduledTasks")
if code != 200:
sys.exit(f"GET /ScheduledTasks -> HTTP {code}: {body[:200]!r}")
for task in json.loads(body):
if task.get("Name") == SCAN_TASK:
state = task.get("State")
return {"state": state, "running": state not in IDLE,
"lastScan": _epoch(task.get("EndTimeUtc"))}
sys.exit(f"the server does not list a {SCAN_TASK!r} task; refusing to guess")
def find_db(value: str) -> pathlib.Path:
wanted = [value] if value != "-" else []
wanted += [p for p in (os.environ.get("JELLYFIN_DB"),) if p]
wanted += list(DB_CANDIDATES)
for candidate in wanted:
path = pathlib.Path(candidate).expanduser()
if path.is_file():
return path
sys.exit("no Jellyfin database found; tried: " + ", ".join(wanted))
def recent_media(db: str, window: int) -> int:
since = time.strftime("%Y-%m-%d %H:%M:%S", time.gmtime(time.time() - window * 60))
con = sqlite3.connect(f"file:{find_db(db)}?mode=ro", uri=True)
try:
return con.execute(
"SELECT COUNT(*) FROM BaseItems WHERE DateModified >= ?", (since,)
).fetchone()[0]
finally:
con.close()
def main() -> None:
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("command", choices=["trigger", "status", "wait"])
ap.add_argument("--url", default="http://127.0.0.1:8096")
ap.add_argument("--key-file", default="~/.config/secrets/jellyfin")
ap.add_argument("--db", default="-", help="jellyfin.db; - searches the known homes")
ap.add_argument("--window", type=int, default=5,
help="minutes of DB writes that still count as recent")
ap.add_argument("--retries", type=int, default=3, help="extra attempts against a 429")
ap.add_argument("--retry-sleep", type=float, default=20.0)
ap.add_argument("--force", action="store_true",
help="trigger even though a scan is already running")
ap.add_argument("--interval", type=float, default=12.0, help="wait: seconds between reads")
ap.add_argument("--require-quiet", type=int, default=3,
help="wait: consecutive quiet reads before the library counts as settled")
ap.add_argument("--after", type=float, default=None,
help="status/wait: an epoch (a journal mtime) the scan must have "
"finished after; status exits non-zero while it has not")
ap.add_argument("--start-window", type=float, default=120.0,
help="wait: seconds to allow the scan to start before reporting "
"that the 204 queued nothing")
ap.add_argument("--timeout", type=float, default=1800.0, help="wait: seconds before giving up")
args = ap.parse_args()
token = read_key(args.key_file)
def state() -> dict:
task = scan_task(args.url, token)
out = {"scan": task["running"], "taskState": task["state"],
"lastScan": task["lastScan"],
"recentMedia": recent_media(args.db, args.window)}
if args.after is not None:
out["scannedAfter"] = bool(task["lastScan"] and task["lastScan"] >= args.after)
return out
if args.command == "status":
current = state()
print(json.dumps(current))
if args.after is not None and not current["scannedAfter"]:
sys.exit(f"the last scan finished before {args.after:.0f}; "
"a refresh is still owed")
return
if args.command == "wait":
start = time.monotonic()
ended_at_entry, quiet, started, first = None, 0, False, True
while True:
current = state()
if first:
first, ended_at_entry = False, current["lastScan"]
if (current["scan"]
or (current["lastScan"] and current["lastScan"] != ended_at_entry)
or current.get("scannedAfter")):
# The settle law: a Running sighting (or a run that finished while
# watching, or a --after the task already passes) must come BEFORE
# any claim that the library is quiet.
started = True
busy = current["scan"] or current["recentMedia"]
quiet = quiet + 1 if started and not busy else 0
print(json.dumps(current | {"quiet": quiet}), flush=True)
if quiet >= args.require_quiet:
return
elapsed = time.monotonic() - start
if not started and elapsed >= args.start_window:
sys.exit(f"{SCAN_TASK} never started: it read {current['taskState']!r} "
f"for {elapsed:.0f}s and the database took no writes. Either "
"the POST queued nothing — a POST that arrives while a scan "
"is already running cancels it, and every later 204 is "
"accepted and discarded, so restart the service and trigger "
"once — or the scan finished before this watch began, in "
"which case ask with --after <journal mtime> instead.")
if elapsed >= args.timeout:
sys.exit(f"still busy after {args.timeout:.0f}s: {current}")
time.sleep(args.interval)
if scan_task(args.url, token)["running"] and not args.force:
sys.exit(f"{SCAN_TASK} is already running on {args.url}: a second "
"POST /Library/Refresh cancels and restarts it, so the index "
"looks frozen while you wait. Poll `wait` instead of triggering "
"again (--force overrides).")
for attempt in range(args.retries + 1):
code, body = request(args.url, token, "POST", "/Library/Refresh")
if code == 204:
print("refresh accepted (204); the scan walks every library")
return
if code == 429:
if attempt < args.retries:
time.sleep(args.retry_sleep)
continue
sys.exit(f"Jellyfin is busy: {args.retries + 1} attempts all answered "
"429. Nothing was queued; run `wait`, then trigger again.")
sys.exit(f"POST /Library/Refresh -> HTTP {code}: {body[:200]!r}")
if __name__ == "__main__":
main()
scripts/media-refresh-plex.sh
Executable
+127
View File
@@ -0,0 +1,127 @@
#!/usr/bin/env bash
# Trigger and poll ONE Plex library section — the step that makes a media move visible.
#
# media-refresh-plex.sh trigger --library KEY
# media-refresh-plex.sh status --library KEY [--after EPOCH]
#
# One section only: refreshing all five costs a full walk of each and buys
# nothing for a rename that touched one tree. And a rename on the array is
# invisible until the section is rescanned — the server is bound at the SSD
# metadata view, so an array-side move is not an event its watcher ever sees.
#
# Two host realities shape this script:
# * PMS's HTTPD closes without a single byte on a request that arrives from
# another netns (the host and the 10.89.x.x container address both read as
# "empty reply"), and it serves TLS to remote clients. So the request rides
# plain http://127.0.0.1:32400 INSIDE the container, shipped in as a file —
# a three-layer ssh/podman/sh quoting chain only produces parse errors.
# * The token is `PlexOnlineToken` in the container's Preferences.xml (not
# PlexHTTPToken, which a claimed server may not carry). It is read inside
# the container and used from a curl config there: never argv (any process
# on the host reads /proc/<pid>/cmdline), never a URL (proxies log URLs).
#
# Progress needs no token and no container: `section_locations.scanned_at` in
# the library database is the same epoch the API prints as `scannedAt`
# (verified 2026-11-04), so `status` reads the database and compares it against
# --after. `trigger` answering 000 means PMS is still warming — retry, never
# restart the service over it. Proof that a rename actually landed is
# scripts/media-verify.py replaying the journal; this script only moves the
# scanner along.
set -euo pipefail
CONTAINER=plex
URL="http://127.0.0.1:32400"
PREFS='/config/Library/Application Support/Plex Media Server/Preferences.xml'
DB="/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.db"
SECTION=""
AFTER=""
TRIES=6
WARM_SLEEP=10
die() { echo "$*" >&2; exit 1; }
CMD="${1:-}"; shift || true
case "$CMD" in
trigger|status) ;;
*) die "usage: $(basename "$0") trigger|status --library KEY [--after EPOCH] [--container NAME] [--db PATH]" ;;
esac
while [ $# -gt 0 ]; do
case "$1" in
--library) SECTION="${2:-}"; shift 2 ;;
--after) AFTER="${2:-}"; shift 2 ;;
--tries) TRIES="${2:-}"; shift 2 ;;
--warm-sleep) WARM_SLEEP="${2:-}"; shift 2 ;;
--container) CONTAINER="${2:-}"; shift 2 ;;
--url) URL="${2:-}"; shift 2 ;;
--prefs) PREFS="${2:-}"; shift 2 ;;
--db) DB="${2:-}"; shift 2 ;;
*) die "unknown argument: $1" ;;
esac
done
[ -n "$SECTION" ] || die "--library is required (the section id from GET /library/sections; Movies is 1)"
command -v python3 >/dev/null || die "python3 is required to read the library database"
if [ "$CMD" = "status" ]; then
[ -r "$DB" ] || die "no readable Plex library database at $DB"
python3 - "$DB" "$SECTION" "$AFTER" <<'PY'
import datetime, sqlite3, sys
db, section, after = sys.argv[1], int(sys.argv[2]), sys.argv[3]
con = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
rows = con.execute(
"SELECT root_path, available, scanned_at FROM section_locations "
"WHERE library_section_id = ? ORDER BY id", (section,)).fetchall()
if not rows:
sys.exit(f"section {section} has no location in {db}; is the id right?")
worst = min(r[2] or 0 for r in rows)
for root, available, at in rows:
iso = (datetime.datetime.fromtimestamp(at, datetime.UTC).strftime("%Y-%m-%dT%H:%M:%SZ")
if at else "never")
print(f" available={available} scanned_at={iso} {root}")
print(f"section={section} scanned_at={worst}")
if after:
if worst < int(after):
sys.exit(f"section {section} was last scanned before {after}; a refresh is still owed")
PY
exit 0
fi
command -v podman >/dev/null || die "podman is required to reach PMS inside its container"
INNER="$(mktemp -t pxscan.XXXXXX)"
trap 'rm -f "$INNER"' EXIT
cat > "$INNER" <<'EOS'
# Runs INSIDE the plex container: $1 section key, $2 base URL, $3 Preferences.xml.
key="$1"; url="$2"; prefs="$3"
[ -r "$prefs" ] || { echo "no readable $prefs inside the container" >&2; exit 3; }
T=$(sed -n 's/.*PlexOnlineToken="\([^"]*\)".*/\1/p' "$prefs" | head -1)
[ -n "$T" ] || { echo "no PlexOnlineToken in $prefs" >&2; exit 3; }
cfg=$(mktemp)
( umask 077
{ printf 'header = "X-Plex-Token: %s"\n' "$T"
printf 'silent\nshow-error\nrequest = "POST"\n'
printf 'url = "%s/library/sections/%s/refresh"\n' "$url" "$key"; } > "$cfg" )
code=$(curl -K "$cfg" -o /dev/null -w '%{http_code}' || true)
shred -u "$cfg" 2>/dev/null || rm -f "$cfg"
[ "$code" = "401" ] && { echo "section $key: HTTP 401 — the token in $prefs is not accepted" >&2; exit 4; }
echo "$code"
EOS
attempt=1
while :; do
podman cp "$INNER" "$CONTAINER:/tmp/media-refresh-plex.sh" \
|| die "could not copy the runner into container $CONTAINER"
code="$(podman exec "$CONTAINER" sh /tmp/media-refresh-plex.sh "$SECTION" "$URL" "$PREFS" || true)"
podman exec "$CONTAINER" rm -f /tmp/media-refresh-plex.sh >/dev/null 2>&1 || true
case "$code" in
204|200) echo "refresh accepted ($code) for section $SECTION"; exit 0 ;;
000)
[ "$attempt" -ge "$TRIES" ] && die "PMS never answered after $TRIES attempts (000); it may still be warming — do not restart the service over it"
echo "section $SECTION: no answer yet ($attempt/$TRIES), PMS is warming" >&2 ;;
*) die "refresh of section $SECTION failed: HTTP ${code:-curl-error}" ;;
esac
attempt=$((attempt + 1))
sleep "$WARM_SLEEP"
done