Merge pull request 'Media refresh drivers: the two scan doors become commands' (#140) from feat/media-refresh-drivers into main

serverAppConfigs-docs/Dankserver/Jellyfin.md

@@ -309,33 +309,36 @@ scripts/jellyfin-providers.sh verify    # plugin + key LENGTHS, never key bytes
 
 ## Triggering a scan
 
-The scan key lives in `~/.config/secrets/jellyfin` (mode 600; the same value is
-`JELLYFIN_API_KEY` in the ignored `dankserver/.env`). It rides a mode-600 curl
-config, never a header argument, in the `MediaBrowser` form: 12.4 authenticates
-only `Authorization: MediaBrowser Token="…"` and refuses a bare `X-Emby-Token`
-for every key, valid or not. Before believing a 401, prove the key against the
-server's own table — `select count(*) from ApiKeys where AccessToken = ?` on
-`jellyfin.db` opened `mode=ro&immutable=1`. The backslashes below matter:
+Run the driver from the deploy checkout: `scripts/media-refresh-jellyfin.py
+trigger`, then `… wait`. It calls the library settled only on the rule below — the
+task seen `Running` (or a run that finished while it watched) first, then
+`--require-quiet` reads with no scan and no writes — and refuses by name when the
+scan never started, because a watcher that saw no `Running` would otherwise report
+the pre-scan library as the fresh one. `… status --after $(stat -c %Y <journal>)`
+is the "has the scan reached my moves" answer, read from the task's `EndTimeUtc`.
 
-```bash
-umask 077
-{ printf 'header = "Authorization: MediaBrowser Token=\\"%s\\""\n' \
-    "$(cat ~/.config/secrets/jellyfin)"
-  echo 'url = "http://127.0.0.1:8096/Library/Refresh"'
-  echo 'request = "POST"'; } >/tmp/jf.hdr
-curl -s -K /tmp/jf.hdr -o /dev/null -w "jellyfin scan: %{http_code}\n"
-shred -u /tmp/jf.hdr
-```
+It carries every law below. Hand-written, the door is a `curl -K` on a mode-600
+config: the scan key lives in `~/.config/secrets/jellyfin` (mode 600; the same
+value is `JELLYFIN_API_KEY` in the ignored `dankserver/.env`) and rides as
+`Authorization: MediaBrowser Token="…"` — 12.4 refuses a bare `X-Emby-Token` for
+every key, valid or not — never as a header argument, because
+`/proc/<pid>/cmdline` is world-readable. `recentMedia` counts
+`BaseItems.DateModified` rows in `jellyfin.db`, which on this host sits in the
+quadlet's config bind at `/home/dank/Containers/jellyfin/config/data/`, not the
+stock `~/.local/share/jellyfin`.
 
 `204` is acceptance; anything else queued no scan, so re-issue the identical
 request instead of diagnosing. A `401` from a key that answers `200` on a
 hand-typed request is this block's quoting, not the server: bash `printf`
 collapses `\"` to `"`, curl ends the quoted value at that quote, and Jellyfin
 receives `Authorization: MediaBrowser Token=` with nothing after it (`\\"` answers
-204, `\"` answers 401 — both run on this host).
-Reading the progress with the same config trick takes **one** URL: a config `url`
-line *plus* a command-line URL makes curl perform both transfers, and the second
-dumps the whole `ScheduledTasks` body onto your stdout.
+204, `\"` answers 401 — both run on this host). Before believing a 401, prove the
+key against the server's own table — `select count(*) from ApiKeys where
+AccessToken = ?` on `jellyfin.db` opened plain `mode=ro` (`immutable=1` only as
+the fallback for a database whose `-shm` cannot be read; it skips the write-ahead
+log, see [[Media Relayout]]). Reading progress with the same config trick takes
+**one** URL: a config `url` line *plus* a command-line URL makes curl perform both
+transfers, and the second dumps the whole `ScheduledTasks` body onto your stdout.
 
 One accepted refresh is the whole procedure. A second one while a scan runs moves
 `Scan Media Library` to `Cancelling`, and once the runner holds `Cancelling` every

serverAppConfigs-docs/Dankserver/Media Duplicate Prune.md

@@ -151,6 +151,18 @@ Each of these is a file that would have been destroyed without it:
 - Media containers only. Grouping every extension buries the report in subtitle
   and cover pairs — and a duration guard that appears to fire on 720 groups is
   counting subtitles, not protecting films.
+- **A short file named after a film is promotion, not a movie.** Thirty-two files
+  in `Movies` — 31 clips and the `.nfo` filed beside one of them — carried real
+  titles (`The Descent (2005) - SD.mkv`) and were 8–20-second h264 video at
+  640×360, 1024×576 or 1280×720, 49 MiB all together —
+  25 of them in a `_release-group watermarks/` folder and the rest at the library
+  root or inside one film's folder. No duplicate screen ever catches them: nothing
+  shares their size, and the feature-length guard above makes them invisible to the
+  similar-title pass too. They were held as `.AGENTTRASH/20261104-promo-clips`
+  (journal `/mnt/entropy/.media-campaign/promo-hold.jsonl`, which survives) and purged
+  the same evening, once both databases read `REAL: 0` against that journal.
+  The census that finds the next batch is a duration floor, not a hash: a video under
+  a minute on a name that reads like a feature is advertising.
 
 ## The hold
 

serverAppConfigs-docs/Dankserver/Media Relayout.md

@@ -99,12 +99,17 @@ interrupted apply still replays the batch and resumes with nothing lost. Batch 1
 is the old behaviour; the only reason to name it is debugging, because on the
 spinning array it spends most of a large apply's wall time in fsyncs.
 
-After it lands, both servers need a scan or the change is invisible. [[Plex]] and
-[[Jellyfin]] own the exact requests; two laws hold at this step: `204` is
-acceptance and anything else queued nothing — re-issue the identical request
-rather than diagnosing — and ONE accepted refresh is the whole procedure, because
-a second while a scan runs moves `Scan Media Library` to `Cancelling` and every
-later `204` is accepted and then discarded.
+After it lands, both servers need a scan or the change is invisible. The two
+drivers are the doors — `scripts/media-refresh-jellyfin.py trigger` then `… wait`,
+and only after it settles, `scripts/media-refresh-plex.sh trigger --library <key>`
+with `… status --library <key> --after $(stat -c %Y <journal>)` as the poll. Run
+them in that order and never together: Jellyfin's only door walks every library,
+every Plex section forks its own scanner walk, and the array is the bottleneck
+both servers share. [[Plex]] and [[Jellyfin]] own the laws behind the drivers; two
+of them decide this step: `204` is acceptance and anything else queued nothing —
+re-issue the identical request rather than diagnosing — and ONE accepted refresh is
+the whole procedure, because a second while a scan runs moves `Scan Media Library`
+to `Cancelling` and every later `204` is accepted and then discarded.
 
 ### Proving the move landed
 

serverAppConfigs-docs/Dankserver/Plex.md

@@ -106,43 +106,39 @@ root runs as the `dank` user, and the deployer adopts the data root with
 ## Triggering a scan
 
 A rename on disk is invisible until the section is rescanned, and a moved file is
-a `Couldn't find the file to stream` 404 until that scan passes it. Run this from
-the host shell through `podman exec`, never from a neighbour container: PMS's
-HTTPD closes without a single byte on any request that arrives from another
-container's netns (10.89.x.x reads as "empty reply", not as a 401). The token
-rides stdin, never argv, where any process on the host could read it out of
-`/proc/<pid>/cmdline`; it lives in `Preferences.xml` inside the RW data bind.
+a `Couldn't find the file to stream` 404 until that scan passes it. Run the
+driver from the host shell; the section keys are `1` Movies, `3` TV Shows,
+`5` Audiobooks, `6` Music, `11` VODs & Live Events:
 
 ```bash
-umask 077
-podman exec plex sh -c "sed -n 's/.*PlexOnlineToken=\"\\([^\"]*\\)\".*/\\1/p' \
-  '/config/Library/Application Support/Plex Media Server/Preferences.xml'" >/tmp/px.tok
-cat >/tmp/pxscan.sh <<'EOS'
-read -r T
-cfg=$(mktemp)
-mk() { { printf 'header = "X-Plex-Token: %s"\nheader = "Accept: application/json"\n' "$T"
-        printf 'url = "%s"\n' "$1"; } > "$cfg"; }
-mk "http://127.0.0.1:32400/library/sections"
-for k in $(curl -s -K "$cfg" | grep -o '"key":"[0-9]*"' | cut -d\" -f4); do
-  mk "http://127.0.0.1:32400/library/sections/$k/refresh"
-  printf 'request = "POST"\n' >> "$cfg"
-  echo "plex section $k: $(curl -s -o /dev/null -w '%{http_code}' -K "$cfg")"
-done
-shred -u "$cfg"
-EOS
-podman cp /tmp/pxscan.sh plex:/tmp/pxscan.sh
-podman exec -i plex sh /tmp/pxscan.sh </tmp/px.tok
-shred -u /tmp/px.tok
+scripts/media-refresh-plex.sh trigger --library 1
+scripts/media-refresh-plex.sh status  --library 1 --after "$(stat -c %Y <journal>)"
 ```
 
-Every section answers `204` — or `200` with a Directory body when
-`Accept: application/json` is sent. A refresh fired while PMS is still warming
-answers `000` instead (the empty reply of gotcha 4, arriving mid-request): retry
-until it answers and never restart the service over it. Inside the quoted
-`printf` format a backslash-newline is LITERAL, not a continuation, so `mk`
-writes two `printf`s: one glued line and one stray backslash is a config file
-curl rejects, and the failure prints as `option -K: is unknown`, never as a bad
-config line.
+Exactly ONE section: a refresh costs a full walk of that tree, and refreshing all
+five for a rename that touched one buys nothing. `trigger` answers
+`refresh accepted (204)` — `200` when a body is requested — and `000` means PMS is
+still warming, which the driver retries out loud; a service restart is never the
+fix. Progress needs no token and no container: `section_locations.scanned_at` in
+the library database carries the same epoch `GET /library/sections` prints as
+`scannedAt` (verified 2026-11-04), and that is what `status` compares `--after`
+against.
+
+The driver ships its runner into the container with `podman cp` rather than
+quoting a command through `podman exec sh -c`, because PMS's HTTPD closes without
+a single byte on a request that arrives from another netns — the host address and
+the `10.89.x.x` container address both read as "empty reply", never as a 401 —
+and because a three-layer `ssh → podman exec → sh -c` quote chain dies on a parse
+error long before it reaches the server. The request therefore rides plain
+`http://127.0.0.1:32400` from inside the container, where insecure connections
+are allowed (gotcha 5). The credential is `PlexOnlineToken` in the container's
+`Preferences.xml` — NOT `PlexHTTPToken`, which a claimed server need not carry —
+read inside the container into a mode-600 curl config there, so it never enters
+argv (`/proc/<pid>/cmdline` is world-readable) and never a URL (proxies log
+URLs). Hand-writing that config has one more trap: inside a quoted `printf`
+format a backslash-newline is LITERAL, so a two-line string becomes one glued
+line plus a stray backslash, and curl answers `option -K: is unknown` — never
+"bad config line".
 
 Proof that a rename landed is not a scan response: `scripts/media-verify.py`
 replays the run's journal against the database — the only read that