docs(vault): the scan pages name their drivers, and a class no screen sees

Plex.md and Jellyfin.md keep every law and point at the driver that now carries
them, so the procedure has one owner and the pages keep the failure signatures.
Jellyfin.md also records where `jellyfin.db` actually lives on this host (the
quadlet's config bind, not the stock path) and demotes `immutable=1` on the
`ApiKeys` read to the `-shm` fallback, because the database is WAL and
`immutable` skips the log. Plex.md gains the live section keys and the
`scanned_at == scannedAt` fact.

Media Relayout.md's hand-off after an apply names both drivers and states the
ordering law: Jellyfin first and alone, since its one door walks every library
while Plex forks a scanner per section, and the array is what both share.

Media Duplicate Prune.md gains the guard the 2026-11-04 hold taught: 32 files
carrying real film titles were 8-20-second h264 clips (49 MiB, 25 of them in a
`_release-group watermarks/` folder), invisible to every duplicate screen
because nothing shares their size and the feature-length guard hides them. The
census that finds the next batch is a duration floor, not a hash.
This commit is contained in:
Frank committed 2026-11-04 11:29:03 -08:00
1 parent e7d7c6eea2
commit 95375683d5
4 files changed
+73 -59

No files matched your search

+23 -20
View File
@@ -309,33 +309,36 @@ scripts/jellyfin-providers.sh verify # plugin + key LENGTHS, never key bytes
## Triggering a scan ## Triggering a scan
The scan key lives in `~/.config/secrets/jellyfin` (mode 600; the same value is Run the driver from the deploy checkout: `scripts/media-refresh-jellyfin.py
`JELLYFIN_API_KEY` in the ignored `dankserver/.env`). It rides a mode-600 curl trigger`, then `… wait`. It calls the library settled only on the rule below — the
config, never a header argument, in the `MediaBrowser` form: 12.4 authenticates task seen `Running` (or a run that finished while it watched) first, then
only `Authorization: MediaBrowser Token="…"` and refuses a bare `X-Emby-Token` `--require-quiet` reads with no scan and no writes — and refuses by name when the
for every key, valid or not. Before believing a 401, prove the key against the scan never started, because a watcher that saw no `Running` would otherwise report
server's own table — `select count(*) from ApiKeys where AccessToken = ?` on the pre-scan library as the fresh one. `… status --after $(stat -c %Y <journal>)`
`jellyfin.db` opened `mode=ro&immutable=1`. The backslashes below matter: is the "has the scan reached my moves" answer, read from the task's `EndTimeUtc`.
```bash It carries every law below. Hand-written, the door is a `curl -K` on a mode-600
umask 077 config: the scan key lives in `~/.config/secrets/jellyfin` (mode 600; the same
{ printf 'header = "Authorization: MediaBrowser Token=\\"%s\\""\n' \ value is `JELLYFIN_API_KEY` in the ignored `dankserver/.env`) and rides as
"$(cat ~/.config/secrets/jellyfin)" `Authorization: MediaBrowser Token="…"` — 12.4 refuses a bare `X-Emby-Token` for
echo 'url = "http://127.0.0.1:8096/Library/Refresh"' every key, valid or not — never as a header argument, because
echo 'request = "POST"'; } >/tmp/jf.hdr `/proc/<pid>/cmdline` is world-readable. `recentMedia` counts
curl -s -K /tmp/jf.hdr -o /dev/null -w "jellyfin scan: %{http_code}\n" `BaseItems.DateModified` rows in `jellyfin.db`, which on this host sits in the
shred -u /tmp/jf.hdr quadlet's config bind at `/home/dank/Containers/jellyfin/config/data/`, not the
``` stock `~/.local/share/jellyfin`.
`204` is acceptance; anything else queued no scan, so re-issue the identical `204` is acceptance; anything else queued no scan, so re-issue the identical
request instead of diagnosing. A `401` from a key that answers `200` on a request instead of diagnosing. A `401` from a key that answers `200` on a
hand-typed request is this block's quoting, not the server: bash `printf` hand-typed request is this block's quoting, not the server: bash `printf`
collapses `\"` to `"`, curl ends the quoted value at that quote, and Jellyfin collapses `\"` to `"`, curl ends the quoted value at that quote, and Jellyfin
receives `Authorization: MediaBrowser Token=` with nothing after it (`\\"` answers receives `Authorization: MediaBrowser Token=` with nothing after it (`\\"` answers
204, `\"` answers 401 — both run on this host). 204, `\"` answers 401 — both run on this host). Before believing a 401, prove the
Reading the progress with the same config trick takes **one** URL: a config `url` key against the server's own table — `select count(*) from ApiKeys where
line *plus* a command-line URL makes curl perform both transfers, and the second AccessToken = ?` on `jellyfin.db` opened plain `mode=ro` (`immutable=1` only as
dumps the whole `ScheduledTasks` body onto your stdout. the fallback for a database whose `-shm` cannot be read; it skips the write-ahead
log, see [[Media Relayout]]). Reading progress with the same config trick takes
**one** URL: a config `url` line *plus* a command-line URL makes curl perform both
transfers, and the second dumps the whole `ScheduledTasks` body onto your stdout.
One accepted refresh is the whole procedure. A second one while a scan runs moves One accepted refresh is the whole procedure. A second one while a scan runs moves
`Scan Media Library` to `Cancelling`, and once the runner holds `Cancelling` every `Scan Media Library` to `Cancelling`, and once the runner holds `Cancelling` every
@@ -151,6 +151,16 @@ Each of these is a file that would have been destroyed without it:
- Media containers only. Grouping every extension buries the report in subtitle - Media containers only. Grouping every extension buries the report in subtitle
and cover pairs — and a duration guard that appears to fire on 720 groups is and cover pairs — and a duration guard that appears to fire on 720 groups is
counting subtitles, not protecting films. counting subtitles, not protecting films.
- **A short file named after a film is promotion, not a movie.** Thirty-two files
in `Movies` carried real titles (`The Descent (2005) - SD.mkv`) and were
8–20-second h264 clips — 640×360, 1024×576 or 1280×720, 49 MiB all together,
25 of them in a `_release-group watermarks/` folder and the rest at the library
root or inside one film's folder. No duplicate screen ever catches them: nothing
shares their size, and the feature-length guard above makes them invisible to the
similar-title pass too. Held as `.AGENTTRASH/20261104-promo-clips`, journalled at
`/mnt/entropy/.media-campaign/promo-hold.jsonl`. The census that finds the next
batch is a duration floor, not a hash: a video under a minute on a name that
reads like a feature is advertising.
## The hold ## The hold
@@ -99,12 +99,17 @@ interrupted apply still replays the batch and resumes with nothing lost. Batch 1
is the old behaviour; the only reason to name it is debugging, because on the is the old behaviour; the only reason to name it is debugging, because on the
spinning array it spends most of a large apply's wall time in fsyncs. spinning array it spends most of a large apply's wall time in fsyncs.
After it lands, both servers need a scan or the change is invisible. [[Plex]] and After it lands, both servers need a scan or the change is invisible. The two
[[Jellyfin]] own the exact requests; two laws hold at this step: `204` is drivers are the doors — `scripts/media-refresh-jellyfin.py trigger` then `… wait`,
acceptance and anything else queued nothing — re-issue the identical request and only after it settles, `scripts/media-refresh-plex.sh trigger --library <key>`
rather than diagnosing — and ONE accepted refresh is the whole procedure, because with `… status --library <key> --after $(stat -c %Y <journal>)` as the poll. Run
a second while a scan runs moves `Scan Media Library` to `Cancelling` and every them in that order and never together: Jellyfin's only door walks every library,
later `204` is accepted and then discarded. every Plex section forks its own scanner walk, and the array is the bottleneck
both servers share. [[Plex]] and [[Jellyfin]] own the laws behind the drivers; two
of them decide this step: `204` is acceptance and anything else queued nothing —
re-issue the identical request rather than diagnosing — and ONE accepted refresh is
the whole procedure, because a second while a scan runs moves `Scan Media Library`
to `Cancelling` and every later `204` is accepted and then discarded.
### Proving the move landed ### Proving the move landed
+29 -33
View File
@@ -106,43 +106,39 @@ root runs as the `dank` user, and the deployer adopts the data root with
## Triggering a scan ## Triggering a scan
A rename on disk is invisible until the section is rescanned, and a moved file is A rename on disk is invisible until the section is rescanned, and a moved file is
a `Couldn't find the file to stream` 404 until that scan passes it. Run this from a `Couldn't find the file to stream` 404 until that scan passes it. Run the
the host shell through `podman exec`, never from a neighbour container: PMS's driver from the host shell; the section keys are `1` Movies, `3` TV Shows,
HTTPD closes without a single byte on any request that arrives from another `5` Audiobooks, `6` Music, `11` VODs & Live Events:
container's netns (10.89.x.x reads as "empty reply", not as a 401). The token
rides stdin, never argv, where any process on the host could read it out of
`/proc/<pid>/cmdline`; it lives in `Preferences.xml` inside the RW data bind.
```bash ```bash
umask 077 scripts/media-refresh-plex.sh trigger --library 1
podman exec plex sh -c "sed -n 's/.*PlexOnlineToken=\"\\([^\"]*\\)\".*/\\1/p' \ scripts/media-refresh-plex.sh status --library 1 --after "$(stat -c %Y <journal>)"
'/config/Library/Application Support/Plex Media Server/Preferences.xml'" >/tmp/px.tok
cat >/tmp/pxscan.sh <<'EOS'
read -r T
cfg=$(mktemp)
mk() { { printf 'header = "X-Plex-Token: %s"\nheader = "Accept: application/json"\n' "$T"
printf 'url = "%s"\n' "$1"; } > "$cfg"; }
mk "http://127.0.0.1:32400/library/sections"
for k in $(curl -s -K "$cfg" | grep -o '"key":"[0-9]*"' | cut -d\" -f4); do
mk "http://127.0.0.1:32400/library/sections/$k/refresh"
printf 'request = "POST"\n' >> "$cfg"
echo "plex section $k: $(curl -s -o /dev/null -w '%{http_code}' -K "$cfg")"
done
shred -u "$cfg"
EOS
podman cp /tmp/pxscan.sh plex:/tmp/pxscan.sh
podman exec -i plex sh /tmp/pxscan.sh </tmp/px.tok
shred -u /tmp/px.tok
``` ```
Every section answers `204` — or `200` with a Directory body when Exactly ONE section: a refresh costs a full walk of that tree, and refreshing all
`Accept: application/json` is sent. A refresh fired while PMS is still warming five for a rename that touched one buys nothing. `trigger` answers
answers `000` instead (the empty reply of gotcha 4, arriving mid-request): retry `refresh accepted (204)` — `200` when a body is requested — and `000` means PMS is
until it answers and never restart the service over it. Inside the quoted still warming, which the driver retries out loud; a service restart is never the
`printf` format a backslash-newline is LITERAL, not a continuation, so `mk` fix. Progress needs no token and no container: `section_locations.scanned_at` in
writes two `printf`s: one glued line and one stray backslash is a config file the library database carries the same epoch `GET /library/sections` prints as
curl rejects, and the failure prints as `option -K: is unknown`, never as a bad `scannedAt` (verified 2026-11-04), and that is what `status` compares `--after`
config line. against.
The driver ships its runner into the container with `podman cp` rather than
quoting a command through `podman exec sh -c`, because PMS's HTTPD closes without
a single byte on a request that arrives from another netns — the host address and
the `10.89.x.x` container address both read as "empty reply", never as a 401 —
and because a three-layer `ssh → podman exec → sh -c` quote chain dies on a parse
error long before it reaches the server. The request therefore rides plain
`http://127.0.0.1:32400` from inside the container, where insecure connections
are allowed (gotcha 5). The credential is `PlexOnlineToken` in the container's
`Preferences.xml` — NOT `PlexHTTPToken`, which a claimed server need not carry —
read inside the container into a mode-600 curl config there, so it never enters
argv (`/proc/<pid>/cmdline` is world-readable) and never a URL (proxies log
URLs). Hand-writing that config has one more trap: inside a quoted `printf`
format a backslash-newline is LITERAL, so a two-line string becomes one glued
line plus a stray backslash, and curl answers `option -K: is unknown` — never
"bad config line".
Proof that a rename landed is not a scan response: `scripts/media-verify.py` Proof that a rename landed is not a scan response: `scripts/media-verify.py`
replays the run's journal against the database — the only read that replays the run's journal against the database — the only read that