docs(vault): the scan pages name their drivers, and a class no screen sees
Plex.md and Jellyfin.md keep every law and point at the driver that now carries them, so the procedure has one owner and the pages keep the failure signatures. Jellyfin.md also records where `jellyfin.db` actually lives on this host (the quadlet's config bind, not the stock path) and demotes `immutable=1` on the `ApiKeys` read to the `-shm` fallback, because the database is WAL and `immutable` skips the log. Plex.md gains the live section keys and the `scanned_at == scannedAt` fact. Media Relayout.md's hand-off after an apply names both drivers and states the ordering law: Jellyfin first and alone, since its one door walks every library while Plex forks a scanner per section, and the array is what both share. Media Duplicate Prune.md gains the guard the 2026-11-04 hold taught: 32 files carrying real film titles were 8-20-second h264 clips (49 MiB, 25 of them in a `_release-group watermarks/` folder), invisible to every duplicate screen because nothing shares their size and the feature-length guard hides them. The census that finds the next batch is a duration floor, not a hash.
This commit is contained in:
1 parent
e7d7c6eea2
commit
95375683d5
4 files changed
+73
-59
No files matched your search
@@ -309,33 +309,36 @@ scripts/jellyfin-providers.sh verify # plugin + key LENGTHS, never key bytes
|
|||||||
|
|
||||||
## Triggering a scan
|
## Triggering a scan
|
||||||
|
|
||||||
The scan key lives in `~/.config/secrets/jellyfin` (mode 600; the same value is
|
Run the driver from the deploy checkout: `scripts/media-refresh-jellyfin.py
|
||||||
`JELLYFIN_API_KEY` in the ignored `dankserver/.env`). It rides a mode-600 curl
|
trigger`, then `… wait`. It calls the library settled only on the rule below — the
|
||||||
config, never a header argument, in the `MediaBrowser` form: 12.4 authenticates
|
task seen `Running` (or a run that finished while it watched) first, then
|
||||||
only `Authorization: MediaBrowser Token="…"` and refuses a bare `X-Emby-Token`
|
`--require-quiet` reads with no scan and no writes — and refuses by name when the
|
||||||
for every key, valid or not. Before believing a 401, prove the key against the
|
scan never started, because a watcher that saw no `Running` would otherwise report
|
||||||
server's own table — `select count(*) from ApiKeys where AccessToken = ?` on
|
the pre-scan library as the fresh one. `… status --after $(stat -c %Y <journal>)`
|
||||||
`jellyfin.db` opened `mode=ro&immutable=1`. The backslashes below matter:
|
is the "has the scan reached my moves" answer, read from the task's `EndTimeUtc`.
|
||||||
|
|
||||||
```bash
|
It carries every law below. Hand-written, the door is a `curl -K` on a mode-600
|
||||||
umask 077
|
config: the scan key lives in `~/.config/secrets/jellyfin` (mode 600; the same
|
||||||
{ printf 'header = "Authorization: MediaBrowser Token=\\"%s\\""\n' \
|
value is `JELLYFIN_API_KEY` in the ignored `dankserver/.env`) and rides as
|
||||||
"$(cat ~/.config/secrets/jellyfin)"
|
`Authorization: MediaBrowser Token="…"` — 12.4 refuses a bare `X-Emby-Token` for
|
||||||
echo 'url = "http://127.0.0.1:8096/Library/Refresh"'
|
every key, valid or not — never as a header argument, because
|
||||||
echo 'request = "POST"'; } >/tmp/jf.hdr
|
`/proc/<pid>/cmdline` is world-readable. `recentMedia` counts
|
||||||
curl -s -K /tmp/jf.hdr -o /dev/null -w "jellyfin scan: %{http_code}\n"
|
`BaseItems.DateModified` rows in `jellyfin.db`, which on this host sits in the
|
||||||
shred -u /tmp/jf.hdr
|
quadlet's config bind at `/home/dank/Containers/jellyfin/config/data/`, not the
|
||||||
```
|
stock `~/.local/share/jellyfin`.
|
||||||
|
|
||||||
`204` is acceptance; anything else queued no scan, so re-issue the identical
|
`204` is acceptance; anything else queued no scan, so re-issue the identical
|
||||||
request instead of diagnosing. A `401` from a key that answers `200` on a
|
request instead of diagnosing. A `401` from a key that answers `200` on a
|
||||||
hand-typed request is this block's quoting, not the server: bash `printf`
|
hand-typed request is this block's quoting, not the server: bash `printf`
|
||||||
collapses `\"` to `"`, curl ends the quoted value at that quote, and Jellyfin
|
collapses `\"` to `"`, curl ends the quoted value at that quote, and Jellyfin
|
||||||
receives `Authorization: MediaBrowser Token=` with nothing after it (`\\"` answers
|
receives `Authorization: MediaBrowser Token=` with nothing after it (`\\"` answers
|
||||||
204, `\"` answers 401 — both run on this host).
|
204, `\"` answers 401 — both run on this host). Before believing a 401, prove the
|
||||||
Reading the progress with the same config trick takes **one** URL: a config `url`
|
key against the server's own table — `select count(*) from ApiKeys where
|
||||||
line *plus* a command-line URL makes curl perform both transfers, and the second
|
AccessToken = ?` on `jellyfin.db` opened plain `mode=ro` (`immutable=1` only as
|
||||||
dumps the whole `ScheduledTasks` body onto your stdout.
|
the fallback for a database whose `-shm` cannot be read; it skips the write-ahead
|
||||||
|
log, see [[Media Relayout]]). Reading progress with the same config trick takes
|
||||||
|
**one** URL: a config `url` line *plus* a command-line URL makes curl perform both
|
||||||
|
transfers, and the second dumps the whole `ScheduledTasks` body onto your stdout.
|
||||||
|
|
||||||
One accepted refresh is the whole procedure. A second one while a scan runs moves
|
One accepted refresh is the whole procedure. A second one while a scan runs moves
|
||||||
`Scan Media Library` to `Cancelling`, and once the runner holds `Cancelling` every
|
`Scan Media Library` to `Cancelling`, and once the runner holds `Cancelling` every
|
||||||
|
|||||||
@@ -151,6 +151,16 @@ Each of these is a file that would have been destroyed without it:
|
|||||||
- Media containers only. Grouping every extension buries the report in subtitle
|
- Media containers only. Grouping every extension buries the report in subtitle
|
||||||
and cover pairs — and a duration guard that appears to fire on 720 groups is
|
and cover pairs — and a duration guard that appears to fire on 720 groups is
|
||||||
counting subtitles, not protecting films.
|
counting subtitles, not protecting films.
|
||||||
|
- **A short file named after a film is promotion, not a movie.** Thirty-two files
|
||||||
|
in `Movies` carried real titles (`The Descent (2005) - SD.mkv`) and were
|
||||||
|
8–20-second h264 clips — 640×360, 1024×576 or 1280×720, 49 MiB all together,
|
||||||
|
25 of them in a `_release-group watermarks/` folder and the rest at the library
|
||||||
|
root or inside one film's folder. No duplicate screen ever catches them: nothing
|
||||||
|
shares their size, and the feature-length guard above makes them invisible to the
|
||||||
|
similar-title pass too. Held as `.AGENTTRASH/20261104-promo-clips`, journalled at
|
||||||
|
`/mnt/entropy/.media-campaign/promo-hold.jsonl`. The census that finds the next
|
||||||
|
batch is a duration floor, not a hash: a video under a minute on a name that
|
||||||
|
reads like a feature is advertising.
|
||||||
|
|
||||||
## The hold
|
## The hold
|
||||||
|
|
||||||
|
|||||||
@@ -99,12 +99,17 @@ interrupted apply still replays the batch and resumes with nothing lost. Batch 1
|
|||||||
is the old behaviour; the only reason to name it is debugging, because on the
|
is the old behaviour; the only reason to name it is debugging, because on the
|
||||||
spinning array it spends most of a large apply's wall time in fsyncs.
|
spinning array it spends most of a large apply's wall time in fsyncs.
|
||||||
|
|
||||||
After it lands, both servers need a scan or the change is invisible. [[Plex]] and
|
After it lands, both servers need a scan or the change is invisible. The two
|
||||||
[[Jellyfin]] own the exact requests; two laws hold at this step: `204` is
|
drivers are the doors — `scripts/media-refresh-jellyfin.py trigger` then `… wait`,
|
||||||
acceptance and anything else queued nothing — re-issue the identical request
|
and only after it settles, `scripts/media-refresh-plex.sh trigger --library <key>`
|
||||||
rather than diagnosing — and ONE accepted refresh is the whole procedure, because
|
with `… status --library <key> --after $(stat -c %Y <journal>)` as the poll. Run
|
||||||
a second while a scan runs moves `Scan Media Library` to `Cancelling` and every
|
them in that order and never together: Jellyfin's only door walks every library,
|
||||||
later `204` is accepted and then discarded.
|
every Plex section forks its own scanner walk, and the array is the bottleneck
|
||||||
|
both servers share. [[Plex]] and [[Jellyfin]] own the laws behind the drivers; two
|
||||||
|
of them decide this step: `204` is acceptance and anything else queued nothing —
|
||||||
|
re-issue the identical request rather than diagnosing — and ONE accepted refresh is
|
||||||
|
the whole procedure, because a second while a scan runs moves `Scan Media Library`
|
||||||
|
to `Cancelling` and every later `204` is accepted and then discarded.
|
||||||
|
|
||||||
### Proving the move landed
|
### Proving the move landed
|
||||||
|
|
||||||
|
|||||||
@@ -106,43 +106,39 @@ root runs as the `dank` user, and the deployer adopts the data root with
|
|||||||
## Triggering a scan
|
## Triggering a scan
|
||||||
|
|
||||||
A rename on disk is invisible until the section is rescanned, and a moved file is
|
A rename on disk is invisible until the section is rescanned, and a moved file is
|
||||||
a `Couldn't find the file to stream` 404 until that scan passes it. Run this from
|
a `Couldn't find the file to stream` 404 until that scan passes it. Run the
|
||||||
the host shell through `podman exec`, never from a neighbour container: PMS's
|
driver from the host shell; the section keys are `1` Movies, `3` TV Shows,
|
||||||
HTTPD closes without a single byte on any request that arrives from another
|
`5` Audiobooks, `6` Music, `11` VODs & Live Events:
|
||||||
container's netns (10.89.x.x reads as "empty reply", not as a 401). The token
|
|
||||||
rides stdin, never argv, where any process on the host could read it out of
|
|
||||||
`/proc/<pid>/cmdline`; it lives in `Preferences.xml` inside the RW data bind.
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
umask 077
|
scripts/media-refresh-plex.sh trigger --library 1
|
||||||
podman exec plex sh -c "sed -n 's/.*PlexOnlineToken=\"\\([^\"]*\\)\".*/\\1/p' \
|
scripts/media-refresh-plex.sh status --library 1 --after "$(stat -c %Y <journal>)"
|
||||||
'/config/Library/Application Support/Plex Media Server/Preferences.xml'" >/tmp/px.tok
|
|
||||||
cat >/tmp/pxscan.sh <<'EOS'
|
|
||||||
read -r T
|
|
||||||
cfg=$(mktemp)
|
|
||||||
mk() { { printf 'header = "X-Plex-Token: %s"\nheader = "Accept: application/json"\n' "$T"
|
|
||||||
printf 'url = "%s"\n' "$1"; } > "$cfg"; }
|
|
||||||
mk "http://127.0.0.1:32400/library/sections"
|
|
||||||
for k in $(curl -s -K "$cfg" | grep -o '"key":"[0-9]*"' | cut -d\" -f4); do
|
|
||||||
mk "http://127.0.0.1:32400/library/sections/$k/refresh"
|
|
||||||
printf 'request = "POST"\n' >> "$cfg"
|
|
||||||
echo "plex section $k: $(curl -s -o /dev/null -w '%{http_code}' -K "$cfg")"
|
|
||||||
done
|
|
||||||
shred -u "$cfg"
|
|
||||||
EOS
|
|
||||||
podman cp /tmp/pxscan.sh plex:/tmp/pxscan.sh
|
|
||||||
podman exec -i plex sh /tmp/pxscan.sh </tmp/px.tok
|
|
||||||
shred -u /tmp/px.tok
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Every section answers `204` — or `200` with a Directory body when
|
Exactly ONE section: a refresh costs a full walk of that tree, and refreshing all
|
||||||
`Accept: application/json` is sent. A refresh fired while PMS is still warming
|
five for a rename that touched one buys nothing. `trigger` answers
|
||||||
answers `000` instead (the empty reply of gotcha 4, arriving mid-request): retry
|
`refresh accepted (204)` — `200` when a body is requested — and `000` means PMS is
|
||||||
until it answers and never restart the service over it. Inside the quoted
|
still warming, which the driver retries out loud; a service restart is never the
|
||||||
`printf` format a backslash-newline is LITERAL, not a continuation, so `mk`
|
fix. Progress needs no token and no container: `section_locations.scanned_at` in
|
||||||
writes two `printf`s: one glued line and one stray backslash is a config file
|
the library database carries the same epoch `GET /library/sections` prints as
|
||||||
curl rejects, and the failure prints as `option -K: is unknown`, never as a bad
|
`scannedAt` (verified 2026-11-04), and that is what `status` compares `--after`
|
||||||
config line.
|
against.
|
||||||
|
|
||||||
|
The driver ships its runner into the container with `podman cp` rather than
|
||||||
|
quoting a command through `podman exec sh -c`, because PMS's HTTPD closes without
|
||||||
|
a single byte on a request that arrives from another netns — the host address and
|
||||||
|
the `10.89.x.x` container address both read as "empty reply", never as a 401 —
|
||||||
|
and because a three-layer `ssh → podman exec → sh -c` quote chain dies on a parse
|
||||||
|
error long before it reaches the server. The request therefore rides plain
|
||||||
|
`http://127.0.0.1:32400` from inside the container, where insecure connections
|
||||||
|
are allowed (gotcha 5). The credential is `PlexOnlineToken` in the container's
|
||||||
|
`Preferences.xml` — NOT `PlexHTTPToken`, which a claimed server need not carry —
|
||||||
|
read inside the container into a mode-600 curl config there, so it never enters
|
||||||
|
argv (`/proc/<pid>/cmdline` is world-readable) and never a URL (proxies log
|
||||||
|
URLs). Hand-writing that config has one more trap: inside a quoted `printf`
|
||||||
|
format a backslash-newline is LITERAL, so a two-line string becomes one glued
|
||||||
|
line plus a stray backslash, and curl answers `option -K: is unknown` — never
|
||||||
|
"bad config line".
|
||||||
|
|
||||||
Proof that a rename landed is not a scan response: `scripts/media-verify.py`
|
Proof that a rename landed is not a scan response: `scripts/media-verify.py`
|
||||||
replays the run's journal against the database — the only read that
|
replays the run's journal against the database — the only read that
|
||||||
|
|||||||
Reference in new issue
Block a user