Media refresh drivers: the two scan doors become commands #140
No files matched your search
@@ -309,33 +309,36 @@ scripts/jellyfin-providers.sh verify # plugin + key LENGTHS, never key bytes
|
|||||||
|
|
||||||
## Triggering a scan
|
## Triggering a scan
|
||||||
|
|
||||||
The scan key lives in `~/.config/secrets/jellyfin` (mode 600; the same value is
|
Run the driver from the deploy checkout: `scripts/media-refresh-jellyfin.py
|
||||||
`JELLYFIN_API_KEY` in the ignored `dankserver/.env`). It rides a mode-600 curl
|
trigger`, then `… wait`. It calls the library settled only on the rule below — the
|
||||||
config, never a header argument, in the `MediaBrowser` form: 12.4 authenticates
|
task seen `Running` (or a run that finished while it watched) first, then
|
||||||
only `Authorization: MediaBrowser Token="…"` and refuses a bare `X-Emby-Token`
|
`--require-quiet` reads with no scan and no writes — and refuses by name when the
|
||||||
for every key, valid or not. Before believing a 401, prove the key against the
|
scan never started, because a watcher that saw no `Running` would otherwise report
|
||||||
server's own table — `select count(*) from ApiKeys where AccessToken = ?` on
|
the pre-scan library as the fresh one. `… status --after $(stat -c %Y <journal>)`
|
||||||
`jellyfin.db` opened `mode=ro&immutable=1`. The backslashes below matter:
|
is the "has the scan reached my moves" answer, read from the task's `EndTimeUtc`.
|
||||||
|
|
||||||
```bash
|
It carries every law below. Hand-written, the door is a `curl -K` on a mode-600
|
||||||
umask 077
|
config: the scan key lives in `~/.config/secrets/jellyfin` (mode 600; the same
|
||||||
{ printf 'header = "Authorization: MediaBrowser Token=\\"%s\\""\n' \
|
value is `JELLYFIN_API_KEY` in the ignored `dankserver/.env`) and rides as
|
||||||
"$(cat ~/.config/secrets/jellyfin)"
|
`Authorization: MediaBrowser Token="…"` — 12.4 refuses a bare `X-Emby-Token` for
|
||||||
echo 'url = "http://127.0.0.1:8096/Library/Refresh"'
|
every key, valid or not — never as a header argument, because
|
||||||
echo 'request = "POST"'; } >/tmp/jf.hdr
|
`/proc/<pid>/cmdline` is world-readable. `recentMedia` counts
|
||||||
curl -s -K /tmp/jf.hdr -o /dev/null -w "jellyfin scan: %{http_code}\n"
|
`BaseItems.DateModified` rows in `jellyfin.db`, which on this host sits in the
|
||||||
shred -u /tmp/jf.hdr
|
quadlet's config bind at `/home/dank/Containers/jellyfin/config/data/`, not the
|
||||||
```
|
stock `~/.local/share/jellyfin`.
|
||||||
|
|
||||||
`204` is acceptance; anything else queued no scan, so re-issue the identical
|
`204` is acceptance; anything else queued no scan, so re-issue the identical
|
||||||
request instead of diagnosing. A `401` from a key that answers `200` on a
|
request instead of diagnosing. A `401` from a key that answers `200` on a
|
||||||
hand-typed request is this block's quoting, not the server: bash `printf`
|
hand-typed request is this block's quoting, not the server: bash `printf`
|
||||||
collapses `\"` to `"`, curl ends the quoted value at that quote, and Jellyfin
|
collapses `\"` to `"`, curl ends the quoted value at that quote, and Jellyfin
|
||||||
receives `Authorization: MediaBrowser Token=` with nothing after it (`\\"` answers
|
receives `Authorization: MediaBrowser Token=` with nothing after it (`\\"` answers
|
||||||
204, `\"` answers 401 — both run on this host).
|
204, `\"` answers 401 — both run on this host). Before believing a 401, prove the
|
||||||
Reading the progress with the same config trick takes **one** URL: a config `url`
|
key against the server's own table — `select count(*) from ApiKeys where
|
||||||
line *plus* a command-line URL makes curl perform both transfers, and the second
|
AccessToken = ?` on `jellyfin.db` opened plain `mode=ro` (`immutable=1` only as
|
||||||
dumps the whole `ScheduledTasks` body onto your stdout.
|
the fallback for a database whose `-shm` cannot be read; it skips the write-ahead
|
||||||
|
log, see [[Media Relayout]]). Reading progress with the same config trick takes
|
||||||
|
**one** URL: a config `url` line *plus* a command-line URL makes curl perform both
|
||||||
|
transfers, and the second dumps the whole `ScheduledTasks` body onto your stdout.
|
||||||
|
|
||||||
One accepted refresh is the whole procedure. A second one while a scan runs moves
|
One accepted refresh is the whole procedure. A second one while a scan runs moves
|
||||||
`Scan Media Library` to `Cancelling`, and once the runner holds `Cancelling` every
|
`Scan Media Library` to `Cancelling`, and once the runner holds `Cancelling` every
|
||||||
|
|||||||
@@ -151,6 +151,16 @@ Each of these is a file that would have been destroyed without it:
|
|||||||
- Media containers only. Grouping every extension buries the report in subtitle
|
- Media containers only. Grouping every extension buries the report in subtitle
|
||||||
and cover pairs — and a duration guard that appears to fire on 720 groups is
|
and cover pairs — and a duration guard that appears to fire on 720 groups is
|
||||||
counting subtitles, not protecting films.
|
counting subtitles, not protecting films.
|
||||||
|
- **A short file named after a film is promotion, not a movie.** Thirty-two files
|
||||||
|
in `Movies` carried real titles (`The Descent (2005) - SD.mkv`) and were
|
||||||
|
8–20-second h264 clips — 640×360, 1024×576 or 1280×720, 49 MiB all together,
|
||||||
|
25 of them in a `_release-group watermarks/` folder and the rest at the library
|
||||||
|
root or inside one film's folder. No duplicate screen ever catches them: nothing
|
||||||
|
shares their size, and the feature-length guard above makes them invisible to the
|
||||||
|
similar-title pass too. Held as `.AGENTTRASH/20261104-promo-clips`, journalled at
|
||||||
|
`/mnt/entropy/.media-campaign/promo-hold.jsonl`. The census that finds the next
|
||||||
|
batch is a duration floor, not a hash: a video under a minute on a name that
|
||||||
|
reads like a feature is advertising.
|
||||||
|
|
||||||
## The hold
|
## The hold
|
||||||
|
|
||||||
|
|||||||
@@ -99,12 +99,17 @@ interrupted apply still replays the batch and resumes with nothing lost. Batch 1
|
|||||||
is the old behaviour; the only reason to name it is debugging, because on the
|
is the old behaviour; the only reason to name it is debugging, because on the
|
||||||
spinning array it spends most of a large apply's wall time in fsyncs.
|
spinning array it spends most of a large apply's wall time in fsyncs.
|
||||||
|
|
||||||
After it lands, both servers need a scan or the change is invisible. [[Plex]] and
|
After it lands, both servers need a scan or the change is invisible. The two
|
||||||
[[Jellyfin]] own the exact requests; two laws hold at this step: `204` is
|
drivers are the doors — `scripts/media-refresh-jellyfin.py trigger` then `… wait`,
|
||||||
acceptance and anything else queued nothing — re-issue the identical request
|
and only after it settles, `scripts/media-refresh-plex.sh trigger --library <key>`
|
||||||
rather than diagnosing — and ONE accepted refresh is the whole procedure, because
|
with `… status --library <key> --after $(stat -c %Y <journal>)` as the poll. Run
|
||||||
a second while a scan runs moves `Scan Media Library` to `Cancelling` and every
|
them in that order and never together: Jellyfin's only door walks every library,
|
||||||
later `204` is accepted and then discarded.
|
every Plex section forks its own scanner walk, and the array is the bottleneck
|
||||||
|
both servers share. [[Plex]] and [[Jellyfin]] own the laws behind the drivers; two
|
||||||
|
of them decide this step: `204` is acceptance and anything else queued nothing —
|
||||||
|
re-issue the identical request rather than diagnosing — and ONE accepted refresh is
|
||||||
|
the whole procedure, because a second while a scan runs moves `Scan Media Library`
|
||||||
|
to `Cancelling` and every later `204` is accepted and then discarded.
|
||||||
|
|
||||||
### Proving the move landed
|
### Proving the move landed
|
||||||
|
|
||||||
|
|||||||
@@ -106,43 +106,39 @@ root runs as the `dank` user, and the deployer adopts the data root with
|
|||||||
## Triggering a scan
|
## Triggering a scan
|
||||||
|
|
||||||
A rename on disk is invisible until the section is rescanned, and a moved file is
|
A rename on disk is invisible until the section is rescanned, and a moved file is
|
||||||
a `Couldn't find the file to stream` 404 until that scan passes it. Run this from
|
a `Couldn't find the file to stream` 404 until that scan passes it. Run the
|
||||||
the host shell through `podman exec`, never from a neighbour container: PMS's
|
driver from the host shell; the section keys are `1` Movies, `3` TV Shows,
|
||||||
HTTPD closes without a single byte on any request that arrives from another
|
`5` Audiobooks, `6` Music, `11` VODs & Live Events:
|
||||||
container's netns (10.89.x.x reads as "empty reply", not as a 401). The token
|
|
||||||
rides stdin, never argv, where any process on the host could read it out of
|
|
||||||
`/proc/<pid>/cmdline`; it lives in `Preferences.xml` inside the RW data bind.
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
umask 077
|
scripts/media-refresh-plex.sh trigger --library 1
|
||||||
podman exec plex sh -c "sed -n 's/.*PlexOnlineToken=\"\\([^\"]*\\)\".*/\\1/p' \
|
scripts/media-refresh-plex.sh status --library 1 --after "$(stat -c %Y <journal>)"
|
||||||
'/config/Library/Application Support/Plex Media Server/Preferences.xml'" >/tmp/px.tok
|
|
||||||
cat >/tmp/pxscan.sh <<'EOS'
|
|
||||||
read -r T
|
|
||||||
cfg=$(mktemp)
|
|
||||||
mk() { { printf 'header = "X-Plex-Token: %s"\nheader = "Accept: application/json"\n' "$T"
|
|
||||||
printf 'url = "%s"\n' "$1"; } > "$cfg"; }
|
|
||||||
mk "http://127.0.0.1:32400/library/sections"
|
|
||||||
for k in $(curl -s -K "$cfg" | grep -o '"key":"[0-9]*"' | cut -d\" -f4); do
|
|
||||||
mk "http://127.0.0.1:32400/library/sections/$k/refresh"
|
|
||||||
printf 'request = "POST"\n' >> "$cfg"
|
|
||||||
echo "plex section $k: $(curl -s -o /dev/null -w '%{http_code}' -K "$cfg")"
|
|
||||||
done
|
|
||||||
shred -u "$cfg"
|
|
||||||
EOS
|
|
||||||
podman cp /tmp/pxscan.sh plex:/tmp/pxscan.sh
|
|
||||||
podman exec -i plex sh /tmp/pxscan.sh </tmp/px.tok
|
|
||||||
shred -u /tmp/px.tok
|
|
||||||
```
|
```
|
||||||
|
|
||||||
Every section answers `204` — or `200` with a Directory body when
|
Exactly ONE section: a refresh costs a full walk of that tree, and refreshing all
|
||||||
`Accept: application/json` is sent. A refresh fired while PMS is still warming
|
five for a rename that touched one buys nothing. `trigger` answers
|
||||||
answers `000` instead (the empty reply of gotcha 4, arriving mid-request): retry
|
`refresh accepted (204)` — `200` when a body is requested — and `000` means PMS is
|
||||||
until it answers and never restart the service over it. Inside the quoted
|
still warming, which the driver retries out loud; a service restart is never the
|
||||||
`printf` format a backslash-newline is LITERAL, not a continuation, so `mk`
|
fix. Progress needs no token and no container: `section_locations.scanned_at` in
|
||||||
writes two `printf`s: one glued line and one stray backslash is a config file
|
the library database carries the same epoch `GET /library/sections` prints as
|
||||||
curl rejects, and the failure prints as `option -K: is unknown`, never as a bad
|
`scannedAt` (verified 2026-11-04), and that is what `status` compares `--after`
|
||||||
config line.
|
against.
|
||||||
|
|
||||||
|
The driver ships its runner into the container with `podman cp` rather than
|
||||||
|
quoting a command through `podman exec sh -c`, because PMS's HTTPD closes without
|
||||||
|
a single byte on a request that arrives from another netns — the host address and
|
||||||
|
the `10.89.x.x` container address both read as "empty reply", never as a 401 —
|
||||||
|
and because a three-layer `ssh → podman exec → sh -c` quote chain dies on a parse
|
||||||
|
error long before it reaches the server. The request therefore rides plain
|
||||||
|
`http://127.0.0.1:32400` from inside the container, where insecure connections
|
||||||
|
are allowed (gotcha 5). The credential is `PlexOnlineToken` in the container's
|
||||||
|
`Preferences.xml` — NOT `PlexHTTPToken`, which a claimed server need not carry —
|
||||||
|
read inside the container into a mode-600 curl config there, so it never enters
|
||||||
|
argv (`/proc/<pid>/cmdline` is world-readable) and never a URL (proxies log
|
||||||
|
URLs). Hand-writing that config has one more trap: inside a quoted `printf`
|
||||||
|
format a backslash-newline is LITERAL, so a two-line string becomes one glued
|
||||||
|
line plus a stray backslash, and curl answers `option -K: is unknown` — never
|
||||||
|
"bad config line".
|
||||||
|
|
||||||
Proof that a rename landed is not a scan response: `scripts/media-verify.py`
|
Proof that a rename landed is not a scan response: `scripts/media-verify.py`
|
||||||
replays the run's journal against the database — the only read that
|
replays the run's journal against the database — the only read that
|
||||||
|
|||||||
Reference in new issue
Block a user