Merge pull request 'feat(media): tvdb names the show — titled episode files, id-backed series folders, park outside' (#102) from feat/tv-tvdb-naming into main

serverAppConfigs-docs/Dankserver/Jellyfin.md

@@ -182,6 +182,48 @@ scripts/jellyfin-providers.sh verify    # plugin + key LENGTHS, never key bytes
 2. Set metadata/language preferences (`LANG=en_US.UTF-8` pinned in the unit).
 3. Configure clients to use `http://192.168.0.235:8096` or the guarded vhost.
 
+## Triggering a scan
+
+The scan key lives in `~/.config/secrets/jellyfin` (mode 600; the same value is
+`JELLYFIN_API_KEY` in the ignored `dankserver/.env`). It rides a mode-600 curl
+config, never a header argument, in the `MediaBrowser` form: 12.4 authenticates
+only `Authorization: MediaBrowser Token="…"` and refuses a bare `X-Emby-Token`
+for every key, valid or not. Before believing a 401, prove the key against the
+server's own table — `select count(*) from ApiKeys where AccessToken = ?` on
+`jellyfin.db` opened `mode=ro&immutable=1`. The backslashes below matter:
+
+```bash
+umask 077
+{ printf 'header = "Authorization: MediaBrowser Token=\\"%s\\""\n' \
+    "$(cat ~/.config/secrets/jellyfin)"
+  echo 'url = "http://127.0.0.1:8096/Library/Refresh"'
+  echo 'request = "POST"'; } >/tmp/jf.hdr
+curl -s -K /tmp/jf.hdr -o /dev/null -w "jellyfin scan: %{http_code}\n"
+shred -u /tmp/jf.hdr
+```
+
+`204` is acceptance; anything else queued no scan, so re-issue the identical
+request instead of diagnosing. A `401` from a key that answers `200` on a
+hand-typed request is this block's quoting, not the server: bash `printf`
+collapses `\"` to `"`, curl ends the quoted value at that quote, and Jellyfin
+receives `Authorization: MediaBrowser Token=` with nothing after it (`\\"` answers
+204, `\"` answers 401 — both run on this host).
+
+One accepted refresh is the whole procedure. A second one while a scan runs moves
+`Scan Media Library` to `Cancelling`, and once the runner holds `Cancelling` every
+later `204` is accepted and then discarded — the status code stops meaning
+anything. The fix is `systemctl --user restart jellyfin.service` (never
+`deploy.sh`: that runs the never-tested metadata-view cutover), one POST after
+health answers, and then believe `ScheduledTasks` State at 130 % CPU, not the
+response code.
+
+**A scan that must see a new folder has to be recursive.** `POST /Library/Refresh`
+is the incremental pass, and inotify does not cross a rootless podman bind, so
+folders created on the host stay invisible to it. The move is
+`POST /Items/{id}/Refresh?Recursive=true` (204) per library, the id from
+`GET /Library/VirtualFolders` — its field is `ItemId`, not `Id` — and that same
+record's `RefreshStatus` is the progress read.
+
 ## See Also
 
 - [[Plex]] — co-resident media server on the same volume

serverAppConfigs-docs/Dankserver/Media Relayout.md

@@ -81,41 +81,12 @@ An apply also leaves the release folders it emptied standing where they were —
 hundreds of them at the library root — which is what `sweep` removes, dry-run
 first, from the same journal.
 
-After it lands, both servers need a scan or the change is invisible.
-
-```bash
-# Run this from the host shell through `podman exec`, never from a neighbour
-# container: Plex's HTTPD closes without a single byte on any request that
-# arrives from another container's netns (10.89.x.x reads as "empty reply", not
-# as a 401). The token rides stdin, never argv, where any process on the host
-# could read it out of /proc/<pid>/cmdline.
-umask 077
-podman exec plex sh -c "sed -n 's/.*PlexOnlineToken=\"\\([^\"]*\\)\".*/\\1/p' \
-  '/config/Library/Application Support/Plex Media Server/Preferences.xml'" >/tmp/px.tok
-podman exec -i plex sh -c 'read -r T
-  for k in $(curl -s -H "X-Plex-Token: $T" -H "Accept: application/json" \
-      http://127.0.0.1:32400/library/sections | grep -o "\"key\":\"[0-9]*\"" | cut -d\" -f4); do
-    curl -s -o /dev/null -w "plex section $k: %{http_code}\n" -X POST \
-      -H "X-Plex-Token: $T" "http://127.0.0.1:32400/library/sections/$k/refresh"
-  done' </tmp/px.tok
-```
-
-[[Jellyfin]]'s scan key lives in `~/.config/secrets/jellyfin` (mode 600; the
-same value is `JELLYFIN_API_KEY` in the ignored `dankserver/.env`). It rides a
-mode-600 curl config, never a header argument, in the `MediaBrowser` form —
-Jellyfin 12.4 refuses a bare `X-Emby-Token`, and the backslashes below matter:
-
-```bash
-umask 077
-{ printf 'header = "Authorization: MediaBrowser Token=\\"%s\\""\n' \
-    "$(cat ~/.config/secrets/jellyfin)"
-  echo 'url = "http://127.0.0.1:8096/Library/Refresh"'
-  echo 'request = "POST"'; } >/tmp/jf.hdr
-curl -s -K /tmp/jf.hdr -o /dev/null -w "jellyfin scan: %{http_code}\n"
-shred -u /tmp/jf.hdr
-```
-
-`204` is acceptance; anything else queued no scan. A `401` from a key that answers `200` on a hand-typed request is this block's quoting, not the server — bash `printf` collapses `\"` to `"`, curl ends the quoted value at that quote, and Jellyfin receives `Authorization: MediaBrowser Token=` with nothing after it (`\\"` answers 204, `\"` answers 401, both run on this host). One accepted refresh is the whole procedure: a second one while a scan runs moves `Scan Media Library` to `Cancelling`, and once the runner holds `Cancelling` every later `204` is accepted and then dropped — the fix is `systemctl --user restart jellyfin.service` (never `deploy.sh`: that runs the metadata-view cutover), one POST after health answers, and then believe `ScheduledTasks` State `Running`, not the status code.
+After it lands, both servers need a scan or the change is invisible. [[Plex]] and
+[[Jellyfin]] own the exact requests; two laws hold at this step: `204` is
+acceptance and anything else queued nothing — re-issue the identical request
+rather than diagnosing — and ONE accepted refresh is the whole procedure, because
+a second while a scan runs moves `Scan Media Library` to `Cancelling` and every
+later `204` is accepted and then discarded.
 
 ### Proving the move landed
 
@@ -208,12 +179,16 @@ show. A name whose marker is `Part 2` is a film and stays put.
 
 `/mnt/entropy/TVShows` is scene packs rather than a library, and Jellyfin has
 already matched and numbered most of it. `scripts/media-shows.py` re-files that
-library into the same shape, and it never guesses: every series name and every
-episode number it writes traces to an authority, and what has none is parked for
-a human to read.
+library into the shape Jellyfin reads, with **tvdb as the name authority**:
+
+    <Series> (<year>)/Season NN/<Series> (<year>) - SxxEyy - <Episode Title>.ext
+
+It never guesses: every series name, episode number and title it writes traces to
+an authority, and what has none is parked for a human to read.
 
 ```bash
-# 1. The authority: Jellyfin's own matched rows (the DB opens read-only).
+# 1. The authority: Jellyfin's own matched rows — each carries the series' TMDb
+#    and Tvdb ids and the title the row holds (the DB opens read-only).
 python3 scripts/media-census.py jellyfin --root /mnt/entropy/TVShows \
     --out /tmp/tv_titles.tsv
 # 2. What is on disk (stat only), and the runtimes the film arm needs.
@@ -221,16 +196,33 @@ python3 scripts/media-census.py enumerate --root /mnt/entropy/TVShows \
     --out /tmp/tv_enum.tsv
 python3 scripts/media-census.py probe --root /mnt/entropy/TVShows \
     --enum /tmp/tv_enum.tsv --jobs 6 --out /tmp/tv_probed.tsv
-#    The probe is the slow step: ffprobe reads both ends of every file.
-# 3. The plan. --tmdb needs TMDB_API_KEY; without it a series no provider dated
-#    is simply named without a year, which Jellyfin accepts.
+#    The probe is the slow step: ffprobe reads both ends of every file. So re-use
+#    the last one — a duration belongs to the file, not to its name: re-key the
+#    previous census through that pass's own journal (`src` -> `dst`) instead of
+#    seeking the array again. It is also the only column the film arm reads, and
+#    only for a file with no marker and no provider row (4,794 of 58,551 today,
+#    all of them covered by a re-key that costs seconds, not an hour).
+# 3. The plan. --tmdb needs TMDB_API_KEY — the v3 key in ~/.config/secrets/tmdb,
+#    which travels as the `api_key` query parameter. Only a v4 request token is
+#    accepted as a Bearer header, and the wrong shape answers 401 on EVERY call,
+#    which the summary line flags as NOT provider-complete. --tvdb reads its
+#    `api_key=` line from ~/.config/secrets/tvdb, never argv. Both write the SAME
+#    cache file, so the plan is reproducible offline afterwards: run it once to
+#    fill the cache, and quote the second run.
+#    Paths are only strings to a dry run, so this step can run on the workstation
+#    against scp'd TSVs — which is where the two provider secrets live.
 python3 scripts/media-shows.py --root /mnt/entropy/TVShows --enum /tmp/tv_enum.tsv \
-    --titles /tmp/tv_titles.tsv --probed /tmp/tv_probed.tsv --out /mnt/entropy/media-relayout-shows-plan.tsv \
-    --exceptions /mnt/entropy/media-relayout-shows-exceptions.txt
+    --titles /tmp/tv_titles.tsv --probed /tmp/tv_probed.tsv \
+    --tmdb-cache /mnt/entropy/media-tv-names.json --tmdb --tvdb \
+    --park /mnt/entropy/media-park/TVShows \
+    --out /mnt/entropy/media-relayout-shows2-plan.tsv \
+    --exceptions /mnt/entropy/media-relayout-shows2-exceptions.txt
 # 4. Read the exceptions file FIRST — it is the review, not a log. Same executor
-#    as above, but on this arm's OWN journal: a shared journal cannot be unwound.
-scripts/media-apply.py apply --plan /mnt/entropy/media-relayout-shows-plan.tsv \
-    --root /mnt/entropy --journal /mnt/entropy/media-relayout-shows.jsonl --apply
+#    as above, but on this arm's OWN journal, NAMED PER PASS: the journal of an
+#    applied pass is its unwind record, so a second pass that reuses the name
+#    destroys the only map back.
+scripts/media-apply.py apply --plan /mnt/entropy/media-relayout-shows2-plan.tsv \
+    --root /mnt/entropy --journal /mnt/entropy/media-relayout-shows2.jsonl --apply
 ```
 
 Evidence, strongest first — the order the planner believes things:
@@ -238,12 +230,19 @@ Evidence, strongest first — the order the planner believes things:
 - the **episode marker the file's own name states** (`S01E02-E03`, `1x04`);
 - the **`Season NN` folder it sits in and the number its name leads with inside
   it**: `Season 19/01 Charles Barkley, Nirvana.avi` is S19E01;
-- the **provider row**: identity (the series name and its year), and the
-  numbering of a file whose own name says nothing;
+- the **provider row**: which episode the file IS, and the numbering of a file
+  whose own name says nothing;
+- the **row's tvdb id, else its TMDb id carried through TMDb's external ids, else
+  an exact tvdb name search**: that id names the SERIES FOLDER and supplies the
+  episode title, because a name the server read off a folder is not a show name;
 - a **bonus folder** anywhere above the file, or Jellyfin's `ExtraType` → the
   series' canonical extra folder;
-- a runtime over `SHORT_MAX` makes it film-shaped, and it leaves for `--movies`
-  only when a movie provider names it — otherwise it parks.
+- a runtime over `SHORT_MAX` makes it film-shaped, and it leaves for `--movies` only
+  when the movie provider's title is a run of words the FILE itself states and its
+  year agrees with the file's: the query is only what `title_of` salvaged from a
+  release name, and the first hit for a short one is a near miss — a 1963 Bond film
+  for the word "russia", *Special Edition I* for a file saying *Edition III*. Else it
+  parks.
 
 Two refusals carry the design, and a dry run on the real library found both:
 
@@ -258,11 +257,25 @@ Two refusals carry the design, and a dry run on the real library found both:
   the file's own leading number, win; a row that still insists on season 1989
   parks its file instead of creating `Season 1989`.
 
-The provider's name is copied verbatim, warts included: Jellyfin carries a Series
-named `Black Clover 001` (92 episodes, no year) because it matched a pack off a
-filename. Renaming a show is Jellyfin's work — a folder rename never touches its
-provider ids — so the mover keeps the name it is handed and strips only the pack
-index a release leads with (`4a. Seed DESTINY` → `Seed DESTINY`).
+**A provider name is evidence only when an id backs it.** Jellyfin names a Series
+after the folder it scanned, so a library filed by release folders ends up with
+one Series item per box set — `24 (2001) Season 1`, `Modern Family S05` — and
+copying those names verbatim launders folder junk into the contract. The planner
+resolves each series to its tvdb id and writes tvdb's name and year; the merged
+identity is also what turns a box set split across five release folders into ONE
+folder, and what makes two files claiming one episode MEET in the duplicate rule.
+A series no id resolves keeps the provider's words and is listed under *no tvdb
+identity* in the exceptions — that list, not a rename, is the review.
+
+**The review pile lives outside the library.** Files parked inside
+`/mnt/entropy/TVShows` were indexed as one show named `_unsorted`, and that
+show's rows then looked like evidence. So a row whose series is the hidden
+top-level folder the file itself sits in is dropped — and the folder is matched
+with its leading underscore stripped, because the server *displays* `_unsorted` as
+`unsorted`: compare the spelled names and the pile survives the relayout as a
+visible series folder that real files get renamed into. `--park` outside the root
+makes the next plan re-file the pile instead of skipping it. Inside a hidden
+top-level folder a file's own words may MATCH a registered show, never invent one.
 
 Release-group folder names are the one thing no provider can read, so the
 pending TVShows naming plan files 179 top-level directories (9,931 files) under
@@ -322,19 +335,6 @@ executor, the one-journal-per-arm law, the sweep and the rollback.
   free space, and `rename(2)` refuses it anyway. Trash goes beside the item —
   `/mnt/entropy/.AGENTTRASH/<date>-<topic>/` — so it costs one inode change and
   rolls back exactly like the move that made it.
-- **A Jellyfin 401 is the header, not the key.** 12.4 authenticates only
-  `Authorization: MediaBrowser Token="…"` and refuses a bare `X-Emby-Token` for
-  every key, valid or not. This page carried "no credential on this box can
-  trigger a scan, the two Seerr keys answer 401" for weeks because of that
-  header — the keys were innocent. Before believing a 401, prove the key against
-  the server's own table: `select count(*) from ApiKeys where AccessToken = ?`
-  on `jellyfin.db` opened `mode=ro&immutable=1`.
-- **A scan that must see a new folder has to be recursive.** `POST
-  /Library/Refresh` is the incremental pass, and inotify does not cross a
-  rootless podman bind, so folders created on the host stay invisible to it. The
-  move is `POST /Items/{id}/Refresh?Recursive=true` (204) per library, the id
-  from `GET /Library/VirtualFolders` — its field is `ItemId`, not `Id` — and
-  that same record's `RefreshStatus` is the progress read.
 - **A file whose name carries no words is bonus material, not a folder.**
   `.Extra 1 ( 1H 27min multisub ).mkv` parses to `1`; a movie folder named `1`
   is not a movie. It becomes a review line telling a human to name the file.
@@ -363,7 +363,7 @@ executor, the one-journal-per-arm law, the sweep and the rollback.
   recreates the folder, so the sweep is undone by the same command as the moves.
   Proof carries downward: inside a folder the journal proves emptied, every
   empty directory goes with it. A release's `Screens/` or `Sample/` is never a
-  move source itself, so it can hold an emptied parent on disk after a sweep.
+  move source itself, so it can hold an emptied parent until the next sweep.
   A failed `rmdir` is not a removal: only a folder that is really gone leaves
   scope, so the next sweep finishes what a permission problem stopped.
 - **An episode-shaped name marks a SERIES, not a film — and the series is the

serverAppConfigs-docs/Dankserver/Plex.md

@@ -87,3 +87,45 @@ root runs as the `dank` user, and the deployer adopts the data root with
    while the HTTP layer finishes loading; the first boot of a years-old
    library took ~10 minutes (2026-09 cutover). The deployer gate now allows
    600 s before failing; later boots are much faster (warm page cache).
+
+## Triggering a scan
+
+A rename on disk is invisible until the section is rescanned, and a moved file is
+a `Couldn't find the file to stream` 404 until that scan passes it. Run this from
+the host shell through `podman exec`, never from a neighbour container: PMS's
+HTTPD closes without a single byte on any request that arrives from another
+container's netns (10.89.x.x reads as "empty reply", not as a 401). The token
+rides stdin, never argv, where any process on the host could read it out of
+`/proc/<pid>/cmdline`; it lives in `Preferences.xml` inside the RW data bind.
+
+```bash
+umask 077
+podman exec plex sh -c "sed -n 's/.*PlexOnlineToken=\"\\([^\"]*\\)\".*/\\1/p' \
+  '/config/Library/Application Support/Plex Media Server/Preferences.xml'" >/tmp/px.tok
+cat >/tmp/pxscan.sh <<'EOS'
+read -r T
+cfg=$(mktemp)
+mk() { { printf 'header = "X-Plex-Token: %s"\nheader = "Accept: application/json"\n' "$T"
+        printf 'url = "%s"\n' "$1"; } > "$cfg"; }
+mk "http://127.0.0.1:32400/library/sections"
+for k in $(curl -s -K "$cfg" | grep -o '"key":"[0-9]*"' | cut -d\" -f4); do
+  mk "http://127.0.0.1:32400/library/sections/$k/refresh"
+  printf 'request = "POST"\n' >> "$cfg"
+  echo "plex section $k: $(curl -s -o /dev/null -w '%{http_code}' -K "$cfg")"
+done
+shred -u "$cfg"
+EOS
+podman cp /tmp/pxscan.sh plex:/tmp/pxscan.sh
+podman exec -i plex sh /tmp/pxscan.sh </tmp/px.tok
+shred -u /tmp/px.tok
+```
+
+Every section answers `204` — or `200` with a Directory body when
+`Accept: application/json` is sent. Inside the quoted `printf` format a
+backslash-newline is LITERAL, not a continuation, so `mk` writes two `printf`s:
+one glued line and one stray backslash is a config file curl rejects, and the
+failure prints as `option -K: is unknown`, never as a bad config line.
+
+Proof that a rename landed is not a scan response: `scripts/media-verify.py`
+replays the run's journal against the database — the only read that
+distinguishes "not scanned yet" from "gone"; see [[Media Relayout]].