From c21c48e61eef35807484e7b4eec26586eb860f1a Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 02:56:03 -0800 Subject: [PATCH 01/13] docs(jellyfin): the door that scans is /Library/Refresh, and the art door names its field MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The page sent the next agent to POST /Items/{id}/Refresh?Recursive=true per library to make a new folder visible. Measured against a Shows library that had never been identified: that route answers 204 and queues nothing — item census byte-identical, ScheduledTasks idle, journal silent, and still a silent 204 after a service restart. The door that works is POST /Library/Refresh, and the progress read is ScheduledTasks -> Scan Media Library -> CurrentProgressPercentage. RefreshStatus on the VirtualFolders record stays Idle through a real scan, so it is not a progress read at all; a 204 cannot distinguish accepted-and-discarded from accepted-and-queued, so the acceptance read is the journal naming folders. Also recorded the art door's wire shape, because guessing it reads as a server bug: the list response carries the address in Url (there is no ImageUrl), the Download POST takes camelCase type/imageUrl and answers 204, and PascalCase or an empty imageUrl answers 500. RemoteImages is empty for items with no ProviderIds, so art coverage is capped by identification, not by the image door. --- serverAppConfigs-docs/Dankserver/Jellyfin.md | 35 ++++++++++++++++---- 1 file changed, 29 insertions(+), 6 deletions(-) diff --git a/serverAppConfigs-docs/Dankserver/Jellyfin.md b/serverAppConfigs-docs/Dankserver/Jellyfin.md index 9d11766..2c4298c 100644 --- a/serverAppConfigs-docs/Dankserver/Jellyfin.md +++ b/serverAppConfigs-docs/Dankserver/Jellyfin.md @@ -119,6 +119,17 @@ store on a build where the automatic image pass will not run at all — it is ho the Movies library got its art: 2,515 of 2,564 films carry a primary now, and the rest have none in TMDb (shorts like `Astartes`, files whose names match nothing). +**The Download door's exact wire shape:** the list response carries the address in +`Url` — there is no `ImageUrl` field — and the POST takes camelCase query params, +`?type=Primary&imageUrl=`, answering `204`; the stored tag then appears in +`ImageTags`. PascalCase params, or an `imageUrl` left empty, answer `500`, which +reads as a server fault and is a field-name fault. + +**`RemoteImages` answers empty for an item with no `ProviderIds`** — no provider id, +nothing to look up. Art coverage is capped by identification, not by the image door: +the items a campaign returns nothing for are the unidentified ones, and no image +retry will change that. + **API shape on 12.x:** `/Items` and `/Items/{id}` need `UserId` when authenticated by API key — without it the answer is `400 Error processing request`, which reads exactly like an authentication failure. `MetadataRefreshMode` @@ -284,12 +295,24 @@ anything. The fix is `systemctl --user restart jellyfin.service`, one POST after health answers, and then believe `ScheduledTasks` State at 130 % CPU, not the response code. -**A scan that must see a new folder has to be recursive.** `POST /Library/Refresh` -is the incremental pass, and inotify does not cross a rootless podman bind, so -folders created on the host stay invisible to it. The move is -`POST /Items/{id}/Refresh?Recursive=true` (204) per library, the id from -`GET /Library/VirtualFolders` — its field is `ItemId`, not `Id` — and that same -record's `RefreshStatus` is the progress read. +**The door that scans is `POST /Library/Refresh` (204), and `ScheduledTasks` is the +progress read.** Measured 2026-11-02 against a `Shows` library that had never been +identified: the per-library `POST /Items/{id}/Refresh?Recursive=true` answers `204` +and queues **nothing** — item census identical afterwards, `Scan Media Library` +idle, journal silent — and it is still an empty `204` after a service restart, so +that status code carries no information at all. `/Library/Refresh` walks every +library; within seconds the journal names folders (`…/TVShows/…`, +`EpisodeMetadataService`) and `GET /ScheduledTasks` reports +`Scan Media Library | Running | 75 %` while the process sits near 200 % CPU. + +**`RefreshStatus` on the `GET /Library/VirtualFolders` record is not a progress +read.** It stayed `Idle` through a scan that was verifiably running at 75 %. The +record's useful field is `ItemId`, and the live library set is still the entries +whose `Locations` are non-empty. + +`inotify` does not cross a rootless podman bind, so folders created on the host +stay invisible until a pass like this runs; every pass that writes into a library +ends by firing one. **`Movies` has no watcher at all.** `EnableRealtimeMonitor` is per library (table in *Data layout*) and it is false for `Movies` and `Collections`. Nothing notices a From 56dfc6d0b90a2c7c6e54d5dac394293ba7da9235 Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 02:57:25 -0800 Subject: [PATCH 02/13] docs(audiobookshelf): the scanner refusal names the file, not the bind The first full scan logged "Invalid data found when processing input" per file and carried on. That line is about the file: the scanner skips it, so the book never appears in the library at all. Census of the 43 refused files: 38 were 0 bytes, 3 held WAV bytes behind a .mp3 name, 2 were unreadable. The check to run first is a byte count, not a server investigation. Full suite: 818 tests, exit 0. --- serverAppConfigs-docs/Dankserver/Audiobookshelf.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md index e079d20..f490b24 100644 --- a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md +++ b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md @@ -124,6 +124,12 @@ the file, not the bind: the same line count grows while the total climbs. cross a rootless podman bind — measured for [[Jellyfin]], whose page owns that law — so the in-container watcher sees nothing a host process files onto the array, and the deployer's re-run of the bootstrapper is the cheap scan trigger. +- **`Invalid data found when processing input` names the file, not the bind.** The + scanner logs it per file and skips the file — the book never appears — so a + missing title beside those lines is library debris, not a server fault. Census of + the refusals on the first full scan: 38 of 43 files were 0 bytes, 3 were WAV + bytes wearing a `.mp3` name, 2 were unreadable. `stat -c %s` before any + investigation of the server. ## Verification From 8889a4ceff5826c952fc0313c61a8fb09550b06a Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 03:00:42 -0800 Subject: [PATCH 03/13] docs(jellyfin): episode stills are out of scope, and the cost that says so A per-episode RemoteImages query is never cached, so the same campaign that dressed 620 series in an afternoon costs ~11 s per episode for one hit in five. Across 42k episodes that is days of provider traffic to dress a fifth of them, and clients show series and season art regardless. Recorded so nobody runs the pass on a hunch; the door and the resumable log are the same if Frank orders it. --- serverAppConfigs-docs/Dankserver/Jellyfin.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/serverAppConfigs-docs/Dankserver/Jellyfin.md b/serverAppConfigs-docs/Dankserver/Jellyfin.md index 2c4298c..6f827ae 100644 --- a/serverAppConfigs-docs/Dankserver/Jellyfin.md +++ b/serverAppConfigs-docs/Dankserver/Jellyfin.md @@ -130,6 +130,13 @@ nothing to look up. Art coverage is capped by identification, not by the image d the items a campaign returns nothing for are the unidentified ones, and no image retry will change that. +**Series and season art is fetched; episode stills are not, deliberately.** Every +episode query is a fresh provider round-trip — nothing caches per episode — measured +at ~11 s per item beside ~2 s per series on the same host, with one hit in five in a +sample. Across 42k episodes that is days of provider traffic to dress a fifth of them, +and clients display the series and season art anyway. If episode stills are ever +ordered, the loop is the same door and its log makes the pass resumable. + **API shape on 12.x:** `/Items` and `/Items/{id}` need `UserId` when authenticated by API key — without it the answer is `400 Error processing request`, which reads exactly like an authentication failure. `MetadataRefreshMode` From 006bbcef2820b7295f7d97f90392e3cfe89b94bb Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 03:18:04 -0800 Subject: [PATCH 04/13] docs(audiobookshelf): paging the items endpoint never terminates itself Two censuses this session reported exactly 1,500 and exactly 4,000 books for one library mid-scan. Cause: an out-of-range offset returns the last item instead of an empty page, so a loop written "until the page is empty" manufactures its own total, and total itself moves while a scan is running. Stop on total; count after the pass settles. --- serverAppConfigs-docs/Dankserver/Audiobookshelf.md | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md index f490b24..153aee5 100644 --- a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md +++ b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md @@ -117,9 +117,13 @@ the file, not the bind: the same line count grows while the total climbs. - **A 200 may carry plain text.** `res.sendStatus(200)` answers the body `OK`, and `/init` and the scan route use it; a client that parses every success as JSON crashes on the one call that matters (`tests/audiobookshelf_init.bats` pins it). -- **The item count is not a field on the library DTO.** The door is - `GET /api/libraries//items?limit=1` → `total`. A poller that guesses a - plausible name (`numItems`) prints `0` forever beside a scan that is working. +- **The item count is not a field on the library DTO, and paging never ends + itself.** The door is `GET /api/libraries//items?limit=1` → `total`; a poller + that guesses a plausible name (`numItems`) prints `0` forever beside a scan that + is working. An out-of-range `offset` returns the last item rather than an empty + page, so a census that loops "until empty" invents its own total — two runs + reported exactly 1,500 and 4,000 for one library — and `total` itself moves while + a scan runs. Stop on `total`, and take the count after the pass settles. - **A fresh scan is the only thing that notices new audio.** `inotify` does not cross a rootless podman bind — measured for [[Jellyfin]], whose page owns that law — so the in-container watcher sees nothing a host process files onto the From f5415667141f6ebca853569c9a4c66eae01d089d Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 03:34:28 -0800 Subject: [PATCH 05/13] docs(audiobookshelf): a book is the shallowest audio folder, and it owns the subtree Measured on the live library: 6 stray files at the root of _unsorted made the whole collection dump ONE item of 5,693 tracks, with 542 real books folded into it; no scan subdivides an existing item. Also pinned: the items endpoint hands back ceil(total/limit)*limit rows because an out-of-range offset repeats the tail (that is where "exactly 1,500" and "exactly 4,000" came from), and DELETE /api/items/:id?hard=1 runs fs.remove() on the item's folder - pointed at a covering item that is the whole subtree, so un-swallowing is a DB-only delete after the stray root files are filed. --- .../Dankserver/Audiobookshelf.md | 23 ++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-) diff --git a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md index 153aee5..5ce6c80 100644 --- a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md +++ b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md @@ -121,13 +121,30 @@ the file, not the bind: the same line count grows while the total climbs. itself.** The door is `GET /api/libraries//items?limit=1` → `total`; a poller that guesses a plausible name (`numItems`) prints `0` forever beside a scan that is working. An out-of-range `offset` returns the last item rather than an empty - page, so a census that loops "until empty" invents its own total — two runs - reported exactly 1,500 and 4,000 for one library — and `total` itself moves while - a scan runs. Stop on `total`, and take the count after the pass settles. + page, so a census that loops "until empty" invents its own total: `limit=500` + against a `total` of 4,628 hands back exactly 5,000 rows — the collected count + is `ceil(total / limit) × limit`, which is why two runs reported "exactly 1,500" + and "exactly 4,000" for one library. Stop on `total`, count *unique ids*, and + take the number after the pass settles (`total` itself moves during a scan). - **A fresh scan is the only thing that notices new audio.** `inotify` does not cross a rootless podman bind — measured for [[Jellyfin]], whose page owns that law — so the in-container watcher sees nothing a host process files onto the array, and the deployer's re-run of the bootstrapper is the cheap scan trigger. +- **A book is the shallowest folder that holds audio — and that item owns the + subtree beneath it.** A folder with audio in its own file list becomes ONE book + and the scanner folds everything below it into that book's tracks instead of + making items: 6 stray files at the root of `_unsorted` turned the whole + collection dump into a single item of 5,693 tracks, titled after whichever file + supplied metadata, with 542 real books inside it. So "hundreds of books are + missing" is a layout fault, not a scan fault — and no scan, forced or not, + subdivides an existing item. Census the layout the server's way: a directory is + a book folder iff its *own* files include an audio extension. +- **`DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()`.** Read + `LibraryItemController.delete`: `hardDelete = req.query.hard == 1` and then + `fs.remove(libraryItemPath)`. Pointed at a covering item, that path is the whole + swallowed subtree — `hard=1` there deletes every audiobook under it off the + array. Un-swallowing is: file the loose root files with the journal-guarded + mover, then delete the stale covering item with NO query string. - **`Invalid data found when processing input` names the file, not the bind.** The scanner logs it per file and skips the file — the book never appears — so a missing title beside those lines is library debris, not a server fault. Census of From f71c7ccce40de6f7f300b4016b49b3c5e9985861 Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 03:41:53 -0800 Subject: [PATCH 06/13] docs(audiobookshelf): the grouping rule, not "shallowest folder with audio" MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Corrects the law pushed earlier today. The mechanism is `groupFileItemsIntoLibraryItemDirs` (utils/scandir.js), called on a live `recurseFiles` walk of the tree EVERY scan: each file attaches to the FIRST directory along its path that already has a group, so one media file at a collection root owns everything beneath it. Measured: one 949 MB `.cbr` hid 652 book folders behind a single 5,693-track item. - "Media file" is audio OR ebook (`audiobooksOnly=false` -> SupportedAudioTypes ∪ SupportedEbookTypes, 17 audio + 6 ebook extensions). An audio-only census cannot see the file that hides an Audiobooks library. - Deleting the covering item does not fix a shadow: grouping is recomputed from disk and the next scan rebuilds the item (measured: `1 Added`, same path). - Un-shadow by MOVE: one file per folder is ABS's own single-book layout. - `?force=1` is not a discovery arm (both arms walk the disk; force re-scans every existing item, 1:33 against 0:04). - Paging corrected: `offset` is ignored outright, `page` is the parameter. - `DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()` of the whole swallowed subtree. Never pass `hard`. Shape table gains the layout row and the move journal; the stale "1,103 shelves" scan count is destroyed. --- .../Dankserver/Audiobookshelf.md | 59 ++++++++++++------- 1 file changed, 38 insertions(+), 21 deletions(-) diff --git a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md index 5ce6c80..837370b 100644 --- a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md +++ b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md @@ -36,6 +36,7 @@ manager) — **Internal-only:** LAN address, no vhost, no DNS name. | Self-healing | `Restart=always`, `RestartSec=5`, `WantedBy=default.target` | | Deploy | `./scripts/deploy.sh dankserver --service audiobookshelf` | | Handler | `deploy_audiobookshelf` in `scripts/deploy-remote.sh` | +| Library layout | one book per folder — ABS groups by the first directory holding a media file (see Laws); every move is journalled at `/mnt/entropy/media-relayout-audiobooks.jsonl` | ## Bootstrap @@ -59,7 +60,7 @@ Dankserver. Change it in **Settings → Users → frank → Password**; the secr then holds a stale value and only matters again if the `config` bind is ever destroyed (a fresh DB makes the script create a new owner from it). -The scan of 1,103 shelves is slow and background; watch `items:` climb from 0, +A full scan is slow and runs in the background; watch the `items:` total climb, never `/healthcheck`. It also reads files that are genuinely broken and logs `[AudioFileScanner] Invalid data found when processing input` per file — that is the file, not the bind: the same line count grows while the total climbs. @@ -120,31 +121,47 @@ the file, not the bind: the same line count grows while the total climbs. - **The item count is not a field on the library DTO, and paging never ends itself.** The door is `GET /api/libraries//items?limit=1` → `total`; a poller that guesses a plausible name (`numItems`) prints `0` forever beside a scan that - is working. An out-of-range `offset` returns the last item rather than an empty - page, so a census that loops "until empty" invents its own total: `limit=500` - against a `total` of 4,628 hands back exactly 5,000 rows — the collected count - is `ceil(total / limit) × limit`, which is why two runs reported "exactly 1,500" - and "exactly 4,000" for one library. Stop on `total`, count *unique ids*, and - take the number after the pass settles (`total` itself moves during a scan). + is working. Paging is on `page` (0-based) plus `limit`, and **`offset` is + ignored**: `limit=500&offset=500` returns the same 500 ids as `offset=0`, while + `page=1` shares none of them. A census that pages on `offset` therefore reads + `ceil(total / limit) × limit` rows containing only `limit` distinct items — the + exact origin of the "exactly 1,500" and "exactly 4,000" counts two runs + reported for one library. Page on `page`, count *unique ids*, stop on `total`, + and take the number after the pass settles (`total` moves during a scan). - **A fresh scan is the only thing that notices new audio.** `inotify` does not cross a rootless podman bind — measured for [[Jellyfin]], whose page owns that law — so the in-container watcher sees nothing a host process files onto the array, and the deployer's re-run of the bootstrapper is the cheap scan trigger. -- **A book is the shallowest folder that holds audio — and that item owns the - subtree beneath it.** A folder with audio in its own file list becomes ONE book - and the scanner folds everything below it into that book's tracks instead of - making items: 6 stray files at the root of `_unsorted` turned the whole - collection dump into a single item of 5,693 tracks, titled after whichever file - supplied metadata, with 542 real books inside it. So "hundreds of books are - missing" is a layout fault, not a scan fault — and no scan, forced or not, - subdivides an existing item. Census the layout the server's way: a directory is - a book folder iff its *own* files include an audio extension. -- **`DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()`.** Read - `LibraryItemController.delete`: `hardDelete = req.query.hard == 1` and then +- **One stray media file at a collection root turns the whole collection into one + book.** Every scan re-walks the tree from disk (`scanFolder` → `recurseFiles`), + and `utils/scandir.js:groupFileItemsIntoLibraryItemDirs` assigns each file to the + FIRST directory along its path that already has a group: the first file seen at a + root owns every deeper file, and the folders below it never become items. One + 949 MB `.cbr` sitting at `_unsorted`'s root collapsed 652 book folders into a + single item of 5,693 tracks, titled after whichever file supplied metadata. + "Hundreds of books are missing" is a layout fault, not a scan fault. +- **"Media file" means audio OR ebook, so an audio-only census cannot see the file + that is hiding the library.** With `audiobooksOnly=false` (the deployed shape) + the predicate is `SupportedAudioTypes` ∪ `SupportedEbookTypes`: 20 audio + extensions (including `mp4`, `mpg`, `webm`) plus `epub pdf mobi azw3 cbr cbz`. A + census reporting "that root holds no loose audio" can be correct and still + useless — the file doing the shadowing here was a comic archive. +- **Deleting the covering item does not un-shadow anything.** Grouping is recomputed + from disk, so the next scan rebuilds the item from the same stray file (measured: + `DELETE`, then scan → `1 Added`, the item back at the same path). The fix is the + MOVE: give every stray root file a folder of its own (one file per folder is + ABS's own single-book layout; formats of one book share a folder), through the + journal-guarded mover, then delete the stale item so the old path is not kept, + then scan. +- **`?force=1` is not a discovery arm.** Both arms of `POST /libraries/:id/scan` + walk the disk; force additionally re-scans every item that already exists + (measured 1:33 against 0:04 on ~4,600 items). Finding new folders needs neither, + so the deployer's cheap trigger stays on the plain arm. +- **`DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()`.** In + `LibraryItemController.delete`, `hardDelete = req.query.hard == 1` and then `fs.remove(libraryItemPath)`. Pointed at a covering item, that path is the whole - swallowed subtree — `hard=1` there deletes every audiobook under it off the - array. Un-swallowing is: file the loose root files with the journal-guarded - mover, then delete the stale covering item with NO query string. + swallowed subtree — `hard=1` there deletes every book beneath it off the array. + Assert the item's `path` first, and never pass `hard`. - **`Invalid data found when processing input` names the file, not the bind.** The scanner logs it per file and skips the file — the book never appears — so a missing title beside those lines is library debris, not a server fault. Census of From ad9563412de56ab671de1dcbfff3badafedb5477 Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 03:59:14 -0800 Subject: [PATCH 07/13] docs(audiobookshelf): what a residual is made of MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Four laws from finishing the un-shadow, correcting one: - A book whose files ALL fail to probe never becomes an item, so a disk-vs-DB folder gap beside `Invalid data` lines is corrupt media, not a lost book. A format-only ffprobe clears files the scanner rejects — do not trust it. - `Disc 01`/`CD 1` subfolders are parts of one item (110 of them here, under a 192-track book); the extension census counts them, ABS does not. - `.cbr`/`.cbz` group like media but `parseComicMetadata` throws `ERAR_BAD_ARCHIVE`, so those items land media-less: the file, not the scanner. - Scan completion is a NEW scan uuid; `podman logs --since` is a sliding window and ages its own evidence out. - `?force=1` is now recorded as neither discovery nor repair: a forced full re-scan left item count, path set and media-less count identical. Verification names the five book folders that need re-acquiring. --- .../Dankserver/Audiobookshelf.md | 41 +++++++++++++++---- 1 file changed, 32 insertions(+), 9 deletions(-) diff --git a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md index 837370b..d7e15a2 100644 --- a/serverAppConfigs-docs/Dankserver/Audiobookshelf.md +++ b/serverAppConfigs-docs/Dankserver/Audiobookshelf.md @@ -153,21 +153,37 @@ the file, not the bind: the same line count grows while the total climbs. ABS's own single-book layout; formats of one book share a folder), through the journal-guarded mover, then delete the stale item so the old path is not kept, then scan. -- **`?force=1` is not a discovery arm.** Both arms of `POST /libraries/:id/scan` - walk the disk; force additionally re-scans every item that already exists - (measured 1:33 against 0:04 on ~4,600 items). Finding new folders needs neither, - so the deployer's cheap trigger stays on the plain arm. +- **`?force=1` is neither a discovery arm nor a repair arm.** Both arms of + `POST /libraries/:id/scan` walk the disk, so finding new folders needs neither + and the deployer's cheap trigger stays on the plain arm. Force additionally + re-scans every item that already exists (measured 1:33 against 0:04 on ~4,600 + items) and repaired nothing on a library whose residuals were file-level: item + count, path set and media-less count identical before and after. - **`DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()`.** In `LibraryItemController.delete`, `hardDelete = req.query.hard == 1` and then `fs.remove(libraryItemPath)`. Pointed at a covering item, that path is the whole swallowed subtree — `hard=1` there deletes every book beneath it off the array. Assert the item's `path` first, and never pass `hard`. - **`Invalid data found when processing input` names the file, not the bind.** The - scanner logs it per file and skips the file — the book never appears — so a - missing title beside those lines is library debris, not a server fault. Census of - the refusals on the first full scan: 38 of 43 files were 0 bytes, 3 were WAV - bytes wearing a `.mp3` name, 2 were unreadable. `stat -c %s` before any - investigation of the server. + scanner logs it per file and skips that file; a book whose files ALL fail never + becomes an item at all, so the disk-vs-DB folder census shows it as a book with + no item. That is corrupt media, not a lost book. Never clear a suspect file with + a lenient probe: `ffprobe -show_entries format=duration` happily returns three + hours for the WAV bytes wearing a `.mp3` name that the scanner refuses. Read the + log's file paths instead, and `stat -c %s` first — 0-byte files are the common + case, and a truncated download reports `moov atom not found`. +- **A `Disc 01` / `CD 1` subfolder is part of a book, not a book.** The extension + census counts those folders; ABS deliberately does not — its grouping attaches + their files to the ancestor item (measured: 110 such folders under one item + carrying all 192 tracks). Read the basename before believing a disk-vs-DB gap. +- **`.cbr`/`.cbz` group like media and never play.** They sit in the grouping + predicate, so they hide books, but the comic parser throws `ERAR_BAD_ARCHIVE` + (`parseComicMetadata` → `CbrComicBookExtractor`) and the item lands with no + media. A media-less row beside those lines is the file, not the scanner. +- **A scan is finished when a NEW scan uuid completes.** Match + `Library scan "" completed in ...`. `podman logs --since 30m` is a sliding + window: the set of visible completion lines changes on its own as old lines age + out, so a poller keyed on that hash reads a mid-scan number and calls it done. ## Verification @@ -178,3 +194,10 @@ stub of the API above: create-once, idempotent re-run, no secret in a URL, refus by URL when the server does not answer, and the deployer calling it only after the health gate) and `tests/media_overlay.bats` (only [[Plex]] and [[Jellyfin]] may reference the metadata view). + +Five book folders hold media no scanner can read, so they stay out of the library: +three 2024 Nebula nominees in the `_unsorted` box (two `.m4b` with no `moov` atom, +one 36-file `.mp3` set that is not MPEG audio at all), `Anthony Heald/Bounty +Hunter Wars - Book 1` and `Star Wars Full Cast Audio Drama/Dark Empire III` — the +latter two also exist in readable form under their author folders. Fix by +re-acquiring the files; the scanner names every one of them in its log. From e9314ea1ee04afa6bc9ce077ff740e2c5aaae748 Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 04:38:40 -0800 Subject: [PATCH 08/13] fix(jellyfin): anchor the provider-key write on the file's real root MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit TheTVDB writes its own configuration file and its serializer omits a null-valued setting entirely, so a fresh install has no element to fill, under a root element called . xml_secret's fallback insertion was hardcoded to , matched nothing, and install refused with "no place to write " — measured on Dankserver 2026-11-02, where the TVDB key has been present in the secret file since October and has never been installable. The anchor now comes from the document's own root, so both shapes Jellyfin actually produces work (the core plugins' and a plugin's ), and the refusal is kept for a file with no root at all. Tests: the plugin-written shape is red before this commit and green after; every other setting in the file is asserted preserved. --- scripts/jellyfin-providers.sh | 13 ++++++++++- tests/jellyfin_providers.bats | 41 ++++++++++++++++++++++++++++++++--- 2 files changed, 50 insertions(+), 4 deletions(-) diff --git a/scripts/jellyfin-providers.sh b/scripts/jellyfin-providers.sh index c982b65..9436e9d 100755 --- a/scripts/jellyfin-providers.sh +++ b/scripts/jellyfin-providers.sh @@ -135,7 +135,18 @@ text = escape(val) if pat.search(src): out = pat.sub(lambda m: f"{m.group(1)}<{elem}>{text}", src, count=1) else: - out = src.replace("", f" <{elem}>{text}\n", 1) + # The plugin owns this file and writes it under ITS OWN root element + # (TheTVDB's is ), and its serializer omits a + # null-valued setting entirely — so the element being filled can be + # absent outright, and a hardcoded anchor then matches + # nothing. Anchor on the document's real root instead. + out = src + root = re.search(r"<([A-Za-z_][\w.:-]*)[^>]*>", src) + if root and ("" % root.group(1)) in src: + name = root.group(1) + out = src.replace( + "" % name, f" <{elem}>{text}\n", 1 + ) if out == src: raise SystemExit(f"{xml_path}: no place to write <{elem}>") diff --git a/tests/jellyfin_providers.bats b/tests/jellyfin_providers.bats index c9e71d0..416da6e 100755 --- a/tests/jellyfin_providers.bats +++ b/tests/jellyfin_providers.bats @@ -13,9 +13,12 @@ # 3. a downloaded plugin artifact is accepted only on its pinned sha256, its # official guid and its pinned version (a swapped or downgraded artifact # is refused before anything lands in plugins/); -# 4. the TMDb key is written into the EXISTING config with every other -# setting preserved — Jellyfin owns that file, and clobbering it would -# silently reset country, language and adult settings; +# 4. a key is written into the EXISTING config with every other setting +# preserved, under whatever root that file's owner chose — Jellyfin's core +# configs say ``, TheTVDB writes `` and +# omits a null setting outright, so the element to fill can be absent and +# the anchor has to come from the file; clobbering the file would silently +# reset country, language and adult settings; # 5. a TVDB config the operator already saved through the UI is kept; # 6. install is idempotent: a second run changes no byte; # 7. whatever is replaced is copied to the trash first (fleet law: never @@ -254,6 +257,38 @@ XML grep -qF 'true' "$cfg" } +@test "the key lands in a plugin-written config with no ApiKey element" { + # What TheTVDB actually writes on a fresh install: its own root element and + # no at all, because its serializer drops null settings. The + # installer used to anchor on and refuse this file. + cat >"$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml" < + + + 1 + true + +XML + run prov install + [ "$status" -eq 0 ] + printf '%s\n' "${lines[@]}" | grep -qiF 'filled the ApiKey' + local cfg="$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml" + grep -qF "$TVDB_KEY" "$cfg" + grep -qF '1' "$cfg" + grep -qF 'true' "$cfg" +} + +@test "a config file with no root element is refused by name" { + # The guard still has to fire when there is genuinely nowhere to write — + # a refusal naming the file beats inventing a document shape. + printf '\n' \ + >"$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml" + run prov install + [ "$status" -ne 0 ] + printf '%s\n' "${lines[@]}" | grep -qF 'Jellyfin.Plugin.Tvdb.xml' + printf '%s\n' "${lines[@]}" | grep -qiF 'no place to write' +} + @test "a pin in the TVDB secret lands in the plugin config" { # A subscriber-shaped key needs the account's PIN beside it; a licensed key # does not, so the element appears only when the secret carries one. From 69effb542def089c8553f6a1d4bb7faa8ea35191 Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 04:47:16 -0800 Subject: [PATCH 09/13] docs(jellyfin): Shows is on providers, and the per-library door stays dead anyway MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The `Shows` flip landed today (one field, proven in the server's own `LibraryOptions` answer), so the posture table no longer calls it a silent file index and `Music`/`Collections` are named as the remaining ones. Two corrections the same day's measurements force: - The per-library `POST /Items/{id}/Refresh` answered `204` and queued nothing both before the flip and after it — zero journal lines for ten minutes, and `Scan Media Library` kept `LastTaskResult=None, EndTimeUtc=None`, the never-run signature. The provider switch was never the reason that door is dead; only `POST /Library/Refresh` runs (204, then `Running` inside 15 s at 33 % CPU), and it walks every library, so a library-scoped pass has no door on this build. That paragraph also records the null-pair read that separates "never ran" from "just finished", and that query+body on the same call answers `400` with an empty body whatever the values. - The key law: a plugin owns its own config file under its own `` root and omits null settings, so TVDB's `` element never existed and `install` refused — which is what `MISSING key` had been reporting since October. PR #121 fixes the writer. The `Shows` art census replaces speculation with the split that matters: 2,908 of 3,558 series+season carry a primary, 624 have no provider id, 26 have an id and no poster. A `Download` campaign over them took 20 s and downloaded nothing, which is the measurement that says identification is the only lever left. --- serverAppConfigs-docs/Dankserver/Jellyfin.md | 60 +++++++++++++++----- 1 file changed, 45 insertions(+), 15 deletions(-) diff --git a/serverAppConfigs-docs/Dankserver/Jellyfin.md b/serverAppConfigs-docs/Dankserver/Jellyfin.md index 6f827ae..0a02a97 100644 --- a/serverAppConfigs-docs/Dankserver/Jellyfin.md +++ b/serverAppConfigs-docs/Dankserver/Jellyfin.md @@ -89,13 +89,16 @@ looks exactly like an unconfigured library. | Library | `CollectionType` | `EnableInternetProviders` | `EnableRealtimeMonitor` | |---|---|---|---| | `Movies` | movies | true (fixed 2026-11-02) | **false** | -| `Shows` | tvshows | **false** | true | +| `Shows` | tvshows | true (fixed 2026-11-02) | true | | `Music` | music | **false** | true | | `Collections` | boxsets | **false** | false | -`Shows` and `Music` are silent file indexes in the sense above: a refresh answers +`Music` and `Collections` are silent file indexes in the sense above: a refresh answers `204`, stores nothing, logs nothing. Any metadata or artwork work on either one -starts by flipping the switch, never by firing a scan. +starts by flipping the switch, never by firing a scan. `Shows` sat in that set +until 2026-11-02; the door is `POST /Library/VirtualFolders/LibraryOptions` with +the library's own `ItemId` (below), and `scripts/jellyfin-providers.sh verify` is +the one command that says which libraries are still in it. **The per-library option door on 12.x is `POST /Library/VirtualFolders/LibraryOptions`** — `GET` there answers `405`, which is how the route announces itself. The 10.x doors @@ -130,6 +133,15 @@ nothing to look up. Art coverage is capped by identification, not by the image d the items a campaign returns nothing for are the unidentified ones, and no image retry will change that. +**A `Download` campaign over a library that is already dressed does nothing in +seconds, and that is the measurement proving identification is the only lever +left.** Run over all 753 series and 2,805 seasons of `Shows` on 2026-11-02 it +classified 3,558 items in 20 s: 2,908 already carried a primary, 624 have no +provider id at all, 26 have an id and no poster in TMDb — zero downloads. The +artless remainder is not an image-door problem: it is the unnamed long tail (clip +collections, `(Warhammer TV)`-shaped directories), and the lever is folder names +or an identify pass, never another retry loop. + **Series and season art is fetched; episode stills are not, deliberately.** Every episode query is a fresh provider round-trip — nothing caches per episode — measured at ~11 s per item beside ~2 s per series on the same host, with one hit in five in a @@ -250,9 +262,17 @@ scripts/jellyfin-providers.sh verify # plugin + key LENGTHS, never key bytes - **The homelab TVDB key is a licensed key** (its JWT claims say `community_supported: false`), so it needs **no PIN** — the optional `TVDB_PIN=` line exists only for a subscriber-shaped key. -- **TVDB is installed and keyed but is not a metadata agent**: TMDb still - identifies everything. Enabling TVDB as a provider would re-fetch show - metadata across the library, so it is a deliberate, separate change. +- **TVDB is installed but is not a metadata agent**: TMDb still identifies + everything. Enabling TVDB as a provider would re-fetch show metadata across + the library, so it is a deliberate, separate change. +- **A plugin owns its own config file, under its own root, and omits null + settings** — so the element an installer wants to fill may simply not exist, + and its root is ``, not ``. TVDB's file had + no `` element at all, which is why `install` refused with + `no place to write ` from October until the anchor fix: the secret sat + valid in `provider-keys` the whole time, and `verify`'s `MISSING key` was this + fault, not a missing key. The writer now anchors on the file's own root and + still refuses by name when a file has none. - **The deployer places the keys before it restarts jellyfin** (`ensure_provider_keys` in `scripts/deploy-remote.sh`) and warns instead of failing when `provider-keys` is absent, so a host without it still deploys. @@ -302,15 +322,25 @@ anything. The fix is `systemctl --user restart jellyfin.service`, one POST after health answers, and then believe `ScheduledTasks` State at 130 % CPU, not the response code. -**The door that scans is `POST /Library/Refresh` (204), and `ScheduledTasks` is the -progress read.** Measured 2026-11-02 against a `Shows` library that had never been -identified: the per-library `POST /Items/{id}/Refresh?Recursive=true` answers `204` -and queues **nothing** — item census identical afterwards, `Scan Media Library` -idle, journal silent — and it is still an empty `204` after a service restart, so -that status code carries no information at all. `/Library/Refresh` walks every -library; within seconds the journal names folders (`…/TVShows/…`, -`EpisodeMetadataService`) and `GET /ScheduledTasks` reports -`Scan Media Library | Running | 75 %` while the process sits near 200 % CPU. +**The door that scans is `POST /Library/Refresh` (204), and `ScheduledTasks` is +the progress read.** A task that has never completed answers +`LastTaskResult=None, EndTimeUtc=None`, and that pair is what separates "never +ran" from "finished a moment ago" — the difference between a door that is dead +and one that merely answered `204`. + +**The per-library `POST /Items/{id}/Refresh` queues nothing, and the provider +switch is not the reason.** Measured twice on 2026-11-02 against `Shows`: before +the flip and again after it, the call answered `204`, the journal wrote zero lines +over the following ten minutes, and `Scan Media Library` kept its never-run +signature; a service restart changes nothing. `POST /Library/Refresh` does run — +`204`, then `Running` inside 15 s at 33 % CPU — but it walks every library, so on +this build a library-scoped metadata pass has no working door and the global one +is the only lever. + +**That endpoint takes its parameters from one source.** The documented query form +(empty body) and the JSON body form (no query) each parse; supplying both at once +answers `400` with an empty body whatever the values, and `ValidationFailure` is +rejected in the body while the query form accepts it. **`RefreshStatus` on the `GET /Library/VirtualFolders` record is not a progress read.** It stayed `Idle` through a scan that was verifiably running at 75 %. The From 0951d909c9c3bf037f7b2b84c257c2b4e6b4844c Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 04:47:16 -0800 Subject: [PATCH 10/13] docs(agents): the bracket trick for a pkill pattern that must be named MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The existing rule says don't `pkill -f` a pattern your own command line carries, and to prefer `pkill -x` — but a remote one-shot often has to name the pattern, and hitting it again today cost a silent `ssh` exit 255 with zero output, which reads like a network fault. Record the escape (`"serve[-]script.sh"`), the zsh glob-expansion trap that makes an unquoted `[s]` a no-op, and the exit-code signature. --- AGENTS.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/AGENTS.md b/AGENTS.md index ffc5ae1..fad25dd 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -735,7 +735,12 @@ linked runbooks. **Ops** - Never `pkill -f` a pattern that appears in your own command line — it kills your ssh session. - Use `pkill -x `. + Use `pkill -x `, or the bracket trick — `pkill -f "serve[-]script.sh"` — + when the pattern has to be named: the bracket makes the pattern stop matching the + command line that carries it. Quote it: a zsh remote glob-expands an unquoted `[s]` + and the kill silently never runs. Re-hit 2026-11-02 — + `ssh host 'pkill -f jfshowswatch.py; …'` killed the remote shell mid-command, and + ssh exits **255 with zero output**, which reads exactly like a network fault. - Both hosts allow passwordless sudo for the deploy user. Use plain `sudo` in committed scripts (no `echo pw | sudo -S`); `FRANK_SUDO_PW` covers hosts without it. - **ComfyUI on Dankclaw runs with GPU access** (`/dev/kfd` + `/dev/dri` via `AddDevice`, From b232033b11ebfc9bd89c5b16499726713d7f46e8 Mon Sep 17 00:00:00 2001 From: Frank Date: Mon, 2 Nov 2026 09:06:04 -0800 Subject: [PATCH 11/13] docs(tv): split the TV arm into its own page, add the folder-identity laws MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Media Relayout.md crossed the OKF 500-line ceiling with the 2026-11-02 TV laws, and the ceiling was right: the TV arm reads a different authority and refuses for different reasons, like the music and audiobook arms that already have pages. The arm moves verbatim to Media TV Relayout.md and gains two laws from the TVShows pass: a folder naming two series is not evidence of two series (ffprobe's embedded title tag answers per file), and right numbering with a wrong name means rename the folder only. The executor page keeps the executor laws and gains the hold-outside---root/--force law; Jellyfin.md gains the settle read (ScheduledTasks State, since /Library/ScanStatus is a 404 on 12.4). Index regenerated with gen-vault-index.py. cwd: /home/blank/.omp/wt/wt-20261030-163847-6d1fc1f · session: 01a18f2e-18b1-7573-bcd5-ac99636f8bcb --- serverAppConfigs-docs/Dankserver/Jellyfin.md | 9 +- .../Dankserver/Media Relayout.md | 173 ++------------- .../Dankserver/Media TV Relayout.md | 201 ++++++++++++++++++ serverAppConfigs-docs/index.md | 1 + 4 files changed, 226 insertions(+), 158 deletions(-) create mode 100644 serverAppConfigs-docs/Dankserver/Media TV Relayout.md diff --git a/serverAppConfigs-docs/Dankserver/Jellyfin.md b/serverAppConfigs-docs/Dankserver/Jellyfin.md index 0a02a97..96dee38 100644 --- a/serverAppConfigs-docs/Dankserver/Jellyfin.md +++ b/serverAppConfigs-docs/Dankserver/Jellyfin.md @@ -3,7 +3,7 @@ type: reference title: Jellyfin description: "Jellyfin on Dankserver: quadlet unit, overlay metadata view, provider keys, ports, access, transcode status." tags: [jellyfin, dankserver, media-server] -timestamp: 2026-10-30T00:00:00Z +timestamp: 2026-11-02T11:00:00Z --- # Jellyfin @@ -328,6 +328,13 @@ the progress read.** A task that has never completed answers ran" from "finished a moment ago" — the difference between a door that is dead and one that merely answered `204`. +**The settle read is `GET /ScheduledTasks`: the row whose `Name` is exactly +`Scan Media Library`, and its `State` (`Idle`|`Running`).** `/Library/ScanStatus` +is a 404 on 12.4, so a watcher built on the old endpoint reports nothing and a +scanning server looks an idle one. Call it settled only after a `Running` +sighting followed by two consecutive `Idle`s: sub-tasks leave gaps that a single +poll reads as finished, and a census taken in one reports the pre-scan library. + **The per-library `POST /Items/{id}/Refresh` queues nothing, and the provider switch is not the reason.** Measured twice on 2026-11-02 against `Shows`: before the flip and again after it, the call answered `204`, the journal wrote zero lines diff --git a/serverAppConfigs-docs/Dankserver/Media Relayout.md b/serverAppConfigs-docs/Dankserver/Media Relayout.md index 3ed15db..0f3248a 100644 --- a/serverAppConfigs-docs/Dankserver/Media Relayout.md +++ b/serverAppConfigs-docs/Dankserver/Media Relayout.md @@ -3,7 +3,7 @@ type: reference title: Media Relayout description: "Census, plan and apply the movie, TV, music and audiobook libraries: the gates, the refusals, the rollback." tags: [dankserver, media-server, jellyfin, plex, procedure] -timestamp: 2026-11-01T01:00:00Z +timestamp: 2026-11-02T11:00:00Z --- # Media Relayout @@ -186,162 +186,10 @@ show. A name whose marker is `Part 2` is a film and stays put. ## The TV library's own relayout -`/mnt/entropy/TVShows` is scene packs rather than a library, and Jellyfin has -already matched and numbered most of it. `scripts/media-shows.py` re-files that -library into the shape Jellyfin reads, with **tvdb as the name authority**: - - () [tvdbid-NNNN]/Season NN/ - () - SxxEyy - .ext - -It never guesses: every series name, episode number and title it writes traces to -an authority, and what has none is parked for a human to read. - -```bash -# 1. The authority: Jellyfin's own matched rows — each carries the series' TMDb -# and Tvdb ids and the title the row holds (the DB opens read-only). -python3 scripts/media-census.py jellyfin --root /mnt/entropy/TVShows \ - --out /tmp/tv_titles.tsv -# 2. What is on disk (stat only), and the runtimes the film arm needs. -python3 scripts/media-census.py enumerate --root /mnt/entropy/TVShows \ - --out /tmp/tv_enum.tsv -python3 scripts/media-census.py probe --root /mnt/entropy/TVShows \ - --enum /tmp/tv_enum.tsv --jobs 6 --out /tmp/tv_probed.tsv -# The probe is the slow step: ffprobe reads both ends of every file. So re-use -# the last one — a duration belongs to the file, not to its name: re-key the -# previous census through that pass's own journal (`src` -> `dst`) instead of -# seeking the array again. It is also the only column the film arm reads, and -# only for a file with no marker and no provider row (4,794 of 58,551 today, -# all of them covered by a re-key that costs seconds, not an hour). -# 3. The plan. --tmdb needs TMDB_API_KEY — the v3 key in ~/.config/secrets/tmdb, -# which travels as the `api_key` query parameter. Only a v4 request token is -# accepted as a Bearer header, and the wrong shape answers 401 on EVERY call, -# which the summary line flags as NOT provider-complete. --tvdb reads its -# `api_key=` line from ~/.config/secrets/tvdb, never argv. Both write the SAME -# cache file, so the plan is reproducible offline afterwards: run it once to -# fill the cache, and quote the second run. -# Paths are only strings to a dry run, so this step can run on the workstation -# against scp'd TSVs — which is where the two provider secrets live. -python3 scripts/media-shows.py --root /mnt/entropy/TVShows --enum /tmp/tv_enum.tsv \ - --titles /tmp/tv_titles.tsv --probed /tmp/tv_probed.tsv \ - --tmdb-cache /mnt/entropy/media-tv-names.json --tmdb --tvdb \ - --park /mnt/entropy/media-park/TVShows \ - --out /mnt/entropy/media-relayout-shows2-plan.tsv \ - --exceptions /mnt/entropy/media-relayout-shows2-exceptions.txt -# 4. Read the exceptions file FIRST — it is the review, not a log. Same executor -# as above, but on this arm's OWN journal, NAMED PER PASS: the journal of an -# applied pass is its unwind record, so a second pass that reuses the name -# destroys the only map back. -scripts/media-apply.py apply --plan /mnt/entropy/media-relayout-shows2-plan.tsv \ - --root /mnt/entropy --journal /mnt/entropy/media-relayout-shows2.jsonl --apply -``` - -Evidence, strongest first — the order the planner believes things: - -- the **episode marker the file's own name states** (`S01E02-E03`, `1x04`); a digit - run longer than three at that position is NOT a number — `S03E2019` and - `S04E47370355` are one box set's airdate fragment and pack index — so such a file - is renumbered by tvdb's title or parks, and is never truncated down to fit; -- the **`Season NN` folder it sits in and the number its name leads with inside - it**: `Season 19/01 Charles Barkley, Nirvana.avi` is S19E01; -- the **provider row**: which episode the file IS, and the numbering of a file - whose own name says nothing; -- the **row's tvdb id, else its TMDb id carried through TMDb's external ids, else - an exact tvdb name search**: that id names the SERIES FOLDER and supplies the - episode title, because a name the server read off a folder is not a show name. - The folder CARRIES the id: `[tvdbid-NNNN]` is what Jellyfin's Tvdb agent and - Plex's scanner both read, so a show whose title is ambiguous resolves by number - instead of by a human guessing which folder is which. The id rides the folder - only — a file name gets 180 characters and the episode title needs the room. -- a **bonus folder** anywhere above the file, or Jellyfin's `ExtraType` → the - series' canonical extra folder; -- a runtime over `SHORT_MAX` makes it film-shaped, and it leaves for `--movies` only - when the movie provider's title is a run of words the FILE itself states and its - year agrees with the file's: the query is only what `title_of` salvaged from a - release name, and the first hit for a short one is a near miss — a 1963 Bond film - for the word "russia", *Special Edition I* for a file saying *Edition III*. Else it - parks. - -Four refusals carry the design: what the planner will NOT do is the point of it. - -- **Folders resolve a series; they never invent one.** A folder's words may - resolve a file to a series a provider has already named — `(CBB) Tengen Toppa - Gurren Lagann - The Complete Series (BD 1080p)` names the registered *Tengen - Toppa Gurren Lagann* — but a bonus subfolder is not a show. An early run made - `…/03. Extra/01. NCOP/NCOP 01.mkv` into a root series called `01 NCOP`. -- **A scraped row does not outrank the filesystem.** Jellyfin numbers - multi-season packs badly: every file under `SNL 16-20/Season 16..19` carries - season 16 episode 20, and one Gundam row says season 2004. The folder, then - the file's own leading number, win. -- **A season that is a YEAR is the date matcher, not a season.** Jellyfin files - specials by airdate, so a row can arrive saying `season 2002`; that is not a - season and it never becomes a folder. The one authority on such a row is tvdb's - episode list read backwards: the title the row carries names the (season, - episode) tvdb gives it. A title tvdb gives to two episodes names nothing, and - the file parks with the reason it always carried. `Season 00` IS a legal - season — specials live there — so a row that really says 0 files there instead - of parking. -- **A season tvdb's own list contradicts gets no tvdb title.** An anime pack - numbered by its release group books `S05E01` in a season tvdb does not hold, or - books more episodes there than tvdb lists: the slot its number points at is a - different episode, and dressing it in that episode's title is a confident lie. - The file still renames to the standard shape; the title does not ride, and the - series is listed under *seasons whose numbering tvdb contradicts* in the - exceptions. A season holding FEWER files than tvdb lists stays plausible — a - partial pack is the normal shape. Neither reading holds for a list tvdb - cut off at its own page limit: the default episode query answers at most - 500 rows and states no total, and One Piece, Crayon Shin-chan and - Detective Conan each answer exactly 500 while holding over a thousand — - a page bounds nothing above itself, so it contradicts nothing. - -**A provider name is evidence only when an id backs it.** Jellyfin names a Series -after the folder it scanned, so a library filed by release folders ends up with -one Series item per box set — `24 (2001) Season 1`, `Modern Family S05` — and -copying those names verbatim launders folder junk into the contract. The planner -resolves each series to its tvdb id and writes tvdb's name and year; the merged -identity is also what turns a box set split across five release folders into ONE -folder, and what makes two files claiming one episode MEET in the duplicate rule. -A series no id resolves keeps the provider's words and is listed under *no tvdb -identity* in the exceptions — that list, not a rename, is the review. - -**The review pile lives outside the library.** Files parked inside -`/mnt/entropy/TVShows` were indexed as one show named `_unsorted`, and that -show's rows then looked like evidence. So a row whose series is the hidden -top-level folder the file itself sits in is dropped — and the folder is matched -with its leading underscore stripped, because the server *displays* `_unsorted` as -`unsorted`: compare the spelled names and the pile survives the relayout as a -visible series folder that real files get renamed into. `--park` outside the root -makes the next plan re-file the pile instead of skipping it. Inside a hidden -top-level folder a file's own words may MATCH a registered show, never invent one. - -**The operator's id map is the door for a name tvdb cannot be searched by.** -`scripts/media-shows.py` reads `scripts/media-shows-tvdb-ids.tsv` by default: one -line per pack folder, `folder tvdb id note`. The file exists because -tvdb titles a record in the show's own language and the English name rides in -*aliases* the name search never reads: `ブラッククローバー` (331753) answers to no -spelling of "Black Clover", and the bare alias `One Piece` belongs to 392276, the -2023 live action, not to the 1999 record (81797) the pack here is. The map -outranks a provider id — a box set's provider id is the server's own guess at a -folder name — and it merges the release folders that are one show. Everything it -touched is written to the exceptions: every folder it named, every row that -matched no folder, every disagreement with a provider id, every id tvdb never -answered. A line whose id is not a number stops the plan, because a typo renames -a whole pack to the wrong show. Only folders the search CANNOT reach belong in -it; a row for a folder the search would resolve hides a live lookup behind a -number nobody re-checks. - -`scripts/media-shows-tvdb.py` makes the sheet the map is built from. It reads a -plan's `mkdir` rows, asks tvdb about every destination folder carrying no -`[tvdbid-`, and writes one row per folder with the verdict that says how much was -guessed to get there: `EXACT` (tvdb's own spelling), `PUNCT` (equal only after -release junk is stripped from the folder's words), `NEAR` (tvdb answered -something else — the best hit is printed for a human and nothing applies it), -`NONE`. Acceptance never moved across the passes that built this: a hit counts -only when tvdb's name equals the query with every non-alphanumeric dropped from -both sides, and the record's year still has to fall inside any year span the -folder names. That gate is what caught three confident picks — Fairy Tail's 2024 -sequel (410031), the 2026 Hokuto no Ken remake (465216) and `Mugyutto! Black -Clover` (378762) offered for Black Clover. It is why the map is approved by hand -and the sheet is never fed to the planner. +`/mnt/entropy/TVShows` is its own arm: a different authority (tvdb), its own +refusals, and its own test for whether a folder really holds two shows — +[[Media TV Relayout]]. It shares this page's executor, the one-journal-per-arm +law, the sweep and the rollback. ## The music and audiobook arms @@ -432,6 +280,14 @@ executor, the one-journal-per-arm law, the sweep and the rollback. `--root` set to the share that holds BOTH libraries: the executor refuses any path outside `--root`, and a journal shared with the movie relayout would undo both runs when only one of them is wanted. +- **A hold folder outside `--root` forces `--force` on BOTH `apply` and + `sweep`.** `sweep` refuses the entire journal when any journalled path sits + outside the root (`this journal belongs to another library`) — the same guard + the plan hits, and it exists to stop one pass unwinding another's files. + `--force` relaxes it and stays safe there: `sweep_targets` marks only + ancestors under `--root`, so a hold folder beyond the root can never be + selected; it only blocks everything that is. The cleaner shape is the law + above — put `--root` on the share that holds both. - **Sidecar metadata does not follow a rename, because it lives on another filesystem.** The media servers write `.nfo` and artwork through the @@ -440,6 +296,9 @@ executor, the one-journal-per-arm law, the sweep and the rollback. `scripts/media-view.sh reap --apply` — stranded sidecars become ghost folders that both servers browse and report as content. Whether this host mounts the view at all is stated on that page; `media-apply.py` stays silent when it does not. + A rename of folders the server never identified strands nothing: Jellyfin + writes a sidecar only for a series it recognised, so `reap` answering + `no ghosts` after such a pass is the expected result, not a broken tool. - **Nothing deletes through the metadata view.** A delete there writes an overlayfs whiteout in the upper layer, which hides the array's copy without removing it; the file returns when the view is unmounted. Deletions in this pipeline happen diff --git a/serverAppConfigs-docs/Dankserver/Media TV Relayout.md b/serverAppConfigs-docs/Dankserver/Media TV Relayout.md new file mode 100644 index 0000000..5c7a9dd --- /dev/null +++ b/serverAppConfigs-docs/Dankserver/Media TV Relayout.md @@ -0,0 +1,201 @@ +--- +type: reference +title: Media TV Relayout +description: "Re-file TVShows from scene packs into the shape Jellyfin reads: authority order, refusals, the two-series split test." +tags: [dankserver, media-server, jellyfin, plex, procedure] +timestamp: 2026-11-02T11:00:00Z +--- + +# Media TV Relayout + +The `/mnt/entropy/TVShows` share arrived as scene packs — release folders whose +names are the release, not the show. This arm re-files it into the shape +[[Jellyfin]] reads, with tvdb as the name authority. The executor, the journal, +the sweep and the rollback belong to [[Media Relayout]]; this page owns the TV +authority order, the refusals, and the test that decides a split. + +`/mnt/entropy/TVShows` is scene packs rather than a library, and Jellyfin has +already matched and numbered most of it. `scripts/media-shows.py` re-files that +library into the shape Jellyfin reads, with **tvdb as the name authority**: + + () [tvdbid-NNNN]/Season NN/ + () - SxxEyy - .ext + +It never guesses: every series name, episode number and title it writes traces to +an authority, and what has none is parked for a human to read. + +```bash +# 1. The authority: Jellyfin's own matched rows — each carries the series' TMDb +# and Tvdb ids and the title the row holds (the DB opens read-only). +python3 scripts/media-census.py jellyfin --root /mnt/entropy/TVShows \ + --out /tmp/tv_titles.tsv +# 2. What is on disk (stat only), and the runtimes the film arm needs. +python3 scripts/media-census.py enumerate --root /mnt/entropy/TVShows \ + --out /tmp/tv_enum.tsv +python3 scripts/media-census.py probe --root /mnt/entropy/TVShows \ + --enum /tmp/tv_enum.tsv --jobs 6 --out /tmp/tv_probed.tsv +# The probe is the slow step: ffprobe reads both ends of every file. So re-use +# the last one — a duration belongs to the file, not to its name: re-key the +# previous census through that pass's own journal (`src` -> `dst`) instead of +# seeking the array again. It is also the only column the film arm reads, and +# only for a file with no marker and no provider row (4,794 of 58,551 today, +# all of them covered by a re-key that costs seconds, not an hour). +# 3. The plan. --tmdb needs TMDB_API_KEY — the v3 key in ~/.config/secrets/tmdb, +# which travels as the `api_key` query parameter. Only a v4 request token is +# accepted as a Bearer header, and the wrong shape answers 401 on EVERY call, +# which the summary line flags as NOT provider-complete. --tvdb reads its +# `api_key=` line from ~/.config/secrets/tvdb, never argv. Both write the SAME +# cache file, so the plan is reproducible offline afterwards: run it once to +# fill the cache, and quote the second run. +# Paths are only strings to a dry run, so this step can run on the workstation +# against scp'd TSVs — which is where the two provider secrets live. +python3 scripts/media-shows.py --root /mnt/entropy/TVShows --enum /tmp/tv_enum.tsv \ + --titles /tmp/tv_titles.tsv --probed /tmp/tv_probed.tsv \ + --tmdb-cache /mnt/entropy/media-tv-names.json --tmdb --tvdb \ + --park /mnt/entropy/media-park/TVShows \ + --out /mnt/entropy/media-relayout-shows2-plan.tsv \ + --exceptions /mnt/entropy/media-relayout-shows2-exceptions.txt +# 4. Read the exceptions file FIRST — it is the review, not a log. Same executor +# as above, but on this arm's OWN journal, NAMED PER PASS: the journal of an +# applied pass is its unwind record, so a second pass that reuses the name +# destroys the only map back. +scripts/media-apply.py apply --plan /mnt/entropy/media-relayout-shows2-plan.tsv \ + --root /mnt/entropy --journal /mnt/entropy/media-relayout-shows2.jsonl --apply +``` + +Evidence, strongest first — the order the planner believes things: + +- the **episode marker the file's own name states** (`S01E02-E03`, `1x04`); a digit + run longer than three at that position is NOT a number — `S03E2019` and + `S04E47370355` are one box set's airdate fragment and pack index — so such a file + is renumbered by tvdb's title or parks, and is never truncated down to fit; +- the **`Season NN` folder it sits in and the number its name leads with inside + it**: `Season 19/01 Charles Barkley, Nirvana.avi` is S19E01; +- the **provider row**: which episode the file IS, and the numbering of a file + whose own name says nothing; +- the **row's tvdb id, else its TMDb id carried through TMDb's external ids, else + an exact tvdb name search**: that id names the SERIES FOLDER and supplies the + episode title, because a name the server read off a folder is not a show name. + The folder CARRIES the id: `[tvdbid-NNNN]` is what Jellyfin's Tvdb agent and + Plex's scanner both read, so a show whose title is ambiguous resolves by number + instead of by a human guessing which folder is which. The id rides the folder + only — a file name gets 180 characters and the episode title needs the room. +- a **bonus folder** anywhere above the file, or Jellyfin's `ExtraType` → the + series' canonical extra folder; +- a runtime over `SHORT_MAX` makes it film-shaped, and it leaves for `--movies` only + when the movie provider's title is a run of words the FILE itself states and its + year agrees with the file's: the query is only what `title_of` salvaged from a + release name, and the first hit for a short one is a near miss — a 1963 Bond film + for the word "russia", *Special Edition I* for a file saying *Edition III*. Else it + parks. + +Four refusals carry the design: what the planner will NOT do is the point of it. + +- **Folders resolve a series; they never invent one.** A folder's words may + resolve a file to a series a provider has already named — `(CBB) Tengen Toppa + Gurren Lagann - The Complete Series (BD 1080p)` names the registered *Tengen + Toppa Gurren Lagann* — but a bonus subfolder is not a show. An early run made + `…/03. Extra/01. NCOP/NCOP 01.mkv` into a root series called `01 NCOP`. +- **A scraped row does not outrank the filesystem.** Jellyfin numbers + multi-season packs badly: every file under `SNL 16-20/Season 16..19` carries + season 16 episode 20, and one Gundam row says season 2004. The folder, then + the file's own leading number, win. +- **A season that is a YEAR is the date matcher, not a season.** Jellyfin files + specials by airdate, so a row can arrive saying `season 2002`; that is not a + season and it never becomes a folder. The one authority on such a row is tvdb's + episode list read backwards: the title the row carries names the (season, + episode) tvdb gives it. A title tvdb gives to two episodes names nothing, and + the file parks with the reason it always carried. `Season 00` IS a legal + season — specials live there — so a row that really says 0 files there instead + of parking. +- **A season tvdb's own list contradicts gets no tvdb title.** An anime pack + numbered by its release group books `S05E01` in a season tvdb does not hold, or + books more episodes there than tvdb lists: the slot its number points at is a + different episode, and dressing it in that episode's title is a confident lie. + The file still renames to the standard shape; the title does not ride, and the + series is listed under *seasons whose numbering tvdb contradicts* in the + exceptions. A season holding FEWER files than tvdb lists stays plausible — a + partial pack is the normal shape. Neither reading holds for a list tvdb + cut off at its own page limit: the default episode query answers at most + 500 rows and states no total, and One Piece, Crayon Shin-chan and + Detective Conan each answer exactly 500 while holding over a thousand — + a page bounds nothing above itself, so it contradicts nothing. + +**A provider name is evidence only when an id backs it.** Jellyfin names a Series +after the folder it scanned, so a library filed by release folders ends up with +one Series item per box set — `24 (2001) Season 1`, `Modern Family S05` — and +copying those names verbatim launders folder junk into the contract. The planner +resolves each series to its tvdb id and writes tvdb's name and year; the merged +identity is also what turns a box set split across five release folders into ONE +folder, and what makes two files claiming one episode MEET in the duplicate rule. +A series no id resolves keeps the provider's words and is listed under *no tvdb +identity* in the exceptions — that list, not a rename, is the review. + +**The review pile lives outside the library.** Files parked inside +`/mnt/entropy/TVShows` were indexed as one show named `_unsorted`, and that +show's rows then looked like evidence. So a row whose series is the hidden +top-level folder the file itself sits in is dropped — and the folder is matched +with its leading underscore stripped, because the server *displays* `_unsorted` as +`unsorted`: compare the spelled names and the pile survives the relayout as a +visible series folder that real files get renamed into. `--park` outside the root +makes the next plan re-file the pile instead of skipping it. Inside a hidden +top-level folder a file's own words may MATCH a registered show, never invent one. + +**The operator's id map is the door for a name tvdb cannot be searched by.** +`scripts/media-shows.py` reads `scripts/media-shows-tvdb-ids.tsv` by default: one +line per pack folder, `folder tvdb id note`. The file exists because +tvdb titles a record in the show's own language and the English name rides in +*aliases* the name search never reads: `ブラッククローバー` (331753) answers to no +spelling of "Black Clover", and the bare alias `One Piece` belongs to 392276, the +2023 live action, not to the 1999 record (81797) the pack here is. The map +outranks a provider id — a box set's provider id is the server's own guess at a +folder name — and it merges the release folders that are one show. Everything it +touched is written to the exceptions: every folder it named, every row that +matched no folder, every disagreement with a provider id, every id tvdb never +answered. A line whose id is not a number stops the plan, because a typo renames +a whole pack to the wrong show. Only folders the search CANNOT reach belong in +it; a row for a folder the search would resolve hides a live lookup behind a +number nobody re-checks. + +`scripts/media-shows-tvdb.py` makes the sheet the map is built from. It reads a +plan's `mkdir` rows, asks tvdb about every destination folder carrying no +`[tvdbid-`, and writes one row per folder with the verdict that says how much was +guessed to get there: `EXACT` (tvdb's own spelling), `PUNCT` (equal only after +release junk is stripped from the folder's words), `NEAR` (tvdb answered +something else — the best hit is printed for a human and nothing applies it), +`NONE`. Acceptance never moved across the passes that built this: a hit counts +only when tvdb's name equals the query with every non-alphanumeric dropped from +both sides, and the record's year still has to fall inside any year span the +folder names. That gate is what caught three confident picks — Fairy Tail's 2024 +sequel (410031), the 2026 Hokuto no Ken remake (465216) and `Mugyutto! Black +Clover` (378762) offered for Black Clover. It is why the map is approved by hand +and the sheet is never fed to the planner. + +## Laws of this arm + +- **A folder that names two series is not evidence of two series — the files + answer.** `ffprobe -show_entries format_tag=title` names the series each file + belongs to. Measured 2026-11-02 on `COW and CHICKEN … Complete Animated TV + Series`, a bundle whose name pairs *Cow and Chicken* with *I Am Weasel*: 103 + of 104 files carry a `Cow and Chicken - SxxEyy - ` tag and none carries + an *I Am Weasel* one. The 7-minute duration class is a half-episode segment, + not a second show, and a file holding both segments needs a remux, which this + executor will never do. Split only when the tags (or a provider episode list) + name both series per file; everything else is a rename. +- **When the numbering is right and only the name is wrong, rename the folder + and nothing else.** Tags whose `Sxx` matches the `Season NN` folder holding + them, per-season counts equal, and no duplicate `SxxEyy` key anywhere prove + the layout is already correct. Renumbering files under that evidence buys + nothing and can silently mislabel an episode. + +## See Also + +[[Media Relayout]] — the executor, the journal, the sweep and the rollback this +arm shares. [[Jellyfin]] — the reader of the resulting shape. [[Plex]] — the +second reader. [[Media Metadata View]] — where a rename's sidecars strand, or +do not. + +## Citations + +- Jellyfin Shows documentation — series, season and episode folder/file naming: + <https://jellyfin.org/docs/general/server/media/shows> diff --git a/serverAppConfigs-docs/index.md b/serverAppConfigs-docs/index.md index 44e044e..498a94d 100644 --- a/serverAppConfigs-docs/index.md +++ b/serverAppConfigs-docs/index.md @@ -57,6 +57,7 @@ is a locator: one row, one clause, no numbers. | [[Media Metadata View]] | The SSD-backed overlay the two media servers read and write: shape, guards, commands, laws, rollback. | | [[Media Music Relayout]] | Re-filing /mnt/entropy/Music by Jellyfin's tag rows: the census, the album key, the refusals. | | [[Media Relayout]] | Census, plan and apply the movie, TV, music and audiobook libraries: the gates, the refusals, the rollback. | +| [[Media TV Relayout]] | Re-file TVShows from scene packs into the shape Jellyfin reads: authority order, refusals, the two-series split test. | | [[Plex]] | Plex Media Server is a **containerized media server on Dankserver**: a podman | | [[redis]] | The Redis cache and task queue for RomM: quadlet unit, network alias topology, and the backup path. | | [[RomM]] | RomM (ROM Manager) is a self-hosted ROM library manager and player. | From a405c5ed35f0f530d74dfba5114c80ddb3de63ab Mon Sep 17 00:00:00 2001 From: Frank <frank@dankware> Date: Mon, 2 Nov 2026 09:51:53 -0800 Subject: [PATCH 12/13] fix(media-apply): refuse a plan row whose verb the tool does not know MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit read_plan's if/elif chain had no else, so any row that was not mkdir, keep or move was dropped in silence. A plan of nothing but typos therefore applied as a successful no-op: on 2026-11-02 a pass meant to rename two TVShows series folders reported "applied: 0 moved" and exit 0, and the journal was never written — nothing told the operator the plan had never been read. An unread row is a plan bug, and every other plan bug in this tool refuses before the first rename, so now this one does too, naming the line and the verb. cwd: /home/blank/.omp/wt/wt-20261030-163847-6d1fc1f · session: 01a18f2e-18b1-7573-bcd5-ac99636f8bcb --- scripts/media-apply.py | 12 +++++-- .../Dankserver/Media Relayout.md | 9 +++-- tests/media_relayout.bats | 33 +++++++++++++++++++ 3 files changed, 49 insertions(+), 5 deletions(-) diff --git a/scripts/media-apply.py b/scripts/media-apply.py index f497cf0..75c3c94 100755 --- a/scripts/media-apply.py +++ b/scripts/media-apply.py @@ -121,14 +121,14 @@ class Journal: def read_plan(path, root, force): """Plan rows: (mkdirs, keeps, moves), header dropped, all paths under root.""" - mkdirs, keeps, moves = [], [], [] + mkdirs, keeps, moves, bad = [], [], [], [] with open(path) as fh: header = fh.readline() if not header.startswith("action\t"): raise Refuse(f"{path} is not a plan (first column is " f"{header.split(chr(9))[0]!r}, expected 'action') —" " regenerate it with scripts/media-plan.py") - for line in fh: + for n, line in enumerate(fh, 2): f = line.rstrip("\n").split("\t") if f[0] == "mkdir": mkdirs.append(f[2]) @@ -136,6 +136,14 @@ def read_plan(path, root, force): keeps.append(f[1]) elif f[0] == "move": moves.append((f[1], f[2], f[3] if len(f) > 3 else "")) + else: + bad.append(f"line {n}: unknown action {f[0]!r} — this tool" + " knows mkdir, keep and move (a rename is a move)") + if bad: + # A row this tool could not read is a plan bug, and a plan bug refuses: + # silently dropping it turns a typo into a no-op run that reports success. + head = "; ".join(bad[:3]) + raise Refuse(head + (f" (+{len(bad) - 3} more)" if len(bad) > 3 else "")) if not force: for p in mkdirs + keeps + [p for m in moves for p in m[:2]]: if p and not (p == root or p.startswith(root + os.sep)): diff --git a/serverAppConfigs-docs/Dankserver/Media Relayout.md b/serverAppConfigs-docs/Dankserver/Media Relayout.md index 3ed15db..b5cb277 100644 --- a/serverAppConfigs-docs/Dankserver/Media Relayout.md +++ b/serverAppConfigs-docs/Dankserver/Media Relayout.md @@ -403,9 +403,12 @@ executor, the one-journal-per-arm law, the sweep and the rollback. - **The journal lives BESIDE the library root, never inside it** (default: `media-relayout.jsonl` next to the root): a log inside the library is a file Jellyfin indexes. -- **Refusals are all-or-nothing.** An occupied target, a vanished source or a - duplicate destination stops the run *before the first rename*, so the operator - fixes one thing and re-runs instead of untangling a half-applied plan. +- **Refusals are all-or-nothing.** An occupied target, a vanished source, a + duplicate destination, or a plan row whose verb is not `mkdir`, `keep` or + `move`, stops the run *before the first rename*, so the operator fixes one + thing and re-runs instead of untangling a half-applied plan. An unread verb is + refused by line number: silently dropping one turned a pass meant to rename + two series folders into `applied: 0 moved` with exit 0 (2026-11-02). `--force` relaxes only the opinion guards; it never permits overwriting. - **A sweep deletes only what the journal proves the run emptied.** A folder is eligible when a journalled file moved out of it or out of something under it, diff --git a/tests/media_relayout.bats b/tests/media_relayout.bats index e3b2af4..c5f300f 100755 --- a/tests/media_relayout.bats +++ b/tests/media_relayout.bats @@ -460,6 +460,39 @@ print(sum(1 for l in open(sys.argv[1]) if json.loads(l).get("op") == "move"))' " [ -e "$LIB/Alien (1979)/Alien.1979.1080p.BluRay.x265.10bit.mkv" ] } +# A hand-written plan row whose verb is not in the executor's vocabulary used +# to be dropped in silence: read_plan's if/elif chain had no else, so a plan of +# nothing but typos applied as a successful no-op (hit for real on 2026-11-02, +# when a pass meant to rename two series folders reported "applied: 0 moved" +# and exit 0). An unread row is a plan bug, and a plan bug refuses. +@test "a plan row with a verb this tool does not know stops the whole run" { + plan + JOURNAL="$BATS_TEST_TMPDIR/journal.jsonl" + at "$LIB/Stray/Stray.S01E01.mkv" 4000 + printf 'rename\t%s\t%s\tfile\n' \ + "$LIB/Stray/Stray.S01E01.mkv" "$LIB/Stray/Stray - S01E01.mkv" >> "$OUT" + run python3 "$APPLY" apply --plan "$OUT" --root "$LIB" --journal "$JOURNAL" --apply + [ "$status" -ne 0 ] + [[ "$output" == *"rename"* ]] + # all-or-nothing: the rows this tool DOES understand did not run either + [ -e "$LIB/Stray/Stray.S01E01.mkv" ] + [ ! -e "$JOURNAL" ] +} + +@test "a plan whose every row is an unknown verb refuses instead of reporting success" { + JOURNAL="$BATS_TEST_TMPDIR/journal.jsonl" + at "$LIB/Stray/Stray.S01E01.mkv" 4000 + { printf 'action\tsrc\tdst\tkind\n' + printf 'rename\t%s\t%s\tfile\n' \ + "$LIB/Stray/Stray.S01E01.mkv" "$LIB/Stray/Stray - S01E01.mkv" + } > "$BATS_TEST_TMPDIR/typo.tsv" + run python3 "$APPLY" apply --plan "$BATS_TEST_TMPDIR/typo.tsv" --root "$LIB" --journal "$JOURNAL" --apply + [ "$status" -ne 0 ] + [[ "$output" == *"unknown action"* ]] + [[ "$output" != *"applied"* ]] + [ -e "$LIB/Stray/Stray.S01E01.mkv" ] +} + @test "a run interrupted between the journal write and the move finishes the job" { plan JOURNAL="$BATS_TEST_TMPDIR/journal.jsonl" From 8f0b1982c26036a8c408c8a1b66dc10340aabd09 Mon Sep 17 00:00:00 2001 From: Frank <frank@dankware> Date: Mon, 2 Nov 2026 13:56:03 -0800 Subject: [PATCH 13/13] docs(tv): the parked-folder merge, and the laws that gate it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A parked folder whose name parses to a registered show merges into the row that owns the show. The proof is the episode rows standing under the owner folder — never the id-less folder list, because one tvdb id can carry two `Series` rows and an id-less folder that parses gets identified on its own and keeps its own folder, offering the show twice. No library file moves while `Scan Media Library` is Running: wait for a Running sighting and then two consecutive Idle polls, because a gap between sub-tasks reads as finished. Both sides speak different path dialects (the server records the container coordinate, the journal speaks the array), so a join that forgets the translation reads as an empty library. Lands on [[Media TV Relayout]] — the page that owns the TV arm — so `Media Relayout.md` stays under the OKF ceiling. --- .../Dankserver/Media TV Relayout.md | 40 +++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/serverAppConfigs-docs/Dankserver/Media TV Relayout.md b/serverAppConfigs-docs/Dankserver/Media TV Relayout.md index 5c7a9dd..a7ed683 100644 --- a/serverAppConfigs-docs/Dankserver/Media TV Relayout.md +++ b/serverAppConfigs-docs/Dankserver/Media TV Relayout.md @@ -171,6 +171,36 @@ sequel (410031), the 2026 Hokuto no Ken remake (465216) and `Mugyutto! Black Clover` (378762) offered for Black Clover. It is why the map is approved by hand and the sheet is never fed to the planner. +**A parked folder merges into the row that owns the show.** The folder that never +got an id usually holds episodes the identified folder for that show lacks, and +the server shows the audience neither: the parked copy is invisible and the owner +reads as a short series. One `media-apply.py` plan per show, built from the +server's own rows: + +- the **owner is the folder whose `Series` row carries the tvdb id**; the parked + side is the folder with the same words and no id (or the same id under a second + row — see the law below); +- the **comparison is episode identity, not filename**: which (season, episode) + the provider row gives each file, read straight out of the server. Release + folders renumber — one park carried `S01E01-11` of a show tvdb books as seasons + 2 and 3, and every episode runs the same eleven minutes, so neither the marker + nor the duration says what one query into the rows says; +- **a season the owner has no file in moves whole**: `mkdir` the `Season NN`, + then one row per file with its name unchanged — the folder shape is the + contract, the file name is not; +- **a subtitle pack is re-keyed, never just moved**: a sidecar attaches only when + its stem equals the video's, so each `.srt` has to land on the OWNER's stem with + the language suffix kept (`… - The Cat and the Claw (1).en.srt`). Moved under + its release-group name it stays a folder of 88 subtitles no player ever offers; +- **the same episode on both sides is the operator's call**, never a plan row: + which copy survives is a quality judgment — the parked copy is usually the + bigger one, and a re-cut is not a worse one — and the parked file often carries + no title to align by. Those pairs are listed and left alone. + +The two sides speak different path dialects, since the server records the +container coordinate ([[Jellyfin]]) and the journal speaks the array; a join that +forgets the translation returns zero rows and looks like an empty library. + ## Laws of this arm - **A folder that names two series is not evidence of two series — the files @@ -187,6 +217,16 @@ and the sheet is never fed to the planner. them, per-season counts equal, and no duplicate `SxxEyy` key anywhere prove the layout is already correct. Renumbering files under that evidence buys nothing and can silently mislabel an episode. +- **One tvdb id can carry two `Series` rows, so the id-less folder list proves + nothing about a merge.** A parked folder whose words parse to a registered show + gets identified on its own and keeps its own folder: the count of id-less + folders drops, and the library now offers the same show twice. Prove a merge by + the episode rows standing under the owner folder, never by that list. +- **Move library files only while `Scan Media Library` is Idle.** A rename that + lands under a running scan is a row the server read a moment ago pointing at a + path that no longer exists. `ScheduledTasks` is the read ([[Jellyfin]]); wait + for a `Running` sighting and then two consecutive `Idle` polls, because a gap + between sub-tasks reads as finished. ## See Also