Media Relayout.md hit the vault's 500-line gate the moment the tvdb contract
landed, and the fat in it was not the new text: the Plex and Jellyfin scan
blocks (35 lines) sat inside the relayout procedure even though they are the
way you talk to those servers after ANY change, and two Laws bullets repeated
the same facts a third time. Both blocks now live on [[Plex]] and
[[Jellyfin]]; the relayout page keeps the two laws an operator needs at that
moment — 204 is acceptance and anything else queued nothing, and one accepted
refresh is the whole procedure, because a second moves `Scan Media Library` to
`Cancelling` and every later 204 is discarded.
Same pass, the dead numbers the vault's own law says to destroy: an applied
music pass's date, an applied audiobook pass's move/park counts, and the 146
sweep stragglers of one specific first sweep. The laws they taught stay.
490 lines, `bats tests/docs_completeness.bats` 6/6 (length violation gone,
link integrity clean).
The TV relayout that landed copied Jellyfin's Series names verbatim, and the
naming census showed what that launders in: one Series item per release folder
(1,107 items for 1,107 top-level dirs), the same show spelled six ways, and
box sets living in 346 separate folders for 821 identities. Jellyfin names a
Series after the folder it scanned, so `24 (2001) Season 1` is a folder echo,
not a show name.
The name authority is now tvdb, reached through the id the server matched:
* media-census.py jellyfin emits two more columns — `title` (the episode row's
own Name) and `tvdb` (the series id, read from BaseItemProviders where
Jellyfin 12 keeps provider ids and from the row's Data blob where older
builds do, so the census does not care which shape the live DB has);
* media-shows.py resolves each series to tvdb — the row's tvdb id, else its
TMDb id through TMDb's external ids, else an exact tvdb name search — and
writes tvdb's name and year. That also collapses the provider's spellings
into ONE identity, which is what turns a box set split over five release
folders into one folder and what makes two files claiming one episode MEET
in the duplicate rule;
* the episode file becomes `<Series> (<year>) - SxxEyy - <Title>`: the title
comes from tvdb's episode list, else the row's own title, else nowhere — no
title means no separator, never an invented one.
Two refusals come with it. A row whose series is the hidden top-level folder the
file itself sits in is NOT evidence: the review pile parked inside the library
was indexed as one show named `_unsorted`, and its rows would have renamed real
shows to it. And inside such a folder a file's own words may MATCH a registered
show but never INVENT one — invention is what filled the pile. `--park`
therefore belongs outside the library, which is also what makes the next plan
re-file the pile instead of skipping it.
Both providers answer offline-reproducibly: every answer is cached in one JSON
file, so a plan can be re-planned without a key. Lookups stay behind --tmdb and
--tvdb, and the tvdb token is read from a mode-600 file, never argv.
tests/media_shows.bats is 23 tests, none of which touch the network — the
fixture Jellyfin DB carries the providers table and the name cache is an input
file. ./tests/run.sh: 753 green.
Tonight's sequence, measured: the post-sweep refresh answered 204 while
`Scan Media Library` sat in `Cancelling`, and Jellyfin's stale-path counts froze
for an hour at 0 % CPU — the request was accepted and discarded. Restarting the
user unit cleared the runner; one POST after that reported State `Running` at
130 % CPU and the counts began to move. So the status code is not the evidence,
and the restart is `systemctl --user restart jellyfin.service`, never
`deploy.sh`, which would run the never-tested metadata-view cutover.
Corrects the lesson written an hour ago, which said a non-204 was transient and
should simply be re-issued. It is reproducible and mechanical: `printf` with
`\"` inside the single-quoted format string drops the backslash, curl then ends
the quoted value at the first bare quote, and Jellyfin receives
`Authorization: MediaBrowser Token=` with an empty key. Both forms were run
against the host in one pass — `\\"` answers 204, `\"` answers 401 — so the
block's double backslashes are load-bearing, and the handoff script that deviated
left a 58k-move library unscanned for 40 minutes.
Also the trap that diagnosis walked into: a second accepted refresh while a scan
is running moves `Scan Media Library` to `Cancelling` and the pass starts over.
One accepted request is the procedure; wait it out.
The vault described the overlay as the live shape. It is not: `media-view.sh
status` answers NOT MOUNTED, /var/lib/media-meta does not exist, and the
installed quadlets under ~/.config/containers/systemd still carry the pre-view
binds (/mnt/entropy:/data:ro for Jellyfin, /mnt/entropy:/mnt/entropy:ro for
Plex) while git binds the view with ConditionPathIsMountPoint. The deploy gate
(deploy-remote.sh:ensure_media_view) has therefore never run: the first
`deploy.sh dankserver --service jellyfin|plex` is an untested cutover of both
media servers, and Jellyfin's read-only sidecar error is today's expected state,
not a fault.
Same commit, the scan-acceptance law from tonight's TV handoff: it logged
`jellyfin scan: 401`, so Jellyfin never rescanned a 58k-move library until the
request was re-issued with the identical key and header form (204). Anything
that is not 204 means no scan was queued — re-issue, do not diagnose.
Every apply and rollback printed "next: scripts/media-view.sh reap", but on
Dankserver there is nothing to reap: the metadata view has never been installed
(`media-view.sh status` -> NOT MOUNTED, `/var/lib/media-meta` absent) and the
live quadlets still bind the array read-only. All three relayout arms tonight
issued that order — 242,084 moves — and it pointed at a directory that does not
exist. Where no overlay is mounted the servers bind the array itself, so the
sidecars moved with the media.
`--metadata-view` (default /var/lib/media-meta/view) names the overlay and the
note now rides only when that path is a mountpoint. Red first: two new cases in
tests/media_relayout.bats (apply and rollback); the shared helper names a real
mountpoint so the pre-existing presence assertion keeps its meaning. 37 ok.
A scan answers 204 whether or not it found anything, so the fleet's documented
proof — every move row has a row at its dst and zero rows still name its src —
was a hand-written sqlite query every time anyone needed it. That is the step a
half-finished relayout is judged on, so it is now a command with an exit code.
It encodes the two doors that bite: Jellyfin stores the container-side path
(/data stands for the share) while Plex stores the host path verbatim and hides
deleted scans behind deleted_at on two tables. The report breaks the missing rows
down by extension, because a .cue or .log has no row to land in and must not read
as a lost album; --ignore-ext keeps those out of the verdict.
Two relayouts writing one journal cannot be rolled back apart: the mover
(Movies -> TVShows) and the library re-filer both pointed at
media-relayout-tv.jsonl. The re-filer now writes media-relayout-shows.jsonl,
and its plan/exceptions move off /tmp, which a reboot eats and which is not a
record.
The page said the music plan "was applied" while 62k of its 131k journal rows
were still being written; a runbook line that gets believed is worse than none.
Also lands the single-writer law the audiobook handoff exists to honour, and the
durable plan path (/tmp is not a record).
Plex's artist row for 272 chapters is a chapter number the ripper put in the
artist tag (0.4, 31, 05), which would have filed them under a shelf called 0.4.
A narrator is a person, so it must contain a letter; the same test is deliberately
not applied to a book title, which may honestly be 1984.
The first real plan filed 1,106 chapters from 14 unrelated top-level folders
onto one shelf, because all 1,106 carry the album tag [Unknown Album] and the
same artist tag. Plex's album title is the authority for the shelf only when it
names a book; a placeholder names nothing, and unrelated books share it. Those
chapters now park with the path they have today, timestamp-glued variants
included; Various Artists stays a legitimate multi-cast narrator.
The linter caps a page at 500 lines and a description at 120 chars: the section
lost four lines of prose (no law went with them) and the description lost its
restatement of the format names. gen-vault-index.py regenerated the locator row.
One page owns the relayout facts: the new section carries the Plex-only reader,
the shape, the four commands, and the two laws a future operator would otherwise
re-litigate (never renumber a chapter; the shelf is the narrator). The music
section said its plan was NOT applied — it was applied on 2026-10-31, and a stale
line in a runbook gets believed, so it now names the journal and the rollback.
The audiobook tree is the one library Jellyfin does not read at all and Plex
reads completely: 58,975 chapters matched into Artist/Album/Track, where the
album is the BOOK and the book's single artist row is the NARRATOR. That is the
authority the new planner files by, never the collection folder a pack made.
Audiobooks/<Narrator>/<Book> (<year>)/<the file's own name>
Two laws are the opposite of the music tool's, both forced by the rows: files
are never renumbered (72% of the books carry chapters with no usable index, and
playback order comes from the embedded track tag either way, so an invented
number can only lie), and the shelf is the narrator, not the author (every book
has exactly one artist row; the author survives in 10% of album titles and in no
column Plex stores, so one rule beats a shelf half authors and half narrators).
media-census.py gains the audiobooks export: the chapters Plex matched under a
root, host paths straight through, nothing outside the root and nothing Plex
deleted. 22 tests in tests/media_audiobooks.bats pin the export's join and the
plan's output.
A scan answers 204 whether or not it found anything, so proof is the journal
replayed against Jellyfin's BaseItems.Path and Plex's media_parts.file: every
move has a row at its dst, no row names its src. Records the four wrong doors:
no sqlite3 binary on the hosts or in either container, Jellyfin's single
BaseItems table (no MediaItems), its container-side /data path prefix, and
Plex's HTTPD closing on ?path= / on /search matching titles not paths.
The music library is rips and discography packs whose filenames say nothing the
matcher needs. The authority is Jellyfin's own audio rows: one per track, with
the album and album-artist ids it carries in BaseItemProviders (the item JSON
holds an empty ProviderIds for albums in this build).
Grouping is what the tags assert, not where files sit: (album-artist tag, album,
year), case-folded. A compilation is one album whose tracks name many performers,
so it files once under Various Artists; two album-artist tags on one album name
are two albums. The folder spelling is the dominant one, because a name taken
from whichever row sorted first depends on row order and re-names albums that are
already filed.
An occupied path is never a destination: the executor refuses occupied targets,
so a plan that aims at one fails after the renames meant to clear it. A file
standing on its own target, or already carrying a collision suffix, is a keep -
otherwise the plan shuffles instead of being a fixed point.
One page owns relayout: the census commands, the order the planner believes
things in, the two refusals a dry run on the real library proved necessary, the
apply and rollback path, and the standing wart that a provider name can itself
be scene junk ("Black Clover 001", 92 episodes) — renaming that is Jellyfin's
job, not the mover's.
A dry run of the TV planner against the real 58k-file library produced a root
series called "01 NCOP" out of a bonus subfolder, copied Jellyfin's stale
numbering (every file under SNL 16-20/Season 16..19 carries season 16 episode
20; one Gundam row says season 2004), and lost subtitles a scene pack had
buried in Subs/<release-name>/.
- Folders now RESOLVE a series and never invent one: a folder's words match a
registered name as a token run ("(CBB) Tengen Toppa Gurren Lagann - The
Complete Series (BD 1080p)" names Tengen Toppa Gurren Lagann), and folder
words alone name nothing, so an unknown pack's bonus folder parks instead of
opening a new show at the library root.
- Numbering priority: the marker the file's own name states, then the Season NN
folder plus the number the name leads with inside it, then the provider row.
A row that still insists on season 1989 parks its file rather than creating
Season 1989.
- The provider row keeps identity: the registered name and its year survive a
file whose own marker numbers it, and the pack index a release leads with
("4a. Seed DESTINY") is folded out of the folder name.
- A sidecar searches every ancestor folder for the video it names, not just the
parent, so Subs/<release-name>/<release>.jpn.srt follows its episode.
- The shared bonus-folder test no longer reads an index as a title: name_words
drops "extras" as release junk, so "01. Extras" reduced to "01" and matched
the "01" of "NCOP 01", cancelling the bonus evidence. Only words can name a
film.
The first run against the real library died in the anchor lookup: a subtitle
whose own name carries the media extension matches BOTH prefix forms of its
video, and the "keep the longest" comparison read the destination slot of the
tuple instead of the length slot. The fixture only ever matched one prefix, so
the seam was green over a crash.
The tail the sidecar keeps is now stripped of that container word: Jellyfin and
Plex both read "Show S01E01.forced.srt", not "Show S01E01.mkv.forced.srt", and
an nfo still carries the media extension because that one is constructed, not
parsed.
The Movies relayout planned from Plex; the TV library has no Plex rows and
58k files whose names lie about which episode they hold. Jellyfin has already
matched and numbered 47,676 of them, so the planner treats that numbering as
the authority and the file name as the fallback: an episode number the name
states, or a provider row, or nothing - and "nothing" parks the file under
_unsorted with a reason instead of inventing a number.
- media-census.py jellyfin: one TSV row per matched episode, its container
path mapped back to the host, series name/year/TMDb id from the Series row
(the episode row carries none of them). Both TSV writers now end lines with
\n so a shell consumer sees the last field clean.
- media-shows.py: the plan. Series folders take the provider's own spelling;
one file claiming two episodes keeps both numbers; two files claiming one
episode - the bigger keeps the standard name, the other parks; bonus
material files under the series' extras folder, never as an episode; a long
file inside the TV library leaves for Movies only when TMDb answers with a
matching title, otherwise it parks. Re-running the plan on a library it
already filed moves nothing.
- tests/media_shows.bats: 15 behaviors, including the executor preflight and
the re-plan stability law.
The page carries the deploy-time shape: which four providers are core plugins
in Jellyfin 12, which one is not, the exact XML each key lives in, the host
file the keys come from and its 600 rule, the three-way pin on the TVDB
artifact with its rollback, and what the deployer does when the keys file is
absent. OMDb is recorded as deliberately unconfigured -- keyed by email,
1000 req/day, and a ToS that forbids storing what it returns. index.md's row
carries the same locator.
Jellyfin 12 ships TMDB, TVDB, StudioImages and MusicBrainz as core plugins:
no DLL, only a configuration file under the data root, so the keys have to
reach that XML from somewhere that is not git. They live in the fleet's
per-service env-file convention -- /home/dank/Containers/jellyfin/provider-keys
(TMDB_API_KEY=, TVDB_API_KEY=, optional TVDB_PIN=, mode 600) -- and
scripts/jellyfin-providers.sh plan|install|verify reads only that file, prints
no key byte, and refuses a keys file that is not mode 600 or that has no line
for the provider about to be written. install is idempotent: it never touches
a key the UI already saved, backs up every XML it edits into
~/.AGENTTRASH/<date>-jellyfin-providers/ first, and only installs the TVDB
plugin when the pinned official zip carries the plugin guid.
The deployer runs install BEFORE it restarts jellyfin, because the plugin and
its keys are read at start-up and a restart-first deploy boots the old state.
A missing keys file warns instead of failing: a host without the keys still
deploys, degraded rather than broken. No plugin configuration XML is tracked.
The container has had one owner since the Quadlet cutover
(dankserver/jellyfin/quadlet/jellyfin.container). The block was a second owner,
and wrong on every path: jellyfin/jellyfin:latest instead of the pinned ghcr
image, /opt/jellyfin/* data roots that no longer exist, and a read-only
/mnt/entropy:/data bind that the metadata view has since replaced.
The planner no longer counts jellyfin as a compose service, so editing the
inventory cannot restart a media server; tests/deployment_scoping.bats pins that
contract for both jellyfin and plex.
The merged door was /mnt/media, which read like a third volume and split the host
state across two trees. It is now /var/lib/media-meta/view, so the array layer,
the SSD upper layer, the kernel workdir, the reap trash and the merged tree are
one directory — and the mount table gains no new /mnt entry beside the array.
systemd names a mount unit after its ESCAPED path, so the repo copy keeps the
readable name media-meta-view.mount while `install` writes the host file as
var-lib-media\x2dmeta-view.mount; `plan` prints both and the page records them, so
nobody hunts for a unit that "does not exist". install also creates the mount
point itself, because systemd will not.
The seam now additionally pins that the unit's Where= is exactly the path both
containers bind — guard and bind cannot drift apart. The Quadlet Lifecycle
phantom-path example is corrected too: it named an invented /mnt/media/*, while
the real phantom is the legacy /opt/jellyfin/* that dankserver/compose.yaml still
carries for a service whose roots moved to /home/dank/Containers/jellyfin.
New owner page Dankserver/Media Metadata View.md: the shape, the two binds and
why Plex keeps the /mnt/entropy container path, the commands, the laws (SSD-only
upper layer, never mount over an unmounted array, never delete through the view
because a whiteout only hides the array's copy, reap after any relayout, inotify
does not cross the merge), the failure signatures, and the rollback.
Jellyfin.md: /data is the view, so Local metadata works; the
"Read-only file system : '/data/…/movie.nfo'" line that used to be expected noise
is now the signature of a missing view. Plex.md: same bind, one fact changed.
Media Relayout.md: two new laws for the metadata that does not follow a rename.
AGENTS.md: runbook-index row plus a summary row. Root index regenerated.
A relayout renames on the raw array; the sidecars the media servers wrote live in
the NVMe upper layer of the view and stay at the old coordinates, which is how a
ghost folder (artwork and an .nfo, no video) is born. apply and rollback now
print the follow-up — scripts/media-view.sh reap — at the moment the operator
needs it, and the existing apply seam asserts the hint is in the output.
Both media servers read a Kodi-format sidecar (movie.nfo / tvshow.nfo /
episode.nfo) only when it sits next to the media file, and the library lives on
the RAID6 array, deliberately mounted read-only for every container and slow to
seek for a thousand tiny XML reads. So neither server binds the array any more:
both bind /mnt/media, an OverlayFS merge of the array (lowerdir, never written)
with /var/lib/media-meta on the NVMe (upperdir, where every write lands).
Jellyfin binds the view rw at /data and is the fleet's only metadata writer;
Plex binds it ro at /mnt/entropy, the container path its own database already
recorded, so only the host side of that bind moved and its library paths keep
resolving.
scripts/media-view.sh plans, installs, verifies, reports and reaps the view. It
refuses to mount over an unmounted array, refuses an upper layer on rotational
storage, and refuses a non-root install. deploy-remote.sh raises the view before
it restarts either media server, both units carry
ConditionPathIsMountPoint=/mnt/media so a missing view skips the start instead of
letting a scan clear the library database, and the planner routes the installer
and dankserver/host/** to the jellyfin+plex pair.
tests/media_overlay.bats (11 seams) pins the resolved shape, both refusals, the
two binds and their guard, the deployer's ordering, the planner mapping, and the
reap behaviour against a fixture (ghosts listed, then moved to the SSD trash --
files that exist only above are sidecars, not ghosts, so only directories are
candidates).
Six single-episode files stayed in the movie library when the threshold refused
them; the hand path is the same executor on a hand-written plan and the same
journal. The trap recorded here is the series year -- never inherit it from the
title whose bonus folder an episode happens to be parked in.
The seam ran through `grep -c`, which returned success on a non-zero count and
let a red suite through: description over the 120-char cap, and a timestamp
that check-okf reads against the local date, so a UTC stamp a day ahead of
Seattle is 'in the future'. index.md regenerated for the new description.
docs_completeness 6/6, exit status checked this time.
X-Emby-Token is refused for every key on 12.4 -- which is why this page
carried the false claim that no credential on the box could trigger a scan and
that the two Seerr keys were dead. The documented snippet now authenticates
with Authorization: MediaBrowser Token=..., reads 204 from /Library/Refresh,
and points at ~/.config/secrets/jellyfin (mirrored as JELLYFIN_API_KEY in the
ignored dankserver/.env). Law: prove a 401 against the ApiKeys table before
blaming the key.
Trashing 31 GiB of duplicate Monk to ~/.AGENTTRASH would have been a
cross-device copy of the thing being deleted, into 187 GiB of free space, on a
volume at 100% -- and rename(2) refuses it regardless. The stage belongs beside
the item.
The destination-hold refusal's evidence, stated exactly: 124 episodes both
sides, same episode list, every byte count identical, 31.5 GiB each way,
head+tail blake2b identical on a 6-episode sample.
Plus the law this came out of: a path-substring filter for a series name also
returns other shows (monk matches Monkey Dust), and whichever file a walk
reaches first standing in for an episode turns two libraries into a fake
duplicate — the same sloppiness aimed the other way deletes a unique file.
ep_of on both sides, and a hash before anything is trashed.
Two fixes found by one live sweep, shipped together because the second is
what makes the first finishable.
1. scripts/media-tv.py — the other half of media-plan.py's refusal to file
TV. Groups episode-shaped files by the series name before the marker
(media-titles.ep_of, imported not restated), writes a plan in the
executor's own format: <library>/<Series> (<year>)/Season <NN>/<own
name>.<ext>. Refuses a group below --min, an episode naming no season,
and an episode the destination library already holds at the same byte
count — the movie share carries a second copy of all 124 Monk episodes
the TV library has had for years; a duplicate is Frank's call, never a
tool's copy. Applied by media-apply.py with its own journal and
--root /mnt/entropy.
2. scripts/media-apply.py — gone meant "rmdir intended", so a folder whose
rmdir failed was blacklisted forever and no later sweep could finish it
(live movie sweep: 4517 targeted, 146 permanently stranded). gone now
means "actually gone": a failed intent is retried by the next sweep.
tests: 9 new tests in media_relayout.bats, each red against the previous
revision — 8 TV + 1 sweep retry behind a chmod 500 parent. ./tests/run.sh
633 green. No live move run yet.
The live sweep removed 2271 husks and left 146 standing, every one a release
folder blocked by a junk subfolder (Screens/, Sample/, a BDMV leaf) that no
file ever moved out of. Eligibility was per-folder, so the blocker was never a
target and its parent could never be removed.
Proof now carries downward: inside a folder the journal shows emptied, every
empty directory qualifies. Nothing under such a folder can hold data, and every
removal is still journalled as rmdir and undone by rollback.
The new test is red against the previous revision, green now. The live sweep
touched no file: 8193 before, 8193 after.
Keyed on the whole episode title, every episode of a box set became a group of
one and never reached the five-episode size that calls something a series, so
the live run filed 124 Monk episodes as 124 movie folders, each named after its
own episode. ep_of now returns the series — the name before SxxEyy / 1x04 /
Part N — and the planner groups and names the quarantine pile by that series,
so one show is one pile: 'Monk - S05E03 - Mr. Monk and the Big Game' is series
monk. The threshold keeps its job: a lone film named Part 2 stays a film.
Re-planning the live census with this moves nothing differently except the 124
Monk files, which now join the TV pile: 168 episode files across 3 series.
An apply leaves every release folder it emptied standing where it was: the
live library run left 1675 husks at the root. sweep takes the same journal
and deletes only folders it can prove this run emptied - a journalled move
out of the folder or something under it, nothing but empty directories left,
no journalled destination beneath it, not a folder the run created. A folder
nobody moved cannot be selected at all, so a hand-made placeholder survives,
and a symlink anywhere inside disqualifies the subtree. Every rmdir is
journalled, so rollback recreates what a sweep removed.
The movie library on the entropy volume is release-folded: the video inside
`Some.Movie.2019.1080p…-FOO/` is named after the release, and Jellyfin matches
a movie by folder and file name, so the noise cost posters, extras and versions.
Three scripts own the fix, and nothing else in the repo moves a media file.
* media-census.py the read-only inputs: a stat pass, an ffprobe header pass
(nice'd, so it is safe beside a live endpoint), and the
Plex metadata cache read that outranks any filename guess
* media-plan.py the Jellyfin-format planner: one folder per movie, the
video named exactly like the folder, extras in the
canonical subfolders, copies of one film as versions. It
opens no media file, so a whole-library plan costs seconds,
and it ends in gates — bad-names, occupied, double-claim,
left-unmoved, unplanned — that must all read zero before a
human points the executor at 24 TB
* media-apply.py the executor: rename(2) only, the intent fsynced to a
write-ahead journal BEFORE the move it authorises, so an
interrupted run is finishable and every run is undoable.
Refusals are all-or-nothing and it never overwrites a file
Laws the fixture pins (tests/media_relayout.bats, 16 tests, each proven red
against a reverted copy of the rule it guards):
* a Plex year of 0 means "indexed, never matched" — trusting it produced
folders "Title (0)": items with no year, no poster, no identity
* a Plex title carrying "/" is sanitised, or "V/H/S/2" became a five-level
tree neither matcher reads as a movie
* a folder inside a movie's folder is bonus territory even when its name is
misspelled ("Exrtas"), unless the file's own name carries another year
* matching words split on everything but a letter or digit — the earlier
tokenizer left "(1960)/extras" glued, so a film could not be found by the
folder holding it and one box set's featurettes drifted into a neighbour
* a name with no words in it is bonus material, never a movie folder named 1
Docs: the runbook page owns the procedure, the gates and the rollback; AGENTS.md
links it from the runbook index and the summary table.
Frank
dank
dank