serverAppConfigs/serverAppConfigs-docs/Dankserver/Media Relayout.md

type, title, description, tags, timestamp

type	title	description	tags	timestamp
reference	Media Relayout	Census, plan and apply the movie, TV, music and audiobook libraries: the gates, the refusals, the rollback.	dankserver media-server jellyfin plex procedure	2026-10-30T00:00:00Z

Media Relayout

The movie library on Entropy Volume arrived as release folders — Some.Movie.2019.1080p.WiKiTaLx.x265-FOO/ with the video inside named after the release. Jellyfin matches a movie by folder and file name, so the release noise cost posters, extras and versions. These scripts turn that library into the shape its media server reads — the movies, the TV packs, the TV that turns up in the movie share, the music library, and the audiobooks, whose only reader is Plex — and nothing else in the repo touches a media file.

Script	Touches media?	Output
scripts/media-census.py	reads headers and the servers' own DBs, never writes	enum, probed, and one authority TSV per library
scripts/media-plan.py	no — TSVs in, TSVs out	the plan + the exception report
scripts/media-apply.py	renames, never copies; sweeps only what a run emptied	the moves, plus a journal
scripts/media-tv.py	no — TSV out	a plan that moves TV into the TV library
scripts/media-shows.py	no — TSV out	a plan that re-files the TV library itself
scripts/media-music.py	no — TSV out	a plan that re-files the music library by its tags
scripts/media-audiobooks.py	no — TSV out	a plan that re-files the audiobooks by the book Plex matched
scripts/media-verify.py	no — reads the servers' DBs read-only	the verdict that every journaled move landed

The seams are tests/media_relayout.bats, tests/media_shows.bats, tests/media_music.bats, tests/media_audiobooks.bats and tests/media_verify.bats; the movie and TV target contracts are quoted from jellyfin.org/docs/general/server/media/movies in media-plan.py's docstring.

Procedure

Run the census on the machine that mounts the library. probe is the slow step (a header read per container) and is niced — it is safe beside a live service because it never competes for the GPU and never writes inside the library.

C=scripts/media-census.py
$C enumerate --root /mnt/entropy/Movies --out /tmp/media-enum.tsv
$C probe     --enum /tmp/media-enum.tsv --out /tmp/media-probed.tsv --jobs 6
$C plex      --root /mnt/entropy/Movies --out /tmp/plex-movies.tsv

plex reads the Plex metadata cache (--db defaults to the path the Plex container bind-mounts at /var/lib/plexmediaserver) with plain mode=ro: a live server is never locked or written, and plain ro is also what makes a scan's uncheckpointed rows visible. The same rule as scripts/media-verify.py — see Proving the move landed.

Then plan — this reads only the three TSVs, so it costs seconds and can run anywhere, including a laptop:

scripts/media-plan.py --root /mnt/entropy/Movies \
    --enum /tmp/media-enum.tsv --probed /tmp/media-probed.tsv \
    --plex /tmp/plex-movies.tsv --out /tmp/jf-plan.tsv --exceptions /tmp/jf-exceptions.txt

Read /tmp/jf-exceptions.txt before applying. It is not a log: every line is a file the plan could not name a film for, a pile that needs a README, or a collision it refused. The printed gate lines are the go/no-go: problems, bad-names, occupied, double-claim, left-unmoved must all be 0 and unplanned=0 must appear in the report. Any non-zero number means a file that would be filed wrong, and the plan is not run until it is understood.

scripts/media-apply.py apply    --plan /tmp/jf-plan.tsv --root /mnt/entropy/Movies   # dry, writes nothing
scripts/media-apply.py apply    --plan /tmp/jf-plan.tsv --root /mnt/entropy/Movies --apply
scripts/media-apply.py verify   --plan /tmp/jf-plan.tsv --journal /mnt/entropy/media-relayout.jsonl
scripts/media-apply.py sweep    --journal /mnt/entropy/media-relayout.jsonl --root /mnt/entropy/Movies   # dry
scripts/media-apply.py sweep    --journal /mnt/entropy/media-relayout.jsonl --root /mnt/entropy/Movies --apply
scripts/media-apply.py rollback --journal /mnt/entropy/media-relayout.jsonl --root /mnt/entropy/Movies --apply

An apply also leaves the release folders it emptied standing where they were — hundreds of them at the library root — which is what sweep removes, dry-run first, from the same journal.

After it lands, both servers need a scan or the change is invisible.

# Run this from the host shell through `podman exec`, never from a neighbour
# container: Plex's HTTPD closes without a single byte on any request that
# arrives from another container's netns (10.89.x.x reads as "empty reply", not
# as a 401). The token rides stdin, never argv, where any process on the host
# could read it out of /proc/<pid>/cmdline.
umask 077
podman exec plex sh -c "sed -n 's/.*PlexOnlineToken=\"\\([^\"]*\\)\".*/\\1/p' \
  '/config/Library/Application Support/Plex Media Server/Preferences.xml'" >/tmp/px.tok
podman exec -i plex sh -c 'read -r T
  for k in $(curl -s -H "X-Plex-Token: $T" -H "Accept: application/json" \
      http://127.0.0.1:32400/library/sections | grep -o "\"key\":\"[0-9]*\"" | cut -d\" -f4); do
    curl -s -o /dev/null -w "plex section $k: %{http_code}\n" -X POST \
      -H "X-Plex-Token: $T" "http://127.0.0.1:32400/library/sections/$k/refresh"
  done' </tmp/px.tok

Jellyfin's scan key lives in ~/.config/secrets/jellyfin (mode 600; the same value is JELLYFIN_API_KEY in the ignored dankserver/.env). It rides a mode-600 curl config, never a header argument, in the MediaBrowser form — Jellyfin 12.4 refuses a bare X-Emby-Token, and the backslashes below matter:

umask 077
{ printf 'header = "Authorization: MediaBrowser Token=\\"%s\\""\n' \
    "$(cat ~/.config/secrets/jellyfin)"
  echo 'url = "http://127.0.0.1:8096/Library/Refresh"'
  echo 'request = "POST"'; } >/tmp/jf.hdr
curl -s -K /tmp/jf.hdr -o /dev/null -w "jellyfin scan: %{http_code}\n"
shred -u /tmp/jf.hdr

204 is acceptance; anything else queued no scan. A 401 from a key that answers 200 on a hand-typed request is this block's quoting, not the server — bash printf collapses \" to ", curl ends the quoted value at that quote, and Jellyfin receives Authorization: MediaBrowser Token= with nothing after it (\\" answers 204, \" answers 401, both run on this host). One accepted refresh is the whole procedure: a second one while a scan runs moves Scan Media Library to Cancelling, and once the runner holds Cancelling every later 204 is accepted and then dropped — the fix is systemctl --user restart jellyfin.service (never deploy.sh: that runs the metadata-view cutover), one POST after health answers, and then believe ScheduledTasks State Running, not the status code.

Proving the move landed

A scan answers 204 whether or not it finds anything, so proof is the journal replayed against both servers' own tables: every move row has a row at its dst, and zero rows still name its src. scripts/media-verify.py is that replay: it opens each database read-only (there is no sqlite3 binary on these hosts or inside either container) and exits non-zero while a row is stale.

Both libraries live in WAL mode, so the tool opens them plain mode=ro: immutable=1 promises sqlite the file never changes, which skips the write-ahead log — a scanner's own uncheckpointed rows then read as missing, and a read beside a running scan can fail as "database disk image is malformed". immutable stays only the fallback for a database whose -shm cannot be read.

scripts/media-verify.py --journal /mnt/entropy/media-relayout-music.jsonl \
    --db-kind jellyfin --ignore-ext jpg,jpeg,png,log,cue,toc,nfo,txt,m3u,accurip

"No row at the destination" is not one verdict but four. never indexed: the server indexes no file of that extension (its own extension census says so) — a sidecar, not a lost film. gone: the file sits at neither src nor dst, because a later pass or the server's own extension sweep renamed it. phantom: a row still naming src while the file has left src. REAL: no row at dst and the file still at src — the only class that is this relayout's own damage. Phantoms are normal for both servers (Plex carries thousands that predate any relayout); re-scan to clear them and never read them as loss.

The owning server decides whether a pass can convict anything: Plex's roots are its sections rows joined to their location rows (Movies, TVShows, Music, Audiobooks, LiveRecording) and Jellyfin holds no audiobook library at all, so the audiobook journal verifies with --db-kind plex or it proves nothing.

Three doors that look right and are not: GET /search?query= matches metadata titles, never file paths; ?path= on /library/sections/N/all makes Plex's HTTPD close without a single byte; and the same HTTPD drops requests that carry no Accept and client-identifier header (see the scan block above for the form that works).

TV in the movie library

A box set that arrives in the movie share stays there until someone moves it: media-plan.py refuses to file TV and parks each series in _TV needs a TV library - <series>. scripts/media-tv.py is the mover. It walks the movie library itself (no census needed), groups episode-shaped names by series, and writes a plan in the executor's own format:

scripts/media-tv.py --root /mnt/entropy/Movies --library /mnt/entropy/TVShows \
    --out /mnt/entropy/media-tv-plan.tsv
scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv --root /mnt/entropy
scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv --root /mnt/entropy \
    --journal /mnt/entropy/media-relayout-tv.jsonl --apply
scripts/media-apply.py sweep --journal /mnt/entropy/media-relayout-tv.jsonl \
    --root /mnt/entropy --apply

The shape it writes is the one both matchers read — Monk (2002)/Season 05/ Monk - S05E03 - Mr. Monk and the Big Game.mkv — keeping the file's own name, because the publisher's spelling beats a re-derivation, with the year taken from the box-set folder. What it refuses it prints and leaves alone:

• fewer than --min (5) files under one series name: one file named Part 2 is a film, not a series;
• an episode whose name states no season number — no invented Season 01;
• a destination two files both claim: the executor refuses the whole plan before its first rename, and only Frank decides which copy is the keeper.
• an episode the destination library already holds at the same byte count. The movie share once carried a second copy of a whole box set — same episode list, every byte count identical. The mover leaves such a series in place and says so; a duplicate is deleted or kept by Frank, never copied into one library by a tool.

A sidecar that shares an episode's own name (.srt, .nfo) moves with it.

Below the threshold the mover prints and leaves, so a lone downloaded episode of a real show stays in the movie library on purpose. Filing one by hand is the same executor on a hand-written plan and the same journal; the trap is the series year. Read it from the file's own name or the folder it owns, never from the title whose bonus folder it happens to be parked in — The Cleveland Show S03E07 inside Die Hard (1988)/extras/ is not a 1988 show. A name whose marker is Part 2 is a film and stays put.

The TV library's own relayout

/mnt/entropy/TVShows is scene packs rather than a library, and Jellyfin has already matched and numbered most of it. scripts/media-shows.py re-files that library into the same shape, and it never guesses: every series name and every episode number it writes traces to an authority, and what has none is parked for a human to read.

# 1. The authority: Jellyfin's own matched rows (the DB opens read-only).
python3 scripts/media-census.py jellyfin --root /mnt/entropy/TVShows \
    --out /tmp/tv_titles.tsv
# 2. What is on disk (stat only), and the runtimes the film arm needs.
python3 scripts/media-census.py enumerate --root /mnt/entropy/TVShows \
    --out /tmp/tv_enum.tsv
python3 scripts/media-census.py probe --root /mnt/entropy/TVShows \
    --enum /tmp/tv_enum.tsv --jobs 6 --out /tmp/tv_probed.tsv
#    The probe is the slow step: ffprobe reads both ends of every file.
# 3. The plan. --tmdb needs TMDB_API_KEY; without it a series no provider dated
#    is simply named without a year, which Jellyfin accepts.
python3 scripts/media-shows.py --root /mnt/entropy/TVShows --enum /tmp/tv_enum.tsv \
    --titles /tmp/tv_titles.tsv --probed /tmp/tv_probed.tsv --out /mnt/entropy/media-relayout-shows-plan.tsv \
    --exceptions /mnt/entropy/media-relayout-shows-exceptions.txt
# 4. Read the exceptions file FIRST — it is the review, not a log. Same executor
#    as above, but on this arm's OWN journal: a shared journal cannot be unwound.
scripts/media-apply.py apply --plan /mnt/entropy/media-relayout-shows-plan.tsv \
    --root /mnt/entropy --journal /mnt/entropy/media-relayout-shows.jsonl --apply

Evidence, strongest first — the order the planner believes things:

• the episode marker the file's own name states (S01E02-E03, 1x04);
• the Season NN folder it sits in and the number its name leads with inside it: Season 19/01 Charles Barkley, Nirvana.avi is S19E01;
• the provider row: identity (the series name and its year), and the numbering of a file whose own name says nothing;
• a bonus folder anywhere above the file, or Jellyfin's ExtraType → the series' canonical extra folder;
• a runtime over SHORT_MAX makes it film-shaped, and it leaves for --movies only when a movie provider names it — otherwise it parks.

Two refusals carry the design, and a dry run on the real library found both:

• Folders resolve a series; they never invent one. A folder's words may resolve a file to a series a provider has already named — (CBB) Tengen Toppa Gurren Lagann - The Complete Series (BD 1080p) names the registered Tengen Toppa Gurren Lagann — but a bonus subfolder is not a show. An early run made …/03. Extra/01. NCOP/NCOP 01.mkv into a root series called 01 NCOP.
• A scraped row does not outrank the filesystem. Jellyfin numbers multi-season packs badly: every file under SNL 16-20/Season 16..19 carries season 16 episode 20, and one Gundam row says season 2004. The folder, then the file's own leading number, win; a row that still insists on season 1989 parks its file instead of creating Season 1989.

The provider's name is copied verbatim, warts included: Jellyfin carries a Series named Black Clover 001 (92 episodes, no year) because it matched a pack off a filename. Renaming a show is Jellyfin's work — a folder rename never touches its provider ids — so the mover keeps the name it is handed and strips only the pack index a release leads with (4a. Seed DESTINY → Seed DESTINY).

The music and audiobook arms

Each is a page of its own, because each reads a different authority and refuses for a different reason: Media Music Relayout files by Jellyfin's track tags, Media Audiobook Relayout by Plex's chapter rows. Both share this page's executor, the one-journal-per-arm law, the sweep and the rollback.

Laws

• A moved file is a 404 in Plex until its section is rescanned. The database keeps the release-folder path, so playback fails Couldn't find the file to stream while the scanner works through the section. Never read such a 404 as a lost file: the file is where the journal says it is.

• Plex's matched title outranks the filename, and a Plex year of 0 means indexed but never matched — trusting it named folders Title (0), which is a Jellyfin item with no year, no poster and no identity. A year that is not a plausible year is no information.

• <>:"/\|?* never survive a name. Plex's own title for V/H/S/2 is V/H/S/2; kept verbatim it became Movies/V/H/S/2 (2013)/ — a tree five levels deep that neither matcher reads as a movie.

• A folder inside a movie's folder is bonus territory even when its name is misspelled. Saboteur (1942)/Exrtas/Saboteur - A Closer Look is a typo for Extras; the vocabulary rule never fired, the featurette became a movie folder, and its words then out-scored the real film and drew that box set's other extras into it. The rule that replaced it needs no metadata: the folder holding the longest film owns the shorter video filed under it, unless that file's own name carries a different year — which is another film speaking.

• Matching words are split on everything that is not a letter or a digit. Splitting on separators but not on / left tokens like (1960)/extras that no other name could ever agree with, so a film could not be found by the folder holding it, and extras drifted into a neighbour. A bare number counts only when it could be a release year, so 1080 and 2160 never pose as agreement between two names.

• Compare two media libraries with the series engine, never a path substring. Filtering paths for monk also returns Monkey Dust, and letting whichever file a walk happens to reach first stand for an episode paired two libraries into a fake duplicate — the same sloppiness pointed the other way deletes a file that is unique. The comparison is media-titles.ep_of on both sides: series, season, episode, bytes, and a hash before anything is trashed.

• A deletion is staged on the item's own filesystem. The trash rule names ~/.AGENTTRASH, but /home and /mnt/entropy are different devices: a 31 GiB stage there is a copy rather than a rename, it has to fit in the 187 GiB of free space, and rename(2) refuses it anyway. Trash goes beside the item — /mnt/entropy/.AGENTTRASH/<date>-<topic>/ — so it costs one inode change and rolls back exactly like the move that made it.

• A Jellyfin 401 is the header, not the key. 12.4 authenticates only Authorization: MediaBrowser Token="…" and refuses a bare X-Emby-Token for every key, valid or not. This page carried "no credential on this box can trigger a scan, the two Seerr keys answer 401" for weeks because of that header — the keys were innocent. Before believing a 401, prove the key against the server's own table: select count(*) from ApiKeys where AccessToken = ? on jellyfin.db opened mode=ro&immutable=1.

• A scan that must see a new folder has to be recursive. POST /Library/Refresh is the incremental pass, and inotify does not cross a rootless podman bind, so folders created on the host stay invisible to it. The move is POST /Items/{id}/Refresh?Recursive=true (204) per library, the id from GET /Library/VirtualFolders — its field is ItemId, not Id — and that same record's RefreshStatus is the progress read.

• A file whose name carries no words is bonus material, not a folder. .Extra 1 ( 1H 27min multisub ).mkv parses to 1; a movie folder named 1 is not a movie. It becomes a review line telling a human to name the file.

• The executor renames and nothing else. rename(2) cannot cross a filesystem, so a plan that would is refused rather than silently copying tens of terabytes.

• The journal is a write-ahead log: the intent is fsynced before the rename. An interrupted run therefore leaves only two states — an intent whose move happened, and an intent whose move never ran — and the next run finishes the job without moving anything twice. A file that moved with no journal line did not move through this tool, and the tool refuses to guess on its behalf.

• The journal lives BESIDE the library root, never inside it (default: media-relayout.jsonl next to the root): a log inside the library is a file Jellyfin indexes.

• Refusals are all-or-nothing. An occupied target, a vanished source or a duplicate destination stops the run before the first rename, so the operator fixes one thing and re-runs instead of untangling a half-applied plan. --force relaxes only the opinion guards; it never permits overwriting.

• A sweep deletes only what the journal proves the run emptied. A folder is eligible when a journalled file moved out of it or out of something under it, it now holds nothing but empty directories, no journalled destination lives beneath it, and it is not a folder the run created. A folder nobody moved — someone's placeholder — cannot be selected at all, and a symlink anywhere inside disqualifies the subtree, because the thing behind a link is not this tool's to delete. Every removal is journalled as rmdir and rollback recreates the folder, so the sweep is undone by the same command as the moves. Proof carries downward: inside a folder the journal proves emptied, every empty directory goes with it. A release's Screens/ or Sample/ is never a move source itself, so it can hold an emptied parent on disk after a sweep. A failed rmdir is not a removal: only a folder that is really gone leaves scope, so the next sweep finishes what a permission problem stopped.

• An episode-shaped name marks a SERIES, not a film — and the series is the name BEFORE the marker. Monk - S05E03 - Mr. Monk and the Big Game is series Monk. Keyed on the whole title instead, every episode became a group of one, never reaching the five-episode size that calls something a series — a whole box set filed as one folder per episode. The threshold still earns its keep: a lone film named Part 2 stays a film. Episodes of a series are parked in _TV needs a TV library - <series> for a human to move to a TV source — the movie tool never files TV.

• One journal is one undo. The TV move runs on its own journal, and with --root set to the share that holds BOTH libraries: the executor refuses any path outside --root, and a journal shared with the movie relayout would undo both runs when only one of them is wanted.

• Sidecar metadata does not follow a rename, because it lives on another filesystem. The media servers write .nfo and artwork through the Media Metadata View (NVMe upper layer); this tool moves bytes on the raw array path, so where that view is mounted a sweep or a move must be followed by scripts/media-view.sh reap --apply — stranded sidecars become ghost folders that both servers browse and report as content. Whether this host mounts the view at all is stated on that page; media-apply.py stays silent when it does not.

• Nothing deletes through the metadata view. A delete there writes an overlayfs whiteout in the upper layer, which hides the array's copy without removing it; the file returns when the view is unmounted. Deletions in this pipeline happen on the raw path only, which is what the journal can undo.

• The music album key is case-folded; the folder name is not. Master Of Puppets and Master of Puppets are one album to the matcher, so the spelling that reaches the folder is the one most tracks carry. Taking it from whichever row sorted first made the folder name depend on row order, and re-planning an already-filed library renamed albums that were already in place.

• An occupied path is never a destination, not even when its occupant is scheduled to leave: the executor refuses an occupied target, so a plan that aims at one fails after the renames that were supposed to clear it. A file standing where the contract puts it is a keep, and a file already carrying a collision suffix keeps that suffix — otherwise the plan is a shuffle, not a fixed point, and a re-run moves albums that are already right.

• media-census.py music reads the BaseItemProviders table, not the item's JSON. This Jellyfin build carries an empty ProviderIds inside BaseItems.Data for albums, so a census that parses the JSON finds no MusicBrainz or Spotify id on any album and reads as "the library is untagged".

Rollback

One command, from the journal of the run: media-apply.py rollback --journal …/media-relayout.jsonl --apply. It recreates any folder a sweep removed, restores every source path, removes only the folders that run created and that are now empty, and refuses everything if any moved file cannot be accounted for. A rolled-back journal refuses to roll back twice.

Roll a chain back newest journal first, or the tool refuses. Each pass re-files what an earlier one moved, so a journal is a clean inverse only until something downstream touches its files; ls -l /mnt/entropy/media-relayout*.jsonl is the order. Refusing is the tool working: on 2026-10-31 the movie journal could not restore files the later TV passes had re-filed, and the stray journal could not restore files Jellyfin's extension sweep had renamed. Roll back what still inverts and leave the rest as the current state — never force it.

See Also

Jellyfin, Plex, Media Metadata View, Entropy Volume, Testing Policy, Storage Policy

Citations

• Jellyfin Movie documentation — folder/file naming, extras subfolders, Movie (2013) - 216p.mkv version and - 4K.mkv handling: https://jellyfin.org/docs/general/server/media/movies