Files
Frank e83162a2ac feat(media): a TV mover that will not file a show the library already has
Two fixes found by one live sweep, shipped together because the second is
what makes the first finishable.

1. scripts/media-tv.py — the other half of media-plan.py's refusal to file
   TV. Groups episode-shaped files by the series name before the marker
   (media-titles.ep_of, imported not restated), writes a plan in the
   executor's own format: <library>/<Series> (<year>)/Season <NN>/<own
   name>.<ext>. Refuses a group below --min, an episode naming no season,
   and an episode the destination library already holds at the same byte
   count — the movie share carries a second copy of all 124 Monk episodes
   the TV library has had for years; a duplicate is Frank's call, never a
   tool's copy. Applied by media-apply.py with its own journal and
   --root /mnt/entropy.

2. scripts/media-apply.py — gone meant "rmdir intended", so a folder whose
   rmdir failed was blacklisted forever and no later sweep could finish it
   (live movie sweep: 4517 targeted, 146 permanently stranded). gone now
   means "actually gone": a failed intent is retried by the next sweep.

tests: 9 new tests in media_relayout.bats, each red against the previous
revision — 8 TV + 1 sweep retry behind a chmod 500 parent. ./tests/run.sh
633 green. No live move run yet.
2026-10-30 20:30:24 -07:00

231 lines
13 KiB
Markdown

---
type: reference
title: Media Relayout
description: "Census, plan and apply the movie library into Jellyfin format: the gates to read, and the rollback."
tags: [dankserver, media-server, jellyfin, plex, procedure]
timestamp: 2026-10-30T00:00:00Z
---
# Media Relayout
The movie library on [[Entropy Volume]] arrived as release folders —
`Some.Movie.2019.1080p.WiKiTaLx.x265-FOO/` with the video inside named after
the release. [[Jellyfin]] matches a movie by folder and file name, so the
release noise cost posters, extras and versions. These scripts turn that
library into Jellyfin's own shape — and move TV back out of it — and nothing
else in the repo touches a media file.
| Script | Touches media? | Output |
|---|---|---|
| `scripts/media-census.py` | reads headers only | `enum`, `probed`, `plex` TSVs |
| `scripts/media-plan.py` | no — TSVs in, TSVs out | the plan + the exception report |
| `scripts/media-apply.py` | **renames, never copies**; sweeps only what a run emptied | the moves, plus a journal |
| `scripts/media-tv.py` | no — TSV out | a plan that moves TV into the TV library |
The seam is `tests/media_relayout.bats`; the target contract is quoted from
[jellyfin.org/docs/general/server/media/movies](https://jellyfin.org/docs/general/server/media/movies)
in `media-plan.py`'s docstring.
## Procedure
Run the census on the machine that mounts the library. `probe` is the slow step
(a header read per container) and is `nice`d — it is safe beside a live service
because it never competes for the GPU and never writes inside the library.
```bash
C=scripts/media-census.py
$C enumerate --root /mnt/entropy/Movies --out /tmp/media-enum.tsv
$C probe --enum /tmp/media-enum.tsv --out /tmp/media-probed.tsv --jobs 6
$C plex --root /mnt/entropy/Movies --out /tmp/plex-movies.tsv
```
`plex` reads the Plex metadata cache (`--db` defaults to the path the
[[Plex]] container bind-mounts at `/var/lib/plexmediaserver`) with
`mode=ro&immutable=1`: a live server is never locked or written.
Then plan — this reads only the three TSVs, so it costs seconds and can run
anywhere, including a laptop:
```bash
scripts/media-plan.py --root /mnt/entropy/Movies \
--enum /tmp/media-enum.tsv --probed /tmp/media-probed.tsv \
--plex /tmp/plex-movies.tsv --out /tmp/jf-plan.tsv --exceptions /tmp/jf-exceptions.txt
```
**Read `/tmp/jf-exceptions.txt` before applying.** It is not a log: every line
is a file the plan could not name a film for, a pile that needs a README, or a
collision it refused. The printed gate lines are the go/no-go:
`problems`, `bad-names`, `occupied`, `double-claim`, `left-unmoved` must all be
0 and `unplanned=0` must appear in the report. Any non-zero number means a file
that would be filed wrong, and the plan is not run until it is understood.
```bash
scripts/media-apply.py apply --plan /tmp/jf-plan.tsv --root /mnt/entropy/Movies # dry, writes nothing
scripts/media-apply.py apply --plan /tmp/jf-plan.tsv --root /mnt/entropy/Movies --apply
scripts/media-apply.py verify --plan /tmp/jf-plan.tsv --journal /mnt/entropy/media-relayout.jsonl
scripts/media-apply.py sweep --journal /mnt/entropy/media-relayout.jsonl --root /mnt/entropy/Movies # dry
scripts/media-apply.py sweep --journal /mnt/entropy/media-relayout.jsonl --root /mnt/entropy/Movies --apply
scripts/media-apply.py rollback --journal /mnt/entropy/media-relayout.jsonl --root /mnt/entropy/Movies --apply
```
An apply also leaves the release folders it emptied standing where they were —
hundreds of them at the library root — which is what `sweep` removes, dry-run
first, from the same journal.
After it lands, both servers need a scan or the change is invisible.
```bash
# Run this from the host shell through `podman exec`, never from a neighbour
# container: Plex's HTTPD closes without a single byte on any request that
# arrives from another container's netns (10.89.x.x reads as "empty reply", not
# as a 401). The token rides stdin, never argv, where any process on the host
# could read it out of /proc/<pid>/cmdline.
umask 077
podman exec plex sh -c "sed -n 's/.*PlexOnlineToken=\"\\([^\"]*\\)\".*/\\1/p' \
'/config/Library/Application Support/Plex Media Server/Preferences.xml'" >/tmp/px.tok
podman exec -i plex sh -c 'read -r T
for k in $(curl -s -H "X-Plex-Token: $T" -H "Accept: application/json" \
http://127.0.0.1:32400/library/sections | grep -o "\"key\":\"[0-9]*\"" | cut -d\" -f4); do
curl -s -o /dev/null -w "plex section $k: %{http_code}\n" -X POST \
-H "X-Plex-Token: $T" "http://127.0.0.1:32400/library/sections/$k/refresh"
done' </tmp/px.tok
```
[[Jellyfin]] has no credential on this box that can trigger a scan: the two keys
in its database are Seerr's, unused since 2026-09-01, and answer 401 to every
call. Mint one (*Dashboard → Advanced → API Keys*, Administrator) and store it in
`~/.config/secrets/jellyfin`, or press *Library → Scan All Libraries* in the web
UI. A scripted scan passes the key in a mode-600 curl config, never in a header
argument:
```bash
umask 077; printf 'header = "X-Emby-Token: %s"\n' "$(cat ~/.config/secrets/jellyfin)" >/tmp/jf.hdr
curl -s -K /tmp/jf.hdr -o /dev/null -w "jellyfin scan: %{http_code}\n" \
-X POST http://127.0.0.1:8096/Library/Refresh
```
## TV in the movie library
A box set that arrives in the movie share stays there until someone moves it:
`media-plan.py` refuses to file TV and parks each series in
`_TV needs a TV library - <series>`. `scripts/media-tv.py` is the mover. It
walks the movie library itself (no census needed), groups episode-shaped names
by series, and writes a plan in the executor's own format:
```bash
scripts/media-tv.py --root /mnt/entropy/Movies --library /mnt/entropy/TVShows \
--out /mnt/entropy/media-tv-plan.tsv
scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv --root /mnt/entropy
scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv --root /mnt/entropy \
--journal /mnt/entropy/media-relayout-tv.jsonl --apply
scripts/media-apply.py sweep --journal /mnt/entropy/media-relayout-tv.jsonl \
--root /mnt/entropy --apply
```
The shape it writes is the one both matchers read — `Monk (2002)/Season 05/
Monk - S05E03 - Mr. Monk and the Big Game.mkv` — keeping the file's own name,
because the publisher's spelling beats a re-derivation, with the year taken
from the box-set folder. What it refuses it prints and leaves alone:
- fewer than `--min` (5) files under one series name: one file named `Part 2`
is a film, not a series;
- an episode whose name states no season number — no invented `Season 01`;
- a destination two files both claim: the executor refuses the whole plan
before its first rename, and only Frank decides which copy is the keeper.
- an episode the destination library already holds at the same byte count. The
movie share carried a second copy of all 124 Monk episodes — the TV library
has had the same box set for years, every episode the same size, ~31 GiB of
it. The mover leaves such a series in place and says so; a duplicate is
deleted or kept by Frank, never copied into one library by a tool.
A sidecar that shares an episode's own name (`.srt`, `.nfo`) moves with it.
## Laws
- **A moved file is a 404 in [[Plex]] until its section is rescanned.** The
database keeps the release-folder path, so playback fails `Couldn't find the
file to stream` while the scanner works through the section. Never read such a
404 as a lost file: the file is where the journal says it is.
- **Plex's matched title outranks the filename**, and a Plex year of `0` means
*indexed but never matched* — trusting it named folders `Title (0)`, which is
a Jellyfin item with no year, no poster and no identity. A year that is not a
plausible year is no information.
- **`<>:"/\|?*` never survive a name.** Plex's own title for *V/H/S/2* is
`V/H/S/2`; kept verbatim it became `Movies/V/H/S/2 (2013)/` — a tree five
levels deep that neither matcher reads as a movie.
- **A folder inside a movie's folder is bonus territory even when its name is
misspelled.** `Saboteur (1942)/Exrtas/Saboteur - A Closer Look` is a typo for
`Extras`; the vocabulary rule never fired, the featurette became a movie
folder, and its words then out-scored the real film and drew that box set's
other extras into it. The rule that replaced it needs no metadata: the folder
holding the longest film owns the shorter video filed under it, unless that
file's own name carries a different year — which is another film speaking.
- **Matching words are split on everything that is not a letter or a digit.**
Splitting on separators but not on `/` left tokens like `(1960)/extras` that
no other name could ever agree with, so a film could not be found by the
folder holding it, and extras drifted into a neighbour. A bare number counts
only when it could be a release year, so `1080` and `2160` never pose as
agreement between two names.
- **A file whose name carries no words is bonus material, not a folder.**
`.Extra 1 ( 1H 27min multisub ).mkv` parses to `1`; a movie folder named `1`
is not a movie. It becomes a review line telling a human to name the file.
- **The executor renames and nothing else.** `rename(2)` cannot cross a
filesystem, so a plan that would is refused rather than silently copying tens
of terabytes.
- **The journal is a write-ahead log: the intent is fsynced before the rename.**
An interrupted run therefore leaves only two states — an intent whose move
happened, and an intent whose move never ran — and the next run finishes the
job without moving anything twice. A file that moved with no journal line did
not move through this tool, and the tool refuses to guess on its behalf.
- **The journal lives BESIDE the library root, never inside it** (default:
`media-relayout.jsonl` next to the root): a log inside the library is a file
Jellyfin indexes.
- **Refusals are all-or-nothing.** An occupied target, a vanished source or a
duplicate destination stops the run *before the first rename*, so the operator
fixes one thing and re-runs instead of untangling a half-applied plan.
`--force` relaxes only the opinion guards; it never permits overwriting.
- **A sweep deletes only what the journal proves the run emptied.** A folder is
eligible when a journalled file moved out of it or out of something under it,
it now holds nothing but empty directories, no journalled destination lives
beneath it, and it is not a folder the run created. A folder nobody moved —
someone's placeholder — cannot be selected at all, and a symlink anywhere
inside disqualifies the subtree, because the thing behind a link is not this
tool's to delete. Every removal is journalled as `rmdir` and `rollback`
recreates the folder, so the sweep is undone by the same command as the moves.
Proof carries downward: inside a folder the journal proves emptied, every
empty directory goes with it. A release's `Screens/` or `Sample/` is never a
move source itself, so 146 of them held their emptied parents on disk after
the first sweep.
A failed `rmdir` is not a removal: only a folder that is really gone leaves
scope, so the next sweep finishes what a permission problem stopped.
- **An episode-shaped name marks a SERIES, not a film — and the series is the
name BEFORE the marker.** `Monk - S05E03 - Mr. Monk and the Big Game` is series
`Monk`. Keyed on the whole title instead, every episode became a group of one
and never reached the five-episode size that calls something a series, so a
box set was filed as 124 movies in 124 folders. The threshold still earns its
keep: a lone film named `Part 2` stays a film. Episodes of a series are parked
in `_TV needs a TV library - <series>` for a human to move to a TV source —
the movie tool never files TV.
- **One journal is one undo.** The TV move runs on its own journal, and with
`--root` set to the share that holds BOTH libraries: the executor refuses any
path outside `--root`, and a journal shared with the movie relayout would
undo both runs when only one of them is wanted.
## Rollback
One command, from the journal of the run:
`media-apply.py rollback --journal …/media-relayout.jsonl --apply`. It recreates
any folder a sweep removed, restores every source path, removes only the folders
that run created *and that are now empty*, and refuses everything if any moved
file cannot be accounted for. A rolled-back journal refuses to roll back twice.
## See Also
[[Jellyfin]], [[Plex]], [[Entropy Volume]], [[Testing Policy]], [[Storage Policy]]
## Citations
- Jellyfin Movie documentation — folder/file naming, extras subfolders,
`Movie (2013) - 216p.mkv` version and `- 4K.mkv` handling:
<https://jellyfin.org/docs/general/server/media/movies>