dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(media): what a verification verdict actually means, and what a rollback can undo

Browse Source 
Three things the 2026-10-31 five-library relayout taught, written where the
next operator reads them:

- A verify run must open the servers' databases plain `mode=ro`. Both keep
  their libraries in WAL mode; `immutable=1` skips the write-ahead log, so a
  scanner's own committed rows read as missing and the read can fail as
  "database disk image is malformed". The census page claimed immutable — now
  it says what the code does.
- "No row at the destination" splits into four classes, and only REAL (no row
  at dst AND the file still at src) is the relayout's own damage. Phantom rows
  naming a pre-move path are normal behaviour for both servers. And the owning
  server decides whether a pass can convict: Jellyfin holds no audiobook
  library, so that journal verifies against Plex or it proves nothing.
- A chain of passes rolls back newest journal first, or the tool refuses; the
  refusals measured that day are the tool working, not failing.

The music and audiobook arms moved to their own pages (progressive disclosure;
the parent drops 175 lines and stays the owner of the shared gates, the
executor, the journal law and the rollback). Stale per-run counts destroyed:
episode counts, byte sizes, move tallies for a pass that already happened.
This commit is contained in:
Frank committed 2026-10-31 10:13:11 -07:00
1 parent 5dbbfcb8a0
commit 7ee831adee
4 files changed
+190 -129
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

serverAppConfigs-docs/Dankserver/Media Audiobook Relayout.md
+74
View File
@@ -0,0 +1,74 @@
---
type: reference
title: Media Audiobook Relayout
description: "Re-filing /mnt/entropy/Audiobooks by Plex's chapter rows: narrator shelves, no renumbering, the refusals."
tags: [dankserver, media-server, plex, media-relayout]
timestamp: 2026-10-31T00:00:00Z
---
# Media Audiobook Relayout
`/mnt/entropy/Audiobooks` is the one library here that [[Jellyfin]] does not read
at all — its music scanner has no audiobook concept — while [[Plex]] indexes all
of it as Artist → Album → Track: the album is the BOOK, that book's single artist
is the NARRATOR, both matched out of the files' own tags. Today one book is split
across `… [Part 1]`, `… [Part 2]` folders three or four deep inside a collection
pack, so a scan serves three quarters of a novel; `scripts/media-audiobooks.py`
re-files the tree so the property both servers' book scanners assume is true —
Audiobooks/<Narrator>/<Book> (<year>)/<the file keeps its own name>
The shared gates, the executor, the journal law and the rollback live in
[[Media Relayout]]; this page is the audiobook arm only.
```bash
# 1. The authority: Plex's chapter rows for this root (the DB opens read-only).
python3 scripts/media-census.py audiobooks --root /mnt/entropy/Audiobooks \
--out /tmp/ab_rows.tsv
# 2. What is on disk (stat only).
python3 scripts/media-census.py enumerate --root /mnt/entropy/Audiobooks \
--out /tmp/ab_enum.tsv
# 3. The plan.
python3 scripts/media-audiobooks.py --root /mnt/entropy/Audiobooks \
--enum /tmp/ab_enum.tsv --rows /tmp/ab_rows.tsv --out /tmp/ab_plan.tsv \
--exceptions /tmp/ab_exceptions.txt
# 4. Read the exceptions file FIRST. Then the same executor, journal and rollback
# as [[Media Relayout]] shows, on its own journal, and refresh the Plex section
# as in step 5 of the movie Procedure — from inside the container, never over
# the host port.
scripts/media-apply.py apply --plan /tmp/ab_plan.tsv --root /mnt/entropy \
--journal /mnt/entropy/media-relayout-audiobooks.jsonl --apply
```
The group is the identity Plex asserts in its own database — **(narrator, book,
year)** — so one book arriving in four part folders becomes one folder, and a
1965 and a 2020 recording that share a title stay two books.
Two laws are forced by what the rows actually hold:
+- **A chapter is never renumbered.** Most books contain chapters whose index Plex
does not know, so a number written into a name would be invented — and it would
buy nothing, because both servers order playback by the embedded track tag, not
by the file name. A wrong number in a name is the number a human later trusts.
Fix order in the tags: a Plex scan re-reads tags and never renames a file.
+- **The shelf is the narrator, not the author.** Every book has exactly one artist
row and it is the narrator; the author survives only inside some album titles
(`Kryptos by Graham McNeill`) and in no column Plex stores, so deriving it would
mean parsing titles where a ripper wrote one and inventing it everywhere else.
The author stays visible in the folder name Plex itself shows.
What it refuses, the music arm's list with one addition: a video container, audio
no chapter row names, and a loose file in a folder holding two books all park under
`_unsorted` with the path they have today — and so does a chapter whose row names
**no book or no narrator**: a placeholder (`[Unknown Album]`, `[Unknown Artist]`)
that unrelated books share, or a chapter number mistagged as the artist (`0.4`).
`Various Artists` is not a placeholder — that is a multi-cast read.
The reviewed plan lives at `/mnt/entropy/media-relayout-audiobooks-plan.tsv`; it
applies on its own journal, behind the music writer, and then refreshes the Plex
section. Verify it with `--db-kind plex` — Jellyfin holds no rows here, so a
Jellyfin pass proves nothing either way.
## See Also
[[Media Relayout]], [[Media Music Relayout]], [[Plex]]
serverAppConfigs-docs/Dankserver/Media Music Relayout.md
+68
View File
@@ -0,0 +1,68 @@
---
type: reference
title: Media Music Relayout
description: "Re-filing /mnt/entropy/Music by Jellyfin's tag rows: the census, the album key, the refusals."
tags: [dankserver, media-server, jellyfin, media-relayout]
timestamp: 2026-10-31T00:00:00Z
---
# Media Music Relayout
`/mnt/entropy/Music` is rips and discography packs, and Jellyfin has matched and
tagged almost all of it. `scripts/media-music.py` re-files it into the shape
Jellyfin's music scanner reads —
Music/<album artist>/<Album> (<year>)/NN - <Title>.ext
— and the authority is the tag row of the track, never the name the ripper gave
the file. The shared gates, the executor, the journal law and the rollback live in
[[Media Relayout]]; this page is the music arm only.
```bash
# 1. The authority: Jellyfin's audio rows, one per track, with the album and
# artist ids its own database carries (the DB opens read-only).
python3 scripts/media-census.py music --root /mnt/entropy/Music \
--out /tmp/music_rows.tsv
# 2. What is on disk (stat only).
python3 scripts/media-census.py enumerate --root /mnt/entropy/Music \
--out /tmp/music_enum.tsv
# 3. The plan.
python3 scripts/media-music.py --root /mnt/entropy/Music --enum /tmp/music_enum.tsv \
--rows /tmp/music_rows.tsv --out /tmp/music_plan.tsv \
--exceptions /tmp/music_exceptions.txt
# 4. Read the exceptions file FIRST — it is the review, not a log. Then the same
# executor, journal and rollback as [[Media Relayout]] shows, on its own journal.
scripts/media-apply.py apply --plan /tmp/music_plan.tsv --root /mnt/entropy \
--journal /mnt/entropy/media-relayout-music.jsonl --apply
```
The group is what the tags assert, not where the files sit: **(album-artist tag,
album, year)**. One album arrives as three disc folders and as two copies of one
rip, and one MusicBrainz release group is one album. A compilation is one album
whose tracks name many performers, so it files once under `Various Artists`:
keying on the performer would file one album once per artist. Two different
album-artist tags on one album name are two albums, which is what Jellyfin's two
album items say.
What it refuses:
+- **A sidecar is placed by the audio around it, never by its own name.** A
`.lrc`, `.txt` or picture that shares a track's stem follows that track and
takes the name the track now has, or Jellyfin cannot find it; anything else
goes to the album its own folder was placed in, or to the folder whose discs
sit one level below it. A folder whose audio belongs to two albums parks its
sidecar rather than guessing.
+- **An image album keeps its name.** A folder holding one audio file and a
`.cue` is a whole album in one file; numbering it `01 - …` tells Jellyfin to
index it as one track.
+- **A video container parks**, because Jellyfin will not index it as music and a
rename to `.m4a` would claim it is; **audio no provider row names parks** too,
because unmatched audio is a tagging job, not a renaming job.
ONE writer at a time on this array: two rename passes race, and neither journal
tells. Verify the run against Jellyfin's own tables — see *Proving the move
landed* in [[Media Relayout]].
## See Also
[[Media Relayout]], [[Jellyfin]], [[Media Audiobook Relayout]]
serverAppConfigs-docs/Dankserver/Media Relayout.md
+46 -129
View File
@@ -47,8 +47,10 @@ $C plex --root /mnt/entropy/Movies --out /tmp/plex-movies.tsv
```
`plex` reads the Plex metadata cache (`--db` defaults to the path the
[[Plex]] container bind-mounts at `/var/lib/plexmediaserver`) with
`mode=ro&immutable=1`: a live server is never locked or written.
[[Plex]] container bind-mounts at `/var/lib/plexmediaserver`) with plain
`mode=ro`: a live server is never locked or written, and plain ro is also what
makes a scan's uncheckpointed rows visible. The same rule as
`scripts/media-verify.py` — see *Proving the move landed*.
Then plan — this reads only the three TSVs, so it costs seconds and can run
anywhere, including a laptop:
@@ -123,14 +125,30 @@ replayed against both servers' own tables: every `move` row has a row at its
replay: it opens each database read-only (there is no `sqlite3` binary on these
hosts or inside either container) and exits non-zero while a row is stale.
Both libraries live in WAL mode, so the tool opens them plain `mode=ro`:
`immutable=1` promises sqlite the file never changes, which skips the
write-ahead log — a scanner's own uncheckpointed rows then read as missing, and
a read beside a running scan can fail as "database disk image is malformed".
`immutable` stays only the fallback for a database whose `-shm` cannot be read.
```bash
scripts/media-verify.py --journal /mnt/entropy/media-relayout-music.jsonl \
--db-kind jellyfin --ignore-ext jpg,jpeg,png,log,cue,toc,nfo,txt,m3u,accurip
```
Ignore only what a server cannot index. A missing `.flac` or `.mkv` is the
fault; a row still naming its `src` means the scan has not followed the rename
yet, so re-run the check after the scan, never before.
"No row at the destination" is not one verdict but four. **never indexed**: the
server indexes no file of that extension (its own extension census says so) — a
sidecar, not a lost film. **gone**: the file sits at neither src nor dst, because
a later pass or the server's own extension sweep renamed it. **phantom**: a row
still naming src while the file has left src. **REAL**: no row at dst *and* the
file still at src — the only class that is this relayout's own damage. Phantoms
are normal for both servers (Plex carries thousands that predate any relayout);
re-scan to clear them and never read them as loss.
The owning server decides whether a pass can convict anything: Plex's roots are
its `sections` rows joined to their location rows (`Movies`, `TVShows`, `Music`,
`Audiobooks`, `LiveRecording`) and Jellyfin holds no audiobook library at all, so
the audiobook journal verifies with `--db-kind plex` or it proves nothing.
Three doors that look right and are not: `GET /search?query=` matches metadata
titles, never file paths; `?path=` on `/library/sections/N/all` makes Plex's
@@ -167,16 +185,15 @@ from the box-set folder. What it refuses it prints and leaves alone:
- a destination two files both claim: the executor refuses the whole plan
before its first rename, and only Frank decides which copy is the keeper.
- an episode the destination library already holds at the same byte count. The
movie share carried a second copy of the whole Monk box set — 124 episodes
both sides, the same episode list, every byte count identical, 31.5 GiB each
way, and head+tail blake2b identical on a 6-episode sample. The mover leaves
such a series in place and says so; a duplicate is deleted or kept by Frank,
never copied into one library by a tool.
movie share once carried a second copy of a whole box set — same episode list,
every byte count identical. The mover leaves such a series in place and says
so; a duplicate is deleted or kept by Frank, never copied into one library by
a tool.
A sidecar that shares an episode's own name (`.srt`, `.nfo`) moves with it.
Below the threshold the mover prints and leaves, so a lone downloaded episode of
a real show stays in the movie library on purpose — six did on 2026-10-30.
a real show stays in the movie library on purpose.
Filing one by hand is the same executor on a hand-written plan and the same
journal; the trap is the series year. Read it from the file's own name or the
folder it owns, never from the title whose bonus folder it happens to be parked
@@ -243,119 +260,12 @@ filename. Renaming a show is Jellyfin's work — a folder rename never touches i
provider ids — so the mover keeps the name it is handed and strips only the pack
index a release leads with (`4a. Seed DESTINY` → `Seed DESTINY`).
## The music library's own relayout
## The music and audiobook arms
`/mnt/entropy/Music` is rips and discography packs, and Jellyfin has matched and
tagged almost all of it. `scripts/media-music.py` re-files it into the shape
Jellyfin's music scanner reads —
Music/<album artist>/<Album> (<year>)/NN - <Title>.ext
— and the authority is the tag row of the track, never the name the ripper gave
the file.
```bash
# 1. The authority: Jellyfin's audio rows, one per track, with the album and
# artist ids its own database carries (the DB opens read-only).
python3 scripts/media-census.py music --root /mnt/entropy/Music \
--out /tmp/music_rows.tsv
# 2. What is on disk (stat only).
python3 scripts/media-census.py enumerate --root /mnt/entropy/Music \
--out /tmp/music_enum.tsv
# 3. The plan.
python3 scripts/media-music.py --root /mnt/entropy/Music --enum /tmp/music_enum.tsv \
--rows /tmp/music_rows.tsv --out /tmp/music_plan.tsv \
--exceptions /tmp/music_exceptions.txt
# 4. Read the exceptions file FIRST — it is the review, not a log. Then the same
# executor, journal and rollback as above, on its own journal.
scripts/media-apply.py apply --plan /tmp/music_plan.tsv --root /mnt/entropy \
--journal /mnt/entropy/media-relayout-music.jsonl --apply
```
The group is what the tags assert, not where the files sit: **(album-artist tag,
album, year)**. One album arrives as three disc folders and as two copies of one
rip, and one MusicBrainz release group is one album. A compilation is one album
whose tracks name many performers, so it files once under `Various Artists`:
keying on the performer would file one album once per artist. Two different
album-artist tags on one album name are two albums, which is what Jellyfin's two
album items say.
What it refuses:
+- **A sidecar is placed by the audio around it, never by its own name.** A
`.lrc`, `.txt` or picture that shares a track's stem follows that track and
takes the name the track now has, or Jellyfin cannot find it; anything else
goes to the album its own folder was placed in, or to the folder whose discs
sit one level below it. A folder whose audio belongs to two albums parks its
sidecar rather than guessing.
+- **An image album keeps its name.** A folder holding one audio file and a
`.cue` is a whole album in one file; numbering it `01 - …` tells Jellyfin to
index it as one track.
+- **A video container parks**, because Jellyfin will not index it as music and a
rename to `.m4a` would claim it is; **audio no provider row names parks** too,
because unmatched audio is a tagging job, not a renaming job.
The plan applies on 2026-10-31 through `/mnt/entropy/media-relayout-music.jsonl`;
its rollback is that journal run through `media-apply.py rollback --apply`. ONE
writer at a time on this array: two rename passes race, and neither journal tells.
## The audiobook library's own relayout
`/mnt/entropy/Audiobooks` is the one library here that [[Jellyfin]] does not read
at all — its music scanner has no audiobook concept — while [[Plex]] indexes all
of it as Artist → Album → Track: the album is the BOOK, that book's single artist
is the NARRATOR, both matched out of the files' own tags. Today one book is split
across `… [Part 1]`, `… [Part 2]` folders three or four deep inside a collection
pack, so a scan serves three quarters of a novel; `scripts/media-audiobooks.py`
re-files the tree so the property both servers' book scanners assume is true —
Audiobooks/<Narrator>/<Book> (<year>)/<the file keeps its own name>
```bash
# 1. The authority: Plex's chapter rows for this root (the DB opens read-only).
python3 scripts/media-census.py audiobooks --root /mnt/entropy/Audiobooks \
--out /tmp/ab_rows.tsv
# 2. What is on disk (stat only).
python3 scripts/media-census.py enumerate --root /mnt/entropy/Audiobooks \
--out /tmp/ab_enum.tsv
# 3. The plan.
python3 scripts/media-audiobooks.py --root /mnt/entropy/Audiobooks \
--enum /tmp/ab_enum.tsv --rows /tmp/ab_rows.tsv --out /tmp/ab_plan.tsv \
--exceptions /tmp/ab_exceptions.txt
# 4. Read the exceptions file FIRST. Then the same executor, journal and rollback
# as above, on its own journal, and refresh the Plex section as in step 5 of
# the movie Procedure — from inside the container, never over the host port.
scripts/media-apply.py apply --plan /tmp/ab_plan.tsv --root /mnt/entropy \
--journal /mnt/entropy/media-relayout-audiobooks.jsonl --apply
```
The group is the identity Plex asserts in its own database — **(narrator, book,
year)** — so one book arriving in four part folders becomes one folder, and a
1965 and a 2020 recording that share a title stay two books.
Two laws are forced by what the rows actually hold:
+- **A chapter is never renumbered.** Most books contain chapters whose index Plex
does not know, so a number written into a name would be invented — and it would
buy nothing, because both servers order playback by the embedded track tag, not
by the file name. A wrong number in a name is the number a human later trusts.
Fix order in the tags: a Plex scan re-reads tags and never renames a file.
+- **The shelf is the narrator, not the author.** Every book has exactly one artist
row and it is the narrator; the author survives only inside some album titles
(`Kryptos by Graham McNeill`) and in no column Plex stores, so deriving it would
mean parsing titles where a ripper wrote one and inventing it everywhere else.
The author stays visible in the folder name Plex itself shows.
What it refuses, the music arm's list with one addition: a video container, audio
no chapter row names, and a loose file in a folder holding two books all park under
`_unsorted` with the path they have today — and so does a chapter whose row names
**no book or no narrator**: a placeholder (`[Unknown Album]`, `[Unknown Artist]`)
that unrelated books share, or a chapter number mistagged as the artist (`0.4`).
`Various Artists` is not a placeholder — that is a multi-cast read.
The plan stands reviewed at `/mnt/entropy/media-relayout-audiobooks-plan.tsv`
(55,114 moves, 7,764 parked, executor preflight refused nothing); it applies on its
own journal, behind the music writer, and then refreshes the Plex section.
Each is a page of its own, because each reads a different authority and refuses
for a different reason: [[Media Music Relayout]] files by Jellyfin's track tags,
[[Media Audiobook Relayout]] by Plex's chapter rows. Both share this page's
executor, the one-journal-per-arm law, the sweep and the rollback.
## Laws
@@ -437,16 +347,15 @@ own journal, behind the music writer, and then refreshes the Plex section.
recreates the folder, so the sweep is undone by the same command as the moves.
Proof carries downward: inside a folder the journal proves emptied, every
empty directory goes with it. A release's `Screens/` or `Sample/` is never a
move source itself, so 146 of them held their emptied parents on disk after
the first sweep.
move source itself, so it can hold an emptied parent on disk after a sweep.
A failed `rmdir` is not a removal: only a folder that is really gone leaves
scope, so the next sweep finishes what a permission problem stopped.
- **An episode-shaped name marks a SERIES, not a film — and the series is the
name BEFORE the marker.** `Monk - S05E03 - Mr. Monk and the Big Game` is series
`Monk`. Keyed on the whole title instead, every episode became a group of one
and never reached the five-episode size that calls something a series, so a
box set was filed as 124 movies in 124 folders. The threshold still earns its
keep: a lone film named `Part 2` stays a film. Episodes of a series are parked
`Monk`. Keyed on the whole title instead, every episode became a group of one,
never reaching the five-episode size that calls something a series — a whole
box set filed as one folder per episode. The threshold still earns its keep:
a lone film named `Part 2` stays a film. Episodes of a series are parked
in `_TV needs a TV library - <series>` for a human to move to a TV source —
the movie tool never files TV.
- **One journal is one undo.** The TV move runs on its own journal, and with
@@ -489,6 +398,14 @@ any folder a sweep removed, restores every source path, removes only the folders
that run created *and that are now empty*, and refuses everything if any moved
file cannot be accounted for. A rolled-back journal refuses to roll back twice.
**Roll a chain back newest journal first, or the tool refuses.** Each pass
re-files what an earlier one moved, so a journal is a clean inverse only until
something downstream touches its files; `ls -l /mnt/entropy/media-relayout*.jsonl`
is the order. Refusing is the tool working: on 2026-10-31 the movie journal could
not restore files the later TV passes had re-filed, and the stray journal could
not restore files Jellyfin's extension sweep had renamed. Roll back what still
inverts and leave the rest as the current state — never force it.
## See Also
[[Jellyfin]], [[Plex]], [[Media Metadata View]], [[Entropy Volume]], [[Testing Policy]], [[Storage Policy]]