Compare commits

..
Author SHA1 Message Date
dank 96944f0b17 feat(media): media-verify.py replays a journal against the servers' own tables
A scan answers 204 whether or not it found anything, so the fleet's documented
proof — every move row has a row at its dst and zero rows still name its src —
was a hand-written sqlite query every time anyone needed it. That is the step a
half-finished relayout is judged on, so it is now a command with an exit code.

It encodes the two doors that bite: Jellyfin stores the container-side path
(/data stands for the share) while Plex stores the host path verbatim and hides
deleted scans behind deleted_at on two tables. The report breaks the missing rows
down by extension, because a .cue or .log has no row to land in and must not read
as a lost album; --ignore-ext keeps those out of the verdict.
2026-10-31 01:02:48 -07:00
dank 33f97cd3b3 docs(media): the TV library arm gets its own journal, and its plan lives on the array
Two relayouts writing one journal cannot be rolled back apart: the mover
(Movies -> TVShows) and the library re-filer both pointed at
media-relayout-tv.jsonl. The re-filer now writes media-relayout-shows.jsonl,
and its plan/exceptions move off /tmp, which a reboot eats and which is not a
record.
2026-10-31 00:55:32 -07:00
dank 254fb41fd0 docs(media): the music relayout is mid-flight, and the array takes one writer
The page said the music plan "was applied" while 62k of its 131k journal rows
were still being written; a runbook line that gets believed is worse than none.
Also lands the single-writer law the audiobook handoff exists to honour, and the
durable plan path (/tmp is not a record).
2026-10-31 00:40:30 -07:00
dank 3613f69593 Merge pull request 'feat(media): the audiobook relayout, filed by the book Plex matched' (#101) from feat/media-audiobook-relayout into main 2026-10-31 00:38:02 -07:00
dank 6bf70c6a64 fix(media): a shelf needs a name, so a mistagged chapter number parks
Plex's artist row for 272 chapters is a chapter number the ripper put in the
artist tag (0.4, 31, 05), which would have filed them under a shelf called 0.4.
A narrator is a person, so it must contain a letter; the same test is deliberately
not applied to a book title, which may honestly be 1984.
2026-10-31 00:34:40 -07:00
dank f1b8b60817 fix(media): a tag-scan placeholder is not a book identity
The first real plan filed 1,106 chapters from 14 unrelated top-level folders
onto one shelf, because all 1,106 carry the album tag [Unknown Album] and the
same artist tag. Plex's album title is the authority for the shelf only when it
names a book; a placeholder names nothing, and unrelated books share it. Those
chapters now park with the path they have today, timestamp-glued variants
included; Various Artists stays a legitimate multi-cast narrator.
2026-10-31 00:31:30 -07:00
dank 4ae2bde9ab docs(media): fit the audiobook section inside the OKF caps, refresh the index
The linter caps a page at 500 lines and a description at 120 chars: the section
lost four lines of prose (no law went with them) and the description lost its
restatement of the format names. gen-vault-index.py regenerated the locator row.
2026-10-31 00:25:58 -07:00
dank ceb5617a42 docs(media): the audiobook arm, and the music plan's applied state
One page owns the relayout facts: the new section carries the Plex-only reader,
the shape, the four commands, and the two laws a future operator would otherwise
re-litigate (never renumber a chapter; the shelf is the narrator). The music
section said its plan was NOT applied — it was applied on 2026-10-31, and a stale
line in a runbook gets believed, so it now names the journal and the rollback.
2026-10-31 00:24:50 -07:00
dank a7ceda95fd feat(media): the audiobook relayout, filed by the book Plex matched
The audiobook tree is the one library Jellyfin does not read at all and Plex
reads completely: 58,975 chapters matched into Artist/Album/Track, where the
album is the BOOK and the book's single artist row is the NARRATOR. That is the
authority the new planner files by, never the collection folder a pack made.

  Audiobooks/<Narrator>/<Book> (<year>)/<the file's own name>

Two laws are the opposite of the music tool's, both forced by the rows: files
are never renumbered (72% of the books carry chapters with no usable index, and
playback order comes from the embedded track tag either way, so an invented
number can only lie), and the shelf is the narrator, not the author (every book
has exactly one artist row; the author survives in 10% of album titles and in no
column Plex stores, so one rule beats a shelf half authors and half narrators).

media-census.py gains the audiobooks export: the chapters Plex matched under a
root, host paths straight through, nothing outside the root and nothing Plex
deleted. 22 tests in tests/media_audiobooks.bats pin the export's join and the
plan's output.
2026-10-31 00:24:45 -07:00
dank a32361ee5e Merge remote-tracking branch 'origin/main' into feat/jellyfin-provider-keys 2026-10-31 00:10:58 -07:00
dank ce6b3b1d3b docs(media): how to prove a relayout landed, read from both servers' tables
A scan answers 204 whether or not it found anything, so proof is the journal
replayed against Jellyfin's BaseItems.Path and Plex's media_parts.file: every
move has a row at its dst, no row names its src. Records the four wrong doors:
no sqlite3 binary on the hosts or in either container, Jellyfin's single
BaseItems table (no MediaItems), its container-side /data path prefix, and
Plex's HTTPD closing on ?path= / on /search matching titles not paths.
2026-10-31 00:10:50 -07:00
dank 382e3eb0b8 Merge pull request 'feat(media): the music library's own relayout, filed by its tags' (#100) from feat/jellyfin-provider-keys into main 2026-10-31 00:04:41 -07:00
Frank 54a393edab Merge remote-tracking branch 'origin/main' into feat/jellyfin-provider-keys 2026-10-30 23:58:34 -07:00
Frank a5459b0b1e docs(media): the music library's own relayout, with the laws it earned 2026-10-30 23:58:34 -07:00
Frank b03931f7ce feat(media): a music relayout that files by the tag row Jellyfin matched
The music library is rips and discography packs whose filenames say nothing the
matcher needs. The authority is Jellyfin's own audio rows: one per track, with
the album and album-artist ids it carries in BaseItemProviders (the item JSON
holds an empty ProviderIds for albums in this build).

Grouping is what the tags assert, not where files sit: (album-artist tag, album,
year), case-folded. A compilation is one album whose tracks name many performers,
so it files once under Various Artists; two album-artist tags on one album name
are two albums. The folder spelling is the dominant one, because a name taken
from whichever row sorted first depends on row order and re-names albums that are
already filed.

An occupied path is never a destination: the executor refuses occupied targets,
so a plan that aims at one fails after the renames meant to clear it. A file
standing on its own target, or already carrying a collision suffix, is a keep -
otherwise the plan shuffles instead of being a fixed point.
2026-10-30 23:58:34 -07:00
dank 82f6ec55c2 Merge pull request 'feat(media): plan a TV relayout from the numbering Jellyfin already has' (#99) from feat/media-tv-relayout into main 2026-10-30 23:38:36 -07:00
Frank 536949c18d docs(media): the TV library's own relayout, with the evidence chain it believes
One page owns relayout: the census commands, the order the planner believes
things in, the two refusals a dry run on the real library proved necessary, the
apply and rollback path, and the standing wart that a provider name can itself
be scene junk ("Black Clover 001", 92 episodes) — renaming that is Jellyfin's
job, not the mover's.
2026-10-30 23:35:14 -07:00
Frank 337afac91e fix(media): a TV plan that follows the authorities it was given
A dry run of the TV planner against the real 58k-file library produced a root
series called "01 NCOP" out of a bonus subfolder, copied Jellyfin's stale
numbering (every file under SNL 16-20/Season 16..19 carries season 16 episode
20; one Gundam row says season 2004), and lost subtitles a scene pack had
buried in Subs/<release-name>/.

- Folders now RESOLVE a series and never invent one: a folder's words match a
  registered name as a token run ("(CBB) Tengen Toppa Gurren Lagann - The
  Complete Series (BD 1080p)" names Tengen Toppa Gurren Lagann), and folder
  words alone name nothing, so an unknown pack's bonus folder parks instead of
  opening a new show at the library root.
- Numbering priority: the marker the file's own name states, then the Season NN
  folder plus the number the name leads with inside it, then the provider row.
  A row that still insists on season 1989 parks its file rather than creating
  Season 1989.
- The provider row keeps identity: the registered name and its year survive a
  file whose own marker numbers it, and the pack index a release leads with
  ("4a. Seed DESTINY") is folded out of the folder name.
- A sidecar searches every ancestor folder for the video it names, not just the
  parent, so Subs/<release-name>/<release>.jpn.srt follows its episode.
- The shared bonus-folder test no longer reads an index as a title: name_words
  drops "extras" as release junk, so "01. Extras" reduced to "01" and matched
  the "01" of "NCOP 01", cancelling the bonus evidence. Only words can name a
  film.
2026-10-30 23:29:34 -07:00
Frank c0c77590b1 fix(media): a sidecar named "episode.mkv.srt" anchors, and loses the container word
The first run against the real library died in the anchor lookup: a subtitle
whose own name carries the media extension matches BOTH prefix forms of its
video, and the "keep the longest" comparison read the destination slot of the
tuple instead of the length slot. The fixture only ever matched one prefix, so
the seam was green over a crash.

The tail the sidecar keeps is now stripped of that container word: Jellyfin and
Plex both read "Show S01E01.forced.srt", not "Show S01E01.mkv.forced.srt", and
an nfo still carries the media extension because that one is constructed, not
parsed.
2026-10-30 23:17:55 -07:00
Frank 441b908937 feat(media): plan a TV relayout from the numbering Jellyfin already has
The Movies relayout planned from Plex; the TV library has no Plex rows and
58k files whose names lie about which episode they hold. Jellyfin has already
matched and numbered 47,676 of them, so the planner treats that numbering as
the authority and the file name as the fallback: an episode number the name
states, or a provider row, or nothing - and "nothing" parks the file under
_unsorted with a reason instead of inventing a number.

  - media-census.py jellyfin: one TSV row per matched episode, its container
    path mapped back to the host, series name/year/TMDb id from the Series row
    (the episode row carries none of them). Both TSV writers now end lines with
    \n so a shell consumer sees the last field clean.
  - media-shows.py: the plan. Series folders take the provider's own spelling;
    one file claiming two episodes keeps both numbers; two files claiming one
    episode - the bigger keeps the standard name, the other parks; bonus
    material files under the series' extras folder, never as an episode; a long
    file inside the TV library leaves for Movies only when TMDb answers with a
    matching title, otherwise it parks. Re-running the plan on a library it
    already filed moves nothing.
  - tests/media_shows.bats: 15 behaviors, including the executor preflight and
    the re-plan stability law.
2026-10-30 23:15:32 -07:00
dank 1a1dbac268 Merge pull request 'Jellyfin metadata providers: keys ride a host file the deployer installs' (#98) from feat/jellyfin-provider-keys into main 2026-10-30 22:48:21 -07:00
Frank 949d3cb6f4 docs(jellyfin): one page owns the provider keys and where they live
The page carries the deploy-time shape: which four providers are core plugins
in Jellyfin 12, which one is not, the exact XML each key lives in, the host
file the keys come from and its 600 rule, the three-way pin on the TVDB
artifact with its rollback, and what the deployer does when the keys file is
absent. OMDb is recorded as deliberately unconfigured -- keyed by email,
1000 req/day, and a ToS that forbids storing what it returns. index.md's row
carries the same locator.
2026-10-30 22:47:49 -07:00
Frank 3ce8d62e70 feat(jellyfin): the provider keys ride a host file the deployer installs
Jellyfin 12 ships TMDB, TVDB, StudioImages and MusicBrainz as core plugins:
no DLL, only a configuration file under the data root, so the keys have to
reach that XML from somewhere that is not git. They live in the fleet's
per-service env-file convention -- /home/dank/Containers/jellyfin/provider-keys
(TMDB_API_KEY=, TVDB_API_KEY=, optional TVDB_PIN=, mode 600) -- and
scripts/jellyfin-providers.sh plan|install|verify reads only that file, prints
no key byte, and refuses a keys file that is not mode 600 or that has no line
for the provider about to be written. install is idempotent: it never touches
a key the UI already saved, backs up every XML it edits into
~/.AGENTTRASH/<date>-jellyfin-providers/ first, and only installs the TVDB
plugin when the pinned official zip carries the plugin guid.

The deployer runs install BEFORE it restarts jellyfin, because the plugin and
its keys are read at start-up and a restart-first deploy boots the old state.
A missing keys file warns instead of failing: a host without the keys still
deploys, degraded rather than broken. No plugin configuration XML is tracked.
2026-10-30 22:47:39 -07:00
dank 38737ab84e Merge pull request 'Media metadata view: Jellyfin writes, Plex reads, the array stays read-only' (#97) from feat/media-metadata-view into main 2026-10-30 22:18:27 -07:00
17 changed files with 4394 additions and 15 deletions

No files matched your search

+5
View File
@@ -476,6 +476,11 @@ def services_for_path(path: str, target: str) -> set[str]:
# The installer is what creates, mounts and reaps the view both media
# servers read, so a change to it is a restart of that pair.
_add(services, "jellyfin", "plex")
if name == "jellyfin-providers.sh":
# The deployer runs this before jellyfin restarts, so a change to
# the installer is a change to the provider state Jellyfin loads at
# start-up: it selects jellyfin and nothing else.
_add(services, "jellyfin")
if name in {"frank-zealot.env", "frank-zealot.service", "serve-frank-zealot.sh"}:
# llama.cpp frank-zealot (2026-09-16) keeps its unit, env and serve
# files here; each one selects frank-zealot only. The ik_llama.cpp
+20
View File
@@ -1216,8 +1216,28 @@ ensure_media_view() {
bash scripts/media-view.sh verify
}
ensure_provider_keys() {
# Jellyfin's metadata-provider keys live in the container-owned plugin
# configuration under the data root, never in git; scripts/jellyfin-providers.sh
# is what places them, reading the per-service host env file (the fleet
# convention: grafana, caddy, pihole, copyparty, romm, seerr) that holds
# TMDB_API_KEY= and TVDB_API_KEY=, mode 600.
# Missing keys mean a keyless provider — a degraded library, not a broken
# deploy — so this warns rather than failing the restart. But when the keys
# ARE placed it must happen before the restart: the plugin and its keys are
# read at start-up, so a restart first would boot the old state.
local keys=/home/dank/Containers/jellyfin/provider-keys
if [ -f "$keys" ]; then
bash scripts/jellyfin-providers.sh install
else
echo "WARN: $keys is absent - provider keys NOT placed"
echo "WARN: create it (TMDB_API_KEY=, TVDB_API_KEY=, chmod 600) and re-deploy jellyfin"
fi
}
deploy_jellyfin() {
ensure_media_view
ensure_provider_keys
cp "dankserver/jellyfin/quadlet/jellyfin.container" "$quadlet_unit_dir/"
systemctl --user daemon-reload
if ! systemctl --user is-active -q jellyfin.service; then
+391
View File
@@ -0,0 +1,391 @@
#!/usr/bin/env bash
# Metadata-provider keys for Jellyfin on Dankserver, placed from the service's
# host env file with the plugin artifact pinned by content hash.
#
# Jellyfin 12 ships TMDb, OMDb, StudioImages and MusicBrainz as CORE plugins:
# they have no DLL, only a configuration file under the data root, and that
# file is created by the running container. So the fleet's provider state lives
# in two places the repo does not track —
#
# plugins/configurations/Jellyfin.Plugin.Tmdb.xml <TmdbApiKey /> (empty = keyless)
# plugins/configurations/Jellyfin.Plugin.Tvdb.xml <ApiKey /> (needs the plugin)
#
# and both are owned by the container user. This tool is the only thing that
# writes them: it is declarative, idempotent, and it never prints a key, so it
# is safe to run while someone watches.
#
# Keys never enter git and never enter argv (a value in argv is world-readable
# through /proc/<pid>/cmdline). They live in the per-service host env file that
# is the fleet's convention for container credentials (grafana, caddy, pihole,
# copyparty, romm, seerr), mode 600:
#
# /home/dank/Containers/jellyfin/provider-keys
# TMDB_API_KEY=<key>
# TVDB_API_KEY=<key>
# TVDB_PIN=<pin> # optional: only a subscriber-shaped TVDB key
# # needs one; a licensed key does not
#
# TVDB itself is not core in 12 (the metadata provider was cut in 10.9), so it
# is installed from the official catalog, pinned three ways: sha256 of the
# artifact, the plugin's own guid, and its version. A matching sha proves only
# that the bytes are the bytes fetched — the guid check is what says it is the
# official plugin, the version check is what says it is the one this file was
# reviewed against.
#
# Subcommands
# plan print the resolved shape and every pending change, touch nothing
# install fetch+verify the plugin, then write the keys (backing up first)
# verify prove the plugin and both keys are actually in place
#
# Nothing here restarts Jellyfin: the deployer owns that
# (`scripts/deploy.sh dankserver --service jellyfin`), and the plugin is read at
# start-up, so a key written while the server runs is not live until then.
set -euo pipefail
# Jellyfin's data root on Dankserver (the rootless quadlet's config volume).
JF_ROOT="${JF_ROOT:-/home/dank/Containers/jellyfin/config}"
PROV_KEYS="${PROV_KEYS:-/home/dank/Containers/jellyfin/provider-keys}"
TRASH_DIR="${TRASH_DIR:-$HOME/.AGENTTRASH/$(date +%Y%m%d)-jellyfin-providers}"
PLUGINS="$JF_ROOT/plugins"
CONF="$PLUGINS/configurations"
TMDB_XML="$CONF/Jellyfin.Plugin.Tmdb.xml"
TVDB_XML="$CONF/Jellyfin.Plugin.Tvdb.xml"
# repo.jellyfin.org catalog entry for TheTVDB (owner: jellyfin), version 24.0.0.0,
# targetAbi 12.0.0.0 — published 2026-09-09, verified against Jellyfin 12.4.0.
TVDB_PLUGIN_URL="${TVDB_PLUGIN_URL:-https://repo.jellyfin.org/files/plugin/thetvdb/thetvdb_24.0.0.0.zip}"
TVDB_PLUGIN_SHA256="${TVDB_PLUGIN_SHA256:-6302fba3857bdde8d186dd099fac09ca43247041f531a528a2c788add70a5319}"
TVDB_PLUGIN_GUID="${TVDB_PLUGIN_GUID:-a677c0da-fac5-4cde-941a-7134223f14c8}"
TVDB_PLUGIN_VERSION="${TVDB_PLUGIN_VERSION:-24.0.0.0}"
PLUGIN_DIR="thetvdb_${TVDB_PLUGIN_VERSION}"
TVDB_DLL="Jellyfin.Plugin.Tvdb.dll"
die() {
printf 'jellyfin-providers: %s\n' "$*" >&2
exit 1
}
usage() {
printf 'usage: %s plan|install|verify\n' "$0" >&2
exit 2
}
# check_keys — refuse anything that is not a readable private file. An open
# mode is an error, not a warning: a provider key readable by every user on the
# box is the failure this check exists to catch.
check_keys() {
[ -f "$PROV_KEYS" ] ||
die "$PROV_KEYS not found (create it with TMDB_API_KEY= and TVDB_API_KEY=, then chmod 600)"
local mode
mode="$(stat -c '%a' "$PROV_KEYS")"
case "$mode" in
600 | 400) ;;
*) die "$PROV_KEYS is mode $mode; provider keys must be 600" ;;
esac
}
# keys_value <VAR> — the value of one variable in the env file, or nothing.
keys_value() {
awk -v k="$1" -F= '$1 == k { sub(/^[^=]*=/, ""); print; exit }' "$PROV_KEYS"
}
# keys_len <VAR> — the length, and a refusal naming the variable when absent.
# The value itself is never printed.
keys_len() {
local v
v="$(keys_value "$1")"
[ -n "$v" ] || die "$PROV_KEYS has no $1= line"
printf '%s' "${#v}"
}
# xml_value <file> <element> — the element's text, empty when it is absent or
# self-closing.
xml_value() {
python3 - "$1" "$2" <<'PYEOF'
import re, sys
src = open(sys.argv[1], encoding="utf-8").read()
m = re.search(r"<%s(?:\s*/>|>(.*?)</%s>)" % (sys.argv[2], sys.argv[2]), src, re.S)
print(m.group(1).strip() if (m and m.group(1)) else "")
PYEOF
}
# xml_secret <file> <element> <var> — set <element> to that variable from the
# env file. The python process reads the env file itself, so the value never
# appears in argv.
xml_secret() {
python3 - "$1" "$2" "$3" "$PROV_KEYS" <<'PYEOF'
import os, re, sys, tempfile
from xml.sax.saxutils import escape
xml_path, elem, var, keys_path = sys.argv[1:5]
val = ""
for line in open(keys_path, encoding="utf-8"):
key, sep, rest = line.partition("=")
if sep and key.strip() == var:
val = rest.strip()
break
if not val:
raise SystemExit(f"{keys_path} has no {var}= line")
src = open(xml_path, encoding="utf-8").read()
pat = re.compile(r"([ \t]*)<%s(?:\s*/>|>.*?</%s>)" % (elem, elem), re.S)
text = escape(val)
if pat.search(src):
out = pat.sub(lambda m: f"{m.group(1)}<{elem}>{text}</{elem}>", src, count=1)
else:
out = src.replace("</Configuration>", f" <{elem}>{text}</{elem}>\n</Configuration>", 1)
if out == src:
raise SystemExit(f"{xml_path}: no place to write <{elem}>")
d, base = os.path.split(xml_path)
fd, tmp = tempfile.mkstemp(prefix=base + ".", dir=d)
with os.fdopen(fd, "w", encoding="utf-8") as fh:
fh.write(out)
os.chmod(tmp, os.stat(xml_path).st_mode & 0o777)
os.replace(tmp, xml_path)
PYEOF
}
# plugin_meta <zip> — "guid version" out of the artifact's own meta.json.
plugin_meta() {
python3 - "$1" <<'PYEOF'
import json, sys, zipfile
meta = json.loads(zipfile.ZipFile(sys.argv[1]).read("meta.json").decode("utf-8"))
print(meta.get("guid", ""), meta.get("version", ""))
PYEOF
}
check_root() {
# Never fabricate a data root: creating plugins/ beside a typo'd path would
# install a plugin Jellyfin never loads and hide the operator's mistake.
[ -d "$CONF" ] || die "no Jellyfin configuration directory at $CONF (is JF_ROOT right?)"
}
check_keys_report() {
check_keys
# A refusal inside a command substitution used as a printf argument would
# be swallowed, so the guard runs as its own statement.
keys_len TMDB_API_KEY >/dev/null
keys_len TVDB_API_KEY >/dev/null
printf '= keys read from %s (mode %s)\n' "$PROV_KEYS" "$(stat -c '%a' "$PROV_KEYS")"
}
# The plugin's state, as one line in the shared vocabulary.
plugin_state() {
if [ -d "$PLUGINS/$PLUGIN_DIR" ]; then
printf '= %s already installed\n' "$PLUGIN_DIR"
return 0
fi
local other
other="$(find "$PLUGINS" -maxdepth 1 -type d -name 'thetvdb_*' ! -name "$PLUGIN_DIR" -printf '%f\n' 2>/dev/null | sort | tr '\n' ' ')"
if [ -n "$other" ]; then
printf '+ install %s (a different version is present: %s)\n' "$PLUGIN_DIR" "$other"
return 0
fi
printf '+ install %s from %s\n' "$PLUGIN_DIR" "${TVDB_PLUGIN_URL#*://}"
}
tmdb_state() {
if [ ! -f "$TMDB_XML" ]; then
printf '! %s is missing — save the TMDb plugin settings once in the UI, then re-run\n' "$TMDB_XML" >&2
return 1
fi
local now want
now="$(xml_value "$TMDB_XML" TmdbApiKey)"
want="$(keys_value TMDB_API_KEY)"
if [ -n "$now" ] && [ "$now" = "$want" ]; then
printf '= TmdbApiKey already set (%s chars)\n' "${#now}"
elif [ -n "$now" ]; then
printf '+ replace the TmdbApiKey in Jellyfin.Plugin.Tmdb.xml\n'
else
printf '+ set TmdbApiKey in Jellyfin.Plugin.Tmdb.xml\n'
fi
}
tvdb_state() {
local now
if [ -f "$TVDB_XML" ]; then
now="$(xml_value "$TVDB_XML" ApiKey)"
if [ -n "$now" ]; then
printf '= Jellyfin.Plugin.Tvdb.xml already has an ApiKey (kept)\n'
else
printf '+ fill the empty ApiKey in Jellyfin.Plugin.Tvdb.xml\n'
fi
else
printf '+ create Jellyfin.Plugin.Tvdb.xml with ApiKey\n'
fi
}
write_plugin() {
local tmp guid version other
other="$(find "$PLUGINS" -maxdepth 1 -type d -name 'thetvdb_*' ! -name "$PLUGIN_DIR" -printf '%f\n' 2>/dev/null | sort | tr '\n' ' ')"
[ -z "$other" ] ||
die "another TheTVDB version is already installed ($other); move it to the trash first — two copies of one plugin means two loads"
tmp="$(mktemp "${TMPDIR:-/tmp}/thetvdb.XXXXXX.zip")"
# The pins are checked before anything is unpacked, so a bad artifact never
# reaches the plugins directory.
curl -fsSL --max-time 300 -o "$tmp" "$TVDB_PLUGIN_URL" ||
{ rm -f "$tmp"; die "could not fetch $TVDB_PLUGIN_URL"; }
local got
got="$(sha256sum "$tmp" | cut -d' ' -f1)"
if [ "$got" != "$TVDB_PLUGIN_SHA256" ]; then
rm -f "$tmp"
die "artifact sha256 is $got, pinned $TVDB_PLUGIN_SHA256 — refusing to install"
fi
read -r guid version <<<"$(plugin_meta "$tmp")"
if [ "$guid" != "$TVDB_PLUGIN_GUID" ]; then
rm -f "$tmp"
die "artifact guid is $guid, expected the official $TVDB_PLUGIN_GUID — refusing to install"
fi
if [ "$version" != "$TVDB_PLUGIN_VERSION" ]; then
rm -f "$tmp"
die "artifact version is $version, pinned $TVDB_PLUGIN_VERSION — refusing to install"
fi
mkdir -p "$PLUGINS/$PLUGIN_DIR"
python3 -m zipfile -e "$tmp" "$PLUGINS/$PLUGIN_DIR/"
rm -f "$tmp"
[ -f "$PLUGINS/$PLUGIN_DIR/$TVDB_DLL" ] ||
die "$PLUGIN_DIR has no $TVDB_DLL — the artifact does not match this plugin"
printf '+ installed %s\n' "$PLUGIN_DIR"
}
backup_to_trash() {
local f="$1"
mkdir -p "$TRASH_DIR"
local dest="$TRASH_DIR/$(basename "$f").$(date +%H%M%S)"
cp -p "$f" "$dest"
printf '+ previous copy kept at %s\n' "$dest"
}
cmd_write_keys() {
local now want
KEYS_CHANGED=0
# TMDb: the running container owns this file and every setting in it. Only
# the key element is touched, so country, language and adult settings
# survive the edit.
[ -f "$TMDB_XML" ] ||
die "$TMDB_XML is missing — save the TMDb plugin settings once in the UI, then re-run"
now="$(xml_value "$TMDB_XML" TmdbApiKey)"
keys_len TMDB_API_KEY >/dev/null
want="$(keys_value TMDB_API_KEY)"
if [ "$now" = "$want" ]; then
printf '= TmdbApiKey already set (%s chars)\n' "${#now}"
else
backup_to_trash "$TMDB_XML"
xml_secret "$TMDB_XML" TmdbApiKey TMDB_API_KEY
printf '+ set TmdbApiKey (%s chars)\n' "${#want}"
KEYS_CHANGED=1
fi
# TVDB: never overwrite a config the operator saved through the UI.
if [ -f "$TVDB_XML" ]; then
now="$(xml_value "$TVDB_XML" ApiKey)"
if [ -n "$now" ]; then
printf '= Jellyfin.Plugin.Tvdb.xml already has an ApiKey (kept)\n'
else
keys_len TVDB_API_KEY >/dev/null
backup_to_trash "$TVDB_XML"
xml_secret "$TVDB_XML" ApiKey TVDB_API_KEY
if [ -n "$(keys_value TVDB_PIN)" ] && [ -z "$(xml_value "$TVDB_XML" Pin)" ]; then
xml_secret "$TVDB_XML" Pin TVDB_PIN
fi
printf '+ filled the ApiKey in Jellyfin.Plugin.Tvdb.xml\n'
KEYS_CHANGED=1
fi
else
keys_len TVDB_API_KEY >/dev/null
cat >"$TVDB_XML" <<XML
<?xml version="1.0" encoding="utf-8"?>
<Configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<ApiKey />
</Configuration>
XML
xml_secret "$TVDB_XML" ApiKey TVDB_API_KEY
if [ -n "$(keys_value TVDB_PIN)" ]; then
xml_secret "$TVDB_XML" Pin TVDB_PIN
fi
printf '+ created Jellyfin.Plugin.Tvdb.xml with ApiKey\n'
KEYS_CHANGED=1
fi
}
cmd_plan() {
check_root
check_keys_report
printf 'root %s\n' "$JF_ROOT"
plugin_state
tmdb_state
tvdb_state
}
cmd_install() {
check_root
check_keys_report
KEYS_CHANGED=0
if [ ! -d "$PLUGINS/$PLUGIN_DIR" ]; then
write_plugin
KEYS_CHANGED=1
else
printf '= %s already installed\n' "$PLUGIN_DIR"
fi
cmd_write_keys
if [ "$KEYS_CHANGED" -eq 1 ]; then
# Nothing is restarted here on purpose: this host may be playing media
# right now, and both the plugin and its keys are read at start-up.
printf 'next: scripts/deploy.sh dankserver --service jellyfin (the plugin and keys load at start-up)\n'
else
printf '= no change\n'
fi
}
cmd_verify() {
check_root
local bad=0
if [ ! -d "$PLUGINS/$PLUGIN_DIR" ]; then
printf 'MISSING plugin directory %s (thetvdb not installed)\n' "$PLUGIN_DIR" >&2
bad=1
else
local dlls abi
dlls="$(find "$PLUGINS/$PLUGIN_DIR" -maxdepth 1 -name '*.dll' | wc -l)"
[ "$dlls" -ge 1 ] || {
printf 'MISSING %s inside %s\n' "$TVDB_DLL" "$PLUGIN_DIR" >&2
bad=1
}
abi="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1])).get("targetAbi","?"))' \
"$PLUGINS/$PLUGIN_DIR/meta.json" 2>/dev/null || echo '?')"
printf 'plugin %s: %s dll(s), targetAbi %s\n' "$PLUGIN_DIR" "$dlls" "$abi"
fi
local pair elem n
for pair in "$TMDB_XML|TmdbApiKey" "$TVDB_XML|ApiKey"; do
elem="${pair##*|}"
if [ ! -f "${pair%%|*}" ]; then
printf 'MISSING %s\n' "${pair%%|*}" >&2
bad=1
continue
fi
if ! python3 -c 'import sys, xml.dom.minidom; xml.dom.minidom.parse(sys.argv[1])' "${pair%%|*}"; then
printf 'UNPARSEABLE %s\n' "${pair%%|*}" >&2
bad=1
continue
fi
n="$(xml_value "${pair%%|*}" "$elem")"
if [ -z "$n" ]; then
printf 'MISSING key: %s has no %s value\n' "$(basename "${pair%%|*}")" "$elem" >&2
bad=1
else
printf '%s %s chars\n' "$elem" "${#n}"
fi
done
[ "$bad" -eq 0 ] || return 1
printf 'providers configured\n'
}
cmd="${1:-}"
case "$cmd" in
plan) cmd_plan ;;
install) cmd_install ;;
verify) cmd_verify ;;
*) usage ;;
esac
+425
View File
@@ -0,0 +1,425 @@
#!/usr/bin/env python3
r"""media-audiobooks.py — a plan that re-files the audiobook library into one folder per book.
media-audiobooks.py --root DIR --enum TSV --rows TSV [--out TSV] [--exceptions FILE]
Plex reads this library and Jellyfin does not: its music scanner has no audiobook
concept, and the tree is indexed entirely by Plex, which matched 58,975 of the
62,879 files here into its Artist/Album/Track shape. So the authority is
`scripts/media-census.py audiobooks` — the BOOK (album row), its NARRATOR (that
book's single artist row) and its CHAPTERS (track rows) — and never the folder a
release pack made. The plan is:
Audiobooks/<Narrator>/<Book> (<year>)/<the file's own name>.<ext>
... the same folder, " (2)" on the name, when two chapters
of one book carry the same name
What that buys is the one property both media servers' book scanners assume: a
folder holds exactly one book. Today 212 top-level folders are a mix of
author-named shelves and collection packs, and the books inside the packs are
split across `… [Part 1]`, `… [Part 2]` folders three or four deep.
Three laws here are the opposite of what a music shelf does, and each is forced
by what the rows actually hold:
* A chapter is NEVER renumbered. 7,556 of the 10,485 books contain at least one
chapter whose index Plex does not know, so a number written into a name would
be invented — and it would buy nothing, because both servers order playback by
the embedded track tag, not by the file name. A wrong number in a name is worse
than none: it is the number a human later trusts. A file already carrying its
own number keeps it, untouched.
* The shelf is the NARRATOR, not the author. Every book has exactly one artist row
and it is the narrator; the author survives only inside 10% of the album titles
(`Kryptos by Graham McNeill`) and in no column Plex stores, so deriving the
author would mean parsing titles for one book in ten and inventing it for the
rest. One rule that always resolves beats a shelf that is half author and half
narrator; the author stays visible in the book folder's own name, which is the
title Plex shows.
* Nothing is guessed at. A chapter row with no book or no narrator parks — a
tag-scan placeholder (`[Unknown Album]`) names neither, since unrelated books
share it and filing on one merges them, and so does a chapter number the ripper
mistagged as the artist (`0.4`). An audio file Plex never matched parks, a
video container parks, and a loose file in a folder holding two books parks.
Nothing is deleted and nothing is overwritten. The output is a list of rename(2)
intents for `scripts/media-apply.py`, whose preflight refuses any target that is
already claimed and whose journal is the rollback.
"""
import argparse
import csv
import os
import re
import sys
from collections import Counter, defaultdict
# What a chapter file can be. .m4b is the audiobook container; the rest are the
# audio a pack ships when it was ripped as music.
AUDIO_EXT = {".mp3", ".m4a", ".m4b", ".ogg", ".oga", ".opus", ".wav", ".flac",
".wma", ".aiff", ".aif", ".m4v", ".aac", ".mka"}
# Never an audiobook, whatever the library it sits in says: Plex reads these as
# video, and renaming one into an audio extension would claim it is a chapter.
VIDEO_EXT = {".mkv", ".mp4", ".webm", ".avi", ".mov", ".mpg", ".mpeg", ".ts",
".m2ts", ".vob", ".ifo", ".wmv", ".iso", ".img", ".bdmv", ".ogm",
".divx"}
# Tag-scan placeholders. These strings are not identities: 1,106 chapters from
# 14 unrelated top-level folders carry the album `[Unknown Album]`, so filing on
# them would merge books that have nothing to do with each other — the exact
# shape this relayout exists to make impossible. Such a chapter parks with the
# path it has today and the fix belongs in its tags. `Various Artists` is NOT a
# placeholder: that is what a multi-cast read carries, and the one book here
# with 1,189 chapters under it is genuinely one book.
PLACEHOLDER_BOOK = {"[unknown album]", "unknown album", "no title", "untitled",
"[untitled]", "unknown title", "n/a", "-"}
PLACEHOLDER_NARRATOR = {"[unknown artist]", "unknown artist", "unknown author",
"<unknown>", "[unknown]", "unknown"}
def identity(value, placeholders, needs_letter=False):
"""The tag value Plex matched, when it names something; '' when it does not.
A trailing parenthesized group is stripped for the test only, because rippers
glue timestamps to the placeholder (`Unknown album (03/03/2015 21:39:55)`).
`needs_letter` is for a NARRATOR alone: a shelf is a person's name, and 272
chapters here carry a chapter number in the artist tag (`0.4`, `31`). The same
test must never judge a BOOK title, which may honestly be `1984`.
"""
v = (value or "").strip()
bare = re.sub(r"\s*\([^()]*\)\s*$", "", v).strip().casefold()
if bare in placeholders or (needs_letter and not any(c.isalpha() for c in v)):
return ""
return v
# The characters Plex documents as breaking playback. They are dropped, not
# substituted: a replacement character is a guess at what the title meant.
BAD_CHARS = set('<>:"/\\|?*')
TRASH_DIR = re.compile(r"(^|/)(\.AGENTTRASH|\.thumbnails|\.Trash-\d+|@eaDir"
r"|lost\+found)(/|$)")
ARGS = None
HELD = {} # path -> (bytes, ext)
ROWS = {} # path -> the chapter row Plex matched
PLACED = {} # chapter path -> the destination the plan gave it
CLAIMED = set() # every destination path the plan has handed out
MOVES, KEEPS = [], []
PARKED = defaultdict(list) # reason -> [path]
NOTES = defaultdict(list) # section -> [line]
AUDIO_BY_DIR = defaultdict(set) # source dir -> chapter files a row names
CHILD_OF = defaultdict(set) # dir -> the dirs directly below it holding chapters
FOLDERS_SEEN = {} # (dir, depth) -> book folders placed below it
SUBTREE_DEPTH = 2 # how far below a folder its chapters may sit
BOOKS = {} # (narrator, book, year) -> the book group
AMBIGUOUS = object()
def dominant(values):
"""The spelling the rows mostly agree on: the most frequent value, ties broken
by sort so the answer never depends on the order rows arrived in."""
counts = Counter(values)
return min(counts.items(), key=lambda kv: (-kv[1], kv[0]))[0]
def rel(path):
"""The path as seen from the library root."""
return os.path.relpath(path, ARGS.root)
def safe(name):
"""A name every client can carry: no reserved character, nothing unprintable,
no trailing dot or space, and short enough to rename."""
out = "".join(c for c in (name or "")
if c not in BAD_CHARS and (c.isprintable() or c == " "))
out = re.sub(r"\s+", " ", out).strip().rstrip(". ")
while out and len(out.encode("utf-8", "ignore")) > 200:
out = out[:-4].rstrip()
return out
def assign(src, folder, name, ext, kind, note):
"""Book one file into `folder` under `name`, renamed if that exact destination
is already taken — by another file in the plan, or by a file standing there
today. Two files claiming one path is the fault that costs a re-run of a 60k
plan, and so is a plan that only works if the renames happen in the right
order: the executor refuses any target that is occupied, so an occupied path
is never a destination, even when its occupant is scheduled to leave.
A file already standing where the contract puts it is a keep, not a move."""
stem = safe(name) or "file"
dst = os.path.join(folder, stem + ext)
if dst == src:
CLAIMED.add(dst.lower()) # it is standing on its own target
KEEPS.append((src, dst, kind))
return dst
# A file already in this folder under this name — or under the numbered
# variant the collision rule gives it — keeps the name it has. Renumbering it
# every run would turn the plan into a shuffle instead of a fixed point.
base = os.path.splitext(os.path.basename(src))[0]
if os.path.dirname(src) == folder and (base == stem
or base.startswith(stem + " (")):
CLAIMED.add(src.lower())
KEEPS.append((src, src, kind))
return src
n = 1
while dst.lower() in CLAIMED:
n += 1
dst = os.path.join(folder, f"{stem} ({n}){ext}")
if n > 1:
NOTES[note].append(rel(dst))
CLAIMED.add(dst.lower())
MOVES.append((src, dst, kind))
return dst
def park(path, reason):
"""Hold a file where it is, under the park folder, keeping its relative path."""
PARKED[reason].append(path)
dst = os.path.join(ARGS.park_dir, rel(path))
if dst != path:
CLAIMED.add(dst.lower())
MOVES.append((path, dst, "park"))
def under_park(path):
"""True for a file already parked: that pile is for a human to review, and
re-parking it every run would only push it deeper."""
return path.startswith(ARGS.park_dir + os.sep)
# ------------------------------------------------------------------------ books
def group_books():
"""Every matched chapter into one group per (narrator, book, year).
The key is the identity Plex asserts in its own database, not the folder the
file sits in: one book arrives as four part folders inside a collection pack,
and two copies of one book in one pack are one book to Plex, whose album item
IS that narrator, title and year. The year is part of the key so a 1965
recording and a 2020 recording that share a title stay two books.
"""
for path, r in sorted(ROWS.items()):
if path not in HELD or under_park(path) or TRASH_DIR.search(path):
continue
d = os.path.dirname(path)
AUDIO_BY_DIR[d].add(path)
CHILD_OF[os.path.dirname(d)].add(d) # so a cover above Part 1/Part 2 sees both
narrator = identity(r["narrator"], PLACEHOLDER_NARRATOR, needs_letter=True)
book = identity(r["book"], PLACEHOLDER_BOOK)
if not book:
park(path, "a chapter row that names no book — Plex's album title is a "
"placeholder like [Unknown Album], so filing on it would "
"merge unrelated books onto one shelf")
continue
if not narrator:
park(path, "a chapter row that names no narrator — the artist row is a "
"placeholder like [Unknown Artist] or a chapter number a "
"ripper mistagged as the artist, so no shelf owns it")
continue
key = (narrator.casefold(), book.casefold(), (r["year"] or "").strip())
g = BOOKS.setdefault(key, {"narrators": [], "books": [], "year": key[2],
"members": [], "indexed": 0})
g["members"].append(path)
g["narrators"].append(narrator)
g["books"].append(book)
if str(r["track"] or "").isdigit():
g["indexed"] += 1
def book_folder(g):
"""The shelf Plex's own rows name: <narrator>/<book> (<year>). None when a
name is made of characters no path can hold — inventing a shelf is filing it
wrong, so those chapters park instead."""
narrator = safe(dominant(g["narrators"]))
book = safe(dominant(g["books"]))
if not narrator or not book:
return None
return os.path.join(ARGS.root, narrator,
f"{book} ({g['year']})" if g["year"] else book)
def place_books():
"""One folder per book; every chapter keeps the name it has today."""
for key in sorted(BOOKS):
g = BOOKS[key]
folder = book_folder(g)
if folder is None:
for path in sorted(g["members"]):
park(path, "a book or narrator Plex spells with characters no "
"folder name can hold")
continue
for path in sorted(g["members"]):
ext = HELD[path][1]
base = os.path.splitext(os.path.basename(path))[0]
# The name the ripper wrote, unchanged: the track tag inside the file
# is what orders playback, and 72% of the books here have chapters no
# row numbers, so any number added here would be invented.
PLACED[path] = assign(path, folder, base, ext, "chapter",
"two chapters of one book carried the same name")
# --------------------------------------------------------------------- sidecars
def folders_below(d, depth):
"""The book folders the chapters in `d`, or within `depth` directories below
it, were placed in. Memoised, because every loose file in a big collection
folder asks the same question about it."""
if (d, depth) not in FOLDERS_SEEN:
seen = {os.path.dirname(PLACED[a]) for a in AUDIO_BY_DIR.get(d, ())
if a in PLACED}
if depth:
for child in CHILD_OF.get(d, ()):
seen |= folders_below(child, depth - 1)
FOLDERS_SEEN[(d, depth)] = seen
return FOLDERS_SEEN[(d, depth)]
def owner_folder(d):
"""The book folder that owns this directory — the first place up the tree
where chapters sit, in the directory itself or within SUBTREE_DEPTH levels
below it, which is how a pack whose parts are `… [Part 1]`/`… [Part 2]` owns
the cover art lying above them. None when no chapter is near enough to say,
AMBIGUOUS when the chapters there belong to more than one book."""
while d.startswith(ARGS.root + os.sep) and d != ARGS.root:
folders = folders_below(d, SUBTREE_DEPTH)
if folders:
return next(iter(folders)) if len(folders) == 1 else AMBIGUOUS
d = os.path.dirname(d)
return None
def place_other():
"""Everything that is not a chapter Plex matched: covers, booklets, EPUBs,
logs, checksums, playlists — and the clutter that is none of those."""
for path, (_size, ext) in sorted(HELD.items()):
if path in ROWS or under_park(path) or TRASH_DIR.search(path):
continue
d = os.path.dirname(path)
base = os.path.splitext(os.path.basename(path))[0]
if ext in VIDEO_EXT:
park(path, "a video container inside the audiobook library: Plex reads "
"it as video, and renaming it would claim it is a chapter")
elif ext in AUDIO_EXT:
park(path, "an audio file no chapter row names — it is not in the "
"library Plex matched, so nothing says which book it is")
else:
owner = owner_folder(d)
if owner is AMBIGUOUS:
park(path, "a file whose folder holds more than one book, so "
"nothing says which one it belongs to")
elif owner:
assign(path, owner, base, ext,
"playlist" if ext in (".m3u", ".m3u8") else "sidecar",
"one book folder would have carried two files of the same"
" sidecar name")
else:
park(path, "not an audiobook file, and no book's folder lies with"
" it to carry it")
# ---------------------------------------------------------------------- outputs
def write_plan():
"""mkdir rows for every destination folder and its whole chain, then the moves.
Parents first: the executor creates directories in the order they are listed."""
chain = set()
for _s, dst, _k in MOVES:
d = os.path.dirname(dst)
while d and d != ARGS.root and d != os.path.dirname(d):
chain.add(d)
d = os.path.dirname(d)
dirs = sorted(chain, key=lambda p: (p.count("/"), p))
with open(ARGS.out, "w") as fh:
fh.write("action\tsrc\tdst\tkind\n")
for d in dirs:
fh.write(f"mkdir\t\t{d}\tdir\n")
for src, dst, kind in sorted(MOVES, key=lambda m: (m[1], m[0])):
fh.write(f"move\t{src}\t{dst}\t{kind}\n")
for src, dst, kind in sorted(KEEPS, key=lambda m: (m[1], m[0])):
fh.write(f"keep\t{src}\t{dst}\t{kind}\n")
parked = [m for m in MOVES if m[2] == "park"]
sized = sum(HELD[s][0] for s, _d, _k in MOVES if s in HELD)
print(f"plan: {len(MOVES) - len(parked)} moves, {len(parked)} parked"
f" ({sized / 2**40:.2f} TiB touched), {len(KEEPS)} already placed,"
f" {len(dirs)} folders -> {ARGS.out}")
def write_exceptions():
with open(ARGS.exceptions, "w") as fh:
fh.write(f"Audiobook relayout exceptions for {ARGS.root}\n\n")
fh.write(f"Everything below is parked under {ARGS.park_dir}, keeping the\n"
"relative path it has today. Nothing is deleted and nothing is\n"
"overwritten: a parked file is one rename from wherever the plan is\n"
"pointed next.\n\n")
for reason in sorted(PARKED, key=lambda r: -len(PARKED[r])):
rs = PARKED[reason]
fh.write(f"{len(rs):>6} files {reason}\n")
for p in sorted(rs, key=rel)[:400]:
fh.write(f" {rel(p)}\n")
if len(rs) > 400:
fh.write(f" … and {len(rs) - 400} more\n")
fh.write("\n")
for section in sorted(NOTES, key=lambda s: -len(NOTES[s])):
lines = sorted(set(NOTES[section]))
fh.write(f"{len(lines):>6} {section}\n")
for line in lines[:200]:
fh.write(f" {line}\n")
if len(lines) > 200:
fh.write(f" … and {len(lines) - 200} more\n")
fh.write("\n")
unindexed = sorted(f"{safe(dominant(g['narrators']))}/{safe(dominant(g['books']))}"
for g in BOOKS.values()
if 0 < g["indexed"] < len(g["members"]))
if unindexed:
fh.write(f"{len(unindexed):>6} books have chapters Plex does not number. Their\n")
fh.write(" files keep the names they carry: playback order comes from the\n"
" track tag inside each file, and a number written here would be\n"
" invented for the chapters with no index. Fix it in the tags, not\n"
" in the names — Plex re-reads tags on a scan and never names files.\n\n")
fh.write(" Playlists move with their book and their contents are never\n"
" rewritten: the paths inside an .m3u go stale.\n")
print(f"exceptions -> {ARGS.exceptions}")
def main():
global ARGS
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("--root", default=os.environ.get("MEDIA_AUDIOBOOKS_ROOT",
"/mnt/entropy/Audiobooks"))
ap.add_argument("--enum", default="/tmp/audiobooks-enum.tsv",
help="the enumerate TSV (bytes, ext, path)")
ap.add_argument("--rows", default="/tmp/plex-audiobooks.tsv",
help="the chapter rows scripts/media-census.py audiobooks wrote")
ap.add_argument("--out", default="/tmp/audiobooks-plan.tsv")
ap.add_argument("--exceptions", default="/tmp/audiobooks-exceptions.txt")
ap.add_argument("--park", default="_unsorted",
help="the folder inside the root that holds what cannot be filed")
ARGS = ap.parse_args()
ARGS.root = os.path.normpath(ARGS.root)
ARGS.park_dir = os.path.join(ARGS.root, ARGS.park)
read_inputs()
# Every path a file occupies today is claimed before a single destination is
# handed out: the executor refuses an occupied target, so a plan that aimed at
# one would only fail after the renames that were supposed to clear it.
CLAIMED.update(p.lower() for p in HELD)
group_books()
place_books()
place_other()
write_plan()
write_exceptions()
def read_inputs():
with open(ARGS.enum) as fh:
for line in fh:
parts = line.rstrip("\n").split("\t")
if len(parts) >= 3:
HELD[parts[2]] = (int(parts[0]), parts[1].lower())
with open(ARGS.rows, newline="") as fh:
for row in csv.DictReader(fh, delimiter="\t"):
if row.get("file"):
ROWS[row["file"]] = row
gone = [p for p in ROWS if p not in HELD]
if gone:
NOTES["a chapter row whose file the census no longer sees"].extend(gone)
if __name__ == "__main__":
main()
+300 -6
View File
@@ -1,13 +1,15 @@
#!/usr/bin/env python3
r"""Read-only census of a movie library — the inputs a relayout plan is built from.
r"""Read-only census of a media library — the inputs a relayout plan is built from.
media-census.py enumerate [--root DIR] [--out TSV]
media-census.py probe [--enum TSV] [--out TSV] [--jobs N]
media-census.py plex [--root DIR] [--db PATH] [--out TSV]
media-census.py jellyfin [--root DIR] [--db PATH] [--container PREFIX] [--out TSV]
media-census.py audiobooks [--root DIR] [--db PATH] [--out TSV]
Run it on the machine that mounts the library. Nothing here writes inside the
library and nothing is deleted; the only outputs are the three TSVs below, and
scripts/media-plan.py reads them instead of touching media.
library and nothing is deleted; the only outputs are the TSVs below, and the
scripts/media-plan.py and scripts/media-shows.py read them instead of touching media.
enum bytes <TAB> ext <TAB> path every file under the root
probed bytes <TAB> path <TAB> dur <TAB> vcodec <TAB> WxH <TAB> fps <TAB> vbit
@@ -18,10 +20,33 @@ scripts/media-plan.py reads them instead of touching media.
plex file <TAB> leaf_title <TAB> … (see PLEX_COLS)
the titles Plex already matched — the authority over any filename guess
jellyfin file <TAB> series <TAB> year <TAB> season <TAB> episode <TAB> extra
<TAB> tmdb (see JF_COLS)
one row per episode Jellyfin has matched and numbered, its path mapped
from the container's view back to the host; `year` and `tmdb` are the
SERIES' values, `extra` is Jellyfin's own extra type when the row is
bonus material rather than an episode
music file <TAB> artist <TAB> album <TAB> year <TAB> disc <TAB> track <TAB> title
<TAB> mb <TAB> performer (see MUSIC_COLS)
one row per track Jellyfin has tagged: the tag read is what the row IS,
so `artist` is the album artist (the track artist when that is empty) and
`album`/`year` are the album item's provider values when the folder is an
album item and the tags' otherwise; `mb` is the album's MusicBrainz
release group, `performer` the track artist the tags carry
audiobooks file <TAB> narrator <TAB> book <TAB> year <TAB> track <TAB> chapter
(see AB_COLS)
one row per chapter Plex matched under the audiobook root: `book` is the
album row Plex built from the files' own tags, `narrator` that book's
artist row — Plex keeps exactly one artist per book, which is why it, and
not an author parsed out of a title, names the shelf. `track` is empty
wherever Plex has no chapter number, which is most of this library.
`probe` is the slow step (every container is opened for its header, ~10 min for
4,800 files on a spindle array at 6 workers) so it is `nice`d: a census beside a
live service must not starve it. `enumerate` is a stat pass, so re-censusing
before a plan is cheap — and a plan is only ever as good as these three files.
before a plan is cheap — and a plan is only ever as good as these files.
"""
import argparse
import csv
@@ -57,6 +82,51 @@ join metadata_items mi on mi.id = m.metadata_item_id
left join metadata_items par on par.id = mi.parent_id
where p.file like ? and p.deleted_at is null and mi.deleted_at is null
"""
JF_DB = "/home/dank/Containers/jellyfin/config/data/jellyfin.db"
JF_COLS = ["file", "series", "year", "season", "episode", "extra", "tmdb"]
# Jellyfin stores the entity's .NET class as its type; the TV tree is these three.
JF_TV = ("MediaBrowser.Controller.Entities.TV.Episode",
"MediaBrowser.Controller.Entities.TV.Season",
"MediaBrowser.Controller.Entities.TV.Series")
JF_SQL = ("""select Id, Path, IndexNumber, ParentIndexNumber, ExtraType, IsVirtualItem
from BaseItems where Type = ? and Path like ?""")
# The music tree is these two entity classes. The track row carries the tags
# Jellyfin read out of the file; the album row carries what the provider matched,
# which is the better folder name of the two.
JF_AUDIO = "MediaBrowser.Controller.Entities.Audio.Audio"
JF_ALBUM = "MediaBrowser.Controller.Entities.Audio.MusicAlbum"
MUSIC_COLS = ["file", "artist", "album", "year", "disc", "track", "title", "mb",
"performer"]
MUSIC_SQL = ("""select b.Path, b.AlbumArtists, b.Artists, b.Album, b.ProductionYear,
b.ParentIndexNumber, b.IndexNumber, b.Name, b.Id,
a.Name, a.ProductionYear, a.Id
from BaseItems b
left join BaseItems a on a.Id = b.ParentId and a.Type = ?
where b.Type = ? and b.Path like ?""")
# The album's identity, in preference order. A release group is the album; a bare
# release id is one reissue of it.
MUSIC_PROVIDERS = ("MusicBrainzReleaseGroup", "MusicBrainzAlbum")
# Plex's audiobook tree is the same Artist/Album/Track shape as its music tree:
# the leaf (10) is a CHAPTER, its parent (9) is the BOOK, and that book's artist
# (8) is the NARRATOR. The track's own `index` is exported because the exceptions
# report counts how many books have chapters Plex does not number — the reason the
# relayout refuses to invent numbers into file names.
PLEX_ARTIST, PLEX_ALBUM, PLEX_TRACK = 8, 9, 10
AB_COLS = ["file", "narrator", "book", "year", "track", "chapter"]
AB_SQL = f"""
select p.file, coalesce(ar.title,''), coalesce(al.title,''), coalesce(al.year,0),
coalesce(mi."index",-1), coalesce(mi.title,'')
from media_parts p
join media_items m on m.id = p.media_item_id
join metadata_items mi on mi.id = m.metadata_item_id
join metadata_items al on al.id = mi.parent_id and al.metadata_type = {PLEX_ALBUM}
left join metadata_items ar on ar.id = al.parent_id
and ar.metadata_type = {PLEX_ARTIST}
where p.file like ? and p.deleted_at is null and mi.deleted_at is null
and al.deleted_at is null
"""
def enumerate_lib(root, out):
@@ -165,7 +235,7 @@ def plex(root, db, out):
"Plex runs rootless here: run this as the container's owner, never "
"with sudo — rootless storage is per-user, so root sees nothing.")
with open(out, "w", newline="") as fh:
w = csv.writer(fh, delimiter="\t")
w = csv.writer(fh, delimiter="\t", lineterminator="\n")
w.writerow(PLEX_COLS)
n = 0
for r in con.execute(PLEX_SQL, (root + "/%",)):
@@ -174,6 +244,197 @@ def plex(root, db, out):
print(f"plex-matched={n} -> {out}")
def jellyfin(root, db, container, out):
"""The episodes Jellyfin has already numbered, for the library at root.
Jellyfin stores each path as the CONTAINER sees it (its quadlet binds the
share at /data), so `container` is the in-container prefix that stands for
root and every exported path is mapped back to the host before it is written
— the planner reads host paths, because it plans moves on the array.
The numbering is what matters, not the name: a release group's file name says
nothing about which episode a file is, and Jellyfin has already answered that
question against a provider. Season/episode come from the episode row; the
series name, its year and its TMDb id come from the Series row the episode
climbs to, because those are the only place Jellyfin keeps them.
"""
try:
con = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
except sqlite3.Error as e:
sys.exit(f"cannot open the Jellyfin database read-only ({db}): {e}\n"
"It is a WAL database: copy it with its -wal and -shm beside it and"
" point --db at the copy. Never sudo it — the data dir belongs to"
" the container's owner, and rootless storage is per-user.")
have = {r[1] for r in con.execute("pragma table_info(BaseItems)")}
need = {"Id", "ParentId", "Type", "Path", "IndexNumber", "ParentIndexNumber",
"ExtraType", "IsVirtualItem", "ProductionYear", "Data"}
if not need <= have:
sys.exit(f"{db} is not a Jellyfin library database (BaseItems has no"
f" {','.join(sorted(need - have))})")
node = {}
for i, pid, typ, name, year, data in con.execute(
"select Id, ParentId, Type, Name, ProductionYear, Data from BaseItems"
" where Type in (?,?,?)", JF_TV):
node[i] = (pid, typ, name, year, data)
def series_of(item_id):
"""Climb episode -> season -> series; empty when the chain is broken."""
for _ in range(4):
if item_id not in node:
return "", "", ""
pid, typ, name, year, data = node[item_id]
if typ.endswith(".Series"):
try:
tmdb = (json.loads(data or "{}")
.get("ProviderIds") or {}).get("Tmdb", "")
except ValueError:
tmdb = ""
return name or "", year or "", tmdb
item_id = pid
return "", "", ""
# The fleet's quadlets bind the share at /data, so the container's view of
# this library is /data/<the library's own folder name> unless told otherwise.
if not container:
container = "/data/" + os.path.basename(os.path.normpath(root))
prefix = container.rstrip("/") + "/"
written = skipped = 0
with open(out, "w", newline="") as fh:
w = csv.writer(fh, delimiter="\t", lineterminator="\n")
w.writerow(JF_COLS)
for iid, path, idx, pidx, extra, virt in con.execute(JF_SQL,
(JF_TV[0], prefix + "%")):
if virt or not path or not path.startswith(prefix):
skipped += 1 # a virtual item is a variant, not a file
continue
sname, syear, tmdb = series_of(iid)
w.writerow([root + "/" + path[len(prefix):],
sname.replace("\t", " "), syear,
"" if pidx is None else pidx,
"" if idx is None else idx,
(extra or "").replace("\t", " "), tmdb])
written += 1
print(f"episodes={written} virtual-or-blank={skipped} -> {out}")
def music(root, db, container, out):
"""The tracks Jellyfin has already tagged, for the library at root.
Jellyfin reads music metadata out of the file itself, so an Audio row IS the
tag read, done once, by the vendor's own parser: its album-artist, album,
year, disc and track columns are what the embedded tags say and its Name is
the track title. That makes this export the whole authority for a music
relayout — the path a ripper chose is not evidence of anything, and the
folder Jellyfin put a track under is often a release pack, not an album.
Two gaps this export answers from the row rather than the tree. Many tracks
sit under a folder that is not an album item at all (a disc folder, or a
mixed one), so the album name and year fall back to the tags when the album
row is absent. And an album's MusicBrainz ids are not in the row's own Data
— the current schema leaves that empty for music — they are rows of
BaseItemProviders keyed by the item's Id.
"""
try:
con = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
except sqlite3.Error as e:
sys.exit(f"cannot open the Jellyfin database read-only ({db}): {e}\n"
"It is a WAL database: copy it with its -wal and -shm beside it and"
" point --db at the copy. Never sudo it — the data dir belongs to"
" the container's owner, and rootless storage is per-user.")
have = {r[1] for r in con.execute("pragma table_info(BaseItems)")}
need = {"Id", "ParentId", "Type", "Path", "Name", "IndexNumber",
"ParentIndexNumber", "ProductionYear", "Album", "AlbumArtists",
"Artists"}
if not need <= have:
sys.exit(f"{db} is not a Jellyfin library database with music columns"
f" (BaseItems has no {','.join(sorted(need - have))})")
# Provider ids, best identity first: the release group is the album, a bare
# release id is one reissue of it and never overrides a group.
mb = {}
if "BaseItemProviders" in {r[0] for r in con.execute(
"select name from sqlite_master where type='table'")}:
rank = {p: i for i, p in enumerate(MUSIC_PROVIDERS)}
best = {}
for iid, pid, val in con.execute(
"select ItemId, ProviderId, ProviderValue from BaseItemProviders"
" where ProviderId in (?, ?)", MUSIC_PROVIDERS):
if iid not in best or rank[pid] < best[iid][0]:
best[iid] = (rank[pid], val)
mb = {k: v[1] for k, v in best.items()}
# The fleet's quadlets bind the share at /data, so the container's view of
# this library is /data/<the library's own folder name> unless told otherwise.
if not container:
container = "/data/" + os.path.basename(os.path.normpath(root))
prefix = container.rstrip("/") + "/"
written = skipped = 0
with open(out, "w", newline="") as fh:
w = csv.writer(fh, delimiter="\t", lineterminator="\n")
w.writerow(MUSIC_COLS)
for (path, aal, arts, tag_album, tag_year, disc, track, name, iid,
alb_name, alb_year, alb_id) in con.execute(
MUSIC_SQL, (JF_ALBUM, JF_AUDIO, prefix + "%")):
if not path or not path.startswith(prefix):
skipped += 1 # a row for a file outside this library
continue
# The album artist owns the folder; the track artist is the fallback
# for the compilations whose album-artist tag was left empty.
w.writerow([root + "/" + path[len(prefix):],
((aal or "").strip() or (arts or "").strip()),
(alb_name or tag_album or "").replace("\t", " "),
alb_year or tag_year or "",
"" if disc is None else disc,
"" if track is None else track,
(name or "").replace("\t", " "),
mb.get(alb_id or "") or mb.get(iid or "") or "",
(arts or "").replace("\t", " ")])
written += 1
print(f"tracks={written} outside-root-or-blank={skipped} -> {out}")
def audiobooks(root, db, out):
"""The chapters Plex matched inside the audiobook library, read-only.
Plex stores HOST paths here (unlike Jellyfin, which stores the container's
view), so no prefix mapping is needed: the part's `file` is the path the
operator has today. Rows for files outside `root` are skipped, because one
Plex library DB carries every library on the box.
The chapter's `index` is exported as an empty field when Plex has none. That
is not a gap to fill in here: three quarters of the books on this box contain
at least one un-indexed chapter, so the relayout that reads this file must
not renumber anything from it.
"""
uri = f"file:{db}?mode=ro&immutable=1"
try:
con = sqlite3.connect(uri, uri=True)
except sqlite3.Error as e:
sys.exit(f"cannot open the Plex database read-only ({db}): {e}\n"
"Plex runs rootless here: run this as the container's owner, never "
"with sudo — rootless storage is per-user, so root sees nothing.")
tables = {r[0] for r in con.execute(
"select name from sqlite_master where type='table'")}
missing = {"media_parts", "media_items", "metadata_items"} - tables
if missing:
sys.exit(f"{db} is not a Plex library database (no "
f"{', '.join(sorted(missing))})")
written = skipped = 0
with open(out, "w", newline="") as fh:
w = csv.writer(fh, delimiter="\t", lineterminator="\n")
w.writerow(AB_COLS)
for file, narrator, book, year, track, chapter in con.execute(
AB_SQL, (root.rstrip("/") + "/%",)):
if not file or not file.startswith(root.rstrip("/") + "/"):
skipped += 1 # a row for a file outside this library
continue
w.writerow([file.replace("\t", " "), narrator.replace("\t", " "),
book.replace("\t", " "), "" if not year else year,
"" if track is None or int(track) < 0 else track,
chapter.replace("\t", " ")])
written += 1
print(f"chapters={written} outside-root={skipped} -> {out}")
def main():
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
sub = ap.add_subparsers(dest="cmd", required=True)
@@ -190,13 +451,46 @@ def main():
x.add_argument("--root", **root)
x.add_argument("--db", default=PLEX_DB)
x.add_argument("--out", default="/tmp/plex-movies.tsv")
j = sub.add_parser("jellyfin",
help="export the episodes Jellyfin has numbered for the root")
j.add_argument("--root", default=os.environ.get("MEDIA_ROOT",
"/mnt/entropy/TVShows"),
help="TV library root on the host")
j.add_argument("--db", default=JF_DB)
j.add_argument("--container", default=None,
help="the in-container prefix that stands for --root"
" (default /data/<root's folder name>)")
j.add_argument("--out", default="/tmp/jellyfin-tv.tsv")
m = sub.add_parser("music",
help="export the tracks Jellyfin has tagged for the root")
m.add_argument("--root", default=os.environ.get("MEDIA_MUSIC_ROOT",
"/mnt/entropy/Music"),
help="music library root on the host")
m.add_argument("--db", default=JF_DB)
m.add_argument("--container", default=None,
help="the in-container prefix that stands for --root"
" (default /data/<root's folder name>)")
m.add_argument("--out", default="/tmp/jellyfin-music.tsv")
b = sub.add_parser("audiobooks",
help="export the chapters Plex matched for the audiobook root")
b.add_argument("--root", default=os.environ.get("MEDIA_AUDIOBOOKS_ROOT",
"/mnt/entropy/Audiobooks"),
help="audiobook library root on the host")
b.add_argument("--db", default=PLEX_DB)
b.add_argument("--out", default="/tmp/plex-audiobooks.tsv")
a = ap.parse_args()
if a.cmd == "enumerate":
enumerate_lib(a.root, a.out)
elif a.cmd == "probe":
probe(a.enum, a.out, a.jobs)
else:
elif a.cmd == "plex":
plex(a.root, a.db, a.out)
elif a.cmd == "music":
music(a.root, a.db, a.container, a.out)
elif a.cmd == "audiobooks":
audiobooks(a.root, a.db, a.out)
else:
jellyfin(a.root, a.db, a.container, a.out)
if __name__ == "__main__":
+445
View File
@@ -0,0 +1,445 @@
#!/usr/bin/env python3
r"""media-music.py — a plan that re-files the music library into Jellyfin's own shape.
media-music.py --root DIR --enum TSV --rows TSV [--out TSV] [--exceptions FILE]
Jellyfin's music contract (https://jellyfin.org/docs/general/server/media/music/)
is not the movie one. Jellyfin reads the metadata out of the file, so a folder is
an ALBUM and nothing else, every disc of an album belongs in that one folder, the
tags decide the track order, and a file name only matters when it carries a
character known to break playback (`< > : " / \ | ? *`) or when a lyric file has
to match the basename of the track it belongs to. The plan is therefore:
Music/<album artist>/<Album> (<year>)/NN - <Title>.<ext>
... the same folder, D.NN - <Title> when the album has discs
Music/Various Artists/... when its tracks name
more than one artist
The authority is the tag row Jellyfin already read — `scripts/media-census.py
music` — never the release-pack folder a ripper made. A file with no row is not
guessed at: it parks under `<root>/_unsorted` with the relative path it has today.
The rules below exist because of what a downloaded discography pack actually
contains:
* A cuesheet beside exactly one audio file means the album is one image with a
track list inside it. That file keeps its own name: numbering it `01` would
claim it is one track, and Jellyfin shows one track either way.
* Sidecars — images, cuesheets, EAC logs, checksums, booklets, playlists —
follow the album the audio in their own folder belongs to. A folder holding
two albums gives a sidecar no single owner, so it parks instead of being
guessed into one.
* A video container (.mkv, .vob, .ifo, …) inside a music library is not music.
Renaming it to .mka/.m4a would tell Jellyfin to index video as audio, so it
parks for a human decision.
* Playlists move with their album and their CONTENTS are never rewritten: the
paths inside an .m3u go stale. Editing a file the operator owns to keep a
convenience file tidy is not this tool's job.
Nothing is deleted and nothing is overwritten. The output is a list of rename(2)
intents for `scripts/media-apply.py`, whose preflight refuses any target that is
already claimed and whose journal is the rollback.
"""
import argparse
import csv
import os
import re
import sys
from collections import Counter, defaultdict
# What Jellyfin will index as music. .mp4/.mkv/.webm are deliberately NOT here:
# the vendor doc says an audio-only stream in those containers is never read as
# music, and this tool renames for a reason it can prove, not to guess at one.
AUDIO_EXT = {".flac", ".mp3", ".m4a", ".m4b", ".ogg", ".oga", ".opus", ".wav",
".wma", ".aiff", ".aif", ".ape", ".dsf", ".dff", ".wv", ".mka",
".mpc", ".aac"}
VIDEO_EXT = {".mkv", ".vob", ".ifo", ".bup", ".mpg", ".mpeg", ".avi", ".mov",
".ts", ".m2ts", ".wmv", ".webm", ".mp4", ".m4v", ".iso", ".img",
".bdmv", ".divx", ".ogm"}
# A lyric has to carry the basename of its track, so it is named after the audio
# file and not after itself. .txt counts only when its stem matches an audio stem:
# a loose .txt in an album folder is a note.
LYRIC_EXT = {".lrc", ".elrc", ".txt"}
IMAGE_EXT = {".jpg", ".jpeg", ".png", ".webp", ".gif", ".bmp", ".tiff"}
# The album-artist strings that mean "compilation", so such an album is not filed
# under one artist who does not own it.
VARIOUS = {"various", "various artists", "unknown", "unknown artist", "unknow"}
# The characters Jellyfin documents as breaking playback. They are dropped, not
# substituted: a replacement character is a guess at what the title meant.
BAD_CHARS = set('<>:"/\\|?*')
TRASH_DIR = re.compile(r"(^|/)(\.AGENTTRASH|\.thumbnails|\.Trash-\d+|@eaDir"
r"|lost\+found)(/|$)")
ARGS = None
HELD = {} # path -> (bytes, ext)
ROWS = {} # path -> the tag row
PLACED = {} # audio path -> the destination the plan gave it
CLAIMED = set() # every destination path the plan has handed out
MOVES, KEEPS = [], []
PARKED = defaultdict(list) # reason -> [path]
NOTES = defaultdict(list) # section -> [line]
CUE_DIRS = set() # source dirs holding a .cue
AUDIO_BY_DIR = defaultdict(set) # source dir -> audio files a row names
CHILD_OF = defaultdict(set) # dir -> the dirs directly below it holding audio
FOLDERS_SEEN = {} # (dir, depth) -> album folders placed below it
SUBTREE_DEPTH = 2 # how far below a folder its audio may sit
ALBUMS = {} # (artist, album, year) -> the album group
AMBIGUOUS = object()
def dominant(values):
"""The spelling the tags mostly agree on: the most frequent value, ties broken
by sort so the answer never depends on the order rows arrived in."""
counts = Counter(values)
return min(counts.items(), key=lambda kv: (-kv[1], kv[0]))[0]
def rel(path):
"""The path as seen from the library root."""
return os.path.relpath(path, ARGS.root)
def safe(name):
"""A name every client can carry: no reserved character, nothing
unprintable, no trailing dot or space, and short enough to rename."""
out = "".join(c for c in (name or "")
if c not in BAD_CHARS and (c.isprintable() or c == " "))
out = re.sub(r"\s+", " ", out).strip().rstrip(". ")
while out and len(out.encode("utf-8", "ignore")) > 200:
out = out[:-4].rstrip()
return out
def assign(src, folder, name, ext, kind, note):
"""Book one file into `folder` under `name`, renamed if that exact destination
is already taken — by another file in the plan, or by a file standing there
today. Two files claiming one path is the fault that costs a re-run of a 90k
plan, and so is a plan that only works if the renames happen in the right
order: the executor refuses any target that is occupied, so an occupied path
is never a destination, even when its occupant is scheduled to leave.
A file already standing where the contract puts it is a keep, not a move."""
stem = safe(name) or "file"
dst = os.path.join(folder, stem + ext)
if dst == src:
CLAIMED.add(dst.lower()) # it is standing on its own target
KEEPS.append((src, dst, kind))
return dst
# A file already in this folder under this name — or under the numbered
# variant the collision rule gives it — keeps the name it has. Renumbering it
# every run would turn the plan into a shuffle instead of a fixed point.
base = os.path.splitext(os.path.basename(src))[0]
if os.path.dirname(src) == folder and (base == stem
or base.startswith(stem + " (")):
CLAIMED.add(src.lower())
KEEPS.append((src, src, kind))
return src
n = 1
while dst.lower() in CLAIMED:
n += 1
dst = os.path.join(folder, f"{stem} ({n}){ext}")
if n > 1:
NOTES[note].append(rel(dst))
CLAIMED.add(dst.lower())
MOVES.append((src, dst, kind))
return dst
def park(path, reason):
"""Hold a file where it is, under the park folder, keeping its relative path."""
PARKED[reason].append(path)
dst = os.path.join(ARGS.park_dir, rel(path))
if dst != path:
CLAIMED.add(dst.lower())
MOVES.append((path, dst, "park"))
def under_park(path):
"""True for a file already parked: that pile is for a human to review, and
re-parking it every run would only push it deeper."""
return path.startswith(ARGS.park_dir + os.sep)
# ---------------------------------------------------------------------- albums
def group_albums():
"""Every tagged track into one group per (album-artist tag, album, year).
The key is the identity the tags assert, not the folder the file sits in: one
album arrives as three discs in three folders, and two copies of an album in
one pack are one album to Jellyfin, whose album item is that name, artist and
year. The artist in the key is the ALBUM-artist tag and never the performer,
because a compilation is one album whose tracks name many performers — keying
on the performer would file one album once per artist.
"""
for path, r in sorted(ROWS.items()):
if path not in HELD or under_park(path):
continue
d = os.path.dirname(path)
AUDIO_BY_DIR[d].add(path)
CHILD_OF[os.path.dirname(d)].add(d) # so a cover above CD1/CD2 sees both
tag = (r["artist"] or "").strip()
album = (r["album"] or "").strip()
performer = (r["performer"] or "").strip()
if not album:
park(path, "a track no album names")
continue
if not tag and not performer:
park(path, "a track no artist names")
continue
key = (tag.casefold(), album.casefold(), (r["year"] or "").strip())
g = ALBUMS.setdefault(key, {"tags": [], "albums": [], "year": key[2],
"members": [], "performers": set(),
"mb": set()})
g["members"].append(path)
g["tags"].append(tag)
g["albums"].append(album)
g["performers"].add(performer or tag)
if r.get("mb"):
g["mb"].add(r["mb"])
def folder_artist(g):
"""Whose shelf the album sits on. The album-artist tag owns it when it names
one; when the tags name only performers, a single performer names the album
and several make it a compilation, which belongs to no one of them. The key is
case-folded, so the spelling that reaches the folder is the one the tracks
mostly carry — see dominant()."""
tags = [t for t in g["tags"] if t]
if tags:
name = dominant(tags)
return "Various Artists" if name.casefold() in VARIOUS else name
if len({p.casefold() for p in g["performers"]}) == 1:
return dominant(g["performers"])
return "Various Artists"
def album_folder(g):
name = safe(dominant(g["albums"])) or "Untitled"
return os.path.join(ARGS.root, safe(folder_artist(g)) or "Unknown Artist",
f"{name} ({g['year']})" if g["year"] else name)
def is_cue_image(path):
"""A folder holding this one audio file and a cuesheet is an album image."""
d = os.path.dirname(path)
return len(AUDIO_BY_DIR.get(d, ())) == 1 and d in CUE_DIRS
def number_of(value, width):
return f"{int(value):0{width}d}" if str(value or "").isdigit() else ""
def place_albums():
for key in sorted(ALBUMS):
g = ALBUMS[key]
folder = album_folder(g)
discs = {g0["disc"] for g0 in (ROWS[p] for p in g["members"]) if g0["disc"]}
tracks = [int(ROWS[p]["track"]) for p in g["members"]
if str(ROWS[p]["track"] or "").isdigit()]
# More than one disc, or a disc numbered past the first: the track number
# alone cannot name a file uniquely, so the disc goes in front of it.
multi = len(discs) > 1 or any(str(d).isdigit() and int(d) > 1 for d in discs)
width = 3 if multi or (max(tracks, default=0) > 99) else 2
for path in sorted(g["members"]):
r = ROWS[path]
ext = HELD[path][1]
base = os.path.splitext(os.path.basename(path))[0]
if is_cue_image(path):
name = safe(base) or "image"
else:
title = safe(r["title"] or base) or "untitled"
num = number_of(r["track"], 2 if multi else width)
if multi and num:
num = f"{r['disc'] or '1'}.{num}"
name = f"{num} - {title}" if num else title
PLACED[path] = assign(path, folder, name, ext, "track",
"two tracks claimed one name inside one album")
# --------------------------------------------------------------------- sidecars
def folders_below(d, depth):
"""The album folders the audio in `d`, or within `depth` directories below it,
was placed in. Memoised, because every loose file in a big discography folder
asks the same question about it."""
if (d, depth) not in FOLDERS_SEEN:
seen = {os.path.dirname(PLACED[a]) for a in AUDIO_BY_DIR.get(d, ())
if a in PLACED}
if depth:
for child in CHILD_OF.get(d, ()):
seen |= folders_below(child, depth - 1)
FOLDERS_SEEN[(d, depth)] = seen
return FOLDERS_SEEN[(d, depth)]
def owner_folder(d):
"""The album folder that owns this directory — the first place up the tree
where audio sits, in the directory itself or within SUBTREE_DEPTH levels below
it, which is how a box with its discs in CD1/CD2 owns the cover lying above
them. None when no audio is near enough to say, AMBIGUOUS when the audio there
belongs to more than one album."""
while d.startswith(ARGS.root + os.sep) and d != ARGS.root:
folders = folders_below(d, SUBTREE_DEPTH)
if folders:
return next(iter(folders)) if len(folders) == 1 else AMBIGUOUS
d = os.path.dirname(d)
return None
def place_sidecars():
"""Everything that is not a tagged track: lyrics, images, cuesheets, logs,
booklets, playlists — and the clutter that is none of those."""
# The lyric rule needs the audio's stem as it is named TODAY, before the plan
# renames it, because that is what the sidecar's own name was copied from.
stem = {(d, os.path.splitext(os.path.basename(a))[0]): a
for d, audios in AUDIO_BY_DIR.items() for a in audios}
for path, (_size, ext) in sorted(HELD.items()):
if path in ROWS or under_park(path) or TRASH_DIR.search(path):
continue
d, base = os.path.dirname(path), os.path.splitext(os.path.basename(path))[0]
if ext in VIDEO_EXT:
park(path, "a video container inside the music library: Jellyfin will "
"not read it as music, and renaming it would claim it is")
elif ext in AUDIO_EXT:
park(path, "an audio file no provider row names — its tags are not in "
"the library Jellyfin has matched")
elif ext in (LYRIC_EXT | IMAGE_EXT) and (d, base) in stem:
# A lyric, or a picture named after its track: it follows that track
# and takes the name the track now has, or Jellyfin cannot find it.
audio = stem[(d, base)]
dst = PLACED.get(audio)
if dst:
folder, name = os.path.dirname(dst), os.path.splitext(
os.path.basename(dst))[0]
assign(path, folder, name, ext, "lyric",
"a sidecar whose name collided with another in its album")
else:
park(path, "a sidecar of a track the plan could not place")
else:
owner = owner_folder(d)
if owner is AMBIGUOUS:
park(path, "a sidecar whose folder holds more than one album, so "
"nothing says which one it belongs to")
elif owner:
assign(path, owner, base, ext,
"playlist" if ext in (".m3u", ".m3u8") else "sidecar",
"one album folder would have carried two files of the same "
"sidecar name")
else:
park(path, "not music, or music with nothing beside it to belong to")
# ---------------------------------------------------------------------- outputs
def write_plan():
"""mkdir rows for every destination folder and its whole chain, then the moves.
Parents first: the executor creates directories in the order they are listed."""
chain = set()
for _s, dst, _k in MOVES:
d = os.path.dirname(dst)
while d and d != ARGS.root and d != os.path.dirname(d):
chain.add(d)
d = os.path.dirname(d)
dirs = sorted(chain, key=lambda p: (p.count("/"), p))
with open(ARGS.out, "w") as fh:
fh.write("action\tsrc\tdst\tkind\n")
for d in dirs:
fh.write(f"mkdir\t\t{d}\tdir\n")
for src, dst, kind in sorted(MOVES, key=lambda m: (m[1], m[0])):
fh.write(f"move\t{src}\t{dst}\t{kind}\n")
for src, dst, kind in sorted(KEEPS, key=lambda m: (m[1], m[0])):
fh.write(f"keep\t{src}\t{dst}\t{kind}\n")
parked = [m for m in MOVES if m[2] == "park"]
sized = sum(HELD[s][0] for s, _d, _k in MOVES if s in HELD)
print(f"plan: {len(MOVES) - len(parked)} moves, {len(parked)} parked"
f" ({sized / 2**40:.2f} TiB touched), {len(KEEPS)} already placed,"
f" {len(dirs)} folders -> {ARGS.out}")
def write_exceptions():
with open(ARGS.exceptions, "w") as fh:
fh.write(f"Music relayout exceptions for {ARGS.root}\n\n")
fh.write(f"Everything below is parked under {ARGS.park_dir}, keeping the\n"
"relative path it has today. Nothing is deleted and nothing is\n"
"overwritten: a parked file is one rename from wherever the plan is\n"
"pointed next.\n\n")
for reason in sorted(PARKED, key=lambda r: -len(PARKED[r])):
rs = PARKED[reason]
fh.write(f"{len(rs):>6} files {reason}\n")
for p in sorted(rs, key=rel)[:400]:
fh.write(f" {rel(p)}\n")
if len(rs) > 400:
fh.write(f" … and {len(rs) - 400} more\n")
fh.write("\n")
for section in sorted(NOTES, key=lambda s: -len(NOTES[s])):
lines = sorted(set(NOTES[section]))
fh.write(f"{len(lines):>6} {section}\n")
for line in lines[:200]:
fh.write(f" {line}\n")
if len(lines) > 200:
fh.write(f" … and {len(lines) - 200} more\n")
fh.write("\n")
no_year = sorted(f"{folder_artist(g)}/{dominant(g['albums'])}"
for g in ALBUMS.values() if not g["year"])
if no_year:
fh.write(f"{len(no_year):>6} albums carry no year in any tag, so their folders\n"
" are named without one; Jellyfin still matches them.\n")
fh.write(" " + ", ".join(no_year[:300]) + "\n\n")
clash = sorted(f"{folder_artist(g)}/{dominant(g['albums'])}"
for g in ALBUMS.values() if len(g["mb"]) > 1)
if clash:
fh.write(f"{len(clash):>6} albums put one folder name on more than one\n"
" MusicBrainz release group — different releases, one name.\n"
" They share the folder Jellyfin's own album item names.\n")
fh.write(" " + ", ".join(clash[:200]) + "\n\n")
fh.write(" Playlists move with their album and their contents are never\n"
" rewritten: the paths inside an .m3u go stale.\n")
print(f"exceptions -> {ARGS.exceptions}")
def main():
global ARGS
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("--root", default=os.environ.get("MEDIA_MUSIC_ROOT",
"/mnt/entropy/Music"))
ap.add_argument("--enum", default="/tmp/music-enum.tsv",
help="the enumerate TSV (bytes, ext, path)")
ap.add_argument("--rows", default="/tmp/jellyfin-music.tsv",
help="the tag rows scripts/media-census.py music wrote")
ap.add_argument("--out", default="/tmp/music-plan.tsv")
ap.add_argument("--exceptions", default="/tmp/music-exceptions.txt")
ap.add_argument("--park", default="_unsorted",
help="the folder inside the root that holds what cannot be filed")
ARGS = ap.parse_args()
ARGS.root = os.path.normpath(ARGS.root)
ARGS.park_dir = os.path.join(ARGS.root, ARGS.park)
read_inputs()
# Every path a file occupies today is claimed before a single destination is
# handed out: the executor refuses an occupied target, so a plan that aimed at
# one would only fail after the renames that were supposed to clear it.
CLAIMED.update(p.lower() for p in HELD)
for path, (_size, ext) in HELD.items():
if ext == ".cue":
CUE_DIRS.add(os.path.dirname(path))
group_albums()
place_albums()
place_sidecars()
write_plan()
write_exceptions()
def read_inputs():
with open(ARGS.enum) as fh:
for line in fh:
parts = line.rstrip("\n").split("\t")
if len(parts) >= 3:
HELD[parts[2]] = (int(parts[0]), parts[1].lower())
with open(ARGS.rows, newline="") as fh:
for row in csv.DictReader(fh, delimiter="\t"):
if row.get("file"):
ROWS[row["file"]] = row
gone = [p for p in ROWS if p not in HELD]
if gone:
NOTES["a tag row whose file the census no longer sees"].extend(gone)
if __name__ == "__main__":
main()
+705
View File
@@ -0,0 +1,705 @@
#!/usr/bin/env python3
r"""Plan a Jellyfin-format relayout of a TV library. DRY RUN ONLY: nothing moves.
media-shows.py [--root DIR] [--enum TSV] [--titles TSV] [--probed TSV]
[--tmdb-cache JSON] [--tmdb] [--movies DIR] [--park DIR]
[--out TSV] [--exceptions TXT]
Inputs are the TSVs scripts/media-census.py exports: `enumerate` lists what
exists, `jellyfin` says which episode each file IS, and `probe` is optional
duration evidence for the movie rule. The TMDb cache is a plain JSON file so a
plan stays reproducible: answers are read from it, and a miss is looked up only
with --tmdb and TMDB_API_KEY in the environment (the key never enters argv, and
never the cache).
Target contract (jellyfin.org/docs/general/server/media/shows):
<root>/<Series> (<year>)/Season <NN>/<Series> SxxEyy.ext
one file holding two episodes SxxE01-E02 specials Season 00
a documented part of an episode SxxE03 Part 1 sidecars <base>.<lang>.srt
bonus material <series>/trailers|featurettes|… — never a season
PLAN is the TSV scripts/media-apply.py replays: `action src dst kind`, every
mkdir row before any move, and `keep` for a file already where the contract puts
it. EXCEPTIONS is the review list: everything parked, and why.
The authority chain, strongest first, for one video file:
1. a provider row — Jellyfin matched this exact path and numbered it.
2. the file's own SxxEyy / 1x04 marker, the series taken from a sibling's
provider row, else from the nearest ancestor folder that has one, else from
the release folder's own words.
3. nothing. An unnumbered file is PARKED under --park with its relative path
kept: an invented `Season 01` steals an episode number from the file that
really is S01E01, and a wrong number is worse than no number.
Guards, and why each is here:
* One series, one folder, one year: the year ANY provider row of a series
states is shared by all of them, so a box set split across five release
folders lands inside ONE series folder instead of five.
* Two files claiming one episode: the bigger keeps the standard name, the other
parks with the reason. Nothing is overwritten and nothing is deleted — which
encode survives is Frank's call, and a parked file is one rename from home.
* A file with no episode evidence that runs long and that the MOVIE provider
names leaves the TV library for --movies under the movie naming contract.
Long and unnamed parks; short and unnumbered is bonus material.
* Series art and the show nfo follow the SERIES, a season image follows its
SEASON, a subtitle or episode nfo follows the episode's new base — an episode
moved without its subtitles is a half-move.
* Hidden files, macOS forks, qBittorrent resume parts and the park pile itself
are never moved. Skipping the park is what makes re-planning stable.
The plan crosses three trees (the library, its park, the movie library), so the
executor runs with --root at the share that holds them, on its own journal —
one journal is one undo:
python3 scripts/media-shows.py --root /mnt/entropy/TVShows \
--enum /tmp/tv-enum.tsv --titles /tmp/jellyfin-tv.tsv \
--out /mnt/entropy/media-tv-plan.tsv
python3 scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/media-tv-relayout.jsonl
"""
import argparse
import collections
import importlib.util
import json
import os
import re
import sys
import urllib.parse
import urllib.request
ARGS = argparse.ArgumentParser(description=__doc__.splitlines()[0],
formatter_class=argparse.RawDescriptionHelpFormatter)
ARGS.add_argument("--root", default=os.environ.get("TV_ROOT", "/mnt/entropy/TVShows"),
help="the TV library the plan reorganises")
ARGS.add_argument("--enum", default="/tmp/tv-enum.tsv",
help="file census (media-census.py enumerate)")
ARGS.add_argument("--titles", default="/tmp/jellyfin-tv.tsv",
help="the provider's numbering (media-census.py jellyfin)")
ARGS.add_argument("--probed", default="",
help="ffprobe census for the movie rule; absent = no durations")
ARGS.add_argument("--tmdb-cache", default="/tmp/tmdb-cache.json",
help="TMDb answers as an input file, so a plan is reproducible")
ARGS.add_argument("--tmdb", action="store_true",
help="look up cache misses (needs TMDB_API_KEY)")
ARGS.add_argument("--movies", default="/mnt/entropy/Movies",
help="movie library a mis-filed film moves into")
ARGS.add_argument("--park", default="",
help="review pile (default: <root>/_unsorted)")
ARGS.add_argument("--out", default="/tmp/tv-plan.tsv")
ARGS.add_argument("--exceptions", default="/tmp/tv-exceptions.txt")
ARGS = ARGS.parse_args()
ROOT = os.path.abspath(ARGS.root)
PARK = os.path.abspath(ARGS.park or os.path.join(ROOT, "_unsorted"))
MOVIES = os.path.abspath(ARGS.movies)
ENGINE = os.path.join(os.path.dirname(os.path.abspath(__file__)), "media-titles.py")
_spec = importlib.util.spec_from_file_location("eng", ENGINE)
eng = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(eng)
try:
CACHE = json.load(open(ARGS.tmdb_cache))
except (OSError, ValueError):
CACHE = {}
API_KEY = os.environ.get("TMDB_API_KEY", "")
VIDEO_EXT = {".mkv", ".mp4", ".avi", ".m4v", ".mov", ".ts", ".m2ts", ".mpg",
".mpeg", ".webm", ".wmv", ".divx", ".ogm", ".flv", ".vob", ".rm",
".rmvb", ".iso", ".img"}
SUB_EXT = {".srt", ".ass", ".ssa", ".sub", ".vtt"}
IMG_EXT = {".jpg", ".jpeg", ".png", ".webp", ".bmp", ".gif", ".tbn"}
NFO_EXT = {".nfo", ".xml"}
# The container's own word, for a sidecar named "episode.mkv.srt": that component
# is not a language flag, and both matchers read the plain form better.
CONTAINER = {e[1:] for e in VIDEO_EXT}
# Files that are never anybody's business: hidden, forks, resume state, thumbnails.
SKIP_NAME = re.compile(r"^(\.|#|dl_state|@eaDir|Thumbs\.db|desktop\.ini)|\.!q$")
# Show-level metadata keeps its own name at the series root; season-level at the
# season root; folder.jpg stays with the folder it was written for.
SERIES_META = re.compile(r"^(tvshow|series|show)(\.nfo|\.xml)?$"
r"|^(poster|tvposter|banner|clearlogo|clearart|logo|art"
r"|landscape|fanart|showbackdrop|backdrop)\d*\."
r"(jpg|jpeg|png|webp|bmp)$", re.I)
SEASON_META = re.compile(r"^season\.nfo$", re.I)
FOLDER_IMAGE = re.compile(r"^folder\.(jpg|jpeg|png|webp|bmp)$", re.I)
SEASON_DIR_RE = re.compile(r"(?i)^\s*(?:season[\s._-]*(\d{1,3})|s(\d{1,2}))\s*$")
BAD_CHARS = re.compile(r'[<>:"/|?*]')
# SxxEyy with an optional second episode, plus the 1x04 form. The second number
# is guarded so a resolution never reads as an episode: "S01E01-1080p" is ONE
# episode, and the longest-match backtracking of \d{1,3} is what the (?![\dpP])
# after it closes off.
EP_RANGE = re.compile(r"(?i)(?<![a-z0-9])s(?:eason[\s._-]*)?(\d{1,2})[\s._-]*"
r"e(?:p(?:isode)[\s._-]*)?(\d{1,3})"
r"(?:[\s._-]*(?:-|to[\s._-]*|ep)[\s._-]*(?:s\d{1,2}[\s._-]*)?e?"
r"(\d{1,3})(?![\dpP]))?")
NX_RE = re.compile(r"(?i)(?<![a-z0-9])(\d{1,2})x(\d{2,3})(?![a-z0-9])")
PART_RE = re.compile(r"(?i)(?:^|[\s._-])(?:part|pt|cd|dvd|disc|disk)[\s._-]?"
r"([1-9]|i{1,3})(?:$|[\s._-])")
# Jellyfin reads these subfolder names as bonus material. The planner emits only
# these spellings, so 40k extras end up in nine folders instead of nine hundred.
EXTRA_DIRS = [
("trailers", r"trailer|teaser|promo|advert|commercial|tv[\s._-]spot|preview"),
("deleted scenes", r"deleted"),
("behind the scenes", r"behind[\s._-]the[\s._-]scenes|making[\s._-]of|bts"
r"|b[\s._-]roll|gag[\s._-]reel|blooper|outtake|archival"
r"|video[\s._-]archive|scene[\s._-]selection|menu|gallery"
r"|stills?|visual[\s._-]effects|workprint"),
("interviews", r"interview|cast[\s._-]and[\s._-]crew|audition|commentary"),
("scene cuts", r"scene[\s._-]cuts?|scenecuts"),
("featurettes", r"featurettes?"),
("shorts", r"shorts?"),
("clips", r"clips?"),
]
# Jellyfin's own ExtraType values, folded onto the same folder names.
EXTRA_KIND = {"trailer": "trailers", "tmbd": "trailers", "intro": "other",
"movie": "other", "scene": "scene cuts", "clip": "clips",
"interview": "interviews", "behindthescenes": "behind the scenes",
"deleted": "deleted scenes", "other": "other", "epilogue": "other",
"opening": "other", "featurette": "featurettes", "short": "shorts"}
SHORT_MAX = 45 * 60 # over this, a file with no episode evidence is film-shaped
def rel(p):
return p[len(ROOT) + 1:] if p.startswith(ROOT + "/") else p
def clean(name):
"""A name the filesystem and both matchers can read."""
name = BAD_CHARS.sub("-", name).replace("\\", "-").strip(" .")
return name[:180] or "untitled"
def title_of(text):
"""The engine's parser, applied to a name that has dots but no extension:
splitext would read "The.Examples" as stem "The", so a sentinel extension is
added and the parser sees an ordinary media name."""
return eng.title_of(text + ".mkv")
def markers(name):
"""(season, [episodes], position) the episode markers a name states; None when
it states none. The position is where the series' own words stop."""
stem = os.path.splitext(name)[0]
m = EP_RANGE.search(stem)
if m:
eps = [int(m.group(2))] + ([int(m.group(3))] if m.group(3) else [])
return int(m.group(1)), eps, m.start()
m = NX_RE.search(stem)
if m:
return int(m.group(1)), [int(m.group(2))], m.start()
return None
LEAD_IDX_RE = re.compile(r"^\s*(\d{1,3})\s*[-\u2013.\s]\s*[A-Za-z]")
PACK_IDX_RE = re.compile(r"^\s*\d+[a-z]?[.)]\s+")
def num(x):
try:
return int(x)
except (TypeError, ValueError):
return 0
def folder_season(path):
"""The season the FOLDER states (`Season 3`, `S3`). A human wrote it, and where
a scraped row disagrees this is the one that survives a re-plan."""
m = SEASON_DIR_RE.match(os.path.basename(os.path.dirname(path)))
return int(next(g for g in m.groups() if g)) if m else None
def leading_index(name):
"""The number a file leads with inside a season folder — `01 Kyle MacLachlan,
Sinead OC.avi` is episode 1. Both matchers read it that way."""
m = LEAD_IDX_RE.match(os.path.splitext(name)[0])
return int(m.group(1)) if m else None
def tmdb(kind, query):
"""A provider answer for a query: the cache first, TMDb once, then never again."""
ck = f"{kind}:{query}"
if ck in CACHE:
hit = CACHE[ck]
# A cache an operator edited by hand can carry the year as a number; the
# comparison below is text, so the wire form is imposed on every hit.
return None if hit is None else {**hit, "year": str(hit.get("year") or "")}
if not (ARGS.tmdb and API_KEY):
return None
path = "movie" if kind == "movie" else "tv"
url = ("https://api.themoviedb.org/3/search/" + path + "?query="
+ urllib.parse.quote(query) + "&include_adult=false")
req = urllib.request.Request(url, headers={
# The key rides a header, never the URL: a token in argv is world-readable.
"Authorization": "Bearer " + API_KEY,
"Accept": "application/json"})
try:
with urllib.request.urlopen(req, timeout=20) as r:
found = json.load(r).get("results") or []
except Exception as e: # noqa: BLE001
print(f"tmdb {kind} lookup failed for {query!r}: {e}", file=sys.stderr)
return None
if not found:
CACHE[ck] = None
return None
top = found[0]
CACHE[ck] = {"name": (top.get("title") or top.get("name") or "").strip(),
"year": str(top.get("year") or "") or (top.get("first_air_date") or "")[:4],
"id": top.get("id")}
return CACHE[ck]
# ------------------------------------------------------------------- the inputs
rows = []
for line in open(ARGS.enum):
size, ext, path = line.rstrip("\n").split("\t")
p = os.path.abspath(path)
base = os.path.basename(p)
if not p.startswith(ROOT + "/") or SKIP_NAME.search(base):
continue
if p.startswith(PARK + "/"):
continue # the park is already reviewed; re-parking is churn
rows.append({"bytes": int(size), "ext": ext.lower(), "path": p, "base": base})
held = {r["path"]: r for r in rows}
titles = {}
with open(ARGS.titles) as fh:
cols = fh.readline().rstrip("\n").split("\t")
for line in fh:
f = line.rstrip("\n").split("\t")
f += [""] * (len(cols) - len(f))
row = dict(zip(cols, f))
p = os.path.abspath(row.get("file", ""))
if p in held:
titles[p] = row
dur = {}
if ARGS.probed and os.path.exists(ARGS.probed):
for line in open(ARGS.probed):
f = line.rstrip("\n").split("\t")
if len(f) > 2:
try:
dur[os.path.abspath(f[1])] = float(f[2] or 0)
except ValueError:
pass
video = [r for r in rows if r["ext"] in VIDEO_EXT]
other = [r for r in rows if r["ext"] not in VIDEO_EXT]
# What a folder already knows: one provider row names the series for every sibling
# that has no row of its own — that is how an anime pack gets its show.
by_dir = collections.defaultdict(list)
for r in video:
by_dir[os.path.dirname(r["path"])].append(r)
dir_series = {}
for d, rs in by_dir.items():
names = collections.Counter(titles[r["path"]]["series"] for r in rs
if r["path"] in titles and titles[r["path"]]["series"])
if names:
dir_series[d] = names.most_common(1)[0][0]
# The registry: every series a provider names, judged before any folder is. Folder
# words may resolve a file to one of these names; they may never invent one. That is
# the whole difference between "NCOP 01.mkv" inside the Gurren Lagann pack, which the
# pack names, and a new root folder called "01 NCOP".
registry = {}
for _p, _t in titles.items():
if _t["series"]:
registry.setdefault(eng.norm(_t["series"]), _t["series"])
def known_in(text):
"""The registered series named inside this text: the longest registered name
whose words appear in it as a run. "(CBB) Tengen Toppa Gurren Lagann - The
Complete Series (BD 1080p)" names "Tengen Toppa Gurren Lagann"."""
hay = " " + eng.norm(text) + " "
best = ""
for k, name in registry.items():
if len(k) >= 5 and f" {k} " in hay and len(k) > len(best):
best = name
return best
# Rows also name their own folders' ancestors, so an already-filed library stays
# where it is when it is re-planned.
for d, name in sorted(dir_series.items()):
a = os.path.dirname(d)
while a and a != ROOT and len(a) > len(ROOT):
if a not in dir_series:
got = known_in(os.path.basename(a))
if got:
dir_series[a] = got
a = os.path.dirname(a)
def series_of_dir(d):
"""The series an ancestor folder belongs to, nearest first: the folder holding
provider rows, else a folder whose own words name a registered show."""
while d and d != ROOT and len(d) > len(ROOT):
if d in dir_series:
return dir_series[d]
got = known_in(os.path.basename(d))
if got:
return got
d = os.path.dirname(d)
return ""
def extras_kind(path, extra_type):
"""The canonical bonus subfolder, from Jellyfin's own extra type first."""
if extra_type:
return EXTRA_KIND.get(eng.norm(extra_type).replace(" ", ""), "other")
for d in rel(path).split("/")[:-1]:
if eng.folder_is_bonus(d):
for name, rx in EXTRA_DIRS:
if re.search(rx, d, re.I):
return name
return "extras"
stem = os.path.splitext(os.path.basename(path))[0]
for name, rx in EXTRA_DIRS:
if re.search(rx, stem, re.I):
return name
return "extras"
# ----------------------------------------------------------- classify every file
series = {} # norm(name) -> {name, year, tmdb}
claimants = [] # (row, name, season, [episodes])
extras = [] # (row, name, bonus folder)
films = [] # film-shaped, no episode evidence
unknown = [] # (row, reason)
def series_name(name):
"""The provider's name as a folder: the pack index a scene release leads with
("4a. Seed DESTINY") falls away, and the doubled spaces it leaves fold."""
return clean(re.sub(r"\s{2,}", " ", PACK_IDX_RE.sub("", name)).strip(" ."))
def note(name, year="", tmdb_id=""):
"""Register a series; the first name wins, the first year and id stick."""
k = eng.norm(name)
s = series.setdefault(k, {"name": series_name(name), "year": "", "tmdb": ""})
if year and not s["year"]:
s["year"] = str(year)
if tmdb_id and not s["tmdb"]:
s["tmdb"] = str(tmdb_id)
def as_extra(r, owner, extra_type):
"""Bonus material needs a series to file under; with none it is parked."""
if not owner:
unknown.append((r, "bonus material with no series to own it"))
return
note(owner)
extras.append((r, owner, extras_kind(r["path"], extra_type)))
for r in video:
p, t = r["path"], titles.get(r["path"])
marked = markers(r["base"])
fs = folder_season(p)
if t and t["extra"]:
as_extra(r, t["series"] or series_of_dir(os.path.dirname(p)), t["extra"])
continue
row = bool(t and t["series"] and (t["season"] or t["episode"]))
if marked or row:
# The provider row names the SHOW: its registered name, its year. The number
# that orders the file comes from the file's own name first, then the folder
# it sits in, then the row — scraped pack rows number episodes badly, where
# every file under SNL 16-20/Season 16..19 carries season 16 episode 20.
owner = (t["series"] if t and t["series"]
else series_of_dir(os.path.dirname(p))
or title_of(r["base"][:marked[2]])[0])
if not eng.norm(owner).replace(" ", ""):
unknown.append((r, "an episode marker with no series name to hang it on"))
continue
if marked:
season, eps = marked[0], marked[1]
else:
season = fs if fs is not None else num(t["season"])
ep = leading_index(r["base"]) or num(t["episode"])
if not 0 < season <= 200 or ep <= 0:
unknown.append((r, f"a provider row of season {season} episode {ep}"))
continue
eps = [ep]
claimants.append((r, owner, season, eps))
note(owner, t["year"] if t else "", t["tmdb"] if t else "")
continue
owner = series_of_dir(os.path.dirname(p))
if fs is not None and owner and leading_index(r["base"]):
claimants.append((r, owner, fs, [leading_index(r["base"])]))
note(owner)
continue
if eng.is_extra(p, title_of(r["base"])[0], dur.get(p, 0.0), ROOT):
as_extra(r, owner, "")
continue
if dur.get(p, 0.0) >= SHORT_MAX or (p not in dur and title_of(r["base"])[1] != "?"):
films.append(r)
continue
unknown.append((r, "no episode marker, no provider row, nothing long enough to be a film"))
# A provider that never states a year still has one: the search is exact, so a
# miss parks nothing — the folder simply carries no year, which Jellyfin accepts.
for k, s in series.items():
if not s["year"]:
got = tmdb("tv", s["name"])
if got and eng.norm(got["name"]) == k:
s["year"] = got["year"]
if ARGS.tmdb:
with open(ARGS.tmdb_cache, "w") as fh:
json.dump(CACHE, fh, indent=1, sort_keys=True)
def series_dir(name):
s = series.get(eng.norm(name))
if not s:
s = {"name": clean(name), "year": ""}
label = f"{s['name']} ({s['year']})" if s["year"] else s["name"]
return f"{ROOT}/{clean(label)}"
# ------------------------------------------------------------------- the plan
moves, keeps, parked = [], [], []
reasons = collections.defaultdict(list)
used = {} # dst -> the file that claimed it
def book(r, dst_dir, name, kind):
"""Claim one destination for one file, or park the file when it is taken."""
dst = f"{dst_dir}/{name}"
if dst == r["path"]:
keeps.append((r["path"], dst, kind))
return
if dst in used:
park(r, "something else in the plan already files there")
return
if dst in held:
park(r, f"{rel(dst)} already occupies that name")
return
used[dst] = r["path"]
moves.append((r["path"], dst, kind))
def park(r, reason):
parked.append((r, reason))
reasons[reason].append(r)
dst = f"{PARK}/{rel(r['path'])}"
if dst in used or dst in held:
dst = f"{PARK}/{rel(r['path'])}.{len(reasons)}"
used[dst] = r["path"]
moves.append((r["path"], dst, "park"))
# Episodes first: two files claiming one number is a decision Frank makes, not one
# the plan sneaks — the bigger copy takes the standard name, the other waits.
by_ep = collections.defaultdict(list)
for r, name, season, eps in claimants:
for e in eps:
by_ep[(eng.norm(name), season, e)].append(r)
winners, losers = set(), {}
for (key, season, e), rs in by_ep.items():
rs.sort(key=lambda r: (-r["bytes"], r["path"]))
winners.add(rs[0]["path"])
for r in rs[1:]:
losers.setdefault(r["path"],
f"duplicate of {series[key]['name']} S{season:02d}E{e:02d}"
" (the bigger copy keeps the standard name)")
for r, name, season, eps in claimants:
if r["path"] in losers:
park(r, losers[r["path"]])
continue
s = series[eng.norm(name)]
label = f" S{season:02d}E" + "-E".join(f"{e:02d}" for e in eps)
part = PART_RE.search(os.path.splitext(r["base"])[0])
tail = f" Part {part.group(1).upper()}" if part and len(eps) == 1 else ""
book(r, f"{series_dir(name)}/Season {season:02d}", s["name"] + label + tail + r["ext"],
"tv")
for r, name, kind in extras:
book(r, f"{series_dir(name)}/{kind}",
clean(os.path.splitext(r["base"])[0]) + r["ext"], "extra")
# A film inside the TV library leaves for the movie library only when the movie
# provider answers with a title that matches the name and, when the name states a
# year, that year too.
for r in films:
parsed, year = title_of(r["base"])
got = tmdb("movie", parsed)
words = eng.norm(got["name"]).split() if got else []
if not got or not eng.mentions(parsed.split(), words) \
or (year != "?" and got["year"] and got["year"] != year):
park(r, "long file with no episode evidence, and no movie provider names it")
continue
label = f"{got['name']} ({got['year']})" if got["year"] else got["name"]
book(r, f"{MOVIES}/{clean(label)}", clean(label) + r["ext"], "movie")
# ------------------------------------------------- sidecars and folder metadata
moved = {} # src -> dst, for every booked file
for src, dst, _kind in moves:
moved[src] = dst
for src, dst, _kind in keeps:
moved[src] = dst
dst_dirs = collections.defaultdict(collections.Counter)
video_dst = collections.defaultdict(list)
for src, dst in moved.items():
if src in {r["path"] for r in video}:
video_dst[os.path.dirname(src)].append((src, dst))
dst_dirs[os.path.dirname(src)][os.path.dirname(dst)] += 1
def series_name_at(d):
"""The series the files of a folder belong to — a registered name, or nothing.
Folder words alone never name a show."""
names = [titles[src]["series"] for src, _ in video_dst.get(d, [])
if src in titles and titles[src]["series"]]
if names:
return collections.Counter(names).most_common(1)[0][0]
return series_of_dir(d)
def season_at(d):
"""The season number a folder carries: its own name, else its files' rows."""
m = SEASON_DIR_RE.match(os.path.basename(d))
if m:
return int(m.group(1) or m.group(2))
nums = [int(titles[src]["season"]) for src, _ in video_dst.get(d, [])
if src in titles and titles[src]["season"]]
return collections.Counter(nums).most_common(1)[0][0] if nums else None
def folder_dst(d):
"""Where a folder's own images go: the one place its files landed, or the
series root when they scattered across several seasons."""
c = dst_dirs.get(d)
if not c:
return None
if len(c) == 1:
return c.most_common(1)[0][0]
owner = series_name_at(d)
return series_dir(owner) if owner else None
def anchor_of(path, stem):
"""The video file this sidecar names: the longest file name it extends, in the
NEAREST folder that holds one. A scene pack buries its subtitles two levels
down (`Pack/Subs/<release-name>/<release>.jpn.srt`), so the search climbs to
the library root instead of stopping at the parent."""
d = os.path.dirname(path)
while len(d) >= len(ROOT) and d.startswith(ROOT):
best = None
for src, dst in video_dst.get(d, []):
base = os.path.basename(src)
for prefix in (os.path.splitext(base)[0], base):
if stem == prefix or stem.startswith(prefix + "."):
if best is None or len(prefix) > best[2]:
best = (src, dst, len(prefix))
if best:
return best[0], best[1]
up = os.path.dirname(d)
if up == d:
break
d = up
return None
def sidecar_name(r, src, dst):
"""The sidecar's own tail — language, flag, the media extension an nfo must
carry — hung on the video file's new base."""
stem = os.path.splitext(os.path.basename(src))[0]
tail = os.path.splitext(r["base"])[0][len(stem):]
# "Show.S01E01.mkv.srt" carries the container's name, not a flag: Jellyfin and
# Plex both read "Show S01E01.srt", so the container word is dropped here.
parts = tail.split(".")
if len(parts) > 1 and parts[1].lower() in CONTAINER:
tail = "".join("." + p for p in parts[2:])
if r["ext"] in NFO_EXT:
tail = os.path.splitext(os.path.basename(src))[1] + tail
return clean(os.path.splitext(os.path.basename(dst))[0] + tail + r["ext"])
for r in other:
p, base = r["path"], r["base"]
stem = os.path.splitext(base)[0]
d = os.path.dirname(p)
if r["ext"] in SUB_EXT or r["ext"] in NFO_EXT or r["ext"] in IMG_EXT:
a = anchor_of(p, stem)
if a:
book(r, os.path.dirname(a[1]), sidecar_name(r, a[0], a[1]),
"subtitle" if r["ext"] in SUB_EXT else
("nfo" if r["ext"] in NFO_EXT else "image"))
continue
if SERIES_META.match(base):
owner = series_name_at(d)
if owner:
book(r, series_dir(owner), base, "meta")
continue
elif SEASON_META.match(base):
owner, season = series_name_at(d), season_at(d)
if owner and season is not None:
book(r, f"{series_dir(owner)}/Season {season:02d}", base, "meta")
continue
elif r["ext"] in IMG_EXT or r["ext"] in NFO_EXT:
dest = folder_dst(d)
if dest:
book(r, dest, base, "meta")
continue
unknown.append((r, "not media, or media with nothing beside it to belong to"))
for r, reason in unknown:
park(r, reason)
# --------------------------------------------------------------------- output
def write_plan():
"""mkdir rows for every destination folder and its whole chain, then the moves.
Parents first: the executor creates directories in the order they are listed."""
chain = set()
for _s, dst, _k in moves:
d = os.path.dirname(dst)
while d and d not in (ROOT, MOVIES, PARK, os.path.dirname(d)):
chain.add(d)
d = os.path.dirname(d)
dirs = sorted(chain, key=lambda p: (p.count("/"), p))
with open(ARGS.out, "w") as fh:
fh.write("action\tsrc\tdst\tkind\n")
for d in dirs:
fh.write(f"mkdir\t\t{d}\tdir\n")
for src, dst, kind in sorted(moves, key=lambda m: (m[1], m[0])):
fh.write(f"move\t{src}\t{dst}\t{kind}\n")
for src, dst, kind in sorted(keeps, key=lambda m: (m[1], m[0])):
fh.write(f"keep\t{src}\t{dst}\t{kind}\n")
sized = sum(held[s]["bytes"] for s, _d, _k in moves if s in held)
print(f"plan: {len(moves) - len(parked)} moves, {len(parked)} parked"
f" ({sized / 2**40:.2f} TiB touched), {len(keeps)} already placed,"
f" {len(dirs)} folders -> {ARGS.out}")
def write_exceptions():
no_year = sorted(s["name"] for s in series.values() if not s["year"])
with open(ARGS.exceptions, "w") as fh:
fh.write(f"TV relayout exceptions for {ROOT}\n\n")
fh.write(f"Everything below is parked under {PARK}, keeping the relative path it\n")
fh.write("has today. Nothing is deleted and nothing is overwritten: a parked file\n")
fh.write("is one rename from wherever the plan is pointed next.\n\n")
for reason in sorted(reasons, key=lambda r: -len(reasons[r])):
rs = reasons[reason]
fh.write(f"{len(rs):>6} files {reason}\n")
for r in sorted(rs, key=lambda r: rel(r["path"]))[:400]:
fh.write(f" {rel(r['path'])}\n")
if len(rs) > 400:
fh.write(f" … and {len(rs) - 400} more\n")
fh.write("\n")
if no_year:
fh.write(f"{len(no_year):>6} series carry no year from any provider, so their\n")
fh.write(" folders are named without one; Jellyfin still matches them.\n")
fh.write(" " + ", ".join(no_year[:300]) + "\n")
print(f"exceptions -> {ARGS.exceptions}")
write_plan()
write_exceptions()
+8 -1
View File
@@ -275,7 +275,14 @@ def is_extra(path, title="", dur=0.0, root=ROOT, matched=False):
bonus_folder = False
for folder in parts[head:-1]:
if folder_is_bonus(folder):
bonus_folder = not mentions(name_words(name), name_words(folder))
# The folder's words are often the film's own name ("Angel's Egg
# Extras/"), and then they are not bonus evidence. Only WORDS can name
# a film: "extras" is release junk that name_words drops, so a folder
# like "01. Extras" reduces to its index — and an index matching an
# index ("NCOP 01") is a coincidence, not a title.
nw = [w for w in name_words(name) if not w.isdigit()]
fw = [w for w in name_words(folder) if not w.isdigit()]
bonus_folder = not fw or not mentions(nw, fw)
break
if dur and dur > SHORT_MAX:
# Runtime vetoes vocabulary, with ONE exception: an explicit bonus folder
+186
View File
@@ -0,0 +1,186 @@
#!/usr/bin/env python3
"""Prove a relayout landed, using the media server's own tables.
A scan answers `204` whether or not it found anything, and a rename that the
scanner has not followed looks exactly like a rename that never happened from
the outside. The only honest proof is the run's journal replayed against the
database the server actually serves from: every `move` row must have a row at
its destination, and NO row may still name its source — a source row is a scan
that has not caught up, and it is the one signature of a library pointing at
files that no longer exist.
Two doors, both read-only, both without a lock while the server runs
(`mode=ro&immutable=1`; there is no `sqlite3` binary on these hosts):
jellyfin `BaseItems.Path`. The container binds the share at `/data`, so the
database holds `/data/<library>/…` where the journal holds
`/mnt/entropy/<library>/…`. Translate, or every row reads as missing.
plex `media_parts.file`, joined up to `metadata_items`. Plex is bound at
the host path, so its rows match the journal verbatim. Filter
`deleted_at is null` on BOTH tables — a deleted scan leaves its parts
behind, and one of those rows is not proof anything landed.
Neither server holds a row for everything the journal renames: a `.log`, `.cue`
or cover image has no row to land in, so its absence is not a fault. The report
therefore breaks the missing rows down by extension, and `--ignore-ext` names
the extensions this server does not index, which are then counted separately and
kept out of the verdict.
Exit codes: 0 every move landed; 1 something is stale or missing (the details are
printed); 2 the journal or the database is not usable at all.
"""
import argparse
import json
import os
import sqlite3
import sys
from collections import Counter
JF_DB = "/home/dank/Containers/jellyfin/config/data/jellyfin.db"
PLEX_DB = ("/var/lib/plexmediaserver/Library/Application Support/Plex Media Server"
"/Plug-in Support/Databases/com.plexapp.plugins.library.db")
# Per kind: the table whose columns are required, those columns, and the query
# that returns every path the server believes it has.
KINDS = {
"jellyfin": {
"table": "BaseItems",
"columns": {"Path"},
"sql": "select Path from BaseItems",
"default_db": JF_DB,
},
"plex": {
"table": "media_parts",
"columns": {"file", "media_item_id", "deleted_at"},
"sql": ("select p.file from media_parts p"
" join media_items m on m.id = p.media_item_id"
" join metadata_items mi on mi.id = m.metadata_item_id"
" where p.deleted_at is null and mi.deleted_at is null"),
"default_db": PLEX_DB,
},
}
def die(msg):
print(f"media-verify: {msg}", file=sys.stderr)
sys.exit(2)
def read_moves(path):
"""The (src, dst) pairs of a journal, in the order the run performed them."""
moves = []
try:
fh = open(path, encoding="utf-8")
except OSError as e:
die(f"cannot read the journal {path}: {e.strerror}")
with fh:
for n, line in enumerate(fh, 1):
line = line.strip()
if not line:
continue
try:
rec = json.loads(line)
except json.JSONDecodeError:
die(f"{path} line {n} is not a journal record")
if rec.get("op") == "move":
if not rec.get("src") or not rec.get("dst"):
die(f"{path} line {n} is a move without both src and dst")
moves.append((rec["src"], rec["dst"]))
return moves
def server_paths(db, kind):
"""Every path the server holds, or a refusal that names what is missing."""
spec = KINDS[kind]
try:
con = sqlite3.connect(f"file:{db}?mode=ro&immutable=1", uri=True)
except sqlite3.Error as e:
die(f"cannot open {db}: {e}")
with con:
have = {r[1] for r in con.execute(f"pragma table_info({spec['table']})")}
if not spec["columns"] <= have:
die(f"{db} is not a {kind} library database ({spec['table']} has no "
f"{','.join(sorted(spec['columns'] - have))})")
try:
return {r[0] for r in con.execute(spec["sql"]) if r[0]}
except sqlite3.Error as e:
die(f"{db} refused the {kind} query: {e}")
def to_host(path, container, share):
"""The database's path in the journal's terms; identity when it is not a
container-side path."""
if not container:
return path
prefix = container.rstrip("/") + "/"
if path.startswith(prefix):
return share.rstrip("/") + "/" + path[len(prefix):]
return path
def main():
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("--journal", required=True,
help="the run's journal (scripts/media-apply.py writes it)")
ap.add_argument("--db-kind", choices=sorted(KINDS), required=True)
ap.add_argument("--db", default=None, help="the server's database on this host")
ap.add_argument("--share", default="/mnt/entropy",
help="the share every journal path lives under")
ap.add_argument("--container", default=None,
help="what the share is called inside the container; empty"
" compares verbatim (default /data for jellyfin, nothing"
" for plex)")
ap.add_argument("--show", type=int, default=20,
help="how many offenders to name before saying 'and N more'")
ap.add_argument("--ignore-ext", default="",
help="comma-separated extensions this server does not index,"
" kept out of the verdict (e.g. log,cue,nfo,srt)")
args = ap.parse_args()
db = args.db or KINDS[args.db_kind]["default_db"]
container = ("/data" if args.db_kind == "jellyfin" else "") \
if args.container is None else args.container
ignore = set()
for ext in args.ignore_ext.split(","):
ext = ext.strip().lower()
if ext:
ignore.add(ext if ext.startswith(".") else "." + ext)
moves = read_moves(args.journal)
stored = {to_host(p, container, args.share) for p in server_paths(db, args.db_kind)}
landed, stale, missing, ignored = 0, [], [], 0
for src, dst in moves:
if os.path.splitext(dst)[1].lower() in ignore:
ignored += 1
continue
at_dst, at_src = dst in stored, src in stored
if at_dst and not at_src:
landed += 1
continue
if at_src:
stale.append(src)
if not at_dst:
missing.append(dst)
print(f"{args.db_kind}: {len(moves)} moves checked")
print(f"landed: {landed}")
print(f"still at src: {len(stale)}")
print(f"missing from the DB: {len(missing)}")
if ignored:
print(f"not indexed (extension ignored): {ignored}")
if missing:
exts = Counter(os.path.splitext(d)[1].lower() or "(no extension)"
for d in missing)
print("missing by extension: "
+ ", ".join(f"{e} {n}" for e, n in sorted(exts.items())))
for label, items in (("stale", stale), ("missing", missing)):
for path in items[:args.show]:
print(f" {label}: {path}")
if len(items) > args.show:
print(f" {label}: … and {len(items) - args.show} more")
return 1 if (stale or missing) else 0
if __name__ == "__main__":
sys.exit(main())
+56 -1
View File
@@ -1,7 +1,7 @@
---
type: reference
title: Jellyfin
description: "Jellyfin on Dankserver: quadlet unit, overlay metadata view layout, ports, access, hardware-transcode status."
description: "Jellyfin on Dankserver: quadlet unit, overlay metadata view, provider keys, ports, access, transcode status."
tags: [jellyfin, dankserver, media-server]
timestamp: 2026-10-30T00:00:00Z
---
@@ -117,6 +117,61 @@ where the LDAP plugin runs. Verified 2026-09-25: `alex` authenticates over
`AuthenticateByName` (HTTP 200; full plugin flow logged — service bind,
search, `ldapUid`, user bind, token).
## Metadata providers and their keys (2026-10-30)
TMDb, OMDb, StudioImages and MusicBrainz are **core plugins** in Jellyfin 12 —
no DLL, just a configuration file the running container owns under the data
root. TVDB is not core (its metadata provider was removed in 10.9), so it is an
official-catalog plugin.
| Provider | Shape on disk | Key lives in |
|---|---|---|
| TMDb | `config/plugins/configurations/Jellyfin.Plugin.Tmdb.xml` → `<TmdbApiKey>` | `/home/dank/Containers/jellyfin/provider-keys` → `TMDB_API_KEY=` (mode 600) |
| TVDB | `config/plugins/thetvdb_24.0.0.0/` + `…Jellyfin.Plugin.Tvdb.xml` → `<ApiKey>` | the same file → `TVDB_API_KEY=` |
| OMDb | `…Jellyfin.Plugin.Omdb.xml` | **not configured** — its key is issued by email, 1 000 req/day, and its ToS forbids storing the returned data; ratings stay Jellyfin's own lookup |
**None of it is tracked in git.** Those XML files belong to the container and
hold credentials, and `tests/jellyfin_providers.bats` fails the suite if one is
ever committed. The keys themselves sit in the per-service host env file that is
the fleet's convention for container credentials (grafana, caddy, pihole,
copyparty, romm, seerr): `provider-keys`, mode 600, beside the data root.
`scripts/jellyfin-providers.sh` is the only thing that writes the XML:
```
scripts/jellyfin-providers.sh plan # what would change; prints no key
scripts/jellyfin-providers.sh install # plugin + keys, previous copy to trash
scripts/jellyfin-providers.sh verify # plugin + key LENGTHS, never key bytes
```
- **A key never enters git, argv, or the tool's output.** The installer reads
that 600-perm file itself (a value in argv is world-readable through
`/proc/<pid>/cmdline`) and refuses it when it is group- or world-readable —
an exposed key is an error, not a warning. A key that shows up in a transcript
or `ps` output has leaked: rotate it with the provider, rewrite
`provider-keys`, re-run `install`.
- **The TVDB artifact is pinned three ways**: sha256
`6302fba3857bdde8d186dd099fac09ca43247041f531a528a2c788add70a5319`, the
official plugin guid `a677c0da-fac5-4cde-941a-7134223f14c8`, and version
`24.0.0.0` (`targetAbi 12.0.0.0`, which fits the deployed 12.4.0). A sha
match alone only proves the bytes are the bytes fetched — the guid check is
what says it is the official plugin. Rollback: move `thetvdb_24.0.0.0/` into
the trash and revert the pin; a second `thetvdb_*` directory is a refusal,
never a quiet second copy.
- **The homelab TVDB key is a licensed key** (its JWT claims say
`community_supported: false`), so it needs **no PIN** — the optional
`TVDB_PIN=` line exists only for a subscriber-shaped key.
- **TVDB is installed and keyed but is not a metadata agent**: TMDb still
identifies everything. Enabling TVDB as a provider would re-fetch show
metadata across the library, so it is a deliberate, separate change.
- **The deployer places the keys before it restarts jellyfin**
(`ensure_provider_keys` in `scripts/deploy-remote.sh`) and warns instead of
failing when `provider-keys` is absent, so a host without it still deploys.
- **Keys and plugins are read at start-up**: after a manual `install` the
change is inert until `./scripts/deploy.sh dankserver --service jellyfin`.
- TMDb's terms are the reason this stays narrow: attribution required, no
caching beyond 6 months, no derivative works. Fine for a personal homelab,
and the reason no bulk enrichment pass runs against it.
## Configuration workflow
1. Create libraries pointing at `/data` folders (movies, shows, audio).
@@ -1,7 +1,7 @@
---
type: reference
title: Media Relayout
description: "Census, plan and apply the movie library into Jellyfin format, and move TV out of it: the gates, refusals, rollback."
description: "Census, plan and apply the movie, TV, music and audiobook libraries: the gates, the refusals, the rollback."
tags: [dankserver, media-server, jellyfin, plex, procedure]
timestamp: 2026-10-30T00:00:00Z
---
@@ -12,17 +12,24 @@ The movie library on [[Entropy Volume]] arrived as release folders —
`Some.Movie.2019.1080p.WiKiTaLx.x265-FOO/` with the video inside named after
the release. [[Jellyfin]] matches a movie by folder and file name, so the
release noise cost posters, extras and versions. These scripts turn that
library into Jellyfin's own shape — and move TV back out of it — and nothing
else in the repo touches a media file.
library into the shape its media server reads — the movies, the TV packs, the TV
that turns up in the movie share, the music library, and the audiobooks, whose
only reader is Plex — and nothing else in the repo touches a media file.
| Script | Touches media? | Output |
|---|---|---|
| `scripts/media-census.py` | reads headers only | `enum`, `probed`, `plex` TSVs |
| `scripts/media-census.py` | reads headers and the servers' own DBs, never writes | `enum`, `probed`, and one authority TSV per library |
| `scripts/media-plan.py` | no — TSVs in, TSVs out | the plan + the exception report |
| `scripts/media-apply.py` | **renames, never copies**; sweeps only what a run emptied | the moves, plus a journal |
| `scripts/media-tv.py` | no — TSV out | a plan that moves TV into the TV library |
| `scripts/media-shows.py` | no — TSV out | a plan that re-files the TV library itself |
| `scripts/media-music.py` | no — TSV out | a plan that re-files the music library by its tags |
| `scripts/media-audiobooks.py` | no — TSV out | a plan that re-files the audiobooks by the book Plex matched |
| `scripts/media-verify.py` | no — reads the servers' DBs read-only | the verdict that every journaled move landed |
The seam is `tests/media_relayout.bats`; the target contract is quoted from
The seams are `tests/media_relayout.bats`, `tests/media_shows.bats`,
`tests/media_music.bats`, `tests/media_audiobooks.bats` and `tests/media_verify.bats`;
the movie and TV target contracts are quoted from
[jellyfin.org/docs/general/server/media/movies](https://jellyfin.org/docs/general/server/media/movies)
in `media-plan.py`'s docstring.
@@ -108,6 +115,29 @@ shred -u /tmp/jf.hdr
`204` is acceptance; the scan then runs on the server's own schedule.
### Proving the move landed
A scan answers `204` whether or not it finds anything, so proof is the journal
replayed against both servers' own tables: every `move` row has a row at its
`dst`, and **zero** rows still name its `src`. `scripts/media-verify.py` is that
replay: it opens each database read-only (there is no `sqlite3` binary on these
hosts or inside either container) and exits non-zero while a row is stale.
```bash
scripts/media-verify.py --journal /mnt/entropy/media-relayout-music.jsonl \
--db-kind jellyfin --ignore-ext jpg,jpeg,png,log,cue,toc,nfo,txt,m3u,accurip
```
Ignore only what a server cannot index. A missing `.flac` or `.mkv` is the
fault; a row still naming its `src` means the scan has not followed the rename
yet, so re-run the check after the scan, never before.
Three doors that look right and are not: `GET /search?query=` matches metadata
titles, never file paths; `?path=` on `/library/sections/N/all` makes Plex's
HTTPD close without a single byte; and the same HTTPD drops requests that carry
no `Accept` and client-identifier header (see the scan block above for the form
that works).
## TV in the movie library
A box set that arrives in the movie share stays there until someone moves it:
@@ -153,6 +183,180 @@ folder it owns, never from the title whose bonus folder it happens to be parked
in — `The Cleveland Show S03E07` inside `Die Hard (1988)/extras/` is not a 1988
show. A name whose marker is `Part 2` is a film and stays put.
## The TV library's own relayout
`/mnt/entropy/TVShows` is scene packs rather than a library, and Jellyfin has
already matched and numbered most of it. `scripts/media-shows.py` re-files that
library into the same shape, and it never guesses: every series name and every
episode number it writes traces to an authority, and what has none is parked for
a human to read.
```bash
# 1. The authority: Jellyfin's own matched rows (the DB opens read-only).
python3 scripts/media-census.py jellyfin --root /mnt/entropy/TVShows \
--out /tmp/tv_titles.tsv
# 2. What is on disk (stat only), and the runtimes the film arm needs.
python3 scripts/media-census.py enumerate --root /mnt/entropy/TVShows \
--out /tmp/tv_enum.tsv
python3 scripts/media-census.py probe --root /mnt/entropy/TVShows \
--enum /tmp/tv_enum.tsv --jobs 6 --out /tmp/tv_probed.tsv
# The probe is the slow step: ffprobe reads both ends of every file.
# 3. The plan. --tmdb needs TMDB_API_KEY; without it a series no provider dated
# is simply named without a year, which Jellyfin accepts.
python3 scripts/media-shows.py --root /mnt/entropy/TVShows --enum /tmp/tv_enum.tsv \
--titles /tmp/tv_titles.tsv --probed /tmp/tv_probed.tsv --out /mnt/entropy/media-relayout-shows-plan.tsv \
--exceptions /mnt/entropy/media-relayout-shows-exceptions.txt
# 4. Read the exceptions file FIRST — it is the review, not a log. Same executor
# as above, but on this arm's OWN journal: a shared journal cannot be unwound.
scripts/media-apply.py apply --plan /mnt/entropy/media-relayout-shows-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/media-relayout-shows.jsonl --apply
```
Evidence, strongest first — the order the planner believes things:
- the **episode marker the file's own name states** (`S01E02-E03`, `1x04`);
- the **`Season NN` folder it sits in and the number its name leads with inside
it**: `Season 19/01 Charles Barkley, Nirvana.avi` is S19E01;
- the **provider row**: identity (the series name and its year), and the
numbering of a file whose own name says nothing;
- a **bonus folder** anywhere above the file, or Jellyfin's `ExtraType` → the
series' canonical extra folder;
- a runtime over `SHORT_MAX` makes it film-shaped, and it leaves for `--movies`
only when a movie provider names it — otherwise it parks.
Two refusals carry the design, and a dry run on the real library found both:
- **Folders resolve a series; they never invent one.** A folder's words may
resolve a file to a series a provider has already named — `(CBB) Tengen Toppa
Gurren Lagann - The Complete Series (BD 1080p)` names the registered *Tengen
Toppa Gurren Lagann* — but a bonus subfolder is not a show. An early run made
`…/03. Extra/01. NCOP/NCOP 01.mkv` into a root series called `01 NCOP`.
- **A scraped row does not outrank the filesystem.** Jellyfin numbers
multi-season packs badly: every file under `SNL 16-20/Season 16..19` carries
season 16 episode 20, and one Gundam row says season 2004. The folder, then
the file's own leading number, win; a row that still insists on season 1989
parks its file instead of creating `Season 1989`.
The provider's name is copied verbatim, warts included: Jellyfin carries a Series
named `Black Clover 001` (92 episodes, no year) because it matched a pack off a
filename. Renaming a show is Jellyfin's work — a folder rename never touches its
provider ids — so the mover keeps the name it is handed and strips only the pack
index a release leads with (`4a. Seed DESTINY` → `Seed DESTINY`).
## The music library's own relayout
`/mnt/entropy/Music` is rips and discography packs, and Jellyfin has matched and
tagged almost all of it. `scripts/media-music.py` re-files it into the shape
Jellyfin's music scanner reads —
Music/<album artist>/<Album> (<year>)/NN - <Title>.ext
— and the authority is the tag row of the track, never the name the ripper gave
the file.
```bash
# 1. The authority: Jellyfin's audio rows, one per track, with the album and
# artist ids its own database carries (the DB opens read-only).
python3 scripts/media-census.py music --root /mnt/entropy/Music \
--out /tmp/music_rows.tsv
# 2. What is on disk (stat only).
python3 scripts/media-census.py enumerate --root /mnt/entropy/Music \
--out /tmp/music_enum.tsv
# 3. The plan.
python3 scripts/media-music.py --root /mnt/entropy/Music --enum /tmp/music_enum.tsv \
--rows /tmp/music_rows.tsv --out /tmp/music_plan.tsv \
--exceptions /tmp/music_exceptions.txt
# 4. Read the exceptions file FIRST — it is the review, not a log. Then the same
# executor, journal and rollback as above, on its own journal.
scripts/media-apply.py apply --plan /tmp/music_plan.tsv --root /mnt/entropy \
--journal /mnt/entropy/media-relayout-music.jsonl --apply
```
The group is what the tags assert, not where the files sit: **(album-artist tag,
album, year)**. One album arrives as three disc folders and as two copies of one
rip, and one MusicBrainz release group is one album. A compilation is one album
whose tracks name many performers, so it files once under `Various Artists`:
keying on the performer would file one album once per artist. Two different
album-artist tags on one album name are two albums, which is what Jellyfin's two
album items say.
What it refuses:
+- **A sidecar is placed by the audio around it, never by its own name.** A
`.lrc`, `.txt` or picture that shares a track's stem follows that track and
takes the name the track now has, or Jellyfin cannot find it; anything else
goes to the album its own folder was placed in, or to the folder whose discs
sit one level below it. A folder whose audio belongs to two albums parks its
sidecar rather than guessing.
+- **An image album keeps its name.** A folder holding one audio file and a
`.cue` is a whole album in one file; numbering it `01 - …` tells Jellyfin to
index it as one track.
+- **A video container parks**, because Jellyfin will not index it as music and a
rename to `.m4a` would claim it is; **audio no provider row names parks** too,
because unmatched audio is a tagging job, not a renaming job.
The plan applies on 2026-10-31 through `/mnt/entropy/media-relayout-music.jsonl`;
its rollback is that journal run through `media-apply.py rollback --apply`. ONE
writer at a time on this array: two rename passes race, and neither journal tells.
## The audiobook library's own relayout
`/mnt/entropy/Audiobooks` is the one library here that [[Jellyfin]] does not read
at all — its music scanner has no audiobook concept — while [[Plex]] indexes all
of it as Artist → Album → Track: the album is the BOOK, that book's single artist
is the NARRATOR, both matched out of the files' own tags. Today one book is split
across `… [Part 1]`, `… [Part 2]` folders three or four deep inside a collection
pack, so a scan serves three quarters of a novel; `scripts/media-audiobooks.py`
re-files the tree so the property both servers' book scanners assume is true —
Audiobooks/<Narrator>/<Book> (<year>)/<the file keeps its own name>
```bash
# 1. The authority: Plex's chapter rows for this root (the DB opens read-only).
python3 scripts/media-census.py audiobooks --root /mnt/entropy/Audiobooks \
--out /tmp/ab_rows.tsv
# 2. What is on disk (stat only).
python3 scripts/media-census.py enumerate --root /mnt/entropy/Audiobooks \
--out /tmp/ab_enum.tsv
# 3. The plan.
python3 scripts/media-audiobooks.py --root /mnt/entropy/Audiobooks \
--enum /tmp/ab_enum.tsv --rows /tmp/ab_rows.tsv --out /tmp/ab_plan.tsv \
--exceptions /tmp/ab_exceptions.txt
# 4. Read the exceptions file FIRST. Then the same executor, journal and rollback
# as above, on its own journal, and refresh the Plex section as in step 5 of
# the movie Procedure — from inside the container, never over the host port.
scripts/media-apply.py apply --plan /tmp/ab_plan.tsv --root /mnt/entropy \
--journal /mnt/entropy/media-relayout-audiobooks.jsonl --apply
```
The group is the identity Plex asserts in its own database — **(narrator, book,
year)** — so one book arriving in four part folders becomes one folder, and a
1965 and a 2020 recording that share a title stay two books.
Two laws are forced by what the rows actually hold:
+- **A chapter is never renumbered.** Most books contain chapters whose index Plex
does not know, so a number written into a name would be invented — and it would
buy nothing, because both servers order playback by the embedded track tag, not
by the file name. A wrong number in a name is the number a human later trusts.
Fix order in the tags: a Plex scan re-reads tags and never renames a file.
+- **The shelf is the narrator, not the author.** Every book has exactly one artist
row and it is the narrator; the author survives only inside some album titles
(`Kryptos by Graham McNeill`) and in no column Plex stores, so deriving it would
mean parsing titles where a ripper wrote one and inventing it everywhere else.
The author stays visible in the folder name Plex itself shows.
What it refuses, the music arm's list with one addition: a video container, audio
no chapter row names, and a loose file in a folder holding two books all park under
`_unsorted` with the path they have today — and so does a chapter whose row names
**no book or no narrator**: a placeholder (`[Unknown Album]`, `[Unknown Artist]`)
that unrelated books share, or a chapter number mistagged as the artist (`0.4`).
`Various Artists` is not a placeholder — that is a multi-cast read.
The plan stands reviewed at `/mnt/entropy/media-relayout-audiobooks-plan.tsv`
(55,114 moves, 7,764 parked, executor preflight refused nothing); it applies on its
own journal, behind the music writer, and then refreshes the Plex section.
## Laws
- **A moved file is a 404 in [[Plex]] until its section is rescanned.** The
@@ -261,6 +465,21 @@ show. A name whose marker is `Part 2` is a film and stays put.
whiteout in the upper layer, which hides the array's copy without removing it;
the file returns when the view is unmounted. Deletions in this pipeline happen
on the raw path only, which is what the journal can undo.
- **The music album key is case-folded; the folder name is not.** `Master Of
Puppets` and `Master of Puppets` are one album to the matcher, so the spelling
that reaches the folder is the one most tracks carry. Taking it from whichever
row sorted first made the folder name depend on row order, and re-planning an
already-filed library renamed albums that were already in place.
- **An occupied path is never a destination**, not even when its occupant is
scheduled to leave: the executor refuses an occupied target, so a plan that
aims at one fails after the renames that were supposed to clear it. A file
standing where the contract puts it is a keep, and a file already carrying a
collision suffix keeps that suffix — otherwise the plan is a shuffle, not a
fixed point, and a re-run moves albums that are already right.
- **`media-census.py music` reads the `BaseItemProviders` table, not the item's
JSON.** This Jellyfin build carries an empty `ProviderIds` inside
`BaseItems.Data` for albums, so a census that parses the JSON finds no
MusicBrainz or Spotify id on any album and reads as "the library is untagged".
## Rollback
+2 -2
View File
@@ -49,10 +49,10 @@ is a locator: one row, one clause, no numbers.
| [[Copyparty]] | Copyparty is a lightweight file server and media streamer that runs as a single Python binary. |
| [[Dankserver Stack]] | Service inventory and routing overview for the media and library host. |
| [[Entropy Volume]] | The Dankserver entropy volume is a RAID-backed LUKS filesystem mounted at |
| [[Jellyfin]] | Jellyfin on Dankserver: quadlet unit, overlay metadata view layout, ports, access, hardware-transcode status. |
| [[Jellyfin]] | Jellyfin on Dankserver: quadlet unit, overlay metadata view, provider keys, ports, access, transcode status. |
| [[mariadb]] | The MariaDB backend for RomM: quadlet unit, network alias, env-file secret handling, and the backup path. |
| [[Media Metadata View]] | The SSD-backed overlay the two media servers read and write: shape, guards, commands, laws, rollback. |
| [[Media Relayout]] | Census, plan and apply the movie library into Jellyfin format, and move TV out of it: the gates, refusals, rollback. |
| [[Media Relayout]] | Census, plan and apply the movie, TV, music and audiobook libraries: the gates, the refusals, the rollback. |
| [[Plex]] | Plex Media Server is a **containerized media server on Dankserver**: a podman |
| [[redis]] | The Redis cache and task queue for RomM: quadlet unit, network alias topology, and the backup path. |
| [[RomM]] | RomM (ROM Manager) is a self-hosted ROM library manager and player. |
+297
View File
@@ -0,0 +1,297 @@
#!/usr/bin/env bats
# Seam: scripts/jellyfin-providers.sh — the only mechanism that puts a
# metadata-provider key into Jellyfin's plugin configuration on Dankserver.
#
# Output vocabulary the tests depend on:
# "+ <action>" something this run will change
# "= <fact>" something already true, so nothing changes
#
# The contracts, in the order the seam tests them:
# 1. plan reports every action and never prints a key;
# 2. a secret file that is not mode 600 is refused by name and nothing is
# written — a key readable by other users is an error, not a warning;
# 3. a downloaded plugin artifact is accepted only on its pinned sha256, its
# official guid and its pinned version (a swapped or downgraded artifact
# is refused before anything lands in plugins/);
# 4. the TMDb key is written into the EXISTING config with every other
# setting preserved — Jellyfin owns that file, and clobbering it would
# silently reset country, language and adult settings;
# 5. a TVDB config the operator already saved through the UI is kept;
# 6. install is idempotent: a second run changes no byte;
# 7. whatever is replaced is copied to the trash first (fleet law: never
# overwrite without a recoverable copy);
# 8. verify answers whether the endpoint is configured, key LENGTH only;
# 9. no plugin configuration XML is tracked in git, the deployer places the
# keys BEFORE it restarts Jellyfin, and the planner routes a change to this
# installer to jellyfin.
load helpers
PROV="$REPO_ROOT/scripts/jellyfin-providers.sh"
TMDB_KEY="tmdbk3a9Zq7Rtestkey"
TVDB_KEY="tvdbk8b2Wt4Ptestkey"
GUID_OFFICIAL="a677c0da-fac5-4cde-941a-7134223f14c8"
# make_plugin <file> <guid> <version> — a zip shaped like the official TheTVDB
# artifact (meta.json + its two DLLs), so extraction is exercised for real.
make_plugin() {
python3 - "$1" "$2" "$3" <<'PYEOF'
import json, sys, zipfile
path, guid, version = sys.argv[1], sys.argv[2], sys.argv[3]
meta = {
"category": "MoviesAndShows",
"guid": guid,
"name": "TheTVDB",
"owner": "jellyfin",
"targetAbi": "12.0.0.0",
"version": version,
}
with zipfile.ZipFile(path, "w") as z:
z.writestr("meta.json", json.dumps(meta))
z.writestr("Jellyfin.Plugin.Tvdb.dll", "dll-bytes")
z.writestr("Tvdb.Sdk.dll", "dll-bytes")
PYEOF
sha256sum "$1" | cut -d' ' -f1
}
setup() {
JF="$BATS_TEST_TMPDIR/jf"
TRASH="$BATS_TEST_TMPDIR/trash"
SEC="$BATS_TEST_TMPDIR/secrets"
mkdir -p "$JF/plugins/configurations" "$TRASH" "$SEC"
cat >"$JF/plugins/configurations/Jellyfin.Plugin.Tmdb.xml" <<XML
<?xml version="1.0" encoding="utf-8"?>
<Configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<MetadataCountryCode>US</MetadataCountryCode>
<PreferOriginalLanguage />
<Language>en</Language>
<EnableCinematicForTrailers>false</EnableCinematicForTrailers>
<IncludeAdult>false</IncludeAdult>
<MaxCastMembers>25</MaxCastMembers>
<AddGenresFromKeywords>true</AddGenresFromKeywords>
<TmdbApiKey />
</Configuration>
XML
KEYS="$SEC/provider-keys"
printf 'TMDB_API_KEY=%s\nTVDB_API_KEY=%s\n' "$TMDB_KEY" "$TVDB_KEY" >"$KEYS"
chmod 600 "$KEYS"
ZIP="$BATS_TEST_TMPDIR/thetvdb.zip"
PIN_SHA="$(make_plugin "$ZIP" "$GUID_OFFICIAL" 24.0.0.0)"
PROV_ENV=(
JF_ROOT="$JF"
PROV_KEYS="$KEYS"
TRASH_DIR="$TRASH"
TVDB_PLUGIN_URL="file://$ZIP"
TVDB_PLUGIN_SHA256="$PIN_SHA"
TVDB_PLUGIN_GUID="$GUID_OFFICIAL"
TVDB_PLUGIN_VERSION=24.0.0.0
)
}
prov() { env "${PROV_ENV[@]}" bash "$PROV" "$@"; }
tree_fingerprint() { find "$JF" -type f | sort | xargs -r sha256sum | sha256sum; }
@test "plan names the plugin and both config files, and prints no key" {
run prov plan
[ "$status" -eq 0 ]
printf '%s\n' "${lines[@]}" | grep -q '^+ install thetvdb_24\.0\.0\.0'
printf '%s\n' "${lines[@]}" | grep -q '^+ set TmdbApiKey'
printf '%s\n' "${lines[@]}" | grep -q '^+ create Jellyfin\.Plugin\.Tvdb\.xml'
# The whole point of running this over a shoulder: a key never appears.
! printf '%s\n' "${lines[@]}" | grep -qF "$TMDB_KEY"
! printf '%s\n' "${lines[@]}" | grep -qF "$TVDB_KEY"
}
@test "plan writes nothing" {
before="$(tree_fingerprint)"
run prov plan
[ "$status" -eq 0 ]
[ "$(tree_fingerprint)" = "$before" ]
}
@test "an already-configured endpoint reports no pending action" {
prov install >/dev/null
run prov plan
[ "$status" -eq 0 ]
! printf '%s\n' "${lines[@]}" | grep -q '^+ '
printf '%s\n' "${lines[@]}" | grep -q '= thetvdb_24\.0\.0\.0 already installed'
printf '%s\n' "${lines[@]}" | grep -q '= TmdbApiKey already set'
printf '%s\n' "${lines[@]}" | grep -q '= Jellyfin\.Plugin\.Tvdb\.xml already has an ApiKey'
}
@test "a keys file that is not mode 600 is refused and nothing is written" {
chmod 644 "$KEYS"
before="$(tree_fingerprint)"
run prov install
[ "$status" -ne 0 ]
printf '%s\n' "${lines[@]}" | grep -qF '600'
[ "$(tree_fingerprint)" = "$before" ]
}
@test "a missing TVDB key is refused by name before anything is fetched" {
printf 'TMDB_API_KEY=%s\n' "$TMDB_KEY" >"$KEYS"
chmod 600 "$KEYS"
run prov install
[ "$status" -ne 0 ]
printf '%s\n' "${lines[@]}" | grep -qF 'TVDB_API_KEY'
[ ! -d "$JF/plugins/thetvdb_24.0.0.0" ]
}
@test "a root with no plugin directory is refused instead of fabricated" {
# Creating plugins/ inside a wrong path would install a plugin Jellyfin
# never loads, and would hide the operator's typo.
run env "${PROV_ENV[@]}" JF_ROOT="$BATS_TEST_TMPDIR/nope" bash "$PROV" install
[ "$status" -ne 0 ]
[ ! -d "$BATS_TEST_TMPDIR/nope/plugins" ]
}
@test "an artifact whose sha256 is not the pinned one is refused" {
run env "${PROV_ENV[@]}" TVDB_PLUGIN_SHA256="$(printf 'f%.0s' {1..64})" \
bash "$PROV" install
[ "$status" -ne 0 ]
printf '%s\n' "${lines[@]}" | grep -qiF 'sha256'
[ ! -d "$JF/plugins/thetvdb_24.0.0.0" ]
}
@test "an artifact that is not the official TVDB plugin is refused" {
# A matching sha proves only that the bytes are the bytes we fetched.
EVIL="$BATS_TEST_TMPDIR/evil.zip"
EVIL_SHA="$(make_plugin "$EVIL" "deadbeef-0000-0000-0000-000000000000" 24.0.0.0)"
run env "${PROV_ENV[@]}" TVDB_PLUGIN_URL="file://$EVIL" TVDB_PLUGIN_SHA256="$EVIL_SHA" \
bash "$PROV" install
[ "$status" -ne 0 ]
printf '%s\n' "${lines[@]}" | grep -qiF 'guid'
[ ! -d "$JF/plugins/thetvdb_24.0.0.0" ]
}
@test "an artifact that is not the pinned version is refused" {
OLD="$BATS_TEST_TMPDIR/old.zip"
OLD_SHA="$(make_plugin "$OLD" "$GUID_OFFICIAL" 22.0.0.0)"
run env "${PROV_ENV[@]}" TVDB_PLUGIN_URL="file://$OLD" TVDB_PLUGIN_SHA256="$OLD_SHA" \
bash "$PROV" install
[ "$status" -ne 0 ]
printf '%s\n' "${lines[@]}" | grep -qiF 'version'
[ ! -d "$JF/plugins/thetvdb_24.0.0.0" ]
}
@test "a different plugin version already in plugins/ is refused, not doubled" {
# Two thetvdb_* directories means Jellyfin loads one plugin twice; the
# operator removes the old one (into the trash) deliberately, not by an
# installer quietly adding a second.
prov install >/dev/null
OLD="$BATS_TEST_TMPDIR/old.zip"
OLD_SHA="$(make_plugin "$OLD" "$GUID_OFFICIAL" 22.0.0.0)"
run env "${PROV_ENV[@]}" TVDB_PLUGIN_URL="file://$OLD" TVDB_PLUGIN_SHA256="$OLD_SHA" \
TVDB_PLUGIN_VERSION=22.0.0.0 bash "$PROV" install
[ "$status" -ne 0 ]
printf '%s\n' "${lines[@]}" | grep -qF 'thetvdb_24.0.0.0'
[ ! -d "$JF/plugins/thetvdb_22.0.0.0" ]
}
@test "install places the plugin, keys TMDb, and preserves every other setting" {
run prov install
[ "$status" -eq 0 ]
! printf '%s\n' "${lines[@]}" | grep -qF "$TMDB_KEY"
! printf '%s\n' "${lines[@]}" | grep -qF "$TVDB_KEY"
[ -f "$JF/plugins/thetvdb_24.0.0.0/meta.json" ]
[ -f "$JF/plugins/thetvdb_24.0.0.0/Jellyfin.Plugin.Tvdb.dll" ]
local cfg="$JF/plugins/configurations/Jellyfin.Plugin.Tmdb.xml"
# Jellyfin owns this file: the settings it wrote must survive the edit.
for setting in MetadataCountryCode Language IncludeAdult MaxCastMembers AddGenresFromKeywords; do
grep -qF "<$setting>" "$cfg" || {
echo "lost $setting"
return 1
}
done
grep -qF '<IncludeAdult>false</IncludeAdult>' "$cfg"
grep -qF "<TmdbApiKey>$TMDB_KEY</TmdbApiKey>" "$cfg"
grep -qF "<ApiKey>$TVDB_KEY</ApiKey>" "$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml"
}
@test "a TVDB config the operator saved through the UI is kept" {
cat >"$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml" <<XML
<?xml version="1.0" encoding="utf-8"?>
<Configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<ApiKey>ui-saved-key</ApiKey>
<EnableSeasonPathMatching>true</EnableSeasonPathMatching>
</Configuration>
XML
run prov install
[ "$status" -eq 0 ]
printf '%s\n' "${lines[@]}" | grep -qiF 'kept'
local cfg="$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml"
grep -qF '<ApiKey>ui-saved-key</ApiKey>' "$cfg"
grep -qF '<EnableSeasonPathMatching>true</EnableSeasonPathMatching>' "$cfg"
}
@test "a pin in the TVDB secret lands in the plugin config" {
# A subscriber-shaped key needs the account's PIN beside it; a licensed key
# does not, so the element appears only when the secret carries one.
printf 'TMDB_API_KEY=%s\nTVDB_API_KEY=%s\nTVDB_PIN=123456\n' "$TMDB_KEY" "$TVDB_KEY" >"$KEYS"
chmod 600 "$KEYS"
prov install >/dev/null
grep -qF '<Pin>123456</Pin>' "$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml"
}
@test "install is idempotent: a second run changes no byte" {
prov install >/dev/null
before="$(tree_fingerprint)"
run prov install
[ "$status" -eq 0 ]
[ "$(tree_fingerprint)" = "$before" ]
printf '%s\n' "${lines[@]}" | grep -qiF 'no change'
# One plugin directory only: two versions in plugins/ means two loads.
[ "$(find "$JF/plugins" -maxdepth 1 -type d -name 'thetvdb_*' | wc -l)" -eq 1 ]
}
@test "what install replaces is copied to the trash first" {
prov install >/dev/null
# The TMDb config existed before this tool touched it, so the pre-edit copy
# must stay recoverable.
[ -n "$(find "$TRASH" -name 'Jellyfin.Plugin.Tmdb.xml*' -print -quit)" ]
grep -qF '<TmdbApiKey />' "$(find "$TRASH" -name 'Jellyfin.Plugin.Tmdb.xml*' | head -1)"
}
@test "verify refuses an unconfigured endpoint and names what is missing" {
run prov verify
[ "$status" -ne 0 ]
printf '%s\n' "${lines[@]}" | grep -qiF 'thetvdb'
printf '%s\n' "${lines[@]}" | grep -qiF 'TmdbApiKey'
}
@test "verify passes once the plugin and both keys are in place" {
prov install >/dev/null
run prov verify
[ "$status" -eq 0 ]
! printf '%s\n' "${lines[@]}" | grep -qF "$TMDB_KEY"
! printf '%s\n' "${lines[@]}" | grep -qF "$TVDB_KEY"
# Length, not bytes: the operator sees the key loaded without reading it.
printf '%s\n' "${lines[@]}" | grep -qE 'TmdbApiKey .*[0-9]+ chars'
}
@test "no plugin configuration XML is tracked in this repo" {
# These files hold provider keys and are owned by the running container;
# tracking them would put credentials in git and guarantee drift.
run git -C "$REPO_ROOT" ls-files
[ "$status" -eq 0 ]
! printf '%s\n' "${lines[@]}" | grep -qF 'plugins/configurations'
! printf '%s\n' "${lines[@]}" | grep -qiF 'Plugin.Tvdb.xml'
}
@test "the planner routes this installer to jellyfin" {
run python3 "$REPO_ROOT/scripts/deploy-plan.py" --target dankserver \
--paths scripts/jellyfin-providers.sh
[ "$status" -eq 0 ]
[ "$output" = "jellyfin" ]
}
@test "the deployer places the keys before it restarts jellyfin" {
# The plugin and both keys are read at start-up, so a restart that runs
# ahead of the placement boots the old, keyless state.
body="$(sed -n '/^deploy_jellyfin()/,/^\}/p' "$REPO_ROOT/scripts/deploy-remote.sh")"
[ -n "$body" ]
up="$(printf '%s\n' "$body" | grep -n 'ensure_provider_keys' | head -1 | cut -d: -f1)"
restart="$(printf '%s\n' "$body" | grep -n 'systemctl --user restart jellyfin.service' | head -1 | cut -d: -f1)"
[ "${up:-999}" -lt "${restart:-0}" ]
}
+438
View File
@@ -0,0 +1,438 @@
#!/usr/bin/env bats
# Seam: the audiobook-library relayout — scripts/media-census.py audiobooks (the
# export of the rows Plex matched) and scripts/media-audiobooks.py (the plan).
#
# The audiobook tree is the one library on this box that Jellyfin does not read
# at all and Plex reads completely: every chapter here is a Plex Track row under
# an Album row (the BOOK) under one Artist row (the NARRATOR). That is the
# authority this seam files by — never the collection folder a release pack made.
#
# Two laws here are the opposite of the music tool's, and both come from what the
# rows actually contain. Files are NEVER renumbered: 7,556 of 10,485 books carry
# at least one chapter whose index Plex does not know, so a `00 -` invented from
# a missing index would order a book wrong, and playback order comes from the
# embedded track tag either way — the name buys nothing and can only lie. And the
# folder artist is the NARRATOR even though a reader thinks in authors: every
# book has exactly one artist row, while the author survives only in 10% of the
# album titles (`Kryptos by Graham McNeill`) and in no column Plex stores, so one
# rule beats a shelf that is half authors and half narrators.
#
# Every test states which authority a file has and asserts where the PLAN files
# it, never how the planner decided.
load helpers
PLAN="$REPO_ROOT/scripts/media-audiobooks.py"
CENSUS="$REPO_ROOT/scripts/media-census.py"
APPLY="$REPO_ROOT/scripts/media-apply.py"
# at PATH SIZE — a file with deterministic content.
at() { mkdir -p "$(dirname "$1")"; head -c "$2" /dev/zero > "$1"; }
# row FILE NARRATOR BOOK YEAR TRACK CHAPTER
# One row of the authority export scripts/media-census.py audiobooks writes.
row() {
printf '%s\t%s\t%s\t%s\t%s\t%s\n' \
"$1" "$2" "$3" "${4:-}" "${5:-}" "$6" >> "$ROWS"
}
# dst SRC — the destination the plan gives SRC, or nothing when it does not move.
dst() {
awk -F '\t' -v s="$1" '$1 == "move" && $2 == s {print $3}' "$OUT"
}
# kind SRC — the kind the plan gave SRC's move row.
kind() {
awk -F '\t' -v s="$1" '$1 == "move" && $2 == s {print $4}' "$OUT"
}
setup() {
AB="$BATS_TEST_TMPDIR/Audiobooks"
PARK="$AB/_unsorted"
ENUM="$BATS_TEST_TMPDIR/enum.tsv"
ROWS="$BATS_TEST_TMPDIR/rows.tsv"
OUT="$BATS_TEST_TMPDIR/plan.tsv"
EXC="$BATS_TEST_TMPDIR/exceptions.txt"
# The header is part of the authority export, so the fixture carries it.
printf 'file\tnarrator\tbook\tyear\ttrack\tchapter\n' > "$ROWS"
# One book of one author, split across the two part folders a pack shipped it
# in, with a cover above part 1 and a playlist beside the chapters.
A="$AB/Stephen King/It (1986)"
at "$A [Part 1]/01 - Chapter 1.mp3" 9000
at "$A [Part 1]/02 - Chapter 2.mp3" 9000
at "$A [Part 1]/cover.jpg" 20
at "$A [Part 1]/It.m3u" 30
at "$A [Part 2]/03 - Chapter 3.mp3" 9000
row "$A [Part 1]/01 - Chapter 1.mp3" "George Guidall" "It" 1986 1 "Chapter 1"
row "$A [Part 1]/02 - Chapter 2.mp3" "George Guidall" "It" 1986 2 "Chapter 2"
row "$A [Part 2]/03 - Chapter 3.mp3" "George Guidall" "It" 1986 3 "Chapter 3"
# A book whose album row carries no year at all.
B="$AB/Collected/Project Gold"
at "$B/01.ogg" 9000
row "$B/01.ogg" "Operator X" "Project Gold" "" 1 "Opening"
# One book title, two narrators: two different books.
C="$AB/Shared"
at "$C/Dune side a.m4b" 9000
at "$C/Dune side b.m4b" 9000
at "$C/Dune 2020.m4b" 9000
row "$C/Dune side a.m4b" "Scott Brick" "Dune" 1965 1 "Prologue"
row "$C/Dune side b.m4b" "Scott Brick" "Dune" 1965 2 "Dune"
row "$C/Dune 2020.m4b" "Malcolm Hillgartner" "Dune" 2020 1 "Book One"
# Two part folders of one book whose chapters share a file name.
D="$AB/Pulp"
at "$D/Book A/vol1.mp3" 9000
at "$D/Book A vol2/vol1.mp3" 9000
row "$D/Book A/vol1.mp3" "Nora Pace" "Hard Food" 2001 1 "Vol 1"
row "$D/Book A vol2/vol1.mp3" "Nora Pace" "Hard Food" 2001 2 "Vol 1"
# The same narrator spelled two ways across one book's chapters.
E="$AB/Spell"
at "$E/kharn 1.mp3" 9000
at "$E/kharn 2.mp3" 9000
at "$E/kharn 3.mp3" 9000
row "$E/kharn 1.mp3" "David Timson" "Kharn" 2013 1 "One"
row "$E/kharn 2.mp3" "David timson" "Kharn" 2013 2 "Two"
row "$E/kharn 3.mp3" "David Timson" "Kharn" 2013 3 "Three"
# A folder holding two different books: its loose note has no owner.
F="$AB/Mixed"
at "$F/book one chapter.mp3" 9000
at "$F/book two chapter.mp3" 9000
at "$F/notes.txt" 10
row "$F/book one chapter.mp3" "Ann Reader" "Book One" 2001 1 "One"
row "$F/book two chapter.mp3" "Bob Reader" "Book Two" 2002 1 "Two"
# A file already parked: the pile is for a human, so a row that would file it
# must not pull it back into the tree.
at "$PARK/Old/nobody.mp3" 9000
row "$PARK/Old/nobody.mp3" "Ann Reader" "Book One" 2001 1 "One"
# The odd cases: no row, a video container, a name carrying a character Plex
# says breaks playback, rows with a blank book and a blank narrator, and a
# matched chapter whose file has no extension.
G="$AB/Odd"
at "$G/loose.mp3" 9000
at "$G/concert.mp4" 9000
at "$G/Bad: Name/01 chapter.mp3" 9000
row "$G/Bad: Name/01 chapter.mp3" "Some Reader" "Trail: Marked" 1999 1 "Marked"
at "$G/blank book.mp3" 9000
row "$G/blank book.mp3" "Some Reader" "" 2000 1 "No Book"
at "$G/blank narrator.mp3" 9000
row "$G/blank narrator.mp3" "" "No Artist" 2000 1 "No Artist"
at "$G/NoExt/part01" 9000
row "$G/NoExt/part01" "Archivist" "Tapes" 1971 1 "Tape One"
# Tag-scan placeholders. Two unrelated books whose album rows both read
# `[Unknown Album]`: the name is shared, so filing on it would merge books
# that have nothing to do with each other. A rip timestamp can be glued to
# the placeholder, and a multi-cast read says `Various Artists` — which IS a
# narrator.
H="$AB/Packs"
at "$H/Stross/Ship/c1.mp3" 9000
at "$H/Leckie/State/c1.mp3" 9000
at "$H/Rip/tape.mp3" 9000
at "$H/NoWho/Real Book.mp3" 9000
at "$H/Multicast/Genesis.mp3" 9000
row "$H/Stross/Ship/c1.mp3" "Various Artists" "[Unknown Album]" "" 1 "Part 1"
row "$H/Leckie/State/c1.mp3" "Various Artists" "[Unknown Album]" "" 1 "Part 1"
row "$H/Rip/tape.mp3" "Ann Reader" "Unknown album (03/03/2015 21:39:55)" "" 1 "One"
row "$H/NoWho/Real Book.mp3" "[Unknown Artist]" "A Named Book" 2001 1 "One"
row "$H/Multicast/Genesis.mp3" "Various Artists" "The Bible" "" 1 "Genesis"
at "$H/Numbered/01 - Grimoire.mp3" 9000
at "$H/Digits/1984.mp3" 9000
row "$H/Numbered/01 - Grimoire.mp3" "0.4" "Grimoire of the Lamb" "" 1 "One"
row "$H/Digits/1984.mp3" "Ann Reader" "1984" 1949 1 "One"
# The enum TSV is what scripts/media-census.py enumerate writes: bytes, ext,
# path, no header.
python3 - "$AB" "$ENUM" <<'PY'
import os, sys
root, out = sys.argv[1], sys.argv[2]
with open(out, "w") as fh:
for dirpath, _dirs, files in os.walk(root):
for fn in files:
p = os.path.join(dirpath, fn)
fh.write(f"{os.path.getsize(p)}\t{os.path.splitext(fn)[1].lower()}\t{p}\n")
PY
}
plan() {
run python3 "$PLAN" --root "$AB" --enum "$ENUM" --rows "$ROWS" \
--out "$OUT" --exceptions "$EXC"
if [ "$status" -ne 0 ]; then
echo "planner said: $output"
return 1
fi
}
# db_with_part DB FILE NARRATOR BOOK YEAR TRACK CHAPTER — one Plex Track row
# under an Album under an Artist, in the three tables the export joins.
db_with_part() {
local db="$1" file="$2" narrator="$3" book="$4" year="$5" track="$6" chapter="$7"
local del="${8:-0}"
python3 - "$db" "$file" "$narrator" "$book" "$year" "$track" "$chapter" "$del" <<'PY'
import sqlite3, sys
db, file, narrator, book, year, track, chapter, deleted = sys.argv[1:9]
c = sqlite3.connect(db)
c.execute("create table if not exists media_parts (id integer primary key,"
" media_item_id, file, deleted_at)")
c.execute("create table if not exists media_items (id integer primary key,"
" metadata_item_id, deleted_at)")
c.execute("""create table if not exists metadata_items (id integer primary key,
parent_id, metadata_type, title, year, "index", deleted_at)""")
now = None if deleted == "0" else "2026-01-01"
ar = c.execute("insert into metadata_items (parent_id, metadata_type, title, year,"
" \"index\", deleted_at) VALUES (NULL, 8, ?, NULL, NULL, NULL)",
(narrator,)).lastrowid
al = c.execute("insert into metadata_items (parent_id, metadata_type, title, year,"
" \"index\", deleted_at) VALUES (?, 9, ?, ?, NULL, NULL)",
(ar, book, int(year) if year else None)).lastrowid
mi = c.execute("insert into metadata_items (parent_id, metadata_type, title, year,"
" \"index\", deleted_at) VALUES (?, 10, ?, NULL, ?, NULL)",
(al, chapter, int(track) if track else -1)).lastrowid
it = c.execute("insert into media_items (metadata_item_id, deleted_at) VALUES (?, NULL)",
(mi,)).lastrowid
c.execute("insert into media_parts (media_item_id, file, deleted_at) VALUES (?,?,?)",
(it, file, now))
c.commit()
PY
}
# ------------------------------------------------------------------ the export
@test "the census exports the narrator, book and chapter Plex matched for a part" {
DB="$BATS_TEST_TMPDIR/plex.db"
db_with_part "$DB" "$BATS_TEST_TMPDIR/Audiobooks/Pack/It part 1/01.mp3" \
"George Guidall" "It" 1986 1 "Chapter 1"
run python3 "$CENSUS" audiobooks --root "$BATS_TEST_TMPDIR/Audiobooks" --db "$DB" \
--out "$BATS_TEST_TMPDIR/exp.tsv"
[ "$status" -eq 0 ]
run head -1 "$BATS_TEST_TMPDIR/exp.tsv"
[ "${lines[0]}" = "file narrator book year track chapter" ]
tail -1 "$BATS_TEST_TMPDIR/exp.tsv" \
| grep -q "^$BATS_TEST_TMPDIR/Audiobooks/Pack/It part 1/01\.mp3 George Guidall It 1986 1 Chapter 1$"
}
@test "the census exports nothing outside the root, and nothing Plex deleted" {
DB="$BATS_TEST_TMPDIR/plex.db"
db_with_part "$DB" "/mnt/other/Library/01.mp3" "Someone" "Elsewhere" 1990 1 "One"
db_with_part "$DB" "$BATS_TEST_TMPDIR/Audiobooks/Gone/01.mp3" "Someone" \
"Deleted" 1990 1 "One" 1
run python3 "$CENSUS" audiobooks --root "$BATS_TEST_TMPDIR/Audiobooks" --db "$DB" \
--out "$BATS_TEST_TMPDIR/exp.tsv"
[ "$status" -eq 0 ]
[ "$(wc -l < "$BATS_TEST_TMPDIR/exp.tsv")" -eq 1 ] # header only
}
# -------------------------------------------------------------------- the plan
@test "a matched chapter files under its narrator and its book, name untouched" {
plan
[ "$(dst "$A [Part 1]/01 - Chapter 1.mp3")" = \
"$AB/George Guidall/It (1986)/01 - Chapter 1.mp3" ]
}
@test "one book arrives in one folder however many part folders it shipped in" {
plan
[ "$(dst "$A [Part 2]/03 - Chapter 3.mp3")" = \
"$AB/George Guidall/It (1986)/03 - Chapter 3.mp3" ]
[ "$(dst "$A [Part 1]/02 - Chapter 2.mp3")" = \
"$AB/George Guidall/It (1986)/02 - Chapter 2.mp3" ]
}
@test "a chapter is never renumbered, whether or not Plex knows its index" {
# The law this guards: playback order comes from the embedded track tag, and
# three quarters of the books have chapters with no index at all, so a
# number written into a name would be invented and would lie.
plan
[ "$(dst "$B/01.ogg")" = "$AB/Operator X/Project Gold/01.ogg" ]
[ "$(dst "$A [Part 1]/01 - Chapter 1.mp3")" = \
"$AB/George Guidall/It (1986)/01 - Chapter 1.mp3" ]
}
@test "a book whose album row has no year gets no parentheses" {
plan
[ "$(dst "$B/01.ogg")" = "$AB/Operator X/Project Gold/01.ogg" ]
}
@test "two chapters claiming one name inside one book are named apart" {
plan
first="$(dst "$D/Book A/vol1.mp3")"
second="$(dst "$D/Book A vol2/vol1.mp3")"
[ "$first" != "$second" ]
[ -n "$first" ] && [ -n "$second" ]
# and the plan still passes the executor's preflight, which refuses a target
# two sources claim.
run python3 "$APPLY" apply --plan "$OUT" --root "$AB"
[ "$status" -eq 0 ]
[[ "$output" != *"double-claim"* ]]
}
@test "a cover follows the book whose folder it lies in" {
plan
[ "$(dst "$A [Part 1]/cover.jpg")" = "$AB/George Guidall/It (1986)/cover.jpg" ]
}
@test "a playlist moves with its book and its bytes are never rewritten" {
plan
[ "$(dst "$A [Part 1]/It.m3u")" = "$AB/George Guidall/It (1986)/It.m3u" ]
before=$(md5sum "$A [Part 1]/It.m3u" | cut -d" " -f1)
run python3 "$APPLY" apply --plan "$OUT" --root "$AB" --journal "$BATS_TEST_TMPDIR/j.jsonl" --apply
[ "$status" -eq 0 ]
[ "$(md5sum "$AB/George Guidall/It (1986)/It.m3u" | cut -d" " -f1)" = "$before" ]
# the stored paths inside it still name the old file names, which the tool
# leaves alone rather than editing a file the operator owns.
[ "$(kind "$A [Part 1]/It.m3u")" != "chapter" ]
}
@test "a note in a folder holding two books has no owner and parks" {
plan
[ "$(kind "$F/notes.txt")" = "park" ]
[ "$(dst "$F/notes.txt")" = "$PARK/Mixed/notes.txt" ]
}
@test "an audio file no row names parks with the path it has today" {
plan
[ "$(kind "$G/loose.mp3")" = "park" ]
[ "$(dst "$G/loose.mp3")" = "$PARK/Odd/loose.mp3" ]
}
@test "a video container parks instead of being renamed into audio" {
plan
[ "$(kind "$G/concert.mp4")" = "park" ]
run grep -c "concert\.mp4" "$EXC"
[ "$output" != "0" ]
}
@test "a character Plex says breaks playback never reaches a destination" {
plan
[ "$(dst "$G/Bad: Name/01 chapter.mp3")" = \
"$AB/Some Reader/Trail Marked (1999)/01 chapter.mp3" ]
run cut -f3 "$OUT"
[[ "$output" != *":"* ]]
}
@test "one book title with two narrators stays two books" {
plan
[ "$(dst "$C/Dune side a.m4b")" = "$AB/Scott Brick/Dune (1965)/Dune side a.m4b" ]
[ "$(dst "$C/Dune 2020.m4b")" = \
"$AB/Malcolm Hillgartner/Dune (2020)/Dune 2020.m4b" ]
}
@test "the spelling most chapters carry names the folder, and they share one folder" {
plan
[ "$(dst "$E/kharn 1.mp3")" = "$AB/David Timson/Kharn (2013)/kharn 1.mp3" ]
[ "$(dst "$E/kharn 2.mp3")" = "$AB/David Timson/Kharn (2013)/kharn 2.mp3" ]
[ "$(dst "$E/kharn 3.mp3")" = "$AB/David Timson/Kharn (2013)/kharn 3.mp3" ]
}
@test "a row with no book title parks rather than inventing a folder" {
plan
[ "$(kind "$G/blank book.mp3")" = "park" ]
}
@test "a row with no narrator parks rather than inventing a shelf" {
plan
[ "$(kind "$G/blank narrator.mp3")" = "park" ]
}
@test "books that share a placeholder album name never share a shelf" {
plan
[ "$(dst "$H/Stross/Ship/c1.mp3")" = "$PARK/Packs/Stross/Ship/c1.mp3" ]
[ "$(dst "$H/Leckie/State/c1.mp3")" = "$PARK/Packs/Leckie/State/c1.mp3" ]
}
@test "a placeholder with a rip timestamp glued to it is still a placeholder" {
plan
[ "$(kind "$H/Rip/tape.mp3")" = "park" ]
}
@test "a narrator placeholder parks even when the book is named" {
plan
[ "$(kind "$H/NoWho/Real Book.mp3")" = "park" ]
}
@test "Various Artists is a multi-cast narrator and files its book" {
plan
[ "$(dst "$H/Multicast/Genesis.mp3")" = \
"$AB/Various Artists/The Bible/Genesis.mp3" ]
}
@test "a chapter number mistagged as the narrator parks" {
plan
[ "$(kind "$H/Numbered/01 - Grimoire.mp3")" = "park" ]
}
@test "a book whose title is digits still files, because only a name must have a letter" {
plan
[ "$(dst "$H/Digits/1984.mp3")" = "$AB/Ann Reader/1984 (1949)/1984.mp3" ]
}
@test "the row, not the extension, decides what a chapter is" {
plan
[ "$(kind "$G/NoExt/part01")" = "chapter" ]
[ "$(dst "$G/NoExt/part01")" = "$AB/Archivist/Tapes (1971)/part01" ]
}
@test "the plan replays through the executor with nothing refused" {
plan
run python3 "$APPLY" apply --plan "$OUT" --root "$AB" \
--journal "$BATS_TEST_TMPDIR/journal.jsonl"
[ "$status" -eq 0 ]
[[ "$output" != *"refusing"* ]]
# The refusal this guards is two sources claiming one target: the executor
# stops the whole run on it, so the plan must never hand a path out twice.
[ -z "$(awk -F '\t' '$1 == "move" {print $3}' "$OUT" | sort | uniq -d)" ]
}
@test "an occupied destination is refused rather than overwritten" {
plan
mkdir -p "$AB/George Guidall/It (1986)"
at "$AB/George Guidall/It (1986)/01 - Chapter 1.mp3" 1
run python3 "$APPLY" apply --plan "$OUT" --root "$AB"
[ "$status" -ne 0 ]
[[ "$output" == *"occupied"* ]]
[ -e "$A [Part 1]/01 - Chapter 1.mp3" ]
}
@test "applying the plan and re-planning moves nothing" {
plan
run python3 "$APPLY" apply --plan "$OUT" --root "$AB" \
--journal "$BATS_TEST_TMPDIR/j.jsonl" --apply
[ "$status" -eq 0 ]
python3 - "$AB" "$ENUM" <<'PY'
import os, sys
root, out = sys.argv[1], sys.argv[2]
with open(out, "w") as fh:
for dirpath, _dirs, files in os.walk(root):
for fn in files:
p = os.path.join(dirpath, fn)
fh.write(f"{os.path.getsize(p)}\t{os.path.splitext(fn)[1].lower()}\t{p}\n")
PY
# The rows still name the paths the chapters were placed at: that is what a
# server re-scan of the moved tree reports.
python3 - "$OUT" "$ROWS" <<'PY'
import csv, sys
out, rows = sys.argv[1], sys.argv[2]
moved = {r[1]: r[2] for r in csv.reader(open(out), delimiter="\t") if r[0] == "move"}
with open(rows) as fh, open(rows + ".2", "w") as fh2:
fh2.write(fh.readline().rstrip("\n") + "\n")
for line in fh:
f = line.split("\t")[0]
fh2.write("\t".join([moved.get(f, f)] + line.rstrip("\n").split("\t")[1:]) + "\n")
PY
run python3 "$PLAN" --root "$AB" --enum "$ENUM" --rows "$ROWS.2" \
--out "$OUT.2" --exceptions "$EXC.2"
[ "$status" -eq 0 ]
[ "$(awk -F '\t' '$1 == "move"' "$OUT.2" | wc -l)" -eq 0 ]
}
@test "nothing under the park root is planned again" {
plan
run awk -F '\t' '$1 == "move" && $2 ~ /_unsorted/' "$OUT"
[ -z "$output" ]
}
+373
View File
@@ -0,0 +1,373 @@
#!/usr/bin/env bats
# Seam: the music-library relayout — scripts/media-census.py music (the tag and
# provider authority export) and scripts/media-music.py (the plan).
#
# Jellyfin's music rule is the opposite of the movie and TV ones: the embedded
# tags are the metadata and the file name is only a fallback title, so the
# authority here is the Audio row Jellyfin already matched (its album-artist,
# album, year, disc and track columns) and NOT the path a ripper chose. Every
# test states which authority a file has — row only, row with no album item,
# sidecar of a file that has a row, nothing at all — and asserts where the PLAN
# files it, never how the planner decided.
#
# The three outcomes that cost a re-run of a 90k-file plan: a track filed under
# the wrong artist (it splits one album across two artists), a destination that
# two tracks claim (the executor refuses, and the operator loses the run), and a
# lyric file that stops matching the basename of the track it belongs to, which
# silently loses the lyrics. Everything below is written to catch those, plus the
# gate that matters most: the plan must replay through scripts/media-apply.py,
# whose preflight is the fleet's only overwrite guard.
load helpers
PLAN="$REPO_ROOT/scripts/media-music.py"
CENSUS="$REPO_ROOT/scripts/media-census.py"
APPLY="$REPO_ROOT/scripts/media-apply.py"
# at PATH SIZE — a file with deterministic content.
at() { mkdir -p "$(dirname "$1")"; head -c "$2" /dev/zero > "$1"; }
# row FILE ARTIST ALBUM YEAR DISC TRACK TITLE [MB] [PERFORMER]
# One row of the authority export scripts/media-census.py music writes.
row() {
printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \
"$1" "$2" "$3" "${4:-}" "${5:-}" "${6:-}" "$7" "${8:-}" "${9:-}" >> "$ROWS"
}
# dst SRC — the destination the plan gives SRC, or nothing when it does not move.
dst() {
awk -F '\t' -v s="$1" '$1 == "move" && $2 == s {print $3}' "$OUT"
}
setup() {
MUSIC="$BATS_TEST_TMPDIR/Music"
PARK="$MUSIC/_unsorted"
ENUM="$BATS_TEST_TMPDIR/enum.tsv"
ROWS="$BATS_TEST_TMPDIR/rows.tsv"
OUT="$BATS_TEST_TMPDIR/plan.tsv"
EXC="$BATS_TEST_TMPDIR/exceptions.txt"
# A discography pack: the artist's own albums buried under "01. Albums",
# with a lyric, a cover, an EAC log and a playlist beside the tracks.
A="$MUSIC/Artist One - Discography [FLAC]/01. Albums/Album X"
at "$A/01. First.flac" 9000
at "$A/02. Second.flac" 9000
at "$A/01. First.lrc" 10
at "$A/cover.jpg" 20
at "$A/rip.log" 10
at "$A/Album X.m3u" 10
# A multi-disc compilation: one album (the album artist is the compilation
# tag), two discs, and a different performer on each track.
B="$MUSIC/Box (2019)"
at "$B/Disc 1/01 Track A.flac" 9000
at "$B/Disc 2/01 Track B.flac" 9000
# The box cover lies above the two disc folders: the discs below it own it.
at "$B/box.jpg" 10
# One image plus its cuesheet: a whole album in one file.
C="$MUSIC/Image Cue"
at "$C/Image.flac" 9000
at "$C/Image.cue" 10
# The hard cases: no row at all, a video container, two tracks claiming one
# number and one title, a track with no number, and a track whose album
# artist tag is empty so only the performer names it.
D="$MUSIC/Odd"
at "$D/loose.flac" 9000
at "$D/concert.mkv" 9000
at "$D/dup1.flac" 9000
at "$D/dup2.flac" 9000
at "$D/unnumbered.flac" 9000
at "$D/lone.flac" 9000
# A sidecar in a folder that holds two different albums.
E="$MUSIC/Mixed"
at "$E/one.flac" 9000
at "$E/two.flac" 9000
at "$E/notes.pdf" 10
# An album title carrying a character Jellyfin says breaks playback.
F="$MUSIC/Question Mark"
at "$F/01. Bang.flac" 9000
# An album whose tags name no album artist at all, and two different
# performers; and one album title carried by two different album artists.
G="$MUSIC/Tagless"
at "$G/one.flac" 9000
at "$G/two.flac" 9000
H="$MUSIC/Same Title"
at "$H/a.flac" 9000
at "$H/b.flac" 9000
# One album whose tracks disagree on the spelling of its name; the minority
# spelling sorts first, so it must not win the folder.
I="$MUSIC/Puppets"
at "$I/one.flac" 9000
at "$I/two.flac" 9000
at "$I/three.flac" 9000
# The authority export carries a header; the planner reads it by column name.
printf 'file\tartist\talbum\tyear\tdisc\ttrack\ttitle\tmb\tperformer\n' > "$ROWS"
row "$A/01. First.flac" "Artist One" "Album X" 2015 1 1 "First Song"
row "$A/02. Second.flac" "Artist One" "Album X" 2015 1 2 "Second Song: Reprise"
row "$B/Disc 1/01 Track A.flac" "Various Artists" "Best Of" 2019 1 1 "Track A" \
"" "Aaa Singer"
row "$B/Disc 2/01 Track B.flac" "Various Artists" "Best Of" 2019 2 1 "Track B" \
"" "Bbb Singer"
row "$C/Image.flac" "Solo" "Single Album" 2001 1 1 "Single Album"
row "$D/dup1.flac" "Dup Artist" "Dupes" "" "" 1 "Same"
row "$D/dup2.flac" "Dup Artist" "Dupes" "" "" 1 "Same"
row "$D/unnumbered.flac" "No Number" "No Number Album" "" "" "" "Untitled"
row "$D/lone.flac" "" "Session Works" "" "" 1 "Lone" "" "Session Singer"
row "$E/one.flac" "Mix Artist" "First Set" 2000 "" 1 "One"
row "$E/two.flac" "Mix Artist" "Second Set" 2001 "" 1 "Two"
row "$F/01. Bang.flac" "Billie" "WHEN WE ALL FALL ASLEEP, WHERE DO WE GO?" \
2019 "" 1 "!!!!!!"
row "$G/one.flac" "" "Two Seasons" 2005 "" 1 "One" "" "Alpha Band"
row "$G/two.flac" "" "Two Seasons" 2005 "" 2 "Two" "" "Beta Band"
row "$H/a.flac" "Aaa Singer" "Same Title" 2011 "" 1 "A Song"
row "$H/b.flac" "Bbb Singer" "Same Title" 2011 "" 1 "B Song"
row "$I/one.flac" "Metallica" "Master Of Puppets" 2015 "" 1 "One"
row "$I/two.flac" "Metallica" "Master of Puppets" 2015 "" 2 "Two"
row "$I/three.flac" "Metallica" "Master of Puppets" 2015 "" 3 "Three"
# The enum TSV is what scripts/media-census.py enumerate writes: bytes, ext,
# path, no header.
python3 - "$MUSIC" "$ENUM" <<'PY'
import os, sys
root, out = sys.argv[1], sys.argv[2]
with open(out, "w") as fh:
for dirpath, _dirs, files in os.walk(root):
for fn in files:
p = os.path.join(dirpath, fn)
fh.write(f"{os.path.getsize(p)}\t{os.path.splitext(fn)[1].lower()}\t{p}\n")
PY
}
plan() {
run python3 "$PLAN" --root "$MUSIC" --enum "$ENUM" --rows "$ROWS" \
--out "$OUT" --exceptions "$EXC"
[ "$status" -eq 0 ] || echo "planner said: $output"
}
@test "the census exports the tag and provider row Jellyfin matched for a track" {
DB="$BATS_TEST_TMPDIR/jellyfin.db"
python3 - "$DB" <<'PY'
import sqlite3, sys, uuid
c = sqlite3.connect(sys.argv[1])
c.execute("""create table BaseItems (Id, ParentId, Type, Name, Path, IndexNumber,
ParentIndexNumber, ProductionYear, Album,
AlbumArtists, Artists, Data)""")
c.execute("""create table BaseItemProviders (ItemId, ProviderId, ProviderValue)""")
AU = "MediaBrowser.Controller.Entities.Audio.Audio"
AL = "MediaBrowser.Controller.Entities.Audio.MusicAlbum"
aid, trk = str(uuid.uuid4()), "/data/Music/Pack/Album X/01. First.flac"
c.execute("insert into BaseItems values (?,?,?,?,?,?,?,?,?,?,?,?)",
(aid, None, AL, "Album X", "/data/Music/Pack/Album X", None, None, 2015,
None, "Artist One", None, '{}'))
c.execute("insert into BaseItemProviders values (?,?,?)",
(aid, "MusicBrainzReleaseGroup", "rg-1234"))
c.execute("insert into BaseItems values (?,?,?,?,?,?,?,?,?,?,?,?)",
(str(uuid.uuid4()), aid, AU, "First Song", trk, 1, 1, 2015,
"album x (the tag)", "Artist One", "Artist One", '{}'))
c.commit()
PY
run python3 "$CENSUS" music --root "$BATS_TEST_TMPDIR/Music" --db "$DB" \
--out "$BATS_TEST_TMPDIR/exp.tsv"
[ "$status" -eq 0 ]
run head -1 "$BATS_TEST_TMPDIR/exp.tsv"
[ "${lines[0]}" = "file artist album year disc track title mb performer" ]
tail -1 "$BATS_TEST_TMPDIR/exp.tsv" \
| grep -q "^$BATS_TEST_TMPDIR/Music/Pack/Album X/01\. First\.flac Artist One Album X 2015 1 1 First Song rg-1234 Artist One$"
}
@test "a track whose folder is no album item still exports from its own tags" {
DB="$BATS_TEST_TMPDIR/jellyfin.db"
python3 - "$DB" <<'PY'
import sqlite3, sys, uuid
c = sqlite3.connect(sys.argv[1])
c.execute("""create table BaseItems (Id, ParentId, Type, Name, Path, IndexNumber,
ParentIndexNumber, ProductionYear, Album,
AlbumArtists, Artists, Data)""")
c.execute("""create table BaseItemProviders (ItemId, ProviderId, ProviderValue)""")
c.execute("insert into BaseItems values (?,?,?,?,?,?,?,?,?,?,?,?)",
(str(uuid.uuid4()), None, "MediaBrowser.Controller.Entities.Audio.Audio",
"Lone", "/data/Music/Pack/lone.flac", 1, None, None,
"Session Works", None, "Session Singer", '{}'))
c.commit()
PY
run python3 "$CENSUS" music --root "$BATS_TEST_TMPDIR/Music" --db "$DB" \
--out "$BATS_TEST_TMPDIR/exp.tsv"
[ "$status" -eq 0 ]
tail -1 "$BATS_TEST_TMPDIR/exp.tsv" \
| grep -q "^$BATS_TEST_TMPDIR/Music/Pack/lone\.flac Session Singer Session Works 1 Lone Session Singer$"
}
@test "a tagged track files under artist, album and year with a numbered title" {
plan
[ "$status" -eq 0 ]
[ "$(dst "$A/01. First.flac")" = "$MUSIC/Artist One/Album X (2015)/01 - First Song.flac" ]
}
@test "the tag title wins over the name the ripper gave the file" {
plan
[ "$(dst "$A/02. Second.flac")" = "$MUSIC/Artist One/Album X (2015)/02 - Second Song Reprise.flac" ]
}
@test "a character Jellyfin says breaks playback never reaches a destination" {
plan
[ "$(dst "$F/01. Bang.flac")" = "$MUSIC/Billie/WHEN WE ALL FALL ASLEEP, WHERE DO WE GO (2019)/01 - !!!!!!.flac" ]
run awk -F '\t' '$1 == "move" && $3 ~ /[<>:"|?*]/ {print $3}' "$OUT"
[ -z "$output" ]
}
@test "every disc of a multi-disc album lands in one folder, numbered apart" {
plan
[ "$(dst "$B/Disc 1/01 Track A.flac")" = "$MUSIC/Various Artists/Best Of (2019)/1.01 - Track A.flac" ]
[ "$(dst "$B/Disc 2/01 Track B.flac")" = "$MUSIC/Various Artists/Best Of (2019)/2.01 - Track B.flac" ]
}
@test "an album whose tracks name several artists files under Various Artists" {
plan
run awk -F '\t' '$1 == "move" && $3 ~ /\/Best Of \(2019\)\/.*\.flac$/ {print $3}' "$OUT"
[ "${#lines[@]}" -eq 2 ]
[[ "${lines[0]}" == "$MUSIC/Various Artists/"* ]]
}
@test "an album with no album artist and two performers is a compilation" {
plan
[ "$(dst "$G/one.flac")" = "$MUSIC/Various Artists/Two Seasons (2005)/01 - One.flac" ]
[ "$(dst "$G/two.flac")" = "$MUSIC/Various Artists/Two Seasons (2005)/02 - Two.flac" ]
}
@test "the same album title under two album artists is two albums" {
plan
[ "$(dst "$H/a.flac")" = "$MUSIC/Aaa Singer/Same Title (2011)/01 - A Song.flac" ]
[ "$(dst "$H/b.flac")" = "$MUSIC/Bbb Singer/Same Title (2011)/01 - B Song.flac" ]
}
@test "the folder carries the album spelling most tracks carry" {
plan
[ "$(dst "$I/one.flac")" = "$MUSIC/Metallica/Master of Puppets (2015)/01 - One.flac" ]
[ "$(dst "$I/two.flac")" = "$MUSIC/Metallica/Master of Puppets (2015)/02 - Two.flac" ]
[ "$(dst "$I/three.flac")" = "$MUSIC/Metallica/Master of Puppets (2015)/03 - Three.flac" ]
}
@test "a lyric file is renamed to match the track it belongs to" {
plan
[ "$(dst "$A/01. First.lrc")" = "$MUSIC/Artist One/Album X (2015)/01 - First Song.lrc" ]
}
@test "the album image, the rip log and the playlist follow the album folder" {
plan
[ "$(dst "$A/cover.jpg")" = "$MUSIC/Artist One/Album X (2015)/cover.jpg" ]
[ "$(dst "$A/rip.log")" = "$MUSIC/Artist One/Album X (2015)/rip.log" ]
[ "$(dst "$A/Album X.m3u")" = "$MUSIC/Artist One/Album X (2015)/Album X.m3u" ]
}
@test "a cover above the disc folders of one album follows that album" {
plan
[ "$(dst "$B/box.jpg")" = "$MUSIC/Various Artists/Best Of (2019)/box.jpg" ]
}
@test "an image carrying a whole album keeps its own name beside its cuesheet" {
plan
[ "$(dst "$C/Image.flac")" = "$MUSIC/Solo/Single Album (2001)/Image.flac" ]
[ "$(dst "$C/Image.cue")" = "$MUSIC/Solo/Single Album (2001)/Image.cue" ]
}
@test "an audio file no provider row names is parked, not filed under an artist" {
plan
[[ "$(dst "$D/loose.flac")" == "$PARK/Odd/loose.flac" ]]
}
@test "a video container in the music library is parked, never renamed to an audio one" {
plan
[[ "$(dst "$D/concert.mkv")" == "$PARK/Odd/concert.mkv" ]]
run awk -F '\t' '$1 == "move" && $3 ~ /\.mka$|\.m4a$/ {print $3}' "$OUT"
[ -z "$output" ]
}
@test "two tracks claiming one number and one title get two destinations" {
plan
[ "$(dst "$D/dup1.flac")" = "$MUSIC/Dup Artist/Dupes/01 - Same.flac" ]
[ "$(dst "$D/dup2.flac")" = "$MUSIC/Dup Artist/Dupes/01 - Same (2).flac" ]
}
@test "an album with no year is named without one, a track with no number likewise" {
plan
[ "$(dst "$D/unnumbered.flac")" = "$MUSIC/No Number/No Number Album/Untitled.flac" ]
}
@test "a track naming only a performer files under that performer" {
plan
[ "$(dst "$D/lone.flac")" = "$MUSIC/Session Singer/Session Works/01 - Lone.flac" ]
}
@test "a sidecar in a folder holding two albums parks instead of guessing" {
plan
[[ "$(dst "$E/notes.pdf")" == "$PARK/Mixed/notes.pdf" ]]
}
@test "the plan creates every folder before the move that needs it" {
plan
python3 - "$OUT" "$MUSIC" <<'PY'
import os, sys
made = set()
for line in open(sys.argv[1]).read().splitlines()[1:]:
action, _src, dest, _kind = line.split("\t")
if action == "mkdir":
made.add(dest)
elif action == "move":
parent = os.path.dirname(dest)
while parent and parent != sys.argv[2]:
assert parent in made, f"{dest} needs {parent}, which is never made"
parent = os.path.dirname(parent)
PY
}
@test "no move lands outside the music library" {
plan
run awk -F '\t' -v l="$MUSIC" \
'$1 == "move" && $3 !~ "^" l "/" {print $3}' "$OUT"
[ -z "$output" ]
}
@test "the executor preflights the plan and no target is claimed twice" {
plan
run python3 "$APPLY" apply --plan "$OUT" --root "$BATS_TEST_TMPDIR" \
--journal "$BATS_TEST_TMPDIR/journal.jsonl"
[ "$status" -eq 0 ]
dup=$(awk -F '\t' '$1 == "move" {print $3}' "$OUT" | sort | uniq -d)
[ -z "$dup" ]
}
@test "re-planning a library the plan already satisfies moves nothing" {
plan
# Applying the plan and re-censusing is what the operator does next: both
# authority files are keyed by path, so both are replayed through the plan
# the way a fresh census of the moved tree would see them. (The enum has no
# header; the rows export does, and its header row is not a path.)
python3 - "$OUT" "$ENUM" "$ROWS" <<'PY'
import os, sys
plan = sys.argv[1]
moves = {}
for line in open(plan).read().splitlines()[1:]:
action, src, dest, _kind = line.split("\t")
if action == "move":
moves[src] = dest
os.makedirs(os.path.dirname(dest), exist_ok=True)
os.rename(src, dest)
for path, col in ((sys.argv[2], 2), (sys.argv[3], 0)):
rows = [line.rstrip("\n").split("\t") for line in open(path)]
with open(path, "w") as fh:
for f in rows:
f[col] = moves.get(f[col], f[col])
fh.write("\t".join(f) + "\n")
PY
plan
run awk -F '\t' '$1 == "move" {print $2 " -> " $3}' "$OUT"
[ -z "$output" ] || { echo "moved again: $output"; false; }
}
+307
View File
@@ -0,0 +1,307 @@
#!/usr/bin/env bats
# Seam: the TV-library relayout — scripts/media-census.py jellyfin (the metadata
# authority export) and scripts/media-shows.py (the plan).
#
# The TV library is 52k files written in five different release layouts, and the
# one thing that makes a rename of that size survivable is that the plan is
# derived from what a metadata provider already matched, not from a guess at the
# file name. So every test here states which authority a file had — name only,
# provider only, both, neither — and asserts where the PLAN files it, never how
# the planner decided. A fixture Jellyfin database stands in for the live one and
# the TMDb cache is an input file, so no test touches the network.
#
# The two outcomes that cost a re-run of a 50k-file plan are a file filed as an
# episode that is not one (it steals an episode number from its real owner) and a
# move that lands on an existing file. Everything below is written to catch
# exactly those, plus the gate that matters most: the plan must be replayable by
# scripts/media-apply.py, whose preflight is the fleet's only overwrite guard.
load helpers
PLAN="$REPO_ROOT/scripts/media-shows.py"
CENSUS="$REPO_ROOT/scripts/media-census.py"
APPLY="$REPO_ROOT/scripts/media-apply.py"
# at PATH SIZE — a file with deterministic content.
at() { mkdir -p "$(dirname "$1")"; head -c "$2" /dev/zero > "$1"; }
# title_row FILE SERIES YEAR SEASON EPISODE [EXTRA] — one row of the authority
# export scripts/media-census.py jellyfin writes.
title_row() {
printf '%s\t%s\t%s\t%s\t%s\t%s\t\n' "$1" "$2" "$3" "${4:-}" "${5:-}" "${6:-}" \
>> "$TITLES"
}
# dst SRC — the destination the plan gives SRC, or nothing when it does not move.
dst() {
awk -F '\t' -v s="$1" '$1 == "move" && $2 == s {print $3}' "$OUT"
}
setup() {
LIB="$BATS_TEST_TMPDIR/TVShows"
PARK="$LIB/_unsorted"
MOVIES="$BATS_TEST_TMPDIR/Movies"
ENUM="$BATS_TEST_TMPDIR/enum.tsv"
PROBED="$BATS_TEST_TMPDIR/probed.tsv"
TITLES="$BATS_TEST_TMPDIR/titles.tsv"
CACHE="$BATS_TEST_TMPDIR/tmdb.json"
OUT="$BATS_TEST_TMPDIR/plan.tsv"
EXC="$BATS_TEST_TMPDIR/exceptions.txt"
: > "$PROBED"
SEASON="$LIB/The Examples (2010) Season 1 S01 (1080p BluRay x265 Silence)"
# An episode its own name states, with a subtitle, a season image and a
# trailer: four things that have to end up in four different places.
at "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.mkv" 9000
at "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.en.srt" 200
at "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.mkv.forced.srt" 250
# A scene pack buries its subtitles two levels down, in Subs/<release-name>/.
at "$SEASON/Subs/The.Examples.S01E01.Pilot.1080p.x265/The.Examples.S01E01.Pilot.1080p.x265.jpn.srt" 220
at "$SEASON/folder.jpg" 300
at "$SEASON/Trailers/teaser.mkv" 1000
at "$SEASON/tvshow.nfo" 400
# An anime pack: no episode marker anywhere in its name, numbered by the
# provider only. Its year is the one thing no authority states.
ANIME="$LIB/Anime Pack [grp] (1080p HEVC)"
at "$ANIME/[grp] Anime Pack - 05.mkv" 5000
# Two releases claiming one episode, and a file holding two episodes.
at "$LIB/Dupes/Show Two S01E01.mkv" 4000
at "$LIB/Dupes/Show.Two.S01E01.720p.other.mkv" 7000
at "$LIB/Dupes/Show.Two.S01E02-E03.720p.mkv" 7000
# A file no authority numbers, and a long file the movie provider knows.
at "$LIB/Orphan Tapes/clip001.mp4" 3000
at "$LIB/Longform/Some Documentary Special (2015).mkv" 8000
# A bonus folder inside a release pack whose own name is not the show's. The
# pack's rows name the show; the bonus folder may never invent a series.
at "$ANIME/01. Extras/NCOP 01.mkv" 1200
# A season folder whose rows are stale: the database numbers every file of this
# pack season 16 episode 20, four seasons' worth of them.
at "$LIB/SNL Pack/Season 19/01 Charles Barkley, Nirvana.avi" 6000
# One episode-shaped file loose at the library root.
at "$LIB/Loose.Show.S02E03.720p.WEB-DL.mkv" 4000
# The enum census lists every file; the planner reads that instead of walking
# the tree, so the fixture writes the same three columns.
python3 - "$LIB" "$ENUM" <<'PY'
import os, sys
root, out = sys.argv[1], sys.argv[2]
with open(out, "w") as fh:
for d, _, fs in os.walk(root):
for f in fs:
p = os.path.join(d, f)
fh.write(f"{os.stat(p).st_size}\t{os.path.splitext(f)[1]}\t{p}\n")
PY
printf 'file\tseries\tyear\tseason\tepisode\textra\ttmdb\n' > "$TITLES"
printf '{}\n' > "$CACHE"
title_row "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.mkv" \
"The Examples" 2010 1 1
title_row "$ANIME/[grp] Anime Pack - 05.mkv" "Anime Pack" "" 1 5
title_row "$LIB/Dupes/Show Two S01E01.mkv" "Show Two" 2016 1 1
title_row "$LIB/Dupes/Show.Two.S01E01.720p.other.mkv" "Show Two" 2016 1 1
title_row "$LIB/Dupes/Show.Two.S01E02-E03.720p.mkv" "Show Two" 2016 1 2
title_row "$LIB/Loose.Show.S02E03.720p.WEB-DL.mkv" "Loose Show" 2019 2 3
title_row "$LIB/SNL Pack/Season 19/01 Charles Barkley, Nirvana.avi" \
"SNL Pack" "" 16 20
}
plan() {
run python3 "$PLAN" --root "$LIB" --enum "$ENUM" --titles "$TITLES" \
--probed "$PROBED" --tmdb-cache "$CACHE" --movies "$MOVIES" \
--out "$OUT" --exceptions "$EXC"
}
@test "the census exports the provider's episode numbering as a TSV" {
DB="$BATS_TEST_TMPDIR/jellyfin.db"
python3 - "$DB" "$SEASON" <<'PY'
import os, sqlite3, sys, uuid
db, season = sys.argv[1], sys.argv[2]
c = sqlite3.connect(db)
pre = "/data/TVShows/" + os.path.basename(season)
c.execute("""create table BaseItems (Id, ParentId, Type, Name, Path, IndexNumber,
ParentIndexNumber, ProductionYear, ExtraType,
IsVirtualItem, Data)""")
sid = str(uuid.uuid4())
c.execute("insert into BaseItems values (?,?,?,?,?,?,?,?,?,?,?)",
(sid, None, "MediaBrowser.Controller.Entities.TV.Series", "The Examples",
pre, None, None, 2010, None, 0,
'{"ProviderIds":{"Tmdb":"1234"}}'))
c.execute("insert into BaseItems values (?,?,?,?,?,?,?,?,?,?,?)",
(str(uuid.uuid4()), sid, "MediaBrowser.Controller.Entities.TV.Episode",
"Pilot", pre + "/The.Examples.S01E01.Pilot.1080p.x265.mkv", 1, 1,
None, None, 0, '{}'))
c.commit()
PY
run python3 "$CENSUS" jellyfin --root "$LIB" --db "$DB" \
--out "$BATS_TEST_TMPDIR/exp.tsv"
[ "$status" -eq 0 ]
run head -1 "$BATS_TEST_TMPDIR/exp.tsv"
[ "${lines[0]}" = "file series year season episode extra tmdb" ]
tail -1 "$BATS_TEST_TMPDIR/exp.tsv" \
| grep -q "^$SEASON/The\.Examples\.S01E01\.Pilot\.1080p\.x265\.mkv The Examples 2010 1 1 1234$"
}
@test "an episode its own name states files as Show SxxEyy under Season NN" {
plan
[ "$status" -eq 0 ]
[ "$(dst "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.mkv")" \
= "$LIB/The Examples (2010)/Season 01/The Examples S01E01.mkv" ]
}
@test "a subtitle follows its episode and keeps its language flag" {
plan
[ "$(dst "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.en.srt")" \
= "$LIB/The Examples (2010)/Season 01/The Examples S01E01.en.srt" ]
}
@test "a subtitle two folders down inside Subs/<release-name>/ still follows its episode" {
plan
[ "$(dst "$SEASON/Subs/The.Examples.S01E01.Pilot.1080p.x265/The.Examples.S01E01.Pilot.1080p.x265.jpn.srt")" \
= "$LIB/The Examples (2010)/Season 01/The Examples S01E01.jpn.srt" ]
}
@test "a subtitle that carries the container's word in its own name still follows the episode" {
plan
[ "$(dst "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.mkv.forced.srt")" \
= "$LIB/The Examples (2010)/Season 01/The Examples S01E01.forced.srt" ]
}
@test "a season image keeps its own name; a series nfo keeps the series folder" {
plan
[ "$(dst "$SEASON/folder.jpg")" = "$LIB/The Examples (2010)/Season 01/folder.jpg" ]
[ "$(dst "$SEASON/tvshow.nfo")" = "$LIB/The Examples (2010)/tvshow.nfo" ]
}
@test "bonus material files as an extra, never as an episode" {
plan
[ "$(dst "$SEASON/Trailers/teaser.mkv")" = "$LIB/The Examples (2010)/trailers/teaser.mkv" ]
[ -z "$(awk -F '\t' '$3 ~ /Season .*teaser/ {print $3}' "$OUT")" ]
}
@test "a file only the provider numbers files at the episode it is matched to" {
plan
[ "$(dst "$ANIME/[grp] Anime Pack - 05.mkv")" \
= "$LIB/Anime Pack/Season 01/Anime Pack S01E05.mkv" ]
}
@test "a bonus folder inside a release pack belongs to the pack's show, not to its own name" {
plan
[ "$(dst "$ANIME/01. Extras/NCOP 01.mkv")" = "$LIB/Anime Pack/extras/NCOP 01.mkv" ]
[ -z "$(awk -F '\t' '$3 ~ /\/01\. Extras\// {print $3}' "$OUT")" ]
}
@test "a Season folder outvotes the stale row its files carry" {
plan
[ "$(dst "$LIB/SNL Pack/Season 19/01 Charles Barkley, Nirvana.avi")" \
= "$LIB/SNL Pack/Season 19/SNL Pack S19E01.avi" ]
}
@test "one file holding two episodes keeps both numbers in its name" {
plan
[ "$(dst "$LIB/Dupes/Show.Two.S01E02-E03.720p.mkv")" \
= "$LIB/Show Two (2016)/Season 01/Show Two S01E02-E03.mkv" ]
}
@test "two files claiming one episode: one keeps the name, the other parks" {
plan
n=$(grep -c " $LIB/Show Two (2016)/Season 01/Show Two S01E01.mkv " "$OUT")
[ "$n" -eq 1 ]
run dst "$LIB/Dupes/Show Two S01E01.mkv"
[[ "$output" == "$PARK/"* ]]
run dst "$LIB/Dupes/Show.Two.S01E01.720p.other.mkv"
[[ "$output" == "$LIB/Show Two (2016)/Season 01/"* ]]
}
@test "a file no authority numbers is parked, never renamed into the library" {
plan
run dst "$LIB/Orphan Tapes/clip001.mp4"
[[ "$output" == "$PARK/Orphan Tapes/clip001.mp4" ]]
grep -q "clip001.mp4" "$EXC"
}
@test "a long file the movie provider knows leaves the TV library for the movie one" {
printf '8000\t%s\t7200\thevc\t1920x1080\t23.976\t4000000\tyuv420p10le\taac:eng\t0\t0\tmatroska,webm\n' \
"$LIB/Longform/Some Documentary Special (2015).mkv" > "$PROBED"
python3 -c 'import json, sys
json.dump({"movie:some documentary":
{"name": "Some Documentary Special", "year": 2015, "id": 55}},
open(sys.argv[1], "w"))' "$CACHE"
plan
[ "$(dst "$LIB/Longform/Some Documentary Special (2015).mkv")" \
= "$MOVIES/Some Documentary Special (2015)/Some Documentary Special (2015).mkv" ]
}
@test "a long file no movie provider knows is parked, not moved into Movies" {
printf '8000\t%s\t7200\thevc\t1920x1080\t23.976\t4000000\tyuv420p10le\taac:eng\t0\t0\tmatroska,webm\n' \
"$LIB/Longform/Some Documentary Special (2015).mkv" > "$PROBED"
plan
run dst "$LIB/Longform/Some Documentary Special (2015).mkv"
[[ "$output" == "$PARK/"* ]]
}
@test "the plan creates every folder before the move that needs it" {
plan
python3 - "$OUT" "$LIB" "$MOVIES" "$PARK" <<'PY'
import os, sys
plan, known = sys.argv[1], set(sys.argv[2:])
made, bad = set(), []
for line in open(plan).read().splitlines()[1:]:
action, _src, dest, _kind = line.split("\t")
if action == "mkdir":
made.add(dest)
elif action == "move":
d = os.path.dirname(dest)
while d not in known and d != os.path.dirname(d):
if d not in made:
bad.append(f"{dest}: {d} was never made")
d = os.path.dirname(d)
print("\n".join(bad[:5]))
sys.exit(1 if bad else 0)
PY
}
@test "no move lands outside the library or the movie library" {
plan
run awk -F '\t' -v l="$LIB" -v m="$MOVIES" \
'$1 == "move" && $3 !~ "^" l "/" && $3 !~ "^" m "/" {print $3}' "$OUT"
[ -z "$output" ]
}
@test "the executor preflights the plan and no target is claimed twice" {
plan
run python3 "$APPLY" apply --plan "$OUT" --root "$BATS_TEST_TMPDIR" \
--journal "$BATS_TEST_TMPDIR/journal.jsonl"
[ "$status" -eq 0 ]
dup=$(awk -F '\t' '$1 == "move" {print $3}' "$OUT" | sort | uniq -d)
[ -z "$dup" ]
}
@test "re-planning a library the plan already satisfies moves nothing" {
plan
# Applying the plan and re-censusing is exactly what the operator does next:
# every authority file is keyed by path, so all three are replayed through the
# plan the way a fresh census would see them. (The enum has no header; the
# titles export does, and its header row is not a path so it survives.)
python3 - "$OUT" "$ENUM" "$TITLES" "$PROBED" <<'PY'
import os, sys
plan = sys.argv[1]
moves = {}
for line in open(plan).read().splitlines()[1:]:
action, src, dest, _kind = line.split("\t")
if action == "move":
moves[src] = dest
os.makedirs(os.path.dirname(dest), exist_ok=True)
os.rename(src, dest)
for path, col in ((sys.argv[2], 2), (sys.argv[3], 0), (sys.argv[4], 1)):
if not os.path.getsize(path):
continue
rows = [line.rstrip("\n").split("\t") for line in open(path)]
with open(path, "w") as fh:
for f in rows:
f[col] = moves.get(f[col], f[col])
fh.write("\t".join(f) + "\n")
PY
plan
[ -z "$(awk -F '\t' '$1 == "move" {print $2}' "$OUT")" ]
}
+212
View File
@@ -0,0 +1,212 @@
#!/usr/bin/env bats
# Seam: scripts/media-verify.py — the replay of a relayout journal against a
# media server's OWN database, which is the documented proof that a rename
# landed (a scan answers 204 whether or not it finds anything).
#
# Every test states what the database holds and asserts what the command prints
# and returns; nothing here reaches into the script's internals. The fixtures are
# minimal sqlite files with the columns the two servers really use, so the seam
# fails if the tool ever reads the wrong table, forgets Plex's `deleted_at`
# filter, or forgets that Jellyfin stores the container-side path (/data/…) while
# Plex stores the host path verbatim.
load helpers
VERIFY="$REPO_ROOT/scripts/media-verify.py"
# jf_mk <db> — the Jellyfin shape: one BaseItems table, Path is the column.
jf_mk() {
python3 - "$1" <<'PY'
import sqlite3, sys
con = sqlite3.connect(sys.argv[1])
con.execute("create table BaseItems (Id INTEGER PRIMARY KEY, Path TEXT, Type TEXT,"
" ParentId INTEGER, Name TEXT)")
con.commit()
PY
}
# jf_row <db> <path> — one item the server believes it has.
jf_row() {
python3 - "$1" "$2" <<'PY'
import sqlite3, sys
con = sqlite3.connect(sys.argv[1])
con.execute("insert into BaseItems (Path, Type) values (?, 'x')", (sys.argv[2],))
con.commit()
PY
}
# px_mk <db> — the Plex shape: media_parts joined up to metadata_items.
px_mk() {
python3 - "$1" <<'PY'
import sqlite3, sys
con = sqlite3.connect(sys.argv[1])
con.execute("create table metadata_items (id INTEGER PRIMARY KEY, title TEXT,"
" deleted_at TEXT)")
con.execute("create table media_items (id INTEGER PRIMARY KEY, metadata_item_id"
" INTEGER, container TEXT)")
con.execute("create table media_parts (id INTEGER PRIMARY KEY, media_item_id"
" INTEGER, file TEXT, deleted_at TEXT)")
con.commit()
PY
}
# px_row <db> <path> [DELETED_AT] — one part; a non-empty DELETED_AT is a row a
# deleted scan left behind, which must never count as proof.
px_row() {
python3 - "$1" "$2" "${3:-}" <<'PY'
import sqlite3, sys
db, path, gone = sys.argv[1], sys.argv[2], sys.argv[3] or None
con = sqlite3.connect(db)
cur = con.execute("insert into metadata_items (title, deleted_at) values ('t', ?)",
(gone,))
mi = cur.lastrowid
cur = con.execute("insert into media_items (metadata_item_id, container)"
" values (?, 'mkv')", (mi,))
con.execute("insert into media_parts (media_item_id, file, deleted_at)"
" values (?, ?, NULL)", (cur.lastrowid, path))
con.commit()
PY
}
# move_row <src> <dst> — one move in the journal.
move_row() {
printf '{"op": "move", "src": "%s", "dst": "%s"}\n' "$1" "$2" >> "$JOURNAL"
}
setup() {
SHARE="$BATS_TEST_TMPDIR/mnt/entropy"
JFDB="$BATS_TEST_TMPDIR/jellyfin.db"
PXDB="$BATS_TEST_TMPDIR/plex.db"
JOURNAL="$BATS_TEST_TMPDIR/journal.jsonl"
: > "$JOURNAL"
jf_mk "$JFDB"
px_mk "$PXDB"
}
@test "every move that the database holds at its destination is reported landed" {
move_row "$SHARE/Movies/Old Name (1999).mkv" "$SHARE/Movies/Old Name (1999)/Old Name (1999).mkv"
move_row "$SHARE/Movies/Two.mkv" "$SHARE/Movies/Two/Two.mkv"
# Jellyfin stores the container-side path: /data stands for the share.
jf_row "$JFDB" "/data/Movies/Old Name (1999)/Old Name (1999).mkv"
jf_row "$JFDB" "/data/Movies/Two/Two.mkv"
run python3 "$VERIFY" --journal "$JOURNAL" --db-kind jellyfin --db "$JFDB" \
--share "$SHARE" --container /data
[ "$status" -eq 0 ]
[ "${lines[0]}" = "jellyfin: 2 moves checked" ]
[ "${lines[1]}" = "landed: 2" ]
[ "${lines[2]}" = "still at src: 0" ]
[ "${lines[3]}" = "missing from the DB: 0" ]
}
@test "a database that has not rescanned yet is refused, naming the stale rows" {
move_row "$SHARE/Movies/Old.mkv" "$SHARE/Movies/Old/Old.mkv"
jf_row "$JFDB" "/data/Movies/Old.mkv"
run python3 "$VERIFY" --journal "$JOURNAL" --db-kind jellyfin --db "$JFDB" \
--share "$SHARE" --container /data
[ "$status" -eq 1 ]
[ "${lines[1]}" = "landed: 0" ]
[ "${lines[2]}" = "still at src: 1" ]
[[ "$output" == *"$SHARE/Movies/Old.mkv"* ]]
}
@test "a moved file no row of the database names is reported missing" {
move_row "$SHARE/Movies/Old.mkv" "$SHARE/Movies/Old/Old.mkv"
run python3 "$VERIFY" --journal "$JOURNAL" --db-kind jellyfin --db "$JFDB" \
--share "$SHARE" --container /data
[ "$status" -eq 1 ]
[ "${lines[3]}" = "missing from the DB: 1" ]
[[ "$output" == *"$SHARE/Movies/Old/Old.mkv"* ]]
}
@test "Plex stores the host path verbatim, so a row at the destination lands" {
move_row "$SHARE/TVShows/Show/Season 01/Show S01E01.mkv" \
"$SHARE/TVShows/Show (2002)/Season 01/Show (2002) S01E01.mkv"
px_row "$PXDB" "$SHARE/TVShows/Show (2002)/Season 01/Show (2002) S01E01.mkv"
run python3 "$VERIFY" --journal "$JOURNAL" --db-kind plex --db "$PXDB" \
--share "$SHARE"
[ "$status" -eq 0 ]
[ "${lines[0]}" = "plex: 1 moves checked" ]
[ "${lines[1]}" = "landed: 1" ]
}
@test "a Plex row a deleted scan left behind is not proof a move landed" {
move_row "$SHARE/TVShows/Old S01E01.mkv" "$SHARE/TVShows/Show/Old S01E01.mkv"
px_row "$PXDB" "$SHARE/TVShows/Show/Old S01E01.mkv" "2026-10-30 00:00:00"
run python3 "$VERIFY" --journal "$JOURNAL" --db-kind plex --db "$PXDB" \
--share "$SHARE"
[ "$status" -eq 1 ]
[ "${lines[1]}" = "landed: 0" ]
[ "${lines[3]}" = "missing from the DB: 1" ]
}
@test "a database without the columns that kind needs is an error, not a zero" {
move_row "$SHARE/Movies/Old.mkv" "$SHARE/Movies/Old/Old.mkv"
run python3 "$VERIFY" --journal "$JOURNAL" --db-kind jellyfin --db "$PXDB" \
--share "$SHARE" --container /data
[ "$status" -eq 2 ]
[[ "$output" == *"$PXDB"* ]]
[[ "$output" == *"BaseItems"* ]]
}
@test "only move rows are counted, so folder operations cannot inflate the tally" {
move_row "$SHARE/Movies/Old.mkv" "$SHARE/Movies/Old/Old.mkv"
jf_row "$JFDB" "/data/Movies/Old/Old.mkv"
printf '{"op": "mkdir", "dst": "%s/Movies/Old"}\n' "$SHARE" >> "$JOURNAL"
printf '{"op": "rmdir", "path": "%s/Movies/Empty"}\n' "$SHARE" >> "$JOURNAL"
run python3 "$VERIFY" --journal "$JOURNAL" --db-kind jellyfin --db "$JFDB" \
--share "$SHARE" --container /data
[ "$status" -eq 0 ]
[ "${lines[0]}" = "jellyfin: 1 moves checked" ]
}
@test "a journal with no moves is clean, not a failure" {
printf '{"op": "mkdir", "dst": "%s/Movies/Old"}\n' "$SHARE" >> "$JOURNAL"
run python3 "$VERIFY" --journal "$JOURNAL" --db-kind jellyfin --db "$JFDB" \
--share "$SHARE" --container /data
[ "$status" -eq 0 ]
[ "${lines[1]}" = "landed: 0" ]
}
@test "the offender list stops at --show so a stale library stays readable" {
for n in 1 2 3 4; do
move_row "$SHARE/Movies/F$n.mkv" "$SHARE/Movies/F$n/F$n.mkv"
done
run python3 "$VERIFY" --journal "$JOURNAL" --db-kind jellyfin --db "$JFDB" \
--share "$SHARE" --container /data --show 2
[ "$status" -eq 1 ]
[ "${lines[3]}" = "missing from the DB: 4" ]
[[ "$output" == *"$SHARE/Movies/F1/F1.mkv"* ]]
[[ "$output" != *"$SHARE/Movies/F4/F4.mkv"* ]]
}
@test "the missing rows are counted by extension, so a sidecar is not mistaken for a lost film" {
move_row "$SHARE/Music/Album/Mandatory Fun.log" "$SHARE/Music/Weird Al/Mandatory Fun (2014)/Mandatory Fun.log"
move_row "$SHARE/Music/Album/Film.mkv" "$SHARE/Music/Weird Al/Mandatory Fun (2014)/Film.mkv"
run python3 "$VERIFY" --journal "$JOURNAL" --db-kind jellyfin --db "$JFDB" \
--share "$SHARE" --container /data
[ "$status" -eq 1 ]
[ "${lines[3]}" = "missing from the DB: 2" ]
[ "${lines[4]}" = "missing by extension: .log 1, .mkv 1" ]
}
@test "--ignore-ext keeps a file the server never indexes out of the verdict" {
move_row "$SHARE/Music/Album/Song.flac" "$SHARE/Music/Weird Al/Mandatory Fun (2014)/Song.flac"
jf_row "$JFDB" "/data/Music/Weird Al/Mandatory Fun (2014)/Song.flac"
move_row "$SHARE/Music/Album/Mandatory Fun.cue" "$SHARE/Music/Weird Al/Mandatory Fun (2014)/Mandatory Fun.cue"
run python3 "$VERIFY" --journal "$JOURNAL" --db-kind jellyfin --db "$JFDB" \
--share "$SHARE" --container /data --ignore-ext log,cue
[ "$status" -eq 0 ]
[ "${lines[1]}" = "landed: 1" ]
[ "${lines[3]}" = "missing from the DB: 0" ]
[ "${lines[4]}" = "not indexed (extension ignored): 1" ]
}