Compare commits
25
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
536949c18d | ||
|
|
337afac91e | ||
|
|
c0c77590b1 | ||
|
|
441b908937 | ||
|
|
949d3cb6f4 | ||
|
|
3ce8d62e70 | ||
|
|
38737ab84e | ||
|
|
4437519430 | ||
|
|
3b29e173c2 | ||
|
|
9706f5d2de | ||
|
|
206dc3baf1 | ||
|
|
f5a3badb56 | ||
|
|
3e4b4f24f7 | ||
|
|
86f5903e74 | ||
|
|
bd4ee78811 | ||
|
|
b7b39cdc92 | ||
|
|
09532e5216 | ||
|
|
e1786e7cf2 | ||
|
|
409a8d8b0c | ||
|
|
e83162a2ac | ||
|
|
5cd9a0c4c2 | ||
|
|
a803aa046a | ||
|
|
73f77a13c3 | ||
|
|
dd76fcf5d4 | ||
|
|
e1760e3316 |
No files matched your search
@@ -808,6 +808,7 @@ per-attempt page and no per-swap ledger anywhere else.
|
||||
| Dankclaw kernel posture (`iommu=pt` since 2026-10-18) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
||||
| Deployment flow | `serverAppConfigs-docs/Architecture/Deployment Flow.md` |
|
||||
| Movie-library relayout (Jellyfin format, Dankserver) | `serverAppConfigs-docs/Dankserver/Media Relayout.md` |
|
||||
| SSD-backed metadata view (Jellyfin writes, Plex reads, Dankserver) | `serverAppConfigs-docs/Dankserver/Media Metadata View.md` |
|
||||
|
||||
There is no page for a retired endpoint and no change diary. A payload swap
|
||||
appends one row to the live endpoint's **Eras** table; when the endpoint itself
|
||||
@@ -850,5 +851,6 @@ history.
|
||||
| Benchmarking (only when ordered) | **Stop and mask the service first**, bench, unmask, restart, verify health. |
|
||||
| Deleting a file | **Move it to trash** — `.AGENTTRASH/` (in-repo) or `~/.AGENTTRASH/YYYYMMDD-topic`. Never `rm`. |
|
||||
| Relayout the movie library | **Census → plan → read the exceptions → `scripts/media-apply.py`** — renames only, journal before every move; never hand-rename, never copy. |
|
||||
| Making media metadata visible to both media servers | **Bind `/var/lib/media-meta/view`** — the overlay `scripts/media-view.sh` installs (array below, NVMe above). Never write the array; never delete through the view. |
|
||||
| Tests exist | **Run them** — verify tests pass after every change. Do not commit failing tests. |
|
||||
| Real secrets | **Never commit** — host `.env` files; `.env.example` holds placeholders only. |
|
||||
+6
-28
@@ -94,34 +94,12 @@ services:
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Jellyfin — media server (movies, TV, music)
|
||||
# ═══════════════════════════════════════════════
|
||||
jellyfin:
|
||||
image: jellyfin/jellyfin:latest
|
||||
container_name: jellyfin
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "8096:8096/tcp"
|
||||
- "8920:8920/tcp"
|
||||
- "7359:7359/udp"
|
||||
- "1900:1900/udp"
|
||||
environment:
|
||||
- JELLYFIN_PublishedServerUrl=http://192.168.0.235:8096
|
||||
- DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true
|
||||
volumes:
|
||||
- /opt/jellyfin/config:/config
|
||||
- /opt/jellyfin/cache:/cache
|
||||
- /mnt/entropy:/data:ro
|
||||
devices:
|
||||
- /dev/dri:/dev/dri
|
||||
group_add:
|
||||
- "992"
|
||||
healthcheck:
|
||||
test: ["CMD", "curl", "-f", "http://localhost:8096/health"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
# Jellyfin left this file when it became a Quadlet: the container is owned by
|
||||
# dankserver/jellyfin/quadlet/jellyfin.container, its data roots are under
|
||||
# /home/dank/Containers/jellyfin, and its library arrives through the SSD
|
||||
# overlay at /var/lib/media-meta/view (scripts/media-view.sh). The block that
|
||||
# was here named /opt/jellyfin/* and a read-only array bind — a second owner for
|
||||
# the container name `jellyfin`, wrong on every path. One owner per container.
|
||||
|
||||
# ═══════════════════════════════════════════════
|
||||
# Calibre-Web — e-book web frontend
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
# /etc/systemd/system/var-lib-media\x2dmeta-view.mount
|
||||
# (repo copy: dankserver/host/systemd/media-meta-view.mount — systemd names a
|
||||
# mount unit after its ESCAPED path, so the host file carries the \x2d spelling)
|
||||
#
|
||||
# Installed by `scripts/media-view.sh install` — the repo copy is the source of
|
||||
# truth (fleet law: never edit host state by hand).
|
||||
#
|
||||
# An OverlayFS merge that shows the media library as one tree while splitting
|
||||
# where its bytes live: lowerdir is the RAID6 array (the source of truth, never
|
||||
# written), upperdir is a directory on the NVMe that receives every write. The
|
||||
# two media servers bind the result at /var/lib/media-meta/view, so Jellyfin's
|
||||
# sidecars and artwork are SSD bytes — which is what makes them fast to browse
|
||||
# and what keeps Plex's NFO agent off the spinners — while the media itself stays
|
||||
# on the array untouched.
|
||||
#
|
||||
# RequiresMountsFor is load-bearing: without the array mounted, lowerdir is an
|
||||
# empty directory and the view would present an empty library as truth.
|
||||
|
||||
[Unit]
|
||||
Description=SSD-backed overlay view of the media library (array + NVMe metadata layer)
|
||||
RequiresMountsFor=/mnt/entropy
|
||||
|
||||
[Mount]
|
||||
What=overlay
|
||||
Where=/var/lib/media-meta/view
|
||||
Type=overlay
|
||||
Options=lowerdir=/mnt/entropy,upperdir=/var/lib/media-meta/upper,workdir=/var/lib/media-meta/work
|
||||
|
||||
[Install]
|
||||
WantedBy=local-fs.target
|
||||
@@ -4,8 +4,9 @@
|
||||
# Unit name: jellyfin.service (podman 5.x drops the container- prefix)
|
||||
#
|
||||
# Data roots migrated from /opt/jellyfin to /home/dank/Containers/jellyfin at
|
||||
# cutover. All of /mnt/entropy is mounted read-only at /data: nothing automated
|
||||
# anywhere writes to the entropy volume. Every
|
||||
# cutover. The library is seen through /var/lib/media-meta/view, the SSD-backed
|
||||
# overlay view (scripts/media-view.sh): the array stays read-only and untouched,
|
||||
# while the sidecars this server writes land on the NVMe. Every
|
||||
# creation-time environment variable of the compose-era container is kept as
|
||||
# an Environment= line (zero drift); none of them are secrets, so this unit
|
||||
# needs no EnvironmentFile.
|
||||
@@ -41,7 +42,12 @@ PublishPort=7359:7359/udp
|
||||
PublishPort=1900:1900/udp
|
||||
Volume=/home/dank/Containers/jellyfin/config:/config
|
||||
Volume=/home/dank/Containers/jellyfin/cache:/cache
|
||||
Volume=/mnt/entropy:/data:ro
|
||||
# The view, writable: Jellyfin is the metadata writer (movie.nfo / tvshow.nfo /
|
||||
# episode.nfo + artwork) and those bytes belong on the SSD. The condition is the
|
||||
# fail-closed guard — starting against a plain, unmounted view directory would let
|
||||
# a scan conclude the library is empty.
|
||||
Volume=/var/lib/media-meta/view:/data
|
||||
UnitConfigItem=ConditionPathIsMountPoint=/var/lib/media-meta/view
|
||||
AddDevice=/dev/dri
|
||||
GroupAdd=keep-groups
|
||||
HealthCmd=curl -f http://localhost:8096/health
|
||||
|
||||
@@ -11,7 +11,12 @@ Environment=PUID=0
|
||||
Environment=PGID=0
|
||||
Environment=TZ=America/Los_Angeles
|
||||
Volume=/var/lib/plexmediaserver:/config
|
||||
Volume=/mnt/entropy:/mnt/entropy:ro
|
||||
# The container path stays /mnt/entropy on purpose: every path Plex recorded in
|
||||
# its own database is written against it, so swapping the HOST source of this bind
|
||||
# to the overlay view re-points Plex at the sidecars without touching its DB.
|
||||
# Read-only: Plex consumes the NFO files Jellyfin writes and writes no metadata.
|
||||
Volume=/var/lib/media-meta/view:/mnt/entropy:ro
|
||||
UnitConfigItem=ConditionPathIsMountPoint=/var/lib/media-meta/view
|
||||
PublishPort=32400:32400
|
||||
|
||||
[Service]
|
||||
|
||||
+16
-1
@@ -155,13 +155,14 @@ DANKCLAW_COMPOSE_SERVICES = set()
|
||||
# Quadlet unit now — the last compose services were the hermes trio, whose units
|
||||
# have owned their containers since 2026-09-12 and whose compose blocks were
|
||||
# dead weight that could only create a duplicate container.
|
||||
# jellyfin left the Dankserver file on 2026-10-30 for the same reason: its Quadlet
|
||||
# has owned the container since the cutover, so a compose edit must not restart it.
|
||||
|
||||
DANKSERVER_COMPOSE_SERVICES = {
|
||||
"mariadb",
|
||||
"redis",
|
||||
"romm",
|
||||
"copyparty",
|
||||
"jellyfin",
|
||||
"calibre-web",
|
||||
}
|
||||
COMPOSE_NAME_MAP = {"mariadb": "romm-mariadb", "redis": "romm-redis"}
|
||||
@@ -421,6 +422,11 @@ def services_for_path(path: str, target: str) -> set[str]:
|
||||
"tautulli": {"tautulli"},
|
||||
}
|
||||
_add(services, *quadlet_services.get(parts[1], set()))
|
||||
if len(parts) >= 2 and parts[1] == "host":
|
||||
# dankserver/host/systemd/media-meta-view.mount is the overlay view the
|
||||
# media servers bind (scripts/media-view.sh installs it on the host),
|
||||
# so editing it restarts jellyfin and plex and nothing else.
|
||||
_add(services, "jellyfin", "plex")
|
||||
if len(parts) >= 2 and parts[1] == "observability":
|
||||
# Dankserver's node-exporter became a Quadlet unit on 2026-09-16
|
||||
# (serverAppConfigs-docs/Infrastructure/Observability.md): its
|
||||
@@ -466,6 +472,15 @@ def services_for_path(path: str, target: str) -> set[str]:
|
||||
# that service (target_services then keeps it only on the host being
|
||||
# planned). Editing the collector restarts the running exporter.
|
||||
_add(services, "gpu-exporter")
|
||||
if name == "media-view.sh":
|
||||
# The installer is what creates, mounts and reaps the view both media
|
||||
# servers read, so a change to it is a restart of that pair.
|
||||
_add(services, "jellyfin", "plex")
|
||||
if name == "jellyfin-providers.sh":
|
||||
# The deployer runs this before jellyfin restarts, so a change to
|
||||
# the installer is a change to the provider state Jellyfin loads at
|
||||
# start-up: it selects jellyfin and nothing else.
|
||||
_add(services, "jellyfin")
|
||||
if name in {"frank-zealot.env", "frank-zealot.service", "serve-frank-zealot.sh"}:
|
||||
# llama.cpp frank-zealot (2026-09-16) keeps its unit, env and serve
|
||||
# files here; each one selects frank-zealot only. The ik_llama.cpp
|
||||
|
||||
@@ -1202,7 +1202,42 @@ deploy_romm_stack() {
|
||||
systemctl --user restart "$service.service"
|
||||
}
|
||||
|
||||
ensure_media_view() {
|
||||
# Both media servers read their library through the SSD-backed overlay view
|
||||
# (scripts/media-view.sh, dankserver/host/systemd/media-meta-view.mount): the
|
||||
# array below stays read-only and every sidecar Jellyfin writes lands on the
|
||||
# NVMe.
|
||||
# Raise it BEFORE either container starts — a start against a bare view
|
||||
# directory would let a scan clear the library database. The units also carry
|
||||
# ConditionPathIsMountPoint, so a missing view skips the start rather than
|
||||
# damaging anything. A relayout must be followed by `media-view.sh reap`
|
||||
# (see the Media Relayout page), which this is not.
|
||||
run_privileged bash scripts/media-view.sh install
|
||||
bash scripts/media-view.sh verify
|
||||
}
|
||||
|
||||
ensure_provider_keys() {
|
||||
# Jellyfin's metadata-provider keys live in the container-owned plugin
|
||||
# configuration under the data root, never in git; scripts/jellyfin-providers.sh
|
||||
# is what places them, reading the per-service host env file (the fleet
|
||||
# convention: grafana, caddy, pihole, copyparty, romm, seerr) that holds
|
||||
# TMDB_API_KEY= and TVDB_API_KEY=, mode 600.
|
||||
# Missing keys mean a keyless provider — a degraded library, not a broken
|
||||
# deploy — so this warns rather than failing the restart. But when the keys
|
||||
# ARE placed it must happen before the restart: the plugin and its keys are
|
||||
# read at start-up, so a restart first would boot the old state.
|
||||
local keys=/home/dank/Containers/jellyfin/provider-keys
|
||||
if [ -f "$keys" ]; then
|
||||
bash scripts/jellyfin-providers.sh install
|
||||
else
|
||||
echo "WARN: $keys is absent - provider keys NOT placed"
|
||||
echo "WARN: create it (TMDB_API_KEY=, TVDB_API_KEY=, chmod 600) and re-deploy jellyfin"
|
||||
fi
|
||||
}
|
||||
|
||||
deploy_jellyfin() {
|
||||
ensure_media_view
|
||||
ensure_provider_keys
|
||||
cp "dankserver/jellyfin/quadlet/jellyfin.container" "$quadlet_unit_dir/"
|
||||
systemctl --user daemon-reload
|
||||
if ! systemctl --user is-active -q jellyfin.service; then
|
||||
@@ -1290,6 +1325,7 @@ deploy_copyparty() {
|
||||
}
|
||||
|
||||
deploy_plex() {
|
||||
ensure_media_view
|
||||
cp "dankserver/plex/quadlet/plex.container" "$quadlet_unit_dir/"
|
||||
systemctl --user daemon-reload
|
||||
if ! systemctl --user is-active -q plex.service; then
|
||||
|
||||
Executable
+391
@@ -0,0 +1,391 @@
|
||||
#!/usr/bin/env bash
|
||||
# Metadata-provider keys for Jellyfin on Dankserver, placed from the service's
|
||||
# host env file with the plugin artifact pinned by content hash.
|
||||
#
|
||||
# Jellyfin 12 ships TMDb, OMDb, StudioImages and MusicBrainz as CORE plugins:
|
||||
# they have no DLL, only a configuration file under the data root, and that
|
||||
# file is created by the running container. So the fleet's provider state lives
|
||||
# in two places the repo does not track —
|
||||
#
|
||||
# plugins/configurations/Jellyfin.Plugin.Tmdb.xml <TmdbApiKey /> (empty = keyless)
|
||||
# plugins/configurations/Jellyfin.Plugin.Tvdb.xml <ApiKey /> (needs the plugin)
|
||||
#
|
||||
# and both are owned by the container user. This tool is the only thing that
|
||||
# writes them: it is declarative, idempotent, and it never prints a key, so it
|
||||
# is safe to run while someone watches.
|
||||
#
|
||||
# Keys never enter git and never enter argv (a value in argv is world-readable
|
||||
# through /proc/<pid>/cmdline). They live in the per-service host env file that
|
||||
# is the fleet's convention for container credentials (grafana, caddy, pihole,
|
||||
# copyparty, romm, seerr), mode 600:
|
||||
#
|
||||
# /home/dank/Containers/jellyfin/provider-keys
|
||||
# TMDB_API_KEY=<key>
|
||||
# TVDB_API_KEY=<key>
|
||||
# TVDB_PIN=<pin> # optional: only a subscriber-shaped TVDB key
|
||||
# # needs one; a licensed key does not
|
||||
#
|
||||
# TVDB itself is not core in 12 (the metadata provider was cut in 10.9), so it
|
||||
# is installed from the official catalog, pinned three ways: sha256 of the
|
||||
# artifact, the plugin's own guid, and its version. A matching sha proves only
|
||||
# that the bytes are the bytes fetched — the guid check is what says it is the
|
||||
# official plugin, the version check is what says it is the one this file was
|
||||
# reviewed against.
|
||||
#
|
||||
# Subcommands
|
||||
# plan print the resolved shape and every pending change, touch nothing
|
||||
# install fetch+verify the plugin, then write the keys (backing up first)
|
||||
# verify prove the plugin and both keys are actually in place
|
||||
#
|
||||
# Nothing here restarts Jellyfin: the deployer owns that
|
||||
# (`scripts/deploy.sh dankserver --service jellyfin`), and the plugin is read at
|
||||
# start-up, so a key written while the server runs is not live until then.
|
||||
set -euo pipefail
|
||||
|
||||
# Jellyfin's data root on Dankserver (the rootless quadlet's config volume).
|
||||
JF_ROOT="${JF_ROOT:-/home/dank/Containers/jellyfin/config}"
|
||||
PROV_KEYS="${PROV_KEYS:-/home/dank/Containers/jellyfin/provider-keys}"
|
||||
TRASH_DIR="${TRASH_DIR:-$HOME/.AGENTTRASH/$(date +%Y%m%d)-jellyfin-providers}"
|
||||
|
||||
PLUGINS="$JF_ROOT/plugins"
|
||||
CONF="$PLUGINS/configurations"
|
||||
TMDB_XML="$CONF/Jellyfin.Plugin.Tmdb.xml"
|
||||
TVDB_XML="$CONF/Jellyfin.Plugin.Tvdb.xml"
|
||||
|
||||
# repo.jellyfin.org catalog entry for TheTVDB (owner: jellyfin), version 24.0.0.0,
|
||||
# targetAbi 12.0.0.0 — published 2026-09-09, verified against Jellyfin 12.4.0.
|
||||
TVDB_PLUGIN_URL="${TVDB_PLUGIN_URL:-https://repo.jellyfin.org/files/plugin/thetvdb/thetvdb_24.0.0.0.zip}"
|
||||
TVDB_PLUGIN_SHA256="${TVDB_PLUGIN_SHA256:-6302fba3857bdde8d186dd099fac09ca43247041f531a528a2c788add70a5319}"
|
||||
TVDB_PLUGIN_GUID="${TVDB_PLUGIN_GUID:-a677c0da-fac5-4cde-941a-7134223f14c8}"
|
||||
TVDB_PLUGIN_VERSION="${TVDB_PLUGIN_VERSION:-24.0.0.0}"
|
||||
PLUGIN_DIR="thetvdb_${TVDB_PLUGIN_VERSION}"
|
||||
TVDB_DLL="Jellyfin.Plugin.Tvdb.dll"
|
||||
|
||||
die() {
|
||||
printf 'jellyfin-providers: %s\n' "$*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
usage() {
|
||||
printf 'usage: %s plan|install|verify\n' "$0" >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
# check_keys — refuse anything that is not a readable private file. An open
|
||||
# mode is an error, not a warning: a provider key readable by every user on the
|
||||
# box is the failure this check exists to catch.
|
||||
check_keys() {
|
||||
[ -f "$PROV_KEYS" ] ||
|
||||
die "$PROV_KEYS not found (create it with TMDB_API_KEY= and TVDB_API_KEY=, then chmod 600)"
|
||||
local mode
|
||||
mode="$(stat -c '%a' "$PROV_KEYS")"
|
||||
case "$mode" in
|
||||
600 | 400) ;;
|
||||
*) die "$PROV_KEYS is mode $mode; provider keys must be 600" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
# keys_value <VAR> — the value of one variable in the env file, or nothing.
|
||||
keys_value() {
|
||||
awk -v k="$1" -F= '$1 == k { sub(/^[^=]*=/, ""); print; exit }' "$PROV_KEYS"
|
||||
}
|
||||
|
||||
# keys_len <VAR> — the length, and a refusal naming the variable when absent.
|
||||
# The value itself is never printed.
|
||||
keys_len() {
|
||||
local v
|
||||
v="$(keys_value "$1")"
|
||||
[ -n "$v" ] || die "$PROV_KEYS has no $1= line"
|
||||
printf '%s' "${#v}"
|
||||
}
|
||||
|
||||
# xml_value <file> <element> — the element's text, empty when it is absent or
|
||||
# self-closing.
|
||||
xml_value() {
|
||||
python3 - "$1" "$2" <<'PYEOF'
|
||||
import re, sys
|
||||
src = open(sys.argv[1], encoding="utf-8").read()
|
||||
m = re.search(r"<%s(?:\s*/>|>(.*?)</%s>)" % (sys.argv[2], sys.argv[2]), src, re.S)
|
||||
print(m.group(1).strip() if (m and m.group(1)) else "")
|
||||
PYEOF
|
||||
}
|
||||
|
||||
# xml_secret <file> <element> <var> — set <element> to that variable from the
|
||||
# env file. The python process reads the env file itself, so the value never
|
||||
# appears in argv.
|
||||
xml_secret() {
|
||||
python3 - "$1" "$2" "$3" "$PROV_KEYS" <<'PYEOF'
|
||||
import os, re, sys, tempfile
|
||||
from xml.sax.saxutils import escape
|
||||
|
||||
xml_path, elem, var, keys_path = sys.argv[1:5]
|
||||
val = ""
|
||||
for line in open(keys_path, encoding="utf-8"):
|
||||
key, sep, rest = line.partition("=")
|
||||
if sep and key.strip() == var:
|
||||
val = rest.strip()
|
||||
break
|
||||
if not val:
|
||||
raise SystemExit(f"{keys_path} has no {var}= line")
|
||||
|
||||
src = open(xml_path, encoding="utf-8").read()
|
||||
pat = re.compile(r"([ \t]*)<%s(?:\s*/>|>.*?</%s>)" % (elem, elem), re.S)
|
||||
text = escape(val)
|
||||
if pat.search(src):
|
||||
out = pat.sub(lambda m: f"{m.group(1)}<{elem}>{text}</{elem}>", src, count=1)
|
||||
else:
|
||||
out = src.replace("</Configuration>", f" <{elem}>{text}</{elem}>\n</Configuration>", 1)
|
||||
if out == src:
|
||||
raise SystemExit(f"{xml_path}: no place to write <{elem}>")
|
||||
|
||||
d, base = os.path.split(xml_path)
|
||||
fd, tmp = tempfile.mkstemp(prefix=base + ".", dir=d)
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as fh:
|
||||
fh.write(out)
|
||||
os.chmod(tmp, os.stat(xml_path).st_mode & 0o777)
|
||||
os.replace(tmp, xml_path)
|
||||
PYEOF
|
||||
}
|
||||
|
||||
# plugin_meta <zip> — "guid version" out of the artifact's own meta.json.
|
||||
plugin_meta() {
|
||||
python3 - "$1" <<'PYEOF'
|
||||
import json, sys, zipfile
|
||||
meta = json.loads(zipfile.ZipFile(sys.argv[1]).read("meta.json").decode("utf-8"))
|
||||
print(meta.get("guid", ""), meta.get("version", ""))
|
||||
PYEOF
|
||||
}
|
||||
|
||||
check_root() {
|
||||
# Never fabricate a data root: creating plugins/ beside a typo'd path would
|
||||
# install a plugin Jellyfin never loads and hide the operator's mistake.
|
||||
[ -d "$CONF" ] || die "no Jellyfin configuration directory at $CONF (is JF_ROOT right?)"
|
||||
}
|
||||
|
||||
check_keys_report() {
|
||||
check_keys
|
||||
# A refusal inside a command substitution used as a printf argument would
|
||||
# be swallowed, so the guard runs as its own statement.
|
||||
keys_len TMDB_API_KEY >/dev/null
|
||||
keys_len TVDB_API_KEY >/dev/null
|
||||
printf '= keys read from %s (mode %s)\n' "$PROV_KEYS" "$(stat -c '%a' "$PROV_KEYS")"
|
||||
}
|
||||
|
||||
# The plugin's state, as one line in the shared vocabulary.
|
||||
plugin_state() {
|
||||
if [ -d "$PLUGINS/$PLUGIN_DIR" ]; then
|
||||
printf '= %s already installed\n' "$PLUGIN_DIR"
|
||||
return 0
|
||||
fi
|
||||
local other
|
||||
other="$(find "$PLUGINS" -maxdepth 1 -type d -name 'thetvdb_*' ! -name "$PLUGIN_DIR" -printf '%f\n' 2>/dev/null | sort | tr '\n' ' ')"
|
||||
if [ -n "$other" ]; then
|
||||
printf '+ install %s (a different version is present: %s)\n' "$PLUGIN_DIR" "$other"
|
||||
return 0
|
||||
fi
|
||||
printf '+ install %s from %s\n' "$PLUGIN_DIR" "${TVDB_PLUGIN_URL#*://}"
|
||||
}
|
||||
|
||||
tmdb_state() {
|
||||
if [ ! -f "$TMDB_XML" ]; then
|
||||
printf '! %s is missing — save the TMDb plugin settings once in the UI, then re-run\n' "$TMDB_XML" >&2
|
||||
return 1
|
||||
fi
|
||||
local now want
|
||||
now="$(xml_value "$TMDB_XML" TmdbApiKey)"
|
||||
want="$(keys_value TMDB_API_KEY)"
|
||||
if [ -n "$now" ] && [ "$now" = "$want" ]; then
|
||||
printf '= TmdbApiKey already set (%s chars)\n' "${#now}"
|
||||
elif [ -n "$now" ]; then
|
||||
printf '+ replace the TmdbApiKey in Jellyfin.Plugin.Tmdb.xml\n'
|
||||
else
|
||||
printf '+ set TmdbApiKey in Jellyfin.Plugin.Tmdb.xml\n'
|
||||
fi
|
||||
}
|
||||
|
||||
tvdb_state() {
|
||||
local now
|
||||
if [ -f "$TVDB_XML" ]; then
|
||||
now="$(xml_value "$TVDB_XML" ApiKey)"
|
||||
if [ -n "$now" ]; then
|
||||
printf '= Jellyfin.Plugin.Tvdb.xml already has an ApiKey (kept)\n'
|
||||
else
|
||||
printf '+ fill the empty ApiKey in Jellyfin.Plugin.Tvdb.xml\n'
|
||||
fi
|
||||
else
|
||||
printf '+ create Jellyfin.Plugin.Tvdb.xml with ApiKey\n'
|
||||
fi
|
||||
}
|
||||
|
||||
write_plugin() {
|
||||
local tmp guid version other
|
||||
other="$(find "$PLUGINS" -maxdepth 1 -type d -name 'thetvdb_*' ! -name "$PLUGIN_DIR" -printf '%f\n' 2>/dev/null | sort | tr '\n' ' ')"
|
||||
[ -z "$other" ] ||
|
||||
die "another TheTVDB version is already installed ($other); move it to the trash first — two copies of one plugin means two loads"
|
||||
|
||||
tmp="$(mktemp "${TMPDIR:-/tmp}/thetvdb.XXXXXX.zip")"
|
||||
# The pins are checked before anything is unpacked, so a bad artifact never
|
||||
# reaches the plugins directory.
|
||||
curl -fsSL --max-time 300 -o "$tmp" "$TVDB_PLUGIN_URL" ||
|
||||
{ rm -f "$tmp"; die "could not fetch $TVDB_PLUGIN_URL"; }
|
||||
local got
|
||||
got="$(sha256sum "$tmp" | cut -d' ' -f1)"
|
||||
if [ "$got" != "$TVDB_PLUGIN_SHA256" ]; then
|
||||
rm -f "$tmp"
|
||||
die "artifact sha256 is $got, pinned $TVDB_PLUGIN_SHA256 — refusing to install"
|
||||
fi
|
||||
read -r guid version <<<"$(plugin_meta "$tmp")"
|
||||
if [ "$guid" != "$TVDB_PLUGIN_GUID" ]; then
|
||||
rm -f "$tmp"
|
||||
die "artifact guid is $guid, expected the official $TVDB_PLUGIN_GUID — refusing to install"
|
||||
fi
|
||||
if [ "$version" != "$TVDB_PLUGIN_VERSION" ]; then
|
||||
rm -f "$tmp"
|
||||
die "artifact version is $version, pinned $TVDB_PLUGIN_VERSION — refusing to install"
|
||||
fi
|
||||
mkdir -p "$PLUGINS/$PLUGIN_DIR"
|
||||
python3 -m zipfile -e "$tmp" "$PLUGINS/$PLUGIN_DIR/"
|
||||
rm -f "$tmp"
|
||||
[ -f "$PLUGINS/$PLUGIN_DIR/$TVDB_DLL" ] ||
|
||||
die "$PLUGIN_DIR has no $TVDB_DLL — the artifact does not match this plugin"
|
||||
printf '+ installed %s\n' "$PLUGIN_DIR"
|
||||
}
|
||||
|
||||
backup_to_trash() {
|
||||
local f="$1"
|
||||
mkdir -p "$TRASH_DIR"
|
||||
local dest="$TRASH_DIR/$(basename "$f").$(date +%H%M%S)"
|
||||
cp -p "$f" "$dest"
|
||||
printf '+ previous copy kept at %s\n' "$dest"
|
||||
}
|
||||
|
||||
cmd_write_keys() {
|
||||
local now want
|
||||
KEYS_CHANGED=0
|
||||
# TMDb: the running container owns this file and every setting in it. Only
|
||||
# the key element is touched, so country, language and adult settings
|
||||
# survive the edit.
|
||||
[ -f "$TMDB_XML" ] ||
|
||||
die "$TMDB_XML is missing — save the TMDb plugin settings once in the UI, then re-run"
|
||||
now="$(xml_value "$TMDB_XML" TmdbApiKey)"
|
||||
keys_len TMDB_API_KEY >/dev/null
|
||||
want="$(keys_value TMDB_API_KEY)"
|
||||
if [ "$now" = "$want" ]; then
|
||||
printf '= TmdbApiKey already set (%s chars)\n' "${#now}"
|
||||
else
|
||||
backup_to_trash "$TMDB_XML"
|
||||
xml_secret "$TMDB_XML" TmdbApiKey TMDB_API_KEY
|
||||
printf '+ set TmdbApiKey (%s chars)\n' "${#want}"
|
||||
KEYS_CHANGED=1
|
||||
fi
|
||||
|
||||
# TVDB: never overwrite a config the operator saved through the UI.
|
||||
if [ -f "$TVDB_XML" ]; then
|
||||
now="$(xml_value "$TVDB_XML" ApiKey)"
|
||||
if [ -n "$now" ]; then
|
||||
printf '= Jellyfin.Plugin.Tvdb.xml already has an ApiKey (kept)\n'
|
||||
else
|
||||
keys_len TVDB_API_KEY >/dev/null
|
||||
backup_to_trash "$TVDB_XML"
|
||||
xml_secret "$TVDB_XML" ApiKey TVDB_API_KEY
|
||||
if [ -n "$(keys_value TVDB_PIN)" ] && [ -z "$(xml_value "$TVDB_XML" Pin)" ]; then
|
||||
xml_secret "$TVDB_XML" Pin TVDB_PIN
|
||||
fi
|
||||
printf '+ filled the ApiKey in Jellyfin.Plugin.Tvdb.xml\n'
|
||||
KEYS_CHANGED=1
|
||||
fi
|
||||
else
|
||||
keys_len TVDB_API_KEY >/dev/null
|
||||
cat >"$TVDB_XML" <<XML
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<Configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
|
||||
<ApiKey />
|
||||
</Configuration>
|
||||
XML
|
||||
xml_secret "$TVDB_XML" ApiKey TVDB_API_KEY
|
||||
if [ -n "$(keys_value TVDB_PIN)" ]; then
|
||||
xml_secret "$TVDB_XML" Pin TVDB_PIN
|
||||
fi
|
||||
printf '+ created Jellyfin.Plugin.Tvdb.xml with ApiKey\n'
|
||||
KEYS_CHANGED=1
|
||||
fi
|
||||
}
|
||||
|
||||
cmd_plan() {
|
||||
check_root
|
||||
check_keys_report
|
||||
printf 'root %s\n' "$JF_ROOT"
|
||||
plugin_state
|
||||
tmdb_state
|
||||
tvdb_state
|
||||
}
|
||||
|
||||
cmd_install() {
|
||||
check_root
|
||||
check_keys_report
|
||||
KEYS_CHANGED=0
|
||||
if [ ! -d "$PLUGINS/$PLUGIN_DIR" ]; then
|
||||
write_plugin
|
||||
KEYS_CHANGED=1
|
||||
else
|
||||
printf '= %s already installed\n' "$PLUGIN_DIR"
|
||||
fi
|
||||
cmd_write_keys
|
||||
if [ "$KEYS_CHANGED" -eq 1 ]; then
|
||||
# Nothing is restarted here on purpose: this host may be playing media
|
||||
# right now, and both the plugin and its keys are read at start-up.
|
||||
printf 'next: scripts/deploy.sh dankserver --service jellyfin (the plugin and keys load at start-up)\n'
|
||||
else
|
||||
printf '= no change\n'
|
||||
fi
|
||||
}
|
||||
|
||||
cmd_verify() {
|
||||
check_root
|
||||
local bad=0
|
||||
if [ ! -d "$PLUGINS/$PLUGIN_DIR" ]; then
|
||||
printf 'MISSING plugin directory %s (thetvdb not installed)\n' "$PLUGIN_DIR" >&2
|
||||
bad=1
|
||||
else
|
||||
local dlls abi
|
||||
dlls="$(find "$PLUGINS/$PLUGIN_DIR" -maxdepth 1 -name '*.dll' | wc -l)"
|
||||
[ "$dlls" -ge 1 ] || {
|
||||
printf 'MISSING %s inside %s\n' "$TVDB_DLL" "$PLUGIN_DIR" >&2
|
||||
bad=1
|
||||
}
|
||||
abi="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1])).get("targetAbi","?"))' \
|
||||
"$PLUGINS/$PLUGIN_DIR/meta.json" 2>/dev/null || echo '?')"
|
||||
printf 'plugin %s: %s dll(s), targetAbi %s\n' "$PLUGIN_DIR" "$dlls" "$abi"
|
||||
fi
|
||||
local pair elem n
|
||||
for pair in "$TMDB_XML|TmdbApiKey" "$TVDB_XML|ApiKey"; do
|
||||
elem="${pair##*|}"
|
||||
if [ ! -f "${pair%%|*}" ]; then
|
||||
printf 'MISSING %s\n' "${pair%%|*}" >&2
|
||||
bad=1
|
||||
continue
|
||||
fi
|
||||
if ! python3 -c 'import sys, xml.dom.minidom; xml.dom.minidom.parse(sys.argv[1])' "${pair%%|*}"; then
|
||||
printf 'UNPARSEABLE %s\n' "${pair%%|*}" >&2
|
||||
bad=1
|
||||
continue
|
||||
fi
|
||||
n="$(xml_value "${pair%%|*}" "$elem")"
|
||||
if [ -z "$n" ]; then
|
||||
printf 'MISSING key: %s has no %s value\n' "$(basename "${pair%%|*}")" "$elem" >&2
|
||||
bad=1
|
||||
else
|
||||
printf '%s %s chars\n' "$elem" "${#n}"
|
||||
fi
|
||||
done
|
||||
[ "$bad" -eq 0 ] || return 1
|
||||
printf 'providers configured\n'
|
||||
}
|
||||
|
||||
cmd="${1:-}"
|
||||
case "$cmd" in
|
||||
plan) cmd_plan ;;
|
||||
install) cmd_install ;;
|
||||
verify) cmd_verify ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
+28
-3
@@ -114,6 +114,13 @@ def read_plan(path, root, force):
|
||||
return mkdirs, keeps, moves
|
||||
|
||||
|
||||
# A relayout moves bytes on the raw array; the sidecars the media servers wrote
|
||||
# live in the SSD overlay and stay at the old coordinates. Naming the follow-up
|
||||
# here is the point of use for that fact — see the Media Metadata View page.
|
||||
METADATA_HINT = ("note: sidecars written through the metadata view did not follow"
|
||||
" these moves — next: scripts/media-view.sh reap")
|
||||
|
||||
|
||||
def do_apply(args, root, journal, mkdirs, keeps, moves):
|
||||
"""Preflight everything, then folders, then renames. Never a partial start."""
|
||||
problems = []
|
||||
@@ -195,6 +202,7 @@ def do_apply(args, root, journal, mkdirs, keeps, moves):
|
||||
f" {dirs_new} folders created, {len(keeps)} files stayed put")
|
||||
print(f"journal: {journal.path} — undo with: media-apply.py rollback"
|
||||
f" --journal {journal.path} --root {root} --apply")
|
||||
print(METADATA_HINT)
|
||||
return 0
|
||||
|
||||
|
||||
@@ -241,10 +249,19 @@ def sweep_targets(root, journal):
|
||||
journalled destination lives under it. A symlink anywhere inside disqualifies
|
||||
the whole subtree: this tool never follows a link, and deleting the folder
|
||||
that holds one would delete someone's mount point.
|
||||
|
||||
Proof carries downward: once a folder qualifies, every empty directory inside
|
||||
it qualifies too. A release's junk (``Screens/``, ``Sample/``) is never itself
|
||||
a move source, so without that rule it holds its emptied parent on disk
|
||||
forever — the parent can never be removed while the child stands.
|
||||
"""
|
||||
moves = [r for r in journal.records if r.get("op") == "move"]
|
||||
made = {r["dst"] for r in journal.records if r.get("op") == "mkdir"}
|
||||
gone = {r["path"] for r in journal.records if r.get("op") == "rmdir"}
|
||||
# An rmdir INTENT that failed left its folder standing, and the next sweep
|
||||
# must be able to finish it: only a folder that is really gone is out of
|
||||
# scope. Counting every intent as done made 146 live survivors permanent.
|
||||
gone = {r["path"] for r in journal.records if r.get("op") == "rmdir"
|
||||
and not os.path.exists(r["path"])}
|
||||
dsts = [r["dst"] for r in moves]
|
||||
marked = set()
|
||||
for r in moves:
|
||||
@@ -268,8 +285,15 @@ def sweep_targets(root, journal):
|
||||
return False
|
||||
return True
|
||||
|
||||
return sorted((d for d in marked if removable(d)),
|
||||
key=lambda q: (-q.count(os.sep), q))
|
||||
roots = [d for d in marked if removable(d)]
|
||||
targets = set(roots)
|
||||
for d in roots:
|
||||
for dirpath, dirnames, _ in os.walk(d):
|
||||
for c in dirnames:
|
||||
p = os.path.join(dirpath, c)
|
||||
if removable(p):
|
||||
targets.add(p)
|
||||
return sorted(targets, key=lambda q: (-q.count(os.sep), q))
|
||||
|
||||
|
||||
def do_sweep(args, root, journal):
|
||||
@@ -362,6 +386,7 @@ def do_rollback(args, root, journal):
|
||||
t=datetime.datetime.now(datetime.timezone.utc).isoformat())
|
||||
print(f"rolled back: {undone} files back where the census found them,"
|
||||
f" {dirs_gone} empty folders removed, {recreated} swept folders recreated")
|
||||
print(METADATA_HINT)
|
||||
return 0
|
||||
|
||||
|
||||
|
||||
+107
-6
@@ -1,13 +1,14 @@
|
||||
#!/usr/bin/env python3
|
||||
r"""Read-only census of a movie library — the inputs a relayout plan is built from.
|
||||
r"""Read-only census of a media library — the inputs a relayout plan is built from.
|
||||
|
||||
media-census.py enumerate [--root DIR] [--out TSV]
|
||||
media-census.py probe [--enum TSV] [--out TSV] [--jobs N]
|
||||
media-census.py plex [--root DIR] [--db PATH] [--out TSV]
|
||||
media-census.py jellyfin [--root DIR] [--db PATH] [--container PREFIX] [--out TSV]
|
||||
|
||||
Run it on the machine that mounts the library. Nothing here writes inside the
|
||||
library and nothing is deleted; the only outputs are the three TSVs below, and
|
||||
scripts/media-plan.py reads them instead of touching media.
|
||||
library and nothing is deleted; the only outputs are the TSVs below, and the
|
||||
scripts/media-plan.py and scripts/media-shows.py read them instead of touching media.
|
||||
|
||||
enum bytes <TAB> ext <TAB> path every file under the root
|
||||
probed bytes <TAB> path <TAB> dur <TAB> vcodec <TAB> WxH <TAB> fps <TAB> vbit
|
||||
@@ -18,10 +19,17 @@ scripts/media-plan.py reads them instead of touching media.
|
||||
plex file <TAB> leaf_title <TAB> … (see PLEX_COLS)
|
||||
the titles Plex already matched — the authority over any filename guess
|
||||
|
||||
jellyfin file <TAB> series <TAB> year <TAB> season <TAB> episode <TAB> extra
|
||||
<TAB> tmdb (see JF_COLS)
|
||||
one row per episode Jellyfin has matched and numbered, its path mapped
|
||||
from the container's view back to the host; `year` and `tmdb` are the
|
||||
SERIES' values, `extra` is Jellyfin's own extra type when the row is
|
||||
bonus material rather than an episode
|
||||
|
||||
`probe` is the slow step (every container is opened for its header, ~10 min for
|
||||
4,800 files on a spindle array at 6 workers) so it is `nice`d: a census beside a
|
||||
live service must not starve it. `enumerate` is a stat pass, so re-censusing
|
||||
before a plan is cheap — and a plan is only ever as good as these three files.
|
||||
before a plan is cheap — and a plan is only ever as good as these files.
|
||||
"""
|
||||
import argparse
|
||||
import csv
|
||||
@@ -57,6 +65,14 @@ join metadata_items mi on mi.id = m.metadata_item_id
|
||||
left join metadata_items par on par.id = mi.parent_id
|
||||
where p.file like ? and p.deleted_at is null and mi.deleted_at is null
|
||||
"""
|
||||
JF_DB = "/home/dank/Containers/jellyfin/config/data/jellyfin.db"
|
||||
JF_COLS = ["file", "series", "year", "season", "episode", "extra", "tmdb"]
|
||||
# Jellyfin stores the entity's .NET class as its type; the TV tree is these three.
|
||||
JF_TV = ("MediaBrowser.Controller.Entities.TV.Episode",
|
||||
"MediaBrowser.Controller.Entities.TV.Season",
|
||||
"MediaBrowser.Controller.Entities.TV.Series")
|
||||
JF_SQL = ("""select Id, Path, IndexNumber, ParentIndexNumber, ExtraType, IsVirtualItem
|
||||
from BaseItems where Type = ? and Path like ?""")
|
||||
|
||||
|
||||
def enumerate_lib(root, out):
|
||||
@@ -165,7 +181,7 @@ def plex(root, db, out):
|
||||
"Plex runs rootless here: run this as the container's owner, never "
|
||||
"with sudo — rootless storage is per-user, so root sees nothing.")
|
||||
with open(out, "w", newline="") as fh:
|
||||
w = csv.writer(fh, delimiter="\t")
|
||||
w = csv.writer(fh, delimiter="\t", lineterminator="\n")
|
||||
w.writerow(PLEX_COLS)
|
||||
n = 0
|
||||
for r in con.execute(PLEX_SQL, (root + "/%",)):
|
||||
@@ -174,6 +190,79 @@ def plex(root, db, out):
|
||||
print(f"plex-matched={n} -> {out}")
|
||||
|
||||
|
||||
def jellyfin(root, db, container, out):
|
||||
"""The episodes Jellyfin has already numbered, for the library at root.
|
||||
|
||||
Jellyfin stores each path as the CONTAINER sees it (its quadlet binds the
|
||||
share at /data), so `container` is the in-container prefix that stands for
|
||||
root and every exported path is mapped back to the host before it is written
|
||||
— the planner reads host paths, because it plans moves on the array.
|
||||
|
||||
The numbering is what matters, not the name: a release group's file name says
|
||||
nothing about which episode a file is, and Jellyfin has already answered that
|
||||
question against a provider. Season/episode come from the episode row; the
|
||||
series name, its year and its TMDb id come from the Series row the episode
|
||||
climbs to, because those are the only place Jellyfin keeps them.
|
||||
"""
|
||||
try:
|
||||
con = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
|
||||
except sqlite3.Error as e:
|
||||
sys.exit(f"cannot open the Jellyfin database read-only ({db}): {e}\n"
|
||||
"It is a WAL database: copy it with its -wal and -shm beside it and"
|
||||
" point --db at the copy. Never sudo it — the data dir belongs to"
|
||||
" the container's owner, and rootless storage is per-user.")
|
||||
have = {r[1] for r in con.execute("pragma table_info(BaseItems)")}
|
||||
need = {"Id", "ParentId", "Type", "Path", "IndexNumber", "ParentIndexNumber",
|
||||
"ExtraType", "IsVirtualItem", "ProductionYear", "Data"}
|
||||
if not need <= have:
|
||||
sys.exit(f"{db} is not a Jellyfin library database (BaseItems has no"
|
||||
f" {','.join(sorted(need - have))})")
|
||||
node = {}
|
||||
for i, pid, typ, name, year, data in con.execute(
|
||||
"select Id, ParentId, Type, Name, ProductionYear, Data from BaseItems"
|
||||
" where Type in (?,?,?)", JF_TV):
|
||||
node[i] = (pid, typ, name, year, data)
|
||||
|
||||
def series_of(item_id):
|
||||
"""Climb episode -> season -> series; empty when the chain is broken."""
|
||||
for _ in range(4):
|
||||
if item_id not in node:
|
||||
return "", "", ""
|
||||
pid, typ, name, year, data = node[item_id]
|
||||
if typ.endswith(".Series"):
|
||||
try:
|
||||
tmdb = (json.loads(data or "{}")
|
||||
.get("ProviderIds") or {}).get("Tmdb", "")
|
||||
except ValueError:
|
||||
tmdb = ""
|
||||
return name or "", year or "", tmdb
|
||||
item_id = pid
|
||||
return "", "", ""
|
||||
|
||||
# The fleet's quadlets bind the share at /data, so the container's view of
|
||||
# this library is /data/<the library's own folder name> unless told otherwise.
|
||||
if not container:
|
||||
container = "/data/" + os.path.basename(os.path.normpath(root))
|
||||
prefix = container.rstrip("/") + "/"
|
||||
written = skipped = 0
|
||||
with open(out, "w", newline="") as fh:
|
||||
w = csv.writer(fh, delimiter="\t", lineterminator="\n")
|
||||
w.writerow(JF_COLS)
|
||||
for iid, path, idx, pidx, extra, virt in con.execute(JF_SQL,
|
||||
(JF_TV[0], prefix + "%")):
|
||||
if virt or not path or not path.startswith(prefix):
|
||||
skipped += 1 # a virtual item is a variant, not a file
|
||||
continue
|
||||
sname, syear, tmdb = series_of(iid)
|
||||
w.writerow([root + "/" + path[len(prefix):],
|
||||
sname.replace("\t", " "), syear,
|
||||
"" if pidx is None else pidx,
|
||||
"" if idx is None else idx,
|
||||
(extra or "").replace("\t", " "), tmdb])
|
||||
written += 1
|
||||
print(f"episodes={written} virtual-or-blank={skipped} -> {out}")
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
sub = ap.add_subparsers(dest="cmd", required=True)
|
||||
@@ -190,13 +279,25 @@ def main():
|
||||
x.add_argument("--root", **root)
|
||||
x.add_argument("--db", default=PLEX_DB)
|
||||
x.add_argument("--out", default="/tmp/plex-movies.tsv")
|
||||
j = sub.add_parser("jellyfin",
|
||||
help="export the episodes Jellyfin has numbered for the root")
|
||||
j.add_argument("--root", default=os.environ.get("MEDIA_ROOT",
|
||||
"/mnt/entropy/TVShows"),
|
||||
help="TV library root on the host")
|
||||
j.add_argument("--db", default=JF_DB)
|
||||
j.add_argument("--container", default=None,
|
||||
help="the in-container prefix that stands for --root"
|
||||
" (default /data/<root's folder name>)")
|
||||
j.add_argument("--out", default="/tmp/jellyfin-tv.tsv")
|
||||
a = ap.parse_args()
|
||||
if a.cmd == "enumerate":
|
||||
enumerate_lib(a.root, a.out)
|
||||
elif a.cmd == "probe":
|
||||
probe(a.enum, a.out, a.jobs)
|
||||
else:
|
||||
elif a.cmd == "plex":
|
||||
plex(a.root, a.db, a.out)
|
||||
else:
|
||||
jellyfin(a.root, a.db, a.container, a.out)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -222,8 +222,9 @@ eps = collections.defaultdict(list)
|
||||
for r in erows:
|
||||
if r["path"] in marks | extras:
|
||||
continue
|
||||
if eng.ep_of(r["path"]):
|
||||
eps[eng.key_of(r["title"])].append(r)
|
||||
ep = eng.ep_of(r["path"])
|
||||
if ep:
|
||||
eps[ep[0]].append(r) # keyed by the SERIES, never the episode title
|
||||
tv_keys = {k for k, v in eps.items() if len(v) >= 5}
|
||||
tv_set = {r["path"] for k, v in eps.items() if k in tv_keys for r in v}
|
||||
# A folder INSIDE a movie's folder is bonus territory even when its own name is
|
||||
@@ -776,19 +777,17 @@ readmes.append(f"README in {ROOT}: this library was relaid out into Jellyfin's "
|
||||
"one-folder-per-movie layout; `_release-group watermarks/`, "
|
||||
"`_unsorted files/` and `_unsorted bonus/` hold files no film claims.")
|
||||
for key in tv_keys:
|
||||
nm = "_TV needs a TV library - " + collections.Counter(
|
||||
r["title"] for r in eps[key]).most_common(1)[0][0]
|
||||
readmes.append(f"README in {ROOT}/{sanitize(nm)}: these are TV episodes found in "
|
||||
"the MOVIES library; move the folder to a TV library source, else "
|
||||
"Jellyfin groups them wrongly as one movie.")
|
||||
readmes.append(f"README in {ROOT}/{sanitize('_TV needs a TV library - ' + key)}: "
|
||||
"these are TV episodes found in the MOVIES library; move the folder "
|
||||
"to a TV library source, else Jellyfin groups them wrongly as one "
|
||||
"movie.")
|
||||
|
||||
# --------------------------------------------------------------------- TV
|
||||
for key in tv_keys:
|
||||
items = eps[key]
|
||||
title = collections.Counter(r["title"] for r in items).most_common(1)[0][0]
|
||||
year = collections.Counter(r["year"] for r in items if r["year"] != "?")
|
||||
yr = year.most_common(1)[0][0] if year else None
|
||||
base = f"_TV needs a TV library - {sanitize(title)}" + (f" ({yr})" if yr else "")
|
||||
base = f"_TV needs a TV library - {sanitize(key)}" + (f" ({yr})" if yr else "")
|
||||
tops = {parts_of(r["path"])[0] for r in items}
|
||||
members = sorted([r["path"] for r in items]
|
||||
+ [q for q in extras | marks if len(parts_of(q)) > 1
|
||||
|
||||
Executable
+705
@@ -0,0 +1,705 @@
|
||||
#!/usr/bin/env python3
|
||||
r"""Plan a Jellyfin-format relayout of a TV library. DRY RUN ONLY: nothing moves.
|
||||
|
||||
media-shows.py [--root DIR] [--enum TSV] [--titles TSV] [--probed TSV]
|
||||
[--tmdb-cache JSON] [--tmdb] [--movies DIR] [--park DIR]
|
||||
[--out TSV] [--exceptions TXT]
|
||||
|
||||
Inputs are the TSVs scripts/media-census.py exports: `enumerate` lists what
|
||||
exists, `jellyfin` says which episode each file IS, and `probe` is optional
|
||||
duration evidence for the movie rule. The TMDb cache is a plain JSON file so a
|
||||
plan stays reproducible: answers are read from it, and a miss is looked up only
|
||||
with --tmdb and TMDB_API_KEY in the environment (the key never enters argv, and
|
||||
never the cache).
|
||||
|
||||
Target contract (jellyfin.org/docs/general/server/media/shows):
|
||||
|
||||
<root>/<Series> (<year>)/Season <NN>/<Series> SxxEyy.ext
|
||||
one file holding two episodes SxxE01-E02 specials Season 00
|
||||
a documented part of an episode SxxE03 Part 1 sidecars <base>.<lang>.srt
|
||||
bonus material <series>/trailers|featurettes|… — never a season
|
||||
|
||||
PLAN is the TSV scripts/media-apply.py replays: `action src dst kind`, every
|
||||
mkdir row before any move, and `keep` for a file already where the contract puts
|
||||
it. EXCEPTIONS is the review list: everything parked, and why.
|
||||
|
||||
The authority chain, strongest first, for one video file:
|
||||
|
||||
1. a provider row — Jellyfin matched this exact path and numbered it.
|
||||
2. the file's own SxxEyy / 1x04 marker, the series taken from a sibling's
|
||||
provider row, else from the nearest ancestor folder that has one, else from
|
||||
the release folder's own words.
|
||||
3. nothing. An unnumbered file is PARKED under --park with its relative path
|
||||
kept: an invented `Season 01` steals an episode number from the file that
|
||||
really is S01E01, and a wrong number is worse than no number.
|
||||
|
||||
Guards, and why each is here:
|
||||
|
||||
* One series, one folder, one year: the year ANY provider row of a series
|
||||
states is shared by all of them, so a box set split across five release
|
||||
folders lands inside ONE series folder instead of five.
|
||||
* Two files claiming one episode: the bigger keeps the standard name, the other
|
||||
parks with the reason. Nothing is overwritten and nothing is deleted — which
|
||||
encode survives is Frank's call, and a parked file is one rename from home.
|
||||
* A file with no episode evidence that runs long and that the MOVIE provider
|
||||
names leaves the TV library for --movies under the movie naming contract.
|
||||
Long and unnamed parks; short and unnumbered is bonus material.
|
||||
* Series art and the show nfo follow the SERIES, a season image follows its
|
||||
SEASON, a subtitle or episode nfo follows the episode's new base — an episode
|
||||
moved without its subtitles is a half-move.
|
||||
* Hidden files, macOS forks, qBittorrent resume parts and the park pile itself
|
||||
are never moved. Skipping the park is what makes re-planning stable.
|
||||
|
||||
The plan crosses three trees (the library, its park, the movie library), so the
|
||||
executor runs with --root at the share that holds them, on its own journal —
|
||||
one journal is one undo:
|
||||
|
||||
python3 scripts/media-shows.py --root /mnt/entropy/TVShows \
|
||||
--enum /tmp/tv-enum.tsv --titles /tmp/jellyfin-tv.tsv \
|
||||
--out /mnt/entropy/media-tv-plan.tsv
|
||||
python3 scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv \
|
||||
--root /mnt/entropy --journal /mnt/entropy/media-tv-relayout.jsonl
|
||||
"""
|
||||
import argparse
|
||||
import collections
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
ARGS = argparse.ArgumentParser(description=__doc__.splitlines()[0],
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ARGS.add_argument("--root", default=os.environ.get("TV_ROOT", "/mnt/entropy/TVShows"),
|
||||
help="the TV library the plan reorganises")
|
||||
ARGS.add_argument("--enum", default="/tmp/tv-enum.tsv",
|
||||
help="file census (media-census.py enumerate)")
|
||||
ARGS.add_argument("--titles", default="/tmp/jellyfin-tv.tsv",
|
||||
help="the provider's numbering (media-census.py jellyfin)")
|
||||
ARGS.add_argument("--probed", default="",
|
||||
help="ffprobe census for the movie rule; absent = no durations")
|
||||
ARGS.add_argument("--tmdb-cache", default="/tmp/tmdb-cache.json",
|
||||
help="TMDb answers as an input file, so a plan is reproducible")
|
||||
ARGS.add_argument("--tmdb", action="store_true",
|
||||
help="look up cache misses (needs TMDB_API_KEY)")
|
||||
ARGS.add_argument("--movies", default="/mnt/entropy/Movies",
|
||||
help="movie library a mis-filed film moves into")
|
||||
ARGS.add_argument("--park", default="",
|
||||
help="review pile (default: <root>/_unsorted)")
|
||||
ARGS.add_argument("--out", default="/tmp/tv-plan.tsv")
|
||||
ARGS.add_argument("--exceptions", default="/tmp/tv-exceptions.txt")
|
||||
ARGS = ARGS.parse_args()
|
||||
|
||||
ROOT = os.path.abspath(ARGS.root)
|
||||
PARK = os.path.abspath(ARGS.park or os.path.join(ROOT, "_unsorted"))
|
||||
MOVIES = os.path.abspath(ARGS.movies)
|
||||
ENGINE = os.path.join(os.path.dirname(os.path.abspath(__file__)), "media-titles.py")
|
||||
_spec = importlib.util.spec_from_file_location("eng", ENGINE)
|
||||
eng = importlib.util.module_from_spec(_spec)
|
||||
_spec.loader.exec_module(eng)
|
||||
|
||||
try:
|
||||
CACHE = json.load(open(ARGS.tmdb_cache))
|
||||
except (OSError, ValueError):
|
||||
CACHE = {}
|
||||
API_KEY = os.environ.get("TMDB_API_KEY", "")
|
||||
|
||||
VIDEO_EXT = {".mkv", ".mp4", ".avi", ".m4v", ".mov", ".ts", ".m2ts", ".mpg",
|
||||
".mpeg", ".webm", ".wmv", ".divx", ".ogm", ".flv", ".vob", ".rm",
|
||||
".rmvb", ".iso", ".img"}
|
||||
SUB_EXT = {".srt", ".ass", ".ssa", ".sub", ".vtt"}
|
||||
IMG_EXT = {".jpg", ".jpeg", ".png", ".webp", ".bmp", ".gif", ".tbn"}
|
||||
NFO_EXT = {".nfo", ".xml"}
|
||||
# The container's own word, for a sidecar named "episode.mkv.srt": that component
|
||||
# is not a language flag, and both matchers read the plain form better.
|
||||
CONTAINER = {e[1:] for e in VIDEO_EXT}
|
||||
# Files that are never anybody's business: hidden, forks, resume state, thumbnails.
|
||||
SKIP_NAME = re.compile(r"^(\.|#|dl_state|@eaDir|Thumbs\.db|desktop\.ini)|\.!q$")
|
||||
# Show-level metadata keeps its own name at the series root; season-level at the
|
||||
# season root; folder.jpg stays with the folder it was written for.
|
||||
SERIES_META = re.compile(r"^(tvshow|series|show)(\.nfo|\.xml)?$"
|
||||
r"|^(poster|tvposter|banner|clearlogo|clearart|logo|art"
|
||||
r"|landscape|fanart|showbackdrop|backdrop)\d*\."
|
||||
r"(jpg|jpeg|png|webp|bmp)$", re.I)
|
||||
SEASON_META = re.compile(r"^season\.nfo$", re.I)
|
||||
FOLDER_IMAGE = re.compile(r"^folder\.(jpg|jpeg|png|webp|bmp)$", re.I)
|
||||
SEASON_DIR_RE = re.compile(r"(?i)^\s*(?:season[\s._-]*(\d{1,3})|s(\d{1,2}))\s*$")
|
||||
BAD_CHARS = re.compile(r'[<>:"/|?*]')
|
||||
# SxxEyy with an optional second episode, plus the 1x04 form. The second number
|
||||
# is guarded so a resolution never reads as an episode: "S01E01-1080p" is ONE
|
||||
# episode, and the longest-match backtracking of \d{1,3} is what the (?![\dpP])
|
||||
# after it closes off.
|
||||
EP_RANGE = re.compile(r"(?i)(?<![a-z0-9])s(?:eason[\s._-]*)?(\d{1,2})[\s._-]*"
|
||||
r"e(?:p(?:isode)[\s._-]*)?(\d{1,3})"
|
||||
r"(?:[\s._-]*(?:-|to[\s._-]*|ep)[\s._-]*(?:s\d{1,2}[\s._-]*)?e?"
|
||||
r"(\d{1,3})(?![\dpP]))?")
|
||||
NX_RE = re.compile(r"(?i)(?<![a-z0-9])(\d{1,2})x(\d{2,3})(?![a-z0-9])")
|
||||
PART_RE = re.compile(r"(?i)(?:^|[\s._-])(?:part|pt|cd|dvd|disc|disk)[\s._-]?"
|
||||
r"([1-9]|i{1,3})(?:$|[\s._-])")
|
||||
# Jellyfin reads these subfolder names as bonus material. The planner emits only
|
||||
# these spellings, so 40k extras end up in nine folders instead of nine hundred.
|
||||
EXTRA_DIRS = [
|
||||
("trailers", r"trailer|teaser|promo|advert|commercial|tv[\s._-]spot|preview"),
|
||||
("deleted scenes", r"deleted"),
|
||||
("behind the scenes", r"behind[\s._-]the[\s._-]scenes|making[\s._-]of|bts"
|
||||
r"|b[\s._-]roll|gag[\s._-]reel|blooper|outtake|archival"
|
||||
r"|video[\s._-]archive|scene[\s._-]selection|menu|gallery"
|
||||
r"|stills?|visual[\s._-]effects|workprint"),
|
||||
("interviews", r"interview|cast[\s._-]and[\s._-]crew|audition|commentary"),
|
||||
("scene cuts", r"scene[\s._-]cuts?|scenecuts"),
|
||||
("featurettes", r"featurettes?"),
|
||||
("shorts", r"shorts?"),
|
||||
("clips", r"clips?"),
|
||||
]
|
||||
# Jellyfin's own ExtraType values, folded onto the same folder names.
|
||||
EXTRA_KIND = {"trailer": "trailers", "tmbd": "trailers", "intro": "other",
|
||||
"movie": "other", "scene": "scene cuts", "clip": "clips",
|
||||
"interview": "interviews", "behindthescenes": "behind the scenes",
|
||||
"deleted": "deleted scenes", "other": "other", "epilogue": "other",
|
||||
"opening": "other", "featurette": "featurettes", "short": "shorts"}
|
||||
SHORT_MAX = 45 * 60 # over this, a file with no episode evidence is film-shaped
|
||||
|
||||
|
||||
def rel(p):
|
||||
return p[len(ROOT) + 1:] if p.startswith(ROOT + "/") else p
|
||||
|
||||
|
||||
def clean(name):
|
||||
"""A name the filesystem and both matchers can read."""
|
||||
name = BAD_CHARS.sub("-", name).replace("\\", "-").strip(" .")
|
||||
return name[:180] or "untitled"
|
||||
|
||||
|
||||
def title_of(text):
|
||||
"""The engine's parser, applied to a name that has dots but no extension:
|
||||
splitext would read "The.Examples" as stem "The", so a sentinel extension is
|
||||
added and the parser sees an ordinary media name."""
|
||||
return eng.title_of(text + ".mkv")
|
||||
|
||||
|
||||
def markers(name):
|
||||
"""(season, [episodes], position) the episode markers a name states; None when
|
||||
it states none. The position is where the series' own words stop."""
|
||||
stem = os.path.splitext(name)[0]
|
||||
m = EP_RANGE.search(stem)
|
||||
if m:
|
||||
eps = [int(m.group(2))] + ([int(m.group(3))] if m.group(3) else [])
|
||||
return int(m.group(1)), eps, m.start()
|
||||
m = NX_RE.search(stem)
|
||||
if m:
|
||||
return int(m.group(1)), [int(m.group(2))], m.start()
|
||||
return None
|
||||
|
||||
|
||||
LEAD_IDX_RE = re.compile(r"^\s*(\d{1,3})\s*[-\u2013.\s]\s*[A-Za-z]")
|
||||
PACK_IDX_RE = re.compile(r"^\s*\d+[a-z]?[.)]\s+")
|
||||
|
||||
|
||||
def num(x):
|
||||
try:
|
||||
return int(x)
|
||||
except (TypeError, ValueError):
|
||||
return 0
|
||||
|
||||
|
||||
def folder_season(path):
|
||||
"""The season the FOLDER states (`Season 3`, `S3`). A human wrote it, and where
|
||||
a scraped row disagrees this is the one that survives a re-plan."""
|
||||
m = SEASON_DIR_RE.match(os.path.basename(os.path.dirname(path)))
|
||||
return int(next(g for g in m.groups() if g)) if m else None
|
||||
|
||||
|
||||
def leading_index(name):
|
||||
"""The number a file leads with inside a season folder — `01 Kyle MacLachlan,
|
||||
Sinead OC.avi` is episode 1. Both matchers read it that way."""
|
||||
m = LEAD_IDX_RE.match(os.path.splitext(name)[0])
|
||||
return int(m.group(1)) if m else None
|
||||
|
||||
|
||||
def tmdb(kind, query):
|
||||
"""A provider answer for a query: the cache first, TMDb once, then never again."""
|
||||
ck = f"{kind}:{query}"
|
||||
if ck in CACHE:
|
||||
hit = CACHE[ck]
|
||||
# A cache an operator edited by hand can carry the year as a number; the
|
||||
# comparison below is text, so the wire form is imposed on every hit.
|
||||
return None if hit is None else {**hit, "year": str(hit.get("year") or "")}
|
||||
if not (ARGS.tmdb and API_KEY):
|
||||
return None
|
||||
path = "movie" if kind == "movie" else "tv"
|
||||
url = ("https://api.themoviedb.org/3/search/" + path + "?query="
|
||||
+ urllib.parse.quote(query) + "&include_adult=false")
|
||||
req = urllib.request.Request(url, headers={
|
||||
# The key rides a header, never the URL: a token in argv is world-readable.
|
||||
"Authorization": "Bearer " + API_KEY,
|
||||
"Accept": "application/json"})
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=20) as r:
|
||||
found = json.load(r).get("results") or []
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"tmdb {kind} lookup failed for {query!r}: {e}", file=sys.stderr)
|
||||
return None
|
||||
if not found:
|
||||
CACHE[ck] = None
|
||||
return None
|
||||
top = found[0]
|
||||
CACHE[ck] = {"name": (top.get("title") or top.get("name") or "").strip(),
|
||||
"year": str(top.get("year") or "") or (top.get("first_air_date") or "")[:4],
|
||||
"id": top.get("id")}
|
||||
return CACHE[ck]
|
||||
|
||||
|
||||
# ------------------------------------------------------------------- the inputs
|
||||
rows = []
|
||||
for line in open(ARGS.enum):
|
||||
size, ext, path = line.rstrip("\n").split("\t")
|
||||
p = os.path.abspath(path)
|
||||
base = os.path.basename(p)
|
||||
if not p.startswith(ROOT + "/") or SKIP_NAME.search(base):
|
||||
continue
|
||||
if p.startswith(PARK + "/"):
|
||||
continue # the park is already reviewed; re-parking is churn
|
||||
rows.append({"bytes": int(size), "ext": ext.lower(), "path": p, "base": base})
|
||||
held = {r["path"]: r for r in rows}
|
||||
|
||||
titles = {}
|
||||
with open(ARGS.titles) as fh:
|
||||
cols = fh.readline().rstrip("\n").split("\t")
|
||||
for line in fh:
|
||||
f = line.rstrip("\n").split("\t")
|
||||
f += [""] * (len(cols) - len(f))
|
||||
row = dict(zip(cols, f))
|
||||
p = os.path.abspath(row.get("file", ""))
|
||||
if p in held:
|
||||
titles[p] = row
|
||||
|
||||
dur = {}
|
||||
if ARGS.probed and os.path.exists(ARGS.probed):
|
||||
for line in open(ARGS.probed):
|
||||
f = line.rstrip("\n").split("\t")
|
||||
if len(f) > 2:
|
||||
try:
|
||||
dur[os.path.abspath(f[1])] = float(f[2] or 0)
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
video = [r for r in rows if r["ext"] in VIDEO_EXT]
|
||||
other = [r for r in rows if r["ext"] not in VIDEO_EXT]
|
||||
|
||||
# What a folder already knows: one provider row names the series for every sibling
|
||||
# that has no row of its own — that is how an anime pack gets its show.
|
||||
by_dir = collections.defaultdict(list)
|
||||
for r in video:
|
||||
by_dir[os.path.dirname(r["path"])].append(r)
|
||||
dir_series = {}
|
||||
for d, rs in by_dir.items():
|
||||
names = collections.Counter(titles[r["path"]]["series"] for r in rs
|
||||
if r["path"] in titles and titles[r["path"]]["series"])
|
||||
if names:
|
||||
dir_series[d] = names.most_common(1)[0][0]
|
||||
|
||||
|
||||
# The registry: every series a provider names, judged before any folder is. Folder
|
||||
# words may resolve a file to one of these names; they may never invent one. That is
|
||||
# the whole difference between "NCOP 01.mkv" inside the Gurren Lagann pack, which the
|
||||
# pack names, and a new root folder called "01 NCOP".
|
||||
registry = {}
|
||||
for _p, _t in titles.items():
|
||||
if _t["series"]:
|
||||
registry.setdefault(eng.norm(_t["series"]), _t["series"])
|
||||
|
||||
|
||||
def known_in(text):
|
||||
"""The registered series named inside this text: the longest registered name
|
||||
whose words appear in it as a run. "(CBB) Tengen Toppa Gurren Lagann - The
|
||||
Complete Series (BD 1080p)" names "Tengen Toppa Gurren Lagann"."""
|
||||
hay = " " + eng.norm(text) + " "
|
||||
best = ""
|
||||
for k, name in registry.items():
|
||||
if len(k) >= 5 and f" {k} " in hay and len(k) > len(best):
|
||||
best = name
|
||||
return best
|
||||
|
||||
|
||||
# Rows also name their own folders' ancestors, so an already-filed library stays
|
||||
# where it is when it is re-planned.
|
||||
for d, name in sorted(dir_series.items()):
|
||||
a = os.path.dirname(d)
|
||||
while a and a != ROOT and len(a) > len(ROOT):
|
||||
if a not in dir_series:
|
||||
got = known_in(os.path.basename(a))
|
||||
if got:
|
||||
dir_series[a] = got
|
||||
a = os.path.dirname(a)
|
||||
|
||||
|
||||
def series_of_dir(d):
|
||||
"""The series an ancestor folder belongs to, nearest first: the folder holding
|
||||
provider rows, else a folder whose own words name a registered show."""
|
||||
while d and d != ROOT and len(d) > len(ROOT):
|
||||
if d in dir_series:
|
||||
return dir_series[d]
|
||||
got = known_in(os.path.basename(d))
|
||||
if got:
|
||||
return got
|
||||
d = os.path.dirname(d)
|
||||
return ""
|
||||
|
||||
|
||||
def extras_kind(path, extra_type):
|
||||
"""The canonical bonus subfolder, from Jellyfin's own extra type first."""
|
||||
if extra_type:
|
||||
return EXTRA_KIND.get(eng.norm(extra_type).replace(" ", ""), "other")
|
||||
for d in rel(path).split("/")[:-1]:
|
||||
if eng.folder_is_bonus(d):
|
||||
for name, rx in EXTRA_DIRS:
|
||||
if re.search(rx, d, re.I):
|
||||
return name
|
||||
return "extras"
|
||||
stem = os.path.splitext(os.path.basename(path))[0]
|
||||
for name, rx in EXTRA_DIRS:
|
||||
if re.search(rx, stem, re.I):
|
||||
return name
|
||||
return "extras"
|
||||
|
||||
|
||||
# ----------------------------------------------------------- classify every file
|
||||
series = {} # norm(name) -> {name, year, tmdb}
|
||||
claimants = [] # (row, name, season, [episodes])
|
||||
extras = [] # (row, name, bonus folder)
|
||||
films = [] # film-shaped, no episode evidence
|
||||
unknown = [] # (row, reason)
|
||||
|
||||
|
||||
def series_name(name):
|
||||
"""The provider's name as a folder: the pack index a scene release leads with
|
||||
("4a. Seed DESTINY") falls away, and the doubled spaces it leaves fold."""
|
||||
return clean(re.sub(r"\s{2,}", " ", PACK_IDX_RE.sub("", name)).strip(" ."))
|
||||
|
||||
|
||||
def note(name, year="", tmdb_id=""):
|
||||
"""Register a series; the first name wins, the first year and id stick."""
|
||||
k = eng.norm(name)
|
||||
s = series.setdefault(k, {"name": series_name(name), "year": "", "tmdb": ""})
|
||||
if year and not s["year"]:
|
||||
s["year"] = str(year)
|
||||
if tmdb_id and not s["tmdb"]:
|
||||
s["tmdb"] = str(tmdb_id)
|
||||
|
||||
|
||||
|
||||
def as_extra(r, owner, extra_type):
|
||||
"""Bonus material needs a series to file under; with none it is parked."""
|
||||
if not owner:
|
||||
unknown.append((r, "bonus material with no series to own it"))
|
||||
return
|
||||
note(owner)
|
||||
extras.append((r, owner, extras_kind(r["path"], extra_type)))
|
||||
|
||||
for r in video:
|
||||
p, t = r["path"], titles.get(r["path"])
|
||||
marked = markers(r["base"])
|
||||
fs = folder_season(p)
|
||||
if t and t["extra"]:
|
||||
as_extra(r, t["series"] or series_of_dir(os.path.dirname(p)), t["extra"])
|
||||
continue
|
||||
row = bool(t and t["series"] and (t["season"] or t["episode"]))
|
||||
if marked or row:
|
||||
# The provider row names the SHOW: its registered name, its year. The number
|
||||
# that orders the file comes from the file's own name first, then the folder
|
||||
# it sits in, then the row — scraped pack rows number episodes badly, where
|
||||
# every file under SNL 16-20/Season 16..19 carries season 16 episode 20.
|
||||
owner = (t["series"] if t and t["series"]
|
||||
else series_of_dir(os.path.dirname(p))
|
||||
or title_of(r["base"][:marked[2]])[0])
|
||||
if not eng.norm(owner).replace(" ", ""):
|
||||
unknown.append((r, "an episode marker with no series name to hang it on"))
|
||||
continue
|
||||
if marked:
|
||||
season, eps = marked[0], marked[1]
|
||||
else:
|
||||
season = fs if fs is not None else num(t["season"])
|
||||
ep = leading_index(r["base"]) or num(t["episode"])
|
||||
if not 0 < season <= 200 or ep <= 0:
|
||||
unknown.append((r, f"a provider row of season {season} episode {ep}"))
|
||||
continue
|
||||
eps = [ep]
|
||||
claimants.append((r, owner, season, eps))
|
||||
note(owner, t["year"] if t else "", t["tmdb"] if t else "")
|
||||
continue
|
||||
owner = series_of_dir(os.path.dirname(p))
|
||||
if fs is not None and owner and leading_index(r["base"]):
|
||||
claimants.append((r, owner, fs, [leading_index(r["base"])]))
|
||||
note(owner)
|
||||
continue
|
||||
if eng.is_extra(p, title_of(r["base"])[0], dur.get(p, 0.0), ROOT):
|
||||
as_extra(r, owner, "")
|
||||
continue
|
||||
if dur.get(p, 0.0) >= SHORT_MAX or (p not in dur and title_of(r["base"])[1] != "?"):
|
||||
films.append(r)
|
||||
continue
|
||||
unknown.append((r, "no episode marker, no provider row, nothing long enough to be a film"))
|
||||
|
||||
# A provider that never states a year still has one: the search is exact, so a
|
||||
# miss parks nothing — the folder simply carries no year, which Jellyfin accepts.
|
||||
for k, s in series.items():
|
||||
if not s["year"]:
|
||||
got = tmdb("tv", s["name"])
|
||||
if got and eng.norm(got["name"]) == k:
|
||||
s["year"] = got["year"]
|
||||
|
||||
if ARGS.tmdb:
|
||||
with open(ARGS.tmdb_cache, "w") as fh:
|
||||
json.dump(CACHE, fh, indent=1, sort_keys=True)
|
||||
|
||||
|
||||
def series_dir(name):
|
||||
s = series.get(eng.norm(name))
|
||||
if not s:
|
||||
s = {"name": clean(name), "year": ""}
|
||||
label = f"{s['name']} ({s['year']})" if s["year"] else s["name"]
|
||||
return f"{ROOT}/{clean(label)}"
|
||||
|
||||
|
||||
# ------------------------------------------------------------------- the plan
|
||||
moves, keeps, parked = [], [], []
|
||||
reasons = collections.defaultdict(list)
|
||||
used = {} # dst -> the file that claimed it
|
||||
|
||||
|
||||
def book(r, dst_dir, name, kind):
|
||||
"""Claim one destination for one file, or park the file when it is taken."""
|
||||
dst = f"{dst_dir}/{name}"
|
||||
if dst == r["path"]:
|
||||
keeps.append((r["path"], dst, kind))
|
||||
return
|
||||
if dst in used:
|
||||
park(r, "something else in the plan already files there")
|
||||
return
|
||||
if dst in held:
|
||||
park(r, f"{rel(dst)} already occupies that name")
|
||||
return
|
||||
used[dst] = r["path"]
|
||||
moves.append((r["path"], dst, kind))
|
||||
|
||||
|
||||
def park(r, reason):
|
||||
parked.append((r, reason))
|
||||
reasons[reason].append(r)
|
||||
dst = f"{PARK}/{rel(r['path'])}"
|
||||
if dst in used or dst in held:
|
||||
dst = f"{PARK}/{rel(r['path'])}.{len(reasons)}"
|
||||
used[dst] = r["path"]
|
||||
moves.append((r["path"], dst, "park"))
|
||||
|
||||
|
||||
# Episodes first: two files claiming one number is a decision Frank makes, not one
|
||||
# the plan sneaks — the bigger copy takes the standard name, the other waits.
|
||||
by_ep = collections.defaultdict(list)
|
||||
for r, name, season, eps in claimants:
|
||||
for e in eps:
|
||||
by_ep[(eng.norm(name), season, e)].append(r)
|
||||
winners, losers = set(), {}
|
||||
for (key, season, e), rs in by_ep.items():
|
||||
rs.sort(key=lambda r: (-r["bytes"], r["path"]))
|
||||
winners.add(rs[0]["path"])
|
||||
for r in rs[1:]:
|
||||
losers.setdefault(r["path"],
|
||||
f"duplicate of {series[key]['name']} S{season:02d}E{e:02d}"
|
||||
" (the bigger copy keeps the standard name)")
|
||||
for r, name, season, eps in claimants:
|
||||
if r["path"] in losers:
|
||||
park(r, losers[r["path"]])
|
||||
continue
|
||||
s = series[eng.norm(name)]
|
||||
label = f" S{season:02d}E" + "-E".join(f"{e:02d}" for e in eps)
|
||||
part = PART_RE.search(os.path.splitext(r["base"])[0])
|
||||
tail = f" Part {part.group(1).upper()}" if part and len(eps) == 1 else ""
|
||||
book(r, f"{series_dir(name)}/Season {season:02d}", s["name"] + label + tail + r["ext"],
|
||||
"tv")
|
||||
|
||||
for r, name, kind in extras:
|
||||
book(r, f"{series_dir(name)}/{kind}",
|
||||
clean(os.path.splitext(r["base"])[0]) + r["ext"], "extra")
|
||||
|
||||
# A film inside the TV library leaves for the movie library only when the movie
|
||||
# provider answers with a title that matches the name and, when the name states a
|
||||
# year, that year too.
|
||||
for r in films:
|
||||
parsed, year = title_of(r["base"])
|
||||
got = tmdb("movie", parsed)
|
||||
words = eng.norm(got["name"]).split() if got else []
|
||||
if not got or not eng.mentions(parsed.split(), words) \
|
||||
or (year != "?" and got["year"] and got["year"] != year):
|
||||
park(r, "long file with no episode evidence, and no movie provider names it")
|
||||
continue
|
||||
label = f"{got['name']} ({got['year']})" if got["year"] else got["name"]
|
||||
book(r, f"{MOVIES}/{clean(label)}", clean(label) + r["ext"], "movie")
|
||||
|
||||
# ------------------------------------------------- sidecars and folder metadata
|
||||
moved = {} # src -> dst, for every booked file
|
||||
for src, dst, _kind in moves:
|
||||
moved[src] = dst
|
||||
for src, dst, _kind in keeps:
|
||||
moved[src] = dst
|
||||
dst_dirs = collections.defaultdict(collections.Counter)
|
||||
video_dst = collections.defaultdict(list)
|
||||
for src, dst in moved.items():
|
||||
if src in {r["path"] for r in video}:
|
||||
video_dst[os.path.dirname(src)].append((src, dst))
|
||||
dst_dirs[os.path.dirname(src)][os.path.dirname(dst)] += 1
|
||||
|
||||
|
||||
def series_name_at(d):
|
||||
"""The series the files of a folder belong to — a registered name, or nothing.
|
||||
Folder words alone never name a show."""
|
||||
names = [titles[src]["series"] for src, _ in video_dst.get(d, [])
|
||||
if src in titles and titles[src]["series"]]
|
||||
if names:
|
||||
return collections.Counter(names).most_common(1)[0][0]
|
||||
return series_of_dir(d)
|
||||
|
||||
|
||||
def season_at(d):
|
||||
"""The season number a folder carries: its own name, else its files' rows."""
|
||||
m = SEASON_DIR_RE.match(os.path.basename(d))
|
||||
if m:
|
||||
return int(m.group(1) or m.group(2))
|
||||
nums = [int(titles[src]["season"]) for src, _ in video_dst.get(d, [])
|
||||
if src in titles and titles[src]["season"]]
|
||||
return collections.Counter(nums).most_common(1)[0][0] if nums else None
|
||||
|
||||
|
||||
def folder_dst(d):
|
||||
"""Where a folder's own images go: the one place its files landed, or the
|
||||
series root when they scattered across several seasons."""
|
||||
c = dst_dirs.get(d)
|
||||
if not c:
|
||||
return None
|
||||
if len(c) == 1:
|
||||
return c.most_common(1)[0][0]
|
||||
owner = series_name_at(d)
|
||||
return series_dir(owner) if owner else None
|
||||
|
||||
|
||||
def anchor_of(path, stem):
|
||||
"""The video file this sidecar names: the longest file name it extends, in the
|
||||
NEAREST folder that holds one. A scene pack buries its subtitles two levels
|
||||
down (`Pack/Subs/<release-name>/<release>.jpn.srt`), so the search climbs to
|
||||
the library root instead of stopping at the parent."""
|
||||
d = os.path.dirname(path)
|
||||
while len(d) >= len(ROOT) and d.startswith(ROOT):
|
||||
best = None
|
||||
for src, dst in video_dst.get(d, []):
|
||||
base = os.path.basename(src)
|
||||
for prefix in (os.path.splitext(base)[0], base):
|
||||
if stem == prefix or stem.startswith(prefix + "."):
|
||||
if best is None or len(prefix) > best[2]:
|
||||
best = (src, dst, len(prefix))
|
||||
if best:
|
||||
return best[0], best[1]
|
||||
up = os.path.dirname(d)
|
||||
if up == d:
|
||||
break
|
||||
d = up
|
||||
return None
|
||||
|
||||
|
||||
def sidecar_name(r, src, dst):
|
||||
"""The sidecar's own tail — language, flag, the media extension an nfo must
|
||||
carry — hung on the video file's new base."""
|
||||
stem = os.path.splitext(os.path.basename(src))[0]
|
||||
tail = os.path.splitext(r["base"])[0][len(stem):]
|
||||
# "Show.S01E01.mkv.srt" carries the container's name, not a flag: Jellyfin and
|
||||
# Plex both read "Show S01E01.srt", so the container word is dropped here.
|
||||
parts = tail.split(".")
|
||||
if len(parts) > 1 and parts[1].lower() in CONTAINER:
|
||||
tail = "".join("." + p for p in parts[2:])
|
||||
if r["ext"] in NFO_EXT:
|
||||
tail = os.path.splitext(os.path.basename(src))[1] + tail
|
||||
return clean(os.path.splitext(os.path.basename(dst))[0] + tail + r["ext"])
|
||||
|
||||
|
||||
for r in other:
|
||||
p, base = r["path"], r["base"]
|
||||
stem = os.path.splitext(base)[0]
|
||||
d = os.path.dirname(p)
|
||||
if r["ext"] in SUB_EXT or r["ext"] in NFO_EXT or r["ext"] in IMG_EXT:
|
||||
a = anchor_of(p, stem)
|
||||
if a:
|
||||
book(r, os.path.dirname(a[1]), sidecar_name(r, a[0], a[1]),
|
||||
"subtitle" if r["ext"] in SUB_EXT else
|
||||
("nfo" if r["ext"] in NFO_EXT else "image"))
|
||||
continue
|
||||
if SERIES_META.match(base):
|
||||
owner = series_name_at(d)
|
||||
if owner:
|
||||
book(r, series_dir(owner), base, "meta")
|
||||
continue
|
||||
elif SEASON_META.match(base):
|
||||
owner, season = series_name_at(d), season_at(d)
|
||||
if owner and season is not None:
|
||||
book(r, f"{series_dir(owner)}/Season {season:02d}", base, "meta")
|
||||
continue
|
||||
elif r["ext"] in IMG_EXT or r["ext"] in NFO_EXT:
|
||||
dest = folder_dst(d)
|
||||
if dest:
|
||||
book(r, dest, base, "meta")
|
||||
continue
|
||||
unknown.append((r, "not media, or media with nothing beside it to belong to"))
|
||||
|
||||
for r, reason in unknown:
|
||||
park(r, reason)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------- output
|
||||
def write_plan():
|
||||
"""mkdir rows for every destination folder and its whole chain, then the moves.
|
||||
Parents first: the executor creates directories in the order they are listed."""
|
||||
chain = set()
|
||||
for _s, dst, _k in moves:
|
||||
d = os.path.dirname(dst)
|
||||
while d and d not in (ROOT, MOVIES, PARK, os.path.dirname(d)):
|
||||
chain.add(d)
|
||||
d = os.path.dirname(d)
|
||||
dirs = sorted(chain, key=lambda p: (p.count("/"), p))
|
||||
with open(ARGS.out, "w") as fh:
|
||||
fh.write("action\tsrc\tdst\tkind\n")
|
||||
for d in dirs:
|
||||
fh.write(f"mkdir\t\t{d}\tdir\n")
|
||||
for src, dst, kind in sorted(moves, key=lambda m: (m[1], m[0])):
|
||||
fh.write(f"move\t{src}\t{dst}\t{kind}\n")
|
||||
for src, dst, kind in sorted(keeps, key=lambda m: (m[1], m[0])):
|
||||
fh.write(f"keep\t{src}\t{dst}\t{kind}\n")
|
||||
sized = sum(held[s]["bytes"] for s, _d, _k in moves if s in held)
|
||||
print(f"plan: {len(moves) - len(parked)} moves, {len(parked)} parked"
|
||||
f" ({sized / 2**40:.2f} TiB touched), {len(keeps)} already placed,"
|
||||
f" {len(dirs)} folders -> {ARGS.out}")
|
||||
|
||||
|
||||
def write_exceptions():
|
||||
no_year = sorted(s["name"] for s in series.values() if not s["year"])
|
||||
with open(ARGS.exceptions, "w") as fh:
|
||||
fh.write(f"TV relayout exceptions for {ROOT}\n\n")
|
||||
fh.write(f"Everything below is parked under {PARK}, keeping the relative path it\n")
|
||||
fh.write("has today. Nothing is deleted and nothing is overwritten: a parked file\n")
|
||||
fh.write("is one rename from wherever the plan is pointed next.\n\n")
|
||||
for reason in sorted(reasons, key=lambda r: -len(reasons[r])):
|
||||
rs = reasons[reason]
|
||||
fh.write(f"{len(rs):>6} files {reason}\n")
|
||||
for r in sorted(rs, key=lambda r: rel(r["path"]))[:400]:
|
||||
fh.write(f" {rel(r['path'])}\n")
|
||||
if len(rs) > 400:
|
||||
fh.write(f" … and {len(rs) - 400} more\n")
|
||||
fh.write("\n")
|
||||
if no_year:
|
||||
fh.write(f"{len(no_year):>6} series carry no year from any provider, so their\n")
|
||||
fh.write(" folders are named without one; Jellyfin still matches them.\n")
|
||||
fh.write(" " + ", ".join(no_year[:300]) + "\n")
|
||||
print(f"exceptions -> {ARGS.exceptions}")
|
||||
|
||||
|
||||
write_plan()
|
||||
write_exceptions()
|
||||
+20
-4
@@ -275,7 +275,14 @@ def is_extra(path, title="", dur=0.0, root=ROOT, matched=False):
|
||||
bonus_folder = False
|
||||
for folder in parts[head:-1]:
|
||||
if folder_is_bonus(folder):
|
||||
bonus_folder = not mentions(name_words(name), name_words(folder))
|
||||
# The folder's words are often the film's own name ("Angel's Egg
|
||||
# Extras/"), and then they are not bonus evidence. Only WORDS can name
|
||||
# a film: "extras" is release junk that name_words drops, so a folder
|
||||
# like "01. Extras" reduces to its index — and an index matching an
|
||||
# index ("NCOP 01") is a coincidence, not a title.
|
||||
nw = [w for w in name_words(name) if not w.isdigit()]
|
||||
fw = [w for w in name_words(folder) if not w.isdigit()]
|
||||
bonus_folder = not fw or not mentions(nw, fw)
|
||||
break
|
||||
if dur and dur > SHORT_MAX:
|
||||
# Runtime vetoes vocabulary, with ONE exception: an explicit bonus folder
|
||||
@@ -322,13 +329,22 @@ def key_of(title):
|
||||
|
||||
|
||||
def ep_of(path):
|
||||
"""(show, episode-token) when the basename carries SxxEyy / 1x04 / Part N."""
|
||||
"""(series, episode-token) when the basename carries SxxEyy / 1x04 / Part N.
|
||||
|
||||
The series is the name BEFORE the marker. The box-set convention writes
|
||||
"Monk - S05E03 - Mr. Monk and the Big Game", and the episode title that
|
||||
follows the marker is not the show's name: keying on the whole title gave
|
||||
every episode of one series its own group, so a 124-episode box set never
|
||||
reached the size that calls something a series and each episode was filed
|
||||
as its own film. A file named only by its marker has nothing before it, so
|
||||
its whole title stands in."""
|
||||
name = os.path.splitext(os.path.basename(path))[0]
|
||||
m = EP_RE.search(name)
|
||||
if not m:
|
||||
return None
|
||||
show, _ = title_of(path)
|
||||
return show, norm(m.group(1)).replace(" ", "")
|
||||
head = name[:m.start()].strip(" ._\u2013-")
|
||||
series = title_of(head)[0] if head else None
|
||||
return series or title_of(path)[0], norm(m.group(1)).replace(" ", "")
|
||||
|
||||
|
||||
def base_words(title, key):
|
||||
|
||||
Executable
+281
@@ -0,0 +1,281 @@
|
||||
#!/usr/bin/env python3
|
||||
"""media-tv.py — move TV episodes out of the movie library into a TV library.
|
||||
|
||||
`scripts/media-plan.py` refuses to file TV: it parks episode files in a
|
||||
`_TV needs a TV library - <series>` pile for a human. This is the other half of
|
||||
that refusal. It walks the movie library, groups the files whose own name
|
||||
carries an episode marker by SERIES, and writes a plan that places each series
|
||||
in the shape both Jellyfin and Plex read without help:
|
||||
|
||||
<library>/<Series> (<year>)/Season <NN>/<the file's own name>.<ext>
|
||||
|
||||
It writes a plan and nothing else. Every rename belongs to
|
||||
`scripts/media-apply.py` — the write-ahead journal, the refusal to overwrite,
|
||||
the all-or-nothing preflight, the rollback. Apply it with `--root` set to the
|
||||
share that holds BOTH libraries (`/mnt/entropy`, not `/mnt/entropy/Movies`) so
|
||||
the executor's own path guard still has teeth, and give it its own journal —
|
||||
one journal is one undo, and a TV move must not be welded to a movie relayout:
|
||||
|
||||
python3 scripts/media-tv.py --root /mnt/entropy/Movies \
|
||||
--library /mnt/entropy/TVShows --out /mnt/entropy/media-tv-plan.tsv
|
||||
python3 scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv \
|
||||
--root /mnt/entropy --journal /mnt/entropy/media-relayout-tv.jsonl
|
||||
python3 scripts/media-apply.py verify --plan /mnt/entropy/media-tv-plan.tsv \
|
||||
--root /mnt/entropy --journal /mnt/entropy/media-relayout-tv.jsonl
|
||||
python3 scripts/media-apply.py sweep --root /mnt/entropy \
|
||||
--journal /mnt/entropy/media-relayout-tv.jsonl --apply
|
||||
|
||||
Guards, and why each one is here:
|
||||
|
||||
* The same five-file threshold as the planner. An episode-shaped name on one
|
||||
file is not a series — `The Godfather Part 2` is a film — so a smaller group
|
||||
is reported and left exactly where it is.
|
||||
* An episode whose name carries no season number is reported and left, never
|
||||
filed into an invented `Season 01`.
|
||||
* A sidecar that shares an episode's own name (`.srt`, `.nfo`, …) follows the
|
||||
episode; an episode without its subtitles is a half-move.
|
||||
* A group that would claim one destination twice is not resolved here. The
|
||||
executor refuses the whole plan before its first rename, which is the right
|
||||
outcome: nobody but Frank decides which copy of an episode is the keeper.
|
||||
* An episode the destination library already holds at the same byte count is
|
||||
left where it is. The TV library has held a complete Monk box set for years
|
||||
while the movie library carried a second copy of all 124 episodes; a mover
|
||||
that did not look where it was filing would have put both inside one library.
|
||||
* A symlink is never followed and never moved.
|
||||
"""
|
||||
import argparse
|
||||
import collections
|
||||
import importlib.util
|
||||
import os
|
||||
import re
|
||||
|
||||
|
||||
def parse_args():
|
||||
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
ap.add_argument("--root", default=os.environ.get("MEDIA_ROOT", "/mnt/entropy/Movies"),
|
||||
help="the movie library to empty of TV")
|
||||
ap.add_argument("--library", default="/mnt/entropy/TVShows",
|
||||
help="the TV library to fill (a different share or folder)")
|
||||
ap.add_argument("--out", default=None,
|
||||
help="plan TSV for scripts/media-apply.py"
|
||||
" (default: media-tv-plan.tsv beside --root)")
|
||||
ap.add_argument("--min", type=int, default=5,
|
||||
help="episode files that make something a series")
|
||||
a = ap.parse_args()
|
||||
if not a.out:
|
||||
a.out = os.path.join(os.path.dirname(os.path.abspath(a.root)),
|
||||
"media-tv-plan.tsv")
|
||||
return a
|
||||
|
||||
|
||||
ARGS = parse_args()
|
||||
ENGINE = os.path.join(os.path.dirname(os.path.abspath(__file__)), "media-titles.py")
|
||||
spec = importlib.util.spec_from_file_location("eng", ENGINE)
|
||||
eng = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(eng)
|
||||
|
||||
BAD_CHARS = re.compile(r'[<>:"/|?*]')
|
||||
# The season and episode, from the token media-titles.ep_of() hands back: s05e03, 1x04.
|
||||
SEASON_RE = re.compile(r"^(?:s(\d{1,2})e(\d{1,4})|(\d{1,2})x(\d{1,4}))$")
|
||||
# Sidecars that follow an episode; macOS forks and resume data do not.
|
||||
SKIP_SIDECAR = re.compile(r"^(\._|\.DS_|dl_state)")
|
||||
# A box set that stamps the show's year into every file name; the folder's year
|
||||
# must not be added on top of it.
|
||||
YEAR_TAIL = re.compile(r"[(\[]\s*(18|19|20)\d\d\s*[)\]]\s*$")
|
||||
SMALL = {"a", "an", "the", "and", "of", "for", "in", "on", "to", "with", "at",
|
||||
"by", "de", "la", "vs"}
|
||||
|
||||
|
||||
def clean(name):
|
||||
"""A name the filesystem and both matchers can read (same law as the plan)."""
|
||||
name = BAD_CHARS.sub("-", name).replace("\\", "-")
|
||||
name = re.sub(r"\s+", " ", name).strip().strip(". ").strip()
|
||||
return name[:180] or "untitled"
|
||||
|
||||
|
||||
def episode_name(path):
|
||||
"""The file's own name, kept: the publisher's spelling beats a re-derivation.
|
||||
|
||||
A dot-separated release name (`monk.s05e03.mkv`) becomes words; a name that
|
||||
already carries spaces is left exactly as it was written."""
|
||||
stem, ext = os.path.splitext(os.path.basename(path))
|
||||
if " " not in stem and stem.count(".") >= 2:
|
||||
stem = stem.replace(".", " ")
|
||||
return clean(stem) + ext.lower()
|
||||
|
||||
|
||||
def head_of(path):
|
||||
"""The publisher's series name: the text BEFORE the episode marker.
|
||||
|
||||
`Monk - S05E03 - Mr. Monk and the Big Game` is `Monk`; the episode title
|
||||
after the marker is not the show's name, and case is kept because it is the
|
||||
library's own spelling, not a normalised key."""
|
||||
name = os.path.splitext(os.path.basename(path))[0]
|
||||
m = eng.EP_RE.search(name)
|
||||
head = name[:m.start()] if m else name
|
||||
return re.sub(r"\s+", " ", re.sub(r"[._]+", " ", head)).strip(" -\u2013").strip()
|
||||
|
||||
|
||||
def title_series(head):
|
||||
"""The series folder in the case a TV library is written in.
|
||||
|
||||
Only a name with no capital at all is retitled: a box set that was
|
||||
lower-cased on the way in would otherwise file `monk (2002)` among
|
||||
`Dexter's Laboratory (1996)`. A name the publisher wrote with case is left
|
||||
exactly as it was written."""
|
||||
if any(c.isupper() for c in head):
|
||||
return head # the publisher wrote it with case; keep it
|
||||
words = head.split()
|
||||
return " ".join(w if i and w.lower() in SMALL else w.capitalize()
|
||||
for i, w in enumerate(words))
|
||||
|
||||
|
||||
def season_episode(token):
|
||||
"""(season, episode) an episode token states, or None when it states none."""
|
||||
m = SEASON_RE.match(token)
|
||||
if not m:
|
||||
return None
|
||||
return int(m.group(1) or m.group(3)), int(m.group(2) or m.group(4))
|
||||
|
||||
|
||||
def sidecars(path):
|
||||
"""The files beside PATH that carry its exact stem — its subtitles, its nfo."""
|
||||
d, stem = os.path.split(path)
|
||||
stem = os.path.splitext(stem)[0]
|
||||
out = []
|
||||
try:
|
||||
entries = sorted(os.listdir(d))
|
||||
except OSError:
|
||||
return out
|
||||
for e in entries:
|
||||
p = os.path.join(d, e)
|
||||
if p == path or os.path.islink(p) or not os.path.isfile(p):
|
||||
continue
|
||||
s, _ext = os.path.splitext(e)
|
||||
if s == stem and not SKIP_SIDECAR.match(e):
|
||||
out.append(p)
|
||||
return out
|
||||
|
||||
|
||||
def episodes_under(root):
|
||||
"""(path, series, token) for every episode-shaped video file under root."""
|
||||
for dirpath, dirnames, filenames in os.walk(root):
|
||||
dirnames[:] = sorted(d for d in dirnames
|
||||
if not d.startswith(".") and d != "@eaDir")
|
||||
for f in filenames:
|
||||
p = os.path.join(dirpath, f)
|
||||
if os.path.islink(p) or not f.lower().endswith(eng.VIDEO_SUFFIX):
|
||||
continue
|
||||
e = eng.ep_of(p)
|
||||
if e:
|
||||
yield p, e[0], e[1]
|
||||
|
||||
|
||||
def walk(root, library):
|
||||
"""Episode files under ROOT keyed by series, ignoring the destination."""
|
||||
groups = collections.defaultdict(list)
|
||||
lib = os.path.abspath(library)
|
||||
for p, series, token in episodes_under(root):
|
||||
if os.path.abspath(p).startswith(lib + os.sep):
|
||||
continue
|
||||
groups[series].append((p, token))
|
||||
return groups
|
||||
|
||||
|
||||
def holdings(library):
|
||||
"""What the destination already carries: (series, season, episode) -> sizes.
|
||||
|
||||
The one expensive read here, and the one that matters. The movie library
|
||||
holds a complete Monk box set that the TV library has held for years: 124
|
||||
episodes, the same episode list, identical byte counts. Moving it would
|
||||
have filed a second copy of a whole show into the library that already had
|
||||
it. Byte count is the test on purpose — two encodes of one episode are two
|
||||
files and both are wanted; the same bytes are the same file."""
|
||||
out = collections.defaultdict(set)
|
||||
for p, series, token in episodes_under(library):
|
||||
se = season_episode(token)
|
||||
if se is None:
|
||||
continue
|
||||
try:
|
||||
out[(series, *se)].add(os.path.getsize(p))
|
||||
except OSError:
|
||||
continue
|
||||
return out
|
||||
|
||||
|
||||
def main():
|
||||
root, library = os.path.abspath(ARGS.root), os.path.abspath(ARGS.library)
|
||||
groups = walk(root, library)
|
||||
held_here = holdings(library)
|
||||
mkdirs = {library}
|
||||
rows, lines, skipped = set(), [], []
|
||||
for series, items in sorted(groups.items()):
|
||||
if len(items) < ARGS.min:
|
||||
skipped.append(f"skipped: {head_of(items[0][0]) or series} — {len(items)} "
|
||||
f"episode-shaped file{'s' if len(items) != 1 else ''},"
|
||||
f" below {ARGS.min}: not a series")
|
||||
continue
|
||||
years = collections.Counter(y for y in (eng.folder_year(p) for p, _t in items)
|
||||
if y)
|
||||
year = years.most_common(1)[0][0] if years else None
|
||||
heads = collections.Counter(head_of(p) for p, _t in items)
|
||||
head = heads.most_common(1)[0][0]
|
||||
if year and YEAR_TAIL.search(head):
|
||||
year = None
|
||||
name = clean(f"{title_series(head)}{f' ({year})' if year else ''}")
|
||||
series_dir = os.path.join(library, name)
|
||||
moved = held = 0
|
||||
for path, token in items:
|
||||
se = season_episode(token)
|
||||
if se is None:
|
||||
skipped.append(f"skipped: {path} — no season number in its name,"
|
||||
" nothing here invents one")
|
||||
continue
|
||||
try:
|
||||
size = os.path.getsize(path)
|
||||
except OSError:
|
||||
continue
|
||||
if size in held_here.get((series, *se), ()):
|
||||
held += 1
|
||||
continue
|
||||
dst_dir = os.path.join(series_dir, f"Season {se[0]:02d}")
|
||||
mkdirs.update([series_dir, dst_dir])
|
||||
base = episode_name(path)
|
||||
rows.add((path, os.path.join(dst_dir, base)))
|
||||
moved += 1
|
||||
for side in sidecars(path):
|
||||
rows.add((side, os.path.join(
|
||||
dst_dir, clean(os.path.splitext(base)[0]
|
||||
+ os.path.splitext(side)[1].lower()))))
|
||||
if moved:
|
||||
lines.append(f"{name}: {moved} episodes"
|
||||
f"{'' if year else ' (no year in any folder name)'}"
|
||||
f" -> {series_dir}")
|
||||
if held:
|
||||
lines.append(f"{name}: {held}"
|
||||
f" {'episode' if held == 1 else 'episodes'} already in"
|
||||
f" {library} at the same byte count — left there")
|
||||
|
||||
# An empty plan creates nothing, not even the library folder: a run that
|
||||
# found no series must not be able to leave a stray directory behind.
|
||||
dirs = sorted(mkdirs, key=lambda p: (p.count(os.sep), p)) if rows else []
|
||||
with open(ARGS.out, "w") as out:
|
||||
out.write("action\tsrc\tdst\tkind\n")
|
||||
for d in dirs:
|
||||
out.write(f"mkdir\t\t{d}\tdir\n")
|
||||
for src, dst in sorted(rows):
|
||||
out.write(f"move\t{src}\t{dst}\ttv\n")
|
||||
|
||||
for line in lines + skipped:
|
||||
print(line)
|
||||
print(f"plan: {len(rows)} moves, {len(dirs)} folders -> {ARGS.out}")
|
||||
if rows:
|
||||
print(f"next: python3 scripts/media-apply.py apply --plan {ARGS.out}"
|
||||
f" --root {os.path.dirname(root)}"
|
||||
f" --journal {os.path.join(os.path.dirname(root),'media-relayout-tv.jsonl')}"
|
||||
" (add --apply to run it)")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Executable
+241
@@ -0,0 +1,241 @@
|
||||
#!/usr/bin/env bash
|
||||
# The SSD-backed metadata view that lets Jellyfin and Plex share one metadata set.
|
||||
#
|
||||
# Both servers read a Kodi-format sidecar (movie.nfo / tvshow.nfo / episode.nfo)
|
||||
# ONLY when it sits next to the media file. The media lives on the RAID6 array at
|
||||
# /mnt/entropy — five spinning disks, deliberately mounted read-only for every
|
||||
# container, and slow to seek for a thousand tiny XML reads. So neither media
|
||||
# server binds the array any more: they bind an OverlayFS merge of it with a
|
||||
# writable layer on the NVMe.
|
||||
#
|
||||
# lowerdir /mnt/entropy RAID6 (ROTA=1) — source of truth, never written
|
||||
# upperdir /var/lib/media-meta/upper NVMe — every sidecar and image byte
|
||||
# workdir /var/lib/media-meta/work NVMe — kernel bookkeeping
|
||||
# Where /var/lib/media-meta/view the merged tree jellyfin and plex bind
|
||||
#
|
||||
# Everything the view needs — layers, bookkeeping, trash, the merged door — lives
|
||||
# under the one directory /var/lib/media-meta, so nothing new appears under /mnt.
|
||||
#
|
||||
# Consequences worth knowing:
|
||||
# * A sidecar write is a few KB onto the SSD; the media bytes never move and the
|
||||
# array is still never written by anything automated.
|
||||
# * Deleting through the view writes a whiteout into the upper layer instead of
|
||||
# deleting from the array. Nothing should delete through the view: deletions
|
||||
# go to .AGENTTRASH on the RAW path, and `reap` clears the resulting ghosts.
|
||||
# * inotify events on the lower layer do not cross the merge, so a file that
|
||||
# arrives on the raw array is invisible to a file-system watcher in the view:
|
||||
# trigger a library refresh (see the Media Relayout page).
|
||||
#
|
||||
# Subcommands
|
||||
# plan [--upper-base D] [--lower D] print the resolved shape, touch nothing
|
||||
# verify prove the live mount IS that shape
|
||||
# install [--upper-base D] create the layers, install+enable+start the unit
|
||||
# reap [--apply] [--upper-base D] [--lower D]
|
||||
# list upper-layer entries whose media is gone
|
||||
# status one line per fact a human wants
|
||||
#
|
||||
# install must run as root (it writes /etc/systemd/system and mounts); the others
|
||||
# are read-only and safe unprivileged.
|
||||
set -euo pipefail
|
||||
|
||||
LOWER=/mnt/entropy
|
||||
UPPER_BASE=/var/lib/media-meta
|
||||
VIEW=$UPPER_BASE/view
|
||||
# systemd names a mount unit after its ESCAPED path ('/' -> '-', a literal '-'
|
||||
# -> '\x2d'), so the unit on the host is var-lib-media\x2dmeta-view.mount. The
|
||||
# repo copy keeps a readable name; install writes it under the escaped one.
|
||||
UNIT='var-lib-media\x2dmeta-view.mount'
|
||||
UNIT_DIR=/etc/systemd/system
|
||||
# The rootless containers write as this host uid (the deploy user). The upper
|
||||
# layer has to accept those writes, so it is owned by them.
|
||||
MEDIA_UID="${MEDIA_VIEW_UID:-1000}"
|
||||
MEDIA_GID="${MEDIA_VIEW_GID:-1000}"
|
||||
|
||||
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
UNIT_SRC="$REPO_ROOT/dankserver/host/systemd/media-meta-view.mount"
|
||||
|
||||
die() {
|
||||
printf 'media-view: %s\n' "$*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
usage() {
|
||||
printf 'usage: %s plan|verify|install|reap|status [options]\n' "$0" >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
cmd="${1:-}"
|
||||
[ -n "$cmd" ] || usage
|
||||
shift || true
|
||||
|
||||
upper_base="$UPPER_BASE"
|
||||
lower="$LOWER"
|
||||
apply=0
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--upper-base) [ $# -ge 2 ] || usage; upper_base="$2"; shift 2 ;;
|
||||
--lower) [ $# -ge 2 ] || usage; lower="$2"; shift 2 ;;
|
||||
--apply) apply=1; shift ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
done
|
||||
|
||||
upper="$upper_base/upper"
|
||||
work="$upper_base/work"
|
||||
|
||||
# The device backing a path, and whether the kernel calls it rotational.
|
||||
rotational_of() {
|
||||
local src rota
|
||||
src="$(findmnt -no SOURCE --target "$1" 2>/dev/null || true)"
|
||||
[ -n "$src" ] || die "cannot find a block device behind $1"
|
||||
rota="$(lsblk -no ROTA "$src" 2>/dev/null | head -1 || true)"
|
||||
[ -n "$rota" ] || die "cannot read the rotational flag of $src"
|
||||
printf '%s %s\n' "$src" "$rota"
|
||||
}
|
||||
|
||||
require_upper_base() {
|
||||
[ -d "$upper_base" ] \
|
||||
|| die "refused: upper base $upper_base does not exist (put it on the SSD first)"
|
||||
}
|
||||
|
||||
do_plan() {
|
||||
printf 'lowerdir=%s\nupperdir=%s\nworkdir=%s\nmountpoint=%s\nunit=%s/%s\n' \
|
||||
"$lower" "$upper" "$work" "$VIEW" "$UNIT_DIR" "$UNIT"
|
||||
}
|
||||
|
||||
do_verify() {
|
||||
local fail=0 fst opts src rota
|
||||
if ! mountpoint -q "$VIEW"; then
|
||||
printf 'refused: %s is not a mountpoint — a plain directory here is a\n' "$VIEW" >&2
|
||||
printf ' library that scans itself empty, not a metadata view.\n' >&2
|
||||
return 1
|
||||
fi
|
||||
fst="$(findmnt -no FSTYPE --mountpoint "$VIEW" 2>/dev/null || true)"
|
||||
if [ "$fst" != overlay ]; then
|
||||
printf 'refused: %s is mounted %s, not overlay\n' "$VIEW" "${fst:-nothing}" >&2
|
||||
fail=1
|
||||
fi
|
||||
opts="$(findmnt -no OPTIONS --mountpoint "$VIEW" 2>/dev/null || true)"
|
||||
local want
|
||||
for want in "lowerdir=$LOWER" "upperdir=$UPPER_BASE/upper" "workdir=$UPPER_BASE/work"; do
|
||||
case ",$opts," in
|
||||
*",${want},"*) ;;
|
||||
*) printf 'refused: %s options lack %s (got: %s)\n' "$VIEW" "$want" "$opts" >&2
|
||||
fail=1 ;;
|
||||
esac
|
||||
done
|
||||
read -r src rota <<<"$(rotational_of "$UPPER_BASE")"
|
||||
if [ "$rota" != 0 ]; then
|
||||
printf 'refused: the upper layer sits on %s which is rotational — metadata\n' "$src" >&2
|
||||
printf ' belongs on the SSD, not on the array.\n' >&2
|
||||
fail=1
|
||||
fi
|
||||
if [ ! -w "$UPPER_BASE/upper" ]; then
|
||||
printf 'refused: %s is not writable\n' "$upper" >&2
|
||||
fail=1
|
||||
fi
|
||||
[ "$fail" -eq 0 ] || return 1
|
||||
printf 'view ok: %s <- overlay(%s,%s)\n' "$VIEW" "$LOWER" "$UPPER_BASE/upper"
|
||||
}
|
||||
|
||||
do_install() {
|
||||
[ "$(id -u)" -eq 0 ] || die "install must run as root (writes $UNIT_DIR and mounts)"
|
||||
require_upper_base
|
||||
mountpoint -q "$LOWER" \
|
||||
|| die "refused: $LOWER is not mounted; overlaying an empty directory would present an empty library"
|
||||
local src rota
|
||||
read -r src rota <<<"$(rotational_of "$upper_base")"
|
||||
[ "$rota" = 0 ] || die "refused: $upper_base is on $src, a rotational device — the metadata layer must be on the SSD"
|
||||
[ -f "$UNIT_SRC" ] || die "refused: $UNIT_SRC is missing from $REPO_ROOT"
|
||||
# systemd will not create a mount point; a missing directory fails the start.
|
||||
install -d -m 0755 -o "$MEDIA_UID" -g "$MEDIA_GID" "$upper" "$work" "$VIEW"
|
||||
install -d -m 0700 "$upper_base/.AGENTTRASH"
|
||||
install -m 0644 "$UNIT_SRC" "$UNIT_DIR/$UNIT"
|
||||
systemctl daemon-reload
|
||||
systemctl enable "$UNIT" >/dev/null 2>&1 || true
|
||||
systemctl start "$UNIT"
|
||||
do_verify
|
||||
}
|
||||
|
||||
# An upper-layer DIRECTORY with no counterpart below is metadata for a media
|
||||
# folder that has been moved or renamed on the RAW path — a ghost the media
|
||||
# servers would still browse. Files are never candidates on their own: a sidecar
|
||||
# (movie.nfo, an .srt) exists ONLY above, by design, and is exactly what the view
|
||||
# is for. A copy-up whose lower file was deleted by hand is the one case this
|
||||
# leaves in place; the raw path is where deletions belong, so it stays visible.
|
||||
do_reap() {
|
||||
[ -d "$upper" ] || die "refused: $upper does not exist (no upper layer at $upper_base)"
|
||||
stamp="$upper_base/.AGENTTRASH/$(date -u +%Y%m%d-%H%M%S)"
|
||||
count=0
|
||||
ghosts=0
|
||||
reap_walk ""
|
||||
if [ "$count" -eq 0 ] && [ "$ghosts" -eq 0 ]; then
|
||||
printf 'reap: no ghosts under %s\n' "$upper"
|
||||
return 0
|
||||
fi
|
||||
if [ "$ghosts" -gt 0 ]; then
|
||||
printf 'reap: %d whiteout%s above — someone deleted THROUGH the view.\n' \
|
||||
"$ghosts" "$([ "$ghosts" -eq 1 ] || printf s)" >&2
|
||||
printf ' Leaving them: removing a whiteout un-hides the file it hides.\n' >&2
|
||||
printf ' Deletions belong in .AGENTTRASH on the raw path.\n' >&2
|
||||
fi
|
||||
if [ "$count" -eq 0 ]; then return 0; fi
|
||||
if [ "$apply" -eq 1 ]; then
|
||||
printf 'reap: moved %d entr%s to %s\n' "$count" \
|
||||
"$([ "$count" -eq 1 ] && printf y || printf ies)" "$stamp"
|
||||
else
|
||||
printf 'reap: %d ghost entr%s above; re-run with --apply to move them to %s\n' \
|
||||
"$count" "$([ "$count" -eq 1 ] && printf y || printf ies)" "$stamp"
|
||||
fi
|
||||
}
|
||||
|
||||
# Descend only where the lower layer still has the directory: a directory that is
|
||||
# missing below is itself the ghost, and its whole subtree goes with it.
|
||||
reap_walk() {
|
||||
local rel="$1" entries abs r name
|
||||
entries="$(find "$upper${rel:+/$rel}" -mindepth 1 -maxdepth 1 -print 2>/dev/null || true)"
|
||||
[ -n "$entries" ] || return 0
|
||||
while IFS= read -r abs; do
|
||||
r="${abs#"$upper"/}"
|
||||
name="${abs##*/}"
|
||||
if [ -c "$abs" ] || printf '%s' "$name" | grep -q '^\.wh\.'; then
|
||||
ghosts=$((ghosts + 1))
|
||||
continue
|
||||
fi
|
||||
if [ -e "$lower/$r" ]; then
|
||||
[ -d "$abs" ] && reap_walk "$r"
|
||||
continue
|
||||
fi
|
||||
[ -d "$abs" ] || continue
|
||||
count=$((count + 1))
|
||||
printf '%s\n' "$r"
|
||||
if [ "$apply" -eq 1 ]; then
|
||||
mkdir -p "$stamp/$(dirname "$r")"
|
||||
mv "$abs" "$stamp/$r"
|
||||
fi
|
||||
done <<<"$entries"
|
||||
}
|
||||
|
||||
do_status() {
|
||||
if mountpoint -q "$VIEW"; then
|
||||
printf 'view: %s (%s)\n' "$VIEW" "$(findmnt -no OPTIONS --mountpoint "$VIEW")"
|
||||
else
|
||||
printf 'view: %s NOT MOUNTED\n' "$VIEW"
|
||||
fi
|
||||
if [ -d "$upper" ]; then
|
||||
printf 'upper: %s %s files %s\n' "$upper" \
|
||||
"$(find "$upper" -type f 2>/dev/null | wc -l)" "$(du -sh "$upper" 2>/dev/null | cut -f1)"
|
||||
else
|
||||
printf 'upper: %s missing\n' "$upper"
|
||||
fi
|
||||
systemctl is-active "$UNIT" 2>/dev/null | sed 's/^/unit: /' || printf 'unit: %s unknown\n' "$UNIT"
|
||||
}
|
||||
|
||||
case "$cmd" in
|
||||
plan) do_plan ;;
|
||||
verify) do_verify ;;
|
||||
install) do_install ;;
|
||||
reap) do_reap ;;
|
||||
status) do_status ;;
|
||||
*) usage ;;
|
||||
esac
|
||||
@@ -94,6 +94,16 @@ for path in "${CHANGED[@]}"; do
|
||||
dankserver/.env.example|dankclaw/.env.example)
|
||||
add_test tests/env_consistency.bats
|
||||
;;
|
||||
# The SSD-backed metadata view: the installer, the host mount unit, and the
|
||||
# two media units that bind the view rather than the raw array.
|
||||
scripts/media-view.sh|dankserver/host/*)
|
||||
add_test tests/media_overlay.bats
|
||||
add_test tests/dankserver_quadlet.bats
|
||||
add_test tests/deployment_scoping.bats
|
||||
;;
|
||||
dankserver/jellyfin/quadlet/*|dankserver/plex/quadlet/*)
|
||||
add_test tests/media_overlay.bats
|
||||
;;
|
||||
shared/homepage/*)
|
||||
add_test tests/homepage_caddy.bats
|
||||
add_test tests/homepage_config.bats
|
||||
|
||||
@@ -195,8 +195,9 @@ deploy.sh, never a live env poke.
|
||||
check every namespace (`sudo podman ps -a`) and native units
|
||||
(`systemctl list-units`) before declaring a service absent. Compare
|
||||
compose mount sources against the real filesystem — compose files name
|
||||
phantom paths (e.g. `/mnt/media/*` vs the real LUKS array at
|
||||
`/mnt/entropy`). Fix compose through git first, deploy second.
|
||||
phantom paths (the legacy `dankserver/compose.yaml` still names `/opt/jellyfin/*`
|
||||
for a service whose roots moved to `/home/dank/Containers/jellyfin`). Fix
|
||||
compose through git first, deploy second.
|
||||
2. **Pre-create bind-mount sources** (`sudo mkdir -p` + `chown 1000:1000`)
|
||||
or creation fails `statfs …: no such file or directory`.
|
||||
3. **Staged bring-up:** infra first (`up -d mariadb redis`), poll
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
---
|
||||
type: reference
|
||||
title: Jellyfin
|
||||
description: "Jellyfin on Dankserver: quadlet unit, entropy-volume data layout, ports, client access, hardware-transcode status."
|
||||
description: "Jellyfin on Dankserver: quadlet unit, overlay metadata view, provider keys, ports, access, transcode status."
|
||||
tags: [jellyfin, dankserver, media-server]
|
||||
timestamp: 2026-09-03T00:00:00Z
|
||||
timestamp: 2026-10-30T00:00:00Z
|
||||
---
|
||||
|
||||
# Jellyfin
|
||||
@@ -26,7 +26,7 @@ public vhost `jelly.franksdank.cloud` behind the [[Anubis]] guard chain →
|
||||
| Image | `ghcr.io/jellyfin/jellyfin:latest` |
|
||||
| Network | `dankserver_appnet`; publishes 8096/tcp, 8920/tcp, 7359/udp, 1900/udp |
|
||||
| Data roots | `/home/dank/Containers/jellyfin/config:/config`, `/home/dank/Containers/jellyfin/cache:/cache` |
|
||||
| Media | `Volume=/mnt/entropy:/data:ro` — the whole [[Entropy Volume]] mounted read-only; nothing automated writes to it |
|
||||
| Media | `Volume=/var/lib/media-meta/view:/data` — the [[Media Metadata View]] overlay, **writable**; sidecars land on the NVMe, the [[Entropy Volume]] array underneath stays read-only and untouched |
|
||||
| GPU | none — no `AddDevice`; CPU decode only (jellyfin-ffmpeg path) |
|
||||
| Env quirks | `DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true`, `LD_PRELOAD=libjemalloc.so.2`, `MALLOC_TRIM_THRESHOLD_=131072` kept verbatim from the compose-era container (zero-drift cutover) |
|
||||
| Health | `curl -f http://localhost:8096/health` every 30 s |
|
||||
@@ -36,11 +36,23 @@ public vhost `jelly.franksdank.cloud` behind the [[Anubis]] guard chain →
|
||||
```
|
||||
/home/dank/Containers/jellyfin/config -> /config (JELLYFIN_DATA_DIR)
|
||||
/home/dank/Containers/jellyfin/cache -> /cache (JELLYFIN_CACHE_DIR, XDG_CACHE_HOME)
|
||||
/mnt/entropy -> /data (read-only media volume)
|
||||
/var/lib/media-meta/view -> /data (the SSD-backed overlay view)
|
||||
```
|
||||
|
||||
Library folders point at `/data` paths; because the mount uses the same
|
||||
absolute path as on the host, library paths survive container recreation.
|
||||
Library folders point at `/data` paths, which are the view's paths: stable
|
||||
across container recreation, and identical to the raw array layout below it.
|
||||
|
||||
`Local metadata` — save metadata to media folders — is enabled, and it works
|
||||
because `/data` is the [[Media Metadata View]] rather than the read-only array:
|
||||
`movie.nfo`, `tvshow.nfo`, `episode.nfo` and artwork are written once, on the
|
||||
NVMe upper layer, and [[Plex]] reads the same files. Jellyfin is the fleet's
|
||||
only metadata writer.
|
||||
|
||||
**Failure signature:** `System.IO.IOException: Read-only file system :
|
||||
'/data/Movies/<film>/movie.nfo'` means the container is binding the raw array
|
||||
instead of the view — the view is unmounted or the quadlet drifted. That used to
|
||||
be the expected state (pre-view, every item update logged it); today it is a
|
||||
fault. Run `scripts/media-view.sh status`.
|
||||
|
||||
## Ports & access
|
||||
|
||||
@@ -105,6 +117,61 @@ where the LDAP plugin runs. Verified 2026-09-25: `alex` authenticates over
|
||||
`AuthenticateByName` (HTTP 200; full plugin flow logged — service bind,
|
||||
search, `ldapUid`, user bind, token).
|
||||
|
||||
## Metadata providers and their keys (2026-10-30)
|
||||
|
||||
TMDb, OMDb, StudioImages and MusicBrainz are **core plugins** in Jellyfin 12 —
|
||||
no DLL, just a configuration file the running container owns under the data
|
||||
root. TVDB is not core (its metadata provider was removed in 10.9), so it is an
|
||||
official-catalog plugin.
|
||||
|
||||
| Provider | Shape on disk | Key lives in |
|
||||
|---|---|---|
|
||||
| TMDb | `config/plugins/configurations/Jellyfin.Plugin.Tmdb.xml` → `<TmdbApiKey>` | `/home/dank/Containers/jellyfin/provider-keys` → `TMDB_API_KEY=` (mode 600) |
|
||||
| TVDB | `config/plugins/thetvdb_24.0.0.0/` + `…Jellyfin.Plugin.Tvdb.xml` → `<ApiKey>` | the same file → `TVDB_API_KEY=` |
|
||||
| OMDb | `…Jellyfin.Plugin.Omdb.xml` | **not configured** — its key is issued by email, 1 000 req/day, and its ToS forbids storing the returned data; ratings stay Jellyfin's own lookup |
|
||||
|
||||
**None of it is tracked in git.** Those XML files belong to the container and
|
||||
hold credentials, and `tests/jellyfin_providers.bats` fails the suite if one is
|
||||
ever committed. The keys themselves sit in the per-service host env file that is
|
||||
the fleet's convention for container credentials (grafana, caddy, pihole,
|
||||
copyparty, romm, seerr): `provider-keys`, mode 600, beside the data root.
|
||||
`scripts/jellyfin-providers.sh` is the only thing that writes the XML:
|
||||
|
||||
```
|
||||
scripts/jellyfin-providers.sh plan # what would change; prints no key
|
||||
scripts/jellyfin-providers.sh install # plugin + keys, previous copy to trash
|
||||
scripts/jellyfin-providers.sh verify # plugin + key LENGTHS, never key bytes
|
||||
```
|
||||
|
||||
- **A key never enters git, argv, or the tool's output.** The installer reads
|
||||
that 600-perm file itself (a value in argv is world-readable through
|
||||
`/proc/<pid>/cmdline`) and refuses it when it is group- or world-readable —
|
||||
an exposed key is an error, not a warning. A key that shows up in a transcript
|
||||
or `ps` output has leaked: rotate it with the provider, rewrite
|
||||
`provider-keys`, re-run `install`.
|
||||
- **The TVDB artifact is pinned three ways**: sha256
|
||||
`6302fba3857bdde8d186dd099fac09ca43247041f531a528a2c788add70a5319`, the
|
||||
official plugin guid `a677c0da-fac5-4cde-941a-7134223f14c8`, and version
|
||||
`24.0.0.0` (`targetAbi 12.0.0.0`, which fits the deployed 12.4.0). A sha
|
||||
match alone only proves the bytes are the bytes fetched — the guid check is
|
||||
what says it is the official plugin. Rollback: move `thetvdb_24.0.0.0/` into
|
||||
the trash and revert the pin; a second `thetvdb_*` directory is a refusal,
|
||||
never a quiet second copy.
|
||||
- **The homelab TVDB key is a licensed key** (its JWT claims say
|
||||
`community_supported: false`), so it needs **no PIN** — the optional
|
||||
`TVDB_PIN=` line exists only for a subscriber-shaped key.
|
||||
- **TVDB is installed and keyed but is not a metadata agent**: TMDb still
|
||||
identifies everything. Enabling TVDB as a provider would re-fetch show
|
||||
metadata across the library, so it is a deliberate, separate change.
|
||||
- **The deployer places the keys before it restarts jellyfin**
|
||||
(`ensure_provider_keys` in `scripts/deploy-remote.sh`) and warns instead of
|
||||
failing when `provider-keys` is absent, so a host without it still deploys.
|
||||
- **Keys and plugins are read at start-up**: after a manual `install` the
|
||||
change is inert until `./scripts/deploy.sh dankserver --service jellyfin`.
|
||||
- TMDb's terms are the reason this stays narrow: attribution required, no
|
||||
caching beyond 6 months, no derivative works. Fine for a personal homelab,
|
||||
and the reason no bulk enrichment pass runs against it.
|
||||
|
||||
## Configuration workflow
|
||||
|
||||
1. Create libraries pointing at `/data` folders (movies, shows, audio).
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
---
|
||||
type: reference
|
||||
title: Media Metadata View
|
||||
description: "The SSD-backed overlay the two media servers read and write: shape, guards, commands, laws, rollback."
|
||||
tags: [dankserver, media-server, jellyfin, plex, storage]
|
||||
timestamp: 2026-10-30T00:00:00Z
|
||||
---
|
||||
|
||||
# Media Metadata View
|
||||
|
||||
`/var/lib/media-meta/view` is an OverlayFS merge of the media library: the RAID6
|
||||
array below, a directory on the NVMe above. It exists so [[Jellyfin]] can write
|
||||
Kodi-format sidecars (`movie.nfo`, `tvshow.nfo`, `episode.nfo`, artwork) that
|
||||
[[Plex]] can read. Sidecar metadata is only useful sitting next to the media
|
||||
file, and the array is read-only and slow — so the media bytes stay there and the
|
||||
*writes* land on SSD. One tree to both servers, one copy of the media.
|
||||
|
||||
The mount point is not a volume and holds nothing while unmounted. It sits inside
|
||||
`/var/lib/media-meta` so that layers, kernel bookkeeping, the reap trash and the
|
||||
merged door are one directory — and so the mount table grows no new `/mnt` entry
|
||||
next to the array.
|
||||
|
||||
## Shape
|
||||
|
||||
| Item | Value |
|
||||
|---|---|
|
||||
| Lower layer | `/mnt/entropy` — the RAID6 array, source of truth, never written |
|
||||
| Upper layer | `/var/lib/media-meta/upper` — NVMe; every write lands here |
|
||||
| Work dir | `/var/lib/media-meta/work` — overlayfs internals, same filesystem |
|
||||
| Mount point | `/var/lib/media-meta/view` — the merged tree both servers bind |
|
||||
| Unit (repo) | `dankserver/host/systemd/media-meta-view.mount` — the source of truth |
|
||||
| Unit (host) | `var-lib-media\x2dmeta-view.mount` — systemd names a mount unit after its escaped path (`-` becomes `\x2d`); `media-view.sh install` writes the repo file under that name |
|
||||
| Installer | `scripts/media-view.sh` — `plan` / `install` / `verify` / `status` / `reap` |
|
||||
| Written by | Jellyfin only (Plex binds read-only); nothing else in the fleet touches it |
|
||||
|
||||
Both consumers guard the view with
|
||||
`UnitConfigItem=ConditionPathIsMountPoint=/var/lib/media-meta/view`, and
|
||||
`scripts/deploy.sh dankserver --service jellyfin|plex` calls `media-view.sh
|
||||
install` + `verify` before it restarts either container.
|
||||
|
||||
| Consumer | Bind | Mode | Why that path |
|
||||
|---|---|---|---|
|
||||
| [[Jellyfin]] | `/var/lib/media-meta/view:/data` | rw | It is the metadata writer; its sidecars become SSD bytes |
|
||||
| [[Plex]] | `/var/lib/media-meta/view:/mnt/entropy` | ro | Every path in the PMS database was recorded as `/mnt/entropy/...`; only the HOST side of the bind moved, so its database still resolves |
|
||||
|
||||
Putting the sidecars inside Plex's reach is not the same as making Plex use them:
|
||||
a library parses `.nfo` only with its *Local media information* agent enabled, and
|
||||
this change touches no library setting.
|
||||
|
||||
## Commands
|
||||
|
||||
```bash
|
||||
scripts/media-view.sh plan # print the resolved shape, touch nothing
|
||||
sudo scripts/media-view.sh install # create dirs, install the unit, mount, enable
|
||||
scripts/media-view.sh verify # is the view really the overlay? (deploy gate)
|
||||
scripts/media-view.sh status # mount options + upper-layer size/file count
|
||||
scripts/media-view.sh reap # list ghost metadata (media moved, sidecars left)
|
||||
sudo scripts/media-view.sh reap --apply # move ghosts to the SSD trash, never delete
|
||||
```
|
||||
|
||||
`--upper-base DIR` and `--lower DIR` relocate the layers for `plan` and `reap`;
|
||||
the defaults are the host values above.
|
||||
|
||||
## Laws
|
||||
|
||||
- **The upper layer must sit on non-rotational storage.** `install` and `verify`
|
||||
refuse via `lsblk -dno ROTA` if the upper base lives on the array: an overlay
|
||||
whose upper is the spinners is the original problem with extra steps, and it
|
||||
silently turns every metadata write back into a seek.
|
||||
- **Never mount the view over an unmounted lower.** `RequiresMountsFor=/mnt/entropy`
|
||||
plus the container `ConditionPathIsMountPoint` guards: an empty `lowerdir`
|
||||
presents an *empty library* as truth, and a media scan would then clear the
|
||||
database of everything. systemd does not create a mount point, so `install`
|
||||
creates `view` itself; without it the mount simply fails.
|
||||
- **Never delete through the view.** A delete writes an overlayfs whiteout
|
||||
(`.wh.<name>`) in the upper layer, which hides the array's copy without
|
||||
removing it — the file reappears the moment the view is unmounted, and the
|
||||
upper layer now carries a landmine. Deletions belong on the raw path
|
||||
(`/mnt/entropy`), which the relayout tool already treats as the only writer.
|
||||
`reap` refuses to remove whiteouts and only warns about them.
|
||||
- **A relayout must be followed by `reap`.** Renames happen on the raw path, so
|
||||
sidecars the view already wrote stay at the old coordinates: a ghost folder
|
||||
the servers browse and find nothing in. `reap` lists upper-layer directories
|
||||
with no counterpart below and moves them (with their subtree) into a dated
|
||||
`.AGENTTRASH/` under the SSD base. It is recursive and file-safe: a sidecar
|
||||
that exists only above is normal, so only *directories* are candidates.
|
||||
- **File-watcher events do not cross the overlay.** Plex's NFO agent and
|
||||
Jellyfin's real-time monitor see inotify events for writes that land *in the
|
||||
upper layer*, not for changes on the array underneath. After any relayout or
|
||||
externally-written metadata, trigger a library refresh — see
|
||||
[[Media Relayout]] for the refresh procedure.
|
||||
|
||||
## Failure signatures
|
||||
|
||||
| Symptom | Meaning |
|
||||
|---|---|
|
||||
| `jellyfin.service` / `plex.service` show `ConditionPathIsMountPoint=... was not met` and stay inactive | The view is not mounted. Do not start them anyway: `scripts/media-view.sh install`, then `verify`. |
|
||||
| Jellyfin log: `Read-only file system : '/data/Movies/…/movie.nfo'` | The container is binding the raw array, not the view — the quadlet drifted back or the view is gone. `media-view.sh status`. |
|
||||
| A folder appears in a media server with artwork/`.nfo` but no video | Ghost metadata from a relayout. `media-view.sh reap`. |
|
||||
| `verify` prints `refused: …/view is not a mountpoint` | A plain directory is sitting where the overlay belongs. That is the fail-closed guard working; `install` on the box. |
|
||||
| `systemctl status` finds no such unit | Look under the escaped name, `var-lib-media\x2dmeta-view.mount`, not the repo file name. |
|
||||
|
||||
## Rollback
|
||||
|
||||
Revert the commit that pointed the two quadlets at the view and re-deploy both
|
||||
services — they bind `/mnt/entropy` read-only again, which is the pre-view shape.
|
||||
The NVMe upper layer can stay: nothing reads it once the binds are back, and it
|
||||
holds no state either server needs. Unmounting alone
|
||||
(`systemctl stop 'var-lib-media\x2dmeta-view.mount'`) is safe while no container
|
||||
binds it, and the array is never modified by any of this.
|
||||
|
||||
## Links
|
||||
|
||||
- [[Jellyfin]] — the metadata writer; its data layout section.
|
||||
- [[Plex]] — the read-only consumer; in-place bind-mount history.
|
||||
- [[Media Relayout]] — the tool that moves media on the raw path and the reason `reap` exists.
|
||||
- [[Entropy Volume]] — the array underneath.
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: Media Relayout
|
||||
description: "Census, plan and apply the movie library into Jellyfin format: the gates to read, and the rollback."
|
||||
description: "Census, plan and apply the movie and TV libraries into Jellyfin format: gates, evidence chain, refusals, rollback."
|
||||
tags: [dankserver, media-server, jellyfin, plex, procedure]
|
||||
timestamp: 2026-10-30T00:00:00Z
|
||||
---
|
||||
@@ -11,17 +11,19 @@ timestamp: 2026-10-30T00:00:00Z
|
||||
The movie library on [[Entropy Volume]] arrived as release folders —
|
||||
`Some.Movie.2019.1080p.WiKiTaLx.x265-FOO/` with the video inside named after
|
||||
the release. [[Jellyfin]] matches a movie by folder and file name, so the
|
||||
release noise cost posters, extras and versions. These three scripts turn that
|
||||
library into Jellyfin's own shape, and nothing else in the repo moves a media
|
||||
file.
|
||||
release noise cost posters, extras and versions. These scripts turn that
|
||||
library into Jellyfin's own shape — the movies, the TV packs, and the TV that
|
||||
turns up in the movie share — and nothing else in the repo touches a media file.
|
||||
|
||||
| Script | Touches media? | Output |
|
||||
|---|---|---|
|
||||
| `scripts/media-census.py` | reads headers only | `enum`, `probed`, `plex` TSVs |
|
||||
| `scripts/media-plan.py` | no — TSVs in, TSVs out | the plan + the exception report |
|
||||
| `scripts/media-apply.py` | **renames, never copies**; sweeps only what a run emptied | the moves, plus a journal |
|
||||
| `scripts/media-tv.py` | no — TSV out | a plan that moves TV into the TV library |
|
||||
| `scripts/media-shows.py` | no — TSV out | a plan that re-files the TV library itself |
|
||||
|
||||
The seam is `tests/media_relayout.bats`; the target contract is quoted from
|
||||
The seams are `tests/media_relayout.bats` and `tests/media_shows.bats`; the target contract is quoted from
|
||||
[jellyfin.org/docs/general/server/media/movies](https://jellyfin.org/docs/general/server/media/movies)
|
||||
in `media-plan.py`'s docstring.
|
||||
|
||||
@@ -68,8 +70,8 @@ scripts/media-apply.py rollback --journal /mnt/entropy/media-relayout.jsonl --ro
|
||||
```
|
||||
|
||||
An apply also leaves the release folders it emptied standing where they were —
|
||||
the live run left 1675 of them at the library root — which is what `sweep`
|
||||
removes, dry-run first, from the same journal.
|
||||
hundreds of them at the library root — which is what `sweep` removes, dry-run
|
||||
first, from the same journal.
|
||||
|
||||
After it lands, both servers need a scan or the change is invisible.
|
||||
|
||||
@@ -90,29 +92,134 @@ podman exec -i plex sh -c 'read -r T
|
||||
done' </tmp/px.tok
|
||||
```
|
||||
|
||||
[[Jellyfin]] has no credential on this box that can trigger a scan: the two keys
|
||||
in its database are Seerr's, unused since 2026-09-01, and answer 401 to every
|
||||
call. Mint one (*Dashboard → Advanced → API Keys*, Administrator) and store it in
|
||||
`~/.config/secrets/jellyfin`, or press *Library → Scan All Libraries* in the web
|
||||
UI. A scripted scan passes the key in a mode-600 curl config, never in a header
|
||||
argument:
|
||||
[[Jellyfin]]'s scan key lives in `~/.config/secrets/jellyfin` (mode 600; the
|
||||
same value is `JELLYFIN_API_KEY` in the ignored `dankserver/.env`). It rides a
|
||||
mode-600 curl config, never a header argument, and it must use the
|
||||
`MediaBrowser` form — Jellyfin 12.4 refuses a bare `X-Emby-Token` for any key:
|
||||
|
||||
```bash
|
||||
umask 077; printf 'header = "X-Emby-Token: %s"\n' "$(cat ~/.config/secrets/jellyfin)" >/tmp/jf.hdr
|
||||
curl -s -K /tmp/jf.hdr -o /dev/null -w "jellyfin scan: %{http_code}\n" \
|
||||
-X POST http://127.0.0.1:8096/Library/Refresh
|
||||
umask 077
|
||||
{ printf 'header = "Authorization: MediaBrowser Token=\\"%s\\""\n' \
|
||||
"$(cat ~/.config/secrets/jellyfin)"
|
||||
echo 'url = "http://127.0.0.1:8096/Library/Refresh"'
|
||||
echo 'request = "POST"'; } >/tmp/jf.hdr
|
||||
curl -s -K /tmp/jf.hdr -o /dev/null -w "jellyfin scan: %{http_code}\n"
|
||||
shred -u /tmp/jf.hdr
|
||||
```
|
||||
|
||||
`204` is acceptance; the scan then runs on the server's own schedule.
|
||||
|
||||
## TV in the movie library
|
||||
|
||||
A box set that arrives in the movie share stays there until someone moves it:
|
||||
`media-plan.py` refuses to file TV and parks each series in
|
||||
`_TV needs a TV library - <series>`. `scripts/media-tv.py` is the mover. It
|
||||
walks the movie library itself (no census needed), groups episode-shaped names
|
||||
by series, and writes a plan in the executor's own format:
|
||||
|
||||
```bash
|
||||
scripts/media-tv.py --root /mnt/entropy/Movies --library /mnt/entropy/TVShows \
|
||||
--out /mnt/entropy/media-tv-plan.tsv
|
||||
scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv --root /mnt/entropy
|
||||
scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv --root /mnt/entropy \
|
||||
--journal /mnt/entropy/media-relayout-tv.jsonl --apply
|
||||
scripts/media-apply.py sweep --journal /mnt/entropy/media-relayout-tv.jsonl \
|
||||
--root /mnt/entropy --apply
|
||||
```
|
||||
|
||||
The shape it writes is the one both matchers read — `Monk (2002)/Season 05/
|
||||
Monk - S05E03 - Mr. Monk and the Big Game.mkv` — keeping the file's own name,
|
||||
because the publisher's spelling beats a re-derivation, with the year taken
|
||||
from the box-set folder. What it refuses it prints and leaves alone:
|
||||
|
||||
- fewer than `--min` (5) files under one series name: one file named `Part 2`
|
||||
is a film, not a series;
|
||||
- an episode whose name states no season number — no invented `Season 01`;
|
||||
- a destination two files both claim: the executor refuses the whole plan
|
||||
before its first rename, and only Frank decides which copy is the keeper.
|
||||
- an episode the destination library already holds at the same byte count. The
|
||||
movie share carried a second copy of the whole Monk box set — 124 episodes
|
||||
both sides, the same episode list, every byte count identical, 31.5 GiB each
|
||||
way, and head+tail blake2b identical on a 6-episode sample. The mover leaves
|
||||
such a series in place and says so; a duplicate is deleted or kept by Frank,
|
||||
never copied into one library by a tool.
|
||||
|
||||
A sidecar that shares an episode's own name (`.srt`, `.nfo`) moves with it.
|
||||
|
||||
Below the threshold the mover prints and leaves, so a lone downloaded episode of
|
||||
a real show stays in the movie library on purpose — six did on 2026-10-30.
|
||||
Filing one by hand is the same executor on a hand-written plan and the same
|
||||
journal; the trap is the series year. Read it from the file's own name or the
|
||||
folder it owns, never from the title whose bonus folder it happens to be parked
|
||||
in — `The Cleveland Show S03E07` inside `Die Hard (1988)/extras/` is not a 1988
|
||||
show. A name whose marker is `Part 2` is a film and stays put.
|
||||
|
||||
## The TV library's own relayout
|
||||
|
||||
`/mnt/entropy/TVShows` is scene packs rather than a library, and Jellyfin has
|
||||
already matched and numbered most of it. `scripts/media-shows.py` re-files that
|
||||
library into the same shape, and it never guesses: every series name and every
|
||||
episode number it writes traces to an authority, and what has none is parked for
|
||||
a human to read.
|
||||
|
||||
```bash
|
||||
# 1. The authority: Jellyfin's own matched rows (the DB opens read-only).
|
||||
python3 scripts/media-census.py jellyfin --root /mnt/entropy/TVShows \
|
||||
--out /tmp/tv_titles.tsv
|
||||
# 2. What is on disk (stat only), and the runtimes the film arm needs.
|
||||
python3 scripts/media-census.py enumerate --root /mnt/entropy/TVShows \
|
||||
--out /tmp/tv_enum.tsv
|
||||
python3 scripts/media-census.py probe --root /mnt/entropy/TVShows \
|
||||
--enum /tmp/tv_enum.tsv --jobs 6 --out /tmp/tv_probed.tsv
|
||||
# The probe is the slow step: ffprobe reads both ends of every file.
|
||||
# 3. The plan. --tmdb needs TMDB_API_KEY; without it a series no provider dated
|
||||
# is simply named without a year, which Jellyfin accepts.
|
||||
python3 scripts/media-shows.py --root /mnt/entropy/TVShows --enum /tmp/tv_enum.tsv \
|
||||
--titles /tmp/tv_titles.tsv --probed /tmp/tv_probed.tsv \
|
||||
--out /tmp/tv_plan.tsv --exceptions /tmp/tv_exceptions.txt
|
||||
# 4. Read the exceptions file FIRST — it is the review, not a log. Then the same
|
||||
# executor, journal and rollback as the movie relayout above.
|
||||
scripts/media-apply.py apply --plan /tmp/tv_plan.tsv --root /mnt/entropy \
|
||||
--journal /mnt/entropy/media-relayout-tv.jsonl --apply
|
||||
```
|
||||
|
||||
Evidence, strongest first — the order the planner believes things:
|
||||
|
||||
- the **episode marker the file's own name states** (`S01E02-E03`, `1x04`);
|
||||
- the **`Season NN` folder it sits in and the number its name leads with inside
|
||||
it**: `Season 19/01 Charles Barkley, Nirvana.avi` is S19E01;
|
||||
- the **provider row**: identity (the series name and its year), and the
|
||||
numbering of a file whose own name says nothing;
|
||||
- a **bonus folder** anywhere above the file, or Jellyfin's `ExtraType` → the
|
||||
series' canonical extra folder;
|
||||
- a runtime over `SHORT_MAX` makes it film-shaped, and it leaves for `--movies`
|
||||
only when a movie provider names it — otherwise it parks.
|
||||
|
||||
Two refusals carry the design, and a dry run on the real library found both:
|
||||
|
||||
- **Folders resolve a series; they never invent one.** A folder's words may
|
||||
resolve a file to a series a provider has already named — `(CBB) Tengen Toppa
|
||||
Gurren Lagann - The Complete Series (BD 1080p)` names the registered *Tengen
|
||||
Toppa Gurren Lagann* — but a bonus subfolder is not a show. An early run made
|
||||
`…/03. Extra/01. NCOP/NCOP 01.mkv` into a root series called `01 NCOP`.
|
||||
- **A scraped row does not outrank the filesystem.** Jellyfin numbers
|
||||
multi-season packs badly: every file under `SNL 16-20/Season 16..19` carries
|
||||
season 16 episode 20, and one Gundam row says season 2004. The folder, then
|
||||
the file's own leading number, win; a row that still insists on season 1989
|
||||
parks its file instead of creating `Season 1989`.
|
||||
|
||||
The provider's name is copied verbatim, warts included: Jellyfin carries a Series
|
||||
named `Black Clover 001` (92 episodes, no year) because it matched a pack off a
|
||||
filename. Renaming a show is Jellyfin's work — a folder rename never touches its
|
||||
provider ids — so the mover keeps the name it is handed and strips only the pack
|
||||
index a release leads with (`4a. Seed DESTINY` → `Seed DESTINY`).
|
||||
|
||||
## Laws
|
||||
|
||||
- **The journal lives beside the library, never in `/tmp`.** The live run's
|
||||
`/mnt/entropy/media-relayout.jsonl` is the only rollback instrument; a journal
|
||||
a reboot clears is not an instrument.
|
||||
- **An apply breaks [[Plex]] playback for every moved item until its section is
|
||||
rescanned.** The database keeps the release-folder path, so streams fail
|
||||
`Couldn't find the file to stream` while the scanner works through the section.
|
||||
Trigger the scans in the same window as the apply, and never read a 404 in that
|
||||
window as a lost file — the file is where the journal says it is.
|
||||
- **A moved file is a 404 in [[Plex]] until its section is rescanned.** The
|
||||
database keeps the release-folder path, so playback fails `Couldn't find the
|
||||
file to stream` while the scanner works through the section. Never read such a
|
||||
404 as a lost file: the file is where the journal says it is.
|
||||
- **Plex's matched title outranks the filename**, and a Plex year of `0` means
|
||||
*indexed but never matched* — trusting it named folders `Title (0)`, which is
|
||||
a Jellyfin item with no year, no poster and no identity. A year that is not a
|
||||
@@ -133,6 +240,32 @@ curl -s -K /tmp/jf.hdr -o /dev/null -w "jellyfin scan: %{http_code}\n" \
|
||||
folder holding it, and extras drifted into a neighbour. A bare number counts
|
||||
only when it could be a release year, so `1080` and `2160` never pose as
|
||||
agreement between two names.
|
||||
- **Compare two media libraries with the series engine, never a path
|
||||
substring.** Filtering paths for `monk` also returns *Monkey Dust*, and
|
||||
letting whichever file a walk happens to reach first stand for an episode
|
||||
paired two libraries into a fake duplicate — the same sloppiness pointed the
|
||||
other way deletes a file that is unique. The comparison is `media-titles.ep_of`
|
||||
on both sides: series, season, episode, bytes, and a hash before anything is
|
||||
trashed.
|
||||
- **A deletion is staged on the item's own filesystem.** The trash rule names
|
||||
`~/.AGENTTRASH`, but `/home` and `/mnt/entropy` are different devices: a 31 GiB
|
||||
stage there is a copy rather than a rename, it has to fit in the 187 GiB of
|
||||
free space, and `rename(2)` refuses it anyway. Trash goes beside the item —
|
||||
`/mnt/entropy/.AGENTTRASH/<date>-<topic>/` — so it costs one inode change and
|
||||
rolls back exactly like the move that made it.
|
||||
- **A Jellyfin 401 is the header, not the key.** 12.4 authenticates only
|
||||
`Authorization: MediaBrowser Token="…"` and refuses a bare `X-Emby-Token` for
|
||||
every key, valid or not. This page carried "no credential on this box can
|
||||
trigger a scan, the two Seerr keys answer 401" for weeks because of that
|
||||
header — the keys were innocent. Before believing a 401, prove the key against
|
||||
the server's own table: `select count(*) from ApiKeys where AccessToken = ?`
|
||||
on `jellyfin.db` opened `mode=ro&immutable=1`.
|
||||
- **A scan that must see a new folder has to be recursive.** `POST
|
||||
/Library/Refresh` is the incremental pass, and inotify does not cross a
|
||||
rootless podman bind, so folders created on the host stay invisible to it. The
|
||||
move is `POST /Items/{id}/Refresh?Recursive=true` (204) per library, the id
|
||||
from `GET /Library/VirtualFolders` — its field is `ItemId`, not `Id` — and
|
||||
that same record's `RefreshStatus` is the progress read.
|
||||
- **A file whose name carries no words is bonus material, not a folder.**
|
||||
`.Extra 1 ( 1H 27min multisub ).mkv` parses to `1`; a movie folder named `1`
|
||||
is not a movie. It becomes a review line telling a human to name the file.
|
||||
@@ -159,6 +292,36 @@ curl -s -K /tmp/jf.hdr -o /dev/null -w "jellyfin scan: %{http_code}\n" \
|
||||
inside disqualifies the subtree, because the thing behind a link is not this
|
||||
tool's to delete. Every removal is journalled as `rmdir` and `rollback`
|
||||
recreates the folder, so the sweep is undone by the same command as the moves.
|
||||
Proof carries downward: inside a folder the journal proves emptied, every
|
||||
empty directory goes with it. A release's `Screens/` or `Sample/` is never a
|
||||
move source itself, so 146 of them held their emptied parents on disk after
|
||||
the first sweep.
|
||||
A failed `rmdir` is not a removal: only a folder that is really gone leaves
|
||||
scope, so the next sweep finishes what a permission problem stopped.
|
||||
- **An episode-shaped name marks a SERIES, not a film — and the series is the
|
||||
name BEFORE the marker.** `Monk - S05E03 - Mr. Monk and the Big Game` is series
|
||||
`Monk`. Keyed on the whole title instead, every episode became a group of one
|
||||
and never reached the five-episode size that calls something a series, so a
|
||||
box set was filed as 124 movies in 124 folders. The threshold still earns its
|
||||
keep: a lone film named `Part 2` stays a film. Episodes of a series are parked
|
||||
in `_TV needs a TV library - <series>` for a human to move to a TV source —
|
||||
the movie tool never files TV.
|
||||
- **One journal is one undo.** The TV move runs on its own journal, and with
|
||||
`--root` set to the share that holds BOTH libraries: the executor refuses any
|
||||
path outside `--root`, and a journal shared with the movie relayout would
|
||||
undo both runs when only one of them is wanted.
|
||||
|
||||
- **Sidecar metadata does not follow a rename, because it lives on another
|
||||
filesystem.** The media servers write `.nfo` and artwork through the
|
||||
[[Media Metadata View]] (NVMe upper layer); this tool moves bytes on the raw
|
||||
array path. After any sweep or TV move, run `scripts/media-view.sh reap`, then
|
||||
the same command with `--apply`: the sidecars left at the old coordinates become
|
||||
ghost folders — artwork and an `.nfo` with no video — that both servers browse
|
||||
and report as content.
|
||||
- **Nothing deletes through the metadata view.** A delete there writes an overlayfs
|
||||
whiteout in the upper layer, which hides the array's copy without removing it;
|
||||
the file returns when the view is unmounted. Deletions in this pipeline happen
|
||||
on the raw path only, which is what the journal can undo.
|
||||
|
||||
## Rollback
|
||||
|
||||
@@ -170,7 +333,7 @@ file cannot be accounted for. A rolled-back journal refuses to roll back twice.
|
||||
|
||||
## See Also
|
||||
|
||||
[[Jellyfin]], [[Plex]], [[Entropy Volume]], [[Testing Policy]], [[Storage Policy]]
|
||||
[[Jellyfin]], [[Plex]], [[Media Metadata View]], [[Entropy Volume]], [[Testing Policy]], [[Storage Policy]]
|
||||
|
||||
## Citations
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ description: "Plex Media Server is a **containerized media server on Dankserver*
|
||||
tags:
|
||||
- dankserver
|
||||
- plex
|
||||
timestamp: 2026-10-21
|
||||
timestamp: 2026-10-30
|
||||
---
|
||||
|
||||
# Plex
|
||||
@@ -13,7 +13,8 @@ Plex Media Server is a **containerized media server on Dankserver**: a podman
|
||||
quadlet (`plex.service`) under the systemd **user** generator, generated from
|
||||
`dankserver/plex/quadlet/plex.container`. It shares the host with a second live
|
||||
media server, [[Jellyfin]] (`jellyfin.service`, listening on 8096); both serve
|
||||
from the same read-only Entropy Volume mount.
|
||||
from the same tree — the [[Media Metadata View]] overlay — so the Kodi-format
|
||||
sidecars Jellyfin writes are the files Plex reads, with no copy and no sync.
|
||||
|
||||
**Host:** Dankserver — **Ports:** 32400/tcp (PMS web + API), 32400 is the
|
||||
only port PMS needs for client streaming; LAN clients connect directly,
|
||||
@@ -26,7 +27,7 @@ public traffic can be fronted by Caddy like every other vhost.
|
||||
| Runtime | podman quadlet, `plex.service` (user manager) — migrated from the native `plexmediaserver.service` OS-package unit (2026-09) |
|
||||
| Image | `lscr.io/linuxserver/plex:1.43.5.11029-6bf8af3f0-ls329` (pinned; verified == the registry's `latest` manifest digest at pin time: `sha256:8f655bad…e9e2334`) |
|
||||
| Data root | `/var/lib/plexmediaserver` bind-mounted **in place** as `/config` (RW) — the migration moved zero bytes |
|
||||
| Media | `/mnt/entropy` mounted at the same absolute path **read-only** — every library path in the PMS database stayed valid, zero re-scan |
|
||||
| Media | `/var/lib/media-meta/view` (the [[Media Metadata View]] overlay) bound at the same absolute path `/mnt/entropy` **read-only** — every library path in the PMS database stayed valid, zero re-scan |
|
||||
| Identity | fleet rootless model — container root maps to the `dank` user (1000:1000); the deployer chowns the data root on first cutover |
|
||||
| Port | `32400/tcp` published on `0.0.0.0` — zero client reconfiguration |
|
||||
| Network | `dankserver_appnet`, alias `plex`; Tautulli reaches it at `http://plex:32400` |
|
||||
@@ -38,7 +39,8 @@ public traffic can be fronted by Caddy like every other vhost.
|
||||
|
||||
The migration moved zero bytes: the container bind-mounts the native data
|
||||
root where it already lived, and the media archive re-mounts at
|
||||
`/mnt/entropy` inside the container, so every absolute library path in the
|
||||
`/mnt/entropy` inside the container — only the HOST side of that bind has moved
|
||||
(array, then the overlay view), so every absolute library path in the
|
||||
PMS database keeps resolving. The one metadata change: rootless podman
|
||||
cannot map the real host uid 997, so the fleet model applies — container
|
||||
root runs as the `dank` user, and the deployer adopts the data root with
|
||||
|
||||
@@ -49,9 +49,10 @@ is a locator: one row, one clause, no numbers.
|
||||
| [[Copyparty]] | Copyparty is a lightweight file server and media streamer that runs as a single Python binary. |
|
||||
| [[Dankserver Stack]] | Service inventory and routing overview for the media and library host. |
|
||||
| [[Entropy Volume]] | The Dankserver entropy volume is a RAID-backed LUKS filesystem mounted at |
|
||||
| [[Jellyfin]] | Jellyfin on Dankserver: quadlet unit, entropy-volume data layout, ports, client access, hardware-transcode status. |
|
||||
| [[Jellyfin]] | Jellyfin on Dankserver: quadlet unit, overlay metadata view, provider keys, ports, access, transcode status. |
|
||||
| [[mariadb]] | The MariaDB backend for RomM: quadlet unit, network alias, env-file secret handling, and the backup path. |
|
||||
| [[Media Relayout]] | Census, plan and apply the movie library into Jellyfin format: the gates to read, and the rollback. |
|
||||
| [[Media Metadata View]] | The SSD-backed overlay the two media servers read and write: shape, guards, commands, laws, rollback. |
|
||||
| [[Media Relayout]] | Census, plan and apply the movie and TV libraries into Jellyfin format: gates, evidence chain, refusals, rollback. |
|
||||
| [[Plex]] | Plex Media Server is a **containerized media server on Dankserver**: a podman |
|
||||
| [[redis]] | The Redis cache and task queue for RomM: quadlet unit, network alias topology, and the backup path. |
|
||||
| [[RomM]] | RomM (ROM Manager) is a self-hosted ROM library manager and player. |
|
||||
|
||||
@@ -133,6 +133,16 @@ REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
|
||||
[ -z "$output" ]
|
||||
}
|
||||
|
||||
@test "a compose edit never restarts a Quadlet-owned media server" {
|
||||
# dankserver/compose.yaml is an inventory; the media servers' Quadlets own
|
||||
# their containers. A compose edit that restarted them would cut real viewers
|
||||
# for a change that cannot affect them.
|
||||
run python3 "$PLANNER" --target dankserver --paths dankserver/compose.yaml
|
||||
[ "$status" -eq 0 ]
|
||||
! printf '%s\n' "$output" | grep -qx jellyfin
|
||||
! printf '%s\n' "$output" | grep -qx plex
|
||||
}
|
||||
|
||||
@test "compose planner narrows a compose edit to the changed service" {
|
||||
local base head
|
||||
base=$(git -C "$REPO_ROOT" rev-parse HEAD)
|
||||
|
||||
Executable
+297
@@ -0,0 +1,297 @@
|
||||
#!/usr/bin/env bats
|
||||
# Seam: scripts/jellyfin-providers.sh — the only mechanism that puts a
|
||||
# metadata-provider key into Jellyfin's plugin configuration on Dankserver.
|
||||
#
|
||||
# Output vocabulary the tests depend on:
|
||||
# "+ <action>" something this run will change
|
||||
# "= <fact>" something already true, so nothing changes
|
||||
#
|
||||
# The contracts, in the order the seam tests them:
|
||||
# 1. plan reports every action and never prints a key;
|
||||
# 2. a secret file that is not mode 600 is refused by name and nothing is
|
||||
# written — a key readable by other users is an error, not a warning;
|
||||
# 3. a downloaded plugin artifact is accepted only on its pinned sha256, its
|
||||
# official guid and its pinned version (a swapped or downgraded artifact
|
||||
# is refused before anything lands in plugins/);
|
||||
# 4. the TMDb key is written into the EXISTING config with every other
|
||||
# setting preserved — Jellyfin owns that file, and clobbering it would
|
||||
# silently reset country, language and adult settings;
|
||||
# 5. a TVDB config the operator already saved through the UI is kept;
|
||||
# 6. install is idempotent: a second run changes no byte;
|
||||
# 7. whatever is replaced is copied to the trash first (fleet law: never
|
||||
# overwrite without a recoverable copy);
|
||||
# 8. verify answers whether the endpoint is configured, key LENGTH only;
|
||||
# 9. no plugin configuration XML is tracked in git, the deployer places the
|
||||
# keys BEFORE it restarts Jellyfin, and the planner routes a change to this
|
||||
# installer to jellyfin.
|
||||
|
||||
load helpers
|
||||
|
||||
PROV="$REPO_ROOT/scripts/jellyfin-providers.sh"
|
||||
TMDB_KEY="tmdbk3a9Zq7Rtestkey"
|
||||
TVDB_KEY="tvdbk8b2Wt4Ptestkey"
|
||||
GUID_OFFICIAL="a677c0da-fac5-4cde-941a-7134223f14c8"
|
||||
|
||||
# make_plugin <file> <guid> <version> — a zip shaped like the official TheTVDB
|
||||
# artifact (meta.json + its two DLLs), so extraction is exercised for real.
|
||||
make_plugin() {
|
||||
python3 - "$1" "$2" "$3" <<'PYEOF'
|
||||
import json, sys, zipfile
|
||||
path, guid, version = sys.argv[1], sys.argv[2], sys.argv[3]
|
||||
meta = {
|
||||
"category": "MoviesAndShows",
|
||||
"guid": guid,
|
||||
"name": "TheTVDB",
|
||||
"owner": "jellyfin",
|
||||
"targetAbi": "12.0.0.0",
|
||||
"version": version,
|
||||
}
|
||||
with zipfile.ZipFile(path, "w") as z:
|
||||
z.writestr("meta.json", json.dumps(meta))
|
||||
z.writestr("Jellyfin.Plugin.Tvdb.dll", "dll-bytes")
|
||||
z.writestr("Tvdb.Sdk.dll", "dll-bytes")
|
||||
PYEOF
|
||||
sha256sum "$1" | cut -d' ' -f1
|
||||
}
|
||||
|
||||
setup() {
|
||||
JF="$BATS_TEST_TMPDIR/jf"
|
||||
TRASH="$BATS_TEST_TMPDIR/trash"
|
||||
SEC="$BATS_TEST_TMPDIR/secrets"
|
||||
mkdir -p "$JF/plugins/configurations" "$TRASH" "$SEC"
|
||||
cat >"$JF/plugins/configurations/Jellyfin.Plugin.Tmdb.xml" <<XML
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<Configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
|
||||
<MetadataCountryCode>US</MetadataCountryCode>
|
||||
<PreferOriginalLanguage />
|
||||
<Language>en</Language>
|
||||
<EnableCinematicForTrailers>false</EnableCinematicForTrailers>
|
||||
<IncludeAdult>false</IncludeAdult>
|
||||
<MaxCastMembers>25</MaxCastMembers>
|
||||
<AddGenresFromKeywords>true</AddGenresFromKeywords>
|
||||
<TmdbApiKey />
|
||||
</Configuration>
|
||||
XML
|
||||
KEYS="$SEC/provider-keys"
|
||||
printf 'TMDB_API_KEY=%s\nTVDB_API_KEY=%s\n' "$TMDB_KEY" "$TVDB_KEY" >"$KEYS"
|
||||
chmod 600 "$KEYS"
|
||||
ZIP="$BATS_TEST_TMPDIR/thetvdb.zip"
|
||||
PIN_SHA="$(make_plugin "$ZIP" "$GUID_OFFICIAL" 24.0.0.0)"
|
||||
PROV_ENV=(
|
||||
JF_ROOT="$JF"
|
||||
PROV_KEYS="$KEYS"
|
||||
TRASH_DIR="$TRASH"
|
||||
TVDB_PLUGIN_URL="file://$ZIP"
|
||||
TVDB_PLUGIN_SHA256="$PIN_SHA"
|
||||
TVDB_PLUGIN_GUID="$GUID_OFFICIAL"
|
||||
TVDB_PLUGIN_VERSION=24.0.0.0
|
||||
)
|
||||
}
|
||||
|
||||
prov() { env "${PROV_ENV[@]}" bash "$PROV" "$@"; }
|
||||
|
||||
tree_fingerprint() { find "$JF" -type f | sort | xargs -r sha256sum | sha256sum; }
|
||||
|
||||
@test "plan names the plugin and both config files, and prints no key" {
|
||||
run prov plan
|
||||
[ "$status" -eq 0 ]
|
||||
printf '%s\n' "${lines[@]}" | grep -q '^+ install thetvdb_24\.0\.0\.0'
|
||||
printf '%s\n' "${lines[@]}" | grep -q '^+ set TmdbApiKey'
|
||||
printf '%s\n' "${lines[@]}" | grep -q '^+ create Jellyfin\.Plugin\.Tvdb\.xml'
|
||||
# The whole point of running this over a shoulder: a key never appears.
|
||||
! printf '%s\n' "${lines[@]}" | grep -qF "$TMDB_KEY"
|
||||
! printf '%s\n' "${lines[@]}" | grep -qF "$TVDB_KEY"
|
||||
}
|
||||
|
||||
@test "plan writes nothing" {
|
||||
before="$(tree_fingerprint)"
|
||||
run prov plan
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$(tree_fingerprint)" = "$before" ]
|
||||
}
|
||||
|
||||
@test "an already-configured endpoint reports no pending action" {
|
||||
prov install >/dev/null
|
||||
run prov plan
|
||||
[ "$status" -eq 0 ]
|
||||
! printf '%s\n' "${lines[@]}" | grep -q '^+ '
|
||||
printf '%s\n' "${lines[@]}" | grep -q '= thetvdb_24\.0\.0\.0 already installed'
|
||||
printf '%s\n' "${lines[@]}" | grep -q '= TmdbApiKey already set'
|
||||
printf '%s\n' "${lines[@]}" | grep -q '= Jellyfin\.Plugin\.Tvdb\.xml already has an ApiKey'
|
||||
}
|
||||
|
||||
@test "a keys file that is not mode 600 is refused and nothing is written" {
|
||||
chmod 644 "$KEYS"
|
||||
before="$(tree_fingerprint)"
|
||||
run prov install
|
||||
[ "$status" -ne 0 ]
|
||||
printf '%s\n' "${lines[@]}" | grep -qF '600'
|
||||
[ "$(tree_fingerprint)" = "$before" ]
|
||||
}
|
||||
|
||||
@test "a missing TVDB key is refused by name before anything is fetched" {
|
||||
printf 'TMDB_API_KEY=%s\n' "$TMDB_KEY" >"$KEYS"
|
||||
chmod 600 "$KEYS"
|
||||
run prov install
|
||||
[ "$status" -ne 0 ]
|
||||
printf '%s\n' "${lines[@]}" | grep -qF 'TVDB_API_KEY'
|
||||
[ ! -d "$JF/plugins/thetvdb_24.0.0.0" ]
|
||||
}
|
||||
|
||||
@test "a root with no plugin directory is refused instead of fabricated" {
|
||||
# Creating plugins/ inside a wrong path would install a plugin Jellyfin
|
||||
# never loads, and would hide the operator's typo.
|
||||
run env "${PROV_ENV[@]}" JF_ROOT="$BATS_TEST_TMPDIR/nope" bash "$PROV" install
|
||||
[ "$status" -ne 0 ]
|
||||
[ ! -d "$BATS_TEST_TMPDIR/nope/plugins" ]
|
||||
}
|
||||
|
||||
@test "an artifact whose sha256 is not the pinned one is refused" {
|
||||
run env "${PROV_ENV[@]}" TVDB_PLUGIN_SHA256="$(printf 'f%.0s' {1..64})" \
|
||||
bash "$PROV" install
|
||||
[ "$status" -ne 0 ]
|
||||
printf '%s\n' "${lines[@]}" | grep -qiF 'sha256'
|
||||
[ ! -d "$JF/plugins/thetvdb_24.0.0.0" ]
|
||||
}
|
||||
|
||||
@test "an artifact that is not the official TVDB plugin is refused" {
|
||||
# A matching sha proves only that the bytes are the bytes we fetched.
|
||||
EVIL="$BATS_TEST_TMPDIR/evil.zip"
|
||||
EVIL_SHA="$(make_plugin "$EVIL" "deadbeef-0000-0000-0000-000000000000" 24.0.0.0)"
|
||||
run env "${PROV_ENV[@]}" TVDB_PLUGIN_URL="file://$EVIL" TVDB_PLUGIN_SHA256="$EVIL_SHA" \
|
||||
bash "$PROV" install
|
||||
[ "$status" -ne 0 ]
|
||||
printf '%s\n' "${lines[@]}" | grep -qiF 'guid'
|
||||
[ ! -d "$JF/plugins/thetvdb_24.0.0.0" ]
|
||||
}
|
||||
|
||||
@test "an artifact that is not the pinned version is refused" {
|
||||
OLD="$BATS_TEST_TMPDIR/old.zip"
|
||||
OLD_SHA="$(make_plugin "$OLD" "$GUID_OFFICIAL" 22.0.0.0)"
|
||||
run env "${PROV_ENV[@]}" TVDB_PLUGIN_URL="file://$OLD" TVDB_PLUGIN_SHA256="$OLD_SHA" \
|
||||
bash "$PROV" install
|
||||
[ "$status" -ne 0 ]
|
||||
printf '%s\n' "${lines[@]}" | grep -qiF 'version'
|
||||
[ ! -d "$JF/plugins/thetvdb_24.0.0.0" ]
|
||||
}
|
||||
|
||||
@test "a different plugin version already in plugins/ is refused, not doubled" {
|
||||
# Two thetvdb_* directories means Jellyfin loads one plugin twice; the
|
||||
# operator removes the old one (into the trash) deliberately, not by an
|
||||
# installer quietly adding a second.
|
||||
prov install >/dev/null
|
||||
OLD="$BATS_TEST_TMPDIR/old.zip"
|
||||
OLD_SHA="$(make_plugin "$OLD" "$GUID_OFFICIAL" 22.0.0.0)"
|
||||
run env "${PROV_ENV[@]}" TVDB_PLUGIN_URL="file://$OLD" TVDB_PLUGIN_SHA256="$OLD_SHA" \
|
||||
TVDB_PLUGIN_VERSION=22.0.0.0 bash "$PROV" install
|
||||
[ "$status" -ne 0 ]
|
||||
printf '%s\n' "${lines[@]}" | grep -qF 'thetvdb_24.0.0.0'
|
||||
[ ! -d "$JF/plugins/thetvdb_22.0.0.0" ]
|
||||
}
|
||||
|
||||
@test "install places the plugin, keys TMDb, and preserves every other setting" {
|
||||
run prov install
|
||||
[ "$status" -eq 0 ]
|
||||
! printf '%s\n' "${lines[@]}" | grep -qF "$TMDB_KEY"
|
||||
! printf '%s\n' "${lines[@]}" | grep -qF "$TVDB_KEY"
|
||||
[ -f "$JF/plugins/thetvdb_24.0.0.0/meta.json" ]
|
||||
[ -f "$JF/plugins/thetvdb_24.0.0.0/Jellyfin.Plugin.Tvdb.dll" ]
|
||||
local cfg="$JF/plugins/configurations/Jellyfin.Plugin.Tmdb.xml"
|
||||
# Jellyfin owns this file: the settings it wrote must survive the edit.
|
||||
for setting in MetadataCountryCode Language IncludeAdult MaxCastMembers AddGenresFromKeywords; do
|
||||
grep -qF "<$setting>" "$cfg" || {
|
||||
echo "lost $setting"
|
||||
return 1
|
||||
}
|
||||
done
|
||||
grep -qF '<IncludeAdult>false</IncludeAdult>' "$cfg"
|
||||
grep -qF "<TmdbApiKey>$TMDB_KEY</TmdbApiKey>" "$cfg"
|
||||
grep -qF "<ApiKey>$TVDB_KEY</ApiKey>" "$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml"
|
||||
}
|
||||
|
||||
@test "a TVDB config the operator saved through the UI is kept" {
|
||||
cat >"$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml" <<XML
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<Configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
|
||||
<ApiKey>ui-saved-key</ApiKey>
|
||||
<EnableSeasonPathMatching>true</EnableSeasonPathMatching>
|
||||
</Configuration>
|
||||
XML
|
||||
run prov install
|
||||
[ "$status" -eq 0 ]
|
||||
printf '%s\n' "${lines[@]}" | grep -qiF 'kept'
|
||||
local cfg="$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml"
|
||||
grep -qF '<ApiKey>ui-saved-key</ApiKey>' "$cfg"
|
||||
grep -qF '<EnableSeasonPathMatching>true</EnableSeasonPathMatching>' "$cfg"
|
||||
}
|
||||
|
||||
@test "a pin in the TVDB secret lands in the plugin config" {
|
||||
# A subscriber-shaped key needs the account's PIN beside it; a licensed key
|
||||
# does not, so the element appears only when the secret carries one.
|
||||
printf 'TMDB_API_KEY=%s\nTVDB_API_KEY=%s\nTVDB_PIN=123456\n' "$TMDB_KEY" "$TVDB_KEY" >"$KEYS"
|
||||
chmod 600 "$KEYS"
|
||||
prov install >/dev/null
|
||||
grep -qF '<Pin>123456</Pin>' "$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml"
|
||||
}
|
||||
|
||||
@test "install is idempotent: a second run changes no byte" {
|
||||
prov install >/dev/null
|
||||
before="$(tree_fingerprint)"
|
||||
run prov install
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$(tree_fingerprint)" = "$before" ]
|
||||
printf '%s\n' "${lines[@]}" | grep -qiF 'no change'
|
||||
# One plugin directory only: two versions in plugins/ means two loads.
|
||||
[ "$(find "$JF/plugins" -maxdepth 1 -type d -name 'thetvdb_*' | wc -l)" -eq 1 ]
|
||||
}
|
||||
|
||||
@test "what install replaces is copied to the trash first" {
|
||||
prov install >/dev/null
|
||||
# The TMDb config existed before this tool touched it, so the pre-edit copy
|
||||
# must stay recoverable.
|
||||
[ -n "$(find "$TRASH" -name 'Jellyfin.Plugin.Tmdb.xml*' -print -quit)" ]
|
||||
grep -qF '<TmdbApiKey />' "$(find "$TRASH" -name 'Jellyfin.Plugin.Tmdb.xml*' | head -1)"
|
||||
}
|
||||
|
||||
@test "verify refuses an unconfigured endpoint and names what is missing" {
|
||||
run prov verify
|
||||
[ "$status" -ne 0 ]
|
||||
printf '%s\n' "${lines[@]}" | grep -qiF 'thetvdb'
|
||||
printf '%s\n' "${lines[@]}" | grep -qiF 'TmdbApiKey'
|
||||
}
|
||||
|
||||
@test "verify passes once the plugin and both keys are in place" {
|
||||
prov install >/dev/null
|
||||
run prov verify
|
||||
[ "$status" -eq 0 ]
|
||||
! printf '%s\n' "${lines[@]}" | grep -qF "$TMDB_KEY"
|
||||
! printf '%s\n' "${lines[@]}" | grep -qF "$TVDB_KEY"
|
||||
# Length, not bytes: the operator sees the key loaded without reading it.
|
||||
printf '%s\n' "${lines[@]}" | grep -qE 'TmdbApiKey .*[0-9]+ chars'
|
||||
}
|
||||
|
||||
@test "no plugin configuration XML is tracked in this repo" {
|
||||
# These files hold provider keys and are owned by the running container;
|
||||
# tracking them would put credentials in git and guarantee drift.
|
||||
run git -C "$REPO_ROOT" ls-files
|
||||
[ "$status" -eq 0 ]
|
||||
! printf '%s\n' "${lines[@]}" | grep -qF 'plugins/configurations'
|
||||
! printf '%s\n' "${lines[@]}" | grep -qiF 'Plugin.Tvdb.xml'
|
||||
}
|
||||
|
||||
@test "the planner routes this installer to jellyfin" {
|
||||
run python3 "$REPO_ROOT/scripts/deploy-plan.py" --target dankserver \
|
||||
--paths scripts/jellyfin-providers.sh
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$output" = "jellyfin" ]
|
||||
}
|
||||
|
||||
@test "the deployer places the keys before it restarts jellyfin" {
|
||||
# The plugin and both keys are read at start-up, so a restart that runs
|
||||
# ahead of the placement boots the old, keyless state.
|
||||
body="$(sed -n '/^deploy_jellyfin()/,/^\}/p' "$REPO_ROOT/scripts/deploy-remote.sh")"
|
||||
[ -n "$body" ]
|
||||
up="$(printf '%s\n' "$body" | grep -n 'ensure_provider_keys' | head -1 | cut -d: -f1)"
|
||||
restart="$(printf '%s\n' "$body" | grep -n 'systemctl --user restart jellyfin.service' | head -1 | cut -d: -f1)"
|
||||
[ "${up:-999}" -lt "${restart:-0}" ]
|
||||
}
|
||||
Executable
+161
@@ -0,0 +1,161 @@
|
||||
#!/usr/bin/env bats
|
||||
# Seam: the SSD-backed metadata view (OverlayFS) that lets Jellyfin write
|
||||
# Kodi-format sidecars for Plex to read, without ever writing to the spindle
|
||||
# array and without letting either media server start against a missing view.
|
||||
#
|
||||
# The contracts, in the order the seam tests them:
|
||||
# 1. the plan puts the writable layer on the SSD and the media layer below it;
|
||||
# 2. bad input is refused by name instead of mounting something surprising;
|
||||
# 3. verify refuses a view that is not mounted (so nothing trusts a directory);
|
||||
# 4. both media units bind the VIEW and refuse to start without it;
|
||||
# 5. only those two units touch the view — the LLM and library tenants keep the
|
||||
# raw volume, so the blast radius is two containers;
|
||||
# 6. the deployer brings the view up BEFORE it restarts either server;
|
||||
# 7. the planner routes a change to these files to both media services.
|
||||
|
||||
load helpers
|
||||
|
||||
VIEW="$REPO_ROOT/scripts/media-view.sh"
|
||||
JELLY="$REPO_ROOT/dankserver/jellyfin/quadlet/jellyfin.container"
|
||||
PLEX="$REPO_ROOT/dankserver/plex/quadlet/plex.container"
|
||||
MOUNT=/var/lib/media-meta/view
|
||||
UNIT="$REPO_ROOT/dankserver/host/systemd/media-meta-view.mount"
|
||||
|
||||
@test "the plan names an overlay with the spindle below and the SSD above" {
|
||||
run bash "$VIEW" plan
|
||||
[ "$status" -eq 0 ]
|
||||
# Lower = the array; upper/work = the NVMe. A user asked for metadata to be
|
||||
# SSD-resident, so a spindle upper layer would defeat the whole mechanism.
|
||||
[ "${lines[0]}" = "lowerdir=/mnt/entropy" ]
|
||||
printf '%s\n' "${lines[@]}" | grep -qx 'upperdir=/var/lib/media-meta/upper'
|
||||
printf '%s\n' "${lines[@]}" | grep -qx 'workdir=/var/lib/media-meta/work'
|
||||
printf '%s\n' "${lines[@]}" | grep -Fqx "mountpoint=$MOUNT"
|
||||
# systemd names a mount unit after its escaped path, backslash and all.
|
||||
printf '%s\n' "${lines[@]}" | grep -Fqx \
|
||||
'unit=/etc/systemd/system/var-lib-media\x2dmeta-view.mount'
|
||||
}
|
||||
|
||||
@test "install refuses a caller who cannot write the unit directory or mount" {
|
||||
# Half an install is worse than none: the unit would exist without the
|
||||
# directories behind it. Refuse by name before anything is created.
|
||||
if [ "$(id -u)" -eq 0 ]; then
|
||||
skip "this seam is running as root"
|
||||
fi
|
||||
run bash "$VIEW" install
|
||||
[ "$status" -ne 0 ]
|
||||
[[ "$output" == *"root"* ]]
|
||||
}
|
||||
|
||||
@test "verify refuses a view that is not mounted" {
|
||||
# Every consumer trusts the mount point to BE the library. On a host where it
|
||||
# is a plain empty directory, verify must fail loudly: that is the difference
|
||||
# between a scan that finds nothing and a scan that empties a library DB.
|
||||
if mountpoint -q "$MOUNT"; then
|
||||
skip "$MOUNT is a real mountpoint on this host"
|
||||
fi
|
||||
run bash "$VIEW" verify
|
||||
[ "$status" -ne 0 ]
|
||||
[[ "$output" == *"$MOUNT"* ]]
|
||||
}
|
||||
|
||||
@test "both media servers bind the view and refuse to start without it" {
|
||||
for unit in "$JELLY" "$PLEX"; do
|
||||
grep -q "^Volume=$MOUNT:" "$unit" || return 1
|
||||
grep -q "^UnitConfigItem=ConditionPathIsMountPoint=$MOUNT$" "$unit" || return 1
|
||||
# The raw array must no longer be a container source: that bind is what
|
||||
# made sidecar sharing impossible (read-only) and metadata slow (seeks).
|
||||
! grep -q '^Volume=/mnt/entropy' "$unit"
|
||||
done
|
||||
# Jellyfin is the writer, so it needs the view writable; Plex only reads the
|
||||
# sidecars, and keeping its bind read-only costs nothing.
|
||||
grep -q "^Volume=$MOUNT:/data$" "$JELLY"
|
||||
grep -q "^Volume=$MOUNT:/mnt/entropy:ro$" "$PLEX"
|
||||
}
|
||||
|
||||
@test "only the two media units reference the view" {
|
||||
run grep -rl --include='*.container' "$MOUNT" "$REPO_ROOT"/dankserver \
|
||||
"$REPO_ROOT"/dankclaw "$REPO_ROOT"/frankbook-pro 2>/dev/null
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$(printf '%s\n' "$output" | wc -l)" -eq 2 ]
|
||||
printf '%s\n' "$output" | grep -q 'jellyfin/quadlet/jellyfin.container'
|
||||
printf '%s\n' "$output" | grep -q 'plex/quadlet/plex.container'
|
||||
}
|
||||
|
||||
@test "the mount unit is an overlay whose upper layer lives on the SSD" {
|
||||
[ -f "$UNIT" ]
|
||||
run grep -E '^Options=lowerdir=/mnt/entropy,upperdir=/var/lib/media-meta/upper,workdir=/var/lib/media-meta/work' "$UNIT"
|
||||
[ "$status" -eq 0 ]
|
||||
# The unit must mount exactly where the two containers bind, or the guard and
|
||||
# the bind disagree and a media server browses an empty directory.
|
||||
run grep -Fqx "Where=$MOUNT" "$UNIT"
|
||||
[ "$status" -eq 0 ]
|
||||
# It must wait for the array to be unlocked: mounting an overlay whose lower
|
||||
# dir is an empty /mnt/entropy would present an empty library, not an error.
|
||||
run grep -E '^RequiresMountsFor=/mnt/entropy$' "$UNIT"
|
||||
[ "$status" -eq 0 ]
|
||||
}
|
||||
|
||||
@test "the deployer raises the view before it restarts either media server" {
|
||||
for svc in jellyfin plex; do
|
||||
body="$(sed -n "/^deploy_${svc}()/,/^\}/p" "$REPO_ROOT/scripts/deploy-remote.sh")"
|
||||
[ -n "$body" ]
|
||||
up="$(printf '%s\n' "$body" | grep -n 'ensure_media_view' | head -1 | cut -d: -f1)"
|
||||
up_line="${up:-999}"
|
||||
restart="$(printf '%s\n' "$body" | grep -n "systemctl --user restart $svc.service" | head -1 | cut -d: -f1)"
|
||||
restart_line="${restart:-0}"
|
||||
[ "$up_line" -lt "$restart_line" ]
|
||||
done
|
||||
}
|
||||
|
||||
@test "the planner routes the view files to both media services" {
|
||||
run python3 "$REPO_ROOT/scripts/deploy-plan.py" --target dankserver --paths \
|
||||
scripts/media-view.sh dankserver/host/systemd/media-meta-view.mount
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$output" = "jellyfin
|
||||
plex" ]
|
||||
}
|
||||
|
||||
@test "a docs-only change still plans nothing for the view" {
|
||||
run python3 "$REPO_ROOT/scripts/deploy-plan.py" --target dankserver --paths \
|
||||
"serverAppConfigs-docs/Dankserver/Media Metadata View.md"
|
||||
[ "$status" -eq 0 ]
|
||||
[ -z "$output" ]
|
||||
}
|
||||
|
||||
@test "reap names the metadata whose media has moved and touches nothing" {
|
||||
# A relayout moves media on the RAW path; the sidecars the view already wrote
|
||||
# stay behind at the old coordinates. Left alone they become folders a media
|
||||
# server browses and finds nothing in.
|
||||
tmp="$(mktemp -d)"
|
||||
mkdir -p "$tmp/meta/upper/Movies/Ghost (2020)" "$tmp/meta/upper/Movies/Keeper (2020)" \
|
||||
"$tmp/meta/upper/TVShows/Big Little Lies/Season 01" "$tmp/meta/work" \
|
||||
"$tmp/array/Movies/Keeper (2020)" "$tmp/array/TVShows"
|
||||
: > "$tmp/meta/upper/Movies/Ghost (2020)/movie.nfo"
|
||||
: > "$tmp/meta/upper/Movies/Keeper (2020)/movie.nfo"
|
||||
: > "$tmp/meta/upper/TVShows/Big Little Lies/Season 01/show.nfo"
|
||||
: > "$tmp/array/Movies/Keeper (2020)/Keeper (2020).mkv"
|
||||
|
||||
run bash "$VIEW" reap --upper-base "$tmp/meta" --lower "$tmp/array"
|
||||
[ "$status" -eq 0 ]
|
||||
printf '%s\n' "$output" | grep -qxF 'Movies/Ghost (2020)'
|
||||
# The series folder is still below, so the ghost is the season inside it — and
|
||||
# it is reported at the level that is actually missing, not deeper.
|
||||
printf '%s\n' "$output" | grep -qxF 'TVShows/Big Little Lies'
|
||||
! printf '%s\n' "$output" | grep -q 'Keeper'
|
||||
[ -f "$tmp/meta/upper/Movies/Ghost (2020)/movie.nfo" ]
|
||||
}
|
||||
|
||||
@test "reap --apply moves ghosts to the SSD trash and keeps live metadata" {
|
||||
tmp="$(mktemp -d)"
|
||||
mkdir -p "$tmp/meta/upper/Movies/Ghost (2020)" "$tmp/meta/upper/Movies/Keeper (2020)" \
|
||||
"$tmp/array/Movies/Keeper (2020)"
|
||||
: > "$tmp/meta/upper/Movies/Ghost (2020)/movie.nfo"
|
||||
: > "$tmp/meta/upper/Movies/Keeper (2020)/movie.nfo"
|
||||
|
||||
run bash "$VIEW" reap --apply --upper-base "$tmp/meta" --lower "$tmp/array"
|
||||
[ "$status" -eq 0 ]
|
||||
[ ! -e "$tmp/meta/upper/Movies/Ghost (2020)" ]
|
||||
[ -f "$tmp/meta/upper/Movies/Keeper (2020)/movie.nfo" ]
|
||||
# Recoverable, not deleted: the item lands in a dated trash under the SSD base.
|
||||
[ "$(find "$tmp/meta/.AGENTTRASH" -name movie.nfo | wc -l)" -eq 1 ]
|
||||
}
|
||||
@@ -21,6 +21,7 @@ load helpers
|
||||
PLAN="$REPO_ROOT/scripts/media-plan.py"
|
||||
CENSUS="$REPO_ROOT/scripts/media-census.py"
|
||||
APPLY="$REPO_ROOT/scripts/media-apply.py"
|
||||
TV="$REPO_ROOT/scripts/media-tv.py"
|
||||
|
||||
# probe_row FILE DUR [RES] [AUDIO] [SUBS] — one probed-TSV row: the twelve
|
||||
# columns media-census.py probe writes (bytes, path, dur, vcodec, res, fps,
|
||||
@@ -47,6 +48,9 @@ setup() {
|
||||
PLEX="$BATS_TEST_TMPDIR/plex.tsv"
|
||||
OUT="$BATS_TEST_TMPDIR/plan.tsv"
|
||||
EXC="$BATS_TEST_TMPDIR/exceptions.txt"
|
||||
TVLIB="$BATS_TEST_TMPDIR/TVShows"
|
||||
TVPLAN="$BATS_TEST_TMPDIR/tv.tsv"
|
||||
TVJ="$BATS_TEST_TMPDIR/tv.jsonl"
|
||||
: > "$PROBED"
|
||||
printf 'file\tleaf_title\tleaf_year\n' > "$PLEX"
|
||||
|
||||
@@ -71,6 +75,20 @@ setup() {
|
||||
at "$LIB/Alien (1979)/._Alien.1979.1080p.BluRay.x265.10bit.mkv" 220
|
||||
# A bonus file whose name is a number: never a movie folder named "1".
|
||||
at "$LIB/Bruce Lee Collection 1971-1981/.Extra 1.mkv" 2700000
|
||||
# A TV box set sitting in the movie library. This one names each file
|
||||
# "Show - SxxEyy - Episode Title", so the episode title comes AFTER the
|
||||
# marker; that convention is what made 124 episodes of one series look like
|
||||
# 124 different films, each under its own threshold.
|
||||
for e in 01 02 03 04 05; do
|
||||
at "$LIB/Monk 2002 Seasons 1 to 8/Season 1/Monk - S01E$e - Mr Monk and the Rival.mkv" 3000000
|
||||
done
|
||||
# The same trouble named the other way round, with no episode title at all.
|
||||
for e in 01 02 03 04 05; do
|
||||
at "$LIB/Big Little Lies Season 1/Big Little Lies S01E$e.mp4" 3000000
|
||||
done
|
||||
# A film whose own name carries a bare "Part 2": one episode-shaped token is
|
||||
# not a series, and it must not be dragged into the TV pile.
|
||||
at "$LIB/The Godfather Part 2 (1974)/The.Godfather.Part.2.1974.1080p.BluRay.mkv" 9000000
|
||||
|
||||
probe_row "$LIB/Alien (1979)/Alien.1979.1080p.BluRay.x265.10bit.mkv" 4700
|
||||
probe_row "$LIB/Alien (1979)/Featurettes/A Bug And A Bite - Making Alien.mkv" 2700
|
||||
@@ -82,11 +100,18 @@ setup() {
|
||||
probe_row "$LIB/VHS 2 (2013)/VHS.2.2013.1080p.BluRay.x265.mkv" 5700
|
||||
probe_row "$LIB/AMERICAN POP/AMERICAN.POP.2023.1080p.WEB-DL.mkv" 6300
|
||||
probe_row "$LIB/Bruce Lee Collection 1971-1981/.Extra 1.mkv" 5220
|
||||
for e in 01 02 03 04 05; do
|
||||
probe_row "$LIB/Monk 2002 Seasons 1 to 8/Season 1/Monk - S01E$e - Mr Monk and the Rival.mkv" 2600
|
||||
probe_row "$LIB/Big Little Lies Season 1/Big Little Lies S01E$e.mp4" 3300
|
||||
done
|
||||
probe_row "$LIB/The Godfather Part 2 (1974)/The.Godfather.Part.2.1974.1080p.BluRay.mkv" 6200
|
||||
plex_row "$LIB/Alien (1979)/Alien.1979.1080p.BluRay.x265.10bit.mkv" "Alien" "1979"
|
||||
plex_row "$LIB/Marnie (1964)/Marnie.1964.1080p.BluRay.x265.mkv" "Marnie" "1964"
|
||||
plex_row "$LIB/VHS 2 (2013)/VHS.2.2013.1080p.BluRay.x265.mkv" "V/H/S/2" "2013"
|
||||
# An unmatched Plex row: the leaf title echoes the file and the year is 0.
|
||||
plex_row "$LIB/AMERICAN POP/AMERICAN.POP.2023.1080p.WEB-DL.mkv" "AMERICAN POP" "0"
|
||||
plex_row "$LIB/The Godfather Part 2 (1974)/The.Godfather.Part.2.1974.1080p.BluRay.mkv" \
|
||||
"The Godfather Part 2" "1974"
|
||||
|
||||
export MEDIA_ROOT="$LIB"
|
||||
run python3 "$CENSUS" enumerate --root "$LIB" --out "$ENUM"
|
||||
@@ -115,6 +140,33 @@ for line in open(sys.argv[1]):
|
||||
' "$OUT" "$1"
|
||||
}
|
||||
|
||||
# tv_plan — the TV mover over the fixture library; its stdout is the operator's
|
||||
# census of what it called a series and what it refused to touch.
|
||||
tv_plan() {
|
||||
run python3 "$TV" --root "$LIB" --library "$TVLIB" --out "$TVPLAN"
|
||||
[ "$status" -eq 0 ]
|
||||
printf '%s\n' "$output" > "$BATS_TEST_TMPDIR/tv.out"
|
||||
}
|
||||
|
||||
# tv_apply — that plan through the same executor. The destination is a sibling
|
||||
# share, so the plan is applied at the root that holds both libraries.
|
||||
tv_apply() {
|
||||
run python3 "$APPLY" apply --plan "$TVPLAN" --root "$BATS_TEST_TMPDIR" \
|
||||
--journal "$TVJ" --apply
|
||||
[ "$status" -eq 0 ]
|
||||
}
|
||||
|
||||
# tv_dst_of SRC — the destination the TV plan gives one file, empty if none.
|
||||
tv_dst_of() {
|
||||
python3 -c '
|
||||
import sys
|
||||
for line in open(sys.argv[1]):
|
||||
f = line.rstrip("\n").split("\t")
|
||||
if len(f) > 2 and f[1] == sys.argv[2]:
|
||||
print(f[2]); break
|
||||
' "$TVPLAN" "$1"
|
||||
}
|
||||
|
||||
@test "a film lands in Title (year)/Title (year).mkv, named by Plex" {
|
||||
plan
|
||||
[ "$(dst_of "$LIB/Alien (1979)/Alien.1979.1080p.BluRay.x265.10bit.mkv")" \
|
||||
@@ -191,6 +243,112 @@ for line in open(sys.argv[1]):
|
||||
grep -q "UNNAMED" "$EXC"
|
||||
}
|
||||
|
||||
@test "a box set in the movie library is quarantined, not filed as movies" {
|
||||
plan
|
||||
local d
|
||||
d="$(dst_of "$LIB/Monk 2002 Seasons 1 to 8/Season 1/Monk - S01E01 - Mr Monk and the Rival.mkv")"
|
||||
[[ "$d" == "$LIB"/_TV\ needs\ a\ TV\ library\ -\ monk* ]]
|
||||
d="$(dst_of "$LIB/Big Little Lies Season 1/Big Little Lies S01E01.mp4")"
|
||||
[[ "$d" == "$LIB"/_TV\ needs\ a\ TV\ library\ -\ big\ little\ lies* ]]
|
||||
# one series is one pile, whatever its episodes are called
|
||||
run python3 -c '
|
||||
import re, sys
|
||||
pile = {}
|
||||
for line in open(sys.argv[1]):
|
||||
f = line.rstrip("\n").split("\t")
|
||||
if len(f) > 2 and f[0] == "move":
|
||||
m = re.search(r"/_TV needs a TV library - ([^/]*)/", f[2] + "/")
|
||||
if m:
|
||||
pile[m.group(1)] = pile.get(m.group(1), 0) + 1
|
||||
for k in sorted(pile):
|
||||
print(f"{k}={pile[k]}")
|
||||
' "$OUT"
|
||||
[ "${#lines[@]}" -eq 2 ]
|
||||
[[ "${lines[0]}" == big\ little\ lies*=5 ]]
|
||||
[[ "${lines[1]}" == monk*=5 ]]
|
||||
}
|
||||
|
||||
@test "a film named Part 2 is not called a series" {
|
||||
plan
|
||||
[ "$(dst_of "$LIB/The Godfather Part 2 (1974)/The.Godfather.Part.2.1974.1080p.BluRay.mkv")" \
|
||||
= "$LIB/The Godfather Part 2 (1974)/The Godfather Part 2 (1974).mkv" ]
|
||||
}
|
||||
|
||||
@test "the TV mover files a box set into the TV library in Jellyfin shape" {
|
||||
tv_plan
|
||||
tv_apply
|
||||
[ -e "$TVLIB/Monk (2002)/Season 01/Monk - S01E01 - Mr Monk and the Rival.mkv" ]
|
||||
[ -e "$TVLIB/Big Little Lies/Season 01/Big Little Lies S01E01.mp4" ]
|
||||
[ ! -e "$LIB/Monk 2002 Seasons 1 to 8/Season 1/Monk - S01E01 - Mr Monk and the Rival.mkv" ]
|
||||
# films are not the TV mover's business
|
||||
[ -e "$LIB/Alien (1979)/Alien.1979.1080p.BluRay.x265.10bit.mkv" ]
|
||||
}
|
||||
|
||||
@test "the TV mover leaves a film whose name only looks like an episode" {
|
||||
tv_plan
|
||||
tv_apply
|
||||
local gf="$LIB/The Godfather Part 2 (1974)/The.Godfather.Part.2.1974.1080p.BluRay.mkv"
|
||||
[ "$(tv_dst_of "$gf")" = "" ]
|
||||
[ -e "$gf" ]
|
||||
grep -q "Godfather.*not a series" "$BATS_TEST_TMPDIR/tv.out"
|
||||
}
|
||||
|
||||
@test "the TV mover keeps an episode's own subtitle with the episode" {
|
||||
at "$LIB/Monk 2002 Seasons 1 to 8/Season 1/Monk - S01E01 - Mr Monk and the Rival.srt" 40000
|
||||
tv_plan
|
||||
tv_apply
|
||||
[ -e "$TVLIB/Monk (2002)/Season 01/Monk - S01E01 - Mr Monk and the Rival.srt" ]
|
||||
}
|
||||
|
||||
@test "a second TV move finds nothing left to move" {
|
||||
tv_plan
|
||||
tv_apply
|
||||
tv_plan
|
||||
grep -q "0 moves" "$BATS_TEST_TMPDIR/tv.out"
|
||||
}
|
||||
|
||||
@test "a show whose own name carries the year is not given it twice" {
|
||||
local d="$LIB/Leah Remini Scientology and the Aftermath (2016)/Season 1"
|
||||
for e in 01 02 03 04 05; do
|
||||
at "$d/Leah Remini Scientology and the Aftermath (2016) - S01E$e - Episode.mkv" 3000000
|
||||
done
|
||||
tv_plan
|
||||
tv_apply
|
||||
[ -e "$TVLIB/Leah Remini Scientology and the Aftermath (2016)/Season 01/Leah Remini Scientology and the Aftermath (2016) - S01E01 - Episode.mkv" ]
|
||||
[ ! -e "$TVLIB/Leah Remini Scientology and the Aftermath (2016) (2016)" ]
|
||||
}
|
||||
|
||||
@test "an all-lowercase series name is written the way the library writes names" {
|
||||
for e in 01 02 03 04 05; do
|
||||
at "$LIB/downton abbey 2010/Season 1/downton abbey - s01e$e - episode title.mkv" 3000000
|
||||
done
|
||||
tv_plan
|
||||
tv_apply
|
||||
# the FOLDER is titled; the file keeps the name the library already holds
|
||||
[ -e "$TVLIB/Downton Abbey (2010)/Season 01/downton abbey - s01e01 - episode title.mkv" ]
|
||||
}
|
||||
|
||||
@test "the mover leaves an episode the TV library already holds" {
|
||||
local src="$LIB/Monk 2002 Seasons 1 to 8/Season 1/Monk - S01E01 - Mr Monk and the Rival.mkv"
|
||||
for e in 01 02 03 04 05; do
|
||||
at "$TVLIB/Monk (2002)/Season 01/Monk - S01E$e - Mr Monk and the Rival.mkv" 3000000
|
||||
done
|
||||
tv_plan
|
||||
[ -z "$(tv_dst_of "$src")" ]
|
||||
[ -e "$src" ]
|
||||
grep -q "Monk.*already in" "$BATS_TEST_TMPDIR/tv.out"
|
||||
}
|
||||
|
||||
@test "an episode the TV library holds in another encode is still moved" {
|
||||
for e in 01 02 03 04 05; do
|
||||
at "$TVLIB/Big Little Lies/Season 01/Big Little Lies S01E$e 1080p.mp4" 4000000
|
||||
done
|
||||
tv_plan
|
||||
tv_apply
|
||||
[ -e "$TVLIB/Big Little Lies/Season 01/Big Little Lies S01E01.mp4" ]
|
||||
[ -e "$TVLIB/Big Little Lies/Season 01/Big Little Lies S01E01 1080p.mp4" ]
|
||||
}
|
||||
|
||||
@test "every file in the census is placed or kept — nothing is dropped" {
|
||||
plan
|
||||
python3 - "$ENUM" "$OUT" <<'PY'
|
||||
@@ -244,6 +402,9 @@ apply_plan() {
|
||||
moved=$(python3 -c 'import json,sys
|
||||
print(sum(1 for l in open(sys.argv[1]) if json.loads(l).get("op") == "move"))' "$JOURNAL")
|
||||
[ "$moved" -eq "$planned" ]
|
||||
# The media servers' sidecars live in the metadata overlay and did not follow
|
||||
# these moves, so the run names that cleanup at the moment it is needed.
|
||||
[[ "$output" == *"media-view.sh reap"* ]]
|
||||
run python3 "$APPLY" verify --plan "$OUT" --root "$LIB" --journal "$JOURNAL"
|
||||
[ "$status" -eq 0 ]
|
||||
}
|
||||
@@ -356,6 +517,33 @@ PY
|
||||
[ -d "$LIB/A Empty Placeholder" ]
|
||||
}
|
||||
|
||||
@test "a swept folder carries the empty junk folders inside it" {
|
||||
mkdir -p "$LIB/AMERICAN POP/Screens" # screenshots, never a move source
|
||||
plan
|
||||
apply_plan
|
||||
run python3 "$APPLY" sweep --root "$LIB" --journal "$JOURNAL" --apply
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" == *"0 left in place"* ]]
|
||||
[ ! -e "$LIB/AMERICAN POP/Screens" ]
|
||||
[ ! -e "$LIB/AMERICAN POP" ]
|
||||
}
|
||||
|
||||
@test "a sweep whose rmdir failed tries those folders again" {
|
||||
plan
|
||||
apply_plan
|
||||
mkdir -p "$LIB/AMERICAN POP/Screens"
|
||||
chmod 500 "$LIB/AMERICAN POP" # readable, but nothing can be removed
|
||||
run python3 "$APPLY" sweep --root "$LIB" --journal "$JOURNAL" --apply
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" == *"left in place"* ]]
|
||||
[ -e "$LIB/AMERICAN POP/Screens" ]
|
||||
chmod 700 "$LIB/AMERICAN POP" # the operator cleared the blocker
|
||||
run python3 "$APPLY" sweep --root "$LIB" --journal "$JOURNAL" --apply
|
||||
[ "$status" -eq 0 ]
|
||||
[[ "$output" != *"left in place: "* ]]
|
||||
[ ! -e "$LIB/AMERICAN POP" ]
|
||||
}
|
||||
|
||||
@test "sweep without --apply reports the husks and changes nothing" {
|
||||
plan
|
||||
apply_plan
|
||||
|
||||
Executable
+307
@@ -0,0 +1,307 @@
|
||||
#!/usr/bin/env bats
|
||||
# Seam: the TV-library relayout — scripts/media-census.py jellyfin (the metadata
|
||||
# authority export) and scripts/media-shows.py (the plan).
|
||||
#
|
||||
# The TV library is 52k files written in five different release layouts, and the
|
||||
# one thing that makes a rename of that size survivable is that the plan is
|
||||
# derived from what a metadata provider already matched, not from a guess at the
|
||||
# file name. So every test here states which authority a file had — name only,
|
||||
# provider only, both, neither — and asserts where the PLAN files it, never how
|
||||
# the planner decided. A fixture Jellyfin database stands in for the live one and
|
||||
# the TMDb cache is an input file, so no test touches the network.
|
||||
#
|
||||
# The two outcomes that cost a re-run of a 50k-file plan are a file filed as an
|
||||
# episode that is not one (it steals an episode number from its real owner) and a
|
||||
# move that lands on an existing file. Everything below is written to catch
|
||||
# exactly those, plus the gate that matters most: the plan must be replayable by
|
||||
# scripts/media-apply.py, whose preflight is the fleet's only overwrite guard.
|
||||
|
||||
load helpers
|
||||
|
||||
PLAN="$REPO_ROOT/scripts/media-shows.py"
|
||||
CENSUS="$REPO_ROOT/scripts/media-census.py"
|
||||
APPLY="$REPO_ROOT/scripts/media-apply.py"
|
||||
|
||||
# at PATH SIZE — a file with deterministic content.
|
||||
at() { mkdir -p "$(dirname "$1")"; head -c "$2" /dev/zero > "$1"; }
|
||||
|
||||
# title_row FILE SERIES YEAR SEASON EPISODE [EXTRA] — one row of the authority
|
||||
# export scripts/media-census.py jellyfin writes.
|
||||
title_row() {
|
||||
printf '%s\t%s\t%s\t%s\t%s\t%s\t\n' "$1" "$2" "$3" "${4:-}" "${5:-}" "${6:-}" \
|
||||
>> "$TITLES"
|
||||
}
|
||||
|
||||
# dst SRC — the destination the plan gives SRC, or nothing when it does not move.
|
||||
dst() {
|
||||
awk -F '\t' -v s="$1" '$1 == "move" && $2 == s {print $3}' "$OUT"
|
||||
}
|
||||
|
||||
setup() {
|
||||
LIB="$BATS_TEST_TMPDIR/TVShows"
|
||||
PARK="$LIB/_unsorted"
|
||||
MOVIES="$BATS_TEST_TMPDIR/Movies"
|
||||
ENUM="$BATS_TEST_TMPDIR/enum.tsv"
|
||||
PROBED="$BATS_TEST_TMPDIR/probed.tsv"
|
||||
TITLES="$BATS_TEST_TMPDIR/titles.tsv"
|
||||
CACHE="$BATS_TEST_TMPDIR/tmdb.json"
|
||||
OUT="$BATS_TEST_TMPDIR/plan.tsv"
|
||||
EXC="$BATS_TEST_TMPDIR/exceptions.txt"
|
||||
: > "$PROBED"
|
||||
|
||||
SEASON="$LIB/The Examples (2010) Season 1 S01 (1080p BluRay x265 Silence)"
|
||||
# An episode its own name states, with a subtitle, a season image and a
|
||||
# trailer: four things that have to end up in four different places.
|
||||
at "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.mkv" 9000
|
||||
at "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.en.srt" 200
|
||||
at "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.mkv.forced.srt" 250
|
||||
# A scene pack buries its subtitles two levels down, in Subs/<release-name>/.
|
||||
at "$SEASON/Subs/The.Examples.S01E01.Pilot.1080p.x265/The.Examples.S01E01.Pilot.1080p.x265.jpn.srt" 220
|
||||
at "$SEASON/folder.jpg" 300
|
||||
at "$SEASON/Trailers/teaser.mkv" 1000
|
||||
at "$SEASON/tvshow.nfo" 400
|
||||
# An anime pack: no episode marker anywhere in its name, numbered by the
|
||||
# provider only. Its year is the one thing no authority states.
|
||||
ANIME="$LIB/Anime Pack [grp] (1080p HEVC)"
|
||||
at "$ANIME/[grp] Anime Pack - 05.mkv" 5000
|
||||
# Two releases claiming one episode, and a file holding two episodes.
|
||||
at "$LIB/Dupes/Show Two S01E01.mkv" 4000
|
||||
at "$LIB/Dupes/Show.Two.S01E01.720p.other.mkv" 7000
|
||||
at "$LIB/Dupes/Show.Two.S01E02-E03.720p.mkv" 7000
|
||||
# A file no authority numbers, and a long file the movie provider knows.
|
||||
at "$LIB/Orphan Tapes/clip001.mp4" 3000
|
||||
at "$LIB/Longform/Some Documentary Special (2015).mkv" 8000
|
||||
# A bonus folder inside a release pack whose own name is not the show's. The
|
||||
# pack's rows name the show; the bonus folder may never invent a series.
|
||||
at "$ANIME/01. Extras/NCOP 01.mkv" 1200
|
||||
# A season folder whose rows are stale: the database numbers every file of this
|
||||
# pack season 16 episode 20, four seasons' worth of them.
|
||||
at "$LIB/SNL Pack/Season 19/01 Charles Barkley, Nirvana.avi" 6000
|
||||
# One episode-shaped file loose at the library root.
|
||||
at "$LIB/Loose.Show.S02E03.720p.WEB-DL.mkv" 4000
|
||||
|
||||
# The enum census lists every file; the planner reads that instead of walking
|
||||
# the tree, so the fixture writes the same three columns.
|
||||
python3 - "$LIB" "$ENUM" <<'PY'
|
||||
import os, sys
|
||||
root, out = sys.argv[1], sys.argv[2]
|
||||
with open(out, "w") as fh:
|
||||
for d, _, fs in os.walk(root):
|
||||
for f in fs:
|
||||
p = os.path.join(d, f)
|
||||
fh.write(f"{os.stat(p).st_size}\t{os.path.splitext(f)[1]}\t{p}\n")
|
||||
PY
|
||||
|
||||
printf 'file\tseries\tyear\tseason\tepisode\textra\ttmdb\n' > "$TITLES"
|
||||
printf '{}\n' > "$CACHE"
|
||||
title_row "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.mkv" \
|
||||
"The Examples" 2010 1 1
|
||||
title_row "$ANIME/[grp] Anime Pack - 05.mkv" "Anime Pack" "" 1 5
|
||||
title_row "$LIB/Dupes/Show Two S01E01.mkv" "Show Two" 2016 1 1
|
||||
title_row "$LIB/Dupes/Show.Two.S01E01.720p.other.mkv" "Show Two" 2016 1 1
|
||||
title_row "$LIB/Dupes/Show.Two.S01E02-E03.720p.mkv" "Show Two" 2016 1 2
|
||||
title_row "$LIB/Loose.Show.S02E03.720p.WEB-DL.mkv" "Loose Show" 2019 2 3
|
||||
title_row "$LIB/SNL Pack/Season 19/01 Charles Barkley, Nirvana.avi" \
|
||||
"SNL Pack" "" 16 20
|
||||
}
|
||||
|
||||
plan() {
|
||||
run python3 "$PLAN" --root "$LIB" --enum "$ENUM" --titles "$TITLES" \
|
||||
--probed "$PROBED" --tmdb-cache "$CACHE" --movies "$MOVIES" \
|
||||
--out "$OUT" --exceptions "$EXC"
|
||||
}
|
||||
|
||||
@test "the census exports the provider's episode numbering as a TSV" {
|
||||
DB="$BATS_TEST_TMPDIR/jellyfin.db"
|
||||
python3 - "$DB" "$SEASON" <<'PY'
|
||||
import os, sqlite3, sys, uuid
|
||||
db, season = sys.argv[1], sys.argv[2]
|
||||
c = sqlite3.connect(db)
|
||||
pre = "/data/TVShows/" + os.path.basename(season)
|
||||
c.execute("""create table BaseItems (Id, ParentId, Type, Name, Path, IndexNumber,
|
||||
ParentIndexNumber, ProductionYear, ExtraType,
|
||||
IsVirtualItem, Data)""")
|
||||
sid = str(uuid.uuid4())
|
||||
c.execute("insert into BaseItems values (?,?,?,?,?,?,?,?,?,?,?)",
|
||||
(sid, None, "MediaBrowser.Controller.Entities.TV.Series", "The Examples",
|
||||
pre, None, None, 2010, None, 0,
|
||||
'{"ProviderIds":{"Tmdb":"1234"}}'))
|
||||
c.execute("insert into BaseItems values (?,?,?,?,?,?,?,?,?,?,?)",
|
||||
(str(uuid.uuid4()), sid, "MediaBrowser.Controller.Entities.TV.Episode",
|
||||
"Pilot", pre + "/The.Examples.S01E01.Pilot.1080p.x265.mkv", 1, 1,
|
||||
None, None, 0, '{}'))
|
||||
c.commit()
|
||||
PY
|
||||
run python3 "$CENSUS" jellyfin --root "$LIB" --db "$DB" \
|
||||
--out "$BATS_TEST_TMPDIR/exp.tsv"
|
||||
[ "$status" -eq 0 ]
|
||||
run head -1 "$BATS_TEST_TMPDIR/exp.tsv"
|
||||
[ "${lines[0]}" = "file series year season episode extra tmdb" ]
|
||||
tail -1 "$BATS_TEST_TMPDIR/exp.tsv" \
|
||||
| grep -q "^$SEASON/The\.Examples\.S01E01\.Pilot\.1080p\.x265\.mkv The Examples 2010 1 1 1234$"
|
||||
}
|
||||
|
||||
@test "an episode its own name states files as Show SxxEyy under Season NN" {
|
||||
plan
|
||||
[ "$status" -eq 0 ]
|
||||
[ "$(dst "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.mkv")" \
|
||||
= "$LIB/The Examples (2010)/Season 01/The Examples S01E01.mkv" ]
|
||||
}
|
||||
|
||||
@test "a subtitle follows its episode and keeps its language flag" {
|
||||
plan
|
||||
[ "$(dst "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.en.srt")" \
|
||||
= "$LIB/The Examples (2010)/Season 01/The Examples S01E01.en.srt" ]
|
||||
}
|
||||
|
||||
@test "a subtitle two folders down inside Subs/<release-name>/ still follows its episode" {
|
||||
plan
|
||||
[ "$(dst "$SEASON/Subs/The.Examples.S01E01.Pilot.1080p.x265/The.Examples.S01E01.Pilot.1080p.x265.jpn.srt")" \
|
||||
= "$LIB/The Examples (2010)/Season 01/The Examples S01E01.jpn.srt" ]
|
||||
}
|
||||
|
||||
@test "a subtitle that carries the container's word in its own name still follows the episode" {
|
||||
plan
|
||||
[ "$(dst "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.mkv.forced.srt")" \
|
||||
= "$LIB/The Examples (2010)/Season 01/The Examples S01E01.forced.srt" ]
|
||||
}
|
||||
|
||||
@test "a season image keeps its own name; a series nfo keeps the series folder" {
|
||||
plan
|
||||
[ "$(dst "$SEASON/folder.jpg")" = "$LIB/The Examples (2010)/Season 01/folder.jpg" ]
|
||||
[ "$(dst "$SEASON/tvshow.nfo")" = "$LIB/The Examples (2010)/tvshow.nfo" ]
|
||||
}
|
||||
|
||||
@test "bonus material files as an extra, never as an episode" {
|
||||
plan
|
||||
[ "$(dst "$SEASON/Trailers/teaser.mkv")" = "$LIB/The Examples (2010)/trailers/teaser.mkv" ]
|
||||
[ -z "$(awk -F '\t' '$3 ~ /Season .*teaser/ {print $3}' "$OUT")" ]
|
||||
}
|
||||
|
||||
@test "a file only the provider numbers files at the episode it is matched to" {
|
||||
plan
|
||||
[ "$(dst "$ANIME/[grp] Anime Pack - 05.mkv")" \
|
||||
= "$LIB/Anime Pack/Season 01/Anime Pack S01E05.mkv" ]
|
||||
}
|
||||
|
||||
@test "a bonus folder inside a release pack belongs to the pack's show, not to its own name" {
|
||||
plan
|
||||
[ "$(dst "$ANIME/01. Extras/NCOP 01.mkv")" = "$LIB/Anime Pack/extras/NCOP 01.mkv" ]
|
||||
[ -z "$(awk -F '\t' '$3 ~ /\/01\. Extras\// {print $3}' "$OUT")" ]
|
||||
}
|
||||
|
||||
@test "a Season folder outvotes the stale row its files carry" {
|
||||
plan
|
||||
[ "$(dst "$LIB/SNL Pack/Season 19/01 Charles Barkley, Nirvana.avi")" \
|
||||
= "$LIB/SNL Pack/Season 19/SNL Pack S19E01.avi" ]
|
||||
}
|
||||
|
||||
|
||||
@test "one file holding two episodes keeps both numbers in its name" {
|
||||
plan
|
||||
[ "$(dst "$LIB/Dupes/Show.Two.S01E02-E03.720p.mkv")" \
|
||||
= "$LIB/Show Two (2016)/Season 01/Show Two S01E02-E03.mkv" ]
|
||||
}
|
||||
|
||||
@test "two files claiming one episode: one keeps the name, the other parks" {
|
||||
plan
|
||||
n=$(grep -c " $LIB/Show Two (2016)/Season 01/Show Two S01E01.mkv " "$OUT")
|
||||
[ "$n" -eq 1 ]
|
||||
run dst "$LIB/Dupes/Show Two S01E01.mkv"
|
||||
[[ "$output" == "$PARK/"* ]]
|
||||
run dst "$LIB/Dupes/Show.Two.S01E01.720p.other.mkv"
|
||||
[[ "$output" == "$LIB/Show Two (2016)/Season 01/"* ]]
|
||||
}
|
||||
|
||||
@test "a file no authority numbers is parked, never renamed into the library" {
|
||||
plan
|
||||
run dst "$LIB/Orphan Tapes/clip001.mp4"
|
||||
[[ "$output" == "$PARK/Orphan Tapes/clip001.mp4" ]]
|
||||
grep -q "clip001.mp4" "$EXC"
|
||||
}
|
||||
|
||||
@test "a long file the movie provider knows leaves the TV library for the movie one" {
|
||||
printf '8000\t%s\t7200\thevc\t1920x1080\t23.976\t4000000\tyuv420p10le\taac:eng\t0\t0\tmatroska,webm\n' \
|
||||
"$LIB/Longform/Some Documentary Special (2015).mkv" > "$PROBED"
|
||||
python3 -c 'import json, sys
|
||||
json.dump({"movie:some documentary":
|
||||
{"name": "Some Documentary Special", "year": 2015, "id": 55}},
|
||||
open(sys.argv[1], "w"))' "$CACHE"
|
||||
plan
|
||||
[ "$(dst "$LIB/Longform/Some Documentary Special (2015).mkv")" \
|
||||
= "$MOVIES/Some Documentary Special (2015)/Some Documentary Special (2015).mkv" ]
|
||||
}
|
||||
|
||||
@test "a long file no movie provider knows is parked, not moved into Movies" {
|
||||
printf '8000\t%s\t7200\thevc\t1920x1080\t23.976\t4000000\tyuv420p10le\taac:eng\t0\t0\tmatroska,webm\n' \
|
||||
"$LIB/Longform/Some Documentary Special (2015).mkv" > "$PROBED"
|
||||
plan
|
||||
run dst "$LIB/Longform/Some Documentary Special (2015).mkv"
|
||||
[[ "$output" == "$PARK/"* ]]
|
||||
}
|
||||
|
||||
@test "the plan creates every folder before the move that needs it" {
|
||||
plan
|
||||
python3 - "$OUT" "$LIB" "$MOVIES" "$PARK" <<'PY'
|
||||
import os, sys
|
||||
plan, known = sys.argv[1], set(sys.argv[2:])
|
||||
made, bad = set(), []
|
||||
for line in open(plan).read().splitlines()[1:]:
|
||||
action, _src, dest, _kind = line.split("\t")
|
||||
if action == "mkdir":
|
||||
made.add(dest)
|
||||
elif action == "move":
|
||||
d = os.path.dirname(dest)
|
||||
while d not in known and d != os.path.dirname(d):
|
||||
if d not in made:
|
||||
bad.append(f"{dest}: {d} was never made")
|
||||
d = os.path.dirname(d)
|
||||
print("\n".join(bad[:5]))
|
||||
sys.exit(1 if bad else 0)
|
||||
PY
|
||||
}
|
||||
|
||||
@test "no move lands outside the library or the movie library" {
|
||||
plan
|
||||
run awk -F '\t' -v l="$LIB" -v m="$MOVIES" \
|
||||
'$1 == "move" && $3 !~ "^" l "/" && $3 !~ "^" m "/" {print $3}' "$OUT"
|
||||
[ -z "$output" ]
|
||||
}
|
||||
|
||||
@test "the executor preflights the plan and no target is claimed twice" {
|
||||
plan
|
||||
run python3 "$APPLY" apply --plan "$OUT" --root "$BATS_TEST_TMPDIR" \
|
||||
--journal "$BATS_TEST_TMPDIR/journal.jsonl"
|
||||
[ "$status" -eq 0 ]
|
||||
dup=$(awk -F '\t' '$1 == "move" {print $3}' "$OUT" | sort | uniq -d)
|
||||
[ -z "$dup" ]
|
||||
}
|
||||
|
||||
@test "re-planning a library the plan already satisfies moves nothing" {
|
||||
plan
|
||||
# Applying the plan and re-censusing is exactly what the operator does next:
|
||||
# every authority file is keyed by path, so all three are replayed through the
|
||||
# plan the way a fresh census would see them. (The enum has no header; the
|
||||
# titles export does, and its header row is not a path so it survives.)
|
||||
python3 - "$OUT" "$ENUM" "$TITLES" "$PROBED" <<'PY'
|
||||
import os, sys
|
||||
plan = sys.argv[1]
|
||||
moves = {}
|
||||
for line in open(plan).read().splitlines()[1:]:
|
||||
action, src, dest, _kind = line.split("\t")
|
||||
if action == "move":
|
||||
moves[src] = dest
|
||||
os.makedirs(os.path.dirname(dest), exist_ok=True)
|
||||
os.rename(src, dest)
|
||||
for path, col in ((sys.argv[2], 2), (sys.argv[3], 0), (sys.argv[4], 1)):
|
||||
if not os.path.getsize(path):
|
||||
continue
|
||||
rows = [line.rstrip("\n").split("\t") for line in open(path)]
|
||||
with open(path, "w") as fh:
|
||||
for f in rows:
|
||||
f[col] = moves.get(f[col], f[col])
|
||||
fh.write("\t".join(f) + "\n")
|
||||
PY
|
||||
plan
|
||||
[ -z "$(awk -F '\t' '$1 == "move" {print $2}' "$OUT")" ]
|
||||
}
|
||||
Reference in new issue
Block a user