Merge remote-tracking branch 'origin/main' into feat/jellyfin-provider-keys
# Conflicts: # AGENTS.md # serverAppConfigs-docs/Dankserver/Media Metadata View.md # serverAppConfigs-docs/index.md
This commit is contained in:
Frank committed
commit
14f340370c
70 files changed
+1898
-4941
No files matched your search
@@ -100,3 +100,12 @@ aborts with `DEPLOY INCOMPLETE: e2e gate FAILED` when it does not pass
|
||||
hand-off list (`HANDOFFS`) instead of a single host. The guard contract is
|
||||
untouched: challenge cleared, no authentik `uidField`, status < 500, and every
|
||||
host NOT in the list still has to land on its own vhost.
|
||||
7. **The runner's resolver is part of the suite.** A 2026-10-31 deploy of one
|
||||
Dankserver quadlet failed the gate with 21 reds spread over SSO, homepage RBAC
|
||||
and every vhost crawl — precisely the specs that navigate to an absolute
|
||||
`*.franksdank.cloud` name, while every spec on the relative `baseURL` stayed
|
||||
green. The error was `net::ERR_NAME_NOT_RESOLVED`: the workstation had lost its
|
||||
wired link, and on the fallback its resolver was no longer pinned to Pi-hole.
|
||||
Nothing on the fleet was wrong — the same suite re-run minutes later passed
|
||||
37/37. Before a mass red gets blamed on the guard chain, run
|
||||
`resolvectl query grafana.franksdank.cloud` on the box that runs the gate.
|
||||
@@ -167,6 +167,43 @@ Gate: `tests/quadlet_image_pins.bats`. Where upstream publishes no version
|
||||
tags (copyparty-ac stops at 1.19.15), pin `:latest@sha256:` and name the
|
||||
version the config label carries in the comment beside it.
|
||||
|
||||
## 4b. Resource control needs host permission (measured 2026-10-31, Dankserver)
|
||||
|
||||
`IOWeight=` in a rootless quadlet does nothing on its own, and nothing says so.
|
||||
The system manager delegates `cpu memory pids` to `user@.service` and **not** `io`;
|
||||
systemd drops a resource line it cannot honour **without a word** — the generator
|
||||
still emits `IOWeight=10`, the container still starts and answers its health check,
|
||||
and the cgroup simply has no `io.weight` file. Same family as
|
||||
`RequiresMountsFor=` (inert in the user manager) and `ConditionPathIsMountPoint=`
|
||||
(fail-dead): the line is accepted, the unit is happy, the setting is dead.
|
||||
|
||||
Two system drop-ins make it real, both placed by `ensure_user_io_delegate` in
|
||||
`scripts/deploy-remote.sh` (repo copies under `dankserver/host/systemd/`):
|
||||
|
||||
| File | Effect |
|
||||
|---|---|
|
||||
| `user.slice.d/io-accounting.conf` → `[Slice] IOAccounting=yes` | puts `io` into `user.slice/cgroup.subtree_control`; lands immediately, no restart |
|
||||
| `user@1000.service.d/io-delegate.conf` → `[Service] DelegateControllers=cpu memory pids io` | the user manager can only request controllers it was delegated; read when it starts, so it lands on **the next reboot** (restarting it kills every rootless container on the host) |
|
||||
|
||||
Read the chain in this order — each line is the failure mode of the one after it:
|
||||
|
||||
```bash
|
||||
cat /sys/fs/cgroup/user.slice/cgroup.subtree_control # wants: io
|
||||
systemctl show user@1000.service -p DelegateControllers # wants: io
|
||||
cat /sys/fs/cgroup/user.slice/user-1000.slice/user@1000.service/app.slice/\<unit\>.service/libpod-payload-*/io.weight
|
||||
```
|
||||
|
||||
`io` missing from the first → the accounting drop-in is absent. Missing from the
|
||||
second → the host has not rebooted since the delegation landed (the deployer prints
|
||||
`NOTE: io is not delegated …` for exactly this). No file at the third → that unit
|
||||
carries no weight.
|
||||
|
||||
Two limits that survive the fix: a container's weight covers **everything** in its
|
||||
cgroup, so a library scan and a playback stream share one weight; and the comparison
|
||||
is made *inside* `user.slice`, so a rootless unit's weight cannot push it below a
|
||||
root unit (`frank-archon`). Shaping that pairing is the weight of `user.slice`
|
||||
itself — a fleet decision, not a unit line.
|
||||
|
||||
## 5. Rootless Quadlet log retrieval
|
||||
|
||||
Step 0 — establish ownership: `systemctl --user status <name>.service`
|
||||
|
||||
@@ -1,51 +1,68 @@
|
||||
---
|
||||
type: reference
|
||||
description: "The Dankserver entropy volume is a RAID-backed LUKS filesystem mounted at"
|
||||
description: "Dankserver's /mnt/entropy volume: the LUKS/array chain, the crypttab + fstab boot path, and the containers bound to it"
|
||||
tags:
|
||||
- dankserver
|
||||
timestamp: 2026-08-28
|
||||
timestamp: 2026-10-31
|
||||
---
|
||||
|
||||
# Entropy Volume
|
||||
|
||||
The Dankserver entropy volume is a RAID-backed LUKS filesystem mounted at
|
||||
`/mnt/entropy`. The command-line helper reuses the host's desktop-integrated
|
||||
`udisks2`, `crypttab`, and `fstab` configuration; it does not store or handle
|
||||
the passphrase itself.
|
||||
`/mnt/entropy` is the ext4 filesystem on the LUKS container holding `/dev/md0`
|
||||
(RAID6, seven disks). The media stack's data lives there, so the mount is a boot
|
||||
dependency for those containers rather than a convenience.
|
||||
|
||||
## Command-line mount
|
||||
|
||||
```bash
|
||||
/home/dank/serverAppConfigs/scripts/mount-entropy.sh
|
||||
```
|
||||
|
||||
Run it as the normal desktop user (`dank`), not from a root shell. `udisksctl`
|
||||
will securely prompt for the LUKS passphrase. `sudo mount /mnt/entropy` then
|
||||
uses the existing `/etc/fstab` entry, preserving its options:
|
||||
## Chain
|
||||
|
||||
```text
|
||||
nosuid,nodev
|
||||
/dev/md0 <- mdadm (its mdadm.conf ARRAY is what makes md0 exist at boot)
|
||||
-> LUKS UUID=cd86c728-459e-4f3a-bcc2-41ecf84685ec keyfile: /etc/entropy
|
||||
-> /dev/mapper/entropy <- systemd-cryptsetup@entropy.service
|
||||
-> ext4 at /mnt/entropy <- mnt-entropy.mount, from /etc/fstab
|
||||
```
|
||||
|
||||
The existing mountpoint is owned by `dank:dankserver` with mode `0755`; the
|
||||
helper deliberately does not run `chmod` or `chown`, so it cannot silently
|
||||
change those permissions.
|
||||
## Files that own it
|
||||
|
||||
## Device path
|
||||
| Repo file | Installs to | Produces at boot |
|
||||
|---|---|---|
|
||||
| `dankserver/host/crypttab` | `/etc/crypttab` | `systemd-cryptsetup@entropy.service` → `/dev/mapper/entropy` |
|
||||
| `dankserver/host/fstab` | `/etc/fstab` | `mnt-entropy.mount` (mountpoint `dank:dankserver`, `0755`) |
|
||||
|
||||
The current storage chain is:
|
||||
Deploy a change by copying the file onto the host, then
|
||||
`systemctl daemon-reload && update-initramfs -u`. The initramfs step is not
|
||||
optional: the cryptsetup hook bakes `/etc/crypttab` into the initrd, and the
|
||||
array is unlocked there, before the real root mounts. `scripts/mount-entropy.sh`
|
||||
(`udisksctl unlock` + `mount`) stays as the manual path for a volume that was
|
||||
never unlocked; on a correct boot it is redundant.
|
||||
|
||||
```text
|
||||
/dev/md0 -> LUKS UUID cd86c728-459e-4f3a-bcc2-41ecf84685ec
|
||||
-> ext4 label entropy
|
||||
-> /mnt/entropy
|
||||
```
|
||||
## Law
|
||||
|
||||
The helper checks that `/dev/md0` exists and exits successfully if the volume
|
||||
is already mounted. It is intentionally specific to this host and volume:
|
||||
that avoids accidentally unlocking or mounting a different disk.
|
||||
**`crypttab`'s device field takes the LUKS UUID, never the md array UUID.**
|
||||
`blkid` prints both on adjacent lines for a RAID-backed container, and the file
|
||||
shipped since 2025 carried the array UUID (`61bd6eaf-…`) where the LUKS container
|
||||
is `cd86c728-…`. The generator produced a perfectly valid unit pointing at a
|
||||
device that never appears, so the mapper came into existence only through a
|
||||
desktop `udisks2` session — the volume looked mounted on every login while every
|
||||
boot failed to mount it. The matching `fstab` line named the same wrong UUID
|
||||
directly instead of the mapper.
|
||||
|
||||
## Related
|
||||
**A LUKS keyfile is hashed whole, so a trailing newline is a different
|
||||
passphrase.** The shipped `/etc/entropy` was 27 bytes — 26 characters plus `\n` —
|
||||
and `cryptsetup` rejected it even after the UUID was corrected. Keyfiles stay
|
||||
`0600` with no trailing newline; prove it with
|
||||
`cryptsetup open --test-passphrase --key-file /etc/entropy UUID=…` *before*
|
||||
relying on it for a boot.
|
||||
|
||||
- [[Storage Policy]]
|
||||
- [[Dankserver Stack]]
|
||||
**Prove a mount fix with the artifacts systemd generates, not with a live
|
||||
mount.** The regenerated `systemd-cryptsetup@entropy.service` naming the right
|
||||
device, `findmnt /mnt/entropy` sourcing `/dev/mapper/entropy`, and a matching
|
||||
checksum between the repo copy and the host copy are the evidence. A mount that
|
||||
exists because someone logged in proves nothing about the boot path — that
|
||||
exactly describes the year this bug survived.
|
||||
|
||||
## Consumers
|
||||
|
||||
`jellyfin`, `copyparty`, and `plex` bind paths under `/mnt/entropy`; with the
|
||||
volume absent they start `active (healthy)` against empty directories, which is
|
||||
how a missing 91 TiB volume hides. See [[Dankserver Stack]] for the stack and
|
||||
[[Storage Policy]] for what may be written here.
|
||||
@@ -18,7 +18,7 @@ clients.
|
||||
public vhost `jelly.franksdank.cloud` behind the [[Anubis]] guard chain →
|
||||
[[Authentik]] outpost (post-auth route proxies `localhost:8096`).
|
||||
|
||||
## Facts (the repo quadlet; the live unit is still the pre-view generation — see Data layout)
|
||||
## Facts (the repo quadlet — deployed as the live unit on 2026-10-31)
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
@@ -26,7 +26,7 @@ public vhost `jelly.franksdank.cloud` behind the [[Anubis]] guard chain →
|
||||
| Image | `ghcr.io/jellyfin/jellyfin:latest` |
|
||||
| Network | `dankserver_appnet`; publishes 8096/tcp, 8920/tcp, 7359/udp, 1900/udp |
|
||||
| Data roots | `/home/dank/Containers/jellyfin/config:/config`, `/home/dank/Containers/jellyfin/cache:/cache` |
|
||||
| Media | `Volume=/var/lib/media-meta/view:/data` in git — the [[Media Metadata View]] overlay, **writable**, sidecars land on the NVMe and the [[Entropy Volume]] array underneath stays untouched. **The live unit is still pre-view: `/mnt/entropy:/data:ro`** — nothing is written into the library until the view is deployed |
|
||||
| Media | `Volume=/var/lib/media-meta/view:/data` — the [[Media Metadata View]] overlay, **writable**, live since the 2026-10-31 cutover: sidecars land on the NVMe and the [[Entropy Volume]] array underneath stays untouched |
|
||||
| GPU | none — no `AddDevice`; CPU decode only (jellyfin-ffmpeg path) |
|
||||
| Env quirks | `DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true`, `LD_PRELOAD=libjemalloc.so.2`, `MALLOC_TRIM_THRESHOLD_=131072` kept verbatim from the compose-era container (zero-drift cutover) |
|
||||
| Health | `curl -f http://localhost:8096/health` every 30 s |
|
||||
@@ -37,26 +37,34 @@ public vhost `jelly.franksdank.cloud` behind the [[Anubis]] guard chain →
|
||||
/home/dank/Containers/jellyfin/config -> /config (JELLYFIN_DATA_DIR)
|
||||
/home/dank/Containers/jellyfin/cache -> /cache (JELLYFIN_CACHE_DIR, XDG_CACHE_HOME)
|
||||
/var/lib/media-meta/view -> /data (git: the overlay view)
|
||||
/mnt/entropy -> /data (live today: the array, read-only)
|
||||
/mnt/entropy -> /data (the overlay's lower layer; also the rollback bind)
|
||||
```
|
||||
|
||||
Library folders point at `/data` paths: stable across container recreation and
|
||||
identical to the raw array layout below. Which side of that path the container
|
||||
is on is the open question on this host — the installed unit binds the read-only
|
||||
array, and `scripts/media-view.sh status` reports `NOT MOUNTED` (2026-10-31).
|
||||
sees is the [[Media Metadata View]] overlay, live since the 2026-10-31 cutover:
|
||||
`/var/lib/media-meta/view:/data`, read-write.
|
||||
|
||||
`Local metadata` — save metadata to media folders — is enabled, and it is the
|
||||
reason the view exists: `movie.nfo`, `tvshow.nfo`, `episode.nfo` and artwork
|
||||
written once on the NVMe upper layer are files [[Plex]] can read too. While
|
||||
`/data` is bound read-only every item update logs the error below and no sidecar
|
||||
is written; Jellyfin becomes the fleet's only metadata writer with the deploy
|
||||
that raises the view.
|
||||
written once on the NVMe upper layer are files [[Plex]] can read too — and since
|
||||
the 2026-10-31 cutover that is what happens: Jellyfin is the fleet's only
|
||||
metadata writer, and every sidecar it saves is an SSD byte the array never sees.
|
||||
|
||||
**Library coverage is the first thing to check when a title is missing.** The
|
||||
four libraries are `movies → /data/Movies`, `tvshows → /data/TVShows`,
|
||||
`music → /data/Music`, `boxsets → /config/data/collections`
|
||||
(`GET /Library/VirtualFolders`). `/data/media-park` is in **none** of them, so
|
||||
everything filed under it — the `_unsorted` release bundles and the `extras/`
|
||||
folders the relayout wrote there — is invisible to Jellyfin by configuration,
|
||||
not by fault: a recursive refresh answers 204, burns CPU for minutes and
|
||||
indexes nothing, because it never looks there. Check coverage before re-scanning.
|
||||
|
||||
**Failure signature:** `System.IO.IOException: Read-only file system :
|
||||
'/data/Movies/<film>/movie.nfo'` means the container is binding the raw array
|
||||
instead of the view. On Dankserver that is today's expected state — the view has
|
||||
never been installed (`scripts/media-view.sh status`). Once it is deployed, the
|
||||
same line is a fault: the quadlet drifted or the mount died.
|
||||
instead of the view — always a fault on this host since the 2026-10-31 cutover.
|
||||
`scripts/media-view.sh status` says which: the quadlet drifted back to the raw
|
||||
array, or the mount died.
|
||||
|
||||
## Ports & access
|
||||
|
||||
@@ -212,8 +220,7 @@ receives `Authorization: MediaBrowser Token=` with nothing after it (`\\"` answe
|
||||
One accepted refresh is the whole procedure. A second one while a scan runs moves
|
||||
`Scan Media Library` to `Cancelling`, and once the runner holds `Cancelling` every
|
||||
later `204` is accepted and then discarded — the status code stops meaning
|
||||
anything. The fix is `systemctl --user restart jellyfin.service` (never
|
||||
`deploy.sh`: that runs the never-tested metadata-view cutover), one POST after
|
||||
anything. The fix is `systemctl --user restart jellyfin.service`, one POST after
|
||||
health answers, and then believe `ScheduledTasks` State at 130 % CPU, not the
|
||||
response code.
|
||||
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
---
|
||||
type: reference
|
||||
title: Media Audiobook Relayout
|
||||
description: "Re-filing /mnt/entropy/Audiobooks by Plex's chapter rows: narrator shelves, no renumbering, the refusals."
|
||||
tags: [dankserver, media-server, plex, media-relayout]
|
||||
timestamp: 2026-10-31T00:00:00Z
|
||||
---
|
||||
|
||||
# Media Audiobook Relayout
|
||||
|
||||
`/mnt/entropy/Audiobooks` is the one library here that [[Jellyfin]] does not read
|
||||
at all — its music scanner has no audiobook concept — while [[Plex]] indexes all
|
||||
of it as Artist → Album → Track: the album is the BOOK, that book's single artist
|
||||
is the NARRATOR, both matched out of the files' own tags. Today one book is split
|
||||
across `… [Part 1]`, `… [Part 2]` folders three or four deep inside a collection
|
||||
pack, so a scan serves three quarters of a novel; `scripts/media-audiobooks.py`
|
||||
re-files the tree so the property both servers' book scanners assume is true —
|
||||
|
||||
Audiobooks/<Narrator>/<Book> (<year>)/<the file keeps its own name>
|
||||
|
||||
The shared gates, the executor, the journal law and the rollback live in
|
||||
[[Media Relayout]]; this page is the audiobook arm only.
|
||||
|
||||
```bash
|
||||
# 1. The authority: Plex's chapter rows for this root (the DB opens read-only).
|
||||
python3 scripts/media-census.py audiobooks --root /mnt/entropy/Audiobooks \
|
||||
--out /tmp/ab_rows.tsv
|
||||
# 2. What is on disk (stat only).
|
||||
python3 scripts/media-census.py enumerate --root /mnt/entropy/Audiobooks \
|
||||
--out /tmp/ab_enum.tsv
|
||||
# 3. The plan.
|
||||
python3 scripts/media-audiobooks.py --root /mnt/entropy/Audiobooks \
|
||||
--enum /tmp/ab_enum.tsv --rows /tmp/ab_rows.tsv --out /tmp/ab_plan.tsv \
|
||||
--exceptions /tmp/ab_exceptions.txt
|
||||
# 4. Read the exceptions file FIRST. Then the same executor, journal and rollback
|
||||
# as [[Media Relayout]] shows, on its own journal, and refresh the Plex section
|
||||
# as in step 5 of the movie Procedure — from inside the container, never over
|
||||
# the host port.
|
||||
scripts/media-apply.py apply --plan /tmp/ab_plan.tsv --root /mnt/entropy \
|
||||
--journal /mnt/entropy/media-relayout-audiobooks.jsonl --apply
|
||||
```
|
||||
|
||||
The group is the identity Plex asserts in its own database — **(narrator, book,
|
||||
year)** — so one book arriving in four part folders becomes one folder, and a
|
||||
1965 and a 2020 recording that share a title stay two books.
|
||||
|
||||
Two laws are forced by what the rows actually hold:
|
||||
|
||||
+- **A chapter is never renumbered.** Most books contain chapters whose index Plex
|
||||
does not know, so a number written into a name would be invented — and it would
|
||||
buy nothing, because both servers order playback by the embedded track tag, not
|
||||
by the file name. A wrong number in a name is the number a human later trusts.
|
||||
Fix order in the tags: a Plex scan re-reads tags and never renames a file.
|
||||
+- **The shelf is the narrator, not the author.** Every book has exactly one artist
|
||||
row and it is the narrator; the author survives only inside some album titles
|
||||
(`Kryptos by Graham McNeill`) and in no column Plex stores, so deriving it would
|
||||
mean parsing titles where a ripper wrote one and inventing it everywhere else.
|
||||
The author stays visible in the folder name Plex itself shows.
|
||||
|
||||
What it refuses, the music arm's list with one addition: a video container, audio
|
||||
no chapter row names, and a loose file in a folder holding two books all park under
|
||||
`_unsorted` with the path they have today — and so does a chapter whose row names
|
||||
**no book or no narrator**: a placeholder (`[Unknown Album]`, `[Unknown Artist]`)
|
||||
that unrelated books share, or a chapter number mistagged as the artist (`0.4`).
|
||||
`Various Artists` is not a placeholder — that is a multi-cast read.
|
||||
|
||||
The reviewed plan lives at `/mnt/entropy/media-relayout-audiobooks-plan.tsv`; it
|
||||
applies on its own journal, behind the music writer, and then refreshes the Plex
|
||||
section. Verify it with `--db-kind plex` — Jellyfin holds no rows here, so a
|
||||
Jellyfin pass proves nothing either way.
|
||||
|
||||
## See Also
|
||||
|
||||
[[Media Relayout]], [[Media Music Relayout]], [[Plex]]
|
||||
@@ -20,7 +20,16 @@ The mount point is not a volume and holds nothing while unmounted. It sits insid
|
||||
merged door are one directory — and so the mount table grows no new `/mnt` entry
|
||||
next to the array.
|
||||
|
||||
**Deployed on Dankserver — verified 2026-11-01.** `mount` shows the overlay on
|
||||
**Deployed on Dankserver — 2026-10-31, both servers.** `plex` binds
|
||||
`/var/lib/media-meta/view:/mnt/entropy:ro` and `jellyfin` binds
|
||||
`/var/lib/media-meta/view:/data:rw`; `findmnt` reports the overlay with
|
||||
`lowerdir=/mnt/entropy upperdir=/var/lib/media-meta/upper`, and each container
|
||||
was proven to resolve a real media file through it — `podman exec <svc> ls -la
|
||||
<its data path>` is the whole proof, because both servers keep the paths their
|
||||
databases recorded. Two service-scoped deploys, each gated by
|
||||
`media-view.sh verify` and the Playwright suite.
|
||||
|
||||
Re-verified 2026-11-01: `mount` shows the overlay on
|
||||
`/var/lib/media-meta/view` (`lowerdir=/mnt/entropy`,
|
||||
`upperdir=/var/lib/media-meta/upper`), and the installed quadlets under
|
||||
`~/.config/containers/systemd/` carry the view binds:
|
||||
@@ -29,6 +38,14 @@ next to the array.
|
||||
order: `scripts/media-view.sh verify` (the deploy gate), then
|
||||
`grep -n Volume ~/.config/containers/systemd/{jellyfin,plex}.container`.
|
||||
|
||||
Moving a bind means restarting the server, and **a restart discards that
|
||||
server's in-flight library scan** — Jellyfin's 7 h 25 m walk ended
|
||||
`"Status": "Aborted"` in `data/ScheduledTasks/<task-id>.js` and nothing resumes
|
||||
it. Renames made on the array while the view was unmounted are invisible to both
|
||||
watchers too, so the additive index has to be asked for by hand; the door is an
|
||||
API key, and the row in the failure table below says what to check when it
|
||||
refuses.
|
||||
|
||||
The cutover itself is an ordinary deploy: `deploy.sh dankserver --service
|
||||
jellyfin|plex` runs `ensure_media_view` (`install` then `verify`) before it
|
||||
restarts the container, so it must not be fired through the middle of a library
|
||||
@@ -51,10 +68,11 @@ leaves the upper layer untouched, and `reap` correctly finds nothing.
|
||||
| Installer | `scripts/media-view.sh` — `plan` / `install` / `verify` / `status` / `reap` |
|
||||
| Written by | Jellyfin only (Plex binds read-only); nothing else in the fleet touches it |
|
||||
|
||||
Both consumers guard the view with
|
||||
`UnitConfigItem=ConditionPathIsMountPoint=/var/lib/media-meta/view`, and
|
||||
`scripts/deploy.sh dankserver --service jellyfin|plex` calls `media-view.sh
|
||||
install` + `verify` before it restarts either container.
|
||||
Both consumers gate their start on the view with
|
||||
`ExecStartPre=/home/dank/serverAppConfigs/scripts/wait-for-mount.sh` (it waits
|
||||
for the mount, then refuses), and `scripts/deploy.sh dankserver --service
|
||||
jellyfin|plex` calls `media-view.sh install` + `verify` before it restarts
|
||||
either container.
|
||||
|
||||
| Consumer | Bind | Mode | Why that path |
|
||||
|---|---|---|---|
|
||||
@@ -85,11 +103,20 @@ the defaults are the host values above.
|
||||
refuse via `lsblk -dno ROTA` if the upper base lives on the array: an overlay
|
||||
whose upper is the spinners is the original problem with extra steps, and it
|
||||
silently turns every metadata write back into a seek.
|
||||
- **Never mount the view over an unmounted lower.** `RequiresMountsFor=/mnt/entropy`
|
||||
plus the container `ConditionPathIsMountPoint` guards: an empty `lowerdir`
|
||||
presents an *empty library* as truth, and a media scan would then clear the
|
||||
database of everything. systemd does not create a mount point, so `install`
|
||||
creates `view` itself; without it the mount simply fails.
|
||||
- **A container that binds the array must refuse to start until the mount
|
||||
answers.** Live 2026-10-31: the array's mount and the media containers raced at
|
||||
boot, podman resolved the bind against the pre-mount directory, and both
|
||||
servers came up blind — Jellyfin read an empty library and began
|
||||
`Removing item, Type: "Season"` by the thousand. Two dependencies people reach
|
||||
for do not work here, both measured on that host: `RequiresMountsFor=` in a
|
||||
**rootless** unit is inert (the user manager cannot order against the system's
|
||||
`mnt-entropy.mount`), and `ConditionPathIsMountPoint=` is fail-*dead* — an
|
||||
unmet condition logs `skipped, unmet condition check` with `NRestarts=0` and no
|
||||
start attempt ever follows, so the unit waits for a human. The working guard is
|
||||
`scripts/wait-for-mount.sh` as `ExecStartPre`: it waits out the race, refuses by
|
||||
name after its window, and `Restart=always` carries the unit from there.
|
||||
systemd does not create a mount point, so `install` creates `view` itself;
|
||||
without it the mount simply fails.
|
||||
- **Never delete through the view.** A delete writes an overlayfs whiteout
|
||||
(`.wh.<name>`) in the upper layer, which hides the array's copy without
|
||||
removing it — the file reappears the moment the view is unmounted, and the
|
||||
@@ -112,10 +139,14 @@ the defaults are the host values above.
|
||||
|
||||
| Symptom | Meaning |
|
||||
|---|---|
|
||||
| `jellyfin.service` / `plex.service` show `ConditionPathIsMountPoint=... was not met` and stay inactive | The view is not mounted. Do not start them anyway: `scripts/media-view.sh install`, then `verify`. |
|
||||
| `jellyfin.service` / `plex.service` fail with `refused: still not mounted after 90s: …` | The view or the array is not mounted. Do not start them anyway: `scripts/media-view.sh install`, then `verify`. |
|
||||
| Servers answer, library looks empty, Jellyfin logs `Removing item, Type: "Season"` by the thousand | Blind bind — the container started before the mount and bound the pre-mount directory. `podman exec jellyfin ls /data` against `ls /mnt/entropy` is the whole diagnosis; restarting the container re-resolves the bind. |
|
||||
| Jellyfin log: `Read-only file system : '/data/Movies/…/movie.nfo'` | The container is binding the raw array, not the view — the quadlet drifted back or the view is gone. `media-view.sh status`. |
|
||||
| A folder appears in a media server with artwork/`.nfo` but no video | Ghost metadata from a relayout. `media-view.sh reap`. |
|
||||
| `verify` prints `refused: …/view is not a mountpoint` | A plain directory is sitting where the overlay belongs. That is the fail-closed guard working; `install` on the box. |
|
||||
| `install` prints `refused: upper base /var/lib/media-meta does not exist (put it on the SSD first)` | A fresh host. `install` deliberately never creates the base — a mistyped `--upper-base` must not build a metadata tree on the wrong filesystem — so `scripts/media-view.sh ensure-base` makes the documented one, which the deployer calls first. |
|
||||
| A media server's scan record reads `"Status": "Aborted"` | Its container restarted (a cutover, an image bump, an OOM) and the walk was discarded; systemd's `Restart=always` restarts the *server*, never the scan. Trigger a refresh — see [[Media Relayout]]. |
|
||||
| The Jellyfin API answers 401 to every token header | Almost always the header **form**, not the key: 12.4 authenticates only `Authorization: MediaBrowser Token="…"` and refuses a bare `X-Emby-Token` for every key, valid or not — and one collapsed backslash in the curl config produces the same 401. `[[Jellyfin]]` owns the block that works. Before concluding anything, prove the key against the server's own table: `select count(*) from ApiKeys where AccessToken = ?` on `jellyfin.db` opened `mode=ro&immutable=1`. |
|
||||
| `systemctl status` finds no such unit | Look under the escaped name, `var-lib-media\x2dmeta-view.mount`, not the repo file name. |
|
||||
|
||||
## Rollback
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
---
|
||||
type: reference
|
||||
title: Media Music Relayout
|
||||
description: "Re-filing /mnt/entropy/Music by Jellyfin's tag rows: the census, the album key, the refusals."
|
||||
tags: [dankserver, media-server, jellyfin, media-relayout]
|
||||
timestamp: 2026-10-31T00:00:00Z
|
||||
---
|
||||
|
||||
# Media Music Relayout
|
||||
|
||||
`/mnt/entropy/Music` is rips and discography packs, and Jellyfin has matched and
|
||||
tagged almost all of it. `scripts/media-music.py` re-files it into the shape
|
||||
Jellyfin's music scanner reads —
|
||||
|
||||
Music/<album artist>/<Album> (<year>)/NN - <Title>.ext
|
||||
|
||||
— and the authority is the tag row of the track, never the name the ripper gave
|
||||
the file. The shared gates, the executor, the journal law and the rollback live in
|
||||
[[Media Relayout]]; this page is the music arm only.
|
||||
|
||||
```bash
|
||||
# 1. The authority: Jellyfin's audio rows, one per track, with the album and
|
||||
# artist ids its own database carries (the DB opens read-only).
|
||||
python3 scripts/media-census.py music --root /mnt/entropy/Music \
|
||||
--out /tmp/music_rows.tsv
|
||||
# 2. What is on disk (stat only).
|
||||
python3 scripts/media-census.py enumerate --root /mnt/entropy/Music \
|
||||
--out /tmp/music_enum.tsv
|
||||
# 3. The plan.
|
||||
python3 scripts/media-music.py --root /mnt/entropy/Music --enum /tmp/music_enum.tsv \
|
||||
--rows /tmp/music_rows.tsv --out /tmp/music_plan.tsv \
|
||||
--exceptions /tmp/music_exceptions.txt
|
||||
# 4. Read the exceptions file FIRST — it is the review, not a log. Then the same
|
||||
# executor, journal and rollback as [[Media Relayout]] shows, on its own journal.
|
||||
scripts/media-apply.py apply --plan /tmp/music_plan.tsv --root /mnt/entropy \
|
||||
--journal /mnt/entropy/media-relayout-music.jsonl --apply
|
||||
```
|
||||
|
||||
The group is what the tags assert, not where the files sit: **(album-artist tag,
|
||||
album, year)**. One album arrives as three disc folders and as two copies of one
|
||||
rip, and one MusicBrainz release group is one album. A compilation is one album
|
||||
whose tracks name many performers, so it files once under `Various Artists`:
|
||||
keying on the performer would file one album once per artist. Two different
|
||||
album-artist tags on one album name are two albums, which is what Jellyfin's two
|
||||
album items say.
|
||||
|
||||
What it refuses:
|
||||
|
||||
+- **A sidecar is placed by the audio around it, never by its own name.** A
|
||||
`.lrc`, `.txt` or picture that shares a track's stem follows that track and
|
||||
takes the name the track now has, or Jellyfin cannot find it; anything else
|
||||
goes to the album its own folder was placed in, or to the folder whose discs
|
||||
sit one level below it. A folder whose audio belongs to two albums parks its
|
||||
sidecar rather than guessing.
|
||||
+- **An image album keeps its name.** A folder holding one audio file and a
|
||||
`.cue` is a whole album in one file; numbering it `01 - …` tells Jellyfin to
|
||||
index it as one track.
|
||||
+- **A video container parks**, because Jellyfin will not index it as music and a
|
||||
rename to `.m4a` would claim it is; **audio no provider row names parks** too,
|
||||
because unmatched audio is a tagging job, not a renaming job.
|
||||
|
||||
ONE writer at a time on this array: two rename passes race, and neither journal
|
||||
tells. Verify the run against Jellyfin's own tables — see *Proving the move
|
||||
landed* in [[Media Relayout]].
|
||||
|
||||
## See Also
|
||||
|
||||
[[Media Relayout]], [[Jellyfin]], [[Media Audiobook Relayout]]
|
||||
@@ -47,8 +47,10 @@ $C plex --root /mnt/entropy/Movies --out /tmp/plex-movies.tsv
|
||||
```
|
||||
|
||||
`plex` reads the Plex metadata cache (`--db` defaults to the path the
|
||||
[[Plex]] container bind-mounts at `/var/lib/plexmediaserver`) with
|
||||
`mode=ro&immutable=1`: a live server is never locked or written.
|
||||
[[Plex]] container bind-mounts at `/var/lib/plexmediaserver`) with plain
|
||||
`mode=ro`: a live server is never locked or written, and plain ro is also what
|
||||
makes a scan's uncheckpointed rows visible. The same rule as
|
||||
`scripts/media-verify.py` — see *Proving the move landed*.
|
||||
|
||||
Then plan — this reads only the three TSVs, so it costs seconds and can run
|
||||
anywhere, including a laptop:
|
||||
@@ -79,6 +81,13 @@ An apply also leaves the release folders it emptied standing where they were —
|
||||
hundreds of them at the library root — which is what `sweep` removes, dry-run
|
||||
first, from the same journal.
|
||||
|
||||
`--batch N` (default 64) makes the journal durable per batch instead of per
|
||||
move: N intents are written and fsynced, then those N rename. The rule a crash
|
||||
relies on — no rename runs before its intent is on disk — is unchanged, so an
|
||||
interrupted apply still replays the batch and resumes with nothing lost. Batch 1
|
||||
is the old behaviour; the only reason to name it is debugging, because on the
|
||||
spinning array it spends most of a large apply's wall time in fsyncs.
|
||||
|
||||
After it lands, both servers need a scan or the change is invisible. [[Plex]] and
|
||||
[[Jellyfin]] own the exact requests; two laws hold at this step: `204` is
|
||||
acceptance and anything else queued nothing — re-issue the identical request
|
||||
@@ -94,14 +103,34 @@ replayed against both servers' own tables: every `move` row has a row at its
|
||||
replay: it opens each database read-only (there is no `sqlite3` binary on these
|
||||
hosts or inside either container) and exits non-zero while a row is stale.
|
||||
|
||||
Both libraries live in WAL mode, so the tool opens them plain `mode=ro`:
|
||||
`immutable=1` promises sqlite the file never changes, which skips the
|
||||
write-ahead log — a scanner's own uncheckpointed rows then read as missing, and
|
||||
a read beside a running scan can fail as "database disk image is malformed".
|
||||
`immutable` stays only the fallback for a database whose `-shm` cannot be read.
|
||||
|
||||
```bash
|
||||
scripts/media-verify.py --journal /mnt/entropy/media-relayout-music.jsonl \
|
||||
--db-kind jellyfin --ignore-ext jpg,jpeg,png,log,cue,toc,nfo,txt,m3u,accurip
|
||||
```
|
||||
|
||||
Ignore only what a server cannot index. A missing `.flac` or `.mkv` is the
|
||||
fault; a row still naming its `src` means the scan has not followed the rename
|
||||
yet, so re-run the check after the scan, never before.
|
||||
"No row at the destination" is not one verdict but four. **never indexed**: the
|
||||
server indexes no file of that extension (its own extension census says so) — a
|
||||
sidecar, not a lost film. **gone**: the file sits at neither src nor dst, because
|
||||
a later pass or the server's own extension sweep renamed it. **phantom**: a row
|
||||
still naming src while the file has left src. **REAL**: no row at dst *and* the
|
||||
file still at src — the only class that is this relayout's own damage. Phantoms
|
||||
are normal for both servers (Plex carries thousands that predate any relayout);
|
||||
re-scan to clear them and never read them as loss.
|
||||
|
||||
`media-verify.py` prints this split itself — `on disk at dst, no row names it`,
|
||||
`REAL — file still at src`, `gone — at neither path` — so nobody hand-stats a
|
||||
thousand paths to find out whether a film is lost or merely unindexed.
|
||||
|
||||
The owning server decides whether a pass can convict anything: Plex's roots are
|
||||
its `sections` rows joined to their location rows (`Movies`, `TVShows`, `Music`,
|
||||
`Audiobooks`, `LiveRecording`) and Jellyfin holds no audiobook library at all, so
|
||||
the audiobook journal verifies with `--db-kind plex` or it proves nothing.
|
||||
|
||||
Three doors that look right and are not: `GET /search?query=` matches metadata
|
||||
titles, never file paths; `?path=` on `/library/sections/N/all` makes Plex's
|
||||
@@ -140,16 +169,15 @@ from the box-set folder. What it refuses it prints and leaves alone:
|
||||
- a destination two files both claim: the executor refuses the whole plan
|
||||
before its first rename, and only Frank decides which copy is the keeper.
|
||||
- an episode the destination library already holds at the same byte count. The
|
||||
movie share carried a second copy of the whole Monk box set — 124 episodes
|
||||
both sides, the same episode list, every byte count identical, 31.5 GiB each
|
||||
way, and head+tail blake2b identical on a 6-episode sample. The mover leaves
|
||||
such a series in place and says so; a duplicate is deleted or kept by Frank,
|
||||
never copied into one library by a tool.
|
||||
movie share once carried a second copy of a whole box set — same episode list,
|
||||
every byte count identical. The mover leaves such a series in place and says
|
||||
so; a duplicate is deleted or kept by Frank, never copied into one library by
|
||||
a tool.
|
||||
|
||||
A sidecar that shares an episode's own name (`.srt`, `.nfo`) moves with it.
|
||||
|
||||
Below the threshold the mover prints and leaves, so a lone downloaded episode of
|
||||
a real show stays in the movie library on purpose — six did on 2026-10-30.
|
||||
a real show stays in the movie library on purpose.
|
||||
Filing one by hand is the same executor on a hand-written plan and the same
|
||||
journal; the trap is the series year. Read it from the file's own name or the
|
||||
folder it owns, never from the title whose bonus folder it happens to be parked
|
||||
@@ -162,7 +190,8 @@ show. A name whose marker is `Part 2` is a film and stays put.
|
||||
already matched and numbered most of it. `scripts/media-shows.py` re-files that
|
||||
library into the shape Jellyfin reads, with **tvdb as the name authority**:
|
||||
|
||||
<Series> (<year>)/Season NN/<Series> (<year>) - SxxEyy - <Episode Title>.ext
|
||||
<Series> (<year>) [tvdbid-NNNN]/Season NN/
|
||||
<Series> (<year>) - SxxEyy - <Episode Title>.ext
|
||||
|
||||
It never guesses: every series name, episode number and title it writes traces to
|
||||
an authority, and what has none is parked for a human to read.
|
||||
@@ -208,14 +237,21 @@ scripts/media-apply.py apply --plan /mnt/entropy/media-relayout-shows2-plan.tsv
|
||||
|
||||
Evidence, strongest first — the order the planner believes things:
|
||||
|
||||
- the **episode marker the file's own name states** (`S01E02-E03`, `1x04`);
|
||||
- the **episode marker the file's own name states** (`S01E02-E03`, `1x04`); a digit
|
||||
run longer than three at that position is NOT a number — `S03E2019` and
|
||||
`S04E47370355` are one box set's airdate fragment and pack index — so such a file
|
||||
is renumbered by tvdb's title or parks, and is never truncated down to fit;
|
||||
- the **`Season NN` folder it sits in and the number its name leads with inside
|
||||
it**: `Season 19/01 Charles Barkley, Nirvana.avi` is S19E01;
|
||||
- the **provider row**: which episode the file IS, and the numbering of a file
|
||||
whose own name says nothing;
|
||||
- the **row's tvdb id, else its TMDb id carried through TMDb's external ids, else
|
||||
an exact tvdb name search**: that id names the SERIES FOLDER and supplies the
|
||||
episode title, because a name the server read off a folder is not a show name;
|
||||
episode title, because a name the server read off a folder is not a show name.
|
||||
The folder CARRIES the id: `[tvdbid-NNNN]` is what Jellyfin's Tvdb agent and
|
||||
Plex's scanner both read, so a show whose title is ambiguous resolves by number
|
||||
instead of by a human guessing which folder is which. The id rides the folder
|
||||
only — a file name gets 180 characters and the episode title needs the room.
|
||||
- a **bonus folder** anywhere above the file, or Jellyfin's `ExtraType` → the
|
||||
series' canonical extra folder;
|
||||
- a runtime over `SHORT_MAX` makes it film-shaped, and it leaves for `--movies` only
|
||||
@@ -225,7 +261,7 @@ Evidence, strongest first — the order the planner believes things:
|
||||
for the word "russia", *Special Edition I* for a file saying *Edition III*. Else it
|
||||
parks.
|
||||
|
||||
Two refusals carry the design, and a dry run on the real library found both:
|
||||
Four refusals carry the design: what the planner will NOT do is the point of it.
|
||||
|
||||
- **Folders resolve a series; they never invent one.** A folder's words may
|
||||
resolve a file to a series a provider has already named — `(CBB) Tengen Toppa
|
||||
@@ -235,8 +271,27 @@ Two refusals carry the design, and a dry run on the real library found both:
|
||||
- **A scraped row does not outrank the filesystem.** Jellyfin numbers
|
||||
multi-season packs badly: every file under `SNL 16-20/Season 16..19` carries
|
||||
season 16 episode 20, and one Gundam row says season 2004. The folder, then
|
||||
the file's own leading number, win; a row that still insists on season 1989
|
||||
parks its file instead of creating `Season 1989`.
|
||||
the file's own leading number, win.
|
||||
- **A season that is a YEAR is the date matcher, not a season.** Jellyfin files
|
||||
specials by airdate, so a row can arrive saying `season 2002`; that is not a
|
||||
season and it never becomes a folder. The one authority on such a row is tvdb's
|
||||
episode list read backwards: the title the row carries names the (season,
|
||||
episode) tvdb gives it. A title tvdb gives to two episodes names nothing, and
|
||||
the file parks with the reason it always carried. `Season 00` IS a legal
|
||||
season — specials live there — so a row that really says 0 files there instead
|
||||
of parking.
|
||||
- **A season tvdb's own list contradicts gets no tvdb title.** An anime pack
|
||||
numbered by its release group books `S05E01` in a season tvdb does not hold, or
|
||||
books more episodes there than tvdb lists: the slot its number points at is a
|
||||
different episode, and dressing it in that episode's title is a confident lie.
|
||||
The file still renames to the standard shape; the title does not ride, and the
|
||||
series is listed under *seasons whose numbering tvdb contradicts* in the
|
||||
exceptions. A season holding FEWER files than tvdb lists stays plausible — a
|
||||
partial pack is the normal shape. Neither reading holds for a list tvdb
|
||||
cut off at its own page limit: the default episode query answers at most
|
||||
500 rows and states no total, and One Piece, Crayon Shin-chan and
|
||||
Detective Conan each answer exactly 500 while holding over a thousand —
|
||||
a page bounds nothing above itself, so it contradicts nothing.
|
||||
|
||||
**A provider name is evidence only when an id backs it.** Jellyfin names a Series
|
||||
after the folder it scanned, so a library filed by release folders ends up with
|
||||
@@ -258,117 +313,42 @@ visible series folder that real files get renamed into. `--park` outside the roo
|
||||
makes the next plan re-file the pile instead of skipping it. Inside a hidden
|
||||
top-level folder a file's own words may MATCH a registered show, never invent one.
|
||||
|
||||
## The music library's own relayout
|
||||
**The operator's id map is the door for a name tvdb cannot be searched by.**
|
||||
`scripts/media-shows.py` reads `scripts/media-shows-tvdb-ids.tsv` by default: one
|
||||
line per pack folder, `folder <TAB> tvdb id <TAB> note`. The file exists because
|
||||
tvdb titles a record in the show's own language and the English name rides in
|
||||
*aliases* the name search never reads: `ブラッククローバー` (331753) answers to no
|
||||
spelling of "Black Clover", and the bare alias `One Piece` belongs to 392276, the
|
||||
2023 live action, not to the 1999 record (81797) the pack here is. The map
|
||||
outranks a provider id — a box set's provider id is the server's own guess at a
|
||||
folder name — and it merges the release folders that are one show. Everything it
|
||||
touched is written to the exceptions: every folder it named, every row that
|
||||
matched no folder, every disagreement with a provider id, every id tvdb never
|
||||
answered. A line whose id is not a number stops the plan, because a typo renames
|
||||
a whole pack to the wrong show. Only folders the search CANNOT reach belong in
|
||||
it; a row for a folder the search would resolve hides a live lookup behind a
|
||||
number nobody re-checks.
|
||||
|
||||
`/mnt/entropy/Music` is rips and discography packs, and Jellyfin has matched and
|
||||
tagged almost all of it. `scripts/media-music.py` re-files it into the shape
|
||||
Jellyfin's music scanner reads —
|
||||
`scripts/media-shows-tvdb.py` makes the sheet the map is built from. It reads a
|
||||
plan's `mkdir` rows, asks tvdb about every destination folder carrying no
|
||||
`[tvdbid-`, and writes one row per folder with the verdict that says how much was
|
||||
guessed to get there: `EXACT` (tvdb's own spelling), `PUNCT` (equal only after
|
||||
release junk is stripped from the folder's words), `NEAR` (tvdb answered
|
||||
something else — the best hit is printed for a human and nothing applies it),
|
||||
`NONE`. Acceptance never moved across the passes that built this: a hit counts
|
||||
only when tvdb's name equals the query with every non-alphanumeric dropped from
|
||||
both sides, and the record's year still has to fall inside any year span the
|
||||
folder names. That gate is what caught three confident picks — Fairy Tail's 2024
|
||||
sequel (410031), the 2026 Hokuto no Ken remake (465216) and `Mugyutto! Black
|
||||
Clover` (378762) offered for Black Clover. It is why the map is approved by hand
|
||||
and the sheet is never fed to the planner.
|
||||
|
||||
Music/<album artist>/<Album> (<year>)/NN - <Title>.ext
|
||||
## The music and audiobook arms
|
||||
|
||||
— and the authority is the tag row of the track, never the name the ripper gave
|
||||
the file.
|
||||
|
||||
```bash
|
||||
# 1. The authority: Jellyfin's audio rows, one per track, with the album and
|
||||
# artist ids its own database carries (the DB opens read-only).
|
||||
python3 scripts/media-census.py music --root /mnt/entropy/Music \
|
||||
--out /tmp/music_rows.tsv
|
||||
# 2. What is on disk (stat only).
|
||||
python3 scripts/media-census.py enumerate --root /mnt/entropy/Music \
|
||||
--out /tmp/music_enum.tsv
|
||||
# 3. The plan.
|
||||
python3 scripts/media-music.py --root /mnt/entropy/Music --enum /tmp/music_enum.tsv \
|
||||
--rows /tmp/music_rows.tsv --out /tmp/music_plan.tsv \
|
||||
--exceptions /tmp/music_exceptions.txt
|
||||
# 4. Read the exceptions file FIRST — it is the review, not a log. Then the same
|
||||
# executor, journal and rollback as above, on its own journal.
|
||||
scripts/media-apply.py apply --plan /tmp/music_plan.tsv --root /mnt/entropy \
|
||||
--journal /mnt/entropy/media-relayout-music.jsonl --apply
|
||||
```
|
||||
|
||||
The group is what the tags assert, not where the files sit: **(album-artist tag,
|
||||
album, year)**. One album arrives as three disc folders and as two copies of one
|
||||
rip, and one MusicBrainz release group is one album. A compilation is one album
|
||||
whose tracks name many performers, so it files once under `Various Artists`:
|
||||
keying on the performer would file one album once per artist. Two different
|
||||
album-artist tags on one album name are two albums, which is what Jellyfin's two
|
||||
album items say.
|
||||
|
||||
What it refuses:
|
||||
|
||||
+- **A sidecar is placed by the audio around it, never by its own name.** A
|
||||
`.lrc`, `.txt` or picture that shares a track's stem follows that track and
|
||||
takes the name the track now has, or Jellyfin cannot find it; anything else
|
||||
goes to the album its own folder was placed in, or to the folder whose discs
|
||||
sit one level below it. A folder whose audio belongs to two albums parks its
|
||||
sidecar rather than guessing.
|
||||
+- **An image album keeps its name.** A folder holding one audio file and a
|
||||
`.cue` is a whole album in one file; numbering it `01 - …` tells Jellyfin to
|
||||
index it as one track.
|
||||
+- **A video container parks**, because Jellyfin will not index it as music and a
|
||||
rename to `.m4a` would claim it is; **audio no provider row names parks** too,
|
||||
because unmatched audio is a tagging job, not a renaming job.
|
||||
|
||||
ONE writer at a time on this array: two rename passes race, and neither journal
|
||||
tells — which is why each arm above carries its own journal name.
|
||||
|
||||
## The audiobook library's own relayout
|
||||
|
||||
`/mnt/entropy/Audiobooks` is the one library here that [[Jellyfin]] does not read
|
||||
at all — its music scanner has no audiobook concept — while [[Plex]] indexes all
|
||||
of it as Artist → Album → Track: the album is the BOOK, that book's single artist
|
||||
is the NARRATOR, both matched out of the files' own tags. Today one book is split
|
||||
across `… [Part 1]`, `… [Part 2]` folders three or four deep inside a collection
|
||||
pack, so a scan serves three quarters of a novel; `scripts/media-audiobooks.py`
|
||||
re-files the tree so the property both servers' book scanners assume is true —
|
||||
|
||||
Audiobooks/<Narrator>/<Book> (<year>)/<the file keeps its own name>
|
||||
|
||||
```bash
|
||||
# 1. The authority: Plex's chapter rows for this root (the DB opens read-only).
|
||||
python3 scripts/media-census.py audiobooks --root /mnt/entropy/Audiobooks \
|
||||
--out /tmp/ab_rows.tsv
|
||||
# 2. What is on disk (stat only).
|
||||
python3 scripts/media-census.py enumerate --root /mnt/entropy/Audiobooks \
|
||||
--out /tmp/ab_enum.tsv
|
||||
# 3. The plan.
|
||||
python3 scripts/media-audiobooks.py --root /mnt/entropy/Audiobooks \
|
||||
--enum /tmp/ab_enum.tsv --rows /tmp/ab_rows.tsv --out /tmp/ab_plan.tsv \
|
||||
--exceptions /tmp/ab_exceptions.txt
|
||||
# 4. Read the exceptions file FIRST. Then the same executor, journal and rollback
|
||||
# as above, on its own journal, and refresh the Plex section as in step 5 of
|
||||
# the movie Procedure — from inside the container, never over the host port.
|
||||
scripts/media-apply.py apply --plan /tmp/ab_plan.tsv --root /mnt/entropy \
|
||||
--journal /mnt/entropy/media-relayout-audiobooks.jsonl --apply
|
||||
```
|
||||
|
||||
The group is the identity Plex asserts in its own database — **(narrator, book,
|
||||
year)** — so one book arriving in four part folders becomes one folder, and a
|
||||
1965 and a 2020 recording that share a title stay two books.
|
||||
|
||||
Two laws are forced by what the rows actually hold:
|
||||
|
||||
+- **A chapter is never renumbered.** Most books contain chapters whose index Plex
|
||||
does not know, so a number written into a name would be invented — and it would
|
||||
buy nothing, because both servers order playback by the embedded track tag, not
|
||||
by the file name. A wrong number in a name is the number a human later trusts.
|
||||
Fix order in the tags: a Plex scan re-reads tags and never renames a file.
|
||||
+- **The shelf is the narrator, not the author.** Every book has exactly one artist
|
||||
row and it is the narrator; the author survives only inside some album titles
|
||||
(`Kryptos by Graham McNeill`) and in no column Plex stores, so deriving it would
|
||||
mean parsing titles where a ripper wrote one and inventing it everywhere else.
|
||||
The author stays visible in the folder name Plex itself shows.
|
||||
|
||||
What it refuses, the music arm's list with one addition: a video container, audio
|
||||
no chapter row names, and a loose file in a folder holding two books all park under
|
||||
`_unsorted` with the path they have today — and so does a chapter whose row names
|
||||
**no book or no narrator**: a placeholder (`[Unknown Album]`, `[Unknown Artist]`)
|
||||
that unrelated books share, or a chapter number mistagged as the artist (`0.4`).
|
||||
`Various Artists` is not a placeholder — that is a multi-cast read.
|
||||
|
||||
This arm applies behind the music writer — one writer at a time — on its own
|
||||
journal, and then refreshes the Plex section.
|
||||
Each is a page of its own, because each reads a different authority and refuses
|
||||
for a different reason: [[Media Music Relayout]] files by Jellyfin's track tags,
|
||||
[[Media Audiobook Relayout]] by Plex's chapter rows. Both share this page's
|
||||
executor, the one-journal-per-arm law, the sweep and the rollback.
|
||||
|
||||
## Laws
|
||||
|
||||
@@ -442,10 +422,10 @@ journal, and then refreshes the Plex section.
|
||||
scope, so the next sweep finishes what a permission problem stopped.
|
||||
- **An episode-shaped name marks a SERIES, not a film — and the series is the
|
||||
name BEFORE the marker.** `Monk - S05E03 - Mr. Monk and the Big Game` is series
|
||||
`Monk`. Keyed on the whole title instead, every episode became a group of one
|
||||
and never reached the five-episode size that calls something a series, so a
|
||||
box set was filed as 124 movies in 124 folders. The threshold still earns its
|
||||
keep: a lone film named `Part 2` stays a film. Episodes of a series are parked
|
||||
`Monk`. Keyed on the whole title instead, every episode became a group of one,
|
||||
never reaching the five-episode size that calls something a series — a whole
|
||||
box set filed as one folder per episode. The threshold still earns its keep:
|
||||
a lone film named `Part 2` stays a film. Episodes of a series are parked
|
||||
in `_TV needs a TV library - <series>` for a human to move to a TV source —
|
||||
the movie tool never files TV.
|
||||
- **One journal is one undo.** The TV move runs on its own journal, and with
|
||||
@@ -488,6 +468,14 @@ any folder a sweep removed, restores every source path, removes only the folders
|
||||
that run created *and that are now empty*, and refuses everything if any moved
|
||||
file cannot be accounted for. A rolled-back journal refuses to roll back twice.
|
||||
|
||||
**Roll a chain back newest journal first, or the tool refuses.** Each pass
|
||||
re-files what an earlier one moved, so a journal is a clean inverse only until
|
||||
something downstream touches its files; `ls -l /mnt/entropy/media-relayout*.jsonl`
|
||||
is the order. Refusing is the tool working: on 2026-10-31 the movie journal could
|
||||
not restore files the later TV passes had re-filed, and the stray journal could
|
||||
not restore files Jellyfin's extension sweep had renamed. Roll back what still
|
||||
inverts and leave the rest as the current state — never force it.
|
||||
|
||||
## See Also
|
||||
|
||||
[[Jellyfin]], [[Plex]], [[Media Metadata View]], [[Entropy Volume]], [[Testing Policy]], [[Storage Policy]]
|
||||
|
||||
@@ -77,7 +77,7 @@ Both prerequisites survive the dnsmasq → Pi-hole swap. Do not remove them.
|
||||
| Port 53 `bind: permission denied` crash loop after a reboot | sysctl conflict: the value must be 53 at boot, and remember `/etc/sysctl.conf` is applied **after** all of `sysctl.d/`. Redeploy pihole — `deploy_pihole` reinstalls the drop-in, neutralizes conflicting drop-ins, and comments out a stale `/etc/sysctl.conf` line |
|
||||
| Single-label name (for example `dankclaw`) answers `127.0.1.1` | The rootless container `/etc/hosts` carries the host entry (`127.0.1.1 <host>`), and FTL serves it. Use the FQDN, or add an `/etc/hosts` entry on the client |
|
||||
| Every `franksdank.cloud` name is NXDOMAIN for LAN clients while external names still answer | FTL holds no local records: `getent hosts <name>` on Dankclaw comes back empty → `podman exec pihole pihole restartdns reload` (the 2026-10-19 pitfall under **Records**) |
|
||||
| One client gets nothing while every other client is fine | FTL rate-limiting that client: `podman exec pihole grep Rate-limiting /var/log/pihole/FTL.log`. A `10.0.2.x` client address is a pasta container's DNS forwarder, not a machine |
|
||||
| One client gets nothing while every other client is fine | FTL rate-limiting that client: `podman exec pihole grep Rate-limiting /var/log/pihole/FTL.log`. A `10.0.2.x` client address is a pasta container's DNS forwarder, not a machine — and **every** rootless container on that host shares the one address, so a single chatty container throttles DNS for all of them (measured 2026-10-31: 8 throttle events in a day, all `10.0.2.100`) |
|
||||
|
||||
## See Also
|
||||
|
||||
|
||||
@@ -48,11 +48,13 @@ is a locator: one row, one clause, no numbers.
|
||||
| [[Calibre-Web]] | Calibre-Web is a web application that browses, reads, and downloads eBooks from an existing Calibre library database. |
|
||||
| [[Copyparty]] | Copyparty is a lightweight file server and media streamer that runs as a single Python binary. |
|
||||
| [[Dankserver Stack]] | Service inventory and routing overview for the media and library host. |
|
||||
| [[Entropy Volume]] | The Dankserver entropy volume is a RAID-backed LUKS filesystem mounted at |
|
||||
| [[Entropy Volume]] | Dankserver's /mnt/entropy volume: the LUKS/array chain, the crypttab + fstab boot path, and the containers bound to it |
|
||||
| [[Jellyfin]] | Jellyfin on Dankserver: quadlet unit, overlay metadata view, provider keys, ports, access, transcode status. |
|
||||
| [[mariadb]] | The MariaDB backend for RomM: quadlet unit, network alias, env-file secret handling, and the backup path. |
|
||||
| [[Media Audiobook Relayout]] | Re-filing /mnt/entropy/Audiobooks by Plex's chapter rows: narrator shelves, no renumbering, the refusals. |
|
||||
| [[Media Duplicate Prune]] | Find and hold duplicate films on the entropy array: the hash ladder, the keep rules, the guards, the purge. |
|
||||
| [[Media Metadata View]] | The SSD-backed overlay the two media servers read and write: shape, guards, commands, laws, rollback. |
|
||||
| [[Media Music Relayout]] | Re-filing /mnt/entropy/Music by Jellyfin's tag rows: the census, the album key, the refusals. |
|
||||
| [[Media Relayout]] | Census, plan and apply the movie, TV, music and audiobook libraries: the gates, the refusals, the rollback. |
|
||||
| [[Plex]] | Plex Media Server is a **containerized media server on Dankserver**: a podman |
|
||||
| [[redis]] | The Redis cache and task queue for RomM: quadlet unit, network alias topology, and the backup path. |
|
||||
|
||||
Reference in new issue
Block a user