0 Fork 0

Files

T

Frank 50e0c5a786 docs(media): the metadata view is live, and duplicate pruning gets a page

`Media Metadata View.md` said "Not deployed on Dankserver — verified
2026-10-31". That was true yesterday and is false now: the overlay is mounted
and both installed quadlets bind it (`view:/data` rw for Jellyfin,
`view:/mnt/entropy:ro` for Plex). The page now states the live shape and the
order that proves it — `media-view.sh verify`, then the `Volume=` lines —
because the whole point of the page is to answer "is the view really serving?".
`AGENTS.md` carried the same stale claim in its summary table.

`Media Duplicate Prune.md` is new: the screen ladder (size, then head+tail
blake2b, then a sparse-window confirmation before anything moves), which copy
survives and why, the guards that each encode a file that would otherwise have
been destroyed (qBittorrent `.!qB` state, year-required with digits kept,
feature length, disc/part exclusion, media containers only), and the law that a
hold on the same volume frees nothing until it is purged. It also carries the
different way `media-verify.py` has to be READ for a prune — `landed: 0` is the
correct end state because no server may index the hold — and the planner bug
that class explains: when the planner re-derived "largest wins" from the hold
list alone it dropped every keeper out of its own group and pruned 17 files
instead of 205.

`Media Relayout.md` keeps a one-line pointer; carrying the whole procedure there
pushed it past the vault's 500-line ceiling, which is the linter telling the
truth about progressive disclosure.

2026-11-01 01:11:26 -07:00

7.9 KiB

Raw Blame History

type, title, description, tags, timestamp

type

title

description

Media Metadata View

/var/lib/media-meta/view is an OverlayFS merge of the media library: the RAID6 array below, a directory on the NVMe above. It exists so Jellyfin can write Kodi-format sidecars (movie.nfo, tvshow.nfo, episode.nfo, artwork) that Plex can read. Sidecar metadata is only useful sitting next to the media file, and the array is read-only and slow — so the media bytes stay there and the writes land on SSD. One tree to both servers, one copy of the media.

The mount point is not a volume and holds nothing while unmounted. It sits inside /var/lib/media-meta so that layers, kernel bookkeeping, the reap trash and the merged door are one directory — and so the mount table grows no new /mnt entry next to the array.

Deployed on Dankserver — verified 2026-11-01. mount shows the overlay on /var/lib/media-meta/view (lowerdir=/mnt/entropy, upperdir=/var/lib/media-meta/upper), and the installed quadlets under ~/.config/containers/systemd/ carry the view binds: Volume=/var/lib/media-meta/view:/data for Jellyfin, Volume=/var/lib/media-meta/view:/mnt/entropy:ro for Plex. Prove it in that order: scripts/media-view.sh verify (the deploy gate), then grep -n Volume ~/.config/containers/systemd/{jellyfin,plex}.container.

The cutover itself is an ordinary deploy: deploy.sh dankserver --service jellyfin|plex runs ensure_media_view (install then verify) before it restarts the container, so it must not be fired through the middle of a library scan. Two consequences are now live rather than theoretical: inotify events for array writes do not cross the merge, so any relayout or prune needs a manual library refresh; and a move can leave ghost sidecars above, so reap runs after a relayout — a pass that moves only media files with no sidecars next to them leaves the upper layer untouched, and reap correctly finds nothing.

Shape

Item	Value
Lower layer	`/mnt/entropy` — the RAID6 array, source of truth, never written
Upper layer	`/var/lib/media-meta/upper` — NVMe; every write lands here
Work dir	`/var/lib/media-meta/work` — overlayfs internals, same filesystem
Mount point	`/var/lib/media-meta/view` — the merged tree both servers bind
Unit (repo)	`dankserver/host/systemd/media-meta-view.mount` — the source of truth
Unit (host)	`var-lib-media\x2dmeta-view.mount` — systemd names a mount unit after its escaped path (`-` becomes `\x2d`); `media-view.sh install` writes the repo file under that name
Installer	`scripts/media-view.sh` — `plan` / `install` / `verify` / `status` / `reap`
Written by	Jellyfin only (Plex binds read-only); nothing else in the fleet touches it

Both consumers guard the view with UnitConfigItem=ConditionPathIsMountPoint=/var/lib/media-meta/view, and scripts/deploy.sh dankserver --service jellyfin|plex calls media-view.sh install + verify before it restarts either container.

Consumer	Bind	Mode	Why that path
Jellyfin	`/var/lib/media-meta/view:/data`	rw	It is the metadata writer; its sidecars become SSD bytes
Plex	`/var/lib/media-meta/view:/mnt/entropy`	ro	Every path in the PMS database was recorded as `/mnt/entropy/...`; only the HOST side of the bind moved, so its database still resolves

Putting the sidecars inside Plex's reach is not the same as making Plex use them: a library parses .nfo only with its Local media information agent enabled, and this change touches no library setting.

Commands

scripts/media-view.sh plan              # print the resolved shape, touch nothing
sudo scripts/media-view.sh install      # create dirs, install the unit, mount, enable
scripts/media-view.sh verify            # is the view really the overlay? (deploy gate)
scripts/media-view.sh status            # mount options + upper-layer size/file count
scripts/media-view.sh reap              # list ghost metadata (media moved, sidecars left)
sudo scripts/media-view.sh reap --apply # move ghosts to the SSD trash, never delete

--upper-base DIR and --lower DIR relocate the layers for plan and reap; the defaults are the host values above.

Laws

The upper layer must sit on non-rotational storage. install and verify refuse via lsblk -dno ROTA if the upper base lives on the array: an overlay whose upper is the spinners is the original problem with extra steps, and it silently turns every metadata write back into a seek.
Never mount the view over an unmounted lower. RequiresMountsFor=/mnt/entropy plus the container ConditionPathIsMountPoint guards: an empty lowerdir presents an empty library as truth, and a media scan would then clear the database of everything. systemd does not create a mount point, so install creates view itself; without it the mount simply fails.
Never delete through the view. A delete writes an overlayfs whiteout (.wh.<name>) in the upper layer, which hides the array's copy without removing it — the file reappears the moment the view is unmounted, and the upper layer now carries a landmine. Deletions belong on the raw path (/mnt/entropy), which the relayout tool already treats as the only writer. reap refuses to remove whiteouts and only warns about them.
A relayout must be followed by reap. Renames happen on the raw path, so sidecars the view already wrote stay at the old coordinates: a ghost folder the servers browse and find nothing in. reap lists upper-layer directories with no counterpart below and moves them (with their subtree) into a dated .AGENTTRASH/ under the SSD base. It is recursive and file-safe: a sidecar that exists only above is normal, so only directories are candidates.
File-watcher events do not cross the overlay. Plex's NFO agent and Jellyfin's real-time monitor see inotify events for writes that land in the upper layer, not for changes on the array underneath. After any relayout or externally-written metadata, trigger a library refresh — see Media Relayout for the refresh procedure.

Failure signatures

Symptom	Meaning
`jellyfin.service` / `plex.service` show `ConditionPathIsMountPoint=... was not met` and stay inactive	The view is not mounted. Do not start them anyway: `scripts/media-view.sh install`, then `verify`.
Jellyfin log: `Read-only file system : '/data/Movies/…/movie.nfo'`	The container is binding the raw array, not the view — the quadlet drifted back or the view is gone. `media-view.sh status`.
A folder appears in a media server with artwork/`.nfo` but no video	Ghost metadata from a relayout. `media-view.sh reap`.
`verify` prints `refused: …/view is not a mountpoint`	A plain directory is sitting where the overlay belongs. That is the fail-closed guard working; `install` on the box.
`systemctl status` finds no such unit	Look under the escaped name, `var-lib-media\x2dmeta-view.mount`, not the repo file name.

Rollback

Revert the commit that pointed the two quadlets at the view and re-deploy both services — they bind /mnt/entropy read-only again, which is the pre-view shape. The NVMe upper layer can stay: nothing reads it once the binds are back, and it holds no state either server needs. Unmounting alone (systemctl stop 'var-lib-media\x2dmeta-view.mount') is safe while no container binds it, and the array is never modified by any of this.

7.9 KiB Raw Blame History