dank
e82c8b2d06
Dankserver rebooted with the array's mount and the media containers racing. A rootless quadlet cannot order itself against the system's mnt-entropy.mount, so `RequiresMountsFor=/mnt/entropy` is inert in the user manager: podman resolved the bind against the pre-mount directory, Jellyfin and Plex came up blind, and Jellyfin read an empty library and began `Removing item, Type: "Season"` by the thousand. Copyparty had no guard at all. `scripts/wait-for-mount.sh` is the gate: it waits for every named mountpoint, then refuses by name. `ConditionPathIsMountPoint=` is NOT the fix — measured on the host, an unmet condition logs `skipped, unmet condition check` with NRestarts=0 and no start attempt ever follows, which is fail-dead, not fail-closed. Jellyfin and Plex swap the condition for the gate; Copyparty gets one; the law is recorded in AGENTS.md and the view page. Verified on the host: the 5.7.0 quadlet generator accepts ExecStartPre in [Service] (exit 0, all three units generated, gate present), the gate waits for a mount that appears mid-window and proceeds, and it refuses a plain directory after its window naming the path. tests/media_overlay.bats pinned the old mechanism by name; its intent — both media servers refuse to start without the view — now rides the gate.
142 lines
8.6 KiB
Markdown
142 lines
8.6 KiB
Markdown
---
|
|
type: reference
|
|
title: Media Metadata View
|
|
description: "The SSD-backed overlay the two media servers read and write: shape, guards, commands, laws, rollback."
|
|
tags: [dankserver, media-server, jellyfin, plex, storage]
|
|
timestamp: 2026-10-30T00:00:00Z
|
|
---
|
|
|
|
# Media Metadata View
|
|
|
|
`/var/lib/media-meta/view` is an OverlayFS merge of the media library: the RAID6
|
|
array below, a directory on the NVMe above. It exists so [[Jellyfin]] can write
|
|
Kodi-format sidecars (`movie.nfo`, `tvshow.nfo`, `episode.nfo`, artwork) that
|
|
[[Plex]] can read. Sidecar metadata is only useful sitting next to the media
|
|
file, and the array is read-only and slow — so the media bytes stay there and the
|
|
*writes* land on SSD. One tree to both servers, one copy of the media.
|
|
|
|
The mount point is not a volume and holds nothing while unmounted. It sits inside
|
|
`/var/lib/media-meta` so that layers, kernel bookkeeping, the reap trash and the
|
|
merged door are one directory — and so the mount table grows no new `/mnt` entry
|
|
next to the array.
|
|
|
|
**Not deployed on Dankserver — verified 2026-10-31.** `scripts/media-view.sh
|
|
status` answers `view: /var/lib/media-meta/view NOT MOUNTED`,
|
|
`/var/lib/media-meta` does not exist, and the installed quadlets under
|
|
`~/.config/containers/systemd/` still carry the pre-view binds — `/mnt/entropy:/data:ro`
|
|
for [[Jellyfin]], `/mnt/entropy:/mnt/entropy:ro` for [[Plex]]. Everything below
|
|
is the shape git describes, not the shape the servers run. The cutover is an
|
|
ordinary deploy: `deploy.sh dankserver --service jellyfin|plex` runs
|
|
`ensure_media_view` (`install` then `verify`) before it restarts the container,
|
|
so it must not be fired through the middle of a library scan. Until that deploy
|
|
lands there is no upper layer, so no relayout can leave ghost metadata and
|
|
`reap` has nothing to do — `media-apply.py` prints that follow-up only when the
|
|
view really is a mountpoint.
|
|
|
|
## Shape
|
|
|
|
| Item | Value |
|
|
|---|---|
|
|
| Lower layer | `/mnt/entropy` — the RAID6 array, source of truth, never written |
|
|
| Upper layer | `/var/lib/media-meta/upper` — NVMe; every write lands here |
|
|
| Work dir | `/var/lib/media-meta/work` — overlayfs internals, same filesystem |
|
|
| Mount point | `/var/lib/media-meta/view` — the merged tree both servers bind |
|
|
| Unit (repo) | `dankserver/host/systemd/media-meta-view.mount` — the source of truth |
|
|
| Unit (host) | `var-lib-media\x2dmeta-view.mount` — systemd names a mount unit after its escaped path (`-` becomes `\x2d`); `media-view.sh install` writes the repo file under that name |
|
|
| Installer | `scripts/media-view.sh` — `plan` / `install` / `verify` / `status` / `reap` |
|
|
| Written by | Jellyfin only (Plex binds read-only); nothing else in the fleet touches it |
|
|
|
|
Both consumers gate their start on the view with
|
|
`ExecStartPre=/home/dank/serverAppConfigs/scripts/wait-for-mount.sh` (it waits
|
|
for the mount, then refuses), and `scripts/deploy.sh dankserver --service
|
|
jellyfin|plex` calls `media-view.sh install` + `verify` before it restarts
|
|
either container.
|
|
|
|
| Consumer | Bind | Mode | Why that path |
|
|
|---|---|---|---|
|
|
| [[Jellyfin]] | `/var/lib/media-meta/view:/data` | rw | It is the metadata writer; its sidecars become SSD bytes |
|
|
| [[Plex]] | `/var/lib/media-meta/view:/mnt/entropy` | ro | Every path in the PMS database was recorded as `/mnt/entropy/...`; only the HOST side of the bind moved, so its database still resolves |
|
|
|
|
Putting the sidecars inside Plex's reach is not the same as making Plex use them:
|
|
a library parses `.nfo` only with its *Local media information* agent enabled, and
|
|
this change touches no library setting.
|
|
|
|
## Commands
|
|
|
|
```bash
|
|
scripts/media-view.sh plan # print the resolved shape, touch nothing
|
|
sudo scripts/media-view.sh install # create dirs, install the unit, mount, enable
|
|
scripts/media-view.sh verify # is the view really the overlay? (deploy gate)
|
|
scripts/media-view.sh status # mount options + upper-layer size/file count
|
|
scripts/media-view.sh reap # list ghost metadata (media moved, sidecars left)
|
|
sudo scripts/media-view.sh reap --apply # move ghosts to the SSD trash, never delete
|
|
```
|
|
|
|
`--upper-base DIR` and `--lower DIR` relocate the layers for `plan` and `reap`;
|
|
the defaults are the host values above.
|
|
|
|
## Laws
|
|
|
|
- **The upper layer must sit on non-rotational storage.** `install` and `verify`
|
|
refuse via `lsblk -dno ROTA` if the upper base lives on the array: an overlay
|
|
whose upper is the spinners is the original problem with extra steps, and it
|
|
silently turns every metadata write back into a seek.
|
|
- **A container that binds the array must refuse to start until the mount
|
|
answers.** Live 2026-10-31: the array's mount and the media containers raced at
|
|
boot, podman resolved the bind against the pre-mount directory, and both
|
|
servers came up blind — Jellyfin read an empty library and began
|
|
`Removing item, Type: "Season"` by the thousand. Two dependencies people reach
|
|
for do not work here, both measured on that host: `RequiresMountsFor=` in a
|
|
**rootless** unit is inert (the user manager cannot order against the system's
|
|
`mnt-entropy.mount`), and `ConditionPathIsMountPoint=` is fail-*dead* — an
|
|
unmet condition logs `skipped, unmet condition check` with `NRestarts=0` and no
|
|
start attempt ever follows, so the unit waits for a human. The working guard is
|
|
`scripts/wait-for-mount.sh` as `ExecStartPre`: it waits out the race, refuses by
|
|
name after its window, and `Restart=always` carries the unit from there.
|
|
systemd does not create a mount point, so `install` creates `view` itself;
|
|
without it the mount simply fails.
|
|
- **Never delete through the view.** A delete writes an overlayfs whiteout
|
|
(`.wh.<name>`) in the upper layer, which hides the array's copy without
|
|
removing it — the file reappears the moment the view is unmounted, and the
|
|
upper layer now carries a landmine. Deletions belong on the raw path
|
|
(`/mnt/entropy`), which the relayout tool already treats as the only writer.
|
|
`reap` refuses to remove whiteouts and only warns about them.
|
|
- **A relayout must be followed by `reap`.** Renames happen on the raw path, so
|
|
sidecars the view already wrote stay at the old coordinates: a ghost folder
|
|
the servers browse and find nothing in. `reap` lists upper-layer directories
|
|
with no counterpart below and moves them (with their subtree) into a dated
|
|
`.AGENTTRASH/` under the SSD base. It is recursive and file-safe: a sidecar
|
|
that exists only above is normal, so only *directories* are candidates.
|
|
- **File-watcher events do not cross the overlay.** Plex's NFO agent and
|
|
Jellyfin's real-time monitor see inotify events for writes that land *in the
|
|
upper layer*, not for changes on the array underneath. After any relayout or
|
|
externally-written metadata, trigger a library refresh — see
|
|
[[Media Relayout]] for the refresh procedure.
|
|
|
|
## Failure signatures
|
|
|
|
| Symptom | Meaning |
|
|
|---|---|
|
|
| `jellyfin.service` / `plex.service` fail with `refused: still not mounted after 90s: …` | The view or the array is not mounted. Do not start them anyway: `scripts/media-view.sh install`, then `verify`. |
|
|
| Servers answer, library looks empty, Jellyfin logs `Removing item, Type: "Season"` by the thousand | Blind bind — the container started before the mount and bound the pre-mount directory. `podman exec jellyfin ls /data` against `ls /mnt/entropy` is the whole diagnosis; restarting the container re-resolves the bind. |
|
|
| Jellyfin log: `Read-only file system : '/data/Movies/…/movie.nfo'` | The container is binding the raw array, not the view — the quadlet drifted back or the view is gone. `media-view.sh status`. |
|
|
| A folder appears in a media server with artwork/`.nfo` but no video | Ghost metadata from a relayout. `media-view.sh reap`. |
|
|
| `verify` prints `refused: …/view is not a mountpoint` | A plain directory is sitting where the overlay belongs. That is the fail-closed guard working; `install` on the box. |
|
|
| `systemctl status` finds no such unit | Look under the escaped name, `var-lib-media\x2dmeta-view.mount`, not the repo file name. |
|
|
|
|
## Rollback
|
|
|
|
Revert the commit that pointed the two quadlets at the view and re-deploy both
|
|
services — they bind `/mnt/entropy` read-only again, which is the pre-view shape.
|
|
The NVMe upper layer can stay: nothing reads it once the binds are back, and it
|
|
holds no state either server needs. Unmounting alone
|
|
(`systemctl stop 'var-lib-media\x2dmeta-view.mount'`) is safe while no container
|
|
binds it, and the array is never modified by any of this.
|
|
|
|
## Links
|
|
|
|
- [[Jellyfin]] — the metadata writer; its data layout section.
|
|
- [[Plex]] — the read-only consumer; in-place bind-mount history.
|
|
- [[Media Relayout]] — the tool that moves media on the raw path and the reason `reap` exists.
|
|
- [[Entropy Volume]] — the array underneath.
|