docs(media): the metadata view is live, and duplicate pruning gets a page

`Media Metadata View.md` said "Not deployed on Dankserver — verified
2026-10-31". That was true yesterday and is false now: the overlay is mounted
and both installed quadlets bind it (`view:/data` rw for Jellyfin,
`view:/mnt/entropy:ro` for Plex). The page now states the live shape and the
order that proves it — `media-view.sh verify`, then the `Volume=` lines —
because the whole point of the page is to answer "is the view really serving?".
`AGENTS.md` carried the same stale claim in its summary table.

`Media Duplicate Prune.md` is new: the screen ladder (size, then head+tail
blake2b, then a sparse-window confirmation before anything moves), which copy
survives and why, the guards that each encode a file that would otherwise have
been destroyed (qBittorrent `.!qB` state, year-required with digits kept,
feature length, disc/part exclusion, media containers only), and the law that a
hold on the same volume frees nothing until it is purged. It also carries the
different way `media-verify.py` has to be READ for a prune — `landed: 0` is the
correct end state because no server may index the hold — and the planner bug
that class explains: when the planner re-derived "largest wins" from the hold
list alone it dropped every keeper out of its own group and pruned 17 files
instead of 205.

`Media Relayout.md` keeps a one-line pointer; carrying the whole procedure there
pushed it past the vault's 500-line ceiling, which is the linter telling the
truth about progressive disclosure.

AGENTS.md

@@ -809,6 +809,7 @@ per-attempt page and no per-swap ledger anywhere else.
 | Deployment flow | `serverAppConfigs-docs/Architecture/Deployment Flow.md` |
 | Movie-library relayout (Jellyfin format, Dankserver) | `serverAppConfigs-docs/Dankserver/Media Relayout.md` |
 | SSD-backed metadata view (Jellyfin writes, Plex reads, Dankserver) | `serverAppConfigs-docs/Dankserver/Media Metadata View.md` |
+| Duplicate films and the purge hold (Dankserver) | `serverAppConfigs-docs/Dankserver/Media Duplicate Prune.md` |
 
 There is no page for a retired endpoint and no change diary. A payload swap
 appends one row to the live endpoint's **Eras** table; when the endpoint itself
@@ -851,6 +852,7 @@ history.
 | Benchmarking (only when ordered) | **Stop and mask the service first**, bench, unmask, restart, verify health. |
 | Deleting a file | **Move it to trash** — `.AGENTTRASH/` (in-repo) or `~/.AGENTTRASH/YYYYMMDD-topic`. Never `rm`. |
 | Relayout the movie library | **Census → plan → read the exceptions → `scripts/media-apply.py`** — renames only, journal before every move; never hand-rename, never copy. |
-| Making media metadata visible to both media servers | **Bind `/var/lib/media-meta/view`** — the overlay `scripts/media-view.sh` installs (array below, NVMe above). Never write the array; never delete through the view. **Not yet deployed on Dankserver: the live quadlets still bind the array read-only — see [[Media Metadata View]].** |
+| Duplicates in the library | **Prune into the hold** — byte-identical copies and smaller copies of the *same work* go to `.AGENTTRASH/`; a different cut is never touched. Screen by size, confirm by hash before moving, purge only on Frank's word. See [[Media Duplicate Prune]]. |
+| Making media metadata visible to both media servers | **Bind `/var/lib/media-meta/view`** — the overlay `scripts/media-view.sh` installs (array below, NVMe above). Never write the array; never delete through the view. **Deployed and live on Dankserver since 2026-11-01: both quadlets bind the view, not the raw array — see [[Media Metadata View]].** |
 | Tests exist | **Run them** — verify tests pass after every change. Do not commit failing tests. |
 | Real secrets | **Never commit** — host `.env` files; `.env.example` holds placeholders only. |

serverAppConfigs-docs/Dankserver/Media Duplicate Prune.md

@@ -0,0 +1,128 @@
+---
+type: reference
+title: Media Duplicate Prune
+description: "Find and hold duplicate films on the entropy array: the hash ladder, the keep rules, the guards, the purge."
+tags: [dankserver, media-server, storage, procedure]
+timestamp: 2026-11-01T01:00:00Z
+---
+
+# Media Duplicate Prune
+
+The movie library holds the same film twice in two different ways — the same file
+downloaded again, or the same film grabbed a second time at another quality — and
+[[Entropy Volume]] is permanently full, so both are worth finding. They need
+different rules, and neither is decided by name similarity alone.
+
+`/mnt/entropy/media-park` was a third kind: a parking lot that held whole shows
+and films nothing served. It is gone; its contents went to `NeedsProcessing`.
+
+## What counts
+
+| Class | Test | Action |
+|---|---|---|
+| Byte-identical | equal size, equal content hash | keep one copy, hold the rest |
+| Same work, worse encode | same title, same year, every runtime within 5 % | keep the largest — Frank's rule |
+| Different cut | same title and year, runtimes more than 5 % apart | **never touched**: an extended and a theatrical version are different products |
+| Same words, different film | title matches, year or duration does not | never grouped — this is the false-merge class that would destroy a unique file |
+
+## The screen
+
+The array cannot afford a full-content read, so identity is established in steps
+and each step only narrows what the next one touches:
+
+1. `scripts/media-census.py enumerate` — size is free, and identical files share it.
+2. Head+tail `blake2b`, 4 MiB at each end, on the size collisions only.
+3. **Before anything moves**, confirm with a sparse-window hash: 32 MiB at eight
+   evenly spaced offsets. Re-hashing a remux-heavy set in full read 150 GiB for
+   the better part of an hour; the sparse window cost 50 s, a file that merely
+   shares its head and tail cannot survive it, and the destination is a
+   recoverable hold rather than a deletion.
+4. Durations come from `scripts/media-census.py probe` — the relayout's own
+   census. Do not hand-roll a second probe: it re-seeks the array for an answer
+   the repo tool already produces, and a probe is the one census step that costs
+   hours instead of seconds.
+
+## Which copy survives
+
+In order: not one inside an `extras`, `featurettes`, `samples` or `trailers`
+folder (an extra is not a copy of the film, and extra names collide across films);
+then the shallowest path; then the file whose name equals its `Title (year)`
+folder; then the largest. For the same-work class the largest is the answer, and
+the canonical-name rule only breaks a tie — without it a `Film - 1080p.mkv` can
+outlive `Film.mkv` for no reason but a space sorting before a dot.
+
+## Guards
+
+Each of these is a file that would have been destroyed without it:
+
+- **`.!qB` is qBittorrent state, not a duplicate.** It can be byte-identical to
+  the finished file while qB still counts the download incomplete; moving it
+  breaks the re-check and re-downloads the film. Report it, leave it. Both
+  leftovers in `Movies` are 13 months stale, so they are Frank's purge decision,
+  not a prune decision.
+- **Require a year, and keep digits in the title.** Stripping digits merged
+  *Akira — Teaser 1* with *Teaser 2*; without a year, two different extras that
+  happened to share a name merged.
+- Both members must be feature length, and a name carrying `disc`, `cd` or `part`
+  is excluded: half a multi-disc release is not a copy of the other half.
+- Media containers only. Grouping every extension buries the report in subtitle
+  and cover pairs — and a duration guard that appears to fire on 720 groups is
+  counting subtitles, not protecting films.
+
+## The hold
+
+Nothing is deleted. Every doomed copy moves into a dated hold with the same
+executor the relayout uses, so any file comes back:
+
+```bash
+scripts/media-apply.py apply   --plan /mnt/entropy/dup-prune-similar-plan.tsv \
+    --root /mnt/entropy --journal /mnt/entropy/dup-prune-similar.jsonl          # dry
+scripts/media-apply.py apply   --plan … --root /mnt/entropy --journal … --apply
+scripts/media-apply.py verify  --plan … --journal /mnt/entropy/dup-prune-similar.jsonl
+scripts/media-apply.py sweep   --journal /mnt/entropy/dup-prune-similar.jsonl --root /mnt/entropy --apply
+scripts/media-apply.py rollback --journal /mnt/entropy/dup-prune-similar.jsonl --root /mnt/entropy --apply
+```
+
+The plan rows are the tool's own format (`action src dst kind`, `mkdir` rows for
+every hold parent), and the planner re-checks the two things that go stale between
+analysis and apply: a file that has moved since the census, and a group whose
+keeper is missing — either one drops the whole group rather than half-applying it.
+
+- **A hold on the same volume frees nothing.** The purge is a separate, approved
+  act and the only thing that moves the free-space number; `df` does not budge
+  when 1 TiB moves sideways.
+- `sweep` removes the release folders the pass emptied. Empty folders dated a year
+  ago are torrent detritus (`Subtitle,info`, `Screens`, `Sample`) from a previous
+  era, not this pass — read the mtimes before blaming the prune.
+- The 2026-11-01 pass held 226 files (~1.1 TiB) in
+  `/mnt/entropy/.AGENTTRASH/20261101-duplicate-prune`, journals
+  `dup-prune-exact.jsonl` and `dup-prune-similar.jsonl`. Purge pending Frank's word.
+
+## After the pass
+
+Both servers still hold rows for files that are now in the hold, and neither
+notices: the media bind is the [[Media Metadata View]], and array changes do not
+cross it as inotify events. Scan in the order the rescan law sets — Jellyfin to
+`Idle`, then Plex's Movies section alone, one accepted refresh each; the requests
+are owned by [[Jellyfin]] and [[Plex]].
+
+`scripts/media-verify.py` reads differently for a prune than for a relayout:
+`landed: 0` is the **correct** end state (no server may index the hold), so the
+verdict is `still at src: 0` with `REAL: 0`, and the rows that simply vanished
+from the database are the pass succeeding.
+
+## Failure signatures
+
+| Symptom | Meaning |
+|---|---|
+| A size group whose members split under the sparse hash | The head+tail screen collided. Those files stay exactly where they are and get reported. |
+| The planner holds far fewer files than the analyzer counted | Two implementations of "largest wins". The analyzer must emit `keep`/`hold` marks and the planner only transcribes them — a planner that re-derives the winner from the hold list alone drops the keeper out of the group and prunes almost nothing. |
+| `df` unchanged after a TiB of moves | Expected: the hold is on the same filesystem. Only the purge frees space. |
+| A film that "disappeared" after a prune | It is in the hold, named by the journal: `media-apply.py rollback --journal … --apply`. |
+
+## Links
+
+- [[Media Relayout]] — the census tools, the executor, the journal format.
+- [[Media Metadata View]] — why neither server sees the change without a scan.
+- [[Jellyfin]] · [[Plex]] — the scan requests, one server at a time.
+- [[Entropy Volume]] — the array that is always full.

serverAppConfigs-docs/Dankserver/Media Metadata View.md

@@ -3,7 +3,7 @@ type: reference
 title: Media Metadata View
 description: "The SSD-backed overlay the two media servers read and write: shape, guards, commands, laws, rollback."
 tags: [dankserver, media-server, jellyfin, plex, storage]
-timestamp: 2026-10-30T00:00:00Z
+timestamp: 2026-11-01T01:00:00Z
 ---
 
 # Media Metadata View
@@ -20,18 +20,23 @@ The mount point is not a volume and holds nothing while unmounted. It sits insid
 merged door are one directory — and so the mount table grows no new `/mnt` entry
 next to the array.
 
-**Not deployed on Dankserver — verified 2026-10-31.** `scripts/media-view.sh
+**Deployed on Dankserver — verified 2026-11-01.** `mount` shows the overlay on
-status` answers `view: /var/lib/media-meta/view NOT MOUNTED`,
+`/var/lib/media-meta/view` (`lowerdir=/mnt/entropy`,
-`/var/lib/media-meta` does not exist, and the installed quadlets under
+`upperdir=/var/lib/media-meta/upper`), and the installed quadlets under
-`~/.config/containers/systemd/` still carry the pre-view binds — `/mnt/entropy:/data:ro`
+`~/.config/containers/systemd/` carry the view binds:
-for [[Jellyfin]], `/mnt/entropy:/mnt/entropy:ro` for [[Plex]]. Everything below
+`Volume=/var/lib/media-meta/view:/data` for [[Jellyfin]],
-is the shape git describes, not the shape the servers run. The cutover is an
+`Volume=/var/lib/media-meta/view:/mnt/entropy:ro` for [[Plex]]. Prove it in that
-ordinary deploy: `deploy.sh dankserver --service jellyfin|plex` runs
+order: `scripts/media-view.sh verify` (the deploy gate), then
-`ensure_media_view` (`install` then `verify`) before it restarts the container,
+`grep -n Volume ~/.config/containers/systemd/{jellyfin,plex}.container`.
-so it must not be fired through the middle of a library scan. Until that deploy
+
-lands there is no upper layer, so no relayout can leave ghost metadata and
+The cutover itself is an ordinary deploy: `deploy.sh dankserver --service
-`reap` has nothing to do — `media-apply.py` prints that follow-up only when the
+jellyfin|plex` runs `ensure_media_view` (`install` then `verify`) before it
-view really is a mountpoint.
+restarts the container, so it must not be fired through the middle of a library
+scan. Two consequences are now live rather than theoretical: inotify events for
+array writes do not cross the merge, so any relayout or prune needs a manual
+library refresh; and a move can leave ghost sidecars above, so `reap` runs after
+a relayout — a pass that moves only media files with no sidecars next to them
+leaves the upper layer untouched, and `reap` correctly finds nothing.
 
 ## Shape
 

serverAppConfigs-docs/Dankserver/Media Relayout.md

@@ -3,7 +3,7 @@ type: reference
 title: Media Relayout
 description: "Census, plan and apply the movie, TV, music and audiobook libraries: the gates, the refusals, the rollback."
 tags: [dankserver, media-server, jellyfin, plex, procedure]
-timestamp: 2026-10-30T00:00:00Z
+timestamp: 2026-11-01T01:00:00Z
 ---
 
 # Media Relayout
@@ -109,6 +109,8 @@ HTTPD close without a single byte; and the same HTTPD drops requests that carry
 no `Accept` and client-identifier header (see the scan block above for the form
 that works).
 
+A duplicate prune is the same census and the same proof, with its own keep rules: [[Media Duplicate Prune]].
+
 ## TV in the movie library
 
 A box set that arrives in the movie share stays there until someone moves it:

serverAppConfigs-docs/index.md

@@ -51,6 +51,7 @@ is a locator: one row, one clause, no numbers.
 | [[Entropy Volume]] | The Dankserver entropy volume is a RAID-backed LUKS filesystem mounted at |
 | [[Jellyfin]] | Jellyfin on Dankserver: quadlet unit, overlay metadata view, provider keys, ports, access, transcode status. |
 | [[mariadb]] | The MariaDB backend for RomM: quadlet unit, network alias, env-file secret handling, and the backup path. |
+| [[Media Duplicate Prune]] | Find and hold duplicate films on the entropy array: the hash ladder, the keep rules, the guards, the purge. |
 | [[Media Metadata View]] | The SSD-backed overlay the two media servers read and write: shape, guards, commands, laws, rollback. |
 | [[Media Relayout]] | Census, plan and apply the movie, TV, music and audiobook libraries: the gates, the refusals, the rollback. |
 | [[Plex]] | Plex Media Server is a **containerized media server on Dankserver**: a podman |