Compare commits

..
Author SHA1 Message Date
Frank a405c5ed35 fix(media-apply): refuse a plan row whose verb the tool does not know
read_plan's if/elif chain had no else, so any row that was not mkdir, keep
or move was dropped in silence. A plan of nothing but typos therefore
applied as a successful no-op: on 2026-11-02 a pass meant to rename two
TVShows series folders reported "applied: 0 moved" and exit 0, and the
journal was never written — nothing told the operator the plan had never
been read. An unread row is a plan bug, and every other plan bug in this
tool refuses before the first rename, so now this one does too, naming the
line and the verb.

cwd: /home/blank/.omp/wt/wt-20261030-163847-6d1fc1f · session: 01a18f2e-18b1-7573-bcd5-ac99636f8bcb
2026-11-02 09:51:53 -08:00
dank 4a33898a6c Merge pull request 'fix(jellyfin): anchor the provider-key write on the file’s real root' (#121) from chore/jellyfin-tvdb-key into main 2026-11-02 07:17:25 -08:00
dank 21c98bd485 Merge pull request 'docs(jellyfin): the door that scans is /Library/Refresh, and the art door names its field' (#120) from docs/jellyfin-scan-door into main 2026-11-02 07:16:52 -08:00
dank a651c81dfb Merge pull request 'feat(audiobookshelf): bootstrap the account, library and scan from the deployer' (#119) from feat/audiobookshelf-bootstrap into main 2026-11-02 07:16:45 -08:00
Frank 0951d909c9 docs(agents): the bracket trick for a pkill pattern that must be named
The existing rule says don't `pkill -f` a pattern your own command line carries,
and to prefer `pkill -x` — but a remote one-shot often has to name the pattern,
and hitting it again today cost a silent `ssh` exit 255 with zero output, which
reads like a network fault. Record the escape (`"serve[-]script.sh"`), the zsh
glob-expansion trap that makes an unquoted `[s]` a no-op, and the exit-code
signature.
2026-11-02 04:47:16 -08:00
Frank 69effb542d docs(jellyfin): Shows is on providers, and the per-library door stays dead anyway
The `Shows` flip landed today (one field, proven in the server's own
`LibraryOptions` answer), so the posture table no longer calls it a silent file
index and `Music`/`Collections` are named as the remaining ones.

Two corrections the same day's measurements force:

- The per-library `POST /Items/{id}/Refresh` answered `204` and queued nothing
  both before the flip and after it — zero journal lines for ten minutes, and
  `Scan Media Library` kept `LastTaskResult=None, EndTimeUtc=None`, the never-run
  signature. The provider switch was never the reason that door is dead; only
  `POST /Library/Refresh` runs (204, then `Running` inside 15 s at 33 % CPU), and
  it walks every library, so a library-scoped pass has no door on this build.
  That paragraph also records the null-pair read that separates "never ran" from
  "just finished", and that query+body on the same call answers `400` with an
  empty body whatever the values.
- The key law: a plugin owns its own config file under its own
  `<PluginConfiguration>` root and omits null settings, so TVDB's `<ApiKey>`
  element never existed and `install` refused — which is what `MISSING key` had
  been reporting since October. PR #121 fixes the writer.

The `Shows` art census replaces speculation with the split that matters: 2,908 of
3,558 series+season carry a primary, 624 have no provider id, 26 have an id and no
poster. A `Download` campaign over them took 20 s and downloaded nothing, which is
the measurement that says identification is the only lever left.
2026-11-02 04:47:16 -08:00
Frank e9314ea1ee fix(jellyfin): anchor the provider-key write on the file's real root
TheTVDB writes its own configuration file and its serializer omits a
null-valued setting entirely, so a fresh install has no <ApiKey> element to
fill, under a root element called <PluginConfiguration>. xml_secret's
fallback insertion was hardcoded to </Configuration>, matched nothing, and
install refused with "no place to write <ApiKey>" — measured on Dankserver
2026-11-02, where the TVDB key has been present in the secret file since
October and has never been installable.

The anchor now comes from the document's own root, so both shapes Jellyfin
actually produces work (the core plugins' <Configuration> and a plugin's
<PluginConfiguration>), and the refusal is kept for a file with no root at
all. Tests: the plugin-written shape is red before this commit and green
after; every other setting in the file is asserted preserved.
2026-11-02 04:38:40 -08:00
Frank ad9563412d docs(audiobookshelf): what a residual is made of
Four laws from finishing the un-shadow, correcting one:

- A book whose files ALL fail to probe never becomes an item, so a disk-vs-DB
  folder gap beside `Invalid data` lines is corrupt media, not a lost book. A
  format-only ffprobe clears files the scanner rejects — do not trust it.
- `Disc 01`/`CD 1` subfolders are parts of one item (110 of them here, under a
  192-track book); the extension census counts them, ABS does not.
- `.cbr`/`.cbz` group like media but `parseComicMetadata` throws `ERAR_BAD_ARCHIVE`,
  so those items land media-less: the file, not the scanner.
- Scan completion is a NEW scan uuid; `podman logs --since` is a sliding window
  and ages its own evidence out.
- `?force=1` is now recorded as neither discovery nor repair: a forced full
  re-scan left item count, path set and media-less count identical.

Verification names the five book folders that need re-acquiring.
2026-11-02 03:59:14 -08:00
Frank f71c7ccce4 docs(audiobookshelf): the grouping rule, not "shallowest folder with audio"
Corrects the law pushed earlier today. The mechanism is
`groupFileItemsIntoLibraryItemDirs` (utils/scandir.js), called on a live
`recurseFiles` walk of the tree EVERY scan: each file attaches to the FIRST
directory along its path that already has a group, so one media file at a
collection root owns everything beneath it. Measured: one 949 MB `.cbr` hid
652 book folders behind a single 5,693-track item.

- "Media file" is audio OR ebook (`audiobooksOnly=false` -> SupportedAudioTypes
  ∪ SupportedEbookTypes, 17 audio + 6 ebook extensions). An audio-only census
  cannot see the file that hides an Audiobooks library.
- Deleting the covering item does not fix a shadow: grouping is recomputed from
  disk and the next scan rebuilds the item (measured: `1 Added`, same path).
- Un-shadow by MOVE: one file per folder is ABS's own single-book layout.
- `?force=1` is not a discovery arm (both arms walk the disk; force re-scans
  every existing item, 1:33 against 0:04).
- Paging corrected: `offset` is ignored outright, `page` is the parameter.
- `DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()` of the whole
  swallowed subtree. Never pass `hard`.

Shape table gains the layout row and the move journal; the stale "1,103
shelves" scan count is destroyed.
2026-11-02 03:43:34 -08:00
Frank f541566714 docs(audiobookshelf): a book is the shallowest audio folder, and it owns the subtree
Measured on the live library: 6 stray files at the root of _unsorted made the
whole collection dump ONE item of 5,693 tracks, with 542 real books folded into
it; no scan subdivides an existing item. Also pinned: the items endpoint hands
back ceil(total/limit)*limit rows because an out-of-range offset repeats the
tail (that is where "exactly 1,500" and "exactly 4,000" came from), and
DELETE /api/items/:id?hard=1 runs fs.remove() on the item's folder - pointed at
a covering item that is the whole subtree, so un-swallowing is a DB-only delete
after the stray root files are filed.
2026-11-02 03:34:28 -08:00
Frank 006bbcef28 docs(audiobookshelf): paging the items endpoint never terminates itself
Two censuses this session reported exactly 1,500 and exactly 4,000 books for one
library mid-scan. Cause: an out-of-range offset returns the last item instead of an
empty page, so a loop written "until the page is empty" manufactures its own
total, and total itself moves while a scan is running. Stop on total; count after
the pass settles.
2026-11-02 03:18:04 -08:00
Frank 8889a4ceff docs(jellyfin): episode stills are out of scope, and the cost that says so
A per-episode RemoteImages query is never cached, so the same campaign that
dressed 620 series in an afternoon costs ~11 s per episode for one hit in five.
Across 42k episodes that is days of provider traffic to dress a fifth of them, and
clients show series and season art regardless. Recorded so nobody runs the pass on
a hunch; the door and the resumable log are the same if Frank orders it.
2026-11-02 03:00:42 -08:00
Frank 56dfc6d0b9 docs(audiobookshelf): the scanner refusal names the file, not the bind
The first full scan logged "Invalid data found when processing input" per file and
carried on. That line is about the file: the scanner skips it, so the book never
appears in the library at all. Census of the 43 refused files: 38 were 0 bytes, 3
held WAV bytes behind a .mp3 name, 2 were unreadable. The check to run first is a
byte count, not a server investigation.

Full suite: 818 tests, exit 0.
2026-11-02 02:57:32 -08:00
Frank c21c48e61e docs(jellyfin): the door that scans is /Library/Refresh, and the art door names its field
The page sent the next agent to POST /Items/{id}/Refresh?Recursive=true per
library to make a new folder visible. Measured against a Shows library that had
never been identified: that route answers 204 and queues nothing — item census
byte-identical, ScheduledTasks idle, journal silent, and still a silent 204 after
a service restart. The door that works is POST /Library/Refresh, and the progress
read is ScheduledTasks -> Scan Media Library -> CurrentProgressPercentage.
RefreshStatus on the VirtualFolders record stays Idle through a real scan, so it
is not a progress read at all; a 204 cannot distinguish accepted-and-discarded
from accepted-and-queued, so the acceptance read is the journal naming folders.

Also recorded the art door's wire shape, because guessing it reads as a server
bug: the list response carries the address in Url (there is no ImageUrl), the
Download POST takes camelCase type/imageUrl and answers 204, and PascalCase or an
empty imageUrl answers 500. RemoteImages is empty for items with no ProviderIds,
so art coverage is capped by identification, not by the image door.
2026-11-02 02:56:12 -08:00
Frank 48ef6d34cd docs(audiobookshelf): the first run is a script, and its re-run is the scan trigger
The page claimed the account can only be created from a LAN address; measured,
the door is POST /init and it answers plain text. Replaced with the bootstrap the
deployer now runs, and with the doors a consumer actually needs: /status reports
isInit with no /api prefix, the item count rides /api/libraries/{id}/items ->
total (the library DTO has no totals), and the list DTO carries media.numTracks
while /api/items/{id} returns media.audioFiles.

New law: the folder watcher cannot cross the rootless podman bind — the same gap
as Jellyfin's auto-scan note — so files added on the host stay invisible until a
scan runs, and the deployer's re-run of the bootstrapper is the cheap trigger.

Testing Policy lists the new seam; index.md regenerated from frontmatter.
2026-11-02 02:55:55 -08:00
Frank 317f36b456 feat(audiobookshelf): bootstrap the account, library and scan from the deployer
A freshly deployed Audiobookshelf answers /healthcheck and is otherwise dead: no
account, no library, no scan. The UI registration step is gone by design — the
doors are POST /init (200 with plain text "OK", not JSON) and /status (isInit),
and there is no /setup route. That is an API-only bootstrap, so it must be code.

scripts/audiobookshelf-init.sh is idempotent: /status -> /init when unregistered,
POST /login for the bearer, POST /api/libraries for the Audiobooks library on
/audiobooks, then POST /api/libraries/{id}/scan. The deployer runs it only after
the health gate has answered.

Credentials live in ~/.config/secrets/audiobookshelf (mode 600): the bearer rides
an Authorization header and the password only a POST body — never a URL or argv,
where any ps on the host would print them. The seam drives an inline stub server
that logs every request line, and asserts: fresh bootstrap creates account,
library and scan; a second run creates nothing new and still fires the
incremental scan; no secret appears in any logged request line; a server that
refuses is reported as "refused: <url>" with exit 1 rather than success; ABS_SCAN
outside 0|1 fails before a single request; the deployer orders init after health.

Measured on Dankserver: run against the already-bootstrapped host it is a no-op
that re-fires the scan; the library then scans (1,500+ books, every item with
audio tracks, none invalid).
2026-11-02 02:55:50 -08:00
dank 7a8c6bd2e3 Merge pull request 'fix(audiobookshelf): set the port with the env names the image actually reads' (#118) from fix/audiobookshelf-image-env into main 2026-11-02 02:28:44 -08:00
Frank 3c035ccef3 fix(audiobookshelf): set the port with the env names the image actually reads
The first deploy of the unit failed its own health gate in the worst possible shape:
systemd said active (running), the engine logged a clean database init, and the
server then listened on :80 under base path /audiobookshelf - so every probe of
:8000 was refused and wait_for_health timed out at 180s beside a container that
looked healthy.

Cause: ABS_PORT / ABS_CONFIG_DIR / ABS_METADATA_DIR / ABS_PUID are the npm
package's names, printed by the server README; this image never reads them. Its own
contract, read off the pinned digest with podman inspect, is PORT=80
CONFIG_PATH=/config METADATA_PATH=/metadata, and the server greps for exactly PORT,
CONFIG_PATH, METADATA_PATH, ROUTER_BASE_PATH. The two path variables appeared to
work only because the image already exports the same values - the coincidence that
made the inert lines look correct.

The unit now speaks the image's names, adds ROUTER_BASE_PATH=/ so the service is
the plain host:port the docs and the homepage card promise, and the dead ABS_*
lines are deleted instead of kept as decoration. Proven against the same digest in
a throwaway container (temp state dirs, loopback-only mapping, removed after):
base path "/", listening on :8000, GET /healthcheck -> 200, and the old prefixed
path -> 404.

Guards: the quadlet seam now refuses an Environment=ABS_* line and requires
PORT/ROUTER_BASE_PATH; the vault page gains the law with its failure signature and
the two-second image-inspect check. Suite: ALL TESTS PASSED, 812 ok.
2026-11-02 02:28:37 -08:00
dank a6ec644ecb Merge pull request 'feat(audiobookshelf): the audiobook server as a boot-persistent quadlet' (#117) from feat/audiobookshelf into main 2026-11-02 02:20:03 -08:00
Frank b7f4b7e148 docs(audiobookshelf): one page for the endpoint, and the rescan law at the relayout
The page owns the shape, the first-run account step (ABS accepts the initial
account only from a local-network address) and the laws: why the array bind is
read-only and what the one-line rollback is if on-disk sidecars are ever wanted;
why the metadata view is not that rollback; why the mount gate, the wget health
check and the amd64 digest pin are each mandatory; and that LAN-only is a posture,
not an accident.

Media Audiobook Relayout gains the third reader: ABS stores absolute paths, so the
relayout procedure has to reach its library too or it keeps rows for folders that
are gone. The vault index is regenerated, not hand-carved.
2026-11-02 02:19:29 -08:00
Frank ed1a20a960 feat(homepage): an Audiobookshelf card on the admin page
Admin category, Media Library Apps section, LAN-direct url and probe like Tautulli:
the service is internal-only, with no vhost and no DNS name, so it earns a card and
no public link. The art is a mono-ab.svg monogram in the existing hexagon family.

Card counts in the e2e specs come from the live registry, so this entry is the test
update as well as the page change - and the admin walk will follow the link, which
is why the service deploys before the homepage ships.
2026-11-02 02:19:29 -08:00
Frank 511d8dca93 feat(audiobookshelf): the audiobook server as a boot-persistent quadlet
Audiobookshelf serves /mnt/entropy/Audiobooks to browsers and phones with its own
library and per-listener positions. The design point of the unit: every byte this
server writes is its own state - SQLite library, accounts, progress, cached covers
- and all of it lands on the NVMe under /home/dank/Containers/audiobookshelf. The
audio bind is :ro, so a stray sidecar write onto the spindle is structurally
impossible rather than merely discouraged, and the array keeps exactly one
sanctioned write path (scripts/media-view.sh).

Three failure modes the unit is shaped around:

- Bind-source ordering. In a rootless unit RequiresMountsFor= is inert and
  ConditionPathIsMountPoint= is fail-dead, so a missing array does not stop the
  start: podman invents an empty directory and the first scan records an empty
  library as truth. ExecStartPre=wait-for-mount.sh is the gate that works here,
  the same one copyparty uses.
- The health binary. The image is Alpine: wget, no curl. A curl HealthCmd reads a
  healthy server as unhealthy and Restart=always turns that into a loop.
- The image pin. podman records the linux/amd64 manifest for what it pulled, not
  the manifest-list digest `latest` points at, so the amd64 digest is the pin that
  names the build this host runs. Upstream version tags stop at 2.19.0 while
  `latest` is newer, so latest@digest is the only form that is both current and
  exact - the jellyfin/romm precedent.

The first-boot scan reads the whole tree once, so IOWeight=10 keeps it behind
playback; that weight is only honoured because the user.slice accounting drop-in
and the user@1000 delegation are installed by the deployer.

Deployer registration is the standard four touch-points (client gate, remote
allowlist + handler + dispatcher, planner order/set/path map, test routing). The
handler creates both state roots before the first start - letting podman invent
them leaves the SQLite file unwritable by the container user under a rootless
first boot - and gates on /healthcheck.
2026-11-02 02:19:24 -08:00
Frank 29aefc3b6f Merge remote-tracking branch 'origin/main' into feat/jellyfin-provider-keys 2026-11-02 02:12:19 -08:00
Frank 2e69215920 docs(jellyfin): name the library doors, the watcher, and the phantom records
Measured 2026-11-02 against the live server. The per-library posture goes on the
page because two libraries are silent file indexes right now: Shows and Music
carry EnableInternetProviders=false, so a refresh on either answers 204 and
stores nothing — any metadata or artwork work there starts by flipping the
switch, not by scanning. Movies additionally has EnableRealtimeMonitor=false, so
a folder moved into /data/Movies stays invisible until a recursive refresh names
it; "Jellyfin does not see the file" is the designed answer, not a fault.

Recorded with it: the option door on 12.x is POST
/Library/VirtualFolders/LibraryOptions (GET answers 405 — that is how the route
is found) while the 10.x /Update and /Refresh routes answer 404 outright, and
the VirtualFolders DTO is PascalCase, so a census written against the old
camelCase keys reads None and misreports a healthy library as unconfigured.
A library record can also be a phantom: a 2026-11-01 capture carried Shows2,
Shows3 and Music2 with empty Locations while only four libraries existed, so the
live set is the entries with non-empty Locations and never the record count.
2026-11-02 02:12:19 -08:00
dank b93e8481f9 Merge pull request 'feat(media): per-episode TV duplicate census, and the Jellyfin library that indexed but fetched nothing' (#116) from feat/jellyfin-provider-keys into main 2026-11-02 02:03:02 -08:00
Frank 51ed5c0b2a docs(media): stat through the overlay can outlive the file it points at
Measured 2026-11-02 while a TV dedupe pass moved folders on the raw path: the
merged tree still answered stat at the old coordinates, link count 0, while ls of
the same parent no longer listed the name. Overlayfs keeps a dentry for a lower
file moved underneath it, so a tool that tests existence with [ -e ] against the
view can believe a file its own readdir denies — the media servers walk by
readdir, so this bites a script and never a user.

Recorded with it: every pass that moves or measures media walks /mnt/entropy, and
the one reader allowed to point at the view is the tool that needs show identity.
2026-11-02 00:44:51 -08:00
Frank 9f9043c037 feat(media): measure a series per episode, and let the tag outrank a borrowed id
media-tvdupes.py censuses a TV library by episode key and never by folder. 10 of
the 25 shows that looked duplicated in the 2026-11-02 census held complementary
halves (Game of Thrones Complete carries no S01 or S08, Modern Family S05 is the
only copy of season 5), so a folder-level verdict would have staged some 340 GB
of unique episodes to recover 54 GiB of true copies.

Show identity comes from the folder's [tvdbid-N] tag, from the tvshow.nfo the
metadata view exposes, or from an operator --ids map. A derived id that
contradicts a tag is dropped and reported, never unioned: Jellyfin's provider
table carries a 2026 special's id on the folder tagged [tvdbid-73838], and
unioning the two makes one show of a 2000 series and a 2026 special whose bigger
files then dominate the original's pilot. Files named by an id key only when the
server's own episode table (the --witness) names them, so a bare number is never
mistaken for an episode.

Seam: tests/media_tvdupes.bats — 12 assertions on which files land in the plan
and which the tool refuses to touch, on a synthetic library.
2026-11-02 00:44:46 -08:00
Frank a25f139281 docs(jellyfin): the art delivery result, and what stays blank on purpose 2026-11-01 11:39:11 -08:00
Frank 52600b28ab fix(jellyfin): refuse a library that indexes files but fetches nothing
The Movies library had indexed 2,571 films and had never asked a provider
for a single one: no TMDb id, no plot, no rating, no poster. Nothing was
broken in the provider chain — the plugin is core in 12.4 and active, its
key is set, image.tmdb.org answers the container — and nothing was broken
in the write path: the overlay view has been read-write since the 2026-10-31
cutover, and movie.nfo lands on the NVMe upper layer today.

The state was two settings in the per-library option file,
config/root/default/<Library>/options.xml (Jellyfin 12 renamed it from
library.xml; the database stores none of these options, so a census that
greps for the old name and then queries the DB concludes nothing is
configured and goes hunting elsewhere):

  <EnableInternetProviders>false</EnableInternetProviders>
  <TypeOptions><ImageFetchers />   (empty, beside a populated
                                    <MetadataFetchers> and a fully
                                    populated <ImageFetcherOrder>)

With the internet switch off every refresh answers 204, stores nothing and
logs nothing, so the item count reads like a healthy scan — this passed the
deploy health gate, the Playwright suite and three agent audits. Order is
not enablement: the fetcher ORDER stays populated while the ENABLED set sits
empty, and the populated metadata list beside it makes the library look well.

verify now audits every library's options.xml and fails with `SILENT INDEX
<Library>: <what>` on providers-off or on a type whose image fetchers are
empty while its metadata fetchers are named; an options.xml that never
carried the switch (BoxSets, written before the property existed) is the
vendor default and stays accepted.

Vault: the page claimed artwork is a sidecar like movie.nfo. It is not — no
image saver is bound for any library, so art goes to the internal store and
Plex never reads it. The claim is corrected, and the RemoteImages diagnostic
that separates a provider fault from a library fault is written down with
the 12.x API shape traps (Items needs UserId; MetadataRefreshMode has no
Full/ValidationFailure, ReplaceAllMetadata=true is what re-runs providers).
2026-11-01 10:31:42 -08:00
Frank 7cf388367e docs(media): the array's write doors, and why survivors are re-verified by size
A folder-rename pass ran on the media array with no journal anywhere: no
jsonl on the array, no bash history, no cron/timer/job artifact, no other
host ever authenticating. The door was GNOME Files mounting the array over
SFTP (gvfsd-sftp -> sftp-server as dank): rename() with no shell and no
journal, bumping only the renamed directory's ctime. Copyparty cannot do
this (its bind is ro at the mount) and the *arr stack has no route to the
array at all (no SMB, no NFS, mini asleep).

Two laws fall out for the prune pipeline: re-verify survivors by size
rather than by the path a stale artifact recorded, and Plex extras rows
naming a held path survive a second section pass.
2026-11-01 09:46:32 -08:00
Frank 13830865f6 docs(media): the Plex leg ran, and a script fed over stdin owns that stdin
The 2026-11-01 morning leg cleared every Plex row naming a held path, so the prune page records the outcome plus the residue that is normal: a section pass rewrites feature rows first and can leave rows for clips in a film's other/ or extras/ folder standing after the refreshing flag is back to 0. Beside REAL: 0 that is a dead disc-menu clip, not a false prune, and no reason to touch the cache or the database.

Plex gains the two operational laws the run taught: a start job issued after an unmask belongs to the manager, not the shell, and lands minutes later surviving the client that issued it; a refresh fired while PMS is warming answers 000, so retry it rather than restart the service.

The trap that made a finished job look hung belongs to the remote job runner: a script fed to bash -s IS the stdin, so any child that reads stdin (podman exec -i) swallows the rest of the script without a word.
2026-11-01 09:38:38 -08:00
dank 010b9affc7 Merge pull request 'media-verify reads the WAL; the metadata view is live; duplicates get a page' (#115) from fix/media-verify-wal-and-prune-record into main 2026-11-01 01:42:20 -07:00
Frank ed5eed6d54 docs(media): the deferred Plex leg, and the guards a detritus sweep must respect 2026-11-01 01:42:02 -07:00
Frank 14f340370c Merge remote-tracking branch 'origin/main' into feat/jellyfin-provider-keys
# Conflicts:
#	AGENTS.md
#	serverAppConfigs-docs/Dankserver/Media Metadata View.md
#	serverAppConfigs-docs/index.md
2026-11-01 01:36:35 -07:00
Frank 50e0c5a786 docs(media): the metadata view is live, and duplicate pruning gets a page
`Media Metadata View.md` said "Not deployed on Dankserver — verified
2026-10-31". That was true yesterday and is false now: the overlay is mounted
and both installed quadlets bind it (`view:/data` rw for Jellyfin,
`view:/mnt/entropy:ro` for Plex). The page now states the live shape and the
order that proves it — `media-view.sh verify`, then the `Volume=` lines —
because the whole point of the page is to answer "is the view really serving?".
`AGENTS.md` carried the same stale claim in its summary table.

`Media Duplicate Prune.md` is new: the screen ladder (size, then head+tail
blake2b, then a sparse-window confirmation before anything moves), which copy
survives and why, the guards that each encode a file that would otherwise have
been destroyed (qBittorrent `.!qB` state, year-required with digits kept,
feature length, disc/part exclusion, media containers only), and the law that a
hold on the same volume frees nothing until it is purged. It also carries the
different way `media-verify.py` has to be READ for a prune — `landed: 0` is the
correct end state because no server may index the hold — and the planner bug
that class explains: when the planner re-derived "largest wins" from the hold
list alone it dropped every keeper out of its own group and pruned 17 files
instead of 205.

`Media Relayout.md` keeps a one-line pointer; carrying the whole procedure there
pushed it past the vault's 500-line ceiling, which is the linter telling the
truth about progressive disclosure.
2026-11-01 01:11:26 -07:00
Frank 649bbafa17 fix(media-verify): read the WAL the server commits through
`mode=ro&immutable=1` claims the file never changes, so sqlite skips the
write-ahead log entirely: every row the server has committed but not yet
checkpointed reads as MISSING, and a read beside a running scanner can fail
outright as "database disk image is malformed". Both servers keep their
libraries in WAL, so plain `mode=ro` is now the first door and `immutable=1`
only the fallback for a database whose `-shm` cannot be read.

The verdict also stopped guessing from the database alone. A file still at its
`src` is REAL (this pass's own damage); a file at neither path is `gone` (a
later pass or the server renamed it); a file sitting at `dst` with no row
naming it is something the server does not index. A row that still names `src`
is none of those — it is the server's stale pointer, and it costs a re-scan,
never a restore.
2026-11-01 01:04:22 -07:00
Frank b7c19592e5 docs(jellyfin): record the live view bind and the library-coverage law
The page still described the pre-cutover generation ("the live unit is
still pre-view", "/mnt/entropy -> /data (live today, read-only)") after
the 2026-10-31 cutover deployed the overlay. Corrected, and added the law
the census tonight taught: Jellyfin's four libraries cover /data/Movies,
TVShows, Music and the collections dir, so anything under /data/media-park
is invisible by configuration — a recursive refresh there returns 204 and
indexes nothing. Check coverage before re-scanning.
2026-10-31 19:43:35 -07:00
dank 6c115c1394 Merge pull request 'fix(deploy): make the view's upper base before installing the view' (#113) from fix/media-view-ensure-base into main 2026-10-31 19:36:52 -07:00
dank 6d79709cc0 fix(deploy): make the view's upper base before installing the view
`ensure_media_view` ran `install` on a host where the documented base directory
did not exist, and `install` refuses by design ("put it on the SSD first") — so
tonight's cutover would have died at the restart step with nobody beside the
box. `media-view.sh ensure-base` now makes the documented directory, and the
deployer calls it first; two seams cover it (the subcommand's behaviour, and the
ordering invariant inside `ensure_media_view`).

Docs flip with the deploy: the view is live on Dankserver (both media servers
verified resolving real media through it), so every "never been deployed" claim
in the view, Plex, Jellyfin and AGENTS pages is corrected, and three laws join
them — a container restart discards the in-flight library scan and nothing
resumes it; Jellyfin 12.4 authenticates only the `MediaBrowser Token` form, so a
401 says nothing about the key until the `ApiKeys` table is read; and Plex
answers 32400 over TLS only, which makes a plaintext health probe read as dead.
2026-10-31 19:33:16 -07:00
dank 0336f42e2a Merge pull request 'test(scoping): stop pinning a purgeable trash archive' (#112) from fix/scoping-seam-disposable-archive into main 2026-10-31 19:30:35 -07:00
dank 0f4a527337 test(scoping): stop pinning a purgeable trash archive
The seam asserted `[ -f scripts/.AGENTTRASH/chat-template.jinja.ik-llama-qwen-sharp ]`
so that the planning assertion "could not pass by the file having vanished". That
reasoning was wrong twice over: the planner's rule is a PATH rule, so it plans
nothing for that path no matter where the bytes sit, and the trash is a purgeable
hold — #111 untracked it, so a fresh worktree or CI checkout has no such file and
the suite went red on a byte that carries no meaning.

Measured: fresh worktree off origin/main, deployment_scoping.bats:39 fails. After
the change: that file 28 ok / 0 fail, ./tests/run.sh 789 ok exit 0.
2026-10-31 19:27:56 -07:00
dank 1d92627935 Merge pull request 'chore: stop tracking .AGENTTRASH so a purge cannot dirty a checkout' (#111) from chore/untrack-agenttrash into main 2026-10-31 18:49:41 -07:00
31 changed files with 2297 additions and 67 deletions

No files matched your search

+8 -2
View File
@@ -735,7 +735,12 @@ linked runbooks.
**Ops**
- Never `pkill -f` a pattern that appears in your own command line — it kills your ssh session.
Use `pkill -x <exact-name>`.
Use `pkill -x <exact-name>`, or the bracket trick — `pkill -f "serve[-]script.sh"` —
when the pattern has to be named: the bracket makes the pattern stop matching the
command line that carries it. Quote it: a zsh remote glob-expands an unquoted `[s]`
and the kill silently never runs. Re-hit 2026-11-02 —
`ssh host 'pkill -f jfshowswatch.py; …'` killed the remote shell mid-command, and
ssh exits **255 with zero output**, which reads exactly like a network fault.
- Both hosts allow passwordless sudo for the deploy user. Use plain `sudo` in committed scripts
(no `echo pw | sudo -S`); `FRANK_SUDO_PW` covers hosts without it.
- **ComfyUI on Dankclaw runs with GPU access** (`/dev/kfd` + `/dev/dri` via `AddDevice`,
@@ -818,6 +823,7 @@ per-attempt page and no per-swap ledger anywhere else.
| Deployment flow | `serverAppConfigs-docs/Architecture/Deployment Flow.md` |
| Movie-library relayout (Jellyfin format, Dankserver) | `serverAppConfigs-docs/Dankserver/Media Relayout.md` |
| SSD-backed metadata view (Jellyfin writes, Plex reads, Dankserver) | `serverAppConfigs-docs/Dankserver/Media Metadata View.md` |
| Duplicate films and the purge hold (Dankserver) | `serverAppConfigs-docs/Dankserver/Media Duplicate Prune.md` |
There is no page for a retired endpoint and no change diary. A payload swap
appends one row to the live endpoint's **Eras** table; when the endpoint itself
@@ -860,6 +866,6 @@ history.
| Benchmarking (only when ordered) | **Stop and mask the service first**, bench, unmask, restart, verify health. |
| Deleting a file | **Move it to trash** — `.AGENTTRASH/` (in-repo) or `~/.AGENTTRASH/YYYYMMDD-topic`. Never `rm`. |
| Relayout the movie library | **Census → plan → read the exceptions → `scripts/media-apply.py`** — renames only, journal before every move; never hand-rename, never copy. |
| Making media metadata visible to both media servers | **Bind `/var/lib/media-meta/view`** — the overlay `scripts/media-view.sh` installs (array below, NVMe above). Never write the array; never delete through the view. **Not yet deployed on Dankserver: the live quadlets still bind the array read-only — see [[Media Metadata View]].** |
| Making media metadata visible to both media servers | **Bind `/var/lib/media-meta/view`** — the overlay `scripts/media-view.sh` installs (array below, NVMe above). Never write the array; never delete through the view. **Deployed on Dankserver 2026-10-31, re-verified 2026-11-01: both quadlets bind the view, not the raw array. A restart for any reason discards that server's in-flight library scan, so re-trigger the refresh — see [[Media Metadata View]].** |
| Tests exist | **Run them** — verify tests pass after every change. Do not commit failing tests. |
| Real secrets | **Never commit** — host `.env` files; `.env.example` holds placeholders only. |
@@ -0,0 +1,66 @@
# Quadlet unit: Audiobookshelf audiobook server (rootless, boot-persistent).
#
# Install path on dankserver: ~/.config/containers/systemd/audiobookshelf.container
# Unit name: audiobookshelf.service (podman 5.x drops the container- prefix)
#
# Everything this server writes is its own state — the SQLite library, accounts,
# playback progress, cached covers — and it lives under
# /home/dank/Containers/audiobookshelf on the NVMe, so no metadata byte ever
# lands on the spindle. The audio itself is bound READ-ONLY off the array: the
# only sanctioned write path into /mnt/entropy is the overlay view
# (scripts/media-view.sh), Audiobookshelf has no need of it, and a read-only bind
# makes a stray sidecar write structurally impossible rather than discouraged.
#
# The digest is the linux/amd64 manifest of the tag, which is what podman
# verifies and what RepoDigests records on this single-arch host. Pinning the
# manifest-list digest instead would let a re-pull resolve to a different image.
# The upstream version tags stop at 2.19.0 (built 2025-02-02) while `latest` is
# newer by a year and a half, so `latest@digest` is the only pin that names the
# build actually running — same reasoning as the jellyfin and romm units.
[Unit]
Description=Audiobookshelf audiobook server container
After=network.target
[Container]
ContainerName=audiobookshelf
Image=ghcr.io/advplyr/audiobookshelf:latest@sha256:37c2057e150694970442579815c646fe6bdc01c95b9d1547c4c9068be8844ef3
Network=pasta
# The names are the image's own: the server reads PORT, CONFIG_PATH, METADATA_PATH
# and ROUTER_BASE_PATH (measured against the digest below — `podman inspect` shows
# PORT=80 CONFIG_PATH=/config METADATA_PATH=/metadata in the image ENV, and the
# server's source greps for exactly these). The ABS_* family printed by the server
# README is the npm package's naming and is INERT here: with ABS_PORT=8000 the
# server listened on :80 under base path /audiobookshelf, so the unit sat
# `active (running)` while every probe to :8000 was refused (measured 2026-11-02,
# first deploy). ROUTER_BASE_PATH=/ is what makes the URL the plain host:port.
Environment=PORT=8000
Environment=ROUTER_BASE_PATH=/
Environment=CONFIG_PATH=/config
Environment=METADATA_PATH=/metadata
Environment=TZ=America/Vancouver
PublishPort=8000:8000
Volume=/home/dank/Containers/audiobookshelf/config:/config
Volume=/home/dank/Containers/audiobookshelf/metadata:/metadata
Volume=/mnt/entropy/Audiobooks:/audiobooks:ro
# The image is Alpine: it ships wget and no curl, so the health check must name
# the binary that exists or the container is unhealthy while serving perfectly.
HealthCmd=wget -q -O /dev/null http://127.0.0.1:8000/healthcheck
HealthInterval=30s
HealthTimeout=10s
HealthRetries=3
[Service]
# Refuse to start until the array is a real mount — with the bind source missing,
# podman invents an empty directory and Audiobookshelf records an empty library as
# truth. See the header of scripts/wait-for-mount.sh for the blind-boot incident.
ExecStartPre=/home/dank/serverAppConfigs/scripts/wait-for-mount.sh /mnt/entropy
Restart=always
RestartSec=5
# The first boot scans the whole audiobook tree. Weighted LOW (default 100) so a
# scan loses to playback when the array is contended. Proportional share, not a
# cap: inert while the disk is idle.
IOWeight=10
[Install]
WantedBy=default.target
+188
View File
@@ -0,0 +1,188 @@
#!/usr/bin/env bash
# audiobookshelf-init.sh — make an Audiobookshelf server usable, from nothing.
#
# Why this is a script and not a browser session (live 2026-11-02, Dankserver):
# a freshly deployed Audiobookshelf answers /healthcheck with 200 while it has no
# user, no library and no scan — "healthy" says nothing about usefulness, and a
# service that only works after someone remembers to click through a first-run
# wizard is not deployed, it is installed. Everything below is the API the React
# client itself calls, so the same procedure runs unattended and lands in git.
#
# The API facts this script depends on, read off the pinned image rather than
# guessed (dist-server/server/Server.js inside the digest this repo pins):
# * the creation route is POST /init with {"newRoot":{"username","password"}}.
# There is NO /setup route in 2.37.x — POST /setup answers "Unauthorized",
# which reads exactly like an auth fault and is instead a missing route.
# * /status (no /api prefix) reports {"isInit": <has a root user>}; every route
# under /api needs the bearer token that POST /login returns.
# * the library count is NOT a field on the library DTO; the door is
# GET /api/libraries/{id}/items -> {"total": N}.
#
# The credential is generated once into a mode-600 host file and never enters
# the repo, the URL, or a process argument list. Same pattern as calibre-server.
#
# usage: audiobookshelf-init.sh # create what is missing, then scan
# ABS_SCAN=0 audiobookshelf-init.sh # bootstrap without a scan
#
# environment (defaults are the deployed shape):
# ABS_URL server base URL (http://127.0.0.1:8000)
# ABS_SECRET credential file (/home/dank/.config/secrets/audiobookshelf)
# ABS_LIBRARY library name to ensure (Audiobooks)
# ABS_FOLDER folder to bind inside the server (/audiobooks)
# ABS_SCAN 1 triggers a scan (default), 0 skips it
#
# exit 0 every step is satisfied (created or already present)
# exit 1 a step was refused or the server misbehaved (named on stderr)
# exit 2 usage / unreadable credential file
set -euo pipefail
ABS_URL=${ABS_URL:-http://127.0.0.1:8000}
ABS_SECRET=${ABS_SECRET:-/home/dank/.config/secrets/audiobookshelf}
ABS_LIBRARY=${ABS_LIBRARY:-Audiobooks}
ABS_FOLDER=${ABS_FOLDER:-/audiobooks}
ABS_SCAN=${ABS_SCAN:-1}
die() { printf 'refused: %s\n' "$*" >&2; exit 1; }
case "$ABS_SCAN" in
0 | 1) ;;
*) printf 'refused: ABS_SCAN wants 0 or 1, got %s\n' "$ABS_SCAN" >&2; exit 2 ;;
esac
# ---- credential -------------------------------------------------------------
umask 077
if [ ! -f "$ABS_SECRET" ]; then
mkdir -p "$(dirname "$ABS_SECRET")"
user=${ABS_ADMIN_USER:-frank}
# `tr … | head -c 24` is the obvious line and it is a trap under
# `set -o pipefail`: head exits at 24 bytes, tr takes SIGPIPE, the pipeline
# status is 141 and set -e kills the script before the credential is ever
# written (measured 2026-11-02). od reads a FIXED byte count and exits on its
# own, so no consumer closes a pipe early; bash slices the length.
pass=$(od -An -tx1 -N 32 /dev/urandom | tr -d ' \n')
pass=${pass:0:24}
[ "${#pass}" -eq 24 ] || die "could not generate a password (no /dev/urandom)"
printf 'ABS_ADMIN_USER=%s\nABS_ADMIN_PASSWORD=%s\n' "$user" "$pass" > "$ABS_SECRET"
printf 'credential: created %s\n' "$ABS_SECRET"
fi
[ -r "$ABS_SECRET" ] || die "$ABS_SECRET exists but is not readable by $(id -un)"
chmod 600 "$ABS_SECRET"
# ---- everything else, over HTTP ---------------------------------------------
# python3 rather than curl so the password and the bearer token never appear in
# any argv (they would be world-readable in /proc/<pid>/cmdline otherwise).
ABS_URL="$ABS_URL" ABS_SECRET="$ABS_SECRET" ABS_LIBRARY="$ABS_LIBRARY" \
ABS_FOLDER="$ABS_FOLDER" ABS_SCAN="$ABS_SCAN" python3 - <<'PY'
import json, os, sys, urllib.error, urllib.request
BASE = os.environ["ABS_URL"].rstrip("/")
LIB = os.environ["ABS_LIBRARY"]
FOLDER = os.environ["ABS_FOLDER"]
SCAN = os.environ["ABS_SCAN"] == "1"
cred = {}
for line in open(os.environ["ABS_SECRET"]):
if "=" in line:
k, _, v = line.strip().partition("=")
cred[k] = v
user = cred.get("ABS_ADMIN_USER") or "frank"
password = cred.get("ABS_ADMIN_PASSWORD") or ""
if not password:
sys.exit("refused: %s has no ABS_ADMIN_PASSWORD" % os.environ["ABS_SECRET"])
def call(method, path, body=None, token=None, want_json=True):
data = None
headers = {"X-Requested-With": "audiobookshelf-init"}
if body is not None:
data = json.dumps(body).encode()
headers["Content-Type"] = "application/json"
if token:
headers["Authorization"] = "Bearer " + token
req = urllib.request.Request(BASE + path, data=data, headers=headers, method=method)
try:
with urllib.request.urlopen(req, timeout=60) as r:
raw = r.read()
if not raw or not want_json:
return r.status, raw
try:
return r.status, json.loads(raw)
except ValueError:
# Express answers sendStatus() with the STATUS TEXT as the body:
# POST /init and the scan route both reply "OK" (measured on the
# pinned 2.37.2 image). A successful call must never be judged by
# whether its body happens to parse as JSON.
return r.status, raw.decode("utf-8", "replace")[:200]
except urllib.error.HTTPError as e:
raw = e.read()
try:
payload = json.loads(raw)
except Exception:
payload = raw.decode("utf-8", "replace")[:200]
return e.code, payload
except urllib.error.URLError as e:
# Not an HTTP answer at all: nothing listening, DNS dead, connection
# refused. HTTPError is a URLError subclass, so this arm must come after.
return 0, "unreachable (%s)" % e.reason
status, st = call("GET", "/status")
if status != 200 or not isinstance(st, dict):
sys.exit("refused: %s/status answered %s (%s) — is the service up?" % (BASE, status, st))
if st.get("isInit"):
print("account: already registered")
else:
code, body = call("POST", "/init", {"newRoot": {"username": user, "password": password}})
if code != 200:
sys.exit("refused: POST /init answered %s (%s)" % (code, body))
print("account: created (%s)" % user)
code, login = call("POST", "/login", {"username": user, "password": password})
if code != 200 or not isinstance(login, dict) or not login.get("user", {}).get("token"):
sys.exit("refused: login answered %s (%s) — the credential file does not match the server"
% (code, login))
token = login["user"]["token"]
print("token: obtained (%s)" % login["user"].get("type", "user"))
code, libs = call("GET", "/api/libraries", token=token)
if code != 200 or not isinstance(libs, dict) or "libraries" not in libs:
sys.exit("refused: GET /api/libraries answered %s (%s)" % (code, libs))
lib = next((l for l in libs["libraries"] if l.get("name") == LIB), None)
if lib:
print("library: exists %s (%s)" % (LIB, lib["id"]))
else:
body = {
"name": LIB,
"provider": "audiobook",
"icon": "book",
"folders": [{"fullPath": FOLDER}],
"settings": {
"coverEncoding": "jpg",
"skipMatchingMediaWithAsin": False,
"skipMatchingMediaWithIsbn": False,
"autoDetectCrossoverAudiobooks": False,
"ignoreAudiobookMetadata": False,
"useAudiobookFileMetadata": False,
},
"providerMetadata": {},
}
code, made = call("POST", "/api/libraries", body, token=token)
if code != 200:
sys.exit("refused: POST /api/libraries answered %s (%s)" % (code, made))
lib = made
print("library: created %s (%s) folder %s" % (LIB, lib["id"], FOLDER))
if SCAN:
code, body = call("POST", "/api/libraries/%s/scan" % lib["id"], {}, token=token,
want_json=False)
if code != 200:
sys.exit("refused: scan answered %s (%s)" % (code, body))
print("scan: accepted")
else:
print("scan: skipped (ABS_SCAN=0)")
code, items = call("GET", "/api/libraries/%s/items?limit=1" % lib["id"], token=token)
if code == 200 and isinstance(items, dict):
print("items: %s (a fresh scan raises this from 0; watch it, not /healthcheck)"
% items.get("total"))
PY
exit 0
+3
View File
@@ -60,6 +60,7 @@ SERVICE_ORDER = (
"calibre-server",
"plex",
"tautulli",
"audiobookshelf",
"prowlarr",
"qbittorrent",
"radarr",
@@ -123,6 +124,7 @@ DANKSERVER_SERVICES = {
"calibre-server",
"plex",
"tautulli",
"audiobookshelf",
"prowlarr",
"qbittorrent",
"radarr",
@@ -420,6 +422,7 @@ def services_for_path(path: str, target: str) -> set[str]:
"copyparty": {"copyparty"},
"plex": {"plex"},
"tautulli": {"tautulli"},
"audiobookshelf": {"audiobookshelf"},
}
_add(services, *quadlet_services.get(parts[1], set()))
if len(parts) >= 2 and parts[1] == "host":
+31
View File
@@ -95,6 +95,7 @@ service_is_allowed() {
dankclaw:frank-tts-engine|dankclaw:frank-tts|\
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
dankserver:audiobookshelf|\
dankserver:prowlarr|dankserver:qbittorrent|dankserver:radarr|\
dankserver:sonarr|dankserver:lidarr|dankserver:readarr|\
dankserver:frank-archon|dankserver:node-exporter|dankserver:cadvisor|dankserver:gpu-exporter|\
@@ -1212,6 +1213,10 @@ ensure_media_view() {
# scripts/wait-for-mount.sh, which waits and then refuses the start. A
# relayout must be followed by `media-view.sh reap`
# (see the Media Relayout page), which this is not.
# A fresh host has no upper base, and `install` refuses to invent one — so
# without this the media deploy dies on a rebuilt machine with
# "put it on the SSD first" (measured 2026-10-31).
run_privileged bash scripts/media-view.sh ensure-base
run_privileged bash scripts/media-view.sh install
bash scripts/media-view.sh verify
}
@@ -1296,6 +1301,31 @@ deploy_calibre_web() {
systemctl --user restart calibre-web.service
}
# Audiobookshelf keeps its whole state (SQLite library, accounts, progress,
# cached covers) on the NVMe under /home/dank/Containers/audiobookshelf; the
# audio is a read-only bind off the array, so the only thing the deployer has to
# create is the two state roots. mkdir names the owner — letting podman invent
# the directories leaves the SQLite file unwritable by the container user on a
# rootless first boot. No `systemctl --user enable`: a generated Quadlet unit
# cannot be enabled, and WantedBy=default.target is what carries it across a
# reboot.
deploy_audiobookshelf() {
cp "dankserver/audiobookshelf/quadlet/audiobookshelf.container" "$quadlet_unit_dir/"
systemctl --user daemon-reload
mkdir -p /home/dank/Containers/audiobookshelf/config \
/home/dank/Containers/audiobookshelf/metadata
systemctl --user restart audiobookshelf.service
# /healthcheck answers once the HTTP server is bound; the library scan that
# follows runs in the background and must not gate the deploy.
wait_for_health audiobookshelf http://127.0.0.1:8000/healthcheck 90
# A server with no root user and no library is "healthy" and useless, so the
# bootstrapper runs after the health gate and its failure fails the deploy:
# it creates the account (credential in the mode-600 host secret file), the
# Audiobooks library over the /audiobooks bind, and fires the scan. Every
# step is idempotent, so a re-deploy costs one incremental scan.
scripts/audiobookshelf-init.sh
}
# The content server's single admin account and the desktop's basic-auth
# credential (same pair) are bootstrapped exactly once: the env file is the
# plaintext store (mode 600), the userdb's pw column is plaintext too, so
@@ -2172,6 +2202,7 @@ deploy_service() {
copyparty) deploy_copyparty ;;
plex) deploy_plex ;;
tautulli) deploy_tautulli ;;
audiobookshelf) deploy_audiobookshelf ;;
alloy) deploy_alloy ;;
prometheus|grafana|loki|node-exporter|cadvisor|alertmanager|blackbox-exporter) deploy_observability "$service" ;;
gpu-exporter) deploy_gpu_exporter ;;
+1
View File
@@ -148,6 +148,7 @@ valid_service() {
dankclaw:frank-dragoon|dankclaw:frank-zealot|\
dankserver:romm-mariadb|dankserver:romm-redis|dankserver:romm|\
dankserver:copyparty|dankserver:jellyfin|dankserver:calibre-web|dankserver:calibre-server|dankserver:plex|dankserver:tautulli|\
dankserver:audiobookshelf|\
dankserver:qbittorrent|dankserver:radarr|dankserver:sonarr|\
dankserver:lidarr|dankserver:readarr|dankserver:node-exporter|dankserver:cadvisor|dankserver:gpu-exporter|\
dankserver:prowlarr|dankserver:frank-archon|\
+73 -2
View File
@@ -35,7 +35,8 @@
# Subcommands
# plan print the resolved shape and every pending change, touch nothing
# install fetch+verify the plugin, then write the keys (backing up first)
# verify prove the plugin and both keys are actually in place
# verify prove the plugin and both keys are in place, and that no library
# is a silent file index (providers off / image fetchers empty)
#
# Nothing here restarts Jellyfin: the deployer owns that
# (`scripts/deploy.sh dankserver --service jellyfin`), and the plugin is read at
@@ -134,7 +135,18 @@ text = escape(val)
if pat.search(src):
out = pat.sub(lambda m: f"{m.group(1)}<{elem}>{text}</{elem}>", src, count=1)
else:
out = src.replace("</Configuration>", f" <{elem}>{text}</{elem}>\n</Configuration>", 1)
# The plugin owns this file and writes it under ITS OWN root element
# (TheTVDB's is <PluginConfiguration>), and its serializer omits a
# null-valued setting entirely — so the element being filled can be
# absent outright, and a hardcoded </Configuration> anchor then matches
# nothing. Anchor on the document's real root instead.
out = src
root = re.search(r"<([A-Za-z_][\w.:-]*)[^>]*>", src)
if root and ("</%s>" % root.group(1)) in src:
name = root.group(1)
out = src.replace(
"</%s>" % name, f" <{elem}>{text}</{elem}>\n</{name}>", 1
)
if out == src:
raise SystemExit(f"{xml_path}: no place to write <{elem}>")
@@ -340,6 +352,64 @@ cmd_install() {
fi
}
# A library whose options.xml carries EnableInternetProviders=false, or an empty
# ImageFetchers list beside a populated MetadataFetchers list, is a SILENT FILE
# INDEX: it indexes every file, answers 204 to every refresh, stores no provider
# id, plot, rating or artwork, and logs nothing — so the item count reads like a
# healthy scan. Jellyfin 12 keeps these options in root/default/<Library>/options.xml
# (it was library.xml) and stores NONE of them in the database, so this audit is
# the only place the state can be read without a browser session.
verify_library_options() {
local root="$JF_ROOT/root/default" f lib problems
if [ ! -d "$root" ]; then
printf '= no library options under %s (nothing audited)\n' "$root"
return 0
fi
local rc=0
for f in "$root"/*/options.xml; do
[ -f "$f" ] || continue
lib="$(basename "$(dirname "$f")")"
problems="$(python3 -s - "$f" <<'PY'
import sys, xml.dom.minidom as X
root = X.parse(sys.argv[1]).documentElement
def child(el, name):
for n in el.childNodes:
if n.nodeType == n.ELEMENT_NODE and n.tagName == name:
return n
def flag(el, name):
n = child(el, name)
return None if n is None else (n.firstChild.data.strip() if n.firstChild else "")
def names(el, name):
n = child(el, name)
return None if n is None else [s.firstChild.data.strip() for s in n.childNodes
if s.nodeType == s.ELEMENT_NODE and s.firstChild]
bad = []
ip = flag(root, "EnableInternetProviders")
if ip is not None and ip.lower() != "true":
bad.append("EnableInternetProviders=%s" % (ip or "empty"))
types = child(root, "TypeOptions")
if types is not None:
for t in types.childNodes:
if t.nodeType != t.ELEMENT_NODE or t.tagName != "TypeOptions":
continue
meta = names(t, "MetadataFetchers") or []
if meta and not (names(t, "ImageFetchers") or []):
bad.append("%s: ImageFetchers empty while MetadataFetchers names %d"
% (flag(t, "Type") or "?", len(meta)))
print("; ".join(bad))
PY
)"
if [ -n "$problems" ]; then
printf 'SILENT INDEX %s: %s\n' "$lib" "$problems" >&2
rc=1
else
printf '= %s: library options fetch from providers\n' "$lib"
fi
done
return "$rc"
}
cmd_verify() {
check_root
local bad=0
@@ -378,6 +448,7 @@ cmd_verify() {
printf '%s %s chars\n' "$elem" "${#n}"
fi
done
verify_library_options || bad=1
[ "$bad" -eq 0 ] || return 1
printf 'providers configured\n'
}
+10 -2
View File
@@ -121,14 +121,14 @@ class Journal:
def read_plan(path, root, force):
"""Plan rows: (mkdirs, keeps, moves), header dropped, all paths under root."""
mkdirs, keeps, moves = [], [], []
mkdirs, keeps, moves, bad = [], [], [], []
with open(path) as fh:
header = fh.readline()
if not header.startswith("action\t"):
raise Refuse(f"{path} is not a plan (first column is "
f"{header.split(chr(9))[0]!r}, expected 'action') —"
" regenerate it with scripts/media-plan.py")
for line in fh:
for n, line in enumerate(fh, 2):
f = line.rstrip("\n").split("\t")
if f[0] == "mkdir":
mkdirs.append(f[2])
@@ -136,6 +136,14 @@ def read_plan(path, root, force):
keeps.append(f[1])
elif f[0] == "move":
moves.append((f[1], f[2], f[3] if len(f) > 3 else ""))
else:
bad.append(f"line {n}: unknown action {f[0]!r} — this tool"
" knows mkdir, keep and move (a rename is a move)")
if bad:
# A row this tool could not read is a plan bug, and a plan bug refuses:
# silently dropping it turns a typo into a no-op run that reports success.
head = "; ".join(bad[:3])
raise Refuse(head + (f" (+{len(bad) - 3} more)" if len(bad) > 3 else ""))
if not force:
for p in mkdirs + keeps + [p for m in moves for p in m[:2]]:
if p and not (p == root or p.startswith(root + os.sep)):
+473
View File
@@ -0,0 +1,473 @@
#!/usr/bin/env python3
"""media-tvdupes.py — find duplicate TV episodes and plan their staging, one
EPISODE at a time.
A TV library grows the same show into several folders: a tagged season box, an
untagged packager drop, a `Complete` folder that is not complete. The folders
look like copies of each other and are not, so the unit of comparison is the
episode and never the folder. A file is droppable only when EVERY episode it
carries is also held, somewhere else in the same show, by a file at least as
large — Frank's rule: the same episode in two places keeps the larger one. A
file that holds one episode nothing else has stays, whatever else it carries.
It writes a plan and moves nothing; `scripts/media-apply.py` is the only thing
that renames. The plan stages into a dated hold, so the pass is undoable from
the journal and frees no space until the hold is purged.
python3 scripts/media-tvdupes.py --library /mnt/entropy/TVShows \
--hold /mnt/entropy/.AGENTTRASH/20261102-tv-duplicates \
--out /mnt/entropy/tv-dupes-plan.tsv
python3 scripts/media-apply.py apply --plan /mnt/entropy/tv-dupes-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/tv-dupes.jsonl \
--metadata-view /var/lib/media-meta/view --apply
python3 scripts/media-apply.py verify --plan /mnt/entropy/tv-dupes-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/tv-dupes.jsonl
python3 scripts/media-apply.py sweep --root /mnt/entropy \
--journal /mnt/entropy/tv-dupes.jsonl --apply # empty folders only
scripts/media-view.sh reap # ghost sidecars above
Then both servers have to be told: the plan moves files on the raw array, and no
watcher crosses the metadata view — [[Media Relayout]] owns the refresh order.
Guards, and why each one is here:
* **Identity before geometry.** Two folders holding the same words are not the
same show: the 2026-11-02 census found `Avatar: The Last Airbender` (1999,
tvdb 74852) beside the 2024 remake (385925), `Star Trek: Voyager` beside
`Star Trek: The Animated Series`, and a German thriller Jellyfin had titled
`Millennium (2022) [tvdbid-472933]` beside `Millennium (1996)`. A folder's
identity is read from three places, in this order: its `[tvdbid-N]` tag, the
`tvshow.nfo` Jellyfin wrote — which lives in the metadata view, not on the
array, and is what `--meta` reads — and a `folder<TAB>id` map passed with
`--ids`. The tag outranks the derived layers: an id that contradicts a folder's
own tag is reported and ignored, never unioned — measured 2026-11-02, Jellyfin's
provider table carried a 2026 special's id on a folder tagged `[tvdbid-73838]`,
and unioning them makes one show of the 2000 series and the special, whose bigger
files then dominate the original's pilot. Two folders are one show when they share
an id, or when neither has one and name and year both agree. Two disjoint id sets
is an absolute stop: the pair is reported as a conflict and nothing under either is
staged. Same name with nothing to decide on is reported unresolved — the fix is the
tag, not a guess.
* **A folder-level verdict destroys content.** 10 of the 25 shows that looked
duplicated held COMPLEMENTARY halves: `Game of Thrones Complete` carries no
S01 and no S08, `Modern Family S05` is the only copy of season 5, `Loiter
Squad Uncensored` is the only copy of season 1. Judged per folder, 20 shows
and some 340 GB of unique episodes would have been staged; judged per episode
the true figure was a sixth of that.
* **A bare number is never an episode.** `256491.mp4` may be an episode id, a
poster, or a season, so a name that states no `SxxEyy` / `1x04` marker is
unkeyed: never staged, and counted in the report. A media server's own episode
table does know those files — pass it with `--witness` and its rows key
whatever the server has matched.
* **The audit re-reads the disk.** Sizes and the survivor's existence are
checked again when the plan is written, because the census is a snapshot: a
file that grew, vanished, or landed in the hold in between must fail the plan,
not surprise the executor.
* **A sidecar that shares an episode's stem follows it.** Subtitles and `.nfo`
left at the old coordinates are ghost bait for `media-view.sh reap`.
* A multi-episode file (`S01E01-02`) is dominated only when every episode in its
range is held elsewhere — and it competes for the survivor seat on each of
them, because the byte law compares files. A pack that states its range in
the name is a name both media servers parse the same way, so the pack winning
against two single-episode rips is the law applied, not a misread.
"""
import argparse
import collections
import importlib.util
import json
import os
import re
import sys
ENGINE = os.path.join(os.path.dirname(os.path.abspath(__file__)), "media-titles.py")
_spec = importlib.util.spec_from_file_location("eng", ENGINE)
eng = importlib.util.module_from_spec(_spec)
_spec.loader.exec_module(eng)
# The executor's definition of a media file, plus the two the title engine omits:
# an audit that disagrees with media-apply.py is a plan that dies at its
# preflight instead of at the census.
VIDEO = tuple(sorted(set(eng.VIDEO_SUFFIX) | {".ts", ".wmv"}))
TAG_RE = re.compile(r"\[\s*tvdbid[\s._-]*(\d+)\s*\]", re.I)
YEAR_RE = re.compile(r"[(\[]\s*((?:19|20)\d\d)\s*[)\]]")
SEASON_EP = re.compile(r"(?i)\bs(?:eason)?[\s._-]*(\d{1,2})[\s._-]*e(?:p(?:isode)?)?"
r"[\s._-]*(\d{1,3})(?:[\s._-]*(?:-|to|\+)[\s._-]*e?"
r"[\s._-]*(\d{1,3}))?")
NXN = re.compile(r"(?i)(?<!\d)(\d{1,2})x(\d{1,3})(?!\d)")
def parse_args():
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("--library", default="/mnt/entropy/TVShows",
help="the TV library to census (never written)")
ap.add_argument("--hold", required=True,
help="dated trash the plan stages into, e.g. "
"/mnt/entropy/.AGENTTRASH/20261102-tv-duplicates")
ap.add_argument("--out", default=None,
help="plan TSV for scripts/media-apply.py (default: "
"tv-dupes-plan.tsv beside --library)")
ap.add_argument("--witness", default=None,
help="TSV of path<TAB>season<TAB>episode out of a media "
"server's episode table; keys files whose names carry"
" no marker")
ap.add_argument("--meta", default=None,
help="the merged metadata view (on the host "
"/var/lib/media-meta/view): tvshow.nfo lives in the"
" upper layer, so this is where an untagged folder's"
" tvdb id is readable")
ap.add_argument("--ids", default=None,
help="TSV of folder<TAB>tvdbid, one line per folder, for"
" identity a lookup settled and the folder name does"
" not state")
ap.add_argument("--json", default=None,
help="also dump the analysis as JSON")
return ap.parse_args()
ARGS = parse_args()
def episode_keys(name):
"""Every (season, episode) an episode name states.
`S01E03`, `Season 1 E3`, `1x04`, and a range `S01E01-02`, which expands so a
two-in-one file is only droppable when both halves are held elsewhere. A
bare number states nothing and yields nothing."""
keys = set()
for m in SEASON_EP.finditer(name):
s, first = int(m.group(1)), int(m.group(2))
keys.add((s, first))
if m.group(3):
keys.update((s, e) for e in range(first + 1, int(m.group(3)) + 1))
keys.update((int(m.group(1)), int(m.group(2))) for m in NXN.finditer(name))
return keys
def folder_identity(folder):
"""(fold_sep name, year, tvdb ids) for a top-level show folder.
Tag and year come in both orders — `Name (2008) [tvdbid-84859]` and `Name
[tvdbid-84859] (2008)` are both in the library — so neither is assumed to
sit at the end, and both are cut out of the name folders compare on."""
ids = frozenset(int(x) for x in TAG_RE.findall(folder))
year = None
for _ in range(3):
y = YEAR_RE.search(folder)
if not y:
break
year = year or y.group(1)
folder = TAG_RE.sub(" ", f"{folder[:y.start()]} {folder[y.end():]}")
return eng.fold_sep(TAG_RE.sub(" ", folder)), year, ids
def nfo_id(path):
"""The tvdb id a `tvshow.nfo` states, or None.
Three shapes are in the library: Jellyfin's `<uniqueid type="tvdb">`, Kodi's
`<tvdbid>`, and a packager's `<id>` + `<id_type>` pair; some carry the id
only in the title line as `[tvdbid-N]`."""
try:
with open(path, encoding="utf-8", errors="replace") as fh:
text = fh.read(16384)
except OSError:
return None
for rx in (r"<tvdbid\s*>\s*(\d+)\s*<",
r"<uniqueid[^>]*type\s*=\s*[\"']?tvdb[^>]*>\s*(\d+)\s*<",
r"<id\s*>\s*(\d+)\s*</id>\s*<id_type>\s*(?:tvdb|thedotdb)",
r"\[\s*tvdbid[\s._-]*(\d+)\s*\]"):
m = re.search(rx, text, re.I)
if m:
return int(m.group(1))
return None
def collect_ids(library, meta, ids_file):
"""top-level folder -> {tvdb ids} from the nfo layer and an operator map.
`meta` is the merged metadata view: Jellyfin's sidecars live in the upper
layer, so a folder's `tvshow.nfo` is not on the raw array. Reading the id
from there is what makes an untagged packager folder identifiable at all —
without it the pair stays unresolved and nothing under it is staged."""
out = collections.defaultdict(set)
if meta:
try:
tops = sorted(os.listdir(library))
except OSError:
tops = []
for folder in tops:
i = nfo_id(os.path.join(meta, folder, "tvshow.nfo"))
if i:
out[folder].add(i)
if ids_file:
with open(ids_file, encoding="utf-8", errors="replace") as fh:
for line in fh:
cols = line.rstrip("\n").split("\t")
if len(cols) < 2 or not cols[1].strip().isdigit():
continue
out[cols[0]].add(int(cols[1].strip()))
return out
def witness_rows(path):
"""absolute path -> {(season, episode)} from a media server's episode table."""
out = collections.defaultdict(set)
if not path:
return out
with open(path, encoding="utf-8", errors="replace") as fh:
for line in fh:
cols = line.rstrip("\n").split("\t")
if len(cols) < 3 or not cols[0].startswith("/"):
continue
try:
out[os.path.abspath(cols[0])].add((int(cols[1]), int(cols[2])))
except ValueError:
continue
return out
def sidecars(path):
"""The files beside PATH carrying its exact stem — subtitles, nfo, artwork."""
folder, stem = os.path.split(path)
stem = os.path.splitext(stem)[0]
out = []
try:
entries = sorted(os.listdir(folder))
except OSError:
return out
for e in entries:
p = os.path.join(folder, e)
if p == path or os.path.islink(p) or not os.path.isfile(p):
continue
if os.path.splitext(e)[0] == stem:
out.append(p)
return out
def scan(library, hold, witness):
"""Every video file under LIBRARY: relative path, size, episode keys.
A file keys from the witness when it has a row (a server matched it), and
from its own name otherwise; with neither it is unkeyed."""
files = []
for dirpath, dirnames, filenames in os.walk(library):
dirnames[:] = sorted(d for d in dirnames
if not d.startswith(".") and d != "@eaDir"
and os.path.abspath(os.path.join(dirpath, d)) != hold)
for f in sorted(filenames):
p = os.path.join(dirpath, f)
if os.path.islink(p) or not f.lower().endswith(VIDEO):
continue
try:
size = os.path.getsize(p)
except OSError:
continue
files.append({"path": os.path.abspath(p),
"rel": os.path.relpath(os.path.abspath(p), library),
"size": size,
"keys": set(witness.get(os.path.abspath(p))
or episode_keys(os.path.splitext(f)[0]))})
return files
def show_groups(folders, extra_ids):
"""(show groups, conflicts, unresolved, disagreements) over top-level folders.
A folder's id is its `[tvdbid-N]` tag; the nfo layer and `--ids` fill a folder
only when it carries no tag. The rule, in order of trust: an id in common makes
one show; two disjoint id sets are two shows, reported as a conflict and never
merged; no id on either side merges only on name AND year (the 1999 miniseries
and the 2015 series are not one show); one side ided and the other bare is
unresolved, because the fix is the tag.
The derived layers never overrule a tag. Measured 2026-11-02: Jellyfin's
provider table carried a 2026 special's tvdb id on the folder tagged
`[tvdbid-73838]`, and unioning the two made one show of the 2000 series and the
special — whose bigger files then dominate the original's pilot. A contradiction
is reported instead, because the tag is the operator's word and the provider
table is a guess that can be wrong.
"""
names = {f: folder_identity(f) for f in folders}
tags = {f: set(names[f][2]) for f in folders}
derived = {f: set(extra_ids.get(f, ())) for f in folders}
disagree = {f: (sorted(tags[f]), sorted(derived[f] - tags[f]))
for f in folders if tags[f] and derived[f] - tags[f]}
ids = {f: (tags[f] or derived[f]) for f in folders}
parent = {f: f for f in folders}
def find(f):
while parent[f] != f:
parent[f] = parent[parent[f]]
f = parent[f]
return f
conflicts, unresolved = [], []
order = sorted(folders)
for i, a in enumerate(order):
for b in order[i + 1:]:
na, ya, _ia = names[a]
nb, yb, _ib = names[b]
if na != nb:
continue
ia, ib = ids[a], ids[b]
if ia & ib:
parent[find(b)] = find(a)
elif ia and ib:
conflicts.append((a, b, f"tvdb {sorted(ia)} vs {sorted(ib)}"))
elif not ia and not ib and ya and ya == yb:
parent[find(b)] = find(a)
elif not ia and not ib:
conflicts.append((a, b,
f"same name, years {ya or '?'} vs {yb or '?'}"))
else:
unresolved.append((a, b, "one folder carries a tvdb id,"
" the other none"))
groups = collections.defaultdict(list)
for f in order:
groups[find(f)].append(f)
return sorted(groups.values()), conflicts, unresolved, disagree
def rank(rec):
"""Survivor order: biggest, then shortest path, then lexicographic.
A strict total order per key means a key's winner is never itself dominated,
so no chain of staging can cycle and exactly one copy of a byte-identical
pair is staged."""
return (-rec["size"], len(rec["path"]), rec["path"])
def main():
library = os.path.abspath(ARGS.library)
hold = os.path.abspath(ARGS.hold)
out = ARGS.out or os.path.join(os.path.dirname(library), "tv-dupes-plan.tsv")
if not os.path.isdir(library):
sys.exit(f"refused: {library} is not a directory")
if hold == library or hold.startswith(library + os.sep):
sys.exit(f"refused: --hold {hold} sits inside the library; a hold the"
" census walks is a plan that eats its own output")
files = scan(library, hold, witness_rows(ARGS.witness))
folders = sorted({r["rel"].split(os.sep)[0] for r in files})
ids = collect_ids(library, ARGS.meta, ARGS.ids)
groups, conflicts, unresolved, disagree = show_groups(folders, ids)
by_show = collections.defaultdict(list)
for members in groups:
for r in files:
if r["rel"].split(os.sep)[0] in members:
by_show[members[0]].append(r)
rows, audits, report, shows_json = [], [], [], []
for show in sorted(by_show):
recs = by_show[show]
holders = collections.defaultdict(list)
for r in recs:
for k in r["keys"]:
holders[k].append(r)
winner = {k: min(v, key=rank) for k, v in holders.items()}
dominated = [r for r in recs if r["keys"]
and all(winner[k] is not r for k in r["keys"])]
if not dominated:
continue
dup_keys = sorted({k for r in dominated for k in r["keys"]})
sole = sum(1 for v in holders.values() if len(v) == 1)
staged_folders = {os.path.dirname(r["rel"]) for r in dominated}
for r in dominated:
for k in r["keys"]:
t = winner[k]
try:
now_src = os.path.getsize(r["path"])
now_twin = os.path.getsize(t["path"])
except OSError:
audits.append(f"gone since the census: {r['rel']}"
f" or its survivor {t['rel']}")
break
if now_twin < now_src:
audits.append(f"survivor smaller on disk: {r['rel']}"
f" {now_src} > {t['rel']} {now_twin}")
for r in dominated:
dst = os.path.join(hold, r["rel"])
rows.append((r["path"], dst))
for side in sidecars(r["path"]):
rows.append((side, os.path.join(hold, os.path.relpath(side, library))))
shows_json.append({"show": show, "files": len(dominated),
"bytes": sum(r["size"] for r in dominated),
"episodes": len(dup_keys),
"sole-episode files left": sole,
"folders": sorted(staged_folders)})
n_ep, n_folder = len(dup_keys), len(staged_folders)
report.append(
f"{show}: stage {len(dominated)} files"
f" ({sum(r['size'] for r in dominated) / 2**30:.1f} GiB) — {n_ep}"
f" episode{n_ep != 1 and 's' or ''} held twice or more; {sole}"
f" episode{sole != 1 and 's' or ''} exist in one copy only, so"
f" {n_folder} folder{n_folder != 1 and 's' or ''} keep their files")
rows = sorted(set(rows))
dests = set()
for src, dst in rows:
if dst in dests:
audits.append(f"destination twice: {dst}")
dests.add(dst)
if os.path.exists(dst):
audits.append(f"hold target occupied: {dst}")
if not os.path.isfile(src):
audits.append(f"source missing: {src}")
unkeyed = collections.Counter(os.path.dirname(r["rel"]) for r in files
if not r["keys"])
if ARGS.json:
with open(ARGS.json, "w") as fh:
json.dump({"shows": shows_json,
"conflicts": [list(c) for c in conflicts],
"unresolved": [list(u) for u in unresolved],
"disagreement": {f: [t, d] for f, (t, d) in disagree.items()},
"unkeyed": dict(unkeyed)}, fh, indent=1)
for line in report:
print(line)
for a, b, why in conflicts:
print(f"CONFLICT (never merged): {a} | {b} — {why}")
for a, b, why in unresolved:
print(f"UNRESOLVED (tag it, then re-run): {a} | {b} — {why}")
for f, (tagged, extra) in sorted(disagree.items()):
print(f"DISAGREE (the tag wins): {f} — tag tvdb {tagged},"
f" derived tvdb {extra}")
if unkeyed:
print(f"unkeyed, never staged: {sum(unkeyed.values())} files in"
f" {len(unkeyed)} folders name no SxxEyy and have no --witness row;"
" most: " + ", ".join(f"{d} ({n})" for d, n in unkeyed.most_common(5)))
# The gate runs before the plan is written: a plan file that exists is a plan
# somebody will point the executor at, so a failed audit leaves no artifact.
if audits:
print(f"AUDIT FAILED ({len(audits)})", file=sys.stderr)
for a in audits[:10]:
print(f" {a}", file=sys.stderr)
print(f"refused: no plan written to {out}", file=sys.stderr)
return 1
mkdirs = sorted({os.path.dirname(dst) for _s, dst in rows},
key=lambda p: (p.count(os.sep), p))
with open(out, "w") as fh:
fh.write("action\tsrc\tdst\tkind\n")
for d in mkdirs:
fh.write(f"mkdir\t\t{d}\tdir\n")
for src, dst in rows:
fh.write(f"move\t{src}\t{dst}\ttvdup\n")
held = sum(os.path.getsize(s) for s, _d in rows if os.path.isfile(s))
print(f"plan: {len(rows)} moves ({held / 2**30:.1f} GiB) into"
f" {len(mkdirs)} hold folders -> {out}")
print("audit: every staged file has a survivor in the same show holding each of"
" its episodes at >= its size on disk, one destination each")
if rows:
root = os.path.dirname(library)
print(f"next: python3 scripts/media-apply.py apply --plan {out} --root"
f" {root} --journal {os.path.join(root, 'tv-dupes.jsonl')}"
" --metadata-view /var/lib/media-meta/view (dry run; add --apply)")
return 0
if __name__ == "__main__":
raise SystemExit(main())
+18 -1
View File
@@ -60,7 +60,7 @@ die() {
}
usage() {
printf 'usage: %s plan|verify|install|reap|status [options]\n' "$0" >&2
printf 'usage: %s plan|ensure-base|verify|install|reap|status [options]\n' "$0" >&2
exit 2
}
@@ -98,6 +98,22 @@ require_upper_base() {
|| die "refused: upper base $upper_base does not exist (put it on the SSD first)"
}
# The SSD placement of the metadata layer is a human choice, so `install` refuses
# to invent a base directory: a mistyped --upper-base must not quietly create a
# tree on whatever filesystem the caller landed on. The deployer owns the
# documented location, so it calls this first — measured 2026-10-31, when a
# fresh-host `install` died with "put it on the SSD first" and nothing in git
# would have created it.
do_ensure_base() {
if [ -d "$upper_base" ]; then
printf 'base: %s exists\n' "$upper_base"
return 0
fi
install -d -m 0755 "$upper_base" 2>/dev/null \
|| die "refused: cannot create $upper_base — it lives outside the checkout, run this as root"
printf 'base: created %s\n' "$upper_base"
}
do_plan() {
printf 'lowerdir=%s\nupperdir=%s\nworkdir=%s\nmountpoint=%s\nunit=%s/%s\n' \
"$lower" "$upper" "$work" "$VIEW" "$UNIT_DIR" "$UNIT"
@@ -234,6 +250,7 @@ do_status() {
case "$cmd" in
plan) do_plan ;;
verify) do_verify ;;
ensure-base) do_ensure_base ;;
install) do_install ;;
reap) do_reap ;;
status) do_status ;;
+7 -1
View File
@@ -104,6 +104,12 @@ for path in "${CHANGED[@]}"; do
dankserver/jellyfin/quadlet/*|dankserver/plex/quadlet/*)
add_test tests/media_overlay.bats
;;
# The Audiobookshelf bootstrapper (account + library + scan over the API)
# and the deployer hook that runs it after the health gate.
scripts/audiobookshelf-init.sh|tests/audiobookshelf_init.bats)
add_test tests/audiobookshelf_init.bats
add_test tests/deployment_flow.bats
;;
shared/homepage/*)
add_test tests/homepage_caddy.bats
add_test tests/homepage_config.bats
@@ -334,7 +340,7 @@ for path in "${CHANGED[@]}"; do
# Cutover-era dankserver quadlet units (romm trio, jellyfin, calibre-web,
# copyparty, plex, tautulli): the seam checks boot-persistence +
# v1-generator contract; deploy-flow seams guard the deployer routing.
dankserver/romm/*|dankserver/jellyfin/*|dankserver/calibre-web/*|dankserver/copyparty/*|dankserver/plex/*|dankserver/tautulli/*)
dankserver/romm/*|dankserver/jellyfin/*|dankserver/calibre-web/*|dankserver/copyparty/*|dankserver/plex/*|dankserver/tautulli/*|dankserver/audiobookshelf/*)
add_test tests/dankserver_quadlet.bats
add_test tests/deployment_flow.bats
add_test tests/deployment_scoping.bats
@@ -23,7 +23,7 @@ This repository tests its configuration at defined seams. The tests run with `ba
| Deployment flow | `tests/deployment_flow.bats`, `tests/deployment_scoping.bats` | scoped dispatcher exists, targets are known, service plans isolate inference, Caddy ownership is guarded, and SSH reachability is checked (skips offline) |
| Quadlet units | `tests/*quadlet*.bats` | every unit is boot-persistent, self-healing, repo-provenanced, secret-free |
| E2E (browser) | `tests/e2e/` + `tests/e2e_scaffold.bats` | Playwright drives the live guard chain; the bats seam asserts structure and credential hygiene. See [[E2E Suite]] |
| Per-service / unit seams | `tests/llm_units.bats`, `tests/frank_{archon,dragoon,zealot}_unit.bats`, `tests/llm_secrets.bats`, `tests/homepage_{caddy,checker,config,deploy,pages}.bats`, `tests/{anubis,authentik,hermes_webui,pihole,mount_entropy,searxng_limiter,autoresearch_measure,changed_tests}.bats`, `tests/{caddy,dankserver,frankbook,marreta,observability,open_webui,searxng}_quadlet.bats`, `tests/comfyui_rocm.bats` | the ~30 named seams the summary rows above do not list: LLM systemd-unit shape and env hygiene, homepage config/pages/deploy behaviour, and one detail-seam per deployed service |
| Per-service / unit seams | `tests/llm_units.bats`, `tests/frank_{archon,dragoon,zealot}_unit.bats`, `tests/llm_secrets.bats`, `tests/homepage_{caddy,checker,config,deploy,pages}.bats`, `tests/{audiobookshelf_init,anubis,authentik,hermes_webui,pihole,mount_entropy,searxng_limiter,autoresearch_measure,changed_tests}.bats`, `tests/{caddy,dankserver,frankbook,marreta,observability,open_webui,searxng}_quadlet.bats`, `tests/comfyui_rocm.bats` | the ~30 named seams the summary rows above do not list: LLM systemd-unit shape and env hygiene, homepage config/pages/deploy behaviour, and one detail-seam per deployed service |
## Focused pre-commit testing
@@ -0,0 +1,203 @@
---
type: reference
description: "Audiobookshelf on Dankserver: quadlet shape, the read-only array bind, state on the SSD, the API bootstrap, laws."
tags:
- dankserver
- audiobooks
- quadlet
timestamp: 2026-11-02
---
# Audiobookshelf
Audiobookshelf (ABS) serves the [[Entropy Volume]] audiobook library to browsers
and phones: it scans `/mnt/entropy/Audiobooks`, keeps its own SQLite library with
per-listener playback positions, and streams the audio. It is **not** a media
server — it shares nothing with [[Plex]] or [[Jellyfin]] and owns no files on the
array.
**Host:** Dankserver — **Unit:** `audiobookshelf.service` (rootless Quadlet, user
manager) — **Internal-only:** LAN address, no vhost, no DNS name.
## Shape
| Item | Value |
|------|-------|
| Unit file | `dankserver/audiobookshelf/quadlet/audiobookshelf.container` (the authority for every number here) |
| Image | `ghcr.io/advplyr/audiobookshelf` pinned by **amd64 manifest digest** — see Laws |
| Port | `Environment=PORT=8000` + `PublishPort=8000:8000` — web UI and API on one port |
| Base path | `Environment=ROUTER_BASE_PATH=/` — the URL is the plain `host:port`, no `/audiobookshelf` prefix |
| Network | `pasta` (standalone; publishes nothing to other containers) |
| Audio bind | `/mnt/entropy/Audiobooks:/audiobooks:ro` — read-only, by design |
| State | `/home/dank/Containers/audiobookshelf/config` → `/config`, `…/metadata` → `/metadata` (both NVMe) |
| Mount gate | `ExecStartPre=…/wait-for-mount.sh /mnt/entropy` |
| Health | `wget` against `/healthcheck` inside the container, 30 s / 3 retries |
| I/O share | `IOWeight=10` — a scan loses the array to every other tenant |
| Self-healing | `Restart=always`, `RestartSec=5`, `WantedBy=default.target` |
| Deploy | `./scripts/deploy.sh dankserver --service audiobookshelf` |
| Handler | `deploy_audiobookshelf` in `scripts/deploy-remote.sh` |
| Library layout | one book per folder — ABS groups by the first directory holding a media file (see Laws); every move is journalled at `/mnt/entropy/media-relayout-audiobooks.jsonl` |
## Bootstrap
`scripts/audiobookshelf-init.sh` — run by the deployer after the health gate, so a
deployed ABS is a *usable* ABS: `/healthcheck` answers 200 on a server with no
user, no library and no scan, and nothing else in the deploy notices the
difference. The script is idempotent; re-running it costs one incremental scan.
| Step | Door | Printed |
|------|------|---------|
| credential | `~/.config/secrets/audiobookshelf` (mode 600, generated once, host-side) | `credential: created …` |
| account | `POST /init` with `{"newRoot":{"username","password"}}` | `account: created (frank)` / `already registered` |
| session | `POST /login` → bearer token | `token: obtained (root)` |
| library | `POST /api/libraries`, folder `/audiobooks`, provider `audiobook` | `library: created Audiobooks (<id>)` / `exists` |
| scan | `POST /api/libraries/<id>/scan` | `scan: accepted` |
| proof | `GET /api/libraries/<id>/items?limit=1` | `items: <total>` |
Sign in at `http://192.168.0.235:8000` as `frank`; the password is
`awk -F= '/^ABS_ADMIN_PASSWORD=/{print $2}' ~/.config/secrets/audiobookshelf` on
Dankserver. Change it in **Settings → Users → frank → Password**; the secret file
then holds a stale value and only matters again if the `config` bind is ever
destroyed (a fresh DB makes the script create a new owner from it).
A full scan is slow and runs in the background; watch the `items:` total climb,
never `/healthcheck`. It also reads files that are genuinely broken and logs
`[AudioFileScanner] Invalid data found when processing input` per file — that is
the file, not the bind: the same line count grows while the total climbs.
## Laws
- **Speak the image's env names, not the README's.** The server reads `PORT`,
`CONFIG_PATH`, `METADATA_PATH` and `ROUTER_BASE_PATH`; the `ABS_*` family printed
by the server's own README is the npm package's naming and is **inert** in this
image. Measured 2026-11-02 on the first deploy: with `ABS_PORT=8000` the server
listened on `:80` under base path `/audiobookshelf`, so `systemctl` said
`active (running)` while every probe of `:8000` was refused and the deploy gate
failed at 180 s. Two seconds with the pinned digest settles it:
`podman inspect --format '{{.Config.Env}}' <image>`.
- **The audio bind stays `:ro`.** ABS writes nothing that belongs beside the
audio: its library, covers, progress and sidecar metadata all land in
`/metadata` on the SSD. A writable bind would put stray writes on the spindle
and hand the array to a service with no reason to hold it. If per-book
`.opf`/`.nfo` sidecars on disk are ever wanted, the rollback is one line —
point the audio bind at the [[Media Metadata View]] the media servers already
use — but that view's upper layer exists for those two servers' sidecars, so a
third writer needs a reason first ([[Media Metadata View]]).
- **Do not bind the raw array without the mount gate.** In a rootless unit
`ConditionPathIsMountPoint=` is fail-*dead* and `RequiresMountsFor=` is inert,
so a missing array does not stop the start: podman invents an empty directory,
ABS indexes it, and an empty library becomes recorded truth. `wait-for-mount.sh`
is the only gate that works here — same law the media servers obey
([[Quadlet Lifecycle]]).
- **The `HealthCmd` must use `wget`.** The image is Alpine and ships `wget`, not
`curl`; a `curl` health command reads a perfectly healthy server as unhealthy
and `Restart=always` turns that into a loop.
- **The image pin is the amd64 manifest digest, not the index digest.**
`podman inspect` reports the platform manifest (`RepoDigests[0]`) for what was
pulled, which differs from the manifest-list digest `latest` points at. Pin the
one this host runs. `latest` is newer than the newest version tag: upstream
stopped publishing version tags after `2.19.0` while `latest` kept moving, so
"highest version tag" is stale here — read the index, don't guess from tags.
- **`IOWeight=10` is inert without the host drop-ins.** `dankserver/host/systemd/`
carries `user.slice.d/io-accounting.conf` and the `user@1000` delegate, and the
deployer installs both; without them systemd silently drops the weight
([[Quadlet Lifecycle]]).
- **No `systemctl --user enable`.** Generated Quadlet units cannot be enabled on
this host; `WantedBy=default.target` in the unit file is what carries the
service across a reboot ([[Quadlet Lifecycle]]).
- **LAN-only until Frank says otherwise.** `0.0.0.0:8000` with no reverse proxy
in front. Any public vhost requires the Anubis + Authentik chain first, and the
outpost proxy provider has to exist before the vhost flips (external exposure
policy in `AGENTS.md`).
- **There is no `/setup` route; the creation door is `POST /init`.** `/status`
(no `/api` prefix) reports `isInit` = "a root user exists", `POST /init` with
`{"newRoot":{"username","password"}}` creates it, `POST /login` returns the
bearer every `/api/*` route wants. `POST /setup` answers `Unauthorized` — that
is a missing route wearing an auth message. Read off `dist-server/server/Server.js`
inside the pinned digest rather than from a forum.
- **A 200 may carry plain text.** `res.sendStatus(200)` answers the body `OK`, and
`/init` and the scan route use it; a client that parses every success as JSON
crashes on the one call that matters (`tests/audiobookshelf_init.bats` pins it).
- **The item count is not a field on the library DTO, and paging never ends
itself.** The door is `GET /api/libraries/<id>/items?limit=1` → `total`; a poller
that guesses a plausible name (`numItems`) prints `0` forever beside a scan that
is working. Paging is on `page` (0-based) plus `limit`, and **`offset` is
ignored**: `limit=500&offset=500` returns the same 500 ids as `offset=0`, while
`page=1` shares none of them. A census that pages on `offset` therefore reads
`ceil(total / limit) × limit` rows containing only `limit` distinct items — the
exact origin of the "exactly 1,500" and "exactly 4,000" counts two runs
reported for one library. Page on `page`, count *unique ids*, stop on `total`,
and take the number after the pass settles (`total` moves during a scan).
- **A fresh scan is the only thing that notices new audio.** `inotify` does not
cross a rootless podman bind — measured for [[Jellyfin]], whose page owns that
law — so the in-container watcher sees nothing a host process files onto the
array, and the deployer's re-run of the bootstrapper is the cheap scan trigger.
- **One stray media file at a collection root turns the whole collection into one
book.** Every scan re-walks the tree from disk (`scanFolder` → `recurseFiles`),
and `utils/scandir.js:groupFileItemsIntoLibraryItemDirs` assigns each file to the
FIRST directory along its path that already has a group: the first file seen at a
root owns every deeper file, and the folders below it never become items. One
949 MB `.cbr` sitting at `_unsorted`'s root collapsed 652 book folders into a
single item of 5,693 tracks, titled after whichever file supplied metadata.
"Hundreds of books are missing" is a layout fault, not a scan fault.
- **"Media file" means audio OR ebook, so an audio-only census cannot see the file
that is hiding the library.** With `audiobooksOnly=false` (the deployed shape)
the predicate is `SupportedAudioTypes` ∪ `SupportedEbookTypes`: 20 audio
extensions (including `mp4`, `mpg`, `webm`) plus `epub pdf mobi azw3 cbr cbz`. A
census reporting "that root holds no loose audio" can be correct and still
useless — the file doing the shadowing here was a comic archive.
- **Deleting the covering item does not un-shadow anything.** Grouping is recomputed
from disk, so the next scan rebuilds the item from the same stray file (measured:
`DELETE`, then scan → `1 Added`, the item back at the same path). The fix is the
MOVE: give every stray root file a folder of its own (one file per folder is
ABS's own single-book layout; formats of one book share a folder), through the
journal-guarded mover, then delete the stale item so the old path is not kept,
then scan.
- **`?force=1` is neither a discovery arm nor a repair arm.** Both arms of
`POST /libraries/:id/scan` walk the disk, so finding new folders needs neither
and the deployer's cheap trigger stays on the plain arm. Force additionally
re-scans every item that already exists (measured 1:33 against 0:04 on ~4,600
items) and repaired nothing on a library whose residuals were file-level: item
count, path set and media-less count identical before and after.
- **`DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()`.** In
`LibraryItemController.delete`, `hardDelete = req.query.hard == 1` and then
`fs.remove(libraryItemPath)`. Pointed at a covering item, that path is the whole
swallowed subtree — `hard=1` there deletes every book beneath it off the array.
Assert the item's `path` first, and never pass `hard`.
- **`Invalid data found when processing input` names the file, not the bind.** The
scanner logs it per file and skips that file; a book whose files ALL fail never
becomes an item at all, so the disk-vs-DB folder census shows it as a book with
no item. That is corrupt media, not a lost book. Never clear a suspect file with
a lenient probe: `ffprobe -show_entries format=duration` happily returns three
hours for the WAV bytes wearing a `.mp3` name that the scanner refuses. Read the
log's file paths instead, and `stat -c %s` first — 0-byte files are the common
case, and a truncated download reports `moov atom not found`.
- **A `Disc 01` / `CD 1` subfolder is part of a book, not a book.** The extension
census counts those folders; ABS deliberately does not — its grouping attaches
their files to the ancestor item (measured: 110 such folders under one item
carrying all 192 tracks). Read the basename before believing a disk-vs-DB gap.
- **`.cbr`/`.cbz` group like media and never play.** They sit in the grouping
predicate, so they hide books, but the comic parser throws `ERAR_BAD_ARCHIVE`
(`parseComicMetadata` → `CbrComicBookExtractor`) and the item lands with no
media. A media-less row beside those lines is the file, not the scanner.
- **A scan is finished when a NEW scan uuid completes.** Match
`Library scan "<uuid>" completed in ...`. `podman logs --since 30m` is a sliding
window: the set of visible completion lines changes on its own as old lines age
out, so a poller keyed on that hash reads a mid-scan number and calls it done.
## Verification
`./tests/run.sh` covers this unit through `tests/dankserver_quadlet.bats`
(boot persistence, the SSD state roots, the `:ro` array bind, the mount gate, the
`wget` health check), `tests/audiobookshelf_init.bats` (the bootstrapper against a
stub of the API above: create-once, idempotent re-run, no secret in a URL, refusal
by URL when the server does not answer, and the deployer calling it only after the
health gate) and `tests/media_overlay.bats` (only [[Plex]] and [[Jellyfin]] may
reference the metadata view).
Five book folders hold media no scanner can read, so they stay out of the library:
three 2024 Nebula nominees in the `_unsorted` box (two `.m4b` with no `moov` atom,
one 36-file `.mp3` set that is not MPEG audio at all), `Anthony Heald/Bounty
Hunter Wars - Book 1` and `Star Wars Full Cast Audio Drama/Dark Empire III` — the
latter two also exist in readable form under their author folders. Fix by
re-acquiring the files; the scanner names every one of them in its log.
@@ -66,3 +66,24 @@ exactly describes the year this bug survived.
volume absent they start `active (healthy)` against empty directories, which is
how a missing 91 TiB volume hides. See [[Dankserver Stack]] for the stack and
[[Storage Policy]] for what may be written here.
## Write doors
Something renamed movie folders on this array with no journal in any of the
`/mnt/entropy/*.jsonl` plans, no bash history, no cron, no timer, and no job
artifact. The census of things that can mutate it:
- **A local process** as `dank` or root. `scripts/media-apply.py` is the only
one that journals; anything else (a hand `mv`, another session's script) is
silent.
- **SFTP by a human.** GNOME Files on a workstation mounts the array through
`gvfsd-sftp`, which runs `sftp-server` on this host as `dank`. Renames and
moves are direct `rename()` calls: no shell, no history, no journal. A folder
rename bumps **only that directory's ctime** — its mtime and every child's
ctime stay at their old values, so the files look untouched while their paths
are gone. Diagnose it with `pgrep -af gvfsd-sftp` on the workstation and
`ss -tnp` for the peer.
- **Copyparty is not a door**: the bind is `ro` at the mount level, so no HTTP
or WebDAV write reaches the array regardless of account flags ([[Copyparty]]).
- **No SMB or NFS server exists here** — the `*arr` stack on FrankBook-Pro
cannot reach the array at all, so it cannot rename, import, or rescan it.
+157 -24
View File
@@ -18,7 +18,7 @@ clients.
public vhost `jelly.franksdank.cloud` behind the [[Anubis]] guard chain →
[[Authentik]] outpost (post-auth route proxies `localhost:8096`).
## Facts (the repo quadlet; the live unit is still the pre-view generation — see Data layout)
## Facts (the repo quadlet — deployed as the live unit on 2026-10-31)
| Item | Value |
|------|-------|
@@ -26,7 +26,7 @@ public vhost `jelly.franksdank.cloud` behind the [[Anubis]] guard chain →
| Image | `ghcr.io/jellyfin/jellyfin:latest` |
| Network | `dankserver_appnet`; publishes 8096/tcp, 8920/tcp, 7359/udp, 1900/udp |
| Data roots | `/home/dank/Containers/jellyfin/config:/config`, `/home/dank/Containers/jellyfin/cache:/cache` |
| Media | `Volume=/var/lib/media-meta/view:/data` in git — the [[Media Metadata View]] overlay, **writable**, sidecars land on the NVMe and the [[Entropy Volume]] array underneath stays untouched. **The live unit is still pre-view: `/mnt/entropy:/data:ro`** — nothing is written into the library until the view is deployed |
| Media | `Volume=/var/lib/media-meta/view:/data` — the [[Media Metadata View]] overlay, **writable**, live since the 2026-10-31 cutover: sidecars land on the NVMe and the [[Entropy Volume]] array underneath stays untouched |
| GPU | none — no `AddDevice`; CPU decode only (jellyfin-ffmpeg path) |
| Env quirks | `DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true`, `LD_PRELOAD=libjemalloc.so.2`, `MALLOC_TRIM_THRESHOLD_=131072` kept verbatim from the compose-era container (zero-drift cutover) |
| Health | `curl -f http://localhost:8096/health` every 30 s |
@@ -37,26 +37,124 @@ public vhost `jelly.franksdank.cloud` behind the [[Anubis]] guard chain →
/home/dank/Containers/jellyfin/config -> /config (JELLYFIN_DATA_DIR)
/home/dank/Containers/jellyfin/cache -> /cache (JELLYFIN_CACHE_DIR, XDG_CACHE_HOME)
/var/lib/media-meta/view -> /data (git: the overlay view)
/mnt/entropy -> /data (live today: the array, read-only)
/mnt/entropy -> /data (the overlay's lower layer; also the rollback bind)
```
Library folders point at `/data` paths: stable across container recreation and
identical to the raw array layout below. Which side of that path the container
is on is the open question on this host — the installed unit binds the read-only
array, and `scripts/media-view.sh status` reports `NOT MOUNTED` (2026-10-31).
sees is the [[Media Metadata View]] overlay, live since the 2026-10-31 cutover:
`/var/lib/media-meta/view:/data`, read-write.
`Local metadata` — save metadata to media folders — is enabled, and it is the
reason the view exists: `movie.nfo`, `tvshow.nfo`, `episode.nfo` and artwork
written once on the NVMe upper layer are files [[Plex]] can read too. While
`/data` is bound read-only every item update logs the error below and no sidecar
is written; Jellyfin becomes the fleet's only metadata writer with the deploy
that raises the view.
reason the view exists: `movie.nfo`, `tvshow.nfo` and `episode.nfo` written once
on the NVMe upper layer are files [[Plex]] can read too — and since the
2026-10-31 cutover that is what happens: Jellyfin is the fleet's only metadata
writer, and every sidecar it saves is an SSD byte the array never sees.
**Artwork is not one of those sidecars.** No image saver is bound for any
library, so posters and fanart go to the internal store
(`config/metadata/library/<xx>/<item-id>/poster.jpg`), which [[Plex]] never
reads — movie art has never existed beside the media on either layer.
**Library coverage is the first thing to check when a title is missing.** The
four libraries are `movies → /data/Movies`, `tvshows → /data/TVShows`,
`music → /data/Music`, `boxsets → /config/data/collections`
(`GET /Library/VirtualFolders`). `/data/media-park` is in **none** of them, so
everything filed under it — the `_unsorted` release bundles and the `extras/`
folders the relayout wrote there — is invisible to Jellyfin by configuration,
not by fault: a recursive refresh answers 204, burns CPU for minutes and
indexes nothing, because it never looks there. Check coverage before re-scanning.
**Failure signature:** `System.IO.IOException: Read-only file system :
'/data/Movies/<film>/movie.nfo'` means the container is binding the raw array
instead of the view. On Dankserver that is today's expected state — the view has
never been installed (`scripts/media-view.sh status`). Once it is deployed, the
same line is a fault: the quadlet drifted or the mount died.
instead of the view — always a fault on this host since the 2026-10-31 cutover.
`scripts/media-view.sh status` says which: the quadlet drifted back to the raw
array, or the mount died.
**`EnableInternetProviders` false turns a library into a silent file index.**
Every refresh answers `204`, stores nothing and logs nothing — no provider ids,
no plot, no rating, no art — while the item count reads like a healthy scan. The
switch lives in `config/root/default/<Library>/options.xml`, Jellyfin 12's name
for the per-library option file: grepping for the old `library.xml` concludes "no
per-library options exist", and the database holds none, so the trail dies there.
Two more traps in the same file: `TypeOptions/ImageFetchers` is the *enabled* set,
so an empty `<ImageFetchers />` switches art off while `ImageFetcherOrder` stays
fully populated — order is not enablement — and the populated `<MetadataFetchers>`
built next to it makes the library look healthy.
**Live posture, measured 2026-11-02** from `GET /Library/VirtualFolders`. The DTO is
PascalCase — `Name`, `CollectionType`, `Locations`, `ItemId`, `RefreshStatus` — and a
census written against the 10.x camelCase keys reads `None` for everything, which
looks exactly like an unconfigured library.
| Library | `CollectionType` | `EnableInternetProviders` | `EnableRealtimeMonitor` |
|---|---|---|---|
| `Movies` | movies | true (fixed 2026-11-02) | **false** |
| `Shows` | tvshows | true (fixed 2026-11-02) | true |
| `Music` | music | **false** | true |
| `Collections` | boxsets | **false** | false |
`Music` and `Collections` are silent file indexes in the sense above: a refresh answers
`204`, stores nothing, logs nothing. Any metadata or artwork work on either one
starts by flipping the switch, never by firing a scan. `Shows` sat in that set
until 2026-11-02; the door is `POST /Library/VirtualFolders/LibraryOptions` with
the library's own `ItemId` (below), and `scripts/jellyfin-providers.sh verify` is
the one command that says which libraries are still in it.
**The per-library option door on 12.x is `POST /Library/VirtualFolders/LibraryOptions`**
— `GET` there answers `405`, which is how the route announces itself. The 10.x doors
are gone: `/Library/VirtualFolders/Update` and `/Library/VirtualFolders/Refresh`
answer `404` on this build, so every recipe naming them fails outright.
**A library record can be a phantom.** A per-library option capture taken 2026-11-01
carried seven entries — `Shows2`, `Shows3`, `Music2` among them, each with
`Locations: []` and a live-looking `ItemId` — while `GET /Library/VirtualFolders` the
same day listed four. A numeric suffix on a library name is what a deleted and
recreated library leaves behind. The live set is the entries whose `Locations` are
non-empty; count those and never the records.
**`RemoteImages` is the call that separates a provider fault from a library
fault.** `GET /Items/{id}/RemoteImages?Type=Primary` returns the provider's real
poster list even with every automatic pass switched off — plugin, key and CDN all
proven by that one response — so a refresh that stores nothing beside it is a
library-switch problem, never a provider problem. It is also the delivery door:
`POST /Items/{id}/RemoteImages/Download` saves the chosen image to the internal
store on a build where the automatic image pass will not run at all — it is how
the Movies library got its art: 2,515 of 2,564 films carry a primary now, and the
rest have none in TMDb (shorts like `Astartes`, files whose names match nothing).
**The Download door's exact wire shape:** the list response carries the address in
`Url` — there is no `ImageUrl` field — and the POST takes camelCase query params,
`?type=Primary&imageUrl=<Url>`, answering `204`; the stored tag then appears in
`ImageTags`. PascalCase params, or an `imageUrl` left empty, answer `500`, which
reads as a server fault and is a field-name fault.
**`RemoteImages` answers empty for an item with no `ProviderIds`** — no provider id,
nothing to look up. Art coverage is capped by identification, not by the image door:
the items a campaign returns nothing for are the unidentified ones, and no image
retry will change that.
**A `Download` campaign over a library that is already dressed does nothing in
seconds, and that is the measurement proving identification is the only lever
left.** Run over all 753 series and 2,805 seasons of `Shows` on 2026-11-02 it
classified 3,558 items in 20 s: 2,908 already carried a primary, 624 have no
provider id at all, 26 have an id and no poster in TMDb — zero downloads. The
artless remainder is not an image-door problem: it is the unnamed long tail (clip
collections, `(Warhammer TV)`-shaped directories), and the lever is folder names
or an identify pass, never another retry loop.
**Series and season art is fetched; episode stills are not, deliberately.** Every
episode query is a fresh provider round-trip — nothing caches per episode — measured
at ~11 s per item beside ~2 s per series on the same host, with one hit in five in a
sample. Across 42k episodes that is days of provider traffic to dress a fifth of them,
and clients display the series and season art anyway. If episode stills are ever
ordered, the loop is the same door and its log makes the pass resumable.
**API shape on 12.x:** `/Items` and `/Items/{id}` need `UserId` when
authenticated by API key — without it the answer is `400 Error processing
request`, which reads exactly like an authentication failure. `MetadataRefreshMode`
accepts `Default` (and `Realtime`) only; `Full` and `ValidationFailure` are
rejected outright, and what forces the providers to re-run is
`ReplaceAllMetadata=true`.
## Ports & access
@@ -164,9 +262,17 @@ scripts/jellyfin-providers.sh verify # plugin + key LENGTHS, never key bytes
- **The homelab TVDB key is a licensed key** (its JWT claims say
`community_supported: false`), so it needs **no PIN** — the optional
`TVDB_PIN=` line exists only for a subscriber-shaped key.
- **TVDB is installed and keyed but is not a metadata agent**: TMDb still
identifies everything. Enabling TVDB as a provider would re-fetch show
metadata across the library, so it is a deliberate, separate change.
- **TVDB is installed but is not a metadata agent**: TMDb still identifies
everything. Enabling TVDB as a provider would re-fetch show metadata across
the library, so it is a deliberate, separate change.
- **A plugin owns its own config file, under its own root, and omits null
settings** — so the element an installer wants to fill may simply not exist,
and its root is `<PluginConfiguration>`, not `<Configuration>`. TVDB's file had
no `<ApiKey>` element at all, which is why `install` refused with
`no place to write <ApiKey>` from October until the anchor fix: the secret sat
valid in `provider-keys` the whole time, and `verify`'s `MISSING key` was this
fault, not a missing key. The writer now anchors on the file's own root and
still refuses by name when a file has none.
- **The deployer places the keys before it restarts jellyfin**
(`ensure_provider_keys` in `scripts/deploy-remote.sh`) and warns instead of
failing when `provider-keys` is absent, so a host without it still deploys.
@@ -212,17 +318,44 @@ receives `Authorization: MediaBrowser Token=` with nothing after it (`\\"` answe
One accepted refresh is the whole procedure. A second one while a scan runs moves
`Scan Media Library` to `Cancelling`, and once the runner holds `Cancelling` every
later `204` is accepted and then discarded — the status code stops meaning
anything. The fix is `systemctl --user restart jellyfin.service` (never
`deploy.sh`: that runs the never-tested metadata-view cutover), one POST after
anything. The fix is `systemctl --user restart jellyfin.service`, one POST after
health answers, and then believe `ScheduledTasks` State at 130 % CPU, not the
response code.
**A scan that must see a new folder has to be recursive.** `POST /Library/Refresh`
is the incremental pass, and inotify does not cross a rootless podman bind, so
folders created on the host stay invisible to it. The move is
`POST /Items/{id}/Refresh?Recursive=true` (204) per library, the id from
`GET /Library/VirtualFolders` — its field is `ItemId`, not `Id` — and that same
record's `RefreshStatus` is the progress read.
**The door that scans is `POST /Library/Refresh` (204), and `ScheduledTasks` is
the progress read.** A task that has never completed answers
`LastTaskResult=None, EndTimeUtc=None`, and that pair is what separates "never
ran" from "finished a moment ago" — the difference between a door that is dead
and one that merely answered `204`.
**The per-library `POST /Items/{id}/Refresh` queues nothing, and the provider
switch is not the reason.** Measured twice on 2026-11-02 against `Shows`: before
the flip and again after it, the call answered `204`, the journal wrote zero lines
over the following ten minutes, and `Scan Media Library` kept its never-run
signature; a service restart changes nothing. `POST /Library/Refresh` does run —
`204`, then `Running` inside 15 s at 33 % CPU — but it walks every library, so on
this build a library-scoped metadata pass has no working door and the global one
is the only lever.
**That endpoint takes its parameters from one source.** The documented query form
(empty body) and the JSON body form (no query) each parse; supplying both at once
answers `400` with an empty body whatever the values, and `ValidationFailure` is
rejected in the body while the query form accepts it.
**`RefreshStatus` on the `GET /Library/VirtualFolders` record is not a progress
read.** It stayed `Idle` through a scan that was verifiably running at 75 %. The
record's useful field is `ItemId`, and the live library set is still the entries
whose `Locations` are non-empty.
`inotify` does not cross a rootless podman bind, so folders created on the host
stay invisible until a pass like this runs; every pass that writes into a library
ends by firing one.
**`Movies` has no watcher at all.** `EnableRealtimeMonitor` is per library (table in
*Data layout*) and it is false for `Movies` and `Collections`. Nothing notices a
folder created under `/data/Movies`: it stays invisible until a recursive refresh
names it. So every pass that writes into a library ends by firing one, and "Jellyfin
does not see the file I just moved in" is the designed answer, not a fault to chase.
## See Also
@@ -11,7 +11,8 @@ timestamp: 2026-10-31T00:00:00Z
`/mnt/entropy/Audiobooks` is the one library here that [[Jellyfin]] does not read
at all — its music scanner has no audiobook concept — while [[Plex]] indexes all
of it as Artist → Album → Track: the album is the BOOK, that book's single artist
is the NARRATOR, both matched out of the files' own tags. Today one book is split
is the NARRATOR, both matched out of the files' own tags. [[Audiobookshelf]]
streams the same tree, read-only, and owns nothing in it. Today one book is split
across `… [Part 1]`, `… [Part 2]` folders three or four deep inside a collection
pack, so a scan serves three quarters of a novel; `scripts/media-audiobooks.py`
re-files the tree so the property both servers' book scanners assume is true —
@@ -36,6 +37,9 @@ python3 scripts/media-audiobooks.py --root /mnt/entropy/Audiobooks \
# as [[Media Relayout]] shows, on its own journal, and refresh the Plex section
# as in step 5 of the movie Procedure — from inside the container, never over
# the host port.
# Audiobookshelf stores absolute paths, so the same refresh has to reach its
# library too ([[Audiobookshelf]]) — otherwise it keeps rows for folders that
# no longer exist.
scripts/media-apply.py apply --plan /tmp/ab_plan.tsv --root /mnt/entropy \
--journal /mnt/entropy/media-relayout-audiobooks.jsonl --apply
```
@@ -0,0 +1,254 @@
---
type: reference
title: Media Duplicate Prune
description: "Find and hold duplicate films and series on the array: the screen, per-episode rule, guards, purge."
tags: [dankserver, media-server, storage, procedure]
timestamp: 2026-11-02T01:00:00Z
---
# Media Duplicate Prune
The movie library holds the same film twice in two different ways — the same file
downloaded again, or the same film grabbed a second time at another quality — and
[[Entropy Volume]] is permanently full, so both are worth finding. They need
different rules, and neither is decided by name similarity alone.
A series duplicates differently again: one show spread over several folders, so
neither the file nor the folder is the unit — see *A series is measured per
episode*.
`/mnt/entropy/media-park` was a third kind: a parking lot that held whole shows
and films nothing served. It is gone; its contents went to `NeedsProcessing`.
## What counts
| Class | Test | Action |
|---|---|---|
| Byte-identical | equal size, equal content hash | keep one copy, hold the rest |
| Same work, worse encode | same title, same year, every runtime within 5 % | keep the largest — Frank's rule |
| Different cut | same title and year, runtimes more than 5 % apart | **never touched**: an extended and a theatrical version are different products |
| Same words, different film | title matches, year or duration does not | never grouped — this is the false-merge class that would destroy a unique file |
| Series split across folders | same show by tvdb id, then the episode `(S,E)` — never the folder name | keep the largest carrier of each episode (`scripts/media-tvdupes.py`) |
## The screen
The array cannot afford a full-content read, so identity is established in steps
and each step only narrows what the next one touches:
1. `scripts/media-census.py enumerate` — size is free, and identical files share it.
2. Head+tail `blake2b`, 4 MiB at each end, on the size collisions only.
3. **Before anything moves**, confirm with a sparse-window hash: 32 MiB at eight
evenly spaced offsets. Re-hashing a remux-heavy set in full read 150 GiB for
the better part of an hour; the sparse window cost 50 s, a file that merely
shares its head and tail cannot survive it, and the destination is a
recoverable hold rather than a deletion.
4. Durations come from `scripts/media-census.py probe` — the relayout's own
census. Do not hand-roll a second probe: it re-seeks the array for an answer
the repo tool already produces, and a probe is the one census step that costs
hours instead of seconds.
## Which copy survives
In order: not one inside an `extras`, `featurettes`, `samples` or `trailers`
folder (an extra is not a copy of the film, and extra names collide across films);
then the shallowest path; then the file whose name equals its `Title (year)`
folder; then the largest. For the same-work class the largest is the answer, and
the canonical-name rule only breaks a tie — without it a `Film - 1080p.mkv` can
outlive `Film.mkv` for no reason but a space sorting before a dot.
## A series is measured per episode
`scripts/media-tvdupes.py` censuses a TV library and writes a staging plan in the
executor's own format; it never moves anything itself.
```bash
python3 scripts/media-tvdupes.py --library /mnt/entropy/TVShows \
--hold /mnt/entropy/.AGENTTRASH/<date>-tv-duplicates \
--meta /var/lib/media-meta/view/TVShows --witness witness.tsv \
--ids /mnt/entropy/tv-tvdb-ids.tsv \
--out /mnt/entropy/tv-dupes-plan.tsv --json /tmp/tv-groups.json
# then media-apply.py apply / verify / sweep / rollback, exactly as for a film prune
```
- **Identity before geometry.** A folder's tvdb id is read from its `[tvdbid-N]`
name tag, from the `tvshow.nfo` the [[Media Metadata View]] exposes (`--meta`),
and from a `folder<TAB>id` map (`--ids`). Build that map from Jellyfin's own
match, not a lookup: `BaseItems` (`Type like '%TV.Series'`) joined to
`BaseItemProviders` where `ProviderId = 'Tvdb'`, the `/data/TVShows/` prefix
stripped to the bare top-level folder name — 616 folders, none carrying two
ids, written to `/mnt/entropy/tv-tvdb-ids.tsv`. Two folders with disjoint ids
are two shows however alike they read — *Avatar: The Last Airbender* (1999)
beside the 2024 remake, *Millennium* (1996) beside a German thriller Jellyfin
had also titled *Millennium* — and a pair whose identity cannot be proven is
reported `UNRESOLVED` and left alone. The fix is the tag, never a guess.
- **A tag outranks every derived id.** The `--ids` map and the `tvshow.nfo` layer
fill an untagged folder and are **dropped** when they contradict its own
`[tvdbid-N]`, which the report prints as `DISAGREE`. Measured the same day: the
provider table carries a 2026 special's id on `Malcolm in the Middle (2000)
[tvdbid-73838]`; unioned, those folders are one show and the special's bigger
files dominate the original's S01E01–E04 — 3.2 GiB of the 2000 series bound for
the hold as "worse copies". The pass that staged 54 GiB ran with no `--ids`, so no
staged file rests on a borrowed id.
- **A folder-level verdict destroys content.** Of 25 structural candidates in the
2026-11-02 census, **10 were complementary halves**: *Game of Thrones: The
Complete Series* holds no S01 and no S08, `Modern Family S05` is the only copy of
season 5, *Loiter Squad Uncensored* the only copy of season 1. Judged per folder
the pass would have staged about 340 GB of unique episodes; judged per episode it
staged 54 GiB of true copies.
- **A file is droppable only when every episode it carries is held elsewhere in the
same show at >= its size** (Frank's rule, applied per episode). A byte-identical
pair ties and the tie breaks by path so exactly one copy stages; a file holding
one unique episode stays whatever else it carries.
- **A bare number is never an episode.** `256383.mp4` may be an episode id, a
poster or a whole season, so key-less names are never staged and are counted in
the report (4,036 after the 2026-11-02 pass, almost all genuine bonus material).
Where a packager named files by id, Jellyfin's own table is the witness: a
`path<TAB>season<TAB>episode` TSV from `BaseItems` where `Type like '%TV.Episode'`
and both index numbers are set, read `mode=ro&immutable=1`, its `/data` prefix
rewritten to the raw array path. Jellyfin 12 keeps `Path`, `ParentIndexNumber` and
`IndexNumber` on `BaseItems` and stores the full CLR type name — confirm with
`pragma table_info` before trusting the query, because an empty witness silently
falls back to name parsing.
Seven clips in the library are still named only by number, and where Jellyfin
was asked its own `Name` for them is the number too, so no database holds a
better title — that residue is naming, not a task.
- **A pack states a range and competes on all of it.** `S01E01-02` is droppable only
when both halves are held elsewhere, and it wins the survivor seat for both when
it is the largest carrier: the byte law compares files, and a name both servers
parse as a range stays indexed as two episodes.
- **A sidecar that shares the doomed copy's stem goes with it** — subtitles, `.nfo`,
artwork keyed on the stem — so a staged duplicate leaves no orphan beside the
survivor. The 2026-11-02 hold is 220 media files and nothing else: the dominated
copies were bare rips, and counting a hold's non-media files is the census that
tells you which case you are in.
- **The audit gates the plan file.** Every survivor is re-statted and every
destination re-checked before `--out` is written; on failure the report prints
`AUDIT FAILED` and writes no plan, so there is nothing to hand the executor.
- **Two folders that look duplicated and share zero episode keys are complementary,
not duplicated.** That is how the two pairs the census still refuses to merge were
settled without touching them: *Star Wars - The Clone Wars* (12 files, 1.3 GiB)
beside *Star Wars The Clone Wars* (121 files, 214 GiB), and the *Batman: The
Animated Series* leftovers (3 files) beside the tagged folder (85) — `shared
episode keys: 0` in both cases, so a tag would have freed nothing and cost a
rename with library consequences. Re-running the whole census with the id map over
all 753 top-level folders (606 already tagged) staged **0 files**: the library is
duplicate-free to the limit of provable identity, and what remains is naming.
## Guards
Each of these is a file that would have been destroyed without it:
- **`.!qB` is qBittorrent state, not a duplicate.** It can be byte-identical to
the finished file while qB still counts the download incomplete; moving it
breaks the re-check and re-downloads the film. Report it, leave it. Both
leftovers in `Movies` are 13 months stale, so they are Frank's purge decision,
not a prune decision.
- **Require a year, and keep digits in the title.** Stripping digits merged
*Akira — Teaser 1* with *Teaser 2*; without a year, two different extras that
happened to share a name merged.
- Both members must be feature length, and a name carrying `disc`, `cd` or `part`
is excluded: half a multi-disc release is not a copy of the other half.
- Media containers only. Grouping every extension buries the report in subtitle
and cover pairs — and a duration guard that appears to fire on 720 groups is
counting subtitles, not protecting films.
## The hold
Nothing is deleted. Every doomed copy moves into a dated hold with the same
executor the relayout uses, so any file comes back:
```bash
scripts/media-apply.py apply --plan /mnt/entropy/dup-prune-similar-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/dup-prune-similar.jsonl # dry
scripts/media-apply.py apply --plan … --root /mnt/entropy --journal … --apply
scripts/media-apply.py verify --plan … --journal /mnt/entropy/dup-prune-similar.jsonl
scripts/media-apply.py sweep --journal /mnt/entropy/dup-prune-similar.jsonl --root /mnt/entropy --apply
scripts/media-apply.py rollback --journal /mnt/entropy/dup-prune-similar.jsonl --root /mnt/entropy --apply
```
The plan rows are the tool's own format (`action src dst kind`, `mkdir` rows for
every hold parent), and the planner re-checks the two things that go stale between
analysis and apply: a file that has moved since the census, and a group whose
keeper is missing — either one drops the whole group rather than half-applying it.
- **A hold on the same volume frees nothing.** The purge is a separate, approved
act and the only thing that moves the free-space number; `df` does not budge
when 1 TiB moves sideways.
- `sweep` removes the release folders the pass emptied. Empty folders dated a year
ago are torrent detritus (`Subtitle,info`, `Screens`, `Sample`) from a previous
era, not this pass — read the mtimes before blaming the prune.
- The 2026-11-01 pass held 226 files (~1.1 TiB) in
`/mnt/entropy/.AGENTTRASH/20261101-duplicate-prune`, journals
`dup-prune-exact.jsonl` and `dup-prune-similar.jsonl`. Purge pending Frank's word.
- The 2026-11-02 TV pass holds 220 files (54 GiB) in
`/mnt/entropy/.AGENTTRASH/20261102-tv-duplicates` — plan `tv-dupes-plan.tsv`,
journal `tv-dupes.jsonl`, 220 moves / 16 hold folders / 7 husks swept — and the
same day's TVShows detritus pass holds 109 files (552 KiB) in
`/mnt/entropy/.AGENTTRASH/20261102-tvshows-detritus` with plan
`tvshows-detritus-plan.tsv` and journal `tvshows-detritus.jsonl`. Both purges
pending Frank's word, beside the 2026-11-01 hold above. All four journals and
plans live at the `entropy` root, not `/tmp`.
- **Re-verify each survivor by size, not by the recorded path, before purging.**
Detection artifacts and plans key on paths, and a folder renamed by an
interactive session ([[Entropy Volume]], *Write doors*) makes its keeper look
deleted while it sits untouched under the new name. The 2026-11-01 audit hit
three such groups: every keeper was present, same bytes, different folder. A
purge decided from the plan's paths alone would have reported survivors gone.
## After the pass
Both servers still hold rows for files that are now in the hold, and neither
notices: the media bind is the [[Media Metadata View]], and array changes do not
cross it as inotify events. Scan in the order the rescan law sets — Jellyfin to
`Idle`, then Plex's Movies section alone, one accepted refresh each; the requests
are owned by [[Jellyfin]] and [[Plex]].
The Plex leg may be deferred on purpose: stop **and mask** Plex (`systemctl --user
stop plex && systemctl --user mask plex`) so its metadata fetches cannot stack on a
live Jellyfin scan, and because a masked unit stays stopped through a reboot —
`unmask` before starting it. The 2026-11-01 morning ran it once Jellyfin reached
`Idle`: one accepted `refresh` on the Movies section cleared every row that named a
held path. The 2026-11-02 TV pass did the same on the TV Shows section (key 3): one
accepted refresh, then `media-verify.py` against that journal reads `landed: 0`,
`still at src: 0`, `REAL: 0` on **both** servers — the correct end state for a prune.
The same evening's detritus pass is a second, separate hold: 423 files (12 MiB) in
`/mnt/entropy/.AGENTTRASH/20261101-detritus-sweep`, plan `detritus-sweep-plan.tsv`,
journal `detritus-sweep.jsonl`, 542 husks removed by `sweep`. It moved files only
from folders whose whole subtree held no media, so it creates no stale rows and
needs no scan. The guards that hold a folder back are the finding: external
subtitle folders (428 of them, 6 GiB) and ripped Blu-ray streams — `.sup` PGS,
`.thd`/`.eac3`/`.dts` audio, `.cci` captions, 25 GiB — are not trash, and an
extension the census does not recognise is a reason to stop, not to sweep.
A folder holding a `.plexmatch` is held back even when its subtree looks empty: that
file is a manual match — a human decision about what the folder is — and detritus
rules do not overrule one.
`scripts/media-verify.py` reads differently for a prune than for a relayout:
`landed: 0` is the **correct** end state (no server may index the hold), so the
verdict is `still at src: 0` with `REAL: 0`, and the rows that simply vanished
from the database are the pass succeeding.
One residue is normal, so do not chase it: a section pass rewrites the feature rows
first, and rows for clips in a film's `other/` or `extras/` subfolder can still name
a held path after the section's `refreshing` flag is back to `0` — and a second
section pass does not clear them either, so do not run one hoping for `0`. Beside
`REAL: 0` that is a dead disc-menu clip — not a false prune, and no reason to touch
the cache, the database, or the hold.
## Failure signatures
| Symptom | Meaning |
|---|---|
| A size group whose members split under the sparse hash | The head+tail screen collided. Those files stay exactly where they are and get reported. |
| The planner holds far fewer files than the analyzer counted | Two implementations of "largest wins". The analyzer must emit `keep`/`hold` marks and the planner only transcribes them — a planner that re-derives the winner from the hold list alone drops the keeper out of the group and prunes almost nothing. |
| `df` unchanged after a TiB of moves | Expected: the hold is on the same filesystem. Only the purge frees space. |
| A film that "disappeared" after a prune | It is in the hold, named by the journal: `media-apply.py rollback --journal … --apply`. |
## Links
- [[Media Relayout]] — the census tools, the executor, the journal format.
- [[Media Metadata View]] — why neither server sees the change without a scan.
- [[Jellyfin]] · [[Plex]] — the scan requests, one server at a time.
- [[Entropy Volume]] — the array that is always full.
@@ -3,7 +3,7 @@ type: reference
title: Media Metadata View
description: "The SSD-backed overlay the two media servers read and write: shape, guards, commands, laws, rollback."
tags: [dankserver, media-server, jellyfin, plex, storage]
timestamp: 2026-10-30T00:00:00Z
timestamp: 2026-11-02T01:00:00Z
---
# Media Metadata View
@@ -20,18 +20,40 @@ The mount point is not a volume and holds nothing while unmounted. It sits insid
merged door are one directory — and so the mount table grows no new `/mnt` entry
next to the array.
**Not deployed on Dankserver — verified 2026-10-31.** `scripts/media-view.sh
status` answers `view: /var/lib/media-meta/view NOT MOUNTED`,
`/var/lib/media-meta` does not exist, and the installed quadlets under
`~/.config/containers/systemd/` still carry the pre-view binds — `/mnt/entropy:/data:ro`
for [[Jellyfin]], `/mnt/entropy:/mnt/entropy:ro` for [[Plex]]. Everything below
is the shape git describes, not the shape the servers run. The cutover is an
ordinary deploy: `deploy.sh dankserver --service jellyfin|plex` runs
`ensure_media_view` (`install` then `verify`) before it restarts the container,
so it must not be fired through the middle of a library scan. Until that deploy
lands there is no upper layer, so no relayout can leave ghost metadata and
`reap` has nothing to do — `media-apply.py` prints that follow-up only when the
view really is a mountpoint.
**Deployed on Dankserver — 2026-10-31, both servers.** `plex` binds
`/var/lib/media-meta/view:/mnt/entropy:ro` and `jellyfin` binds
`/var/lib/media-meta/view:/data:rw`; `findmnt` reports the overlay with
`lowerdir=/mnt/entropy upperdir=/var/lib/media-meta/upper`, and each container
was proven to resolve a real media file through it — `podman exec <svc> ls -la
<its data path>` is the whole proof, because both servers keep the paths their
databases recorded. Two service-scoped deploys, each gated by
`media-view.sh verify` and the Playwright suite.
Re-verified 2026-11-01: `mount` shows the overlay on
`/var/lib/media-meta/view` (`lowerdir=/mnt/entropy`,
`upperdir=/var/lib/media-meta/upper`), and the installed quadlets under
`~/.config/containers/systemd/` carry the view binds:
`Volume=/var/lib/media-meta/view:/data` for [[Jellyfin]],
`Volume=/var/lib/media-meta/view:/mnt/entropy:ro` for [[Plex]]. Prove it in that
order: `scripts/media-view.sh verify` (the deploy gate), then
`grep -n Volume ~/.config/containers/systemd/{jellyfin,plex}.container`.
Moving a bind means restarting the server, and **a restart discards that
server's in-flight library scan** — Jellyfin's 7 h 25 m walk ended
`"Status": "Aborted"` in `data/ScheduledTasks/<task-id>.js` and nothing resumes
it. Renames made on the array while the view was unmounted are invisible to both
watchers too, so the additive index has to be asked for by hand; the door is an
API key, and the row in the failure table below says what to check when it
refuses.
The cutover itself is an ordinary deploy: `deploy.sh dankserver --service
jellyfin|plex` runs `ensure_media_view` (`install` then `verify`) before it
restarts the container, so it must not be fired through the middle of a library
scan. Two consequences are now live rather than theoretical: inotify events for
array writes do not cross the merge, so any relayout or prune needs a manual
library refresh; and a move can leave ghost sidecars above, so `reap` runs after
a relayout — a pass that moves only media files with no sidecars next to them
leaves the upper layer untouched, and `reap` correctly finds nothing.
## Shape
@@ -112,6 +134,19 @@ the defaults are the host values above.
upper layer*, not for changes on the array underneath. After any relayout or
externally-written metadata, trigger a library refresh — see
[[Media Relayout]] for the refresh procedure.
- **`stat` through the view can outlive the file.** Measured 2026-11-02 while a TV
dedupe pass moved folders on the raw path: the merged tree still answered `stat`
at the old coordinates — link count `0`, and `ls` of the same parent no longer
listed the name. Overlayfs keeps a dentry for a lower file that was moved
underneath it, so a script that tests existence with `[ -e ]` against the view can
believe a file its own `readdir` denies. Prove existence by listing the parent.
The media servers walk by `readdir`, so ghosts never reach the library — this bites
a tool, never a user, which is why no server symptom accompanies it.
- **Tools that move or measure media walk the raw array, never the view** — every
relayout, prune and detritus pass, and every journal it writes. The view is for
Jellyfin's writes and Plex's reads only. One deliberate exception: `tvshow.nfo`
sidecars live in the upper layer, so a tool reading show identity points at the
view (`media-tvdupes.py --meta /var/lib/media-meta/view/TVShows`).
## Failure signatures
@@ -122,7 +157,11 @@ the defaults are the host values above.
| Jellyfin log: `Read-only file system : '/data/Movies/…/movie.nfo'` | The container is binding the raw array, not the view — the quadlet drifted back or the view is gone. `media-view.sh status`. |
| A folder appears in a media server with artwork/`.nfo` but no video | Ghost metadata from a relayout. `media-view.sh reap`. |
| `verify` prints `refused: …/view is not a mountpoint` | A plain directory is sitting where the overlay belongs. That is the fail-closed guard working; `install` on the box. |
| `install` prints `refused: upper base /var/lib/media-meta does not exist (put it on the SSD first)` | A fresh host. `install` deliberately never creates the base — a mistyped `--upper-base` must not build a metadata tree on the wrong filesystem — so `scripts/media-view.sh ensure-base` makes the documented one, which the deployer calls first. |
| A media server's scan record reads `"Status": "Aborted"` | Its container restarted (a cutover, an image bump, an OOM) and the walk was discarded; systemd's `Restart=always` restarts the *server*, never the scan. Trigger a refresh — see [[Media Relayout]]. |
| The Jellyfin API answers 401 to every token header | Almost always the header **form**, not the key: 12.4 authenticates only `Authorization: MediaBrowser Token="…"` and refuses a bare `X-Emby-Token` for every key, valid or not — and one collapsed backslash in the curl config produces the same 401. `[[Jellyfin]]` owns the block that works. Before concluding anything, prove the key against the server's own table: `select count(*) from ApiKeys where AccessToken = ?` on `jellyfin.db` opened `mode=ro&immutable=1`. |
| `systemctl status` finds no such unit | Look under the escaped name, `var-lib-media\x2dmeta-view.mount`, not the repo file name. |
| Stale dentry through the view: `stat` succeeds with link count `0` while the parent's `ls` omits the name | The lower file was moved on the raw path after the view cached its dentry. Nothing is lost; existence through the view is proved by the parent listing. |
## Rollback
@@ -3,7 +3,7 @@ type: reference
title: Media Relayout
description: "Census, plan and apply the movie, TV, music and audiobook libraries: the gates, the refusals, the rollback."
tags: [dankserver, media-server, jellyfin, plex, procedure]
timestamp: 2026-10-30T00:00:00Z
timestamp: 2026-11-01T01:00:00Z
---
# Media Relayout
@@ -138,6 +138,8 @@ HTTPD close without a single byte; and the same HTTPD drops requests that carry
no `Accept` and client-identifier header (see the scan block above for the form
that works).
A duplicate prune is the same census and the same proof, with its own keep rules — films by title and duration, a series per episode: [[Media Duplicate Prune]].
## TV in the movie library
A box set that arrives in the movie share stays there until someone moves it:
@@ -401,9 +403,12 @@ executor, the one-journal-per-arm law, the sweep and the rollback.
- **The journal lives BESIDE the library root, never inside it** (default:
`media-relayout.jsonl` next to the root): a log inside the library is a file
Jellyfin indexes.
- **Refusals are all-or-nothing.** An occupied target, a vanished source or a
duplicate destination stops the run *before the first rename*, so the operator
fixes one thing and re-runs instead of untangling a half-applied plan.
- **Refusals are all-or-nothing.** An occupied target, a vanished source, a
duplicate destination, or a plan row whose verb is not `mkdir`, `keep` or
`move`, stops the run *before the first rename*, so the operator fixes one
thing and re-runs instead of untangling a half-applied plan. An unread verb is
refused by line number: silently dropping one turned a pass meant to rename
two series folders into `applied: 0 moved` with exit 0 (2026-11-02).
`--force` relaxes only the opinion guards; it never permits overwriting.
- **A sweep deletes only what the journal proves the run emptied.** A folder is
eligible when a journalled file moved out of it or out of something under it,
+25 -7
View File
@@ -27,7 +27,7 @@ public traffic can be fronted by Caddy like every other vhost.
| Runtime | podman quadlet, `plex.service` (user manager) — migrated from the native `plexmediaserver.service` OS-package unit (2026-09) |
| Image | `lscr.io/linuxserver/plex:1.43.5.11029-6bf8af3f0-ls329` (pinned; verified == the registry's `latest` manifest digest at pin time: `sha256:8f655bad…e9e2334`) |
| Data root | `/var/lib/plexmediaserver` bind-mounted **in place** as `/config` (RW) — the migration moved zero bytes |
| Media | git: `/var/lib/media-meta/view` (the [[Media Metadata View]] overlay) bound at the same absolute path `/mnt/entropy` **read-only**; live today: `/mnt/entropy` itself, read-only — the view has never been installed. Either way every library path in the PMS database stays valid, zero re-scan |
| Media | `/var/lib/media-meta/view` (the [[Media Metadata View]] overlay) bound at the same absolute path `/mnt/entropy` **read-only** — live since the 2026-10-31 cutover, so the PMS database's recorded paths keep resolving with zero re-scan |
| Identity | fleet rootless model — container root maps to the `dank` user (1000:1000); the deployer chowns the data root on first cutover |
| Port | `32400/tcp` published on `0.0.0.0` — zero client reconfiguration |
| Network | `dankserver_appnet`, alias `plex`; Tautulli reaches it at `http://plex:32400` |
@@ -40,7 +40,7 @@ public traffic can be fronted by Caddy like every other vhost.
The migration moved zero bytes: the container bind-mounts the native data
root where it already lived, and the media archive re-mounts at
`/mnt/entropy` inside the container — only the HOST side of that bind has moved
(the array today; the overlay view in git has never been deployed), so every
(the array until the 2026-10-31 view cutover, the overlay since), so every
absolute library path in the
PMS database keeps resolving. The one metadata change: rootless podman
cannot map the real host uid 997, so the fleet model applies — container
@@ -86,7 +86,22 @@ root runs as the `dank` user, and the deployer adopts the data root with
4. **Cold container boot is slow.** PMS pre-binds 32400 and empty-replies
while the HTTP layer finishes loading; the first boot of a years-old
library took ~10 minutes (2026-09 cutover). The deployer gate now allows
600 s before failing; later boots are much faster (warm page cache).
600 s before failing; later boots are much faster (warm page cache). A
`systemctl --user start` issued after an `unmask` is a job in the manager,
not in the shell that issued it: it lands minutes later and survives that
client dying, so a wrapper script that looks hung is usually only waiting —
read `is-active` and `podman ps` before declaring a fault, and give every
step of such a script its own `timeout`.
5. **`32400` answers TLS only to remote clients.** PMS requires secure
connections, so `curl http://192.168.0.235:32400/identity` from any other
machine closes without a single byte (curl exit 52) — that reads as a dead
server beside a healthy one, and it is how a post-cutover health check gets
misread. Probe with `curl -k https://192.168.0.235:32400/identity`. Two
consequences: `plex.franksdank.cloud` works **only** because its `transport
http` block carries `tls_insecure_skip_verify` (that option makes Caddy speak
TLS to PMS's self-signed cert — delete it and the vhost 502s), and the scan
block below rides `http://127.0.0.1` **inside** the container, where insecure
connections are allowed.
## Triggering a scan
@@ -121,10 +136,13 @@ shred -u /tmp/px.tok
```
Every section answers `204` — or `200` with a Directory body when
`Accept: application/json` is sent. Inside the quoted `printf` format a
backslash-newline is LITERAL, not a continuation, so `mk` writes two `printf`s:
one glued line and one stray backslash is a config file curl rejects, and the
failure prints as `option -K: is unknown`, never as a bad config line.
`Accept: application/json` is sent. A refresh fired while PMS is still warming
answers `000` instead (the empty reply of gotcha 4, arriving mid-request): retry
until it answers and never restart the service over it. Inside the quoted
`printf` format a backslash-newline is LITERAL, not a continuation, so `mk`
writes two `printf`s: one glued line and one stray backslash is a config file
curl rejects, and the failure prints as `option -K: is unknown`, never as a bad
config line.
Proof that a rename landed is not a scan response: `scripts/media-verify.py`
replays the run's journal against the database — the only read that
@@ -107,6 +107,12 @@ job.sh watch # refreshing status table
`ssh 'bash -c "python3 -c \"…\""'` quoting broke repeatedly (unterminated
quotes; `grep -c` returning `"0\n0"` because `|| echo 0` appended a line)
every failure disappeared once the script was a file.
- **A script fed over stdin owns that stdin.** `ssh host 'bash -s' <<'EOS'` hands
the script itself to stdin, so any child that reads stdin — `podman exec -i`,
`ssh -t`, a bare `read` — swallows everything after it and the remainder
vanishes without a word. It reads exactly like a hang: the commands before the
child print, the ones after never existed. Drop `-i`, or give every child
`< /dev/null`.
## Verify before trusting it (3am law)
+2
View File
@@ -44,6 +44,7 @@ is a locator: one row, one clause, no numbers.
| Page | Owns |
|---|---|
| [[Audiobookshelf]] | Audiobookshelf on Dankserver: quadlet shape, the read-only array bind, state on the SSD, the API bootstrap, laws. |
| [[Calibre]] | Calibre is the e-book library manager that backs Calibre-Web. |
| [[Calibre-Web]] | Calibre-Web is a web application that browses, reads, and downloads eBooks from an existing Calibre library database. |
| [[Copyparty]] | Copyparty is a lightweight file server and media streamer that runs as a single Python binary. |
@@ -52,6 +53,7 @@ is a locator: one row, one clause, no numbers.
| [[Jellyfin]] | Jellyfin on Dankserver: quadlet unit, overlay metadata view, provider keys, ports, access, transcode status. |
| [[mariadb]] | The MariaDB backend for RomM: quadlet unit, network alias, env-file secret handling, and the backup path. |
| [[Media Audiobook Relayout]] | Re-filing /mnt/entropy/Audiobooks by Plex's chapter rows: narrator shelves, no renumbering, the refusals. |
| [[Media Duplicate Prune]] | Find and hold duplicate films and series on the array: the screen, per-episode rule, guards, purge. |
| [[Media Metadata View]] | The SSD-backed overlay the two media servers read and write: shape, guards, commands, laws, rollback. |
| [[Media Music Relayout]] | Re-filing /mnt/entropy/Music by Jellyfin's tag rows: the census, the album key, the refusals. |
| [[Media Relayout]] | Census, plan and apply the movie, TV, music and audiobook libraries: the gates, the refusals, the rollback. |
+1
View File
@@ -0,0 +1 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#3fc1c9" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#3fc1c9" text-anchor="middle">AB</text></svg>

After

Width:  |  Height:  |  Size: 293 B

+13
View File
@@ -511,6 +511,19 @@
"showLink": true,
"art": "/img/apps/mono-tt.svg"
},
{
"name": "Audiobookshelf",
"category": "admin",
"section": "Media Library Apps",
"description": "Audiobook server for the entropy library: its own SQLite library, its own login, per-listener positions.",
"url": "http://192.168.0.235:8000",
"docs": "",
"probe": "192.168.0.235:8000",
"method": "http",
"wip": false,
"showLink": true,
"art": "/img/apps/mono-ab.svg"
},
{
"name": "Postgres",
"category": "admin",
+200
View File
@@ -0,0 +1,200 @@
#!/usr/bin/env bats
# Seam: scripts/audiobookshelf-init.sh — the Audiobookshelf bootstrapper.
#
# A freshly deployed Audiobookshelf answers /healthcheck 200 while it has no
# user, no library and no scan, so "healthy" proves nothing about usefulness.
# This seam pins what the bootstrapper must make true, against a stub that
# speaks the same API as the pinned image (POST /init + POST /login at the
# root, bearer-authenticated /api/*, library count only on /items):
# 1. an unregistered server gains the account, the library and a scan, and the
# credential lands mode 600 with both keys;
# 2. a second run creates nothing new (exactly one /init, one library) and
# still fires the incremental scan;
# 3. neither the bearer token nor the password ever rides a URL — a secret in
# a URL or an argv is world-readable in /proc/<pid>/cmdline, which is how
# an HF token leaked into a transcript on 2026-09-16;
# 4. a server that cannot be reached is refused by URL with exit 1, never
# reported as success (a green deploy over a dead API is a lie);
# 5. ABS_SCAN outside 0/1 is a usage error raised before anything is touched;
# 6. the deployer runs the bootstrapper only after the health gate answered.
load helpers
setup() {
INIT="$REPO_ROOT/scripts/audiobookshelf-init.sh"
TMP="$(mktemp -d)"
PORT=18937
cat > "$TMP/stub.py" <<'PY'
import json, sys
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
port, logf = int(sys.argv[1]), sys.argv[2]
state = {"init": False, "user": None, "pw": None, "libs": []}
def log(line):
with open(logf, "a") as f:
f.write(line + "\n")
class H(BaseHTTPRequestHandler):
def log_message(self, *a):
pass
def send(self, code, body, as_json=False):
data = json.dumps(body).encode() if as_json else str(body).encode()
self.send_response(code)
self.send_header("Content-Type", "application/json" if as_json else "text/plain")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def body(self):
n = int(self.headers.get("Content-Length") or 0)
return json.loads(self.rfile.read(n) or b"{}")
def authed(self):
return self.headers.get("Authorization") == "Bearer stub-token"
def do_GET(self):
path = self.path.split("?")[0]
log("GET %s auth=%s" % (self.path, "yes" if self.headers.get("Authorization") else "no"))
if path == "/status":
return self.send(200, {"app": "audiobookshelf", "isInit": state["init"]}, True)
if not self.authed():
return self.send(401, "Unauthorized")
if path == "/api/libraries":
return self.send(200, {"libraries": state["libs"]}, True)
if path.endswith("/items"):
return self.send(200, {"total": 42, "results": []}, True)
return self.send(404, "nope")
def do_POST(self):
path = self.path.split("?")[0]
payload = self.body()
log("POST %s auth=%s" % (self.path, "yes" if self.headers.get("Authorization") else "no"))
if path == "/init":
if state["init"]:
return self.send(500, "already initialized")
root = payload.get("newRoot", {})
state["init"] = True
state["user"] = root.get("username")
state["pw"] = root.get("password")
log("INIT %s" % root.get("username"))
return self.send(200, "OK")
if path == "/login":
if not state["init"]:
return self.send(401, "Unauthorized")
if (payload.get("username"), payload.get("password")) != (state["user"], state["pw"]):
return self.send(401, "Unauthorized")
return self.send(200, {"user": {"username": state["user"], "type": "root",
"id": "u1", "token": "stub-token"}}, True)
if not self.authed():
return self.send(401, "Unauthorized")
if path == "/api/libraries":
lib = {"id": "lib-1", "name": payload.get("name"), "folders": payload.get("folders", [])}
state["libs"].append(lib)
log("LIBRARY %s %s" % (lib["name"], lib["folders"]))
return self.send(200, lib, True)
if path.endswith("/scan"):
log("SCAN")
return self.send(200, "OK")
return self.send(404, "nope")
ThreadingHTTPServer(("127.0.0.1", port), H).serve_forever()
PY
python3 "$TMP/stub.py" "$PORT" "$TMP/requests.log" > "$TMP/stub.out" 2>&1 &
STUB_PID=$!
for _ in $(seq 1 50); do
curl -s -o /dev/null "http://127.0.0.1:$PORT/status" && break
sleep 0.1
done
}
teardown() {
[ -n "${STUB_PID:-}" ] && kill "$STUB_PID" 2>/dev/null || true
rm -rf "$TMP"
}
boot() {
env ABS_URL="http://127.0.0.1:$PORT" ABS_SECRET="$TMP/cred" \
ABS_ADMIN_USER=frank ABS_LIBRARY=Audiobooks ABS_FOLDER=/audiobooks \
ABS_SCAN="${SCAN:-1}" bash "$INIT"
}
@test "an unregistered server gains the account, the library and the scan" {
run boot
[ "$status" -eq 0 ]
printf '%s\n' "${lines[@]}" | grep -qx "credential: created $TMP/cred"
printf '%s\n' "${lines[@]}" | grep -q "^account: created (frank)"
printf '%s\n' "${lines[@]}" | grep -q "^token: obtained (root)"
printf '%s\n' "${lines[@]}" | grep -q "^library: created Audiobooks (lib-1)"
printf '%s\n' "${lines[@]}" | grep -q "^scan: accepted"
printf '%s\n' "${lines[@]}" | grep -q "^items: 42"
# The state roots of this service hold an account credential, so the file
# that carries it must never be group- or world-readable.
[ "$(stat -c %a "$TMP/cred")" = "600" ]
grep -qx 'ABS_ADMIN_USER=frank' "$TMP/cred"
grep -Eq '^ABS_ADMIN_PASSWORD=[0-9a-f]{24}$' "$TMP/cred"
# The folder the library indexes is the bind inside the container, not the
# host path — a library pointed at /mnt/entropy would answer 0 items and
# look like an empty audiobook collection.
grep -q "LIBRARY Audiobooks \[{'fullPath': '/audiobooks'}\]" "$TMP/requests.log"
}
@test "a second run creates nothing new and still fires the incremental scan" {
boot > /dev/null
[ -e "$TMP/requests.log" ]
run boot
[ "$status" -eq 0 ]
printf '%s\n' "${lines[@]}" | grep -q "^account: already registered"
printf '%s\n' "${lines[@]}" | grep -q "^library: exists Audiobooks (lib-1)"
printf '%s\n' "${lines[@]}" | grep -q "^scan: accepted"
# No second credential, no second account, no duplicate library: the deployer
# may run this on every re-deploy of the service.
! printf '%s\n' "${lines[@]}" | grep -q "credential: created"
[ "$(grep -c '^INIT ' "$TMP/requests.log")" -eq 1 ]
[ "$(grep -c '^LIBRARY ' "$TMP/requests.log")" -eq 1 ]
[ "$(grep -c '^SCAN$' "$TMP/requests.log")" -eq 2 ]
}
@test "the bearer token and the password never ride a URL" {
boot
# The stub logs the request line (method + path). A secret showing up there
# means it travelled as a query parameter — visible in access logs, shell
# history and /proc/<pid>/cmdline.
! awk '{print $2}' "$TMP/requests.log" | grep -q 'stub-token'
pw=$(awk -F= '/^ABS_ADMIN_PASSWORD=/{print $2}' "$TMP/cred")
[ -n "$pw" ]
! awk '{print $2}' "$TMP/requests.log" | grep -q -- "$pw"
# and the authenticated routes only answered once the header was present
grep -q '^GET /api/libraries auth=yes$' "$TMP/requests.log"
}
@test "a server that does not answer is refused by URL, not reported as success" {
run env ABS_URL="http://127.0.0.1:1" ABS_SECRET="$TMP/cred" bash "$INIT"
[ "$status" -eq 1 ]
[[ "$output" == *refused:* ]]
[[ "$output" == *"127.0.0.1:1"* ]]
}
@test "ABS_SCAN outside 0/1 is a usage error raised before anything is touched" {
run env ABS_URL="http://127.0.0.1:$PORT" ABS_SECRET="$TMP/cred" ABS_SCAN=2 \
bash "$INIT"
[ "$status" -eq 2 ]
[[ "$output" == *ABS_SCAN* ]]
[ ! -e "$TMP/cred" ]
}
@test "the deployer bootstraps only after the health gate has answered" {
body=$(awk '/^deploy_audiobookshelf\(\)/,/^}/' "$REPO_ROOT/scripts/deploy-remote.sh")
[ -n "$body" ]
printf '%s\n' "$body" | grep -q \
'wait_for_health audiobookshelf http://127\.0\.0\.1:8000/healthcheck'
printf '%s\n' "$body" | grep -q 'scripts/audiobookshelf-init\.sh'
printf '%s\n' "$body" | awk '
/wait_for_health audiobookshelf/ { health = NR }
/audiobookshelf-init\.sh/ { init = NR }
END { exit !(health && init && init > health) }'
}
+60
View File
@@ -127,3 +127,63 @@ load helpers
grep -q "dankserver/host/systemd/$f" "$REPO_ROOT/scripts/deploy-remote.sh" || return 1
done
}
@test "audiobookshelf unit exists, is boot-persistent and self-healing" {
unit="$REPO_ROOT/dankserver/audiobookshelf/quadlet/audiobookshelf.container"
[ -f "$unit" ] || return 1
grep -q '^WantedBy=default.target$' "$unit" || return 1
grep -q '^Restart=always$' "$unit" || return 1
grep -q '^RestartSec=' "$unit" || return 1
}
@test "audiobookshelf keeps every byte it writes on the SSD" {
# The point of the unit: its library database, accounts, progress and cached
# covers are all server state, and server state belongs on the NVMe. A bind
# of /mnt/entropy that is not :ro would silently put writes on the spindle.
unit="$REPO_ROOT/dankserver/audiobookshelf/quadlet/audiobookshelf.container"
grep -q '^Volume=/home/dank/Containers/audiobookshelf/config:/config$' "$unit" || return 1
grep -q '^Volume=/home/dank/Containers/audiobookshelf/metadata:/metadata$' "$unit" || return 1
grep -q '^Volume=/mnt/entropy/Audiobooks:/audiobooks:ro$' "$unit" || return 1
# The overlay view is reserved for the two media servers that write sidecars
# (tests/media_overlay.bats pins that set); a reader must not join it.
! grep -q 'media-meta/view' "$unit" || return 1
}
@test "audiobookshelf refuses to start against an unmounted array" {
# Without the gate podman invents an empty bind source and the first scan
# records an empty library as truth.
unit="$REPO_ROOT/dankserver/audiobookshelf/quadlet/audiobookshelf.container"
grep -q '^ExecStartPre=.*/wait-for-mount.sh /mnt/entropy$' "$unit" || return 1
}
@test "audiobookshelf health check names a binary its image actually ships" {
# The image is Alpine: wget yes, curl no. A curl HealthCmd reads the
# container as unhealthy while it is serving perfectly.
unit="$REPO_ROOT/dankserver/audiobookshelf/quadlet/audiobookshelf.container"
grep -q '^HealthCmd=wget ' "$unit" || return 1
! grep -q '^HealthCmd=curl' "$unit" || return 1
grep -q '^PublishPort=8000:8000$' "$unit" || return 1
}
@test "the audiobookshelf deployer creates both state roots before it starts" {
body="$(sed -n '/^deploy_audiobookshelf()/,/^\}/p' "$REPO_ROOT/scripts/deploy-remote.sh")"
mkdir_line="$(printf '%s\n' "$body" | grep -n 'mkdir -p' | head -1 | cut -d: -f1)"
start_line="$(printf '%s\n' "$body" | grep -n 'systemctl --user restart' | head -1 | cut -d: -f1)"
[ -n "$mkdir_line" ] && [ -n "$start_line" ] || return 1
[ "$mkdir_line" -lt "$start_line" ] || return 1
printf '%s\n' "$body" | grep -q 'audiobookshelf/config' || return 1
printf '%s\n' "$body" | grep -q 'audiobookshelf/metadata' || return 1
printf '%s\n' "$body" | grep -q 'wait_for_health audiobookshelf' || return 1
}
@test "audiobookshelf sets its port and base path with the names the image reads" {
# Measured 2026-11-02 on the first deploy: ABS_PORT / ABS_CONFIG_DIR are the npm
# package's names and this image ignores them, so the server came up on :80
# under base path /audiobookshelf and the deploy gate timed out beside a unit
# that was active (running). PORT and ROUTER_BASE_PATH are what the image's own
# ENV block and the server source actually use.
unit="$REPO_ROOT/dankserver/audiobookshelf/quadlet/audiobookshelf.container"
grep -q '^Environment=PORT=8000$' "$unit" || return 1
grep -q '^Environment=ROUTER_BASE_PATH=/$' "$unit" || return 1
! grep -qE '^Environment=ABS_' "$unit" || return 1
}
+6 -5
View File
@@ -28,15 +28,16 @@ REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
! grep -q 'frank_npu\|frank-npu' "$REMOTE_DEPLOY"
}
@test "the retired ik chat template plans nothing, and its archive is still here" {
@test "the retired ik chat template plans nothing" {
# scripts/chat-template.jinja existed only for the ik_llama.cpp runtime
# (retired 2026-09-16 in favour of halogen-flash-server again). The
# llama.cpp frank-zealot runtime serves the template the GGUF itself
# carries, which is the one with reasoning_effort, preserve_thinking and
# the tool-call grammar. Editing the archive must deploy nothing -- and the
# archive must still be present, so this cannot pass by the file having
# vanished.
[ -f "$REPO_ROOT/scripts/.AGENTTRASH/chat-template.jinja.ik-llama-qwen-sharp" ]
# the tool-call grammar. The planner's rule is a PATH rule, so an edit at
# that path deploys nothing no matter where the bytes currently sit: the seam
# deliberately does not assert the archive exists. Trash is a purgeable hold
# (untracked in #111, and the deletion policy says the purger deletes it), so
# pinning it fails every fresh checkout and proves nothing about planning.
run python3 "$PLANNER" --target dankclaw --paths scripts/chat-template.jinja
[ "$status" -eq 0 ]
[ -z "$output" ]
+130 -3
View File
@@ -13,9 +13,12 @@
# 3. a downloaded plugin artifact is accepted only on its pinned sha256, its
# official guid and its pinned version (a swapped or downgraded artifact
# is refused before anything lands in plugins/);
# 4. the TMDb key is written into the EXISTING config with every other
# setting preserved — Jellyfin owns that file, and clobbering it would
# silently reset country, language and adult settings;
# 4. a key is written into the EXISTING config with every other setting
# preserved, under whatever root that file's owner chose — Jellyfin's core
# configs say `<Configuration>`, TheTVDB writes `<PluginConfiguration>` and
# omits a null setting outright, so the element to fill can be absent and
# the anchor has to come from the file; clobbering the file would silently
# reset country, language and adult settings;
# 5. a TVDB config the operator already saved through the UI is kept;
# 6. install is idempotent: a second run changes no byte;
# 7. whatever is replaced is copied to the trash first (fleet law: never
@@ -24,6 +27,9 @@
# 9. no plugin configuration XML is tracked in git, the deployer places the
# keys BEFORE it restarts Jellyfin, and the planner routes a change to this
# installer to jellyfin.
# 10. verify refuses a library that is a SILENT FILE INDEX — internet providers
# switched off, or a type whose ImageFetchers is empty while its
# MetadataFetchers list is populated (the 2026-11-01 missing-posters fault).
load helpers
@@ -59,6 +65,32 @@ setup() {
TRASH="$BATS_TEST_TMPDIR/trash"
SEC="$BATS_TEST_TMPDIR/secrets"
mkdir -p "$JF/plugins/configurations" "$TRASH" "$SEC"
# The per-library option file Jellyfin 12 owns (it was library.xml). Conforming
# by default so the pre-existing verify tests keep testing what they tested.
mkdir -p "$JF/root/default/Movies"
cat >"$JF/root/default/Movies/options.xml" <<XML
<?xml version="1.0" encoding="utf-8"?>
<LibraryOptions xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<EnableInternetProviders>true</EnableInternetProviders>
<MetadataSavers>
<string>Nfo</string>
</MetadataSavers>
<TypeOptions>
<TypeOptions>
<Type>Movie</Type>
<MetadataFetchers>
<string>TheMovieDb</string>
</MetadataFetchers>
<ImageFetchers>
<string>TheMovieDb</string>
</ImageFetchers>
<ImageFetcherOrder>
<string>TheMovieDb</string>
</ImageFetcherOrder>
</TypeOptions>
</TypeOptions>
</LibraryOptions>
XML
cat >"$JF/plugins/configurations/Jellyfin.Plugin.Tmdb.xml" <<XML
<?xml version="1.0" encoding="utf-8"?>
<Configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
@@ -225,6 +257,38 @@ XML
grep -qF '<EnableSeasonPathMatching>true</EnableSeasonPathMatching>' "$cfg"
}
@test "the key lands in a plugin-written config with no ApiKey element" {
# What TheTVDB actually writes on a fresh install: its own root element and
# no <ApiKey /> at all, because its serializer drops null settings. The
# installer used to anchor on </Configuration> and refuse this file.
cat >"$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml" <<XML
<?xml version="1.0" encoding="utf-8"?>
<PluginConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<SubscriberPIN />
<CacheDurationInHours>1</CacheDurationInHours>
<OriginalTitle>true</OriginalTitle>
</PluginConfiguration>
XML
run prov install
[ "$status" -eq 0 ]
printf '%s\n' "${lines[@]}" | grep -qiF 'filled the ApiKey'
local cfg="$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml"
grep -qF "<ApiKey>$TVDB_KEY</ApiKey>" "$cfg"
grep -qF '<CacheDurationInHours>1</CacheDurationInHours>' "$cfg"
grep -qF '<OriginalTitle>true</OriginalTitle>' "$cfg"
}
@test "a config file with no root element is refused by name" {
# The guard still has to fire when there is genuinely nowhere to write —
# a refusal naming the file beats inventing a document shape.
printf '<?xml version="1.0" encoding="utf-8"?>\n' \
>"$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml"
run prov install
[ "$status" -ne 0 ]
printf '%s\n' "${lines[@]}" | grep -qF 'Jellyfin.Plugin.Tvdb.xml'
printf '%s\n' "${lines[@]}" | grep -qiF 'no place to write'
}
@test "a pin in the TVDB secret lands in the plugin config" {
# A subscriber-shaped key needs the account's PIN beside it; a licensed key
# does not, so the element appears only when the secret carries one.
@@ -270,6 +334,69 @@ XML
printf '%s\n' "${lines[@]}" | grep -qE 'TmdbApiKey .*[0-9]+ chars'
}
@test "verify refuses a library whose internet providers are switched off" {
# The whole library becomes a file index: refreshes answer 204 and store
# nothing, and the item count still looks like a healthy scan.
prov install >/dev/null
sed -i 's#<EnableInternetProviders>true</EnableInternetProviders>#<EnableInternetProviders>false</EnableInternetProviders>#' \
"$JF/root/default/Movies/options.xml"
run prov verify
[ "$status" -ne 0 ]
printf '%s\n' "${lines[@]}" | grep -qiF 'SILENT INDEX Movies'
printf '%s\n' "${lines[@]}" | grep -qiF 'EnableInternetProviders'
}
@test "verify refuses a type with no image fetchers while metadata fetchers are named" {
prov install >/dev/null
cat >"$JF/root/default/Movies/options.xml" <<XML
<?xml version="1.0" encoding="utf-8"?>
<LibraryOptions>
<EnableInternetProviders>true</EnableInternetProviders>
<TypeOptions>
<TypeOptions>
<Type>Movie</Type>
<MetadataFetchers>
<string>TheMovieDb</string>
</MetadataFetchers>
<ImageFetchers />
<ImageFetcherOrder>
<string>TheMovieDb</string>
</ImageFetcherOrder>
</TypeOptions>
</TypeOptions>
</LibraryOptions>
XML
run prov verify
[ "$status" -ne 0 ]
printf '%s\n' "${lines[@]}" | grep -qiF 'ImageFetchers empty'
}
@test "an older options.xml that never carried the switch is not refused" {
# BoxSets on this host were written before the property existed; absence is
# the vendor default, not a switched-off library, so it must not fail.
prov install >/dev/null
mkdir -p "$JF/root/default/Collections"
cat >"$JF/root/default/Collections/options.xml" <<XML
<?xml version="1.0" encoding="utf-8"?>
<LibraryOptions>
<TypeOptions>
<TypeOptions>
<Type>BoxSet</Type>
<MetadataFetchers>
<string>TheMovieDb</string>
</MetadataFetchers>
<ImageFetchers>
<string>TheMovieDb</string>
</ImageFetchers>
</TypeOptions>
</TypeOptions>
</LibraryOptions>
XML
run prov verify
[ "$status" -eq 0 ]
printf '%s\n' "${lines[@]}" | grep -qiF '= Collections'
}
@test "no plugin configuration XML is tracked in this repo" {
# These files hold provider keys and are owned by the running container;
# tracking them would put credentials in git and guarantee drift.
+26 -1
View File
@@ -10,7 +10,8 @@
# 4. both media units bind the VIEW and refuse to start without it;
# 5. only those two units touch the view — the LLM and library tenants keep the
# raw volume, so the blast radius is two containers;
# 6. the deployer brings the view up BEFORE it restarts either server;
# 6. the deployer creates the upper base and brings the view up BEFORE it
# restarts either server;
# 7. the planner routes a change to these files to both media services.
load helpers
@@ -110,6 +111,30 @@ UNIT="$REPO_ROOT/dankserver/host/systemd/media-meta-view.mount"
done
}
@test "ensure-base creates a base directory that install refuses to invent" {
base="$BATS_TEST_TMPDIR/fresh-base"
run bash "$VIEW" ensure-base --upper-base "$base"
[ "$status" -eq 0 ]
[ -d "$base" ]
# The deployer runs this on every media deploy, so a second call must be a
# no-op and must say so rather than recreate anything.
run bash "$VIEW" ensure-base --upper-base "$base"
[ "$status" -eq 0 ]
[[ "${lines[0]}" == *"exists"* ]]
}
@test "the media deployer creates the upper base before it installs the view" {
body="$(sed -n '/^ensure_media_view()/,/^\}/p' "$REPO_ROOT/scripts/deploy-remote.sh")"
[ -n "$body" ]
base="$(printf '%s\n' "$body" | grep -n 'media-view.sh ensure-base' | head -1 | cut -d: -f1)"
inst="$(printf '%s\n' "$body" | grep -n 'media-view.sh install' | head -1 | cut -d: -f1)"
# Order is the whole test: install refuses a missing base, so an installer
# that runs first dies on a rebuilt host (measured 2026-10-31).
[ -n "$base" ]
[ -n "$inst" ]
[ "$base" -lt "$inst" ]
}
@test "the planner routes the view files to both media services" {
run python3 "$REPO_ROOT/scripts/deploy-plan.py" --target dankserver --paths \
scripts/media-view.sh dankserver/host/systemd/media-meta-view.mount
+33
View File
@@ -460,6 +460,39 @@ print(sum(1 for l in open(sys.argv[1]) if json.loads(l).get("op") == "move"))' "
[ -e "$LIB/Alien (1979)/Alien.1979.1080p.BluRay.x265.10bit.mkv" ]
}
# A hand-written plan row whose verb is not in the executor's vocabulary used
# to be dropped in silence: read_plan's if/elif chain had no else, so a plan of
# nothing but typos applied as a successful no-op (hit for real on 2026-11-02,
# when a pass meant to rename two series folders reported "applied: 0 moved"
# and exit 0). An unread row is a plan bug, and a plan bug refuses.
@test "a plan row with a verb this tool does not know stops the whole run" {
plan
JOURNAL="$BATS_TEST_TMPDIR/journal.jsonl"
at "$LIB/Stray/Stray.S01E01.mkv" 4000
printf 'rename\t%s\t%s\tfile\n' \
"$LIB/Stray/Stray.S01E01.mkv" "$LIB/Stray/Stray - S01E01.mkv" >> "$OUT"
run python3 "$APPLY" apply --plan "$OUT" --root "$LIB" --journal "$JOURNAL" --apply
[ "$status" -ne 0 ]
[[ "$output" == *"rename"* ]]
# all-or-nothing: the rows this tool DOES understand did not run either
[ -e "$LIB/Stray/Stray.S01E01.mkv" ]
[ ! -e "$JOURNAL" ]
}
@test "a plan whose every row is an unknown verb refuses instead of reporting success" {
JOURNAL="$BATS_TEST_TMPDIR/journal.jsonl"
at "$LIB/Stray/Stray.S01E01.mkv" 4000
{ printf 'action\tsrc\tdst\tkind\n'
printf 'rename\t%s\t%s\tfile\n' \
"$LIB/Stray/Stray.S01E01.mkv" "$LIB/Stray/Stray - S01E01.mkv"
} > "$BATS_TEST_TMPDIR/typo.tsv"
run python3 "$APPLY" apply --plan "$BATS_TEST_TMPDIR/typo.tsv" --root "$LIB" --journal "$JOURNAL" --apply
[ "$status" -ne 0 ]
[[ "$output" == *"unknown action"* ]]
[[ "$output" != *"applied"* ]]
[ -e "$LIB/Stray/Stray.S01E01.mkv" ]
}
@test "a run interrupted between the journal write and the move finishes the job" {
plan
JOURNAL="$BATS_TEST_TMPDIR/journal.jsonl"
+215
View File
@@ -0,0 +1,215 @@
#!/usr/bin/env bats
# Seam: scripts/media-tvdupes.py — the episode-level duplicate census for the TV
# library.
#
# The failure this seam exists to prevent is the one that almost cost real
# content: two folders of one show look like copies, and a folder-level "keep
# the bigger folder" deletes the half of the series only the smaller folder
# holds (the 2026-11-02 census found 10 such complementary pairs). So every
# assertion here is about WHICH files land in the plan and WHICH the tool
# refuses to touch — never about its internals. Everything runs on a synthetic
# library in the test's own tmpdir: no media server, no network, no array.
load helpers
TOOL="$REPO_ROOT/scripts/media-tvdupes.py"
# at PATH SIZE — a file of deterministic bytes.
at() { mkdir -p "$(dirname "$1")"; head -c "$2" /dev/zero > "$1"; }
# sources FILE — the sources the plan moves, one per line.
sources() { awk -F'\t' '$1=="move"{print $2}' "$1"; }
# destinations FILE — where the plan puts them.
destinations() { awk -F'\t' '$1=="move"{print $3}' "$1"; }
# plan_moves FILE — how many files the plan stages.
plan_moves() {
[ -f "$1" ] || { echo 0; return; }
awk -F'\t' '$1=="move"' "$1" | wc -l
}
setup() {
LIB="$BATS_TEST_TMPDIR/TVShows"
HOLD="$BATS_TEST_TMPDIR/.AGENTTRASH/20991231-tv-duplicates"
PLAN="$BATS_TEST_TMPDIR/plan.tsv"
mkdir -p "$LIB" "$HOLD"
}
@test "the smaller copy of an episode is staged and the larger stays put" {
at "$LIB/Doc Who (2005) [tvdbid-78804]/Season 01/Doc Who - S01E01 - Pilot.mkv" 4000
at "$LIB/Doc Who (2005) [tvdbid-78804]/Season 01/backup/Doc.Who.S01E01.720p.mkv" 1200
run python3 "$TOOL" --library "$LIB" --hold "$HOLD" --out "$PLAN"
[ "$status" -eq 0 ]
[ "$(plan_moves "$PLAN")" -eq 1 ]
run sources "$PLAN"
[[ "$output" == *"backup/Doc.Who.S01E01.720p.mkv"* ]]
[[ "$output" != *"Pilot.mkv"* ]]
run destinations "$PLAN"
[[ "$output" == "$HOLD/"* ]]
}
@test "an equal-size pair stages exactly one copy" {
at "$LIB/Equal (2001) [tvdbid-11]/Season 01/Equal - S01E01.mkv" 2500
at "$LIB/Equal (2001) [tvdbid-11]/Rip/Equal - S01E01.mkv" 2500
run python3 "$TOOL" --library "$LIB" --hold "$HOLD" --out "$PLAN"
[ "$status" -eq 0 ]
[ "$(plan_moves "$PLAN")" -eq 1 ]
}
@test "a file holding one episode nothing else has is never staged" {
# The shape that defeats a folder-level verdict: Season 02 exists only here,
# so the folder that looks like a leftover is the only copy of an episode.
at "$LIB/King (2010) [tvdbid-21]/Season 01/King - S01E01 - One.mkv" 3000
at "$LIB/King (2010) [tvdbid-21]/Season 01/backup/King - S01E01 - One.mkv" 900
at "$LIB/King (2010) [tvdbid-21]/Season 02/King - S02E01 - Two.mkv" 800
run python3 "$TOOL" --library "$LIB" --hold "$HOLD" --out "$PLAN"
[ "$status" -eq 0 ]
run sources "$PLAN"
[[ "$output" != *"S02E01"* ]]
[[ "$output" == *"backup/King - S01E01 - One.mkv"* ]]
}
@test "a two-in-one pack competes for every episode in its range" {
at "$LIB/Range (2012) [tvdbid-31]/Season 01/Range - S01E01.mkv" 1500
at "$LIB/Range (2012) [tvdbid-31]/Season 01/Range - S01E02.mkv" 1500
at "$LIB/Range (2012) [tvdbid-31]/Packs/Range - S01E01-02.mkv" 2400
# The pack is the biggest carrier of BOTH episodes, so the two rips go.
run python3 "$TOOL" --library "$LIB" --hold "$HOLD" --out "$PLAN"
[ "$(plan_moves "$PLAN")" -eq 2 ]
run sources "$PLAN"
[[ "$output" == *"Season 01/Range - S01E01.mkv"* ]]
[[ "$output" != *"Packs/Range - S01E01-02.mkv"* ]]
# Make the season file the bigger carrier of E02 and the pack becomes the
# only carrier of E01: it stays, however small it is.
rm "$LIB/Range (2012) [tvdbid-31]/Season 01/Range - S01E01.mkv"
rm "$LIB/Range (2012) [tvdbid-31]/Season 01/Range - S01E02.mkv"
at "$LIB/Range (2012) [tvdbid-31]/Season 01/Range - S01E02.mkv" 9000
run python3 "$TOOL" --library "$LIB" --hold "$HOLD" --out "$PLAN"
[ "$(plan_moves "$PLAN")" -eq 0 ]
}
@test "two tvdb ids on the same words are never merged" {
at "$LIB/Avatar The Last Airbender (2024) [tvdbid-385925]/Season 01/Avatar - S01E01.mkv" 9000
at "$LIB/Avatar The Last Airbender (1999) [tvdbid-74852]/Season 01/Avatar - S01E01.mkv" 500
run python3 "$TOOL" --library "$LIB" --hold "$HOLD" --out "$PLAN"
[ "$status" -eq 0 ]
[ "$(plan_moves "$PLAN")" -eq 0 ]
[[ "$output" == *CONFLICT* ]]
[[ "$output" == *"tvdb [74852] vs [385925]"* ]]
}
@test "a derived id that contradicts the folder's own tag never merges them" {
# Measured 2026-11-02: Jellyfin's provider table carried a 2026 special's tvdb
# id on the folder tagged [tvdbid-73838]. Unioning the two makes one show of a
# 2000 series and a 2026 special, and the special's bigger files dominate the
# original's pilot. The tag is the operator's word; the nfo and `--ids` layers
# fill an untagged folder and never overrule a tagged one.
at "$LIB/Middle (2000) [tvdbid-73838]/Season 01/Middle - S01E01 - Pilot.mkv" 800
at "$LIB/Middle (2026) [tvdbid-457779]/Season 01/Middle - S01E01 - Episode 1.mkv" 1100
IDS="$BATS_TEST_TMPDIR/ids.tsv"
printf '%s\t%s\n' "Middle (2000) [tvdbid-73838]" 457779 > "$IDS"
run python3 "$TOOL" --library "$LIB" --hold "$HOLD" --out "$PLAN" --ids "$IDS"
[ "$status" -eq 0 ]
[ "$(plan_moves "$PLAN")" -eq 0 ]
[[ "$output" == *"DISAGREE"* ]]
[[ "$output" == *"tag tvdb [73838]"* ]]
[[ "$output" == *"derived tvdb [457779]"* ]]
}
@test "an untagged drop is identified by the tvshow.nfo in the metadata view" {
at "$LIB/Clone Wars/Season 01/Clone Wars - S01E01.mkv" 1200
at "$LIB/Clone Wars (2008) [tvdbid-84859]/Season 01/Clone Wars - S01E01.mkv" 4000
VIEW="$BATS_TEST_TMPDIR/view"
mkdir -p "$VIEW/Clone Wars"
printf '%s\n' '<tvshow><title>Clone Wars</title>' \
'<uniqueid type="tvdb" default="true">84859</uniqueid></tvshow>' \
> "$VIEW/Clone Wars/tvshow.nfo"
run python3 "$TOOL" --library "$LIB" --hold "$HOLD" --out "$PLAN" --meta "$VIEW"
[ "$status" -eq 0 ]
[ "$(plan_moves "$PLAN")" -eq 1 ]
run sources "$PLAN"
[[ "$output" == *"Clone Wars/Season 01/Clone Wars - S01E01.mkv"* ]]
# Without the nfo layer nothing identifies the drop, so nothing moves.
run python3 "$TOOL" --library "$LIB" --hold "$HOLD" \
--out "$BATS_TEST_TMPDIR/no-meta.tsv"
[ "$status" -eq 0 ]
[[ "$output" == *UNRESOLVED* ]]
[ "$(plan_moves "$BATS_TEST_TMPDIR/no-meta.tsv")" -eq 0 ]
}
@test "a file named by an id is keyed only when the witness names it" {
BIG="$LIB/Edd (1999) [tvdbid-75897]/Season 01/Edd - S01E01 - Down the Line.mkv"
SMALL="$LIB/Edd (1999) [tvdbid-75897]/Season 01/104348.mkv"
at "$BIG" 2100
at "$SMALL" 700
WIT="$BATS_TEST_TMPDIR/witness.tsv"
printf '%s\t1\t1\n' "$SMALL" > "$WIT"
run python3 "$TOOL" --library "$LIB" --hold "$HOLD" --out "$PLAN"
[ "$(plan_moves "$PLAN")" -eq 0 ]
[[ "$output" == *"unkeyed, never staged: 1 files"* ]]
run python3 "$TOOL" --library "$LIB" --hold "$HOLD" --out "$PLAN" --witness "$WIT"
[ "$(plan_moves "$PLAN")" -eq 1 ]
run sources "$PLAN"
[[ "$output" == *104348.mkv* ]]
}
@test "the sidecars of a staged episode are staged with it" {
at "$LIB/Side (2003) [tvdbid-41]/Season 01/Side - S01E01 - First.mkv" 3000
at "$LIB/Side (2003) [tvdbid-41]/Rip/Side - S01E01 - First.mkv" 1000
at "$LIB/Side (2003) [tvdbid-41]/Rip/Side - S01E01 - First.srt" 400
at "$LIB/Side (2003) [tvdbid-41]/Season 01/Side - S01E01 - First.srt" 450
run python3 "$TOOL" --library "$LIB" --hold "$HOLD" --out "$PLAN"
[ "$(plan_moves "$PLAN")" -eq 2 ]
run sources "$PLAN"
[[ "$output" == *"Rip/Side - S01E01 - First.srt"* ]]
[[ "$output" != *"Season 01/Side - S01E01 - First.srt"* ]]
}
@test "a hold inside the library is refused" {
at "$LIB/Hostage (2007) [tvdbid-51]/Season 01/Hostage - S01E01.mkv" 100
run python3 "$TOOL" --library "$LIB" --hold "$LIB/.AGENTTRASH/hold" \
--out "$PLAN"
[ "$status" -ne 0 ]
[[ "$output" == *"--hold"* ]]
}
@test "a plan whose hold target is occupied is refused, plan and all" {
at "$LIB/Gone (2009) [tvdbid-61]/Season 01/Gone - S01E01.mkv" 1000
at "$LIB/Gone (2009) [tvdbid-61]/Rip/Gone - S01E01.mkv" 400
run python3 "$TOOL" --library "$LIB" --hold "$HOLD" --out "$PLAN"
[ "$(plan_moves "$PLAN")" -eq 1 ]
# The hold already holds one of the destinations: the executor would refuse
# it, so a plan that cannot run must not be handed over at all.
mkdir -p "$HOLD/Gone (2009) [tvdbid-61]/Rip"
at "$HOLD/Gone (2009) [tvdbid-61]/Rip/Gone - S01E01.mkv" 400
rm "$PLAN"
run python3 "$TOOL" --library "$LIB" --hold "$HOLD" --out "$PLAN"
[ "$status" -eq 1 ]
[[ "$output" == *"hold target occupied"* ]]
[ ! -e "$PLAN" ]
}
@test "a library with nothing duplicated writes an empty plan" {
at "$LIB/Unique (2015) [tvdbid-71]/Season 01/Unique - S01E01.mkv" 1000
at "$LIB/Other Show (2016) [tvdbid-72]/Season 01/Other Show - S01E01.mkv" 1000
run python3 "$TOOL" --library "$LIB" --hold "$HOLD" --out "$PLAN"
[ "$status" -eq 0 ]
[ "$(plan_moves "$PLAN")" -eq 0 ]
[[ "$output" == *"plan: 0 moves"* ]]
}