Commit Graph
1582 Commits
Author SHA1 Message Date
Frank 51ed5c0b2a docs(media): stat through the overlay can outlive the file it points at
Measured 2026-11-02 while a TV dedupe pass moved folders on the raw path: the
merged tree still answered stat at the old coordinates, link count 0, while ls of
the same parent no longer listed the name. Overlayfs keeps a dentry for a lower
file moved underneath it, so a tool that tests existence with [ -e ] against the
view can believe a file its own readdir denies — the media servers walk by
readdir, so this bites a script and never a user.

Recorded with it: every pass that moves or measures media walks /mnt/entropy, and
the one reader allowed to point at the view is the tool that needs show identity.
2026-11-02 00:44:51 -08:00
Frank 9f9043c037 feat(media): measure a series per episode, and let the tag outrank a borrowed id
media-tvdupes.py censuses a TV library by episode key and never by folder. 10 of
the 25 shows that looked duplicated in the 2026-11-02 census held complementary
halves (Game of Thrones Complete carries no S01 or S08, Modern Family S05 is the
only copy of season 5), so a folder-level verdict would have staged some 340 GB
of unique episodes to recover 54 GiB of true copies.

Show identity comes from the folder's [tvdbid-N] tag, from the tvshow.nfo the
metadata view exposes, or from an operator --ids map. A derived id that
contradicts a tag is dropped and reported, never unioned: Jellyfin's provider
table carries a 2026 special's id on the folder tagged [tvdbid-73838], and
unioning the two makes one show of a 2000 series and a 2026 special whose bigger
files then dominate the original's pilot. Files named by an id key only when the
server's own episode table (the --witness) names them, so a bare number is never
mistaken for an episode.

Seam: tests/media_tvdupes.bats — 12 assertions on which files land in the plan
and which the tool refuses to touch, on a synthetic library.
2026-11-02 00:44:46 -08:00
Frank a25f139281 docs(jellyfin): the art delivery result, and what stays blank on purpose 2026-11-01 11:39:11 -08:00
Frank 52600b28ab fix(jellyfin): refuse a library that indexes files but fetches nothing
The Movies library had indexed 2,571 films and had never asked a provider
for a single one: no TMDb id, no plot, no rating, no poster. Nothing was
broken in the provider chain — the plugin is core in 12.4 and active, its
key is set, image.tmdb.org answers the container — and nothing was broken
in the write path: the overlay view has been read-write since the 2026-10-31
cutover, and movie.nfo lands on the NVMe upper layer today.

The state was two settings in the per-library option file,
config/root/default/<Library>/options.xml (Jellyfin 12 renamed it from
library.xml; the database stores none of these options, so a census that
greps for the old name and then queries the DB concludes nothing is
configured and goes hunting elsewhere):

  <EnableInternetProviders>false</EnableInternetProviders>
  <TypeOptions><ImageFetchers />   (empty, beside a populated
                                    <MetadataFetchers> and a fully
                                    populated <ImageFetcherOrder>)

With the internet switch off every refresh answers 204, stores nothing and
logs nothing, so the item count reads like a healthy scan — this passed the
deploy health gate, the Playwright suite and three agent audits. Order is
not enablement: the fetcher ORDER stays populated while the ENABLED set sits
empty, and the populated metadata list beside it makes the library look well.

verify now audits every library's options.xml and fails with `SILENT INDEX
<Library>: <what>` on providers-off or on a type whose image fetchers are
empty while its metadata fetchers are named; an options.xml that never
carried the switch (BoxSets, written before the property existed) is the
vendor default and stays accepted.

Vault: the page claimed artwork is a sidecar like movie.nfo. It is not — no
image saver is bound for any library, so art goes to the internal store and
Plex never reads it. The claim is corrected, and the RemoteImages diagnostic
that separates a provider fault from a library fault is written down with
the 12.x API shape traps (Items needs UserId; MetadataRefreshMode has no
Full/ValidationFailure, ReplaceAllMetadata=true is what re-runs providers).
2026-11-01 10:31:42 -08:00
Frank 7cf388367e docs(media): the array's write doors, and why survivors are re-verified by size
A folder-rename pass ran on the media array with no journal anywhere: no
jsonl on the array, no bash history, no cron/timer/job artifact, no other
host ever authenticating. The door was GNOME Files mounting the array over
SFTP (gvfsd-sftp -> sftp-server as dank): rename() with no shell and no
journal, bumping only the renamed directory's ctime. Copyparty cannot do
this (its bind is ro at the mount) and the *arr stack has no route to the
array at all (no SMB, no NFS, mini asleep).

Two laws fall out for the prune pipeline: re-verify survivors by size
rather than by the path a stale artifact recorded, and Plex extras rows
naming a held path survive a second section pass.
2026-11-01 09:46:32 -08:00
Frank 13830865f6 docs(media): the Plex leg ran, and a script fed over stdin owns that stdin
The 2026-11-01 morning leg cleared every Plex row naming a held path, so the prune page records the outcome plus the residue that is normal: a section pass rewrites feature rows first and can leave rows for clips in a film's other/ or extras/ folder standing after the refreshing flag is back to 0. Beside REAL: 0 that is a dead disc-menu clip, not a false prune, and no reason to touch the cache or the database.

Plex gains the two operational laws the run taught: a start job issued after an unmask belongs to the manager, not the shell, and lands minutes later surviving the client that issued it; a refresh fired while PMS is warming answers 000, so retry it rather than restart the service.

The trap that made a finished job look hung belongs to the remote job runner: a script fed to bash -s IS the stdin, so any child that reads stdin (podman exec -i) swallows the rest of the script without a word.
2026-11-01 09:38:38 -08:00
dank 010b9affc7 Merge pull request 'media-verify reads the WAL; the metadata view is live; duplicates get a page' (#115) from fix/media-verify-wal-and-prune-record into main 2026-11-01 01:42:20 -07:00
Frank ed5eed6d54 docs(media): the deferred Plex leg, and the guards a detritus sweep must respect 2026-11-01 01:42:02 -07:00
Frank 14f340370c Merge remote-tracking branch 'origin/main' into feat/jellyfin-provider-keys
# Conflicts:
#	AGENTS.md
#	serverAppConfigs-docs/Dankserver/Media Metadata View.md
#	serverAppConfigs-docs/index.md
2026-11-01 01:36:35 -07:00
Frank 50e0c5a786 docs(media): the metadata view is live, and duplicate pruning gets a page
`Media Metadata View.md` said "Not deployed on Dankserver — verified
2026-10-31". That was true yesterday and is false now: the overlay is mounted
and both installed quadlets bind it (`view:/data` rw for Jellyfin,
`view:/mnt/entropy:ro` for Plex). The page now states the live shape and the
order that proves it — `media-view.sh verify`, then the `Volume=` lines —
because the whole point of the page is to answer "is the view really serving?".
`AGENTS.md` carried the same stale claim in its summary table.

`Media Duplicate Prune.md` is new: the screen ladder (size, then head+tail
blake2b, then a sparse-window confirmation before anything moves), which copy
survives and why, the guards that each encode a file that would otherwise have
been destroyed (qBittorrent `.!qB` state, year-required with digits kept,
feature length, disc/part exclusion, media containers only), and the law that a
hold on the same volume frees nothing until it is purged. It also carries the
different way `media-verify.py` has to be READ for a prune — `landed: 0` is the
correct end state because no server may index the hold — and the planner bug
that class explains: when the planner re-derived "largest wins" from the hold
list alone it dropped every keeper out of its own group and pruned 17 files
instead of 205.

`Media Relayout.md` keeps a one-line pointer; carrying the whole procedure there
pushed it past the vault's 500-line ceiling, which is the linter telling the
truth about progressive disclosure.
2026-11-01 01:11:26 -07:00
Frank 649bbafa17 fix(media-verify): read the WAL the server commits through
`mode=ro&immutable=1` claims the file never changes, so sqlite skips the
write-ahead log entirely: every row the server has committed but not yet
checkpointed reads as MISSING, and a read beside a running scanner can fail
outright as "database disk image is malformed". Both servers keep their
libraries in WAL, so plain `mode=ro` is now the first door and `immutable=1`
only the fallback for a database whose `-shm` cannot be read.

The verdict also stopped guessing from the database alone. A file still at its
`src` is REAL (this pass's own damage); a file at neither path is `gone` (a
later pass or the server renamed it); a file sitting at `dst` with no row
naming it is something the server does not index. A row that still names `src`
is none of those — it is the server's stale pointer, and it costs a re-scan,
never a restore.
2026-11-01 01:04:22 -07:00
Frank b7c19592e5 docs(jellyfin): record the live view bind and the library-coverage law
The page still described the pre-cutover generation ("the live unit is
still pre-view", "/mnt/entropy -> /data (live today, read-only)") after
the 2026-10-31 cutover deployed the overlay. Corrected, and added the law
the census tonight taught: Jellyfin's four libraries cover /data/Movies,
TVShows, Music and the collections dir, so anything under /data/media-park
is invisible by configuration — a recursive refresh there returns 204 and
indexes nothing. Check coverage before re-scanning.
2026-10-31 19:43:35 -07:00
dank 6c115c1394 Merge pull request 'fix(deploy): make the view's upper base before installing the view' (#113) from fix/media-view-ensure-base into main 2026-10-31 19:36:52 -07:00
dank 6d79709cc0 fix(deploy): make the view's upper base before installing the view
`ensure_media_view` ran `install` on a host where the documented base directory
did not exist, and `install` refuses by design ("put it on the SSD first") — so
tonight's cutover would have died at the restart step with nobody beside the
box. `media-view.sh ensure-base` now makes the documented directory, and the
deployer calls it first; two seams cover it (the subcommand's behaviour, and the
ordering invariant inside `ensure_media_view`).

Docs flip with the deploy: the view is live on Dankserver (both media servers
verified resolving real media through it), so every "never been deployed" claim
in the view, Plex, Jellyfin and AGENTS pages is corrected, and three laws join
them — a container restart discards the in-flight library scan and nothing
resumes it; Jellyfin 12.4 authenticates only the `MediaBrowser Token` form, so a
401 says nothing about the key until the `ApiKeys` table is read; and Plex
answers 32400 over TLS only, which makes a plaintext health probe read as dead.
2026-10-31 19:33:16 -07:00
dank 0336f42e2a Merge pull request 'test(scoping): stop pinning a purgeable trash archive' (#112) from fix/scoping-seam-disposable-archive into main 2026-10-31 19:30:35 -07:00
dank 0f4a527337 test(scoping): stop pinning a purgeable trash archive
The seam asserted `[ -f scripts/.AGENTTRASH/chat-template.jinja.ik-llama-qwen-sharp ]`
so that the planning assertion "could not pass by the file having vanished". That
reasoning was wrong twice over: the planner's rule is a PATH rule, so it plans
nothing for that path no matter where the bytes sit, and the trash is a purgeable
hold — #111 untracked it, so a fresh worktree or CI checkout has no such file and
the suite went red on a byte that carries no meaning.

Measured: fresh worktree off origin/main, deployment_scoping.bats:39 fails. After
the change: that file 28 ok / 0 fail, ./tests/run.sh 789 ok exit 0.
2026-10-31 19:27:56 -07:00
dank 1d92627935 Merge pull request 'chore: stop tracking .AGENTTRASH so a purge cannot dirty a checkout' (#111) from chore/untrack-agenttrash into main 2026-10-31 18:49:41 -07:00
dank 8ae4d1c710 chore: stop tracking .AGENTTRASH — the purger deletes tracked files, dirtying every host
The deletion policy moves in-repo deletions into .AGENTTRASH/ beside the item,
and .gitignore line 25 already declares it ignored. But 37 retired payloads were
committed before that rule existed, and gitignore never applies to tracked paths.
Consequence measured 2026-10-31: Dankserver's deploy checkout reported 37 tracked
deletions — exactly every tracked trash file — because the approved purger had
done its job. Any host that purges is permanently dirty, and a future pull that
touches one of those paths aborts mid-deploy with 'local changes would be
overwritten' while the tree is in fact clean of real edits.

Untrack the four dirs; the bytes stay on disk and in history (they always were
recoverable from git, which is what the trash policy's 'temporary hold, not an
archive' means). tests/deployment_scoping.bats asserts the staged chat template
exists on disk — it still does, so the seam is untouched.
2026-10-31 18:46:53 -07:00
dank 6a4291f224 Merge pull request 'feat(fleet): give the media servers an I/O share that actually exists' (#110) from feat/media-io-weight into main 2026-10-31 18:43:49 -07:00
dank 183a6d5442 feat(fleet): give the media servers an I/O share that actually exists
A Plex section scan walked 50k files at ~850 r/s and nothing bounded it.
`IOWeight=` in a quadlet looks like the fix and does nothing: systemd delegates
`cpu memory pids` to user@.service and NOT io, and it drops a resource line it
cannot honour without a word — the container boots healthy and its cgroup has no
io.weight file at all. Same family as RequiresMountsFor= (inert in the user
manager) and ConditionPathIsMountPoint= (fail-dead).

Place the two system drop-ins that make the controller reachable (user.slice
IOAccounting=yes puts io in user.slice/cgroup.subtree_control, measured
immediately; user@1000.service DelegateControllers gains io and lands at the
next reboot, because restarting the user manager would kill every rootless
container on the host), then ask for the share: IOWeight=10 on jellyfin and plex.
10 is a convention, not a measured optimum — proportional share, only bites
under contention, only requirement is far below the default 100.

ensure_user_io_delegate places both from git and prints NOTE: io is not
delegated … inert until the next reboot, so the quadlet line never reads as live
config. Verified on the host: subtree_control flipped to `cpu io memory pids`,
the drop-in is attached and parsed (systemctl cat), and the podman 5.7 generator
emits IOWeight=10 for both units (bare key; UnitConfigItem= not needed).

Two limits recorded instead of hidden: a container's weight covers its whole
cgroup (scan and playback share it), and the comparison happens inside
user.slice, so it cannot push a rootless unit below a root unit such as
frank-archon — that pairing is user.slice's own weight, a separate call.

Docs: Quadlet Lifecycle 4b (the three-command read of the chain), Pi-hole (all
rootless containers share one 10.0.2.100, so one chatty container throttles DNS
for the host — 8 events today), E2E Suite gotcha 7 (the gate depends on the
runner's resolver: 21 reds on a lost wired link, 37/37 minutes later).

Seam: an in-repo IOWeight= may only exist when both drop-ins exist AND the
deployer places them. ./tests/run.sh: 789 ok, exit 0.
2026-10-31 18:42:27 -07:00
dank 6d274acc47 Merge pull request 'fix(fleet): refuse a container start until the mount it binds has answered' (#109) from fix/array-mount-gate into main 2026-10-31 18:20:40 -07:00
dank 3344b51fbd Merge pull request 'perf(media): the apply fsyncs the journal once per batch, not per move' (#108) from perf/media-apply-batch into main 2026-10-31 18:20:17 -07:00
dank e82c8b2d06 fix(fleet): refuse a container start until the mount it binds has answered
Dankserver rebooted with the array's mount and the media containers racing. A
rootless quadlet cannot order itself against the system's mnt-entropy.mount, so
`RequiresMountsFor=/mnt/entropy` is inert in the user manager: podman resolved
the bind against the pre-mount directory, Jellyfin and Plex came up blind, and
Jellyfin read an empty library and began `Removing item, Type: "Season"` by the
thousand. Copyparty had no guard at all.

`scripts/wait-for-mount.sh` is the gate: it waits for every named mountpoint,
then refuses by name. `ConditionPathIsMountPoint=` is NOT the fix — measured on
the host, an unmet condition logs `skipped, unmet condition check` with
NRestarts=0 and no start attempt ever follows, which is fail-dead, not
fail-closed. Jellyfin and Plex swap the condition for the gate; Copyparty gets
one; the law is recorded in AGENTS.md and the view page.

Verified on the host: the 5.7.0 quadlet generator accepts ExecStartPre in
[Service] (exit 0, all three units generated, gate present), the gate waits for
a mount that appears mid-window and proceeds, and it refuses a plain directory
after its window naming the path.

tests/media_overlay.bats pinned the old mechanism by name; its intent — both
media servers refuse to start without the view — now rides the gate.
2026-10-31 18:17:02 -07:00
dank f7083f175d perf(media): the apply fsyncs the journal once per batch, not per move
The 58k-rename apply on the spinning array spent most of its wall time in
fsyncs: one durability flush per rename, and a quadratic `Journal.already()`
scan over a growing record list on top of it. Both are the journal's, not the
renames'.

- `Journal` indexes move intents by (src, dst) so `already()` is O(1).
- `--batch N` (default 64) writes N intents, fsyncs once, then performs those
  N renames. The invariant is unchanged: no rename runs before its own intent
  is durable, so a crash mid-batch replays the batch and resumes.
- Tests: a mid-batch kill loses nothing and the rerun completes; batching
  changes no final path and no journal line.
2026-10-31 18:04:37 -07:00
dank 75b04dc313 Merge pull request 'fix(media): a number too long to be an episode is not one' (#107) from fix/episode-marker-guard into main 2026-10-31 14:51:03 -07:00
Frank 2625ad1083 fix(media): a number too long to be an episode is not one
The Attack on Titan box set on the array names two files S03E2019 and
S04E47370355, and the provider rows the metadata server recorded for them carry
the same junk. EP_RANGE capped its first episode group at three digits without
guarding the run, so the pass-2 plan filed both under E201 and E473 — slots
that belong to real episodes — inside 進撃の巨人 (2013) [tvdbid-267440].

- guard the first episode group the way the second was already guarded, so a
  longer digit run states no marker at all; a glued resolution stays legal,
  because S01E01-1080p is still ONE episode
- cap a provider row's episode at EP_MAX: such a row is renumbered from tvdb's
  title when the series' id answers, and parks with a reason that names the
  junk when nothing answers
- split the operator-map exceptions into the two things they are: a folder that
  IS on the array whose files never became a series (bonus material with no
  owner: Fairy Tail, Kimetsu no Yaiba, Dorohedoro) is no longer reported as a
  folder the array no longer holds, and the missing-folder hint says the review
  pile counts as moved because the map reads top-level folders only
- the vault's evidence list carries the law; the four tests were read red
  against the shipping planner before any of it changed

Replanning the two real files now parks them under their own names with the
reasons "a provider row of season 3 episode 2019" and
"… season 4 episode 47370355".
2026-10-31 14:49:12 -07:00
dank 9c25b1cccb Merge pull request 'feat(media): the operator's map gives a folder the id its name cannot reach' (#106) from feat/media-tvdb-id-map into main 2026-10-31 13:15:37 -07:00
Frank 7aa32d1b64 feat(media): the operator's map gives a folder the id its name cannot reach
Tvdb titles a record in the show's own language and keeps the English name in
aliases the name search never reads, so the rule that accepts only tvdb's own
spelling of a query can never resolve them: ブラッククローバー (331753) holds no
alias at all, and the bare alias "One Piece" belongs to 392276, the 2023 live
action, not to the 1999 record (81797) that 1,072-file pack is. Seventeen pack
folders on the array — 3,507 files — are in that class.

media-shows.py now reads scripts/media-shows-tvdb-ids.tsv by default: one line
per pack folder, folder <TAB> tvdb id <TAB> note, every id read from tvdb's own
record. The map outranks a provider id, because a box set's provider id is the
server's own guess at a folder name, and it merges the release folders that are
one show. Everything it touched goes to the exceptions: every folder it named,
every row that matched no folder, every disagreement with a provider id, every id
tvdb never answered. The file is read fail-closed — an id that is not a number,
or one folder named twice, stops the plan, because a typo there renames a pack of
1,268 files to the wrong show.

scripts/media-shows-tvdb.py is the sheet the map was built from, promoted out of
/tmp so the next run is reproducible: it reads a plan's mkdir rows, asks tvdb
about every destination folder carrying no [tvdbid-], and grades the answer
EXACT / PUNCT / NEAR / NONE by how much was guessed to reach it. Acceptance is
unchanged from the passes it replaces — bare-form equality plus the year-span
guard — and that gate is what rejected three confident picks: Fairy Tail's 2024
sequel (410031), the 2026 Hokuto no Ken remake (465216) and Mugyutto! Black
Clover (378762) offered for Black Clover. The sheet is for the operator; the map
is what the planner reads, and only after he approves the rows.

Tests: seven for the map's behaviour — rename to the record it names, two release
folders becoming one series, precedence over a provider id and the line stating
the disagreement, an id tvdb never answered, a row whose folder is gone, a
malformed id refusing the run, a folder named twice refusing the run — plus one
that the shipped file parses and never stops a plan.
2026-10-31 13:10:32 -07:00
dank 576c3421b8 Merge pull request 'feat(media): the folder carries its tvdb id; specials and year-seasons resolve' (#105) from feat/shows-ids-and-specials into main 2026-10-31 12:56:35 -07:00
Frank e4656cfd7a fix(media): a tvdb episode page is not the episode list
Tvdb's default episode query answers at most 500 rows and states no total.
One Piece (1999), Crayon Shin-chan (1992) and Detective Conan (1996) each
answer exactly 500 while holding over a thousand, so the contradiction rule
added in the previous commit read every long-running show as contradicted:
the pack books more episodes than "the authority holds", and the rule that
exists to stop a wrong anime title would have stripped the right one from
every file of the season.

A capped list now records that it is capped (`tvdb:capped:<id>`, so a warm
cache answers the same question a cold one did) and contradicts nothing;
titles for the episodes the page does hold still ride.

Test: a 500-row page with a 501-episode pack keeps ` - Episode 1` and lists
no contradiction. Red before the change, proved in a copy of the tree with
the cap raised past the fixture.
2026-10-31 12:40:49 -07:00
Frank acab4cef22 feat(media): the folder carries its tvdb id; specials and year-seasons resolve
Three defects in the TVShows planner, all three visible in the applied plan:

- A series folder named a show by words alone. Jellyfin's Tvdb agent and
  Plex's scanner both read `[tvdbid-NNNN]`, so the resolved id now rides the
  series folder — and only the folder: a file name gets 180 characters and
  the episode title needs the room.
- 1,412 files parked because their provider row said `season 0`. Season 0 is
  a legal season; Jellyfin's contract is `Season 00` for specials.
- 1,033 files parked because Jellyfin's date matcher writes the AIRDATE YEAR
  into the season column. Such a row is now renumbered by the title tvdb
  states for the series' id, read backwards off its own episode list. A
  title tvdb gives to two episodes names no episode, and that file parks
  with the reason it always carried.

A season tvdb's own episode list CONTRADICTS — a release pack books more
episodes there than the authority holds, the anime numbering hazard — keeps
its rename and loses the tvdb title, because the slot its number points at
is a different episode. Those series are listed in the exceptions file,
which is where the operator reads them.

Tests: six through the fixture (five red before the change; the ambiguous
title is a guard that the recovery must never guess). Full bats suite green.
2026-10-31 12:27:15 -07:00
dank 04802b3e48 Merge pull request 'fix(media): verify read the library the way the servers write it' (#103) from fix/media-verify-wal-read into main 2026-10-31 12:12:23 -07:00
dank 44e55ad172 Merge pull request 'feat(media): tvdb names the show — titled episode files, id-backed series folders, park outside' (#102) from feat/tv-tvdb-naming into main 2026-10-31 12:12:15 -07:00
dank 132805a25a Merge pull request 'fix(dankserver): mount /mnt/entropy from its LUKS mapper at boot' (#104) from fix/dankserver-entropy-automount into main 2026-10-31 12:04:08 -07:00
Frank 2406641bc6 fix(dankserver): mount /mnt/entropy from its LUKS mapper at boot
The crypttab entry named the md RAID array UUID (61bd6eaf-...) where a luks
entry takes the LUKS container UUID (cd86c728-...), so
systemd-cryptsetup@entropy.service could never produce /dev/mapper/entropy and
the fstab entry could not resolve either; the volume only appeared when a
desktop session let udisks2 unlock it, which kept the broken boot path hidden
for a year. The keyfile was wrong as well: 27 bytes, i.e. the 26-character
passphrase plus a trailing newline, and cryptsetup hashes the file whole.

Records the host config as dankserver/host/{crypttab,fstab} (the dankclaw/host
precedent), rewrites the Entropy Volume page to own the chain, the UUID law and
the keyfile contract, and comments out the now-unmountable /dev/md0 line.

Proven on the host: /etc/{crypttab,fstab} byte-identical to the repo copies,
reboot into 7.0.0-42-generic mounts /mnt/entropy from /dev/mapper/entropy, all
12 containers returned, frank-archon back to 141 t/s (was 6.7 t/s on a wedged
amdgpu queue). ./tests/run.sh: ALL TESTS PASSED (749).
2026-10-31 11:56:39 -07:00
Frank dbcf8a2c35 docs(media): the verifier prints the four verdict classes itself
The taxonomy was written as something an operator assembled by hand; the tool now
emits `on disk at dst` / `REAL` / `gone` lines, which is the difference between
reading `missing from the DB: 1851` as 1851 lost albums and reading it correctly.

cwd: /home/blank/.omp/wt/wt-20261030-163847-6d1fc1f · session: wt-20261030-163847-6d1fc1f
2026-10-31 10:29:33 -07:00
Frank a5a9c04da2 feat(media): the verdict splits what the database alone cannot
"No row at the destination" was one number, and it conflated three different
things: a sidecar the server never indexes, a file a later pass renamed again,
and a file that never left its source — the last one being actual damage. The
runbook already named the four classes; the tool never computed them, so an
operator reading `missing from the DB: 1851` had no way to tell a booklet.pdf
from a lost album and might have run a rollback over a false alarm.

`missing` rows are now stat'd on both paths and reported as
`on disk at dst, no row names it` / `REAL — file still at src` /
`gone — at neither path`, with offenders listed in that order. A row still
naming `src` stays a separate line: that is the server's own stale pointer, a
phantom, and it costs a re-scan rather than a restore.

Two new tests (REAL, and gone-vs-sidecar) plus two existing layout assertions
loosened from line positions to content. 14/14 in tests/media_verify.bats.
2026-10-31 10:28:44 -07:00
Frank c95ae20bfb Merge main into feat/tv-tvdb-naming: the arms moved to their own pages
main split the music and audiobook arms out of Media Relayout.md and put a
locator in their place, so the branch's trims inside those sections now live on
the two child pages, and the parent keeps main's locator. The two Jellyfin laws
the branch moved to Jellyfin.md stay moved; the sweep law keeps the branch's
wording ("until the next sweep"), which is the one the next line depends on.

Full suite: 758 tests, all green.
2026-10-31 10:22:34 -07:00
Frank a0dfa61623 docs(media): the TVShows id-map proposal — where it lives and what may be applied unreviewed 2026-10-31 10:18:16 -07:00
Frank 4b67c3c4d8 fix(media): the census read Plex the same wrong way
The two Plex arms of media-census.py — `plex` (the movie planner's title
authority) and `audiobooks` (the chapter rows the audiobook plan files by) —
opened the library `mode=ro&immutable=1` while the Jellyfin arms already used
plain `mode=ro`. Same defect as media-verify.py: an uncheckpointed row of a
running scan is invisible, so a just-scanned item is simply absent from the
authority file a plan is built from, and the pass cannot say why.

Both now go through open_plex_ro(), which tries plain ro first and keeps
immutable only as the fallback for a database whose -shm cannot be read. The
new audiobooks test leaves a chapter row committed-but-uncheckpointed and
asserts the export still carries it; red before the change, green after.
2026-10-31 10:13:17 -07:00
Frank 7ee831adee docs(media): what a verification verdict actually means, and what a rollback can undo
Three things the 2026-10-31 five-library relayout taught, written where the
next operator reads them:

- A verify run must open the servers' databases plain `mode=ro`. Both keep
  their libraries in WAL mode; `immutable=1` skips the write-ahead log, so a
  scanner's own committed rows read as missing and the read can fail as
  "database disk image is malformed". The census page claimed immutable — now
  it says what the code does.
- "No row at the destination" splits into four classes, and only REAL (no row
  at dst AND the file still at src) is the relayout's own damage. Phantom rows
  naming a pre-move path are normal behaviour for both servers. And the owning
  server decides whether a pass can convict: Jellyfin holds no audiobook
  library, so that journal verifies against Plex or it proves nothing.
- A chain of passes rolls back newest journal first, or the tool refuses; the
  refusals measured that day are the tool working, not failing.

The music and audiobook arms moved to their own pages (progressive disclosure;
the parent drops 175 lines and stays the owner of the shared gates, the
executor, the journal law and the rollback). Stale per-run counts destroyed:
episode counts, byte sizes, move tallies for a pass that already happened.
2026-10-31 10:13:11 -07:00
Frank afad21dda5 fix(media): verify read the library the way the servers write it
A verify run beside a running scan reported rows the scanner had just
written as MISSING, and could fail outright as "database disk image is
malformed". Both libraries live in WAL mode, and the verifier opened them
with immutable=1, which tells sqlite the file never changes: it skips the
write-ahead log, so every committed-but-uncheckpointed row is invisible.
134 MB of jellyfin.db-wal was riding past the verifier's eyes during the
2026-10-31 rescan.

Open plain mode=ro first — sqlite then reads the WAL through its -shm —
and keep immutable=1 only as the fallback for a database whose -shm
cannot be read. The bats fixture leaves a journaled move uncheckpointed
and proves the verifier still counts it landed.
2026-10-31 10:00:11 -07:00
Frank f430e2264d docs(plex): the scan recipe kept its own promise about the token
The block said the token rides stdin and never argv, then handed it to curl as
`-H "X-Plex-Token: $T"` — world-readable in /proc/<pid>/cmdline. It now writes
a mode-600 curl config inside the container, which is also how the overnight
rescan was actually issued (five sections, all accepted). Two truths the old
block got wrong are stated: a section answers 200 with a Directory body when
Accept: application/json is sent, and a backslash-newline inside a quoted
printf format is literal, so the config builder writes two printfs.
2026-10-31 09:50:22 -07:00
Frank 82a2183d40 docs(media): a film handoff needs the file's own words and year
The provider's first hit for a salvaged query is a near miss, and the page said
only that a provider must name the file. The bullet now states the two checks and
the two real misfiles they exist to stop.
2026-10-31 07:39:45 -07:00
Frank c70f9b3f6d fix(media): a film leaves the TV library only for a film it actually names
The first tvdb-grade plan handed 12 files to the movie library, and two of them
were wrong in a way the exceptions file could not show: an episode of the
documentary series `Russia 1985-1999: TraumaZone` was renamed to the 1963 Bond
film `From Russia with Love`, and `Gundam 00 Special Edition III - Return The
World` became `Special Edition I - Celestial Being`.

Two guards, each alone insufficient:

- `title_of` salvages a short query from a long release name ("russia", "mobile
  suit gundam 00"), and tmdb() took the FIRST search result. The call site then
  compared the provider's words against that salvaged query, so any title merely
  CONTAINING the query passed. The provider's title must now be a run of words the
  FILE itself states: `Special Edition I: Celestial Being` is not in a filename
  that says `Special Edition III - Return The World`.
- tmdb() read the year from `year`/`first_air_date`, which the MOVIE search does
  not return — it returns `release_date`. Every film answer therefore arrived
  year-less and the year guard at the call site could never fire. That is how a
  1963 answer survived a filename stating 1991.

Tests: a provider answer the file does not name parks, and nothing lands in the
movie library; an answer from another year is refused. The year-from-the-wire half
is proven by the live re-plan, which now dates every movie answer.
2026-10-31 07:38:54 -07:00
Frank 32b8901bff docs(media): the pile is matched with its underscore stripped
The server drops the leading underscore from the hidden folder's display name, so
the park-echo rule that compares spelled names lets the review pile survive as a
visible series. The law now says which spelling is compared; the mechanism stays
with the guard's docstring.
2026-10-31 07:28:22 -07:00
Frank 6d03bbc8e7 fix(media): the review pile is not a series, and a folder carries the year once
Two destination defects the first tvdb-grade plan exposed in its own output —
both visible by reading the plan's folder names, neither caught by any gate.

1. Jellyfin sanitizes a hidden folder's display name: `_unsorted` comes back as
   `unsorted`. The park-echo guard compared the row's series against the folder
   with the underscore, so 1,091 pile rows survived as a real series and the plan
   renamed 617 files out of the pile into a VISIBLE folder named `unsorted` —
   inventing the junk show the guard exists to prevent. The comparison now strips
   leading underscores from both sides.

2. TVDb hands back some series with the year already on the name (`Acapulco
   (2021)`). label_of appended the year anyway, producing `Acapulco (2021)
   (2021)` — a folder no matcher and no human reaches for twice. A name that
   already ends in `(YYYY)` is left alone.

Both are covered by fixtures: a pile row whose series is the sanitized spelling
must park, never file under the pile's name; a provider name carrying its year
produces one year in the path and the plan contains no `(YYYY) (YYYY)` anywhere.
2026-10-31 07:27:47 -07:00
Frank 32b58dd787 fix(media): TMDb's v3 key travels as a query parameter; Bearer is a JWT only
The first tvdb-grade TV plan built 49,771 moves with not ONE TMDb answer: every
search and every external_ids call came back 401. The credential on this box is
the v3 key (~/.config/secrets/tmdb, `api_key=`), and TMDb takes a v3 key only as
the `api_key` query parameter — a v4 request token is the only thing the Bearer
header accepts. Measured on the same credential in one pass: the query form
resolves `New Girl (2011)`, the Bearer form raises HTTP 401.

tmdb_request() now picks the transport from the shape it was handed (three
dot-separated segments = a JWT → Bearer, anything else → query parameter), and
the old header comment's reasoning is corrected: a token in argv is
world-readable in /proc/<pid>/cmdline, but this URL is built inside the process
and never reaches argv, so the query form leaks nothing.

The failure mode that let it look like a finished plan is fixed with it. The
401s were stderr-only, so the summary printed a confident move count while every
series folder still carried the metadata provider's folder-echo name — the exact
junk this contract exists to undo. Provider health is now part of the summary
line and of the exceptions file, and a test covers it offline with
`env -u TMDB_API_KEY`: a plan that asked and could not ask says NOT
provider-complete in both places, and still writes the moves it can justify,
because the answers that did arrive are worth their rename.
2026-10-31 07:16:46 -07:00
Frank 0a365274b6 docs(agent): a PR comment posts to the issues endpoint, and an empty jq body is silent
Measured while commenting on PR 102: POST /pulls/102/comments answers 405 to a
plain body (it is the review-comment store); the conversation thread is
POST /issues/<PR-number>/comments, which answers 201 and returns an html_url on
the PR. Gitea maintains one issue index, so the PR number is the same number.

The trap that hid it: building the body with jq --rawfile b /dev/stdin inside a
command substitution feeds jq nothing (the substitution does not inherit the
heredoc), Gitea answers 422, and jq -r .id on the error prints nothing — a
silent no-op that looked like a posted comment. The body goes in a file.
2026-10-31 07:11:13 -07:00
Frank e531e90278 docs(media): a scan request belongs to the server's own page
Media Relayout.md hit the vault's 500-line gate the moment the tvdb contract
landed, and the fat in it was not the new text: the Plex and Jellyfin scan
blocks (35 lines) sat inside the relayout procedure even though they are the
way you talk to those servers after ANY change, and two Laws bullets repeated
the same facts a third time. Both blocks now live on [[Plex]] and
[[Jellyfin]]; the relayout page keeps the two laws an operator needs at that
moment — 204 is acceptance and anything else queued nothing, and one accepted
refresh is the whole procedure, because a second moves `Scan Media Library` to
`Cancelling` and every later 204 is discarded.

Same pass, the dead numbers the vault's own law says to destroy: an applied
music pass's date, an applied audiobook pass's move/park counts, and the 146
sweep stragglers of one specific first sweep. The laws they taught stay.

490 lines, `bats tests/docs_completeness.bats` 6/6 (length violation gone,
link integrity clean).
2026-10-31 07:10:28 -07:00