Commit Graph
15 Commits
Author SHA1 Message Date
dank 8ae4d1c710 chore: stop tracking .AGENTTRASH — the purger deletes tracked files, dirtying every host
The deletion policy moves in-repo deletions into .AGENTTRASH/ beside the item,
and .gitignore line 25 already declares it ignored. But 37 retired payloads were
committed before that rule existed, and gitignore never applies to tracked paths.
Consequence measured 2026-10-31: Dankserver's deploy checkout reported 37 tracked
deletions — exactly every tracked trash file — because the approved purger had
done its job. Any host that purges is permanently dirty, and a future pull that
touches one of those paths aborts mid-deploy with 'local changes would be
overwritten' while the tree is in fact clean of real edits.

Untrack the four dirs; the bytes stay on disk and in history (they always were
recoverable from git, which is what the trash policy's 'temporary hold, not an
archive' means). tests/deployment_scoping.bats asserts the staged chat template
exists on disk — it still does, so the seam is untouched.
2026-10-31 18:46:53 -07:00
dank 183a6d5442 feat(fleet): give the media servers an I/O share that actually exists
A Plex section scan walked 50k files at ~850 r/s and nothing bounded it.
`IOWeight=` in a quadlet looks like the fix and does nothing: systemd delegates
`cpu memory pids` to user@.service and NOT io, and it drops a resource line it
cannot honour without a word — the container boots healthy and its cgroup has no
io.weight file at all. Same family as RequiresMountsFor= (inert in the user
manager) and ConditionPathIsMountPoint= (fail-dead).

Place the two system drop-ins that make the controller reachable (user.slice
IOAccounting=yes puts io in user.slice/cgroup.subtree_control, measured
immediately; user@1000.service DelegateControllers gains io and lands at the
next reboot, because restarting the user manager would kill every rootless
container on the host), then ask for the share: IOWeight=10 on jellyfin and plex.
10 is a convention, not a measured optimum — proportional share, only bites
under contention, only requirement is far below the default 100.

ensure_user_io_delegate places both from git and prints NOTE: io is not
delegated … inert until the next reboot, so the quadlet line never reads as live
config. Verified on the host: subtree_control flipped to `cpu io memory pids`,
the drop-in is attached and parsed (systemctl cat), and the podman 5.7 generator
emits IOWeight=10 for both units (bare key; UnitConfigItem= not needed).

Two limits recorded instead of hidden: a container's weight covers its whole
cgroup (scan and playback share it), and the comparison happens inside
user.slice, so it cannot push a rootless unit below a root unit such as
frank-archon — that pairing is user.slice's own weight, a separate call.

Docs: Quadlet Lifecycle 4b (the three-command read of the chain), Pi-hole (all
rootless containers share one 10.0.2.100, so one chatty container throttles DNS
for the host — 8 events today), E2E Suite gotcha 7 (the gate depends on the
runner's resolver: 21 reds on a lost wired link, 37/37 minutes later).

Seam: an in-repo IOWeight= may only exist when both drop-ins exist AND the
deployer places them. ./tests/run.sh: 789 ok, exit 0.
2026-10-31 18:42:27 -07:00
dank e82c8b2d06 fix(fleet): refuse a container start until the mount it binds has answered
Dankserver rebooted with the array's mount and the media containers racing. A
rootless quadlet cannot order itself against the system's mnt-entropy.mount, so
`RequiresMountsFor=/mnt/entropy` is inert in the user manager: podman resolved
the bind against the pre-mount directory, Jellyfin and Plex came up blind, and
Jellyfin read an empty library and began `Removing item, Type: "Season"` by the
thousand. Copyparty had no guard at all.

`scripts/wait-for-mount.sh` is the gate: it waits for every named mountpoint,
then refuses by name. `ConditionPathIsMountPoint=` is NOT the fix — measured on
the host, an unmet condition logs `skipped, unmet condition check` with
NRestarts=0 and no start attempt ever follows, which is fail-dead, not
fail-closed. Jellyfin and Plex swap the condition for the gate; Copyparty gets
one; the law is recorded in AGENTS.md and the view page.

Verified on the host: the 5.7.0 quadlet generator accepts ExecStartPre in
[Service] (exit 0, all three units generated, gate present), the gate waits for
a mount that appears mid-window and proceeds, and it refuses a plain directory
after its window naming the path.

tests/media_overlay.bats pinned the old mechanism by name; its intent — both
media servers refuse to start without the view — now rides the gate.
2026-10-31 18:17:02 -07:00
dank f7083f175d perf(media): the apply fsyncs the journal once per batch, not per move
The 58k-rename apply on the spinning array spent most of its wall time in
fsyncs: one durability flush per rename, and a quadratic `Journal.already()`
scan over a growing record list on top of it. Both are the journal's, not the
renames'.

- `Journal` indexes move intents by (src, dst) so `already()` is O(1).
- `--batch N` (default 64) writes N intents, fsyncs once, then performs those
  N renames. The invariant is unchanged: no rename runs before its own intent
  is durable, so a crash mid-batch replays the batch and resumes.
- Tests: a mid-batch kill loses nothing and the rerun completes; batching
  changes no final path and no journal line.
2026-10-31 18:04:37 -07:00
dank 96944f0b17 feat(media): media-verify.py replays a journal against the servers' own tables
A scan answers 204 whether or not it found anything, so the fleet's documented
proof — every move row has a row at its dst and zero rows still name its src —
was a hand-written sqlite query every time anyone needed it. That is the step a
half-finished relayout is judged on, so it is now a command with an exit code.

It encodes the two doors that bite: Jellyfin stores the container-side path
(/data stands for the share) while Plex stores the host path verbatim and hides
deleted scans behind deleted_at on two tables. The report breaks the missing rows
down by extension, because a .cue or .log has no row to land in and must not read
as a lost album; --ignore-ext keeps those out of the verdict.
2026-10-31 01:02:48 -07:00
dank 33f97cd3b3 docs(media): the TV library arm gets its own journal, and its plan lives on the array
Two relayouts writing one journal cannot be rolled back apart: the mover
(Movies -> TVShows) and the library re-filer both pointed at
media-relayout-tv.jsonl. The re-filer now writes media-relayout-shows.jsonl,
and its plan/exceptions move off /tmp, which a reboot eats and which is not a
record.
2026-10-31 00:55:32 -07:00
dank 254fb41fd0 docs(media): the music relayout is mid-flight, and the array takes one writer
The page said the music plan "was applied" while 62k of its 131k journal rows
were still being written; a runbook line that gets believed is worse than none.
Also lands the single-writer law the audiobook handoff exists to honour, and the
durable plan path (/tmp is not a record).
2026-10-31 00:40:30 -07:00
dank 6bf70c6a64 fix(media): a shelf needs a name, so a mistagged chapter number parks
Plex's artist row for 272 chapters is a chapter number the ripper put in the
artist tag (0.4, 31, 05), which would have filed them under a shelf called 0.4.
A narrator is a person, so it must contain a letter; the same test is deliberately
not applied to a book title, which may honestly be 1984.
2026-10-31 00:34:40 -07:00
dank f1b8b60817 fix(media): a tag-scan placeholder is not a book identity
The first real plan filed 1,106 chapters from 14 unrelated top-level folders
onto one shelf, because all 1,106 carry the album tag [Unknown Album] and the
same artist tag. Plex's album title is the authority for the shelf only when it
names a book; a placeholder names nothing, and unrelated books share it. Those
chapters now park with the path they have today, timestamp-glued variants
included; Various Artists stays a legitimate multi-cast narrator.
2026-10-31 00:31:30 -07:00
dank 4ae2bde9ab docs(media): fit the audiobook section inside the OKF caps, refresh the index
The linter caps a page at 500 lines and a description at 120 chars: the section
lost four lines of prose (no law went with them) and the description lost its
restatement of the format names. gen-vault-index.py regenerated the locator row.
2026-10-31 00:25:58 -07:00
dank ceb5617a42 docs(media): the audiobook arm, and the music plan's applied state
One page owns the relayout facts: the new section carries the Plex-only reader,
the shape, the four commands, and the two laws a future operator would otherwise
re-litigate (never renumber a chapter; the shelf is the narrator). The music
section said its plan was NOT applied — it was applied on 2026-10-31, and a stale
line in a runbook gets believed, so it now names the journal and the rollback.
2026-10-31 00:24:50 -07:00
dank a7ceda95fd feat(media): the audiobook relayout, filed by the book Plex matched
The audiobook tree is the one library Jellyfin does not read at all and Plex
reads completely: 58,975 chapters matched into Artist/Album/Track, where the
album is the BOOK and the book's single artist row is the NARRATOR. That is the
authority the new planner files by, never the collection folder a pack made.

  Audiobooks/<Narrator>/<Book> (<year>)/<the file's own name>

Two laws are the opposite of the music tool's, both forced by the rows: files
are never renumbered (72% of the books carry chapters with no usable index, and
playback order comes from the embedded track tag either way, so an invented
number can only lie), and the shelf is the narrator, not the author (every book
has exactly one artist row; the author survives in 10% of album titles and in no
column Plex stores, so one rule beats a shelf half authors and half narrators).

media-census.py gains the audiobooks export: the chapters Plex matched under a
root, host paths straight through, nothing outside the root and nothing Plex
deleted. 22 tests in tests/media_audiobooks.bats pin the export's join and the
plan's output.
2026-10-31 00:24:45 -07:00
dank a32361ee5e Merge remote-tracking branch 'origin/main' into feat/jellyfin-provider-keys 2026-10-31 00:10:58 -07:00
dank ce6b3b1d3b docs(media): how to prove a relayout landed, read from both servers' tables
A scan answers 204 whether or not it found anything, so proof is the journal
replayed against Jellyfin's BaseItems.Path and Plex's media_parts.file: every
move has a row at its dst, no row names its src. Records the four wrong doors:
no sqlite3 binary on the hosts or in either container, Jellyfin's single
BaseItems table (no MediaItems), its container-side /data path prefix, and
Plex's HTTPD closing on ?path= / on /search matching titles not paths.
2026-10-31 00:10:50 -07:00
dank 272e06daf6 feat(media): the movie-library relayout toolchain — census, plan, executor
The movie library on the entropy volume is release-folded: the video inside
`Some.Movie.2019.1080p…-FOO/` is named after the release, and Jellyfin matches
a movie by folder and file name, so the noise cost posters, extras and versions.
Three scripts own the fix, and nothing else in the repo moves a media file.

  * media-census.py  the read-only inputs: a stat pass, an ffprobe header pass
                     (nice'd, so it is safe beside a live endpoint), and the
                     Plex metadata cache read that outranks any filename guess
  * media-plan.py    the Jellyfin-format planner: one folder per movie, the
                     video named exactly like the folder, extras in the
                     canonical subfolders, copies of one film as versions. It
                     opens no media file, so a whole-library plan costs seconds,
                     and it ends in gates — bad-names, occupied, double-claim,
                     left-unmoved, unplanned — that must all read zero before a
                     human points the executor at 24 TB
  * media-apply.py   the executor: rename(2) only, the intent fsynced to a
                     write-ahead journal BEFORE the move it authorises, so an
                     interrupted run is finishable and every run is undoable.
                     Refusals are all-or-nothing and it never overwrites a file

Laws the fixture pins (tests/media_relayout.bats, 16 tests, each proven red
against a reverted copy of the rule it guards):

  * a Plex year of 0 means "indexed, never matched" — trusting it produced
    folders "Title (0)": items with no year, no poster, no identity
  * a Plex title carrying "/" is sanitised, or "V/H/S/2" became a five-level
    tree neither matcher reads as a movie
  * a folder inside a movie's folder is bonus territory even when its name is
    misspelled ("Exrtas"), unless the file's own name carries another year
  * matching words split on everything but a letter or digit — the earlier
    tokenizer left "(1960)/extras" glued, so a film could not be found by the
    folder holding it and one box set's featurettes drifted into a neighbour
  * a name with no words in it is bonus material, never a movie folder named 1

Docs: the runbook page owns the procedure, the gates and the rollback; AGENTS.md
links it from the runbook index and the summary table.
2026-10-30 19:33:44 -07:00