Compare commits

..
Author SHA1 Message Date
dank fab17129e2 feat(media): dedupe + relayout campaign tools for Audiobooks and eBooks
media-ab-reorg.py: planner for the ABS layout {Author}/{Series}/{Seq -
Year - Title {Narrator}} — claims one book folder per DB item (tracks
of duplicate recordings must not interleave), gates any book folder
that would nest above another (ABS's swallow defect), routes comic and
ebook-only items to intake, holds dedup losers under the campaign dir.
media-ab-verifydups.py: full-blake2b confirmation of size+head/tail
collision survivors. media-ebook-dedupe.py: dedupe the eBooks pile
against the Calibre library (identity key title-core+first-surname,
OPF metadata authority) and emit the calibredb ingest list.
abs-post-reorg.py: server-side ABS driver (cover export, DB-row wipe —
never hard=1 — scan, status) with credentials out of argv.
2026-11-04 14:45:29 -08:00
dank f46bf60939 fix(alloy): make the copier's refusal case actually refuse
The engine-rejects test corrupted its own cfg file, but the copier always
renders from the repo config, so the gate validated a good file and
installed it; the test only surfaced where it was written, not where it
broke, because '!' suppresses errexit. Add an ALLOY_CONFIG_SRC seam
(alongside the *_CONF_ROOT seams) so the refusal case feeds the copier
the broken config it claims to test, and pin the repo-relative sed paths
to $REPO_ROOT — run.sh cds into tests/, so the relative path was dead
on any box that has the alloy image.
2026-11-04 14:45:29 -08:00
dank b3eb9c6314 Merge pull request 'feat(homepage): Audiobookshelf's own mark, and frank-tts in the Model Information block' (#150) from feat/homepage-abs-icon-and-tts-card into main 2026-11-04 13:50:50 -08:00
dank a3ddb4c65a Merge pull request 'fix(caddy): the *arr stack and qBittorrent are LAN-only at the edge' (#151) from fix/arr-lan-only into main 2026-11-04 13:49:33 -08:00
Frank b5173033b6 fix(caddy): *arr stack and qBittorrent are LAN-only at the edge
A missing Porkbun record was never the gate: Caddy issues one certificate per
name and every one is in public CT logs, so the *arr hostnames were discoverable
and the edge accepted them. Each of the five vhosts now carries a remote_ip
source gate (LAN proxies through, everything else gets 404) - the posture the
Sonarr comment already claimed. AGENTS.md records the LAN-gate shape as the
required form of internal-only, and Caddy.md records why DNS secrecy is fiction.
2026-11-04 13:45:25 -08:00
Frank 53e4a777b4 feat(homepage): Audiobookshelf's own mark, and the voice endpoint in the model block
Audiobookshelf carried the improvised "AB" hex monogram — the fallback for
apps with no icon. Simple Icons carries its brand mark, so the card now
uses /img/apps/audiobookshelf.svg the way Plex and Jellyfin use theirs; the
monogram went to .AGENTTRASH.

frank-tts (Qwen3-TTS 1.7 B base talker on the gfx1151 iGPU) was an
admin-only status tile, so the household read about five cloned voices in
/docs and saw no sign of the door that speaks them. It is now a Model
Information card on the user page beside the three chat models, carrying
the ask-Frank permission note and a privacy note about cloned voices.

miCard grew the shape that requires: art falls back to the card's own `art`
when a card has no StarCraft `unit`, the accent falls back to `.mi-tool`,
and a `tagline` with no `sfx` renders as text instead of a play button that
plays nothing. No audio asset ships with the card on purpose: every pack on
that endpoint is a replica of a real person's voice, so a sample clip would
put a synthetic person's voice in the repo and on the page.

Seams: the model-card contract splits unit cards (equivalence, supervision,
playable voice line) from tool cards (permission note), and the voice
stack's homepage seam now pins category=user with the reason.
2026-11-04 13:39:37 -08:00
dank 52d6da4f49 Merge pull request 'docs(caddy): the exposure gates measured, and the guarded set published' (#149) from docs/caddy-exposure-gates into main 2026-11-04 13:36:24 -08:00
Frank d138788411 Merge origin/main into docs/caddy-exposure-gates 2026-11-04 13:32:20 -08:00
Frank 6c169fc4ad docs(caddy): record the two exposure gates and the published guard set
Measured 2026-11-04: the router's 80/443 forward plus Caddy's vhost list are the
only gates — a public A record buys discoverability, not access — so internal-
only means an absent record. Publishes the outstanding fact: the whole
authentik-guarded set now carries a record; the exempt self-authenticating names
and the legacy redirects do not. Adds the dynamic-WAN caveat and the 1800 s
negative-cache trap that makes a fresh record look broken.
2026-11-04 13:32:17 -08:00
dank e6a85176c4 Merge pull request 'feat(homepage): give the not-open-yet apps their own section, below AI' (#148) from feat/homepage-wip-section into main 2026-11-04 13:20:57 -08:00
dank 9ff7ce6a12 Merge pull request 'docs(frank-archon): record the crash signature — clean-looking exit, silent engine' (#147) from docs/frank-archon-crash-signature into main 2026-11-04 13:19:50 -08:00
Frank f25be876fb docs(frank-archon): record the crash signature — clean-looking exit, silent engine
A 2026-11-04 crash left no trace in the engine's own log and reported
Result=success with NRestarts=1: the unit's ExecStart ends in 'exec podman
wait', whose exit status is 0, while the container's real code is printed to
stdout. The only copy of the reason was the client-facing error text. Both
traps now live in the endpoint page's Laws, with the observed trigger shape
(concurrent triple-digit-thousand-token prefills) and the instruments that
ruled out OOM, GPU hang, MCE and swap pressure.
2026-11-04 13:18:43 -08:00
Frank bad78079d7 feat(homepage): the not-open-yet apps get their own section, below AI
Calibre-Web, RomM and Seerr sat inside "Media Library Apps" carrying a WIP
badge and a "not open to family users yet" sentence each, so the household
grid mixed doors they can use with doors they cannot. They now render under
their own "Work in progress" heading, placed after AI on the user page, and
the copy drops the restated "Work in progress:" for the ask-Frank line the
section heading no longer needs to repeat.

- app.js: SECTION_ORDER.user gains "Work in progress" last.
- services.json: the three cards' section, plus one-line copy trim.
- homepage_config.bats / homepage_pages.bats: the section contract now
  includes the new heading and pins the new order (the old test pinned AI
  last, which this change deliberately overturns).
- Homepage.md: the classification list and the SECTION_ORDER/wip contract
  lines catch up (the user order in the page was already stale, and "down"
  was an undocumented wip value).

The admin page is untouched: its WIP cards (Gluetun, mariadb, valkey,
Postgres) are plumbing inside Infrastructure, not applications.
2026-11-04 13:09:16 -08:00
dank fee8121d47 Merge pull request 'docs(openwebui): re-key the model rows at swap time; default model is frank-archon' (#146) from docs/openwebui-model-rows-default into main 2026-11-04 13:08:52 -08:00
Frank 1b5c11fb60 docs(openwebui): re-key the model rows at swap time; default is frank-archon
The picker's decoration rows carried two payload-swap drifts: frank-dragoon
still named halogen 0.16.2 (live 0.18.20) and frank-zealot build 10777 /
3x262,144 / MTP off (live build 12118 / 3x200,704 / draft-mtp armed). Worse,
their params.context.length values had swapped, so OpenWebUI advertised more
window than the zealot slot grants and gave up 61k tokens on dragoon. Live
rows re-keyed through the admin API and ui.default_models set to
frank-archon; the swap procedure in AGENTS.md now names the re-key step.
2026-11-04 13:06:54 -08:00
dank cae283047f Merge pull request 'docs(authentik,openwebui): the roles claim mints the OpenWebUI role; jason joins the census' (#144) from docs/authentik-jason-provision into main 2026-11-04 12:57:07 -08:00
dank 51007ac604 Merge pull request 'fix(homepage): Jellyfin takes the family login; document the voice models' (#145) from docs/homepage-jellyfin-login-and-tts into main 2026-11-04 12:56:44 -08:00
Frank f7c41966b0 fix(homepage): Jellyfin takes the family login; document the voice models
Jellyfin authenticates against Authentik over LDAP, so telling the household
that it "will not take the family password" sent people to the wrong door. The
tile's provider id (alex -> LdapAuthenticationProviderPlugin) and a live bind +
memberOf search against the running outpost are the proof; six places repeated
the claim, and all six now say to sign in with the franksdank.cloud username and
password. Added the failure mode that actually happens: a password that works on
the homepage but is refused here means the name is not in jellyfin_users yet.
Plex and the e-book door stay the two genuinely separate accounts.

Marreta's tile loses its VPN/headless-browser mechanics and keeps one honest
clause: "does not work on all sites".

The models page gains the TTS pair the household keeps meeting: frank-tts
(Qwen3 talker + 12 Hz codec on the iGPU, five cloned voices, live list at
/v1/audio/voices) and Kokoro (82M, stock presets, runs in the user's own
browser) - which is why the AI chat's speaker list never matches the server's.

Verified in a browser against an isolated copy of the tree: both tiles render
the new copy, #models carries both tables, zero broken images, and every
retired phrase is absent from the rendered text. ./tests/run.sh: 841 pass.
2026-11-04 12:49:42 -08:00
Frank 25114ca622 Merge remote-tracking branch 'origin/main' into wt/20261104-104444 2026-11-04 12:42:19 -08:00
Frank a07f492b89 docs(authentik,openwebui): the roles claim mints the OpenWebUI role
Provisioned jason (pk 24, human-user template) and drove his first SSO
login in a clean browser: the row appeared as role='user' with chat and
wilwheaton TTS working, no manual step. The pending-then-flip law is
therefore wrong for a claim-carrying account - ENABLE_OAUTH_ROLE_MANAGEMENT
+ the roles scope decide it, and 0.12.5 refuses a claim-less login with 403
instead of falling back to ui.default_user_role. Census gains jason.
2026-11-04 12:42:19 -08:00
dank 9961ed62c0 Merge pull request 'docs(jellyfin,authentik): the LDAP admin filter is inert; the auth door lives in Users' (#143) from docs/jellyfin-ldap-sso-laws into main 2026-11-04 12:27:00 -08:00
Frank 70be24e9d5 docs(jellyfin,authentik): the LDAP admin filter is inert; the auth door lives in Users
Diagnosed 'authentik login does not work in Jellyfin' 2026-11-04: nothing was
broken. The Jellyfin admin account 'dank' is a LOCAL-provider account, and the
operator's authentik identity (akadmin) was not in jellyfin_users, so the plugin
search found nothing and Jellyfin fell back to local auth - same UI message,
opposite fix. akadmin is now in jellyfin_users (directory data, DB-owned).

- Jellyfin: the admin-filter law (DN match, so a group-OU base grants nothing;
  memberOf shape is the working one), the AuthenticationProviderId diagnostic,
  re-verified LDAP path (bind + memberOf search from Dankserver).
- Authentik Outpost Reference: settle the token-intent argument (the plugin
  sends a password, never a token), verify the bind DN live, and record the
  instruments/traps: the outpost journal never shows a bind, Event is keyed by
  created/action, Outpost has no apps, and piping ak shell steals its heredoc.
- Authentik: stop restating the LDAP outpost digest - the unit file owns it.
2026-11-04 12:25:57 -08:00
dank 70aa40a24a Merge pull request 'The --after help names the trigger’s epoch, not a file mtime' (#142) from fix/after-epoch-help into main 2026-11-04 12:25:46 -08:00
dank 8f2c3bb378 docs(scripts): --after names the trigger's epoch, never a file mtime
Both drivers' help text recommended `--after "$(stat -c %Y <journal>)"`. Measured
2026-11-04 that epoch is answered by whatever scan finished before the question was
asked: on Jellyfin it reads an earlier task's EndTimeUtc, and the Plex door cannot be
a waiter at all, since PMS stamps section_locations.scanned_at when it accepts the
request. The laws are on the vault pages (main 146c28d); this puts the warning at
point of use, in the text an agent reads before calling the door.
2026-11-04 12:24:56 -08:00
dank 146c28da9f docs(plex,jellyfin): the scan doors are not waiters; supersedes b82655d
`status` compares a recorded timestamp with `--after`. Measured 2026-11-04: Plex
answered `settled` in the same second its own trigger was accepted (stamp
19:46:59Z for a request accepted 19:46:58Z) and still answered `settled` from that
one stamp 30 minutes later while the walk ran. So no epoch survives it — a journal
mtime fails and the trigger's epoch fails, because the trigger satisfies the door it
was asked to wait for. That is the correction of b82655d, which prescribed the
trigger's epoch for this door.

Jellyfin's `EndTimeUtc` only advances when a scan finishes, so there the trigger's
epoch is sound and a file mtime is not; its `wait` already refuses an epoch an
earlier task passes. The proof a scan did its work stays `media-verify.py`
replaying the journal: a scan can clear rows but cannot invent one.
2026-11-04 12:20:59 -08:00
dank b82655d30a docs(plex): --after takes the trigger's epoch; a file mtime reports a false settle
The page's own example passed the journal mtime. Measured 2026-11-04: a chain
using it fell straight through its wait loop, because status only asks whether
the section has scanned since that epoch and the section had been walked earlier
the same day. The verify that followed read a database that had not yet processed
that day's renames — a false success, which is the direction that gets believed.
2026-11-04 12:17:20 -08:00
dank 0161ab2aa8 Merge pull request 'Media refresh drivers: the two scan doors become commands' (#140) from feat/media-refresh-drivers into main 2026-11-04 12:15:38 -08:00
dank 085a5d570c Merge pull request 'docs(openwebui): the browser-kokoro switch is the user's settings row, not browser storage' (#141) from docs/openwebui-tts-kokoro-user-row into main 2026-11-04 12:14:35 -08:00
Frank 9ca69d4e7e Merge remote-tracking branch 'origin/main' into wt/20261104-104444 2026-11-04 12:09:57 -08:00
Frank 64b06b9426 docs(openwebui): the browser-kokoro switch is the user's settings row, not browser storage
`wilwheaton` kept failing in OpenWebUI with the 29-name Kokoro enum even
after the dropdown was set to Default. The page's 2026-10-18 conclusion —
"no user.settings row carries an audio override, so the choice lives in that
browser's own storage" — was a probe-depth bug: the personal audio settings
live in table `user`, and the 0.12.x front end nests them under `ui`
(`settings.ui.audio.tts.engine`). The row did carry `browser-kokoro`, and the
SPA re-applies the server copy on every load, so the UI fix never stuck.

Corrected facts on the owner page:
- where the override really lives, and the version-dependent nesting;
- repair BOTH engine and voice: engine "" still sends the stored voice, so a
  Kokoro-era `voice: "alloy"` returns as frank-tts's own 400;
- the one-line tells: an `/wasm/ort-wasm-simd-threaded.jsep.wasm` fetch means
  Kokoro.js is live in that browser, and uvicorn logs every request so a
  missing `/api/v1/audio/speech` line is evidence;
- deployed build is 0.12.5 (pin `0b399a8b`, bumped 2026-10-30) — the page
  still claimed 0.11.4 — and the 401 paragraph's duplicated sentence is gone.

Verified 2026-11-04: with the row repaired, `POST /api/v1/audio/speech` as a
real user over the vhost returns 200 audio/mpeg and frank-tts logs
`speech voice=wilwheaton`.
2026-11-04 12:09:57 -08:00
dank e519f6b554 docs(vault): the promo-clip hold is purged; merge main's purge record 2026-11-04 11:59:00 -08:00
Frank 1e0247f29e Merge main into feat/media-refresh-drivers (main moved: the purge record) 2026-11-04 11:58:36 -08:00
dank 71e7e94b94 docs(media): the 2026-11-04 purge, and the gate that proves a hold is safe to delete
The duplicate-prune, TV-duplicate, Ghibli and promo-clip holds went out of the
array trash on Frank's word: ~1.2 TB back. This replaces the three "purge pending
Frank's word" rows with what happened.

Two procedures the page did not carry:

- The purge gate. `REAL: 0` in both databases only says no row points at a moved
  path; it says nothing about whether the film survived. Every held file needs a
  same-work carrier in the live library at >= its size, re-derived at purge time,
  because the relayout renames folders AND keeper files, accents defeat ASCII
  matching (Nausicaä), and a carrier is usually another container at another
  size. That supersedes the 2026-11-01 "re-verify by size" law, which is exactly
  the test that reports survivors as lost.
- The purge mechanics. The repo purger acts on a root's own .AGENTTRASH and
  nothing else, so a partial purge is staged into a scratch root while the
  unapproved buckets sit parked at the array root as .HOLD-<date>-*.

./tests/run.sh: 824 assertions, all pass.
2026-11-04 11:58:21 -08:00
Frank 24d793eb7d docs(vault): the promo-clip hold is 31 clips and one sidecar 2026-11-04 11:30:07 -08:00
Frank 95375683d5 docs(vault): the scan pages name their drivers, and a class no screen sees
Plex.md and Jellyfin.md keep every law and point at the driver that now carries
them, so the procedure has one owner and the pages keep the failure signatures.
Jellyfin.md also records where `jellyfin.db` actually lives on this host (the
quadlet's config bind, not the stock path) and demotes `immutable=1` on the
`ApiKeys` read to the `-shm` fallback, because the database is WAL and
`immutable` skips the log. Plex.md gains the live section keys and the
`scanned_at == scannedAt` fact.

Media Relayout.md's hand-off after an apply names both drivers and states the
ordering law: Jellyfin first and alone, since its one door walks every library
while Plex forks a scanner per section, and the array is what both share.

Media Duplicate Prune.md gains the guard the 2026-11-04 hold taught: 32 files
carrying real film titles were 8-20-second h264 clips (49 MiB, 25 of them in a
`_release-group watermarks/` folder), invisible to every duplicate screen
because nothing shares their size and the feature-length guard hides them. The
census that finds the next batch is a duration floor, not a hash.
2026-11-04 11:29:03 -08:00
Frank e7d7c6eea2 feat(scripts): the two media scan doors become commands
Both media servers' scan procedure lived only as a hand-written curl block in
the vault, and that block has bitten this fleet twice: the escaped-quote form
that reads exactly like a dead API key, and a config `url` line meeting a
command-line URL so curl quietly performs a second transfer and dumps the whole
task table on your stdout. Jellyfin's door is now
`scripts/media-refresh-jellyfin.py` (trigger / status / wait) and Plex's is
`scripts/media-refresh-plex.sh` (trigger / status);
`tests/media_refresh.bats` pins the contracts an operator observes, against a
stub and with no live server.

The Jellyfin `wait` encodes the settle law instead of a poll count: the Scan
Media Library task must be seen Running — or finish while watching, or already
pass the `--after` it is given — before any quiet read counts toward settled.
A watcher that saw no Running otherwise reports the pre-scan library as the
fresh one, which is how a rename looks applied while nothing has rescanned.
Plex's `status` needs no token at all: `section_locations.scanned_at` carries
the same epoch the API prints as `scannedAt` (measured on section 1 today).

Neither driver lets a secret reach argv or a URL: the Jellyfin key rides a
mode-600 curl config as the quoted MediaBrowser form, and the Plex token is
read inside the container, where the HTTPD actually answers, into a config file
that is shredded there.
2026-11-04 11:29:03 -08:00
dank e1bf1d6e80 Merge pull request 'Household docs: rewrite the family guide, and make the model cards tell the truth' (#139) from docs/household-guide-refresh into main 2026-11-03 04:08:09 -08:00
Frank ef306397a5 docs(vault): the four-slot archon contract, and pages that contradicted themselves
frank-archon.md's intro promised 2 conversations while its own shape table said
MAX_SEQS=4; AGENTS.md carried --max-num-seqs 2 and a 44,928 MiB host pool against
a live 4 and 49,152. Both now quote the env.

dankclaw-iommu-translation-off.md re-transcribed frank-dragoon's boot ledger,
including a pool figure that was superseded the same day, and asserted in the
present tense that no NPU endpoint is deployed - falsified by the XDNA2 embedding
lane on 2026-10-19. Both facts now belong to [[frank-dragoon]] alone.

Deployment Flow.md had a sentence broken by an earlier link removal.

Census: every tunable in the three scripts/<svc>.env against its endpoint page
(only truncated-sha and elided-path forms, which the pages omit on purpose), live
/health + /v1/models on all three endpoints, check-okf.py clean at 125 pages, no
duplicate titles, no page unlinked from an index.
2026-11-03 04:07:46 -08:00
Frank 3896dce364 fix(homepage): the model cards speak the live shape
Three numbers on the family-facing model cards had drifted from scripts/*.env:
archon said three submissions where FRANK_ARCHON_MAX_SEQS=4 (1bb6399), zealot
said 262k context where the contract is 200,704 a slot (CTX_SIZE 602112 / PARALLEL
3, and llama.cpp 400s over the slot rather than truncating), and zealot's warning
still promised censored-base refusals after the Uncensored-LoRA went into the load
line (LORA_FILE set, LORA_SCALE=1). archon's role dropped an unverifiable
"fastest"; zealot's card dropped a tokens/s figure nobody is comparing.

Deletes the assertion that zealot's warning must not claim uncensored output. It
was written 2026-10-17, when the censored base was live; the adapter landed after
it, so the pin now enforces a false safety claim on the one surface the household
reads. The general invariant survives and still fires: a card whose spec says
uncensored or abliterated must carry the legal warning (same seam, lines 100-104).
2026-11-03 04:07:46 -08:00
Frank 3f82c4d5d8 docs(homepage): rewrite the household guide for the people who use it
Task-first instead of app-first: start here, pick what you want to do, open the
door. Every door gets its exact address, the button to press, and the failure the
household actually hits.

Screenshots are files now (img/onboarding/*.webp, 14 of them, all captured logged
in as a standard user) instead of inline data URIs: the page is 23 KB of prose
instead of 30 KB of base64, and the images cache on their own. Zero-JS kept.

Facts re-checked against the live services rather than against the old page: the
front door is www.franksdank.cloud (the apex 308s there); the standard user's
Open WebUI account sees all three models and opens on frank-dragoon (read from
/api/models as that user); WebDAV answers only on the house network (207 on
192.168.0.235:3923, a guard challenge through the public door); Audiobookshelf
iOS is a limited-seat TestFlight beta, Android is the Play release.
2026-11-03 04:07:39 -08:00
Frank 5030f83c55 docs(jellyfin): one URL per curl config read, or the second transfer dumps the body 2026-11-03 03:42:21 -08:00
dank b875256500 Merge pull request 'feat(jellyfin): big-screen theme for the TV - repo CSS in the Media Center slot' (#138) from feat/jellyfin-big-screen-theme into main 2026-11-03 03:40:04 -08:00
Frank 51abc6550d feat(jellyfin): big-screen theme for the TV, mounted into the wmc slot 2026-11-03 03:38:15 -08:00
dank 64f489023e Merge pull request 'feat(homepage): Jellyfin goes live, Audiobookshelf joins the household, Marreta says what it is' (#137) from feat/homepage-media-cards into main 2026-11-03 03:32:49 -08:00
Frank 3511e04ae9 docs(jellyfin): record the API-key ledger and the rotation door 2026-11-03 03:27:48 -08:00
Frank 21b7ad15d9 chore(homepage): retire the unreferenced dankserver registry copy
shared/homepage/dankserver.services.json was read by nothing: no reference
anywhere in the repo, and deploy_homepage never copies it. It sat beside the
real registry looking like a second live card set, which is exactly how a
future agent gets misled about what the page renders. Moved to the gitignored
shared/homepage/.AGENTTRASH/ beside it, so it stays recoverable.
2026-11-03 03:23:13 -08:00
Frank 4ad55c28a0 feat(homepage): Jellyfin live, Audiobookshelf to the household page, Marreta says what it is
Jellyfin's card still read "Work in progress: not open to family users yet"
with no link; the server answers 12.4.0 on :8096 and carries a second
(non-admin) account, so the card flips to live status with its Open link on
http://jelly.franksdank.cloud. It has no Authentik link, and the copy says so.

Audiobookshelf moves category admin -> user. A card renders on exactly one
page, so it leaves /admin: Authentik SSO made it a household app, and the
household page is the door that matters.

Marreta's description now names what it is - the content-cleaning proxy in the
Gluetun netns with a headless companion - and its honest limit.

Also in this change:
- docs.html: a "Media & files" line for the audiobook door and its SSO, plus
  the pre-existing unclosed <ul> under frank-dragoon, which made the whole
  frank-archon block parse inside the dragoon bullet.
- Caddyfile: the jelly. vhost gains the exemption reason the External Exposure
  Policy requires (comment only - Caddy strips comments, so no proxy bounce).
- Homepage.md: classification list gains Audiobookshelf, and the section now
  states that services.json is the authority for placement/WIP/link.
2026-11-03 03:20:08 -08:00
dank c9a8469d04 Merge pull request 'docs(audiobookshelf): the wide listener default is a decision, not a defect' (#136) from docs/abs-listener-default-decision into main 2026-11-03 03:09:19 -08:00
Frank 5794b07c32 docs(audiobookshelf): the wide listener default is a decision, not a defect
The SSO paragraph left "whether an access policy should be bound to the
audiobookshelf application" open, which invites the next agent to relitigate
it. Frank's answer of 2026-11-03: accepted on purpose — every signer becomes a
listener with all libraries and downloads, no access policy on the application,
narrowing done per account in Settings -> Users. What that makes load-bearing is
authentik's own signup flow, now named in the same paragraph as the thing that
must stay operator-only.
2026-11-03 03:08:36 -08:00
dank 1a47b831ee Merge pull request 'docs(audiobookshelf): /api/login is not a route — and what auto-registration grants' (#135) from docs/abs-login-path-and-listener-shape into main 2026-11-03 03:04:05 -08:00
Frank 5be66df2bb docs(audiobookshelf): /api/login is not a route
A wrong path was mistaken for a dead break-glass credential. Local login is
POST <base>/login; /api/* is behind the bearer guard, which answers a bare 401
and logs nothing, so a bad path looks exactly like a bad password. The recorded
value bcrypt-matches users.pash, so local really is the door when authentik is
down, and the rate-limit line was protecting a path that cannot carry a login.

Also records what auto-registration actually grants, measured from the state DB
because GET /api/users returns the limited shape: every SSO signer arrives with
accessAllLibraries and download already on, and carries no local password.
2026-11-03 02:28:11 -08:00
dank d3aedd1837 Merge pull request 'Every authentik account that signs in to Audiobookshelf becomes a listener' (#134) from feat/audiobookshelf-sso-default-listener into main 2026-11-03 02:19:14 -08:00
Frank 66766f23ef docs(audiobookshelf): the IdP is the library gate now
Rewrite the listener onboarding, the role row, and the fail-closed law: the
claim grants every signer, revocation happens on the ABS side (disable or
delete the user), and what actually guards the library is who can reach the
audiobookshelf application in authentik. Name both e2e gates in Verification.
2026-11-03 02:06:20 -08:00
Frank 5cf5416471 test(e2e): pin the Audiobookshelf default listener grant
spec 24- signs in through SSO as qa-e2e — an identity in no Audiobookshelf
group — and must land inside the app with type "user". It is the counterpart
to 23-, which pins the admin escalation: together they catch both directions
of a scope-mapping regression (household locked out, or household made site
admin). Verified red against the pre-change mapping — the identity never
received a token — and green after re-provisioning: 4 passed.

Joined to the standard project, which is the only project whose identity
observes the default.
2026-11-03 02:06:20 -08:00
Frank 30186d8365 feat(authentik): make every Audiobookshelf SSO signer a listener
The abs_groups claim decided the listener role from membership of
audiobookshelf_users, a group nobody was ever added to, while the ABS
application carries no access policy of its own. The result: seventeen
household accounts authenticated at the IdP successfully and were then
refused by the app — a policy decision that reads as a broken login.

The claim now answers ["user"] for every signer and appends "admin" only
for audiobookshelf_admins, so the IdP login is the gate and onboarding a
listener is exactly "create the authentik account". Admins keep the exact
claim they always received, ["admin","user"], in that order: ABS's
group-tag matching has never been measured for order-sensitivity and the
working admin login is not where to find out. audiobookshelf_users is
deleted rather than left empty — a group that decides nothing is a trap
for the next reader, who will assume it gates something.

Two side fixes found by running the script:
 - Two comments inside the unquoted PYEOF heredoc carried backticks, so
   bash command-substituted them locally on EVERY run ("audiobookshelf_users:
   command not found", "user: command not found") and spliced the output
   into the python program. Names in that body are plain now, with a note
   on why. A quoted heredoc is not an option: the body interpolates.
 - The default-grant probe picked authentik's AnonymousUser row, then its
   admin superuser. It now walks real non-superuser accounts, preferring
   the fleet's plain-user marker group, and prints up to three claims.
2026-11-03 02:06:20 -08:00
dank 3a6f7c7280 Merge pull request 'feat(caddy): Audiobookshelf outside the guard chain, so the phone app works from anywhere' (#133) from feat/audiobookshelf-public-app-access into main 2026-11-03 01:56:51 -08:00
dank c7643fb4d0 docs: regenerate the vault index row for the new Audiobookshelf description 2026-11-03 01:55:06 -08:00
dank 882a9f6808 docs: own the un-guard procedure and the browser-only law
AGENTS.md states the three exposure postures and why the guard chain cannot front a
non-browser client. Infrastructure/Caddy.md gains *Unguard a vhost* — route first,
IdP rows last, audit lists after — since deleting the proxy provider while traffic
still enters :8016 hands browsers authentik's error page. The Audiobookshelf page
retires its two-doors law (the guarded vhost as a browser-only door, remote
listening via VPN) for one address every client carries, adds how a listener gets an
account, and says plainly what the exemption costs.
2026-11-03 01:54:08 -08:00
dank aa57c1a894 fix(authentik): register the mobile OIDC redirect on every ABS origin
The mobile-redirect URI was registered on the LAN origin only, because the phone
had to be configured with the LAN address while the vhost was guarded. Now that the
app rides the public name it sends the public mobile-redirect URI, and the RP's
STRICT matching answers an unregistered one with "Redirect URI Error" — a phone
locked out at the last step of a login that looked healthy. Both callback URIs are
now emitted for every entry in BASES.
2026-11-03 01:54:08 -08:00
dank 2a48751283 feat(caddy): serve Audiobookshelf outside the guard chain
The Audiobookshelf phone app could never sign in over the public name. The guard
chain answered its /ping and /socket.io/ handshake with Anubis challenge HTML, so
the app evaluated 'isInit' in e.data on a non-object and reported "Failed to
contact server" while the container's request log recorded nothing at all. Anubis
demands a JavaScript proof-of-work and the authentik outpost works from a session
cookie: the chain is browser-only by construction, and a native client, a podcast
RSS reader or an OPDS client cannot pass it.

Audiobookshelf authenticates itself (local accounts plus authentik OIDC), so it
joins jelly/plex/*arr in the exempt set: the public site block dials the container
directly, keeping the X-Forwarded-Proto pin its OIDC origin check depends on, and
the @audiobooks route leaves the post-auth listener so one host has one owner. The
IdP rows that guarded the name go after this deploys, never before.

tests/e2e/specs/14-guard.spec.ts loses audiobooks as a guarded target and gains the
exemption assertion (app bytes to an anonymous client, the app's own 401, never
challenge HTML); tests/anubis.bats pins the route shape.
2026-11-03 01:54:08 -08:00
dank 6083903ccb Merge pull request 'fix(e2e): judge only body-carrying script responses in the ABS SSO spec' (#132) from fix/e2e-mime-304 into main 2026-11-03 01:42:28 -08:00
dank ee9eb353fb fix(e2e): judge only body-carrying script responses in the ABS SSO spec
A 304 carries no Content-Type by spec, and the OIDC bounce revalidates the entry script (Cache-Control: max-age=0 + ETag), so the blanket MIME assertion failed on a healthy bundle. The HTML-stub class is still caught: 200s are still judged, and the count floor now runs on the same filtered set.
2026-11-03 01:39:39 -08:00
dank df597bb1b1 Merge pull request 'fix(abs-guard): audiobooks.franksdank.cloud completes OIDC (forwarded scheme + provider OAuth defaults)' (#131) from fix/abs-guard-oauth-defaults into main 2026-11-03 01:26:32 -08:00
Frank 23ad0ca317 chore(abs): name the Audiobookshelf door audiobooks.franksdank.cloud
`abs.` was a three-letter abbreviation of the container; the library it serves is
/mnt/entropy/Audiobooks, and `audiobooks.` says what the door is. Renamed in one
commit on purpose: the pihole seam fails the suite if a Caddyfile vhost has no
addn-hosts record, so any split would leave a red commit, and the `abs` name was
never deployed at the IdP with this fix, so nothing has to keep answering it —
no alias, no redirect, no stale URI.

Moves together: the Caddyfile site block, its matcher (@abs -> @audiobooks), the
LAN record, ABS_EXTRA_BASES in authentik-audiobookshelf-sso.sh (that script
REPLACES the provider's redirect-URI list, so the old origin's callback leaves the
RP when it is next run), the homepage admin tile, the guard spec's GUARDED list and
leak marker, and the three vault pages that name the door. authentik's provider row
follows in the deploy window (external_host + name), which re-derives its
redirect_uris through scripts/authentik-proxy-provider-defaults.sh; the Application
row keeps its own name, "Audiobookshelf Guard" — it names the app, not the vhost.
2026-11-03 01:23:54 -08:00
Frank 71d5804eaf docs(caddy): put both auth walls in the guard-a-new-vhost procedure
Step 1 gains the closing sentence the abs-proxy incident demands: the ak-shell
snippet that creates a ProxyProvider with objects.create() must be followed by
scripts/authentik-proxy-provider-defaults.sh, or the guard fails before any
credential. Step 2 gains the forwarded-scheme clause for any app that derives its
own origin from the request. Each fact keeps one owner — the laws stay on
[[Authentik Outpost Reference]] (provisioning rule 3) and [[Audiobookshelf]]; this
page only says where in the procedure they bite. Outpost page timestamp bumped to
match its substantive edit.
2026-11-03 01:17:12 -08:00
Frank 5ef141eb17 fix(caddy): tell Audiobookshelf its real scheme so the OIDC callback validates
ABS checks every OIDC callback against the origin it believes it is served from:
getRequestOrigin() is x-forwarded-proto://Host (dist-server/server/utils/requestUtils.js,
read in the pinned image digest 2026-11-03). The post-auth http://:8017 listener is fed
over plain http, so Caddy's own X-Forwarded-Proto is http, ABS derives
http://abs.franksdank.cloud, and answers its own https callback with

    400 {"message":"Invalid callback URL - must be same-origin"}

Measured at the app with the callback held fixed: Host abs.franksdank.cloud alone
reproduces the 400 (as does every other Host, including the bare LAN IP), and adding
`X-Forwarded-Proto: https` turns it into a 302 to /application/o/authorize/ carrying
the registered redirect_uri. The edge for a guarded name is TLS-only, so https is the
truthful value, not a spoof; the LAN door needs nothing because there browser and
server already agree on http, which is why only the guarded path was broken.
2026-11-03 01:16:40 -08:00
Frank fadbe10420 fix(authentik): census and repair the OAuth defaults on every proxy provider
A proxy provider built by hand in `ak shell` never runs
ProxyProvider.set_oauth_defaults() — the REST serializer does, the ORM does not —
so the row looks fully configured while redirect_uris is empty (they are derived
from external_host by _get_callback_url()), grant_types is [], and
authorization_flow is NULL. Each one kills a guarded login in a different,
unrelated-looking way, all before any credential is requested: authentik's own
"Redirect URI Error" page, then a bodiless HTTP 400 from /application/o/authorize/
once the URIs are filled by hand, and a 500 NoneType.slug in the flow planner.
abs-proxy (abs.franksdank.cloud) hit the first two on 2026-11-03.

The script calls the vendor's own set_oauth_defaults() and points the
authorization flow at default-provider-authorization-implicit-consent for every
proxy provider, idempotently; a provider that already matches is untouched, so a
deliberate deviation is never flattened. `--check` is the census (exit 1 on
drift, writes nothing) for gate use. Applied live 2026-11-03: abs-proxy fixed,
the other ten (REST-created) already compliant, census now reports drift: 0.

This is authentik DB state and no deployable file carries it — a blueprint
re-import or an ak-shell re-creation reverts it and the symptom returns, so the
law and the failure signatures are recorded on the outpost page as provisioning
rule 3.
2026-11-03 01:16:35 -08:00
dank 17ea2f4110 Merge pull request 'feat(caddy): give Audiobookshelf a guarded abs.franksdank.cloud address' (#130) from feat/abs-vhost into main 2026-11-03 00:57:47 -08:00
Frank 603513a15e docs: the guard-a-new-vhost procedure and Audiobookshelf's second door
Infrastructure/Caddy.md now owns the order of operations for exposing a vhost
(proxy provider + application + embedded-outpost claim, then the route, then the
name and the chain audit), because the IdP row is invisible state that nobody
recreates. Audiobookshelf's Internal-only row and law become Two doors, one app;
the homepage link and index.md follow.
2026-11-03 00:55:02 -08:00
Frank 5e0592381a test(e2e): audit the guard chain over abs.franksdank.cloud
The guard spec's GUARDED list is the lockstep with the Caddyfile; abs joins it
with the app's own SSO-button label as the leak marker - a string the Audiobookshelf
login renders and the IdP gate never prints.
2026-11-03 00:55:02 -08:00
Frank ef89f77ea7 feat(authentik): register the ABS browser callback per origin
ABS builds redirect_uri from the origin of the request, so a second door is a
second URI: ABS_EXTRA_BASES (default the guarded vhost) now contributes a
callback each, while the mobile-redirect URI stays on the LAN origin - the phone
is a non-browser client and cannot solve the guard's challenge, so it is
configured with the LAN address. Measured on the live provider: three URIs,
grants and scopes unchanged.
2026-11-03 00:55:02 -08:00
Frank 9e2584ec8b feat(homepage): point the Audiobookshelf tile at the vhost
The tile carried the bare LAN IP. The probe stays the LAN host:port so the
homepage checker keeps measuring the app rather than the auth chain in front of
it.
2026-11-03 00:55:02 -08:00
Frank a5dfc1ee99 feat(caddy): serve Audiobookshelf at a guarded abs.franksdank.cloud
Provider-first exposure: the IdP row (proxy provider abs-proxy + application
"Audiobookshelf Guard", claimed by the embedded outpost) already exists, so the
route can ship. The site block proxies everything to anubis:8923 like every other
guarded vhost, and the post-auth :8017 handler routes the host to
192.168.0.235:8000. The container carries ROUTER_BASE_PATH=/audiobookshelf, so
the app has no root: the bare / is redirected onto the base path inside the
post-auth handler, never pre-auth, so an anonymous client only ever meets the
gate.
2026-11-03 00:55:02 -08:00
Frank 7e72126a07 docs(audiobookshelf): an SSO identity cannot land on an existing account
authOpenIDMatchExistingBy is unset on this deployment, so ABS matches an
OpenID userinfo on sub only and auto-registers everyone else. A new authentik
username that equals an existing ABS username therefore dies on the unique
constraint inside the callback, reported only as 'Unauthorized' - which is
exactly what a future agent would hit by 'just creating a frank user' in the
IdP to reach the root account. Records the two real doors (match by username,
which links and keeps root, since setUserGroup refuses to downgrade a root;
or a non-colliding name), and corrects the admin-membership row: akadmin is
the operator's own authentik account, already granted site admin.
2026-11-02 20:29:30 -08:00
dank 80a65b013f Merge pull request 'fix(authentik): provision the Audiobookshelf provider with grant types and OpenID scopes' (#129) from fix/abs-grant-types into main 2026-11-02 20:24:53 -08:00
Frank 54ec72da17 docs(audiobookshelf): the two IdP laws the dead login taught, and the gate
Records that an ORM-created provider has no grant types and that the claim
mapping is not the whole property_mappings list, with the failure signatures
to recognise; names qa-e2e-admin as the admin-group fixture and the new
Playwright spec as the live verification.
2026-11-02 20:21:21 -08:00
Frank d82b8efdd7 test(e2e): gate the Audiobookshelf SSO round trip end to end
The service answers on a LAN address with no vhost, so it was in no
guarded-vhost list and no spec touched it: every deploy gate passed while its
login was dead. The gate signs in as qa-e2e-admin and asserts the two
consumer-visible outcomes that the two defect classes break - every .js the
SPA loads arrives as JavaScript (a bundle built for the wrong base path comes
back text/html and boots nothing), and /api/me answers 200 with type 'admin'
(true only while grant types, the OpenID scope mappings and the group claim
all hold). Joined to the admin project, so it rides every deploy e2e gate.
2026-11-02 20:21:21 -08:00
Frank 27b3c24065 fix(authentik): provision the Audiobookshelf provider with grant types and OpenID scopes
The provider row this script creates was missing two fields, and each one is
a dead login that names nothing on the wire (both measured 2026-11-02 in the
browser):

* grant_types defaults to [] on an ORM-created provider in authentik 2026.8,
  so /application/o/authorize/ refused the code flow with oauthlib's generic
  'The request is otherwise malformed'; the log line that names the cause is
  'Invalid grant_type for provider'. Set authorization_code + refresh_token,
  and reconcile it on re-run.
* property_mappings held only the abs_groups claim mapping, so the granted
  token was scoped shorter than the scopes ABS requests and ABS's /userinfo
  call died 'insufficient_scope' after a good code exchange. Attach the
  built-in OpenID openid/profile/email mappings beside the claim.

Also provisions qa-e2e-admin into the admin group: it is the identity the new
Playwright gate signs in as, so a blueprint re-import restores the fixture.
2026-11-02 20:21:21 -08:00
dank 7e9bfbc2be Merge pull request 'fix(audiobookshelf): serve under /audiobookshelf, send the OIDC redirect subfolder' (#128) from fix/abs-base-path-sso into main 2026-11-02 20:02:57 -08:00
Frank f5410bd253 docs(audiobookshelf): the probe routes answer at the root under either base path
Measured on both shapes: ping returns success:true and healthcheck returns OK at the root whether or not the app is mounted under a prefix, which is why the phone app's connectivity check works before it ever resolves the base path. The paragraph now says which routes were probed instead of asserting 200s in general, and the frontmatter description fits the OKF 120-char budget the linter enforces.
2026-11-02 19:53:16 -08:00
Frank 98522cecba docs(jellyfin): correct a future-dated measurement stamp 2026-11-02 19:50:44 -08:00
Frank 3138e782bd docs(audiobookshelf): the base-path law, the SSO contract, the phone address
The page owned the URL as a plain host:port; every client now carries the /audiobookshelf prefix. Added the two laws a next operator needs (why the base path is not a choice, why the redirect subfolder must be sent) and the Single sign-on section naming which script owns which half.
2026-11-02 19:50:44 -08:00
Frank 393b77dfd2 feat(audiobookshelf): provision authentik SSO and send the redirect subfolder
The live round trip died on authentik with Redirect URI Error. ABS builds redirect_uri as <origin><authOpenIDSubfolderForRedirectURLs>/auth/openid/callback, and that key is unset on a database created before SSO (the 2.17.4 migration only fills a DB that already had OIDC enabled), so the URL carried the literal string undefined. Arming through the API therefore has to send the value; it is the mounted base path, so it is derived from ABS_URL rather than becoming a second knob an operator has to keep in sync.

authentik-audiobookshelf-sso.sh is the IdP half kept in git: provider, groups, the abs_groups scope mapping and the redirect URIs. That state lives in authentik's database, and a blueprint re-import reverts hand-made rows silently, so it needs a re-runnable door. Tokens are HS256 because no provider on the fleet carries a signing key, which makes the client secret the signature key.

The stub server now records the auth-settings PATCH body, so the seam asserts the subfolder that reaches the wire follows ABS_URL for both the prefixed and root shapes, and a second seam pins that the provider script and the bootstrapper build the same redirect path.
2026-11-02 19:50:44 -08:00
Frank cfdd3435bc fix(audiobookshelf): mount the app at the base path its SPA was built for
ROUTER_BASE_PATH=/ served a healthy API and a web UI that could never boot. The index.html in the pinned image is built with routerBasePath:/audiobookshelf, so every script request fell through to the SPA fallback and came back as text/html, which Chrome refuses to execute. healthcheck, the API and the phone app all stayed fine, which is exactly how this hid behind an API-only deploy check.

The deployer now reads the mounted base path out of the unit and hands it to the bootstrapper as ABS_URL, so the API door, the login route and the OIDC redirect subfolder cannot drift from the unit. healthcheck answers at the root under either base path, so the health gate is unchanged.
2026-11-02 19:50:44 -08:00
dank f3a6b4f715 Merge pull request 'docs(jellyfin): the recursive Movie list is not proof an item is indexed' (#126) from docs/jellyfin-verification-read into main 2026-11-02 19:14:33 -08:00
dank 5c888ff948 Merge pull request 'docs(audiobookshelf): the phone app's connection facts and where its token lives' (#127) from docs/abs-phone-app into main 2026-11-02 19:11:23 -08:00
Frank 03e0d84380 docs(audiobookshelf): the phone app's connection facts and where its token lives
The page told an agent how to sign in from a browser and nothing about the phone.
Measured 2026-11-03 on the deployed service:

- Server Address `http://192.168.0.235:8000`, user `frank`, plain HTTP, no base-path
  prefix; `/healthcheck` and `/ping` answer 200 from a second host on the LAN and
  `iptables -S INPUT` has no rule for 8000, so no phone-side setup is needed.
  `download: true` on the root account is what enables offline listening.
- `POST /login` puts the bearer at `user.token`, not top level. Reading
  `response.token` yields "", the client sends `Authorization: Bearer ` with nothing
  after it, and every `/api/*` answers 401 as though the password were wrong — which
  it was not. That is the failure I hit inside five minutes of scripting this.
- The LAN-only law gains its consequence for clients: the Anubis + Authentik chain
  serves browsers, so guarding a vhost does not make the phone app work remotely.
  Remote listening wants a VPN into the LAN; the LAN address is then still correct.

bats: 822 seams green.
2026-11-02 19:09:23 -08:00
Frank a5603d8f01 - 2026-11-02 18:52:13 -08:00
dank 988c4899d4 Merge pull request 'docs(media): point plan/journal/park paths at .media-campaign and NeedsProcessing' (#125) from docs/media-artifact-paths into main 2026-11-02 18:42:34 -08:00
Frank 77e9973ff5 docs(media): point plan/journal/park paths at .media-campaign and NeedsProcessing
The campaign-artifact home moved (42bc482) but the copy-paste surfaces still
named the old ones, and a tool's usage block is exactly what an agent runs:

- scripts/media-shows.py, media-tv.py, media-tvdupes.py: usage examples wrote
  plans and journals at the array root. media-apply.py defaults its journal to
  <root>/media-relayout.jsonl, so any run copied from those lines re-creates
  the root-level clutter the vault now forbids. Each block now names
  /mnt/entropy/.media-campaign/ and says why --journal is explicit.
- media-shows.py: --park named the retired media-park; the pile is
  /mnt/entropy/NeedsProcessing/TVShows.
- Jellyfin.md: library-coverage law named /data/media-park; names the pile by
  its current path and records when it moved.
- Media TV Relayout.md: the runnable --park path named media-park.

bats: 822 seams green.
2026-11-02 18:02:48 -08:00
Frank 625263dd48 Merge remote-tracking branch 'origin/main' into feat/jellyfin-provider-keys 2026-11-02 17:44:49 -08:00
Frank 42bc482ed8 docs(media): campaign artifacts live in /mnt/entropy/.media-campaign
The array root carried 48 planning artifacts (journals, plans, run logs)
accumulated by the relayout campaigns since 2026-11-02. They sit beside the
libraries now, which is how a media server ends up indexing a journal.

Moved all 48 (219.8 MiB) into the dot-directory `/mnt/entropy/.media-campaign`
with a journaled `media-apply` run (plan, apply, verify — 48 verified at their
targets, 0 root files left). Jellyfin and Plex ignore dot-directories, and the
overlay view exposes the array root to both servers, so the same pollution
existed on the view side.

Every vault page that named a root-level journal path is rewritten to the new
home (26 occurrences across 6 pages), and the law is stated once, in
[[Media Relayout]], which owns the tool chain: `media-apply.py` defaults its
journal to `media-relayout.jsonl` beside `--root`, so a run that omits
`--journal` recreates exactly the root-level log this now forbids.
2026-11-02 17:44:49 -08:00
dank aea24353ad Merge pull request 'docs(jellyfin): the switches live under LibraryOptions; a pinned art pass is not a provider census' (#124) from feat/jellyfin-provider-keys into main 2026-11-02 17:39:21 -08:00
Frank d1493de31a Merge remote-tracking branch 'origin/main' into feat/jellyfin-provider-keys
# Conflicts:
#	serverAppConfigs-docs/Dankserver/Audiobookshelf.md
#	serverAppConfigs-docs/Dankserver/Jellyfin.md
2026-11-02 15:04:46 -08:00
Frank 93725f42b3 docs(jellyfin): correct the posture table and three API/art laws
- the per-library switches are under LibraryOptions, not top-level DTO keys
- Shows was flipped on 2026-11-02; Music is the only silent file index left
- never pin Providers= on a TV art loop: the TVDB-only offers disappear
- /Items?Ids= needs no UserId where /Items/{id} does, so verify writes there
2026-11-02 15:03:48 -08:00
dank f80148c5db Merge pull request 'fix(media-apply): refuse a plan row whose verb the tool does not know' (#123) from fix/media-apply-unknown-action into main 2026-11-02 14:07:55 -08:00
dank 541067e6b2 Merge pull request 'docs(tv): split the TV arm into its own page, add the folder-identity laws' (#122) from docs/tv-folder-identity-law into main 2026-11-02 14:03:56 -08:00
Frank 8f0b1982c2 docs(tv): the parked-folder merge, and the laws that gate it
A parked folder whose name parses to a registered show merges into the row
that owns the show. The proof is the episode rows standing under the owner
folder — never the id-less folder list, because one tvdb id can carry two
`Series` rows and an id-less folder that parses gets identified on its own
and keeps its own folder, offering the show twice.

No library file moves while `Scan Media Library` is Running: wait for a
Running sighting and then two consecutive Idle polls, because a gap between
sub-tasks reads as finished. Both sides speak different path dialects (the
server records the container coordinate, the journal speaks the array), so a
join that forgets the translation reads as an empty library.

Lands on [[Media TV Relayout]] — the page that owns the TV arm — so
`Media Relayout.md` stays under the OKF ceiling.
2026-11-02 13:56:03 -08:00
Frank a405c5ed35 fix(media-apply): refuse a plan row whose verb the tool does not know
read_plan's if/elif chain had no else, so any row that was not mkdir, keep
or move was dropped in silence. A plan of nothing but typos therefore
applied as a successful no-op: on 2026-11-02 a pass meant to rename two
TVShows series folders reported "applied: 0 moved" and exit 0, and the
journal was never written — nothing told the operator the plan had never
been read. An unread row is a plan bug, and every other plan bug in this
tool refuses before the first rename, so now this one does too, naming the
line and the verb.

cwd: /home/blank/.omp/wt/wt-20261030-163847-6d1fc1f · session: 01a18f2e-18b1-7573-bcd5-ac99636f8bcb
2026-11-02 09:51:53 -08:00
Frank b232033b11 docs(tv): split the TV arm into its own page, add the folder-identity laws
Media Relayout.md crossed the OKF 500-line ceiling with the 2026-11-02 TV
laws, and the ceiling was right: the TV arm reads a different authority and
refuses for different reasons, like the music and audiobook arms that already
have pages. The arm moves verbatim to Media TV Relayout.md and gains two laws
from the TVShows pass: a folder naming two series is not evidence of two
series (ffprobe's embedded title tag answers per file), and right numbering
with a wrong name means rename the folder only. The executor page keeps the
executor laws and gains the hold-outside---root/--force law; Jellyfin.md gains
the settle read (ScheduledTasks State, since /Library/ScanStatus is a 404 on
12.4). Index regenerated with gen-vault-index.py.

cwd: /home/blank/.omp/wt/wt-20261030-163847-6d1fc1f · session: 01a18f2e-18b1-7573-bcd5-ac99636f8bcb
2026-11-02 09:06:22 -08:00
dank 4a33898a6c Merge pull request 'fix(jellyfin): anchor the provider-key write on the file’s real root' (#121) from chore/jellyfin-tvdb-key into main 2026-11-02 07:17:25 -08:00
dank 21c98bd485 Merge pull request 'docs(jellyfin): the door that scans is /Library/Refresh, and the art door names its field' (#120) from docs/jellyfin-scan-door into main 2026-11-02 07:16:52 -08:00
dank a651c81dfb Merge pull request 'feat(audiobookshelf): bootstrap the account, library and scan from the deployer' (#119) from feat/audiobookshelf-bootstrap into main 2026-11-02 07:16:45 -08:00
Frank 0951d909c9 docs(agents): the bracket trick for a pkill pattern that must be named
The existing rule says don't `pkill -f` a pattern your own command line carries,
and to prefer `pkill -x` — but a remote one-shot often has to name the pattern,
and hitting it again today cost a silent `ssh` exit 255 with zero output, which
reads like a network fault. Record the escape (`"serve[-]script.sh"`), the zsh
glob-expansion trap that makes an unquoted `[s]` a no-op, and the exit-code
signature.
2026-11-02 04:47:16 -08:00
Frank 69effb542d docs(jellyfin): Shows is on providers, and the per-library door stays dead anyway
The `Shows` flip landed today (one field, proven in the server's own
`LibraryOptions` answer), so the posture table no longer calls it a silent file
index and `Music`/`Collections` are named as the remaining ones.

Two corrections the same day's measurements force:

- The per-library `POST /Items/{id}/Refresh` answered `204` and queued nothing
  both before the flip and after it — zero journal lines for ten minutes, and
  `Scan Media Library` kept `LastTaskResult=None, EndTimeUtc=None`, the never-run
  signature. The provider switch was never the reason that door is dead; only
  `POST /Library/Refresh` runs (204, then `Running` inside 15 s at 33 % CPU), and
  it walks every library, so a library-scoped pass has no door on this build.
  That paragraph also records the null-pair read that separates "never ran" from
  "just finished", and that query+body on the same call answers `400` with an
  empty body whatever the values.
- The key law: a plugin owns its own config file under its own
  `<PluginConfiguration>` root and omits null settings, so TVDB's `<ApiKey>`
  element never existed and `install` refused — which is what `MISSING key` had
  been reporting since October. PR #121 fixes the writer.

The `Shows` art census replaces speculation with the split that matters: 2,908 of
3,558 series+season carry a primary, 624 have no provider id, 26 have an id and no
poster. A `Download` campaign over them took 20 s and downloaded nothing, which is
the measurement that says identification is the only lever left.
2026-11-02 04:47:16 -08:00
Frank e9314ea1ee fix(jellyfin): anchor the provider-key write on the file's real root
TheTVDB writes its own configuration file and its serializer omits a
null-valued setting entirely, so a fresh install has no <ApiKey> element to
fill, under a root element called <PluginConfiguration>. xml_secret's
fallback insertion was hardcoded to </Configuration>, matched nothing, and
install refused with "no place to write <ApiKey>" — measured on Dankserver
2026-11-02, where the TVDB key has been present in the secret file since
October and has never been installable.

The anchor now comes from the document's own root, so both shapes Jellyfin
actually produces work (the core plugins' <Configuration> and a plugin's
<PluginConfiguration>), and the refusal is kept for a file with no root at
all. Tests: the plugin-written shape is red before this commit and green
after; every other setting in the file is asserted preserved.
2026-11-02 04:38:40 -08:00
Frank 8889a4ceff docs(jellyfin): episode stills are out of scope, and the cost that says so
A per-episode RemoteImages query is never cached, so the same campaign that
dressed 620 series in an afternoon costs ~11 s per episode for one hit in five.
Across 42k episodes that is days of provider traffic to dress a fifth of them, and
clients show series and season art regardless. Recorded so nobody runs the pass on
a hunch; the door and the resumable log are the same if Frank orders it.
2026-11-02 03:00:42 -08:00
Frank f1ae1ddb36 docs(audiobookshelf): the scanner refusal names the file, not the bind
The first full scan logged "Invalid data found when processing input" per file and
carried on. That line is about the file: the scanner skips it, so the book never
appears in the library at all. Census of the 43 refused files: 38 were 0 bytes, 3
held WAV bytes behind a .mp3 name, 2 were unreadable. The check to run first is a
byte count, not a server investigation.

Full suite: 818 tests, exit 0.
2026-11-02 02:57:25 -08:00
Frank c21c48e61e docs(jellyfin): the door that scans is /Library/Refresh, and the art door names its field
The page sent the next agent to POST /Items/{id}/Refresh?Recursive=true per
library to make a new folder visible. Measured against a Shows library that had
never been identified: that route answers 204 and queues nothing — item census
byte-identical, ScheduledTasks idle, journal silent, and still a silent 204 after
a service restart. The door that works is POST /Library/Refresh, and the progress
read is ScheduledTasks -> Scan Media Library -> CurrentProgressPercentage.
RefreshStatus on the VirtualFolders record stays Idle through a real scan, so it
is not a progress read at all; a 204 cannot distinguish accepted-and-discarded
from accepted-and-queued, so the acceptance read is the journal naming folders.

Also recorded the art door's wire shape, because guessing it reads as a server
bug: the list response carries the address in Url (there is no ImageUrl), the
Download POST takes camelCase type/imageUrl and answers 204, and PascalCase or an
empty imageUrl answers 500. RemoteImages is empty for items with no ProviderIds,
so art coverage is capped by identification, not by the image door.
2026-11-02 02:56:12 -08:00
Frank 22cd17d064 docs(jellyfin): the door that scans is /Library/Refresh, and the art door names its field
The page sent the next agent to POST /Items/{id}/Refresh?Recursive=true per
library to make a new folder visible. Measured against a Shows library that had
never been identified: that route answers 204 and queues nothing — item census
byte-identical, ScheduledTasks idle, journal silent, and still a silent 204 after
a service restart. The door that works is POST /Library/Refresh, and the progress
read is ScheduledTasks -> Scan Media Library -> CurrentProgressPercentage.
RefreshStatus on the VirtualFolders record stays Idle through a real scan, so it
is not a progress read at all; a 204 cannot distinguish accepted-and-discarded
from accepted-and-queued, so the acceptance read is the journal naming folders.

Also recorded the art door's wire shape, because guessing it reads as a server
bug: the list response carries the address in Url (there is no ImageUrl), the
Download POST takes camelCase type/imageUrl and answers 204, and PascalCase or an
empty imageUrl answers 500. RemoteImages is empty for items with no ProviderIds,
so art coverage is capped by identification, not by the image door.
2026-11-02 02:56:03 -08:00
74 changed files with 4510 additions and 720 deletions

No files matched your search

+38 -7
View File
@@ -227,9 +227,28 @@ See `serverAppConfigs-docs/Architecture/Testing Policy.md`.
## External Exposure Policy (MANDATORY)
Every Caddy-proxied service is either behind the auth chain (Anubis :8923 →
Authentik embedded outpost) or explicitly internal-only — no public DNS name,
no vhost. New services default to internal-only until guarded.
Every Caddy-proxied service is in exactly one of three postures: behind the
auth chain (Anubis :8923 → Authentik embedded outpost), **exempt by
construction** because it authenticates itself (`auth.`, git smart-HTTP, Gitea,
OpenWebUI, Jellyfin, Plex, Audiobookshelf, the *arr stack), or internal-only —
served to nobody off-LAN. New services default to internal-only until guarded.
**Internal-only is a source gate, not a missing DNS record.** Caddy answers
every Host it knows, and its per-name certificates are published in Certificate
Transparency logs, so a name with no Porkbun A record is still in a stranger's
reach (`crt.sh` lists it). A household-only service therefore carries an edge
gate — `@lan remote_ip 127.0.0.1/32 ::1/128 192.168.0.0/16 10.0.0.0/8
172.16.0.0/12`, `handle @lan { reverse_proxy … }`, `handle { respond 404 }` —
which is what the *arr stack and qBittorrent carry (2026-11-04: Frank's law,
"never leaving the home boxes"). A future household-only vhost gets the same
block, not merely an absent record.
The guard chain is a **browser-only device**: Anubis demands a JavaScript
proof-of-work and the outpost works from a session cookie, so any non-browser
client — a native app, a podcast RSS reader, an OPDS client, `git`, a
service-to-service call — is answered with challenge HTML where it expected
JSON or a socket payload. A service that must serve such a client cannot sit
behind the chain; it has to own its authentication.
When guarding a new vhost — provider first, then route:
@@ -248,6 +267,10 @@ When guarding a new vhost — provider first, then route:
Exempt categories (document the reason in the vhost comment): the IdP itself
(`auth.`), git smart-HTTP endpoints, services with their own complete auth.
Unguarding a vhost is the mirror image of the procedure above — the route flips
first and the IdP rows go last — and `serverAppConfigs-docs/Infrastructure/Caddy.md`
*Unguard a vhost* owns the steps.
## Deletion & Trash Policy (MANDATORY)
Never `rm`. Move deletions into trash so they stay recoverable:
@@ -265,7 +288,10 @@ archive.
**Swap the model. Do not instrument it.** A payload change is: edit the trio
(env / serve script / unit) → `scripts/deploy.sh <host> --service <svc>` →
`/health` answers → one chat turn, one tool call, and one vision probe on an
endpoint that serves images. That is the entire procedure.
endpoint that serves images → re-key that endpoint's OpenWebUI `model` row
(`meta.description` + `params.context.length`, API door in [[OpenWebUI]]). That is
the entire procedure — no deployer step touches `webui.db`, and a stale
`context.length` makes OpenWebUI advertise a window the endpoint does not grant.
- **No benchmarking unless Frank orders it.** No pre-swap baseline battery,
no A/B arms, no decode/prefill ladders, no acceptance tables, no numbers
@@ -476,8 +502,8 @@ disagrees with it is the bug.
`scripts/frank-archon.service`, `scripts/ttm-preflight.sh`, template
`scripts/qwen38-froggeric-v22.5.jinja`; image digest-pinned
`stilldeadcode/radiance@sha256:d590e300…` (radiance 1.8.1); container name = unit name.
- Shape: TP=2 over `wht6`, `--max-num-seqs 2`, `--max-model-len 200000`,
`expert_tiered` placement, 44,928 MiB pinned host pool per rank, `fp8` KV,
- Shape: TP=2 over `wht6`, `--max-num-seqs 4`, `--max-model-len 200000`,
`expert_tiered` placement, 49,152 MiB pinned host pool per rank, `fp8` KV,
MTP depth 2, MNBT 2048, `--busy-prefill-chunk auto`, prefix cache 8 GiB host
tier only, n-gram table pinned to RAM. The per-card weights/KV split is derived
by the engine's budget resolver from `--gpu-headroom-mib`, the cache bound and
@@ -735,7 +761,12 @@ linked runbooks.
**Ops**
- Never `pkill -f` a pattern that appears in your own command line — it kills your ssh session.
Use `pkill -x <exact-name>`.
Use `pkill -x <exact-name>`, or the bracket trick — `pkill -f "serve[-]script.sh"` —
when the pattern has to be named: the bracket makes the pattern stop matching the
command line that carries it. Quote it: a zsh remote glob-expands an unquoted `[s]`
and the kill silently never runs. Re-hit 2026-11-02 —
`ssh host 'pkill -f jfshowswatch.py; …'` killed the remote shell mid-command, and
ssh exits **255 with zero output**, which reads exactly like a network fault.
- Both hosts allow passwordless sudo for the deploy user. Use plain `sudo` in committed scripts
(no `echo pw | sudo -S`); `FRANK_SUDO_PW` covers hosts without it.
- **ComfyUI on Dankclaw runs with GPU access** (`/dev/kfd` + `/dev/dri` via `AddDevice`,
+83 -15
View File
@@ -22,13 +22,16 @@
# CHALLENGE everyone else) → Caddy's internal listener http://:8016 (auth
# decision; Host headers survive every hop) → authenticated traffic is
# forwarded back to http://:8017 and routed straight to the app.
# Apps with their own auth (Gitea, OpenWebUI, Jellyfin, *arr, Pi-hole) and the
# Authentik IdP itself (auth.franksdank.cloud — its login flow must never sit
# behind a challenge) are exempt by construction: their vhosts never route
# through Anubis. ComfyUI left this exempt set on 2026-08-31 and Grafana left
# it on 2026-09-01 (OAuth2 relying-party mode against authentik): both now sit
# behind the guard chain; Grafana authenticates users itself via the OAuth2
# code flow, so no identity header is forwarded to it.
# Apps with their own auth (Gitea, OpenWebUI, Jellyfin, Plex, Audiobookshelf,
# *arr, Pi-hole) and the Authentik IdP itself (auth.franksdank.cloud — its login
# flow must never sit behind a challenge) are exempt by construction: their
# vhosts never route through Anubis. The chain is browser-only by nature — a
# native app client, a podcast RSS reader or a git client solves no JavaScript
# proof, so anything that must serve one has to own its own authentication.
# ComfyUI left this exempt set on 2026-08-31 and Grafana left it on 2026-09-01
# (OAuth2 relying-party mode against authentik): both now sit behind the guard
# chain; Grafana authenticates users itself via the OAuth2 code flow, so no
# identity header is forwarded to it.
# The pihole vhost keeps its basicauth edge; the hash placeholder below is
# rendered by deploy.sh from CADDY_BASICAUTH_HASH (host .env, git-ignored) into
# /home/dank/Containers/caddy/Caddyfile, which the container mounts.
@@ -264,20 +267,67 @@ files.franksdank.cloud {
}
}
# --- Jellyfin ---
# --- Audiobookshelf (exempt by construction: own auth) --------------------
# Caddy → app: no Anubis, no outpost. The guard chain answers a native client's
# /ping, /api/login and /socket.io/ handshake with challenge HTML, so behind it
# the phone app parsed a non-object where the socket `init` payload belongs and
# podcast RSS/OPDS feeds broke the same way. ABS authenticates itself (local
# accounts plus authentik OIDC), so it joins jelly/plex/*arr outside the chain
# (2026-11-03); the IdP rows that guarded this name — proxy provider
# `audiobooks-proxy` and application "Audiobookshelf Guard" — are retired with
# the change, after the route flips. Accepted exposure: /audiobookshelf/api/login
# is internet-visible with no rate limiter on either side (this Caddy build has
# no rate_limit module; ABS 2.37.2 ships no lockout) — the same class every other
# own-auth vhost here lives with, so shared accounts need long unique passwords.
# ROUTER_BASE_PATH=/audiobookshelf, so the bare root is a convenience hop onto it.
# X-Forwarded-Proto must say https: ABS derives its OIDC origin from
# x-forwarded-proto://Host and this hop is plain http, so without the pin its own
# https callback is a 400 "Invalid callback URL - must be same-origin".
audiobooks.franksdank.cloud {
redir / /audiobookshelf/ permanent
reverse_proxy 192.168.0.235:8000 {
header_up X-Forwarded-Proto https
}
}
# --- Jellyfin (exempt by construction: own auth) --------------------------
# No Anubis, no outpost: Jellyfin carries its own complete login (its own
# account store, no Authentik link), which is the policy's exempt category —
# the same family as plex/ABS/*arr around it. Accepted exposure: the sign-in
# form is internet-visible with no rate limiter on either side (this Caddy
# build ships no rate_limit module), so accounts need long unique passwords.
# The homepage card opened it to the household on 2026-11-03; accounts are
# created in Jellyfin's own admin UI, not by SSO.
jelly.franksdank.cloud {
reverse_proxy 192.168.0.235:8096
}
# --- *arr Stack (exempt by construction: own auth) ---
# --- *arr Stack + qBittorrent: LAN-ONLY at the edge (2026-11-04) -----------
# Frank's law: these never leave the home boxes. DNS secrecy is NOT a gate —
# Caddy issues one certificate per name and every one is in public CT logs
# (crt.sh lists prowlarr/radarr/lidarr/readarr/qbittorrent.franksdank.cloud),
# so a stranger can send the Host. The gate is the source address: a LAN client
# proxies through, anything off-LAN gets 404. `remote_ip` reads the real client
# address because the router DNATs the forward (no proxy hop in between).
prowlarr.franksdank.cloud {
reverse_proxy 192.168.0.112:9696
@lan remote_ip 127.0.0.1/32 ::1/128 192.168.0.0/16 10.0.0.0/8 172.16.0.0/12
handle @lan {
reverse_proxy 192.168.0.112:9696
}
handle {
respond 404
}
}
radarr.franksdank.cloud {
reverse_proxy 192.168.0.112:7878
@lan remote_ip 127.0.0.1/32 ::1/128 192.168.0.0/16 10.0.0.0/8 172.16.0.0/12
handle @lan {
reverse_proxy 192.168.0.112:7878
}
handle {
respond 404
}
}
# --- Sonarr: internal-only by construction (retired 2026-09-04) -----------
@@ -288,15 +338,33 @@ radarr.franksdank.cloud {
# zero-auth UI to the internet via manual DNS — retired.
lidarr.franksdank.cloud {
reverse_proxy 192.168.0.112:8686
@lan remote_ip 127.0.0.1/32 ::1/128 192.168.0.0/16 10.0.0.0/8 172.16.0.0/12
handle @lan {
reverse_proxy 192.168.0.112:8686
}
handle {
respond 404
}
}
readarr.franksdank.cloud {
reverse_proxy 192.168.0.112:8787
@lan remote_ip 127.0.0.1/32 ::1/128 192.168.0.0/16 10.0.0.0/8 172.16.0.0/12
handle @lan {
reverse_proxy 192.168.0.112:8787
}
handle {
respond 404
}
}
qbittorrent.franksdank.cloud {
reverse_proxy 192.168.0.112:8080
@lan remote_ip 127.0.0.1/32 ::1/128 192.168.0.0/16 10.0.0.0/8 172.16.0.0/12
handle @lan {
reverse_proxy 192.168.0.112:8080
}
handle {
respond 404
}
}
# --- Internal listener: auth decision layer ---------------------------------
+1
View File
@@ -26,6 +26,7 @@
192.168.0.143 romm.franksdank.cloud
192.168.0.143 files.franksdank.cloud
192.168.0.143 jelly.franksdank.cloud
192.168.0.143 audiobooks.franksdank.cloud
192.168.0.143 radarr.franksdank.cloud
192.168.0.143 lidarr.franksdank.cloud
192.168.0.143 readarr.franksdank.cloud
@@ -33,9 +33,16 @@ Network=pasta
# README is the npm package's naming and is INERT here: with ABS_PORT=8000 the
# server listened on :80 under base path /audiobookshelf, so the unit sat
# `active (running)` while every probe to :8000 was refused (measured 2026-11-02,
# first deploy). ROUTER_BASE_PATH=/ is what makes the URL the plain host:port.
# first deploy).
# ROUTER_BASE_PATH MUST be /audiobookshelf: the SPA in /app/client/dist/index.html is
# built with `routerBasePath:"/audiobookshelf"` baked in, so it asks for its assets under
# that prefix no matter what the server is mounted at. With `/` the server answers the
# page but every script request falls through to the SPA fallback and comes back as
# text/html — Chrome refuses to execute it and the UI never leaves "Loading…"
# (measured 2026-11-03; the API and /healthcheck were fine, which is why an API-only
# verification missed it).
Environment=PORT=8000
Environment=ROUTER_BASE_PATH=/
Environment=ROUTER_BASE_PATH=/audiobookshelf
Environment=CONFIG_PATH=/config
Environment=METADATA_PATH=/metadata
Environment=TZ=America/Vancouver
@@ -45,6 +52,8 @@ Volume=/home/dank/Containers/audiobookshelf/metadata:/metadata
Volume=/mnt/entropy/Audiobooks:/audiobooks:ro
# The image is Alpine: it ships wget and no curl, so the health check must name
# the binary that exists or the container is unhealthy while serving perfectly.
# /healthcheck answers `OK` at the root AND under the base path (measured on both
# base paths), so the probe below is correct whatever ROUTER_BASE_PATH says.
HealthCmd=wget -q -O /dev/null http://127.0.0.1:8000/healthcheck
HealthInterval=30s
HealthTimeout=10s
@@ -47,6 +47,12 @@ Volume=/home/dank/Containers/jellyfin/cache:/cache
# is the guard: started against a plain, unmounted view directory a scan decides
# the library is empty and deletes what it cannot see.
Volume=/var/lib/media-meta/view:/data
# Big-screen theme for the TV. The repo file replaces the container's "Media
# Center" theme slot; the upstream dark palette stays live because the file
# @imports it, so an image bump cannot strand a vendored CSS snapshot. Read-only
# — the web app only fetches it. A client opts in with
# Display Settings -> Theme -> "Media Center".
Volume=/home/dank/serverAppConfigs/dankserver/jellyfin/theme/wmc/theme.css:/jellyfin/jellyfin-web/themes/wmc/theme.css:ro
AddDevice=/dev/dri
GroupAdd=keep-groups
HealthCmd=curl -f http://localhost:8096/health
+53
View File
@@ -0,0 +1,53 @@
/*
* Dank fleet big-screen theme for jellyfin-web.
*
* Deployed by bind-mounting THIS file over the container's "Media Center"
* theme (`/jellyfin/jellyfin-web/themes/wmc/theme.css`) — see the Volume line
* in dankserver/jellyfin/quadlet/jellyfin.container. The slot is deliberately
* one nobody picks by accident, and its name is the only thing that changes:
* the colors are the live upstream dark theme, pulled in by @import below, so
* this file never vendors a snapshot of upstream CSS and cannot drift when the
* Jellyfin image moves.
*
* Why it exists: the living-room client (Jellyfin Desktop on danklivingroom)
* renders this server-served web app at 3840x2160 with no client-side scaling,
* so 14 px body text lands on a big panel viewed from three metres. `zoom` on
* the root element is the one lever that scales the whole UI — the layout code
* measures layout px, so card grids re-flow to fewer, larger columns instead of
* overflowing.
*
* Adoption is per device, not global: Display Settings -> Theme ->
* "Media Center" on the screen that wants it. That is the point — a phone
* browser on the same server must not get a 1.75x interface.
*
* Change the size by editing the single number. 1.5 is mild, 1.75 is the
* three-metre default, 2.0 is for a projector.
*/
@import url("../dark/theme.css");
:root {
/* Marker the e2e gate reads back; also a tweak hook for a per-client
override injected by anyone who really wants one. */
--dank-big-screen: 1;
--dank-big-screen-zoom: 1.75;
}
html {
zoom: var(--dank-big-screen-zoom);
}
/* A remote moves focus in discrete jumps, so the ring has to survive a
three-metre read. */
.emby-focus,
button:focus-visible,
a:focus-visible,
.inputElement:focus-visible {
outline: 3px solid var(--accentColor, #00a4dc) !important;
outline-offset: 2px !important;
}
/* Scrollbars are mouse furniture; on a TV they are noise at this size. */
* {
scrollbar-width: none;
}
+204
View File
@@ -0,0 +1,204 @@
#!/usr/bin/env python3
"""abs-post-reorg.py — drive the Audiobookshelf side of the 2026-11-04 relayout.
Runs ON Dankserver (the ABS host). Credentials come from the machine-local
file ~/.config/secrets/audiobookshelf and never enter argv; requests go
through urllib so neither the password nor the bearer token is world-readable
in /proc. The delete arm is the DB-row arm ONLY — `hard=1` is fs.remove() of
the item folder and must never reach this file (see the Audiobookshelf vault
page; after the move the covering paths are whole books).
Subcommands
covers --list LIST.tsv copy each ABS /metadata cover onto the book
folder ("srchost<TAB>dstarray" per line);
run BEFORE the mover so the cover rides the
book's move, or AFTER with dst already final
wipe [--apply] delete every library item DB row (progress
included — the campaign accepted losing the
11 progress rows); refuses without --apply
scan [--wait] trigger the library scan; --wait polls
isFetchingLibrary then prints the item total
status item total + isMissing census
Paging law (measured twice): page on `page` (0-based) + `limit`; `offset` is
ignored outright. Count unique ids; stop on `total`.
"""
import argparse
import json
import os
import shutil
import sys
import time
import urllib.error
import urllib.request
BASE = "http://127.0.0.1:8000/audiobookshelf"
SECRET = os.environ.get("ABS_SECRET", "/home/dank/.config/secrets/audiobookshelf")
def cred():
kv = {}
with open(SECRET) as fh:
for line in fh:
if "=" in line:
k, _, v = line.strip().partition("=")
kv[k] = v
return kv.get("ABS_ADMIN_USER", "frank"), kv.get("ABS_ADMIN_PASSWORD")
def call(method, path, body=None, token=None, base=BASE):
data = json.dumps(body).encode() if body is not None else None
req = urllib.request.Request(base + path, data=data, method=method)
req.add_header("Content-Type", "application/json")
if token:
req.add_header("Authorization", f"Bearer {token}")
try:
with urllib.request.urlopen(req, timeout=30) as r:
raw = r.read().decode()
try:
return r.status, json.loads(raw)
except json.JSONDecodeError:
return r.status, raw.strip()
except urllib.error.HTTPError as e:
return e.code, e.read().decode()[:200]
def login():
user, password = cred()
if not password:
sys.exit(f"refused: {SECRET} carries no ABS_ADMIN_PASSWORD")
code, body = call("POST", "/login", {"username": user, "password": password})
if code != 200:
sys.exit(f"login failed {code}: {body}")
return body["user"]["token"] # the token is inside user, not top level
def library_id(token, name="Audiobooks"):
code, libs = call("GET", "/api/libraries", token=token)
if code != 200:
sys.exit(f"libraries failed {code}: {libs}")
for lib in libs.get("libraries", []):
if lib["name"] == name:
return lib["id"]
sys.exit(f"no library named {name}")
def all_item_ids(token, lid):
ids, page = [], 0
code, one = call("GET", f"/api/libraries/{lid}/items?limit=1", token=token)
total = one.get("total", -1) if code == 200 else -1
while True:
code, body = call("GET", f"/api/libraries/{lid}/items?page={page}&limit=500",
token=token)
if code != 200:
sys.exit(f"items page {page} failed {code}")
results = body.get("results", [])
if not results:
break
ids += [it["id"] for it in results]
page += 1
if total >= 0 and page * 500 >= total:
break
return list(dict.fromkeys(ids)), total
def cmd_covers(args):
copied = missing = exists = 0
for line in open(args.list):
src, _, dst = line.rstrip("\n").partition("\t")
if not dst:
continue
if os.path.exists(dst):
exists += 1
continue
if not os.path.exists(src):
missing += 1
continue
os.makedirs(os.path.dirname(dst), exist_ok=True)
shutil.copy2(src, dst)
copied += 1
print(f"covers copied: {copied} already there: {exists} missing source: {missing}")
def cmd_wipe(args, token, lid):
ids, total = all_item_ids(token, lid)
print(f"library {lid}: {len(ids)} items (total={total})")
if not args.apply:
print("dry run: nothing deleted (add --apply)")
return
gone = 0
for i, iid in enumerate(ids):
code, _ = call("DELETE", f"/api/items/{iid}", token=token)
if code == 200:
gone += 1
elif i < 3:
print(f"delete {iid}: {code}")
if i % 500 == 499:
print(f" {i + 1}/{len(ids)} deleted")
print(f"deleted {gone}/{len(ids)} item rows")
def cmd_scan(args, token, lid):
code, body = call("POST", f"/api/libraries/{lid}/scan", token=token)
print(f"scan: {code} {body if code != 200 else 'accepted'}")
if not args.wait:
return
last = -1
while True:
time.sleep(30)
code, lib = call("GET", f"/api/libraries/{lid}", token=token)
code2, one = call("GET", f"/api/libraries/{lid}/items?limit=1", token=token)
total = one.get("total", -1) if code2 == 200 else -1
fetching = lib.get("isFetchingLibrary") if code == 200 else None
print(f" fetching={fetching} items={total}")
if fetching is False and total == last and total >= 0:
break
last = total
print(f"scan settled at {total} items")
def cmd_status(token, lid):
missing = 0
seen, page = set(), 0
while True:
code, body = call("GET", f"/api/libraries/{lid}/items?page={page}&limit=500",
token=token)
results = body.get("results", []) if code == 200 else []
if not results:
break
for it in results:
if it["id"] not in seen:
seen.add(it["id"])
if it.get("isMissing"):
missing += 1
page += 1
code, lib = call("GET", f"/api/libraries/{lid}", token=token)
print(f"items: {len(seen)} isMissing: {missing} "
f"pending tasks: {lib.get('pendingTasks') if code == 200 else '?'}")
def main():
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
sub = ap.add_subparsers(dest="cmd", required=True)
c = sub.add_parser("covers")
c.add_argument("--list", required=True)
w = sub.add_parser("wipe")
w.add_argument("--apply", action="store_true")
s = sub.add_parser("scan")
s.add_argument("--wait", action="store_true")
sub.add_parser("status")
args = ap.parse_args()
token = login()
lid = library_id(token)
if args.cmd == "covers":
cmd_covers(args)
elif args.cmd == "wipe":
cmd_wipe(args, token, lid)
elif args.cmd == "scan":
cmd_scan(args, token, lid)
else:
cmd_status(token, lid)
if __name__ == "__main__":
main()
+127 -4
View File
@@ -21,26 +21,64 @@
# The credential is generated once into a mode-600 host file and never enters
# the repo, the URL, or a process argument list. Same pattern as calibre-server.
#
# Single sign-on (authentik) is armed by the same run, and only when the credential
# file carries ABS_OIDC_CLIENT_SECRET, so a local-only deployment is unchanged by a
# re-run. These OIDC facts came from the same pinned image, and each one is a trap:
# * isOpenIDAuthSettingsValid demands issuer + authorization + token + userinfo +
# jwks + clientID + clientSecret + signingAlgorithm at once. Miss one and the
# server SILENTLY strips 'openid' from authActiveAuthMethods and falls back to
# ['local'] while answering 200 everywhere — so the only honest check is what
# /status says AFTER the write, which is what this script does.
# * the algorithm is HS256, not the image's RS256 default: no provider on the
# fleet's authentik carries a signing key, so the client secret IS the signature
# key (scripts/authentik-audiobookshelf-sso.sh). Left at RS256, every id_token
# fails verification and the failure looks like a broken IdP.
# * the auth block lives on PATCH /api/auth-settings. PATCH /api/settings has a
# 20-key allowlist (PATCHABLE_SETTINGS_KEYS) with NO auth key in it, so writing
# the OIDC config there answers 200 and changes nothing — measured on this host
# 2026-11-02. The keys /api/auth-settings accepts are exactly its own
# authenticationSettings getters; anything else is skipped with a debug line.
# * authOpenIDSubfolderForRedirectURLs must be sent explicitly. The 2.17.4 migration only
# fills it on a DB that ALREADY had OIDC on, so a database created before SSO leaves it
# UNDEFINED and the server builds redirect_uri
# "http://host/undefined/auth/openid/callback" — authentik answers the authorize request
# with "Redirect URI Error" (measured 2026-11-02). This script derives the value from
# ABS_URL's path, so it cannot drift from the unit's ROUTER_BASE_PATH.
# * 'local' stays in authActiveAuthMethods on purpose: if authentik is ever down,
# the root account must still be able to sign into its own server.
#
# usage: audiobookshelf-init.sh # create what is missing, then scan
# ABS_SCAN=0 audiobookshelf-init.sh # bootstrap without a scan
# ABS_OIDC=0 audiobookshelf-init.sh # never touch the auth methods
#
# environment (defaults are the deployed shape):
# ABS_URL server base URL (http://127.0.0.1:8000)
# ABS_URL server base URL INCLUDING the mounted base path
# (http://127.0.0.1:8000/audiobookshelf)
# ABS_SECRET credential file (/home/dank/.config/secrets/audiobookshelf)
# ABS_LIBRARY library name to ensure (Audiobooks)
# ABS_FOLDER folder to bind inside the server (/audiobooks)
# ABS_SCAN 1 triggers a scan (default), 0 skips it
# ABS_OIDC auto (default) arms SSO when the credential file has a client
# secret, 1 requires one, 0 leaves the auth methods alone
# ABS_OIDC_ISSUER authentik issuer URL for this client
# ABS_OIDC_CLIENT_ID provider client id (audiobookshelf)
# ABS_OIDC_ALG id_token signing algorithm the IdP actually uses (HS256)
# ABS_OIDC_SUBFOLDER override for the redirect-URI subfolder (default: ABS_URL's path)
#
# exit 0 every step is satisfied (created or already present)
# exit 1 a step was refused or the server misbehaved (named on stderr)
# exit 2 usage / unreadable credential file
set -euo pipefail
ABS_URL=${ABS_URL:-http://127.0.0.1:8000}
ABS_URL=${ABS_URL:-http://127.0.0.1:8000/audiobookshelf}
ABS_SECRET=${ABS_SECRET:-/home/dank/.config/secrets/audiobookshelf}
ABS_LIBRARY=${ABS_LIBRARY:-Audiobooks}
ABS_FOLDER=${ABS_FOLDER:-/audiobooks}
ABS_SCAN=${ABS_SCAN:-1}
ABS_OIDC=${ABS_OIDC:-auto}
ABS_OIDC_ISSUER=${ABS_OIDC_ISSUER:-https://auth.franksdank.cloud/application/o/audiobookshelf/}
ABS_OIDC_CLIENT_ID=${ABS_OIDC_CLIENT_ID:-audiobookshelf}
ABS_OIDC_ALG=${ABS_OIDC_ALG:-HS256}
die() { printf 'refused: %s\n' "$*" >&2; exit 1; }
@@ -49,6 +87,11 @@ case "$ABS_SCAN" in
*) printf 'refused: ABS_SCAN wants 0 or 1, got %s\n' "$ABS_SCAN" >&2; exit 2 ;;
esac
case "$ABS_OIDC" in
auto | 0 | 1) ;;
*) printf 'refused: ABS_OIDC wants auto, 0 or 1, got %s\n' "$ABS_OIDC" >&2; exit 2 ;;
esac
# ---- credential -------------------------------------------------------------
umask 077
if [ ! -f "$ABS_SECRET" ]; then
@@ -72,8 +115,10 @@ chmod 600 "$ABS_SECRET"
# python3 rather than curl so the password and the bearer token never appear in
# any argv (they would be world-readable in /proc/<pid>/cmdline otherwise).
ABS_URL="$ABS_URL" ABS_SECRET="$ABS_SECRET" ABS_LIBRARY="$ABS_LIBRARY" \
ABS_FOLDER="$ABS_FOLDER" ABS_SCAN="$ABS_SCAN" python3 - <<'PY'
import json, os, sys, urllib.error, urllib.request
ABS_FOLDER="$ABS_FOLDER" ABS_SCAN="$ABS_SCAN" ABS_OIDC="$ABS_OIDC" \
ABS_OIDC_ISSUER="$ABS_OIDC_ISSUER" ABS_OIDC_CLIENT_ID="$ABS_OIDC_CLIENT_ID" \
ABS_OIDC_ALG="$ABS_OIDC_ALG" python3 - <<'PY'
import json, os, sys, urllib.error, urllib.parse, urllib.request
BASE = os.environ["ABS_URL"].rstrip("/")
LIB = os.environ["ABS_LIBRARY"]
@@ -184,5 +229,83 @@ code, items = call("GET", "/api/libraries/%s/items?limit=1" % lib["id"], token=t
if code == 200 and isinstance(items, dict):
print("items: %s (a fresh scan raises this from 0; watch it, not /healthcheck)"
% items.get("total"))
# ---- single sign-on (authentik) --------------------------------------------------
# The provider this points at is provisioned by scripts/authentik-audiobookshelf-sso.sh;
# the two share exactly one value, the client secret in the credential file.
mode = os.environ["ABS_OIDC"]
oidc_secret = cred.get("ABS_OIDC_CLIENT_SECRET") or ""
if mode == "0":
print("oidc: skipped (ABS_OIDC=0)")
elif not oidc_secret:
if mode == "1":
sys.exit("refused: ABS_OIDC=1 but %s has no ABS_OIDC_CLIENT_SECRET — run "
"scripts/authentik-audiobookshelf-sso.sh first" % os.environ["ABS_SECRET"])
print("oidc: not configured (no ABS_OIDC_CLIENT_SECRET in the credential file)"
" — local auth only")
else:
client_id = os.environ["ABS_OIDC_CLIENT_ID"]
issuer = os.environ["ABS_OIDC_ISSUER"].rstrip("/") + "/"
# redirect_uri = <origin><subfolder>/auth/openid/callback, and the subfolder IS the
# base path the app is mounted under — taken from ABS_URL so unit and config agree.
subfolder = os.environ.get("ABS_OIDC_SUBFOLDER")
if subfolder is None:
subfolder = urllib.parse.urlparse(os.environ["ABS_URL"]).path.rstrip("/")
origin = issuer.split("/application/")[0]
body = {
"authActiveAuthMethods": ["local", "openid"],
"authOpenIDIssuerURL": issuer,
"authOpenIDAuthorizationURL": origin + "/application/o/authorize/",
"authOpenIDTokenURL": origin + "/application/o/token/",
"authOpenIDUserInfoURL": origin + "/application/o/userinfo/",
"authOpenIDJwksURL": issuer + "jwks/",
"authOpenIDLogoutURL": issuer + "end-session/",
"authOpenIDClientID": client_id,
"authOpenIDClientSecret": oidc_secret,
"authOpenIDTokenSigningAlgorithm": os.environ["ABS_OIDC_ALG"],
"authOpenIDSubfolderForRedirectURLs": subfolder,
"authOpenIDButtonText": "Sign in with Authentik",
# not auto-launch: the login page keeps offering the local form, which is the
# break-glass path when the IdP is unreachable.
"authOpenIDAutoLaunch": False,
# a household user's first SSO sign-in creates their account. The role comes from
# the abs_groups claim, which grants the listener role to every signer; only the
# authentik admin group escalates. Revoking a listener therefore happens HERE in
# ABS (disable or delete the user), never by removing an IdP group membership.
"authOpenIDAutoRegister": True,
"authOpenIDGroupClaim": "abs_groups",
# The app's own scheme lives HERE, never in the IdP: ABS swaps it for
# <base>/auth/openid/mobile-redirect before the provider sees a redirect_uri.
"authOpenIDMobileRedirectURIs": ["audiobookshelf://oauth"],
}
code, res = call("PATCH", "/api/auth-settings", body, token=token)
if code != 200:
sys.exit("refused: PATCH /api/auth-settings answered %s (%s)" % (code, res))
# Verify twice: what was stored, and what the server will honour. A config that
# misses any of the eight required OIDC values is accepted with 200 and then
# silently dropped at the next load, leaving authMethods=['local'].
code, stored = call("GET", "/api/auth-settings", token=token)
if code != 200 or not isinstance(stored, dict):
sys.exit("refused: GET /api/auth-settings answered %s (%s)" % (code, stored))
for key, want in (("authOpenIDIssuerURL", issuer),
("authOpenIDTokenSigningAlgorithm", os.environ["ABS_OIDC_ALG"]),
("authOpenIDSubfolderForRedirectURLs", subfolder),
("authOpenIDButtonText", "Sign in with Authentik")):
if stored.get(key) != want:
sys.exit("refused: stored %s=%r, expected %r" % (key, stored.get(key), want))
if sorted(stored.get("authActiveAuthMethods") or []) != ["local", "openid"]:
sys.exit("refused: stored authActiveAuthMethods=%r, expected local+openid "
"('openid' is dropped when the block is incomplete)"
% stored.get("authActiveAuthMethods"))
code, after = call("GET", "/status")
methods = after.get("authMethods") if isinstance(after, dict) else None
if not isinstance(methods, list) or "openid" not in methods:
sys.exit("refused: /status reports authMethods=%s after a stored config that "
"looks complete — restart the service, the strategy is only armed at "
"load for a config that was invalid when the server read it" % methods)
print("oidc: armed (issuer %s | alg %s | claim abs_groups | redirect subfolder %r | "
"client %s | secret %s chars | authMethods %s)"
% (issuer, os.environ["ABS_OIDC_ALG"], subfolder, client_id, len(oidc_secret),
methods))
PY
exit 0
+294
View File
@@ -0,0 +1,294 @@
#!/usr/bin/env bash
# authentik-audiobookshelf-sso.sh — provision the authentik side of Audiobookshelf SSO.
#
# Why a script: this state lives in authentik's DATABASE, not in any repo file, and a
# blueprint re-import or a rebuild-from-scratch reverts hand-made provider rows
# silently. Same reasoning (and same door) as
# authentik-set-login-session-duration.sh: ssh into Dankclaw, `ak shell` from stdin.
# Re-run this after such a rebuild.
#
# The contract with Audiobookshelf, read off the pinned image (2.37.2,
# dist-server/server/auth/OidcAuthStrategy.js + objects/settings/ServerSettings.js)
# rather than a forum, because each of these is a dead-login trap:
#
# * Two URIs per origin ABS is reached at — <base>/auth/openid/callback and
# <base>/auth/openid/mobile-redirect — for EVERY entry in BASES (the LAN address
# and the public vhost). ABS SWAPS the app's own audiobookshelf://oauth for the
# mobile-redirect URL before the IdP ever sees a redirect_uri, so registering the
# app scheme with authentik — the obvious move — is what makes phone logins fail.
# The app scheme belongs in ABS's authOpenIDMobileRedirectURIs list (its built-in
# default), not here. The mobile URI exists on every origin because the app sends
# the one belonging to the door it was configured with: the vhost started serving
# non-browser clients on 2026-11-03 (it left the guard chain), so a phone pointed
# at the public name sends the public mobile-redirect URI, and STRICT matching
# answers an unregistered one with "Redirect URI Error".
# * <base> CARRIES THE BASE PATH. ABS builds redirect_uri as
# <origin><authOpenIDSubfolderForRedirectURLs>/auth/openid/callback, and that subfolder is
# the ROUTER_BASE_PATH the container is mounted at — /audiobookshelf, which is also the only
# base path the shipped SPA can boot under. Left unset (the fresh-DB default) the URL
# becomes "/undefined/…"; a provider registered without the prefix mismatches the same way
# (measured 2026-11-02, "Redirect URI Error"). audiobookshelf-init.sh derives its copy from
# ABS_URL, so the two sides each move with one variable.
# * ABS requests the scopes "openid profile email <GroupClaim> [<PermsClaim>]": the
# CLAIM NAME is a scope name. So authentik needs a scope mapping whose scope_name
# is exactly the claim name, or the claim never arrives and the callback dies on
# "Group claim <name> not found in userinfo".
# * the claim's VALUES must be ABS role names — the callback picks the first of
# ['admin','user','guest'] present in the list. Group names like
# audiobookshelf_admins are therefore translated by the mapping, never passed
# through. A user in neither group gets [] and is denied (fail closed).
# * tokens are HS256 here: no provider on this box carries a signing_key (census
# 2026-11-02), which makes the client secret the signature key. ABS's own default
# is RS256, so the ABS side MUST be set to HS256 or every id_token fails
# verification.
# * provider.authorization_flow must be non-NULL or /application/o/authorize/
# answers a 500 with 'NoneType' object has no attribute 'slug' (measured on
# grafana, same authentik build).
# * provider.grant_types MUST contain authorization_code. authentik 2026.8 defaults an
# ORM-created provider to the EMPTY list, and /application/o/authorize/ then refuses the
# code flow with oauthlib's generic "The request is otherwise malformed"; the only line
# that names the cause is the server log's 'Invalid grant_type for provider'. The
# redirect-URI check runs FIRST, so a provider carrying both faults shows only the URI
# error until the URI is fixed (measured 2026-11-02, where the login survived the URI fix
# and died here).
# * property_mappings must carry the built-in OpenID scope mappings ('openid', 'profile',
# 'email') alongside the claim mapping. A provider holding only the claim mapping grants a
# token scoped shorter than the scopes ABS requests, and ABS's /userinfo call then dies
# 'insufficient_scope (The request requires higher privileges than provided by the access
# token)' after authentik has already issued a good code (measured 2026-11-02).
#
# Idempotent: every object is created or asserted. Safe to re-run.
#
# usage: ABS_OIDC_CLIENT_SECRET=<secret> authentik-audiobookshelf-sso.sh
# (the secret is the value stored in
# /home/dank/.config/secrets/audiobookshelf on Dankserver — get it with
# ssh dankserver awk -F= '/^ABS_OIDC_CLIENT_SECRET=/{print $2}' \
# ~/.config/secrets/audiobookshelf)
#
# environment:
# ABS_OIDC_CLIENT_SECRET REQUIRED. The provider's client secret.
# ABS_OIDC_CLIENT_ID provider/client id (audiobookshelf)
# ABS_OIDC_GROUPS_CLAIM claim ABS reads for roles (abs_groups)
# ABS_OIDC_USERNAME authentik user made admin (akadmin)
# There is deliberately NO user list. The claim grants the LISTENER role to every
# authentik account that signs in (2026-11-03), so onboarding a listener is exactly
# "create the authentik account". Before this, a household account authenticated at
# the IdP and was then refused by ABS for being in no group — a policy decision that
# read as a broken login. Revocation lives on the ABS side (disable or delete the
# user there): a group can no longer withhold a role the claim always sends.
# ABS_E2E_ADMIN authentik user kept in the ADMIN group for the Playwright gate
# (qa-e2e-admin — tests/e2e/specs/23-audiobookshelf-sso.spec.ts
# signs in as this identity, so a re-run restores the fixture)
# ABS_BASE ABS URL as the browser sees it, base path included
# (http://192.168.0.235:8000/audiobookshelf — the LAN
# origin, reached directly, which is also what the phone
# app is configured with)
# ABS_EXTRA_BASES space-separated additional origins, base path included
# (default: https://audiobooks.franksdank.cloud/audiobookshelf
# — the guarded vhost, dankclaw/Caddyfile).
# ABS builds its redirect_uri from the ORIGIN of the
# request, so each origin the app is served at needs its
# own registered URI — an unregistered origin is a
# redirect_uri_mismatch at the IdP, before any credential.
# Set it empty to register the LAN origin alone.
# AUTHENTIK_HOST ssh target (dankclaw)
set -euo pipefail
SECRET="${ABS_OIDC_CLIENT_SECRET:?set ABS_OIDC_CLIENT_SECRET (see the header)}"
[ "${#SECRET}" -ge 32 ] || { echo "refused: ABS_OIDC_CLIENT_SECRET is too short to be a secret" >&2; exit 2; }
CLIENT_ID="${ABS_OIDC_CLIENT_ID:-audiobookshelf}"
CLAIM="${ABS_OIDC_GROUPS_CLAIM:-abs_groups}"
ADMIN_GROUP="audiobookshelf_admins"
SSO_USER="${ABS_OIDC_USERNAME:-akadmin}"
E2E_ADMIN="${ABS_E2E_ADMIN:-qa-e2e-admin}"
BASE="${ABS_BASE:-http://192.168.0.235:8000/audiobookshelf}"
EXTRA_BASES="${ABS_EXTRA_BASES:-https://audiobooks.franksdank.cloud/audiobookshelf}"
HOST="${AUTHENTIK_HOST:-dankclaw}"
# The claim expression: every signer is a listener; only the admin group escalates.
# Stored verbatim so a later read of the provider shows exactly this policy. The default
# lives HERE and nowhere else — ABS reads this list off the token, so an account in no
# Audiobookshelf group still arrives with ["user"] and is auto-registered as a listener
# instead of being refused after a successful authentication.
read -r -d '' EXPR <<PYEXPR || true
groups = {g.name for g in user.groups.all()}
# Order is deliberate: an admin's claim comes out ["admin","user"], byte-identical to what
# this mapping emitted before the default existed, because ABS's group-tag matching has never
# been measured for order-sensitivity and the working admin login is not the place to find out.
roles = []
if "$ADMIN_GROUP" in groups:
roles.append("admin")
roles.append("user")
return {"$CLAIM": roles}
PYEXPR
# NOTE: the heredoc below is UNQUOTED on purpose — its body interpolates the variables
# above. The consequence is that a backtick anywhere in that python body is a LOCAL command
# substitution whose output is spliced into the program before it ever runs, and comments
# are not exempt, because bash does not know what a python comment is. Write names plainly
# in the body: measured 2026-11-03, two comments in this body had the shell run
# audiobookshelf_users and user as commands on every execution of this script.
ssh "$HOST" 'podman exec -i authentik-server ak shell' <<PYEOF
from authentik.core.models import Application, Group, User
from authentik.flows.models import Flow
from authentik.providers.oauth2.models import (
ClientType, OAuth2Provider, RedirectURI, RedirectURIMatchingMode, RedirectURIType, ScopeMapping)
ADMIN_GROUP = "$ADMIN_GROUP"
CLAIM, CLIENT_ID, SECRET, BASE = "$CLAIM", "$CLIENT_ID", "$SECRET", "$BASE"
BASES = [BASE] + """$EXTRA_BASES""".split()
SSO_USER, E2E_ADMIN = "$SSO_USER", "$E2E_ADMIN"
EXPR = '''$EXPR'''
# The expression reads user.groups, NOT user.ak_groups: authentik 2026.8 logs
# "User.ak_groups is deprecated … use User.groups" on every evaluation of the old
# name (measured on this run), and a stored mapping that leans on a deprecated
# accessor is a future upgrade failure sitting inside the login path.
Group.objects.get_or_create(name=ADMIN_GROUP)
admin_g = Group.objects.get(name=ADMIN_GROUP)
# The audiobookshelf_users group is retired. Its membership used to be the only door to the
# listener role; the claim now sends that role to every signer, so the group decides
# nothing — and a group that decides nothing is a trap for the next reader, who will
# assume it gates something and start adding people to it.
for stale in Group.objects.filter(name="audiobookshelf_users"):
stale.delete()
print("group: retired audiobookshelf_users (the claim grants every signer)")
# --- the claim mapping (scope_name IS the claim name ABS requests) ---------------
mapping = ScopeMapping.objects.filter(name="audiobookshelf-groups").first()
if mapping is None:
mapping = ScopeMapping.objects.create(name="audiobookshelf-groups", scope_name=CLAIM,
expression=EXPR,
description="Audiobookshelf roles: every signer is a listener, the admin group escalates")
print("mapping: created (scope_name %s)" % CLAIM)
elif mapping.scope_name != CLAIM or mapping.expression != EXPR:
mapping.scope_name, mapping.expression = CLAIM, EXPR
mapping.save(update_fields=["scope_name", "expression"])
print("mapping: scope/expression updated (scope_name %s)" % CLAIM)
else:
print("mapping: already current")
# Prove the expression runs and yields an ABS role BEFORE anyone tries to log in:
# authentik execs the expression inside a function whose context has the name "user".
_src = "def _probe(user, request=None):\n" + "".join(" %s\n" % ln for ln in EXPR.splitlines())
_ns = {}
exec(_src, _ns)
# --- provider --------------------------------------------------------------------
# One callback pair per origin (LAN address and the public vhost): ABS derives the
# browser callback from the request origin and the mobile callback from the door the
# app was configured with, so each origin needs both or that door cannot sign in.
uris = [RedirectURI(matching_mode=RedirectURIMatchingMode.STRICT, url="%s%s" % (b, p))
for b in BASES
for p in ("/auth/openid/callback", "/auth/openid/mobile-redirect")]
authz_flow = Flow.objects.get(slug="default-provider-authorization-implicit-consent")
# Three things beyond the name, each one a silent dead login: the grant-type list (2026.8
# defaults it empty), the built-in OpenID scope mappings beside the claim mapping, and the
# redirect URIs with the base path baked in.
GRANTS = ["authorization_code", "refresh_token"]
BUILTIN_SCOPES = ["authentik default OAuth Mapping: OpenID 'openid'",
"authentik default OAuth Mapping: OpenID 'profile'",
"authentik default OAuth Mapping: OpenID 'email'"]
scoped = {m.pk: m for m in ScopeMapping.objects.filter(name__in=BUILTIN_SCOPES)}
scoped[mapping.pk] = mapping
_have = {m.name for m in scoped.values()}
if len(_have) < len(BUILTIN_SCOPES):
print("scope mappings: NOT PROVIDED by this authentik install: %s"
% ", ".join(sorted(set(BUILTIN_SCOPES) - _have)))
p = OAuth2Provider.objects.filter(name="audiobookshelf").first()
if p is None:
p = OAuth2Provider(name="audiobookshelf", client_id=CLIENT_ID,
client_type=ClientType.CONFIDENTIAL, client_secret=SECRET,
authorization_flow=authz_flow, grant_types=list(GRANTS))
p.redirect_uris = uris
p.save()
p.property_mappings.set(list(scoped.values()))
print("provider: created (grant_types %s)" % GRANTS)
else:
changed = []
if p.client_id != CLIENT_ID:
p.client_id = CLIENT_ID
changed.append("client_id")
if p.client_secret != SECRET:
p.client_secret = SECRET
changed.append("client_secret")
if p.authorization_flow_id != authz_flow.pk:
p.authorization_flow = authz_flow
changed.append("authorization_flow")
if list(p.grant_types) != GRANTS:
p.grant_types = list(GRANTS)
changed.append("grant_types")
if [(u.url, str(u.matching_mode)) for u in p.redirect_uris] != \
[(u.url, str(u.matching_mode)) for u in uris]:
p.redirect_uris = uris
changed.append("redirect_uris")
if changed:
p.save()
if sorted(m.pk for m in p.property_mappings.all()) != sorted(scoped):
p.property_mappings.set(list(scoped.values()))
changed.append("property_mappings")
print("provider: asserted (%s)" % (", ".join(changed) or "nothing to change"))
# --- application (the launchpad entry; binds the provider) ------------------------
app = Application.objects.filter(slug="audiobookshelf").first()
if app is None:
app = Application.objects.create(name="audiobookshelf", slug="audiobookshelf",
provider=p, meta_launch_url=BASE)
print("application: created")
elif app.provider_id != p.pk or app.meta_launch_url != BASE:
app.provider, app.meta_launch_url = p, BASE
app.save(update_fields=["provider", "meta_launch_url"])
print("application: asserted")
else:
print("application: already bound")
# --- membership --------------------------------------------------------------------
root = User.objects.get(username=SSO_USER)
root.groups.add(admin_g)
# The deploy gate's Playwright spec signs in as this identity, so the admin fixture is
# provisioned with the rest: a blueprint re-import that drops it fails the gate, not a phone.
e2e = User.objects.filter(username=E2E_ADMIN).first()
if e2e is None:
print("e2e fixture: authentik user %s absent — the e2e SSO spec will fail" % E2E_ADMIN)
else:
e2e.groups.add(admin_g)
print("e2e fixture: %s in %s" % (E2E_ADMIN, ADMIN_GROUP))
print("admin: %s in %s" % (SSO_USER, ADMIN_GROUP))
# The default is printed at provision time, not merely asserted: an expression that regressed
# to returning nothing for a non-member refuses every household login, and this is where a
# re-run notices it. The population matters — authentik ships an AnonymousUser row, a
# superuser, and ak-outpost-* service identities, and proving the default on one of those says
# nothing about the friend who logs in. Prefer the fleet's plain-user marker group (owned by
# serverAppConfigs-docs/Infrastructure/Authentik.md; a preference only, since that group
# belongs to other consumers), which normally lands on qa-e2e — the identity the e2e gate
# signs in as. is_superuser is a python property on authentik's User, not a queryable field.
pool = User.objects.filter(is_active=True).exclude(groups__name__in=[ADMIN_GROUP])
pool = pool.exclude(username__in=[root.username, E2E_ADMIN, "AnonymousUser"])
pool = [u for u in pool.order_by("username")
if not u.is_superuser and not u.username.startswith("ak-outpost-")]
household = [u for u in pool if any(g.name == "Non-Admin Users" for g in u.groups.all())]
for u in (household or pool)[:3]:
print("default probe: %s %s -> %s" % (
u.username, sorted(g.name for g in u.groups.all()),
_ns["_probe"](user=u, request=None)))
# --- verification: the claim the callback will actually receive ---------------------
for probe in [root] + ([e2e] if e2e else []):
print("claim for %s -> %s" % (probe.username, _ns["_probe"](user=probe, request=None)))
p.refresh_from_db()
print("provider %s | client_id %s | flow %s | grants %s | uris %s" % (
p.name, p.client_id, p.authorization_flow.slug, list(p.grant_types),
[u.url for u in p.redirect_uris]))
print("scopes: %s" % sorted(m.scopemapping.scope_name for m in p.property_mappings.all()))
PYEOF
cat <<DONE
Next: arm the Audiobookshelf side (issuer, client, HS256, claim) —
ssh dankserver 'ABS_OIDC=1 bash /home/dank/serverAppConfigs/scripts/audiobookshelf-init.sh'
It reads the same secret from /home/dank/.config/secrets/audiobookshelf and verifies
/status answers openid among its auth methods.
DONE
+95
View File
@@ -0,0 +1,95 @@
#!/usr/bin/env bash
# Census and repair the OAuth2 defaults on every authentik PROXY provider.
#
# A proxy provider built by hand in `ak shell` never runs
# ProxyProvider.set_oauth_defaults() — the REST serializer calls it, the ORM does
# not — so the row looks perfectly configured (external_host, internal_host,
# application bound, embedded-outpost claim present) while three OAuth facts are
# blank. Each one kills the guarded login in a different, unrelated-looking way,
# all BEFORE any credential is asked for:
#
# * redirect_uris empty -> authentik answers its own page:
# "Redirect URI Error — The request fails due to a missing, invalid, or
# mismatching redirection URI (redirect_uri)". Measured on the Audiobookshelf
# guard provider, 2026-11-03.
# * grant_types empty -> the authorize endpoint refuses the outpost's
# authorization-code request with a bodiless HTTP 400 (Chromium renders
# "This page isn't working / HTTP ERROR 400"). Same provider, one step
# later, after the URIs were filled in by hand.
# * authorization_flow NULL -> the authorize view crashes building its flow
# planner: AttributeError 'NoneType' object has no attribute 'slug', HTTP
# 500. That is what authentik-grafana-sso-fix.sh exists for.
#
# So: a guard that shows ANY of those three pages is not an Anubis problem and
# not a Caddy problem — run this script before touching the route.
#
# Repair calls the VENDOR's own set_oauth_defaults() (grant_types = authz code +
# client credentials + password, confidential client, the stock scope mappings,
# redirect_uris derived from external_host by _get_callback_url) and points the
# authorization flow at default-provider-authorization-implicit-consent, the flow
# every working provider here uses. Providers that already match are untouched,
# so a deliberate per-provider deviation is never silently flattened.
#
# This is authentik DB state. It is NOT in any deployable file: a blueprint
# re-import, or an ak-shell re-creation of a provider, reverts these facts and
# the symptom returns verbatim. Re-run after any such rebuild. Idempotent.
#
# Usage: authentik-proxy-provider-defaults.sh [--check]
# no argument repair what drifts, print one line per provider
# --check census only, exit 1 if anything drifts (gate use)
# override host: AUTHENTIK_HOST=…
set -euo pipefail
HOST="${AUTHENTIK_HOST:-dankclaw}"
MODE="${1:-}"
case "$MODE" in
"" | --check) : ;;
*) echo "usage: $(basename "$0") [--check]" >&2; exit 2 ;;
esac
ssh "$HOST" 'podman exec -i authentik-server ak shell' <<PYEOF
from authentik.flows.models import Flow
from authentik.providers.proxy.models import ProxyProvider, _get_callback_url
CHECK = $( [ "$MODE" = "--check" ] && echo True || echo False )
FLOW = "default-provider-authorization-implicit-consent"
WANT_GRANTS = ["authorization_code", "client_credentials", "password"]
flow = Flow.objects.filter(slug=FLOW).first()
if not flow and not CHECK:
raise SystemExit("flow %s is missing; nothing was changed" % FLOW)
drift = 0
for p in ProxyProvider.objects.all().order_by("pk"):
want_uris = [u.url for u in _get_callback_url(p.external_host)]
have_uris = [u.url for u in p.redirect_uris]
bad = []
if [str(g) for g in p.grant_types] != WANT_GRANTS:
bad.append("grant_types=%s" % [str(g) for g in p.grant_types])
if have_uris != want_uris:
bad.append("redirect_uris=%s" % have_uris)
if p.authorization_flow_id is None:
bad.append("authorization_flow=None")
if not bad:
print("ok %s (ext %s)" % (p.name, p.external_host))
continue
drift += 1
if CHECK:
print("DRIFT %s: %s" % (p.name, "; ".join(bad)))
continue
print("fix %s: %s" % (p.name, "; ".join(bad)))
p.set_oauth_defaults()
if p.authorization_flow_id is None:
p.authorization_flow = flow
p.save()
after = ProxyProvider.objects.get(pk=p.pk)
print(" -> grants %s, %d redirect URIs, flow %s"
% ([str(g) for g in after.grant_types], len(after.redirect_uris),
after.authorization_flow and after.authorization_flow.slug))
print("providers checked, drift: %d" % drift)
raise SystemExit(1 if (CHECK and drift) else 0)
PYEOF
echo "Verify: walk the guarded vhost in a browser — Anubis solves, authentik"
echo "renders its login form (no Redirect URI Error, no HTTP 400), and the"
echo "callback lands back on the app."
+13 -5
View File
@@ -323,11 +323,13 @@ copy_alloy_config() {
# The repo config carries the token GRAFANA_HOST_LABEL; the installed copy
# carries the host's name, so Loki streams say WHICH machine's journal a
# line came from without a second config file per host.
# *_CONF_ROOT are test seams: the bats cases execute these copiers for
# real against a sandbox with a stubbed run_privileged.
# *_CONF_ROOT and ALLOY_CONFIG_SRC are test seams: the bats cases execute
# these copiers for real against a sandbox with a stubbed run_privileged;
# ALLOY_CONFIG_SRC lets the refusal case feed a deliberately broken config.
local root="${ALLOY_CONF_ROOT:-/opt/alloy}" tmp
local src="${ALLOY_CONFIG_SRC:-shared/alloy/config.alloy}"
tmp="$(mktemp)"
sed "s/GRAFANA_HOST_LABEL/$TARGET/g" shared/alloy/config.alloy > "$tmp"
sed "s/GRAFANA_HOST_LABEL/$TARGET/g" "$src" > "$tmp"
if ! grep -q "host = \"$TARGET\"," "$tmp" || grep -q "GRAFANA_HOST_LABEL" "$tmp"; then
echo "alloy config: host-label substitution failed" >&2
rm -f "$tmp"
@@ -1315,7 +1317,8 @@ deploy_audiobookshelf() {
mkdir -p /home/dank/Containers/audiobookshelf/config \
/home/dank/Containers/audiobookshelf/metadata
systemctl --user restart audiobookshelf.service
# /healthcheck answers once the HTTP server is bound; the library scan that
# /healthcheck answers once the HTTP server is bound, at the root whatever
# ROUTER_BASE_PATH says (measured on both base paths); the library scan that
# follows runs in the background and must not gate the deploy.
wait_for_health audiobookshelf http://127.0.0.1:8000/healthcheck 90
# A server with no root user and no library is "healthy" and useless, so the
@@ -1323,7 +1326,12 @@ deploy_audiobookshelf() {
# it creates the account (credential in the mode-600 host secret file), the
# Audiobooks library over the /audiobooks bind, and fires the scan. Every
# step is idempotent, so a re-deploy costs one incremental scan.
scripts/audiobookshelf-init.sh
# The bootstrapper has to speak the same base path the unit mounts (the API, the
# login route and the OIDC redirect subfolder all hang off it), so the URL is read
# out of the unit file here instead of being restated in the script's default.
abs_base=$(awk -F= '/^Environment=ROUTER_BASE_PATH=/{print $3}' \
dankserver/audiobookshelf/quadlet/audiobookshelf.container)
ABS_URL="http://127.0.0.1:8000${abs_base}" scripts/audiobookshelf-init.sh
}
# The content server's single admin account and the desktop's basic-auth
+12 -1
View File
@@ -135,7 +135,18 @@ text = escape(val)
if pat.search(src):
out = pat.sub(lambda m: f"{m.group(1)}<{elem}>{text}</{elem}>", src, count=1)
else:
out = src.replace("</Configuration>", f" <{elem}>{text}</{elem}>\n</Configuration>", 1)
# The plugin owns this file and writes it under ITS OWN root element
# (TheTVDB's is <PluginConfiguration>), and its serializer omits a
# null-valued setting entirely — so the element being filled can be
# absent outright, and a hardcoded </Configuration> anchor then matches
# nothing. Anchor on the document's real root instead.
out = src
root = re.search(r"<([A-Za-z_][\w.:-]*)[^>]*>", src)
if root and ("</%s>" % root.group(1)) in src:
name = root.group(1)
out = src.replace(
"</%s>" % name, f" <{elem}>{text}</{elem}>\n</{name}>", 1
)
if out == src:
raise SystemExit(f"{xml_path}: no place to write <{elem}>")
+420
View File
@@ -0,0 +1,420 @@
#!/usr/bin/env python3
"""media-ab-reorg.py — plan the Audiobooks dedupe + ABS-structure relayout.
2026-11-04 campaign. Turns the Audiobooks library into the layout Audiobookshelf
documents (https://audiobookshelf.org/docs/documentation/libraries/book-library/
directory-structure/):
{Author}/[{Series}/]{seq} - {year} - {Title} {Narrator}/…files…
The authority for every book's author/series/title/year/narrator is the ABS
database dump (--items, JSON lines: [relPath, isMissing, title, authors, series,
seq, year, narratorsJSON, coverPath, ebookFlag]); files are attached to their
book by longest-prefix match on the item's relPath, which is how ABS's own
grouping works. Folder names are then re-emitted so ABS's folder parser reads
the same metadata back off disk.
Three outputs, all consumable by scripts/media-apply.py (run everything with
--root /mnt/entropy; hold/intake/park live outside the library on purpose):
--plan mkdir/keep/move rows: dedupe losers -> --hold, ebook-only items
and comics -> --intake (the eBooks pile feeds Calibre next),
unowned media -> --park, everything else -> the new layout.
--covers rows "srchost \\t dstarray": covers ABS keeps in /metadata get a
copy beside the book (cp, it crosses filesystems); run this
BEFORE apply so the cover lands inside the moved folder.
--exceptions every judgement call: name collisions, byte-dups that pair two
different books, collapsed items, stray files.
Dedupe law: only CONFIRMED byte-identities (--dups from
media-ab-verifydups.py) remove a file. One copy survives per book: inside a
folder the cleanest-named file wins; across folders the item whose root matches
its author wins. Equal bytes under two different book identities keep both and
raise an exception — a trailer embedded in two books is not a duplicate.
Rerunnable: a second run against a moved library emits keep rows, not moves.
"""
import argparse
import json
import os
import re
from collections import defaultdict
AUDIO = {"m4b", "mp3", "m4a", "flac", "opus", "ogg", "oga", "mp4", "aac", "wma",
"aiff", "aif", "wav", "webm", "webma", "mka", "awb", "caf", "mpg",
"mpg2", "mpeg"}
EBOOK = {"epub", "pdf", "mobi", "azw3", "cbr", "cbz", "fb2"}
COMIC = {"cbr", "cbz"}
MEDIA = AUDIO | {"epub", "pdf", "mobi", "azw3", "cbr", "cbz"}
PLACEHOLDER_AUTHOR = {"unknown", "unknown artist", "various", "various artists",
"various authors", "n/a", ""}
BAD_CHARS = re.compile(r'[<>:"/\\|?*]')
DISC_DIR = re.compile(r"^(disc|cd|disk)\s*\d", re.I)
KEEP_HINT = re.compile(r"\s*\((?:2|3|copy|backup|duplicate)\)\s*$", re.I)
META_HOST = "/home/dank/Containers/audiobookshelf"
def clean(name):
name = BAD_CHARS.sub(" ", name or "")
name = re.sub(r"\s+", " ", name).strip().strip(".")
return name or "Untitled"
def clean_author(name):
"""DB author string -> folder name: one author, no 'zz' sort artifact."""
name = re.sub(r"^zz", "", (name or "").strip())
return clean(name)
def first_author(authors):
parts = [clean_author(p) for p in (authors or "").split("|")]
parts = [p for p in parts if p]
return parts[0] if parts else "Unknown"
def parse_narrators(blob):
try:
v = json.loads(blob) if blob else []
except (TypeError, json.JSONDecodeError):
v = []
if isinstance(v, str):
v = [v]
return [str(x).strip() for x in (v or []) if str(x).strip()]
def core_title(title, author):
"""Title as the book folder should show it: release prefixes stripped."""
t = (title or "Untitled").strip()
a = (author or "").strip()
if a and t.lower().startswith(a.lower() + " -"):
t = t[len(a):].lstrip(" -")
return t.strip() or "Untitled"
def bookname(seq, year, title, narrators):
seq = (seq or "").strip()
seq = seq if re.fullmatch(r"\d+(\.\d+)?", seq) else ""
year = (year or "").strip()
year = year if re.fullmatch(r"(1[5-9]|20)\d\d", year) else ""
t = title
if seq:
# drop a sequence prefix already carried inside the title
t = re.sub(rf"^{re.escape(seq)}\s*[\-.]\s+", "", t)
if year:
t = re.sub(rf"^{year}\s*[\-.]\s+", "", t)
t = re.sub(rf"\s*\({year}\)\s*$", "", t)
name = t
if seq and year:
name = f"{seq} - {year} - {name}"
elif seq:
name = f"{seq} - {name}"
elif year:
name = f"{year} - {name}"
if len(narrators) == 1:
name = f"{name} {{{clean(narrators[0])}}}"
return clean(name)
class Campaign:
def __init__(self, args):
self.root = args.root.rstrip("/")
self.lib = self.root + "/Audiobooks"
self.hold = args.hold
self.intake = args.intake
self.park = args.park
self.items = {} # relPath -> item dict (healthy only)
self.stale = [] # isMissing=1 rows
self.moved = {} # src -> (dst, kind)
self.claims = {} # dst -> src (collision guard)
self.loser_files = set()
self.loser_items = set()
self.exc = defaultdict(list)
# ------------------------------------------------------------- inputs
def load_items(self, path):
for line in open(path):
r = json.loads(line)
rel = r[0]
item = {"rel": rel, "title": r[2] or "", "authors": r[3] or "",
"series": r[4] or "", "seq": r[5] or "", "year": r[6] or "",
"narr": parse_narrators(r[7]), "cover": r[8] or "",
"ebook": r[9]}
if r[1]:
self.stale.append(rel)
continue
self.items[rel] = item
def load_files(self, path):
files = []
for line in open(path):
sz, ext, p = line.rstrip("\n").split("\t", 2)
if p.startswith(self.lib + "/"):
files.append((int(sz), ext.lstrip(".").lower(), p))
return files
def owner(self, d):
"""The healthy DB item whose folder is d or an ancestor of d."""
rel = os.path.relpath(d, self.lib)
while rel and rel != ".":
if rel in self.items:
return rel
rel = os.path.dirname(rel)
return None
# ------------------------------------------------------------- dedupe
def identity(self, item_rel):
if item_rel is None:
return None
it = self.items[item_rel]
return (first_author(it["authors"]).lower(),
core_title(it["title"], first_author(it["authors"])).lower(),
(it["year"] or "").strip())
def root_score(self, item_rel):
it = self.items[item_rel]
top = item_rel.split("/")[0]
s = 0
if top.lower() == first_author(it["authors"]).lower():
s += 2
if not top.startswith("_") and "various" not in top.lower():
s += 1
return (s, -len(item_rel))
def dedupe(self, files, dups_path):
by_hash = defaultdict(list)
if dups_path:
for line in open(dups_path):
h, sz, p = line.rstrip("\n").split("\t", 2)
if p.startswith(self.lib + "/"):
by_hash[h].append(p)
owner_of = {}
dirs = {os.path.dirname(p) for _, _, p in files}
for d in dirs:
owner_of[d] = self.owner(d)
files_by_owner = defaultdict(list)
for _, _, p in files:
files_by_owner[owner_of[os.path.dirname(p)]].append(p)
for h, paths in by_hash.items():
owners = {owner_of.get(os.path.dirname(p)) for p in paths}
owners = {o for o in owners if o is not None}
per_item = defaultdict(list)
for p in paths:
per_item[owner_of.get(os.path.dirname(p))].append(p)
if owners:
ids = {o: self.identity(o) for o in owners}
# group items by identity; loose (None) files follow their own rule
same_as_loser = []
keepers = []
byid = defaultdict(list)
for o in owners:
byid[ids[o]].append(o)
for ident, os_ in byid.items():
os_.sort(key=self.root_score, reverse=True)
keepers.append(os_[0])
same_as_loser.extend(os_[1:])
keepers = set(keepers)
loser_items = set(same_as_loser)
else:
loser_items = set()
keepers = set()
for o, ps in per_item.items():
if o is None:
continue # loose files decided in placement
if o in loser_items:
self.loser_files.update(ps)
continue
if len(ps) > 1 and o in keepers:
# redundant copies inside the surviving folder
keeper = min(ps, key=lambda p: (
bool(KEEP_HINT.search(os.path.splitext(p)[0])),
len(p), p))
self.loser_files.update(p for p in ps if p != keeper)
for o in loser_items:
self.loser_items.add(o)
# collapsed items: every media file lost
for o, ps in files_by_owner.items():
if o is None:
continue
media = [p for p in ps if os.path.splitext(p)[1].lstrip(".").lower()
in MEDIA]
if media and all(p in self.loser_files for p in media):
self.loser_items.add(o)
self.exc["collapsed items (all files were byte-duplicates)"].append(o)
# ---------------------------------------------------------- placement
def target(self, item):
author = first_author(item["authors"])
series = clean(item["series"]) if item["series"] else ""
title = core_title(item["title"], author)
name = bookname(item["seq"], item["year"], title, item["narr"])
parts = [self.lib, clean(author)]
if series:
parts.append(clean(series))
parts.append(name)
return os.path.join(*parts)
def claim(self, dst, src, why):
"""One destination, one source; collisions rename with a marker."""
if dst not in self.claims:
self.claims[dst] = src
return dst
stem, ext = os.path.splitext(dst)
n = 2
while f"{stem} ({n}){ext}" in self.claims:
n += 1
dst2 = f"{stem} ({n}){ext}"
self.claims[dst2] = src
self.exc[why].append(f"{src} -> renamed to {dst2}")
return dst2
def place(self, files):
audio_by_item = defaultdict(int)
for _, ext, p in files:
o = self.owner(os.path.dirname(p))
if o and ext in AUDIO:
audio_by_item[o] += 1
cover_dirs = {}
# Claim one BOOK FOLDER per item: two items for the same title (the
# _unsorted era filed some books 3 times) must not merge their tracks
# into one item — the loser gets a variant folder instead.
for rel in sorted(self.items, key=lambda r: self.root_score(r),
reverse=True):
it = self.items[rel]
t = self.target(it)
if t in self.claims:
n = 2
while f"{t} (variant {n})" in self.claims:
n += 1
t = f"{t} (variant {n})"
self.exc["same title filed by two DB items"].append(f"{rel} -> {t}")
self.claims[t] = rel
cover_dirs[rel] = t
if (it["cover"].startswith("/metadata") and audio_by_item[rel]
and rel not in self.loser_items):
src = META_HOST + it["cover"]
self.covers.append((src, os.path.join(t,
"cover" + os.path.splitext(src)[1].lower())))
# Containment gate: an item folder holding media files must never sit
# above another item folder — ABS's grouping would swallow the child
# (the docs' PROBLEM example). Rename the ANCESTOR book out of the way.
for _ in range(3):
moved = False
folders = sorted(set(cover_dirs.values()))
for anc in folders:
for d in folders:
if d != anc and d.startswith(anc + os.sep):
n = 1
cand = f"{anc} (book)" if n == 1 else f"{anc} (book {n})"
while cand in self.claims:
n += 1
cand = f"{anc} (book {n})"
self.exc["book folder nested above another book"].append(
f"{anc} -> {cand}")
self.claims[cand] = self.claims.pop(anc)
for r, t in cover_dirs.items():
if t == anc:
cover_dirs[r] = cand
self.covers = [(s, d.replace(anc + os.sep, cand + os.sep, 1))
for s, d in self.covers]
moved = True
break
if moved:
break
if not moved:
break
for _, ext, p in files:
if p in self.loser_files:
d = os.path.join(self.hold, os.path.relpath(p, self.root))
self.move(p, d, "dup")
continue
d = os.path.dirname(p)
o = self.owner(d)
if o is None:
rel = os.path.relpath(p, self.lib)
if ext in MEDIA:
self.move(p, os.path.join(self.park, rel), "park")
else:
self.move(p, os.path.join(self.park, rel), "park-junk")
continue
it = self.items[o]
if ext in COMIC:
key = clean(f"{first_author(it['authors'])} - {core_title(it['title'], '')}")
dst = os.path.join(self.intake, "_comics", key, os.path.basename(p))
self.move(p, self.claim(dst, p, "intake comic collisions"), "comic")
continue
if audio_by_item[o] == 0: # ebook-only item belongs to the ebook stack
key = f"{first_author(it['authors'])} - {bookname(it['seq'], it['year'], core_title(it['title'], first_author(it['authors'])), it['narr'])}"
dst = os.path.join(self.intake, key, os.path.relpath(p, os.path.join(self.lib, o)))
self.move(p, self.claim(dst, p, "intake ebook collisions"), "ebook-only")
continue
rel_in = os.path.relpath(p, os.path.join(self.lib, o))
dst = os.path.join(cover_dirs[o], rel_in)
if dst == p:
self.kept.append(p)
self.claims.setdefault(dst, p)
continue
self.move(p, self.claim(dst, p, "target collisions"), "move")
def move(self, src, dst, kind):
self.moves.append((src, dst, kind))
def write(self, plan_path, exc_path):
dirs = {os.path.dirname(dst) for _, dst, _ in self.moves}
dirs |= {os.path.dirname(d) for _, d in self.covers}
all_dirs = set()
for d in dirs:
while d and d != self.root:
all_dirs.add(d)
d = os.path.dirname(d)
with open(plan_path, "w") as fh:
fh.write("action\tsrc\tdst\tkind\n")
for d in sorted(all_dirs):
fh.write(f"mkdir\t\t{d}\tcampaign\n")
for p in self.kept:
fh.write(f"keep\t{p}\t\tkeep\n")
for src, dst, kind in sorted(self.moves):
fh.write(f"move\t{src}\t{dst}\t{kind}\n")
if self.exc:
with open(exc_path, "w") as fh:
for why in sorted(self.exc):
fh.write(f"# {why}\n")
for line in sorted(set(self.exc[why])):
fh.write(line + "\n")
fh.write("\n")
mv = [m for m in self.moves if m[2] != "dup"]
print(f"plan: {len(all_dirs)} folders, {len(self.moves)} moves "
f"({len(mv)} relayout, {len(self.moves)-len(mv)} dup-hold), "
f"{len(self.kept)} keep, {len(self.covers)} cover-copies")
def main():
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("--root", default="/mnt/entropy")
ap.add_argument("--enum", required=True)
ap.add_argument("--items", required=True)
ap.add_argument("--dups", default=None)
ap.add_argument("--hold", default="/mnt/entropy/.media-campaign/dup-hold-20261104")
ap.add_argument("--intake", default="/mnt/entropy/eBooks/_from-audiobooks")
ap.add_argument("--park", default="/mnt/entropy/NeedsProcessing/_from-audiobooks-20261104")
ap.add_argument("--plan", required=True)
ap.add_argument("--exceptions", required=True)
ap.add_argument("--covers", required=True)
args = ap.parse_args()
c = Campaign(args)
c.moves, c.kept, c.covers = [], [], []
c.load_items(args.items)
files = c.load_files(args.enum)
c.dedupe(files, args.dups)
c.place(files)
c.write(args.plan, args.exceptions)
with open(args.covers, "w") as fh:
for src, dst in sorted(set(c.covers)):
fh.write(f"{src}\t{dst}\n")
print(f"covers: {len(set(c.covers))} stale DB rows (delete after scan): {len(c.stale)}")
if __name__ == "__main__":
main()
+89
View File
@@ -0,0 +1,89 @@
#!/usr/bin/env python3
"""media-ab-verifydups.py — confirm byte-duplicates among size-group survivors.
Stage 2 of the 2026-11-04 Audiobooks dedupe. Stage 1 (partial blake2b of the
first+last 4 MiB, done by the campaign's hashcand.py) over-claims: two files
of equal size and equal head+tail are CANDIDATES, not copies. This tool takes
the candidate files from --partial (rows: size, hash, path), groups them by
(size, partial-hash), full-hashes only members of groups with 2+ members, and
writes verdicts: one line per file whose full hash ties it to at least one
other file.
Output rows: full-hash <TAB> size <TAB> path (sorted by hash) — a hash that
appears once is not written at all; singletons are not duplicates.
Read-only. Never moves, never deletes. Run it after the partial pass finishes
(pgrep it first; two hashers on one output file have raced before).
"""
import argparse
import hashlib
import os
from collections import defaultdict
def full_hash(path, chunk=1 << 20):
h = hashlib.blake2b(digest_size=32)
with open(path, "rb") as fh:
for block in iter(lambda: fh.read(chunk), b""):
h.update(block)
return h.hexdigest()
def main():
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("--partial", required=True, help="partial-hashes.tsv")
ap.add_argument("--out", required=True, help="full-hash verdicts TSV")
ap.add_argument("--progress-every", type=int, default=50)
args = ap.parse_args()
groups = defaultdict(list)
with open(args.partial) as fh:
for line in fh:
f = line.rstrip("\n").split("\t")
if len(f) != 3 or f[0] == "ERR":
continue
groups[(int(f[0]), f[1])].append(f[2])
survivors = sorted({p for g in groups.values() if len(g) > 1 for p in g})
print(f"survivor files to full-hash: {len(survivors)}")
done = set()
if os.path.exists(args.out):
with open(args.out) as fh:
for line in fh:
done.add(line.rstrip("\n").split("\t", 2)[2])
full = {}
for line in open(args.out) if os.path.exists(args.out) else []:
h, s, p = line.rstrip("\n").split("\t", 2)
full[p] = h
todo = [p for p in survivors if p not in done]
out = open(args.out, "a")
for i, p in enumerate(todo):
try:
full[p] = full_hash(p)
except OSError as e:
print(f"SKIP {p}: {e}")
continue
if i % args.progress_every == args.progress_every - 1:
out.flush()
out.close()
# Rewrite the whole verdict file keeping only TRUE duplicates (2+ fulls).
by_hash = defaultdict(list)
for p in survivors:
if p in full:
by_hash[full[p]].append(p)
with open(args.out, "w") as out:
for h, ps in sorted(by_hash.items()):
if len(ps) > 1:
for p in sorted(ps):
out.write(f"{h}\t{os.path.getsize(p)}\t{p}\n")
dups = {h: ps for h, ps in by_hash.items() if len(ps) > 1}
n = sum(len(ps) - 1 for ps in dups.values())
waste = sum((len(ps) - 1) * os.path.getsize(ps[0]) for ps in dups.values())
print(f"true dup sets: {len(dups)} redundant files: {n} "
f"waste: {waste / 2**30:.1f} GiB")
if __name__ == "__main__":
main()
+10 -2
View File
@@ -121,14 +121,14 @@ class Journal:
def read_plan(path, root, force):
"""Plan rows: (mkdirs, keeps, moves), header dropped, all paths under root."""
mkdirs, keeps, moves = [], [], []
mkdirs, keeps, moves, bad = [], [], [], []
with open(path) as fh:
header = fh.readline()
if not header.startswith("action\t"):
raise Refuse(f"{path} is not a plan (first column is "
f"{header.split(chr(9))[0]!r}, expected 'action') —"
" regenerate it with scripts/media-plan.py")
for line in fh:
for n, line in enumerate(fh, 2):
f = line.rstrip("\n").split("\t")
if f[0] == "mkdir":
mkdirs.append(f[2])
@@ -136,6 +136,14 @@ def read_plan(path, root, force):
keeps.append(f[1])
elif f[0] == "move":
moves.append((f[1], f[2], f[3] if len(f) > 3 else ""))
else:
bad.append(f"line {n}: unknown action {f[0]!r} — this tool"
" knows mkdir, keep and move (a rename is a move)")
if bad:
# A row this tool could not read is a plan bug, and a plan bug refuses:
# silently dropping it turns a typo into a no-op run that reports success.
head = "; ".join(bad[:3])
raise Refuse(head + (f" (+{len(bad) - 3} more)" if len(bad) > 3 else ""))
if not force:
for p in mkdirs + keeps + [p for m in moves for p in m[:2]]:
if p and not (p == root or p.startswith(root + os.sep)):
+239
View File
@@ -0,0 +1,239 @@
#!/usr/bin/env python3
"""media-ebook-dedupe.py — dedupe the eBooks pile against the Calibre library.
2026-11-04 campaign, ebook side. The Calibre library (/mnt/entropy/CalibreBooks,
6,289 books) is DB-owned and healthy; /mnt/entropy/eBooks is a raw pile whose
best books already live in the library, several of them twice in the pile.
Audiobooks' orphaned ebooks/comics land in the pile first (media-ab-reorg's
intake), so this campaign runs after the audiobook move.
Inputs
--meta ebook-meta.tsv: JSON lines {p, sz, title, authors, ident} for
epub (embedded OPF) and mobi/azw3 (Palm header title); the
producer is the campaign's opfcensus.py.
--enum enum TSV (size, ext, path) for every tree.
--calibre calibre-books.tsv: JSON lines [id, title, authors, series, path,
pubdate, formats] dumped from metadata.db.
--dups full blake2b verdicts (hash, size, path) — only confirmed
byte-identities remove a file. Optional: without it, identity
rules alone run and every hash-dependent pair stays a KEEP.
Outputs
--plan media-apply plan (--root /mnt/entropy): hash-duplicates of a
file already in CalibreBooks -> HOLD; byte-identical files inside
the pile keep the best format of one book once; non-book junk
(mp4/mp3/images) -> PARK.
--ingest one JSON line per surviving BOOK (a set of format files of one
identity): {paths, title, authors, source}. The ingest runner
feeds these to calibredb inside the calibre-web image.
--exceptions every identity pair the file-level evidence could not settle.
Match law: identity is (core title, first author surname). Calibre's
"Dinniman| Matt" and an OPF "Matt Dinniman" share surname "dinniman". Title
core drops the leading article and all non-alphanumerics. Identity alone NEVER
deletes against the pile — it only deletes when the same identity sits in the
Calibre library with the same format (the pile copy adds nothing), or when a
byte-hash confirms the pair. Format preference when the pile holds one book in
several copies: epub > azw3 > mobi > pdf > fb2 > lit > rtf > txt.
"""
import argparse
import json
import os
import re
import unicodedata
from collections import defaultdict
BOOK_EXT = {"epub", "azw3", "mobi", "pdf", "fb2", "lit", "rtf", "txt", "cbr", "cbz"}
FORMAT_RANK = {"epub": 0, "azw3": 1, "mobi": 2, "pdf": 3, "fb2": 4,
"cbr": 5, "cbz": 5, "lit": 6, "rtf": 7, "txt": 8}
ARTICLE = re.compile(r"^(the|a|an)\s+", re.I)
def ascii_fold(s):
return "".join(c for c in unicodedata.normalize("NFKD", s or "")
if not unicodedata.combining(c))
def title_core(title):
t = ascii_fold(title).lower()
t = ARTICLE.sub("", t)
t = re.sub(r"\(.*?\)", " ", t) # (1998), (Unabridged) — noise
t = re.sub(r"[^a-z0-9 ]", " ", t)
t = re.sub(r"\b(1[5-9]|20)\d\d\b", " ", t) # years
return " ".join(t.split())[:120]
def surname(author):
a = ascii_fold(author).strip()
a = re.split(r"[;,]", a)[0] if "," in a else " ".join(a.split()[-1:])
return re.sub(r"[^a-z]", "", a.lower())
def first_author(authors):
parts = [p.strip() for p in re.split(r"[;&]", authors or "") if p.strip()]
return parts[0] if parts else ""
def stem_title(path):
s = os.path.splitext(os.path.basename(path))[0]
return ascii_fold(s)
class Campaign:
def __init__(self, args):
self.args = args
self.root = args.root.rstrip("/")
self.pile = self.root + "/eBooks"
self.lib = self.root + "/CalibreBooks"
self.hold = args.hold
self.park = args.park
self.meta = {} # path -> meta dict
self.pile_files = [] # (sz, ext, path)
self.cal_by_path = {} # file path under CalibreBooks -> calibre row
self.cal_identity = {} # (tcore, surname) -> calibre row
self.drop = {} # path -> reason
self.exc = defaultdict(list)
self.moves = []
self.dirs_pile_book = defaultdict(list) # book dir (pile) -> files
# ------------------------------------------------------------- inputs
def load(self):
for line in open(self.args.meta):
r = json.loads(line)
self.meta[r["p"]] = r
by_hash = defaultdict(list)
if self.args.dups:
for line in open(self.args.dups):
h, sz, p = line.rstrip("\n").split("\t", 2)
by_hash[h].append(p)
self.dup_sets = [ps for ps in by_hash.values() if len(ps) > 1]
for line in open(self.args.enum):
sz, ext, p = line.rstrip("\n").split("\t", 2)
ext = ext.lstrip(".").lower()
if p.startswith(self.pile + "/") and p not in self.park:
self.pile_files.append((int(sz), ext, p))
for line in open(self.args.calibre):
row = json.loads(line)
self.cal_identity[(title_core(row[1]), surname(first_author(row[2])))] = row
def identity(self, p):
m = self.meta.get(p)
if m and m.get("title"):
t = title_core(m["title"])
a = surname(first_author(m.get("authors", "")))
if t:
return (t, a)
return (title_core(stem_title(p)), "")
# ------------------------------------------------------------- rules
def run(self):
pile_paths = {p for _, _, p in self.pile_files}
# R1/R2: confirmed byte-duplicates
for ps in self.dup_sets:
in_lib = [p for p in ps if p.startswith(self.lib + "/")]
in_pile = [p for p in ps if p.startswith(self.pile + "/")]
if in_lib:
for p in in_pile:
self.drop[p] = "hash-duplicate of CalibreBooks file"
if len(in_pile) > 1:
keep = min(in_pile, key=lambda p: (FORMAT_RANK.get(
os.path.splitext(p)[1].lstrip(".").lower(), 9), len(p), p))
for p in in_pile:
if p != keep:
self.drop[p] = "hash-duplicate inside the pile"
# R3/R4: identity against the library (same format only deletes)
cal_files = defaultdict(set) # identity -> formats in the library
for key, row in self.cal_identity.items():
cal_files[key] = set(f for f in row[6].split("; ") if f)
for sz, ext, p in self.pile_files:
if p in self.drop or ext not in BOOK_EXT:
continue
key = self.identity(p)
if key[0] and key in self.cal_identity:
if ext in cal_files[key]:
self.drop[p] = f"same identity+format already in Calibre ({key[0][:40]})"
else:
self.exc["pile holds a format the library lacks (add-format on ingest)"].append(
f"{p}\t{key[0][:50]}\t{self.cal_identity[key][0]}")
# R5: surviving files grouped by (identity, parent-ish book dir)
books = defaultdict(list)
for sz, ext, p in self.pile_files:
if p in self.drop:
continue
if ext not in BOOK_EXT:
self.moves.append((p, os.path.join(self.park, os.path.relpath(p, self.root)),
"non-book"))
continue
m = self.meta.get(p, {})
if m.get("err") and ext in ("epub",):
self.exc["unreadable container (park for a human)"].append(p)
self.moves.append((p, os.path.join(self.park, os.path.relpath(p, self.root)),
"broken-book"))
continue
key = self.identity(p)
books[key].append(p)
for key, ps in books.items():
first = min(ps, key=lambda p: (FORMAT_RANK.get(
os.path.splitext(p)[1].lstrip(".").lower(), 9), len(p), p))
m = self.meta.get(first, {})
self.ingest = {
"key": list(key),
"paths": sorted(ps),
"title": m.get("title") or stem_title(first),
"authors": m.get("authors") or "",
"ident": m.get("ident") or "",
}
self.ingest_list.append(self.ingest)
for p, why in self.drop.items():
self.moves.append((p, os.path.join(self.hold, os.path.relpath(p, self.root)), "dup"))
def write(self):
dirs = {os.path.dirname(d) for _, d, _ in self.moves}
all_dirs = set()
for d in dirs:
while d and d != self.root:
all_dirs.add(d)
d = os.path.dirname(d)
with open(self.args.plan, "w") as fh:
fh.write("action\tsrc\tdst\tkind\n")
for d in sorted(all_dirs):
fh.write(f"mkdir\t\t{d}\tcampaign\n")
for src, dst, kind in sorted(self.moves):
fh.write(f"move\t{src}\t{dst}\t{kind}\n")
with open(self.args.ingest, "w") as fh:
for b in self.ingest_list:
fh.write(json.dumps(b, ensure_ascii=False) + "\n")
with open(self.args.exceptions, "w") as fh:
for why in sorted(self.exc):
fh.write(f"# {why}\n")
for line in sorted(set(self.exc[why])):
fh.write(line + "\n")
fh.write("\n")
ndrop = sum(1 for _, _, k in self.moves if k == "dup")
print(f"pile books to ingest: {len(self.ingest_list)} "
f"files dropped as duplicates: {ndrop} "
f"non-book/broken moved out: {sum(1 for _, _, k in self.moves if k in ('non-book', 'broken-book'))}")
def main():
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("--root", default="/mnt/entropy")
ap.add_argument("--meta", required=True)
ap.add_argument("--enum", required=True)
ap.add_argument("--calibre", required=True)
ap.add_argument("--dups", default=None)
ap.add_argument("--hold", default="/mnt/entropy/.media-campaign/dup-hold-ebooks-20261104")
ap.add_argument("--park", default="/mnt/entropy/NeedsProcessing/_from-ebooks-20261104")
ap.add_argument("--plan", required=True)
ap.add_argument("--ingest", required=True)
ap.add_argument("--exceptions", required=True)
args = ap.parse_args()
c = Campaign(args)
c.ingest_list = []
c.load()
c.run()
c.write()
if __name__ == "__main__":
main()
+223
View File
@@ -0,0 +1,223 @@
#!/usr/bin/env python3
"""Trigger and poll Jellyfin's library scan — the step that makes a media move visible.
media-refresh-jellyfin.py trigger [--force]
media-refresh-jellyfin.py status [--after EPOCH]
media-refresh-jellyfin.py wait [--after EPOCH] [--interval S] [--require-quiet N]
`status` answers {"scan": true, "taskState": "Running", "recentMedia": 17}: the
Scan Media Library row plus the rows the database has taken in recently; with
`--after` (the trigger's epoch — a file mtime is already passed by an earlier scan) it
also answers `scannedAfter` and exits non-zero
while the last finished scan predates it. `wait` calls the library settled only
on the page's law: a sighting of the task Running — or a run that finished while
it watched, or a `--after` the task already passes — and THEN `--require-quiet`
consecutive reads with no scan and no writes. Both the scan and the metadata
fetch must be quiet, and the Running sighting is what stops a watcher from
declaring a pre-scan library settled (Dankserver/Jellyfin.md).
Laws this encodes:
* Auth is `Authorization: MediaBrowser Token="…"`. A bare X-Emby-Token answers
401 (measured on franksdankcloud 12.4.0), which reads exactly like a dead
key. The key comes from a mode-600 file and never rides argv or a URL:
/proc/<pid>/cmdline is world-readable.
* `POST /Library/Refresh` (204) is the ONLY door that queues work on 12.4, and
it walks every library. `POST /Items/{id}/Refresh` answers 204 and queues
NOTHING (measured twice, 2026-11-02) — so there is no --library here.
* A SECOND accepted POST cancels and restarts the scan already running, so
progress looks frozen while you hammer it. `trigger` therefore refuses while
the Scan Media Library task is not Idle; --force says you want the restart.
* 429 means busy. Waiting is the only response that helps, and it is bounded
so a wedged server ends the run with a named failure instead of a loop.
* `RefreshStatus` on /Library/VirtualFolders stays Idle through a live scan;
never poll it. The task table is the only honest read.
Defaults are this fleet's layout: http://127.0.0.1:8096, the key at
~/.config/secrets/jellyfin, and the database found by trying, in order, $JELLYFIN_DB,
~/.local/share/jellyfin/jellyfin.db and the quadlet's config bind.
"""
import argparse
import json
import os
import pathlib
import sqlite3
import sys
import time
import urllib.error
import urllib.request
from datetime import datetime, timezone
SCAN_TASK = "Scan Media Library"
IDLE = ("Idle",) # Running, Cancelling and Waiting all mean: not settled
DB_CANDIDATES = ("~/.local/share/jellyfin/jellyfin.db",
"/home/dank/Containers/jellyfin/config/data/jellyfin.db")
def _epoch(iso: str | None) -> float | None:
"""Jellyfin's `2026-11-04T18:31:33Z` EndTimeUtc as a unix epoch, or None."""
if not iso:
return None
for fmt in ("%Y-%m-%dT%H:%M:%S.%fZ", "%Y-%m-%dT%H:%M:%SZ"):
try:
return datetime.strptime(iso, fmt).replace(tzinfo=timezone.utc).timestamp()
except ValueError:
continue
sys.exit(f"the server sent an unparsable EndTimeUtc: {iso!r}")
def read_key(value: str) -> str:
path = pathlib.Path(value).expanduser()
try:
token = path.read_text().strip()
except OSError as exc:
sys.exit(f"no Jellyfin API key at {path} ({exc.strerror}); "
"that file is the credential this driver authenticates with")
if not token:
sys.exit(f"Jellyfin API key file {path} is empty")
return token
def request(base: str, token: str, method: str, path: str,
timeout: float = 30.0) -> tuple[int, bytes]:
req = urllib.request.Request(
base.rstrip("/") + path, method=method,
headers={"Authorization": f'MediaBrowser Token="{token}"'})
try:
with urllib.request.urlopen(req, timeout=timeout) as resp:
return resp.status, resp.read()
except urllib.error.HTTPError as exc:
return exc.code, exc.read()
def scan_task(base: str, token: str) -> dict:
"""The one task row that means something, and when its last run finished."""
code, body = request(base, token, "GET", "/ScheduledTasks")
if code != 200:
sys.exit(f"GET /ScheduledTasks -> HTTP {code}: {body[:200]!r}")
for task in json.loads(body):
if task.get("Name") == SCAN_TASK:
state = task.get("State")
return {"state": state, "running": state not in IDLE,
"lastScan": _epoch(task.get("EndTimeUtc"))}
sys.exit(f"the server does not list a {SCAN_TASK!r} task; refusing to guess")
def find_db(value: str) -> pathlib.Path:
wanted = [value] if value != "-" else []
wanted += [p for p in (os.environ.get("JELLYFIN_DB"),) if p]
wanted += list(DB_CANDIDATES)
for candidate in wanted:
path = pathlib.Path(candidate).expanduser()
if path.is_file():
return path
sys.exit("no Jellyfin database found; tried: " + ", ".join(wanted))
def recent_media(db: str, window: int) -> int:
since = time.strftime("%Y-%m-%d %H:%M:%S", time.gmtime(time.time() - window * 60))
con = sqlite3.connect(f"file:{find_db(db)}?mode=ro", uri=True)
try:
return con.execute(
"SELECT COUNT(*) FROM BaseItems WHERE DateModified >= ?", (since,)
).fetchone()[0]
finally:
con.close()
def main() -> None:
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
ap.add_argument("command", choices=["trigger", "status", "wait"])
ap.add_argument("--url", default="http://127.0.0.1:8096")
ap.add_argument("--key-file", default="~/.config/secrets/jellyfin")
ap.add_argument("--db", default="-", help="jellyfin.db; - searches the known homes")
ap.add_argument("--window", type=int, default=5,
help="minutes of DB writes that still count as recent")
ap.add_argument("--retries", type=int, default=3, help="extra attempts against a 429")
ap.add_argument("--retry-sleep", type=float, default=20.0)
ap.add_argument("--force", action="store_true",
help="trigger even though a scan is already running")
ap.add_argument("--interval", type=float, default=12.0, help="wait: seconds between reads")
ap.add_argument("--require-quiet", type=int, default=3,
help="wait: consecutive quiet reads before the library counts as settled")
ap.add_argument("--after", type=float, default=None,
help="status/wait: an epoch the scan must have finished after — pass "
"the trigger's, not a file mtime an earlier scan already passes")
ap.add_argument("--start-window", type=float, default=120.0,
help="wait: seconds to allow the scan to start before reporting "
"that the 204 queued nothing")
ap.add_argument("--timeout", type=float, default=1800.0, help="wait: seconds before giving up")
args = ap.parse_args()
token = read_key(args.key_file)
def state() -> dict:
task = scan_task(args.url, token)
out = {"scan": task["running"], "taskState": task["state"],
"lastScan": task["lastScan"],
"recentMedia": recent_media(args.db, args.window)}
if args.after is not None:
out["scannedAfter"] = bool(task["lastScan"] and task["lastScan"] >= args.after)
return out
if args.command == "status":
current = state()
print(json.dumps(current))
if args.after is not None and not current["scannedAfter"]:
sys.exit(f"the last scan finished before {args.after:.0f}; "
"a refresh is still owed")
return
if args.command == "wait":
start = time.monotonic()
ended_at_entry, quiet, started, first = None, 0, False, True
while True:
current = state()
if first:
first, ended_at_entry = False, current["lastScan"]
if (current["scan"]
or (current["lastScan"] and current["lastScan"] != ended_at_entry)
or current.get("scannedAfter")):
# The settle law: a Running sighting (or a run that finished while
# watching, or a --after the task already passes) must come BEFORE
# any claim that the library is quiet.
started = True
busy = current["scan"] or current["recentMedia"]
quiet = quiet + 1 if started and not busy else 0
print(json.dumps(current | {"quiet": quiet}), flush=True)
if quiet >= args.require_quiet:
return
elapsed = time.monotonic() - start
if not started and elapsed >= args.start_window:
sys.exit(f"{SCAN_TASK} never started: it read {current['taskState']!r} "
f"for {elapsed:.0f}s and the database took no writes. Either "
"the POST queued nothing — a POST that arrives while a scan "
"is already running cancels it, and every later 204 is "
"accepted and discarded, so restart the service and trigger "
"once — or the scan finished before this watch began, in "
"which case ask with --after <journal mtime> instead.")
if elapsed >= args.timeout:
sys.exit(f"still busy after {args.timeout:.0f}s: {current}")
time.sleep(args.interval)
if scan_task(args.url, token)["running"] and not args.force:
sys.exit(f"{SCAN_TASK} is already running on {args.url}: a second "
"POST /Library/Refresh cancels and restarts it, so the index "
"looks frozen while you wait. Poll `wait` instead of triggering "
"again (--force overrides).")
for attempt in range(args.retries + 1):
code, body = request(args.url, token, "POST", "/Library/Refresh")
if code == 204:
print("refresh accepted (204); the scan walks every library")
return
if code == 429:
if attempt < args.retries:
time.sleep(args.retry_sleep)
continue
sys.exit(f"Jellyfin is busy: {args.retries + 1} attempts all answered "
"429. Nothing was queued; run `wait`, then trigger again.")
sys.exit(f"POST /Library/Refresh -> HTTP {code}: {body[:200]!r}")
if __name__ == "__main__":
main()
+130
View File
@@ -0,0 +1,130 @@
#!/usr/bin/env bash
# Trigger and poll ONE Plex library section — the step that makes a media move visible.
#
# media-refresh-plex.sh trigger --library KEY
# media-refresh-plex.sh status --library KEY [--after EPOCH]
#
# One section only: refreshing all five costs a full walk of each and buys
# nothing for a rename that touched one tree. And a rename on the array is
# invisible until the section is rescanned — the server is bound at the SSD
# metadata view, so an array-side move is not an event its watcher ever sees.
#
# Two host realities shape this script:
# * PMS's HTTPD closes without a single byte on a request that arrives from
# another netns (the host and the 10.89.x.x container address both read as
# "empty reply"), and it serves TLS to remote clients. So the request rides
# plain http://127.0.0.1:32400 INSIDE the container, shipped in as a file —
# a three-layer ssh/podman/sh quoting chain only produces parse errors.
# * The token is `PlexOnlineToken` in the container's Preferences.xml (not
# PlexHTTPToken, which a claimed server may not carry). It is read inside
# the container and used from a curl config there: never argv (any process
# on the host reads /proc/<pid>/cmdline), never a URL (proxies log URLs).
#
# Progress needs no token and no container: `section_locations.scanned_at` in
# the library database is the same epoch the API prints as `scannedAt`
# (verified 2026-11-04), so `status` reads the database and compares it against
# --after — which makes it a census of the last RECORDED scan, never a waiter: PMS
# stamps that epoch when it accepts the request, so the door answers `settled` in the
# same second its own `trigger` ran (measured 2026-11-04; no epoch survives it).
# `trigger` answering 000 means PMS is still warming — retry, never
# restart the service over it. Proof that a rename actually landed is
# scripts/media-verify.py replaying the journal; this script only moves the
# scanner along.
set -euo pipefail
CONTAINER=plex
URL="http://127.0.0.1:32400"
PREFS='/config/Library/Application Support/Plex Media Server/Preferences.xml'
DB="/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Plug-in Support/Databases/com.plexapp.plugins.library.db"
SECTION=""
AFTER=""
TRIES=6
WARM_SLEEP=10
die() { echo "$*" >&2; exit 1; }
CMD="${1:-}"; shift || true
case "$CMD" in
trigger|status) ;;
*) die "usage: $(basename "$0") trigger|status --library KEY [--after EPOCH] [--container NAME] [--db PATH]" ;;
esac
while [ $# -gt 0 ]; do
case "$1" in
--library) SECTION="${2:-}"; shift 2 ;;
--after) AFTER="${2:-}"; shift 2 ;;
--tries) TRIES="${2:-}"; shift 2 ;;
--warm-sleep) WARM_SLEEP="${2:-}"; shift 2 ;;
--container) CONTAINER="${2:-}"; shift 2 ;;
--url) URL="${2:-}"; shift 2 ;;
--prefs) PREFS="${2:-}"; shift 2 ;;
--db) DB="${2:-}"; shift 2 ;;
*) die "unknown argument: $1" ;;
esac
done
[ -n "$SECTION" ] || die "--library is required (the section id from GET /library/sections; Movies is 1)"
command -v python3 >/dev/null || die "python3 is required to read the library database"
if [ "$CMD" = "status" ]; then
[ -r "$DB" ] || die "no readable Plex library database at $DB"
python3 - "$DB" "$SECTION" "$AFTER" <<'PY'
import datetime, sqlite3, sys
db, section, after = sys.argv[1], int(sys.argv[2]), sys.argv[3]
con = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
rows = con.execute(
"SELECT root_path, available, scanned_at FROM section_locations "
"WHERE library_section_id = ? ORDER BY id", (section,)).fetchall()
if not rows:
sys.exit(f"section {section} has no location in {db}; is the id right?")
worst = min(r[2] or 0 for r in rows)
for root, available, at in rows:
iso = (datetime.datetime.fromtimestamp(at, datetime.UTC).strftime("%Y-%m-%dT%H:%M:%SZ")
if at else "never")
print(f" available={available} scanned_at={iso} {root}")
print(f"section={section} scanned_at={worst}")
if after:
if worst < int(after):
sys.exit(f"section {section} was last scanned before {after}; a refresh is still owed")
PY
exit 0
fi
command -v podman >/dev/null || die "podman is required to reach PMS inside its container"
INNER="$(mktemp -t pxscan.XXXXXX)"
trap 'rm -f "$INNER"' EXIT
cat > "$INNER" <<'EOS'
# Runs INSIDE the plex container: $1 section key, $2 base URL, $3 Preferences.xml.
key="$1"; url="$2"; prefs="$3"
[ -r "$prefs" ] || { echo "no readable $prefs inside the container" >&2; exit 3; }
T=$(sed -n 's/.*PlexOnlineToken="\([^"]*\)".*/\1/p' "$prefs" | head -1)
[ -n "$T" ] || { echo "no PlexOnlineToken in $prefs" >&2; exit 3; }
cfg=$(mktemp)
( umask 077
{ printf 'header = "X-Plex-Token: %s"\n' "$T"
printf 'silent\nshow-error\nrequest = "POST"\n'
printf 'url = "%s/library/sections/%s/refresh"\n' "$url" "$key"; } > "$cfg" )
code=$(curl -K "$cfg" -o /dev/null -w '%{http_code}' || true)
shred -u "$cfg" 2>/dev/null || rm -f "$cfg"
[ "$code" = "401" ] && { echo "section $key: HTTP 401 — the token in $prefs is not accepted" >&2; exit 4; }
echo "$code"
EOS
attempt=1
while :; do
podman cp "$INNER" "$CONTAINER:/tmp/media-refresh-plex.sh" \
|| die "could not copy the runner into container $CONTAINER"
code="$(podman exec "$CONTAINER" sh /tmp/media-refresh-plex.sh "$SECTION" "$URL" "$PREFS" || true)"
podman exec "$CONTAINER" rm -f /tmp/media-refresh-plex.sh >/dev/null 2>&1 || true
case "$code" in
204|200) echo "refresh accepted ($code) for section $SECTION"; exit 0 ;;
000)
[ "$attempt" -ge "$TRIES" ] && die "PMS never answered after $TRIES attempts (000); it may still be warming — do not restart the service over it"
echo "section $SECTION: no answer yet ($attempt/$TRIES), PMS is warming" >&2 ;;
*) die "refresh of section $SECTION failed: HTTP ${code:-curl-error}" ;;
esac
attempt=$((attempt + 1))
sleep "$WARM_SLEEP"
done
+10 -4
View File
@@ -78,10 +78,16 @@ one journal is one undo:
python3 scripts/media-shows.py --root /mnt/entropy/TVShows \
--enum /tmp/tv-enum.tsv --titles /tmp/jellyfin-tv.tsv \
--tvdb --tmdb --out /mnt/entropy/media-tv-plan.tsv \
--park /mnt/entropy/media-park/TVShows
python3 scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/media-tv-relayout.jsonl
--tvdb --tmdb --out /mnt/entropy/.media-campaign/media-tv-plan.tsv \
--park /mnt/entropy/NeedsProcessing/TVShows
python3 scripts/media-apply.py apply \
--plan /mnt/entropy/.media-campaign/media-tv-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/.media-campaign/media-tv-relayout.jsonl
Every plan, journal and log lives in /mnt/entropy/.media-campaign/ — the array
root holds libraries only. media-apply.py defaults its journal to
<root>/media-relayout.jsonl, so pass --journal explicitly or the run puts the
log back on the root.
"""
import argparse
import collections
+13 -6
View File
@@ -17,13 +17,20 @@ the executor's own path guard still has teeth, and give it its own journal —
one journal is one undo, and a TV move must not be welded to a movie relayout:
python3 scripts/media-tv.py --root /mnt/entropy/Movies \
--library /mnt/entropy/TVShows --out /mnt/entropy/media-tv-plan.tsv
python3 scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/media-relayout-tv.jsonl
python3 scripts/media-apply.py verify --plan /mnt/entropy/media-tv-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/media-relayout-tv.jsonl
--library /mnt/entropy/TVShows \
--out /mnt/entropy/.media-campaign/media-tv-plan.tsv
python3 scripts/media-apply.py apply \
--plan /mnt/entropy/.media-campaign/media-tv-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/.media-campaign/media-relayout-tv.jsonl
python3 scripts/media-apply.py verify \
--plan /mnt/entropy/.media-campaign/media-tv-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/.media-campaign/media-relayout-tv.jsonl
python3 scripts/media-apply.py sweep --root /mnt/entropy \
--journal /mnt/entropy/media-relayout-tv.jsonl --apply
--journal /mnt/entropy/.media-campaign/media-relayout-tv.jsonl --apply
Plans and journals live in `/mnt/entropy/.media-campaign/`, never loose on the
array root: `media-apply.py` defaults its journal to `<root>/media-relayout.jsonl`,
so `--journal` is always explicit here.
Guards, and why each one is here:
+8 -6
View File
@@ -16,14 +16,16 @@ the journal and frees no space until the hold is purged.
python3 scripts/media-tvdupes.py --library /mnt/entropy/TVShows \
--hold /mnt/entropy/.AGENTTRASH/20261102-tv-duplicates \
--out /mnt/entropy/tv-dupes-plan.tsv
python3 scripts/media-apply.py apply --plan /mnt/entropy/tv-dupes-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/tv-dupes.jsonl \
--out /mnt/entropy/.media-campaign/tv-dupes-plan.tsv
python3 scripts/media-apply.py apply \
--plan /mnt/entropy/.media-campaign/tv-dupes-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/.media-campaign/tv-dupes.jsonl \
--metadata-view /var/lib/media-meta/view --apply
python3 scripts/media-apply.py verify --plan /mnt/entropy/tv-dupes-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/tv-dupes.jsonl
python3 scripts/media-apply.py verify \
--plan /mnt/entropy/.media-campaign/tv-dupes-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/.media-campaign/tv-dupes.jsonl
python3 scripts/media-apply.py sweep --root /mnt/entropy \
--journal /mnt/entropy/tv-dupes.jsonl --apply # empty folders only
--journal /mnt/entropy/.media-campaign/tv-dupes.jsonl --apply # empty folders only
scripts/media-view.sh reap # ghost sidecars above
Then both servers have to be told: the plan moves files on the raw array, and no
@@ -68,8 +68,8 @@ service on the other host.
| Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. |
The shared chat template maps to the inference service on the selected target.
The `frank-npu` endpoint is retired () and has no deploy handler
an unrelated large-model deployment never touches it.
The `frank-npu` endpoint is retired and has no deploy handler, so an unrelated
large-model deployment never touches it.
## Caddy safety
+1 -1
View File
@@ -222,5 +222,5 @@ harness bundle and measured the day of the flip:
| Endpoint answers, synthesis fails | the engine: `podman healthcheck run frank-tts-engine`, then `systemctl --user status frank-tts-engine.service` |
| First request of the day is ~0.6 s slower than the rest — or every name returns 400 | the warm pass did not run; it carries `-`, so it cannot fail the unit. `journalctl --user -u frank-tts-engine` for the warmer's lines, then `podman exec frank-tts-engine /app/warm-voices.sh` — the registry is memory-only (law 1 of [[Frank TTS Engine]]) |
| The unit never appears after a deploy | the generator dropped it: `frank-tts-engine.network` is missing from `~/.config/containers/systemd/` |
| A consumer rejects `wilwheaton` by listing 29 names (`af_heart … bm_fable`) and no request ever reaches this endpoint | that consumer is synthesizing **in the browser** with Kokoro.js, whose voice pack is fixed — OpenWebUI 0.11.4 ships it as the `browser-kokoro` TTS engine; see [[OpenWebUI]] |
| A consumer rejects `wilwheaton` by listing 29 names (`af_heart … bm_fable`) and no request ever reaches this endpoint | that consumer is synthesizing **in the browser** with Kokoro.js, whose voice pack is fixed. In OpenWebUI the `browser-kokoro` choice is that user's own `user.settings` row (`ui.audio.tts.engine`) and the SPA re-serves it on every load, so flipping the dropdown does not stick; see [[OpenWebUI]] |
---
+121 -38
View File
@@ -21,7 +21,7 @@ https://github.com/open-webui/open-webui
| Item | Value |
|------|-------|
| Image | `ghcr.io/open-webui/open-webui:main` (verified from the live quadlet unit) |
| Deployed build | `0.11.4` — `GET http://127.0.0.1:3005/api/version` → `{"version":"0.11.4"}` (2026-10-18). `:main` is the rolling image, not a stable pin, so the running build lags upstream: releases `0.12.0` (2026-10-10) through `0.12.3` (2026-10-16) exist and are not deployed (https://github.com/open-webui/open-webui/releases). The 2026-09-21 `0.11.4` security/access-control advisory is what pinned this era of the page. Imagegen endpoint is `POST /api/v1/images/generations` (the `comfyui` engine is still upstream). |
| Deployed build | `0.12.5` — `GET http://127.0.0.1:3005/api/version` → `{"version":"0.12.5"}` (re-measured 2026-11-04). The repo pin `@sha256:0b399a8b…` (`dankclaw/open-webui/quadlet/open-webui.container`, bumped 2026-10-30) is what makes that build reproducible; `:main` is a rolling tag, so re-measure `/api/version` instead of quoting a build from this page. Imagegen endpoint is `POST /api/v1/images/generations` (the `comfyui` engine is still upstream). |
| Port | 3005 (host) → 8080 (container). Port 3000 is Gitea's HTTP, so it is not usable. |
| Data volume | `/home/dank/Containers/open-webui/data` → `/app/backend/data` |
| Backends | The picker's content is `webui.db` state, not repo state: as recorded on 2026-10-07 it exposes **one** model, `frank-zealot`, with `frank-dragoon`/`frank-archon` configured but disabled — *unverified 2026-10-18* (the DB is the authority and is not readable from the repo). What the repo does fix is the container's own OpenAI target: `OPENAI_API_BASE_URL=http://192.168.0.143:7770` (`dankclaw/open-webui/quadlet/open-webui.container:32`, same value in the live container env) — see [The LLM backends](#the-llm-backends) |
@@ -109,6 +109,39 @@ The Ollama connection is disabled (`ollama.enable=false`).
> connection here only hides it from this picker — the endpoint itself keeps
> serving every other client (the harness, Hermes, API consumers).
## Default model, descriptions, and the admin API door
**The default model is `ui.default_models`** — one row of the config table, a
comma-separated **string**, not a list. Live value `frank-archon` since
2026-11-04 (per Frank). The front end resolves the picker as *the browser's own
stored pick, else `config.default_models.split(",")`*, so the value decides what
a fresh profile or a cleared-storage session lands on; after a user picks
something themselves, their pick keeps winning. Verified 2026-11-04 in a clean
browser: signed in with `localStorage` empty → the pill reads
`Selected model: frank-archon`.
Two admin-token doors, both driven against loopback `:3005` — service-to-service
traffic never rides the public vhost:
- **Config:** `POST /api/v1/configs/import` with
`{"config":{"ui.default_models":"frank-archon"}}` — the general per-key setter
(no `PATCH /configs/{key}` route exists in 0.12.5; verified against
`routers/` + `main.py`), and it returns the whole config so the write reads
back in the same call.
- **A model row:** `POST /api/v1/models/model/update` with a full `ModelForm`
(`id`, `name`, `meta`, `params`, `is_active`). **`meta` and `params` REPLACE,
they do not merge:** read the row first, change one field, send everything
back — a partial payload silently drops `meta.capabilities` and the
profile/background images.
**An API key is not accepted everywhere.** Routes guarded by `get_human_user`
(`/api/v1/auths/`, `utils/auth.py`) answer **403 `A human session is required.`**
for any token whose `typ` is not `session` — that is not a dead key. The two
mutation doors above take `get_admin_user` / `get_verified_user`, which do accept
the key. Prove a key on `GET /api/v1/models` with **no trailing slash**:
`/api/v1/models/` falls through to the SPA and answers `200` + `text/html`, which
looks like success and parses as nothing.
## Container introspection (DB and backend python)
- Working dir for imports: `/app/backend`; package name `open_webui`
@@ -163,14 +196,11 @@ account under Profile → API Keys). Anonymous web visitors get the `pending`
role (`ui.default_user_role=pending`), which cannot chat. Verified
2026-10-07 from loopback `:3005` and over the public vhost:
`GET /api/v1/models/list`, `GET /api/v1/models` and
`POST /api/chat/completions` all return **401** with no key (also with a
bogus bearer token); only `/api/config`, `/api/version` and `/health`
`POST /api/chat/completions` all return **401** with no key (also with a bogus bearer
token); only `/api/config`, `/api/version` and `/health` answer publicly, and none of
them expose the model list. An external program needs an account + API key — there is
no keyless access. Re-verified 2026-10-18 from loopback: `/health` → `{"status":true}`
`/api/version` → 0.11.4, `/api/v1/models/list`, `/api/v1/models` and
`POST /api/chat/completions` → 401; `/api/config` additionally publishes
no keyless access. Re-measured 2026-11-04 from loopback: `/api/version` → `0.12.5`,
and `/api/config` publishes
`features.enable_login_form=false`, `features.enable_signup=false` and
`oauth.providers.oidc="authentik"` — the SSO-only posture, visible from outside.
@@ -219,18 +249,26 @@ browser-driven and only touches the two exempt vhosts (`ai.` and `auth.`).
into the existing local account with the same email — the pre-SSO admin
(frank8465@protonmail.com) keeps its data. `ENABLE_OAUTH_SIGNUP=true` lets
other `openwebui_users` create accounts on first login.
- **First login mints role `pending`.** `ui.default_user_role=pending` (config
DB, verified 2026-10-09), so a freshly-provisioned `openwebui_users` account
completes SSO and then cannot chat (picker empty) until its role is flipped:
`UPDATE "user" SET role='user' WHERE email='<user>@franksdank.cloud';` —
run with host `python3` over ssh against
- **First login mints the role from the claim, not from
`ui.default_user_role`.** The quadlet sets `ENABLE_OAUTH_ROLE_MANAGEMENT=true`
and `OAUTH_ROLES_CLAIM` defaults to `roles`, so the `roles: ["user"]` claim
decides the role. Measured 2026-11-04: a freshly-provisioned
`openwebui_users` account completed SSO in a clean browser and the row
appeared as `role='user'` — chat answered and Read Aloud synthesized, with no
manual step. `ui.default_user_role=pending` (config DB) is the fallback for an
account the claim does *not* describe, and 0.12.5's `get_user_role` **refuses
the login with 403** when role management is on and no roles claim is
readable — so "SSO succeeded but I cannot chat" points at a missing
`openwebui_users` group/entitlement, not at a row waiting to be flipped. Flip
only a row that is actually stuck:
`UPDATE "user" SET role='user' WHERE email='<user>@franksdank.cloud';` — run
with host `python3` over ssh against
`/home/dank/Containers/open-webui/data/webui.db` (no `sqlite3` CLI on
Dankclaw; table is `user`, singular; no `active` column). No per-user model
grant is needed: the picker model `frank-zealot` carries the global
`access_grant` row `(model, frank-zealot, user, *, read)`, so any
`role='user'` account sees it. Flip on FIRST-LOGIN SIGHTING — the row does
not exist until the account signs in (2026-10-09: five provisioned accounts
had no row).
Dankclaw; table is `user`, singular; no `active` column). The row does not
exist until the account signs in, so census it after a first-login sighting
(2026-10-09: five provisioned accounts had no row). No per-user model grant is
needed: the picker models carry the global `access_grant` row
`(model, <id>, user, *, read)`, so any `role='user'` account sees them.
- **IdP reachability:** the container is on its own bridge (10.89.3.0/24) and
cannot reach `authentik-server:9000` by name. It reaches the IdP over the
public URL `https://auth.franksdank.cloud` — Caddy publishes `0.0.0.0:443`
@@ -305,28 +343,58 @@ port is the rootless quadlet `frank-tts` (this repo:
400 listing the voices that exist, never a silent fallback.
**The 29-voice `af_* … bm_fable` list is NOT frank-tts.** That enum belongs
to **Kokoro.js running in the browser**: the deployed front end (0.11.4)
ships an in-page Kokoro implementation (chunk carrying a frozen voice table
`{af_heart:{name:"Heart",…}}` and `_validate_voice`, model
`onnx-community/Kokoro-82M-v1.0-ONNX`, plus a personal *Kokoro.js dtype*
setting). Its engine select takes `""` (Default) or `browser-kokoro`; when
`browser-kokoro` is chosen the page synthesizes locally and rejects any
voice outside its fixed pack — so `wilwheaton` fails with
`Voice: "wilwheaton" not found. Should be one of: af_heart, …` **without a
single request reaching the server**. Personal audio settings are
browser: no `user.settings` row and no container env carries an audio
override, yet the toast still appears ⇒ the choice lives in that browser's
own storage. Fix in the UI: Settings → Audio → TTS Engine → **Default**,
which hands synthesis to the admin config (frank-tts); picking `af_heart`
instead keeps Kokoro, and Kokoro cannot clone — fixed pack only.
to **Kokoro.js running in the browser**: the deployed front end ships an
in-page Kokoro implementation (frozen voice table plus `_validate_voice`,
model `onnx-community/Kokoro-82M-v1.0-ONNX`, and a personal *Kokoro.js dtype*
setting). Its engine select takes `""` (Default) or `browser-kokoro`; with
`browser-kokoro` the page synthesizes locally and rejects any voice outside
its fixed pack, so `wilwheaton` fails with `Voice: "wilwheaton" not found.
Should be one of: af_heart, …` **without a single request leaving the
browser**. Kokoro cannot clone — fixed pack only.
Verified 2026-10-18 from inside the container with the exact OpenWebUI body
(`{"model":"frank-tts","input":…,"voice":"narrator1",
"response_format":"mp3","speed":1.0}`) → **200, `audio/mpeg`, 7,629 B**,
while `podman logs open-webui` recorded **zero** `audio/speech` requests for
the failing click — that contrast is the instrument: a server-path failure
leaves a log line and a frank-tts error, a `browser-kokoro` failure leaves
neither.
**The switch is a DB row, not browser storage.** Personal audio settings are
the `settings` JSON column of table `user`, and in 0.12.x they nest under
`ui`: `ui.audio.tts.engine`, `.voice`, `.defaultVoice`. In 0.11.x the same
keys sat at the top level (`audio.tts.*`), and the 2026-10-18 probe listed
only top-level keys, found no audio key, and concluded the choice lived in
that browser's own storage — a probe-depth bug. The row carried the override
the whole way; the 2026-10-30 pin bump to 0.12.5 moved the keys, not the bug.
The SPA reads that row at load (`GET /api/v1/users/user/settings`) and posts
the **whole** blob back whenever any preference changes
(`POST …/settings/update`, body `{...settings}`). So the dropdown can fix it —
but a second tab still holding the old copy writes `browser-kokoro` straight
back, which is the loop that makes the toast return after it was "fixed". The
user row is the authority here; the admin `config` table was never the problem
(its `audio.tts.engine` has been `openai` throughout). Repair
`settings.ui.audio.tts` with the image's stdlib `sqlite3` (`mode=ro`; quote
identifiers, `user` and `group` are reserved words — there is no `sqlite3` CLI
in the image or on the host), then reload every tab of that account.
**Repair the engine *and* the voice.** With engine `""` the front end still
sends its stored voice in the request body, so a Kokoro-era `voice`/
`defaultVoice` survives the engine fix as frank-tts's own rejection —
measured 2026-11-04: `voice: "alloy"` → **400 `unknown voice 'alloy';
available voices: dr.kern, garrickhagon, jeffhays, vaati, wilwheaton`**. Set
`ui.audio.tts` to `{"engine": "", "voice": "wilwheaton", "defaultVoice":
"wilwheaton"}` and reload. Engine `""` means "the admin config decides", and
the handler fills `voice` from `audio.tts.voice` only when the body omits it.
**Two tells settle the case in one line:**
- a session that fetched `/wasm/ort-wasm-simd-threaded.jsep.wasm` loaded
onnxruntime, so Kokoro.js is running in that browser — the server path
never asks for that file;
- a server-path failure leaves a uvicorn request line for
`/api/v1/audio/speech` **and** a frank-tts log line; a `browser-kokoro`
failure leaves neither. uvicorn logs every request, so silence is evidence.
Verified 2026-11-04: the handler's own outbound shape
(`{"model":"frank-tts","input":…,"voice":"wilwheaton"}` with
`Authorization: Bearer none`, the configured `audio.tts.openai.api_key`) →
**200 `audio/wav`** — a body without `response_format` comes back WAV and
either MIME is browser-playable, so OpenWebUI accepts it. The full route as a
real user over the vhost, `POST /api/v1/audio/speech` → **200 `audio/mpeg`**,
with frank-tts logging `speech voice=wilwheaton`.
## Retired instances
@@ -396,6 +464,21 @@ neither.
app process env; the `.webui_secret_key` file happens to equal it — use it
explicitly, `podman exec -e`).
- **2026-11-04 — decoration rows drifted again; default model set to
`frank-archon`.** Two payload swaps had gone by without anyone touching the
picker: `frank-dragoon`'s row still advertised halogen `0.16.2`, and
`frank-zealot`'s still said `build 10777` / `3×262,144` / *MTP off* — live is
`0.18.20`, `build 12118`, `3 × 200,704`, MTP armed. The prose is cosmetic; the
`params.context.length` values had **swapped**, and that one bites:
`frank-zealot` advertised 262,144 while the endpoint refuses anything past its
200,704 slot, so a long zealot chat rides the endpoint's 400 instead of
compacting, and `frank-dragoon` advertised 200,704 against a 262,144 ceiling,
giving up 61k tokens of window for nothing. Both rows re-keyed through the API
door in [Default model, descriptions, and the admin API door](#default-model-descriptions-and-the-admin-api-door),
descriptions rewritten from the env files, and `ui.default_models` set to
`frank-archon`. **Law: a model swap is not finished until that endpoint's
`model` row is re-keyed as well — no deployer step touches `webui.db`.**
## See Also
- [[Dankclaw Stack]]
@@ -1,11 +1,11 @@
---
type: reference
description: "Audiobookshelf on Dankserver: quadlet shape, the read-only array bind, state on the SSD, the API bootstrap, laws."
description: "Audiobookshelf on Dankserver: quadlet shape, array bind, SSD state, API bootstrap, authentik SSO, public vhost, laws."
tags:
- dankserver
- audiobooks
- quadlet
timestamp: 2026-11-02
timestamp: 2026-11-03
---
# Audiobookshelf
@@ -16,8 +16,8 @@ per-listener playback positions, and streams the audio. It is **not** a media
server — it shares nothing with [[Plex]] or [[Jellyfin]] and owns no files on the
array.
**Host:** Dankserver — **Unit:** `audiobookshelf.service` (rootless Quadlet, user
manager) — **Internal-only:** LAN address, no vhost, no DNS name.
**Host:** Dankserver — **Unit:** `audiobookshelf.service` (rootless Quadlet, user manager) —
**Doors:** LAN `http://192.168.0.235:8000/audiobookshelf` (plain HTTP, nothing in front) and public `https://audiobooks.franksdank.cloud/audiobookshelf/` — Caddy straight to the container, **exempt from the guard chain** because the app must answer non-browser clients ([[Caddy]] *Unguard a vhost*). The public name is the address every client carries: Pi-hole answers it with `192.168.0.143` on the LAN, Porkbun answers it with the WAN address outside, and Caddy serves a real certificate either way.
## Shape
@@ -26,7 +26,7 @@ manager) — **Internal-only:** LAN address, no vhost, no DNS name.
| Unit file | `dankserver/audiobookshelf/quadlet/audiobookshelf.container` (the authority for every number here) |
| Image | `ghcr.io/advplyr/audiobookshelf` pinned by **amd64 manifest digest** — see Laws |
| Port | `Environment=PORT=8000` + `PublishPort=8000:8000` — web UI and API on one port |
| Base path | `Environment=ROUTER_BASE_PATH=/` — the URL is the plain `host:port`, no `/audiobookshelf` prefix |
| Base path | `Environment=ROUTER_BASE_PATH=/audiobookshelf` — every URL below carries this prefix, and the shipped SPA cannot boot without it (Laws) |
| Network | `pasta` (standalone; publishes nothing to other containers) |
| Audio bind | `/mnt/entropy/Audiobooks:/audiobooks:ro` — read-only, by design |
| State | `/home/dank/Containers/audiobookshelf/config` → `/config`, `…/metadata` → `/metadata` (both NVMe) |
@@ -36,7 +36,8 @@ manager) — **Internal-only:** LAN address, no vhost, no DNS name.
| Self-healing | `Restart=always`, `RestartSec=5`, `WantedBy=default.target` |
| Deploy | `./scripts/deploy.sh dankserver --service audiobookshelf` |
| Handler | `deploy_audiobookshelf` in `scripts/deploy-remote.sh` |
| Library layout | one book per folder — ABS groups by the first directory holding a media file (see Laws); every move is journalled at `/mnt/entropy/media-relayout-audiobooks.jsonl` |
| Single sign-on | authentik provider `audiobookshelf`; the local `frank` account stays enabled as break-glass — see *Single sign-on* |
| Library layout | one book per folder — ABS groups by the first directory holding a media file (see Laws); every move is journalled at `/mnt/entropy/.media-campaign/media-relayout-audiobooks.jsonl` |
## Bootstrap
@@ -54,17 +55,109 @@ difference. The script is idempotent; re-running it costs one incremental scan.
| scan | `POST /api/libraries/<id>/scan` | `scan: accepted` |
| proof | `GET /api/libraries/<id>/items?limit=1` | `items: <total>` |
Sign in at `http://192.168.0.235:8000` as `frank`; the password is
Sign in at `http://192.168.0.235:8000/audiobookshelf` as `frank`; the password is
`awk -F= '/^ABS_ADMIN_PASSWORD=/{print $2}' ~/.config/secrets/audiobookshelf` on
Dankserver. Change it in **Settings → Users → frank → Password**; the secret file
then holds a stale value and only matters again if the `config` bind is ever
destroyed (a fresh DB makes the script create a new owner from it).
That value is the live password, not a leftover: it bcrypt-matches `users.pash` in the state
DB (2026-11-03), so `local` does open the door when authentik is down.
A full scan is slow and runs in the background; watch the `items:` total climb,
never `/healthcheck`. It also reads files that are genuinely broken and logs
`[AudioFileScanner] Invalid data found when processing input` per file — that is
the file, not the bind: the same line count grows while the total climbs.
## Phone app
The official **Audiobookshelf** app connects with Server Address
`https://audiobooks.franksdank.cloud/audiobookshelf` — the base path belongs in the address —
username `frank`, and the same password as the web UI, or the SSO button. One address works
at home and away: on the LAN Pi-hole answers that name with `192.168.0.143` and Caddy dials
the container; off-LAN the same name resolves to the WAN address and takes the same route
(inbound 443 is forwarded at the router — fetched from a datacenter node 2026-11-03). The raw
LAN address still answers and is the break-glass door whenever Caddy is down. `/ping`
(`{"success":true}`) and `/healthcheck` (`OK`) answer at the ROOT whatever `ROUTER_BASE_PATH`
says, so the app's connectivity probe works before it resolves the base path at all, and
`download: true` on the account is what makes offline listening work.
**A home device that bypasses Pi-hole cannot reach it.** The G36 does not hairpin: resolving
the name to the WAN address from inside the LAN times out (measured 2026-11-03). Only a
client pinned to an external resolver hits this — everything using the router's DNS gets the
LAN answer.
**The login token is at `user.token`, not top level.** `POST /login` answers
`{ user, serverSettings, ereaderDevices, Source, userDefaultLibraryId }` and the bearer sits
inside the user object. A client that reads `response.token` gets the empty string, sends
`Authorization: Bearer ` with nothing after it, and every `/api/*` call then answers `401`
as though the password were wrong.
**`POST /api/login` is not a route.** Local login is `POST <base>/login`, and the `/api`
bearer guard answers anything else with a bare `401 Unauthorized` while logging **nothing** —
no `[LocalAuth]` line — so a wrong path is indistinguishable from a wrong password unless the
journal is read too. Same family as the absent `/setup` route named in
`scripts/audiobookshelf-init.sh`. To settle a suspected stale credential without signing in:
bcrypt-compare the recorded value against `users.pash` in
`/home/dank/Containers/audiobookshelf/config/absdatabase.sqlite`, opened read-only
(`file:<path>?mode=ro`).
## Listeners (accounts)
The door is public now, so the account is the only gate between a listener and the library.
Two ways in, both per listener:
- **Local account** — **Settings → Users → Add User**: type `user`, tick only the libraries
they may hear, and leave `download` off unless they need offline listening. Hand them the
server address above and their own password.
- **Household SSO** — create the authentik account, and that is the whole of it: the first
**Sign in with Authentik** auto-registers the ABS account (`authOpenIDAutoRegister`) as a
listener, because the claim grants that role to every signer. To shut a listener off,
disable or delete their user in Audiobookshelf — no IdP group membership can withhold it.
What auto-registration hands out (read off the state DB, 2026-11-03): `accessAllLibraries:
true` and `download: true` — every household signer lands on the **whole** library with
offline downloads already on. Narrow it per account in **Settings → Users → <name>**. The DB
is the census when no admin token is at hand; `GET /api/users` returns the *limited* shape
(no `active`, no `librariesAccessible`), so a missing key there is not a denial, and an
account created by SSO has `pash` NULL with its IdP link at `extraData.authOpenIDSub`.
Nothing rate-limits `POST /audiobookshelf/login` — this Caddy build carries no `rate_limit`
module and ABS 2.37.2 ships no lockout — so the form is a guessing target exactly like
`jelly.`, `plex.` and the *arr logins already are. The mitigation is credential hygiene:
long unique passwords per listener, never a reuse of the root password.
## Single sign-on (authentik)
Both doors take the household identity: the web UI puts **Sign in with Authentik** beside
the local form, and the phone app offers SSO on its server-address screen. The local
`frank` account stays enabled on purpose — `authActiveAuthMethods` is `local` **and**
`openid`, and `local` is how you get in when the IdP is down.
| Piece | Owner |
|-------|-------|
| Provider `audiobookshelf`, the admin group, the scope mapping | `scripts/authentik-audiobookshelf-sso.sh` — re-run it after an authentik rebuild, because this state lives in the IdP's database and a blueprint re-import reverts it silently |
| Client secret | `ABS_OIDC_CLIENT_SECRET` in `~/.config/secrets/audiobookshelf` on Dankserver (mode 600; never in the repo, a URL or an argv) |
| The ABS half: issuer, the four IdP URLs, claim, auto-register | `scripts/audiobookshelf-init.sh`, armed automatically when that secret exists in the credential file |
| Role at first SSO sign-in | the `abs_groups` claim: every signer gets `user`; `audiobookshelf_admins` adds `admin`. Nothing in the IdP can deny the listener role — the login itself is the gate (law since 2026-11-03; the `audiobookshelf_users` group was retired, it had zero members and every household account was being refused) |
| Admin group members | `akadmin` (the operator's own authentik admin account — sign in as it and the claim grants site admin) and `qa-e2e-admin` (the identity the deploy gate signs in as, so the script re-adds it every run; drop it and the gate fails, not a phone) |
| Guard state of the door | none since 2026-11-03: proxy provider `audiobooks-proxy` and application "Audiobookshelf Guard" were deleted and the embedded outpost un-claimed them, because that chain answers a native client with challenge HTML ([[Caddy]] *Unguard a vhost* for the order). The SSO relying party `audiobookshelf` above is a different object and stays |
| Redirect URIs on the RP | `<base>/auth/openid/callback` **and** `<base>/auth/openid/mobile-redirect` for **both** origins (LAN address and public name): the app sends the mobile URI belonging to the door it was configured with, and STRICT matching answers an unregistered one with "Redirect URI Error" |
With `authOpenIDAutoRegister` on, a first sign-in creates the account and the claim alone
decides its type — and the claim always carries `user`, so **the IdP is now the library gate**:
any authentik account that can reach this application becomes a listener. That is the
intent (the household signs in with the accounts it already has), and its breadth is
accepted on purpose: no access policy is bound to the `audiobookshelf` application, and
the wide auto-registered default (every library, downloads on) is **not a defect to fix** —
Frank's call, 2026-11-03. Narrowing stays a per-account judgement in **Settings → Users**.
The one thing still worth watching is authentik's own invitation/signup flow: it must
remain something only Frank drives, because an account created there is a listener the
moment it signs in. Revoking one person is a disable/delete of their ABS user; revoking
everyone is that signup flow.
Both halves — authentik's provider row and ABS's settings — are asserted end to end by the Playwright gate named in *Verification*. `23-` signs in as `qa-e2e-admin` and must land inside the app with site-admin rights; `24-` signs in as the standard identity, which is in no Audiobookshelf group, and must land as a listener. Together they pin the escalation and the default: a mapping that stops granting silently locks the household out, and a mapping that over-grants turns the household into site admins.
## Laws
- **Speak the image's env names, not the README's.** The server reads `PORT`,
@@ -75,6 +168,54 @@ the file, not the bind: the same line count grows while the total climbs.
`active (running)` while every probe of `:8000` was refused and the deploy gate
failed at 180 s. Two seconds with the pinned digest settles it:
`podman inspect --format '{{.Config.Env}}' <image>`.
- **The URL carries `/audiobookshelf`, and that is not a style choice.** The SPA in
`/app/client/dist/index.html` is built with `routerBasePath:"/audiobookshelf"`, so it asks
for its assets under that prefix whatever the server is mounted at. Mounted at `/`, every
script request falls through to the SPA fallback, comes back `text/html`, Chrome refuses to
execute it, and the page never leaves "Loading…" — while `/healthcheck`, the API and the
phone app stay perfectly healthy, which is how this hid behind an API-only verification
(measured 2026-11-02). Prove a UI claim in a browser, never from `/health`.
- **The OIDC redirect subfolder is a value you send, not a default you inherit.** The 2.17.4
upgrade only fills `authOpenIDSubfolderForRedirectURLs` on a database that ALREADY had OIDC
enabled, so a DB created before SSO leaves it unset and the server builds
`http://host/undefined/auth/openid/callback`. authentik answers the authorize request with
"Redirect URI Error" and nothing on the ABS side looks wrong (measured 2026-11-02). The
provider's registered URI must equal `<origin><subfolder>/auth/openid/callback`, so each
script derives the value from one variable of its own (`ABS_URL`, `ABS_BASE`) and
`tests/audiobookshelf_init.bats` pins what reaches the wire.
- **An ORM-created provider has no grant types.** authentik 2026.8 defaults `grant_types` to
`[]`, and `/application/o/authorize/` then refuses the code flow with oauthlib's generic
"The request is otherwise malformed" — the wire names nothing, the server log line that names
it is `Invalid grant_type for provider`. Worse, the redirect-URI check runs *first*, so a
provider carrying both faults shows only "Redirect URI Error" until the URI is fixed, and the
second wall appears after the fix ships (measured 2026-11-02: the browser reached authentik,
got `code`+`state`, and ABS still bounced to `/login?error=undefined`). The script sets
`authorization_code` + `refresh_token`; a blueprint re-import reverts the row silently.
- **The guarded door has to tell the truth about its scheme.** ABS validates every OIDC
`callback` against the origin it believes it is served from — `getRequestOrigin()` is
`x-forwarded-proto://Host` (`server/utils/requestUtils.js`, read in the pinned digest
2026-11-03) — so behind a TLS edge that reaches the container over plain http it computes
`http://audiobooks.franksdank.cloud` and answers its own https callback with a 400 "Invalid callback
URL - must be same-origin" before the IdP is ever contacted. The Caddy `:8017` route pins
`header_up X-Forwarded-Proto https` for exactly that reason ([[Caddy]] *Guard a new vhost*);
the LAN door needs nothing, because there browser and server agree on `http`.
- **The claim mapping is not the whole `property_mappings` list.** A provider holding only the
custom mapping grants a token scoped shorter than the scopes ABS requests
(`openid profile email abs_groups`): the code is issued and exchanged normally, and ABS's
`/userinfo` call then dies `insufficient_scope (The request requires higher privileges than
provided by the access token)` — a fault that reads like an ABS bug and is an IdP row. Attach
the built-in OpenID `openid`/`profile`/`email` mappings beside the claim mapping, which is
what every working RP on this IdP carries (measured 2026-11-02).
- **With `authOpenIDMatchExistingBy` unset, an SSO identity never lands on an existing ABS
account, and a name collision is a hard denial.** In that mode `findUserFromOpenIdUserInfo`
matches on `sub` only, so every first sign-in auto-registers a NEW user; if the IdP username
equals an ABS username — the local root `frank` is the one here — the create hits the unique
constraint and the callback dies `Unauthorized`. So "create a `frank` user in authentik" does
not reach the root account: either set `authOpenIDMatchExistingBy` to `username` (links the
IdP account to the existing one, and `setUserGroup` refuses to downgrade a root — root plus an
`admin` claim is allowed, root plus `user` is denied) or pick a non-colliding IdP name. Read
off `server/models/User.js` and `server/auth/OidcAuthStrategy.js` in the pinned digest
(2026-11-02).
- **The audio bind stays `:ro`.** ABS writes nothing that belongs beside the
audio: its library, covers, progress and sidecar metadata all land in
`/metadata` on the SSD. A writable bind would put stray writes on the spindle
@@ -105,10 +246,17 @@ the file, not the bind: the same line count grows while the total climbs.
- **No `systemctl --user enable`.** Generated Quadlet units cannot be enabled on
this host; `WantedBy=default.target` in the unit file is what carries the
service across a reboot ([[Quadlet Lifecycle]]).
- **LAN-only until Frank says otherwise.** `0.0.0.0:8000` with no reverse proxy
in front. Any public vhost requires the Anubis + Authentik chain first, and the
outpost proxy provider has to exist before the vhost flips (external exposure
policy in `AGENTS.md`).
- **One door for every client, and it sits outside the guard chain on purpose.**
`https://audiobooks.franksdank.cloud/audiobookshelf` is what browsers and apps use;
`http://192.168.0.235:8000/audiobookshelf` is the raw LAN socket that same Caddy dials,
so it doubles as break-glass when the proxy is down. Behind the chain (until
2026-11-03) the vhost answered the phone app's `/ping` and `/socket.io/` handshake with
Anubis' challenge HTML — Anubis demands a JavaScript proof and the outpost needs a
session cookie, and a native client solves neither — so the app evaluated
`'isInit' in e.data` on a non-object, reported "Failed to contact server", and the
container's request log never recorded the attempt. Podcast RSS and OPDS readers died
the same death. ABS's own auth is the gate, as it is for [[Plex]] and [[Jellyfin]]
([[Caddy]] *Unguard a vhost*).
- **There is no `/setup` route; the creation door is `POST /init`.** `/status`
(no `/api` prefix) reports `isInit` = "a root user exists", `POST /init` with
`{"newRoot":{"username","password"}}` creates it, `POST /login` returns the
@@ -188,12 +336,29 @@ the file, not the bind: the same line count grows while the total climbs.
## Verification
`./tests/run.sh` covers this unit through `tests/dankserver_quadlet.bats`
(boot persistence, the SSD state roots, the `:ro` array bind, the mount gate, the
`wget` health check), `tests/audiobookshelf_init.bats` (the bootstrapper against a
stub of the API above: create-once, idempotent re-run, no secret in a URL, refusal
by URL when the server does not answer, and the deployer calling it only after the
health gate) and `tests/media_overlay.bats` (only [[Plex]] and [[Jellyfin]] may
reference the metadata view).
(boot persistence, the SSD state roots, the `:ro` array bind, the mount gate, the `wget`
health check, the base path the shipped SPA needs), `tests/audiobookshelf_init.bats` (the
bootstrapper against a stub of the API above: create-once, idempotent re-run, no secret in a
URL, refusal by URL when the server does not answer, the deployer calling it only after the
health gate, and the SSO arm sending the mounted base path as the redirect subfolder);
`tests/media_overlay.bats` (only [[Plex]] and [[Jellyfin]] may reference the metadata view);
and the live Playwright gates `tests/e2e/specs/23-audiobookshelf-sso.spec.ts` and
`tests/e2e/specs/24-audiobookshelf-sso-default-listener.spec.ts`, run by `tests/e2e/run.sh` at
the end of every `deploy.sh`. `23-` (the `admin` project) signs in as `qa-e2e-admin` against
`http://192.168.0.235:8000/audiobookshelf` and asserts the two things no stub can see: every
`.js` the app loads arrives as JavaScript (a bundle built for the wrong base path arrives as
`text/html`, which boots nothing), and `/api/me` answers 200 with `type: "admin"` — true only
while grant types, the OpenID scope mappings and the group claim all hold together. `24-` (the
`standard` project) signs in as `qa-e2e`, an account in no Audiobookshelf group, and must come
back `type: "user"`: the default grant that lets the household in, so a scope-mapping
regression surfaces as a failed sign-in in the gate instead of as a friend's complaint.
The public door is pinned by the guard audit `tests/e2e/specs/14-guard.spec.ts`, which now
carries an **exemption** assertion where it used to carry a guarded one: an anonymous raw
client must get the app's own `{"success":true}` on `/ping`, a machine-format socket
handshake, and the app's own `401` on `/audiobookshelf/api/libraries` — never challenge
HTML. `tests/anubis.bats` pins the same shape in the Caddyfile: the vhost never proxies to
`anubis:8923`, and the post-auth listener carries no `@audiobooks` route.
Five book folders hold media no scanner can read, so they stay out of the library:
three 2024 Nebula nominees in the `_unsorted` box (two `.m4b` with no `moov` atom,
+236 -50
View File
@@ -3,7 +3,7 @@ type: reference
title: Jellyfin
description: "Jellyfin on Dankserver: quadlet unit, overlay metadata view, provider keys, ports, access, transcode status."
tags: [jellyfin, dankserver, media-server]
timestamp: 2026-10-30T00:00:00Z
timestamp: 2026-11-03T11:40:00Z
---
# Jellyfin
@@ -58,12 +58,23 @@ reads — movie art has never existed beside the media on either layer.
**Library coverage is the first thing to check when a title is missing.** The
four libraries are `movies → /data/Movies`, `tvshows → /data/TVShows`,
`music → /data/Music`, `boxsets → /config/data/collections`
(`GET /Library/VirtualFolders`). `/data/media-park` is in **none** of them, so
everything filed under it — the `_unsorted` release bundles and the `extras/`
folders the relayout wrote there — is invisible to Jellyfin by configuration,
(`GET /Library/VirtualFolders`). The review pile — `/data/NeedsProcessing`, the
former `media-park`, drained into it 2026-11-02 — is in **none** of them, so
everything filed there is invisible to Jellyfin by configuration,
not by fault: a recursive refresh answers 204, burns CPU for minutes and
indexes nothing, because it never looks there. Check coverage before re-scanning.
**Which read proves an item is indexed: the folder view, never the recursive type
list.** `GET /Items?IncludeItemTypes=Movie&Recursive=true` answered 2,101 items on
2026-11-02 while `GET /Items?ParentId=<Movies ItemId>` listed 2,548 children — every
folder on disk — and the eight folders the recursive read "lost" (three `Gundam 00
Special Edition`, five `Sailor Moon Memorial`) each answer as `Type: Movie` through
`SearchTerm` and sit in the folder listing. The two reads are not interchangeable, so
the recursive list is a wrong instrument for "did the refresh index what I moved": its
shortfall is grouping, not missing media, and acting on it starts a scan that fixes
nothing. Verify per item — `ParentId` listing or `SearchTerm` — and use the recursive
count only as a trend line.
**Failure signature:** `System.IO.IOException: Read-only file system :
'/data/Movies/<film>/movie.nfo'` means the container is binding the raw array
instead of the view — always a fault on this host since the 2026-10-31 cutover.
@@ -81,21 +92,27 @@ so an empty `<ImageFetchers />` switches art off while `ImageFetcherOrder` stays
fully populated — order is not enablement — and the populated `<MetadataFetchers>`
built next to it makes the library look healthy.
**Live posture, measured 2026-11-02** from `GET /Library/VirtualFolders`. The DTO is
PascalCase — `Name`, `CollectionType`, `Locations`, `ItemId`, `RefreshStatus` — and a
census written against the 10.x camelCase keys reads `None` for everything, which
looks exactly like an unconfigured library.
**Live posture, measured 2026-11-02** from `GET /Library/VirtualFolders`, cross-checked
against each library's `config/root/default/<Library>/options.xml`. The switches are NOT
top-level DTO keys: the folder carries `Name`, `CollectionType`, `Locations`, `ItemId`,
`PrimaryImageItemId`, `RefreshStatus` and `LibraryOptions`, and every per-library option
sits inside `LibraryOptions`. Reading `EnableInternetProviders` off the folder itself
answers `None` for all four — the camelCase trap one level deeper, and it looks exactly
like an unconfigured library.
| Library | `CollectionType` | `EnableInternetProviders` | `EnableRealtimeMonitor` |
|---|---|---|---|
| `Movies` | movies | true (fixed 2026-11-02) | **false** |
| `Shows` | tvshows | **false** | true |
| `Shows` | tvshows | true (fixed 2026-11-02) | true |
| `Music` | music | **false** | true |
| `Collections` | boxsets | **false** | false |
`Shows` and `Music` are silent file indexes in the sense above: a refresh answers
`Music` and `Collections` are silent file indexes in the sense above: a refresh answers
`204`, stores nothing, logs nothing. Any metadata or artwork work on either one
starts by flipping the switch, never by firing a scan.
starts by flipping the switch, never by firing a scan. `Shows` sat in that set
until 2026-11-02; the door is `POST /Library/VirtualFolders/LibraryOptions` with
the library's own `ItemId` (below), and `scripts/jellyfin-providers.sh verify` is
the one command that says which libraries are still in it.
**The per-library option door on 12.x is `POST /Library/VirtualFolders/LibraryOptions`**
— `GET` there answers `405`, which is how the route announces itself. The 10.x doors
@@ -119,9 +136,41 @@ store on a build where the automatic image pass will not run at all — it is ho
the Movies library got its art: 2,515 of 2,564 films carry a primary now, and the
rest have none in TMDb (shorts like `Astartes`, files whose names match nothing).
**API shape on 12.x:** `/Items` and `/Items/{id}` need `UserId` when
authenticated by API key — without it the answer is `400 Error processing
request`, which reads exactly like an authentication failure. `MetadataRefreshMode`
**The Download door's exact wire shape:** the list response carries the address in
`Url` — there is no `ImageUrl` field — and the POST takes camelCase query params,
`?type=Primary&imageUrl=<Url>`, answering `204`; the stored tag then appears in
`ImageTags`. PascalCase params, or an `imageUrl` left empty, answer `500`, which
reads as a server fault and is a field-name fault.
**`RemoteImages` answers empty for an item with no `ProviderIds`** — no provider id,
nothing to look up. Art coverage is capped by identification, not by the image door:
the items a campaign returns nothing for are the unidentified ones, and no image
retry will change that.
**A `Download` pass reports zero only when it is pinned to one provider.** A TMDb-pinned
campaign over `Shows` classified every item in 20 s, downloaded nothing, and concluded that
identification was the only lever left. The same door queried with no `Providers=` filter
answered for 16 seasons that same day — every offer from `TheTVDB`, which the pinned form
never asks — and all 16 are dressed now. A pinned answer proves one provider is empty; the
unpinned answer is the provider census, and a TV art loop that pins is simply the wrong tool.
Post-fill the artless remainder measures 86 items (80 seasons, 6 series): 61 carry no provider
id at all — the unnamed long tail (`3a BBW - Series (2010-11)`, `History Of Africa With Zeinab
Badawi`, `(Warhammer TV)`-shaped directories), where the lever is folder names or an identify
pass — and 25 hold an id and draw no offer from any provider. No image retry moves either
group, which is the half of the old conclusion that survives.
**Series and season art is fetched; episode stills are not, deliberately.** Every
episode query is a fresh provider round-trip — nothing caches per episode — measured
at ~11 s per item beside ~2 s per series on the same host, with one hit in five in a
sample. Across 42k episodes that is days of provider traffic to dress a fifth of them,
and clients display the series and season art anyway. If episode stills are ever
ordered, the loop is the same door and its log makes the pass resumable.
**API shape on 12.x:** `/Items/{id}` needs `UserId` when authenticated by API key — without
it the answer is `400 Error processing request`, which reads exactly like an authentication
failure, and it bites hardest as a *verification* read directly after a write that succeeded.
The `/Items` list route, `?Ids=<id>` included, needs no `UserId`, so verify writes there.
`MetadataRefreshMode`
accepts `Default` (and `Realtime`) only; `Full` and `ValidationFailure` are
rejected outright, and what forces the providers to re-run is
`ReplaceAllMetadata=true`.
@@ -139,7 +188,7 @@ Client access paths and the public-exposure policy (auth wall required before
any public vhost; Plex-style claim tokens do not exist here — Jellyfin uses its
own users/API keys) follow the fleet [[Authentik]]/[[Anubis]] chain pattern.
## Authentik LDAP authentication (2026-09-25)
## Authentik LDAP authentication (live since 2026-09-25, re-verified 2026-11-04)
Jellyfin has no native external-auth, so it authenticates users through the
**LDAP-Auth plugin** (`jellyfin-plugin-ldapauth`) against an [[Authentik]] LDAP
@@ -155,17 +204,35 @@ user to verify the password.
publishing `389`/`636` on the Dankclaw host for cross-host binds.
- Service account `svc-jellyfin-ldap` (DN
`cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud`).
- Access groups `jellyfin_users` (search filter
`(memberOf=cn=jellyfin_users,ou=groups,dc=franksdank,dc=cloud)`) and
`jellyfin_admins` (admin filter `(cn=jellyfin_admins)`). Add a user to
`jellyfin_users` to grant them Jellyfin access; `jellyfin_admins` grants
the Jellyfin admin role.
- Access group `jellyfin_users`: the plugin's search filter is
`(memberOf=cn=jellyfin_users,ou=groups,dc=franksdank,dc=cloud)`, so a user
who is not in that group is **invisible** to the plugin and the login fails
as "Wrong username or password". Add members with
`Group.objects.get(name="jellyfin_users").users.add(user)` in `ak shell`
(directory data — the DB owns it, not this repo).
`jellyfin_admins` is **inert as configured** — see the admin-filter law.
**Jellyfin side** (plugin config, set via admin → Plugins → LDAP-Auth):
- Server `192.168.0.143:389` (plain LDAP over the trusted LAN), Base DN
`dc=franksdank,dc=cloud`, bind user = the service-account DN, search
attributes `uid,cn,mail,displayName`, uid attr `uid`, username attr `cn`
`CreateUsersFromLdap=true`, all folders enabled.
- Admin base `ou=groups,dc=franksdank,dc=cloud`, admin filter
`(cn=jellyfin_admins)`, `EnableLdapAdminFilterMemberUid=false`.
**Law — the admin filter grants nothing unless it can return the user's own
DN.** With `EnableLdapAdminFilterMemberUid=false` the plugin searches
`LdapAdminBaseDn` with `LdapAdminFilter` and sets the admin permission only when
a returned entry's DN equals the bound user's DN
(`LDAP-Auth/LDAPAuthenticationProviderPlugin.cs`). A group-OU base returns the
*group* entry, whose DN never matches, so **no LDAP user can ever become a
Jellyfin admin** — measured 2026-11-04: the search answers
`cn=jellyfin_admins,ou=groups,…` for every login. Working shape if group-granted
admins are ever wanted: base `dc=franksdank,dc=cloud` +
`(memberOf=cn=jellyfin_admins,ou=groups,dc=franksdank,dc=cloud)`. Deliberate
today: the server's own admin account stays **local**, which is the break-glass
door when authentik or the outpost is down. A *malformed* admin filter is not
inert — the plugin throws for every login, so this knob is all-or-nothing.
**Gotcha — the bind credential is a PASSWORD, not a token.** The LDAP bind is
verified by authentik's default authentication flow (standard-password
@@ -175,10 +242,23 @@ password and the bind fails with `flow error password: Invalid`. The
service-account password is stored only in the Jellyfin plugin config (the
`LdapBindPassword` field) — it is never committed to this repo.
**Verification (2026-09-25):** `ldapsearch` bind + the `jellyfin_users`
search both succeed from Dankclaw; a real user (`alex`, in `jellyfin_users`)
logged into the web UI with their authentik password and the plugin created
the matching Jellyfin user.
**Verification (re-taken 2026-11-04, from Dankserver):** the container reaches
`192.168.0.143:389`; the service-account bind plus the `jellyfin_users` search
return every member; LDAP-Auth 23.0.0.0 loads on Jellyfin 12.4; the `alex` row
still carries
`AuthenticationProviderId=Jellyfin.Plugin.LDAP_Auth.LdapAuthenticationProviderPlugin`.
**Law — "the authentik login doesn't work" has two opposite causes; read the
Users table before touching authentik.** Copy `config/data/jellyfin.db` with its
`-wal`/`-shm` and read it read-only (never sqlite the live file):
`Users.AuthenticationProviderId` names the door — `…Users.DefaultAuthentication
Provider` = local password, `…LDAP_Auth.LdapAuthenticationProviderPlugin` =
authentik. Because the LDAP search is scoped to `jellyfin_users`, a name that
exists only as a local account is rejected by the plugin and Jellyfin then
silently falls back to the local provider: same UI message, opposite fix. The
instrument for "did a bind ever reach authentik" is the **event registry**, never
the outpost journal — see [[Authentik Outpost Reference]].
**Gotcha — the login endpoint changed in Jellyfin 12.1.** The legacy
`POST /Users/Auth/Password` route (and its `LegacyEmbyRouteRewriteMiddleware`)
was removed; probing it now returns a silent 401 with no plugin log, which
@@ -232,9 +312,17 @@ scripts/jellyfin-providers.sh verify # plugin + key LENGTHS, never key bytes
- **The homelab TVDB key is a licensed key** (its JWT claims say
`community_supported: false`), so it needs **no PIN** — the optional
`TVDB_PIN=` line exists only for a subscriber-shaped key.
- **TVDB is installed and keyed but is not a metadata agent**: TMDb still
identifies everything. Enabling TVDB as a provider would re-fetch show
metadata across the library, so it is a deliberate, separate change.
- **TVDB is installed but is not a metadata agent**: TMDb still identifies
everything. Enabling TVDB as a provider would re-fetch show metadata across
the library, so it is a deliberate, separate change.
- **A plugin owns its own config file, under its own root, and omits null
settings** — so the element an installer wants to fill may simply not exist,
and its root is `<PluginConfiguration>`, not `<Configuration>`. TVDB's file had
no `<ApiKey>` element at all, which is why `install` refused with
`no place to write <ApiKey>` from October until the anchor fix: the secret sat
valid in `provider-keys` the whole time, and `verify`'s `MISSING key` was this
fault, not a missing key. The writer now anchors on the file's own root and
still refuses by name when a file has none.
- **The deployer places the keys before it restarts jellyfin**
(`ensure_provider_keys` in `scripts/deploy-remote.sh`) and warns instead of
failing when `provider-keys` is absent, so a host without it still deploys.
@@ -252,30 +340,40 @@ scripts/jellyfin-providers.sh verify # plugin + key LENGTHS, never key bytes
## Triggering a scan
The scan key lives in `~/.config/secrets/jellyfin` (mode 600; the same value is
`JELLYFIN_API_KEY` in the ignored `dankserver/.env`). It rides a mode-600 curl
config, never a header argument, in the `MediaBrowser` form: 12.4 authenticates
only `Authorization: MediaBrowser Token="…"` and refuses a bare `X-Emby-Token`
for every key, valid or not. Before believing a 401, prove the key against the
server's own table — `select count(*) from ApiKeys where AccessToken = ?` on
`jellyfin.db` opened `mode=ro&immutable=1`. The backslashes below matter:
Run the driver from the deploy checkout: `scripts/media-refresh-jellyfin.py
trigger`, then `… wait`. It calls the library settled only on the rule below — the
task seen `Running` (or a run that finished while it watched) first, then
`--require-quiet` reads with no scan and no writes — and refuses by name when the
scan never started, because a watcher that saw no `Running` would otherwise report
the pre-scan library as the fresh one. `… status --after <the trigger's epoch>` is the
"has the scan reached my moves" answer, read from the task's `EndTimeUtc` — and that
timestamp only advances when a scan *finishes*, so the epoch must be one your trigger
could not have followed: a journal mtime is already passed by whatever scan ran before
you asked, and bare `status` cannot tell that walk from yours, while `wait` refuses such
an epoch by name. The Plex door is worse still — [[Plex]].
```bash
umask 077
{ printf 'header = "Authorization: MediaBrowser Token=\\"%s\\""\n' \
"$(cat ~/.config/secrets/jellyfin)"
echo 'url = "http://127.0.0.1:8096/Library/Refresh"'
echo 'request = "POST"'; } >/tmp/jf.hdr
curl -s -K /tmp/jf.hdr -o /dev/null -w "jellyfin scan: %{http_code}\n"
shred -u /tmp/jf.hdr
```
It carries every law below. Hand-written, the door is a `curl -K` on a mode-600
config: the scan key lives in `~/.config/secrets/jellyfin` (mode 600; the same
value is `JELLYFIN_API_KEY` in the ignored `dankserver/.env`) and rides as
`Authorization: MediaBrowser Token="…"` — 12.4 refuses a bare `X-Emby-Token` for
every key, valid or not — never as a header argument, because
`/proc/<pid>/cmdline` is world-readable. `recentMedia` counts
`BaseItems.DateModified` rows in `jellyfin.db`, which on this host sits in the
quadlet's config bind at `/home/dank/Containers/jellyfin/config/data/`, not the
stock `~/.local/share/jellyfin`.
`204` is acceptance; anything else queued no scan, so re-issue the identical
request instead of diagnosing. A `401` from a key that answers `200` on a
hand-typed request is this block's quoting, not the server: bash `printf`
collapses `\"` to `"`, curl ends the quoted value at that quote, and Jellyfin
receives `Authorization: MediaBrowser Token=` with nothing after it (`\\"` answers
204, `\"` answers 401 — both run on this host).
204, `\"` answers 401 — both run on this host). Before believing a 401, prove the
key against the server's own table — `select count(*) from ApiKeys where
AccessToken = ?` on `jellyfin.db` opened plain `mode=ro` (`immutable=1` only as
the fallback for a database whose `-shm` cannot be read; it skips the write-ahead
log, see [[Media Relayout]]). Reading progress with the same config trick takes
**one** URL: a config `url` line *plus* a command-line URL makes curl perform both
transfers, and the second dumps the whole `ScheduledTasks` body onto your stdout.
One accepted refresh is the whole procedure. A second one while a scan runs moves
`Scan Media Library` to `Cancelling`, and once the runner holds `Cancelling` every
@@ -284,12 +382,41 @@ anything. The fix is `systemctl --user restart jellyfin.service`, one POST after
health answers, and then believe `ScheduledTasks` State at 130 % CPU, not the
response code.
**A scan that must see a new folder has to be recursive.** `POST /Library/Refresh`
is the incremental pass, and inotify does not cross a rootless podman bind, so
folders created on the host stay invisible to it. The move is
`POST /Items/{id}/Refresh?Recursive=true` (204) per library, the id from
`GET /Library/VirtualFolders` — its field is `ItemId`, not `Id` — and that same
record's `RefreshStatus` is the progress read.
**The door that scans is `POST /Library/Refresh` (204), and `ScheduledTasks` is
the progress read.** A task that has never completed answers
`LastTaskResult=None, EndTimeUtc=None`, and that pair is what separates "never
ran" from "finished a moment ago" — the difference between a door that is dead
and one that merely answered `204`.
**The settle read is `GET /ScheduledTasks`: the row whose `Name` is exactly
`Scan Media Library`, and its `State` (`Idle`|`Running`).** `/Library/ScanStatus`
is a 404 on 12.4, so a watcher built on the old endpoint reports nothing and a
scanning server looks an idle one. Call it settled only after a `Running`
sighting followed by two consecutive `Idle`s: sub-tasks leave gaps that a single
poll reads as finished, and a census taken in one reports the pre-scan library.
**The per-library `POST /Items/{id}/Refresh` queues nothing, and the provider
switch is not the reason.** Measured twice on 2026-11-02 against `Shows`: before
the flip and again after it, the call answered `204`, the journal wrote zero lines
over the following ten minutes, and `Scan Media Library` kept its never-run
signature; a service restart changes nothing. `POST /Library/Refresh` does run —
`204`, then `Running` inside 15 s at 33 % CPU — but it walks every library, so on
this build a library-scoped metadata pass has no working door and the global one
is the only lever.
**That endpoint takes its parameters from one source.** The documented query form
(empty body) and the JSON body form (no query) each parse; supplying both at once
answers `400` with an empty body whatever the values, and `ValidationFailure` is
rejected in the body while the query form accepts it.
**`RefreshStatus` on the `GET /Library/VirtualFolders` record is not a progress
read.** It stayed `Idle` through a scan that was verifiably running at 75 %. The
record's useful field is `ItemId`, and the live library set is still the entries
whose `Locations` are non-empty.
`inotify` does not cross a rootless podman bind, so folders created on the host
stay invisible until a pass like this runs; every pass that writes into a library
ends by firing one.
**`Movies` has no watcher at all.** `EnableRealtimeMonitor` is per library (table in
*Data layout*) and it is false for `Movies` and `Collections`. Nothing notices a
@@ -297,6 +424,65 @@ folder created under `/data/Movies`: it stays invisible until a recursive refres
names it. So every pass that writes into a library ends by firing one, and "Jellyfin
does not see the file I just moved in" is the designed answer, not a fault to chase.
## API-key ledger and rotation
`/Auth/Keys` is the ledger. `GET` lists every key **with its `AccessToken` in
cleartext**, so an admin token in hand prints the whole fleet's keys;
`POST /Auth/Keys?app=<name>` mints one and answers `204` **with no body** — the
new token is read back from the `GET`, never from the create response.
`DELETE /Auth/Keys/{token}` revokes **by the token string**, and a parameter that
matches nothing answers `204` too, so `204` there is not proof anything was
removed. Discover a route's verb from the `Allow` header of an `OPTIONS` (or
`GET`) response — never by firing `DELETE` at a guess.
- `IsActive` reads `False` on rows that authenticate `200`. It is not a usability
signal; only an authenticated request is.
- `DateLastActivity` never moves off `0001-01-01` on 12.4, so "is this key in
use?" cannot be answered from the ledger. The census is repo + host credential
stores + the `Devices` table (whose `DateLastActivity` *is* real).
- Rotation order: mint → rewrite **both** stores (`~/.config/secrets/jellyfin`
and `JELLYFIN_API_KEY` in the ignored `dankserver/.env`) → prove the new key on
`/System/Info` → `DELETE` the old token → prove `401`. Revoking while a store
still holds that value breaks the scan door silently.
- Blast radius as measured 2026-11-03: no repo script reads `JELLYFIN_API_KEY`;
Seerr keeps a key of its own in its own DB (which of its two rows it uses was
never confirmed), so revoke by token and never by app name.
## Big-screen theme (the TV)
`dankserver/jellyfin/theme/wmc/theme.css` is repo-owned and bind-mounted over the
image's **Media Center** theme slot (the `Volume=` line in
`dankserver/jellyfin/quadlet/jellyfin.container`). It exists for
`danklivingroom`, which renders this server-served web app at 3840×2160 with no
client-side scaling, so 14 px text arrives on a panel read from three metres.
The file is four rules, and its first line is the design:
`@import url("../dark/theme.css")` pulls in the **live upstream** dark palette, so
the repo never vendors a CSS snapshot and a Jellyfin image bump cannot strand one.
That works because jellyfin-web applies a theme as
`<link rel="stylesheet" href="themes/<id>/theme.css">` — a relative URL resolved
against the CSS file, not the document. The size lives in one number:
`--dank-big-screen-zoom` (1.75 ≈ three metres; 1.5 mild, 2.0 projector). `zoom` on
the root element is the right lever: the card-grid code measures layout pixels, so
fewer, larger columns are computed rather than overflow produced.
- **Adoption is per device, and it stays that way.** The theme a client renders
comes from its own settings (localStorage first), so there is no server-side
"everyone gets the big UI" knob. On the screen that wants it:
**Display Settings → Theme → Media Center.** That is also the honest label —
the dropdown names are compiled into the bundle, so a "Big Screen" entry will
never appear; do not go looking for one.
- The mount is `:ro` and the app only fetches the file, so the failure mode of a
stale `@import` target is slightly wrong colors, never a dead web app.
- **Editing the CSS needs no restart.** The bind hands out the host file's live
bytes, so a changed zoom number is on screen at the next page load. The planner
agrees by construction: only `dankserver/<svc>/quadlet/**` maps to a service, so
a `theme/**`-only commit plans nothing — that is correct, not a gap to "fix",
because restarting a media server for a font size is the wrong trade. Only the
first deploy of the `Volume=` line restarts anything.
- **Rollback is one line:** drop the `Volume=` and redeploy; nothing in the
database records the theme, so no cleanup follows.
## See Also
- [[Plex]] — co-resident media server on the same volume
@@ -41,7 +41,7 @@ python3 scripts/media-audiobooks.py --root /mnt/entropy/Audiobooks \
# library too ([[Audiobookshelf]]) — otherwise it keeps rows for folders that
# no longer exist.
scripts/media-apply.py apply --plan /tmp/ab_plan.tsv --root /mnt/entropy \
--journal /mnt/entropy/media-relayout-audiobooks.jsonl --apply
--journal /mnt/entropy/.media-campaign/media-relayout-audiobooks.jsonl --apply
```
The group is the identity Plex asserts in its own database — **(narrator, book,
@@ -68,7 +68,7 @@ no chapter row names, and a loose file in a folder holding two books all park un
that unrelated books share, or a chapter number mistagged as the artist (`0.4`).
`Various Artists` is not a placeholder — that is a multi-cast read.
The reviewed plan lives at `/mnt/entropy/media-relayout-audiobooks-plan.tsv`; it
The reviewed plan lives at `/mnt/entropy/.media-campaign/media-relayout-audiobooks-plan.tsv`; it
applies on its own journal, behind the music writer, and then refreshes the Plex
section. Verify it with `--db-kind plex` — Jellyfin holds no rows here, so a
Jellyfin pass proves nothing either way.
@@ -65,8 +65,8 @@ executor's own format; it never moves anything itself.
python3 scripts/media-tvdupes.py --library /mnt/entropy/TVShows \
--hold /mnt/entropy/.AGENTTRASH/<date>-tv-duplicates \
--meta /var/lib/media-meta/view/TVShows --witness witness.tsv \
--ids /mnt/entropy/tv-tvdb-ids.tsv \
--out /mnt/entropy/tv-dupes-plan.tsv --json /tmp/tv-groups.json
--ids /mnt/entropy/.media-campaign/tv-tvdb-ids.tsv \
--out /mnt/entropy/.media-campaign/tv-dupes-plan.tsv --json /tmp/tv-groups.json
# then media-apply.py apply / verify / sweep / rollback, exactly as for a film prune
```
@@ -76,7 +76,7 @@ python3 scripts/media-tvdupes.py --library /mnt/entropy/TVShows \
match, not a lookup: `BaseItems` (`Type like '%TV.Series'`) joined to
`BaseItemProviders` where `ProviderId = 'Tvdb'`, the `/data/TVShows/` prefix
stripped to the bare top-level folder name — 616 folders, none carrying two
ids, written to `/mnt/entropy/tv-tvdb-ids.tsv`. Two folders with disjoint ids
ids, written to `/mnt/entropy/.media-campaign/tv-tvdb-ids.tsv`. Two folders with disjoint ids
are two shows however alike they read — *Avatar: The Last Airbender* (1999)
beside the 2024 remake, *Millennium* (1996) beside a German thriller Jellyfin
had also titled *Millennium* — and a pair whose identity cannot be proven is
@@ -151,6 +151,18 @@ Each of these is a file that would have been destroyed without it:
- Media containers only. Grouping every extension buries the report in subtitle
and cover pairs — and a duration guard that appears to fire on 720 groups is
counting subtitles, not protecting films.
- **A short file named after a film is promotion, not a movie.** Thirty-two files
in `Movies` — 31 clips and the `.nfo` filed beside one of them — carried real
titles (`The Descent (2005) - SD.mkv`) and were 8–20-second h264 video at
640×360, 1024×576 or 1280×720, 49 MiB all together —
25 of them in a `_release-group watermarks/` folder and the rest at the library
root or inside one film's folder. No duplicate screen ever catches them: nothing
shares their size, and the feature-length guard above makes them invisible to the
similar-title pass too. They were held as `.AGENTTRASH/20261104-promo-clips`
(journal `/mnt/entropy/.media-campaign/promo-hold.jsonl`, which survives) and purged
the same evening, once both databases read `REAL: 0` against that journal.
The census that finds the next batch is a duration floor, not a hash: a video under
a minute on a name that reads like a feature is advertising.
## The hold
@@ -158,12 +170,12 @@ Nothing is deleted. Every doomed copy moves into a dated hold with the same
executor the relayout uses, so any file comes back:
```bash
scripts/media-apply.py apply --plan /mnt/entropy/dup-prune-similar-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/dup-prune-similar.jsonl # dry
scripts/media-apply.py apply --plan /mnt/entropy/.media-campaign/dup-prune-similar-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/.media-campaign/dup-prune-similar.jsonl # dry
scripts/media-apply.py apply --plan … --root /mnt/entropy --journal … --apply
scripts/media-apply.py verify --plan … --journal /mnt/entropy/dup-prune-similar.jsonl
scripts/media-apply.py sweep --journal /mnt/entropy/dup-prune-similar.jsonl --root /mnt/entropy --apply
scripts/media-apply.py rollback --journal /mnt/entropy/dup-prune-similar.jsonl --root /mnt/entropy --apply
scripts/media-apply.py verify --plan … --journal /mnt/entropy/.media-campaign/dup-prune-similar.jsonl
scripts/media-apply.py sweep --journal /mnt/entropy/.media-campaign/dup-prune-similar.jsonl --root /mnt/entropy --apply
scripts/media-apply.py rollback --journal /mnt/entropy/.media-campaign/dup-prune-similar.jsonl --root /mnt/entropy --apply
```
The plan rows are the tool's own format (`action src dst kind`, `mkdir` rows for
@@ -179,21 +191,57 @@ keeper is missing — either one drops the whole group rather than half-applying
era, not this pass — read the mtimes before blaming the prune.
- The 2026-11-01 pass held 226 files (~1.1 TiB) in
`/mnt/entropy/.AGENTTRASH/20261101-duplicate-prune`, journals
`dup-prune-exact.jsonl` and `dup-prune-similar.jsonl`. Purge pending Frank's word.
- The 2026-11-02 TV pass holds 220 files (54 GiB) in
`dup-prune-exact.jsonl` and `dup-prune-similar.jsonl`.
- The 2026-11-02 TV pass held 220 files (54 GiB) in
`/mnt/entropy/.AGENTTRASH/20261102-tv-duplicates` — plan `tv-dupes-plan.tsv`,
journal `tv-dupes.jsonl`, 220 moves / 16 hold folders / 7 husks swept — and the
same day's TVShows detritus pass holds 109 files (552 KiB) in
same day's TVShows detritus pass staged 109 files (552 KiB) in
`/mnt/entropy/.AGENTTRASH/20261102-tvshows-detritus` with plan
`tvshows-detritus-plan.tsv` and journal `tvshows-detritus.jsonl`. Both purges
pending Frank's word, beside the 2026-11-01 hold above. All four journals and
plans live at the `entropy` root, not `/tmp`.
- **Re-verify each survivor by size, not by the recorded path, before purging.**
Detection artifacts and plans key on paths, and a folder renamed by an
interactive session ([[Entropy Volume]], *Write doors*) makes its keeper look
deleted while it sits untouched under the new name. The 2026-11-01 audit hit
three such groups: every keeper was present, same bytes, different folder. A
purge decided from the plan's paths alone would have reported survivors gone.
`tvshows-detritus-plan.tsv` and journal `tvshows-detritus.jsonl`. All four
journals and plans live at the `entropy` root, not `/tmp`.
- **Purged 2026-11-04, on Frank's word, after the gate below:** the duplicate
prune, the TV duplicate pass, the Ghibli duplicates (6 files — both `Nausicaä`
copies among them) and the 2026-11-04 promo-clip hold left the array trash,
~1.2 TB freed (the array went 218 GB → 1.4 TB available). The journals survive as
the only record of where a purged file came from; rollback after a purge is
impossible, which is why the gate runs first.
### The purge gate: a carrier for every held file
`REAL: 0` in both databases says only that no row points at a moved path; it says
nothing about whether the film survived. So a purge waits on a stronger proof:
**every held file has a same-work carrier in the live library at >= its size,
re-derived at purge time**, never copied from the plan. The 2026-11-04 purge re-found
all 452 held files this way; the first pass of the gate could not place 142 of them,
and every one of those was a matcher gap, not a lost film. What the re-derivation has
to absorb:
- **The layout moves under the plan.** The relayout renames folders to
`Title (YYYY)` and `[tvdbid-N]/Season N` and renames the keeper files too, so the
paths *and* the sizes a plan recorded no longer describe the library. Keys are work
identity — normalized title+year for a film, `(S,E)` for a TV pack — never a path.
- **Fold accents, then compare.** The library holds `Nausicaä`; a plan, a
`find -ipath "*Nausicaa*"`, and an ASCII substring search all miss it. NFKD-fold and
drop combining marks, or a real keeper reads absent for two rounds and the file
beside it starts to look like an orphan.
- **The carrier is often another container, and usually larger.** A held `.mp4` can be
kept as the sibling `.mkv`; keep-larger means the keeper is frequently far bigger (a
377 MB held episode matched a 501 MB keeper; a 1.6 GiB film stands beside a 34 GiB
BDRemux of itself in another folder). "Same folder, same bytes" reports survivors as
lost — it is the wrong test.
- **An exact twin can sit in a sibling folder**, so "no same-size file in the folder"
needs an array-wide name search before it means anything.
### Running the purge
`scripts/purge-agenttrash.sh` is the only door — never `rm`. It acts on exactly one
object: a root's own `.AGENTTRASH`. A partial purge is therefore **staged, not
filtered**: move the approved buckets into a scratch root
(`.PURGE-<date>/.AGENTTRASH/`) and point the purger at that root, and park anything
not approved at the array root as `.HOLD-<date>-*`, out of its reach. Dry-run first —
the plan must print exactly one `plan empty` row, and it is the same row `--execute`
acts on — and send `--log` into `.media-campaign/`, because `/tmp` is not a record.
Then read `df`: a purge is the only media act that moves the free-space number.
## After the pass
@@ -33,7 +33,7 @@ python3 scripts/media-music.py --root /mnt/entropy/Music --enum /tmp/music_enum.
# 4. Read the exceptions file FIRST — it is the review, not a log. Then the same
# executor, journal and rollback as [[Media Relayout]] shows, on its own journal.
scripts/media-apply.py apply --plan /tmp/music_plan.tsv --root /mnt/entropy \
--journal /mnt/entropy/media-relayout-music.jsonl --apply
--journal /mnt/entropy/.media-campaign/media-relayout-music.jsonl --apply
```
The group is what the tags assert, not where the files sit: **(album-artist tag,
@@ -3,7 +3,7 @@ type: reference
title: Media Relayout
description: "Census, plan and apply the movie, TV, music and audiobook libraries: the gates, the refusals, the rollback."
tags: [dankserver, media-server, jellyfin, plex, procedure]
timestamp: 2026-11-01T01:00:00Z
timestamp: 2026-11-02T11:00:00Z
---
# Media Relayout
@@ -71,12 +71,23 @@ that would be filed wrong, and the plan is not run until it is understood.
```bash
scripts/media-apply.py apply --plan /tmp/jf-plan.tsv --root /mnt/entropy/Movies # dry, writes nothing
scripts/media-apply.py apply --plan /tmp/jf-plan.tsv --root /mnt/entropy/Movies --apply
scripts/media-apply.py verify --plan /tmp/jf-plan.tsv --journal /mnt/entropy/media-relayout.jsonl
scripts/media-apply.py sweep --journal /mnt/entropy/media-relayout.jsonl --root /mnt/entropy/Movies # dry
scripts/media-apply.py sweep --journal /mnt/entropy/media-relayout.jsonl --root /mnt/entropy/Movies --apply
scripts/media-apply.py rollback --journal /mnt/entropy/media-relayout.jsonl --root /mnt/entropy/Movies --apply
scripts/media-apply.py verify --plan /tmp/jf-plan.tsv --journal /mnt/entropy/.media-campaign/media-relayout.jsonl
scripts/media-apply.py sweep --journal /mnt/entropy/.media-campaign/media-relayout.jsonl --root /mnt/entropy/Movies # dry
scripts/media-apply.py sweep --journal /mnt/entropy/.media-campaign/media-relayout.jsonl --root /mnt/entropy/Movies --apply
scripts/media-apply.py rollback --journal /mnt/entropy/.media-campaign/media-relayout.jsonl --root /mnt/entropy/Movies --apply
```
**Every campaign artifact — plan, journal, exceptions file, run log — lives in
`/mnt/entropy/.media-campaign/`.** The array root carries libraries and nothing
else: a root strewn with `*.jsonl` and `*-plan.tsv` gets browsed as content, and
the dot-directory is ignored by both media servers. Relocated 2026-11-02, 48 files,
undo with `media-apply.py rollback --journal /mnt/entropy/.media-campaign/media-campaign-relocate.jsonl --root /mnt/entropy --apply`.
The holds and trash (`.AGENTTRASH`, `.HOLD-<date>-*`) stay at the root on purpose:
the purge machinery is given those paths as `--hold`, and a hold that moves out
from under the tool that made it is a hold nobody can find. Name `--journal`
explicitly on every run — the tool's default is `media-relayout.jsonl` beside
`--root`, so a bare run recreates exactly the root-level log this page forbids.
An apply also leaves the release folders it emptied standing where they were —
hundreds of them at the library root — which is what `sweep` removes, dry-run
first, from the same journal.
@@ -88,12 +99,17 @@ interrupted apply still replays the batch and resumes with nothing lost. Batch 1
is the old behaviour; the only reason to name it is debugging, because on the
spinning array it spends most of a large apply's wall time in fsyncs.
After it lands, both servers need a scan or the change is invisible. [[Plex]] and
[[Jellyfin]] own the exact requests; two laws hold at this step: `204` is
acceptance and anything else queued nothing — re-issue the identical request
rather than diagnosing — and ONE accepted refresh is the whole procedure, because
a second while a scan runs moves `Scan Media Library` to `Cancelling` and every
later `204` is accepted and then discarded.
After it lands, both servers need a scan or the change is invisible. The two
drivers are the doors — `scripts/media-refresh-jellyfin.py trigger` then `… wait`,
and only after it settles, `scripts/media-refresh-plex.sh trigger --library <key>`
with `… status --library <key> --after $(stat -c %Y <journal>)` as the poll. Run
them in that order and never together: Jellyfin's only door walks every library,
every Plex section forks its own scanner walk, and the array is the bottleneck
both servers share. [[Plex]] and [[Jellyfin]] own the laws behind the drivers; two
of them decide this step: `204` is acceptance and anything else queued nothing —
re-issue the identical request rather than diagnosing — and ONE accepted refresh is
the whole procedure, because a second while a scan runs moves `Scan Media Library`
to `Cancelling` and every later `204` is accepted and then discarded.
### Proving the move landed
@@ -110,7 +126,7 @@ a read beside a running scan can fail as "database disk image is malformed".
`immutable` stays only the fallback for a database whose `-shm` cannot be read.
```bash
scripts/media-verify.py --journal /mnt/entropy/media-relayout-music.jsonl \
scripts/media-verify.py --journal /mnt/entropy/.media-campaign/media-relayout-music.jsonl \
--db-kind jellyfin --ignore-ext jpg,jpeg,png,log,cue,toc,nfo,txt,m3u,accurip
```
@@ -150,11 +166,11 @@ by series, and writes a plan in the executor's own format:
```bash
scripts/media-tv.py --root /mnt/entropy/Movies --library /mnt/entropy/TVShows \
--out /mnt/entropy/media-tv-plan.tsv
scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv --root /mnt/entropy
scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv --root /mnt/entropy \
--journal /mnt/entropy/media-relayout-tv.jsonl --apply
scripts/media-apply.py sweep --journal /mnt/entropy/media-relayout-tv.jsonl \
--out /mnt/entropy/.media-campaign/media-tv-plan.tsv
scripts/media-apply.py apply --plan /mnt/entropy/.media-campaign/media-tv-plan.tsv --root /mnt/entropy
scripts/media-apply.py apply --plan /mnt/entropy/.media-campaign/media-tv-plan.tsv --root /mnt/entropy \
--journal /mnt/entropy/.media-campaign/media-relayout-tv.jsonl --apply
scripts/media-apply.py sweep --journal /mnt/entropy/.media-campaign/media-relayout-tv.jsonl \
--root /mnt/entropy --apply
```
@@ -186,162 +202,10 @@ show. A name whose marker is `Part 2` is a film and stays put.
## The TV library's own relayout
`/mnt/entropy/TVShows` is scene packs rather than a library, and Jellyfin has
already matched and numbered most of it. `scripts/media-shows.py` re-files that
library into the shape Jellyfin reads, with **tvdb as the name authority**:
<Series> (<year>) [tvdbid-NNNN]/Season NN/
<Series> (<year>) - SxxEyy - <Episode Title>.ext
It never guesses: every series name, episode number and title it writes traces to
an authority, and what has none is parked for a human to read.
```bash
# 1. The authority: Jellyfin's own matched rows — each carries the series' TMDb
# and Tvdb ids and the title the row holds (the DB opens read-only).
python3 scripts/media-census.py jellyfin --root /mnt/entropy/TVShows \
--out /tmp/tv_titles.tsv
# 2. What is on disk (stat only), and the runtimes the film arm needs.
python3 scripts/media-census.py enumerate --root /mnt/entropy/TVShows \
--out /tmp/tv_enum.tsv
python3 scripts/media-census.py probe --root /mnt/entropy/TVShows \
--enum /tmp/tv_enum.tsv --jobs 6 --out /tmp/tv_probed.tsv
# The probe is the slow step: ffprobe reads both ends of every file. So re-use
# the last one — a duration belongs to the file, not to its name: re-key the
# previous census through that pass's own journal (`src` -> `dst`) instead of
# seeking the array again. It is also the only column the film arm reads, and
# only for a file with no marker and no provider row (4,794 of 58,551 today,
# all of them covered by a re-key that costs seconds, not an hour).
# 3. The plan. --tmdb needs TMDB_API_KEY — the v3 key in ~/.config/secrets/tmdb,
# which travels as the `api_key` query parameter. Only a v4 request token is
# accepted as a Bearer header, and the wrong shape answers 401 on EVERY call,
# which the summary line flags as NOT provider-complete. --tvdb reads its
# `api_key=` line from ~/.config/secrets/tvdb, never argv. Both write the SAME
# cache file, so the plan is reproducible offline afterwards: run it once to
# fill the cache, and quote the second run.
# Paths are only strings to a dry run, so this step can run on the workstation
# against scp'd TSVs — which is where the two provider secrets live.
python3 scripts/media-shows.py --root /mnt/entropy/TVShows --enum /tmp/tv_enum.tsv \
--titles /tmp/tv_titles.tsv --probed /tmp/tv_probed.tsv \
--tmdb-cache /mnt/entropy/media-tv-names.json --tmdb --tvdb \
--park /mnt/entropy/media-park/TVShows \
--out /mnt/entropy/media-relayout-shows2-plan.tsv \
--exceptions /mnt/entropy/media-relayout-shows2-exceptions.txt
# 4. Read the exceptions file FIRST — it is the review, not a log. Same executor
# as above, but on this arm's OWN journal, NAMED PER PASS: the journal of an
# applied pass is its unwind record, so a second pass that reuses the name
# destroys the only map back.
scripts/media-apply.py apply --plan /mnt/entropy/media-relayout-shows2-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/media-relayout-shows2.jsonl --apply
```
Evidence, strongest first — the order the planner believes things:
- the **episode marker the file's own name states** (`S01E02-E03`, `1x04`); a digit
run longer than three at that position is NOT a number — `S03E2019` and
`S04E47370355` are one box set's airdate fragment and pack index — so such a file
is renumbered by tvdb's title or parks, and is never truncated down to fit;
- the **`Season NN` folder it sits in and the number its name leads with inside
it**: `Season 19/01 Charles Barkley, Nirvana.avi` is S19E01;
- the **provider row**: which episode the file IS, and the numbering of a file
whose own name says nothing;
- the **row's tvdb id, else its TMDb id carried through TMDb's external ids, else
an exact tvdb name search**: that id names the SERIES FOLDER and supplies the
episode title, because a name the server read off a folder is not a show name.
The folder CARRIES the id: `[tvdbid-NNNN]` is what Jellyfin's Tvdb agent and
Plex's scanner both read, so a show whose title is ambiguous resolves by number
instead of by a human guessing which folder is which. The id rides the folder
only — a file name gets 180 characters and the episode title needs the room.
- a **bonus folder** anywhere above the file, or Jellyfin's `ExtraType` → the
series' canonical extra folder;
- a runtime over `SHORT_MAX` makes it film-shaped, and it leaves for `--movies` only
when the movie provider's title is a run of words the FILE itself states and its
year agrees with the file's: the query is only what `title_of` salvaged from a
release name, and the first hit for a short one is a near miss — a 1963 Bond film
for the word "russia", *Special Edition I* for a file saying *Edition III*. Else it
parks.
Four refusals carry the design: what the planner will NOT do is the point of it.
- **Folders resolve a series; they never invent one.** A folder's words may
resolve a file to a series a provider has already named — `(CBB) Tengen Toppa
Gurren Lagann - The Complete Series (BD 1080p)` names the registered *Tengen
Toppa Gurren Lagann* — but a bonus subfolder is not a show. An early run made
`…/03. Extra/01. NCOP/NCOP 01.mkv` into a root series called `01 NCOP`.
- **A scraped row does not outrank the filesystem.** Jellyfin numbers
multi-season packs badly: every file under `SNL 16-20/Season 16..19` carries
season 16 episode 20, and one Gundam row says season 2004. The folder, then
the file's own leading number, win.
- **A season that is a YEAR is the date matcher, not a season.** Jellyfin files
specials by airdate, so a row can arrive saying `season 2002`; that is not a
season and it never becomes a folder. The one authority on such a row is tvdb's
episode list read backwards: the title the row carries names the (season,
episode) tvdb gives it. A title tvdb gives to two episodes names nothing, and
the file parks with the reason it always carried. `Season 00` IS a legal
season — specials live there — so a row that really says 0 files there instead
of parking.
- **A season tvdb's own list contradicts gets no tvdb title.** An anime pack
numbered by its release group books `S05E01` in a season tvdb does not hold, or
books more episodes there than tvdb lists: the slot its number points at is a
different episode, and dressing it in that episode's title is a confident lie.
The file still renames to the standard shape; the title does not ride, and the
series is listed under *seasons whose numbering tvdb contradicts* in the
exceptions. A season holding FEWER files than tvdb lists stays plausible — a
partial pack is the normal shape. Neither reading holds for a list tvdb
cut off at its own page limit: the default episode query answers at most
500 rows and states no total, and One Piece, Crayon Shin-chan and
Detective Conan each answer exactly 500 while holding over a thousand —
a page bounds nothing above itself, so it contradicts nothing.
**A provider name is evidence only when an id backs it.** Jellyfin names a Series
after the folder it scanned, so a library filed by release folders ends up with
one Series item per box set — `24 (2001) Season 1`, `Modern Family S05` — and
copying those names verbatim launders folder junk into the contract. The planner
resolves each series to its tvdb id and writes tvdb's name and year; the merged
identity is also what turns a box set split across five release folders into ONE
folder, and what makes two files claiming one episode MEET in the duplicate rule.
A series no id resolves keeps the provider's words and is listed under *no tvdb
identity* in the exceptions — that list, not a rename, is the review.
**The review pile lives outside the library.** Files parked inside
`/mnt/entropy/TVShows` were indexed as one show named `_unsorted`, and that
show's rows then looked like evidence. So a row whose series is the hidden
top-level folder the file itself sits in is dropped — and the folder is matched
with its leading underscore stripped, because the server *displays* `_unsorted` as
`unsorted`: compare the spelled names and the pile survives the relayout as a
visible series folder that real files get renamed into. `--park` outside the root
makes the next plan re-file the pile instead of skipping it. Inside a hidden
top-level folder a file's own words may MATCH a registered show, never invent one.
**The operator's id map is the door for a name tvdb cannot be searched by.**
`scripts/media-shows.py` reads `scripts/media-shows-tvdb-ids.tsv` by default: one
line per pack folder, `folder <TAB> tvdb id <TAB> note`. The file exists because
tvdb titles a record in the show's own language and the English name rides in
*aliases* the name search never reads: `ブラッククローバー` (331753) answers to no
spelling of "Black Clover", and the bare alias `One Piece` belongs to 392276, the
2023 live action, not to the 1999 record (81797) the pack here is. The map
outranks a provider id — a box set's provider id is the server's own guess at a
folder name — and it merges the release folders that are one show. Everything it
touched is written to the exceptions: every folder it named, every row that
matched no folder, every disagreement with a provider id, every id tvdb never
answered. A line whose id is not a number stops the plan, because a typo renames
a whole pack to the wrong show. Only folders the search CANNOT reach belong in
it; a row for a folder the search would resolve hides a live lookup behind a
number nobody re-checks.
`scripts/media-shows-tvdb.py` makes the sheet the map is built from. It reads a
plan's `mkdir` rows, asks tvdb about every destination folder carrying no
`[tvdbid-`, and writes one row per folder with the verdict that says how much was
guessed to get there: `EXACT` (tvdb's own spelling), `PUNCT` (equal only after
release junk is stripped from the folder's words), `NEAR` (tvdb answered
something else — the best hit is printed for a human and nothing applies it),
`NONE`. Acceptance never moved across the passes that built this: a hit counts
only when tvdb's name equals the query with every non-alphanumeric dropped from
both sides, and the record's year still has to fall inside any year span the
folder names. That gate is what caught three confident picks — Fairy Tail's 2024
sequel (410031), the 2026 Hokuto no Ken remake (465216) and `Mugyutto! Black
Clover` (378762) offered for Black Clover. It is why the map is approved by hand
and the sheet is never fed to the planner.
`/mnt/entropy/TVShows` is its own arm: a different authority (tvdb), its own
refusals, and its own test for whether a folder really holds two shows —
[[Media TV Relayout]]. It shares this page's executor, the one-journal-per-arm
law, the sweep and the rollback.
## The music and audiobook arms
@@ -403,9 +267,12 @@ executor, the one-journal-per-arm law, the sweep and the rollback.
- **The journal lives BESIDE the library root, never inside it** (default:
`media-relayout.jsonl` next to the root): a log inside the library is a file
Jellyfin indexes.
- **Refusals are all-or-nothing.** An occupied target, a vanished source or a
duplicate destination stops the run *before the first rename*, so the operator
fixes one thing and re-runs instead of untangling a half-applied plan.
- **Refusals are all-or-nothing.** An occupied target, a vanished source, a
duplicate destination, or a plan row whose verb is not `mkdir`, `keep` or
`move`, stops the run *before the first rename*, so the operator fixes one
thing and re-runs instead of untangling a half-applied plan. An unread verb is
refused by line number: silently dropping one turned a pass meant to rename
two series folders into `applied: 0 moved` with exit 0 (2026-11-02).
`--force` relaxes only the opinion guards; it never permits overwriting.
- **A sweep deletes only what the journal proves the run emptied.** A folder is
eligible when a journalled file moved out of it or out of something under it,
@@ -432,6 +299,14 @@ executor, the one-journal-per-arm law, the sweep and the rollback.
`--root` set to the share that holds BOTH libraries: the executor refuses any
path outside `--root`, and a journal shared with the movie relayout would
undo both runs when only one of them is wanted.
- **A hold folder outside `--root` forces `--force` on BOTH `apply` and
`sweep`.** `sweep` refuses the entire journal when any journalled path sits
outside the root (`this journal belongs to another library`) — the same guard
the plan hits, and it exists to stop one pass unwinding another's files.
`--force` relaxes it and stays safe there: `sweep_targets` marks only
ancestors under `--root`, so a hold folder beyond the root can never be
selected; it only blocks everything that is. The cleaner shape is the law
above — put `--root` on the share that holds both.
- **Sidecar metadata does not follow a rename, because it lives on another
filesystem.** The media servers write `.nfo` and artwork through the
@@ -440,6 +315,9 @@ executor, the one-journal-per-arm law, the sweep and the rollback.
`scripts/media-view.sh reap --apply` — stranded sidecars become ghost folders
that both servers browse and report as content. Whether this host mounts the
view at all is stated on that page; `media-apply.py` stays silent when it does not.
A rename of folders the server never identified strands nothing: Jellyfin
writes a sidecar only for a series it recognised, so `reap` answering
`no ghosts` after such a pass is the expected result, not a broken tool.
- **Nothing deletes through the metadata view.** A delete there writes an overlayfs
whiteout in the upper layer, which hides the array's copy without removing it;
the file returns when the view is unmounted. Deletions in this pipeline happen
@@ -0,0 +1,241 @@
---
type: reference
title: Media TV Relayout
description: "Re-file TVShows from scene packs into the shape Jellyfin reads: authority order, refusals, the two-series split test."
tags: [dankserver, media-server, jellyfin, plex, procedure]
timestamp: 2026-11-02T11:00:00Z
---
# Media TV Relayout
The `/mnt/entropy/TVShows` share arrived as scene packs — release folders whose
names are the release, not the show. This arm re-files it into the shape
[[Jellyfin]] reads, with tvdb as the name authority. The executor, the journal,
the sweep and the rollback belong to [[Media Relayout]]; this page owns the TV
authority order, the refusals, and the test that decides a split.
`/mnt/entropy/TVShows` is scene packs rather than a library, and Jellyfin has
already matched and numbered most of it. `scripts/media-shows.py` re-files that
library into the shape Jellyfin reads, with **tvdb as the name authority**:
<Series> (<year>) [tvdbid-NNNN]/Season NN/
<Series> (<year>) - SxxEyy - <Episode Title>.ext
It never guesses: every series name, episode number and title it writes traces to
an authority, and what has none is parked for a human to read.
```bash
# 1. The authority: Jellyfin's own matched rows — each carries the series' TMDb
# and Tvdb ids and the title the row holds (the DB opens read-only).
python3 scripts/media-census.py jellyfin --root /mnt/entropy/TVShows \
--out /tmp/tv_titles.tsv
# 2. What is on disk (stat only), and the runtimes the film arm needs.
python3 scripts/media-census.py enumerate --root /mnt/entropy/TVShows \
--out /tmp/tv_enum.tsv
python3 scripts/media-census.py probe --root /mnt/entropy/TVShows \
--enum /tmp/tv_enum.tsv --jobs 6 --out /tmp/tv_probed.tsv
# The probe is the slow step: ffprobe reads both ends of every file. So re-use
# the last one — a duration belongs to the file, not to its name: re-key the
# previous census through that pass's own journal (`src` -> `dst`) instead of
# seeking the array again. It is also the only column the film arm reads, and
# only for a file with no marker and no provider row (4,794 of 58,551 today,
# all of them covered by a re-key that costs seconds, not an hour).
# 3. The plan. --tmdb needs TMDB_API_KEY — the v3 key in ~/.config/secrets/tmdb,
# which travels as the `api_key` query parameter. Only a v4 request token is
# accepted as a Bearer header, and the wrong shape answers 401 on EVERY call,
# which the summary line flags as NOT provider-complete. --tvdb reads its
# `api_key=` line from ~/.config/secrets/tvdb, never argv. Both write the SAME
# cache file, so the plan is reproducible offline afterwards: run it once to
# fill the cache, and quote the second run.
# Paths are only strings to a dry run, so this step can run on the workstation
# against scp'd TSVs — which is where the two provider secrets live.
python3 scripts/media-shows.py --root /mnt/entropy/TVShows --enum /tmp/tv_enum.tsv \
--titles /tmp/tv_titles.tsv --probed /tmp/tv_probed.tsv \
--tmdb-cache /mnt/entropy/.media-campaign/media-tv-names.json --tmdb --tvdb \
--park /mnt/entropy/NeedsProcessing/TVShows \
--out /mnt/entropy/.media-campaign/media-relayout-shows2-plan.tsv \
--exceptions /mnt/entropy/.media-campaign/media-relayout-shows2-exceptions.txt
# 4. Read the exceptions file FIRST — it is the review, not a log. Same executor
# as above, but on this arm's OWN journal, NAMED PER PASS: the journal of an
# applied pass is its unwind record, so a second pass that reuses the name
# destroys the only map back.
scripts/media-apply.py apply --plan /mnt/entropy/.media-campaign/media-relayout-shows2-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/.media-campaign/media-relayout-shows2.jsonl --apply
```
Evidence, strongest first — the order the planner believes things:
- the **episode marker the file's own name states** (`S01E02-E03`, `1x04`); a digit
run longer than three at that position is NOT a number — `S03E2019` and
`S04E47370355` are one box set's airdate fragment and pack index — so such a file
is renumbered by tvdb's title or parks, and is never truncated down to fit;
- the **`Season NN` folder it sits in and the number its name leads with inside
it**: `Season 19/01 Charles Barkley, Nirvana.avi` is S19E01;
- the **provider row**: which episode the file IS, and the numbering of a file
whose own name says nothing;
- the **row's tvdb id, else its TMDb id carried through TMDb's external ids, else
an exact tvdb name search**: that id names the SERIES FOLDER and supplies the
episode title, because a name the server read off a folder is not a show name.
The folder CARRIES the id: `[tvdbid-NNNN]` is what Jellyfin's Tvdb agent and
Plex's scanner both read, so a show whose title is ambiguous resolves by number
instead of by a human guessing which folder is which. The id rides the folder
only — a file name gets 180 characters and the episode title needs the room.
- a **bonus folder** anywhere above the file, or Jellyfin's `ExtraType` → the
series' canonical extra folder;
- a runtime over `SHORT_MAX` makes it film-shaped, and it leaves for `--movies` only
when the movie provider's title is a run of words the FILE itself states and its
year agrees with the file's: the query is only what `title_of` salvaged from a
release name, and the first hit for a short one is a near miss — a 1963 Bond film
for the word "russia", *Special Edition I* for a file saying *Edition III*. Else it
parks.
Four refusals carry the design: what the planner will NOT do is the point of it.
- **Folders resolve a series; they never invent one.** A folder's words may
resolve a file to a series a provider has already named — `(CBB) Tengen Toppa
Gurren Lagann - The Complete Series (BD 1080p)` names the registered *Tengen
Toppa Gurren Lagann* — but a bonus subfolder is not a show. An early run made
`…/03. Extra/01. NCOP/NCOP 01.mkv` into a root series called `01 NCOP`.
- **A scraped row does not outrank the filesystem.** Jellyfin numbers
multi-season packs badly: every file under `SNL 16-20/Season 16..19` carries
season 16 episode 20, and one Gundam row says season 2004. The folder, then
the file's own leading number, win.
- **A season that is a YEAR is the date matcher, not a season.** Jellyfin files
specials by airdate, so a row can arrive saying `season 2002`; that is not a
season and it never becomes a folder. The one authority on such a row is tvdb's
episode list read backwards: the title the row carries names the (season,
episode) tvdb gives it. A title tvdb gives to two episodes names nothing, and
the file parks with the reason it always carried. `Season 00` IS a legal
season — specials live there — so a row that really says 0 files there instead
of parking.
- **A season tvdb's own list contradicts gets no tvdb title.** An anime pack
numbered by its release group books `S05E01` in a season tvdb does not hold, or
books more episodes there than tvdb lists: the slot its number points at is a
different episode, and dressing it in that episode's title is a confident lie.
The file still renames to the standard shape; the title does not ride, and the
series is listed under *seasons whose numbering tvdb contradicts* in the
exceptions. A season holding FEWER files than tvdb lists stays plausible — a
partial pack is the normal shape. Neither reading holds for a list tvdb
cut off at its own page limit: the default episode query answers at most
500 rows and states no total, and One Piece, Crayon Shin-chan and
Detective Conan each answer exactly 500 while holding over a thousand —
a page bounds nothing above itself, so it contradicts nothing.
**A provider name is evidence only when an id backs it.** Jellyfin names a Series
after the folder it scanned, so a library filed by release folders ends up with
one Series item per box set — `24 (2001) Season 1`, `Modern Family S05` — and
copying those names verbatim launders folder junk into the contract. The planner
resolves each series to its tvdb id and writes tvdb's name and year; the merged
identity is also what turns a box set split across five release folders into ONE
folder, and what makes two files claiming one episode MEET in the duplicate rule.
A series no id resolves keeps the provider's words and is listed under *no tvdb
identity* in the exceptions — that list, not a rename, is the review.
**The review pile lives outside the library.** Files parked inside
`/mnt/entropy/TVShows` were indexed as one show named `_unsorted`, and that
show's rows then looked like evidence. So a row whose series is the hidden
top-level folder the file itself sits in is dropped — and the folder is matched
with its leading underscore stripped, because the server *displays* `_unsorted` as
`unsorted`: compare the spelled names and the pile survives the relayout as a
visible series folder that real files get renamed into. `--park` outside the root
makes the next plan re-file the pile instead of skipping it. Inside a hidden
top-level folder a file's own words may MATCH a registered show, never invent one.
**The operator's id map is the door for a name tvdb cannot be searched by.**
`scripts/media-shows.py` reads `scripts/media-shows-tvdb-ids.tsv` by default: one
line per pack folder, `folder <TAB> tvdb id <TAB> note`. The file exists because
tvdb titles a record in the show's own language and the English name rides in
*aliases* the name search never reads: `ブラッククローバー` (331753) answers to no
spelling of "Black Clover", and the bare alias `One Piece` belongs to 392276, the
2023 live action, not to the 1999 record (81797) the pack here is. The map
outranks a provider id — a box set's provider id is the server's own guess at a
folder name — and it merges the release folders that are one show. Everything it
touched is written to the exceptions: every folder it named, every row that
matched no folder, every disagreement with a provider id, every id tvdb never
answered. A line whose id is not a number stops the plan, because a typo renames
a whole pack to the wrong show. Only folders the search CANNOT reach belong in
it; a row for a folder the search would resolve hides a live lookup behind a
number nobody re-checks.
`scripts/media-shows-tvdb.py` makes the sheet the map is built from. It reads a
plan's `mkdir` rows, asks tvdb about every destination folder carrying no
`[tvdbid-`, and writes one row per folder with the verdict that says how much was
guessed to get there: `EXACT` (tvdb's own spelling), `PUNCT` (equal only after
release junk is stripped from the folder's words), `NEAR` (tvdb answered
something else — the best hit is printed for a human and nothing applies it),
`NONE`. Acceptance never moved across the passes that built this: a hit counts
only when tvdb's name equals the query with every non-alphanumeric dropped from
both sides, and the record's year still has to fall inside any year span the
folder names. That gate is what caught three confident picks — Fairy Tail's 2024
sequel (410031), the 2026 Hokuto no Ken remake (465216) and `Mugyutto! Black
Clover` (378762) offered for Black Clover. It is why the map is approved by hand
and the sheet is never fed to the planner.
**A parked folder merges into the row that owns the show.** The folder that never
got an id usually holds episodes the identified folder for that show lacks, and
the server shows the audience neither: the parked copy is invisible and the owner
reads as a short series. One `media-apply.py` plan per show, built from the
server's own rows:
- the **owner is the folder whose `Series` row carries the tvdb id**; the parked
side is the folder with the same words and no id (or the same id under a second
row — see the law below);
- the **comparison is episode identity, not filename**: which (season, episode)
the provider row gives each file, read straight out of the server. Release
folders renumber — one park carried `S01E01-11` of a show tvdb books as seasons
2 and 3, and every episode runs the same eleven minutes, so neither the marker
nor the duration says what one query into the rows says;
- **a season the owner has no file in moves whole**: `mkdir` the `Season NN`,
then one row per file with its name unchanged — the folder shape is the
contract, the file name is not;
- **a subtitle pack is re-keyed, never just moved**: a sidecar attaches only when
its stem equals the video's, so each `.srt` has to land on the OWNER's stem with
the language suffix kept (`… - The Cat and the Claw (1).en.srt`). Moved under
its release-group name it stays a folder of 88 subtitles no player ever offers;
- **the same episode on both sides is the operator's call**, never a plan row:
which copy survives is a quality judgment — the parked copy is usually the
bigger one, and a re-cut is not a worse one — and the parked file often carries
no title to align by. Those pairs are listed and left alone.
The two sides speak different path dialects, since the server records the
container coordinate ([[Jellyfin]]) and the journal speaks the array; a join that
forgets the translation returns zero rows and looks like an empty library.
## Laws of this arm
- **A folder that names two series is not evidence of two series — the files
answer.** `ffprobe -show_entries format_tag=title` names the series each file
belongs to. Measured 2026-11-02 on `COW and CHICKEN … Complete Animated TV
Series`, a bundle whose name pairs *Cow and Chicken* with *I Am Weasel*: 103
of 104 files carry a `Cow and Chicken - SxxEyy - <title>` tag and none carries
an *I Am Weasel* one. The 7-minute duration class is a half-episode segment,
not a second show, and a file holding both segments needs a remux, which this
executor will never do. Split only when the tags (or a provider episode list)
name both series per file; everything else is a rename.
- **When the numbering is right and only the name is wrong, rename the folder
and nothing else.** Tags whose `Sxx` matches the `Season NN` folder holding
them, per-season counts equal, and no duplicate `SxxEyy` key anywhere prove
the layout is already correct. Renumbering files under that evidence buys
nothing and can silently mislabel an episode.
- **One tvdb id can carry two `Series` rows, so the id-less folder list proves
nothing about a merge.** A parked folder whose words parse to a registered show
gets identified on its own and keeps its own folder: the count of id-less
folders drops, and the library now offers the same show twice. Prove a merge by
the episode rows standing under the owner folder, never by that list.
- **Move library files only while `Scan Media Library` is Idle.** A rename that
lands under a running scan is a row the server read a moment ago pointing at a
path that no longer exists. `ScheduledTasks` is the read ([[Jellyfin]]); wait
for a `Running` sighting and then two consecutive `Idle` polls, because a gap
between sub-tasks reads as finished.
## See Also
[[Media Relayout]] — the executor, the journal, the sweep and the rollback this
arm shares. [[Jellyfin]] — the reader of the resulting shape. [[Plex]] — the
second reader. [[Media Metadata View]] — where a rename's sidecars strand, or
do not.
## Citations
- Jellyfin Shows documentation — series, season and episode folder/file naming:
<https://jellyfin.org/docs/general/server/media/shows>
+47 -33
View File
@@ -106,43 +106,57 @@ root runs as the `dank` user, and the deployer adopts the data root with
## Triggering a scan
A rename on disk is invisible until the section is rescanned, and a moved file is
a `Couldn't find the file to stream` 404 until that scan passes it. Run this from
the host shell through `podman exec`, never from a neighbour container: PMS's
HTTPD closes without a single byte on any request that arrives from another
container's netns (10.89.x.x reads as "empty reply", not as a 401). The token
rides stdin, never argv, where any process on the host could read it out of
`/proc/<pid>/cmdline`; it lives in `Preferences.xml` inside the RW data bind.
a `Couldn't find the file to stream` 404 until that scan passes it. Run the
driver from the host shell; the section keys are `1` Movies, `3` TV Shows,
`5` Audiobooks, `6` Music, `11` VODs & Live Events:
```bash
umask 077
podman exec plex sh -c "sed -n 's/.*PlexOnlineToken=\"\\([^\"]*\\)\".*/\\1/p' \
'/config/Library/Application Support/Plex Media Server/Preferences.xml'" >/tmp/px.tok
cat >/tmp/pxscan.sh <<'EOS'
read -r T
cfg=$(mktemp)
mk() { { printf 'header = "X-Plex-Token: %s"\nheader = "Accept: application/json"\n' "$T"
printf 'url = "%s"\n' "$1"; } > "$cfg"; }
mk "http://127.0.0.1:32400/library/sections"
for k in $(curl -s -K "$cfg" | grep -o '"key":"[0-9]*"' | cut -d\" -f4); do
mk "http://127.0.0.1:32400/library/sections/$k/refresh"
printf 'request = "POST"\n' >> "$cfg"
echo "plex section $k: $(curl -s -o /dev/null -w '%{http_code}' -K "$cfg")"
done
shred -u "$cfg"
EOS
podman cp /tmp/pxscan.sh plex:/tmp/pxscan.sh
podman exec -i plex sh /tmp/pxscan.sh </tmp/px.tok
shred -u /tmp/px.tok
scripts/media-refresh-plex.sh trigger --library 1
python3 scripts/media-verify.py --db-kind plex \
--journal /mnt/entropy/.media-campaign/<journal>.jsonl # the real proof
```
Every section answers `204` — or `200` with a Directory body when
`Accept: application/json` is sent. A refresh fired while PMS is still warming
answers `000` instead (the empty reply of gotcha 4, arriving mid-request): retry
until it answers and never restart the service over it. Inside the quoted
`printf` format a backslash-newline is LITERAL, not a continuation, so `mk`
writes two `printf`s: one glued line and one stray backslash is a config file
curl rejects, and the failure prints as `option -K: is unknown`, never as a bad
config line.
Exactly ONE section: a refresh costs a full walk of that tree, and refreshing all
five for a rename that touched one buys nothing. `trigger` answers
`refresh accepted (204)` — `200` when a body is requested — and `000` means PMS is
still warming, which the driver retries out loud; a service restart is never the
fix.
**`status` is a census, not a waiter: no epoch makes it watch a scan.** It reads
`section_locations.scanned_at` — the epoch of the last scan PMS has *recorded* for the
section, the same value `GET /library/sections` prints as `scannedAt` (verified
2026-11-04) — and compares it against `--after`. Measured twice on 2026-11-04: a chain
fell through its wait loop in the same second its own trigger was accepted, and a direct
read 30 minutes later still answered `settled` off that one stamp
(`2026-11-04T19:46:59Z`, for a request accepted at `19:46:58Z`) while the walk ran. A
journal's mtime fails, and so does the trigger's own epoch — the request satisfies the
very door it was asked to wait for. The failure direction is a false *success*, never a
false alarm, which is what makes it worth the warning: ask `status` whether a scan got
recorded, and never let a chain depend on it.
**The proof that a scan did its work is the journal replayed against the database:**
`scripts/media-verify.py --db-kind plex --journal <journal>` names every row that still
points at a source path (`still at src`) and every entry whose file is physically still
there (`REAL`). Both at zero is what gates a purge, and it needs no settle signal at all
— a scan can clear rows but cannot invent one. Jellyfin's door has no such trap:
`scripts/media-refresh-jellyfin.py wait` watches the scan task itself and refuses by
name when no task reaches `Running`.
The driver ships its runner into the container with `podman cp` rather than
quoting a command through `podman exec sh -c`, because PMS's HTTPD closes without
a single byte on a request that arrives from another netns — the host address and
the `10.89.x.x` container address both read as "empty reply", never as a 401 —
and because a three-layer `ssh → podman exec → sh -c` quote chain dies on a parse
error long before it reaches the server. The request therefore rides plain
`http://127.0.0.1:32400` from inside the container, where insecure connections
are allowed (gotcha 5). The credential is `PlexOnlineToken` in the container's
`Preferences.xml` — NOT `PlexHTTPToken`, which a claimed server need not carry —
read inside the container into a mode-600 curl config there, so it never enters
argv (`/proc/<pid>/cmdline` is world-readable) and never a URL (proxies log
URLs). Hand-writing that config has one more trap: inside a quoted `printf`
format a backslash-newline is LITERAL, so a two-line string becomes one glued
line plus a stray backslash, and curl answers `option -K: is unknown` — never
"bad config line".
Proof that a rename landed is not a scan response: `scripts/media-verify.py`
replays the run's journal against the database — the only read that
@@ -4,7 +4,7 @@ description: "Outpost, proxy-provider, OAuth2, LDAP and flow configuration for t
tags:
- infrastructure
- authentik
timestamp: 2026-10-19
timestamp: 2026-11-03
---
# Authentik Outpost Reference
@@ -56,7 +56,7 @@ correctly — the outpost matches on hostname.
## Provisioning rules for proxy providers
Two provisioning rules, each verified live by an outage:
Three provisioning rules, each verified live by an outage or a dead login:
1. **All three flows are required.** A ProxyProvider created via raw API/ORM
with any of `authentication_flow`/`authorization_flow`/`invalidation_flow`
@@ -80,6 +80,16 @@ Two provisioning rules, each verified live by an outage:
the app. No error is logged anywhere — the SSO "works" and the app never
appears. Create the Application with `provider_id` set (the UI does this
automatically; raw API/ORM drives must do it explicitly).
3. **A raw ORM create skips the OAuth defaults.** The REST serializer calls
`ProxyProvider.set_oauth_defaults()`; `ProxyProvider.objects.create()` does not. The row
then looks fully configured — external_host, internal_host, application bound, outpost
claimed — while `redirect_uris` is empty (those URIs are *derived* from `external_host` by
`_get_callback_url`) and `grant_types` is `[]`. Both kill the guard before any credential,
in this order: authentik renders its own **"Redirect URI Error"** page, and once the URIs
are filled by hand the next wall is a bodiless **HTTP 400** out of `/application/o/authorize/`
(both measured on the Audiobookshelf guard provider, 2026-11-03). Census and repair in one idempotent pass, all
providers at once: `scripts/authentik-proxy-provider-defaults.sh`; `--check` prints the
census and exits 1 on drift, and a provider that already matches is left alone.
## OAuth2 relying-party quirks (this fork; corrected live 2026-10-20)
@@ -138,36 +148,46 @@ Read from the live database (read-only `SELECT`s against the
| Object | Value |
|---|---|
| LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud`, `search_mode=direct`, `bind_mode=direct` |
| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` *(unverified 2026-10-18: base DN confirmed in the DB, the `ou=users` path was not probed — an LDAP bind needs the bind secret)* |
| Service account | `svc-jellyfin-ldap` (user pk 8). **It is not passwordless**: its password field is a real hash, not Django's `!`-prefixed unusable marker (`starts_with(password,'!')` → `f`) — an earlier revision's "passwordless/unusable" claim is wrong, which matters because an account with a usable password can sign in through the ordinary password flow *(unverified 2026-10-18: whether anyone holds that password)* |
| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — an earlier revision recorded `intent=verification`; the live value is `app_password` |
| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` — **verified by a live bind 2026-11-04** (the bind and the `memberOf` search both succeed, so the `ou=users` path is right) |
| Service account | `svc-jellyfin-ldap` (user pk 8). It has a usable password and authenticates through the default flow — that password is held by Jellyfin in the plugin config (`LdapBindPassword`) and is what the bind actually uses (live bind 2026-11-04) |
| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — and **unused by Jellyfin**, which has no token field (see the rule below) |
| RBAC role | `svc-jellyfin-ldap-search`, carrying exactly one object-level permission `authentik_providers_ldap.search_full_directory` (verified via `guardian_roleobjectpermission`) |
| Groups | `jellyfin_users`, `jellyfin_admins` (both exist in `authentik_core_group`) |
| Application | name `Jellyfin`, **slug `jellyfin-ldap`** (an earlier revision said slug `jellyfin`, which does not exist) |
Hard-won rules (the first is now contested by the live database):
Hard-won rules:
- **Token intent: contested, do not trust either value blindly.** An earlier
revision asserted that a bind token with `intent=app_password` is rejected by
the flow's password stage with `Invalid password` and only
`intent=verification` passes. The live token for `svc-jellyfin-ldap` is
`app_password` (`authentik_core_token`, 2026-10-18), so either the rule is
wrong or the current Jellyfin bind path is not that token. Nothing observable
settles it today: the LDAP outpost's retained log holds only its 5-minute
`initialized direct binder` / `Update providers` config cycles and no bind
activity *(unverified 2026-10-18: no bind was performed — reproducing it
requires the bind secret)*. Re-verify with a real bind before changing the
token's intent either way.
- **Token intent is moot for Jellyfin.** The earlier argument (a bind token with
`intent=app_password` is rejected by the flow's password stage, only
`intent=verification` passes) never applied here: LDAP-Auth sends the service
account's **password** from `LdapBindPassword`, not a token. A live bind plus a
`memberOf=cn=jellyfin_users` search (11 entries) succeeded from Dankserver on
2026-11-04. Leave the token row alone.
- **Search needs `search_full_directory`.** Without the RBAC grant the
outpost returns only the bind user itself — a `(memberOf=…)` filter then
matches zero entries and the plugin logs `Found no users matching X`.
The permission is object-level on the provider; grant it via an RBAC role
and retest.
- **The outpost log never shows a bind.** Its retained journal is only the
5-minute `initialized direct binder` / `Update providers` config cycle; the
absence of a bind line is not evidence of anything. Instruments that do answer:
`ldapsearch` with the plugin's own DN, password and filter, and the authentik
event registry (`login` / `login_failed` with `client_ip` from the podman
gateway, since the published port SNATs).
- `ak shell` import path for the provider model in 2026.8:
`from authentik.providers.ldap.models import LDAPProvider`.
- (unverified 2026-10-18: importing it outside a configured Django shell fails
with `ImproperlyConfigured`, and no `ak shell` was driven against the live
auth service for this check)*
`from authentik.providers.ldap.models import LDAPProvider` (verified inside a
live `ak shell`, 2026-11-04 — it only fails outside a configured Django shell).
- **`ak shell` field names, 2026.8:** `Event` is keyed by `created` and `action`
— there is no `timestamp` and no `type`; `Outpost` has `providers` but **no
`apps`**. Introspect first: `[f.name for f in Model._meta.get_fields()]`.
- **Never pipe `ak shell` inside a heredoc.** `podman exec -i … ak shell | grep
-v '^{'` binds the heredoc to `grep` (the redirection belongs to the LAST
element of the pipeline), so the shell gets no stdin and prints nothing at all.
And `2>/dev/null` on `ak shell` eats the traceback, which looks identical to "no
rows". Clean pattern: have the script write its results to a file and read that
file back —
`podman exec -i authentik-server ak shell >/dev/null 2>/dev/null <<'PY' … open("/tmp/x.txt","w") … PY`
then `podman exec authentik-server cat /tmp/x.txt`.
## Self-serve user-settings flow (password change)
@@ -145,18 +145,17 @@ relying-party quirks are reference lookup: [[Authentik Outpost Reference]].
The second outpost — `dankclaw/authentik/quadlet/authentik-outpost-ldap.container`
→ `authentik-outpost-ldap.service`, one of the five units in this stack —
serves LDAP for Jellyfin's LDAP-Auth plugin (`enabled`=generated, `active`
`Restart=always`, container up, 2026-10-18). The image is pinned by its **amd64
single-platform digest**
`ghcr.io/goauthentik/ldap@sha256:2e7b259eb8c37faac916946ddf48ec681ee9d3a9f724fe8521bf161973f13cf9`
(`dankclaw/authentik/quadlet/authentik-outpost-ldap.container:26`, matching the
running image in `podman ps`, 2026-10-18). The digest in this page before 2026-10-18
(`…fa33bc8f…`) is superseded. The reason for pinning a digest at all: podman
5.7.0 refuses the plain tag because it resolves to a *nested* manifest list
(index → per-platform manifest lists) — `unexpectedly received a manifest list
instead of a manifest for a single image` — first hit on `:2026.8.0`, still the
rule at the current `2026.8.3` pin. Resolve once with `podman manifest inspect
<image> | jq -r '.manifests[] | select(.platform.architecture=="amd64") |
.digest'` and pin that digest.
`Restart=always`, container up, re-checked 2026-11-04). **The image pin has one
owner: the `Image=` line of
`dankclaw/authentik/quadlet/authentik-outpost-ldap.container`** — never restate
the digest here, because every pin bump turns this page into a lie (the digest
cited here went stale twice: `…fa33bc8f…`, then `…2e7b259e…`). The reason a
digest is pinned at all: podman 5.7.0 refuses the plain tag because it resolves
to a *nested* manifest list (index → per-platform manifest lists) —
`unexpectedly received a manifest list instead of a manifest for a single image`
— first hit on `:2026.8.0` and still the rule. Resolve once with
`podman manifest inspect <image> | jq -r '.manifests[] |
select(.platform.architecture=="amd64") | .digest'` and pin that digest.
- Ports **389** (ldap) and **636** (ldaps) publish on the LAN
(`PublishPort=389:3389`, `636:6636`
@@ -393,9 +392,10 @@ downstream —
- `Non-Admin Users` = the plain-user marker group (precedents: qa-e2e, and
every human account).
- **The human-user template** (ridley, kassie, alex, artem, kirby, ry, nic,
debo, freya, mitch, eric). The pks in this page are confirmed against
`authentik_core_user` on 2026-10-18: ridley 13, kassie 14, alex 15,
artem 16, kirby 17, ry 18, nic 19, debo 20, freya 21, mitch 22, eric 23 —
debo, freya, mitch, eric, jason). The pks were confirmed against
`authentik_core_user` on 2026-10-18 and the table re-read 2026-11-04: ridley
13, kassie 14, alex 15, artem 16, kirby 17, ry 18, nic 19, debo 20, freya 21,
mitch 22, eric 23, **jason 24 (created 2026-11-04)** —
and every one carries the `<user>@franksdank.cloud` address the template
prescribes. Creation dates (2026-10-07 → 2026-10-15) were not re-checked
- (unverified 2026-10-18)*. Template: `username` +
+110 -3
View File
@@ -4,7 +4,7 @@ description: "Caddy is the single reverse proxy for both hosts."
tags:
- infrastructure
- caddy
timestamp: 2026-08-29
timestamp: 2026-11-03
---
# Caddy
@@ -90,7 +90,111 @@ pihole.franksdank.cloud {
After rendering, the line reads `dankuser <hash>` (one complete hash) — `tests/deployment_flow.bats:96-99` pins the `dankuser` name and the single-hash form, and `tests/caddyfile.bats:75` pins the bare `dankuser REPLACE_WITH_HASH` template. Do not put a `$2a$14$` prefix in the template: it would render a double-prefixed, invalid hash (2026-08-22 incident — every basicauth vhost returned 401 for every password; `tests/caddyfile.bats:67-72` forbids the prefix). `tests/deployment_flow.bats` pins the render contract.
Everything else authenticates, not basicauth. The guarded set — `www.` (homepage), `search.`, `grafana.`, `prometheus.`, `loki.`, `seerr.`, `marreta.`, `romm.`, `files.` — proxies **all** traffic to `anubis:8923`, which verifies the client and forwards it back to Caddy's internal listener `http://:8016`; Authentik's proxy-provider mode serves the login inline (this build exposes **no** `forward_auth` endpoint) and forwards authenticated traffic to `http://:8017`, where the request is routed straight to its app (`dankclaw/Caddyfile:14-24,283-423`). The exempt-by-construction set never enters that chain: `auth.` (the IdP itself), `ai.` and `gitea.` (own logins), `hermes.` and `hermesadmin.` (app-enforced passwords), `pihole.` (the basicauth edge above), and the media/*arr vhosts. Grafana left basicauth on 2026-09-01 and is an OAuth2 relying party against Authentik (`dankclaw/Caddyfile:122-132`). See [[Homepage]] and [[Authentik]].
Everything else authenticates, not basicauth. The guarded set — `www.` (homepage), `search.`, `grafana.`, `prometheus.`, `loki.`, `seerr.`, `marreta.`, `romm.`, `files.`, `notebook.` — proxies **all** traffic to `anubis:8923`, which verifies the client and forwards it back to Caddy's internal listener `http://:8016`; Authentik's proxy-provider mode serves the login inline (this build exposes **no** `forward_auth` endpoint) and forwards authenticated traffic to `http://:8017`, where the request is routed straight to its app (`dankclaw/Caddyfile`: the header comment and the two internal listeners). The exempt-by-construction set never enters that chain: `auth.` (the IdP itself), `ai.` and `gitea.` (own logins), `hermes.` and `hermesadmin.` (app-enforced passwords), `pihole.` (the basicauth edge above), the media/*arr vhosts, and `audiobooks.` — Audiobookshelf enforces its own auth (local accounts plus authentik OIDC) and must answer native app clients, podcast RSS and OPDS, which the chain structurally cannot deliver: it answers a client that solves no JavaScript proof-of-work with challenge HTML wherever that client expects JSON or a socket payload ([[Audiobookshelf]]). Grafana left basicauth on 2026-09-01 and is an OAuth2 relying party against Authentik. See [[Homepage]] and [[Authentik]].
## Guard a new vhost
Order is the whole procedure: the IdP row has to exist **before** the route flips, or
authenticated traffic reaches the outpost with no matching provider and the user gets
authentik's error page instead of the app.
1. **Provider first.** A proxy provider plus an application, claimed by the embedded
outpost. No blueprint owns this state — it lives only in the IdP's database, which is
why it is easy to forget:
```
ssh dankclaw 'podman exec -i authentik-server ak shell' <<'PY'
from authentik.providers.proxy.models import ProxyProvider
from authentik.core.models import Application
from authentik.outposts.models import Outpost
NAME, EXT = "example", "https://example.franksdank.cloud/"
pp = ProxyProvider.objects.filter(name=f"{NAME}-proxy").first()
if pp is None:
pp = ProxyProvider.objects.create(name=f"{NAME}-proxy", external_host=EXT,
internal_host="http://caddy:8017")
if not Application.objects.filter(provider=pp).exists():
Application.objects.create(name=f"{NAME.title()} Guard", slug=f"{NAME}-guard",
provider=pp)
o = Outpost.objects.get(managed="goauthentik.io/outposts/embedded")
o.providers.add(pp) # an unclaimed provider is invisible to the outpost
print(pp.name, pp.external_host, o.providers.count())
PY
```
`external_host` is the exact vhost URL **with the trailing slash**: the embedded
outpost routes on the ORIGINAL `Host` header alone, so anything else matches zero
providers. Never print a provider's fields wholesale — a proxy provider carries a
client secret, and it lands in whatever transcript the command runs in.
`ProxyProvider.objects.create()` is not the API: the serializer's
`set_oauth_defaults()` never runs, so the row lands with `redirect_uris` and
`grant_types` empty, and the guard fails *before* any credential — authentik's
"Redirect URI Error" page, then a bodiless HTTP 400 once the URIs are filled
by hand. Run `scripts/authentik-proxy-provider-defaults.sh` after the snippet
(idempotent, covers every provider; `--check` is the census); law and failure
signatures in [[Authentik Outpost Reference]].
2. **Then the route.** The public site block proxies the whole request to `anubis:8923`
with `header_up X-Real-Ip {remote_host}`, and the post-auth `http://:8017` listener
gains a `@<name> host <vhost>` handler pointing at the app — copy `romm`/`files`,
which are the same shape including a Dankserver LAN-port upstream.
An app that derives its own origin from the request needs the scheme pinned:
Audiobookshelf checks every OIDC callback against `x-forwarded-proto://Host`; this
listener is fed over plain http so Caddy's own `X-Forwarded-Proto` is `http`, and the
app answers a 400 "Invalid callback URL - must be same-origin" for its own https
callback (measured 2026-11-03, while this vhost was still guarded — it is exempt
now, and the same pin sits on its edge block instead).
The edge for a guarded name is TLS-only, so `header_up X-Forwarded-Proto
https` is the truthful value ([[Audiobookshelf]]).
3. **Then the name and the audit.** A `dankclaw/pihole/addn-hosts` line (the `pihole`
seam fails the build if a Caddyfile vhost has no record), a Porkbun A record if the
name must resolve off-LAN ([[LAN-DNS]]), and the subdomain added to `GUARDED` in
`tests/e2e/specs/14-guard.spec.ts` with an app-specific leak marker — that spec is
the deploy gate's proof that the guard, not the app, answers an anonymous client.
## Unguard a vhost
The same three artifacts, in the opposite order. Getting it wrong takes the app
down for browsers while looking healthy:
1. **Route first.** Move the public site block off `anubis:8923` to a direct
`reverse_proxy` (keeping any origin-pinning `header_up` the app needs), delete
the `@<name>` handler from the `http://:8017` listener, and deploy. Traffic
stops entering the auth listener at `:8016`, so the IdP row goes unused — and
nothing breaks.
2. **Then the IdP rows**, once the route is verified:
```
ssh dankclaw 'podman exec -i authentik-server ak shell' <<'PY'
from authentik.providers.proxy.models import ProxyProvider
from authentik.outposts.models import Outpost
pp = ProxyProvider.objects.filter(name="audiobooks-proxy").first()
if pp:
o = Outpost.objects.get(managed="goauthentik.io/outposts/embedded")
o.providers.remove(pp)
app = pp.application # the "<Name> Guard" application row
if app:
app.delete()
pp.delete()
print("providers now claimed:", Outpost.objects.get(
managed="goauthentik.io/outposts/embedded").providers.count())
PY
```
Deleting the provider **before** the route flips is the wrong order: the
outpost then matches zero providers for a Host that still arrives, and every
browser gets authentik's error page instead of the app.
3. **Then the audit lists.** Drop the subdomain from `GUARDED` in
`tests/e2e/specs/14-guard.spec.ts` (and its leak marker), and replace it
there with an exemption assertion — the deploy gate must keep proving the
exemption is deliberate, the way `plex` does: app bytes to an anonymous
client, zero challenge HTML, and the app's own `401` on an unauthenticated
API call rather than the gate's `200`.
What the exemption costs, said plainly: the app's login endpoint becomes
internet-visible with no rate limiter in front of it unless the edge carries one
(this Caddy build has no `rate_limit` module) or the app ships a lockout (ABS
2.37.2 does not). That is the same exposure `jelly.`, `plex.` and the *arr
vhosts already live with; the mitigation is the credentials.
## Operations
@@ -113,7 +217,10 @@ Certificates and ACME state live in the caddy data volume (`data/caddy/`). Back
- A bare compose container is **not** boot-persistent: after the 2026-08-26 reboot Caddy stayed down for hours because `restart: unless-stopped` does not survive shutdown's explicit stop. It now runs only from its quadlet unit, and the repo has no `dankclaw/compose.yaml` left to re-add it to (`ls`, 2026-10-18).
- `admin off` disables the config file watcher: rendered changes then sit on disk while Caddy serves its stale in-memory config until the container is recreated, and nothing logs an error (2026-08-22: a broken basicauth hash survived several deploys this way). `tests/caddyfile.bats` forbids `admin off`.
- The basicauth template must stay a bare `REPLACE_WITH_HASH` token (no `$2a$14$` prefix) — the rendered value is the complete hash. A double prefix silently rejects every password with 401; no Caddy error is logged.
- A public vhost needs three things: a Pi-hole `addn-hosts` line, a Porkbun A record, and the Caddyfile block. See [[LAN-DNS]]. The deliberate exception is an **internal-only** name: `hermes.franksdank.cloud` and `hermesadmin.franksdank.cloud` are in `addn-hosts` and the Caddyfile but have no public record, which is what keeps them LAN-only — DNS-01 issuance needs no public A record (`dankclaw/Caddyfile:173-179`).
- **Exposure has exactly two gates: the router's 80/443 forward to Caddy, and Caddy's vhost list.** Caddy matches on SNI/Host, so *every* name in the Caddyfile already answers a WAN client that sends that name — a public A record buys **discoverability, not access** (measured 2026-11-04). A published name needs a Pi-hole `addn-hosts` line, a Porkbun A record and the Caddyfile block ([[LAN-DNS]]); DNS-01 issuance needs no public record, so an **internal-only** name is one with no A record. Posture measured 2026-11-04: the whole authentik-guarded set is published — `www.`, `search.`, `romm.`, `files.`, `grafana.`, `prometheus.`, `loki.`, `seerr.`, `marreta.`, `notebook.` — while the names with no public record are the self-authenticating exempt ones (`pihole.`, `hermes.`, `hermesadmin.`, `prowlarr.`, `radarr.`, `lidarr.`, `readarr.`, `qbittorrent.`) and the legacy redirects (`dankclaw.`, `dankserver.`). Every A record is a copy of the WAN address, which is dynamic — republish after an IP change.
- **A missing DNS record is not a gate: the names are public anyway.** Caddy issues one certificate per name and every issuance is published in Certificate Transparency logs — `crt.sh` for `%.franksdank.cloud` lists the live *and* retired set (`sonarr.`, `comfyui.`, `chat.`, `jellyfin.`, the whole *arr stack), so "LAN-only because there is no A record" is fiction and a stranger can send the Host. A household-only service needs a **source gate in its vhost**, which is what the *arr stack and qBittorrent now carry: a LAN client proxies through, anything off-LAN gets `404`. Keep those five gated even though they have no A record.
- **The LAN gate is `remote_ip`, and it reads the true client.** The G36 DNATs the 80/443 forward straight to Caddy with no proxy hop, so `@lan remote_ip 127.0.0.1/32 ::1/128 192.168.0.0/16 10.0.0.0/8 172.16.0.0/12` sees either a LAN address or the internet address — never the router's. Verify the split without waiting for a WAN client: a LAN request must reach the upstream (a dead upstream still proves it passed the gate, because the answer is its `502`, not the gate's `404`), and an external fetcher must get the `404`.
- **A new record looks like it failed for up to the SOA's negative TTL.** The zone's negative cache is 1800 s, so a resolver that asked for the name *before* the create keeps answering NXDOMAIN (1.1.1.1 did, while Quad9 and the authoritative NS had the record seconds after the create) — check `dig @<authoritative NS>` before re-creating anything.
- The G36 has no NAT loopback, so LAN clients must resolve the name to the LAN IP. That is the Pi-hole role in [[LAN-DNS]]. *(unverified 2026-10-18: hairpin behaviour is only observable from a WAN-side client; nothing on the LAN or in this repo can prove it)*
- Caddy serves HTTPS on 443 and HTTP on 80 (redirects). The router needs no 80/443 port forward for LAN use.
- `transport http {versions h2c 1.1}` on a vhost makes Caddy send an h2c
@@ -28,11 +28,12 @@ descriptions and a `docs` link to `/docs#models`.
**Sections + WIP split (2026-09-21):** both pages now group their panels
under section headings rendered by shared `renderSection()`/`renderPanel()`
components in `app.js`. The user page orders **AI, Web Apps, Media Library
Apps**; the admin page orders **Media Library Apps, Observability, AI,
Infrastructure**. The WIP badge split into **WIP: OPEN** (work in progress)
and **WIP: CLOSED** (work finished by decision — e.g. a card that is
deliberately unprobeable) via the `wip` value (`"open"` / `"closed"`).
components in `app.js`. The user page orders **Web Apps, Media Library
Apps, AI, Work in progress** (the three not-open-yet apps moved under their
own heading, below AI, on 2026-11-04); the admin page orders **Media Library
Apps, Observability, AI, Infrastructure**. The WIP badge split into **WIP:
OPEN** (work in progress) and **WIP: CLOSED** (work finished by decision — e.g.
a card that is deliberately unprobeable) via the `wip` value.
Tautulli (Plex statistics, `192.168.0.235:8181`) joined the admin page's
Media Library Apps section.
@@ -314,9 +315,10 @@ flowchart TD
## Service classification
**User services** (root page), in section order:
- **AI** — OpenWebUI, and the three LLM fleet cards (frank-dragoon, frank-archon, frank-zealot — status + docs link, no Open link).
- **Web Apps** — SearXNG, Gitea, Copyparty, Marreta.
- **Media Library Apps** — Calibre-Web, RomM, Plex, Jellyfin, Seerr.
- **Media Library Apps** — Plex, Jellyfin, Audiobookshelf (moved up from the admin page 2026-11-03, once Authentik SSO made it a household app).
- **AI** — OpenWebUI, the three LLM fleet cards (frank-dragoon, frank-archon, frank-zealot) and `frank-tts`, the cloned-voice endpoint (moved up from the admin page 2026-11-04 so the household sees whether the voice door is up). All four are status + docs link, no Open link; the fused **Model Information** panel renders under this heading.
- **Work in progress** — Calibre-Web, RomM, Seerr. WIP badge, no Open link, no household account yet: the ask-Frank door. See the docs page's *Three doors that are not open to the household yet*.
**Admin services** (`/admin/`), in section order:
- **Media Library Apps** — the *arr stack (Prowlarr, Sonarr, Radarr, Lidarr, Readarr), qBittorrent, and Tautulli — live on FrankBook-Pro and Dankserver, operator-only: no vhosts (Tautulli included), direct IP:port access only. See [[FrankBook-Pro]].
@@ -324,6 +326,12 @@ flowchart TD
- **AI** — Hermes, ComfyUI.
- **Infrastructure** — Caddy, dnsmasq, Gluetun, and the low-level backends (mariadb, valkey, Postgres). The backends show status only. They get a link only where a UI is meaningful.
**The registry is the authority.** `shared/homepage/services.json` decides which
page a card renders on (`category`), whether it shows a WIP badge instead of live
status (`wip`), and whether it gets an Open link (`showLink` + `url`). The two
lists above are a locator only; when they disagree with the registry, the page is
the bug — fix the page.
## Layout
```
@@ -426,15 +434,22 @@ agent knowledge base (`~/.pi/agent/docs/caddy/caddyfile.md`).
- `category` — `user` or `admin`. Selects the page.
- `section` — the heading the panel renders under. Must be one of the page's
`SECTION_ORDER` entries (user: AI, Web Apps, Media Library Apps; admin:
Media Library Apps, Observability, AI, Infrastructure), or the panel is
silently dropped from the page.
`SECTION_ORDER` entries (user: Web Apps, Media Library Apps, AI, Work in
progress; admin: Media Library Apps, Observability, AI, Infrastructure), or
the panel is silently dropped from the page.
- `url` — the "Open" link (the Caddy site).
- `docs` — the per-service docs page (`/docs/<svc>/`).
- `probe` — the backend `host:port` the checker probes.
- `method` — `tcp` or `http`. The probe type for this service (Q10).
- `wip` — `false`, or `"open"` / `"closed"`. When set, the badge shows
**WIP: OPEN** or **WIP: CLOSED** and the checker skips the probe.
- `wip` — `false`, or `"open"` / `"closed"` / `"down"`. When set, the badge shows
**WIP: OPEN** / **WIP: CLOSED** / **WIP: DOWN** and the checker skips the probe.
- `model` — present only on Model Information cards. Such a card renders in the
fused `renderModelInfo()` panel (`mi-*` styles) instead of the app grid, and its
`description` is unused. `unit` is the StarCraft identity: art
`/img/unit-<unit>.jpg`, accent `.mi-<unit>`, voice lines from `/sfx/`. A card
with no `unit` is a tool (today: `frank-tts`) — art falls back to the card's own
`art`, the accent is `.mi-tool`, and a `tagline` with no `sfx` renders as plain
text rather than a play button.
**A card must be probeable from the monitor's host, or it must be `wip`.** The
checker runs on Dankclaw, so its reach is: that host's own loopback, its own
@@ -197,23 +197,18 @@ and PerfOpt is now a bonus nobody is waiting for rather than the reason to flip.
**Post-boot gates.** 0 failed units, `systemd` running. 32 IOMMU groups — 31
`identity` + 1 `DMA`, and that one belongs to the Genesys Logic SD card reader
(`c2:00.0`, a kernel quirk, not a fleet device). The iGPU (`c5:00.0`) sits in group
20 `identity` with ATS capability; interrupt remapping is enabled. frank-dragoon's
boot ledger is unchanged to the MiB — 62.9 GiB weights (0.8 of them vision) + 5.5
GiB KV + 5.7 GiB working = 74.1 GiB, two slots over ONE 200,704-position pool, MTP
depth 2, prompt cache ON, no clamp line — and Gitea, the Caddy edge and Grafana all
answered 200. (Later the same day the pool DOUBLING to 401,408 positions for
slot parity was COMMITTED (`2b12716`) and **deployed the same evening**: the
boot now reserves `401408 positions (about 11.0 GiB)` and the engine-side total
is `79.6 GiB in all` (62.9 weights + 11.0 KV + 5.7 working)
see [[frank-dragoon]].)
20 `identity` with ATS capability; interrupt remapping is enabled. The flip moved
frank-dragoon's boot ledger by nothing measurable — its shape and every number in it are
owned by [[frank-dragoon]] — and Gitea, the Caddy edge and Grafana all answered 200.
**The NPU came back with the flip.** `amdxdna` bound to `0000:c6:00.1` (Strix/
Krackan/Strix Halo NPU rev 11), firmware `amdnpu/17f0_11/npu_7.sbin`, `Initialized
amdxdna_accel_driver 0.7.0 … on minor 0`, IOMMU group 26 `identity`, PASID cap width
10 (`Exec- Priv-`). `/dev/accel/accel0` is `crw-rw---- root:render 261,0` and `dank`
is in `render`, so a quadlet needs `AddDevice=/dev/accel/accel0` +
`GroupAdd=keep-groups`. **No NPU endpoint is deployed** — stays the
history of the retired endpoint; what changed is only its precondition.
`GroupAdd=keep-groups`. The flip brought the device back, not a service: no NPU chat
endpoint was deployed then. The XDNA2 lane returned to work a day later as frank-dragoon's
small-model lane ([[frank-dragoon]]) — the shape and its laws are owned there.
**The `pp_dpm_fclk` pin (level 7, 2000 MHz) went in on 2026-10-19**, with the first thing that needed it: the
vendor's root host unit, installed and enabled by `frank-dragoon`'s deployer while its NPU lane is named. Pinning
+16 -1
View File
@@ -20,7 +20,7 @@ timestamp: 2026-10-19
The radiance engine (C++/HIP, `libr4d` kernels built for gfx1201, no Python in the serving
path) driving `podman run` straight from the digest-pinned image under a **root** systemd
unit; container name = unit name. Host Dankserver (192.168.0.235), port 7770, both Radeon AI
PRO R9700s. Capacity contract **2 conversations × 200,000 tokens at fp8 KV** — at 1.8.1 that pair
PRO R9700s. Capacity contract **4 conversations × 200,000 tokens at fp8 KV** — at 1.8.1 that pair
is the **demand signal** the budget resolver sizes the cache against, not a KV allocation (Laws).
A payload swap is a `MODEL_FILE` line plus a re-download — see Eras.
@@ -525,6 +525,21 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
- `--prefix-cache-dir` creation is **not recursive** — the serve script `mkdir -p`s the path or
the disk tier silently never opens. The tier does survive a payload swap when arch and
tokenizer match (it recovered 900,176 blocks on the 2026-10-15 swap; `cache_n` proves a hit).
- **A radiance crash reads as a clean exit unless you know where to look, and the engine never
says why.** The unit's `ExecStart` ends in `exec podman wait`, so `systemctl show -p
ExecMainStatus` reports **0** after a container death — that is `podman wait`'s own status, not
the engine's — and `Result=success` beside `NRestarts=1` is the *normal* fingerprint of a
crash. The container's real code is the bare digit `podman wait` prints to stdout
(`serve-frank-archon.sh[NNNN]: 1`). The engine's fatal path writes **nothing** to its log: no
stderr line, no message before the exit and no core. The only copy of the reason is the error
the client received — `the engine stopped on an error and is not serving (<errno>)`, where the
parenthesis is the failing call's errno text (`invalid argument` = EINVAL, 2026-11-04). So
grep the *client* logs before concluding the journal is empty: omp keeps the provider error in
`~/.omp/logs/omp.<date>.<pid>.log` (`agent turn ended with provider error`), and OpenWebUI
shows it as the chat's error toast. Seen once (2026-11-04, restart counter 1 after 3d20h up):
three concurrent triple-digit-thousand-token agent turns prefilled at once, prefill rate
collapsed, all in-flight streams failed at the same instant, then a silent teardown — kernel
logged only the KFD SVM workqueue churning, with no OOM, GPU hang, MCE or swap pressure.
### Dead ends — do not re-try
+2 -1
View File
@@ -44,7 +44,7 @@ is a locator: one row, one clause, no numbers.
| Page | Owns |
|---|---|
| [[Audiobookshelf]] | Audiobookshelf on Dankserver: quadlet shape, the read-only array bind, state on the SSD, the API bootstrap, laws. |
| [[Audiobookshelf]] | Audiobookshelf on Dankserver: quadlet shape, array bind, SSD state, API bootstrap, authentik SSO, public vhost, laws. |
| [[Calibre]] | Calibre is the e-book library manager that backs Calibre-Web. |
| [[Calibre-Web]] | Calibre-Web is a web application that browses, reads, and downloads eBooks from an existing Calibre library database. |
| [[Copyparty]] | Copyparty is a lightweight file server and media streamer that runs as a single Python binary. |
@@ -57,6 +57,7 @@ is a locator: one row, one clause, no numbers.
| [[Media Metadata View]] | The SSD-backed overlay the two media servers read and write: shape, guards, commands, laws, rollback. |
| [[Media Music Relayout]] | Re-filing /mnt/entropy/Music by Jellyfin's tag rows: the census, the album key, the refusals. |
| [[Media Relayout]] | Census, plan and apply the movie, TV, music and audiobook libraries: the gates, the refusals, the rollback. |
| [[Media TV Relayout]] | Re-file TVShows from scene packs into the shape Jellyfin reads: authority order, refusals, the two-series split test. |
| [[Plex]] | Plex Media Server is a **containerized media server on Dankserver**: a podman |
| [[redis]] | The Redis cache and task queue for RomM: quadlet unit, network alias topology, and the backup path. |
| [[RomM]] | RomM (ROM Manager) is a self-hosted ROM library manager and player. |
+6 -4
View File
@@ -10,7 +10,7 @@ const CATEGORY = window.location.pathname.indexOf('/admin') !== -1 ? 'admin' : '
// same renderSection()/renderPanel() below. SECTION_ORDER fixes each page's
// section order; a section with no services renders nothing.
const SECTION_ORDER = {
user: ['Web Apps', 'Media Library Apps', 'AI'],
user: ['Web Apps', 'Media Library Apps', 'AI', 'Work in progress'],
admin: ['Media Library Apps', 'Observability', 'AI', 'Infrastructure'],
};
@@ -79,15 +79,17 @@ function voiceBtn(text, files) {
function miCard(svc, state) {
const m = svc.model;
const art = svc.art || '/img/unit-' + m.unit + '.jpg';
const parts = ['<div class="mi-top">' +
'<img class="mi-art" src="/img/unit-' + esc(m.unit) + '.jpg" alt="' + esc(m.unit) + '">' +
'<img class="mi-art" src="' + esc(art) + '" alt="' + esc(m.unit || svc.name) + '">' +
'<div class="mi-head"><div class="mi-id"><span class="badge ' + state + '">' + BADGE_LABEL[state] + '</span>' +
'<span class="mi-name">' + esc(svc.name) + '</span></div>' +
'<div class="mi-role">' + esc(m.role) + '</div>' +
(m.hf ? '<a class="mi-hf" href="' + esc(m.hf) +
'" target="_blank" rel="noopener">model on huggingface &#8599;</a>' : '') +
'</div></div>',
voiceBtn(m.tagline, m.sfx)
(m.sfx && m.sfx.length ? voiceBtn(m.tagline, m.sfx)
: m.tagline ? '<div class="mi-tag">' + esc(m.tagline) + '</div>' : '')
];
if (m.equiv) parts.push('<div class="mi-equiv">' + esc(m.equiv) + '</div>');
if (m.spec) parts.push('<div class="mi-spec">' + esc(m.spec) + '</div>');
@@ -99,7 +101,7 @@ function miCard(svc, state) {
if (m.warn) parts.push('<details class="mi-warn"><summary>&#9888; privacy note</summary>' +
'<div class="warn-body">' + esc(m.warn) + '</div></details>');
const card = document.createElement('article');
card.className = 'mi-card mi-' + esc(m.unit);
card.className = 'mi-card mi-' + esc(m.unit || 'tool');
card.innerHTML = parts.join('');
return card;
}
-35
View File
@@ -1,35 +0,0 @@
{
"host": "dankserver",
"services": [
{
"name": "Homepage",
"url": "/",
"description": "This page. Password-protected service index.",
"docs": "serverAppConfigs-docs/Infrastructure/Homepage.md"
},
{
"name": "RomM",
"url": "http://romm.franksdank.cloud",
"description": "Retro game manager and player for the ROM collection.",
"docs": "serverAppConfigs-docs/Dankserver/RomM.md"
},
{
"name": "Copyparty",
"url": "http://files.franksdank.cloud",
"description": "High-speed file sharing and streaming server.",
"docs": "serverAppConfigs-docs/Dankserver/Copyparty.md"
},
{
"name": "Calibre-Web",
"url": "http://books.franksdank.cloud",
"description": "E-book library browser and reader.",
"docs": "serverAppConfigs-docs/Dankserver/Calibre-Web.md"
},
{
"name": "Plex",
"url": "http://plex.franksdank.cloud/web/index.html",
"description": "Media server (existing host service, proxied).",
"docs": "serverAppConfigs-docs/Dankserver/Dankserver Stack.md"
}
]
}
+426 -159
View File
@@ -17,191 +17,458 @@
</header>
<main class="docs">
<section id="getting-in">
<h1>Getting in</h1>
<p>Everything here lives behind two doors. The first is <strong>Anubis</strong>, a
quiet bot-check: the first time you visit any guarded page in a session you see a
short "making sure you're not a bot" screen. It runs once. The second door is
<strong>Authentik</strong>, the family single sign-on: log in once and every
guarded service (this homepage, search, chat, media requests, and friends) opens
for you without a second password.</p>
<ul>
<li>Need an account? Ask Frank. Accounts are issued by hand; there is no signup.</li>
<li>Your clearance lives in a cookie. Clearing cookies means re-clearing both doors.</li>
<li>If a page asks you to log in repeatedly, that is a server-side problem — tell Frank, don't brute-force it.</li>
</ul>
<h1>The household guide</h1>
<p class="sub">Everything running on the computers in this house, what it is for,
and which door to knock on. Written for people who did not build it.</p>
<section id="start">
<h2>Start here</h2>
<p class="kicker">Three steps and you are in</p>
<p><strong>1. Open the front door:</strong>
<a href="https://www.franksdank.cloud">https://www.franksdank.cloud</a><br>
It works from anywhere &mdash; phone data, a hotel, a friend's house. The first page you
see is a robot check: your browser does a few seconds of arithmetic so the internet's
scanners can't wander in. Then type the family username and password Frank gave you.
There is no signup button on purpose.</p>
<p><strong>2. Pick what you want to do:</strong></p>
<table>
<tr><th>I want to&hellip;</th><th>Open</th><th>Needs its own login?</th></tr>
<tr><td>watch a film or a show</td><td><a href="https://jelly.franksdank.cloud">Jellyfin</a></td><td>no &mdash; your franksdank.cloud login</td></tr>
<tr><td>watch it on a TV that already has Plex</td><td><a href="https://plex.franksdank.cloud/web">Plex</a></td><td>yes &mdash; a Plex account, Frank invites it</td></tr>
<tr><td>listen to an audiobook</td><td><a href="https://audiobooks.franksdank.cloud/audiobookshelf">Audiobookshelf</a></td><td>no &mdash; the family password</td></tr>
<tr><td>ask a computer a question, privately</td><td><a href="https://ai.franksdank.cloud">AI chat</a></td><td>no</td></tr>
<tr><td>search the web without being followed</td><td><a href="https://search.franksdank.cloud">Search</a></td><td>no</td></tr>
<tr><td>read a news article without the junk</td><td><a href="https://marreta.franksdank.cloud">Marreta</a></td><td>no</td></tr>
<tr><td>grab a file off the big disk</td><td><a href="https://files.franksdank.cloud">Files</a></td><td>no (read-only)</td></tr>
<tr><td>ebooks, retro games, request a film</td><td>not open to the household yet &mdash; <a href="#notopen">see below</a></td><td>&mdash;</td></tr>
</table>
<p><strong>3. On your phone,</strong> open the front door in Safari or Chrome and choose
<em>Add to Home Screen</em>. You get an icon that behaves like an app. Details in
<a href="#phone">On your phone</a>.</p>
<figure>
<img src="/img/onboarding/homepage.webp" alt="The homepage: one tile per service, each with a short description">
<figcaption>The front door itself: one tile per thing the house runs, and a line about what
each is for. The badges say whether it's awake.</figcaption>
</figure>
<div class="toc">
<a href="#doors">The two doors</a>
<a href="#phone">On your phone</a>
<a href="#jellyfin">Jellyfin</a>
<a href="#plex">Plex</a>
<a href="#audiobookshelf">Audiobookshelf</a>
<a href="#notopen">Not open yet</a>
<a href="#files">Files</a>
<a href="#search">Search</a>
<a href="#marreta">Read an article</a>
<a href="#ai">AI chat</a>
<a href="#gitea">Gitea</a>
<a href="#models">The models</a>
<a href="#trouble">When something misbehaves</a>
</div>
</section>
<section id="troubleshooting">
<h1>When something looks broken</h1>
<ul>
<li><strong>A status card says DOWN.</strong> Reload the page once; the checker
re-probes every few minutes. If it stays down, that service really is unhappy — tell Frank.</li>
<li><strong>"status unavailable" on the homepage.</strong> The page couldn't read
its status file. Hard-reload (Ctrl/Cmd-Shift-R).</li>
<li><strong>Connection timeout on any device.</strong> Our names only resolve
on the home network (or the VPN). If you are on guest Wi-Fi, a phone hotspot,
or out of the house, the pages will not load — that is the security working.
Connect to the house network and retry.</li>
<li><strong>The bot-check spins forever.</strong> Let it finish; heavy JS blocking
(privacy extensions) can starve it. Pause the blocker, reload, resume.</li>
</ul>
<section id="doors">
<h2>The two doors</h2>
<p>Almost everything here is behind two checks, one after the other.</p>
<ul>
<li><strong>Anubis</strong> is the bouncer. It makes a browser prove it is a browser
(that robot-check page) and stops the automatic scanners that comb the internet for
open servers. It never asks for a password.</li>
<li><strong>Authentik</strong> is the front desk. It asks who you are, once per device
per few days, and after that you drift from app to app without being stopped again.</li>
</ul>
<p>One account opens: the homepage, search, Marreta, the AI chat, Files, Gitea, and Jellyfin
(there it is typed as username + password, not the automatic sign-in).</p>
<figure>
<img src="/img/onboarding/door-authentik.webp" alt="The sign-in screen: an identification field, then a password field">
<figcaption>The front desk. Your username first, then your password. Nothing here signs
you up for anything; if you don't have an account yet, ask Frank.</figcaption>
</figure>
<p class="key-warn">Two doors keep their own accounts and will not take the family
password: <strong>Plex</strong> (a Plex account, its own e-mail
login) and the e-book door. Frank creates those by hand. <strong>Jellyfin does take the
family login</strong> &mdash; your franksdank.cloud username and password.</p>
<p class="foot-note">Want your own password? It lives at
<a href="https://auth.franksdank.cloud/if/flow/default-user-settings-flow/">account settings</a>
&mdash; the same button you get at the bottom of the homepage.</p>
</section>
<section id="media">
<h1>Media &amp; files</h1>
<ul>
<li><strong>Movies &amp; TV:</strong> request with <a href="https://seerr.franksdank.cloud">Seerr</a>;
watch with <a href="https://plex.franksdank.cloud/web/index.html">Plex</a> or
<a href="https://jelly.franksdank.cloud">Jellyfin</a>. Seerr is the front door —
requests ride the VPN tunnel from there.</li>
<li><strong>Books:</strong> <a href="https://books.franksdank.cloud">Calibre-Web</a> — the e-book library, with its own login.</li>
<li><strong>Games:</strong> <a href="https://romm.franksdank.cloud">RomM</a> — ROM library and emulator collection.</li>
<li><strong>Raw files:</strong> <a href="https://files.franksdank.cloud">Copyparty</a> — direct downloads and shares.</li>
<li><strong>Search:</strong> <a href="https://search.franksdank.cloud">SearXNG</a> — metasearch, VPN-routed, no AI snoop. <details class="key-warn"><summary>&#9888; privacy note</summary><div class="warn-body">Searches are obfuscated, not anonymous: SearXNG hides you from the upstream engines and the VPN tunnel hides the traffic from the LAN &mdash; but the upstream search API keys are tied to Frank&rsquo;s identity. Don&rsquo;t treat it as Tor.</div></details></li>
</ul>
<section id="phone">
<h2>On your phone</h2>
<p><strong>Get the icon.</strong> In the browser: <em>Share</em> (iPhone) or the
<em>&#8942;</em> menu (Android) &rarr; <em>Add to Home Screen</em>. Same trick on a tablet.
There is no app store anything to install for the household doors; they are web pages
wearing a good coat.</p>
<figure>
<img src="/img/onboarding/phone-homepage.webp" alt="The homepage on a phone, one row per service">
<figcaption>The front door on a 5-inch screen. Same tiles, same links, no app.</figcaption>
</figure>
<p><strong>Apps that do have a real client</strong> (better than the web page on a phone):</p>
<table>
<tr><th>What</th><th>Where</th><th>Server address to type</th></tr>
<tr><td>Jellyfin</td><td>Google Play &amp; the App Store (also Fire TV, Roku, most smart TVs)</td><td><code>https://jelly.franksdank.cloud</code></td></tr>
<tr><td>Plex</td><td>on nearly every TV ever sold</td><td>nothing &mdash; sign in to your Plex account</td></tr>
<tr><td>Audiobookshelf</td><td>Google Play (Android, solid). The iPhone build is a beta with a limited number of seats &mdash; use the browser, or SoundLeaf / ShelfPlayer</td><td><code>https://audiobooks.franksdank.cloud/audiobookshelf</code></td></tr>
</table>
<p><strong>The one address that only works at home.</strong> The raw file shelf also speaks
WebDAV on <code>http://192.168.0.235:3923/</code> &mdash; that's what a real file-manager app
mounts. Two catches, both measured: it is <em>http</em>, not https (the phone will warn you;
it is fine on the house Wi-Fi), and it answers only on the house network. From elsewhere the
public door hands a file app the robot-check page instead of files, because a file app can't
solve it. Off-site, use the browser.</p>
<ul>
<li>iPhone: the built-in Files app does <em>not</em> do WebDAV. <em>FileBrowser</em> or
<em>Owlfiles</em> do.</li>
<li>Android: <em>Cx File Explorer</em>, <em>Material Files</em>, <em>Solid Explorer</em> all do.</li>
</ul>
</section>
<section id="chat">
<h1>Talking to the AI</h1>
<p>The everyday front door is <a href="https://ai.franksdank.cloud">Open&nbsp;WebUI</a>,
the family chat app. Open the address, click <strong>Continue with authentik</strong>,
and use the same single sign-on as everything else — no separate password. Your
conversations are saved to your account, so history follows you between devices.</p>
<img src="img/openwebui-chat.png" alt="Open WebUI chat screen with a question and the AI's answer">
<p>Below the text box, the model box on the right chooses which model answers you.
Family accounts get <code>frank-zealot</code> — the fleet's fast all-rounder,
which also sees images: drag a photo or screenshot straight into the chat box.
Frank's account can additionally reach the bigger models.</p>
<img src="img/openwebui-model-picker.png" alt="The Open WebUI welcome screen with the family model selected in the model box">
<p>Open WebUI keeps your chat history; the raw model endpoints (used by developer
tools, see <a href="#api-access">API access</a>) do not.</p>
<section id="jellyfin">
<h2>Jellyfin &mdash; films and TV</h2>
<p><a href="https://jelly.franksdank.cloud">jelly.franksdank.cloud</a> &mdash; free, no
account of any company's, and the posters and episode guides are all filled in.</p>
<h3>Getting in</h3>
<ul>
<li>Pick your name. There is no list of names on the login screen &mdash; type yours.</li>
<li>Type your <strong>franksdank.cloud password</strong> &mdash; the same one that opens the
homepage, search and the AI chat. There is no separate Jellyfin password.</li>
<li>Refused, but the same password opens the homepage? Your name hasn't been given library
access yet &mdash; ask Frank, it's a one-line change on his side.</li>
<li>No name yet? Ask Frank to make one. There is no self-service signup, on purpose.</li>
</ul>
<figure>
<img src="/img/onboarding/jellyfin-login.webp" alt="The Jellyfin login screen: a username field and a password field">
<figcaption>The empty login. Username, password, <em>Sign in</em>. Nothing to register.</figcaption>
</figure>
<h3>The first ten minutes</h3>
<ul>
<li>Home shows <em>Continue watching</em> and what's new. Press a poster; it starts.</li>
<li>The <strong>&#8942;</strong> menu on a poster is where the useful things live: mark
watched, add to a playlist, download it to the device.</li>
<li>On a TV the app asks for a <em>Server address</em>: <code>https://jelly.franksdank.cloud</code>.</li>
<li>Settings &rarr; theme: the big-screen look is installed for the living-room box.
Themes are per device, so change it only on the one you're sitting in front of.</li>
</ul>
<h3>If it misbehaves</h3>
<ul>
<li>It rejects a password that works everywhere else? Your name isn't on the library-access
list yet &mdash; ask Frank.</li>
<li>A film with a blank poster is missing artwork, not missing files &mdash; Frank can refresh it.</li>
<li>Buffering away from home usually means the pipe out of the house, not the server.</li>
</ul>
</section>
<section id="plex">
<h2>Plex &mdash; the same shelf, different shop window</h2>
<p><a href="https://plex.franksdank.cloud/web">plex.franksdank.cloud/web</a> bounces you to
Plex's own sign-in, and that is correct: Plex is closed-source and insists on a Plex account.</p>
<figure>
<img src="/img/onboarding/plex-login.webp" alt="Plex's own sign-in page on plex.tv">
<figcaption>Plex keeps its own front desk. You need a Plex account (an e-mail address is
enough); Frank invites it into the library.</figcaption>
</figure>
<ul>
<li>Jellyfin and Plex read <strong>the same shelf of files</strong>. The same films are in
both; the artwork, watch-state and profiles are separate.</li>
<li>Choose Plex when your TV already has the Plex app on it, or you want Plex's trailers
and music. Otherwise Jellyfin is the one the household uses.</li>
<li>Need a Plex account of your own? Ask Frank to invite it; he can also tell you which of
the two has the thing you're looking for.</li>
</ul>
</section>
<section id="audiobookshelf">
<h2>Audiobookshelf &mdash; audiobooks and podcasts</h2>
<p><a href="https://audiobooks.franksdank.cloud/audiobookshelf">audiobooks.franksdank.cloud/audiobookshelf</a>
&mdash; the family password works here.</p>
<figure>
<img src="/img/onboarding/audiobookshelf-login.webp" alt="The Audiobookshelf login page with username and password fields">
<figcaption>Two ways in: the password Frank gave you, or <em>Sign in with Authentik</em>,
which is the same family account.</figcaption>
</figure>
<ul>
<li>Your place in every book follows you between phone, laptop and the kitchen tablet.</li>
<li>On the phone the Android app is the best version: it will do <em>downloads for offline
listening</em>. A web page can't play offline; if you need it for a flight, use an app.</li>
<li>iPhone: the official build is a TestFlight beta with a small number of seats, so either
use the browser + <em>Add to Home Screen</em>, or a third-party client
(<em>SoundLeaf</em>, <em>ShelfPlayer</em>).</li>
<li>Adding the server in any app: <code>https://audiobooks.franksdank.cloud/audiobookshelf</code>.</li>
</ul>
<figure>
<img src="/img/onboarding/audiobookshelf-library.webp" alt="An empty audiobook library with a scan button">
<figcaption>An empty library. Books appear when Frank scans, and then they're yours to keep playing.</figcaption>
</figure>
</section>
<section id="notopen">
<h2>Three doors that are not open to the household yet</h2>
<ul>
<li><strong>Calibre-Web</strong> (e-books) &mdash;
<a href="https://books.franksdank.cloud">books.franksdank.cloud</a>. Its own login, and
Frank hasn't decided who gets in. Ask.</li>
<li><strong>RomM</strong> (retro games, plays them in the browser) &mdash;
<a href="https://romm.franksdank.cloud">romm.franksdank.cloud</a>. Needs its own account,
and it is not open to family users yet. Ask.</li>
<li><strong>Seerr</strong> (the "would you watch this?" request box) &mdash;
<a href="https://seerr.franksdank.cloud">seerr.franksdank.cloud</a>. Not open yet: tell
Frank what you want to watch and he'll add it.</li>
</ul>
<figure>
<img src="/img/onboarding/calibre-login.webp" alt="The Calibre-Web login page">
<figcaption>What a closed door looks like: a login that wants credentials of its own.</figcaption>
</figure>
</section>
<section id="files">
<h2>Files &mdash; the raw shelf</h2>
<p><a href="https://files.franksdank.cloud">files.franksdank.cloud</a> is the honest view of
the big disk: folders, dates, sizes, no cleverness. <strong>Read-only</strong> for the
household &mdash; you can take things out, nothing you do here deletes anything.</p>
<figure>
<img src="/img/onboarding/copyparty.webp" alt="A folder listing with sizes, dates and a download link on each row">
<figcaption>Each row has a download link; the <em>zip</em> link at the bottom takes the
whole folder as one archive. <em>control-panel</em> and the <em>up</em> link navigate.</figcaption>
</figure>
<ul>
<li>Good for: a photo you want the original of, a document, a game ROM, a song to put on another device.</li>
<li>Bad for: watching video (the player is the wrong tool &mdash; use Jellyfin) and for
saving your own work (it can't write).</li>
<li>Works in any browser, on any machine, from anywhere.</li>
</ul>
</section>
<section id="search">
<h2>Search &mdash; SearXNG</h2>
<p><a href="https://search.franksdank.cloud">search.franksdank.cloud</a> asks a lot of search
engines at once and merges the answers, then leaves no trail behind. Outbound requests ride
the house VPN.</p>
<figure>
<img src="/img/onboarding/search-results.webp" alt="Search results merged from many engines, with engine checkboxes on the left">
<figcaption>One question, many engines. The boxes on the left choose which engines answer.</figcaption>
</figure>
<ul>
<li>Make it your browser's search engine: browser settings &rarr; search &rarr; add a custom
engine with <code>https://search.franksdank.cloud/search?q=%s</code>.</li>
<li>On a phone there is no app; bookmark it or put the homepage on your home screen.</li>
</ul>
<figure>
<img src="/img/onboarding/phone-search.webp" alt="A search result page on a phone">
<figcaption>Same results, thumb-sized.</figcaption>
</figure>
<details class="key-warn">
<summary>The privacy catch worth knowing</summary>
<div class="warn-body">
<p>This hides your question from Google, Bing and the rest &mdash; it does not hide it from
<em>this house</em>. Queries are logged on this server so Frank can tell a broken search
from a slow one. And because results are fetched from here, the search engines see one
house address, not your phone: your habits don't get stitched to your face. If you need a
genuinely anonymous search, that's Tor or a paid service, not this.</p>
</div>
</details>
</section>
<section id="marreta">
<h2>Read an article without the junk &mdash; Marreta</h2>
<p><a href="https://marreta.franksdank.cloud">marreta.franksdank.cloud</a>. Paste a link, get
the article: no cookie banner, no video autoplay, no sixteen sidebars. It also runs an RSS
reader once Frank feeds it news.</p>
<figure>
<img src="/img/onboarding/marreta.webp" alt="Marreta's empty home page with a URL box and an Add as app hint">
<figcaption>Paste the link. The app itself offers <em>Add as app</em> in its menu, which is
the cleanest way to keep it on a phone.</figcaption>
</figure>
<ul>
<li>Some pages with hard paywalls still come back thin. Nothing in the house can fix that.</li>
<li>A white screen after pasting? Pause your ad blocker for this site &mdash; the app warns
about exactly that.</li>
</ul>
</section>
<section id="ai">
<h2>AI chat &mdash; Open WebUI</h2>
<p><a href="https://ai.franksdank.cloud">ai.franksdank.cloud</a>. The family password, or
<em>Continue with authentik</em> on the login card, which is the same thing.</p>
<figure>
<img src="/img/onboarding/openwebui-login.webp" alt="The Open WebUI welcome card with a Continue with authentik button">
<figcaption>Press <em>Continue with authentik</em> and the family login does the rest.</figcaption>
</figure>
<ul>
<li>Nothing you type here goes to a company's server. It goes to the computers in this house.</li>
<li>Install it as an app: in Chrome or Edge, the menu &rarr; <em>Install site as an app</em>;
in Safari, share &rarr; <em>Add to Home Screen</em>.</li>
<li>Conversations are kept per account, so your history is yours and Frank's is Frank's.</li>
</ul>
<h3>Which model do I pick?</h3>
<p>You are offered all three, and the app opens on <code>frank-dragoon</code>. Leave it there
unless you have a reason. Short version (longer version in <a href="#models">The models</a>):</p>
<ul>
<li><code>frank-dragoon</code> &mdash; fast, sees pictures, doesn't lecture. Good default.</li>
<li><code>frank-archon</code> &mdash; the big careful one on the two-card server. Slower per
answer, best at long and difficult work. Polite by construction.</li>
<li><code>frank-zealot</code> &mdash; small, quick, holds a very long conversation. Also
doesn't lecture.</li>
</ul>
<p class="key-warn">Two of the three will answer almost anything, including adult and dangerous
material, without a warning. That is deliberate for Frank's work, not a toy for a child:
conversations stay on the house machines and are not visible to Frank's phone. If that isn't
the setup you want for a kid, keep them off this door and ask Frank about a filtered one.</p>
</section>
<section id="gitea">
<h2>Gitea &mdash; Frank's workshop</h2>
<p><a href="https://gitea.franksdank.cloud">gitea.franksdank.cloud</a> holds the code that runs
this house: the configs, the deploy scripts, the tests. It has its own accounts (the top right
offers <em>Register</em>, or <em>Sign in with OpenID</em> to ride the family login), but it is a
workshop, not a household door. You very likely don't need it.</p>
<figure>
<img src="/img/onboarding/gitea-signin.webp" alt="The Gitea sign-in form with a password box and a Sign in with OpenID button">
<figcaption>If you ever do need it: <em>Register</em> for its own account, or
<em>Sign in with OpenID</em> for the family one.</figcaption>
</figure>
</section>
<section id="models">
<h1>The running models</h1>
<p>Three models serve the family, named for the Protoss units of StarCraft.
The homepage "Model Information" panel shows whether each one is up right now —
and plays the unit's voice if you click its quoted line.</p>
<h2>The models</h2>
<p class="kicker">What's actually answering you</p>
<p>Three machines in this house do the thinking. None of them phone a company. Each is a
different trade-off of speed, memory and how much it can hold in its head at once; the chat
app above is just a window onto them.</p>
<table>
<tr><th>Name</th><th>Runs on</th><th>Good at</th><th>Notes</th></tr>
<tr>
<td><code>frank-dragoon</code></td>
<td>the Strix Halo machine's built-in graphics</td>
<td>fast answers, long chats, reading pictures, tools</td>
<td>two conversations at a time; abliterated, so no refusals</td>
</tr>
<tr>
<td><code>frank-archon</code></td>
<td>the media server, two big graphics cards</td>
<td>the hardest work: long code, careful reasoning, big documents</td>
<td>four conversations at a time; keeps its guardrails</td>
</tr>
<tr>
<td><code>frank-zealot</code></td>
<td>the RTX 3090 in the corner</td>
<td>very long conversations, cheap and quick; it is what Frank's own coding agent runs on</td>
<td>three conversations at 200,000 tokens each; no refusals</td>
</tr>
</table>
<p class="foot-note">The exact weights: <code>frank-dragoon</code> is a halogen conversion of
Qwen3.8-Flash-Next v2, abliterated
(<a href="https://huggingface.co/Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated">Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated</a>);
<code>frank-archon</code> is the official Qwen3.8-Flash-Next re-packed for that engine
(<a href="https://huggingface.co/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe">StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe</a>);
<code>frank-zealot</code> is Ternary-Bonsai-2-9B in a 2-bit packing
(<a href="https://huggingface.co/prism-ml/Ternary-Bonsai-2-9B-gguf">prism-ml/Ternary-Bonsai-2-9B-gguf</a>)
with a small adapter fitted that removes its refusal reflex. All three accept pictures. Ask
Frank before pointing a script at one of them directly; the address is on the homepage.</p>
<p class="foot-note">Naming: the Open&nbsp;WebUI picker and the raw API show
exactly the Protoss names used on this page (renamed 2026-10-09). The name you
read here is the name you type into a config file.</p>
<h2>frank-zealot — the fast utility model</h2>
<ul>
<li><strong>Model:</strong> <a href="https://huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf" target="_blank" rel="noopener">Ternary-Bonsai-2-27B</a> (PQ2_0 ternary weights) on the RTX&nbsp;3090 — roughly 77 tokens per second.</li>
<li><strong>Equivalent:</strong> last year's frontier level (2025).</li>
<li><strong>Capacity:</strong> 200k-token context, two submissions at a time; sees images.</li>
<li><strong>Inputs:</strong> text, image.</li>
<li><strong>Best for:</strong> code editor autocomplete, agentic search, RAG, research subagents, translation, roleplay, and summarization — any small task that doesn't need much intelligence, and anything with a picture. This is the model Open&nbsp;WebUI offers to family accounts.</li>
<li><strong>Compaction:</strong> if your agent harness supports setting a compaction model, set this one. A compaction job severely slows the bigger models for far longer than it slows this one.</li>
<li><strong>Supervision:</strong> requires tight human supervision.</li>
<li><strong>Warning:</strong> <details class="key-warn"><summary>&#9888; censored base model</summary><div class="warn-body">Base-alignment (censored) model: refusals are possible on sensitive topics, and any model can still be steered into risky output. Uncensored work belongs on frank-dragoon. Take this warning seriously &mdash; don&rsquo;t ruin this server for everybody.</div></details></li>
</ul>
<h2>frank-dragoon — deep reasoning</h2>
<ul>
<li><strong>Model:</strong> <a href="https://huggingface.co/Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated" target="_blank" rel="noopener">Qwen3.8-Flash-Next v2 native abliterated</a> (halogen flash-server, ~62&nbsp;GiB) on the Strix Halo iGPU &mdash; the base checkpoint itself, converted to the engine&rsquo;s native format and censorship-removed, thinking before it answers.</li>
<li><strong>Equivalent:</strong> Claude Sonnet 5 / OpenAI-Luna (Summer 2026 frontier intelligence).</li>
<li><strong>Capacity:</strong> 200k-token context, two submissions at a time; sees images.</li>
<li><strong>Inputs:</strong> text, image.</li>
<li><strong>Best for:</strong> complex agentic tasks, programming, system administration, and "claw"-style personal agents.</li>
<li><strong>Supervision:</strong> safe for autonomous, low-supervision use.</li>
<li><strong>Warning:</strong> <details class="key-warn"><summary>&#9888; uncensored model</summary><div class="warn-body">Uncensored model: output can include adult content, content that describes harmful, hateful, or illegal acts, content concerning &ldquo;sensitive offensive cybersecurity topics&rdquo;, and other output that may carry severe legal risk. Take this warning seriously &mdash; don&rsquo;t ruin this server for everybody.</div></details></li>
<h2>frank-archon — the big one</h2>
<ul>
<li><strong>Model:</strong> our own build — a GPTQ-w4 quant of <a href="https://huggingface.co/frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700" target="_blank" rel="noopener">Swift1.5-Qwen3.8-Flash-Next</a>, quantized and tuned in-house (radiance, ~114&nbsp;GiB) on twin AMD R9700s with more than 200&nbsp;GB of system RAM — the biggest brain on the network and the fastest runtime.</li>
<li><strong>Equivalent:</strong> Claude Opus 4.8 / OpenAI-Terra (Summer 2026 frontier intelligence).</li>
<li><strong>Capacity:</strong> 200k-token context, three submissions at a time; sees images.</li>
<li><strong>Inputs:</strong> text, image.</li>
<li><strong>Best for:</strong> software engineering, system administration, and instruction tasks. <strong>&#9888; Ask Frank for permission before use</strong> &mdash; this is his primary model; he uses it more than anything else on the fleet.</li>
</ul>
<p class="foot-note">Which one to pick? Small utility work, autocomplete, RAG,
programming, and personal agents: <code>frank-dragoon</code>. Software
engineering, sysadmin, and instruction tasks: <code>frank-archon</code> (ask
Frank first). Every name on this page is the id the Open&nbsp;WebUI picker and
the raw API answer to.</p>
<h3>And the voices</h3>
<p>Two speech models live in this house, and they are different animals. One of them
speaks as a <em>specific person</em>; the other is a pack of stock voices that ships inside
the tools you already use.</p>
<table>
<tr><th>Name</th><th>Runs on</th><th>Good at</th><th>Notes</th></tr>
<tr>
<td><code>frank-tts</code></td>
<td>the Strix Halo machine's built-in graphics, sharing the card with <code>frank-dragoon</code></td>
<td>cloning: it speaks as one particular voice, chosen by name</td>
<td>five voices today &mdash; <code>dr.kern</code>, <code>garrickhagon</code>,
<code>jeffhays</code>, <code>vaati</code>, <code>wilwheaton</code>; ask for one by name and
you get it. No key, but it is a house-hardware door: it answers on the house network only</td>
</tr>
<tr>
<td><strong>Kokoro</strong></td>
<td>your own browser, or Frank's own terminal &mdash; no server in the loop</td>
<td>instant plain narration, quick and cheap</td>
<td>about two dozen preset voices with coded names (<code>af_heart</code>,
<code>bm_george</code>&hellip;). It cannot clone, and it will never answer to the five names
above</td>
</tr>
</table>
<ul>
<li><strong>The AI chat's speaker button is Kokoro.</strong> That is why it offers a
different list of names, and why it starts talking a moment before anything on the server
could. It speaks in your browser, on your machine.</li>
<li><strong>The podcast maker uses the clones.</strong> An episode's two hosts are
<code>wilwheaton</code> and <code>vaati</code>, rendered by <code>frank-tts</code>. That door
is Frank's for now.</li>
<li><strong>Frank's coding agent narrates through the clones too</strong> &mdash; the default
voice is <code>wilwheaton</code>.</li>
<li><strong>Want your own voice in the list?</strong> Ask Frank. It needs about twenty
seconds of you talking plainly, once; the voice then carries your name.</li>
</ul>
<p class="foot-note">The exact weights: <code>frank-tts</code> is a Qwen3 talker
(1.7&nbsp;billion parameters, 4-bit) plus its 12&nbsp;Hz audio codec, driven by
<code>qwentts.cpp</code>; Kokoro is a much smaller 82-million-parameter model whose voices are
baked in. A cloned voice is not a setting &mdash; it is a directory of reference audio on the
server, which is why adding one is a fifteen-minute job and not a checkbox. Programmers on the
house network: <code>POST http://192.168.0.143:7772/v1/audio/speech</code> with
<code>{"input": "&hellip;", "voice": "vaati"}</code> gets you WAV or MP3, and
<a href="http://192.168.0.143:7772/v1/audio/voices"><code>/v1/audio/voices</code></a> answers
the live list, so this page can never lie to you for long.</p>
</section>
<section id="api-access">
<h1>API access — the AI on your own laptop</h1>
<p>Developer assistants like <strong>Opencode</strong> and <strong>Pi</strong> are
chat programs that run on your own computer and can read your files, run commands,
and edit code while they talk. They are all built to speak the same standard
language as the big commercial products (the "OpenAI-compatible" API), and our
server speaks it too. You give the program three things — an address, a personal
key, and a model name — and your laptop thinks through our hardware.</p>
<h2>Step 1 — get your personal key</h2>
<ol>
<li>Open <a href="https://ai.franksdank.cloud">ai.franksdank.cloud</a> and log in.</li>
<li>Click your circle (bottom-left corner) → <strong>Settings</strong> → <strong>Account</strong>.</li>
<li>Scroll to <strong>API keys</strong> → press <strong>Show</strong> (it flips to
<strong>Hide</strong>) — the key appears as a row of dots.</li>
<li>Press the <strong>copy icon</strong> beside the key (the eye previews it;
the <strong>⋯</strong> menu rotates it later). The key looks like <code>sk-3f2a…</code>.</li>
</ol>
<img src="img/openwebui-api-keys.png" alt="The Open WebUI Account settings page, scrolled to the API keys section with the masked key, preview, copy and rotate controls">
<p class="key-warn">The key is a password. Anyone who has it can use our AI as
you. Paste it only into the program's config file — never into a shared chat,
a screenshot, or a git repository.</p>
<h2>Step 2 — the three details every tool asks for</h2>
<ul>
<li><strong>Base URL:</strong> <code>https://ai.franksdank.cloud/api/v1</code></li>
<li><strong>Model:</strong> <code>frank-zealot</code> — the family utility model (this is the id the API expects)</li>
<li><strong>API key:</strong> the <code>sk-…</code> from step 1 — sent as the header <code>Authorization: Bearer sk-…</code></li>
</ul>
<p>Proof that it works, from any terminal (replace the key):</p>
<pre><code>curl https://ai.franksdank.cloud/api/v1/chat/completions \
<h2>Let your own tools use it</h2>
<p class="kicker">For the household's programmers</p>
<p>Frank can hand you an API key. It works on one door
&mdash; <code>https://ai.franksdank.cloud/api/v1</code> &mdash; and speaks the plain OpenAI
dialect, so anything that can talk to a hosted model can talk to the house.</p>
<pre><code>curl https://ai.franksdank.cloud/api/v1/chat/completions \
-H "Authorization: Bearer PASTE-YOUR-KEY" \
-H "Content-Type: application/json" \
-d '{"model": "frank-zealot", "messages": [{"role": "user", "content": "Hello!"}]}'</code></pre>
<h2>Step 3a — Opencode</h2>
<p>Install once (<code>npm install -g opencode-ai</code>), then create the file
<code>~/.config/opencode/opencode.json</code> with:</p>
<pre><code>{
"$schema": "https://opencode.ai/config.json",
"provider": {
"frank": {
"npm": "@ai-sdk/openai-compatible",
"name": "Frank fleet",
"options": {
"baseURL": "https://ai.franksdank.cloud/api/v1",
"apiKey": "PASTE-YOUR-KEY"
},
"models": {
"frank-zealot": { "name": "frank-zealot" }
}
}
}
}</code></pre>
<p>Start <code>opencode</code> and pick the model with <code>/model</code> —
it appears as <code>frank/frank-zealot</code>.</p>
<h2>Step 3b — Pi</h2>
<p>Create the file <code>~/.pi/agent/models.json</code> with:</p>
<pre><code>{
-d '{
"model": "frank-dragoon",
"messages": [{"role": "user", "content": "What is on the shelf?"}]
}'</code></pre>
<p>For <a href="https://github.com/badlogic/pi-mono">Pi</a>, drop this in
<code>models.json</code>:</p>
<pre><code>{
"providers": {
"frank": {
"baseUrl": "https://ai.franksdank.cloud/api/v1",
"api": "openai-completions",
"apiKey": "PASTE-YOUR-KEY",
"models": [
{ "id": "frank-zealot" }
{ "id": "frank-dragoon" }
]
}
}
}</code></pre>
<p>Then start Pi with <code>pi --model frank/frank-zealot</code>.</p>
<p>Then start Pi with <code>pi --model frank/frank-dragoon</code>.</p>
<p class="foot-note">The key is for the house AI only &mdash; it is not your Authentik password,
and losing it costs nothing beyond asking Frank to rotate it. Want a different model than the
one you were given? Ask Frank to widen the key.</p>
</section>
<p class="foot-note">The key works only for the family AI — it is not your
Authentik password, and losing it costs nothing beyond asking Frank to rotate it.
Want a bigger model (<code>frank-archon</code>, <code>frank-dragoon</code>)? Ask Frank to raise your access.</p>
<section id="trouble">
<h2>When something misbehaves</h2>
<ul>
<li><strong>The robot check page keeps coming back.</strong> Your browser is blocking the
scripts it needs (or an extension is). Allow scripts for that site, or try another browser.
Still looping? Ask Frank &mdash; the guard can be told you're a person.</li>
<li><strong>"Invalid password" on the front desk.</strong> Right password, wrong door: the
family login does not open Plex or the e-book door. Jellyfin does take it, but only for the
names Frank has given library access.</li>
<li><strong>The AI is slow.</strong> Each model only holds a couple of conversations at once
(two, three or four, depending which). It is a queue, not a crash &mdash; or Frank is
mid-conversation; the big one is on the same machine as the films.</li>
<li><strong>The AI invents something confidently.</strong> All three are small models living
on house hardware; they will make things up rather than say "I don't know". Check anything
that matters.</li>
<li><strong>A film is missing.</strong> It is probably not on the shelf yet, or it landed
yesterday and the library hasn't been told. Say so and Frank will scan.</li>
<li><strong>A file app can't reach the shelf from outside the house.</strong> Expected: the
public door is browser-shaped. Use the browser, or mount the home address on the house Wi-Fi.</li>
<li><strong>Something is genuinely down.</strong> The homepage badge says
<span style="color: var(--down)">DOWN</span> &mdash; tell Frank which one. Screenshots help.</li>
</ul>
<p class="foot-note">Anything missing, wrong, or written like a manual instead of a person:
that's a bug in this page. Tell Frank, or tell the machine that wrote it.</p>
</section>
</main>
@@ -0,0 +1 @@
<svg fill="#82612C" role="img" viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg"><title>Audiobookshelf</title><path d="M12 0A12 12 0 0 0 0 12a12 12 0 0 0 12 12 12 12 0 0 0 12-12A12 12 0 0 0 12 0Zm-.023.402A11.598 11.598 0 0 1 23.575 12a11.598 11.598 0 0 1-11.598 11.598A11.598 11.598 0 0 1 .378 12 11.598 11.598 0 0 1 11.977.402Zm0 1.776a7.093 7.093 0 0 0-7.092 7.093v1.536a6.395 6.395 0 0 0-.439.33.35.35 0 0 0-.126.27v1.84a.36.36 0 0 0 .126.272c.22.182.722.564 1.504.956v.179c0 .483.31.873.694.873.384 0 .694-.392.694-.873v-4.415c0-.483-.31-.873-.694-.873-.369 0-.67.359-.694.812h-.002v-.91a6.027 6.027 0 1 1 12.054.003v.91c-.025-.454-.326-.813-.695-.813-.384 0-.694.391-.694.873v4.415c0 .483.31.873.694.873.384 0 .695-.392.695-.873v-.179a7.964 7.964 0 0 0 1.503-.956.35.35 0 0 0 .126-.272v-1.843a.342.342 0 0 0-.124-.27 5.932 5.932 0 0 0-.438-.329V9.271a7.093 7.093 0 0 0-7.092-7.093zm-3.34 5.548a.84.84 0 0 0-.84.84v9.405c0 .464.376.84.84.84h.866a.84.84 0 0 0 .84-.84V8.566a.84.84 0 0 0-.84-.84Zm2.905 0a.84.84 0 0 0-.84.84v9.405c0 .464.377.84.84.84h.867a.84.84 0 0 0 .84-.84V8.566a.84.84 0 0 0-.84-.84zm2.908 0a.84.84 0 0 0-.84.84v9.405c0 .464.376.84.84.84h.867a.84.84 0 0 0 .84-.84V8.566a.84.84 0 0 0-.84-.84zM8.112 9.983h1.915v.2H8.112Zm2.906 0h1.915v.2h-1.915Zm2.908 0h1.915v.2h-1.915zm-7.58 9.119a.633.633 0 0 0 0 1.265h11.26a.632.632 0 0 0 0-1.265z"/></svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

-1
View File
@@ -1 +0,0 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 96 96"><polygon points="48,6 86,27 86,69 48,90 10,69 10,27" fill="#1c1300" stroke="#3fc1c9" stroke-width="3"/><text x="48" y="60" font-family="monospace" font-size="30" font-weight="bold" fill="#3fc1c9" text-anchor="middle">AB</text></svg>

Before

Width:  |  Height:  |  Size: 293 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 38 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 4.0 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 87 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 10 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 48 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 49 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 71 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 43 KiB

+32 -23
View File
@@ -57,8 +57,8 @@
{
"name": "Calibre-Web",
"category": "user",
"section": "Media Library Apps",
"description": "E-book library. Work in progress: not open to family users yet.",
"section": "Work in progress",
"description": "E-book library. Not open to family users yet — ask Frank.",
"url": "",
"docs": "",
"probe": "192.168.0.235:8083",
@@ -70,8 +70,8 @@
{
"name": "RomM",
"category": "user",
"section": "Media Library Apps",
"description": "ROM and emulator library. Work in progress: not open to family users yet.",
"section": "Work in progress",
"description": "ROM and emulator library. Not open to family users yet — ask Frank.",
"url": "",
"docs": "",
"probe": "192.168.0.235:8085",
@@ -110,20 +110,20 @@
"name": "Jellyfin",
"category": "user",
"section": "Media Library Apps",
"description": "Open source media server. Work in progress: not open to family users yet.",
"url": "",
"description": "Open source media server for the film and TV library. Sign in with your franksdank.cloud username and password — the same login as the rest of the house; ask Frank if yours isn't in yet.",
"url": "http://jelly.franksdank.cloud",
"docs": "",
"probe": "192.168.0.235:8096",
"method": "http",
"wip": "down",
"showLink": false,
"wip": false,
"showLink": true,
"art": "/img/apps/jellyfin.svg"
},
{
"name": "Seerr",
"category": "user",
"section": "Media Library Apps",
"description": "Download requests. Work in progress: not open to family users yet.",
"section": "Work in progress",
"description": "Download requests. Not open to family users yet — ask Frank.",
"url": "",
"docs": "",
"probe": "127.0.0.1:5055",
@@ -136,7 +136,7 @@
"name": "Marreta",
"category": "user",
"section": "Web Apps",
"description": "Clean, ad-free article reader. Paste any URL, get just the article.",
"description": "Paywall sledgehammer: hand it an article URL and it will attempt to smash its paywall. Note: does not work on all sites.",
"url": "http://marreta.franksdank.cloud",
"docs": "",
"probe": "127.0.0.1:8082",
@@ -314,7 +314,7 @@
"zealot-honor.ogg"
],
"equiv": "Equivalent to last year's frontier level (2025).",
"spec": "Ternary-Bonsai-2-9B (PQ2_0 ternary weights, MTP head in the file) on an NVIDIA RTX 3090 — ~167 tokens/s; 262k-token context, three submissions at a time; sees images.",
"spec": "Ternary-Bonsai-2-9B (PQ2_0 ternary weights, MTP head in the file) with the uncensoring adapter fitted at load, on an NVIDIA RTX 3090 — 200k-token conversations, three at a time; sees images.",
"inputs": [
"text",
"image"
@@ -327,7 +327,7 @@
]
},
"supervision": "Requires tight human supervision.",
"warn": "Base-alignment (censored) model: refusals are possible on sensitive topics, and any model can still be steered into risky output. Take this warning seriously — don’t ruin this server for everybody.",
"warn": "Uncensored model — a refusal-removal adapter is fitted at load, so refusals are not a safeguard here: output can include adult content, content that describes harmful, hateful, or illegal acts, and other output that may carry severe legal risk. Take this warning seriously — don’t ruin this server for everybody.",
"compaction": "If your agent harness lets you set a compaction model, set this one: a compaction job on the bigger models blocks them for far longer."
},
"url": "",
@@ -350,7 +350,7 @@
"dragoon-tagline.ogg"
],
"equiv": "Equivalent to Claude Sonnet 5 / OpenAI-Luna (Summer 2026 frontier intelligence).",
"spec": "Qwen3.8-Flash-Next v2 native abliterated (halogen flash-server, ~62 GiB) on the Strix Halo iGPU — 200k-token context, two submissions at a time; sees images.",
"spec": "Qwen3.8-Flash-Next v2 native abliterated (halogen flash-server, ~62 GiB) on the Strix Halo iGPU — 200k-token conversations, two at a time; sees images.",
"inputs": [
"text",
"image"
@@ -378,14 +378,14 @@
"section": "AI",
"model": {
"unit": "archon",
"role": "Frank's strongest and fastest model runtime",
"role": "Frank's strongest model",
"hf": "https://huggingface.co/StillDeadcode/qwen3.8-next-flash-fp8-iq4r-moe",
"tagline": "...Power Overwhelming...",
"sfx": [
"archon-power.ogg"
],
"equiv": "Equivalent to Claude Opus 4.8 / OpenAI-Terra (Summer 2026 frontier intelligence).",
"spec": "The engine author's published quant: 4-bit codebook experts + int8 trunk of Qwen3.8-Flash-Next (radiance, ~114 GiB) on twin AMD R9700s with more than 200 GB of system RAM — 200k-token context, three submissions at a time; sees images.",
"spec": "The engine author's published quant: 4-bit codebook experts + int8 trunk of the official (censored) Qwen3.8-Flash-Next (radiance, ~114 GiB) on twin AMD R9700s with more than 200 GB of system RAM — 200k-token conversations, four at a time; sees images.",
"inputs": [
"text",
"image"
@@ -513,16 +513,16 @@
},
{
"name": "Audiobookshelf",
"category": "admin",
"category": "user",
"section": "Media Library Apps",
"description": "Audiobook server for the entropy library: its own SQLite library, its own login, per-listener positions.",
"url": "http://192.168.0.235:8000",
"description": "Audiobook server for the library: sign in with your Authentik account and every listener keeps their own position.",
"url": "http://audiobooks.franksdank.cloud/audiobookshelf/",
"docs": "",
"probe": "192.168.0.235:8000",
"method": "http",
"wip": false,
"showLink": true,
"art": "/img/apps/mono-ab.svg"
"art": "/img/apps/audiobookshelf.svg"
},
{
"name": "Postgres",
@@ -552,11 +552,20 @@
},
{
"name": "frank-tts",
"category": "admin",
"category": "user",
"section": "AI",
"description": "The fleet's cloned-voice speech endpoint: POST /v1/audio/speech with voice=<name>, audio back in that voice. Published on 7772; the engine behind it (frank-tts-engine) is unpublished on purpose. It is also the harness speech role.",
"model": {
"role": "The fleet's voice — it speaks, it does not chat",
"hf": "https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF",
"tagline": "Your library, read aloud in a voice you know.",
"spec": "Qwen3-TTS 1.7 B base talker (Q4_K_M) with its 12 Hz tokenizer, run by qwentts.cpp on the gfx1151 iGPU. The frank-tts door answers POST /v1/audio/speech with voice=<name> and hands back audio in that voice.",
"inputs": ["text"],
"use": "Narration: Open Notebook episodes, the operator harness's spoken output, and anything that should sound like a person instead of a text-to-speech robot.",
"permission": "Ask Frank for a voice — each pack is built from that person's own recording.",
"warn": "Cloned voices. Every pack on this endpoint is a replica of a real person's voice. Audio coming out of it is not that person speaking and must never be passed off as them."
},
"url": "",
"docs": "",
"docs": "/docs#models",
"probe": "127.0.0.1:7772",
"method": "http",
"wip": false,
+13
View File
@@ -183,12 +183,23 @@ body::after {
.docs details.key-warn[open] summary::after { content: " \25BE"; }
.docs details.key-warn .warn-body { margin-top: 4px; }
.docs .foot-note { color: var(--dim); }
.docs h3 { color: var(--head); font-size: 1.05rem; margin-top: 16px; }
.docs figure { margin: 14px 0; }
.docs figcaption { color: var(--dim); font-size: .85rem; }
.docs table { border-collapse: collapse; width: 100%; margin: 10px 0; }
.docs th, .docs td { border: 1px solid var(--border); padding: 6px 8px; text-align: left; vertical-align: top; }
.docs th { color: var(--head); }
.docs .toc { display: grid; grid-template-columns: repeat(auto-fill, minmax(150px, 1fr)); gap: 4px; margin: 12px 0; }
.docs .toc a { display: block; border: 1px solid var(--border); padding: 6px 8px; text-decoration: none; }
.docs .kicker { color: var(--dim); text-transform: uppercase; letter-spacing: .12em; font-size: .72rem; }
html[data-bare="1"] .docs h1, html[data-bare="1"] .docs h2 { color: #000; border-color: #000; }
html[data-bare="1"] .docs a, html[data-bare="1"] .docs code { color: #000; }
html[data-bare="1"] .docs pre { background: none; border-color: #000; }
html[data-bare="1"] .docs pre code { color: #000; }
html[data-bare="1"] .docs img { border-color: #000; }
html[data-bare="1"] .docs .key-warn { color: #000; }
html[data-bare="1"] .docs h3, html[data-bare="1"] .docs th { color: #000; }
html[data-bare="1"] .docs th, html[data-bare="1"] .docs td, html[data-bare="1"] .docs .toc a { border-color: #000; }
/* Bare mode (Q27): strip to default browser rendering. */
html[data-bare="1"] body { background: #fff; color: #000; font-family: serif; text-shadow: none; font-size: 16px; }
@@ -252,6 +263,7 @@ html[data-bare="1"] .top { border-image: none; }
.mi-zealot { --mi: #4fb4ff; } /* Zealot: Khai's blue blades */
.mi-archon { --mi: #a8ecff; } /* Archon: pure psychic light */
.mi-dragoon { --mi: #5fe0d0; } /* Dragoon: phase-crystal teal */
.mi-tool { --mi: #ffc86b; } /* frank-tts: amber — a tool, not a unit */
.mi-top { display: flex; align-items: center; gap: 10px; }
.mi-art {
width: 56px;
@@ -283,6 +295,7 @@ html[data-bare="1"] .top { border-image: none; }
.mi-voice:hover { text-shadow: 0 0 8px var(--mi); }
.mi-voice.mi-speaking { animation: mi-pulse 0.7s ease-in-out infinite; }
@keyframes mi-pulse { 50% { text-shadow: 0 0 12px var(--mi); } }
.mi-tag { color: var(--mi); margin: 4px 0; }
.mi-equiv { color: var(--head); font-style: italic; margin: 4px 0; }
.mi-spec { color: var(--fg); margin: 4px 0; }
.mi-inputs { color: var(--dim); }
+11 -2
View File
@@ -75,8 +75,10 @@ PY
}
@test "exempt vhosts never route through anubis (auth, plex, own-auth apps)" {
# The IdP login flow and git smart-HTTP clients cannot solve challenges;
# these vhosts bypass the chain by construction.
# The IdP login flow, git smart-HTTP clients, and any non-browser client (the
# Audiobookshelf phone app, podcast RSS readers) cannot solve a challenge, so
# services that enforce their own complete auth bypass the chain by
# construction — the gate would answer their API calls with HTML.
local block
block=$(awk '/^auth\.franksdank\.cloud/,/^}/' "$CADDY")
echo "$block" | grep -q 'reverse_proxy authentik-server:9000'
@@ -84,4 +86,11 @@ PY
block=$(awk '/^plex\.franksdank\.cloud/,/^}/' "$CADDY")
echo "$block" | grep -q '192\.168\.0\.235:32400'
! echo "$block" | grep -q anubis
block=$(awk '/^audiobooks\.franksdank\.cloud/,/^}/' "$CADDY")
echo "$block" | grep -q '192\.168\.0\.235:8000'
! echo "$block" | grep -q anubis
# The exemption is total: the post-auth listener must not still carry an
# audiobooks route. A leftover handle there is a second owner for one host,
# and its X-Forwarded-Proto pin would silently disagree with the edge.
! grep -q '@audiobooks host audiobooks.franksdank.cloud' "$CADDY"
}
+80 -4
View File
@@ -17,6 +17,10 @@
# reported as success (a green deploy over a dead API is a lie);
# 5. ABS_SCAN outside 0/1 is a usage error raised before anything is touched;
# 6. the deployer runs the bootstrapper only after the health gate answered.
# 7. the SSO arm sends the base path: ABS builds its redirect_uri as
# <origin><authOpenIDSubfolderForRedirectURLs>/auth/openid/callback, and the
# value must be the path the app is mounted under — never the literal
# "undefined" a fresh database leaves there.
load helpers
@@ -29,7 +33,7 @@ import json, sys
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
port, logf = int(sys.argv[1]), sys.argv[2]
state = {"init": False, "user": None, "pw": None, "libs": []}
state = {"init": False, "user": None, "pw": None, "libs": [], "oidc": None}
def log(line):
@@ -37,6 +41,12 @@ def log(line):
f.write(line + "\n")
def norm(p):
# ABS mounts every route under ROUTER_BASE_PATH. The stub accepts the API at the
# root or under that prefix, so a test can still assert which one the client used.
return p[len("/audiobookshelf"):] if p.startswith("/audiobookshelf") else p
class H(BaseHTTPRequestHandler):
def log_message(self, *a):
pass
@@ -57,20 +67,24 @@ class H(BaseHTTPRequestHandler):
return self.headers.get("Authorization") == "Bearer stub-token"
def do_GET(self):
path = self.path.split("?")[0]
path = norm(self.path.split("?")[0])
log("GET %s auth=%s" % (self.path, "yes" if self.headers.get("Authorization") else "no"))
if path == "/status":
return self.send(200, {"app": "audiobookshelf", "isInit": state["init"]}, True)
methods = ["local", "openid"] if state["oidc"] else ["local"]
return self.send(200, {"app": "audiobookshelf", "isInit": state["init"],
"authMethods": methods}, True)
if not self.authed():
return self.send(401, "Unauthorized")
if path == "/api/libraries":
return self.send(200, {"libraries": state["libs"]}, True)
if path == "/api/auth-settings":
return self.send(200, state["oidc"] or {}, True)
if path.endswith("/items"):
return self.send(200, {"total": 42, "results": []}, True)
return self.send(404, "nope")
def do_POST(self):
path = self.path.split("?")[0]
path = norm(self.path.split("?")[0])
payload = self.body()
log("POST %s auth=%s" % (self.path, "yes" if self.headers.get("Authorization") else "no"))
if path == "/init":
@@ -101,6 +115,21 @@ class H(BaseHTTPRequestHandler):
return self.send(200, "OK")
return self.send(404, "nope")
def do_PATCH(self):
path = norm(self.path.split("?")[0])
payload = self.body()
log("PATCH %s auth=%s" % (self.path, "yes" if self.headers.get("Authorization") else "no"))
if not self.authed():
return self.send(401, "Unauthorized")
if path == "/api/auth-settings":
# Stored and echoed verbatim: the test asserts the wire, not the stub's
# opinion about which keys are patchable.
state["oidc"] = payload
for k in sorted(payload):
log("OIDCFG %s=%r" % (k, payload[k]))
return self.send(200, payload, True)
return self.send(404, "nope")
ThreadingHTTPServer(("127.0.0.1", port), H).serve_forever()
PY
@@ -198,3 +227,50 @@ boot() {
/audiobookshelf-init\.sh/ { init = NR }
END { exit !(health && init && init > health) }'
}
@test "the SSO arm sends the base path as the redirect subfolder" {
# What authentik has registered must equal what ABS builds:
# <origin><authOpenIDSubfolderForRedirectURLs>/auth/openid/callback. A database
# created before SSO leaves that key unset and ABS then sends the literal
# "/undefined/auth/openid/callback" — measured 2026-11-02, where the live login died
# on authentik's "Redirect URI Error" page. So the value has to reach the wire, and
# it has to be the path the app is served under.
umask 077
cat > "$TMP/cred" <<'CRED'
ABS_ADMIN_USER=frank
ABS_ADMIN_PASSWORD=0123456789abcdef0123456789
ABS_OIDC_CLIENT_SECRET=0123456789abcdef0123456789abcdef0123456789abcdef0123456789
CRED
run env ABS_URL="http://127.0.0.1:$PORT/audiobookshelf" ABS_SECRET="$TMP/cred" \
ABS_SCAN=0 bash "$INIT"
[ "$status" -eq 0 ]
printf '%s\n' "${lines[@]}" | grep -q "^oidc: armed"
grep -q "^OIDCFG authOpenIDSubfolderForRedirectURLs='/audiobookshelf'$" "$TMP/requests.log"
# every route the bootstrapper calls rides under the same prefix it armed with
grep -q "^POST /audiobookshelf/init auth=no$" "$TMP/requests.log"
grep -q "^GET /audiobookshelf/api/auth-settings auth=yes$" "$TMP/requests.log"
# A root-mounted deployment sends the EMPTY subfolder (the value the 2.17.4 upgrade
# uses), not whatever the last URL happened to carry.
run env ABS_URL="http://127.0.0.1:$PORT" ABS_SECRET="$TMP/cred" ABS_SCAN=0 bash "$INIT"
[ "$status" -eq 0 ]
[ "$(grep "^OIDCFG authOpenIDSubfolderForRedirectURLs=" "$TMP/requests.log" | tail -1)" \
= "OIDCFG authOpenIDSubfolderForRedirectURLs=''" ]
}
@test "the app and the IdP provider script build the same redirect path" {
# The provider row is written by scripts/authentik-audiobookshelf-sso.sh from
# ABS_BASE; the app builds its redirect_uri from the base path the deployer hands
# audiobookshelf-init.sh, which comes from the unit. Drift between the two is a
# redirect_uri mismatch — the login dies at the IdP before any credential is issued.
unit="$REPO_ROOT/dankserver/audiobookshelf/quadlet/audiobookshelf.container"
mounted=$(awk -F= '/^Environment=ROUTER_BASE_PATH=/{print $3}' "$unit")
[ -n "$mounted" ]
provider=$(grep -m1 '^BASE=' "$REPO_ROOT/scripts/authentik-audiobookshelf-sso.sh" \
| grep -oE 'https?://[^"}]+')
bootstrapper=$(grep -m1 '^ABS_URL=' "$INIT" | grep -oE 'https?://[^"}]+')
# Origins legitimately differ (LAN address vs loopback); the PATH must not.
[ "${provider#http://192.168.0.235:8000}/auth/openid/callback" \
= "${bootstrapper#http://127.0.0.1:8000}/auth/openid/callback" ]
[ "${bootstrapper#http://127.0.0.1:8000}" = "$mounted" ]
}
+6 -1
View File
@@ -182,8 +182,13 @@ load helpers
# under base path /audiobookshelf and the deploy gate timed out beside a unit
# that was active (running). PORT and ROUTER_BASE_PATH are what the image's own
# ENV block and the server source actually use.
# The base path is not a free choice either: /app/client/dist/index.html inside the
# pinned image carries routerBasePath:"/audiobookshelf", so a server mounted at / hands
# out a page whose every script request falls through to the SPA fallback and comes back
# text/html — the UI never leaves "Loading…" while the API stays healthy (measured
# 2026-11-02, which is exactly how it hid behind an API-only verification).
unit="$REPO_ROOT/dankserver/audiobookshelf/quadlet/audiobookshelf.container"
grep -q '^Environment=PORT=8000$' "$unit" || return 1
grep -q '^Environment=ROUTER_BASE_PATH=/$' "$unit" || return 1
grep -q '^Environment=ROUTER_BASE_PATH=/audiobookshelf$' "$unit" || return 1
! grep -qE '^Environment=ABS_' "$unit" || return 1
}
+9 -3
View File
@@ -35,14 +35,20 @@ export default defineConfig({
// 2026-10-19 - the newest guarded app, in neither GUARDED list nor any
// spec until now: the 17-hermes "uncovered vhost broke silently" class).
// 22- joins Prometheus's app-is-really-there proof (its own API), closing
// the "not a login wall but maybe a 404" hole 20- closed for Grafana.
testMatch: /(10|12|13|16|17|18|20|21|22)-.*\.spec\.ts/,
// the "not a login wall but maybe a 404" hole 20- closed for Grafana. 24-
// joins 2026-11-03 (Audiobookshelf's DEFAULT listener grant) and belongs to
// this project precisely because the standard identity is in no Audiobookshelf
// group — the admin project could not observe the default.
testMatch: /(10|12|13|16|17|18|20|21|22|24)-.*\.spec\.ts/,
use: { storageState: ".auth/qa-e2e.json" },
dependencies: ["setup"],
},
{
name: "admin",
testMatch: /11-.*\.spec\.ts/,
// 23- joins 2026-11-02 (Audiobookshelf SSO): it needs BOTH halves of the admin
// fixture — a live authentik session to ride the OIDC round trip, and the
// qa-e2e-admin group membership that grants site admin in the app.
testMatch: /(11|23)-.*\.spec\.ts/,
use: { storageState: ".auth/qa-e2e-admin.json" },
dependencies: ["setup"],
},
+58 -4
View File
@@ -6,9 +6,10 @@ import { expect, test } from "@playwright/test";
// navigation would skip. These tests drive the browser as an UNAUTHENTICATED
// user (the guest project has no storage state) and navigate directly to each
// guarded vhost's root path: the guard must present authentik's flow UI in
// place and must never leak app content. plex is the documented exemption —
// PMS enforces its own auth — so it must be reachable unauthenticated with
// zero challenge pages, proving the exemption is deliberate, not accidental.
// place and must never leak app content. plex and audiobooks are the documented
// exemptions — PMS and Audiobookshelf enforce their own auth — so they must be
// reachable unauthenticated with zero challenge pages, proving the exemption is
// deliberate, not accidental.
const BASE = "https://www.franksdank.cloud";
@@ -17,7 +18,9 @@ const BASE = "https://www.franksdank.cloud";
// added here or the guard-chain audit is incomplete.
const GUARDED = ["www", "search", "romm", "files", "seerr", "grafana", "prometheus", "loki", "notebook"];
// Markers that prove app content leaked past the guard to an anonymous user.
// Markers that prove app content leaked past the guard to an anonymous user. Each
// marker must be a string the APP renders that the gate itself could never print:
// authentik's gate never renders an app's UI text, because it IS the IdP.
const LEAK_MARKERS: Record<string, string[]> = {
"search.franksdank.cloud": ["SearXNG preferences"],
"romm.franksdank.cloud": ["RomM"],
@@ -98,3 +101,54 @@ test("plex is exempt from the guard chain and serves PMS content unauthenticated
const resp = await page.request.get("https://plex.franksdank.cloud/");
expect(resp.status(), "PMS must answer directly, not error").toBeLessThan(500);
});
// Audiobookshelf is exempt by construction (2026-11-03). The phone app is a
// non-browser client and solves no JS proof-of-work, so behind the guard chain it
// was answered with challenge HTML wherever the socket `init` payload belongs —
// every remote sign-in died as a TypeError on a non-object, and the container's
// request log never saw a byte of it. ABS enforces its own auth (local accounts
// plus authentik OIDC), exactly like PMS above. Asserted from the CLIENT side: the
// endpoints an app must call before it holds any credential are answered by the
// app, and the app's own 401 — not a 200 of HTML — is what an anonymous API call gets.
test("audiobookshelf answers an unauthenticated app client with app bytes, not gate bytes", async ({
request,
}) => {
const ABS = "https://audiobooks.franksdank.cloud";
// The probe every Audiobookshelf client sends first. The app answers it at the
// ROOT whatever ROUTER_BASE_PATH says, so this is the vhost's own behaviour.
const ping = await request.get(`${ABS}/ping`, { timeout: 30_000 });
expect(ping.status(), "/ping must be answered by the app").toBe(200);
expect(await ping.json(), "/ping must carry the app's own payload").toEqual({ success: true });
// The socket.io handshake. Engine.IO answers this in a machine format; the gate
// answered it in HTML, which is the defect this exemption exists to kill.
const sock = await request.get(`${ABS}/audiobookshelf/socket.io/?EIO=4&transport=polling`, {
timeout: 30_000,
});
expect(
await sock.text(),
"the socket handshake was answered by the gate, not Engine.IO",
).not.toContain("Making sure you're not a bot");
expect(
sock.headers()["content-type"] ?? "",
"the socket handshake must not arrive as HTML",
).not.toMatch(/text\/html/);
// An API call with no credential: the app refuses it. A gate would answer 200
// with HTML, so the status code alone distinguishes the two owners.
const api = await request.get(`${ABS}/audiobookshelf/api/libraries`, { timeout: 30_000 });
expect(api.status(), "the app's own auth gate must own unauthenticated API calls").toBe(401);
});
test("audiobookshelf login renders the app's own sign-in to an anonymous browser", async ({ page }) => {
test.setTimeout(120_000);
await page.goto("https://audiobooks.franksdank.cloud/audiobookshelf/login");
expect(new URL(page.url()).hostname).toBe("audiobooks.franksdank.cloud");
const body = await page.locator("body").innerText();
expect(body, "the bot challenge still fronts this vhost").not.toContain("Making sure you're not a bot");
expect(await page.locator('input[name="uidField"]').count(), "the IdP gate form must not front the app").toBe(0);
// The app answers, and its login screen carries SSO beside the local form.
await expect(page.getByText("Sign in with Authentik")).toBeVisible({ timeout: 90_000 });
});
@@ -0,0 +1,95 @@
import { expect, test } from "@playwright/test";
// Audiobookshelf: the SSO round trip must land a session with a real permission level.
//
// The service answers on a LAN address with no DNS name and no Caddy vhost, so it is in no
// guarded-vhost list and no other spec ever fetched a byte of it — every deploy gate passed
// while its login was dead. Two defect classes, both caught live on 2026-11-02:
//
// 1. The bundle must be BUILT for /audiobookshelf (ROUTER_BASE_PATH in the quadlet). Built
// for "/" the container still answers /healthcheck and still serves index.html, so the
// deploy health gate is green while the browser sees a blank page: every <script>
// resolves against the root, misses the static allowlist, and falls through to the SPA
// catch-all that returns index.html as text/html. Hence the MIME assertion below — the
// one thing that distinguishes a booted SPA from a served HTML stub.
// 2. The authentik provider row must carry grant_types AND the built-in OpenID scope
// mappings (provisioned by scripts/authentik-audiobookshelf-sso.sh). A blueprint
// re-import reverts hand-made provider rows silently, and the wire errors are generic:
// oauthlib's "The request is otherwise malformed" for an empty grant list, then
// "insufficient_scope" from ABS's /userinfo call for a short scope. Only the level
// assertion below — that the abs_groups claim granted site admin — sees either.
const HOST = "192.168.0.235:8000";
const ABS = `http://${HOST}/audiobookshelf`;
test("SSO round trip boots the SPA and grants a real Audiobookshelf session", async ({
page,
}) => {
test.setTimeout(180_000);
const scripts: { url: string; type: string; status: number }[] = [];
page.on("response", (res) => {
const u = new URL(res.url());
if (u.host === HOST && u.pathname.endsWith(".js")) {
scripts.push({ url: u.pathname, type: res.headers()["content-type"] ?? "", status: res.status() });
}
});
await page.goto(`${ABS}/login`, { waitUntil: "domcontentloaded" });
const sso = page.getByText("Sign in with Authentik");
await expect(sso).toBeVisible();
await sso.click();
// The admin project's storageState carries a live authentik session, so this is a silent
// round trip: authorize -> code -> callback -> token. Still on /login means the IdP
// refused the authorize request or ABS rejected the code exchange.
await expect
.poll(() => !new URL(page.url()).pathname.endsWith("/login"), { timeout: 90_000 })
.toBe(true);
await expect
.poll(
async () => {
const u = new URL(page.url());
return u.host === HOST && u.pathname.startsWith("/audiobookshelf");
},
{ timeout: 30_000 },
)
.toBe(true);
// Defect 1 gate: nothing the app loaded may be HTML dressed as a script. The render proof
// is the SSO button above — it only exists once the bundle has executed, so no DOM-shape
// guess is needed (the shell's own mount markup says nothing about whether it booted).
// Only responses that carried a body can be judged: the OIDC bounce re-validates the entry
// script (Cache-Control: max-age=0 + ETag) and a 304 carries no Content-Type by spec.
const bodies = scripts.filter((s) => s.status === 200);
expect(bodies.length, "the SPA loaded no JavaScript").toBeGreaterThan(2);
for (const s of bodies) {
expect(s.type, `${s.url} arrived as ${s.type}`).toContain("javascript");
}
// The token the app landed on is the credential its own API must accept. Give the SPA a
// moment to finish the exchange: the URL leaves /login the instant authentik bounces back
// through the callback route, which is BEFORE the app has stored what it exchanged.
const grabToken = () =>
page.evaluate(() => {
for (let i = 0; i < localStorage.length; i++) {
const v = localStorage.getItem(localStorage.key(i) as string) || "";
if (/^[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+$/.test(v)) return v;
}
const c = document.cookie.match(/(?:^|;\s*)abs_token=([^;]+)/);
return c ? decodeURIComponent(c[1]) : "";
});
await expect.poll(grabToken, { timeout: 60_000 }).not.toBe("");
const token = await grabToken();
const me = await page.request.get(`${ABS}/api/me`, {
headers: { Authorization: `Bearer ${token}` },
});
expect(me.status()).toBe(200);
const user = await me.json();
expect(user.username).toBe("qa-e2e-admin");
// Site role, derived ONLY from the abs_groups claim: it says "admin" if and only if the
// scope mapping reached ABS with audiobookshelf_admins in the list. A short token scope
// (missing OpenID mappings) never gets this far — the exchange dies before it.
expect(user.type).toBe("admin");
});
@@ -0,0 +1,53 @@
import { expect, test } from "@playwright/test";
// Audiobookshelf grants the LISTENER role to anyone the IdP lets sign in (2026-11-03).
//
// The role used to come from membership of `audiobookshelf_users` — a group nobody was ever
// added to. The household accounts exist in authentik, and every one of them was denied at ABS
// AFTER a successful authentication, which reads as a broken login rather than a policy decision.
// The scope mapping now answers `abs_groups: ["user"]` for every signer and reserves `admin` for
// `audiobookshelf_admins`, so the IdP is the only gate to the library and onboarding a listener
// is exactly "create the account".
//
// This is the counterpart to 23-: that spec proves the admin escalation, this one proves the
// DEFAULT grant, from an identity in neither group. Against the old mapping it is red — ABS
// refuses the sign-in and the SPA never leaves /login.
const HOST = "192.168.0.235:8000";
const ABS = `http://${HOST}/audiobookshelf`;
test("an SSO signer in no Audiobookshelf group lands as a listener", async ({ page }) => {
test.setTimeout(180_000);
await page.goto(`${ABS}/login`, { waitUntil: "domcontentloaded" });
const sso = page.getByText("Sign in with Authentik");
await expect(sso).toBeVisible();
await sso.click();
// The standard project's storageState carries a live authentik session, so the round trip is
// silent: authorize -> code -> callback -> token. Still being on /login means the app refused
// the identity (the denial this change removes) or the exchange failed.
await expect
.poll(() => !new URL(page.url()).pathname.endsWith("/login"), { timeout: 90_000 })
.toBe(true);
const grabToken = () =>
page.evaluate(() => {
for (let i = 0; i < localStorage.length; i++) {
const v = localStorage.getItem(localStorage.key(i) as string) || "";
if (/^[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+$/.test(v)) return v;
}
const c = document.cookie.match(/(?:^|;\s*)abs_token=([^;]+)/);
return c ? decodeURIComponent(c[1]) : "";
});
await expect.poll(grabToken, { timeout: 60_000 }).not.toBe("");
const me = await page.request.get(`${ABS}/api/me`, {
headers: { Authorization: `Bearer ${await grabToken()}` },
});
expect(me.status()).toBe(200);
const user = await me.json();
expect(user.username).toBe("qa-e2e");
// The default, not a grant: the listener role for a signer in neither group. Being `user`
// rather than `admin` is also the proof that the default never escalates.
expect(user.type).toBe("user");
});
+10 -5
View File
@@ -981,9 +981,12 @@ DRV
# there is no address the checker could dial, and a card that can never be
# green is worse than no card. The façade carries the pair's signal — and
# since the 2026-10-18 deploy that signal is a real probe (`wip: false`), not
# a placeholder. It stays `showLink: false` and admin-only: an HTTP API has no
# page to open, and the checker counts ANY HTTP response as up, so the card is
# honest without a link and without a `/health` path in the probe.
# a placeholder. It stays `showLink: false`: an HTTP API has no page to open,
# and the checker counts ANY HTTP response as up, so the card is honest
# without a link and without a `/health` path in the probe.
# 2026-11-04: the card moved from admin to the user page's Model Information
# block — the household guide already documents the voices (/docs#models), so
# the family reading about them should also see whether the door is up.
python3 - "$REPO_ROOT" <<'PY'
import json, os, sys
root = sys.argv[1]
@@ -996,8 +999,10 @@ assert fts["wip"] is False, \
"the endpoint is deployed; a wip card would hide its real state: " + repr(fts.get("wip"))
assert fts["showLink"] is False and fts["url"] == "", \
"an API endpoint has no UI to link to — it is a status card, not a tile"
assert fts["category"] == "admin", \
"consumers dial this; it is not an app the user homepage should advertise"
assert fts["category"] == "user" and fts["section"] == "AI", \
"the voice endpoint belongs to the user page's Model Information block: " + repr(fts["category"])
assert isinstance(fts.get("model"), dict) and fts["model"].get("role"), \
"the card renders as a Model Information card, which needs a model object"
assert os.path.exists(f"{root}/shared/homepage{fts['art']}"), f"missing art {fts['art']}"
PY
}
+22 -14
View File
@@ -37,7 +37,7 @@ PY
import json, sys
data = json.load(open(sys.argv[1]))
allowed = {
"user": {"AI", "Web Apps", "Media Library Apps"},
"user": {"AI", "Web Apps", "Media Library Apps", "Work in progress"},
"admin": {"Media Library Apps", "Observability", "AI", "Infrastructure"},
}
for svc in data:
@@ -76,23 +76,33 @@ PY
@test "the Model Information component is complete: text, art, voices, warnings" {
# The fused model panel is the family's only description of the fleet.
# Every model card must carry: equivalence note, supervision level, the
# unit art, at least one playable voice line, and (for uncensored models)
# the legal warning. The compaction recommendation must name the zealot.
# Every card must carry: role, spec, use, an inputs list, and art that
# exists. A card with a StarCraft unit identity (a chat model) must also
# carry the equivalence note, the supervision level, and a playable unit
# voice line whose file ships; a unit-less tool card (frank-tts speaks, it
# does not chat) quotes no unit, so it carries the ask-Frank permission
# note instead. Uncensored models must carry the legal warning. The
# compaction recommendation must name the zealot.
python3 - "$REPO_ROOT/shared/homepage/services.json" "$REPO_ROOT/shared/homepage" <<'PY'
import json, os, sys
services = json.load(open(sys.argv[1]))
root = sys.argv[2]
fleet = [s for s in services if "model" in s]
assert {s["name"] for s in fleet} == {"frank-dragoon", "frank-archon", "frank-zealot"}, [s["name"] for s in fleet]
assert {s["name"] for s in fleet} == {"frank-dragoon", "frank-archon", "frank-zealot", "frank-tts"}, [s["name"] for s in fleet]
for s in fleet:
m = s["model"]
assert m["equiv"].strip(), f"{s['name']} has no equivalence note"
assert m["supervision"].strip(), f"{s['name']} has no supervision note"
assert m["sfx"], f"{s['name']} has no voice line"
assert os.path.isfile(os.path.join(root, "img", "unit-" + m["unit"] + ".jpg")), f"{s['name']} art missing"
for f in m["sfx"] + m.get("voice", {}).get("sfx", []):
assert m["role"].strip() and m["spec"].strip() and m["use"].strip(), f"{s['name']} needs role, spec and use"
assert m.get("inputs"), f"{s['name']} has no inputs list"
art = s.get("art") or "img/unit-%s.jpg" % m["unit"]
assert os.path.isfile(os.path.join(root, art.lstrip("/"))), f"{s['name']} art missing: {art}"
for f in list(m.get("sfx", [])) + m.get("voice", {}).get("sfx", []):
assert os.path.isfile(os.path.join(root, "sfx", f)), f"{s['name']} voice {f} missing"
if m.get("unit"):
assert m["equiv"].strip(), f"{s['name']} has no equivalence note"
assert m["supervision"].strip(), f"{s['name']} has no supervision note"
assert m.get("sfx"), f"{s['name']} has no voice line"
else:
assert m.get("permission", "").strip(), f"{s['name']} has no permission note"
zealot = next(s for s in fleet if s["name"] == "frank-zealot")
assert "compaction" in zealot["model"].get("compaction", ""), "zealot must be recommended as compaction model"
assert zealot["model"].get("compaction", "") != "", "compaction advice must be its own visible field"
@@ -102,13 +112,11 @@ for s in fleet:
uncensored = "ncensored" in m["spec"] or "bliterated" in m["spec"]
if uncensored:
assert "legal" in m.get("warn", "").lower(), f"{s['name']} is uncensored but has no legal warning"
assert [s["name"] for s in fleet] == ["frank-zealot", "frank-dragoon", "frank-archon"], \
"card render order must be zealot, dragoon, archon: " + str([s["name"] for s in fleet])
assert [s["name"] for s in fleet] == ["frank-zealot", "frank-dragoon", "frank-archon", "frank-tts"], \
"card render order must be zealot, dragoon, archon, then the voice tool: " + str([s["name"] for s in fleet])
w = {s["name"]: s["model"].get("warn", "") for s in fleet}
assert "legal risk" in w["frank-dragoon"].lower() and "everybody" in w["frank-dragoon"].lower(), \
"the abliterated endpoint must carry the common legal warning"
assert "censored" in w["frank-zealot"].lower() and "uncensored" not in w["frank-zealot"].lower(), \
"zealot ships the censored base since 2026-10-17: its card warning must not claim uncensored output"
archon = next(s for s in fleet if s["name"] == "frank-archon")
assert "permission" in archon["model"], "archon card needs the ask-Frank permission callout"
PY
+2 -2
View File
@@ -41,8 +41,8 @@ setup() {
grep -q "repeat(auto-fill" "$BASE/style.css"
}
@test "the user page keeps the AI section at the bottom" {
grep -q "user: \['Web Apps', 'Media Library Apps', 'AI'\]" "$BASE/app.js"
@test "the user page puts the Work in progress section last, below AI" {
grep -q "user: \['Web Apps', 'Media Library Apps', 'AI', 'Work in progress'\]" "$BASE/app.js"
}
@test "WIP badges distinguish open from closed work" {
+38 -3
View File
@@ -13,9 +13,12 @@
# 3. a downloaded plugin artifact is accepted only on its pinned sha256, its
# official guid and its pinned version (a swapped or downgraded artifact
# is refused before anything lands in plugins/);
# 4. the TMDb key is written into the EXISTING config with every other
# setting preserved — Jellyfin owns that file, and clobbering it would
# silently reset country, language and adult settings;
# 4. a key is written into the EXISTING config with every other setting
# preserved, under whatever root that file's owner chose — Jellyfin's core
# configs say `<Configuration>`, TheTVDB writes `<PluginConfiguration>` and
# omits a null setting outright, so the element to fill can be absent and
# the anchor has to come from the file; clobbering the file would silently
# reset country, language and adult settings;
# 5. a TVDB config the operator already saved through the UI is kept;
# 6. install is idempotent: a second run changes no byte;
# 7. whatever is replaced is copied to the trash first (fleet law: never
@@ -254,6 +257,38 @@ XML
grep -qF '<EnableSeasonPathMatching>true</EnableSeasonPathMatching>' "$cfg"
}
@test "the key lands in a plugin-written config with no ApiKey element" {
# What TheTVDB actually writes on a fresh install: its own root element and
# no <ApiKey /> at all, because its serializer drops null settings. The
# installer used to anchor on </Configuration> and refuse this file.
cat >"$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml" <<XML
<?xml version="1.0" encoding="utf-8"?>
<PluginConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<SubscriberPIN />
<CacheDurationInHours>1</CacheDurationInHours>
<OriginalTitle>true</OriginalTitle>
</PluginConfiguration>
XML
run prov install
[ "$status" -eq 0 ]
printf '%s\n' "${lines[@]}" | grep -qiF 'filled the ApiKey'
local cfg="$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml"
grep -qF "<ApiKey>$TVDB_KEY</ApiKey>" "$cfg"
grep -qF '<CacheDurationInHours>1</CacheDurationInHours>' "$cfg"
grep -qF '<OriginalTitle>true</OriginalTitle>' "$cfg"
}
@test "a config file with no root element is refused by name" {
# The guard still has to fire when there is genuinely nowhere to write —
# a refusal naming the file beats inventing a document shape.
printf '<?xml version="1.0" encoding="utf-8"?>\n' \
>"$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml"
run prov install
[ "$status" -ne 0 ]
printf '%s\n' "${lines[@]}" | grep -qF 'Jellyfin.Plugin.Tvdb.xml'
printf '%s\n' "${lines[@]}" | grep -qiF 'no place to write'
}
@test "a pin in the TVDB secret lands in the plugin config" {
# A subscriber-shaped key needs the account's PIN beside it; a licensed key
# does not, so the element appears only when the secret carries one.
+361
View File
@@ -0,0 +1,361 @@
#!/usr/bin/env bats
# Seam: scripts/media-refresh-jellyfin.py + scripts/media-refresh-plex.sh —
# the two library-refresh drivers every media pass ends with.
#
# The rescan is the part of a media move that actually makes it visible, and
# both servers punish the naive version of it. Jellyfin 12.4: the quoted
# MediaBrowser header is the only auth form; POST /Library/Refresh is the only
# door that queues work; a SECOND accepted POST cancels the scan already
# running, so the index looks frozen while you hammer it; 429 means wait. Plex:
# its HTTPD answers nobody outside its own netns and speaks TLS to everyone who
# is inside the LAN, so the request has to be shipped INTO the container; the
# token is PlexOnlineToken in the container's Preferences.xml; and progress is
# readable without any of that, from section_locations.scanned_at. This seam
# pins, against stubs that speak those APIs:
# 1. the Jellyfin driver authenticates as `Authorization: MediaBrowser
# Token="…"`, and an auth refusal ends the run naming the code;
# 2. a trigger while Scan Media Library is running is REFUSED with no POST at
# all (the cancel-restart trap); --force overrides;
# 3. a 429 is waited out with a bounded number of attempts, never hammered,
# and a still-busy server ends in a non-zero exit that says 429;
# 4. `status` prints {"scan": …, "recentMedia": …} from the task table plus a
# fixture DB, and `wait` returns only after N consecutive quiet reads;
# 5. the Plex driver carries the token from the container's Preferences.xml
# into a header — never argv, never a URL;
# 6. a Plex trigger that gets no answer is a warming server: retried, bounded,
# and reported as 000 rather than a service restart;
# 7. Plex `status` answers from section_locations.scanned_at and is exit-0
# only when that epoch has passed --after;
# 8. a missing credential or database is refused by NAME before any request.
load helpers
setup() {
JF="$REPO_ROOT/scripts/media-refresh-jellyfin.py"
PX="$REPO_ROOT/scripts/media-refresh-plex.sh"
TMP="$(mktemp -d)"
PORT=18947
printf 'stubkey\n' > "$TMP/jf-key"
printf '<Preferences PlexOnlineToken="stubplextoken"/>\n' > "$TMP/prefs.xml"
echo Idle > "$TMP/scan-state"
echo ok > "$TMP/refresh-mode"
echo 0 > "$TMP/polls"
mkdir -p "$TMP/bin" "$TMP/container"
# A podman that is really a local directory: the shim runs the runner the
# driver copied in, so the in-container half is exercised for real.
cat > "$TMP/bin/podman" <<'SH'
#!/usr/bin/env bash
echo "podman $*" >> "$PODMAN_LOG"
case "$1" in
cp) cp "$2" "$CONTAINER_FS/$(basename "$3" | sed 's#.*:##')" ;;
exec) shift; [ "$1" = "$CONTAINER" ] || exit 1; shift
[ "$1" = rm ] && exit 0
[ "$1" = sh ] || exit 1; shift
script="$1"; shift
sh "$CONTAINER_FS/$(basename "$script")" "$@" ;;
*) exit 1 ;;
esac
SH
chmod +x "$TMP/bin/podman"
cat > "$TMP/stub.py" <<'PY'
import json, os, sys
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
port, logf = int(sys.argv[1]), sys.argv[2]
def read(var, default=""):
# Scenarios are flipped by rewriting a file; the env var names that file.
try:
return open(os.environ.get(var, "")).read().strip()
except OSError:
return default
def log(line):
with open(logf, "a") as f:
f.write(line + "\n")
class H(BaseHTTPRequestHandler):
def log_message(self, *a):
pass
def send(self, code, body, as_json=False):
data = json.dumps(body).encode() if as_json else str(body).encode()
self.send_response(code)
self.send_header("Content-Type", "application/json" if as_json else "text/xml")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def jf_authed(self):
# Jellyfin 12.4 accepts the quoted MediaBrowser form ONLY: a bare
# X-Emby-Token reads as unauthenticated here, exactly as it does live.
return self.headers.get("Authorization") == 'MediaBrowser Token="stubkey"'
def do_GET(self):
path = self.path.split("?")[0]
log("GET %s auth=%s" % (self.path, "yes" if self.headers.get("Authorization") else "no"))
if path == "/System/Info":
if self.jf_authed():
return self.send(200, {"ServerName": "stub", "Version": "12.4.0"}, True)
return self.send(401, "Unauthorized")
if path == "/ScheduledTasks":
with open(os.environ["POLL"]) as f:
n = int(f.read().strip() or 0)
with open(os.environ["POLL"], "w") as f:
f.write(str(n + 1))
state = read("SCAN", "Idle")
flip = read("FLIP", "")
if flip and n + 1 >= int(flip):
state = "Idle"
return self.send(200, [{"Name": "Audio Normalization", "State": "Running"},
{"Name": "Scan Media Library", "State": state,
"EndTimeUtc": read("ENDED", "") or None}], True)
return self.send(404, "nope")
def do_POST(self):
path = self.path.split("?")[0]
log("POST %s auth=%s" % (self.path, "yes" if self.headers.get("Authorization") else "no"))
if path == "/Library/Refresh":
if not self.jf_authed():
return self.send(401, "Unauthorized")
if read("MODE", "ok") == "busy":
return self.send(429, "too many requests")
return self.send(204, "")
if path.endswith("/library/refresh") or "/library/sections/" in path:
if self.headers.get("X-Plex-Token") != "stubplextoken":
return self.send(401, "Unauthorized")
return self.send(204, "")
return self.send(404, "nope")
ThreadingHTTPServer(("127.0.0.1", port), H).serve_forever()
PY
export SCAN="$TMP/scan-state" MODE="$TMP/refresh-mode" POLL="$TMP/polls"
export FLIP="$TMP/flip-after"
export ENDED="$TMP/scan-ended"
export PODMAN_LOG="$TMP/podman.log" CONTAINER_FS="$TMP/container" CONTAINER=plex
: > "$PODMAN_LOG"
python3 "$TMP/stub.py" "$PORT" "$TMP/requests.log" > "$TMP/stub.out" 2>&1 &
STUB_PID=$!
for _ in $(seq 1 50); do
curl -s -o /dev/null "http://127.0.0.1:$PORT/System/Info" && break
sleep 0.1
done
}
teardown() {
[ -n "${STUB_PID:-}" ] && kill "$STUB_PID" 2>/dev/null || true
rm -rf "$TMP"
}
plex_db() { # plex_db <path> <scanned_at epoch>
python3 - "$1" "$2" <<'PY'
import sqlite3, sys
db = sqlite3.connect(sys.argv[1])
db.execute("CREATE TABLE section_locations (id INTEGER PRIMARY KEY, library_section_id INT,"
" root_path TEXT, available INT, scanned_at INT)")
db.execute("INSERT INTO section_locations VALUES (1, 1, '/mnt/entropy/Movies', 1, ?)",
(int(sys.argv[2]),))
db.commit()
PY
}
# ---------------------------------------------------------------- jellyfin
@test "the jellyfin driver authenticates as MediaBrowser Token, never argv" {
run python3 "$JF" trigger --url "http://127.0.0.1:$PORT" --key-file "$TMP/jf-key"
[ "$status" -eq 0 ]
grep -q '^POST /Library/Refresh' "$TMP/requests.log"
! grep -q "stubkey" "$TMP/requests.log"
}
@test "a server that rejects the auth form is a failure, not a success" {
# The stub 401s anything but the quoted form; the driver must report it
# rather than exit 0 — a green trigger over a 401 is a lie.
printf 'wrongkey\n' > "$TMP/bad-key"
run python3 "$JF" trigger --url "http://127.0.0.1:$PORT" --key-file "$TMP/bad-key"
[ "$status" -ne 0 ]
[[ "$output" == *"401"* ]]
}
@test "a trigger during a running scan is refused with no POST at all" {
echo Running > "$TMP/scan-state"
run python3 "$JF" trigger --url "http://127.0.0.1:$PORT" --key-file "$TMP/jf-key"
[ "$status" -ne 0 ]
[[ "$output" == *"running"* ]]
! grep -q '^POST /Library/Refresh' "$TMP/requests.log"
}
@test "--force triggers anyway while a scan runs" {
echo Running > "$TMP/scan-state"
run python3 "$JF" trigger --url "http://127.0.0.1:$PORT" --key-file "$TMP/jf-key" --force
[ "$status" -eq 0 ]
grep -q '^POST /Library/Refresh' "$TMP/requests.log"
}
@test "a busy server is waited out with a bounded number of attempts" {
echo busy > "$TMP/refresh-mode"
run python3 "$JF" trigger --url "http://127.0.0.1:$PORT" --key-file "$TMP/jf-key" \
--retries 2 --retry-sleep 0.1
[ "$status" -ne 0 ]
[[ "$output" == *"429"* ]]
posts=$(grep -c '^POST /Library/Refresh' "$TMP/requests.log")
[ "$posts" -le 3 ]
}
@test "status reports the scan task and recent media writes" {
python3 - "$TMP/jellyfin.db" <<'PY'
import sqlite3, sys, time
db = sqlite3.connect(sys.argv[1])
db.execute("CREATE TABLE BaseItems (Id TEXT, DateModified TEXT)")
now = int(time.time())
for i in range(3):
db.execute("INSERT INTO BaseItems VALUES (?, datetime(?, 'unixepoch'))", (str(i), now - 30))
db.execute("INSERT INTO BaseItems VALUES ('old', datetime(?, 'unixepoch'))", (now - 86400,))
db.commit()
PY
run python3 "$JF" status --url "http://127.0.0.1:$PORT" --key-file "$TMP/jf-key" \
--db "$TMP/jellyfin.db" --window 5
[ "$status" -eq 0 ]
python3 -c 'import json,sys; d=json.loads(sys.argv[1]); assert d["scan"] is False, d; assert d["recentMedia"] == 3, d' "$output"
echo Running > "$TMP/scan-state"
run python3 "$JF" status --url "http://127.0.0.1:$PORT" --key-file "$TMP/jf-key" \
--db "$TMP/jellyfin.db" --window 5
python3 -c 'import json,sys; assert json.loads(sys.argv[1])["scan"] is True' "$output"
}
@test "wait returns only after the scan reads quiet repeatedly" {
python3 - "$TMP/jellyfin.db" <<'PY'
import sqlite3, sys
db = sqlite3.connect(sys.argv[1])
db.execute("CREATE TABLE BaseItems (Id TEXT, DateModified TEXT)")
db.commit()
PY
echo Running > "$TMP/scan-state"
echo 4 > "$TMP/flip-after"
run python3 "$JF" wait --url "http://127.0.0.1:$PORT" --key-file "$TMP/jf-key" \
--db "$TMP/jellyfin.db" --interval 0 --require-quiet 2 --timeout 30
[ "$status" -eq 0 ]
[[ "${lines[-1]}" == *'"quiet": 2'* ]]
# every read before poll 4 was busy, so no earlier line may claim quiet
[ "$(printf '%s\n' "$output" | grep -c '"quiet": 0')" -ge 3 ]
}
@test "wait gives up out loud when the library never settles" {
python3 - "$TMP/jellyfin.db" <<'PY'
import sqlite3, sys
db = sqlite3.connect(sys.argv[1])
db.execute("CREATE TABLE BaseItems (Id TEXT, DateModified TEXT)")
db.commit()
PY
echo Running > "$TMP/scan-state"
run python3 "$JF" wait --url "http://127.0.0.1:$PORT" --key-file "$TMP/jf-key" \
--db "$TMP/jellyfin.db" --interval 0 --timeout 0.2
[ "$status" -ne 0 ]
[[ "$output" == *"still busy"* ]]
}
@test "wait refuses a scan that never started instead of calling the library settled" {
python3 - "$TMP/jellyfin.db" <<'PY'
import sqlite3, sys
db = sqlite3.connect(sys.argv[1])
db.execute("CREATE TABLE BaseItems (Id TEXT, DateModified TEXT)")
db.commit()
PY
echo Idle > "$TMP/scan-state"
run python3 "$JF" wait --url "http://127.0.0.1:$PORT" --key-file "$TMP/jf-key" \
--db "$TMP/jellyfin.db" --interval 0.05 --start-window 0.4 --timeout 60
[ "$status" -ne 0 ]
[[ "$output" == *"never started"* ]]
[[ "$output" == *"cancels it"* ]]
# a watcher that never saw the scan may not hand back a quiet count
! grep -Eq '"quiet": [1-9]' <<<"$output"
}
@test "status --after refuses to certify a scan that predates the moves" {
python3 - "$TMP/jellyfin.db" <<'PY'
import sqlite3, sys
db = sqlite3.connect(sys.argv[1])
db.execute("CREATE TABLE BaseItems (Id TEXT, DateModified TEXT)")
db.commit()
PY
owed=$(python3 -c 'import time; print(int(time.time()) - 7200)')
future=$(python3 -c 'import time; print(int(time.time()) + 7200)')
date -u -d '3 hours ago' +%Y-%m-%dT%H:%M:%SZ > "$TMP/scan-ended"
run python3 "$JF" status --url "http://127.0.0.1:$PORT" --key-file "$TMP/jf-key" \
--db "$TMP/jellyfin.db" --after "$future"
[ "$status" -ne 0 ]
[[ "$output" == *"still owed"* ]]
date -u +%Y-%m-%dT%H:%M:%SZ > "$TMP/scan-ended"
run python3 "$JF" status --url "http://127.0.0.1:$PORT" --key-file "$TMP/jf-key" \
--db "$TMP/jellyfin.db" --after "$owed"
[ "$status" -eq 0 ]
[[ "$output" == *'"scannedAfter": true'* ]]
}
@test "a missing jellyfin credential file is refused by name" {
run python3 "$JF" trigger --url "http://127.0.0.1:$PORT" --key-file "$TMP/nope"
[ "$status" -ne 0 ]
[[ "$output" == *"$TMP/nope"* ]]
! grep -q '^POST' "$TMP/requests.log"
}
# --------------------------------------------------------------------- plex
@test "the plex driver carries the token from Preferences.xml into a header" {
run env PATH="$TMP/bin:$PATH" bash "$PX" trigger --library 1 \
--url "http://127.0.0.1:$PORT" --prefs "$TMP/prefs.xml"
[ "$status" -eq 0 ]
[[ "$output" == *"refresh accepted (204)"* ]]
grep -q "^POST /library/sections/1/refresh" "$TMP/requests.log"
# the token reaches PMS as a header only: never in a URL, never in an argv
! grep -q "stubplextoken" "$TMP/requests.log"
! grep -q "stubplextoken" "$TMP/podman.log"
}
@test "a plex trigger with no answer is a warming server, retried and named" {
run env PATH="$TMP/bin:$PATH" bash "$PX" trigger --library 1 \
--url "http://127.0.0.1:1" --prefs "$TMP/prefs.xml" --tries 2 --warm-sleep 0.1
[ "$status" -ne 0 ]
[[ "$output" == *"000"* ]]
[[ "$output" == *"warming"* ]] || [[ "$output" == *"never answered"* ]]
[ "$(grep -c '^podman cp' "$TMP/podman.log")" -eq 2 ]
}
@test "a plex trigger with no token in Preferences.xml fails before any request" {
printf '<Preferences/>\n' > "$TMP/prefs-empty.xml"
run env PATH="$TMP/bin:$PATH" bash "$PX" trigger --library 1 \
--url "http://127.0.0.1:$PORT" --prefs "$TMP/prefs-empty.xml"
[ "$status" -ne 0 ]
[[ "$output" == *"PlexOnlineToken"* ]]
! grep -q '^POST' "$TMP/requests.log"
}
@test "plex status reads the scan epoch from the database" {
plex_db "$TMP/library.db" 1000
run bash "$PX" status --library 1 --db "$TMP/library.db"
[ "$status" -eq 0 ]
[[ "$output" == *"section=1 scanned_at=1000"* ]]
[[ "$output" == *"/mnt/entropy/Movies"* ]]
run bash "$PX" status --library 1 --db "$TMP/library.db" --after 2000
[ "$status" -ne 0 ]
[[ "$output" == *"still owed"* ]]
run bash "$PX" status --library 1 --db "$TMP/library.db" --after 500
[ "$status" -eq 0 ]
}
@test "a plex status with no readable database is refused by name" {
run bash "$PX" status --library 1 --db "$TMP/no-library.db"
[ "$status" -ne 0 ]
[[ "$output" == *"$TMP/no-library.db"* ]]
}
@test "plex status names a section that has no location" {
plex_db "$TMP/library.db" 1000
run bash "$PX" status --library 77 --db "$TMP/library.db"
[ "$status" -ne 0 ]
[[ "$output" == *"section 77"* ]]
}
+33
View File
@@ -460,6 +460,39 @@ print(sum(1 for l in open(sys.argv[1]) if json.loads(l).get("op") == "move"))' "
[ -e "$LIB/Alien (1979)/Alien.1979.1080p.BluRay.x265.10bit.mkv" ]
}
# A hand-written plan row whose verb is not in the executor's vocabulary used
# to be dropped in silence: read_plan's if/elif chain had no else, so a plan of
# nothing but typos applied as a successful no-op (hit for real on 2026-11-02,
# when a pass meant to rename two series folders reported "applied: 0 moved"
# and exit 0). An unread row is a plan bug, and a plan bug refuses.
@test "a plan row with a verb this tool does not know stops the whole run" {
plan
JOURNAL="$BATS_TEST_TMPDIR/journal.jsonl"
at "$LIB/Stray/Stray.S01E01.mkv" 4000
printf 'rename\t%s\t%s\tfile\n' \
"$LIB/Stray/Stray.S01E01.mkv" "$LIB/Stray/Stray - S01E01.mkv" >> "$OUT"
run python3 "$APPLY" apply --plan "$OUT" --root "$LIB" --journal "$JOURNAL" --apply
[ "$status" -ne 0 ]
[[ "$output" == *"rename"* ]]
# all-or-nothing: the rows this tool DOES understand did not run either
[ -e "$LIB/Stray/Stray.S01E01.mkv" ]
[ ! -e "$JOURNAL" ]
}
@test "a plan whose every row is an unknown verb refuses instead of reporting success" {
JOURNAL="$BATS_TEST_TMPDIR/journal.jsonl"
at "$LIB/Stray/Stray.S01E01.mkv" 4000
{ printf 'action\tsrc\tdst\tkind\n'
printf 'rename\t%s\t%s\tfile\n' \
"$LIB/Stray/Stray.S01E01.mkv" "$LIB/Stray/Stray - S01E01.mkv"
} > "$BATS_TEST_TMPDIR/typo.tsv"
run python3 "$APPLY" apply --plan "$BATS_TEST_TMPDIR/typo.tsv" --root "$LIB" --journal "$JOURNAL" --apply
[ "$status" -ne 0 ]
[[ "$output" == *"unknown action"* ]]
[[ "$output" != *"applied"* ]]
[ -e "$LIB/Stray/Stray.S01E01.mkv" ]
}
@test "a run interrupted between the journal write and the move finishes the job" {
plan
JOURNAL="$BATS_TEST_TMPDIR/journal.jsonl"
+3 -4
View File
@@ -799,14 +799,13 @@ assert not any('container_network' in e or 'container_fs_' in e for e in names),
skip "franksdank/alloy image not present on this box"
fi
local sandbox="$BATS_TEST_TMPDIR/opt" cfg="$BATS_TEST_TMPDIR/broken.alloy"
sed 's/GRAFANA_HOST_LABEL/dankclaw/g' shared/alloy/config.alloy > "$cfg"
sed 's/GRAFANA_HOST_LABEL/dankclaw/g' "$REPO_ROOT/shared/alloy/config.alloy" > "$cfg"
printf 'bogus_attr = "x"\n' >> "$cfg"
! (
cd "$REPO_ROOT"
eval "$(sed -n '/^copy_alloy_config()/,/^}/p' scripts/deploy-remote.sh)"
run_privileged() { "$@"; }
cp() { if [ "$1" = "$cfg" ]; then return 1; fi; command cp "$@"; }
export TARGET=dankclaw ALLOY_CONF_ROOT="$sandbox"
export TARGET=dankclaw ALLOY_CONF_ROOT="$sandbox" ALLOY_CONFIG_SRC="$cfg"
copy_alloy_config
)
[ ! -f "$sandbox/config/config.alloy" ]
@@ -820,7 +819,7 @@ assert not any('container_network' in e or 'container_fs_' in e for e in names),
skip "franksdank/alloy image not present on this box"
fi
local cfg="$BATS_TEST_TMPDIR/rendered.alloy"
sed 's/GRAFANA_HOST_LABEL/dankclaw/g' shared/alloy/config.alloy > "$cfg"
sed 's/GRAFANA_HOST_LABEL/dankclaw/g' "$REPO_ROOT/shared/alloy/config.alloy" > "$cfg"
podman run --rm -v "$cfg:/c.alloy:ro" --entrypoint /usr/bin/alloy \
localhost/franksdank/alloy:latest validate /c.alloy
}