Compare commits

..
Author SHA1 Message Date
Frank 23ad0ca317 chore(abs): name the Audiobookshelf door audiobooks.franksdank.cloud
`abs.` was a three-letter abbreviation of the container; the library it serves is
/mnt/entropy/Audiobooks, and `audiobooks.` says what the door is. Renamed in one
commit on purpose: the pihole seam fails the suite if a Caddyfile vhost has no
addn-hosts record, so any split would leave a red commit, and the `abs` name was
never deployed at the IdP with this fix, so nothing has to keep answering it —
no alias, no redirect, no stale URI.

Moves together: the Caddyfile site block, its matcher (@abs -> @audiobooks), the
LAN record, ABS_EXTRA_BASES in authentik-audiobookshelf-sso.sh (that script
REPLACES the provider's redirect-URI list, so the old origin's callback leaves the
RP when it is next run), the homepage admin tile, the guard spec's GUARDED list and
leak marker, and the three vault pages that name the door. authentik's provider row
follows in the deploy window (external_host + name), which re-derives its
redirect_uris through scripts/authentik-proxy-provider-defaults.sh; the Application
row keeps its own name, "Audiobookshelf Guard" — it names the app, not the vhost.
2026-11-03 01:23:54 -08:00
Frank 71d5804eaf docs(caddy): put both auth walls in the guard-a-new-vhost procedure
Step 1 gains the closing sentence the abs-proxy incident demands: the ak-shell
snippet that creates a ProxyProvider with objects.create() must be followed by
scripts/authentik-proxy-provider-defaults.sh, or the guard fails before any
credential. Step 2 gains the forwarded-scheme clause for any app that derives its
own origin from the request. Each fact keeps one owner — the laws stay on
[[Authentik Outpost Reference]] (provisioning rule 3) and [[Audiobookshelf]]; this
page only says where in the procedure they bite. Outpost page timestamp bumped to
match its substantive edit.
2026-11-03 01:17:12 -08:00
Frank 5ef141eb17 fix(caddy): tell Audiobookshelf its real scheme so the OIDC callback validates
ABS checks every OIDC callback against the origin it believes it is served from:
getRequestOrigin() is x-forwarded-proto://Host (dist-server/server/utils/requestUtils.js,
read in the pinned image digest 2026-11-03). The post-auth http://:8017 listener is fed
over plain http, so Caddy's own X-Forwarded-Proto is http, ABS derives
http://abs.franksdank.cloud, and answers its own https callback with

    400 {"message":"Invalid callback URL - must be same-origin"}

Measured at the app with the callback held fixed: Host abs.franksdank.cloud alone
reproduces the 400 (as does every other Host, including the bare LAN IP), and adding
`X-Forwarded-Proto: https` turns it into a 302 to /application/o/authorize/ carrying
the registered redirect_uri. The edge for a guarded name is TLS-only, so https is the
truthful value, not a spoof; the LAN door needs nothing because there browser and
server already agree on http, which is why only the guarded path was broken.
2026-11-03 01:16:40 -08:00
Frank fadbe10420 fix(authentik): census and repair the OAuth defaults on every proxy provider
A proxy provider built by hand in `ak shell` never runs
ProxyProvider.set_oauth_defaults() — the REST serializer does, the ORM does not —
so the row looks fully configured while redirect_uris is empty (they are derived
from external_host by _get_callback_url()), grant_types is [], and
authorization_flow is NULL. Each one kills a guarded login in a different,
unrelated-looking way, all before any credential is requested: authentik's own
"Redirect URI Error" page, then a bodiless HTTP 400 from /application/o/authorize/
once the URIs are filled by hand, and a 500 NoneType.slug in the flow planner.
abs-proxy (abs.franksdank.cloud) hit the first two on 2026-11-03.

The script calls the vendor's own set_oauth_defaults() and points the
authorization flow at default-provider-authorization-implicit-consent for every
proxy provider, idempotently; a provider that already matches is untouched, so a
deliberate deviation is never flattened. `--check` is the census (exit 1 on
drift, writes nothing) for gate use. Applied live 2026-11-03: abs-proxy fixed,
the other ten (REST-created) already compliant, census now reports drift: 0.

This is authentik DB state and no deployable file carries it — a blueprint
re-import or an ak-shell re-creation reverts it and the symptom returns, so the
law and the failure signatures are recorded on the outpost page as provisioning
rule 3.
2026-11-03 01:16:35 -08:00
dank 17ea2f4110 Merge pull request 'feat(caddy): give Audiobookshelf a guarded abs.franksdank.cloud address' (#130) from feat/abs-vhost into main 2026-11-03 00:57:47 -08:00
Frank 603513a15e docs: the guard-a-new-vhost procedure and Audiobookshelf's second door
Infrastructure/Caddy.md now owns the order of operations for exposing a vhost
(proxy provider + application + embedded-outpost claim, then the route, then the
name and the chain audit), because the IdP row is invisible state that nobody
recreates. Audiobookshelf's Internal-only row and law become Two doors, one app;
the homepage link and index.md follow.
2026-11-03 00:55:02 -08:00
Frank 5e0592381a test(e2e): audit the guard chain over abs.franksdank.cloud
The guard spec's GUARDED list is the lockstep with the Caddyfile; abs joins it
with the app's own SSO-button label as the leak marker - a string the Audiobookshelf
login renders and the IdP gate never prints.
2026-11-03 00:55:02 -08:00
Frank ef89f77ea7 feat(authentik): register the ABS browser callback per origin
ABS builds redirect_uri from the origin of the request, so a second door is a
second URI: ABS_EXTRA_BASES (default the guarded vhost) now contributes a
callback each, while the mobile-redirect URI stays on the LAN origin - the phone
is a non-browser client and cannot solve the guard's challenge, so it is
configured with the LAN address. Measured on the live provider: three URIs,
grants and scopes unchanged.
2026-11-03 00:55:02 -08:00
Frank 9e2584ec8b feat(homepage): point the Audiobookshelf tile at the vhost
The tile carried the bare LAN IP. The probe stays the LAN host:port so the
homepage checker keeps measuring the app rather than the auth chain in front of
it.
2026-11-03 00:55:02 -08:00
Frank a5dfc1ee99 feat(caddy): serve Audiobookshelf at a guarded abs.franksdank.cloud
Provider-first exposure: the IdP row (proxy provider abs-proxy + application
"Audiobookshelf Guard", claimed by the embedded outpost) already exists, so the
route can ship. The site block proxies everything to anubis:8923 like every other
guarded vhost, and the post-auth :8017 handler routes the host to
192.168.0.235:8000. The container carries ROUTER_BASE_PATH=/audiobookshelf, so
the app has no root: the bare / is redirected onto the base path inside the
post-auth handler, never pre-auth, so an anonymous client only ever meets the
gate.
2026-11-03 00:55:02 -08:00
Frank 7e72126a07 docs(audiobookshelf): an SSO identity cannot land on an existing account
authOpenIDMatchExistingBy is unset on this deployment, so ABS matches an
OpenID userinfo on sub only and auto-registers everyone else. A new authentik
username that equals an existing ABS username therefore dies on the unique
constraint inside the callback, reported only as 'Unauthorized' - which is
exactly what a future agent would hit by 'just creating a frank user' in the
IdP to reach the root account. Records the two real doors (match by username,
which links and keeps root, since setUserGroup refuses to downgrade a root;
or a non-colliding name), and corrects the admin-membership row: akadmin is
the operator's own authentik account, already granted site admin.
2026-11-02 20:29:30 -08:00
dank 80a65b013f Merge pull request 'fix(authentik): provision the Audiobookshelf provider with grant types and OpenID scopes' (#129) from fix/abs-grant-types into main 2026-11-02 20:24:53 -08:00
Frank 54ec72da17 docs(audiobookshelf): the two IdP laws the dead login taught, and the gate
Records that an ORM-created provider has no grant types and that the claim
mapping is not the whole property_mappings list, with the failure signatures
to recognise; names qa-e2e-admin as the admin-group fixture and the new
Playwright spec as the live verification.
2026-11-02 20:21:21 -08:00
Frank d82b8efdd7 test(e2e): gate the Audiobookshelf SSO round trip end to end
The service answers on a LAN address with no vhost, so it was in no
guarded-vhost list and no spec touched it: every deploy gate passed while its
login was dead. The gate signs in as qa-e2e-admin and asserts the two
consumer-visible outcomes that the two defect classes break - every .js the
SPA loads arrives as JavaScript (a bundle built for the wrong base path comes
back text/html and boots nothing), and /api/me answers 200 with type 'admin'
(true only while grant types, the OpenID scope mappings and the group claim
all hold). Joined to the admin project, so it rides every deploy e2e gate.
2026-11-02 20:21:21 -08:00
Frank 27b3c24065 fix(authentik): provision the Audiobookshelf provider with grant types and OpenID scopes
The provider row this script creates was missing two fields, and each one is
a dead login that names nothing on the wire (both measured 2026-11-02 in the
browser):

* grant_types defaults to [] on an ORM-created provider in authentik 2026.8,
  so /application/o/authorize/ refused the code flow with oauthlib's generic
  'The request is otherwise malformed'; the log line that names the cause is
  'Invalid grant_type for provider'. Set authorization_code + refresh_token,
  and reconcile it on re-run.
* property_mappings held only the abs_groups claim mapping, so the granted
  token was scoped shorter than the scopes ABS requests and ABS's /userinfo
  call died 'insufficient_scope' after a good code exchange. Attach the
  built-in OpenID openid/profile/email mappings beside the claim.

Also provisions qa-e2e-admin into the admin group: it is the identity the new
Playwright gate signs in as, so a blueprint re-import restores the fixture.
2026-11-02 20:21:21 -08:00
dank 7e9bfbc2be Merge pull request 'fix(audiobookshelf): serve under /audiobookshelf, send the OIDC redirect subfolder' (#128) from fix/abs-base-path-sso into main 2026-11-02 20:02:57 -08:00
Frank f5410bd253 docs(audiobookshelf): the probe routes answer at the root under either base path
Measured on both shapes: ping returns success:true and healthcheck returns OK at the root whether or not the app is mounted under a prefix, which is why the phone app's connectivity check works before it ever resolves the base path. The paragraph now says which routes were probed instead of asserting 200s in general, and the frontmatter description fits the OKF 120-char budget the linter enforces.
2026-11-02 19:53:16 -08:00
Frank 98522cecba docs(jellyfin): correct a future-dated measurement stamp 2026-11-02 19:50:44 -08:00
Frank 3138e782bd docs(audiobookshelf): the base-path law, the SSO contract, the phone address
The page owned the URL as a plain host:port; every client now carries the /audiobookshelf prefix. Added the two laws a next operator needs (why the base path is not a choice, why the redirect subfolder must be sent) and the Single sign-on section naming which script owns which half.
2026-11-02 19:50:44 -08:00
Frank 393b77dfd2 feat(audiobookshelf): provision authentik SSO and send the redirect subfolder
The live round trip died on authentik with Redirect URI Error. ABS builds redirect_uri as <origin><authOpenIDSubfolderForRedirectURLs>/auth/openid/callback, and that key is unset on a database created before SSO (the 2.17.4 migration only fills a DB that already had OIDC enabled), so the URL carried the literal string undefined. Arming through the API therefore has to send the value; it is the mounted base path, so it is derived from ABS_URL rather than becoming a second knob an operator has to keep in sync.

authentik-audiobookshelf-sso.sh is the IdP half kept in git: provider, groups, the abs_groups scope mapping and the redirect URIs. That state lives in authentik's database, and a blueprint re-import reverts hand-made rows silently, so it needs a re-runnable door. Tokens are HS256 because no provider on the fleet carries a signing key, which makes the client secret the signature key.

The stub server now records the auth-settings PATCH body, so the seam asserts the subfolder that reaches the wire follows ABS_URL for both the prefixed and root shapes, and a second seam pins that the provider script and the bootstrapper build the same redirect path.
2026-11-02 19:50:44 -08:00
Frank cfdd3435bc fix(audiobookshelf): mount the app at the base path its SPA was built for
ROUTER_BASE_PATH=/ served a healthy API and a web UI that could never boot. The index.html in the pinned image is built with routerBasePath:/audiobookshelf, so every script request fell through to the SPA fallback and came back as text/html, which Chrome refuses to execute. healthcheck, the API and the phone app all stayed fine, which is exactly how this hid behind an API-only deploy check.

The deployer now reads the mounted base path out of the unit and hands it to the bootstrapper as ABS_URL, so the API door, the login route and the OIDC redirect subfolder cannot drift from the unit. healthcheck answers at the root under either base path, so the health gate is unchanged.
2026-11-02 19:50:44 -08:00
dank f3a6b4f715 Merge pull request 'docs(jellyfin): the recursive Movie list is not proof an item is indexed' (#126) from docs/jellyfin-verification-read into main 2026-11-02 19:14:33 -08:00
dank 5c888ff948 Merge pull request 'docs(audiobookshelf): the phone app's connection facts and where its token lives' (#127) from docs/abs-phone-app into main 2026-11-02 19:11:23 -08:00
Frank 03e0d84380 docs(audiobookshelf): the phone app's connection facts and where its token lives
The page told an agent how to sign in from a browser and nothing about the phone.
Measured 2026-11-03 on the deployed service:

- Server Address `http://192.168.0.235:8000`, user `frank`, plain HTTP, no base-path
  prefix; `/healthcheck` and `/ping` answer 200 from a second host on the LAN and
  `iptables -S INPUT` has no rule for 8000, so no phone-side setup is needed.
  `download: true` on the root account is what enables offline listening.
- `POST /login` puts the bearer at `user.token`, not top level. Reading
  `response.token` yields "", the client sends `Authorization: Bearer ` with nothing
  after it, and every `/api/*` answers 401 as though the password were wrong — which
  it was not. That is the failure I hit inside five minutes of scripting this.
- The LAN-only law gains its consequence for clients: the Anubis + Authentik chain
  serves browsers, so guarding a vhost does not make the phone app work remotely.
  Remote listening wants a VPN into the LAN; the LAN address is then still correct.

bats: 822 seams green.
2026-11-02 19:09:23 -08:00
Frank a5603d8f01 - 2026-11-02 18:52:13 -08:00
dank 988c4899d4 Merge pull request 'docs(media): point plan/journal/park paths at .media-campaign and NeedsProcessing' (#125) from docs/media-artifact-paths into main 2026-11-02 18:42:34 -08:00
Frank 77e9973ff5 docs(media): point plan/journal/park paths at .media-campaign and NeedsProcessing
The campaign-artifact home moved (42bc482) but the copy-paste surfaces still
named the old ones, and a tool's usage block is exactly what an agent runs:

- scripts/media-shows.py, media-tv.py, media-tvdupes.py: usage examples wrote
  plans and journals at the array root. media-apply.py defaults its journal to
  <root>/media-relayout.jsonl, so any run copied from those lines re-creates
  the root-level clutter the vault now forbids. Each block now names
  /mnt/entropy/.media-campaign/ and says why --journal is explicit.
- media-shows.py: --park named the retired media-park; the pile is
  /mnt/entropy/NeedsProcessing/TVShows.
- Jellyfin.md: library-coverage law named /data/media-park; names the pile by
  its current path and records when it moved.
- Media TV Relayout.md: the runnable --park path named media-park.

bats: 822 seams green.
2026-11-02 18:02:48 -08:00
Frank 625263dd48 Merge remote-tracking branch 'origin/main' into feat/jellyfin-provider-keys 2026-11-02 17:44:49 -08:00
Frank 42bc482ed8 docs(media): campaign artifacts live in /mnt/entropy/.media-campaign
The array root carried 48 planning artifacts (journals, plans, run logs)
accumulated by the relayout campaigns since 2026-11-02. They sit beside the
libraries now, which is how a media server ends up indexing a journal.

Moved all 48 (219.8 MiB) into the dot-directory `/mnt/entropy/.media-campaign`
with a journaled `media-apply` run (plan, apply, verify — 48 verified at their
targets, 0 root files left). Jellyfin and Plex ignore dot-directories, and the
overlay view exposes the array root to both servers, so the same pollution
existed on the view side.

Every vault page that named a root-level journal path is rewritten to the new
home (26 occurrences across 6 pages), and the law is stated once, in
[[Media Relayout]], which owns the tool chain: `media-apply.py` defaults its
journal to `media-relayout.jsonl` beside `--root`, so a run that omits
`--journal` recreates exactly the root-level log this now forbids.
2026-11-02 17:44:49 -08:00
dank aea24353ad Merge pull request 'docs(jellyfin): the switches live under LibraryOptions; a pinned art pass is not a provider census' (#124) from feat/jellyfin-provider-keys into main 2026-11-02 17:39:21 -08:00
Frank d1493de31a Merge remote-tracking branch 'origin/main' into feat/jellyfin-provider-keys
# Conflicts:
#	serverAppConfigs-docs/Dankserver/Audiobookshelf.md
#	serverAppConfigs-docs/Dankserver/Jellyfin.md
2026-11-02 15:04:46 -08:00
Frank 93725f42b3 docs(jellyfin): correct the posture table and three API/art laws
- the per-library switches are under LibraryOptions, not top-level DTO keys
- Shows was flipped on 2026-11-02; Music is the only silent file index left
- never pin Providers= on a TV art loop: the TVDB-only offers disappear
- /Items?Ids= needs no UserId where /Items/{id} does, so verify writes there
2026-11-02 15:03:48 -08:00
dank f80148c5db Merge pull request 'fix(media-apply): refuse a plan row whose verb the tool does not know' (#123) from fix/media-apply-unknown-action into main 2026-11-02 14:07:55 -08:00
dank 541067e6b2 Merge pull request 'docs(tv): split the TV arm into its own page, add the folder-identity laws' (#122) from docs/tv-folder-identity-law into main 2026-11-02 14:03:56 -08:00
Frank 8f0b1982c2 docs(tv): the parked-folder merge, and the laws that gate it
A parked folder whose name parses to a registered show merges into the row
that owns the show. The proof is the episode rows standing under the owner
folder — never the id-less folder list, because one tvdb id can carry two
`Series` rows and an id-less folder that parses gets identified on its own
and keeps its own folder, offering the show twice.

No library file moves while `Scan Media Library` is Running: wait for a
Running sighting and then two consecutive Idle polls, because a gap between
sub-tasks reads as finished. Both sides speak different path dialects (the
server records the container coordinate, the journal speaks the array), so a
join that forgets the translation reads as an empty library.

Lands on [[Media TV Relayout]] — the page that owns the TV arm — so
`Media Relayout.md` stays under the OKF ceiling.
2026-11-02 13:56:03 -08:00
Frank a405c5ed35 fix(media-apply): refuse a plan row whose verb the tool does not know
read_plan's if/elif chain had no else, so any row that was not mkdir, keep
or move was dropped in silence. A plan of nothing but typos therefore
applied as a successful no-op: on 2026-11-02 a pass meant to rename two
TVShows series folders reported "applied: 0 moved" and exit 0, and the
journal was never written — nothing told the operator the plan had never
been read. An unread row is a plan bug, and every other plan bug in this
tool refuses before the first rename, so now this one does too, naming the
line and the verb.

cwd: /home/blank/.omp/wt/wt-20261030-163847-6d1fc1f · session: 01a18f2e-18b1-7573-bcd5-ac99636f8bcb
2026-11-02 09:51:53 -08:00
Frank b232033b11 docs(tv): split the TV arm into its own page, add the folder-identity laws
Media Relayout.md crossed the OKF 500-line ceiling with the 2026-11-02 TV
laws, and the ceiling was right: the TV arm reads a different authority and
refuses for different reasons, like the music and audiobook arms that already
have pages. The arm moves verbatim to Media TV Relayout.md and gains two laws
from the TVShows pass: a folder naming two series is not evidence of two
series (ffprobe's embedded title tag answers per file), and right numbering
with a wrong name means rename the folder only. The executor page keeps the
executor laws and gains the hold-outside---root/--force law; Jellyfin.md gains
the settle read (ScheduledTasks State, since /Library/ScanStatus is a 404 on
12.4). Index regenerated with gen-vault-index.py.

cwd: /home/blank/.omp/wt/wt-20261030-163847-6d1fc1f · session: 01a18f2e-18b1-7573-bcd5-ac99636f8bcb
2026-11-02 09:06:22 -08:00
dank 4a33898a6c Merge pull request 'fix(jellyfin): anchor the provider-key write on the file’s real root' (#121) from chore/jellyfin-tvdb-key into main 2026-11-02 07:17:25 -08:00
dank 21c98bd485 Merge pull request 'docs(jellyfin): the door that scans is /Library/Refresh, and the art door names its field' (#120) from docs/jellyfin-scan-door into main 2026-11-02 07:16:52 -08:00
dank a651c81dfb Merge pull request 'feat(audiobookshelf): bootstrap the account, library and scan from the deployer' (#119) from feat/audiobookshelf-bootstrap into main 2026-11-02 07:16:45 -08:00
Frank 0951d909c9 docs(agents): the bracket trick for a pkill pattern that must be named
The existing rule says don't `pkill -f` a pattern your own command line carries,
and to prefer `pkill -x` — but a remote one-shot often has to name the pattern,
and hitting it again today cost a silent `ssh` exit 255 with zero output, which
reads like a network fault. Record the escape (`"serve[-]script.sh"`), the zsh
glob-expansion trap that makes an unquoted `[s]` a no-op, and the exit-code
signature.
2026-11-02 04:47:16 -08:00
Frank 69effb542d docs(jellyfin): Shows is on providers, and the per-library door stays dead anyway
The `Shows` flip landed today (one field, proven in the server's own
`LibraryOptions` answer), so the posture table no longer calls it a silent file
index and `Music`/`Collections` are named as the remaining ones.

Two corrections the same day's measurements force:

- The per-library `POST /Items/{id}/Refresh` answered `204` and queued nothing
  both before the flip and after it — zero journal lines for ten minutes, and
  `Scan Media Library` kept `LastTaskResult=None, EndTimeUtc=None`, the never-run
  signature. The provider switch was never the reason that door is dead; only
  `POST /Library/Refresh` runs (204, then `Running` inside 15 s at 33 % CPU), and
  it walks every library, so a library-scoped pass has no door on this build.
  That paragraph also records the null-pair read that separates "never ran" from
  "just finished", and that query+body on the same call answers `400` with an
  empty body whatever the values.
- The key law: a plugin owns its own config file under its own
  `<PluginConfiguration>` root and omits null settings, so TVDB's `<ApiKey>`
  element never existed and `install` refused — which is what `MISSING key` had
  been reporting since October. PR #121 fixes the writer.

The `Shows` art census replaces speculation with the split that matters: 2,908 of
3,558 series+season carry a primary, 624 have no provider id, 26 have an id and no
poster. A `Download` campaign over them took 20 s and downloaded nothing, which is
the measurement that says identification is the only lever left.
2026-11-02 04:47:16 -08:00
Frank ad9563412d docs(audiobookshelf): what a residual is made of
Four laws from finishing the un-shadow, correcting one:

- A book whose files ALL fail to probe never becomes an item, so a disk-vs-DB
  folder gap beside `Invalid data` lines is corrupt media, not a lost book. A
  format-only ffprobe clears files the scanner rejects — do not trust it.
- `Disc 01`/`CD 1` subfolders are parts of one item (110 of them here, under a
  192-track book); the extension census counts them, ABS does not.
- `.cbr`/`.cbz` group like media but `parseComicMetadata` throws `ERAR_BAD_ARCHIVE`,
  so those items land media-less: the file, not the scanner.
- Scan completion is a NEW scan uuid; `podman logs --since` is a sliding window
  and ages its own evidence out.
- `?force=1` is now recorded as neither discovery nor repair: a forced full
  re-scan left item count, path set and media-less count identical.

Verification names the five book folders that need re-acquiring.
2026-11-02 03:59:14 -08:00
Frank f71c7ccce4 docs(audiobookshelf): the grouping rule, not "shallowest folder with audio"
Corrects the law pushed earlier today. The mechanism is
`groupFileItemsIntoLibraryItemDirs` (utils/scandir.js), called on a live
`recurseFiles` walk of the tree EVERY scan: each file attaches to the FIRST
directory along its path that already has a group, so one media file at a
collection root owns everything beneath it. Measured: one 949 MB `.cbr` hid
652 book folders behind a single 5,693-track item.

- "Media file" is audio OR ebook (`audiobooksOnly=false` -> SupportedAudioTypes
  ∪ SupportedEbookTypes, 17 audio + 6 ebook extensions). An audio-only census
  cannot see the file that hides an Audiobooks library.
- Deleting the covering item does not fix a shadow: grouping is recomputed from
  disk and the next scan rebuilds the item (measured: `1 Added`, same path).
- Un-shadow by MOVE: one file per folder is ABS's own single-book layout.
- `?force=1` is not a discovery arm (both arms walk the disk; force re-scans
  every existing item, 1:33 against 0:04).
- Paging corrected: `offset` is ignored outright, `page` is the parameter.
- `DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()` of the whole
  swallowed subtree. Never pass `hard`.

Shape table gains the layout row and the move journal; the stale "1,103
shelves" scan count is destroyed.
2026-11-02 03:43:34 -08:00
Frank f541566714 docs(audiobookshelf): a book is the shallowest audio folder, and it owns the subtree
Measured on the live library: 6 stray files at the root of _unsorted made the
whole collection dump ONE item of 5,693 tracks, with 542 real books folded into
it; no scan subdivides an existing item. Also pinned: the items endpoint hands
back ceil(total/limit)*limit rows because an out-of-range offset repeats the
tail (that is where "exactly 1,500" and "exactly 4,000" came from), and
DELETE /api/items/:id?hard=1 runs fs.remove() on the item's folder - pointed at
a covering item that is the whole subtree, so un-swallowing is a DB-only delete
after the stray root files are filed.
2026-11-02 03:34:28 -08:00
Frank 006bbcef28 docs(audiobookshelf): paging the items endpoint never terminates itself
Two censuses this session reported exactly 1,500 and exactly 4,000 books for one
library mid-scan. Cause: an out-of-range offset returns the last item instead of an
empty page, so a loop written "until the page is empty" manufactures its own
total, and total itself moves while a scan is running. Stop on total; count after
the pass settles.
2026-11-02 03:18:04 -08:00
Frank 8889a4ceff docs(jellyfin): episode stills are out of scope, and the cost that says so
A per-episode RemoteImages query is never cached, so the same campaign that
dressed 620 series in an afternoon costs ~11 s per episode for one hit in five.
Across 42k episodes that is days of provider traffic to dress a fifth of them, and
clients show series and season art regardless. Recorded so nobody runs the pass on
a hunch; the door and the resumable log are the same if Frank orders it.
2026-11-02 03:00:42 -08:00
Frank 56dfc6d0b9 docs(audiobookshelf): the scanner refusal names the file, not the bind
The first full scan logged "Invalid data found when processing input" per file and
carried on. That line is about the file: the scanner skips it, so the book never
appears in the library at all. Census of the 43 refused files: 38 were 0 bytes, 3
held WAV bytes behind a .mp3 name, 2 were unreadable. The check to run first is a
byte count, not a server investigation.

Full suite: 818 tests, exit 0.
2026-11-02 02:57:32 -08:00
Frank f1ae1ddb36 docs(audiobookshelf): the scanner refusal names the file, not the bind
The first full scan logged "Invalid data found when processing input" per file and
carried on. That line is about the file: the scanner skips it, so the book never
appears in the library at all. Census of the 43 refused files: 38 were 0 bytes, 3
held WAV bytes behind a .mp3 name, 2 were unreadable. The check to run first is a
byte count, not a server investigation.

Full suite: 818 tests, exit 0.
2026-11-02 02:57:25 -08:00
Frank c21c48e61e docs(jellyfin): the door that scans is /Library/Refresh, and the art door names its field
The page sent the next agent to POST /Items/{id}/Refresh?Recursive=true per
library to make a new folder visible. Measured against a Shows library that had
never been identified: that route answers 204 and queues nothing — item census
byte-identical, ScheduledTasks idle, journal silent, and still a silent 204 after
a service restart. The door that works is POST /Library/Refresh, and the progress
read is ScheduledTasks -> Scan Media Library -> CurrentProgressPercentage.
RefreshStatus on the VirtualFolders record stays Idle through a real scan, so it
is not a progress read at all; a 204 cannot distinguish accepted-and-discarded
from accepted-and-queued, so the acceptance read is the journal naming folders.

Also recorded the art door's wire shape, because guessing it reads as a server
bug: the list response carries the address in Url (there is no ImageUrl), the
Download POST takes camelCase type/imageUrl and answers 204, and PascalCase or an
empty imageUrl answers 500. RemoteImages is empty for items with no ProviderIds,
so art coverage is capped by identification, not by the image door.
2026-11-02 02:56:12 -08:00
Frank 22cd17d064 docs(jellyfin): the door that scans is /Library/Refresh, and the art door names its field
The page sent the next agent to POST /Items/{id}/Refresh?Recursive=true per
library to make a new folder visible. Measured against a Shows library that had
never been identified: that route answers 204 and queues nothing — item census
byte-identical, ScheduledTasks idle, journal silent, and still a silent 204 after
a service restart. The door that works is POST /Library/Refresh, and the progress
read is ScheduledTasks -> Scan Media Library -> CurrentProgressPercentage.
RefreshStatus on the VirtualFolders record stays Idle through a real scan, so it
is not a progress read at all; a 204 cannot distinguish accepted-and-discarded
from accepted-and-queued, so the acceptance read is the journal naming folders.

Also recorded the art door's wire shape, because guessing it reads as a server
bug: the list response carries the address in Url (there is no ImageUrl), the
Download POST takes camelCase type/imageUrl and answers 204, and PascalCase or an
empty imageUrl answers 500. RemoteImages is empty for items with no ProviderIds,
so art coverage is capped by identification, not by the image door.
2026-11-02 02:56:03 -08:00
Frank 48ef6d34cd docs(audiobookshelf): the first run is a script, and its re-run is the scan trigger
The page claimed the account can only be created from a LAN address; measured,
the door is POST /init and it answers plain text. Replaced with the bootstrap the
deployer now runs, and with the doors a consumer actually needs: /status reports
isInit with no /api prefix, the item count rides /api/libraries/{id}/items ->
total (the library DTO has no totals), and the list DTO carries media.numTracks
while /api/items/{id} returns media.audioFiles.

New law: the folder watcher cannot cross the rootless podman bind — the same gap
as Jellyfin's auto-scan note — so files added on the host stay invisible until a
scan runs, and the deployer's re-run of the bootstrapper is the cheap trigger.

Testing Policy lists the new seam; index.md regenerated from frontmatter.
2026-11-02 02:55:55 -08:00
Frank 317f36b456 feat(audiobookshelf): bootstrap the account, library and scan from the deployer
A freshly deployed Audiobookshelf answers /healthcheck and is otherwise dead: no
account, no library, no scan. The UI registration step is gone by design — the
doors are POST /init (200 with plain text "OK", not JSON) and /status (isInit),
and there is no /setup route. That is an API-only bootstrap, so it must be code.

scripts/audiobookshelf-init.sh is idempotent: /status -> /init when unregistered,
POST /login for the bearer, POST /api/libraries for the Audiobooks library on
/audiobooks, then POST /api/libraries/{id}/scan. The deployer runs it only after
the health gate has answered.

Credentials live in ~/.config/secrets/audiobookshelf (mode 600): the bearer rides
an Authorization header and the password only a POST body — never a URL or argv,
where any ps on the host would print them. The seam drives an inline stub server
that logs every request line, and asserts: fresh bootstrap creates account,
library and scan; a second run creates nothing new and still fires the
incremental scan; no secret appears in any logged request line; a server that
refuses is reported as "refused: <url>" with exit 1 rather than success; ABS_SCAN
outside 0|1 fails before a single request; the deployer orders init after health.

Measured on Dankserver: run against the already-bootstrapped host it is a no-op
that re-fires the scan; the library then scans (1,500+ books, every item with
audio tracks, none invalid).
2026-11-02 02:55:50 -08:00
31 changed files with 1894 additions and 260 deletions

No files matched your search

+6 -1
View File
@@ -735,7 +735,12 @@ linked runbooks.
**Ops**
- Never `pkill -f` a pattern that appears in your own command line — it kills your ssh session.
Use `pkill -x <exact-name>`.
Use `pkill -x <exact-name>`, or the bracket trick — `pkill -f "serve[-]script.sh"` —
when the pattern has to be named: the bracket makes the pattern stop matching the
command line that carries it. Quote it: a zsh remote glob-expands an unquoted `[s]`
and the kill silently never runs. Re-hit 2026-11-02 —
`ssh host 'pkill -f jfshowswatch.py; …'` killed the remote shell mid-command, and
ssh exits **255 with zero output**, which reads exactly like a network fault.
- Both hosts allow passwordless sudo for the deploy user. Use plain `sudo` in committed scripts
(no `echo pw | sudo -S`); `FRANK_SUDO_PW` covers hosts without it.
- **ComfyUI on Dankclaw runs with GPU access** (`/dev/kfd` + `/dev/dri` via `AddDevice`,
+32
View File
@@ -264,6 +264,20 @@ files.franksdank.cloud {
}
}
# --- Audiobookshelf (guarded) ---
# Anubis → Authentik → app, like romm/files. The container carries
# ROUTER_BASE_PATH=/audiobookshelf, so this vhost serves the app under that
# path; the bare root is redirected onto it in the post-auth listener, never
# pre-auth. Guard state in the IdP: proxy provider `audiobooks-proxy`
# (external_host https://audiobooks.franksdank.cloud/, internal_host
# http://caddy:8017, application "Audiobookshelf Guard"), claimed by the
# embedded outpost.
audiobooks.franksdank.cloud {
reverse_proxy anubis:8923 {
header_up X-Real-Ip {remote_host}
}
}
# --- Jellyfin ---
jelly.franksdank.cloud {
reverse_proxy 192.168.0.235:8096
@@ -410,6 +424,24 @@ http://:8017 {
reverse_proxy 192.168.0.235:3923
}
@audiobooks host audiobooks.franksdank.cloud
handle @audiobooks {
# The app answers only under its ROUTER_BASE_PATH; the bare root is a
# convenience hop, taken post-auth so nothing reaches the app
# pre-authentication.
redir / /audiobookshelf/ permanent
# Audiobookshelf validates every OIDC callback against the origin it
# believes it is served from — getRequestOrigin() is
# x-forwarded-proto://Host — and a mismatch is a bodiless 400
# "Invalid callback URL - must be same-origin" before the IdP is ever
# contacted (measured 2026-11-03). This listener is fed over plain http
# by the outpost, so Caddy's own X-Forwarded-Proto here says http; the
# edge for this name is TLS-only, so https is the truthful value.
reverse_proxy 192.168.0.235:8000 {
header_up X-Forwarded-Proto https
}
}
@prowlarr host prowlarr.franksdank.cloud
handle @prowlarr {
reverse_proxy 192.168.0.112:9696
+1
View File
@@ -26,6 +26,7 @@
192.168.0.143 romm.franksdank.cloud
192.168.0.143 files.franksdank.cloud
192.168.0.143 jelly.franksdank.cloud
192.168.0.143 audiobooks.franksdank.cloud
192.168.0.143 radarr.franksdank.cloud
192.168.0.143 lidarr.franksdank.cloud
192.168.0.143 readarr.franksdank.cloud
@@ -33,9 +33,16 @@ Network=pasta
# README is the npm package's naming and is INERT here: with ABS_PORT=8000 the
# server listened on :80 under base path /audiobookshelf, so the unit sat
# `active (running)` while every probe to :8000 was refused (measured 2026-11-02,
# first deploy). ROUTER_BASE_PATH=/ is what makes the URL the plain host:port.
# first deploy).
# ROUTER_BASE_PATH MUST be /audiobookshelf: the SPA in /app/client/dist/index.html is
# built with `routerBasePath:"/audiobookshelf"` baked in, so it asks for its assets under
# that prefix no matter what the server is mounted at. With `/` the server answers the
# page but every script request falls through to the SPA fallback and comes back as
# text/html — Chrome refuses to execute it and the UI never leaves "Loading…"
# (measured 2026-11-03; the API and /healthcheck were fine, which is why an API-only
# verification missed it).
Environment=PORT=8000
Environment=ROUTER_BASE_PATH=/
Environment=ROUTER_BASE_PATH=/audiobookshelf
Environment=CONFIG_PATH=/config
Environment=METADATA_PATH=/metadata
Environment=TZ=America/Vancouver
@@ -45,6 +52,8 @@ Volume=/home/dank/Containers/audiobookshelf/metadata:/metadata
Volume=/mnt/entropy/Audiobooks:/audiobooks:ro
# The image is Alpine: it ships wget and no curl, so the health check must name
# the binary that exists or the container is unhealthy while serving perfectly.
# /healthcheck answers `OK` at the root AND under the base path (measured on both
# base paths), so the probe below is correct whatever ROUTER_BASE_PATH says.
HealthCmd=wget -q -O /dev/null http://127.0.0.1:8000/healthcheck
HealthInterval=30s
HealthTimeout=10s
+309
View File
@@ -0,0 +1,309 @@
#!/usr/bin/env bash
# audiobookshelf-init.sh — make an Audiobookshelf server usable, from nothing.
#
# Why this is a script and not a browser session (live 2026-11-02, Dankserver):
# a freshly deployed Audiobookshelf answers /healthcheck with 200 while it has no
# user, no library and no scan — "healthy" says nothing about usefulness, and a
# service that only works after someone remembers to click through a first-run
# wizard is not deployed, it is installed. Everything below is the API the React
# client itself calls, so the same procedure runs unattended and lands in git.
#
# The API facts this script depends on, read off the pinned image rather than
# guessed (dist-server/server/Server.js inside the digest this repo pins):
# * the creation route is POST /init with {"newRoot":{"username","password"}}.
# There is NO /setup route in 2.37.x — POST /setup answers "Unauthorized",
# which reads exactly like an auth fault and is instead a missing route.
# * /status (no /api prefix) reports {"isInit": <has a root user>}; every route
# under /api needs the bearer token that POST /login returns.
# * the library count is NOT a field on the library DTO; the door is
# GET /api/libraries/{id}/items -> {"total": N}.
#
# The credential is generated once into a mode-600 host file and never enters
# the repo, the URL, or a process argument list. Same pattern as calibre-server.
#
# Single sign-on (authentik) is armed by the same run, and only when the credential
# file carries ABS_OIDC_CLIENT_SECRET, so a local-only deployment is unchanged by a
# re-run. These OIDC facts came from the same pinned image, and each one is a trap:
# * isOpenIDAuthSettingsValid demands issuer + authorization + token + userinfo +
# jwks + clientID + clientSecret + signingAlgorithm at once. Miss one and the
# server SILENTLY strips 'openid' from authActiveAuthMethods and falls back to
# ['local'] while answering 200 everywhere — so the only honest check is what
# /status says AFTER the write, which is what this script does.
# * the algorithm is HS256, not the image's RS256 default: no provider on the
# fleet's authentik carries a signing key, so the client secret IS the signature
# key (scripts/authentik-audiobookshelf-sso.sh). Left at RS256, every id_token
# fails verification and the failure looks like a broken IdP.
# * the auth block lives on PATCH /api/auth-settings. PATCH /api/settings has a
# 20-key allowlist (PATCHABLE_SETTINGS_KEYS) with NO auth key in it, so writing
# the OIDC config there answers 200 and changes nothing — measured on this host
# 2026-11-02. The keys /api/auth-settings accepts are exactly its own
# authenticationSettings getters; anything else is skipped with a debug line.
# * authOpenIDSubfolderForRedirectURLs must be sent explicitly. The 2.17.4 migration only
# fills it on a DB that ALREADY had OIDC on, so a database created before SSO leaves it
# UNDEFINED and the server builds redirect_uri
# "http://host/undefined/auth/openid/callback" — authentik answers the authorize request
# with "Redirect URI Error" (measured 2026-11-02). This script derives the value from
# ABS_URL's path, so it cannot drift from the unit's ROUTER_BASE_PATH.
# * 'local' stays in authActiveAuthMethods on purpose: if authentik is ever down,
# the root account must still be able to sign into its own server.
#
# usage: audiobookshelf-init.sh # create what is missing, then scan
# ABS_SCAN=0 audiobookshelf-init.sh # bootstrap without a scan
# ABS_OIDC=0 audiobookshelf-init.sh # never touch the auth methods
#
# environment (defaults are the deployed shape):
# ABS_URL server base URL INCLUDING the mounted base path
# (http://127.0.0.1:8000/audiobookshelf)
# ABS_SECRET credential file (/home/dank/.config/secrets/audiobookshelf)
# ABS_LIBRARY library name to ensure (Audiobooks)
# ABS_FOLDER folder to bind inside the server (/audiobooks)
# ABS_SCAN 1 triggers a scan (default), 0 skips it
# ABS_OIDC auto (default) arms SSO when the credential file has a client
# secret, 1 requires one, 0 leaves the auth methods alone
# ABS_OIDC_ISSUER authentik issuer URL for this client
# ABS_OIDC_CLIENT_ID provider client id (audiobookshelf)
# ABS_OIDC_ALG id_token signing algorithm the IdP actually uses (HS256)
# ABS_OIDC_SUBFOLDER override for the redirect-URI subfolder (default: ABS_URL's path)
#
# exit 0 every step is satisfied (created or already present)
# exit 1 a step was refused or the server misbehaved (named on stderr)
# exit 2 usage / unreadable credential file
set -euo pipefail
ABS_URL=${ABS_URL:-http://127.0.0.1:8000/audiobookshelf}
ABS_SECRET=${ABS_SECRET:-/home/dank/.config/secrets/audiobookshelf}
ABS_LIBRARY=${ABS_LIBRARY:-Audiobooks}
ABS_FOLDER=${ABS_FOLDER:-/audiobooks}
ABS_SCAN=${ABS_SCAN:-1}
ABS_OIDC=${ABS_OIDC:-auto}
ABS_OIDC_ISSUER=${ABS_OIDC_ISSUER:-https://auth.franksdank.cloud/application/o/audiobookshelf/}
ABS_OIDC_CLIENT_ID=${ABS_OIDC_CLIENT_ID:-audiobookshelf}
ABS_OIDC_ALG=${ABS_OIDC_ALG:-HS256}
die() { printf 'refused: %s\n' "$*" >&2; exit 1; }
case "$ABS_SCAN" in
0 | 1) ;;
*) printf 'refused: ABS_SCAN wants 0 or 1, got %s\n' "$ABS_SCAN" >&2; exit 2 ;;
esac
case "$ABS_OIDC" in
auto | 0 | 1) ;;
*) printf 'refused: ABS_OIDC wants auto, 0 or 1, got %s\n' "$ABS_OIDC" >&2; exit 2 ;;
esac
# ---- credential -------------------------------------------------------------
umask 077
if [ ! -f "$ABS_SECRET" ]; then
mkdir -p "$(dirname "$ABS_SECRET")"
user=${ABS_ADMIN_USER:-frank}
# `tr … | head -c 24` is the obvious line and it is a trap under
# `set -o pipefail`: head exits at 24 bytes, tr takes SIGPIPE, the pipeline
# status is 141 and set -e kills the script before the credential is ever
# written (measured 2026-11-02). od reads a FIXED byte count and exits on its
# own, so no consumer closes a pipe early; bash slices the length.
pass=$(od -An -tx1 -N 32 /dev/urandom | tr -d ' \n')
pass=${pass:0:24}
[ "${#pass}" -eq 24 ] || die "could not generate a password (no /dev/urandom)"
printf 'ABS_ADMIN_USER=%s\nABS_ADMIN_PASSWORD=%s\n' "$user" "$pass" > "$ABS_SECRET"
printf 'credential: created %s\n' "$ABS_SECRET"
fi
[ -r "$ABS_SECRET" ] || die "$ABS_SECRET exists but is not readable by $(id -un)"
chmod 600 "$ABS_SECRET"
# ---- everything else, over HTTP ---------------------------------------------
# python3 rather than curl so the password and the bearer token never appear in
# any argv (they would be world-readable in /proc/<pid>/cmdline otherwise).
ABS_URL="$ABS_URL" ABS_SECRET="$ABS_SECRET" ABS_LIBRARY="$ABS_LIBRARY" \
ABS_FOLDER="$ABS_FOLDER" ABS_SCAN="$ABS_SCAN" ABS_OIDC="$ABS_OIDC" \
ABS_OIDC_ISSUER="$ABS_OIDC_ISSUER" ABS_OIDC_CLIENT_ID="$ABS_OIDC_CLIENT_ID" \
ABS_OIDC_ALG="$ABS_OIDC_ALG" python3 - <<'PY'
import json, os, sys, urllib.error, urllib.parse, urllib.request
BASE = os.environ["ABS_URL"].rstrip("/")
LIB = os.environ["ABS_LIBRARY"]
FOLDER = os.environ["ABS_FOLDER"]
SCAN = os.environ["ABS_SCAN"] == "1"
cred = {}
for line in open(os.environ["ABS_SECRET"]):
if "=" in line:
k, _, v = line.strip().partition("=")
cred[k] = v
user = cred.get("ABS_ADMIN_USER") or "frank"
password = cred.get("ABS_ADMIN_PASSWORD") or ""
if not password:
sys.exit("refused: %s has no ABS_ADMIN_PASSWORD" % os.environ["ABS_SECRET"])
def call(method, path, body=None, token=None, want_json=True):
data = None
headers = {"X-Requested-With": "audiobookshelf-init"}
if body is not None:
data = json.dumps(body).encode()
headers["Content-Type"] = "application/json"
if token:
headers["Authorization"] = "Bearer " + token
req = urllib.request.Request(BASE + path, data=data, headers=headers, method=method)
try:
with urllib.request.urlopen(req, timeout=60) as r:
raw = r.read()
if not raw or not want_json:
return r.status, raw
try:
return r.status, json.loads(raw)
except ValueError:
# Express answers sendStatus() with the STATUS TEXT as the body:
# POST /init and the scan route both reply "OK" (measured on the
# pinned 2.37.2 image). A successful call must never be judged by
# whether its body happens to parse as JSON.
return r.status, raw.decode("utf-8", "replace")[:200]
except urllib.error.HTTPError as e:
raw = e.read()
try:
payload = json.loads(raw)
except Exception:
payload = raw.decode("utf-8", "replace")[:200]
return e.code, payload
except urllib.error.URLError as e:
# Not an HTTP answer at all: nothing listening, DNS dead, connection
# refused. HTTPError is a URLError subclass, so this arm must come after.
return 0, "unreachable (%s)" % e.reason
status, st = call("GET", "/status")
if status != 200 or not isinstance(st, dict):
sys.exit("refused: %s/status answered %s (%s) — is the service up?" % (BASE, status, st))
if st.get("isInit"):
print("account: already registered")
else:
code, body = call("POST", "/init", {"newRoot": {"username": user, "password": password}})
if code != 200:
sys.exit("refused: POST /init answered %s (%s)" % (code, body))
print("account: created (%s)" % user)
code, login = call("POST", "/login", {"username": user, "password": password})
if code != 200 or not isinstance(login, dict) or not login.get("user", {}).get("token"):
sys.exit("refused: login answered %s (%s) — the credential file does not match the server"
% (code, login))
token = login["user"]["token"]
print("token: obtained (%s)" % login["user"].get("type", "user"))
code, libs = call("GET", "/api/libraries", token=token)
if code != 200 or not isinstance(libs, dict) or "libraries" not in libs:
sys.exit("refused: GET /api/libraries answered %s (%s)" % (code, libs))
lib = next((l for l in libs["libraries"] if l.get("name") == LIB), None)
if lib:
print("library: exists %s (%s)" % (LIB, lib["id"]))
else:
body = {
"name": LIB,
"provider": "audiobook",
"icon": "book",
"folders": [{"fullPath": FOLDER}],
"settings": {
"coverEncoding": "jpg",
"skipMatchingMediaWithAsin": False,
"skipMatchingMediaWithIsbn": False,
"autoDetectCrossoverAudiobooks": False,
"ignoreAudiobookMetadata": False,
"useAudiobookFileMetadata": False,
},
"providerMetadata": {},
}
code, made = call("POST", "/api/libraries", body, token=token)
if code != 200:
sys.exit("refused: POST /api/libraries answered %s (%s)" % (code, made))
lib = made
print("library: created %s (%s) folder %s" % (LIB, lib["id"], FOLDER))
if SCAN:
code, body = call("POST", "/api/libraries/%s/scan" % lib["id"], {}, token=token,
want_json=False)
if code != 200:
sys.exit("refused: scan answered %s (%s)" % (code, body))
print("scan: accepted")
else:
print("scan: skipped (ABS_SCAN=0)")
code, items = call("GET", "/api/libraries/%s/items?limit=1" % lib["id"], token=token)
if code == 200 and isinstance(items, dict):
print("items: %s (a fresh scan raises this from 0; watch it, not /healthcheck)"
% items.get("total"))
# ---- single sign-on (authentik) --------------------------------------------------
# The provider this points at is provisioned by scripts/authentik-audiobookshelf-sso.sh;
# the two share exactly one value, the client secret in the credential file.
mode = os.environ["ABS_OIDC"]
oidc_secret = cred.get("ABS_OIDC_CLIENT_SECRET") or ""
if mode == "0":
print("oidc: skipped (ABS_OIDC=0)")
elif not oidc_secret:
if mode == "1":
sys.exit("refused: ABS_OIDC=1 but %s has no ABS_OIDC_CLIENT_SECRET — run "
"scripts/authentik-audiobookshelf-sso.sh first" % os.environ["ABS_SECRET"])
print("oidc: not configured (no ABS_OIDC_CLIENT_SECRET in the credential file)"
" — local auth only")
else:
client_id = os.environ["ABS_OIDC_CLIENT_ID"]
issuer = os.environ["ABS_OIDC_ISSUER"].rstrip("/") + "/"
# redirect_uri = <origin><subfolder>/auth/openid/callback, and the subfolder IS the
# base path the app is mounted under — taken from ABS_URL so unit and config agree.
subfolder = os.environ.get("ABS_OIDC_SUBFOLDER")
if subfolder is None:
subfolder = urllib.parse.urlparse(os.environ["ABS_URL"]).path.rstrip("/")
origin = issuer.split("/application/")[0]
body = {
"authActiveAuthMethods": ["local", "openid"],
"authOpenIDIssuerURL": issuer,
"authOpenIDAuthorizationURL": origin + "/application/o/authorize/",
"authOpenIDTokenURL": origin + "/application/o/token/",
"authOpenIDUserInfoURL": origin + "/application/o/userinfo/",
"authOpenIDJwksURL": issuer + "jwks/",
"authOpenIDLogoutURL": issuer + "end-session/",
"authOpenIDClientID": client_id,
"authOpenIDClientSecret": oidc_secret,
"authOpenIDTokenSigningAlgorithm": os.environ["ABS_OIDC_ALG"],
"authOpenIDSubfolderForRedirectURLs": subfolder,
"authOpenIDButtonText": "Sign in with Authentik",
# not auto-launch: the login page keeps offering the local form, which is the
# break-glass path when the IdP is unreachable.
"authOpenIDAutoLaunch": False,
# a household user's first SSO sign-in creates their account; the group claim
# decides the role, and a user in neither group is denied (fail closed).
"authOpenIDAutoRegister": True,
"authOpenIDGroupClaim": "abs_groups",
# The app's own scheme lives HERE, never in the IdP: ABS swaps it for
# <base>/auth/openid/mobile-redirect before the provider sees a redirect_uri.
"authOpenIDMobileRedirectURIs": ["audiobookshelf://oauth"],
}
code, res = call("PATCH", "/api/auth-settings", body, token=token)
if code != 200:
sys.exit("refused: PATCH /api/auth-settings answered %s (%s)" % (code, res))
# Verify twice: what was stored, and what the server will honour. A config that
# misses any of the eight required OIDC values is accepted with 200 and then
# silently dropped at the next load, leaving authMethods=['local'].
code, stored = call("GET", "/api/auth-settings", token=token)
if code != 200 or not isinstance(stored, dict):
sys.exit("refused: GET /api/auth-settings answered %s (%s)" % (code, stored))
for key, want in (("authOpenIDIssuerURL", issuer),
("authOpenIDTokenSigningAlgorithm", os.environ["ABS_OIDC_ALG"]),
("authOpenIDSubfolderForRedirectURLs", subfolder),
("authOpenIDButtonText", "Sign in with Authentik")):
if stored.get(key) != want:
sys.exit("refused: stored %s=%r, expected %r" % (key, stored.get(key), want))
if sorted(stored.get("authActiveAuthMethods") or []) != ["local", "openid"]:
sys.exit("refused: stored authActiveAuthMethods=%r, expected local+openid "
"('openid' is dropped when the block is incomplete)"
% stored.get("authActiveAuthMethods"))
code, after = call("GET", "/status")
methods = after.get("authMethods") if isinstance(after, dict) else None
if not isinstance(methods, list) or "openid" not in methods:
sys.exit("refused: /status reports authMethods=%s after a stored config that "
"looks complete — restart the service, the strategy is only armed at "
"load for a config that was invalid when the server read it" % methods)
print("oidc: armed (issuer %s | alg %s | claim abs_groups | redirect subfolder %r | "
"client %s | secret %s chars | authMethods %s)"
% (issuer, os.environ["ABS_OIDC_ALG"], subfolder, client_id, len(oidc_secret),
methods))
PY
exit 0
+262
View File
@@ -0,0 +1,262 @@
#!/usr/bin/env bash
# authentik-audiobookshelf-sso.sh — provision the authentik side of Audiobookshelf SSO.
#
# Why a script: this state lives in authentik's DATABASE, not in any repo file, and a
# blueprint re-import or a rebuild-from-scratch reverts hand-made provider rows
# silently. Same reasoning (and same door) as
# authentik-set-login-session-duration.sh: ssh into Dankclaw, `ak shell` from stdin.
# Re-run this after such a rebuild.
#
# The contract with Audiobookshelf, read off the pinned image (2.37.2,
# dist-server/server/auth/OidcAuthStrategy.js + objects/settings/ServerSettings.js)
# rather than a forum, because each of these is a dead-login trap:
#
# * The browser callback URI <base>/auth/openid/callback for EVERY origin ABS is
# reached at, plus <base>/auth/openid/mobile-redirect on ABS_BASE only. ABS SWAPS
# the app's own audiobookshelf://oauth for the mobile-redirect URL before the IdP
# ever sees a redirect_uri, so registering the app scheme with authentik — the
# obvious move — is what makes phone logins fail. The app scheme belongs in ABS's
# authOpenIDMobileRedirectURIs list (its built-in default), not here. The mobile
# URI is registered on the LAN origin only because the phone is configured with
# the LAN address: it is a non-browser client and cannot solve the guard
# challenge that sits in front of the vhost.
# * <base> CARRIES THE BASE PATH. ABS builds redirect_uri as
# <origin><authOpenIDSubfolderForRedirectURLs>/auth/openid/callback, and that subfolder is
# the ROUTER_BASE_PATH the container is mounted at — /audiobookshelf, which is also the only
# base path the shipped SPA can boot under. Left unset (the fresh-DB default) the URL
# becomes "/undefined/…"; a provider registered without the prefix mismatches the same way
# (measured 2026-11-02, "Redirect URI Error"). audiobookshelf-init.sh derives its copy from
# ABS_URL, so the two sides each move with one variable.
# * ABS requests the scopes "openid profile email <GroupClaim> [<PermsClaim>]": the
# CLAIM NAME is a scope name. So authentik needs a scope mapping whose scope_name
# is exactly the claim name, or the claim never arrives and the callback dies on
# "Group claim <name> not found in userinfo".
# * the claim's VALUES must be ABS role names — the callback picks the first of
# ['admin','user','guest'] present in the list. Group names like
# audiobookshelf_admins are therefore translated by the mapping, never passed
# through. A user in neither group gets [] and is denied (fail closed).
# * tokens are HS256 here: no provider on this box carries a signing_key (census
# 2026-11-02), which makes the client secret the signature key. ABS's own default
# is RS256, so the ABS side MUST be set to HS256 or every id_token fails
# verification.
# * provider.authorization_flow must be non-NULL or /application/o/authorize/
# answers a 500 with 'NoneType' object has no attribute 'slug' (measured on
# grafana, same authentik build).
# * provider.grant_types MUST contain authorization_code. authentik 2026.8 defaults an
# ORM-created provider to the EMPTY list, and /application/o/authorize/ then refuses the
# code flow with oauthlib's generic "The request is otherwise malformed"; the only line
# that names the cause is the server log's 'Invalid grant_type for provider'. The
# redirect-URI check runs FIRST, so a provider carrying both faults shows only the URI
# error until the URI is fixed (measured 2026-11-02, where the login survived the URI fix
# and died here).
# * property_mappings must carry the built-in OpenID scope mappings ('openid', 'profile',
# 'email') alongside the claim mapping. A provider holding only the claim mapping grants a
# token scoped shorter than the scopes ABS requests, and ABS's /userinfo call then dies
# 'insufficient_scope (The request requires higher privileges than provided by the access
# token)' after authentik has already issued a good code (measured 2026-11-02).
#
# Idempotent: every object is created or asserted. Safe to re-run.
#
# usage: ABS_OIDC_CLIENT_SECRET=<secret> authentik-audiobookshelf-sso.sh
# (the secret is the value stored in
# /home/dank/.config/secrets/audiobookshelf on Dankserver — get it with
# ssh dankserver awk -F= '/^ABS_OIDC_CLIENT_SECRET=/{print $2}' \
# ~/.config/secrets/audiobookshelf)
#
# environment:
# ABS_OIDC_CLIENT_SECRET REQUIRED. The provider's client secret.
# ABS_OIDC_CLIENT_ID provider/client id (audiobookshelf)
# ABS_OIDC_GROUPS_CLAIM claim ABS reads for roles (abs_groups)
# ABS_OIDC_USERNAME authentik user made admin (akadmin)
# ABS_OIDC_USERS space-separated authentik usernames given the USER group
# (default: nobody — an empty list is fail-closed, and the
# household decides who listens to audiobooks)
# ABS_E2E_ADMIN authentik user kept in the ADMIN group for the Playwright gate
# (qa-e2e-admin — tests/e2e/specs/23-audiobookshelf-sso.spec.ts
# signs in as this identity, so a re-run restores the fixture)
# ABS_BASE ABS URL as the browser sees it, base path included
# (http://192.168.0.235:8000/audiobookshelf — the LAN
# origin, reached directly, which is also what the phone
# app is configured with)
# ABS_EXTRA_BASES space-separated additional origins, base path included
# (default: https://audiobooks.franksdank.cloud/audiobookshelf
# — the guarded vhost, dankclaw/Caddyfile).
# ABS builds its redirect_uri from the ORIGIN of the
# request, so each origin the app is served at needs its
# own registered URI — an unregistered origin is a
# redirect_uri_mismatch at the IdP, before any credential.
# Set it empty to register the LAN origin alone.
# AUTHENTIK_HOST ssh target (dankclaw)
set -euo pipefail
SECRET="${ABS_OIDC_CLIENT_SECRET:?set ABS_OIDC_CLIENT_SECRET (see the header)}"
[ "${#SECRET}" -ge 32 ] || { echo "refused: ABS_OIDC_CLIENT_SECRET is too short to be a secret" >&2; exit 2; }
CLIENT_ID="${ABS_OIDC_CLIENT_ID:-audiobookshelf}"
CLAIM="${ABS_OIDC_GROUPS_CLAIM:-abs_groups}"
ADMIN_GROUP="audiobookshelf_admins"
USER_GROUP="audiobookshelf_users"
SSO_USER="${ABS_OIDC_USERNAME:-akadmin}"
MEMBERS="${ABS_OIDC_USERS:-}"
E2E_ADMIN="${ABS_E2E_ADMIN:-qa-e2e-admin}"
BASE="${ABS_BASE:-http://192.168.0.235:8000/audiobookshelf}"
EXTRA_BASES="${ABS_EXTRA_BASES:-https://audiobooks.franksdank.cloud/audiobookshelf}"
HOST="${AUTHENTIK_HOST:-dankclaw}"
# The claim expression: translate group membership into ABS role names. It is stored
# verbatim so a later read of the provider shows exactly this policy.
read -r -d '' EXPR <<PYEXPR || true
groups = {g.name for g in user.groups.all()}
roles = []
if "$ADMIN_GROUP" in groups:
roles.append("admin")
if groups & {"$ADMIN_GROUP", "$USER_GROUP"}:
roles.append("user")
return {"$CLAIM": roles}
PYEXPR
ssh "$HOST" 'podman exec -i authentik-server ak shell' <<PYEOF
from authentik.core.models import Application, Group, User
from authentik.flows.models import Flow
from authentik.providers.oauth2.models import (
ClientType, OAuth2Provider, RedirectURI, RedirectURIMatchingMode, RedirectURIType, ScopeMapping)
ADMIN_GROUP, USER_GROUP = "$ADMIN_GROUP", "$USER_GROUP"
CLAIM, CLIENT_ID, SECRET, BASE = "$CLAIM", "$CLIENT_ID", "$SECRET", "$BASE"
BASES = [BASE] + """$EXTRA_BASES""".split()
SSO_USER, E2E_ADMIN = "$SSO_USER", "$E2E_ADMIN"
MEMBERS = """$MEMBERS""".split()
EXPR = '''$EXPR'''
# The expression reads user.groups, NOT user.ak_groups: authentik 2026.8 logs
# "User.ak_groups is deprecated … use User.groups" on every evaluation of the old
# name (measured on this run), and a stored mapping that leans on a deprecated
# accessor is a future upgrade failure sitting inside the login path.
for name in (ADMIN_GROUP, USER_GROUP):
Group.objects.get_or_create(name=name)
admin_g, user_g = Group.objects.get(name=ADMIN_GROUP), Group.objects.get(name=USER_GROUP)
# --- the claim mapping (scope_name IS the claim name ABS requests) ---------------
mapping = ScopeMapping.objects.filter(name="audiobookshelf-groups").first()
if mapping is None:
mapping = ScopeMapping.objects.create(name="audiobookshelf-groups", scope_name=CLAIM,
expression=EXPR,
description="Audiobookshelf role names from group membership")
print("mapping: created (scope_name %s)" % CLAIM)
elif mapping.scope_name != CLAIM or mapping.expression != EXPR:
mapping.scope_name, mapping.expression = CLAIM, EXPR
mapping.save(update_fields=["scope_name", "expression"])
print("mapping: scope/expression updated (scope_name %s)" % CLAIM)
else:
print("mapping: already current")
# Prove the expression runs and yields an ABS role BEFORE anyone tries to log in:
# authentik execs the expression inside a function whose context has `user`.
_src = "def _probe(user, request=None):\n" + "".join(" %s\n" % ln for ln in EXPR.splitlines())
_ns = {}
exec(_src, _ns)
# --- provider --------------------------------------------------------------------
# One browser callback per origin (LAN address and the guarded vhost), because ABS
# derives redirect_uri from the request origin; the mobile URI stays on the LAN origin.
uris = [RedirectURI(matching_mode=RedirectURIMatchingMode.STRICT,
url="%s/auth/openid/callback" % b) for b in BASES]
uris.append(RedirectURI(matching_mode=RedirectURIMatchingMode.STRICT,
url="%s/auth/openid/mobile-redirect" % BASE))
authz_flow = Flow.objects.get(slug="default-provider-authorization-implicit-consent")
# Three things beyond the name, each one a silent dead login: the grant-type list (2026.8
# defaults it empty), the built-in OpenID scope mappings beside the claim mapping, and the
# redirect URIs with the base path baked in.
GRANTS = ["authorization_code", "refresh_token"]
BUILTIN_SCOPES = ["authentik default OAuth Mapping: OpenID 'openid'",
"authentik default OAuth Mapping: OpenID 'profile'",
"authentik default OAuth Mapping: OpenID 'email'"]
scoped = {m.pk: m for m in ScopeMapping.objects.filter(name__in=BUILTIN_SCOPES)}
scoped[mapping.pk] = mapping
_have = {m.name for m in scoped.values()}
if len(_have) < len(BUILTIN_SCOPES):
print("scope mappings: NOT PROVIDED by this authentik install: %s"
% ", ".join(sorted(set(BUILTIN_SCOPES) - _have)))
p = OAuth2Provider.objects.filter(name="audiobookshelf").first()
if p is None:
p = OAuth2Provider(name="audiobookshelf", client_id=CLIENT_ID,
client_type=ClientType.CONFIDENTIAL, client_secret=SECRET,
authorization_flow=authz_flow, grant_types=list(GRANTS))
p.redirect_uris = uris
p.save()
p.property_mappings.set(list(scoped.values()))
print("provider: created (grant_types %s)" % GRANTS)
else:
changed = []
if p.client_id != CLIENT_ID:
p.client_id = CLIENT_ID
changed.append("client_id")
if p.client_secret != SECRET:
p.client_secret = SECRET
changed.append("client_secret")
if p.authorization_flow_id != authz_flow.pk:
p.authorization_flow = authz_flow
changed.append("authorization_flow")
if list(p.grant_types) != GRANTS:
p.grant_types = list(GRANTS)
changed.append("grant_types")
if [(u.url, str(u.matching_mode)) for u in p.redirect_uris] != \
[(u.url, str(u.matching_mode)) for u in uris]:
p.redirect_uris = uris
changed.append("redirect_uris")
if changed:
p.save()
if sorted(m.pk for m in p.property_mappings.all()) != sorted(scoped):
p.property_mappings.set(list(scoped.values()))
changed.append("property_mappings")
print("provider: asserted (%s)" % (", ".join(changed) or "nothing to change"))
# --- application (the launchpad entry; binds the provider) ------------------------
app = Application.objects.filter(slug="audiobookshelf").first()
if app is None:
app = Application.objects.create(name="audiobookshelf", slug="audiobookshelf",
provider=p, meta_launch_url=BASE)
print("application: created")
elif app.provider_id != p.pk or app.meta_launch_url != BASE:
app.provider, app.meta_launch_url = p, BASE
app.save(update_fields=["provider", "meta_launch_url"])
print("application: asserted")
else:
print("application: already bound")
# --- membership --------------------------------------------------------------------
root = User.objects.get(username=SSO_USER)
root.groups.add(admin_g)
# The deploy gate's Playwright spec signs in as this identity, so the admin fixture is
# provisioned with the rest: a blueprint re-import that drops it fails the gate, not a phone.
e2e = User.objects.filter(username=E2E_ADMIN).first()
if e2e is None:
print("e2e fixture: authentik user %s absent — the e2e SSO spec will fail" % E2E_ADMIN)
else:
e2e.groups.add(admin_g)
print("e2e fixture: %s in %s" % (E2E_ADMIN, ADMIN_GROUP))
print("admin: %s in %s" % (SSO_USER, ADMIN_GROUP))
for name in MEMBERS:
u = User.objects.get(username=name)
u.groups.add(user_g)
print("users: %s in %s" % (", ".join(MEMBERS) or "(none — fail closed)", USER_GROUP))
# --- verification: the claim the callback will actually receive ---------------------
for probe in [root] + ([e2e] if e2e else []) + [User.objects.get(username=n) for n in MEMBERS]:
print("claim for %s -> %s" % (probe.username, _ns["_probe"](user=probe, request=None)))
p.refresh_from_db()
print("provider %s | client_id %s | flow %s | grants %s | uris %s" % (
p.name, p.client_id, p.authorization_flow.slug, list(p.grant_types),
[u.url for u in p.redirect_uris]))
print("scopes: %s" % sorted(m.scopemapping.scope_name for m in p.property_mappings.all()))
PYEOF
cat <<DONE
Next: arm the Audiobookshelf side (issuer, client, HS256, claim) —
ssh dankserver 'ABS_OIDC=1 bash /home/dank/serverAppConfigs/scripts/audiobookshelf-init.sh'
It reads the same secret from /home/dank/.config/secrets/audiobookshelf and verifies
/status answers openid among its auth methods.
DONE
+95
View File
@@ -0,0 +1,95 @@
#!/usr/bin/env bash
# Census and repair the OAuth2 defaults on every authentik PROXY provider.
#
# A proxy provider built by hand in `ak shell` never runs
# ProxyProvider.set_oauth_defaults() — the REST serializer calls it, the ORM does
# not — so the row looks perfectly configured (external_host, internal_host,
# application bound, embedded-outpost claim present) while three OAuth facts are
# blank. Each one kills the guarded login in a different, unrelated-looking way,
# all BEFORE any credential is asked for:
#
# * redirect_uris empty -> authentik answers its own page:
# "Redirect URI Error — The request fails due to a missing, invalid, or
# mismatching redirection URI (redirect_uri)". Measured on the Audiobookshelf
# guard provider, 2026-11-03.
# * grant_types empty -> the authorize endpoint refuses the outpost's
# authorization-code request with a bodiless HTTP 400 (Chromium renders
# "This page isn't working / HTTP ERROR 400"). Same provider, one step
# later, after the URIs were filled in by hand.
# * authorization_flow NULL -> the authorize view crashes building its flow
# planner: AttributeError 'NoneType' object has no attribute 'slug', HTTP
# 500. That is what authentik-grafana-sso-fix.sh exists for.
#
# So: a guard that shows ANY of those three pages is not an Anubis problem and
# not a Caddy problem — run this script before touching the route.
#
# Repair calls the VENDOR's own set_oauth_defaults() (grant_types = authz code +
# client credentials + password, confidential client, the stock scope mappings,
# redirect_uris derived from external_host by _get_callback_url) and points the
# authorization flow at default-provider-authorization-implicit-consent, the flow
# every working provider here uses. Providers that already match are untouched,
# so a deliberate per-provider deviation is never silently flattened.
#
# This is authentik DB state. It is NOT in any deployable file: a blueprint
# re-import, or an ak-shell re-creation of a provider, reverts these facts and
# the symptom returns verbatim. Re-run after any such rebuild. Idempotent.
#
# Usage: authentik-proxy-provider-defaults.sh [--check]
# no argument repair what drifts, print one line per provider
# --check census only, exit 1 if anything drifts (gate use)
# override host: AUTHENTIK_HOST=…
set -euo pipefail
HOST="${AUTHENTIK_HOST:-dankclaw}"
MODE="${1:-}"
case "$MODE" in
"" | --check) : ;;
*) echo "usage: $(basename "$0") [--check]" >&2; exit 2 ;;
esac
ssh "$HOST" 'podman exec -i authentik-server ak shell' <<PYEOF
from authentik.flows.models import Flow
from authentik.providers.proxy.models import ProxyProvider, _get_callback_url
CHECK = $( [ "$MODE" = "--check" ] && echo True || echo False )
FLOW = "default-provider-authorization-implicit-consent"
WANT_GRANTS = ["authorization_code", "client_credentials", "password"]
flow = Flow.objects.filter(slug=FLOW).first()
if not flow and not CHECK:
raise SystemExit("flow %s is missing; nothing was changed" % FLOW)
drift = 0
for p in ProxyProvider.objects.all().order_by("pk"):
want_uris = [u.url for u in _get_callback_url(p.external_host)]
have_uris = [u.url for u in p.redirect_uris]
bad = []
if [str(g) for g in p.grant_types] != WANT_GRANTS:
bad.append("grant_types=%s" % [str(g) for g in p.grant_types])
if have_uris != want_uris:
bad.append("redirect_uris=%s" % have_uris)
if p.authorization_flow_id is None:
bad.append("authorization_flow=None")
if not bad:
print("ok %s (ext %s)" % (p.name, p.external_host))
continue
drift += 1
if CHECK:
print("DRIFT %s: %s" % (p.name, "; ".join(bad)))
continue
print("fix %s: %s" % (p.name, "; ".join(bad)))
p.set_oauth_defaults()
if p.authorization_flow_id is None:
p.authorization_flow = flow
p.save()
after = ProxyProvider.objects.get(pk=p.pk)
print(" -> grants %s, %d redirect URIs, flow %s"
% ([str(g) for g in after.grant_types], len(after.redirect_uris),
after.authorization_flow and after.authorization_flow.slug))
print("providers checked, drift: %d" % drift)
raise SystemExit(1 if (CHECK and drift) else 0)
PYEOF
echo "Verify: walk the guarded vhost in a browser — Anubis solves, authentik"
echo "renders its login form (no Redirect URI Error, no HTTP 400), and the"
echo "callback lands back on the app."
+13 -1
View File
@@ -1315,9 +1315,21 @@ deploy_audiobookshelf() {
mkdir -p /home/dank/Containers/audiobookshelf/config \
/home/dank/Containers/audiobookshelf/metadata
systemctl --user restart audiobookshelf.service
# /healthcheck answers once the HTTP server is bound; the library scan that
# /healthcheck answers once the HTTP server is bound, at the root whatever
# ROUTER_BASE_PATH says (measured on both base paths); the library scan that
# follows runs in the background and must not gate the deploy.
wait_for_health audiobookshelf http://127.0.0.1:8000/healthcheck 90
# A server with no root user and no library is "healthy" and useless, so the
# bootstrapper runs after the health gate and its failure fails the deploy:
# it creates the account (credential in the mode-600 host secret file), the
# Audiobooks library over the /audiobooks bind, and fires the scan. Every
# step is idempotent, so a re-deploy costs one incremental scan.
# The bootstrapper has to speak the same base path the unit mounts (the API, the
# login route and the OIDC redirect subfolder all hang off it), so the URL is read
# out of the unit file here instead of being restated in the script's default.
abs_base=$(awk -F= '/^Environment=ROUTER_BASE_PATH=/{print $3}' \
dankserver/audiobookshelf/quadlet/audiobookshelf.container)
ABS_URL="http://127.0.0.1:8000${abs_base}" scripts/audiobookshelf-init.sh
}
# The content server's single admin account and the desktop's basic-auth
+10 -2
View File
@@ -121,14 +121,14 @@ class Journal:
def read_plan(path, root, force):
"""Plan rows: (mkdirs, keeps, moves), header dropped, all paths under root."""
mkdirs, keeps, moves = [], [], []
mkdirs, keeps, moves, bad = [], [], [], []
with open(path) as fh:
header = fh.readline()
if not header.startswith("action\t"):
raise Refuse(f"{path} is not a plan (first column is "
f"{header.split(chr(9))[0]!r}, expected 'action') —"
" regenerate it with scripts/media-plan.py")
for line in fh:
for n, line in enumerate(fh, 2):
f = line.rstrip("\n").split("\t")
if f[0] == "mkdir":
mkdirs.append(f[2])
@@ -136,6 +136,14 @@ def read_plan(path, root, force):
keeps.append(f[1])
elif f[0] == "move":
moves.append((f[1], f[2], f[3] if len(f) > 3 else ""))
else:
bad.append(f"line {n}: unknown action {f[0]!r} — this tool"
" knows mkdir, keep and move (a rename is a move)")
if bad:
# A row this tool could not read is a plan bug, and a plan bug refuses:
# silently dropping it turns a typo into a no-op run that reports success.
head = "; ".join(bad[:3])
raise Refuse(head + (f" (+{len(bad) - 3} more)" if len(bad) > 3 else ""))
if not force:
for p in mkdirs + keeps + [p for m in moves for p in m[:2]]:
if p and not (p == root or p.startswith(root + os.sep)):
+10 -4
View File
@@ -78,10 +78,16 @@ one journal is one undo:
python3 scripts/media-shows.py --root /mnt/entropy/TVShows \
--enum /tmp/tv-enum.tsv --titles /tmp/jellyfin-tv.tsv \
--tvdb --tmdb --out /mnt/entropy/media-tv-plan.tsv \
--park /mnt/entropy/media-park/TVShows
python3 scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/media-tv-relayout.jsonl
--tvdb --tmdb --out /mnt/entropy/.media-campaign/media-tv-plan.tsv \
--park /mnt/entropy/NeedsProcessing/TVShows
python3 scripts/media-apply.py apply \
--plan /mnt/entropy/.media-campaign/media-tv-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/.media-campaign/media-tv-relayout.jsonl
Every plan, journal and log lives in /mnt/entropy/.media-campaign/ — the array
root holds libraries only. media-apply.py defaults its journal to
<root>/media-relayout.jsonl, so pass --journal explicitly or the run puts the
log back on the root.
"""
import argparse
import collections
+13 -6
View File
@@ -17,13 +17,20 @@ the executor's own path guard still has teeth, and give it its own journal —
one journal is one undo, and a TV move must not be welded to a movie relayout:
python3 scripts/media-tv.py --root /mnt/entropy/Movies \
--library /mnt/entropy/TVShows --out /mnt/entropy/media-tv-plan.tsv
python3 scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/media-relayout-tv.jsonl
python3 scripts/media-apply.py verify --plan /mnt/entropy/media-tv-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/media-relayout-tv.jsonl
--library /mnt/entropy/TVShows \
--out /mnt/entropy/.media-campaign/media-tv-plan.tsv
python3 scripts/media-apply.py apply \
--plan /mnt/entropy/.media-campaign/media-tv-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/.media-campaign/media-relayout-tv.jsonl
python3 scripts/media-apply.py verify \
--plan /mnt/entropy/.media-campaign/media-tv-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/.media-campaign/media-relayout-tv.jsonl
python3 scripts/media-apply.py sweep --root /mnt/entropy \
--journal /mnt/entropy/media-relayout-tv.jsonl --apply
--journal /mnt/entropy/.media-campaign/media-relayout-tv.jsonl --apply
Plans and journals live in `/mnt/entropy/.media-campaign/`, never loose on the
array root: `media-apply.py` defaults its journal to `<root>/media-relayout.jsonl`,
so `--journal` is always explicit here.
Guards, and why each one is here:
+8 -6
View File
@@ -16,14 +16,16 @@ the journal and frees no space until the hold is purged.
python3 scripts/media-tvdupes.py --library /mnt/entropy/TVShows \
--hold /mnt/entropy/.AGENTTRASH/20261102-tv-duplicates \
--out /mnt/entropy/tv-dupes-plan.tsv
python3 scripts/media-apply.py apply --plan /mnt/entropy/tv-dupes-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/tv-dupes.jsonl \
--out /mnt/entropy/.media-campaign/tv-dupes-plan.tsv
python3 scripts/media-apply.py apply \
--plan /mnt/entropy/.media-campaign/tv-dupes-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/.media-campaign/tv-dupes.jsonl \
--metadata-view /var/lib/media-meta/view --apply
python3 scripts/media-apply.py verify --plan /mnt/entropy/tv-dupes-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/tv-dupes.jsonl
python3 scripts/media-apply.py verify \
--plan /mnt/entropy/.media-campaign/tv-dupes-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/.media-campaign/tv-dupes.jsonl
python3 scripts/media-apply.py sweep --root /mnt/entropy \
--journal /mnt/entropy/tv-dupes.jsonl --apply # empty folders only
--journal /mnt/entropy/.media-campaign/tv-dupes.jsonl --apply # empty folders only
scripts/media-view.sh reap # ghost sidecars above
Then both servers have to be told: the plan moves files on the raw array, and no
+6
View File
@@ -104,6 +104,12 @@ for path in "${CHANGED[@]}"; do
dankserver/jellyfin/quadlet/*|dankserver/plex/quadlet/*)
add_test tests/media_overlay.bats
;;
# The Audiobookshelf bootstrapper (account + library + scan over the API)
# and the deployer hook that runs it after the health gate.
scripts/audiobookshelf-init.sh|tests/audiobookshelf_init.bats)
add_test tests/audiobookshelf_init.bats
add_test tests/deployment_flow.bats
;;
shared/homepage/*)
add_test tests/homepage_caddy.bats
add_test tests/homepage_config.bats
@@ -23,7 +23,7 @@ This repository tests its configuration at defined seams. The tests run with `ba
| Deployment flow | `tests/deployment_flow.bats`, `tests/deployment_scoping.bats` | scoped dispatcher exists, targets are known, service plans isolate inference, Caddy ownership is guarded, and SSH reachability is checked (skips offline) |
| Quadlet units | `tests/*quadlet*.bats` | every unit is boot-persistent, self-healing, repo-provenanced, secret-free |
| E2E (browser) | `tests/e2e/` + `tests/e2e_scaffold.bats` | Playwright drives the live guard chain; the bats seam asserts structure and credential hygiene. See [[E2E Suite]] |
| Per-service / unit seams | `tests/llm_units.bats`, `tests/frank_{archon,dragoon,zealot}_unit.bats`, `tests/llm_secrets.bats`, `tests/homepage_{caddy,checker,config,deploy,pages}.bats`, `tests/{anubis,authentik,hermes_webui,pihole,mount_entropy,searxng_limiter,autoresearch_measure,changed_tests}.bats`, `tests/{caddy,dankserver,frankbook,marreta,observability,open_webui,searxng}_quadlet.bats`, `tests/comfyui_rocm.bats` | the ~30 named seams the summary rows above do not list: LLM systemd-unit shape and env hygiene, homepage config/pages/deploy behaviour, and one detail-seam per deployed service |
| Per-service / unit seams | `tests/llm_units.bats`, `tests/frank_{archon,dragoon,zealot}_unit.bats`, `tests/llm_secrets.bats`, `tests/homepage_{caddy,checker,config,deploy,pages}.bats`, `tests/{audiobookshelf_init,anubis,authentik,hermes_webui,pihole,mount_entropy,searxng_limiter,autoresearch_measure,changed_tests}.bats`, `tests/{caddy,dankserver,frankbook,marreta,observability,open_webui,searxng}_quadlet.bats`, `tests/comfyui_rocm.bats` | the ~30 named seams the summary rows above do not list: LLM systemd-unit shape and env hygiene, homepage config/pages/deploy behaviour, and one detail-seam per deployed service |
## Focused pre-commit testing
@@ -1,11 +1,11 @@
---
type: reference
description: "Audiobookshelf on Dankserver: quadlet shape, the read-only array bind, state on the SSD, first-run account, laws."
description: "Audiobookshelf on Dankserver: quadlet shape, array bind, SSD state, API bootstrap, authentik SSO, guarded vhost, laws."
tags:
- dankserver
- audiobooks
- quadlet
timestamp: 2026-11-02
timestamp: 2026-11-03
---
# Audiobookshelf
@@ -16,8 +16,8 @@ per-listener playback positions, and streams the audio. It is **not** a media
server — it shares nothing with [[Plex]] or [[Jellyfin]] and owns no files on the
array.
**Host:** Dankserver — **Unit:** `audiobookshelf.service` (rootless Quadlet, user
manager) — **Internal-only:** LAN address, no vhost, no DNS name.
**Host:** Dankserver — **Unit:** `audiobookshelf.service` (rootless Quadlet, user manager) —
**Doors:** LAN `http://192.168.0.235:8000/audiobookshelf` (direct, what the phone app uses) and guarded `https://audiobooks.franksdank.cloud/audiobookshelf/`.
## Shape
@@ -26,7 +26,7 @@ manager) — **Internal-only:** LAN address, no vhost, no DNS name.
| Unit file | `dankserver/audiobookshelf/quadlet/audiobookshelf.container` (the authority for every number here) |
| Image | `ghcr.io/advplyr/audiobookshelf` pinned by **amd64 manifest digest** — see Laws |
| Port | `Environment=PORT=8000` + `PublishPort=8000:8000` — web UI and API on one port |
| Base path | `Environment=ROUTER_BASE_PATH=/` — the URL is the plain `host:port`, no `/audiobookshelf` prefix |
| Base path | `Environment=ROUTER_BASE_PATH=/audiobookshelf` — every URL below carries this prefix, and the shipped SPA cannot boot without it (Laws) |
| Network | `pasta` (standalone; publishes nothing to other containers) |
| Audio bind | `/mnt/entropy/Audiobooks:/audiobooks:ro` — read-only, by design |
| State | `/home/dank/Containers/audiobookshelf/config` → `/config`, `…/metadata` → `/metadata` (both NVMe) |
@@ -36,20 +36,76 @@ manager) — **Internal-only:** LAN address, no vhost, no DNS name.
| Self-healing | `Restart=always`, `RestartSec=5`, `WantedBy=default.target` |
| Deploy | `./scripts/deploy.sh dankserver --service audiobookshelf` |
| Handler | `deploy_audiobookshelf` in `scripts/deploy-remote.sh` |
| Single sign-on | authentik provider `audiobookshelf`; the local `frank` account stays enabled as break-glass — see *Single sign-on* |
| Library layout | one book per folder — ABS groups by the first directory holding a media file (see Laws); every move is journalled at `/mnt/entropy/.media-campaign/media-relayout-audiobooks.jsonl` |
## First run
## Bootstrap
1. Open `http://192.168.0.235:8000` **from the LAN**. ABS accepts the initial
account creation only from a local-network address; a remote address gets a
refusal, not a form.
2. Create the owner account, then add one **Audi Book** library with folder
`/audiobooks`. The path is the in-container name of the read-only bind.
3. The first scan reads the whole tree once. It runs in the background — the
deploy gate is already green by then, and `IOWeight=10` keeps it off the
critical path.
`scripts/audiobookshelf-init.sh` — run by the deployer after the health gate, so a
deployed ABS is a *usable* ABS: `/healthcheck` answers 200 on a server with no
user, no library and no scan, and nothing else in the deploy notices the
difference. The script is idempotent; re-running it costs one incremental scan.
Nothing in this repo holds an ABS credential: the account is created in the
browser and lives in the `config` bind on the SSD.
| Step | Door | Printed |
|------|------|---------|
| credential | `~/.config/secrets/audiobookshelf` (mode 600, generated once, host-side) | `credential: created …` |
| account | `POST /init` with `{"newRoot":{"username","password"}}` | `account: created (frank)` / `already registered` |
| session | `POST /login` → bearer token | `token: obtained (root)` |
| library | `POST /api/libraries`, folder `/audiobooks`, provider `audiobook` | `library: created Audiobooks (<id>)` / `exists` |
| scan | `POST /api/libraries/<id>/scan` | `scan: accepted` |
| proof | `GET /api/libraries/<id>/items?limit=1` | `items: <total>` |
Sign in at `http://192.168.0.235:8000/audiobookshelf` as `frank`; the password is
`awk -F= '/^ABS_ADMIN_PASSWORD=/{print $2}' ~/.config/secrets/audiobookshelf` on
Dankserver. Change it in **Settings → Users → frank → Password**; the secret file
then holds a stale value and only matters again if the `config` bind is ever
destroyed (a fresh DB makes the script create a new owner from it).
A full scan is slow and runs in the background; watch the `items:` total climb,
never `/healthcheck`. It also reads files that are genuinely broken and logs
`[AudioFileScanner] Invalid data found when processing input` per file — that is
the file, not the bind: the same line count grows while the total climbs.
## Phone app
The official **Audiobookshelf** app connects with Server Address
`http://192.168.0.235:8000/audiobookshelf` — the base path belongs in the address — username
`frank`, and the same password as the web UI, or the SSO button; plain HTTP. Measured from a
second host 2026-11-02 and re-measured on both base paths: `/ping` (`{"success":true}`) and
`/healthcheck` (`OK`) answer at the ROOT whatever `ROUTER_BASE_PATH` says, so the app's
connectivity probe works before it resolves the base path at all, and `iptables -S INPUT`
carries no rule for 8000 — a phone on `192.168.0.0/24` connects with no further setup, and
`download: true` on the root account is what makes offline listening work. Away
from the LAN, see the two-doors law in *Laws* — the guarded vhost is not the remedy.
**The login token is at `user.token`, not top level.** `POST /login` answers
`{ user, serverSettings, ereaderDevices, Source, userDefaultLibraryId }` and the bearer sits
inside the user object. A client that reads `response.token` gets the empty string, sends
`Authorization: Bearer ` with nothing after it, and every `/api/*` call then answers `401`
as though the password were wrong.
## Single sign-on (authentik)
Both doors take the household identity: the web UI puts **Sign in with Authentik** beside
the local form, and the phone app offers SSO on its server-address screen. The local
`frank` account stays enabled on purpose — `authActiveAuthMethods` is `local` **and**
`openid`, and `local` is how you get in when the IdP is down.
| Piece | Owner |
|-------|-------|
| Provider `audiobookshelf`, the two groups, the scope mapping | `scripts/authentik-audiobookshelf-sso.sh` — re-run it after an authentik rebuild, because this state lives in the IdP's database and a blueprint re-import reverts it silently |
| Client secret | `ABS_OIDC_CLIENT_SECRET` in `~/.config/secrets/audiobookshelf` on Dankserver (mode 600; never in the repo, a URL or an argv) |
| The ABS half: issuer, the four IdP URLs, claim, auto-register | `scripts/audiobookshelf-init.sh`, armed automatically when that secret exists in the credential file |
| Role at first SSO sign-in | the `abs_groups` claim: `audiobookshelf_admins` → `admin`, `audiobookshelf_users` → `user`, in neither → denied (fail closed) |
| Admin group members | `akadmin` (the operator's own authentik admin account — sign in as it and the claim grants site admin) and `qa-e2e-admin` (the identity the deploy gate signs in as, so the script re-adds it every run; drop it and the gate fails, not a phone) |
| The guard in front of the vhost | proxy provider `audiobooks-proxy` (external_host `https://audiobooks.franksdank.cloud/`, internal_host `http://caddy:8017`) + application "Audiobookshelf Guard" (its own name tracks the app, not the vhost), claimed by the embedded outpost — the ak-shell recipe in [[Caddy]] *Guard a new vhost* creates it; without that row the chain answers authentik's error page, not the login |
The IdP saying yes is not the same as being allowed in: with `authOpenIDAutoRegister` on,
a first sign-in creates the account, and the claim alone decides the type. A user in
neither group is refused after a successful authentication — that is the mapping working,
not a broken login.
Both halves — authentik's provider row and ABS's settings — are asserted end to end by the Playwright gate named in *Verification*: a sign-in as `qa-e2e-admin` that must land inside the app with site-admin rights.
## Laws
@@ -61,6 +117,54 @@ browser and lives in the `config` bind on the SSD.
`active (running)` while every probe of `:8000` was refused and the deploy gate
failed at 180 s. Two seconds with the pinned digest settles it:
`podman inspect --format '{{.Config.Env}}' <image>`.
- **The URL carries `/audiobookshelf`, and that is not a style choice.** The SPA in
`/app/client/dist/index.html` is built with `routerBasePath:"/audiobookshelf"`, so it asks
for its assets under that prefix whatever the server is mounted at. Mounted at `/`, every
script request falls through to the SPA fallback, comes back `text/html`, Chrome refuses to
execute it, and the page never leaves "Loading…" — while `/healthcheck`, the API and the
phone app stay perfectly healthy, which is how this hid behind an API-only verification
(measured 2026-11-02). Prove a UI claim in a browser, never from `/health`.
- **The OIDC redirect subfolder is a value you send, not a default you inherit.** The 2.17.4
upgrade only fills `authOpenIDSubfolderForRedirectURLs` on a database that ALREADY had OIDC
enabled, so a DB created before SSO leaves it unset and the server builds
`http://host/undefined/auth/openid/callback`. authentik answers the authorize request with
"Redirect URI Error" and nothing on the ABS side looks wrong (measured 2026-11-02). The
provider's registered URI must equal `<origin><subfolder>/auth/openid/callback`, so each
script derives the value from one variable of its own (`ABS_URL`, `ABS_BASE`) and
`tests/audiobookshelf_init.bats` pins what reaches the wire.
- **An ORM-created provider has no grant types.** authentik 2026.8 defaults `grant_types` to
`[]`, and `/application/o/authorize/` then refuses the code flow with oauthlib's generic
"The request is otherwise malformed" — the wire names nothing, the server log line that names
it is `Invalid grant_type for provider`. Worse, the redirect-URI check runs *first*, so a
provider carrying both faults shows only "Redirect URI Error" until the URI is fixed, and the
second wall appears after the fix ships (measured 2026-11-02: the browser reached authentik,
got `code`+`state`, and ABS still bounced to `/login?error=undefined`). The script sets
`authorization_code` + `refresh_token`; a blueprint re-import reverts the row silently.
- **The guarded door has to tell the truth about its scheme.** ABS validates every OIDC
`callback` against the origin it believes it is served from — `getRequestOrigin()` is
`x-forwarded-proto://Host` (`server/utils/requestUtils.js`, read in the pinned digest
2026-11-03) — so behind a TLS edge that reaches the container over plain http it computes
`http://audiobooks.franksdank.cloud` and answers its own https callback with a 400 "Invalid callback
URL - must be same-origin" before the IdP is ever contacted. The Caddy `:8017` route pins
`header_up X-Forwarded-Proto https` for exactly that reason ([[Caddy]] *Guard a new vhost*);
the LAN door needs nothing, because there browser and server agree on `http`.
- **The claim mapping is not the whole `property_mappings` list.** A provider holding only the
custom mapping grants a token scoped shorter than the scopes ABS requests
(`openid profile email abs_groups`): the code is issued and exchanged normally, and ABS's
`/userinfo` call then dies `insufficient_scope (The request requires higher privileges than
provided by the access token)` — a fault that reads like an ABS bug and is an IdP row. Attach
the built-in OpenID `openid`/`profile`/`email` mappings beside the claim mapping, which is
what every working RP on this IdP carries (measured 2026-11-02).
- **With `authOpenIDMatchExistingBy` unset, an SSO identity never lands on an existing ABS
account, and a name collision is a hard denial.** In that mode `findUserFromOpenIdUserInfo`
matches on `sub` only, so every first sign-in auto-registers a NEW user; if the IdP username
equals an ABS username — the local root `frank` is the one here — the create hits the unique
constraint and the callback dies `Unauthorized`. So "create a `frank` user in authentik" does
not reach the root account: either set `authOpenIDMatchExistingBy` to `username` (links the
IdP account to the existing one, and `setUserGroup` refuses to downgrade a root — root plus an
`admin` claim is allowed, root plus `user` is denied) or pick a non-colliding IdP name. Read
off `server/models/User.js` and `server/auth/OidcAuthStrategy.js` in the pinned digest
(2026-11-02).
- **The audio bind stays `:ro`.** ABS writes nothing that belongs beside the
audio: its library, covers, progress and sidecar metadata all land in
`/metadata` on the SSD. A writable bind would put stray writes on the spindle
@@ -91,14 +195,109 @@ browser and lives in the `config` bind on the SSD.
- **No `systemctl --user enable`.** Generated Quadlet units cannot be enabled on
this host; `WantedBy=default.target` in the unit file is what carries the
service across a reboot ([[Quadlet Lifecycle]]).
- **LAN-only until Frank says otherwise.** `0.0.0.0:8000` with no reverse proxy
in front. Any public vhost requires the Anubis + Authentik chain first, and the
outpost proxy provider has to exist before the vhost flips (external exposure
policy in `AGENTS.md`).
- **Two doors, one app.** `http://192.168.0.235:8000/audiobookshelf` is the raw
LAN address — plain HTTP, nothing in front, what the phone app is configured
with; `https://audiobooks.franksdank.cloud/audiobookshelf/` is the guarded vhost
(Caddy → Anubis → the Authentik outpost → Caddy `:8017` → this container),
the house chain for browser traffic — [[Caddy]] *Guard a new vhost*. That
chain serves browsers only: the phone app is a non-browser client and solves
no challenge, so remote listening wants a VPN into the LAN, not the vhost.
- **There is no `/setup` route; the creation door is `POST /init`.** `/status`
(no `/api` prefix) reports `isInit` = "a root user exists", `POST /init` with
`{"newRoot":{"username","password"}}` creates it, `POST /login` returns the
bearer every `/api/*` route wants. `POST /setup` answers `Unauthorized` — that
is a missing route wearing an auth message. Read off `dist-server/server/Server.js`
inside the pinned digest rather than from a forum.
- **A 200 may carry plain text.** `res.sendStatus(200)` answers the body `OK`, and
`/init` and the scan route use it; a client that parses every success as JSON
crashes on the one call that matters (`tests/audiobookshelf_init.bats` pins it).
- **The item count is not a field on the library DTO, and paging never ends
itself.** The door is `GET /api/libraries/<id>/items?limit=1` → `total`; a poller
that guesses a plausible name (`numItems`) prints `0` forever beside a scan that
is working. Paging is on `page` (0-based) plus `limit`, and **`offset` is
ignored**: `limit=500&offset=500` returns the same 500 ids as `offset=0`, while
`page=1` shares none of them. A census that pages on `offset` therefore reads
`ceil(total / limit) × limit` rows containing only `limit` distinct items — the
exact origin of the "exactly 1,500" and "exactly 4,000" counts two runs
reported for one library. Page on `page`, count *unique ids*, stop on `total`,
and take the number after the pass settles (`total` moves during a scan).
- **A fresh scan is the only thing that notices new audio.** `inotify` does not
cross a rootless podman bind — measured for [[Jellyfin]], whose page owns that
law — so the in-container watcher sees nothing a host process files onto the
array, and the deployer's re-run of the bootstrapper is the cheap scan trigger.
- **One stray media file at a collection root turns the whole collection into one
book.** Every scan re-walks the tree from disk (`scanFolder` → `recurseFiles`),
and `utils/scandir.js:groupFileItemsIntoLibraryItemDirs` assigns each file to the
FIRST directory along its path that already has a group: the first file seen at a
root owns every deeper file, and the folders below it never become items. One
949 MB `.cbr` sitting at `_unsorted`'s root collapsed 652 book folders into a
single item of 5,693 tracks, titled after whichever file supplied metadata.
"Hundreds of books are missing" is a layout fault, not a scan fault.
- **"Media file" means audio OR ebook, so an audio-only census cannot see the file
that is hiding the library.** With `audiobooksOnly=false` (the deployed shape)
the predicate is `SupportedAudioTypes` ∪ `SupportedEbookTypes`: 20 audio
extensions (including `mp4`, `mpg`, `webm`) plus `epub pdf mobi azw3 cbr cbz`. A
census reporting "that root holds no loose audio" can be correct and still
useless — the file doing the shadowing here was a comic archive.
- **Deleting the covering item does not un-shadow anything.** Grouping is recomputed
from disk, so the next scan rebuilds the item from the same stray file (measured:
`DELETE`, then scan → `1 Added`, the item back at the same path). The fix is the
MOVE: give every stray root file a folder of its own (one file per folder is
ABS's own single-book layout; formats of one book share a folder), through the
journal-guarded mover, then delete the stale item so the old path is not kept,
then scan.
- **`?force=1` is neither a discovery arm nor a repair arm.** Both arms of
`POST /libraries/:id/scan` walk the disk, so finding new folders needs neither
and the deployer's cheap trigger stays on the plain arm. Force additionally
re-scans every item that already exists (measured 1:33 against 0:04 on ~4,600
items) and repaired nothing on a library whose residuals were file-level: item
count, path set and media-less count identical before and after.
- **`DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()`.** In
`LibraryItemController.delete`, `hardDelete = req.query.hard == 1` and then
`fs.remove(libraryItemPath)`. Pointed at a covering item, that path is the whole
swallowed subtree — `hard=1` there deletes every book beneath it off the array.
Assert the item's `path` first, and never pass `hard`.
- **`Invalid data found when processing input` names the file, not the bind.** The
scanner logs it per file and skips that file; a book whose files ALL fail never
becomes an item at all, so the disk-vs-DB folder census shows it as a book with
no item. That is corrupt media, not a lost book. Never clear a suspect file with
a lenient probe: `ffprobe -show_entries format=duration` happily returns three
hours for the WAV bytes wearing a `.mp3` name that the scanner refuses. Read the
log's file paths instead, and `stat -c %s` first — 0-byte files are the common
case, and a truncated download reports `moov atom not found`.
- **A `Disc 01` / `CD 1` subfolder is part of a book, not a book.** The extension
census counts those folders; ABS deliberately does not — its grouping attaches
their files to the ancestor item (measured: 110 such folders under one item
carrying all 192 tracks). Read the basename before believing a disk-vs-DB gap.
- **`.cbr`/`.cbz` group like media and never play.** They sit in the grouping
predicate, so they hide books, but the comic parser throws `ERAR_BAD_ARCHIVE`
(`parseComicMetadata` → `CbrComicBookExtractor`) and the item lands with no
media. A media-less row beside those lines is the file, not the scanner.
- **A scan is finished when a NEW scan uuid completes.** Match
`Library scan "<uuid>" completed in ...`. `podman logs --since 30m` is a sliding
window: the set of visible completion lines changes on its own as old lines age
out, so a poller keyed on that hash reads a mid-scan number and calls it done.
## Verification
`./tests/run.sh` covers this unit through `tests/dankserver_quadlet.bats`
(boot persistence, the SSD state roots, the `:ro` array bind, the mount gate, the
`wget` health check) and `tests/media_overlay.bats` (only [[Plex]] and
[[Jellyfin]] may reference the metadata view).
(boot persistence, the SSD state roots, the `:ro` array bind, the mount gate, the `wget`
health check, the base path the shipped SPA needs), `tests/audiobookshelf_init.bats` (the
bootstrapper against a stub of the API above: create-once, idempotent re-run, no secret in a
URL, refusal by URL when the server does not answer, the deployer calling it only after the
health gate, and the SSO arm sending the mounted base path as the redirect subfolder);
`tests/media_overlay.bats` (only [[Plex]] and [[Jellyfin]] may reference the metadata view);
and the live Playwright gate `tests/e2e/specs/23-audiobookshelf-sso.spec.ts`, run by
`tests/e2e/run.sh` at the end of every `deploy.sh` — the `admin` project signs in as
`qa-e2e-admin` against `http://192.168.0.235:8000/audiobookshelf` and asserts the two things
no stub can see: every `.js` the app loads arrives as JavaScript (a bundle built for the wrong
base path arrives as `text/html`, which boots nothing), and `/api/me` answers 200 with
`type: "admin"` — true only while grant types, the OpenID scope mappings and the group claim
all hold together.
Five book folders hold media no scanner can read, so they stay out of the library:
three 2024 Nebula nominees in the `_unsorted` box (two `.m4b` with no `moov` atom,
one 36-file `.mp3` set that is not MPEG audio at all), `Anthony Heald/Bounty
Hunter Wars - Book 1` and `Star Wars Full Cast Audio Drama/Dark Empire III` — the
latter two also exist in readable form under their author folders. Fix by
re-acquiring the files; the scanner names every one of them in its log.
+109 -23
View File
@@ -3,7 +3,7 @@ type: reference
title: Jellyfin
description: "Jellyfin on Dankserver: quadlet unit, overlay metadata view, provider keys, ports, access, transcode status."
tags: [jellyfin, dankserver, media-server]
timestamp: 2026-10-30T00:00:00Z
timestamp: 2026-11-02T11:00:00Z
---
# Jellyfin
@@ -58,12 +58,23 @@ reads — movie art has never existed beside the media on either layer.
**Library coverage is the first thing to check when a title is missing.** The
four libraries are `movies → /data/Movies`, `tvshows → /data/TVShows`,
`music → /data/Music`, `boxsets → /config/data/collections`
(`GET /Library/VirtualFolders`). `/data/media-park` is in **none** of them, so
everything filed under it — the `_unsorted` release bundles and the `extras/`
folders the relayout wrote there — is invisible to Jellyfin by configuration,
(`GET /Library/VirtualFolders`). The review pile — `/data/NeedsProcessing`, the
former `media-park`, drained into it 2026-11-02 — is in **none** of them, so
everything filed there is invisible to Jellyfin by configuration,
not by fault: a recursive refresh answers 204, burns CPU for minutes and
indexes nothing, because it never looks there. Check coverage before re-scanning.
**Which read proves an item is indexed: the folder view, never the recursive type
list.** `GET /Items?IncludeItemTypes=Movie&Recursive=true` answered 2,101 items on
2026-11-02 while `GET /Items?ParentId=<Movies ItemId>` listed 2,548 children — every
folder on disk — and the eight folders the recursive read "lost" (three `Gundam 00
Special Edition`, five `Sailor Moon Memorial`) each answer as `Type: Movie` through
`SearchTerm` and sit in the folder listing. The two reads are not interchangeable, so
the recursive list is a wrong instrument for "did the refresh index what I moved": its
shortfall is grouping, not missing media, and acting on it starts a scan that fixes
nothing. Verify per item — `ParentId` listing or `SearchTerm` — and use the recursive
count only as a trend line.
**Failure signature:** `System.IO.IOException: Read-only file system :
'/data/Movies/<film>/movie.nfo'` means the container is binding the raw array
instead of the view — always a fault on this host since the 2026-10-31 cutover.
@@ -81,21 +92,27 @@ so an empty `<ImageFetchers />` switches art off while `ImageFetcherOrder` stays
fully populated — order is not enablement — and the populated `<MetadataFetchers>`
built next to it makes the library look healthy.
**Live posture, measured 2026-11-02** from `GET /Library/VirtualFolders`. The DTO is
PascalCase — `Name`, `CollectionType`, `Locations`, `ItemId`, `RefreshStatus` — and a
census written against the 10.x camelCase keys reads `None` for everything, which
looks exactly like an unconfigured library.
**Live posture, measured 2026-11-02** from `GET /Library/VirtualFolders`, cross-checked
against each library's `config/root/default/<Library>/options.xml`. The switches are NOT
top-level DTO keys: the folder carries `Name`, `CollectionType`, `Locations`, `ItemId`,
`PrimaryImageItemId`, `RefreshStatus` and `LibraryOptions`, and every per-library option
sits inside `LibraryOptions`. Reading `EnableInternetProviders` off the folder itself
answers `None` for all four — the camelCase trap one level deeper, and it looks exactly
like an unconfigured library.
| Library | `CollectionType` | `EnableInternetProviders` | `EnableRealtimeMonitor` |
|---|---|---|---|
| `Movies` | movies | true (fixed 2026-11-02) | **false** |
| `Shows` | tvshows | **false** | true |
| `Shows` | tvshows | true (fixed 2026-11-02) | true |
| `Music` | music | **false** | true |
| `Collections` | boxsets | **false** | false |
`Shows` and `Music` are silent file indexes in the sense above: a refresh answers
`Music` and `Collections` are silent file indexes in the sense above: a refresh answers
`204`, stores nothing, logs nothing. Any metadata or artwork work on either one
starts by flipping the switch, never by firing a scan.
starts by flipping the switch, never by firing a scan. `Shows` sat in that set
until 2026-11-02; the door is `POST /Library/VirtualFolders/LibraryOptions` with
the library's own `ItemId` (below), and `scripts/jellyfin-providers.sh verify` is
the one command that says which libraries are still in it.
**The per-library option door on 12.x is `POST /Library/VirtualFolders/LibraryOptions`**
— `GET` there answers `405`, which is how the route announces itself. The 10.x doors
@@ -119,9 +136,41 @@ store on a build where the automatic image pass will not run at all — it is ho
the Movies library got its art: 2,515 of 2,564 films carry a primary now, and the
rest have none in TMDb (shorts like `Astartes`, files whose names match nothing).
**API shape on 12.x:** `/Items` and `/Items/{id}` need `UserId` when
authenticated by API key — without it the answer is `400 Error processing
request`, which reads exactly like an authentication failure. `MetadataRefreshMode`
**The Download door's exact wire shape:** the list response carries the address in
`Url` — there is no `ImageUrl` field — and the POST takes camelCase query params,
`?type=Primary&imageUrl=<Url>`, answering `204`; the stored tag then appears in
`ImageTags`. PascalCase params, or an `imageUrl` left empty, answer `500`, which
reads as a server fault and is a field-name fault.
**`RemoteImages` answers empty for an item with no `ProviderIds`** — no provider id,
nothing to look up. Art coverage is capped by identification, not by the image door:
the items a campaign returns nothing for are the unidentified ones, and no image
retry will change that.
**A `Download` pass reports zero only when it is pinned to one provider.** A TMDb-pinned
campaign over `Shows` classified every item in 20 s, downloaded nothing, and concluded that
identification was the only lever left. The same door queried with no `Providers=` filter
answered for 16 seasons that same day — every offer from `TheTVDB`, which the pinned form
never asks — and all 16 are dressed now. A pinned answer proves one provider is empty; the
unpinned answer is the provider census, and a TV art loop that pins is simply the wrong tool.
Post-fill the artless remainder measures 86 items (80 seasons, 6 series): 61 carry no provider
id at all — the unnamed long tail (`3a BBW - Series (2010-11)`, `History Of Africa With Zeinab
Badawi`, `(Warhammer TV)`-shaped directories), where the lever is folder names or an identify
pass — and 25 hold an id and draw no offer from any provider. No image retry moves either
group, which is the half of the old conclusion that survives.
**Series and season art is fetched; episode stills are not, deliberately.** Every
episode query is a fresh provider round-trip — nothing caches per episode — measured
at ~11 s per item beside ~2 s per series on the same host, with one hit in five in a
sample. Across 42k episodes that is days of provider traffic to dress a fifth of them,
and clients display the series and season art anyway. If episode stills are ever
ordered, the loop is the same door and its log makes the pass resumable.
**API shape on 12.x:** `/Items/{id}` needs `UserId` when authenticated by API key — without
it the answer is `400 Error processing request`, which reads exactly like an authentication
failure, and it bites hardest as a *verification* read directly after a write that succeeded.
The `/Items` list route, `?Ids=<id>` included, needs no `UserId`, so verify writes there.
`MetadataRefreshMode`
accepts `Default` (and `Realtime`) only; `Full` and `ValidationFailure` are
rejected outright, and what forces the providers to re-run is
`ReplaceAllMetadata=true`.
@@ -232,9 +281,17 @@ scripts/jellyfin-providers.sh verify # plugin + key LENGTHS, never key bytes
- **The homelab TVDB key is a licensed key** (its JWT claims say
`community_supported: false`), so it needs **no PIN** — the optional
`TVDB_PIN=` line exists only for a subscriber-shaped key.
- **TVDB is installed and keyed but is not a metadata agent**: TMDb still
identifies everything. Enabling TVDB as a provider would re-fetch show
metadata across the library, so it is a deliberate, separate change.
- **TVDB is installed but is not a metadata agent**: TMDb still identifies
everything. Enabling TVDB as a provider would re-fetch show metadata across
the library, so it is a deliberate, separate change.
- **A plugin owns its own config file, under its own root, and omits null
settings** — so the element an installer wants to fill may simply not exist,
and its root is `<PluginConfiguration>`, not `<Configuration>`. TVDB's file had
no `<ApiKey>` element at all, which is why `install` refused with
`no place to write <ApiKey>` from October until the anchor fix: the secret sat
valid in `provider-keys` the whole time, and `verify`'s `MISSING key` was this
fault, not a missing key. The writer now anchors on the file's own root and
still refuses by name when a file has none.
- **The deployer places the keys before it restarts jellyfin**
(`ensure_provider_keys` in `scripts/deploy-remote.sh`) and warns instead of
failing when `provider-keys` is absent, so a host without it still deploys.
@@ -284,12 +341,41 @@ anything. The fix is `systemctl --user restart jellyfin.service`, one POST after
health answers, and then believe `ScheduledTasks` State at 130 % CPU, not the
response code.
**A scan that must see a new folder has to be recursive.** `POST /Library/Refresh`
is the incremental pass, and inotify does not cross a rootless podman bind, so
folders created on the host stay invisible to it. The move is
`POST /Items/{id}/Refresh?Recursive=true` (204) per library, the id from
`GET /Library/VirtualFolders` — its field is `ItemId`, not `Id` — and that same
record's `RefreshStatus` is the progress read.
**The door that scans is `POST /Library/Refresh` (204), and `ScheduledTasks` is
the progress read.** A task that has never completed answers
`LastTaskResult=None, EndTimeUtc=None`, and that pair is what separates "never
ran" from "finished a moment ago" — the difference between a door that is dead
and one that merely answered `204`.
**The settle read is `GET /ScheduledTasks`: the row whose `Name` is exactly
`Scan Media Library`, and its `State` (`Idle`|`Running`).** `/Library/ScanStatus`
is a 404 on 12.4, so a watcher built on the old endpoint reports nothing and a
scanning server looks an idle one. Call it settled only after a `Running`
sighting followed by two consecutive `Idle`s: sub-tasks leave gaps that a single
poll reads as finished, and a census taken in one reports the pre-scan library.
**The per-library `POST /Items/{id}/Refresh` queues nothing, and the provider
switch is not the reason.** Measured twice on 2026-11-02 against `Shows`: before
the flip and again after it, the call answered `204`, the journal wrote zero lines
over the following ten minutes, and `Scan Media Library` kept its never-run
signature; a service restart changes nothing. `POST /Library/Refresh` does run —
`204`, then `Running` inside 15 s at 33 % CPU — but it walks every library, so on
this build a library-scoped metadata pass has no working door and the global one
is the only lever.
**That endpoint takes its parameters from one source.** The documented query form
(empty body) and the JSON body form (no query) each parse; supplying both at once
answers `400` with an empty body whatever the values, and `ValidationFailure` is
rejected in the body while the query form accepts it.
**`RefreshStatus` on the `GET /Library/VirtualFolders` record is not a progress
read.** It stayed `Idle` through a scan that was verifiably running at 75 %. The
record's useful field is `ItemId`, and the live library set is still the entries
whose `Locations` are non-empty.
`inotify` does not cross a rootless podman bind, so folders created on the host
stay invisible until a pass like this runs; every pass that writes into a library
ends by firing one.
**`Movies` has no watcher at all.** `EnableRealtimeMonitor` is per library (table in
*Data layout*) and it is false for `Movies` and `Collections`. Nothing notices a
@@ -41,7 +41,7 @@ python3 scripts/media-audiobooks.py --root /mnt/entropy/Audiobooks \
# library too ([[Audiobookshelf]]) — otherwise it keeps rows for folders that
# no longer exist.
scripts/media-apply.py apply --plan /tmp/ab_plan.tsv --root /mnt/entropy \
--journal /mnt/entropy/media-relayout-audiobooks.jsonl --apply
--journal /mnt/entropy/.media-campaign/media-relayout-audiobooks.jsonl --apply
```
The group is the identity Plex asserts in its own database — **(narrator, book,
@@ -68,7 +68,7 @@ no chapter row names, and a loose file in a folder holding two books all park un
that unrelated books share, or a chapter number mistagged as the artist (`0.4`).
`Various Artists` is not a placeholder — that is a multi-cast read.
The reviewed plan lives at `/mnt/entropy/media-relayout-audiobooks-plan.tsv`; it
The reviewed plan lives at `/mnt/entropy/.media-campaign/media-relayout-audiobooks-plan.tsv`; it
applies on its own journal, behind the music writer, and then refreshes the Plex
section. Verify it with `--db-kind plex` — Jellyfin holds no rows here, so a
Jellyfin pass proves nothing either way.
@@ -65,8 +65,8 @@ executor's own format; it never moves anything itself.
python3 scripts/media-tvdupes.py --library /mnt/entropy/TVShows \
--hold /mnt/entropy/.AGENTTRASH/<date>-tv-duplicates \
--meta /var/lib/media-meta/view/TVShows --witness witness.tsv \
--ids /mnt/entropy/tv-tvdb-ids.tsv \
--out /mnt/entropy/tv-dupes-plan.tsv --json /tmp/tv-groups.json
--ids /mnt/entropy/.media-campaign/tv-tvdb-ids.tsv \
--out /mnt/entropy/.media-campaign/tv-dupes-plan.tsv --json /tmp/tv-groups.json
# then media-apply.py apply / verify / sweep / rollback, exactly as for a film prune
```
@@ -76,7 +76,7 @@ python3 scripts/media-tvdupes.py --library /mnt/entropy/TVShows \
match, not a lookup: `BaseItems` (`Type like '%TV.Series'`) joined to
`BaseItemProviders` where `ProviderId = 'Tvdb'`, the `/data/TVShows/` prefix
stripped to the bare top-level folder name — 616 folders, none carrying two
ids, written to `/mnt/entropy/tv-tvdb-ids.tsv`. Two folders with disjoint ids
ids, written to `/mnt/entropy/.media-campaign/tv-tvdb-ids.tsv`. Two folders with disjoint ids
are two shows however alike they read — *Avatar: The Last Airbender* (1999)
beside the 2024 remake, *Millennium* (1996) beside a German thriller Jellyfin
had also titled *Millennium* — and a pair whose identity cannot be proven is
@@ -158,12 +158,12 @@ Nothing is deleted. Every doomed copy moves into a dated hold with the same
executor the relayout uses, so any file comes back:
```bash
scripts/media-apply.py apply --plan /mnt/entropy/dup-prune-similar-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/dup-prune-similar.jsonl # dry
scripts/media-apply.py apply --plan /mnt/entropy/.media-campaign/dup-prune-similar-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/.media-campaign/dup-prune-similar.jsonl # dry
scripts/media-apply.py apply --plan … --root /mnt/entropy --journal … --apply
scripts/media-apply.py verify --plan … --journal /mnt/entropy/dup-prune-similar.jsonl
scripts/media-apply.py sweep --journal /mnt/entropy/dup-prune-similar.jsonl --root /mnt/entropy --apply
scripts/media-apply.py rollback --journal /mnt/entropy/dup-prune-similar.jsonl --root /mnt/entropy --apply
scripts/media-apply.py verify --plan … --journal /mnt/entropy/.media-campaign/dup-prune-similar.jsonl
scripts/media-apply.py sweep --journal /mnt/entropy/.media-campaign/dup-prune-similar.jsonl --root /mnt/entropy --apply
scripts/media-apply.py rollback --journal /mnt/entropy/.media-campaign/dup-prune-similar.jsonl --root /mnt/entropy --apply
```
The plan rows are the tool's own format (`action src dst kind`, `mkdir` rows for
@@ -33,7 +33,7 @@ python3 scripts/media-music.py --root /mnt/entropy/Music --enum /tmp/music_enum.
# 4. Read the exceptions file FIRST — it is the review, not a log. Then the same
# executor, journal and rollback as [[Media Relayout]] shows, on its own journal.
scripts/media-apply.py apply --plan /tmp/music_plan.tsv --root /mnt/entropy \
--journal /mnt/entropy/media-relayout-music.jsonl --apply
--journal /mnt/entropy/.media-campaign/media-relayout-music.jsonl --apply
```
The group is what the tags assert, not where the files sit: **(album-artist tag,
@@ -3,7 +3,7 @@ type: reference
title: Media Relayout
description: "Census, plan and apply the movie, TV, music and audiobook libraries: the gates, the refusals, the rollback."
tags: [dankserver, media-server, jellyfin, plex, procedure]
timestamp: 2026-11-01T01:00:00Z
timestamp: 2026-11-02T11:00:00Z
---
# Media Relayout
@@ -71,12 +71,23 @@ that would be filed wrong, and the plan is not run until it is understood.
```bash
scripts/media-apply.py apply --plan /tmp/jf-plan.tsv --root /mnt/entropy/Movies # dry, writes nothing
scripts/media-apply.py apply --plan /tmp/jf-plan.tsv --root /mnt/entropy/Movies --apply
scripts/media-apply.py verify --plan /tmp/jf-plan.tsv --journal /mnt/entropy/media-relayout.jsonl
scripts/media-apply.py sweep --journal /mnt/entropy/media-relayout.jsonl --root /mnt/entropy/Movies # dry
scripts/media-apply.py sweep --journal /mnt/entropy/media-relayout.jsonl --root /mnt/entropy/Movies --apply
scripts/media-apply.py rollback --journal /mnt/entropy/media-relayout.jsonl --root /mnt/entropy/Movies --apply
scripts/media-apply.py verify --plan /tmp/jf-plan.tsv --journal /mnt/entropy/.media-campaign/media-relayout.jsonl
scripts/media-apply.py sweep --journal /mnt/entropy/.media-campaign/media-relayout.jsonl --root /mnt/entropy/Movies # dry
scripts/media-apply.py sweep --journal /mnt/entropy/.media-campaign/media-relayout.jsonl --root /mnt/entropy/Movies --apply
scripts/media-apply.py rollback --journal /mnt/entropy/.media-campaign/media-relayout.jsonl --root /mnt/entropy/Movies --apply
```
**Every campaign artifact — plan, journal, exceptions file, run log — lives in
`/mnt/entropy/.media-campaign/`.** The array root carries libraries and nothing
else: a root strewn with `*.jsonl` and `*-plan.tsv` gets browsed as content, and
the dot-directory is ignored by both media servers. Relocated 2026-11-02, 48 files,
undo with `media-apply.py rollback --journal /mnt/entropy/.media-campaign/media-campaign-relocate.jsonl --root /mnt/entropy --apply`.
The holds and trash (`.AGENTTRASH`, `.HOLD-<date>-*`) stay at the root on purpose:
the purge machinery is given those paths as `--hold`, and a hold that moves out
from under the tool that made it is a hold nobody can find. Name `--journal`
explicitly on every run — the tool's default is `media-relayout.jsonl` beside
`--root`, so a bare run recreates exactly the root-level log this page forbids.
An apply also leaves the release folders it emptied standing where they were —
hundreds of them at the library root — which is what `sweep` removes, dry-run
first, from the same journal.
@@ -110,7 +121,7 @@ a read beside a running scan can fail as "database disk image is malformed".
`immutable` stays only the fallback for a database whose `-shm` cannot be read.
```bash
scripts/media-verify.py --journal /mnt/entropy/media-relayout-music.jsonl \
scripts/media-verify.py --journal /mnt/entropy/.media-campaign/media-relayout-music.jsonl \
--db-kind jellyfin --ignore-ext jpg,jpeg,png,log,cue,toc,nfo,txt,m3u,accurip
```
@@ -150,11 +161,11 @@ by series, and writes a plan in the executor's own format:
```bash
scripts/media-tv.py --root /mnt/entropy/Movies --library /mnt/entropy/TVShows \
--out /mnt/entropy/media-tv-plan.tsv
scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv --root /mnt/entropy
scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv --root /mnt/entropy \
--journal /mnt/entropy/media-relayout-tv.jsonl --apply
scripts/media-apply.py sweep --journal /mnt/entropy/media-relayout-tv.jsonl \
--out /mnt/entropy/.media-campaign/media-tv-plan.tsv
scripts/media-apply.py apply --plan /mnt/entropy/.media-campaign/media-tv-plan.tsv --root /mnt/entropy
scripts/media-apply.py apply --plan /mnt/entropy/.media-campaign/media-tv-plan.tsv --root /mnt/entropy \
--journal /mnt/entropy/.media-campaign/media-relayout-tv.jsonl --apply
scripts/media-apply.py sweep --journal /mnt/entropy/.media-campaign/media-relayout-tv.jsonl \
--root /mnt/entropy --apply
```
@@ -186,162 +197,10 @@ show. A name whose marker is `Part 2` is a film and stays put.
## The TV library's own relayout
`/mnt/entropy/TVShows` is scene packs rather than a library, and Jellyfin has
already matched and numbered most of it. `scripts/media-shows.py` re-files that
library into the shape Jellyfin reads, with **tvdb as the name authority**:
<Series> (<year>) [tvdbid-NNNN]/Season NN/
<Series> (<year>) - SxxEyy - <Episode Title>.ext
It never guesses: every series name, episode number and title it writes traces to
an authority, and what has none is parked for a human to read.
```bash
# 1. The authority: Jellyfin's own matched rows — each carries the series' TMDb
# and Tvdb ids and the title the row holds (the DB opens read-only).
python3 scripts/media-census.py jellyfin --root /mnt/entropy/TVShows \
--out /tmp/tv_titles.tsv
# 2. What is on disk (stat only), and the runtimes the film arm needs.
python3 scripts/media-census.py enumerate --root /mnt/entropy/TVShows \
--out /tmp/tv_enum.tsv
python3 scripts/media-census.py probe --root /mnt/entropy/TVShows \
--enum /tmp/tv_enum.tsv --jobs 6 --out /tmp/tv_probed.tsv
# The probe is the slow step: ffprobe reads both ends of every file. So re-use
# the last one — a duration belongs to the file, not to its name: re-key the
# previous census through that pass's own journal (`src` -> `dst`) instead of
# seeking the array again. It is also the only column the film arm reads, and
# only for a file with no marker and no provider row (4,794 of 58,551 today,
# all of them covered by a re-key that costs seconds, not an hour).
# 3. The plan. --tmdb needs TMDB_API_KEY — the v3 key in ~/.config/secrets/tmdb,
# which travels as the `api_key` query parameter. Only a v4 request token is
# accepted as a Bearer header, and the wrong shape answers 401 on EVERY call,
# which the summary line flags as NOT provider-complete. --tvdb reads its
# `api_key=` line from ~/.config/secrets/tvdb, never argv. Both write the SAME
# cache file, so the plan is reproducible offline afterwards: run it once to
# fill the cache, and quote the second run.
# Paths are only strings to a dry run, so this step can run on the workstation
# against scp'd TSVs — which is where the two provider secrets live.
python3 scripts/media-shows.py --root /mnt/entropy/TVShows --enum /tmp/tv_enum.tsv \
--titles /tmp/tv_titles.tsv --probed /tmp/tv_probed.tsv \
--tmdb-cache /mnt/entropy/media-tv-names.json --tmdb --tvdb \
--park /mnt/entropy/media-park/TVShows \
--out /mnt/entropy/media-relayout-shows2-plan.tsv \
--exceptions /mnt/entropy/media-relayout-shows2-exceptions.txt
# 4. Read the exceptions file FIRST — it is the review, not a log. Same executor
# as above, but on this arm's OWN journal, NAMED PER PASS: the journal of an
# applied pass is its unwind record, so a second pass that reuses the name
# destroys the only map back.
scripts/media-apply.py apply --plan /mnt/entropy/media-relayout-shows2-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/media-relayout-shows2.jsonl --apply
```
Evidence, strongest first — the order the planner believes things:
- the **episode marker the file's own name states** (`S01E02-E03`, `1x04`); a digit
run longer than three at that position is NOT a number — `S03E2019` and
`S04E47370355` are one box set's airdate fragment and pack index — so such a file
is renumbered by tvdb's title or parks, and is never truncated down to fit;
- the **`Season NN` folder it sits in and the number its name leads with inside
it**: `Season 19/01 Charles Barkley, Nirvana.avi` is S19E01;
- the **provider row**: which episode the file IS, and the numbering of a file
whose own name says nothing;
- the **row's tvdb id, else its TMDb id carried through TMDb's external ids, else
an exact tvdb name search**: that id names the SERIES FOLDER and supplies the
episode title, because a name the server read off a folder is not a show name.
The folder CARRIES the id: `[tvdbid-NNNN]` is what Jellyfin's Tvdb agent and
Plex's scanner both read, so a show whose title is ambiguous resolves by number
instead of by a human guessing which folder is which. The id rides the folder
only — a file name gets 180 characters and the episode title needs the room.
- a **bonus folder** anywhere above the file, or Jellyfin's `ExtraType` → the
series' canonical extra folder;
- a runtime over `SHORT_MAX` makes it film-shaped, and it leaves for `--movies` only
when the movie provider's title is a run of words the FILE itself states and its
year agrees with the file's: the query is only what `title_of` salvaged from a
release name, and the first hit for a short one is a near miss — a 1963 Bond film
for the word "russia", *Special Edition I* for a file saying *Edition III*. Else it
parks.
Four refusals carry the design: what the planner will NOT do is the point of it.
- **Folders resolve a series; they never invent one.** A folder's words may
resolve a file to a series a provider has already named — `(CBB) Tengen Toppa
Gurren Lagann - The Complete Series (BD 1080p)` names the registered *Tengen
Toppa Gurren Lagann* — but a bonus subfolder is not a show. An early run made
`…/03. Extra/01. NCOP/NCOP 01.mkv` into a root series called `01 NCOP`.
- **A scraped row does not outrank the filesystem.** Jellyfin numbers
multi-season packs badly: every file under `SNL 16-20/Season 16..19` carries
season 16 episode 20, and one Gundam row says season 2004. The folder, then
the file's own leading number, win.
- **A season that is a YEAR is the date matcher, not a season.** Jellyfin files
specials by airdate, so a row can arrive saying `season 2002`; that is not a
season and it never becomes a folder. The one authority on such a row is tvdb's
episode list read backwards: the title the row carries names the (season,
episode) tvdb gives it. A title tvdb gives to two episodes names nothing, and
the file parks with the reason it always carried. `Season 00` IS a legal
season — specials live there — so a row that really says 0 files there instead
of parking.
- **A season tvdb's own list contradicts gets no tvdb title.** An anime pack
numbered by its release group books `S05E01` in a season tvdb does not hold, or
books more episodes there than tvdb lists: the slot its number points at is a
different episode, and dressing it in that episode's title is a confident lie.
The file still renames to the standard shape; the title does not ride, and the
series is listed under *seasons whose numbering tvdb contradicts* in the
exceptions. A season holding FEWER files than tvdb lists stays plausible — a
partial pack is the normal shape. Neither reading holds for a list tvdb
cut off at its own page limit: the default episode query answers at most
500 rows and states no total, and One Piece, Crayon Shin-chan and
Detective Conan each answer exactly 500 while holding over a thousand —
a page bounds nothing above itself, so it contradicts nothing.
**A provider name is evidence only when an id backs it.** Jellyfin names a Series
after the folder it scanned, so a library filed by release folders ends up with
one Series item per box set — `24 (2001) Season 1`, `Modern Family S05` — and
copying those names verbatim launders folder junk into the contract. The planner
resolves each series to its tvdb id and writes tvdb's name and year; the merged
identity is also what turns a box set split across five release folders into ONE
folder, and what makes two files claiming one episode MEET in the duplicate rule.
A series no id resolves keeps the provider's words and is listed under *no tvdb
identity* in the exceptions — that list, not a rename, is the review.
**The review pile lives outside the library.** Files parked inside
`/mnt/entropy/TVShows` were indexed as one show named `_unsorted`, and that
show's rows then looked like evidence. So a row whose series is the hidden
top-level folder the file itself sits in is dropped — and the folder is matched
with its leading underscore stripped, because the server *displays* `_unsorted` as
`unsorted`: compare the spelled names and the pile survives the relayout as a
visible series folder that real files get renamed into. `--park` outside the root
makes the next plan re-file the pile instead of skipping it. Inside a hidden
top-level folder a file's own words may MATCH a registered show, never invent one.
**The operator's id map is the door for a name tvdb cannot be searched by.**
`scripts/media-shows.py` reads `scripts/media-shows-tvdb-ids.tsv` by default: one
line per pack folder, `folder <TAB> tvdb id <TAB> note`. The file exists because
tvdb titles a record in the show's own language and the English name rides in
*aliases* the name search never reads: `ブラッククローバー` (331753) answers to no
spelling of "Black Clover", and the bare alias `One Piece` belongs to 392276, the
2023 live action, not to the 1999 record (81797) the pack here is. The map
outranks a provider id — a box set's provider id is the server's own guess at a
folder name — and it merges the release folders that are one show. Everything it
touched is written to the exceptions: every folder it named, every row that
matched no folder, every disagreement with a provider id, every id tvdb never
answered. A line whose id is not a number stops the plan, because a typo renames
a whole pack to the wrong show. Only folders the search CANNOT reach belong in
it; a row for a folder the search would resolve hides a live lookup behind a
number nobody re-checks.
`scripts/media-shows-tvdb.py` makes the sheet the map is built from. It reads a
plan's `mkdir` rows, asks tvdb about every destination folder carrying no
`[tvdbid-`, and writes one row per folder with the verdict that says how much was
guessed to get there: `EXACT` (tvdb's own spelling), `PUNCT` (equal only after
release junk is stripped from the folder's words), `NEAR` (tvdb answered
something else — the best hit is printed for a human and nothing applies it),
`NONE`. Acceptance never moved across the passes that built this: a hit counts
only when tvdb's name equals the query with every non-alphanumeric dropped from
both sides, and the record's year still has to fall inside any year span the
folder names. That gate is what caught three confident picks — Fairy Tail's 2024
sequel (410031), the 2026 Hokuto no Ken remake (465216) and `Mugyutto! Black
Clover` (378762) offered for Black Clover. It is why the map is approved by hand
and the sheet is never fed to the planner.
`/mnt/entropy/TVShows` is its own arm: a different authority (tvdb), its own
refusals, and its own test for whether a folder really holds two shows —
[[Media TV Relayout]]. It shares this page's executor, the one-journal-per-arm
law, the sweep and the rollback.
## The music and audiobook arms
@@ -403,9 +262,12 @@ executor, the one-journal-per-arm law, the sweep and the rollback.
- **The journal lives BESIDE the library root, never inside it** (default:
`media-relayout.jsonl` next to the root): a log inside the library is a file
Jellyfin indexes.
- **Refusals are all-or-nothing.** An occupied target, a vanished source or a
duplicate destination stops the run *before the first rename*, so the operator
fixes one thing and re-runs instead of untangling a half-applied plan.
- **Refusals are all-or-nothing.** An occupied target, a vanished source, a
duplicate destination, or a plan row whose verb is not `mkdir`, `keep` or
`move`, stops the run *before the first rename*, so the operator fixes one
thing and re-runs instead of untangling a half-applied plan. An unread verb is
refused by line number: silently dropping one turned a pass meant to rename
two series folders into `applied: 0 moved` with exit 0 (2026-11-02).
`--force` relaxes only the opinion guards; it never permits overwriting.
- **A sweep deletes only what the journal proves the run emptied.** A folder is
eligible when a journalled file moved out of it or out of something under it,
@@ -432,6 +294,14 @@ executor, the one-journal-per-arm law, the sweep and the rollback.
`--root` set to the share that holds BOTH libraries: the executor refuses any
path outside `--root`, and a journal shared with the movie relayout would
undo both runs when only one of them is wanted.
- **A hold folder outside `--root` forces `--force` on BOTH `apply` and
`sweep`.** `sweep` refuses the entire journal when any journalled path sits
outside the root (`this journal belongs to another library`) — the same guard
the plan hits, and it exists to stop one pass unwinding another's files.
`--force` relaxes it and stays safe there: `sweep_targets` marks only
ancestors under `--root`, so a hold folder beyond the root can never be
selected; it only blocks everything that is. The cleaner shape is the law
above — put `--root` on the share that holds both.
- **Sidecar metadata does not follow a rename, because it lives on another
filesystem.** The media servers write `.nfo` and artwork through the
@@ -440,6 +310,9 @@ executor, the one-journal-per-arm law, the sweep and the rollback.
`scripts/media-view.sh reap --apply` — stranded sidecars become ghost folders
that both servers browse and report as content. Whether this host mounts the
view at all is stated on that page; `media-apply.py` stays silent when it does not.
A rename of folders the server never identified strands nothing: Jellyfin
writes a sidecar only for a series it recognised, so `reap` answering
`no ghosts` after such a pass is the expected result, not a broken tool.
- **Nothing deletes through the metadata view.** A delete there writes an overlayfs
whiteout in the upper layer, which hides the array's copy without removing it;
the file returns when the view is unmounted. Deletions in this pipeline happen
@@ -0,0 +1,241 @@
---
type: reference
title: Media TV Relayout
description: "Re-file TVShows from scene packs into the shape Jellyfin reads: authority order, refusals, the two-series split test."
tags: [dankserver, media-server, jellyfin, plex, procedure]
timestamp: 2026-11-02T11:00:00Z
---
# Media TV Relayout
The `/mnt/entropy/TVShows` share arrived as scene packs — release folders whose
names are the release, not the show. This arm re-files it into the shape
[[Jellyfin]] reads, with tvdb as the name authority. The executor, the journal,
the sweep and the rollback belong to [[Media Relayout]]; this page owns the TV
authority order, the refusals, and the test that decides a split.
`/mnt/entropy/TVShows` is scene packs rather than a library, and Jellyfin has
already matched and numbered most of it. `scripts/media-shows.py` re-files that
library into the shape Jellyfin reads, with **tvdb as the name authority**:
<Series> (<year>) [tvdbid-NNNN]/Season NN/
<Series> (<year>) - SxxEyy - <Episode Title>.ext
It never guesses: every series name, episode number and title it writes traces to
an authority, and what has none is parked for a human to read.
```bash
# 1. The authority: Jellyfin's own matched rows — each carries the series' TMDb
# and Tvdb ids and the title the row holds (the DB opens read-only).
python3 scripts/media-census.py jellyfin --root /mnt/entropy/TVShows \
--out /tmp/tv_titles.tsv
# 2. What is on disk (stat only), and the runtimes the film arm needs.
python3 scripts/media-census.py enumerate --root /mnt/entropy/TVShows \
--out /tmp/tv_enum.tsv
python3 scripts/media-census.py probe --root /mnt/entropy/TVShows \
--enum /tmp/tv_enum.tsv --jobs 6 --out /tmp/tv_probed.tsv
# The probe is the slow step: ffprobe reads both ends of every file. So re-use
# the last one — a duration belongs to the file, not to its name: re-key the
# previous census through that pass's own journal (`src` -> `dst`) instead of
# seeking the array again. It is also the only column the film arm reads, and
# only for a file with no marker and no provider row (4,794 of 58,551 today,
# all of them covered by a re-key that costs seconds, not an hour).
# 3. The plan. --tmdb needs TMDB_API_KEY — the v3 key in ~/.config/secrets/tmdb,
# which travels as the `api_key` query parameter. Only a v4 request token is
# accepted as a Bearer header, and the wrong shape answers 401 on EVERY call,
# which the summary line flags as NOT provider-complete. --tvdb reads its
# `api_key=` line from ~/.config/secrets/tvdb, never argv. Both write the SAME
# cache file, so the plan is reproducible offline afterwards: run it once to
# fill the cache, and quote the second run.
# Paths are only strings to a dry run, so this step can run on the workstation
# against scp'd TSVs — which is where the two provider secrets live.
python3 scripts/media-shows.py --root /mnt/entropy/TVShows --enum /tmp/tv_enum.tsv \
--titles /tmp/tv_titles.tsv --probed /tmp/tv_probed.tsv \
--tmdb-cache /mnt/entropy/.media-campaign/media-tv-names.json --tmdb --tvdb \
--park /mnt/entropy/NeedsProcessing/TVShows \
--out /mnt/entropy/.media-campaign/media-relayout-shows2-plan.tsv \
--exceptions /mnt/entropy/.media-campaign/media-relayout-shows2-exceptions.txt
# 4. Read the exceptions file FIRST — it is the review, not a log. Same executor
# as above, but on this arm's OWN journal, NAMED PER PASS: the journal of an
# applied pass is its unwind record, so a second pass that reuses the name
# destroys the only map back.
scripts/media-apply.py apply --plan /mnt/entropy/.media-campaign/media-relayout-shows2-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/.media-campaign/media-relayout-shows2.jsonl --apply
```
Evidence, strongest first — the order the planner believes things:
- the **episode marker the file's own name states** (`S01E02-E03`, `1x04`); a digit
run longer than three at that position is NOT a number — `S03E2019` and
`S04E47370355` are one box set's airdate fragment and pack index — so such a file
is renumbered by tvdb's title or parks, and is never truncated down to fit;
- the **`Season NN` folder it sits in and the number its name leads with inside
it**: `Season 19/01 Charles Barkley, Nirvana.avi` is S19E01;
- the **provider row**: which episode the file IS, and the numbering of a file
whose own name says nothing;
- the **row's tvdb id, else its TMDb id carried through TMDb's external ids, else
an exact tvdb name search**: that id names the SERIES FOLDER and supplies the
episode title, because a name the server read off a folder is not a show name.
The folder CARRIES the id: `[tvdbid-NNNN]` is what Jellyfin's Tvdb agent and
Plex's scanner both read, so a show whose title is ambiguous resolves by number
instead of by a human guessing which folder is which. The id rides the folder
only — a file name gets 180 characters and the episode title needs the room.
- a **bonus folder** anywhere above the file, or Jellyfin's `ExtraType` → the
series' canonical extra folder;
- a runtime over `SHORT_MAX` makes it film-shaped, and it leaves for `--movies` only
when the movie provider's title is a run of words the FILE itself states and its
year agrees with the file's: the query is only what `title_of` salvaged from a
release name, and the first hit for a short one is a near miss — a 1963 Bond film
for the word "russia", *Special Edition I* for a file saying *Edition III*. Else it
parks.
Four refusals carry the design: what the planner will NOT do is the point of it.
- **Folders resolve a series; they never invent one.** A folder's words may
resolve a file to a series a provider has already named — `(CBB) Tengen Toppa
Gurren Lagann - The Complete Series (BD 1080p)` names the registered *Tengen
Toppa Gurren Lagann* — but a bonus subfolder is not a show. An early run made
`…/03. Extra/01. NCOP/NCOP 01.mkv` into a root series called `01 NCOP`.
- **A scraped row does not outrank the filesystem.** Jellyfin numbers
multi-season packs badly: every file under `SNL 16-20/Season 16..19` carries
season 16 episode 20, and one Gundam row says season 2004. The folder, then
the file's own leading number, win.
- **A season that is a YEAR is the date matcher, not a season.** Jellyfin files
specials by airdate, so a row can arrive saying `season 2002`; that is not a
season and it never becomes a folder. The one authority on such a row is tvdb's
episode list read backwards: the title the row carries names the (season,
episode) tvdb gives it. A title tvdb gives to two episodes names nothing, and
the file parks with the reason it always carried. `Season 00` IS a legal
season — specials live there — so a row that really says 0 files there instead
of parking.
- **A season tvdb's own list contradicts gets no tvdb title.** An anime pack
numbered by its release group books `S05E01` in a season tvdb does not hold, or
books more episodes there than tvdb lists: the slot its number points at is a
different episode, and dressing it in that episode's title is a confident lie.
The file still renames to the standard shape; the title does not ride, and the
series is listed under *seasons whose numbering tvdb contradicts* in the
exceptions. A season holding FEWER files than tvdb lists stays plausible — a
partial pack is the normal shape. Neither reading holds for a list tvdb
cut off at its own page limit: the default episode query answers at most
500 rows and states no total, and One Piece, Crayon Shin-chan and
Detective Conan each answer exactly 500 while holding over a thousand —
a page bounds nothing above itself, so it contradicts nothing.
**A provider name is evidence only when an id backs it.** Jellyfin names a Series
after the folder it scanned, so a library filed by release folders ends up with
one Series item per box set — `24 (2001) Season 1`, `Modern Family S05` — and
copying those names verbatim launders folder junk into the contract. The planner
resolves each series to its tvdb id and writes tvdb's name and year; the merged
identity is also what turns a box set split across five release folders into ONE
folder, and what makes two files claiming one episode MEET in the duplicate rule.
A series no id resolves keeps the provider's words and is listed under *no tvdb
identity* in the exceptions — that list, not a rename, is the review.
**The review pile lives outside the library.** Files parked inside
`/mnt/entropy/TVShows` were indexed as one show named `_unsorted`, and that
show's rows then looked like evidence. So a row whose series is the hidden
top-level folder the file itself sits in is dropped — and the folder is matched
with its leading underscore stripped, because the server *displays* `_unsorted` as
`unsorted`: compare the spelled names and the pile survives the relayout as a
visible series folder that real files get renamed into. `--park` outside the root
makes the next plan re-file the pile instead of skipping it. Inside a hidden
top-level folder a file's own words may MATCH a registered show, never invent one.
**The operator's id map is the door for a name tvdb cannot be searched by.**
`scripts/media-shows.py` reads `scripts/media-shows-tvdb-ids.tsv` by default: one
line per pack folder, `folder <TAB> tvdb id <TAB> note`. The file exists because
tvdb titles a record in the show's own language and the English name rides in
*aliases* the name search never reads: `ブラッククローバー` (331753) answers to no
spelling of "Black Clover", and the bare alias `One Piece` belongs to 392276, the
2023 live action, not to the 1999 record (81797) the pack here is. The map
outranks a provider id — a box set's provider id is the server's own guess at a
folder name — and it merges the release folders that are one show. Everything it
touched is written to the exceptions: every folder it named, every row that
matched no folder, every disagreement with a provider id, every id tvdb never
answered. A line whose id is not a number stops the plan, because a typo renames
a whole pack to the wrong show. Only folders the search CANNOT reach belong in
it; a row for a folder the search would resolve hides a live lookup behind a
number nobody re-checks.
`scripts/media-shows-tvdb.py` makes the sheet the map is built from. It reads a
plan's `mkdir` rows, asks tvdb about every destination folder carrying no
`[tvdbid-`, and writes one row per folder with the verdict that says how much was
guessed to get there: `EXACT` (tvdb's own spelling), `PUNCT` (equal only after
release junk is stripped from the folder's words), `NEAR` (tvdb answered
something else — the best hit is printed for a human and nothing applies it),
`NONE`. Acceptance never moved across the passes that built this: a hit counts
only when tvdb's name equals the query with every non-alphanumeric dropped from
both sides, and the record's year still has to fall inside any year span the
folder names. That gate is what caught three confident picks — Fairy Tail's 2024
sequel (410031), the 2026 Hokuto no Ken remake (465216) and `Mugyutto! Black
Clover` (378762) offered for Black Clover. It is why the map is approved by hand
and the sheet is never fed to the planner.
**A parked folder merges into the row that owns the show.** The folder that never
got an id usually holds episodes the identified folder for that show lacks, and
the server shows the audience neither: the parked copy is invisible and the owner
reads as a short series. One `media-apply.py` plan per show, built from the
server's own rows:
- the **owner is the folder whose `Series` row carries the tvdb id**; the parked
side is the folder with the same words and no id (or the same id under a second
row — see the law below);
- the **comparison is episode identity, not filename**: which (season, episode)
the provider row gives each file, read straight out of the server. Release
folders renumber — one park carried `S01E01-11` of a show tvdb books as seasons
2 and 3, and every episode runs the same eleven minutes, so neither the marker
nor the duration says what one query into the rows says;
- **a season the owner has no file in moves whole**: `mkdir` the `Season NN`,
then one row per file with its name unchanged — the folder shape is the
contract, the file name is not;
- **a subtitle pack is re-keyed, never just moved**: a sidecar attaches only when
its stem equals the video's, so each `.srt` has to land on the OWNER's stem with
the language suffix kept (`… - The Cat and the Claw (1).en.srt`). Moved under
its release-group name it stays a folder of 88 subtitles no player ever offers;
- **the same episode on both sides is the operator's call**, never a plan row:
which copy survives is a quality judgment — the parked copy is usually the
bigger one, and a re-cut is not a worse one — and the parked file often carries
no title to align by. Those pairs are listed and left alone.
The two sides speak different path dialects, since the server records the
container coordinate ([[Jellyfin]]) and the journal speaks the array; a join that
forgets the translation returns zero rows and looks like an empty library.
## Laws of this arm
- **A folder that names two series is not evidence of two series — the files
answer.** `ffprobe -show_entries format_tag=title` names the series each file
belongs to. Measured 2026-11-02 on `COW and CHICKEN … Complete Animated TV
Series`, a bundle whose name pairs *Cow and Chicken* with *I Am Weasel*: 103
of 104 files carry a `Cow and Chicken - SxxEyy - <title>` tag and none carries
an *I Am Weasel* one. The 7-minute duration class is a half-episode segment,
not a second show, and a file holding both segments needs a remux, which this
executor will never do. Split only when the tags (or a provider episode list)
name both series per file; everything else is a rename.
- **When the numbering is right and only the name is wrong, rename the folder
and nothing else.** Tags whose `Sxx` matches the `Season NN` folder holding
them, per-season counts equal, and no duplicate `SxxEyy` key anywhere prove
the layout is already correct. Renumbering files under that evidence buys
nothing and can silently mislabel an episode.
- **One tvdb id can carry two `Series` rows, so the id-less folder list proves
nothing about a merge.** A parked folder whose words parse to a registered show
gets identified on its own and keeps its own folder: the count of id-less
folders drops, and the library now offers the same show twice. Prove a merge by
the episode rows standing under the owner folder, never by that list.
- **Move library files only while `Scan Media Library` is Idle.** A rename that
lands under a running scan is a row the server read a moment ago pointing at a
path that no longer exists. `ScheduledTasks` is the read ([[Jellyfin]]); wait
for a `Running` sighting and then two consecutive `Idle` polls, because a gap
between sub-tasks reads as finished.
## See Also
[[Media Relayout]] — the executor, the journal, the sweep and the rollback this
arm shares. [[Jellyfin]] — the reader of the resulting shape. [[Plex]] — the
second reader. [[Media Metadata View]] — where a rename's sidecars strand, or
do not.
## Citations
- Jellyfin Shows documentation — series, season and episode folder/file naming:
<https://jellyfin.org/docs/general/server/media/shows>
@@ -4,7 +4,7 @@ description: "Outpost, proxy-provider, OAuth2, LDAP and flow configuration for t
tags:
- infrastructure
- authentik
timestamp: 2026-10-19
timestamp: 2026-11-03
---
# Authentik Outpost Reference
@@ -56,7 +56,7 @@ correctly — the outpost matches on hostname.
## Provisioning rules for proxy providers
Two provisioning rules, each verified live by an outage:
Three provisioning rules, each verified live by an outage or a dead login:
1. **All three flows are required.** A ProxyProvider created via raw API/ORM
with any of `authentication_flow`/`authorization_flow`/`invalidation_flow`
@@ -80,6 +80,16 @@ Two provisioning rules, each verified live by an outage:
the app. No error is logged anywhere — the SSO "works" and the app never
appears. Create the Application with `provider_id` set (the UI does this
automatically; raw API/ORM drives must do it explicitly).
3. **A raw ORM create skips the OAuth defaults.** The REST serializer calls
`ProxyProvider.set_oauth_defaults()`; `ProxyProvider.objects.create()` does not. The row
then looks fully configured — external_host, internal_host, application bound, outpost
claimed — while `redirect_uris` is empty (those URIs are *derived* from `external_host` by
`_get_callback_url`) and `grant_types` is `[]`. Both kill the guard before any credential,
in this order: authentik renders its own **"Redirect URI Error"** page, and once the URIs
are filled by hand the next wall is a bodiless **HTTP 400** out of `/application/o/authorize/`
(both measured on the Audiobookshelf guard provider, 2026-11-03). Census and repair in one idempotent pass, all
providers at once: `scripts/authentik-proxy-provider-defaults.sh`; `--check` prints the
census and exits 1 on drift, and a provider that already matches is left alone.
## OAuth2 relying-party quirks (this fork; corrected live 2026-10-20)
+59 -2
View File
@@ -4,7 +4,7 @@ description: "Caddy is the single reverse proxy for both hosts."
tags:
- infrastructure
- caddy
timestamp: 2026-08-29
timestamp: 2026-11-03
---
# Caddy
@@ -90,7 +90,64 @@ pihole.franksdank.cloud {
After rendering, the line reads `dankuser <hash>` (one complete hash) — `tests/deployment_flow.bats:96-99` pins the `dankuser` name and the single-hash form, and `tests/caddyfile.bats:75` pins the bare `dankuser REPLACE_WITH_HASH` template. Do not put a `$2a$14$` prefix in the template: it would render a double-prefixed, invalid hash (2026-08-22 incident — every basicauth vhost returned 401 for every password; `tests/caddyfile.bats:67-72` forbids the prefix). `tests/deployment_flow.bats` pins the render contract.
Everything else authenticates, not basicauth. The guarded set — `www.` (homepage), `search.`, `grafana.`, `prometheus.`, `loki.`, `seerr.`, `marreta.`, `romm.`, `files.` — proxies **all** traffic to `anubis:8923`, which verifies the client and forwards it back to Caddy's internal listener `http://:8016`; Authentik's proxy-provider mode serves the login inline (this build exposes **no** `forward_auth` endpoint) and forwards authenticated traffic to `http://:8017`, where the request is routed straight to its app (`dankclaw/Caddyfile:14-24,283-423`). The exempt-by-construction set never enters that chain: `auth.` (the IdP itself), `ai.` and `gitea.` (own logins), `hermes.` and `hermesadmin.` (app-enforced passwords), `pihole.` (the basicauth edge above), and the media/*arr vhosts. Grafana left basicauth on 2026-09-01 and is an OAuth2 relying party against Authentik (`dankclaw/Caddyfile:122-132`). See [[Homepage]] and [[Authentik]].
Everything else authenticates, not basicauth. The guarded set — `www.` (homepage), `search.`, `grafana.`, `prometheus.`, `loki.`, `seerr.`, `marreta.`, `romm.`, `files.`, `notebook.`, `abs.` — proxies **all** traffic to `anubis:8923`, which verifies the client and forwards it back to Caddy's internal listener `http://:8016`; Authentik's proxy-provider mode serves the login inline (this build exposes **no** `forward_auth` endpoint) and forwards authenticated traffic to `http://:8017`, where the request is routed straight to its app (`dankclaw/Caddyfile`: the header comment and the two internal listeners). The exempt-by-construction set never enters that chain: `auth.` (the IdP itself), `ai.` and `gitea.` (own logins), `hermes.` and `hermesadmin.` (app-enforced passwords), `pihole.` (the basicauth edge above), and the media/*arr vhosts. Grafana left basicauth on 2026-09-01 and is an OAuth2 relying party against Authentik (`dankclaw/Caddyfile:122-132`). See [[Homepage]] and [[Authentik]].
## Guard a new vhost
Order is the whole procedure: the IdP row has to exist **before** the route flips, or
authenticated traffic reaches the outpost with no matching provider and the user gets
authentik's error page instead of the app.
1. **Provider first.** A proxy provider plus an application, claimed by the embedded
outpost. No blueprint owns this state — it lives only in the IdP's database, which is
why it is easy to forget:
```
ssh dankclaw 'podman exec -i authentik-server ak shell' <<'PY'
from authentik.providers.proxy.models import ProxyProvider
from authentik.core.models import Application
from authentik.outposts.models import Outpost
NAME, EXT = "audiobooks", "https://audiobooks.franksdank.cloud/"
pp = ProxyProvider.objects.filter(name=f"{NAME}-proxy").first()
if pp is None:
pp = ProxyProvider.objects.create(name=f"{NAME}-proxy", external_host=EXT,
internal_host="http://caddy:8017")
if not Application.objects.filter(provider=pp).exists():
Application.objects.create(name=f"{NAME.title()} Guard", slug=f"{NAME}-guard",
provider=pp)
o = Outpost.objects.get(managed="goauthentik.io/outposts/embedded")
o.providers.add(pp) # an unclaimed provider is invisible to the outpost
print(pp.name, pp.external_host, o.providers.count())
PY
```
`external_host` is the exact vhost URL **with the trailing slash**: the embedded
outpost routes on the ORIGINAL `Host` header alone, so anything else matches zero
providers. Never print a provider's fields wholesale — a proxy provider carries a
client secret, and it lands in whatever transcript the command runs in.
`ProxyProvider.objects.create()` is not the API: the serializer's
`set_oauth_defaults()` never runs, so the row lands with `redirect_uris` and
`grant_types` empty, and the guard fails *before* any credential — authentik's
"Redirect URI Error" page, then a bodiless HTTP 400 once the URIs are filled
by hand. Run `scripts/authentik-proxy-provider-defaults.sh` after the snippet
(idempotent, covers every provider; `--check` is the census); law and failure
signatures in [[Authentik Outpost Reference]].
2. **Then the route.** The public site block proxies the whole request to `anubis:8923`
with `header_up X-Real-Ip {remote_host}`, and the post-auth `http://:8017` listener
gains a `@<name> host <vhost>` handler pointing at the app — copy `romm`/`files`,
which are the same shape including a Dankserver LAN-port upstream.
An app that derives its own origin from the request needs the scheme pinned:
Audiobookshelf checks every OIDC callback against `x-forwarded-proto://Host`; this
listener is fed over plain http so Caddy's own `X-Forwarded-Proto` is `http`, and the
app answers a 400 "Invalid callback URL - must be same-origin" for its own https
callback. The edge for a guarded name is TLS-only, so `header_up X-Forwarded-Proto
https` is the truthful value ([[Audiobookshelf]]).
3. **Then the name and the audit.** A `dankclaw/pihole/addn-hosts` line (the `pihole`
seam fails the build if a Caddyfile vhost has no record), a Porkbun A record if the
name must resolve off-LAN ([[LAN-DNS]]), and the subdomain added to `GUARDED` in
`tests/e2e/specs/14-guard.spec.ts` with an app-specific leak marker — that spec is
the deploy gate's proof that the guard, not the app, answers an anonymous client.
## Operations
+2 -1
View File
@@ -44,7 +44,7 @@ is a locator: one row, one clause, no numbers.
| Page | Owns |
|---|---|
| [[Audiobookshelf]] | Audiobookshelf on Dankserver: quadlet shape, the read-only array bind, state on the SSD, first-run account, laws. |
| [[Audiobookshelf]] | Audiobookshelf on Dankserver: quadlet shape, array bind, SSD state, API bootstrap, authentik SSO, guarded vhost, laws. |
| [[Calibre]] | Calibre is the e-book library manager that backs Calibre-Web. |
| [[Calibre-Web]] | Calibre-Web is a web application that browses, reads, and downloads eBooks from an existing Calibre library database. |
| [[Copyparty]] | Copyparty is a lightweight file server and media streamer that runs as a single Python binary. |
@@ -57,6 +57,7 @@ is a locator: one row, one clause, no numbers.
| [[Media Metadata View]] | The SSD-backed overlay the two media servers read and write: shape, guards, commands, laws, rollback. |
| [[Media Music Relayout]] | Re-filing /mnt/entropy/Music by Jellyfin's tag rows: the census, the album key, the refusals. |
| [[Media Relayout]] | Census, plan and apply the movie, TV, music and audiobook libraries: the gates, the refusals, the rollback. |
| [[Media TV Relayout]] | Re-file TVShows from scene packs into the shape Jellyfin reads: authority order, refusals, the two-series split test. |
| [[Plex]] | Plex Media Server is a **containerized media server on Dankserver**: a podman |
| [[redis]] | The Redis cache and task queue for RomM: quadlet unit, network alias topology, and the backup path. |
| [[RomM]] | RomM (ROM Manager) is a self-hosted ROM library manager and player. |
+1 -1
View File
@@ -516,7 +516,7 @@
"category": "admin",
"section": "Media Library Apps",
"description": "Audiobook server for the entropy library: its own SQLite library, its own login, per-listener positions.",
"url": "http://192.168.0.235:8000",
"url": "http://audiobooks.franksdank.cloud/audiobookshelf/",
"docs": "",
"probe": "192.168.0.235:8000",
"method": "http",
+276
View File
@@ -0,0 +1,276 @@
#!/usr/bin/env bats
# Seam: scripts/audiobookshelf-init.sh — the Audiobookshelf bootstrapper.
#
# A freshly deployed Audiobookshelf answers /healthcheck 200 while it has no
# user, no library and no scan, so "healthy" proves nothing about usefulness.
# This seam pins what the bootstrapper must make true, against a stub that
# speaks the same API as the pinned image (POST /init + POST /login at the
# root, bearer-authenticated /api/*, library count only on /items):
# 1. an unregistered server gains the account, the library and a scan, and the
# credential lands mode 600 with both keys;
# 2. a second run creates nothing new (exactly one /init, one library) and
# still fires the incremental scan;
# 3. neither the bearer token nor the password ever rides a URL — a secret in
# a URL or an argv is world-readable in /proc/<pid>/cmdline, which is how
# an HF token leaked into a transcript on 2026-09-16;
# 4. a server that cannot be reached is refused by URL with exit 1, never
# reported as success (a green deploy over a dead API is a lie);
# 5. ABS_SCAN outside 0/1 is a usage error raised before anything is touched;
# 6. the deployer runs the bootstrapper only after the health gate answered.
# 7. the SSO arm sends the base path: ABS builds its redirect_uri as
# <origin><authOpenIDSubfolderForRedirectURLs>/auth/openid/callback, and the
# value must be the path the app is mounted under — never the literal
# "undefined" a fresh database leaves there.
load helpers
setup() {
INIT="$REPO_ROOT/scripts/audiobookshelf-init.sh"
TMP="$(mktemp -d)"
PORT=18937
cat > "$TMP/stub.py" <<'PY'
import json, sys
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
port, logf = int(sys.argv[1]), sys.argv[2]
state = {"init": False, "user": None, "pw": None, "libs": [], "oidc": None}
def log(line):
with open(logf, "a") as f:
f.write(line + "\n")
def norm(p):
# ABS mounts every route under ROUTER_BASE_PATH. The stub accepts the API at the
# root or under that prefix, so a test can still assert which one the client used.
return p[len("/audiobookshelf"):] if p.startswith("/audiobookshelf") else p
class H(BaseHTTPRequestHandler):
def log_message(self, *a):
pass
def send(self, code, body, as_json=False):
data = json.dumps(body).encode() if as_json else str(body).encode()
self.send_response(code)
self.send_header("Content-Type", "application/json" if as_json else "text/plain")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def body(self):
n = int(self.headers.get("Content-Length") or 0)
return json.loads(self.rfile.read(n) or b"{}")
def authed(self):
return self.headers.get("Authorization") == "Bearer stub-token"
def do_GET(self):
path = norm(self.path.split("?")[0])
log("GET %s auth=%s" % (self.path, "yes" if self.headers.get("Authorization") else "no"))
if path == "/status":
methods = ["local", "openid"] if state["oidc"] else ["local"]
return self.send(200, {"app": "audiobookshelf", "isInit": state["init"],
"authMethods": methods}, True)
if not self.authed():
return self.send(401, "Unauthorized")
if path == "/api/libraries":
return self.send(200, {"libraries": state["libs"]}, True)
if path == "/api/auth-settings":
return self.send(200, state["oidc"] or {}, True)
if path.endswith("/items"):
return self.send(200, {"total": 42, "results": []}, True)
return self.send(404, "nope")
def do_POST(self):
path = norm(self.path.split("?")[0])
payload = self.body()
log("POST %s auth=%s" % (self.path, "yes" if self.headers.get("Authorization") else "no"))
if path == "/init":
if state["init"]:
return self.send(500, "already initialized")
root = payload.get("newRoot", {})
state["init"] = True
state["user"] = root.get("username")
state["pw"] = root.get("password")
log("INIT %s" % root.get("username"))
return self.send(200, "OK")
if path == "/login":
if not state["init"]:
return self.send(401, "Unauthorized")
if (payload.get("username"), payload.get("password")) != (state["user"], state["pw"]):
return self.send(401, "Unauthorized")
return self.send(200, {"user": {"username": state["user"], "type": "root",
"id": "u1", "token": "stub-token"}}, True)
if not self.authed():
return self.send(401, "Unauthorized")
if path == "/api/libraries":
lib = {"id": "lib-1", "name": payload.get("name"), "folders": payload.get("folders", [])}
state["libs"].append(lib)
log("LIBRARY %s %s" % (lib["name"], lib["folders"]))
return self.send(200, lib, True)
if path.endswith("/scan"):
log("SCAN")
return self.send(200, "OK")
return self.send(404, "nope")
def do_PATCH(self):
path = norm(self.path.split("?")[0])
payload = self.body()
log("PATCH %s auth=%s" % (self.path, "yes" if self.headers.get("Authorization") else "no"))
if not self.authed():
return self.send(401, "Unauthorized")
if path == "/api/auth-settings":
# Stored and echoed verbatim: the test asserts the wire, not the stub's
# opinion about which keys are patchable.
state["oidc"] = payload
for k in sorted(payload):
log("OIDCFG %s=%r" % (k, payload[k]))
return self.send(200, payload, True)
return self.send(404, "nope")
ThreadingHTTPServer(("127.0.0.1", port), H).serve_forever()
PY
python3 "$TMP/stub.py" "$PORT" "$TMP/requests.log" > "$TMP/stub.out" 2>&1 &
STUB_PID=$!
for _ in $(seq 1 50); do
curl -s -o /dev/null "http://127.0.0.1:$PORT/status" && break
sleep 0.1
done
}
teardown() {
[ -n "${STUB_PID:-}" ] && kill "$STUB_PID" 2>/dev/null || true
rm -rf "$TMP"
}
boot() {
env ABS_URL="http://127.0.0.1:$PORT" ABS_SECRET="$TMP/cred" \
ABS_ADMIN_USER=frank ABS_LIBRARY=Audiobooks ABS_FOLDER=/audiobooks \
ABS_SCAN="${SCAN:-1}" bash "$INIT"
}
@test "an unregistered server gains the account, the library and the scan" {
run boot
[ "$status" -eq 0 ]
printf '%s\n' "${lines[@]}" | grep -qx "credential: created $TMP/cred"
printf '%s\n' "${lines[@]}" | grep -q "^account: created (frank)"
printf '%s\n' "${lines[@]}" | grep -q "^token: obtained (root)"
printf '%s\n' "${lines[@]}" | grep -q "^library: created Audiobooks (lib-1)"
printf '%s\n' "${lines[@]}" | grep -q "^scan: accepted"
printf '%s\n' "${lines[@]}" | grep -q "^items: 42"
# The state roots of this service hold an account credential, so the file
# that carries it must never be group- or world-readable.
[ "$(stat -c %a "$TMP/cred")" = "600" ]
grep -qx 'ABS_ADMIN_USER=frank' "$TMP/cred"
grep -Eq '^ABS_ADMIN_PASSWORD=[0-9a-f]{24}$' "$TMP/cred"
# The folder the library indexes is the bind inside the container, not the
# host path — a library pointed at /mnt/entropy would answer 0 items and
# look like an empty audiobook collection.
grep -q "LIBRARY Audiobooks \[{'fullPath': '/audiobooks'}\]" "$TMP/requests.log"
}
@test "a second run creates nothing new and still fires the incremental scan" {
boot > /dev/null
[ -e "$TMP/requests.log" ]
run boot
[ "$status" -eq 0 ]
printf '%s\n' "${lines[@]}" | grep -q "^account: already registered"
printf '%s\n' "${lines[@]}" | grep -q "^library: exists Audiobooks (lib-1)"
printf '%s\n' "${lines[@]}" | grep -q "^scan: accepted"
# No second credential, no second account, no duplicate library: the deployer
# may run this on every re-deploy of the service.
! printf '%s\n' "${lines[@]}" | grep -q "credential: created"
[ "$(grep -c '^INIT ' "$TMP/requests.log")" -eq 1 ]
[ "$(grep -c '^LIBRARY ' "$TMP/requests.log")" -eq 1 ]
[ "$(grep -c '^SCAN$' "$TMP/requests.log")" -eq 2 ]
}
@test "the bearer token and the password never ride a URL" {
boot
# The stub logs the request line (method + path). A secret showing up there
# means it travelled as a query parameter — visible in access logs, shell
# history and /proc/<pid>/cmdline.
! awk '{print $2}' "$TMP/requests.log" | grep -q 'stub-token'
pw=$(awk -F= '/^ABS_ADMIN_PASSWORD=/{print $2}' "$TMP/cred")
[ -n "$pw" ]
! awk '{print $2}' "$TMP/requests.log" | grep -q -- "$pw"
# and the authenticated routes only answered once the header was present
grep -q '^GET /api/libraries auth=yes$' "$TMP/requests.log"
}
@test "a server that does not answer is refused by URL, not reported as success" {
run env ABS_URL="http://127.0.0.1:1" ABS_SECRET="$TMP/cred" bash "$INIT"
[ "$status" -eq 1 ]
[[ "$output" == *refused:* ]]
[[ "$output" == *"127.0.0.1:1"* ]]
}
@test "ABS_SCAN outside 0/1 is a usage error raised before anything is touched" {
run env ABS_URL="http://127.0.0.1:$PORT" ABS_SECRET="$TMP/cred" ABS_SCAN=2 \
bash "$INIT"
[ "$status" -eq 2 ]
[[ "$output" == *ABS_SCAN* ]]
[ ! -e "$TMP/cred" ]
}
@test "the deployer bootstraps only after the health gate has answered" {
body=$(awk '/^deploy_audiobookshelf\(\)/,/^}/' "$REPO_ROOT/scripts/deploy-remote.sh")
[ -n "$body" ]
printf '%s\n' "$body" | grep -q \
'wait_for_health audiobookshelf http://127\.0\.0\.1:8000/healthcheck'
printf '%s\n' "$body" | grep -q 'scripts/audiobookshelf-init\.sh'
printf '%s\n' "$body" | awk '
/wait_for_health audiobookshelf/ { health = NR }
/audiobookshelf-init\.sh/ { init = NR }
END { exit !(health && init && init > health) }'
}
@test "the SSO arm sends the base path as the redirect subfolder" {
# What authentik has registered must equal what ABS builds:
# <origin><authOpenIDSubfolderForRedirectURLs>/auth/openid/callback. A database
# created before SSO leaves that key unset and ABS then sends the literal
# "/undefined/auth/openid/callback" — measured 2026-11-02, where the live login died
# on authentik's "Redirect URI Error" page. So the value has to reach the wire, and
# it has to be the path the app is served under.
umask 077
cat > "$TMP/cred" <<'CRED'
ABS_ADMIN_USER=frank
ABS_ADMIN_PASSWORD=0123456789abcdef0123456789
ABS_OIDC_CLIENT_SECRET=0123456789abcdef0123456789abcdef0123456789abcdef0123456789
CRED
run env ABS_URL="http://127.0.0.1:$PORT/audiobookshelf" ABS_SECRET="$TMP/cred" \
ABS_SCAN=0 bash "$INIT"
[ "$status" -eq 0 ]
printf '%s\n' "${lines[@]}" | grep -q "^oidc: armed"
grep -q "^OIDCFG authOpenIDSubfolderForRedirectURLs='/audiobookshelf'$" "$TMP/requests.log"
# every route the bootstrapper calls rides under the same prefix it armed with
grep -q "^POST /audiobookshelf/init auth=no$" "$TMP/requests.log"
grep -q "^GET /audiobookshelf/api/auth-settings auth=yes$" "$TMP/requests.log"
# A root-mounted deployment sends the EMPTY subfolder (the value the 2.17.4 upgrade
# uses), not whatever the last URL happened to carry.
run env ABS_URL="http://127.0.0.1:$PORT" ABS_SECRET="$TMP/cred" ABS_SCAN=0 bash "$INIT"
[ "$status" -eq 0 ]
[ "$(grep "^OIDCFG authOpenIDSubfolderForRedirectURLs=" "$TMP/requests.log" | tail -1)" \
= "OIDCFG authOpenIDSubfolderForRedirectURLs=''" ]
}
@test "the app and the IdP provider script build the same redirect path" {
# The provider row is written by scripts/authentik-audiobookshelf-sso.sh from
# ABS_BASE; the app builds its redirect_uri from the base path the deployer hands
# audiobookshelf-init.sh, which comes from the unit. Drift between the two is a
# redirect_uri mismatch — the login dies at the IdP before any credential is issued.
unit="$REPO_ROOT/dankserver/audiobookshelf/quadlet/audiobookshelf.container"
mounted=$(awk -F= '/^Environment=ROUTER_BASE_PATH=/{print $3}' "$unit")
[ -n "$mounted" ]
provider=$(grep -m1 '^BASE=' "$REPO_ROOT/scripts/authentik-audiobookshelf-sso.sh" \
| grep -oE 'https?://[^"}]+')
bootstrapper=$(grep -m1 '^ABS_URL=' "$INIT" | grep -oE 'https?://[^"}]+')
# Origins legitimately differ (LAN address vs loopback); the PATH must not.
[ "${provider#http://192.168.0.235:8000}/auth/openid/callback" \
= "${bootstrapper#http://127.0.0.1:8000}/auth/openid/callback" ]
[ "${bootstrapper#http://127.0.0.1:8000}" = "$mounted" ]
}
+6 -1
View File
@@ -182,8 +182,13 @@ load helpers
# under base path /audiobookshelf and the deploy gate timed out beside a unit
# that was active (running). PORT and ROUTER_BASE_PATH are what the image's own
# ENV block and the server source actually use.
# The base path is not a free choice either: /app/client/dist/index.html inside the
# pinned image carries routerBasePath:"/audiobookshelf", so a server mounted at / hands
# out a page whose every script request falls through to the SPA fallback and comes back
# text/html — the UI never leaves "Loading…" while the API stays healthy (measured
# 2026-11-02, which is exactly how it hid behind an API-only verification).
unit="$REPO_ROOT/dankserver/audiobookshelf/quadlet/audiobookshelf.container"
grep -q '^Environment=PORT=8000$' "$unit" || return 1
grep -q '^Environment=ROUTER_BASE_PATH=/$' "$unit" || return 1
grep -q '^Environment=ROUTER_BASE_PATH=/audiobookshelf$' "$unit" || return 1
! grep -qE '^Environment=ABS_' "$unit" || return 1
}
+4 -1
View File
@@ -42,7 +42,10 @@ export default defineConfig({
},
{
name: "admin",
testMatch: /11-.*\.spec\.ts/,
// 23- joins 2026-11-02 (Audiobookshelf SSO): it needs BOTH halves of the admin
// fixture — a live authentik session to ride the OIDC round trip, and the
// qa-e2e-admin group membership that grants site admin in the app.
testMatch: /(11|23)-.*\.spec\.ts/,
use: { storageState: ".auth/qa-e2e-admin.json" },
dependencies: ["setup"],
},
+6 -2
View File
@@ -15,9 +15,12 @@ const BASE = "https://www.franksdank.cloud";
// Every vhost that Caddy routes through anubis:8923 (see dankclaw/Caddyfile).
// Keep this list in lockstep with the Caddyfile: a new guarded vhost must be
// added here or the guard-chain audit is incomplete.
const GUARDED = ["www", "search", "romm", "files", "seerr", "grafana", "prometheus", "loki", "notebook"];
const GUARDED = ["www", "search", "romm", "files", "seerr", "grafana", "prometheus", "loki", "notebook", "audiobooks"];
// Markers that prove app content leaked past the guard to an anonymous user.
// Markers that prove app content leaked past the guard to an anonymous user. Each
// marker must be a string the APP renders that the gate itself could never print —
// for audiobooks that is the Audiobookshelf login's own SSO button label (the app's
// door), which authentik's gate never renders because it IS the IdP.
const LEAK_MARKERS: Record<string, string[]> = {
"search.franksdank.cloud": ["SearXNG preferences"],
"romm.franksdank.cloud": ["RomM"],
@@ -26,6 +29,7 @@ const LEAK_MARKERS: Record<string, string[]> = {
"grafana.franksdank.cloud": ["Dashboards"],
"prometheus.franksdank.cloud": ["Expression Browser", "Graph"],
"notebook.franksdank.cloud": ["Open Notebook"],
"audiobooks.franksdank.cloud": ["Sign in with Authentik"],
};
test("homepage root (/) is guarded for unauthenticated users", async ({ page }) => {
@@ -0,0 +1,92 @@
import { expect, test } from "@playwright/test";
// Audiobookshelf: the SSO round trip must land a session with a real permission level.
//
// The service answers on a LAN address with no DNS name and no Caddy vhost, so it is in no
// guarded-vhost list and no other spec ever fetched a byte of it — every deploy gate passed
// while its login was dead. Two defect classes, both caught live on 2026-11-02:
//
// 1. The bundle must be BUILT for /audiobookshelf (ROUTER_BASE_PATH in the quadlet). Built
// for "/" the container still answers /healthcheck and still serves index.html, so the
// deploy health gate is green while the browser sees a blank page: every <script>
// resolves against the root, misses the static allowlist, and falls through to the SPA
// catch-all that returns index.html as text/html. Hence the MIME assertion below — the
// one thing that distinguishes a booted SPA from a served HTML stub.
// 2. The authentik provider row must carry grant_types AND the built-in OpenID scope
// mappings (provisioned by scripts/authentik-audiobookshelf-sso.sh). A blueprint
// re-import reverts hand-made provider rows silently, and the wire errors are generic:
// oauthlib's "The request is otherwise malformed" for an empty grant list, then
// "insufficient_scope" from ABS's /userinfo call for a short scope. Only the level
// assertion below — that the abs_groups claim granted site admin — sees either.
const HOST = "192.168.0.235:8000";
const ABS = `http://${HOST}/audiobookshelf`;
test("SSO round trip boots the SPA and grants a real Audiobookshelf session", async ({
page,
}) => {
test.setTimeout(180_000);
const scripts: { url: string; type: string }[] = [];
page.on("response", (res) => {
const u = new URL(res.url());
if (u.host === HOST && u.pathname.endsWith(".js")) {
scripts.push({ url: u.pathname, type: res.headers()["content-type"] ?? "" });
}
});
await page.goto(`${ABS}/login`, { waitUntil: "domcontentloaded" });
const sso = page.getByText("Sign in with Authentik");
await expect(sso).toBeVisible();
await sso.click();
// The admin project's storageState carries a live authentik session, so this is a silent
// round trip: authorize -> code -> callback -> token. Still on /login means the IdP
// refused the authorize request or ABS rejected the code exchange.
await expect
.poll(() => !new URL(page.url()).pathname.endsWith("/login"), { timeout: 90_000 })
.toBe(true);
await expect
.poll(
async () => {
const u = new URL(page.url());
return u.host === HOST && u.pathname.startsWith("/audiobookshelf");
},
{ timeout: 30_000 },
)
.toBe(true);
// Defect 1 gate: nothing the app loaded may be HTML dressed as a script. The render proof
// is the SSO button above — it only exists once the bundle has executed, so no DOM-shape
// guess is needed (the shell's own mount markup says nothing about whether it booted).
expect(scripts.length, "the SPA loaded no JavaScript").toBeGreaterThan(2);
for (const s of scripts) {
expect(s.type, `${s.url} arrived as ${s.type}`).toContain("javascript");
}
// The token the app landed on is the credential its own API must accept. Give the SPA a
// moment to finish the exchange: the URL leaves /login the instant authentik bounces back
// through the callback route, which is BEFORE the app has stored what it exchanged.
const grabToken = () =>
page.evaluate(() => {
for (let i = 0; i < localStorage.length; i++) {
const v = localStorage.getItem(localStorage.key(i) as string) || "";
if (/^[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+\.[A-Za-z0-9_-]+$/.test(v)) return v;
}
const c = document.cookie.match(/(?:^|;\s*)abs_token=([^;]+)/);
return c ? decodeURIComponent(c[1]) : "";
});
await expect.poll(grabToken, { timeout: 60_000 }).not.toBe("");
const token = await grabToken();
const me = await page.request.get(`${ABS}/api/me`, {
headers: { Authorization: `Bearer ${token}` },
});
expect(me.status()).toBe(200);
const user = await me.json();
expect(user.username).toBe("qa-e2e-admin");
// Site role, derived ONLY from the abs_groups claim: it says "admin" if and only if the
// scope mapping reached ABS with audiobookshelf_admins in the list. A short token scope
// (missing OpenID mappings) never gets this far — the exchange dies before it.
expect(user.type).toBe("admin");
});
+33
View File
@@ -460,6 +460,39 @@ print(sum(1 for l in open(sys.argv[1]) if json.loads(l).get("op") == "move"))' "
[ -e "$LIB/Alien (1979)/Alien.1979.1080p.BluRay.x265.10bit.mkv" ]
}
# A hand-written plan row whose verb is not in the executor's vocabulary used
# to be dropped in silence: read_plan's if/elif chain had no else, so a plan of
# nothing but typos applied as a successful no-op (hit for real on 2026-11-02,
# when a pass meant to rename two series folders reported "applied: 0 moved"
# and exit 0). An unread row is a plan bug, and a plan bug refuses.
@test "a plan row with a verb this tool does not know stops the whole run" {
plan
JOURNAL="$BATS_TEST_TMPDIR/journal.jsonl"
at "$LIB/Stray/Stray.S01E01.mkv" 4000
printf 'rename\t%s\t%s\tfile\n' \
"$LIB/Stray/Stray.S01E01.mkv" "$LIB/Stray/Stray - S01E01.mkv" >> "$OUT"
run python3 "$APPLY" apply --plan "$OUT" --root "$LIB" --journal "$JOURNAL" --apply
[ "$status" -ne 0 ]
[[ "$output" == *"rename"* ]]
# all-or-nothing: the rows this tool DOES understand did not run either
[ -e "$LIB/Stray/Stray.S01E01.mkv" ]
[ ! -e "$JOURNAL" ]
}
@test "a plan whose every row is an unknown verb refuses instead of reporting success" {
JOURNAL="$BATS_TEST_TMPDIR/journal.jsonl"
at "$LIB/Stray/Stray.S01E01.mkv" 4000
{ printf 'action\tsrc\tdst\tkind\n'
printf 'rename\t%s\t%s\tfile\n' \
"$LIB/Stray/Stray.S01E01.mkv" "$LIB/Stray/Stray - S01E01.mkv"
} > "$BATS_TEST_TMPDIR/typo.tsv"
run python3 "$APPLY" apply --plan "$BATS_TEST_TMPDIR/typo.tsv" --root "$LIB" --journal "$JOURNAL" --apply
[ "$status" -ne 0 ]
[[ "$output" == *"unknown action"* ]]
[[ "$output" != *"applied"* ]]
[ -e "$LIB/Stray/Stray.S01E01.mkv" ]
}
@test "a run interrupted between the journal write and the move finishes the job" {
plan
JOURNAL="$BATS_TEST_TMPDIR/journal.jsonl"