Compare commits

...
Author SHA1 Message Date
Frank b232033b11 docs(tv): split the TV arm into its own page, add the folder-identity laws
Media Relayout.md crossed the OKF 500-line ceiling with the 2026-11-02 TV
laws, and the ceiling was right: the TV arm reads a different authority and
refuses for different reasons, like the music and audiobook arms that already
have pages. The arm moves verbatim to Media TV Relayout.md and gains two laws
from the TVShows pass: a folder naming two series is not evidence of two
series (ffprobe's embedded title tag answers per file), and right numbering
with a wrong name means rename the folder only. The executor page keeps the
executor laws and gains the hold-outside---root/--force law; Jellyfin.md gains
the settle read (ScheduledTasks State, since /Library/ScanStatus is a 404 on
12.4). Index regenerated with gen-vault-index.py.

cwd: /home/blank/.omp/wt/wt-20261030-163847-6d1fc1f · session: 01a18f2e-18b1-7573-bcd5-ac99636f8bcb
2026-11-02 09:06:22 -08:00
dank 4a33898a6c Merge pull request 'fix(jellyfin): anchor the provider-key write on the file’s real root' (#121) from chore/jellyfin-tvdb-key into main 2026-11-02 07:17:25 -08:00
dank 21c98bd485 Merge pull request 'docs(jellyfin): the door that scans is /Library/Refresh, and the art door names its field' (#120) from docs/jellyfin-scan-door into main 2026-11-02 07:16:52 -08:00
dank a651c81dfb Merge pull request 'feat(audiobookshelf): bootstrap the account, library and scan from the deployer' (#119) from feat/audiobookshelf-bootstrap into main 2026-11-02 07:16:45 -08:00
Frank e9314ea1ee fix(jellyfin): anchor the provider-key write on the file's real root
TheTVDB writes its own configuration file and its serializer omits a
null-valued setting entirely, so a fresh install has no <ApiKey> element to
fill, under a root element called <PluginConfiguration>. xml_secret's
fallback insertion was hardcoded to </Configuration>, matched nothing, and
install refused with "no place to write <ApiKey>" — measured on Dankserver
2026-11-02, where the TVDB key has been present in the secret file since
October and has never been installable.

The anchor now comes from the document's own root, so both shapes Jellyfin
actually produces work (the core plugins' <Configuration> and a plugin's
<PluginConfiguration>), and the refusal is kept for a file with no root at
all. Tests: the plugin-written shape is red before this commit and green
after; every other setting in the file is asserted preserved.
2026-11-02 04:38:40 -08:00
Frank ad9563412d docs(audiobookshelf): what a residual is made of
Four laws from finishing the un-shadow, correcting one:

- A book whose files ALL fail to probe never becomes an item, so a disk-vs-DB
  folder gap beside `Invalid data` lines is corrupt media, not a lost book. A
  format-only ffprobe clears files the scanner rejects — do not trust it.
- `Disc 01`/`CD 1` subfolders are parts of one item (110 of them here, under a
  192-track book); the extension census counts them, ABS does not.
- `.cbr`/`.cbz` group like media but `parseComicMetadata` throws `ERAR_BAD_ARCHIVE`,
  so those items land media-less: the file, not the scanner.
- Scan completion is a NEW scan uuid; `podman logs --since` is a sliding window
  and ages its own evidence out.
- `?force=1` is now recorded as neither discovery nor repair: a forced full
  re-scan left item count, path set and media-less count identical.

Verification names the five book folders that need re-acquiring.
2026-11-02 03:59:14 -08:00
Frank f71c7ccce4 docs(audiobookshelf): the grouping rule, not "shallowest folder with audio"
Corrects the law pushed earlier today. The mechanism is
`groupFileItemsIntoLibraryItemDirs` (utils/scandir.js), called on a live
`recurseFiles` walk of the tree EVERY scan: each file attaches to the FIRST
directory along its path that already has a group, so one media file at a
collection root owns everything beneath it. Measured: one 949 MB `.cbr` hid
652 book folders behind a single 5,693-track item.

- "Media file" is audio OR ebook (`audiobooksOnly=false` -> SupportedAudioTypes
  ∪ SupportedEbookTypes, 17 audio + 6 ebook extensions). An audio-only census
  cannot see the file that hides an Audiobooks library.
- Deleting the covering item does not fix a shadow: grouping is recomputed from
  disk and the next scan rebuilds the item (measured: `1 Added`, same path).
- Un-shadow by MOVE: one file per folder is ABS's own single-book layout.
- `?force=1` is not a discovery arm (both arms walk the disk; force re-scans
  every existing item, 1:33 against 0:04).
- Paging corrected: `offset` is ignored outright, `page` is the parameter.
- `DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()` of the whole
  swallowed subtree. Never pass `hard`.

Shape table gains the layout row and the move journal; the stale "1,103
shelves" scan count is destroyed.
2026-11-02 03:43:34 -08:00
Frank f541566714 docs(audiobookshelf): a book is the shallowest audio folder, and it owns the subtree
Measured on the live library: 6 stray files at the root of _unsorted made the
whole collection dump ONE item of 5,693 tracks, with 542 real books folded into
it; no scan subdivides an existing item. Also pinned: the items endpoint hands
back ceil(total/limit)*limit rows because an out-of-range offset repeats the
tail (that is where "exactly 1,500" and "exactly 4,000" came from), and
DELETE /api/items/:id?hard=1 runs fs.remove() on the item's folder - pointed at
a covering item that is the whole subtree, so un-swallowing is a DB-only delete
after the stray root files are filed.
2026-11-02 03:34:28 -08:00
Frank 006bbcef28 docs(audiobookshelf): paging the items endpoint never terminates itself
Two censuses this session reported exactly 1,500 and exactly 4,000 books for one
library mid-scan. Cause: an out-of-range offset returns the last item instead of an
empty page, so a loop written "until the page is empty" manufactures its own
total, and total itself moves while a scan is running. Stop on total; count after
the pass settles.
2026-11-02 03:18:04 -08:00
Frank 56dfc6d0b9 docs(audiobookshelf): the scanner refusal names the file, not the bind
The first full scan logged "Invalid data found when processing input" per file and
carried on. That line is about the file: the scanner skips it, so the book never
appears in the library at all. Census of the 43 refused files: 38 were 0 bytes, 3
held WAV bytes behind a .mp3 name, 2 were unreadable. The check to run first is a
byte count, not a server investigation.

Full suite: 818 tests, exit 0.
2026-11-02 02:57:32 -08:00
Frank 48ef6d34cd docs(audiobookshelf): the first run is a script, and its re-run is the scan trigger
The page claimed the account can only be created from a LAN address; measured,
the door is POST /init and it answers plain text. Replaced with the bootstrap the
deployer now runs, and with the doors a consumer actually needs: /status reports
isInit with no /api prefix, the item count rides /api/libraries/{id}/items ->
total (the library DTO has no totals), and the list DTO carries media.numTracks
while /api/items/{id} returns media.audioFiles.

New law: the folder watcher cannot cross the rootless podman bind — the same gap
as Jellyfin's auto-scan note — so files added on the host stay invisible until a
scan runs, and the deployer's re-run of the bootstrapper is the cheap trigger.

Testing Policy lists the new seam; index.md regenerated from frontmatter.
2026-11-02 02:55:55 -08:00
Frank 317f36b456 feat(audiobookshelf): bootstrap the account, library and scan from the deployer
A freshly deployed Audiobookshelf answers /healthcheck and is otherwise dead: no
account, no library, no scan. The UI registration step is gone by design — the
doors are POST /init (200 with plain text "OK", not JSON) and /status (isInit),
and there is no /setup route. That is an API-only bootstrap, so it must be code.

scripts/audiobookshelf-init.sh is idempotent: /status -> /init when unregistered,
POST /login for the bearer, POST /api/libraries for the Audiobooks library on
/audiobooks, then POST /api/libraries/{id}/scan. The deployer runs it only after
the health gate has answered.

Credentials live in ~/.config/secrets/audiobookshelf (mode 600): the bearer rides
an Authorization header and the password only a POST body — never a URL or argv,
where any ps on the host would print them. The seam drives an inline stub server
that logs every request line, and asserts: fresh bootstrap creates account,
library and scan; a second run creates nothing new and still fires the
incremental scan; no secret appears in any logged request line; a server that
refuses is reported as "refused: <url>" with exit 1 rather than success; ABS_SCAN
outside 0|1 fails before a single request; the deployer orders init after health.

Measured on Dankserver: run against the already-bootstrapped host it is a no-op
that re-fires the scan; the library then scans (1,500+ books, every item with
audio tracks, none invalid).
2026-11-02 02:55:50 -08:00
12 changed files with 791 additions and 178 deletions

No files matched your search

+188
View File
@@ -0,0 +1,188 @@
#!/usr/bin/env bash
# audiobookshelf-init.sh — make an Audiobookshelf server usable, from nothing.
#
# Why this is a script and not a browser session (live 2026-11-02, Dankserver):
# a freshly deployed Audiobookshelf answers /healthcheck with 200 while it has no
# user, no library and no scan — "healthy" says nothing about usefulness, and a
# service that only works after someone remembers to click through a first-run
# wizard is not deployed, it is installed. Everything below is the API the React
# client itself calls, so the same procedure runs unattended and lands in git.
#
# The API facts this script depends on, read off the pinned image rather than
# guessed (dist-server/server/Server.js inside the digest this repo pins):
# * the creation route is POST /init with {"newRoot":{"username","password"}}.
# There is NO /setup route in 2.37.x — POST /setup answers "Unauthorized",
# which reads exactly like an auth fault and is instead a missing route.
# * /status (no /api prefix) reports {"isInit": <has a root user>}; every route
# under /api needs the bearer token that POST /login returns.
# * the library count is NOT a field on the library DTO; the door is
# GET /api/libraries/{id}/items -> {"total": N}.
#
# The credential is generated once into a mode-600 host file and never enters
# the repo, the URL, or a process argument list. Same pattern as calibre-server.
#
# usage: audiobookshelf-init.sh # create what is missing, then scan
# ABS_SCAN=0 audiobookshelf-init.sh # bootstrap without a scan
#
# environment (defaults are the deployed shape):
# ABS_URL server base URL (http://127.0.0.1:8000)
# ABS_SECRET credential file (/home/dank/.config/secrets/audiobookshelf)
# ABS_LIBRARY library name to ensure (Audiobooks)
# ABS_FOLDER folder to bind inside the server (/audiobooks)
# ABS_SCAN 1 triggers a scan (default), 0 skips it
#
# exit 0 every step is satisfied (created or already present)
# exit 1 a step was refused or the server misbehaved (named on stderr)
# exit 2 usage / unreadable credential file
set -euo pipefail
ABS_URL=${ABS_URL:-http://127.0.0.1:8000}
ABS_SECRET=${ABS_SECRET:-/home/dank/.config/secrets/audiobookshelf}
ABS_LIBRARY=${ABS_LIBRARY:-Audiobooks}
ABS_FOLDER=${ABS_FOLDER:-/audiobooks}
ABS_SCAN=${ABS_SCAN:-1}
die() { printf 'refused: %s\n' "$*" >&2; exit 1; }
case "$ABS_SCAN" in
0 | 1) ;;
*) printf 'refused: ABS_SCAN wants 0 or 1, got %s\n' "$ABS_SCAN" >&2; exit 2 ;;
esac
# ---- credential -------------------------------------------------------------
umask 077
if [ ! -f "$ABS_SECRET" ]; then
mkdir -p "$(dirname "$ABS_SECRET")"
user=${ABS_ADMIN_USER:-frank}
# `tr … | head -c 24` is the obvious line and it is a trap under
# `set -o pipefail`: head exits at 24 bytes, tr takes SIGPIPE, the pipeline
# status is 141 and set -e kills the script before the credential is ever
# written (measured 2026-11-02). od reads a FIXED byte count and exits on its
# own, so no consumer closes a pipe early; bash slices the length.
pass=$(od -An -tx1 -N 32 /dev/urandom | tr -d ' \n')
pass=${pass:0:24}
[ "${#pass}" -eq 24 ] || die "could not generate a password (no /dev/urandom)"
printf 'ABS_ADMIN_USER=%s\nABS_ADMIN_PASSWORD=%s\n' "$user" "$pass" > "$ABS_SECRET"
printf 'credential: created %s\n' "$ABS_SECRET"
fi
[ -r "$ABS_SECRET" ] || die "$ABS_SECRET exists but is not readable by $(id -un)"
chmod 600 "$ABS_SECRET"
# ---- everything else, over HTTP ---------------------------------------------
# python3 rather than curl so the password and the bearer token never appear in
# any argv (they would be world-readable in /proc/<pid>/cmdline otherwise).
ABS_URL="$ABS_URL" ABS_SECRET="$ABS_SECRET" ABS_LIBRARY="$ABS_LIBRARY" \
ABS_FOLDER="$ABS_FOLDER" ABS_SCAN="$ABS_SCAN" python3 - <<'PY'
import json, os, sys, urllib.error, urllib.request
BASE = os.environ["ABS_URL"].rstrip("/")
LIB = os.environ["ABS_LIBRARY"]
FOLDER = os.environ["ABS_FOLDER"]
SCAN = os.environ["ABS_SCAN"] == "1"
cred = {}
for line in open(os.environ["ABS_SECRET"]):
if "=" in line:
k, _, v = line.strip().partition("=")
cred[k] = v
user = cred.get("ABS_ADMIN_USER") or "frank"
password = cred.get("ABS_ADMIN_PASSWORD") or ""
if not password:
sys.exit("refused: %s has no ABS_ADMIN_PASSWORD" % os.environ["ABS_SECRET"])
def call(method, path, body=None, token=None, want_json=True):
data = None
headers = {"X-Requested-With": "audiobookshelf-init"}
if body is not None:
data = json.dumps(body).encode()
headers["Content-Type"] = "application/json"
if token:
headers["Authorization"] = "Bearer " + token
req = urllib.request.Request(BASE + path, data=data, headers=headers, method=method)
try:
with urllib.request.urlopen(req, timeout=60) as r:
raw = r.read()
if not raw or not want_json:
return r.status, raw
try:
return r.status, json.loads(raw)
except ValueError:
# Express answers sendStatus() with the STATUS TEXT as the body:
# POST /init and the scan route both reply "OK" (measured on the
# pinned 2.37.2 image). A successful call must never be judged by
# whether its body happens to parse as JSON.
return r.status, raw.decode("utf-8", "replace")[:200]
except urllib.error.HTTPError as e:
raw = e.read()
try:
payload = json.loads(raw)
except Exception:
payload = raw.decode("utf-8", "replace")[:200]
return e.code, payload
except urllib.error.URLError as e:
# Not an HTTP answer at all: nothing listening, DNS dead, connection
# refused. HTTPError is a URLError subclass, so this arm must come after.
return 0, "unreachable (%s)" % e.reason
status, st = call("GET", "/status")
if status != 200 or not isinstance(st, dict):
sys.exit("refused: %s/status answered %s (%s) — is the service up?" % (BASE, status, st))
if st.get("isInit"):
print("account: already registered")
else:
code, body = call("POST", "/init", {"newRoot": {"username": user, "password": password}})
if code != 200:
sys.exit("refused: POST /init answered %s (%s)" % (code, body))
print("account: created (%s)" % user)
code, login = call("POST", "/login", {"username": user, "password": password})
if code != 200 or not isinstance(login, dict) or not login.get("user", {}).get("token"):
sys.exit("refused: login answered %s (%s) — the credential file does not match the server"
% (code, login))
token = login["user"]["token"]
print("token: obtained (%s)" % login["user"].get("type", "user"))
code, libs = call("GET", "/api/libraries", token=token)
if code != 200 or not isinstance(libs, dict) or "libraries" not in libs:
sys.exit("refused: GET /api/libraries answered %s (%s)" % (code, libs))
lib = next((l for l in libs["libraries"] if l.get("name") == LIB), None)
if lib:
print("library: exists %s (%s)" % (LIB, lib["id"]))
else:
body = {
"name": LIB,
"provider": "audiobook",
"icon": "book",
"folders": [{"fullPath": FOLDER}],
"settings": {
"coverEncoding": "jpg",
"skipMatchingMediaWithAsin": False,
"skipMatchingMediaWithIsbn": False,
"autoDetectCrossoverAudiobooks": False,
"ignoreAudiobookMetadata": False,
"useAudiobookFileMetadata": False,
},
"providerMetadata": {},
}
code, made = call("POST", "/api/libraries", body, token=token)
if code != 200:
sys.exit("refused: POST /api/libraries answered %s (%s)" % (code, made))
lib = made
print("library: created %s (%s) folder %s" % (LIB, lib["id"], FOLDER))
if SCAN:
code, body = call("POST", "/api/libraries/%s/scan" % lib["id"], {}, token=token,
want_json=False)
if code != 200:
sys.exit("refused: scan answered %s (%s)" % (code, body))
print("scan: accepted")
else:
print("scan: skipped (ABS_SCAN=0)")
code, items = call("GET", "/api/libraries/%s/items?limit=1" % lib["id"], token=token)
if code == 200 and isinstance(items, dict):
print("items: %s (a fresh scan raises this from 0; watch it, not /healthcheck)"
% items.get("total"))
PY
exit 0
+6
View File
@@ -1318,6 +1318,12 @@ deploy_audiobookshelf() {
# /healthcheck answers once the HTTP server is bound; the library scan that
# follows runs in the background and must not gate the deploy.
wait_for_health audiobookshelf http://127.0.0.1:8000/healthcheck 90
# A server with no root user and no library is "healthy" and useless, so the
# bootstrapper runs after the health gate and its failure fails the deploy:
# it creates the account (credential in the mode-600 host secret file), the
# Audiobooks library over the /audiobooks bind, and fires the scan. Every
# step is idempotent, so a re-deploy costs one incremental scan.
scripts/audiobookshelf-init.sh
}
# The content server's single admin account and the desktop's basic-auth
+12 -1
View File
@@ -135,7 +135,18 @@ text = escape(val)
if pat.search(src):
out = pat.sub(lambda m: f"{m.group(1)}<{elem}>{text}</{elem}>", src, count=1)
else:
out = src.replace("</Configuration>", f" <{elem}>{text}</{elem}>\n</Configuration>", 1)
# The plugin owns this file and writes it under ITS OWN root element
# (TheTVDB's is <PluginConfiguration>), and its serializer omits a
# null-valued setting entirely — so the element being filled can be
# absent outright, and a hardcoded </Configuration> anchor then matches
# nothing. Anchor on the document's real root instead.
out = src
root = re.search(r"<([A-Za-z_][\w.:-]*)[^>]*>", src)
if root and ("</%s>" % root.group(1)) in src:
name = root.group(1)
out = src.replace(
"</%s>" % name, f" <{elem}>{text}</{elem}>\n</{name}>", 1
)
if out == src:
raise SystemExit(f"{xml_path}: no place to write <{elem}>")
+6
View File
@@ -104,6 +104,12 @@ for path in "${CHANGED[@]}"; do
dankserver/jellyfin/quadlet/*|dankserver/plex/quadlet/*)
add_test tests/media_overlay.bats
;;
# The Audiobookshelf bootstrapper (account + library + scan over the API)
# and the deployer hook that runs it after the health gate.
scripts/audiobookshelf-init.sh|tests/audiobookshelf_init.bats)
add_test tests/audiobookshelf_init.bats
add_test tests/deployment_flow.bats
;;
shared/homepage/*)
add_test tests/homepage_caddy.bats
add_test tests/homepage_config.bats
@@ -23,7 +23,7 @@ This repository tests its configuration at defined seams. The tests run with `ba
| Deployment flow | `tests/deployment_flow.bats`, `tests/deployment_scoping.bats` | scoped dispatcher exists, targets are known, service plans isolate inference, Caddy ownership is guarded, and SSH reachability is checked (skips offline) |
| Quadlet units | `tests/*quadlet*.bats` | every unit is boot-persistent, self-healing, repo-provenanced, secret-free |
| E2E (browser) | `tests/e2e/` + `tests/e2e_scaffold.bats` | Playwright drives the live guard chain; the bats seam asserts structure and credential hygiene. See [[E2E Suite]] |
| Per-service / unit seams | `tests/llm_units.bats`, `tests/frank_{archon,dragoon,zealot}_unit.bats`, `tests/llm_secrets.bats`, `tests/homepage_{caddy,checker,config,deploy,pages}.bats`, `tests/{anubis,authentik,hermes_webui,pihole,mount_entropy,searxng_limiter,autoresearch_measure,changed_tests}.bats`, `tests/{caddy,dankserver,frankbook,marreta,observability,open_webui,searxng}_quadlet.bats`, `tests/comfyui_rocm.bats` | the ~30 named seams the summary rows above do not list: LLM systemd-unit shape and env hygiene, homepage config/pages/deploy behaviour, and one detail-seam per deployed service |
| Per-service / unit seams | `tests/llm_units.bats`, `tests/frank_{archon,dragoon,zealot}_unit.bats`, `tests/llm_secrets.bats`, `tests/homepage_{caddy,checker,config,deploy,pages}.bats`, `tests/{audiobookshelf_init,anubis,authentik,hermes_webui,pihole,mount_entropy,searxng_limiter,autoresearch_measure,changed_tests}.bats`, `tests/{caddy,dankserver,frankbook,marreta,observability,open_webui,searxng}_quadlet.bats`, `tests/comfyui_rocm.bats` | the ~30 named seams the summary rows above do not list: LLM systemd-unit shape and env hygiene, homepage config/pages/deploy behaviour, and one detail-seam per deployed service |
## Focused pre-commit testing
@@ -1,6 +1,6 @@
---
type: reference
description: "Audiobookshelf on Dankserver: quadlet shape, the read-only array bind, state on the SSD, first-run account, laws."
description: "Audiobookshelf on Dankserver: quadlet shape, the read-only array bind, state on the SSD, the API bootstrap, laws."
tags:
- dankserver
- audiobooks
@@ -36,20 +36,34 @@ manager) — **Internal-only:** LAN address, no vhost, no DNS name.
| Self-healing | `Restart=always`, `RestartSec=5`, `WantedBy=default.target` |
| Deploy | `./scripts/deploy.sh dankserver --service audiobookshelf` |
| Handler | `deploy_audiobookshelf` in `scripts/deploy-remote.sh` |
| Library layout | one book per folder — ABS groups by the first directory holding a media file (see Laws); every move is journalled at `/mnt/entropy/media-relayout-audiobooks.jsonl` |
## First run
## Bootstrap
1. Open `http://192.168.0.235:8000` **from the LAN**. ABS accepts the initial
account creation only from a local-network address; a remote address gets a
refusal, not a form.
2. Create the owner account, then add one **Audi Book** library with folder
`/audiobooks`. The path is the in-container name of the read-only bind.
3. The first scan reads the whole tree once. It runs in the background — the
deploy gate is already green by then, and `IOWeight=10` keeps it off the
critical path.
`scripts/audiobookshelf-init.sh` — run by the deployer after the health gate, so a
deployed ABS is a *usable* ABS: `/healthcheck` answers 200 on a server with no
user, no library and no scan, and nothing else in the deploy notices the
difference. The script is idempotent; re-running it costs one incremental scan.
Nothing in this repo holds an ABS credential: the account is created in the
browser and lives in the `config` bind on the SSD.
| Step | Door | Printed |
|------|------|---------|
| credential | `~/.config/secrets/audiobookshelf` (mode 600, generated once, host-side) | `credential: created …` |
| account | `POST /init` with `{"newRoot":{"username","password"}}` | `account: created (frank)` / `already registered` |
| session | `POST /login` → bearer token | `token: obtained (root)` |
| library | `POST /api/libraries`, folder `/audiobooks`, provider `audiobook` | `library: created Audiobooks (<id>)` / `exists` |
| scan | `POST /api/libraries/<id>/scan` | `scan: accepted` |
| proof | `GET /api/libraries/<id>/items?limit=1` | `items: <total>` |
Sign in at `http://192.168.0.235:8000` as `frank`; the password is
`awk -F= '/^ABS_ADMIN_PASSWORD=/{print $2}' ~/.config/secrets/audiobookshelf` on
Dankserver. Change it in **Settings → Users → frank → Password**; the secret file
then holds a stale value and only matters again if the `config` bind is ever
destroyed (a fresh DB makes the script create a new owner from it).
A full scan is slow and runs in the background; watch the `items:` total climb,
never `/healthcheck`. It also reads files that are genuinely broken and logs
`[AudioFileScanner] Invalid data found when processing input` per file — that is
the file, not the bind: the same line count grows while the total climbs.
## Laws
@@ -95,10 +109,95 @@ browser and lives in the `config` bind on the SSD.
in front. Any public vhost requires the Anubis + Authentik chain first, and the
outpost proxy provider has to exist before the vhost flips (external exposure
policy in `AGENTS.md`).
- **There is no `/setup` route; the creation door is `POST /init`.** `/status`
(no `/api` prefix) reports `isInit` = "a root user exists", `POST /init` with
`{"newRoot":{"username","password"}}` creates it, `POST /login` returns the
bearer every `/api/*` route wants. `POST /setup` answers `Unauthorized` — that
is a missing route wearing an auth message. Read off `dist-server/server/Server.js`
inside the pinned digest rather than from a forum.
- **A 200 may carry plain text.** `res.sendStatus(200)` answers the body `OK`, and
`/init` and the scan route use it; a client that parses every success as JSON
crashes on the one call that matters (`tests/audiobookshelf_init.bats` pins it).
- **The item count is not a field on the library DTO, and paging never ends
itself.** The door is `GET /api/libraries/<id>/items?limit=1` → `total`; a poller
that guesses a plausible name (`numItems`) prints `0` forever beside a scan that
is working. Paging is on `page` (0-based) plus `limit`, and **`offset` is
ignored**: `limit=500&offset=500` returns the same 500 ids as `offset=0`, while
`page=1` shares none of them. A census that pages on `offset` therefore reads
`ceil(total / limit) × limit` rows containing only `limit` distinct items — the
exact origin of the "exactly 1,500" and "exactly 4,000" counts two runs
reported for one library. Page on `page`, count *unique ids*, stop on `total`,
and take the number after the pass settles (`total` moves during a scan).
- **A fresh scan is the only thing that notices new audio.** `inotify` does not
cross a rootless podman bind — measured for [[Jellyfin]], whose page owns that
law — so the in-container watcher sees nothing a host process files onto the
array, and the deployer's re-run of the bootstrapper is the cheap scan trigger.
- **One stray media file at a collection root turns the whole collection into one
book.** Every scan re-walks the tree from disk (`scanFolder` → `recurseFiles`),
and `utils/scandir.js:groupFileItemsIntoLibraryItemDirs` assigns each file to the
FIRST directory along its path that already has a group: the first file seen at a
root owns every deeper file, and the folders below it never become items. One
949 MB `.cbr` sitting at `_unsorted`'s root collapsed 652 book folders into a
single item of 5,693 tracks, titled after whichever file supplied metadata.
"Hundreds of books are missing" is a layout fault, not a scan fault.
- **"Media file" means audio OR ebook, so an audio-only census cannot see the file
that is hiding the library.** With `audiobooksOnly=false` (the deployed shape)
the predicate is `SupportedAudioTypes` ∪ `SupportedEbookTypes`: 20 audio
extensions (including `mp4`, `mpg`, `webm`) plus `epub pdf mobi azw3 cbr cbz`. A
census reporting "that root holds no loose audio" can be correct and still
useless — the file doing the shadowing here was a comic archive.
- **Deleting the covering item does not un-shadow anything.** Grouping is recomputed
from disk, so the next scan rebuilds the item from the same stray file (measured:
`DELETE`, then scan → `1 Added`, the item back at the same path). The fix is the
MOVE: give every stray root file a folder of its own (one file per folder is
ABS's own single-book layout; formats of one book share a folder), through the
journal-guarded mover, then delete the stale item so the old path is not kept,
then scan.
- **`?force=1` is neither a discovery arm nor a repair arm.** Both arms of
`POST /libraries/:id/scan` walk the disk, so finding new folders needs neither
and the deployer's cheap trigger stays on the plain arm. Force additionally
re-scans every item that already exists (measured 1:33 against 0:04 on ~4,600
items) and repaired nothing on a library whose residuals were file-level: item
count, path set and media-less count identical before and after.
- **`DELETE /api/items/:id` is a DB row; `?hard=1` is `fs.remove()`.** In
`LibraryItemController.delete`, `hardDelete = req.query.hard == 1` and then
`fs.remove(libraryItemPath)`. Pointed at a covering item, that path is the whole
swallowed subtree — `hard=1` there deletes every book beneath it off the array.
Assert the item's `path` first, and never pass `hard`.
- **`Invalid data found when processing input` names the file, not the bind.** The
scanner logs it per file and skips that file; a book whose files ALL fail never
becomes an item at all, so the disk-vs-DB folder census shows it as a book with
no item. That is corrupt media, not a lost book. Never clear a suspect file with
a lenient probe: `ffprobe -show_entries format=duration` happily returns three
hours for the WAV bytes wearing a `.mp3` name that the scanner refuses. Read the
log's file paths instead, and `stat -c %s` first — 0-byte files are the common
case, and a truncated download reports `moov atom not found`.
- **A `Disc 01` / `CD 1` subfolder is part of a book, not a book.** The extension
census counts those folders; ABS deliberately does not — its grouping attaches
their files to the ancestor item (measured: 110 such folders under one item
carrying all 192 tracks). Read the basename before believing a disk-vs-DB gap.
- **`.cbr`/`.cbz` group like media and never play.** They sit in the grouping
predicate, so they hide books, but the comic parser throws `ERAR_BAD_ARCHIVE`
(`parseComicMetadata` → `CbrComicBookExtractor`) and the item lands with no
media. A media-less row beside those lines is the file, not the scanner.
- **A scan is finished when a NEW scan uuid completes.** Match
`Library scan "<uuid>" completed in ...`. `podman logs --since 30m` is a sliding
window: the set of visible completion lines changes on its own as old lines age
out, so a poller keyed on that hash reads a mid-scan number and calls it done.
## Verification
`./tests/run.sh` covers this unit through `tests/dankserver_quadlet.bats`
(boot persistence, the SSD state roots, the `:ro` array bind, the mount gate, the
`wget` health check) and `tests/media_overlay.bats` (only [[Plex]] and
[[Jellyfin]] may reference the metadata view).
`wget` health check), `tests/audiobookshelf_init.bats` (the bootstrapper against a
stub of the API above: create-once, idempotent re-run, no secret in a URL, refusal
by URL when the server does not answer, and the deployer calling it only after the
health gate) and `tests/media_overlay.bats` (only [[Plex]] and [[Jellyfin]] may
reference the metadata view).
Five book folders hold media no scanner can read, so they stay out of the library:
three 2024 Nebula nominees in the `_unsorted` box (two `.m4b` with no `moov` atom,
one 36-file `.mp3` set that is not MPEG audio at all), `Anthony Heald/Bounty
Hunter Wars - Book 1` and `Star Wars Full Cast Audio Drama/Dark Empire III` — the
latter two also exist in readable form under their author folders. Fix by
re-acquiring the files; the scanner names every one of them in its log.
+8 -1
View File
@@ -3,7 +3,7 @@ type: reference
title: Jellyfin
description: "Jellyfin on Dankserver: quadlet unit, overlay metadata view, provider keys, ports, access, transcode status."
tags: [jellyfin, dankserver, media-server]
timestamp: 2026-10-30T00:00:00Z
timestamp: 2026-11-02T11:00:00Z
---
# Jellyfin
@@ -328,6 +328,13 @@ the progress read.** A task that has never completed answers
ran" from "finished a moment ago" — the difference between a door that is dead
and one that merely answered `204`.
**The settle read is `GET /ScheduledTasks`: the row whose `Name` is exactly
`Scan Media Library`, and its `State` (`Idle`|`Running`).** `/Library/ScanStatus`
is a 404 on 12.4, so a watcher built on the old endpoint reports nothing and a
scanning server looks an idle one. Call it settled only after a `Running`
sighting followed by two consecutive `Idle`s: sub-tasks leave gaps that a single
poll reads as finished, and a census taken in one reports the pre-scan library.
**The per-library `POST /Items/{id}/Refresh` queues nothing, and the provider
switch is not the reason.** Measured twice on 2026-11-02 against `Shows`: before
the flip and again after it, the call answered `204`, the journal wrote zero lines
@@ -3,7 +3,7 @@ type: reference
title: Media Relayout
description: "Census, plan and apply the movie, TV, music and audiobook libraries: the gates, the refusals, the rollback."
tags: [dankserver, media-server, jellyfin, plex, procedure]
timestamp: 2026-11-01T01:00:00Z
timestamp: 2026-11-02T11:00:00Z
---
# Media Relayout
@@ -186,162 +186,10 @@ show. A name whose marker is `Part 2` is a film and stays put.
## The TV library's own relayout
`/mnt/entropy/TVShows` is scene packs rather than a library, and Jellyfin has
already matched and numbered most of it. `scripts/media-shows.py` re-files that
library into the shape Jellyfin reads, with **tvdb as the name authority**:
<Series> (<year>) [tvdbid-NNNN]/Season NN/
<Series> (<year>) - SxxEyy - <Episode Title>.ext
It never guesses: every series name, episode number and title it writes traces to
an authority, and what has none is parked for a human to read.
```bash
# 1. The authority: Jellyfin's own matched rows — each carries the series' TMDb
# and Tvdb ids and the title the row holds (the DB opens read-only).
python3 scripts/media-census.py jellyfin --root /mnt/entropy/TVShows \
--out /tmp/tv_titles.tsv
# 2. What is on disk (stat only), and the runtimes the film arm needs.
python3 scripts/media-census.py enumerate --root /mnt/entropy/TVShows \
--out /tmp/tv_enum.tsv
python3 scripts/media-census.py probe --root /mnt/entropy/TVShows \
--enum /tmp/tv_enum.tsv --jobs 6 --out /tmp/tv_probed.tsv
# The probe is the slow step: ffprobe reads both ends of every file. So re-use
# the last one — a duration belongs to the file, not to its name: re-key the
# previous census through that pass's own journal (`src` -> `dst`) instead of
# seeking the array again. It is also the only column the film arm reads, and
# only for a file with no marker and no provider row (4,794 of 58,551 today,
# all of them covered by a re-key that costs seconds, not an hour).
# 3. The plan. --tmdb needs TMDB_API_KEY — the v3 key in ~/.config/secrets/tmdb,
# which travels as the `api_key` query parameter. Only a v4 request token is
# accepted as a Bearer header, and the wrong shape answers 401 on EVERY call,
# which the summary line flags as NOT provider-complete. --tvdb reads its
# `api_key=` line from ~/.config/secrets/tvdb, never argv. Both write the SAME
# cache file, so the plan is reproducible offline afterwards: run it once to
# fill the cache, and quote the second run.
# Paths are only strings to a dry run, so this step can run on the workstation
# against scp'd TSVs — which is where the two provider secrets live.
python3 scripts/media-shows.py --root /mnt/entropy/TVShows --enum /tmp/tv_enum.tsv \
--titles /tmp/tv_titles.tsv --probed /tmp/tv_probed.tsv \
--tmdb-cache /mnt/entropy/media-tv-names.json --tmdb --tvdb \
--park /mnt/entropy/media-park/TVShows \
--out /mnt/entropy/media-relayout-shows2-plan.tsv \
--exceptions /mnt/entropy/media-relayout-shows2-exceptions.txt
# 4. Read the exceptions file FIRST — it is the review, not a log. Same executor
# as above, but on this arm's OWN journal, NAMED PER PASS: the journal of an
# applied pass is its unwind record, so a second pass that reuses the name
# destroys the only map back.
scripts/media-apply.py apply --plan /mnt/entropy/media-relayout-shows2-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/media-relayout-shows2.jsonl --apply
```
Evidence, strongest first — the order the planner believes things:
- the **episode marker the file's own name states** (`S01E02-E03`, `1x04`); a digit
run longer than three at that position is NOT a number — `S03E2019` and
`S04E47370355` are one box set's airdate fragment and pack index — so such a file
is renumbered by tvdb's title or parks, and is never truncated down to fit;
- the **`Season NN` folder it sits in and the number its name leads with inside
it**: `Season 19/01 Charles Barkley, Nirvana.avi` is S19E01;
- the **provider row**: which episode the file IS, and the numbering of a file
whose own name says nothing;
- the **row's tvdb id, else its TMDb id carried through TMDb's external ids, else
an exact tvdb name search**: that id names the SERIES FOLDER and supplies the
episode title, because a name the server read off a folder is not a show name.
The folder CARRIES the id: `[tvdbid-NNNN]` is what Jellyfin's Tvdb agent and
Plex's scanner both read, so a show whose title is ambiguous resolves by number
instead of by a human guessing which folder is which. The id rides the folder
only — a file name gets 180 characters and the episode title needs the room.
- a **bonus folder** anywhere above the file, or Jellyfin's `ExtraType` → the
series' canonical extra folder;
- a runtime over `SHORT_MAX` makes it film-shaped, and it leaves for `--movies` only
when the movie provider's title is a run of words the FILE itself states and its
year agrees with the file's: the query is only what `title_of` salvaged from a
release name, and the first hit for a short one is a near miss — a 1963 Bond film
for the word "russia", *Special Edition I* for a file saying *Edition III*. Else it
parks.
Four refusals carry the design: what the planner will NOT do is the point of it.
- **Folders resolve a series; they never invent one.** A folder's words may
resolve a file to a series a provider has already named — `(CBB) Tengen Toppa
Gurren Lagann - The Complete Series (BD 1080p)` names the registered *Tengen
Toppa Gurren Lagann* — but a bonus subfolder is not a show. An early run made
`…/03. Extra/01. NCOP/NCOP 01.mkv` into a root series called `01 NCOP`.
- **A scraped row does not outrank the filesystem.** Jellyfin numbers
multi-season packs badly: every file under `SNL 16-20/Season 16..19` carries
season 16 episode 20, and one Gundam row says season 2004. The folder, then
the file's own leading number, win.
- **A season that is a YEAR is the date matcher, not a season.** Jellyfin files
specials by airdate, so a row can arrive saying `season 2002`; that is not a
season and it never becomes a folder. The one authority on such a row is tvdb's
episode list read backwards: the title the row carries names the (season,
episode) tvdb gives it. A title tvdb gives to two episodes names nothing, and
the file parks with the reason it always carried. `Season 00` IS a legal
season — specials live there — so a row that really says 0 files there instead
of parking.
- **A season tvdb's own list contradicts gets no tvdb title.** An anime pack
numbered by its release group books `S05E01` in a season tvdb does not hold, or
books more episodes there than tvdb lists: the slot its number points at is a
different episode, and dressing it in that episode's title is a confident lie.
The file still renames to the standard shape; the title does not ride, and the
series is listed under *seasons whose numbering tvdb contradicts* in the
exceptions. A season holding FEWER files than tvdb lists stays plausible — a
partial pack is the normal shape. Neither reading holds for a list tvdb
cut off at its own page limit: the default episode query answers at most
500 rows and states no total, and One Piece, Crayon Shin-chan and
Detective Conan each answer exactly 500 while holding over a thousand —
a page bounds nothing above itself, so it contradicts nothing.
**A provider name is evidence only when an id backs it.** Jellyfin names a Series
after the folder it scanned, so a library filed by release folders ends up with
one Series item per box set — `24 (2001) Season 1`, `Modern Family S05` — and
copying those names verbatim launders folder junk into the contract. The planner
resolves each series to its tvdb id and writes tvdb's name and year; the merged
identity is also what turns a box set split across five release folders into ONE
folder, and what makes two files claiming one episode MEET in the duplicate rule.
A series no id resolves keeps the provider's words and is listed under *no tvdb
identity* in the exceptions — that list, not a rename, is the review.
**The review pile lives outside the library.** Files parked inside
`/mnt/entropy/TVShows` were indexed as one show named `_unsorted`, and that
show's rows then looked like evidence. So a row whose series is the hidden
top-level folder the file itself sits in is dropped — and the folder is matched
with its leading underscore stripped, because the server *displays* `_unsorted` as
`unsorted`: compare the spelled names and the pile survives the relayout as a
visible series folder that real files get renamed into. `--park` outside the root
makes the next plan re-file the pile instead of skipping it. Inside a hidden
top-level folder a file's own words may MATCH a registered show, never invent one.
**The operator's id map is the door for a name tvdb cannot be searched by.**
`scripts/media-shows.py` reads `scripts/media-shows-tvdb-ids.tsv` by default: one
line per pack folder, `folder <TAB> tvdb id <TAB> note`. The file exists because
tvdb titles a record in the show's own language and the English name rides in
*aliases* the name search never reads: `ブラッククローバー` (331753) answers to no
spelling of "Black Clover", and the bare alias `One Piece` belongs to 392276, the
2023 live action, not to the 1999 record (81797) the pack here is. The map
outranks a provider id — a box set's provider id is the server's own guess at a
folder name — and it merges the release folders that are one show. Everything it
touched is written to the exceptions: every folder it named, every row that
matched no folder, every disagreement with a provider id, every id tvdb never
answered. A line whose id is not a number stops the plan, because a typo renames
a whole pack to the wrong show. Only folders the search CANNOT reach belong in
it; a row for a folder the search would resolve hides a live lookup behind a
number nobody re-checks.
`scripts/media-shows-tvdb.py` makes the sheet the map is built from. It reads a
plan's `mkdir` rows, asks tvdb about every destination folder carrying no
`[tvdbid-`, and writes one row per folder with the verdict that says how much was
guessed to get there: `EXACT` (tvdb's own spelling), `PUNCT` (equal only after
release junk is stripped from the folder's words), `NEAR` (tvdb answered
something else — the best hit is printed for a human and nothing applies it),
`NONE`. Acceptance never moved across the passes that built this: a hit counts
only when tvdb's name equals the query with every non-alphanumeric dropped from
both sides, and the record's year still has to fall inside any year span the
folder names. That gate is what caught three confident picks — Fairy Tail's 2024
sequel (410031), the 2026 Hokuto no Ken remake (465216) and `Mugyutto! Black
Clover` (378762) offered for Black Clover. It is why the map is approved by hand
and the sheet is never fed to the planner.
`/mnt/entropy/TVShows` is its own arm: a different authority (tvdb), its own
refusals, and its own test for whether a folder really holds two shows —
[[Media TV Relayout]]. It shares this page's executor, the one-journal-per-arm
law, the sweep and the rollback.
## The music and audiobook arms
@@ -432,6 +280,14 @@ executor, the one-journal-per-arm law, the sweep and the rollback.
`--root` set to the share that holds BOTH libraries: the executor refuses any
path outside `--root`, and a journal shared with the movie relayout would
undo both runs when only one of them is wanted.
- **A hold folder outside `--root` forces `--force` on BOTH `apply` and
`sweep`.** `sweep` refuses the entire journal when any journalled path sits
outside the root (`this journal belongs to another library`) — the same guard
the plan hits, and it exists to stop one pass unwinding another's files.
`--force` relaxes it and stays safe there: `sweep_targets` marks only
ancestors under `--root`, so a hold folder beyond the root can never be
selected; it only blocks everything that is. The cleaner shape is the law
above — put `--root` on the share that holds both.
- **Sidecar metadata does not follow a rename, because it lives on another
filesystem.** The media servers write `.nfo` and artwork through the
@@ -440,6 +296,9 @@ executor, the one-journal-per-arm law, the sweep and the rollback.
`scripts/media-view.sh reap --apply` — stranded sidecars become ghost folders
that both servers browse and report as content. Whether this host mounts the
view at all is stated on that page; `media-apply.py` stays silent when it does not.
A rename of folders the server never identified strands nothing: Jellyfin
writes a sidecar only for a series it recognised, so `reap` answering
`no ghosts` after such a pass is the expected result, not a broken tool.
- **Nothing deletes through the metadata view.** A delete there writes an overlayfs
whiteout in the upper layer, which hides the array's copy without removing it;
the file returns when the view is unmounted. Deletions in this pipeline happen
@@ -0,0 +1,201 @@
---
type: reference
title: Media TV Relayout
description: "Re-file TVShows from scene packs into the shape Jellyfin reads: authority order, refusals, the two-series split test."
tags: [dankserver, media-server, jellyfin, plex, procedure]
timestamp: 2026-11-02T11:00:00Z
---
# Media TV Relayout
The `/mnt/entropy/TVShows` share arrived as scene packs — release folders whose
names are the release, not the show. This arm re-files it into the shape
[[Jellyfin]] reads, with tvdb as the name authority. The executor, the journal,
the sweep and the rollback belong to [[Media Relayout]]; this page owns the TV
authority order, the refusals, and the test that decides a split.
`/mnt/entropy/TVShows` is scene packs rather than a library, and Jellyfin has
already matched and numbered most of it. `scripts/media-shows.py` re-files that
library into the shape Jellyfin reads, with **tvdb as the name authority**:
<Series> (<year>) [tvdbid-NNNN]/Season NN/
<Series> (<year>) - SxxEyy - <Episode Title>.ext
It never guesses: every series name, episode number and title it writes traces to
an authority, and what has none is parked for a human to read.
```bash
# 1. The authority: Jellyfin's own matched rows — each carries the series' TMDb
# and Tvdb ids and the title the row holds (the DB opens read-only).
python3 scripts/media-census.py jellyfin --root /mnt/entropy/TVShows \
--out /tmp/tv_titles.tsv
# 2. What is on disk (stat only), and the runtimes the film arm needs.
python3 scripts/media-census.py enumerate --root /mnt/entropy/TVShows \
--out /tmp/tv_enum.tsv
python3 scripts/media-census.py probe --root /mnt/entropy/TVShows \
--enum /tmp/tv_enum.tsv --jobs 6 --out /tmp/tv_probed.tsv
# The probe is the slow step: ffprobe reads both ends of every file. So re-use
# the last one — a duration belongs to the file, not to its name: re-key the
# previous census through that pass's own journal (`src` -> `dst`) instead of
# seeking the array again. It is also the only column the film arm reads, and
# only for a file with no marker and no provider row (4,794 of 58,551 today,
# all of them covered by a re-key that costs seconds, not an hour).
# 3. The plan. --tmdb needs TMDB_API_KEY — the v3 key in ~/.config/secrets/tmdb,
# which travels as the `api_key` query parameter. Only a v4 request token is
# accepted as a Bearer header, and the wrong shape answers 401 on EVERY call,
# which the summary line flags as NOT provider-complete. --tvdb reads its
# `api_key=` line from ~/.config/secrets/tvdb, never argv. Both write the SAME
# cache file, so the plan is reproducible offline afterwards: run it once to
# fill the cache, and quote the second run.
# Paths are only strings to a dry run, so this step can run on the workstation
# against scp'd TSVs — which is where the two provider secrets live.
python3 scripts/media-shows.py --root /mnt/entropy/TVShows --enum /tmp/tv_enum.tsv \
--titles /tmp/tv_titles.tsv --probed /tmp/tv_probed.tsv \
--tmdb-cache /mnt/entropy/media-tv-names.json --tmdb --tvdb \
--park /mnt/entropy/media-park/TVShows \
--out /mnt/entropy/media-relayout-shows2-plan.tsv \
--exceptions /mnt/entropy/media-relayout-shows2-exceptions.txt
# 4. Read the exceptions file FIRST — it is the review, not a log. Same executor
# as above, but on this arm's OWN journal, NAMED PER PASS: the journal of an
# applied pass is its unwind record, so a second pass that reuses the name
# destroys the only map back.
scripts/media-apply.py apply --plan /mnt/entropy/media-relayout-shows2-plan.tsv \
--root /mnt/entropy --journal /mnt/entropy/media-relayout-shows2.jsonl --apply
```
Evidence, strongest first — the order the planner believes things:
- the **episode marker the file's own name states** (`S01E02-E03`, `1x04`); a digit
run longer than three at that position is NOT a number — `S03E2019` and
`S04E47370355` are one box set's airdate fragment and pack index — so such a file
is renumbered by tvdb's title or parks, and is never truncated down to fit;
- the **`Season NN` folder it sits in and the number its name leads with inside
it**: `Season 19/01 Charles Barkley, Nirvana.avi` is S19E01;
- the **provider row**: which episode the file IS, and the numbering of a file
whose own name says nothing;
- the **row's tvdb id, else its TMDb id carried through TMDb's external ids, else
an exact tvdb name search**: that id names the SERIES FOLDER and supplies the
episode title, because a name the server read off a folder is not a show name.
The folder CARRIES the id: `[tvdbid-NNNN]` is what Jellyfin's Tvdb agent and
Plex's scanner both read, so a show whose title is ambiguous resolves by number
instead of by a human guessing which folder is which. The id rides the folder
only — a file name gets 180 characters and the episode title needs the room.
- a **bonus folder** anywhere above the file, or Jellyfin's `ExtraType` → the
series' canonical extra folder;
- a runtime over `SHORT_MAX` makes it film-shaped, and it leaves for `--movies` only
when the movie provider's title is a run of words the FILE itself states and its
year agrees with the file's: the query is only what `title_of` salvaged from a
release name, and the first hit for a short one is a near miss — a 1963 Bond film
for the word "russia", *Special Edition I* for a file saying *Edition III*. Else it
parks.
Four refusals carry the design: what the planner will NOT do is the point of it.
- **Folders resolve a series; they never invent one.** A folder's words may
resolve a file to a series a provider has already named — `(CBB) Tengen Toppa
Gurren Lagann - The Complete Series (BD 1080p)` names the registered *Tengen
Toppa Gurren Lagann* — but a bonus subfolder is not a show. An early run made
`…/03. Extra/01. NCOP/NCOP 01.mkv` into a root series called `01 NCOP`.
- **A scraped row does not outrank the filesystem.** Jellyfin numbers
multi-season packs badly: every file under `SNL 16-20/Season 16..19` carries
season 16 episode 20, and one Gundam row says season 2004. The folder, then
the file's own leading number, win.
- **A season that is a YEAR is the date matcher, not a season.** Jellyfin files
specials by airdate, so a row can arrive saying `season 2002`; that is not a
season and it never becomes a folder. The one authority on such a row is tvdb's
episode list read backwards: the title the row carries names the (season,
episode) tvdb gives it. A title tvdb gives to two episodes names nothing, and
the file parks with the reason it always carried. `Season 00` IS a legal
season — specials live there — so a row that really says 0 files there instead
of parking.
- **A season tvdb's own list contradicts gets no tvdb title.** An anime pack
numbered by its release group books `S05E01` in a season tvdb does not hold, or
books more episodes there than tvdb lists: the slot its number points at is a
different episode, and dressing it in that episode's title is a confident lie.
The file still renames to the standard shape; the title does not ride, and the
series is listed under *seasons whose numbering tvdb contradicts* in the
exceptions. A season holding FEWER files than tvdb lists stays plausible — a
partial pack is the normal shape. Neither reading holds for a list tvdb
cut off at its own page limit: the default episode query answers at most
500 rows and states no total, and One Piece, Crayon Shin-chan and
Detective Conan each answer exactly 500 while holding over a thousand —
a page bounds nothing above itself, so it contradicts nothing.
**A provider name is evidence only when an id backs it.** Jellyfin names a Series
after the folder it scanned, so a library filed by release folders ends up with
one Series item per box set — `24 (2001) Season 1`, `Modern Family S05` — and
copying those names verbatim launders folder junk into the contract. The planner
resolves each series to its tvdb id and writes tvdb's name and year; the merged
identity is also what turns a box set split across five release folders into ONE
folder, and what makes two files claiming one episode MEET in the duplicate rule.
A series no id resolves keeps the provider's words and is listed under *no tvdb
identity* in the exceptions — that list, not a rename, is the review.
**The review pile lives outside the library.** Files parked inside
`/mnt/entropy/TVShows` were indexed as one show named `_unsorted`, and that
show's rows then looked like evidence. So a row whose series is the hidden
top-level folder the file itself sits in is dropped — and the folder is matched
with its leading underscore stripped, because the server *displays* `_unsorted` as
`unsorted`: compare the spelled names and the pile survives the relayout as a
visible series folder that real files get renamed into. `--park` outside the root
makes the next plan re-file the pile instead of skipping it. Inside a hidden
top-level folder a file's own words may MATCH a registered show, never invent one.
**The operator's id map is the door for a name tvdb cannot be searched by.**
`scripts/media-shows.py` reads `scripts/media-shows-tvdb-ids.tsv` by default: one
line per pack folder, `folder <TAB> tvdb id <TAB> note`. The file exists because
tvdb titles a record in the show's own language and the English name rides in
*aliases* the name search never reads: `ブラッククローバー` (331753) answers to no
spelling of "Black Clover", and the bare alias `One Piece` belongs to 392276, the
2023 live action, not to the 1999 record (81797) the pack here is. The map
outranks a provider id — a box set's provider id is the server's own guess at a
folder name — and it merges the release folders that are one show. Everything it
touched is written to the exceptions: every folder it named, every row that
matched no folder, every disagreement with a provider id, every id tvdb never
answered. A line whose id is not a number stops the plan, because a typo renames
a whole pack to the wrong show. Only folders the search CANNOT reach belong in
it; a row for a folder the search would resolve hides a live lookup behind a
number nobody re-checks.
`scripts/media-shows-tvdb.py` makes the sheet the map is built from. It reads a
plan's `mkdir` rows, asks tvdb about every destination folder carrying no
`[tvdbid-`, and writes one row per folder with the verdict that says how much was
guessed to get there: `EXACT` (tvdb's own spelling), `PUNCT` (equal only after
release junk is stripped from the folder's words), `NEAR` (tvdb answered
something else — the best hit is printed for a human and nothing applies it),
`NONE`. Acceptance never moved across the passes that built this: a hit counts
only when tvdb's name equals the query with every non-alphanumeric dropped from
both sides, and the record's year still has to fall inside any year span the
folder names. That gate is what caught three confident picks — Fairy Tail's 2024
sequel (410031), the 2026 Hokuto no Ken remake (465216) and `Mugyutto! Black
Clover` (378762) offered for Black Clover. It is why the map is approved by hand
and the sheet is never fed to the planner.
## Laws of this arm
- **A folder that names two series is not evidence of two series — the files
answer.** `ffprobe -show_entries format_tag=title` names the series each file
belongs to. Measured 2026-11-02 on `COW and CHICKEN … Complete Animated TV
Series`, a bundle whose name pairs *Cow and Chicken* with *I Am Weasel*: 103
of 104 files carry a `Cow and Chicken - SxxEyy - <title>` tag and none carries
an *I Am Weasel* one. The 7-minute duration class is a half-episode segment,
not a second show, and a file holding both segments needs a remux, which this
executor will never do. Split only when the tags (or a provider episode list)
name both series per file; everything else is a rename.
- **When the numbering is right and only the name is wrong, rename the folder
and nothing else.** Tags whose `Sxx` matches the `Season NN` folder holding
them, per-season counts equal, and no duplicate `SxxEyy` key anywhere prove
the layout is already correct. Renumbering files under that evidence buys
nothing and can silently mislabel an episode.
## See Also
[[Media Relayout]] — the executor, the journal, the sweep and the rollback this
arm shares. [[Jellyfin]] — the reader of the resulting shape. [[Plex]] — the
second reader. [[Media Metadata View]] — where a rename's sidecars strand, or
do not.
## Citations
- Jellyfin Shows documentation — series, season and episode folder/file naming:
<https://jellyfin.org/docs/general/server/media/shows>
+2 -1
View File
@@ -44,7 +44,7 @@ is a locator: one row, one clause, no numbers.
| Page | Owns |
|---|---|
| [[Audiobookshelf]] | Audiobookshelf on Dankserver: quadlet shape, the read-only array bind, state on the SSD, first-run account, laws. |
| [[Audiobookshelf]] | Audiobookshelf on Dankserver: quadlet shape, the read-only array bind, state on the SSD, the API bootstrap, laws. |
| [[Calibre]] | Calibre is the e-book library manager that backs Calibre-Web. |
| [[Calibre-Web]] | Calibre-Web is a web application that browses, reads, and downloads eBooks from an existing Calibre library database. |
| [[Copyparty]] | Copyparty is a lightweight file server and media streamer that runs as a single Python binary. |
@@ -57,6 +57,7 @@ is a locator: one row, one clause, no numbers.
| [[Media Metadata View]] | The SSD-backed overlay the two media servers read and write: shape, guards, commands, laws, rollback. |
| [[Media Music Relayout]] | Re-filing /mnt/entropy/Music by Jellyfin's tag rows: the census, the album key, the refusals. |
| [[Media Relayout]] | Census, plan and apply the movie, TV, music and audiobook libraries: the gates, the refusals, the rollback. |
| [[Media TV Relayout]] | Re-file TVShows from scene packs into the shape Jellyfin reads: authority order, refusals, the two-series split test. |
| [[Plex]] | Plex Media Server is a **containerized media server on Dankserver**: a podman |
| [[redis]] | The Redis cache and task queue for RomM: quadlet unit, network alias topology, and the backup path. |
| [[RomM]] | RomM (ROM Manager) is a self-hosted ROM library manager and player. |
+200
View File
@@ -0,0 +1,200 @@
#!/usr/bin/env bats
# Seam: scripts/audiobookshelf-init.sh — the Audiobookshelf bootstrapper.
#
# A freshly deployed Audiobookshelf answers /healthcheck 200 while it has no
# user, no library and no scan, so "healthy" proves nothing about usefulness.
# This seam pins what the bootstrapper must make true, against a stub that
# speaks the same API as the pinned image (POST /init + POST /login at the
# root, bearer-authenticated /api/*, library count only on /items):
# 1. an unregistered server gains the account, the library and a scan, and the
# credential lands mode 600 with both keys;
# 2. a second run creates nothing new (exactly one /init, one library) and
# still fires the incremental scan;
# 3. neither the bearer token nor the password ever rides a URL — a secret in
# a URL or an argv is world-readable in /proc/<pid>/cmdline, which is how
# an HF token leaked into a transcript on 2026-09-16;
# 4. a server that cannot be reached is refused by URL with exit 1, never
# reported as success (a green deploy over a dead API is a lie);
# 5. ABS_SCAN outside 0/1 is a usage error raised before anything is touched;
# 6. the deployer runs the bootstrapper only after the health gate answered.
load helpers
setup() {
INIT="$REPO_ROOT/scripts/audiobookshelf-init.sh"
TMP="$(mktemp -d)"
PORT=18937
cat > "$TMP/stub.py" <<'PY'
import json, sys
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
port, logf = int(sys.argv[1]), sys.argv[2]
state = {"init": False, "user": None, "pw": None, "libs": []}
def log(line):
with open(logf, "a") as f:
f.write(line + "\n")
class H(BaseHTTPRequestHandler):
def log_message(self, *a):
pass
def send(self, code, body, as_json=False):
data = json.dumps(body).encode() if as_json else str(body).encode()
self.send_response(code)
self.send_header("Content-Type", "application/json" if as_json else "text/plain")
self.send_header("Content-Length", str(len(data)))
self.end_headers()
self.wfile.write(data)
def body(self):
n = int(self.headers.get("Content-Length") or 0)
return json.loads(self.rfile.read(n) or b"{}")
def authed(self):
return self.headers.get("Authorization") == "Bearer stub-token"
def do_GET(self):
path = self.path.split("?")[0]
log("GET %s auth=%s" % (self.path, "yes" if self.headers.get("Authorization") else "no"))
if path == "/status":
return self.send(200, {"app": "audiobookshelf", "isInit": state["init"]}, True)
if not self.authed():
return self.send(401, "Unauthorized")
if path == "/api/libraries":
return self.send(200, {"libraries": state["libs"]}, True)
if path.endswith("/items"):
return self.send(200, {"total": 42, "results": []}, True)
return self.send(404, "nope")
def do_POST(self):
path = self.path.split("?")[0]
payload = self.body()
log("POST %s auth=%s" % (self.path, "yes" if self.headers.get("Authorization") else "no"))
if path == "/init":
if state["init"]:
return self.send(500, "already initialized")
root = payload.get("newRoot", {})
state["init"] = True
state["user"] = root.get("username")
state["pw"] = root.get("password")
log("INIT %s" % root.get("username"))
return self.send(200, "OK")
if path == "/login":
if not state["init"]:
return self.send(401, "Unauthorized")
if (payload.get("username"), payload.get("password")) != (state["user"], state["pw"]):
return self.send(401, "Unauthorized")
return self.send(200, {"user": {"username": state["user"], "type": "root",
"id": "u1", "token": "stub-token"}}, True)
if not self.authed():
return self.send(401, "Unauthorized")
if path == "/api/libraries":
lib = {"id": "lib-1", "name": payload.get("name"), "folders": payload.get("folders", [])}
state["libs"].append(lib)
log("LIBRARY %s %s" % (lib["name"], lib["folders"]))
return self.send(200, lib, True)
if path.endswith("/scan"):
log("SCAN")
return self.send(200, "OK")
return self.send(404, "nope")
ThreadingHTTPServer(("127.0.0.1", port), H).serve_forever()
PY
python3 "$TMP/stub.py" "$PORT" "$TMP/requests.log" > "$TMP/stub.out" 2>&1 &
STUB_PID=$!
for _ in $(seq 1 50); do
curl -s -o /dev/null "http://127.0.0.1:$PORT/status" && break
sleep 0.1
done
}
teardown() {
[ -n "${STUB_PID:-}" ] && kill "$STUB_PID" 2>/dev/null || true
rm -rf "$TMP"
}
boot() {
env ABS_URL="http://127.0.0.1:$PORT" ABS_SECRET="$TMP/cred" \
ABS_ADMIN_USER=frank ABS_LIBRARY=Audiobooks ABS_FOLDER=/audiobooks \
ABS_SCAN="${SCAN:-1}" bash "$INIT"
}
@test "an unregistered server gains the account, the library and the scan" {
run boot
[ "$status" -eq 0 ]
printf '%s\n' "${lines[@]}" | grep -qx "credential: created $TMP/cred"
printf '%s\n' "${lines[@]}" | grep -q "^account: created (frank)"
printf '%s\n' "${lines[@]}" | grep -q "^token: obtained (root)"
printf '%s\n' "${lines[@]}" | grep -q "^library: created Audiobooks (lib-1)"
printf '%s\n' "${lines[@]}" | grep -q "^scan: accepted"
printf '%s\n' "${lines[@]}" | grep -q "^items: 42"
# The state roots of this service hold an account credential, so the file
# that carries it must never be group- or world-readable.
[ "$(stat -c %a "$TMP/cred")" = "600" ]
grep -qx 'ABS_ADMIN_USER=frank' "$TMP/cred"
grep -Eq '^ABS_ADMIN_PASSWORD=[0-9a-f]{24}$' "$TMP/cred"
# The folder the library indexes is the bind inside the container, not the
# host path — a library pointed at /mnt/entropy would answer 0 items and
# look like an empty audiobook collection.
grep -q "LIBRARY Audiobooks \[{'fullPath': '/audiobooks'}\]" "$TMP/requests.log"
}
@test "a second run creates nothing new and still fires the incremental scan" {
boot > /dev/null
[ -e "$TMP/requests.log" ]
run boot
[ "$status" -eq 0 ]
printf '%s\n' "${lines[@]}" | grep -q "^account: already registered"
printf '%s\n' "${lines[@]}" | grep -q "^library: exists Audiobooks (lib-1)"
printf '%s\n' "${lines[@]}" | grep -q "^scan: accepted"
# No second credential, no second account, no duplicate library: the deployer
# may run this on every re-deploy of the service.
! printf '%s\n' "${lines[@]}" | grep -q "credential: created"
[ "$(grep -c '^INIT ' "$TMP/requests.log")" -eq 1 ]
[ "$(grep -c '^LIBRARY ' "$TMP/requests.log")" -eq 1 ]
[ "$(grep -c '^SCAN$' "$TMP/requests.log")" -eq 2 ]
}
@test "the bearer token and the password never ride a URL" {
boot
# The stub logs the request line (method + path). A secret showing up there
# means it travelled as a query parameter — visible in access logs, shell
# history and /proc/<pid>/cmdline.
! awk '{print $2}' "$TMP/requests.log" | grep -q 'stub-token'
pw=$(awk -F= '/^ABS_ADMIN_PASSWORD=/{print $2}' "$TMP/cred")
[ -n "$pw" ]
! awk '{print $2}' "$TMP/requests.log" | grep -q -- "$pw"
# and the authenticated routes only answered once the header was present
grep -q '^GET /api/libraries auth=yes$' "$TMP/requests.log"
}
@test "a server that does not answer is refused by URL, not reported as success" {
run env ABS_URL="http://127.0.0.1:1" ABS_SECRET="$TMP/cred" bash "$INIT"
[ "$status" -eq 1 ]
[[ "$output" == *refused:* ]]
[[ "$output" == *"127.0.0.1:1"* ]]
}
@test "ABS_SCAN outside 0/1 is a usage error raised before anything is touched" {
run env ABS_URL="http://127.0.0.1:$PORT" ABS_SECRET="$TMP/cred" ABS_SCAN=2 \
bash "$INIT"
[ "$status" -eq 2 ]
[[ "$output" == *ABS_SCAN* ]]
[ ! -e "$TMP/cred" ]
}
@test "the deployer bootstraps only after the health gate has answered" {
body=$(awk '/^deploy_audiobookshelf\(\)/,/^}/' "$REPO_ROOT/scripts/deploy-remote.sh")
[ -n "$body" ]
printf '%s\n' "$body" | grep -q \
'wait_for_health audiobookshelf http://127\.0\.0\.1:8000/healthcheck'
printf '%s\n' "$body" | grep -q 'scripts/audiobookshelf-init\.sh'
printf '%s\n' "$body" | awk '
/wait_for_health audiobookshelf/ { health = NR }
/audiobookshelf-init\.sh/ { init = NR }
END { exit !(health && init && init > health) }'
}
+38 -3
View File
@@ -13,9 +13,12 @@
# 3. a downloaded plugin artifact is accepted only on its pinned sha256, its
# official guid and its pinned version (a swapped or downgraded artifact
# is refused before anything lands in plugins/);
# 4. the TMDb key is written into the EXISTING config with every other
# setting preserved — Jellyfin owns that file, and clobbering it would
# silently reset country, language and adult settings;
# 4. a key is written into the EXISTING config with every other setting
# preserved, under whatever root that file's owner chose — Jellyfin's core
# configs say `<Configuration>`, TheTVDB writes `<PluginConfiguration>` and
# omits a null setting outright, so the element to fill can be absent and
# the anchor has to come from the file; clobbering the file would silently
# reset country, language and adult settings;
# 5. a TVDB config the operator already saved through the UI is kept;
# 6. install is idempotent: a second run changes no byte;
# 7. whatever is replaced is copied to the trash first (fleet law: never
@@ -254,6 +257,38 @@ XML
grep -qF '<EnableSeasonPathMatching>true</EnableSeasonPathMatching>' "$cfg"
}
@test "the key lands in a plugin-written config with no ApiKey element" {
# What TheTVDB actually writes on a fresh install: its own root element and
# no <ApiKey /> at all, because its serializer drops null settings. The
# installer used to anchor on </Configuration> and refuse this file.
cat >"$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml" <<XML
<?xml version="1.0" encoding="utf-8"?>
<PluginConfiguration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<SubscriberPIN />
<CacheDurationInHours>1</CacheDurationInHours>
<OriginalTitle>true</OriginalTitle>
</PluginConfiguration>
XML
run prov install
[ "$status" -eq 0 ]
printf '%s\n' "${lines[@]}" | grep -qiF 'filled the ApiKey'
local cfg="$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml"
grep -qF "<ApiKey>$TVDB_KEY</ApiKey>" "$cfg"
grep -qF '<CacheDurationInHours>1</CacheDurationInHours>' "$cfg"
grep -qF '<OriginalTitle>true</OriginalTitle>' "$cfg"
}
@test "a config file with no root element is refused by name" {
# The guard still has to fire when there is genuinely nowhere to write —
# a refusal naming the file beats inventing a document shape.
printf '<?xml version="1.0" encoding="utf-8"?>\n' \
>"$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml"
run prov install
[ "$status" -ne 0 ]
printf '%s\n' "${lines[@]}" | grep -qF 'Jellyfin.Plugin.Tvdb.xml'
printf '%s\n' "${lines[@]}" | grep -qiF 'no place to write'
}
@test "a pin in the TVDB secret lands in the plugin config" {
# A subscriber-shaped key needs the account's PIN beside it; a licensed key
# does not, so the element appears only when the secret carries one.