docs(media): the TV library's own relayout, with the evidence chain it believes

One page owns relayout: the census commands, the order the planner believes
things in, the two refusals a dry run on the real library proved necessary, the
apply and rollback path, and the standing wart that a provider name can itself
be scene junk ("Black Clover 001", 92 episodes) — renaming that is Jellyfin's
job, not the mover's.

AGENTS.md

@@ -807,6 +807,8 @@ per-attempt page and no per-swap ledger anywhere else.
 | `frank-tts` — cloned-voice speech, Dankclaw | `serverAppConfigs-docs/Dankclaw/Frank TTS.md` |
 | Dankclaw kernel posture (`iommu=pt` since 2026-10-18) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
 | Deployment flow | `serverAppConfigs-docs/Architecture/Deployment Flow.md` |
+| Movie-library relayout (Jellyfin format, Dankserver) | `serverAppConfigs-docs/Dankserver/Media Relayout.md` |
+| SSD-backed metadata view (Jellyfin writes, Plex reads, Dankserver) | `serverAppConfigs-docs/Dankserver/Media Metadata View.md` |
 
 There is no page for a retired endpoint and no change diary. A payload swap
 appends one row to the live endpoint's **Eras** table; when the endpoint itself
@@ -848,5 +850,7 @@ history.
 | Writing a doc | **One owner per fact** — edit the endpoint/service page; everywhere else links. Never restate a number. |
 | Benchmarking (only when ordered) | **Stop and mask the service first**, bench, unmask, restart, verify health. |
 | Deleting a file | **Move it to trash** — `.AGENTTRASH/` (in-repo) or `~/.AGENTTRASH/YYYYMMDD-topic`. Never `rm`. |
+| Relayout the movie library | **Census → plan → read the exceptions → `scripts/media-apply.py`** — renames only, journal before every move; never hand-rename, never copy. |
+| Making media metadata visible to both media servers | **Bind `/var/lib/media-meta/view`** — the overlay `scripts/media-view.sh` installs (array below, NVMe above). Never write the array; never delete through the view. |
 | Tests exist | **Run them** — verify tests pass after every change. Do not commit failing tests. |
 | Real secrets | **Never commit** — host `.env` files; `.env.example` holds placeholders only. |

dankserver/compose.yaml

@@ -94,34 +94,12 @@ services:
       timeout: 10s
       retries: 3
 
-  # ═══════════════════════════════════════════════
-  # Jellyfin — media server (movies, TV, music)
-  # ═══════════════════════════════════════════════
-  jellyfin:
-    image: jellyfin/jellyfin:latest
-    container_name: jellyfin
-    restart: unless-stopped
-    ports:
-      - "8096:8096/tcp"
-      - "8920:8920/tcp"
-      - "7359:7359/udp"
-      - "1900:1900/udp"
-    environment:
-      - JELLYFIN_PublishedServerUrl=http://192.168.0.235:8096
-      - DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true
-    volumes:
-      - /opt/jellyfin/config:/config
-      - /opt/jellyfin/cache:/cache
-      - /mnt/entropy:/data:ro
-    devices:
-      - /dev/dri:/dev/dri
-    group_add:
-      - "992"
-    healthcheck:
-      test: ["CMD", "curl", "-f", "http://localhost:8096/health"]
-      interval: 30s
-      timeout: 10s
-      retries: 3
+  # Jellyfin left this file when it became a Quadlet: the container is owned by
+  # dankserver/jellyfin/quadlet/jellyfin.container, its data roots are under
+  # /home/dank/Containers/jellyfin, and its library arrives through the SSD
+  # overlay at /var/lib/media-meta/view (scripts/media-view.sh). The block that
+  # was here named /opt/jellyfin/* and a read-only array bind — a second owner for
+  # the container name `jellyfin`, wrong on every path. One owner per container.
 
   # ═══════════════════════════════════════════════
   # Calibre-Web — e-book web frontend

dankserver/host/systemd/media-meta-view.mount

@@ -0,0 +1,30 @@
+# /etc/systemd/system/var-lib-media\x2dmeta-view.mount
+# (repo copy: dankserver/host/systemd/media-meta-view.mount — systemd names a
+# mount unit after its ESCAPED path, so the host file carries the \x2d spelling)
+#
+# Installed by `scripts/media-view.sh install` — the repo copy is the source of
+# truth (fleet law: never edit host state by hand).
+#
+# An OverlayFS merge that shows the media library as one tree while splitting
+# where its bytes live: lowerdir is the RAID6 array (the source of truth, never
+# written), upperdir is a directory on the NVMe that receives every write. The
+# two media servers bind the result at /var/lib/media-meta/view, so Jellyfin's
+# sidecars and artwork are SSD bytes — which is what makes them fast to browse
+# and what keeps Plex's NFO agent off the spinners — while the media itself stays
+# on the array untouched.
+#
+# RequiresMountsFor is load-bearing: without the array mounted, lowerdir is an
+# empty directory and the view would present an empty library as truth.
+
+[Unit]
+Description=SSD-backed overlay view of the media library (array + NVMe metadata layer)
+RequiresMountsFor=/mnt/entropy
+
+[Mount]
+What=overlay
+Where=/var/lib/media-meta/view
+Type=overlay
+Options=lowerdir=/mnt/entropy,upperdir=/var/lib/media-meta/upper,workdir=/var/lib/media-meta/work
+
+[Install]
+WantedBy=local-fs.target

dankserver/jellyfin/quadlet/jellyfin.container

@@ -4,8 +4,9 @@
 # Unit name: jellyfin.service (podman 5.x drops the container- prefix)
 #
 # Data roots migrated from /opt/jellyfin to /home/dank/Containers/jellyfin at
-# cutover. All of /mnt/entropy is mounted read-only at /data: nothing automated
-# anywhere writes to the entropy volume. Every
+# cutover. The library is seen through /var/lib/media-meta/view, the SSD-backed
+# overlay view (scripts/media-view.sh): the array stays read-only and untouched,
+# while the sidecars this server writes land on the NVMe. Every
 # creation-time environment variable of the compose-era container is kept as
 # an Environment= line (zero drift); none of them are secrets, so this unit
 # needs no EnvironmentFile.
@@ -41,7 +42,12 @@ PublishPort=7359:7359/udp
 PublishPort=1900:1900/udp
 Volume=/home/dank/Containers/jellyfin/config:/config
 Volume=/home/dank/Containers/jellyfin/cache:/cache
-Volume=/mnt/entropy:/data:ro
+# The view, writable: Jellyfin is the metadata writer (movie.nfo / tvshow.nfo /
+# episode.nfo + artwork) and those bytes belong on the SSD. The condition is the
+# fail-closed guard — starting against a plain, unmounted view directory would let
+# a scan conclude the library is empty.
+Volume=/var/lib/media-meta/view:/data
+UnitConfigItem=ConditionPathIsMountPoint=/var/lib/media-meta/view
 AddDevice=/dev/dri
 GroupAdd=keep-groups
 HealthCmd=curl -f http://localhost:8096/health

dankserver/plex/quadlet/plex.container

@@ -11,7 +11,12 @@ Environment=PUID=0
 Environment=PGID=0
 Environment=TZ=America/Los_Angeles
 Volume=/var/lib/plexmediaserver:/config
-Volume=/mnt/entropy:/mnt/entropy:ro
+# The container path stays /mnt/entropy on purpose: every path Plex recorded in
+# its own database is written against it, so swapping the HOST source of this bind
+# to the overlay view re-points Plex at the sidecars without touching its DB.
+# Read-only: Plex consumes the NFO files Jellyfin writes and writes no metadata.
+Volume=/var/lib/media-meta/view:/mnt/entropy:ro
+UnitConfigItem=ConditionPathIsMountPoint=/var/lib/media-meta/view
 PublishPort=32400:32400
 
 [Service]

scripts/calib-push.py

@@ -22,9 +22,17 @@ Mechanics notes:
   prompts do not bias the Hessians (the corpus is drawn round-robin over
   source classes, so loss is class-neutral as long as failures are rare;
   a systematic 4xx is loud in the log and must be fixed, not ignored).
-- a round that seals no tap bytes is a FAILURE, not a retry: the engine
-  answering 200 while the tap directory is wrong, full or read-only is
-  how a corpus gets pushed four times for nothing.
+- a round that seals no tap bytes is NORMAL and continues: RADIANCE_CALIB_TOKENS
+  is a pooled floor and this campaign's corpus is 10.03M tokens against a 12M
+  floor, so round 1 seals nothing by construction. A tap that LOSES bytes fails
+  the run, and a run that never sees the engine's completion line exits non-zero.
+  The first version failed on a flat round, which aborted exactly one round too
+  early (2026-10-29).
+- before any prompt, the tap must carry the rig's own marker
+  (`--tap-marker`, written by run-qwen38-bf16.sh calib) naming THIS
+  container, and that container must be running. Measured 2026-10-29:
+  with neither check, four rounds of 10M tokens went into restored
+  production after the rig's boot had died, and recorded nothing.
 
 Usage: calib-push.py [--port 7770] [--parallel 64] [--container arbench]
        [--corpus /home/dank/corpora/out/calib.jsonl] [--max-rounds 4]
@@ -107,6 +115,61 @@ def tap_bytes(path):
     return total
 
 
+def round_verdict(before, after, done):
+    """What one round's tap bytes mean.
+
+    'done'   the engine logged that it sealed the tap — the run is over
+    'shrink' the tap LOST bytes: reset, pruned or replaced under us
+    'grew'   sealed bytes this round
+    'flat'   sealed nothing — which is legitimate: RADIANCE_CALIB_TOKENS is a
+             pooled floor, and a round that does not reach it has nothing to
+             write yet (the campaign's corpus is 10.03M tokens against a 12M
+             floor, so round 1 is always flat). Failing on it aborted the run
+             exactly one round too early.
+    """
+    if done:
+        return "done"
+    if after < before:
+        return "shrink"
+    if after == before:
+        return "flat"
+    return "grew"
+
+
+def require_rig(a):
+    """Refuse unless the rig that opened this tap is up and serving right now.
+
+    Three facts, in order: the runner wrote its marker into the tap dir, the
+    marker names the container this pusher targets, and that container is
+    running. A marker without a live container is a boot that died; a live
+    container without a marker is some other engine on the port — on 2026-10-29
+    it was restored production, which took four rounds of 10M tokens before the
+    old driver admitted it had recorded nothing.
+    """
+    marker = os.path.join(a.tap_dir, a.tap_marker)
+    if not os.path.isfile(marker):
+        qc.refuse(f"no rig marker at {marker}: this tap was not opened by "
+                  "run-qwen38-bf16.sh calib, so a push here would feed whatever "
+                  "answers the port and record nothing")
+    try:
+        lines = open(marker, encoding="utf-8", errors="replace").read().splitlines()
+    except OSError as exc:
+        qc.refuse(f"cannot read the rig marker {marker}: {exc}")
+    meta = dict(line.split("=", 1) for line in lines if "=" in line)
+    who = meta.get("container", "")
+    if who != a.container:
+        qc.refuse(f"the rig marker names container {who!r} and this pusher targets "
+                  f"{a.container!r} — wrong tap, or a stale marker")
+    try:
+        p = subprocess.run(["podman", "inspect", "-f", "{{.State.Running}}", a.container],
+                           capture_output=True, text=True, timeout=30)
+    except Exception as exc:
+        qc.refuse(f"cannot inspect the rig container {a.container}: {exc}")
+    if p.returncode != 0 or p.stdout.strip() != "true":
+        qc.refuse(f"the rig container {a.container} is not running — whatever is "
+                  "answering the port, it is not the calibration rig")
+
+
 def parse_args():
     ap = argparse.ArgumentParser()
     ap.add_argument("--port", type=int, default=7770)
@@ -114,6 +177,8 @@ def parse_args():
     ap.add_argument("--container", default="arbench")
     ap.add_argument("--model", default="frank-archon")
     ap.add_argument("--tap-dir", default="/home/dank/campaign/qwen38-w5nl64/calib")
+    ap.add_argument("--tap-marker", default=".rig-boot",
+                    help="the file run-qwen38-bf16.sh calib writes into the tap dir; without it the pusher refuses, because an unmarked tap is not a rig")
     ap.add_argument("--corpus", default="/home/dank/corpora/out/calib.jsonl")
     ap.add_argument("--max-rounds", type=int, default=4)
     return ap.parse_args()
@@ -136,6 +201,7 @@ def main():
     a = parse_args()
     url = f"http://127.0.0.1:{a.port}/v1/completions"
     prompts = load_prompts(a)
+    require_rig(a)
     chars = sum(len(p) for p in prompts)
     print(f"corpus: {len(prompts)} prompts, ~{chars // 4} tokens (chars/4)",
           flush=True)
@@ -156,11 +222,15 @@ def main():
         if tap_done(a.container):
             break
         after = tap_bytes(a.tap_dir)
-        if after <= before:
-            print(f"FAIL: the tap at {a.tap_dir} sealed nothing this round "
-                  f"({before} B before, {after} B after) — the engine is serving "
-                  "without recording (wrong dir, full disk, read-only mount)", flush=True)
+        verdict = round_verdict(before, after, False)
+        if verdict == "shrink":
+            print(f"FAIL: the tap at {a.tap_dir} LOST bytes this round "
+                  f"({before} B before, {after} B after) — it was reset or replaced "
+                  "under us", flush=True)
             sys.exit(1)
+        if verdict == "flat":
+            print(f"round {rnd}: the tap sealed nothing yet (byte floor not reached; "
+                  f"{before} B) — continuing", flush=True)
     else:
         print("max rounds exhausted, tap not done", flush=True)
         sys.exit(1)

scripts/convert-qwen38-w5nl64.sh

@@ -47,7 +47,12 @@
 set -u -o pipefail
 
 SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-IMAGE=${IMAGE:-stilldeadcode/radiance@sha256:d590e300234bf6d2f942a0b8d456d183153fd808f28820e6c2fa69c9389178ed}
+# THE TOOLCHAIN PIN. `:latest` on 2026-10-30 is v1.14.0 (revision 4a61d7296bc5),
+# the same image the tap producer boots: a tap must be converted by the line that
+# is current, and rad-convert's `gptq:calib=<dir>` input contract is version-stable
+# (RADGRAM2 + gram.r<rank>.<stem>.bin), so nothing else about the recipe moves with
+# it. Production stays on its own pin — this is the campaign's toolchain.
+IMAGE=${IMAGE:-stilldeadcode/radiance@sha256:69add58f7de1ed2bca2b1e20a4cc75717557df3f73569a178eca5aafe8c44093}
 CKPT=${CKPT:-/home/dank/models/Qwen3.8-Flash-Next}
 CAMPAIGN=${CAMPAIGN:-/home/dank/campaign/qwen38-w5nl64}
 RECIPE=${RECIPE:-$SELF/recipes/qwen4exp-qwen38-w5nl64-bf16trunk.recipe}

scripts/deploy-plan.py

@@ -155,13 +155,14 @@ DANKCLAW_COMPOSE_SERVICES = set()
 # Quadlet unit now — the last compose services were the hermes trio, whose units
 # have owned their containers since 2026-09-12 and whose compose blocks were
 # dead weight that could only create a duplicate container.
+# jellyfin left the Dankserver file on 2026-10-30 for the same reason: its Quadlet
+# has owned the container since the cutover, so a compose edit must not restart it.
 
 DANKSERVER_COMPOSE_SERVICES = {
     "mariadb",
     "redis",
     "romm",
     "copyparty",
-    "jellyfin",
     "calibre-web",
 }
 COMPOSE_NAME_MAP = {"mariadb": "romm-mariadb", "redis": "romm-redis"}
@@ -421,6 +422,11 @@ def services_for_path(path: str, target: str) -> set[str]:
                 "tautulli": {"tautulli"},
             }
             _add(services, *quadlet_services.get(parts[1], set()))
+        if len(parts) >= 2 and parts[1] == "host":
+            # dankserver/host/systemd/media-meta-view.mount is the overlay view the
+            # media servers bind (scripts/media-view.sh installs it on the host),
+            # so editing it restarts jellyfin and plex and nothing else.
+            _add(services, "jellyfin", "plex")
         if len(parts) >= 2 and parts[1] == "observability":
             # Dankserver's node-exporter became a Quadlet unit on 2026-09-16
             # (serverAppConfigs-docs/Infrastructure/Observability.md): its
@@ -466,6 +472,15 @@ def services_for_path(path: str, target: str) -> set[str]:
             # that service (target_services then keeps it only on the host being
             # planned). Editing the collector restarts the running exporter.
             _add(services, "gpu-exporter")
+        if name == "media-view.sh":
+            # The installer is what creates, mounts and reaps the view both media
+            # servers read, so a change to it is a restart of that pair.
+            _add(services, "jellyfin", "plex")
+        if name == "jellyfin-providers.sh":
+            # The deployer runs this before jellyfin restarts, so a change to
+            # the installer is a change to the provider state Jellyfin loads at
+            # start-up: it selects jellyfin and nothing else.
+            _add(services, "jellyfin")
         if name in {"frank-zealot.env", "frank-zealot.service", "serve-frank-zealot.sh"}:
             # llama.cpp frank-zealot (2026-09-16) keeps its unit, env and serve
             # files here; each one selects frank-zealot only. The ik_llama.cpp

scripts/deploy-remote.sh

@@ -1202,7 +1202,42 @@ deploy_romm_stack() {
     systemctl --user restart "$service.service"
 }
 
+ensure_media_view() {
+    # Both media servers read their library through the SSD-backed overlay view
+    # (scripts/media-view.sh, dankserver/host/systemd/media-meta-view.mount): the
+    # array below stays read-only and every sidecar Jellyfin writes lands on the
+    # NVMe.
+    # Raise it BEFORE either container starts — a start against a bare view
+    # directory would let a scan clear the library database. The units also carry
+    # ConditionPathIsMountPoint, so a missing view skips the start rather than
+    # damaging anything. A relayout must be followed by `media-view.sh reap`
+    # (see the Media Relayout page), which this is not.
+    run_privileged bash scripts/media-view.sh install
+    bash scripts/media-view.sh verify
+}
+
+ensure_provider_keys() {
+    # Jellyfin's metadata-provider keys live in the container-owned plugin
+    # configuration under the data root, never in git; scripts/jellyfin-providers.sh
+    # is what places them, reading the per-service host env file (the fleet
+    # convention: grafana, caddy, pihole, copyparty, romm, seerr) that holds
+    # TMDB_API_KEY= and TVDB_API_KEY=, mode 600.
+    # Missing keys mean a keyless provider — a degraded library, not a broken
+    # deploy — so this warns rather than failing the restart. But when the keys
+    # ARE placed it must happen before the restart: the plugin and its keys are
+    # read at start-up, so a restart first would boot the old state.
+    local keys=/home/dank/Containers/jellyfin/provider-keys
+    if [ -f "$keys" ]; then
+        bash scripts/jellyfin-providers.sh install
+    else
+        echo "WARN: $keys is absent - provider keys NOT placed"
+        echo "WARN: create it (TMDB_API_KEY=, TVDB_API_KEY=, chmod 600) and re-deploy jellyfin"
+    fi
+}
+
 deploy_jellyfin() {
+    ensure_media_view
+    ensure_provider_keys
     cp "dankserver/jellyfin/quadlet/jellyfin.container" "$quadlet_unit_dir/"
     systemctl --user daemon-reload
     if ! systemctl --user is-active -q jellyfin.service; then
@@ -1290,6 +1325,7 @@ deploy_copyparty() {
 }
 
 deploy_plex() {
+    ensure_media_view
     cp "dankserver/plex/quadlet/plex.container" "$quadlet_unit_dir/"
     systemctl --user daemon-reload
     if ! systemctl --user is-active -q plex.service; then

scripts/jellyfin-providers.sh

@@ -0,0 +1,391 @@
+#!/usr/bin/env bash
+# Metadata-provider keys for Jellyfin on Dankserver, placed from the service's
+# host env file with the plugin artifact pinned by content hash.
+#
+# Jellyfin 12 ships TMDb, OMDb, StudioImages and MusicBrainz as CORE plugins:
+# they have no DLL, only a configuration file under the data root, and that
+# file is created by the running container. So the fleet's provider state lives
+# in two places the repo does not track —
+#
+#   plugins/configurations/Jellyfin.Plugin.Tmdb.xml   <TmdbApiKey /> (empty = keyless)
+#   plugins/configurations/Jellyfin.Plugin.Tvdb.xml   <ApiKey />     (needs the plugin)
+#
+# and both are owned by the container user. This tool is the only thing that
+# writes them: it is declarative, idempotent, and it never prints a key, so it
+# is safe to run while someone watches.
+#
+# Keys never enter git and never enter argv (a value in argv is world-readable
+# through /proc/<pid>/cmdline). They live in the per-service host env file that
+# is the fleet's convention for container credentials (grafana, caddy, pihole,
+# copyparty, romm, seerr), mode 600:
+#
+#   /home/dank/Containers/jellyfin/provider-keys
+#       TMDB_API_KEY=<key>
+#       TVDB_API_KEY=<key>
+#       TVDB_PIN=<pin>        # optional: only a subscriber-shaped TVDB key
+#                             # needs one; a licensed key does not
+#
+# TVDB itself is not core in 12 (the metadata provider was cut in 10.9), so it
+# is installed from the official catalog, pinned three ways: sha256 of the
+# artifact, the plugin's own guid, and its version. A matching sha proves only
+# that the bytes are the bytes fetched — the guid check is what says it is the
+# official plugin, the version check is what says it is the one this file was
+# reviewed against.
+#
+# Subcommands
+#   plan      print the resolved shape and every pending change, touch nothing
+#   install   fetch+verify the plugin, then write the keys (backing up first)
+#   verify    prove the plugin and both keys are actually in place
+#
+# Nothing here restarts Jellyfin: the deployer owns that
+# (`scripts/deploy.sh dankserver --service jellyfin`), and the plugin is read at
+# start-up, so a key written while the server runs is not live until then.
+set -euo pipefail
+
+# Jellyfin's data root on Dankserver (the rootless quadlet's config volume).
+JF_ROOT="${JF_ROOT:-/home/dank/Containers/jellyfin/config}"
+PROV_KEYS="${PROV_KEYS:-/home/dank/Containers/jellyfin/provider-keys}"
+TRASH_DIR="${TRASH_DIR:-$HOME/.AGENTTRASH/$(date +%Y%m%d)-jellyfin-providers}"
+
+PLUGINS="$JF_ROOT/plugins"
+CONF="$PLUGINS/configurations"
+TMDB_XML="$CONF/Jellyfin.Plugin.Tmdb.xml"
+TVDB_XML="$CONF/Jellyfin.Plugin.Tvdb.xml"
+
+# repo.jellyfin.org catalog entry for TheTVDB (owner: jellyfin), version 24.0.0.0,
+# targetAbi 12.0.0.0 — published 2026-09-09, verified against Jellyfin 12.4.0.
+TVDB_PLUGIN_URL="${TVDB_PLUGIN_URL:-https://repo.jellyfin.org/files/plugin/thetvdb/thetvdb_24.0.0.0.zip}"
+TVDB_PLUGIN_SHA256="${TVDB_PLUGIN_SHA256:-6302fba3857bdde8d186dd099fac09ca43247041f531a528a2c788add70a5319}"
+TVDB_PLUGIN_GUID="${TVDB_PLUGIN_GUID:-a677c0da-fac5-4cde-941a-7134223f14c8}"
+TVDB_PLUGIN_VERSION="${TVDB_PLUGIN_VERSION:-24.0.0.0}"
+PLUGIN_DIR="thetvdb_${TVDB_PLUGIN_VERSION}"
+TVDB_DLL="Jellyfin.Plugin.Tvdb.dll"
+
+die() {
+    printf 'jellyfin-providers: %s\n' "$*" >&2
+    exit 1
+}
+
+usage() {
+    printf 'usage: %s plan|install|verify\n' "$0" >&2
+    exit 2
+}
+
+# check_keys — refuse anything that is not a readable private file. An open
+# mode is an error, not a warning: a provider key readable by every user on the
+# box is the failure this check exists to catch.
+check_keys() {
+    [ -f "$PROV_KEYS" ] ||
+        die "$PROV_KEYS not found (create it with TMDB_API_KEY= and TVDB_API_KEY=, then chmod 600)"
+    local mode
+    mode="$(stat -c '%a' "$PROV_KEYS")"
+    case "$mode" in
+    600 | 400) ;;
+    *) die "$PROV_KEYS is mode $mode; provider keys must be 600" ;;
+    esac
+}
+
+# keys_value <VAR> — the value of one variable in the env file, or nothing.
+keys_value() {
+    awk -v k="$1" -F= '$1 == k { sub(/^[^=]*=/, ""); print; exit }' "$PROV_KEYS"
+}
+
+# keys_len <VAR> — the length, and a refusal naming the variable when absent.
+# The value itself is never printed.
+keys_len() {
+    local v
+    v="$(keys_value "$1")"
+    [ -n "$v" ] || die "$PROV_KEYS has no $1= line"
+    printf '%s' "${#v}"
+}
+
+# xml_value <file> <element> — the element's text, empty when it is absent or
+# self-closing.
+xml_value() {
+    python3 - "$1" "$2" <<'PYEOF'
+import re, sys
+src = open(sys.argv[1], encoding="utf-8").read()
+m = re.search(r"<%s(?:\s*/>|>(.*?)</%s>)" % (sys.argv[2], sys.argv[2]), src, re.S)
+print(m.group(1).strip() if (m and m.group(1)) else "")
+PYEOF
+}
+
+# xml_secret <file> <element> <var> — set <element> to that variable from the
+# env file. The python process reads the env file itself, so the value never
+# appears in argv.
+xml_secret() {
+    python3 - "$1" "$2" "$3" "$PROV_KEYS" <<'PYEOF'
+import os, re, sys, tempfile
+from xml.sax.saxutils import escape
+
+xml_path, elem, var, keys_path = sys.argv[1:5]
+val = ""
+for line in open(keys_path, encoding="utf-8"):
+    key, sep, rest = line.partition("=")
+    if sep and key.strip() == var:
+        val = rest.strip()
+        break
+if not val:
+    raise SystemExit(f"{keys_path} has no {var}= line")
+
+src = open(xml_path, encoding="utf-8").read()
+pat = re.compile(r"([ \t]*)<%s(?:\s*/>|>.*?</%s>)" % (elem, elem), re.S)
+text = escape(val)
+if pat.search(src):
+    out = pat.sub(lambda m: f"{m.group(1)}<{elem}>{text}</{elem}>", src, count=1)
+else:
+    out = src.replace("</Configuration>", f"  <{elem}>{text}</{elem}>\n</Configuration>", 1)
+if out == src:
+    raise SystemExit(f"{xml_path}: no place to write <{elem}>")
+
+d, base = os.path.split(xml_path)
+fd, tmp = tempfile.mkstemp(prefix=base + ".", dir=d)
+with os.fdopen(fd, "w", encoding="utf-8") as fh:
+    fh.write(out)
+os.chmod(tmp, os.stat(xml_path).st_mode & 0o777)
+os.replace(tmp, xml_path)
+PYEOF
+}
+
+# plugin_meta <zip> — "guid version" out of the artifact's own meta.json.
+plugin_meta() {
+    python3 - "$1" <<'PYEOF'
+import json, sys, zipfile
+meta = json.loads(zipfile.ZipFile(sys.argv[1]).read("meta.json").decode("utf-8"))
+print(meta.get("guid", ""), meta.get("version", ""))
+PYEOF
+}
+
+check_root() {
+    # Never fabricate a data root: creating plugins/ beside a typo'd path would
+    # install a plugin Jellyfin never loads and hide the operator's mistake.
+    [ -d "$CONF" ] || die "no Jellyfin configuration directory at $CONF (is JF_ROOT right?)"
+}
+
+check_keys_report() {
+    check_keys
+    # A refusal inside a command substitution used as a printf argument would
+    # be swallowed, so the guard runs as its own statement.
+    keys_len TMDB_API_KEY >/dev/null
+    keys_len TVDB_API_KEY >/dev/null
+    printf '= keys read from %s (mode %s)\n' "$PROV_KEYS" "$(stat -c '%a' "$PROV_KEYS")"
+}
+
+# The plugin's state, as one line in the shared vocabulary.
+plugin_state() {
+    if [ -d "$PLUGINS/$PLUGIN_DIR" ]; then
+        printf '= %s already installed\n' "$PLUGIN_DIR"
+        return 0
+    fi
+    local other
+    other="$(find "$PLUGINS" -maxdepth 1 -type d -name 'thetvdb_*' ! -name "$PLUGIN_DIR" -printf '%f\n' 2>/dev/null | sort | tr '\n' ' ')"
+    if [ -n "$other" ]; then
+        printf '+ install %s (a different version is present: %s)\n' "$PLUGIN_DIR" "$other"
+        return 0
+    fi
+    printf '+ install %s from %s\n' "$PLUGIN_DIR" "${TVDB_PLUGIN_URL#*://}"
+}
+
+tmdb_state() {
+    if [ ! -f "$TMDB_XML" ]; then
+        printf '! %s is missing — save the TMDb plugin settings once in the UI, then re-run\n' "$TMDB_XML" >&2
+        return 1
+    fi
+    local now want
+    now="$(xml_value "$TMDB_XML" TmdbApiKey)"
+    want="$(keys_value TMDB_API_KEY)"
+    if [ -n "$now" ] && [ "$now" = "$want" ]; then
+        printf '= TmdbApiKey already set (%s chars)\n' "${#now}"
+    elif [ -n "$now" ]; then
+        printf '+ replace the TmdbApiKey in Jellyfin.Plugin.Tmdb.xml\n'
+    else
+        printf '+ set TmdbApiKey in Jellyfin.Plugin.Tmdb.xml\n'
+    fi
+}
+
+tvdb_state() {
+    local now
+    if [ -f "$TVDB_XML" ]; then
+        now="$(xml_value "$TVDB_XML" ApiKey)"
+        if [ -n "$now" ]; then
+            printf '= Jellyfin.Plugin.Tvdb.xml already has an ApiKey (kept)\n'
+        else
+            printf '+ fill the empty ApiKey in Jellyfin.Plugin.Tvdb.xml\n'
+        fi
+    else
+        printf '+ create Jellyfin.Plugin.Tvdb.xml with ApiKey\n'
+    fi
+}
+
+write_plugin() {
+    local tmp guid version other
+    other="$(find "$PLUGINS" -maxdepth 1 -type d -name 'thetvdb_*' ! -name "$PLUGIN_DIR" -printf '%f\n' 2>/dev/null | sort | tr '\n' ' ')"
+    [ -z "$other" ] ||
+        die "another TheTVDB version is already installed ($other); move it to the trash first — two copies of one plugin means two loads"
+
+    tmp="$(mktemp "${TMPDIR:-/tmp}/thetvdb.XXXXXX.zip")"
+    # The pins are checked before anything is unpacked, so a bad artifact never
+    # reaches the plugins directory.
+    curl -fsSL --max-time 300 -o "$tmp" "$TVDB_PLUGIN_URL" ||
+        { rm -f "$tmp"; die "could not fetch $TVDB_PLUGIN_URL"; }
+    local got
+    got="$(sha256sum "$tmp" | cut -d' ' -f1)"
+    if [ "$got" != "$TVDB_PLUGIN_SHA256" ]; then
+        rm -f "$tmp"
+        die "artifact sha256 is $got, pinned $TVDB_PLUGIN_SHA256 — refusing to install"
+    fi
+    read -r guid version <<<"$(plugin_meta "$tmp")"
+    if [ "$guid" != "$TVDB_PLUGIN_GUID" ]; then
+        rm -f "$tmp"
+        die "artifact guid is $guid, expected the official $TVDB_PLUGIN_GUID — refusing to install"
+    fi
+    if [ "$version" != "$TVDB_PLUGIN_VERSION" ]; then
+        rm -f "$tmp"
+        die "artifact version is $version, pinned $TVDB_PLUGIN_VERSION — refusing to install"
+    fi
+    mkdir -p "$PLUGINS/$PLUGIN_DIR"
+    python3 -m zipfile -e "$tmp" "$PLUGINS/$PLUGIN_DIR/"
+    rm -f "$tmp"
+    [ -f "$PLUGINS/$PLUGIN_DIR/$TVDB_DLL" ] ||
+        die "$PLUGIN_DIR has no $TVDB_DLL — the artifact does not match this plugin"
+    printf '+ installed %s\n' "$PLUGIN_DIR"
+}
+
+backup_to_trash() {
+    local f="$1"
+    mkdir -p "$TRASH_DIR"
+    local dest="$TRASH_DIR/$(basename "$f").$(date +%H%M%S)"
+    cp -p "$f" "$dest"
+    printf '+ previous copy kept at %s\n' "$dest"
+}
+
+cmd_write_keys() {
+    local now want
+    KEYS_CHANGED=0
+    # TMDb: the running container owns this file and every setting in it. Only
+    # the key element is touched, so country, language and adult settings
+    # survive the edit.
+    [ -f "$TMDB_XML" ] ||
+        die "$TMDB_XML is missing — save the TMDb plugin settings once in the UI, then re-run"
+    now="$(xml_value "$TMDB_XML" TmdbApiKey)"
+    keys_len TMDB_API_KEY >/dev/null
+    want="$(keys_value TMDB_API_KEY)"
+    if [ "$now" = "$want" ]; then
+        printf '= TmdbApiKey already set (%s chars)\n' "${#now}"
+    else
+        backup_to_trash "$TMDB_XML"
+        xml_secret "$TMDB_XML" TmdbApiKey TMDB_API_KEY
+        printf '+ set TmdbApiKey (%s chars)\n' "${#want}"
+        KEYS_CHANGED=1
+    fi
+
+    # TVDB: never overwrite a config the operator saved through the UI.
+    if [ -f "$TVDB_XML" ]; then
+        now="$(xml_value "$TVDB_XML" ApiKey)"
+        if [ -n "$now" ]; then
+            printf '= Jellyfin.Plugin.Tvdb.xml already has an ApiKey (kept)\n'
+        else
+            keys_len TVDB_API_KEY >/dev/null
+            backup_to_trash "$TVDB_XML"
+            xml_secret "$TVDB_XML" ApiKey TVDB_API_KEY
+            if [ -n "$(keys_value TVDB_PIN)" ] && [ -z "$(xml_value "$TVDB_XML" Pin)" ]; then
+                xml_secret "$TVDB_XML" Pin TVDB_PIN
+            fi
+            printf '+ filled the ApiKey in Jellyfin.Plugin.Tvdb.xml\n'
+            KEYS_CHANGED=1
+        fi
+    else
+        keys_len TVDB_API_KEY >/dev/null
+        cat >"$TVDB_XML" <<XML
+<?xml version="1.0" encoding="utf-8"?>
+<Configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
+  <ApiKey />
+</Configuration>
+XML
+        xml_secret "$TVDB_XML" ApiKey TVDB_API_KEY
+        if [ -n "$(keys_value TVDB_PIN)" ]; then
+            xml_secret "$TVDB_XML" Pin TVDB_PIN
+        fi
+        printf '+ created Jellyfin.Plugin.Tvdb.xml with ApiKey\n'
+        KEYS_CHANGED=1
+    fi
+}
+
+cmd_plan() {
+    check_root
+    check_keys_report
+    printf 'root  %s\n' "$JF_ROOT"
+    plugin_state
+    tmdb_state
+    tvdb_state
+}
+
+cmd_install() {
+    check_root
+    check_keys_report
+    KEYS_CHANGED=0
+    if [ ! -d "$PLUGINS/$PLUGIN_DIR" ]; then
+        write_plugin
+        KEYS_CHANGED=1
+    else
+        printf '= %s already installed\n' "$PLUGIN_DIR"
+    fi
+    cmd_write_keys
+    if [ "$KEYS_CHANGED" -eq 1 ]; then
+        # Nothing is restarted here on purpose: this host may be playing media
+        # right now, and both the plugin and its keys are read at start-up.
+        printf 'next: scripts/deploy.sh dankserver --service jellyfin (the plugin and keys load at start-up)\n'
+    else
+        printf '= no change\n'
+    fi
+}
+
+cmd_verify() {
+    check_root
+    local bad=0
+    if [ ! -d "$PLUGINS/$PLUGIN_DIR" ]; then
+        printf 'MISSING plugin directory %s (thetvdb not installed)\n' "$PLUGIN_DIR" >&2
+        bad=1
+    else
+        local dlls abi
+        dlls="$(find "$PLUGINS/$PLUGIN_DIR" -maxdepth 1 -name '*.dll' | wc -l)"
+        [ "$dlls" -ge 1 ] || {
+            printf 'MISSING %s inside %s\n' "$TVDB_DLL" "$PLUGIN_DIR" >&2
+            bad=1
+        }
+        abi="$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1])).get("targetAbi","?"))' \
+            "$PLUGINS/$PLUGIN_DIR/meta.json" 2>/dev/null || echo '?')"
+        printf 'plugin %s: %s dll(s), targetAbi %s\n' "$PLUGIN_DIR" "$dlls" "$abi"
+    fi
+    local pair elem n
+    for pair in "$TMDB_XML|TmdbApiKey" "$TVDB_XML|ApiKey"; do
+        elem="${pair##*|}"
+        if [ ! -f "${pair%%|*}" ]; then
+            printf 'MISSING %s\n' "${pair%%|*}" >&2
+            bad=1
+            continue
+        fi
+        if ! python3 -c 'import sys, xml.dom.minidom; xml.dom.minidom.parse(sys.argv[1])' "${pair%%|*}"; then
+            printf 'UNPARSEABLE %s\n' "${pair%%|*}" >&2
+            bad=1
+            continue
+        fi
+        n="$(xml_value "${pair%%|*}" "$elem")"
+        if [ -z "$n" ]; then
+            printf 'MISSING key: %s has no %s value\n' "$(basename "${pair%%|*}")" "$elem" >&2
+            bad=1
+        else
+            printf '%s %s chars\n' "$elem" "${#n}"
+        fi
+    done
+    [ "$bad" -eq 0 ] || return 1
+    printf 'providers configured\n'
+}
+
+cmd="${1:-}"
+case "$cmd" in
+plan) cmd_plan ;;
+install) cmd_install ;;
+verify) cmd_verify ;;
+*) usage ;;
+esac

scripts/kld-gate-qwen38.sh

@@ -1,44 +1,53 @@
 #!/usr/bin/env bash
 # kld-gate-qwen38.sh — RUNS ON DANKSERVER (or anywhere the reports are). The campaign's
-# acceptance gate: three KLD reports against ONE reference recording, one verdict.
+# acceptance gate: one verdict from the KLD reports of the containers that ride one reference
+# recording.
 #
 # The reference is the BF16 container built from the vendor checkpoint (run-qwen38-bf16.sh
 # kldref, in the same drained window that records the calibration grams). Every scored
 # container is measured against that one recording, so the KL scale runs from 0 = the
-# checkpoint itself downward in faithfulness. Three containers ride it:
+# checkpoint itself downward in faithfulness.
 #
-#   fp8.json    the published FP8-native release   — the achievable end of the interval
-#   iq4r.json   the payload frank-archon serves    — the bar to beat
-#   cand.json   the arm under test                 — the subject
+# TWO MODES, because the disk decides how many anchors can exist at once:
 #
-# The FP8 number is what makes the interval absolute instead of relative: "half the gap"
-# is half of the distance between the 4-bit payload and the near-lossless release, not half
-# of an invented distance to the checkpoint. That is also why the FP8 payload is FETCHED and
-# scored, never converted: it is a second opinion about what this checkpoint tolerates, not a
-# step of the campaign. It lands LAST (disk sequencing) — by then the 335 GiB bf16 container
-# is gone, so its 172.8 GiB fits.
+#   gate IQ4R.json CAND.json                    candidate vs the served payload
+#   gate FP8.json IQ4R.json CAND.json           the same, inside the absolute interval
 #
-# The law, in order of severity, on the mean KL of one slice (--slice all | a corpus source):
-#   1. KL(fp8) < KL(iq4r)                 the anchors are ordered. If the near-lossless
-#                                         release does NOT sit closer to the checkpoint than
-#                                         the 4-bit payload, the corpus, the tokenizer or the
-#                                         boot shape is wrong and no arm number means anything.
-#   2. KL(fp8) < KL(cand) < KL(iq4r)      the candidate lies strictly inside the interval —
-#                                         outside it, something is mis-measured, not better
-#   3. KL(cand) <= KL(iq4r) - R x gap     it closes at least R (--recover, default 0.5) of the
-#                                         gap = KL(iq4r) - KL(fp8): at least half the way from
-#                                         what is served toward the release (agreed 2026-10-27)
-#   4. top-1(cand) >= top-1(iq4r)         the agreement the harness keys on does not fall
+# iq4r.json is the payload frank-archon serves — the bar to beat. When the FP8-native release
+# (172.8 GiB, fetched last because of the peak-disk law) is on disk, pass its report first:
+# "half the gap" is then half of the distance between the 4-bit payload and the near-lossless
+# release, not half of an invented distance to the checkpoint. Without it the gate still
+# answers the campaign's question — does the candidate beat what is served — and says so.
+#
+# The law, on the summary of one slice (--slice all | a corpus source):
+#
+#   candidate vs served (2 reports)
+#   1. mean KL(cand) < mean KL(iq4r)         the served payload is beaten on the corpus
+#   2. p99.9(cand) <= p99.9(iq4r)            and the tail is no worse — a mean can hide a few
+#                                            positions taking the whole hit
+#   3. top-1(cand) >= top-1(iq4r)            the agreement the harness keys on does not fall
+#   4. every source slice passes 1-3         a win bought by regressing one distribution
+#                                            (code, prose, tools) is not a win
+#
+#   inside the interval (3 reports), laws 1-4 above (1-3 renumbered to the interval and the
+#   recovery fraction, as agreed 2026-10-27) plus:
+#   5. p99.9(cand) <= p99.9(iq4r)            the tail law
+#   6. every source slice passes            the slice law
 #
 # Booting the containers is NOT this script's job: `run-qwen38-bf16.sh score RAD OUT.json`
 # writes the reports and owns the single engine argv (same pin, --tp 2, --kv-cache-dtype fp8,
-# same step size, --deterministic, no speculation) that makes the three numbers comparable.
+# same step size, --deterministic, no speculation) that makes the numbers comparable.
 # This script is arithmetic, so it re-runs against old reports and is testable without a GPU.
 #
-# Usage: kld-gate-qwen38.sh FP8.json IQ4R.json CAND.json [--slice NAME] [--recover F]
+# Usage: kld-gate-qwen38.sh IQ4R.json CAND.json [--slice NAME]
+#        kld-gate-qwen38.sh FP8.json IQ4R.json CAND.json [--slice NAME] [--recover F]
 set -uo pipefail
 
-[ "$#" -ge 3 ] || { echo "usage: kld-gate-qwen38.sh FP8.json IQ4R.json CAND.json [--slice NAME] [--recover F]" >&2; exit 2; }
+if [ "$#" -lt 2 ]; then
+    echo "usage: kld-gate-qwen38.sh IQ4R.json CAND.json [--slice NAME]" >&2
+    echo "       kld-gate-qwen38.sh FP8.json IQ4R.json CAND.json [--slice NAME] [--recover F]" >&2
+    exit 2
+fi
 python3 - "$@" <<'PY'
 import json, sys
 
@@ -52,13 +61,15 @@ while i < len(a):
     else:
         files.append(a[i])
         i += 1
-if len(files) != 3:
-    sys.exit("usage: kld-gate-qwen38.sh FP8.json IQ4R.json CAND.json [--slice NAME] [--recover F]")
+if len(files) not in (2, 3):
+    sys.exit("usage: kld-gate-qwen38.sh IQ4R.json CAND.json [--slice NAME]\n"
+             "       kld-gate-qwen38.sh FP8.json IQ4R.json CAND.json [--slice NAME] [--recover F]")
 o = dict(zip(opts[0::2], opts[1::2]))
 slice_ = o.get("--slice", "all")
 recover = float(o.get("--recover", 0.5))
 if not 0.0 < recover <= 1.0:
     sys.exit(f"--recover must be in (0,1], got {recover}")
+interval = len(files) == 3
 
 
 def die(msg):
@@ -66,11 +77,15 @@ def die(msg):
     raise SystemExit(2)
 
 
-def read(path):
+def load(path):
     try:
-        d = json.load(open(path))
+        return json.load(open(path))
     except Exception as e:
         die(f"cannot read the KLD report {path}: {e}")
+
+
+def summary(path, d):
+    """The slice this gate grades: 'all', or one corpus source."""
     if slice_ == "all":
         g = d.get("all")
     else:
@@ -80,23 +95,25 @@ def read(path):
                 f"{', '.join(sorted(d.get('by_source') or {})) or '(none)'}")
     if g is None or "kld" not in g or "top1_agreement" not in g:
         die(f"{path} has no {slice_!r} summary — wrong report version?")
-    return d, g
+    return g
 
 
+docs = {label: load(path) for label, path in
+        zip(("fp8", "iq4r", "cand") if interval else ("iq4r", "cand"), files)}
 rows = []
-for label, path in zip(("fp8", "iq4r", "cand"), files):
-    d, g = read(path)
-    rows.append((label, path, d.get("model", "?"), g["kld"]["mean"], g["kld"]["p99_9"],
-                 g["top1_agreement"], g["positions"]))
+for label, path in zip(("fp8", "iq4r", "cand") if interval else ("iq4r", "cand"), files):
+    g = summary(path, docs[label])
+    rows.append((label, path, docs[label].get("model", "?"), g["kld"]["mean"],
+                 g["kld"]["p99_9"], g["top1_agreement"], g["positions"]))
 
-print(f"reference: BF16 container; slice: {slice_}")
+print(f"reference: BF16 container; slice: {slice_}; "
+      + ("inside the FP8 interval" if interval else "candidate vs served (no FP8 anchor)"))
 print(f"{'':6} {'model':38} {'KL mean':>10} {'KL p99.9':>10} {'top-1':>8}  positions")
 for label, path, model, kl, p999, top1, n in rows:
     print(f"{label:6} {str(model)[:38]:38} {kl:10.6f} {p999:10.4f} {100*top1:7.2f}%  {n}")
-r = {label: (kl, top1) for label, _p, _m, kl, _q, top1, _n in rows}
-kl_fp8, kl_iq4r, kl_cand = r["fp8"][0], r["iq4r"][0], r["cand"][0]
-t_iq4r, t_cand = r["iq4r"][1], r["cand"][1]
-gap = kl_iq4r - kl_fp8
+r = {label: (kl, p999, top1) for label, _p, _m, kl, p999, top1, _n in rows}
+kl_cand, p_cand, t_cand = r["cand"]
+kl_iq4r, p_iq4r, t_iq4r = r["iq4r"]
 ok = True
 
 
@@ -106,17 +123,64 @@ def law(n, text, passed):
     print(f"  [{'PASS' if passed else 'FAIL'}] {n}. {text}")
 
 
-law(1, f"anchors ordered: fp8 {kl_fp8:.6f} < iq4r {kl_iq4r:.6f} (gap {gap:.6f})",
-    gap > 0)
-law(2, f"candidate inside the interval: {kl_fp8:.6f} < {kl_cand:.6f} < {kl_iq4r:.6f}",
-    kl_fp8 < kl_cand < kl_iq4r)
-need = kl_iq4r - recover * gap
-law(3, f"candidate closes {100*max((kl_iq4r - kl_cand)/gap, 0):.1f}% of the "
-       f"{gap:.6f} gap (needs {100*recover:.0f}%, i.e. KL <= {need:.6f})"
-       if gap > 0 else "gap is not positive; cannot score recovery",
-    gap > 0 and kl_cand <= need)
-law(4, f"candidate top-1 {100*t_cand:.2f}% >= served iq4r top-1 {100*t_iq4r:.2f}%",
-    t_cand >= t_iq4r)
+def tail_law(n):
+    law(n, f"candidate KL p99.9 {p_cand:.4f} <= served iq4r {p_iq4r:.4f}",
+        p_cand <= p_iq4r)
+
+
+# --- the slice law: every corpus source must pass the metric laws -------------
+def slice_law(n):
+    """A win that regresses one source class is not a win: check them all."""
+    mine = docs["cand"].get("by_source") or {}
+    theirs = docs["iq4r"].get("by_source") or {}
+    if not mine or not theirs:
+        law(n, "no per-source summaries to check", True)
+        return
+    missing = sorted(set(theirs) - set(mine))
+    extra = sorted(set(mine) - set(theirs))
+    if missing or extra:
+        law(n, "same source slices in both reports"
+            + (f" (missing from cand: {', '.join(missing)})" if missing else "")
+            + (f" (only in cand: {', '.join(extra)})" if extra else ""), False)
+        return
+    bad = []
+    for src in sorted(theirs):
+        c, q = mine[src], theirs[src]
+        worse_mean = c["kld"]["mean"] >= q["kld"]["mean"]
+        worse_tail = c["kld"]["p99_9"] > q["kld"]["p99_9"]
+        worse_top1 = c["top1_agreement"] < q["top1_agreement"]
+        if worse_mean or worse_tail or worse_top1:
+            why = ",".join(m for m, w in (("mean", worse_mean), ("p99.9", worse_tail),
+                                          ("top-1", worse_top1)) if w)
+            bad.append(f"{src} ({why})")
+    law(n, "every source slice passes (mean, p99.9, top-1)"
+        + (f" — regressed: {'; '.join(bad)}" if bad else ""), not bad)
+
+
+if interval:
+    kl_fp8 = r["fp8"][0]
+    gap = kl_iq4r - kl_fp8
+    law(1, f"anchors ordered: fp8 {kl_fp8:.6f} < iq4r {kl_iq4r:.6f} (gap {gap:.6f})",
+        gap > 0)
+    law(2, f"candidate inside the interval: {kl_fp8:.6f} < {kl_cand:.6f} < {kl_iq4r:.6f}",
+        kl_fp8 < kl_cand < kl_iq4r)
+    need = kl_iq4r - recover * gap
+    law(3, f"candidate closes {100*max((kl_iq4r - kl_cand)/gap, 0):.1f}% of the "
+           f"{gap:.6f} gap (needs {100*recover:.0f}%, i.e. KL <= {need:.6f})"
+           if gap > 0 else "gap is not positive; cannot score recovery",
+        gap > 0 and kl_cand <= need)
+    law(4, f"candidate top-1 {100*t_cand:.2f}% >= served iq4r top-1 {100*t_iq4r:.2f}%",
+        t_cand >= t_iq4r)
+    tail_law(5)
+    slice_law(6)
+else:
+    law(1, f"candidate beats the served payload: KL {kl_cand:.6f} < iq4r {kl_iq4r:.6f}",
+        kl_cand < kl_iq4r)
+    tail_law(2)
+    law(3, f"candidate top-1 {100*t_cand:.2f}% >= served iq4r top-1 {100*t_iq4r:.2f}%",
+        t_cand >= t_iq4r)
+    slice_law(4)
+
 print("KLD GATE " + ("PASS" if ok else "FAIL"))
 sys.exit(0 if ok else 1)
 PY

scripts/media-apply.py

@@ -0,0 +1,434 @@
+#!/usr/bin/env python3
+"""media-apply.py — the only thing that moves movie-library files.
+
+Reads the plan written by scripts/media-plan.py (rows: `action  src  dst  kind`,
+plus `mkdir` rows) and performs it with rename(2) — bytes are never copied, so
+an interrupted run has destroyed nothing. Every run is resumable and every run
+is undoable: the journal is a write-ahead log, one JSON line per intent, fsynced
+BEFORE the rename it authorises. That ordering is the whole design: a crash
+leaves either an intent whose move happened or an intent whose move never ran,
+and both are recognised on the next run. A file that moved with no journal line
+cannot have been moved by this tool, and saying so is better than guessing.
+
+Four subcommands:
+
+    apply    --plan PLAN [--root ROOT] [--journal J] [--apply]
+             no --apply => report only and write NOTHING, not even folders
+    rollback --journal J [--root ROOT] [--apply]
+             back to the paths the census saw; folders this run created are
+             removed only when empty, never anything else
+    verify   --plan PLAN --journal J [--root ROOT]
+             every planned file is at its target with the journalled byte count
+    sweep    --journal J [--root ROOT] [--apply]
+             removes the folders THIS RUN emptied — the journal has to show a
+             file that moved out of one, so a folder someone made by hand can
+             never be removed by this tool, and rollback puts them back
+
+Refusals (exit 2, before the first rename, with nothing touched): a target
+already occupied by a foreign file, a source that has gone, a duplicate target,
+a path outside --root, a journal that was already rolled back, a move that would
+cross a filesystem (rename cannot do it and a silent copy at 24 TB would not).
+
+--force relaxes only the guards that are opinion (journal already rolled back,
+paths outside --root); it never allows overwriting a file.
+
+The journal defaults to `media-relayout.jsonl` BESIDE the library root, never
+inside it: a log inside the library is a file Jellyfin will index.
+"""
+import argparse
+import datetime
+import json
+import os
+import sys
+
+DEFAULT_ROOT = os.environ.get("MEDIA_ROOT", "/mnt/entropy/Movies")
+
+
+class Refuse(Exception):
+    """A gate the operator has to see; every refusal leaves the library alone."""
+
+
+def journal_default(root):
+    return os.path.join(os.path.dirname(os.path.abspath(root)), "media-relayout.jsonl")
+
+
+class Journal:
+    """Append-only JSONL of intents. Writes are fsynced before the tool acts."""
+
+    def __init__(self, path, writing):
+        self.path, self.writing = path, writing
+        self.records = []
+        if os.path.exists(path):
+            for line in open(path):
+                line = line.strip()
+                if line:
+                    self.records.append(json.loads(line))
+        self._fh = None
+
+    def already(self, dst, src):
+        """An intent for this exact (src, dst) was written by an earlier run."""
+        return any(r.get("op") == "move" and r.get("dst") == dst
+                   and r.get("src") == src for r in self.records)
+
+    @property
+    def rolled_back(self):
+        return any(r.get("op") == "rollback" for r in self.records)
+
+    def record(self, **fields):
+        self.records.append(fields)
+        if not self.writing:
+            return
+        if self._fh is None:
+            self._fh = open(self.path, "a")
+        self._fh.write(json.dumps(fields) + "\n")
+        self._fh.flush()
+        os.fsync(self._fh.fileno())
+
+    def close(self):
+        if self._fh:
+            self._fh.close()
+
+
+def read_plan(path, root, force):
+    """Plan rows: (mkdirs, keeps, moves), header dropped, all paths under root."""
+    mkdirs, keeps, moves = [], [], []
+    with open(path) as fh:
+        header = fh.readline()
+        if not header.startswith("action\t"):
+            raise Refuse(f"{path} is not a plan (first column is "
+                         f"{header.split(chr(9))[0]!r}, expected 'action') —"
+                         " regenerate it with scripts/media-plan.py")
+        for line in fh:
+            f = line.rstrip("\n").split("\t")
+            if f[0] == "mkdir":
+                mkdirs.append(f[2])
+            elif f[0] == "keep":
+                keeps.append(f[1])
+            elif f[0] == "move":
+                moves.append((f[1], f[2], f[3] if len(f) > 3 else ""))
+    if not force:
+        for p in mkdirs + keeps + [p for m in moves for p in m[:2]]:
+            if p and not (p == root or p.startswith(root + os.sep)):
+                raise Refuse(f"{p} is outside --root {root}"
+                             " (a plan built for another library?)")
+    return mkdirs, keeps, moves
+
+
+# A relayout moves bytes on the raw array; the sidecars the media servers wrote
+# live in the SSD overlay and stay at the old coordinates. Naming the follow-up
+# here is the point of use for that fact — see the Media Metadata View page.
+METADATA_HINT = ("note: sidecars written through the metadata view did not follow"
+                 " these moves — next: scripts/media-view.sh reap")
+
+
+def do_apply(args, root, journal, mkdirs, keeps, moves):
+    """Preflight everything, then folders, then renames. Never a partial start."""
+    problems = []
+    for p in keeps:
+        if not os.path.exists(p):
+            problems.append(f"missing: keep row source is gone: {p}")
+    seen = {}
+    for src, dst, _kind in moves:
+        if src == dst:
+            problems.append(f"no-op: move row whose target is its source: {src}")
+        if dst in seen:
+            problems.append(f"double-claim: {dst} claimed by {seen[dst]} and {src}")
+        seen[dst] = src
+        if journal.already(dst, src) and os.path.exists(dst) and not os.path.exists(src):
+            continue                                    # this intent already ran
+        if not os.path.exists(src):
+            if os.path.exists(dst):
+                problems.append(f"missing: {src} is gone and {dst} carries no journal"
+                                " line — it moved outside this tool, refusing to guess")
+            else:
+                problems.append(f"missing: {src} is gone and {dst} is not there either")
+            continue
+        if os.path.exists(dst):
+            problems.append(f"occupied: target already holds another file: {dst}")
+    if problems:
+        print(f"refusing: {len(problems)} problem(s), nothing moved")
+        for p in sorted(problems)[:20]:
+            print("  " + p)
+        if len(problems) > 20:
+            print(f"  ... and {len(problems) - 20} more")
+        return 2
+
+    if not args.apply:
+        print(f"dry run: {len(moves)} moves, {len(mkdirs)} folders, {len(keeps)} files stay"
+              " — nothing written (pass --apply to run)")
+        for src, dst, _k in moves[:5]:
+            print(f"  would move {src} -> {dst}")
+        return 0
+
+    made = {r.get("dst") for r in journal.records if r.get("op") == "mkdir"}
+    dirs_new = 0
+    for d in sorted(mkdirs, key=lambda q: q.count(os.sep)):
+        if d in made or os.path.isdir(d):
+            continue
+        journal.record(op="mkdir", dst=d)
+        os.makedirs(d, exist_ok=True)
+        dirs_new += 1
+
+    # same-filesystem gate: rename cannot cross a device, and a silent copy of a
+    # library this size is not an option anyone would choose.
+    cross = []
+    for src, dst, _k in moves:
+        if journal.already(dst, src) and os.path.exists(dst) and not os.path.exists(src):
+            continue
+        parent = os.path.dirname(dst)
+        if os.path.exists(parent) and os.stat(parent).st_dev != os.stat(src).st_dev:
+            cross.append(dst)
+    if cross:
+        print(f"refusing: {len(cross)} moves would cross a filesystem"
+              f" (first: {cross[0]}) — this tool only renames")
+        return 2
+
+    moved = done = 0
+    for src, dst, _kind in moves:
+        if journal.already(dst, src):
+            if os.path.exists(dst) and not os.path.exists(src):
+                done += 1
+                continue
+            if os.path.exists(src) and not os.path.exists(dst):
+                os.rename(src, dst)             # the intent ran, the move did not
+                moved += 1
+                continue
+        st = os.stat(src)
+        journal.record(op="move", src=src, dst=dst, bytes=st.st_size, mode=st.st_mode,
+                       mtime=st.st_mtime, ino=st.st_ino, dev=st.st_dev)
+        os.rename(src, dst)
+        moved += 1
+    print(f"applied: {moved} moved, {done} already done in an earlier run,"
+          f" {dirs_new} folders created, {len(keeps)} files stayed put")
+    print(f"journal: {journal.path} — undo with: media-apply.py rollback"
+          f" --journal {journal.path} --root {root} --apply")
+    print(METADATA_HINT)
+    return 0
+
+
+def do_verify(root, journal, moves, keeps):
+    """Every planned file sits at its target carrying the journalled bytes."""
+    journalled = {(r["src"], r["dst"]): r for r in journal.records
+                  if r.get("op") == "move"}
+    todo = moves or [(r["src"], r["dst"], "") for r in journal.records
+                     if r.get("op") == "move"]
+    bad, missing, bytes_total = [], [], 0
+    for src, dst, _kind in todo:
+        rec = journalled.get((src, dst))
+        if not os.path.exists(dst):
+            missing.append(dst)
+            continue
+        size = os.path.getsize(dst)
+        bytes_total += size
+        if rec and rec.get("bytes") not in (None, size):
+            bad.append(f"{dst}: {size} bytes on disk, {rec['bytes']} at move time")
+        if os.path.exists(src) and src != dst:
+            bad.append(f"{src}: still in place as well as at {dst}")
+    missing += [p for p in keeps if not os.path.exists(p)]
+    if bad or missing:
+        print(f"verify FAILED: {len(missing)} absent, {len(bad)} wrong")
+        for line in (missing + bad)[:20]:
+            print("  " + line)
+        return 2
+    empty = [r["dst"] for r in journal.records
+             if r.get("op") == "mkdir" and os.path.isdir(r["dst"])
+             and not os.listdir(r["dst"])]
+    print(f"verified: {len(todo)} files at their targets"
+          f" ({bytes_total / 2**40:.2f} TiB), {len(keeps)} kept")
+    if empty:
+        print(f"note: {len(empty)} folders are empty (left in place — see the trash law)")
+    return 0
+
+
+def sweep_targets(root, journal):
+    """The folders this run emptied, deepest first — never one it did not touch.
+
+    Eligibility is evidence, not appearance. A folder is a target only when the
+    journal shows a file moved out of it or out of something beneath it, it holds
+    nothing but empty directories now, it is not a folder this run created, and no
+    journalled destination lives under it. A symlink anywhere inside disqualifies
+    the whole subtree: this tool never follows a link, and deleting the folder
+    that holds one would delete someone's mount point.
+
+    Proof carries downward: once a folder qualifies, every empty directory inside
+    it qualifies too. A release's junk (``Screens/``, ``Sample/``) is never itself
+    a move source, so without that rule it holds its emptied parent on disk
+    forever — the parent can never be removed while the child stands.
+    """
+    moves = [r for r in journal.records if r.get("op") == "move"]
+    made = {r["dst"] for r in journal.records if r.get("op") == "mkdir"}
+    # An rmdir INTENT that failed left its folder standing, and the next sweep
+    # must be able to finish it: only a folder that is really gone is out of
+    # scope. Counting every intent as done made 146 live survivors permanent.
+    gone = {r["path"] for r in journal.records if r.get("op") == "rmdir"
+            and not os.path.exists(r["path"])}
+    dsts = [r["dst"] for r in moves]
+    marked = set()
+    for r in moves:
+        d = os.path.dirname(r["src"])
+        while d.startswith(root + os.sep):
+            marked.add(d)
+            d = os.path.dirname(d)
+
+    def removable(d):
+        if d in made or d in gone or os.path.islink(d):
+            return False
+        if any(p == d or p.startswith(d + os.sep) for p in dsts):
+            return False
+        try:
+            entries = os.listdir(d)
+        except OSError:
+            return False
+        for e in entries:
+            c = os.path.join(d, e)
+            if os.path.islink(c) or not os.path.isdir(c) or not removable(c):
+                return False
+        return True
+
+    roots = [d for d in marked if removable(d)]
+    targets = set(roots)
+    for d in roots:
+        for dirpath, dirnames, _ in os.walk(d):
+            for c in dirnames:
+                p = os.path.join(dirpath, c)
+                if removable(p):
+                    targets.add(p)
+    return sorted(targets, key=lambda q: (-q.count(os.sep), q))
+
+
+def do_sweep(args, root, journal):
+    """Remove the empty husks an apply left standing at the library root."""
+    moves = [r for r in journal.records if r.get("op") == "move"]
+    if not moves:
+        raise Refuse("the journal holds no moves — a sweep follows an apply, it"
+                     " does not guess at emptiness")
+    outside = sorted({p for r in moves for p in (r["src"], r["dst"])
+                      if not (p == root or p.startswith(root + os.sep))})
+    if outside and not args.force:
+        raise Refuse(f"{len(outside)} journalled paths are outside --root {root}"
+                     f" (first: {outside[0]}) — this journal belongs to another"
+                     " library")
+    targets = sweep_targets(root, journal)
+    if not targets:
+        print("swept: 0 folders — nothing this run emptied is still standing empty")
+        return 0
+    if not args.apply:
+        print(f"dry run: {len(targets)} folders this run emptied would be removed"
+              " (pass --apply to run)")
+        for d in targets[:5]:
+            print(f"  would remove {d}")
+        return 0
+    removed = 0
+    for d in targets:
+        journal.record(op="rmdir", path=d)
+        try:
+            os.rmdir(d)
+        except OSError as e:
+            print(f"left in place: {d} ({e})")
+            continue
+        removed += 1
+    print(f"swept: {removed} folders removed, {len(targets) - removed} left in place")
+    print(f"journal: {journal.path} — rollback recreates them with their files")
+    return 0
+
+
+def do_rollback(args, root, journal):
+    """Undo every journalled move, newest first, after a full preflight."""
+    moves = [r for r in journal.records if r.get("op") == "move"]
+    made = [r["dst"] for r in journal.records if r.get("op") == "mkdir"]
+    swept = [r["path"] for r in journal.records if r.get("op") == "rmdir"]
+    problems = []
+    for r in reversed(moves):
+        src, dst = r["src"], r["dst"]
+        if not args.force and not (src.startswith(root + os.sep)
+                                   or dst.startswith(root + os.sep)):
+            problems.append(f"outside --root: {dst}")
+            continue
+        if os.path.exists(dst) and os.path.exists(src):
+            problems.append(f"occupied: both paths exist, refusing to choose:"
+                            f" {src} | {dst}")
+        elif os.path.exists(dst):
+            pass
+        elif os.path.exists(src):
+            continue                                    # already undone
+        else:
+            problems.append(f"missing: {dst} is gone and {src} never came back")
+    if problems:
+        print(f"refusing: {len(problems)} problem(s), nothing moved back")
+        for p in problems[:20]:
+            print("  " + p)
+        return 2
+    if not args.apply:
+        print(f"dry run: would move {len(moves)} files back to their census paths"
+              f" and recreate {len(set(swept))} swept folders (pass --apply)")
+        return 0
+    undone = 0
+    recreated = 0
+    for d in sorted(set(swept), key=lambda q: q.count(os.sep)):
+        if not os.path.isdir(d):
+            os.makedirs(d, exist_ok=True)
+            recreated += 1
+    for r in reversed(moves):
+        if os.path.exists(r["dst"]) and not os.path.exists(r["src"]):
+            os.makedirs(os.path.dirname(r["src"]), exist_ok=True)
+            os.rename(r["dst"], r["src"])
+            undone += 1
+    dirs_gone = 0
+    for d in sorted(set(made), key=lambda q: -q.count(os.sep)):
+        if d == root or not os.path.isdir(d) or os.listdir(d):
+            continue
+        try:
+            os.rmdir(d)
+            dirs_gone += 1
+        except OSError as e:
+            print(f"left in place: {d} ({e})")
+    journal.record(op="rollback", undone=undone, dirs_removed=dirs_gone,
+                   t=datetime.datetime.now(datetime.timezone.utc).isoformat())
+    print(f"rolled back: {undone} files back where the census found them,"
+          f" {dirs_gone} empty folders removed, {recreated} swept folders recreated")
+    print(METADATA_HINT)
+    return 0
+
+
+def main():
+    ap = argparse.ArgumentParser(description="apply or undo a media relayout plan")
+    ap.add_argument("cmd", choices=["apply", "rollback", "verify", "sweep"])
+    ap.add_argument("--plan")
+    ap.add_argument("--root", default=DEFAULT_ROOT)
+    ap.add_argument("--journal")
+    ap.add_argument("--apply", action="store_true", help="actually change the library")
+    ap.add_argument("--force", action="store_true", help="relax the opinion guards")
+    args = ap.parse_args()
+
+    root = os.path.abspath(args.root)
+    if not os.path.isdir(root):
+        raise Refuse(f"no library at {root}")
+    journal_path = args.journal or journal_default(root)
+    journal = Journal(journal_path, args.apply)
+    try:
+        if args.cmd in ("rollback", "verify", "sweep") and not os.path.exists(journal_path):
+            raise Refuse(f"no journal at {journal_path} — nothing to {args.cmd}")
+        if args.cmd in ("rollback", "sweep") and not args.force and journal.rolled_back:
+            raise Refuse("this journal has already been rolled back"
+                         " (--force to replay it)")
+        if args.cmd in ("apply", "verify") and not args.plan:
+            raise Refuse(f"{args.cmd} needs --plan")
+        mkdirs, keeps, moves = (read_plan(args.plan, root, args.force)
+                                if args.plan else ([], [], []))
+        if args.cmd == "apply":
+            return do_apply(args, root, journal, mkdirs, keeps, moves)
+        if args.cmd == "verify":
+            return do_verify(root, journal, moves, keeps)
+        if args.cmd == "sweep":
+            return do_sweep(args, root, journal)
+        return do_rollback(args, root, journal)
+    finally:
+        journal.close()
+
+
+if __name__ == "__main__":
+    try:
+        sys.exit(main())
+    except Refuse as r:
+        print(f"refusing: {r}")
+        sys.exit(2)

scripts/media-census.py

@@ -0,0 +1,304 @@
+#!/usr/bin/env python3
+r"""Read-only census of a media library — the inputs a relayout plan is built from.
+
+  media-census.py enumerate [--root DIR] [--out TSV]
+  media-census.py probe     [--enum TSV] [--out TSV] [--jobs N]
+  media-census.py plex      [--root DIR] [--db PATH] [--out TSV]
+  media-census.py jellyfin  [--root DIR] [--db PATH] [--container PREFIX] [--out TSV]
+
+Run it on the machine that mounts the library. Nothing here writes inside the
+library and nothing is deleted; the only outputs are the TSVs below, and the
+scripts/media-plan.py and scripts/media-shows.py read them instead of touching media.
+
+  enum    bytes <TAB> ext <TAB> path                  every file under the root
+  probed  bytes <TAB> path <TAB> dur <TAB> vcodec <TAB> WxH <TAB> fps <TAB> vbit
+          <TAB> pix <TAB> audio <TAB> subs <TAB> spare <TAB> format
+          one row per video-container file; `audio` is "codec:lang+codec:lang",
+          `subs` the subtitle-stream count, `spare` the audio streams carrying
+          neither a default nor a forced tag
+  plex    file <TAB> leaf_title <TAB> … (see PLEX_COLS)
+          the titles Plex already matched — the authority over any filename guess
+
+  jellyfin file <TAB> series <TAB> year <TAB> season <TAB> episode <TAB> extra
+          <TAB> tmdb      (see JF_COLS)
+          one row per episode Jellyfin has matched and numbered, its path mapped
+          from the container's view back to the host; `year` and `tmdb` are the
+          SERIES' values, `extra` is Jellyfin's own extra type when the row is
+          bonus material rather than an episode
+
+`probe` is the slow step (every container is opened for its header, ~10 min for
+4,800 files on a spindle array at 6 workers) so it is `nice`d: a census beside a
+live service must not starve it. `enumerate` is a stat pass, so re-censusing
+before a plan is cheap — and a plan is only ever as good as these files.
+"""
+import argparse
+import csv
+import json
+import os
+import re
+import sqlite3
+import subprocess
+import sys
+from concurrent.futures import ThreadPoolExecutor
+
+# Containers ffprobe can read. .iso/.img are deliberately NOT here: a disc image is
+# a mounted volume, not a stream, and the planner treats it as a movie file instead.
+PROBE_EXT = {".mkv", ".mp4", ".avi", ".m4v", ".mov", ".ts", ".m2ts", ".mpg",
+             ".mpeg", ".webm", ".wmv", ".divx", ".ogm", ".bdmv", ".vob"}
+# Trash and thumbnail caches are not library content, and this repo's trash policy
+# means .AGENTTRASH can sit inside a movie folder at any time.
+SKIP_DIR = re.compile(r"(^|/)(\.AGENTTRASH|\.thumbnails|\.Trash-\d+|@eaDir|lost\+found)(/|$)")
+PLEX_DB = ("/var/lib/plexmediaserver/Library/Application Support/Plex Media Server"
+           "/Plug-in Support/Databases/com.plexapp.plugins.library.db")
+PLEX_COLS = ["file", "leaf_title", "leaf_orig", "leaf_edition", "leaf_year",
+             "parent_title", "parent_orig", "parent_year", "parent_edition",
+             "container", "vcodec", "acodec", "width", "height"]
+PLEX_SQL = """
+select p.file, coalesce(mi.title,''), coalesce(mi.original_title,''),
+       coalesce(mi.edition_title,''), coalesce(mi.year,0),
+       coalesce(par.title,''), coalesce(par.original_title,''),
+       coalesce(par.year,0), coalesce(par.edition_title,''),
+       m.container, m.video_codec, m.audio_codec, m.width, m.height
+from media_parts p
+join media_items m on m.id = p.media_item_id
+join metadata_items mi on mi.id = m.metadata_item_id
+left join metadata_items par on par.id = mi.parent_id
+where p.file like ? and p.deleted_at is null and mi.deleted_at is null
+"""
+JF_DB = "/home/dank/Containers/jellyfin/config/data/jellyfin.db"
+JF_COLS = ["file", "series", "year", "season", "episode", "extra", "tmdb"]
+# Jellyfin stores the entity's .NET class as its type; the TV tree is these three.
+JF_TV = ("MediaBrowser.Controller.Entities.TV.Episode",
+         "MediaBrowser.Controller.Entities.TV.Season",
+         "MediaBrowser.Controller.Entities.TV.Series")
+JF_SQL = ("""select Id, Path, IndexNumber, ParentIndexNumber, ExtraType, IsVirtualItem
+              from BaseItems where Type = ? and Path like ?""")
+
+
+def enumerate_lib(root, out):
+    """One row per real file under root: size, extension, path."""
+    rows = []
+    for dirpath, dirnames, filenames in os.walk(root):
+        dirnames[:] = [d for d in dirnames
+                       if not SKIP_DIR.search(os.path.join(dirpath, d))]
+        for fn in filenames:
+            p = os.path.join(dirpath, fn)
+            if SKIP_DIR.search(p):
+                continue
+            try:
+                st = os.lstat(p)
+            except OSError:
+                continue
+            if not os.path.isfile(p) or os.path.islink(p):
+                continue                # links and sockets are not library content
+            rows.append((st.st_size, os.path.splitext(fn)[1].lower(), p))
+    with open(out, "w") as fh:
+        for size, ext, p in rows:
+            fh.write(f"{size}\t{ext}\t{p}\n")
+    vid = [r for r in rows if r[1] in PROBE_EXT]
+    print(f"files={len(rows)} bytes={sum(r[0] for r in rows)} "
+          f"probes={len(vid)} probe_bytes={sum(r[0] for r in vid)} -> {out}")
+
+
+def probe_one(path):
+    """ffprobe header read -> the probed TSV row (never the whole file)."""
+    try:
+        os.nice(19)                     # a census must never outrank the media server
+    except OSError:
+        pass
+    try:
+        size = os.stat(path).st_size
+    except OSError:
+        return None
+    cmd = ["ffprobe", "-v", "error", "-print_format", "json",
+           "-show_format", "-show_streams", path]
+    try:
+        done = subprocess.run(cmd, capture_output=True, text=True, timeout=180)
+        data = json.loads(done.stdout or "{}")
+    except (subprocess.TimeoutExpired, subprocess.SubprocessError,
+            json.JSONDecodeError, OSError):
+        data = {}
+    st = data.get("streams", [])
+    fmt = data.get("format", {})
+    v = next((s for s in st if s.get("codec_type") == "video"), None)
+    aud = [s for s in st if s.get("codec_type") == "audio"]
+    subs = [s for s in st if s.get("codec_type") == "subtitle"]
+    fps = "-"
+    if v and v.get("r_frame_rate"):
+        try:
+            n, d = (int(x) for x in v["r_frame_rate"].split("/", 1))
+            fps = f"{round(n / d * 100) / 100:g}" if d else "0"
+        except ValueError:
+            pass
+
+    def num(x):
+        try:
+            return round(float(x))
+        except (TypeError, ValueError):
+            return 0
+
+    return [size, path, num(fmt.get("duration")),
+            (v or {}).get("codec_name", "-"),
+            f"{v['width']}x{v['height']}" if v else "-",
+            fps, num(fmt.get("bit_rate")), (v or {}).get("pix_fmt", "-"),
+            "+".join(f"{a.get('codec_name', '?')}:{a.get('language', '?')}" for a in aud),
+            len(subs),
+            sum(1 for a in aud if not re.search(
+                r"default|forced", json.dumps(a.get("tags", {})), re.I)),
+            fmt.get("format_name", "-") or "-"]
+
+
+def probe(enum, out, jobs):
+    """Probe every video-container row of the enum TSV, in parallel."""
+    paths = []
+    with open(enum) as fh:
+        for line in fh:
+            f = line.rstrip("\n").split("\t", 2)
+            if len(f) == 3 and f[1] in PROBE_EXT:
+                paths.append(f[2])
+    print(f"probing {len(paths)} files with {jobs} workers", flush=True)
+    done = unreadable = 0
+    with open(out, "w") as fh, ThreadPoolExecutor(max_workers=jobs) as pool:
+        for row in pool.map(probe_one, paths):
+            if row is None:
+                continue
+            if row[3] == "-":
+                unreadable += 1         # header unreadable: the row still gets written
+            done += 1
+            fh.write("\t".join(str(x) for x in row) + "\n")
+            if done % 250 == 0:
+                print(f"  {done}/{len(paths)}", flush=True)
+    print(f"probed={done} unreadable={unreadable} -> {out}")
+
+
+def plex(root, db, out):
+    """Plex's matched titles for files under root, from its DB, read-only."""
+    uri = f"file:{db}?mode=ro&immutable=1"
+    try:
+        con = sqlite3.connect(uri, uri=True)
+    except sqlite3.Error as e:
+        sys.exit(f"cannot open the Plex database read-only ({db}): {e}\n"
+                 "Plex runs rootless here: run this as the container's owner, never "
+                 "with sudo — rootless storage is per-user, so root sees nothing.")
+    with open(out, "w", newline="") as fh:
+        w = csv.writer(fh, delimiter="\t", lineterminator="\n")
+        w.writerow(PLEX_COLS)
+        n = 0
+        for r in con.execute(PLEX_SQL, (root + "/%",)):
+            w.writerow([str(x).replace("\t", " ").replace("\n", " ") for x in r])
+            n += 1
+    print(f"plex-matched={n} -> {out}")
+
+
+def jellyfin(root, db, container, out):
+    """The episodes Jellyfin has already numbered, for the library at root.
+
+    Jellyfin stores each path as the CONTAINER sees it (its quadlet binds the
+    share at /data), so `container` is the in-container prefix that stands for
+    root and every exported path is mapped back to the host before it is written
+    — the planner reads host paths, because it plans moves on the array.
+
+    The numbering is what matters, not the name: a release group's file name says
+    nothing about which episode a file is, and Jellyfin has already answered that
+    question against a provider. Season/episode come from the episode row; the
+    series name, its year and its TMDb id come from the Series row the episode
+    climbs to, because those are the only place Jellyfin keeps them.
+    """
+    try:
+        con = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
+    except sqlite3.Error as e:
+        sys.exit(f"cannot open the Jellyfin database read-only ({db}): {e}\n"
+                 "It is a WAL database: copy it with its -wal and -shm beside it and"
+                 " point --db at the copy. Never sudo it — the data dir belongs to"
+                 " the container's owner, and rootless storage is per-user.")
+    have = {r[1] for r in con.execute("pragma table_info(BaseItems)")}
+    need = {"Id", "ParentId", "Type", "Path", "IndexNumber", "ParentIndexNumber",
+            "ExtraType", "IsVirtualItem", "ProductionYear", "Data"}
+    if not need <= have:
+        sys.exit(f"{db} is not a Jellyfin library database (BaseItems has no"
+                 f" {','.join(sorted(need - have))})")
+    node = {}
+    for i, pid, typ, name, year, data in con.execute(
+            "select Id, ParentId, Type, Name, ProductionYear, Data from BaseItems"
+            " where Type in (?,?,?)", JF_TV):
+        node[i] = (pid, typ, name, year, data)
+
+    def series_of(item_id):
+        """Climb episode -> season -> series; empty when the chain is broken."""
+        for _ in range(4):
+            if item_id not in node:
+                return "", "", ""
+            pid, typ, name, year, data = node[item_id]
+            if typ.endswith(".Series"):
+                try:
+                    tmdb = (json.loads(data or "{}")
+                            .get("ProviderIds") or {}).get("Tmdb", "")
+                except ValueError:
+                    tmdb = ""
+                return name or "", year or "", tmdb
+            item_id = pid
+        return "", "", ""
+
+    # The fleet's quadlets bind the share at /data, so the container's view of
+    # this library is /data/<the library's own folder name> unless told otherwise.
+    if not container:
+        container = "/data/" + os.path.basename(os.path.normpath(root))
+    prefix = container.rstrip("/") + "/"
+    written = skipped = 0
+    with open(out, "w", newline="") as fh:
+        w = csv.writer(fh, delimiter="\t", lineterminator="\n")
+        w.writerow(JF_COLS)
+        for iid, path, idx, pidx, extra, virt in con.execute(JF_SQL,
+                                                             (JF_TV[0], prefix + "%")):
+            if virt or not path or not path.startswith(prefix):
+                skipped += 1          # a virtual item is a variant, not a file
+                continue
+            sname, syear, tmdb = series_of(iid)
+            w.writerow([root + "/" + path[len(prefix):],
+                        sname.replace("\t", " "), syear,
+                        "" if pidx is None else pidx,
+                        "" if idx is None else idx,
+                        (extra or "").replace("\t", " "), tmdb])
+            written += 1
+    print(f"episodes={written} virtual-or-blank={skipped} -> {out}")
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    sub = ap.add_subparsers(dest="cmd", required=True)
+    root = {"default": os.environ.get("MEDIA_ROOT", "/mnt/entropy/Movies"),
+            "help": "library root"}
+    e = sub.add_parser("enumerate", help="stat every file under the root")
+    e.add_argument("--root", **root)
+    e.add_argument("--out", default="/tmp/media-enum.tsv")
+    p = sub.add_parser("probe", help="ffprobe every video container")
+    p.add_argument("--enum", default="/tmp/media-enum.tsv")
+    p.add_argument("--out", default="/tmp/media-probed.tsv")
+    p.add_argument("--jobs", type=int, default=6)
+    x = sub.add_parser("plex", help="export Plex's matched titles for the root")
+    x.add_argument("--root", **root)
+    x.add_argument("--db", default=PLEX_DB)
+    x.add_argument("--out", default="/tmp/plex-movies.tsv")
+    j = sub.add_parser("jellyfin",
+                       help="export the episodes Jellyfin has numbered for the root")
+    j.add_argument("--root", default=os.environ.get("MEDIA_ROOT",
+                                                    "/mnt/entropy/TVShows"),
+                   help="TV library root on the host")
+    j.add_argument("--db", default=JF_DB)
+    j.add_argument("--container", default=None,
+                   help="the in-container prefix that stands for --root"
+                        " (default /data/<root's folder name>)")
+    j.add_argument("--out", default="/tmp/jellyfin-tv.tsv")
+    a = ap.parse_args()
+    if a.cmd == "enumerate":
+        enumerate_lib(a.root, a.out)
+    elif a.cmd == "probe":
+        probe(a.enum, a.out, a.jobs)
+    elif a.cmd == "plex":
+        plex(a.root, a.db, a.out)
+    else:
+        jellyfin(a.root, a.db, a.container, a.out)
+
+
+if __name__ == "__main__":
+    main()

scripts/media-plan.py

@@ -0,0 +1,881 @@
+#!/usr/bin/env python3
+r"""Plan a Jellyfin-format relayout of a movie library.  DRY RUN ONLY: nothing moves.
+
+  media-plan.py [--root DIR] [--enum TSV] [--probed TSV] [--plex TSV]
+                [--out TSV] [--exceptions TXT]
+
+Inputs are the TSVs scripts/media-census.py exports, plus the optional Plex
+metadata-cache export whose clean titles outrank the filename parser; the title
+authority for everything Plex never matched is scripts/media-titles.py beside this
+file. No media file is opened, so planning the whole library costs seconds.
+
+PLAN is a TSV the executor (scripts/media-apply.py) replays verbatim —
+`action<TAB>src<TAB>dst<TAB>kind` with action in mkdir | move | keep, ordered so
+parents exist before children. EXCEPTIONS is the human review list: every file the
+plan could not name a film for, every collision it refused, the READMEs the
+executor must write into the piles it created.
+
+Target contract (jellyfin.org/docs/general/server/media/movies):
+  * one folder per movie, named  Title (year)
+  * the video inside is named exactly like the folder
+  * extra copies of one film are versions: "<base> - <label>.ext"; the base must
+    match the folder name character-for-character or Jellyfin splits them apart
+  * extras live in canonical subfolders (trailers, featurettes, deleted scenes,
+    interviews, samples, shorts, clips, other, extras, backdrops, theme-music)
+  * sidecars <base>.<lang>.<flag>.<ext>; images <base>-<poster|backdrop|...>
+  * <>:"/\|?* break the matcher, so they never survive here
+  * VIDEO_TS/BDMV folders support neither multiple versions nor parts
+"""
+import argparse
+import collections
+import csv
+import importlib.util
+import os
+import re
+
+# Inputs are the census TSVs scripts/media-census.py writes; the engine beside
+# this file is the title authority. Nothing here opens a media file.
+def parse_args():
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    ap.add_argument("--root", default=os.environ.get("MEDIA_ROOT", "/mnt/entropy/Movies"),
+                    help="library root the plan moves files inside")
+    ap.add_argument("--enum", default="/tmp/media-enum.tsv", help="file census TSV")
+    ap.add_argument("--probed", default="/tmp/media-probed.tsv", help="ffprobe census TSV")
+    ap.add_argument("--plex", default="/tmp/plex-movies.tsv",
+                    help="Plex metadata export (clean titles); absent = parser only")
+    ap.add_argument("--out", default="/tmp/jf-plan.tsv")
+    ap.add_argument("--exceptions", default="/tmp/jf-exceptions.txt")
+    return ap.parse_args()
+
+
+ARGS = parse_args()
+ROOT = ARGS.root
+ENUM, PROBE, PLEX = ARGS.enum, ARGS.probed, ARGS.plex
+PLAN, EXCEPTS = ARGS.out, ARGS.exceptions
+ENGINE = os.path.join(os.path.dirname(os.path.abspath(__file__)), "media-titles.py")
+
+spec = importlib.util.spec_from_file_location("eng", ENGINE)
+eng = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(eng)
+
+VIDEO_EXT = {".mkv", ".mp4", ".avi", ".m4v", ".mov", ".ts", ".m2ts", ".mpg",
+             ".mpeg", ".webm", ".wmv", ".divx", ".ogm", ".flv", ".vob", ".iso",
+             ".img", ".mpv", ".3gp", ".rm", ".rmvb", ".m2v", ".mpv"}
+DISC_DIRS = {"BDMV", "VIDEO_TS", "CERTIFICATE", "AACS", "MLPI", "CLIPINF"}
+SUB_EXT = {".srt", ".ass", ".ssa", ".sub", ".vtt"}
+IMG_EXT = {".jpg", ".jpeg", ".png", ".webp", ".bmp", ".gif", ".tbn"}
+AUDIO_DIR_RE = re.compile(r"soundtrack|score|\bost\b|music|song|album", re.I)
+SUB_KEEP_RE = re.compile(r"^(?!.*\d)(?!.*[._])(?!.*\[)(?!.*\()([A-Za-z][A-Za-z -]{1,24})$")
+MUSIC_TOP_RE = re.compile(r"(?i)soundtrack|score|\bost\b|music|album|song")
+SUB_DIR_OK = re.compile(r"(?i)^$|subs?|subtitles?|srt|custom ?subs?(\[.*\])?"
+                        r"|(additional|extra) languages|languages|resources")
+AUDIO_DIR_OK = re.compile(r"(?i)^$|audio(s)?|extra audio|additional audio"
+                          r"|(additional|extra) languages|commentar(y|ies)"
+                          r"|audio commentary|audio tracks?")
+
+
+AUDIO_EXT = {".mp3", ".flac", ".m4a", ".aac", ".dts", ".wav", ".ac3", ".mka",
+             ".opus", ".ogg", ".wma"}
+NFO_EXT = {".nfo"}
+IMG_PRIMARY = {"poster", "folder", "cover", "default", "movie", "show"}
+IMG_OTHER = {"backdrop": "backdrop", "fanart": "backdrop", "background": "backdrop",
+             "banner": "banner", "logo": "logo", "clearlogo": "logo",
+             "thumb": "thumb", "landscape": "thumb", "art": "backdrop",
+             "clearart": "backdrop", "jacket": "poster", "cdart": "poster",
+             "discart": "poster", "disc": "poster"}
+RES_RE = re.compile(r"\b(2160|4320|4k|1080|720|576|480|360|240)(i|p)\b", re.I)
+# The characters Jellyfin's matcher cannot carry in a file or folder name. "/" is
+# one of them and it is the dangerous one: Plex titles KEEP it ("V/H/S/2", and this
+# library holds six of those films), so leaving it in a name turns ONE movie folder
+# into a nested tree — Movies/V/H/S/2 (2013)/… — and Jellyfin then sees four
+# folders, none of them a movie.
+BAD_CHARS = re.compile(r'[<>:"/|?*]')
+PART_RE = re.compile(r"(?:^|[\s._-])(cd|dvd|part|pt|disc|disk|disco)"
+                     r"[\s._-]?([1-9abcd])(?:$|[\s._-])", re.I)
+LANGS = {"en", "eng", "it", "ita", "ja", "jpn", "de", "deu", "fr", "fra", "es",
+         "spa", "pt", "por", "ru", "rus", "zh", "chi", "ko", "kor", "sv", "no",
+         "da", "nl", "pl", "cs", "hu", "fi", "tr", "el", "he", "ar", "uk", "ro",
+         "bg", "hr", "sr", "sk", "et", "lv", "lt", "id", "th", "vi", "fa", "la"}
+FLAGS = {"default", "forced", "foreign", "sdh", "cc"}
+NOISE = {"sub", "subs", "subtitle", "subtitles", "srt", "ass", "ext", "external",
+         "the", "a", "an", "of", "and", "in", "on", "to", "for", "with", "ita",
+         "eng", "english", "italian", "dual", "audio", "forced", "default"}
+# Canonical title where the library only ever carries a localised name.
+OVERRIDES = {"28 giorni dopo": "28 Days Later",
+             "28 settimane dopo": "28 Weeks Later"}
+
+rows = []
+for line in open(ENUM):
+    size, ext, path = line.rstrip("\n").split("\t")
+    rows.append({"bytes": int(size), "ext": ext.lower(), "path": path})
+ext_by = {r["path"]: r["ext"] for r in rows}
+size_by = {r["path"]: r["bytes"] for r in rows}
+
+
+def rel(p):
+    return p[len(ROOT) + 1:] if p.startswith(ROOT + "/") else p
+
+
+def parts_of(p):
+    return [q for q in rel(p).split("/") if q]
+
+
+def is_qb(p):
+    return ".!q" in p.lower()
+
+
+def disc_root(p, fold=True):
+    """Absolute root of the disc tree containing p (the MOVIE folder that holds
+    BDMV/ or VIDEO_TS/), else None. `fold` also folds bookkeeping dirs that sit
+    beside the tree (CERTIFICATE/, AACS/) into the same movie folder."""
+    ps = parts_of(p)
+    kinds = DISC_DIRS if fold else {"BDMV", "VIDEO_TS"}
+    for i, q in enumerate(ps):
+        if q.upper() in kinds:
+            return ROOT + "/" + "/".join(ps[:i])
+    return None
+
+
+def row_name(r):
+    """The name a disc-tree entry should be identified by: the engine stores the
+    disc's own folder for it, because "BDMV/index.bdmv" is not a title."""
+    return r.get("name_src") or r["path"]
+
+
+def sanitize(name):
+    name = BAD_CHARS.sub("-", name).replace("\\", "-")
+    name = re.sub(r"\s+", " ", name).strip().strip(". ").strip()
+    return (name[:180] or "untitled")
+
+
+def ok_year(y):
+    """True for a year a real film could carry. Plex stores 0 for a file it indexed
+    but never MATCHED, and "0" is a digit string: trusting it named 67 movie
+    folders "Title (0)" — Jellyfin items with no identity, no year and no poster."""
+    return bool(re.fullmatch(r"(18|19|20)\d\d", y or "")) and 1870 <= int(y) <= 2035
+
+
+WORD_SPLIT = re.compile(r"[^0-9a-z]+")
+
+
+def words_of(text):
+    """The matching words of a name — or of a whole path, which is how callers
+    pass a file together with the folders it sits in.
+
+    Split on everything that is not a letter or a digit: eng.norm does not touch
+    "/", so feeding it "…/Psycho (1960)/Extras" left glued tokens like
+    "(1960)/extras" and "(1954)" that no other name could ever agree with, and a
+    film could not be found by the folder that holds it — which is how one box
+    set's featurettes ended up in another film's folder. Both sides of every
+    comparison below go through here. A bare number counts only when it could be
+    a release year, so release junk ("1080", "2160", "2", "7") never poses as
+    agreement between two names.
+    """
+    out = set()
+    for w in WORD_SPLIT.split(eng.norm(text)):
+        if not w or w in eng.STOP:
+            continue
+        if w.isdigit() and not 1870 <= int(w) <= 2100:
+            continue
+        out.add(w)
+    return out
+
+
+# ------------------------------------------------------------------ Plex titles
+# Plex has matched a large part of the library's media files to real titles
+# ("Alien", not "Alien.Directors.Cut.1979.1080p.BRrip.x265…"). Where the cache
+# holds a MATCH, its title wins; the filename parser carries everything else
+# (extras, disc trees, TV, and whatever Plex never matched).
+def plex_titles():
+    if not os.path.exists(PLEX):
+        return {}
+    out = {}
+    with open(PLEX, newline="") as fh:
+        for r in csv.DictReader(fh, delimiter="\t"):
+            t, y = r["leaf_title"].strip(), r["leaf_year"].strip()
+            # Only a match is an authority, and a match carries a year: an
+            # unmatched row echoes the filename, which the parser reads better
+            # than a copied release name. The junk filter stays for the matches
+            # whose title still carries the release.
+            if t and ok_year(y) and not re.search(
+                    r"2160p|1080p|x26[45]|hevc|bluray|web-?dl", t.lower()):
+                out[r["file"]] = (t, y)
+    return out
+
+
+PLEX_T = plex_titles()
+
+# ------------------------------------------------------------- classification
+erows = eng.load(PROBE)
+for r in erows:
+    for k, v in OVERRIDES.items():
+        if k in r["title"]:
+            r["title"] = v
+            r["key"] = eng.key_of(r["title"])
+            r["base"] = eng.base_words(r["title"], r["key"])
+marks = {r["path"] for r in erows if eng.is_watermark(r["path"], r["bytes"])}
+extras = {r["path"] for r in erows
+          if r["path"] not in marks
+          and eng.is_extra(r["path"], r["title"], r["dur"], ROOT,
+                           matched=r["path"] in PLEX_T)}
+eps = collections.defaultdict(list)
+for r in erows:
+    if r["path"] in marks | extras:
+        continue
+    ep = eng.ep_of(r["path"])
+    if ep:
+        eps[ep[0]].append(r)      # keyed by the SERIES, never the episode title
+tv_keys = {k for k, v in eps.items() if len(v) >= 5}
+tv_set = {r["path"] for k, v in eps.items() if k in tv_keys for r in v}
+# A folder INSIDE a movie's folder is bonus territory even when its own name is
+# not bonus vocabulary: "Saboteur (1942)/Exrtas/Saboteur - A Closer Look" is a
+# misspelled Extras folder, so the engine's folder rule never fired and the
+# featurette became a movie folder of its own — whose words then out-scored the
+# real Saboteur and drew that box set's other extras into it. The rule needs no
+# metadata: the folder holding the longest film owns the shorter video filed
+# under it, provided that shorter file's own name speaks the film's name — a
+# whole other film filed there stays a film, exactly as the engine's folder rule
+# exempts it. A different year in the file's own name IS that other film
+# speaking, so it vetoes the rule.
+def longest_of(rows):
+    """The longest video in each folder: that folder's feature."""
+    out = {}
+    for r in rows:
+        if r["dur"] > 0:
+            d = os.path.dirname(r["path"])
+            if d not in out or r["dur"] > out[d]["dur"]:
+                out[d] = r
+    return out
+
+
+def not_disc(r):
+    """A disc tree's innards belong to the tree, never to the folder's film."""
+    return not any(c in DISC_DIRS for c in r["path"].split("/"))
+
+
+FEATURE_OF = longest_of([r for r in erows if r["path"] not in marks and not_disc(r)])
+
+
+def under_a_feature(path, dur):
+    """True when a folder above this file holds a longer film that this file's
+    own name names."""
+    d = os.path.dirname(os.path.dirname(path))
+    while len(d) > len(ROOT):
+        f = FEATURE_OF.get(d)
+        if f:
+            if f["dur"] < dur:
+                return False
+            theirs = (words_of(os.path.basename(f["path"]))
+                      | words_of(os.path.basename(d)))
+            if not words_of(os.path.basename(path)) & theirs:
+                return False
+            y1 = re.search(r"(?:18|19|20)\d\d", os.path.basename(path))
+            y2 = re.search(r"(?:18|19|20)\d\d", os.path.basename(f["path"]))
+            return not (y1 and y2 and y1.group() != y2.group())
+        d = os.path.dirname(d)
+    return False
+
+
+extras |= {r["path"] for r in erows
+           if r["dur"] > 0 and r["path"] not in marks | extras | tv_set
+           and not_disc(r) and under_a_feature(r["path"], r["dur"])}
+film_rows = [r for r in erows if r["path"] not in marks | extras | tv_set]
+
+buckets = collections.defaultdict(list)
+for r in film_rows:
+    buckets[eng.key_of(r["title"])].append(r)
+
+
+def res_label(r):
+    m = RES_RE.search(r["res"] or "")
+    if m:
+        n, kind = m.group(1).lower(), m.group(2).lower()
+        return "2160p" if n in ("2160", "4k") else f"{n}{kind}"
+    try:
+        w = int((r["res"] or "0x0").split("x")[0])
+    except (ValueError, IndexError):
+        w = 0
+    return "2160p" if w >= 3000 else ("1080p" if w >= 1400 else "SD")
+
+
+def version_label(r):
+    bits = [res_label(r)]
+    vc = (r["vcodec"] or "").lower()
+    if vc in ("av1", "libaom", "libsvtav1"):
+        bits.append("AV1")
+    elif vc in ("vc1", "wmv3"):
+        bits.append("VC1")
+    if r["bytes"] > 40 * 2**30:
+        bits.append("REMUX")
+    if "imax" in r["path"].lower():
+        bits.append("IMAX")
+    return sanitize(" ".join(bits))
+
+
+def group_name(items):
+    titles = collections.Counter(r["title"] for r in items)
+    ascii_t = [t for t in titles if t.isascii()]
+    title = (collections.Counter(ascii_t) if ascii_t else titles).most_common(1)[0][0]
+    years = [r["year"] for r in items if r["year"] != "?"] or \
+            [r["soft"] for r in items if r["soft"]]
+    year = collections.Counter(years).most_common(1)[0][0] if years else None
+    return sanitize(f"{title} ({year})" if year else title)
+
+
+groups = []
+for key, items in buckets.items():
+    for c in eng.dur_clusters(items):
+        g = {"rows": c, "base": group_name(c)}
+        known = [PLEX_T[r["path"]] for r in c if r["path"] in PLEX_T]
+        if known:
+            # one film, one name: a Plex match for any copy names every copy
+            t, yr = collections.Counter(known).most_common(1)[0][0]
+            g["plex"] = t
+            g["base"] = sanitize(f"{t} ({yr})" if ok_year(yr) else t)
+        groups.append(g)
+clash = {b for b, n in collections.Counter(g["base"] for g in groups).items() if n > 1}
+for g in groups:
+    if g["base"] in clash:
+        # two genuinely different films share title+year: separate folders, and
+        # say so in the name, because Jellyfin can only match one item per folder
+        g["base"] = sanitize(g["base"] + " - " + version_label(g["rows"][0]))
+        g["flag"] = "same title and year as another film"
+
+moves, mkdirs, problems, readmes = [], set(), [], []
+# --------------------------------------------------------------- unnamed groups
+# A movie folder has to READ as a title. The engine's runtime veto says any file
+# over 45 minutes that is not named like an extra is a film whatever it is called
+# — true for Jellyfin, but a 118-minute documentary named "Extra 1" then produced
+# a movie folder named "1", and folders named "1", "2", "3" are Jellyfin items
+# with no identity whose numeric keys went on to capture every subtitle named
+# "3_Eng.srt" in the library. A year proves a dated feature ("300 (2006)" is a
+# real film), so the gate is: a year, or a word that is not release noise. With
+# neither, the rows are bonus material until a human names them.
+def named(g):
+    if re.search(r"\(\d{4}\)$", g["base"]):
+        return True
+    return any(w.isalpha() and len(w) >= 2 for w in eng.name_words(g["base"]))
+
+
+unnamed = [g for g in groups if not named(g)]
+groups = [g for g in groups if named(g)]
+for g in unnamed:
+    for r in g["rows"]:
+        extras.add(r["path"])
+        problems.append(
+            f"UNNAMED film {rel(r['path'])} parses to {g['base']!r} with no year: "
+            f"bonus pile, not a folder — give the file a real name and re-plan")
+
+
+group_of = {}
+for g in groups:
+    g["dir"] = f"{ROOT}/{g['base']}"
+    for r in g["rows"]:
+        group_of[r["path"]] = g
+        dr = disc_root(r["path"])
+        if dr:
+            group_of[dr] = g
+by_path = {r["path"]: r for r in erows}
+
+# -------------------------------------------------------------- bookkeeping
+claimed = {}
+existing = set(rows_map := {r["path"] for r in rows})
+
+
+at_target, booked_src = set(), set()
+
+
+def book(src, dst, kind):
+    if src == dst:
+        at_target.add(src)
+        booked_src.add(src)
+        return
+    if os.environ.get("JF_TRACE") and (src in booked_src or dst.lower() in claimed):
+        why = "rebook" if src in booked_src else "collision"
+        problems.append(f"TRACE {why} kind={kind} src={rel(src)} dst={rel(dst)}")
+    if src in booked_src:          # one source may only ever move once
+        problems.append(f"RE-BOOK {rel(src)} -> {rel(dst)} (already planned)")
+        return
+    if dst.lower() in claimed:
+        problems.append(f"COLLISION {rel(dst)}\n    from {rel(src)}\n    and  {rel(claimed[dst.lower()])}")
+        return
+    claimed[dst.lower()] = src
+    booked_src.add(src)
+    comp = dst.split("/")
+    for i in range(1, len(comp)):
+        mkdirs.add("/".join(comp[:i]))
+    moves.append((src, dst, kind))
+
+
+def uniq_dir(d, stem, ext, hint=""):
+    """A destination that no other source already took."""
+    cand = f"{d}/{stem}{ext}"
+    if cand.lower() not in claimed:
+        return cand
+    n = 2
+    while True:
+        cand = f"{d}/{stem} {n}{ext}"
+        if cand.lower() not in claimed:
+            return cand
+        n += 1
+
+
+def media_dir(p):
+    dr = disc_root(p)
+    return os.path.dirname(dr) if dr else os.path.dirname(p)
+
+
+# ------------------------------------------------------------------ films
+# film / disc moves: the largest copy carries the folder's own name, the rest
+# become Jellyfin versions; a disc tree moves as a whole into the movie folder
+
+
+def vext(p):
+    return ext_by.get(p) or os.path.splitext(p)[1].lower()
+disc_children = collections.defaultdict(list)
+for q in rows:
+    dr = disc_root(q["path"])
+    if dr:
+        disc_children[dr].append(q["path"])
+for g in groups:
+    primary_taken = False
+    disc_kinds = set()
+    for r in sorted(g["rows"], key=lambda r: -r["bytes"]):
+        dr = disc_root(r["path"])
+        if dr:
+            inner = os.path.basename(dr).upper()
+            if inner in disc_kinds:
+                problems.append(
+                    f"DISC-VERSION {g['base']}: extra disc tree {rel(dr)} cannot "
+                    f"share the folder — discs have no multiple versions")
+                continue
+            disc_kinds.add(inner)
+            for src in disc_children[dr]:
+                tail = os.path.relpath(src, dr)
+                fold = disc_root(src, fold=False)
+                if fold != dr and os.path.dirname(src) != dr:
+                    tail = os.path.basename(dr) + "/" + tail
+                book(src, f"{g['dir']}/{tail}", "disc-file")
+            continue
+        label = "" if not primary_taken else f" - {version_label(r)}"
+        dst = f"{g['dir']}/{g['base']}{label}{vext(r['path'])}"
+        n = 2
+        while dst.lower() in claimed:
+            dst = f"{g['dir']}/{g['base']}{label} {n}{vext(r['path'])}"
+            n += 1
+        r["dst"] = dst
+        primary_taken = True
+        book(r["path"], dst, "film")
+    g["primary"] = primary_taken
+
+# ------------------------------------------------------------------ extras
+EXTRA_KIND = [("trailers", r"trailer|teaser|preview|promo|tv spot|advert"),
+              ("deleted scenes", r"deleted|alt scene|gag reel|blooper|outtake"),
+              ("interviews", r"interview|cast and crew|session piece|commentary"),
+              ("featurettes", r"featurette|behind the scenes|making of|documentary"
+                              r"|heritage|archival|video archive|music video"),
+              ("samples", r"sample"),
+              ("shorts", r"\bshort\b|short film"),
+              ("clips", r"\bclip\b|scene selection|montage|epilogue|isolated score"),
+              ("other", r"menu|logo|gallery|still|trivia|storyboard|screen test"
+                        r"|audition|test footage|watermark|workprint|poster"
+                        r"|picture in picture|singalong|karaoke|original score")]
+THEME_RE = re.compile(r"theme|opening song|ending song|soundtrack|score"
+                      r"|\bost\b|music|song", re.I)
+BACKDROP_RE = re.compile(r"backdrop|intro", re.I)
+
+
+def extras_kind(path):
+    low = path.lower()
+    if THEME_RE.search(low):
+        return "theme-music"
+    if BACKDROP_RE.search(low):
+        return "backdrops"
+    for name, rx in EXTRA_KIND:
+        if re.search(rx, low):
+            return name
+    return "extras"
+
+
+def attachable_title(path):
+    t, _y = eng.title_of(path)
+    return eng.key_of(t) if t and not eng.is_watermark(path, size_by.get(path, 0)) else None
+
+
+for g in groups:
+    g["tops"] = {parts_of(r["path"])[0] for r in g["rows"]}
+top_groups = collections.defaultdict(list)
+for g in groups:
+    for t in g["tops"]:
+        top_groups[t].append(g)
+group_by_key = collections.defaultdict(list)
+for g in groups:
+    for r in g["rows"]:
+        group_by_key[eng.key_of(r["title"])].append(g)
+group_by_dir = {}
+for g in groups:
+    for r in g["rows"]:
+        group_by_dir.setdefault(media_dir(r["path"]), g)
+        dr = disc_root(r["path"])
+        if dr:
+            group_by_dir.setdefault(os.path.dirname(dr), g)
+
+
+def group_words(g):
+    """The words that identify a film group: its titles and the names of the
+    folders its files live in."""
+    if "words" not in g:
+        w = set()
+        if g.get("plex"):
+            w |= words_of(g["plex"])
+        for r in g["rows"]:
+            w |= words_of(r["title"])
+            w |= words_of(os.path.basename(media_dir(r["path"])))
+        g["words"] = w
+    return g["words"]
+
+
+def score_words(text, groups):
+    """Ranks film groups by how many of their words a path carries."""
+    words = words_of(text)
+    out = [(len(words & group_words(g)), g) for g in groups]
+    out.sort(key=lambda x: -x[0])
+    return out
+
+
+def group_for_orphan(path):
+    """Which movie folder an extra/sidecar belongs to. A collection folder holding
+    several films must be resolved by WORDS, never by position: taking the first
+    film of the folder attached every loose subtitle in a box set to whichever
+    movie happened to sort first."""
+    stem = os.path.splitext(os.path.basename(path))[0]
+    key = attachable_title(path)
+    if key and group_by_key.get(key):
+        cands = group_by_key[key]
+        top = parts_of(path)[0] if len(parts_of(path)) > 1 else ""
+        same_top = [g for g in cands if any(parts_of(r["path"])[0] == top
+                                            for r in g["rows"] if len(parts_of(r["path"])) > 1)]
+        pool = same_top or cands
+        return pool[0] if len(pool) == 1 else score_words(stem, pool)[0][1]
+    gtop = parts_of(path)[0] if len(parts_of(path)) > 1 else ""
+    pool = top_groups.get(gtop, [])
+    if len(pool) > 1:                    # a loose collection folder (box set, bonus
+        n, g = score_words(stem + " " + os.path.dirname(path), pool)[0]
+        return g if n >= 2 else None     # album) is resolved by words, not position
+    if len(pool) == 1:
+        return pool[0]
+    d = os.path.dirname(path)
+    while d and d != ROOT:
+        if d in group_by_dir:
+            return group_by_dir[d]
+        d = os.path.dirname(d)
+    return None
+
+
+tv_tops = {parts_of(p)[0] for p in tv_set}
+def film_folder_for(path):
+    """A movie folder for a lone feature that has no group: derived from its own
+    title when a year proves it, else None (goes to the review pile)."""
+    t, y = eng.title_of(path)
+    if not t or y == "?":
+        return None
+    d = uniq_dir(ROOT, sanitize(f"{t} ({y})"), "", "")
+    mkdirs.add(d)
+    return d
+
+
+for p in sorted(extras):
+    if disc_root(p):
+        continue
+    if len(parts_of(p)) > 1 and parts_of(p)[0] in tv_tops:
+        continue                        # a special inside a TV set: staged with it
+    g = group_for_orphan(p)
+    ext = ext_by.get(p, os.path.splitext(p)[1].lower())
+    stem = os.path.splitext(os.path.basename(p))[0]
+    if g is None:
+        solo = film_folder_for(p)
+        if solo:
+            dst = f"{solo}/{sanitize(stem)}{ext}"
+            problems.append(f"REVIEW lone feature {rel(p)}: gets its own folder "
+                            f"{os.path.basename(solo)}")
+        else:
+            dst = f"{ROOT}/_unsorted bonus/{extras_kind(p)}/{sanitize(stem)}{ext}"
+            problems.append(f"NO-FILM extra {rel(p)}")
+    else:
+        dst = f"{g['dir']}/{extras_kind(p)}/{sanitize(stem)}{ext}"
+    book(p, uniq_dir(os.path.dirname(dst), os.path.basename(dst)[:-(len(ext) or 1)],
+                     ext, os.path.dirname(p)), "extra")
+
+# Disc trees the engine never saw (a MakeMKV scratch folder, an AACS remnant):
+# they belong to the film of the folder they sit in, kept as a named subtree.
+for q in rows:
+    pth = q["path"]
+    dr = disc_root(pth)
+    if not dr or pth in by_path or pth in booked_src:
+        continue
+    g = group_by_dir.get(dr) or (top_groups[parts_of(pth)[0]][0]
+                                 if len(top_groups.get(parts_of(pth)[0], [])) == 1 else None)
+    if g is None:
+        continue
+    inner = os.path.basename(dr)
+    problems.append(f"REVIEW disc remnant {rel(dr)} moves inside {g['base']} "
+                    f"(a second master: compare before keeping both)")
+    for src in disc_children[dr]:
+        if src not in booked_src:
+            book(src, f"{g['dir']}/{inner}/{os.path.relpath(src, dr)}", "disc-file")
+
+
+# ----------------------------------------------------------------- sidecars
+media_by_dir = collections.defaultdict(list)
+for r in erows:
+    if r["path"] in marks:
+        continue
+    dst = r.get("dst")
+    if dst:
+        media_by_dir[media_dir(r["path"])].append(r)
+
+
+
+def sidecar_target(path, e):
+    d = os.path.dirname(path)
+    stem = os.path.splitext(os.path.basename(path))[0]
+    if d == ROOT:
+        # A file loose in the library ROOT names no film. media_by_dir[ROOT] is the
+        # set of root-level MOVIE files, and scoring a stray name against it
+        # attached every root-level leftover to whichever root-level film was
+        # largest — the plan sent ".DS_Store" into "Titanic (1997)/other/" that
+        # way. A root-level stray is nobody's sidecar: it goes to the review pile.
+        return None, None
+    cands = media_by_dir.get(d, [])
+    if not cands:
+        nd = os.path.dirname(d)          # a subs/ or Extras/ folder beside the media
+        while nd and nd != ROOT and not media_by_dir.get(nd):
+            nd = os.path.dirname(nd)
+        if media_by_dir.get(nd):
+            if nd != d:                  # a NAMED subfolder (Subs/, Extras/, the
+                g = group_for_orphan(path)   # box set's own root) carries evidence
+                dirs = {os.path.dirname(r.get("dst", "")) for r in media_by_dir[nd]}
+                if g is None or g["dir"] not in dirs:
+                    # ...but only a STRONG word match outranks the folder the file
+                    # sits in. A weak one is an accident: "3_Eng.srt" scored 1
+                    # against a junk group named "3" and escaped its own movie's
+                    # Subs/ folder, while "03 xXx 3 Return Of Xander Cage" scores 4
+                    # and genuinely belongs to the other film of the set.
+                    n = score_words(stem, [g])[0][0] if g else 0
+                    if n >= 2:
+                        return (g, None)
+                cands = media_by_dir[nd]
+    if not cands:
+        g = group_for_orphan(path)
+        return (g, None) if g else (None, None)
+    if len(cands) > 1:
+        w = [t for t in eng.norm(stem).split() if t not in NOISE]
+        cands = sorted(cands, key=lambda r: (
+            -len([t for t in eng.norm(os.path.basename(r["path"])).split() if t in w]),
+            -r["bytes"]))
+    return (None, cands[0])
+
+
+for q in rows:
+    p, e = q["path"], q["ext"]
+    if p in by_path or p in extras or p in marks or is_qb(p) or disc_root(p):
+        continue
+    if os.path.basename(p).startswith("._"):
+        # macOS AppleDouble resource forks: a few hundred bytes carrying a name
+        # COPIED FROM ANOTHER FILM. Word-matching them to "their" film scattered
+        # dead files through four movie folders (and they are video extensions in
+        # the AppleDouble case, so they also skipped every other filter). They
+        # claim nothing: one pile, and they stay in it — nothing is deleted here.
+        book(p, uniq_dir(f"{ROOT}/_macos sidecars", sanitize(os.path.basename(p))[:120],
+                         "", os.path.dirname(p)), "junk")
+        problems.append(f"JUNK AppleDouble sidecar {rel(p)} (macOS resource fork)")
+        continue
+    if e == ".iso":         # a DVD image: Jellyfin plays it as a movie file
+        g = group_for_orphan(p)
+        stem = os.path.splitext(os.path.basename(p))[0]
+        if g is not None:
+            book(p, uniq_dir(g["dir"], g["base"], e, stem) if g.get("primary") is not None
+                 else f"{g['dir']}/{g['base']}{e}", "iso")
+        else:
+            solo = film_folder_for(p)
+            if solo:
+                book(p, f"{solo}/{sanitize(stem)}.iso", "iso")
+                problems.append(f"REVIEW disc image {rel(p)}: DVD rip, alone in its "
+                                f"folder -> {os.path.basename(solo)}")
+            else:
+                book(p, f"{ROOT}/_unsorted files/{sanitize(os.path.basename(p))}", "orphan")
+                problems.append(f"NO-FILM disc image {rel(p)}")
+        continue
+    if e in VIDEO_EXT:
+        # A video the engine never probed (unreadable header) and not an AppleDouble
+        # remnant: the plan does not guess at it, so it stays where it is.
+        continue
+    stem = os.path.splitext(os.path.basename(p))[0]
+    g, media = sidecar_target(p, e)
+    if media is not None:
+        base = os.path.splitext(os.path.basename(media["dst"]))[0]
+        d = os.path.dirname(media["dst"])
+    elif g is not None:
+        base, d = g["base"], g["dir"]
+    else:
+        # Nothing names this file's film. NEVER scatter it: an archive set or an
+        # album tree must move as the unit it is, so the whole loose folder goes
+        # to the review pile and a film folder's own leftovers stay with it.
+        ps = parts_of(p)
+        if len(ps) < 2:
+            dst = f"{ROOT}/_unsorted files/{sanitize(stem)}{e}"
+        elif len(top_groups.get(ps[0], [])):
+            dst = f"{top_groups[ps[0]][0]['dir']}/other/{'/'.join(sanitize(c) for c in ps[1:])}"
+        else:
+            dst = f"{ROOT}/_unsorted files/{'/'.join(sanitize(c) for c in ps)}"
+        book(dst=dst, src=p, kind="orphan")
+        problems.append(f"NO-OWNER sidecar {rel(p)}")
+        continue
+    if e in IMG_EXT:
+        low = eng.norm(stem)
+        word = next((w for w in IMG_PRIMARY if re.search(rf"\b{w}\b", low)), None)
+        kind = "poster" if word else next(
+            (IMG_OTHER[w] for w in IMG_OTHER if re.search(rf"\b{w}\b", low)), None)
+        if re.search(r"\btheme\b", low):
+            dst = f"{d}/theme{e}"
+        else:
+            dst = f"{d}/{base}-{kind or 'art'}{e}" if kind else f"{d}/{sanitize(stem)}{e}"
+        book(p, uniq_dir(os.path.dirname(dst), os.path.basename(dst)[:-(len(e) or 1)],
+                         e, os.path.dirname(p)), "image")
+    elif e in NFO_EXT:
+        book(p, uniq_dir(d, base, ".nfo", os.path.dirname(p)), "nfo")
+    elif e in SUB_EXT | AUDIO_EXT:
+        toks = [t for t in re.split(r"[._\-\s]+", stem) if t]
+        langs = [t.lower() for t in toks if t.lower() in LANGS]
+        flags = [t.lower() for t in toks if t.lower() in FLAGS]
+        keep = [sanitize(t) for t in toks
+                if t.lower() not in langs + flags + list(NOISE)
+                and not re.fullmatch(r"\d{3,4}p?|cd\d+|\d{4}|\d+ch|\d+.\d", t.lower())]
+        bits = [base] + ([sanitize(" ".join(keep[:4]))] if keep else []) + langs + flags
+        if e in SUB_EXT:
+            # subtitles: <movie>/srt[/language]
+            d = f"{d}/srt"
+            pd = os.path.basename(os.path.dirname(p))
+            if SUB_KEEP_RE.match(pd) and not SUB_DIR_OK.match(pd):
+                d = f"{d}/{sanitize(pd)}"      # a real language folder survives
+        elif e in AUDIO_EXT and AUDIO_DIR_RE.search(os.path.dirname(p)):
+            d = f"{d}/theme-music"                 # bonus audio: one canonical folder
+            pd = os.path.basename(os.path.dirname(p))
+            if not MUSIC_TOP_RE.search(pd):        # an album folder keeps its name
+                d = f"{d}/{sanitize(pd)}"
+        else:
+                pd = os.path.basename(os.path.dirname(p))
+                if not AUDIO_DIR_OK.match(pd):
+                    d = f"{d}/{sanitize(pd)}"
+        stem2 = ".".join(bits) or sanitize(os.path.basename(p))
+        book(p, uniq_dir(d, stem2, e, os.path.basename(p)), "track")
+    else:
+        book(p, uniq_dir(f"{d}/other", sanitize(os.path.basename(p))[:120], e,
+                         os.path.dirname(p)), "misc")
+
+readmes.append(f"README in {ROOT}: this library was relaid out into Jellyfin's "
+               "one-folder-per-movie layout; `_release-group watermarks/`, "
+               "`_unsorted files/` and `_unsorted bonus/` hold files no film claims.")
+for key in tv_keys:
+    readmes.append(f"README in {ROOT}/{sanitize('_TV needs a TV library - ' + key)}: "
+                   "these are TV episodes found in the MOVIES library; move the folder "
+                   "to a TV library source, else Jellyfin groups them wrongly as one "
+                   "movie.")
+
+# --------------------------------------------------------------------- TV
+for key in tv_keys:
+    items = eps[key]
+    year = collections.Counter(r["year"] for r in items if r["year"] != "?")
+    yr = year.most_common(1)[0][0] if year else None
+    base = f"_TV needs a TV library - {sanitize(key)}" + (f" ({yr})" if yr else "")
+    tops = {parts_of(r["path"])[0] for r in items}
+    members = sorted([r["path"] for r in items]
+                     + [q for q in extras | marks if len(parts_of(q)) > 1
+                        and parts_of(q)[0] in tops],
+                     key=lambda q: q)
+    for src in members:
+        rel_parts = rel(src).split("/")
+        tail = "/".join(sanitize(c) for c in rel_parts[:-1] + [rel_parts[-1]])
+        book(src, f"{ROOT}/{base}/{tail}", "tv")
+for p in sorted(marks):
+    stem, ext = os.path.splitext(os.path.basename(p))
+    book(p, uniq_dir(f"{ROOT}/_release-group watermarks", sanitize(stem)[:120], ext,
+                     os.path.dirname(p)), "watermark")
+for q in rows:
+    if is_qb(q["path"]):
+        problems.append(f"SKIPPED qBittorrent partial {rel(q['path'])}")
+        book(q["path"], q["path"], "qb-partial")     # stays: resume data
+
+# --------------------------------------------------------------- debug hook
+if os.environ.get("JF_ISO"):
+    x = ROOT + "/High Anxiety (Mel Brooks) [1977] nordic subs  DVDrip/High Anxiety [1977].iso"
+    print("attachable", attachable_title(x), "| dir", os.path.dirname(x) in group_by_dir,
+          "| g", group_for_orphan(x) and group_for_orphan(x)["base"])
+    raise SystemExit
+if os.environ.get("JF_DEBUG"):
+    import json
+    unp = sorted({r["path"] for r in rows} - {m[0] for m in moves} - at_target)
+    print("at_target", len(at_target))
+    print("kinds-unplanned", collections.Counter(os.path.splitext(u)[1] for u in unp).most_common(8))
+    for up in unp[:20]:
+        d = os.path.dirname(up)
+        print("UNP", up[:90], "| mediadir?", d in media_by_dir, "| by_path", up in by_path,
+              "| extras", up in extras, "| discroot", disc_root(up), "| ext", os.path.splitext(up)[1])
+
+# -------------------------------------------------------------- validation
+srcs = [m[0] for m in moves]
+dup_src = {s for s, n in collections.Counter(srcs).items() if n > 1}
+for s in dup_src:
+    problems.append(f"DOUBLE-CLAIM source {rel(s)}")
+all_src = set(srcs)
+untouched = {r["path"] for r in rows} - all_src
+bad_names, into_alive, lost = [], [], []
+for src, dst, kind in moves:
+    for comp in dst[len(ROOT) + 1:].split("/"):
+        if (BAD_CHARS.search(comp) or comp != comp.strip(". ") or not comp
+                or len(comp) > 200):
+            bad_names.append(f"{kind}: {comp!r} in {rel(dst)}")
+    if dst in untouched:                       # something staying put is there
+        into_alive.append(f"{rel(src)} -> {rel(dst)} (occupied by a file that stays)")
+    if dst.lower() in {s.lower() for s in untouched}:
+        into_alive.append(f"{rel(src)} -> {rel(dst)} (case-collides with a survivor)")
+    if dst.startswith(src + "/") and src in all_src:
+        lost.append(f"{rel(src)} moves into its own subtree")
+# every accounted-for source must exist exactly once in the plan
+covered = {r["path"] for r in rows}
+not_planned = sorted(covered - all_src - at_target)
+
+with open(PLAN, "w") as out:
+    out.write("action\tsrc\tdst\tkind\n")
+    for d in sorted(mkdirs, key=lambda q: (q.count("/"), q)):
+        if d.startswith(ROOT + "/") and d != ROOT:
+            out.write(f"mkdir\t\t{d}\tdir\n")
+    for p in sorted(booked_src - {m[0] for m in moves}):
+        out.write(f"keep\t{p}\t{p}\n")
+    for src, dst, kind in sorted(moves, key=lambda m: m[0]):
+        out.write(f"move\t{src}\t{dst}\t{kind}\n")
+
+kinds = collections.Counter(k for _s, _d, k in moves)
+films_n = sum(1 for _s, _d, k in moves if k == "film")
+folders = len({os.path.dirname(d) for _s, d, k in moves if k in ("film", "disc-file")})
+with open(EXCEPTS, "w") as out:
+    out.write(f"groups={len(groups)} moves={len(moves)} folders={folders}\n")
+    out.write(f"kinds={dict(kinds)}\n")
+    out.write(f"at-target-already={len(at_target)} unplanned={len(not_planned)}\n")
+    out.write("\n=== WHAT THE EXECUTOR MUST ALSO WRITE ===\n")
+    out.write("\n".join(sorted(set(readmes))) + "\n")
+    out.write("\n=== PROBLEMS ===\n")
+    out.write("\n".join(sorted(set(problems))) + "\n")
+    out.write("\n=== BAD NAMES ===\n" + "\n".join(sorted(set(bad_names))) + "\n")
+    out.write("\n=== TARGETS OCCUPIED BY SURVIVORS ===\n"
+              + "\n".join(sorted(set(into_alive))) + "\n")
+    out.write("\n=== LEFTOVER SOURCES (first 60) ===\n"
+              + "\n".join(rel(p) for p in not_planned[:60]) + "\n")
+print(f"groups={len(groups)} folders={folders} moves={len(moves)} "
+      f"mkdirs={len(mkdirs)} film-files={films_n}")
+print(f"kinds={dict(kinds)}")
+print(f"problems={len(set(problems))} bad-names={len(set(bad_names))} "
+      f"occupied={len(set(into_alive))} double-claim={len(dup_src)}")
+print(f"left-unmoved={len(not_planned)}")
+print(f"tv-keys={len(tv_keys)} extras={len(extras)} marks={len(marks)} "
+      f"disc-trees={len(disc_children)}")

scripts/media-shows.py

@@ -0,0 +1,705 @@
+#!/usr/bin/env python3
+r"""Plan a Jellyfin-format relayout of a TV library. DRY RUN ONLY: nothing moves.
+
+ media-shows.py [--root DIR] [--enum TSV] [--titles TSV] [--probed TSV]
+                [--tmdb-cache JSON] [--tmdb] [--movies DIR] [--park DIR]
+                [--out TSV] [--exceptions TXT]
+
+Inputs are the TSVs scripts/media-census.py exports: `enumerate` lists what
+exists, `jellyfin` says which episode each file IS, and `probe` is optional
+duration evidence for the movie rule. The TMDb cache is a plain JSON file so a
+plan stays reproducible: answers are read from it, and a miss is looked up only
+with --tmdb and TMDB_API_KEY in the environment (the key never enters argv, and
+never the cache).
+
+Target contract (jellyfin.org/docs/general/server/media/shows):
+
+    <root>/<Series> (<year>)/Season <NN>/<Series> SxxEyy.ext
+    one file holding two episodes   SxxE01-E02      specials       Season 00
+    a documented part of an episode SxxE03 Part 1   sidecars       <base>.<lang>.srt
+    bonus material                  <series>/trailers|featurettes|… — never a season
+
+PLAN is the TSV scripts/media-apply.py replays: `action src dst kind`, every
+mkdir row before any move, and `keep` for a file already where the contract puts
+it. EXCEPTIONS is the review list: everything parked, and why.
+
+The authority chain, strongest first, for one video file:
+
+  1. a provider row — Jellyfin matched this exact path and numbered it.
+  2. the file's own SxxEyy / 1x04 marker, the series taken from a sibling's
+     provider row, else from the nearest ancestor folder that has one, else from
+     the release folder's own words.
+  3. nothing. An unnumbered file is PARKED under --park with its relative path
+     kept: an invented `Season 01` steals an episode number from the file that
+     really is S01E01, and a wrong number is worse than no number.
+
+Guards, and why each is here:
+
+  * One series, one folder, one year: the year ANY provider row of a series
+    states is shared by all of them, so a box set split across five release
+    folders lands inside ONE series folder instead of five.
+  * Two files claiming one episode: the bigger keeps the standard name, the other
+    parks with the reason. Nothing is overwritten and nothing is deleted — which
+    encode survives is Frank's call, and a parked file is one rename from home.
+  * A file with no episode evidence that runs long and that the MOVIE provider
+    names leaves the TV library for --movies under the movie naming contract.
+    Long and unnamed parks; short and unnumbered is bonus material.
+  * Series art and the show nfo follow the SERIES, a season image follows its
+    SEASON, a subtitle or episode nfo follows the episode's new base — an episode
+    moved without its subtitles is a half-move.
+  * Hidden files, macOS forks, qBittorrent resume parts and the park pile itself
+    are never moved. Skipping the park is what makes re-planning stable.
+
+The plan crosses three trees (the library, its park, the movie library), so the
+executor runs with --root at the share that holds them, on its own journal —
+one journal is one undo:
+
+    python3 scripts/media-shows.py --root /mnt/entropy/TVShows \
+        --enum /tmp/tv-enum.tsv --titles /tmp/jellyfin-tv.tsv \
+        --out /mnt/entropy/media-tv-plan.tsv
+    python3 scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv \
+        --root /mnt/entropy --journal /mnt/entropy/media-tv-relayout.jsonl
+"""
+import argparse
+import collections
+import importlib.util
+import json
+import os
+import re
+import sys
+import urllib.parse
+import urllib.request
+
+ARGS = argparse.ArgumentParser(description=__doc__.splitlines()[0],
+                               formatter_class=argparse.RawDescriptionHelpFormatter)
+ARGS.add_argument("--root", default=os.environ.get("TV_ROOT", "/mnt/entropy/TVShows"),
+                  help="the TV library the plan reorganises")
+ARGS.add_argument("--enum", default="/tmp/tv-enum.tsv",
+                  help="file census (media-census.py enumerate)")
+ARGS.add_argument("--titles", default="/tmp/jellyfin-tv.tsv",
+                  help="the provider's numbering (media-census.py jellyfin)")
+ARGS.add_argument("--probed", default="",
+                  help="ffprobe census for the movie rule; absent = no durations")
+ARGS.add_argument("--tmdb-cache", default="/tmp/tmdb-cache.json",
+                  help="TMDb answers as an input file, so a plan is reproducible")
+ARGS.add_argument("--tmdb", action="store_true",
+                  help="look up cache misses (needs TMDB_API_KEY)")
+ARGS.add_argument("--movies", default="/mnt/entropy/Movies",
+                  help="movie library a mis-filed film moves into")
+ARGS.add_argument("--park", default="",
+                  help="review pile (default: <root>/_unsorted)")
+ARGS.add_argument("--out", default="/tmp/tv-plan.tsv")
+ARGS.add_argument("--exceptions", default="/tmp/tv-exceptions.txt")
+ARGS = ARGS.parse_args()
+
+ROOT = os.path.abspath(ARGS.root)
+PARK = os.path.abspath(ARGS.park or os.path.join(ROOT, "_unsorted"))
+MOVIES = os.path.abspath(ARGS.movies)
+ENGINE = os.path.join(os.path.dirname(os.path.abspath(__file__)), "media-titles.py")
+_spec = importlib.util.spec_from_file_location("eng", ENGINE)
+eng = importlib.util.module_from_spec(_spec)
+_spec.loader.exec_module(eng)
+
+try:
+    CACHE = json.load(open(ARGS.tmdb_cache))
+except (OSError, ValueError):
+    CACHE = {}
+API_KEY = os.environ.get("TMDB_API_KEY", "")
+
+VIDEO_EXT = {".mkv", ".mp4", ".avi", ".m4v", ".mov", ".ts", ".m2ts", ".mpg",
+             ".mpeg", ".webm", ".wmv", ".divx", ".ogm", ".flv", ".vob", ".rm",
+             ".rmvb", ".iso", ".img"}
+SUB_EXT = {".srt", ".ass", ".ssa", ".sub", ".vtt"}
+IMG_EXT = {".jpg", ".jpeg", ".png", ".webp", ".bmp", ".gif", ".tbn"}
+NFO_EXT = {".nfo", ".xml"}
+# The container's own word, for a sidecar named "episode.mkv.srt": that component
+# is not a language flag, and both matchers read the plain form better.
+CONTAINER = {e[1:] for e in VIDEO_EXT}
+# Files that are never anybody's business: hidden, forks, resume state, thumbnails.
+SKIP_NAME = re.compile(r"^(\.|#|dl_state|@eaDir|Thumbs\.db|desktop\.ini)|\.!q$")
+# Show-level metadata keeps its own name at the series root; season-level at the
+# season root; folder.jpg stays with the folder it was written for.
+SERIES_META = re.compile(r"^(tvshow|series|show)(\.nfo|\.xml)?$"
+                         r"|^(poster|tvposter|banner|clearlogo|clearart|logo|art"
+                         r"|landscape|fanart|showbackdrop|backdrop)\d*\."
+                         r"(jpg|jpeg|png|webp|bmp)$", re.I)
+SEASON_META = re.compile(r"^season\.nfo$", re.I)
+FOLDER_IMAGE = re.compile(r"^folder\.(jpg|jpeg|png|webp|bmp)$", re.I)
+SEASON_DIR_RE = re.compile(r"(?i)^\s*(?:season[\s._-]*(\d{1,3})|s(\d{1,2}))\s*$")
+BAD_CHARS = re.compile(r'[<>:"/|?*]')
+# SxxEyy with an optional second episode, plus the 1x04 form. The second number
+# is guarded so a resolution never reads as an episode: "S01E01-1080p" is ONE
+# episode, and the longest-match backtracking of \d{1,3} is what the (?![\dpP])
+# after it closes off.
+EP_RANGE = re.compile(r"(?i)(?<![a-z0-9])s(?:eason[\s._-]*)?(\d{1,2})[\s._-]*"
+                      r"e(?:p(?:isode)[\s._-]*)?(\d{1,3})"
+                      r"(?:[\s._-]*(?:-|to[\s._-]*|ep)[\s._-]*(?:s\d{1,2}[\s._-]*)?e?"
+                      r"(\d{1,3})(?![\dpP]))?")
+NX_RE = re.compile(r"(?i)(?<![a-z0-9])(\d{1,2})x(\d{2,3})(?![a-z0-9])")
+PART_RE = re.compile(r"(?i)(?:^|[\s._-])(?:part|pt|cd|dvd|disc|disk)[\s._-]?"
+                     r"([1-9]|i{1,3})(?:$|[\s._-])")
+# Jellyfin reads these subfolder names as bonus material. The planner emits only
+# these spellings, so 40k extras end up in nine folders instead of nine hundred.
+EXTRA_DIRS = [
+    ("trailers", r"trailer|teaser|promo|advert|commercial|tv[\s._-]spot|preview"),
+    ("deleted scenes", r"deleted"),
+    ("behind the scenes", r"behind[\s._-]the[\s._-]scenes|making[\s._-]of|bts"
+                          r"|b[\s._-]roll|gag[\s._-]reel|blooper|outtake|archival"
+                          r"|video[\s._-]archive|scene[\s._-]selection|menu|gallery"
+                          r"|stills?|visual[\s._-]effects|workprint"),
+    ("interviews", r"interview|cast[\s._-]and[\s._-]crew|audition|commentary"),
+    ("scene cuts", r"scene[\s._-]cuts?|scenecuts"),
+    ("featurettes", r"featurettes?"),
+    ("shorts", r"shorts?"),
+    ("clips", r"clips?"),
+]
+# Jellyfin's own ExtraType values, folded onto the same folder names.
+EXTRA_KIND = {"trailer": "trailers", "tmbd": "trailers", "intro": "other",
+              "movie": "other", "scene": "scene cuts", "clip": "clips",
+              "interview": "interviews", "behindthescenes": "behind the scenes",
+              "deleted": "deleted scenes", "other": "other", "epilogue": "other",
+              "opening": "other", "featurette": "featurettes", "short": "shorts"}
+SHORT_MAX = 45 * 60        # over this, a file with no episode evidence is film-shaped
+
+
+def rel(p):
+    return p[len(ROOT) + 1:] if p.startswith(ROOT + "/") else p
+
+
+def clean(name):
+    """A name the filesystem and both matchers can read."""
+    name = BAD_CHARS.sub("-", name).replace("\\", "-").strip(" .")
+    return name[:180] or "untitled"
+
+
+def title_of(text):
+    """The engine's parser, applied to a name that has dots but no extension:
+    splitext would read "The.Examples" as stem "The", so a sentinel extension is
+    added and the parser sees an ordinary media name."""
+    return eng.title_of(text + ".mkv")
+
+
+def markers(name):
+    """(season, [episodes], position) the episode markers a name states; None when
+    it states none. The position is where the series' own words stop."""
+    stem = os.path.splitext(name)[0]
+    m = EP_RANGE.search(stem)
+    if m:
+        eps = [int(m.group(2))] + ([int(m.group(3))] if m.group(3) else [])
+        return int(m.group(1)), eps, m.start()
+    m = NX_RE.search(stem)
+    if m:
+        return int(m.group(1)), [int(m.group(2))], m.start()
+    return None
+
+
+LEAD_IDX_RE = re.compile(r"^\s*(\d{1,3})\s*[-\u2013.\s]\s*[A-Za-z]")
+PACK_IDX_RE = re.compile(r"^\s*\d+[a-z]?[.)]\s+")
+
+
+def num(x):
+    try:
+        return int(x)
+    except (TypeError, ValueError):
+        return 0
+
+
+def folder_season(path):
+    """The season the FOLDER states (`Season 3`, `S3`). A human wrote it, and where
+    a scraped row disagrees this is the one that survives a re-plan."""
+    m = SEASON_DIR_RE.match(os.path.basename(os.path.dirname(path)))
+    return int(next(g for g in m.groups() if g)) if m else None
+
+
+def leading_index(name):
+    """The number a file leads with inside a season folder — `01 Kyle MacLachlan,
+    Sinead OC.avi` is episode 1. Both matchers read it that way."""
+    m = LEAD_IDX_RE.match(os.path.splitext(name)[0])
+    return int(m.group(1)) if m else None
+
+
+def tmdb(kind, query):
+    """A provider answer for a query: the cache first, TMDb once, then never again."""
+    ck = f"{kind}:{query}"
+    if ck in CACHE:
+        hit = CACHE[ck]
+        # A cache an operator edited by hand can carry the year as a number; the
+        # comparison below is text, so the wire form is imposed on every hit.
+        return None if hit is None else {**hit, "year": str(hit.get("year") or "")}
+    if not (ARGS.tmdb and API_KEY):
+        return None
+    path = "movie" if kind == "movie" else "tv"
+    url = ("https://api.themoviedb.org/3/search/" + path + "?query="
+           + urllib.parse.quote(query) + "&include_adult=false")
+    req = urllib.request.Request(url, headers={
+        # The key rides a header, never the URL: a token in argv is world-readable.
+        "Authorization": "Bearer " + API_KEY,
+        "Accept": "application/json"})
+    try:
+        with urllib.request.urlopen(req, timeout=20) as r:
+            found = json.load(r).get("results") or []
+    except Exception as e:                                # noqa: BLE001
+        print(f"tmdb {kind} lookup failed for {query!r}: {e}", file=sys.stderr)
+        return None
+    if not found:
+        CACHE[ck] = None
+        return None
+    top = found[0]
+    CACHE[ck] = {"name": (top.get("title") or top.get("name") or "").strip(),
+                 "year": str(top.get("year") or "") or (top.get("first_air_date") or "")[:4],
+                 "id": top.get("id")}
+    return CACHE[ck]
+
+
+# ------------------------------------------------------------------- the inputs
+rows = []
+for line in open(ARGS.enum):
+    size, ext, path = line.rstrip("\n").split("\t")
+    p = os.path.abspath(path)
+    base = os.path.basename(p)
+    if not p.startswith(ROOT + "/") or SKIP_NAME.search(base):
+        continue
+    if p.startswith(PARK + "/"):
+        continue                    # the park is already reviewed; re-parking is churn
+    rows.append({"bytes": int(size), "ext": ext.lower(), "path": p, "base": base})
+held = {r["path"]: r for r in rows}
+
+titles = {}
+with open(ARGS.titles) as fh:
+    cols = fh.readline().rstrip("\n").split("\t")
+    for line in fh:
+        f = line.rstrip("\n").split("\t")
+        f += [""] * (len(cols) - len(f))
+        row = dict(zip(cols, f))
+        p = os.path.abspath(row.get("file", ""))
+        if p in held:
+            titles[p] = row
+
+dur = {}
+if ARGS.probed and os.path.exists(ARGS.probed):
+    for line in open(ARGS.probed):
+        f = line.rstrip("\n").split("\t")
+        if len(f) > 2:
+            try:
+                dur[os.path.abspath(f[1])] = float(f[2] or 0)
+            except ValueError:
+                pass
+
+video = [r for r in rows if r["ext"] in VIDEO_EXT]
+other = [r for r in rows if r["ext"] not in VIDEO_EXT]
+
+# What a folder already knows: one provider row names the series for every sibling
+# that has no row of its own — that is how an anime pack gets its show.
+by_dir = collections.defaultdict(list)
+for r in video:
+    by_dir[os.path.dirname(r["path"])].append(r)
+dir_series = {}
+for d, rs in by_dir.items():
+    names = collections.Counter(titles[r["path"]]["series"] for r in rs
+                                if r["path"] in titles and titles[r["path"]]["series"])
+    if names:
+        dir_series[d] = names.most_common(1)[0][0]
+
+
+# The registry: every series a provider names, judged before any folder is. Folder
+# words may resolve a file to one of these names; they may never invent one. That is
+# the whole difference between "NCOP 01.mkv" inside the Gurren Lagann pack, which the
+# pack names, and a new root folder called "01 NCOP".
+registry = {}
+for _p, _t in titles.items():
+    if _t["series"]:
+        registry.setdefault(eng.norm(_t["series"]), _t["series"])
+
+
+def known_in(text):
+    """The registered series named inside this text: the longest registered name
+    whose words appear in it as a run. "(CBB) Tengen Toppa Gurren Lagann - The
+    Complete Series (BD 1080p)" names "Tengen Toppa Gurren Lagann"."""
+    hay = " " + eng.norm(text) + " "
+    best = ""
+    for k, name in registry.items():
+        if len(k) >= 5 and f" {k} " in hay and len(k) > len(best):
+            best = name
+    return best
+
+
+# Rows also name their own folders' ancestors, so an already-filed library stays
+# where it is when it is re-planned.
+for d, name in sorted(dir_series.items()):
+    a = os.path.dirname(d)
+    while a and a != ROOT and len(a) > len(ROOT):
+        if a not in dir_series:
+            got = known_in(os.path.basename(a))
+            if got:
+                dir_series[a] = got
+        a = os.path.dirname(a)
+
+
+def series_of_dir(d):
+    """The series an ancestor folder belongs to, nearest first: the folder holding
+    provider rows, else a folder whose own words name a registered show."""
+    while d and d != ROOT and len(d) > len(ROOT):
+        if d in dir_series:
+            return dir_series[d]
+        got = known_in(os.path.basename(d))
+        if got:
+            return got
+        d = os.path.dirname(d)
+    return ""
+
+
+def extras_kind(path, extra_type):
+    """The canonical bonus subfolder, from Jellyfin's own extra type first."""
+    if extra_type:
+        return EXTRA_KIND.get(eng.norm(extra_type).replace(" ", ""), "other")
+    for d in rel(path).split("/")[:-1]:
+        if eng.folder_is_bonus(d):
+            for name, rx in EXTRA_DIRS:
+                if re.search(rx, d, re.I):
+                    return name
+            return "extras"
+    stem = os.path.splitext(os.path.basename(path))[0]
+    for name, rx in EXTRA_DIRS:
+        if re.search(rx, stem, re.I):
+            return name
+    return "extras"
+
+
+# ----------------------------------------------------------- classify every file
+series = {}                       # norm(name) -> {name, year, tmdb}
+claimants = []                    # (row, name, season, [episodes])
+extras = []                       # (row, name, bonus folder)
+films = []                        # film-shaped, no episode evidence
+unknown = []                      # (row, reason)
+
+
+def series_name(name):
+    """The provider's name as a folder: the pack index a scene release leads with
+    ("4a. Seed DESTINY") falls away, and the doubled spaces it leaves fold."""
+    return clean(re.sub(r"\s{2,}", " ", PACK_IDX_RE.sub("", name)).strip(" ."))
+
+
+def note(name, year="", tmdb_id=""):
+    """Register a series; the first name wins, the first year and id stick."""
+    k = eng.norm(name)
+    s = series.setdefault(k, {"name": series_name(name), "year": "", "tmdb": ""})
+    if year and not s["year"]:
+        s["year"] = str(year)
+    if tmdb_id and not s["tmdb"]:
+        s["tmdb"] = str(tmdb_id)
+
+
+
+def as_extra(r, owner, extra_type):
+    """Bonus material needs a series to file under; with none it is parked."""
+    if not owner:
+        unknown.append((r, "bonus material with no series to own it"))
+        return
+    note(owner)
+    extras.append((r, owner, extras_kind(r["path"], extra_type)))
+
+for r in video:
+    p, t = r["path"], titles.get(r["path"])
+    marked = markers(r["base"])
+    fs = folder_season(p)
+    if t and t["extra"]:
+        as_extra(r, t["series"] or series_of_dir(os.path.dirname(p)), t["extra"])
+        continue
+    row = bool(t and t["series"] and (t["season"] or t["episode"]))
+    if marked or row:
+        # The provider row names the SHOW: its registered name, its year. The number
+        # that orders the file comes from the file's own name first, then the folder
+        # it sits in, then the row — scraped pack rows number episodes badly, where
+        # every file under SNL 16-20/Season 16..19 carries season 16 episode 20.
+        owner = (t["series"] if t and t["series"]
+                 else series_of_dir(os.path.dirname(p))
+                 or title_of(r["base"][:marked[2]])[0])
+        if not eng.norm(owner).replace(" ", ""):
+            unknown.append((r, "an episode marker with no series name to hang it on"))
+            continue
+        if marked:
+            season, eps = marked[0], marked[1]
+        else:
+            season = fs if fs is not None else num(t["season"])
+            ep = leading_index(r["base"]) or num(t["episode"])
+            if not 0 < season <= 200 or ep <= 0:
+                unknown.append((r, f"a provider row of season {season} episode {ep}"))
+                continue
+            eps = [ep]
+        claimants.append((r, owner, season, eps))
+        note(owner, t["year"] if t else "", t["tmdb"] if t else "")
+        continue
+    owner = series_of_dir(os.path.dirname(p))
+    if fs is not None and owner and leading_index(r["base"]):
+        claimants.append((r, owner, fs, [leading_index(r["base"])]))
+        note(owner)
+        continue
+    if eng.is_extra(p, title_of(r["base"])[0], dur.get(p, 0.0), ROOT):
+        as_extra(r, owner, "")
+        continue
+    if dur.get(p, 0.0) >= SHORT_MAX or (p not in dur and title_of(r["base"])[1] != "?"):
+        films.append(r)
+        continue
+    unknown.append((r, "no episode marker, no provider row, nothing long enough to be a film"))
+
+# A provider that never states a year still has one: the search is exact, so a
+# miss parks nothing — the folder simply carries no year, which Jellyfin accepts.
+for k, s in series.items():
+    if not s["year"]:
+        got = tmdb("tv", s["name"])
+        if got and eng.norm(got["name"]) == k:
+            s["year"] = got["year"]
+
+if ARGS.tmdb:
+    with open(ARGS.tmdb_cache, "w") as fh:
+        json.dump(CACHE, fh, indent=1, sort_keys=True)
+
+
+def series_dir(name):
+    s = series.get(eng.norm(name))
+    if not s:
+        s = {"name": clean(name), "year": ""}
+    label = f"{s['name']} ({s['year']})" if s["year"] else s["name"]
+    return f"{ROOT}/{clean(label)}"
+
+
+# ------------------------------------------------------------------- the plan
+moves, keeps, parked = [], [], []
+reasons = collections.defaultdict(list)
+used = {}                          # dst -> the file that claimed it
+
+
+def book(r, dst_dir, name, kind):
+    """Claim one destination for one file, or park the file when it is taken."""
+    dst = f"{dst_dir}/{name}"
+    if dst == r["path"]:
+        keeps.append((r["path"], dst, kind))
+        return
+    if dst in used:
+        park(r, "something else in the plan already files there")
+        return
+    if dst in held:
+        park(r, f"{rel(dst)} already occupies that name")
+        return
+    used[dst] = r["path"]
+    moves.append((r["path"], dst, kind))
+
+
+def park(r, reason):
+    parked.append((r, reason))
+    reasons[reason].append(r)
+    dst = f"{PARK}/{rel(r['path'])}"
+    if dst in used or dst in held:
+        dst = f"{PARK}/{rel(r['path'])}.{len(reasons)}"
+    used[dst] = r["path"]
+    moves.append((r["path"], dst, "park"))
+
+
+# Episodes first: two files claiming one number is a decision Frank makes, not one
+# the plan sneaks — the bigger copy takes the standard name, the other waits.
+by_ep = collections.defaultdict(list)
+for r, name, season, eps in claimants:
+    for e in eps:
+        by_ep[(eng.norm(name), season, e)].append(r)
+winners, losers = set(), {}
+for (key, season, e), rs in by_ep.items():
+    rs.sort(key=lambda r: (-r["bytes"], r["path"]))
+    winners.add(rs[0]["path"])
+    for r in rs[1:]:
+        losers.setdefault(r["path"],
+                          f"duplicate of {series[key]['name']} S{season:02d}E{e:02d}"
+                          " (the bigger copy keeps the standard name)")
+for r, name, season, eps in claimants:
+    if r["path"] in losers:
+        park(r, losers[r["path"]])
+        continue
+    s = series[eng.norm(name)]
+    label = f" S{season:02d}E" + "-E".join(f"{e:02d}" for e in eps)
+    part = PART_RE.search(os.path.splitext(r["base"])[0])
+    tail = f" Part {part.group(1).upper()}" if part and len(eps) == 1 else ""
+    book(r, f"{series_dir(name)}/Season {season:02d}", s["name"] + label + tail + r["ext"],
+         "tv")
+
+for r, name, kind in extras:
+    book(r, f"{series_dir(name)}/{kind}",
+         clean(os.path.splitext(r["base"])[0]) + r["ext"], "extra")
+
+# A film inside the TV library leaves for the movie library only when the movie
+# provider answers with a title that matches the name and, when the name states a
+# year, that year too.
+for r in films:
+    parsed, year = title_of(r["base"])
+    got = tmdb("movie", parsed)
+    words = eng.norm(got["name"]).split() if got else []
+    if not got or not eng.mentions(parsed.split(), words) \
+            or (year != "?" and got["year"] and got["year"] != year):
+        park(r, "long file with no episode evidence, and no movie provider names it")
+        continue
+    label = f"{got['name']} ({got['year']})" if got["year"] else got["name"]
+    book(r, f"{MOVIES}/{clean(label)}", clean(label) + r["ext"], "movie")
+
+# ------------------------------------------------- sidecars and folder metadata
+moved = {}                                          # src -> dst, for every booked file
+for src, dst, _kind in moves:
+    moved[src] = dst
+for src, dst, _kind in keeps:
+    moved[src] = dst
+dst_dirs = collections.defaultdict(collections.Counter)
+video_dst = collections.defaultdict(list)
+for src, dst in moved.items():
+    if src in {r["path"] for r in video}:
+        video_dst[os.path.dirname(src)].append((src, dst))
+    dst_dirs[os.path.dirname(src)][os.path.dirname(dst)] += 1
+
+
+def series_name_at(d):
+    """The series the files of a folder belong to — a registered name, or nothing.
+    Folder words alone never name a show."""
+    names = [titles[src]["series"] for src, _ in video_dst.get(d, [])
+             if src in titles and titles[src]["series"]]
+    if names:
+        return collections.Counter(names).most_common(1)[0][0]
+    return series_of_dir(d)
+
+
+def season_at(d):
+    """The season number a folder carries: its own name, else its files' rows."""
+    m = SEASON_DIR_RE.match(os.path.basename(d))
+    if m:
+        return int(m.group(1) or m.group(2))
+    nums = [int(titles[src]["season"]) for src, _ in video_dst.get(d, [])
+            if src in titles and titles[src]["season"]]
+    return collections.Counter(nums).most_common(1)[0][0] if nums else None
+
+
+def folder_dst(d):
+    """Where a folder's own images go: the one place its files landed, or the
+    series root when they scattered across several seasons."""
+    c = dst_dirs.get(d)
+    if not c:
+        return None
+    if len(c) == 1:
+        return c.most_common(1)[0][0]
+    owner = series_name_at(d)
+    return series_dir(owner) if owner else None
+
+
+def anchor_of(path, stem):
+    """The video file this sidecar names: the longest file name it extends, in the
+    NEAREST folder that holds one. A scene pack buries its subtitles two levels
+    down (`Pack/Subs/<release-name>/<release>.jpn.srt`), so the search climbs to
+    the library root instead of stopping at the parent."""
+    d = os.path.dirname(path)
+    while len(d) >= len(ROOT) and d.startswith(ROOT):
+        best = None
+        for src, dst in video_dst.get(d, []):
+            base = os.path.basename(src)
+            for prefix in (os.path.splitext(base)[0], base):
+                if stem == prefix or stem.startswith(prefix + "."):
+                    if best is None or len(prefix) > best[2]:
+                        best = (src, dst, len(prefix))
+        if best:
+            return best[0], best[1]
+        up = os.path.dirname(d)
+        if up == d:
+            break
+        d = up
+    return None
+
+
+def sidecar_name(r, src, dst):
+    """The sidecar's own tail — language, flag, the media extension an nfo must
+    carry — hung on the video file's new base."""
+    stem = os.path.splitext(os.path.basename(src))[0]
+    tail = os.path.splitext(r["base"])[0][len(stem):]
+    # "Show.S01E01.mkv.srt" carries the container's name, not a flag: Jellyfin and
+    # Plex both read "Show S01E01.srt", so the container word is dropped here.
+    parts = tail.split(".")
+    if len(parts) > 1 and parts[1].lower() in CONTAINER:
+        tail = "".join("." + p for p in parts[2:])
+    if r["ext"] in NFO_EXT:
+        tail = os.path.splitext(os.path.basename(src))[1] + tail
+    return clean(os.path.splitext(os.path.basename(dst))[0] + tail + r["ext"])
+
+
+for r in other:
+    p, base = r["path"], r["base"]
+    stem = os.path.splitext(base)[0]
+    d = os.path.dirname(p)
+    if r["ext"] in SUB_EXT or r["ext"] in NFO_EXT or r["ext"] in IMG_EXT:
+        a = anchor_of(p, stem)
+        if a:
+            book(r, os.path.dirname(a[1]), sidecar_name(r, a[0], a[1]),
+                 "subtitle" if r["ext"] in SUB_EXT else
+                 ("nfo" if r["ext"] in NFO_EXT else "image"))
+            continue
+    if SERIES_META.match(base):
+        owner = series_name_at(d)
+        if owner:
+            book(r, series_dir(owner), base, "meta")
+            continue
+    elif SEASON_META.match(base):
+        owner, season = series_name_at(d), season_at(d)
+        if owner and season is not None:
+            book(r, f"{series_dir(owner)}/Season {season:02d}", base, "meta")
+            continue
+    elif r["ext"] in IMG_EXT or r["ext"] in NFO_EXT:
+        dest = folder_dst(d)
+        if dest:
+            book(r, dest, base, "meta")
+            continue
+    unknown.append((r, "not media, or media with nothing beside it to belong to"))
+
+for r, reason in unknown:
+    park(r, reason)
+
+
+# --------------------------------------------------------------------- output
+def write_plan():
+    """mkdir rows for every destination folder and its whole chain, then the moves.
+    Parents first: the executor creates directories in the order they are listed."""
+    chain = set()
+    for _s, dst, _k in moves:
+        d = os.path.dirname(dst)
+        while d and d not in (ROOT, MOVIES, PARK, os.path.dirname(d)):
+            chain.add(d)
+            d = os.path.dirname(d)
+    dirs = sorted(chain, key=lambda p: (p.count("/"), p))
+    with open(ARGS.out, "w") as fh:
+        fh.write("action\tsrc\tdst\tkind\n")
+        for d in dirs:
+            fh.write(f"mkdir\t\t{d}\tdir\n")
+        for src, dst, kind in sorted(moves, key=lambda m: (m[1], m[0])):
+            fh.write(f"move\t{src}\t{dst}\t{kind}\n")
+        for src, dst, kind in sorted(keeps, key=lambda m: (m[1], m[0])):
+            fh.write(f"keep\t{src}\t{dst}\t{kind}\n")
+    sized = sum(held[s]["bytes"] for s, _d, _k in moves if s in held)
+    print(f"plan: {len(moves) - len(parked)} moves, {len(parked)} parked"
+          f" ({sized / 2**40:.2f} TiB touched), {len(keeps)} already placed,"
+          f" {len(dirs)} folders -> {ARGS.out}")
+
+
+def write_exceptions():
+    no_year = sorted(s["name"] for s in series.values() if not s["year"])
+    with open(ARGS.exceptions, "w") as fh:
+        fh.write(f"TV relayout exceptions for {ROOT}\n\n")
+        fh.write(f"Everything below is parked under {PARK}, keeping the relative path it\n")
+        fh.write("has today. Nothing is deleted and nothing is overwritten: a parked file\n")
+        fh.write("is one rename from wherever the plan is pointed next.\n\n")
+        for reason in sorted(reasons, key=lambda r: -len(reasons[r])):
+            rs = reasons[reason]
+            fh.write(f"{len(rs):>6} files  {reason}\n")
+            for r in sorted(rs, key=lambda r: rel(r["path"]))[:400]:
+                fh.write(f"          {rel(r['path'])}\n")
+            if len(rs) > 400:
+                fh.write(f"          … and {len(rs) - 400} more\n")
+            fh.write("\n")
+        if no_year:
+            fh.write(f"{len(no_year):>6} series carry no year from any provider, so their\n")
+            fh.write("       folders are named without one; Jellyfin still matches them.\n")
+            fh.write("       " + ", ".join(no_year[:300]) + "\n")
+    print(f"exceptions -> {ARGS.exceptions}")
+
+
+write_plan()
+write_exceptions()

scripts/media-titles.py

@@ -0,0 +1,579 @@
+#!/usr/bin/env python3
+r"""The fleet's release-filename title parser: the grouping authority the relayout
+plan uses for every file the media server never matched.
+
+  report:  media-titles.py report [--probed TSV] [--out TSV]
+           the similar-copies census (DUP / REVIEW / EP-DUP groups) over the
+           ffprobe census scripts/media-census.py writes.
+  parse:   media-titles.py parse NAME [NAME…]
+           what title_of() makes of a name — the way to check a naming accident
+           before it merges two films into one folder.
+
+scripts/media-plan.py imports this file; over there the paths are arguments.
+
+Method — three independent signals, so a naming accident cannot merge two films:
+  * franchise KEY  : first two significant title tokens. Stable across parts of a
+    collection ("resident evil", "jump street") but NOT film-specific — it only
+    selects candidates.
+  * DURATION       : the identity check. Same master encodes differ by at most a
+    couple of seconds; different films / cuts do not collide.
+  * SIZE SPREAD + YEAR: decide the tier. Two files same size AND same duration
+    are byte-twins (the hash pass confirms); same duration but different sizes are
+    two different releases of one film (deletable redundancy).
+
+TV episodes (SxxEyy in the name) are grouped separately: their uniform runtime
+would otherwise collapse a whole series into one "duplicate" group.
+"""
+import argparse
+import collections, os, re, sys, unicodedata
+
+# Library root: the folder directly under this one names the film, and every
+# folder below it is structure (Extras/, Featurettes/, Disc 1/, BDMV/). Tests pass
+# their own; nothing here reads media, only the census TSVs about it.
+ROOT = os.environ.get("MEDIA_ROOT", "/mnt/entropy/Movies")
+
+# Release-tag vocabulary: never part of a title, so the title stops here.
+JUNK = {"5.1", "7.1", "2.0", "1.0", "6.1", "4.0", "extras"}
+JUNK_RE = re.compile(
+    r"^("
+    r"\d{3,5}p|\d{1,2}k|uhd|fhd|imax|x ?26[45]|hevc|h ?26[45]|avc|av1|x ?av1|vp[89]"
+    r"|divx|xvid|mpeg-?2?|\d{1,2}-?bit|hdr ?10\+?|hdr|dolby ?vision|dvh?|sdr|hybrid"
+    r"|re-?mux|bd-?re-?mux|blu-?ray|bdrip|brrip|bdmux|bd|web-?(rip|dl)?|hdtv|hdrip"
+    r"|dvd-?(rip|r|scr)?|hdcam|telesync|screener|ppv|re-?encode|extras?"
+    r"|ddp? ?\d(\.\d)?|dd\d(\.\d)?|dts-?hd(\.ma)?|dts|truehd|atmos|e-?ac-?3?|ac-?3"
+    r"|aac\d?(\.\d)?|lpcm|flac|opus|mp3|pcm|\d\.\d(\.\d)?|5 ?1|7 ?1|2 ?0|1 ?0|6 ?1|4 ?0"
+    r"|subs?-?(it|en|eng|ita)?|subbed|dual(-?audio)?|multi|dub(bed)?"
+    r"|ita|eng|en|it|de|fr|es|pl|jp|jap|lat|esp|ger|fre|rus|zho|hin|tam|tel"
+    r"|repack|proper|internal|limited|unrated|extended|director'?s|cut|remaster(ed)?|uncut"
+    r"|special|edition|version|crc32|sha1|sfv|par2|nfo|nzb|mux|complete|unpack"
+    r"|certified|scene|p2p|part\d*|cd\d+|disc\d+|disk\d+|vob|idx|srt|ass|ssa|sup"
+    r"|m3u|cue|log|txt|mkv|mp4|avi|imdb\d[\d.]*|yts(\.\w+)?|rarbg|collection"
+    r")$", re.I)
+RES_RE = re.compile(r"^\d{3,5}p$|^\d{1,2}k$|^\d+x\d+$", re.I)
+YEAR_RE = re.compile(r"^(19|20)\d{2}$")
+SEP_RE = re.compile(r"[._\s\-\[\](){}+!~,;:&@#/*'\"`´|]+")
+IDX_RE = re.compile(r"^\d{1,3}[a-z]?$")
+WORD_RE = re.compile(r"[A-Za-z0-9']+$")
+EP_RE = re.compile(r"(?i)(?<![a-z0-9])(s(?:eason[\s._-]*)?\d{1,2}[\s._-]*e(?:p(?:isode)?[\s._-]*)?"
+                   r"\d{1,3}|s\d{1,2}\s?e\d{1,3}|\d{1,2}x\d{2,3}|part[\s._-]*\d{1,2})")
+STOP = {"a", "an", "the", "il", "la", "le", "i", "un", "une", "el", "los", "las", "de",
+        "del", "du", "di", "e", "ed", "y", "and", "et", "for", "of", "to", "in", "on",
+        "vs", "avec", "con"}
+
+
+def norm(s):
+    s = unicodedata.normalize("NFKD", s)
+    return "".join(c for c in s if not unicodedata.combining(c)).lower().strip()
+
+
+def fold_sep(s):
+    """norm(), plus: every character that is left over is not a letter, digit or
+    apostrophe, so it is a SEPARATOR whatever codepoint it hides behind — not part
+    of the word. Release groups file titles with lookalikes for punctuation the
+    filesystem forbids: "V ∕ H ∕ S ∕ 2" carries U+2215 DIVISION SLASH, "S꞉ Viral"
+    U+A789, and en dashes hide in "Star Wars – A New Hope". NFKD does NOT fold
+    those (no decomposition mapping), and the scrubber inside title_of DELETES
+    what it does not recognise, so five different V/H/S films squashed into the one
+    title "vhs2" and would have shared one folder. Deleting is the wrong operation:
+    they separate. Accents are gone by then — norm() removed the combining marks —
+    so "Amélie" stays "amelie" and never splits into "ame lie"."""
+    return " ".join("".join(c if c.isalnum() or c == "'" else " "
+                            for c in norm(s)).split())
+
+def title_of(path):
+    """(title, year) from the basename.
+
+    Two passes. (1) Tokenize; every 4-digit year becomes a sentinel in place.
+    (2) Find the boundary: the first release-noise token (resolution / codec /
+    source / audio / language / group vocabulary) ends the title, so trailing
+    tags can never leak in. (3) Collect up to that boundary: leading index
+    numbers ("03."), leading year sentinels ("1940 - Fantasia") and leading noise
+    are skipped while nothing is collected; once a real word appears a sentinel
+    CLOSES the title. A title sitting BEFORE the year sentinel ("300.2006.…" or
+    "12. Monkeys. 1995") is a numeric title and is prepended to what follows.
+    """
+    # fold_sep BEFORE tokenizing, so a lookalike separator separates (see its
+    # docstring): the tokens below are then plain alnum/apostrophe words.
+    name = fold_sep(os.path.splitext(os.path.basename(path))[0])
+    raw = [x for x in SEP_RE.split(name) if x]
+    years = [t for t in raw if YEAR_RE.match(t)]
+    two_year = len(years) > 1                       # "2001…1968": one of them is the title
+    toks, year = [], None
+    for t in raw:
+        if YEAR_RE.match(t):
+            year = t                                    # last year wins
+            toks.append("\x00")
+        else:
+            toks.append(t)
+    stop, seen_word = len(toks), False
+    for i, t in enumerate(toks):
+        if t == "\x00":
+            continue
+        c = norm(re.sub(r"[^A-Za-z0-9']", "", t))
+        if (not c or RES_RE.match(c) or JUNK_RE.match(c) or c in JUNK
+                or not WORD_RE.match(c)) and not (IDX_RE.match(t) and not seen_word):
+            stop = i
+            break
+        if c.isalpha():
+            seen_word = True
+    # A leading number is a track index ("01. Bye Bye Baby", "05. Casablanca") unless
+    # it is three digits or more ("007", "000", "1408") or the name carries no year
+    # at all ("12 Monkeys", "831"). A 1-2 digit prefix beside a year is an index;
+    # the rare real title that looks like one ("0 Things I Hate About You (1999)")
+    # loses its zero to the parser — Plex's matched title overrides it anyway.
+    lead = re.split(r"[\s._-]", name, maxsplit=1)[0]
+    track_idx = bool(re.fullmatch(r"\d{1,2}[a-z]?", lead)) and bool(year)
+    keep, num_title = [], None
+    for t in toks[:stop]:
+        if t == "\x00":
+            if keep:
+                break                                   # year closes the title
+            if two_year and t is not None and years[0] != year:
+                # two DIFFERENT years ("2001…1968"): the leading one is the title;
+                # "1940 - Fantasia (1940)" repeats one year, so it is not a title
+                num_title = num_title or years[0]
+            continue                                    # leading year: not the title
+        if track_idx and IDX_RE.match(t) and not keep and t == lead:
+            continue                                    # "03." / "1c." ordering
+        c = norm(re.sub(r"[^A-Za-z0-9']", "", t))
+        if not keep and not c.isalpha():
+            num_title = num_title or (c or None)
+            continue                                    # numeric title candidate
+        keep.append(c)
+        if len(keep) >= 8:
+            break
+    keep = [k for k in keep if not re.fullmatch(r"(disc|disk|cd|dvd|part)\d*", k)]
+    title = " ".join(keep)
+    if not keep:
+        # nothing collected: a pure-numeric leading token is the title even when it
+        # sits where the release year sits ("1917.2019.2160p…"), and "300.2006…" too
+        for t in raw:
+            c = norm(re.sub(r"[^A-Za-z0-9']", "", t))
+            if re.fullmatch(r"(?:[12]\d{3}|\d{1,3})", c) and (c != year or two_year):
+                return c, year or "?"    # never "2160p" / "1080": they are not pure digits
+    if (num_title and title and not keep[0].isdigit()
+            and (two_year or not track_idx)):
+        # a leading number that only echoes the release year ("1917.2019.2160p…") is
+        # not a title; one that stands alone ("300.2006…", "1900") is
+        title = f"{num_title} {title}"
+    title = title or num_title or norm(name).replace("\x00", " ").strip()[:40]
+    return title, year or "?"
+
+
+# Bonus-material vocabulary: names that say nothing about the film, so an extras
+# file otherwise pairs with an extras file of another movie. "Short", "Spot" and
+# "Clip" are deliberately absent — they are film titles (Short Circuit, Spot,
+# Clips) — and the running-time guard in is_extra() carries the rest.
+SEP_CLS = r"[\s._-]+"
+
+
+def _bonus_rx(word):
+    return r"(?<![a-z0-9])" + re.sub(r"[ \-]+", lambda _: SEP_CLS, word.lower()) + r"(?![a-z0-9])"
+
+
+EXTRA_WORDS = [
+    "trailer", "teaser", "featurette", "featurettes", "promo", "promotional",
+    "tv spot", "advert", "commercial", "behind the scenes", "making of",
+    "deleted scene", "deleted scenes", "deleted footage", "gag reel", "blooper",
+    "blooper reel", "bloopers", "outtake", "outtakes", "scene selection", "menu",
+    "menus", "gallery", "still gallery", "stills", "poster", "posters",
+    "montage", "preview", "previews", "epilogue", "visual effects",
+    "music video", "singalong", "karaoke", "trivia", "storyboard", "workprint",
+    "test footage", "screen test", "audition", "cast and crew", "isolated score",
+    "picture in picture", "original score", "session piece", "sample",
+    "watermark",
+]
+# Words that describe bonus material which can run LONG (a feature-length
+# commentary-track cut, a documentary about the film). Trusted only below
+# LONG_BONUS_MAX, so a 40-minute "commentary" release stays a film.
+EXTRA_LONG_WORDS = ["interview", "interviews", "commentary", "video archive",
+                    "archival", "documentary"]
+EXTRA_NAME_RE = re.compile(
+    "|".join([_bonus_rx(w) for w in EXTRA_WORDS] + [r"scene[\s._-]?\d{1,3}"]), re.I)
+EXTRA_LONG_NAME_RE = re.compile(
+    "|".join(_bonus_rx(w) for w in EXTRA_LONG_WORDS), re.I)
+# Bonus-folder vocabulary. fullmatch = the folder IS an extras subtree; a bare
+# search = it only mentions extras ("... + Extras" box sets hold feature films).
+EXTRA_FOLDER_RE = re.compile(
+    r"(?:extras?|bonus(?:[\s._-]material)?|specials?|special[\s._-]featurettes?|"
+    r"featurettes?|deleted(?:[\s._-](?:scenes?|footage))?|trailers?|"
+    r"making[\s._-]of|behind[\s._-]the[\s._-]scenes|shorts?|interviews?|"
+    r"archiv(?:al|es)|video[\s._-]archive)", re.I)
+VIDEO_SUFFIX = (".mkv", ".mp4", ".avi", ".m4v", ".mov", ".m2ts", ".mpg", ".webm")
+
+# Anything longer than this is a feature film whatever it is called: a 118-minute
+# file named "The Menu" or "Trailer" is a movie, not bonus material.
+SHORT_MAX = 45 * 60
+
+# Long-shaped bonus vocabulary is only believed below this: a 40-minute
+# commentary cut is an alternate version of the film, not a featurette.
+LONG_BONUS_MAX = 25 * 60
+
+
+def name_words(x):
+    """Bare words of a file or folder name: no release metadata, no year."""
+    x = os.path.splitext(x)[0] if x.lower().endswith(VIDEO_SUFFIX) else x
+    return [w for w in SEP_RE.split(fold_sep(x))
+            if w and not YEAR_RE.match(w) and not RES_RE.match(w)
+            and not JUNK_RE.match(w) and w not in JUNK]
+
+
+def mentions(short, long_):
+    """True when one word list appears in the other, anywhere: a release names
+    the film at the front ("Alien (1979) - Trailer") or in the middle
+    ("Movies/Alien (1979)/Extra - Logo.mkv"), so a start-of-string rule fails."""
+    a, b = sorted([short, long_], key=len)
+    if not a:
+        return False
+    return any(b[i:i + len(a)] == a for i in range(len(b) - len(a) + 1))
+
+
+
+def folder_is_bonus(folder):
+    """True when a folder name carries bonus vocabulary as WHOLE words: the
+    vocabulary is matched against a padded word string, so "Extras" decides and
+    a word that merely CONTAINS one ("Extraordinary", "The Extras") does not."""
+    return bool(EXTRA_FOLDER_RE.search(" " + " ".join(SEP_RE.split(norm(folder))) + " "))
+
+
+def is_extra(path, title="", dur=0.0, root=ROOT, matched=False):
+    """True for bonus material, which must never be counted as a film: a file
+    called "Trailer", "Menu", "Sample" or "Gallery" carries no film identity, so
+    extras otherwise pair with each other across the whole library.
+
+    Evidence, strongest first:
+      * RUNTIME VETOES EVERYTHING. The film "The Menu", the film "Trailer", the
+        franchise "Trailer Park Boys" and "ET — The Extra-Terrestrial" all carry
+        bonus vocabulary and run 100+ minutes: anything over SHORT_MAX is a film —
+        unless it sits in an explicit bonus folder and metadata never matched it.
+        Bonus words that describe long-form material (commentary, documentary,
+        interview) are believed only
+        under LONG_BONUS_MAX, so a feature-length commentary cut stays a film.
+      * The file's OWN name carries bonus vocabulary -> bonus.
+      * ANY folder of the path below the film's own root folder carries bonus
+        vocabulary ("Alien (1979)/Extras/", ".../Featurettes/Deleted Scenes/",
+        "Angel's Egg (1985)/04b. Angel's Egg Extras/") -> bonus. The ROOT folder
+        is exempt because it names the film, and a film may be titled with a
+        bonus word; and a file whose own name IS that folder's film name is the
+        feature, not the bonus ("Ghost in the Shell Movies/Extras/
+        Ghost in the Shell - CM3.mp4" is a bonus, but a whole other film filed
+        there stays a film).
+    Bonus vocabulary is matched as whole words only, so the film
+    "Angel's Egg - Extra - Logo" inside "04b. Angel's Egg Extras" is reached by
+    the folder rule, and "…Extras" as a fragment of a word is never evidence.
+    """
+    parts = [q for q in path.split("/") if q]
+    if not parts:
+        return False
+    name = (os.path.splitext(parts[-1])[0]
+            if parts[-1].lower().endswith(VIDEO_SUFFIX) else "")
+    # A bonus folder anywhere BELOW the film's own folder decides for its
+    # contents; the film's own folder is the film's name, whatever words it
+    # contains, so it is exempt. It is the first component AFTER ROOT's own
+    # components — those are the mount path, not folders of the library.
+    head = len([q for q in root.split("/") if q]) + 1
+    bonus_folder = False
+    for folder in parts[head:-1]:
+        if folder_is_bonus(folder):
+            # The folder's words are often the film's own name ("Angel's Egg
+            # Extras/"), and then they are not bonus evidence. Only WORDS can name
+            # a film: "extras" is release junk that name_words drops, so a folder
+            # like "01. Extras" reduces to its index — and an index matching an
+            # index ("NCOP 01") is a coincidence, not a title.
+            nw = [w for w in name_words(name) if not w.isdigit()]
+            fw = [w for w in name_words(folder) if not w.isdigit()]
+            bonus_folder = not fw or not mentions(nw, fw)
+            break
+    if dur and dur > SHORT_MAX:
+        # Runtime vetoes vocabulary, with ONE exception: an explicit bonus folder
+        # and no metadata match. A 90-minute "Feature Length Rough Cut" sitting in
+        # a movie's Featurettes/ is a disc extra, and nothing in the library claims
+        # it as a film. Long-form bonus words still describe a film, so a
+        # feature-length commentary cut or documentary keeps its runtime veto.
+        if bonus_folder and not matched and not EXTRA_LONG_NAME_RE.search(name):
+            return True
+        return False
+    if EXTRA_NAME_RE.search(name):
+        return True
+    if dur and dur <= LONG_BONUS_MAX and EXTRA_LONG_NAME_RE.search(name):
+        return True
+    return bonus_folder
+
+# Release-group watermark clips: a group drops a tiny ad ("ETRG.mp4",
+# "RARBG.com.mp4") into many different films, so its name carries no film
+# identity at all and it would otherwise pair with itself across the library.
+WATERMARK_MAX = 20 * 2**20
+WATERMARK_RE = re.compile(r"(?i)^(rarbg(\.com)?|etrg|tgx|fbt|cpbg|yify|yts|"
+                          r"rarbg\.com|playnomore|imprs|gauntlet|dfx|macmote|"
+                          r"psi-axiz|acm|tlf|sa8|ebp|gtv|w4g|don|aAF|enjoy|hd4u)$")
+
+
+def is_watermark(path, size):
+    """True for a release group's own watermark video, not a film or an extra."""
+    return (size < WATERMARK_MAX
+            and bool(WATERMARK_RE.match(os.path.splitext(os.path.basename(path))[0])))
+
+
+
+def key_of(title):
+    """Franchise key: first two alphabetic tokens (numeric-only title = itself).
+    Deliberately NOT film-specific — it only selects candidates for the duration
+    check, so a two-token collision (two films starting "star trek") cannot by
+    itself create a duplicate group."""
+    toks = [t for t in title.split() if t]
+    words = [t for t in toks if not t.isdigit()]
+    if not words:
+        return " ".join(toks)
+    sig = [t for t in words if t not in STOP] or words
+    return " ".join(sig[:2])
+
+
+def ep_of(path):
+    """(series, episode-token) when the basename carries SxxEyy / 1x04 / Part N.
+
+    The series is the name BEFORE the marker. The box-set convention writes
+    "Monk - S05E03 - Mr. Monk and the Big Game", and the episode title that
+    follows the marker is not the show's name: keying on the whole title gave
+    every episode of one series its own group, so a 124-episode box set never
+    reached the size that calls something a series and each episode was filed
+    as its own film. A file named only by its marker has nothing before it, so
+    its whole title stands in."""
+    name = os.path.splitext(os.path.basename(path))[0]
+    m = EP_RE.search(name)
+    if not m:
+        return None
+    head = name[:m.start()].strip(" ._\u2013-")
+    series = title_of(head)[0] if head else None
+    return series or title_of(path)[0], norm(m.group(1)).replace(" ", "")
+
+
+def base_words(title, key):
+    """Title tokens beyond the franchise key — the FILM identity part.
+    "evil dead rise" under key "evil dead" -> {rise}; "28 days later" -> {}."""
+    kt = set(key.split())
+    return frozenset(t for t in title.split() if t not in kt and t not in STOP
+                     and not t.isdigit() and not YEAR_RE.match(t))
+
+
+def folder_year(path):
+    """Soft year evidence: a 19xx/20xx token in any parent directory name
+    (collection folders are written both "(2002)" and "[2009]"; a range such as
+    "(1998-2000)" deliberately yields nothing)."""
+    for d in reversed(os.path.dirname(path).split("/")):
+        ys = [y for y in re.split(r"[^0-9]+", d) if YEAR_RE.match(y)]
+        if len(ys) == 1:
+            return ys[0]
+    return None
+
+
+def same_film(a, b):
+    """Two files are copies of ONE film only when the film-identity tokens agree
+    (one side may carry none) AND no year evidence contradicts them. The key says
+    the franchise matches; this says the actual film matches — it is what keeps
+    'Evil Dead' (2013) apart from 'Evil Dead Rise' (2023) and the four Austin
+    Powers apart when their runtimes happen to agree."""
+    ba, bb = a["base"], b["base"]
+    if ba and bb and ba != bb:
+        return False
+    ya = a["year"] if a["year"] != "?" else a["soft"]
+    yb = b["year"] if b["year"] != "?" else b["soft"]
+    if ya and yb and ya != yb:
+        return False
+    return True
+
+
+def dur_close(a, b):
+    if a <= 0 or b <= 0:
+        return True                                   # unknown duration: let titles decide
+    return abs(a - b) <= max(2.0, 0.003 * max(a, b))
+
+
+# Blu-ray folder trees: ffprobe only sees the clip it is pointed at, so each tree
+# is folded into ONE entry, described by its longest main-feature clip (measured
+# with ffprobe on the largest BDMV/STREAM file) and sized by the whole tree.
+BDMV_MAIN = {
+    "Avengers.Age.of.Ultron.2015.2160p.BluRay.HEVC.TrueHD.7.1.Atmos-TERMiNAL":
+        dict(dur=8478.2, vcodec="hevc", res="3840x2160", pix="yuv420p10le", fps=23.976),
+    "Full.Metal.Jacket.1987.2160p.BluRay.HEVC.DTS-HD.MA.5.1-EATDIK":
+        dict(dur=6993.3, vcodec="hevc", res="3840x2160", pix="yuv420p10le", fps=23.976),
+}
+
+
+def load(probed):
+    """Probe rows -> film entries. AppleDouble (._*) sidecars are dropped; each
+    BDMV tree collapses to one entry carrying the tree's total size."""
+    trees = {}
+    rows = []
+    for line in open(probed):
+        f = line.rstrip("\n").split("\t")
+        if len(f) < 12:
+            continue
+        b, p, dur, vc, res, fps, vbit, pix, aud, subs, _d, fmt = f[:12]
+        base = os.path.basename(p)
+        if base.startswith("._") or base == ".DS_Store":
+            continue                                    # macOS sidecar, not media
+        m = re.search(r"/(BDMV|VIDEO_TS)/", p)
+        root = p[:m.start()] if m else None
+        if root or re.search(r"/(CERTIFICATE|AACS|MLPI|CLIPINF|PLAYLIST|BACKUP|REGISTRL)/", p):
+            if not root:
+                continue      # disc bookkeeping: folds into its tree, never a film
+        if root:
+            t = trees.setdefault(root, dict(bytes=0, audio=set(), subs=0))
+            t["bytes"] += int(b)
+            t["audio"].update(a for a in aud.split("+") if a)
+            t["subs"] += int(subs or 0)
+            continue
+        try:
+            dur = float(dur)
+        except ValueError:
+            dur = 0.0
+        rows.append(dict(bytes=int(b), path=p, dur=dur, vcodec=vc, res=res, fps=fps,
+                         vbit=int(float(vbit or 0)), pix=pix, audio=aud, subs=subs, fmt=fmt))
+    for root, t in trees.items():
+        folder = os.path.basename(root)
+        m = BDMV_MAIN.get(folder, {})
+        rows.append(dict(bytes=t["bytes"], path=root + "/BDMV/index.bdmv",
+                         name_src=folder, dur=m.get("dur", 0.0),
+                         vcodec=m.get("vcodec", "hevc"), res=m.get("res", "-"),
+                         fps=m.get("fps", 0), vbit=0, pix=m.get("pix", "-"),
+                         audio="+".join(sorted(t["audio"])), subs=str(t["subs"]),
+                         fmt="BDMV tree"))
+    for r in rows:
+        r["title"], r["year"] = title_of(r.get("name_src") or r["path"])
+        r["key"] = key_of(r["title"])
+        r["base"] = base_words(r["title"], r["key"])
+        r["soft"] = folder_year(r["path"])
+        r["ep"] = ep_of(r["path"])
+    return rows
+
+
+def dur_clusters(items):
+    """Group files that are the SAME FILM and the SAME MASTER duration. Candidate
+    buckets come from the franchise key; same_film then rejects the different
+    installments whose runtimes happen to agree."""
+    clusters = []
+    for r in sorted(items, key=lambda r: -r["dur"]):
+        for c in clusters:
+            if dur_close(c[0]["dur"], r["dur"]) and same_film(c[0], r):
+                c.append(r)
+                break
+        else:
+            clusters.append([r])
+    return clusters
+
+
+def tier(groups):
+    """Split same-film/same-duration clusters: DUP = different releases of one
+    film (real, deletable redundancy); REVIEW = identical size AND duration
+    (byte-twins — the hash pass is the authority) or clashing year evidence."""
+    dup, review = [], []
+    for g in groups:
+        g.sort(key=lambda r: -r["bytes"])
+        years = {(r["year"] if r["year"] != "?" else r["soft"]) for r in g}
+        years.discard(None)
+        spread = (g[0]["bytes"] - g[-1]["bytes"]) / g[0]["bytes"] if g[0]["bytes"] else 0
+        if len(years) > 1 or spread < 0.005:
+            review.append(g)
+        else:
+            dup.append(g)
+    return dup, review
+
+
+def redun(g):
+    return sum(r["bytes"] for r in sorted(g, key=lambda r: -r["bytes"])[1:])
+
+
+def line(r):
+    return (f"{r['bytes']/2**30:8.2f} GiB {r['res']:>9} {r['vcodec']:<7} "
+            f"{r['pix']:<11} {r['dur']/60:5.0f}m {r['audio'][:34]:<34}")
+
+
+def dump(tag, groups, out, note=""):
+    for gi, g in enumerate(groups, 1):
+        g.sort(key=lambda r: -r["bytes"])
+        print(f"\n### {tag}-{gi}. {g[0]['title']} ({g[0]['year']}) — {len(g)} files, "
+              f"{redun(g)/2**30:.1f} GiB redundant {note}".rstrip())
+        for rank, r in enumerate(g):
+            print(f"  {line(r)} [{r['year']}] "
+                  f"{r['path'].replace(ROOT + '/', '')}")
+            out.write(f"{tag}\t{gi}\t{rank}\t{r['bytes']}\t{r['res']}\t{r['vcodec']}\t"
+                      f"{r['pix']}\t{int(r['dur'])}\t{r['audio']}\t{r['title']}|{r['year']}\t"
+                      f"{r['path']}\n")
+
+
+def report(probed, out_path):
+    rows = load(probed)
+    for r in rows:
+        r["mark"] = is_watermark(r["path"], r["bytes"])
+        r["extra"] = not r["mark"] and is_extra(r["path"], r["title"], r["dur"])
+    marks = [r for r in rows if r["mark"]]
+    movies = [r for r in rows if not r["ep"] and not r["extra"] and not r["mark"]]
+    episodes = [r for r in rows if r["ep"] and not r["extra"] and not r["mark"]]
+    extras = [r for r in rows if r["extra"]]
+    print(f"census: {len(rows)} video entries = {len(movies)} films + {len(episodes)} "
+          f"TV episodes + {len(extras)} bonus/trailers "
+          f"({sum(r['bytes'] for r in extras)/2**30:.1f} GiB) "
+          f"+ {len(marks)} release-group watermarks")
+
+    out = open(out_path, "w")
+
+    buckets = collections.defaultdict(list)
+    for r in movies:
+        buckets[r["key"]].append(r)
+    cand = {k: v for k, v in buckets.items() if len(v) > 1}
+    clusters = [c for v in cand.values() for c in dur_clusters(v) if len(c) > 1]
+    dup, review = tier(clusters)
+    dup.sort(key=lambda g: -redun(g))
+    review.sort(key=lambda g: -redun(g))
+    dump("DUP", dup, out)
+    dump("REVIEW", review, out, "— same key+duration, sizes near-identical OR years clash: RECHECK")
+
+
+    ebuf = collections.defaultdict(list)
+    for r in episodes:
+        ebuf[(r["ep"][0], r["ep"][1])].append(r)
+    edup, ereview = tier([c for v in ebuf.values() if len(v) > 1
+                          for c in dur_clusters(v) if len(c) > 1])
+    edup.sort(key=lambda g: -redun(g))
+    dump("EP-DUP", edup, out)
+
+    shows = collections.Counter()
+    for r in episodes:
+        shows[r["ep"][0]] += 1
+    multi = {s: n for s, n in shows.items() if n > 4}
+    print("\n=== TV-series content inside Movies/ (>4 episodes) ===")
+    for s, n in sorted(multi.items(), key=lambda kv: -kv[1]):
+        print(f"  {n:4d} episodes  {s}")
+    out.close()
+
+    print(f"\nDUP groups={len(dup)} files={sum(len(g) for g in dup)} "
+          f"redundant={sum(redun(g) for g in dup)/2**40:.2f} TiB", file=sys.stderr)
+    print(f"REVIEW groups={len(review)} "
+          f"redundant={sum(redun(g) for g in review)/2**40:.2f} TiB", file=sys.stderr)
+    print(f"EP-DUP groups={len(edup)} "
+          f"ep-redundant={sum(redun(g) for g in edup)/2**30:.1f} GiB", file=sys.stderr)
+
+
+def parse(names):
+    for n in names:
+        t, y = title_of(n)
+        print(f"{t!r} ({y})\tkey={key_of(t)!r}\t{n[:70]}")
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    sub = ap.add_subparsers(dest="cmd", required=True)
+    rep = sub.add_parser("report", help="similar-copy census over a probe TSV")
+    rep.add_argument("--probed", default="/tmp/media-probed.tsv")
+    rep.add_argument("--out", default="/tmp/media-similar.tsv")
+    par = sub.add_parser("parse", help="show the parsed title/year of a name")
+    par.add_argument("names", nargs="+")
+    a = ap.parse_args()
+    if a.cmd == "report":
+        report(a.probed, a.out)
+    else:
+        parse(a.names)
+
+
+if __name__ == "__main__":
+    main()

scripts/media-tv.py

@@ -0,0 +1,281 @@
+#!/usr/bin/env python3
+"""media-tv.py — move TV episodes out of the movie library into a TV library.
+
+`scripts/media-plan.py` refuses to file TV: it parks episode files in a
+`_TV needs a TV library - <series>` pile for a human. This is the other half of
+that refusal. It walks the movie library, groups the files whose own name
+carries an episode marker by SERIES, and writes a plan that places each series
+in the shape both Jellyfin and Plex read without help:
+
+    <library>/<Series> (<year>)/Season <NN>/<the file's own name>.<ext>
+
+It writes a plan and nothing else. Every rename belongs to
+`scripts/media-apply.py` — the write-ahead journal, the refusal to overwrite,
+the all-or-nothing preflight, the rollback. Apply it with `--root` set to the
+share that holds BOTH libraries (`/mnt/entropy`, not `/mnt/entropy/Movies`) so
+the executor's own path guard still has teeth, and give it its own journal —
+one journal is one undo, and a TV move must not be welded to a movie relayout:
+
+    python3 scripts/media-tv.py --root /mnt/entropy/Movies \
+        --library /mnt/entropy/TVShows --out /mnt/entropy/media-tv-plan.tsv
+    python3 scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv \
+        --root /mnt/entropy --journal /mnt/entropy/media-relayout-tv.jsonl
+    python3 scripts/media-apply.py verify --plan /mnt/entropy/media-tv-plan.tsv \
+        --root /mnt/entropy --journal /mnt/entropy/media-relayout-tv.jsonl
+    python3 scripts/media-apply.py sweep --root /mnt/entropy \
+        --journal /mnt/entropy/media-relayout-tv.jsonl --apply
+
+Guards, and why each one is here:
+
+* The same five-file threshold as the planner. An episode-shaped name on one
+  file is not a series — `The Godfather Part 2` is a film — so a smaller group
+  is reported and left exactly where it is.
+* An episode whose name carries no season number is reported and left, never
+  filed into an invented `Season 01`.
+* A sidecar that shares an episode's own name (`.srt`, `.nfo`, …) follows the
+  episode; an episode without its subtitles is a half-move.
+* A group that would claim one destination twice is not resolved here. The
+  executor refuses the whole plan before its first rename, which is the right
+  outcome: nobody but Frank decides which copy of an episode is the keeper.
+* An episode the destination library already holds at the same byte count is
+  left where it is. The TV library has held a complete Monk box set for years
+  while the movie library carried a second copy of all 124 episodes; a mover
+  that did not look where it was filing would have put both inside one library.
+* A symlink is never followed and never moved.
+"""
+import argparse
+import collections
+import importlib.util
+import os
+import re
+
+
+def parse_args():
+    ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    ap.add_argument("--root", default=os.environ.get("MEDIA_ROOT", "/mnt/entropy/Movies"),
+                    help="the movie library to empty of TV")
+    ap.add_argument("--library", default="/mnt/entropy/TVShows",
+                    help="the TV library to fill (a different share or folder)")
+    ap.add_argument("--out", default=None,
+                    help="plan TSV for scripts/media-apply.py"
+                         " (default: media-tv-plan.tsv beside --root)")
+    ap.add_argument("--min", type=int, default=5,
+                    help="episode files that make something a series")
+    a = ap.parse_args()
+    if not a.out:
+        a.out = os.path.join(os.path.dirname(os.path.abspath(a.root)),
+                             "media-tv-plan.tsv")
+    return a
+
+
+ARGS = parse_args()
+ENGINE = os.path.join(os.path.dirname(os.path.abspath(__file__)), "media-titles.py")
+spec = importlib.util.spec_from_file_location("eng", ENGINE)
+eng = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(eng)
+
+BAD_CHARS = re.compile(r'[<>:"/|?*]')
+# The season and episode, from the token media-titles.ep_of() hands back: s05e03, 1x04.
+SEASON_RE = re.compile(r"^(?:s(\d{1,2})e(\d{1,4})|(\d{1,2})x(\d{1,4}))$")
+# Sidecars that follow an episode; macOS forks and resume data do not.
+SKIP_SIDECAR = re.compile(r"^(\._|\.DS_|dl_state)")
+# A box set that stamps the show's year into every file name; the folder's year
+# must not be added on top of it.
+YEAR_TAIL = re.compile(r"[(\[]\s*(18|19|20)\d\d\s*[)\]]\s*$")
+SMALL = {"a", "an", "the", "and", "of", "for", "in", "on", "to", "with", "at",
+         "by", "de", "la", "vs"}
+
+
+def clean(name):
+    """A name the filesystem and both matchers can read (same law as the plan)."""
+    name = BAD_CHARS.sub("-", name).replace("\\", "-")
+    name = re.sub(r"\s+", " ", name).strip().strip(". ").strip()
+    return name[:180] or "untitled"
+
+
+def episode_name(path):
+    """The file's own name, kept: the publisher's spelling beats a re-derivation.
+
+    A dot-separated release name (`monk.s05e03.mkv`) becomes words; a name that
+    already carries spaces is left exactly as it was written."""
+    stem, ext = os.path.splitext(os.path.basename(path))
+    if " " not in stem and stem.count(".") >= 2:
+        stem = stem.replace(".", " ")
+    return clean(stem) + ext.lower()
+
+
+def head_of(path):
+    """The publisher's series name: the text BEFORE the episode marker.
+
+    `Monk - S05E03 - Mr. Monk and the Big Game` is `Monk`; the episode title
+    after the marker is not the show's name, and case is kept because it is the
+    library's own spelling, not a normalised key."""
+    name = os.path.splitext(os.path.basename(path))[0]
+    m = eng.EP_RE.search(name)
+    head = name[:m.start()] if m else name
+    return re.sub(r"\s+", " ", re.sub(r"[._]+", " ", head)).strip(" -\u2013").strip()
+
+
+def title_series(head):
+    """The series folder in the case a TV library is written in.
+
+    Only a name with no capital at all is retitled: a box set that was
+    lower-cased on the way in would otherwise file `monk (2002)` among
+    `Dexter's Laboratory (1996)`. A name the publisher wrote with case is left
+    exactly as it was written."""
+    if any(c.isupper() for c in head):
+        return head            # the publisher wrote it with case; keep it
+    words = head.split()
+    return " ".join(w if i and w.lower() in SMALL else w.capitalize()
+                    for i, w in enumerate(words))
+
+
+def season_episode(token):
+    """(season, episode) an episode token states, or None when it states none."""
+    m = SEASON_RE.match(token)
+    if not m:
+        return None
+    return int(m.group(1) or m.group(3)), int(m.group(2) or m.group(4))
+
+
+def sidecars(path):
+    """The files beside PATH that carry its exact stem — its subtitles, its nfo."""
+    d, stem = os.path.split(path)
+    stem = os.path.splitext(stem)[0]
+    out = []
+    try:
+        entries = sorted(os.listdir(d))
+    except OSError:
+        return out
+    for e in entries:
+        p = os.path.join(d, e)
+        if p == path or os.path.islink(p) or not os.path.isfile(p):
+            continue
+        s, _ext = os.path.splitext(e)
+        if s == stem and not SKIP_SIDECAR.match(e):
+            out.append(p)
+    return out
+
+
+def episodes_under(root):
+    """(path, series, token) for every episode-shaped video file under root."""
+    for dirpath, dirnames, filenames in os.walk(root):
+        dirnames[:] = sorted(d for d in dirnames
+                             if not d.startswith(".") and d != "@eaDir")
+        for f in filenames:
+            p = os.path.join(dirpath, f)
+            if os.path.islink(p) or not f.lower().endswith(eng.VIDEO_SUFFIX):
+                continue
+            e = eng.ep_of(p)
+            if e:
+                yield p, e[0], e[1]
+
+
+def walk(root, library):
+    """Episode files under ROOT keyed by series, ignoring the destination."""
+    groups = collections.defaultdict(list)
+    lib = os.path.abspath(library)
+    for p, series, token in episodes_under(root):
+        if os.path.abspath(p).startswith(lib + os.sep):
+            continue
+        groups[series].append((p, token))
+    return groups
+
+
+def holdings(library):
+    """What the destination already carries: (series, season, episode) -> sizes.
+
+    The one expensive read here, and the one that matters. The movie library
+    holds a complete Monk box set that the TV library has held for years: 124
+    episodes, the same episode list, identical byte counts. Moving it would
+    have filed a second copy of a whole show into the library that already had
+    it. Byte count is the test on purpose — two encodes of one episode are two
+    files and both are wanted; the same bytes are the same file."""
+    out = collections.defaultdict(set)
+    for p, series, token in episodes_under(library):
+        se = season_episode(token)
+        if se is None:
+            continue
+        try:
+            out[(series, *se)].add(os.path.getsize(p))
+        except OSError:
+            continue
+    return out
+
+
+def main():
+    root, library = os.path.abspath(ARGS.root), os.path.abspath(ARGS.library)
+    groups = walk(root, library)
+    held_here = holdings(library)
+    mkdirs = {library}
+    rows, lines, skipped = set(), [], []
+    for series, items in sorted(groups.items()):
+        if len(items) < ARGS.min:
+            skipped.append(f"skipped: {head_of(items[0][0]) or series} — {len(items)} "
+                           f"episode-shaped file{'s' if len(items) != 1 else ''},"
+                           f" below {ARGS.min}: not a series")
+            continue
+        years = collections.Counter(y for y in (eng.folder_year(p) for p, _t in items)
+                                    if y)
+        year = years.most_common(1)[0][0] if years else None
+        heads = collections.Counter(head_of(p) for p, _t in items)
+        head = heads.most_common(1)[0][0]
+        if year and YEAR_TAIL.search(head):
+            year = None
+        name = clean(f"{title_series(head)}{f' ({year})' if year else ''}")
+        series_dir = os.path.join(library, name)
+        moved = held = 0
+        for path, token in items:
+            se = season_episode(token)
+            if se is None:
+                skipped.append(f"skipped: {path} — no season number in its name,"
+                               " nothing here invents one")
+                continue
+            try:
+                size = os.path.getsize(path)
+            except OSError:
+                continue
+            if size in held_here.get((series, *se), ()):
+                held += 1
+                continue
+            dst_dir = os.path.join(series_dir, f"Season {se[0]:02d}")
+            mkdirs.update([series_dir, dst_dir])
+            base = episode_name(path)
+            rows.add((path, os.path.join(dst_dir, base)))
+            moved += 1
+            for side in sidecars(path):
+                rows.add((side, os.path.join(
+                    dst_dir, clean(os.path.splitext(base)[0]
+                                   + os.path.splitext(side)[1].lower()))))
+        if moved:
+            lines.append(f"{name}: {moved} episodes"
+                         f"{'' if year else ' (no year in any folder name)'}"
+                         f" -> {series_dir}")
+        if held:
+            lines.append(f"{name}: {held}"
+                         f" {'episode' if held == 1 else 'episodes'} already in"
+                         f" {library} at the same byte count — left there")
+
+    # An empty plan creates nothing, not even the library folder: a run that
+    # found no series must not be able to leave a stray directory behind.
+    dirs = sorted(mkdirs, key=lambda p: (p.count(os.sep), p)) if rows else []
+    with open(ARGS.out, "w") as out:
+        out.write("action\tsrc\tdst\tkind\n")
+        for d in dirs:
+            out.write(f"mkdir\t\t{d}\tdir\n")
+        for src, dst in sorted(rows):
+            out.write(f"move\t{src}\t{dst}\ttv\n")
+
+    for line in lines + skipped:
+        print(line)
+    print(f"plan: {len(rows)} moves, {len(dirs)} folders -> {ARGS.out}")
+    if rows:
+        print(f"next: python3 scripts/media-apply.py apply --plan {ARGS.out}"
+              f" --root {os.path.dirname(root)}"
+              f" --journal {os.path.join(os.path.dirname(root),'media-relayout-tv.jsonl')}"
+              "   (add --apply to run it)")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

scripts/media-view.sh

@@ -0,0 +1,241 @@
+#!/usr/bin/env bash
+# The SSD-backed metadata view that lets Jellyfin and Plex share one metadata set.
+#
+# Both servers read a Kodi-format sidecar (movie.nfo / tvshow.nfo / episode.nfo)
+# ONLY when it sits next to the media file. The media lives on the RAID6 array at
+# /mnt/entropy — five spinning disks, deliberately mounted read-only for every
+# container, and slow to seek for a thousand tiny XML reads. So neither media
+# server binds the array any more: they bind an OverlayFS merge of it with a
+# writable layer on the NVMe.
+#
+#   lowerdir  /mnt/entropy                RAID6 (ROTA=1) — source of truth, never written
+#   upperdir  /var/lib/media-meta/upper   NVMe — every sidecar and image byte
+#   workdir   /var/lib/media-meta/work    NVMe — kernel bookkeeping
+#   Where     /var/lib/media-meta/view    the merged tree jellyfin and plex bind
+#
+# Everything the view needs — layers, bookkeeping, trash, the merged door — lives
+# under the one directory /var/lib/media-meta, so nothing new appears under /mnt.
+#
+# Consequences worth knowing:
+#   * A sidecar write is a few KB onto the SSD; the media bytes never move and the
+#     array is still never written by anything automated.
+#   * Deleting through the view writes a whiteout into the upper layer instead of
+#     deleting from the array. Nothing should delete through the view: deletions
+#     go to .AGENTTRASH on the RAW path, and `reap` clears the resulting ghosts.
+#   * inotify events on the lower layer do not cross the merge, so a file that
+#     arrives on the raw array is invisible to a file-system watcher in the view:
+#     trigger a library refresh (see the Media Relayout page).
+#
+# Subcommands
+#   plan   [--upper-base D] [--lower D]  print the resolved shape, touch nothing
+#   verify                               prove the live mount IS that shape
+#   install [--upper-base D]             create the layers, install+enable+start the unit
+#   reap   [--apply] [--upper-base D] [--lower D]
+#                                       list upper-layer entries whose media is gone
+#   status                               one line per fact a human wants
+#
+# install must run as root (it writes /etc/systemd/system and mounts); the others
+# are read-only and safe unprivileged.
+set -euo pipefail
+
+LOWER=/mnt/entropy
+UPPER_BASE=/var/lib/media-meta
+VIEW=$UPPER_BASE/view
+# systemd names a mount unit after its ESCAPED path ('/' -> '-', a literal '-'
+# -> '\x2d'), so the unit on the host is var-lib-media\x2dmeta-view.mount. The
+# repo copy keeps a readable name; install writes it under the escaped one.
+UNIT='var-lib-media\x2dmeta-view.mount'
+UNIT_DIR=/etc/systemd/system
+# The rootless containers write as this host uid (the deploy user). The upper
+# layer has to accept those writes, so it is owned by them.
+MEDIA_UID="${MEDIA_VIEW_UID:-1000}"
+MEDIA_GID="${MEDIA_VIEW_GID:-1000}"
+
+REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
+UNIT_SRC="$REPO_ROOT/dankserver/host/systemd/media-meta-view.mount"
+
+die() {
+    printf 'media-view: %s\n' "$*" >&2
+    exit 1
+}
+
+usage() {
+    printf 'usage: %s plan|verify|install|reap|status [options]\n' "$0" >&2
+    exit 2
+}
+
+cmd="${1:-}"
+[ -n "$cmd" ] || usage
+shift || true
+
+upper_base="$UPPER_BASE"
+lower="$LOWER"
+apply=0
+while [ $# -gt 0 ]; do
+    case "$1" in
+        --upper-base) [ $# -ge 2 ] || usage; upper_base="$2"; shift 2 ;;
+        --lower)      [ $# -ge 2 ] || usage; lower="$2"; shift 2 ;;
+        --apply)      apply=1; shift ;;
+        *) usage ;;
+    esac
+done
+
+upper="$upper_base/upper"
+work="$upper_base/work"
+
+# The device backing a path, and whether the kernel calls it rotational.
+rotational_of() {
+    local src rota
+    src="$(findmnt -no SOURCE --target "$1" 2>/dev/null || true)"
+    [ -n "$src" ] || die "cannot find a block device behind $1"
+    rota="$(lsblk -no ROTA "$src" 2>/dev/null | head -1 || true)"
+    [ -n "$rota" ] || die "cannot read the rotational flag of $src"
+    printf '%s %s\n' "$src" "$rota"
+}
+
+require_upper_base() {
+    [ -d "$upper_base" ] \
+        || die "refused: upper base $upper_base does not exist (put it on the SSD first)"
+}
+
+do_plan() {
+    printf 'lowerdir=%s\nupperdir=%s\nworkdir=%s\nmountpoint=%s\nunit=%s/%s\n' \
+        "$lower" "$upper" "$work" "$VIEW" "$UNIT_DIR" "$UNIT"
+}
+
+do_verify() {
+    local fail=0 fst opts src rota
+    if ! mountpoint -q "$VIEW"; then
+        printf 'refused: %s is not a mountpoint — a plain directory here is a\n' "$VIEW" >&2
+        printf '  library that scans itself empty, not a metadata view.\n' >&2
+        return 1
+    fi
+    fst="$(findmnt -no FSTYPE --mountpoint "$VIEW" 2>/dev/null || true)"
+    if [ "$fst" != overlay ]; then
+        printf 'refused: %s is mounted %s, not overlay\n' "$VIEW" "${fst:-nothing}" >&2
+        fail=1
+    fi
+    opts="$(findmnt -no OPTIONS --mountpoint "$VIEW" 2>/dev/null || true)"
+    local want
+    for want in "lowerdir=$LOWER" "upperdir=$UPPER_BASE/upper" "workdir=$UPPER_BASE/work"; do
+        case ",$opts," in
+            *",${want},"*) ;;
+            *) printf 'refused: %s options lack %s (got: %s)\n' "$VIEW" "$want" "$opts" >&2
+               fail=1 ;;
+        esac
+    done
+    read -r src rota <<<"$(rotational_of "$UPPER_BASE")"
+    if [ "$rota" != 0 ]; then
+        printf 'refused: the upper layer sits on %s which is rotational — metadata\n' "$src" >&2
+        printf '  belongs on the SSD, not on the array.\n' >&2
+        fail=1
+    fi
+    if [ ! -w "$UPPER_BASE/upper" ]; then
+        printf 'refused: %s is not writable\n' "$upper" >&2
+        fail=1
+    fi
+    [ "$fail" -eq 0 ] || return 1
+    printf 'view ok: %s <- overlay(%s,%s)\n' "$VIEW" "$LOWER" "$UPPER_BASE/upper"
+}
+
+do_install() {
+    [ "$(id -u)" -eq 0 ] || die "install must run as root (writes $UNIT_DIR and mounts)"
+    require_upper_base
+    mountpoint -q "$LOWER" \
+        || die "refused: $LOWER is not mounted; overlaying an empty directory would present an empty library"
+    local src rota
+    read -r src rota <<<"$(rotational_of "$upper_base")"
+    [ "$rota" = 0 ] || die "refused: $upper_base is on $src, a rotational device — the metadata layer must be on the SSD"
+    [ -f "$UNIT_SRC" ] || die "refused: $UNIT_SRC is missing from $REPO_ROOT"
+    # systemd will not create a mount point; a missing directory fails the start.
+    install -d -m 0755 -o "$MEDIA_UID" -g "$MEDIA_GID" "$upper" "$work" "$VIEW"
+    install -d -m 0700 "$upper_base/.AGENTTRASH"
+    install -m 0644 "$UNIT_SRC" "$UNIT_DIR/$UNIT"
+    systemctl daemon-reload
+    systemctl enable "$UNIT" >/dev/null 2>&1 || true
+    systemctl start "$UNIT"
+    do_verify
+}
+
+# An upper-layer DIRECTORY with no counterpart below is metadata for a media
+# folder that has been moved or renamed on the RAW path — a ghost the media
+# servers would still browse. Files are never candidates on their own: a sidecar
+# (movie.nfo, an .srt) exists ONLY above, by design, and is exactly what the view
+# is for. A copy-up whose lower file was deleted by hand is the one case this
+# leaves in place; the raw path is where deletions belong, so it stays visible.
+do_reap() {
+    [ -d "$upper" ] || die "refused: $upper does not exist (no upper layer at $upper_base)"
+    stamp="$upper_base/.AGENTTRASH/$(date -u +%Y%m%d-%H%M%S)"
+    count=0
+    ghosts=0
+    reap_walk ""
+    if [ "$count" -eq 0 ] && [ "$ghosts" -eq 0 ]; then
+        printf 'reap: no ghosts under %s\n' "$upper"
+        return 0
+    fi
+    if [ "$ghosts" -gt 0 ]; then
+        printf 'reap: %d whiteout%s above — someone deleted THROUGH the view.\n' \
+            "$ghosts" "$([ "$ghosts" -eq 1 ] || printf s)" >&2
+        printf '  Leaving them: removing a whiteout un-hides the file it hides.\n' >&2
+        printf '  Deletions belong in .AGENTTRASH on the raw path.\n' >&2
+    fi
+    if [ "$count" -eq 0 ]; then return 0; fi
+    if [ "$apply" -eq 1 ]; then
+        printf 'reap: moved %d entr%s to %s\n' "$count" \
+            "$([ "$count" -eq 1 ] && printf y || printf ies)" "$stamp"
+    else
+        printf 'reap: %d ghost entr%s above; re-run with --apply to move them to %s\n' \
+            "$count" "$([ "$count" -eq 1 ] && printf y || printf ies)" "$stamp"
+    fi
+}
+
+# Descend only where the lower layer still has the directory: a directory that is
+# missing below is itself the ghost, and its whole subtree goes with it.
+reap_walk() {
+    local rel="$1" entries abs r name
+    entries="$(find "$upper${rel:+/$rel}" -mindepth 1 -maxdepth 1 -print 2>/dev/null || true)"
+    [ -n "$entries" ] || return 0
+    while IFS= read -r abs; do
+        r="${abs#"$upper"/}"
+        name="${abs##*/}"
+        if [ -c "$abs" ] || printf '%s' "$name" | grep -q '^\.wh\.'; then
+            ghosts=$((ghosts + 1))
+            continue
+        fi
+        if [ -e "$lower/$r" ]; then
+            [ -d "$abs" ] && reap_walk "$r"
+            continue
+        fi
+        [ -d "$abs" ] || continue
+        count=$((count + 1))
+        printf '%s\n' "$r"
+        if [ "$apply" -eq 1 ]; then
+            mkdir -p "$stamp/$(dirname "$r")"
+            mv "$abs" "$stamp/$r"
+        fi
+    done <<<"$entries"
+}
+
+do_status() {
+    if mountpoint -q "$VIEW"; then
+        printf 'view: %s (%s)\n' "$VIEW" "$(findmnt -no OPTIONS --mountpoint "$VIEW")"
+    else
+        printf 'view: %s NOT MOUNTED\n' "$VIEW"
+    fi
+    if [ -d "$upper" ]; then
+        printf 'upper: %s %s files %s\n' "$upper" \
+            "$(find "$upper" -type f 2>/dev/null | wc -l)" "$(du -sh "$upper" 2>/dev/null | cut -f1)"
+    else
+        printf 'upper: %s missing\n' "$upper"
+    fi
+    systemctl is-active "$UNIT" 2>/dev/null | sed 's/^/unit: /' || printf 'unit: %s unknown\n' "$UNIT"
+}
+
+case "$cmd" in
+    plan)    do_plan ;;
+    verify)  do_verify ;;
+    install) do_install ;;
+    reap)    do_reap ;;
+    status)  do_status ;;
+    *)       usage ;;
+esac

scripts/qwen38-calib-window.sh

@@ -0,0 +1,143 @@
+#!/usr/bin/env bash
+# qwen38-calib-window.sh — RUNS ON DANKSERVER. The calibration window, and nothing else:
+# boot the BF16 rig, wait until it is actually serving, push the corpus until the engine says
+# the tap is full, stop the rig, restore frank-archon, and bank the reference container.
+#
+# The reference recording is already taken (kld-ref/kld.json exists), so this window is the
+# calibration tap alone — the one job that needs both cards after it. The kldref phase is NOT
+# re-run here: the chained launcher that used to do that moved the banked kld-ref into trash
+# first, which is how a 16-hour phase gets paid for twice.
+#
+# Every phase is gated on the previous one, because the last time this ran unguarded
+# (2026-10-29) the rig's boot died of an OOM, the health loop quietly fell through, and the
+# pusher fed four rounds of 10M tokens into RESTORED PRODUCTION with nothing recorded:
+#
+#   1. the runner must still be alive and its container must answer /health AND own the
+#      listening socket, or this script kills it, restores the service and exits non-zero —
+#      no push, ever. The socket clause is not decoration: the runner stops frank-archon
+#      first, the dying process keeps its listener for a moment, and a rig container that
+#      is still loading has not bound the port yet, so `podman inspect` + `/health` both
+#      report success for the WRONG process (measured 2026-10-30: `health ok waited=0s`,
+#      then the pusher refused on the container name and the window died at second zero);
+#   2. the pusher refuses on its own unless the rig wrote its marker into the tap dir, the
+#      marker names this container, and the container is running (scripts/calib-push.py
+#      --tap-marker), so a second lock survives a mistake in this file;
+#   3. the reference container is banked into trash only when BOTH the serve and the push
+#      exited 0 and the tap dir holds grams. The checkpoint that reproduces it stays.
+#
+# knobs: Q38_WINDOW_CALIB_CORPUS (calib-7168.jsonl — the re-chunked file the rig's pool can
+#        hold), Q38_WINDOW_HEALTH_TIMEOUT (seconds to wait for /health), Q38_WINDOW_ROUNDS
+#        (pusher rounds; one is 10M tokens), Q38_WINDOW_KEEP_RAD=1 to leave the .rad in place.
+# Usage: scripts/qwen38-calib-window.sh [--dry-run]
+set -uo pipefail
+cd "$(dirname "${BASH_SOURCE[0]}")/.." || exit 9
+
+DRY=0
+[ "${1:-}" = "--dry-run" ] && DRY=1
+say() { printf '%s %s\n' "$(date -Is)" "$*"; }
+
+MODELS="${FRANK_QWEN38_MODELS:-/home/dank/models}"
+CAMPAIGN="${FRANK_QWEN38_CAMPAIGN:-/home/dank/campaign/qwen38-w5nl64}"
+RAD="${FRANK_QWEN38_RAD:-$MODELS/rad/qwen38-bf16.rad}"
+CORPUS="${Q38_WINDOW_CALIB_CORPUS:-/home/dank/corpora/out/calib-7168.jsonl}"
+CALIB="${FRANK_QWEN38_CALIB_DIR:-$CAMPAIGN/calib}"
+HEALTH_TIMEOUT="${Q38_WINDOW_HEALTH_TIMEOUT:-3600}"
+ROUNDS="${Q38_WINDOW_ROUNDS:-6}"
+RIG=qwen38-bf16
+SERVICE=frank-archon
+PY="${PY:-python3}"
+
+# ---- preflight: every fact by identity, one owner (scripts/qwen38-preflight.sh) ----
+# The checks that used to live here were proxies, and each lied at least once: the space check
+# asked a directory that merely LOOKED related, `/health` answered for a dying production
+# process, `podman inspect` named a container that existed but had bound nothing. The shared
+# script binds each fact to its object — cwd, the digest the tag actually resolves to, free
+# space on the tap's own filesystem, every /dev/kfd holder's HOST pid against the service's,
+# the :7770 listener's host pid, the corpus and the reference recording. A resumed window
+# keeps an existing tap, so the tap-empty fact is waived.
+[ -f "$RAD" ]    || { echo "REFUSED: no BF16 reference container at $RAD — nothing to serve; scripts/build-qwen38-bf16-rad.sh" >&2; exit 2; }
+FRANK_QWEN38_CAMPAIGN="$CAMPAIGN" \
+Q38_WINDOW_CALIB_CORPUS="$CORPUS" \
+Q38_CALIB_MIN_FREE_MIB="${Q38_CALIB_MIN_FREE_MIB:-61440}" \
+    bash scripts/qwen38-preflight.sh --allow-existing-tap || exit 2
+! [[ "$HEALTH_TIMEOUT" =~ ^[0-9]+$ ]] && { echo "REFUSED: Q38_WINDOW_HEALTH_TIMEOUT must be seconds (got: $HEALTH_TIMEOUT)" >&2; exit 2; }
+! [[ "$ROUNDS" =~ ^[0-9]+$ ]] && { echo "REFUSED: Q38_WINDOW_ROUNDS must be a round count (got: $ROUNDS)" >&2; exit 2; }
+
+if [ "$DRY" = 1 ]; then
+    echo "dry run: calib window — the service is left running, nothing is mounted or written"
+    echo "would run: bash scripts/run-qwen38-bf16.sh calib          (rig boots, writes $CALIB/.rig-boot)"
+    echo "would wait: /health up within ${HEALTH_TIMEOUT}s, else kill the rig, restore $SERVICE, exit 1"
+    echo "would run: $PY scripts/calib-push.py --container $RIG --corpus $CORPUS --max-rounds $ROUNDS"
+    echo "would bank: $RAD -> ~/.AGENTTRASH/$(date +%Y%m%d)-qwen38-bf16-rad/  (only if both phases exit 0)"
+    exit 0
+fi
+
+say "calib window start pid=$$ corpus=$CORPUS rounds=$ROUNDS"
+[ -n "$(ls -A "$CALIB" 2>/dev/null)" ] && say "tap dir $CALIB is not empty; the engine's token floor is pooled, so an earlier partial tap still counts"
+
+# ---- phase 1: the rig -------------------------------------------------------
+say "phase=calib serve begin"
+bash scripts/run-qwen38-bf16.sh calib & spid=$!
+
+rig_up() {
+    podman inspect -f '{{.State.Running}}' "$RIG" 2>/dev/null | grep -qx true || return 1
+    curl -sf --max-time 10 -o /dev/null http://127.0.0.1:7770/health || return 1
+    # The answer must come from the CONTAINER. The runner stops frank-archon first, and the
+    # dying production process keeps its listener for a moment while the freshly created
+    # rig container is still loading — so `container running` + `/health` 200 both hold
+    # before the rig has bound anything. Measured 2026-10-30: `health ok waited=0s`, then the
+    # pusher refused on the container name and the window died at second zero. Ask the kernel
+    # who owns the listening socket and require it to be a rig process.
+    local lpid
+    lpid=$(ss -H -ltnp 'sport = :7770' 2>/dev/null | grep -o 'pid=[0-9]*' | head -1 | cut -d= -f2)
+    [ -n "${lpid:-}" ] || return 1
+    podman top "$RIG" pid 2>/dev/null | grep -qw "$lpid"
+}
+
+waited=0
+until rig_up; do
+    if ! kill -0 "$spid" 2>/dev/null; then
+        wait "$spid"; crc=$?
+        say "phase=calib serve end rc=$crc (the rig exited before it ever answered /health)"
+        echo "FAIL: the rig boot died (rc=$crc) — nothing was pushed; read /home/dank/logs/qwen38_bf16_calib.log" >&2
+        exit 1
+    fi
+    if [ "$waited" -ge "$HEALTH_TIMEOUT" ]; then
+        say "phase=calib health never came after ${HEALTH_TIMEOUT}s; killing the rig"
+        podman stop "$RIG" >/dev/null 2>&1 || true
+        wait "$spid"
+        echo "FAIL: the rig never answered /health within ${HEALTH_TIMEOUT}s — nothing was pushed" >&2
+        exit 1
+    fi
+    sleep 15
+    waited=$((waited + 15))
+done
+say "phase=calib health ok waited=${waited}s"
+
+# ---- phase 2: the tap -------------------------------------------------------
+say "phase=calib push begin"
+"$PY" scripts/calib-push.py --port 7770 --parallel 8 --container "$RIG" \
+    --tap-dir "$CALIB" --corpus "$CORPUS" --max-rounds "$ROUNDS"
+prc=$?
+say "phase=calib push end rc=$prc"
+
+podman stop "$RIG" >/dev/null 2>&1 || true
+wait "$spid"; crc=$?
+say "phase=calib serve end rc=$crc"
+
+if [ "$prc" -ne 0 ] || [ "$crc" -ne 0 ]; then
+    echo "FAIL: serve rc=$crc push rc=$prc — the tap is NOT banked and $RAD stays: $(basename "$RAD")" >&2
+    exit 1
+fi
+[ -n "$(ls -A "$CALIB" 2>/dev/null)" ] || { echo "FAIL: both phases exited 0 but the tap dir $CALIB is empty" >&2; exit 1; }
+say "TAP BANKED: $(ls -1 "$CALIB" | wc -l) file(s) in $CALIB"
+
+# ---- phase 3: bank the reference container ----------------------------------
+if [ "${Q38_WINDOW_KEEP_RAD:-0}" = "1" ]; then
+    say "reference .rad kept (Q38_WINDOW_KEEP_RAD=1)"
+else
+    T="${Q38_WINDOW_TRASH_ROOT:-$HOME/.AGENTTRASH}/$(date +%Y%m%d)-qwen38-bf16-rad"
+    mkdir -p "$T" || { echo "FAIL: cannot create trash dir $T" >&2; exit 1; }
+    mv "$RAD" "$T/" && say "reference .rad banked in $T (the checkpoint that rebuilds it stays)"
+fi
+say "calib window end"

scripts/qwen38-preflight.sh

@@ -0,0 +1,136 @@
+#!/usr/bin/env bash
+# qwen38-preflight.sh — the facts a calibration window must have BEFORE it takes the cards,
+# each checked by IDENTITY and not by proxy. Every line is PASS or FAIL with the reason.
+#
+# Why this file exists: on 2026-10-30 a window died at second zero because `/health` answered
+# 200 from the *production* process that was still draining its listener, while the freshly
+# created rig container had not bound anything. Other runs died from a port that answered for
+# the wrong process, a /dev/kfd held by an orphaned engine, a tap dir on a 35 GiB filesystem,
+# and a launcher whose cwd silently lacked the repo. None of those were math problems; all of
+# them were a check that measured something adjacent to the thing it meant.
+#
+# Usage:  bash scripts/qwen38-preflight.sh [--allow-existing-tap]
+# Exit:   0 all PASS · 2 any FAIL · 3 a fact could not be evaluated (missing tool)
+set -uo pipefail
+
+SELF_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(cd "$SELF_DIR/.." && pwd)"
+
+CAMPAIGN="${FRANK_QWEN38_CAMPAIGN:-/home/dank/campaign/qwen38-w5nl64}"
+TAP="$CAMPAIGN/calib"
+CORPUS="${Q38_WINDOW_CALIB_CORPUS:-/home/dank/corpora/out/calib-7168.jsonl}"
+NEED_MIB="${Q38_CALIB_MIN_FREE_MIB:-61440}"
+ALLOW_EXISTING=0
+[ "${1:-}" = "--allow-existing-tap" ] && ALLOW_EXISTING=1
+
+# The producer pin is the calib image; the window fails closed if the tag on this host is not it.
+WANT_DIGEST="${Q38_PREFLIGHT_IMAGE_DIGEST:-sha256:69add58f7de1ed2bca2b1e20a4cc75717557df3f73569a178eca5aafe8c44093}"
+IMAGE_REF="${Q38_CALIB_IMAGE:-stilldeadcode/radiance@$WANT_DIGEST}"
+SERVICE="frank-archon"
+PORT=7770
+
+fails=0
+ok()   { printf 'PASS  %-14s %s\n' "$1" "$2"; }
+bad()  { printf 'FAIL  %-14s %s\n' "$1" "$2"; fails=$((fails + 1)); }
+skip() { printf 'WARN  %-14s %s\n' "$1" "$2"; }
+
+# 1. cwd — a detached launcher that lost its cwd fails here in a second, not in the log later.
+if [ "$(cd "$REPO_ROOT" && pwd)" = "$(pwd)" ] && [ -x "$REPO_ROOT/scripts/run-qwen38-bf16.sh" ]; then
+    ok cwd "running from the repo root ($REPO_ROOT)"
+else
+    bad cwd "cwd is $(pwd), not the repo root ($REPO_ROOT) — every relative script path would break"
+fi
+
+# 2. image — resolve the REF to a digest. A stale local `:latest` is a different model of the world.
+if command -v podman >/dev/null 2>&1; then
+    # An image carries EVERY digest it was pulled under (a tag plus the by-digest pull), so this
+    # asks whether the pin is one of them — reading RepoDigests[0] reports a healthy image as wrong.
+    got="$(podman image inspect -f '{{range .RepoDigests}}{{println .}}{{end}}' "$IMAGE_REF" 2>/dev/null \
+           | sed 's/.*@//' | grep -c "^${WANT_DIGEST}$")"
+    if [ "${got:-0}" -ge 1 ]; then
+        ok image "$IMAGE_REF carries ${WANT_DIGEST:0:19}…"
+    elif podman image exists "$IMAGE_REF" 2>/dev/null; then
+        bad image "$IMAGE_REF is present but does not carry $WANT_DIGEST (a retagged or older pull)"
+    else
+        bad image "$IMAGE_REF is not present on this host"
+    fi
+else
+    skip image "podman not on PATH — cannot resolve $IMAGE_REF"
+fi
+
+# 3. space — df the FILE PATH the seal writes to, never a directory that merely looks related.
+if [ -d "$TAP" ] || mkdir -p "$TAP" 2>/dev/null; then
+    avail="$(df --output=avail -B 1M "$TAP" 2>/dev/null | tail -1 | tr -d ' ')"
+    if [[ "${avail:-}" =~ ^[0-9]+$ ]] && [ "$avail" -ge "$NEED_MIB" ]; then
+        ok space "$avail MiB free on $(df --output=target "$TAP" | tail -1) (need $NEED_MIB)"
+    else
+        bad space "${avail:-unknown} MiB free on $TAP, need $NEED_MIB — one seal writes ~43 GB"
+    fi
+    # 4. tap cleanliness — stale grams from another run must never be mixed in.
+    n=$(ls -A "$TAP" 2>/dev/null | grep -c . || true)
+    if [ "$n" -eq 0 ] || [ "$ALLOW_EXISTING" = 1 ]; then
+        ok tap-empty "$n entries in $TAP"
+    else
+        bad tap-empty "$n entries in $TAP — rotate them to .AGENTTRASH first"
+    fi
+else
+    bad space "cannot create $TAP"
+fi
+
+# 5. device — every /dev/kfd holder must be a process of the service we are about to stop. An
+#    orphaned engine from a previous run makes the next boot die with `startup stopped: device
+#    error`. Identity comes from podman's own host-pid list: `podman top <ctr> pid` prints
+#    CONTAINER-internal pids (1, 7, …), which compare against nothing, and a rootless cgroup
+#    carries the container ID, never the service name — both are proxies that lie.
+PROD_CTR="${Q38_PREFLIGHT_PROD_CONTAINER:-$SERVICE}"
+service_pids() { podman top "$PROD_CTR" hpid 2>/dev/null | tail -n +2; }
+if [ -e /dev/kfd ]; then
+    mine="$(service_pids | tr '\n' ' ')"
+    foreign=""
+    for p in $(fuser /dev/kfd 2>/dev/null); do
+        case " $mine " in
+            *" $p "*) : ;;                                          # the service we will stop
+            *) foreign="$foreign $p:$(cat /proc/$p/comm 2>/dev/null)" ;;
+        esac
+    done
+    if [ -z "$foreign" ]; then
+        ok device "no foreign /dev/kfd holders"
+    else
+        bad device "foreign holders:$foreign — kill by pid, never by pattern"
+    fi
+else
+    skip device "/dev/kfd absent — not an AMD host?"
+fi
+
+# 6. port — a listener on :7770 must be a process of the service we stop, never an unknown one.
+lpid="$(ss -H -ltnp "sport = :$PORT" 2>/dev/null | grep -o 'pid=[0-9]*' | head -1 | cut -d= -f2)"
+if [ -z "${lpid:-}" ]; then
+    ok port ":$PORT is free"
+else
+    mine="$(service_pids | tr '\n' ' ')"
+    case " $mine " in
+        *" $lpid "*) ok port ":$PORT is held by $SERVICE (host pid $lpid) — the window stops it" ;;
+        *)           bad port ":$PORT is held by host pid $lpid ($(cat /proc/$lpid/comm 2>/dev/null)), not $SERVICE" ;;
+    esac
+fi
+
+# 7. corpus — the push source must exist and be the re-chunked file the window documents.
+if [ -s "$CORPUS" ]; then
+    ok corpus "$(wc -l < "$CORPUS") prompts in $CORPUS"
+else
+    bad corpus "$CORPUS is missing or empty"
+fi
+
+# 8. reference — the KL gate has nothing to compare against without it.
+if [ -s "$CAMPAIGN/kld-ref/kld.json" ]; then
+    ok reference "$(du -h "$CAMPAIGN/kld-ref/kld.json" | cut -f1) kld.json present"
+else
+    bad reference "$CAMPAIGN/kld-ref/kld.json is missing"
+fi
+
+echo
+if [ "$fails" -gt 0 ]; then
+    echo "PREFLIGHT FAILED: $fails fact(s). The window must not take the cards."
+    exit 2
+fi
+echo "PREFLIGHT PASSED: every fact checked by identity."

scripts/run-qwen38-bf16.sh

@@ -28,10 +28,18 @@
 #                     (RADIANCE_CALIB_DIR + RADIANCE_CALIB_TOKENS=12000000). A
 #                     separate driver — scripts/calib-push.py --port 7770
 #                     --container qwen38-bf16 --corpus
-#                     /home/dank/corpora/out/calib.jsonl — pushes the corpus; at
-#                     12M pooled positions the engine writes the Grams and the
-#                     tap switches itself off (calib-push prints TAP_DONE). Stop
-#                     this script after that; the trap brings frank-archon back.
+#                     /home/dank/corpora/out/calib-7168.jsonl — pushes the corpus;
+#                     at the floor the engine writes the Grams and the tap switches
+#                     itself off (calib-push prints TAP_DONE). Stop this script
+#                     after that; the trap brings frank-archon back. THE FLOOR IS
+#                     THE CAMPAIGN'S RULE (12 M tokens, Frank 2026-10-30): the
+#                     served iq4r this arm competes with was calibrated on a 10 M-
+#                     token set, so the arm is calibrated at least as deeply. One
+#                     round of the re-chunked corpus is ~9.31 M tokens, so sealing
+#                     takes round 1 plus part of round 2 — a round that seals
+#                     nothing is NORMAL (the floor is pooled), and the pusher's
+#                     round_verdict treats a flat round as "continue", never as a
+#                     failure. One seal writes ~40 GiB.
 #
 # Shape: the vendor's bf16 calibration lines — expert_tiered placement with the
 # weights disk tier, deterministic (the resident set pinned to the loader's plan,
@@ -92,7 +100,8 @@
 #        the container reads through /models), FRANK_QWEN38_LOG_DIR, and the step
 #        and slot counts inherited as FRANK_ARCHON_MAX_BATCHED_TOKENS /
 #        FRANK_ARCHON_MAX_SEQS — no per-mode literals (the audit law: a mode that
-#        carries its own step measures a shape nothing serves) —
+#        carries its own step measures a shape nothing serves; calib is exempt, see
+#        Q38_CALIB_MNBT / Q38_CALIB_SEQS below — its output is never compared) —
 #        and FRANK_QWEN38_MAX_MODEL_LEN — the window every mode boots at (32,768;
 #        see the corpus census above, and raise it together with the corpus).
 # kldref refuses (exit 2) when the longest record of the corpus it is about to
@@ -112,6 +121,18 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 # shellcheck disable=SC1091
 source "$SCRIPT_DIR/frank-archon.env"
 IMAGE="${FRANK_ARCHON_IMAGE:?FRANK_ARCHON_IMAGE must name the radiance image}"
+# THE CALIBRATION PIN: the LATEST radiance, pinned by digest because a tag is a
+# moving target and a tap must name the producer that wrote it. `:latest` on
+# 2026-10-30 resolves to v1.14.0 (revision 4a61d7296bc5), and on that image a
+# calib boot prints the arming line — "recorded passes: off, the program
+# accumulates calibration Grams (gram_accum)" — where the production pin
+# (1.8.1) prints nothing of the kind: a 17-hour run on 1.8.1 wrote ZERO bytes
+# into the tap dir while /health, /stats and the push all looked healthy
+# (measured 2026-10-30). The tap code is identical in both trees' sources, so
+# the shipped 1.8.1 build is what drops it; do not spend another window on it.
+# Any other image must be named deliberately with --image.
+CALIB_IMAGE="${Q38_CALIB_IMAGE:-stilldeadcode/radiance@sha256:69add58f7de1ed2bca2b1e20a4cc75717557df3f73569a178eca5aafe8c44093}"
+IMAGE_EXPLICIT=0
 
 MODE=""
 POS=()
@@ -121,7 +142,7 @@ while [ $# -gt 0 ]; do
         kldref|calib|score) MODE="$1" ;;
         --dry-run)          DRY=1 ;;
         --image)            [ "$#" -ge 2 ] || { echo "--image needs a reference" >&2; exit 2; }
-                            IMAGE="$2"; shift ;;
+                            IMAGE="$2"; IMAGE_EXPLICIT=1; shift ;;
         *)
             # score's two positional arguments, and nothing else may be bare.
             if [ "$MODE" = score ] && [ "${#POS[@]}" -lt 2 ]; then
@@ -151,7 +172,7 @@ TEMPLATE="$SCRIPT_DIR/qwen38-froggeric-v22.5.jinja"
 LOG_DIR="${FRANK_QWEN38_LOG_DIR:-/home/dank/logs}"
 LOG="$LOG_DIR/qwen38_bf16_${MODE}.log"
 KLD_REF_DIR="$CAMPAIGN/kld-ref"
-CALIB_DIR="$CAMPAIGN/calib"
+CALIB_DIR="${FRANK_QWEN38_CALIB_DIR:-$CAMPAIGN/calib}"
 C=qwen38-bf16
 SERVICE=frank-archon
 # The card split, taken from the file that owns it instead of a private pair:
@@ -185,6 +206,24 @@ if ! [[ "$MNBT" =~ ^[0-9]+$ ]]; then
     exit 2
 fi
 
+# Calib's shape is NOT a measuring shape. kldref and score produce KL numbers that are only
+# comparable across boots at an identical shape, which is why they inherit production's step and
+# slots to the letter. The tap's output is layer Hessians, never a number any report compares, so
+# its step and slot count are its own — and they are the tap's dominant cost: a 7,168-token
+# document cut into four 2,048-token steps re-streams each layer's routed expert union four
+# times from the container, so one wide step is worth up to ~4x the read volume. Measured
+# 2026-10-29 at the inherited 2,048/4: prefill 71 tok/s and a 47-hour tap; the probe exists to
+# find out what the wide step is worth before the campaign pays for two days of drained serving.
+# Both default to the inherited value, so nothing moves unless a probe asks it to.
+CALIB_MNBT="${Q38_CALIB_MNBT:-$MNBT}"
+CALIB_SEQS="${Q38_CALIB_SEQS:-$MAX_SEQS}"
+for pair in "Q38_CALIB_MNBT=$CALIB_MNBT" "Q38_CALIB_SEQS=$CALIB_SEQS"; do
+    if ! [[ "${pair#*=}" =~ ^[0-9]+$ ]]; then
+        echo "REFUSED: ${pair%%=*} must be a number (got: ${pair#*=})" >&2
+        exit 2
+    fi
+done
+
 # The window every mode boots at. Over-length is a REFUSAL, not a truncation (the
 # env file says so for production), so this is the number the corpus must fit in.
 MAX_MODEL_LEN="${FRANK_QWEN38_MAX_MODEL_LEN:-32768}"
@@ -193,6 +232,25 @@ if ! [[ "$MAX_MODEL_LEN" =~ ^[0-9]+$ ]]; then
     exit 2
 fi
 
+# Where the n-gram table lives. On the bf16 payload it is 95.37 GiB and `ram`
+# PINS it: the calibration boot died at "startup stopped: out of memory" with
+# 230.22 GiB pinned of 246.07 GiB and 8.06 GiB available (measured 2026-10-29),
+# while the reference recording and every score boot had already succeeded with
+# the same pool and slots. The difference that matters is the job: calib only
+# needs the routed weights prefilled, so it reads the table from the container's
+# own file tier and the pin never happens; kldref and score keep production's
+# placement, because their numbers are only comparable when every other term of
+# the shape is identical. FRANK_QWEN38_NGRAM_PLACEMENT overrides either default.
+case "$MODE" in
+    calib) NGRAM_PLACEMENT="${FRANK_QWEN38_NGRAM_PLACEMENT:-disk}" ;;
+    *)     NGRAM_PLACEMENT="${FRANK_QWEN38_NGRAM_PLACEMENT:-ram}" ;;
+esac
+case "$NGRAM_PLACEMENT" in
+    auto|vram|ram|disk) ;;
+    *) echo "REFUSED: FRANK_QWEN38_NGRAM_PLACEMENT must be auto|vram|ram|disk (got: $NGRAM_PLACEMENT)" >&2
+       exit 2 ;;
+esac
+
 # Every container this campaign boots is read through the ro /models mount, so
 # the host path must live under it; the container-side spelling is what goes
 # into the argv. Same rule for the container a score pass opens.
@@ -230,7 +288,7 @@ ARGS=(--tp 2 --placement expert_tiered --weights-disk-tier
       --host-pool-mib "$HOST_POOL_MIB" --gpu-headroom-mib "$GPU_HEADROOM_MIB"
       --deterministic --num-speculative-tokens 0
       --kv-cache-dtype fp8 --max-model-len "$MAX_MODEL_LEN"
-      --ngram-placement ram
+      --ngram-placement "$NGRAM_PLACEMENT"
       --override-chat-template /models/tpl.jinja)
 MOUNTS=(-v "$MODELS:/models:ro")
 TAPENV=()
@@ -276,8 +334,23 @@ case "$MODE" in
     # buys throughput with the expert slab, so price it like any other
     # VRAM knob. The slots follow frank-archon.env's FRANK_ARCHON_MAX_SEQS
     # through MAX_SEQS, exactly as kldref and score do.
+    [ "$IMAGE_EXPLICIT" = 1 ] || IMAGE="$CALIB_IMAGE"
+    # ONE TAP SEAL WRITES ~40 GiB. Measured 2026-10-30: a 8,192-token floor wrote
+    # 39.4 GiB into the tap dir within seconds; pointed at a 37 GiB root it filled
+    # the filesystem to zero, after which every write came back short ("calibration
+    # cannot write", "write was short") and the system itself began refusing
+    # writes. The room is demanded up front, on the tap's own filesystem.
+    if [ "$DRY" != 1 ]; then
+        avail_mib=$(df --output=avail -B 1M "$CALIB_DIR" 2>/dev/null | tail -1 | tr -d ' ')
+        [[ "$avail_mib" =~ ^[0-9]+$ ]] || avail_mib=0
+        need_mib="${Q38_CALIB_MIN_FREE_MIB:-61440}"
+        if [ "$avail_mib" -lt "$need_mib" ]; then
+            echo "REFUSED: tap dir $CALIB_DIR has ${avail_mib} MiB free; one seal writes ~40 GiB and this run needs ${need_mib} MiB (Q38_CALIB_MIN_FREE_MIB overrides)" >&2
+            exit 2
+        fi
+    fi
     ARGS+=(--model "$MODEL_IN" --port 7770
-           --max-num-seqs "$MAX_SEQS" --max-num-batched-tokens "$MNBT")
+           --max-num-seqs "$CALIB_SEQS" --max-num-batched-tokens "$CALIB_MNBT")
     # The tap is only real if this directory exists and the engine can write it:
     # without the check a typo'd CALIB_DIR auto-creates a root-owned empty dir in
     # whatever path was given, and the run burns hours serving prompts into a tap
@@ -289,6 +362,17 @@ case "$MODE" in
     MOUNTS+=(-v "$CALIB_DIR:/data/calib")
     TAPENV=(-e RADIANCE_CALIB_DIR=/data/calib
             -e "RADIANCE_CALIB_TOKENS=${FRANK_QWEN38_CALIB_TOKENS:-12000000}")
+    # The tap's own authorization. A pusher must see this marker before it sends
+    # a prompt (calib-push.py --tap-marker), it names the container so a push
+    # aimed anywhere else refuses, and a boot that fails removes it again below.
+    # Measured 2026-10-29: with no marker, the chain pushed 4 rounds x 10M tokens
+    # into RESTORED PRODUCTION after the rig's boot died, and nothing recorded.
+    if [ "$DRY" = 1 ]; then
+        :
+    else
+        printf 'container=%s\nimage=%s\nmode=calib\nstarted=%s\npid=%s\n' \
+            "$C" "$IMAGE" "$(date -Is)" "$$" > "$CALIB_DIR/.rig-boot"
+    fi
     ;;
 esac
 
@@ -368,6 +452,19 @@ cp -f "$TEMPLATE" "$TPL" || {
 # where the service reported `active` while its engine starved on the rig's VRAM.
 restore() {
     podman rm -f "$C" >/dev/null 2>&1 || true
+    # Removing the container does NOT kill its engine. Measured 2026-10-30: a rig
+    # radiance kept /dev/kfd after `podman rm -f`, production's
+    # ar_oneshot_2rank_wht6 init then failed `device error`, the engine exited 0,
+    # and Restart=always looped it 100 times with /health never answering. The
+    # service is stopped for the whole run, so any radiance still holding the
+    # device here is an orphan — kill it BY PID (pkill -f matches the caller's
+    # own command line, which is how a session gets killed instead of the orphan).
+    for p in $(fuser /dev/kfd 2>/dev/null); do
+        if [ "$(cat "/proc/$p/comm" 2>/dev/null)" = radiance ]; then
+            echo "restore: killing orphaned radiance pid $p still holding /dev/kfd" >&2
+            kill -9 "$p" 2>/dev/null || true
+        fi
+    done
     sudo systemctl start "$SERVICE" >/dev/null 2>&1 || true
 }
 trap restore EXIT
@@ -377,5 +474,12 @@ podman rm -f "$C" >/dev/null 2>&1 || true
 
 "${CMD[@]}" 2>&1 | tee "$LOG"
 rc=${PIPESTATUS[0]}
+# A boot that failed must not leave the tap marked as rig-authorized: the marker
+# is what calib-push.py trusts, and the point of it is that a dead rig can never
+# be pushed into again.
+if [ "$MODE" = calib ] && [ "$rc" -ne 0 ] && [ -f "$CALIB_DIR/.rig-boot" ]; then
+    mv -f "$CALIB_DIR/.rig-boot" "$CALIB_DIR/.rig-boot.failed-$(date +%Y%m%d-%H%M%S)" 2>/dev/null || true
+    echo "the rig boot failed (rc=$rc): the tap marker is retired, so no pusher can take this tap for a live rig"
+fi
 echo "run-qwen38-bf16 $MODE exited rc=$rc; $SERVICE restored; log: $LOG"
 exit "$rc"

scripts/test-changed.sh

@@ -94,6 +94,16 @@ for path in "${CHANGED[@]}"; do
         dankserver/.env.example|dankclaw/.env.example)
             add_test tests/env_consistency.bats
             ;;
+        # The SSD-backed metadata view: the installer, the host mount unit, and the
+        # two media units that bind the view rather than the raw array.
+        scripts/media-view.sh|dankserver/host/*)
+            add_test tests/media_overlay.bats
+            add_test tests/dankserver_quadlet.bats
+            add_test tests/deployment_scoping.bats
+            ;;
+        dankserver/jellyfin/quadlet/*|dankserver/plex/quadlet/*)
+            add_test tests/media_overlay.bats
+            ;;
         shared/homepage/*)
             add_test tests/homepage_caddy.bats
             add_test tests/homepage_config.bats

serverAppConfigs-docs/Architecture/Quadlet Lifecycle.md

@@ -195,8 +195,9 @@ deploy.sh, never a live env poke.
    check every namespace (`sudo podman ps -a`) and native units
    (`systemctl list-units`) before declaring a service absent. Compare
    compose mount sources against the real filesystem — compose files name
-   phantom paths (e.g. `/mnt/media/*` vs the real LUKS array at
-   `/mnt/entropy`). Fix compose through git first, deploy second.
+   phantom paths (the legacy `dankserver/compose.yaml` still names `/opt/jellyfin/*`
+   for a service whose roots moved to `/home/dank/Containers/jellyfin`). Fix
+   compose through git first, deploy second.
 2. **Pre-create bind-mount sources** (`sudo mkdir -p` + `chown 1000:1000`)
    or creation fails `statfs …: no such file or directory`.
 3. **Staged bring-up:** infra first (`up -d mariadb redis`), poll

serverAppConfigs-docs/Dankserver/Jellyfin.md

@@ -1,9 +1,9 @@
 ---
 type: reference
 title: Jellyfin
-description: "Jellyfin on Dankserver: quadlet unit, entropy-volume data layout, ports, client access, hardware-transcode status."
+description: "Jellyfin on Dankserver: quadlet unit, overlay metadata view, provider keys, ports, access, transcode status."
 tags: [jellyfin, dankserver, media-server]
-timestamp: 2026-09-03T00:00:00Z
+timestamp: 2026-10-30T00:00:00Z
 ---
 
 # Jellyfin
@@ -26,7 +26,7 @@ public vhost `jelly.franksdank.cloud` behind the [[Anubis]] guard chain →
 | Image | `ghcr.io/jellyfin/jellyfin:latest` |
 | Network | `dankserver_appnet`; publishes 8096/tcp, 8920/tcp, 7359/udp, 1900/udp |
 | Data roots | `/home/dank/Containers/jellyfin/config:/config`, `/home/dank/Containers/jellyfin/cache:/cache` |
-| Media | `Volume=/mnt/entropy:/data:ro` — the whole [[Entropy Volume]] mounted read-only; nothing automated writes to it |
+| Media | `Volume=/var/lib/media-meta/view:/data` — the [[Media Metadata View]] overlay, **writable**; sidecars land on the NVMe, the [[Entropy Volume]] array underneath stays read-only and untouched |
 | GPU | none — no `AddDevice`; CPU decode only (jellyfin-ffmpeg path) |
 | Env quirks | `DOTNET_SYSTEM_IO_DISABLEFILELOCKING=true`, `LD_PRELOAD=libjemalloc.so.2`, `MALLOC_TRIM_THRESHOLD_=131072` kept verbatim from the compose-era container (zero-drift cutover) |
 | Health | `curl -f http://localhost:8096/health` every 30 s |
@@ -36,11 +36,23 @@ public vhost `jelly.franksdank.cloud` behind the [[Anubis]] guard chain →
 ```
 /home/dank/Containers/jellyfin/config   -> /config   (JELLYFIN_DATA_DIR)
 /home/dank/Containers/jellyfin/cache    -> /cache    (JELLYFIN_CACHE_DIR, XDG_CACHE_HOME)
-/mnt/entropy                            -> /data     (read-only media volume)
+/var/lib/media-meta/view                -> /data     (the SSD-backed overlay view)
 ```
 
-Library folders point at `/data` paths; because the mount uses the same
-absolute path as on the host, library paths survive container recreation.
+Library folders point at `/data` paths, which are the view's paths: stable
+across container recreation, and identical to the raw array layout below it.
+
+`Local metadata` — save metadata to media folders — is enabled, and it works
+because `/data` is the [[Media Metadata View]] rather than the read-only array:
+`movie.nfo`, `tvshow.nfo`, `episode.nfo` and artwork are written once, on the
+NVMe upper layer, and [[Plex]] reads the same files. Jellyfin is the fleet's
+only metadata writer.
+
+**Failure signature:** `System.IO.IOException: Read-only file system :
+'/data/Movies/<film>/movie.nfo'` means the container is binding the raw array
+instead of the view — the view is unmounted or the quadlet drifted. That used to
+be the expected state (pre-view, every item update logged it); today it is a
+fault. Run `scripts/media-view.sh status`.
 
 ## Ports & access
 
@@ -105,6 +117,61 @@ where the LDAP plugin runs. Verified 2026-09-25: `alex` authenticates over
 `AuthenticateByName` (HTTP 200; full plugin flow logged — service bind,
 search, `ldapUid`, user bind, token).
 
+## Metadata providers and their keys (2026-10-30)
+
+TMDb, OMDb, StudioImages and MusicBrainz are **core plugins** in Jellyfin 12 —
+no DLL, just a configuration file the running container owns under the data
+root. TVDB is not core (its metadata provider was removed in 10.9), so it is an
+official-catalog plugin.
+
+| Provider | Shape on disk | Key lives in |
+|---|---|---|
+| TMDb | `config/plugins/configurations/Jellyfin.Plugin.Tmdb.xml` → `<TmdbApiKey>` | `/home/dank/Containers/jellyfin/provider-keys` → `TMDB_API_KEY=` (mode 600) |
+| TVDB | `config/plugins/thetvdb_24.0.0.0/` + `…Jellyfin.Plugin.Tvdb.xml` → `<ApiKey>` | the same file → `TVDB_API_KEY=` |
+| OMDb | `…Jellyfin.Plugin.Omdb.xml` | **not configured** — its key is issued by email, 1 000 req/day, and its ToS forbids storing the returned data; ratings stay Jellyfin's own lookup |
+
+**None of it is tracked in git.** Those XML files belong to the container and
+hold credentials, and `tests/jellyfin_providers.bats` fails the suite if one is
+ever committed. The keys themselves sit in the per-service host env file that is
+the fleet's convention for container credentials (grafana, caddy, pihole,
+copyparty, romm, seerr): `provider-keys`, mode 600, beside the data root.
+`scripts/jellyfin-providers.sh` is the only thing that writes the XML:
+
+```
+scripts/jellyfin-providers.sh plan      # what would change; prints no key
+scripts/jellyfin-providers.sh install   # plugin + keys, previous copy to trash
+scripts/jellyfin-providers.sh verify    # plugin + key LENGTHS, never key bytes
+```
+
+- **A key never enters git, argv, or the tool's output.** The installer reads
+  that 600-perm file itself (a value in argv is world-readable through
+  `/proc/<pid>/cmdline`) and refuses it when it is group- or world-readable —
+  an exposed key is an error, not a warning. A key that shows up in a transcript
+  or `ps` output has leaked: rotate it with the provider, rewrite
+  `provider-keys`, re-run `install`.
+- **The TVDB artifact is pinned three ways**: sha256
+  `6302fba3857bdde8d186dd099fac09ca43247041f531a528a2c788add70a5319`, the
+  official plugin guid `a677c0da-fac5-4cde-941a-7134223f14c8`, and version
+  `24.0.0.0` (`targetAbi 12.0.0.0`, which fits the deployed 12.4.0). A sha
+  match alone only proves the bytes are the bytes fetched — the guid check is
+  what says it is the official plugin. Rollback: move `thetvdb_24.0.0.0/` into
+  the trash and revert the pin; a second `thetvdb_*` directory is a refusal,
+  never a quiet second copy.
+- **The homelab TVDB key is a licensed key** (its JWT claims say
+  `community_supported: false`), so it needs **no PIN** — the optional
+  `TVDB_PIN=` line exists only for a subscriber-shaped key.
+- **TVDB is installed and keyed but is not a metadata agent**: TMDb still
+  identifies everything. Enabling TVDB as a provider would re-fetch show
+  metadata across the library, so it is a deliberate, separate change.
+- **The deployer places the keys before it restarts jellyfin**
+  (`ensure_provider_keys` in `scripts/deploy-remote.sh`) and warns instead of
+  failing when `provider-keys` is absent, so a host without it still deploys.
+- **Keys and plugins are read at start-up**: after a manual `install` the
+  change is inert until `./scripts/deploy.sh dankserver --service jellyfin`.
+- TMDb's terms are the reason this stays narrow: attribution required, no
+  caching beyond 6 months, no derivative works. Fine for a personal homelab,
+  and the reason no bulk enrichment pass runs against it.
+
 ## Configuration workflow
 
 1. Create libraries pointing at `/data` folders (movies, shows, audio).

serverAppConfigs-docs/Dankserver/Media Metadata View.md

@@ -0,0 +1,117 @@
+---
+type: reference
+title: Media Metadata View
+description: "The SSD-backed overlay the two media servers read and write: shape, guards, commands, laws, rollback."
+tags: [dankserver, media-server, jellyfin, plex, storage]
+timestamp: 2026-10-30T00:00:00Z
+---
+
+# Media Metadata View
+
+`/var/lib/media-meta/view` is an OverlayFS merge of the media library: the RAID6
+array below, a directory on the NVMe above. It exists so [[Jellyfin]] can write
+Kodi-format sidecars (`movie.nfo`, `tvshow.nfo`, `episode.nfo`, artwork) that
+[[Plex]] can read. Sidecar metadata is only useful sitting next to the media
+file, and the array is read-only and slow — so the media bytes stay there and the
+*writes* land on SSD. One tree to both servers, one copy of the media.
+
+The mount point is not a volume and holds nothing while unmounted. It sits inside
+`/var/lib/media-meta` so that layers, kernel bookkeeping, the reap trash and the
+merged door are one directory — and so the mount table grows no new `/mnt` entry
+next to the array.
+
+## Shape
+
+| Item | Value |
+|---|---|
+| Lower layer | `/mnt/entropy` — the RAID6 array, source of truth, never written |
+| Upper layer | `/var/lib/media-meta/upper` — NVMe; every write lands here |
+| Work dir | `/var/lib/media-meta/work` — overlayfs internals, same filesystem |
+| Mount point | `/var/lib/media-meta/view` — the merged tree both servers bind |
+| Unit (repo) | `dankserver/host/systemd/media-meta-view.mount` — the source of truth |
+| Unit (host) | `var-lib-media\x2dmeta-view.mount` — systemd names a mount unit after its escaped path (`-` becomes `\x2d`); `media-view.sh install` writes the repo file under that name |
+| Installer | `scripts/media-view.sh` — `plan` / `install` / `verify` / `status` / `reap` |
+| Written by | Jellyfin only (Plex binds read-only); nothing else in the fleet touches it |
+
+Both consumers guard the view with
+`UnitConfigItem=ConditionPathIsMountPoint=/var/lib/media-meta/view`, and
+`scripts/deploy.sh dankserver --service jellyfin|plex` calls `media-view.sh
+install` + `verify` before it restarts either container.
+
+| Consumer | Bind | Mode | Why that path |
+|---|---|---|---|
+| [[Jellyfin]] | `/var/lib/media-meta/view:/data` | rw | It is the metadata writer; its sidecars become SSD bytes |
+| [[Plex]] | `/var/lib/media-meta/view:/mnt/entropy` | ro | Every path in the PMS database was recorded as `/mnt/entropy/...`; only the HOST side of the bind moved, so its database still resolves |
+
+Putting the sidecars inside Plex's reach is not the same as making Plex use them:
+a library parses `.nfo` only with its *Local media information* agent enabled, and
+this change touches no library setting.
+
+## Commands
+
+```bash
+scripts/media-view.sh plan              # print the resolved shape, touch nothing
+sudo scripts/media-view.sh install      # create dirs, install the unit, mount, enable
+scripts/media-view.sh verify            # is the view really the overlay? (deploy gate)
+scripts/media-view.sh status            # mount options + upper-layer size/file count
+scripts/media-view.sh reap              # list ghost metadata (media moved, sidecars left)
+sudo scripts/media-view.sh reap --apply # move ghosts to the SSD trash, never delete
+```
+
+`--upper-base DIR` and `--lower DIR` relocate the layers for `plan` and `reap`;
+the defaults are the host values above.
+
+## Laws
+
+- **The upper layer must sit on non-rotational storage.** `install` and `verify`
+  refuse via `lsblk -dno ROTA` if the upper base lives on the array: an overlay
+  whose upper is the spinners is the original problem with extra steps, and it
+  silently turns every metadata write back into a seek.
+- **Never mount the view over an unmounted lower.** `RequiresMountsFor=/mnt/entropy`
+  plus the container `ConditionPathIsMountPoint` guards: an empty `lowerdir`
+  presents an *empty library* as truth, and a media scan would then clear the
+  database of everything. systemd does not create a mount point, so `install`
+  creates `view` itself; without it the mount simply fails.
+- **Never delete through the view.** A delete writes an overlayfs whiteout
+  (`.wh.<name>`) in the upper layer, which hides the array's copy without
+  removing it — the file reappears the moment the view is unmounted, and the
+  upper layer now carries a landmine. Deletions belong on the raw path
+  (`/mnt/entropy`), which the relayout tool already treats as the only writer.
+  `reap` refuses to remove whiteouts and only warns about them.
+- **A relayout must be followed by `reap`.** Renames happen on the raw path, so
+  sidecars the view already wrote stay at the old coordinates: a ghost folder
+  the servers browse and find nothing in. `reap` lists upper-layer directories
+  with no counterpart below and moves them (with their subtree) into a dated
+  `.AGENTTRASH/` under the SSD base. It is recursive and file-safe: a sidecar
+  that exists only above is normal, so only *directories* are candidates.
+- **File-watcher events do not cross the overlay.** Plex's NFO agent and
+  Jellyfin's real-time monitor see inotify events for writes that land *in the
+  upper layer*, not for changes on the array underneath. After any relayout or
+  externally-written metadata, trigger a library refresh — see
+  [[Media Relayout]] for the refresh procedure.
+
+## Failure signatures
+
+| Symptom | Meaning |
+|---|---|
+| `jellyfin.service` / `plex.service` show `ConditionPathIsMountPoint=... was not met` and stay inactive | The view is not mounted. Do not start them anyway: `scripts/media-view.sh install`, then `verify`. |
+| Jellyfin log: `Read-only file system : '/data/Movies/…/movie.nfo'` | The container is binding the raw array, not the view — the quadlet drifted back or the view is gone. `media-view.sh status`. |
+| A folder appears in a media server with artwork/`.nfo` but no video | Ghost metadata from a relayout. `media-view.sh reap`. |
+| `verify` prints `refused: …/view is not a mountpoint` | A plain directory is sitting where the overlay belongs. That is the fail-closed guard working; `install` on the box. |
+| `systemctl status` finds no such unit | Look under the escaped name, `var-lib-media\x2dmeta-view.mount`, not the repo file name. |
+
+## Rollback
+
+Revert the commit that pointed the two quadlets at the view and re-deploy both
+services — they bind `/mnt/entropy` read-only again, which is the pre-view shape.
+The NVMe upper layer can stay: nothing reads it once the binds are back, and it
+holds no state either server needs. Unmounting alone
+(`systemctl stop 'var-lib-media\x2dmeta-view.mount'`) is safe while no container
+binds it, and the array is never modified by any of this.
+
+## Links
+
+- [[Jellyfin]] — the metadata writer; its data layout section.
+- [[Plex]] — the read-only consumer; in-place bind-mount history.
+- [[Media Relayout]] — the tool that moves media on the raw path and the reason `reap` exists.
+- [[Entropy Volume]] — the array underneath.

serverAppConfigs-docs/Dankserver/Media Relayout.md

@@ -0,0 +1,342 @@
+---
+type: reference
+title: Media Relayout
+description: "Census, plan and apply the movie and TV libraries into Jellyfin format: gates, evidence chain, refusals, rollback."
+tags: [dankserver, media-server, jellyfin, plex, procedure]
+timestamp: 2026-10-30T00:00:00Z
+---
+
+# Media Relayout
+
+The movie library on [[Entropy Volume]] arrived as release folders —
+`Some.Movie.2019.1080p.WiKiTaLx.x265-FOO/` with the video inside named after
+the release. [[Jellyfin]] matches a movie by folder and file name, so the
+release noise cost posters, extras and versions. These scripts turn that
+library into Jellyfin's own shape — the movies, the TV packs, and the TV that
+turns up in the movie share — and nothing else in the repo touches a media file.
+
+| Script | Touches media? | Output |
+|---|---|---|
+| `scripts/media-census.py` | reads headers only | `enum`, `probed`, `plex` TSVs |
+| `scripts/media-plan.py` | no — TSVs in, TSVs out | the plan + the exception report |
+| `scripts/media-apply.py` | **renames, never copies**; sweeps only what a run emptied | the moves, plus a journal |
+| `scripts/media-tv.py` | no — TSV out | a plan that moves TV into the TV library |
+| `scripts/media-shows.py` | no — TSV out | a plan that re-files the TV library itself |
+
+The seams are `tests/media_relayout.bats` and `tests/media_shows.bats`; the target contract is quoted from
+[jellyfin.org/docs/general/server/media/movies](https://jellyfin.org/docs/general/server/media/movies)
+in `media-plan.py`'s docstring.
+
+## Procedure
+
+Run the census on the machine that mounts the library. `probe` is the slow step
+(a header read per container) and is `nice`d — it is safe beside a live service
+because it never competes for the GPU and never writes inside the library.
+
+```bash
+C=scripts/media-census.py
+$C enumerate --root /mnt/entropy/Movies --out /tmp/media-enum.tsv
+$C probe     --enum /tmp/media-enum.tsv --out /tmp/media-probed.tsv --jobs 6
+$C plex      --root /mnt/entropy/Movies --out /tmp/plex-movies.tsv
+```
+
+`plex` reads the Plex metadata cache (`--db` defaults to the path the
+[[Plex]] container bind-mounts at `/var/lib/plexmediaserver`) with
+`mode=ro&immutable=1`: a live server is never locked or written.
+
+Then plan — this reads only the three TSVs, so it costs seconds and can run
+anywhere, including a laptop:
+
+```bash
+scripts/media-plan.py --root /mnt/entropy/Movies \
+    --enum /tmp/media-enum.tsv --probed /tmp/media-probed.tsv \
+    --plex /tmp/plex-movies.tsv --out /tmp/jf-plan.tsv --exceptions /tmp/jf-exceptions.txt
+```
+
+**Read `/tmp/jf-exceptions.txt` before applying.** It is not a log: every line
+is a file the plan could not name a film for, a pile that needs a README, or a
+collision it refused. The printed gate lines are the go/no-go:
+`problems`, `bad-names`, `occupied`, `double-claim`, `left-unmoved` must all be
+0 and `unplanned=0` must appear in the report. Any non-zero number means a file
+that would be filed wrong, and the plan is not run until it is understood.
+
+```bash
+scripts/media-apply.py apply    --plan /tmp/jf-plan.tsv --root /mnt/entropy/Movies   # dry, writes nothing
+scripts/media-apply.py apply    --plan /tmp/jf-plan.tsv --root /mnt/entropy/Movies --apply
+scripts/media-apply.py verify   --plan /tmp/jf-plan.tsv --journal /mnt/entropy/media-relayout.jsonl
+scripts/media-apply.py sweep    --journal /mnt/entropy/media-relayout.jsonl --root /mnt/entropy/Movies   # dry
+scripts/media-apply.py sweep    --journal /mnt/entropy/media-relayout.jsonl --root /mnt/entropy/Movies --apply
+scripts/media-apply.py rollback --journal /mnt/entropy/media-relayout.jsonl --root /mnt/entropy/Movies --apply
+```
+
+An apply also leaves the release folders it emptied standing where they were —
+hundreds of them at the library root — which is what `sweep` removes, dry-run
+first, from the same journal.
+
+After it lands, both servers need a scan or the change is invisible.
+
+```bash
+# Run this from the host shell through `podman exec`, never from a neighbour
+# container: Plex's HTTPD closes without a single byte on any request that
+# arrives from another container's netns (10.89.x.x reads as "empty reply", not
+# as a 401). The token rides stdin, never argv, where any process on the host
+# could read it out of /proc/<pid>/cmdline.
+umask 077
+podman exec plex sh -c "sed -n 's/.*PlexOnlineToken=\"\\([^\"]*\\)\".*/\\1/p' \
+  '/config/Library/Application Support/Plex Media Server/Preferences.xml'" >/tmp/px.tok
+podman exec -i plex sh -c 'read -r T
+  for k in $(curl -s -H "X-Plex-Token: $T" -H "Accept: application/json" \
+      http://127.0.0.1:32400/library/sections | grep -o "\"key\":\"[0-9]*\"" | cut -d\" -f4); do
+    curl -s -o /dev/null -w "plex section $k: %{http_code}\n" -X POST \
+      -H "X-Plex-Token: $T" "http://127.0.0.1:32400/library/sections/$k/refresh"
+  done' </tmp/px.tok
+```
+
+[[Jellyfin]]'s scan key lives in `~/.config/secrets/jellyfin` (mode 600; the
+same value is `JELLYFIN_API_KEY` in the ignored `dankserver/.env`). It rides a
+mode-600 curl config, never a header argument, and it must use the
+`MediaBrowser` form — Jellyfin 12.4 refuses a bare `X-Emby-Token` for any key:
+
+```bash
+umask 077
+{ printf 'header = "Authorization: MediaBrowser Token=\\"%s\\""\n' \
+    "$(cat ~/.config/secrets/jellyfin)"
+  echo 'url = "http://127.0.0.1:8096/Library/Refresh"'
+  echo 'request = "POST"'; } >/tmp/jf.hdr
+curl -s -K /tmp/jf.hdr -o /dev/null -w "jellyfin scan: %{http_code}\n"
+shred -u /tmp/jf.hdr
+```
+
+`204` is acceptance; the scan then runs on the server's own schedule.
+
+## TV in the movie library
+
+A box set that arrives in the movie share stays there until someone moves it:
+`media-plan.py` refuses to file TV and parks each series in
+`_TV needs a TV library - <series>`. `scripts/media-tv.py` is the mover. It
+walks the movie library itself (no census needed), groups episode-shaped names
+by series, and writes a plan in the executor's own format:
+
+```bash
+scripts/media-tv.py --root /mnt/entropy/Movies --library /mnt/entropy/TVShows \
+    --out /mnt/entropy/media-tv-plan.tsv
+scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv --root /mnt/entropy
+scripts/media-apply.py apply --plan /mnt/entropy/media-tv-plan.tsv --root /mnt/entropy \
+    --journal /mnt/entropy/media-relayout-tv.jsonl --apply
+scripts/media-apply.py sweep --journal /mnt/entropy/media-relayout-tv.jsonl \
+    --root /mnt/entropy --apply
+```
+
+The shape it writes is the one both matchers read — `Monk (2002)/Season 05/
+Monk - S05E03 - Mr. Monk and the Big Game.mkv` — keeping the file's own name,
+because the publisher's spelling beats a re-derivation, with the year taken
+from the box-set folder. What it refuses it prints and leaves alone:
+
+- fewer than `--min` (5) files under one series name: one file named `Part 2`
+  is a film, not a series;
+- an episode whose name states no season number — no invented `Season 01`;
+- a destination two files both claim: the executor refuses the whole plan
+  before its first rename, and only Frank decides which copy is the keeper.
+- an episode the destination library already holds at the same byte count. The
+  movie share carried a second copy of the whole Monk box set — 124 episodes
+  both sides, the same episode list, every byte count identical, 31.5 GiB each
+  way, and head+tail blake2b identical on a 6-episode sample. The mover leaves
+  such a series in place and says so; a duplicate is deleted or kept by Frank,
+  never copied into one library by a tool.
+
+A sidecar that shares an episode's own name (`.srt`, `.nfo`) moves with it.
+
+Below the threshold the mover prints and leaves, so a lone downloaded episode of
+a real show stays in the movie library on purpose — six did on 2026-10-30.
+Filing one by hand is the same executor on a hand-written plan and the same
+journal; the trap is the series year. Read it from the file's own name or the
+folder it owns, never from the title whose bonus folder it happens to be parked
+in — `The Cleveland Show S03E07` inside `Die Hard (1988)/extras/` is not a 1988
+show. A name whose marker is `Part 2` is a film and stays put.
+
+## The TV library's own relayout
+
+`/mnt/entropy/TVShows` is scene packs rather than a library, and Jellyfin has
+already matched and numbered most of it. `scripts/media-shows.py` re-files that
+library into the same shape, and it never guesses: every series name and every
+episode number it writes traces to an authority, and what has none is parked for
+a human to read.
+
+```bash
+# 1. The authority: Jellyfin's own matched rows (the DB opens read-only).
+python3 scripts/media-census.py jellyfin --root /mnt/entropy/TVShows \
+    --out /tmp/tv_titles.tsv
+# 2. What is on disk (stat only), and the runtimes the film arm needs.
+python3 scripts/media-census.py enumerate --root /mnt/entropy/TVShows \
+    --out /tmp/tv_enum.tsv
+python3 scripts/media-census.py probe --root /mnt/entropy/TVShows \
+    --enum /tmp/tv_enum.tsv --jobs 6 --out /tmp/tv_probed.tsv
+#    The probe is the slow step: ffprobe reads both ends of every file.
+# 3. The plan. --tmdb needs TMDB_API_KEY; without it a series no provider dated
+#    is simply named without a year, which Jellyfin accepts.
+python3 scripts/media-shows.py --root /mnt/entropy/TVShows --enum /tmp/tv_enum.tsv \
+    --titles /tmp/tv_titles.tsv --probed /tmp/tv_probed.tsv \
+    --out /tmp/tv_plan.tsv --exceptions /tmp/tv_exceptions.txt
+# 4. Read the exceptions file FIRST — it is the review, not a log. Then the same
+#    executor, journal and rollback as the movie relayout above.
+scripts/media-apply.py apply --plan /tmp/tv_plan.tsv --root /mnt/entropy \
+    --journal /mnt/entropy/media-relayout-tv.jsonl --apply
+```
+
+Evidence, strongest first — the order the planner believes things:
+
+- the **episode marker the file's own name states** (`S01E02-E03`, `1x04`);
+- the **`Season NN` folder it sits in and the number its name leads with inside
+  it**: `Season 19/01 Charles Barkley, Nirvana.avi` is S19E01;
+- the **provider row**: identity (the series name and its year), and the
+  numbering of a file whose own name says nothing;
+- a **bonus folder** anywhere above the file, or Jellyfin's `ExtraType` → the
+  series' canonical extra folder;
+- a runtime over `SHORT_MAX` makes it film-shaped, and it leaves for `--movies`
+  only when a movie provider names it — otherwise it parks.
+
+Two refusals carry the design, and a dry run on the real library found both:
+
+- **Folders resolve a series; they never invent one.** A folder's words may
+  resolve a file to a series a provider has already named — `(CBB) Tengen Toppa
+  Gurren Lagann - The Complete Series (BD 1080p)` names the registered *Tengen
+  Toppa Gurren Lagann* — but a bonus subfolder is not a show. An early run made
+  `…/03. Extra/01. NCOP/NCOP 01.mkv` into a root series called `01 NCOP`.
+- **A scraped row does not outrank the filesystem.** Jellyfin numbers
+  multi-season packs badly: every file under `SNL 16-20/Season 16..19` carries
+  season 16 episode 20, and one Gundam row says season 2004. The folder, then
+  the file's own leading number, win; a row that still insists on season 1989
+  parks its file instead of creating `Season 1989`.
+
+The provider's name is copied verbatim, warts included: Jellyfin carries a Series
+named `Black Clover 001` (92 episodes, no year) because it matched a pack off a
+filename. Renaming a show is Jellyfin's work — a folder rename never touches its
+provider ids — so the mover keeps the name it is handed and strips only the pack
+index a release leads with (`4a. Seed DESTINY` → `Seed DESTINY`).
+
+## Laws
+
+- **A moved file is a 404 in [[Plex]] until its section is rescanned.** The
+  database keeps the release-folder path, so playback fails `Couldn't find the
+  file to stream` while the scanner works through the section. Never read such a
+  404 as a lost file: the file is where the journal says it is.
+- **Plex's matched title outranks the filename**, and a Plex year of `0` means
+  *indexed but never matched* — trusting it named folders `Title (0)`, which is
+  a Jellyfin item with no year, no poster and no identity. A year that is not a
+  plausible year is no information.
+- **`<>:"/\|?*` never survive a name.** Plex's own title for *V/H/S/2* is
+  `V/H/S/2`; kept verbatim it became `Movies/V/H/S/2 (2013)/` — a tree five
+  levels deep that neither matcher reads as a movie.
+- **A folder inside a movie's folder is bonus territory even when its name is
+  misspelled.** `Saboteur (1942)/Exrtas/Saboteur - A Closer Look` is a typo for
+  `Extras`; the vocabulary rule never fired, the featurette became a movie
+  folder, and its words then out-scored the real film and drew that box set's
+  other extras into it. The rule that replaced it needs no metadata: the folder
+  holding the longest film owns the shorter video filed under it, unless that
+  file's own name carries a different year — which is another film speaking.
+- **Matching words are split on everything that is not a letter or a digit.**
+  Splitting on separators but not on `/` left tokens like `(1960)/extras` that
+  no other name could ever agree with, so a film could not be found by the
+  folder holding it, and extras drifted into a neighbour. A bare number counts
+  only when it could be a release year, so `1080` and `2160` never pose as
+  agreement between two names.
+- **Compare two media libraries with the series engine, never a path
+  substring.** Filtering paths for `monk` also returns *Monkey Dust*, and
+  letting whichever file a walk happens to reach first stand for an episode
+  paired two libraries into a fake duplicate — the same sloppiness pointed the
+  other way deletes a file that is unique. The comparison is `media-titles.ep_of`
+  on both sides: series, season, episode, bytes, and a hash before anything is
+  trashed.
+- **A deletion is staged on the item's own filesystem.** The trash rule names
+  `~/.AGENTTRASH`, but `/home` and `/mnt/entropy` are different devices: a 31 GiB
+  stage there is a copy rather than a rename, it has to fit in the 187 GiB of
+  free space, and `rename(2)` refuses it anyway. Trash goes beside the item —
+  `/mnt/entropy/.AGENTTRASH/<date>-<topic>/` — so it costs one inode change and
+  rolls back exactly like the move that made it.
+- **A Jellyfin 401 is the header, not the key.** 12.4 authenticates only
+  `Authorization: MediaBrowser Token="…"` and refuses a bare `X-Emby-Token` for
+  every key, valid or not. This page carried "no credential on this box can
+  trigger a scan, the two Seerr keys answer 401" for weeks because of that
+  header — the keys were innocent. Before believing a 401, prove the key against
+  the server's own table: `select count(*) from ApiKeys where AccessToken = ?`
+  on `jellyfin.db` opened `mode=ro&immutable=1`.
+- **A scan that must see a new folder has to be recursive.** `POST
+  /Library/Refresh` is the incremental pass, and inotify does not cross a
+  rootless podman bind, so folders created on the host stay invisible to it. The
+  move is `POST /Items/{id}/Refresh?Recursive=true` (204) per library, the id
+  from `GET /Library/VirtualFolders` — its field is `ItemId`, not `Id` — and
+  that same record's `RefreshStatus` is the progress read.
+- **A file whose name carries no words is bonus material, not a folder.**
+  `.Extra 1 ( 1H 27min multisub ).mkv` parses to `1`; a movie folder named `1`
+  is not a movie. It becomes a review line telling a human to name the file.
+- **The executor renames and nothing else.** `rename(2)` cannot cross a
+  filesystem, so a plan that would is refused rather than silently copying tens
+  of terabytes.
+- **The journal is a write-ahead log: the intent is fsynced before the rename.**
+  An interrupted run therefore leaves only two states — an intent whose move
+  happened, and an intent whose move never ran — and the next run finishes the
+  job without moving anything twice. A file that moved with no journal line did
+  not move through this tool, and the tool refuses to guess on its behalf.
+- **The journal lives BESIDE the library root, never inside it** (default:
+  `media-relayout.jsonl` next to the root): a log inside the library is a file
+  Jellyfin indexes.
+- **Refusals are all-or-nothing.** An occupied target, a vanished source or a
+  duplicate destination stops the run *before the first rename*, so the operator
+  fixes one thing and re-runs instead of untangling a half-applied plan.
+  `--force` relaxes only the opinion guards; it never permits overwriting.
+- **A sweep deletes only what the journal proves the run emptied.** A folder is
+  eligible when a journalled file moved out of it or out of something under it,
+  it now holds nothing but empty directories, no journalled destination lives
+  beneath it, and it is not a folder the run created. A folder nobody moved —
+  someone's placeholder — cannot be selected at all, and a symlink anywhere
+  inside disqualifies the subtree, because the thing behind a link is not this
+  tool's to delete. Every removal is journalled as `rmdir` and `rollback`
+  recreates the folder, so the sweep is undone by the same command as the moves.
+  Proof carries downward: inside a folder the journal proves emptied, every
+  empty directory goes with it. A release's `Screens/` or `Sample/` is never a
+  move source itself, so 146 of them held their emptied parents on disk after
+  the first sweep.
+  A failed `rmdir` is not a removal: only a folder that is really gone leaves
+  scope, so the next sweep finishes what a permission problem stopped.
+- **An episode-shaped name marks a SERIES, not a film — and the series is the
+  name BEFORE the marker.** `Monk - S05E03 - Mr. Monk and the Big Game` is series
+  `Monk`. Keyed on the whole title instead, every episode became a group of one
+  and never reached the five-episode size that calls something a series, so a
+  box set was filed as 124 movies in 124 folders. The threshold still earns its
+  keep: a lone film named `Part 2` stays a film. Episodes of a series are parked
+  in `_TV needs a TV library - <series>` for a human to move to a TV source —
+  the movie tool never files TV.
+- **One journal is one undo.** The TV move runs on its own journal, and with
+  `--root` set to the share that holds BOTH libraries: the executor refuses any
+  path outside `--root`, and a journal shared with the movie relayout would
+  undo both runs when only one of them is wanted.
+
+- **Sidecar metadata does not follow a rename, because it lives on another
+  filesystem.** The media servers write `.nfo` and artwork through the
+  [[Media Metadata View]] (NVMe upper layer); this tool moves bytes on the raw
+  array path. After any sweep or TV move, run `scripts/media-view.sh reap`, then
+  the same command with `--apply`: the sidecars left at the old coordinates become
+  ghost folders — artwork and an `.nfo` with no video — that both servers browse
+  and report as content.
+- **Nothing deletes through the metadata view.** A delete there writes an overlayfs
+  whiteout in the upper layer, which hides the array's copy without removing it;
+  the file returns when the view is unmounted. Deletions in this pipeline happen
+  on the raw path only, which is what the journal can undo.
+
+## Rollback
+
+One command, from the journal of the run:
+`media-apply.py rollback --journal …/media-relayout.jsonl --apply`. It recreates
+any folder a sweep removed, restores every source path, removes only the folders
+that run created *and that are now empty*, and refuses everything if any moved
+file cannot be accounted for. A rolled-back journal refuses to roll back twice.
+
+## See Also
+
+[[Jellyfin]], [[Plex]], [[Media Metadata View]], [[Entropy Volume]], [[Testing Policy]], [[Storage Policy]]
+
+## Citations
+
+- Jellyfin Movie documentation — folder/file naming, extras subfolders,
+  `Movie (2013) - 216p.mkv` version and `- 4K.mkv` handling:
+  <https://jellyfin.org/docs/general/server/media/movies>

serverAppConfigs-docs/Dankserver/Plex.md

@@ -4,7 +4,7 @@ description: "Plex Media Server is a **containerized media server on Dankserver*
 tags:
   - dankserver
   - plex
-timestamp: 2026-10-21
+timestamp: 2026-10-30
 ---
 
 # Plex
@@ -13,7 +13,8 @@ Plex Media Server is a **containerized media server on Dankserver**: a podman
 quadlet (`plex.service`) under the systemd **user** generator, generated from
 `dankserver/plex/quadlet/plex.container`. It shares the host with a second live
 media server, [[Jellyfin]] (`jellyfin.service`, listening on 8096); both serve
-from the same read-only Entropy Volume mount.
+from the same tree — the [[Media Metadata View]] overlay — so the Kodi-format
+sidecars Jellyfin writes are the files Plex reads, with no copy and no sync.
 
 **Host:** Dankserver — **Ports:** 32400/tcp (PMS web + API), 32400 is the
 only port PMS needs for client streaming; LAN clients connect directly,
@@ -26,7 +27,7 @@ public traffic can be fronted by Caddy like every other vhost.
 | Runtime | podman quadlet, `plex.service` (user manager) — migrated from the native `plexmediaserver.service` OS-package unit (2026-09) |
 | Image | `lscr.io/linuxserver/plex:1.43.5.11029-6bf8af3f0-ls329` (pinned; verified == the registry's `latest` manifest digest at pin time: `sha256:8f655bad…e9e2334`) |
 | Data root | `/var/lib/plexmediaserver` bind-mounted **in place** as `/config` (RW) — the migration moved zero bytes |
-| Media | `/mnt/entropy` mounted at the same absolute path **read-only** — every library path in the PMS database stayed valid, zero re-scan |
+| Media | `/var/lib/media-meta/view` (the [[Media Metadata View]] overlay) bound at the same absolute path `/mnt/entropy` **read-only** — every library path in the PMS database stayed valid, zero re-scan |
 | Identity | fleet rootless model — container root maps to the `dank` user (1000:1000); the deployer chowns the data root on first cutover |
 | Port | `32400/tcp` published on `0.0.0.0` — zero client reconfiguration |
 | Network | `dankserver_appnet`, alias `plex`; Tautulli reaches it at `http://plex:32400` |
@@ -38,7 +39,8 @@ public traffic can be fronted by Caddy like every other vhost.
 
 The migration moved zero bytes: the container bind-mounts the native data
 root where it already lived, and the media archive re-mounts at
-`/mnt/entropy` inside the container, so every absolute library path in the
+`/mnt/entropy` inside the container — only the HOST side of that bind has moved
+(array, then the overlay view), so every absolute library path in the
 PMS database keeps resolving. The one metadata change: rootless podman
 cannot map the real host uid 997, so the fleet model applies — container
 root runs as the `dank` user, and the deployer adopts the data root with

serverAppConfigs-docs/LLM/Radiance Architecture Plugins.md

@@ -0,0 +1,287 @@
+---
+type: "howto"
+title: "Radiance Architecture Plugins — declaring a model"
+description: "Writing a radiance architecture plugin: declaring weights, buffers, KV groups and ops, then issuing the step."
+tags:
+  - llm
+  - radiance
+  - plugins
+  - howto
+timestamp: 2026-10-30
+---
+
+# Radiance architecture plugins
+
+An architecture plugin declares a model in two phases: `rad_arch_declare` names every weight,
+buffer, KV group and op **once**, against the model's metadata; `rad_arch_step` issues the ops for
+one step. The engine picks the plugin by the exact `(rad_arch_id, rad_arch_quant)` pair — no
+fallback — so one family at two weight formats is two `.so` files, and branching on a quant
+descriptor inside one plugin is the anti-pattern the split exists to prevent.
+
+The in-tree plugins are the templates: `arch/llama_fp8/llama_fp8.cpp` (the smallest complete
+decoder), `arch/gemma3`, `arch/qwen35_bf16`, and the header-only blocks in `arch/common/`
+(`rad_block_attn*.h`, `rad_block_mlp*.h`, `rad_block_moe_fp8.h`, `rad_block_gdn*.h`,
+`rad_block_dflash2.h`, `rad_block_mtp*.h`, `rad_block_vit.h`, `rad_block_ple.h`,
+`rad_block_glm_*.h`, `rad_qsa.h`, …). Include those by `<arch/rad_arch.h>` — they are installed
+beside the ABI and the in-tree plugins include them by that spelling.
+
+## Exports
+
+```c
+const char* rad_arch_id(void);              /* "llama", "qwen35", ... */
+const char* rad_arch_quant(void);           /* "" unquantised, else the descriptor served */
+int         rad_arch_serves_imports(void);  /* optional: 0 declines an imported quantisation */
+int         rad_arch_probe(const RadModelMeta* meta, RadArchProbe* out);   /* optional */
+int         rad_arch_declare(RadBuilder* b, const RadModelMeta* meta, const RadBuildCtx* ctx);
+void        rad_arch_step(struct RadCtx* c, const struct RadBatch* batch);
+const RadChatFormat*  rad_arch_chat_format(const RadModelMeta* meta);      /* optional */
+const RadAudioFormat* rad_arch_audio_format(const RadModelMeta* meta);     /* optional */
+```
+
+Everything but declare/step is optional. `RAD_ARCH_PLUGIN(NS, ID, QUANT, VERSION, DESC)` in
+`arch/common/rad_arch.h` emits the whole export block, including `rad_plugin_info` (kind
+`RAD_PLUGIN_ARCH`). The plugin's **name must be unique**; the architecture id may repeat.
+
+`rad_arch_serves_imports` exists because a checkpoint whose quantisation was imported weight by
+weight (Quark, ParoQuant, GPTQ, NVFP4) names its producing tool, which no plugin claims: the family's
+one quantised plugin serves it, and two are refused by name. A second quantised plugin that reads
+only its own containers returns 0.
+
+## Declare: the metadata
+
+`RadModelMeta` is what the loader read from the container's header before any plugin ran: `arch_id`,
+`name`, `quant`, `n_layers`, `n_embd`, `n_head`, `n_head_kv`, **`head_dim`**, `n_ff`, `n_vocab`,
+`n_ctx_train`, expert counts, `rms_eps`, `rope_theta`, and a free-form `kv_key`/`kv_val` table for
+everything else (`rad_meta_geti/getf/gets`). Read dimensions from here — one plugin covers every size
+in a family.
+
+**`head_dim` is read, never derived.** On the Qwen hybrids it is 256 while `hidden / heads` is 213;
+a derived value silently breaks projection widths, KV page size and the attention kernel's band.
+
+`RadBuildCtx` is the deployment the plugin must declare against:
+
+| field | meaning |
+|---|---|
+| `rank`, `world_size` | declare runs once per rank; **dimensions are already divided** — the core inserts no collectives |
+| `max_tok`, `max_seqs`, `max_ctx`, `max_spec` | the step bounds: buffer sizing, kernel bands, the scheduler's window |
+| `max_mixed_tok` | the most a multi-sequence step may carry; a wider step is one sequence (prefill span) |
+| `tp_wire_lossy`, `tp_wire_min_bytes` | invert into `all_reduce`'s `exact`; below the floor a message must go exact |
+| `kv_dtype`, `kv_quant` | the attention cache width and, under fp8, its narrow layout (`RAD_KVQ_V4`, `RAD_KVQ_K6V4`) |
+| `attn_prefill_fp8` | `RAD_ATTN_FP8_*` bits to pass as `attn_paged`'s `prefill_fp8` |
+| `kv_resident_blocks`, `kv_stream_ok` | KV streaming: the ring depth, and permission to declare the streamed-only ops |
+| `state_dtype` | `RAD_F16` when `--state-cache-dtype f16` is served by this plugin |
+| `is_draft`, `scope` | a drafter/encoder declare; `scope` prefixes declared names (`draft`, `vision`, `audio`) |
+| `shape_probe` | a **sizing** declare at a smaller `max_tok`; must not touch state the real declare wrote |
+| `max_enc_patches`, `max_audio_frames` | the encoder pass sizes |
+| `max_out_rows` | logits rows the KL mode needs (0 = the plugin's own rule) |
+| `cvec` | the loaded control vector, or null |
+| `tp_attn_rows` | row-split prefill attention requested |
+| `draft_coupling`, `draft_tree` | declare the coupled/tree forms |
+| `weight_prefetch_off` | declare no weight prefetch beside collectives |
+
+## Declare: weights
+
+```c
+static const RadWeightDecl d = {
+    .dtype = RAD_BF16, .rank = 2, .shape = {n_embd, n_ff},
+    .access = RAD_ACCESS_PER_TOKEN, .shard = RAD_SHARD_ROW,
+    .group = grp_layer(l), .optional = 0,
+    .n_row_parts = 0, .source = nullptr, .planes = nullptr,
+};
+rad_weight w = rad_decl_weight(b, "blk.0.mlp.gate.weight", &d);
+```
+
+- **`access`** tells the placement planner how the weight is read: `PER_TOKEN` (dense),
+  `PER_REQUEST` (an encoder tower, read once per request), `CONDITIONAL` (routed experts),
+  `RARE`, `VOCAB` (lm_head/embedding: read every token but huge, and one is a gather).
+- **`shard`**: `NONE` (replicated), `ROW` (split dim 0), `COL` (split dim 1, needs a following
+  all-reduce). The plugin says; the core does not infer. Uneven splits use
+  `rad_weight_shard_span`/`_parts` (a width that does not divide into whole 128-column blocks; the
+  delta net's `[q|k|v|z]` of unequal part widths).
+- **`group`** is the unit of movement (`layer`, `expert`, `slot`); an expert's gate/up/down share one,
+  because the unit of transfer is an expert, not a tensor.
+- **`row_parts`** describes what a `SHARD_ROW` weight stacks along dim 0 (`[gate|up]`,
+  `[q|k|v|z]`). **A tensor-parallel rank owns a slice of each stacked projection**, never a
+  contiguous slice of the stack; the parts cannot be inferred from a fused container tensor, and a
+  contiguous slice produces fluent wrong text.
+- **`source`/`planes`** view an existing stored weight: a block-fp8 linear's `codes,scale` are one
+  logical weight taken by a GEMM as two operands. Null `source` is the weight of its own name; null
+  `planes` takes every plane.
+- A dtype other than the plane's is legal only as an exact widening (a bf16 norm read as f32).
+
+The **name map** (`rad_decl_name_map`) is the plugin's map from checkpoint tensor names to declared
+weight names, and it is the only place convert-time fusions live: `COPY`, `CONCAT`, `WNORM` (a
+weight-normalised pair folded to the plain weight), `ROWS`, `TRANSPOSE`, `EMA` (an EnCodec codebook),
+`TAP`, `COLS`. `rad-convert` reads this map from the plugin, so it needs no knowledge of the model.
+`RAD_MAP_COPY` with several sources means "first present wins" (a tied lm_head); `CONCAT` requires
+all. Ask for a weight's encoding with `rad_weight_encoding` **after** its map entry when the graph
+depends on the format (fp8 GEMM vs plain, rotated expert kernels).
+
+## Declare: buffers and KV groups
+
+`rad_decl_buffer` names every activation: `RAD_BUF_TRANSIENT` (liveness-analysed, may share bytes),
+`RAD_BUF_PERSIST` (across steps: a calibration accumulator, a routing histogram), or
+`RAD_BUF_DERIVED` (computed once per step by the core from the batch — `derive` names the
+derivation, e.g. `conv_state_index`). Buffers live in one arena sized once, not an allocator per
+step.
+
+`rad_decl_kv_group` declares each state kind: `RAD_KV_FULL` (paged per-token, growing),
+`RAD_KV_WINDOW` (bounded by a window), `RAD_KV_LINEAR` (per-sequence fixed state, GDN),
+`RAD_KV_CONV` (a rolling window of `width-1+num_spec`), `RAD_KV_RING` (window floor set at runtime).
+The **block size is not declared** — it comes from the resolved attention kernel's constraints; read
+it back with `rad_kv_block_size`. `rad_bind_layer_kv` binds each layer so the block manager knows how
+many state instances a sequence owns; `rad_kv_group_zero` (a rank with no heads),
+`rad_kv_group_replicated` (every rank holds the same bytes), and `rad_kv_group_dense_read` (read over
+the whole history every step, so it cannot stream) are the three refinements.
+
+## Declare: ops
+
+```c
+rad_op h = RAD_OP(b, "gemm_w4a8_nt",
+                  RAD_PARAMS(RAD_RANGE("M", 1, max_tok), RAD_INT("N", 8192), RAD_INT("K", 5120)),
+                  RAD_WEIGHTS(w));
+rad_op_reads (b, h, RAD_BUFS(b_x), n);
+rad_op_writes(b, h, RAD_BUFS(b_y), n);
+```
+
+- Parameter names and operand order come from `docs/OPS.md`; a new name is allowed only when the
+  vocabulary has none, and the first plugin to declare it fixes its schema.
+- `RAD_RANGE` on exactly one parameter per op gives the selector its bucket table. The core carries
+  every parameter opaquely except `RAD_PROLE_SEQ_CHUNK` (cut steps on it) and `RAD_PROLE_CAPACITY`.
+- `rad_op_reads`/`rad_op_writes` are the buffer planner's liveness. An undeclared use is assumed live
+  for the whole program: sound but wasteful.
+- **Fusion is selection.** Ask for the fused op; if `rad_op_resolved` is false, emit the chain. A
+  fusion's `replaces` hook is a checkable equivalence, not a rewrite.
+- `rad_op_declared(b, "op")` asks whether any library declares an optional op; use it before
+  emitting a fused form only an optional library defines.
+- Buffers touched from the **second lane** go through `rad_buf_concurrent`, and an architecture whose
+  lane-1 work lies in one stretch declares `rad_lane1_window_begin/end` around it. The runtime
+  refuses a lane-1 op outside the window; the window lets buffers used only outside it share bytes.
+
+Step output: `rad_declare_logits` names the buffer the sampler reads (its dim 0 bounds `n_out`; a
+[n_tok, vocab] logits buffer at a large vocabulary is gigabytes — declare the gather +
+`logits_gemm` shape instead). An embedding program declares `rad_declare_pooled` instead (one row a
+sequence, no sampler, no generation). `rad_declare_scores` opts into host-side scored steps.
+
+## Declare: drafters, encoders, generators
+
+- `rad_declare_drafter(RadDrafterDecl)` — depth, block, proposal shape. `rad_drafter_shared_kv` (a
+  drafter with no cache of its own), `rad_drafter_tree` (a tree over the proposals),
+  `rad_drafter_short_block` (a second, shorter pass) are refinements. Declare the drafter **last**.
+- `rad_declare_encoder(RadEncoderDecl)` — a vision/audio tower, media preprocessed by the core.
+- `rad_declare_generator` / `rad_declare_audio_generator` / `rad_declare_video_generator` — image,
+  audio and video generation programs; the engine sizes the step from the probe's `gen_*` fields
+  before declare.
+- Optional format exports let a model state what it knows and the server learn nothing new:
+  `rad_arch_chat_format` (a data description of reasoning/tool-call markup, replacing template
+  analysis), `rad_arch_audio_format` (how a speech model hears: clip tokens, prompt, language,
+  transcript start), `rad_arch_text_prepare` (rewrite request text into training form),
+  `rad_arch_video_positions`/`rad_arch_video_pack` (video pass geometry and packing),
+  `rad_arch_encode_geom` (encoder pass geometry). Each struct grows only at its end behind
+  `struct_size`, so adding a field does not move the ABI.
+
+**Media preprocessing is the core's**, not the plugin's: resizing, patching, placeholder expansion
+and positions in six processor families (Qwen-VL, Gemma 4, LFM2-VL, Unlimited-OCR, GLM-5.3, Pixtral
+for Nemotron), chosen by `vision_config.model_type` plus owning-model checks; audio through Gemma 4's
+extractor or Qwen3-ASR's Whisper one, chosen by `audio_config.model_type`. A vision model with another
+pipeline needs a core change.
+
+## Step: issuing the graph
+
+`RadBatch` is built once per step and shared by every block: `phase` (`PREFILL`/`DECODE`/`MIXED`),
+`n_tok`, `n_seq`, `token_ids`, `positions`, `cu_seqlens`, `seq_ids`, `lora_slot`, `n_out`/`out_ids`,
+`q_lens`/`ctx_lens`/`max_q_len`/`max_ctx_len`, the KV-group array, the speculative fields
+(`n_spec`, `draft_pass`, …).
+
+```c
+static void step(RadCtx* c, const RadBatch* batch) {
+    if (batch->draft_pass > 0) { m.ds.step(c, batch); return; }      /* query pass */
+    if (batch->draft_pass < 0) { m.ds.context(c, batch); return; }   /* context pass */
+    RAD_ISSUE(c, m.op_embed, praw(batch->token_ids, RAD_I32, batch->n_tok),
+              RAD_W(m.w_tok), brows(m.b_x, batch->n_tok));
+    for (auto& layer : m.layers) { layer.attn.step(c, batch); layer.mlp.step(c, batch); }
+    /* ...heads; the lm_head only over batch->n_out rows... */
+}
+```
+
+- `rad_issue(c, op, operands…, n)` / `RAD_ISSUE` (n = the batch's ranged extent) /
+  `RAD_ISSUE_N` (an explicit row count). `rad_step_fail(c, what)` sets the status helpers return.
+- **A mixed step is the trap.** Continuous batching produces steps with a prefill chunk beside decode
+  rows. A block that issues one path for the whole step sends its decode rows down the chunked path,
+  which commits recurrent state directly and takes no `num_accepted`; a rejected draft cannot be
+  rolled back, and the model stays fluent while drifting. Issue the recurrent path over
+  `RadBatch::n_seq_decode` sequences and the chunked path over the rest (the core orders steps
+  decode-first, so each half is contiguous; `cu_seqlens` holds absolute offsets, so pass
+  `cu + n_seq_decode` with the token operands at their base).
+- A step issued on lane 1 outside the declared window fails by name. An op the core appends behind a
+  declared one (runtime LoRA) counts at that op's position.
+- **Zero rows is not a call**: issuing an op over zero rows (a zero ranged extent, or all-empty
+  sequences) returns `RAD_E_SHAPE` and writes nothing. The caller skips the issue, as the
+  architectures' decode and prefill halves do.
+- `n_out` is 0 for a chunk that finishes no sequence — do not run the lm_head.
+
+## Probe: what the plugin measured
+
+`rad_arch_probe(meta, out)` runs before declare. Its fields are the engine's permissions:
+
+| field | effect |
+|---|---|
+| `draft_depth`, `draft_depth_fixed` | the container's spec depth; a block drafter's trained block is the only depth it serves |
+| `shape_probe_ok` | the plugin tolerates a sizing declare, so the activation arena can be lent |
+| `enc_merge`, `audio_frames` | encoder pass granularity; longest audio clip |
+| `gen_rows`, `gen_patches`, `gen_audio`, `gen_ctx`, `gen_video` | generator sizing before declare |
+| `kv_quant_ok` | bits of the narrow KV layouts this model's kernels serve |
+| `chain_pays`, `tree_pays` | the plugin measured chained launches / draft trees faster |
+| `kv_stream_ok` | the architecture can stream a KV group (sparse, block-addressed, with `kv_resolve`) |
+| `control_vector_ok` | the plugin applies `RadBuildCtx::cvec` |
+| `tp_attn_rows_ok` | the plugin serves row-split prefill attention |
+| `state_f16_ok` | the plugin declares recurrent state at f16 and its kernels read it |
+
+A flag whose probe bit is 0 is **refused at startup with the plugin named** (`--kv-cache-dtype
+fp8_v4`, `--control-vector`, `--tp-attn-split rows`, `--state-cache-dtype f16`, `--kv-resident`).
+That is the design: a deployment switch the plugin cannot honour is an error, never a silent
+substitution.
+
+## Worked pattern: the smallest complete plugin
+
+`arch/llama_fp8/llama_fp8.cpp` builds twice from one source — `llama_fp8` (quant `fp8_e4m3`) and
+`llama_dense_fp8` (quant `""`), differing only in the descriptor and the graph-dump text. It shows
+the whole shape:
+
+1. `struct Layer { AttnFP8 attn; MlpFP8 mlp; }`, `struct Model` holding `Names`, `Geom`, buffer and
+   op handles.
+2. `declare()`: name map entries first, then weights (with `rad_weight_encoding` where the head's
+   format decides the kernel), buffers, KV group, the embedding and its all-reduce, then each layer's
+   `declare()` from the shared blocks, the final norm, gather and lm_head, **the drafter last**, and
+   `rad_note` lines explaining the shape.
+3. `step()`: draft passes first (sign selects query/context), then embedding (+ exact all-reduce
+   when needed), each layer's `step()`, the final norm, and the head only over `n_out`.
+4. `probe()`: reads `draft.block_size` from the metadata and refuses any other depth for a block
+   drafter.
+5. `RAD_ARCH_PLUGIN(...)` at the bottom.
+
+Add a new family by copying a neighbour, replacing the name map and geometry, and composing the
+blocks you need; add a new *format* by cloning the plugin with another `(id, quant)` pair and the
+ops that format needs.
+
+## Verifying a new plugin
+
+```sh
+radiance --model m.rad --debug-graph          # every op, its resolved kernel, and why
+radiance --model m.rad --debug-placement      # the plan
+rad-convert $SRC -o m.rad --plan-only -v      # a convert that plans will boot
+rad-kbench -m m.rad --kernels mylib,libr4d,libref --report k.md --bench
+rad-tune -m m.rad --tp 2                      # only at the shapes you serve
+```
+
+`tests/arch_test.cpp` drives a plugin's builder the way the core does, including lane-window
+liveness, so the pinned graphs and steps are the ones the core plans. A plugin that declares a weight
+form no kernel `layout` accepts is a declare-time error naming the weight, encoding and kernel — the
+same check `rad-convert` runs before writing.
+
+## Related
+
+- [[Radiance Plugins]] — kernel libraries, encodings, the build path, ABI law.
+- [[Radiance Conversion]] — how the name map drives a convert, and `--set`/descriptor selection.
+- [[Radiance Model Families]] — the existing `(id, quant)` pairs and their recipes.
+- [[Radiance Tools]] — `rad-kbench`, `rad-tune`, `rad-info --plugins`.

serverAppConfigs-docs/LLM/Radiance Conversion.md

@@ -0,0 +1,354 @@
+---
+type: "howto"
+title: "Radiance Conversion — checkpoints into .rad containers"
+description: "Turning a checkpoint into a .rad container: rad-convert, the recipe language, quantisers, reuse and export."
+tags:
+  - llm
+  - radiance
+  - quantization
+  - howto
+timestamp: 2026-10-30
+---
+
+# Radiance conversion
+
+`rad-convert` turns a Hugging Face checkpoint into a `.rad` container, choosing each weight's
+encoding from a **recipe**. `rad-export` wraps it for finetunes (a merged checkpoint or a PEFT
+adapter) with the stock model's recipe. Neither opens a GPU: conversion is CPU-only and safe beside a
+running server.
+
+Fetch checkpoints with `scripts/hfget.sh <org/repo> <dir>` (resumable; reads `.safetensors`,
+skips `.bin`/ONNX/GGUF when safetensors exist; `HF_TOKEN` for gated repos; writes a
+`.hfget-manifest` of content keys). `radiance --model <owner/name>` runs the same fetch itself.
+
+## What a conversion does
+
+1. Read `config.json` (or GGUF metadata), and the shard index if present.
+2. Load the architecture, kernel and quantiser plugins and run **declare**, as the engine does.
+3. Resolve every op and ask each resolved kernel's `layout` hook about the planned encoding. A
+   recipe no kernel can serve is refused **here**, before any I/O.
+4. Write each logical weight once: the checkpoint's bytes where no rule matches, otherwise the
+   planes the matching quantiser writes, a block of rows at a time.
+5. Measure every quantised weight against its source through the core's decoder and print the worst.
+6. Bake the tokeniser, chat template, metadata, recipe and any draft head.
+
+```sh
+build/bin/rad-convert /srv/models/hf/Qwen/Qwen3.8-27B -o qwen3.8-27b-bf16.rad \
+    --recipe data/recipes/q38-27b-bf16.recipe
+build/bin/rad-convert /srv/models/hf/Qwen/Qwen3.8-Flash-Next -o flashnext-bf16.rad --max-ctx 4096
+```
+
+## Every `rad-convert` option
+
+| option | meaning |
+|---|---|
+| `<input>` | a `.gguf`, a `.safetensors`, an `index.json`, or a checkpoint directory |
+| `-o, --out FILE` | the container to write |
+| `--recipe FILE` | one rule a line: `PATTERN QUANTISER key=value …`; repeatable; first match decides |
+| `--quant RULE` | `PATTERN=QUANTISER[:key=value,…]`, placed **ahead of** the recipe's rules; repeatable |
+| `--list-quantizers` | every loaded quantiser and the options it takes, then exit |
+| `--arch ID` | override the architecture id read from the checkpoint |
+| `--home DIR` | `$RADIANCE_HOME` (kernels, architectures, quantizers) |
+| `--kernels A:B` | kernel libraries to load, first wins; declare checks the recipe against them |
+| `--tokenizer F` | `tokenizer.json`; its declared chain is baked instead of reconstructed from a pre-tokenizer name. Required for a GGUF, without it the chain is a guess |
+| `--tokenizer2 F` | a second `tokenizer.json` baked as the second vocab (a generator reading its prompt through two tokenisers); keep `tokenizer_config.json` beside it |
+| `--add FILE` | merge one more `.safetensors`, tensor names kept; repeatable; a name in two files is refused |
+| `--config DIR` | read `config.json`, `generation_config.json`, `tokenizer.json` from DIR for a single-file checkpoint |
+| `--chat-template F` | a Jinja template baked in, overriding the checkpoint's (or its absence); trailing newlines stripped; without one `/v1/chat/completions` answers 501 |
+| `--imatrix F` | llama.cpp imatrix: expert-popularity profile, per-column importances, error weighting; required by `ggml` types `iq2_xxs`, `iq2_xs`, `iq1_s` |
+| `--set K=V` | a metadata key, seen by declare **and** written into the container; repeatable |
+| `--max-tok N` | the `max_tok` declare runs at (default 8192, or a generator's largest pass) |
+| `--max-ctx N` | the context declare runs at (default: the checkpoint's training context) |
+| `--max-spec N` | the `max_spec` declare runs at (default 4, or the block a drafter states); `0` leaves a draft head out |
+| `--draft-model DIR` | a drafter shipped as its own repository, merged under `draft.` |
+| `--kv-cache-dtype T` | `bf16` or `fp8`: the cache width declare runs at; the container is identical at either |
+| `--reuse FILE` | copy every weight an earlier container holds with the same encoding, quantiser, options and shape |
+| `--changed FILE` | with `--reuse`: checkpoint tensors (one name a line) that differ; every weight reading one is converted |
+| `--in-place` | with `--reuse`: extend that container; new weights go past its end, the header last; keeps `<file>.pre-append` |
+| `--lora FILE\|DIR` | merge a PEFT adapter into the checkpoint tensors before converting (`W + strength · scale · B·A`); repeatable, merged in order; each merged tensor joins `--changed` |
+| `--lora-strength F` | strength for every `--lora` (default 1) |
+| `--lora-map FILE` | JSON map for adapters whose module names differ: `{"strip":[prefix…],"modules":{"<pat>":[target…]}}`; one `*` matches a block index; several targets mean a fused module (rows split in order) |
+| `--plan-only` | declare, plan every encoding, print the plan, stop before writing |
+| `--report N` | print the N worst quantisation errors (default 20) |
+| `-v, --verbose` | per-weight detail |
+
+`--reuse` matches a weight on its **encoding, quantiser, options and shape — not on its source
+bytes** — and carries whatever calibration made the old planes. Use it only when source and
+calibration are unchanged, or name the changed tensors with `--changed`. `--in-place` refuses to
+change a weight the container holds or to add an expert.
+
+## The recipe language
+
+```
+# PATTERN                      QUANTISER  OPTIONS
+blk.*.ffn_*_exps.*.weight      gptq       codes=i4 group=128 scale=bf16 transform=fwht128 calib=$CALIB
+blk.*.ple_ngram.weight         rtn        codes=fp8_e4m3 block=*x* scale=bf16 rule=fixed scale_value=1.99317932128906e-4
+output.weight                  rtn        codes=fp8_e4m3 block=128x128 scale=bf16
+blk.*.weight                   rtn        codes=fp8_e4m3 block=128x128 scale=bf16
+```
+
+- One rule a line: `PATTERN QUANTISER [key=value …]`. `#` starts a comment.
+- `PATTERN` is a glob over the **whole logical weight name**: `*` crosses dots, `?` is one character.
+- **The first matching rule decides.** No rule → the checkpoint's own encoding, which must be one
+  the engine serves as is: `bf16`, `f16`, `f32`, or block-scaled FP8 (`X.weight` with
+  `X.weight_scale_inv`). Anything else is named and refused.
+- `$NAME` / `${NAME}` in a value comes from the environment; an **unset variable is refused**, never
+  expanded empty (so an empty `$CALIB` cannot silently produce an uncalibrated convert).
+- A quantiser may **decline** a weight (`RAD_E_UNSUPPORTED`), which keeps the checkpoint's encoding
+  — `rtn` declines a rank-1 tensor, so a pattern that also matches a norm leaves the norm alone.
+- An option the quantiser does not declare is refused, by name, before anything is quantised.
+- Two plugins naming the same quantiser are refused, both named.
+
+## Quantisers
+
+| quantiser | what it does |
+|---|---|
+| `rtn` | round to nearest on a grid; each block's scale chosen by `rule` |
+| `gptq` | error feedback against the layer's input Hessian (`calib=DIR`, damped by `damp` = 0.01 of the mean diagonal by default); `act_order=1` quantises columns by descending input energy and stores the permutation; `cd=N` adds coordinate-descent sweeps |
+| `awq` | per-input-column scaling to a searched power, then a per-block clip search, both against the Hessian; column scales are the last scale level |
+| `autoround` | sign-gradient descent on a rounding offset per value and a range factor per block; `iters` steps; slow (a 27B dense model takes most of a working day at defaults; `iters=50` a quarter) |
+| `paroquant` | channel scaling plus `stages` Givens rotations learned against the Hessian, then AutoRound on the rotated weight; the `givens` transform |
+| `m2xfp4` | M2XFP's weight-side Sg-EM: MXFP4 plus one metadata byte per group of 32; its own `m2xfp4` scheme; `adaptive=0` pins the bias, `wide=1` widens it |
+| `ggml` | a llama.cpp format (`q4_0`, `q4_1`, `q5_0`, `q5_1`, `q8_0`, `q2_K`…`q6_K`, `iq4_nl`, `iq4_xs`, `iq2_xxs`, `iq2_xs`, `iq2_s`, `iq3_xxs`, `iq3_s`, `iq1_s`, `iq1_m`), by llama.cpp's own search, imatrix-weighted, codes byte for byte |
+| `cast` | a float weight in another float dtype (`dtype=` f32, bf16, f16, fp8_e4m3, fp8_e5m2), no scale |
+| `import` | rad-convert's own, not a plugin: write the numbers a quantised release ships, in the spelling the kernels read, instead of quantising again |
+
+Every quantiser except `m2xfp4`, `ggml`, `cast` and `import` takes the grid options:
+
+| option | values |
+|---|---|
+| `codes` | `i2`…`i8`, `i16`, `i32` symmetric; `u1`…`u8`, `u16` with `zero=`; `fp8_e4m3`, `fp8_e5m2`, `fp6_e2m3`, `fp6_e3m2`, `fp4_e2m1` |
+| `table` | a codebook the codes index: `nf4`, `iq4nl`, `w4nl`, `w5nl`, `binary`, `ternary`, or `;`-separated values |
+| `group=N` / `block=RxC` | where the scales sit; `*` is a whole extent |
+| `scale` | f32, bf16, f16, fp8_e4m3, e8m0; under `scale2` also an integer sub-scale `u4`…`u8`, `i6`, `i8` |
+| `scale2`, `group2` / `block2` | a second scale level the first is relative to (default the whole tensor) |
+| `scale2_value` | the second level fixed at this value, exact in `scale2`'s dtype |
+| `zero` | an integer zero point per block — an asymmetric grid |
+| `transform=fwhtN` | rotate each row by the Hadamard first |
+| `rule` | `absmax`, `mx` (OCP shared exponent), `fixed` (`scale_value=`), `search` (least squared error over candidates and refits, imatrix-weighted) |
+| `clamp` | a signed code's range: `full` or `sym` |
+
+Named grids: MXFP4 = `codes=fp4_e2m1 group=32 scale=e8m0`; NVFP4 =
+`codes=fp4_e2m1 group=16 scale=fp8_e4m3 scale2=f32`; NF4 = `table=nf4 group=64`; a K-quant-like grid
+= `codes=i4 group=32 scale=u6 scale2=f16 group2=256`; a ternary BitNet weight = `table=ternary`.
+
+**A container can hold only what some kernel reads.** rad-convert checks every planned encoding
+against the `layout` hooks of the kernels declare resolved and refuses one no kernel accepts. libr4d
+reads the plain floats; block-scaled fp8 and fp8 rows; int8 block rows plain or rotated; routed
+experts as fp8, int4 (`i4*bf16[1x128]/fwht128`) or the `w4nl`/`w5nl` codebooks under E4M3 scales;
+the dense w4, w8 and mxfp4 GEMM rows; and the 2-bit draft head. See [[Radiance Plugins]] for the
+encoding model.
+
+## Importing a release someone else quantised
+
+These forms are read weight by weight and written in the spelling the kernels read; a recipe rule
+re-quantises from the exact dequantisation where the family's kernels want another form.
+
+| `quantization_config` | read as | notes |
+|---|---|---|
+| `quark` MXFP4 (fp4, group 32, e8m0) | `fp4_e2m1*e8m0[1x32]` | byte for byte; also servable as it stands |
+| `paroquant_mxfp4` | `fp4_e2m1*e8m0[1x32]/givens` | codes/exponents byte for byte plus learned rotations as `givens` tables |
+| `paroquant`, `int5-bitplane`, plain `paroquant` | `u5*f16[1x128]-u8[1x128]/givens`, `u4*…` | AWQ-packed nibbles re-laid as a code bit stream; f16 scales, integer zeros, rotations |
+| `compressed-tensors` `pack-quantized`, 4-bit `int`, group, act-order none/static/weight | `i4*<scale>[1xG]` | `weight_packed` made signed beside `weight_scale`; group/dynamic reorder is refused |
+| same with `symmetric:false` (AWQ) | `u4*<scale>[1xG]-u4[1xG]` | nibbles plus `weight_zero_point`, bit for bit |
+| `auto-round` `auto_round:auto_gptq`, `gptq` v1 | `i4*bf16[1x128]` or `i8*bf16[1x128]` | scales rounded to bf16; convert-time; `import` keeps shipped f16 |
+| `nvfp4-pack-quantized` | `fp4_e2m1*f32[1x16]` | `weight_scale / weight_global_scale`; **no kernel serves NVFP4**, so it always takes a rule |
+| `modelopt` NVFP4 / FP8 / MIXED_PRECISION | NVFP4 `fp4_e2m1*f32[1x16]`; FP8 `fp8_e4m3*f32[1x*]` | ModelOpt's second scale **multiplies**; `input_scale` is not read |
+| compressed-tensors all-FP8 groups | `fp8_e4m3*<scale>[1x*]` or `[RxC]` | nothing imported: stock FP8 with its scale; a per-tensor scale is written once a row |
+| GGUF `Q8_0`, `Q4_K`, `Q6_K` | `i8*f16[1x32]`, affine u4/u6…, `i6*i8…` | blocks split bit for bit; **not servable as they stand** |
+
+The width is read **per tensor**, so a mixed release imports int4 experts and int8 projections
+together. What an import cannot read (an asymmetric GPTQ zero point, a group-reordering act-order
+index, a GPTQ group under 128, a missing/zero global scale, an unlisted format) is refused by name.
+The release's `quantization_config` names the tool, not a served form, so it is kept as
+`imported_quantization_config.*`.
+
+`import` options: `codes=uN` makes a symmetric grid unsigned under a zero point of `2^(N-1)`;
+`zero=` the zero plane's dtype (`u4`, `u8`); `scale=` f16/bf16/f32, each shipped scale rounded to
+nearest; `block=RxC` repeats an FP8 scale shipped per row/tensor over that grid. It reads an imported
+integer grid or FP8 with a per-row/per-tensor scale and keeps a plain float weight as it is.
+
+**NVFP4 to MXFP4 rounds twice** (e4m3 per 16 → power of two per 32), so it loses more than MXFP4 made
+from bf16. Prefer a calibrated MXFP4 container when one exists; the quality verdict is a KL gate, not
+a file size.
+
+## The descriptor law (`--set`)
+
+The architecture plugin is chosen by the exact `(arch id, quant descriptor)` pair. A convert-time
+choice that must also hold at serve time is stated once with `--set`, which hands it to declare and
+writes it into the container. Storage format is not such a choice (the encoding is in the container
+and the plugin reads it there), but the **serving plugin** is: a `quantization_config.*` key
+re-derives the descriptor.
+
+```sh
+rad-convert <bf16 Qwen3.8-27B> -o q38-27b-fp8.rad \
+    --recipe data/recipes/q38-27b-fp8block-df2.recipe \
+    --set quantization_config.quant_method=fp8 --set quantization_config.fmt=e4m3 \
+    --set "quantization_config.weight_block_size=128 128" \
+    --set quantization_config.activation_scheme=dynamic
+```
+
+A bf16 checkpoint whose recipe writes Qwen's block FP8 must therefore be declared *as that format*;
+otherwise the selection pair does not match any plugin and the convert (or the boot) is refused. The
+same law bites the other way when a release's own `quant_method` names a pair no plugin claims.
+
+## Inspect and validate the result
+
+```sh
+rad-info out.rad                  # architecture, source quant, recipe, sections
+rad-info --recipe out.rad         # the expanded rules — did the intended one win?
+rad-info -v out.rad | head -40    # every tensor, its encoding, and what made it
+rad-kbench -m out.rad --home $RADIANCE_HOME --report kbench.md
+radiance --model out.rad --debug-graph     # every op resolves
+```
+
+The converter's own error report (the worst `--report N` weights) measures each quantised weight
+against its source through the core decoder. `rad-kbench` adds the artifact test: weight operands
+read from the container and passed through each kernel's `layout`/`relayout` hooks. Both are on
+[[Radiance Tools]].
+
+## Calibration (GPTQ, AWQ, AutoRound, ParoQuant)
+
+`calib=DIR` is a directory an engine run wrote with `RADIANCE_CALIB_DIR` set: each layer's **Gram
+matrices of its input**, in the domain the kernel multiplies it in. A block that characterises an
+activation declares an f32 `[K,K]` PERSIST buffer and a `gram_accum` op; at `RADIANCE_CALIB_TOKENS`
+(default 65536) the run writes the files.
+
+Three-step bootstrap: convert with plain `rtn` → serve that container with `RADIANCE_CALIB_DIR` →
+push real prompts through prefill → convert again with the Hessians. The second moment is robust to
+the first container's small quantisation error. A weight with no file is quantised uncalibrated and
+the converter says so; `awq`, `autoround` and `paroquant` fall back to the imatrix diagonal when one
+is given.
+
+- The accumulator must be **PERSIST**: a transient buffer may share bytes and be overwritten between
+  the launches it sums.
+- Files are per rank; the consumer sums them. Under expert parallelism a rank's `down` input covers
+  only its experts' tokens; `gate_up` input is the full token set on every rank, so the sum scales
+  `H` by world size — GPTQ is invariant to a positive scale.
+- Under tensor parallelism a column-split linear sees only its rank's columns, so a tap may write
+  `gram.r<rank>.<stem>.part.bin`; libquant places the parts on the diagonal in rank order, leaving
+  cross-rank products zero. They must cover K and be in the model's own domain.
+- `--imatrix` supplies the expert-popularity profile the placement planner ranks units by, the
+  per-column importances a quantiser uses, and the error report's weighting.
+
+## Reuse, drafters, merging
+
+```sh
+# re-quantise only what changed
+rad-convert $SRC -o new.rad --recipe new.recipe --reuse old.rad --changed changed.txt
+
+# merge a drafter shipped as its own repo (DFlash2, DSpark, MTP assistant)
+rad-convert $SRC -o out.rad --draft-model /srv/models/z-lab/Qwen3.8-27B-DFlash2 \
+    --recipe data/recipes/q38-27b-mxfp4-df2.recipe
+
+# merge a PEFT adapter at convert time
+rad-convert $SRC -o tuned.rad --lora ~/run/adapter --lora-strength 1.0 \
+    --recipe data/recipes/q38-27b-mxfp4-df2.recipe
+```
+
+- A drafter's tensors go in under `draft.`, its config under `draft.` keys. It has no embedding and
+  no `lm_head` of its own; it uses the target's.
+- `--lora-map` exists for adapters trained on names the checkpoint does not use; a fused target
+  (`qkv`) takes several targets and splits the adapter's rows in order.
+- `--add` + `--config` + `--tokenizer2` assemble a ComfyUI-style release of separate single files
+  (Anima: diffusion model, text encoder, VAE). A name in two files is refused.
+
+## `rad-export` — finetunes without hand-writing the command
+
+```sh
+rad-export ~/outputs/my-run/checkpoint-400 --dry-run   # what it would run
+rad-export ~/outputs/my-run/checkpoint-400             # adapter -> <name>.rad
+rad-export ~/exports/merged -o ~/models/mine.rad --recipe q38-27b-fp8block-df2
+```
+
+| flag | default | meaning |
+|---|---|---|
+| `SRC` | required | a merged 16-bit checkpoint (`config.json` + `*.safetensors`) or a PEFT adapter (`adapter_config.json` + `adapter_model.safetensors`) |
+| `-o, --out FILE` | SRC's folder name + `.rad`, beside SRC or in `out_dir` | the container |
+| `--recipe NAME\|FILE\|auto` | `auto` | a recipe name, a file, or the one recipe whose header names the base model, takes a bf16 checkpoint and needs no calibration |
+| `--base DIR` | the adapter's `base_model_name_or_path`, from disk or the HF cache, else downloaded | the base a LoRA merges into |
+| `--draft DIR\|ORG/NAME` | the config, else the Hub repo the recipe's usage line names | the drafter when the recipe merges one; a usage line with a placeholder asks for `--draft` |
+| `--image IMAGE` | `$RADIANCE_IMAGE`, the config's, then `stilldeadcode/radiance` | run `rad-convert` in this image (docker, CPU-only, no network, **not pulled**) |
+| `--rad-convert PATH` | — | run this binary instead of docker; recipes come from its `share/radiance` |
+| `--memory M` | — | docker memory limit for the convert |
+| `--dry-run` | — | resolve recipe, drafter and paths; print the `rad-convert` line; write nothing |
+| `--keep-merged` | — | keep the merged checkpoint beside OUT |
+| `--config FILE` | `~/.config/rad-export/config.json` | the config below |
+| `--list-recipes [--json]` | — | list the recipes the image/installed tool carries |
+
+- Input: a merged 16-bit checkpoint (`config.json` + `*.safetensors`) or a PEFT LoRA adapter
+  (`adapter_config.json` + `adapter_model.safetensors`).
+- An adapter is merged **exactly as PEFT's `merge_and_unload`** computes it on CPU: `scale` =
+  alpha/r (alpha/√r with rsLoRA), `rank_pattern`/`alpha_pattern` honoured, delta in fp32 rounded to
+  the adapter's dtype. It streams tensor by tensor into 1 GiB shards, so a 27B merge needs a few GiB
+  of RAM, not the model twice. Plain linear LoRA only: embedding LoRA, trained biases, DoRA, saved
+  modules, convolution LoRA, a rank other than the config's, and a missing target are refused before
+  anything is written. The adapter's tokeniser/template replace the base's where shipped.
+- The recipe comes from the image (`/opt/radiance/share/radiance/recipes`) or the installed
+  `rad-convert`'s share; `auto` matches the base model's name in recipe headers and asks when several
+  fit. A recipe needing calibration is listed but never picked by `auto` — a finetune's Hessians are
+  not the stock model's.
+- It runs `rad-convert` on the CPU, under Docker as the caller with no network, and writes the log to
+  `OUT.log`.
+- `~/.config/rad-export/config.json` pins what auto cannot know: `image`, `out_dir`,
+  `recipes{model:recipe}`, `drafts{recipe:dir}`, `bases{repo:dir}`.
+- It is Python 3 with no dependencies; merging needs `torch`/`safetensors`, fetching a missing base or
+  drafter needs `huggingface_hub`. The image carries no Python; extract the script with
+  `docker run --rm --entrypoint busybox stilldeadcode/radiance cat /opt/radiance/bin/rad-export`.
+
+## Scoring a container (the KL mode)
+
+```sh
+# once, on the bf16 container
+radiance --model bf16.rad --kld-record /srv/kld/q38 --kld-corpus corpus.jsonl
+# each candidate, same flags otherwise
+radiance --model quant.rad --kld-ref /srv/kld/q38 --kld-out report.json
+```
+
+- The corpus is JSONL, one `{"prompt", "score_from"?, "source"?}` a line; only `prompt` is required.
+  Positions `[score_from, n-1)` are scored; `source` groups the report. Keep the scoring corpus
+  disjoint from the calibration corpus.
+- `--kld-record` writes the reference's log-probabilities (f16 over the vocabulary, the 16 most
+  likely tokens exactly, the token ids). DIR is never overwritten and is read only once `kld.json`
+  exists, which is written last.
+- `--kld-ref` scores this model's f32 logits against stored rows: mean, median, 99/99.9/99.99th
+  percentile KL, top-1 agreement, and both models' perplexity. `--kld-out` writes the report; `FILE.rows`
+  is `[positions, 4]` f32 (KL, both NLLs, argmax agreement) for position-by-position comparison.
+- The mode turns speculation and the prefix cache off, turns the step pipeline off, and refuses an
+  explicit spec depth above 0. The logits buffer holds a row per token of a step, so a large
+  `--max-num-batched-tokens` costs pinned host memory as well as VRAM.
+- `--kld-lora NAME` scores every document through a served adapter, against a reference recorded
+  from the merged checkpoint; `--kld-lora-sources PREFIX` runs only matching documents through it,
+  the rest as the base, in the same steps — each document's rows should be byte-identical to the
+  all-adapter and all-base runs.
+
+The gate the fleet uses (mean KL ≤ 0.03, top-1 ≥ 95 %) is on [[frank-archon]] and
+[[Quantization Production]]; the mode here is only the instrument.
+
+## Convert laws paid for in full
+
+- **`--plan-only` first.** A plan that resolves every op will boot; it costs seconds against an hour
+  of I/O on a large checkpoint.
+- **Declare runs at the step and context bounds, and some graphs depend on them.** Qwen4-Exp sizes
+  its QSA buffers by `--max-ctx` and refuses a `--max-tok`/context pair whose score plane would pass
+  2 GiB; the defaults (8192 tokens, the checkpoint's training context) fit most models, and a
+  generator's largest pass may override `--max-tok`.
+- **Scale names follow the tensor.** `scale_name()` appends `_scale_inv` to the *tensor* name, so a
+  stacked `…experts.gate_up_proj` keeps its scales at `…gate_up_proj_scale_inv`.
+- **File shadowing: first wins.** The reader opens index-named files sorted and emplaces every tensor
+  each file physically holds, first-wins. A merged PLE shard must sort first
+  (`00-ple.merged.safetensors`) or a raw shard wins the name.
+- **Never write through a staging symlink.** A checkpoint may symlink its raw index; `open(path,"w")`
+  writes through it and corrupts the checkpoint. Write a temp file and `os.replace`.
+- **One encoding per expert table.** A whole expert kept bf16 is the only thing legal beside a
+  table; a split tier (down=fp8, gate_up=w4) is unservable.
+- **The descriptor pair has no fallback** (above), and **convert is card-less**: GPTQ solves on CPU
+  BLAS, so the GPU is irrelevant to production and the card only decides what can be decoded.
+
+## Related
+
+- [[Radiance Model Families]] — the (arch id, quant descriptor) table and the command per model.
+- [[Radiance Plugins]] — quantiser and kernel-plugin contracts behind these options.
+- [[Radiance Tools]] — `rad-info`, `rad-kbench`, `rad-tune`.
+- [[Quantization Production]] — the fleet's calibration/KLD campaign and its gates.
+- [[frank-archon]] — the deployment whose recipes and convert laws these are.

serverAppConfigs-docs/LLM/Radiance Engine.md

@@ -0,0 +1,140 @@
+---
+type: "reference"
+title: "Radiance Engine — the inference server behind frank-archon"
+description: "The radiance inference engine: its shape, plugin model, container format, and how the reference pages fit together."
+tags:
+  - llm
+  - radiance
+  - reference
+timestamp: 2026-10-30
+---
+
+# Radiance Engine — shape, plugin model, container
+
+Radiance is StillDeadcode's inference server for AMD GPUs, written in C++20 and HIP: language
+models, plus image, video, speech and music generation and embeddings. One HTTP front end speaks
+OpenAI's chat, completions, Responses, images, audio and video APIs, Anthropic's Messages API and
+llama-server's native one. Nothing in the serving path is Python.
+
+The fleet's only radiance endpoint is [[frank-archon]] (2× R9700, TP=2, Dankserver). **This page
+and its siblings own the engine reference; the live deployment shape, its Eras and its measured
+laws stay on the endpoint page.** The numbers here describe upstream `main` as read 2026-10-30
+(release 1.14.0, plugin ABI 31). The fleet pin is older (1.8.1): a flag below that post-dates the pin
+is a boot refusal, never a silent no-op, and the endpoint page records the CLI families that pin
+itself retired relative to its own rollback target.
+
+Source: `https://codeberg.org/StillDeadcode/radiance` at `8da043c`. Engine docs live in `docs/`
+(`GUIDE.md` for serving, `TOOLS.md` for tools, `PLUGIN.md` for plugins, `ARCHITECTURES.md` per
+family, `OPS.md` for the op vocabulary, `spec.md` for the design).
+
+## The four moving parts
+
+```mermaid
+graph LR
+    A[Checkpoint<br/>safetensors / GGUF] -->|rad-convert + recipe| B[.rad container]
+    B --> C[radiance engine]
+    C --> D[OpenAI / Anthropic /<br/>llama-server wire APIs]
+    P1[kernel library .so<br/>libr4d, libavx, libref] --> C
+    P2[architecture .so<br/>one per arch id × quant descriptor] --> C
+    P3[quantiser .so<br/>libquant] -->|rad-convert| A
+```
+
+- **Engine** — `core/`: declare phase, placement/memory planner, scheduler, KV manager, sampler,
+  tokeniser, HTTP server, media preprocessing.
+- **Kernel libraries** — implement ops for hardware. `libr4d` targets RDNA4 (`gfx1200`,
+  `gfx1201`); `libavx` serves the host backend (AVX-512/AVX2) with no GPU at all; `libref` is the
+  slow reference implementation every other library is checked against.
+- **Architecture plugins** — declare one model family at one weight format: its weights, buffers,
+  KV groups and the ops it issues each step. The engine picks one by an **exact (architecture id,
+  quantisation descriptor) pair, with no fallback**.
+- **Quantiser plugins** — `libquant`'s `rtn`, `gptq`, `awq`, `autoround`, `paroquant`, `m2xfp4`,
+  `ggml`, `cast`, `import`; they write a weight's numbers into an encoding's planes.
+- **Tools** — `rad-info`, `rad-convert`, `rad-export`, `rad-kbench`, `rad-tune`, `rad-schemas`,
+  `rad-tokcheck`, plus `rad-chatparse`, `rad-avx-check`, `r4d_selftest` in the build tree.
+
+## Two phases, one static graph
+
+`rad_arch_declare` runs once per rank and names every weight, buffer, KV group and op, in order.
+The core resolves every op against the kernel hierarchy per band, per domain, and completes the
+whole plan before a byte of weights is read. `rad_arch_step` then only *issues* handles obtained at
+declare; it may not allocate and may not synchronise. A model whose declare succeeds is fully known
+statically, which is what makes `--debug-graph`, the buffer liveness plan and `rad-convert`'s
+plan-only check possible.
+
+Consequences a plugin author must respect:
+
+- An op that never resolves is `RAD_NULL_HANDLE` and only fails if issued. A fusion is a *selection*:
+  ask for the fused op, emit the chain when `rad_op_resolved` is false.
+- Declaration order is the buffer planner's liveness. A drafter must be declared after the trunk
+  it runs behind, or trunk and drafter buffers silently share bytes.
+- Every op in the graph must have a row in a loaded kernel library. `libref` implements the
+  conventional vocabulary in `docs/OPS.md`; device-only ops (most fusions) exist only in `libr4d`.
+
+## The `.rad` container (RAD2)
+
+A container is one file holding everything the server needs: weights as **canonical encoding
+planes**, the tokeniser (and optionally a second vocab), the chat template, metadata, the recipe it
+was made with, and any draft head. It names no kernel library.
+
+- Planes are kernel-independent: codes, scales, zero points, codebooks — row-major, bit-packed
+  sub-byte where needed. A kernel rearranges them into its own layout at load (`layout`/`relayout`
+  hooks), losslessly, so a kernel library can change layouts without reconverting the model.
+- Encodings are data, not an enum: `RadEncoding` names the scheme (`plain`, `affine`, or a
+  quantiser's own), the planes by role, the scale grouping, and a transform (`fwhtN`, `perm`,
+  `givens`). That is how one file can hold block-FP8, int4, MXFP4, NVFP4-imported, ParoQuant
+  rotations and llama.cpp grids.
+- `rad-info` is the reader: sections (strings, metadata, directory, planes, encodings, vocab,
+  profile, data), `--meta`, `--vocab`, `--recipe`, `--plugins`, and `-v` for every tensor with its
+  encoding and what made it.
+
+See [[Radiance Conversion]] for producing one and [[Radiance Tools]] for inspecting it.
+
+## Selection key and version law
+
+- A checkpoint's architecture id is its `config.json` `model_type` without underscores; `rad-convert
+  --arch ID` overrides it. `rad_arch_quant` supplies the quantisation descriptor (empty for an
+  unquantised checkpoint).
+- **No fallback.** A raw official FP8 checkpoint whose `quantization_config.quant_method` is `fp8`
+  is not the pair the `qwen4exp_fp8` plugin claims (`(qwen4exp, "")`), so both serve and convert
+  refuse it until the descriptor is changed with `--set` or a staged `config.json`. The deployment
+  lesson is on [[frank-archon]]; the mechanism is on [[Radiance Conversion]].
+- `RAD_ABI_VERSION` covers every header a plugin compiles against. A plugin reporting a different
+  number is refused at load, with both numbers printed. It moves on any shape change, not on a new
+  optional export; rebuild when it changes. Release version and ABI are independent: `/version`
+  reports the release, `rad-info --plugins` reports what loaded.
+
+## Environment
+
+| variable | meaning |
+|---|---|
+| `RADIANCE_HOME` | plugin search path: `:`-separated homes, each holding `kernels/`, `architectures/`, `quantizers/`; first wins, same-name files shadow later homes. Compiled-in default is `<prefix>/share/radiance` |
+| `RADIANCE_KERNELS` | default for `--kernels`: ordered kernel libraries, first wins; unnamed libraries are not loaded |
+| `RADIANCE_TUNE` | pin a kernel's tuned axes for a measurement (a measurement aid, not a deployment setting) |
+| `RADIANCE_CALIB_DIR` | a serve run records per-layer input Gram matrices here (the GPTQ calibration tap) |
+| `RADIANCE_CALIB_TOKENS` | tokens after which the tap stops and writes (default 65536) |
+| `RADIANCE_LOG`, `RADIANCE_LOG_STEPS` | log level; per-step token counts |
+| `RADIANCE_DEBUG_RESID` | residual-stream probe per (layer, site) |
+| `RADIANCE_DFLASH_Q_TEMP` | temperature of a DFlash2 coupled walk's candidate softmax |
+
+The full list of debug knobs (`RADIANCE_DEBUG_*`, `RADIANCE_DUMP_*`, `RADIANCE_OP_ORACLE`, …) is in
+the source's `getenv` calls; none is a deployment setting.
+
+## Where the pages are
+
+| Page | Owns |
+|---|---|
+| [[Radiance Serve Options]] | every serve flag, grouped, with defaults and the law behind each knob |
+| [[Radiance HTTP API]] | endpoints, request fields, streaming, tool calls, refusal rules |
+| [[Radiance Conversion]] | `rad-convert`, the recipe language, quantisers, imports, reuse, `rad-export`, KL scoring |
+| [[Radiance Model Families]] | which plugin claims which model, and the conversion command per family |
+| [[Radiance Plugins]] | kernel and quantiser plugins: ABI, rows, hooks, encodings, build and checks |
+| [[Radiance Architecture Plugins]] | declaring a model: weights, buffers, KV groups, ops, step, probe |
+| [[Radiance Tools]] | `rad-info`, `rad-kbench`, `rad-tune`, `rad-schemas`, `rad-tokcheck`, harnesses |
+| [[Radiance Operations]] | the container contract, host prerequisites, readiness, ledgers, restarts |
+| [[frank-archon]] | the fleet's live radiance endpoint: shape, Eras, rollback, measured laws |
+
+## Sources
+
+- `README.md`, `docs/GUIDE.md` §1–§4, `docs/TOOLS.md`, `docs/PLUGIN.md`, `docs/DOCKER.md`,
+  `docs/ARCHITECTURES.md`, `docs/OPS.md`, `spec.md` §2–§4, `abi/*.h`, `core/config.cpp`,
+  `core/rad_core.h` at commit `8da043c` (codeberg.org/StillDeadcode/radiance).

serverAppConfigs-docs/LLM/Radiance HTTP API.md

@@ -0,0 +1,198 @@
+---
+type: "reference"
+title: "Radiance HTTP API — endpoints, fields, refusals"
+description: "The HTTP surface radiance serves: endpoints, request fields, streaming, tools, and refusal rules."
+tags:
+  - llm
+  - radiance
+  - api
+  - reference
+timestamp: 2026-10-30
+---
+
+# Radiance HTTP API
+
+One server serves one model. It answers OpenAI's chat/completions/Responses/images/audio/video APIs,
+Anthropic's Messages API and llama-server's native one, so their clients and SDKs work unchanged.
+`model` in a request is accepted and **not enforced**.
+
+## Endpoints
+
+| route | what it serves |
+|---|---|
+| `POST /v1/chat/completions` | chat: tools, reasoning, streaming, grammars |
+| `POST /v1/completions` | raw text completion |
+| `POST /v1/embeddings` | an embedding head, else 501 |
+| `POST /v1/images/generations`, `/v1/images/edits` | OpenAI images API; needs an image model, else 501 |
+| `POST /v1/systemone` | LiquidAI d1 decisions: named yes/no, choice and score questions as probabilities; needs a d1 container, else 501 |
+| `POST /v1/audio/transcriptions` | OpenAI speech-to-text; needs a transcribing plugin, else 501 |
+| `GET /v1/realtime` (WebSocket) | streaming transcription session |
+| `POST /v1/audio/speech` | speech (Breeze-TTS-2, drama-3) or a song (MiniMax-Music3), whole or streamed |
+| `GET /v1/audio/speech/stream` (WebSocket) | text streamed in, each sentence's audio out |
+| `POST /v1/videos`; `GET /v1/videos[/:id[/content]]`; `DELETE /v1/videos/:id` | video jobs made in the background, downloaded as MP4 |
+| `POST /v1/messages`, `/v1/messages/count_tokens` | Anthropic Messages, for Claude Code and the Anthropic SDKs |
+| `POST /v1/responses`, `/v1/responses/input_tokens` | OpenAI Responses, for Codex and SDK clients |
+| `POST /completion`, `/completions`, `/infill` | llama-server native completion and fill-in-the-middle |
+| `GET /v1/models`, `/v1/models/:id` | what is loaded, with `max_model_len`, `root`, `capabilities` |
+| `GET /health`, `/v1/health`, `/ping`, `POST /ping` | liveness |
+| `GET /load`, `/version`, `/props`, `/server_info`, `/diagnostics` | load; release version; llama-server props; full configuration; the startup report as JSON (`/diagnostics` is same-machine-only without an API key) |
+| `POST /apply-template`, `/v1/chat/completions/input_tokens` | the rendered prompt; its token count |
+| `GET /metrics` | Prometheus |
+| `POST /tokenize`, `/detokenize`; `GET /get_tokenizer_info` | vLLM's shape and llama-server's at once |
+| `POST /reset_prefix_cache`; `DELETE /sessions/<label>`, `/sessions?id=<id>` | drop every cached prefix; forget a stored conversation |
+| `GET /`, `/dashboard`; `GET /stats`, `/graph`, `/sessions` | the dashboard and the JSON it is drawn from |
+
+Unversioned shorthands exist for chat, chat input tokens, embeddings, images, audio transcriptions,
+audio speech, models, responses and completions, for a client given the bare host as base URL.
+`/completions` **without** `/v1` is llama-server's native completion. A path nothing serves is a JSON
+404.
+
+A text prompt to `/v1/completions`, `/v1/embeddings` or `/tokenize` reads control tokens as tokens
+(as vLLM and llama.cpp do), so a client rendering its own template can send `<|im_start|>user\n…`.
+The model's declared BOS is prepended unless the prompt already opens with it.
+
+## Chat
+
+```sh
+curl -s localhost:8000/v1/chat/completions -H 'Content-Type: application/json' \
+  -d '{"model":"m","messages":[{"role":"user","content":"hi"}],"max_tokens":200,"temperature":0.7}'
+```
+
+- **No chat template → 501** on this route (the container carries one; `--override-chat-template`
+  replaces it). Use `/v1/completions` instead.
+- Thinking off has three spellings: `chat_template_kwargs.enable_thinking=false`,
+  top-level `enable_thinking=false`, or `reasoning_effort:"none"`.
+- Reasoning returns in `choices[].message.reasoning_content`, separate from `content`.
+  **Reasoning counts against `max_tokens`**; a reply that runs out inside it has empty `content` and
+  `finish_reason:"length"`. `usage.completion_tokens_details.reasoning_tokens` tells that case from a
+  model that answered nothing.
+- `thinking_token_budget` (aliases `thinking_budget`, `thinking_budget_tokens`,
+  `reasoning_budget_tokens`) closes the reasoning block at N tokens and forces the close, so it holds
+  under speculation and at any temperature. It needs a one-token end marker; otherwise the response
+  has no `completion_tokens_details` and a budget is a 400.
+- `continue_final_message` continues a conversation ending in an assistant message instead of
+  answering it; `add_generation_prompt` opens a new turn.
+- Tool calls: `tools`, `tool_choice` (`auto`, `none`, `required`, named function),
+  `parallel_tool_calls`. `required`/named hold the whole reply to calls after the reasoning block and
+  cannot combine with `response_format` or `grammar`.
+- Structured output: `response_format` (`json_object` or `json_schema`), llama-server's `json_schema`
+  shorthand, or a raw GBNF `grammar`. A chat template that drops the grammar makes the request a
+  refusal.
+
+## Streaming
+
+```sh
+curl -N localhost:8000/v1/chat/completions -H 'Content-Type: application/json' \
+  -d '{"model":"m","messages":[{"role":"user","content":"Count to ten."}],
+       "max_tokens":100,"stream":true,"stream_options":{"include_usage":true}}'
+```
+
+Server-sent events in OpenAI's delta format, terminated by `data: [DONE]`. **Nothing is sent while
+the prompt is read**: the first event (the `role: assistant` chunk) goes out with the first generated
+token, so a client timing its first event measures time-to-first-token. `return_progress` adds
+llama-server's `prompt_progress` chunks during prefill; `timings_per_token` adds `timings` to every
+chunk.
+
+## Completions, embeddings, images, audio, video, tokens
+
+- **Completions** — `/v1/completions` takes `prompt` (string or token ids), `echo`, `logprobs`,
+  `preserved_tokens`. `/completion` is llama-server's native route; `/infill` is fill-in-the-middle.
+- **Embeddings** — `input` is text, a batch, or token ids; `encoding_format` `float`/`base64`;
+  `dimensions` truncates to the first N and renormalises; `prompt_name` prepends the model's named
+  prompt. Pooling is the model's; the engine does not renormalise.
+- **Images** — `/v1/images/generations` returns `b64_json`; `size` defaults to 2048×2048. Edits take
+  one picture or up to ten (`image[]`), multipart as OpenAI's SDKs send them, with an optional mask
+  (repaint only where transparent). `partial_images` streams previews as SSE.
+- **Speech to text** — `/v1/audio/transcriptions`; `/v1/realtime` streams audio in and words back as
+  they are spoken. The model's prompt shape comes from its plugin's audio format, not from the server.
+- **Speech and music** — `/v1/audio/speech` (whole or streamed) with `instructions` describing a voice
+  (Breeze-TTS-2) or speakers written in the input (`<|speaker:1|>`, `[laughs]` for drama-3); a music
+  model takes the lyrics as `input` and the description as `instructions`. `/v1/audio/speech/stream`
+  streams text in and audio out as each sentence completes.
+- **Video** — `/v1/videos` is a job: a video with its stereo soundtrack from a prompt, 5–14 s, up to
+  1344×768, made in the background; `GET /v1/videos/:id/content` downloads the MP4.
+- **Tokenising** — `POST /tokenize` accepts vLLM's shape (`prompt` or `messages` in, `prompt` out) and
+  llama-server's (`content` in/out, `with_pieces`). `POST /detokenize` reverses it.
+  `GET /get_tokenizer_info` reports the tokeniser configuration.
+
+## Request fields
+
+Accepted fields, in OpenAI's, vLLM's and llama-server's spellings:
+
+| field | notes |
+|---|---|
+| `model` | accepted, not enforced |
+| `messages` / `prompt` / `input` | chat / completion / embedding |
+| `max_tokens`, `max_completion_tokens`, `n_predict` | default is what the context leaves; `-1` is llama-server's "no limit"; clamped to context and to `--max-tokens-cap` |
+| `temperature`, `top_p`, `top_k`, `min_p`, `typical_p` | sampler |
+| `presence_penalty`, `frequency_penalty`, `repetition_penalty`/`repeat_penalty`, `repeat_last_n` | penalties |
+| `xtc_probability`, `xtc_threshold`, `dry_*` | XTC and DRY |
+| `seed` | absent means "pick one", not 0 |
+| `n` | 1..`--max-n`; prefix cache makes the fan-out nearly free after the first |
+| `stop` | string or array, bounded by `--max-stop-strings`/`--max-stop-bytes` |
+| `priority` | scheduler hint |
+| `ignore_eos` | generate to `max_tokens` whatever the model emits |
+| `stream`, `stream_options.include_usage`, `timings_per_token`, `return_progress` | streaming controls |
+| `samplers` | llama-server's order; this engine's chain is penalties, DRY, top-k, top-p, min-p, typical, XTC, with temperature first or last; any other list is refused by name |
+| `session_id` | labels the conversation on `/sessions`, dashboard and `DELETE` |
+| `control_vector` | `true` applies the deployment's vector to this request; absent takes the deployment default; a deployment with no vector refuses `true` |
+| `logprobs`, `top_logprobs`, `n_probs` | up to 20 alternatives; not with `echo` |
+| `tools`, `tool_choice`, `parallel_tool_calls`, legacy `functions`/`function_call` | chat only |
+| `add_generation_prompt`, `chat_template_kwargs`, `continue_final_message` | chat only |
+| `reasoning_format`, `reasoning_effort`, `enable_thinking`, `thinking_token_budget`, `preserve_thinking` | chat only; levels are the template's (`capabilities.reasoning_efforts`); `none`/`minimal` turn thinking off |
+| `response_format`, `json_schema`, `grammar` | structured output; `grammar` and `response_format` are mutually exclusive |
+| `echo`, `preserved_tokens` | `/v1/completions` only |
+| `encoding_format`, `dimensions`, `prompt_name`, `image_tokens` | `/v1/embeddings` only |
+| `images_config` | image models only: a named processor mode or an object restating served sizes |
+
+## What is refused, ignored, and clamped
+
+The contract: **a 200 means every field that could change the answer was honoured.** Fields fall
+into four classes.
+
+- **Inert** — accepted and ignored because they cannot change the answer: OpenAI/OpenRouter
+  bookkeeping (`user`, `metadata`, `store`, `service_tier`, `prompt_cache_key`, …) and llama-server's
+  own speed/slot knobs (`cache_prompt`, `id_slot`, `n_cache_reuse`, `backend_sampling`,
+  `speculative.*`, …). The prefix cache and speculation here are the server's and never change output.
+- **Guarded** — known, unimplemented, and able to change the answer: accepted at their off value,
+  refused by name otherwise. `logit_bias` (empty), `mirostat` (0), `top_n_sigma` (≤0),
+  `dynatemp_range` (≤0), `min_keep` (0), `post_sampling_probs` (false), `best_of` (≤1), `suffix`
+  (empty), `lora` (empty — an adapter is chosen by `model`), `modalities`/`audio` (`["text"]`/null),
+  `n_indent` (0), `t_max_predict_ms` (≤0), `return_tokens` (false), `grammar_lazy`,
+  `grammar_triggers`, `preserved_tokens` (empty on chat).
+- **Ignored** — anything else unknown, logged at debug. An unknown field is far more often a client's
+  extension than a misunderstood request.
+- **`max_tokens` is the exception**: if prompt + `max_tokens` exceeds the context, it is **clamped**
+  and served, ending `finish_reason:"length"`, because refusing would produce no tokens instead of
+  fewer. A prompt that does not fit on its own is a 400 `context_length_exceeded` naming
+  `n_prompt_tokens`, `n_ctx`, and how many tokens must go.
+
+## Anthropic, Responses, llama-server
+
+- `/v1/messages` is translated into a chat request, so chat defaults, refusals and limits apply.
+  Streamed (`message_start`, content blocks, `message_delta`, `message_stop`) or not:
+  `ANTHROPIC_BASE_URL=http://host:8000 ANTHROPIC_API_KEY=$KEY claude` works. `/v1/messages/count_tokens`
+  returns the prompt length.
+- `/v1/responses` maps `instructions`/`input` (strings, messages, `function_call`,
+  `function_call_output`, `reasoning` items) into the chat request; `max_output_tokens`,
+  `reasoning.effort`, `text.format` map to chat fields; undefined fields pass through. The event stream
+  carries `output_index`, `content_index`, `sequence_number`, and opens with `response.created` and
+  `response.in_progress`. A reply cut at `max_output_tokens` is `incomplete`. Function tools a
+  `namespace` groups are named `<namespace>__<name>`; built-in tools and freeform custom tools are
+  left out.
+- `/completion`, `/completions`, `/infill` are llama-server's own shapes, for llama.vim/llama.vscode.
+
+## Auth, CORS, errors
+
+- `--api-key KEY` requires `Authorization: Bearer KEY` or `X-Api-Key: KEY` on every request except
+  `/health`, `/v1/health`, `/ping` and the dashboard page. An empty key is refused at startup.
+- CORS headers are sent unless `--no-cors`.
+- Errors are JSON. Over-context is 400 `context_length_exceeded`; a full queue is 429 with
+  `Retry-After`; a capability the model lacks is 501; a guarded field is 400 naming the field and its
+  accepted off value.
+
+## Related
+
+- [[Radiance Serve Options]] — the defaults every omitted field takes.
+- [[Radiance Operations]] — `/health` semantics, metrics, dashboard.
+- [[frank-archon]] — the harness contract built on this surface.

serverAppConfigs-docs/LLM/Radiance Model Families.md

@@ -0,0 +1,142 @@
+---
+type: "reference"
+title: "Radiance Model Families — arch ids, descriptors and recipes"
+description: "Which radiance plugin claims which model family, and the conversion command and recipe of record for each."
+tags:
+  - llm
+  - radiance
+  - models
+  - reference
+timestamp: 2026-10-30
+---
+
+# Radiance model families
+
+The selection key is the exact **(architecture id, quantisation descriptor)** pair, so a family at
+two weight formats is two `.so` files. A checkpoint's id is its `config.json` `model_type` without
+underscores (`qwen3_5` → `qwen35`); `rad-convert --arch ID` overrides it. `data/recipes/` is the
+authority: **read the recipe's header** — it names the exact `rad-convert` line, its `--set` keys
+and whether it needs calibration.
+
+## The plugin table
+
+| plugin | id | quant | what it serves |
+|---|---|---|---|
+| `llama_fp8` | `llama` | `fp8_e4m3` | plain LLaMA decoder (MiniCPM5, Llama 3, derivatives reporting `model_type: llama`) from a checkpoint holding E4M3 codes + `weight_scale_inv` |
+| `llama_dense_fp8` | `llama` | *(empty)* | the same graph from a bf16 checkpoint, linears made block fp8 by a recipe |
+| `qwen35_bf16` | `qwen35` | *(empty)* | Qwen3.5/3.6/3.8 dense hybrid (GDN + gated attention), bf16 |
+| `qwen35_fp8` | `qwen35` | `fp8_e4m3` | the same, block-scaled FP8 weights and activations |
+| `qwen35_w8a8` | `qwen35` | `w8a8_fwht128` | int8 trunk: i8 weights, bf16 scale per 128 of K, 128-wide Hadamard, int8 activations |
+| `qwen35moe_fp8` | `qwen35moe` | `fp8_e4m3` | `qwen35_fp8` with a routed feed-forward |
+| `qwen4exp_fp8` | `qwen4exp` | *(empty)* | Qwen4-Exp / Qwen3.8-Flash-Next: a bf16 checkpoint quantised at convert, or Qwen's FP8 release as it ships |
+| `glm5next` | `glm5next` | *(empty)* | GLM-5.3-Flash: four mHC residual streams, 34 KDA + 11 sparse MLA layers with a DSA indexer, 288 MXFP4 experts top-8, block-FP8 attention/dense at f32 scales, MTP drafter, vision tower |
+| `hyv3_bf16` | `hyv3` | *(empty)* | Tencent Hy-Memory-30B-A3B bf16; bf16/W4/W8A8/block-FP8/MXFP4 by recipe |
+| `hyv3_fp8` | `hyv3` | `fp8` | Hy-Memory-30B-A3B-FP8: imported per-tensor E4M3, one dense layer then sigmoid-routed MoE |
+| `hyv3_gguf` | `hyv3` | `gguf_ftype_15` | a llama.cpp Hy-Memory Q4_K_M GGUF, dequantised and recipe-converted |
+| `gemma4_bf16` | `gemma4` | `compressed-tensors` | Gemma 4 31B dense (and E2B/E4B at one rank): sliding-window + proportional-RoPE global attention, GELU, softcapped logits; int4/Q8 codes per 128; assistant (shared-KV), DFlash or DFlash2 drafter; vision |
+| `gemma4` | `gemma4` | *(empty)* | Gemma 4 text from Google's bf16 checkpoint (26B-A4B MoE, dense sizes) with z-lab's DFlash |
+| `gemma3` | `gemma3` | *(empty)* | Gemma 3 dense (TranslateGemma 27B): sliding-window + global attention, GemmaRMSNorm, GELU; bf16 or int8/MXFP4/asym-int4 linears; DFlash2 drafter |
+| `lfm2_bf16` | `lfm2` | *(empty)* | LiquidAI LFM2 / LFM2-VL, bf16, one rank |
+| `embgemma2` | `embeddinggemma2` | *(empty)* | EmbeddingGemma 2: bidirectional encoder with Gemma 4 vision tower and audio conformer, pooled |
+| `nomicbert` | `nomicbert` | *(empty)* | nomic-embed-text-v2-moe: bidirectional post-norm encoder, 8 experts top-2 on every second layer, mean-pooled |
+| `nemotron35_embed` | `mistral3` | *(empty)* | Nemotron 3.5 Embed 1B/8B and Rerank 8B: bf16 text/image retrieval with a Pixtral tower |
+| `qwenimage21` | `qwenimage21` | *(empty)* | Qwen-Image 2.1 / 2.1-Turbo image generator and editor (diffusers pipeline) |
+| `krea2` | `krea2` | *(empty)* | Krea 2 Raw/Turbo: Qwen3-VL-4B tapped layers, 12.9B single-stream DiT, Wan 2.1 VAE |
+| `sdxl` | `sdxl` | *(empty)* | Stable Diffusion XL and finetunes (Illustrious, NoobAI, Pony), single-file checkpoint |
+| `flux3` | `flux3` | *(empty)* | FLUX.3 [klein] and base: image generation and edits from up to ten pictures |
+| `flux3distilled` | `flux3distilled` | *(empty)* | the same plugin under the step-distilled release's own pipeline name |
+| `anima` | `anima` | *(empty)* | Anima Base/Aesthetic/Turbo from the release's three single files |
+| `qwen3asr` | `qwen3asr` | *(empty)* | Qwen3-ASR: Whisper-style audio encoder over a dense Qwen3 decoder, speech to text |
+| `minimaxmusic3` | `minimaxmusic3` | *(empty)* | MiniMax-Music3: Qwen3-8B frame model, depth decoder, flow-matching DiT, DAC vocoder |
+| `breeze` | `breeze` | *(empty)* | Breeze-TTS-2: T5Gemma 2 text encoder, Qwen3 backbone, 16-codebook depth decoder, Qwen3-TTS 12 Hz codec |
+| `fishomni` | `fishomni` | `bsgemm-moe` | Fish Audio drama-3: Qwen3.5-35B-A3B backbone, block-FP8 experts at f32 scales, 13-layer codebook tower, Fish S2 DAC/codec |
+| `minimaxh3` | `minimaxh3` | *(empty)* | MiniMax-H3 video with stereo sound: Qwen3-VL-32B text encoder, 33B DiT, tiled ViT decoder, BigVGAN audio |
+| `ltx2` | `ltx2` | *(empty)* | LTX-2.5 base/distilled: Gemma 4 LM + text connectors, duration head, audio-video DiT, causal video VAE, audio VAE |
+
+## Recipes of record, by family
+
+`$SRC` is a downloaded checkpoint; `$CALIB` is a calibration directory (see
+[[Radiance Conversion]]). Paths are shortened to the recipe name.
+
+| target | command |
+|---|---|
+| Qwen3.8-27B bf16 | `rad-convert $SRC -o q38-27b-bf16.rad --recipe data/recipes/q38-27b-bf16.recipe` |
+| Qwen3.8-27B block fp8 + DFlash2 | `rad-convert $SRC -o q38-27b-fp8.rad --recipe data/recipes/q38-27b-fp8block-df2.recipe --draft-model $DFLASH2 --set quantization_config.quant_method=fp8 --set quantization_config.fmt=e4m3 --set "quantization_config.weight_block_size=128 128" --set quantization_config.activation_scheme=dynamic` |
+| Qwen3.8-27B MXFP4 + DFlash2 | `rad-convert $SRC -o q38-27b-mxfp4.rad --recipe data/recipes/q38-27b-mxfp4-df2.recipe --draft-model $DFLASH2 --set quantization_config.quant_method=fp8 --set quantization_config.fmt=e4m3` |
+| Qwen3.8-27B w4a16 (an int4 release) | `rad-convert $SRC -o q38-27b-w4a16.rad --recipe data/recipes/q38-27b-w4a16.recipe` |
+| Qwen3.8-27B Quark AWQ-MXFP4 | `rad-convert $SRC -o q38-27b-quark.rad --recipe data/recipes/q38-27b-quark-mxfp4-df2.recipe --draft-model $DFLASH2` |
+| Qwen3.8-27B ParoQuant int4/int5/MXFP4 | `rad-convert $SRC -o q38-27b-paro.rad --recipe data/recipes/q38-27b-paro-mxfp4-df2.recipe --draft-model $DFLASH2` |
+| Qwen3.8-27B NVFP4 (compressed-tensors or ModelOpt) | `--recipe data/recipes/q38-27b-nvfp4-df2.recipe` or `q38-27b-modelopt-nvfp4.recipe`; every NVFP4 weight must take a rule (no kernel serves NVFP4) |
+| Qwen3.8-Flash-Next bf16 reference | `rad-convert $SRC -o flashnext-bf16.rad --max-ctx 4096` (no recipe; served with `--weights-disk-tier`) |
+| Qwen3.8-Flash-Next four-bit production | `CALIB=$CALIB rad-convert $SRC -o w4nl64-i8lm.rad --recipe data/recipes/qwen4exp-w4nl64-i8-hc8m.recipe` (GPTQ experts + int8 trunk) |
+| Qwen3.8-Flash-Next uncalibrated four-bit | `--recipe data/recipes/qwen4exp-w4.recipe` (rtn, no calibration) |
+| Flash-Next from an AutoRound int4 release | `--recipe data/recipes/qwen4exp-autoround-i8-hc8m.recipe` |
+| Gemma 4 31B w4a8 (no calibration) | `--recipe data/recipes/gemma4-31b-dense-bf16-w4a8.recipe --set quantization_config.quant_method=compressed-tensors --draft-model $ASSISTANT` |
+| Gemma 4 31B w4a8 + DFlash / DFlash2 | `gemma4-31b-dense-bf16-w4a8-df.recipe` / `-w4a8-df2.recipe`, same `--set` |
+| Gemma 4 31B Q8 grid | `gemma4-31b-dense-bf16-q8-df.recipe` |
+| Gemma 4 26B-A4B + DFlash | `gemma4-26b-a4b-df.recipe`; `-pad768` when the experts were padded to 768 (down may then be block fp8) |
+| Gemma 4 E4B (Nemotron Bolt) | `gemma4-e4b-bf16-df.recipe`, `-i8g-`, `-a4-`, `-w4a8-`, `-w8a8-` variants |
+| TranslateGemma 27B | `translategemma-27b-i8a8.recipe` (calibration container), then `CALIB=$CALIB translategemma-27b-mxfp4-gptq.recipe` (or `-df2`) |
+| GLM-5.3-Flash | `rad-convert $AMD_QUARK -o glm53.rad --recipe data/recipes/glm53-mxfp4.recipe` |
+| Hy-Memory-30B-A3B | `hy-memory-30b-a3b-bf16-fp8.recipe`, `-mxfp4`, `-w4`, `-w8`, `hy-memory-30b-a3b-fp8.recipe`, `hy-memory-30b-a3b-gguf-q4k-to-w4.recipe` |
+| MiniCPM5-2B | `minicpm5-2b-dspark.recipe` (or `-bf16-dspark`) with `--draft-model $DSPARK` |
+| Qwen3.6-35B-A3B | `q36-35b-a3b-fp8.recipe` |
+| Qwen-Image 2.1 | `qwen-image-2.1-fp8.recipe` (Turbo uses the same file) or `-bf16` |
+| Krea 2 | `krea-2-fp8.recipe`, `krea-2-bf16.recipe`, `krea-2-fp8-unscaled.recipe` for a plain-FP8 finetune file |
+| FLUX.3 [klein] | `flux-3-klein-fp8.recipe` |
+| SDXL / finetunes | `sdxl-bf16.recipe --tokenizer <openai/clip-vit-large-patch14>/tokenizer.json` |
+| Anima | `anima-fp8.recipe` (or `-bf16`) + `--add` ×2 + `--config` + `--tokenizer2` + `--arch anima --set anima.variant=aesthetic` |
+| LTX-2.5 | `ltx-2.5-fp8.recipe` (or `-bf16`); full DiT: symlink a folder whose `transformer/` is `transformer_full/` and `--set scheduler.use_dynamic_shifting=1 --set scheduler.shift_terminal=0.1` |
+| MiniMax-H3 / FastH3 | `minimax-h3-fp8.recipe`, `minimax-h3-w8a8.recipe`, `minimax-h3-curve-*` for curve-form checkpoints |
+| MiniMax-Music3 | `minimax-music3-bf16.recipe` |
+| Breeze-TTS-2 | `breeze-tts2-bf16.recipe` |
+| drama-3 | `drama3-fp8.recipe` |
+| nomic-embed-v2-moe | `nomic-embed-v2-moe-bf16.recipe` |
+| Unlimited-OCR | `unlimited-ocr-bf16.recipe` |
+| Qwen3-ASR | no container: serve the checkpoint directory as it ships |
+| EmbeddingGemma 2, Nemotron Embed/Rerank | serve the checkpoint, or convert bf16 with no recipe |
+
+A checkpoint served at its own precision needs no container at all:
+`radiance --model /srv/models/hf/Qwen/Qwen3.8-27B-FP8`. Convert when you want a tier only a container
+has (quantised encodings, a drafter, a second vocab) or a smaller file.
+
+## Family-specific traps
+
+- **`head_dim` is read, never derived.** On the Qwen hybrids it is 256 while `hidden_size /
+  num_attention_heads` is 213; a derived value silently breaks projection widths, KV page size and the
+  attention kernel's band.
+- **A fused projection shards by part, not by contiguous rows.** A tensor-parallel rank owns a slice
+  of *each* stacked projection in a fused `[gate|up]` or `[q|k|v|z]`; a contiguous slice would hand
+  rank 0 the whole gate, and the model would produce fluent wrong text. This is declared through
+  `RadWeightDecl::row_parts`, never inferred from the container.
+- **Gemma 4 26B-A4B:** the routed experts' `gate_up` may be block fp8, but `down` stays bf16 unless the
+  experts were padded to 768 — hence the `-pad768` recipe.
+- **GLM-5.3-Flash:** every weight is served as stored except the latent attention's small `kv_b`,
+  decoded to bf16. Its KV cache is bf16 only (the plugin refuses `fp8` at startup), and at 172 GiB it
+  needs `--tp 8`.
+- **TranslateGemma 27B:** the checkpoint's template takes structured turns naming two languages and
+  refuses the plain turn the server renders at startup. The container therefore serves a plain Gemma 3
+  template, and a client sends on `/v1/completions` the prompt the checkpoint's own template renders.
+- **ParoQuant containers made before 1.5.0 are refused at startup.** Convert again with the old
+  container as `--reuse`; every tensor that did not change is copied.
+- **SDXL carries no tokeniser.** Pass CLIP's `tokenizer.json` beside the single file. A Krea 2
+  finetune shipped as one DiT file in Krea's key names is rewritten into a directory whose
+  `transformer/` holds it (`scripts/krea2_native.py`), with the release's other parts copied in.
+- **Anima** ships as three single files: `--add` the text encoder and VAE, `--config` a Qwen3-0.6B-Base
+  directory (config + tokenizer), `--tokenizer2` ComfyUI's T5 tokenizer, `--arch anima`,
+  `--set anima.variant=…`.
+- **drama-3's codec is `codec.pth`**, a `torch.save` file read as data, never unpickled.
+- **Generators that load LoRA files declare their module aliases**; the checkpoint and a ComfyUI port
+  disagree in ways that load without error and generate garbage. The per-plugin alias table is in
+  `docs/ARCHITECTURES.md` ("Adapter key names of the generators").
+- **Licences travel with the checkpoint, not the engine.** Qwen-Image 2.1 is non-commercial;
+  MiniMax-H3's licence excludes the EU, UK, Republic of Korea and USA; drama-3's community licence
+  needs a commercial licence to serve speech to others, and cloning a real person needs consent;
+  Breeze-TTS-2 is research/non-commercial.
+
+## Related
+
+- [[Radiance Conversion]] — the tool, recipe language and quantisers used above.
+- [[Radiance Serve Options]] — the flags each family's compose file uses.
+- [[Radiance Architecture Plugins]] — writing a plugin for a family not in this table.
+- [[Quantization Production]] — the fleet's campaign and gates.

serverAppConfigs-docs/LLM/Radiance Operations.md

@@ -0,0 +1,240 @@
+---
+type: "howto"
+title: "Radiance Operations — running, reading, restarting"
+description: "Running a radiance server: the container contract, host prerequisites, readiness, ledgers and safe restarts."
+tags:
+  - llm
+  - radiance
+  - operations
+  - howto
+timestamp: 2026-10-30
+---
+
+# Radiance operations
+
+This is the engine-level runbook. The fleet's deployed shape, its unit and its deploy path live on
+[[frank-archon]]; the flags themselves on [[Radiance Serve Options]].
+
+## Install
+
+```sh
+git clone https://codeberg.org/StillDeadcode/radiance.git && cd radiance
+./build.sh                                   # configure, build for the cards present, run fast tests
+./build.sh --no-hip                          # host backend only
+cmake -S . -B build -G Ninja -DRAD_GPU_TARGETS='gfx1201'   # cards this machine lacks
+cmake --install build                        # after -DCMAKE_INSTALL_PREFIX=/opt/radiance at first configure
+```
+
+Needs Linux, git, a C++20 compiler, CMake 3.21+, Ninja; for device kernels ROCm 10.1 or 7.2
+(`hipcc`, `ROCM_PATH` if not `/opt/rocm`).
+
+- **Name the prefix at the first configure.** The default plugin path `<prefix>/share/radiance` is
+  compiled in from that configure; `cmake --install build --prefix DIR` alone gives a tree that looks
+  for its plugins elsewhere.
+- The prefix gets `bin/`, `include/radiance/`, `lib/cmake/radiance/`, `libexec/radiance/`,
+  `share/radiance/` (plugins and recipes) and `share/doc/radiance/`.
+- With ROCm 10.1 the build turns VOPD pairing off because its LLVM 24 pairs in heap-address order, so
+  two builds of one source can differ. The Docker image's compiler has the fix and keeps pairing
+  (faster fp8 prefill GEMMs); declare such a compiler with `-DRAD_HIP_VOPD_REPEATABLE=ON`. ROCm 7.2 is
+  unaffected.
+
+Run without Docker:
+
+```sh
+export RADIANCE_HOME=$PWD/build/radiance_home   # plugins; an install finds its own
+build/bin/radiance --model /srv/models/rad/minicpm5-2b-fp8.rad --port 8000 <flags>
+build/bin/radiance --model StillDeadcode/minicpm5-2b-fp8 --port 8000 <flags>
+```
+
+`--model` also takes a repo id: the server runs `scripts/hfget.sh` into `--download-dir`, serves the
+one `.rad` at the repo's top level, or the directory as a checkpoint when it holds none. A repo with
+several containers is refused, naming them. Every start checks the hub; a start that cannot reach it
+serves the copy a complete earlier fetch left, and a transfer that fails part-way stops startup.
+
+## Container
+
+The published image is `stilldeadcode/radiance`, kernels built for `gfx1201`. The host needs Docker
+and an `amdgpu` driver supporting ROCm 10.1 — **not** ROCm itself.
+
+```sh
+docker run -d --name radiance --network host --init --ulimit memlock=-1 \
+    --device /dev/kfd --device /dev/dri --security-opt seccomp=unconfined \
+    --group-add "$(getent group render | cut -d: -f3)" \
+    --group-add "$(getent group video | cut -d: -f3)" \
+    -v /srv/models/rad:/models:ro \
+    radiance --model /models/m.rad --tp 2 --host :: --port 8000
+```
+
+Each option is load-bearing:
+
+| option | why |
+|---|---|
+| `--device /dev/kfd --device /dev/dri`, `render`/`video` groups | the cards |
+| `--security-opt seccomp=unconfined` | Docker's default profile refuses io_uring (a disk-tier n-gram table's reads) and the NUMA calls the ROCm runtime makes for host memory |
+| `--init` | an init process as PID 1 forwards SIGTERM; the engine finishes its step and releases the cards |
+| `--ulimit memlock=-1` | a host-side pinned n-gram copy can be tens of GiB, far past Docker's default |
+| `--network host` | the server binds the host's addresses directly, IPv4 and IPv6, no NAT |
+
+The image is `FROM scratch` around the stripped install tree: busybox as the only shell, plus `curl`,
+`jq` and a CA bundle for `hfget.sh`; **no Python, no package manager**. Tools (`rad-convert`,
+`rad-info`, `rad-kbench`, `rad-tune`, `rad-schemas`, `rad-tokcheck`) are on `PATH`; recipes are in
+`/opt/radiance/share/radiance/recipes/`. Its entrypoint is `radiance`. It sets
+`GPU_MAX_ALLOC_PERCENT=100` and `GPU_MAX_HEAP_SIZE=100` because the HIP runtime's defaults cap one
+allocation and the heap below what the placement plan hands out.
+
+Build your own with `docker/build.sh` (`-t 'gfx1200;gfx1201'` for targets, `-j` compile jobs, `-E`
+to skip tests, `-s host` to build remotely). The context is `git archive` of the commit, so the image
+holds only committed content. The `rootfs` stage fails the build if any ELF cannot be resolved inside
+a chroot of the tree.
+
+Compose: one file per model plus `common.yaml`, copied to the serving host with an `.env` setting the
+image, models directory (mounted read-only at `/models`), API key, port, user and GPU groups. Every
+model listens on the same port, and most use both cards, so **run one at a time**. The models
+directory must be on a local filesystem that does direct I/O (ext4, XFS, btrfs); on btrfs with
+compression, a container copied into a `chattr +C` directory is stored uncompressed and direct reads
+work.
+
+A plugin of your own is mounted and put first on the search path; nothing in the image changes:
+
+```sh
+-v /srv/radiance-plugins:/plugins:ro -e RADIANCE_HOME=/plugins:/opt/radiance/share/radiance
+```
+
+## Host prerequisites (upstream's two-R9700 checklist)
+
+| item | setting |
+|---|---|
+| firmware | Above 4G Decoding on (Resizable BAR needs it); CSM off (pure UEFI) |
+| kernel command line | `iommu=pt amdgpu.ppfeaturemask=0xfff7ffff amdgpu.ras_enable=0` |
+| power/clock | OverDrive bit needed for `ppfeaturemask`; apply a power cap only while no process holds `/dev/kfd` |
+| memlock | `--ulimit memlock=-1` (container) / `LimitMEMLOCK=infinity` (systemd) when host memory is pinned |
+| two engines, one tmpfs table | `amdgpu.no_system_mem_limit=1`, or the second pin can be refused |
+| storage | a local fs with O_DIRECT for the container; a tmpfs sized for the table for `--ngram-placement ram=DIR` |
+
+Why each: `iommu=pt` keeps DMA identity so the cards can reach each other's memory and pinned host
+pages; `ppfeaturemask` enables the power controls; `ras_enable=0` turns VRAM ECC off (ECC reserves
+about 2 GiB a card). Verify peer access with a rate in `lspci`/the card tables — a missing cell means
+no peer access; check `iommu=pt` and the BAR size. The fleet's own boot-line law for Dankclaw
+(`iommu=pt`, tracked in the repo) is on [[dankclaw-iommu-translation-off]]; Dankserver's cards are a
+different box and its posture is on [[Dankserver System Specs]].
+
+## Readiness: the receipt, then `/health`
+
+Startup states every budget **before anything is allocated**, then loads. `/health` (and
+`/v1/health`, `/ping`) answers only once the load completes, so a bare health gate is honest. While it
+loads, expect 503 for minutes; do not restart over it. `--live` draws the live view (redirected, one
+`greppable key=value` line). `/diagnostics` returns the startup report as JSON — **same-machine-only
+without an API key** — with every option and its source and the hardware read again now.
+
+Read the receipt in this order:
+
+1. `elastic` — what the experts and KV cache share after non-expert weights.
+2. The split: `experts … / kv cache …`, and `unclaimed` if any.
+3. `kv: the pool backs N % of the worst case … M full-length sessions` — the referee for whether the
+   demand signal (`--max-model-len` × `--max-num-seqs`) is funded.
+4. `activation arena` and what each size lends; `prefill staging`; `hardware queues` and their VRAM.
+5. `host_pool` opened/used, `vram_weights`, the PLE or n-gram placement, the vision tower's placement.
+6. `device: the N cards reach each other's memory directly` or the failed pairs;
+   `expert prefetch on/off`; `chunk geometry`; `speculation:`; `chained launches`; `draft trees`.
+7. `card N settled at … free` — memory taken after the budget read comes out of headroom; a warning
+   names the `--gpu-headroom-mib` that clears it.
+
+A boot refusal prints the reason and exits nonzero; when it happens inside `podman run`, the
+container stops at once and the unit's restart policy re-runs the same refusal. A unit in
+`activating (auto-restart)` with a fast nonzero exit is therefore a **preflight/refusal or a boot
+refusal**, not an engine fault: read the journal line and the argv, not the GPU.
+
+## Lifecycle and restart
+
+- SIGTERM finishes the current step and releases the cards. In a container, `--init` forwards it;
+  `podman stop -t N`/`docker stop` is the graceful path.
+- A systemd unit should `exec podman wait` (or run in the foreground) so the script's lifetime is the
+  container's: a crash is a nonzero exit and the restart policy is real. Set `TimeoutStopSec` above
+  the stop grace, or systemd records `Failed with result timeout` on a healthy stop.
+- `Restart=always` rather than `on-failure` for a server whose front end can exit cleanly after a
+  worker fault.
+- Model load is slow (minutes cold). The deployer's LLM wait budget is on [[Deployment Flow]]; a
+  container restart loop during a legitimate load is worse than waiting.
+
+## Two engines on one set of cards
+
+- `ROCR_VISIBLE_DEVICES` picks each engine's cards explicitly.
+- `--weight-prefetch off`: a weight read beside a collective puts a queue switch into every decode
+  step when another server shares the cards.
+- `--ngram-placement ram=DIR`: two engines on separate card sets hold one table in host memory
+  (`DIR/ngram-<id>.tab` in a tmpfs, `MAP_SHARED`, pinned). The `<id>` hashes the container path, size,
+  mtime and the table's place, so a file never serves another model. The first engine fills it; a
+  second verifies samples and only pins it; the last to leave deletes it. The driver counts a shared
+  page against every process that pins it, so the pinned total is twice the table.
+- The engine prefers discrete cards (all of the first one's architecture), integrated only when
+  there is nothing else; on integrated graphics device memory is host memory and the budget leaves an
+  eighth of host RAM to the host.
+
+## Logs and diagnostics
+
+- `-v` / `-vv`; `RADIANCE_LOG` sets the level without a flag. `RADIANCE_LOG_STEPS=1` prints every
+  step's token count (how a chunk geometry is confirmed in practice).
+- The engine writes to stderr. Under a container, the journal holds only the container's own output;
+  a rootless `podman logs` covers only the current instance across recreations.
+- `/metrics` is Prometheus text: engine steps by kind and their seconds, decode/prefill rates,
+  prefill chunks held back, steps held by interleave, draft acceptance and depth, prompt-lookup
+  proposals held, KV bytes and the tokens they address, stream misses and bytes, prefix hits and
+  evictions, preemptions and queue depth. `/stats`,
+  `/graph`, `/sessions` back the dashboard. `scripts/statsdump.py` records `/stats` to JSONL with
+  time and gap, for a machine with no scraper (its key comes from `--api-key` or
+  `RADIANCE_API_KEY`).
+- `RADIANCE_PROFILE_EVERY`, `RADIANCE_DUMP_BUF`/`RADIANCE_DUMP_DIR`, `RADIANCE_OP_ORACLE`(+) and the
+  `RADIANCE_DEBUG_*` family are debug instruments; they are not deployment settings.
+
+## Verifying a deploy (fleet law)
+
+`deploy.sh` green is not proof of what is running. A leftover systemd drop-in or a stale container
+identity can serve a different model at a different shape while `/health` is 200. After deploying
+any recently-benched service:
+
+1. `systemctl show <svc> -p DropInPaths` — must be empty.
+2. `tr '\0' ' ' </proc/$(pidof radiance)/cmdline` (or the container's `Config.Cmd`) — the argv you
+   meant.
+3. The boot ledger — the shape you meant.
+4. A behavioural probe — `/v1/models` for the served id, one chat turn, one tool call, and a vision
+   probe with a solid colour and an exact expected word.
+
+`/v1/models` is an echo on some runtimes; on radiance it is authoritative for the alias, but the
+argv and the ledger are what prove the payload and shape. The three-pin revert discipline for a
+payload swap is on [[Model Swap and Rollback]]; the live deployment is [[frank-archon]].
+
+## Security posture
+
+- `--api-key` requires a bearer token or `X-Api-Key` on every route except `/health`, `/v1/health`,
+  `/ping` and the dashboard; the key is a command-line argument, so it is readable in
+  `/proc/<pid>/cmdline` by any local user of that host. On a single-tenant inference box the network
+  boundary is the real control; do not run one where untrusted local users share the host.
+- `/diagnostics` is same-machine-only without a key, because it prints the whole configuration.
+- `--security-opt seccomp=unconfined` is a genuine widening, required by io_uring and the ROCm
+  runtime's NUMA calls; it belongs on a dedicated model host, not a general-purpose one.
+- Mount model directories read-only; a container with `/models` writable adds nothing a serve needs.
+- A control vector can remove refusal behaviour. Loading one is a safety decision, not a
+  performance one; the flag is `--control-vector`, and `--control-vector disabled` is the default.
+
+## Failure signatures at startup
+
+| what you see | what it is |
+|---|---|
+| unknown/retired flag, or a flag refused by name | a boot refusal — the pin does not have it; change the trio, not the invocation |
+| "no plugin claims `<arch>` at `<quant>`; served at: …" | the (id, descriptor) pair has no plugin; fix the descriptor with `--set`/convert |
+| "reports ABI N, engine M" | rebuild the plugin against this engine |
+| budget refusal / `host_pool` short | the shape does not fit this card/host; lower demand or pin more host RAM |
+| `--ngram-placement ram` refuses | memlock short; raise it or use `auto`/`disk` |
+| `--kv-cache-dtype fp8_v4`/`k6_v4` refused, plugin named | no kernels for this geometry |
+| `--p2p on` refuses, pairs named | the driver has no peer access; use `auto`/`off` |
+| first launch of a never-run kernel faults | too little `--gpu-headroom-mib` for its code object |
+| unit loop, no container, no engine log | a preflight refusal (TTM/memlock/digest/model) re-run by `Restart=always` |
+
+## Related
+
+- [[Radiance Serve Options]] — every knob and its law.
+- [[Radiance HTTP API]] — readiness and the wire.
+- [[Radiance Tools]] — how to corroborate a receipt.
+- [[frank-archon]] — the fleet's radiance unit, guard chain and Eras.
+- [[Model Swap and Rollback]] — the swap discipline this page's verification serves.
+- [[Deployment Flow]] — how a committed change reaches the host.

serverAppConfigs-docs/LLM/Radiance Plugins.md

@@ -0,0 +1,371 @@
+---
+type: "howto"
+title: "Radiance Plugins — kernel libraries and quantisers"
+description: "Writing radiance kernel and quantiser plugins: the ABI, kernel rows, hooks, encodings, and the build path."
+tags:
+  - llm
+  - radiance
+  - plugins
+  - howto
+timestamp: 2026-10-30
+---
+
+# Radiance plugins — kernels and quantisers
+
+Three kinds of `.so` extend the engine, each exporting a few C symbols and linking **nothing of
+radiance's** (they compile against the installed `abi/` headers):
+
+| kind | value | contract |
+|---|---|---|
+| kernel library | `RAD_PLUGIN_KERNEL` (1) | ops for some hardware: a card family, the CPU, an oracle |
+| architecture | `RAD_PLUGIN_ARCH` (2) | one model family at one weight format — see [[Radiance Architecture Plugins]] |
+| quantiser | `RAD_PLUGIN_QUANT` (3) | weights → encoding planes, for `rad-convert` |
+
+In-tree plugins use the same interface a third party does: each builds with `rad_add_plugin` and is
+found and ranked by what it declares, never by its name. `ctest -L oot` rebuilds every bundled
+plugin out of tree against an install.
+
+## Identity and ABI
+
+```c
+#include "rad_abi.h"   /* required; pulls rad_types.h and rad_encoding.h */
+
+uint32_t rad_plugin_abi_version(void) { return RAD_ABI_VERSION; }
+
+const RadPluginInfo* rad_plugin_info(void) {
+    static const RadPluginInfo i = {
+        RAD_PLUGIN_KERNEL, "mykernels", "0.1", "a GEMM or two", "gfx1100",
+    };
+    return &i;
+}
+```
+
+- `RAD_ABI_VERSION` (31 at this commit) is bumped on any shape change in a header a plugin compiles
+  against. Return the macro; a mismatch is refused at load with both numbers printed.
+- `build_target` is printed, not trusted: the targets the library was built for (`"gfx1100"`,
+  `"gfx11-generic"`, several separated by spaces/commas), `"host"`, or `""` for portable. The engine
+  checks the library's actual code objects instead (§"A library for a new card").
+- Optional exports, resolved by exact name: `rad_plugin_open` (return `RAD_E_UNSUPPORTED` to decline
+  this machine and unload), `rad_plugin_close`, `rad_kernel_concurrent`, `rad_kernel_collective`,
+  `rad_plugin_reference`, `rad_kernel_reference`, `rad_kernel_chain`.
+
+## What a kernel library exports
+
+```c
+int                rad_kernel_schema_count(void);
+const RadOpSchema* rad_kernel_schema_at(int i);
+int                rad_kernel_count(void);
+const RadKernelInfo* rad_kernel_at(int i);
+```
+
+A row declares one op implementation and what it can serve:
+
+```c
+static const RadConstraint c_gemm[] = {
+    RAD_CLE("M", 64),              /* M <= 64            */
+    RAD_CDIV("K", 128),            /* K divisible by 128 */
+    RAD_CIN("dtype", "fp8a8 w4a8") /* dtype in that set  */
+};
+
+static const RadKernelInfo k[] = {{
+    .name = "mygemm_m64", .op = "gemm_nt_q", .family = "gemm",
+    .computes = "row-major A times column-major B, fp8 in, bf16 out",
+    .shape    = "M<=64, K%128==0",
+    .dtypes   = "fp8a8, w4a8",
+    .domain   = RAD_DOMAIN_DEVICE,
+    .priority = 50,
+    .constraints = c_gemm, .n_constraints = 3,
+    .launch = my_launch,
+}};
+```
+
+- Operators: `RAD_CEQ`, `RAD_CLE`, `RAD_CGE`, `RAD_CDIV`, `RAD_CIN`. The engine takes bucket
+  boundaries from `LE`/`GE`/`EQ` and resolves once per band; `DIV` and `IN` are not intervals.
+- **Declare only the band you serve.** The engine believes the claim; a kernel that says `M <= 2048`
+  but is correct to 64 is wrong at concurrency.
+- `device.*` keys are matched against what the engine measured about the ranks' cards, not against
+  op parameters: `RAD_CEQ(RAD_FACT_P2P, 1)` means the kernel touches another card's memory. libr4d
+  pairs every collective with a `_host` twin requiring `device.p2p == 0`, so exactly one matches any
+  machine.
+- **Priority breaks ties within one plugin**, highest first, then declaration order. **Between
+  plugins the hierarchy is absolute**: the first library in `--kernels` order with any matching row
+  supplies the op, however specialised a later library's row is. That makes a user's library an
+  override, not a competitor.
+- A derivative parameter (`RAD_DERIVED`) is supplied by the kernel; a constraint on a key the
+  geometry omits does not hold during matching, and the winner's `RAD_C_EQ` writes it into the
+  resolved geometry. `RAD_PROLE_SEQ_CHUNK` marks the parameter the scheduler must cut steps on;
+  `RAD_PROLE_CAPACITY` marks a count the kernel only bounds itself by.
+
+## Operands and extents
+
+`RadArgs.t` is positional in the schema's order; each `RadTensor` carries the extent the caller
+narrowed it to, usually smaller than the buffer. Size loops and grids from those extents, never from
+a declared maximum. Where an op takes `cu_seqlens`, derive the sequence count from its own length
+(`n = len - 1`) and each sequence's rows from `cu[n]` and `cu[n+1]`.
+
+An absent optional operand is a **null `RadTensor.data`**, not a shorter operand count.
+`rad_arg_in(a, i)` fetches one correctly; `rad_tensor_rows`, `rad_tensor_lastdim`,
+`rad_tensor_row_pitch`, `rad_args_geti_or`, `rad_args_getf_or`, `rad_args_gets_or` are the other
+everyday helpers in `rad_plugin.h`.
+
+## Optional hooks
+
+| hook | meaning |
+|---|---|
+| `init` / `fini` | per-resolved-instance state; `RadArgs.instance` hands it back. `init` runs once per (op, band, domain) after selection and before load |
+| `scratch` | bytes of scratch one launch needs; sized at declare from the band's upper bound |
+| `layout` / `relayout` | the kernel stores this weight in its own order, and how the loader produces it from the canonical planes |
+| `unrelayout` | the inverse, so a tool holding stored bytes can compare against a dense oracle |
+| `opd_shape` | size and fill every operand, so a tool can call the kernel cold |
+| `replaces` | the chain of conventional ops this kernel computes in one launch |
+| `tunables` / `tune_valid` | the instantiation axes, and which combinations are legal at a shape |
+
+`replaces` must be **exact** and is compared byte for byte against the chain — a kernel that is only
+close publishes nothing. `tunables` declares axes and legal values, not named variants; the core
+takes the cross product, and `deflt` must be launchable (the loader refuses the plugin otherwise).
+`rad-tune` measures survivors and writes the cache the selector reads; the chosen point arrives as
+ordinary parameters (`rad_args_geti("sk")`), with no variant index.
+
+`layout` answers: `RAD_OK` (own form, `relayout` will produce it), `RAD_E_UNSUPPORTED` (stored as
+is — the common case, and the only form that can be read in place from the file/mapped tier),
+`RAD_E_DTYPE` (this encoding is not mine — a declare-time error), `RAD_E_SHAPE`, or `RAD_LAYOUT_ANY`
+(reads the bytes without interpreting them; claims no storage). A weight only `LAYOUT_ANY` readers
+touch is stored as it is.
+
+## What a launch may do
+
+On a card the `RadStream` is **the engine's own AQL queue**, not a `hipStream_t`. The engine binary
+defines and serves these HIP entry points:
+
+- `hipLaunchKernel` (and `<<<>>>`, `hipLaunchKernelGGL`);
+- `hipMemcpyAsync`, `hipMemcpy2DAsync`, `hipMemsetAsync`;
+- `hipStreamSynchronize` (not from a launch — a launch must not synchronise);
+- `hipMemset` (runs on the engine's queue and waits);
+- `hipGetLastError` / `hipPeekAtLastError`, with HIP's semantics.
+
+**Nothing else in HIP takes an engine stream.** `hipModuleLaunchKernel`, `hipExtLaunchKernel`,
+cooperative launches, `hipLaunchHostFunc`, and libraries that launch through them (rocBLAS,
+hipBLASLt, MIOpen, hipRTC) would hand HIP a queue it does not know. Header-only device code
+(Composable Kernel templates, rocPRIM primitives) compiled into your `.hip` works. `hipEventRecord`
+and `hipStreamWaitEvent` on an engine stream are refused by name.
+
+The engine also reads the code object, which limits a kernel to: **no hostcall** (`printf`/`assert`
+are refused at the first launch), **no dynamic stack**, **≤ 768 bytes of kernel arguments**, **≤ 64
+KiB static+dynamic LDS per dispatch**, and an **uncompressed fat binary**. Device memory a kernel
+needs comes from the `scratch` hook, never `hipMalloc` — the engine's allocator knows what is left.
+
+`rad_plugin.h`'s conversions define the result: the reference contract is f32 accumulation, IEEE
+round-to-nearest-even, **no FMA and no reassociation**. A plugin whose bf16 narrowing truncates
+cannot be checked against the reference. `rad_load_f32`/`rad_store_f32`/`rad_load_bits`/
+`rad_store_bits` read and write any core dtype from a plane.
+
+## Colours of agreement
+
+- `rad_kernel_concurrent(i)` — kernel `i` touches only its operands and scratch, so independent
+  launches may overlap.
+- `rad_kernel_collective(i)` — it exchanges data with other ranks; the engine keeps bulk link
+  traffic away. `2` promises more: the launch ends the hold itself once the other ranks' bytes have
+  landed, through a `link_gate` word it reaches at system scope; returning `RAD_LINK_ENDED` says it
+  did.
+- `rad_kernel_chain(i)` — the kernel can wait in its own prologue instead of behind the packet
+  processor's barrier bit. Chained forms are recognised by `r4dchain` in the name and use a
+  `RAD_CHAIN_REC_BYTES` record at `RAD_CHAIN_KARG_OFF`.
+- `rad_plugin_reference` / `rad_kernel_reference` — an oracle library (or an oracle row beside a
+  device row). A reference library ranks below every unnamed library, is refused for serving unless
+  `--debug-accept-reference-kernels`, and is what `rad-kbench` checks against.
+
+## Defining a new op
+
+Only when `docs/OPS.md` has no name for it. The vocabulary document owns the rules: what a schema
+must declare, and that **the first plugin in hierarchy order to declare an op fixes its schema** — a
+later plugin declaring it differently is refused at load, both named. `ctest -R ops_doc` checks the
+document against the registries, so the rules live there only. An op in `docs/OPS.md` is describable
+by `libref`, and therefore checkable, even when your kernel cannot describe itself; an op you invent
+runs only on a library that implements it.
+
+A schema is positional data, not code:
+
+```c
+static const RadParamSpec params[] = {
+    {"M", RAD_P_INT, RAD_REQUIRED, RAD_PROLE_NONE},   /* the ranged extent */
+    {"n", RAD_P_INT, RAD_REQUIRED, RAD_PROLE_NONE},
+    {"eps", RAD_P_F64, RAD_REQUIRED, RAD_PROLE_NONE},
+    {"dtype", RAD_P_STR, RAD_REQUIRED, RAD_PROLE_NONE},
+};
+static const RadOperandSpec opds[] = {
+    {"x", RAD_OPD_IN, 0}, {"w", RAD_OPD_WEIGHT, 1}, {"y", RAD_OPD_OUT, 0},
+};
+static const RadOpSchema schemas[] = {
+    {"my_op", params, 4, opds, 3, "what it computes, for a human"},
+};
+```
+
+`rad_kernel_schema_count/at` return this table. `RAD_REQUIRED`/`RAD_OPTIONAL`/`RAD_DERIVED` and the
+operand roles (`IN`, `OUT`, `INOUT`, `WEIGHT`, `WTAB`) are the whole vocabulary; the kernel's
+constraints and launch do the rest.
+
+## Quantiser plugins
+
+`abi/rad_quant.h` is the whole contract:
+
+```c
+int                     rad_quant_count(void);
+const RadQuantizerInfo* rad_quant_at(int i);
+
+typedef struct RadQuantizerInfo {
+    const char*           name;       /* what a recipe names */
+    const char*           doc;
+    const RadQuantOption* options;    /* key, RAD_P_INT|F64|STR, default text, doc */
+    int                   n_options;
+    RadQuantEncodingFn    encoding;   /* the encoding this quantiser would write */
+    RadQuantRowBlockFn    row_block;  /* rows per quantize() call; 0 = whole weight */
+    RadQuantizeFn         quantize;   /* logical rows -> planes */
+    RadQuantDecodeFn      decode;     /* optional, only for a scheme of its own */
+} RadQuantizerInfo;
+```
+
+The flow for each weight a recipe rule picks: ask `encoding` (given name, shape, and — at declare —
+no movement group), let the architecture declare against that encoding, then hand the weight over in
+row blocks, so a 51 GB table streams. Other calibration data is the quantiser's own option
+(`calib=DIR`); the output is **canonical planes only**, so every kernel that accepts the encoding and
+the core's decoder can read it — that is how the converter measures error without knowing the method.
+
+A quantiser's own scheme must be decoded by its `decode` hook (the oracle, `rad-kbench` and the
+error report call it), and must return `RAD_E_UNSUPPORTED` for an encoding it did not write. Serving a
+new encoding also needs a kernel library whose `layout` accepts it and an architecture that declares
+the weight at it — a new format is a new kernel row plus a plugin, never a converter-only change.
+
+Two plugins naming the same quantiser are refused, both named.
+
+### The encoding model (`abi/rad_encoding.h`)
+
+An encoding is the code type, an optional codebook, the grouping of scales/zero points/minimums, and
+a transform. It is **data, not an enum**, so a plugin can invent one. The core can size, slice and
+place any encoding from the struct alone, and decodes the `plain` and `affine` schemes itself:
+
+```
+q  = code                                   codes TILED, block [1x1], or [1xV] with a grid
+q  = grid[code][k]                          grid TABLE [entries x V]
+q  = table[code]                            table TABLE [1 x entries]
+q  = -q if the sign bit is set              signs TILED u1
+q  = q - zero                               zero TILED
+v  = q * scale * scale.1 * ...              every scale level
+v  = v - min * min.1 * ...                  a minimum in the value's domain
+w  = v . T^-1                               the transform, per row over columns
+```
+
+So NVFP4 is fp4 codes, e4m3 scale per 16, f32 `scale.1` per tensor; Q4_K is u4 codes, u6 scale and
+u6 min per 32 each times an f16 per 256; IQ2_XXS is u8 codes per 8 into a 256×8 grid, a sign bit per
+value and two scale levels.
+
+- Planes are **TILED** (cover the weight, element `(i,j)` covering a `[br,bc]` block; `0` extent means
+  the whole extent) or **TABLE** (a codebook, grid, rotation parameter; every slice carries it).
+- Sub-byte elements are a bit stream along the row, lowest bits first; every row starts on a byte.
+- Transforms: `""`, `fwhtN` (Hadamard before quantisation; the `1/N` lives in the stored scale),
+  `perm` (GPTQ act-order, a TABLE `t.perm`), `givens` (ParoQuant rotations: `t.pairs`, `t.angle`,
+  optional `t.scale`, per-part `t.part`, composed `t.dense`).
+
+A kernel only *rearranges* planes (`relayout`); choosing numbers is the quantiser's job. That split
+is why a container never has to be reconverted when a kernel library changes its layout.
+
+## Build, install, search path
+
+```cmake
+cmake_minimum_required(VERSION 3.21)
+project(mykernels LANGUAGES CXX HIP)     # CXX alone for a host-only plugin
+find_package(radiance REQUIRED)
+rad_plugin_defaults()                    # C++20, optimising build unless asked otherwise
+rad_add_plugin(mykernels kernels SOURCES rows.cpp HIP_SOURCES kernels.hip)
+```
+
+The kind is `kernels`, `architectures` or `quantizers` — the directory the engine loads it from.
+Only `HIP_SOURCES` compile as HIP: for `RAD_GPU_TARGETS` or a `GPU_TARGETS` list, with `-DNDEBUG`
+outside Debug, uncompressed, against the HIP runtime the installed radiance found. `find_package`
+also sets `RADIANCE_ABI_VERSION` (to fail at configure time rather than at `dlopen`),
+`RADIANCE_HOME`, `RADIANCE_KERNELS_DIR`, `RADIANCE_ARCHITECTURES_DIR`,
+`RADIANCE_QUANTIZERS_DIR`. `cmake --install` puts the plugin under the home of the radiance you built
+against; `RAD_PLUGIN_INSTALL_HOME` overrides that.
+
+**`$RADIANCE_HOME` is a search path**: homes separated by `:`, the first winning; each may hold
+`kernels/`, `architectures/`, `quantizers/`, and an earlier home's same-name file shadows a later
+one. Your plugins go ahead of an installation without copying it:
+
+```sh
+RADIANCE_HOME=$HOME/radiance-plugins:/opt/radiance/share/radiance radiance --model m.rad
+# in the container:
+-v /my/plugins:/plugins -e RADIANCE_HOME=/plugins:/opt/radiance/share/radiance
+```
+
+The **kernel hierarchy** is `--kernels` (or `$RADIANCE_KERNELS`): `--kernels mykernels,libr4d` puts
+yours before libr4d. An entry matches a plugin's name or its file (`libr4d`, `r4d`, `libr4d.so`, a
+path). An entry matching nothing is warned about by name. With no list, every library found loads,
+reference last.
+
+## A library for a new card
+
+The engine's own device code is compiled for every AMD family's generic target
+(`gfx9-generic`, `gfx9-4-generic`, `gfx10-1-generic`, `gfx10-3-generic`, `gfx11-generic`,
+`gfx12-generic`) beside the build's targets, so one engine binary starts anywhere. What it computes
+there depends on the installed kernel library.
+
+- Configure with `-DRAD_GPU_TARGETS=gfx942` (or `gfx11-generic`). At load the engine asks the runtime
+  which targets the card accepts, in order (chip, then family: `gfx1036`, then `gfx10-3-generic`),
+  with the card's modes (`gfx942:sramecc+:xnack-`), and takes the best fit in the fat binary.
+- **A library with nothing for this card is left out, not failed**: its device rows leave selection,
+  its host rows stay, and the log says what it carries versus what the card runs. One installation
+  can carry a library per card. `rad_plugin_open` may decline explicitly with `RAD_E_UNSUPPORTED`.
+- The engine uses discrete cards if any exist (all of the first one's architecture), integrated only
+  when there is nothing else; `ROCR_VISIBLE_DEVICES` chooses explicitly. On integrated graphics,
+  device memory is host memory and the budget leaves an eighth of host RAM to the host.
+- **fp8 bytes are OCP E4M3** (`e4m3fn`, bias 7, max 448) wherever the engine reads them: container
+  weights, an fp8 KV cache, activations. A card whose fp8 is `e4m3fnuz` (gfx942) converts in its
+  `relayout` hook (the codes keep their bits except `0x80` and decode to half the value, so the scale
+  doubles) and keeps activations/KV in OCP bytes, or converts at its own boundaries.
+- **Tensor parallelism is ops you supply.** `all_reduce`, `all_gather` and the fused collectives are
+  rows like any other; the engine inserts none and assumes no link.
+
+## Checking your work
+
+```sh
+rad-kbench --kernels mylib,libr4d,libref --report mylib.md --bench
+```
+
+- It replays a recorded fixture of `libref`'s answers over real model geometries
+  (`$RADIANCE_HOME/kernels.rkb`, plus `kernels_moe.rkb`), running every library with a row for a
+  case. Correctness uses sampled values **and** the full-tensor norm **and** seeded projections;
+  completeness is the op-by-library matrix; speed is timed on the same bytes.
+- Red zones: every device allocation carries a poisoned 256-byte margin, verified after the launch.
+- Fusion: a row with `replaces` is run against its named chain, out of your library, compared with
+  `memcmp`.
+- A skip is named, not failed — usually a missing `opd_shape`. `--strict` also fails a stale fixture.
+- `rad-avx-check --ref libref.so --avx mylib.so` checks a **host** library without a card, op by op,
+  at every ISA level and with parallel regions forced onto several threads (for a library exporting
+  `avx_isa_force`/`avx_par_force`).
+- `rad-schemas mylib.so libref.so` prints every schema disagreement at once (it exits 1 on any), and
+  works on a plugin the engine cannot load.
+- `--debug-selection` prints which row took each band and which constraint refused each other
+  candidate; `/graph` on a running server shows the declared graph and which declarations the run
+  phase issued.
+- `rad-info --plugins` lists every plugin on the search path as the engine loads it: kind, version,
+  declarations, reference status, whether its device code runs on this card, and its file.
+
+## A worked example to copy
+
+`plugin/spechub/` is the smallest real kernel plugin in the tree: two ops (`mtp_couple_propose`,
+`mtp_couple_verify`), each with a device row and a host reference row. Read it in this order:
+
+- `exports.cpp` — the whole export block: info, ABI, schema count/at, kernel count/at, plus
+  `rad_kernel_reference` classifying device rows 0 and host rows 1 so the reference gate does not
+  refuse the device rows.
+- `spechub.cpp` — the rows, their constraints and their argument readers.
+- `spechub.hip` / `ref_spechub.cpp` — the device launch and the host oracle.
+- `CMakeLists.txt` — the `rad_add_plugin` call, the optional `enable_language(HIP)`, and
+  `-ffp-contract=off` because an FMA in the ordered FP64 mass sums would move a token off the host
+  reference.
+
+## Related
+
+- [[Radiance Architecture Plugins]] — the third plugin kind, one level up.
+- [[Radiance Conversion]] — how a quantiser is driven by a recipe.
+- [[Radiance Tools]] — `rad-kbench`, `rad-tune`, `rad-schemas`, `rad-avx-check` in full.
+- [[Radiance Serve Options]] — `--kernels`, `--radiance-home`, the tuning aids.

serverAppConfigs-docs/LLM/Radiance Serve Options.md

@@ -0,0 +1,240 @@
+---
+type: "reference"
+title: "Radiance Serve Options — every flag and the law behind it"
+description: "Every radiance serve flag grouped by task, with its default and the law that decides when to move it."
+tags:
+  - llm
+  - radiance
+  - reference
+  - serving
+timestamp: 2026-10-30
+---
+
+# Radiance serve options
+
+`radiance --help` prints this table. Every numeric option takes a whole number (not `12k`, not
+`auto`, unless the option names those); a malformed value exits 2. Upstream `main` at 2026-10-30;
+the fleet is pinned older — see the pin note on [[Radiance Engine]].
+
+The one thing to read before any table: **the engine derives the VRAM split; no flag sets either
+pool.** The startup receipt prints every term before anything is allocated.
+
+## Model, plugins, container
+
+| flag | default | meaning |
+|---|---|---|
+| `--model PATH` | required | a `.rad` container, a checkpoint served as it stands, or a Hugging Face repo id (`owner/name`) fetched before startup |
+| `--download-dir DIR` | `$XDG_CACHE_HOME/radiance/hf`, else `~/.cache/radiance/hf` | where a repo id is fetched |
+| `--radiance-home DIRS` | compiled prefix | plugin homes, `:`-separated, first wins |
+| `--kernels LIST` | every library found | kernel libraries, `,`/`:` separated, first wins; a library not named is not loaded |
+| `--served-model-name NAME` | container's name | the id `/v1/models` lists and responses carry; a request may name any model |
+| `--override-chat-template PATH` | container's | a Jinja template served instead; an unreadable or empty file stops startup |
+| `--generation-config PATH` | none | sampler defaults from a `generation_config.json` (else `<model>.generation.json`, then the container's `generation.*` metadata) |
+| `--control-vector off\|on\|disabled` | `disabled` | load the GGUF control vector found for the model and start requests off/on; `disabled` declares no projection and refuses a request that asks |
+| `--api-key KEY` | none | require `Authorization: Bearer KEY` or `X-Api-Key: KEY`, except `/health`, `/v1/health`, `/ping`, the dashboard |
+| `--mm-max-patches N` | `auto` (16384 with a vision tower) | patches one encoder pass carries: the largest image, 16×16 px a patch (14×14 on GLM-5.3). `0` serves text only and keeps the tower off the card |
+
+## Batch shape, context, scheduler
+
+| flag | default | meaning |
+|---|---|---|
+| `--max-num-batched-tokens N` | 2048 | the largest token count one step may carry; on MoE the knob that sets prefill chunk size, and the activation arena's size |
+| `--max-num-seqs N` | 256 | the most sequences one step may carry; the cache's **demand**, never its funding |
+| `--max-model-len N` | 0 = model's training context | context bound per request. Longer is a 400, never a truncation. Reserves nothing per request; it is the worst case the pool is sized against |
+| `--busy-prefill-chunk auto\|off\|N` | `auto` | a prefill chunk's share of a step that has other work: `auto` leaves decode rows and waiting requests room, `off` takes the whole step, `N` caps it |
+| `--prefill-interleave N` | 0 (off) | decode-only steps after each chunk taken beside decoders, at most 64; buys decoder cadence with the prompt's TTFT |
+| `--prefill-span N` | 1 | checkpoint intervals a prefill **alone in its step** may carry, from its second interval; needs `--max-num-batched-tokens` of N intervals. Every other step is planned at the shared budget; checkpoints inside a span are not written |
+| `--preempt-wait-ms N` | 5000 | how long a preempted prefill waits for the whole-recompute gate before reclaiming blocks and preempting lower-priority work; `0` falls back next step |
+| `--rope-scaling SPEC` | the checkpoint's | `yarn:FACTOR[:ORIG]` writes static YaRN over the container's own rotary keys with no reconvert; `none` takes a checkpoint's YaRN off |
+| `--max-queued-requests N` | 8 × `--max-num-seqs` | admitted, unfinished requests; past it the answer is 429 |
+| `--retry-after S` | 1 | the `Retry-After` a 429 carries |
+
+## Memory and placement
+
+| flag | default | meaning |
+|---|---|---|
+| `--placement MODE` | `auto` | `all_vram`, `layer_offload`, `expert_tiered`, `auto` |
+| `--gpu-headroom-mib N` | 96 | VRAM left unclaimed on every card; too little makes the first launch of a never-run kernel fail inside HIP |
+| `--host-pool-mib N` | 0 | pinned host tier for weights, **per rank**; also the KV lever under expert tiering (below) |
+| `--host-headroom-mib N` | 4096 | host RAM left free when startup pins memory; a snapshot store's floor takes the shortfall back only down to 1 GiB |
+| `--kv-cache-dtype T` | `bf16` | `bf16`, `fp8`, and on Qwen3.8-27B/-Flash-Next `fp8_v4` (V int4) or `k6_v4` (K int6 + V int4). A real quantisation; GLM-5.3 is bf16-only |
+| `--state-cache-dtype T` | `f32` | `f32` or `f16`: the recurrent delta-net state cache; f16 halves state pool, checkpoints and traffic and is lossy — `qwen35` plugins only |
+| `--attn-prefill-fp8 MODE` | `off` | `off`, `qk`, `pv`, `both`: prefill attention products in E4M3 over an `fp8`/`fp8_v4`/`k6_v4` cache; lossy, Qwen3.8-27B only where kernels exist |
+| `--kv-resident N` | 0 (off) | stream the sparse attention group's KV from pinned host RAM, keeping an N-token ring on the card. Raised to 32768 and to one decode step's selection; ignored with a warning where unsupported |
+| `--embedding-placement ram\|vram` | `ram` | token embedding pinned in host memory and read in place, or on the card. A tied head or a host that cannot pin keeps it on the card; an explicit `ram` refuses instead |
+| `--vision-tower-placement vram\|ram` | `vram` | encoder towers on the card, or pinned in host memory and read once per image, giving their VRAM to the KV cache |
+| `--ngram-placement auto\|vram\|ram\|disk` | `auto` | where an n-gram embedding table lives: all of it on every card, a pinned host copy the card reads, or rows from the container through a row cache. `auto` takes the first that fits. `ram=DIR` makes the pinned copy a tmpfs file shared by two engines |
+| `--weights-disk-tier` | off | read weights that fit neither VRAM nor the host pool from the container with O_DIRECT; every step is paced by the drive |
+| `--row-prefetch on\|off` | on | ask for a host gather's pages before its kernel; output-neutral |
+| `--mapped-rewarm on\|off` | on | at startup, re-ask the page cache for a mapped table's pages an earlier run left behind |
+| `--deterministic` | off | pin placement to the loader's plan so the resident set cannot move under the heat engine; it does **not** make output bit-identical across batch positions |
+
+## KV cache, host tier, checkpoints
+
+| flag | default | meaning |
+|---|---|---|
+| `--prefix-cache-mib N` | 0 (off) | copy finished conversations' KV to host memory so a return restores instead of re-prefilling; LRU when full; off while a KV group streams |
+| `--prefix-cache-ttl S` | 3600 | seconds an unused cached conversation is kept; `0` keeps it until memory is wanted |
+| `--no-prefix-cache` | cache on | disable prefix caching |
+| `--checkpoint-interval N` | 2048 | linear-attention state checkpoint interval, in tokens; also clamps a hybrid model's prefill chunk |
+| `--checkpoint-slots N` | `auto` | snapshots held **in total**, in pinned host memory; `auto` = per-policy need for each sequence within an eighth of host RAM; `0` keeps none |
+| `--checkpoint-policy NAME` | `geometric-backoff` | also `tip-only`, `keep-all`; retention only — writes happen at every interval unless a span skips them |
+| `--prefix-resume exact\|nearest` | `exact` | `exact` resumes only where a cold prefill starts a chunk, so a hit answers bit-identically; `nearest` takes the deepest snapshot, up to a chunk less re-prefilled |
+
+## Speculation
+
+| flag | default | meaning |
+|---|---|---|
+| `--num-speculative-tokens N` | `auto` (the drafter's own point; 0 with no head) | speculative window; 0 disables. The ceiling under `adaptive` |
+| `--spec-ngram K` | 0 (off) | prompt lookup alone, up to K drafts a step from the sequence's own n-grams; leaves a container's draft head out; sets the window to K |
+| `--spec-ngram-min N` / `--spec-ngram-max N` | 2 / 4 (max 15) | pattern lengths prompt lookup tries, longest first |
+| `--draft-window fixed\|adaptive` | `fixed` | `adaptive` follows each sequence's acceptance and the measured step cost |
+| `--draft-short-from N` | `auto` | a block drafter that declares a shorter block runs it from N decoding sequences; `0` never; `auto` = never for a chain, from 2 for a tree |
+| `--draft-coupling off\|on` | off | sample each proposal from the drafter's own shortlist and couple the target's draw to it; same output law, different tokens for a fixed seed, pays at high temperature |
+| `--draft-model PATH` | — | **not implemented** at serve: merge the drafter at convert time (`rad-convert --draft-model`) |
+
+## Tensor parallelism and collectives
+
+| flag | default | meaning |
+|---|---|---|
+| `--tp N` | 1 | tensor-parallel ranks; declare divides dimensions by world size and inserts no collective |
+| `--tp-wire exact\|wht6` | `exact` | `wht6` is lossy: rotated 6-bit at 2 ranks, tiered int8 over a two-shot at 3–4, unavailable at 8 |
+| `--tp-wire-min-kb N` | 128 | smallest all-reduce (KiB) the lossy wire compresses; keeps single-sequence decode exact |
+| `--tp-attn-split heads\|rows` | `heads` | a prefill step's attention split: `rows` gives each rank all heads of its rows plus an all-gather; faster where attention dominates, not bit-identical, plugin-gated |
+| `--weight-prefetch on\|off` | on | read a GEMM's weight beside the collective before it; `off` when another server shares the cards |
+| `--p2p auto\|on\|off` | `auto` | `auto` asks the driver; `on` refuses to start without peer access; `off` exchanges through pinned host memory |
+
+## LoRA adapters
+
+| flag | default | meaning |
+|---|---|---|
+| `--lora NAME=PATH` | none | serve an adapter: a PEFT directory as model NAME (text), or a `.safetensors` a request names in `loras` (image/video). Repeatable |
+| `--lora-dir DIR` | none | generators: any `NAME.safetensors` under DIR may be requested |
+| `--max-loras N` | count of `--lora`, 2 with `--lora-dir` alone | adapter slots resident at once |
+| `--max-lora-rank R` | 16 text / 64 generator | slot rank; a larger text adapter is refused, a larger generator combination truncated |
+| `--lora-target-modules LIST` | the adapters' own | HF module leaf names adapters may target; needed with `--max-loras` and no `--lora` |
+
+## Request defaults and bounds
+
+A request that names a field wins. These set defaults for requests that do not.
+
+| flag | default | meaning |
+|---|---|---|
+| `--temp F`, `--top-k N`, `--top-p F`, `--min-p F` | from the model | sampler defaults; `top_k` 0 is the 1024 most likely candidates |
+| `--typical-p F` | 1 | typical sampling |
+| `--presence-penalty F`, `--frequency-penalty F` | 0 | range [-2, 2] |
+| `--repetition-penalty F`, `--repeat-last-n N` | 1, 64 | `-1` looks back over the whole context |
+| `--dry-multiplier F`, `--dry-base F`, `--dry-allowed-length N`, `--dry-penalty-last-n N` | 0, 1.75, 2, -1 | DRY; multiplier 0 is off |
+| `--dry-sequence-breakers JSON` | `["\n", ":", "\"", "*"]` | at most 16 strings |
+| `--xtc-probability F`, `--xtc-threshold F` | 0, 0.1 | range [0, 1] |
+| `--default-max-tokens N` | `auto` | `max_tokens` for a request that sends none; `auto` is what the context leaves |
+| `--max-tokens-cap N` | 0 (none) | a larger request is **cut to it**, not refused; reply ends `finish_reason: "length"` |
+| `--max-n N` | max(8, `--max-num-seqs`) | the largest fan-out `n` |
+| `--max-stop-strings N`, `--max-stop-bytes N` | 64, 4096 | stop-string limits |
+| `--chat-template-kwargs JSON` | none | variables for the template on every chat request; a JSON object or `@FILE`; repeatable |
+| `--reasoning-effort S` | template's own | default `reasoning_effort`; passed to the template verbatim, so the legal set is the template's |
+| `--reasoning-effort-map MAP` | none | `FROM=TO,...`, checked against the template at startup; `none`/`minimal` turn thinking off |
+| `--thinking-token-budget N` | none | default budget: at N reasoning tokens the block is closed and the answer follows |
+| `--reasoning-format auto\|none` | `auto` | `auto` returns reasoning as `reasoning_content`; `none` leaves it inline |
+
+## Server and HTTP
+
+| flag | default | meaning |
+|---|---|---|
+| `--host ADDR` | 0.0.0.0 | listen address |
+| `--port N` | 8000 | listen port |
+| `--http-threads N` | one per hardware thread, at least 4 | workers kept while idle |
+| `--read-timeout S` | 30 | a client may go silent this long while sending |
+| `--write-timeout S` | 600 | one write to a client may block this long |
+| `--keep-alive-timeout S` | 5 | idle connection hold |
+| `--max-body-mib N` | 512 | largest body; images and video ride base64 inside it |
+| `--no-cors` | CORS on | send no CORS headers |
+
+## Media preprocessing
+
+Each replaces what the container's preprocessor config states, and does nothing for a model that
+takes no media.
+
+| flag | default | meaning |
+|---|---|---|
+| `--image-min-pixels N`, `--image-max-pixels N` | container's | the band an image resizes into; one encoder pass still bounds the top |
+| `--video-min-pixels N`, `--video-max-pixels N` | container's | same, for a video's sampled frames together |
+| `--video-fps F` | container's | frames sampled per second |
+| `--video-min-frames N`, `--video-max-frames N` | container's | frame-count bounds |
+| `--video-max-frame-tokens N` | container's | the most prompt tokens one frame becomes; `0` no cap; a token-budget processor takes 70/140/280/560/1120 |
+| `--image-tokens N` | container's | a token-budget processor's image budget (Gemma 4's fixed set; GLM-5.3's ceiling ≤ `--mm-max-patches`/4) |
+| `--max-source-pixels N` | 67108864 | largest input the decoder accepts before resize |
+
+## Quality measurement (instead of serving)
+
+| flag | meaning |
+|---|---|
+| `--kld-record DIR` | run `--kld-corpus` through prefill and write this model's per-position log-probabilities to DIR: the reference |
+| `--kld-ref DIR` | score this model against the reference: mean/median/percentile KL and top-1 agreement |
+| `--kld-corpus FILE` | JSONL, one `{"prompt", "score_from"?, "source"?}` a line; `--kld-record` only |
+| `--kld-out FILE` | the report as JSON; `FILE.rows` holds every position's numbers |
+| `--kld-lora NAME`, `--kld-lora-sources PREFIX` | score through a served adapter, all documents or a `source` prefix |
+
+The mode turns speculation and the prefix cache off and refuses `--num-speculative-tokens > 0`.
+See [[Radiance Conversion]] for the corpus and the scoring discipline.
+
+## Diagnostics
+
+| flag | meaning |
+|---|---|
+| `--debug-graph` | print the declared graph, every op's resolved kernel and why, then exit |
+| `--debug-selection` | the selection table only |
+| `--debug-placement` | the placement plan only |
+| `--debug-tape-dump` | the first recorded pass of each kind (`TAPEDUMP` on stderr, every rank) |
+| `--debug-draft-oracle FILE` | verify-path test: prompt lookup proposes from FILE with 30 % corrupted drafts; the text must not move |
+| `--debug-accept-reference-kernels` | serve even with `libref` rows — not a production configuration |
+| `--profile-ops` | per-op device timing |
+| `--expert-stats FILE` | routing counts per layer and step kind, JSON every 30 s and at shutdown |
+| `--expert-profile FILE` | rank the expert plane by a stats file's `decode` counts instead of the container profile |
+| `--expert-pin-top F` | pin the hottest fraction F of routed experts so heat never demotes them |
+| `--draft-expert-placement host\|auto` | pin a draft head's experts in the host pool before any strategy runs |
+| `--live` | the terminal live view; redirected, one `key=value` line |
+| `-v`, `-vv` | verbose, very verbose |
+
+## The laws behind the knobs
+
+**The split is derived, and the receipt is the referee.** Non-expert weights come off the top of
+the card; the remainder (`elastic`) is shared by the expert slab and the KV cache. `--max-model-len`
+× `--max-num-seqs` is the cache's **demand**; the funding is `elastic − expert_floor`. Under a
+lending cache (`auto`/`expert_tiered`, no `--deterministic`, VMM present) the cache takes what it
+can address and lends unused blocks back to experts. **Every MiB added to `--host-pool-mib` lowers
+the expert floor one-for-one and the cache takes it** — until the cache already holds its whole
+ceiling, where the extra host room shows up as unclaimed. Read the boot line
+`kv: the pool backs N % of the worst case …`, never arithmetic.
+
+**The activation arena comes off the budget before the split**, so both sides pay for
+`--max-num-batched-tokens`. While steps are small most of it is lent back to the expert slab; a
+larger step reclaims it synchronously. That is why a bigger chunk can cost decode residency. On a
+hybrid model the chunk is clamped to `--checkpoint-interval` before the token budget, and startup
+prints which bound applies.
+
+**The KV dtype law.** `bf16` and `fp8` are engine-wide; `fp8_v4`/`k6_v4` are narrower layouts whose
+kernels exist only for particular attention geometries, and the server refuses them elsewhere by
+plugin name. A drafter's cache stays fp8. Recurrent state and the QSA indexer keep their own widths.
+
+**The slot law.** A slot costs recurrent state, block tables at full context, snapshot-store RAM
+and all-reduce rows — never speed by itself. Raising `--max-num-seqs` raises demand; whether it is
+funded is the resolver's answer.
+
+**The spec law.** Measure speculation by step time, not tok/s. Prompt lookup pays where the reply
+copies its context and costs a verify row elsewhere. `adaptive` moves the depth only after eight
+steps and a 2 % gain, because a new window is a new pass shape.
+
+**Batch shape is part of the configuration.** A different chunk, span or spec depth changes the
+summation order, so greedy text can differ at a tie. Reproducibility holds per configuration, never
+across one.
+
+**Unknown request fields are ignored; known-but-unimplemented fields are guarded** (accepted at
+their off value, refused otherwise) — the list is on [[Radiance HTTP API]].
+
+## Related
+
+- [[Radiance HTTP API]] — what the flags expose on the wire.
+- [[Radiance Operations]] — the container contract and readiness.
+- [[Radiance Plugins]] — how a flag reaches a plugin (`RadBuildCtx`).
+- [[frank-archon]] — the measured deployment of this resolver on two R9700s.

serverAppConfigs-docs/LLM/Radiance Tools.md

@@ -0,0 +1,322 @@
+---
+type: "reference"
+title: "Radiance Tools — inspect, check, tune, tokenise"
+description: "The radiance inspection, measurement and tokeniser tools, with the flags and verdicts each one reports."
+tags:
+  - llm
+  - radiance
+  - tools
+  - reference
+timestamp: 2026-10-30
+---
+
+# Radiance tools
+
+Six binaries and `rad-export` are installed beside `radiance`; `rad-avx-check`, `r4d_selftest` and
+`rad-chatparse` live in the build tree. Every tool that loads plugins honours `RADIANCE_HOME`,
+`RADIANCE_KERNELS` and `--kernels` as the engine does. A numeric option takes a whole number: `12k`,
+`auto` (where the option does not name it) or an empty value exits 2.
+
+`rad-convert`, `rad-kbench` and `rad-tune` run the engine's startup with a different last step: the
+same plugins, in the same hierarchy, through the same declare phase, so they resolve kernels exactly
+as a serve would.
+
+## `rad-info` — what a container holds
+
+```
+rad-info [options] <model.rad>
+rad-info --plugins [--home DIRS]
+
+  -v, --verbose      list every tensor, not just the per-unit summary
+      --meta         metadata table, then stop
+      --vocab        vocab summary and normaliser chain, then stop
+      --profile N    top N expert-profile entries (default 20, 0 = all)
+      --recipe       the recipe the container was converted with, then stop
+      --plugins      every plugin the search path offers, as the engine loads it
+      --home DIRS    search path for --plugins (default $RADIANCE_HOME)
+```
+
+It opens the container read-only and loads no plugins (except for `--plugins`): a container names no
+kernel library. Output is aligned; byte counts are binary units and each count says what it counts.
+
+```sh
+rad-info m.rad                    # format, architecture, model, source quant, recipe, sections
+rad-info --recipe m.rad           # the recipe, expanded, one rule a line
+rad-info -v m.rad | head -40      # every tensor, encoding, shape and what made it
+rad-info --plugins                # kind, version, declarations, reference, device fit, file
+```
+
+The section table (strings, metadata, directory, planes, encodings, vocab, profile, data) is the
+container's layout; the metadata table, the weight directory grouped by layer and movement unit, the
+vocab, and the expert profile follow.
+
+## `rad-kbench` — correctness, completeness, speed
+
+```
+rad-kbench [options]                       replay the recorded fixture (no model)
+rad-kbench -m <model.rad> [options]        check against a real container
+rad-kbench -m <model.rad> --record FILE    record a new fixture
+rad-kbench -m <model.rad> --skeleton FILE  write a metadata-only copy to record from
+```
+
+**Replay** needs no model, weights or card. Geometries, seeds and `libref`'s reduced answers are
+checked in as `data/kernels.rkb` (dense model, block drafter, gqa 6) and `data/kernels_moe.rkb`
+(routed MoE, MTP head, gqa 8). `--fixture` is repeatable; `ctest` runs three replays — the other
+libraries against each recording, and `libref` against both at tolerance zero (its regression test,
+since a live comparison moves both sides together).
+
+Each library is asked for its row as the engine would ask it with that library alone: same
+constraints, derived parameters and weight-encoding filter, so a row the engine would pass over does
+not run here either (`--all-rows` runs every row whose constraints hold).
+
+**Against a container** (`-m`), weight operands are read from it: each weight's canonical planes,
+which the kernel gets through its own `layout`/`relayout` hooks and the oracle gets decoded, and
+every expert of a weight table. This is the only test of the artifact and of the layout hooks.
+
+**Recording** takes the reference's answer once at every point of every declared op, whichever
+library resolved it. Weights are drawn from a seed, not read from the container, so any machine can
+reproduce the reference and no card is needed. A skeleton (tables intact, data a hole) records the
+same fixture in a few megabytes.
+
+```sh
+rad-kbench -m model.rad --skeleton model.skel.rad
+rad-kbench -m model.skel.rad --max-cases 1 --record data/kernels.rkb
+rad-kbench -m moe.rad        --max-cases 1 --record data/kernels_moe.rkb
+```
+
+### Verdicts
+
+- An output is compared on sampled positions, the full-tensor norm, and 32 seeded random-sign
+  projections (which estimate rel_l2 to about 25 %). The largest of the three is the error.
+- Tolerances are per op and activation dtype (`dtype`, else `q_dtype`, else `kv_dtype`). Copies,
+  gathers, scatters and cache stores are exact. Every op in the fixtures has a row of its own.
+- Ids are compared exactly; where the contract leaves a tie open (`sample_argmax`, `router_topk`,
+  `sample_topk`), an id indexing a value equal to the reference id's is accepted.
+- A case whose reference and output are both identically zero checked nothing: **vacuous**, not
+  passed.
+- A failure, a red-zone overrun, a fusion that differs from its chain, or a host kernel that wrote
+  into an input exits 1. `--strict` also fails a stale fixture.
+- `--coverage DIR` keeps one file per fixture/library of the rows a replay reaches
+  (`data/kbench_rows/`); a listed row not reached fails, unless a memory skip kept its cases out.
+- Every pass ends with a line saying where its time went (draw, hashing, layout, upload, launch,
+  read-back, comparison, cache, timing).
+
+### Timing
+
+`--bench` times every case that passes, on the bytes it was checked on.
+
+- Device cases are timed **after** the checks, operands still on the device, after one sustained
+  warm-up. A time is device timestamps around a batch of back-to-back launches divided by the launch
+  count; the batch is queued behind a gate the host opens once submission is done, so neither the
+  host rate nor a sync is in the interval. Two untimed launches precede every batch; batches repeat
+  until the median's 95 % interval is within 1 % or `--bench-reps` is reached.
+- **Cold by rotation**: weights and inputs of 1/32 of the footprint or more are read from copies
+  spanning the footprint (`--bench-flush`, default 256 MiB, at least twice an R9700's caches), so
+  each launch reads them from memory as streamed weights do. Smaller inputs stay warm, as the
+  previous op leaves them.
+- In-place operands are restored from a pristine copy before every launch; the restores are timed
+  alone and subtracted.
+- Timing draws expert offsets/routing skewed (Zipf 0.5) and block tables as permutations; the
+  checked launch keeps the recorded draw.
+- The header prints the calibration: launch floor and streaming-read bandwidth warm and cold. If a
+  smaller cold probe runs >1.5× the largest's rate the rotation was cache-served and device times
+  are withheld. Throttler bits that came on during a batch repeat it, up to three times.
+- Host cases are timed in their own column, after an untimed launch, weights rotated over at least
+  256 MiB (twice LLC), for at most two seconds a case.
+
+### Flags
+
+| flag | meaning |
+|---|---|
+| `--fixture FILE` | replay recording (default `$RADIANCE_HOME/kernels.rkb`); repeatable |
+| `--record FILE` | with `-m`: write a fixture |
+| `--skeleton FILE` | with `-m`: write a sparse container copy |
+| `--rerecord FILE` | replay: write the fixtures' cases again with the reference's answers |
+| `--record-rows FILE` | replay: record a row-driven fixture from the cases |
+| `-m, --model FILE` | the `.rad` to read weight operands from |
+| `--home DIR`, `--kernels A:B` | plugin home; kernel libraries (default every one found) |
+| `--op NAME` | only this op (repeatable prefix match) |
+| `--ref NAME` | the oracle plugin's name (default: the library declaring itself a reference) |
+| `--no-ref` | replay without the reference among the tested libraries |
+| `--strict` | replay: a stale fixture fails |
+| `--all-rows` | run every row whose constraints hold |
+| `--coverage DIR`, `--coverage-write` | the row-coverage ratchet |
+| `--no-cache`, `--cache-dir DIR` | run the reference instead of reusing cached answers; cache location |
+| `--seed N` | base seed; every case derives its own |
+| `--tol X` | override every tolerance (bisecting only) |
+| `--max-ctx N` | deepest context considered (declare and the attention sweep) |
+| `--max-tok N` | `max_tok` declare runs at (default 512, small on purpose) |
+| `--mm-max-patches N` | encoder pass declare runs at (default 2048 when a tower is present; 0 leaves it out) |
+| `--kv-cache-dtype T` | `bf16` or `fp8`, as on the engine |
+| `--num-speculative-tokens N` | draft depth to declare at (default: the container's) |
+| `--max-cases N` | runs per distinct case (default 4; 0 = every one) |
+| `--max-bytes MIB` | skip a case needing more than this, by name (default: half available memory, 512–8192) |
+| `--no-fusion` | skip the fusion oracle |
+| `--bench`, `--bench-iters N`, `--bench-reps N`, `--bench-warm-ms N`, `--bench-flush MIB` | timing |
+| `--timing-only` | time every device kernel, check nothing (refuses `--record`) |
+| `--report FILE`, `--analysis FILE` | markdown report; splice in `tools/kbench-analysis.sh` output |
+| `-v` | every case, not only failures |
+
+Notes: `--max-bytes` matters on a loaded machine — one case holds each operand as drawn, as the
+kernel's copy, and on the device; without the cap a small machine is OOM-killed with no indication of
+the op. `--bench` times cold on purpose: an R9700's 52 MB bf16 weight read warm comes from cache at
+1.45 TB/s against 565 GB/s from memory, so warm times rank shapes but do not size a roofline. A
+kernel that cannot lay out an encoding is a **skip**, not a failure (`RAD_E_DTYPE`), and the report
+names the kernel, operand and encoding.
+
+## `rad-tune` — measure the instantiation space
+
+```
+rad-tune -m <model.rad> [options]
+
+  -m, --model FILE   the .rad whose declared graph supplies the shapes
+      --home DIR     $RADIANCE_HOME; cache at <home>/tune/<machine>.tune
+      --kernels A:B  kernel libraries to load, first wins
+      --op NAME      only ops whose name starts with NAME
+      --machine KEY  override the machine key (writing a cache for another machine)
+      --reps N       timing reps, median reported (default 5)
+      --iters N      launches per rep (default 32)
+      --max-tok N    max_tok declare runs at (default 8192)
+      --tp N         tensor-parallel width to declare at (default 1)
+      --num-speculative-tokens N|auto   draft depth to declare at
+      --dry-run      list what would be measured
+  -v, --verbose      every candidate, not just the winner
+```
+
+Kernels declare tunable axes, not named variants; `rad-tune` walks the cross product at this model's
+shapes and writes the winner per (op, geometry) to the cache the selector reads at startup.
+
+- **Declare at the shapes you serve**: `--tp N` divides every sharded N; a declare at world size 1
+  measures a program no `--tp 2` serve runs. `--num-speculative-tokens` puts the drafter's kernels in
+  the measured graph.
+- At `--tp > 1` every rank declares at once, each on its own thread and device: a collective's
+  `init` is a rendezvous, so declaring rank 0 alone hangs.
+- A collective is measured as the exchange: per-rank operands (own seeds, so identical halves cannot
+  pass), own scratch and stream, own instance; every rank fires at once and the clock stops when the
+  last stream drains. Non-collective rows run on rank 0 alone.
+- `RADIANCE_TUNE` pins a kernel's axes to compare a tuned cache against the defaults — a measurement
+  aid, not a deployment setting.
+
+## `rad-schemas` — the op vocabulary diff
+
+```
+rad-schemas libref.so                 print one plugin's vocabulary
+rad-schemas libr4d.so libref.so       diff them, in hierarchy order
+```
+
+The first plugin in hierarchy order to declare an op fixes its schema; a later plugin declaring the
+same op differently is refused at load, both named (arguments are positional, so disagreeing schemas
+would make one call mean different things). `rad-schemas` prints every disagreement at once, so a new
+plugin can be conformed before it is loaded beside another. It exits 1 on any disagreement. It links
+no core — it `dlopen`s plugins and reads static tables, so it works on a plugin the engine cannot
+load.
+
+## `rad-avx-check` — host libraries without a card
+
+```
+rad-avx-check --ref libref.so --avx LIB.so [options]
+
+  --ref PATH     the oracle plugin      (default libref.so)
+  --avx PATH     the plugin under test  (default libavx.so)
+  --op NAME      only this op
+  --level N      only this ISA level (0 scalar, 1 avx, 2 avx2, 3 avx512)
+  --threads N    rerun every case with each parallel region on N threads (default 4; 0 skips)
+  --tol X        override every tolerance
+  -v             every case
+```
+
+Checks a host library op by op against `libref`, on the CPU, with neither engine nor card: two
+`dlopen`s, a geometry table (`libavx/avx_cases.cpp`), libref's operand descriptions and hand-built
+cases. For every case: schemas as the loader compares them, outputs within `rad-kbench`'s tolerances,
+guard bytes, inputs unchanged, and an oracle output that is not just the draw. `libavx` is also
+checked at every ISA level the CPU executes and with parallel regions forced onto several threads.
+A new host library is checked with the same command; it is treated as a host library (not `libavx`),
+so libref ops it has no row for are listed rather than failed, and the ISA/thread sweeps run only if
+it exports `avx_isa_force`/`avx_par_force`. A row returning `RAD_E_UNSUPPORTED` for a geometry libref
+serves fails: the selector would resolve to it and the step would fail at issue.
+
+## `r4d_selftest` — libr4d's own check and timer
+
+```
+r4d_selftest <libr4d.so> [--case <row>]
+r4d_selftest <libr4d.so> --perf N K M [iters]      (and the other --perf* modes in its usage)
+```
+
+With no mode it runs the catalogue and static checks, then each row's smoke and numeric cases in a
+child process of its own, with fusion claims compared byte for byte (`ctest` labels `slow`, `gpu`).
+`--case` runs one row in-process as the reproducer. `--perf` covers the fp8 decode GEMMs and head;
+`--perfmxd` the dense OCP-MXFP4 rows with the weight bandwidth each reaches; `--perfmx` the MXFP4
+expert GEMM; `--cold MIB` times each launch alone behind a cache flush. `--perfar8` prints the
+eight-rank all-reduce shard weights and times `lockstep`/`ring`/`split` forms at every piece count;
+`R4D_SELFTEST_ONE_CARD=1` runs them on one card for their bits only.
+
+## `rad-tokcheck` — tokeniser against its references
+
+```
+rad-tokcheck [options] <tokenizer.json> <corpus-file>...
+
+  --hf-dir DIR    compare against HuggingFace ids in DIR/<corpus basename>.ids
+                  (little-endian int32, encode(text, add_special_tokens=False))
+  --no-ref        skip the reference encoder (slow)
+  --user          encode with parse_special off
+  --turns N       turns in the multi-turn measurement (default 40; 0 skips)
+  --repeat N      timing repeats, best taken (default 3)
+  --context N     ids shown either side of a difference (default 6)
+```
+
+Per corpus file: ids match `tools/tokref`'s reference encoder (llama.cpp's splitters and
+`std::regex`); ids match HuggingFace `tokenizers` read from files made elsewhere; and speed — MB/s
+for both encoders, plus the cost of re-tokenising a growing conversation with and without the
+segment cache. Any difference prints the ids and surrounding text and exits nonzero, so it gates a
+change.
+
+## `rad-chatparse` — where a tool call goes wrong
+
+```
+rad-chatparse --request REQ.json --rad MODEL.rad --reply R.txt
+rad-chatparse --render-raw T.jinja --context CTX.json
+```
+
+Renders one OpenAI chat request as the server would (container template or
+`--override-chat-template`, `--chat-template-kwargs`, the plugin's declared reply format, or the
+template's own with `--no-plugin`), then reads each reply file as the model's output: parsed whole,
+parsed through the SSE stream as a client reassembles it, and run through the request's tool-call
+grammar. It prints one JSON object; exit 1 when the two parses disagree or the grammar refuses a
+token. No weights are loaded.
+
+## The KL mode
+
+`radiance --kld-record/--kld-ref/--kld-corpus/--kld-out/--kld-lora` is a mode of the engine, not a
+separate tool: same placement and kernels, prefill instead of listening, then exit. The flags and the
+corpus format are on [[Radiance Conversion]].
+
+## Shell harnesses (`scripts/`)
+
+They drive a running engine: bring a model up, measure it, assert on what came back. Each header
+records why it measures the way it does.
+
+| script | what it does |
+|---|---|
+| `mcserve.sh`, `fnserve.sh` | bring MiniCPM5-2B+DSpark / Flash-Next up and leave it running (env knobs: `P TP MLEN SEQS M KVD HR MNBT EXTRA …`) |
+| `fnstop.sh` | stop whichever is running and wait for exit |
+| `convdspark.sh`, `convfn.sh` | rebuild the two reference containers (`convfn.sh` needs `CALIB=`) |
+| `hfget.sh` | resumable Hugging Face fetch with a content manifest |
+| `fnpf.sh`, `fnconc.sh`, `cstep2.sh`, `ident.sh`, `cardclock.py` | attach to a live server: prefill rate; prefill-vs-decode contention; steady-state decode ms/step per concurrency; run-to-run hashes; which clock limit binds and which rank the TP step waits for |
+| `fndrop.sh` | disconnect a client mid-request and check every other answer against a reference |
+| `tierident.sh`, `tiercross.sh` | whether a session that left VRAM returns byte for byte (attention and recurrent state), including two sessions in flight |
+| `mctools.sh`, `mcstream.sh`, `mcagent.sh`, `mcdecl.sh` | tool-call parsing shapes; streamed tool calls on raw SSE; speculation on an agentic workload; whether the drafter proposes |
+| `dprofd.sh`, `fntrace.sh`, `mctrace.sh`, `oracle.sh` | per-op device timing; rocprofv3 traces; every op against libref on the operands the model issued |
+| `kv-parity.sh`, `kv-stage-parity-64k.sh` | KV streaming: covering ring and staged prefill are byte-identical to streaming off |
+
+**Not all of them leave a server alone.** `fnserve.sh`, `mcserve.sh`, `fnstop.sh`, `fntrace.sh`,
+`dprofd.sh`, `mctrace.sh`, `mcagent.sh` and `mcdecl.sh` kill any running engine (`pkill` on the
+binary), whatever port it is on; `oracle.sh` stops whatever listens on its own port. `cstep2.sh`,
+`ident.sh`, `tierident.sh`, `tiercross.sh`, `mctools.sh`, `mcstream.sh`, `hfget.sh`, `convfn.sh`,
+`convdspark.sh`, `fnpf.sh`, `fnconc.sh`, `fndrop.sh` and `cardclock.py` leave it running
+(`fndrop.sh` deliberately disconnects clients). Never point the stopping ones at a production port.
+
+## Related
+
+- [[Radiance Conversion]] — `rad-convert`, `rad-export`, the KL mode.
+- [[Radiance Plugins]] — what `rad-kbench`/`rad-tune`/`rad-schemas` check.
+- [[Radiance Operations]] — the startup receipt these tools corroborate.

serverAppConfigs-docs/LLM/frank-archon.md

@@ -562,6 +562,52 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
   fork bomb inside the container. Kill with `podman rm -f` (the cgroup owns every generation)
   never pattern-kill an interpreter name on this host; use `TORCH_LOGS` for dynamo questions.
 
+### The quantization pipeline
+
+- **A calibration tap is only a tap when the rig that opened it is running.** The runner writes
+  `.rig-boot` into the tap dir (container, image, pid) and retires it when its boot exits non-zero;
+  `calib-push.py` refuses without that marker, refuses when it names another container, and refuses
+  when the named container is not running; `qwen38-calib-window.sh` stops on a failed phase instead
+  of falling through its health wait. All three exist because on 2026-10-29 the unguarded chain's
+  rig boot died of an OOM, the health loop expired without failing, and the pusher fed four rounds
+  of 10M tokens into **restored production** — 86 minutes of live prefill that recorded nothing.
+- **A calibration step has an OPTIMUM, and it is not the widest step that boots.** The tap's cost is
+  expert re-reads per document (a 7,168-token document cut into four 2,048-token steps re-streams
+  each layer's routed expert union four times), which argues for the widest step — but the measured
+  curve turns over: at the 32,768 calibration window one 2-way prefill of the same document ran
+  **106 tok/s at an 8,192-token step and 76 tok/s at 16,384**, with the computed activation arena
+  growing 3.39 → 4.50 GiB, so the wider step pays more compute than it saves in reads. The
+  200,000-window `QSA score plane` refusal that made the wide step look impossible is an artifact of
+  serving the production window: the same 16,384-token step needs a 512 MiB plane at 32,768.
+  `Q38_CALIB_MNBT` is the only knob — the KL modes keep production's step, because their numbers are
+  only comparable at one shape.
+- **A container that exists, a port that answers, and a `/health` 200 are three different claims —
+  bind identity before believing any of them.** Measured 2026-10-30: `podman inspect` named the rig
+  while the port still belonged to the *dying* production process (`health ok waited=0s`, window dead
+  at second zero), and the identity check written to catch that compared two different number spaces:
+  `podman top <ctr> pid` prints **container-internal** pids (`1`, `7`), never host pids, and a rootless
+  cgroup carries the container ID rather than the service name — so both "matches" can only ever fail.
+  The binding is `podman top <ctr> hpid` against the listener from
+  `ss -H -ltnp 'sport = :7770'`, and `fuser /dev/kfd` likewise resolved through podman.
+  `scripts/qwen38-preflight.sh` owns those facts now (cwd, the digest a tag actually resolves to,
+  free space on the tap's **own** filesystem, stale grams, every GPU holder and the listener by host
+  pid), prints `PASS`/`FAIL` per fact, and `qwen38-calib-window.sh` calls it before a card is touched.
+  Corollary for every ad-hoc driver: a detached launcher that loses its cwd fails with
+  `No such file or directory` in a log nobody reads — assert cwd, and never bypass the orchestrator.
+- **The n-gram table's pin is what starves a rig boot, and the tier is a knob.** On the BF16 payload
+  the table is 95.37 GiB, `--ngram-placement ram` pins it, and the calibration boot died at
+  `startup stopped: out of memory` with 230.22 GiB pinned of 246.07 GiB and 8.06 GiB available —
+  while the reference recording and every score boot succeeded at the same pool and slot count.
+  Calibration only needs the routed weights prefilled, so it reads the table from the container's
+  file tier (`--ngram-placement disk`); kldref and score keep production's placement, because their
+  numbers are only comparable at an identical shape.
+- **Acceptance is three metrics and every slice, not a mean.** `scripts/kld-gate-qwen38.sh` grades
+  mean KL, the p99.9 tail, top-1 agreement and every `by_source` slice (code, prose, tools):
+  a mean can hide a few positions taking the whole hit, and an overall win can be bought by
+  regressing one distribution. The FP8-native anchor is an interval, not a requirement — without
+  the 172.8 GiB fetch the gate runs in two-report mode (candidate vs the served payload) and says
+  so on its first line.
+
 ## Contract (the wire)
 
 - **Live surface:** `/health` (== `/ping`), `/v1/models`, `/v1/chat/completions`
@@ -638,14 +684,19 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
   rebuild), `scripts/download-frank-archon-swift15.sh` (the BF16 source a w4dn8 re-convert
   needs), `scripts/hfget.sh` / `scripts/hf-fetch.sh`; template
   `scripts/qwen38-froggeric-v22.5.jinja`.
-- Convert campaign (dormant, kept for a future quant): `scripts/convert-swift15-w4dn8.sh`
-  `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`, `scripts/run-swift15-bf16.sh`
-  `scripts/kld-gate-swift15.sh`, `scripts/kld-then-calib.sh`, `scripts/calib-push.py`
+- The campaign (qwen38-w5nl64, live): `scripts/run-qwen38-bf16.sh` (kldref / calib / score — the
+  ONE measuring shape), `scripts/qwen38-calib-window.sh` (the gated calibration window),
+  `scripts/convert-qwen38-w5nl64.sh` with
+  `scripts/recipes/qwen4exp-qwen38-w5nl64-bf16trunk.recipe` (the arm driver and its template),
+  `scripts/kld-gate-qwen38.sh` (acceptance), `scripts/calib-push.py`,
   `scripts/make-calib-corpus.py`, `scripts/rechunk-kld-corpus.py`,
   `scripts/qwen38_common.py` (the tokenizer, JSONL and refusal plumbing those three
   corpus tools share — one `n_tokens`, so every tool counts the same way),
-  `scripts/derive-protected-experts.py`
-  `scripts/apply-protected-cast.sh`, `scripts/verify-swift15-ple.py`.
+  `scripts/derive-protected-experts.py`, `scripts/apply-protected-cast.sh`.
+- Retired swift15-era campaign scripts, superseded by the above and kept only as reference:
+  `scripts/convert-swift15-w4dn8.sh`, `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`,
+  `scripts/run-swift15-bf16.sh`, `scripts/kld-gate-swift15.sh`, `scripts/kld-then-calib.sh`,
+  `scripts/verify-swift15-ple.py`.
 - Probes: `scripts/probe-frank-archon-kv.py` (reads slots/window/dtype from the deployed env so
   it cannot drift; leg D = decode), `scripts/frank-archon-expert-activity.py`
   `scripts/autoresearch-frank-archon.sh` + `scripts/autoresearch/frank-archon-measure.py`.
@@ -655,5 +706,6 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
 - Retired-era leftovers in-tree: `scripts/patches/glm53-*.patch`
   `scripts/patches/gpu_worker-kv-cap.patch` (fork archaeology; none applies byte-exact today).
 
-Related: [[frank-dragoon]] (the abliterated family), [[frank-zealot]],
----
+Related: [[frank-dragoon]] (the abliterated family), [[frank-zealot]], and the engine reference —
+[[Radiance Engine]], [[Radiance Serve Options]], [[Radiance Conversion]], [[Radiance Plugins]],
+[[Radiance Operations]].

serverAppConfigs-docs/LLM/index.md

@@ -13,6 +13,20 @@ the endpoint's **Eras** table, and the numbers a retired era measured are gone.
 | [[frank-dragoon]] | The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it. |
 | [[frank-zealot]] | The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws. |
 
+## Radiance engine reference
+
+| Page | Owns |
+|---|---|
+| [[Radiance Engine]] | The radiance inference engine: its shape, plugin model, container format, and how the reference pages fit together. |
+| [[Radiance Serve Options]] | Every radiance serve flag grouped by task, with its default and the law that decides when to move it. |
+| [[Radiance HTTP API]] | The HTTP surface radiance serves: endpoints, request fields, streaming, tools, and refusal rules. |
+| [[Radiance Conversion]] | Turning a checkpoint into a .rad container: rad-convert, the recipe language, quantisers, reuse and export. |
+| [[Radiance Model Families]] | Which radiance plugin claims which model family, and the conversion command and recipe of record for each. |
+| [[Radiance Plugins]] | Writing radiance kernel and quantiser plugins: the ABI, kernel rows, hooks, encodings, and the build path. |
+| [[Radiance Architecture Plugins]] | Writing a radiance architecture plugin: declaring weights, buffers, KV groups and ops, then issuing the step. |
+| [[Radiance Tools]] | The radiance inspection, measurement and tokeniser tools, with the flags and verdicts each one reports. |
+| [[Radiance Operations]] | Running a radiance server: the container contract, host prerequisites, readiness, ledgers and safe restarts. |
+
 ## Retired endpoints
 
 | Page | Owns |

serverAppConfigs-docs/index.md

@@ -49,8 +49,10 @@ is a locator: one row, one clause, no numbers.
 | [[Copyparty]] | Copyparty is a lightweight file server and media streamer that runs as a single Python binary. |
 | [[Dankserver Stack]] | Service inventory and routing overview for the media and library host. |
 | [[Entropy Volume]] | The Dankserver entropy volume is a RAID-backed LUKS filesystem mounted at |
-| [[Jellyfin]] | Jellyfin on Dankserver: quadlet unit, entropy-volume data layout, ports, client access, hardware-transcode status. |
+| [[Jellyfin]] | Jellyfin on Dankserver: quadlet unit, overlay metadata view, provider keys, ports, access, transcode status. |
 | [[mariadb]] | The MariaDB backend for RomM: quadlet unit, network alias, env-file secret handling, and the backup path. |
+| [[Media Metadata View]] | The SSD-backed overlay the two media servers read and write: shape, guards, commands, laws, rollback. |
+| [[Media Relayout]] | Census, plan and apply the movie and TV libraries into Jellyfin format: gates, evidence chain, refusals, rollback. |
 | [[Plex]] | Plex Media Server is a **containerized media server on Dankserver**: a podman |
 | [[redis]] | The Redis cache and task queue for RomM: quadlet unit, network alias topology, and the backup path. |
 | [[RomM]] | RomM (ROM Manager) is a self-hosted ROM library manager and player. |
@@ -162,6 +164,15 @@ is a locator: one row, one clause, no numbers.
 | [[PCIe Link and Atomics]] | Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives. |
 | [[Quantization by Hardware Architecture]] | This page maps LLM quantization formats to the GPU and CPU architectures that support them. |
 | [[Quantization Production]] | Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file. |
+| [[Radiance Architecture Plugins]] | Writing a radiance architecture plugin: declaring weights, buffers, KV groups and ops, then issuing the step. |
+| [[Radiance Conversion]] | Turning a checkpoint into a .rad container: rad-convert, the recipe language, quantisers, reuse and export. |
+| [[Radiance Engine]] | The radiance inference engine: its shape, plugin model, container format, and how the reference pages fit together. |
+| [[Radiance HTTP API]] | The HTTP surface radiance serves: endpoints, request fields, streaming, tools, and refusal rules. |
+| [[Radiance Model Families]] | Which radiance plugin claims which model family, and the conversion command and recipe of record for each. |
+| [[Radiance Operations]] | Running a radiance server: the container contract, host prerequisites, readiness, ledgers and safe restarts. |
+| [[Radiance Plugins]] | Writing radiance kernel and quantiser plugins: the ABI, kernel rows, hooks, encodings, and the build path. |
+| [[Radiance Serve Options]] | Every radiance serve flag grouped by task, with its default and the law that decides when to move it. |
+| [[Radiance Tools]] | The radiance inspection, measurement and tokeniser tools, with the flags and verdicts each one reports. |
 | [[RDNA4 MoE Expert Offload VRAM Budget]] | Expert-offload arithmetic on the RDNA4 fork: movable terms, the utilisation law, graph charge, ceilings, context ladder. |
 | [[Shared Memory Architecture Dankclaw]] | Dankclaw uses AMD Strix Halo APU with unified memory architecture. |
 

tests/deployment_scoping.bats

@@ -133,6 +133,16 @@ REMOTE_DEPLOY="$REPO_ROOT/scripts/deploy-remote.sh"
     [ -z "$output" ]
 }
 
+@test "a compose edit never restarts a Quadlet-owned media server" {
+    # dankserver/compose.yaml is an inventory; the media servers' Quadlets own
+    # their containers. A compose edit that restarted them would cut real viewers
+    # for a change that cannot affect them.
+    run python3 "$PLANNER" --target dankserver --paths dankserver/compose.yaml
+    [ "$status" -eq 0 ]
+    ! printf '%s\n' "$output" | grep -qx jellyfin
+    ! printf '%s\n' "$output" | grep -qx plex
+}
+
 @test "compose planner narrows a compose edit to the changed service" {
     local base head
     base=$(git -C "$REPO_ROOT" rev-parse HEAD)

tests/jellyfin_providers.bats

@@ -0,0 +1,297 @@
+#!/usr/bin/env bats
+# Seam: scripts/jellyfin-providers.sh — the only mechanism that puts a
+# metadata-provider key into Jellyfin's plugin configuration on Dankserver.
+#
+# Output vocabulary the tests depend on:
+#   "+ <action>"    something this run will change
+#   "= <fact>"      something already true, so nothing changes
+#
+# The contracts, in the order the seam tests them:
+#   1. plan reports every action and never prints a key;
+#   2. a secret file that is not mode 600 is refused by name and nothing is
+#      written — a key readable by other users is an error, not a warning;
+#   3. a downloaded plugin artifact is accepted only on its pinned sha256, its
+#      official guid and its pinned version (a swapped or downgraded artifact
+#      is refused before anything lands in plugins/);
+#   4. the TMDb key is written into the EXISTING config with every other
+#      setting preserved — Jellyfin owns that file, and clobbering it would
+#      silently reset country, language and adult settings;
+#   5. a TVDB config the operator already saved through the UI is kept;
+#   6. install is idempotent: a second run changes no byte;
+#   7. whatever is replaced is copied to the trash first (fleet law: never
+#      overwrite without a recoverable copy);
+#   8. verify answers whether the endpoint is configured, key LENGTH only;
+#   9. no plugin configuration XML is tracked in git, the deployer places the
+#      keys BEFORE it restarts Jellyfin, and the planner routes a change to this
+#      installer to jellyfin.
+
+load helpers
+
+PROV="$REPO_ROOT/scripts/jellyfin-providers.sh"
+TMDB_KEY="tmdbk3a9Zq7Rtestkey"
+TVDB_KEY="tvdbk8b2Wt4Ptestkey"
+GUID_OFFICIAL="a677c0da-fac5-4cde-941a-7134223f14c8"
+
+# make_plugin <file> <guid> <version> — a zip shaped like the official TheTVDB
+# artifact (meta.json + its two DLLs), so extraction is exercised for real.
+make_plugin() {
+    python3 - "$1" "$2" "$3" <<'PYEOF'
+import json, sys, zipfile
+path, guid, version = sys.argv[1], sys.argv[2], sys.argv[3]
+meta = {
+    "category": "MoviesAndShows",
+    "guid": guid,
+    "name": "TheTVDB",
+    "owner": "jellyfin",
+    "targetAbi": "12.0.0.0",
+    "version": version,
+}
+with zipfile.ZipFile(path, "w") as z:
+    z.writestr("meta.json", json.dumps(meta))
+    z.writestr("Jellyfin.Plugin.Tvdb.dll", "dll-bytes")
+    z.writestr("Tvdb.Sdk.dll", "dll-bytes")
+PYEOF
+    sha256sum "$1" | cut -d' ' -f1
+}
+
+setup() {
+    JF="$BATS_TEST_TMPDIR/jf"
+    TRASH="$BATS_TEST_TMPDIR/trash"
+    SEC="$BATS_TEST_TMPDIR/secrets"
+    mkdir -p "$JF/plugins/configurations" "$TRASH" "$SEC"
+    cat >"$JF/plugins/configurations/Jellyfin.Plugin.Tmdb.xml" <<XML
+<?xml version="1.0" encoding="utf-8"?>
+<Configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
+  <MetadataCountryCode>US</MetadataCountryCode>
+  <PreferOriginalLanguage />
+  <Language>en</Language>
+  <EnableCinematicForTrailers>false</EnableCinematicForTrailers>
+  <IncludeAdult>false</IncludeAdult>
+  <MaxCastMembers>25</MaxCastMembers>
+  <AddGenresFromKeywords>true</AddGenresFromKeywords>
+  <TmdbApiKey />
+</Configuration>
+XML
+    KEYS="$SEC/provider-keys"
+    printf 'TMDB_API_KEY=%s\nTVDB_API_KEY=%s\n' "$TMDB_KEY" "$TVDB_KEY" >"$KEYS"
+    chmod 600 "$KEYS"
+    ZIP="$BATS_TEST_TMPDIR/thetvdb.zip"
+    PIN_SHA="$(make_plugin "$ZIP" "$GUID_OFFICIAL" 24.0.0.0)"
+    PROV_ENV=(
+        JF_ROOT="$JF"
+        PROV_KEYS="$KEYS"
+        TRASH_DIR="$TRASH"
+        TVDB_PLUGIN_URL="file://$ZIP"
+        TVDB_PLUGIN_SHA256="$PIN_SHA"
+        TVDB_PLUGIN_GUID="$GUID_OFFICIAL"
+        TVDB_PLUGIN_VERSION=24.0.0.0
+    )
+}
+
+prov() { env "${PROV_ENV[@]}" bash "$PROV" "$@"; }
+
+tree_fingerprint() { find "$JF" -type f | sort | xargs -r sha256sum | sha256sum; }
+
+@test "plan names the plugin and both config files, and prints no key" {
+    run prov plan
+    [ "$status" -eq 0 ]
+    printf '%s\n' "${lines[@]}" | grep -q '^+ install thetvdb_24\.0\.0\.0'
+    printf '%s\n' "${lines[@]}" | grep -q '^+ set TmdbApiKey'
+    printf '%s\n' "${lines[@]}" | grep -q '^+ create Jellyfin\.Plugin\.Tvdb\.xml'
+    # The whole point of running this over a shoulder: a key never appears.
+    ! printf '%s\n' "${lines[@]}" | grep -qF "$TMDB_KEY"
+    ! printf '%s\n' "${lines[@]}" | grep -qF "$TVDB_KEY"
+}
+
+@test "plan writes nothing" {
+    before="$(tree_fingerprint)"
+    run prov plan
+    [ "$status" -eq 0 ]
+    [ "$(tree_fingerprint)" = "$before" ]
+}
+
+@test "an already-configured endpoint reports no pending action" {
+    prov install >/dev/null
+    run prov plan
+    [ "$status" -eq 0 ]
+    ! printf '%s\n' "${lines[@]}" | grep -q '^+ '
+    printf '%s\n' "${lines[@]}" | grep -q '= thetvdb_24\.0\.0\.0 already installed'
+    printf '%s\n' "${lines[@]}" | grep -q '= TmdbApiKey already set'
+    printf '%s\n' "${lines[@]}" | grep -q '= Jellyfin\.Plugin\.Tvdb\.xml already has an ApiKey'
+}
+
+@test "a keys file that is not mode 600 is refused and nothing is written" {
+    chmod 644 "$KEYS"
+    before="$(tree_fingerprint)"
+    run prov install
+    [ "$status" -ne 0 ]
+    printf '%s\n' "${lines[@]}" | grep -qF '600'
+    [ "$(tree_fingerprint)" = "$before" ]
+}
+
+@test "a missing TVDB key is refused by name before anything is fetched" {
+    printf 'TMDB_API_KEY=%s\n' "$TMDB_KEY" >"$KEYS"
+    chmod 600 "$KEYS"
+    run prov install
+    [ "$status" -ne 0 ]
+    printf '%s\n' "${lines[@]}" | grep -qF 'TVDB_API_KEY'
+    [ ! -d "$JF/plugins/thetvdb_24.0.0.0" ]
+}
+
+@test "a root with no plugin directory is refused instead of fabricated" {
+    # Creating plugins/ inside a wrong path would install a plugin Jellyfin
+    # never loads, and would hide the operator's typo.
+    run env "${PROV_ENV[@]}" JF_ROOT="$BATS_TEST_TMPDIR/nope" bash "$PROV" install
+    [ "$status" -ne 0 ]
+    [ ! -d "$BATS_TEST_TMPDIR/nope/plugins" ]
+}
+
+@test "an artifact whose sha256 is not the pinned one is refused" {
+    run env "${PROV_ENV[@]}" TVDB_PLUGIN_SHA256="$(printf 'f%.0s' {1..64})" \
+        bash "$PROV" install
+    [ "$status" -ne 0 ]
+    printf '%s\n' "${lines[@]}" | grep -qiF 'sha256'
+    [ ! -d "$JF/plugins/thetvdb_24.0.0.0" ]
+}
+
+@test "an artifact that is not the official TVDB plugin is refused" {
+    # A matching sha proves only that the bytes are the bytes we fetched.
+    EVIL="$BATS_TEST_TMPDIR/evil.zip"
+    EVIL_SHA="$(make_plugin "$EVIL" "deadbeef-0000-0000-0000-000000000000" 24.0.0.0)"
+    run env "${PROV_ENV[@]}" TVDB_PLUGIN_URL="file://$EVIL" TVDB_PLUGIN_SHA256="$EVIL_SHA" \
+        bash "$PROV" install
+    [ "$status" -ne 0 ]
+    printf '%s\n' "${lines[@]}" | grep -qiF 'guid'
+    [ ! -d "$JF/plugins/thetvdb_24.0.0.0" ]
+}
+
+@test "an artifact that is not the pinned version is refused" {
+    OLD="$BATS_TEST_TMPDIR/old.zip"
+    OLD_SHA="$(make_plugin "$OLD" "$GUID_OFFICIAL" 22.0.0.0)"
+    run env "${PROV_ENV[@]}" TVDB_PLUGIN_URL="file://$OLD" TVDB_PLUGIN_SHA256="$OLD_SHA" \
+        bash "$PROV" install
+    [ "$status" -ne 0 ]
+    printf '%s\n' "${lines[@]}" | grep -qiF 'version'
+    [ ! -d "$JF/plugins/thetvdb_24.0.0.0" ]
+}
+
+@test "a different plugin version already in plugins/ is refused, not doubled" {
+    # Two thetvdb_* directories means Jellyfin loads one plugin twice; the
+    # operator removes the old one (into the trash) deliberately, not by an
+    # installer quietly adding a second.
+    prov install >/dev/null
+    OLD="$BATS_TEST_TMPDIR/old.zip"
+    OLD_SHA="$(make_plugin "$OLD" "$GUID_OFFICIAL" 22.0.0.0)"
+    run env "${PROV_ENV[@]}" TVDB_PLUGIN_URL="file://$OLD" TVDB_PLUGIN_SHA256="$OLD_SHA" \
+        TVDB_PLUGIN_VERSION=22.0.0.0 bash "$PROV" install
+    [ "$status" -ne 0 ]
+    printf '%s\n' "${lines[@]}" | grep -qF 'thetvdb_24.0.0.0'
+    [ ! -d "$JF/plugins/thetvdb_22.0.0.0" ]
+}
+
+@test "install places the plugin, keys TMDb, and preserves every other setting" {
+    run prov install
+    [ "$status" -eq 0 ]
+    ! printf '%s\n' "${lines[@]}" | grep -qF "$TMDB_KEY"
+    ! printf '%s\n' "${lines[@]}" | grep -qF "$TVDB_KEY"
+    [ -f "$JF/plugins/thetvdb_24.0.0.0/meta.json" ]
+    [ -f "$JF/plugins/thetvdb_24.0.0.0/Jellyfin.Plugin.Tvdb.dll" ]
+    local cfg="$JF/plugins/configurations/Jellyfin.Plugin.Tmdb.xml"
+    # Jellyfin owns this file: the settings it wrote must survive the edit.
+    for setting in MetadataCountryCode Language IncludeAdult MaxCastMembers AddGenresFromKeywords; do
+        grep -qF "<$setting>" "$cfg" || {
+            echo "lost $setting"
+            return 1
+        }
+    done
+    grep -qF '<IncludeAdult>false</IncludeAdult>' "$cfg"
+    grep -qF "<TmdbApiKey>$TMDB_KEY</TmdbApiKey>" "$cfg"
+    grep -qF "<ApiKey>$TVDB_KEY</ApiKey>" "$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml"
+}
+
+@test "a TVDB config the operator saved through the UI is kept" {
+    cat >"$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml" <<XML
+<?xml version="1.0" encoding="utf-8"?>
+<Configuration xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
+  <ApiKey>ui-saved-key</ApiKey>
+  <EnableSeasonPathMatching>true</EnableSeasonPathMatching>
+</Configuration>
+XML
+    run prov install
+    [ "$status" -eq 0 ]
+    printf '%s\n' "${lines[@]}" | grep -qiF 'kept'
+    local cfg="$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml"
+    grep -qF '<ApiKey>ui-saved-key</ApiKey>' "$cfg"
+    grep -qF '<EnableSeasonPathMatching>true</EnableSeasonPathMatching>' "$cfg"
+}
+
+@test "a pin in the TVDB secret lands in the plugin config" {
+    # A subscriber-shaped key needs the account's PIN beside it; a licensed key
+    # does not, so the element appears only when the secret carries one.
+    printf 'TMDB_API_KEY=%s\nTVDB_API_KEY=%s\nTVDB_PIN=123456\n' "$TMDB_KEY" "$TVDB_KEY" >"$KEYS"
+    chmod 600 "$KEYS"
+    prov install >/dev/null
+    grep -qF '<Pin>123456</Pin>' "$JF/plugins/configurations/Jellyfin.Plugin.Tvdb.xml"
+}
+
+@test "install is idempotent: a second run changes no byte" {
+    prov install >/dev/null
+    before="$(tree_fingerprint)"
+    run prov install
+    [ "$status" -eq 0 ]
+    [ "$(tree_fingerprint)" = "$before" ]
+    printf '%s\n' "${lines[@]}" | grep -qiF 'no change'
+    # One plugin directory only: two versions in plugins/ means two loads.
+    [ "$(find "$JF/plugins" -maxdepth 1 -type d -name 'thetvdb_*' | wc -l)" -eq 1 ]
+}
+
+@test "what install replaces is copied to the trash first" {
+    prov install >/dev/null
+    # The TMDb config existed before this tool touched it, so the pre-edit copy
+    # must stay recoverable.
+    [ -n "$(find "$TRASH" -name 'Jellyfin.Plugin.Tmdb.xml*' -print -quit)" ]
+    grep -qF '<TmdbApiKey />' "$(find "$TRASH" -name 'Jellyfin.Plugin.Tmdb.xml*' | head -1)"
+}
+
+@test "verify refuses an unconfigured endpoint and names what is missing" {
+    run prov verify
+    [ "$status" -ne 0 ]
+    printf '%s\n' "${lines[@]}" | grep -qiF 'thetvdb'
+    printf '%s\n' "${lines[@]}" | grep -qiF 'TmdbApiKey'
+}
+
+@test "verify passes once the plugin and both keys are in place" {
+    prov install >/dev/null
+    run prov verify
+    [ "$status" -eq 0 ]
+    ! printf '%s\n' "${lines[@]}" | grep -qF "$TMDB_KEY"
+    ! printf '%s\n' "${lines[@]}" | grep -qF "$TVDB_KEY"
+    # Length, not bytes: the operator sees the key loaded without reading it.
+    printf '%s\n' "${lines[@]}" | grep -qE 'TmdbApiKey .*[0-9]+ chars'
+}
+
+@test "no plugin configuration XML is tracked in this repo" {
+    # These files hold provider keys and are owned by the running container;
+    # tracking them would put credentials in git and guarantee drift.
+    run git -C "$REPO_ROOT" ls-files
+    [ "$status" -eq 0 ]
+    ! printf '%s\n' "${lines[@]}" | grep -qF 'plugins/configurations'
+    ! printf '%s\n' "${lines[@]}" | grep -qiF 'Plugin.Tvdb.xml'
+}
+
+@test "the planner routes this installer to jellyfin" {
+    run python3 "$REPO_ROOT/scripts/deploy-plan.py" --target dankserver \
+        --paths scripts/jellyfin-providers.sh
+    [ "$status" -eq 0 ]
+    [ "$output" = "jellyfin" ]
+}
+
+@test "the deployer places the keys before it restarts jellyfin" {
+    # The plugin and both keys are read at start-up, so a restart that runs
+    # ahead of the placement boots the old, keyless state.
+    body="$(sed -n '/^deploy_jellyfin()/,/^\}/p' "$REPO_ROOT/scripts/deploy-remote.sh")"
+    [ -n "$body" ]
+    up="$(printf '%s\n' "$body" | grep -n 'ensure_provider_keys' | head -1 | cut -d: -f1)"
+    restart="$(printf '%s\n' "$body" | grep -n 'systemctl --user restart jellyfin.service' | head -1 | cut -d: -f1)"
+    [ "${up:-999}" -lt "${restart:-0}" ]
+}

tests/media_overlay.bats

@@ -0,0 +1,161 @@
+#!/usr/bin/env bats
+# Seam: the SSD-backed metadata view (OverlayFS) that lets Jellyfin write
+# Kodi-format sidecars for Plex to read, without ever writing to the spindle
+# array and without letting either media server start against a missing view.
+#
+# The contracts, in the order the seam tests them:
+#   1. the plan puts the writable layer on the SSD and the media layer below it;
+#   2. bad input is refused by name instead of mounting something surprising;
+#   3. verify refuses a view that is not mounted (so nothing trusts a directory);
+#   4. both media units bind the VIEW and refuse to start without it;
+#   5. only those two units touch the view — the LLM and library tenants keep the
+#      raw volume, so the blast radius is two containers;
+#   6. the deployer brings the view up BEFORE it restarts either server;
+#   7. the planner routes a change to these files to both media services.
+
+load helpers
+
+VIEW="$REPO_ROOT/scripts/media-view.sh"
+JELLY="$REPO_ROOT/dankserver/jellyfin/quadlet/jellyfin.container"
+PLEX="$REPO_ROOT/dankserver/plex/quadlet/plex.container"
+MOUNT=/var/lib/media-meta/view
+UNIT="$REPO_ROOT/dankserver/host/systemd/media-meta-view.mount"
+
+@test "the plan names an overlay with the spindle below and the SSD above" {
+    run bash "$VIEW" plan
+    [ "$status" -eq 0 ]
+    # Lower = the array; upper/work = the NVMe. A user asked for metadata to be
+    # SSD-resident, so a spindle upper layer would defeat the whole mechanism.
+    [ "${lines[0]}" = "lowerdir=/mnt/entropy" ]
+    printf '%s\n' "${lines[@]}" | grep -qx 'upperdir=/var/lib/media-meta/upper'
+    printf '%s\n' "${lines[@]}" | grep -qx 'workdir=/var/lib/media-meta/work'
+    printf '%s\n' "${lines[@]}" | grep -Fqx "mountpoint=$MOUNT"
+    # systemd names a mount unit after its escaped path, backslash and all.
+    printf '%s\n' "${lines[@]}" | grep -Fqx \
+        'unit=/etc/systemd/system/var-lib-media\x2dmeta-view.mount'
+}
+
+@test "install refuses a caller who cannot write the unit directory or mount" {
+    # Half an install is worse than none: the unit would exist without the
+    # directories behind it. Refuse by name before anything is created.
+    if [ "$(id -u)" -eq 0 ]; then
+        skip "this seam is running as root"
+    fi
+    run bash "$VIEW" install
+    [ "$status" -ne 0 ]
+    [[ "$output" == *"root"* ]]
+}
+
+@test "verify refuses a view that is not mounted" {
+    # Every consumer trusts the mount point to BE the library. On a host where it
+    # is a plain empty directory, verify must fail loudly: that is the difference
+    # between a scan that finds nothing and a scan that empties a library DB.
+    if mountpoint -q "$MOUNT"; then
+        skip "$MOUNT is a real mountpoint on this host"
+    fi
+    run bash "$VIEW" verify
+    [ "$status" -ne 0 ]
+    [[ "$output" == *"$MOUNT"* ]]
+}
+
+@test "both media servers bind the view and refuse to start without it" {
+    for unit in "$JELLY" "$PLEX"; do
+        grep -q "^Volume=$MOUNT:" "$unit" || return 1
+        grep -q "^UnitConfigItem=ConditionPathIsMountPoint=$MOUNT$" "$unit" || return 1
+        # The raw array must no longer be a container source: that bind is what
+        # made sidecar sharing impossible (read-only) and metadata slow (seeks).
+        ! grep -q '^Volume=/mnt/entropy' "$unit"
+    done
+    # Jellyfin is the writer, so it needs the view writable; Plex only reads the
+    # sidecars, and keeping its bind read-only costs nothing.
+    grep -q "^Volume=$MOUNT:/data$" "$JELLY"
+    grep -q "^Volume=$MOUNT:/mnt/entropy:ro$" "$PLEX"
+}
+
+@test "only the two media units reference the view" {
+    run grep -rl --include='*.container' "$MOUNT" "$REPO_ROOT"/dankserver \
+        "$REPO_ROOT"/dankclaw "$REPO_ROOT"/frankbook-pro 2>/dev/null
+    [ "$status" -eq 0 ]
+    [ "$(printf '%s\n' "$output" | wc -l)" -eq 2 ]
+    printf '%s\n' "$output" | grep -q 'jellyfin/quadlet/jellyfin.container'
+    printf '%s\n' "$output" | grep -q 'plex/quadlet/plex.container'
+}
+
+@test "the mount unit is an overlay whose upper layer lives on the SSD" {
+    [ -f "$UNIT" ]
+    run grep -E '^Options=lowerdir=/mnt/entropy,upperdir=/var/lib/media-meta/upper,workdir=/var/lib/media-meta/work' "$UNIT"
+    [ "$status" -eq 0 ]
+    # The unit must mount exactly where the two containers bind, or the guard and
+    # the bind disagree and a media server browses an empty directory.
+    run grep -Fqx "Where=$MOUNT" "$UNIT"
+    [ "$status" -eq 0 ]
+    # It must wait for the array to be unlocked: mounting an overlay whose lower
+    # dir is an empty /mnt/entropy would present an empty library, not an error.
+    run grep -E '^RequiresMountsFor=/mnt/entropy$' "$UNIT"
+    [ "$status" -eq 0 ]
+}
+
+@test "the deployer raises the view before it restarts either media server" {
+    for svc in jellyfin plex; do
+        body="$(sed -n "/^deploy_${svc}()/,/^\}/p" "$REPO_ROOT/scripts/deploy-remote.sh")"
+        [ -n "$body" ]
+        up="$(printf '%s\n' "$body" | grep -n 'ensure_media_view' | head -1 | cut -d: -f1)"
+        up_line="${up:-999}"
+        restart="$(printf '%s\n' "$body" | grep -n "systemctl --user restart $svc.service" | head -1 | cut -d: -f1)"
+        restart_line="${restart:-0}"
+        [ "$up_line" -lt "$restart_line" ]
+    done
+}
+
+@test "the planner routes the view files to both media services" {
+    run python3 "$REPO_ROOT/scripts/deploy-plan.py" --target dankserver --paths \
+        scripts/media-view.sh dankserver/host/systemd/media-meta-view.mount
+    [ "$status" -eq 0 ]
+    [ "$output" = "jellyfin
+plex" ]
+}
+
+@test "a docs-only change still plans nothing for the view" {
+    run python3 "$REPO_ROOT/scripts/deploy-plan.py" --target dankserver --paths \
+        "serverAppConfigs-docs/Dankserver/Media Metadata View.md"
+    [ "$status" -eq 0 ]
+    [ -z "$output" ]
+}
+
+@test "reap names the metadata whose media has moved and touches nothing" {
+    # A relayout moves media on the RAW path; the sidecars the view already wrote
+    # stay behind at the old coordinates. Left alone they become folders a media
+    # server browses and finds nothing in.
+    tmp="$(mktemp -d)"
+    mkdir -p "$tmp/meta/upper/Movies/Ghost (2020)" "$tmp/meta/upper/Movies/Keeper (2020)" \
+        "$tmp/meta/upper/TVShows/Big Little Lies/Season 01" "$tmp/meta/work" \
+        "$tmp/array/Movies/Keeper (2020)" "$tmp/array/TVShows"
+    : > "$tmp/meta/upper/Movies/Ghost (2020)/movie.nfo"
+    : > "$tmp/meta/upper/Movies/Keeper (2020)/movie.nfo"
+    : > "$tmp/meta/upper/TVShows/Big Little Lies/Season 01/show.nfo"
+    : > "$tmp/array/Movies/Keeper (2020)/Keeper (2020).mkv"
+
+    run bash "$VIEW" reap --upper-base "$tmp/meta" --lower "$tmp/array"
+    [ "$status" -eq 0 ]
+    printf '%s\n' "$output" | grep -qxF 'Movies/Ghost (2020)'
+    # The series folder is still below, so the ghost is the season inside it — and
+    # it is reported at the level that is actually missing, not deeper.
+    printf '%s\n' "$output" | grep -qxF 'TVShows/Big Little Lies'
+    ! printf '%s\n' "$output" | grep -q 'Keeper'
+    [ -f "$tmp/meta/upper/Movies/Ghost (2020)/movie.nfo" ]
+}
+
+@test "reap --apply moves ghosts to the SSD trash and keeps live metadata" {
+    tmp="$(mktemp -d)"
+    mkdir -p "$tmp/meta/upper/Movies/Ghost (2020)" "$tmp/meta/upper/Movies/Keeper (2020)" \
+        "$tmp/array/Movies/Keeper (2020)"
+    : > "$tmp/meta/upper/Movies/Ghost (2020)/movie.nfo"
+    : > "$tmp/meta/upper/Movies/Keeper (2020)/movie.nfo"
+
+    run bash "$VIEW" reap --apply --upper-base "$tmp/meta" --lower "$tmp/array"
+    [ "$status" -eq 0 ]
+    [ ! -e "$tmp/meta/upper/Movies/Ghost (2020)" ]
+    [ -f "$tmp/meta/upper/Movies/Keeper (2020)/movie.nfo" ]
+    # Recoverable, not deleted: the item lands in a dated trash under the SSD base.
+    [ "$(find "$tmp/meta/.AGENTTRASH" -name movie.nfo | wc -l)" -eq 1 ]
+}

tests/media_relayout.bats

@@ -0,0 +1,623 @@
+#!/usr/bin/env bats
+# Seam: the movie-library relayout — scripts/media-census.py (the read-only
+# census), scripts/media-plan.py (the plan) and scripts/media-apply.py (the only
+# thing that moves files).
+#
+# The library is 8k files on a 100 %-full array, so the two things this seam
+# exists to catch are the ones that cost a re-run of a 20-minute census: a file
+# that ends up in the WRONG movie folder, and a move that destroys something.
+# Everything below runs on a synthetic library in the test's own tmpdir: no
+# ffprobe, no network, no server. The probe step is the one part a fixture
+# cannot run (it opens real containers), so the tests feed the planner the
+# probed TSV in the column order scripts/media-census.py exports — which pins
+# that contract as a side effect.
+#
+# The plan's own gate lines (unplanned, left-unmoved, double-claim, occupied)
+# are asserted as output, because they are how the operator learns the plan is
+# trustworthy before pointing the executor at 24 TB.
+
+load helpers
+
+PLAN="$REPO_ROOT/scripts/media-plan.py"
+CENSUS="$REPO_ROOT/scripts/media-census.py"
+APPLY="$REPO_ROOT/scripts/media-apply.py"
+TV="$REPO_ROOT/scripts/media-tv.py"
+
+# probe_row FILE DUR [RES] [AUDIO] [SUBS] — one probed-TSV row: the twelve
+# columns media-census.py probe writes (bytes, path, dur, vcodec, res, fps,
+# vbit, pix, audio, subs, spare, fmt).
+probe_row() {
+    local p="$1" dur="$2" res="${3:-1920x1080}" aud="${4:-aac+aac}" subs="${5:-0}"
+    printf '%s\t%s\t%s\thevc\t%s\t23.976\t5000000\t8bit\t%s\t%s\t0\tmatroska,webm\n' \
+        "$(stat -c %s "$p")" "$p" "$dur" "$res" "$aud" "$subs" >> "$PROBED"
+}
+
+# plex_row FILE TITLE YEAR — one Plex metadata row; the planner reads these
+# three columns and treats a row without a plausible year as no match at all.
+plex_row() {
+    printf '%s\t%s\t%s\n' "$1" "$2" "$3" >> "$PLEX"
+}
+
+# at PATH SIZE — a file with deterministic content, big enough to matter.
+at() { mkdir -p "$(dirname "$1")"; head -c "$2" /dev/zero > "$1"; }
+
+setup() {
+    LIB="$BATS_TEST_TMPDIR/Movies"
+    ENUM="$BATS_TEST_TMPDIR/enum.tsv"
+    PROBED="$BATS_TEST_TMPDIR/probed.tsv"
+    PLEX="$BATS_TEST_TMPDIR/plex.tsv"
+    OUT="$BATS_TEST_TMPDIR/plan.tsv"
+    EXC="$BATS_TEST_TMPDIR/exceptions.txt"
+    TVLIB="$BATS_TEST_TMPDIR/TVShows"
+    TVPLAN="$BATS_TEST_TMPDIR/tv.tsv"
+    TVJ="$BATS_TEST_TMPDIR/tv.jsonl"
+    : > "$PROBED"
+    printf 'file\tleaf_title\tleaf_year\n' > "$PLEX"
+
+    # Two films that share a box set and a word, plus their bonus material:
+    # the shape that made one film's featurettes land in another's folder.
+    at "$LIB/Alien (1979)/Alien.1979.1080p.BluRay.x265.10bit.mkv" 4000
+    at "$LIB/Alien (1979)/Featurettes/A Bug And A Bite - Making Alien.mkv" 3300000
+    at "$LIB/Alien (1979)/Extras/Extra - Logo.mkv" 900000
+    at "$LIB/Alien (1979)/subs/3_Eng.srt" 40000
+    at "$LIB/Marnie (1964)/Marnie.1964.1080p.BluRay.x265.mkv" 5000000
+    at "$LIB/Marnie (1964)/Featurettes/The Trouble With Marnie.mkv" 2400000
+    # A long extra: over 45 minutes, so only the folder may call it bonus.
+    at "$LIB/Marnie (1964)/Featurettes/Commentary And Archives.mkv" 2600000
+    # A misspelled bonus folder: no bonus vocabulary, so only the folder that
+    # holds the feature can decide that this is not a film.
+    at "$LIB/Marnie (1964)/Exrtas/Marnie - A Closer Look.mkv" 2500000
+    # A Plex title that carries "/" and a Plex row that was never matched.
+    at "$LIB/VHS 2 (2013)/VHS.2.2013.1080p.BluRay.x265.mkv" 4200000
+    at "$LIB/AMERICAN POP/AMERICAN.POP.2023.1080p.WEB-DL.mkv" 3100000
+    # Junk the layout has to park somewhere other than a movie folder.
+    at "$LIB/.DS_Store" 6148
+    at "$LIB/Alien (1979)/._Alien.1979.1080p.BluRay.x265.10bit.mkv" 220
+    # A bonus file whose name is a number: never a movie folder named "1".
+    at "$LIB/Bruce Lee Collection 1971-1981/.Extra 1.mkv" 2700000
+    # A TV box set sitting in the movie library. This one names each file
+    # "Show - SxxEyy - Episode Title", so the episode title comes AFTER the
+    # marker; that convention is what made 124 episodes of one series look like
+    # 124 different films, each under its own threshold.
+    for e in 01 02 03 04 05; do
+        at "$LIB/Monk 2002 Seasons 1 to 8/Season 1/Monk - S01E$e - Mr Monk and the Rival.mkv" 3000000
+    done
+    # The same trouble named the other way round, with no episode title at all.
+    for e in 01 02 03 04 05; do
+        at "$LIB/Big Little Lies Season 1/Big Little Lies S01E$e.mp4" 3000000
+    done
+    # A film whose own name carries a bare "Part 2": one episode-shaped token is
+    # not a series, and it must not be dragged into the TV pile.
+    at "$LIB/The Godfather Part 2 (1974)/The.Godfather.Part.2.1974.1080p.BluRay.mkv" 9000000
+
+    probe_row "$LIB/Alien (1979)/Alien.1979.1080p.BluRay.x265.10bit.mkv" 4700
+    probe_row "$LIB/Alien (1979)/Featurettes/A Bug And A Bite - Making Alien.mkv" 2700
+    probe_row "$LIB/Alien (1979)/Extras/Extra - Logo.mkv" 30
+    probe_row "$LIB/Marnie (1964)/Marnie.1964.1080p.BluRay.x265.mkv" 4800
+    probe_row "$LIB/Marnie (1964)/Featurettes/The Trouble With Marnie.mkv" 1200
+    probe_row "$LIB/Marnie (1964)/Featurettes/Commentary And Archives.mkv" 3000
+    probe_row "$LIB/Marnie (1964)/Exrtas/Marnie - A Closer Look.mkv" 2600
+    probe_row "$LIB/VHS 2 (2013)/VHS.2.2013.1080p.BluRay.x265.mkv" 5700
+    probe_row "$LIB/AMERICAN POP/AMERICAN.POP.2023.1080p.WEB-DL.mkv" 6300
+    probe_row "$LIB/Bruce Lee Collection 1971-1981/.Extra 1.mkv" 5220
+    for e in 01 02 03 04 05; do
+        probe_row "$LIB/Monk 2002 Seasons 1 to 8/Season 1/Monk - S01E$e - Mr Monk and the Rival.mkv" 2600
+        probe_row "$LIB/Big Little Lies Season 1/Big Little Lies S01E$e.mp4" 3300
+    done
+    probe_row "$LIB/The Godfather Part 2 (1974)/The.Godfather.Part.2.1974.1080p.BluRay.mkv" 6200
+    plex_row "$LIB/Alien (1979)/Alien.1979.1080p.BluRay.x265.10bit.mkv" "Alien" "1979"
+    plex_row "$LIB/Marnie (1964)/Marnie.1964.1080p.BluRay.x265.mkv" "Marnie" "1964"
+    plex_row "$LIB/VHS 2 (2013)/VHS.2.2013.1080p.BluRay.x265.mkv" "V/H/S/2" "2013"
+    # An unmatched Plex row: the leaf title echoes the file and the year is 0.
+    plex_row "$LIB/AMERICAN POP/AMERICAN.POP.2023.1080p.WEB-DL.mkv" "AMERICAN POP" "0"
+    plex_row "$LIB/The Godfather Part 2 (1974)/The.Godfather.Part.2.1974.1080p.BluRay.mkv" \
+        "The Godfather Part 2" "1974"
+
+    export MEDIA_ROOT="$LIB"
+    run python3 "$CENSUS" enumerate --root "$LIB" --out "$ENUM"
+    [ "$status" -eq 0 ]
+}
+
+# plan — run the planner over the fixture, keeping its plan TSV, its exception
+# report and the gate lines it prints (that stdout is the operator's evidence
+# that the plan accounts for every file, so the tests read it too).
+plan() {
+    PLANOUT="$BATS_TEST_TMPDIR/plan.out"
+    run python3 "$PLAN" --root "$LIB" --enum "$ENUM" --probed "$PROBED" \
+        --plex "$PLEX" --out "$OUT" --exceptions "$EXC"
+    [ "$status" -eq 0 ]
+    printf '%s\n' "$output" > "$PLANOUT"
+}
+
+# dst_of SRC — the destination the plan gives one file.
+dst_of() {
+    python3 -c '
+import sys
+for line in open(sys.argv[1]):
+    f = line.rstrip("\n").split("\t")
+    if len(f) > 2 and f[1] == sys.argv[2]:
+        print(f[2]); break
+' "$OUT" "$1"
+}
+
+# tv_plan — the TV mover over the fixture library; its stdout is the operator's
+# census of what it called a series and what it refused to touch.
+tv_plan() {
+    run python3 "$TV" --root "$LIB" --library "$TVLIB" --out "$TVPLAN"
+    [ "$status" -eq 0 ]
+    printf '%s\n' "$output" > "$BATS_TEST_TMPDIR/tv.out"
+}
+
+# tv_apply — that plan through the same executor. The destination is a sibling
+# share, so the plan is applied at the root that holds both libraries.
+tv_apply() {
+    run python3 "$APPLY" apply --plan "$TVPLAN" --root "$BATS_TEST_TMPDIR" \
+        --journal "$TVJ" --apply
+    [ "$status" -eq 0 ]
+}
+
+# tv_dst_of SRC — the destination the TV plan gives one file, empty if none.
+tv_dst_of() {
+    python3 -c '
+import sys
+for line in open(sys.argv[1]):
+    f = line.rstrip("\n").split("\t")
+    if len(f) > 2 and f[1] == sys.argv[2]:
+        print(f[2]); break
+' "$TVPLAN" "$1"
+}
+
+@test "a film lands in Title (year)/Title (year).mkv, named by Plex" {
+    plan
+    [ "$(dst_of "$LIB/Alien (1979)/Alien.1979.1080p.BluRay.x265.10bit.mkv")" \
+        = "$LIB/Alien (1979)/Alien (1979).mkv" ]
+}
+
+@test "a Plex title that carries a slash becomes one folder, not a nested tree" {
+    plan
+    local d; d="$(dst_of "$LIB/VHS 2 (2013)/VHS.2.2013.1080p.BluRay.x265.mkv")"
+    [ "$(dirname "$d")" = "$LIB/V-H-S-2 (2013)" ]
+    [ ! -e "$LIB/V" ]
+    # the folder must be ONE mkdir: an un-sanitized name would make the
+    # executor build Movies/V/H/S/2 (2013) instead
+    run grep -F "mkdir" "$OUT"
+    [[ "$output" == *"$(printf '%s' "$LIB/V-H-S-2 (2013)")"* ]]
+    run awk -F'\t' '$1=="mkdir"{print $3}' "$OUT"
+    [[ "$output" != *"$LIB/V/"* ]]
+}
+
+@test "an unmatched Plex row never names a folder with a zero year" {
+    plan
+    run awk -F'\t' '{print $3}' "$OUT"
+    [[ "$output" != *"(0)"* ]]
+    # the filename's own year carries the folder, the folder and the file share
+    # the name (Jellyfin's rule), and the film sits at the library root rather
+    # than in an unsorted pile
+    local d; d="$(dst_of "$LIB/AMERICAN POP/AMERICAN.POP.2023.1080p.WEB-DL.mkv")"
+    [ "$(dirname "$(dirname "$d")")" = "$LIB" ]
+    [[ "$(basename "$(dirname "$d")")" == *" (2023)" ]]
+    [ "$(basename "$d")" = "$(basename "$(dirname "$d")").mkv" ]
+}
+
+@test "a bonus folder keeps its featurettes inside its own film, not a neighbour's" {
+    plan
+    local d
+    d="$(dst_of "$LIB/Alien (1979)/Featurettes/A Bug And A Bite - Making Alien.mkv")"
+    [ "$(dirname "$(dirname "$d")")" = "$LIB/Alien (1979)" ]
+    d="$(dst_of "$LIB/Marnie (1964)/Featurettes/The Trouble With Marnie.mkv")"
+    [ "$(dirname "$(dirname "$d")")" = "$LIB/Marnie (1964)" ]
+    d="$(dst_of "$LIB/Alien (1979)/Extras/Extra - Logo.mkv")"
+    [ "$(dirname "$(dirname "$d")")" = "$LIB/Alien (1979)" ]
+    # the feature-length extra is bonus too: it sits in a bonus folder and no
+    # metadata matched it, so it may not become a movie of its own
+    d="$(dst_of "$LIB/Marnie (1964)/Featurettes/Commentary And Archives.mkv")"
+    [ "$(dirname "$(dirname "$d")")" = "$LIB/Marnie (1964)" ]
+}
+
+@test "a folder inside a movie folder is bonus even when its name is misspelled" {
+    plan
+    local d; d="$(dst_of "$LIB/Marnie (1964)/Exrtas/Marnie - A Closer Look.mkv")"
+    [ "$(dirname "$(dirname "$d")")" = "$LIB/Marnie (1964)" ]
+}
+
+@test "a subtitle follows the film of the folder it sits in" {
+    plan
+    local d; d="$(dst_of "$LIB/Alien (1979)/subs/3_Eng.srt")"
+    [ "$(dirname "$(dirname "$d")")" = "$LIB/Alien (1979)" ]
+}
+
+@test "macOS sidecars are parked in one pile, never inside a movie folder" {
+    plan
+    local d; d="$(dst_of "$LIB/Alien (1979)/._Alien.1979.1080p.BluRay.x265.10bit.mkv")"
+    [[ "$(dirname "$d")" == "$LIB"/_* ]]
+    d="$(dst_of "$LIB/.DS_Store")"
+    [[ "$(dirname "$d")" == "$LIB"/_* ]]
+    grep -q "AppleDouble" "$EXC"
+}
+
+@test "a file whose name is a number becomes a review line, not a folder named 1" {
+    plan
+    run awk -F'\t' '$1=="mkdir"{print $3}' "$OUT"
+    [[ "$output" != "$LIB/1"$'\n'* ]] && [[ "$output" != "$LIB/1" ]]
+    [[ "$output" != *"$LIB/1/"* ]]
+    grep -q "UNNAMED" "$EXC"
+}
+
+@test "a box set in the movie library is quarantined, not filed as movies" {
+    plan
+    local d
+    d="$(dst_of "$LIB/Monk 2002 Seasons 1 to 8/Season 1/Monk - S01E01 - Mr Monk and the Rival.mkv")"
+    [[ "$d" == "$LIB"/_TV\ needs\ a\ TV\ library\ -\ monk* ]]
+    d="$(dst_of "$LIB/Big Little Lies Season 1/Big Little Lies S01E01.mp4")"
+    [[ "$d" == "$LIB"/_TV\ needs\ a\ TV\ library\ -\ big\ little\ lies* ]]
+    # one series is one pile, whatever its episodes are called
+    run python3 -c '
+import re, sys
+pile = {}
+for line in open(sys.argv[1]):
+    f = line.rstrip("\n").split("\t")
+    if len(f) > 2 and f[0] == "move":
+        m = re.search(r"/_TV needs a TV library - ([^/]*)/", f[2] + "/")
+        if m:
+            pile[m.group(1)] = pile.get(m.group(1), 0) + 1
+for k in sorted(pile):
+    print(f"{k}={pile[k]}")
+' "$OUT"
+    [ "${#lines[@]}" -eq 2 ]
+    [[ "${lines[0]}" == big\ little\ lies*=5 ]]
+    [[ "${lines[1]}" == monk*=5 ]]
+}
+
+@test "a film named Part 2 is not called a series" {
+    plan
+    [ "$(dst_of "$LIB/The Godfather Part 2 (1974)/The.Godfather.Part.2.1974.1080p.BluRay.mkv")" \
+        = "$LIB/The Godfather Part 2 (1974)/The Godfather Part 2 (1974).mkv" ]
+}
+
+@test "the TV mover files a box set into the TV library in Jellyfin shape" {
+    tv_plan
+    tv_apply
+    [ -e "$TVLIB/Monk (2002)/Season 01/Monk - S01E01 - Mr Monk and the Rival.mkv" ]
+    [ -e "$TVLIB/Big Little Lies/Season 01/Big Little Lies S01E01.mp4" ]
+    [ ! -e "$LIB/Monk 2002 Seasons 1 to 8/Season 1/Monk - S01E01 - Mr Monk and the Rival.mkv" ]
+    # films are not the TV mover's business
+    [ -e "$LIB/Alien (1979)/Alien.1979.1080p.BluRay.x265.10bit.mkv" ]
+}
+
+@test "the TV mover leaves a film whose name only looks like an episode" {
+    tv_plan
+    tv_apply
+    local gf="$LIB/The Godfather Part 2 (1974)/The.Godfather.Part.2.1974.1080p.BluRay.mkv"
+    [ "$(tv_dst_of "$gf")" = "" ]
+    [ -e "$gf" ]
+    grep -q "Godfather.*not a series" "$BATS_TEST_TMPDIR/tv.out"
+}
+
+@test "the TV mover keeps an episode's own subtitle with the episode" {
+    at "$LIB/Monk 2002 Seasons 1 to 8/Season 1/Monk - S01E01 - Mr Monk and the Rival.srt" 40000
+    tv_plan
+    tv_apply
+    [ -e "$TVLIB/Monk (2002)/Season 01/Monk - S01E01 - Mr Monk and the Rival.srt" ]
+}
+
+@test "a second TV move finds nothing left to move" {
+    tv_plan
+    tv_apply
+    tv_plan
+    grep -q "0 moves" "$BATS_TEST_TMPDIR/tv.out"
+}
+
+@test "a show whose own name carries the year is not given it twice" {
+    local d="$LIB/Leah Remini Scientology and the Aftermath (2016)/Season 1"
+    for e in 01 02 03 04 05; do
+        at "$d/Leah Remini Scientology and the Aftermath (2016) - S01E$e - Episode.mkv" 3000000
+    done
+    tv_plan
+    tv_apply
+    [ -e "$TVLIB/Leah Remini Scientology and the Aftermath (2016)/Season 01/Leah Remini Scientology and the Aftermath (2016) - S01E01 - Episode.mkv" ]
+    [ ! -e "$TVLIB/Leah Remini Scientology and the Aftermath (2016) (2016)" ]
+}
+
+@test "an all-lowercase series name is written the way the library writes names" {
+    for e in 01 02 03 04 05; do
+        at "$LIB/downton abbey 2010/Season 1/downton abbey - s01e$e - episode title.mkv" 3000000
+    done
+    tv_plan
+    tv_apply
+    # the FOLDER is titled; the file keeps the name the library already holds
+    [ -e "$TVLIB/Downton Abbey (2010)/Season 01/downton abbey - s01e01 - episode title.mkv" ]
+}
+
+@test "the mover leaves an episode the TV library already holds" {
+    local src="$LIB/Monk 2002 Seasons 1 to 8/Season 1/Monk - S01E01 - Mr Monk and the Rival.mkv"
+    for e in 01 02 03 04 05; do
+        at "$TVLIB/Monk (2002)/Season 01/Monk - S01E$e - Mr Monk and the Rival.mkv" 3000000
+    done
+    tv_plan
+    [ -z "$(tv_dst_of "$src")" ]
+    [ -e "$src" ]
+    grep -q "Monk.*already in" "$BATS_TEST_TMPDIR/tv.out"
+}
+
+@test "an episode the TV library holds in another encode is still moved" {
+    for e in 01 02 03 04 05; do
+        at "$TVLIB/Big Little Lies/Season 01/Big Little Lies S01E$e 1080p.mp4" 4000000
+    done
+    tv_plan
+    tv_apply
+    [ -e "$TVLIB/Big Little Lies/Season 01/Big Little Lies S01E01.mp4" ]
+    [ -e "$TVLIB/Big Little Lies/Season 01/Big Little Lies S01E01 1080p.mp4" ]
+}
+
+@test "every file in the census is placed or kept — nothing is dropped" {
+    plan
+    python3 - "$ENUM" "$OUT" <<'PY'
+import sys
+enum = {l.rstrip("\n").split("\t")[2] for l in open(sys.argv[1])}
+placed = set()
+for line in open(sys.argv[2]):
+    f = line.rstrip("\n").split("\t")
+    if f[0] in ("move", "keep"):
+        placed.add(f[1])
+missing = enum - placed
+assert not missing, f"{len(missing)} census files with no plan row: {sorted(missing)[:3]}"
+PY
+    grep -q "left-unmoved=0" "$PLANOUT"
+    grep -q "double-claim=0" "$PLANOUT"
+    grep -q "occupied=0" "$PLANOUT"
+    grep -q "bad-names=0" "$PLANOUT"
+    grep -q "unplanned=0" "$EXC"
+}
+
+# ---------------------------------------------------------------- executor
+
+# apply_plan — plan, then apply for real; the journal goes in the tmpdir.
+apply_plan() {
+    JOURNAL="$BATS_TEST_TMPDIR/journal.jsonl"
+    run python3 "$APPLY" apply --plan "$OUT" --root "$LIB" --journal "$JOURNAL" --apply
+}
+
+@test "a run without --apply says what it would do and touches nothing" {
+    plan
+    JOURNAL="$BATS_TEST_TMPDIR/journal.jsonl"
+    run python3 "$APPLY" apply --plan "$OUT" --root "$LIB" --journal "$JOURNAL"
+    [ "$status" -eq 0 ]
+    [[ "$output" == *"--apply"* ]]
+    [ -e "$LIB/Alien (1979)/Alien.1979.1080p.BluRay.x265.10bit.mkv" ]
+    [ ! -e "$LIB/Alien (1979)/Alien (1979).mkv" ]
+    # a dry run may not even build the folders: an operator must be able to
+    # run it against the live library with no --apply and change nothing
+    [ ! -d "$LIB/V-H-S-2 (2013)" ]
+    [ ! -e "$JOURNAL" ]
+}
+
+@test "apply moves every planned file, keeps its bytes, and journals each move" {
+    plan
+    apply_plan
+    [ "$status" -eq 0 ]
+    [ -e "$LIB/Alien (1979)/Alien (1979).mkv" ]
+    [ ! -e "$LIB/Alien (1979)/Alien.1979.1080p.BluRay.x265.10bit.mkv" ]
+    local planned moved
+    planned=$(awk -F'\t' '$1=="move"' "$OUT" | wc -l)
+    moved=$(python3 -c 'import json,sys
+print(sum(1 for l in open(sys.argv[1]) if json.loads(l).get("op") == "move"))' "$JOURNAL")
+    [ "$moved" -eq "$planned" ]
+    # The media servers' sidecars live in the metadata overlay and did not follow
+    # these moves, so the run names that cleanup at the moment it is needed.
+    [[ "$output" == *"media-view.sh reap"* ]]
+    run python3 "$APPLY" verify --plan "$OUT" --root "$LIB" --journal "$JOURNAL"
+    [ "$status" -eq 0 ]
+}
+
+@test "a target already holding a different file stops the whole run" {
+    plan
+    JOURNAL="$BATS_TEST_TMPDIR/journal.jsonl"
+    at "$LIB/Alien (1979)/Alien (1979).mkv" 7
+    run python3 "$APPLY" apply --plan "$OUT" --root "$LIB" --journal "$JOURNAL" --apply
+    [ "$status" -ne 0 ]
+    [[ "$output" == *"occupied"* ]]
+    # nothing moved at all: the other films are still where the census saw them
+    [ -e "$LIB/Marnie (1964)/Marnie.1964.1080p.BluRay.x265.mkv" ]
+    [ ! -e "$JOURNAL" ]
+}
+
+@test "a plan row whose source has gone stops the run instead of inventing a move" {
+    plan
+    JOURNAL="$BATS_TEST_TMPDIR/journal.jsonl"
+    rm "$LIB/Marnie (1964)/Marnie.1964.1080p.BluRay.x265.mkv"
+    run python3 "$APPLY" apply --plan "$OUT" --root "$LIB" --journal "$JOURNAL" --apply
+    [ "$status" -ne 0 ]
+    [[ "$output" == *"missing"* ]]
+    [ -e "$LIB/Alien (1979)/Alien.1979.1080p.BluRay.x265.10bit.mkv" ]
+}
+
+@test "a run interrupted between the journal write and the move finishes the job" {
+    plan
+    JOURNAL="$BATS_TEST_TMPDIR/journal.jsonl"
+    # The journal is written before each move, so an interruption leaves two
+    # states and only two: an intent whose move happened, and an intent whose
+    # move never ran. Both are simulated here by hand.
+    python3 - "$OUT" "$JOURNAL" <<'PY'
+import json, os, sys
+moves = [r for r in (l.rstrip("\n").split("\t") for l in open(sys.argv[1]))
+         if r[0] == "move"][:3]
+with open(sys.argv[2], "w") as fh:
+    for i, r in enumerate(moves):
+        fh.write(json.dumps({"op": "move", "src": r[1], "dst": r[2],
+                             "bytes": os.path.getsize(r[1])}) + "\n")
+        if i < 2:                     # the first two moves actually ran
+            os.makedirs(os.path.dirname(r[2]), exist_ok=True)
+            os.rename(r[1], r[2])
+PY
+    run python3 "$APPLY" apply --plan "$OUT" --root "$LIB" --journal "$JOURNAL" --apply
+    [ "$status" -eq 0 ]
+    run python3 "$APPLY" verify --plan "$OUT" --root "$LIB" --journal "$JOURNAL"
+    [ "$status" -eq 0 ]
+    python3 - "$OUT" "$JOURNAL" <<'PY'
+import json, os, sys
+moves = [r for r in (l.rstrip("\n").split("\t") for l in open(sys.argv[1]))
+         if r[0] == "move"]
+journal = [json.loads(l) for l in open(sys.argv[2])]
+seen = [j["dst"] for j in journal]
+assert len(seen) == len(set(seen)), "a destination was journalled twice"
+for r in moves:
+    assert os.path.exists(r[2]) and not os.path.exists(r[1]), r[2]
+PY
+}
+
+@test "rollback puts every file back where the census found it" {
+    plan
+    apply_plan
+    [ "$status" -eq 0 ]
+    run python3 "$APPLY" rollback --root "$LIB" --journal "$JOURNAL" --apply
+    [ "$status" -eq 0 ]
+    [ -e "$LIB/Alien (1979)/Alien.1979.1080p.BluRay.x265.10bit.mkv" ]
+    [ -e "$LIB/Marnie (1964)/Marnie.1964.1080p.BluRay.x265.mkv" ]
+    [ -e "$LIB/Alien (1979)/subs/3_Eng.srt" ]
+    [ ! -e "$LIB/Alien (1979)/Alien (1979).mkv" ]
+    # the folders the run created are empty afterwards and must not litter the
+    # library: this one never existed before the plan ran
+    [ -d "$LIB/VHS 2 (2013)" ]
+    [ ! -e "$LIB/V-H-S-2 (2013)" ]
+}
+
+@test "rollback refuses when a moved file has gone missing" {
+    plan
+    apply_plan
+    [ "$status" -eq 0 ]
+    rm "$LIB/Alien (1979)/Alien (1979).mkv"
+    run python3 "$APPLY" rollback --root "$LIB" --journal "$JOURNAL" --apply
+    [ "$status" -ne 0 ]
+    [[ "$output" == *"missing"* ]]
+    # the refusal is all-or-nothing: another film did not move back
+    [ -e "$LIB/Marnie (1964)/Marnie (1964).mkv" ]
+}
+
+# ---------------------------------------------------------------- sweep
+#
+# A relayout leaves the release folders it emptied standing at the library root:
+# 1675 of them after the live run, each holding nothing but empty subfolders. The
+# sweep pass removes only folders THIS RUN emptied — proven from the journal,
+# never from "it looks empty" — so a folder someone made by hand is untouchable,
+# and every removal rides the same rollback as the moves.
+
+@test "sweep removes only the folders this run emptied" {
+    plan
+    apply_plan
+    mkdir -p "$LIB/A Empty Placeholder"      # someone's folder, never a move source
+    run python3 "$APPLY" sweep --root "$LIB" --journal "$JOURNAL" --apply
+    [ "$status" -eq 0 ]
+    [[ "$output" == *"swept"* ]]
+    [ ! -e "$LIB/AMERICAN POP" ]
+    [ ! -e "$LIB/VHS 2 (2013)" ]
+    [ ! -e "$LIB/Alien (1979)/Featurettes" ]
+    [ ! -e "$LIB/Alien (1979)/subs" ]
+    [ ! -e "$LIB/Marnie (1964)/Exrtas" ]
+    # a folder the run never moved a file out of survives, empty or not
+    [ -d "$LIB/A Empty Placeholder" ]
+}
+
+@test "a swept folder carries the empty junk folders inside it" {
+    mkdir -p "$LIB/AMERICAN POP/Screens"   # screenshots, never a move source
+    plan
+    apply_plan
+    run python3 "$APPLY" sweep --root "$LIB" --journal "$JOURNAL" --apply
+    [ "$status" -eq 0 ]
+    [[ "$output" == *"0 left in place"* ]]
+    [ ! -e "$LIB/AMERICAN POP/Screens" ]
+    [ ! -e "$LIB/AMERICAN POP" ]
+}
+
+@test "a sweep whose rmdir failed tries those folders again" {
+    plan
+    apply_plan
+    mkdir -p "$LIB/AMERICAN POP/Screens"
+    chmod 500 "$LIB/AMERICAN POP"          # readable, but nothing can be removed
+    run python3 "$APPLY" sweep --root "$LIB" --journal "$JOURNAL" --apply
+    [ "$status" -eq 0 ]
+    [[ "$output" == *"left in place"* ]]
+    [ -e "$LIB/AMERICAN POP/Screens" ]
+    chmod 700 "$LIB/AMERICAN POP"          # the operator cleared the blocker
+    run python3 "$APPLY" sweep --root "$LIB" --journal "$JOURNAL" --apply
+    [ "$status" -eq 0 ]
+    [[ "$output" != *"left in place: "* ]]
+    [ ! -e "$LIB/AMERICAN POP" ]
+}
+
+@test "sweep without --apply reports the husks and changes nothing" {
+    plan
+    apply_plan
+    local before jlines
+    before=$(find "$LIB" -type d | wc -l)
+    jlines=$(wc -l < "$JOURNAL")
+    run python3 "$APPLY" sweep --root "$LIB" --journal "$JOURNAL"
+    [ "$status" -eq 0 ]
+    [[ "$output" == *"--apply"* ]]
+    [ "$(find "$LIB" -type d | wc -l)" -eq "$before" ]
+    [ "$(wc -l < "$JOURNAL")" -eq "$jlines" ]
+    [ -e "$LIB/AMERICAN POP" ]
+}
+
+@test "a folder that still holds content is never swept" {
+    plan
+    apply_plan
+    run python3 "$APPLY" sweep --root "$LIB" --journal "$JOURNAL" --apply
+    [ "$status" -eq 0 ]
+    # the film folder, the folder the run created, and the piles all stay
+    [ -d "$LIB/Marnie (1964)" ]
+    [ -e "$LIB/Marnie (1964)/Marnie (1964).mkv" ]
+    [ -d "$LIB/V-H-S-2 (2013)" ]
+    [ -d "$LIB/_unsorted bonus/extras" ]
+}
+
+@test "rollback after a sweep brings the folders and the files back" {
+    plan
+    apply_plan
+    run python3 "$APPLY" sweep --root "$LIB" --journal "$JOURNAL" --apply
+    [ "$status" -eq 0 ]
+    [ ! -e "$LIB/AMERICAN POP" ]
+    run python3 "$APPLY" rollback --root "$LIB" --journal "$JOURNAL" --apply
+    [ "$status" -eq 0 ]
+    [ -e "$LIB/AMERICAN POP/AMERICAN.POP.2023.1080p.WEB-DL.mkv" ]
+    [ -e "$LIB/VHS 2 (2013)/VHS.2.2013.1080p.BluRay.x265.mkv" ]
+    [ -e "$LIB/Alien (1979)/Featurettes/A Bug And A Bite - Making Alien.mkv" ]
+    [ -e "$LIB/Alien (1979)/subs/3_Eng.srt" ]
+}
+
+@test "a second sweep has nothing left to do and says so" {
+    plan
+    apply_plan
+    run python3 "$APPLY" sweep --root "$LIB" --journal "$JOURNAL" --apply
+    [ "$status" -eq 0 ]
+    local before
+    before=$(find "$LIB" -type d | wc -l)
+    run python3 "$APPLY" sweep --root "$LIB" --journal "$JOURNAL" --apply
+    [ "$status" -eq 0 ]
+    [[ "$output" == *"swept: 0"* ]]
+    [ "$(find "$LIB" -type d | wc -l)" -eq "$before" ]
+}
+
+@test "sweep never follows a symlink and never removes the folder it guards" {
+    plan
+    apply_plan
+    mkdir -p "$LIB/Elsewhere"
+    at "$LIB/Elsewhere/keep.txt" 10
+    ln -s "$LIB/Elsewhere" "$LIB/AMERICAN POP/pointer"
+    run python3 "$APPLY" sweep --root "$LIB" --journal "$JOURNAL" --apply
+    [ "$status" -eq 0 ]
+    # a link is content, not nothing: the husk it sits in survives untouched
+    [ -d "$LIB/AMERICAN POP" ]
+    [ -L "$LIB/AMERICAN POP/pointer" ]
+    [ -e "$LIB/Elsewhere/keep.txt" ]
+}
+
+@test "sweep refuses a journal that belongs to another library" {
+    plan
+    apply_plan
+    mkdir -p "$BATS_TEST_TMPDIR/Other"
+    run python3 "$APPLY" sweep --root "$BATS_TEST_TMPDIR/Other" \
+        --journal "$JOURNAL" --apply
+    [ "$status" -ne 0 ]
+    [[ "$output" == *"outside"* ]]
+    [ -d "$LIB/AMERICAN POP" ]
+}

tests/media_shows.bats

@@ -0,0 +1,307 @@
+#!/usr/bin/env bats
+# Seam: the TV-library relayout — scripts/media-census.py jellyfin (the metadata
+# authority export) and scripts/media-shows.py (the plan).
+#
+# The TV library is 52k files written in five different release layouts, and the
+# one thing that makes a rename of that size survivable is that the plan is
+# derived from what a metadata provider already matched, not from a guess at the
+# file name. So every test here states which authority a file had — name only,
+# provider only, both, neither — and asserts where the PLAN files it, never how
+# the planner decided. A fixture Jellyfin database stands in for the live one and
+# the TMDb cache is an input file, so no test touches the network.
+#
+# The two outcomes that cost a re-run of a 50k-file plan are a file filed as an
+# episode that is not one (it steals an episode number from its real owner) and a
+# move that lands on an existing file. Everything below is written to catch
+# exactly those, plus the gate that matters most: the plan must be replayable by
+# scripts/media-apply.py, whose preflight is the fleet's only overwrite guard.
+
+load helpers
+
+PLAN="$REPO_ROOT/scripts/media-shows.py"
+CENSUS="$REPO_ROOT/scripts/media-census.py"
+APPLY="$REPO_ROOT/scripts/media-apply.py"
+
+# at PATH SIZE — a file with deterministic content.
+at() { mkdir -p "$(dirname "$1")"; head -c "$2" /dev/zero > "$1"; }
+
+# title_row FILE SERIES YEAR SEASON EPISODE [EXTRA] — one row of the authority
+# export scripts/media-census.py jellyfin writes.
+title_row() {
+    printf '%s\t%s\t%s\t%s\t%s\t%s\t\n' "$1" "$2" "$3" "${4:-}" "${5:-}" "${6:-}" \
+        >> "$TITLES"
+}
+
+# dst SRC — the destination the plan gives SRC, or nothing when it does not move.
+dst() {
+    awk -F '\t' -v s="$1" '$1 == "move" && $2 == s {print $3}' "$OUT"
+}
+
+setup() {
+    LIB="$BATS_TEST_TMPDIR/TVShows"
+    PARK="$LIB/_unsorted"
+    MOVIES="$BATS_TEST_TMPDIR/Movies"
+    ENUM="$BATS_TEST_TMPDIR/enum.tsv"
+    PROBED="$BATS_TEST_TMPDIR/probed.tsv"
+    TITLES="$BATS_TEST_TMPDIR/titles.tsv"
+    CACHE="$BATS_TEST_TMPDIR/tmdb.json"
+    OUT="$BATS_TEST_TMPDIR/plan.tsv"
+    EXC="$BATS_TEST_TMPDIR/exceptions.txt"
+    : > "$PROBED"
+
+    SEASON="$LIB/The Examples (2010) Season 1 S01 (1080p BluRay x265 Silence)"
+    # An episode its own name states, with a subtitle, a season image and a
+    # trailer: four things that have to end up in four different places.
+    at "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.mkv" 9000
+    at "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.en.srt" 200
+    at "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.mkv.forced.srt" 250
+    # A scene pack buries its subtitles two levels down, in Subs/<release-name>/.
+    at "$SEASON/Subs/The.Examples.S01E01.Pilot.1080p.x265/The.Examples.S01E01.Pilot.1080p.x265.jpn.srt" 220
+    at "$SEASON/folder.jpg" 300
+    at "$SEASON/Trailers/teaser.mkv" 1000
+    at "$SEASON/tvshow.nfo" 400
+    # An anime pack: no episode marker anywhere in its name, numbered by the
+    # provider only. Its year is the one thing no authority states.
+    ANIME="$LIB/Anime Pack [grp] (1080p HEVC)"
+    at "$ANIME/[grp] Anime Pack - 05.mkv" 5000
+    # Two releases claiming one episode, and a file holding two episodes.
+    at "$LIB/Dupes/Show Two S01E01.mkv" 4000
+    at "$LIB/Dupes/Show.Two.S01E01.720p.other.mkv" 7000
+    at "$LIB/Dupes/Show.Two.S01E02-E03.720p.mkv" 7000
+    # A file no authority numbers, and a long file the movie provider knows.
+    at "$LIB/Orphan Tapes/clip001.mp4" 3000
+    at "$LIB/Longform/Some Documentary Special (2015).mkv" 8000
+    # A bonus folder inside a release pack whose own name is not the show's. The
+    # pack's rows name the show; the bonus folder may never invent a series.
+    at "$ANIME/01. Extras/NCOP 01.mkv" 1200
+    # A season folder whose rows are stale: the database numbers every file of this
+    # pack season 16 episode 20, four seasons' worth of them.
+    at "$LIB/SNL Pack/Season 19/01 Charles Barkley, Nirvana.avi" 6000
+    # One episode-shaped file loose at the library root.
+    at "$LIB/Loose.Show.S02E03.720p.WEB-DL.mkv" 4000
+
+    # The enum census lists every file; the planner reads that instead of walking
+    # the tree, so the fixture writes the same three columns.
+    python3 - "$LIB" "$ENUM" <<'PY'
+import os, sys
+root, out = sys.argv[1], sys.argv[2]
+with open(out, "w") as fh:
+    for d, _, fs in os.walk(root):
+        for f in fs:
+            p = os.path.join(d, f)
+            fh.write(f"{os.stat(p).st_size}\t{os.path.splitext(f)[1]}\t{p}\n")
+PY
+
+    printf 'file\tseries\tyear\tseason\tepisode\textra\ttmdb\n' > "$TITLES"
+    printf '{}\n' > "$CACHE"
+    title_row "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.mkv" \
+        "The Examples" 2010 1 1
+    title_row "$ANIME/[grp] Anime Pack - 05.mkv" "Anime Pack" "" 1 5
+    title_row "$LIB/Dupes/Show Two S01E01.mkv" "Show Two" 2016 1 1
+    title_row "$LIB/Dupes/Show.Two.S01E01.720p.other.mkv" "Show Two" 2016 1 1
+    title_row "$LIB/Dupes/Show.Two.S01E02-E03.720p.mkv" "Show Two" 2016 1 2
+    title_row "$LIB/Loose.Show.S02E03.720p.WEB-DL.mkv" "Loose Show" 2019 2 3
+    title_row "$LIB/SNL Pack/Season 19/01 Charles Barkley, Nirvana.avi" \
+        "SNL Pack" "" 16 20
+}
+
+plan() {
+    run python3 "$PLAN" --root "$LIB" --enum "$ENUM" --titles "$TITLES" \
+        --probed "$PROBED" --tmdb-cache "$CACHE" --movies "$MOVIES" \
+        --out "$OUT" --exceptions "$EXC"
+}
+
+@test "the census exports the provider's episode numbering as a TSV" {
+    DB="$BATS_TEST_TMPDIR/jellyfin.db"
+    python3 - "$DB" "$SEASON" <<'PY'
+import os, sqlite3, sys, uuid
+db, season = sys.argv[1], sys.argv[2]
+c = sqlite3.connect(db)
+pre = "/data/TVShows/" + os.path.basename(season)
+c.execute("""create table BaseItems (Id, ParentId, Type, Name, Path, IndexNumber,
+                                      ParentIndexNumber, ProductionYear, ExtraType,
+                                      IsVirtualItem, Data)""")
+sid = str(uuid.uuid4())
+c.execute("insert into BaseItems values (?,?,?,?,?,?,?,?,?,?,?)",
+          (sid, None, "MediaBrowser.Controller.Entities.TV.Series", "The Examples",
+           pre, None, None, 2010, None, 0,
+           '{"ProviderIds":{"Tmdb":"1234"}}'))
+c.execute("insert into BaseItems values (?,?,?,?,?,?,?,?,?,?,?)",
+          (str(uuid.uuid4()), sid, "MediaBrowser.Controller.Entities.TV.Episode",
+           "Pilot", pre + "/The.Examples.S01E01.Pilot.1080p.x265.mkv", 1, 1,
+           None, None, 0, '{}'))
+c.commit()
+PY
+    run python3 "$CENSUS" jellyfin --root "$LIB" --db "$DB" \
+        --out "$BATS_TEST_TMPDIR/exp.tsv"
+    [ "$status" -eq 0 ]
+    run head -1 "$BATS_TEST_TMPDIR/exp.tsv"
+    [ "${lines[0]}" = "file	series	year	season	episode	extra	tmdb" ]
+    tail -1 "$BATS_TEST_TMPDIR/exp.tsv" \
+        | grep -q "^$SEASON/The\.Examples\.S01E01\.Pilot\.1080p\.x265\.mkv	The Examples	2010	1	1		1234$"
+}
+
+@test "an episode its own name states files as Show SxxEyy under Season NN" {
+    plan
+    [ "$status" -eq 0 ]
+    [ "$(dst "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.mkv")" \
+        = "$LIB/The Examples (2010)/Season 01/The Examples S01E01.mkv" ]
+}
+
+@test "a subtitle follows its episode and keeps its language flag" {
+    plan
+    [ "$(dst "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.en.srt")" \
+        = "$LIB/The Examples (2010)/Season 01/The Examples S01E01.en.srt" ]
+}
+
+@test "a subtitle two folders down inside Subs/<release-name>/ still follows its episode" {
+    plan
+    [ "$(dst "$SEASON/Subs/The.Examples.S01E01.Pilot.1080p.x265/The.Examples.S01E01.Pilot.1080p.x265.jpn.srt")" \
+        = "$LIB/The Examples (2010)/Season 01/The Examples S01E01.jpn.srt" ]
+}
+
+@test "a subtitle that carries the container's word in its own name still follows the episode" {
+    plan
+    [ "$(dst "$SEASON/The.Examples.S01E01.Pilot.1080p.x265.mkv.forced.srt")" \
+        = "$LIB/The Examples (2010)/Season 01/The Examples S01E01.forced.srt" ]
+}
+
+@test "a season image keeps its own name; a series nfo keeps the series folder" {
+    plan
+    [ "$(dst "$SEASON/folder.jpg")" = "$LIB/The Examples (2010)/Season 01/folder.jpg" ]
+    [ "$(dst "$SEASON/tvshow.nfo")" = "$LIB/The Examples (2010)/tvshow.nfo" ]
+}
+
+@test "bonus material files as an extra, never as an episode" {
+    plan
+    [ "$(dst "$SEASON/Trailers/teaser.mkv")" = "$LIB/The Examples (2010)/trailers/teaser.mkv" ]
+    [ -z "$(awk -F '\t' '$3 ~ /Season .*teaser/ {print $3}' "$OUT")" ]
+}
+
+@test "a file only the provider numbers files at the episode it is matched to" {
+    plan
+    [ "$(dst "$ANIME/[grp] Anime Pack - 05.mkv")" \
+        = "$LIB/Anime Pack/Season 01/Anime Pack S01E05.mkv" ]
+}
+
+@test "a bonus folder inside a release pack belongs to the pack's show, not to its own name" {
+    plan
+    [ "$(dst "$ANIME/01. Extras/NCOP 01.mkv")" = "$LIB/Anime Pack/extras/NCOP 01.mkv" ]
+    [ -z "$(awk -F '\t' '$3 ~ /\/01\. Extras\// {print $3}' "$OUT")" ]
+}
+
+@test "a Season folder outvotes the stale row its files carry" {
+    plan
+    [ "$(dst "$LIB/SNL Pack/Season 19/01 Charles Barkley, Nirvana.avi")" \
+        = "$LIB/SNL Pack/Season 19/SNL Pack S19E01.avi" ]
+}
+
+
+@test "one file holding two episodes keeps both numbers in its name" {
+    plan
+    [ "$(dst "$LIB/Dupes/Show.Two.S01E02-E03.720p.mkv")" \
+        = "$LIB/Show Two (2016)/Season 01/Show Two S01E02-E03.mkv" ]
+}
+
+@test "two files claiming one episode: one keeps the name, the other parks" {
+    plan
+    n=$(grep -c "	$LIB/Show Two (2016)/Season 01/Show Two S01E01.mkv	" "$OUT")
+    [ "$n" -eq 1 ]
+    run dst "$LIB/Dupes/Show Two S01E01.mkv"
+    [[ "$output" == "$PARK/"* ]]
+    run dst "$LIB/Dupes/Show.Two.S01E01.720p.other.mkv"
+    [[ "$output" == "$LIB/Show Two (2016)/Season 01/"* ]]
+}
+
+@test "a file no authority numbers is parked, never renamed into the library" {
+    plan
+    run dst "$LIB/Orphan Tapes/clip001.mp4"
+    [[ "$output" == "$PARK/Orphan Tapes/clip001.mp4" ]]
+    grep -q "clip001.mp4" "$EXC"
+}
+
+@test "a long file the movie provider knows leaves the TV library for the movie one" {
+    printf '8000\t%s\t7200\thevc\t1920x1080\t23.976\t4000000\tyuv420p10le\taac:eng\t0\t0\tmatroska,webm\n' \
+        "$LIB/Longform/Some Documentary Special (2015).mkv" > "$PROBED"
+    python3 -c 'import json, sys
+json.dump({"movie:some documentary":
+           {"name": "Some Documentary Special", "year": 2015, "id": 55}},
+          open(sys.argv[1], "w"))' "$CACHE"
+    plan
+    [ "$(dst "$LIB/Longform/Some Documentary Special (2015).mkv")" \
+        = "$MOVIES/Some Documentary Special (2015)/Some Documentary Special (2015).mkv" ]
+}
+
+@test "a long file no movie provider knows is parked, not moved into Movies" {
+    printf '8000\t%s\t7200\thevc\t1920x1080\t23.976\t4000000\tyuv420p10le\taac:eng\t0\t0\tmatroska,webm\n' \
+        "$LIB/Longform/Some Documentary Special (2015).mkv" > "$PROBED"
+    plan
+    run dst "$LIB/Longform/Some Documentary Special (2015).mkv"
+    [[ "$output" == "$PARK/"* ]]
+}
+
+@test "the plan creates every folder before the move that needs it" {
+    plan
+    python3 - "$OUT" "$LIB" "$MOVIES" "$PARK" <<'PY'
+import os, sys
+plan, known = sys.argv[1], set(sys.argv[2:])
+made, bad = set(), []
+for line in open(plan).read().splitlines()[1:]:
+    action, _src, dest, _kind = line.split("\t")
+    if action == "mkdir":
+        made.add(dest)
+    elif action == "move":
+        d = os.path.dirname(dest)
+        while d not in known and d != os.path.dirname(d):
+            if d not in made:
+                bad.append(f"{dest}: {d} was never made")
+            d = os.path.dirname(d)
+print("\n".join(bad[:5]))
+sys.exit(1 if bad else 0)
+PY
+}
+
+@test "no move lands outside the library or the movie library" {
+    plan
+    run awk -F '\t' -v l="$LIB" -v m="$MOVIES" \
+        '$1 == "move" && $3 !~ "^" l "/" && $3 !~ "^" m "/" {print $3}' "$OUT"
+    [ -z "$output" ]
+}
+
+@test "the executor preflights the plan and no target is claimed twice" {
+    plan
+    run python3 "$APPLY" apply --plan "$OUT" --root "$BATS_TEST_TMPDIR" \
+        --journal "$BATS_TEST_TMPDIR/journal.jsonl"
+    [ "$status" -eq 0 ]
+    dup=$(awk -F '\t' '$1 == "move" {print $3}' "$OUT" | sort | uniq -d)
+    [ -z "$dup" ]
+}
+
+@test "re-planning a library the plan already satisfies moves nothing" {
+    plan
+    # Applying the plan and re-censusing is exactly what the operator does next:
+    # every authority file is keyed by path, so all three are replayed through the
+    # plan the way a fresh census would see them. (The enum has no header; the
+    # titles export does, and its header row is not a path so it survives.)
+    python3 - "$OUT" "$ENUM" "$TITLES" "$PROBED" <<'PY'
+import os, sys
+plan = sys.argv[1]
+moves = {}
+for line in open(plan).read().splitlines()[1:]:
+    action, src, dest, _kind = line.split("\t")
+    if action == "move":
+        moves[src] = dest
+        os.makedirs(os.path.dirname(dest), exist_ok=True)
+        os.rename(src, dest)
+for path, col in ((sys.argv[2], 2), (sys.argv[3], 0), (sys.argv[4], 1)):
+    if not os.path.getsize(path):
+        continue
+    rows = [line.rstrip("\n").split("\t") for line in open(path)]
+    with open(path, "w") as fh:
+        for f in rows:
+            f[col] = moves.get(f[col], f[col])
+            fh.write("\t".join(f) + "\n")
+PY
+    plan
+    [ -z "$(awk -F '\t' '$1 == "move" {print $2}' "$OUT")" ]
+}

tests/qwen38_campaign.bats

@@ -200,6 +200,9 @@ DERIVE="$REPO_ROOT/scripts/derive-protected-experts.py"
 APPLY="$REPO_ROOT/scripts/apply-protected-cast.sh"
 CONVERT="$REPO_ROOT/scripts/convert-qwen38-w5nl64.sh"
 GATE="$REPO_ROOT/scripts/kld-gate-qwen38.sh"
+RUNNER="$REPO_ROOT/scripts/run-qwen38-bf16.sh"
+PUSHER="$REPO_ROOT/scripts/calib-push.py"
+WINDOW="$REPO_ROOT/scripts/qwen38-calib-window.sh"
 TEMPLATE="$REPO_ROOT/scripts/recipes/qwen4exp-qwen38-w5nl64-bf16trunk.recipe"
 
 # make_gram <dir> <layer> <expert> <energy> <rows> — a RADGRAM2 down-Gram whose
@@ -525,3 +528,411 @@ gate_case() { # gate_case <kl fp8> <kl iq4r> <kl cand> <top1 cand> [extra flags.
     [[ "$output" == *"has no slice 'code'"* ]]
     [[ "$output" == *"it has: ops"* ]]
 }
+
+# --- the n-gram tier: the pin that starved the calibration boot ---------------
+
+@test "the calibration boot reads the n-gram table from the file tier" {
+    # 2026-10-29: 95.37 GiB pinned beside a 65.59 GiB/rank pool left 8.06 GiB of
+    # RAM and the boot died "startup stopped: out of memory". calib only needs the
+    # routed weights prefilled, so it reads the table off the container's file tier.
+    run bash "$RUNNER" calib --dry-run
+    [ "$status" -eq 0 ]
+    grep -A1 -x -- '--ngram-placement' <<<"$output" | grep -qx 'disk'
+}
+
+@test "the recording and every score pass keep production's n-gram placement" {
+    # Those numbers are only comparable when the shape is identical; the tier is
+    # part of the shape.
+    run bash "$RUNNER" kldref --dry-run
+    [ "$status" -eq 0 ]
+    grep -A1 -x -- '--ngram-placement' <<<"$output" | grep -qx 'ram'
+}
+
+@test "an unknown n-gram placement is refused by name before the boot" {
+    run env FRANK_QWEN38_NGRAM_PLACEMENT=ssd bash "$RUNNER" calib --dry-run
+    [ "$status" -eq 2 ]
+    [[ "$output" == *"auto|vram|ram|disk"* ]]
+}
+
+# --- the pusher: a tap without a live rig is not a tap ------------------------
+
+# stub_podman <dir> <running|absent> — a podman that answers the one question the
+# pusher asks: is the rig container running?
+stub_podman() {
+    mkdir -p "$1/bin"
+    if [ "$2" = running ]; then
+        printf '#!/usr/bin/env bash\necho true\n' > "$1/bin/podman"
+    elif [ "$2" = stopped ]; then
+        printf '#!/usr/bin/env bash\necho false\n' > "$1/bin/podman"
+    else
+        printf '#!/usr/bin/env bash\nexit 125\n' > "$1/bin/podman"
+    fi
+    chmod +x "$1/bin/podman"
+}
+
+pusher_fixture() { # pusher_fixture <marker-container or empty>
+    D="$BATS_TEST_TMPDIR/push"
+    mkdir -p "$D/tap" "$D/bin"
+    printf '{"prompt":"hello"}\n' > "$D/corpus.jsonl"
+    if [ -n "$1" ]; then
+        printf 'container=%s\nimage=radiance@sha256:test\nmode=calib\npid=1\n' "$1" \
+            > "$D/tap/.rig-boot"
+    fi
+    echo "$D"
+}
+
+@test "the pusher refuses a tap that carries no rig marker: production is not a rig" {
+    # With no marker, the old driver fed four rounds of 10M tokens into restored
+    # production after the rig's boot had died, and recorded nothing.
+    D="$(pusher_fixture '')"
+    run python3 "$PUSHER" --corpus "$D/corpus.jsonl" --tap-dir "$D/tap" --port 1
+    [ "$status" -eq 2 ]
+    [[ "$output" == *"no rig marker"* ]]
+}
+
+@test "the pusher refuses a marker that names another container" {
+    D="$(pusher_fixture arbench)"
+    run python3 "$PUSHER" --corpus "$D/corpus.jsonl" --tap-dir "$D/tap" \
+        --container qwen38-bf16 --port 1
+    [ "$status" -eq 2 ]
+    [[ "$output" == *"names container 'arbench'"* ]]
+    [[ "$output" == *"targets 'qwen38-bf16'"* ]]
+}
+
+@test "the pusher refuses when the rig container is not running" {
+    D="$(pusher_fixture qwen38-bf16)"
+    stub_podman "$D" stopped
+    run env PATH="$D/bin:$PATH" python3 "$PUSHER" --corpus "$D/corpus.jsonl" \
+        --tap-dir "$D/tap" --container qwen38-bf16 --port 1
+    [ "$status" -eq 2 ]
+    [[ "$output" == *"is not running"* ]]
+}
+
+# --- the acceptance gate without an FP8 anchor -------------------------------
+
+# kld_report_slices <path> <model> <kl all> <kl code> <top1 all> <top1 code>
+kld_report_slices() {
+    python3 - "$@" <<'PY'
+import json, sys
+path, model, kl_all, kl_code, t_all, t_code = sys.argv[1:7]
+def g(kl, t, n=25000):
+    return {"kld": {"mean": float(kl), "p99_9": float(kl) * 4},
+            "top1_agreement": float(t), "positions": n}
+json.dump({"model": model, "all": g(kl_all, t_all),
+           "by_source": {"code": g(kl_code, t_code), "ops": g(kl_all, t_all)}},
+          open(path, "w"))
+PY
+}
+
+gate2() { # gate2 <kl iq4r> <kl cand> <top1 iq4r> <top1 cand> [extra]
+    D="$BATS_TEST_TMPDIR/kl2"
+    mkdir -p "$D"
+    kld_report_slices "$D/iq4r.json" iq4r-moe "$1" "$1" "$3" "$3"
+    kld_report_slices "$D/cand.json" w5nl64 "$2" "$2" "$4" "$4"
+    run bash "$GATE" "$D/iq4r.json" "$D/cand.json" "${@:5}"
+}
+
+@test "a candidate that beats the served payload on mean, tail and top-1 passes without an anchor" {
+    gate2 0.0190 0.0090 0.962 0.971
+    [ "$status" -eq 0 ]
+    [[ "$output" == *"candidate vs served (no FP8 anchor)"* ]]
+    [[ "$output" == *"KLD GATE PASS"* ]]
+}
+
+@test "a better mean with a worse tail fails: the tail is where a mean hides its damage" {
+    D="$BATS_TEST_TMPDIR/kl3"
+    mkdir -p "$D"
+    kld_report_slices "$D/iq4r.json" iq4r-moe 0.0190 0.0190 0.962 0.962
+    python3 - "$D/cand.json" <<'PY'
+import json, sys
+# mean better than the served payload, p99.9 worse: the shape a mean-only gate waves through
+g = {"kld": {"mean": 0.0090, "p99_9": 0.4000}, "top1_agreement": 0.971, "positions": 25000}
+json.dump({"model": "w5nl64", "all": g, "by_source": {"code": dict(g), "ops": dict(g)}},
+          open(sys.argv[1], "w"))
+PY
+    run bash "$GATE" "$D/iq4r.json" "$D/cand.json"
+    [ "$status" -eq 1 ]
+    [[ "$output" == *"[FAIL] 2."* ]]
+    [[ "$output" == *"KL p99.9"* ]]
+}
+
+@test "a candidate that wins overall but regresses one source slice fails" {
+    D="$BATS_TEST_TMPDIR/kl4"
+    mkdir -p "$D"
+    kld_report_slices "$D/iq4r.json" iq4r-moe 0.0190 0.0190 0.962 0.962
+    # 'all' is better, 'code' is worse: exactly the trade a mean-only gate cannot see
+    kld_report_slices "$D/cand.json" w5nl64 0.0090 0.0300 0.971 0.950
+    run bash "$GATE" "$D/iq4r.json" "$D/cand.json"
+    [ "$status" -eq 1 ]
+    [[ "$output" == *"[PASS] 1."* ]]
+    [[ "$output" == *"[FAIL] 4."* ]]
+    [[ "$output" == *"regressed: code"* ]]
+}
+
+# --- the calibration window: gated, and it says so ---------------------------
+
+window_fixture() {
+    D="$BATS_TEST_TMPDIR/win"
+    mkdir -p "$D/campaign/kld-ref" "$D/campaign/calib" "$D/rad" "$D/corpora" "$D/bin"
+    echo '{}' > "$D/campaign/kld-ref/kld.json"
+    : > "$D/rad/qwen38-bf16.rad"
+    echo "{\"p\":1}" > "$D/corpora/calib.jsonl"
+    # the window's preflight resolves the image digest through podman and asks about the
+    # port/device by host pid; these stubs answer for a quiet host
+    cat > "$D/bin/podman" <<'EOS'
+#!/usr/bin/env bash
+case "${1:-}" in
+    image) echo "sha256:69add58f7de1ed2bca2b1e20a4cc75717557df3f73569a178eca5aafe8c44093" ;;
+esac
+EOS
+    printf '#!/usr/bin/env bash\nexit 0\n' > "$D/bin/ss"
+    printf '#!/usr/bin/env bash\nexit 0\n' > "$D/bin/fuser"
+    chmod +x "$D/bin/podman" "$D/bin/ss" "$D/bin/fuser"
+    echo "$D"
+}
+
+@test "the calibration window refuses before touching a card without a reference recording" {
+    D="$(window_fixture)"
+    mv "$D/campaign/kld-ref/kld.json" "$D/campaign/kld-ref/kld.json.absent"
+    run env PATH="$D/bin:$PATH" \
+        FRANK_QWEN38_CAMPAIGN="$D/campaign" \
+        FRANK_QWEN38_RAD="$D/rad/qwen38-bf16.rad" \
+        Q38_WINDOW_CALIB_CORPUS="$D/corpora/calib.jsonl" \
+        Q38_CALIB_MIN_FREE_MIB=1 bash "$WINDOW"
+    [ "$status" -eq 2 ]
+    [[ "$output" == *"FAIL  reference"* ]]
+    [[ "$output" == *"must not take the cards"* ]]
+}
+
+@test "the calibration window's dry run prints the gated plan, including what it would bank" {
+    D="$(window_fixture)"
+    run env PATH="$D/bin:$PATH" \
+        FRANK_QWEN38_CAMPAIGN="$D/campaign" \
+        FRANK_QWEN38_RAD="$D/rad/qwen38-bf16.rad" \
+        Q38_WINDOW_CALIB_CORPUS="$D/corpora/calib.jsonl" \
+        Q38_CALIB_MIN_FREE_MIB=1 bash "$WINDOW" --dry-run
+    [ "$status" -eq 0 ]
+    [[ "$output" == *"run-qwen38-bf16.sh calib"* ]]
+    [[ "$output" == *"else kill the rig"* ]]
+    [[ "$output" == *"calib-push.py"* ]]
+    [[ "$output" == *"AGENTTRASH"* ]]
+}
+
+@test "a flat calibration round is not a failure: the tap's floor is pooled" {
+    # round 1 of a 10.03M-token corpus against a 12M floor seals nothing, and the
+    # first version of this gate aborted the run there — one round too early.
+    run python3 - "$PUSHER" <<'PY'
+import importlib.util, sys
+spec = importlib.util.spec_from_file_location("cp", sys.argv[1])
+m = importlib.util.module_from_spec(spec)
+spec.loader.exec_module(m)
+print(m.round_verdict(8192, 8192, False),   # flat: continue
+      m.round_verdict(8192, 4096, False),   # shrink: fail
+      m.round_verdict(8192, 20000, False),  # grew: continue
+      m.round_verdict(0, 0, True))          # done: stop
+PY
+    [ "$status" -eq 0 ]
+    [ "$output" = "flat shrink grew done" ]
+}
+
+@test "the calibration boot's step is its own knob; the KL modes keep production's" {
+    # Calib is the only phase whose cost is read volume per document: four 2,048-token steps
+    # re-stream each layer's routed experts four times. Measured 2026-10-29: 71 tok/s, 47 h tap.
+    run env Q38_CALIB_MNBT=8192 bash "$RUNNER" calib --dry-run
+    [ "$status" -eq 0 ]
+    grep -A1 -x -- '--max-num-batched-tokens' <<<"$output" | grep -qx '8192'
+    # and the same knob must NOT reach the measuring shape those numbers are compared in
+    run env Q38_CALIB_MNBT=8192 bash "$RUNNER" kldref --dry-run
+    [ "$status" -eq 0 ]
+    ! grep -qx '8192' <<<"$output"
+    run env Q38_CALIB_MNBT=wide bash "$RUNNER" calib --dry-run
+    [ "$status" -eq 2 ]
+    [[ "$output" == *"Q38_CALIB_MNBT must be a number"* ]]
+}
+
+# ---- the window must not mistake a dying production socket for the rig -------
+# 2026-10-30: the runner stopped frank-archon, the rig container was still loading,
+# and /health answered 200 from the production process draining its listener. The
+# window logged `health ok waited=0s`, the pusher refused on the container name, and
+# a 12M window died at second zero. The gate must prove the LISTENER is in the rig.
+
+tap_window_fixture() {
+    export W="$BATS_TEST_TMPDIR/window"
+    mkdir -p "$W/scripts" "$W/campaign/calib" "$W/campaign/kld-ref" "$W/models/rad" "$W/corpora" "$W/bin"
+    cp "$BATS_TEST_DIRNAME/../scripts/qwen38-calib-window.sh" "$W/scripts/"
+    cp "$BATS_TEST_DIRNAME/../scripts/qwen38-preflight.sh" "$W/scripts/"
+    echo "{}" > "$W/campaign/kld-ref/kld.json"
+    : > "$W/models/rad/qwen38-bf16.rad"
+    echo "{\"p\":1}" > "$W/corpora/calib.jsonl"
+    # a gram so the empty-tap check passes on the positive path
+    : > "$W/campaign/calib/gram.r0.blk.0.ffn_down_exps.0.bin"
+
+    cat > "$W/scripts/run-qwen38-bf16.sh" <<'EOS'
+#!/usr/bin/env bash
+[ -n "${STUB_RIG_MARKER:-}" ] && : > "$STUB_RIG_MARKER"
+sleep "${STUB_RIG_SLEEP:-3}"
+exit 0
+EOS
+    cat > "$W/bin/py" <<'EOS'
+#!/usr/bin/env bash
+[ -n "${STUB_PUSH_SENTINEL:-}" ] && : > "$STUB_PUSH_SENTINEL"
+exit 0
+EOS
+    cat > "$W/bin/podman" <<'EOS'
+#!/usr/bin/env bash
+case "${1:-}" in
+    inspect) echo true ;;
+    # the PREFLIGHT asks who owns :7770 (production, per LISTEN_PID); the WINDOW asks which
+    # pids belong to the rig (RIG_PID). Both stubs are the same binary, so keep them apart.
+    image)   echo "sha256:69add58f7de1ed2bca2b1e20a4cc75717557df3f73569a178eca5aafe8c44093" ;;
+    top)     echo PID
+             # the WINDOW asks "which pids are the rig's", the PREFLIGHT asks "which are
+             # production's". One stub, two answers: that separation IS the identity lesson.
+             case "${2:-}" in
+                 qwen38-bf16) echo "${RIG_PID:-424242}" ;;
+                 *)           echo "${LISTEN_PID:-424242}" ;;
+             esac ;;
+    *)       : ;;
+esac
+EOS
+    printf '#!/usr/bin/env bash\nexit 0\n' > "$W/bin/fuser"
+    cat > "$W/bin/ss" <<'EOS'
+#!/usr/bin/env bash
+echo "LISTEN 0 4096 *:7770 *:* users:((\"radiance\",pid=${LISTEN_PID:-424242},fd=9))"
+EOS
+    cat > "$W/bin/curl" <<'EOS'
+#!/usr/bin/env bash
+exit 0
+EOS
+    chmod +x "$W/scripts/run-qwen38-bf16.sh" "$W/bin/py" "$W/bin/podman" "$W/bin/ss" "$W/bin/curl" "$W/bin/fuser"
+}
+
+tap_window_env() {
+    env PATH="$W/bin:$PATH" \
+        FRANK_QWEN38_CAMPAIGN="$W/campaign" \
+        FRANK_QWEN38_RAD="$W/models/rad/qwen38-bf16.rad" \
+        Q38_WINDOW_CALIB_CORPUS="$W/corpora/calib.jsonl" \
+        Q38_CALIB_MIN_FREE_MIB=1 \
+        Q38_WINDOW_HEALTH_TIMEOUT=1 \
+        PY="$W/bin/py" \
+        STUB_RIG_MARKER="$W/rig.marker" \
+        STUB_RIG_SLEEP=0 \
+        STUB_PUSH_SENTINEL="$W/push.ran" \
+        bash "$W/scripts/qwen38-calib-window.sh"
+}
+
+@test "a health answer from a listener outside the rig is not the rig: no push, window fails" {
+    tap_window_fixture
+    # health 200, container "running", but the listener pid belongs to production
+    export LISTEN_PID=999999 RIG_PID=424242
+    run tap_window_env
+    [ "$status" -ne 0 ]
+    [ -z "$(ls "$W/push.ran" 2>/dev/null)" ]
+    [[ "$output" == *"before it ever answered /health"* ]]
+}
+
+@test "when the listener is inside the rig the window proceeds and banks the tap" {
+    tap_window_fixture
+    export LISTEN_PID=424242 RIG_PID=424242
+    run tap_window_env
+    [ "$status" -eq 0 ]
+    [ -e "$W/push.ran" ]
+    [[ "$output" == *"TAP BANKED"* ]]
+}
+
+# ---- the preflight: every fact by IDENTITY, and it must fail loudly -------------------------
+# Each case here is a lie that cost a real window on 2026-10-30.
+
+preflight_fixture() {
+    export PF="$BATS_TEST_TMPDIR/pf"
+    mkdir -p "$PF/campaign/calib" "$PF/campaign/kld-ref" "$PF/bin"
+    echo '{}' > "$PF/campaign/kld-ref/kld.json"
+    echo '{"p":1}' > "$PF/corpus.jsonl"
+    cat > "$PF/bin/podman" <<'EOS'
+#!/usr/bin/env bash
+case "$1" in
+  image) # an image carries every digest it was pulled under; the pin is one of them
+         printf '%s\n' "docker.io/stilldeadcode/radiance@sha256:4beb81eea874e3e308fbffbe1a9402a2f845c19e68ea612ba949b43bd5cb7cb0"
+         printf '%s\n' "docker.io/stilldeadcode/radiance@${STUB_IMAGE_DIGEST:-sha256:69add58f7de1ed2bca2b1e20a4cc75717557df3f73569a178eca5aafe8c44093}" ;;
+  top)   echo HPID; echo "${STUB_PROD_PIDS:-4242}" ;;
+esac
+EOS
+    cat > "$PF/bin/ss" <<'EOS'
+#!/usr/bin/env bash
+[ -n "${STUB_LISTENER_PID:-}" ] && echo "LISTEN 0 4096 *:7770 *:* users:((\"x\",pid=${STUB_LISTENER_PID},fd=9))"
+exit 0
+EOS
+    cat > "$PF/bin/fuser" <<'EOS'
+#!/usr/bin/env bash
+[ -n "${STUB_KFD_PID:-}" ] && echo "${STUB_KFD_PID}"
+exit 0
+EOS
+    chmod +x "$PF/bin/podman" "$PF/bin/ss" "$PF/bin/fuser"
+}
+
+run_preflight() {
+    ( cd "$BATS_TEST_DIRNAME/.." && env PATH="$PF/bin:$PATH" \
+        FRANK_QWEN38_CAMPAIGN="$PF/campaign" \
+        Q38_WINDOW_CALIB_CORPUS="$PF/corpus.jsonl" \
+        Q38_CALIB_MIN_FREE_MIB=1 \
+        bash scripts/qwen38-preflight.sh "$@" )
+}
+
+@test "preflight passes when every fact is true, and says so" {
+    preflight_fixture
+    run run_preflight
+    [ "$status" -eq 0 ]
+    [[ "$output" == *"PREFLIGHT PASSED"* ]]
+    [[ "$output" == *"carries sha256:69add58f"* ]]
+}
+
+@test "preflight accepts the service's own process on the port, not just any answer" {
+    preflight_fixture
+    export STUB_PROD_PIDS=4242 STUB_LISTENER_PID=4242 STUB_KFD_PID=4242
+    run run_preflight
+    [ "$status" -eq 0 ]
+    [[ "$output" == *"held by frank-archon (host pid 4242)"* ]]
+}
+
+@test "preflight fails when the tag resolves to a different digest" {
+    preflight_fixture
+    export STUB_IMAGE_DIGEST="sha256:deadbeefdeadbeef"
+    run run_preflight
+    [ "$status" -eq 2 ]
+    [[ "$output" == *"does not carry sha256:69add58f"* ]]
+}
+
+@test "preflight fails when :7770 belongs to a host pid the service does not own" {
+    preflight_fixture
+    export STUB_LISTENER_PID=999999
+    run run_preflight
+    [ "$status" -eq 2 ]
+    [[ "$output" == *":7770 is held by host pid 999999"* ]]
+}
+
+@test "preflight fails when another process holds the GPU" {
+    preflight_fixture
+    export STUB_KFD_PID=999998
+    run run_preflight
+    [ "$status" -eq 2 ]
+    [[ "$output" == *"foreign holders:"* ]]
+}
+
+@test "preflight fails when the tap dir already holds grams from another run" {
+    preflight_fixture
+    : > "$PF/campaign/calib/gram.r0.blk.0.ffn_down_exps.0.bin"
+    run run_preflight
+    [ "$status" -eq 2 ]
+    [[ "$output" == *"rotate them to .AGENTTRASH"* ]]
+    run run_preflight --allow-existing-tap
+    [ "$status" -eq 0 ]
+}
+
+@test "preflight fails when the launcher is not in the repo root" {
+    preflight_fixture
+    run env PATH="$PF/bin:$PATH" FRANK_QWEN38_CAMPAIGN="$PF/campaign" \
+        Q38_WINDOW_CALIB_CORPUS="$PF/corpus.jsonl" Q38_CALIB_MIN_FREE_MIB=1 \
+        bash -c "cd /tmp && bash '$BATS_TEST_DIRNAME/../scripts/qwen38-preflight.sh'"
+    [ "$status" -eq 2 ]
+    [[ "$output" == *"not the repo root"* ]]
+}

tests/qwen38_reference.bats

@@ -157,12 +157,15 @@ flag_value() {
     [[ "$output" != *"--vram-kv-mib"* ]]
 }
 
-@test "the calib dry run opens the tap at 12M tokens and serves on 7770" {
+@test "the calib dry run opens the tap at the 12M floor, on the latest pin, and serves on 7770" {
     run bash "$RUNNER" calib --dry-run
     [ "$status" -eq 0 ]
     flat="$(oneline)"
     [[ "$flat" == *"-e RADIANCE_CALIB_TOKENS=12000000"* ]]
     [[ "$flat" == *"-e RADIANCE_CALIB_DIR=/data/calib"* ]]
+    # the tap producer is the campaign's own pin (latest radiance), not production's
+    [[ "$flat" == *"radiance@sha256:69add58f"* ]]
+    [[ "$flat" != *"radiance@sha256:d590e300"* ]]
     [[ "$flat" == *"--port 7770"* ]]
     [[ "$flat" == *":/data/calib "* ]]
     # calib serves; it does not record