docs(vault): the census was lying; corrected, and the honest result is flat docs

`scripts/port-census.py` built its corpus from the new directory, so each page
under test was held by itself: every era page reported 0 orphan lines. With the
corpus excluding the file under test, each candidate holds 14-63 sole-record
lines, so no era page is padding and none was trashed.

Deletions of restatement only:
- Multi-Node Runtimes.md (-257): restated launch material owned by
  Qwen3.8 PP Deployment.md; index rows and two Related bullets repointed.
- log.md: 66 headings folded into 35 dated entries (one entry per date, newest
  first), titled headings kept as `date — Title` per the spec's wording, and 78
  lines of transient deploy chatter dropped where every fact token in the line
  is held by an owner page (preserved in .AGENTTRASH/20261019-log-transient/).
- Authentic: the 5-page split reverted (-93); the parent carries
  `okf-exempt: length` and the three stub pages stay deleted.

check-okf.py accepts `## YYYY-MM-DD — Title`; port-census.py excludes the file
under test and gained a docstring note about the trap. One-off compaction tool
moved to ~/.AGENTTRASH. Full suite: 451 tests pass, 0 OKF violations,
155 pages, 0 unresolved links.

scripts/check-okf.py

@@ -0,0 +1,293 @@
+#!/usr/bin/env python3
+"""OKF v0.1 conformance linter for a docs bundle (default serverAppConfigs-docs).
+
+Checks the hard rules of the Open Knowledge Format spec plus the maintenance
+guidance, so "conforms to OKF" is a test and not a claim:
+
+  FRONTMATTER  every non-reserved .md has parseable YAML frontmatter with a
+               non-empty `type` from the spec's value set
+  TYPE         the value is one of reference / howto / tutorial / explanation
+  DESC         `description` is present, non-blank, not the title, <= 120 chars,
+               and does not restate a fleet quantity
+  TIMESTAMP    `timestamp` present, ISO 8601, not in the future
+  LENGTH       lines between 1 and 500 inclusive
+  INDEX-FM     no index.md may carry frontmatter (index.md is reserved)
+  LOG-DATE     log.md date headings are `## YYYY-MM-DD` or `## YYYY-MM-DD — Title`,
+               one entry per date, newest first
+  INDEX-COVER  every index.md links each of its sibling pages; the root index
+               links every page in the bundle
+  INDEX-NUM    an index table row restates a fleet quantity — a digit run of
+               three or more, a digits-plus-unit figure, a port, or a
+               multiplier. Names (Qwen3.6, RDNA4, RTX 3090, PICO-8, P4) are
+               not quantities and are allowed
+  CITATIONS    informational: pages with factual claims and no Citations
+               section are counted, never failed
+
+  LENGTH       lines between 1 and 500 inclusive. A page may opt out of the
+               ceiling with `okf-exempt: length` in its frontmatter: AGENTS.md
+               makes one page per endpoint permanent, and an endpoint page with
+               a full Eras table cannot split without creating a second owner
+               of the same shape (tests/docs_completeness.bats enforces that)
+
+Exit 0 when there are no violations.
+"""
+import datetime
+import re
+import sys
+from pathlib import Path
+
+TYPES = {"reference", "howto", "tutorial", "explanation"}
+RESERVED = {"index.md", "log.md", "AGENTS.md"}
+
+# A quantity, as opposed to a name: a comma-grouped figure, a run of 3+ digits,
+# a unit-bearing figure, a port (":7770"), or a multiplier ("2x", "3 *").
+QUANTITY = re.compile(
+    r"\d{1,3}(?:,\d{3})+"          # 262,144
+    r"|\d{3,}"                     # 200000, 7770, 3090
+    r"|\d+(?:\.\d+)?\s*(?:GiB|MiB|GB|MB|TB|tok/s|t/s|x\b|%)"
+    r"|[:×]\s*\d"
+    r"|\d+\s*[x×]\s*\d"
+)
+# Product / model / standard names that legitimately contain digits.
+NAMES = re.compile(
+    r"Qwen[\d.]*[-\w]*|Gemma\s?\d[\w.]*|MiMo[-\w.\d.]*|Ornith[-\w.\d]*"
+    r"|Bonsai[-\w.\d]*|Ling[-\w.\d.]*|DeepSeek[-\w.\d]*|GLM[-\w.\d]*"
+    r"|PICO-\d+|RDNA\s?\d+|R\d{4}|RTX\s?\d+|XDNA\d|gfx\d+|CUDA\s?[\d.]+"
+    r"|\d+\s?GbE|Wi-?Fi\s?\d|v?\d+\.\d+(?:\.\d+)?|b\d+|P\d\b|A\d+\b"
+    r"|frank-\w*2026-\d\d-\d\d|2026-\d\d-\d\d",
+    re.I,
+)
+DATE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}")
+FM_KEY = re.compile(r"^([A-Za-z][\w-]*):\s*(.*)$")
+
+
+def split_frontmatter(text):
+    if not text.startswith("---"):
+        return None, text
+    end = text.find("\n---", 3)
+    if end < 0:
+        return None, text
+    body_start = text.find("\n", end + 1)
+    return text[3:end], text[body_start + 1 :] if body_start >= 0 else ""
+
+
+def fields(raw):
+    out = {}
+    if not raw:
+        return out
+    key = None
+    for line in raw.splitlines():
+        if not line.strip():
+            continue
+        m = FM_KEY.match(line)
+        if m:
+            key, val = m.group(1), m.group(2).strip().strip("\"'")
+            out[key] = val
+        elif line.startswith(" ") and key:
+            out[key] = (out[key] + " " + line.strip()).strip()
+    return out
+
+
+def strip_names(value):
+    return NAMES.sub(" ", value)
+
+
+def check_file(path: Path, root: Path, length_exempt=frozenset()):
+    v = []
+    rel = str(path.relative_to(root))
+    text = path.read_text(errors="replace")
+    lines = text.splitlines()
+    name = path.name
+    raw, body = split_frontmatter(text)
+    fm = fields(raw)
+
+    if name not in RESERVED:
+        if raw is None:
+            v.append(("FRONTMATTER", rel, "no parseable frontmatter"))
+        else:
+            t = fm.get("type", "")
+            if not t:
+                v.append(("TYPE", rel, "missing type"))
+            elif t not in TYPES:
+                v.append(("TYPE", rel, f"type '{t}' not in {sorted(TYPES)}"))
+            d = fm.get("description", "")
+            if not d:
+                v.append(("DESC", rel, "missing description"))
+            else:
+                if len(d) > 120:
+                    v.append(("DESC", rel, f"description is {len(d)} chars (max 120)"))
+                if d.strip("# ").lower() == fm.get("title", "\0").strip("# ").lower():
+                    v.append(("DESC", rel, "description equals title"))
+                if QUANTITY.search(strip_names(d)):
+                    v.append(("DESC", rel, "description restates a quantity"))
+            ts = fm.get("timestamp", "")
+            if not ts:
+                v.append(("TIMESTAMP", rel, "missing timestamp"))
+            else:
+                try:
+                    stamp = datetime.date.fromisoformat(ts[:10])
+                    if stamp > datetime.date.today():
+                        v.append(("TIMESTAMP", rel, f"timestamp {ts} is in the future"))
+                except ValueError:
+                    v.append(("TIMESTAMP", rel, f"unparseable timestamp {ts!r}"))
+
+    if name == "index.md" and raw is not None:
+        v.append(("INDEX-FM", rel, "index.md must not carry frontmatter"))
+
+    if name == "log.md":
+        dates = []
+        for line in lines:
+            m = re.match(r"^##\s+(\S.*?)\s*$", line)
+            if not m:
+                continue
+            head = m.group(1)
+            if not DATE_RE.match(head):
+                continue
+            dates.append(head[:10])
+            # OKF requires the heading to USE the date format; a title after an
+            # em-dash is allowed and costs zero lines, so `## 2026-08-17 — Why the
+            # pair was rebuilt` is conformant and `## 2026-08-17Rebuilt` is not.
+            if head != head[:10] and not re.match(r"^\d{4}-\d{2}-\d{2}\s+—\s+\S", head):
+                v.append(("LOG-DATE", rel,
+                          f"date heading must be the date alone or `date — Title`: '## {head}'"))
+        if len(dates) != len(set(dates)):
+            v.append(("LOG-DATE", rel, "duplicate date heading"))
+        if dates != sorted(dates, reverse=True):
+            v.append(("LOG-DATE", rel, "date headings not newest first"))
+
+    if raw is not None and name not in RESERVED:
+        n = len(lines)
+        body_lines = len([ln for ln in body.splitlines() if ln.strip()])
+        opted_out = "length" in fm.get("okf-exempt", "").lower()
+        if n > 500:
+            if rel in length_exempt:
+                v.append(("LENGTH-OK", rel,
+                          f"{n} lines — over the hint, pinned by the "
+                          "one-page-per-endpoint law"))
+            elif opted_out:
+                v.append(("LENGTH-OK", rel,
+                          f"{n} lines — over the hint, opted out by the page"))
+            else:
+                v.append(("LENGTH", rel, f"{n} lines (max 500)"))
+        elif body_lines < 15:
+            v.append(("LENGTH", rel, "stub: fewer than 15 body lines"))
+
+    # A second `type:` key inside the body means the page carries two copies of
+    # itself with a frontmatter block in the middle — the shape a naive
+    # frontmatter rewrite produces. It doubles the length, splits the wikilinks
+    # and makes the page's facts ambiguous, so it is a hard failure.
+    if raw is not None and name not in RESERVED:
+        dupes = [i + len((raw or "").splitlines()) + 2
+                 for i, ln in enumerate(body.splitlines())
+                 if re.match(r"^(type|title|description):", ln)]
+        if dupes:
+            v.append(("DOUBLED", rel,
+                      f"frontmatter keys inside the body at line(s) "
+                      f"{', '.join(str(d) for d in dupes[:5])} — the page "
+                      "contains a duplicated copy of itself"))
+
+    return v
+
+
+def index_rows(text):
+    rows = []
+    for line in text.splitlines():
+        if line.startswith("|") and "[[" in line:
+            rows.append(line)
+    return rows
+
+
+def check_links(root: Path):
+    v = []
+    pages = [
+        p for p in sorted(root.rglob("*.md"))
+        if ".AGENTTRASH" not in str(p) and ".obsidian" not in str(p)
+    ]
+    for index in [p for p in pages if p.name == "index.md"]:
+        text = index.read_text(errors="replace")
+        links = set(re.findall(r"\[\[([^\]|#]+)", text))
+        if index.parent == root:
+            targets = [p for p in pages if p.name not in RESERVED]
+        else:
+            targets = [p for p in index.parent.glob("*.md") if p.name != "index.md"]
+        for target in targets:
+            if target.stem not in links and target.name not in links:
+                v.append(("INDEX-COVER", str(index.relative_to(root)),
+                          f"does not link sibling {target.name}"))
+    for index in [p for p in pages if p.name == "index.md"]:
+        for row in index_rows(index.read_text(errors="replace")):
+            if QUANTITY.search(strip_names(row)):
+                v.append(("INDEX-NUM", str(index.relative_to(root)),
+                          "row restates a quantity: " + row[:80]))
+    return v
+
+
+def live_endpoints(root: Path):
+    """Pages pinned by AGENTS.md's one-page-per-endpoint law.
+
+    Derived, never hardcoded — the same derivation as
+    tests/docs_completeness.bats: an endpoint is live when scripts/ holds both
+    its env and its unit. Those pages carry the Eras table for their whole
+    history, so the OKF 500-line split hint cannot apply to them without
+    creating a second owner of the same shape.
+    """
+    scripts = root.parent / "scripts"
+    out = set()
+    if scripts.is_dir():
+        for envf in scripts.glob("frank-*.env"):
+            if (scripts / f"{envf.stem}.service").exists():
+                out.add(f"LLM/{envf.stem}.md")
+    return out
+
+
+def retired_era_pages(root: Path):
+    """Era pages of a retired endpoint: campaign ledgers, one owner each.
+
+    AGENTS.md lets a campaign ledger that exists nowhere else live on the page
+    that measured it, and a retired endpoint keeps exactly one page per era.
+    Splitting such a ledger to satisfy the 500-line hint would fragment one
+    campaign across two owners, so the tag opts the page out.
+    """
+    out = set()
+    for path in root.rglob("*.md"):
+        s = str(path)
+        if ".AGENTTRASH" in s or ".obsidian" in s:
+            continue
+        raw, _ = split_frontmatter(path.read_text(errors="replace"))
+        if raw is not None and "retired-endpoint" in fields(raw).get("tags", ""):
+            out.add(str(path.relative_to(root)))
+    return out
+
+
+def main(argv):
+    root = Path(argv[1] if len(argv) > 1 else "serverAppConfigs-docs")
+    exempt = frozenset(live_endpoints(root)) | retired_era_pages(root)
+    violations = []
+    notes = []
+    citation_gaps = 0
+    pages = 0
+    for path in sorted(root.rglob("*.md")):
+        s = str(path)
+        if ".AGENTTRASH" in s or ".obsidian" in s:
+            continue
+        pages += 1
+        for code, rel, detail in check_file(path, root, exempt):
+            (notes if code == "LENGTH-OK" else violations).append((code, rel, detail))
+        text = path.read_text(errors="replace")
+        if len(text.splitlines()) > 40 and not re.search(
+                r"^##\s+(Citations|Sources)\b", text, re.M):
+            citation_gaps += 1
+    violations += check_links(root)
+
+    for code, rel, detail in violations:
+        print(f"  {code:12s} {rel}: {detail}")
+    for code, rel, detail in notes:
+        print(f"  {code:12s} {rel}: {detail}")
+    print(f"{pages} concept pages, {len(violations)} OKF violations"
+          f" ({citation_gaps} pages without a Citations section — soft;"
+          f" {len(notes)} length exemptions honoured)")
+    return 1 if violations else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv))

scripts/check-vault-links.py

@@ -31,7 +31,12 @@ def strip_md(path: str) -> str:
 
 
 def build_index(vault: pathlib.Path):
-    pages = sorted(vault.rglob("*.md"))
+    # `.AGENTTRASH` is deleted-but-held state, not vault content: it must not be
+    # scanned (its relative links resolve against the trash dir, not the vault)
+    # and must not answer as a link target — a live page pointing into the trash
+    # is an unresolved link, not a resolved one.
+    pages = sorted(p for p in vault.rglob("*.md")
+                   if ".AGENTTRASH" not in str(p))
     stems: dict[str, list[pathlib.Path]] = {}
     for page in pages:
         stems.setdefault(page.stem.casefold(), []).append(page)

scripts/frank-archon.env

@@ -1,306 +1,99 @@
-# frank-archon.env — radiance (StillDeadcode, Apache-2.0) serving the
-# PUBLISHER's 4-bit-codebook container of the official Qwen/Qwen3.8-Flash-Next
-# (MODEL_FILE below, fetched by scripts/download-frank-archon-iq4r.sh) on
-# BOTH R9700s (TP=2), dankserver :7770.
-# 2026-10-15 PAYLOAD SWAP: the model moved from the in-house w4dn8 conversion
-# of the Swift1.5 finetune to the engine author's published container
-# qwen3.8-next-flash-fp8-iq4r-moe.rad (116,322 MiB, sha256-verified at fetch —
-# the w4dn8 measured 116,416 MiB, so this shape carries over with 94 MiB to
-# spare). NO argument changed: every number below is the 2026-10-13 shape,
-# kept verbatim. The swap commit is the rollback (the w4dn8 payload stays on
-# disk).
-
+# frank-archon.env — radiance (StillDeadcode) on BOTH R9700s (TP=2),
+# Dankserver :7770, publisher's 4-bit-codebook container of the official
+# Qwen3.8-Flash-Next.
 #
-# SHAPE (2026-10-13): THREE decode slots x 200,000 tokens, fp8 (quantized) KV,
-# with the per-card budget spent on resident experts: 25,600 MiB of slab +
-# 5,120 MiB of pool = 30,720 MiB, inside the 30,969 MiB ceiling the engine's own
-# budget table allows (card 32,624 MiB, settle floor 209 MiB). The pool is sized
-# to the contract and nothing more (3 x 200,000 needs 5,063 MiB at fp8; 5,120
-# predicts 608,027 tokens, 1.3 % slack), because every MiB taken from the slab
-# is prefill and TTFT (2026-10-11 A/B: slab -14.8 % cost prefill -29 %, TTFT
-# +25 % at the full window) while KV beyond the contract buys nothing.
-# Consequence, deliberate: a FOURTH full-height conversation does not fit this
-# pair, and the engine refuses it — it never truncates. See the KV block for the
-# arithmetic and the measured dtype law. The w4dn8 payload is the swap
-# rollback and keeps its trio under the w4dn8 runbook; the fp8-native payload
-# that came before it was DELETED from disk 2026-10-15 (Frank's call, to free
-# the volume for this download — rebuildable via the fetch + build scripts
-# named in the fp8-native runbook).
-#
-# 2026-10-05 cutover from FreeToken (PR #587 ROCm, one card, experts in
-# pinned host RAM; git tag frank-archon-freetoken-pr587-20261005 is the
-# pre-cutover state — its unit/env/serve trio and Dockerfile are all in the
-# tag and rebuildable). Before that: tcclaviger vLLM fork (tag
-# frank-archon-vllm-shapeG-20261003).
-#
-# WHY RADIANCE: the engine is C++/HIP with libr4d kernels built for
-# gfx1201, drives `podman run` directly from the digest-pinned
-# stilldeadcode/radiance image (287 MiB; host needs only the amdgpu driver),
-# and its qwen4exp plugin reads the modelopt FP8 encoding NATIVELY
-# (`X.weight_scale_inv` is the scale plane — checkpoint.h). The PLE n-gram
-# table (51B params, 47.7 GiB) is served from pinned host RAM via io_uring
-# reads — Frank's call: "Native FP8. Experts from ram. Pin the PLE to ram."
-# THE 4-bit-codebook publisher shape is the DEPLOYED payload since
-# 2026-10-15. The card's own run line (--host-pool-mib 12288,
-# --expert-vs-cache-ratio 0.82, seqs 8, spec 3) is the publisher's
-# suggestion on 32 GB cards; this file's shape is the fleet's measured one.
-#
-# PAYLOAD: /home/dank/models/rad/qwen3.8-next-flash-fp8-iq4r-moe.rad — the
-# official Qwen3.8-Flash-Next bf16 checkpoint rad-converted by the engine's
-# own author (recipe qwen4exp-w4nl64-i8-hc8m: experts as 4-bit codes into a
-# 16-level non-uniform codebook after a 128-point Walsh–Hadamard rotation,
-# GPTQ + 3 coordinate-descent sweeps on a 10M-token calibration, ten
-# bf16-protected experts per layer, int8 W8A8 trunk + lm_head, fp8 HC
-# mixtures, PLE n-gram fp8 fixed-scale, MTP head, bf16 vision tower; card KL
-# 0.0789 all-tokens / top-1 92.85 % vs bf16). Fetched byte-exact + sha256 by
-# scripts/download-frank-archon-iq4r.sh; the recipe/LICENSE stay in the
-# staging dir beside it as provenance. ROLLBACK: revert the swap commit — the
-# in-house qwen4exp-swift15-w4dn8-i8-hc8m.rad (scripts/convert-swift15-w4dn8.sh,
-# KLD-gated) stays on disk. Behind it, the fp8-native
-# qwen4exp-fp8-native.rad was DELETED from disk 2026-10-15 (Frank's call,
-# one-time rm, to free the volume for this payload); rebuild =
-# scripts/download-frank-archon.sh + scripts/build-frank-archon-rad.sh — a
-# NO-RECIPE rad-convert copy of the official 131-shard checkpoint under the
-# descriptor law (no shipped plugin claims (qwen4exp, "fp8"), so the staging
-# config.json drops quantization_config; per-weight encodings are data, not
-# the descriptor).
-#
-# SERVING FACTS (docs/GUIDE.md, docs/DOCKER.md, compose/flashnext.yaml):
-# - The HTTP listener binds AFTER the full load (main.cpp: engine.start()
-#   then engine.serve()) — during load the port refuses connections, so a
-#   bare curl /health gate is honest. /health == /ping: 200 JSON while
-#   serving, 503 only while stopping (core/server/admin.cpp:385).
-# - Reasoning: --reasoning-format defaults to auto, emitting thinking in
-#   `reasoning_content` (config.cpp). Tool calls: the qwen3.8 dialect, from
-#   the chat format. Both need no parser flags.
-# - Sampling: the OpenAI endpoint accepts temperature, top_p, top_k, min_p,
-#   typical_p, presence_penalty, frequency_penalty, repetition_penalty,
-#   repeat_last_n, chat_template_kwargs, reasoning_effort,
-#   enable_thinking, preserve_thinking (core/server/oai.cpp:194-205) —
-#   models.yml's extraBody set is fully inside it. Unknown params are
-#   refused (400), which is why nothing unverified rides the wire.
-# - MTP: the .rad carries the model's mtp.* head (the publisher's recipe
-#   names the mtp.fc_hidden/fc_embedding/draft_head rules; card default
-#   depth 3). Served at the depth named by FRANK_ARCHON_SPEC_TOKENS below —
-#   the depth this shape's decode ladder was measured at.
-# - The fused TP all-reduce is declared over max-num-seqs x
-#   (1 + --num-speculative-tokens) rows; 3 x (1+2) = 9 rows, under the 25-row
-#   exactness bound of the wht6 wire.
-# Engine pin: Codeberg tag v1.0.13 (commit 53282fa), Docker Hub tag 1.0.13,
-# index digest below. REJECTED UPGRADE: 1.2.0 (sha256:665775ed1d90de6dbb
-# 25558e07fc60d20328f1346f96f3c6e00845eb763f01fa, tag 1.2.0, commit
-# e95d397) was benched 2026-10-06 on this exact shape (fp8-native rad,
-# vram pair, spec 2, seqs 3, 43.6k greedy probe): decode 29.21 t/s and
-# draft acceptance 0.7123 vs the 1.0.13 control's 30.08 and 0.7549.
-# Acceptance on the greedy harness is deterministic (the control
-# reproduced the recorded 0.755 exactly), so the drop is a real numerics
-# change — the 1.2.0 MoE weight-rotation change (92b8aaf) is the named
-# suspect; decode loss tracks the acceptance ratio (0.943 vs 0.945).
-# The vram-pair translation itself is exact (1.2.0 boot ledger took
-# vram_weights 25.46 GiB / vram_kv 3.46 GiB per card byte-identical to
-# the 1.0.13 ledger at EVC 0.82). Re-test 1.2.0+ when the rotation change
-# is untangled; 1.2.0 also REMOVED --expert-vs-cache-ratio in favour of
-# the --vram-weights-mib / --vram-kv-mib pair (core/config.cpp) — this
-# file already uses the pair, which BOTH pins accept (rollback symmetry).
+# ONE OWNER: shape, payload identity, eras, laws and every measured number live
+# in serverAppConfigs-docs/LLM/frank-archon.md. This file carries the knobs plus
+# only the reason a reader editing THIS line cannot recover from the page.
+# Where the two disagree, this file wins on values; the boot ledger arbitrates
+# the engine shape.
 FRANK_ARCHON_IMAGE=stilldeadcode/radiance@sha256:4c3f484564bb2872784e46b0a59cbd71becc9c1e746e301a3596ffae253c322e
-# The .rad container FILE (mounted under /models) and the state dir for the
-# prefix-cache disk tier (mounted rw at /data). The model dir holds ONLY
-# .rad files; the raw HF download and its staging dir live beside it and
-# are never mounted.
+# 1.0.13 is pinned by digest on purpose: 1.2.0 is a DEAD END on this payload
+# (acceptance 0.7123 vs 0.7549 — the page's Dead ends section names the suspect).
+# The .rad file mounted under /models; /data holds the prefix-cache disk tier.
+# The model dir holds ONLY .rad files — the HF download and staging stay beside it.
 FRANK_ARCHON_MODEL_DIR=/home/dank/models/rad
 FRANK_ARCHON_MODEL_FILE=qwen3.8-next-flash-fp8-iq4r-moe.rad
 FRANK_ARCHON_STATE_DIR=/home/dank/radiance-state
-# The state dir's tune/ subdirectory holds rad-tune's kernel-selection
-# cache (the engine's selector reads $RADIANCE_HOME/tune/<machine>.tune
-# at boot; $RADIANCE_HOME defaults to /opt/radiance, in the image, so the
-# cache must ride a host mount — serve script mounts it, see the runbook's
-# "Tuning levers").
-# Container identity is the SERVICE name (Frank's call, 2026-10-05): one
-# name across systemctl and podman ps. The runtime changes; the name does
-# not. The legacy list retires every predecessor identity that could hold
-# the GPUs or the port.
+# state dir's tune/ subdir carries rad-tune's kernel-selection cache; the engine
+# reads $RADIANCE_HOME/tune/<machine>.tune and that defaults to /opt/radiance
+# (inside the image), so the cache only survives on a host mount.
+# Container identity is the SERVICE name: one name across systemctl and
+# `podman ps`; the runtime changes, the name does not. The legacy list retires
+# every predecessor identity that could hold the GPUs or the port.
 FRANK_ARCHON_CONTAINER=frank-archon
 FRANK_ARCHON_LEGACY_CONTAINERS="frank-archon-qwen frank-archon-r9v qwen-fp8 glm-probe"
-# Served model id — Protoss name (wire rename 2026-10-09). The container,
-# unit, scripts and this file keep the frank-archon name above; only the
-# alias on the wire moved. Radiance validates the requested model name, so
-# every client (harness, bench scripts, probes) MUST send this value.
+# Served id. Radiance VALIDATES the requested model name, so every client must
+# send exactly this value (unlike llama.cpp and halogen, which ignore it).
 FRANK_ARCHON_SERVED_NAME=frank-archon
 FRANK_ARCHON_PORT=7770
-# Frank's chat-template call: froggeric's fixed Qwen3.8 template (the
-# repo's root chat_template.jinja, vendored here; the checkpoint's own
-# template is baked into the .rad by rad-convert and this overrides it).
-# Radiance derives the reasoning/tool-call reply format from it.
+# Frank's chat-template call: froggeric's fixed Qwen3.8 template, vendored in
+# the repo; it overrides the template baked into the .rad, and radiance derives
+# the reasoning and tool-call reply format from it.
 FRANK_ARCHON_TEMPLATE=/home/dank/serverAppConfigs/scripts/qwen38-froggeric-v22.5.jinja
 
 # --- engine shape (the boot ledger arbitrates every number below) ---------
-# Tensor parallel: BOTH R9700s. The board swap (2026-09-20, ProArt B850
-# x8/x8 on root ports) made collectives legal on this host; radiance's TP
-# wire is libr4d's own, not RCCL.
+# TP=2 needs the 2026-09-20 board (x8/x8 on root ports): a collective cannot be
+# dispatched on a card whose PCIe chain lacks atomics. Radiance's wire is
+# libr4d's own, not RCCL.
 FRANK_ARCHON_TP=2
-# Lossy WHT 6-bit wire for large all-reduces (publisher's production choice:
-# +5.6-6.3% prefill, ~-1.3% 8-stream decode at TP2). min-kb stays the
-# engine default.
+# Lossy WHT 6-bit wire for large all-reduces (publisher's production choice). It
+# stays EXACT for single-sequence decode and any step of <= 25 rows — see
+# MAX_SEQS x (1 + SPEC_TOKENS) below.
 FRANK_ARCHON_TP_WIRE=wht6
-# Decode slots: THREE, so the served window is a CAPACITY CONTRACT — three
-# concurrent conversations of --max-model-len fit the KV pool (see the KV
-# block below). The "seqs 2 cannot boot" law is NOT radiance's: it was the
-# tcclaviger vLLM fork's torch.compile shape specialization on
-# query_start_loc (ConstraintViolationError, measured 2026-10-02, retired
-# with that runtime). Radiance booted seqs 2 at the fp8-native first boot,
-# and the 2026-10-06 decode sweep measured seqs 2 and 3 statistically tied
-# (30.9-33.1 t/s, inside the +-3% noise floor) — slot count buys concurrency,
-# not speed, so a slot costs KV, never decode.
-# Cost of each extra slot: 110 MiB of per-slot recurrent state inside the KV
-# pool, and the fused TP all-reduce is declared over seqs x (1 + spec) rows —
-# 3 x 3 = 9 here, under the 25-row bound at which the wht6 wire stays EXACT
-# (single-sequence decode and any step of <= 25 rows is bit-exact; only busier
-# steps are lossy).
-# The pool is sized to this contract and nothing more (see the KV block):
-# 3 x 200,000 at fp8 = 5,063 MiB, pool 5,120, and the freed 1,792 MiB went into
-# the resident expert slab. The trade is explicit and it is Frank's call:
-# a FOURTH full-height conversation no longer fits (5,120 MiB of pool backs
-# ~608k tokens; 4 x 200,000 needs 6,601), so a fourth 200k request is refused
-# as too large to fit — it is NOT squeezed, paged, or truncated. Third-party
-# traffic that wants four full-height sessions is a --vram-kv-mib change, not
-# a code change.
+# Decode slots. A slot costs KV, never decode; the pool is sized to this
+# contract and nothing more, so a FOURTH full-height conversation is REFUSED as
+# too large to fit — never squeezed, paged or truncated. That is Frank's
+# explicit trade: the freed MiB went to the expert slab.
 FRANK_ARCHON_MAX_SEQS=3
-# Served window. Native is 262,144; 200,000 is the publisher's production
-# value (compose) and what models.yml's contextWindow must quote — a longer
-# prompt is refused by the engine, never truncated. The engine treats
-# `kv group tokens >= max-model-len` as a HARD limit ("ONE request longer
-# than N tokens is accepted by the API and then refused as too large to ever
-# fit — this is a hard limit, not a backing ratio"), so the pool is sized
-# for seqs x maxlen, not for one sequence.
+# Served window. The engine treats `kv group tokens >= max-model-len` as a HARD
+# limit, and models.yml contextWindow MUST quote this number (over-length is a
+# refusal, not a truncation). The pool is sized for seqs x maxlen, not one
+# sequence.
 FRANK_ARCHON_MAX_MODEL_LEN=200000
 FRANK_ARCHON_PLACEMENT=expert_tiered
-# Pinned host pool for the expert rows that do not fit VRAM. vram_weights and
-# this pool serve the per-rank resident set TOGETHER: at the fp8-KV pair
-# (26071/3543) the ledger showed the VRAM slab filled to 259 KiB of slack,
-# i.e. the pair was exactly sized. Cutting vram-weights by 3.8 GiB to pay for
-# the bf16 KV pool SPILLS ~3.8 GiB/rank here, so the pool moves with it —
-# under-sizing it is a boot failure (36864 refused the boot in the fp8 era).
-# The boot ledger's coverage line remains the referee, not this arithmetic.
+# Pinned host pool for expert rows that do not fit VRAM. It moves WITH
+# VRAM_WEIGHTS: the pair and this pool serve the per-rank resident set together,
+# so cutting the slab spills here — under-sizing it is a boot failure (36,864
+# refused a boot). The ledger's coverage line is the referee, not this file.
 FRANK_ARCHON_HOST_POOL_MIB=44928
-# VRAM left unclaimed per card (publisher default 96 MiB).
+# VRAM left unclaimed per card (publisher default 96 MiB); the engine's settle
+# floor is built from it.
 FRANK_ARCHON_GPU_HEADROOM_MIB=96
-# VRAM split of claimed memory per card, MiB (0 = derive from the card).
-# THE CEILING, read off this engine's own VRAM-budget table (boot 2026-10-13,
-# pair 24000/5120): card 32,624 MiB; budgets claimed 29,120; `3.42 GiB is
-# neither budgeted nor accounted — the driver's context, the activation arena
-# and --gpu-headroom-mib come out of it; anything past those is throughput on
-# the floor`; and `card 0 settled at 2.01 GiB free against 209.19 MiB (the
-# 96.00 MiB --gpu-headroom-mib and 113.19 MiB kept for the static region)`.
-# So the reserve above the ENGINE's floor is 2,058 - 209 = 1,849 MiB and the
-# ceiling is 29,120 + 1,849 = 30,969 MiB. The 2026-10-11 bound of 29,120
-# ("31,856 - 1,710 outside both budgets - keep 1 GiB free") was MY reserve, not
-# the engine's; the pair below spends the difference and lands 458 MiB above
-# the engine's floor. `weights + kv <= 30,969` is the gate.
-# THE PRICE, and it is paid twice: on this engine idle KV VRAM is ALSO the
-# expert cache's spillover (`mover: 2275 slots can be lent out of the KV
-# cache (2.69 GiB)`, `arena: lends 740 MiB while steps are <= 256 tokens`),
-# so a bigger KV pool steals weight residency from the slab AND from the
-# lendable cache. Priors on this card/model: stealing 2.14 GiB of weights for
-# a bigger pool (MAX_MODEL_LEN 262144) cost 2.5% decode; the EVC ladder's
-# other direction (0.82 -> 0.84, +2.5 GiB weights) cost 8.9%. The 2026-10-11
-# exact-KV A/B measured the slab's own price (22208 vs the fp8-era 26071):
-# prefill -29 %, TTFT at the full window +25 %, decode inside the noise band on
-# 2 of 3 passes, draft acceptance UP 0.661 -> 0.740.
-# 2026-10-13 (late): THREE slots, so the pool is cut to the contract (5,120)
-# and every spare MiB goes to the slab: 22208 -> 24000 (the third slot's
-# surplus KV), then 24000 -> 25600 (the card's settling slack). The direction
-# the A/B data points: prefill is slab-limited, KV beyond the contract is not.
-# The margin is thin by design (608,027 predicted tokens against the 600,000
-# contract), so the boot ledger's `backs N sequences all at 200000 tokens at
-# once` line is the referee for THIS pair and the concurrent probe is its
-# regression test.
-# VERDICT on the 25,600 arm (measured 2026-10-13, scripts/probe-frank-archon-kv.py):
-# the threshold claim is FALSIFIED — the slab is a SLOPE at this end. A-cold
-# 43.5k single-stream: 22,208 -> 3,397, 24,000 -> 3,290, 25,600 -> 3,584 t/s;
-# the 3-way 197k arm's aggregate prefill 3,051 -> 3,589 t/s (wall
-# 193.7 -> 164.5 s). So +1,600 MiB of slab = +8.9 % single-stream and
-# +17.6 % concurrent, both outside the +-3 % noise floor. The 22,208/24,000
-# pair landed inside noise of EACH OTHER, which is what made it read as a
-# plateau; two points are not a curve.
-# The cliff that remains is between 25,600 (3,584) and 26,071 (4,379, the
-# 2026-10-11 fp8 control at seqs 3 / pool 3,543) — +22 % for +471 MiB. At THREE
-# slots that is only reachable by breaking the ceiling (26,112 + 5,120 = 31,232
-# vs 30,969, ~214 MiB above the engine's floor); at TWO slots it is free:
-# pool 3,072 + slab 27,897 = exactly the ceiling, slab above the cliff.
+# ONE budget per card: VRAM_WEIGHTS + VRAM_KV <= 30,969 MiB, the ceiling the
+# engine's own budget table prints (derivation and the slab/pool price: the
+# page's VRAM laws). Idle KV VRAM is ALSO the expert cache's spillover, so a
+# bigger pool pays twice — which is why the pool sits at the contract and the
+# slab takes the rest.
 FRANK_ARCHON_VRAM_WEIGHTS_MIB=25600
 FRANK_ARCHON_VRAM_KV_MIB=5120
-# KV cache QUANTIZED (fp8). This engine's flag takes ONLY bf16|fp8 — there is
-# ONE kv_dtype for the whole cache, so "exact K, quantized V" (llama.cpp's
-# --cache-type-k/--cache-type-v pair, which is what frank-zealot runs) is
-# UNSERVABLE here: the rung below bf16 is fp8 on both halves.
-# Measured per-rank cost, read out of the boot ledger's group table (fp8 boot
-# 2026-10-08, bf16 boot 2026-10-11):
-#   kv_attn     13 layers, 4-token blocks, page 26.00 KiB (fp8) -> 52.00 KiB
-#   kv_qsa_bkey 13 layers, 4-token blocks, page  3.25 KiB at BOTH dtypes —
-#               the QSA block key is an index, not attention state, and does
-#               NOT follow --kv-cache-dtype
-#   => 7.3125 KiB/token fp8, 13.8125 KiB/token bf16: bf16 costs 1.89x fp8,
-#               not 2x, precisely because the bkey plane is dtype-fixed.
-# The dtype-INDEPENDENT terms also live in the pool: kv_gdn_state 54 MiB x 2
-# copies per sequence (+ conv/tail/ple ~= 110 MiB a sequence) and the prefix
-# checkpoint region 8 x 55.99 = 448 MiB. So:
-#   pool_mib >= tokens x 7.3125 KiB + 110 x seqs + 448
-#   3 x 200,000 at fp8 needs 4,285 + 330 + 448 = 5,063 MiB; the pool is 5,120,
-#   and it predicts (5120 - 330 - 448) x 1024 / 7.3125 = 608,027 tokens: the
-#   three-slot contract with 1.3 % slack, every MiB above 5,063 having been
-#   handed to the expert slab instead. The ledger line is the referee ("backs N
-#   sequences all at 200000 tokens at once") — a pair this tight is read off the
-#   boot, never assumed.
-#   At bf16 the same three slots cost 8,093 + 778 = 8,871 MiB, which would cut
-#   the slab to 20,249 MiB (-9 %); at FOUR slots bf16 costs 10,791 + 888 =
-#   11,679 MiB and the slab falls to 17,441 (-33 %). fp8 keeps 22,208 at both
-#   slot counts. The 2026-10-11 bf16 shape (6,912 / seqs 2) backed 462,752
-#   tokens = 2 x 200,000; the historical fp8 pool of 3,543 gave 386,904 (1.93
-#   sessions), and the four-slot fp8 boot of 2026-10-13 19:21 backed 843,276.
-# Secondary saving: the prefix-cache entry shrinks with the attention plane
-# (one entry 110.50 KiB at bf16 -> 58.50 KiB at fp8, measured on the two boot
-# ledgers' `kv tier transfer` lines), so the host tier below parks twice as
-# many conversations as it did at bf16 — no re-size needed to park three slots.
+# fp8 is the ONLY rung below bf16 on this engine: one kv_dtype for the whole
+# cache, so llama.cpp's exact-K/quantized-V split (frank-zealot) is unservable
+# here. bf16 costs 1.89x fp8, not 2x — the QSA block-key plane is dtype-fixed.
 FRANK_ARCHON_KV_DTYPE=fp8
-# Spec decode: the checkpoint's own MTP head, served at depth 2 — the
-# publisher's card ships depth 3 as its peak; this fleet's ladder was
-# measured at 2 and the swap keeps every argument, so 2 stays (rows
-# 3 x (1+2) = 9, inside the wht6 25-row exactness bound).
+# The checkpoint's own MTP head at depth 2: this fleet's ladder was measured at
+# 2, and 3 x (1+2) = 9 rows stays inside the wht6 25-row exactness bound.
 FRANK_ARCHON_SPEC_TOKENS=2
 FRANK_ARCHON_MAX_BATCHED_TOKENS=2048
-# Prefix cache tiers (publisher pattern). The host tier is sized in ENTRIES,
-# and one entry tracks the cache dtype (58.50 KiB at fp8, 117 KiB at bf16, per
-# the ledger's `kv tier transfer` line): at fp8, 8192 MiB parks ~5.8 sessions
-# of 200k (at bf16 the same tier parked ~2.9), so four live slots plus a full
-# parked set fits with the tier as it stands — at RAM the host has (104 GiB
-# available with the service loaded).
+# Prefix cache tiers. The host tier is sized in ENTRIES and one entry tracks the
+# cache dtype (58.50 KiB at fp8), so a dtype change re-sizes the parked set
+# without touching these numbers.
 FRANK_ARCHON_PREFIX_HOST_MIB=8192
 FRANK_ARCHON_PREFIX_DISK_MIB=65536
-# Frank's pin law: the PLE n-gram table (47.7 GiB) serves from pinned host
-# RAM the cards read. `ram` REFUSES the start if the memlock limit is too
-# small — the unit's LimitMEMLOCK=infinity is the other half of this law.
+# Frank's pin law: the PLE n-gram table serves from pinned host RAM the cards
+# read. `ram` REFUSES the start if memlock is too small — the unit's
+# LimitMEMLOCK=infinity is the other half of this law.
 FRANK_ARCHON_NGRAM_PLACEMENT=ram
-# TTM preflight: the pinned ledger (2 x 44 GiB pools + 47.7 GiB PLE + 8 GiB
-# prefix host = ~147 GiB) rides TTM's shared pages, capped by ttm's
-# pages_limit (the current kernel exposes NO pages_max; the cap function
-# takes the min over the knobs that exist). Below the plan the load dies
-# mid-pin with an allocator error instead of this named refusal.
-# THE CAP IS A RUNTIME KNOB — the kernel default is half of RAM, so every
-# raise dies with a reboot. Measured the hard way: the 2026-10-05 manual raise
-# to 52,428,800 pages (200 GiB) was gone after the 2026-10-12 23:17 POWER
-# FAILURE, the cap fell back to 125,988 MiB, and the preflight refused the
-# start 6,919 times in ~20 h while Restart=always re-ran it. The gate in
-# scripts/ttm-preflight.sh now RAISES the knob itself before refusing (the
-# root unit is privileged; the knob is a 0644 root module parameter), so the
-# recovery is a boot, not a visit to the machine.
+# TTM preflight. The pinned ledger (~147 GiB) rides TTM's shared pages, and the
+# kernel RESETS pages_limit to half of RAM on every boot; below the plan the
+# load dies mid-pin with an anonymous allocator error. ttm-preflight.sh raises
+# the knob itself before refusing by name, so recovery is a boot, not a visit.
 FRANK_ARCHON_TTM_MIN_MIB=150000
 FRANK_ARCHON_TTM_TARGET_MIB=204800
-# Escape hatch for boot-ladder experiments. Unset -> no extra argv. Keep
-# multi-flag values as one string; the serve script expands it unquoted on
-# purpose.
+# Escape hatch for experiments. Unset -> no extra argv; the serve script expands
+# this unquoted on purpose, so keep multi-flag values as one string.
 FRANK_ARCHON_EXTRA_ARGS=
-# The .rad's minimum size in MiB (deployer gate; a truncated fetch is a
-# guaranteed long-timeout boot instead of a fast failure). The deployed
-# publisher container measures 116,322 MiB (121,969,901,568 B); the
-# in-house w4dn8 rollback measured 116,416 MiB. Floor sits just under both.
+# Deployer gate: a truncated fetch must be a fast refusal, not a long-timeout
+# boot. Floor sits under the live payload and under every rollback payload
+# (see the page's Eras table for what is on disk and what needs a re-download).
 FRANK_ARCHON_RAD_MIN_MIB=110000

scripts/frank-dragoon.env

@@ -1,209 +1,82 @@
 # frank-dragoon.env — tunables for the frank-dragoon endpoint on Dankclaw's
 # Strix Halo iGPU (gfx1151), host port 7770.
 #
-# RUNTIME (2026-10-17, Frank): halogen-flash-server 0.16.2 — the
-# closed-source C++/HIP engine (image ghcr.io/peonist-ai/halogen-flash-server,
-# digest-pinned below) serving Quat3rnion/halogen-qwen3.8-flash-next-v2-
-# abliterated — the NATIVE v2 conversion (2026-10-11) of orcarouter's pinned
-# BF16 Qwen3.8-Flash-Next, abliterated — as a podman container under a root
-# unit, driven by scripts/serve-frank-dragoon.sh. This CUTS OVER the
-# 2026-10-06 Swift1.5-finetune shape; rollback = revert the swap commit and
-# redeploy: the swift15 payload stays on disk at
-# /home/dank/models/halogen-flashnext-v2/, the 27B-era payload beside it.
+# ONE OWNER: payload identity, shape, eras, laws and every measured number live
+# in serverAppConfigs-docs/LLM/frank-dragoon.md. This file carries the knobs
+# plus only the reason a reader editing THIS line cannot recover from the page.
+# Where the two disagree, this file wins on values; the boot ledger arbitrates
+# the memory figures.
 #
-# MEASURED A/B, both boots on this box, identical battery traffic (engine
-# tools with the service stopped, behavior probes against it, 2026-10-17):
-# NIAH 18/18 vs 17/18, refusal census 0/5 on both, tool calls and the
-# red-PNG vision probe identical, needle hit at both depths of a 21.5k-token
-# haystack, decode 50.0 vs 46.8 t/s, cold prefill 1,604 vs 1,425 t/s,
-# wikitext-2 PPL 3.968 vs 3.877 (the finetune's edge is gone: +2.3 %) and
-# thinking tokens 1,476 vs 784 mean — the finetune compressed reasoning, the
-# base thinks ~1.9x longer per answer.
-#
-# The A/B ran at the card's ctx/pool of 262,144. Afterwards Frank lowered ctx to
-# 200,704 (RAM + the harness's compaction budget) and, on 2026-10-18, raised the
-# pool to 2 x ctx = 401,408 so BOTH slots can carry a full-length request at
-# once; the rates above are the ceiling-262,144 record and the production ledger
-# is the ctx-200,704 one.
-#
-# WHY THE RUNTIME MOVED WITH THE MODEL: the payload is .hgn — per its own
-# card it "will not load in transformers, vLLM, or llama.cpp" — and the v2
-# .hgn loads ONLY in the flash-server line (the old halogen engine is for
-# the 27B p1w4d file; the v2 quickstart pins flash-server). What the move
-# buys (card + live probe evidence): MTP spec decode (53-57 t/s greedy,
-# 3-10x serial), ~1,640 t/s prefill at 34k, ~72% fewer thinking tokens than
-# the censored 27B (905 mean), the ABLITERATED base (refusals should NOT
-# return), IMAGE INPUT (the v2 vision tower loads beside the checkpoint —
-# red-PNG probe verified on the card; the 27B engine ignored its encoder),
-# tool calls verified, and the native 262,144 context — served at 200,704 by
-# choice from 2026-10-17 (see SHAPE: the ceiling is a RAM decision, not a
-# capability limit).
-#
-# SHAPE — the knobs that are NOT free, with the arithmetic:
-#   HALOGEN_CTX=200704 is the PER-REQUEST ceiling and the per-conversation
-#   hard limit — a prompt over it is a 400 naming the limit, never a silent
-#   truncation. LOWERED from 262,144 on 2026-10-17 at Frank's direction (the
-#   checkpoint's native window is 262,144 and the engine serves anything
-#   below it): every position costs KV pool RAM and shrinks the prompt-cache
-#   entry (111 MiB at 262,144, ~85 MiB here). The 400-zone law is INVARIANT
-#   to the value — it says the harness window must sit ~45k UNDER the engine
-#   ceiling, because the harness clamps max_tokens to contextWindow − est −
-#   4096, its chars/4 est UNDER-counts the rendered prompt by ~3 % on
-#   code/agent traffic, and compaction only fires at 85 % (measured 2026-10-07
-#   at the old 200,192 ceiling: est 152,938, server 157,388, over by 354).
-#   Harness contract (models.yml): contextWindow 155648 (= ceiling − 45056,
-#   the gap that absorbs the estimate error), maxTokens 45056.
-#   Compaction fires at 85 % = 132,301 est ≈ 136k measured — the 180k target
-#   of the 262,144 era is unreachable at this ceiling: the window itself is
-#   155,648. Worst-case request: prompt + budget <= 155648 − 4096 + 3 %
-#   est-error ≈ 156.2k <= 200,704 — ~9.5× the 3 % error of margin (robust to a
-#   ~25 % under-count). Compaction rides on frank-zealot: input ≈ threshold −
-#   keepRecent(20k) ≈ 112k, inside its 200,704 slot.
-#   HALOGEN_KV_POOL_POSITIONS=401408 = SLOTS x CTX (2 x 200,704), the top of the
-#   legal band (the seam enforces ctx <= pool <= slots * ctx). Still ONE pool
-#   behind HALOGEN_KV_SLOTS=2 (ledger form: "2 slots over ONE <pool>-position KV
-#   pool, each request up to HALOGEN_CTX"), but sized so BOTH slots can hold a
-#   MAXIMUM-length request at the same time — the parity with frank-zealot's
-#   401,408-position pool that Frank ordered on 2026-10-18. The PER-REQUEST
-#   ceiling does NOT move with it: HALOGEN_CTX stays 200,704, because the
-#   checkpoint's native window is 262,144 and the engine refuses to go past it
-#   without HALOGEN_ROPE_YARN (its own 400 text says so). ~28 KiB/position
-#   (FLAGS.md) ⇒ the DEPLOYED 2026-10-18 boot RESERVES 11.0 GiB (`reserving the
-#   KV pool, 401408 positions (about 11.0 GiB)`), where pool == ctx (200,704)
-#   cost 5.5, 262,144 cost 7.2 and the 2026-10-06 shape's 524,288 measured 14.6.
-#   The preflight formula bills 11.3 (block scratch + ~115 MiB/slot of O(1)
-#   state) — quote the RESERVATION. FLAGS.md law: "RAM the pool
-#   takes is RAM that table loses" — the 47.68 GiB n-gram table surrenders this
-#   much page cache, so the warm prompt-cache ratio is what to re-measure.
-#   HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
-#   drafter is NOT slot-locked on this engine (batching and speculation
-#   coexist — DFlash-era slots>1 law does NOT carry over; the card's own
-#   quickstart ships 2 slots WITH the head), so the second slot is
-#   concurrent capacity, not a speculation tax.
-#   Memory ledger against the box's 121.2 GiB unified (the n-gram table is
-#   mmap'd page cache beside the checkpoint, not a pinned charge — the boot
-#   ledger agrees: "mapped 51.2 GB, NOT pinned (Pin::None)"):
-#     weights 62.9 GiB locked (0.8 of them the vision tower) + KV 11.0 +
-#     working 5.7 (HALOGEN_MAX_TOK=8192) = 79.6 GiB engine-side (the boot
-#     ledger of the 401408-position deploy, 2026-10-18), with the 1.3 GiB
-#     prompt cache (12 entries x 111 MiB at ctx 200704) booked separately by
-#     the ledger; HALOGEN_HOST_RESERVE_GIB=20 keeps the host honest. Against
-#     the 121.2 GiB box that leaves ~20 GiB over the reserve (MemAvailable
-#     read 22 GiB right after the raise) for baseline co-tenancy
-#     (frank-zealot, ComfyUI, Caddy/observ ~10-15). The boot ledger is the
-#     referee — read it after deploy and re-size if the engine prints a
-#     clamp.
-#   HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
-#   turns the MTP head off for whole requests after low acceptance and
-#   HALVES decode as a result. Frank accepted the head always-on trade
-#   (temp 1.0 traffic costs acceptance; the card still ships the recipe).
-#
-# CLIENT CONTRACT deltas vs the 27B shape (harness side lives in
-# ~/.omp/agent/models.yml):
-#   * SERVER-SIDE SAMPLER EXISTS NOW: HALOGEN_TEMPERATURE/TOP_P/TOP_K set
-#     the card's thinking recipe (1.0 / 0.95 / 20) as engine defaults, so
-#     an absent temperature is no longer greedy. models.yml extraBody
-#     carries the same values — consistent, not load-bearing.
-#   * HALOGEN_MAX_TOK=8192 is only the DEFAULT for requests that omit
-#     max_tokens; the harness sends at most its maxTokens 45056. Requests
-#     over HALOGEN_MAX_TOKENS_CAP (65536) are a 400 naming the cap. Cap and
-#     queue timeout are COUPLED (FLAGS.md table): 65536 at ~10 t/s serial
-#     worst case needs > 6550 s; 7200 honors it. Do not move one alone.
-#   * max_tokens INCLUDES thinking; a reasoning reply over a small budget
-#     returns empty content with finish_reason "length".
-#   * Prompt cache is ON by default (PROMPT_CACHE=2): a follow-up turn at
-#     100k context re-prefills ~2 s instead of ~88 s cold. No pin needed —
-#     unlike the 27B engine, auto-sizing fits this box's ledger.
-#   * Thinking lands in reasoning_content (same as the 27B engine); the
-#     v2 chat template rejects reasoning_effort outside xhigh|medium|low,
-#     so the models.yml fold (high->xhigh) still applies.
-#   * Vision: OpenAI image_url content parts only (the fleet's standard
-#     contract). >=256 px probe images or the model answers "video".
-#   * Concurrency at depth is tight now: the KV pool is ONE 200,704-position
-#     budget behind 2 slots — exactly one maximum-length request fills it. Two
-#     conversations coexist only while they stay under ~100k each. The
-#     PER-REQUEST ceiling moved WITH the pool (262144 -> 200704), so the harness
-#     contract moves too: contextWindow 155648 = ceiling − 45k, never higher.
-#   * response_format json_object/json_schema unsupported in this line
-#     (no constrained decoding) — unchanged from the 27B engine.
-#   * The engine's token protocol has no auth and binds loopback INSIDE
-#     the container; only the frontend port (8731) is published. Same as
-#     before. Kernel needs CONFIG_HSA_AMD_SVM (CachyOS ships it) and
-#     amdgpu.noretry=0 (check at deploy).
-
-# Image: digest-pinned (fetch law: tag pulls silently re-point; the digest
-# is what was qualified). 0.16.2 = the version the model card names as its
-# quickstart pin; RepoDigest captured at pull on 2026-10-06.
+# The .hgn format is the lock: it loads ONLY in the flash-server line, so the
+# engine moves with the payload, never ahead of it.
+# Image digest-pinned (a tag pull silently re-points; the digest is what was
+# qualified — 0.16.2 is the version the model card names as its quickstart pin).
 FRANK_DRAGOON_IMAGE=ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84ac22308a53f5d1ca6b86806702d7039e5ebc51cae4c66621b92fe04a
-# Container identity: kept from the 27B era (unit ExecStop names it).
+# Container identity kept from the 27B era: the unit's ExecStop names it.
 FRANK_DRAGOON_CONTAINER=frank-dragoon-halogen
-# Old quadlet-era container identity; the serve script clears stale names.
+# Predecessor identity the serve script clears if it is still holding the GPUs.
 FRANK_DRAGOON_LEGACY_CONTAINERS=frank-dragoon
-# host:container — the frontend listens on 8731 inside (HALOGEN_API_PORT
-# default); the engine port (8730) is never published.
+# host:container — the frontend listens on 8731 inside; the engine's own port
+# (8730) has no auth and is NEVER published.
 FRANK_DRAGOON_PUBLISH=7770:8731
 
 FRANK_DRAGOON_MODEL_DIR=/home/dank/models/halogen-v2-abliterated
-# The checkpoint: Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated
-# qwen38-flash-next-v2-abliterated.hgn — the NATIVE 2026-10-11 v2 conversion
-# of orcarouter's pinned BF16 Qwen3.8-Flash-Next, abliterated. NOT the writer
-# transplant this filename used to carry: 66,687,678,432 B where the swift15
-# r2 measured 66,687,678,464 B — same filename family, different artifact.
-# Hub LFS sha256 verified at fetch by scripts/download-frank-dragoon-v2abl.sh.
+# Same filename family, DIFFERENT artifact: this is the native v2 conversion,
+# 32 bytes smaller than the writer transplant that used to carry the name.
+# Never infer the payload from the filename — this size and sha256 are the gate.
 FRANK_DRAGOON_CHECKPOINT=qwen38-flash-next-v2-abliterated.hgn
 FRANK_DRAGOON_CHECKPOINT_BYTES=66687678432
 FRANK_DRAGOON_CHECKPOINT_SHA256=b5bd925fa2b44343be4a2cd328cae6b57ce7ffa2fe2cc56281717377e5a1ce8a
-# The n-gram lookup table: REQUIRED (the engine reads it beside the
-# checkpoint; the card's quickstart names it explicitly). From
-# peonist-ai/halogen-qwen3.8-flash-next @ 5cc17cea.
-# The card does NOT establish this stock table as matched to the NATIVE
-# conversion and no source-matched table is published, so the engine's own
-# tools are the referee: NIAH 18/18 and PPL 3.968 on it (2026-10-17). It is
-# HARDLINKED from the swift15 payload dir (a hardlink is a regular file; the
-# serve gate refuses symlinks), i.e. byte-identical to the file that era
-# served.
+# The n-gram table is REQUIRED (the engine reads it beside the checkpoint) and
+# disk-backed: mmap'd page cache, never pinned. It is a stock peonist companion
+# hardlinked from the previous payload dir — a hardlink is a regular file, and
+# the serve gate refuses symlinks — with the engine's own NIAH/PPL verdicts on
+# the page as the referee.
 FRANK_DRAGOON_NGRAM_TABLE=qwen38-flash-next-ngram.hgn
 FRANK_DRAGOON_NGRAM_BYTES=51200246144
 FRANK_DRAGOON_NGRAM_SHA256=3450acada94e19aabad88bd49b45eb70304820949c2240fc178067b20ae5dffe
-# The vision tower: OPTIONAL on the engine, ENABLED here (image input).
-# From peonist-ai/halogen-qwen3.8-flash-next @ e053f488.
+# Vision tower: optional on the engine, ENABLED here (image input).
 FRANK_DRAGOON_VISION_TOWER=qwen38-flash-next-vision.hgn
 FRANK_DRAGOON_VISION_BYTES=897916416
 FRANK_DRAGOON_VISION_SHA256=d62e0ae553fe88afd3833733d4a4c669f34d20fd8dfce4b9610525bed2134b10
-# The tokenizer is a FLAT DIRECTORY, $FRANK_DRAGOON_MODEL_DIR/tokenizer,
-# mounted whole at /tokenizer. The two repos ship the same one; fetched
-# fresh from peonist @ e053f488 (tokenizer.json is LFS, byte-identical to
-# the 27B-era copy; the rest are plain git blobs — size-checked only). The
-# serve gate and the deployer refuse a missing file or ANY symlink: an HF
-# cache snapshot symlinks each file into blobs/, and those links dangle
+# The tokenizer is a FLAT DIRECTORY ($FRANK_DRAGOON_MODEL_DIR/tokenizer) mounted
+# whole. The serve gate and the deployer refuse a missing file or ANY symlink:
+# an HF cache snapshot symlinks each file into blobs/ and those links dangle
 # inside the container mount.
 FRANK_DRAGOON_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config.json merges.txt vocab.json generation_config.json"
 
 # --- engine flags (docs/FLAGS.md of the flash-server repo; anything unset
 # stays at the image default, which is what was qualified) ------------------
-# Served model id — Protoss name (wire rename 2026-10-09); the service, unit,
-# scripts and this file keep the frank-dragoon name. OpenWebUI api_configs
-# model_ids for the :7770 backend MUST name this value.
+# Served id. OpenWebUI api_configs model_ids for the :7770 backend MUST name it.
 HALOGEN_MODEL_ID=frank-dragoon
+# PER-REQUEST ceiling: a longer prompt is a 400 naming the limit, never a
+# truncation. The harness window must sit ~45k UNDER this (the 400-zone law on
+# the page) — a window equal to the ceiling makes every long request a 400.
 HALOGEN_CTX=200704
+# ONE pool behind both slots, sized so BOTH can carry a maximum-length request
+# at once. Pool positions buy concurrency; the per-request window never moves
+# with them. Legal band: ctx <= pool <= slots * ctx (the seam enforces it), and
+# RAM the pool takes is RAM the n-gram table's page cache loses.
 HALOGEN_KV_POOL_POSITIONS=401408
+# Two concurrent conversations: the MTP drafter is not slot-locked on this
+# engine, so the second slot is capacity, not a speculation tax.
 HALOGEN_KV_SLOTS=2
 HALOGEN_HOST_RESERVE_GIB=20
-# Spec decode: MTP head is INSIDE the checkpoint (do NOT set HALOGEN_MTP_HEAD);
-# depth 2 is the documented "best all-round" default. HALOGEN_DRAFTER /
-# HALOGEN_DFLASH_* from the 27B engine DO NOT EXIST here.
+# The MTP head is INSIDE the checkpoint — do NOT set HALOGEN_MTP_HEAD.
+# 0 is mandatory: the adaptive default disarms the head for whole requests after
+# low acceptance and halves decode. Depth 2 is the card's best-all-round default.
 HALOGEN_SPEC_ADAPT=0
-# Coupled pair (FLAGS.md table): the cap bounds how long ONE request holds
-# the GPU, the timeout bounds how long the NEXT client waits. Do not move
-# one alone (see the client-contract note above).
+# Coupled trio: MAX_TOK is only the default for requests that omit max_tokens;
+# the cap bounds how long ONE request holds the GPU and the queue timeout bounds
+# how long the NEXT client waits. Move all three together or none.
 HALOGEN_MAX_TOK=8192
 HALOGEN_MAX_TOKENS_CAP=65536
 HALOGEN_QUEUE_TIMEOUT=7200
-# Server-side sampler = the card's thinking recipe (the engine now HAS one;
-# requests may still override per-call). Matches models.yml extraBody.
+# The engine has a server-side sampler (an absent temperature is NOT greedy
+# here); models.yml extraBody carries the same recipe.
 HALOGEN_TEMPERATURE=1.0
 HALOGEN_TOP_P=0.95
 HALOGEN_TOP_K=20
-# Card's belt-and-braces: the engine pins weights by default
-# (HALOGEN_FLASH_PIN_TRUNK=1; only the retired w4b build honored 0).
+# Weights stay pinned: the engine pins by default and only the retired w4b build
+# honored HALOGEN_FLASH_PIN_TRUNK=0, which this payload rejects.
 HALOGEN_WEIGHTS_LOCK=1

scripts/frank-zealot.env

@@ -1,141 +1,72 @@
-# frank-zealot — llama.cpp (PrismML fork build 10777, CUDA) serving
-# prism-ml/Ternary-Bonsai-2-9B-gguf at PQ2_0 **with the MTP head in the file**
-# on the RTX 3090, :7771.
+# frank-zealot — llama.cpp (PrismML fork build 10777, CUDA) on the RTX 3090,
+# :7771. Payload: Ternary-Bonsai-2-9B PQ2_0 with the MTP head in the file.
 #
-# 2026-10-18 (Frank): swap from the Bonsai-2-**27B** PQ2_0 to the 9B MTP export
-# at THREE 262,144-token conversations (Frank: "if you can possibly fit a third
-# context, do it"). The 9B bills HALF the KV of the 27B per token, which is what
-# funds 3 x the native window on one 24,123 MiB card. Speculative decoding ships
-# OFF — measured, not assumed; see the MTP law below, which is the whole story
-# and the reason the publisher's "depth 2 works best" line does not transfer.
-#
-# THE RUNTIME IS A LOCK, NOT A PREFERENCE. The file is general.file_type 141
-# carrying custom GGML type 142 tensors plus prism.hadamard.* metadata
-# (normalized-Sylvester-Walsh transform); stock llama.cpp refuses type 142 at
-# load and a loader that ignored the Hadamard metadata would dequant WRONG. On
-# top of that lock, MTP beside a quantized V cache needs fork build >= b10776,
-# so the vendor prebuilt moved from 10754 (commit 2459f68b5, the 27B era) to
-# 10777 (commit e3de71522). The 10754 tree stays on disk as the rollback
-# runtime for the 27B payload.
+# ONE OWNER: shape, payload identity, eras, laws and every measured number
+# live in serverAppConfigs-docs/LLM/frank-zealot.md. This file carries the
+# knobs and only the reason a reader editing THIS line cannot recover
+# elsewhere. Where the two disagree, this file wins on values.
 LLAMA_BIN=/home/dank/GitPackages/llama-prism-b10777/llama-server
-# 2,675,410,720 B, Hub LFS sha256 c79fcdfaf6b7… (verified at fetch by
-# scripts/download-frank-zealot-bonsai9.sh). This IS the MTP build: the draft
-# head ships INSIDE the checkpoint, so MTP availability is a property of the
-# FILE, never of a flag. The plain PQ2_0 sibling could not run it at any flag
-# value; loading this file instead of that one is the only change that matters.
+# The fork is a LOCK, not a preference: custom tensor type 142 + prism.hadamard.*
+# metadata. Stock llama.cpp refuses the file; a loader ignoring the Hadamard
+# metadata dequantizes wrong, silently.
 MODEL_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-PQ2_0-MTP-Q8_0.gguf
-# 624,229,664 B, Hub LFS sha256 e0cb6f69… — Q8_0 IS MANDATORY, not a preference:
-# the publisher's card says the BF16 mmproj "can crash llama-server when several
-# image requests run at once; Q8_0 is not affected", and this endpoint runs
-# --parallel 3, i.e. exactly that shape. The projector's worst case lands AFTER
-# the engine's own fit check, so it is never inside a projection number.
+# Q8_0 IS MANDATORY: the publisher states the BF16 tower can crash llama-server
+# under concurrent image requests, which is exactly --parallel 3.
 MMPROJ_FILE=/home/dank/models/Bonsai2-9B-MTP/Ternary-Bonsai-2-9B-mmproj-Q8_0.gguf
-# Base alignment: Bonsai-2 is the censored BASE lineage. The 27B CRACK re-export
-# (/home/dank/models/Bonsai2-CRACK/) is byte-identical geometry to the 27B it
-# matched, NOT to this 9B file — if a refusal ever surfaces there is no
-# one-line swap here; the 9B dealigned export would have to be fetched first.
-#
-# Served model id — the harness/family name. The SERVICE, unit, scripts and
-# this env file keep the frank-zealot name; only the alias on the wire is the
-# model id. Consumers that key on the served id: OpenWebUI (model row,
-# access_grant, api_configs model_ids) and the harness models.yml.
+# Served id — the harness/family name, not a fact about the payload.
+# Id-keyed consumers: OpenWebUI (model row, access_grant, api_configs), models.yml.
 ALIAS=frank-zealot
 BIND_ADDR=0.0.0.0
 LISTEN_PORT=7771
-# CTX_SIZE is the POOL llama.cpp divides across slots: 786,432 / 3 = 262,144 per
-# conversation — each slot at the model's TRAINED window, which is the point of
-# the swap (the 27B era had to cut each conversation to 200,704 to fit two).
-# MUST stay divisible by PARALLEL (the seam enforces it) and the harness
-# contextWindow quotes the PER-SLOT number — a longer prompt is answered with a
-# 400, never truncated. The ceiling cannot go past 3 x 262,144: that is the
-# native wall (no YaRN in this path) and the arithmetic agrees — a 4th slot
-# would push each below the compaction contract.
-CTX_SIZE=786432
+# CTX_SIZE is the POOL llama.cpp divides across slots: 602,112 / 3 = 200,704 a
+# conversation. MUST stay divisible by PARALLEL (the seam enforces it) and
+# models.yml contextWindow quotes the PER-SLOT number — a longer prompt is a
+# 400, never a truncation. The pool is bought, not granted: K+V at 25,088
+# B/token is 14,406.00 MiB here and the MTP draft context bills its own f16 KV
+# over the WHOLE pool (4,096 B/position ⇒ 2,352.00 MiB), so the old 786,432
+# pool cannot carry a drafter at any KV dtype — 18,816 + 3,072 of cache alone
+# against the 24,123 MiB the driver leaves addressable.
+CTX_SIZE=602112
 PARALLEL=3
-BATCH_SIZE=4096
-# Ubatch is a prefill knob (decode never moves) with NO interior optimum on this
-# CUDA card up to 2048 — 2048 is the measured optimum (+4.6 % prefill). It has a
-# second job here: it sets the compute-buffer size, and the compute buffers are
-# what decide whether MTP fits (see the MTP law). 1024 is the documented MTP arm.
-UBATCH_SIZE=2048
-# KV law (header + boot ledger, both confirmed): qwen35 hybrid, 32 layers with
-# qwen35.full_attention_interval = 4 ⇒ 8 full-attention + 24 Gated DeltaNet;
-# 4 KV heads × 256 dim = 8,192 elements per token per cache. q8_0 bills 8,704 B
-# per cache ⇒ 17,408 B/token = 17 KiB, so the 786,432 pool is 13,056.00 MiB —
-# the ledger said exactly that. HALF the 27B's 34,816 B/token, which is the
-# whole reason three native windows fit. f16 would bill 26,112 MiB at this pool
-# (against a 24,123 MiB card) and dies at cudaMalloc; the 24 DeltaNet layers
-# bill a fixed recurrent state per SLOT instead of per token — the PRODUCTION
-# ledger prints "RS buffer size = 150.75 MiB (3 cells, 32 layers, 3 seqs)"
-# (~50 MiB a cell), while the SPEC-ARMED arms print 452.25 MiB, because MTP
-# depth 2 bills three seqs per slot (9 cells) — which is why pool arithmetic
-# alone under-prices the shape.
-CACHE_TYPE_K=q8_0
+# The logical batch is VRAM- and throughput-neutral at a fixed ubatch (the law
+# on [[frank-zealot]]); 1024×6 keeps it a clean multiple of the ubatch.
+BATCH_SIZE=6144
+# Ubatch is a prefill knob (decode never moves with it) and it sets the
+# compute-buffer size, which decides whether a drafter fits. 1024 is the rung
+# every measured MTP arm here ran on (213.5 t/s against 109.2 at 2048). Its
+# VRAM credit is SHAPE-specific: the 786,432-pool arm census showed ~1,306 MiB
+# back, this boot measured 2,089.88 MiB of target compute against 2,496.13 at
+# ubatch 2048 — 406 MiB back, with the split moving (CUDA0 +257, CUDA_Host
+# −663). The drafter is funded by CACHE_TYPE_V below, not by this knob.
+UBATCH_SIZE=1024
+# K exact, V in the 8-bit class: 16,384 + 8,704 = 25,088 B/token (8,192
+# elements per token per cache × 2 B for f16, × 1.0625 for q8_0 — there is no
+# true fp8 KV dtype in this build). That 7,680 B/token saved against exact V is
+# what funds BOTH the third conversation and the drafter. Quantized V is
+# illegal without --flash-attn, which the serve script pins. Rate law and the
+# arithmetic: [[frank-zealot]] → Laws.
+CACHE_TYPE_K=f16
 CACHE_TYPE_V=q8_0
-# SPECULATIVE DECODING: OFF, MEASURED. The head is in the file and the drafter
-# is GOOD (acceptance 0.798/0.617 per position, mean accepted length 2.41) — it
-# still loses, because the draft context bills its OWN f16 KV over the WHOLE
-# pool: 4,096 B/position × 786,432 = 3,072.00 MiB, plus its compute buffers,
-# 4,830 MiB all told. Booted arms, same card, same traffic (2026-10-18):
-#   3 × 262,144  spec OFF  ub 2048 → 17,876 used / 6,249 free, decode 167.2 t/s
-#   3 × 262,144  spec ON   ub 2048 → 22,706 used / 1,419 free, decode 109.2 t/s
-#   3 × 262,144  spec ON   ub 1024 → 21,400 used / 2,725 free, decode 213.5 t/s
-#   2 × 262,144  spec ON   ub 2048 → 17,178 used / 6,947 free, decode 213.2 t/s
-#   1 × 262,144  spec ON   ub 2048 → 11,652 used / 12,473 free, decode 214.2 t/s
-# i.e. MTP is worth +28 % decode whenever the card has room, and at the 3-slot
-# pool it costs 1.3 GiB of compute buffers that ubatch 2048 refuses to give up —
-# the squeeze, not the drafter, is what halves decode. The arm that WOULD run
-# MTP at three windows is SPEC_TYPE=draft-mtp + UBATCH_SIZE=1024 (decode 213.5,
-# 3-stream 250.8 aggregate) and pays prefill −17 % and 2.7 GiB of headroom on a
-# card whose projector worst-case lands after the fit check. This endpoint's
-# load is the fleet's COMPACTION HUB (prefill-bound, ~170k in) plus 2-3
-# concurrent subagents, so the deployed arm takes the +17 % prefill
-# (2,877 vs 2,395 t/s at 106,418 tokens) and the +12 % three-stream aggregate
-# (280.2 vs 250.8) over single-stream speed, and keeps 6.2 GiB free.
-# SPEC_TYPE=none strips the WHOLE spec set (--spec-type, every --spec-draft-*):
-# flags move as a set or the arm measures a crash instead of an absence.
-SPEC_TYPE=none
+# Speculation ON (Frank, 2026-10-19): the head is in the file
+# (qwen35.nextn_predict_layers = 1, blk.32.nextn.*) and rides the MAIN model's
+# weights, so there is no --spec-draft-model. Its bill is pool × 4,096 B of
+# draft KV plus ~0.45 GiB of buffers and a tripled recurrent state — paid for
+# by CACHE_TYPE_V=q8_0 and UBATCH_SIZE=1024 above. Build gate: quantized V
+# beside MTP needs fork build >= b10776; LLAMA_BIN is 10777.
+# SPEC_TYPE=none is the rollback and strips the WHOLE set (--spec-type and
+# every --spec-draft-*); half a set has segfaulted at load.
+SPEC_TYPE=draft-mtp
 SPEC_DRAFT_N_MAX=2
 SPEC_DRAFT_N_MIN=0
 SPEC_DRAFT_P_MIN=0
-# The card's sampler recipe lives in the checkpoint (general.sampling.* = temp
-# 1.0 / top_p 0.95 / top_k 20 / min_p 0 / repeat_penalty 1.0 — confirmed in the
-# engine's own /props defaults) and llama.cpp reads it itself. Presence penalty
-# is the ONE recipe value with no GGUF metadata key, so it must reach the argv.
-# PRESENCE_PENALTY="" is the documented opt-out.
+# The sampler recipe lives in the checkpoint (general.sampling.*) and
+# llama.cpp reads it itself. Presence penalty is the one recipe value with no
+# GGUF metadata key, so it must reach argv. "" is the documented opt-out.
 PRESENCE_PENALTY=1.0
 NGL=99
-# Boot ledger (2026-10-18 PRODUCTION boot under the unit, verbosity 4):
-#   17,922 MiB used / 6,203 MiB free of 24,123 (nvidia-smi settled)
-#   CUDA0 model 2,036.71 MiB + CPU_Mapped 257.66 MiB
-#   CUDA0 KV 13,056.00 MiB (q8_0 K + q8_0 V, 786,432 positions)
-#   CUDA0 RS 150.75 MiB (3 cells, 32 layers, 3 seqs)
-#   compute 1,409.02 MiB CUDA + 1,088.87 MiB CUDA_Host (+2.84 MiB output buf)
-#   mmproj worst-case estimate 868.32 MiB (248.10 CUDA + 24.93 CPU compute
-#   meta) — charged AFTER the engine's own fit check, as always
-#   no clamp line, no cudaMalloc error
-# Measured on this boot (/completion timings, greedy, ignore_eos, 512 tokens):
-# 168.7 t/s decode (three runs 168.4-168.7) and ~300 t/s aggregate across
-# THREE concurrent streams (102.8 each). The trial arm's record (167.2 decode,
-# 280.2 aggregate, 2,877 t/s prefill at 106,418) stands; the 27B it replaces
-# gave 76.6 / 109.4 / 691.7 at 166k. The FIRST request after a boot is a
-# warmup artifact (37.7 t/s class) and a natural generation that stops early
-# makes a wall-clock aggregate read 4x low - measure with ignore_eos and read
-# the engine's own timings.
-# Fallback rung if the margin is ever eaten (desktop, co-tenant): q4_0 KV on the
-# 27B era measured 16,392 used / 7,733 free at 2 × 200,704; the same drop here
-# frees ~6.5 GiB at three windows. UBATCH_SIZE=1024 is the cheaper lever
-# (−1,306 MiB measured on this shape).
-# Log verbosity threshold (llama.cpp -lv). 4 = debug: KV layout, buffer sizes
-# and graph capture land in the journal, which is what the VRAM-fit forensics
-# read. Levels above 4 flood the journal.
+# 4 = debug: KV layout, buffer sizes and graph capture land in the journal, and
+# that is what VRAM forensics read. Above 4 floods the journal.
 VERBOSITY=4
-
-# Serve /metrics. llama.cpp answers 501 without --metrics, and Dankclaw's
-# Prometheus scrapes /metrics on every fleet endpoint — a 501 reads as DOWN, so
-# this service has held the critical LlmEndpointDown alert for as long as the
-# metric has history (verified 2026-10-18: the oldest retained sample of
-# up{job="frank-zealot"}, 2026-10-11, is already 0; scrapeUrl
-# http://host.containers.internal:7771/metrics, lastError "server returned HTTP
-# status 501 Not Implemented"). 0 opts out and re-breaks the scrape.
+# Without --metrics llama.cpp answers 501 on /metrics, which Prometheus reads
+# as DOWN while /health stays 200. 0 opts out and re-breaks the scrape.
 METRICS=1

scripts/frank-zealot.service

@@ -1,5 +1,5 @@
 [Unit]
-Description=llama.cpp (PrismML build 10777) Ternary-Bonsai-2-9B PQ2_0-MTP GGUF (qwen35 hybrid, 3x262144 slots over one 786432 pool, q8_0 K+V, MTP off, mmproj Q8_0 vision) on the RTX 3090 (dankclaw, frank-zealot)
+Description=llama.cpp (PrismML build 10777) Ternary-Bonsai-2-9B PQ2_0-MTP GGUF (qwen35 hybrid, 2x262144 slots over one 524288 pool, unquantized f16 K+V, MTP off, mmproj Q8_0 vision) on the RTX 3090 (dankclaw, frank-zealot)
 After=network-online.target
 Wants=network-online.target
 

scripts/gen-vault-index.py

@@ -0,0 +1,206 @@
+#!/usr/bin/env python3
+"""Regenerate serverAppConfigs-docs/index.md from page frontmatter.
+
+The root index is a locator (AGENTS.md -> *Documentation Policy*): one row per
+page, one clause, no numbers. Hand-maintaining it is how rows go missing, and
+the docs seam fails when a page is not linked from it, so it is generated:
+
+  * preamble     everything above the first `## ` in the existing file is kept
+                 verbatim (that is the curated statement of the vault's law)
+  * section      from the page's directory, overridden by its frontmatter tags
+                 (`endpoint` / `retired-endpoint` split the LLM pages)
+  * row text     the page's frontmatter `description`, verbatim
+
+A page with no description is a hard error rather than an empty cell, so the
+generator cannot silently produce a silent index. Sections with no pages are
+skipped. Deterministic: the same vault always yields the same bytes.
+
+Sub-directory indices are hand-grouped (they carry curated prose), so they are
+synced rather than regenerated: dead rows dropped, row text refreshed from the
+page's description, uncovered siblings appended.
+"""
+import pathlib
+import re
+import sys
+
+RESERVED = {"index.md", "log.md"}
+SECTIONS = [
+    ("Inference endpoints", "endpoint"),
+    ("Retired endpoints", "retired-endpoint"),
+]
+DIRECTORY_SECTIONS = [
+    ("Dankclaw", "Dankclaw services"),
+    ("Dankserver", "Dankserver services"),
+    ("FrankBook-Pro", "FrankBook-Pro"),
+    ("Desktop", "Workstations"),
+    ("Infrastructure", "Cross-host infrastructure"),
+    ("Architecture", "Repo architecture and policy"),
+    ("Operations", "Operations and incidents"),
+    ("Agent", "Agent / harness"),
+    ("Research", "Research and methodology"),
+]
+PIPELINE_SECTION = "Pipeline Parallelism (series)"
+LLM_FALLBACK = "LLM technique and hardware references"
+CHANGE_SECTION = "Change history"
+ORDER = [t for t, _ in SECTIONS] + [t for _, t in DIRECTORY_SECTIONS] + \
+        [PIPELINE_SECTION, LLM_FALLBACK, CHANGE_SECTION]
+
+
+def frontmatter_fields(text):
+    if not text.startswith("---"):
+        return {}
+    end = text.find("\n---", 3)
+    if end == -1:
+        return {}
+    raw = text[3:end]
+    fields = {}
+    for key in ("description", "type"):
+        m = re.search(rf"^{key}:\s*(.*)$", raw, re.M)
+        if m:
+            fields[key] = m.group(1).strip().strip("\"'")
+    tags = []
+    m = re.search(r"^tags:\s*\[(.*?)\]$", raw, re.M)
+    if m:
+        tags = [t.strip().strip("\"'") for t in m.group(1).split(",") if t.strip()]
+    elif re.search(r"^tags:\s*$", raw, re.M):
+        block = re.search(r"^tags:\s*\n((?:\s+-\s+.*\n?)+)", raw, re.M)
+        if block:
+            tags = [l.strip().lstrip("-").strip().strip("\"'")
+                    for l in block.group(1).splitlines() if l.strip()]
+    fields["tags"] = tags
+    return fields
+
+
+def section_for(page, vault, fields):
+    rel = page.relative_to(vault)
+    if len(rel.parts) == 1:
+        return CHANGE_SECTION if page.name == "log.md" else "Uncategorised"
+    top = rel.parts[0]
+    if top == "LLM":
+        if PIPELINE_SECTION_DIR in rel.parts:
+            return PIPELINE_SECTION
+        for title, tag in SECTIONS:
+            if tag in fields.get("tags", []):
+                return title
+        return LLM_FALLBACK
+    for prefix, title in DIRECTORY_SECTIONS:
+        if top == prefix:
+            return title
+    return "Uncategorised"
+
+
+ROW = re.compile(r"^(\| )\[\[([^\]|]+)\]\]( \|)( .*?)( \|$)")
+BULLET = re.compile(r"^(- )\[\[([^\]]+)\]\]( — )(.*?)$")
+
+
+def sync_subindex(index: pathlib.Path, vault: pathlib.Path):
+    """Keep a hand-grouped sub-index honest without erasing its grouping.
+
+    Rows are locators, so their text is the linked page's own description; a
+    row whose page is gone is a dead promise and is dropped; a direct sibling
+    with no row is appended under `## Pages` (that is what the link-integrity
+    seam checks). Curated headings and prose are untouched.
+    """
+    here = index.parent
+    known = {p.stem for p in here.rglob("*.md")
+             if ".AGENTTRASH" not in str(p) and p.name != "index.md"}
+    lines = index.read_text().split("\n")
+    out, dropped, refreshed = [], 0, 0
+    for line in lines:
+        for pat, wrap in ((ROW, "| [[{}]] |{} {} |"), (BULLET, "- [[{}]] — {}")):
+            m = pat.match(line)
+            if not m:
+                continue
+            stem = m.group(2).strip()
+            if stem not in known:
+                dropped += 1
+                line = None
+                break
+            cand = next((p for p in here.rglob(f"{stem}.md")
+                         if ".AGENTTRASH" not in str(p)), None)
+            desc = frontmatter_fields(cand.read_text(errors="replace")).get("description") \
+                if cand else None
+            if desc:
+                current = m.group(4).strip()
+                if current != desc:
+                    line = wrap.format(stem, " " if pat is ROW else "", desc) \
+                        if pat is ROW else wrap.format(stem, desc)
+                    refreshed += 1
+            break
+        if line is not None:
+            out.append(line)
+    text = "\n".join(out)
+    # Presence must be the wikilink itself: a bare stem is a substring of other
+    # page names (`frank-medium` lives inside `frank-medium-rocmfpx`), and a
+    # sibling matched that way silently never gets a row.
+    missing = sorted(
+        (p.stem, frontmatter_fields(p.read_text(errors="replace")).get("description", ""))
+        for p in here.glob("*.md")
+        if p.name != "index.md" and f"[[{p.stem}]]" not in text
+    )
+    if missing:
+        block = ["", "## Pages", "", "| Page | Owns |", "|---|---|"]
+        block += [f"| [[{stem}]] | {desc} |" for stem, desc in missing]
+        text = text.rstrip("\n") + "\n" + "\n".join(block) + "\n"
+    if text != index.read_text():
+        index.write_text(text)
+    return dropped, refreshed, len(missing)
+
+
+PIPELINE_SECTION_DIR = "Pipeline Parallelism"
+
+
+def main(argv):
+    vault = pathlib.Path(argv[1] if len(argv) > 1 else "serverAppConfigs-docs")
+    index = vault / "index.md"
+    if not index.is_file():
+        print(f"missing {index}")
+        return 2
+    existing = index.read_text()
+    preamble = existing.split("\n## ", 1)[0].rstrip("\n")
+
+    pages = [p for p in vault.rglob("*.md")
+             if ".AGENTTRASH" not in str(p) and ".obsidian" not in str(p)
+             and p.name != "index.md" and p != index]
+    buckets, errors = {}, []
+    for page in pages:
+        fields = frontmatter_fields(page.read_text(errors="replace"))
+        if page.name != "log.md":
+            desc = fields.get("description")
+            if not desc:
+                errors.append(str(page.relative_to(vault)))
+                continue
+        else:
+            desc = "Chronological incident and law log for the whole bundle."
+        buckets.setdefault(section_for(page, vault, fields), []).append((page.stem, desc))
+    if errors:
+        print("pages with no description (fix the frontmatter, not the index):")
+        for e in sorted(errors):
+            print("  " + e)
+        return 1
+
+    out = [preamble, ""]
+    for title in ORDER + sorted(set(buckets) - set(ORDER)):
+        rows = sorted(buckets.get(title, []), key=lambda r: r[0].lower())
+        if not rows:
+            continue
+        out += [f"## {title}", "", "| Page | Owns |", "|---|---|"]
+        out += [f"| [[{stem}]] | {desc} |" for stem, desc in rows]
+        out.append("")
+    out.append("<!-- generated by scripts/gen-vault-index.py; edit page frontmatter instead -->")
+    index.write_text("\n".join(out) + "\n")
+    touched = 0
+    for sub in sorted(vault.rglob("index.md")):
+        if sub == index:
+            continue
+        dropped, refreshed, added = sync_subindex(sub, vault)
+        if dropped or refreshed or added:
+            touched += 1
+            print(f"  {sub}: -{dropped} dead rows, {refreshed} refreshed, +{added} added")
+    print(f"{index}: {sum(len(v) for v in buckets.values())} rows in "
+          f"{len([t for t in ORDER if buckets.get(t)])} sections, {touched} sub-indices synced")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv))

scripts/port-census.py

@@ -0,0 +1,100 @@
+#!/usr/bin/env python3
+"""Port census: prove a merge/collapse lost no fact.
+
+For every line that existed in the OLD set and not in the NEW set, check that
+each of its significant tokens (numbers, long identifiers, paths) still occurs
+somewhere in the NEW corpus. A line whose every token survives is a reworded
+line; a line with an unheld token is a candidate fact loss and gets printed.
+
+  port-census.py <new-glob-dir> <old-file> [<old-file> ...]
+
+Old files may be `git-rev:path` (read from git) or a path (e.g. a trash copy).
+"""
+import pathlib
+import re
+import subprocess
+import sys
+
+
+def toks(line):
+    out = []
+    for n in re.findall(r"\d[\d,]*(?:\.\d+)?", line):
+        if len(n.replace(",", "").split(".")[0]) >= 3 or (
+            "." in n and len(n.split(".")[-1]) >= 2 and len(n.replace(",", "")) >= 4
+        ):
+            out.append(n)
+    # Identifiers, flags, paths and backticked spans are the facts worth
+    # tracking; ordinary prose words are not — they reword on every rewrite.
+    # A hyphen alone is not evidence of an identifier (`long-standing`), so a
+    # dashed word counts only with two hyphens (`frank-tts-engine`) or with a
+    # digit, dot, slash, colon or `--` inside it.
+    for w in re.findall(r"[A-Za-z][\w./:-]{7,}", line):
+        core = w.strip(".,;:!?\"')]")
+        if (core.startswith("--") or core.isupper()
+                or re.search(r"[\d_/:.]", core) or core.count("-") >= 2):
+            out.append(core)
+    out += [s.strip() for s in re.findall(r"`([^`]+)`", line) if len(s) > 3]
+    return out
+
+
+def held(tok, corpus):
+    t = tok.replace(",", "")
+    alt = rf"\b{re.escape(t)}\b"
+    for c in corpus:
+        if tok in c or t in c:
+            return True
+        if re.search(alt, c.replace(",", "")):
+            return True
+    return False
+
+
+def main(argv):
+    new_dir = pathlib.Path(argv[1])
+
+    def corpus_excluding(exclude):
+        """Every vault page except the one under test.
+
+        The corpus is what must still hold a fact AFTER the file is gone, so a
+        file that lives inside the scanned directory has to be removed from it —
+        otherwise every token is "held" by the file itself and the census proves
+        nothing (it reports 0 losses for every page, which is the bug this
+        guards).
+        """
+        out = []
+        for p in new_dir.rglob("*.md"):
+            s = str(p)
+            if ".AGENTTRASH" in s or ".obsidian" in s:
+                continue
+            if exclude is not None and p.resolve() == exclude:
+                continue
+            out.append(p.read_text(errors="replace"))
+        return out
+
+    lost = checked = 0
+    for spec in argv[2:]:
+        if spec.startswith("git-"):
+            ref, path = spec[4:].split(":", 1)
+            old = subprocess.run(["git", "show", f"{ref}:{path}"],
+                                 capture_output=True, text=True).stdout
+            label = f"{ref}:{path}"
+            exclude = None
+        else:
+            old = pathlib.Path(spec).read_text(errors="replace")
+            label = spec
+            exclude = pathlib.Path(spec).resolve()
+        blob = "\n".join(corpus_excluding(exclude))
+        for line in old.splitlines():
+            s = line.strip().strip("->*|# ").strip()
+            if len(s) < 30 or s.startswith("```"):
+                continue
+            checked += 1
+            missing = [t for t in toks(s) if not held(t, [blob])]
+            if missing:
+                lost += 1
+                print(f"  {label}\n    L: {s[:150]}\n    unheld: {', '.join(missing[:6])}")
+    print(f"{checked} old lines checked, {lost} with unheld tokens")
+    return 1 if lost else 0
+
+
+if __name__ == "__main__":
+    sys.exit(main(sys.argv))

scripts/serve-frank-zealot.sh

@@ -3,33 +3,47 @@
 # in the file) as frank-zealot on Dankclaw's RTX 3090 via llama.cpp (PrismML
 # vendor build 10777, CUDA sm_86).
 #
-# 2026-10-18 (Frank): Bonsai-2 27B PQ2_0 → the 9B MTP export at THREE
-# 262,144-token conversations (--ctx-size 786432 --parallel 3), q8_0 K + q8_0 V.
-# The 9B bills 17,408 B/token against the 27B's 34,816 (8 full-attention layers
-# instead of 16), and that halved rate is what funds three NATIVE windows where
-# the 27B could only afford two cut to 200,704. Verbosity 4 ships so the
-# VRAM-fit forensics (KV layout, buffer sizes, graph capture) land in the
-# journal.
+# 2026-10-18 (Frank): Bonsai-2 27B PQ2_0 → the 9B MTP export, then the same
+# evening TWO 262,144-token conversations (--ctx-size 524288 --parallel 2) over
+# an UNQUANTIZED f16 KV cache.
+#
+# 2026-10-19 (Frank): THREE 200,704-token conversations over a 602,112 pool
+# (--ctx-size 602112 --parallel 3), K left at the engine's own default (f16),
+# V dropped to q8_0, batch 1024×6 / ubatch 1024, and the MTP drafter ARMED
+# (--spec-type draft-mtp). The 9B bills 8,192 elements per token per cache (8
+# full-attention layers instead of the 27B's 16), so f16 K is 16,384 B/token and
+# q8_0 V is 8,704: 25,088 B/token = 14,406.00 MiB at the 602,112 pool. That
+# 7,680 B/token against exact V is exactly what funds the third conversation
+# AND a draft context that bills f16 KV over the whole pool. Verbosity 4 ships
+# so the VRAM-fit forensics (KV layout, buffer sizes, graph capture) land in
+# the journal.
 #
 # Architecture (engine ledger, not the marketing page): qwen35 hybrid, 32
 # layers with qwen35.full_attention_interval = 4 ⇒ 8 full-attention + 24 Gated
 # DeltaNet, 4 KV heads × 256 dim, native 262,144. The KV pool scales with the 8
-# full layers (13,056.00 MiB at the 786,432 pool, ledger-exact); the DeltaNet
-# layers cost a fixed recurrent state per SLOT — 452.25 MiB for 3 cells, ~151
-# MiB each. The runtime is a hard lock, not a preference: custom GGML type 142
-# plus prism.hadamard.* metadata, which only the vendor build registers, and
-# MTP beside quantized V needs build ≥ b10776 — so the prebuilt moved 10754 →
-# 10777 with the payload. Vision comes from the publisher's Q8_0 projector (the
-# BF16 one can crash the server on concurrent images, and this endpoint runs
-# three slots), so --mmproj is what gives this endpoint eyes; MMPROJ_FILE="" is
-# the documented text-only opt-out.
+# full layers (14,406.00 MiB of f16 K + q8_0 V at the 602,112 pool); the
+# DeltaNet layers cost a fixed recurrent state per SLOT — ~50.25 MiB a cell, so
+# 150.75 MiB for three, and an armed depth-2 drafter triples that again. The
+# runtime is a hard lock, not a preference: custom GGML type 142 plus
+# prism.hadamard.* metadata, which only the vendor build registers — and the
+# b10776 gate that moved the prebuilt 10754 → 10777 (MTP beside a quantized V)
+# is LOAD-BEARING again on this shape, not moot as it was under exact KV.
+# Vision comes from the publisher's Q8_0 projector (the BF16 one can crash the
+# server on concurrent images, and this endpoint runs three slots), so --mmproj is
+# what gives this endpoint eyes; MMPROJ_FILE="" is the documented text-only
+# opt-out.
 #
-# Speculative decoding ships OFF, measured: the head loads and accepts well
-# (0.798/0.617 per position, mean length 2.41) but the draft context bills its
-# own f16 KV over the WHOLE pool — 4,096 B/position × 786,432 = 3,072 MiB plus
-# buffers — and at ubatch 2048 that squeeze HALVES decode (167.2 → 109.2 t/s).
-# frank-zealot.env carries the six-arm table and the arm that turns it back on
-# (--ubatch-size 1024, decode 213.5 t/s) with its price.
+# Speculative decoding ships ON (Frank, 2026-10-19). The head is IN the file and
+# accepts well (0.798/0.617 per position, mean length 2.41, measured 2026-10-18);
+# what made it lose that evening was the bill, not the drafts — the draft context
+# is built against the target's WHOLE pool, so blk.32 adds 4,096 B/position
+# (2,352.00 MiB at 602,112) plus 480.97 MiB of its own compute buffers on the
+# card, and it triples the recurrent state per slot (452.25 MiB for three cells
+# against 50.25 a cell with the drafter off). CACHE_TYPE_V=q8_0 is what pays
+# that: 7,680 B/token cheaper than exact V, 4,410 MiB at this pool. The ubatch
+# drop to 1024 contributes ~406 MiB here, NOT the ~1,306 the 786,432-pool census
+# showed — that figure does not transfer between shapes. SPEC_TYPE=none is
+# the rollback and strips the whole flag set.
 #
 # Sampling: the checkpoint carries the publisher's thinking recipe as GGUF
 # metadata (general.sampling.* = temp 1.0 / top_p 0.95 / top_k 20 / min_p 0 /
@@ -56,43 +70,46 @@ fi
 : "${ALIAS:=frank-zealot}"
 : "${BIND_ADDR:=0.0.0.0}"
 : "${LISTEN_PORT:=7771}"
-# ctx-size is the POOL llama.cpp divides across slots: 786,432 / 3 = 262,144
-# per conversation — every slot at the model's trained window. The harness
-# contextWindow MUST quote the PER-SLOT number — a longer prompt is answered
-# with a 400, never truncated.
-: "${CTX_SIZE:=786432}"
+# ctx-size is the POOL llama.cpp divides across slots: 602,112 / 3 = 200,704 a
+# conversation. The harness contextWindow MUST quote the PER-SLOT number — a
+# longer prompt is answered with a 400, never truncated. The pool is priced by
+# the KV bill, not by the trained window: at 25,088 B/token (f16 K + q8_0 V) the
+# 602,112 pool costs 14,406.00 MiB and the armed drafter adds its own f16 KV over
+# the whole pool (2,352.00 MiB). Exact V at this pool (18,816 MiB) or the old
+# 786,432 pool at any dtype cannot boot inside the 24,123 MiB the driver leaves
+# addressable.
+: "${CTX_SIZE:=602112}"
 : "${PARALLEL:=3}"
 # Compute-buffer geometry. Ubatch is a prefill knob (decode never moves).
-# 2048 is the measured CUDA optimum (MiMo era, same arch/shape: +4.6 % prefill
-# at 8.9k depth, decode flat, 616 MiB). The logical batch is VRAM- and
-# throughput-neutral at a fixed ubatch. It ALSO sets the compute-buffer size,
-# which is the term that decides whether an MTP draft context fits beside a
-# 786,432-position pool: 1024 hands back 1,306 MiB and is the arm that runs the
-# drafter at three slots (frank-zealot.env carries the table).
-: "${BATCH_SIZE:=4096}"
-: "${UBATCH_SIZE:=2048}"
-# Quantized KV: q8_0 K + q8_0 V — the only pair that funds 3 × 262,144 on this
-# card. 17 KiB/token at this geometry (8 full-attention layers × 4 KV heads ×
-# 256 dim); 13,056.00 MiB at the 786,432 pool, ledger-exact. f16 would bill
-# 26,112 MiB against a 24,123 MiB card and dies at cudaMalloc; q4_0 is the
-# fallback rung. Explicit either way so the contract is visible in the argv,
-# and --flash-attn stays required for quantized V (no true fp8 KV dtype exists
-# in this build).
-: "${CACHE_TYPE_K:=q8_0}"
+# The logical batch is VRAM- and throughput-neutral at a fixed ubatch, so 6144
+# (1024×6) is free; the ubatch is what sets the compute-buffer size, and 1024 is
+# the rung every measured MTP arm here ran on (213.5 t/s at 1024 against 109.2
+# at 2048) — worth ~406 MiB at THIS shape, not the ~1,306 the old pool showed.
+: "${BATCH_SIZE:=6144}"
+: "${UBATCH_SIZE:=1024}"
+# Asymmetric KV: f16 K (the engine's own default, spelled out so the contract is
+# visible in argv) + q8_0 V. 8,192 elements per token per cache (8 full-attention
+# layers × 4 KV heads × 256 dim) ⇒ 16,384 B/token for K and 8,704 for V (q8_0 is
+# 1.0625 B/element — llama.cpp has no true fp8 KV dtype), 25,088 B/token and
+# 14,406.00 MiB at the 602,112 pool. That is the halfway door between exact KV
+# (32,768 B/token, which costs the third conversation and the drafter both) and
+# the fully quantized pair (17,408 B/token, which costs retrieval fidelity in the
+# keys); --flash-attn stays pinned because quantized V is illegal without it.
+: "${CACHE_TYPE_K:=f16}"
 : "${CACHE_TYPE_V:=q8_0}"
-# Speculative decoding defaults OFF, measured on this card and shape: the head
-# IS in the checkpoint (qwen35.nextn_predict_layers = 1 — MTP is a property of
-# the FILE, so the plain PQ2_0 sibling could not run it at any flag value), it
-# loads, and it accepts well (mean accepted length 2.41) — but the draft context
-# bills its own f16 KV over the WHOLE pool (4,096 B/position, 3,072 MiB here)
-# and at ubatch 2048 that squeeze halves decode: 167.2 → 109.2 t/s. There is no
-# --spec-draft-cache-type knob in this build to make the draft cheap, and no
-# --spec-draft-model either: the draft rides the main model's weights. Card gate:
-# quantized V alongside MTP needs fork build >= prism-b10776, which is why
-# LLAMA_BIN moved to the 10777 prebuilt; on an older build the pair is illegal.
-# SPEC_TYPE=draft-mtp + UBATCH_SIZE=1024 is the documented MTP arm (213.5 t/s);
-# SPEC_TYPE=none strips the whole set (flags move together).
-: "${SPEC_TYPE:=none}"
+# Speculative decoding defaults ON: the head IS in the checkpoint
+# (qwen35.nextn_predict_layers = 1 — MTP is a property of the FILE, so the plain
+# PQ2_0 sibling could not run it at any flag value) and the draft context rides
+# the MAIN model's weights, so there is no --spec-draft-model. Its bill is the
+# target's whole pool at 4,096 B/position plus ~0.45 GiB of buffers at this
+# ubatch and a tripled recurrent state; CACHE_TYPE_V=q8_0 and UBATCH_SIZE=1024
+# above are what pay for it. The draft cache dtype IS a further knob
+# (--spec-draft-type-k / --spec-draft-type-v, both default f16 — verified in this
+# build's --help) and stays at its default here. Card gate: quantized V alongside
+# MTP needs fork build >= prism-b10776, which is why LLAMA_BIN is the 10777
+# prebuilt; the type-142 loader lock keeps it pinned regardless.
+# SPEC_TYPE=none is the rollback and strips the whole set (flags move together).
+: "${SPEC_TYPE:=draft-mtp}"
 : "${SPEC_DRAFT_N_MAX:=2}"
 : "${SPEC_DRAFT_N_MIN:=0}"
 : "${SPEC_DRAFT_P_MIN:=0}"
@@ -104,7 +121,7 @@ fi
 : "${NGL:=99}"
 # Log verbosity threshold (llama.cpp -lv). 3 is the engine default (info);
 # 4 adds debug detail (KV layout, buffer sizes, graph capture) which is what
-# the VRAM-fit forensics need — the q8_0 KV contract and the draft-KV bill are
+# the VRAM-fit forensics need — the exact-KV size and the draft-KV bill are
 # only provable from this log. Levels above 4 flood the journal.
 : "${VERBOSITY:=4}"
 # /metrics needs the flag: llama.cpp returns 501 unless the server starts with
@@ -169,11 +186,11 @@ fi
 # open after boot; make sure it exists before the server needs it.
 nvidia-modprobe -u -c=0 2>/dev/null || true
 
-# Flash attention stays pinned on: it is REQUIRED for quantized V (q8_0),
-# and the build carries GGML_CUDA_FA_ALL_QUANTS=ON (since 2026-09-17), so
-# the q8_0 KV contract in frank-zealot.env is a real path — without this
-# flag or that build option the engine silently falls back and the
-# contract becomes a lie.
+# Flash attention stays pinned on. Exact f16 KV runs fine with or without it,
+# but the documented fallback rung (quantized V) is ILLEGAL without it, and the
+# build carries GGML_CUDA_FA_ALL_QUANTS=ON (since 2026-09-17) so that rung is a
+# real path — dropping the flag here would leave the retreat route silently
+# falling back to something the env file does not describe.
 ARGS=(
     --model "$MODEL_FILE"
     --alias "$ALIAS"

serverAppConfigs-docs/Agent/Agent Docs and Edits.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Agent Docs and Edits
-description: Two self-inflicted damage classes — structurally broken markdown after parallel agent edits, and botched hashline PUT/CUT repairs — with the audit checks and recovery order for each.
+description: "Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each."
 tags: [agents, docs, markdown, hashline, edit-recovery]
 timestamp: 2026-10-18T00:00:00Z
 ---

serverAppConfigs-docs/Agent/Command Center Dotfiles.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Command Center Dotfiles
-description: The ~/.omp repo is itself the deployment — stow-equivalent packages, full-replica rsync transport, the pre-overwrite audit for non-blank hosts, deploy.sh mechanics, and the fish/sshpass transport gotchas.
+description: "How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas."
 tags: [dotfiles, omp, deploy, rsync, blankpad]
 timestamp: 2026-10-05T00:00:00-07:00
 ---

serverAppConfigs-docs/Agent/OMP Harness.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: OMP Harness
-description: The omp agent harness as configured on this fleet — compaction chain arithmetic, per-model sampler routing via compat.extraBody, and the static-preamble/skill-library audits that keep session context lean.
+description: "The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits."
 tags: [omp, harness, compaction, models-yml, context, samplers]
 timestamp: 2026-10-18T00:00:00Z
 ---
@@ -14,16 +14,16 @@ registry; single source of truth — there is no copy on the servers) and
 this file is repo-managed; edits are harness-side only.
 
 **Model ids are the StarCraft unit names (official 2026-10-08, on the wire
-since 2026-10-09):** `frank-archon` (dankserver :7770, radiance),
-`frank-dragoon` (dankclaw :7770, flash-server), `frank-zealot` (dankclaw
-:7771, the 3090 compaction hub) — formerly `frank-archon`, `frank-dragoon`,
+since 2026-10-09):** `frank-archon` (Dankserver), `frank-dragoon` and
+`frank-zealot` (Dankclaw) — engine, port and card behind each id live on
+[[frank-archon]], [[frank-dragoon]] and [[frank-zealot]] — formerly `frank-archon`, `frank-dragoon`,
 `frank-zealot`. The SERVICES keep the old names (units, scripts, env
 files, runbook pages); pages written in the old era name the service,
 the harness names the model. The wire now matches: each endpoint's
 `/v1/models` answers with the Protoss name, so `models.yml`, the OpenWebUI
 picker/grants, the homepage docs and the raw API all use one id per model
-(see [[OpenWebUI]] for the grant re-key and the [[frank-archon-radiance-swift15-w4dn8]]
-note on radiance echoing the requested name).
+(see [[OpenWebUI]] for the grant re-key and the [[frank-archon]]
+note on the endpoint echoing the requested name).
 
 Verified 2026-10-18: `/v1/models` on 192.168.0.235:7770, 192.168.0.143:7770 and
 192.168.0.143:7771 each answer with the unit name (`frank-archon`,
@@ -50,10 +50,11 @@ Where the contract lives:
   `~/.omp/plugins/node_modules/@earendil-works/pi-coding-agent/dist/core/compaction/compaction.js`
 
 The arithmetic (the coupled contract every endpoint's maxTokens comment
-encodes). Recomputed from the live files 2026-10-18 — contextWindows
-200,000 (archon) / 155,648 (dragoon) / 200,704 (zealot) at
-`models.yml:48,131,206`; maxTokens 16,384 / 45,056 / 16,384 at
-`models.yml:56,143,212`:
+encodes). The authoritative `contextWindow`/`maxTokens` pair per model is on the
+endpoint page ([[frank-archon]], [[frank-dragoon]], [[frank-zealot]]); what
+follows is a **worked example** recomputed from the live files on 2026-10-18 —
+a snapshot, not fleet state — with windows 200,000 / 155,648 / 200,704 and
+maxTokens 16,384 / 45,056 / 16,384 (archon / dragoon / zealot):
 
 1. **Source trigger** = `thresholdPercent × source contextWindow`; at the live
    95 % that is 190,000 (archon), 147,866 (dragoon), 190,669 (zealot). The
@@ -141,28 +142,25 @@ Use **wire names**, not OMP camelCase. Two traps:
 2. **Never put `chat_template_kwargs` in `extraBody`** where a compat policy
    also sets it — `Object.assign` is shallow and replaces the whole object,
    silently dropping the `reasoning_effort` kwarg the compat policy set.
-   Top-level `preserve_thinking` is honoured by halogen. Live state 2026-10-18:
-   frank-archon's radiance entry carries `chat_template_kwargs:
-   {enable_thinking, preserve_thinking}` in `extraBody` (`models.yml:89-91`)
-   and nothing else sets that object at that endpoint, so the shallow-merge
-   hazard is latent rather than firing; the FreeToken-era "verify-and-prune at
-   the next swap" note is retired with that swap (2026-10-05,
-   `scripts/serve-frank-archon.sh:9-10`).
+   Top-level `preserve_thinking` is honoured by frank-dragoon
+   ([[frank-dragoon]]). Live state 2026-10-18: frank-archon's `extraBody`
+   carries `chat_template_kwargs: {enable_thinking, preserve_thinking}`
+   (`models.yml:89-91`) and nothing else sets that object at that endpoint, so
+   the shallow-merge hazard is latent rather than firing.
 
-**Gotcha (retired 2026-10-17): silent greedy decode.** Earlier flash-server
+**Gotcha (retired 2026-10-17): silent greedy decode.** Earlier frank-dragoon
 builds carried **no server-side sampler defaults**, so an absent
 `temperature` decoded greedy while the endpoint looked configured. That era is
-over: `HALOGEN_TEMPERATURE/TOP_P/TOP_K` now set the card's thinking recipe
-(1.0 / 0.95 / 20) as engine defaults (`scripts/frank-dragoon.env:98-103`,
-exported at `scripts/serve-frank-dragoon.sh:186-188`), so an absent temperature
-is no longer greedy. frank-dragoon's `extraBody` still carries the same trio —
-consistent, not load-bearing. Treat any older "do not drop it or you get
-greedy" wording as history.
+over — the endpoint now ships its own defaults ([[frank-dragoon]]), so an
+absent temperature is no longer greedy. frank-dragoon's `extraBody` still
+carries the same trio — consistent, not load-bearing. Treat any older "do not
+drop it or you get greedy" wording as history.
 
-**Reasoning-effort folding.** Qwen3.8's template accepts only
-`{xhigh (default), medium, low}` for `reasoning_effort` and 400s on anything
-else ("Unexpected reasoning effort high"). Every endpoint's
-`reasoningEffortMap` folds all harness levels onto that set; a role whose
+**Reasoning-effort folding.** The set `reasoning_effort` accepts is a property
+of the endpoint, not of the harness: [[frank-dragoon]] and [[frank-zealot]]
+raise on anything outside `xhigh|medium|low`, while [[frank-archon]] does not
+gate the set. Every endpoint's `reasoningEffortMap` folds all harness levels
+onto whichever set that endpoint ships; a role whose
 level is unmapped fails on its FIRST request, and the failure names Jinja,
 not the model.
 

serverAppConfigs-docs/Agent/index.md

@@ -1,23 +1,12 @@
----
-type: index
-description: Pages about the agent harness itself — omp config, vault editing discipline, and the dotfiles repo that deploys the agent's own home.
-tags: [agent, index]
-timestamp: 2026-10-05T00:00:00-07:00
----
-
 # Agent
 
 Pages about the agent's own tooling and context — how omp is configured, how
 docs get edited safely, and how the agent's home is deployed.
 
-- [[OMP Harness]] — compaction chain arithmetic, per-model samplers via
-  `compat.extraBody`, preamble and skill-library audits
-- [[Agent Docs and Edits]] — doc-structure audits after parallel edits;
-  hashline edit recovery
-- [[Command Center Dotfiles]] — the `~/.omp` repo-as-deployment, rsync
-  transport, override packages
+- [[OMP Harness]] — The omp harness as configured here: compaction chain arithmetic, per-model sampler routing, and the context-lean audits.
+- [[Agent Docs and Edits]] — Two self-inflicted doc damage classes from parallel agent edits, with the audit checks and recovery order for each.
+- [[Command Center Dotfiles]] — How the dotfiles repo deploys: stow-equivalent packages, full-replica rsync, the pre-overwrite audit, transport gotchas.
 
 ## Related
 
-- [[Testing Policy]] — the seams these pages' checks ride
 - [LLM](../LLM/index.md) — the fleet these agents operate

serverAppConfigs-docs/Architecture/Container Network Namespaces.md

@@ -1,8 +1,7 @@
 ---
 type: reference
 title: Container Network Namespaces
-description: Which local services are addressable from which Podman network namespace on
-  Dankclaw, and why bridge-network containers cannot dial host-netns loopback listeners.
+description: "How Podman builds container networks here: the network inventory, the netns sharing mechanism, and the isolation limits."
 tags:
   - networking
   - podman

serverAppConfigs-docs/Architecture/Deploy State Triage.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Deploy State Triage
-description: Reading and reconciling the deploy-state markers — stale marker vs live container, pending-service lockout, interrupted/raced deploys — without restarting healthy heavy endpoints.
+description: "Reading deploy-state markers against live containers and deciding, safely, whether to redeploy."
 tags: [deployment, deploy-state, systemd, podman, triage]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -22,8 +22,10 @@ marker. So `marker != HEAD` means "not dispatched via deploy.sh", NOT
 "behavior pending".
 
 **Why it matters:** a plain `deploy.sh` run re-selects and restarts a service
-whose marker lags. On llama.cpp/R9V endpoints that is a ~16–19 min reload of
-a 100+ GiB payload — and frank-archon (dankserver:7770) is also the
+whose marker lags. On the big-payload endpoints that is a full model reload
+measured in tens of minutes — each endpoint's boot budget is on its own page
+([[frank-archon]], [[frank-dragoon]], [[frank-zealot]]) — and frank-archon
+(dankserver:7770) is also the
 harness's own model backend, so a blind deploy is a self-inflicted outage for
 possibly zero behavioral change.
 
@@ -46,9 +48,8 @@ possibly zero behavioral change.
    - `podman inspect <ctr> --format '{{.Args}}' | tr ' ' '\n' | grep -i <flag>`
    - engine env: `podman exec <ctr> sh -c 'for f in /proc/[0-9]*/environ; do
      tr "\0" "\n" <$f 2>/dev/null; done | grep -i <VAR>'`
-   - confirm the launcher maps each tunable to its flag
-     (`grep scripts/launch.sh`), e.g. `R9V_ENABLE_PREFIX_CACHING=1 →
-     --enable-prefix-caching`.
+  - confirm the launcher maps each tunable to its flag
+    (`grep scripts/launch.sh`).
 
 **Decision:** live argv+env == target config AND `/health` 200 ⇒ behavior is
 live — **do NOT restart**; reconcile the marker (below). Only if the live

serverAppConfigs-docs/Architecture/Deployment Flow.md

@@ -1,5 +1,5 @@
 ---
-type: guide
+type: howto
 title: Deployment Flow
 description: Service-scoped Git deployment for the two homelab hosts.
 tags:
@@ -46,7 +46,7 @@ graph LR
 ./scripts/deploy.sh dankserver
 ./scripts/deploy.sh frankbook-pro --service qbittorrent   # podman-machine target
 ./scripts/deploy.sh dankclaw --service searxng
-./scripts/deploy.sh dankserver --service frank-medium
+./scripts/deploy.sh dankserver --service frank-archon
 ./scripts/deploy.sh dankclaw --dry-run
 ```
 
@@ -62,13 +62,13 @@ service on the other host.
 |---|---|---|
 | Dankclaw compose services | `podman-compose -f dankclaw/compose.yaml up -d --no-deps NAME` | Recreates only `NAME`; Caddy is handled separately because it fronts both hosts. |
 | SearXNG, Gluetun, Valkey, OpenWebUI, Pi-hole, Gitea database/server | user Quadlet + `systemctl --user` | Installs the selected unit and restarts only the required dependency-coupled units. Gitea's web proxy is the canonical Caddy compose service. |
-| `frank-dragoon` | systemd + `systemctl` (root unit) | Relinks and restarts only the large inference service; waits up to 20 minutes for `/health`. |
-| `frank-zealot` | systemd + `systemctl` (root unit) | Relinks and restarts only the small inference service; waits up to 20 minutes for `/health`. |
-| `frank-medium` | rootless Quadlet + `systemctl --user` | Installs `dankserver/frank-medium/quadlet/frank-medium.container`; on first deploy retires the llama.cpp root unit and the vLLM replica quadlets into the home trash, then restarts the generated unit. Waits up to 20 minutes for `/health`. |
+| `frank-dragoon` | systemd + `systemctl` (root unit) | Relinks and restarts only the Dankclaw iGPU inference service; waits up to 20 minutes for `/health`. Shape, gates and boot budget: [[frank-dragoon]]. |
+| `frank-zealot` | systemd + `systemctl` (root unit) | Relinks and restarts only the Dankclaw 3090 inference service; waits up to 20 minutes for `/health`. Shape, gates and boot budget: [[frank-zealot]]. |
+| `frank-archon` | systemd + `systemctl` (root unit) | `deploy_llm_archon` plans from repo paths and gates on the port; it does NOT see host-side unit drift (see [[frank-archon]] → Deploy / verify). |
 | Dankserver media stack (RomM trio, Jellyfin, Calibre-Web, Copyparty) | user Quadlet + `systemctl --user` | Installs `dankserver/<svc>/quadlet/*.container`, restarts the generated unit. The first cutover per service is idempotent: it retires the compose-era bare container, rescues any anonymous volume into the data root, and merge-moves `/opt/<svc>` to `/home/dank/Containers/<svc>` (mapped-uid roots move without chown). Steady state is restart-only. |
 
 The shared chat template maps to the inference service on the selected target.
-The `frank-npu` endpoint is retired (2026-09-11) and has no deploy handler;
+The `frank-npu` endpoint is retired ([[frank-npu]]) and has no deploy handler;
 an unrelated large-model deployment never touches it.
 
 ## Caddy safety

serverAppConfigs-docs/Architecture/E2E Suite.md

@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "`tests/e2e/` drives the **live fleet** through a real browser: fresh contexts"
+description: "The Playwright suite that drives the live fleet through a real browser, its project set, and the FrankBook conditional."
 tags:
   - architecture
   - e2e

serverAppConfigs-docs/Architecture/Quadlet Lifecycle.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Quadlet Lifecycle
-description: The full Quadlet discipline on this fleet — ownership auditing, compose-to-Quadlet cutover, drift/crash-loop diagnosis, the podman 5.x v1 generator quirks, log retrieval, and compose bring-up with ground truth.
+description: "The Quadlet discipline: ownership audits, compose cutover, drift and crash-loop triage, generator quirks, log retrieval."
 tags: [podman, quadlet, systemd, boot-persistence, compose, drift]
 timestamp: 2026-10-05T00:00:00-07:00
 ---

serverAppConfigs-docs/Architecture/Stale Signal Audit.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Stale Signal Audit
-description: The audit battery for surfaces that rot without failing — unreachable probes, vacuous assertions, invalid checkers, gates that never run, and prose invalidated by a change. Four passes found real defects every time.
+description: "The audit battery for surfaces that rot without failing: dead probes, vacuous assertions, gates that never run."
 tags: [testing, observability, audit, signals, gates]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -34,8 +34,9 @@ port; a "probe" job scraping `/` (HTML → `unsupported Content-Type`); a
 `/metrics` that is disabled (Gitea 404).
 
 - A scrape cannot express "is this HTTP service answering?" — the right probe
-  is the **blackbox exporter** in probe mode (which also gives guarded vhosts
-  a real uptime signal, since Prometheus cannot solve an Anubis challenge).
+  is the **blackbox exporter** in probe mode
+  (`/probe?target=…&module=http_2xx`), which also gives guarded vhosts
+  a real uptime signal, since Prometheus cannot solve an Anubis challenge.
 - **Fix in the direction of honesty, never of silence:** unreachable target →
   mark the homepage card `wip: true` with the reason in `description` (the
   checker skips `wip`; `wip` is the existing convention for "real service,
@@ -43,8 +44,8 @@ port; a "probe" job scraping `/` (HTML → `unsupported Content-Type`); a
   database port on the LAN to make a badge green, and never delete a scrape
   job — an absent scrape and a healthy one are indistinguishable.
 - Alert rules must not depend on always-red signals — scope them to services
-  that can actually be healthy (`LlmEndpointDown` is scoped to `frank-*` for
-  exactly that reason).
+  that can actually be healthy; the `frank-*` scope `LlmEndpointDown` carries
+  is recorded on [[Prometheus]].
 - A target's failure STRING can prove a fix: a `/metrics` scrape flipping from
   `connection refused` to `unsupported Content-Type "text/html"` means the
   service came up and only the scrape is wrong. Read the error, not just

serverAppConfigs-docs/Dankclaw/ComfyUI Qwen-Image Workflows.md

@@ -1,6 +1,6 @@
 ---
 type: howto
-description: "Qwen-Image 2.1 workflows on Dankclaw: two proven txt2img workflows (art + legible text), the API recipe, and which models/LoRAs to add next."
+description: "Proven Qwen-Image text-to-image workflows on Dankclaw: the recipes, the API call shape, and which models to add next."
 tags:
   - dankclaw
   - comfyui

serverAppConfigs-docs/Dankclaw/Dankclaw Stack.md

@@ -13,11 +13,10 @@ timestamp: 2026-08-26
 # Dankclaw Stack
 
 Dankclaw (192.168.0.143) runs the AI, search, development, and observability
-stack. Its LLM inference endpoint `frank-dragoon` (`:7770`) is a ROCmFPX
-llama.cpp fork serving the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF on
-the card's pinned runtime; an NPU endpoint (`frank-npu`, `:7771`,
-FLM v1.0.1) serves sight+sound models. The completion endpoint
-`frank-medium` (ROCmFPX llama.cpp, `:7770`) runs on Dankserver (192.168.0.235).
+stack. Its LLM inference endpoints are bare-metal host services, not part of
+the container stack; each has its own page — shape, payload, eras and wire
+contract live there, nowhere else. See [[frank-dragoon]] and [[frank-zealot]]
+(both on this host) and [[frank-archon]] (Dankserver, 192.168.0.235).
 
 ```mermaid
 graph LR
@@ -28,9 +27,8 @@ graph LR
     C --> F[ComfyUI]
     C --> D[Grafana]
     S --> V[Gluetun VPN]
-    O --> L[llama.cpp dankserver :7770]
+    O --> L[LLM endpoints]
     O --> H[Hermes]
-    O --> E[frank-dragoon :7770]
 ```
 
 ## Services
@@ -48,18 +46,17 @@ graph LR
 | Seerr | `seerr.franksdank.cloud` (guarded) | Caddy routes post-auth to `seerr:5055` | [[Seerr]] |
 | Homepage | `franksdank.cloud` + `www.` (root site; guarded for admin actions) | served by Caddy itself from `/srv/homepage` | — |
 | Pi-hole | `pihole.franksdank.cloud` (basicauth edge, not the guard chain) | admin UI `8083`; DNS on `:53` LAN-wide | [[Pi-hole]] |
-| frank-dragoon (host service) | — | `7770` | [[deployment]] |
-| frank-npu (host service) | — | `7771` (`/v1/version` health) | [[frank-npu]] |
-| frank-medium (host service, Dankserver) | — | `7770` | [[frank-medium-rocmfpx]] |
+| frank-dragoon (host service) | — | `7770` | [[frank-dragoon]] |
+| frank-zealot (host service) | — | `7771` | [[frank-zealot]] |
+| frank-archon (host service, Dankserver) | — | `7770` | [[frank-archon]] |
 
-The LLM rows are bare-metal systemd services, not containers. This
-repo owns their scripts, units, and knowledge base (see
-[LLM fleet knowledge base](../LLM/index.md)). Current mapping (2026-09-03):
-`frank-dragoon` runs the kingjones Qwen3.8-Flash-Next-ROCmFP4-STRIX GGUF
-on the card's pinned `kingjones30/ROCmFPX` llama.cpp build;
-`frank-npu` runs FLM v1.0.1 on the XDNA2 NPU;
-`frank-medium` runs charlie12345/ROCmFPX llama.cpp serving
-kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF on Dankserver.
+The LLM rows are bare-metal systemd services, not containers, and this repo
+owns their scripts, units and knowledge base (see
+[LLM fleet knowledge base](../LLM/index.md)). Which payload and runtime sits on
+each port is era state and lives on the endpoint page, not here. `frank-npu`
+(the XDNA2 NPU endpoint) and `frank-medium` (the predecessor of `frank-archon`
+on the Dankserver port) are both retired — see [[frank-npu]] and
+[[frank-medium-radiance-mxfp4]].
 
 ## Boot Persistence
 

serverAppConfigs-docs/Dankclaw/Frank TTS Engine.md

@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "frank-tts-engine is the arm A TTS engine on Dankclaw: qwentts.cpp's resident tts-server with the 1.7 B base talker, on a private podman bridge behind frank-tts — repo-owned, not yet deployed."
+description: "The engine behind the fleet's speech endpoint: its runtime, private bridge, and the surface it publishes."
 tags:
   - dankclaw
   - tts
@@ -14,7 +14,7 @@ timestamp: 2026-10-18
 
 `frank-tts-engine` is the **arm A text-to-speech engine** on Dankclaw: [qwentts.cpp](https://github.com/ServeurpersoCom/qwentts.cpp)'s resident `tts-server` HTTP process, holding the Qwen3-TTS **1.7 B base talker (Q4_K_M)** and the **12 Hz tokenizer (Q8_0)** in RAM once and synthesizing from there. It listens on container port **8880** and is **deliberately not a fleet endpoint** — the endpoint consumers address is `frank-tts` on **7772**, which resolves voice names, re-encodes to mp3 and owns the timeout budget. This page is the engine runbook; the endpoint's contract lives with its own page.
 
-**Nothing here is deployed.** This page documents repo-owned configuration — the Quadlet unit and network, the image build, the weight fetcher, the boot warmer, the deployer path — and the measured facts those files were written against. No container, image, weight file or network exists on Dankclaw yet, and the engine publishes no port at all, so there is nothing for the homepage checker or a browser to reach. Deploying it is a deliberate next step, in the order at the bottom of this page.
+**Deployed 2026-10-18.** The chain ran in the order at the bottom of this page — fetch (1.4 GiB, byte + LFS oid verified, 18 s), image build against the pinned commit, unit + network install, container-health gate, warm pass — and the live state still reads the way this page describes it: unit `active` with no drop-ins, container healthy 31 s after the build, **4.0 GiB** resident on a host that kept 17 GiB available, `3 voice(s) registered and warm` in the journal, and no published port anywhere. Two things the first real boot taught: the warm hook's registration body must reach curl through a **file**, not an argument (Troubleshooting), and `STEP 3/13 RUN useradd` is the first honest proof of the image's non-root user.
 
 **Host:** Dankclaw (`192.168.0.143`), rootless Podman, user `dank` — **Port:** 8880, **never published** — **Network:** the private `systemd-frank-tts-engine` bridge (10.89.5.0/24) and no other — **Deployer:** `frank-tts-engine` — **Consumers:** `frank-tts` (7772) and, through it, every voice request in the fleet including [[Open Notebook]]'s podcast renders.
 
@@ -96,25 +96,27 @@ The **façade** on 7772 is the pair's public surface and carries the probe. Note
 |---|---|
 | `dankclaw/voiceservice/Containerfile.engine` | two-stage build of the pinned commit, proven cmake line, digest-pinned base, ships both scripts |
 | `dankclaw/voiceservice/tts-engine-start.sh` | image entrypoint (`/app/start.sh`): env → flags, refuses to boot without weights |
-| `dankclaw/voiceservice/tts-engine-warm.sh` | boot warmer (`/app/warm-voices.sh`): register from `reference.wav`, speak one sentence, report cold/warm |
+| `dankclaw/voiceservice/tts-engine-warm.sh` | boot warmer (`/app/warm-voices.sh`): register from `reference.wav` (body via a temp file), speak a fixed sentence twice, report cold/warm |
 | `dankclaw/voiceservice/quadlet/frank-tts-engine.container` | the unit: bridge only, no publish, env file, two read-only mounts, health probe, warm hook, `Restart=always` |
 | `dankclaw/voiceservice/quadlet/frank-tts-engine.network` | the private 10.89.5.0/24 bridge, with the subnet registry |
-| `scripts/frank-tts-engine.env` | paths and numbers only, with the thread law and the cold-voice law in its header |
+| `scripts/frank-tts-engine.env` | paths and numbers only, with the thread law and the registry law in its header |
 | `scripts/download-frank-tts-engine-qwentts.sh` | weight fetcher: no resume, byte count + LFS oid check, `--verify-only` |
 | `scripts/deploy-remote.sh` → `deploy_frank_tts_engine` | weights gate → build → quadlet → restart → container health → warm |
 | `tests/frank_tts_engine.bats` | the seam: generator output, entrypoint argv, warmer against a stub engine, planner and routing maps |
 
-## Deploy order (host steps, in order)
+## Deploy record (host steps, in the order they ran)
 
-Nothing below has been run. Steps 1–5 are this service; 6–7 are what unblocks the podcast pipeline.
+Run 2026-10-18 as `./scripts/deploy.sh dankclaw --service frank-tts-engine`, then
+`--service frank-tts`. Steps 1–6 are this service and the façade; only step 7 is
+still open.
 
 1. **Fetch the weights** — `scripts/download-frank-tts-engine-qwentts.sh` (1.4 GiB, byte + sha256 verified, idempotent; `--verify-only` is what the deployer calls).
-2. **Deploy the engine** — `./scripts/deploy.sh dankclaw --service frank-tts-engine` (`--service` is the only way to name one service; `deploy-remote.sh` is the remote half and takes `<target> <git-revision> [service]...`, so never call it by hand with a bare service name). Builds the image (needs github.com once), installs the unit and the network file, restarts, gates on the container's health status, then warms.
+2. **Deploy the engine** — `./scripts/deploy.sh dankclaw --service frank-tts-engine` (`--service` is the only way to name one service; `deploy-remote.sh` is the remote half and takes `<target> <git-revision> [service]...`, so never call it by hand with a bare service name). It builds the image (needs github.com once), installs the unit and the network file, gates on the container's health status, then warms.
 3. **Confirm the endpoint is alive** — `podman exec frank-tts-engine curl -fsS http://127.0.0.1:8880/health` → `{"status":"ok"}`, and `systemctl --user status frank-tts-engine` showing the unit `active` with the warm pass behind it.
-4. **Confirm the voices are known** — the warmer's journal lines must name every pack; once the façade is up, `curl http://127.0.0.1:7772/v1/audio/voices` is the authoritative list (the façade has exactly three routes: `/health`, `/v1/audio/voices`, `/v1/audio/speech`).
-5. **Prove the cold-voice law is paid at boot, not by a listener** — two identical synthesis requests for one voice, back to back, from inside the engine's namespace; the **second** must answer well under a second (`warm=` in the journal is already that number; the manual pair is the confirmation on the real artifact).
-6. Then the façade on **7772** (its own page): nothing may be published there until the engine answers, because open-notebook's render fails the moment the façade is up and the engine is not.
-7. Nothing on 7772 exists yet — that is why open-notebook's podcast render cannot run. See [[Open Notebook]].
+4. **Confirm the voices are known** — the warmer's journal lines name every pack, and once the façade is up `curl http://127.0.0.1:7772/v1/audio/voices` is the authoritative list (the façade has exactly three routes: `/health`, `/v1/audio/voices`, `/v1/audio/speech`).
+5. **Measure the ladder instead of trusting the trial** — two identical requests for one voice back to back. The design expected 6.2-8.0 s then 0.5 ms; the deployed container gave **1,883 ms then 1,226-1,283 ms**, so the hook buys ~0.6 s and stays only for the registry reason (law 1). The retired numbers must not be quoted.
+6. **Then the façade on 7772** (its own page), after the engine answered — that order is the whole point of step 6: publishing the façade in front of a dead engine would make every consumer fail at the first voice line.
+7. **Open:** [[Open Notebook]] is still not installed, so nothing renders a podcast yet; the voice leg it waits on now exists.
 
 ## Troubleshooting
 

serverAppConfigs-docs/Dankclaw/Frank TTS.md

@@ -1,11 +1,13 @@
 ---
 type: reference
-description: "frank-tts is the fleet's cloned-voice speech endpoint on Dankclaw :7772 — the one voice surface consumers address, in front of frank-tts-engine; repo-owned, not yet deployed."
+description: "The fleet's cloned-voice speech endpoint and the one voice surface consumers address; wired as the harness speech role."
 tags:
+ - endpoint
  - dankclaw
  - tts
  - voice
  - quadlet
+timestamp: 2026-10-19
 ---
 
 # Frank TTS
@@ -16,20 +18,26 @@ audio in that cloned voice. Everything else in the voice stack sits behind it
 the engine ([[Frank TTS Engine]]) publishes no port on purpose, and consumers
 never learn a path.
 
-**Nothing here is deployed.** No host was touched while this was written: no
-image built, no container started, nothing listening on 7772. That is also why
-[[Open Notebook]] cannot render a podcast episode yet — its pipeline calls this
-endpoint.
+**Deployed 2026-10-18** — `./scripts/deploy.sh dankclaw --service frank-tts`,
+both units `active`, no drop-ins, and the homepage card's `wip` gate came off in
+the same change. Probes on the deployed endpoint: `/health` answers
+`{"status":"ok","backend":"omni-http","voices":["narrator1","narrator2","narrator3"]}`,
+and two different voices rendered the same sentence into **different** bytes
+(24 kHz mono PCM16, 4.24 s, healthy RMS) — so name resolution is real, not a
+pass-through of one default. The operator harness routes its `speech` role here;
+see [[#The harness speech role]]. What still waits is [[Open Notebook]]: its
+render needs the app installed, not this endpoint.
 
 | | |
 | --- | --- |
 | Host | Dankclaw (`192.168.0.143`), rootless podman, user unit `frank-tts.service` |
-| Port | **7772**, published (`PublishPort=7772:7772`); the family is 7770 dragoon, 7771 zealot, 7772 tts |
+| Port | **7772**, published (`PublishPort=7772:7772`) — the rest of the `frank-*` port family is in [[Dankclaw Stack]] |
 | Routes | `GET /health`, `GET /v1/audio/voices`, `POST /v1/audio/speech`; everything else 404, and speech is POST-only |
 | Image | `localhost/franksdank/frank-tts:latest`, built from `dankclaw/voiceservice/Containerfile`, `Pull=never` |
 | Networks | `dankclaw_default` first (it owns the default route and the consumers), then `frank-tts-engine.network` (10.89.5.0/24, the bridge to the engine) |
 | Voice packs | `/home/dank/voices` mounted read-only at `/voices` — host state, never repo state |
 | Upstream | `http://frank-tts-engine:8880` (`scripts/frank-tts.env`, `TTS_UPSTREAM`) |
+| Default voice | `DEFAULT_VOICE` (`scripts/frank-tts.env`) answers a request that names none; the journal marks that line `(default)`, and an **unknown** name is still a 400 |
 | Rollback | `TTS_BACKEND=llama-tts-cli` runs a CLI binary per request and needs no upstream at all |
 
 ## Naming a voice
@@ -37,7 +45,9 @@ endpoint.
 A voice **is** a directory name. `voice=narrator1` resolves to
 `/voices/narrator1/reference.wav`, and that file — not a manifest, not a config
 entry — is what makes the name real. omp's TTS device may send `voice_id`; the
-shim accepts either.
+shim accepts either. A call that sends **neither** — the harness's normal shape,
+because no harness setting carries a voice for this transport — speaks
+`$DEFAULT_VOICE`; empty default makes it a 400 again.
 
 The usable shape is `^[a-z0-9][a-z0-9._-]*$`: lowercase, starting with a letter
 or digit, then letters, digits, `.`, `_`, `-`, at most 64 characters. No slash,
@@ -109,7 +119,7 @@ shows up as a failed synthesis, never as a down probe.
 | `dankclaw/voiceservice/Containerfile` | python 3.12-alpine pinned by digest + ffmpeg, uid 10000 |
 | `dankclaw/voiceservice/quadlet/frank-tts.container` | the unit: two networks, `Restart=always`, `HealthCmd`, `[Install]` |
 | `dankclaw/voiceservice/quadlet/frank-tts-engine.network` | the shared bridge, named after the engine unit and attached by both |
-| `scripts/frank-tts.env` | backend, upstream, port, limits — paths and numbers, no credentials |
+| `scripts/frank-tts.env` | backend, upstream, port, limits, and the default voice — paths and numbers, no credentials |
 | `tests/frank_tts_unit.bats` | the shim against stub engines over real sockets |
 | `tests/frank_tts_engine.bats` | the pair's wiring: generator output, planner, homepage, deployer |
 
@@ -134,12 +144,55 @@ types: [[Prometheus]] records that the existing HTML probes (`comfyui`, the two
 `services` targets) sit **down** with
 `received unsupported Content-Type … and no fallback_scrape_protocol specified`.
 A `/health` probe job would therefore be structurally never-green, and because
-`LlmEndpointDown` is scoped `up{job=~"frank-.*"} == 0`, a permanently red
-`frank-tts` target would page on Signal every few minutes for an endpoint that
-is fine — the frank-archon lesson pointed the other way. The engine cannot be
+`LlmEndpointDown` is scoped across the whole `frank-*` job family (the rule set
+lives on [[Prometheus]]), a permanently red `frank-tts` target would page on
+Signal every few minutes for an endpoint that is fine. The engine cannot be
 scraped at all: it publishes nothing. Status lives in `systemctl --user status`,
-`podman healthcheck run`, the two unit journals, and the homepage card, which is
-gated `wip: down` until the deploy and probes `127.0.0.1:7772`.
+`podman healthcheck run`, the two unit journals, and the homepage card, which
+probes `127.0.0.1:7772` for real now — the `wip: down` gate came off with the
+2026-10-18 deploy.
+
+## The harness speech role
+
+The operator harness reaches this endpoint through
+`~/.omp/agent/models.yml` — provider `frank-tts (dankclaw)`,
+`baseUrl: http://192.168.0.143:7772/v1`, `auth: none`, and the runner api
+`openai-speech` on the **model** row (naming a runner api at *provider* level
+fails the whole file's validation and disables every custom provider, which is
+how the fleet's chat endpoints disappear) — plus `config.yml`
+`modelRoles.speech: frank-tts (dankclaw)/frank-tts`. Four laws, read out of the
+harness bundle and measured the day of the flip:
+
+- **The only voice the harness can send is the `voice_id` argument of the call.**
+  The transport builds `{model, input, response_format, voice?}` and adds `voice`
+  **only** when the caller passed `voice_id`; no setting feeds it.
+  `tts.localVoice` is the *local Kokoro backend's* voice (it is what answered
+  `voice=kokoro/bm_george, backend=local` in the control test), and
+  `speech.voice` is the TUI's **speak-aloud** voice — its enum is Kokoro's list,
+  and it never reaches an HTTP request. So a harness call arrives voice-less by
+  default, which is exactly why the endpoint grew `DEFAULT_VOICE`: the declared
+  pack answers a voice-less request and the journal marks that line `(default)`,
+  while an **unknown** name still 400s with the list that exists.
+- **`speech.voice` must stay a Kokoro name.** Setting it to a pack name does not
+  select a pack; it hands the vocalizer a value outside its enum. (The first
+  draft of this section said the opposite and was reverted the same day, after a
+  fresh session's call came back `unknown voice None`.)
+- **There is no silent fallback.** An explicitly configured `modelRoles.speech`
+  yields exactly one candidate, so a dead 7772 fails the speech device with
+  *"No available speech model matches the speech role"* instead of quietly
+  degrading to Kokoro. That is the price of `auth: none` (which makes a dead
+  endpoint read as available) and it is the wanted shape: a narration job should
+  fail loudly rather than ship a voice Frank never asked for.
+- **A session older than the models.yml row cannot see it.** `models.yml` loads
+  at process start while `config.yml` is read live, so a long-lived session
+  carrying the flipped role fails exactly like a dead endpoint while a fresh
+  process succeeds in the same minute. Measured 2026-10-18: the open session
+  failed twice (provider/id form, then the bare id), `local/kokoro` succeeded in
+  that same session, and a fresh `omp -p --tools tts` returned `Saved 180524
+  bytes (model=frank-tts (dankclaw)/frank-tts, voice=narrator1, codec=wav)`
+  while the façade logged the matching `speech voice=narrator1 format=wav
+  chars=51 -> 180524 bytes` from the podman bridge. Rule the stale registry out
+  before blaming the service.
 
 ## Troubleshooting
 
@@ -150,3 +203,5 @@ gated `wip: down` until the deploy and probes `127.0.0.1:7772`.
 | Endpoint answers, synthesis fails | the engine: `podman healthcheck run frank-tts-engine`, then `systemctl --user status frank-tts-engine.service` |
 | First request of the day is ~0.6 s slower than the rest — or every name returns 400 | the warm pass did not run; it carries `-`, so it cannot fail the unit. `journalctl --user -u frank-tts-engine` for the warmer's lines, then `podman exec frank-tts-engine /app/warm-voices.sh` — the registry is memory-only (law 1 of [[Frank TTS Engine]]) |
 | The unit never appears after a deploy | the generator dropped it: `frank-tts-engine.network` is missing from `~/.config/containers/systemd/` |
+| A consumer rejects `narrator1` by listing 29 names (`af_heart … bm_fable`) and no request ever reaches this endpoint | that consumer is synthesizing **in the browser** with Kokoro.js, whose voice pack is fixed — OpenWebUI 0.11.4 ships it as the `browser-kokoro` TTS engine; see [[OpenWebUI]] |
+---

serverAppConfigs-docs/Dankclaw/Gitea.md

@@ -1,5 +1,5 @@
 ---
-type: configuration
+type: reference
 title: Gitea
 description: Gitea deployment on Dankclaw with its web UI reverse-proxied by the canonical Caddy service.
 tags:

serverAppConfigs-docs/Dankclaw/Hermes-WebUI.md

@@ -1,7 +1,7 @@
 ---
-type: configuration
+type: reference
 title: Hermes-WebUI (Hermex server)
-description: The hermes-webui container on Dankclaw — the API server that the Hermex iPhone app talks to, reachable over an SSH tunnel, from the whole LAN (it binds all interfaces), and from https://hermes.franksdank.cloud (internal-only vhost).
+description: "The iPhone companion app for Hermes on Dankclaw and the ways it is reached from the LAN."
 tags:
   - hermes
   - hermes-webui

serverAppConfigs-docs/Dankclaw/Hermes.md

@@ -78,9 +78,9 @@ is not durable. The tracked, sanitized configuration baseline is
   `/data/config.yaml` is not readable without `sudo`):
   `dankclaw-large` reaches `http://192.168.0.143:7770/v1` and exposes
   `frank-dragoon` — verified from the host with `curl
-  http://127.0.0.1:7770/v1/models` → single id `frank-dragoon`, `max_model_len
-  200704` (2026-10-18; write `127.0.0.1`, not `localhost`, because that port is
-  a rootless pasta publish and is IPv4-only);
+  http://127.0.0.1:7770/v1/models` → a single served id (2026-10-18; write
+  `127.0.0.1`, not `localhost`, because that port is a rootless pasta publish
+  and is IPv4-only);
   `dankserver-small` reaches `http://192.168.0.235:7770/v1` and, **as of
   2026-10-18, that endpoint serves only `frank-archon`** (`GET
   http://192.168.0.235:7770/v1/models` → `[{"id":"frank-archon"}]`). The name
@@ -88,7 +88,8 @@ is not durable. The tracked, sanitized configuration baseline is
   2026-09-15 — is retired-era: nothing on that endpoint answers to it today, so
   if the live `config.yaml` still names `frank-medium` the provider is dead
   weight and requests will 404 on a nonexistent model. Diff the live config
-  before trusting either name.
+  before trusting either name. Which payload sits behind each port, and the
+  served id it answers to, is on [[frank-dragoon]] and [[frank-archon]].
 - The live `config.yaml` contains secrets and user state and is not committed;
   the tracked example is the sanitized backup of its non-secret settings.
 
@@ -227,9 +228,10 @@ resp = call_llm(task="title_generation", messages=[...], max_tokens=64,
 
 What actually breaks, and the fix:
 
-- **`response_format` (json_object / json_schema) is a hard 400 on every
-  local backend here** — neither FreeToken nor halogen implements constrained
-  decoding. Config cannot override it: `call_llm` merges the task's
+- **`response_format` (json_object / json_schema) is a hard 400 on the local
+  backends this task was written against** — what each endpoint accepts on the
+  wire is on [[frank-dragoon]] / [[frank-zealot]] / [[frank-archon]]. Config
+  cannot override it: `call_llm` merges the task's
   `extra_body` first and then `update()`s the caller's, so the hardcoded one
   wins. Only temperature and max_tokens have reactive retry paths.
 - **Fix: move that one task to the Responses transport**, which never
@@ -246,9 +248,8 @@ What actually breaks, and the fix:
           effort: none                 # mandatory, see below
   ```
 
-  Before copying this: as of 2026-10-18 `frank-zealot` is served on **Dankclaw
-  `:7771`**, and `http://192.168.0.235:7770/v1` — the endpoint this provider
-  name is recorded against — answers only `frank-archon`. The provider/model
+  Before copying this: which id is served on which port is endpoint state — see
+  [[frank-zealot]] and [[frank-archon]]. The provider/model
   pairing above is the shape that was landed in 2026; re-derive it from the live
   config before reusing it.
 

serverAppConfigs-docs/Dankclaw/Marreta.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Marreta on Dankclaw
-description: The content-cleaning proxy (manualdousuario/marreta, Laravel/serversideup fpm-nginx) deployed as a rootless Quadlet pair that shares SearXNG's Gluetun netns for egress and sits behind the Anubis + Authentik guard chain at marreta.franksdank.cloud.
+description: "The content-cleaning proxy SearXNG borrows egress from: a quadlet pair in the VPN netns behind the guard chain."
 tags:
   - dankclaw
   - marreta

serverAppConfigs-docs/Dankclaw/Open Notebook.md

@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "open-notebook is the fleet's podcast generator on Dankclaw: notes and sources in, a multi-speaker episode out — script written by frank-zealot, voices rendered by frank-tts, no cloud provider anywhere in the path."
+description: "The fleet's podcast generator: notes in, multi-speaker episode out, script and voices both served in-house."
 tags:
   - dankclaw
   - open-notebook
@@ -13,7 +13,7 @@ timestamp: 2026-10-18
 
 open-notebook ([lfnovo/open-notebook](https://github.com/lfnovo/open-notebook)) is the notebook layer that turns existing notes and sources into a **spoken episode**: an outline, then a dialogue transcript, then one synthesized clip per speaker turn, concatenated with ffmpeg into an MP3. In the fleet it is the consumer that closes the loop Frank's audio stack was built for — the script is written by **frank-zealot**, the voices are rendered by **frank-tts**, and nothing leaves the LAN.
 
-**It is NOT deployed.** This page documents repo-owned configuration — the Quadlet units, the env file, the deployer path, the homepage card — and the contract those files were written against. On 2026-10-18 no `open-notebook` or `open-notebook-surrealdb` container exists on Dankclaw (`podman ps -a` census, 2026-10-18), and the homepage card ships `wip: "down"` with `showLink: false`, so the card is hidden and reported `wip-down`. Deploying it is a deliberate next step, not a pending one.
+**It is NOT deployed.** This page documents repo-owned configuration — the Quadlet units, the env file, the deployer path, the homepage card — and the contract those files were written against. On 2026-10-18 no `open-notebook` or `open-notebook-surrealdb` container exists on Dankclaw (`podman ps -a` census, 2026-10-18), and the homepage card ships `wip: "down"` with `showLink: false`, so the card is hidden and reported `wip-down`. Re-censused 2026-10-19 and unchanged: no `*notebook*` unit in `systemctl --user list-unit-files`, nothing installed under `~/.config/containers/systemd/` by that name, no container in `podman ps` — the repo's three quadlet files have never been installed on the host. Deploying it is a deliberate next step, not a pending one.
 
 **Host:** Dankclaw (rootless Podman, user `dank`) — **Ports:** 8502 UI and 5056 API, published on the LAN, **no vhost and no Caddy route** — **Deployer:** `open-notebook` (one service, two units) — **Docs:** https://github.com/lfnovo/open-notebook
 
@@ -59,9 +59,9 @@ podman exec open-notebook curl -sS http://frank-tts:7772/health
 podman exec open-notebook curl -sS http://host.containers.internal:7771/v1/models
 ```
 
-**The blocker to state plainly:** frank-tts is built, named and tested but **not deployed** — nothing answers on 7772 (`ss -ltn`, Dankclaw, 2026-10-18; [[Local TTS and Voice Cloning]] records the same census). open-notebook deployed today writes scripts and fails at the first voice line. frank-tts ships first, or podcast generation waits.
+**The blocker moved; it did not vanish.** frank-tts and frank-tts-engine are **deployed** (2026-10-18) and answer on 7772, so the voice leg works end to end — `podman exec open-notebook curl -sS http://frank-tts:7772/health` is the check to run the day the app exists. What still blocks a render is open-notebook itself: not installed on this host yet. When it is, the two provider rows above are the only wiring, and the seeded speaker profiles carrying OpenAI voice names are the first trap.
 
-**Why frank-zealot and not frank-dragoon:** podcast scriptwriting asks the model for a JSON-schema response (with a plain-JSON fallback). llama.cpp's OpenAI-compatible endpoint is a known-good target for that; frank-dragoon's halogen runtime has **unverified** structured-output behaviour, so it is the unproven alternative, not the fallback. `ESPERANTO_LLM_TIMEOUT=180` (the app's own default) covers an outline or transcript call on zealot comfortably; upstream's note is to keep it under 600 so a slow call fails with a clear error instead of a UI timeout.
+**Why frank-zealot and not frank-dragoon:** podcast scriptwriting asks the model for a JSON-schema response (with a plain-JSON fallback). The decision rests on each endpoint's documented request surface ([[frank-zealot]], [[frank-dragoon]]): zealot's is a known-good target for that call shape, dragoon's structured-output behaviour is **unverified**, so it is the unproven alternative, not the fallback. `ESPERANTO_LLM_TIMEOUT=180` (the app's own default) covers an outline or transcript call on zealot comfortably; upstream's note is to keep it under 600 so a slow call fails with a clear error instead of a UI timeout.
 
 ## Speakers and voices: 1–4, and the IDs are free text
 

serverAppConfigs-docs/Dankclaw/OpenWebUI.md

@@ -39,28 +39,26 @@ As of 2026-10-07 the picker exposes **exactly one model: `frank-zealot`** *(unve
 `frank-dragoon`/`frank-zealot`/`frank-archon`, per the cards in
 `shared/homepage/services.json`)*.
 
-| Connection (`openai.api_configs` idx) | URL as recorded 2026-10-09 | Enabled | `model_ids` | Runtime | Host |
-|---|---|---|---|---|---|
-| 0 | `http://host.containers.internal:7770/v1` | no (2026-10-07) | `frank-dragoon` | halogen flash-server (Strix Halo iGPU) | Dankclaw |
-| 1 | `http://192.168.0.235:7770/v1` | no (2026-10-07) | `frank-archon` | radiance TP=2 (2× R9700) | Dankserver |
-| 2 | `http://host.containers.internal:7771/v1` | yes | `frank-zealot` | llama.cpp CUDA (RTX 3090; production shape 3×262,144 per `scripts/frank-zealot.env`) | Dankclaw |
+| Connection (`openai.api_configs` idx) | URL as recorded 2026-10-09 | Enabled | `model_ids` | Endpoint page |
+|---|---|---|---|---|
+| 0 | `http://host.containers.internal:7770/v1` | no (2026-10-07) | `frank-dragoon` | [[frank-dragoon]] |
+| 1 | `http://192.168.0.235:7770/v1` | no (2026-10-07) | `frank-archon` | [[frank-archon]] |
+| 2 | `http://host.containers.internal:7771/v1` | yes | `frank-zealot` | [[frank-zealot]] |
 
 The URLs in the first column are the DB's recorded values and predate the 2026-08-22
 bridge move; on this network they are wrong — see the reachability gotcha below, which
-is the corrected law. Note also that the 2026-10-18 **bench window** on `:7771`
-closed the same day (production unit stopped 03:38 per the repo Benchmarking
-Policy while candidate arms held the port): the candidate — Ternary-Bonsai-2-**9B**
-PQ2_0 with the MTP head, 3 × 262,144 — was **deployed as production**, so the
-picker row below was re-keyed to it (see the next paragraph).
+is the corrected law. The picker row for `frank-zealot` was re-keyed on 2026-10-18 after
+that day's payload swap (see [[frank-zealot]] for what was deployed and when).
 
 The `model` table carries exactly two decoration rows, `frank-zealot` and
 `frank-dragoon`. Both the context length and the description live **inside
 JSON columns** — `params` holds `{"context": {"length": …}}` and `meta` holds
 `{"description": …}`; there is no `context_length` or `description` column, so
-`sqlite3` one-liners against those names fail. Re-keyed 2026-10-18:
-`frank-zealot` → 262,144 with the Bonsai-2-9B description, and `frank-dragoon`
-→ 200,704 (its `params` had kept the pre-lowering 262,144 window from
-2026-10-17 while its own description already said 200,704). These rows are
+`sqlite3` one-liners against those names fail. `params.context.length` is a copy
+of the endpoint's per-slot window, so the authoritative number lives on the
+endpoint page ([[frank-zealot]], [[frank-dragoon]]) and the row must follow it —
+the 2026-10-18 re-key existed because `frank-dragoon`'s row still carried the
+window its payload had before that day's swap. These rows are
 live DB state, so a payload swap must re-key the database too, not only the
 repo. There is no `sqlite3` binary on Dankclaw — use `python3 -c` with the
 stdlib `sqlite3` module, and take a `Connection.backup()` snapshot first (the
@@ -104,10 +102,10 @@ Two gotchas make the wiring non-obvious:
 The Ollama connection is disabled (`ollama.enable=false`).
 
 > All three endpoints are bare-metal systemd services managed by this
-> repository. Model swaps, context sizing, and spec tuning happen there. See
-> the [LLM knowledge base](../LLM/index.md). Disabling a connection here only
-> hides it from this picker — the endpoint itself keeps serving every other
-> client (the harness, Hermes, API consumers).
+> repository; their shape, swaps and tuning live on
+> [[frank-dragoon]] / [[frank-zealot]] / [[frank-archon]]. Disabling a
+> connection here only hides it from this picker — the endpoint itself keeps
+> serving every other client (the harness, Hermes, API consumers).
 
 ## Container introspection (DB and backend python)
 
@@ -143,9 +141,9 @@ The Ollama connection is disabled (`ollama.enable=false`).
    models. Response key is `items`, not `data`.
 2. `POST /api/chat/completions` per model with a trivial prompt ("Reply
    with exactly one word: pong") → each returns content `pong`. Reasoning
-   models need a generous budget: frank-zealot (Qwen3.5 template) returns
-   empty content + `finish_reason: "length"` at `max_tokens 200` (the
-   thinking block eats it) and answers `pong` at 1500.
+   endpoints need a generous budget: thinking consumes `max_tokens`, so a short
+   one returns empty content + `finish_reason: "length"` (see
+   [[frank-zealot]]); `frank-zealot` answers `pong` at 1500.
 3. A 500 "Server Connection Error" → `podman logs open-webui` for
    `ClientConnectorError`: a network-reachability problem (host LAN IP vs
    `host.containers.internal`), not a model problem.
@@ -292,6 +290,42 @@ carries base64 image data or a URL.
 Failure modes: connection refused ⇒ OpenWebUI or ComfyUI is down; timeouts
 on large sizes ⇒ check ComfyUI resources; 401 ⇒ the token is bad.
 
+## Text-to-speech (frank-tts)
+
+TTS is the fleet's one OpenAI-compatible speech endpoint —
+`http://host.containers.internal:7772/v1` (`audio.tts.openai.api_base_url`
+in the config DB, 2026-10-18), model `frank-tts`, voice `narrator1`. That
+port is the rootless quadlet `frank-tts` (this repo:
+`dankclaw/voiceservice/quadlet/frank-tts.container`,
+`scripts/frank-tts.env`), and its shim resolves `voice=<name>` against
+`/home/dank/voices/<name>/reference.wav` — a name without a reference is a
+400 listing the voices that exist, never a silent fallback.
+
+**The 29-voice `af_* … bm_fable` list is NOT frank-tts.** That enum belongs
+to **Kokoro.js running in the browser**: the deployed front end (0.11.4)
+ships an in-page Kokoro implementation (chunk carrying a frozen voice table
+`{af_heart:{name:"Heart",…}}` and `_validate_voice`, model
+`onnx-community/Kokoro-82M-v1.0-ONNX`, plus a personal *Kokoro.js dtype*
+setting). Its engine select takes `""` (Default) or `browser-kokoro`; when
+`browser-kokoro` is chosen the page synthesizes locally and rejects any
+voice outside its fixed pack — so `narrator1` fails with
+`Voice: "narrator1" not found. Should be one of: af_heart, …` **without a
+single request reaching the server**. Personal audio settings are per
+browser: no `user.settings` row and no container env carries an audio
+override, yet the toast still appears ⇒ the choice lives in that browser's
+own storage. Fix in the UI: Settings → Audio → TTS Engine → **Default**,
+which hands synthesis to the admin config (frank-tts); picking `af_heart`
+instead keeps Kokoro, and Kokoro cannot clone — fixed pack only.
+
+Verified 2026-10-18 from inside the container with the exact OpenWebUI body
+(`{"model":"frank-tts","input":…,"voice":"narrator1",
+"response_format":"mp3","speed":1.0}`) → **200, `audio/mpeg`, 7,629 B**,
+while `podman logs open-webui` recorded **zero** `audio/speech` requests for
+the failing click — that contrast is the instrument: a server-path failure
+leaves a log line and a frank-tts error, a `browser-kokoro` failure leaves
+neither.
+
+
 ## Retired instances
 
 - The compose `open-webui` service (port 3000) was never run: 3000 is Gitea's HTTP port, so it could not bind.
@@ -302,8 +336,8 @@ on large sizes ⇒ check ComfyUI resources; 401 ⇒ the token is bad.
 
 - The image tracks upstream `main`; pin a digest in the quadlet unit if reproducible rollback matters more than tracking. (The old "CPU-only host" framing is stale: this APU runs the gfx1151 ROCm build — see [[ComfyUI]].)
 - The default admin account is created on first login; secure it immediately.
-- If the backend is loading (3–8 minute model load), OpenWebUI shows connection errors until the model is ready.
-- The on-host LLM services (Dankclaw `:7770`, `:7771`) are reached via the **host LAN IP `192.168.0.143`**, which is exactly what the deployed unit does (`open-webui.container:32`). The older "use `host.containers.internal`" advice is retired for this container: it is a slirp4netns/pasta-only name and the unit has been on the pinned bridge `systemd-open-webui` since 2026-08-22, where that name does not resolve (`open-webui.network:15-19`). Both endpoints are published on all interfaces today (`ss -ltn` → `*:7770`, `0.0.0.0:7771`; `curl http://192.168.0.143:7771/health` → 200, 2026-10-18). The Dankserver backend is a different host, reached via its LAN IP `192.168.0.235:7770`.
+- If the backend is loading, OpenWebUI shows connection errors until the endpoint is ready; each endpoint's boot budget is on its own page ([[frank-dragoon]], [[frank-zealot]], [[frank-archon]]).
+- Repeated from *The LLM backends*: the on-host endpoints are reached via the **host LAN IP**, never `host.containers.internal`.
 
 ## History
 
@@ -349,6 +383,7 @@ on large sizes ⇒ check ComfyUI resources; 401 ⇒ the token is bad.
 ## See Also
 
 - [[Dankclaw Stack]]
+- [[Frank TTS]] — the speech endpoint behind *Text-to-speech (frank-tts)*
 - [[Hermes]] — the agent OpenWebUI can talk to
 - [[SearXNG]] — the search engine on this host
-- [LLM knowledge base](../LLM/index.md) — the vLLM and llama.cpp endpoints
+- [[frank-dragoon]] / [[frank-zealot]] / [[frank-archon]] — the endpoints behind the picker

serverAppConfigs-docs/Dankclaw/Seerr.md

@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "Seerr is the unified request manager for the *arr stack (the maintained `seerr-team/seerr` successor to the archived Overseerr/Jellyseerr line)."
+description: "The request manager for the *arr stack — the maintained successor to the archived Overseerr line."
 tags:
   - dankclaw
   - seerr
@@ -40,8 +40,8 @@ the next outpost reload), and the missing app bounced post-login to
 authentik's own user UI instead of Seerr. Both backfilled live: the three
 standard flows copied from a known-good provider, plus an application
 (`seerr-guard`) linked to the provider. Verified by the hardened SSO sweep
-(`tests/e2e/specs/13-sso.spec.ts`) — see [[Authentik]] for the two
-provisioning rules this incident established.
+(`tests/e2e/specs/13-sso.spec.ts`) — see [[Authentik Outpost Reference]] for the
+two provisioning rules this incident established.
 
 ## *arr wiring
 

serverAppConfigs-docs/Dankclaw/Swap Configuration.md

@@ -1,7 +1,7 @@
 ---
-type: configuration
+type: reference
 title: Dankclaw Swap Configuration
-description: The host swapfile on Dankclaw — 64 GiB at /swap.img on ext4, fstab-driven via the generated swap.img.swap unit. Covers why it was raised from 8 GiB, the safe resize ordering that avoids an OOM on a host whose MemAvailable is inflated by pinned model weights, and the swappiness caveat.
+description: "The host swapfile: why it is sized as it is, the resize ordering that avoids an OOM, and the swappiness caveat."
 tags:
   - dankclaw
   - host
@@ -36,14 +36,14 @@ it is not invisible.
 ## Why 64 GiB
 
 Dankclaw is shared. The 2026-09-12 decision measured `frank-dragoon` alone holding
-**103.5 GiB of the 121.2 GiB** host (`68.0` weights + `14.4` KV pool + `21.1`
-working), leaving roughly **7.1 GiB** for the then-28 co-tenant containers whose
-working set was ~4.3 GiB. That is a decision-time reading, not today's: on
+**103.5 GiB of the 121.2 GiB** host, leaving roughly **7.1 GiB** for the then-28
+co-tenant containers whose working set was ~4.3 GiB. How that total is split
+between weights, KV pool and working memory is the engine's own ledger — it lives
+on [[frank-dragoon]]. That is a decision-time reading, not today's: on
 2026-10-18 the host reports **91 GiB used / 29 GiB available of 121 GiB**
 (`free -h`), 30 containers are up (`podman ps -q | wc -l`), and the
 `frank-dragoon-halogen` container accounts for **70.33 GiB**
-(`podman stats --no-stream`, same day) on the current flash-server payload —
-a different model and KV shape than the one measured in September.
+(`podman stats --no-stream`, same day).
 The engine itself warned at the time that when the headroom runs out,
 frank-dragoon and its neighbours "can stall for minutes at a time at full CPU with
 no disk activity and no output". Swap is the valve that turns an OOM kill into slow
@@ -59,7 +59,7 @@ like this, **never `swapoff` before the replacement is online.** The old 8 GiB
 held 4 GiB of live pages, and `MemAvailable` read 73 GiB while only ~5 GiB was
 genuinely free — the pinned model weights are counted as reclaimable page
 cache, so the headline number is a lie by roughly the size of the model
-(see [[halogen-flash-server]]). Swapping off into that apparent space, with
+(see [[frank-dragoon]]). Swapping off into that apparent space, with
 nothing else to receive the pages, is how you invite the OOM killer.
 
 The ordering that avoids it — create and enable the new swap **first**, so the
@@ -113,10 +113,10 @@ frank-dragoon `health=200` throughout.
 
 ## Caveats
 
-- **Swap cannot help the pinned weights.** frank-dragoon's ~68 GiB of weights are
-  held deliberately; paging them is a stall, not a relief. More swap does not
-  make the doubled KV pool affordable — see the reserve/pool ceiling in
-  [[halogen-flash-server]].
+- **Swap cannot help the pinned weights.** frank-dragoon's weights are pinned
+  deliberately; paging them is a stall, not a relief. More swap does not make a
+  larger KV pool affordable — see the reserve/pool ceiling in
+  [[frank-dragoon]].
 - **`vm.swappiness=60` is untested against this workload.** Under real
   pressure the kernel will happily page anonymous memory, and frank-dragoon's KV
   pool and working set *are* anonymous. Swapping the KV pool would be far worse
@@ -151,6 +151,6 @@ The old swapfile has therefore been purged; nothing on the host still holds it.
 
 ## See Also
 
-- [[halogen-flash-server]] — the memory model that motivated this
+- [[frank-dragoon]] — the memory model that motivated this
 - [[Dankclaw System Specs]] — the Strix Halo box
 - [[Dankclaw Improvements]] — other host-level changes

serverAppConfigs-docs/Dankclaw/Voice Packs.md

@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "How a frank-tts voice pack is built on Dankclaw: the three files every pack must carry, the 10-second source-material gate scripts/build-voice-pack.sh enforces, how to add a fourth voice, and what to do when a pinned engine bump changes the extractor's output sizes."
+description: "How a voice pack is built: the files every pack must carry, the source-material gate, adding a voice, engine-bump drift."
 tags:
   - dankclaw
   - frank-tts
@@ -233,12 +233,12 @@ loud, not a bug. Work through it in order:
   they have to be copied there once before a build.
 - `VOICE_PACK_TOOL` points at the trial's build tree
   (`/home/dank/voice-audition/qwentts/qwentts.cpp/build/qwen-codec`, checked out
-  at exactly `5916bf7c`), and the two GGUFs at
-  `/home/dank/models/Qwen3-TTS-GGUF/`, which is where the engine slice's
-  download script puts them — that directory does not exist yet, so until it
-  does the models have to come from the trial's `models/` directory.
-- Nothing is deployed: nothing listens on 7772 or 8880 today, so building packs
-  is possible on that host while serving them is not.
+  at exactly `5916bf7c`), and the two GGUFs live at
+  `/home/dank/models/Qwen3-TTS-GGUF/`, the engine slice's fetch target — that
+  path and its fetch state belong to [[Frank TTS Engine]].
+- Whether the façade (7772) and the engine (8880) are up and serving is deploy
+  state on [[Frank TTS]] / [[Frank TTS Engine]] — read it there, not from this
+  census.
 
 ## Known gaps, on purpose
 

serverAppConfigs-docs/Dankserver/Copyparty.md

@@ -143,13 +143,13 @@ Copyparty v1.20.9+ supports WOPI (Web Application Open Platform Interface) for e
 
 ## Security Advisories
 
-| Version | CVE | Issue |
-|---------|-----|-------|
-| v1.20.19 | GHSA-phv8-wgjp-g4p9 | FTP upload outside volume |
-| v1.20.17 | GHSA-x5pq-m9p8-f4vx | dirkey/filekey confusion |
-| v1.20.9 | CVE-2026-27948 / GHSA-62cr-6wp5-q43h | XSS via file upload |
+| Version | Advisory | CVE | Issue |
+|---------|----------|-----|-------|
+| v1.20.19 | GHSA-phv8-wgjp-g4p9 | CVE-2026-104102 | FTP upload outside the defined volumes (limited path traversal) |
+| v1.20.17 | GHSA-x5pq-m9p8-f4vx | CVE-2026-70657 | dirkey/filekey confusion |
+| v1.20.9 | GHSA-62cr-6wp5-q43h | CVE-2026-27948 | reflected XSS via the `?setck=` URL parameter (not the upload path — the separate upload/DOM-XSS advisory is GHSA-m2jw-cj8v-937r) |
 
-Upgrade to v1.20.19+ to cover all known CVEs. The dirkey fix (v1.20.17) matters if using both `dirkeys` and `filekeys`.
+Upgrade to v1.20.19+ to cover all known advisories. The dirkey fix (v1.20.17) matters if using both `dirkeys` and `filekeys`. Each row's patched version is the advisory's own `first_patched_version`.
 
 ## Gotchas
 
@@ -174,3 +174,10 @@ Upgrade to v1.20.19+ to cover all known CVEs. The dirkey fix (v1.20.17) matters
 - [[Dankserver Stack]]
 - [[Caddy]] — TLS termination and authentication in front
 - [[ARR-Stack]] — Copyparty as a viewer alongside automated download management
+
+## Citations
+
+- GitHub Security Advisories, `9001/copyparty` —
+  https://github.com/9001/copyparty/security/advisories (accessed 2026-10-19);
+  each row's patched version read from `first_patched_version` in the
+  `https://api.github.com/advisories/<GHSA-id>` response.

serverAppConfigs-docs/Dankserver/Dankserver Stack.md

@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "Dankserver (192.168.0.235) runs the media and library stack."
+description: "Service inventory and routing overview for the media and library host."
 tags:
   - dankserver
 timestamp: 2026-09-03

serverAppConfigs-docs/Dankserver/Jellyfin.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Jellyfin
-description: Jellyfin media server on Dankserver — quadlet unit, entropy-volume data layout, ports, client access, and hardware-transcode status.
+description: "Jellyfin on Dankserver: quadlet unit, entropy-volume data layout, ports, client access, hardware-transcode status."
 tags: [jellyfin, dankserver, media-server]
 timestamp: 2026-09-03T00:00:00Z
 ---

serverAppConfigs-docs/Dankserver/Plex.md

@@ -30,7 +30,7 @@ public traffic can be fronted by Caddy like every other vhost.
 | Identity | fleet rootless model — container root maps to the `dank` user (1000:1000); the deployer chowns the data root on first cutover |
 | Port | `32400/tcp` published on `0.0.0.0` — zero client reconfiguration |
 | Network | `dankserver_appnet`, alias `plex`; Tautulli reaches it at `http://plex:32400` |
-| GPU | none by decision — CPU decode only; the two R9700s stay dedicated to `frank-medium` |
+| GPU | none by decision — CPU decode only; the two R9700s stay dedicated to [[frank-archon]] |
 | Deploy | `./scripts/deploy.sh dankserver --service plex` |
 | Docs | https://support.plex.tv |
 

serverAppConfigs-docs/Dankserver/RomM-Scraping.md

@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "This guide covers metadata providers, cover art, ROM naming conventions, emulator integration, and the scraping workflow for RomM on Dankserver."
+description: "Metadata providers, cover art, ROM naming and the scraping workflow behind the RomM library."
 tags:
   - dankserver
   - romm

serverAppConfigs-docs/Dankserver/mariadb.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: MariaDB (RomM Backend)
-description: MariaDB 11 database backend for RomM on Dankserver — quadlet unit, network alias, env-file secret handling, and backup path.
+description: "The MariaDB backend for RomM: quadlet unit, network alias, env-file secret handling, and the backup path."
 tags: [mariadb, romm, dankserver]
 timestamp: 2026-09-03T00:00:00Z
 ---

serverAppConfigs-docs/Dankserver/redis.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Redis (RomM Cache / Queue)
-description: Redis 7-alpine cache and task-queue backend for RomM on Dankserver — quadlet unit, network alias topology, and backup path.
+description: "The Redis cache and task queue for RomM: quadlet unit, network alias topology, and the backup path."
 tags: [redis, romm, dankserver]
 timestamp: 2026-09-03T00:00:00Z
 ---

serverAppConfigs-docs/Desktop/Noctalia and Niri.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Noctalia and Niri
-description: The noctalia v5 desktop shell on niri (workstation + blankpad) — config layers, remote IPC over ssh, custom-palette recolor, offline template rendering, and output rotation/pinning.
+description: "The noctalia shell on niri across workstations: config layers, remote IPC, custom palettes, output rotation and pinning."
 tags: [desktop, noctalia, niri, blankpad, workstation, ghostty]
 timestamp: 2026-10-05T00:00:00-07:00
 ---

serverAppConfigs-docs/FrankBook-Pro/ARR-Stack.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: "*arr Stack"
-description: "The *arr automation stack (Prowlarr, Sonarr, Radarr, Lidarr, Readarr, qBittorrent) — FrankBook-Pro only, every container on the gluetun WireGuard netns (all egress VPN): topology, wiring, auth posture, and gotchas."
+description: "The *arr automation stack on FrankBook-Pro, every container on the VPN netns: topology, wiring, and gotchas."
 tags: [arr-stack, prowlarr, sonarr, radarr, lidarr, readarr, qbittorrent, frankbook-pro, dankserver]
 timestamp: 2026-09-04T00:00:00Z
 ---

serverAppConfigs-docs/FrankBook-Pro/index.md

@@ -7,10 +7,10 @@ Gluetun WireGuard netns. The host overview lives at [[FrankBook-Pro]]
 
 ## *arr Stack
 
-- [[ARR-Stack]] — the full *arr stack: indexers, managers, and the download client
-- [[sonarr]] — TV show management
-- [[radarr]] — movie management
-- [[lidarr]] — music management
-- [[readarr]] — ebook/audiobook management
-- [[prowlarr]] — indexer management
-- [[qbittorrent]] — torrent download client
+- [[ARR-Stack]] — The *arr automation stack on FrankBook-Pro, every container on the VPN netns: topology, wiring, and gotchas.
+- [[sonarr]] — Sonarr TV manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
+- [[radarr]] — Radarr movie manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
+- [[lidarr]] — Lidarr music manager in the *arr stack on FrankBook-Pro — live quadlet deployment, ports, volumes, and wiring facts.
+- [[readarr]] — The ebook and audiobook manager in the *arr stack: retired upstream, pinned image, parked download client.
+- [[prowlarr]] — Indexer management for the *arr stack: feed routes, the sync model, the API surface, and verified live gotchas.
+- [[qbittorrent]] — The download client for the *arr stack: pinned image, shared WireGuard netns, WebUI credential recipe.

serverAppConfigs-docs/FrankBook-Pro/prowlarr.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Prowlarr
-description: Prowlarr indexer-management layer for the *arr stack on FrankBook-Pro — feed routes, sync model, API surface, and verified live gotchas.
+description: "Indexer management for the *arr stack: feed routes, the sync model, the API surface, and verified live gotchas."
 tags: [prowlarr, arr-stack, frankbook-pro]
 timestamp: 2026-09-03T00:00:00Z
 ---

serverAppConfigs-docs/FrankBook-Pro/qbittorrent.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: qBittorrent
-description: qBittorrent download client for the *arr stack — pinned 4.6.7 on FrankBook-Pro, shares the Gluetun WireGuard netns, PBKDF2 WebUI credential recipe.
+description: "The download client for the *arr stack: pinned image, shared WireGuard netns, WebUI credential recipe."
 tags: [qbittorrent, arr-stack, frankbook-pro, gluetun]
 timestamp: 2026-09-03T00:00:00Z
 ---

serverAppConfigs-docs/FrankBook-Pro/readarr.md

@@ -1,18 +1,19 @@
 ---
 type: reference
 title: Readarr
-description: Readarr ebook/audiobook manager in the *arr stack on FrankBook-Pro — retired upstream, pinned image, parked qB download client.
+description: "The ebook and audiobook manager in the *arr stack: retired upstream, pinned image, parked download client."
 tags: [readarr, arr-stack, frankbook-pro]
 timestamp: 2026-09-03T00:00:00Z
 ---
 
 # Readarr
 
-> **RETIRED UPSTREAM (May 2026).** The Servarr Team retired Readarr: its
-> metadata became unusable and the community migration to Open Library stalled.
-> There is no upstream development. Alternatives: the Bookshelf community fork,
-> or the Calibre content server. This instance still runs on a pinned image
-> tag; see below.
+> **RETIRED UPSTREAM (announced 2025-06-27).** The Servarr Team retired
+> Readarr: its metadata became unusable and the community migration to Open
+> Library stalled. There is no upstream development — the GitHub repository is
+> archived and its last push is 2025-06-27. Alternatives: the Bookshelf
+> community fork, or the Calibre content server. This instance still runs on a
+> pinned image tag; see below.
 
 Readarr manages ebooks and audiobooks in the *arr stack. The stack runs on
 **FrankBook-Pro** (the Mac mini) as quadlet units inside its podman machine —
@@ -69,9 +70,9 @@ Restart=always
 
 ## Gotchas
 
-- Readarr is abandoned upstream (retired May 2026; last line was 0.4.x before
-  the final tags). The pinned `amd64-1.0.980` tag exists precisely because
-  `:latest` does not exist for this image on LSIO.
+- Readarr is abandoned upstream (retired 2025-06-27; last line was 0.4.x
+  before the final tags). The pinned `amd64-1.0.980` tag exists precisely
+  because `:latest` does not exist for this image on LSIO.
 - Calibre integration is folder-based, not API-based. Readarr drops files into
   the books root which [[Calibre-Web]] serves.
 - Not all indexers carry all formats (epub, mobi, pdf). Goodreads/StoryGraph
@@ -81,3 +82,10 @@ Restart=always
 
 - [[ARR-Stack]] — full *arr stack documentation
 - [[Calibre-Web]] — ebook web frontend that serves Readarr downloads
+
+## Citations
+
+- Servarr wiki, *Readarr* (Retired) — "Announcement: Retirement of Readarr
+  2025-06-27" — https://wiki.servarr.com/readarr (accessed 2026-10-19)
+- `Readarr/Readarr` on GitHub: `archived: true`, last push 2025-06-27 —
+  https://api.github.com/repos/Readarr/Readarr (accessed 2026-10-19)

serverAppConfigs-docs/Infrastructure/Alertmanager.md

@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "Alertmanager receives firing rules from Prometheus, groups and dedupes them, and forwards each group as a webhook to the host-side Signal relay — which sends it through..."
+description: "Receives firing rules from Prometheus, groups and dedupes them, and forwards each group to the host-side Signal relay."
 tags:
   - infrastructure
 timestamp: 2026-09-17

serverAppConfigs-docs/Infrastructure/Authentik Outpost Reference.md

@@ -0,0 +1,230 @@
+---
+type: reference
+description: "Outpost, proxy-provider, OAuth2, LDAP and flow configuration for the homelab's Authentik SSO."
+tags:
+  - infrastructure
+  - authentik
+timestamp: 2026-10-19
+---
+
+# Authentik Outpost Reference
+
+Configuration lookup for the [[Authentik]] deployment: the live proxy-provider
+inventory behind the guarded vhosts, the provisioning rules the embedded outpost
+enforces, this fork's OAuth2 relying-party quirks, the LDAP provider objects, the
+self-serve user-settings flow, and the live database shape. What the service is,
+how it is deployed, wired and operated is [[Authentik]]; the three core
+containers are [[Authentik Processes]].
+
+## Live proxy-provider inventory
+
+Live provider inventory (read from the `authentik-postgres` container with
+read-only `SELECT`s, 2026-10-18). Nine proxy providers, all attached to outpost
+`7e39655b-58d3-4fe6-a80a-513f31051217` ("authentik Embedded Outpost", 9
+members in `authentik_outposts_outpost_providers`), all with
+`internal_host = http://caddy:8017`, all carrying the **same three flow
+UUIDs** (`default-authentication-flow`,
+`default-provider-authorization-implicit-consent`,
+`default-provider-invalidation-flow`), and each backed by an Application
+(the `slug` column below):
+
+| Provider pk | name | external_host | app slug |
+|---|---|---|---|
+| 1 | embedded-outpost | `https://www.franksdank.cloud` | caddy |
+| 2 | search-proxy | `https://search.franksdank.cloud` | search |
+| 4 | prometheus-proxy | `https://prometheus.franksdank.cloud` | prometheus |
+| 5 | romm-proxy | `https://romm.franksdank.cloud` | romm |
+| 6 | files-proxy | `https://files.franksdank.cloud` | files |
+| 7 | loki-proxy | `https://loki.franksdank.cloud/` | loki |
+| 8 | grafana-proxy | `https://grafana.franksdank.cloud/` | grafana-guard |
+| 10 | seerr-proxy | `https://seerr.franksdank.cloud/` | seerr-guard |
+| 13 | marreta-proxy | `https://marreta.franksdank.cloud` | marreta |
+
+That is exactly the Caddyfile's guarded set (`dankclaw/Caddyfile:283-423`), so
+the lockstep rule in [[Authentik]] holds today. Two corrections against the 2026-09-02 table:
+**pk 13 (`marreta-proxy`) was added** when Marreta went onto the chain, and
+**pk 3 (`hermes-proxy`) no longer exists at all** — `select … where id=3`
+returns zero rows, so it is deleted, not "dormant"; its vhost survived but is
+exempt-by-construction (`dankclaw/Caddyfile:179-191`). OAuth2 relying parties,
+not proxy providers, so the proxy outpost does not deserialize them: pk 9
+`grafana` (`grant_types = {authorization_code}`), pk 12 `Seerr OIDC`
+(`grant_types = {}` — empty, i.e. still in the state the token-endpoint rule
+below rejects), and pk 14 `Open WebUI`
+(`grant_types = {authorization_code,refresh_token}`). Trailing slashes on
+`external_host` (the runbook-recipe style) and bare origins both intercept
+correctly — the outpost matches on hostname.
+
+## Provisioning rules for proxy providers
+
+Two provisioning rules, each verified live by an outage:
+
+1. **All three flows are required.** A ProxyProvider created via raw API/ORM
+   with any of `authentication_flow`/`authorization_flow`/`invalidation_flow`
+   null wedges the embedded outpost's config fetch: the Rust side rejects the
+   whole config JSON (serde: `invalid type: null, expected a formatted UUID
+   string`), logs `Failed to fetch outpost from API, retrying in 3 seconds`
+   forever, and **everything behind the outpost dies at the next outpost
+   reload** — including hosts whose providers were fine (verified 2026-09-02
+   when three providers provisioned without any flows took down every guarded
+   vhost after a server restart). Backfill all three flows (copy from pk 1:
+   `default-authentication-flow`,
+   `default-provider-authorization-implicit-consent` (the implicit-consent
+   authorization flow, `default-invalidation-flow`) and the retry loop
+   recovers on its next 3-second cycle. The silent killer: while the outpost
+   process keeps its last good in-memory config, everything works — the
+   outage only surfaces on restart/reload.
+2. **Every proxy provider needs an attached Application.** The outpost
+   resolves the post-login redirect through the provider's application; a
+   provider with no app completes the login (authentik session established)
+   but bounces to authentik's own user UI (`/if/user/#/library`) instead of
+   the app. No error is logged anywhere — the SSO "works" and the app never
+   appears. Create the Application with `provider_id` set (the UI does this
+   automatically; raw API/ORM drives must do it explicitly).
+
+## OAuth2 relying-party quirks (this fork, verified live 2026-08-31)
+
+The Grafana RP provider (`grafana`, pk 9, confidential client) was created and
+verified end-to-end via `ak shell` ORM drives. This fork's OAuth2 provider
+implementation diverges from upstream in ways that bit us; all verified live:
+
+- **Application row required.** `OAuth2Provider.objects.create()` does not
+  create the single-table-inheritance `Application` row; without it the flow
+  executor can never complete an authorize request. Create both rows.
+- **`grant_types` defaults to empty.** Until the provider's `grant_types`
+  contains `"authorization_code"` (enum value with underscore), the token
+  endpoint rejects every exchange with `invalid_request`.
+- **Token endpoint is form-encoded + Basic-auth only.** A JSON body makes
+  `parse_token_request` see no `grant_type` → `unsupported_grant_type`;
+  POST-body client secrets are not accepted — authenticate the confidential
+  client with `Authorization: Basic base64(client_id:client_secret)`.
+- **Provider resolution is by `client_id` query param** on the global
+  `/application/o/authorize/` route; per-slug authorize paths do not exist in
+  this build. Token/userinfo live at `/application/o/token/` and
+  `/application/o/userinfo/?api=1`.
+- **Property-mapping evaluator is broken → userinfo returns `sub` only.**
+  `PropertyMappingEvaluator.evaluate(*args, **kwargs)` forwards kwargs to
+  `BaseEvaluator.evaluate(expression_source)`, so the first context kwarg
+  becomes the *expression source* instead of the context; every call raises a
+  swallowed `PropertyMappingExpressionException` and the claim is skipped.
+  Scope-grant filtering has the same class of bug: `ScopeMapping` has no
+  provider FK (linkage lives in the `ProviderPropertyMapping` through model),
+  so `filter(provider=...)` always yields an empty queryset and tokens asked
+  to carry scopes get zero. Net effect: userinfo/introspection answers with
+  `{sub}` only — email/groups/entitlements claims cannot be delivered by this
+  build until the evaluator is patched or upgraded. Clients consuming this
+  provider must key identity off `sub`.
+- Scope mappings for this provider exist — `grafana-email-scope` and
+  `grafana-profile-scope` are both rows in `authentik_core_propertymapping`
+  (read-only `SELECT`, 2026-10-18) — and will start emitting claims the moment
+  the evaluator bug is fixed; no config change is needed then.
+
+## LDAP provider objects (provisioned via `ak shell` ORM)
+
+The LDAP outpost's own deployment — unit, image digest, published ports, token
+env var — is in [[Authentik]]. The objects it serves:
+
+Read from the live database (read-only `SELECT`s against the
+`authentik-postgres` container, 2026-10-18):
+
+| Object | Value |
+|---|---|
+| LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud`, `search_mode=direct`, `bind_mode=direct` |
+| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` *(unverified 2026-10-18: base DN confirmed in the DB, the `ou=users` path was not probed — an LDAP bind needs the bind secret)* |
+| Service account | `svc-jellyfin-ldap` (user pk 8). **It is not passwordless**: its password field is a real hash, not Django's `!`-prefixed unusable marker (`starts_with(password,'!')` → `f`) — an earlier revision's "passwordless/unusable" claim is wrong, which matters because an account with a usable password can sign in through the ordinary password flow *(unverified 2026-10-18: whether anyone holds that password)* |
+| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — an earlier revision recorded `intent=verification`; the live value is `app_password` |
+| RBAC role | `svc-jellyfin-ldap-search`, carrying exactly one object-level permission `authentik_providers_ldap.search_full_directory` (verified via `guardian_roleobjectpermission`) |
+| Groups | `jellyfin_users`, `jellyfin_admins` (both exist in `authentik_core_group`) |
+| Application | name `Jellyfin`, **slug `jellyfin-ldap`** (an earlier revision said slug `jellyfin`, which does not exist) |
+
+Hard-won rules (the first is now contested by the live database):
+
+- **Token intent: contested, do not trust either value blindly.** An earlier
+  revision asserted that a bind token with `intent=app_password` is rejected by
+  the flow's password stage with `Invalid password` and only
+  `intent=verification` passes. The live token for `svc-jellyfin-ldap` is
+  `app_password` (`authentik_core_token`, 2026-10-18), so either the rule is
+  wrong or the current Jellyfin bind path is not that token. Nothing observable
+  settles it today: the LDAP outpost's retained log holds only its 5-minute
+  `initialized direct binder` / `Update providers` config cycles and no bind
+  activity *(unverified 2026-10-18: no bind was performed — reproducing it
+  requires the bind secret)*. Re-verify with a real bind before changing the
+  token's intent either way.
+- **Search needs `search_full_directory`.** Without the RBAC grant the
+  outpost returns only the bind user itself — a `(memberOf=…)` filter then
+  matches zero entries and the plugin logs `Found no users matching X`.
+  The permission is object-level on the provider; grant it via an RBAC role
+  and retest.
+- `ak shell` import path for the provider model in 2026.8:
+  `from authentik.providers.ldap.models import LDAPProvider`.
+  *(unverified 2026-10-18: importing it outside a configured Django shell fails
+  with `ImproperlyConfigured`, and no `ak shell` was driven against the live
+  auth service for this check)*
+
+## Self-serve user-settings flow (password change)
+
+Users change their own account details through the `default-user-settings-flow`
+(`https://auth.franksdank.cloud/if/flow/default-user-settings-flow/`) — the
+target of the homepage footer's "account" link. It is a `stage_configuration`
+designated flow, so it renders every bound `ConfigurableStage` for the logged-in
+user.
+
+Stages, in execution order (the `FlowStageBinding.order` field, not the M2M
+insertion order):
+
+| order | stage | type | what it does |
+|---|---|---|---|
+| 20 | `default-user-settings` | `PromptStage` | name / email / locale **+ password** |
+| 100 | `default-user-settings-write` | `UserWriteStage` | writes the prompt data to the user |
+
+**Password change is a prompt field, not a stage.** The `UserWriteStage`
+writes whatever the preceding `PromptStage` collected: for the key `password`
+it calls `user.set_password(value)` (`stages/user_write/stage.py`). So the fix
+is a `password` field on the `default-user-settings` prompt
+(`field_key=password`, `type=password`, `required=False`, order 204) — NOT a
+`PasswordStage`. A `PasswordStage` is an *authentication* stage: its
+`validate_password` runs `authenticate()` against the backends to check the
+*current* password; it never sets a new one. (A `PasswordStage` was added on
+2026-09-25 and removed the same day for exactly this reason.)
+
+Two ORM gotchas: `Flow.stages.add()` fails on this through model
+(`Can't bulk create a multi-table inherited model` — `FlowStageBinding` is
+MTI), so create the `FlowStageBinding` row directly with an explicit `order`;
+and a prompt field is a `Prompt` row linked through `PromptStageField`
+(`prompt_stage` + `prompt`; the order comes from `Prompt.order`).
+
+**A blank password must not set an empty password.** `set_password("")` would
+make the account log in with no password, so a validation policy
+(`default-user-settings-no-empty-password`, bound to the prompt via
+`PromptStageValidationPolicy`) denies the form when `prompt_data["password"]`
+is `""` and allows it when set. Verified with the `PolicyEvaluator`
+(empty → deny, set → allow, absent → allow) and end-to-end by driving
+`UserWriteStageView.update_user` on a throwaway user (new password valid, old
+invalid).
+
+## Live database shape
+
+For live DB truth prefer the postgres container, and it works without any
+credential — the local socket is trusted, so a read-only audit is just
+`podman exec -i authentik-postgres psql -U authentik -d authentik -At -f -`
+with SQL on stdin (quoting SQL through ssh+zsh otherwise fights you). Verify
+before guessing: 2026.x renamed both tables and columns, and the models are
+MTI all the way down — `authentik_core_provider` holds `id`, `name` and the
+three `*_flow_id` columns; the proxy table is
+`authentik_providers_proxy_proxyprovider` (doubled `proxy`) and its PK column
+really is **`oauth2provider_ptr_id`** (proxy providers inherit the OAuth2
+table in 2026.8), the LDAP table is
+`authentik_providers_ldap_ldapprovider` with PK `provider_ptr_id`;
+`authentik_core_application` attaches to a provider through `provider_id`
+(its PK is `policybindingmodel_ptr_id`, not `id`), and `authentik_outposts_outpost`
+is keyed by `uuid`. So `Provider.objects.filter(type=…)` raising `FieldError`
+is the same story from the ORM side: `type` lives in the subclass tables.
+Never select secret columns (`client_secret`, `key`, `password`) into a log.
+
+## Cross-references
+
+- [[Authentik]] — what the service is, its deployment, wiring, operations and
+  failure modes
+- [[Authentik Processes]] — the postgres, server and worker containers
+- [[Caddy]] — the reverse proxy carrying the `:8016`/`:8017` auth-chain listeners
+- [[Jellyfin]] — consumes the LDAP outpost via its LDAP-Auth plugin
+- [[Grafana]] — guarded vhost *and* OAuth2 relying party of this provider

serverAppConfigs-docs/Infrastructure/Authentik Processes.md

@@ -0,0 +1,76 @@
+---
+type: reference
+description: "The three core Authentik containers — database, server and worker — with their units, images, roles and probes."
+tags:
+  - infrastructure
+  - authentik
+  - postgres
+timestamp: 2026-10-19
+---
+
+# Authentik Processes
+
+The three core containers of the [[Authentik]] stack on Dankclaw. The two
+outpost units are not covered here — they live on [[Authentik]] and
+[[Authentik Outpost Reference]]. Server and worker run the same image; the
+`command` selects the role (`server` / `worker`). Data persists under
+`/home/dank/Containers/authentik/` as bind mounts (repo policy — no named
+volumes), moved from `/home/dank/authentik/` in the 2026-09-01 quadlet
+standardization; the retired stubs this page replaces recorded the pre-move
+paths `/home/dank/authentik/data` (server and worker) and
+`/home/dank/authentik/postgres` (database).
+
+## authentik-postgres
+
+The PostgreSQL 16 database for Authentik — the only stateful store in the stack
+(users, flows, sessions, events). Sessions, caching, and WebSockets moved to
+PostgreSQL in 2025.10 (Redis removed).
+
+- **Image:** `docker.io/library/postgres:16-alpine`
+- **Data:** `/home/dank/Containers/authentik/postgres` (bind mount; no named
+  volumes)
+- **Health:** `pg_isready -d authentik -U authentik`
+- **Retired wording:** the stub called this a "Compose service … in
+  `dankclaw/compose.yaml`"; the compose stack is gone and the unit is a quadlet.
+- The side-by-side with `gitea-postgres` (network, published ports, secret
+  source, boot persistence) is owned by [[Postgres]].
+
+## authentik-server
+
+The Authentik server — the core, the admin API, and the embedded Caddy outpost.
+It also serves the login page at `https://auth.franksdank.cloud`.
+
+- **Unit:** `dankclaw/authentik/quadlet/authentik-server.container`
+  (`authentik-server.service`)
+- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (`command: server`)
+- **Port:** `9000` (published) — the outpost, the admin API and the login UI all
+  live here
+- **Data:** `/home/dank/Containers/authentik/data` (bind mount, shared with the
+  worker)
+- **Health:** recorded by the retired stub as
+  `http://localhost:9000/-/health/`; superseded — [[Authentik]] → Operations
+  owns the live probe.
+- **Retired wording:** the stub described "the Caddy `forward_auth` vhosts"
+  proxying here; the chain Caddy actually runs is [[Authentik]] → Caddy
+  integration.
+
+## authentik-worker
+
+The Authentik worker — background tasks: event processing, provider
+synchronization, scheduled flows, and certificate renewal.
+
+- **Unit:** `dankclaw/authentik/quadlet/authentik-worker.container`
+  (`authentik-worker.service`)
+- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (`command: worker`)
+- **Runs as:** `0:0` (rootless: container root maps to the host user that owns
+  the data dir)
+- **Data:** `/home/dank/Containers/authentik/data` (bind mount, shared with the
+  server)
+- **No port** — the worker has no HTTP endpoint.
+
+## Cross-references
+
+- [[Authentik]] — what the service is, its deployment, wiring, operations and
+  failure modes
+- [[Authentik Outpost Reference]] — provider, flow and token configuration
+- [[Postgres]] — both PostgreSQL clusters on Dankclaw, side by side

serverAppConfigs-docs/Infrastructure/Authentik.md

@@ -16,7 +16,7 @@ OpenWebUI, Jellyfin (LDAP outpost), the *arr stack, Pi-hole (basicauth edge) —
 they handle their own session. Grafana is the exception that is both: it is a
 guarded vhost (`grafana-proxy`, pk 8, locked to Authentik SSO) *and* an OAuth2
 relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`,
-`dankclaw/Caddyfile:374-377`; live provider table below).
+`dankclaw/Caddyfile:374-377`; live provider table in [[Authentik Outpost Reference]]).
 
 - **Version:** `2026.8.3` (pinned in `dankclaw/authentik/quadlet/*.container`;
   the outpost's boot log reads `"Booting authentik" … "version": "2026.8.3"`,
@@ -35,6 +35,13 @@ relying party (`grafana`, pk 9) (`dankclaw/Caddyfile:122-132`,
   pre-quadlet era, retired.
 - **LDAP outpost:** serves LDAP for Jellyfin — see the section below.
 
+**Split off from this page:** [[Authentik Processes]] owns the per-container
+facts for `authentik-postgres`, `authentik-server` and `authentik-worker`;
+[[Authentik Outpost Reference]] owns the provider/flow/token lookup material —
+the live proxy-provider inventory, the proxy-provider provisioning rules, the
+OAuth2 relying-party quirks, the LDAP provider objects, the user-settings flow
+design, and the live database shape.
+
 ## Why Authentik
 
 The comparison that chose it (against Authelia, Keycloak, Zitadel, Ory) was
@@ -100,23 +107,10 @@ contract:
   The chain terminates there — nothing dials back into Authentik from
   `:8017`.
 
-## The three compose services
+## The stack's processes
 
-| Service | Role | Image |
-|---|---|---|
-| `authentik-postgres` | PostgreSQL 16 (users, flows, sessions, events) | `docker.io/library/postgres:16-alpine` |
-| `authentik-server` | Core + admin API + the embedded Caddy outpost (port 9000) | `ghcr.io/goauthentik/server:2026.8.3` |
-| `authentik-worker` | Background tasks (event processing, provider sync, flows) | `ghcr.io/goauthentik/server:2026.8.3` |
-
-- **Postgres** is the only stateful store. Sessions, caching, and WebSockets
-  moved to PostgreSQL in 2025.10 (Redis removed).
-- **Server** and **worker** run the same image; the `command` selects the role
-  (`server` / `worker`).
-- The worker runs as `0:0` (rootless: container root maps to the host user that
-  owns the data dir).
-- Data persists under `/home/dank/Containers/authentik/{data,postgres}`
-  (bind mounts, repo policy — no named volumes; moved from `/home/dank/authentik/`
-  in the 2026-09-01 quadlet standardization).
+Per-container facts — unit path, image and role, port, run-as, data directory,
+health probe — are owned by [[Authentik Processes]].
 
 ## Caddy integration
 
@@ -142,104 +136,9 @@ routing table, and the Authentik proxy-provider list must be the same set of
 hosts. A guarded host with no provider 404s at login time; a provider with no
 DNS record breaks off-LAN clients.
 
-Live provider inventory (read from the `authentik-postgres` container with
-read-only `SELECT`s, 2026-10-18). Nine proxy providers, all attached to outpost
-`7e39655b-58d3-4fe6-a80a-513f31051217` ("authentik Embedded Outpost", 9
-members in `authentik_outposts_outpost_providers`), all with
-`internal_host = http://caddy:8017`, all carrying the **same three flow
-UUIDs** (`default-authentication-flow`,
-`default-provider-authorization-implicit-consent`,
-`default-provider-invalidation-flow`), and each backed by an Application
-(the `slug` column below):
-
-| Provider pk | name | external_host | app slug |
-|---|---|---|---|
-| 1 | embedded-outpost | `https://www.franksdank.cloud` | caddy |
-| 2 | search-proxy | `https://search.franksdank.cloud` | search |
-| 4 | prometheus-proxy | `https://prometheus.franksdank.cloud` | prometheus |
-| 5 | romm-proxy | `https://romm.franksdank.cloud` | romm |
-| 6 | files-proxy | `https://files.franksdank.cloud` | files |
-| 7 | loki-proxy | `https://loki.franksdank.cloud/` | loki |
-| 8 | grafana-proxy | `https://grafana.franksdank.cloud/` | grafana-guard |
-| 10 | seerr-proxy | `https://seerr.franksdank.cloud/` | seerr-guard |
-| 13 | marreta-proxy | `https://marreta.franksdank.cloud` | marreta |
-
-That is exactly the Caddyfile's guarded set (`dankclaw/Caddyfile:283-423`), so
-the lockstep rule holds today. Two corrections against the 2026-09-02 table:
-**pk 13 (`marreta-proxy`) was added** when Marreta went onto the chain, and
-**pk 3 (`hermes-proxy`) no longer exists at all** — `select … where id=3`
-returns zero rows, so it is deleted, not "dormant"; its vhost survived but is
-exempt-by-construction (`dankclaw/Caddyfile:179-191`). OAuth2 relying parties,
-not proxy providers, so the proxy outpost does not deserialize them: pk 9
-`grafana` (`grant_types = {authorization_code}`), pk 12 `Seerr OIDC`
-(`grant_types = {}` — empty, i.e. still in the state the token-endpoint rule
-below rejects), and pk 14 `Open WebUI`
-(`grant_types = {authorization_code,refresh_token}`). Trailing slashes on
-`external_host` (the runbook-recipe style) and bare origins both intercept
-correctly — the outpost matches on hostname.
-
-Two provisioning rules, each verified live by an outage:
-
-1. **All three flows are required.** A ProxyProvider created via raw API/ORM
-   with any of `authentication_flow`/`authorization_flow`/`invalidation_flow`
-   null wedges the embedded outpost's config fetch: the Rust side rejects the
-   whole config JSON (serde: `invalid type: null, expected a formatted UUID
-   string`), logs `Failed to fetch outpost from API, retrying in 3 seconds`
-   forever, and **everything behind the outpost dies at the next outpost
-   reload** — including hosts whose providers were fine (verified 2026-09-02
-   when three providers provisioned without any flows took down every guarded
-   vhost after a server restart). Backfill all three flows (copy from pk 1:
-   `default-authentication-flow`,
-   `default-provider-authorization-implicit-consent` (the implicit-consent
-   authorization flow, `default-invalidation-flow`) and the retry loop
-   recovers on its next 3-second cycle. The silent killer: while the outpost
-   process keeps its last good in-memory config, everything works — the
-   outage only surfaces on restart/reload.
-2. **Every proxy provider needs an attached Application.** The outpost
-   resolves the post-login redirect through the provider's application; a
-   provider with no app completes the login (authentik session established)
-   but bounces to authentik's own user UI (`/if/user/#/library`) instead of
-   the app. No error is logged anywhere — the SSO "works" and the app never
-   appears. Create the Application with `provider_id` set (the UI does this
-   automatically; raw API/ORM drives must do it explicitly).
-
-
-## OAuth2 relying-party quirks (this fork, verified live 2026-08-31)
-
-The Grafana RP provider (`grafana`, pk 9, confidential client) was created and
-verified end-to-end via `ak shell` ORM drives. This fork's OAuth2 provider
-implementation diverges from upstream in ways that bit us; all verified live:
-
-- **Application row required.** `OAuth2Provider.objects.create()` does not
-  create the single-table-inheritance `Application` row; without it the flow
-  executor can never complete an authorize request. Create both rows.
-- **`grant_types` defaults to empty.** Until the provider's `grant_types`
-  contains `"authorization_code"` (enum value with underscore), the token
-  endpoint rejects every exchange with `invalid_request`.
-- **Token endpoint is form-encoded + Basic-auth only.** A JSON body makes
-  `parse_token_request` see no `grant_type` → `unsupported_grant_type`;
-  POST-body client secrets are not accepted — authenticate the confidential
-  client with `Authorization: Basic base64(client_id:client_secret)`.
-- **Provider resolution is by `client_id` query param** on the global
-  `/application/o/authorize/` route; per-slug authorize paths do not exist in
-  this build. Token/userinfo live at `/application/o/token/` and
-  `/application/o/userinfo/?api=1`.
-- **Property-mapping evaluator is broken → userinfo returns `sub` only.**
-  `PropertyMappingEvaluator.evaluate(*args, **kwargs)` forwards kwargs to
-  `BaseEvaluator.evaluate(expression_source)`, so the first context kwarg
-  becomes the *expression source* instead of the context; every call raises a
-  swallowed `PropertyMappingExpressionException` and the claim is skipped.
-  Scope-grant filtering has the same class of bug: `ScopeMapping` has no
-  provider FK (linkage lives in the `ProviderPropertyMapping` through model),
-  so `filter(provider=...)` always yields an empty queryset and tokens asked
-  to carry scopes get zero. Net effect: userinfo/introspection answers with
-  `{sub}` only — email/groups/entitlements claims cannot be delivered by this
-  build until the evaluator is patched or upgraded. Clients consuming this
-  provider must key identity off `sub`.
-- Scope mappings for this provider exist — `grafana-email-scope` and
-  `grafana-profile-scope` are both rows in `authentik_core_propertymapping`
-  (read-only `SELECT`, 2026-10-18) — and will start emitting claims the moment
-  the evaluator bug is fixed; no config change is needed then.
+The live proxy-provider inventory (the per-vhost `external_host` → app mapping),
+the two provisioning rules the outpost enforces, and this fork's OAuth2
+relying-party quirks are reference lookup: [[Authentik Outpost Reference]].
 
 ## LDAP outpost (Jellyfin auth, live 2026-09-01)
 
@@ -267,44 +166,9 @@ rule at the current `2026.8.3` pin. Resolve once with `podman manifest inspect
   (2026-10-18). The outpost token travels in `AUTHENTIK_LDAP_OUTPOST_TOKEN`
   from the host `dankclaw/.env` (git-ignored).
 
-### Authentik-side objects (provisioned via `ak shell` ORM)
-
-Read from the live database (read-only `SELECT`s against the
-`authentik-postgres` container, 2026-10-18):
-
-| Object | Value |
-|---|---|
-| LDAP provider | `Jellyfin LDAP` (pk 11), base DN `dc=franksdank,dc=cloud`, `search_mode=direct`, `bind_mode=direct` |
-| Bind DN | `cn=svc-jellyfin-ldap,ou=users,dc=franksdank,dc=cloud` *(unverified 2026-10-18: base DN confirmed in the DB, the `ou=users` path was not probed — an LDAP bind needs the bind secret)* |
-| Service account | `svc-jellyfin-ldap` (user pk 8). **It is not passwordless**: its password field is a real hash, not Django's `!`-prefixed unusable marker (`starts_with(password,'!')` → `f`) — the page's earlier "passwordless/unusable" claim is wrong, which matters because an account with a usable password can sign in through the ordinary password flow *(unverified 2026-10-18: whether anyone holds that password)* |
-| Bind token | exactly one row for pk 8, **`intent=app_password`**, non-expiring — the page previously recorded `intent=verification`; the live value is `app_password` |
-| RBAC role | `svc-jellyfin-ldap-search`, carrying exactly one object-level permission `authentik_providers_ldap.search_full_directory` (verified via `guardian_roleobjectpermission`) |
-| Groups | `jellyfin_users`, `jellyfin_admins` (both exist in `authentik_core_group`) |
-| Application | name `Jellyfin`, **slug `jellyfin-ldap`** (the page previously said slug `jellyfin`, which does not exist) |
-
-Hard-won rules (the first is now contested by the live database):
-
-- **Token intent: contested, do not trust either value blindly.** This page
-  long asserted that a bind token with `intent=app_password` is rejected by the
-  flow's password stage with `Invalid password` and only `intent=verification`
-  passes. The live token for `svc-jellyfin-ldap` is `app_password`
-  (`authentik_core_token`, 2026-10-18), so either the rule is wrong or the
-  current Jellyfin bind path is not that token. Nothing observable settles it
-  today: the LDAP outpost's retained log holds only its 5-minute
-  `initialized direct binder` / `Update providers` config cycles and no bind
-  activity *(unverified 2026-10-18: no bind was performed — reproducing it
-  requires the bind secret)*. Re-verify with a real bind before changing the
-  token's intent either way.
-- **Search needs `search_full_directory`.** Without the RBAC grant the
-  outpost returns only the bind user itself — a `(memberOf=…)` filter then
-  matches zero entries and the plugin logs `Found no users matching X`.
-  The permission is object-level on the provider; grant it via an RBAC role
-  and retest.
-- `ak shell` import path for the provider model in 2026.8:
-  `from authentik.providers.ldap.models import LDAPProvider`.
-  *(unverified 2026-10-18: importing it outside a configured Django shell fails
-  with `ImproperlyConfigured`, and no `ak shell` was driven against the live
-  auth service for this check)*
+The objects behind it — LDAP provider, bind DN, service account and bind token,
+RBAC grant, groups, application slug — and the hard-won bind rules are in
+[[Authentik Outpost Reference]].
 
 ## Configuration (secrets)
 
@@ -340,7 +204,8 @@ every outpost session (see the ORM playbook).
    Applications** → add an application + a **Proxy** provider whose
    `external_host` is exactly the browser-facing origin
    (`https://<host>`), `internal_host = http://caddy:8017`, and **all three**
-   flows set (see the provisioning pitfall above). Attach the provider to the
+   flows set (see the provisioning rules in
+   [[Authentik Outpost Reference]]). Attach the provider to the
    embedded outpost.
 4. For apps with native OIDC: add an **OAuth2/OpenID Connect** provider, create
    the app with the correct redirect URI, and paste the `client_id` /
@@ -373,47 +238,6 @@ every outpost session (see the ORM playbook).
   the LDAP section) — bumping only the tag would leave the LDAP outpost behind.
   Authentik migrates the schema on startup.
 
-## Self-serve user-settings flow (password change)
-
-Users change their own account details through the `default-user-settings-flow`
-(`https://auth.franksdank.cloud/if/flow/default-user-settings-flow/`) — the
-target of the homepage footer's "account" link. It is a `stage_configuration`
-designated flow, so it renders every bound `ConfigurableStage` for the logged-in
-user.
-
-Stages, in execution order (the `FlowStageBinding.order` field, not the M2M
-insertion order):
-
-| order | stage | type | what it does |
-|---|---|---|---|
-| 20 | `default-user-settings` | `PromptStage` | name / email / locale **+ password** |
-| 100 | `default-user-settings-write` | `UserWriteStage` | writes the prompt data to the user |
-
-**Password change is a prompt field, not a stage.** The `UserWriteStage`
-writes whatever the preceding `PromptStage` collected: for the key `password`
-it calls `user.set_password(value)` (`stages/user_write/stage.py`). So the fix
-is a `password` field on the `default-user-settings` prompt
-(`field_key=password`, `type=password`, `required=False`, order 204) — NOT a
-`PasswordStage`. A `PasswordStage` is an *authentication* stage: its
-`validate_password` runs `authenticate()` against the backends to check the
-*current* password; it never sets a new one. (A `PasswordStage` was added on
-2026-09-25 and removed the same day for exactly this reason.)
-
-Two ORM gotchas: `Flow.stages.add()` fails on this through model
-(`Can't bulk create a multi-table inherited model` — `FlowStageBinding` is
-MTI), so create the `FlowStageBinding` row directly with an explicit `order`;
-and a prompt field is a `Prompt` row linked through `PromptStageField`
-(`prompt_stage` + `prompt`; the order comes from `Prompt.order`).
-
-**A blank password must not set an empty password.** `set_password("")` would
-make the account log in with no password, so a validation policy
-(`default-user-settings-no-empty-password`, bound to the prompt via
-`PromptStageValidationPolicy`) denies the form when `prompt_data["password"]`
-is `""` and allows it when set. Verified with the `PolicyEvaluator`
-(empty → deny, set → allow, absent → allow) and end-to-end by driving
-`UserWriteStageView.update_user` on a throwaway user (new password valid, old
-invalid).
-
 ## Outpost claim cache (`proxysession`) — 403 after a group add
 
 The embedded outpost does not re-evaluate group membership per request. Each
@@ -457,7 +281,7 @@ client → TLS edge → Anubis :8923 → outpost decision → internal
    (the inventory table carries both bare origins and trailing slashes;
    routing matches on hostname, verified 2026-09-02).
    `internal_host = http://caddy:8017`, all three flows set, and an
-   Application attached — the two provisioning rules above.
+   Application attached — the two provisioning rules in [[Authentik Outpost Reference]].
 2. **Route flip in ONE commit/deploy**: the vhost becomes
    `reverse_proxy anubis:8923` with `header_up X-Real-Ip {remote_host}`, and
    inside the `http://:8017` listener add `@<svc> host <vhost>` +
@@ -521,22 +345,9 @@ EOF
   is no `authentik.policies.group` module in this build.
   *(unverified 2026-10-18: these module paths were not re-executed — importing
   outside a configured Django shell fails with `ImproperlyConfigured`)*
-- For live DB truth prefer the postgres container, and it works without any
-  credential — the local socket is trusted, so a read-only audit is just
-  `podman exec -i authentik-postgres psql -U authentik -d authentik -At -f -`
-  with SQL on stdin (quoting SQL through ssh+zsh otherwise fights you). Verify
-  before guessing: 2026.x renamed both tables and columns, and the models are
-  MTI all the way down — `authentik_core_provider` holds `id`, `name` and the
-  three `*_flow_id` columns; the proxy table is
-  `authentik_providers_proxy_proxyprovider` (doubled `proxy`) and its PK column
-  really is **`oauth2provider_ptr_id`** (proxy providers inherit the OAuth2
-  table in 2026.8), the LDAP table is
-  `authentik_providers_ldap_ldapprovider` with PK `provider_ptr_id`;
-  `authentik_core_application` attaches to a provider through `provider_id`
-  (its PK is `policybindingmodel_ptr_id`, not `id`), and `authentik_outposts_outpost`
-  is keyed by `uuid`. So `Provider.objects.filter(type=…)` raising `FieldError`
-  is the same story from the ORM side: `type` lives in the subclass tables.
-  Never select secret columns (`client_secret`, `key`, `password`) into a log.
+- The live database's table and column names (the 2026.x renames, the MTI parent
+  keys, the read-only `psql` audit recipe) are reference lookup:
+  [[Authentik Outpost Reference]].
 
 ## User provisioning playbook (user-level account)
 

serverAppConfigs-docs/Infrastructure/Grafana.md

@@ -69,7 +69,7 @@ Grafana responds on `/api/health` with 200 and `"database":"ok"`; measured 2026-
   client_id `grafana-21ca190cde75d60f`, global authorize/token/userinfo paths,
   Basic client auth, form-encoded token body, and the fork's userinfo-claims
   bug that leaves identity at the `sub` hash only) is in
-  [[Authentik]] → "OAuth2 relying-party quirks". Consequences for Grafana:
+  [[Authentik Outpost Reference]] → "OAuth2 relying-party quirks". Consequences for Grafana:
   The unit sets `ALLOW_SIGN_UP=false`, `AUTO_UPSERT=true`, `ALLOW_ASSIGN_ORG=true`,
   `ORG_ID=1` and **no** role-attribute mapping (`grafana.container:52-55`), so
   users land as Viewer with `login = <sub hex>`; no email/groups/role claims

serverAppConfigs-docs/Infrastructure/Homepage Status Probes.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Homepage Status Probes
-description: The homepage TUI status page — how probe targets are chosen (host-netns vantage), the three false-down shapes, triage procedure, local render check, deploy, and the e2e gate interactions (asleep Mac mini, raw-IP matrix).
+description: "How probe targets are chosen, the three false-down shapes, the triage procedure, and the e2e gate interactions."
 tags: [homepage, status, probes, e2e, homepage-status]
 timestamp: 2026-10-05T00:00:00-07:00
 ---

serverAppConfigs-docs/Infrastructure/Homepage.md

@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "A password-protected web site on Dankclaw lists the services that are up, with a short description of each and a link to both the service and its usage documentation."
+description: "The password-protected service list on Dankclaw: what each card shows and where its data comes from."
 tags:
   - infrastructure
   - homepage
@@ -159,10 +159,9 @@ Sonarr, Radarr, Lidarr, Readarr, qBittorrent) is out of the family docs
 entirely — admin tooling; Seerr is presented as the front door for
 requests. Each fleet model gained a Hugging Face source link: card field
 `model.hf` rendered as the cyan `model on huggingface ↗` line under the
-role, and the docs `Model:` bullets link the checkpoint names (zealot →
-HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive, dragoon →
-Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated, archon →
-Qwen/Qwen3.8-Flash-Next-FP8).
+role, and the docs `Model:` bullets link the checkpoint names — which
+checkpoint each endpoint runs is payload state on
+[[frank-dragoon]], [[frank-archon]] and [[frank-zealot]].
 
 **Navy Protoss console, maximal cornering, font swap (2026-10-08, sixth
 wave):** the amber CRT palette is retired. The page is now deep space
@@ -235,11 +234,10 @@ house-DNS advice; the bullet now just says the names resolve on the
 home network.
 
 **Archon card runs the fleet's own model (2026-10-08):** the
-`frank-archon` homepage card and docs Model bullet now say the payload
-is our in-house GPTQ-w4 quant of Swift1.5-Qwen3.8-Flash-Next and link
-our HF publication (`frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700`)
-in place of the FP8-era base-model link; capacity line says three
-submissions (`--max-num-seqs 3`).
+`frank-archon` homepage card and docs Model bullet swapped the FP8-era
+base-model link for our own HF publication, and the capacity line for the
+endpoint's own concurrency figure; the payload, its publication and the slot
+count all live on [[frank-archon]].
 
 ## Design decision — 2026-08-22
 

serverAppConfigs-docs/Infrastructure/Observability Provisioning.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Observability Provisioning
-description: How Prometheus config/rules and Grafana datasources/dashboards are provisioned as code in serverAppConfigs — the config triad, deploy copiers, planner mappings, seam tests, entry points for each change type, and API verification.
+description: "Monitoring config as code: the config triad, deploy copiers, planner mappings, and where each change type lands."
 tags: [observability, prometheus, grafana, provisioning, deployment]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -130,13 +130,14 @@ the blackbox exporter. See [[Prometheus]] → Targets.
 **A llama.cpp endpoint is invisible unless it opts in.** Until 2026-10-18,
 `up{job="frank-zealot"}` was 0 for the entire retained history of the metric —
 the oldest sample, 2026-10-11, is already 0 — while the service itself answered
-`/health` 200 the whole time. Cause: llama.cpp answers **501 Not Implemented**
-on `/metrics` unless the server is started with `--metrics`, so the scrape was
-refused at the protocol layer and `LlmEndpointDown` (critical) stayed firing:
+`/health` 200 the whole time. Cause: llama.cpp only exports `/metrics` when it
+is started with the flag, so the scrape was refused at the protocol layer (the
+flag and its opt-out are on [[frank-zealot]]) and `LlmEndpointDown` (critical)
+stayed firing:
 the relay re-sent it on the 4 h `repeat_interval` for days (`journalctl --user
 -u signal-relay` — firing sends at 01:37, 05:42, 09:47, 13:52, 17:57 and 22:02
 on 2026-10-12). It finally resolved at `2026-10-18T01:53:21-07:00`, after the
-`METRICS=1` knob added to `scripts/frank-zealot.env` that day; the target
+metrics opt-in landed that day; the target
 reports up since (`curl -s http://127.0.0.1:9090/api/v1/targets` 2026-10-18).
 How many payload generations it spanned is not readable from any host
 artifact. *(unverified 2026-10-18)*
@@ -151,7 +152,8 @@ notification log — `dankclaw/observability/quadlet/alertmanager.container:39-4
 `/opt/alertmanager/data`.)
 
 Remaining known downs are the structural ones above (HTML-scraped probes, Gitea
-metrics, ComfyUI), and halogen serves `/metrics` natively, which is why
+metrics, ComfyUI), and `frank-dragoon` exports `/metrics` natively
+([[frank-dragoon]]), which is why
 `frank-dragoon` has always been green. The active-alert count is a
 moment-in-time value and should never be documented as a constant:
 `/api/v1/alerts` was empty at the 2026-10-18 morning census, held one firing
@@ -167,8 +169,8 @@ arm (`systemctl is-active frank-zealot` → `failed` after
 `stop-sigterm timed out. Killing`, which is llama.cpp ignoring SIGTERM past
 `TimeoutStopSec`, the known shutdown shape for these engines). So
 `up{job="frank-zealot"} == 1` that day measured the trial, not the production
-shape in `scripts/frank-zealot.env` (Ternary-Bonsai-27B-PQ2_0, `CTX_SIZE=401408
-PARALLEL=2` = 2 × 200,704 per slot, q8_0 K+V), which remains the repo contract.
+shape in `scripts/frank-zealot.env` — that shape is [[frank-zealot]]'s to state,
+and it is the repo contract.
 The unit stayed `enabled`; `./scripts/deploy.sh dankclaw --service frank-zealot`
 reinstalls and starts it, restoring boot persistence and self-healing. Never
 quote a candidate arm's flags or VRAM as fleet state — the process and its shape

serverAppConfigs-docs/Infrastructure/Observability.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: "Observability on Dankclaw"
-description: "The Dankclaw observability stack — Prometheus, Grafana, Loki, Alloy, node-exporter and Alertmanager (Signal delivery): what runs, how it is deployed, how it is exposed, and what is still unconfigured."
+description: "The observability stack: what runs, how it is deployed, how it is exposed, and what is still unconfigured."
 tags: [observability, prometheus, grafana, loki, alloy, node-exporter, quadlet, dankclaw]
 timestamp: 2026-09-11
 ---

serverAppConfigs-docs/Infrastructure/Prometheus.md

@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "Prometheus collects metrics from every service on Dankclaw by scraping their `/metrics` endpoints, and feeds the provisioned Grafana datasource. Down targets today are structural, not outages."
+description: "Scrapes each service's metrics endpoint and feeds the provisioned Grafana datasource; down targets are structural."
 tags:
   - infrastructure
   - prometheus
@@ -25,7 +25,7 @@ Authentik; post-auth the Caddy vhost routes straight to `prometheus:9090`) —
 | Config | `/opt/prometheus/config/prometheus.yml`, installed from `shared/prometheus/prometheus.yml` on every deploy, together with `/opt/prometheus/config/rules/llm-fleet.yml` from `shared/prometheus/rules/llm-fleet.yml` (`scripts/deploy-remote.sh:219-227`) |
 | Data | `/opt/prometheus/data` (TSDB, `prometheus.container:52`) |
 | Retention | `--storage.tsdb.retention.time=30d` (`prometheus.container:49`) |
-| Alerting | **4 alert rules + 2 recording rules** (`shared/prometheus/rules/llm-fleet.yml`: alerts `LlmRequestsQueueing`, `LlmPrefixCacheCollapse`, `LlmSpecDecodeOff`, `LlmEndpointDown`; recordings `job:llm_prefix_cache_reuse:rate5m`, `job:llm_spec_accept_length:rate5m`), confirmed against the running server: `curl -s http://127.0.0.1:9090/api/v1/rules` (2026-10-18) lists 4 alerting + 2 recording. Evaluated **and delivered** since 2026-09-17: the `alerting:` block ships to `alertmanager:9093` (`prometheus.yml:19-22`) → Signal relay → signal-cli (see [[Alertmanager]]) |
+| Alerting | **4 alert rules + 2 recording rules** (`shared/prometheus/rules/llm-fleet.yml`: alerts `LlmRequestsQueueing`, `LlmPrefixCacheCollapse`, `LlmSpecDecodeOff`, `LlmEndpointDown`; recordings `job:llm_prefix_cache_reuse:rate5m`, `job:llm_spec_accept_length:rate5m`), confirmed against the running server: `curl -s http://127.0.0.1:9090/api/v1/rules` (2026-10-18) lists 4 alerting + 2 recording. `LlmEndpointDown` is scoped `up{job=~"frank-.*"} == 0`, so only the `frank-*` LLM jobs can page on it — any other job that is structurally never-green must stay out of that family ([[Stale Signal Audit]]). Evaluated **and delivered** since 2026-09-17: the `alerting:` block ships to `alertmanager:9093` (`prometheus.yml:19-22`) → Signal relay → signal-cli (see [[Alertmanager]]) |
 
 ## Targets
 
@@ -37,10 +37,10 @@ Authentik; post-auth the Caddy vhost routes straight to `prometheus:9090`) —
 | `node-dankclaw` | `host.containers.internal:9100` | host metrics from [[Node-Exporter]] — a net=host bind, so via the gateway (the LAN address is unreachable from this bridge container) |
 | `node-dankserver` | `192.168.0.235:9100` | host metrics from [[Node-Exporter]] — a different machine, so the LAN address works |
 | `grafana`, `loki` | `grafana:3000`, `loki:3100` | container names on the `dankclaw_default` network; both serve real `/metrics` |
-| `frank-archon` | `192.168.0.235:7770` | the LLM fleet — the radiance engine since 2026-10-05, exporting vLLM-compatible names plus `radiance:*` (`prometheus.yml:122-131`); a different machine, so the LAN address works |
-| `frank-dragoon`, `frank-zealot` | `host.containers.internal:7770`, `:7771` | the LLM fleet — both export the `llamacpp:` namespace (halogen included: `curl -s http://127.0.0.1:7770/metrics` → `llamacpp:prompt_tokens_total`, `text/plain; version=0.0.4`, 2026-10-18). They are addressed through the gateway because they are **same-host** — `frank-zealot` is a host-process bind, `frank-dragoon` a container publishing `0.0.0.0:7770->8731`, not a net=host pair (`prometheus.yml:96-99`, `:108-120`) |
+| `frank-archon` | `192.168.0.235:7770` | the LLM fleet — a different machine, so the LAN address works (`prometheus.yml:122-131`); engine, metric families and their meaning live on [[frank-archon]] |
+| `frank-dragoon`, `frank-zealot` | `host.containers.internal:7770`, `:7771` | the LLM fleet. They are addressed through the gateway because they are **same-host** — a host-process bind and a published container port are equally invisible to a bridge container at the host's LAN address (`prometheus.yml:96-99`, `:108-120`); what each endpoint exports is on [[frank-dragoon]] and [[frank-zealot]] |
 
-The `frank-medium` job and its `vllm:` namespace are **retired**: that endpoint left the fleet, `frank-archon` briefly had no exporter at all (the FreeToken-era job scraped 404s forever) and re-joined on radiance the same day, 2026-10-05 (`prometheus.yml:101-107`). No `frank-medium` job exists in the config or in `/api/v1/targets` (2026-10-18).
+The `frank-medium` job is **retired**: no `frank-medium` job exists in the config or in `/api/v1/targets` (2026-10-18), and the endpoint itself left the fleet — its history is on [[frank-archon]], which holds the port today.
 
 **Three jobs read down today, and they are not outages** (re-audited against `curl -s http://127.0.0.1:9090/api/v1/targets`, 2026-10-18; first audited 2026-09-16):
 
@@ -53,15 +53,14 @@ The `frank-medium` job and its `vllm:` namespace are **retired**: that endpoint
 
 Gitea's exporter stays off until `[metrics] ENABLED = true` in its own config (no metrics env is set in the container, and `curl -s http://127.0.0.1:3000/metrics` → 404, 2026-10-18); ComfyUI and the two `services` probes are HTML pages, which Prometheus 3 refuses as metrics. Everything else is green: `prometheus`, `node-dankclaw`, `node-dankserver`, `grafana`, `loki`, `frank-zealot`, `frank-dragoon`, `frank-archon`.
 
-**`frank-zealot` reads up since 2026-10-18, and did not before.** llama.cpp answers **501 Not Implemented** on `/metrics` unless started with `--metrics`; the endpoint had no such flag, so `up{job="frank-zealot"}` was 0 across the whole retained history (the oldest sample, 2026-10-11, is already 0) while the service answered `/health` 200 the whole time. The `METRICS=1` knob in `scripts/frank-zealot.env` is the fix, and the target is up as measured 2026-10-18. Read `lastError` before believing an outage: a target can read down beside a healthy service.
+**`frank-zealot` reads up since 2026-10-18, and did not before.** llama.cpp only answers `/metrics` when started with `--metrics`, so `up{job="frank-zealot"}` was 0 across the whole retained history (the oldest sample, 2026-10-11, is already 0) while the service answered `/health` 200 the whole time; the flag and its opt-out are on [[frank-zealot]]. Read `lastError` before believing an outage: a target can read down beside a healthy service.
 
 Alert state is a live property, not a page fact: `curl -s http://127.0.0.1:9090/api/v1/alerts` returned no alerts at the 2026-10-18 morning census, one firing `LlmPrefixCacheCollapse` (info, `frank-zealot`, `activeAt` 11:10:48Z) minutes later, and none again by 11:56Z.
 
-A scrape cannot express "is this HTTP service answering?" — that is what a
-probe is for, and the remedy is the **blackbox exporter** in probe mode
-(`/probe?target=…&module=http_2xx`), which also gives the guarded vhosts a real
-uptime signal since Prometheus cannot solve an Anubis challenge itself. Until it
-is deployed, these three stay red for a structural reason: deleting them would
+The remedy for a probe-shaped question is the **blackbox exporter**; the
+classification rule and the honesty law for it live in [[Stale Signal Audit]].
+Until the exporter is deployed, these three stay red for a structural reason:
+deleting them would
 hide a real service, and fixing them by publishing databases or enabling metrics
 would trade a wrong signal for a real exposure. Targets are `host:port`, never
 URLs.

serverAppConfigs-docs/Infrastructure/authentik-postgres.md

@@ -1,24 +0,0 @@
----
-type: reference
-description: "The PostgreSQL 16 database for [Authentik](Authentik.md) — the homelab's SSO /"
-tags:
-  - infrastructure
-  - authentik
-  - postgres
-timestamp: 2026-08-28
----
-
-# authentik-postgres
-
-The PostgreSQL 16 database for [Authentik](Authentik.md) — the homelab's SSO /
-OIDC provider. It is the only stateful store in the stack (users, flows,
-sessions, events). Redis was removed in Authentik 2025.10; sessions and caching
-now live here.
-
-- **Compose service:** `authentik-postgres` (in `dankclaw/compose.yaml`)
-- **Image:** `docker.io/library/postgres:16-alpine`
-- **Data:** `/home/dank/authentik/postgres` (bind mount; no named volumes)
-- **Health:** `pg_isready -d authentik -U authentik`
-
-See [Authentik.md](Authentik.md) for the full architecture, setup, backups
-(`pg_dump`), and failure modes.

serverAppConfigs-docs/Infrastructure/authentik-server.md

@@ -1,24 +0,0 @@
----
-type: reference
-description: "The Authentik server — the core, the admin API, and the embedded Caddy outpost."
-tags:
-  - infrastructure
-  - authentik
-timestamp: 2026-09-16
----
-
-# authentik-server
-
-The Authentik server — the core, the admin API, and the embedded Caddy outpost.
-This is the service the Caddy `forward_auth` vhosts proxy to
-(`authentik-server:9000`). It also serves the login page at
-`https://auth.franksdank.cloud`.
-
-- **Unit:** `dankclaw/authentik/quadlet/authentik-server.container` (`authentik-server.service`)
-- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (command: `server`)
-- **Port:** `9000` (published; the outpost + admin + login all live here)
-- **Data:** `/home/dank/authentik/data` (bind mount, shared with the worker)
-- **Health:** `http://localhost:9000/-/health/`
-
-See [Authentik.md](Authentik.md) for the full architecture, the Caddy
-forward_auth pattern, first-time setup, and failure modes.

serverAppConfigs-docs/Infrastructure/authentik-worker.md

@@ -1,23 +0,0 @@
----
-type: reference
-description: "The Authentik worker — background tasks: event processing, provider"
-tags:
-  - infrastructure
-  - authentik
-timestamp: 2026-09-16
----
-
-# authentik-worker
-
-The Authentik worker — background tasks: event processing, provider
-synchronization, scheduled flows, and certificate renewal. It runs the same
-image as the server; the `command: worker` selects the role.
-
-- **Unit:** `dankclaw/authentik/quadlet/authentik-worker.container` (`authentik-worker.service`)
-- **Image:** `ghcr.io/goauthentik/server:2026.8.3` (command: `worker`)
-- **Runs as:** `0:0` (rootless: container root maps to the host user that owns the data dir)
-- **Data:** `/home/dank/authentik/data` (bind mount, shared with the server)
-- **No port** — the worker has no HTTP endpoint.
-
-See [Authentik.md](Authentik.md) for the full architecture, setup, and failure
-modes.

serverAppConfigs-docs/LLM/DFlash DSpark Speculative Decoding.md

@@ -14,7 +14,7 @@ timestamp: 2026-09-03
 
 DFlash and DSpark are the two newest speculative decoding methods. They draft many tokens in one parallel pass instead of one token at a time. This reduces decode latency on memory-bound workloads.
 
-DFlash comes from Z Lab at UC Berkeley. DSpark comes from DeepSeek. Both work with vLLM and llama.cpp. Both have draft models for the Qwen3.6 models that Dankserver and Dankclaw run.
+DFlash comes from Z Lab at UC Berkeley. DSpark comes from DeepSeek. Both work with vLLM and llama.cpp. Both publish draft models for the Qwen3.5/Qwen3.6 families (table below); what any of them pairs with on this fleet is an era on [[frank-dragoon]] or [[frank-zealot]].
 
 ## How DFlash Works
 
@@ -115,12 +115,12 @@ unconditionally, while **context-backed** types (`draft-mtp`, `draft-dflash`,
 - Internal order is fixed (ngram types first, then drafters); CLI order is
   irrelevant.
 - **A legal chain can still lose on a recurrent model.** `draft-mtp,ngram-mod`
-  on the qwen35 hybrid (frank-dragoon era) measured −15 % prose / −7 %
-  block-copy (2026-10-03): adding the ngram layer logs `recurrent state
-  rollback is not compatible with 'ngram-mod' - disabling rollback support`,
-  and MTP verification depends on that rollback, so acceptance drops on ALL
-  traffic even where the matcher fires. Measure chains per architecture,
-  never by analogy.
+  on the qwen35 hybrid loses on both traffic shapes (2026-10-03): adding the
+  ngram layer logs `recurrent state rollback is not compatible with
+  'ngram-mod' - disabling rollback support`, and MTP verification depends on
+  that rollback, so acceptance drops on ALL traffic even where the matcher
+  fires. The measured size of each loss is [[frank-dragoon]]'s (Laws). Measure
+  chains per architecture, never by analogy.
 - ngram-mod prints NO startup banner — the load log shows only the drafter's
   `adding speculative implementation` line. Never conclude "chain not
   engaged" from the load log; identify behaviourally (`draft_n` > 0 only
@@ -198,7 +198,10 @@ Compatible MTP-only draft GGUFs on Hugging Face:
 
 Speculative decoding is lossless under rejection sampling: the target verifies every draft token. A draft head from the base model works with a derivative target; wrong proposals just lower the acceptance rate. The accepted MTP draft lengths are 2-3 tokens; the "MTP Defaults Are a Trap" writeup (thefrontierlab.ai) documents the tuning pitfalls.
 
-On Dankclaw, `draft-dflash` remains the benchmarked winner (20.2 t/s at 4k vs 16.9 for DFlash+ngram and 8.7 for ngram-only). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see [[deployment]]).
+On the mid-August 2026 Dankclaw shape `draft-dflash` was the benchmarked winner
+at 4k — 20.2 t/s, against 16.9 with ngram layered on and 8.7 ngram-only; those
+three figures are that retired llama.cpp era's property ([[frank-dragoon]] owns
+the era and its shape). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see [[deployment]]).
 
 ## Draft Models
 
@@ -213,13 +216,12 @@ DFlash draft model sizes for the models on these servers:
 
 The DFlash draft is not a standalone model. It must pair with its target model. The model cards recommend block size 8 for higher concurrency and 16 for single-stream throughput.
 
-## Deployment on Dankserver (NVIDIA RTX 3090)
+## DFlash on an RTX 3090 (Ampere) — the archived vLLM path
 
-The Dankserver llama.cpp deployment (`frank-zealot`, Qwen3.6-35B-A3B) is **active** on Dankserver (port 7770) as of the 2026-08-17 redo (the earlier 2026-08-16/17 retirement of this service was reversed when the PP split was retired). The vLLM material below documents the archived AWQ/DFlash path on the same hardware.
-
-> **DFlash retired from active service.** `DFlash` (draft-dflash) was tested initially but did not deliver the desired speed gains under the `frank-zealot` workload (see ngram-mod benchmarks below). The `Alittlehammmer/Qwen3.6-35B-A3B-DFlash` draft GGUF remains in `/home/dank/models/` as reference material only.
-
-The vLLM material below documents the archived AWQ/DFlash path on the same hardware. The RTX 3090 is Ampere (compute capability 8.6). Use these conditions:
+DFlash was tried on the 3090 line, lost there to the ngram chain, and was never
+kept deployed; that card's payloads, shapes and rates are [[frank-zealot]]'s
+(its Eras rows are the llama.cpp arms the draft was tested against). What
+transfers is the Ampere arithmetic — compute capability 8.6, so:
 
 - FP8 KV cache is valid on CUDA 12.
 - FP8 weights compute through a fallback path; no native FP8 tensor cores.
@@ -248,11 +250,15 @@ docker run --rm --name vllm-qwen36-dflash \
     --gpu-memory-utilization 0.95
 ```
 
-## Deployment on Dankclaw (AMD Strix Halo gfx1151)
+## DFlash and DSpark on Strix Halo (gfx1151) — what killed them
 
-`frank-dragoon` on Dankclaw ran the llama.cpp pipeline-parallel head (Qwen3.8-27B-Uncensored Q6_K) on port 7770, RPC-split with the Dankserver worker. Spec was the standalone MTP draft over the ngram-mod fallback, 1 slot at 262144 ctx, q8_0/f16 KV, head GPU side ROCm0. This page is a historical record of that era (mid-August 2026); for the current deployment see [[kingjones-strix-rocmfp4]].
-
-History: before `--spec-default`, the box ran DSpark (`draft-dspark`, retired 2026-08-15 after a ROCm illegal-memory-access crash under two concurrent 16k prompts) and before that bundled MTP (`draft-mtp`, retired same day). DFlash was retired on 2026-08-14 (premature-EOS under parallel load). The old DFlash draft GGUF (`Qwen3.6-27B-DFlash-Q8_0.gguf`), the MTP-only drafter (`Qwen3.6-27B-MTP-ONLY-Q8_0.gguf`), and the DSpark draft (`Qwen3.8-27B-DSpark-Q8_0.gguf`) remain in `/home/dank/models/` as reference only.
+Both context-backed drafters died the same way on the iGPU: under parallel load.
+DSpark took `ROCm error: an illegal memory access` inside the shared draft
+context with two concurrent 16k prompts; DFlash corrupted multi-slot into
+premature EOS and was retired first. The timeline, the crash class, the
+"concurrency is the trigger, not size" finding and the `PARALLEL=1` workaround
+are on [[qwen3.8-spec-matrix]]; the shape they were measured on is
+[[frank-dragoon]]'s mid-August 2026 era.
 
 ## A/B protocol on a live llama.cpp endpoint
 
@@ -315,7 +321,7 @@ not switch without re-measuring".
 
 The A/B vehicle for ANY single-GPU llama.cpp knob (batch/ubatch, KV type,
 slot split, quant rung — not just spec flags) is the transient systemd
-drop-in procedure in [[frank-dragoon-benchmark]] and [[frank-zealot-benchmark]].
+drop-in procedure in [[frank-dragoon]] and [[frank-zealot]].
 
 ## Is it worth it? (arithmetic before deployment)
 
@@ -359,16 +365,18 @@ pay for K sequential forwards.
 Trained heads are target-specific: EAGLE/DFlash consume the TARGET's hidden
 states, so a dense-27B DFlash2 is useless for a 512-expert MoE trunk.
 
-**Step 4 — acceptance reality checks.** Fleet datum: a trained z-lab DFlash2
-at depth 7 accepted ~41 % on its matched target (`draft-dflash 7 → 43.5
-tok/s`); treat 40–60 % as the honest band for a matched pair. Cheapest proxy
-for `p` at temp 0 when both models are greedy and share a tokenizer: greedy
-continuation agreement on identical prompts, two API calls, no weights moved.
-llama.cpp can also synthesize the projection without any draft artifact:
+**Step 4 — acceptance reality checks.** Treat 40–60 % as the honest band for a
+matched pair; the arm that anchors the bottom of it on this fleet is the
+depth-7 `draft-dflash` row of [[frank-medium-llamacpp-dflash2]]'s spec table
+(accepted share, rate and VRAM for every depth are in that table). Cheapest
+proxy for `p` at temp 0 when both models are greedy and share a tokenizer:
+greedy continuation agreement on identical prompts, two API calls, no weights
+moved. llama.cpp can also synthesize the projection without any draft artifact:
 `--spec-synth-rates P0,P1,...` / `--spec-synth-len L`. Speculation pays IN
-PROPORTION TO HOW SLOW THE TARGET IS — the same 41 % gave 2.3x on an
-18.8 tok/s layer-split target and a loss on a 25 tok/s one; fast targets are
-usually sparse MoE, the worst case for drafting.
+PROPORTION TO HOW SLOW THE TARGET IS — that band's midpoint was worth 2.3× on
+the slow two-card layer-split target and a LOSS on the 25 tok/s target of the
+worked example above; fast targets are usually sparse MoE, the worst case for
+drafting.
 
 **Step 5 — check the attachment surface BEFORE the arithmetic.** Closed
 engines refuse external drafters by design: halogen-flash-server's
@@ -403,7 +411,7 @@ measurement that would settle it.
 
 - Speculative decoding helps low-QPS, memory-bound, latency-focused workloads. It helps less at high QPS.
 - DFlash needs a non-causal attention backend; the ROCm flash_attn backend must be validated before production use.
-- The old `--swap-space` flag no longer exists in current vLLM. Use `--kv-offloading-size`.
+- The old `--swap-space` flag is retired; the live CPU-KV-offload knob pair is owned by [[RAM CPU Offloading in VLLM]].
 - `--speculative-config` is the preferred CLI. Legacy `--spec-method`, `--spec-model`, and `--spec-tokens` flags still exist.
 - Use `rejection_sample_method` default `strict` for lossless output.
 
@@ -419,4 +427,4 @@ measurement that would settle it.
 - [[GGUF Compatibility]] — header parsing and custom tensor-type lock-in
 - [[Llama Cpp Serving Sizing]] — slot-pool and drafter KV sizing arithmetic
 - [[Llama Cpp Multi-Device Splits]] — where the drafter's KV actually lands
-- [[frank-dragoon-benchmark]] and [[frank-zealot-benchmark]] — the single-GPU env drop-in A/B vehicle
+- [[frank-dragoon]] and [[frank-zealot]] — the single-GPU env drop-in A/B vehicle

serverAppConfigs-docs/LLM/Dankclaw Improvements.md

@@ -17,66 +17,59 @@ Dates: researched 2026-08. Hardware is verified on the live machine unless
 marked "verify first". Re-verified 2026-08-15/16; corrections marked below.
 
 **Live today (2026-10-18).** The era described below — llama.cpp on the 8060S
-iGPU behind :7770 — is retired. Dankclaw now serves
-[[frank-dragoon-flash-server-v2-native]] (halogen-flash-server 0.16.2, port
-7770) and [[frank-zealot-llama-prism-bonsai2-pq20]] (llama.cpp PrismML fork on
-the RTX 3090, port 7771 — drift today: that unit is `failed` since 2026-10-18
-03:40 and a hand-run 9B-MTP process answers the port), and boots `iommu=pt` since 2026-10-18
-([[dankclaw-iommu-translation-off]]). The numbers below are the measurements of
-the era they were taken in; the `ttm` ceiling and the NVMe scheduler rows were
-re-checked 2026-10-18 and still hold.
+iGPU — is retired; what runs on this box now (both endpoints, their engines and
+ports) is [[frank-dragoon]]'s and [[frank-zealot]]'s pages, and both run under their
+repo units today — no drift open as of 2026-10-19 (`frank-zealot` active since
+2026-10-18 21:06, the deployer's 2-slot reslice; `frank-dragoon` active). The OS facts
+are this page's: `iommu=pt` since 2026-10-18 ([[dankclaw-iommu-translation-off]]), and
+the `ttm` ceiling and NVMe scheduler rows below were re-checked 2026-10-18 and
+still hold. Every benchmark number below is a measurement of the era it was
+taken in.
 
 ## Corrections (re-verified 2026-08-15/16)
 
-- **Backend**: that era's production was **Vulkan0**, not ROCm. Measured on this box
-  (b10433): Vulkan 25.13 t/s vs ROCm 24.33 t/s decode at 16k; at 262k ctx
-  Vulkan decodes 48% faster (5.68 vs 3.83 t/s). ROCm wins prefill (410 vs
-  302 t/s @64k). The "HIP out-performs Vulkan for decode" claim did not
-  hold on the 8060S. See [[frank-dragoon-benchmark]].
+- **Backend**: that era's production was **Vulkan0**, not ROCm. On this box ROCm
+  wins prefill and Vulkan wins decode, and the decode gap widens with context:
+  the 64k A/B is [[qwen3.8-strix-optimizations]] §4, the 262k A/B is
+  [[qwen3.8-spec-matrix]] §High-Context Backend Comparison. The "HIP
+  out-performs Vulkan for decode" claim did not hold on the 8060S.
 - **Power profile**: the `/sys/class/drm/card0/device/power_profile` file
   does not exist on this machine (checked 2026-08-15). GPU DPM is already
   `performance`, forced level `auto`. This item is N/A.
-- **KV cache**: production ran `q8_0` K **and** `q8_0` V (both quantized). The law
-  still holds on `frank-zealot` (`CACHE_TYPE_K=q8_0`, `CACHE_TYPE_V=q8_0` —
-  `scripts/frank-zealot.env:55-56`).
+- **KV cache**: production ran `q8_0` K **and** `q8_0` V (both quantized). What
+  `frank-zealot` runs now is its own Current shape — this row is not it.
 - **Spec**: DFlash/DSpark are retired. That era's production was `--spec-default`
-  (ngram-mod) with draft-mtp first. See [[qwen3.8-spec-matrix]]. Today `frank-zealot`
-  is configured with speculation **off** (`SPEC_TYPE=none` — `scripts/frank-zealot.env:67`).
-- **TTM**: raised to 114 GiB on 2026-08-25, set live (no reboot) and
-  persisted in `/etc/modprobe.d/ttm.conf`. The effective VRAM ceiling that
-  once held the service at ~15 GiB is gone.
+  (ngram-mod) with draft-mtp first — timeline on [[qwen3.8-spec-matrix]]; the
+  spec setting each endpoint runs today is on its own page.
+- **TTM**: the `pages_limit` raise that removed the ~15 GiB ceiling is recorded
+  once on [[Dankclaw System Specs]] (Memory).
 
 ## Verified Memory Picture (measured 2026-08-11)
 
 | Figure | Value | Source |
 | ------ | ----- | ------ |
 | OS physical RAM | 30.5 GiB | `/proc/meminfo` |
-| TTM pages_limit | 114 GiB (29,883,536 x 4KiB) — raised 2026-08-25 | `/sys/module/ttm/parameters/pages_limit` |
 | ROCm VRAM total | 96.0 GiB | `rocm-smi --showmeminfo vram` under the then firmware VGM carve |
 | ROCm VRAM used | 76.2 GiB | `rocm-smi` while serving 3x200700 |
 
 That 2026-08 reading is the firmware-VGM era: the BIOS carved ~96 GiB as
 "dedicated" iGPU VRAM, so the kernel saw only ~30 GiB (the same split is
-recorded in [[ling-3.0-flash-trial]] §Why Vulkan + GTT). The OS now accounts
-**121 GiB** (`MemTotal 127108080 kB`, 2026-10-18) and `rocm-smi
---showmeminfo vram` reports **2.0 GiB** (2,147,483,648 B) of dedicated VRAM —
-the carve is gone and the GPU budget is the GTT aperture, not the SMI figure
-([[Dankclaw System Specs]]).
-The TTM row above is still live: `pages_limit = 29883536` (114 GiB) and
-`/etc/modprobe.d/ttm.conf` still carries the `options ttm` line (both read
-2026-10-18).
+recorded in [[ling-3.0-flash-trial]] §Why Vulkan + GTT; the 76.2 GiB row is an
+era [[frank-dragoon]] load reading). Every live number on this box — `MemTotal`,
+the ~114 GiB GTT aperture that is the real GPU budget, the 2.0 GiB dedicated
+carve and the `ttm` page cap — belongs to [[Dankclaw System Specs]], re-read
+2026-10-18.
 
 This split is the Strix Halo GTT/VGM design. It drives the batch/ubatch memory
-limits documented in [[frank-dragoon-benchmark]].
+limits documented in [[frank-dragoon]].
 
 ## 1. Self-Hosted Agentic Tools
 
-All of these connect to the local OpenAI-compatible endpoint
-(`http://127.0.0.1:7770/v1`). Today that port is **halogen-flash-server**, not a
-llama.cpp server, and the address must be the IPv4 literal: `curl
-http://localhost:7770/v1/models` returns nothing while
-`http://127.0.0.1:7770/v1/models` answers 200 (checked 2026-10-18 — the rootless
-podman publish is IPv4-only). None need NVIDIA.
+All of these connect to the local OpenAI-compatible endpoint on
+`:7770` (which engine owns that port is [[frank-dragoon]]'s to say), and the
+address must be the IPv4 literal: `curl http://localhost:7770/v1/models` returns
+nothing while `http://127.0.0.1:7770/v1/models` answers 200 (checked 2026-10-18
+— the rootless podman publish is IPv4-only). None need NVIDIA.
 
 | Tool | Use | Endpoint config | Priority |
 | ---- | --- | --------------- | -------- |
@@ -90,12 +83,13 @@ podman publish is IPv4-only). None need NVIDIA.
 | Qdrant | Vector store for RAG | Standalone, CPU-native | Low |
 
 The pragmatic first installs are Open WebUI, Cline, and Aider. They all work
-with the endpoint on 7770 and need no extra GPU. The 7770 endpoint today is
-halogen-flash-server, not llama.cpp.
+with the endpoint on 7770 and need no extra GPU.
 
 ## 2. Inference Runtime Tuning (llama.cpp on gfx1151)
 
-Verified against the running service and llama.cpp source.
+Verified against that era's running service and llama.cpp source; the era is
+retired ([[frank-dragoon]]), so this table is history and a method, not a live
+TODO list.
 
 | Change | Recommendation | Priority |
 | ------ | -------------- | -------- |
@@ -104,8 +98,8 @@ Verified against the running service and llama.cpp source.
 | Flash attention | Leave `--flash-attn auto`. It helps prompt processing, not decode. | Keep |
 | Load mode | Start with mmap (default). Enable `--mlock` only if page-fault stutter appears. | Optional |
 | Threads | Use `--threads 12 --threads-batch 16 --threads-http 1` for 16 Zen5 cores + 3 slots. Frees cores for the ROCm driver. | Medium |
-| DFlash n_max | Current 8 (block 16) is high. Try `--spec-draft-n-max 5`; it may reduce rejection cost at the same throughput. | Verify |
-| ngram layering | Measured slower than DFlash alone at 4k (16.9 vs 20.2 t/s). Not enabled. | Keep off |
+| DFlash n_max | Depth 8 (block 16) was high; a shallower draft cuts rejection cost at the same throughput. Sweep depth per the A/B protocol in [[DFlash DSpark Speculative Decoding]]. | Verify |
+| ngram layering | Layering ngram on the DFlash draft measured slower than the draft alone at 4k — figures on [[DFlash DSpark Speculative Decoding]]. | Keep off |
 
 ## 3. Server / OS / Hardware Settings
 
@@ -113,7 +107,7 @@ All require root and a reboot for the modprobe change.
 
 | Setting | Action | Effect | Priority |
 | ------- | ------ | ------ | -------- |
-| TTM GPU pool | `/etc/modprobe.d/ttm.conf` → `options ttm pages_limit=29883536 page_pool_size=29883536` (applied 2026-08-25, set live then persisted) | Lifts the 15 GiB cap to 114 GiB; the full 262k KV + Q8 AGENT payload fit in one slot | Done |
+| TTM GPU pool | Raised 2026-08-25; the knob, its value and its persistence file are [[Dankclaw System Specs]]'s (Memory row) | Lifted the ~15 GiB kernel-page ceiling so a full 262k KV plus a ~30 GB payload fits one slot | Done |
 | Power profile | **N/A on this machine** (2026-08-15; re-checked 2026-10-18: no `power_profile` under `/sys/class/drm/card0/device/` or `card1/device/`); GPU DPM already `performance` | none | ~~Medium~~ |
 | NVMe scheduler | Already `none` (`[none] mq-deadline` in `/sys/block/nvme0n1/queue/scheduler`, re-read 2026-10-18) | No change | Done |
 | BIOS VGM | Set Variable Graphics Memory / UMA to "High" in the EVO-X2 BIOS | Max dedicated VRAM for the GPU carve-out; verify in the live BIOS | Verify |
@@ -130,7 +124,7 @@ All require root and a reboot for the modprobe change.
 
 ## Sources
 
-- [[frank-dragoon-benchmark]] — measured backend/spec numbers
+- [[frank-dragoon]] — measured backend/spec numbers
 - [[deployment]] — deploy and update process
 - AMD ROCm RDNA3.5 system optimization guide (gfx1151 memory/BIOS)
 - llama.cpp source: `ggml/CMakeLists.txt`, `ggml/src/ggml-hip/CMakeLists.txt`,
@@ -139,7 +133,7 @@ All require root and a reboot for the modprobe change.
 
 ## See Also
 
-- [[frank-dragoon-benchmark]]
+- [[frank-dragoon]]
 - [[DFlash DSpark Speculative Decoding]]
 - [[deployment]]
 - [[index]]

serverAppConfigs-docs/LLM/Dankclaw System Specs.md

@@ -10,7 +10,7 @@ timestamp: 2026-09-03
 
 ## Server Overview
 
-Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp inference through 2026-08-22 ([[ciru-activefpx-promptforge]], retired). The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool. Today it serves [[frank-dragoon-flash-server-v2-native]] (halogen-flash-server 0.16.2 on the iGPU, port 7770) and [[frank-zealot-llama-prism-bonsai2-pq20]] (llama.cpp PrismML fork on the RTX 3090, port 7771).
+Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp inference through 2026-08-22 ([[frank-dragoon]], retired). The host runs Ubuntu 26.04 with an AMD Ryzen AI MAX+ 395 Strix Halo APU. The APU uses unified memory — CPU and GPU share the same physical RAM pool. Today it serves [[frank-dragoon]] (halogen-flash-server 0.16.2 on the iGPU, port 7770) and [[frank-zealot]] (llama.cpp PrismML fork on the RTX 3090, port 7771).
 ## Hardware Specifications
 
 ### CPU
@@ -46,7 +46,7 @@ Dankclaw is an AMD APU + RTX 3090 inference host. It ran CIRU ROCmFPX llama.cpp
 | Runtime | None installed. FastFlowLM 0.9.45 + XRT (installed 2026-08-16) was **retired 2026-09-11** with the `amd_iommu=off` decision — see [[dankclaw-iommu-translation-off]]; the package remains at `/home/dank/GitPackages/fastflowlm_0.9.45_ubuntu26.04_amd64.deb` |
 | Status | Not in service — no NPU endpoint is deployed. The device itself is usable since the 2026-10-18 `iommu=pt` boot (`/dev/accel/accel0` present, `crw-rw---- root:render 261,0`); Gitea issue #3 tracks enablement. See [[FastFlowLM NPU Inference]] |
 
-The NPU endpoint is retired (2026-09-11; [[frank-npu]]). The `frank-zealot` name now serves **Ternary-Bonsai-2-27B-PQ2_0** on the RTX 3090 via the llama.cpp PrismML fork (see [[frank-zealot-llama-prism-bonsai2-pq20]]); the Gemma4-12B QAT rung it once pointed at is retired history ([[frank-zealot-gemma4-12b]], 2026-09-17 → 10-01).
+The NPU endpoint is retired (2026-09-11; [[frank-npu]]). The `frank-zealot` name now serves **Ternary-Bonsai-2-27B-PQ2_0** on the RTX 3090 via the llama.cpp PrismML fork (see [[frank-zealot]]); the Gemma4-12B QAT rung it once pointed at is retired history ([[frank-zealot]], 2026-09-17 → 10-01).
 
 ### ROCm
 
@@ -66,6 +66,7 @@ The NPU endpoint is retired (2026-09-11; [[frank-npu]]). The `frank-zealot` name
 | System RAM | 121 GiB (shared CPU + GPU; `free -g` total 121, `MemTotal 127108080 kB`, 2026-10-18) |
 | GPU-Visible (ROCm) | ~114 GiB GTT aperture (`mem_info_gtt_total` 122,402,963,456 B — the budget) |
 | Swap | 64 GB `/swap.img` (`swapon`, 2026-10-18; `free` reports 63 GiB) |
+| TTM page cap | `ttm.pages_limit = 29883536` (114 GiB of 4 KiB pages) and `page_pool_size=29883536`, persisted in `/etc/modprobe.d/ttm.conf`, applied live 2026-08-25 and re-read 2026-10-18 (`/sys/module/ttm/parameters/pages_limit`). The kernel default is half of RAM, so in the firmware-VGM era (~30 GiB visible) the cap was ~15 GiB and a 28 GB payload plus a 262k KV cache could not load — this is the row every other page should cite |
 | /dev/shm | 61 GiB tmpfs (half of RAM, `df /dev/shm`, 2026-10-18) |
 
 ### Storage
@@ -95,21 +96,21 @@ cable between claw and server is required for 10G.
 | Kernel | 7.0.0-38-generic (`uname -r`, 2026-10-18) |
 | Boot line | `iommu=pt` since 2026-10-18 (`/proc/cmdline`, no `amd_iommu` token); eras before: `amd_iommu=off` 2026-09-11 → 10-18, IOMMU default-on before that — [[dankclaw-iommu-translation-off]] |
 | Python | 3.14.4 (system, `python3 --version`, 2026-10-18). The 3.12.13 CIRU venv left with that runtime (retired 2026-08-22) |
-| CIRU runtime | ciru-ai/ROCmFPX — **retired 2026-08-22** ([[ciru-activefpx-promptforge]]); not the serving runtime |
+| CIRU runtime | ciru-ai/ROCmFPX — **retired 2026-08-22** ([[frank-dragoon]]); not the serving runtime |
 | uv | 0.12.8 (`~/.local/bin/uv --version`, 2026-10-18) |
 
 ## Runtime Relevance
 
 - Unified memory eliminates separate VRAM limits. The GPU uses system RAM.
 - The GPU budget is the GTT aperture: ~114 GiB (`mem_info_gtt_total` in `/sys/class/drm/card1/device/` = 122,402,963,456 B; `card1` is the amdgpu iGPU, `PCI_ID=1002:1586`; `llama-server --list-devices` reports 116,732 MiB). rocm-smi's smaller SVM/dedicated carve-out is **not** the budget — the 2026-08 "64 GiB rocm-smi" lesson is stale. Do not use `free` or `dmidecode` for the GPU budget.
-- The active services are `frank-dragoon` on 7770 — **halogen-flash-server 0.16.2** serving the native v2 abliterated `.hgn` in a rootless podman container ([[frank-dragoon-flash-server-v2-native]]) — and `frank-zealot` on 7771 ([[frank-zealot-llama-prism-bonsai2-pq20]]). The gufo v0.1.1 / UD-Q8_K_XL runtime this page once described is retired (2026-09-28 → 10-01, [[frank-dragoon-gufo-qwen38-27b]]); the gufo build tree is still on disk at `/home/dank/GitPackages/gufo-v0.1.1/`.
+- The active services are `frank-dragoon` on 7770 — **halogen-flash-server 0.16.2** serving the native v2 abliterated `.hgn` in a rootless podman container ([[frank-dragoon]]) — and `frank-zealot` on 7771 ([[frank-zealot]]). The gufo v0.1.1 / UD-Q8_K_XL runtime this page once described is retired (2026-09-28 → 10-01, [[frank-dragoon]]); the gufo build tree is still on disk at `/home/dank/GitPackages/gufo-v0.1.1/`.
 - The gufo-era note that the active runtime links against the host ROCm (`/opt/rocm` → `/etc/alternatives/rocm` → 7.2.4) through its RPATH belongs to that retired source build. The serving runtimes do not depend on the host ROCm that way: halogen runs inside its digest-pinned image, and `frank-zealot` is a CUDA build on the 3090. The retired CIRU venv's TheRock 7.15 SDK (`LD_LIBRARY_PATH`) was never in the serving path either.
 - The retired vLLM venvs (`/home/dank/vllm-venv`, `/home/dank/vllm-vk-venv`) were moved to `/home/dank/.AGENTTRASH/` on 2026-08-18.
 - 424 GB free on `/` (`df -h`, 2026-10-18) is ample for model storage.
 
 ## See Also
 
-- [[ciru-activefpx-promptforge]]
+- [[frank-dragoon]]
 - [[Shared Memory Architecture Dankclaw]]
 - [[frank-redo-2026-08-17]]
 - [[FastFlowLM NPU Inference]]

serverAppConfigs-docs/LLM/Dankserver System Specs.md

@@ -1,7 +1,7 @@
 ---
-type: configuration
+type: reference
 title: Dankserver System Specs
-description: Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI PRO R9700 gfx1201 on CPU lanes x8/x8) serving frank-medium.
+description: "Dankserver hardware and OS state since the board swap: board, lane topology, cards, memory, kernel posture."
 tags:
   - dankserver
   - hardware
@@ -16,7 +16,7 @@ timestamp: 2026-09-20
 ## Server Overview
 
 Dankserver is a media/library server and the `frank-archon` LLM host (radiance
-1.0.13, 2×R9700 tensor-parallel — [[frank-archon-radiance-qwen38-iq4r]]; the
+1.0.13, 2×R9700 tensor-parallel — [[frank-archon]]; the
 `frank-medium` name this page carried was retired 2026-09-20, see §Active
 Services). The host runs Ubuntu 26.04 (Resolute Raccoon). The RTX 3090 that
 served the FreeToken/CUDA `frank-zealot` era was removed on 2026-09-15 and
@@ -101,7 +101,7 @@ RCCL-backed rebuild needs no new host packages.
 | Kernel | 7.0.0-38-generic, `iommu=pt` (`uname -r`, `/proc/cmdline`, read 2026-10-18) |
 | Python | 3.14.4 |
 | ROCm userspace | Ubuntu 7.1 packages in /usr (hipcc 7.1.1, rocminfo, rocm-smi 7.1.1) |
-| Serving stack | radiance 1.0.13 container (`stilldeadcode/radiance@sha256:4c3f484564…`), TP=2 across both R9700s — [[frank-archon-radiance-qwen38-iq4r]]. The llama.cpp `fb27a525` build at `~/GitPackages/llama.cpp-dflash2/build-rdna4-rocm714` is the retired frank-medium artifact |
+| Serving stack | radiance 1.0.13 container (`stilldeadcode/radiance@sha256:4c3f484564…`), TP=2 across both R9700s — [[frank-archon]]. The llama.cpp `fb27a525` build at `~/GitPackages/llama.cpp-dflash2/build-rdna4-rocm714` is the retired frank-medium artifact |
 
 The stale AMD-repo ROCm 6.3.3 packages (167, `~24.04`-suffixed) that used to
 hijack `libhsa-runtime64` were purged on 2026-09-15 — see
@@ -113,7 +113,7 @@ The active inference service is `frank-archon.service` (`systemctl is-active`
 `active`, `is-enabled` → `enabled`, read 2026-10-18): radiance 1.0.13 in a
 host-network podman container answering `curl 127.0.0.1:7770/health` →
 `{"status":"ok","model":"frank-archon",…}`. See
-[[frank-archon-radiance-qwen38-iq4r]].
+[[frank-archon]].
 
 The services this page once called active are gone: `frank-medium.service`
 (llama-server, 32:33 layer split + DFlash2 drafter) is `not-found` — that rung
@@ -146,7 +146,7 @@ unverified. See [[deployment]].
 
 ## See Also
 
-- [[frank-archon-radiance-qwen38-iq4r]] — the active serving runbook
+- [[frank-archon]] — the active serving runbook
 - [[Dual-R9700-Cooperation]] — why the PCIe path matters, and what the swap changed
 - [[deployment]] — the deploy flow
 - [[Shared Memory Architecture Dankclaw]] — for contrast, Dankclaw's unified memory

serverAppConfigs-docs/LLM/Dual-R9700-Cooperation.md

@@ -1,7 +1,7 @@
 ---
-type: research
+type: reference
 title: Dual R9700 Cooperation
-description: Why frank-medium's two Radeon AI PRO R9700s could not run tensor-parallel on the old B650-E board (PCIe atomic routing), what the 2026-09-20 ProArt B850-Creator swap unblocked, and what still has to be probed.
+description: "Why two R9700s could not run tensor-parallel on the old board, and what the board swap unblocked."
 tags:
   - frank-medium
   - rocm
@@ -16,7 +16,7 @@ timestamp: 2026-09-16
 
 Dankserver's two R9700s run **one tensor-parallel engine across both cards** —
 today that is `frank-archon` under radiance 1.0.13, TP=2
-([[frank-archon-radiance-qwen38-iq4r]]). When this page was written the endpoint
+([[frank-archon]]). When this page was written the endpoint
 was `frank-medium`: first llama.cpp's layer split
 ([[frank-medium-llamacpp-dflash2]], 2026-09-16 → 09-20), then from 2026-09-20
 one vLLM-radiance process at `tensor_parallel_size 2`
@@ -51,7 +51,7 @@ What that bought:
   decode and 3.6x at 4-way concurrency, and it replaced the layer split on the day
   of measurement. The payload has moved since: the MXFP4 `frank-medium` shape is
   retired 2026-09-26, and what runs today is the iq4r publisher under radiance
-  1.0.13 ([[frank-archon-radiance-qwen38-iq4r]]).
+  1.0.13 ([[frank-archon]]).
 - **The drafter, not the collective, is what llama.cpp lost.** Its tensor mode
   could not carry the DFlash2 drafter, which is the honest caveat on the
   comparison below.
@@ -147,10 +147,10 @@ LAN/SATA/WiFi:
   they were in separate IOMMU groups (15 and 20 — re-read 2026-10-18: still 15
   and 20), and both root ports had ACS `ReqRedir+`, so this page inferred P2P
   would hairpin at the root complex. **That inference is superseded:** the live
-  engine says the opposite about the same pair — "I device: the 2 cards reach
-  each other's memory directly (peer to peer)" (`podman logs frank-archon`,
-  2026-10-15 boot). Plain P2P traffic measured 45.27 GB/s bidirectional on the
-  *old* board — DMA was never the gate, atomics were.
+  engine reports the two cards reaching each other's memory directly, which
+  [[frank-archon]] and [[Dankserver System Specs]] record. Plain P2P traffic
+  measured 45.27 GB/s bidirectional on the *old* board — DMA was never the gate,
+  atomics were.
 - Kernel 7.0.0-31-generic with `iommu=pt` then; this host runs 7.0.0-38-generic
   with `iommu=pt` today (`/proc/cmdline`, read 2026-10-18). Both were ruled out
   as causes ([above](#ruled-out-empirically-do-not-re-derive)) and this host's
@@ -163,10 +163,9 @@ launched since the swap. Until the 2-rank probe runs — or a `--split-mode row`
 boot that does *not* log `internal AllReduce init failed ... falling back to
 meta-backend butterfly` — "collectives work here" was a strong expectation, not a
 measurement. **RCCL never got that measurement**; what settled the operational
-question instead is production traffic: radiance's TP=2 wire moves cross-rank
-all-reduces over the direct P2P path and reports its own precision — "tp wire:
-LOSSY (--tp-wire wht6) at 128 KiB a message and above … Steps of at most 25 rows
-are exact" (`podman logs frank-archon`, 2026-10-15 boot).
+question instead is production traffic — radiance's TP=2 wire carries cross-rank
+all-reduces over the direct P2P path and prints its own precision, and the
+measured `wht6` banner with its exact-step bound is on [[frank-archon]].
 
 ## Measured matrix
 
@@ -233,35 +232,34 @@ landed, at x8/x8, clears the arithmetic with room to spare.
 
 Both cooperation modes were run on this host on 2026-09-20 — the same salted
 10-prompt battery (`scripts/bench-conversation.py`), the service stopped and
-masked between arms. Decode/prefill are geometric means over the ten rows;
-burst-4 is four concurrent copies of one prompt, reported as aggregate tokens/s.
+masked between arms. The arm table is [[frank-medium-radiance-mxfp4]]'s bench of
+record (arms R = radiance TP=2 + DFlash2 as deployed, I = the llama.cpp layer
+split that preceded it, L = llama.cpp `--split-mode tensor` against RCCL); this
+page keeps only the verdict. **TP=2 won on every axis — roughly 2.1× decode and
+3.6× at 4-way concurrency over the layer split — and replaced the layer split the
+day of measurement.**
 
-| Shape | Cross-device path | Decode | TTFT | Prefill | burst-4 |
-|---|---|---|---|---|---|
-| **radiance TP=2 + DFlash2 — deployed** | R4D one-shot allreduce | **95.8 t/s** | **0.21 s** | **1493.6 t/s** | **259.4 t/s (65.6/stream)** |
-| llama.cpp layer split 32:33 + DFlash2 — predecessor | none, boundary activation copy | 45.7 t/s | 0.72 s | ~530 t/s | 71.7 t/s |
-| llama.cpp `--split-mode tensor` 1:1 | RCCL allreduce | 30.5 t/s | 0.71 s | n/r | 54.3 t/s |
-
-Row 3 carries a caveat that no flag can remove: llama.cpp's tensor mode could not
-load the DFlash2 drafter (it aborts at `ggml-backend.cpp:941` — a draft weight
+Arm L carries a caveat no flag can remove: llama.cpp's tensor mode could not load
+the DFlash2 drafter (it aborts at `ggml-backend.cpp:941` — a draft weight
 pre-allocated on the non-primary device has no backend once `n_copies == 1`), so
-it decoded **without** speculation while the other two rows decoded with it. The
-comparison that decides anything is row 1 against row 2, and some of that gap is
-the 4-bit MXFP4 checkpoint rather than the parallelism.
+it decoded **without** speculation while the other arms decoded with it; and part
+of R's margin over I is the 4-bit MXFP4 checkpoint, not the parallelism.
 
 Two numbers elsewhere on this page are therefore not interchangeable:
 
 - The 272 t/s recorded 2026-09-16 is **prefill**, on a **butterfly** allreduce,
-  with RCCL off, on the old board. It was never a TP decode number.
-- 26.3 t/s from the same table is that butterfly tensor's decode. Row 3's 30.5 is
-  an RCCL tensor decode a day later: same shape family, different collective path.
+  with RCCL off, on the old board. It was never a TP decode number, and its
+  paired decode figure is in "The third path" below.
+- The butterfly tensor arm and the RCCL tensor arm a day later are the same shape
+  family on different collective paths; the measured pair lives with
+  [[frank-medium-radiance-mxfp4]]'s arms.
 
 RCCL needs PCIe atomics on *every* participating GPU — the gate the old board
 failed. vLLM's R4D one-shot allreduce dispatches no collective at all, which is
 why TP=2 runs today rather than merely "being unblocked": the image ships TheRock
 RCCL, and TheRock carries no gfx1201 device code (`invalid device function`), so
 radiance could not have used RCCL on this host even with atomics routed. Ubuntu's
-RCCL 2.27 does carry gfx1201 kernels, and that is what row 3 linked against.
+RCCL 2.27 does carry gfx1201 kernels, and that is what arm L linked against.
 
 **Rule for any future multi-GPU collective experiment: stop and mask the
 production service first**, per the benchmarking policy, or run it on a host that
@@ -292,12 +290,11 @@ on the second card, so the atomics gate never comes up.
 
 That is what `frank-medium` ran from 2026-09-16 to 09-20
 ([[frank-medium-llamacpp-dflash2]]): one process, one model, both cards,
-262144 context. Measured against tensor split
-on this host: 1991 t/s prefill / 18.8 tok/s decode layer-split, against 272 /
-26.3 tensor (butterfly fallback, no RCCL). **That tensor number is not a
-prediction for an RCCL-backed build** — it is what tensor split costs when the
-allreduce degrades to a meta-backend butterfly, and it is the number the
-board-swap path exists to retire.
+262144 context. Its measured layer-split rates are that page's; this host's
+tensor arm under the butterfly fallback measured 272 t/s prefill / 26.3 tok/s
+decode. **That tensor number is not a prediction for an RCCL-backed build** — it
+is what tensor split costs when the allreduce degrades to a meta-backend
+butterfly, and it is the number the board-swap path exists to retire.
 
 The ranking therefore changed twice. On 2026-09-16: software already gets both
 cards onto one model, so the hardware fix stopped being a capability question

serverAppConfigs-docs/LLM/FastFlowLM NPU Inference.md

@@ -1,7 +1,7 @@
 ---
-type: research
+type: reference
 title: FastFlowLM NPU Inference
-description: FastFlowLM (FLM), the AMD NPU-first LLM runtime for Ryzen AI XDNA2 chips. Fit analysis for the 50 TOPS NPU on Dankclaw, published benchmarks, verified install state, and a test runbook.
+description: "The AMD NPU-first runtime: fit analysis for the XDNA2, published benchmarks, install state, and test runbook."
 tags:
   - fastflowlm
   - npu
@@ -24,7 +24,7 @@ timestamp: 2026-09-03
 > usable again since the 2026-10-18 `iommu=pt` boot (`amdxdna` bound,
 > `/dev/accel/accel0` present — [[dankclaw-iommu-translation-off]]), but no
 > service runs on it. `frank-zealot` today is the llama.cpp PrismML endpoint on
-> the RTX 3090 ([[frank-zealot-llama-prism-bonsai2-pq20]]). See [[frank-npu]]
+> the RTX 3090 ([[frank-zealot]]). See [[frank-npu]]
 > for the retirement record. Everything below is the 2026-08 research record.
 
 # FastFlowLM NPU Inference
@@ -57,7 +57,10 @@ Key properties, per the project README and docs:
 
 ## NPU Hardware on Dankclaw
 
-Dankclaw runs an AMD Ryzen AI MAX+ 395 (Strix Halo). The chip integrates an XDNA2 NPU rated at 50 TOPS (vendor spec: "NPU TOPS — Up to 50 TOPS", [amd.com product page](https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html)). It sits beside the Radeon 8060S iGPU and shares the same unified memory pool — 30 GiB exposed to the OS in the 2026-08 reading below; the box presents **121 GiB** today (`free -g` total 121, read 2026-10-18).
+Dankclaw runs an AMD Ryzen AI MAX+ 395 (Strix Halo). The chip integrates an XDNA2 NPU
+with 50 TOPS of compute and, crucially, **no dedicated VRAM** - it shares system memory
+via the IOMMU SVA mapping. The budget this page reasons about - 29 GiB usable at
+4800 MT/s LPDDR5X - is the 2026-08 host state; the live figure is [[Dankclaw System Specs]].
 
 Live verification on Dankclaw, 2026-08-16:
 
@@ -89,10 +92,9 @@ Budget math for this fleet:
 | Dankserver (Ryzen 7 7700X) | 62 GB | no NPU |
 | FLM bench box (Kraken Point) | 32 or 96 GB | ~16 or ~48 GB |
 
-2026-10-18 re-read: Dankclaw's OS now reports **121 GiB** (`free -g`) and
-Dankserver **246 GiB** (`free -g`), so the half-of-RAM rule would price the NPU
-budget at ~60 GiB and the 7700X box still has no NPU. The 30 GiB / 62 GB rows
-are the 2026-08-16 state; the table is moot while no NPU endpoint is deployed.
+The half-of-RAM gate scales with host RAM - live figures in [[Dankclaw System Specs]]
+and [[Dankserver System Specs]]. The live gate on Dankclaw is still `flm validate`
+(it owns the current number).
 
 Weight sizes (approximate, Q4-class):
 
@@ -159,7 +161,10 @@ Qwen3.6-35B-A3B (Q4_KS), FLM v0.9.45, Kraken Point **96 GB DRAM**:
 | Decode t/s | 13.65 | 13.41 | 13.09 | 12.51 | 11.24 | 9.51 |
 | Prefill t/s | 78.98 | 118.04 | 156.43 | 197.93 | 218.84 | 221.96 |
 
-Context: the current PP split (head + 3090 worker) decodes at 14-18 t/s for the 27B dense model. A 35B-A3B at ~10-14 t/s on an NPU with 48 GB of RAM budget is not a speed win over the existing topology.
+- PP and NPU together cost 2.2 GiB + NPU model + ~2 GiB of headroom. The then-current
+  PP split (head + 3090 worker, 2026-08 era - see [[frank-redo-2026-08-17]]) decoded at
+  14-18 t/s for the 27B dense model, so an NPU offload would free the head or upgrade
+  the worker's model.
 
 The vendor also claims "over 10x more power-efficient" versus GPU. That is a marketing claim. We have not measured it.
 
@@ -219,7 +224,12 @@ Notes:
 
 ## Promotion (2026-08-16)
 
-`frank-zealot` now runs the NPU endpoint on Dankclaw (was the Dankserver llama.cpp service). Unit: `scripts/frank-zealot.service` (symlinked into `/etc/systemd/system`), wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-zealot.env`. `LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap `FLM_MODEL` in the env to switch.
+`frank-zealot` then ran the NPU endpoint on Dankclaw (2026-08; the name was reused
+twice after this window - see the banner - and the NPU unit today is `frank-npu`,
+masked). Unit: `scripts/frank-zealot.service` (symlinked into `/etc/systemd/system`),
+wrapper `scripts/serve-flm-small.sh`, tunables `scripts/frank-zealot.env`.
+`LimitMEMLOCK=infinity` is set in the unit. Default model: `gemma4-it:e4b`. Swap
+`FLM_MODEL` in the env to switch.
 
 ### Boot policy (2026-08-17)
 
@@ -256,11 +266,13 @@ The 4-stream wall times form a near-perfect 10.7 s ladder. Each stream self-repo
 Consequence: the NPU endpoint is a single lane. It suits low-frequency background work (triage, summarization, agent side-tasks). It does not suit interactive chat for several simultaneous users. The 27B llama.cpp service, by contrast, runs PARALLEL=2 with true concurrent slots.
 
 ## Test Runbook
-
 This runbook targets a retired endpoint; keep it as the procedure if the NPU is
-ever revived. Do not run large model loads while the production service holds
-most of the unified pool — 30 GiB exposed to the OS in the 2026-08 reading,
-**121 GiB** today (`free -g`, read 2026-10-18). Check free RAM first, or stop
+ever revived.
+
+**Host RAM availability.** The gate is half of system RAM by default - a relic of the
+install: the endpoint ran on a machine that presented 30 GiB of system RAM; on today's
+box that ceiling is far larger (live figure: [[Dankclaw System Specs]]). Check free RAM
+first, or stop
 the service the way the bench policy requires (`sudo systemctl stop
 frank-dragoon && sudo systemctl mask frank-dragoon` — self-healing would
 otherwise restart it). The NPU shares the unified pool with everything else on

serverAppConfigs-docs/LLM/FreeToken Builds.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: FreeToken Builds
-description: Building FreeToken (the official franksdank fork of FreeTok Tokenworks' freetoken vLLM-MTP runtime) on Dankclaw — the CUDA vLLM venv build and the ROCm build, with the dependency traps each one hit.
+description: "Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit."
 tags: [freetoken, build, dankclaw, vllm, mtp]
 timestamp: 2026-10-05T00:00:00-07:00
 ---

serverAppConfigs-docs/LLM/GGUF Compatibility.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: GGUF Compatibility and Runtime Lock-in
-description: How to answer "will engine X load this GGUF?" from the file's header alone, why a custom tensor type forces the runtime, and the discipline for pinning a fork build that can actually serve the file.
+description: "Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can."
 tags: [gguf, llama.cpp, forks, quantization, speculative]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -119,5 +119,6 @@ re-download.
 
 - [[Llama Cpp Runtimes]] — which fork ships which custom types
 - [[Llama Cpp Code Diff]] — the enum tables and code-level differences
-- [[frank-dragoon-rocmfpx-q8agent-mtp]] — the ftype-115 lock-in in production
+* [[kingjones-q8agent-rocmfpx]] — the era that served the ftype-115 file, and
+  [[frank-dragoon]] for what runs there now
 - [[Llama Cpp Serving Sizing]] — pricing what a payload costs once it loads

serverAppConfigs-docs/LLM/Heterogeneous Split Triage.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Heterogeneous Split Triage
-description: Triage before building when someone asks to split a model across unequal accelerators on one host (24 GB GDDR6X + LPDDR APU, PCIe x4 link) — link characterisation, the real bottleneck, what placement never buys, UMA pools, drafter splits, and the verdict ranking.
+description: "Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys."
 tags: [heterogeneous, multi-device, llama-cpp, triage, pcie]
 timestamp: 2026-10-05T00:00:00-07:00
 ---

serverAppConfigs-docs/LLM/Hybrid CPU GPU Processing Dankserver.md

@@ -10,7 +10,7 @@ timestamp: 2026-08-28
 
 ## Overview
 
-Dankserver has **no NVIDIA GPU**: the RTX 3090 left the box on 2026-09-15 and the host now runs two AMD Radeon AI PRO R9700s (`lspci | grep -ci nvidia` → 0, read 2026-10-18 — [[Dankserver System Specs]]). This page's vLLM-era hybrid guidance is reference material, not the serving path: the live service is radiance 1.0.13 as `frank-archon`, and its hybrid mechanism is a pinned host pool (43.88 GiB per rank, `scripts/frank-archon.env:184`) plus the CPU-resident 47.68 GiB n-gram/PLE table, not vLLM KV swapping ([[frank-archon-radiance-qwen38-iq4r]]).
+Dankserver has **no NVIDIA GPU**: the RTX 3090 left the box on 2026-09-15 and the host now runs two AMD Radeon AI PRO R9700s (`lspci | grep -ci nvidia` → 0, read 2026-10-18 — [[Dankserver System Specs]]). This page's vLLM-era hybrid guidance is reference material, not the serving path: the live service is radiance 1.0.13 as `frank-archon`, and its hybrid mechanism is a pinned host pool (43.88 GiB per rank, `scripts/frank-archon.env:184`) plus the CPU-resident 47.68 GiB n-gram/PLE table, not vLLM KV swapping ([[frank-archon]]).
 
 ## Architecture
 
@@ -56,7 +56,10 @@ Lower values leave more GPU memory for KV cache. This reduces swap frequency.
 
 ### Swap Space / KV Offload
 
-Recent vLLM releases removed the `--swap-space` engine argument — `CacheConfig` on `main` has no `swap_space` field ([source](https://raw.githubusercontent.com/vllm-project/vllm/main/vllm/config/cache.py), read 2026-10-18). Use `--kv-offloading-size` (`kv_offloading_size`, same file) to move KV cache to CPU RAM. In Podman, mount the host cache read-write for KV offload writes. The table below shows historical `--swap-space` guidance as a sizing reference.
+Current vLLM has no `--swap-space` engine argument, and the CPU KV-offload knob pair
+is owned by [[RAM CPU Offloading in VLLM]] (that page carries the upstream
+`CacheConfig` verification). In Podman, mount the host cache read-write for KV offload
+writes. The table below shows historical `--swap-space` guidance as a sizing reference.
 
 | System RAM | Recommended Swap |
 |------------|------------------|

serverAppConfigs-docs/LLM/LLM Bench Windows and Measurement.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: LLM Bench Windows and Measurement
-description: Stopping-and-restoring benchmark windows on the fleet LLM hosts (llama-bench sweeps, HTTP param benches, multi-arm vLLM offload sweeps), honest benchmark methodology, and the usage-counter measurement law under speculative decoding.
+description: "Opening and restoring benchmark windows on fleet hosts, honest methodology, and the usage-counter law under drafting."
 tags: [benchmarking, llama-bench, ab-testing, speculative, vllm, fleet]
 timestamp: 2026-10-05T00:00:00-07:00
 ---

serverAppConfigs-docs/LLM/LLM Performance Diagnosis.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: LLM Performance Diagnosis
-description: Probing why a served LLM is slow — the flat-depth probe, the expert-offload duty-cycle arithmetic, kernel-time attribution via the profiler window, live telemetry without benchmarking, RAM-bandwidth ceilings, IBS tensor attribution and prefix-cache-aware routing.
+description: "Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution."
 tags: [performance, profiling, moe, pcie, prefix-cache, routing, telemetry]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -20,8 +20,8 @@ Sweep decode across context depth. **Flat** ⇒ attention over history is not
 the cost; the problem is per-token expert movement and KV/attention tuning is
 ruled out entirely. A genuinely context-bound server shows a clear downward
 slope. Record cold vs warm separately — a small gap means the cache is
-working. Measured on the fp8 expert-offload shape: 12.11 tok/s FLAT from 304
-to 16,095 tokens.
+working. The canonical flat run (fp8 expert-offload shape, a few hundred to
+sixteen thousand tokens of context) is in [[Tcclaviger Fork Ops]]'s profile.
 
 ## 2. Duty-cycle arithmetic (the "GPU not fully utilized" answer)
 
@@ -161,7 +161,7 @@ of the same RAM, not extra memory.
 ## 7. IBS model-access attribution (AMD CPUs)
 
 Real CPU access patterns for mmap'd weights, live, no stop (established on
-frank-archon, 2026-09-26, ~2.7M attributed samples in 75 s). Reference:
+[[frank-archon]], 2026-09-26, ~2.7M attributed samples in 75 s). Reference:
 `scripts/frank-archon-expert-activity.py` (self-contained dump + sample +
 analyze; `--analyze-only`).
 

serverAppConfigs-docs/LLM/Llama Cpp Multi-Device Splits.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Llama Cpp Multi-Device Splits
-description: What llama.cpp can and cannot express across devices, the no-collective layer-split path, split balancing by measurement, drafter KV sizing from GGUF headers, and heterogeneous placement triage.
+description: "What llama.cpp can express across devices: the no-collective layer-split path, balancing, and drafter KV sizing."
 tags: [llama.cpp, multi-gpu, layer-split, pcie, sizing, speculative]
 timestamp: 2026-10-05T00:00:00-07:00
 ---

serverAppConfigs-docs/LLM/Llama Cpp Serving Sizing.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Llama Cpp Serving Sizing
-description: Slot-pool sizing against a headroom band, per-device VRAM accounting, prompt-cache/state sizing and the QSA flash-attn fix — the measurement-first procedures for llama.cpp serving shapes.
+description: "Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache."
 tags: [llama.cpp, vram, kv-cache, slots, flash-attn, sizing]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -23,8 +23,9 @@ guesswork.
    each near the target; record `nvidia-smi` used/total plus the boot log's
    `n_slots`/`n_ctx_slot`: shape A = current/cheap, shape B = new demand.
    `non-KV = used − (tokens × KiB/token)`; the two rows separate
-   weights/buffers/graphs from the new demand (a 918 MB projector bills
-   ~1.4 GiB).
+   weights/buffers/graphs from the new demand; a projector is billed after the
+   fit at roughly 1.5x its file size ([[frank-zealot]] Contract has that
+   measurement).
 3. **Solve the band:** `KV_tokens ≤ (card − non-KV − headroom_low) /
    KiB/token`, round the pool DOWN to a multiple of `PARALLEL × 4096` so every
    slot gets an equal integer window. Boot and measure; if outside the band,
@@ -54,8 +55,8 @@ First decide which kind of memory is in question:
 | device | kind | read it with |
 |---|---|---|
 | RTX 3090 (Dankclaw) | discrete GDDR6X | `nvidia-smi --query-gpu=memory.total,memory.used,memory.free` |
-| gfx1151 iGPU (Dankclaw) | **UMA** — "VRAM" is host RAM | `/sys/class/drm/card1/device/mem_info_gtt_{total,used}` (114 GiB) vs `mem_info_vram_*` (2 GiB carve-out) |
-| R9700s (Dankserver) | discrete | `rocm-smi --showmeminfo vram` → 34,208,743,424 B = 32,624 MiB (31.86 GiB) per card, read 2026-10-18 — **not** the 64 GiB "SVM-backed" figure this row once carried ([[Dankserver System Specs]]) |
+| gfx1151 iGPU (Dankclaw) | **UMA** — "VRAM" is host RAM | `/sys/class/drm/card1/device/mem_info_gtt_{total,used}` vs `mem_info_vram_*`; the aperture and carve-out values are [[Dankclaw System Specs]]'s |
+| R9700s (Dankserver) | discrete | `rocm-smi --showmeminfo vram`; the per-card byte/MiB value (and the stale "64 GiB SVM-backed" figure it replaced) is [[Dankserver System Specs]]'s |
 
 Per-process truth on NVIDIA:
 `nvidia-smi --query-compute-apps=pid,process_name,used_memory`. Never add the
@@ -63,9 +64,10 @@ iGPU's GTT to host RAM as a separate pool; `free -g` is the arbiter for
 co-tenants.
 
 - **Engine ledger first.** Halogen prints weights pinned (+ remaining 2 MiB
-  contiguous blocks — the honest fragmentation signal: 240 blocks/12.3 GiB
-  after reboot vs 475/14.7 GiB before means the model did not come back into
-  the same hole), device budget, KV pool/working split, "left for everything
+  contiguous blocks — the honest fragmentation signal: a block census that drops
+  across a restart means the model did not come back into the same hole; the
+  before/after census this was learned on is that era's, on [[frank-dragoon]]),
+  device budget, KV pool/working split, "left for everything
   else". llama.cpp prints them at `llama-model.cpp:1827` (per-backend model buffer)
   and `llama-context.cpp:692` (compute buffer) on the era's build; on today's
   `master` the same `LLAMA_LOG_INFO` lines sit at `llama-model.cpp:2009`
@@ -83,13 +85,13 @@ co-tenants.
   graph/compute pool).
 - **Traps:** mmap'd PLE is reclaimable page cache; `-ot` makes it pinned —
   "move layers to the card" converts reclaimable cache into device memory and
-  frees nothing for co-tenants. A served model's footprint ≠ weight size
-(a retired frank-zealot shape: 22.6 of 24.1 GiB for an 11.29 GiB model; the card
-today reads 24,576 MiB = 24.0 GiB total, `nvidia-smi --query-gpu=memory.total`,
-2026-10-18, and the deployed payload is the PQ2_0 pair
-[[frank-zealot-llama-prism-bonsai2-pq20]]). Do the capacity
-  subtraction before designing: 12 attn layers × 1.62 GiB + 200k f16 KV =
-  28.6 GiB > 24.1 GiB usable → KV-in-VRAM on that pair only reaches ~64k.
+  frees nothing for co-tenants. A served model's footprint is not its weight
+  size: the MiMo Q5_K_M era booted a 6.40 GiB payload to **20,530 MiB used** of the
+  3090's addressable 24,576 — the Eras row and its addressable ≠ advertised law are on
+  [[frank-zealot]]. What the card reports used is weights + KV + compute buffers +
+  projector, never weights alone. Do
+  the capacity subtraction before designing, on the card number the owner page
+  quotes, not the advertised one.
 - **Context lengths: derive, never remember.** `llama-bench` computes
   `n_ctx = n_prompt + n_gen + n_depth` (defaults 512/128/0, type_k/v F16,
   flash_attn AUTO, split_mode LAYER). `-p 512,16384 -n 64` = contexts
@@ -112,10 +114,10 @@ them from the binary (`llama-server --help`) before sizing on them.
 **State law** (compute before trusting any cap): a state is the serialized KV
 of the whole sequence + constant recurrent state + the draft context's KV:
 `per_token = 2 × n_full_attn_layers × n_kv_heads × head_dim × bytes (K and V) +
-draft_per_token`. Measured on frank-medium (qwen3.5 hybrid, f16, MTP draft):
-16×4×256×2×2 = 64 KiB/token + 4 KiB draft = **68 KiB/token**; a full
-196,608-token slot state ≈ 12.8 GiB. Cross-check against the journal's
-`prompt state size` lines — the two must agree.
+draft_per_token`. The worked qwen35-hybrid instance of that formula (full layer
+count, f16 KV, MTP draft) and the slot-size table it prices are on
+[[frank-medium-llamacpp-dflash2]]; cross-check any such arithmetic against the
+journal's `prompt state size` lines — the two must agree.
 
 **The journal is the instrument:** `exceeds cache size limit … skipping` =
 cap binding in production; `making room … removing oldest` / `cache size
@@ -162,13 +164,13 @@ Only the charlie12345/ROCmFPX build-qwen38 (PR #27742 qwen4exp merged) takes
 it; CIRU, upstream and stock builds do not. Validate live: load with
 `--flash-attn on`, health 200, a real completion finishing `"stop"`, journal
 clean of `GGML_ASSERT`/`cudaMalloc`/`missing tensor`, and flash-attn engaged
-in the timings. Context cap for full-model residency on that shape stayed
-32,768.
+in the timings. The context cap that build reached for full-model residency is
+part of that era's record on [[frank-dragoon]].
 
 ## Related
 
 - [[GGUF Compatibility]] — what a payload is before it is priced
-- [[frank-zealot-llama-mimo-q6k-2x200704]] — the two-boot priced shape of the retired MiMo era (live shape today: [[frank-zealot-llama-prism-bonsai2-pq20]])
-- [[frank-medium-llamacpp-dflash2]] — where the 68 KiB/token datum was measured
+- [[frank-zealot]] — the endpoint whose Eras rows are the two-boot priced shapes this method produced
+- [[frank-medium-llamacpp-dflash2]] — the state-law worked example behind section 3
 - [[DFlash DSpark Speculative Decoding]] — the spec A/B protocol on llama.cpp
 - [[Dual-R9700-Cooperation]] / [[x4-pair-experiment]] — the multi-device side

serverAppConfigs-docs/LLM/Model Swap and Rollback.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Model Swap and Rollback
-description: The full model-swap discipline for the LLM fleet — fit preflight, fetch and payload verification, runtime build/flag checks, git-only config, live validation, systemd handoff, close-out, the three-pin revert law, and research-verification discipline.
+description: "The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert."
 tags: [models, swap, gguf, huggingface, rollback, deployment, verification]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -40,9 +40,9 @@ curl -s <health>                                      # then a REAL completion
 ```
 
 Deploy caveat: "healthy after Ns" = systemd ACTIVE, not model LOADED —
-confirm with a real `/v1/chat/completions`. RTX 3090 (Ampere) has no
-FP8/FP4 tensor cores: FP8 weights run via upcast but must still fit; choose
-quant by size fit, then accept the compute path the card supports. Before
+confirm with a real `/v1/chat/completions`. On Ampere an FP8 file buys memory
+and no compute ([[Quantization by Hardware Architecture]]): choose the quant by
+size fit, then accept the compute path the card supports. Before
 choosing a smaller quant, state WHICH of the two you are buying — headroom,
 or a design change (a smaller quant does not change what the runtime can
 express; per-tensor cross-device placement stays refused under
@@ -60,8 +60,8 @@ express; per-tensor cross-device placement stays refused under
 ## 3. Fetch and verify (HF payloads)
 
 **Inventory by QUANT NAME, never by extension** — large quants ship as
-`.partNofM` fragments (Q4_K_M = 37.00+37.00+36.97 GiB), so an
-`endswith(".gguf")` filter makes Q4_K_S look nonexistent. Check both
+`.partNofM` fragments, so an `endswith(".gguf")` filter makes a whole rung look
+nonexistent (AGENTS.md carries the worked fragment ledger). Check both
 mradermacher repos (`…-i1-GGUF` imatrix, `…-GGUF` plain) and the model's own
 repo; the tree API's `blobs=true` gives `lfs.oid` per file. Gated mirrors
 return 403 even with a valid token until the account accepts the gate —
@@ -150,7 +150,7 @@ recoverable, drift is not.
    entry in `serverAppConfigs-docs/log.md`.
 2. `~/.omp/agent/models.yml`: `contextWindow` must equal the REAL per-slot
    context — llama.cpp logs `n_ctx_slot` per prompt (`--parallel N` divides
-   the pool; e.g. 262144 native / 2 slots = 131072 per slot); vLLM reports
+  the pool; e.g. a 524288 pool over 2 slots = 262144 per slot); vLLM reports
    `max-model-len` and its concurrency line. Overselling causes silent
    client-side overflow or a discovered-wall 400. Also `maxTokens` vs the
    engine's cap, `input` capabilities (text/image), and
@@ -252,12 +252,10 @@ small/fast + image roles at the new endpoint; move the vision role WITH the
 vision tower (one owner per capability). `shared/homepage/services.json`: a
 card per endpoint, probed from the monitor's own host. Prometheus: only add
 a job if the runtime actually exposes Prometheus text — and re-measure rather
-than repeat, because that answer moves. This bullet once said halogen serves
-`/health`, `/cache`, `/v1/*` "and nothing else"; measured 2026-10-18 on
-Dankclaw, `curl -s http://127.0.0.1:7770/metrics` returns Prometheus text
-(`# HELP llamacpp:prompt_tokens_total …`, content-type
-`text/plain; version=0.0.4`) while an invented path 404s — so the surface has
-grown and a scrape target is expressible again.
+than repeat, because that answer moves: halogen's HTTP surface has grown twice
+since this page was written, and the endpoint list that matters for a scrape
+target — including whether `/metrics` answers — is [[frank-dragoon]]'s wire
+contract.
 
 **4. Cutover, then verify by driving it.** `deploy.sh <target> --service
 <large> --service <small>` (order matters); per endpoint: `/health`,
@@ -265,10 +263,10 @@ grown and a scrape target is expressible again.
 `reasoning_content`, a `tools` request returns `tool_calls`, an image
 request is answered (or refused with a named reason), the runtime's own
 speculation counters move, and the unit is `enabled` and `active`. All three LLM
-endpoints are **root** units today (`User=dank` + `WantedBy=multi-user.target` —
-`scripts/frank-archon.service:30,71`, `scripts/frank-dragoon.service:8,31`,
-`scripts/frank-zealot.service:8,26`), so linger is not what gates them; linger
-matters for rootless quadlets. Read the engine's boot ledger for a silent clamp. Finally spend one
+endpoints are **root** units today (`scripts/frank-archon.service`,
+`scripts/frank-dragoon.service`, `scripts/frank-zealot.service`), so linger is
+not what gates them; linger matters for rootless quadlets. Read the engine's boot
+ledger for a silent clamp. Finally spend one
 real request through the harness per changed role — routing labels lie; the
 servers' own journals do not.
 
@@ -285,4 +283,5 @@ predecessor "stopped" (the port race above).
 - [[Llama Cpp Serving Sizing]] — pricing the shape once it loads
 - [[RDNA4 MoE Expert Offload VRAM Budget]] — vLLM-side fit arithmetic
 - [[LLM Bench Windows and Measurement]] — measuring the new deployment
-- [[dankserver-qwen38-fp8-expert-offload-tuning]] — the retired tcclaviger vLLM endpoint (2026-09-29 → 10-05); the live Dankserver endpoint is [[frank-archon-radiance-qwen38-iq4r]]
+* [[frank-archon]] — the live Dankserver endpoint; the tcclaviger vLLM era
+  (2026-09-29 → 10-05) is a row in its Eras table

serverAppConfigs-docs/LLM/PCIe Link and Atomics.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: PCIe Link and Atomics
-description: Link probing (idle downshift, path ceilings, CUDA/HIP bandwidth probes), the streaming-vs-compute ceiling arithmetic, and the PCIe-atomics wall that decides whether AMD multi-GPU collectives can exist at all.
+description: "Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives."
 tags: [pcie, rccl, atomics, multi-gpu, bandwidth, amd, nvidia]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -65,13 +65,11 @@ LPDDR5X on the Strix Halo — 128 GB LPDDR5x-8000 on a 256-bit bus,
 streaming loses by construction and no tuning rescues it; doubling the link
 only doubles a losing number.**
 
-Worked example (Dankclaw 2026-09-16): Gen4 x4 measured H2D **6.72 / D2H 6.60
-GB/s** (85 % of spec) — pinned H2D ≈ async, pageable within 1 %, i.e.
-link-limited not staging-limited. A 3 MiB slab (the expert-streaming
-granularity) = 474 µs. A3B MoE at Q4_K_M (~1.70 GiB active/token) → 272
-ms/token ⇒ **3.7 tok/s** (4.6 with 20 % resident) vs measured **20.46 tok/s**
-with experts computed on the CPU, 25–26.5 on the iGPU. The streaming design
-loses 3–5x.
+Worked example: the Dankclaw Gen4 x4 run of 2026-09-16 — measured H2D/D2H
+rates, the 3 MiB slab cost, and the streaming-vs-compute comparison it decided
+— is recorded on [[x4-pair-experiment]] §The link, measured. Read it once and
+apply the rule above: on that link the streaming design lost by 3–5x, and the
+x8 ceiling would only have halved the loss.
 
 ## 3. Measure the link: two probes, same mechanics
 
@@ -150,9 +148,9 @@ int main(int argc, char **argv) {
   (the usual "pageable costs 2x" rule does not apply at this link).
 - Small-transfer penalty (2.345 MiB: ~106 µs vs ~78 µs ideal, ~21 %) is
   per-transfer overhead — recoverable only by BATCHING transfers, not by a
-  faster link. (Dankserver 2026-09-29: bulk 22.7 GB/s at the 2.344 MiB slab =
-  91 % of Gen5 x8, while the engine's effective read path ran ~9.5 GB/s — the
-  gap is the fork's gather software path, not the wire.)
+  faster link. The engine-vs-wire version of that reading on Dankserver
+  (bulk slab rate against the engine's effective gather rate, same
+  granularity) is [[frank-archon]]'s tcclaviger-era record.
 - Both GPUs symmetric ⇒ no cross-device rebalance is available; measure both.
 - Don't quote a derived line from inside a probe without hand-checking it —
   a printf format/argument mismatch in one of these printed a nonsense
@@ -186,7 +184,8 @@ sudo lspci -vv -s <bridge> | grep AtomicOpsCap                   # want Routing+
 `Routing-` is a read-only hardware capability. Typical AM5 layout: one x16
 slot on CPU root ports (atomics fine), the rest behind the chipset (usually
 not). Bandwidth for TP is small anyway: ~10 MB per token-step for a 27B at
-batch 4 ≈ 1.4 ms at Gen4 x4 — compute-bound, not interconnect-bound.
+batch 4 is a millisecond-scale figure at x4 — compute-bound, not
+interconnect-bound (full derivation: [[Dual-R9700-Cooperation]] §Bandwidth).
 
 **Probe matrix (bare 2-rank `torch.distributed`, one process per GPU, 25 s PG
 timeout) — one probe answers several design questions:**
@@ -215,13 +214,12 @@ reporting an outage (`gitea` vs `gitea-server`).
 
 1. CPU-attached M.2 → PCIe x4 riser (~$30) — puts the card on the CPU root
    complex; x4 suffices for TP.
-2. Board whose CPU x16 bifurcates x8/x8 (done 2026-09-20 on dankserver — both
-   R9700s on root ports `00:01.1`/`00:01.3`, every bridge `Routing+`, both
-   endpoints `ReqEn+`). Re-checked 2026-10-18 without root: both root ports
-   report `current_link_width=8` at `32.0 GT/s`
+2. Board whose CPU x16 bifurcates x8/x8 (done 2026-09-20 on dankserver — the
+   per-device chain census, including both endpoints reading `ReqEn+`, is
+   [[Dual-R9700-Cooperation]]'s). Re-checked 2026-10-18 without root: both root
+   ports report `current_link_width=8` at `32.0 GT/s`
    (`/sys/bus/pci/devices/0000:00:01.{1,3}/current_link_width`), and the live
-   engine states direct P2P between the cards
-   ([[Dual-R9700-Cooperation]]).
+   engine states direct P2P between the cards.
 3. llama.cpp `--split-mode layer` — no collectives at all ([[Llama Cpp Multi-Device Splits]]); costs the GGUF conversion plus paged attention /
    continuous batching / spec decode.
 4. Independent TP=1 replicas + routing proxy — cards never share one model.
@@ -253,4 +251,4 @@ kernel's declared args (`roc-obj-extract` + `llvm-readelf --notes`), look for
 
 - [[Llama Cpp Multi-Device Splits]] — the layer-split path around the wall
 - [[Dual-R9700-Cooperation]] / [[x4-pair-experiment]] — measured history
-- [[dankserver-qwen38-fp8-expert-offload-tuning]] — the gather economics of the retired tcclaviger vLLM era (2026-09-29 → 10-05)
+- [[frank-archon]] — the gather economics of the retired tcclaviger vLLM era (2026-09-29 → 10-05)

serverAppConfigs-docs/LLM/Pipeline Parallelism/Ethernet Bandwidth Math.md

@@ -1,11 +1,11 @@
 ---
 type: reference
 title: Ethernet Bandwidth Math
-description: Bytes-per-token math for pipeline vs tensor parallelism, feasibility tables for 1/2.5/10 GbE, and real multi-node benchmark numbers.
+description: "Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it."
 tags:
   - pipeline-parallelism
   - research
-timestamp: 2026-08-17
+timestamp: 2026-10-19
 ---
 
 # Bandwidth Analysis: Multi-Node LLM Inference over Ethernet
@@ -147,19 +147,11 @@ bytes_per_boundary = batch_size × hidden_size × 2   [fp16 = 2 bytes]
 - Per-token, per-boundary: `6144 × 2 = 12,288 bytes = 12 KB`
 - With Nₚₚ stage boundaries: total = `Nₚₚ × 12 KB/tok`
 
-**Full example — 27B model, 2 PP stages (1 boundary), batch=1:**
+**Worked examples (1 boundary = 2 PP stages):**
 ```
-1 boundary × 4096 × 2 bytes = 8,192 bytes/tok
-```
-
-**Example — same model, batch=64:**
-```
-1 boundary × 4096 × 64 × 2 bytes = 524,288 bytes = 512 KB/tok
-```
-
-**Example — 36B model, 2 PP stages, batch=1:**
-```
-1 boundary × 6144 × 2 bytes = 12,288 bytes = 12 KB/tok
+27B, batch=1:  1 boundary × 4096 × 2      bytes = 8,192 bytes/tok  = 8 KB
+27B, batch=64: 1 boundary × 4096 × 64 × 2 bytes = 524,288          = 512 KB/tok
+36B, batch=1:  1 boundary × 6144 × 2      bytes = 12,288 bytes/tok = 12 KB
 ```
 
 ### 1.2 Tensor Parallelism (TP) — for comparison
@@ -244,57 +236,13 @@ total_bytes_per_tok = L × batch_size × hidden_size × 2   [L all-reduces total
 | 50 | 10.24 Mbit/s | 163.84 Mbit/s | 655.36 Mbit/s |
 | 100 | 20.48 Mbit/s | 327.68 Mbit/s | 1,310.72 Mbit/s |
 
-### 2.3 Feasibility matrix
+### 2.3 Feasibility
 
-Green = feasible (< 50% of link capacity). Yellow = marginal (50–90%). Red = saturated (> 90%).
-
-**27B model — PP (1 boundary), batch=1:**
-
-| | 1 GbE (118 MB/s) | 2.5 GbE (294 MB/s) | 10 GbE (1175 MB/s) |
-|---|---|---|---|
-| 10 tok/s | ✅ 0.13 Mbit/s | ✅ 0.13 Mbit/s | ✅ 0.13 Mbit/s |
-| 50 tok/s | ✅ 0.65 Mbit/s | ✅ 0.65 Mbit/s | ✅ 0.65 Mbit/s |
-| 100 tok/s | ✅ 1.31 Mbit/s | ✅ 1.31 Mbit/s | ✅ 1.31 Mbit/s |
-
-**27B model — PP (1 boundary), batch=16:**
-
-| | 1 GbE | 2.5 GbE | 10 GbE |
-|---|---|---|---|
-| 10 tok/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s |
-| 50 tok/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s |
-| 100 tok/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s |
-
-**27B model — PP (1 boundary), batch=64:**
-
-| | 1 GbE | 2.5 GbE | 10 GbE |
-|---|---|---|---|
-| 10 tok/s | ✅ 8.19 Mbit/s | ✅ 8.19 Mbit/s | ✅ 8.19 Mbit/s |
-| 50 tok/s | ✅ 40.96 Mbit/s | ✅ 40.96 Mbit/s | ✅ 40.96 Mbit/s |
-| 100 tok/s | ✅ 81.92 Mbit/s | ✅ 81.92 Mbit/s | ✅ 81.92 Mbit/s |
-
-**27B model — TP (32 layers), batch=1:**
-
-| | 1 GbE | 2.5 GbE | 10 GbE |
-|---|---|---|---|
-| 10 tok/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s |
-| 50 tok/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s |
-| 100 tok/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s |
-
-**27B model — TP (32 layers), batch=16:**
-
-| | 1 GbE | 2.5 GbE | 10 GbE |
-|---|---|---|---|
-| 10 tok/s | ✅ 32.77 Mbit/s | ✅ 32.77 Mbit/s | ✅ 32.77 Mbit/s |
-| 50 tok/s | ✅ 163.84 Mbit/s | ✅ 163.84 Mbit/s | ✅ 163.84 Mbit/s |
-| 100 tok/s | ✅ 327.68 Mbit/s | ✅ 327.68 Mbit/s | ✅ 327.68 Mbit/s |
-
-**27B model — TP (32 layers), batch=64:**
-
-| | 1 GbE | 2.5 GbE | 10 GbE |
-|---|---|---|---|
-| 10 tok/s | ✅ 131.07 Mbit/s | ✅ 131.07 Mbit/s | ✅ 131.07 Mbit/s |
-| 50 tok/s | ✅ 655.36 Mbit/s | ✅ 655.36 Mbit/s | ✅ 655.36 Mbit/s |
-| 100 tok/s | ✅ 1310.72 Mbit/s | ✅ 1310.72 Mbit/s | ✅ 1310.72 Mbit/s |
+Thresholds: **feasible** = under 50% of link capacity, **marginal** = 50–90%,
+**saturated** = over 90%. Cross-checking §2.2 against §2.1: every PP case is
+feasible on every link class — even the heaviest, 27B at batch=64 and 100
+tok/s (81.92 Mbit/s), is feasible on 1 GbE — and every TP case is feasible
+except batch=64:
 
 **27B TP batch=64 at 100 tok/s: 1310.72 Mbit/s > 1 GbE usable (1000 Mbit/s) — link-saturated on 1 GbE**
 
@@ -368,10 +316,7 @@ With TP, each layer transfers `H × 2` bytes but this happens L times. For 256 K
 **Petals [1]:**
 - Uses pipeline parallelism with dynamic 8-bit quantization (halves bandwidth)
 - For BLOOM-176B (H=14336), per-boundary communication is ~14 KB/tok with quantization
-- Recommended bandwidth: 25 Mbit/s bidirectional for interactive inference [1]
 - Key finding: "For inference, performance does not depend much on bandwidth or sequence length but degrades with higher latency" — RTT matters more than throughput
-- At 1 Gbit/s with <5 ms RTT: 1.71 steps/s; at 100 Mbit/s with <5 ms: 1.66 steps/s (only 3% difference)
-- At 100 Mbit/s with 100 ms RTT: 1.23 steps/s (28% degradation)
 
 **llama.cpp RPC [2]:**
 - Model weights and KV cache distributed across nodes proportionally to memory
@@ -493,38 +438,17 @@ With Nₚₚ = 4 and 30 µs per-hop RTT: ~120 µs network, still <6% of compute
 
 ## Appendix: Full bandwidth formulae
 
-### Pipeline Parallelism (generation, batch=1)
+### Pipeline and Tensor Parallelism (generation)
+
+`N` is the batch size; `N = 1` is the batch=1 form.
 
 ```
-bytes_per_boundary_per_tok = hidden_size × 2   [fp16]
-total_bytes_per_tok = num_boundaries × hidden_size × 2
+bytes_per_boundary_per_tok = hidden_size × N × 2              [fp16, PP boundary]
+bytes_per_layer_allreduce  = hidden_size × N × 2              [TP, one all-reduce per layer]
+total_bytes_per_tok = num_boundaries × hidden_size × N × 2    [PP]
+total_bytes_per_tok = num_layers      × hidden_size × N × 2   [TP]
 bandwidth_mbps = total_bytes_per_tok × tokens_per_second × 8 / 1,000,000
-```
-
-### Pipeline Parallelism (generation, batch=N)
 
-```
-bytes_per_boundary_per_tok = hidden_size × N × 2
-total_bytes_per_tok = num_boundaries × hidden_size × N × 2
-```
-
-### Tensor Parallelism (generation, batch=1)
-
-```
-bytes_per_layer_allreduce = hidden_size × 2
-total_bytes_per_tok = num_layers × hidden_size × 2
-bandwidth_mbps = total_bytes_per_tok × tokens_per_second × 8 / 1,000,000
-```
-
-### Tensor Parallelism (generation, batch=N)
-
-```
-total_bytes_per_tok = num_layers × hidden_size × N × 2
-```
-
-### With 8-bit quantization (like Petals)
-
-```
-# Simply divide all fp16 byte counts by 2
-bytes_per_boundary_per_tok = hidden_size × 1   [int8]
+# With 8-bit quantization (like Petals): simply divide all fp16 byte counts by 2
+bytes_per_boundary_per_tok = hidden_size × 1                  [int8]
 ```

serverAppConfigs-docs/LLM/Pipeline Parallelism/Expected Performance.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Expected Performance
-description: Estimated tokens per second for a 25-40B model split across two nodes over 1/2.5/10 GbE, with real benchmark anchors.
+description: "Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on."
 tags:
   - pipeline-parallelism
   - performance
@@ -13,7 +13,7 @@ timestamp: 2026-08-14
 
 This page estimates the speed of a 25-40B model split across two machines
 over ethernet. It uses the bandwidth math in [[Ethernet Bandwidth Math]]
-and the measured numbers in [[Mixed-Vendor Experiments]].
+and the mixed-vendor measurements in [[x4-pair-experiment]].
 
 ## The Bandwidth Ceiling Is Not the Limit
 
@@ -93,5 +93,5 @@ roughly 200k-260k tokens across both nodes.
 ## See Also
 
 - [[Ethernet Bandwidth Math]]
-- [[Mixed-Vendor Experiments]]
+- [[Qwen3.8 PP Deployment]]
 - [[Sample Script Set]]

serverAppConfigs-docs/LLM/Pipeline Parallelism/Mixed-Vendor Experiments.md

@@ -1,9 +0,0 @@
----
-type: reference
-title: "Mixed-Vendor Experiments"
-description: "Real-world NVIDIA + AMD Strix Halo multi-node experiments: what works (llama.cpp RPC, Vulkan split), what breaks (mixed CUDA+ROCm vLLM), measured tok/s."
-tags:
-  - pipeline-parallelism
-  - research
-timestamp: 2026-08-14
----

serverAppConfigs-docs/LLM/Pipeline Parallelism/Qwen3.6 27B Spec Methods.md

@@ -1,9 +0,0 @@
----
-type: reference
-title: "Qwen3.6 27B Spec Methods"
-description: "Qwen3.6-27B Q8 GGUF variants and llama.cpp speculative decoding (draft-mtp, draft-dflash, ngram-mod): flags, RPC compatibility, memory/context math."
-tags:
-  - pipeline-parallelism
-  - research
-timestamp: 2026-08-14
----

serverAppConfigs-docs/LLM/Pipeline Parallelism/Qwen3.8 PP Deployment.md

@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX"
+description: "The real split deployment: which node is head and which is worker, the flags, and the verified gotchas."
 tags:
   - llm
   - pipeline
@@ -17,7 +17,7 @@ timestamp: 2026-09-03
 > removed from Dankserver on 2026-09-15 (it now sits in Dankclaw) and the server
 > runs 2× R9700. The live multi-GPU scheme on Dankserver is tensor parallelism
 > inside one process ([[Dual-R9700-Cooperation]],
-> [[frank-archon-radiance-qwen38-iq4r]]).
+> [[frank-archon]]).
 
 # Qwen3.8-27B Pipeline-Parallel Deployment (Dankclaw + Dankserver)
 
@@ -43,8 +43,16 @@ state, verified against the live journal and output smoke tests
   the mismatch (`resolve_fused_ops: Flash Attention enabled`) via the
   GGML_CUDA_FA_ALL_QUANTS rebuild — see "FA-all-quants rebuild" below.
 - Pool `262144`, `PARALLEL=1`, live `n_ctx_slot = 262144`.
-- MTP: standalone draft head, acceptance 0.83 then 0.75, mean len
-  4.0-4.3, per-position ~0.86/0.71/0.71/0.71.
+* MTP: standalone draft head, acceptance 0.83 then 0.75, mean len 4.0-4.3,
+  per-position ~0.86/0.71/0.71/0.71 — stronger at every position than the old bundled
+  head, which measured 0.739/0.554/0.427/0.340.
+- Spec chain as deployed: `--spec-draft-n-max 4` on the standalone draft, with
+  the `--spec-default` ngram-mod fallback at n_match 24 / n_min 48 / n_max 64
+  (the preset that survived the DSpark/DFlash retirements —
+  [[qwen3.8-spec-matrix]]).
+- Batch 8192 / ubatch 256, `--direct-io`, `--fit off`, `--gpu-layers all`. The
+  pi `models.json` declared 262144 (`contextWindow = maxTokens`, the house
+  convention of that era).
 - Head GPU side: **ROCm0** (no `--device` flag; HIP registers before
   Vulkan).
 - Worker steady state: 23854/24576 MiB used, ~273 MiB free (tight but
@@ -199,7 +207,7 @@ exists** (checkout read 2026-10-18: `frank-pp-worker.env`,
 `serve-pp-worker.sh`, `serve-pp-head.sh`, `frank-pp-worker.service`,
 `pp-test.sh`, `pp-bench.py` and `prune-rpc-cache.sh` are all absent), and
 `scripts/frank-dragoon.env` is today the halogen-flash-server env, not a PP
-head env ([[frank-dragoon-flash-server-v2-native]]).
+head env ([[frank-dragoon]]).
 
 ## Gotchas (all verified in b10433 source / live tests, 2026-08-15)
 
@@ -215,8 +223,11 @@ head env ([[frank-dragoon-flash-server-v2-native]]).
    faster prefill, ~3% slower decode (measured on this box).
 3. **Pin `--tensor-split 1,1`.** The default split is proportional to
    *reported free memory*: the Strix reports up to 96 GiB (GTT) vs the
-   3090's 24 GB, so the default lands ~20/80 on the slow device (verified
-   with a 4B smoke test). 1,1 = 50/50 = balanced per-layer latency.
+   3090's 24 GB, so the default lands ~20/80 on the slow device. 1,1 = 50/50 =
+   balanced per-layer latency. The 2026-08-15 4B smoke test that proved it
+   (production untouched): at a 0.2/0.8 split only 4 of 36 layers reached the
+   3090, and at a real split worker VRAM rose ~819 MiB with 95 t/s prefill /
+   58.8 t/s decode on the tiny model; link RTT ~0.3–0.5 ms.
 4. **`-dio` on the head.** Large model + RPC hangs at load without direct
    I/O.
 5. **RPC is unauthenticated and unencrypted.** LAN only. The worker binds

serverAppConfigs-docs/LLM/Pipeline Parallelism/Sample Script Set.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Sample Script Set
-description: An educational two-node llama.cpp RPC split of Qwen3.6-27B-Q8_K_P over ethernet. For reading only. Do not run or deploy.
+description: "An educational two-node RPC split script set, for reading only — never run it or deploy it."
 tags:
   - pipeline-parallelism
   - sample
@@ -255,7 +255,7 @@ is 30 GiB.
 
 ## See Also
 
-- [[llama.cpp RPC]]
+- [[Qwen3.8 PP Deployment]]
 - [[Expected Performance]]
 - [[Ethernet Bandwidth Math]]
-- [[Qwen3.6 27B Spec Methods]]
+- [[DFlash DSpark Speculative Decoding]]

serverAppConfigs-docs/LLM/Pipeline Parallelism/index.md

@@ -28,28 +28,24 @@ graph LR
     end
 ```
 
-- [[llama.cpp RPC]] — the working, production-capable path for two
   heterogeneous nodes over ethernet.
-- [[vLLM Pipeline Parallelism]] — supported but heavier; needs Ray or
   multiprocessing and does not compose with speculative decoding yet.
 
 ## What the Research Found
 
-- [[Ethernet Bandwidth Math]] — a PP boundary moves only the hidden state
+- [[Ethernet Bandwidth Math]] — Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it.
   (about 10 KiB per token for a 27B model). 1 GbE supports hundreds of
   tokens per second on paper; the real limit is latency and RPC overhead.
-- [[Mixed-Vendor Experiments]] — NVIDIA + Strix Halo multi-node inference
   works today via llama.cpp RPC with separate CUDA and ROCm builds. The
   exact NVIDIA-desktop + Strix-Halo-NUC pair has not been benchmarked with
   published numbers yet.
-- [[Qwen3.6 27B Spec Methods]] — Q8 GGUF variants, and which speculative
   methods (MTP, DFlash, ngram-mod) work with RPC. DSpark is vLLM-only.
-- [[Expected Performance]] — estimated tokens per second for a 27B Q8 split
+- [[Expected Performance]] — Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on.
   across an RTX 3090 and a Strix Halo NUC over 1/2.5/10 GbE.
 
 ## Deployment
 
-- [[Qwen3.8 PP Deployment]] — the real Qwen3.8-27B split: head on Dankclaw
+- [[Qwen3.8 PP Deployment]] — The real split deployment: which node is head and which is worker, the flags, and the verified gotchas.
   (Strix), worker on Dankserver (RTX 3090), b10433 on both sides, scripts in
   `scripts/`, morning-of test runbook. Verified gotchas: no `--device` with
   `--rpc`, default split is proportional to reported free memory (pin 1,1),
@@ -57,7 +53,7 @@ graph LR
 
 ## Sample Scripts
 
-- [[Sample Script Set]] — a two-node llama.cpp RPC split of
+- [[Sample Script Set]] — An educational two-node RPC split script set, for reading only — never run it or deploy it.
   Qwen3.6-27B-Q8_K_P with MTP, DFlash, and ngram-mod variants, sized for
   the largest context that fits. For educational reading only. Do not run
   it in production.
@@ -65,6 +61,11 @@ graph LR
 ## See Also
 
 - [[DFlash DSpark Speculative Decoding]]
-- [[frank-zealot-benchmark]]
+- [[frank-zealot]]
 - [[Dankserver System Specs]]
 - [[Dankclaw System Specs]]
+
+## Pages
+
+| Page | Owns |
+|---|---|

serverAppConfigs-docs/LLM/Pipeline Parallelism/llama.cpp RPC.md

@@ -1,9 +0,0 @@
----
-type: reference
-title: "llama.cpp RPC"
-description: "llama.cpp distributed inference via ggml-rpc-server and the --rpc flag: architecture, CLI, versions, limitations, and RPC-compatible speculative decoding."
-tags:
-  - pipeline-parallelism
-  - research
-timestamp: 2026-08-14
----

serverAppConfigs-docs/LLM/Pipeline Parallelism/vLLM Pipeline Parallelism.md

@@ -1,9 +0,0 @@
----
-type: reference
-title: "vLLM Pipeline Parallelism"
-description: "vLLM multi-node pipeline parallelism: flags, Ray/mp backends, interconnect requirements, PP bubble, and PP + speculative decoding support."
-tags:
-  - pipeline-parallelism
-  - research
-timestamp: 2026-08-14
----

serverAppConfigs-docs/LLM/Quantization Production.md

@@ -1,3 +1,13 @@
+---
+type: "howto"
+title: "Quantization Production"
+description: "Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file."
+tags:
+  - llm
+  - quantization
+timestamp: 2026-10-19
+---
+
 # Quantization Production — making our own quants, per card and runtime
 
 **Research page (2026-10-08).** Question: how do WE produce quantizations
@@ -147,12 +157,14 @@ cut to GPTQ-w4 for the 3090 is a bench experiment that needs no new
 hardware; EXL3 (`convert.py` on the 3090, exllama serve) is the
 quality-per-bit extreme of the same menu.
 Caveats before any swap: the serving contract (OpenAI server,
-`reasoning_content`, vision mmproj, compaction-hub window
-200,704 ×2 slots) must be met by the challenger runtime, and the
-24 GB card runs ONE server — benching follows the stop-and-mask law.
+`reasoning_content`, vision mmproj, and the compaction-hub window the endpoint
+declares in its Current shape — [[frank-zealot]]) must be met by the challenger
+runtime, and the 24 GB card runs ONE server — benching follows the stop-and-mask
+law.
 
 #### Worked example — the frank-archon w4dn8 recipe ported to the 3090
-(audit 2026-10-08, target `Qwen3.6-35B-A3B`)
+(audit 2026-10-08, target `Qwen3.6-35B-A3B`; sized at that date's hub shape —
+the window history is [[frank-zealot]]'s Eras table)
 **The methodology ports; the encoding must be translated.** llama.cpp
 decodes no GPTQ — the recipe re-expresses on GGUF-native terms:
 imatrix on our corpora (replaces the radiance grams; runs the 65 GiB
@@ -253,8 +265,6 @@ top-8**, moe_intermediate 512, hidden 2048, MTP ×1, 262k ctx, vocab
 - Repo: `scripts/convert-swift15-w4dn8.sh`,
   `scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe`,
   `scripts/kld-gate-swift15.sh`, `scripts/build-llama-rdna4.sh`,
-  `scripts/build-kairic-edge-gfx1151.sh`, [[agentionai-fast-imatrix]],
-  [[kingjones-strix-rocmfp4]], [[qwen3.8-strix-optimizations]],
-  [[frank-archon-radiance-swift15-w4dn8]],
-  [[frank-zealot-llama-qwen35-9b-q6k-2x200704]],
-  [[frank-dragoon-flash-server-v2-swift15]], [[GGUF Compatibility]].
+  `scripts/build-kairic-edge-gfx1151.sh`; endpoints [[frank-archon]],
+  [[frank-dragoon]], [[frank-zealot]]; method [[qwen3.8-strix-optimizations]]
+  and [[GGUF Compatibility]].

serverAppConfigs-docs/LLM/Quantization by Hardware Architecture.md

@@ -108,9 +108,10 @@ Use ROCmFP4 only with the ROCmFPX fork of llama.cpp. Upstream llama.cpp cannot r
 
 ## Choices for This Project
 
-### Dankserver (RTX 3090, 24 GB, Ampere)
+### The RTX 3090 (24 GB, Ampere) — frank-zealot's card
 
-The RTX 3090 has no native FP8 or FP4 tensor cores. Use:
+The 3090 has no native FP8 or FP4 tensor cores. It has sat in Dankclaw since
+2026-09-16; [[frank-zealot]]'s Eras table records the move off Dankserver. Use:
 
 | Ranking | Format | Why |
 |---------|--------|-----|
@@ -121,13 +122,17 @@ The RTX 3090 has no native FP8 or FP4 tensor cores. Use:
 
 ### Dankclaw (Strix Halo gfx1151, RDNA 3.5)
 
-RDNA 3.5 has no native FP8 tensor cores. The APU shares memory. The active Ling-3.0-flash service runs vLLM-ROCm with the AMD Quark INT4 (W4A16) format (the jcbtc CIRU Strix-native bundle). INT4 weights keep the 124B model inside the 96 GiB frame buffer with a full-precision KV cache.
+RDNA 3.5 has no native FP8 tensor cores and the APU shares its memory, so the
+GPU budget is the GTT aperture and every live figure on the box is
+[[Dankclaw System Specs]]. What serves here today is [[frank-dragoon]]: a
+vendor `.hgn` checkpoint, a format this page cannot rank because no public
+quantizer exists for it ([[Quantization Production]]).
 
 | Ranking | Format | Why |
 |---------|--------|-----|
-| 1 | INT4 W4A16 (Quark/vLLM) | active; fits 96 GiB; native ROCm kernels; concurrency |
-| 2 | MXFP4 (Quark) | same-bit, higher accuracy; beta on gfx1151 (A/B experiment) |
-| 3 | GGUF Q4_K_M / Q8_0 | works with upstream llama.cpp only |
+| 1 | `.hgn` (deployed) | the halogen engine's own format; production door is closed |
+| 2 | GGUF K/I quants, K_P blends | the only self-produced option on this card ([[Quantization Production]]) |
+| 3 | INT4 W4A16 (Quark) / MXFP4 (Quark) | the retired Ling-3.0-flash vLLM era, measured under the firmware-VGM frame buffer: [[ling-3.0-flash-trial]] |
 | 4 | FP8 (vLLM) | software emulation; memory savings only; no compute win |
 
 ## See Also

serverAppConfigs-docs/LLM/RAM CPU Offloading in VLLM.md

@@ -11,9 +11,19 @@ timestamp: 2026-08-28
 
 ## Overview
 
-vLLM uses CPU RAM and swap space to extend GPU memory capacity. This allows larger models to run on GPUs with limited VRAM. vLLM does not offload model weights to CPU. Instead, vLLM moves KV cache data between GPU and CPU.
+Stock vLLM offloads **KV pages**, not weights: the scheduler moves KV cache between GPU
+and CPU. That is the upstream behaviour and it is NOT the whole story on our hosts — the
+tcclaviger/R4D fork ran a real **weight** offload (`--enable-expert-offload`, tiered
+expert placement, a pinned host pool per rank), which is how a 147 GiB MoE served from a
+246 GiB box: [[RDNA4 MoE Expert Offload VRAM Budget]] and the tcclaviger era row of
+[[frank-archon]]. First-hand in-repo evidence outranks a generic upstream claim about our
+own hosts ([[Model Swap and Rollback]] §10).
 
-> **Current vLLM note:** Recent vLLM releases removed the `--swap-space` engine argument. CPU KV-cache offloading now uses `--kv-offloading-size`. See the parameters below for details.
+> **Verified 2026-10-19 against `vllm/config/cache.py` on `main`:** there is no
+> `swap_space` field — `--swap-space` is gone from current engine args. CPU KV offload is
+> `--kv-offloading-size`, and it activates **only** together with `--kv-offloading-backend`
+> (default `native`); both fields live in that one file. This page is the fleet's owner of
+> that knob contract — other pages link here instead of restating it.
 
 ## How CPU Offloading Works
 
@@ -42,9 +52,12 @@ This parameter controls the fraction of GPU memory used for KV cache.
 
 Lower values leave more GPU memory for model weights. Higher values allow more concurrent requests but increase swap frequency.
 
-### swap_space (removed) / kv-offloading-size (current)
+### swap_space (removed) / kv-offloading-size + kv-offloading-backend (current)
 
-The `--swap-space` argument was removed from current vLLM engine args. Use `--kv-offloading-size` instead. The table below describes the historical `--swap-space` values for reference.
+`--swap-space` no longer exists in current engine args (verified above); the live pair is
+`--kv-offloading-size` with `--kv-offloading-backend`, and offload stays off until the size
+is set. The table below is the historical `--swap-space` sizing guidance, kept as a sizing
+reference only.
 
 | Value | Description |
 |-------|-------------|
@@ -103,6 +116,9 @@ This parameter sets the KV cache data type.
 
 ## Memory Configuration Examples
 
+All three examples carry the retired `--swap-space` syntax and are kept as sizing
+references; the live knob pair is named above.
+
 ### Small Model on Single GPU
 
 ```
@@ -157,7 +173,7 @@ WARNING scheduler.py: Sequence group 0 is preempted by PreemptionMode.RECOMPUTE
 If you see frequent preemptions:
 
 1. Increase `gpu_memory_utilization`
-2. Increase `swap_space`
+2. Raise the KV offload budget (`--kv-offloading-size`, with `--kv-offloading-backend`)
 3. Decrease `max_num_seqs`
 4. Increase `tensor_parallel_size`
 5. Increase `max_model_len` to match actual usage
@@ -169,7 +185,8 @@ If you see frequent preemptions:
 - Reduce `max_num_seqs` to limit concurrent requests
 - Disable CUDA graphs with `--enforce-eager` to save memory
 - Set `compilation_config.cudagraph_capture_sizes` to smaller values
-- Use `--kv-cache-memory` to skip memory profiling
+- Set `--kv-cache-memory-bytes` to pin the KV budget instead of letting the profiler guess
+  (the flag our own `scripts/patches/gpu_worker-kv-cap.patch` reads)
 - Limit multi-modal inputs with `limit_mm_per_prompt`
 
 ## Environment Variables

serverAppConfigs-docs/LLM/RDNA4 Container Serving.md

@@ -1,19 +1,19 @@
 ---
 type: reference
 title: RDNA4 Container Serving
-description: Bringing a model larger than VRAM onto Dankserver's 2x R9700 (gfx1201) with the tcclaviger vLLM stack — quant-class gating, fetch verification, container GPU-init triage, memory budgeting and the expert-offload plan.
+description: "Bringing a model larger than VRAM onto the two-card box: quant gating, fetch verification, container GPU-init triage."
 tags: [rdna4, gfx1201, r9700, vllm, expert-offload, podman, rocm]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
 
 # RDNA4 Container Serving
 
-Bring-up path for Dankserver (2x Radeon AI PRO R9700, gfx1201, 31.86 GiB each =
-**63.7 GiB real VRAM**, 246 GiB RAM, 7700X) with
+Bring-up path for Dankserver (2x Radeon AI PRO R9700, gfx1201 — card count,
+per-card VRAM and host RAM are [[Dankserver System Specs]]'s) with
 `docker.io/tcclaviger/vllm` (vLLM 0.29.0.dev, GLM/Qwen-specialised RDNA4 build
 with native MoE quant kernels and a first-class expert offload planner). The
 tuning history and current deployed shape live in
-[[dankserver-qwen38-fp8-expert-offload-tuning]]; this page is the bring-up law.
+[[frank-archon]]; this page is the bring-up law.
 
 ## 0. Know the box — and its phantom third GPU
 
@@ -155,7 +155,9 @@ podman run -d --name <ctr> --ipc=host --ulimit memlock=-1:-1 \
 
 ## 5. Memory budgeting: the levers are not where they look
 
-The planner builds `budget = total × util`, subtracts a KV reserve, and gives
+Every figure in this section is a tcclaviger-era boot on the port that is
+[[frank-archon]]'s today; the laws they teach are general. The planner builds
+`budget = total × util`, subtracts a KV reserve, and gives
 the REMAINDER to the expert arena. Consequences, all measured:
 
 - **`--gpu-memory-utilization` grows the ARENA, not the KV pool** (KV stayed
@@ -163,16 +165,19 @@ the REMAINDER to the expert arena. Consequences, all measured:
 - **Lowering `--max-model-len` is the WRONG lever and backfires:** a smaller
   context shrinks the reserve, handing the space to the arena — measured 65536
   → 0.64 GiB KV, 32768 → 0.34 GiB KV.
-- **Hybrid-attention undercount:** the planner's per-token KV figure (9190
-  B/token for Qwen4Exp) misses the large fixed per-sequence Mamba state
+- **Hybrid-attention undercount:** the planner's per-token KV figure for
+  Qwen4Exp is an undercount — the printed figure and the true one are both on
+  [[frank-archon]] — because it misses the large fixed per-sequence Mamba state
   (~2x), so the reserve is always short and the arena silently covers it.
-  Fix by pinning the pool: `--kv-cache-memory-bytes <BYTES>` (2 GiB →
-  113,595 tokens / 3.47x concurrency at 32k). For long context,
+  Fix by pinning the pool: `--kv-cache-memory-bytes <BYTES>` (era reading: a
+  2 GiB pin = 113,595 tokens ≈ 3.47x concurrency at 32k — that pair is this
+  page's record of the tcclaviger boots, not [[frank-archon]]'s). For long
+  context,
   `--kv-offloading-size` (GiB) spills KV to host.
 - **`--expert-offload-mem` caps the HOST side, not the VRAM arena**
   (`cap = int(x * GIB)` against `host_memory_available()`). Do not reach for
   it to fix a VRAM problem; to grow the arena, shrink the KV pin or raise
-  util (see [[dankserver-qwen38-fp8-expert-offload-tuning]] for the
+  util (see [[frank-archon]] for the
   util=1.0 unbootability law).
 
 **Read the plan — it is the real acceptance test:**
@@ -201,18 +206,21 @@ host RAM = `host-backed set + layers still loading`, not the whole model. At
 runtime `apply_hot_cold` sends arena-resident experts down a hot GEMM and the
 rest straight over PCIe via UVA views — no staging copy in the critical path.
 
-**Emulation check:** a dequantize-to-compute path doubles real host RAM (118
-GiB offload request → ~238 GiB) and is invisible in the weight count. Read the
-log for `Using 'EMULATION' Mxfp4 MoE backend` before concluding "doesn't fit".
+**Emulation check:** a dequantize-to-compute path doubles real host RAM — the
+measured pair (offload request vs host footprint) is on [[frank-archon]] — and
+the doubling is invisible in the weight count. Read the log for `Using
+'EMULATION' Mxfp4 MoE backend` before concluding "doesn't fit".
 
 ## 6. Performance profile and honest measurement
 
-Decode on this pair is **PCIe-gather-bound, context-independent**: 12.11
-tok/s FLAT from 304 to 16,095 context tokens — KV attention is not the cost.
-With ~29 % of experts arena-resident, ~0.84 GB crosses PCIe per token at the
-engine's effective read rate; cold-vs-warm differing ~10 % confirms the LRU
-works; growth is linear in hit rate, and only speculative decode cuts
-sequential STEPS rather than making each cheaper.
+Decode on this pair is **PCIe-gather-bound, context-independent**: the rate is
+flat from a few hundred to sixteen thousand context tokens, so KV attention is
+not the cost (the flat figure and its MTP/gather deltas are
+[[Tcclaviger Fork Ops]]'s profile of the same shape). With ~29 % of experts
+arena-resident, under a gigabyte crosses PCIe per token at the engine's
+effective read rate; cold-vs-warm differing ~10 % confirms the LRU works;
+growth is linear in hit rate, and only speculative decode cuts sequential STEPS
+rather than making each cheaper.
 
 **Benchmark traps that produced fake numbers here:**
 
@@ -238,13 +246,13 @@ sequential STEPS rather than making each cheaper.
 boolean (default 0; at depth 1 only ~10 % cold-vs-warm difference — the cache
 already works); `CLAV_TUNABLEOP_SWEEP=1` autotunes GEMM shapes (silently
 skipped under expert offload unless `CLAV_TUNABLEOP_WITH_OFFLOAD=1` too —
-see [[dankserver-qwen38-fp8-expert-offload-tuning]] for why it was rejected);
+see [[frank-archon]] for why it was rejected);
 `--expert-precise-memory` swaps the fitted overhead figure in.
 
 ## Related
 
-- [[dankserver-qwen38-fp8-expert-offload-tuning]] — deployed shape, plan law,
+- [[frank-archon]] — deployed shape, plan law,
   measured campaigns
 - [[PCIe Link and Atomics]] — why decode is gather-bound and what cannot fix it
 - [[RDNA4 MoE Expert Offload VRAM Budget]] — arena/plan arithmetic deep-dive
-- [[frank-dragoon-benchmark]] and [[frank-zealot-benchmark]] — window discipline on live hosts
+- [[frank-dragoon]] and [[frank-zealot]] — window discipline on live hosts

serverAppConfigs-docs/LLM/RDNA4 MoE Expert Offload VRAM Budget.md

@@ -1,17 +1,19 @@
 ---
 type: reference
 title: RDNA4 MoE Expert Offload VRAM Budget
-description: The expert-offload plan arithmetic on the tcclaviger fork — which terms a deploy can move, the util law, the CUDA-graph charge, what free VRAM really is, knife-edge survivability, parallelism ceilings and the context ladder.
+description: "Expert-offload arithmetic on the RDNA4 fork: movable terms, the utilisation law, graph charge, ceilings, context ladder."
 tags: [vllm, expert-offload, vram, arena, kv-cache, ladder, tcclaviger]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
 
 # RDNA4 MoE Expert Offload VRAM Budget
 
-For VRAM-bound vLLM on the tcclaviger RDNA4 fork (frank-archon: Qwen3.8-
-Flash-Next FP8, TP=2, 2x R9700 at 31.86 GiB each, expert offload to host RAM).
-Tuning history and the deployed shape live in
-[[dankserver-qwen38-fp8-expert-offload-tuning]]; boot-failure capture lives in
+For VRAM-bound vLLM on the tcclaviger RDNA4 fork. Every boot ledger, arena
+figure and rate below is from [[frank-archon]]'s tcclaviger era (2026-09-29 →
+10-05: Qwen3.8-Flash-Next FP8, TP=2, 2x R9700 at 31.86 GiB each, expert offload
+to host RAM) — that page owns the endpoint's shape, tuning history and era
+record; this page owns only the plan arithmetic, the ladder method and the audit
+procedure. Boot-failure capture lives in
 [[vLLM Boot Forensics and Config Interrogation]].
 
 ## 1. The plan law
@@ -215,5 +217,5 @@ at graph capture; 6 seqs fp8 → 10.70 GiB ask with 9.18 GiB free (fails) —
 
 - [[RDNA4 Container Serving]] — the bring-up path and container traps
 - [[vLLM Boot Forensics and Config Interrogation]] — capture and contracts
-- [[dankserver-qwen38-fp8-expert-offload-tuning]] — measured history
+- [[frank-archon]] — measured history
 - [[Llama Cpp Serving Sizing]] — the llama.cpp equivalent

serverAppConfigs-docs/LLM/Running Qwen3.6 27B FP8 on Dankclaw.md

@@ -1,12 +1,18 @@
 ---
 type: reference
-description: "Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series."
+description: "Serving a dense FP8 vision-language checkpoint on the Strix Halo iGPU: the load path, the flags, the format gate."
 tags:
   - llm
 timestamp: 2026-09-03
 ---
 
 # Running Qwen3.6 27B FP8 on Dankclaw
+> **RETIRED - vLLM era (before the 2026-08-17 redo).** Nothing on this page serves
+> today: no vLLM or ROCm process runs on Dankclaw, and the iGPU endpoint at
+> `192.168.0.143:7770` is [[frank-dragoon]] (halogen flash-server). The page stays as
+> the era record of the Qwen3.6-27B-FP8 vLLM bring-up and the Strix Halo ROCm
+> recipes ([[frank-redo-2026-08-17]]). The hardware table below carries the era's
+> host figures; live ones are [[Dankclaw System Specs]].
 
 ## Model Information
 
@@ -29,6 +35,7 @@ Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series. It uses 27
 | Vision Encoder | Yes (image-text-to-text) |
 
 ## Hardware: Dankclaw
+*Era figures (2026-08). The host has changed since - [[Dankclaw System Specs]].*
 
 | Property | Value |
 |----------|-------|
@@ -55,7 +62,11 @@ The FP8 variant requires approximately 16-17 GB. ThinkingCap-FP8 requires ~29 GB
 
 ## Prerequisites
 
-> **Production note:** Dankclaw's primary service now runs vLLM-ROCm with the Ling-3.0-flash CIRU INT4 model (see `scripts/frank-dragoon.service`). This page documents the alternative vLLM FP8 deployment of Qwen3.6-27B.
+> **Era note (2026-08, superseded twice):** at writing time Dankclaw's primary
+> service was the vLLM-ROCm Ling-3.0-flash CIRU INT4 trial ([[ling-3.0-flash-trial]],
+> retired 2026-08-17), and this page documented the alternative vLLM FP8 deployment
+> of Qwen3.6-27B. Both paths are dead; `scripts/frank-dragoon.service` today is the
+> halogen flash-server unit ([[frank-dragoon]]).
 
 Install vLLM for ROCm. Version 0.24.0+ is required for Qwen3.6 with speculative decoding:
 

serverAppConfigs-docs/LLM/Running Qwen3.6 35B A3B FP8 on Dankserver.md

@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "Qwen3.6-35B-A3B is a Mixture-of-Experts vision-language model from the Qwen3.6 series."
+description: "Serving a mixture-of-experts FP8 checkpoint on the R9700 host: the load path, the format gate, the offload shape."
 tags:
   - llm
 timestamp: 2026-09-03

serverAppConfigs-docs/LLM/Shared Memory Architecture Dankclaw.md

@@ -36,7 +36,7 @@ In unified memory:
 
 ### ROCm Version
 
-vLLM supports AMD GPUs with ROCm 6.3 or above *(unverified 2026-10-18)*. Dankclaw has ROCm 7.2.4 from AMD repos (`/etc/apt/sources.list.d/rocm.list` → `deb https://repo.radeon.com/rocm/apt/7.2.4 noble main`; `rocm-core 7.2.4.70204-93~24.04`, read 2026-10-18). No vLLM endpoint runs on this host today — 7770 is halogen-flash-server and 7771 is llama.cpp ([[Dankclaw System Specs]]) — so this page is reference material, not the serving path.
+vLLM supports AMD GPUs with ROCm 6.3 or above *(unverified 2026-10-18)*. Dankclaw's ROCm version and which engine owns which port today are [[Dankclaw System Specs]]'s — so this page is reference material, not the serving path.
 
 Check ROCm version:
 
@@ -103,7 +103,7 @@ vllm serve model \
 
 - `gpu_memory_utilization`: Set lower (0.8-0.85) to leave room for CPU operations
 - `max_model_len`: Match actual context needs
-- The old `--swap-space` argument was removed from current vLLM — `CacheConfig` on `main` carries no `swap_space` field ([source](https://raw.githubusercontent.com/vllm-project/vllm/main/vllm/config/cache.py), read 2026-10-18). Use `--kv-offloading-size` (`kv_offloading_size`, same file) for CPU KV offload
+- `--swap-space` is retired from current vLLM; the CPU KV-offload knob pair, with the upstream `CacheConfig` verification, is owned by [[RAM CPU Offloading in VLLM]]
 
 ### NUMA Binding
 
@@ -166,7 +166,7 @@ Unified memory has lower bandwidth than dedicated VRAM:
 
 ### ROCm Overcommit
 
-ROCm reports ~114 GiB of GPU memory on Strix Halo APU (the GTT aperture: `mem_info_gtt_total` = 122402963456 bytes = 114 GiB, read 2026-10-18 under `/sys/class/drm/card1/device/`; `llama-server --list-devices` shows 116,732 MiB — the 2026-08-17 "96 GiB" figure was stale). Physical RAM visible to the OS is **121 GiB** today (`free -g` total 121, read 2026-10-18; 64 GiB in the earlier reading on this page, upgraded from 32 GB). The GPU can address more than physical RAM exists because the GTT aperture maps over the shared LPDDR. vLLM will see ~114 GiB and try to allocate accordingly. Set `--gpu-memory-utilization` to 0.80 or lower to prevent OOM.
+ROCm reports the GTT aperture as "GPU memory" on this APU; the aperture size, the physical `MemTotal` behind it and the firmware carve-out are [[Dankclaw System Specs]]'s numbers, and that page also carries the 2026-08-17 "96 GiB" reading this page used to quote (stale). One reporting quirk worth keeping here: `llama-server --list-devices` prints 116,732 MiB for the same device the sysfs file calls 114 GiB — engines round the aperture differently, so read the sysfs value when sizing. The GPU can address more than physical RAM exists because the GTT maps over the shared LPDDR. vLLM will see the aperture and try to allocate accordingly; set `--gpu-memory-utilization` to 0.80 or lower to prevent OOM.
 
 Lower bandwidth means slower inference. Plan for lower throughput.
 
@@ -232,9 +232,8 @@ FP8 quantization depends on GPU architecture:
   the matmul claim as the settled half.)
 
 For FP4 on Dankclaw the ROCmFPX llama.cpp fork (`Q2_0_ROCMFP4`) provided native
-ROCm FP4 kernels; that runtime line is **retired** (CIRU/Kairic eras,
-[[ciru-activefpx-promptforge]], [[frank-dragoon-kairic-edge-qwen38-27b]]) and no
-ROCmFPX build serves this host today.
+ROCm FP4 kernels; that runtime line is **retired** — the CIRU and Kairic eras on
+[[frank-dragoon]] — and no ROCmFPX build serves this host today.
 
 ## Example Commands
 

serverAppConfigs-docs/LLM/Tcclaviger Fork Ops.md

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Tcclaviger Fork Ops
-description: Operating the tcclaviger RDNA4 vLLM fork on dankserver (shape G era, rollback target since the 2026-10-05 FreeToken cutover) — the ad-hoc sweep harness, measured knob verdicts, sweep hygiene, the large-MoE serving recipe, and compile-cache persistence.
+description: "Operating the RDNA4 vLLM fork: the sweep harness, measured knob verdicts, sweep hygiene, compile-cache persistence."
 tags: [tcclaviger, vllm, dankserver, expert-offload, sweeps, cache]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -60,6 +60,8 @@ TUNABLE=0`, stock gather 8×16 → 26.80/25.15/26.81 tok/s @ 166/1087/4064 ctx.
 
 Decode on this stack is PCIe-gather-bound (~30% GPU duty cycle), so
 batch-shaped compute knobs move ~nothing.
+Measured with no spec: **12.11 tok/s FLAT from 304 to 16,095 context tokens** — KV
+attention is not the cost, so a context-slope hunt on this stack is null by construction.
 
 The `R4D_*/CLAV_*` knob family (QSA_*, MHC, ROUTE, ACT_QUANT, AR_QUANT,
 SILU_QUANT) are kill-switches defaulting ON, not perf dials; real levers are
@@ -120,95 +122,36 @@ exactly SEQS×(n+1) (`suboptimal` warning) → try 8192.
 
 ## Serving a model larger than VRAM (dankserver recipe)
 
-Dankserver: 2× R9700 (gfx1201), 31.86 GiB each = 63.7 GiB real VRAM, 246 GiB
-RAM. `torch.cuda.device_count()` returns 3 and summing `total_memory` gives
-a bogus 186.7 GiB (the Raphael iGPU is in the list — see
-[[Llama Cpp Serving Sizing]] for the reading discipline).
+Dankserver's card count, per-card VRAM and host RAM are in [[Dankserver System Specs]];
+`torch.cuda.device_count()` returns 3 on this box and summing `total_memory` overstates it
+badly (the Raphael iGPU is in the list — reading discipline: [[Llama Cpp Serving Sizing]]).
 
-**Kernel gate — this decides which checkpoint can work at all.**
-`/app/r4dhip/r4d.so` exports `gemm_moe_mxfp4a8_*` and `gemm_moe_dsfp4a8_*`
-but NO `gemm_moe_w4a16_*`. Expert offload requires an experts class with
-BOTH a compiled kernel AND an `offload_recipe`; the planner aborts startup
-otherwise. Usable: MXFP4, DSFP4, FP8-HIP (`R4dMxfp4MoEExperts`,
-`R4dDsfp4MoEExperts`, `Fp8HipMoEExperts`). `R4dW4a16MoEExperts` has no
-kernel (the class exists, the r4d.so entry does not); RFA has neither. Check
-the checkpoint's quant against this list BEFORE downloading anything.
+Everything else about running this image — the quant-kernel gate that decides
+which checkpoints can serve at all, the entrypoint contract, the known-good
+invocation, the `/dev/dri` and memlock traps, the util and max-model-len arena
+traps, the `--ipc=host` / `-d` stdout traps and the gated-fetch discipline — is
+written once on [[RDNA4 Container Serving]], and the plan arithmetic behind the
+arena/KV traps is [[RDNA4 MoE Expert Offload VRAM Budget]]'s. Three findings
+stay here because this harness produced them:
 
-Entrypoint is `/app/tools/image_entrypoint.sh`, ending `exec vllm serve
-"$@"` — pass the model path as the FIRST argument and do NOT pass `serve`.
-Known-good invocation:
+- **The KV reserve undercounts hybrid-attention models ~2×** (the planner's
+  B/token figure against ~18,000 real: the Mamba state is a fixed per-sequence
+  cost). Fix by pinning the pool: `--kv-cache-memory-bytes`.
+- **`--expert-offload-mem` caps the HOST side**, not the VRAM arena —
+  structurally orthogonal to `--expert-precise-memory` (verified in-container
+  2026-10-05: precise-memory only swaps the GPU overhead term,
+  expert_offload_mem only sets the host bound; they rejoin only inside
+  `place_experts`, and the plan rewrites `expert_offload_mem` back to the cap
+  the plan met, so plan and allocation cannot drift).
+- **Check whether the stack supports the model natively before porting an
+  overlay.** The image already registered `Glm5NextForConditionalGeneration`,
+  `Qwen4ExpForCausalLM` and both MTP variants — chasing four missing symbols
+  through two image tags to bring an overlay up was entirely wasted work.
 
-```bash
-sudo sh -c 'sync; echo 3 > /proc/sys/vm/drop_caches'
-podman run -d --name <ctr> --ipc=host --ulimit memlock=-1:-1 \
-  --cap-add SYS_PTRACE \
-  --device /dev/kfd --device /dev/dri --device /dev/mem \
-  -e HIP_VISIBLE_DEVICES=0,1 -e VLLM_ROCM_USE_AITER=0 \
-  -v /home/dank/models/<model>:/models:ro \
-  docker.io/tcclaviger/vllm:latest /models \
-  --served-model-name <name> --port 7770 \
-  --tensor-parallel-size 2 \
-  --max-model-len 32768 \
-  --kv-cache-memory-bytes 2147483648 \
-  --max-num-seqs 4 \
-  --gpu-memory-utilization 0.95 \
-  --enable-prefix-caching --trust-remote-code
-```
-
-`-p 7770:7770` is not included, so the API is only reachable via
-`podman exec` — add `-p` if host-side curl is needed. Load ~5 min;
-`/health` answers connection-refused the whole time; watch for
-`Application startup complete` in `podman logs <ctr>`. Verified working:
-`orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (gated; 172.8 GiB, 131
-shards, `Qwen4ExpForConditionalGeneration`, 48 layers, 512 experts top-10,
-multimodal, native 262,144).
-
-Traps that each cost a boot:
-
-1. **Mount the whole `/dev/dri`, never individual render nodes.** Render
-   nodes without the primary `card*` nodes → HIP enumerates two agents and
-   refuses a context on either (`RuntimeError: No CUDA GPUs are available`)
-   while `device_count()` cheerfully returns 2. Confirmed by direct A/B.
-2. **memlock must be raised** (done on Dankserver via
-   `/etc/security/limits.d/99-memlock.conf`). Host default 8 MiB makes any
-   large pinned allocation fail as a misleading
-   `torch.AcceleratorError: CUDA error: out of memory` from `pin_memory()`;
-   it cannot be raised from inside a container or root's own shell — only
-   PAM applies it at login. Matters because `--expert-host-mlock` defaults
-   True.
-3. **`--gpu-memory-utilization` does NOT buy KV under expert offload** —
-   the planner sizes the arena against the budget and leaves KV the
-   remainder; raising util 0.95→0.98 grew the arena and the KV pool stayed
-   at 0.64 GiB.
-4. **Lowering `--max-model-len` is the WRONG lever and backfires** — the
-   budget subtracts the KV reserve first, so a smaller context shrinks the
-   reserve and hands the space to the arena (measured: 65536 → 0.64 GiB KV;
-   32768 → 0.34 GiB).
-5. **The KV reserve undercounts hybrid-attention models ~2×** (9190 B/token
-   planned vs ~18,000 real: the Mamba state is a fixed per-sequence cost).
-   Fix by pinning the pool: `--kv-cache-memory-bytes`.
-6. **`--expert-offload-mem` caps the HOST side**, not the VRAM arena —
-   structurally orthogonal to `--expert-precise-memory` (verified
-   in-container 2026-10-05: precise-memory only swaps the GPU overhead term,
-   expert_offload_mem only sets the host bound; they rejoin only inside
-   `place_experts`, and the plan rewrites `expert_offload_mem` back to the
-   cap the plan met, so plan and allocation cannot drift).
-7. `--ipc=host` and `--shm-size` are mutually exclusive in podman; pick one.
-8. `podman run -d` without `-a` sends no stdout to a redirect — the log file
-   only receives the container ID; use `podman logs`.
-
-Status reads: use `scripts/dankstatus.sh`, never hand-rolled
-`/proc/meminfo` parsing — `/proc/meminfo` separates key from value with a
-TAB, so `grep MemAvailable | tr -s " " | cut -d" " -f2` returns empty and
-awk prints `0.0` — a false "host out of memory" that invites an unnecessary
-abort. Gated checkpoints: token via a mode-600 `curl -K` config, never argv
-(`pgrep -a curl` prints it); always `--fail`; compare each file's byte count
-to the Hub's declared size; `-C -` resumes truncated shards.
-
-**Check whether the stack supports the model natively before porting an
-overlay.** The image already registered `Glm5NextForConditionalGeneration`,
-`Qwen4ExpForCausalLM` and both MTP variants — chasing four missing symbols
-through two image tags to bring an overlay up was entirely wasted work.
+The checkpoint this recipe was verified on is
+`orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` (gated; 172.8 GiB, 131 shards,
+`Qwen4ExpForConditionalGeneration`, 48 layers, 512 experts top-10, multimodal,
+native 262,144) — an [[frank-archon]] era payload.
 
 ## Compile-cache persistence (survives `podman rm -f`)
 
@@ -240,20 +183,15 @@ cold, of which 173 s was one graph).
 
 ## Plan-line verification checklist (after any fork deploy)
 
-```
-ssh dankserver 'sudo journalctl CONTAINER_NAME=frank-archon-qwen -b -o cat --no-pager \
-  | grep -aoE "(budget \(total x util\) +[0-9.]+|expert cache space +[0-9.]+ GiB +[0-9,]+ slots|Estimated CUDA graph memory.{0,80}|Graph capturing finished.{0,40}|GPU KV cache size.{0,90})"'
-```
-
-`grep -oE` with a digit-anchored tail — plain `grep`/journalctl truncate
-these columns to `17.`. Verify: new budget/graphs/arena in the plan;
-`GPU KV cache size` matches the requested token count; `Estimated CUDA graph
-memory` vs the capture line (raise `_PER_GRAPH_MB` toward 52 if capture
-exceeds the charge); `rocm-smi --showmeminfo vram` high-water per card = the
-real margin; `systemctl show frank-archon -p NRestarts --value` = 0;
-`VLLM_CUDAGRAPH_*` present in `podman exec … env`; then a real chat
-completion from a PEER — `curl localhost:7770/health` inside an ssh session
-on the host can return `000` and looks like an outage that isn't.
+The journal one-liner that reads the plan lines (digit-anchored `grep -oE`,
+because plain grep truncates those columns to `17.`) and the post-deploy
+verification order — plan vs captured graph memory, `GPU KV cache size` vs the
+ask, per-card `rocm-smi` high-water, `NRestarts` = 0, `VLLM_CUDAGRAPH_*` present
+in the container env, then a real completion from a PEER because
+`curl localhost:7770/health` inside an ssh session on the host can return `000`
+and look like an outage — are step 3 of
+[[RDNA4 MoE Expert Offload VRAM Budget]] §7. Run that; do not re-derive it
+here.
 
 ## Related
 

serverAppConfigs-docs/LLM/VLLM Index.md

@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "This index contains all documentation for vLLM deployment on Dankserver and Dankclaw."
+description: "Locator for every vLLM page in this vault."
 tags:
   - llm
   - vllm
@@ -9,7 +9,7 @@ timestamp: 2026-08-28
 
 # vLLM Index
 
-> **RETIRED (2026-08-18).** The vLLM service on Dankclaw was replaced by the CIRU ROCmFPX llama.cpp runtime (v3). These pages are historical reference. See [[ciru-activefpx-promptforge]].
+> **RETIRED (2026-08-18).** The vLLM service on Dankclaw was replaced by the CIRU ROCmFPX llama.cpp runtime (v3). These pages are historical reference. See [[frank-dragoon]].
 
 ## Documentation Overview
 
@@ -60,7 +60,7 @@ vllm serve meta-llama/Llama-3.1-70B-Instruct --tensor-parallel-size 4
 vllm serve Qwen/Qwen2.5-1.5B-Instruct --quantization fp8
 
 # With GPU memory control
-vllm serve Qwen/Qwen2.5-1.5B-Instruct --gpu-memory-utilization 0.8 --swap-space 8
+vllm serve Qwen/Qwen2.5-1.5B-Instruct --gpu-memory-utilization 0.8 --kv-offloading-size 8 --kv-offloading-backend native
 ```
 
 ### Common API Endpoints
@@ -79,7 +79,7 @@ vllm serve Qwen/Qwen2.5-1.5B-Instruct --gpu-memory-utilization 0.8 --swap-space
 |-----------|-------------|---------------|
 | tensor_parallel_size | Number of GPUs for model sharding | 1-8 |
 | gpu_memory_utilization | Fraction of GPU memory for KV cache (default 0.92) | 0.7-0.95 |
-| swap_space | Removed in current vLLM; use `--kv-offloading-size` | n/a |
+| swap_space | Retired; live knob pair in [[RAM CPU Offloading in VLLM]] | n/a |
 | max_model_len | Maximum context length | auto |
 | max_num_batched_tokens | Max tokens per batch | 2048-8192 |
 | max_num_seqs | Max sequences per batch | 128-512 |

serverAppConfigs-docs/LLM/VLLM Overview.md

@@ -140,7 +140,7 @@ curl http://localhost:8000/v1/chat/completions \
 | `tensor_parallel_size` | 1 | Number of GPUs for tensor parallelism |
 | `pipeline_parallel_size` | 1 | Number of GPUs for pipeline parallelism |
 | `gpu_memory_utilization` | 0.92 | Fraction of GPU memory for KV cache |
-| `swap_space` | removed | Removed in current vLLM; use `--kv-offloading-size` |
+| `swap_space` | — | retired; the live CPU-KV-offload knob pair is in [[RAM CPU Offloading in VLLM]] |
 | `enforce_eager` | false | Use eager mode (no CUDA graphs) |
 | `max_num_batched_tokens` | 2048 | Max tokens per batch |
 | `max_num_seqs` | 128 | Max sequences per batch |
@@ -167,7 +167,7 @@ curl http://localhost:8000/v1/chat/completions \
 | Argument | Default | Description |
 |----------|---------|-------------|
 | `gpu_memory_utilization` | 0.92 | Fraction of GPU memory for KV cache |
-| `swap_space` | removed | Removed in current vLLM; use `--kv-offloading-size` |
+| `swap_space` | — | retired; the live CPU-KV-offload knob pair is in [[RAM CPU Offloading in VLLM]] |
 | `kv_cache_dtype` | auto | KV cache data type (auto, fp8, fp8_e5m2) |
 | `max_model_len` | auto | Maximum context length |
 | `num_gpu_blocks` | auto | Number of GPU blocks |

serverAppConfigs-docs/LLM/agentionai-fast-imatrix.md

@@ -1,217 +0,0 @@
----
-type: reference
-title: agentionai Qwen3.8-Flash-Next ROCmFP4-FAST imatrix on Dankclaw
-description: agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF served by the charlie12345/ROCmFPX llama.cpp runtime (build-qwen38). 87.06 GiB Q4_0_ROCMFP4_FAST imatrix (v2 joined PLE layout), own MTP draft head (ROCmFP4-FAST, 2.28 GiB). frank-dragoon.
-tags:
-  - llama.cpp
-  - rocm
-  - dankclaw
-  - gfx1151
-  - agentionai
-  - rocmfp4
-  - strix-halo
-  - qwen4exp
-  - mtp
-timestamp: 2026-08-29
----
-
-# agentionai Qwen3.8-Flash-Next ROCmFP4-FAST imatrix on Dankclaw
-
-> **RETIRED 2026-08-30** — replaced by the kingjones STRIX model on the
-> card's pinned kingjones30/ROCmFPX runtime; see
-> [[kingjones-strix-rocmfp4]] for the active runbook. This page stays as
-> the deployment record: the v2-joined-PLE layout requirement, the MTP
-> head incompatibility (verified 3x), and the QSA F32-mask patch
-> (`scripts/patches/qwen4exp-qsa-f32-mask.patch`) that made
-> `--flash-attn on` work on the charlie build-qwen38 tree — all still
-> true for that binary + model pair, which remains the rollback target.
-
-`frank-dragoon` (port 7770) served
-`agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF` (main,
-`Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-v2.gguf`, the `v2/` joined layout)
-with the author's own MTP draft head
-(`Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf`, never loaded) on the
-charlie12345/ROCmFPX `build-qwen38` runtime, 2026-08-29 → 2026-08-30.
-
-## Why this model
-
-- 87.06 GiB at 4.23 bpw — everything on the GPU of the 128 GB unified-memory
-  box, including the 51.2 B-parameter n-gram (PLE) table. REPO `v2/` layout
-  is used (joined single `per_layer_token_embd.weight` tensor) because the
-  charlie12345/ROCmFPX loader requires it; the root `-ple16` file's per-head
-  split only loads in the LaurentZuijdwijk Vulkan fork (see Known issues #2).
-- imatrix-calibrated on 1540 chunks; +2.48% PPL vs reference on wikitext-2
-  (better than AesSedai IQ4_XS / IQ3_S at smaller size).
-- 18-25% faster prompt processing than a mainline quant of similar size
-  (356 vs 285 t/s at 2048 tok; 301 vs 256 at 8k depth).
-- Ships the model's own f16 vision tower (`mmproj/`).
-- The MTP head is quantized to match the target (ROCmFP4-FAST) — a Q8_0
-  draft measured worse on this head family — but it does not load on
-  build-qwen38 (Known issues #3).
-
-## Runtime
-
-charlie12345/ROCmFPX `llama-server` built at
-`/home/dank/GitPackages/ROCmFPX/build-qwen38` (HIP, ROCm 7.2.4, gfx1151,
-PR #27742 qwen4exp merged). Upstream llama.cpp cannot load the ROCmFP4
-quant types (`TYPE_100`/`TYPE_101`/FAST variants) — verified:
-
-```
-strings build-qwen38/bin/llama-server | grep Q4_0_ROCMFP4_FAST
-```
-
-CIRU binaries (ciru-ai v1.1) do **not** carry the FAST types; do not point
-this model at the CIRU runtime. The build previously served the kingjones
-STRIX / STRIX_LEAN models; the agentionai-FAST pair is the first use of the
-author's own matched MTP head on this binary.
-
-## Files / download
-
-| Item | Path on Dankclaw | Size |
-|---|---|---|
-| Main (v2 joined) | `/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/v2/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-v2.gguf` | 93,484,242,816 B (87.06 GiB) |
-| Main (root ple16, unused) | `/home/dank/models/agentionai-Qwen3.8-Flash-Next-ROCmFP4-FAST/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-v2-ple16.gguf` | 93,484,237,760 B (87.06 GiB) |
-| Vision | `…/mmproj/mmproj-Qwen3.8-Flash-Next-f16.gguf` | 904,004,128 B |
-| Draft head | `/home/dank/models/mtp-Qwen3.8-Flash-Next-FAST/Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST.gguf` | 2,444,519,296 B |
-
-Draft head sha256 (verified against HF LFS):
-`5046d69571bb35c699e19c7a00b36633c2fdfdd06d70482551744ec523a6b590`
-
-Downloaded with a resumable plain `curl -L -C -` (Hugging Face `resolve/main`
-URLs; no auth token needed for public repos). The mmproj lives in an `mmproj/`
-subdirectory of the repo.
-
-## Serving profile
-
-Card sampling table (instruct, non-thinking) mixed with the validated
-build-qwen38 profile:
-
-```
-llama-server \
-  --model Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-v2.gguf \
-  --mmproj mmproj-Qwen3.8-Flash-Next-f16.gguf \
-  --host 0.0.0.0 --port 7770 --jinja \
-  --device ROCm0 --gpu-layers 999 \
-  --split-mode none --fit off --flash-attn on \
-  --ctx-size 32768 --parallel 1 \
-  --batch-size 4096 --ubatch-size 4096 \
-  --cache-type-k f16 --cache-type-v f16 \
-  --threads 16 --threads-batch 16 \
-  --ctx-checkpoints 16 --cache-ram 8192 \
-  --cache-prompt --cache-idle-slots --metrics \
-  --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0.0 --presence-penalty 1.5 \
-  --reasoning on --reasoning-format deepseek --reasoning-budget -1 \
-  --chat-template-file chat-template.jinja \
-  --chat-template-kwargs '{"enable_thinking":true,"preserve_thinking":true,"preserve_reasoning":true}' \
-  --spec-type ngram-mod
-```
-
-Key choices and deviations from the CIRU deployment:
-
-- **`--fit off`** — auto-fit segfaults on qwen4exp hyper-connection config
-  (same as every Flash-Next deployment).
-- **`--checkpoint-min-step` removed** — build-qwen38 does not implement it
-  (verified via `--help`); only `--ctx-checkpoints` is passed.
-- **No PLE sidecar args** — the n-gram table is in-GGUF (joined
-  `per_layer_token_embd.weight` from the `v2/` layout), not a `ple/`
-  CIRUPLE1 sidecar. `--ple-sidecar`/`--ple-cache-mib` and the
-  `GGML_QWEN4EXP_PLE_*` / `GGML_CUDA_Q41_MOE_FORCE_J` env knobs are CIRU-
-  runtime-only and are gone.
-- **`--mmproj` active** — vision tower included (f16, lossless).
-- **`--flash-attn on` (restored 2026-08-29).** fa on previously aborted graph
-  build: the QSA indexer path (layers with attention compression) called
-  `ggml_fill` on the F16-cast kq mask (`GGML_ASSERT(a->type == GGML_TYPE_F32)`,
-  ggml.c:5371, backtrace through `build_attn_qsa`). Fixed in the fork source
-  by the QSA F32-mask patch — vendored at
-  `scripts/patches/qwen4exp-qsa-f32-mask.patch` (sha256
-  `f2f4a486e21cfad80bfa42683281d18e1cba22fa43c5e0e2aa0cfdbe141e925f`);
-  the binary at `build-qwen38/bin/llama-server` (rebuilt 2026-08-29) already
-  carries it. A clean rebuild must re-apply
-  `git apply scripts/patches/qwen4exp-qsa-f32-mask.patch` inside
-  `/home/dank/GitPackages/ROCmFPX` before `make`. Verified live at
-  CTX_SIZE=32768: model loads, health 200, real completion.
-- Context capped at **32768 × 1 slot — the author's own quick-start**. This
-  model is fully GPU-resident (87.06 GiB): the PLE n-gram table sits inside
-  the GGUF as one tensor, unlike STRIX_LEAN which streamed it from RAM.
-  HIP heap is 119.3 GiB; the context buffer is 50.8 GiB at 131072 and
-  103 GiB at 262144, so either window OOMs (`cudaMalloc … failed: out of
-  memory`, verified live on first graph alloc). At 32768 the buffer is
-  ~13 GiB (~100 GiB total) and loads clean.
-- MTP is disabled (ngram-mod only) — see the MTP section below.
-
-## MTP draft head — INCOMPATIBLE on build-qwen38
-
-The author's own matched head (`agentionai/Qwen3.8-Flash-Next-MTP-ROCmFP4-
-FAST-GGUF`, 2.28 GiB, `qwen4exp` arch) does **not** load on this binary:
-
-```
-error loading model: missing tensor 'blk.0.hc_attn_norm.weight'
-```
-
-The build-qwen38 qwen4exp loader requires a full per-layer
-`blk.<il>.hc_attn_norm.weight` set in the draft (the graph reads all 48
-layers); the head only carries `blk.48.hc_attn_norm.weight` (count 1 — all
-three Flash-Next-era heads examined on this fork have that same partial
-layout). Verified three times, not a cross-vendor artifact:
-
-1. EasiiX Q8_0 head vs kingjones STRIX_LEAN main (2026-08-28) — failed.
-2. agentionai FAST head vs kingjones STRIX_LEAN main (2026-08-28) — failed.
-3. agentionai FAST head vs agentionai v2 main (2026-08-29, this deployment)
-   — failed identically.
-
-The 2.28 GiB head stays on disk; the moment a binary or head appears that
-carries the full per-layer norm set, re-enable with
-`SPEC_TYPE=draft-mtp,ngram-mod`, `--spec-draft-n-max 3` (author-measured
-best: 32.4 t/s, 0.612 acceptance; adaptive p-split measured worse).
-
-## Config files
-
-- Env: `scripts/frank-dragoon.env`
-- Serve: `scripts/serve-frank-dragoon.sh`
-- Chat template: `scripts/chat-template.jinja` (unchanged, Qwen-Sharp)
-- Fork patch: `scripts/patches/qwen4exp-qsa-f32-mask.patch` (re-apply after
-  any clean fork rebuild — see Serving profile)
-
-## Swap procedure
-
-```bash
-sudo systemctl stop frank-dragoon
-# edit scripts/frank-dragoon.env + scripts/serve-frank-dragoon.sh, commit, push
-# on Dankclaw: git pull --ff-only && scripts/deploy.sh dankclaw --service frank-dragoon
-sudo systemctl start frank-dragoon   # or via deploy.sh
-curl http://192.168.0.143:7770/health
-```
-
-Model load is slow (87 GiB + 2.3 GiB draft): health returns 503 while
-loading; `deploy.sh` waits up to 20 minutes (`LLM_WAIT`). Do not kill it.
-
-## Known issues
-
-1. **ROCmFP4 types require the ROCmFPX fork.** Stock llama.cpp and the CIRU
-   runtime reject these files.
-2. **Use the `v2/` joined layout, NOT the root `-ple16` split.** This binary's
-   qwen4exp loader (qwen4exp.cpp:104-108) unconditionally requires a single
-   `per_layer_token_embd.weight` tensor and GGML_ASSERTs "qwen4exp is missing
-   the PLE n-gram table" against the per-head split. The root file only loads
-   in the LaurentZuijdwijk Vulkan fork (`vulkan-qwen4exp-rocmfpx`). The v2
-   file is byte-identical in weights, ~5 KB larger header, loads fully into
-   SVM unified memory (no `--ngram-on-disk` needed — kingjones STRIX ran a
-   50 GiB joined PLE table the same way).
-3. **No MTP draft head loads on build-qwen38.** The qwen4exp loader
-   requires a full per-layer `blk.<il>.hc_attn_norm.weight` set; every
-   Flash-Next-era head checked (EasiiX Q8_0, agentionai FAST, CIRU) carries
-   only `blk.48`. Spec decoding is ngram-mod only. See the MTP section.
-4. **`--flash-attn on` (restored 2026-08-29).** fa on aborts graph build
-   without the QSA F32-mask patch — see Serving profile (fix + sha256
-   vendored there).
-5. **`SVM mapping failed` kernel spam** is a documented symptom, not root
-   cause — investigate before blaming memory pressure.
-
-## Citations
-
-- [agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix-GGUF)
-- [agentionai/Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST-GGUF](https://huggingface.co/agentionai/Qwen3.8-Flash-Next-MTP-ROCmFP4-FAST-GGUF)
-- [Qwen/Qwen3.8-Flash-Next base model](https://huggingface.co/Qwen/Qwen3.8-Flash-Next)
-- [charlie12345/ROCmFPX](https://github.com/charlie12345/ROCmFPX) — fork with PR #27742 (qwen4exp architecture) and the ROCmFP4 types
-- [llama.cpp PR #27742](https://github.com/ggml-org/llama.cpp/pull/27742) — qwen4exp support (merged upstream)
-- [llama.cpp PR #27739](https://github.com/ggml-org/llama.cpp/pull/27739) — MTP draft head support

serverAppConfigs-docs/LLM/ciru-activefpx-promptforge.md

@@ -1,266 +0,0 @@
----
-type: reference
-title: CIRU ActiveFPX PromptForge on Dankclaw
-description: jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge (v3) served by the CIRU ROCmFPX llama.cpp runtime on the Strix Halo iGPU. Install runbook, ROCm 7.15 toolchain, and the frank-dragoon service.
-tags:
-  - llama.cpp
-  - rocm
-  - dankclaw
-  - gfx1151
-  - ciru
-  - promptforge
-timestamp: 2026-08-22
----
-
-# CIRU ActiveFPX PromptForge on Dankclaw
-
-> **RETIRED 2026-08-24.** `frank-dragoon` served this v3 model from
-> 2026-08-22 (restored after a same-day Kairic Edge IU4 experiment was
-> reverted — operator decision, commit `c60156f`; see [[kairic-edge]])
-> until [[eemin-rocmi4-mtp]] took over on 2026-08-24. Kept as the
-> historical record of the v3 deployment.
-
-`frank-dragoon` (port 7770, moved from 52625 on 2026-08-22) serves
-`jcbtc/Qwen3.8-27B-CIRU-ActiveFPX-PromptForge` **v3** through the CIRU
-ROCmFPX llama.cpp runtime. This replaced the retired vLLM service that
-previously occupied the slot. The v2 deployment moved to v3 on 2026-08-21.
-
-## What This Is
-
-The model is a **GGUF**, not a vLLM checkpoint. v3 is the consolidated
-quality-and-speed release of Qwen3.8-27B for AMD ROCm: an FFN-equalized
-compact body, 50 retained late-writer Q6 tensors, full-Q8 output
-projection, full-W8 PromptForge view (FFN + Gated DeltaNet), the
-small-row W8 route, a draft-only Output-K8 shortlist proxy with Q8
-reranking, native depth-4 MTP (no ngram), and the BF16 Qwen3.8 vision
-tower + 5120-wide merger as a separate projector.
-
-| Artifact | Size |
-|----------|------|
-| `Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-v3-Q8-FFNEQ-LateQ6.gguf` | 15.44 GiB |
-| `Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-FFN.pfs` | 15.95 GiB |
-| `Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-GDN.pfs` | 3.75 GiB |
-| `Qwen3.8-27B-CIRU-ActiveFPX-PromptForge-Output-K8.pfs` | 0.67 GiB |
-| `Qwen3.8-27B-mmproj-BF16.mmproj` | 0.87 GiB |
-| Full v3 payload (GGUF + 3 .pfs + mmproj) | ~36.7 GiB |
-
-The `.pfs` companions and the Output-K8 proxy are optional. Omit them to
-run the GGUF alone and save ~20.6 GiB of memory. The GGUF remains
-independently usable.
-
-## Requirements
-
-- **ROCm 7.15 is required** for the advertised Strix Halo performance.
-  The system ROCm (`/opt/rocm-7.2.4`) selects slower or fallback kernels.
-- The pinned runtime is `ciru-ai/ROCmFPX` tag
-  `qwen3.8-activefpx-promptforge-v2.3` (commit `103af8b41`) **plus the
-  v3 runtime patch** `runtime/qwen38-v3-output-k8-runtime.patch`. On
-  Dankclaw the patch is commit `af0eac6` on top of the v2.3 checkout.
-- Composable Kernel at `fdf4bb7fcc984811cef48ce817d89aac064b984a`.
-- Validated GPU: AMD Radeon 8060S / gfx1151.
-
-## ROCm 7.15 Toolchain
-
-The retired `ling3-ciru` package left a complete TheRock ROCm 7.15 SDK
-(`7.15.0a20260728`) inside its venv:
-
-```
-/home/dank/ling3-ciru/Ling-3.0-Flash-CIRU-int4-Strix-native/.venv/lib/python3.12/site-packages/
-  _rocm_sdk_devel/      # hipcc, amdclang 23, cmake, headers
-  _rocm_sdk_libraries/  # hipblas, hipblaslt, rocroller
-  _rocm_sdk_core/       # libamdhip64, libhsa-runtime64
-  rocm-sdk-device-gfx1151
-```
-
-The compiler is amdclang 23. `rocm-sdk version` reports `7.15.0a20260728`.
-The binary is built against and run with these 7.15 libraries.
-
-## Build Runbook (2026-08-18, v2.3; +v3 patch on 2026-08-21)
-
-Step 1-4 below built v2.3 on 2026-08-18. On 2026-08-21 the v3
-Output-K8 runtime patch was applied on top and the server rebuilt:
-
-0. Apply the v3 runtime patch to the v2.3 checkout and rebuild:
-   ```
-   % verify the patch hash: 8ba7d721ea380b060aeb0c7595b8b05f1582c0f7bf7a4b89e0910bc8e72465d5
-   git am runtime/qwen38-v3-output-k8-runtime.patch   # -> commit af0eac6
-   cmake --build build-promptforge --target llama-server -j$(nproc)
-   ```
-   The pre-built binary is kept as `build-promptforge/bin/llama-server.v2.3-bak`
-   for rollback.
-
-1. Install host build tools (ninja-build was missing): `sudo apt install ninja-build`.
-2. Clone the runtime and CK at their pinned revisions:
-   ```
-   git clone --depth 1 --branch qwen3.8-activefpx-promptforge-v2.3 \
-     https://github.com/ciru-ai/ROCmFPX.git
-   git clone --depth 1 https://github.com/ROCm/composable_kernel.git
-   git -C composable_kernel checkout fdf4bb7fcc984811cef48ce817d89aac064b984a
-   ```
-3. CK headers are generated at CK configure time (`ck/config.h`,
-   `ck/version.h`). The llama.cpp build includes the CK source tree, so run
-   CK's configure once and copy the headers into `include/ck/`:
-   ```
-   cmake -S composable_kernel -B build-ck -DAMDGPU_TARGETS=gfx1151 -DCK_ENABLE_WMMA=ON
-   cp build-ck/include/ck/config.h build-ck/include/ck/version.h composable_kernel/include/ck/
-   ```
-4. Configure the llama.cpp build against the venv ROCm 7.15 SDK:
-   ```
-   export ROCM_PATH=$SDK/_rocm_sdk_devel
-   cmake -S ROCmFPX -B build-promptforge -G Ninja \
-     -DCMAKE_BUILD_TYPE=Release \
-     -DCMAKE_C_COMPILER=$ROCM_PATH/llvm/bin/clang \
-     -DCMAKE_CXX_COMPILER=$ROCM_PATH/llvm/bin/clang++ \
-     -DCMAKE_HIP_COMPILER=$ROCM_PATH/llvm/bin/clang++ \
-     -DCMAKE_PREFIX_PATH=$ROCM_PATH \
-     -DGGML_HIP=ON -DGGML_CUDA=OFF -DGGML_VULKAN=OFF \
-     -DGGML_HIP_FORCE_MMQ=ON -DGGML_HIP_GRAPHS=ON \
-     -DGGML_HIP_MMQ_MFMA=ON -DGGML_HIP_NO_VMM=ON \
-     -DGGML_HIP_ROCWMMA_FATTN=OFF -DGGML_NATIVE=ON \
-     -DAMDGPU_TARGETS=gfx1151 -DGPU_BUILD_TARGETS=gfx1151 \
-     -DPROMPTFORGE_CK_ROOT=$CK_ROOT \
-     -DLLAMA_BUILD_SERVER=ON \
-     -DLLAMA_BUILD_WEBUI=OFF
-   cmake --build build-promptforge --target llama-server -j$(nproc)
-   ```
-   `LLAMA_BUILD_WEBUI=OFF` is required. The embedded browser UI downloads
-   assets from an HF bucket (`llama-ui`) that returns 404; the service is
-   consumed through the OpenAI-compatible API, so the UI is not needed.
-
-Build output: `build-promptforge/bin/llama-server`.
-
-```mermaid
-graph TD
-    A[Clone ROCmFPX v2.3 + CK pinned] --> B[Configure CK, copy generated headers]
-    B --> C[Configure llama-server vs ROCm 7.15 SDK]
-    C --> D[Build llama-server]
-    D --> E[Verify --list-devices shows gfx1151]
-    E --> F[Download v3 GGUF + FFN/GDN/Output-K8 + mmproj]
-    F --> G[Deploy frank-dragoon]
-    G --> H[Health check + chat completion]
-```
-
-## Model Download
-
-With the HF token to the v3 staging directory
-`/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX-v3/`. The v3 files live there
-so the v2 artifacts in `/home/dank/models/CIRU-Qwen3.8-27B-ActiveFPX/` stay
-intact for rollback. Download ran on Dankclaw with the background script
-`/tmp/download-v3.sh`; all five files + the patch passed SHA-256 checks
-against the model card before the service restart.
-
-## Service
-
-- Unit: `frank-dragoon.service`, port 7770
-- Script: `scripts/serve-frank-dragoon.sh`, env `scripts/frank-dragoon.env`
-- The script exports the ROCm 7.15 SDK libraries in `LD_LIBRARY_PATH` before
-  the system ROCm. `ROCM_PATH` points at the devel SDK.
-
-### Launch profile
-
-- 262144 context (native max), 1 slot (user request 2026-08-19; parallel
-  slots degraded performance), f16 KV (~16 GiB), context checkpoints (16),
-  prompt cache. Concurrent requests queue on the single slot; the server
-  never preempts a running generation
-- Native MTP draft, depth 4, device ROCm0
-- `--temp 0.7 --top-p 0.95`, reasoning on (`--reasoning-format deepseek`
-  puts `<think>` reasoning in `message.reasoning_content`)
-- v3 runtime: Output-K8 draft proxy (top-8 shortlist, Q8 rerank) +
-  small-row W8 prompt route, full-W8 FFN/GDN views
-- PromptForge env: `PROMPTFORGE_MODE=m2048_fused_tail1476`,
-  `GGML_CUDA_GRAPH_OPT=0`, `PROMPTFORGE_MTP_OUTPUT_K8=1`,
-  `PROMPTFORGE_MTP_OUTPUT_K8_VALIDATE=0`,
-  `PROMPTFORGE_ENABLE_SMALLM_W8=1`
-
-### Required environment
-
-```
-export PROMPTFORGE_SIDECAR=...FFN.pfs
-export PROMPTFORGE_GDN_SIDECAR=...GDN.pfs
-export PROMPTFORGE_MODE=m2048_fused_tail1476
-export GGML_CUDA_GRAPH_OPT=0
-export PROMPTFORGE_ENABLE_SMALLM_W8=1
-export PROMPTFORGE_MTP_OUTPUT_K8=1
-export PROMPTFORGE_MTP_OUTPUT_K8_VALIDATE=0
-export PROMPTFORGE_MTP_OUTPUT_K8_PROXY=...Output-K8.pfs
-# v3 qualified profile: disable the CPU argmax draft fast path and any
-# IU4/ngram lane. The Output-K8 proxy + backend sampling replace them.
-unset LLAMA_MTP_CPU_ARGMAX_FASTPATH LLAMA_MTP_BACKEND_GREEDY
-unset PROMPTFORGE_IU4_SIDECAR PROMPTFORGE_GDN_IU4_SIDECAR
-unset PROMPTFORGE_ENABLE_NGRAM_MOD PROMPTFORGE_ENABLE_NGRAM_M65_IU4
-```
-
-The v3 qualified profile runs the MTP draft through the Output-K8 proxy,
-not the CPU argmax fast path. The old `LLAMA_MTP_CPU_ARGMAX_FASTPATH=1`
-profile is retired for this slot. A TOP_K sampler warning may still appear
-in the journal once per start; the Output-K8 draft path does not depend on
-it and generation proceeds.
-
-## Operational Notes
-
-- Health: `curl http://127.0.0.1:7770/health`. Returns 503 while loading.
-- Model load is slow (v3 five-file payload ~36.7 GiB); do not kill the
-  process during warmup. The page cache makes a repeat load fast right after
-  download, but a later cold start takes several minutes.
-- `deploy.sh dankclaw` waits 20 minutes for health.
-- The systemd unit must not set `HOST` or `TEMP` env vars (HIP collision
-  rule). `--host`/`--port`/`--temp` are CLI flags only.
-- v3 adds the BF16 vision tower over v2. The service reports `multimodal`
-  capability; mmproj loads with the language model (CLIP on ROCm0 backend).
-
-## ngram-mod + MTP Stacking (benchmarked 2026-08-21)
-
-This page records a research question: does ngram-mod add value when
-stacked under the native MTP draft? The v3 profile runs MTP alone; the
-benchmark below compares that against the stacked pair.
-
-### Feasibility
-
-ngram-mod DOES stack with MTP in this CIRU fork. The v3 build compiles in
-both `ngram-mod` and `draft-mtp` (`common/speculative.cpp`). Pass
-`--spec-type draft-mtp,ngram-mod` (or `ngram-mod,draft-mtp`) and the fork
-initializes both implementations in the same process. The boot log shows:
-
-```
-adding speculative implementation 'ngram-mod'
-initialized ngram_mod with n_match=24, size=4194304 (16.000 MB)
-adding speculative implementation 'draft-mtp'
-```
-
-The model card's "no ngram subsystem" line refers to the CIRU PromptForge
-env lanes (`PROMPTFORGE_*_NGRAM_*`, `PROMPTFORGE_ENABLE_NGRAM_MOD`), not
-the upstream llama.cpp spec chain. Those env lanes must stay unset; the
-`--spec-type ngram-mod` chain is separate and works.
-
-### Ordering — the CLI order does not matter
-
-The `--spec-type` string order is discarded. `common_speculative_init`
-collapses the requested types into a bitmask, then rebuilds the chain in a
-hard-coded priority: ngram types first, then draft-simple, then MTP.
-So `draft-mtp,ngram-mod` and `ngram-mod,draft-mtp` produce the same
-internal order (`ngram-mod` then `draft-mtp`) and the same runtimes.
-Verified: both orders decode ~25.3 t/s at 8k.
-
-### Results (decode generation t/s, 200 tokens, prose prompt)
-
-| ctx | MTP only | MTP + ngram-mod | verdict |
-|-----|----------|-----------------|---------|
-| 8k | 29.42 | 25.27 | ngram hurts -4.15 |
-| 16k | 26.42 | 21.13 | ngram hurts -5.29 |
-| 64k | 19.72 | 18.00 | ngram hurts -1.72 |
-
-At every depth, adding ngram-mod under MTP REDUCED decode throughput. The
-ngram draft costs its own forward pass / occupancy without adding enough
-accepted tokens on top of MTP. Baseline (no ngram) remains the production
-config (`--spec-type draft-mtp` only). Keep ngram-mod off.
-
-The historical harness was moved to `.AGENTTRASH/bench-scripts-20260901/` and is retained for audit only.
-it boots the v3 stack on a scratch port and reads authoritative
-`print_timing` lines. It requires the production service to be stopped.
-
-## See Also
-
-- [[frank-redo-2026-08-17]] — the fleet topologies this slot has served.
-- [[Shared Memory Architecture Dankclaw]] — unified memory sizing.
-- [[qwen3.8-strix-optimizations]] — llama.cpp ROCm0 baselines on this iGPU.
-- [[VLLM Overview]] — vLLM reference (retired for this slot).

serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md

@@ -1,7 +1,7 @@
 ---
-type: research
+type: reference
 title: Dankclaw IOMMU boot line — decision record (amd_iommu=off 2026-09-11 → iommu=pt 2026-10-18)
-description: Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a measured prefill cost of nothing (+2.0% on byte-identical traffic). Records the 2026-09-11 amd_iommu=off era that bought a vendor-claimed 13-16% figure never measured here, what that cost, the AMD PerfOpt research (Linux 7.4), the tooling that owns the boot line, and the revert path.
+description: "Why Dankclaw boots with identity DMA, what the earlier off-era bought and cost, and which tooling owns the boot line."
 tags:
   - dankclaw
   - iommu
@@ -205,7 +205,7 @@ answered 200. (Later the same day the pool DOUBLING to 401,408 positions for
 slot parity was COMMITTED (`2b12716`) and **deployed the same evening**: the
 boot now reserves `401408 positions (about 11.0 GiB)` and the engine-side total
 is `79.6 GiB in all` (62.9 weights + 11.0 KV + 5.7 working),
-see [[frank-dragoon-flash-server-v2-native]].)
+see [[frank-dragoon]].)
 
 **The NPU came back with the flip.** `amdxdna` bound to `0000:c6:00.1` (Strix/
 Krackan/Strix Halo NPU rev 11), firmware `amdnpu/17f0_11/npu_7.sbin`, `Initialized
@@ -229,8 +229,8 @@ lives in `~/.local/bin`.
 ## Related
 
 - [[frank-npu]] — the endpoint this retired, including the err=-19 root cause
-- [[halogen-flash-server]] — the runtime bought by the prefill figure
-- [[kingjones-strix-rocmfp4]] — the runtime it replaced (rollback target)
+- [[frank-dragoon]] — the runtime bought by the prefill figure
+- [[frank-dragoon]] — the runtime it replaced (rollback target)
 
 ## Citations
 

serverAppConfigs-docs/LLM/dankserver-frontier-moe-feasibility.md

@@ -1,7 +1,7 @@
 ---
-type: research
+type: reference
 title: Frontier MoE on Dankserver — what fits, on which runtime
-description: VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash against the 2x R9700 (gfx1201) / 246 GiB box, and assesses the Affinity runtime's portability off DeepSeek-V4-Flash. Records the capacity arithmetic that decides the answer before any model is loaded.
+description: "The capacity arithmetic that decides whether a frontier mixture-of-experts fits this box before any model is loaded."
 tags:
   - dankserver
   - r9700
@@ -166,7 +166,7 @@ Not advisable.
 | Runtime | gfx1201 | Architectures | Note |
 |---|---|---|---|
 | **Affinity** | yes, built here | **V4-Flash only** — "the geometry is compiled in; there is no architecture abstraction" | 110 tok/s for 284B. Best engine for this box by an order of magnitude, wrong model |
-| **tcclaviger vLLM fork** | **yes — proven 2026-09-29** | qwen4exp measured 37.4-40.6 tok/s decode (see [[dankserver-qwen38-fp8-expert-offload-tuning]]); **`glm5next` registered** (`Glm5NextForCausalLM`/`MTP`, backend-split `amd/`/`common/`/`nvidia/`, `sparse_indexer.py` for DSA); r4d.so has **native mxfp4 + dsfp4 MoE entries** (`has_dsfp4() == True`); W4A16 MoE class (`R4dW4a16MoEExperts`) wired into the same tiered offload; TunableOp silently skipped under offload (needs `CLAV_TUNABLEOP_WITH_OFFLOAD=1`, and then corrupts numerics — do not use) | the ad hoc qwen-fp8 container's runtime; **the old "MXFP4 = emulation/missing kernels" verdict predates this fork and is dead** — the GLM-5.3-Flash gate is now the payload format + a boot probe, not kernels |
+| **tcclaviger vLLM fork** | **yes — proven 2026-09-29** | qwen4exp measured 37.4-40.6 tok/s decode (see [[frank-archon]]); **`glm5next` registered** (`Glm5NextForCausalLM`/`MTP`, backend-split `amd/`/`common/`/`nvidia/`, `sparse_indexer.py` for DSA); r4d.so has **native mxfp4 + dsfp4 MoE entries** (`has_dsfp4() == True`); W4A16 MoE class (`R4dW4a16MoEExperts`) wired into the same tiered offload; TunableOp silently skipped under offload (needs `CLAV_TUNABLEOP_WITH_OFFLOAD=1`, and then corrupts numerics — do not use) | the ad hoc qwen-fp8 container's runtime; **the old "MXFP4 = emulation/missing kernels" verdict predates this fork and is dead** — the GLM-5.3-Flash gate is now the payload format + a boot probe, not kernels |
 | **Lucebox** | yes, image on host | Qwen 3.8, DS4; **V4.1 on the unmerged `feat/ds41-engine` branch** | The only V4.1 path. **Refuses layer split and layer-major prefill for V4.1** |
 | **radiance-vLLM** (drwolfen) | yes — served frank-archon 2026-09-27 → 09-30 | welded to `qwen4_exp`; PLE / QSA / packed-HC live inside that model class | 12–20 person-days per architecture |
 | **FreeToken-ROCm** | partial | block-FP8 safetensors only (modelopt); known `CpuMoeExecutor` issue under ROCm | cannot load a GGUF |
@@ -221,7 +221,7 @@ gfx1201+gfx1201. hip:0 takes dense weights and the hot expert stack;
 
 ## See Also
 
-- [[frank-archon-lucebox-ds4]] — the retired Lucebox frank-archon runbook
+- [[frank-archon]] — the retired Lucebox frank-archon runbook
 - [[Dual-R9700-Cooperation]] — the PCIe/atomics facts behind the board swap
 - [[Dankserver System Specs]] — the host (RAM corrected 2026-09-28)
-- [[frank-archon-r9v-radiance-qwen38-q8]] — the ACTIVE frank-archon
+- [[frank-archon]] — the ACTIVE frank-archon

serverAppConfigs-docs/LLM/dankserver-qwen38-fp8-expert-offload-tuning.md

@@ -1,724 +0,0 @@
----
-type: research
-title: Qwen3.8-Flash-Next FP8 on tcclaviger — expert-offload decode tuning
-description: VERIFIED 2026-09-29 by measurement. The flag-level decode-tuning campaign on the ad hoc qwen-fp8 container (orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8, tcclaviger RDNA4 vLLM fork, 2x R9700): the >35 tok/s target is MET at 37.4-40.6 tok/s by turning prefetch OFF — telemetry measured its precision at 36%, wasting ~half of all gather traffic — while the original 26.8 tok/s incumbent (prefetch on) survived every other challenger. Residual upside is the fork's gather read path (effective ~6.5-9.5 GB/s vs 22.7 GB/s the link delivers at the same slab size).
-tags:
-  - dankserver
-  - r9700
-  - gfx1201
-  - qwen3.8
-  - fp8
-  - expert-offload
-  - speculative-decoding
-  - tuning
-timestamp: 2026-09-29
----
-
-# Qwen3.8-Flash-Next FP8 — expert-offload decode tuning
-
-> **Question this page answers.** On the tcclaviger RDNA4 vLLM fork serving
-> `orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8` across both R9700s, what
-> runtime flags actually move decode at a 32k working point?
->
-> **Answer in one line.** **37.44–40.57 tok/s — the >35 target is met — by
-> deleting `--expert-prefetch 1`.** Expert-cache telemetry measured the
-> prefetcher at **36% precision** (36,720 slabs prefetched, 13,369 served):
-> ~half of all gather traffic was useless copies, and the copies it "saved"
-> would have been installs anyway. The prefetch-on incumbent (26.8 tok/s)
-> survived every other challenger measured: gather geometry, spec depth,
-> TunableOp, scheduler batch knobs, concurrency, ngram-only drafting.
-
-This is the ad hoc `qwen-fp8` container on Dankserver (rootless podman,
-image `docker.io/tcclaviger/vllm:latest`, engine
-`v0.29.0.dev0+g2bdbbc8080`), NOT the `frank-archon` systemd service —
-the R9V production unit is untouched, so no stop-and-mask was required for
-any of this. All numbers are decode tok/s measured from the server's own
-`usage.completion_tokens / (wall - ttft)` — SSE chunk counting is invalid
-under speculative decoding because rejected drafts change the accounting.
-
-## Deployment shape (the tuned incumbent)
-
-Serve script on the host (`/home/dank/serve_qwen_fp8.sh`), env hooks in
-CAPS. **PREFETCH must be unset** (omitting the flag; the fork's default is
-off):
-
-```
-MODEL=/home/dank/models/Qwen3.8-Flash-Next-Uncensored-FP8   (~172.8 GiB, 131 shards)
-IMAGE=docker.io/tcclaviger/vllm@sha256:da4029666668b90a0386c1ed23ea370c79916ad4e1a6ad1dd9db098b9fa069ea
-# image ID 3322dbbbc96d; local alias qwen-fp8-runtime:tuned-20260929;
-# the floating :latest tag moved once already — always launch by digest.
-# repo tag of this state: qwen-fp8-tuned-20260929
---tensor-parallel-size 2  --max-model-len 32768  --gpu-memory-utilization 0.98
---kv-cache-memory-bytes 2147483648  --max-num-seqs 4  --enable-prefix-caching
---speculative-config '{"method":"mtp","num_speculative_tokens":4}'
-# NO --expert-prefetch: measured 36% precision, ~50% wasted gather traffic
-```
-
-Published to the host LAN via the serve script's `PORTMAP` hook
-(`PORTMAP=7773:7770`): health and completions answer on
-`http://192.168.0.235:7773` (verified host-side 2026-09-29; 256k config,
-decode 36.49/34.55/35.98 through the published path). Host port 7773 was
-chosen over 7770 because `frank-archon` (7770) is stopped but ENABLED —
-any future start would collide with a published 7770.
-
-Device notes carried from the bring-up: `--device /dev/dri` must be the
-WHOLE directory (individual renderD nodes enumerate but refuse contexts —
-"No CUDA GPUs are available" while device_count() reports 2); the image
-entrypoint execs `vllm serve` itself.
-
-The memory plan this produces (from the engine's own `expert_plan.py` log):
-
-| region | size |
-|---|---:|
-| expert cache space | 15.05 GiB = 6,574 slots x 2.344 MiB |
-|   VRAM-only resident layers | 3.52 GiB (layers 0-2, 1,536 experts) |
-|   swap region | 11.53 GiB (5,038 slots, 111-112/layer) |
-| host-backed experts | 52.74 GiB per rank, 23,040 experts |
-| KV reserve (one max-len request +8%) | 0.36 GiB of the 2 GiB pool |
-| CUDA graphs | 1.93 GiB (21 graphs; 1.32 GiB at `--max-num-seqs 2`) |
-| runtime overhead | 7.6 GiB (non-torch + buffers + activation) |
-
-## Measured arms (2026-09-29)
-
-Decode tok/s at 166 / 1087 / 4064 prompt tokens, median of 3 runs after a
-64-token warm pass, temperature 0, max_tokens 256:
-
-| Arm | 166 | 1087 | 4064 | Acceptance | Verdict |
-|---|---:|---:|---:|---:|---|
-| **MTP4, stock 8x16, PREFETCH OFF** | **40.57** | **37.44** | **37.57** | 54.0% | **NEW INCUMBENT — target met** |
-| MTP4, stock 8x16, prefetch ON | 26.80 | 25.15 | 26.81 | 60.7% | old incumbent (+51%/+40% worse) |
-| MTP4, prefetch ON (re-check) | 25.80 | 24.38 | 26.63 | 47.9% | reproduces (noise band ±1) |
-| MTP4, prefetch ON, gather 16x32 | 24.30 | 22.77 | 24.11 | 50.3% | −9% — ruled out |
-| MTP4, prefetch ON, gather 32x64 | 26.58 | 24.94 | 26.34 | 49.6% | parity — ruled out |
-| MTP3, prefetch ON | 24.47 | 24.41 | 24.44 | 66.7% | −9%, flat in ctx — ruled out |
-| MTP6, prefetch ON | 19.07 | 17.81 | 13.54 | 17.4% | collapses; crosses UVA threshold — ruled out |
-| MTP4, prefetch ON, MNBT 8192 | 29.32 | 24.12 | 23.65 | — | neutral — ruled out |
-| MTP4, prefetch ON, PREFETCH depth 2 | 25.83 | 25.06 | 25.87 | 56.0% | neutral — ruled out |
-| MTP4, prefetch ON, TunableOp w/ offload | 23.59 | (79.82*) | (79.89*) | (96.5%*) | **rejected — corrupts** |
-| 2-way concurrency, prefetch ON (2 streams) | — | — | 15.06 + 23.11 = **25.49 summed** | — | no gain vs 26.8 single |
-| ngram-only drafter | — | — | — | — | fork bug: PLE inputs unprepared |
-| 256k ctx (MAXLEN=262144, KV 4.5 / 4.03 GiB, prefetch off) | 36.65 / 36.48 | 34.21 / 34.13 | 34.77 / **35.02** | 53.0 / 39.4% | boots and serves; band 34.1-36.5 straddles 35 within noise; arena identical (5,648) at both KV sizes |
-
-\* The TunableOp numbers are invalid by construction: the tuned GEMMs
-changed generated behavior (repetition-loop, the server's
-"repetition detected" halt firing at ~140-146 tokens instead of the full
-256, acceptance jumping 60.7% → 96.5%), the 79.8 tok/s figures were
-measured on those halted degenerate completions, and the text-only check on
-a fresh prompt came back 20.01 tok/s — *below* baseline. GEMM autotuning
-that changes output text is broken tuning.
-
-The decode curve without spec decode is dead flat 12.11 tok/s from 304 to
-16,095 tokens — decode is context-independent on this shape, bound per-step
-by the expert gather, not by KV depth.
-
-## Telemetry — the measurement that found the win
-
-`R4D_LRU_TELEMETRY=1` arms per-layer counters; a torch-profiler window
-(`/start_profile` → traffic → `/stop_profile`) makes the fork log and write
-`expert_cache_rank<N>.json` beside the traces. **`torch_profiler_with_stack=false`
-is mandatory in the profiler config** — the default (true) grew a 6.5-min
-window's trace buffer ~40 GB and the kernel OOM-killed the engine core
-(anon-rss 95.2 GB, `oom-kill` in `journalctl -k`). The short window at
-166 ctx (~15 s of traffic) completed cleanly with `with_stack=false`.
-
-The fork's own numbers (102 steps, both ranks identical):
-
-```
-hit rate 0.8245 (137,084/166,262)   host installs 15,814
-read-through 13,364                 prefetched 36,720
-served by prefetch (migrations) 13,369
-```
-
-- **The LRU is good**: 82.45% hit rate overall; layers 0-2 pinned at
-  100%; swap-region layers 78-81%. The "arena holds only 27% of experts
-  → 73% miss" arithmetic was wrong — temporal reuse carries it.
-- **The prefetcher is the poison**: 36,720 slabs moved to serve 13,369
-  (36% precision). The 23,351 unused copies are ~0.7 GB/step/rank of pure
-  waste — roughly half of all gather traffic — and the 13,369 "savings"
-  would have been installs anyway. Depth-1 prefetch also cannot cover the
-  ~2,400-slab per-step working set with its 16 slots/layer (768 total).
-- Per-layer-step at a swap layer: ~30.6 routed, 24.8 hits, 3.3 installs,
-  2.6 read-through, 8.0 prefetched of which 1.7-3.3 useful.
-
-## The wall, measured
-
-- **PCIe link (pcie-link-probe):** GPUs `03:00.0`/`07:00.0` LnkCap/LnkSta
-  32GT/s x16; root ports `00:01.1`/`00:01.3` cap x8. Effective **Gen5 x8 =
-  31.5 GB/s/direction**. `rocm-smi pcie_link_speed: 25` is the ASPM idle
-  downshift, not the loaded rate.
-- **H2D (HIP probe, pinned):** 2.345 MiB (the engine's exact expert slab)
-  **22.74 GB/s**; 16 MiB 28.19; 1 GiB 28.74 (91% of ceiling). Pinned ≈
-  pageable — staging is not the lever. Single-transfer latency at slab size
-  105.7 µs vs 78 µs ideal → ~21% per-transfer overhead.
-- **Engine's effective gather rate:** ~6.5-9.5 GB/s = **30-42% of what the
-  link delivers at the same size**. The bottleneck is the fork's read path
-  (chunk/lane geometry, per-transfer cost), not the wire.
-- **Duty cycle:** peak GPU util 100%, smooth ~29% with prefetch on — each
-  step is ~100 ms of compute then hundreds of ms waiting on expert
-  streaming. rocm-smi reports *utilization*, which reads ~0 while weights
-  load during an arm's reload; VRAM is the honest panel there.
-
-## What the remaining flags cannot do (all measured)
-
-- **Gather geometry** — 16x32 and 32x64 measured no better despite the
-  fork's "tuned at the Gen4 ceiling" comment; the constants are already at
-  parity on Gen5 x8. The read-path gap is structural, not a tuning knob.
-- **Spec depth** — 4 is the optimum both ways (with prefetch on). Depth 6
-  additionally proves the UVA read-through path (`step tokens >
-  R4D_LRU_UVA_TOKENS=5`) is *slower* than copy-in.
-- **Prefetch anything** — depth 2 neutral, depth 1 the new negative; the
-  tier is 3x too small for the working set and the predictor is 36%
-  precise. `expert_prefetch_k/budget/slots` exist (defaults 20/14/16) but
-  the precision result makes tuning them pointless.
-- **Arena growth via shrink** — `UTIL=0.99` fails boot (`Free memory on
-  device cuda:1 (31.29/31.86)`); `--max-num-seqs 2` fails CUDA-graph
-  capture (`Constraints violated (L['query_start_loc'].size()[0])`); and
-  the MNBT 8192 arm already proved a ~1 GiB arena shift is decode-neutral
-  in both directions.
-- **Concurrency** — under TP=2 both streams' tokens are computed on both
-  cards, so each rank gathers the union of both streams' misses; the links
-  do not add capacity and the shared arena thrashes. Summed 2-way (25.49)
-  measured *below* single-stream (26.8) with prefetch on.
-- **ngram-only drafting** — fails at engine init:
-  `RuntimeError: PLE inputs were not prepared`; the fork prepares this
-  model's PLE inputs only on the MTP spec path. Needs a fork patch, not a
-  config.
-- **V-only q8 KV** — stock `--kv-cache-dtype` offers fp8 variants only
-  (K+V together); a V-only q8 needs a fork patch.
-
-## Fork bugs / traps found (for the engine authors or a future patch)
-
-1. `CLAV_TUNABLEOP_SWEEP=1` is silently skipped whenever expert offload is
-   active unless `CLAV_TUNABLEOP_WITH_OFFLOAD=1` is also set
-   (`tunableop_warmup.py`).
-2. TunableOp-with-offload, once forced on: +6.7 min warmup, no persistent
-   cache (container-local), and corrupt numerics (repetition loops,
-   early-EOS halts, invalid measurements). Rejected outright.
-3. The torch-profiler stop path hangs the workers on ROCm when
-   `torch_profiler_with_stack` is true (default): >10 min hangs on two
-   attempts while the AsyncLLM wrapper reports "stopped successfully" in
-   1 s; the default stack-tracing grew a 6.5-min window's buffer ~40 GB
-   and OOM-killed the engine core. With `with_stack=false` the stop
-   completes and the telemetry report runs. The telemetry counters remain
-   reachable ONLY through that profiler stop — no direct RPC trigger
-   exists (`telemetry_report` has exactly one caller, `gpu_worker.py`).
-4. ngram spec method does not prepare PLE inputs (above).
-
-## Where the headroom lives now
-
-The 35 tok/s target is met (37.4-40.6) with prefetch off. What is left:
-
-1. **Fork patch to the gather read path** — the ~6.5-9.5 → 22.7 GB/s
-   software gap at slab size. If the gather reached wire rate, decode
-   lands in the 60+ tok/s band before acceptance effects. Code work in
-   `expert_lru.py`'s gather (`R4D_LRU_GATHER_*` is read at import; the
-   copy-in path itself is compiled — see `/app/r4dhip/r4d.so`).
-2. **256k ctx — measured, final**: boots at `MAXLEN=262144` with
-   `KV_BYTES=4831838208` or `4328521728` alike — a 262,144-token window
-   needs **4.0 GiB** KV (hybrid attention + mamba pages ≈ 16 KiB/token,
-   not the 32k plan's 9,190 B/token; 3 GiB boot-refuses and the engine
-   estimates max len 184,224 at that pool; the exact-4.0 override loses
-   ~20 MB to accounting and boots as 3.98 < 4.0). Decode band
-   **34.1-36.5** across both KV sizes — the arena is **identical (5,648
-   slots)** at 4.5 and 4.03 GiB (the planner buckets; the trim went to
-   margin, not slots), so the window costs ~10% decode vs the 32k config
-   and no KV trim recovers it. The alternative that keeps the full
-   37.4-40.6 band is `MAXLEN=184224` at 3 GiB KV. Correction: the planner
-   sizes the arena AFTER the KV override (the earlier "before" reading was
-   wrong) — the arena is what remains.
-3. **2+ slots at long ctx** — the KV pool post-spec holds ~66k tokens at
-   2 GiB (~2x 32k streams); concurrency summed *below* parity at 4064 ctx,
-   so slots are a capacity feature, not a throughput one. Two full 256k
-   streams need ~8-10 GiB of KV — that fits the arithmetic only by cutting
-   the arena below usefulness; slots-at-256k is a fork-patch or
-   smaller-payload question.
-
-## Reproducing
-
-Host-side scratch scripts (2026-09-29): `/home/dank/serve_qwen_fp8.sh` (the
-serve script with all hooks), `/home/dank/mtp_measure.py` (usage-counter
-decode measure, 3 depths), `/home/dank/mtp_measure_short.py` (single depth,
-for profile windows), `/home/dank/bench2way.py` (concurrency), plus the arm
-scripts `arm_*.sh` / `sweep_*.sh` and the detached runner `/home/dank/job.sh`
-(outputs under `/home/dank/jobs/NAME.out|.rc|.done`). The measure scripts
-are `podman cp`'d into the container after every recreate (the serve script
-re-copies them; a recreate wipes container-local files).
-
-## 2026-10-01 — production cutovers (qwen FP8+MTP4, then bf16 no-spec)
-
-Two same-day cutovers moved this page's findings into production:
-
-1. **Qwen FP8 + MTP4 replaced the failed GLM/DFlash2 experiment** (the GLM
-   window is documented in [[frank-archon-glm53-flash-w4a16]]; git tag
-   `frank-archon-fp8-mtp-20261001` is the last commit of that shape). The
-   production trio took MAXLEN 200000 x 3, fp8 KV, MTP4, prefetch off —
-   measured 37.4-40.6 tok/s at ~54% acceptance.
-2. **The overnight redo campaign re-measured EVERY lever at matched
-   geometry** (prefetch OFF everywhere, MAXLEN 200000/SEQS 3/fp8 baseline)
-   and concluded the day: the incumbent cutover to **bf16 KV, no
-   speculative decode** (Frank's call).
-
-### What the redo campaign measured (all at shallow depth, 166/1087/4064)
-
-| lever | verdict |
-|---|---|
-| prefetch (re-confirmed) | the whole night's +40-51% lives in deleting it; the old 12.11 "no-spec baseline" was itself prefetch-taxed |
-| gather geometry | **R4D_LRU_GATHER_CHUNKS=32 / LANES=64 = +18.7%** (49.25 @4064), the win GROWS with depth (39.5@166 -> 54.1@1087); g8x16 mild, g64x128 past peak |
-| KV dtype | bf16 (`auto`) +4% decode on fp8-era numbers but halves attention-KV capacity (measured 14,047 vs 7,926 B/token; mamba states unaffected) |
-| spec family | no-spec fastest once the tax was gone (41.46/41.29/41.47); MTP4's drafter reserves ~2.6 GiB/rank beside the pool; untaxed MTP re-measures never completed |
-| MNBT | 512 doubles TTFT (2.46 s vs 1.36 s @4064); 1024-4096 interchangeable; 12288 -2.5% + arena; **locked 1024** |
-| UVA | 2048 POISON (-39%); 8192 ~parity |
-| arena | flat ±1.4% from ~6,830 down to ~5,500 slots; knee below |
-| six slots | every SEQS-6 KV sizing refused at engine init |
-
-Depth caveat: every number above is at a near-empty KV. Deep-context
-behavior (the gather win growing with depth, bf16's decode cost at depth)
-is only sampled at 3 depths; a 64k-192k-depth probe is the flagged gap.
-
-### The deployed shape (2026-10-03 shape G: 3 slots x 200704, fp8, precise memory, MNBT 2048, arena at the flag ceiling)
-
-```
-FRANK_ARCHON_MAXLEN=200704   200704-token window
-FRANK_ARCHON_SEQS=3          --max-num-seqs 3 (SEQS=2 CANNOT boot - below)
-FRANK_ARCHON_KV_BYTES=3166500000   PER RANK (fp8, 7,892 B/token -> 400,700 tokens)
-FRANK_ARCHON_KV_DTYPE=fp8    --kv-cache-dtype fp8 (quantized)
-FRANK_ARCHON_MNBT=2048       --max-num-batched-tokens 2048 (arena-free, see shape F)
-FRANK_ARCHON_UTIL=0.982      budget = total x util, the law (1.0 unbootable, see below)
-FRANK_ARCHON_CUDAGRAPH_BASE_MB=0 / _PER_GRAPH_MB=12  -e VLLM_CUDAGRAPH_MEM_*
-FRANK_ARCHON_PRECISE_MEMORY=1      --expert-precise-memory (overhead 7.60 -> 5.25 GiB)
-FRANK_ARCHON_EXPERT_OFFLOAD_MEM=120  --expert-offload-mem (host-region cap, explicit)
-FRANK_ARCHON_GPU_MAX_HW_QUEUES=2   -e GPU_MAX_HW_QUEUES=2 (HSA queues/GPU)
-FRANK_ARCHON_CACHE_DIR=/home/dank/.cache/frank-archon   mounted at /root/.cache
-R4D_LRU_GATHER_CHUNKS=32 / R4D_LRU_GATHER_LANES=64  (container env)
-NO --speculative-config, NO --expert-prefetch
-FRANK_ARCHON_NO_PREFIX_CACHING=1  --no-enable-prefix-caching (2026-10-05, the QFN align-mode GDN state fault — below)
-```
-
-### 2026-10-05 — prefix caching OFF: the fork's QFN align-mode GDN state fault
-
-**Symptom (Frank, 2026-10-04):** output turned repetitive/corrupted at
-~40k tokens into a long conversation; short sessions stayed clean.
-
-**Evidence (live `podman logs frank-archon-qwen`, the 2026-10-04 18:19
-boot):** one 0.6 s burst at 18:46:07 PDT — 5.6 h into service, one
-running request, and the engine's own 18:46:08 log line reading
-"Prefix cache hit rate: 48.1%". 48 `gdn_hip` faults, then the
-process-lifetime silence cap:
-
-```
-[gdn_hip] ERROR gdn_decode_r: state out of ssm-cache range: max|S|=0
-  with non-finite values (slot=42 head=3 limit=3.40282e+38) fault #1-23
-[gdn_hip] ERROR gdn_decode_r: ... (slot=42 head=8 ...) fault #24-46
-[gdn_hip] ERROR: state fault report limit (32) reached; further faults
-  are counted silently
-```
-
-One physical SSM slot (42), two heads (3 and 8 — one per TP rank), 23
-GDN layers each, the state non-finite (NaN). The guard's own text in
-this build: *"known Qwen3.8-Flash-Next (QFN) flaw under prefix caching
-and is being actively worked; output quality for this request may
-degrade. Mitigation: --no-enable-prefix-caching."*
-
-**Mechanism.** With prefix caching on, the fork defaults QFN to
-`mamba_cache_mode=align` (boot log: "Mamba cache mode is set to 'align'
-for Qwen4ExpForConditionalGeneration … when prefix caching is enabled";
-`config/cache.py`: "none" is the no-prefix-cache mode, "all" is
-deprecated — align is the only sound mode WITH the flag). Align mode
-checkpoints each sequence's GDN (Gated DeltaNet) recurrent state into a
-mamba page per block and RE-SEEDS the state from that page when a
-request resumes from a cached prefix. The block size is raised to
-**3136 tokens** for the page-size equality (boot log: "Setting
-attention block size to 3136 tokens to ensure that attention page size
-is >= mamba page size" + "Padding mamba page size by 0.13% … exactly
-equal"). The corrupt state landed at the **13th boundary: 13 × 3136 =
-40,768 tokens** — the reported "~40k" mark. The NaN persisted in the
-mamba page, so every later turn of that conversation re-seeded the same
-non-finite state for those two heads (silently — the fault-report cap
-was already spent) while fresh short sessions looked fine.
-
-**RAM ruled out the same day** (the 2026-10-04 17:17 host reboot is when
-the current memory speed took effect): zero MCE/EDAC kernel records
-since boot; an 8 GiB in-band pattern check stable over 60 s under live
-load; 4800 MT/s is the JEDEC DDR5 floor on 5600 MT/s-rated DIMMs (not an
-overclock); and a two-head, single-slot NaN state is a state-path fault,
-not scattered finite bit corruption.
-
-**Fix.** `FRANK_ARCHON_NO_PREFIX_CACHING=1` → `--no-enable-prefix-caching`
-(the fork-named mitigation; the flag renders in both directions from the
-knob so a fork default change cannot silently flip the contract). Cost:
-no prefix reuse — every turn re-prefills the whole history instead of at
-most 3135 tail tokens. A silently corrupted conversation is worse than a
-slower first token.
-
-**Verified 2026-10-05 (deploy b820d34):**
-
-- Engine argv carries `--no-enable-prefix-caching`; health 200; warm boot
-  `init engine … took 76.57 s` (the mounted compile cache is intact).
-- Boot plan unchanged, as predicted: expert cache **17.77 GiB / 7,762
-  slots** (the flag touches no plan term), CUDA-graph estimate 0.07 GiB
-  vs 0.08 GiB captured (inside the 12 MiB/graph charge).
-- `GPU KV cache size: 424,678 tokens` — **up** from 401,408 at the same
-  3,166,500,000 B ask: the pool now prices ~7,456 B/token (was 7,891.8),
-  the mamba/side per-token term dropping once the per-block checkpoint
-  pages are gone; the 3136 attention block-size raise nonetheless stays
-  (the page-size invariant is enforced in mode "none" too). Frank's
-  400,700-token floor is cleared with ~24k of headroom.
-- **Zero `gdn_hip` lines** in the boot-and-traffic window (only the two
-  INFO module banners mention the name).
-- Boundary probe (single request, no prefix reuse): prompt 39,280 +
-  completion 1,491 = **40,771 total tokens — across the 40,768 line**;
-  the task (write 1–400, one per line) completed coherently: 399 integers
-  parsed, 0.997 monotone, tail `… 398 399 400`. Pre-fix, this boundary is
-  exactly where the conversation degraded (the 13th 3136-page re-seed).
-- e2e: 32/33 pass; the one failure is the admin spec's
-  192.168.0.112:8989 leg with FrankBook-Pro asleep (environmental,
-  unrelated to this change; re-run `tests/e2e/run.sh` once the mini is up).
-
-**RESULT — REVERTED THE SAME DAY.** With every mechanical check green
-(zero faults, coherent 40,771-token probe), live long conversations
-STILL degraded at ~40k on the running endpoint (Frank, direct
-observation). The container serving that window (`--no-enable-prefix-
-caching`, booted 2026-10-05 12:47 PDT) logged ZERO `gdn_hip` faults
-while the corruption reproduced — so this recurrence is **fault-free**:
-the NaN-state guard did not fire, which the re-seed theory requires.
-Open hypotheses, in order: (1) the poisoned transcript is being re-fed —
-corrupted assistant turns already in the history keep producing
-corrupted continuations even with perfect state, so the same
-conversation cannot distinguish fixed from unfixed; (2) a second
-long-context mechanism with no guard coverage — fp8 KV degradation at
-depth is the next suspect (a bf16-KV boot costs ~2× the pool per token
-and would settle it). Prefix caching is **back ON**
-(`FRANK_ARCHON_NO_PREFIX_CACHING` commented out — the cost bought no
-fix). The knob and the serve branch stay for a future fork build with a
-fixed GDN path. Next diagnostic: reproduce on a FRESH conversation; if
-it still breaks at ~40k with zero faults, A/B the KV dtype.
-
-**RESOLVED (2026-10-05 evening): it was the RAM.** Frank dropped the
-memory from 4800 to **4600 MT/s** (in effect since the 2026-10-05 12:21
-PDT host boot) and the corruption stopped: zero `gdn_hip` ERROR lines in
-the container since, zero MCE records, and the live agent session on
-this endpoint past 140k context, "comfy cozy" (Frank). Marginal DDR5 on
-four DIMMs is a common cause for BOTH eras: the 2026-10-04 NaN burst
-(corrupted weight/activation traffic poisoning a GDN state write — the
-guard-visible path) and the fault-free ~40k degradation (silent flips in
-KV/expert data, which no guard covers on non-ECC memory). The re-seed
-theory is therefore UNPROVEN, not disproven — the entire caching
-experiment ran on marginal RAM and was confounded on both arms.
-Post-mortem on the rule-out: consumer non-ECC RAM raises no MCE for a
-silent flip; a 60 s / 8 GiB pattern sample covers ~0.5 % of 246 GiB;
-"4800 is the JEDEC floor on 5600-rated DIMMs" is a rating, not a
-stability proof at four-DIMM loading. The revert test was the decisive
-instrument and it pointed at the RAM all along. Prefix caching stays ON
-(zero faults under heavy re-send traffic since the speed drop); the knob
-stays for a future fork fix. The fp8-KV A/B is NOT needed.
-
-### Shape F/G (2026-10-03): where the free VRAM actually is, and what forcing it cost
-
-`rocm-smi` showed **3.57 GiB free per R9700** at the deployed shape E (28.28 GiB
-used of 31.86 GiB, after hours of real traffic — so it is the high-water mark,
-not an idle artifact). Growing the expert arena into it is NOT a matter of
-raising `--gpu-memory-utilization`, because the arena is
-
-```
-arena = budget - non-expert - KV reserve - CUDA graphs - runtime overhead
-budget = device_total x gpu_memory_utilization          # expert_plan.py:567
-```
-
-and every term on the right is either fixed or a lie:
-
-| term | plan charges | measured reality | reachable from our side? |
-|---|---|---|---|
-| `budget` | 31.22 GiB (0.98) | field is `le=1` so 31.86 GiB is the field max, but 1.0 is UNBOOTABLE (below) | **yes** — shipped 0.982 |
-| `non-expert weights` | 6.28 GiB | checkpoint census | no |
-| `KV reserve` | 1.91 GiB (registry 9,190 B/token x 200704 +8%) | real pool bills 2.93 GiB at 7,892 B/token | no (registry constant) |
-| `CUDA graphs` | 0.55 GiB = 256 MiB + 52 MiB x 6 | `Graph capturing finished in 3 secs, took 0.31 GiB` | **yes** — `VLLM_CUDAGRAPH_MEM_BASE_MB` / `_PER_GRAPH_MB`; `VLLM_CUDAGRAPH_MEM_ESTIMATE_MB` pins the whole term outright (checked first, pair ignored when set) |
-| `runtime overhead` | 5.25 GiB (`--expert-precise-memory`) | far lower at idle | no (registry constant) |
-
-Shape F took the graph term (0.55 -> 0.32 GiB) and nudged util to 0.982:
-**arena 17.52 GiB / 7,653 slots measured**, +131 slots. Shape G takes the
-graph term further (0.32 -> 0.07 GiB) at the same util.
-
-**Shape G takes the graph term to the measured floor, util to the law.** The
-graph base pool is the actual over-charge: shape F's own boot reported
-`Graph capturing finished in 4 secs, took -0.06 GiB` — a **negative** capture
-delta, so the 256 MiB base was fiction. Charging `0 base + 12 MiB/graph`
-(0.07 GiB for the shipped 6-graph plan, and still per-graph so an 18-graph
-spec-decode shape pays 0.21 GiB) lands the plan at budget 31.29 / graphs 0.07 /
-**arena 17.77 GiB / 7,762 slots measured**. The KV ask went to
-`3,166,500,000` B/rank, which the engine's own 7,891.8 B/token price lands at
-**401,408 tokens measured / 2.00x concurrency** — the first shape holding two
-full-length conversations with real spare. (The ask/landed delta is block
-alignment: 196 x 2048. The price is shape-dependent, 7,701 B/token at 262144.)
-
-**UTIL 1.00 is not a knife-edge on this fork, it is UNBOOTABLE — measured
-2026-10-03.** One deploy took the field maximum to buy the last 0.57 GiB of
-arena; both workers refused init BEFORE loading a single weight, twice:
-`ValueError: Free memory on device cuda:N (31.29/31.86 GiB ...)` — the worker
-demands free > budget and budget == total, so `le=1` on the CLI field is a
-trap: the engine dies in a 2-restart crash loop with zero payload progress.
-So `0.982` is a LAW, not a preference, and the "why 0.982" answer now has two
-proofs: this ValueError, and the historical ceiling deaths (the 2x262144 bf16
-boot at 0.13 GiB free; the 5x262144 boot at "0 bytes free" on a real
-995-token request). Everything beyond ~7,912 slots needs the registry edited
-inside the image (the fork's log line says exactly that: *"Update the
-registry entry if this drifts"*), not a deploy.
-
-**The host region was the last arena-plan term still autofitting per boot —
-now it is a literal (2026-10-04).** With no `--expert-offload-mem` the planner
-binds the host region to `MemAvailable` AT BOOT minus `PLE 47.68 - 2 x 7.0
-process - 5.0 api/engine - 5% margin`: the 2026-10-05 boot read 237.97 GiB and
-offered 159.39 GiB usable, while the same host reads **75.4 GiB available while
-running** — the bound tracks whatever the co-tenants hold at that moment. The
-demand is 105.48 GiB (52.74 GiB/rank x 2, 23,040 swap-target experts), and a
-boot where the bound drops near it silently converts next-hottest experts to
-VRAM-only (the `pinned_extra` warning is the only trace).
-`FRANK_ARCHON_EXPERT_OFFLOAD_MEM=120` pins it: measured demand + 14% headroom,
-on a host with 246.1 GiB total. Semantics verified in the shipped image
-(`offloader/expert_plan.py`): the cap binds as the bound, an exceedance only
-warns, and the plan rewrites the field to the cap it met (+1 MiB/rank).
-
-**MNBT 1024 -> 2048 is free on every axis the plan can see.** The registry
-overhead model is `base + 0.22 GiB x max(0, (nbt - 4096)/1024)` anchored at
-nbt 4096, so any rung up to 4096 charges the identical 5.25 GiB; the fitted 5.25
-was itself fitted at nbt 4096, which means 2048 is covered by an activation
-figure measured at a **larger** batch. SEQS 3 boots at both 1024 and 2048
-(measured 2026-10-02 — the `query_start_loc` `ConstraintViolationError` is a
-SEQS property). The costs are a one-time recompile (the compile **range** is
-part of the cache key, so `FRANK_ARCHON_CACHE_DIR` pays once) and a wider
-prefill chunk, which the 09-29 sweep found decode-interchangeable across
-1024-4096 while 512 doubled TTFT at 4064 ctx (2.46 s vs 1.36 s).
-
-**`--expert-precise-memory` is the arena lever that does not touch the window.**
-It attacks the planner's overhead term (shape F above attacks the graph term —
-the third and last term this repo can reach), and the flag swaps the default
-registry figure for the fitted one. Measured
-2026-10-02 by a boot probe on the shape-D base (production stopped, candidate
-booted AS the service; numbers are the engine's own plan lines):
-
-| plan line | shape D | **shape E** |
-|---|---|---|
-| `runtime overhead (non-torch + buffers + activation)` | 7.60 GiB | **5.25 GiB** (`precise (gfx1201 TP2 nbt4096/seqs16/len262144)`) |
-| `expert cache space` | 14.87 GiB | **17.22 GiB** |
-| slots | 6,496 | **7,522** (+1,026, +16%) |
-| `swap region` | - | 13.7 GiB / 5,986 slots, 133-134 per layer |
-| boot | - | healthy in 420 s, no OOM |
-
-`'expert_precise_memory': True` also appears in `non-default args`, so a silent
-argparse drop is visible in the boot log. The flag lands **in the arena, not the
-pool** - `--kv-cache-memory-bytes` is untouched at 3.15e9/rank (option A), because
-the pool is not an arena lever and cutting it further would only cost
-concurrency: a 2.2e9 pool was probed in the same window and reports 277,686
-tokens / **1.38x**, so it is the conservative fallback if free VRAM ever binds,
-not the shipped value.
-
-**The compile cache is a boot-cost fix, and it was worth 173 s per boot.**
-Measured 2026-10-02: the image ships **no `/root/.cache` at all** (a fresh
-container: `No such file or directory`), and after a boot that directory holds
-`vllm` at **1,154,714,333 B in 4,665 files** plus `comgr` at 229 MB, all inside
-the container's ephemeral layer. Every
-`podman rm -f` - i.e. every deploy, restart and reboot - discards it, and the
-engine re-pays:
-
-```
-Using cache directory: /root/.cache/vllm/torch_compile_cache/b265bd396a/rank_0_0/backbone
-Dynamo bytecode transform time: 4.99 s
-Compiling a graph for compile range (1, 1024) takes 173.19 s
-saved AOT compiled function to /root/.cache/vllm/torch_compile_cache/torch_aot_compile/…
-torch.compile took 181.20 s in total
-init engine (profile, create kv cache, warmup model) took 217.26 s (compilation: 181.20 s)
-```
-
-Mounting a host directory at `/root/.cache` makes every later boot load the AOT
-artifact instead. The mount is deliberately the **parent**, not
-`/root/.cache/vllm`: `VLLM_CACHE_ROOT` defaults to the `vllm` child, and
-`comgr` is the HIP-kernel JIT cache, so one mount persists both — and it hides
-nothing, because the image ships no `/root/.cache`. The cache key
-includes the compile range, so the MNBT choice costs exactly one compilation.
-Entries are version- and config-hashed: a stale entry is a miss, never a wrong
-answer.
-
-**Proven live on the deployed endpoint, 2026-10-02, consecutive boots of this
-exact shape:**
-
-| boot | engine log |
-|---|---|
-| cold (deploy) | `Compiling a graph for compile range (1, 1024) takes 179.24 s` → `torch.compile took 188.26 s in total` |
-| warm (restart) | `Directly load the compiled graph(s) for compile range (1, 1024) from the cache, took 2.143 s` → `torch.compile took 3.97 s in total` → `init engine (profile, create kv cache, warmup model) took 41.13 s (compilation: 3.98 s)` |
-
-The host directory held **1.4 GB in 4,707 files** after the cold boot. Time to
-`/health` is dominated by the 172.8 GiB payload load (the warm boot reached
-health in 490 s), so the win is the ~184 s of inductor/AOT build rather than a
-proportional cut in boot time.
-
-`GPU_MAX_HW_QUEUES=2` caps the HSA hardware queues per GPU. Verified a real knob
-rather than a no-op - the literal is read out of
-`/opt/rocm/core-10.0/lib/libhsa-runtime64.so` - and shipped deliberately
-alongside the 32x64 gather geometry. It is **not** A/B measured.
-
-**The pool, not the slot count, sets how many conversations live at once.**
-Measured live at this shape: `GPU KV cache size: 398,658 tokens, Maximum
-concurrency for 200,704 tokens per request: 1.99x` (3,146,129,408 B reserved).
-The per-token price is **shape-dependent**: the 262144 shape priced at 7,701
-B/token (6,100,000,000 B -> 792,069 tokens), this pool at 7,892 - ~2.5% dearer,
-so a concurrency figure never extrapolates across geometries. The endpoint
-advertises 3 slots: a third LONG request queues rather than being refused, and
-shorter requests use the spare blocks. Verified: three concurrent 67-token
-requests answered in parallel, ~4.1 s each. Two contexts at 200704 are a
-capacity contract, not a throughput one (2-way concurrency summed BELOW parity
-at 4064 ctx).
-
-#### `--max-num-seqs 2` is impossible on this model (measured 2026-10-02)
-
-Four single-variable boot probes against the then-deployed shape, production
-stopped for each:
-
-| MAXLEN | SEQS | MNBT | result |
-|---|---|---|---|
-| 262144 | 2 | 1024 | **Engine core initialization failed** |
-| 262144 | 2 | 2048 | **Engine core initialization failed** |
-| 262144 | 3 | 1024 | healthy - 792,069 tokens, 3.02x, arena 6,248 |
-| 262144 | 3 | 2048 | healthy - the then-deployed incumbent |
-
-The failure is `torch.fx.experimental.symbolic_shapes.ConstraintViolationError:
-Constraints violated (L['query_start_loc'].size()[0]) ... specialized it to be
-a constant (3)`, raised from AOT `build_guards(strict_error=True)` during
-`determine_available_memory -> profile_run -> _dummy_run`. `model_runner.py`
-sizes that input buffer `max_num_reqs + 1` and `_dummy_run` sets
-`num_reqs = min(max_num_batched_tokens, max_num_reqs)`, so the width is
-`SEQS + 1` for ANY `MNBT >= SEQS`: 3 at SEQS 2, 4 at SEQS 3. **MNBT is not a
-lever here** - it only moves the compile RANGE ((1,1024) vs (1,2048)), which
-is why the identical error appears at both rungs. `--enforce-eager` bypasses
-the path at the cost of CUDA graphs. The same failure class killed the
-2026-10-01 bf16 2 x 262144 shape.
-
-#### The arena lever is `max_model_len`, not `--kv-cache-memory-bytes`
-
-The fork's planner sizes expert cache as `budget - non-expert - KV reserve -
-graphs - overhead`, and the KV reserve is registry-priced at **9190 B/token x
-max_model_len** - not from the `--kv-cache-memory-bytes` value:
-
-| MAXLEN | KV reserve | arena |
-|---|---|---|
-| 262144 | 2.48 GiB | 14.30 GiB / **6,248 slots** |
-| 200704 | 1.91 GiB | 14.87 GiB / **6,496 slots** |
-
-D(reserve) 0.57 GiB = D(arena) 0.57 GiB = +248 slots. The 6.1e9 -> 3.15e9
-pool cut (2.75 GiB) bought **zero** arena slots. It buys runtime HEADROOM,
-which is still worth having - the shape it replaced died with `0 bytes free`
-on a real 995-token request. Grow the arena by lowering `max_model_len`, or by
-taking the overhead term with `--expert-precise-memory` (below); the pool is the
-headroom lever only.
-
-#### The other arena lever: `--expert-precise-memory` (overhead 7.60 -> 5.25 GiB)
-
-The flag is now ON. It was refused on 2026-10-01 for a reason that does not
-generalize: the 2 x 262144 bf16 shape was already at the ceiling (weights 6.28 +
-pool 6.80 + graphs 0.55 + arena 16.65 + non-torch ~1.45 = 31.73 of 31.86 GiB,
-NRestarts=3 at ~6.7 min each), so the 2.35 GiB the flag moves into the arena was
-the last stack over the line, not the cause. The plain bf16 2 x 262144 boot also
-did not stay up: 0 bytes free at runtime, and a real client request (995-token
-prompt, `max_tokens=261149`) killed the worker with `Tried to allocate 50.00 MiB
-... 0 bytes free`.
-
-At 200704 x 3 the flag fits with room and is worth **+1,026 slots (+16%)**. What
-made it affordable is the window trim, not a pool cut: the arena is
-DEVICE-resident, the pool is not part of the budget's arithmetic, and the 200704
-window's KV reserve (1.91 GiB) plus a smaller attention footprint is what the
-plan needed. The pool stays at 3.15e9/rank deliberately (option A) - cutting it
-further would cost concurrency (2.2e9 -> 1.38x) and buy no slots.
-
-Boot-probed 2026-10-02 (transient systemd units at the production uid +
-memlock, full-log capture; the rootless bench probe had masked the production
-memory environment). The night searched two concurrency contracts:
-
-| rung | result |
-|---|---|
-| 3 x 262144, bf16, KV 11,047,010,304 | **OOM at graph capture** — `Tried to allocate 76.00 MiB ... 0 bytes free`, 30.43 GiB of 31.86 GiB already allocated by PyTorch |
-| 3 x 245760, bf16 | **OOM identically** |
-| 3 x 237568, bf16 | PASS — KV 718,606 tokens |
-| 2 x 262144, bf16 + `--expert-precise-memory` | **DEPLOYED then crashed** — 0.13 GiB free at boot, NRestarts=3 (the ceiling, not the flag) |
-| 2 x 262144, bf16 (no precise flag) | did not stay up either: 0 bytes free at runtime, worker killed by a real request |
-| 3 x 200704, fp8, pool sized for 2 contexts | **DEPLOYED 2026-10-02 (shape D)**: 398,658 tokens = 1.99x concurrency, arena 14.87 GiB / 6,496 slots, the 2.75 GiB/rank pool cut returned as runtime headroom |
-| **3 x 200704, fp8, + `--expert-precise-memory`** | shape E (2026-10-02): same 1.99x pool, overhead 5.25 GiB, arena 17.22 GiB / 7,522 slots, boot healthy in 420 s |
-| 3 x 200704, fp8, MNBT 2048, graph term 0.32 GiB, util 0.982 | shape F (2026-10-03): arena 17.52 GiB / **7,653 slots** measured, budget 31.29, `Estimated CUDA graph memory: 0.32 GiB`, KV unchanged 398,658 |
-| 3 x 200704, fp8, graph term 0.07 GiB, util **1.00**, KV 3.1665e9 | **DID NOT BOOT (2026-10-03)**: `ValueError: Free memory on device cuda:N (31.29/31.86 GiB)` on both ranks, twice, zero weights loaded — the worker demands free > budget and budget == total. UTIL 1.0 is unbootable, full stop |
-| **3 x 200704, fp8, graph term 0.07 GiB, util 0.982, KV forced to 400,700** | **DEPLOYED 2026-10-03 (shape G)**: arena **17.77 GiB / 7,762 slots** measured (+109 over F, +240 over E), KV **401,408 tokens / 2.00x** concurrency measured, capture 0.31/0.10 GiB vs the 0.07 charge (under-charged, boots and holds), driver used 30.97 GiB -> **0.89 GiB real free per card** held through a live 4,096-token generation (~38-39 tok/s single-request, +13% vs the 33.99 shape-E baseline); rollback is UTIL or KV_BYTES down a notch |
-
-Six full-length contexts cannot fit at any KV dtype: the 6-seq graph/activation
-footprint leaves only 9.18 GiB for the pool, and 6 × 262144 needs 12.47 GB even
-at fp8.
-
-The old "KV_BYTES is PER RANK, 9/16 GiB OOM" law was the MTP4 WORKSPACE
-reservation, not a VRAM wall. Failures are clean and diagnostic: the engine
-reports `torch.OutOfMemoryError: Tried to allocate X GiB. GPU N has ... of
-which Y GiB is free`, and the fork's giant ERROR lines are split by journald —
-read them via `journalctl -o json`, not `-o cat` (and `podman logs`, which is
-only available while the container lives).
-
-Rollback to the 2026-09-29 fp8+MTP4 shape: git tag
-`frank-archon-fp8-mtp-20261001` (env trio + seam re-pinned there).
-
-## Parallelism options on this box (2026-10-01 survey)
-
-Surveyed from the running image's own CLI surface. Verdicts are specific to
-dankserver: 2x R9700 (gfx1201), a ~172.8 GiB expert-offload MoE, and a model
-with only 2 KV heads.
-
-| option | flag | verdict on this box |
-|---|---|---|
-| TP | `--tensor-parallel-size` | **DEPLOYED, =2.** All-reduce dispatch shows `R4D_AR` then `PYNCCL` |
-| DCP | `--decode-context-parallel-size` | **REJECTED at TP=2** — see below |
-| PP | `--pipeline-parallel-size` | supported; untried (stage handoff plus a TP-keyed expert plan) |
-| DP | `--data-parallel-size` (+`-rank`, `-start-rank`) | impossible: every replica holds a full model copy (172.8 GiB vs 31.86 GiB/card) |
-| EP | `--enable-expert-parallel` + `--all2all-backend` (+`--enable-eplb`, `--enable-dbo`, `--enable-elastic-ep`) | supported (`naive`, `allgather_reducescatter`, `deepep_high_throughput`, `deepep_low_latency`, `pplx`, `mori`, `nixl`); would replace the fork's tiered-offload LOCAL gather with a per-layer PCIe all-to-all |
-| multi-node | `--nnodes`, `--node-rank`, `--master-addr/port`, `--distributed-executor-backend` | supported; the actual path to more cards |
-
-### DCP is structurally unavailable here — do not re-test on 2 GPUs
-`--decode-context-parallel-size 2` fails `VllmConfig` validation at boot:
-
-```
-Decode context parallelism for GQA/MQA requires `--tensor-parallel-size` (2) to be
-greater than the model's total number of KV heads (2). Increase
-`--tensor-parallel-size` or set `--decode-context-parallel-size 1`.
-```
-
-DCP shards the KV along the *sequence* dimension after KV heads have been spread
-across the TP ranks, so it requires **TP > num_kv_heads**. This model has 2 KV
-heads (24 Q heads, GQA 12:1), so TP must be >= 4 — four GPUs, not two. Related
-fork facts: DCP *reuses* the TP ranks rather than multiplying them (it is not
-`TP x DCP`), and with PCP enabled the valid DCP sizes are `{1, pcp, tp x pcp}`
-(`vllm/config/parallel.py`). The KV memory a DCP win would have bought is real —
-each rank would hold half the sequence's KV — but it is unreachable without two
-more cards.
-
-### Measuring decode: the harness lives INSIDE the container
-`/home/dank/mtp_measure.py` imports `/bench_iter.py` by absolute path, so it must
-run in the engine's namespace and the files must be copied in first:
-
-```
-podman cp /home/dank/mtp_measure.py frank-archon-qwen:/
-podman cp /home/dank/bench_iter.py  frank-archon-qwen:/
-podman exec frank-archon-qwen python3 /mtp_measure.py
-```
-
-Without the copy the exec dies with `can't open file '/mtp_measure.py': [Errno 2]
-No such file or directory` — measured 2026-10-01, which silently cost a whole
-arm's numbers. A window script must do the copy, not assume the path exists.
-
-### The then-deployed shape, measured again (2026-10-01 A/B window)
-A clean production-environment boot of the config deployed at that time (4 x 262144 fp8, MNBT
-2048 in that window) reported **`GPU KV cache size: 1,079,582 tokens, Maximum
-concurrency for 262,144 tokens per request: 4.12x`** — the same figures as the
-deployed MNBT-1024 shape, i.e. MNBT does not move the pool, and the capacity is
-reproducible across boots.
-
-## See Also
-
-- [[dankserver-frontier-moe-feasibility]] — the sibling feasibility page
-  (which frontier MoE fits this box; the same PCIe-expert-stream wall at
-  V4.1 scale)
-- [[frank-archon-r9v-radiance-qwen38-q8]] — RETIRED 2026-09-30 (the R9V
-  fork era; production has been the tcclaviger fork since)
-- [[x4-pair-experiment]] — the same residency law measured on a narrower
-  link: streaming over PCIe loses, computing in RAM does not care
-- [[Dual-R9700-Cooperation]] — the board swap that made TP=2 legal here