The 2026-10-19 write-up inferred a host restart from FTL's 16:25 start line. It was wrong: uptime -s reads 2026-10-18 01:24 (two days up) and the user journal shows 'Stopping pihole.service' at 16:25:23 with StartedAt 16:25:27 — a plain container restart inside the open-notebook deploy window. That moves where the verification belongs: after every pihole restart, not just after a reboot.
open-notebook resolves frank-dragoon-halogen to 10.89.1.39, GET :8731/v1/models answers, and the engine logs the app's POST /v1/embeddings 200. That makes the 2026-10-19 dial census a pre-move record: pasta's forwarding rule is unchanged, the endpoint was moved out of pasta, so the 'menu is frank-tts + frank-archon' line is now past tense. Frontmatter date was still 2026-09-02 against October content; bumped.
The 2026-10-19 reboot left every franksdank.cloud name NXDOMAIN for LAN clients while external names kept answering from cache; the file was mounted and readable and dns.hosts was intact, and 'pihole restartdns reload' restored every name in seconds. Mechanism not established. Instrument and remedy now on the owner page, plus a failure-mode row for the FTL rate-limit case seen beside it (10.0.2.x = a pasta container's forwarder).
open-notebook's embedding default now points at qwen3-embedding-0.6b on frank-dragoon's XDNA2 lane, proven through the app's own paths (model test 1024 dims, worker 'Successfully embedded', vector search ranking two unrelated notes 0.659/0.292 and 0.906/0.318). Records the two app-side traps: defaults store the model ID and a name is accepted then fails in the worker, and POST /api/embed 500s on an upstream RecordID typing bug after the vector is already written. frank-dragoon's lane section moves from 'off until the mount fix' to armed-and-serving with the boot-ledger contract and the measured idle RSS; the page's stale 'not on dankclaw_default' reachability claims are retired.
The armed boot reached the engine and died in its own preflight:
halogen npu: /dev/accel/accel0 is here, but this process cannot open it
(crw-rw---- nobody:nogroup).
The node is root:render 0660 and a rootless container maps exactly one uid/gid,
so `render` is unmapped inside it and `--group-add keep-groups` — the vendor's
advice, and the ComfyUI pattern for /dev/kfd — cannot grant a group the
namespace does not map. Measured 2026-10-19 from a `dank` shell already in
`render`: without the flag denied, with the flag denied, with the udev mode
0666 the open succeeds (OPEN_OK). Same wall 99-comfyui-gpu-access.rules already
solved for the GPU compute nodes, so the accel node gets its counterpart:
dankclaw/udev/98-halogen-npu-access.rules — KERNEL=="accel[0-9]*",
SUBSYSTEM=="accel", MODE="0666"
The deployer installs the rule, reloads rules, re-triggers the node and only
then checks the mode; the serve script refuses any node that is not
other-writable, naming the rule, so this cannot return as a crash loop (189
restarts while chat was down is the failure mode being closed).
Both checks read the permission bit directly: `[ -o FILE ]` is not
"other-writable" — it means "exists" and `-o` doubles as the OR operator inside
test, which is why the first cut of the gate refused everything.
Dragoon seam: 33 -> 35 tests. The fixture node is 0666 like the installed one,
a 0660 node is refused with the rule named and nothing rendered, and the rule
file's shape plus the deployer's install/trigger are asserted.
The containment gate added in #44 parsed scripts/frank-dragoon.env through a
$CHECKOUT variable that does not exist in deploy-remote.sh, so the very next
deploy died at `set -u` before it touched the unit:
./scripts/deploy-remote.sh: line 1445: CHECKOUT: unbound variable
deploy-remote.sh runs from the checkout root and every other path in that block
is a parity-twin default (LLM_DRAGOON_CKPT/_VISION/_TOK_DIR), so the payload dir
gets the same treatment: LLM_DRAGOON_MODEL_DIR, defaulted to the deployed path,
compared directly. The seam now asserts that twin against the env file next to
the NPU twins it already checks, and the containment comparison itself is
verified both ways (inside accepted; the old outside path and a shared-prefix
sibling both refused).
frank-dragoon's lane section: the staging dir must live inside the payload dir
(the image ships no /models, so a nested :ro bind has no mountpoint to land on),
the entrypoint's rejection of HALOGEN_NPU_DIR, and the fabric-clock paragraph
corrected — the vendor's root host unit is installed and reports `held (manual)`
as of today, so "nothing is held" is retired along with the hand-write it
replaced.
dankclaw-iommu-translation-off: the deferred `pp_dpm_fclk` pin is no longer
deferred; the fact is owned by [[frank-dragoon]] and this page links it.
The lane's first boot died at container init, not at the engine:
runc create failed: ... create mountpoint for /models/npu mount:
mkdirat .../merged/models/npu: read-only file system
The image ships no /models, and the serve script binds the payload dir
there read-only, so a nested `-v $NPU_DIR:/models/npu:ro` had to create
its own mountpoint underneath a read-only parent — impossible. The
engine's own entrypoint confirms the container path is fixed: it rejects
HALOGEN_NPU_DIR outright ("our NPU models live under /models/npu/<id>/").
Fix is the mount shape, not the engine: FRANK_DRAGOON_NPU_DIR now lives
inside FRANK_DRAGOON_MODEL_DIR, so the payload's single read-only mount
puts the files at /models/npu/<id>/ by itself and the nested bind is
gone. Verified on the running container: after the move, `podman exec
frank-dragoon-halogen ls /models/npu/qwen3-embedding-0.6b` answers with
the staged tree while the lane is still armed off.
Both gates refuse the old shape by name (serve + deployer parity) so a
future payload swap that moves the model dir cannot reintroduce a crash
that lands after the deployer has already stopped the running unit.
The 2026-10-19 legs table blamed the frank-dragoon refusal on a root/rootless
netavark rule-set split. It is not that: the container is rootless (the root
unit's User=dank runs podman run), it is on pasta, and pasta installs no DNAT
for rootless-bridge traffic while refusing a second network outright. The
gateway alias does reach it (200) and is link-local, which is the same dead end
the Open Notebook URL guard hit. Measured with the move onto the bridge now in
the deployer, including the /proc/net/tcp line that shows 8730 is loopback-bound
and so stays private after the move.
Co-Authored-By: omp <omp@dankclaw>
An endpoint on podman's rootless default (pasta) has no address another
container can use, measured on 2026-10-19 from inside open-notebook:
- 192.168.0.143:7770 connection refused (pasta forwards the
published port in the host netns; no netavark
DNAT exists for traffic arriving from a
rootless bridge)
- host.containers.internal:7770 200, but the alias resolves to 169.254.1.2 —
link-local, which open-notebook's URL guard
refuses to store
- frank-dragoon-halogen:8731 name unknown: different netns
- --network pasta --network dankclaw_default
"cannot set multiple networks without bridge
network mode" — pasta cannot be added to
So reaching the NPU endpoints from an app container means moving the container
onto dankclaw_default (FRANK_DRAGOON_NETWORK; empty keeps today's shape). This
is not an exposure change: the engine's unauthenticated token port binds the
container's own loopback (/proc/net/tcp: 127.0.0.1:8730 against 0.0.0.0:8731),
so only the API port becomes name-resolvable — the same trust the published
7770 already carries on the LAN. The deployer refuses a network that does not
exist rather than starting a unit that restart-loops on it.
Co-Authored-By: omp <omp@dankclaw>
halogen-flash-server already ships an NPU path on the pin we run: for each
model named in HALOGEN_NPU_MODELS it loads the vendor's precompiled AIE
programs (XDNA2 executes only AIE-compiled graphs) and answers them on
/v1/embeddings inside the SAME container behind port 7770. That gives the
fleet an in-house embedding endpoint without touching the iGPU trunk.
The lane is opt-in and one line wide: HALOGEN_NPU_MODELS blank is the
pre-lane endpoint byte for byte (no device, no mounts, no engine env).
The three host preconditions the vendor names are gated in the serve script
and mirrored in the deployer (the 2026-10-01 twin law), each refused by name:
- the staged model files. /models is read-only and HALOGEN_DOWNLOAD stays
unset so the container opens no outbound connection at boot, so
scripts/download-frank-dragoon-npu.sh stages them from the pinned
revision with the catalog's own byte counts and sha256s (the engine
re-checks both at boot);
- the distro XRT's libxrt_coreutil/libxrt_core/libxrt_driver_xdna, each
mounted twice — where the image looks for XRT and at its own path —
because those libraries find each other where they were installed and
an /opt/xilinx/xrt of symlinks dangles inside the container;
- the GPU fabric clock HELD. On Strix Halo the iGPU and the NPU co-running
hang the machine (and corrupt NPU results) while FCLK changes speed, and
halogen refuses to start in that state; it cannot read host sysfs from
inside, so the hold is proven host-side. dankclaw/host/ carries the
vendor's halogen-fabric-clock{,.service} verbatim at tag v0.16.2 (sha256
a3a4f7f4e0142e71… / 5f9f1cc4c85a1c02…); the deployer installs, enables
and gates it only while the lane is named, and releases the clock plus
drops the boot hook when it is not.
After health, the deployer's lane gate is a real embedding through
/v1/embeddings — a 200 on /health proves nothing about the NPU programs.
Planner: dankclaw/host/halogen-fabric-clock* re-dispatches frank-dragoon
(the deployer installs it); dankclaw/host/grub and the fetch script plan
nothing.
Seam: 11 new tests — the off arm really renders nothing lane-specific, the
on arm renders the device/mounts/env, every missing precondition refuses by
name, the deployer twins and the host unit shape are pinned.
Co-Authored-By: omp <omp@dankclaw>
- frank-dragoon: measured the response_format contract. json_schema and json_object answer
schema-valid JSON at temperature 0 and draw a 400 at any sampled temperature, and because
the sampler default is server-side an absent temperature is sampled. The page claimed
"response_format is unsupported", which sends the next agent past a working feature.
- frank-dragoon: new "The NPU lane" section. The pinned 0.16.2 image already ships the
five-program AIE catalog including qwen3-embedding-0.6b, so /v1/embeddings is one env var
plus the accel device, the distro-XRT mount and the fabric-clock hold; embeddinggemma-2
needs 0.18.3+, the stock-kernel-driver memory-budget fix 0.18.8+. Host census recorded:
no /opt/xilinx/xrt, distro XRT 1:2.21.75 with libxrt_driver_xdna, clocks unheld.
- Open Notebook: scriptwriter probe verdict — greedy-only, and open-notebook's model record
is {credential, name, provider, type} with no per-row config, so it cannot ask for greedy;
frank-archon stays the writer. The embedding gap now has a named door instead of "build a
new llama.cpp slot".
- Frank TTS: "the NPU is not a door at all" was too broad. It is not a door for the TTS
family; halogen's precompiled catalog is a door on the same device.
- System Specs: NPU runtime row corrected to the distro XRT actually installed, and the
stale "frank-zealot serves 27B" line retired in favour of [[frank-zealot]] as sole owner.
Tests: ./tests/run.sh, 455 passed.
The page described an undeployed service whose scriptwriter was going to be
frank-zealot. Both halves are now false, and the reason is worth more than the
correction.
- Status, ports, coordinates: deployed 2026-10-19, guarded vhost, API published
on loopback only. The first episode is the proof: 747 s of speech in 301 s of
wall clock (2.5x real time, knobs at 1), download route 200 with the bearer
and 401 without.
- New section for the guard chain: provider-before-route, one upstream because
the image rewrites /api/* itself, and the API_URL law — autodetect appends
:5055, which is seerr's port on this host, so an unset API_URL yields a UI
that loads and an app that cannot call itself.
- Provisioned records table (credentials, model rows, dank-duo, dank-daily) and
the two contract traps measured on the live API: generate takes profile
NAMES, and the fleet has no /v1/embeddings, so every note save fails its
embed_note command while renders from content work fine.
- Why frank-archon writes the scripts: frank-zealot is reachable from this
netns only at 169.254.1.2, and open-notebook's URL guard refuses to store a
link-local base URL with no override. Census lands on [[Container Network
Namespaces]], which also gains the root-vs-rootless publish rule — a
root-published port is invisible to a rootless container, so "publish the
port" was the wrong remedy as written there.
- Frank TTS: the request's model field is ignored; only voice selects a pack.
Tests: ./tests/run.sh, 455 passed.
The service came up internal-only: UI on the LAN at 8502, API at 5056, both
published on every interface, with only its own shared password in front. That
put the episode library outside the fleet's auth chain and one port scan away
from anything on the network.
- Caddy: guarded vhost notebook.franksdank.cloud (Anubis -> Authentik; provider
notebook-proxy, internal_host caddy:8017) plus the post-auth route to the
published UI port. One upstream is enough, because the image's own Next.js
layer rewrites /api/* to the FastAPI beside it — so the API needs no outward
face at all, and 5056 moves to loopback, where only the deployer's health gate
dials it.
- scripts/open-notebook.env: API_URL and CORS_ORIGINS become the single guarded
origin. Left to itself the app auto-detects <host>:5055, the port seerr holds
on this host; a LAN-origin API_URL would also take the browser around the
guard chain, where its authenticated session does not ride.
- addn-hosts: the LAN A record. The homepage card goes live and links the vhost.
- Seam: the API origin is derived from whatever the Caddyfile routes to 8502, so
renaming the vhost moves the test instead of leaving it blessing a stale name,
and a new test proves the vhost hands the WHOLE request to Anubis.
podman wraps a bare HealthCmd as CMD-SHELL (measured on Dankclaw's podman 5.7:
--health-cmd "/surreal is-ready" stores {"Test":["CMD-SHELL","/surreal is-ready"]}),
and the pinned SurrealDB v2 image is distroless — no /bin/sh, not even `ls`.
So the probe could never run: every 30 s interval exited 1 in ~30 ms with empty
output while the database was listening and answering, and open-notebook's first
deploy died at the gate ("did not become healthy in 90s"). The hand-check
`podman exec … /surreal is-ready` printed OK, which is why the old comment
believed the probe worked.
The JSON array survives the Quadlet generator (--health-cmd "[\"/surreal\",\"is-ready\"]")
and stores ["CMD","/surreal","is-ready"], which execs directly: a throwaway
instance of the pinned digest went healthy on that form. The seam is re-pinned
to the shell-free contract; the old assertion forbade quotes in any HealthCmd,
which is what pinned the broken shape.
The refusal message still named "the /data state dir"; /data left with the
prefix disk tier. The serve script mounts /models ro, the rad-tune cache and the
chat template only, and tests/frank_archon_unit.bats asserts no :/data mount
exists, so the line an operator reads at the moment a boot refuses pointed at a
mount that is gone.
Text only. The guard still refuses: the block was run standalone with the
variable unset and every neighbour set - it prints the new message and exits 1.
bash -n clean, ./tests/run.sh green (454 tests).
The t/s ladder and the TTFT delta triple measure a configuration neither
deployed nor compared; per the purpose test they are dead weight. What survives
is the knob pricing (halving the shared chunk costs roughly a fifth of prefill
throughput), the long-prompt-only shape of the span win, and the method. Run-on
now names the era the census was taken in.
The endpoint moved to radiance 1.8.1 with MAX_SEQS=2 and the
--vram-weights-mib/--vram-kv-mib pair removed (PR #30); env, serve script and
bats carried the new shape while the page and the AGENTS.md block still
described the 1.0.13 3-slot era, so every point-of-use number was wrong.
Re-derived from the live checkout and the 2026-10-19 11:10 boot ledger, not
from the stale pre-#31 draft: runtime block (digest, version oracle, the two
CLI breaks that are boot refusals), shape table (demand-signal rows, derived
VRAM split, restored spec/batch row, host-only prefix tier, step sharing),
container contract (no /data mount), the measured 2-slot ledger, a new Eras row
for the retired 1.0.13 era whose rollback is a digest plus a trio rewrite, and
the laws: pool sizing and the per-card budget re-scoped to the pair era, a new
resolver law (funds ~90% of the worst case, host RAM below a slab step is inert,
MAX_SEQS is a demand signal), the sludge law marked measured-at-3-slots with the
fix ordered-not-verified, the knob-availability law flipped, and the slots law
now pricing the third slot in KV (4.20 -> 2.76 GiB).
`systemctl status` said `2x262144 slots over one 524288 pool, unquantized f16
K+V, MTP off` — three payload eras and two reslices out of date, and still
wrong about the thing that decides whether the endpoint can be trusted (the KV
dtype and whether the drafter is armed). It now quotes the env: 3x200704 over
one 602112 pool, f16 K + q8_0 V, MTP armed, adapter, Q8_0 vision.
Touching the unit selects the service in the planner, so this deliberately does
NOT come with a deploy: the hub is serving live conversations. The revision
marker stays behind until the next real frank-zealot change.
Before/after on the two probe prompts (both opened `I cannot provide…`; neither
now carries a marker — one ran to its 2048-token cap, the other finished at
1930), the residual `Important Disclaimer` heading named for what it is (style,
not the circuit), and the same-boot proof that chat/tools/vision still answer.
Provenance now points at the boot ledger: the fork prints every
`prism.abliterate.*` key at each start, so the PTQ1_0-vs-PQ2_0 mismatch is
visible to anyone who boots the service — the page links that instrument
instead of restating it.
- Shape table: an Adapter row (bytes, sha, what it edits, and the PTQ1_0
provenance asymmetry), the fetch line now names both fetchers, the flag list
carries --lora-scaled.
- One paragraph: the adapter changes behaviour, not the budget equation, and the
probe that proves it must budget >= 2048 tokens because the thinking channel
eats a short budget whole and returns empty content with finish_reason=length.
- Eras: the new era's rollback is one env line on a file already loaded.
- Laws: provenance is metadata, not a gate (tensor names, not hashes; an
embedding rank would crash, which is why the publisher left token_embd alone);
the fork pin now has a third reason, since older builds refuse the adapter with
`unsupported prism.hadamard.version: 2`. Base-alignment law updated: the 9B
refusal escape hatch was a fetch, and it has now been taken.
- AGENTS.md: topology payload cell and the zealot operator block.
Frank ordered the adapter; this adds it to the shape as one knob and one flag.
- `LORA_FILE` / `LORA_SCALE` in the env, `--lora-scaled <file>:<scale>` as ONE
argv token so a scale can never reach the engine without its adapter. The
rollback follows the mmproj convention exactly: `LORA_FILE=` (explicitly
empty) drops the flag entirely, no rebuild, and `LORA_SCALE=0` is the card's
own "inert but present" value.
- New fetcher `scripts/download-frank-zealot-uncensored-lora.sh`: `-f`, byte
count, sha256, `--verify-only`, idempotent. The file is NOT LFS (a plain git
object), so the pinned sha is the content hash, not an LFS oid.
- The deployer gates on the adapter beside the pack and the projector, and its
error names the ADAPTER's fetcher — sending an operator to the bonsai9 script
for a 3 MB file is how a payload ships without it.
- Seam: path+scale pinned as one token, the scale knob including 0, absence on
the empty opt-out, a refusal to boot on a missing adapter, and the env ↔
deployer anti-drift pair extended to LORA_FILE/LLM_ZEALOT_LORA. 454/454.
Provenance, stated where it will be read: the adapter's own metadata names
`Ternary-Bonsai-2-9B-PTQ1_0.gguf` (3561df7a…) as the pack the direction was
fitted against. This endpoint serves the PQ2_0+MTP+Q8_0 packing (c79fcdfa…),
which is not that file and is not on this host. The loader matches tensor names,
not hashes, so it applies — and build 10777 is the build the card verified on —
but the strength is calibrated on a sibling quantization. The proof is therefore
the refusal probe against the live endpoint, captured before and after, and not
the card's table.
The cutover PR merged the runtime; this carries the surface that has to be
true afterwards.
- Engine page: the shape is now `--device Vulkan0` on the gfx1151 as a
co-tenant beside frank-dragoon; the registry/thread/streaming laws gain a
fourth — the device law — whose numbers stay in the verdict report, and the
deploy record gets the cutover steps including the `glslc` configure gate.
The CPU-era arm/rollback prose keeps the qualitative argument and drops the
dead numbers per the purpose test.
- Endpoint page: "Where the engine computes" now says iGPU + co-tenancy, and
states the NPU verdict in one paragraph instead of a research tour.
- Quadlet comment: it claimed supplementary groups are what grant render
access. On this host the udev rule world-writes the render nodes and the
container's `id` shows no groups at all, so keep-groups is belt-and-braces;
the RSS note now matches the device shape instead of the CPU trial.
- Verdict report: gate 2 (contention) recorded as measured after the cutover,
and the build-stage prediction corrected — the package is `glslc`, not
`libvulkan-dev glslang-dev`.
- index.md regenerated (one description changed).
The NPU path is a dead end for this payload (verdict:
report/tts-npu-spike-verdict-2026-10-19.md); the iGPU is the accelerator that
exists on this box today, and it is already shared with frank-dragoon.
- Containerfile.engine: -DGGML_VULKAN=ON, plus the three packages the door
needs to configure — libvulkan-dev, spirv-headers, and glslc, which Ubuntu's
glslang-tools does NOT ship (find_package(Vulkan COMPONENTS glslc REQUIRED)
fails without it) — and libvulkan1 + mesa-vulkan-drivers (the RADV ICD) at
runtime. No NVIDIA ICD on purpose: one Vulkan device in the enumeration is
what makes the NAME Vulkan0 safe to write down.
- tts-engine-start.sh: QWENTTS_DEVICE -> --device, default Vulkan0. A value
that is set but EMPTY passes no flag at all, which is the CPU rollback; that
is why the default is written as an -v test and not with :=, because := also
fires on an empty value and would silently undo the rollback.
- quadlet: AddDevice=/dev/dri + GroupAdd=keep-groups (a rootless container that
drops its supplementary groups enumerates no render device). The node is not
named: this host is hybrid, renderD128 is the NVIDIA card and renderD129 the
amdgpu iGPU.
- env: QWENTTS_DEVICE=Vulkan0 plus the device law in the header, including the
rollback line.
- tests: the two CPU-only assertions now assert the Vulkan shape, and a new one
pins the rollback (empty device => no --device in argv). 451/451 green.
Measured before writing any of it: the configure in the digest-pinned base
answers "Found Vulkan: ... found components: glslc" and "Including Vulkan
backend"; gate 1 (RADV GFX1151 visible to a rootless container off this base)
is recorded in the verdict report.
Two conflicts, both from the 2026-10-19 vault compaction on main:
* frontmatter description — main shortened it to satisfy OKF DESC
(max 120 chars); this branch's longer wording would have failed the
seam. Kept main's shape and folded the new §9 in by naming
step-shape attribution instead of profiler attribution.
* the Related bullet for [[vLLM Porting and Compile Triage]] — that
page was destroyed on main, so the link is dropped rather than
resurrected; this branch's [[frank-archon]] row is kept.
Index rows regenerated with scripts/gen-vault-index.py.
The trial's quadlet line said AddDevice=/dev/dri/renderD128. Resolved the nodes by
driver instead of assuming: on this hybrid host renderD128 -> nvidia and
renderD129 -> amdgpu, so the guess pointed the engine at the wrong card. Also
records the gate-1 command that works (vulkaninfo --text; --batch prints help and
exits 1) and the probe script path.
vulkaninfo inside the engine's own base image (ubuntu:24.04 + mesa-vulkan-drivers
vulkan-tools, --device /dev/dri, rootless) reports Radeon 8060S Graphics (RADV
GFX1151) / driver radv / API 1.4.318 with two heaps: heap 0 host-visible 38.67 GiB
(budget 32.58) and heap 1 DEVICE_LOCAL 77.33 GiB (budget 65.16) with frank-dragoon
loaded. ggml's Vulkan backend asks for DEVICE_LOCAL, so the ~4 GiB the TTS engine
needs lands in the 65 GiB budget: memory is not the constraint, and the 2 GiB DRM
carve-out never enters the picture. Both heaps are GTT — the same host RAM, so the
budget is a ceiling on mappings, not free silicon memory.
Traps recorded: vulkaninfo --batch is not a flag in Ubuntu 24.04's vulkan-tools
(prints help, exits 1; use --text), and the container has no NVIDIA ICD, so Vulkan
device 0 happens to be the iGPU — select by name, not index.
Probe: /home/dank/spikes/npu-tts/vk_probe.sh on Dankclaw. Enumeration only; no
allocation, no compute, production untouched.
The overnight spike settled whether frank-tts can run on the Strix Halo NPU.
It cannot, and the wall is the loader, not the silicon: AMD's own Linux
runtime opens /dev/accel/accel0, succeeds at the GEM ioctls, then declines
the NuTTS-Air graph without emitting a single log line — and the one run that
did load has never reproduced. The accelerator that does exist is the gfx1151
iGPU behind -DGGML_VULKAN=ON, a flag our own Containerfile.engine writes OFF,
and Frank set the target shape to co-tenancy: the engine resident beside
frank-dragoon, not a handover.
- report/tts-npu-spike-verdict-2026-10-19.md owns the measured result: the
exoneration chain, the two arms' real-time bars (12.5 codes/s deployed vs
50.00 for NuTTS-Air), the 65,536-entry speech-token census (ids
151,671-217,206), the host gates (rootless cannot read accel0; dpkg-deb the
vendor XRT, never dpkg -i), the iGPU census (18,523 of 116,733 MiB GTT in
use, 17 GiB host RAM available, GTT pages are host pages), the two gates
that decide co-tenancy (RADV heap for an APU tenant; dragoon's TTFT with a
synthesis in flight), and the five-file change surface for the trial.
- report/npu-tts-runtime-landscape-2026-10-19.md and
report/qwen3-tts-payload-portability-2026-10-19.md are the research slices;
both now point at the verdict page instead of ending in loose ends.
- The vault page gains "Where the engine computes", so an operator doing a TTS
task learns the NPU is a closed door without re-walking it.
No fleet change: frank-tts / frank-tts-engine still serve narrator1-3 on CPU.
tests: ./tests/run.sh 450/450 green.
FRANK_ARCHON_MAX_SEQS 3 -> 2. At 1.8.1 the budget resolver sizes the KV cache
from the cache bound (--max-model-len x --max-num-seqs), --gpu-headroom-mib and
--host-pool-mib, and the shipped CLI has no KV-size flag (the
--vram-weights-mib/--vram-kv-mib pair was removed at 1.3.0). The live boot
ledger reads that cache at 4.20 GiB = 556,348 tokens = 2.8 sessions at 200,000,
so a third full-height conversation is refused while /health answers green and
the harness advertises capacity the engine cannot honour. Two slots is what the
shape actually serves at the full window; a slot costs KV, never decode, and the
fused TP all-reduce drops from 9 rows to 2 x (1 + spec) = 6, still inside
wht6's 25-row bit-exact bound.
FRANK_ARCHON_HOST_POOL_MIB back to 44928 and FRANK_ARCHON_TTM_MIN_MIB back to
150000, reverting PR #30. That raise bought cache the resolver never hands out:
the two boot ledgers differ in exactly one line -- host_pool 44.47 -> 45.09 GiB
-- while vram_weights (25.81 GiB) and vram_kv (4.20 GiB / 556,348 tokens, both
ranks) are byte-identical, so the extra 640 MiB sat unclaimed in the host pool
while the TTM floor moved out from under the preflight's arithmetic. The
compute-bound cap on --host-pool-mib is unchanged either way (45,568 -> 22,222
MiB of experts either side of the 22,222 clamp).
The bats seam pins the new contract: MAX_SEQS=2, pool 44,928, and the ledger
floor that computes to 148,920 MiB at that pool against the 150,000 minimum.
*Documentation Policy* rewritten: a page earns its bytes only if it changes what
the reader does next. Measurements are disposable — a rate for anything not
running and not being compared is destroyed, not filed in an Eras row or a
technique page. "It is the only copy" is explicitly NOT a reason to keep a fact.
`log.md` retired from the policy; the Runbook Index no longer promises a page per
retired endpoint (rollback is a commit, not a document).
*Key Lessons Learned* purged of the dead: the dual-R9700 board-swap narrative and
the layer-split/x4-pair experiment laws (no live endpoint uses either), the QSA
patch note and the retired Kairic/FastMTP traps, and the throughput figures inside
the laws that survive — the law stays, the number goes. Two lessons added, both
from this campaign's own failures: a verdict is a lead, the scout's and the tool's
alike; and never let a census decide what to keep, because a tool can say where a
fact lives but not whether anything needs it.
997 → 820 lines.
The vault was 155 pages / 31,210 lines, most of it measurement ledgers for
payloads that no longer run. A throughput figure for a runtime nobody operates is
not a record: AI moves faster than the vault, so the number is wrong before
anyone reads it, and a wrong number in a runbook gets believed. Per the new
purpose test in `AGENTS.md` → *Documentation Policy*, it is destroyed.
Gone (`.AGENTTRASH/20261019-destruction/`, KILL-LIST.txt is the manifest):
- the `frank-medium` endpoint and its five era pages, `frank-npu`, FastFlowLM,
the ikllama/kingjones/dealign/frank-redo eras, the ling/swift15/freetoken/ubatch
campaign pages, qwen3.8 spec-matrix and strix-optimizations, the two
"Running Qwen3.6 …" pages;
- the whole `Pipeline Parallelism/` folder (no live script passes a pipeline flag)
and the multi-device/x4-pair/hybrid/offload experiment pages;
- every vLLM page — no vLLM endpoint exists on this fleet;
- `LLM/deployment.md` (`Architecture/Deployment Flow.md` is the test-referenced
owner) and the external surveys (SGLang, Lemonade, llama.cpp forks/diff,
self-hosted app census, the TTS survey) that were obsolete by construction;
- `log.md`, 2,380 lines. Git holds the chronology; a causal law belongs to the
page that owns the thing it governs, never to a diary.
Survivors: 101 pages, 17,542 lines, 0 OKF violations, 0 unresolved links. Eras
tables keep the window, payload, runtime, reason and rollback pointer; the rate
columns are blanked. The live shapes stay verbatim — those numbers are what the
deployer and `models.yml` consume.
Point-of-use enforcement needs hands, not advice:
- `destroy-vault.py` — moves the pages on a kill list to `.AGENTTRASH/` and then
deletes every reference to them from the survivors, including the whole line
when the dead link was the line's only content. Stems are seeded from the trash
so the pass is re-runnable; `pathlib.Path.stem` is NOT used to compare link
targets, because it strips the real suffix off names like `sd.cpp`.
- `axe-measurements.py` — deletes bench/throughput ledger sections and rate rows,
blanks any table cell that reports a rate, and folds Eras older than the current
one into a single row that says the numbers were destroyed and that rollback is
a commit. Never touches a shape number or a Law/Wire/Rollback section.
- `repair-damage.py` — fixes the two artifacts those passes can leave (a list
marker that lost its space, a table separator left without its header).
- `restore-punctuation.py` — restores a line whose only difference from HEAD is
stripped trailing punctuation. Written after `repair-damage.py`'s first version
matched every wrapped prose line ending in a comma and deleted 660 of them: the
rule is now `… see .` only, and the incident is recorded in its docstring.
- `gen-vault-index.py` — never walks into `.AGENTTRASH/`, and sweeps rows whose
page cell is empty (a description left nameless by a destruction).
Full suite: 451 tests pass.
The 1.8.1 boot resolved the budget resolver to 4.20 GiB of cache a card —
556,348 tokens, 2.8 sessions at full height — so three concurrent
full-height requests are refused with a 400 at admission: the endpoint's
3-slot contract is broken by the cutover, silently, because /health and
every shorter conversation still answer.
The resolver gives the expert plane whatever the pinned host pool cannot
hold and the cache the rest, so the pool is the lever: +640 MiB a rank
moves that much expert demand into host RAM and the cache wins it back.
Tokens scale with the GiB won (8.11 KiB/token at fp8), so ~0.63 GiB buys
~44,000 tokens — 600,000 with margin — at ~2.5% off the expert slab.
TTM floor moves with the pool (2 x pool + 48,824 + prefix + MNBT =
150,200 MiB computed by the ledger test).
Verified by the boot ledger's `kv:` line and the probe's capacity leg,
not by this arithmetic.
The decode stall is step contention (measured: 18.7 ms decode-only steps
against 317 ms steps that carry a prefill chunk, decode at 13 t/s median
while prefill runs), and the two engine knobs that fix it arrived after this
endpoint's pin. Moving to 1.8.1 (2026-10-18) brings both, and brings two CLI
breaks with them:
- --vram-weights-mib / --vram-kv-mib were REMOVED at 1.3.0. The per-card
split between the expert slab and the KV pool is now derived by the
engine's budget resolver from --gpu-headroom-mib, the cache bound
(--max-model-len, --max-num-seqs) and --host-pool-mib, and the cache may
lend idle blocks to the expert slab. The slab size and the pool's token
count are boot-ledger lines from here on, so the capacity contract
(3 x 200,000 fp8) is verified live by the probe's capacity leg instead of
computed from an env number.
- The prefix cache has no disk tier any more: --prefix-cache-dir and
--prefix-cache-disk-mib are refusals and --prefix-cache-host-mib is
--prefix-cache-mib. 64 GiB of parked KV on NVMe becomes re-prefill, so
prefix_hit_rate is the thing to watch after this boot.
env: image re-pinned to the 1.8.1 digest; the pair and the disk tier dropped;
FRANK_ARCHON_PREFIX_CACHE_MIB replaces the host/disk pair; new
FRANK_ARCHON_BUSY_PREFILL_CHUNK=auto (explicit, because auto vs off is the
difference between decode waiting one chunk and waiting a whole step) and
FRANK_ARCHON_PREFILL_INTERLEAVE=0 (opt-in through a guarded argv array).
serve: argv follows the new CLI, /data mount and the pre-created kvcache path
go with the disk tier, and the retired env keys are no longer required (they
would abort ExecStart). autoresearch-frank-archon.sh takes the same CLI, with
a cold prefill now coming from --prefix-cache-mib 0 instead of a per-run disk
directory. run-swift15-bf16.sh is untouched: it pins its own engines by design.
tests: the two budget-arithmetic seams asserted knobs that no longer exist and
are replaced by one seam that asserts the retired knobs are ABSENT from env and
argv - a stale line is a boot refusal, not a no-op. The container-contract seam
now asserts the tune mount and the absence of /data.
`scripts/port-census.py` built its corpus from the new directory, so each page
under test was held by itself: every era page reported 0 orphan lines. With the
corpus excluding the file under test, each candidate holds 14-63 sole-record
lines, so no era page is padding and none was trashed.
Deletions of restatement only:
- Multi-Node Runtimes.md (-257): restated launch material owned by
Qwen3.8 PP Deployment.md; index rows and two Related bullets repointed.
- log.md: 66 headings folded into 35 dated entries (one entry per date, newest
first), titled headings kept as `date — Title` per the spec's wording, and 78
lines of transient deploy chatter dropped where every fact token in the line
is held by an owner page (preserved in .AGENTTRASH/20261019-log-transient/).
- Authentic: the 5-page split reverted (-93); the parent carries
`okf-exempt: length` and the three stub pages stay deleted.
check-okf.py accepts `## YYYY-MM-DD — Title`; port-census.py excludes the file
under test and gained a docstring note about the trap. One-off compaction tool
moved to ~/.AGENTTRASH. Full suite: 451 tests pass, 0 OKF violations,
155 pages, 0 unresolved links.
`Multi-Node Runtimes.md` (+257) restated launch material that
[[Qwen3.8 PP Deployment]] (build target, two-process RPC table, cache-prune
cron), [[Sample Script Set]] (runnable scripts), [[Ethernet Bandwidth Math]]
and [[Expected Performance]] already own, and mixed-vendor numbers owned by
[[x4-pair-experiment]]. Consolidation means the facts end up in ONE place, not
that a new page is written beside the old owners: the four stubs it replaced
were frontmatter-only (9 lines each) and needed no successor. Page trashed,
three inbound links repointed at the real owners.
main moved under this campaign — PR #25/#26 resliced the RTX 3090 endpoint.
The frontmatter conflict is resolved in favour of the live shape, confirmed
by re-probe the same day: `--ctx-size 602112 --parallel 3` (3 × 200,704),
`--cache-type-k f16 --cache-type-v q8_0`, `--spec-type draft-mtp`, `--jinja
--reasoning auto`, `--metrics`. The 262,144 figures that remain on the page
are era rows and the checkpoint's trained window, not the live shape.
log.md's 2026-10-19 fleet-state bullet now records both probes and the law
this taught: a docs pass that reports live shapes must re-probe before
writing the number, because main moving mid-campaign makes the first probe
stale.
The ROCmFPX row said "single card, pinned at 36e9acd": `36e9acd` is the
SHA that `scripts/build-rocmfpx.sh` pins for the *build*, and the era page
records the deployed commit as `c49ebdb` running a 2× R9700 `-sm layer`
split over `ROCm0,ROCm1` at 131,072 × 1 slot with draft-MTP pinned to
`ROCm1`. The 2026-09-20 row skipped a payload generation: the same day the
endpoint swapped MXFP4 → FP8 E4M3 128×128 ([[frank-medium-model-swap]])
before MXFP6 arrived on 09-21. Both cells now quote the era pages.
- frank-archon KV dtype law: `bf16|fp8` is a **radiance** restriction. Stock
vLLM advertises `auto|fp8|fp8_e5m2|fp8_e4m3` (docs.vllm.ai engine args), and
even there the advertised set is not the usable set — several accepted
values crash engine init at the attention layer's own dtype check
(vllm-project/vllm#42587). The page stated the restriction as if upstream
had it.
- Open Notebook: re-censused 2026-10-19 — no unit file, no installed quadlet,
no container. The page already said NOT deployed; it now says what was
checked, so the claim is re-checkable instead of remembered.
Four bullets in Key Lessons Learned carried narrative a vault page owns: the
2026-09-20 board swap ([[Dual-R9700-Cooperation]]), the `--override-tensor`
gate ([[Llama Cpp Multi-Device Splits]]), the x4 link budget
([[x4-pair-experiment]]), and the silent-RAM incident plus the Prometheus
`--metrics` law ([[frank-archon]], [[frank-zealot]]). Each keeps its law, its
numbers and its error strings; the causal narration that duplicated a page is
what went. The RAM bullet linked to [[log]] — the incident is owned by
[[frank-archon]], so the link now names the owner.
A hyphen or a trailing period inside a word was enough to make it a tracked
token, so `long-standing`, `deployed:` and `cleaned.` were reported as lost
facts while real identifiers hid behind them. Require a digit, dot, slash,
colon, `--`, two hyphens (`frank-tts-engine`) or ALL-CAPS in the stripped
word. The census of the log rewrite went from 979 noisy lines to a set whose
unheld tokens are all fact-shaped.
Structural repair:
- 13 pages were the same document twice (each exactly 2x its HEAD size),
a second frontmatter block whose closing fence had been glued onto a body
line by a frontmatter rewrite that re-read its own output. Un-doubled by
cutting at the second frontmatter and keeping the copy whose body is a
superset; the orphan census reports 0 lost tokens against HEAD.
- log.md: one entry per date, bare `## YYYY-MM-DD` headings, newest first —
66 dated headings merged into 34, titles demoted to bold leads, every
content line verified present. The file had been descending at the head
and ascending at the tail.
Ownership moves:
- New `LLM/frank-medium.md` is the locator for the retired Dankserver
endpoint: era windows quoted from each era page's own banner, no numbers
restated, and the two unresolved date conflicts recorded instead of
silently resolved. `frank-archon.md` links to it instead of restating the
pre-rename window.
- The four Pipeline-Parallelism stubs (frontmatter only, no body) are gone;
`Multi-Node Runtimes.md` owns runtime construction, launch, mixed-vendor
verdicts and cross-split spec decode, and the four links that pointed at
the stubs were repointed at it or at the drafter page.
- Authentik's three stub pages fold into `Authentik.md`, with the process
detail and the outpost reference split out so the parent stays under the
length hint.
Verified corrections (host census + live probe, 2026-10-19):
- The MXFP4 checkpoint two pages called the "on-disk rollback" is gone:
/home/dank/models/ holds StillDeadcode, rad and tpl.jinja. Stated once, on
the era page that owns that payload; the swap page now links to it.
- frank-medium-radiance-mxfp4: the 4.24/7 accept rate belongs to the DFlash2
arm measured on the FP8 target, not to the MXFP4 era; the 3-family soak run
census is kept once (155 requests) instead of twice (~120 and 155).
- Fleet shapes re-confirmed against the live endpoints before touching any
page: 3x200000 fp8 KV, 2x200704, 2x262144.
Index rows are regenerated, not hand-carved. Net -1,849 vault lines.
Add scripts/check-okf.py: enforces the OKF v0.1 hard rules the repo had
been carrying by convention — parseable frontmatter with type/title/
description/timestamp, description shape (length, not-equals-title, no
restated quantity), the reserved-name and index.md rules, log.md's date
heading shape (bare ISO date, one entry per date, newest first), the
500-line hint, and the doubled-page signature (a second `type:` key in
the body). Citations are reported as a soft tally, not a failure. The
500-line ceiling is escapable two ways, both explainable: `okf-exempt:
length` in a page's own frontmatter, and pages derived as one-per-
endpoint — live endpoints from scripts/, retired-endpoint era pages from
their tag (AGENTS.md lets a campaign ledger live on the page that
measured it).
scripts/gen-vault-index.py: sibling-presence was tested as a bare stem
substring, so a page whose name is a prefix of another page's name
(`frank-medium` inside `frank-medium-rocmfpx`) silently never got an
index row. Test the wikilink form instead.
scripts/check-vault-links.py: `.AGENTTRASH` is deleted-but-held state,
not vault content. It is now excluded from both the scan set and the
target index, so a live link into the trash reads as unresolved instead
of resolved.
tests/docs_completeness.bats: the same exclusion in both page walks —
trashed pages need no index row, and a trashed page must not satisfy the
docs requirement for a live compose service. Adds the OKF conformance
and index-freshness tests.
scripts/port-census.py: the port-before-cut orphan census used to prove
a merge lost no fact, kept so the next collapse can re-run it.
Live step-shape census on frank-archon (radiance 1.0.13, boot of 2026-10-15):
decode-only steps run 18.7 ms, prefill+mixed steps 317 ms — 60 % of engine
wall-clock inside 8 % of the steps — and a day of Prometheus shows decode at
13.3 t/s median while prefill runs >=1000 t/s against 106-240 t/s when it does
not. The lag test says the penalty is confined to the prefill window: no
lingering expert-residency tail, and preemptions are zero.
Adds the method as LLM Performance Diagnosis S9 (counter arithmetic, the lagged
bucket correlation that separates step contention from residency churn, and the
four radiance knobs with their published costs), and the endpoint's own numbers
plus the verdict as two frank-archon laws: --busy-prefill-chunk and
--prefill-interleave are absent from this pin (proven via the running
container's --help), so MNBT is the only crank here and an image bump is gated
by the 1.2.0 numerics dead end.
No behavior change; docs only.
The boot of record for the new shape, from the verbosity-4 journal plus
settled nvidia-smi: target KV 14,406.00 (K f16 9,408.00 + V q8_0 4,998.00),
draft KV 2,352.00 over the same 200,704 cells x 3 seqs, recurrent state
452.25, compute 1,665.00 + 424.88 (draft 480.97 + 424.86), weights 2,283.29
+ 257.66, projector 595.29 + 248.10 + 24.93, settled 22,870 used / 1,255
free with n_slots = 3 / n_ctx_slot = 200704.
Two inherited numbers did not survive it, and both are corrected in the env,
the serve rationale and the page:
- The ubatch 2048 -> 1024 step was quoted at "-~1,306 MiB". That was the
786,432 / q8_0-pair arm. At THIS shape the target's compute buffers went
2,496.13 -> 2,089.88: 406 MiB, with CUDA0 going UP 257 while CUDA_Host
went down 663. q8_0 V (4,410 MiB, not the 3,840 previously written) is
what funds the drafter, alone.
- The fit gate priced 22,428 against a real 22,870: conservative by 442
MiB, so it stays honest as written.
And one finding stated plainly: this is the first boot on this card that
misses llama.cpp's own free-memory target - the engine printed "cannot meet
free memory target of 1,867 MiB, need to reduce device memory by 102 MiB"
and settled at 1,255 MiB free, under the ~2 GiB band every previous era
held. Cheapest rungs back are CTX_SIZE=401408 or SPEC_TYPE=none, both one
line.
Probes on the same boot go in too: chat turn (448 tokens, 1,755 chars of
thinking - empty content at max_tokens 64 is the documented trap), tools
(finish_reason tool_calls, {"a":24,"b":7}), vision (512 px solid red ->
"Red", 280 prompt tokens), and the drafter's own statistics (acceptance
0.706 / 0.763, mean accepted length 2.41 / 2.53).
The endpoint page owns its shape: the flags row, the budget equation behind
it (25,088 B/token, 14,406.00 MiB of cache + 2,352.00 MiB of draft cache
inside 24,123 MiB addressable), the "MTP is ON" section replacing the "OFF by
measurement" one, the harness contract at the per-slot 200,704, and the
fallback rungs rewritten as deltas FROM the shipped shape.
The 2026-10-18 exact-KV boot (KV 16,384.00 / RS 100.50 / compute 1,408.06 +
1,088.07 / mmproj 868.32 charged after the fit check, settled 21,176/2,949)
and its contended rate floors move into that era's Eras row with the one-line
rollback, so the two pages that carried them collapse to one. Rates for the
new shape are deliberately absent - no bench was ordered, and the fit gate,
the health gate and the boot ledger are the evidence a swap needs.
AGENTS.md follows the same numbers: contextWindow 200704 is the per-slot
window and the trio's shape line.
Frank's order, 2026-10-19: --parallel 3 at 200,704 a conversation (pool
602,112), K at the engine default (f16), V to q8_0, batch 1024x6 / ubatch
1024, and the MTP drafter armed explicitly.
The three requests are one budget equation. At 8,192 elements per token per
cache the pair now bills 25,088 B/token = 14,406.00 MiB over the 602,112
pool, and the armed draft context adds its own f16 KV over that whole pool
(4,096 B/position = 2,352.00 MiB) plus buffers and recurrent cells. Dropping
V to the 8-bit class frees 4,410 MiB against exact V and the ubatch drop
hands back ~1,306 - that is what pays for the third slot AND the drafter,
neither of which exact V could fund.
The serve script's written-down fallbacks move with the env so a missing env
file cannot boot the retired shape, and the seam now: asserts the per-slot
contract (divides evenly, <= the trained 262,144), bills an armed drafter in
the VRAM fit gate, proves the gate still REFUSES both boots this shape can
walk into, pins the shipped MTP set (and the absence of --spec-draft-model,
which would load the trunk twice), and pins SPEC_TYPE=none as the rollback
that strips the whole set.
Key Lessons Learned 483 -> 340 lines. Every law survives; only the essays
went. Port-before-cut census over the diff: 454 deleted lines, 0 orphaned
facts (every number still has a holder in the vault).
- Each lesson is now one bullet with the page that owns the detail.
- De-duplicated the page's own internal repeats: the layer-split law was
told twice (PCIe and ROCm sections), the FA-quants law twice more.
- CORRECTION, verified on dankclaw 2026-10-19: `GGML_CUDA_FA_ALL_QUANTS` is
deprecated upstream (master's ggml/cmake/common.cmake tells you to use
GGML_CUDA_FA_QUANTS=all), the successor's default pair list
`q4_0-q4_0;q8_0-q8_0;f16-f16;bf16-bf16` compiles the quantized-KV kernels
with the old flag OFF (master build-cuda cache: FA_ALL_QUANTS=OFF), and
neither llama tree references the old macro under ggml/src/ggml-cuda/.
The "required or prefill collapses ~12x" law is obsolete; the new law is
to read the CMakeCache pair list.
- frank-zealot: ported the three facts AGENTS.md alone carried, and verified
them against the host first — the 2026-10-18 13:16 cutover boot really did
load PTQ1_0 at `new slot, n_ctx = 133888` x3 (journal), the mmproj fit
check really printed 873.10 MiB against a 21,439 MiB projection, and the
warmup spread is 2.6 -> 1,258 t/s prefill.
- New tooling: scripts/check-okf.py (OKF v0.1 conformance linter) and
scripts/gen-vault-index.py (root index regenerated from frontmatter).
Co-Authored-By: omp <omq@openclaw.sh>
The 2026-10-19 vault de-dup campaign sent four read-only scouts over the same
pages and had to parent-verify every claim: one inverted era-page law (flagged
pages stale for documenting the retired payloads they exist to record), one
listed files that do not exist, one reported duplications grep disproved, one
cited a line past the end of the file it judged. Record the rule beside the
other evidence laws so the next agent does not integrate a yield unchecked.
Refs: vault de-dup campaign, PR #24.
The "`--swap-space` is removed; use `--kv-offloading-size`" claim was carried by
six pages, two of them with their own copy of the upstream source citation. It
now lives only in [[RAM CPU Offloading in VLLM]], which carries the 2026-10-19
verification against `vllm/config/cache.py` (no `swap_space`; `kv_offloading_size`
activates only with `kv_offloading_backend`):
- `VLLM Overview` (two argument tables), `VLLM Index` (parameter table),
`DFlash DSpark Speculative Decoding`, `Shared Memory Architecture Dankclaw`,
`Hybrid CPU GPU Processing Dankserver` -> cite the owner.
- `VLLM Index`'s example command used the dead flag; it now shows the live pair.
- `dankserver-vram-growth`: banner disambiguates the 2026-08 `frank-zealot`
(llama.cpp on the 3090 when that card was in Dankserver) from today's endpoint
of the same name on Dankclaw, so a retired crash signature does not read as a
live fault on a same-card-class service.
Refs: vault de-dup campaign, clusters C2 and C6.
The root and LLM index pages still advertised two retired endpoints' pages as
live procedure:
- `frank-medium-radiance-mxfp4`: frontmatter said "ACTIVE frank-medium
runbook" and the body opened "The ACTIVE frank-medium deployment". Both
re-keyed to the era it ran (2026-09-21 -> 10-10), with [[frank-archon]]'s
Eras table named as the owner of that shape.
- `frank-medium-model-swap`: read as live how-to for a retired endpoint.
Re-keyed to historical, pointing at [[Model Swap and Rollback]] for the
current swap law.
- Both index pages' rows re-keyed to match (rows are hand-copied from
frontmatter - no generator - so both sites move together).
Refs: vault de-dup campaign, cluster C5.
- `Running Qwen3.6 27B FP8 on Dankclaw`: had no banner at all and opened as a
live how-to. Added the RETIRED banner (iGPU endpoint today is
[[frank-dragoon]]), era-tagged the hardware table, and rewrote the
"primary service now runs vLLM-ROCm with Ling CIRU INT4" production note,
which was false by three fleet generations.
- `ling-3.0-flash-vllm-plan`: "frank-dragoon is the Qwen3.8-27B PP head" ->
past tense with the 2026-08-17 redo as the anchor and the live owner named.
- `frank-redo-2026-08-17` banner: stripped restated live shapes that had
already drifted (radiance 1.0.13, PrismML b10754 - the fleet runs 10777)
and the 3090-provenance clause owned by the PP deployment page; banner is
now locator-only (names + ports + links).
Verified before cutting: 457.9 t/s and the ping-RTT figures are solo-held by
Ethernet Bandwidth Math.md (no duplicate), so both scout-reported "duplicates"
were kept.
Refs: vault de-dup campaign, cluster C3.
One fact, one owner (de-dup campaign C4):
- `FastFlowLM NPU Inference.md` carried the host RAM figures three times
(29 GiB budget input kept as era data; the 2026-10-18 `free -g` re-reads
of 121/246 GiB cut -> [[Dankclaw System Specs]]/[[Dankserver System Specs]]).
- The 14-18 t/s PP-split decode is this page's own solo-held era measurement:
kept, tagged to the 2026-08 era with [[frank-redo-2026-08-17]].
- Promotion section: `frank-zealot` "now runs the NPU endpoint" era-tagged
(the name has been reused twice since; the NPU unit is `frank-npu`, masked).
- Repaired the Test Runbook splice introduced by the range edit above.
Refs: vault de-dup campaign, cluster C4.
Parent-written (scout summaries were corroborated against the files, not
trusted): every restatement of an endpoint shape now cites the endpoint page,
and four claims contradicted by today's fleet state are corrected.
- Quantization Production: the challenger-contract line quoted the retired
compaction-hub window; now cites [[frank-zealot]]'s Current shape. The 3090
worked example is era-tagged. Sources listed [[frank-dragoon]] three times.
- Quantization by Hardware Architecture: said the 3090 lives on Dankserver
(it moved to Dankclaw 2026-09-16; the Eras table owns the move) and ranked
the retired Ling-3.0 Quark INT4 service as the ACTIVE Dankclaw workload. The
active service is [[frank-dragoon]] on a closed `.hgn` format.
- GGUF Compatibility: Related claimed the ftype-115 ROCmFPX file was "in
production"; that is an era now, so it points at [[kingjones-q8agent-rocmfpx]].
- Model Swap and Rollback: dropped the fragment byte-count ledger (AGENTS.md
owns it), the Ampere FP8 hardware claim ([[Quantization by Hardware
Architecture]] owns it), halogen's HTTP surface ([[frank-dragoon]]'s wire
contract) and three brittle unit-file line refs (accepted loss: line numbers
are derivable, the root-unit law stays). Fixed the per-slot example, which
described the retired 3-slot arithmetic, and a Related bullet that called
[[frank-archon]] retired while linking the same page twice.
Census over the diff: 1 orphan line, the deleted unit line-number refs above.
The page claimed the unit had been failed since 2026-10-18 03:40 with a
hand-run process on the port. Verified live: both units are active under
their repo units (zealot since 21:06, the deployer's 2-slot reslice).
The dedupe passes kept trying to port single-owner experiment figures into
endpoint Eras tables. The law now states the test explicitly: two pages
carrying one figure is the defect; a figure with exactly one holder is fine
wherever it lives.
The 3-slot -> 2-slot reslice entry and the contended-rate entry carried the
full measurement ledger; the numbers belong to [[frank-zealot]] (shape block +
Eras rows), so the entry keeps the causal story (pool halving funds exact KV;
rates taken under a live consumer are floors) and links for the figures.
Enforces "one fact, one owner": technique pages keep mechanism, laws, traps
and verdicts; per-endpoint shapes, digests and era measurements are cited to
the endpoint page that owns them.
Also repairs the four pointers the pass left empty (each verified against the
cited owner before landing):
- EMULATION dequant doubles host RAM (118 GiB -> ~238 GiB) -> [[frank-archon]]
- 12.11 tok/s flat 304 -> 16,095 tokens -> [[Tcclaviger Fork Ops]]
- bundled-head acceptance ladder 0.739/0.554/0.427/0.340 -> [[Qwen3.8 PP Deployment]]
- llama.cpp PR #23897 dedup provenance -> [[qwen3.8-spec-matrix]]
Replaces the retired-shape illustration in Llama Cpp Serving Sizing with the
owned MiMo Q5_K_M row (6.40 GiB payload -> 20,530 MiB used), and unwraps a
hard-wrapped [[Dankserver System Specs]] link that failed link integrity.
Orphan census over the diff: 0 numbers lost (comma-insensitive match against
the whole vault plus AGENTS.md).
Conflicts and how each was resolved:
- AGENTS.md: kept the 2026-10-18 Model Swaps law (a payload change is the trio
edit, the deploy, /health and one probe; no bench unless Frank orders one).
The incoming Benchmarking-Policy wording it replaced is not lost: its one
durable fact -- on the PrismML fork build 10777 the per-request `timings`
object only reaches you on /v1/chat/completions, because a greedy ignore_eos
run on the raw /completion path answers 500 "Content-only format" -- is now
carried inside "When a bench IS ordered", step 2, and in [[frank-zealot]]'s
Contract section.
- frank-zealot-llama-prism-bonsai9-mtp.md: kept deleted. PR #22's additions
(the exact-KV boot ledger, the contended floor rates, the /completion
instrument trap) were ported into LLM/frank-zealot.md before the merge, which
is where the one-page-per-endpoint law says they live.
- log.md: auto-merged, then compressed to incident plus law -- the reslice
decision, the "a KV byte rate belongs to the payload it was measured on"
lesson that buried the invented 26,112 MiB figure, and the instrument law.
The boot ledger and the floor table now live only on [[frank-zealot]].
log.md drops 466 lines of per-swap ledger prose (1,169 lines out, 703 back) and
keeps the causal stories: why the host went down, why a knob resets at boot, why
a stale drop-in served a different model for 24 minutes, why the RAM was the
fault. Date sections are unchanged (65 before, 65 after) -- this is compression
inside sections, not deletion of days.
Every number the compression would have orphaned was ported to its owner first,
and a census of the deleted lines against the whole vault is what found them:
- frank-zealot: the two boots that priced non-KV (20,190/KV 12,800 => 7,390
without the projector; 22,972/KV 14,113 => 8,859 with it, so the vision tower
costs ~1.4 GiB for a 918 MB file), the 1,153 MiB-free 3-slot boot that made
the seam price non-KV at 9 GiB, and the f16 wall's own figures (fit 33,198 vs
23,558 free on a 25,088.00 MiB KV request).
- frank-archon: the pool-cut-is-headroom-not-arena law (2.75 GiB/rank bought
zero slots; d(reserve) = d(arena) = +248 slots; 14.87 GiB / 6,496 slots
against a 398,658-token pool), the MNBT compile-RANGE vs SEQS+1 law, and the
2 x 196,919 prefix-cache proof (393,837 tokens resident above the pool's
386,904; 94.1/102.3/117.6 t/s, acceptance 0.7284/0.7525).
- frank-dragoon: the KL arm of the 2026-10-17 A/B (mean 0.155 / median 0.057,
top-1 agreement 85.1 %) joins the PPL and NIAH figures already on the era row.
- Payload and Disk Reclaim: the 121 GB layer census lesson -- a huge image layer
can be the engine image, not a payload.
Also fixes a wikilink the rewrite had hard-wrapped across two lines, which the
link checker reads as a target that does not exist.
New seam: 'one page per live LLM endpoint'. The live set is derived from
scripts/ (an endpoint has both an env and a unit), and any <endpoint>-*.md page
in the vault fails it -- the enforcement of the one-page-per-endpoint law, so a
re-created era page cannot pass tests.
18 pages cut their per-endpoint shape, payload, port and window claims and
link [[frank-archon]] / [[frank-dragoon]] / [[frank-zealot]] instead. Several
of the cut claims had already drifted -- the stack page still described the
kingjones ROCmFP4 era on ROCmFPX, OpenWebUI carried the 27B/3-slot bench
narrative, Hermes named a retired max_model_len and the old service name, and
the OMP harness page's live-window block had drifted off frank-zealot's real
262,144 (it is now labelled a dated worked example whose authority is the
endpoint page).
Two facts gain a single owner instead of two: the up{job=~"frank-.*"} scope
lives in Infrastructure/Prometheus.md, and the blackbox/Anubis probe form in
Architecture/Stale Signal Audit.md, each cited from the other page.
Voice Packs and Frank TTS carried live-state claims that contradicted the
engine page's 2026-10-18 deploy record (nothing listening on 7772/8880, model
dir absent); they now point at the pages that own host state.
Every payload swap a live endpoint has ever run is now one row in that
endpoint's Eras table, with its rollback path in the same row. No per-attempt
page, no per-swap ledger, nowhere else.
- Added: LLM/frank-archon.md (radiance, 2x R9700 TP=2), LLM/frank-dragoon.md
(halogen flash-server, 19 eras), LLM/frank-zealot.md (llama.cpp PrismML fork
on the 3090). Each carries Current shape, Eras, Laws, wire contract, files.
- Deleted: 45 per-payload runbooks -- the frank-dragoon line (11 including the
pre-name halogen/kingjones/kairic/CIRU/eemin/agentionai/dflash2 pages and the
benchmark page), the frank-zealot line (llama.cpp/vLLM/ik_llama/SGLang eras),
the frank-archon line (vLLM/FreeToken/R9V/llama.cpp/radiance eras), plus
frank-medium, frank-npu and the ubatch/PCIe one-offs that owned nothing.
- Rewired: every wikilink and markdown-path reference to a folded page points
at the endpoint page instead; the three Research see-also lists and the
ubatch sweep's runbook link included. No anchored links existed in the set.
- Regenerated both index pages as pure locators -- one row, one clause, no
numbers -- so the docs_completeness locator law holds for all 153 pages.
tests/docs_completeness.bats green: index links every page, no unresolved
links. The pages moved to .AGENTTRASH/20261018-doc-dedupe/LLM/ (gitignored,
and git history holds every byte).
Every KEY=value is byte-identical to the previous revision (diffed against
git show HEAD: -- 28/25/20 keys). What left is the per-swap narration: the
archon header drops 207 lines of era prose, dragoon 127, zealot 104. Each
header now keeps the format lock, the digest/identity law, and one line per
knob explaining what a reader editing that line cannot recover anywhere.
The facts moved to their one owner: LLM/frank-archon.md, LLM/frank-dragoon.md,
LLM/frank-zealot.md. One stale claim dies with the trim -- the dragoon header
still said the KV pool is ONE 200,704-position budget behind 2 slots, which
contradicted its own 401,408 paragraph; the page carries the correct ledger.
Also kills two wrong arithmetic claims in the zealot header: the q8_0-V and
CTX_SIZE=401408 rungs each free 3,840 MiB at the 524,288 pool, not 8,192 and
not ~6.7 GiB.
The runbook index collapses to three live endpoints plus a retired-row
locator: a payload swap is an Eras row on the endpoint's vault page, never
a page of its own. Restated shapes, digests and measured numbers are
replaced with links to [[frank-archon]], [[frank-dragoon]], [[frank-zealot]]
and scripts/<svc>.env, which is the authority for every value.
Carries the frank-zealot instrument correction from d457985: on the PrismML
fork build 10777 the per-request timings object only reaches you on
/v1/chat/completions — the raw /completion path answers 500 on a greedy
ignore_eos run.
Deployed shape verified live: KV 16,384.00 MiB (ledger-exact 32,768 B/token x
524,288), RS 100.50 MiB (2 cells), settled nvidia-smi 21,176 used / 2,949 free,
n_slots 2, n_ctx_slot 262144, DropInPaths empty.
Rates recorded as FLOORS: another consumer drove the endpoint through the whole
window (56 prompts on the other slot in three minutes, 96% GPU-util). Single
stream 134.1-141.5 (145.8/163.2 in a lighter minute) vs the 168.7 q8_0 record;
two-stream aggregate 256.6-265.1 vs ~300 across three; prefill 3,159.5 at
106,282 beats the 2,877 record while contended; decode at 106k depth 71.5 (new
baseline, no prior). Per-stream concurrency improved: ~130 each vs 102.8.
Instrument correction: the PrismML fork build 10777 answers 500 'The model
produced output that does not match the expected Content-only format' for a
greedy ignore_eos run on the raw /completion path, so the per-request timings
must be read off /v1/chat/completions. AGENTS.md Benchmarking Policy corrected.
AGENTS.md: topology row, the frank-zealot runbook section (shape, KV
accounting, MTP verdict, compaction-hub concurrency), and two Key
Lessons bullets -- the draft-context bill restated with the real knob
names, plus a new lesson that a quant's scale overhead belongs in its
byte rate (the 6.25 % invented by doubling 13,056.00 MiB).
LLM/frank-zealot-llama-prism-bonsai9-mtp.md: retitled to the live shape;
the 3-slot q8_0 boot ledger and its measurements are relabelled as the
arm this replaced (byte-identical numbers, honest attribution), the
corrected f16 rate derivation is added, the MTP census is labelled
q8_0-era, the fallback rungs are re-derived for the 524,288 pool, and the
boot-ledger section carries an explicit PROJECTION that must be replaced
with the measured ledger from the first f16 boot.
Pointers re-pointed (live-shape sentences only; every historical ledger
left byte-identical): LLM/index.md, vault index.md, Dankclaw/OpenWebUI.md,
and the retired-runbook "today" pointers in mimo-q5km, mimo-q6k,
qwen35-9b-q6k, swift15, sglang-mimo-v26. log.md gains the dated entry.
Frank: "shift frank-zealot to 2 parallels of unquantized kv".
Shape: CTX_SIZE 786432 -> 524288, PARALLEL 3 -> 2, CACHE_TYPE_K/V
q8_0 -> f16. The per-slot window is UNCHANGED at 262,144 (the native
trained window, and models.yml contextWindow), so what the exact cache
buys is fidelity and what it pays is one concurrent conversation.
The POOL is the term that had to move: exact KV bills 32,768 B/token at
this geometry (8 full-attention layers x 4 KV heads x 256 dim, 2 B,
K and V), so the old 786,432 pool asks 24,576 MiB against the 24,123 MiB
the driver leaves addressable. 524,288 positions = 16,384 MiB, projected
~21,200 used / ~2,900 free from the 13:19 boot ledger (only the KV pool
and one recurrent-state cell move).
Also corrects a repo-wide arithmetic error: "f16 would bill 26,112 MiB"
doubled the q8_0 ledger figure INCLUDING its 1/32 scale overhead. q8_0 is
1.0625 B/element, f16 exactly 2.0, so the true ratio is 1.882x. The wall
stands; the number was wrong.
And it corrects a false claim about the drafter: this build DOES expose
--spec-draft-type-k / --spec-draft-type-v (verified against
llama-server --help), so the draft cache is quantizable -- the
exact-KV-plus-MTP exclusion is a measurement gap, not a law, and the
1,088 MiB quantized-draft arm is recorded as unmeasured.
Seam: tests/frank_zealot_unit.bats re-pins the shape contract (2 slots,
524,288 pool, 262,144 per slot) and adds a VRAM fit gate that computes
dtype x pool + the measured non-KV ledger terms against 24,123 MiB with
>=1 GiB of slack -- and proves the gate REFUSES exact KV at the old
pool, so passing means something. 18/18 green.
A consumer that lists the 29-name af_*/am_*/bf_*/bm_* pack and refuses
narrator1 without a request reaching the façade is synthesizing in the
browser. Frank TTS's troubleshooting table gains that symptom; OpenWebUI's
See Also gains the service page.
The `Voice "narrator1" not found. Should be one of: af_heart, …, bm_fable`
toast is OpenWebUI 0.11.4's bundled Kokoro.js validator: the enum is
Kokoro's fixed 29-voice pack, not frank-tts's registry. Frank-tts answers
200 audio/mpeg for the exact OpenWebUI body from inside the container, and
the failing click leaves no `audio/speech` line in the container log — that
contrast is the instrument. Admin config DB is correct
(engine=openai, base URL :7772/v1, model frank-tts, voice narrator1); no
container env and no user row carries an audio override, so the
`browser-kokoro` choice is per-browser storage. Fix: Settings → Audio →
TTS Engine → Default.
#18 shipped DEFAULT_VOICE and #17 explained why the harness cannot name a voice from a setting, but the endpoint page still described voice resolution as if every request carried a name. One contract row, one paragraph in Naming a voice (a call that names nothing is the harness default shape, not a client bug), the env-file row now lists the default voice among its knobs, and the worklog records the rollout: both merges, the one-dispatch deploy with the 33/33 gate, the card flipping to up on the next checker tick, and the previously failing call returning 430124 bytes with the journal reading "speech voice=narrator1 (default)".
The omp speech device forwards a voice only when the call passes voice_id - there is no setting that feeds one (speech.voice is the TUI speak-aloud Kokoro enum, tts.localVoice belongs to the local backend) - so every harness voiceover arrives with no voice key and the shim answered 400 "unknown voice None". Measured on the deployed endpoint 2026-10-18, from a restarted session with the speech role pointed here.
DEFAULT_VOICE makes the absent case a spoken operator choice instead of a code surprise: the pack named there answers a voice-less request and the journal line is marked (default), so an unrequested voice never reads later like a requested one. An UNKNOWN name still 400s with the list that exists - the no-silent-fallback law that this endpoint was built on is untouched, and the strict shape stays the code default (empty DEFAULT_VOICE), which the second new seam test pins. Ships narrator1 in scripts/frank-tts.env.
The section this PR adds claimed speech.voice was the forwarded voice key. It is not: the openai-speech transport builds {model, input, response_format, voice?} and sets voice only when the CALL passed voice_id, tts.localVoice is the local Kokoro backend key, and speech.voice is the TUI speak-aloud voice whose enum is Kokoro-only. A restarted session proved it the honest way - the tts device arrived at 7772 with no voice key and got "unknown voice None; available voices: narrator1, narrator2, narrator3" - so the fresh-process success earlier that day was the run that was told to pass voice_id, not evidence of a setting.
The law now reads: voice comes from voice_id alone, so a voice-less call is the harness default shape (the endpoint grew DEFAULT_VOICE in its own PR to answer it), and speech.voice must stay a Kokoro name - a pack name there breaks the vocalizer and does nothing for synthesis. config.yml reverted to am_fenrir on the operator box, and the worklog entry that recorded narrator1 as wiring carries the correction.
Six statements still claimed the endpoint did not exist: both vault descriptions, the two index.md bullets, the Open Notebook blocker, and the engine page deploy order that opened with "Nothing below has been run". All re-keyed to the deployed state, with the ledger the deploys printed: engine healthy 31 s after the build at 4.0 GiB resident, three voices warm, the ladder measured at 1883 ms then 1226-1283 ms (not the retired 6.2-8.0 s), and two voices rendering the same sentence into different bytes.
Adds a The harness speech role section to the endpoint page with the three wiring laws measured the day of the flip: speech.voice is the forwarded key (tts.localVoice belongs to Kokoro), an explicit speech role has no fallback candidate, and models.yml loads at process start while config.yml is read live - so a session older than the frank-tts row fails exactly like a dead endpoint. Fresh-process proof: omp -p --tools tts saved 180524 bytes with voice=narrator1 while the facade logged the matching speech line.
The endpoint deployed on 2026-10-18, so the admin card must carry its state rather than a placeholder: wip becomes false (the registry schema wants the key on every service; false is what the live cards use) and the description drops the "Hidden until deployed" line. showLink stays false and the card stays admin-only: an HTTP API has no page to open, and homepage-status.py counts ANY HTTP response as up, so the card is honest with no link and no /health path in the probe. The voice seam that pinned wip == down is re-keyed to the deployed contract and now also refuses a link or a user-category placement.
PR #15 corrected the engine's page, the research brief's law bullet and the
warmer's header, but the retired ladder had six more homes: both quadlet files
("6.2-8.0 s… 0.5 ms", "~6-8 s each", "paying the 5.5 s cold decode"),
`scripts/frank-tts-engine.env` (the whole law paragraph and the WARM_TEXT note),
`scripts/frank-tts.env`, the [[Frank TTS]] troubleshooting row that told an
operator to expect ~5.5 s, the research brief's consequences paragraph, and one
test comment.
All re-keyed to what was measured on the deployed container 2026-10-18: the
first synthesis of a freshly registered voice costs 1,883 ms against 1,226-1,283
ms afterwards, so the warm pass buys about 0.6 s — and the reason it exists at
all, which the pages never led with, is that the registry is memory-only: without
the hook every voice name is a 400 until somebody synthesizes. The trial's dated
worklog entry stays as written; the correction points at it.
No behavior change: comments, unit lore and docs only, so no cfhash move and no
image rebuild.
The first real boot gave the chance to measure what warming actually buys, and
the trial's numbers were wrong:
- Empty registry, register only, then time: the FIRST synthesis of a freshly
registered voice cost 1 883 ms; the next two 1 283 and 1 226 ms. A ~110-word
request cost 9.4 s and then 9.8 s.
- So the published 6.2–8.0 s cold / first-audio-byte 5 530 ms / later-requests
0.5 ms ladder does not reproduce, and request time tracks the audio generated
rather than snapshot state. The 0.5 ms figure is an internal state-restore
cost, never an end-to-end request time.
- Warming still earns its place for a different reason than the page claimed:
the registry is memory-only and re-registering does not warm, so a boot without
the hook leaves the façade answering 400 for every name. The saving is ~0.6 s.
Law 1 of [[Frank TTS Engine]] is re-titled the registry law and carries the
measured numbers; its example journal line is a real one from today; the
troubleshooting row for "first request of the day is slow" — the retired law's
symptom — is replaced by the argument-cap fault that actually happened. The same
ladder in the research brief is annotated rather than erased, and [[Voice Packs]]
step 6 stops claiming the hook registers only packs it has not seen (the engine
replaces a name it already holds).
The deploy of the newly built engine image came back `0 voice(s) warm` with
`/app/warm-voices.sh: line 152: /usr/bin/curl: Argument list too long` above
every line, and the unit was `active` with a green `/health` beside it. The hook
built `{"name":…,"wav_b64":…}` in a shell variable and passed it as ONE curl
argument; the kernel caps a single argument at MAX_ARG_STRLEN = 131,072 bytes,
and the shipped reference clips base64 to about 1.27 MB, so curl was never
executed and the hook reported the absence as "HTTP no response".
The body now goes to a mktemp file and curl reads `@file`; the audio enters
through a redirection instead of a command substitution, so the script never
holds a 1.3 MB string at all. Every pack in the fleet is ~1 MB, i.e. ~13x the
cliff, so this was always going to happen — it took the first real packs to show
it because no fixture was ever that big.
The regression test therefore ships a 300 KB pack against the seam's REAL stub
listener with the REAL curl: it fails with the same kernel error the host did
(`Argument list too long`, then the 400-for-every-name summary), and passes once
the body is a file. Proved red before the fix, green after.
The vault link check reads [[Frank TTS Engine]] literally, so wrapping it made
the target "Frank\nTTS Engine" and the unresolved-link seam failed. Rewrapped,
not reworded.
Three faults, all of the reporting-success-while-doing-nothing species: the
build that could not start (no adduser in the base), the warm pass that could
not warm (podman's --env-file keeps the quotes), and the image that would never
rebuild (cfhash over the Containerfile alone). Records the proofs, the two
behaviour seams, and what stays undone: nothing is deployed, #2 phase 1 stays
open until 7772 answers.
Three pages, all from faults found this session:
- `LLM/deployment.md` gains "Locally built images": which images have no
registry digest, what the `cfhash` skip is for, why it must cover every COPY'd
file (the silent shape: build skipped, /health green, old code serving), and
the base-image half of the law — the engine's `useradd`, because the pinned
ubuntu:24.04 has no `adduser`.
- `Dankclaw/Frank TTS Engine.md` troubleshooting gains the two lines an operator
actually needs: every voice cold on a fresh boot with a hand-run hook
succeeding (= `WARM_TEXT` arrived quoted, `podman exec … env | grep WARM_TEXT`
proves it), and a 502 from the façade beside a 200 `/health` from the engine
(model loaded ≠ voices registered). The stale-warmer row is re-keyed to the
new rebuild rule.
- `Dankclaw/Voice Packs.md` step 4 was wrong: it told the next agent to restart
the engine *instead of* the warm hook because the hook "only registers packs it
has not seen". It registers every pack on every run and the engine replaces a
name it already holds — `tools/tts-server.cpp:148` at the pinned commit:
"Re-registering a name replaces the previous entry." A restart is what an
engine bump needs (the registry is memory-only), never worse, only slower.
- `Dankclaw/Frank TTS.md` deploy command re-keyed to `./scripts/deploy.sh
dankclaw --service frank-tts`, the way it is actually deployed; the raw
`deploy-remote.sh` form is plumbing.
build_compose_image skips `podman build` when the installed image's `cfhash`
label matches a hash taken from the checkout now — the skip is what keeps
frank-tts-engine's ten-minute recursive source clone off every re-deploy. The
hash covered the Containerfile ALONE, so any change to a file the image COPYs
was invisible: edit dankclaw/voiceservice/app.py or tts-engine-warm.sh, re-run
the deploy, and podman reports the existing image is fine. The redeployed
container runs the previous code, /health answers 200, and the e2e gate has no
way to see it. It bit the boot warmer twice this week.
Each case arm now names its own extra=(…) — the paths its Containerfile COPYs —
and the label hashes the Containerfile plus those. The two arms are kept apart
deliberately: the façade bakes app.py and tts-start.sh, the engine bakes
tts-engine-start.sh and tts-engine-warm.sh, and comfyui/loki/alloy have no extra
files, so their hash is unchanged and they do not rebuild on this commit.
Hashing rather than word-splitting because these are repo-relative paths under
a fixed checkout root.
The seam drives the real function against a stub podman and asserts observable
output — did the deploy ask podman to build, or did it skip? — in five steps:
first deploy builds, unchanged tree does not, editing the engine's COPY'd warm
hook rebuilds the ENGINE image and not the façade, editing app.py rebuilds the
façade. Reverted scripts/deploy-remote.sh to prove it red on the old hash: the
warm-hook step is exactly where it fails.
`WARM_TEXT` reaches the container **with its quotes**. The unit reads the env
file through podman's `--env-file`, which takes the literal remainder of the
line — no shell parsing — while bash `source` strips one layer. The hook
interpolated the raw value into the synthesis body, so with the quotes intact
the JSON was invalid and every voice logged "registered, but the warm synthesis
failed" while a hand-run hook (bash shape) succeeded. Reproduced both ways
against a real `podman run --env-file`, not by reading the man page.
The hook now strips exactly one matched pair — only when both ends are the same
quote AND the inner content carries another quote or a backslash, so a value
that legitimately contains quotes is never mangled — and then JSON-escapes what
is left, which is what makes a sentence like `He said "hi\" once"` arrive as
itself instead of corrupting the body.
The seam gains a test that reads `scripts/frank-tts-engine.env` BOTH ways —
`source` and the way `podman --env-file` does — and asserts the engine receives
the same sentence either way, in both shapes. A test that only sources the file
cannot see this bug, which is how it survived the first deploy.
The runtime stage of Containerfile.engine ran `adduser --system …`. The pinned
ubuntu:24.04 base ships no adduser, so the build died at that RUN line with
crun's "executable file not found in $PATH".
Verified in the base itself on Dankclaw, 2026-10-18: `command -v adduser`
prints nothing, and the exact replacement line — `useradd --system --uid 10000
--home-dir /app --shell /usr/sbin/nologin tts` — answers `uid=10000(tts)
gid=999(tts)`. Same account as before: numeric uid/gid 10000, non-login shell,
no password entry. Its one warning ("uid 10000 is greater than SYS_UID_MAX
999") is cosmetic and is what keeps the uid equal to the façade's, so the
comment says so before someone "fixes" it by dropping --system.
Stage 2 of the voice stack. Repo state only: nothing deployed, no host touched,
no container started.
- `frank-tts` is now registered everywhere a dankclaw service has to be:
`SERVICE_ORDER` (after `frank-tts-engine`, before the consumers), the planner's
dankclaw set, `deploy-remote.sh`'s allowlist + dispatch + a
`deploy_frank_tts` handler, and `deploy.sh`'s dispatch regex — plus the
`build_compose_image` arm naming `dankclaw/voiceservice/Containerfile`, without
which the build silently returns 0 and the unit's `Pull=never` tag never
exists.
- The unit joins TWO networks, in order: `dankclaw_default` first (default route,
where consumers and open-notebook live), then the voice bridge. `TTS_UPSTREAM`
moves from `host.containers.internal:8880` to `http://frank-tts-engine:8880` —
the actual fix for a shape that could never work, since a bridge container
cannot reach a host-loopback bind and the engine publishes nothing at all.
- `Wants=`/`After=` on the engine, never `Requires=`: the `llama-tts-cli` rollback
backend needs no upstream, so a broken engine may delay this endpoint but must
not take its unit down.
- The handler installs the engine's quadlet pair too, because a referenced
`Network=` file the generator cannot find means no façade unit is produced at
all, and `--only frank-tts` has no other way to get it. It creates
`/home/dank/voices` as the deploy user (rootless podman would auto-create a
missing bind source root-owned, which reads as "no voices"), and gates on
podman's recorded health.
- Voice naming is now written down where it can be found: a voice is a lowercase
directory name matching `^[a-z0-9][a-z0-9._-]*$`, at most 64 chars. That shape
is the intersection of two rules that disagree — the shim would accept mixed
case, the engine lowercases names — so a mixed-case pack resolves on one side
only, and the boot warmer refuses it loudly. `narrator1/2/3` are load-bearing
for open-notebook.
- Homepage card for the pair, `wip: down` and hidden, probing `127.0.0.1:7772`;
the engine gets no card because it publishes nothing and could never be probed.
- **No Prometheus job, deliberately.** No `/metrics` exists on the shim, this
Prometheus 3 refuses HTML/JSON bodies (the existing `comfyui` and `services`
probes are documented down for exactly that), and `LlmEndpointDown` is scoped
`up{job=~"frank-.*"}` — so a `/health` job would page on Signal for a healthy
endpoint. Recorded in the page instead of quietly skipped.
- Vault: new [[Frank TTS]] page linked from the index; [[Frank TTS Engine]] now
points at it for the pair's public signal; log entry for the day.
- Seam grows to 25 tests: the generator's rendering of the pair (both
attachments, first-network order, one publish, Wants-not-Requires, bounded
per-unit blocks), the env file's upstream checked as a network JOIN rather than
a string, planner separation and order, the handler's steps, the card's gating,
routing that selects both seams for either deployable.
Repo state only: nothing deployed, no host touched, no container started.
- `frank-tts-engine` on Dankclaw: `qwentts.cpp` commit
5916bf7c79dbb4e66fc059cdc507a79a23a28bb4, resident `tts-server`, 1.7 B base
talker Q4_K_M + Q8_0 tokenizer, container port 8880. It is deliberately NOT a
fleet endpoint — `frank-tts` on 7772 is the endpoint consumers address.
- Publishes NOTHING. A host-loopback publish would have been useless to the
façade (netavark DNATs published ports only, so a bridge container cannot
reach a host loopback bind — the shape that 502'd hermes) and would have been
the one LAN-reachable surface of an unauthenticated synthesiser. Both voice
containers sit on one committed quadlet network, `frank-tts-engine.network`
pinned to 10.89.5.0/24 (next free /24 after open-notebook's 10.89.4.0/24), and
peers address the engine by container name. Diagnostics and the deployer's
health gate run inside the netns (`podman exec`, `wait_container_healthy`).
- Cold-voice law honoured in the unit: the first request for a registered voice
decodes its whole reference to snapshot codec state (6.2-8.0 s, first audio
byte 5 530 ms), later ones restore in 0.5 ms, and the registry is memory-only.
`ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh`
with `TimeoutStartSec=900`; the `-` prefix keeps a failed warm from
crash-looping the endpoint under `Restart=always`. It logs cold=/warm= per
voice, so the under-a-second second request is proven on every boot.
- Thread law recorded: `tts-server` has no `--threads`; src/backend.h uses
hardware_concurrency()/2, so a cpuset pin silently changes the thread count.
- Image builds from the pinned commit with the proven line (GGML_BLAS=OFF
forced by this host, every GPU backend off, -j 8, no -DGGML_NATIVE because
ggml defaults to -march=native for the building host). Two stages; the runtime
stage COPYs both the entrypoint and the warmer, because a container that
publishes nothing is reached with `podman exec`, not a port.
- `scripts/download-frank-tts-engine-qwentts.sh` fetches the two GGUFs with byte
and LFS-oid checks (talker 1 219 245 248 B, tokenizer 291 150 624 B); the
deployer gates on --verify-only instead of pulling 1.4 GiB mid-deploy.
- Registered in SERVICE_ORDER, DANKCLAW_SERVICES, the deploy-remote.sh dispatch,
its pre-deploy requirement, deploy.sh's regex, the planner's path map (the
whole `dankclaw/voiceservice/` tree mapped to nothing before this), and
test-changed.sh ABOVE the `dankclaw/*.container` case whose `*` crosses `/`.
- New seam tests/frank_tts_engine.bats (19): the Quadlet generator's own output
for network/publish/boot/warm shape, entrypoint argv driven from the tracked
env file, the warmer against a stub tts-server over a real socket
(registration keys, ref_text absent, late engine, hung synthesis, name
refusal), the fetcher's verifier, the pinned cmake line, and both routing
tables. Suite: 422 passed.
- Vault: [[Frank TTS Engine]] linked from the index; log.md entry for the day.
The page a new operator needs: what the three required files are and who
reads each one, how a pack is built (with the extractor's usage text
verbatim, because the flags are not guessable and there is no -o), the
10-second gate and where it fires twice, how to add a fourth voice
including the warm hook that registers a pack without restarting the
engine, and the engine-bump procedure for when the artifact sizes move.
Plus the host census as of 2026-10-18 — the trial packs carry no
artifacts and their manifest is the trial's own schema, so the builder's
first real run is a --force migration — and the three gaps left open on
purpose: no curated ICL transcript, no 16 kHz variant, no loudness pass.
Linked from the Dankclaw services list; log entry appended oldest-first.
scripts/voice-packs.env carries the decisions worth a diff and nothing
else: the pack list, each voice's source file, crop offset, crop length
and pinned source digest, the extractor path with its engine repo and
commit, and the artifact arithmetic. Paths and numbers only — the pack
directories themselves stay host state under /home/dank/voices.
tests/voice_pack_unit.bats pins behaviour, from inputs to observable
output: a crop inside the first ten seconds exits nonzero naming the
invariant and creates nothing, a legal one builds the whole pack, the
reference comes out 24 kHz mono inside 18-25 s (ffprobe, on a locally
generated tone), the manifest round-trips through jq with offsets and a
sha256 that matches the source bytes, a re-run without --force leaves
every byte untouched, --force retires rather than deletes, --dry-run
needs no extractor at all, and a stub extractor that changed its artifact
size cannot publish a voice. No engine binary, no GPU, no network, no
fleet host: qwen-codec is stood in by a stub with the real tool's I/O
shape, the way frank_tts_unit.bats drives its backends.
test-changed.sh routes both new paths to that seam instead of letting
them land on the generic scripts/* syntax fallback.
Turns a source recording plus a crop offset into the pack frank-tts
resolves by name: reference.wav (24 kHz mono pcm_s16le, 18-25 s), the
engine's own reference.spk / reference.rvq made from it, and manifest.json
provenance (source path + sha256, crop start/end, sample rate, duration,
tool, engine commit, voice name).
The extractor is qwen-codec from the pinned qwentts.cpp tree, and the
flags are its own usage text, not a guess: --model <codec gguf>, --talker
<base talker gguf>, -i <wav>, optional --device; there is no -o, so it
names both artifacts next to the input with the extension swapped, which
is why the pack's WAV is staged under its final stem.
The 10-second source-material invariant is enforced, not annotated: an
offset inside the forbidden region exits 1 before a directory is created,
and the gate is asserted a second time from the manifest that was just
written, together with the recorded sha256. Artifact sizes are checked
against the codec's own arithmetic (8,192 B x-vector; 22 B per 12.5 Hz
frame), so an engine bump fails the build instead of shipping a pack the
engine rejects at synthesis time — and the crop is refused up front when
it is too short to fill the .rvq window.
reference.wav is the file the façade gates on, so it lands last, after
the artifacts and the manifest: a voice name never exists on disk without
its artifacts behind it. Nothing is ever deleted — an existing pack is
refused unless --force, which retires it whole to .AGENTTRASH/ — and
--dry-run runs every step except the extractor, so the seam can test the
gate without the engine.
Frank heard every audition pair and passed it, so likeness stopped being the
tiebreak and the engine was picked on the live-output criterion: arm A
qwentts.cpp 5916bf7, resident tts-server, 1.7 B talker, llama-tts-cli kept as
the rollback. Records the discriminators (first audio byte, warm RTF, spread),
the three consequences that are configuration (boot-warm every voice, no cpuset
pin, WAV-only registration), the two follow-ups left open on purpose, and the
corrected addressing law: a host-loopback publish is unreachable from a bridge
container, so the pair shares one quadlet network.
Mirror of command-center clarification: first dogfooding round stamped the
build worktree instead of the session's own working directory. Rule text
now says which one, and to note the build worktree in the body.
Mirror of dank/command-center's rule (there: PR off docs/session-trailer-rule).
Every agent-opened issue/PR body ends with cwd + omp session id so machine-filed
artifacts trace back to the box and session that produced them.
Refs #11 (command-center tracker)
The env file first landed with ESPERANTO_TTS_TIMEOUT equal to SHIM_TIMEOUT=600.
Equality is a race: both budgets expire at the same instant, so the case the
number existed to prevent — the worker abandoning a synthesis frank-tts is still
finishing — is exactly what happens on the tail of a long line. Nested budgets
go inside-out, which is the rule scripts/frank-tts.env states for its own
LLAMA_TIMEOUT; the client waits one round-trip longer than the server, so a
frank-tts give-up arrives labelled as frank-tts's error. 660.
The seam's assertion moves from equality to `>` for the same reason, and now
also holds ESPERANTO_LLM_TIMEOUT under upstream's 600 s web-UI patience — above
it the failure surfaces as a UI timeout with no name on it. The scriptwriting
number itself stays at the app's 180 s default: frank-zealot answers an outline
or a transcript well inside it, which corrects the earlier commit message's
claim that both timeouts were raised past upstream's 300 s.
Frank
