The omp speech device forwards a voice only when the call passes voice_id - there is no setting that feeds one (speech.voice is the TUI speak-aloud Kokoro enum, tts.localVoice belongs to the local backend) - so every harness voiceover arrives with no voice key and the shim answered 400 "unknown voice None". Measured on the deployed endpoint 2026-10-18, from a restarted session with the speech role pointed here.
DEFAULT_VOICE makes the absent case a spoken operator choice instead of a code surprise: the pack named there answers a voice-less request and the journal line is marked (default), so an unrequested voice never reads later like a requested one. An UNKNOWN name still 400s with the list that exists - the no-silent-fallback law that this endpoint was built on is untouched, and the strict shape stays the code default (empty DEFAULT_VOICE), which the second new seam test pins. Ships narrator1 in scripts/frank-tts.env.
PR #15 corrected the engine's page, the research brief's law bullet and the
warmer's header, but the retired ladder had six more homes: both quadlet files
("6.2-8.0 s… 0.5 ms", "~6-8 s each", "paying the 5.5 s cold decode"),
`scripts/frank-tts-engine.env` (the whole law paragraph and the WARM_TEXT note),
`scripts/frank-tts.env`, the [[Frank TTS]] troubleshooting row that told an
operator to expect ~5.5 s, the research brief's consequences paragraph, and one
test comment.
All re-keyed to what was measured on the deployed container 2026-10-18: the
first synthesis of a freshly registered voice costs 1,883 ms against 1,226-1,283
ms afterwards, so the warm pass buys about 0.6 s — and the reason it exists at
all, which the pages never led with, is that the registry is memory-only: without
the hook every voice name is a 400 until somebody synthesizes. The trial's dated
worklog entry stays as written; the correction points at it.
No behavior change: comments, unit lore and docs only, so no cfhash move and no
image rebuild.
The first real boot gave the chance to measure what warming actually buys, and
the trial's numbers were wrong:
- Empty registry, register only, then time: the FIRST synthesis of a freshly
registered voice cost 1 883 ms; the next two 1 283 and 1 226 ms. A ~110-word
request cost 9.4 s and then 9.8 s.
- So the published 6.2–8.0 s cold / first-audio-byte 5 530 ms / later-requests
0.5 ms ladder does not reproduce, and request time tracks the audio generated
rather than snapshot state. The 0.5 ms figure is an internal state-restore
cost, never an end-to-end request time.
- Warming still earns its place for a different reason than the page claimed:
the registry is memory-only and re-registering does not warm, so a boot without
the hook leaves the façade answering 400 for every name. The saving is ~0.6 s.
Law 1 of [[Frank TTS Engine]] is re-titled the registry law and carries the
measured numbers; its example journal line is a real one from today; the
troubleshooting row for "first request of the day is slow" — the retired law's
symptom — is replaced by the argument-cap fault that actually happened. The same
ladder in the research brief is annotated rather than erased, and [[Voice Packs]]
step 6 stops claiming the hook registers only packs it has not seen (the engine
replaces a name it already holds).
The deploy of the newly built engine image came back `0 voice(s) warm` with
`/app/warm-voices.sh: line 152: /usr/bin/curl: Argument list too long` above
every line, and the unit was `active` with a green `/health` beside it. The hook
built `{"name":…,"wav_b64":…}` in a shell variable and passed it as ONE curl
argument; the kernel caps a single argument at MAX_ARG_STRLEN = 131,072 bytes,
and the shipped reference clips base64 to about 1.27 MB, so curl was never
executed and the hook reported the absence as "HTTP no response".
The body now goes to a mktemp file and curl reads `@file`; the audio enters
through a redirection instead of a command substitution, so the script never
holds a 1.3 MB string at all. Every pack in the fleet is ~1 MB, i.e. ~13x the
cliff, so this was always going to happen — it took the first real packs to show
it because no fixture was ever that big.
The regression test therefore ships a 300 KB pack against the seam's REAL stub
listener with the REAL curl: it fails with the same kernel error the host did
(`Argument list too long`, then the 400-for-every-name summary), and passes once
the body is a file. Proved red before the fix, green after.
The vault link check reads [[Frank TTS Engine]] literally, so wrapping it made
the target "Frank\nTTS Engine" and the unresolved-link seam failed. Rewrapped,
not reworded.
Three faults, all of the reporting-success-while-doing-nothing species: the
build that could not start (no adduser in the base), the warm pass that could
not warm (podman's --env-file keeps the quotes), and the image that would never
rebuild (cfhash over the Containerfile alone). Records the proofs, the two
behaviour seams, and what stays undone: nothing is deployed, #2 phase 1 stays
open until 7772 answers.
Three pages, all from faults found this session:
- `LLM/deployment.md` gains "Locally built images": which images have no
registry digest, what the `cfhash` skip is for, why it must cover every COPY'd
file (the silent shape: build skipped, /health green, old code serving), and
the base-image half of the law — the engine's `useradd`, because the pinned
ubuntu:24.04 has no `adduser`.
- `Dankclaw/Frank TTS Engine.md` troubleshooting gains the two lines an operator
actually needs: every voice cold on a fresh boot with a hand-run hook
succeeding (= `WARM_TEXT` arrived quoted, `podman exec … env | grep WARM_TEXT`
proves it), and a 502 from the façade beside a 200 `/health` from the engine
(model loaded ≠ voices registered). The stale-warmer row is re-keyed to the
new rebuild rule.
- `Dankclaw/Voice Packs.md` step 4 was wrong: it told the next agent to restart
the engine *instead of* the warm hook because the hook "only registers packs it
has not seen". It registers every pack on every run and the engine replaces a
name it already holds — `tools/tts-server.cpp:148` at the pinned commit:
"Re-registering a name replaces the previous entry." A restart is what an
engine bump needs (the registry is memory-only), never worse, only slower.
- `Dankclaw/Frank TTS.md` deploy command re-keyed to `./scripts/deploy.sh
dankclaw --service frank-tts`, the way it is actually deployed; the raw
`deploy-remote.sh` form is plumbing.
build_compose_image skips `podman build` when the installed image's `cfhash`
label matches a hash taken from the checkout now — the skip is what keeps
frank-tts-engine's ten-minute recursive source clone off every re-deploy. The
hash covered the Containerfile ALONE, so any change to a file the image COPYs
was invisible: edit dankclaw/voiceservice/app.py or tts-engine-warm.sh, re-run
the deploy, and podman reports the existing image is fine. The redeployed
container runs the previous code, /health answers 200, and the e2e gate has no
way to see it. It bit the boot warmer twice this week.
Each case arm now names its own extra=(…) — the paths its Containerfile COPYs —
and the label hashes the Containerfile plus those. The two arms are kept apart
deliberately: the façade bakes app.py and tts-start.sh, the engine bakes
tts-engine-start.sh and tts-engine-warm.sh, and comfyui/loki/alloy have no extra
files, so their hash is unchanged and they do not rebuild on this commit.
Hashing rather than word-splitting because these are repo-relative paths under
a fixed checkout root.
The seam drives the real function against a stub podman and asserts observable
output — did the deploy ask podman to build, or did it skip? — in five steps:
first deploy builds, unchanged tree does not, editing the engine's COPY'd warm
hook rebuilds the ENGINE image and not the façade, editing app.py rebuilds the
façade. Reverted scripts/deploy-remote.sh to prove it red on the old hash: the
warm-hook step is exactly where it fails.
`WARM_TEXT` reaches the container **with its quotes**. The unit reads the env
file through podman's `--env-file`, which takes the literal remainder of the
line — no shell parsing — while bash `source` strips one layer. The hook
interpolated the raw value into the synthesis body, so with the quotes intact
the JSON was invalid and every voice logged "registered, but the warm synthesis
failed" while a hand-run hook (bash shape) succeeded. Reproduced both ways
against a real `podman run --env-file`, not by reading the man page.
The hook now strips exactly one matched pair — only when both ends are the same
quote AND the inner content carries another quote or a backslash, so a value
that legitimately contains quotes is never mangled — and then JSON-escapes what
is left, which is what makes a sentence like `He said "hi\" once"` arrive as
itself instead of corrupting the body.
The seam gains a test that reads `scripts/frank-tts-engine.env` BOTH ways —
`source` and the way `podman --env-file` does — and asserts the engine receives
the same sentence either way, in both shapes. A test that only sources the file
cannot see this bug, which is how it survived the first deploy.
The runtime stage of Containerfile.engine ran `adduser --system …`. The pinned
ubuntu:24.04 base ships no adduser, so the build died at that RUN line with
crun's "executable file not found in $PATH".
Verified in the base itself on Dankclaw, 2026-10-18: `command -v adduser`
prints nothing, and the exact replacement line — `useradd --system --uid 10000
--home-dir /app --shell /usr/sbin/nologin tts` — answers `uid=10000(tts)
gid=999(tts)`. Same account as before: numeric uid/gid 10000, non-login shell,
no password entry. Its one warning ("uid 10000 is greater than SYS_UID_MAX
999") is cosmetic and is what keeps the uid equal to the façade's, so the
comment says so before someone "fixes" it by dropping --system.
Stage 2 of the voice stack. Repo state only: nothing deployed, no host touched,
no container started.
- `frank-tts` is now registered everywhere a dankclaw service has to be:
`SERVICE_ORDER` (after `frank-tts-engine`, before the consumers), the planner's
dankclaw set, `deploy-remote.sh`'s allowlist + dispatch + a
`deploy_frank_tts` handler, and `deploy.sh`'s dispatch regex — plus the
`build_compose_image` arm naming `dankclaw/voiceservice/Containerfile`, without
which the build silently returns 0 and the unit's `Pull=never` tag never
exists.
- The unit joins TWO networks, in order: `dankclaw_default` first (default route,
where consumers and open-notebook live), then the voice bridge. `TTS_UPSTREAM`
moves from `host.containers.internal:8880` to `http://frank-tts-engine:8880` —
the actual fix for a shape that could never work, since a bridge container
cannot reach a host-loopback bind and the engine publishes nothing at all.
- `Wants=`/`After=` on the engine, never `Requires=`: the `llama-tts-cli` rollback
backend needs no upstream, so a broken engine may delay this endpoint but must
not take its unit down.
- The handler installs the engine's quadlet pair too, because a referenced
`Network=` file the generator cannot find means no façade unit is produced at
all, and `--only frank-tts` has no other way to get it. It creates
`/home/dank/voices` as the deploy user (rootless podman would auto-create a
missing bind source root-owned, which reads as "no voices"), and gates on
podman's recorded health.
- Voice naming is now written down where it can be found: a voice is a lowercase
directory name matching `^[a-z0-9][a-z0-9._-]*$`, at most 64 chars. That shape
is the intersection of two rules that disagree — the shim would accept mixed
case, the engine lowercases names — so a mixed-case pack resolves on one side
only, and the boot warmer refuses it loudly. `narrator1/2/3` are load-bearing
for open-notebook.
- Homepage card for the pair, `wip: down` and hidden, probing `127.0.0.1:7772`;
the engine gets no card because it publishes nothing and could never be probed.
- **No Prometheus job, deliberately.** No `/metrics` exists on the shim, this
Prometheus 3 refuses HTML/JSON bodies (the existing `comfyui` and `services`
probes are documented down for exactly that), and `LlmEndpointDown` is scoped
`up{job=~"frank-.*"}` — so a `/health` job would page on Signal for a healthy
endpoint. Recorded in the page instead of quietly skipped.
- Vault: new [[Frank TTS]] page linked from the index; [[Frank TTS Engine]] now
points at it for the pair's public signal; log entry for the day.
- Seam grows to 25 tests: the generator's rendering of the pair (both
attachments, first-network order, one publish, Wants-not-Requires, bounded
per-unit blocks), the env file's upstream checked as a network JOIN rather than
a string, planner separation and order, the handler's steps, the card's gating,
routing that selects both seams for either deployable.
Repo state only: nothing deployed, no host touched, no container started.
- `frank-tts-engine` on Dankclaw: `qwentts.cpp` commit
5916bf7c79dbb4e66fc059cdc507a79a23a28bb4, resident `tts-server`, 1.7 B base
talker Q4_K_M + Q8_0 tokenizer, container port 8880. It is deliberately NOT a
fleet endpoint — `frank-tts` on 7772 is the endpoint consumers address.
- Publishes NOTHING. A host-loopback publish would have been useless to the
façade (netavark DNATs published ports only, so a bridge container cannot
reach a host loopback bind — the shape that 502'd hermes) and would have been
the one LAN-reachable surface of an unauthenticated synthesiser. Both voice
containers sit on one committed quadlet network, `frank-tts-engine.network`
pinned to 10.89.5.0/24 (next free /24 after open-notebook's 10.89.4.0/24), and
peers address the engine by container name. Diagnostics and the deployer's
health gate run inside the netns (`podman exec`, `wait_container_healthy`).
- Cold-voice law honoured in the unit: the first request for a registered voice
decodes its whole reference to snapshot codec state (6.2-8.0 s, first audio
byte 5 530 ms), later ones restore in 0.5 ms, and the registry is memory-only.
`ExecStartPost=-/usr/bin/podman exec frank-tts-engine /app/warm-voices.sh`
with `TimeoutStartSec=900`; the `-` prefix keeps a failed warm from
crash-looping the endpoint under `Restart=always`. It logs cold=/warm= per
voice, so the under-a-second second request is proven on every boot.
- Thread law recorded: `tts-server` has no `--threads`; src/backend.h uses
hardware_concurrency()/2, so a cpuset pin silently changes the thread count.
- Image builds from the pinned commit with the proven line (GGML_BLAS=OFF
forced by this host, every GPU backend off, -j 8, no -DGGML_NATIVE because
ggml defaults to -march=native for the building host). Two stages; the runtime
stage COPYs both the entrypoint and the warmer, because a container that
publishes nothing is reached with `podman exec`, not a port.
- `scripts/download-frank-tts-engine-qwentts.sh` fetches the two GGUFs with byte
and LFS-oid checks (talker 1 219 245 248 B, tokenizer 291 150 624 B); the
deployer gates on --verify-only instead of pulling 1.4 GiB mid-deploy.
- Registered in SERVICE_ORDER, DANKCLAW_SERVICES, the deploy-remote.sh dispatch,
its pre-deploy requirement, deploy.sh's regex, the planner's path map (the
whole `dankclaw/voiceservice/` tree mapped to nothing before this), and
test-changed.sh ABOVE the `dankclaw/*.container` case whose `*` crosses `/`.
- New seam tests/frank_tts_engine.bats (19): the Quadlet generator's own output
for network/publish/boot/warm shape, entrypoint argv driven from the tracked
env file, the warmer against a stub tts-server over a real socket
(registration keys, ref_text absent, late engine, hung synthesis, name
refusal), the fetcher's verifier, the pinned cmake line, and both routing
tables. Suite: 422 passed.
- Vault: [[Frank TTS Engine]] linked from the index; log.md entry for the day.
The page a new operator needs: what the three required files are and who
reads each one, how a pack is built (with the extractor's usage text
verbatim, because the flags are not guessable and there is no -o), the
10-second gate and where it fires twice, how to add a fourth voice
including the warm hook that registers a pack without restarting the
engine, and the engine-bump procedure for when the artifact sizes move.
Plus the host census as of 2026-10-18 — the trial packs carry no
artifacts and their manifest is the trial's own schema, so the builder's
first real run is a --force migration — and the three gaps left open on
purpose: no curated ICL transcript, no 16 kHz variant, no loudness pass.
Linked from the Dankclaw services list; log entry appended oldest-first.
scripts/voice-packs.env carries the decisions worth a diff and nothing
else: the pack list, each voice's source file, crop offset, crop length
and pinned source digest, the extractor path with its engine repo and
commit, and the artifact arithmetic. Paths and numbers only — the pack
directories themselves stay host state under /home/dank/voices.
tests/voice_pack_unit.bats pins behaviour, from inputs to observable
output: a crop inside the first ten seconds exits nonzero naming the
invariant and creates nothing, a legal one builds the whole pack, the
reference comes out 24 kHz mono inside 18-25 s (ffprobe, on a locally
generated tone), the manifest round-trips through jq with offsets and a
sha256 that matches the source bytes, a re-run without --force leaves
every byte untouched, --force retires rather than deletes, --dry-run
needs no extractor at all, and a stub extractor that changed its artifact
size cannot publish a voice. No engine binary, no GPU, no network, no
fleet host: qwen-codec is stood in by a stub with the real tool's I/O
shape, the way frank_tts_unit.bats drives its backends.
test-changed.sh routes both new paths to that seam instead of letting
them land on the generic scripts/* syntax fallback.
Turns a source recording plus a crop offset into the pack frank-tts
resolves by name: reference.wav (24 kHz mono pcm_s16le, 18-25 s), the
engine's own reference.spk / reference.rvq made from it, and manifest.json
provenance (source path + sha256, crop start/end, sample rate, duration,
tool, engine commit, voice name).
The extractor is qwen-codec from the pinned qwentts.cpp tree, and the
flags are its own usage text, not a guess: --model <codec gguf>, --talker
<base talker gguf>, -i <wav>, optional --device; there is no -o, so it
names both artifacts next to the input with the extension swapped, which
is why the pack's WAV is staged under its final stem.
The 10-second source-material invariant is enforced, not annotated: an
offset inside the forbidden region exits 1 before a directory is created,
and the gate is asserted a second time from the manifest that was just
written, together with the recorded sha256. Artifact sizes are checked
against the codec's own arithmetic (8,192 B x-vector; 22 B per 12.5 Hz
frame), so an engine bump fails the build instead of shipping a pack the
engine rejects at synthesis time — and the crop is refused up front when
it is too short to fill the .rvq window.
reference.wav is the file the façade gates on, so it lands last, after
the artifacts and the manifest: a voice name never exists on disk without
its artifacts behind it. Nothing is ever deleted — an existing pack is
refused unless --force, which retires it whole to .AGENTTRASH/ — and
--dry-run runs every step except the extractor, so the seam can test the
gate without the engine.
Frank heard every audition pair and passed it, so likeness stopped being the
tiebreak and the engine was picked on the live-output criterion: arm A
qwentts.cpp 5916bf7, resident tts-server, 1.7 B talker, llama-tts-cli kept as
the rollback. Records the discriminators (first audio byte, warm RTF, spread),
the three consequences that are configuration (boot-warm every voice, no cpuset
pin, WAV-only registration), the two follow-ups left open on purpose, and the
corrected addressing law: a host-loopback publish is unreachable from a bridge
container, so the pair shares one quadlet network.
Mirror of command-center clarification: first dogfooding round stamped the
build worktree instead of the session's own working directory. Rule text
now says which one, and to note the build worktree in the body.
Mirror of dank/command-center's rule (there: PR off docs/session-trailer-rule).
Every agent-opened issue/PR body ends with cwd + omp session id so machine-filed
artifacts trace back to the box and session that produced them.
Refs #11 (command-center tracker)
The env file first landed with ESPERANTO_TTS_TIMEOUT equal to SHIM_TIMEOUT=600.
Equality is a race: both budgets expire at the same instant, so the case the
number existed to prevent — the worker abandoning a synthesis frank-tts is still
finishing — is exactly what happens on the tail of a long line. Nested budgets
go inside-out, which is the rule scripts/frank-tts.env states for its own
LLAMA_TIMEOUT; the client waits one round-trip longer than the server, so a
frank-tts give-up arrives labelled as frank-tts's error. 660.
The seam's assertion moves from equality to `>` for the same reason, and now
also holds ESPERANTO_LLM_TIMEOUT under upstream's 600 s web-UI patience — above
it the failure surfaces as a UI timeout with no name on it. The scriptwriting
number itself stays at the app's 180 s default: frank-zealot answers an outline
or a transcript well inside it, which corrects the earlier commit message's
claim that both timeouts were raised past upstream's 300 s.
Page registered in the vault index (the docs-completeness seam fails an
unregistered page) and logged in the bundle change log.
What the page owes the reader: the provider contract (frank-zealot for scripts
with a /v1 base URL, frank-tts for voices with 1-4 speakers mapped to the
narrator1/2/3 voice packs typed as free text), why Test Connection and Discover
Models fail cosmetically against frank-tts rather than being a fault, the
encryption key as the precondition for saving any credential, and the honest
blocker — frank-tts is not deployed and nothing listens on 7772, so a generation
run writes its script and fails on the first voice.
Episode destination is stated as interim and cites Gitea issue #7 (human-only,
agents must not work it): the app keeps its named volume, one bind mount puts
finished MP3s under /home/dank/media/podcasts, and deleting that one line is the
whole rollback. Rollback section also covers the full undo, including that the
podman network only disappears once the units are gone.
Structure only: what the units declare and how the scripts route them, never how
the app behaves inside a container. 19 tests.
The ones carrying real weight: the app unit has no publishable 8000 anywhere and
the database unit publishes nothing at all; the API remap off host 5055 is only
correct if API_URL names the same host port, so the seam parses the PublishPort
and compares; each unit reads the repo tunable file before the host secret file,
because podman's last-match-wins makes the order load-bearing; both health
probes are argument-only (a shell string there is a unit that starts dead); the
database password appears only as an env reference, never in Exec=; and the
episode host bind appears exactly once, since a second mount would silently
shadow the first.
The deployer assertions read the functions themselves: the gate URL path is
/health on the remapped port and must not name 5055, and the seed function
contains the blank-secret refusal plus the no-clobber test. No test asserts a
made-up endpoint or a cloud URL, and nothing touches the network.
The case has to sit where it does: the dankclaw/*.container glob's `*` crosses
`/`, so placed below it, dankclaw/open-notebook/quadlet/open-notebook.container
matched that generic case and ran the open-webui/pihole/searxng seams instead of
its own. Same reason the env path is grouped here — scripts/*.env would otherwise
send it to the generic env seams.
Internal-only: port 8502 direct, no vhost field, no Caddy change — exposure
stays behind an explicit decision, and the app carries its own password.
urlStatus=hidden is deliberate: the homepage probe runs on the gateway and the
pair is not deployed, so a live card would render dead on day one. Flipping it
to live is the only homepage change the deploy needs; the seam asserts the
hidden state while the service is repo-only and requires the required card
fields either way.
mono-on.svg follows the mono-* icon set the sibling cards use, so the card does
not fall back to the generic dot.
Registration is load-bearing, not cosmetic: deploy-plan.py refuses any service
it does not know, so SERVICE_ORDER + DANKCLAW_SERVICES are what let `--only` and
`--auto` address the pair at all; the dispatch case in deploy-remote.sh runs it;
the requirement in check_prerequisites gates a manual deploy; and the dispatch
regex in deploy.sh's host-mode branch decides whether a changed path dispatches
a service or falls through to "no service mapped" — an omission there is silent,
which is exactly how the AGENTS.md deployer-registration law was earned.
The deploy seeds /home/dank/Containers/open-notebook/env (0600) from the host
env and refuses to start the pair while OPEN_NOTEBOOK_ENCRYPTION_KEY or
OPEN_NOTEBOOK_PASSWORD is blank: without the key no provider credential saves
and without the password the API — including the episode-audio route — is open
to the LAN with no auth. A seeded file is never rewritten, so re-deploying
cannot clobber a rotated key, which would silently make every stored credential
undecryptable. SURREAL_PASS is derived from SURREAL_PASSWORD in the same write.
The gate names paths that answer, not invented ones: the database is judged by
its own container health status (/surreal is-ready, verified against the pinned
digest) and the app by wait_for_health on /health at the published 5056 mapping —
the route is upstream's and is on the auth middleware's exclude list, so it
stays 200 after the password is set. 180 s covers Next.js build + startup and a
degraded-but-listening first boot, matching the sibling LLM deploys.
Both digest pins are checked against local storage before restart, because
Pull=never means a missing image would otherwise surface as a unit stuck in
"Start request queued" (the 203/EXEC trap's sibling). The interim episode
directory is created before the app starts: a missing bind source makes podman
create it as root-owned and the app's mkdir would then fail.
The two knobs that keep the box alive: TTS_BATCH_SIZE=1 and
OPEN_NOTEBOOK_WORKER_MAX_TASKS=1. Upstream ships 5 x 5; frank-tts spawns one
~8 GiB, 8-thread engine process per synthesis request on a machine that also
serves two LLMs, so a five-way fan-out is a self-inflicted shutdown. The worker
knob is real, not decorative: the image's supervisord expands
${OPEN_NOTEBOOK_WORKER_MAX_TASKS:-5} into the worker command
(supervisord.conf:25).
Both ESPERANTO_* timeouts go above upstream's 300 s defaults, sized for CPU
synthesis (SHIM_TIMEOUT=600, LLAMA_TTS_THREADS=8) — one episode is dozens of
per-line /audio/speech calls and a timeout kills the episode, not the line.
API_URL is forced for the 5055 -> 5056 remap; SURREAL_URL addresses the database
by container name; CORS_ORIGINS is origin-form (scheme://host:port), the form
the app compares against. OPEN_NOTEBOOK_PASSWORD stays absent here on purpose:
the app treats a set-but-empty value as auth-on.
Secrets stay placeholders in the tracked file and in dankclaw/.env.example, which
lists SURREAL_PASSWORD rather than SURREAL_PASS — the deployer derives the
surreal CLI's spelling from it when it seeds, so the app and the database cannot
be handed different values.
Two containers, because that is what upstream runs (docker-compose.yml:6,13):
the app image under supervisord, plus SurrealDB. The `v1-latest-single` tag is
not in play — it bundles its own database, is deprecated upstream, and would
give the host a second hidden DB.
Both images are digest-pinned with Pull=never and a documented one-time pull,
the frank-dragoon posture: a Quadlet on :latest re-resolves on the first restart
after an unrelated daemon-reload, and the failure is silent.
Port map paid for by what is already on the host: seerr holds 127.0.0.1:5055
(since 2026-10-14, live `ss -ltn` agrees), so the API leaves on 5056 and the env
file has to set API_URL to match, because the frontend derives the API URL from
the window origin when it is unset. 8502 publishes on 0.0.0.0 — the service
carries its own password and the episode-audio route is unauthenticated. 8000 is
published nowhere, where upstream exposes it on loopback with root credentials.
The app joins dankclaw_default first and the private systemd-open-notebook
second: it reaches the published fleet endpoints through the former
(host.containers.internal, the arrangement Prometheus is proven against) and
finds its database by container name through the latter. Generated --network
flags keep file order, so the tunable file is listed before the host secret file
in both units — podman applies last-match-wins.
No UserMapping: the app runs as root in its namespace (no USER in the Dockerfile)
and writes only paths owned by that root. Exec= carries no database credential —
`exec start --user --pass` puts it in world-readable argv; the image's own
entrypoint passes it on stdin, so only the container env holds it.
Both candidate :8880 engines were measured on Dankclaw's CPU (no GPU,
no container, nothing deployed). arm A qwentts.cpp 5916bf7 resident at
RTF 0.55-0.57 warm / 0.67 ICL / 1.65 cold; arm B mainline llama-tts at
0.93-0.99 with a +-0.25 spread. Records the four configuration laws the
trial produced (boot-warm every voice, .spk+.rvq replay is byte-identical,
streaming != buffered, /v1/audio/voices rejects a lone spk_b64), the
likeness measurement's resolution limit, and re-keys the stale
8890/voiceshim shape on that page to frank-tts on 7772. The engine pick
stays open: the likeness metric cannot rank clones, so the audition clips
decide.
The section mandated a source-test-push loop but never stated a branch
regime - agents pushed straight to main. Fleet alignment (operator
decision 2026-10-18, mirrors dank/command-center): main is the only
long-lived branch; branch fresh off origin/main per work item; PRs
target main with --no-ff; merge only on Frank's explicit OK; tags on
main. Their tests/run.sh suite: ALL TESTS PASSED before push.
The dragoon-window provenance bullet still said the zealot arm owed a
measurement — it does not: the numbers are recovered and in the fetch script and
runbook. Adds the frank-tts phase-1 entry (naming law, port 7772, voice-pack law,
384/384 suite, PR #4 merged as 49c4e19, and the measured Gitea merge-field law).
Phase 1 of the voice-over goal (Gitea issue #2). One stdlib-only HTTP facade in
front of whichever TTS engine wins the engine decision: POST /v1/audio/speech
with voice=<name> answers audio in THAT voice, where a name is real exactly when
/voices/<name>/reference.wav exists — an unknown name is a 400 listing the
voices that are, never a silent fallback (a wrong-voice endpoint stays wrong for
weeks). voice_id works as the alias the omp harness sends; response_format=mp3
is really mp3 (ffmpeg re-encode); oversized input is refused with 414 instead of
half-narrated; /health reports the backend and the resolved voice list.
Two backends behind the one surface, chosen by TTS_BACKEND: omni-http forwards to
an OpenAI-shaped upstream carrying the voice as base64 reference_audio, and
llama-tts-cli runs a local binary per request with --tts-speaker-file at nice 19
so narration never starves the text endpoints. Both are driven in the seam
through stubs — no GPU, no containers.
Named frank-tts, a first-class fleet model endpoint: port 7772 in the 7770/7771
family, alias naming per the StarCraft law, rootless boot-persistent self-healing
quadlet with a read-only /home/dank/voices mount. Voice-pack law recorded in the
env header: reference.wav PLUS the engine's own speaker artifact (required, so a
voice is reusable in other runtimes), plus manifest.json provenance with offsets
always past the first 10 s of the source file.
Not deployed: the engine decision, the pack builder, deployer/homepage/Prometheus
registration and the omp models.yml runner row are phase 2.
Frank's 02:36 instruction (archive 01a14c93) ordered the 27B PTQ1_0 arm at
2 x 200,704 with the Q8 mmproj and batch/ubatch 3072/1536; it served
production 03:14:39-03:38:57 (pid 191358, journal type ptq1_0: 402 tensors).
Recovered its numbers from that process's own print_timing lines: ~48 t/s
single-stream decode, 25-27 t/s per stream at two streams, prefill 1,295 t/s
at 30,637 -> 943 at 97,856 -> 801 at 141,365. It bought the promised 1,201 MiB
and lost 37% of decode against the PQ2_0 record, so the publisher's Ampere
warning generalizes to GA102; Frank scratched the 27B plan at 03:33 for the 9B.
Every 'unfinished experiment' claim is re-keyed, and the AGENTS.md law gains
the durable lesson: a bench verdict that never reaches the repo is
indistinguishable from a rogue arm.
Post-deploy probe on frank-dragoon: one 101,987-token prompt through
/v1/chat/completions took 81.5 s cold (1,251 t/s of prefill) and 0.3 s on two
repeats — HALOGEN_CACHE_INPLACE=1 keeps the KV rows in the pool, so doubling the
pool evicted nothing the prompt cache needed. The RAM price is still real:
MemAvailable 26 -> 22 GiB.
Census of the same engine's surface corrected the Benchmarking Policy, which
quoted llama.cpp instruments at a halogen endpoint: frank-dragoon answers
/health, /metrics, /v1/models and POST /v1/chat/completions and 404s /ping,
/props, /completion and /tokenize, and no response carries Server-Timing. The
/completion `timings` object and the ttft;dur= header are frank-zealot's; on
frank-dragoon the instrument is wall clock plus usage.prompt_tokens.
Deployed 2026-10-18 (e2e 33/33). The boot ledger prints
`reserving the KV pool, 401408 positions (about 11.0 GiB)` and
`62.9 GiB of weights locked … 11.0 GiB of KV pool, 5.7 GiB of working memory,
79.6 GiB in all`, so the ~10.8 GiB that every page quoted from the
~28 KiB/position arithmetic is replaced by the reservation the engine actually
makes (11.0) and its preflight bill (11.3). Host MemAvailable fell 26 -> 22 GiB
across the raise: the FLAGS.md collision (pool RAM is n-gram page cache) paid in
full, so the warm prompt-cache ratio is flagged as the number to watch.
All "committed but not yet deployed" clauses retired: AGENTS.md topology row,
KV-pool law and memory ledger, frank-dragoon.env lore, both index pages, the
iommu page, and the runbook chronology (third production boot recorded).
The sweep also caught the LLM index still describing frank-zealot as
Bonsai-2-27B at 2 x 200,704 in its intro and topology diagram - re-keyed to the
live 9B: 8 full-attention + 24 Gated DeltaNet + blk.32 MTP, 3 x 262,144,
tensor type 141.
The cutover deploy's FIRST boot was not the shipped payload: a stale
frank-zealot.service.d/ptq1.conf from the bench window still pointed
FRANK_ZEALOT_ENV_FILE at /tmp/zealot-arm-current.env, so the unit booted the
unmeasured 27B PTQ1_0 arm (3 slots of 133,888, 21,188 MiB) while /health
answered 200, /v1/models echoed the alias and the e2e gate passed 33/33.
Drop-in retired to ~/.AGENTTRASH/20261018-zealot-ptq1-arm/; the 13:19 restart
is the shape the docs now quote (n_ctx_slot 262144, KV 13,056.00 MiB,
settled 17,922/6,203).
Corrections the production ledger forced:
- RS buffer is 150.75 MiB (3 cells) on the shipped spec-off shape; 452.25 MiB
belongs to the SPEC-ARMED arms, where MTP depth 2 bills three seqs per slot.
The drafter triples the recurrent state on top of its 3,072 MiB of f16 KV.
- Decode measured 168.4-168.7 t/s single-stream and ~300 t/s aggregate over
three streams (102.8 each). Wall-clock timing of a generation that stops on
its own reads 4x low (72 t/s for a 24-token countdown) - measure with
ignore_eos and read the engine's timings.
- OpenWebUI stores both decoration values inside JSON columns (params/meta),
and frank-dragoon's context.length had drifted to 262,144 against its own
200,704 ceiling; both re-keyed in the live DB.
New law in AGENTS.md: after deploying a recently-benched LLM service, check
DropInPaths, then /proc/<pid>/cmdline, then the ledger. Health checks cannot
see the payload.
The docs-fact census wave re-read the live hosts and re-keyed every page
it could: runtime/payload identities, ports, sizes, ledger figures, unit
state, and the retirement banners that drifted as the fleet moved. The
wave was interrupted mid-run, so this commit carries only what passed the
integrity gates: every touched page parses (front matter or H1+status
convention intact, balanced fences, no truncated tables, trailing
newline), the docs_completeness and link-integrity seams are green, and
bats is 384/384.
Numbers that this session proved on live hosts are keyed to the measured
value, not the page's earlier claim.
Swap the 3090 endpoint from Ternary-Bonsai-2-27B PQ2_0 (2 x 200,704,
fork build 10754) to Ternary-Bonsai-2-9B PQ2_0 with the MTP head IN the
file, three conversations at the trained 262,144 window
(--ctx-size 786432 --parallel 3), on vendor fork build 10777 -- the
minimum build that pairs MTP with a quantized V cache. q8_0 K + q8_0 V
is the only KV that fits: 13,056 MiB against a 26,112 MiB f16 pair.
Settled boot 17,876 used / 6,249 free.
MTP is armed and then DECLINED by measurement: the draft context bills
its own f16 KV over the whole pool (3,072 MiB) and at UBATCH_SIZE=2048
that halves decode (167.2 -> 109.2 t/s). The documented MTP arm
(SPEC_TYPE=draft-mtp + UBATCH_SIZE=1024, 213.5 t/s) is recorded in the
env header rather than left to rediscovery.
Two bugs the trial caught: ${SPEC_PRESENCE_PENALTY:=0.0} was a
default-assignment, so the env value always won and the no-presence-penalty
MTP arm was unreachable (now ${VAR=0.0}); and the Q8_0 mmproj is the
publisher's requirement for concurrent image requests, which matters at
--parallel 3.
contextWindow is now 262144 -- the widest in the fleet -- so :7771 is the
compaction hub. 27B runbook re-banned RETIRED with its bench-window
section closed; indexes, homepage card, OpenWebUI doc and the worklog
re-keyed.
frank-dragoon's total in-flight context was HALF frank-zealot's: zealot runs
one 401,408-position pool split into two 200,704 conversations, while dragoon
ran ONE 200,704-position pool behind its two slots, so a single maximum-length
request filled it and two long conversations had to stay under ~100k each.
HALOGEN_KV_POOL_POSITIONS 200704 -> 401408 = slots x ctx, the top of the seam's
legal band (ctx <= pool <= slots * ctx). The PER-REQUEST ceiling does not move:
HALOGEN_CTX stays 200704, because the checkpoint's native window is 262,144 and
the engine's own 400 text refuses to exceed it without HALOGEN_ROPE_YARN. What
this buys is concurrency, not a longer conversation, which is why the harness
contextWindow (155648 = ceiling - 45056) is untouched.
Cost: ~10.8 GiB of KV where 200,704 cost 5.5 (262,144 cost 7.2; the
2026-10-06 shape's 524,288 measured 14.6), taken from the page cache the 51.2 GB
n-gram table reads through — the warm prompt-cache ratio is the number to
re-measure after the boot. Engine-side total goes 74.1 -> ~79.4 GiB of 121.2.
Also carries scripts/download-frank-zealot-bonsai2-ptq1.sh: the PTQ1_0 candidate
payload fetcher (5,946,648,928 B, LFS sha256-verified, mmproj hardlinked), with
the publisher's own throughput table in its header — the packing is smaller but
the card measures it SLOWER on Ampere, so the swap is decided on our card, not
on the card's A100 row.
The scrape has never worked: lastError is "server returned HTTP status 501 Not
Implemented" on http://host.containers.internal:7771/metrics, and the oldest
retained sample of up{job="frank-zealot"} (2026-10-11) is already 0 -- the target
has been red across four payload generations while /health answered 200 the whole
time, keeping the critical LlmEndpointDown alert permanently active. llama.cpp
gates /metrics behind --metrics; halogen serves it natively, which is why the
asymmetry looked like a 3090-only problem.
METRICS=1 in the shipped env, --metrics reaches the argv through EXTRA_ARGS, and
the seam asserts both directions (default ships the flag, METRICS=0 strips it).
Diagnosed via /api/v1/targets lastError rather than the alert list, which is also
the lesson: Alertmanager here has no persistence, so a reboot re-stamps startsAt
and a weeks-old fault looks fresh.
Window 01:24-01:35. Byte-identical A/B (rebuild proved it: same 1,287,656 B corpus,
same billed token counts 59,024 / 21,568): cold prefill 1,585.1 -> 1,616.0 t/s
(+2.0 %), warm 208,531 -> 214,131, needle hit/hit both boots. The 13-16 % that
justified amd_iommu=off since 2026-09-11 does not exist on this host; the decode
delta (-9.7 %) is instrument noise -- three repeats of one request on the same boot
gave 31.8/46.9/47.8 t/s.
NPU precondition restored: amdxdna bound 0000:c6:00.1 (driver 0.7.0, firmware
amdnpu/17f0_11/npu_7.sbin), group 26 identity, PASID width 10, /dev/accel/accel0
root:render with dank in render. No endpoint deployed, so frank-npu stays retired
-- only its rationale changes. Gates: 0 failed units, 32 groups (31 identity + the
SD reader's DMA quirk), iGPU c5:00.0 identity with ATS, dragoon ledger unchanged
(74.1 GiB, no clamp, listening at 39.8 s).
Deferred on purpose: the pp_dpm_fclk pin (changes iGPU timing for everyone, nothing
co-runs yet) and PerfOpt (7.0.0-38 has zero perfopt symbols, 26.04 caps at 7.0.0-38.38).
Re-keyed: decision record banner/front matter/verification/revert + new execution
section, frank-npu rationale, AGENTS key lesson, both index pages.
/tmp is tmpfs: the 2026-10-18 IOMMU reboot deleted /tmp/halogen-eval mid-A/B and
the off-arm numbers nearly went unreproducible. One command restores the corpus
(wikitext-2 raw test, blanks dropped, 2891 lines / 1287656 B -- the byte count is
the identity check because battery.py sizes haystacks in characters), the 512px
probe PNG, and battery.py itself; WARM=1 also pre-reads the 51.2 GB n-gram table
so a cold page cache cannot masquerade as a measured regression. Must run through
a login shell: uv lives in ~/.local/bin.
The pending-entry grep anchored on "menuentry 'Ubuntu'" with -A4 and matched no
linux line, so the script printed nothing where it should have shown the next
boot's command line; the PENDING line then said "is in grub.cfg" when the loop
only ever compared the tracked file against /proc/cmdline. Now the default
entry's linux line is parsed with awk (first linux line of the first Ubuntu
menuentry, which is what GRUB_DEFAULT=0 selects), its iommu token is printed,
and a mismatch against the tracked file warns by name.
The 2026-09-11 trade bought a vendor-claimed 13-16% prefill figure that was
never measured here; the 2026-10-18 research (same decision record) shows what
identity mode actually costs the iGPU is the 1:1 page walk plus ATS/PRI/PASID/
GCR3 -- worth 2-4% on this silicon class, and removable outright by AMD PerfOpt
when the distro ships Linux 7.4. Meanwhile pt is what err=-19 asks for: the
XDNA2 NPU binds its own IOPT_TYPE_SVA domain and only needs the IOMMU enabled.
Measured off-vs-pt prefill arms follow in the docs commit; frank-archon
(Dankserver) is untouched.
/etc/default/grub was host-only state while the IOMMU boot line was the most
consequential kernel decision on the box (the 2026-09-11 amd_iommu=off adoption
lived only in the decision record). Fetched byte-exact (1581 B) and paired with
scripts/set-dankclaw-bootline.sh: refuses any host but dankclaw, shows the diff,
installs 0644 root:root, runs update-grub, names the pending token, and reboots
only on --reboot. The deployer plans nothing under dankclaw/host/ by design —
kernel state is not a container restart.
Reading patch v2 1/2 verbatim: amd_iommu_enable_perfopt() rehoming needs the
device ALREADY on the identity domain and switches ATS/PRI/PASID/GCR3 off via
the skip_caps path. So an iGPU in identity mode still pays a 1:1 page walk plus
those four capabilities until PerfOpt exists -- iommu=pt recovers most of the
gap, PerfOpt the last part. The first pass claimed pt alone already avoids the
cost; that was wrong and is retracted here and in the posted issue comment
(edit 23).
Also re-keys frank-npu.md, which carried the dead law 'iommu=pt is not a middle
ground' as its retirement rationale: identity mode + SVA is exactly the revival
path (amdxdna binds its own IOPT_TYPE_SVA domain), and the cost is single-digit
percent on this silicon class. Procedure step 2 now demands a local off-vs-pt
measurement instead of inheriting the vendor's 13-16%, whose slow-arm boot line
our record never identified.
Records the 2026-10-18 answer to 'does a recent kernel change retire
amd_iommu=off': PerfOpt (iommu tree 2026-09-24, Linux 7.4) stops GPA->SPA
translation for the privileged iGPU while the IOMMU stays enabled for every
other device, the NPU included. Local census proves the fix is absent here
(zero perfopt symbols in System.map-7.0.0-38, archive caps at 7.0.0-38.38),
and the September 13-16% figure was translated-domain cost, not strict-flush
cost -- lazy DMA is the Ubuntu default, so iommu=pt already avoids it. The
corrected procedure therefore flips to iommu=pt now and treats PerfOpt as a
2-4% later bonus on this silicon class (Phoronix desktop arm), with the NPU
legal under identity mode because amdxdna binds its own SVA domain.
Re-keys the AGENTS.md key-lesson bullet; no host change.
The full-suite re-run after the reset fix came back "1 flaky": 12-crawl reached
plex.franksdank.cloud, the guard cleared, and Plex Web — signed out in the
qa-e2e profile — handed the browser to its own hosted sign-in surface, so the
hostname never settled on the vhost (60 s poll, Expected plex.franksdank.cloud /
Received app.plex.tv). settleThroughGuard now takes a HANDOFFS landing list per
vhost; only plex.franksdank.cloud gets one, every other host still has to land on
its own vhost, and the guard assertions (Anubis cleared, no authentik uidField,
status < 500) are unchanged. Worklog and E2E gotcha 6 carry both symptoms: the
ERR_ABORTED reset and the hand-off.
Two faces of the same verification round.
12-crawl returned to the homepage between links with a bare page.goto, and Plex
Web (signed out, so it renders its own sign-in iframe for its Plex-account flow)
keeps redirecting the top frame: Chromium cancelled our goto with
net::ERR_ABORTED, outside the try, so the whole crawl died — once as a 420 s
timeout in the deploy gate, once at 5.4 s on the same link. backHome() tolerates
that one error class and re-issues; any other error still propagates, and the
per-app verdict (guard settled, hostname, no authentik uidField, status < 500)
is unchanged. Plex was healthy throughout (container up 2 days, 32400 answering
HTTPS 401 / plain-HTTP empty reply), so this was app behaviour, not a fault —
recorded as gotcha 6 on the E2E page.
The 200,704 boot printed 5.5 GiB reserved and 74.1 GiB in all (not the ~5.4/~74
my arithmetic predicted), so every prose figure is re-keyed to the ledger, and
the runbook gains the second production-boot section: marker revision, pool
reservation, shape line, prompt-cache entry size, and the refusal contract at the
new ceiling (400 in 0.86 s naming 200704 and the 96,405-token overage, before any
prefill). Cost stated (two maximum-length conversations never coexist; the 180k
compaction target is unreachable inside a 155,648 window) and purchase stated
(1.7 GiB of page cache, 9.1 GiB against the 2026-10-06 shape).
Frank's call the same evening as the native-payload cutover: HALOGEN_CTX and
HALOGEN_KV_POOL_POSITIONS both 262144 -> 200704 (a multiple of 4096, the rung
frank-zealot already runs). The checkpoint's native window is 262,144 and the
engine serves anything below it, so this is a RAM decision, not a capability
limit: the KV reservation drops 7.2 GiB -> ~5.4 (the 2026-10-06 pool measured
14.6), and by the FLAGS.md law -- RAM the pool takes is RAM that n-gram table
loses -- the freed positions become page cache for the 47.68 GiB table that
long prompts read. Engine-side total ~74 GiB instead of 75.8.
The 400-zone law is invariant to the value, so the harness moves WITH the
ceiling: models.yml contextWindow 217088 -> 155648 = ceiling - 45056, keeping
the gap that absorbs the chars/4 under-count (worst-case request ~156.2k vs
the 200,704 ceiling, ~44k margin). Compaction now fires at 85% = 132,300 est
(~136k measured); the 180k target of the 262,144 era is unreachable because
the window itself is 155,648. maxTokens 45056 and the coupled
cap/timeout 65536/7200 are untouched.
Cost stated plainly: with pool == ctx, ONE maximum-length request fills the
pool, so two long conversations never coexist (two of <=100k do). Acceptable
because frank-dragoon carries only the slow role and the security-reviewer
subagent, not the interactive workhorse.
Seam re-keyed: the fit-law test now proves the invariant at the new pair, and
the pool law (ctx <= pool <= slots * ctx) still holds with pool == ctx.
Deploy verification for fcd7630 (22:20): healthy in ~12 s, e2e 33/33, and the
boot ledger settles the shape — 62.9 GiB weights locked (0.8 vision) + 7.2 GiB
KV in ONE 262,144-position pool (6,656 MiB, two slots) + 5.7 GiB working =
75.8 GiB in all, n-gram 47.68 GiB read from disk, prompt cache ON (12 entries),
MTP depth-2, no clamp line. Live probes: 27*59 -> 1593, tool call, red PNG ->
Red. The two KV figures are reconciled (engine preflight FORMULA 7.4 GiB
includes block scratch + per-slot O(1) state; the RESERVATION is 7.2).
Also corrects two stale claims: the retired swift15 rows named 200,192 ctx
(raised to 262,144 on 2026-10-07 inside that same era), and the OpenWebUI
frank-dragoon row's description is now re-keyed in the live DB (backup
webui.db.bak-20261017b).
New LLM/frank-dragoon-flash-server-v2-native.md carries the shape with the
three deliberate deviations from the card (served id, SPEC_ADAPT=0, engine
pin 0.16.2), the memory ledger with the PIN_TRUNK rejection, the full
measured A/B table, and the companion law (hardlinked stock peonist files,
engine-tool certified). The swift15 page is re-banned RETIRED 2026-10-06 →
10-17 with its rollback terms; both indexes and AGENTS.md (topology table,
service section, runbook index) are re-keyed, and the root index also stops
describing the halogen-27B page as ACTIVE.
The three artifacts that produced the native-vs-swift15 verdict, so the next
payload swap starts from instruments instead of ad-hoc scripts:
census-frank-dragoon.sh — the engine's own checkpoint tools (verify/ppl/niah)
over BOTH payloads under one knob set; the pool stays 524288 deliberately
because the ppl pass scores 297,053 ids as two 262,144-token sequences.
battery-frank-dragoon.py — behavior battery against the live service: refusal
census, thinking-token census, tool-call turns, the red-PNG vision probe,
needle-in-haystack, decode/prefill/prompt-cache rates; discards the
post-boot warmup request and records the engine's 400 body (which names the
exact token overage) instead of dying on it.
trial-frank-dragoon.sh — the guarded candidate window: fetch gate, verify,
stop+mask, census, boot through a transient FRANK_DRAGOON_ENV_FILE drop-in,
battery, trap-guarded restore.
Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated is now the NATIVE
2026-10-11 v2 conversion of orcarouter's pinned BF16 (66,687,678,432 B,
LFS sha256 b5bd925f...) — 32 bytes smaller than the writer transplant it
used to ship under the same filename. The n-gram table, vision tower and
tokenizer are the stock peonist companions, hardlinked out of the swift15
payload dir: the release publishes only the main file and no source-matched
pair exists, so the engine's own tools are the referee (niah 18/18, ppl
3.968 with mean KL 0.155 against the swift15 logits, 2026-10-17).
Memory saving, per the card: HALOGEN_KV_POOL_POSITIONS 524288 -> 262144,
one pool behind both slots (boot ledger: 7.2 GiB, where the old pool
measured 14.6). HALOGEN_CTX stays 262144 so the per-request ceiling and the
harness contract (contextWindow 217088) do not move; only two MAXIMUM-length
conversations lose the ability to coexist.
Measured against the swift15 boot on identical traffic: cold prefill
+12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18, PPL +2.3 %, thinking
tokens x1.9 (the finetune's reasoning compression is gone), refusals 0/5 on
both.
Quat3rnion/halogen-qwen3.8-flash-next-v2-abliterated @main replaced the file
on 2026-10-11: a complete HIP native-v2 conversion of orcarouter's pinned
BF16 (66,687,678,432 B, sha256 b5bd925f), not the r1/r2 writer transplant
the filename used to carry. The release publishes ONLY the main file, and its
card states that the stock peonist n-gram/vision companions are NOT established
as interchangeable (source-derived table would be 51,200,246,084 B against
peonist's 51,200,246,144 B), so the script hardlinks the verified peonist trio
into the new payload dir — a hardlink is a regular file, so the serve gate's
no-symlink law holds and 47.7 GiB is not copied — and leaves the companion
question to the engine's own verify/ppl/niah tools plus the boot ledger.
Left uncommitted by the deploy session: the production decode rate (75.8 t/s,
within 1% of the trial arm), the shipped boot ledger, the live probe census,
and the picker-row law (OpenWebUI meta.description is live DB state a swap
must follow into the database, written without restarting open-webui).
The Swift1.5 w4dn8 .rad, the GLM-4.7 GGUF, and every derived model cache
were purged from dankserver (trash-staged, emptied with approval; 763 GiB
free). Rollback of the iq4r swap is now an HF re-download of the
byte-exact frank8465 mirror (LFS sha256 09dd9306...). Corrects the w4dn8
runbook header (still said LIVE) and rebuilds both Rollback sections;
log.md carries the purge record.
- production boot reproduced the census (22,664/1,461, fit 21,439 leaving
2,119 >= 1,872, KV 13,328.00 MiB at n_ctx_seq 200,704, n_slots 2), live
probes green (pong + reasoning_content, tool_calls, 512px -> Red, 75.8 t/s),
e2e 33/33
- records the two live-path stragglers the swap had to catch: the serve
script's retired-payload DEFAULTS and the OpenWebUI model row description
(live DB state, re-keyed with a backup, no forced restart)
- serve script: header + every fallback default re-keyed to the Bonsai-2 trio
(vendor LLAMA_BIN, PQ2_0 MODEL_FILE, Q8_0 MMPROJ) and CACHE_TYPE_*:=q8_0 —
a defaulted run must land on the shipped shape, never on the retired 9B/f16
pair (the trial window proved a silent default is how a swap goes wrong)
- unit Description now names the payload + KV dtype that `systemctl status`
will show during the next incident
- OpenWebUI decoration-row description re-keyed, with the law that the row is
live DB state a swap must follow into the database
- card/docs: prism-ml payload link, PQ2_0 spec, ~77 t/s (measured single
stream, replacing the 9B's ~95)
- warning re-keyed: zealot is no longer an uncensored endpoint, so the card
carries the base-alignment notice and points uncensored work at dragoon;
the test's "common uncensored text" equality (zealot==dragoon) is retired
with the payload it described — dragoon alone carries the legal warning
- env: Bonsai2 payload pair, CACHE_TYPE_K/V=q8_0 (f16 OOMs at 2x200,704:
34 KiB vs 64 KiB/token over the 16 full-attention layers), measured boot
ledger 22,664 used / 1,461 free incl. the 873 MiB mmproj that lands AFTER
the engine fit check, q4_0 fallback rung, CRACK one-line MODEL_FILE swap
- deployer: runtime/model/mmproj gates re-keyed to the PrismML vendor build
(the only loader for GGML tensor type 142 + prism.hadamard.* metadata)
- seam: payload trio pinned; KV argv assertions now derived from CACHE_TYPE_*
in the env file — the wrong-var-name f16 fallback that produced the first
failed trial window can no longer pass silently
kv_attn 26.00 KiB fp8 pages at 607,740 tokens (identical to the w4dn8
boot), slab 17,610/17,658 slots per rank (above w4dn8's 17,548 — ten
bf16-protected experts per layer pack more of the 4-bit table), sampler
9 rows, MTP 2/step, KV lending 4.22 GiB, prefix tier recovered 900,176
blocks across the payload swap. Vision probe green (reasoning read the
solid-red PNG; empty content at 64 tokens = the thinking-budget law).
e2e gate RED on three dankclaw specs: Marreta was down since the 01:41
dankclaw reboot (marreta-browser dependency-failed at boot beside
gluetun, not re-armed); pair restarted manually, redeploy rerunning
the gate.
New runbook frank-archon-radiance-qwen38-iq4r.md (publisher container,
payload chain, shape-verbatim cutover record with the boot-ledger and
measurement slots left open for the first boot, rollback). LLM index,
root vault index, and AGENTS.md (fleet table row, section, runbook
index: iq4r ACTIVE, w4dn8 RETIRED/on-disk, fp8-native deleted). Full
suite green.
Model swap on the radiance endpoint: MODEL_FILE moves from the in-house
qwen4exp-swift15-w4dn8-i8-hc8m.rad (116,416 MiB) to the engine author's
published qwen3.8-next-flash-fp8-iq4r-moe.rad (116,322 MiB, official
Qwen3.8-Flash-Next bf16 -> 4-bit non-uniform codebook experts + int8
trunk, card KL 0.0789 / top-1 92.85 % vs bf16). NO argument changed:
25600/5120 vram pair, 44928 host pool, fp8 KV, 3 x 200000, MTP depth 2,
wht6, MNBT 2048, froggeric v22.5 — the 2026-10-13 shape carries verbatim.
New fetch script scripts/download-frank-archon-iq4r.sh (hfget + exact
byte count 121,969,901,568 + LFS sha256, --verify-only default, ungated,
no token on the command line). Unit/serve/deployer lore re-keyed; the
seam pins the new payload id + sha256 and keeps the w4dn8 rollback
provenance. Homepage card points at the publisher repo.
Rollback: revert this commit (w4dn8 .rad stays on disk). The fp8-native
rad was deleted from dankserver 2026-10-15 (Frank's one-time rm, volume
for this payload); rebuild path documented.
Measured, 512 tokens, greedy, cold, at the deployed 3 x 200,000 fp8 shape:
prefill 22,208 -> 3,397, 24,000 -> 3,290, 25,600 -> 3,290 t/s at 43.5k —
the third slab step bought nothing, so the slab is a threshold whose cliff is
the 2026-10-11 arm at 26,071 (4,379), not a slope. Decode ladder: 99.0 t/s
single-stream at 43.5k, 93.3 at 197k (nearly window-free), and 3 concurrent
streams 113.9/43.3/10.9 = 18.4 t/s aggregate — the slot count is paid in
decode concurrency, the axis the shape hunt never measured because probe leg
B generates 24 tokens. probe-frank-archon-kv.py gains leg D (N-stream decode,
512 tokens, per-stream + aggregate) so no resize is priced on prefill alone.
Measured A/B on the shipped pair 25600/5120: A-cold 43.5k prefill 3,290 ->
3,584 t/s (+8.9 %) and the 3-way 197k aggregate 3,051 -> 3,589 t/s (+17.6 %,
wall 193.7 -> 164.5 s). The prediction I shipped with the arm -- slab is a
threshold below ~26 GiB, so 1,600 MiB buys ~0 -- is falsified by the measurement
it asked for. Two slab points inside noise of EACH OTHER (22,208 / 24,000) are
not a plateau. The cliff is above 25,600: 26,071 measured 4,379 t/s.
Ceiling re-derived from the engine's own budget table (card 32,624 MiB, pair at
29,120 settling 2,058 MiB free against its 209.19 MiB floor => 30,969 claimable).
The 29,120 bound was my 1 GiB \`keep free\` reserve; the seam now derives the
number, so the next resize starts from the engine's arithmetic.
Ledger for the record: slab 16,235 -> 17,548 slots while the pinned host pool
hands the same experts back (8,893 -> 7,580) -- the transfer is the mechanism,
the pinned pool is the slow path. Capacity contract live-verified at 3 slots:
3 x 199,984-token prompts accepted (599,952 tokens), fourth refused as too
large to ever fit.
Probe law learned: the oversized-prompt refusal names the prompt's token count
BEFORE prefill (free calibration) and fires on prompt tokens ALONE, not on
prompt + max_tokens; the window is ~514,286 chars at 2.5714 chars/token.
The 2 GiB/card that \`rocm-smi\` shows free is not idle: the engine's budget
table calls it driver context + the 896 MiB activation arena + 96 MiB headroom,
and says the rest is `throughput on the floor`. Its own settle line is the
number that matters — `card 0 settled at 2.01 GiB free against 209.19 MiB (the
96.00 MiB --gpu-headroom-mib and 113.19 MiB kept for the static region)` — so
1,849 MiB above the ENGINE's floor is claimable and the real ceiling is 30,969
MiB, not the 29,120 my 1 GiB reserve invented on 2026-10-11.
Shipped pair 25600/5120 = 30,720: +1,600 MiB of slab, ~458 MiB above the floor.
The slab is already saturated at 24,000 (mover took all of it, 451 KiB slack),
so this is the only direction left — KV is at its 3-slot floor (5,063 needed of
5,120).
Named prediction, because today's two points say the slab is a THRESHOLD and not
a slope (22,208 -> 3,397 t/s, 24,000 -> 3,290 t/s at 43.5k single-stream): this
arm buys ~1,600 resident expert slots and ~0 prefill. The point that moves
prefill is 26,071 (2026-10-11: 4,379 t/s), fundable only at two slots. Prefill
moving > +-3 % here falsifies the threshold reading.
Seam: the ceiling gate now derives from the engine's budget table (30,969) and
is the single owner of the pair arithmetic; the duplicate I put in the shape
test is removed.
Frank's call: three slots, and spend the KV the third slot doesn't need on
resident experts. `--max-num-seqs 3`, `--vram-kv-mib 6912 -> 5120`,
`--vram-weights-mib 22208 -> 24000` — the pair is still exactly the measured
29,120 MiB/card ceiling. The pool funds 608,027 tokens against the 600,000-token
contract (3 x 200,000, +1.3 %), and the slab gains 8.1 %.
The direction is the 2026-10-11 A/B: prefill is slab-limited, KV beyond the
contract buys nothing. At fp8 the slab curve measured on this host is
22,208 -> 3,397 t/s and 26,071 -> 4,379 t/s at 43.5k, i.e. ~0.25 t/s per MiB, so
24,000 predicts ~3,850 t/s single-stream. Falsifiable; the probe is the check.
The cost is named, not hidden: a FOURTH full-height conversation no longer fits
(4 x 200,000 needs 6,601 MiB of pool) and is refused as too large to ever fit —
radiance never truncates. Reverting to four slots is one env pair.
Seam: two new arithmetic gates, because both failures are boot-then-400 shape
faults that no flag check catches — the pair must respect the per-card ceiling,
and the pool must back seqs x max-model-len at the dtype's measured rate plus
the dtype-independent terms (110 MiB a sequence of recurrent state, 448 MiB of
prefix checkpoints).
Probe on the live service: four concurrent 196,960-token chats, 4/4 exact
marker retrieval, 787,842 tokens in 238.4 s (3,305 t/s aggregate prefill = the
single-stream ceiling, so the four streams shared one pipeline and the KV pool
never ran short); prefix-cache repeat answered in 1.0 s with cache_n=43008.
Recorded with the caveat that 8 generated tokens do not make a decode rate.
Runbook gains the 2026-10-13 section (symptom signature, the runtime-knob law,
the shape arithmetic, the boot ledger that confirms it: 843,276 tokens =
4 x 200,000, 12 all-reduce rows, 5.87 GiB of KV lent to the slab). AGENTS.md
gains the fleet-row update, the KV dtype law at four slots, the sysfs-raise
lesson under Key Lessons, and the pinned-RAM paragraph now points at
scripts/ttm-preflight.sh as the repair. LLM index entry + topology node match.
a9da65e put the raise inside the serve script, which runs as `User=dank`: the
knob is root-owned 0644, so `-w` is false and the write silently failed — the
gate printed its refusal on a cap it had just failed to move. The privileged
path is systemd's `+` prefix: `ExecStartPre=+scripts/ttm-preflight.sh --raise`
runs as root even under `User=`, before ExecStart, so `Restart=always` can
never outrun it. The lib is now also executable (`--raise` sources the env file
and verifies), and keeps a `sudo -n` fallback for a human running the gate from
a shell.
Seam: the refusal cases run with a PATH that has `cat`+`bash` and NO `sudo`
(root writes a 444 file, so a passwordless dev box would have tested nothing),
and both now assert the gate's own exit 1, not merely non-zero. New seam covers
the unit line and the `--raise` entry point end to end (writable knob -> raised,
no args -> usage exit 2).
The TTM_TARGET_MIB guard I added on 1a6a78e was written `:"...` (no space), so
bash parsed the expansion as one word and executed `cmd :204800` as the
command `:204800`: exit 127 `command not found` on every start, before the
ttm gate ran. Caught on the deploy: the unit crash-looped 34 times in ~4 min
with exactly that journal line.
probe-frank-archon-kv.py now reads the slot count, window and KV dtype from the
deployed env on the host, so probe B exercises the capacity contract that is
actually live (4 x 200,000 today) instead of a hardcoded pair, and each probe's
marker starts with a unique word because show() scores the hit on that word.
The 2026-10-12 23:17 power failure reset /sys/module/ttm/parameters/pages_limit
from the manual 2026-10-05 raise (200 GiB) back to the kernel default (half of
RAM = 125,988 MiB), below the pinned-RAM ledger floor of 150,000 MiB. The serve
script's preflight named the refusal and exited 1; Restart=always re-ran it
6,919 times over ~20 hours and frank-archon served nobody.
scripts/ttm-preflight.sh (sourced by the serve script) now RAISES the knob
itself before refusing: the unit is root, the knob is a 0644 root module
parameter, and the cap is a runtime knob whose default is half of RAM — so a
manual sysfs write is not a repair procedure, it is a countdown to the next
reboot. FRANK_ARCHON_TTM_TARGET_MIB=204800 records the host ceiling; the gate
re-reads the cap after writing and only then refuses, so a knob the kernel
will not move is still a named failure.
Shape change (Frank's call): FOUR decode slots x 200,000 tokens with fp8
(quantized) KV. The pool must back seqs x max-model-len, so fp8 is what funds
the fourth slot: at bf16 the four slots cost 11,679 MiB of pool and would cut
the resident expert slab to 17,441 MiB (-33 %); at fp8 they cost 6,601 MiB, so
the pool stays 6,912 MiB (predicts 843,566 tokens), the VRAM pair stays at the
measured 29,120 MiB/card ceiling, the weights slab and the 44 GiB host pool are
untouched, and the fused TP all-reduce rides 4 x (1+2) = 12 rows, inside the
25-row bound where the wht6 wire is bit-exact.
Runbook section `2026-10-11 — unquantized KV at 2 x 200,000`: the dtype law
(only bf16|fp8, no K/V split; the QSA block key is dtype-fixed, which is why
bf16 costs 1.89x and not 2x), the pool formula that reproduces both live
boots within 0.1%, the seqs-3-to-2 provenance (the tcclaviger fork's
torch.compile constraint, not radiance's), the boot ledger, and the A/B.
The A/B, fp8 control arm through a transient FRANK_ARCHON_ENV_FILE drop-in:
prefill -29% (4,379 -> 3,126 cold), TTFT at the full window +25% (the 2 x 197k
pair: 110.5 s -> 138.1 s), decode inside the +-3% band on two of three passes,
draft acceptance UP 0.661 -> 0.740. The price is prefill, not decode. Open
call for Frank: keep exact KV, or take the cheap fp8 2 x 200k fit (pool 3,648).
Also corrects two stale index entries (w4dn8 said STAGED, fp8-native said
ACTIVE), adds the worklog entry, and commits the verification tool as
scripts/probe-frank-archon-kv.py — it carries the probe law that cost two
false negatives: radiance answers in reasoning_content, so a chat probe with a
small budget reads as empty content + finish_reason stop on a healthy server.
Tests 368/368.
The KV pool at the fp8 shape backed 386,904 tokens — 1.93 sessions of the
served 200,000 window, so a second full-height conversation preempted to the
host/disk prefix tiers. Moving the cache to bf16 (the engine's only
unquantized rung; `--kv-cache-dtype` takes bf16|fp8 and has no per-tensor
K/V split) doubles what a token costs: 7.3125 -> 14.625 KiB per rank
(kv_attn 26.00 -> 52.00 KiB per 4-token page + kv_qsa_bkey 3.25 -> 6.50).
Pool 3543 -> 6912 MiB (437k tokens, >= 2 x 200,000), and the weights slab
pays for all of it: 26071 -> 22208 MiB, the pair pinned to the measured
card ceiling 29,120 MiB (card 31,856 - 1,710 consumed outside both budgets
- 1,024 free). The resident set that leaves VRAM spills into the pinned host
pool, so 40960 -> 44928 MiB, and the prefix host tier doubles with the entry
size (58.50 -> 117 KiB): 4096 -> 8192 MiB. Pinned ledger ~147 GiB, ttm floor
140000 -> 150000 (cap 200 GiB).
--max-num-seqs 3 -> 2: two slots x the window is the capacity contract, and
the "seqs 2 cannot boot" law is the retired tcclaviger fork's torch.compile
query_start_loc constraint, not radiance's (radiance booted seqs 2 in the
fp8-native era; the 2026-10-06 sweep had 2 and 3 tied inside +-3%). Bonus:
the fused all-reduce is now declared over 2 x 3 = 6 rows, under the 25-row
bound at which the wht6 wire is bit-exact.
Cost, paid twice on this engine: idle KV VRAM is also the expert cache's
spillover (lends 2.69 GiB + 740 MiB of arena), so the pool steals weight
residency from the slab AND the lendable cache. Priors: 2.14 GiB of stolen
weights cost 2.5% decode (MAX_MODEL_LEN 262144), the EVC ladder's other
direction cost 8.9%. Prediction -3..-8% vs the fp8 control (136 t/s at 44.5k,
acceptance 0.7246); the deploy boot ledger and live probes are the referee.
Seams: MAX_SEQS/HOST_POOL/KV_DTYPE/PREFIX_HOST literals follow the shape, and
two new invariants guard it — the KV pool must back seqs x max-model-len at
the declared dtype (an engine hard limit: an over-long request is accepted
then refused as too large to ever fit; the formula reproduces the fp8 boot's
386,904 to within 0.1%), and the VRAM pair must stay inside the ceiling.
The 2026-10-09 wire aliases become the service names everywhere:
systemd units, serve scripts, env files and FRANK_* variables,
deploy-plan/deploy-remote service keys, container names, Prometheus
job names + alert rules, Grafana dashboard queries, bats seams,
runbook filenames, homepage/AGENTS lore, and historical prose.
Rollback tags renamed in lockstep. Residue in the tree: zero.
Retired frank-medium/small/npu names are history and stay.
Known casualty: scripts/patches/*.patch context lines were rewritten;
those retired-era vLLM-fork patches no longer apply byte-exact.
- AGENTS.md Model Config: the Protoss names are now the ids on the wire
(per-endpoint knob named); verify a rename on /v1/models, never on a chat
response — radiance echoes the requested name
- OMP Harness.md: harness id == served id; one id per model everywhere
- Homepage.md: the docs-page "api id" annotations and the switchover promise
are retired
- OpenWebUI.md: the id-keyed trio (model row, access_grant, api_configs
model_ids) is a foreign-key set — re-key it in the same deploy; plus the
contenteditable-composer probe note
- frank-secondary/frank-large/frank-primary runbooks: served ids
frank-zealot/frank-dragoon/frank-archon, and the bench.conf drop-in that
made every frank-primary deploy since 2026-10-08 04:09 restart the staging
worktree behind a green health gate
The 2026-10-08 rename moved the harness model ids and the homepage cards;
the served aliases on the wire stayed frank-primary/frank-large/
frank-secondary. This closes the gap: the alias each endpoint announces is
now the family name, while the services, units, scripts, env files and
runbooks keep the old names (container identity stays stable).
- scripts/frank-secondary.env ALIAS, scripts/frank-large.env
HALOGEN_MODEL_ID, scripts/frank-primary.env FRANK_PRIMARY_SERVED_NAME
(+ the two serve-script defaults/gate messages)
- seams re-pinned: frank_secondary/frank_large/frank_primary unit bats
- fleet harnesses retargeted: bench-probe, deep-probe, calib-push,
fleet-probes-swift15, autoresearch(+measure), bench-threads-batch-http,
bench-decode
- shared/homepage/docs.html: api-id annotations and the "switchover will
rename the ids" footnote removed — the page's names are now the ids;
opencode/Pi/curl examples send frank-zealot
- dankclaw/hermes/config.yaml.example: frank-large -> frank-dragoon and the
live 262144 context (Hermes routes by served id; see Hermes.md)
Consumers with live state re-keyed out-of-band in the same window:
OpenWebUI (model row, access_grant, openai.api_configs model_ids) and the
Hermes config.yaml.
The panel title carries a right-aligned 'AI guide ->' link to
/docs#models (the running-models section), at the point of use.
Bare mode recolors the link; layout is flex baseline, doctrine-minimal.
Seam: homepage_config.bats 6/6; headless smoke census + screenshot.
- Icons: Copyparty -> improvised amber folder, Marreta -> crumbling
wall (bricks + red crack), Calibre-Web -> book, OpenWebUI -> official
logo.png from openwebui.com, Seerr -> official seerr.dev
os_logo_filled.svg.
- Tiles with an open link are clickable whole (.panel.clickable: hover
lift + glow; inner link clicks pass through).
- Any DOWN panel (app tile or model card) gets a red gradient wash and a
slow pulsing glow (.panel.down, .mi-card:has(.badge.down)).
- docs / admin / account-settings buttons moved from the footer to the
header (nav.top-links); footer keeps the deployment stamp.
- Zealot compaction advice promoted from the Use paragraph to a bordered
glowing model.compaction callout (.mi-comp); component seam pinned to
the new field.
- Every app card carries hex-clipped art: 19 amber-tinted brand marks
from the Simple Icons CDN plus 19 improvised monogram SVGs, wired via
a per-service "art" field; homepage_config.bats pins art presence for
all 32 apps.
- Model cards render zealot -> dragoon -> archon (registry order pinned
in the seam).
- Dragoon and zealot share one common uncensored legal warning (adult
content, harmful/hateful/illegal descriptions, sensitive offensive
cybersecurity topics, severe legal risk, "don't ruin this server for
everybody"), hover label on card and docs.
- Archon gets a bordered red permission callout: ask Frank before use.
- SearXNG obfuscation-not-anonymity note moved to a card-level hover
warning (.svc-warn) and a docs hover note.
- Gitea: self-service signup flow; no need to ask Frank.
- Footer docs/admin/account-settings become glowing bordered buttons;
the admin button is shown only to homepage-admins members via a new
Caddy /whoami endpoint echoing the outpost-injected X-Authentik-*
headers (spoof-proof; the /admin* 403 stays the real gate). e2e pins
both sides.
- Docs naming swept: Protoss names primary, bracketed api ids, explicit
note that a future switchover renames the ids themselves.
The Protoss model cards render as .mi-name inside one full-width panel;
the panel-name census now counts both render styles, keeping services.json
the single source of truth.
The model cards become the StarCraft trio: frank-large -> frank-dragoon,
frank-primary -> frank-archon, frank-secondary -> frank-zealot (family
names only; endpoint/API ids unchanged). The three cards fuse into one
Model Information component: per-unit Protoss accent colours and hex-
clipped unit art, 15px type, clickable StarCraft unit-voice quotations
(original .ogg files from the fandom wiki, served from /sfx/), supervision
notes (dragoon/archon safe for autonomous use, zealot tight supervision),
compaction-model recommendation for frank-zealot, and uncensored-model
legal notes rendered as hover tooltips instead of wall text. docs.html
carries the same renames plus api-id mapping.
App-card updates: Seerr, Calibre-Web, RomM, Jellyfin set to WIP: DOWN
with Open links removed (new wip "down" badge state); Copyparty notes it
serves all other media apps content including WIP apps; Gitea and Plex
noted as needing logins separate from SSO; SearXNG noted as obfuscated,
not anonymous (upstream API keys tie to Frank).
Plumbing: Caddy @data allowlist grew /sfx/*; deploy_homepage copies sfx/;
homepage_config.bats pins the new probe names, the component fields,
art/voice-file presence, and the wip-down schema.
Every fleet model card (and the #models docs section) now states, in
plain family-facing terms, what the model is equivalent to:
frank-large ~ Claude Sonnet 5 / OpenAI-Luna, frank-primary ~ Claude
Opus 4.8 / OpenAI-Terra (both Summer 2026 frontier), frank-secondary ~
last year (2025) frontier. Rendered as a new details.equiv line
(desc-equiv, styled in the head color, bare-mode clean); pinned in
homepage_config.bats.
The harness clamps max_tokens to min(45056, contextWindow - est - 4096),
its chars/4 est under-counts the rendered prompt by ~3 % (measured:
152,938 -> 157,388 and 158,906 -> 165,301), and compaction fires only at
85 % -- so with window == engine ceiling 200192, every request in the
est ~146k-184k zone was a guaranteed 400. HALOGEN_CTX=262144 (native) +
KV pool 524288; harness window 217088 = ceiling - 45k absorbs the
estimate error (worst-case request ~219k <= 262144). models.yml updated
out-of-repo: contextWindow 217088, maxTokens 45056.
The @data path allowlist on the post-auth listener did not include /img/*,
so docs screenshots fell through to the index.html catch-all and the
browser got 200 text/html for every PNG (docs images broken live
2026-10-08). Pinned by the homepage_caddy seam.
- style.css: green phosphor -> amber CRT-AMBIOS palette; docs styles for
screenshots, code blocks, key warning (full + bare mode)
- index/admin/docs: VT323 from the Google Fonts CDN
- docs.html: rewrite #chat/#models for the live fleet (flash-server v2,
radiance native FP8, Qwen3.5-9B Q6_K); new #api-access with OpenWebUI
key walkthrough + Opencode and Pi configs; three OpenWebUI screenshots
- services.json: AI card details match the current fleet
- deploy-remote.sh: ship shared/homepage/img/ to /srv/homepage/
- docs: Homepage.md theme wave + OpenWebUI.md picker access-control law
rad_arch_declare rejects the split tier (down block-fp8 + gate_up w4): the
qwen4exp_fp8 kernel serves one encoding per expert table (w4nl64a8h) with only
whole bf16 experts beside it. Recipe now sends gate_up AND down through the
GPTQ w4 rule (down calibrated by the Grams, gate_up uncalibrated), drops the
down-fp8 rule, and lowers the size floor 126000->108000 MiB for the ~19 GiB
lighter container. Convert script: checkpoint lives under models/ukisai/, and
the in-container path must keep the subdir (basename stripped it).
OpenWebUI model picker narrowed to exactly frank-secondary (connections
frank-large/frank-primary disabled in openai.api_configs, model table rebuilt,
stale frank-small model_id retargeted to the served alias). Documented the
auth-gated API surface (anonymous probes all 401; keys per account).
Authentik roster gains artem (pk 16) in the human-user template.
kassie (pk 14) and alex (pk 15, deleted pk 7 then recreated) now share the
ridley shape: <user>@franksdank.cloud + Non-Admin Users + jellyfin_users +
openwebui_users. Audit before destroy: the old alex row carried no MFA
device, token, entitlement, or proxy session - checked across the MFA
device tables, authentik_core_token, and providers_proxy_proxysession
(the fleet's column names differ from the ORM names: events.user is
denormalized JSON, certificatekeypair has no user_id). Events survive the
delete, so the audit trail keeps its shape.
Co-Authored-By: omp <omp@franksdank.cloud>
Deploy verified: nvidia-smi 21,436 MiB used / 2,689 free at ubatch 2048 -
the MiMo-priced projection (~21,430/~2,700) landed within 11 MiB, and the
verbosity-4 ledger confirms the KV law to the MiB (CUDA0 KV buffer
12,544.00 MiB = K(f16) 6,272 + V(f16) 6,272 = 32 KiB/token over the
401,408 pool), compute buffers 1,136 MiB CUDA0 + 848 MiB host, mmproj
879 + 248 MiB. Playwright gate: 33 passed.
Two lessons recorded: the 203/EXEC boot loop my shebang-less rewrite
caused (journal signature + the head -1 check), and that llama.cpp's
"n_ctx_seq < n_ctx_train" line is informational when the context is BELOW
the trained window - the Gemma4 drafter warning was the opposite case.
The unit's Description named the Swift-1.5 era (and said 2x131072/q8_0/
draft-mtp); it now names the Q6_K shape. Cosmetic, so it is NOT deployed
as its own restart: the next frank-secondary deploy relinks it.
Co-Authored-By: omp <omp@franksdank.cloud>
Capacity trade: the 9B dense-hybrid (qwen35: 32 layers = 8 full-attention
+ 24 Gated DeltaNet, 4 KV heads x 256 dim, native 262,144, MTP-less,
natively multimodal) costs ~7 GiB of weights where the 26B MoE cost
~16.6 GiB. The freed room buys back the second conversation slot and
exact KV: --ctx-size 401408 --parallel 2 (two 200,704 windows), f16 K +
f16 V, MTP off (the file carries no nextn/mtp tensors - flag-only enable
is impossible), vision via the publisher's BF16 mmproj, ubatch 1024
(the CUDA decode-optimal value), verbosity 4, publisher sampler recipe
(0.6/0.95/20/0) in models.yml.
Hosts with the MiMo-V2.6-Distill Q6_K boots at this geometry: 21,276 MiB
used / 2,849 free at ubatch 1024, 40,960 tokens of fp16 KV at ubatch
2048. Compaction stays routed here; window math is unchanged (maxTokens
16,384 keeps the worst-case harness request inside the slot).
Gemma4 payload (Q4_K_M + mmproj + MTP assistant) stays on disk as the
rollback; its runbook is retired in place, keeping the attention_k_eq_v
pair-billing law.
Co-Authored-By: omp <omp@franksdank.cloud>
Verbosity 4 paid for itself: the engine's own llama_kv_cache lines give the
exact accounting — the cache bills the K+V pair TWICE under attention_k_eq_v
(6.125 B/elem at f16 K + q8_0 V), reproducing 3,001.25 MiB (5 full layers at
n_head_kv 2) + 306.25 MiB (25 SWA rings at n_head_kv 8, window 1024) to the
byte. The earlier "bill K-only" note was wrong; full layers have 2 KV heads,
not 8.
Deployed boot: 21,868 used / 2,257 free — the f16-K raise (+2,161 MiB KV) is
funded by the ubatch 2048 -> 1024 cut (~616 MiB of compute buffer) plus
variance, so exact K came essentially free. Measured decode 139.9 t/s temp 0;
vision probe returns Red at prompt_tokens 151.
Payload: HauhauCS/Gemma4-26B-A4B-QAT-Uncensored Q4_K_M (16,796,015,520 B)
+ the publisher's BF16 mmproj, /home/dank/models/Gemma4-26B-A4B/.
Shape: ONE 200,704-token slot (--parallel 1; 2x200,704 does not fit the
24 GiB card), f16 K + q8_0 V under flash attention, batch 4096 / ubatch
1024, --log-verbosity 4, MTP OFF (the gemma4-assistant file is on disk,
not loaded: its draft KV is built against the target's full window,
measured +1.9 GiB -> 23,806/319 free, no margin beside the f16-K raise).
Measured boots (nvidia-smi): q8_0 K + q8_0 V / ubatch 2048 / MTP off =
21,890 used / 2,235 free; with the MTP assistant = 23,806 / 319.
Engine facts recorded: gemma4 hybrid attention (25 sliding-window + 5
full-attention layers) so KV scales with 5 layers only, and
attention_k_eq_v is honoured by build 5fc4f3c (pool bills K-only; the
measured q8_0 pool = 2.05 GiB = the K-only figure).
Serve script gained VERBOSITY (+ --log-verbosity in the argv), the
SPEC_DRAFT_FILE drafter path, and defaults/lore retargeted to Gemma4.
deploy-remote.sh payload gates follow the env. The frank-secondary bats
seam is rewritten for the new contract (argv-based) and fixes a
pre-existing vacuity: serve_argv regenerates the test env, so
mutations made before it were silently undone - tests that mutate now
exec through run_stage. frank-secondary is the fleet compaction hub
(200,704 is the only window that fits a worst-case compaction input).
The KLD engine exited clean with the record complete (kld.json rows:
50235, 24 GiB, 5 files), but the wrapper died rc=127: a mid-run scp of
the calib-shape edit shifted bash's lazy-read offset — the fleet's own
script-swap law, biting on the same class as the autoresearch collision.
Resume path: SKIP_KLD=1 verifies the receipt (rows + size + files) and
goes straight to calib. Never scp a script a running process owns.
Execution facts, blocker trail (plugin-tap probe law, weights-slab
formula, KLD_MOUNT), engine verdict, and the post-chain order. Linked
from the docs index per the completeness seam; the cutover runbook
replaces it.
MNBT 65536 / seqs 8: MOE-W4.md:244 measures 640 tok/s at a 32k step
vs 930 at 55k — the KLD record's own rate confirmed step size drives
disk-tier prefill. 131k resident tokens fits the 3072 pool (~237k
tokens/rank at fp8). Cuts the 10M-token calibration from ~4.3 h to
~3 h.
apply-protected-cast.sh splices derive-protected-experts.py output into
the recipe's PROTECTED-BEGIN/END section (refuses to re-derive, trashes
the pre-edit recipe). kld-gate-swift15.sh boots the candidate on the
serving pin in --kld-ref mode (pool fully resident, no disk tier) and
enforces mean KL<=0.03 / top-1>=95% from the JSON report.
One-GPU serialization: the chain waits for the kld driver log to close
rc=0, sanity-checks kld-ref, boots the calibration server, pushes the
corpus at parallel 16 (4 engine slots; queue absorbs the rest), then
kills the server so its trap restores production.
kld died: 'cannot read --kld-corpus /data/kld.jsonl' — KLD_MOUNT was
referenced but never defined, so the corpus/ref-dir mounts silently
expanded to nothing. Add them (nested order: after the /data mount).
feat(convert-swift15-w4dn8): final recipe convert on 1.0.13
Convert with the SERVING pin so the .rad crosses no version boundary;
only the Gram files cross (RADGRAM2 magic, gram.r%d.%s.bin naming and
calib_gram/calib_resolve/calib_stem are byte-identical between the
1.0.13 and 1.2.0 libquant). Host-backend run: prod can stay up.
Gates: grams present (>=48), bf16 container deleted, disk floor,
rad-info gptq receipt, size floor 126000 MiB.
The weights slab derives as card - headroom - kv, and the disk tier
refuses under 9.57 GiB (4.21 layer buffers + 5.36 fixed). 4608 starved
it (8.68 budget, DID NOT FIT). 3072 leaves 10.2 for weights; 4x16k fp8
slots fit the pool, recording is prefill-bound so slot count only
queues pushes.
kld stays on 1.0.13 (record on the scoring engine) with --vram-kv-mib
4608 + 16 slots: the ratio-derived pool came up SHORT by 217 MiB at
32x16k. calib moves to 1.2.0: the RADIANCE_CALIB_TOKENS tap is a 1.2.0
qwen4exp plugin feature (1.0.13 plugin has the DIR half only), and
Grams are bf16 activations - engine-invariant. --vram-kv-mib 9984
covers 64x16k fp8 slots.
Engine v1.2.0 (Codeberg e95d397, Docker Hub tag 1.2.0, index digest
665775ed...) removes --expert-vs-cache-ratio. The explicit
--vram-weights-mib 26071 / --vram-kv-mib 3543 pair transcribes the live
1.0.13 boot ledger at ratio 0.82 (weights 25.46 GiB, KV 3.46 GiB per
card); bracket arms scale KV linearly in (1-r): 0.90 -> 1968, 0.95 -> 984.
Serve script, autoresearch harness, and bats gates follow; the bats
shape asserts are realigned to the deployed seqs=3/spec=2 sweep result
(95923d3, df6b5bb) they failed to track.
- download-frank-primary-swift15.sh: ukisai/Swift1.5-Qwen3.8-Flash-Next
(BF16, 131 shards + index + configs, 360037355038 B = 335.3 GiB, ungated)
via the vendored hfget.sh; verify = listed sizes + 131-shard weight-map
count + every shard's sha256 against the Hub LFS oid. Pattern cloned
from scripts/download-frank-primary.sh.
- qwen4exp-swift15-w4dn8-i8-hc8m.recipe: routed experts with DOWN at
block-fp8 (rtn codes=fp8_e4m3 block=128x128 scale=bf16, the served
FP8-native tile form; moe_gemm_fp8a8 reads it; minicpm5-2b is the
shipped bf16->fp8a8 precedent) and gate_up at w4nl64 GPTQ cd=3;
hc/MTP-fc fp8, trunk i8, lm_head i8, PLE fixed scale, draft_head u2
(inert). Protected-expert cast lines carry the BASE checkpoint's names
as a warm start; the derivation step rewrites them from Swift's own
calibration recording before the recipe is ever run (law documented
in the file).
23,120 used / 1,005 free at boot, 763 settled after a vision probe
(encoder buffers resident). Rung cost 746 MiB, not the transferable
616 prediction. Prefill baseline 1,209 t/s @ 3,760 recorded; decode
unchanged as the law says. 2048 = ceiling for this shape.
Frank's order the same night. The 1024->2048 rung measured +4.6 %
prefill for 616 MiB on the 3090 (2026-10-05 boot pair); the 112k cut
left 1,751 MiB free, so it rides with ~1.1 GiB predicted margin.
Decode is ubatch-invariant; batch 4096 >= 2048 keeps the seam happy.
Cross-cutting rename of the current llama.cpp endpoint on the RTX 3090
(port 7771) from frank-small to frank-secondary:
- Service trio: scripts/{frank-small.env,frank-small.service,
serve-frank-small.sh,download-frank-small.sh} → frank-secondary.*
- Test: tests/frank_small_unit.bats → tests/frank_secondary_unit.bats
- Runbooks: serverAppConfigs-docs/LLM/frank-small-*.md →
frank-secondary-*.md (9 files); the FreeToken runbook stays
freetoken-frank-small.md (it documents the retired CUDA-era service)
- Identifiers: FRANK_SMALL→FRANK_SECONDARY, llm_small→llm_secondary,
LLM_SMALL_*→LLM_SECONDARY_*, deploy_llm_small→deploy_llm_secondary
- Historical references to the OLD frank-small (FreeToken/CUDA on the
3090, and the Dankserver service that became frank-medium) are
preserved verbatim — they describe what the service was called
at the time.
The 2026-10-06 cutover crash-looped frank-large 8 boots (03:09:39-03:11:42):
the engine's pin gate refused to start while host MemAvailable was ~74.7 GiB,
3.34 GiB short of the 78.11 GiB it needs (62.11 GiB checkpoint + 16 GiB floor).
It self-resolved in ~2 min when the dying 27B engine freed RAM, but left the
host with only ~1.8 GiB MemAvailable - a restart would crash-loop again.
serve-frank-large.sh now checks MemAvailable against the pin gate BEFORE
podman run and WAITS up to FRANK_LARGE_RAM_WAIT (default 600 s) for the RAM
to clear, so a transient dip boots once instead of crash-looping; if it never
clears it refuses loudly. The 16 GiB floor is the engine's (not
HALOGEN_HOST_RESERVE_GIB=20, a separate overall-budget knob).
frank_large_unit.bats: FRANK_LARGE_MEMINFO seam + a refuse test. Runbook +
log: the pin-gate law, the incident, and the fix.
autoresearch.sh (dev box) scp's the candidate scripts/frank-primary.env
+ the bench script to dankserver and runs it over ssh.
scripts/autoresearch-frank-primary.sh (dankserver) follows the bench
policy: stop frank-primary (relink unit: stop, not mask), boot a
scratch radiance container (arbench) on the same two cards with the
candidate argv, poll /health, run ONE self-contained greedy completion
(fixed ~140k-char real-text prompt -> ~32k tokens, 256-token decode),
parse the engine timings into METRIC lines, tear the scratch down, and
restore production via a trap. Fresh prefix-cache tier per run for cold
prefill. Baseline validated: decode_tps=31.70, prefill_tps=465.4,
ttft_ms=93745, draft_accept=0.831 at prompt_n=43627.
The writer's 31b1ec3 added LLM/frank-large-flash-server-v2-swift15.md
but did not wiki-link it from index.md, which failed the docs-index
seam (test 135). Add the link as the new ACTIVE frank-large runbook
(supersedes halogen 0.1.4).
The engine's kernel selector reads $RADIANCE_HOME/tune/<machine>.tune at
boot; $RADIANCE_HOME defaults to /opt/radiance, which lives in the image,
so a cache written by a --rm scratch container dies with it. Pre-create
$FRANK_PRIMARY_STATE_DIR/tune and bind-mount it at
/opt/radiance/share/radiance/tune so the rad-tune cache (generated in a
stopped-service window) survives and is read at every production boot.
AGENTS.md: row + section carry the measured 177,109 MiB container, floor
177000, live cutover 2026-10-06. Runbook: the three convert laws the live
runs proved (scale-name, file-shadow, symlink-through), the measured boot
(125.26 GiB page-in 38.1 s, PLE pin 47.68 GiB/43.9 s, ~2 min to health),
the MTP/vision/tool/reasoning contract results, and the SSD tier fix.
The engine's create is not recursive: booting with only the state dir
mounted left the SSD prefix tier off ('cannot create
/data/kvcache/frank-primary: No such file or directory'). The serve
script now mkdir -p's the full path before podman run.
Stop-window A/B, same 80k filler prompt, 128-token essays, runtime
timings: HALOGEN_FA_TQ=256 (upstream issue #12's carve workaround) moves
decode ±0.3% (noise, draft acceptance identical) and costs 9.5% prefill
on this topology. The depth collapse sits deeper than the FA tile knob;
the default stands. The harness's slow/advisor roles moved to
frank-primary (fp8 KV 7.32 KiB/token = 8.7x less KV read at depth).
The first deploy refused: floor 178000 > actual 177109 MiB - the plan-only
estimate (172.76 GiB) prices the data planes but not the container's
metadata tail. Floor 177000 sits just under the real payload.
Probe on the live endpoint (2026-10-06): 17.3k cold prefill 27.0 s ->
warm turn resumes at cache_n 16,384, prefill 1.7 s. The boot WARNING
'smaller than ONE full-context entry' sizes against the native 262,144
window, not our 200,700 slot — documented as expected in the checklist.
Tuning notes record the depth law (decode 5.5 t/s @ 46k -> 3.3-3.8 @ 80k
vs 30-50 shallow; KV reads ~8 GB/s effective; no supported flag touches
decode) and the AGENTS.md unit paragraph regains the unit-facts
parenthetical the reflow dropped.
Plan-only measured the .rad at 172.76 GiB (176,906 MiB) - the checkpoint's
own fp8 mass, byte-stacked experts plus the PLE re-encoded at the
checkpoint's scale - not the ~161 GiB first estimate. Floor 158000 -> 178000.
Auto cache-sizing is OFF on this box: the boot ledger refused it at
36.9 GB (2 x full-entry) floor vs 35.7 GB free, so every deep agent
request re-prefills its whole 79-81k context (84.9-179.8 s measured on
serve_api lines, 2026-10-06) for ~500-650 new tokens per turn. Pin
HALOGEN_CACHE_MB=16384 (>= one full-slot entry at ~70 KB/token) — the
documented override; the per-store reserve re-check keeps the host safe.
Also: the serve script's 'podman run -d > LOG' captured only the
container ID — the runbook's boot-ledger grep read a one-line hex file
and the ledger existed nowhere on disk. Foreground 'podman run' makes
the ledger land in the log file as documented; exit-code and ExecStop
semantics are unchanged. Test seam updated for both.
The reader opens every index-named file in sorted order and emplaces
EVERY tensor each file physically holds, first-wins (checkpoint.cpp
add_shard + shard_list). The raw model-000NN shards carry tensors named
...ple.ple_embedding.ngram_embedding.shard_0.weight ([2500012,160] fp8)
- the identical name the plugin asks the merged table under - so
ple.merged.safetensors lost the emplace race and the plan read one raw
shard as the codes (declared [320001536,160] vs codes [2500012,160]).
Rename to 00-ple.merged.safetensors: '00-' wins the sort, the merged
bf16 tensor wins the name, table_bf16 declares and the rtn rule makes
the fp8 container form.
Runbook reshaped: 2 x 131072 no-spec identity, slot-2 arithmetic (drafter ~0.4 GiB + ubatch ~0.3 GiB bought the second 4,352 MiB KV slot), ngram-mod skipped on the recurrent-rollback law, MTP references kept (44.8-52.1% acceptance, 58.9-66.9 t/s), rollback = one revert of 812d9eb. AGENTS fleet row + section, LLM/index.md intro + entry, root index entry all flipped to the 2026-10-06 shape. 360/360 suite green; e2e gate 33/33 green at deploy.
CTX 262144 / PARALLEL 2 = two 131,072 conversations; SPEC_TYPE=none strips both speculative flags in the serve script (ngram-mod skipped on purpose — on a qwen35 hybrid it disables recurrent-state rollback); ubatch 1024->512 claws back ~0.3 GiB. Budget: non-KV ~15.5 GiB + two q8_0 slots 8,704 MiB lands AT the card — the 2-3 GiB floor is knowingly violated by explicit order; nvidia-smi free at boot is the measured truth. Seam retargeted: shape (2 slots, 131072/slot), budget (two fit, three don't), spec-off (absence of both flags), unit Description.
Run 3 rewrote the staged index THROUGH the relative symlink into the raw
checkpoint: the raw index.json was restamped with the stacked layout
(605 KB smaller), the raw verify then failed 143/144 and the overnight
run died before stacking. Two fixes:
- stack-qwen38-official-fp8.py: index/config writes are now temp+os.replace
(an atomic swap replaces the symlink itself; the raw tree stays pristine).
- build-frank-primary-rad.sh: the stage's index.json is COPIED like
config.json; only shards stay relative symlinks.
- release the numpy frombuffer view before closing the source mmap
(Python 3.14 BufferError: cannot close exported pointers exist).
- stacked scale planes rename: checkpoint.cpp scale_name() appends
_scale_inv to the STACKED tensor name verbatim (no .weight segment) -
...experts.gate_up_proj_scale_inv, not ...gate_up_proj.weight_scale_inv.
Boot ledger lines, 19,404 used / 4,721 MiB free (the q8_0 proof), decode arms (66.1 t/s @0.8, 66.9 @0, 58.9 @recipe 1.0 with acceptance 52.1%/52.2%/44.8%), vision Red probe, reasoning_effort xhigh/high contract, e2e 12-crawl transient + full-gate green rerun. Unit Description updated in-repo; the relinked unit picks it up at the next deploy/restart (no bounce for cosmetics). 360/360 suite green.
The parallel session's uncommitted swap, committed at Frank's direction
(commit whatever's in the working tree):
- llama.cpp master on the RTX 3090 now serves ukisai's
Swift-1.5-Qwen3.8-27B GSQ-RCO IQ3_S WITH the in-file MTP head, ONE
131,072-token slot at Frank's 128k cap, q8_0 K + q8_0 V, vision through
the base-model projector; MiMo-V2.6 Q6_K (2x200704 f16 KV) retired to
rollback target.
- env/serve/download + seam twins updated together (deployer payload gates
rode the previous commit's deploy-remote.sh).
- docs: new swift15 runbook; MiMo Q6K runbook retired; AGENTS.md topology
and both vault indexes follow (root index ACTIVE/RETIRED flip, test 135
seam green).
- comfyui: two qwen-image t2i workflow drops under dankclaw/comfyui/
workflows/ (12 KiB).
Replace the FreeToken PR587 endpoint (single-GPU: its TP=2 dies in the
pynccl.cu ROCm JIT) with the radiance engine on both R9700s, TP=2 via
libr4d's fused one-shot all-reduce. Official Qwen/Qwen3.8-Flash-Next-FP8
payload at native block-FP8 precision:
- scripts/build-frank-primary-rad.sh: staging (quantization_config drop =
plugin SELECTION law) -> checkpoint reshape -> rad-convert with ONE
--quant rule (PLE rtn fixed-scale); plan-only first.
- scripts/stack-qwen38-official-fp8.py: the two value-preserving reshapes
(experts byte-stacked into the plugin's stacked layout; PLE's 128
scaleless fp8 shards decoded to one bf16 shard with the checkpoint's own
scale; the rtn rule re-encodes at that same scale - codes round-trip
exactly and land the one-element scale plane the plugin declares).
- scripts/hfget.sh + verified download (131/131 shards sha256-match).
- froggeric v22.5 chat template vendored (--override-chat-template).
- serve script (pinned ledger: 2x40 GiB pools + 47.7 GiB PLE + prefix host),
TTM preflight fixed for kernels with pages_limit only (no pages_max -
an absent knob is skipped, not read as zero); pages_limit raised to
52,428,800 pages = 200 GiB on the host pre-cutover.
- root unit, relinked; deployer gates image digest + .rad floor 158,000 MiB
+ health; prometheus re-adds the /metrics job.
- rollback: tag frank-primary-freetoken-pr587-20261005 (FreeToken trio +
Dockerfile); payload + image stay on disk until the window closes.
deploy-remote.sh also carries the parallel session's frank-small payload
gates (swift15) - committed here per Frank's direction to commit the tree.
FreeToken has no /metrics endpoint — the job scraped 404s forever, a
structural never-green, not an outage (the Lucebox DS4.1 precedent). The
LlmEndpointDown alert regex self-adjusts (frank-.* matches existing series
only). Comments updated; seam now asserts frank-primary's absence. If
FreeToken grows /metrics, re-add the job.
FreeToken's /health returns 200 with {"status":"loading"} while the
engine loads — the FastAPI front end binds before the scheduler is up.
Measured 2026-10-06: the deploy gate went green after 6s while 131 shards
were still paging in (vLLM 503'd during load, which made the bare curl
honest there). The frank-primary gate now loops until status:ok; the
MainPID stray-port guard is unchanged.
Measured first boot: the auto attention backend is qsa_sparse, which
overrides page size to 64 and refuses --num-tokens 300000 ('nearest valid
values: 299968 or 300032'). 299,968 x 24 KiB = 7.0 GiB still clears the
177,356-token harness worst case and one native 262,144 conversation.
Replaces the tcclaviger vLLM fork with FreeToken at PR #587 head 77a7201
built on dankserver into localhost/frank-primary-freetoken:pr587-77a7201
(scripts/docker/frank-primary-freetoken.Dockerfile: digest-pinned
rocm/pytorch rocm7.14 torch-2.11.0 base, libhiprtc dev symlink, curated
--no-deps install per docs/install_amd.md).
Shape (first-boot conservative): ONE R9700 (second card idle;
--tensor-parallel-size exists for future 2-card tuning), KV pool
--num-tokens 300000 (7.2 GiB at 24 KiB/token; clears the 177,356-token
harness worst case and one native 262,144 conversation), MoE LRU
--moe-cache-rate 0.10, --max-running-requests 2, --max-output-tokens
36864. Parsers AUTO: thinking -> reasoning_content natively (models.yml
flips reasoningContentField on the harness side). Experts + PLE pin
~170 GiB host RAM — root unit keeps LimitMEMLOCK=infinity.
Container identity renamed to the unit name (Frank's call): frank-primary;
vLLM-era frank-primary-qwen joins the legacy cleanup list. Deployer image
gate becomes podman image exists (a local build carries no RepoDigests;
the old digest grep refused exactly this shape). Rollback tag:
frank-primary-vllm-shapeG-20261003.
- FROM pinned to sha256:a223aee1... (the pulled rocm/pytorch rocm7.14
pytorch 2.11.0 manifest); a re-pushed tag cannot change the stack.
- The pip-ROCm torch ships libhiprtc.so.7 only; the PR's _cpu_moe links
-lhiprtc and ld needs the unversioned symlink in the already-linked
_rocm_sdk_core/lib (first build died: cannot find -lhiprtc).
The parallel session's stash pop left GGUF Compatibility, Multi-Device
Splits, Serving Sizing, PCIe Link and Atomics, RDNA4 Container Serving,
vLLM Porting triage and RDNA4 MoE Budget unindexed and carrying 9
unresolved wikilinks (LLM Bench Windows / Model Swap / AGENTS targets
that never existed, a [[nodiscard]] C++ attribute parsed as a wiki link,
and two multi-line-wrapped link tokens). All repointed to real pages;
six rows added to the vault index; seam green.
1862dc3 repriced the Ornith rung (nine full-attn layers: f16/f16 = dead
boot) and flipped CACHE_TYPE_V to q8_0, but its note that the seam was
'retargeted in 8f53df7' covered only the payload paths — the KV test
still asserted the 2026-10-03 unquantized-both-halves contract and held
the suite red. Test now pins the shipped contract: K=f16, V=q8_0,
flash-attn on.
The default-project-set rewrite dropped non---project args (e.g. --list),
so 'run.sh --list' silently RAN the whole suite instead of listing.
Args now ride alongside the default set; an explicit --project still
takes over. Verified: default --list = 33 tests listed; opt-in --list =
34 with the frankbook walk.
The rogue session's swap left this env reprice uncommitted in the tree
(2026-10-05): Ornith is a 33-block qwen35 hybrid with NINE full-attn
layers (18 KiB/token/half), +1,900 MiB of weights vs MiMo Q6_K — f16/f16
lands ~379 MiB free, a dead boot; pool-stepping cannot rescue it because
the compaction invariant needs slot >= 191,148. The only lever that
keeps 2 x 200,704 AND the 2-3 GiB floor is q8_0 V (~2.87 GiB free). K
stays f16. Deployer gate + seam already retargeted in 8f53df7; fetch
script manifest already flipped in 3ee88d7. MEASURE after boot; under
1.5 GiB free, UBATCH_SIZE down to 1024, never the window.
The 17:47 Ornith-1.5-9B swap (77decb5) updated frank-small.env and
serve-frank-small.sh but left two consumers on the MiMo-Q6K paths:
deploy-remote.sh's LLM_SMALL_MODEL_FILE/LLM_SMALL_MMPROJ defaults (the
payload-existence gate checked files the unit no longer loads) and
frank_small_unit.bats (stubs + env assertion), which held the whole
bats suite red. Both now point at the Ornith-Q8 payload; docs updated
with the conditional frankbook e2e project.
The rogue session's Ornith commits (3ee88d7/77decb5) swapped the env and
fetch script but left the deploy handler gates and the seam asserting
the MiMo-Q6K paths — the seam failures 195-203 and a deploy that gates
a file the unit no longer reads. Flip both to Ornith-Q8 (Q8_0 GGUF +
BF16 mmproj), keeping the anti-drift twin structure intact.
The 11-admin spec walked links that terminate on FrankBook-Pro
(192.168.0.112), which sleeps when idle — every deploy gate timed out
there (measured 2026-10-05, two deploy windows). Split: the frankbook
links move to tests/e2e/specs/19-frankbook.spec.ts under a frankbook
project that run.sh excludes by default and deploy.sh triggers ONLY
when a frankbook blast-radius path changed (the spec, frankbook-pro/,
the homepage registry, the Caddyfile arr vhosts). deploy.sh computes
the trigger from the same BASE..HEAD revision range the plan uses.
The first constants were wrong in three ways, found while resuming the
checkpoint (the file was COMPLETE; the manifest was not):
- checkpoint is 35,865,565,184 B / LFS sha256 274c3fc7... (not
35,880,329,102 / 40874e7a...), 1352 tensors per the card;
- the tokenizer is a FLAT tokenizer/ DIRECTORY of 5 files mounted whole
at /tokenizer (model-card docker-run parity; the engine's
HALOGEN_TOKENIZER default is the directory) — there is no
frank-large.tokenizer.gguf;
- plain git blobs (vocab/merges/jinja/config) carry no LFS oid, so the
fetch script size-checks them and sha-verifies only the two LFS
objects (checkpoint, tokenizer.json).
Gates flipped together (serve + deployer twins): tokenizer dir must be a
real directory and every named file a regular file, no symlinks (HF
snapshot links dangle in the mount). Fetch made idempotent: a file at
the published size skips the transfer (resuming it is a CDN 416 under
set -e). Seam rewritten to the same contract, 17/17.
curl drops the Range header across the HF /resolve/ 302, so '-C -'
against the resolve URL pulls the 1 KB redirect body and exits 0 with
the file still 14 MiB short (measured 2026-10-05). Resolve the
effective URL first; the Xet CDN honors Range.
Runtime swap ROCmFPX llama.cpp -> peonist-ai/halogen (image digest
dc0a39a0, tag 0.1.4), serving qwen3.8-27b-p1w4d-d2.hgn (33.4 GiB, the
.hgn format loads in no other engine) with HALOGEN_DRAFTER=2 (DFlash2)
on ONE slot of 200700 ctx: slots>1 disables the drafter (engine law).
Accepted: TEXT-ONLY (vision encoder ignored; frank-small owns images)
and the censored base (rollback = revert this commit; payload, fork
build and build-rocmfpx.sh pin stay on disk).
- serve script: podman run under the root unit, exec podman wait so
Restart=always is real; gates image digest, checkpoint byte-count,
flat regular-file tokenizer, GPU nodes; engine stdout to log.
- deploy handler: gates the artifacts, sweeps the legacy bare
llama-server + old container name by NAME (never pkill llama-server:
frank-small), retires the 2026-09 quadlet, port cutover 7770.
- fetch script: -C - resume, -f, byte count, sha256, --verify-only.
- seam rewritten (17 tests): halogen is now the deployed runtime.
Both indexes stated the pre-doubling VRAM as current. Now 2,279 MiB free with
the 4096/2048 pair named, so no index contradicts the runbook it links to.
The AGENTS.md lesson "`--ubatch-size` has an OPTIMUM on this stack" was load-
bearing and over-general: it was measured on the 2x R9700 layer split, and the
same knob on the CUDA 3090 behaves the opposite way (curve climbs monotonically
to 2048). That difference is now the reason frank-small ships ubatch 2048, so
the lesson names the backend it was measured on and points at the counterpoint.
Adds the 2026-10-05 boot-pair A/B to the runbook (+4.6 % prefill at 8,928
tokens for 616 MiB, decode flat, the sub-ubatch prompt as the control), marks
the superseded 21,276/2,849 VRAM row, and logs the day. The old "ubatch 2048
failed the scratch boot" line stays visible as what it was: a predecessor-shape
observation, not a law.
BATCH_SIZE 2048 -> 4096, UBATCH_SIZE 1024 -> 2048.
Why the ubatch move is right on THIS card: the prefill curve is
card-specific. On the 2x R9700 layer-split sibling it peaks at ubatch 512
and every step up costs GiB for no speed; on this CUDA 3090 it climbs
monotonically to 2048 (710 -> 1188 t/s on the 35B MoE sweep, 1944 -> 2423
t/s on the Ornith 8k sweep) with decode flat to under 1 t/s on every rung.
The logical batch is free either way: two sweeps agree that at a fixed
ubatch, 2048/4096/8192 give identical prefill speed and identical peak
VRAM, because the compute buffers are sized by the ubatch alone.
The risk is all in the ubatch, and it is documented rather than assumed:
the step cost ~1,244 MiB of peak graph VRAM on the 35B MoE, the shape this
replaces is 2,815 MiB free, and the Q5_K_M predecessor recorded "ubatch
2048 failed the scratch boot" with MORE free than that. Compute buffers
are allocated at graph capture during boot, so a failure is loud and fast;
the step-down rungs (4096/1536, then 4096/1024) are named in the env.
The seam stops hard-coding a rung: serve_argv now carries BATCH_SIZE and
UBATCH_SIZE the way it already carried CTX_SIZE et al., and the pair test
asserts the argv follows the shipped env plus the llama.cpp invariant
ubatch <= batch. Verified both ways: retuning the env to 8192/4096 keeps
the seam green, inverting the pair to 2048/4096 turns it red.
rad-convert --plan-only with the deployed orcarouter FP8 payload:
'no architecture plugin claims qwen4exp at quantisation fp8; served at
(unquantised)'. Our quantization is reproducible from a bf16/abliterated
source by recipe (libr4d reads fp8 experts), not from our file. Image
pulled on dankserver, no GPU touched.
The prefix-caching OFF/ON experiment was confounded (both arms ran on
the marginal speed); the align-mode re-seed theory is unproven, not
disproven. Caching ships ON with zero gdn_hip faults under heavy
re-send traffic on stable RAM; knob stays commented. Corrects the
env's 'RAM RULED OUT' claim and adds the AGENTS.md lesson.
Frank: still corrupted at ~40k with --no-enable-prefix-caching deployed
and mechanically verified (zero gdn_hip faults in the container serving
that window, coherent 40,771-token boundary probe). The recurrence is
FAULT-FREE, so the align-mode page re-seed is not the (only) mechanism
and the mitigation bought whole-history re-prefill per turn for nothing.
Revert: knob FRANK_PRIMARY_NO_PREFIX_CACHING commented out -> the serve
branch renders --enable-prefix-caching (the shipped contract). The knob,
the branch and the seam stay documented for a future fixed fork build.
Open hypotheses recorded in the runbook: re-fed poisoned transcript (the
same conversation cannot distinguish fixed from unfixed), or a second
fault-free long-context path — fp8 KV at depth is the next suspect. Next
diagnostic: fresh-conversation reproduction, then a KV-dtype A/B.
Deploy b820d34, measured: --no-enable-prefix-caching in the engine argv,
health 200, warm boot 76.57 s (compile cache intact), expert plan
unchanged (17.77 GiB / 7,762 slots), GPU KV cache 424,678 tokens (up
from 401,408 at the same byte ask - pool now prices ~7,456 B/token with
the per-block mamba checkpoint pages gone; still clears the 400,700
floor), zero gdn_hip lines, and the 40,771-token boundary probe
(13 x 3136 = 40,768 crossed) decoded the 1-400 count coherently
(0.997 monotone, tail ...398 399 400). e2e 32/33 - the failure is the
admin spec's FrankBook-Pro leg (mini asleep), environmental.
Two bash defects in the 9d8b185 prefix-cache wiring, both measured live
on the 2026-10-05 deploy attempt (unit crash-looped 190x, exit 127,
endpoint down for the whole 3600 s health window):
1. A # comment block inside the backslash-continued podman run
terminates the continuation: the comment swallows the joined line
and the flag line then runs as its own command
('--no-enable-prefix-caching: command not found'). The comment moves
above the run block, with a no-comments-in-chain note.
2. The ${VAR:+A}${VAR:--B} double-expansion is ambiguous: when the
variable is set, ${VAR:--B} parses as a SUBSTRING OFFSET, not the
:- default — measured render was '--no-enable-prefix-caching1'.
The flag is now branched explicitly into
FRANK_PRIMARY_PREFIX_CACHE_FLAG (exactly one of the two flags
reaches the engine; unset keeps the historical flag).
Seam: the render test sources the shipped env through the branch and
asserts the exact flag word the engine receives (render-assert, the
same pattern the spec knobs use). bash -n clean; all 22 frank-primary
seams pass.
Measured 2026-10-04 18:46:07 PDT (live podman logs): 48 gdn_hip state
faults in one 0.6 s burst — 'state out of ssm-cache range: max|S|=0 with
non-finite values (slot=42 head=3|8)', one per TP rank, 23 GDN layers
each — then the process-lifetime report cap. The guard's own text in
this build names it: 'known Qwen3.8-Flash-Next (QFN) flaw under prefix
caching … Mitigation: --no-enable-prefix-caching'.
Mechanism: prefix caching puts QFN on mamba_cache_mode 'align', which
checkpoints each sequence's GDN recurrent state into a mamba page per
3136-token block and re-seeds it from that page when a request resumes
from a cached prefix. The NaN state landed at the 13th boundary
(13 x 3136 = 40,768 — the reported ~40k mark) and persisted in the page:
every later turn of that conversation re-seeded the same non-finite
state for two heads, silently (fault-report cap already spent).
RAM ruled out the same day: zero MCE/EDAC records since the boot under
test, 8 GiB in-band pattern stable under live load, 4800 MT/s is the
JEDEC floor on 5600-rated DIMMs, and the two-head single-slot NaN
signature is a state-path fault, not scattered bit corruption.
Fix: FRANK_PRIMARY_NO_PREFIX_CACHING=1 -> --no-enable-prefix-caching
(flag renders in both directions from the knob). Cost: every turn
re-prefills the whole history instead of at most 3135 tail tokens.
Seam updated: the prefix-cache pin now asserts the conditional render
and the shipped OFF value. Runbook carries the evidence + acceptance
criteria; all 368 tests pass.
The host region was the ONE arena-plan term still autofitting per boot: with
no cap the planner binds it to MemAvailable AT BOOT minus PLE/process/api/
margin terms (boot 2026-10-05: 237.97 GiB -> 159.39 GiB usable; the same host
reads 75.4 GiB available while running). Demand is 105.48 GiB
(52.74/rank x 2 ranks, 23,040 swap-target experts); a boot where co-tenant
RAM squeezed the bound near that silently pins next-hottest experts VRAM-only
(pinned_extra warning, smaller swap region, no flag change). 120 GiB =
measured demand + 14% headroom on 246.1 GiB total. Semantics verified in the
shipped image: cap binds as the bound, exceedance only warns.
The GPU-side arena is unchanged (deterministic arithmetic over the same
constants): verify after boot that host reads '--expert-offload-mem' and
slots stay 7,762. Seam: red first (wiring + value), then green; full suite
368+1 green.
Verified in the shipped image (v1/worker/gpu/cudagraph_mem.py):
estimate_cudagraph_memory() checks VLLM_CUDAGRAPH_MEM_ESTIMATE_MB FIRST and
returns that MiB total outright, ignoring the BASE/PER_GRAPH pair — the only
way to state the CUDA-graph plan term directly instead of deriving it. We
keep shipping the pair (0/12) so a future spec-decode shape re-prices
automatically; documented the switch condition.
Also corrected two stale shape-G claims in the runbook that predated the
1.00 revert: the deployed shape block and the lever table both still said
util 1.0 / 'le=1 ceiling', while shape G ships 0.982.
- new active runbook frank-small-llama-mimo-q6k-2x200704.md: 21,276 used /
2,849 true free (inside the 2-3 GiB floor), decode 94.44 t/s vs 104.84 on
Q5_K_M - the rung costs a measured -10 % (weight-bandwidth-bound card),
512px red-PNG vision verified, and the new probe-size law (>=256 px: a
10x10 image undersupplies image tokens and the model answers 'video'),
nvidia-smi free-field law (driver holds ~450 MiB the subtraction hides)
- frank-small-llama-mimo-q5km.md retired; it keeps the boot-pair table and
becomes the Q5 rollback reference (3,571 free, 104.84 t/s measured)
- AGENTS.md fleet row, endpoint section, --parallel lesson, runbook index
(active row + two retired rows); LLM + root index entries
- Q6_K GGUF-vs-GGUF diff proven: template and all 46 KV keys byte-identical
to Q5_K_M - the vision probe scare was probe size, not payload
The freed third slot funded exact V; the remaining ~3.5 GiB of honest
nvidia-smi free (3,571, not the 4,046 total-minus-used I quoted - the
driver holds ~450 MiB of reserve) funds the +875 MiB of Q6_K weights.
Projected landing ~2.7 GiB free, inside the 2-3 GiB floor; measure on
boot, step the pool one 12,288-token increment if it lands under 2 GiB.
- payload dir MiMo-Q6K/ made self-contained (mmproj hardlinked from
MiMo-Q5KM/, link-count-2 verified)
- env/serve/deployer/bats repointed; download script now fetches and
verifies Q6_K (7,793,710,624 B, ef96d05a...) with Q5_K_M documented as
the rollback; payload sha verified live on the host before deploy
- seam green 14/14 against the stub engine
- new active runbook frank-large-rocmfpx-q6agent-uncensored-mtp.md: the
Q6_0_ROCMFPX_AGENT file (ftype 114, 339 type-102 + 159 type-103 tensors),
27.6-28.1 t/s at 86.8% draft acceptance, the measured ngram-mod chain
rejection (-15% prose: recurrent-state rollback disabled), the
bench-window hardening laws
- frank-large-rocmfpx-q8agent-mtp.md retired; rollback section corrected
(gufo payloads removed from disk 2026-10-03 - that rollback is now a
re-download; the aligned Q8 file is the direct one, still on disk)
- frank-small-llama-mimo-q5km.md reshaped to 2 x 200704 with fully
unquantized f16 KV: boot-pair table gains the measured row (20,530 used /
4,046 free), 104.84 tok/s, slot-window contract at 200704, rollback to
the 3x180224 + q8_0-V shape documented
- AGENTS.md: fleet table rows, both endpoint sections, the chain law
extended with the recurrent-model measurement, the --parallel law with
the reslice, runbook index (active + retired)
- LLM/index.md + root index entries updated
Record the live numbers and the UTIL 1.0 verdict (unbootable: worker init
demands free > budget; ValueError on both ranks before a single weight loads).
Driver used 30.97 of 31.86 GiB; the 0.89 GiB/card margin held through a live
4,096-token generation at ~38-39 tok/s.
Worker init refuses budget >= device total on BOTH ranks:
ValueError: Free memory on device cuda:0 (31.29/31.86 GiB ...)
NRestarts=2 crash loop, zero weights loaded. Experts stay at the v1 shape
(graph term 0.07 GiB, KV 3.1665e9 = the 400,700-token ask); only the budget
returns under the law. Arena 17.78 GiB / ~7,912 slots predicted.
Two plan terms were still unspent: the CUDA-graph charge (shape F's own boot
reported a NEGATIVE capture delta, -0.06 GiB, so the 256 MiB base pool was
fiction -> charge 0 + 12 MiB/graph = 0.07 GiB) and the budget (util 0.982 -> 1.0,
the field's le=1 maximum). Together +0.82 GiB of expert cache: plan target
18.35 GiB / ~8,016 slots. KV_BYTES 3.15e9 -> 3.1665e9 = the 400,700-token ask at
the engine's measured 7,891.8 B/token. Cost of the force, recorded: driver
high-water ~30.4 GiB of 31.86, so ~1.5 GiB of real margin per card; rollback is
UTIL=0.982.
The arena is budget - non-expert - KV reserve - graphs - overhead. With util
capped at 0.982 (never 1.00) the graph estimate is the only plan term this repo
can shrink: the fork charges an analytic 256 MiB + 52 MiB/graph = 0.55 GiB where
the engine's own post-capture line reports 0.31 GiB, so pricing it at
256 + 12/graph returns ~0.24 GiB to the expert arena. MNBT 1024 -> 2048 is
arena-free: the registry overhead is base + 0.22 GiB per 1024 ABOVE its nbt 4096
anchor, and the fitted 5.25 GiB was fitted AT 4096.
Frank's order, two messages: "extend the max context of frank-small to
200700 per individual stream without further quantization", then
"reduce it to 2 streams for a total of 401400-ish tokens, set ubatch to
1k", then "unquantize the KV if we have extra space".
CTX_SIZE 540672 -> 401408 with PARALLEL 3 -> 2 gives n_ctx_slot 200,704:
above the 200,700 the harness compacts at, so a full-length request
lands in one stream instead of a 400 exceed_context_size_error. The pool
must divide evenly across slots and stay 256-aligned for the FA path,
which is why it is 401,408 and not 401,400.
CACHE_TYPE_V q8_0 -> f16: the freed third slot is exactly what pays for
unquantized values (24.5 -> 32 KiB/token), and the budget lands back in
the 2-3 GiB band Frank set on 2026-10-02 instead of leaving ~6 GiB of
the card idle. The asymmetric-KV contract that created this endpoint is
spent with it; both --cache-type flags stay explicit.
--batch-size/--ubatch-size are now passed (2048/1024). The endpoint had
never passed either, so it ran on engine defaults nobody wrote down.
Franksdank-Runbook: serverAppConfigs-docs/LLM/frank-small-llama-mimo-q5km.md
Frank's order: replace frank-large with
kingjones777/Qwen3.8-27B-Uncensored-ROCmFP4-STRIX-MTP-GGUF's
Qwen3.8-27B-Uncensored-Q6_0_ROCMFPX_AGENT.gguf, following that card.
The file is ftype 114 — 339 GGML_TYPE_Q6_0_ROCMFPX (102) + 159
GGML_TYPE_Q8_0_ROCMFPX (103) + 353 F32 over 851 tensors,
qwen35/64 layers/full_attention_interval 4/4 KV heads/262,144 window —
read out of its own header. The pinned ROCmFPX build already knows both
types (ggml.h 102/103 plus convert/cpy/getrows/mmq/fattn device paths
for 102), so no rebuild: the fork stays at 36e9acd and the payload
directory is the only new host state.
Card compliance, all already our flags: Q4_0 draft head (not Q8_0),
--spec-draft-n-max 4 (the knee on both cards), --spec-draft-ngl 99,
--jinja, --fit off, -fa on, -dio, one slot at full window, thinking with
the card's own sampling triple (which the file also embeds as its
general.sampling defaults). Kept from the deployed shape: the BF16
projector (this repo ships only a Q8_0 one; BF16 is the file verified
4/4 on spatial ground truth here) and the byte-identical MTP head, which
the new script hardlinks instead of re-fetching 1.6 GiB.
Franksdank-Runbook: serverAppConfigs-docs/LLM/frank-large-rocmfpx-q6agent-uncensored-mtp.md
Four harness gaps that block an honest MTP re-measurement:
- measure: AR_TEMPS sweeps sampling temperatures (default "0,0.7"). Draft
ACCEPTANCE is sampling-dependent — a greedy draft matches a greedy target
exactly and is rejected more under a non-greedy target — so judging a spec
arm only at temperature 0 judges half the contract. The unsuffixed keys stay
temperature 0, so every historical number remains comparable.
- window: capture an evidence.log sidecar (expert-plan lines, compile path,
graph sizes, /metrics spec-decode and cache counters, OOM signatures). The
arena/overhead/reserve numbers exist ONLY in the boot log; an arm that loses
decode speed is otherwise unexplainable afterwards.
- orchestrator: FRANK_PRIMARY_AR_ENV_FILE stages an env other than the repo's,
which is how arms A/B without mutating the checkout (the fingerprint covers
whatever is staged, so the memo can never answer one arm with another's
numbers), plus METRIC lines for the swept temperatures and the evidence dump.
- scripts/autoresearch/frank-primary-arms.sh: runs a sequence of full windows
(baseline, MTP3+probabilistic with a step-4 capture ladder at precise memory
ON, the same at the default overhead, and the vendor SEQS-16 shape), gating
each staged env on its JSON knobs still parsing AFTER sourcing — an unquoted
{"temperature":0.7} loses its quotes at source time and reaches the engine as
{temperature:0.7}, which is a 20-minute boot wasted.
Three engine knobs that were impossible to set without editing the run line:
FRANK_PRIMARY_SPEC (--speculative-config), FRANK_PRIMARY_COMPCONFIG
(--compilation-config) and FRANK_PRIMARY_GEN_CONFIG
(--override-generation-config). Each holds the whole JSON value; empty or
unset means the flag is absent, so the shipped shape E argv is unchanged.
The seam re-pins the law that justified the omission: "speculative decode is
ABSENT BY MEASUREMENT" compared the wrong pair — c1248d4 retired MTP by
weighing fp8+MTP4 (37.4-40.6 t/s, 54% acceptance) against the bf16 no-spec
boot (36.4-39.4), bf16 then proved unbootable, and fp8 came back WITHOUT MTP.
fp8+spec was never measured against fp8+no-spec on one shape, the no-spec
incumbent runs ~30 t/s under the fleet's 35 t/s target, and every vendor
flash-next recipe ships MTP3.
Also in the seam: a dead assertion is deleted (a failing `! grep` mid-test is
exempt from bats' errexit, so the pre-precise-memory `! grep
--expert-precise-memory` never ran), and the knob test asserts by RENDERING the
flag from the sourced env instead of grepping for an assignment.
Live after the 1ac4d48 deploy: 'expert_precise_memory': True, runtime overhead
5.25 GiB, expert cache space 17.22 GiB / 7522 slots, swap region 13.7 GiB /
5986 slots, GPU KV cache 398,658 tokens / 1.99x, pool 3,150,000,000, MNBT 1024
-- the probe's projection is now production.
Compile cache proven on consecutive boots of that shape: cold 'Compiling a
graph for compile range (1, 1024) takes 179.24 s' / 'torch.compile took
188.26 s'; warm 'Directly load the compiled graph(s) ... from the cache, took
2.143 s' / 'torch.compile took 3.97 s in total' / 'init engine ... took 41.13 s
(compilation: 3.98 s)'. Host dir 1.4 GB / 4,707 files. Time to /health stays
payload-bound (490 s warm), so the claim is ~184 s of inductor/AOT build per
restart, not a proportional boot cut.
Tests: 367/367.
Measured by boot probe on the shape-D base (production stopped, candidate
booted AS the service), plan numbers read off the engine's own log:
runtime overhead 7.60 -> 5.25 GiB (precise gfx1201 TP2 registry)
expert cache 14.87 -> 17.22 GiB
slots 6,496 -> 7,522 (+1,026, +16%)
swap region 13.7 GiB / 5,986 slots
boot healthy in 420 s, no OOM
'--expert-precise-memory': True also appears in non-default args, so a silent
argparse drop is visible. The pool is NOT cut (option A) because the pool is
not an arena lever: 3.15e9/rank keeps shape D's measured 1.99x concurrency,
and a 2.2e9 arm probed in the same window (277,686 tokens, 1.38x) is recorded
as the conservative fallback.
Boot-cost bug found while diffing Frank's 3xR9700 recipe: the image ships no
/root/.cache at all, our boot writes 1.1 GB of vLLM AOT/torch_compile
artifacts (4,665 files) plus 229 MB of comgr JIT kernels into the ephemeral
layer, and podman rm -f discards them — so every deploy/restart/reboot re-pays
'torch.compile took 181.20 s in total' (173.19 s of it one graph). Mounting
FRANK_PRIMARY_CACHE_DIR at /root/.cache persists both and hides nothing.
GPU_MAX_HW_QUEUES=2 added at Frank's direction (real HSA knob); not A/B
measured. Retracted my own recipe-diff claim that cudagraph_capture_sizes
needed pinning: capture is 3 PIECEWISE + 2 FULL graphs, 3 s, 0.31 GiB.
Tests: 367/367.
`deploy.sh <target>` with no `--service` also walks the per-service revision
markers and restarts everything behind HEAD whose paths changed — a drain, not
a sync. That is how the latent plex gate bug surfaced: the backlog fired all
at once and plex's http:// readiness probe, which can never succeed against a
PMS that drops plain HTTP off-loopback, burned its full 600 s and aborted the
run with every later service left pending.
Both facts recorded: AGENTS.md gains the drain law under Deployment, log.md
gains the plex entry with the evidence chain.
deploy_plex gated on `curl http://127.0.0.1:32400/identity`, which can never
succeed: Plex accepts the TCP connection (it arrives on the container's eth0)
and then answers with nothing, so curl reports "Empty reply from server" (52).
Measured 2026-10-02 on a plex that was healthy the whole time —
`https://127.0.0.1:32400/identity` returned 200 in 44 ms while the gate burned
its full 600 s and failed the deploy.
The repo already knew this law in another place: tests/caddyfile.bats pins the
plex vhosts to a TLS upstream with the comment "PMS drops plain HTTP
off-loopback". The deployer's gate never got the same treatment, and plex's
per-service revision had been behind long enough that the gate had not run
against this posture until now.
-k because PMS serves its own self-signed cert on 32400.
index.md intro and mermaid nodes still described 4 x 262144 fp8/bf16 and
frank-small at 2 x 262144; the active-deployment list had no frank-primary
entry at all. Fixed both, plus a malformed GLM list item.
log.md gains the shape-D entry (the SEQS=2 constraint, the arena-lever
measurement, the 2x2 probe table, the measured 1.99x), and the frank-small
entry's stale consequence bullet now points at the resolution.
frank-small runbook: the compaction-fit note is corrected - the 200704 window
moved frank-primary's trigger to 140,492, which fits a 180,224 slot again.
Comments/docs only - no functional change. The live boot of c769ff2 reported:
GPU KV cache size: 398,658 tokens,
Maximum concurrency for 200,704 tokens per request: 1.99x
expert cache space 14.87 GiB / 6,496 slots of 2.344 MiB
The arena prediction (6,496) landed exactly; the concurrency prediction (2.04x)
did not, because the per-token KV price is SHAPE-DEPENDENT: 7,701 B/token at
MAXLEN 262144 (6.1e9 -> 792,069 tokens) but 7,892 at 200704 (3.146e9 ->
398,658). Never extrapolate a concurrency figure across geometries.
Also smoke-verified: three concurrent 67-token requests answered in parallel,
~4.1 s each - the 3 slots are real, and the pool admits 2 full-length.
Re-applies the desired shape incrementally on the proven-good fp8 3x262144
base, after bisecting why the first attempt (30904ba) crash-looped.
Root cause: `--max-num-seqs 2` cannot compile on this model. AOT
build_guards(strict_error=True) inside profile_run -> _dummy_run specializes
`query_start_loc`'s width to SEQS + 1 (3 at SEQS 2, 4 at SEQS 3) and the
ConstraintViolationError kills engine init ~7 min in. MNBT is orthogonal:
SEQS 2 failed at MNBT 1024 AND 2048; SEQS 3 passed at both. KV dtype was a
red herring. Measured by four single-variable boot probes.
Arena lever measured: the expert arena is sized from `max_model_len` via the
registry-priced KV reserve (9190 B/token x max_model_len), NOT from
--kv-cache-memory-bytes. 262144 -> 14.30 GiB / 6,248 slots; 200704 ->
14.87 GiB / 6,496 slots. The 6.1e9 -> 3.15e9 pool cut bought zero arena and
is a runtime-headroom lever.
Shape: MAXLEN 200704, SEQS 3 (unchanged - the only slot count that compiles),
KV_BYTES 3.15e9/rank = 2.04x full-length concurrency at 7,701 B/token fp8,
MNBT 1024, gather 32x64, no spec decode, no prefetch, no precise-memory.
The 2 x 200704 fp8 / MNBT 1024 shape (30904ba) crash-loops at boot: the
engine dies in determine_available_memory -> profile_run -> _dummy_run with
the SAME torch.compile dynamo constraint that killed the bf16 2 x 262144
shape on 2026-10-01:
RuntimeError: Worker failed with error 'Constraints violated
(L['query_start_loc'].size()[0])! You marked it as dynamic but your code
specialized it to be a constant (3)'
Not an OOM. Restoring the last shape measured serving (fp8 3 x 262144 /
KV 6.1e9 / MNBT 2048 / arena 6,248) so the endpoint serves again; the
desired edits are re-applied one at a time from this base.
The 2026-10-01 bf16 shape (2 x 262144, --expert-precise-memory)
crash-looped at graph capture — the arena is DEVICE-resident and that
shape had 0.13 GiB to spare. The plain bf16 2 x 262144 also did not stay
up: 0 bytes free at runtime, and a real client request (995-token prompt,
max_tokens=261149) killed the worker with .
The correction is a pool cut, not a downgrade. FP8 KV (7,701 B/token),
two 200704 windows (401,408 tokens -> 3.091e9 B floor, pinned 3.15e9),
MNBT 1024. The fork's expert planner sizes the arena AFTER the KV
override, so the ~2.95 GiB/rank reclaimed becomes arena: ~7,500 slots
vs 6,248. Two contexts at 200704 is a capacity contract, not a
throughput one (2-way concurrency summed BELOW parity at 4064 ctx), so
the slot buys the VRAM headroom long generations need.
Refs [[dankserver-qwen38-fp8-expert-offload-tuning]].
- scripts/frank-primary.env: MAXLEN 200704, SEQS 2, KV_BYTES 3.15e9,
MNBT 1024, rewritten shape comment
- scripts/serve-frank-primary.sh: header comment updated
- tests/frank_primary_unit.bats: assertions updated, new seam pins the
arena-floor pool law (the pool must move with a context change or the
reduction buys nothing)
- docs + AGENTS.md: fleet topology and deployed shape corrected
Benchmark entrypoint: bash autoresearch.sh
Goal: Make frank-primary faster: raise decode tok/s on the tcclaviger vLLM endpoint (dankserver :7770, Qwen3.8-Flash-Next-Uncensored-FP8, 2x R9700, automatic tiered-expert offload). Context size may be altered (MAXLEN/SEQS/KV_BYTES and friends); quantization may NOT be altered (the 131-shard FP8 payload and the digest-pinned image are frozen).
Runbook, both indexes, AGENTS.md (topology row, endpoint section, three new
Key Lessons, one stale SGLang label in the bench list) and the bundle log now
describe the deployed shape: pool 540,672, 180,224 tokens per conversation,
f16 K + q8_0 V, mmproj vision, 21,684 used / 2,441 free of 24,576 MiB, 106.8
tok/s decode, 3,814 t/s prefill.
The claims that mattered:
- three contexts buy 180,224 tokens per conversation, not the native 262,144,
and a longer prompt is a hard 400 - so models.yml contextWindow is 180224
and frank-primary's 183,501-token compaction request no longer fits here
(forward pointer added to the retired SGLang runbook, which asserted it)
- the vision tower bills ~1.4 GiB for a 918 MB file, established by subtracting
two measured boots, not by a formula
- /v1/chat/completions ignores the legacy top-level 'images' field: the model
invents a color; only the image_url content part reaches it
Measured after the first 3-slot deploy: 3 x 196,608 booted (n_slots=3,
n_ctx_slot=196608, multimodal loaded) but used 22,972/24,576 MiB - 1,153 MiB
free, outside the 2-3 GiB band. The two measurements now recorded in
frank-small.env price the vision tower at ~1.4 GiB (non-KV 7,390 -> 8,859
MiB), so the pool drops 49,152 tokens (16,384/slot, 1,176 MiB) to land ~2.7
GiB free. The bats budget gate is recalibrated to the measured 9 GiB non-KV
cost, which now rejects the 3 x 196,608 shape instead of blessing it.
CTX_SIZE 589824 / PARALLEL 3 -> three independent 196,608-token conversations
(a mamba hybrid cannot share one KV cache, so per-slot window = pool/slots;
3 x 262,144 would need 18.4 GiB of KV alone and busts the card).
Vision comes back via bartowski's mmproj-MiMo-V2.6-Distill-Qwen-9B-f16.gguf
(918,166,048 B, sha256 ff348f31..., fetched + verified by
scripts/download-frank-small.sh): the Q5_K_M GGUF ships no v.* tensors, so
--mmproj IS the image path. MMPROJ_FILE without a colon default so an explicit
empty value is a real text-only opt-out, and the deployer now gates on the
projector exactly as the serve script does.
Budget is measured, not guessed: the 2 x 262,144 shape used 20,190/24,576 MiB,
which prices non-KV cost at 7,390 MiB. Three slots + projector land near
22.5 GiB, i.e. ~2 GiB spare - the bottom of the requested band, to be
confirmed by measurement and laddered if it lands outside 2-3 GiB.
Staged 59 GiB of retired LLM payloads on Dankclaw to
~dank/.AGENTTRASH/20261002-retired-llm-payloads/ (vLLM MiMo BF16, Qwen3.5-9B
MTP source, both DFlash dirs, Gemma4-12B-QAT, SGLang MiMo INT8) and corrected
every doc claim about what is on disk: gufo's rollback pair (Q4_K_XL weights +
the Q8_K_XL DFlash2 drafter) and frank-small's Q6_K stay in place, so the
rollback paths in the tags' env files still resolve. The vLLM-era frank-large
fetch trio moves to old/ with its runtime; runbook links updated.
frank-large: new runbook frank-large-rocmfpx-q8agent-mtp.md — the ftype-115
tensor-type lock-in that forced the fork, the pinned build (set_rows history),
the measured MTP A/B (17.0 vs 7.6 t/s decode, acceptance 0.462), the live
client contract (reasoning_content, effort fold, vision restored) and rollback.
frank-small: new runbook frank-small-llama-mimo-q5km.md; q6k marked retired
(rollback target). gufo runbook marked retired (rollback target). Both indexes,
the fleet topology table, the vision/models.yml notes, the benchmarking law and
three new Key Lessons updated.
The repo's highest-quality rung is ftype 115: 311 tensors of
GGML_TYPE_Q8_0_ROCMFPX, a type only the ROCmFPX fork reads, so the runtime
moves from gufo to scripts/build-rocmfpx.sh's pinned fork build (36e9acd) —
the first build on this host measured to load this file without the
2026-08-25 set_rows segfault. One session at the native 262,144 window, f16
KV (64 KiB/token, 16 GiB), the repo's Q4_0 MTP head drafting 4 tokens, plus
the bundled BF16 projector so the endpoint has vision again.
The serve script now honors caller environment overrides over the env file
(a plain source let the file win, which made single-run overrides no-ops) and
defaults apply only when a value is unset, so an explicit LLAMA_BIN='' is the
error it reads as.
One quant step down from Q6_K on the same CUDA build and the same shape:
2 x 262,144 with K f16 + V q8_0. The 0.86 GiB freed from the weights is
headroom, not a new window — the native 262,144 cap is already reached,
and f16 V would still cost 3.96 GiB more than the pool can carry.
Live battery green after deploy #3: alias, reasoning split into
`reasoning`, tool_calls, /metrics acceptance pos0 69% pos1 44% - the
cross-model MTP head performs as measured in the bench window.
The first crash-looping boot (51 restarts): serve received the host path
/home/dank/models/... but the models tree is mounted at /models inside the
container, so transformers failed 'Repo id must be in the form
namespace/repo_name' before the engine even built. Same conversion the spec
draft already used. Pinned by bats. Root docs index links the new runbook
(seam required it) and the work log records the crash-loop plus the 52 G
payload reclaim.
The 2026-10-01 first deploy refused /home/dank/models/Qwen3.5-9B because
the handler checked for a single model.safetensors (the dflash layout);
the Qwen3.5-9B MTP head ships 4 shards behind model.safetensors.index.json.
Same dual-layout acceptance as the serve script, now mirrored and gated by
the seam. The gate fired before the stop, so the live gufo unit was never
touched (health 200 throughout).
New runbook carries the roof math, the measured three-arm spec table, the
greedy-text numerics trap, the reasoning-field contract change and the
rollback tag. LLM index and AGENTS.md fleet table, bench policy and runbook
index follow; the frank-small index entries catch up to the SGLang cutover
(entry was missed on 2026-10-01). Work log records the cutover.
Engine: digest-pinned docker.io/kyuz0/vllm-therock-gfx1151 (vLLM 0.30,
TheRock ROCm 10, Triton GDN kernels); the serve script gates the image by
REPO DIGEST, the payload (config + index + exact shard count), the spec
draft (BOTH weight layouts: single model.safetensors for dflash, sharded
index for the Qwen3.5-9B MTP head), and whole /dev/dri, sweeps the bench
identity vllm-mimo, then exec podman wait so a container death trips
Restart=always. Shape: 4 x 262144 at util 0.45 (KV grows only on the 8
full-attention layers), MTP 2 draft tokens.
Measured 2026-10-01 window (frank-large stopped): base 13.1 t/s decode
(~94% of the LPDDR5x BF16 roof), DFlash-8 17.4, MTP-2 19.8 with 71%/44%
per-position acceptance - the cross-model head works because vLLM reads the
draft from a separate path while MiMo ships zero mtp.* tensors. Spec arms
do not preserve greedy bf16 text (batched-verify reduction order).
The seam was rewritten around the new laws (18 tests): unit self-healing,
digest gating, engine wiring, the measured spec table, the retired gufo
gates, the deployer MainPID check. frank_small port test follows the
PORT -> FRANK_LARGE_PUBLISH rename. Rollback: tag frank-large-gufo-q4kxl-20261001.
All three verify every file by byte count + Hub LFS sha256, use -f (a bare
curl -o reports success on a 403 error body), support --verify-only, and
never put a token on the command line. Payloads are on Dankclaw and verified
against these manifests.
bf16 at 2 x 262144 fails with and without --expert-precise-memory. The real
error (captured from the journal this time, container fixed via journald
rather than podman logs):
RuntimeError: Worker failed with error 'Constraints violated
(L['query_start_loc'].size()[0])! You marked it as dynamic but your code
specialized it to be a constant (3)'
- a torch.compile/dynamo dynamic-shape violation, NOT an OOM. Ruled out a
stale compile cache: ~/.cache/vllm holds only modelinfos (16K) and each
podman run gets a fresh container filesystem.
The deltas from the last shape measured serving (fp8 3 x 262144 / MNBT 2048 /
792,069 tokens / 3.02x / arena 6,248) are KV dtype (fp8 -> bf16) and SEQS
(3 -> 2). Restoring the known-good shape; bisect bf16 vs SEQS in a window,
with --enforce-eager as the bypass.
NRestarts climbed to 3, each attempt ~6.7 min then 'Deactivated successfully'
(API server drains after an EngineCore fatal error; Restart=always turned the
silent death into a visible loop). The arena is device-resident, so the budget
decides it: weights 6.28 + pool 6.80 (bf16 2 x 262144) + graphs 0.55 + arena
16.65 (precise overhead 5.25) + non-torch 1.45 = 31.73 of 31.86 GiB -> ~0.13
GiB free at boot, and graph capture is what allocates transiently.
Dropping the flag restores the 7.60 GiB default overhead: arena 14.31 GiB,
total ~29.4 GiB, ~2.4 GiB free. bf16 2 x 262144 (Frank's KV change) stays.
The serve script keeps the opt-in flag; the seam now asserts it is ABSENT.
Live-verified post-deploy: context_len=262144, max_running_requests=2,
max_total_num_tokens=606208 in the boot log; a 200,000-token prompt
(compaction height is 183,501) prefilled+completed in one request
(97.3 s, ~2,056 tok/s eager); two concurrent 50k requests answered.
Docs-only change.
Frank's call. Payload: ajgazin's abliterated Swift-1.5 UD-Q8_K_XL (29.3 GiB,
SWIFT OPEN LICENSE) -> unsloth/Qwen3.8-27B-GGUF UD-Q4_K_XL (16.35 GiB,
sha256 3f227079...b01e, vanilla/Apache-2.0). NOTE: the endpoint is no longer
uncensored — that was one of the reasons for the Swift payload on 2026-09-28.
New fetcher scripts/download-frank-large-q4.sh (no auth header: repo is public
and ungated; size + sha256 verified after transfer). The speculative drafter
and its fetcher are unchanged.
Shape: --sessions 4 --context 163840 with unquantized KV. gufo exposes no KV
quantization lever (execution_policy.hpp pins kv_cache_storage to kFp16), so
f16 is inherent.
KV arithmetic: f16 KV = 64 KiB/token (16 full-attention layers x 4 KV heads x
256 head_dim x 2 x 2B; the other 48 layers are linear-attention with a
per-sequence fp32 state). 4 x 163840 = 655,360 tokens = 40.0 GiB KV, each
session exactly 10 GiB. Budget ~62 GiB of the 114 GiB GTT.
Seam retargeted (FETCH -> the q4 script, TARGET_STUB, context/sessions pins).
The KV pool stays pinned at 606,208 tokens (card is full: 531 MiB free at
the old shape; autofit's 687,388 would leave 0.53 GB and OOM on the first
long request). 2 x 262,144 = 524,288 leaves ~82k tokens of pool slack.
Motivation: harness compaction. frank-primary's contextWindow 262,144
compacts at 70% = 183,501 tokens, and the compaction request carries a
32,768-token summary budget -> 216,269 must fit ONE frank-small request.
The 151,552 slot window could not hold it; the native window does.
No spec-decode interaction: checkpoint ships zero mtp.* tensors (config
lies), seam keeps every --speculative-* flag away from the engine.
Both of Frank's changes at once.
KV is bf16 (--kv-cache-dtype auto) at two full-size windows (2 x 262144).
bf16 measures 13,839 B/token (bf16 3x241664 boot: 11,047,010,304 B -> 798,280
tokens), decomposing as 12,276 attention + 1,563 mamba/side caches vs fp8's
6,138 + 1,563 = 7,701. 524,288 tokens needs 7.256e9 B; pinned at 7.3e9
(~527,500 tokens, 3,206 margin). Pool cost: 5.68 -> 6.76 GiB per rank.
--expert-precise-memory selects the fitted overhead (5.25 GiB) over the default
(7.60 GiB = fitted + compile/graph-capture headroom). It moves only the plan's
term, so the expert arena goes 6,248 -> ~7,278 slots (~1,025 more) at the
cost of boot-time capture slack. Overhead adds 0.22 GiB/1024 nbt above 4096, so
MNBT 2048 pays nothing.
Seam + docs + AGENTS row + log updated.
The gate failed twice in the same spec for two different reasons and then passed
33/33 on a manual re-run. Future sessions should re-run before treating the
DEPLOY INCOMPLETE banner as a real fleet fault.
ExecStop runs 'podman stop -t 120' but the unit never set TimeoutStopSec, so
systemd's 90 s default cut the graceful stop short: journal shows 'Stopping
timed out. Terminating.' then 'Failed with result timeout' on 2026-10-01
13:27:17 — every stop of this unit was logging a failed stop. 180 s covers the
engine's drain with margin, and the seam now guards the RELATIONSHIP
(TimeoutStopSec > the podman grace) rather than a literal.
StartLimitIntervalSec is a [Unit] directive; the copy in [Service] made systemd
log 'Unknown key StartLimitIntervalSec in section Service' on every load. Seam
asserts it appears exactly once and not in [Service].
Frank's call 2026-10-01: MAXLEN 262144 x SEQS 3 = 786,432 tokens of contract.
Measured fp8 KV costs 7,698 B/token (4-slot boot 8,311,013,376 B -> 1,079,582
tokens; 5-slot boot 10,388,766,720 B -> 1,350,182 tokens), so the pool floor
for the contract is 6.054e9 B. Pinned at 6,100,000,000 (~792k tokens, ~5.9k
margin) so the remaining ~2.2 GiB versus the 4-slot shape goes to the expert
ARENA - the fork sizes its decode-critical expert cache from the remainder
after weights, pool and workspace.
MNBT 2048 (Frank): the arena sweep measured 6,499 slots at every rung <= 4096,
so the larger batch costs the arena nothing.
Seam retargeted (3 seqs / 6.1e9 / MNBT 2048); docs + AGENTS row + log updated,
including the closed DCP investigation.
--decode-context-parallel-size 2 fails VllmConfig validation at boot:
'Decode context parallelism for GQA/MQA requires --tensor-parallel-size (2) to
be greater than the model's total number of KV heads (2)'. DCP shards KV along
the sequence dimension AFTER KV heads spread across TP ranks, so it needs
TP > num_kv_heads; this model has 2 KV heads, so TP >= 4 — four GPUs, not two.
Recorded with the rest of the survey (PP/DP/EP/multi-node verdicts for this
box), the container-injection requirement for /home/dank/mtp_measure.py
(it imports /bench_iter.py by absolute path and must be copied in), and the
re-measured deployed capacity (1,079,582 tokens / 4.12x).
DCP is passed only when FRANK_PRIMARY_DCP is set, so the deployed shape is
unchanged. Verified in this fork: DCP reuses the TP ranks when PCP is
disabled (tp % dcp == 0), so TP=2 + DCP=2 fits the two R9700s with no extra
ranks; the flash_attn backend carries DCP support, the 36 linear-attention
layers of this hybrid do not — which is what the A/B window tests.
Replaces llama.cpp Gemma4-12B-QAT (payload + binary retained as rollback).
SGLang in a self-contained python3.12 venv with an extracted CUDA 13.0
nvcc toolchain for the sgl-kernel/flashinfer JIT.
Shape: fp8 KV, pinned 606,208-token pool / 4 x 151,552 slots, bf16 mamba
states pinned at 32 slots (the engine autofills the ssm pool to a fixed
~5.1 GB regardless of dtype), eager prefill (no headroom for graph
capture), no spec decode (the checkpoint ships no mtp.* tensors despite
mtp_num_hidden_layers=1 — NEXTN drafts with an uninitialised 4.24 GiB
ghost at 0% acceptance).
Verified: 83.9 tok/s decode, 170k single prompt, 4 concurrent 54k
requests, vision (in-checkpoint tower), reasoning_content field.
Five 262144-token contexts passed every boot gate (KV 1,350,182 / 5.15x) but
left 0 bytes free at runtime: a real client request (995-token prompt,
max_tokens=261149, sampling params straight from models.yml) killed Worker_TP1
with torch.OutOfMemoryError 'Tried to allocate 50.00 MiB ... 0 bytes free'.
Boot is not proof of runtime survivability.
Four slots x 262144 fp8 (KV_BYTES 8,311,013,376/rank) returns ~1.93 GiB per
rank for activations/graph capture.
Unit: Restart=always instead of on-failure — a runtime worker fault makes
vLLM's API server exit clean, so ExecStart returned 0 and on-failure never
fired, leaving the endpoint down until a human noticed. Also
StartLimitIntervalSec=0 and OOMScoreAdjust=-500.
Frank's call after the night ladder: five contexts at the model's full native
262,144 window beat six shortened ones. Six full-length contexts do NOT fit at
any KV dtype (6 x 241664 fp8 asked 10.70 GiB against 9.18 GiB free — the
6-seq graph/activation footprint leaves too little for the pool); 5 x 262144
at fp8 PASSED on the first rung: KV_BYTES 10,388,766,720/rank, measured
'GPU KV cache size: 1,350,182 tokens, Maximum concurrency for 262,144 tokens
per request: 5.15x' (39,462 tokens of margin).
Also documented: the SEQS=3 bf16 max (241,664, KV 732,359 / 3.03x) measured
earlier the same night, and the root-cause read that the 7.2 GiB KV ceiling
law was the MTP4 workspace, not VRAM. Seam and harness (contextWindow 262144)
re-pinned.
Night ladder result (Frank's subtract-16k method): 262144 and 245760 both OOM
at CUDA graph capture ('Tried to allocate 76.00 MiB ... 0 bytes free'), 237568
passed (KV 718,606 tokens), and the +4096 bonus step PASSED -> 241664 is max
possible and DEPLOYED at util 0.98 (KV 732,359 tokens / 3.03x concurrency at
241,664-token requests). Util stays 0.98: lowering it shrinks the engine's own
budget instead of buying slack.
Client contract restored (the GLM rollback d91379d dropped all three and the
qwen rollback inherited the gap): --reasoning-parser qwen3 fixes the malformed
thinking block (thinking was landing inline in content with a dangling
</think> and a null reasoning field), --enable-auto-tool-choice with
--tool-call-parser qwen3_xml makes tools work, and
--default-chat-template-kwargs enables enable_thinking + preserve_thinking.
Harness: contextWindow 241664 (compaction gap recomputed), preserve_thinking
moved into chat_template_kwargs.
The 0.98 boot died at KV allocation: torch OOM tried 10.28 GiB (the exact
3x262144 bf16 pool = 11,047,010,304 bytes) with 9.46 GiB free per rank.
The rootless bench probe had booted the same numbers; the root unit's
pinned-expert path holds ~0.8 GiB/rank more. Util 0.90 returns ~2.5 GiB/rank
of workspace with the pool ask unchanged; measured KV capacity law
(798,280 tokens at 11.05e9) unchanged.
Overnight redo campaign verdict + Frank's call: the fp8+MTP4 shape is retired
to git tag frank-primary-fp8-mtp-20261001. The new incumbent: KV cache
unquantized (auto), no speculative decode, KV_BYTES 11,047,010,304/rank
(engine-measured 798,280 tokens, 3.05x concurrency at 262,144-token
requests), MNBT 1024, MoE gather 32x64 (+18.7% decode, grows with depth).
Measured at the deployed shape: 39.38/38.79/36.42 tok/s (166/1087/4064 ctx).
Seam re-pinned: the 7.2 GiB/rank KV ceiling was the MTP4 workspace, not VRAM;
spec decode absent by measurement; bf16 trade documented (clean cache +
262k windows for ~14% decode at 4064 ctx).
- AGENTS.md: fleet table + frank-primary section back to qwen FP8 + MTP4
(37.4-40.6 tok/s, 54% acceptance); GLM runbook moved to the retired index.
- runbook: RETIRED verdict — DFlash2 drafter served and drafted but
acceptance pinned 1.00-1.38 on every text class including verbatim
repetition (mis-wired aux capture, not a weak drafter); 5x slower than
baseline at ~1.2 acceptance.
- log.md: 2026-10-01 rollback entry.
- deploy-remote.sh: LLM_PRIMARY_CONTAINER back to frank-primary-qwen;
glm-probe joins the legacy sweep list.
- frank_primary_unit.bats: re-pinned to the qwen shape — KV qualification
law (7.2 GiB/rank, 3.65x concurrency; 9/16 GiB OOM), fp8 q8/q8 KV
PRESENT, MTP4 PRESENT (3.1-3.4x over the 12.11 no-spec baseline), no tool
dialect, automatic offload. The old 'MTP buys zero' law carried the
prefetch tax and is retired.
DFlash2 experiment declared failed 2026-10-01: the drafter serves and
drafts (12.8 tok/s proposed) but acceptance is pinned at 1.0-1.38 on ALL
text classes including verbatim repetition — the aux-hidden-state capture
semantics are wrong (single-stream entering-state vs post-layer/comb is
the untested alternative). At ~1.2 acceptance the 8-token verify tax makes
decode 5x SLOWER than the 9.9-10.0 baseline (1.6-1.8 tok/s measured).
Restores the verified 37-40 tok/s shape: qwen3.8-flash-next FP8, MTP4,
prefetch off, 200k x 3 seqs, fp8 KV.
use_aux_hidden_state_outputs lives on the gpu_model_runner instance
(gpu_model_runner.py:623/678), not on the spec config, so the first
registry check never fired and the reserve stayed at 13,600 B/token.
Switch to method in {dflash, dspark, extract_hidden_states}.
The dflash boot cleared the unifier and the kpool tail (02/03) and died on
KV capacity: on the generic path the mamba states are NOT aliased into the
MLA pages (the glm5 tensor layout's whole point), so the capacity estimator
demanded ~30.6 GiB fixed against 0.92 GiB available (the plan's KV reserve
registry only knows the target's 13,600 B/token).
Patch 04 keeps the aliased glm5 pool and appends the drafter's uniform
group and per-layer hidden-state groups beside it: grouping partitions
hidden + draft specs, the layout returns a 10-tuple, the builder sizes one
shared backing allocation (pool + extras), and the memory estimator adds
the extras. Patch 04b extends the GLM registry kv_bytes_per_token by the
drafter (4,096 B/token) and hidden-state (5 x hidden x 2 B/token) costs so
the expert plan reserves enough.
Local smoke tests (CPU torch + real spec classes) validate grouping shape,
layout recognition, builder block math and memory accounting against
synthetic GLM+draft+hidden spec sets. Apply-verified: reproduces shas
5521e50b... / 1a623d5c...
Patch 02 (MLA-multiple max page) cleared the KV page-size unifier: the
dflash boot now reaches cudagraph capture. New blocker: the kpool decode
kernel asserts tail_kv_cache.shape[2] == pool_size; KpoolTailSpec pins
block_size == index_kpool, so the unifier's divisible branch must never
grow it. Patch 03 = 02 + a KpoolTailSpec branch that pads page_size_padded
instead. Apply-verified: reproduces patched sha 5093e4b7... Runbook
updated with the full KV grouping journey.
Runbook: the five-version evolution recorded, each trap named (method
hand-write, Protocol-base MRO kill, wrong-wrapper attrs). The decisive
discovery: the runner checks the LOADED instance, which for GLM-5.3-Flash
is the multimodal Glm5NextForConditionalGeneration holding the text model
as self.language_model — v4's attrs on Glm5NextForCausalLM were invisible
to it. v5 (attrs + methods on both wrappers) passes: both TP workers log
'Using Eagle3 auxiliary layers from config: (6, 15, 25, 34, ...)' and the
DFlash2 drafter resolves + loads. Log: the DFlash bullet updated from
'the trunk simply does not implement' to 'gap closed; serving end state
still unproven (window RAM contention interrupted the boot twice)'.
The v4 boot named the actual miss: model_runner.py:425 calls
set_eagle3_aux_hidden_state_layers(self.model) on the instance model_runner
LOADED -- and for GLM-5.3-Flash that is Glm5NextForConditionalGeneration (the
multimodal wrapper; APIServer log: 'Resolved architecture:
Glm5NextForConditionalGeneration'), which wraps the text model as
self.language_model = Glm5NextForCausalLM (glm5next/common/model.py:1113).
v4 put the three presence-checked attributes only on Glm5NextForCausalLM,
so the isinstance check on the conditional still failed with the same
RuntimeError.
v5 adds to BOTH wrappers, MRO-safely (attribute route, no Protocol base):
- Glm5NextForConditionalGeneration: supports_eagle3 / has_own_lm_head /
has_own_embed_tokens + the two methods delegating to self.language_model
(the pixtral.py:498 pattern);
- Glm5NextForCausalLM: the same three attributes + the two methods that
unwrap to self.model (the EagleModelMixin trunk -- v1's correct bodies).
Also proven by MRO simulation: the ForCausalLM's five-base ordering cannot
take SupportsEagle3 as a base in ANY position, so inheritance was never a
viable route for this model.
base 008eeaa2... -> patched bc9913a1..., apply-verified against the pinned
image (patch -p1 reproduces the sha256 and matches the mounted file).
The v4 window boot named the real presence-check rule: isinstance against a
runtime_checkable Protocol walks __protocol_attrs__ (which includes members
inherited from BASE protocols) and getattr_static()s each one off the instance,
breaking on the first missing attribute. SupportsEagleBase declares
has_own_lm_head/has_own_embed_tokens as plain instance-var annotations, so
v3's bare supports_eagle3 flag still failed. Replicated in-image with stock
typing: flag-only class -> False; flag + the two base members -> True.
v4 declares all three on Glm5NextForCausalLM:
supports_eagle3: ClassVar[Literal[True]] = True
has_own_lm_head: bool = False
has_own_embed_tokens: bool = False
(False is correct for the two: the wrapper delegates embed/lm_head to its
trunk. The MRO-safe attribute path is kept from v3.)
base 008eeaa2... -> patched 60f27aad..., apply-verified against the pinned
image (patch -p1 reproduces the sha256 and matches the mounted file).
v2 (inheriting SupportsEagle3 as a base) taught the real lesson: the isinstance
check PASSES and the class then dies at creation with
'Cannot create a consistent method resolution order (MRO) for bases ...'
-- GLM's outer wrapper already inherits vLLM Protocols (SupportsPP,
MixtureOfExperts, HasInnerState, IsHybrid) whose ordering is incompatible with
appending another Protocol base.
v3 avoids the MRO entirely: declare
supports_eagle3: ClassVar[Literal[True]] = True
as a class attribute on Glm5NextForCausalLM. Runtime-checkable Protocols do
presence-based isinstance, so the bare flag satisfies supports_eagle3(model)
without touching the MRO. The aux-layer plumbing stays on the trunk via
EagleModelMixin.
base 008eeaa2... -> patched 4c9a632b..., apply-verified against the pinned
image (patch -p1 reproduces the sha256 and matches the mounted dflash-patch
file byte for byte).
v1 hand-wrote set_aux_hidden_state_layers/get_eagle3_default_aux_hidden_state_layers
and the boot reproduced the identical 'Model does not support EAGLE3 interface'.
Reason: supports_eagle3() is an isinstance check against a runtime-checkable
Protocol whose members include a required ClassVar flag
(interfaces.py:1663 supports_eagle3: ClassVar[Literal[True]] = True), so methods
alone do not satisfy it. Inheriting SupportsEagle3 -- the idiom every model in
this image uses, including the fork's own DFlashLagunaForCausalLM -- supplies the
flag, SupportsEagleBase's has_own_lm_head/has_own_embed_tokens, and working
default implementations of both methods; the hand-written methods are deleted.
base 008eeaa2... -> patched 5eea89c5..., apply-verified against the pinned image.
Runbook: corrected method name, recorded the Protocol trap as the generalizable
lesson for future spec-decode ports.
The frank-primary section still carried the retracted framing ('a latent bug the compile
exposes' in the mHC/KDA path) and listed only three compile patches. It now records:
six apply-verified artifacts across two WIP changelists (compile 01-05 + the DFlash2
enabling patch), the measured facts (lru_cache does not protect a probe; suppress_errors
does not cover breaks; the trunk can never satisfy fullgraph because its indexer
metadata syncs to host), the single remaining blocker
(KeyError: 'in_proj_qkvbfg_a', raised in a callee dynamo inlines into
Glm5NextDecoderLayer.forward, not at any of the four nn_modules lookups and not a literal
string lookup -> currently a dynamo-side name-resolution failure, not a demonstrated fork
bug), and the two operational gotchas this session paid for (the launcher silently
dropping VLLM_COMPILE_FULLGRAPH=0, and the PYTHONPATH sitecustomize fork bomb). The
DFlash2 paragraph now says the aux-hidden-state contract is written, and that the
engine's refusal was literally 'Model does not support EAGLE3 interface'.
tests/run.sh: 362 seams pass.
Clarifies scope in the artifact header: torch.compiler.disable is itself a graph break,
so 05 helps only a hypothetical fullgraph=True route (where the workspace acquisition
would be better restructured out of the forward than made opaque). It is kept because
the discovery is real and the wrapper is a no-op when nothing is compiling, but it is
not required for the break-tolerant route whose only confirmed blocker is the dynamo
name-map KeyError.
Also verified in this pass: glm53-dflash-01-aux-hidden-states.patch applies cleanly to
the pinned image and reproduces its recorded patched sha256 (4ff6ff8d...), so all six
artifacts in scripts/patches are now apply-verified.
Recording the incident: a diagnostic sitecustomize.py mounted at /probe on PYTHONPATH
produced a self-replicating chain of python3 processes 19+ levels deep inside the probe
container (sitecustomize runs in EVERY interpreter; the hook imported torch/vLLM at
startup; something in that import chain spawned a helper python which inherited
PYTHONPATH and the hook, and spawned again). Found alive 18 minutes later, 33 probe-tagged
processes. Killed with 'podman rm -f glm-probe' (the container cgroup owns every
generation) and verified by counting /proc/<pid>/environ for the marker variable --
never by pattern-killing an interpreter name, since this host runs python in production
containers (romm, jellyfin, plex, glances). Fixed at the source with an env marker that
children inherit, verified with a two-generation test: gen 1 installs, gen 2 stays inert.
The runbook now carries the hazard, the guard, the kill recipe, the conjunction the
restore must verify, and the no-overlapping-jobs rule. Patch 06
(glm53-dflash-01-aux-hidden-states.patch) is documented in the DFlash2 section, and the
/probe diagnostic is marked retired -- TORCH_LOGS answers dynamo questions with no
startup hook.
tests/run.sh: 362 seams pass.
glm53-dflash-01-aux-hidden-states.patch: implements the interface whose absence makes
the engine refuse the drafter with 'Model does not support EAGLE3 interface'. Adds
EagleModelMixin to Glm5NextModel, captures the single-stream hidden state entering each
requested layer in the loop, returns (hidden_states, aux_hidden_states) as a tuple when
the list is non-empty, and exposes set_aux_hidden_state_layers /
get_eagle3_default_aux_hidden_state_layers on Glm5NextForCausalLM -- the contract read
out of interfaces.py:1694/1712, eagle3_utils.py:26-31, deepseek_v2.py:1551-1585 and
cudagraph_utils.py:621, with kimi_k25/pixtral as the delegation reference.
Independent of the compile changelist (01-05); it patches the same file as 01, so 01
goes first if both are applied. Scope is honest in the header: this makes the interface
present (the prerequisite for driving DFlash2), it does not promise draft quality and is
not a decode-speed fix.
Five runs of the changelist now reproduce the same single failure with breaks genuinely
tolerated (patch 01+02+03+04, VLLM_COMPILE_FULLGRAPH=0, VLLM_USE_AOT_COMPILE=0,
R4D_GLM_FUSE_QKV_RMSNORM=0): the trunk TRACES ('Dynamo bytecode transform time: 1.08 s')
and the boot dies on KeyError: 'in_proj_qkvbfg_a'.
What it is NOT, each ruled out by measurement rather than argument:
- not one of dynamo's four nn_modules lookups: /home/dank/dynamo-probe wraps three of
them and stayed silent over two runs, with the probe's install line confirmed in both
the API-server and worker processes;
- not a dynamo-internal failure: TORCHDYNAMO_VERBOSE=1 adds no internal stack trace;
- not a literal string lookup: the name appears tree-wide only as the KDA layer's
attribute (kda.py:217, used at :407) and in the load_weights shard table
(model.py:780-785); the four sites in the
quantization utils do compute exactly this bare key but every one is guarded
(), so none can raise it;
- not the pluggable-layer registry, which is keyed by class name.
What it IS: a plain Python KeyError raised inside the traced
Glm5NextDecoderLayer.forward, in a callee dynamo INLINED (frame chain
profile_run -> _dummy_run -> execute_model -> glm4_1v.py:2358 -> the compile wrapper ->
eval_frame.compile_wrapper -> Glm5NextModel.forward:748 -> module._call_impl ->
Glm5NextDecoderLayer.forward:456; the reported line is that method's def line, which is
how inlined frames surface).
Patch 03's header and the runbook now record the remaining instrument (TORCH_LOGS=+dynamo
and grep, or wrap the inlining candidates — the mHC CustomOps and the indexer op — with a
KeyError probe), and the two breaks that are real but non-fatal in this configuration
(patch 04's fused-norm launcher cell, patch 05's workspace acquisition).
tests/run.sh: 362 seams pass.
This engine ignores SIGTERM: after systemctl stop its container sits in podman ps
state 'Stopping' for 20+ minutes WHILE STILL ANSWERING /health with 200. So a liveness
probe is a lying oracle for a window restore — twice a restore reported PROD HEALTHY
while systemctl is-active said failed and the runtime mask was still in place, because
the old container was still serving.
Records the conjunction the restore must verify (is-active == active AND is-enabled ==
enabled AND no /run/systemd/system/<unit> mask symlink AND the container is owned by
the unit and not Stopping) and the four-step repair recipe (mv the runtime mask away —
it survives stop and blocks start; daemon-reload; reset-failed — a failed unit refuses
to start; podman rm -f — podman stop hangs on the same SIGTERM).
Also records the overlap hazard: a later job's 'podman rm -f glm-probe' cleanup
deleted an in-flight job's container, so one compile round produced no evidence at all
('no container with name glm-probe found'). Check proc:// for running jobs before
opening a window.
tests/run.sh: 362 seams pass.
Root-caused the moving failures: they were my own scratch launcher. Its compile
branches were restructured and the '-e VLLM_COMPILE_FULLGRAPH=0' line was dropped, so
four consecutive boots mounted the wrapper patch while _FULLGRAPH still defaulted to
True — and with fullgraph=True EVERY break is an error, so each boot reported whichever
break came first (cell -> workspace -> compiler-disable). I had blamed
TORCH_LOGS=graph_breaks for the difference; that was wrong, the flag only logs. The
launcher sets the env var again, and patch 04's header, the runbook and the work log
now carry the true run history as a table instead of the bad theory.
New artifact: scripts/patches/glm53-compile-05-workspace-acquisition-opaque.patch —
the indexer acquires its shared workspace INSIDE the forward (its profile-run branch
literally says 'Reserve workspace for indexer during profiling run'), and
v1/worker/workspace.py:_ensure_workspace_size reads ContextVar/ubatch state, which
dynamo cannot trace. Wrap both call sites in a torch.compiler.disable'd helper
(narrower than decorating the workspace manager, which every model would go through;
a no-op when not compiling).
All five artifacts verified against the pinned image: applied in sequence with patch
-p1 they reproduce their recorded patched sha256 exactly (6e9cb81b…, 0751852e…,
732363bb…, 84969993…, c154115a…).
tests/run.sh: 362 seams pass.
Working the dyno name-map blocker one step at a time, with the tools to see it:
- All four patch artifacts are now genuine diff -u patches (they were prose + pseudo
hunks before, useless to patch(1)). Verified against the pinned image: each applies
clean and reproduces its recorded patched sha256 (6e9cb81b… / 0751852e… / 732363bb…
/ 84969993…).
- New: scripts/patches/glm53-compile-04-fused-qkv-rmsnorm-optout.patch — takes GLM's
MLA block off the fused-norm JIT kernel (R4D_GLM_FUSE_QKV_RMSNORM=0; the flag is
already a constructor arg glm5next sets at attention.py:583). Reason: in
models/common/ops/fused_qk_rmsnorm.py the kernel object is bound at line 223, AFTER
its use at line 211 — the launcher family (@kernel_launcher, 39 call sites) is what
breaks dynamo's break analysis.
- New diagnostics, both reusable and recorded: TORCH_LOGS=graph_breaks +
VLLM_USE_AOT_COMPILE=0 to name each break with its user stack, and
/home/dank/dynamo-probe (sitecustomize.py on PYTHONPATH, VLLM_DYNAMO_DUMP_ATTRS=1)
to print the dynamo name-map miss detail. Gotchas: a .pth on PYTHONPATH is NOT
executed, and the scratch launcher reset ENVARGS (now ${ENVARGS:-}), which silently
dropped env vars and wasted a boot.
Measurement discipline correction: I first recorded the 'Read uninitialized cell' and
'Unsupported method call' failures as fatal blockers. They are not established as
such — they appear only when TORCH_LOGS=graph_breaks is set, while a flag-free
break-tolerance run gets past break handling and dies on the name-map KeyError. The
likely mechanism is that graph_breaks logging makes dynamo build a break
*explanation*, and that analysis itself trips on the uninitialized cell. Patch 04's
header, the runbook and the work log now carry the confound explicitly rather than
the overclaim.
tests/run.sh: 362 seams pass.
DFlash2: fetched incoai/GLM-5.3-Flash-DFlash2 (2,342,169,800 B, sha256 matches the
Hub LFS oid) and tried --speculative-config {"method":"dflash","model":"/draft"}.
Config and drafter are accepted; the worker dies with 'Model does not support
EAGLE3 interface' - so the only gap is the target-side aux-hidden-state contract
(set_aux_hidden_state_layers + per-layer capture + the two-value forward return,
as deepseek_v2.py demonstrates twice in this image). Recorded the exact interface,
the GLM-specific unknown (what the single-stream state entering layer k means for
an mHC multi-stream trunk) and the honest expectation: no speedup on a
token-proportional pointwise bound.
fp8 KV: recorded the deciding measurement - the fork's KV reserve model is
dtype-blind (the fp8 boot printed the identical 1760 MiB / 13600 B-per-token plan),
so fp8 cannot buy capacity even if the gfx12 MLA path were rewritten. Verdict: do
not spend effort there.
Compile blocker narrowed from 'a latent GLM bug' to dynamo's name map: the
KeyError is thrown by torch/_dynamo/output_graph.py's nn_modules lookup (:3000,
registered :1259/:1384, _modules[..] prefixes stripped :1225), i.e. the compiler
cannot resolve a submodule by name while tracing the decoder layer. Next step is
TORCHDYNAMO_VERBOSE=1, with the note that the scratch launcher resets ENVARGS.
tests/run.sh: 362 seams pass.
A torch-profiler kernel-time window (496 decode tokens, rank0 trace parsed for
device-activity time) on the deployed GLM-5.3-Flash shape falsifies this page's
own "the expert gather is the bound" explanation:
46.18 s of kernel time / 496 tokens = ~93 ms per token (actual step ~104 ms)
34.8 % vectorized_elementwise_kernel 708,500 launches (1,428/token)
10.0 % r4d_gemm_moe_w4a16_nt_b16_m64 (the MoE GEMM)
6.7 % elementwise_kernel_manual_unroll
6.4 % wvSplitK_hf_sml_ (split-K GEMM)
8.9 % router: sbtopk + warpMergeSortTopK + bitonicSortKVInPlace
6.5 % r4d_mhc_* (hyper-connections)
1.05 % r4d_moe_lru_gather_kernel <-- the retracted suspect
The earlier arithmetic was circular (it assumed the gather consumed the whole
step). The real bound is unfused pointwise glue: the fork compiles its own Qwen
trunk (support_torch_compile in models/qwen4_exp/amd/model.py) and not its GLM
trunk, so the engine boots with "provides neither a compiled submodule nor
breakable CUDA graphs. Overriding to FULL_DECODE_ONLY." That also explains why
MTP4 measured flat — pointwise cost scales with the token count, so spec decode
amortizes nothing here — and it retires arena/hit-rate tuning (the gather is 1 %).
Staged as documented patch artifacts (not wired into any serve script; the
engine does not yet boot with them) — scripts/patches/glm53-compile-0{1,2,3}*.patch:
-01 models/glm5next/common/model.py decorate Glm5NextModel (R4D_GLM_COMPILE)
-02 model_executor/kernels/r4d_lib.py freeze 10 capability probes that do
os.path.exists/.so lookups inside the traced forward. functools.lru_cache
does NOT protect them: dynamo re-traces the wrapped body even when warm
(verified in-image); a plain module-global read compiles.
-03 compilation/wrapper.py make fullgraph=True opt-out, because
suppress_errors does NOT cover graph breaks and this trunk can never
satisfy fullgraph (indexer metadata syncs to host — .item() at
glm5next/amd/sparse_indexer.py:263,419).
With all three the trunk traces (Dynamo bytecode transform time 1.08 s) and dies
on KeyError: in_proj_qkvbfg_a in the mHC/KDA path — a latent bug the compile
exposes, and the remaining blocker.
Also records two structural constraints: no new HIP kernels are possible (r4d.so
ships without sources), which binary-blocks the fused router at GLM's 288
experts; and DFlash2 for GLM is a TARGET-side port, not a drafter port
(incoai/GLM-5.3-Flash-DFlash2 is architectures ["DFlash2DraftModel"] +
model_type qwen3, already mapped at registry.py:636; glm5next lacks only the
aux-hidden-state plumbing deepseek_v2.py demonstrates).
Runbook frontmatter, intro, mermaid and wall section, AGENTS.md and the work log
corrected to match the measurement. tests/run.sh: 362 seams pass.
Two operational facts the cutover surfaced: PYTORCH_CUDA_ALLOC_CONF=
expandable_segments:True is part of the qualified shape (the plan fills VRAM to
its modelled headroom, so the default allocator's fragmentation fails
profile/graph capture — measured on the deployed unit), and the deploy's e2e
gate drives the admin open-links spec, which walks every guarded vhost
including 192.168.0.112:8989 on FrankBook-Pro — a sleeping Mac mini fails that
spec and reports DEPLOY INCOMPLETE while the engine is healthy.
The deployed unit failed its first boot with the engine core OOMing during
profile/graph capture: "Tried to allocate 662.00 MiB. GPU 0 has a total
capacity of 31.86 GiB of which 240.00 MiB is free ... try setting
PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True". The expert-offload plan
fills VRAM to its modelled headroom by design, so the default allocator's
fragmentation leaves the capture phase nothing — every successful bring-up
probe (both scratch-port and 7770) carried this env, and the repo recipe
dropped it.
Also pins the seam: the setting is part of the qualified shape, not an
optional tuning.
Payload: Intel/GLM-5.3-Flash-W4A16-AutoRound (AutoRound symmetric int4 group
128, auto_round:auto_gptq, 34 shards + 2 extras, 113,074 tensors, ~170 GiB).
It is the only W4A16 checkpoint this fork can serve with expert offload: its
quant_method "auto-round" is overridden to "inc", which routes the MoE to
libr4d's grouped W4A16 GEMM (R4dW4a16MoEMethod). A compressed-tensors W4A16
build of the same model resolves to TritonWNA16Experts, which has no offload
recipe, and refuses to serve at all (170 GiB of experts cannot fit 64 GiB of
VRAM, so offload is mandatory).
Shape: tcclaviger RDNA4 vLLM fork (same digest-pinned image as the 2026-09-29
qwen cutover), TP=2, 131,072 ctx x 2 seqs, bf16 KV (fp8 is ILLEGAL here — it
disables the sparse MLA backend's rope-free Triton path, the only MLA decode
gfx12 has), util 0.98, NBATCH 2048, --enable-expert-offload
--expert-precise-memory, prefix caching, glm47 tool + reasoning parsers,
MTP and expert prefetch absent by measurement.
Measured: decode 9.9-10.0 tok/s, prefill 918 tok/s (warm), arena 14.51 GiB =
2,383 slots/rank over 12,096 host-backed experts (73.64 GiB pinned/rank).
THE 35 tok/s TARGET IS NOT MET and the shortfall is measured: live telemetry
(R4D_LRU_TELEMETRY=1) gives a 0.53-0.66 per-layer hit rate, a step moves
0.82 GiB of misses in ~100 ms = 8.2 GB/s effective against a link measured at
22.74 GB/s — the same structural gather-path gap the qwen campaign found.
Growing the arena 13.08 -> 14.51 GiB changed decode by nothing.
Rejected by measurement: MTP4 (head loads, acceptance 3.76-4.90, but decode
unchanged and prefill collapses 918 -> 255 — a verification forward gathers the
union of the batch's experts), NBATCH 8192 (identical prefill).
Also: scripts/patches/gpu_worker-kv-cap.patch (fork-local fix for a KV-reserve
model that under-counts GLM's per-sequence KDA state; not needed by this shape),
runbook, AGENTS.md row + section, both indexes, work log.
Rollback: 2339fae (qwen FP8 + MTP4).
The classic 'podman images | grep -q' pattern races under set -o pipefail:
grep exits on its first match, podman dies SIGPIPE (141) writing the
remaining RepoDigests lines, and pipefail promotes 141 to the pipeline
status — the gate reads 'no match' and refuses to deploy (2026-09-30,
dankserver). Same bug in the serve script (it would restart-loop the
same way). Fix: capture podman's output into a variable first, grep the
variable. Patched in the R9V September deploys by luck of timing.
Unquoted value + set -a sourcing parses 'qwen-fp8' as a command ->
ExecStart exit 127 -> unit restart loop (caught on the first deploy).
The deployer's wait_for_health was killed and will re-run on the fresh
deploy.
Replace the R9V radiance fork with the tcclaviger RDNA4 vLLM stack (the
digest-pinned image the 2026-09-29 tuning campaign measured), serving
orcarouter/Qwen3.8-Flash-Next-Uncensored-FP8 (131 shards, 172.8 GiB,
already on disk) with MTP4 spec decode.
Qualified by boot probe on 2026-09-29:
- MAXLEN=200000 x SEQS=3 (600k total) at --kv-cache-dtype fp8 (q8/q8 KV):
'GPU KV cache size: 729,545 tokens; Maximum concurrency for 200,000
tokens per request: 3.65x' at KV_BYTES 7.2 GiB PER RANK.
- KV_BYTES is per-rank: global 16G and 9G both OOM (worker tried to
allocate the full size; ~7.5 GiB/rank is the ceiling with the arena).
- prefetch OFF by law (telemetry: 36% precision, ~50% wasted traffic);
TunableOp off (corrupts numerics).
- decode at this shape: 42.94/34.55/36.09 tok/s; arena 6,032 slots.
Trio: frank-primary.env (all FRANK_PRIMARY_* keys), serve-frank-primary.sh
(direct podman run; entrypoint execs vllm serve; waits on the container),
frank-primary.service (root unit, same ExecStop law, new container name
frank-primary-qwen). Deployer gates rewritten (image digest + shard count
+ weight-map index). download-frank-primary.sh is now inventory/verify
(repo is GATED — 401 anonymous, masked LFS hashes; payload already on
disk). Seam rewritten for the new contract. Pre-cutover tag:
frank-primary-r9v-q8-20260929.
PORTMAP=7773:7770 hook; health 200 on 127.0.0.1 and 192.168.0.235,
decode through the published path in band. 7773 chosen over 7770
(frank-primary owns 7770 when it returns). Log bullet recovered from a
misplaced insert into the 2026-09-28 cleanup section.
Repo tag qwen-fp8-tuned-20260929 at f532e03; serve script launches the
tcclaviger image by digest sha256:da402966... (local alias
qwen-fp8-runtime:tuned-20260929) — the floating :latest is no longer
referenced.
Source reading in the qwen-fp8 container: glm5next is registered with an
amd/ backend dir + DSA indexer; r4d.so has native moe_dsfp4/moe_dsfp4_mt1
entries (has_dsfp4()==True), so the old 'MXFP4 = emulation, missing
kernels' verdict is dead for this fork. DSFP4 = MXFP4 weights + DeepSeek
activation law. Offload recipes exist for mxfp4 and W4A16 (R4dW4a16MoEExperts
in offloader/expert.py:114). GLM gate is now the payload format
(compressed-tensors MXFP4/W4A16, none public) + a HIP boot probe.
The KV trim is inert: arena identical (5,648 slots) at 4.5 and 4.03 GiB
(planner buckets; the trim went to margin). Final 256k config
MAXLEN=262144 KV_BYTES=4328521728 prefetch off; full-speed alternative
MAXLEN=184224 at 3 GiB.
Asked for iterative decode/prefill improvement at a 32k working point
without touching the model or the quantization. There isn't one: gufo's
served settings are already at this engine's tuned point, so no config
change was made and none is proposed.
Measured on the repo's own prose and shell sources at 32,762 prompt
tokens, salted per run against a warm prefix cache, 3-5 repetitions:
* --prefill-chunk 512 -> 8192 moves prefill 410.7-412.4 t/s. A 16x change
in chunk size, 0.4% out. The lever is dead at this depth.
* --draft-tokens 5 -> 10 gives 13.79-15.86 t/s decode, all inside the
run-to-run noise. The 5-rep confirmation REVERSED the 3-rep leaders:
dt9+chunk4096 looked best and came last (12.86), the deployed config
came first (14.71).
* --draft-policy fixed vs adaptive: no effect. --min-draft-tokens:
rejected by the engine ("DFlash2 requires ... 1"). MTP: unsupported for
Q8 targets (token_embd.weight is Q8_0).
* --context 262144 -> 40960, tested as a diagnostic: no gain, and it
would break models.yml's declared contextWindow. Not shipped.
What does move the number is speculation itself — 6.65 t/s off vs ~15 t/s
on, 5.3x — and it is already enabled.
The two traps that cost most of the wall time are now in AGENTS.md's
benchmarking policy, because both produced FALSE results:
1. gufo bench cannot rank speculative settings and inverted the ranking.
Its corpus reports acceptance=0.933 +/- 0.000 with an identical trace
sha256 across draft widths — the same highly-predictable text every
run. It reported 29-37 t/s where real traffic gives ~15, and made
dt16 look 6% faster than dt7 when real text made it 14% slower.
2. A live A/B needs >=128 generated tokens. At 8, inter_token is
first-token noise; the first pass produced a nonsense ordering that
the 128-token pass reversed.
Plus the Server-Timing header is `ttft;dur=402.485` — the unit rides in
the NAME half, so splitting on "=" keys the dict "ttft;dur" and every
lookup silently returns nothing. Four configs reported "no Server-Timing"
while the header was present on all of them.
The disk was at 93% with 137G free. Nine retired payloads are now moved
to ~/.AGENTTRASH/20260928-retired-models/ with a MANIFEST.md giving a
restore command per item — moved, not deleted, and nothing is freed until
the purge is approved. Every one maps to a runtime already retired in the
runbook index.
Only the .gguf was taken from Qwen3.8-27B-UD-Q8_K_XL/: the live
frank-large DFlash2 drafter lives in that directory and stays put. The
manifest warns against restoring the directory wholesale.
Kept: checkpoints/ (23G) — unreferenced by the repo, but ComfyUI is live
and its workflows were not searched, so it is not confirmed dead. Also
logs/, because x4e-run.log is a SPARSE file (3.3G apparent, 4K real) and
frees nothing.
The reference check caught a live bug: serve-frank-small.sh's fallback
MODEL_DIR still pointed at the retired GSQ-RCO payload frank-small served
on 2026-09-16/17. frank-small.env has always overridden it, so the drift
was invisible — but a missing or unreadable env file would have turned a
config problem into a "missing asset" naming a model that is no longer
there. Repointed at Gemma4-12B-QAT/, which the env file already sets.
Not taken, by request: podman images (279G reclaimable, 95%) and the
uv+pip caches (73G). Together 352G of regenerable garbage, larger than
what was reclaimed here.
The 29.3G payload rollback target went with the rest, so the payload-only
rollback is now a re-fetch first; the runbook and AGENTS.md say so.
The target moves from unsloth/Qwen3.8-27B-GGUF:UD-Q8_K_XL to
ajgazin/Swift-1.5-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF's UD-Q8_K_XL —
an abliterated fine-tune of the same base, under the non-OSI SWIFT OPEN
LICENCE 1.0. Port, alias, context, drafter and client contract are
unchanged.
Compatibility was settled from the GGUF headers before fetching anything:
both files parse to the same architecture (qwen35), the same 866 tensors
with identical names in identical order, zero per-tensor ggml_type or dim
mismatches, and the same MTP head. Two metadata differences that look
alarming are no-ops, confirmed against gufo's source — general.file_type is
deliberately not consulted (the file gufo already served records Q4_K_M
while containing no Q4_K tensor), and Swift's extra
qwen35.attention.recurrent_layers key is never read; layer kinds come from
full_attention_interval, which both files carry as 4 and which agrees.
The load test agreed: 9.2 s to listening, 66.5 GiB of the 116.7 GiB iGPU,
no loader warnings, and a green contract battery including a two-turn tool
loop.
The DFlash2 drafter did not move, so the payload is now two files in two
directories: DRAFTER_DIR is its own tunable. That broke the deployer's
stray-run sweep, which keyed on the target filename and would have stopped
matching — a stray run would then have held 7770 past the port gate. It
now keys on the drafter, which every frank-large command line carries.
The fetch script sends no auth header: the repo is public, and a token on
the curl command line is world-readable via /proc/<pid>/cmdline. The
anonymous transfer ran at 1.8 MiB/s; the host token through a mode-600
curl -K config reached 7 MiB/s.
Test seam: the mutation pass found four real gaps. Two fetch assertions
were satisfied by a header comment rather than the URL assignment, so
repointing a repository left them green, and nothing bound the env's
MODEL_FILE/DRAFTER_FILE to the filenames the fetch scripts download. Both
are now explicit. A harness bug worth recording: a bats --filter that
matches no test reports zero "not ok", indistinguishable from a mutation
that survived — the harness now checks the 1..N count. 16/16 caught.
The ad-hoc status commands kept producing wrong readings, most seriously
reporting "MemAvailable 0.0 GiB" on a host that actually had 240 GiB free.
Cause: /proc/meminfo separates key from value with a TAB, so
grep MemAvailable /proc/meminfo | tr -s " " | cut -d" " -f2
returns an empty field and awk prints 0.0. A false "host is out of memory"
reading is worse than no reading, because it invites an unnecessary abort.
dankstatus.sh parses on a colon split (tab-agnostic), cross-checks
MemAvailable against MemTotal before printing either, and refuses to print
a zero value. It also pins the rocm-smi CSV column order (device, total,
used) which was previously read swapped, and takes only the first line of
systemctl is-active/is-enabled, which emit more than one line here.
Incidental finding while validating: rocm-smi reports card2 (the 7700X iGPU)
as 0.50 GiB total, not the 123.0 GiB torch.cuda reports. torch was exposing
the GTT aperture. The device is smaller than documented, not larger.
Tests: 346/346 pass (no test surface touched).
Public, ungated repo so no auth header is sent — a token on the curl
command line is world-readable via /proc/<pid>/cmdline. Size and sha256
are the Hub LFS values (31457990272 / 9076f99a…), verified after fetch.
Not wired into frank-large.env: the gufo load test has to pass first.
Live numbers from the 2026-09-28 deploy: 13 s load to listening, RSS
30.6 GiB, ttft 549.8 ms / inter-token 24.2 ms on a short prompt, and a
green contract battery (health, ready, /v1/models at 262,144, content +
reasoning_content, tool_calls, SSE, and a two-turn tool loop).
Also recorded: gufo accepts every request field the harness sends in
extraBody and the full reasoning_effort set, both verified 200 — which
is why models.yml's reasoningEffortMap fold is now optional rather than
required, and why it is being kept anyway.
The .AGENTTRASH archive is ignored and exists only in a primary checkout,
so a filesystem grep fails there while passing in a worktree. git grep
searches the tracked tree, which is what the assertion is about.
gufo is a vertical inference engine built for exactly this hardware
(Ryzen AI MAX+ 395 / Radeon 8060S, gfx1151), and Dankclaw's frank-large
has swapped runtimes seven times in search of a better one. This one
ships no prebuilt Linux binary, so the runtime is BUILT FROM SOURCE on
the host and the payload, port, alias and client contract stay exactly
where they are.
Runtime: gufo v0.1.1 (b0f8673) at /home/dank/GitPackages/gufo-v0.1.1,
built by the new scripts/build-gufo.sh. It links the host ROCm 7.2.4
through its RPATH — the opposite of the unsloth prebuilt it replaces,
which carried its own — so two host workarounds are encoded in that
script: Ubuntu's librocwmma-dev installs a truncated rocWMMA header set
that shadows AMD's and cannot be overridden with -I in HIP mode, and the
ROCm clang prefers Ubuntu 26.04's GCC 16 headers over the GCC 15 whose
libstdc++ is actually installed.
Shape is unchanged: one session, full 262144 ctx, the same UD-Q8_K_XL
target and the same DFlash2 drafter, greedy, text-only. Speculation
moves from llama.cpp's fixed depth 4 to gufo's adaptive block policy
capped at 7 proposals.
Client-contract deltas, all documented: the chat template is compiled
into the binary (no --chat-template-file, so the Qwen-Sharp 73k-token
failure mode is gone), reasoning_effort now takes the full OpenAI set
(so models.yml's reasoningEffortMap fold is no longer required — kept
anyway, so this commit does not also move every high role from xhigh to
high effort), and there is no KV-quant or prompt-cache-RAM flag.
The test seam now runs the serve script against a stub engine and
asserts on the argv it actually execs, instead of grepping the script's
text. The deploy handler gates on the runtime binary and returns before
it stops the serving unit.
Measured trade, from gufo's own Q8_K_XL tables against llama.cpp
b11069: prefill +37% to +80%, cold load 4.5x faster, decode +7% at zero
depth — but -14% at 65,536 tokens of depth and -24% at 131,072. On the
8-bit target the crossover is ~32k, and frank-large runs one 262,144
conversation. The Q4_K_XL target shows no such regression.
Answers Frank's question: given the 2x R9700 box, which of DeepSeek-V4.1-Flash,
MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash runs best, on what runtime, and can
Affinity / vllm-radiance / FreeToken-AMD be modified to run them fast.
Sizing, measured from the HF API rather than the model cards:
- DS4.1 payload 475.27 GiB, of which 189.13 GiB is Engram (two 94.56 GiB
shards, the layer 1 and 14 lookup tables). Lucebox never maps them, so
they do not evict the expert page cache.
- 240 routed experts x ~11 MB per token at the 37-of-384-per-layer residency
the hot stack can hold = ~2.4 GB across PCIe per token. At 55 GB/s real on
Gen5 x8 that is a ~23 tok/s ceiling, under the 35 tok/s goal before any
attention or dense work.
- The backend refuses layer-major prefill for V4.1, so the only legal
prefill is token-by-token; 1000 t/s is ~20x away.
Affinity (Codeberg StillDeadcode) builds clean here (hipcc 23, gfx1201, zero
errors) and claims 110 tok/s for the 284B V4-Flash, because the 101 GiB
container leaves the cold expert set in a pinned host pool in RAM. V4.1 is
2.5x too big to repeat that. Port scope, from both checkpoints' real
safetensors indices: expert machinery ports unchanged (identical tensor
naming, 15,360 tensors at 40x384), 26/32 per-layer tensors and 34/35 config
keys present. Five gaps - Engram, the disjoint {0,1,2} vs {0,4,128} ratio
vocabulary, CED, the DSpark draft geometry, vision. num_hash_layers is the
only key V4.1 lacks, so model.h:36's default of 3 silently survives and
would route layers 0-2 through a table that does not exist.
Also corrects: Dankserver RAM 60 -> 246 GiB (stale since the 2026-09-20 board
swap) and storage; the top-level index credited V4.1 with V4's geometry
(43 layers / 256 experts vs the actual 40 / 384).
New ACTIVE runbook (setup/doctor PASS=34/FAIL=0, UVA transport verified
in serving workers, wire-verified alias/reasoning/metrics), index +
vault entries flipped: llama.cpp b11030 endpoint RETIRED 2026-09-27 to
rollback target. Topology diagram updated.
download-frank-primary-fp8.sh + the fp8 manifest fetched the vLLM 0.30
payload that never deployed; the R9V cutover's payload is the R9V setup
flow's (packages + PLE, state-gated). Scripts move to old/ per policy.
Replace the vLLM 0.30 UVA-offload wiring (never deployed; llama.cpp
b11030 endpoint replaced by Frank's order, downtime window open) with
the R9V radiance-vLLM shape on the same root unit:
- serve-frank-primary.sh: layer the on-host qualified setup state
(104 tuned R9V_* keys, /home/dank/r9v-state/uncensored/setup.json)
under repo production overrides, then exec the fork's own launcher
through the docker->podman shim and block on docker wait so a crashed
engine is a nonzero ExecStart (launch.sh --detach alone would fake
green). Gates: pinned image by OCI manifest digest in RepoDigests,
PLE at pinned byte count (stat -c %s), all GGUF shards + mmproj.
- frank-primary.env: thin production overrides only (LAN bind 0.0.0.0,
:7770, alias frank-primary, container frank-primary-r9v, GPU BDFs);
tuned engine shape stays in the setup state by design.
- frank-primary.service: same root-unit shape (User=dank,
LimitMEMLOCK=infinity for the tiered-expert pin), plus ExecStop
podman stop — docker wait cannot stop the container child.
- deploy-remote.sh deploy_llm_primary: state-driven gates (state ready,
image digest present in rootless storage, PLE bytes, shard files)
replace the fp030 shard-index gate; false-green MainPID check kept.
- tests/frank_primary_unit.bats: seam rewritten for the R9V shape.
Unit restart SIGKILLs conmon before --rm cleanup runs, leaving the container
name registered: every restart then dies on name-in-use while the orphan
holds the port and GPUs (observed live 2026-09-27, NRestarts=6 with the
container Up and the unit auto-restarting). The serve script now clears its
own name before exec podman run; the deployer stops the unit and removes
the container BEFORE waiting on the port.
HOME inside the container pointed at the root-owned WORKDIR /app, so
vLLM's registry cache write hit PermissionError; HOME now points at the
cache volume.
The vLLM cutover itself is blocked on a structural wall (AMD Qwen4Exp impl
creates the 51.2B PLE table as a bf16 on-GPU param: 47.69 GiB transient on a
31.86 GiB card; the pinned-host UVA path that fits this hardware is
CUDA-gated). llama.cpp is restored live at :7770; host HEAD unchanged.
The llama.cpp b11030 trio becomes the vLLM 0.30 trio ahead of the
FP8 cutover on :7770 — same port, same alias, no client moves.
Serve shape: locally built ROCm gfx1201 image
(localhost/frank-primary:vllm030) execs via podman as dank from the
root unit (LimitMEMLOCK=infinity legalizes the pin). UVA offload:
--offload-backend uva pins the PLE family (47.75 GiB, 138 tensors:
131 n-gram table shards + 7 per-layer conv/proj) + routed experts
(114.86 GiB, 150,528 tensors) in host RAM — 162.60 GiB combined,
~81.3 GiB per rank under TP=2, hence --cpu-offload-gb 82 (80/card
strands 2.6 GiB of experts on the cards). Parsers per the host's
own Ornith-era precedent: qwen3_xml tools + qwen3 reasoning (vLLM
0.30 emits the channel as `reasoning` — reasoning_content is
renamed server-side) + enable_thinking/preserve_thinking template
kwargs. Spec decode stays env-wired and OFF at bring-up.
Deployer deploy_llm_primary rewritten: image gate, index-driven
shard-count gate (131), stale-container cleanup, port wait, and the
false-green hardening — health is only green when the unit's own
MainPID is live behind the answer (2026-09-26 lesson).
Seam tests rewritten to the vLLM contract (15 gates): offload
arithmetic, parser wiring, no-llama.cpp-ism, deployer gates. Suite
349/349 green.
scripts/manifests/qwen3.8-flash-next-fp8.manifest: 144 entries (131
safetensors shards + aux) with Hub LFS sha256s for
Qwen/Qwen3.8-Flash-Next-FP8. download-frank-primary-fp8.sh wraps
hf-fetch.sh: idempotent fetch, full sha256 re-verify of an on-disk
payload (verified live on dankserver: all 144 files match, ~10 min).
Part of the frank-primary vLLM cutover prep; the serve trio itself
lands with the image deploy.
The stock SearXNG image ships a native kagi engine; enabling it needs an
API key that must never enter git, and SearXNG does not expand env vars
in settings.yml (the live secret_key literally was '${SEARXNG_SECRET}'
until now — the app tolerated the placeholder string silently).
- settings.yml: kagi engine enabled (weight 2, kg shortcut, paging off so
exactly one API credit per aggregated query, timeout 8.0 vs 5.0 scrapers
so a slow paid query is not a wasted credit); api_key stays ${KAGI_API_KEY}.
- new scripts/searxng-render-settings.py: deploy-time renderer, allowlisted
to KAGI_API_KEY/SEARXNG_SECRET/SEARXNG_BASE_URL, fails the deploy loudly
on an unresolvable placeholder, idempotent on redeploys.
- deploy-remote.sh copy_searxng_config runs the renderer against
/opt/searxng/.env after copying settings.yml into /opt/searxng/config.
- KAGI_API_KEY placeholder added to dankclaw/.env.example (real key staged
into the host-side /opt/searxng/.env, mode 600, out of band).
- bats: engine wiring, renderer substitution, loud-failure, deployer seam.
- docs: SearXNG.md Kagi section + renderer law; Providers.md carve-out on
the no-credentials policy (paid engine acceptable when bounded + ordered).
Verified live on dankclaw: direct API probe via the gluetun netns (200,
data.search[] shape), then an aggregated query returned 29/29 results
tagged kagi, zero unresponsive engines.
The first no-redirect HEAD fix read X-Linked-Size on every 3xx, but
non-LFS files on a Xet-backed repo answer 307 to HF's own
/api/resolve-cache WITHOUT any X-Linked-Size header. The fallback to the
stub's Content-Length recorded the redirect page size (254-264 bytes)
instead of the file size, so hf-fetch.sh failed the size check on every
aux file it verified against the regenerated manifest (README.md
"65400 != 254", config.json "72514 != 258", 7 files on the FP8 repo).
Distinguish the two redirect shapes: 3xx WITH X-Linked-Size = LFS/Xet,
whose X-Linked-* headers are the truth and whose redirect must NOT be
followed (the Xet bridge ETag is the CAS hash, not the sha256); 3xx
WITHOUT = non-LFS, where the Location (followed once, via urljoin)
answers 200 with the real Content-Length and a git-blob-sha ETag that
the 64-hex gate correctly rejects (size-only check for those files).
Verified against Qwen/Qwen3.8-Flash-Next-FP8: all 7 previously-failing
aux files now record their on-disk byte counts exactly (README.md 65400,
config.json 72514, merges.txt 3353259, ...), shard shas unchanged
(model-00002 6841fe21...), total 172.82 GiB across 144 files.
On Xet-backed repos (Qwen/Qwen3.8-Flash-Next-FP8), the resolve endpoint 302s
to a CDN bridge whose final ETag is the Xet CAS chunk-dag hash, NOT the
content sha256. urllib follows redirects, so the manifest recorded the Xet
hash and every downloaded shard failed sha256 verification against a hash no
download could ever match (8/8 shards FAIL, all sizes correct, on-disk bytes
matching the Tree API's lfs.oid). Classic LFS CDNs serve ETag=sha256, which
is why this was invisible on prior fetches.
Fix: HEAD via http.client, which does not follow redirects. 200 = direct
answer (Content-Length + ETag, non-LFS); 302 = LFS/Xet redirect
(X-Linked-Size + X-Linked-ETag = content sha256). Verified against the FP8
repo: shard model-00002-of-00131 now records 6841fe21..., the sha256 of the
bytes already on disk (cross-checked against the Tree API lfs.oid).
Instrument the live server to answer "can -ot target the most active
layers": AMD IBS op sampling on llama-server during 4 mixed concurrent
generations, sampled load addresses mapped through /proc/<pid>/maps to
backing tensors with extents derived from consecutive GGUF header
offsets (no quant type-size tables - a wrong per-element size silently
mis-buckets tensor tails, first run lost 59% of expert samples to a
junk bucket).
Measured (75 s window, 2.7M expert-tensor samples): per-block read
volume is FLAT, 0.93x-1.08x of mean across all 40 CPU-resident blocks.
MoE top-k routing fires every block every token, so "most active
layers" do not exist at -ot granularity - any 8 GPU-resident blocks
are equivalent, the lever is COUNT, already at the 5/3 VRAM ceiling.
Real skew is per-EXPERT (top 10% of 48x512 cells carry 40.5% of
samples), below tensor-override granularity. No config change.
scripts/frank-primary-expert-activity.py: dump + sample + analyze in
one entry point; --analyze-only re-analyzes saved samples. Instruments
production serving in place; not a benchmark.
Blocks 0-4 on ROCm0 + 45-47 on ROCm1. This exact shape was probed and
battery-verified earlier the same session: card0 2.54 GiB free at boot,
2.11 GiB after the vision graph (steady under load), card1 unchanged
at ~3.3 GiB. Card1 cannot take another block (0.82 GiB would OOM the
vision graph + boot drift), so 5/3 is the physical limit at the
4x262144 / V-q8_0 shape. OT assertion pins the new value.
- CONTEXT_SIZE=1048576 / PARALLEL_SLOTS=4: four conversations, each the
full native 262144 window (per-slot window unchanged; slots divide
the pool, they never multiply it).
- CACHE_TYPE_V=q8_0 / CACHE_TYPE_K=f16: main KV drops 24,576 -> 18,816
B/token (-23.4%); the shared MTP drafter's KV ignores the V type
(measured: 2,048 MiB draft pool = the f16 law at 1,048,576 cells).
- OT_OVERRIDES: blocks 0-3 on ROCm0 and 45-47 on ROCm1 stay
GPU-resident (2.42/2.49 GiB per block, measured); the rest stay
CPU-computed. Measured free under load: ~4.9 GB card0 / ~3.4 GB
card1; a 5th block on card0 lands at 2.27 GB, under the 3 GB
checkpoint floor, so blocks are atomic and 4 is the safe fill.
- serve script: --cache-type-k/v env-wired tunables; ONE joined
--override-tensor flag (b11030 deprecates repeated flags, and the
loader is first-match-wins, so the specific patterns must precede
the broad exps=CPU sweep).
- seams: the window test asserts per-slot = 262144 at 4 slots; new
KV-type test; OT exact-value assertion pins the ordering law.
29,696 B/token ctx-scaled (12 of 48 layers carry KV - full_attention_interval=4;
indexer and draft KV included); weights/RS/compute are ctx-independent.
262k = 18.75 GiB total / 9.37 per card; 100k = 14.3 / 7.1. Parallel
slots split the pool rather than multiplying it.
Both b11030 serve scripts never passed --metrics, so /metrics answered
501 and Prometheus could not scrape frank-large (silently down since
its deploy) or frank-primary. frank-primary also gets -lv 4 (VERBOSITY
env tunable, default 4) for the full per-buffer load breakdown.
Battery 6/6: alias, reasoning_content, tool_calls, vision (probe image
described exactly), reasoning_effort high -> 200. Decode 11.8 t/s at
depth-0 prose (draft 109/accepted 72), 0.864 mean len 2.73 on tool
traffic. VRAM 29%/36% (output-head card carries the drafter). Also
records the ROCm1 drafter law in the runbook body (borrowed output.weight
+ n_copies==1 abort) and the two cutover traps: foreign-process health
false green, and llama-server SIGTERM cleanup hang.
The shared MTP head borrows the main model output head, which a -ts 1,1
layer split places on ROCm1. With tensor overrides set, n_copies==1 in the
draft's scheduler, and a borrowed output.weight in a ROCm1 buffer aborts the
draft graph (ggml-backend.cpp:941) when the draft context is on ROCm0 —
the unit crash-looped 4x at ~12 s per attempt during the 2026-09-26 cutover.
Probed live on the host: ROCm0 aborts identically, ROCm1 loads and serves.
Discovery, not guesswork: dump the GGUF v3 headers of the fetched parts
(scripts/dump-gguf.py, added — a dependency-free reader that never
touches tensor data, so a 50 GiB part scans in milliseconds) before
the first load.
What the dump says (parts 2-3, 2026-09-26):
- The 51B-param PLE n-gram table is ONE tensor: per_layer_token_embd.
weight, dims [160, 320001536], Q8_0, 54.4 GiB — exactly the whole of
part 3. It is the same tensor the shared MTP head borrows, so the
=CPU override is required for placement AND for the draft load
(GPU-resident borrow: "invalid vector subscript").
- The model is qwen4exp, 512 experts, top-10 routing
(qwen4exp.ple.ngram_size=3, heads_per_ngram=8).
- Part 1 is metadata-only (ntensors=0); parts 2-6 carry tensors.
The placeholder pattern ngram_embeddings matched nothing — dropped from
OT_OVERRIDES (env, serve-script default, seam, AGENTS.md, runbook,
LLM index) before it could masquerade as a working override at load
time. Deployed set: exps=CPU,per_layer_token_embd.weight=CPU.
Suite: 346 green.
The 2026-09-26 same-day cutover that ended the Lucebox DS4.1 detour:
Dankserver's :7770 endpoint is now unsloth llama.cpp (the b11030-mix
prebuilt, ROCm gfx120X, bundled runtime) serving
unsloth/Qwen3.8-Flash-Next-GGUF at Q8_0 — 175.30 GiB across six sha256-
verified split parts — on both Radeon AI PRO R9700s, as a ROOT unit (the
frank-large shape: no container, no generated unit).
The deployed contract:
- Placement law: layer split -ts 1,1 with -ngl 999; the routed experts
and the 51B-param n-gram (PLE) table are COMPUTED in host RAM via
-ot exps=CPU,ngram_embeddings=CPU (the x4-pair measurement: streaming
over PCIe loses at 3.7 t/s, computing in RAM does not care).
per_layer_token_embd.weight=CPU is required by the shared MTP head's
borrow path (a GPU-resident borrow crashes the draft load).
mmap stays on; --cache-ram 16384.
- Spec chain draft-mtp,ngram-mod: the shared Q8_0 MTP head
(mtp-...-shared-Q8_0.gguf, 2,786,568,256 B) at depth 2, with the
draftless ngram-mod matcher chained behind it — the only legal way to
layer two spec implementations in one process.
- Vision explicit: --mmproj names the BF16 projector (this build never
autoloads; a server without it comes up healthy and text-only).
- Reasoning: --reasoning-format deepseek → reasoning_content; the
Qwen3.8 template's {xhigh, medium, low} reasoning_effort gate folded
by reasoningEffortMap in models.yml.
- One stream, the model's full 262,144 context; greedy server default
with Qwen-recommended sampling riding compat.extraBody.
The deployer's deploy_llm_primary is rewritten for the root unit: it
gates every artifact at its declared byte count (stat -c %s, never wc -c
— which read the whole 377 GiB lucebox file for minutes on 2026-09-26),
retires the lucebox user quadlet to dated home trash, waits out port
7770, relinks/enables/restarts the root unit and waits up to 1800 s for
/health. The quadlet-era tree under dankserver/frank-primary/ is
archived (dankserver/.AGENTTRASH/20260926-llamacpp-cutover/ on this
box; git history everywhere else) and its planner mapping replaced by
the scripts/ trio.
Observability flips back to the llamacpp namespace: a frank-primary
Prometheus job at 192.168.0.235:7770 (cross-host LAN — the netavark
published-port quirk that forces host.containers.internal for the two
Dankclaw endpoints is same-host only), the LlmEndpointDown runbook
text, and the dashboard description for three llama.cpp endpoints.
Docs: the new runbook (frank-primary-llamacpp-qwen38-flash-next-q8.md),
RETIRED banners on the lucebox and radiance runbooks and their index
entries, the AGENTS.md fleet section rewritten for the root-unit shape,
and the homepage frank-medium card renamed to frank-primary with the
Q8_0 Flash-Next spec (docs.html #models and the modality note updated:
images on frank-primary and frank-small, audio on frank-small only).
models.yml: the dankserver provider is "llamacpp (dankserver)" serving
frank-primary, contextWindow 262144, maxTokens 65536 (compaction-gap
law: 183,501 + 65,536 <= 262,144), input [text, image].
Suite: 346 green (frank_primary_unit.bats rewritten for the root unit —
16 tests pinning the placement law, the shared-head requirement, the
explicit projector, HIP env hygiene (BIND_HOST, never bare HOST), and
the deployer's byte-count gates; the quadlet-era planner and
changed-tests mappings rewritten for the trio; the observability scrape
test asserts three llamacpp jobs again).
Rollback: the lucebox image (localhost/lucebox-rocm:gfx1201) and the
378 GiB payload stay on the host; the Qwen payload stays fetched, so a
re-swap in either direction is a redeploy, not a refetch.
Fetches into /home/dank/models/Qwen3.8-Flash-Next-GGUF/ keeping the Hub
layout: the six Q8_0 split parts (175.30 GiB — never concatenated, part
1's header advertises file_count=6), the shared Q8_0 MTP drafter head
(2.6 GiB — "shared" borrows the target's embeddings; unsloth's MTP guide
names it the recommended head), and the BF16 vision projector. Total
191,919,144,448 bytes.
Sizes and sha256s captured from the resolve endpoint's x-linked-etag/
x-linked-size on 2026-09-26; every artifact is checked for exact byte
count AND sha256, curl -f so an auth error fails instead of writing the
error body, resumable via -C -. The repo is public: no token machinery.
The frank-medium retirement tail the 2026-09-26 lucebox cutover left
uncommitted: the radiance quadlet tree, its swap/fetch tooling and its
seam test are deleted (archive copies live in dankserver/.AGENTTRASH/
20260926-frank-medium-radiance/ and scripts/.AGENTTRASH/20260926-
frank-medium-radiance/), the planner/allowlist/test-changed mappings
drop the service, and frank-large's chat template moves to scripts/
(its old home was the retired frank-medium dir).
Finished here:
- tests/frank_primary_unit.bats: the lucebox quadlet seam (boot
persistence, host-network bind, HIP_VISIBLE_DEVICES=0,1, HIP-reserved
env-name guard, checkout-mounted entrypoint/healthcheck, first-start
patience, deployer payload/port gates).
- test-changed.sh: explicit .AGENTTRASH/ case — retirement commits move
files to trash, and the wrapper refused those additions with "no
focused-test mapping".
- observability: the vllm: namespace left with the radiance runtime.
Rules keep only llamacpp:* series (prefix-cache reuse, spec-decode,
queueing, endpoint-down); the dashboard drops the vLLM KV/latency
panels; prometheus.yml documents that frank-primary (lucebox) has no
/metrics and therefore no scrape job. llm-fleet.json was left
corrupted mid-edit (missing opening brace) and is rewritten from HEAD.
Suite: 339 green (was 7 red).
The runtime's auto budget is gpu_free - kv - warm - safety (22.35 GiB), which
leaves only the 0.50 GiB safety margin free while the fit check demands the
1.50 GiB discrete headroom, so auto-sizing refuses on any 32 GiB target:
"device 0 memory: 30.97 of 31.86 GiB used, 0.89 GiB free (headroom 1.50
GiB): does not fit". 18000 MB is the value DS41.md measured its R9700 rows
at: ~1500 experts resident, ~5 GiB free on device 0.
--ds4-prefill dense made requires_monolithic_model() true, and a monolithic
load asks ggml for the entire 187 GiB expert set as one device-0 buffer:
"allocating 191932.25 MiB on device 0: cudaMalloc failed: out of memory",
followed by "monolithic HIP load required for dense prefill". Approximate
prefill (dense, sparse) is expressible only where the experts are resident
(in-process owners). exact feeds the prompt token by token, which is the
path the hybrid tier streams experts for. Chunk follows DS41.md's
cold-expert recipe (1024).
wc -c falls back to reading the whole stream when its fstat shortcut does
not apply, and this gate runs over a 377 GiB GGUF: the 2026-09-26 cutover
deploy sat in wc for minutes (30 s of CPU, NVMe-bound) before it could
reach the cutover at all. stat -c %s is metadata-only and equally exact.
The llama.cpp hunk (ggml_cuda_node_launches_nothing) was needed on main's
tree; the branch restructured that file and compiles clean, so only the
apt-list and cmake-3.31 hunks remain. Regenerated from the host clone.
The deepseek41 architecture exists only on Luce-Org/lucebox's
feat/ds41-engine branch; main rejects the checkpoint with
"unsupported model architecture 'deepseek41'". Re-aim the image build at
that branch and fix the launch shape, which had been copied from the V4
DS4.md recipe:
- drop --ds4-fused-decode and --ds4-fused-verify-f16-kv, and use
--ds4-prefill dense: V4.1's validate_model_features refuses the V4-only
prefill/fused paths (layer split and --mmproj too).
- --chunk 4096 (the V4.1 dense-prefill measurement point), --max-ctx
131072 (the V4.1 default window), --prefix/--prefill-cache-slots 0 as
the DS41.md bring-up recipe.
- drop --cache-type-k/v: the DeepSeek4 backend ignores them; the MLA
compressed caches are F16 by design.
- expose BOTH R9700s and set LUCE_EXPERT_STREAM_DEVICE=1, so hip:0 owns
the dense weights plus the hot expert stack while hip:1 holds the
streamed-expert slot cache instead of competing with it.
- quadlet: 384 routed experts per layer, Engram tables, 40 layers.
Replaces frank-medium (vllm-radiance TP=2) on :7770. The 377.2 GiB
43-layer MoE (256 routed experts, top-6) is mmapped: dense work and the
MLA KV cache on the R9700, routed experts computed on CPU from the
page cache (246 GiB RAM). The unit bypasses the image entrypoint
(Qwen/DFlash-only flag set) and execs luce_server with the DS4 launch
shape; --ds4-fused-decode's monolithic-load fallback IS the serving
mode here. deploy_llm_primary gates on image + model byte count,
stops/masks the radiance unit, waits for :7770, installs and starts.
Runbook: frank-primary-lucebox-ds4.md.
With a missing file, -C - sends Range: bytes=0-, which HF's xethub
CDN rejects with 400. Xet-backed repos (e.g. Lucebox/DeepSeek-V4.1-
Flash-ROCMFP23-GGUF) fail on the first fetch; the old LFS backend
tolerated the zero-offset range.
Dim the spec line (--dim) so the card reads pale role -> dim spec -> bright
spring-green inputs -> dim use. The inputs line now pops as the bright middle
tier and the spec/use are de-emphasized as reference detail.
Add --input: #00ff9d to the palette and point .desc-inputs at it, so the
Inputs line is visually distinct from the bright-green spec (--fg), the
pale-green role (--head), and the dim-green use (--dim).
Each LLM card now lists its accepted inputs as a line (Inputs: text · image ·
audio), sourced from the harness models.yml:
- frank-large: text (no vision projector)
- frank-medium: text, image (Gemma4 vision path)
- frank-small: text, image, audio (Gemma4 projector audio encoder; audio is
not in models.yml's input list only because OMP's schema rejects it, but the
endpoint accepts input_audio parts)
New .desc-inputs CSS + bare-mode reset; /docs model guide gains an Inputs line
per model. bats 30/30 + live render verified.
Each LLM card now carries a details object (role/spec/use) rendered as a
bold pale-green role, a bright-green spec line, and a dim 'Use:' line.
app.js descHTML() falls back to the plain description for non-LLM cards;
new .desc-role/.desc-spec/.desc-use CSS + bare-mode reset.
Roles made neutral (no nicknames): frank-large = deep reasoning,
frank-medium = primary model, frank-small = fast utility model.
The /docs model guide was fixed — it had drifted to retired model info
(frank-large as Qwen3.8-Flash-Next 87.3 GiB/200k, frank-medium as FP8
~31 GiB) — now Qwen3.8-27B UD-Q8_K_XL 31.5 GiB/262k and MXFP4/W4A8
19.4 GiB, reordered large->medium->small. bats 30/30 + live render.
frank-large: cybersecurity + debugging (requires Frank's permission)
frank-medium: primary intelligent model (surpasses Claude Sonnet 5 per
Artificial Analysis) — software engineering, sysadmin, instruction tasks
frank-small: fast/cheap utility model — agentic search, RAG, research
subagents, translation, roleplay, summarization, small utility tasks
Updated both the status cards (services.json) and the /docs model guide.
The auth works via the modern /Users/AuthenticateByName endpoint (what the
web UI uses). The 401 was a test artifact from probing the dead legacy route.
No config change needed; LDAP-Auth plugin + authentik 2026.8.3 outpost healthy.
The UserWriteStage writes prompt_data['password'] via user.set_password().
A PasswordStage only authenticates the current password. Document the
correct mechanism (password field on the default-user-settings prompt +
empty-password guard policy) and the throwaway-user verification.
The default-user-settings-flow was missing a PasswordStage, so users could
edit name/email/locale but not their password. Added a PasswordStage
(InbuiltBackend) bound at order 101. Documented the flow's stage table and
the two ORM gotchas (MTI through model, required backends array).
The local username/password path is retired (ENABLE_LOGIN_FORM +
ENABLE_PASSWORD_AUTH false, ui.enable_login_form DB override false) and
qa-e2e-admin is out of the authentik Admins group. Documented in the SSO
section and History.
The authentik OIDC round-trip is verified, so retire the local
username/password path. ENABLE_LOGIN_FORM hides the form and
ENABLE_PASSWORD_AUTH refuses password logins; the only way in is now
the OIDC flow, which folds into the local account by email. The
ui.enable_login_form config-DB override is flipped to false in the
same change (the DB wins over env, so both must agree).
Footer link opens the authentik default-user-settings-flow in a new tab,
giving users a self-service entry point to change their password and
manage MFA/credentials without admin intervention.
systemd splits unquoted Environment= values on whitespace, so the
quadlet quotes OAUTH_SCOPES to keep all four scopes; update the bats
assertion to expect the quoted form (it was checking the old
unquoted line and failing).
Records the LDAP-Auth plugin wiring: the Jellyfin LDAP provider +
outpost on Dankclaw, the svc-jellyfin-ldap service account, the
jellyfin_users/jellyfin_admins access groups, and the key gotcha that
the bind credential is a PASSWORD (set via Recovery), not an API/app
token — the default auth flow's standard-password backend rejects
tokens with 'flow error password: Invalid'. Verified end-to-end:
alex (in jellyfin_users) logs in with their authentik password and the
plugin creates the matching Jellyfin user.
systemd splits unquoted Environment= values on whitespace, so
'OAUTH_SCOPES=openid email profile roles' arrived as 'openid' and the
authorize request carried a single scope — the id_token then lacked the
email claim and the callback failed. Quoted, all four scopes flow.
Also: assign the default openid/email/profile ScopeMappings to the
provider (2026.x no longer auto-creates them; without them the token
grant is the intersection with the provider's mappings = roles only,
and userinfo 403s).
fill-verify-Enter with a single inputValue read per stage (identification
and password); the toHaveValue assertion hard-failed every attempt when the
re-render wipe landed inside its 5s window (00-auth.setup failure 2026-09-24)
Boot 2026-09-22 at 0.98 fill: KV pool auto-sized to 425,047 tokens (1.62x
the 262,144 window) with 14.42 GiB per card available for KV. The DFlash2
drafter on the Swift finetune measures mean acceptance length 7.00 / 85.7%
(easy) and 5.08 / 58.3% (varied) — it survives the base-model change.
Replaces the estimated ~420k / ~17.6 GiB figures with the measured values.
0.99 failed the vLLM startup memory check before weight load: the R9700
reports 31.86 GiB total but only 31.23 GiB free at device-init (the ROCm/HIP
context + NCCL + radiance custom all-reduce consume the 0.63 GiB gap), so
0.99 (31.54 GiB) exceeded free and the container crash-looped. 0.98 (31.22
GiB) is the ceiling and the prior production value.
Also fixed the runbook KV math that had shipped with the 0.99 change: the
~730k pool estimate was impossible (MXFP6's 25.16 GiB weights > MXFP4's
19.37 GiB, so the pool is smaller, not larger than MXFP4's 486k) and the
34.2 GiB/card figure was not the R9700 total. Recomputed ~420k tokens at 0.98
fill.
The Qwen-Image 2.1 VAE is a video VAE (5D convs, time_conv, depth-3).
ComfyUI detects it as WanVAE (latent_dim=3), but EmptyLatentImage produces
a 4D latent. The decode function's memory_used_decode lambda accesses
shape[4] which is out of range for 4D.
Fix: expand 4D latents to 5D in the decode function when latent_dim==3.
Added to the patch script for persistence across container recreations.
The serve script now passes --chat-template-file pointing at the
Qwen-Sharp v22.4.0 template. Without it the binary falls back to the
built-in peg-native template, which produces a ~73,600-token prompt for
a ~250-token request. With the sharp template wired in, the same request
produces a 62-175-token prompt.
The serve script only passed --chat-template-kwargs, so the binary fell
back to the built-in peg-native template. A 250-token request produced a
73,600-token prompt (280x the expected ~260). Wiring in --chat-template-file
drops the prompt to 62 tokens. Draft acceptance 0.79 (mean 4.14), model
answers correctly, reasoning lands in reasoning_content.
The qwen_image21 detection check and GGUF node arch list patch are
applied by a startup wrapper (comfyui-start.sh) that runs before
ComfyUI starts. The patch script and wrapper are baked into the image
so they survive container recreations. The quadlet's Exec now points
at the wrapper instead of main.py directly.
The hermesadmin.franksdank.cloud dashboard (9119 — config, API keys,
sessions) is a distinct admin surface from the web UI client. Add it as a
third AI-section card; probe 9119 (302 login redirect counts as up, verified
on Dankclaw).
The admin Infrastructure section carried the retired 'dnsmasq' (probe-only,
no link) — Pi-hole FTL (a dnsmasq fork) replaced it on 2026-08-24. Replace it
with a linked Pi-hole card (pihole.franksdank.cloud, probe 8083 — the 403
basicauth counts as up). The Hermes card was probe-only; add the web UI
client link (hermes.franksdank.cloud). Both probes verified up on Dankclaw.
Knowledge-base follow-up to a45f396. The runbook, both index one-liners, and
the AGENTS.md topology row all still declared FP8 active; flipped to
just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq with unquantized bf16 KV (pool
486,464 tokens, full 262,144 window), and recorded the causal nuance: fp8 KV
was on both arms, so the FP8-vs-MXFP4 gap was weight precision, not KV - clean
KV is a quality hypothesis for W4A8, not a throughput one. No verdict yet.
The quadlet's "TEXT ONLY ... 400 here" comment was wrong: vision works on the
MXFP4 checkpoint (Qwen3_5ForConditionalGeneration carries the vision tower
through the Gemma4 merge path, unquantized) - verified by naming a solid red
48x48 PNG "Red". Comment corrected; harness models.yml input stays [text,image].
Docs-only; no re-deploy (the endpoint already serves vision).
Swap the serving checkpoint back to
just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq (quark/W4A8) and serve it with
KV_DTYPE=auto (unquantized bf16 KV) instead of the hardcoded fp8. fp8 KV
stacked on MXFP4's already-quantized W4A8 weights compounds attention-path
precision loss; clean KV tests whether that was hurting output quality. The
freed MXFP4 weights (19.37 vs 30.9 GB) buy the KV pool back, so the full
262,144 native window fits unquantized - MAXLEN returns to 262144.
The entrypoint gains a KV_DTYPE tunable (default fp8, the historical behavior),
shared across both quantization paths; the quadlet sets auto. Rollback: SNAP
back to the FP8 checkpoint with KV_DTYPE=fp8 and MAXLEN=192000.
The LLM backends section said two ROCmFPX endpoints; the fleet now serves
three (frank-large/medium/small) with different runtimes. Rewrite it, fix the
gotcha that claimed host.containers.internal does not resolve on a bridge
(it does — verified end-to-end for both on-host endpoints), and add a History
entry for the 2026-09-21 model-picker rebuild.
The per-key config DB already pointed web search at SearXNG (engine=searxng,
query URL http://192.168.0.143:8081/search), but only inside the data volume.
The quadlet now carries the same values as seeding env vars, so a fresh data
dir comes up wired, and the doc records the access path: the internal
host-published port, not the search. vhost — the Anubis->Authentik guard
chain is browser-only, and the bridge egress sits in SearXNG's pass_ip, so
the 4-req/IP/hour JSON cap never applies.
The user page now orders Web Apps, Media Library Apps, AI — the models sit
at the bottom, below the apps people click. The AI card and /docs#models
descriptions drop the infrastructure (runtimes, GPUs, tensor parallelism,
draft heads) in favor of what users care about: the hosted model, its
size, the context/parallel capacity, and what it's good at.
The build default is 8192 MiB. dankclaw is memory-tight (frank-large's
70 GiB GTT plus the rest of the stack) and Gemma4 runs fully on the 3090
(-ngl 999), so the CPU-side cache is not the hot path. Cap it at 2048 to
give back ~6 GiB of LPDDR. Gemma4 is not a DeltaNet model, so the
>12.8 GiB CACHE_RAM floor does not apply.
Wired through frank-small.env (CACHE_RAM) -> serve-frank-small.sh
(--cache-ram), with a wiring seam so the cap cannot be silently unwired.
Each section's panels fill an auto-fill row (as many per row as the
viewport allows) and the sections stack vertically — the side-by-side
column layout was the wrong read of 'render the sections horizontally'.
The board rendered vertically for a user whose browser had cached the
pre-sectioning style.css: deploys overwrite /srv/homepage in place, and
nothing told the browser to revalidate. The @home handler on the post-auth
listener now sends Cache-Control: no-cache, so every load is an ETag
round-trip (a cheap 304 when unchanged) and a deploy is always visible on
the next navigation.
The first cutover crash-looped the unit with 'invalid split file name':
the fetch script concatenated the three split parts into a single GGUF,
but part 1's header carries ggml.file_count=3, so the byte-correct
concatenation still advertises itself as a split and the loader demands
a -NNNNN-of-NNNNN filename.
- serve-frank-large.sh: -m points at part 1; llama.cpp loads the split
natively (parts 2-3 auto-discovered in the same directory)
- deploy-remote.sh: LLM_LARGE_MODEL gate path follows
- download-frank-large-unsloth.sh: assembly removed; the parts ARE the
canonical payload, verified against live Hub LFS oids
- docs: runbook model section + troubleshooting note, AGENTS.md, log
- the redundant 87 GiB assembled file is staged on dankclaw at
~/.AGENTTRASH/20260921-frank-large-assembled/
Both pages now group their panels under section headings rendered by shared
renderSection()/renderPanel() components in app.js (SECTION_ORDER fixes each
page's order: user = AI, Web Apps, Media Library Apps; admin = Media Library
Apps, Observability, AI, Infrastructure). The WIP badge splits into WIP: OPEN
and WIP: CLOSED via the wip value ("open"/"closed"); the checker passes the
value through as wip-open/wip-closed. The frank-large card follows the
unsloth llama.cpp cutover, frank-medium follows the FP8 checkpoint, and the
/docs model guide matches. Tautulli (Plex statistics, 192.168.0.235:8181)
joins the admin page's Media Library Apps section with an open link.
Only the 8084 forward is exposed; the desktop GUI runs inside the
container netns and is internal-only. Drop the now-unneeded
CUSTOM_HTTPS_PORT (no host-port conflict exists inside the netns).
lscr.io/linuxserver/calibre:9.15.0 (desktop build) running calibre-server
as the s6 main service on internal 8090 -> host 8084, auth-gated with a
plaintext userdb (mode 600) bootstrapped once by the deploy handler.
The desktop GUI is kept on 8080/8182, basic-authed with the same
credential pair via the machine-local env file.
The verification looped over every LFS file under UD-IQ4_XS/ and MTP/ in
the live Hub tree — the five MTP head variants we do not fetch (BF16,
Q4_K_M, Q8_0, shared-BF16, shared-Q4_K_M) printed MISSING, set ok=False,
and set -e killed the script silently between verification and assembly
after a 3-hour download. Verify exactly the FILES set instead (passed to
python as argv), and report NOT IN HUB TREE when an expected file has
vanished upstream. Also fix the layout comment: the transport parts live
in UD-IQ4_XS/, not parts/.
New runbook frank-large-unsloth-qwen38-flash-next.md (runtime, payload,
deployed config, the explicit -md, the 114 GiB GTT memory model, client
contract, rollback). frank-large-ciru-orca.md marked RETIRED (rollback
target, payload and runtime intact on disk). AGENTS.md topology row,
frank-large section, harness note, runbook index, and the GPU-budget
lesson (GTT aperture ~114 GiB, not the stale 64 GiB rocm-smi figure —
the CIRU Orca held 92.6 GiB on the same device). LLM index header,
mermaid, active list, and the vault root index updated. Work-log entry.
Replace the CIRU v4.4.1 Orca runtime with the unsloth llama.cpp prebuilt
(b11030-mix-5ff778e, ROCm gfx1151, bundles its own ROCm) serving
unsloth/Qwen3.8-Flash-Next-GGUF:UD-IQ4_XS (87.3 GiB, three HF split parts
concatenated) on the Strix Halo iGPU. One conversation, 200,000 ctx,
PARALLEL_SLOTS=1, MTP depth 2 via the explicit -md shared-Q8_0 head
(sidecar auto-discovery does not search the mtp/ subfolder), text-only.
Fleet contract unchanged: port 7770, alias frank-large.
- serve-frank-large.sh: unsloth binary, -m/-md, --spec-type draft-mtp
--spec-draft-n-max 2, --jinja, preserve_thinking; gates on binary,
GGUF, MTP head; conditional --mmproj on ENABLE_VISION=1
- download-frank-large-unsloth.sh: resumable fetch of 3 parts + MTP head
+ mmproj, each verified against its LFS sha256 fetched live from the
Hub tree API; concatenates parts, records/verifies SHA256SUMS
- deploy-remote.sh: LLM_LARGE_MODEL/LLM_LARGE_DRAFT paths, pkill pattern
on the model path (not the binary name), wait_port_free gate
- tests: seams for 200k/1-slot, explicit -md, text-only, handler gates,
sweep safety
Runbook deployed-flags table and AGENTS.md now name the template
(froggeric v22.4.0, repo-tracked, terse by default) and the rollback
target; the stale qwen-fixed-v22.3.jinja row and the long-gone
MM_MERGE_ENGINE note are gone.
The template was staged in the working tree (then stashed by a peer for the
prometheus deploy) but never committed, so the live container kept serving
/patches/qwen-fixed-v22.3.jinja. Commit the repo-tracked template and flip
the quadlet: CHAT_TEMPLATE now points at it, mounted read-only at
/templates/qwen-sharp-v22.4.0.jinja. The entrypoint already passes
--chat-template on the shared exec path, so this applies to the FP8 shape.
Rollback: one env flip back to /patches/qwen-fixed-v22.3.jinja.
node-exporter is net=host, so its 192.168.0.143:9100 bind is a host bind,
not a published port — the bridge-networked Prometheus container cannot
reach its own host's LAN IP there (netavark DNATs published ports only).
The node-dankclaw target has been red by construction since the
observability cutover while the container reported healthy. Point it at
host.containers.internal:9100 (netavark delivers that to the host LAN IP,
where node-exporter listens) — the same rule frank-large and frank-small
already use. node-dankserver is a different machine and keeps its LAN
address. Update the cross-file contract test and the Targets table.
DFlash2 (100.0 t/s decode, 287.1 t/s burst-4) beats MTP self-speculation
(85.9 t/s, 179.7 t/s) despite a lower accept length (4.24 vs 5.66): the
small separate drafter steps faster than the target's own MTP layer applied
autoregressively. DFlash2 is the deployed shape; both swap-script paths now
default to dflash/7. Docs updated.
The entrypoint now reads the model's weight quantization from config.json
and serves the plain FP8 path (no MXFP4 patches, --quantization=fp8, MTP
self-speculation, ROCM_AITER unified attention with the R4D all-reduce)
or the MXFP4 patch-and-serve path. A model swap is now: change SNAP
(+ SPEC_METHOD/SPEC), restart.
- radiance-entrypoint.sh: quantization detection, gated MXFP4 env vars
and patch block, --quantization fp8 flag
- quadlet: SNAP -> Qwen3.8-27B-Uncensored-FP8, SPEC_METHOD=mtp, SPEC=8,
KV_MEM=0 (profile the shape, the documented plain-FP8 default)
- deploy-remote.sh: model-aware gates (read SNAP/SPEC_METHOD from the
quadlet; drafter gate only for dflash, r4d.so gate only for mxfp4)
- tests: quantization-aware drafter/spec and KV-pool tests
- hf-fetch.sh: generic, resumable, verified, idempotent HF fetch tool
- frank-medium-fp8.manifest: 17 files (10 config + 7 shards) with sizes
and LFS sha256
- shared/prometheus/rules/llm-fleet.yml: the llamacpp reuse recording rule
divided cached by prompt, but llama.cpp's prompt_tokens_total EXCLUDES
cached tokens (its own HELP line), so the ratio was unbounded — measured
77 on a fully-warm frank-large, meaning LlmPrefixCacheCollapse could never
fire on that side. Now cached / (cached + prompt), with the traffic floor
counting both terms too; the header documents the unit trap. Also fixed the
LlmEndpointDown description: frank-large's CIRU llama.cpp build DOES export
the llamacpp:* family (verified live), so the alert covers all three
endpoints.
- shared/grafana/provisioning/dashboards/llm-fleet.json: re-cut for two
namespaces. Every panel reads both (the endpoint picker selects); the
KV-pool-fill panel retired 2026-09-16 returned with
vllm:kv_cache_usage_perc; the latency p95 panel returned with the vLLM
histograms (TTFT/ITL/e2e); the alerts panel is scoped to the fleet's
Llm.* rules; the stale halogen-era description is gone. All 21 panel
expressions validated against the live endpoints.
- runbook + log + Observability: prefix caching measured, not inferred —
reuse happens at 1,648-token block granularity (attention block size padded
to the mamba page size); an identical 13,021-token prompt went 2.30 s cold
to 0.68 s warm with 9,888 tokens (76 %) reused, while a 2,521-token prompt
registered zero hits. Rule counts corrected to 6 alerts + 3 recording rules.
./tests/run.sh: 325 pass.
The radiance TP=2 cutover changed the metric namespace frank-medium exports
(llamacpp:* -> vllm:*) and left two scrape targets addressing host processes
by an address a container cannot reach.
- shared/prometheus/rules/llm-fleet.yml: two namespaces side by side. Added
LlmRequestsWaiting, LlmKvCacheSaturated (vllm-only gauges llama.cpp never
had), the job:llm_vllm_prefix_cache_reuse:rate5m and
job:llm_spec_accept_length:rate5m recording rules, and re-cut
LlmPrefixCacheCollapse / LlmSpecDecodeOff to OR both families with
per-family traffic floors so an idle endpoint cannot fire them. Every
expression validated against live Prometheus.
- shared/prometheus/prometheus.yml: frank-small -> host.containers.internal:7771
and frank-large scraped at host.containers.internal:7770. Both are HOST
processes on Dankclaw and netavark DNATs published container ports only, so
the LAN-IP targets were red by construction; frank-large's CIRU llama.cpp
build does export the llamacpp:* family, contra the old comment.
- tests/observability_quadlet.bats: the two seams now encode the new contract
(per-endpoint reachability by process type; every named series must be on the
measured list for its engine; engine-independent rules must name both).
- homepage docs.html/services.json: frank-medium sees pictures, frank-large is
CIRU Orca with a single seat (was halogen w4b, two seats, 200k).
- docs: radiance runbook is ACTIVE (deployed flags, three-arm battery, verified
capabilities, rollback, replica-era sections labelled); llama.cpp runbook
carries the SUPERSEDED banner; Dual-R9700-Cooperation states the measured
verdict; Observability records the namespace split and the DNAT lesson;
AGENTS.md topology/frank-medium/Model Config/Benchmarking Policy/runbook
index flipped; both indexes updated; work-log entry added.
Also repaired agent-pass damage: prose prepended above frontmatter openers,
duplicated runbook bodies, a literal tool-call token pasted into a runbook, and
an index destroyed by diff artifacts (restored from HEAD, flips re-applied).
Vault link check: 128 pages, 0 unresolved. ./tests/run.sh: 325 pass.
The conversation battery (scripts/bench-conversation.py, salted, same 10
prompts, 2026-09-20) measured three arms on the two R9700s:
arm I llama.cpp layer split 32:33 + DFlash2, Q8_0 geo decode 45.7 t/s, TTFT 0.72 s, burst-4 71.7 t/s
arm L llama.cpp tensor split 1:1 (RCCL), no drafter geo decode 30.5 t/s, TTFT 0.71 s, burst-4 54.3 t/s
arm R radiance vLLM TP=2 + DFlash2-FP8, MXFP4 geo decode 95.8 t/s, TTFT 0.21 s, burst-4 259.4 t/s
Radiance wins every axis, so it takes the alias. The board swap is what made
arm R possible: TP=2 was archived on 2026-09-16 because the second card sat
behind B650 chipset bridges with AtomicOpsCap: Routing-, and the ProArt
B850-Creator put both cards on CPU GPP root ports (Gen5 x8, Routing+, ReqEn+).
The R4D one-shot all-reduce needs no RCCL collective anyway.
- un-retire dankserver/frank-medium/: the TP=2 quadlet (banner rewritten —
it is the deployed shape now), radiance-entrypoint.sh, the healthcheck.
The replica pair and its affinity proxy stay archived; one process owns
both cards now.
- retire scripts/frank-medium.{service,env} + serve-frank-medium.sh and
bench-spec-ab.py to scripts/.AGENTTRASH/20260920-llama-layersplit/ as the
rollback path. build-llama-rdna4.sh and download-frank-medium-gguf.sh
stay: they are the tooling that rollback needs.
- deploy-remote.sh: deploy_llm_medium now gates on the image (pull once,
re-tag to the localhost name the quadlet resolves), the MXFP4 checkpoint,
the drafter, the pinned patch clone and the patched libr4d; retires the
llama root unit into the home trash instead of relinking it; then
install_quadlet + wait_container_healthy 1800s (first start compiles).
retire_vllm_medium_quadlets becomes retire_vllm_replica_quadlets — the
a/b/proxy trio only, since frank-medium is installed again.
- planner + test-changed map dankserver/frank-medium/** to frank-medium
and its seam; the scripts-trio mapping is gone with the files.
- tests/frank_medium_unit.bats rewritten for the container contract: boot
persistence of a GENERATED unit (no enable on this host's v1 generator),
TP=2 with the atomics-era NCCL workarounds kept out, the DFlash2 depth vs
the MXFP4 decode-kernel row band, the KV pin's coupling to the shape it
was calibrated at plus pool/VRAM arithmetic, the endpoint/tool/reasoning
contract, and the archive invariants.
- homepage card + docs updated, and they were stale twice over: they still
described the ajgazin Swift 35:30 / 393k payload, not the deployed shape.
Known regression, documented in the unit: vision. The MXFP4 checkpoint is
language_model_only and the launcher loads no projector, so images are a 400
here; the harness's vision role already routes to frank-small.
The warm-up request was a warm prefix-cache hit on the next run, so it
hid exactly what the arms were meant to differ on (prefill). Token
counts now come from the streamed usage of each job itself, and --salt
defeats cross-arm inheritance.
bench-conversation.py answers 'what does this endpoint feel like': ten real
jobs streamed once, TTFT split from decode tok/s, salted so no arm inherits a
warm cache, optional concurrent burst for capacity. Needed to compare the two
tensor-parallel runtimes the 2026-09-20 board swap unblocked.
dankserver-route-primary.sh pins eno1 (the fleet address, .235) as the
preferred default route after the board swap brought up eno2 with DHCP and a
lower metric; eno2 stays up as failover. nmcli device reapply, not connection
up, so the link this session rides is not torn down.
librccl DT_NEEDEDs librocm-core.so.1, so a GGML_HIP_RCCL=ON binary would not
exec from its bundled lib dir (measured: 'error while loading shared
libraries: librocm-core.so.1'). The layer-split build never pulled it in.
Layer split needs no collectives, so RCCL stayed OFF. The 2026-09-20 board
swap put both R9700s on CPU root ports with atomic routing, so a
tensor-parallel (row-split) runtime is now buildable: pass
GGML_HIP_RCCL=ON with a distinct BUILD_DIR beside the production one.
Reanalysis after the 2026-09-20 swap to an ASUS ProArt B850-Creator WiFi
NEO: both R9700s now sit on CPU GPP root ports (x8/x8 Gen5 bifurcation),
every bridge reads AtomicOpsCap Routing+, and both endpoints read
AtomicOpsCtl ReqEn+ (CAP_EXP+28.w = 0x1440) — the exact precondition
Dual-R9700-Cooperation's test plan named. TP=2 is therefore unblocked,
not proven: no collective has been dispatched since the swap,
GGML_HIP_RCCL is still OFF, and the 272 t/s "tensor" number on record
measured a butterfly fallback.
- Dual-R9700-Cooperation: short answer rewritten, re-measured 2026-09-20
topology table, rebuild-hypothesis door marked moot, probe plan is now
standing work.
- Dankserver System Specs: Motherboard section; GPU BDFs 0b:00.0 ->
07:00.0, iGPU 10:00.0 -> 11:00.0, kernel 7.0.0-31, SMT on, and the new
eno2 (192.168.0.247) second 5GbE port holding the low-metric default
route while eno1 keeps the fleet's static .235.
- frank-medium-llamacpp-dflash2: rationale and hardware row split past
from present; dated work-log sections stay as history.
- AGENTS.md + both indexes: no longer assert a wall that no longer exists.
Docs only; frank-medium stayed active and healthy throughout (verified
/health). ./tests/run.sh 331/331.
The frank-medium runbook's deployed-flags table still carried the
predecessor's values (35,30 / 393216 / 196608 / draft-mtp / BF16 mmproj);
updated to the deployed shape (32,33 / 327680 / 163840 / draft-dflash / f16
mmproj) from scripts/frank-medium.env. The halogen runbook still described
itself as 'the active frank-large runbook'; marked RETIRED 2026-09-19 with a
pointer to [[frank-large-ciru-orca]].
The CIRU launcher gates on hip/lib/libamdhip64.so + rocr/lib/libhsa-runtime64.so
under its runtime root and exits 2 without them; the build links the private
venv through its RUNPATH, so point those entries at the venv's unversioned
devel symlinks (idempotent ln -sfn). Without this the unit crash-looped for
the full 2400s health gate on first deploy.
Operator call 2026-09-19: switch the iGPU endpoint from
halogen-flash-server to jcbtc/Qwen3.8-Flash-CIRU-STRIX-Orca on the CIRU
v4.4.1 runtime with the QSA window hotfix (host build against the private
ROCm 10 SDK), one stream of the full 262144 trained window. The service
leaves the quadlet shape for a root unit (the same shape as frank-small and
frank-medium: a private-SDK host binary has no image to install and no
generated unit to dodge); the halogen quadlet is archived, the deploy
handler retires it on the host, gates on 7770, and its stray-run sweep
matches the model path so it cannot take frank-small down.
Operator call 2026-09-19: switch the target to
orcarouter/Qwen3.8-27B-Uncensored-GGUF Q8_0 (gated repo; the fetch script now
requires an HF token and verifies sha256, not just size), take the drafter to
the Q4_K_M rung of z-lab/Qwen3.8-27B-DFlash2-GGUF, cut the pool to two
163840-token conversations (327680, entirely inside the trained 262144
window), and balance the tensors across the cards (32:33) in place of the
35:30 x16-lean. The in-file MTP head stays one SPEC_TYPE change away.
The vLLM detour (a2b39b1, same day) is retired: the Q4_K_M llama.cpp setup
is the requested shape and the vLLM KV-quant options on sm_86 all measured
dead (fp8 hardware-impossible, int8 page-size unification, turboquant
mm-prefix). Root unit restored, vLLM quadlet/Dockerfile/fetch script moved
to .AGENTTRASH, runbook listed as same-day-reverted. Rollback target:
frank-small-vllm-gemma4.
int8_per_token_head: page-size unification fails (scale tensor breaks
the 256/512 head-dim ratio). turboquant_k8v4: backend validation fails
(no mm-prefix attention, which Gemma4's vision/audio path requires).
fp8: hardware-dead on sm_86. bf16 it is; history documented in
frank-small.env, seam pins auto + no backend pin.
The VLLM_ATTENTION_BACKEND env var does not exist in this vLLM dev
build (warns 'Unknown vLLM environment variable' and is ignored), so
the pin landed nowhere and Gemma4's hook force-selected TRITON_ATTN
again. Pass --attention-backend TURBOQUANT as a CLI flag instead.
Gemma4's config hook force-selects TRITON_ATTN for the mixed 256/512
head dims whenever no backend is selected, and TRITON_ATTN rejects the
turboquant KV dtype (measured: 'kv_cache_dtype not supported', crash
loop). The TURBOQUANT backend accepts any head size and the
turboquant_* dtypes, and the Gemma4 MTP speculator carries the
target's selected backend through to the draft layers.
int8_per_token_head keeps a separate fp32 scale tensor per (token,
head); Gemma4's hybrid head dims (SWA 256 / full 512) make the
per-token page bytes 4160 vs 8256, not an integer ratio, so
unify_kv_cache_spec_page_size raises NotImplementedError — nine
crash-loop restarts measured. TurboQuant k8v4 packs its block scales
inside the tensor, keeps the 2:1 page ratio, and has a dedicated
backend with an explicit Ampere path. 289,647 -> ~772k-token pool.
fp8 KV is hardware-dead on the 3090 (sm_86, no cvt.e4m3x2), so int8 is
the cheapest quant the TRITON_ATTN backend can run: pure ALU dequant,
same backend, no fp8 tensor cores needed. 289,647 -> ~579k-token pool,
two -> four full-context sessions. Quality verified against the bf16
baseline (needle-in-haystack at 213k + short battery) before the swap.
podman 5.7.0's v1 quadlet generator rejects both ContainerEntrypoint
and Entry; the key is Entrypoint= (pkg/systemd/quadlet/quadlet.go
KeyEntrypoint, mapped to --entrypoint).
The podman 5.x v1 quadlet generator on dankclaw rejects
ContainerEntrypoint with 'unsupported key in group Container', so the
unit was never generated and the deploy died at restart with 'Unit
frank-small.service not found'. Entry= is the v1 spelling.
Runtime swap on the RTX 3090 (port 7771):
- vLLM gemma-x86_64-cu129 dev build, digest-pinned, audio extra pip-installed
into localhost/vllm/vllm-openai:frank-small (Dockerfile, no COPY lines)
- google/gemma-4-12B-it-qat-w4a16-ct (10.26 GiB int4) + Google's bf16 MTP
assistant (845 MB) — the w4a16-ct assistant cannot draft in vLLM (drafter
path skips Marlin; acceptance 1.00), bf16 measures 2.80-3.56
- root systemd unit -> rootless Quadlet (dankclaw/frank-small/quadlet/);
manual CUDA device+lib mounts (no nvidia-container-toolkit on host)
- bf16 KV (fp8 hardware-dead on sm_86), 289,647-token pool, 2 concurrent
full-context sessions, CUDA graphs on
- native text+image+audio: the fleet's multimodal endpoint
- measured: pp@67k 1293 t/s, pp@213k 608 t/s, tg@213k ~13.6 t/s,
tg@85k ~17.6 t/s
- host driver 610.57.04 (Ubuntu 26.04); sm_86 is correct, no arch override
- planner/scoping: frank-small now selected by dankclaw/frank-small/ +
env/serve/download trio; old unit retired to scripts/.AGENTTRASH/
- docs: new runbook frank-small-vllm-gemma4.md; AGENTS.md, LLM index,
top docs index updated; old runbook moved to retired
--cache-ram caps the total bytes of saved prompt states in HOST RAM, and
alloc() refuses any single state larger than the whole cap. That refusal was
firing in production: 137 times in the two days from 2026-09-16, at 11.0-11.4
GiB per state, so every conversation past ~170k tokens was uncacheable and
re-prefilled from scratch on its next turn.
The state law at this shape is 68 KiB/token (16 full-attention layers x 4 kv
heads x 256 dim x 2 B x 2 for K/V, plus the MTP draft context's 4096 B), so
8192 MiB held ~120k tokens and 16384 holds ~240k: a FULL 196608-token slot
state (~12.8 GiB) now fits with ~3 GiB spare.
Host RAM only, allocated on demand and never pinned, so the VRAM fit is
untouched. New seam: the cap must clear one full slot's state and stay under
half the host's RAM (it fails on the old 8192 and passes on 16384). Also
corrects the adjacent claim that 0 is the build's default for
--ctx-checkpoints -- this binary's --help says 32.
Isolated-port (:7790) 2k-token decode with AVG rocm-smi util over the
request (peak util lies: a layer split touches both cards every token):
35:30 draft ROCm1 31.52 t/s util0 41% util1 98% <- deployed
36:29 draft ROCm1 31.49 t/s util0 43% util1 98%
35:30 draft ROCm0 31.52 t/s util0 42% util1 98%
36:29 draft ROCm0 31.48 t/s util0 43% util1 98%
The x1 card (ROCm1) stays the 98% bottleneck in every configuration.
Moving the MTP drafter to the x16 card does not flip the balance: the
per-step cost is the verify phase (both cards run the full target) plus
the logits, which are pinned to the card holding the last layers. A
target-layer move (36:29) buys 2 points on the x16 card and erodes its
VRAM margin (fit free 0.45 GiB) for no throughput change. The x1 card is
compute-bound, not PCIe-bound (~5.2 MB activation per ubatch on the link),
so saturating it costs no tokens. 35:30 is the balance; no split change.
A pending entry for frank-default-sightsound (FLM NPU endpoint retired
2026-09-11) survived in the dankclaw state store and made every --pending
run exit 2 at the dispatcher before the Playwright e2e gate could
execute. The pending pass now drops entries the target no longer allows
into ~/.AGENTTRASH and keeps going. Behavioral seam test added.
- services.json: frank-large/medium/small move from admin-only to the user
page with current descriptions (halogen w4b iGPU; 35:30 layer-split MTP
27B on the R9700s; Gemma4-12B QAT vision path on the 3090) and docs
links to /docs#models
- docs.html: static user documentation (access chain, model guide, media
how-tos, troubleshooting); served at /docs via the post-auth homepage
listener; shipped by deploy_homepage; footer + index link to it
- tests: pages seam asserts the docs contract (page, link, Caddy route,
deployer copy)
- vault: Homepage.md synced (docs landing shipped, cards reclassified)
The two R9700s sit on unequal PCIe paths: ROCm0 (03:00.0) is CPU-root-port
Gen4 x16 (~31.5 GB/s); ROCm1 (0b:00.0) rides the B650 chipset through
bridges negotiating Gen4 x1 (~1.97 GB/s). With the MTP drafter, output
layers and heavier compute buffers pinned to the last-layers card, live
rocm-smi read 98% busy on the x1 card vs 40% on the x16 card. Moving one
DeltaNet layer (34) to ROCm0 shifts ~0.43 GiB off the bottlenecked card;
the seam fit lands 30.59/30.01 with gap 0.58 GiB, inside floor and slack.
36:29 fails both gates, so this lever is spent.
signal-cli 0.14.7 parses an all-digit -a value as a phone number (the
numeric account name never resolves), and the message is a -m option,
not a positional (a positional is parsed as a second recipient).
Verified live: corrected invocation returns type SUCCESS.
Every webhook died on 'cannot access local variable _last_sent' because
the prune comprehension rebound the module global inside the function.
Prune in place instead.
The bridge-subnet-only pair dropped the host's own health probe
(source 192.168.0.143/127.0.0.1) and failed a healthy deploy. The
deployer now installs the full set: ACCEPT 127.0.0.1, ACCEPT the host
IP, ACCEPT 10.89.0.0/16, then DROP.
169.254.1.2 is not a local address on this host (netavark runs its
bridges in aardvark-dns' netns), so the relay cannot bind it — and a
loopback bind refuses container traffic. Probe-socket proof: a host
socket on 192.168.0.143:18094 answers the container's 169.254.1.2:18094
dial while the LAN IP refuses from the container. The relay binds
192.168.0.143:18093 and the deployer installs an iptables INPUT pair
(ACCEPT 10.89.0.0/16, DROP rest) so the unauthenticated endpoint stays
off the LAN.
A socket bound to 127.0.0.1 refuses container traffic that arrives via
the podman bridge: the first live webhook failed with 'dial tcp
169.254.1.2:18093: connection refused'. The relay now binds the
host-gateway endpoint — link-local to the podman bridge, off the LAN —
and the deployer's health probe follows it there.
- runbook: identity now 4x262144, audio verified, throughput table with
the OOM and negative-result rows, deployed flags updated to reality.
- AGENTS.md frank-small paragraph matches the deployed shape.
- log.md: the sweep entry.
- scripts/frank-small.env: my earlier line-range edit had clobbered
NGL=999 and FLASH_ATTN=on out of the file; the seams caught it. The
live service was unaffected because the script's defaults match the
declared values exactly, but the file contract is restored.
Suite: ALL TESTS PASSED (321), vault 125 pages 0 unresolved.
Swept on the 3090 at the deployed 4x262144 geometry, salted API probes:
- ubatch 512: pp@254k ~918 t/s, tg@250k 90-204 (noisy)
- ubatch 1024: pp@254k 962 t/s (+4.8%), tg@250k 257-302
- ubatch 2048: OOM — the MTP draft context needs ~3.3 GiB of pp compute
buffer against 2.2 GiB headroom; do not try.
- draft depth 4 + poll 100 decoded WORSE at 250k (230-257) than 3/50:
the longer verification round re-reads more long-ctx attention than the
extra accepted token earns. Kept 3/50, now with the evidence attached.
Idle VRAM 23,300 MiB; all ubatch-1024 variants idled identically
(compute buffers reserved at max graph size), so 8192 buys the deepest
prefill window for free over 4096.
The 393216/34:31/MTP deploy measured 30.78 GiB on ROCm1 against the
fit's 28.32 — the MTP draft buffers (one nextn layer's KV, 4096 B/pos,
plus draft buffers) came in 2.46 GiB above the estimate. Recalibrate
the seam's MTP constant to the measured 4.20 GiB total charge on card
1 and drop the floor 1.5 -> 1.0 GiB for this shape (ROCm1's 1.08 GiB
headroom is real; 35:30/36:29 fail card 0's floor at this pool, so
34:31 is the only split that fits 2x196k).
AGENTS.md and the runbook move to the MTP drafter: 34:31, 393216 pool
(196608 per conversation), the DFlash2 sidecar documented as a
one-line revert, and a dated part-2 section with the fit/measured
table and the recalibration.
Both were implicit build defaults (3 and 50); the throughput sweep needs
to move them without editing the exec line each time. Defaults preserve
current behavior exactly. Additive only.
DFlash2's sidecar (1.92 GiB + buffers on ROCm1) kept the 393216 pool off the table; the MTP head lives in the target file, so its cost is one nextn layer's KV (4096 B/pos) plus small buffers. Fit: ROCm0 29.77 / ROCm1 28.32 GiB, over the floor, 34:31 the balance point at this pool. n_max 3 is the measured MTP winner. The seam's fit is now drafter-aware, the slack is the deployed integer gap (1483 MiB), the ctx cap 393216 with the rope caveat (top 131k extrapolates on the base rope).
CTX_SIZE=1048576 / PARALLEL=4: every slot gets the model's full trained
window. Measured fit arithmetic on the 3090: 1 full-context slot idles
at 11,996 MiB, 4 at 22,346 -> 3,450 MiB per slot, ~8,546 MiB fixed.
A 5th slot projects 25.8 GiB past the 24,576 MiB card, and the KV pool
allocates upfront, so 4 is the ceiling. Explicit CTX_SIZE bypasses --fit
on purpose: fit maximises the TOTAL pool and would divide a smaller
window across slots at parallel>1. The slot-grid seam now pins both the
full-window division and the budget inequality so a future capacity
bump fails pre-deploy, not in the boot allocation.
Audio verified while measuring: the mmproj's audio encoder transcribes
input_audio parts verbatim (JFK WAV sample). Header comments updated.
AGENTS.md topology, Active Services, the CUDA-quant-KV lesson (f16 is no
longer why), and the runbook index. New frank-small-gemma4-12b runbook:
payload digests, the SWA/q8 arithmetic behind --fit's 262k window, the
MTP-sidecar + ngram-mod chain, the always-accepted reasoning_effort, and
the fit-probe error that looks like a fault at boot. The Qwen runbook and
both indexes carry the retirement; work log updated.
AGENTS.md topology row and section, and the runbook: the pool is 327680
positions (163840 per conversation over two slots), the split is 36:29 -- one
layer past the 34:31 balance at this context, on purpose, for headroom on the
drafter card (fit 29.19/26.04, measured 28.40/26.66 GiB at load, 3.46/5.20
headroom) -- and vision is on via the explicit --mmproj flag, because this
build has no autoload (the first deploy came up text-only and 500'd image
requests). The rope caveat is written down where the cap lives: the GGUF has
no scaling keys, so the top 65k of the pool extrapolates on the base rope.
The Qwen3.8-27B IQ3_S file carried its own MTP head; Gemma4's ships as a
separate assistant file (mtp-gemma-4-12B-it.gguf, arch gemma4-assistant,
4 nextn layers) that loads via --model-draft and shares the target's last
layers' KV. The draftless ngram-mod matcher chains behind it in the same
--spec-type value — two context-backed drafters cannot share one process,
so this chain is the only shape that carries both.
KV is q8_0 (16 KiB/token/layer vs 32 at f16) and CTX_SIZE is unset so
--fit sizes the pool against what the 7.4 GiB of payloads leave on the
24 GiB card. build-cuda was rebuilt with -DGGML_CUDA_FA_ALL_QUANTS=ON,
which quantized KV on CUDA requires. The three seams that pinned the Qwen
contract (MTP-in-file, the Qwen mmproj name, the f16 KV arithmetic) now
pin the Gemma4 contract instead, and the deploy handler's presence check
points at the new payload.
The 36:29/327680/vision deploy came up healthy and text-only: image requests
answered 500 'image input is not supported - hint: ... provide the mmproj'
with no mmproj line in the load log, so llama-server fb27a525 does not
autoload mmproj*.gguf from the model directory. Name the file, the way
frank-small's serve script already does, and pin the flag in the seam.
2026-09-17 01:54:26 -07:00
464 changed files with 46782 additions and 6081 deletions
"text":"A detailed illustration of a 1930s Soviet industrial design bureau at night: a well-measured engineer at a drafting table, a warm desk lamp, rolled blueprints, gears and calipers on the desk, deep red and cream palette, gouache painterly style, intricate mechanical detail, soft shadows",
"clip":["6",0]
}
},
"4":{
"class_type":"CLIPTextEncode",
"_meta":{"title":"Negative prompt"},
"inputs":{
"text":"lowres, blurry, deformed, extra limbs, watermark, signature, jpeg artifacts",
"text":"Clean minimalist movie-poster design, dark charcoal background with a subtle vintage grain. Centered large bold white sans-serif text reading \"SOLIDARITY OF THE SOVIET COMPUTER SYSTEMS\", perfectly legible, crisp letterforms, even letter spacing. Below it, a smaller line of yellow text reading \"DANKCLAW DANKSERVER EST 2026\", equally legible. Flat graphic design, high contrast, generous margins, no other words, no watermark",
# Reported by GET /v1/models, and what scripts/frank-tts.env's
# TTS_UPSTREAM_MODEL names. A model id, not a filename: the file name would
# leak a quantization choice into every consumer's config.
: "${QWENTTS_ALIAS:=frank-tts-engine}"
# 0.0.0.0 inside the container is not "public". tts-server defaults to
# 127.0.0.1 (its own usage prints `--host <ip> Listen address (default:
# 127.0.0.1)`), and inside a container netns a loopback bind is reachable by
# nothing outside that namespace — the façade dials this container by its bridge
# address on the private voice network, which is why the address has to be
# broad. The unit PUBLISHES NOTHING (see
# dankclaw/voiceservice/quadlet/frank-tts-engine.container: no PublishPort at
# all, not even 127.0.0.1:8880), so the only namespaces that can reach
# 0.0.0.0:8880 are that bridge and this container's own loopback — and its own
# loopback is exactly where the boot warmer and the health probe speak.
: "${QWENTTS_HOST:=0.0.0.0}"
: "${QWENTTS_PORT:=8880}"
# The language label used when a request omits one. `auto` is the upstream
# default and what the trial ran.
: "${QWENTTS_LANG:=auto}"
# One request at a time. The server-side queue is a serialised FIFO at this
# setting, which is what keeps the measured per-voice timings true; batching
# on a CPU build splits the same cores between two syntheses and the warm RTF
# margin (0.55-0.67 against real time 1.0) disappears. The engine's own
# default is also 1; it is written here because it is a decision, not an
# accident.
: "${QWENTTS_MAX_BATCH:=1}"
# Fail loudly BEFORE binding, so the journal says which artifact is missing
# instead of the unit reporting a container that starts and dies. This is the
# self-healing path: Restart=always re-runs it every RestartSec, and the same
# sentence repeats until the operator fetches the weights
# (scripts/download-frank-tts-engine-qwentts.sh).
for artifact in "$QWENTTS_MODEL""$QWENTTS_CODEC";do
if[[ ! -r "$artifact"]];then
echo"frank-tts-engine: cannot read ${artifact}; run scripts/download-frank-tts-engine-qwentts.sh on the host (the /models mount is read-only and holds the two GGUF files)" >&2
This document describes the deployment of a subsurface aquifer monitoring network for the Cascadia sedimentary basin. It is written for the operations crew and the regional water authority. Every sensor node, relay, aggregation service, calibration procedure, and failure mode below carries a stable alphanumeric identifier that appears in every table and in the field notes.
## Sensor nodes
Node SN-000-08B9 sits in a discharge lens at 359.0 m below datum and reports a salinity of 2.38 ppt with the fractured interval reading 5.40 degC at dawn and the wellhead porous 1.72 m3/d while the electrode pair returns 3157 uS/cm.
Node SN-001-0A65 sits in a percolate lens at 25.9 m below datum and steps down to a salinity of 2.13 ppt with the diapir interval reading 8.55 degC at dawn and the wellhead silty 0.67 m3/d while the electrode pair returns 979 uS/cm.
Node SN-002-0571 sits in a diapir lens at 287.8 m below datum and peaks at a salinity of 7.63 ppt with the karst interval reading 4.26 degC at dawn and the wellhead spooned 11.50 m3/d while the electrode pair returns 2987 uS/cm.
Node SN-003-0CE9 sits in a confining lens at 169.5 m below datum and logs a salinity of 0.96 ppt with the alluvial interval reading 5.03 degC at dawn and the wellhead salinized 12.08 m3/d while the electrode pair returns 2607 uS/cm.
Node SN-004-0E2A sits in a freshened lens at 70.4 m below datum and settles near a salinity of 8.68 ppt with the spooned interval reading 4.88 degC at dawn and the wellhead siphon 4.38 m3/d while the electrode pair returns 2707 uS/cm.
Node SN-005-0DDD sits in a percolate lens at 44.6 m below datum and reports a salinity of 6.25 ppt with the alluvial interval reading 10.85 degC at dawn and the wellhead brackish 13.99 m3/d while the electrode pair returns 3618 uS/cm.
Node SN-006-0EFC sits in a till lens at 309.5 m below datum and reports a salinity of 3.49 ppt with the lacustrine interval reading 7.38 degC at dawn and the wellhead spooned 3.21 m3/d while the electrode pair returns 1253 uS/cm.
Node SN-007-0B02 sits in a recharge lens at 262.0 m below datum and steps down to a salinity of 2.38 ppt with the discharge interval reading 8.18 degC at dawn and the wellhead perched 4.05 m3/d while the electrode pair returns 3900 uS/cm.
Node SN-008-0448 sits in a karst lens at 406.5 m below datum and reports a salinity of 7.32 ppt with the upconing interval reading 6.17 degC at dawn and the wellhead fluvial 0.75 m3/d while the electrode pair returns 1448 uS/cm.
Node SN-009-043C sits in a alluvial lens at 159.3 m below datum and flags a salinity of 6.20 ppt with the percolate interval reading 7.60 degC at dawn and the wellhead brackish 13.01 m3/d while the electrode pair returns 2025 uS/cm.
Node SN-010-0FF5 sits in a karst lens at 264.2 m below datum and flags a salinity of 7.05 ppt with the freshened interval reading 7.88 degC at dawn and the wellhead siphon 8.41 m3/d while the electrode pair returns 1635 uS/cm.
Node SN-011-098A sits in a drain lens at 365.7 m below datum and reports a salinity of 8.11 ppt with the wedge interval reading 5.51 degC at dawn and the wellhead glacial 2.52 m3/d while the electrode pair returns 2916 uS/cm.
Node SN-012-0AE8 sits in a till lens at 259.6 m below datum and steps down to a salinity of 9.13 ppt with the recharge interval reading 11.60 degC at dawn and the wellhead spooned 0.46 m3/d while the electrode pair returns 3077 uS/cm.
Node SN-013-0B83 sits in a salinized lens at 171.2 m below datum and peaks at a salinity of 2.83 ppt with the lacustrine interval reading 5.56 degC at dawn and the wellhead discharge 0.35 m3/d while the electrode pair returns 3083 uS/cm.
Node SN-014-032E sits in a drain lens at 249.6 m below datum and flags a salinity of 1.09 ppt with the salinized interval reading 9.58 degC at dawn and the wellhead perched 12.00 m3/d while the electrode pair returns 2221 uS/cm.
Node SN-015-004F sits in a aeolian lens at 264.4 m below datum and reports a salinity of 7.34 ppt with the salinized interval reading 8.76 degC at dawn and the wellhead perched 0.31 m3/d while the electrode pair returns 1483 uS/cm.
Node SN-016-07C6 sits in a aeolian lens at 155.9 m below datum and reports a salinity of 0.64 ppt with the mineralized interval reading 11.75 degC at dawn and the wellhead diluted 13.46 m3/d while the electrode pair returns 524 uS/cm.
Node SN-017-06C5 sits in a salinized lens at 72.1 m below datum and averages a salinity of 4.52 ppt with the siphon interval reading 8.93 degC at dawn and the wellhead confining 3.98 m3/d while the electrode pair returns 3687 uS/cm.
Node SN-018-0E64 sits in a siphon lens at 334.9 m below datum and steps down to a salinity of 6.73 ppt with the salinized interval reading 7.63 degC at dawn and the wellhead freshened 9.64 m3/d while the electrode pair returns 1681 uS/cm.
Node SN-019-0122 sits in a spooned lens at 117.2 m below datum and logs a salinity of 3.24 ppt with the fractured interval reading 9.26 degC at dawn and the wellhead karst 3.50 m3/d while the electrode pair returns 1065 uS/cm.
Node SN-020-021D sits in a karst lens at 414.3 m below datum and recovers to a salinity of 0.76 ppt with the glacial interval reading 6.25 degC at dawn and the wellhead silty 9.60 m3/d while the electrode pair returns 1041 uS/cm.
Node SN-021-06C6 sits in a percolate lens at 379.2 m below datum and drifts to a salinity of 7.61 ppt with the wedge interval reading 5.84 degC at dawn and the wellhead karst 1.65 m3/d while the electrode pair returns 1913 uS/cm.
Node SN-022-0DC9 sits in a spooned lens at 327.1 m below datum and reports a salinity of 9.25 ppt with the freshened interval reading 5.05 degC at dawn and the wellhead silty 5.90 m3/d while the electrode pair returns 1544 uS/cm.
Node SN-023-0717 sits in a karst lens at 222.0 m below datum and steps down to a salinity of 4.02 ppt with the alluvial interval reading 6.60 degC at dawn and the wellhead siphon 3.77 m3/d while the electrode pair returns 3892 uS/cm.
Node SN-024-02A9 sits in a fractured lens at 479.7 m below datum and drifts to a salinity of 7.88 ppt with the silty interval reading 12.53 degC at dawn and the wellhead discharge 13.18 m3/d while the electrode pair returns 3592 uS/cm.
Node SN-025-0C8F sits in a wedge lens at 434.3 m below datum and holds a salinity of 1.63 ppt with the alluvial interval reading 4.22 degC at dawn and the wellhead till 5.73 m3/d while the electrode pair returns 3905 uS/cm.
Node SN-026-04BF sits in a colluvial lens at 460.0 m below datum and averages a salinity of 9.36 ppt with the diapir interval reading 8.98 degC at dawn and the wellhead brackish 10.29 m3/d while the electrode pair returns 802 uS/cm.
Node SN-027-0DA4 sits in a silty lens at 265.7 m below datum and steps down to a salinity of 7.06 ppt with the percolate interval reading 4.69 degC at dawn and the wellhead freshened 8.42 m3/d while the electrode pair returns 2201 uS/cm.
Node SN-028-01EB sits in a perched lens at 49.5 m below datum and recovers to a salinity of 1.83 ppt with the silty interval reading 9.32 degC at dawn and the wellhead drain 9.69 m3/d while the electrode pair returns 1126 uS/cm.
Node SN-029-042C sits in a karst lens at 30.6 m below datum and averages a salinity of 0.86 ppt with the percolate interval reading 9.85 degC at dawn and the wellhead recharge 8.16 m3/d while the electrode pair returns 1452 uS/cm.
## Relays
Relay RL-00-02181 reburses the upconing trunk from the brackish margin and peaks at a round-trip time of 131.8 ms with a frame loss of 1.04 percent across the night window.
Relay RL-01-00262 reburses the spooned trunk from the fluvial margin and holds a round-trip time of 372.2 ms with a frame loss of 3.65 percent across the night window.
Relay RL-02-03691 reburses the recharge trunk from the percolate margin and steps down to a round-trip time of 398.5 ms with a frame loss of 0.29 percent across the night window.
Relay RL-03-0E117 reburses the lacustrine trunk from the confining margin and averages a round-trip time of 373.9 ms with a frame loss of 3.44 percent across the night window.
Relay RL-04-0B41B reburses the aeolian trunk from the colluvial margin and peaks at a round-trip time of 70.7 ms with a frame loss of 3.25 percent across the night window.
Relay RL-05-0D12D reburses the recharge trunk from the diluted margin and steps down to a round-trip time of 264.7 ms with a frame loss of 0.03 percent across the night window.
Relay RL-06-043B4 reburses the colluvial trunk from the upconing margin and reports a round-trip time of 49.5 ms with a frame loss of 3.42 percent across the night window.
Relay RL-07-09AD6 reburses the fractured trunk from the freshened margin and averages a round-trip time of 225.3 ms with a frame loss of 1.06 percent across the night window.
Relay RL-08-04394 reburses the brackish trunk from the fluvial margin and steps down to a round-trip time of 88.6 ms with a frame loss of 2.47 percent across the night window.
Relay RL-09-017A0 reburses the wedge trunk from the lacustrine margin and drifts to a round-trip time of 363.0 ms with a frame loss of 3.30 percent across the night window.
Relay RL-10-0217D reburses the mineralized trunk from the lacustrine margin and peaks at a round-trip time of 26.2 ms with a frame loss of 1.30 percent across the night window.
Relay RL-11-08F97 reburses the perched trunk from the freshened margin and logs a round-trip time of 181.8 ms with a frame loss of 2.75 percent across the night window.
Relay RL-12-02628 reburses the fractured trunk from the glacial margin and steps down to a round-trip time of 378.5 ms with a frame loss of 2.69 percent across the night window.
Relay RL-13-06E06 reburses the silty trunk from the mineralized margin and flags a round-trip time of 153.4 ms with a frame loss of 2.15 percent across the night window.
## Aggregation services
Aggregator AG-00-3BB logs a queue depth of 5.2 messages and a histogram bucket of 6.82 seconds before the dedupe pass settles the stream for the next rotation.
Aggregator AG-01-0FF reports a queue depth of 86.4 messages and a histogram bucket of 4.10 seconds before the dedupe pass settles the stream for the next rotation.
Aggregator AG-02-37F drifts to a queue depth of 34.7 messages and a histogram bucket of 10.25 seconds before the dedupe pass settles the stream for the next rotation.
Aggregator AG-03-3B3 averages a queue depth of 93.5 messages and a histogram bucket of 13.59 seconds before the dedupe pass settles the stream for the next rotation.
Aggregator AG-04-33E flags a queue depth of 83.2 messages and a histogram bucket of 17.58 seconds before the dedupe pass settles the stream for the next rotation.
Aggregator AG-05-2F2 settles near a queue depth of 84.8 messages and a histogram bucket of 0.84 seconds before the dedupe pass settles the stream for the next rotation.
Aggregator AG-06-2AD averages a queue depth of 75.4 messages and a histogram bucket of 7.65 seconds before the dedupe pass settles the stream for the next rotation.
Aggregator AG-07-06E drifts to a queue depth of 26.5 messages and a histogram bucket of 14.26 seconds before the dedupe pass settles the stream for the next rotation.
Aggregator AG-08-0E3 averages a queue depth of 83.9 messages and a histogram bucket of 15.51 seconds before the dedupe pass settles the stream for the next rotation.
Aggregator AG-09-138 averages a queue depth of 78.6 messages and a histogram bucket of 8.50 seconds before the dedupe pass settles the stream for the next rotation.
## Calibration procedures
Procedure CP-00-047 steps the probe through a two-point rinse, then applies a trim of 0.2084 at the 66 milli-bar mark and records a residual of 0.02052 before the next field seal is crimped.
Procedure CP-01-0A4 steps the probe through a two-point rinse, then applies a trim of 0.7793 at the 96 milli-bar mark and records a residual of 0.01468 before the next field seal is crimped.
Procedure CP-02-054 steps the probe through a two-point rinse, then applies a trim of 0.4634 at the 68 milli-bar mark and records a residual of 0.04234 before the next field seal is crimped.
Procedure CP-03-02D steps the probe through a two-point rinse, then applies a trim of 0.8459 at the 12 milli-bar mark and records a residual of 0.04855 before the next field seal is crimped.
Procedure CP-04-098 steps the probe through a two-point rinse, then applies a trim of 0.5267 at the 97 milli-bar mark and records a residual of 0.00287 before the next field seal is crimped.
Procedure CP-05-09B steps the probe through a two-point rinse, then applies a trim of 0.3968 at the 72 milli-bar mark and records a residual of 0.01637 before the next field seal is crimped.
Procedure CP-06-030 steps the probe through a two-point rinse, then applies a trim of 0.8860 at the 12 milli-bar mark and records a residual of 0.04508 before the next field seal is crimped.
Procedure CP-07-0EC steps the probe through a two-point rinse, then applies a trim of 0.2367 at the 70 milli-bar mark and records a residual of 0.00108 before the next field seal is crimped.
Procedure CP-08-0BC steps the probe through a two-point rinse, then applies a trim of 0.7278 at the 68 milli-bar mark and records a residual of 0.04707 before the next field seal is crimped.
Procedure CP-09-0AA steps the probe through a two-point rinse, then applies a trim of 0.6069 at the 37 milli-bar mark and records a residual of 0.00443 before the next field seal is crimped.
## Failure modes
Mode FM-00-27 declares a fluvial excursion when two independent probes disagree beyond the band for three rotations; the fleet reports a mean time between failures of 36902 hours for this class of excursion.
Mode FM-01-13 declares a drain excursion when two independent probes disagree beyond the band for three rotations; the fleet peaks at a mean time between failures of 29042 hours for this class of excursion.
Mode FM-02-14 declares a till excursion when two independent probes disagree beyond the band for three rotations; the fleet peaks at a mean time between failures of 26976 hours for this class of excursion.
Mode FM-03-0C declares a diapir excursion when two independent probes disagree beyond the band for three rotations; the fleet drifts to a mean time between failures of 22578 hours for this class of excursion.
Mode FM-04-39 declares a perched excursion when two independent probes disagree beyond the band for three rotations; the fleet recovers to a mean time between failures of 27383 hours for this class of excursion.
Mode FM-05-19 declares a siphon excursion when two independent probes disagree beyond the band for three rotations; the fleet logs a mean time between failures of 23152 hours for this class of excursion.
## End of document
Summarize the network's dominant risk in one paragraph, then list the three node identifiers whose readings most limit the model this month.
die "no source for '$voice': pass --source or set ${source_var} in $ENV_FILE"
fi
[ -n "$offset"]|| die "no crop offset for '$voice': pass --offset or set ${offset_var}"
[ -n "$duration"]|| die "no crop duration for '$voice': pass --duration or set ${duration_var}"
is_seconds "$offset"|| die "crop offset '$offset' is not a number of seconds"
is_seconds "$duration"|| die "crop duration '$duration' is not a number of seconds"
[ -f "$source"]&&[ -r "$source"]|| die "source not readable: $source"
# === THE GATE. Source seconds 0..10 are forbidden voice material, full stop.
if ! fcmp "$offset"'>='"$SOURCE_FORBIDDEN_SECONDS";then
die "source-material invariant violated: the crop must start past the first ${SOURCE_FORBIDDEN_SECONDS} s of the source file, and '$voice' asks for ${offset} s into $source. Pick an offset at or past ${SOURCE_FORBIDDEN_SECONDS} s — nothing was written."
is_seconds "$source_duration"|| die "ffprobe reports no numeric duration for $source: '$source_duration'"
source_bytes="$(stat -c%s "$source")"
if ! fcmp "$(awk -v a="$offset" -v d="$duration"'BEGIN { print a + d }')"'<='"$source_duration";then
die "crop ends at $(awk -v a="$offset" -v d="$duration"'BEGIN { print a + d }') s but $source is only $source_duration s; the crop would come out short"
fi
if ! fcmp "$duration"'>='"$MIN_SECONDS"|| ! fcmp "$duration"'<='"$MAX_SECONDS";then
die "crop duration $duration s is outside the ${MIN_SECONDS}-${MAX_SECONDS} s reference window"
fi
local crop_end planned_rvq
crop_end="$(awk -v a="$offset" -v d="$duration"'BEGIN { print a + d }')"
planned_rvq="$(rvq_bytes_for "$duration")"
if ! fcmp "$planned_rvq"'>='"$RVQ_MIN_BYTES"|| ! fcmp "$planned_rvq"'<='"$RVQ_MAX_BYTES";then
die "a $duration s crop packs to ${planned_rvq} B of .rvq, outside the recorded engine window ${RVQ_MIN_BYTES}-${RVQ_MAX_BYTES} B; lengthen the crop (that window starts at $(awk -v m="$RVQ_MIN_BYTES" -v b="$BYTES_PER_SECOND"'BEGIN { print m / b }') s) — the source, not the codec, sets this floor"
echo"EXTRACT $(basename "$TOOL") at $ENGINE_COMMIT (device ${DEVICE:-auto}, $THREADS threads)"
if ! nice -n "$NICE_LEVEL" env OMP_NUM_THREADS="$THREADS"\
"${extractor[@]}" 2>&1| tee "$stage_log" >&2;then
die "$TOOL failed on $stage_wav (its output is in $stage_log); the pack was not published"
fi
[ -f "$spk_file"]|| die "$TOOL wrote no $spk_file (see $stage_log)"
[ -f "$rvq_file"]|| die "$TOOL wrote no $rvq_file (see $stage_log)"
spk_bytes="$(stat -c%s "$spk_file")"
rvq_bytes="$(stat -c%s "$rvq_file")"
local expect_rvq
expect_rvq="$(rvq_bytes_for "$wav_duration")"
["$spk_bytes"="$SPK_BYTES"]||
die "reference.spk is $spk_bytes B, expected ${SPK_BYTES} B (enc_dim x f32). The engine changed its speaker encoder — see [[Voice Packs]] before shipping this pack"
["$rvq_bytes"="$expect_rvq"]||
die "reference.rvq is $rvq_bytes B, expected ${expect_rvq} B for a ${wav_duration} s crop (22 B per 12.5 Hz frame)"
if ! fcmp "$rvq_bytes"'>='"$RVQ_MIN_BYTES"|| ! fcmp "$rvq_bytes"'<='"$RVQ_MAX_BYTES";then
die "reference.rvq is $rvq_bytes B, outside the recorded engine window ${RVQ_MIN_BYTES}-${RVQ_MAX_BYTES} B — an engine bump? see [[Voice Packs]]"
'(.crop.offset_seconds | type == "number") and .invariant.forbidden_source_seconds == $min and (.crop.offset_seconds >= $min) and (.invariant.name == "source-material invariant")'\
"$stage_manifest" >/dev/null ||
die "manifest assertion failed: .crop.offset_seconds ($recorded_offset) does not clear the ${SOURCE_FORBIDDEN_SECONDS} s source-material invariant"
["$recorded_sha"="$source_sha"]||
die "manifest assertion failed: .source.sha256 ($recorded_sha) is not the hash of $source ($source_sha)"
jq -e '(.voice | type == "string") and (.reference.sample_rate | type == "number") and (.reference.duration_seconds | type == "number") and (.artifacts.engine_commit | length == 40)'\
"$stage_manifest" >/dev/null ||
die "manifest assertion failed: voice / reference / engine provenance fields are not all present"
echo"MANIFEST crop offset $recorded_offset s clears the ${SOURCE_FORBIDDEN_SECONDS} s invariant"
if["$DRY_RUN" -eq 1];then
echo"MANIFEST_BEGIN"
cat "$stage_manifest"
echo"MANIFEST_END"
echo"PLAN would write $pack_dir/{reference.wav,reference.spk,reference.rvq,manifest.json}"
echo"DRYRUN $voice: inputs valid, crop cut and probed, extractor not run"
else
# reference.wav is the gate the shim reads, so it lands last: the voice
# name never exists on disk in a state the artifacts are not behind it.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Frank
dank
Comrade Chief Designer
Comrade Chief Designer
