docs(frank-medium): record the MXFP4 + clean-KV cut; correct vision claim

Knowledge-base follow-up to a45f396. The runbook, both index one-liners, and
the AGENTS.md topology row all still declared FP8 active; flipped to
just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq with unquantized bf16 KV (pool
486,464 tokens, full 262,144 window), and recorded the causal nuance: fp8 KV
was on both arms, so the FP8-vs-MXFP4 gap was weight precision, not KV - clean
KV is a quality hypothesis for W4A8, not a throughput one. No verdict yet.

The quadlet's "TEXT ONLY ... 400 here" comment was wrong: vision works on the
MXFP4 checkpoint (Qwen3_5ForConditionalGeneration carries the vision tower
through the Gemma4 merge path, unquantized) - verified by naming a solid red
48x48 PNG "Red". Comment corrected; harness models.yml input stays [text,image].
Docs-only; no re-deploy (the endpoint already serves vision).
This commit is contained in:
Frank committed 2026-09-21 17:08:33 -07:00
1 parent a45f396c39
commit d8dbd4f5bc
6 files changed
+76 -46

No files matched your search

+22
View File
@@ -11,6 +11,28 @@ timestamp: 2026-09-14
# Bundle Change Log
## 2026-09-21 — frank-medium cut back to MXFP4 with unquantized bf16 KV
- **The change.** frank-medium (dankserver :7770) served the FP8 checkpoint
(`orcarouter/Qwen3.8-27B-Uncensored-FP8`) with fp8 KV from 2026-09-20 to
09-21; it is now cut back to the MXFP4 checkpoint
(`just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq`, quark/W4A8) with
`--kv-cache-dtype auto` — unquantized bf16 KV instead of the endpoint's
historical fp8. `KV_DTYPE` is now a quadlet tunable (entrypoint default fp8);
`MAXLEN` stays at the native 262,144. Commit a45f396.
- **Why.** The operator's read that fp8 KV was hurting MXFP4. The nuance: fp8
KV sat on BOTH the FP8 and MXFP4 arms (hardcoded on the common exec line in
every commit since the MXFP4 era), so it could not explain the FP8-vs-MXFP4
*throughput* gap — that was weight precision. But MXFP4 is W4A8, and fp8 KV
on top compounds attention-path precision loss; clean KV tests whether that
was degrading *output quality*. No verdict yet — this is the setup.
- **Measured on deploy.** `kv_cache_dtype=auto`, no fp8 banner; pool auto-sized
to **486,464 tokens** (`Maximum concurrency for 262,144 tokens: 1`) — exactly
half MXFP4's 943,581-token fp8 pool (the 2x f16/fp8 ratio), and MXFP4's freed
weights (19.4 vs 30.9 GiB) keep the full 262,144 window fitting. Vision
re-verified on MXFP4 (a solid red 48x48 PNG named "Red"); the quadlet's old
"TEXT ONLY" comment was wrong and is corrected. e2e 32/32.
## 2026-09-21 — Calibre Content Server deployed (dankserver :8084)
- **The change.** The Calibre Content Server (`calibre-server`) is now a