docs: frank-small is Gemma4-12B QAT now — new runbook, Qwen retired as rollback target

AGENTS.md topology, Active Services, the CUDA-quant-KV lesson (f16 is no
longer why), and the runbook index. New frank-small-gemma4-12b runbook:
payload digests, the SWA/q8 arithmetic behind --fit's 262k window, the
MTP-sidecar + ngram-mod chain, the always-accepted reasoning_effort, and
the fit-probe error that looks like a fault at boot. The Qwen runbook and
both indexes carry the retirement; work log updated.
This commit is contained in:
Frank
2026-09-17 02:18:44 -07:00
parent 0a25351caa
commit 87caca4894
6 changed files with 226 additions and 29 deletions
+27
View File
@@ -793,3 +793,30 @@ timestamp: 2026-09-14
contiguous 2 MiB blocks left (240 before the refresh).
- Runbook: [[halogen-flash-server]] gains a "Refreshing the weights" note; the
quant comparison is appended to [[x4-pair-experiment]].
## 2026-09-17 — frank-small swapped to Gemma4-12B QAT (Q4_K_M)
- **Cutover:** `frank-small` (Dankclaw RTX 3090, :7771) replaced
ISTA-DASLab Qwen3.8-27B-GSQ-RCO IQ3_S with
`HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced` Q4_K_M —
7.38 GiB main + 175 MiB BF16 mmproj (with audio encoder) + 254 MiB
`mtp-gemma-4-12B-it.gguf` assistant sidecar, all sha256-verified.
- **Spec decode:** the MTP head is now a separate `gemma4-assistant`
file loaded via `--model-draft` (shares the target's last layers' KV,
`llama-model.cpp:2662`), chained behind draftless `ngram-mod`:
`--spec-type draft-mtp,ngram-mod`. Measured acceptance 0.38–0.46,
mean draft len ~2.2.
- **KV:** q8_0 K and V; `build-cuda` rebuilt with
`-DGGML_CUDA_FA_ALL_QUANTS=ON` (required, else ~12x prefill collapse).
`CTX_SIZE` unset → `--fit on` chose the full 262,144-token window;
idle card usage 11,996 / 24,576 MiB (was 22,362).
- **Template behavior:** Gemma4 accepts every `reasoning_effort` level
(all seven probed: 200) — the Qwen3.8 gate is gone; harness map kept
as a safety net.
- **Verified live:** health 200, chat + `reasoning_content`, `tool_calls`,
vision (live JPEG), deploy e2e gate passed (31 passed, 1 flaky crawl).
- **Harness:** `models.yml` frank-small `contextWindow: 262144`.
- **Note:** the boot log's `Gemma4Assistant requires ctx_other to be
set` error is `--fit`'s own probe — normal, self-recovers seconds later.
- Runbook: [[frank-small-gemma4-12b]] (active);
[[frank-small-qwen38-gsq-rco]] retired 2026-09-17 as rollback target.