docs: frank-small is Gemma4-12B QAT now — new runbook, Qwen retired as rollback target
AGENTS.md topology, Active Services, the CUDA-quant-KV lesson (f16 is no longer why), and the runbook index. New frank-small-gemma4-12b runbook: payload digests, the SWA/q8 arithmetic behind --fit's 262k window, the MTP-sidecar + ngram-mod chain, the always-accepted reasoning_effort, and the fit-probe error that looks like a fault at boot. The Qwen runbook and both indexes carry the retirement; work log updated.
This commit is contained in:
1 parent
0a25351caa
commit
87caca4894
6 files changed
+226
-29
No files matched your search
@@ -87,11 +87,15 @@ frank-small, port 7771]
|
||||
overlay) on the gfx1151 iGPU — two 200k conversations over one
|
||||
400,384-position pool, deployed flags, memory model, client contract,
|
||||
troubleshooting
|
||||
- [[frank-small-qwen38-gsq-rco]] — ACTIVE frank-small runbook since 2026-09-16
|
||||
- [[frank-small-gemma4-12b]] — ACTIVE frank-small runbook since 2026-09-17
|
||||
(root systemd unit, port 7771): mainline llama.cpp serving
|
||||
ISTA-DASLab/Qwen3.8-27B-GSQ-RCO IQ3_S on the RTX 3090, with the model's own
|
||||
Multi-Token Prediction head for speculative decoding and the BF16 projector
|
||||
for the fleet's image path
|
||||
HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced Q4_K_M on the RTX
|
||||
3090, with the separate MTP assistant sidecar chained behind ngram-mod,
|
||||
q8_0 KV, and `--fit` sizing the full 262k window into ~12 GiB
|
||||
- [[frank-small-qwen38-gsq-rco]] — RETIRED 2026-09-17 (rollback target): the
|
||||
prior frank-small runbook (2026-09-16 → 09-17), ISTA-DASLab
|
||||
Qwen3.8-27B-GSQ-RCO IQ3_S with the model's own in-file MTP head and
|
||||
f16 KV at 131k
|
||||
- [[x4-pair-experiment]] — MEASURED 2026-09-16: what mainline llama.cpp can and
|
||||
cannot do with the 3090 + Strix Halo pair, over their only link (the 3090's
|
||||
Gen4 x4). The tensor inventory that decides any split, the single flag (`-ot`)
|
||||
|
||||
Reference in new issue
Block a user