docs: frank-small is Gemma4-12B QAT now — new runbook, Qwen retired as rollback target

AGENTS.md topology, Active Services, the CUDA-quant-KV lesson (f16 is no
longer why), and the runbook index. New frank-small-gemma4-12b runbook:
payload digests, the SWA/q8 arithmetic behind --fit's 262k window, the
MTP-sidecar + ngram-mod chain, the always-accepted reasoning_effort, and
the fit-probe error that looks like a fault at boot. The Qwen runbook and
both indexes carry the retirement; work log updated.
This commit is contained in:
Frank committed 2026-09-17 02:18:44 -07:00
1 parent 0a25351caa
commit 87caca4894
6 files changed
+226 -29

No files matched your search

@@ -1,7 +1,7 @@
---
type: reference
title: frank-small on Dankclaw — llama.cpp, Qwen3.8-27B GSQ-RCO IQ3_S
description: The active frank-small runbook — mainline llama.cpp on Dankclaw's RTX 3090 serving ISTA-DASLab Qwen3.8-27B-GSQ-RCO at IQ3_S with the model's own MTP head and the BF16 vision projector: identity, KV arithmetic, deployed flags, the MTP contract, the client contract, and what to check when it misbehaves.
description: The RETIRED frank-small runbook (2026-09-16 → 09-17, rollback target) — mainline llama.cpp on Dankclaw's RTX 3090 serving ISTA-DASLab Qwen3.8-27B-GSQ-RCO at IQ3_S with the model's own in-file MTP head and the BF16 vision projector: identity, KV arithmetic, deployed flags, the MTP contract, the client contract, and what to check when it misbehaves.
tags:
- llm
- frank-small
@@ -16,10 +16,11 @@ timestamp: 2026-09-16
# frank-small — llama.cpp on the RTX 3090
> **ACTIVE since 2026-09-16.** Created when Dankclaw's inference was re-split:
> [[halogen-flash-server]] took the gfx1151 iGPU back for `frank-large`, which
> freed the RTX 3090 for this endpoint. Two endpoints, one host: `frank-large`
> on `:7770`, `frank-small` on `:7771`.
> **RETIRED 2026-09-17** — replaced by [[frank-small-gemma4-12b]]. Kept as
> the rollback target and the record of the deployment it succeeded.
> Originally ACTIVE from 2026-09-16, created when Dankclaw's inference was
> re-split: [[halogen-flash-server]] took the gfx1151 iGPU back for
> `frank-large`, which freed the RTX 3090 for this endpoint.
## Identity