docs: frank-small is Gemma4-12B QAT now — new runbook, Qwen retired as rollback target
AGENTS.md topology, Active Services, the CUDA-quant-KV lesson (f16 is no longer why), and the runbook index. New frank-small-gemma4-12b runbook: payload digests, the SWA/q8 arithmetic behind --fit's 262k window, the MTP-sidecar + ngram-mod chain, the always-accepted reasoning_effort, and the fit-probe error that looks like a fault at boot. The Qwen runbook and both indexes carry the retirement; work log updated.
This commit is contained in:
1 parent
0a25351caa
commit
87caca4894
6 files changed
+226
-29
No files matched your search
@@ -1,7 +1,7 @@
|
||||
---
|
||||
type: reference
|
||||
title: frank-small on Dankclaw — llama.cpp, Qwen3.8-27B GSQ-RCO IQ3_S
|
||||
description: The active frank-small runbook — mainline llama.cpp on Dankclaw's RTX 3090 serving ISTA-DASLab Qwen3.8-27B-GSQ-RCO at IQ3_S with the model's own MTP head and the BF16 vision projector: identity, KV arithmetic, deployed flags, the MTP contract, the client contract, and what to check when it misbehaves.
|
||||
description: The RETIRED frank-small runbook (2026-09-16 → 09-17, rollback target) — mainline llama.cpp on Dankclaw's RTX 3090 serving ISTA-DASLab Qwen3.8-27B-GSQ-RCO at IQ3_S with the model's own in-file MTP head and the BF16 vision projector: identity, KV arithmetic, deployed flags, the MTP contract, the client contract, and what to check when it misbehaves.
|
||||
tags:
|
||||
- llm
|
||||
- frank-small
|
||||
@@ -16,10 +16,11 @@ timestamp: 2026-09-16
|
||||
|
||||
# frank-small — llama.cpp on the RTX 3090
|
||||
|
||||
> **ACTIVE since 2026-09-16.** Created when Dankclaw's inference was re-split:
|
||||
> [[halogen-flash-server]] took the gfx1151 iGPU back for `frank-large`, which
|
||||
> freed the RTX 3090 for this endpoint. Two endpoints, one host: `frank-large`
|
||||
> on `:7770`, `frank-small` on `:7771`.
|
||||
> **RETIRED 2026-09-17** — replaced by [[frank-small-gemma4-12b]]. Kept as
|
||||
> the rollback target and the record of the deployment it succeeded.
|
||||
> Originally ACTIVE from 2026-09-16, created when Dankclaw's inference was
|
||||
> re-split: [[halogen-flash-server]] took the gfx1151 iGPU back for
|
||||
> `frank-large`, which freed the RTX 3090 for this endpoint.
|
||||
|
||||
## Identity
|
||||
|
||||
|
||||
Reference in new issue
Block a user