frank-small: revert vLLM cutover — back to llama.cpp Gemma4-12B Q4_K_M
The vLLM detour (a2b39b1, same day) is retired: the Q4_K_M llama.cpp setup
is the requested shape and the vLLM KV-quant options on sm_86 all measured
dead (fp8 hardware-impossible, int8 page-size unification, turboquant
mm-prefix). Root unit restored, vLLM quadlet/Dockerfile/fetch script moved
to .AGENTTRASH, runbook listed as same-day-reverted. Rollback target:
frank-small-vllm-gemma4.
This commit is contained in:
1 parent
bab8f7f839
commit
374041cd7a
15 files changed
+495
-637
No files matched your search
@@ -166,17 +166,15 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
|
||||
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
|
||||
- [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02)
|
||||
- [[frank-small-vllm-gemma4]] — the ACTIVE frank-small runbook since
|
||||
2026-09-19: vLLM serving
|
||||
google/gemma-4-12B-it-qat-w4a16-ct (compressed-tensors w4a16, Marlin)
|
||||
with Google's bf16 MTP assistant (bf16 KV, 289,647-token pool, 2
|
||||
full-context sessions, native text+image+audio) on
|
||||
Dankclaw's RTX 3090, port 7771
|
||||
- [[frank-small-gemma4-12b]] — RETIRED 2026-09-19 (rollback target):
|
||||
mainline llama.cpp serving
|
||||
- [[frank-small-gemma4-12b]] — the ACTIVE frank-small runbook since
|
||||
2026-09-17: mainline llama.cpp serving
|
||||
HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced at Q4_K_M
|
||||
(MTP assistant sidecar + ngram-mod, q8_0 KV, 4 slots of 262k) on
|
||||
Dankclaw's RTX 3090
|
||||
(MTP assistant sidecar + ngram-mod, q8_0 KV, --fit → 262k) on
|
||||
Dankclaw's RTX 3090, port 7771
|
||||
- [[frank-small-vllm-gemma4]] — RETIRED 2026-09-19 (same-day revert,
|
||||
rollback target): vLLM serving google/gemma-4-12B-it-qat-w4a16-ct (w4a16
|
||||
Marlin) with the bf16 MTP assistant on the RTX 3090 — bf16 KV, 289,647-
|
||||
token pool, 2 concurrent full-context sessions, native text+image+audio
|
||||
- [[frank-small-qwen38-gsq-rco]] — RETIRED 2026-09-17 (rollback target):
|
||||
the prior frank-small runbook (2026-09-16 → 09-17), ISTA-DASLab
|
||||
Qwen3.8-27B-GSQ-RCO IQ3_S with own in-file MTP head + BF16 projector
|
||||
|
||||
Reference in new issue
Block a user