docs+scripts: reconcile payload inventory after both swaps

Staged 59 GiB of retired LLM payloads on Dankclaw to
~dank/.AGENTTRASH/20261002-retired-llm-payloads/ (vLLM MiMo BF16, Qwen3.5-9B
MTP source, both DFlash dirs, Gemma4-12B-QAT, SGLang MiMo INT8) and corrected
every doc claim about what is on disk: gufo's rollback pair (Q4_K_XL weights +
the Q8_K_XL DFlash2 drafter) and frank-small's Q6_K stay in place, so the
rollback paths in the tags' env files still resolve. The vLLM-era frank-large
fetch trio moves to old/ with its runtime; runbook links updated.
This commit is contained in:
Frank
2026-10-02 02:31:39 -07:00
parent 72bd4c967e
commit 8f129180d8
9 changed files with 44 additions and 26 deletions
+14 -13
View File
@@ -177,8 +177,8 @@ frank-small, port 7771]
(bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
- [[deployment]] — deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[frank-large-gufo-qwen38-27b]] — **the ACTIVE frank-large runbook
again since 2026-10-01 night** (root systemd unit, port 7770): gufo
- [[frank-large-gufo-qwen38-27b]] — **RETIRED 2026-10-02 (the rollback
target)** (root systemd unit, port 7770): gufo
(gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from
source on Dankclaw, serving unsloth/Qwen3.8-27B-GGUF **UD-Q4_K_XL**
with the z-lab DFlash2 Q4_K_M drafter — ONE session at the full native
@@ -228,23 +228,24 @@ frank-small, port 7771]
.hgn + quality overlay) on the gfx1151 iGPU — two 200k conversations over one
400,384-position pool, deployed flags, memory model, client contract,
troubleshooting (2026-09-16 → 09-19)
- [[frank-small-llama-mimo-q6k]] — **the ACTIVE frank-small runbook
since 2026-10-01 night** (root systemd unit, port 7771): llama.cpp
(master, CUDA sm_86 build) serving bartowski/MiMo-V2.6-Distill-
Qwen-9B-GGUF **Q6_K** on the RTX 3090 — TWO native 262,144-token
sessions with ASYMMETRIC KV (K f16 unquantized + V q8_0, which forces
flash-attn), ~20.9/24.6 GiB measured, text-only (the GGUF has no
vision), reasoning_content, no spec decode by default
- [[frank-small-llama-mimo-q6k]] — **RETIRED 2026-10-01 → 10-02 (rollback
target)**: llama.cpp (master, CUDA sm_86 build) serving
bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** on the RTX 3090 — TWO
native 262,144-token sessions with ASYMMETRIC KV (K f16 unquantized + V
q8_0, which forces flash-attn), ~20.9/24.6 GiB measured, text-only (the
GGUF has no vision), reasoning_content, no spec decode; payload still at
`/home/dank/models/MiMo-Q6K/`
- [[frank-small-sglang-mimo-v26]] — RETIRED 2026-10-01 (hours; replaced
by [[frank-small-llama-mimo-q6k]] for the asymmetric KV SGLang cannot
express; venv + INT8 payload on disk as rollback): SGLang in a
express; venv still on disk, INT8 payload staged to
`~dank/.AGENTTRASH/20261002-retired-llm-payloads/`): SGLang in a
self-contained python3.12 venv serving leteschlead MiMo-V2.6 INT8
W8A16 AutoRound, 2 x 262144 ctx, fp8 KV, in-checkpoint vision,
83.9 tok/s decode
- [[frank-small-gemma4-12b]] — RETIRED 2026-09-17 → 2026-10-01 (replaced by
[[frank-small-sglang-mimo-v26]] → [[frank-small-llama-mimo-q6k]];
rollback target, payload and binary intact on disk): mainline llama.cpp serving
intact on disk): mainline llama.cpp serving
[[frank-small-sglang-mimo-v26]] → the llama.cpp MiMo shape; payload staged
to `~dank/.AGENTTRASH/20261002-retired-llm-payloads/Gemma4-12B-QAT/`):
mainline llama.cpp serving
HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced Q4_K_M on the RTX
3090, with the separate MTP assistant sidecar chained behind ngram-mod,
q8_0 KV, and `--fit` sizing the full 262k window into ~12 GiB.