docs+scripts: reconcile payload inventory after both swaps
Staged 59 GiB of retired LLM payloads on Dankclaw to ~dank/.AGENTTRASH/20261002-retired-llm-payloads/ (vLLM MiMo BF16, Qwen3.5-9B MTP source, both DFlash dirs, Gemma4-12B-QAT, SGLang MiMo INT8) and corrected every doc claim about what is on disk: gufo's rollback pair (Q4_K_XL weights + the Q8_K_XL DFlash2 drafter) and frank-small's Q6_K stay in place, so the rollback paths in the tags' env files still resolve. The vLLM-era frank-large fetch trio moves to old/ with its runtime; runbook links updated.
This commit is contained in:
@@ -177,8 +177,8 @@ frank-small, port 7771]
|
||||
(bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
|
||||
- [[deployment]] — deploy and update process for both servers
|
||||
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
||||
- [[frank-large-gufo-qwen38-27b]] — **the ACTIVE frank-large runbook
|
||||
again since 2026-10-01 night** (root systemd unit, port 7770): gufo
|
||||
- [[frank-large-gufo-qwen38-27b]] — **RETIRED 2026-10-02 (the rollback
|
||||
target)** (root systemd unit, port 7770): gufo
|
||||
(gufo-org/gufo) v0.1.1, the Strix Halo inference engine, built from
|
||||
source on Dankclaw, serving unsloth/Qwen3.8-27B-GGUF **UD-Q4_K_XL**
|
||||
with the z-lab DFlash2 Q4_K_M drafter — ONE session at the full native
|
||||
@@ -228,23 +228,24 @@ frank-small, port 7771]
|
||||
.hgn + quality overlay) on the gfx1151 iGPU — two 200k conversations over one
|
||||
400,384-position pool, deployed flags, memory model, client contract,
|
||||
troubleshooting (2026-09-16 → 09-19)
|
||||
- [[frank-small-llama-mimo-q6k]] — **the ACTIVE frank-small runbook
|
||||
since 2026-10-01 night** (root systemd unit, port 7771): llama.cpp
|
||||
(master, CUDA sm_86 build) serving bartowski/MiMo-V2.6-Distill-
|
||||
Qwen-9B-GGUF **Q6_K** on the RTX 3090 — TWO native 262,144-token
|
||||
sessions with ASYMMETRIC KV (K f16 unquantized + V q8_0, which forces
|
||||
flash-attn), ~20.9/24.6 GiB measured, text-only (the GGUF has no
|
||||
vision), reasoning_content, no spec decode by default
|
||||
- [[frank-small-llama-mimo-q6k]] — **RETIRED 2026-10-01 → 10-02 (rollback
|
||||
target)**: llama.cpp (master, CUDA sm_86 build) serving
|
||||
bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** on the RTX 3090 — TWO
|
||||
native 262,144-token sessions with ASYMMETRIC KV (K f16 unquantized + V
|
||||
q8_0, which forces flash-attn), ~20.9/24.6 GiB measured, text-only (the
|
||||
GGUF has no vision), reasoning_content, no spec decode; payload still at
|
||||
`/home/dank/models/MiMo-Q6K/`
|
||||
- [[frank-small-sglang-mimo-v26]] — RETIRED 2026-10-01 (hours; replaced
|
||||
by [[frank-small-llama-mimo-q6k]] for the asymmetric KV SGLang cannot
|
||||
express; venv + INT8 payload on disk as rollback): SGLang in a
|
||||
express; venv still on disk, INT8 payload staged to
|
||||
`~dank/.AGENTTRASH/20261002-retired-llm-payloads/`): SGLang in a
|
||||
self-contained python3.12 venv serving leteschlead MiMo-V2.6 INT8
|
||||
W8A16 AutoRound, 2 x 262144 ctx, fp8 KV, in-checkpoint vision,
|
||||
83.9 tok/s decode
|
||||
- [[frank-small-gemma4-12b]] — RETIRED 2026-09-17 → 2026-10-01 (replaced by
|
||||
[[frank-small-sglang-mimo-v26]] → [[frank-small-llama-mimo-q6k]];
|
||||
rollback target, payload and binary intact on disk): mainline llama.cpp serving
|
||||
intact on disk): mainline llama.cpp serving
|
||||
[[frank-small-sglang-mimo-v26]] → the llama.cpp MiMo shape; payload staged
|
||||
to `~dank/.AGENTTRASH/20261002-retired-llm-payloads/Gemma4-12B-QAT/`):
|
||||
mainline llama.cpp serving
|
||||
HauhauCS/Gemma4-12B-QAT-Uncensored-HauhauCS-Balanced Q4_K_M on the RTX
|
||||
3090, with the separate MTP assistant sidecar chained behind ngram-mod,
|
||||
q8_0 KV, and `--fit` sizing the full 262k window into ~12 GiB.
|
||||
|
||||
Reference in New Issue
Block a user