frank-small: Swift-1.5 IQ3_S-mtp swap (parallel session) + fleet docs index
The parallel session's uncommitted swap, committed at Frank's direction (commit whatever's in the working tree): - llama.cpp master on the RTX 3090 now serves ukisai's Swift-1.5-Qwen3.8-27B GSQ-RCO IQ3_S WITH the in-file MTP head, ONE 131,072-token slot at Frank's 128k cap, q8_0 K + q8_0 V, vision through the base-model projector; MiMo-V2.6 Q6_K (2x200704 f16 KV) retired to rollback target. - env/serve/download + seam twins updated together (deployer payload gates rode the previous commit's deploy-remote.sh). - docs: new swift15 runbook; MiMo Q6K runbook retired; AGENTS.md topology and both vault indexes follow (root index ACTIVE/RETIRED flip, test 135 seam green). - comfyui: two qwen-image t2i workflow drops under dankclaw/comfyui/ workflows/ (12 KiB).
This commit is contained in:
@@ -181,11 +181,20 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
### Deployment and Topology
|
||||
|
||||
- [[deployment]] — unified deploy and update process for both servers
|
||||
- [[frank-primary-freetoken-pr587]] — the ACTIVE frank-primary runbook
|
||||
(2026-10-05): FreeToken built at PR #587 head `77a7201` on the host into
|
||||
- [[frank-primary-radiance-qwen38-native-fp8]] — the ACTIVE frank-primary
|
||||
runbook (2026-10-05): the StillDeadcode radiance engine (digest-pinned
|
||||
287 MiB image, libr4d gfx1201 kernels) serving the OFFICIAL
|
||||
`Qwen/Qwen3.8-Flash-Next-FP8` at native block-FP8 (no-recipe convert
|
||||
after the descriptor-staging drop) on BOTH R9700s TP=2 `wht6`, experts
|
||||
in a 40 GiB/rank pinned host pool, PLE 47.7 GiB pinned to RAM, fp8 KV
|
||||
7.32 KiB/token, MTP depth 3, 2 × 200,000 first boot; rollback is the
|
||||
FreeToken one-card era (tag `frank-primary-freetoken-pr587-20261005`),
|
||||
then the tcclaviger shape-G era (tag `frank-primary-vllm-shapeG-20261003`)
|
||||
- [[frank-primary-freetoken-pr587]] — RETIRED 2026-10-05 (superseded the
|
||||
same day by the radiance cutover above; kept as the rollback runbook):
|
||||
FreeToken built at PR #587 head `77a7201` on the host into
|
||||
`frank-primary-freetoken:pr587-77a7201`, ONE R9700 with experts in pinned
|
||||
host RAM behind a 0.10 GPU LRU, 2 slots, parsers AUTO; rollback is the
|
||||
tcclaviger shape-G era (tag `frank-primary-vllm-shapeG-20261003`)
|
||||
host RAM behind a 0.10 GPU LRU, 2 slots, parsers AUTO
|
||||
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
||||
- [[frank-primary-r9v-radiance-qwen38-q8]] — RETIRED 2026-09-30 (two swaps
|
||||
back; rollback target): the R9V radiance-vLLM fork (docker-pinned bundle
|
||||
@@ -359,14 +368,20 @@ The LLM inference fleet on the two hosts. Full index:
|
||||
- [[ciru-activefpx-promptforge]] — the RETIRED v3 CIRU PromptForge runtime on Dankclaw (superseded 2026-08-24)
|
||||
- [[kingjones-strix-mtp]] — the kingjones Qwen3.8-27B ROCmFP4 STRIX MTP model on Dankclaw (active)
|
||||
- [[dealign-ornith-crack]] — the retired Ornith-1.5-35B-A3B CRACK model on Dankserver (retired 2026-09-02)
|
||||
- [[frank-small-llama-mimo-q6k-2x200704]] — the ACTIVE frank-small runbook
|
||||
since 2026-10-03: llama.cpp master (CUDA sm_86) serving
|
||||
bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** on Dankclaw's RTX 3090,
|
||||
port 7771 — TWO conversations at 200,704 tokens each with fully
|
||||
unquantized f16 KV and mmproj vision; 94.4 tok/s decode (the Q6_K rung
|
||||
costs a measured −10 % vs Q5_K_M — weight-bandwidth), batch 4096 / ubatch
|
||||
2048 (+4.6 % prefill, measured 2026-10-05), 2,279 MiB free, and
|
||||
the ≥256 px image-probe trap (tiny images answer "video")
|
||||
- [[frank-small-llama-swift15-iq3s-mtp-131072]] — the ACTIVE frank-small
|
||||
runbook since 2026-10-05 (evening): llama.cpp master (CUDA sm_86) serving
|
||||
ukisai's Swift-1.5-Qwen3.8-27B GSQ-RCO IQ3_S WITH the in-file MTP head on
|
||||
the RTX 3090 — ONE 131,072-token conversation at Frank's 128k cap, q8_0
|
||||
K + q8_0 V, vision through the base-model projector; the arithmetic that
|
||||
makes one slot the maximum
|
||||
- [[frank-small-llama-mimo-q6k-2x200704]] — RETIRED 2026-10-05 (evening,
|
||||
superseded by swift15): llama.cpp master serving
|
||||
bartowski/MiMo-V2.6-Distill-Qwen-9B-GGUF **Q6_K** — TWO conversations at
|
||||
200,704 tokens each with fully unquantized f16 KV and mmproj vision;
|
||||
94.4 tok/s decode (the Q6_K rung costs a measured −10 % vs Q5_K_M —
|
||||
weight-bandwidth), batch 4096 / ubatch 2048 (+4.6 % prefill, measured
|
||||
2026-10-05), 2,279 MiB free, and the ≥256 px image-probe trap (tiny
|
||||
images answer "video")
|
||||
- [[frank-small-llama-mimo-q5km]] — RETIRED 2026-10-02 → 10-03 (rollback
|
||||
target, payload on disk; owns the measured boot-pair table): same runtime
|
||||
and shape one rung lower — 104.8 tok/s, 3,571 MiB true free
|
||||
|
||||
Reference in New Issue
Block a user