docs: record fleet double swap (large -> kingjones STRIX on pinned fork, small -> Huihui FP8 on FreeToken)

- kingjones-strix-rocmfp4 reactivated as the ACTIVE large runbook:
  runtime build recipe (WEBUI=OFF gotcha), deployed serving profile,
  fa-on-works-unpatched evidence, MTP heads documented unloadable,
  swap procedure + 2026-08-30 validation record
- agentionai-fast-imatrix marked RETIRED (rollback record kept)
- freetoken-frank-default-small: model/lineage/template sections now
  Huihui abliterated FP8; new operational quirk: chat API 422 without
  explicit model field
- LLM index + AGENTS fleet table/runtime sections current; both
  services' history chains fixed (small was still Tiel in AGENTS.md)
- worklog entry added; models.json contextWindow for large corrected
  to the real per-slot 131072 (llama clamps n_ctx_train/parallel)
This commit is contained in:
Frank
2026-08-30 02:35:38 -07:00
parent aa9c1d065a
commit 0937cfb047
6 changed files with 268 additions and 152 deletions
+17 -16
View File
@@ -1,10 +1,10 @@
# LLM Fleet — Knowledge Base
Documentation for the LLM inference fleet on two local machines
(2026-08-29): FreeToken v0.1.2 on Dankserver (frank-default-small,
Qwen3.6-35B-A3B-FP8) and ROCmFPX llama.cpp on Dankclaw
(frank-default-large, agentionai Qwen3.8-Flash-Next ROCmFP4-FAST
imatrix + MTP draft). Both services run the Qwen-Sharp chat template.
(2026-08-30): FreeToken v0.1.2 on Dankserver (frank-default-small,
Huihui-Qwen3.6-35B-A3B-abliterated-FP8) and kingjones30/ROCmFPX
llama.cpp on Dankclaw (frank-default-large, kingjones777
Qwen3.8-Flash-Next ROCmFP4-STRIX). Both services run the Qwen-Sharp chat template.
> 2026-08-22: the former `vllmScripts` repository is merged into the
> serverAppConfigs repository. This knowledge base moved from the old repo's
@@ -19,17 +19,17 @@ graph LR
A[RTX 3090 24GB + 60GB RAM]
B2[FreeToken v0.1.2
frank-default-small, port 7770]
C[Qwen3.6-35B-A3B-FP8
C[Huihui-Qwen3.6-35B-A3B-abliterated-FP8
block-fp8, 20% MoE residency, 453k KV / 4 streams]
A --> B2 --> C
end
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
E2[gfx1151 iGPU 64 GiB unified memory]
E[charlie12345/ROCmFPX build-qwen38
E[kingjones30/ROCmFPX build
frank-default-large, port 7770]
F[agentionai Qwen3.8-Flash-Next
ROCmFP4-FAST imatrix + MTP draft, 262K ctx / 1 slot]
F[kingjones777 Qwen3.8-Flash-Next
ROCmFP4-STRIX, 401k ctx / 2 slots, mmap PLE]
E2 --> E --> F
end
@@ -47,19 +47,20 @@ ROCmFP4-FAST imatrix + MTP draft, 262K ctx / 1 slot]
## Active Deployment
- [[freetoken-frank-default-small]] — the active frank-default-small
runbook: FreeToken v0.1.2 serving Qwen3.6-35B-A3B-FP8 on Dankserver
(2026-09-02)
runbook: FreeToken v0.1.2 serving batsclamp/Huihui-Qwen3.6-35B-A3B-
abliterated-FP8 on Dankserver (swapped 2026-08-30; same format/pools)
- [[freetoken-benchmark-sweep]] — KV pool vs decode speed sweep
(bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
- [[deployment]] — deploy and update process for both servers
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
- [[agentionai-fast-imatrix]] — ACTIVE frank-default-large runbook:
agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB
Q4_0_ROCMFP4_FAST, per-head PLE in GGUF) + own MTP draft head
(ROCmFP4-FAST, 2.28 GiB), build-qwen38 runtime (2026-08-29)
- [[kingjones-strix-rocmfp4]] — retired frank-default-large runbook:
- [[kingjones-strix-rocmfp4]] — ACTIVE frank-default-large runbook:
kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF, 177B params
(56B MoE + 51.2B PLE), single 113.47 GiB GGUF, STRIX tier
(56B MoE + 51.2B PLE), single 113.47 GiB GGUF, on the card's pinned
kingjones30/ROCmFPX fork (swapped 2026-08-30)
- [[agentionai-fast-imatrix]] — retired frank-default-large runbook:
agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB
Q4_0_ROCMFP4_FAST imatrix, v2 joined PLE) on build-qwen38
(2026-08-29 → 2026-08-30); rollback binary + payload retained
- [[ubatch-benchmark-large]] — active-model llama-bench ubatch sweep plan and results
- [[kingjones-q8agent-rocmfpx]] — the retired Q8_0-AGENT investigation and `set_rows` failure
- [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B embedded-MTP model