docs: record fleet double swap (large -> kingjones STRIX on pinned fork, small -> Huihui FP8 on FreeToken)
- kingjones-strix-rocmfp4 reactivated as the ACTIVE large runbook: runtime build recipe (WEBUI=OFF gotcha), deployed serving profile, fa-on-works-unpatched evidence, MTP heads documented unloadable, swap procedure + 2026-08-30 validation record - agentionai-fast-imatrix marked RETIRED (rollback record kept) - freetoken-frank-default-small: model/lineage/template sections now Huihui abliterated FP8; new operational quirk: chat API 422 without explicit model field - LLM index + AGENTS fleet table/runtime sections current; both services' history chains fixed (small was still Tiel in AGENTS.md) - worklog entry added; models.json contextWindow for large corrected to the real per-slot 131072 (llama clamps n_ctx_train/parallel)
This commit is contained in:
@@ -1,10 +1,10 @@
|
||||
# LLM Fleet — Knowledge Base
|
||||
|
||||
Documentation for the LLM inference fleet on two local machines
|
||||
(2026-08-29): FreeToken v0.1.2 on Dankserver (frank-default-small,
|
||||
Qwen3.6-35B-A3B-FP8) and ROCmFPX llama.cpp on Dankclaw
|
||||
(frank-default-large, agentionai Qwen3.8-Flash-Next ROCmFP4-FAST
|
||||
imatrix + MTP draft). Both services run the Qwen-Sharp chat template.
|
||||
(2026-08-30): FreeToken v0.1.2 on Dankserver (frank-default-small,
|
||||
Huihui-Qwen3.6-35B-A3B-abliterated-FP8) and kingjones30/ROCmFPX
|
||||
llama.cpp on Dankclaw (frank-default-large, kingjones777
|
||||
Qwen3.8-Flash-Next ROCmFP4-STRIX). Both services run the Qwen-Sharp chat template.
|
||||
|
||||
> 2026-08-22: the former `vllmScripts` repository is merged into the
|
||||
> serverAppConfigs repository. This knowledge base moved from the old repo's
|
||||
@@ -19,17 +19,17 @@ graph LR
|
||||
A[RTX 3090 24GB + 60GB RAM]
|
||||
B2[FreeToken v0.1.2
|
||||
frank-default-small, port 7770]
|
||||
C[Qwen3.6-35B-A3B-FP8
|
||||
C[Huihui-Qwen3.6-35B-A3B-abliterated-FP8
|
||||
block-fp8, 20% MoE residency, 453k KV / 4 streams]
|
||||
A --> B2 --> C
|
||||
end
|
||||
|
||||
subgraph Dankclaw[AMD Strix Halo - Ubuntu 26.04]
|
||||
E2[gfx1151 iGPU 64 GiB unified memory]
|
||||
E[charlie12345/ROCmFPX build-qwen38
|
||||
E[kingjones30/ROCmFPX build
|
||||
frank-default-large, port 7770]
|
||||
F[agentionai Qwen3.8-Flash-Next
|
||||
ROCmFP4-FAST imatrix + MTP draft, 262K ctx / 1 slot]
|
||||
F[kingjones777 Qwen3.8-Flash-Next
|
||||
ROCmFP4-STRIX, 401k ctx / 2 slots, mmap PLE]
|
||||
E2 --> E --> F
|
||||
end
|
||||
|
||||
@@ -47,19 +47,20 @@ ROCmFP4-FAST imatrix + MTP draft, 262K ctx / 1 slot]
|
||||
## Active Deployment
|
||||
|
||||
- [[freetoken-frank-default-small]] — the active frank-default-small
|
||||
runbook: FreeToken v0.1.2 serving Qwen3.6-35B-A3B-FP8 on Dankserver
|
||||
(2026-09-02)
|
||||
runbook: FreeToken v0.1.2 serving batsclamp/Huihui-Qwen3.6-35B-A3B-
|
||||
abliterated-FP8 on Dankserver (swapped 2026-08-30; same format/pools)
|
||||
- [[freetoken-benchmark-sweep]] — KV pool vs decode speed sweep
|
||||
(bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
|
||||
- [[deployment]] — deploy and update process for both servers
|
||||
- [[frank-redo-2026-08-17]] — the 2026-08-17 topology change
|
||||
- [[agentionai-fast-imatrix]] — ACTIVE frank-default-large runbook:
|
||||
agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB
|
||||
Q4_0_ROCMFP4_FAST, per-head PLE in GGUF) + own MTP draft head
|
||||
(ROCmFP4-FAST, 2.28 GiB), build-qwen38 runtime (2026-08-29)
|
||||
- [[kingjones-strix-rocmfp4]] — retired frank-default-large runbook:
|
||||
- [[kingjones-strix-rocmfp4]] — ACTIVE frank-default-large runbook:
|
||||
kingjones777/Qwen3.8-Flash-Next-ROCmFP4-STRIX-GGUF, 177B params
|
||||
(56B MoE + 51.2B PLE), single 113.47 GiB GGUF, STRIX tier
|
||||
(56B MoE + 51.2B PLE), single 113.47 GiB GGUF, on the card's pinned
|
||||
kingjones30/ROCmFPX fork (swapped 2026-08-30)
|
||||
- [[agentionai-fast-imatrix]] — retired frank-default-large runbook:
|
||||
agentionai/Qwen3.8-Flash-Next-ROCmFP4-FAST-imatrix (87.06 GiB
|
||||
Q4_0_ROCMFP4_FAST imatrix, v2 joined PLE) on build-qwen38
|
||||
(2026-08-29 → 2026-08-30); rollback binary + payload retained
|
||||
- [[ubatch-benchmark-large]] — active-model llama-bench ubatch sweep plan and results
|
||||
- [[kingjones-q8agent-rocmfpx]] — the retired Q8_0-AGENT investigation and `set_rows` failure
|
||||
- [[eemin-rocmi4-mtp]] — the former caf-eemin Qwen3.8-27B embedded-MTP model
|
||||
|
||||
Reference in New Issue
Block a user