dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: frank-medium runbook, fleet topology and client cutover

Browse Source 
New runbook page serverAppConfigs-docs/LLM/frank-medium-rocmfpx.md records
the hardware swap, the self-contained gfx1201 build, the measured layer-vs-
tensor decision (llama-bench multi-device syntax gotcha included), the
tensor-mode draft abort, served-path numbers, and the post-cleanup host ROCm
posture. AGENTS.md, README, docs indexes, Dankserver System Specs,
deployment/Hermes/OpenWebUI/Plex/Stack pages follow the rename;
freetoken-frank-small is marked RETIRED and kingjones-q8agent SUPERSEDED.
Work log entry added.
This commit is contained in:
Frank committed 2026-09-15 19:27:04 -07:00
1 parent d53962fec5
commit d4024a9180
15 files changed
+426 -80
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

serverAppConfigs-docs/LLM/index.md
+24 -18
View File
@@ -1,9 +1,10 @@
# LLM Fleet — Knowledge Base
Documentation for the LLM inference fleet on two local machines
(2026-09-11): FreeToken v0.1.2 on Dankserver (frank-small,
Huihui-Qwen3.6-35B-A3B-abliterated-FP8) and halogen-flash-server 0.5.8 on
Dankclaw (frank-large, peonist-ai Qwen3.8-Flash-Next w4b).
Documentation for the LLM inference fleet on the two hosts
(2026-09-15): charlie12345/ROCmFPX llama.cpp on Dankserver (frank-medium,
kingjones777 Qwen3.8-27B-ROCmFPX-Q8_0-AGENT on 2x Radeon AI PRO R9700) and
halogen-flash-server 0.5.8 on Dankclaw (frank-large, peonist-ai
Qwen3.8-Flash-Next w4b).
> 2026-08-22: the former `vllmScripts` repository is merged into the
> serverAppConfigs repository. This knowledge base moved from the old repo's
@@ -14,12 +15,13 @@ Dankclaw (frank-large, peonist-ai Qwen3.8-Flash-Next w4b).
```mermaid
graph LR
subgraph Dankserver[NVIDIA Dankserver - Ubuntu 26.04]
A[RTX 3090 24GB + 60GB RAM]
B2[FreeToken v0.1.2
frank-small, port 7770]
C[Huihui-Qwen3.6-35B-A3B-abliterated-FP8
block-fp8, 20% MoE residency, 453k KV / 4 streams]
subgraph Dankserver[AMD Dankserver - Ubuntu 26.04]
A[2x Radeon AI PRO R9700
gfx1201, 32 GiB each]
B2[ROCmFPX llama.cpp
frank-medium, port 7770]
C[Qwen3.8-27B-ROCmFPX-Q8_0-AGENT
layer split, 131k ctx / 1 slot, MTP]
A --> B2 --> C
end
@@ -40,14 +42,18 @@ w4b .hgn + overlay, 262k ctx / 4 slots]
| Server | GPU | VRAM / Memory | OS |
| --------------------------------------- | ----------------------------- | -------------------- | ------------ |
| [[Dankserver System Specs\|Dankserver]] | NVIDIA RTX 3090 | 24 GB VRAM, 62 GB RAM | Ubuntu 26.04 |
| [[Dankserver System Specs\|Dankserver]] | 2x AMD Radeon AI PRO R9700 | 2x 32 GB VRAM, 60 GB RAM | Ubuntu 26.04 |
| [[Dankclaw System Specs\|Dankclaw]] | AMD Radeon 8060S (Strix Halo) | gfx1151 unified memory | Ubuntu 26.04 |
## Active Deployment
- [[freetoken-frank-small]] — the active frank-small
runbook: FreeToken v0.1.2 serving batsclamp/Huihui-Qwen3.6-35B-A3B-
abliterated-FP8 on Dankserver (swapped 2026-08-30; same format/pools)
- [[frank-medium-rocmfpx]] — the ACTIVE frank-medium runbook:
charlie12345/ROCmFPX llama.cpp serving kingjones777/
Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF (MTP draft on) on Dankserver's
2x Radeon AI PRO R9700 (since 2026-09-15; replaced frank-small)
- [[freetoken-frank-small]] — RETIRED 2026-09-15 with the RTX 3090: the
FreeToken v0.1.2 frank-small runbook (batsclamp Huihui-Qwen3.6-35B-A3B-
abliterated-FP8), kept as the retirement record
- [[freetoken-benchmark-sweep]] — KV pool vs decode speed sweep
(bs1/bs3/bs4/bs6, 8k–700k KV) on Dankserver RTX 3090
- [[deployment]] — deploy and update process for both servers
@@ -80,7 +86,7 @@ w4b .hgn + overlay, 262k ctx / 4 slots]
on Dankclaw (superseded 2026-08-24 by the kingjones ROCmFP4 model)
- [[kairic-edge]] — the retired Kairic Edge IU4 experiment
(deployed and reverted 2026-08-22)
- [[Dankserver System Specs]] — the RTX 3090 box (frank-small)
- [[Dankserver System Specs]] — the 2x R9700 box (frank-medium)
- [[Dankclaw System Specs]] — the Strix Halo gfx1151 box (frank-large,
on halogen-flash-server since 2026-09-11)
- [[frank-npu]] — RETIRED 2026-09-11: the FLM NPU sight+sound endpoint
@@ -106,8 +112,8 @@ w4b .hgn + overlay, 262k ctx / 4 slots]
- [[VLLM Overview]] — installation, CLI, API, architecture, quantization, parallelism
- [[VLLM Index]] — quick reference with commands, endpoints, and parameter table
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]] — Qwen3.6-35B-A3B hardware
notes and sizing (this architecture family is what frank-small
serves today in the Huihui abliterated FP8 variant — see
notes and sizing (this architecture family is what the retired frank-small
served in the Huihui abliterated FP8 variant — see
[[freetoken-frank-small]]; the Ornith-1.5-35B-A3B CRACK checkpoint
that preceded it is archived at [[dealign-ornith-crack]])
@@ -136,7 +142,7 @@ w4b .hgn + overlay, 262k ctx / 4 slots]
- [[Llama Cpp Runtimes]] — mainline vs charlie12345/ROCmFPX vs ik_llama.cpp: positioning, quantization, server fit
- [[Llama Cpp Code Diff]] — code-level quantization diff of the three runtimes + server recommendations
- [[SGLang]] — the structured-generation serving engine; when it fits Dankserver (3090) vs Dankclaw (Strix Halo)
- [[SGLang]] — the structured-generation serving engine; when it fits Dankserver (2x R9700) vs Dankclaw (Strix Halo)
## Related