Frank
6a8cbde260
The root and LLM index pages still advertised two retired endpoints' pages as live procedure: - `frank-medium-radiance-mxfp4`: frontmatter said "ACTIVE frank-medium runbook" and the body opened "The ACTIVE frank-medium deployment". Both re-keyed to the era it ran (2026-09-21 -> 10-10), with [[frank-archon]]'s Eras table named as the owner of that shape. - `frank-medium-model-swap`: read as live how-to for a retired endpoint. Re-keyed to historical, pointing at [[Model Swap and Rollback]] for the current swap law. - Both index pages' rows re-keyed to match (rows are hand-copied from frontmatter - no generator - so both sites move together). Refs: vault de-dup campaign, cluster C5.
90 lines
8.8 KiB
Markdown
90 lines
8.8 KiB
Markdown
# LLM fleet — index
|
|
|
|
The three live endpoint pages are the only pages that carry a shape. Everything
|
|
else here is a technique, a piece of hardware, or a trial that taught a law — and
|
|
each of those owns exactly that one thing. The incidents behind the laws are in
|
|
[[log]].
|
|
|
|
## Live endpoints
|
|
|
|
| Page | Owns |
|
|
|---|---|
|
|
| [[frank-archon]] | The one frank-archon page: the live radiance shape and harness contract (read off scripts/frank-archon.env), the Eras… |
|
|
| [[frank-dragoon]] | The one runbook for frank-dragoon, the Strix Halo iGPU endpoint (192.168.0.143:7770): current shape from the env,… |
|
|
| [[frank-zealot]] | The one frank-zealot reference: the live shape on Dankclaw's RTX 3090 (:7771) exactly as scripts/frank-zealot.env… |
|
|
|
|
## Retired endpoints
|
|
|
|
| Page | Owns |
|
|
|---|---|
|
|
| [[frank-medium-batch-ubatch]] | llama-bench sweep of --batch-size against --ubatch-size on frank-medium (2x Radeon AI PRO R9700, layer split) at the… |
|
|
| [[frank-medium-llamacpp-dflash2]] | The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on… |
|
|
| [[frank-medium-model-swap]] | RETIRED-era swap procedure for the frank-medium endpoint (`:7770` is now [[frank-archon]]) — the… |
|
|
| [[frank-medium-radiance-mxfp4]] | RETIRED frank-medium era runbook — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8,… |
|
|
| [[frank-medium-rocmfpx]] | charlie12345/ROCmFPX llama.cpp serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF as frank-medium on Dankserver… |
|
|
| [[frank-npu]] | The retired FastFlowLM endpoint on Dankclaw's XDNA2 NPU — what it served, why it died on 2026-09-11 with the… |
|
|
| [[frank-redo-2026-08-17]] | This page records the full redo of the inference fleet on 2026-08-17. |
|
|
| [[ikllama-ornith-dankserver]] | ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-zealot on the RTX 3090. Q8_0 model, KV K default f16 / V… |
|
|
| [[kingjones-q8agent-rocmfpx]] | **Status (historical, 2026-09-13):** was to deploy on Dankclaw. |
|
|
| [[qwen3.8-spec-matrix]] | This page records the speculative-decoding benchmark history and the |
|
|
|
|
## Pipeline Parallelism (series)
|
|
|
|
| Page | Owns |
|
|
|---|---|
|
|
| [[Ethernet Bandwidth Math]] | Bytes-per-token math for pipeline vs tensor parallelism, feasibility tables for 1/2.5/10 GbE, and real multi-node… |
|
|
| [[Expected Performance]] | Estimated tokens per second for a 25-40B model split across two nodes over 1/2.5/10 GbE, with real benchmark anchors. |
|
|
| [[llama.cpp RPC]] | llama.cpp distributed inference via ggml-rpc-server and the --rpc flag: architecture, CLI, versions, limitations, and… |
|
|
| [[Mixed-Vendor Experiments]] | Real-world NVIDIA + AMD Strix Halo multi-node experiments: what works (llama.cpp RPC, Vulkan split), what breaks… |
|
|
| [[Qwen3.6 27B Spec Methods]] | Qwen3.6-27B Q8 GGUF variants and llama.cpp speculative decoding (draft-mtp, draft-dflash, ngram-mod): flags, RPC… |
|
|
| [[Qwen3.8 PP Deployment]] | Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX |
|
|
| [[Sample Script Set]] | An educational two-node llama.cpp RPC split of Qwen3.6-27B-Q8_K_P over ethernet. For reading only. Do not run or… |
|
|
| [[vLLM Pipeline Parallelism]] | vLLM multi-node pipeline parallelism: flags, Ray/mp backends, interconnect requirements, PP bubble, and PP +… |
|
|
|
|
## Techniques, hardware, trials
|
|
|
|
| Page | Owns |
|
|
|---|---|
|
|
| [[Dankclaw Improvements]] | This page collects concrete, sourced recommendations to improve Dankclaw |
|
|
| [[Dankclaw System Specs]] | Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference. |
|
|
| [[dankclaw-iommu-translation-off]] | Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a… |
|
|
| [[Dankserver System Specs]] | Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI… |
|
|
| [[dankserver-frontier-moe-feasibility]] | VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and… |
|
|
| [[dankserver-vram-growth]] | Investigation of the recurring cuBLAS crash on frank-zealot. What grows, what does not, and what to do next. |
|
|
| [[dealign-ornith-crack]] | dealignai/Ornith-1.5-35B-A3B-UNCENSORED (CRACK Q6_K) served by upstream llama.cpp on the RTX 3090. MoE model with… |
|
|
| [[deployment]] | Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot). |
|
|
| [[DFlash DSpark Speculative Decoding]] | DFlash and DSpark are the two newest speculative decoding methods. |
|
|
| [[Dual-R9700-Cooperation]] | Why frank-medium's two Radeon AI PRO R9700s could not run tensor-parallel on the old B650-E board (PCIe atomic… |
|
|
| [[FastFlowLM NPU Inference]] | FastFlowLM (FLM), the AMD NPU-first LLM runtime for Ryzen AI XDNA2 chips. Fit analysis for the 50 TOPS NPU on… |
|
|
| [[FreeToken Builds]] | Building FreeToken (the official franksdank fork of FreeTok Tokenworks' freetoken vLLM-MTP runtime) on Dankclaw — the… |
|
|
| [[freetoken-benchmark-sweep]] | KV pool vs decode speed sweep across bs1/bs3/bs4/bs6 on Dankserver with Qwen3.6-35B-A3B-FP8 via FreeToken v0.1.2. |
|
|
| [[GGUF Compatibility]] | How to answer "will engine X load this GGUF?" from the file's header alone, why a custom tensor type forces the… |
|
|
| [[Heterogeneous Split Triage]] | Triage before building when someone asks to split a model across unequal accelerators on one host (24 GB GDDR6X +… |
|
|
| [[Hybrid CPU GPU Processing Dankserver]] | Dankserver uses NVIDIA GPU with system RAM for hybrid processing. |
|
|
| [[ling-3.0-flash-trial]] | `bailingmoe3-support` (commit `db4480bc8`, build b10411) at `/home/dank/llama.cpp-ling`. |
|
|
| [[ling-3.0-flash-vllm-plan]] | This page was the plan for a vLLM-ROCm deployment of `jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native` (INT4 W4A16, ~62… |
|
|
| [[Llama Cpp Multi-Device Splits]] | What llama.cpp can and cannot express across devices, the no-collective layer-split path, split balancing by… |
|
|
| [[Llama Cpp Serving Sizing]] | Slot-pool sizing against a headroom band, per-device VRAM accounting, prompt-cache/state sizing and the QSA… |
|
|
| [[LLM Bench Windows and Measurement]] | Stopping-and-restoring benchmark windows on the fleet LLM hosts (llama-bench sweeps, HTTP param benches, multi-arm… |
|
|
| [[LLM Performance Diagnosis]] | Probing why a served LLM is slow — the flat-depth probe, the expert-offload duty-cycle arithmetic, kernel-time… |
|
|
| [[Model Swap and Rollback]] | The full model-swap discipline for the LLM fleet — fit preflight, fetch and payload verification, runtime build/flag… |
|
|
| [[PCIe Link and Atomics]] | Link probing (idle downshift, path ceilings, CUDA/HIP bandwidth probes), the streaming-vs-compute ceiling arithmetic,… |
|
|
| [[Quantization by Hardware Architecture]] | This page maps LLM quantization formats to the GPU and CPU architectures that support them. |
|
|
| [[Quantization Production]] | — |
|
|
| [[qwen3.8-strix-optimizations]] | This page answers one question: **how do we make Qwen3.8-27B serve faster on |
|
|
| [[RAM CPU Offloading in VLLM]] | vLLM uses CPU RAM and swap space to extend GPU memory capacity. |
|
|
| [[RDNA4 Container Serving]] | Bringing a model larger than VRAM onto Dankserver's 2x R9700 (gfx1201) with the tcclaviger vLLM stack — quant-class… |
|
|
| [[RDNA4 MoE Expert Offload VRAM Budget]] | The expert-offload plan arithmetic on the tcclaviger fork — which terms a deploy can move, the util law, the… |
|
|
| [[Running Qwen3.6 27B FP8 on Dankclaw]] | Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series. |
|
|
| [[Running Qwen3.6 35B A3B FP8 on Dankserver]] | Qwen3.6-35B-A3B is a Mixture-of-Experts vision-language model from the Qwen3.6 series. |
|
|
| [[Shared Memory Architecture Dankclaw]] | Dankclaw uses AMD Strix Halo APU with unified memory architecture. |
|
|
| [[swift15-campaign-worklog]] | — |
|
|
| [[Tcclaviger Fork Ops]] | Operating the tcclaviger RDNA4 vLLM fork on dankserver (shape G era, rollback target since the 2026-10-05 FreeToken… |
|
|
| [[ubatch-benchmark-large]] | Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment. |
|
|
| [[vLLM Boot Forensics and Config Interrogation]] | Capturing a vLLM container boot failure's real root cause from journald (the container log is already gone),… |
|
|
| [[VLLM Index]] | This index contains all documentation for vLLM deployment on Dankserver and Dankclaw. |
|
|
| [[VLLM Overview]] | vLLM is a fast and easy library for large language model inference and serving. |
|
|
| [[vLLM Porting and Compile Triage]] | Forward-porting a model architecture to a new vLLM major (additive-only discipline, gap enumeration, shims that… |
|
|
| [[vllm_research_findings]] | vLLM v0.26.0 (latest stable, released July 25, 2026) supports DFlash speculative decoding with the `--spec-method… |
|
|
| [[x4-pair-experiment]] | What mainline llama.cpp can and cannot do with a 3090 + Strix Halo combination, measured 2026-09-16 on Dankclaw — the… |
|