serverAppConfigs/serverAppConfigs-docs/LLM/index.md

# LLM fleet — index

The three live endpoint pages are the only pages that carry a shape. Everything
else here is a technique, a piece of hardware, or a trial that taught a law — and
each of those owns exactly that one thing. The incidents behind the laws are in
[[log]].

## Live endpoints

| Page | Owns |
|---|---|
| [[frank-archon]] | The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught. |
| [[frank-dragoon]] | The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it. |
| [[frank-zealot]] | The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws. |

## Retired endpoints

| Page | Owns |
|---|---|
| [[frank-medium-batch-ubatch]] | The batch-versus-ubatch sweep on the layer-split pair at its deployed shape, and the rebalance the sweep forced. |
| [[frank-medium-llamacpp-dflash2]] | Retired era: the layer-split llama.cpp shape with a drafter, its ladder measurements, and the balance it settled on. |
| [[frank-medium-model-swap]] | Retired-era swap procedure: the fetch toolchain, the entrypoint contract, the capability battery, the rollback path. |
| [[frank-medium-radiance-mxfp4]] | Retired era: the tensor-parallel radiance shape on two cards, its quant-class swap, and the on-disk rollback payload. |
| [[frank-medium-rocmfpx]] | Retired era: the ROCmFPX fork serving an agent-tuned checkpoint on the two-card box, and the loader pin it required. |
| [[frank-npu]] | The retired NPU endpoint: what it served, why it died with the IOMMU decision, and where every dependent route went. |
| [[frank-redo-2026-08-17]] | The day the whole inference fleet was rebuilt: what came down, what replaced it, and why it was done that way. |
| [[ikllama-ornith-dankserver]] | Retired era: the ik_llama fork with a quantized value cache and CPU expert offload, and the build flags it required. |
| [[kingjones-q8agent-rocmfpx]] | Historical: a checkpoint and loader pairing staged for this host and never deployed on it. |
| [[qwen3.8-spec-matrix]] | The speculative-decoding benchmark history for this endpoint, arm by arm, and what each arm ruled out. |

## Pipeline Parallelism (series)

| Page | Owns |
|---|---|
| [[Ethernet Bandwidth Math]] | Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it. |
| [[Expected Performance]] | Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on. |
| [[Qwen3.8 PP Deployment]] | The real split deployment: which node is head and which is worker, the flags, and the verified gotchas. |
| [[Sample Script Set]] | An educational two-node RPC split script set, for reading only — never run it or deploy it. |

## Techniques, hardware, trials

| Page | Owns |
|---|---|
| [[Dankclaw Improvements]] | This page collects concrete, sourced recommendations to improve Dankclaw |
| [[Dankclaw System Specs]] | Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference. |
| [[dankclaw-iommu-translation-off]] | Why Dankclaw boots with identity DMA, what the earlier off-era bought and cost, and which tooling owns the boot line. |
| [[Dankserver System Specs]] | Dankserver hardware and OS state since the board swap: board, lane topology, cards, memory, kernel posture. |
| [[dankserver-frontier-moe-feasibility]] | The capacity arithmetic that decides whether a frontier mixture-of-experts fits this box before any model is loaded. |
| [[dankserver-vram-growth]] | Investigation of the recurring cuBLAS crash on frank-zealot. What grows, what does not, and what to do next. |
| [[dealign-ornith-crack]] | Retired era: a mixture-of-experts checkpoint served with CPU expert offload and n-gram drafting on a single card. |
| [[deployment]] | Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot). |
| [[DFlash DSpark Speculative Decoding]] | DFlash and DSpark are the two newest speculative decoding methods. |
| [[Dual-R9700-Cooperation]] | Why two R9700s could not run tensor-parallel on the old board, and what the board swap unblocked. |
| [[FastFlowLM NPU Inference]] | The AMD NPU-first runtime: fit analysis for the XDNA2, published benchmarks, install state, and test runbook. |
| [[FreeToken Builds]] | Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit. |
| [[freetoken-benchmark-sweep]] | KV pool versus decode speed for the FP8 mixture-of-experts payload on the FreeToken runtime. |
| [[GGUF Compatibility]] | Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can. |
| [[Heterogeneous Split Triage]] | Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys. |
| [[Hybrid CPU GPU Processing Dankserver]] | Dankserver uses NVIDIA GPU with system RAM for hybrid processing. |
| [[ling-3.0-flash-trial]] | Trial of a fork build for a new mixture-of-experts architecture: what loaded, what the harness refused, and the verdict. |
| [[Llama Cpp Multi-Device Splits]] | What llama.cpp can express across devices: the no-collective layer-split path, balancing, and drafter KV sizing. |
| [[Llama Cpp Serving Sizing]] | Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache. |
| [[LLM Bench Windows and Measurement]] | Opening and restoring benchmark windows on fleet hosts, honest methodology, and the usage-counter law under drafting. |
| [[LLM Performance Diagnosis]] | Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution. |
| [[Model Swap and Rollback]] | The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert. |
| [[PCIe Link and Atomics]] | Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives. |
| [[Quantization by Hardware Architecture]] | This page maps LLM quantization formats to the GPU and CPU architectures that support them. |
| [[Quantization Production]] | Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file. |
| [[qwen3.8-strix-optimizations]] | How a served checkpoint got faster on the Strix Halo iGPU: the lever ladder and what each lever actually bought. |
| [[RAM CPU Offloading in VLLM]] | vLLM uses CPU RAM and swap space to extend GPU memory capacity. |
| [[RDNA4 Container Serving]] | Bringing a model larger than VRAM onto the two-card box: quant gating, fetch verification, container GPU-init triage. |
| [[RDNA4 MoE Expert Offload VRAM Budget]] | Expert-offload arithmetic on the RDNA4 fork: movable terms, the utilisation law, graph charge, ceilings, context ladder. |
| [[Running Qwen3.6 27B FP8 on Dankclaw]] | Serving a dense FP8 vision-language checkpoint on the Strix Halo iGPU: the load path, the flags, the format gate. |
| [[Running Qwen3.6 35B A3B FP8 on Dankserver]] | Serving a mixture-of-experts FP8 checkpoint on the R9700 host: the load path, the format gate, the offload shape. |
| [[Shared Memory Architecture Dankclaw]] | Dankclaw uses AMD Strix Halo APU with unified memory architecture. |
| [[swift15-campaign-worklog]] | Dated worklog of the campaign that converted and served a Swift checkpoint on the radiance endpoint, arm by arm. |
| [[Tcclaviger Fork Ops]] | Operating the RDNA4 vLLM fork: the sweep harness, measured knob verdicts, sweep hygiene, compile-cache persistence. |
| [[ubatch-benchmark-large]] | The wide ubatch sweep for a deployed ROCmFPX shape, run through the standardized harness. |
| [[vLLM Boot Forensics and Config Interrogation]] | Recovering a container boot failure's real cause from journald, and interrogating a live endpoint's true config. |
| [[VLLM Index]] | Locator for every vLLM page in this vault. |
| [[VLLM Overview]] | vLLM is a fast and easy library for large language model inference and serving. |
| [[vLLM Porting and Compile Triage]] | Forward-porting a model architecture: additive-only discipline, shims that survive worker spawn, compile triage. |
| [[vllm_research_findings]] | Upstream support survey for speculative decoding: which flags, releases and pull requests carry each method. |
| [[x4-pair-experiment]] | Measured limits of a consumer card plus APU pair: the tensor inventory, the flag that kills cross-device placement. |

## Pages

| Page | Owns |
|---|---|
| [[frank-medium]] | Locator for the retired Dankserver LLM endpoint: which era page owns which payload, and where the dates disagree. |