dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

docs(vault): destroy the dead weight — 53 pages and the change diary

Browse Source 
The vault was 155 pages / 31,210 lines, most of it measurement ledgers for
payloads that no longer run. A throughput figure for a runtime nobody operates is
not a record: AI moves faster than the vault, so the number is wrong before
anyone reads it, and a wrong number in a runbook gets believed. Per the new
purpose test in `AGENTS.md` → *Documentation Policy*, it is destroyed.

Gone (`.AGENTTRASH/20261019-destruction/`, KILL-LIST.txt is the manifest):
- the `frank-medium` endpoint and its five era pages, `frank-npu`, FastFlowLM,
  the ikllama/kingjones/dealign/frank-redo eras, the ling/swift15/freetoken/ubatch
  campaign pages, qwen3.8 spec-matrix and strix-optimizations, the two
  "Running Qwen3.6 …" pages;
- the whole `Pipeline Parallelism/` folder (no live script passes a pipeline flag)
  and the multi-device/x4-pair/hybrid/offload experiment pages;
- every vLLM page — no vLLM endpoint exists on this fleet;
- `LLM/deployment.md` (`Architecture/Deployment Flow.md` is the test-referenced
  owner) and the external surveys (SGLang, Lemonade, llama.cpp forks/diff,
  self-hosted app census, the TTS survey) that were obsolete by construction;
- `log.md`, 2,380 lines. Git holds the chronology; a causal law belongs to the
  page that owns the thing it governs, never to a diary.

Survivors: 101 pages, 17,542 lines, 0 OKF violations, 0 unresolved links. Eras
tables keep the window, payload, runtime, reason and rollback pointer; the rate
columns are blanked. The live shapes stay verbatim — those numbers are what the
deployer and `models.yml` consume.
This commit is contained in:
Frank
2026-10-19 10:56:37 -07:00
parent 8775e64ed0
commit 327e78b950
126 changed files with 443 additions and 14157 deletions
Download Patch File Download Diff File Expand all files Collapse all files
serverAppConfigs-docs/LLM/index.md
+3 -44
View File
@@ -1,9 +1,9 @@
# LLM fleet — index
The three live endpoint pages are the only pages that carry a shape. Everything
else here is a technique, a piece of hardware, or a trial that taught a law — and
each of those owns exactly that one thing. The incidents behind the laws are in
[[log]].
else here is a technique or a piece of hardware, and each of those owns exactly
that one thing. There is no worklog and no era page: a payload swap is one row in
the endpoint's **Eras** table, and the numbers a retired era measured are gone.
## Live endpoints
@@ -17,25 +17,11 @@ each of those owns exactly that one thing. The incidents behind the laws are in
| Page | Owns |
|---|---|
| [[frank-medium-batch-ubatch]] | The batch-versus-ubatch sweep on the layer-split pair at its deployed shape, and the rebalance the sweep forced. |
| [[frank-medium-llamacpp-dflash2]] | Retired era: the layer-split llama.cpp shape with a drafter, its ladder measurements, and the balance it settled on. |
| [[frank-medium-model-swap]] | Retired-era swap procedure: the fetch toolchain, the entrypoint contract, the capability battery, the rollback path. |
| [[frank-medium-radiance-mxfp4]] | Retired era: the tensor-parallel radiance shape on two cards, its quant-class swap, and the on-disk rollback payload. |
| [[frank-medium-rocmfpx]] | Retired era: the ROCmFPX fork serving an agent-tuned checkpoint on the two-card box, and the loader pin it required. |
| [[frank-npu]] | The retired NPU endpoint: what it served, why it died with the IOMMU decision, and where every dependent route went. |
| [[frank-redo-2026-08-17]] | The day the whole inference fleet was rebuilt: what came down, what replaced it, and why it was done that way. |
| [[ikllama-ornith-dankserver]] | Retired era: the ik_llama fork with a quantized value cache and CPU expert offload, and the build flags it required. |
| [[kingjones-q8agent-rocmfpx]] | Historical: a checkpoint and loader pairing staged for this host and never deployed on it. |
| [[qwen3.8-spec-matrix]] | The speculative-decoding benchmark history for this endpoint, arm by arm, and what each arm ruled out. |
## Pipeline Parallelism (series)
| Page | Owns |
|---|---|
| [[Ethernet Bandwidth Math]] | Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it. |
| [[Expected Performance]] | Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on. |
| [[Qwen3.8 PP Deployment]] | The real split deployment: which node is head and which is worker, the flags, and the verified gotchas. |
| [[Sample Script Set]] | An educational two-node RPC split script set, for reading only — never run it or deploy it. |
## Techniques, hardware, trials
@@ -45,46 +31,19 @@ each of those owns exactly that one thing. The incidents behind the laws are in
| [[Dankclaw System Specs]] | Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference. |
| [[dankclaw-iommu-translation-off]] | Why Dankclaw boots with identity DMA, what the earlier off-era bought and cost, and which tooling owns the boot line. |
| [[Dankserver System Specs]] | Dankserver hardware and OS state since the board swap: board, lane topology, cards, memory, kernel posture. |
| [[dankserver-frontier-moe-feasibility]] | The capacity arithmetic that decides whether a frontier mixture-of-experts fits this box before any model is loaded. |
| [[dankserver-vram-growth]] | Investigation of the recurring cuBLAS crash on frank-zealot. What grows, what does not, and what to do next. |
| [[dealign-ornith-crack]] | Retired era: a mixture-of-experts checkpoint served with CPU expert offload and n-gram drafting on a single card. |
| [[deployment]] | Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot). |
| [[DFlash DSpark Speculative Decoding]] | DFlash and DSpark are the two newest speculative decoding methods. |
| [[Dual-R9700-Cooperation]] | Why two R9700s could not run tensor-parallel on the old board, and what the board swap unblocked. |
| [[FastFlowLM NPU Inference]] | The AMD NPU-first runtime: fit analysis for the XDNA2, published benchmarks, install state, and test runbook. |
| [[FreeToken Builds]] | Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit. |
| [[freetoken-benchmark-sweep]] | KV pool versus decode speed for the FP8 mixture-of-experts payload on the FreeToken runtime. |
| [[GGUF Compatibility]] | Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can. |
| [[Heterogeneous Split Triage]] | Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys. |
| [[Hybrid CPU GPU Processing Dankserver]] | Dankserver uses NVIDIA GPU with system RAM for hybrid processing. |
| [[ling-3.0-flash-trial]] | Trial of a fork build for a new mixture-of-experts architecture: what loaded, what the harness refused, and the verdict. |
| [[Llama Cpp Multi-Device Splits]] | What llama.cpp can express across devices: the no-collective layer-split path, balancing, and drafter KV sizing. |
| [[Llama Cpp Serving Sizing]] | Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache. |
| [[LLM Bench Windows and Measurement]] | Opening and restoring benchmark windows on fleet hosts, honest methodology, and the usage-counter law under drafting. |
| [[LLM Performance Diagnosis]] | Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution. |
| [[Model Swap and Rollback]] | The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert. |
| [[PCIe Link and Atomics]] | Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives. |
| [[Quantization by Hardware Architecture]] | This page maps LLM quantization formats to the GPU and CPU architectures that support them. |
| [[Quantization Production]] | Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file. |
| [[qwen3.8-strix-optimizations]] | How a served checkpoint got faster on the Strix Halo iGPU: the lever ladder and what each lever actually bought. |
| [[RAM CPU Offloading in VLLM]] | vLLM uses CPU RAM and swap space to extend GPU memory capacity. |
| [[RDNA4 Container Serving]] | Bringing a model larger than VRAM onto the two-card box: quant gating, fetch verification, container GPU-init triage. |
| [[RDNA4 MoE Expert Offload VRAM Budget]] | Expert-offload arithmetic on the RDNA4 fork: movable terms, the utilisation law, graph charge, ceilings, context ladder. |
| [[Running Qwen3.6 27B FP8 on Dankclaw]] | Serving a dense FP8 vision-language checkpoint on the Strix Halo iGPU: the load path, the flags, the format gate. |
| [[Running Qwen3.6 35B A3B FP8 on Dankserver]] | Serving a mixture-of-experts FP8 checkpoint on the R9700 host: the load path, the format gate, the offload shape. |
| [[Shared Memory Architecture Dankclaw]] | Dankclaw uses AMD Strix Halo APU with unified memory architecture. |
| [[swift15-campaign-worklog]] | Dated worklog of the campaign that converted and served a Swift checkpoint on the radiance endpoint, arm by arm. |
| [[Tcclaviger Fork Ops]] | Operating the RDNA4 vLLM fork: the sweep harness, measured knob verdicts, sweep hygiene, compile-cache persistence. |
| [[ubatch-benchmark-large]] | The wide ubatch sweep for a deployed ROCmFPX shape, run through the standardized harness. |
| [[vLLM Boot Forensics and Config Interrogation]] | Recovering a container boot failure's real cause from journald, and interrogating a live endpoint's true config. |
| [[VLLM Index]] | Locator for every vLLM page in this vault. |
| [[VLLM Overview]] | vLLM is a fast and easy library for large language model inference and serving. |
| [[vLLM Porting and Compile Triage]] | Forward-porting a model architecture: additive-only discipline, shims that survive worker spawn, compile triage. |
| [[vllm_research_findings]] | Upstream support survey for speculative decoding: which flags, releases and pull requests carry each method. |
| [[x4-pair-experiment]] | Measured limits of a consumer card plus APU pair: the tensor inventory, the flag that kills cross-device placement. |
## Pages
| Page | Owns |
|---|---|
| [[frank-medium]] | Locator for the retired Dankserver LLM endpoint: which era page owns which payload, and where the dates disagree. |