docs(vault): one fact, one owner — repair the doubled pages, single-source the facts

Structural repair: - 13 pages were the same document twice (each exactly 2x its HEAD size), a second frontmatter block whose closing fence had been glued onto a body line by a frontmatter rewrite that re-read its own output. Un-doubled by cutting at the second frontmatter and keeping the copy whose body is a superset; the orphan census reports 0 lost tokens against HEAD. - log.md: one entry per date, bare `## YYYY-MM-DD` headings, newest first — 66 dated headings merged into 34, titles demoted to bold leads, every content line verified present. The file had been descending at the head and ascending at the tail. Ownership moves: - New `LLM/frank-medium.md` is the locator for the retired Dankserver endpoint: era windows quoted from each era page's own banner, no numbers restated, and the two unresolved date conflicts recorded instead of silently resolved. `frank-archon.md` links to it instead of restating the pre-rename window. - The four Pipeline-Parallelism stubs (frontmatter only, no body) are gone; `Multi-Node Runtimes.md` owns runtime construction, launch, mixed-vendor verdicts and cross-split spec decode, and the four links that pointed at the stubs were repointed at it or at the drafter page. - Authentik's three stub pages fold into `Authentik.md`, with the process detail and the outpost reference split out so the parent stays under the length hint. Verified corrections (host census + live probe, 2026-10-19): - The MXFP4 checkpoint two pages called the "on-disk rollback" is gone: /home/dank/models/ holds StillDeadcode, rad and tpl.jinja. Stated once, on the era page that owns that payload; the swap page now links to it. - frank-medium-radiance-mxfp4: the 4.24/7 accept rate belongs to the DFlash2 arm measured on the FP8 target, not to the MXFP4 era; the 3-family soak run census is kept once (155 requests) instead of twice (~120 and 155). - Fleet shapes re-confirmed against the live endpoints before touching any page: 3x200000 fp8 KV, 2x200704, 2x262144. Index rows are regenerated, not hand-carved. Net -1,849 vault lines.
2026-10-19 05:00:43 -07:00
1 parent 3880503057
commit a5fe045b58
@@ -1,7 +1,7 @@
 ---
-type: configuration
+type: reference
 title: Dankserver System Specs
-description: Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI PRO R9700 gfx1201 on CPU lanes x8/x8) serving frank-medium.
+description: "Dankserver hardware and OS state since the board swap: board, lane topology, cards, memory, kernel posture."
 tags:
  - dankserver
  - hardware
@@ -1,7 +1,7 @@
 ---
-type: research
+type: reference
 title: Dual R9700 Cooperation
-description: Why frank-medium's two Radeon AI PRO R9700s could not run tensor-parallel on the old B650-E board (PCIe atomic routing), what the 2026-09-20 ProArt B850-Creator swap unblocked, and what still has to be probed.
+description: "Why two R9700s could not run tensor-parallel on the old board, and what the board swap unblocked."
 tags:
  - frank-medium
  - rocm
@@ -1,7 +1,7 @@
 ---
-type: research
+type: reference
 title: FastFlowLM NPU Inference
-description: FastFlowLM (FLM), the AMD NPU-first LLM runtime for Ryzen AI XDNA2 chips. Fit analysis for the 50 TOPS NPU on Dankclaw, published benchmarks, verified install state, and a test runbook.
+description: "The AMD NPU-first runtime: fit analysis for the XDNA2, published benchmarks, install state, and test runbook."
 tags:
  - fastflowlm
  - npu
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: FreeToken Builds
-description: Building FreeToken (the official franksdank fork of FreeTok Tokenworks' freetoken vLLM-MTP runtime) on Dankclaw — the CUDA vLLM venv build and the ROCm build, with the dependency traps each one hit.
+description: "Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit."
 tags: [freetoken, build, dankclaw, vllm, mtp]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: GGUF Compatibility and Runtime Lock-in
-description: How to answer "will engine X load this GGUF?" from the file's header alone, why a custom tensor type forces the runtime, and the discipline for pinning a fork build that can actually serve the file.
+description: "Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can."
 tags: [gguf, llama.cpp, forks, quantization, speculative]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Heterogeneous Split Triage
-description: Triage before building when someone asks to split a model across unequal accelerators on one host (24 GB GDDR6X + LPDDR APU, PCIe x4 link) — link characterisation, the real bottleneck, what placement never buys, UMA pools, drafter splits, and the verdict ranking.
+description: "Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys."
 tags: [heterogeneous, multi-device, llama-cpp, triage, pcie]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: LLM Bench Windows and Measurement
-description: Stopping-and-restoring benchmark windows on the fleet LLM hosts (llama-bench sweeps, HTTP param benches, multi-arm vLLM offload sweeps), honest benchmark methodology, and the usage-counter measurement law under speculative decoding.
+description: "Opening and restoring benchmark windows on fleet hosts, honest methodology, and the usage-counter law under drafting."
 tags: [benchmarking, llama-bench, ab-testing, speculative, vllm, fleet]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: LLM Performance Diagnosis
-description: Probing why a served LLM is slow — the flat-depth probe, the expert-offload duty-cycle arithmetic, kernel-time attribution via the profiler window, live telemetry without benchmarking, RAM-bandwidth ceilings, IBS tensor attribution and prefix-cache-aware routing.
+description: "Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution."
 tags: [performance, profiling, moe, pcie, prefix-cache, routing, telemetry]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Llama Cpp Multi-Device Splits
-description: What llama.cpp can and cannot express across devices, the no-collective layer-split path, split balancing by measurement, drafter KV sizing from GGUF headers, and heterogeneous placement triage.
+description: "What llama.cpp can express across devices: the no-collective layer-split path, balancing, and drafter KV sizing."
 tags: [llama.cpp, multi-gpu, layer-split, pcie, sizing, speculative]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Llama Cpp Serving Sizing
-description: Slot-pool sizing against a headroom band, per-device VRAM accounting, prompt-cache/state sizing and the QSA flash-attn fix — the measurement-first procedures for llama.cpp serving shapes.
+description: "Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache."
 tags: [llama.cpp, vram, kv-cache, slots, flash-attn, sizing]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Model Swap and Rollback
-description: The full model-swap discipline for the LLM fleet — fit preflight, fetch and payload verification, runtime build/flag checks, git-only config, live validation, systemd handoff, close-out, the three-pin revert law, and research-verification discipline.
+description: "The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert."
 tags: [models, swap, gguf, huggingface, rollback, deployment, verification]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: PCIe Link and Atomics
-description: Link probing (idle downshift, path ceilings, CUDA/HIP bandwidth probes), the streaming-vs-compute ceiling arithmetic, and the PCIe-atomics wall that decides whether AMD multi-GPU collectives can exist at all.
+description: "Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives."
 tags: [pcie, rccl, atomics, multi-gpu, bandwidth, amd, nvidia]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -1,11 +1,11 @@
 ---
 type: reference
 title: Ethernet Bandwidth Math
-description: Bytes-per-token math for pipeline vs tensor parallelism, feasibility tables for 1/2.5/10 GbE, and real multi-node benchmark numbers.
+description: "Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it."
 tags:
  - pipeline-parallelism
  - research
-timestamp: 2026-08-17
+timestamp: 2026-10-19
 ---

 # Bandwidth Analysis: Multi-Node LLM Inference over Ethernet
@@ -147,19 +147,11 @@ bytes_per_boundary = batch_size × hidden_size × 2   [fp16 = 2 bytes]
 - Per-token, per-boundary: `6144 × 2 = 12,288 bytes = 12 KB`
 - With Nₚₚ stage boundaries: total = `Nₚₚ × 12 KB/tok`

-**Full example — 27B model, 2 PP stages (1 boundary), batch=1:**
+**Worked examples (1 boundary = 2 PP stages):**
 ```
-1 boundary × 4096 × 2 bytes = 8,192 bytes/tok
-```
-
-**Example — same model, batch=64:**
-```
-1 boundary × 4096 × 64 × 2 bytes = 524,288 bytes = 512 KB/tok
-```
-
-**Example — 36B model, 2 PP stages, batch=1:**
-```
-1 boundary × 6144 × 2 bytes = 12,288 bytes = 12 KB/tok
+27B, batch=1:  1 boundary × 4096 × 2      bytes = 8,192 bytes/tok  = 8 KB
+27B, batch=64: 1 boundary × 4096 × 64 × 2 bytes = 524,288          = 512 KB/tok
+36B, batch=1:  1 boundary × 6144 × 2      bytes = 12,288 bytes/tok = 12 KB
 ```

 ### 1.2 Tensor Parallelism (TP) — for comparison
@@ -244,57 +236,13 @@ total_bytes_per_tok = L × batch_size × hidden_size × 2   [L all-reduces total
 | 50 | 10.24 Mbit/s | 163.84 Mbit/s | 655.36 Mbit/s |
 | 100 | 20.48 Mbit/s | 327.68 Mbit/s | 1,310.72 Mbit/s |

-### 2.3 Feasibility matrix
+### 2.3 Feasibility

-Green = feasible (< 50% of link capacity). Yellow = marginal (50–90%). Red = saturated (> 90%).
-
-**27B model — PP (1 boundary), batch=1:**
-
-| | 1 GbE (118 MB/s) | 2.5 GbE (294 MB/s) | 10 GbE (1175 MB/s) |
-|---|---|---|---|
-| 10 tok/s | ✅ 0.13 Mbit/s | ✅ 0.13 Mbit/s | ✅ 0.13 Mbit/s |
-| 50 tok/s | ✅ 0.65 Mbit/s | ✅ 0.65 Mbit/s | ✅ 0.65 Mbit/s |
-| 100 tok/s | ✅ 1.31 Mbit/s | ✅ 1.31 Mbit/s | ✅ 1.31 Mbit/s |
-
-**27B model — PP (1 boundary), batch=16:**
-
-| | 1 GbE | 2.5 GbE | 10 GbE |
-|---|---|---|---|
-| 10 tok/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s |
-| 50 tok/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s |
-| 100 tok/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s |
-
-**27B model — PP (1 boundary), batch=64:**
-
-| | 1 GbE | 2.5 GbE | 10 GbE |
-|---|---|---|---|
-| 10 tok/s | ✅ 8.19 Mbit/s | ✅ 8.19 Mbit/s | ✅ 8.19 Mbit/s |
-| 50 tok/s | ✅ 40.96 Mbit/s | ✅ 40.96 Mbit/s | ✅ 40.96 Mbit/s |
-| 100 tok/s | ✅ 81.92 Mbit/s | ✅ 81.92 Mbit/s | ✅ 81.92 Mbit/s |
-
-**27B model — TP (32 layers), batch=1:**
-
-| | 1 GbE | 2.5 GbE | 10 GbE |
-|---|---|---|---|
-| 10 tok/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s | ✅ 2.05 Mbit/s |
-| 50 tok/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s | ✅ 10.24 Mbit/s |
-| 100 tok/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s | ✅ 20.48 Mbit/s |
-
-**27B model — TP (32 layers), batch=16:**
-
-| | 1 GbE | 2.5 GbE | 10 GbE |
-|---|---|---|---|
-| 10 tok/s | ✅ 32.77 Mbit/s | ✅ 32.77 Mbit/s | ✅ 32.77 Mbit/s |
-| 50 tok/s | ✅ 163.84 Mbit/s | ✅ 163.84 Mbit/s | ✅ 163.84 Mbit/s |
-| 100 tok/s | ✅ 327.68 Mbit/s | ✅ 327.68 Mbit/s | ✅ 327.68 Mbit/s |
-
-**27B model — TP (32 layers), batch=64:**
-
-| | 1 GbE | 2.5 GbE | 10 GbE |
-|---|---|---|---|
-| 10 tok/s | ✅ 131.07 Mbit/s | ✅ 131.07 Mbit/s | ✅ 131.07 Mbit/s |
-| 50 tok/s | ✅ 655.36 Mbit/s | ✅ 655.36 Mbit/s | ✅ 655.36 Mbit/s |
-| 100 tok/s | ✅ 1310.72 Mbit/s | ✅ 1310.72 Mbit/s | ✅ 1310.72 Mbit/s |
+Thresholds: **feasible** = under 50% of link capacity, **marginal** = 50–90%,
+**saturated** = over 90%. Cross-checking §2.2 against §2.1: every PP case is
+feasible on every link class — even the heaviest, 27B at batch=64 and 100
+tok/s (81.92 Mbit/s), is feasible on 1 GbE — and every TP case is feasible
+except batch=64:

 **27B TP batch=64 at 100 tok/s: 1310.72 Mbit/s > 1 GbE usable (1000 Mbit/s) — link-saturated on 1 GbE**

@@ -368,10 +316,7 @@ With TP, each layer transfers `H × 2` bytes but this happens L times. For 256 K
 **Petals [1]:**
 - Uses pipeline parallelism with dynamic 8-bit quantization (halves bandwidth)
 - For BLOOM-176B (H=14336), per-boundary communication is ~14 KB/tok with quantization
- Recommended bandwidth: 25 Mbit/s bidirectional for interactive inference [1]
 - Key finding: "For inference, performance does not depend much on bandwidth or sequence length but degrades with higher latency" — RTT matters more than throughput
- At 1 Gbit/s with <5 ms RTT: 1.71 steps/s; at 100 Mbit/s with <5 ms: 1.66 steps/s (only 3% difference)
- At 100 Mbit/s with 100 ms RTT: 1.23 steps/s (28% degradation)

 **llama.cpp RPC [2]:**
 - Model weights and KV cache distributed across nodes proportionally to memory
@@ -493,38 +438,17 @@ With Nₚₚ = 4 and 30 µs per-hop RTT: ~120 µs network, still <6% of compute

 ## Appendix: Full bandwidth formulae

-### Pipeline Parallelism (generation, batch=1)
+### Pipeline and Tensor Parallelism (generation)
+
+`N` is the batch size; `N = 1` is the batch=1 form.

 ```
-bytes_per_boundary_per_tok = hidden_size × 2   [fp16]
-total_bytes_per_tok = num_boundaries × hidden_size × 2
+bytes_per_boundary_per_tok = hidden_size × N × 2              [fp16, PP boundary]
+bytes_per_layer_allreduce  = hidden_size × N × 2              [TP, one all-reduce per layer]
+total_bytes_per_tok = num_boundaries × hidden_size × N × 2    [PP]
+total_bytes_per_tok = num_layers      × hidden_size × N × 2   [TP]
 bandwidth_mbps = total_bytes_per_tok × tokens_per_second × 8 / 1,000,000
-```
-
-### Pipeline Parallelism (generation, batch=N)

-```
-bytes_per_boundary_per_tok = hidden_size × N × 2
-total_bytes_per_tok = num_boundaries × hidden_size × N × 2
-```
-
-### Tensor Parallelism (generation, batch=1)
-
-```
-bytes_per_layer_allreduce = hidden_size × 2
-total_bytes_per_tok = num_layers × hidden_size × 2
-bandwidth_mbps = total_bytes_per_tok × tokens_per_second × 8 / 1,000,000
-```
-
-### Tensor Parallelism (generation, batch=N)
-
-```
-total_bytes_per_tok = num_layers × hidden_size × N × 2
-```
-
-### With 8-bit quantization (like Petals)
-
-```
-# Simply divide all fp16 byte counts by 2
-bytes_per_boundary_per_tok = hidden_size × 1   [int8]
+# With 8-bit quantization (like Petals): simply divide all fp16 byte counts by 2
+bytes_per_boundary_per_tok = hidden_size × 1                  [int8]
 ```
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Expected Performance
-description: Estimated tokens per second for a 25-40B model split across two nodes over 1/2.5/10 GbE, with real benchmark anchors.
+description: "Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on."
 tags:
  - pipeline-parallelism
  - performance
@@ -13,7 +13,7 @@ timestamp: 2026-08-14

 This page estimates the speed of a 25-40B model split across two machines
 over ethernet. It uses the bandwidth math in [[Ethernet Bandwidth Math]]
-and the measured numbers in [[Mixed-Vendor Experiments]].
+and the mixed-vendor measurements in [[Multi-Node Runtimes]].

 ## The Bandwidth Ceiling Is Not the Limit

@@ -93,5 +93,5 @@ roughly 200k-260k tokens across both nodes.
 ## See Also

 - [[Ethernet Bandwidth Math]]
- [[Mixed-Vendor Experiments]]
+- [[Multi-Node Runtimes]]
 - [[Sample Script Set]]
@@ -1,9 +0,0 @@
---
-type: reference
-title: "Mixed-Vendor Experiments"
-description: "Real-world NVIDIA + AMD Strix Halo multi-node experiments: what works (llama.cpp RPC, Vulkan split), what breaks (mixed CUDA+ROCm vLLM), measured tok/s."
-tags:
-  - pipeline-parallelism
-  - research
-timestamp: 2026-08-14
---
@@ -0,0 +1,257 @@
+---
+type: reference
+title: Multi-Node Runtimes
+description: "How each two-machine runtime is built and launched, what it can combine with, and the mixed-vendor verdicts."
+tags:
+  - pipeline-parallelism
+  - llamacpp
+  - vllm
+  - speculative
+timestamp: 2026-10-19
+---
+
+# Multi-Node Runtimes
+
+Two runtimes can put one model on two machines: llama.cpp over the ggml-rpc
+backend, and vLLM with pipeline parallelism. This page owns the cross-runtime
+facts — how each is built and launched, what each can and cannot be combined
+with, and the mixed-vendor verdicts. Facts owned by a sibling page are linked,
+not restated. Bytes-per-token arithmetic and link budgets belong to
+[[Ethernet Bandwidth Math]], throughput estimates to [[Expected Performance]],
+the real deployment and its verified gotchas to [[Qwen3.8 PP Deployment]], the
+educational scripts to [[Sample Script Set]], drafter mechanisms to
+[[DFlash DSpark Speculative Decoding]], single-host multi-GPU placement to
+[[Llama Cpp Multi-Device Splits]].
+
+## What each runtime is
+
+Both distribute by **layers**, not inside layers.
+
+| | llama.cpp over ggml-rpc | vLLM pipeline parallelism |
+| --- | --- | --- |
+| unit of distribution | a contiguous slice of layers (`--split-mode layer`, the default) | a contiguous slice of layers (`--pipeline-parallel-size`) |
+| process model | one `llama-server`/`llama-cli` on the head; a `ggml-rpc-server` per worker | one `vllm serve` per node, one engine across them |
+| what crosses the wire | whatever graph work the head offloads to the remote devices | the hidden state at each stage boundary, point to point |
+| KV cache | distributed with the weights; the KV of a layer lives with the owner of that layer | each stage holds the KV of the layers it owns |
+| speculation together with the split | yes in llama.cpp (see below) | no, per vLLM's own docs |
+
+Upstream's own summary is why layer-wise is the ethernet choice: pipeline
+parallel "minimizes data transfers between GPUs but requires many tokens to
+scale well", while tensor parallel "does multiple cross-GPU reductions per
+layer … much more bottlenecked by the GPU interconnect speed".
+
+Do not carry the tensor path across machines: `--split-mode tensor` is
+experimental, needs flash attention, rejects quantized KV, and is not
+implemented for the MoE and hybrid architectures the guide lists — and
+per-tensor placement across vendors is not expressible (§1 of [[Llama Cpp Multi-Device Splits]]).
+
+## llama.cpp over ggml-rpc
+
+### Build and launch
+
+Both sides take `-DGGML_RPC=ON` next to that host's own backend:
+
+```bash
+cmake .. -DGGML_CUDA=ON -DGGML_RPC=ON    # a CUDA worker; -DGGML_HIP=ON for an AMD one
+cmake --build . --config Release
+```
+
+Worker side — `ggml-rpc-server` exposes ggml devices on the remote host:
+
+```bash
+bin/ggml-rpc-server                      # endpoint 127.0.0.1:50052 by default
+bin/ggml-rpc-server --device CUDA0 -c    # or CUDA_VISIBLE_DEVICES=0; -c = local cache
+```
+
+- It exposes every accelerator it can see; a host with no accelerator exposes a
+  single `CPU` device. `--device` and `CUDA_VISIBLE_DEVICES` are equivalent
+  ways to narrow that set.
+- `-c` keeps large tensors in `$HOME/.cache/llama.cpp/rpc` (redirect with
+  `LLAMA_CACHE`) so a big model does not re-stream per load; that cache
+  filling the worker disk is a recorded fleet incident ([[Qwen3.8 PP Deployment]]).
+
+Head side — build the local backends plus `-DGGML_RPC=ON`, then name the
+workers:
+
+```bash
+llama-server -m model.gguf --rpc 192.168.88.10:50052,192.168.88.11:50052 -ngl 99
+```
+
+`--rpc` is a comma-separated list of `host:port`, one entry per
+`ggml-rpc-server`; the head's own devices stay in the pool next to the remote
+ones. Launch scripts for this shape: [[Sample Script Set]].
+
+### Split, context, KV
+
+- Default placement: weights **and** KV across all devices, local and remote,
+  "in proportion to each device's available memory". Override with
+  `--tensor-split`, whose proportions follow the order in `--device`. Why the
+  fleet always pins it (a Strix reports a huge GTT window, so the automatic
+  split lands on the slow device) is [[Qwen3.8 PP Deployment]]'s.
+- `-sm layer` (the default) is pipeline parallelism: each device holds a
+  contiguous slice of layers and the KV cache of a layer lives on the device
+  that owns that layer.
+- KV types are the usual `-ctk` / `-ctv`, `f16` by default. `--fit` is on by
+  default and sizes unset arguments to device memory; turn it off to hold a
+  context you set on purpose. Context is one global `--ctx-size` — the RPC
+  README documents no per-node context or per-node KV knob.
+- Upstream's out-of-memory ladder (stated for tensor mode, useful order
+  anyway): lower `--ctx-size`, then `--parallel`, then `--n-gpu-layers`.
+- Transport is TCP; RDMA is negotiated at the handshake with no flag changes
+  and falls back to TCP unless both peers can use it (Linux RoCEv2 via
+  `libibverbs`, macOS RDMA over Thunderbolt 5 via `librdma`). It is point to
+  point — name the peer's RDMA-capable address in `--rpc` or the session stays
+  on TCP. `GGML_RPC_NO_RDMA=1` forces TCP; `GGML_RPC_DEBUG` traces the backend.
+
+Upstream calls the backend a proof of concept, "fragile and insecure" (see the
+Security section). Two limits shape the design: decode is bound by the per-step
+round trip, not by the link's bytes ([[Ethernet Bandwidth Math]]), and every
+node must run the identical build or the handshake hangs ([[Qwen3.8 PP Deployment]]).
+
+## vLLM pipeline parallelism
+
+> **Provenance.** Flags and mechanics below were recorded 2026-08-14 and are
+> not re-checked upstream in this pass (see "Unverified in this pass"); the
+> incompatibility lines, the method catalog and the issue history are re-verified.
+
+- `--pipeline-parallel-size` divides the layers into contiguous stages;
+  `--tensor-parallel-size` still works inside a node, so the canonical two-node
+  shape is tp = GPUs per node, pp = nodes.
+- Two backends: Ray (`--distributed-executor-backend ray`, the worker joining
+  a `ray start --head`) or multiprocessing (`--distributed-executor-backend mp`
+  with `--nnodes`, `--node-rank`, `--master-addr`, `--headless` on the
+  follower). `mp` avoids the Ray dependency; Ray brings the cluster manager.
+- What the interconnect must carry: one activation tensor per stage boundary
+  per forward pass, point to point — not a per-layer reduction. That is the
+  whole cross-node traffic story; the byte arithmetic and link budgets are
+  [[Ethernet Bandwidth Math]]. Pin the NIC vLLM talks on
+  (`NCCL_SOCKET_IFNAME`) rather than letting it guess an interface.
+- The PP bubble: with N stages and one sequence in flight, one stage works
+  while N-1 idle, so pp=2 costs half the machine at batch one. Batching is the
+  cure — pipeline parallelism "requires many tokens to scale well" — and the
+  microbatch formula is [[Ethernet Bandwidth Math]] §5.3.
+- KV and context: each stage keeps the KV of its own layers, so the pool grows
+  with the stage count, and `--max-model-len` is not reduced by the split
+  (recorded 2026-08-14, unverified here).
+- What it cannot combine with, in vLLM's words: "Pipeline parallelism is not
+  composable with speculative decoding as of `vllm<=0.15.0`", and "Speculative
+  decoding with a draft models is not supported in `vllm<=0.10.0`". Composing
+  the two has been requested since 2024 and closed stale (#6911, #10615); PP
+  also carries a Qwen3.5 bug report closed stale without a fix (#36643).
+- Method catalog at v0.25.1: EAGLE, MTP, draft model, PARD, MLP, n-gram, suffix
+  decoding, hidden-state extraction, a custom proposer class, dynamic
+  speculative decoding; `dflash` is an accepted `method` value.
+
+## Speculative decoding across the split
+
+llama.cpp chains several implementations behind one comma-separated
+`--spec-type` list; the split does not change the chain, only where its passes
+run. Mechanisms, chaining laws and drafter KV sizing are
+[[DFlash DSpark Speculative Decoding]]'s; only the split interaction is here.
+
+| Method | llama.cpp type | Over ggml-rpc | With vLLM PP | Why |
+| --- | --- | --- | --- | --- |
+| MTP | `draft-mtp` | works; not documented upstream, but run this way on this fleet's split head with acceptance measured ([[Qwen3.8 PP Deployment]]) | no | Context-backed drafter: it needs a draft context and runs in the head process, so RPC devices are simply more devices under it |
+| DFlash | `draft-dflash` | same gate, same status as MTP | no | Same draft-context gate (`ctx_dft`); the drafter's KV lands on whichever device the draft context is built on — the one thing to check on a split |
+| ngram-mod | `ngram-mod` | safest of the four: no draft model, no draft context, no remote draft pass | no | Draftless — proposals come from cached token n-grams in the head, so the split adds no device work |
+| DSpark | `draft-dspark` | **unproven over RPC** — not a missing feature | no | `draft-dspark` exists upstream; nobody has measured it with the target split across two machines |
+
+**Correction (2026-10-19).** A retired stub in this directory asserted "DSpark
+is vLLM-only." False. `draft-dspark` is in upstream llama.cpp — parsed in
+`common/arg.cpp`, mapped to `COMMON_SPECULATIVE_TYPE_DRAFT_DSPARK` in
+`common/speculative.cpp`, where it carries a Markov head and an anchor-first
+block layout and aborts with `DSpark draft has no confidence head: please set
+--spec-draft-p-min 0` when the drafter has no confidence head. What is unproven
+is running it over RPC, and that is the honest limit of the claim.
+
+The vLLM column is uniform because that statement is blanket: it names pipeline
+parallelism, not a method, so n-gram and suffix are excluded as written. vLLM
+gives no mechanism with it; "the drafter must sit next to the target's layers,
+which a stage split separates" is a reading, not upstream's words.
+
+Upstream likewise documents neither for nor against `--rpc` plus
+`--spec-type`. What the implementation does say is structural: every
+speculative implementation is built in the head process against the head's
+device set, and draftless types need no draft context at all. Routing draft
+passes to a remote worker, however, has never been tested.
+
+## Mixed-vendor reality
+
+What works today:
+
+- **One vendor per node.** RPC is backend-agnostic by construction: upstream's
+  own diagram has a host exposing CUDA0+CUDA1, a host exposing Metal, and a
+  host exposing CUDA0+CPU, all driven by one main host over TCP. Each node
+  builds its own backend plus `-DGGML_RPC=ON`; nothing speaks CUDA-to-ROCm.
+- **Identical build tag on every node**, worker restarted before head. The
+  verified version-skew gotcha is [[Qwen3.8 PP Deployment]]'s.
+- **Speculation on the head** while the split carries the target (table above).
+- **RDMA when both peers have it**, auto-negotiated at the handshake; the size
+  of the gain is third-party and unverified here.
+
+Unproven or broken:
+
+- **One binary holding both CUDA and ROCm for a same-host layer split**:
+  reported to compile and then list only CUDA devices (issue #11506,
+  third-party, unverified). Vulkan is the reported single-binary cross-vendor
+  path, and single-host placement is [[Llama Cpp Multi-Device Splits]]'s, whose
+  verdict is that per-tensor placement across a vendor mix never works.
+- **vLLM across a CUDA node and a ROCm node**: no known experiment; both ends
+  run the same backend (recorded 2026-08-14). AMD's own two-Halo clustering
+  guide is tensor parallelism over RCCL, not a mixed-vendor split.
+- **DSpark over RPC**: unproven, see the correction above.
+- **The exact NVIDIA-desktop + Strix-Halo-NUC pair has no published
+  benchmark**, and the third-party mixed-vendor figures it is estimated from
+  (Metal+CUDA over 10 GbE, two Strix Halos over USB4, a three-card CUDA+ROCm set
+  over 1 GbE, a Vulkan triple in one box, an AMD four-node Strix cluster) live on
+  [[Expected Performance]] and [[Ethernet Bandwidth Math]], each named with its
+  model and hardware. None was re-verified in this pass.
+
+## Security
+
+`ggml-rpc-server` has **no authentication and no encryption**. Upstream's own
+warning: the backend is "in a proof-of-concept development stage. As such, the
+functionality is fragile and insecure. Never run the RPC server on an open
+network or in a sensitive environment!" It binds `127.0.0.1:50052` by default —
+safe by accident. Bind a routable address and any host that can reach the port
+can allocate memory there and supply the tensors it computes on. Trusted LAN
+only, firewalled or tunnelled; never a public address.
+
+## Unverified in this pass
+
+Every item was first recorded 2026-08-14, in the research that preceded this
+page. Nothing is silently deleted; none is backed by the sources cited below.
+
+- **RPC protocol version and handshake rule** — recorded as protocol `5.0.0`
+  with "major must match, server minor must be ≥ client". The README's own
+  banner prints `Starting RPC server v3.0.0`; the conflict is unresolved.
+- **RPC internals** — a hard cap of 16 servers, synchronous graph compute, no
+  tensor copy between two RPC servers, workers loading tensors on first use, a
+  hash-keyed cache above roughly 10 MB, asserts on quantized tensors whose rows
+  are not 512-aligned, and `-rpc` on `llama-bench` (the README names only
+  `llama-cli` and `llama-server`).
+- **vLLM flag surface** — `--pipeline-parallel-size`, `--tensor-parallel-size`,
+  `--distributed-executor-backend {ray,mp}`, `--nnodes`, `--node-rank`,
+  `--master-addr`, `--headless`, the `ray start --head` join,
+  `NCCL_SOCKET_IFNAME`.
+- **vLLM mechanics** — one activation handoff per stage boundary per forward
+  pass, a worked ~512 MB handoff for a hidden-5120 model at batch 8 and 2048
+  positions, `--max-model-len` unchanged by PP, ~50% idle at pp=2 batch one.
+- **vLLM throughput verdicts** for a 27B across two nodes over gigabit-class
+  links; superseded as an estimate by [[Expected Performance]].
+- **Mixed-vendor claims** — a CUDA+HIP build listing only CUDA devices
+  (#11506), Vulkan as the universal cross-vendor backend, AMD's two-Halo
+  clustering guide being RCCL tensor parallelism, WiFi costing about ten times
+  wired, the slowest hop setting the decode pace, RDMA worth a few percent.
+- **Draft over RPC** — that the draft model stays on the head, and that routing
+  draft passes to a remote worker has never been confirmed.
+
+## Citations
+
+- [llama.cpp RPC README](https://github.com/ggml-org/llama.cpp/blob/master/tools/rpc/README.md), accessed 2026-10-19 — server usage and device exposure, `-DGGML_RPC=ON` on both sides, the `--rpc host:port` list, proportional weight+KV split and `--tensor-split`, the `-c` local cache, RDMA negotiation and `GGML_RPC_NO_RDMA`, the proof-of-concept and no-authentication warning.
+- [llama.cpp multi-GPU guide](https://github.com/ggml-org/llama.cpp/blob/master/docs/multi-gpu.md), accessed 2026-10-19 — split modes (`-sm layer` = pipeline parallelism, KV with the layer owner), `-ts`/`-ngl`/`-ctk`/`-ctv`/`-fit`, the pipeline-versus-tensor summary, the tensor-mode limits.
+- [vLLM speculative decoding docs, v0.25.1](https://docs.vllm.ai/en/v0.25.1/features/speculative_decoding/), accessed 2026-10-19 — the pipeline-parallelism incompatibility statement, the draft-model version floor, the method catalog.
+- [llama.cpp `common/speculative.cpp`](https://github.com/ggml-org/llama.cpp/blob/master/common/speculative.cpp), accessed 2026-10-19 — the speculative implementation set including `draft-dspark`, its draft-context gate, its confidence-head error.
+- vLLM issue #6911, "[Feature]: Combine pipeline parallelism with speculative decoding", opened 2024-07-29, closed not-planned 2025-03-28, accessed 2026-10-19.
+- vLLM issue #10615, "[Usage]: Does speculative decoding support pipeline parallelism?", opened 2024-11-25, closed not-planned 2025-04-18, accessed 2026-10-19.
+- vLLM issue #36643, "[Bug]: Qwen3.5 does not work with pipeline parallelism", opened 2026-03-10, closed not-planned 2026-09-24, accessed 2026-10-19.
@@ -1,9 +0,0 @@
---
-type: reference
-title: "Qwen3.6 27B Spec Methods"
-description: "Qwen3.6-27B Q8 GGUF variants and llama.cpp speculative decoding (draft-mtp, draft-dflash, ngram-mod): flags, RPC compatibility, memory/context math."
-tags:
-  - pipeline-parallelism
-  - research
-timestamp: 2026-08-14
---
@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX"
+description: "The real split deployment: which node is head and which is worker, the flags, and the verified gotchas."
 tags:
  - llm
  - pipeline
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Sample Script Set
-description: An educational two-node llama.cpp RPC split of Qwen3.6-27B-Q8_K_P over ethernet. For reading only. Do not run or deploy.
+description: "An educational two-node RPC split script set, for reading only — never run it or deploy it."
 tags:
  - pipeline-parallelism
  - sample
@@ -255,7 +255,7 @@ is 30 GiB.

 ## See Also

- [[llama.cpp RPC]]
+- [[Multi-Node Runtimes]]
 - [[Expected Performance]]
 - [[Ethernet Bandwidth Math]]
- [[Qwen3.6 27B Spec Methods]]
+- [[DFlash DSpark Speculative Decoding]]
@@ -28,28 +28,24 @@ graph LR
    end
 ```

- [[llama.cpp RPC]] — the working, production-capable path for two
  heterogeneous nodes over ethernet.
- [[vLLM Pipeline Parallelism]] — supported but heavier; needs Ray or
  multiprocessing and does not compose with speculative decoding yet.

 ## What the Research Found

- [[Ethernet Bandwidth Math]] — a PP boundary moves only the hidden state
+- [[Ethernet Bandwidth Math]] — Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it.
  (about 10 KiB per token for a 27B model). 1 GbE supports hundreds of
  tokens per second on paper; the real limit is latency and RPC overhead.
- [[Mixed-Vendor Experiments]] — NVIDIA + Strix Halo multi-node inference
  works today via llama.cpp RPC with separate CUDA and ROCm builds. The
  exact NVIDIA-desktop + Strix-Halo-NUC pair has not been benchmarked with
  published numbers yet.
- [[Qwen3.6 27B Spec Methods]] — Q8 GGUF variants, and which speculative
  methods (MTP, DFlash, ngram-mod) work with RPC. DSpark is vLLM-only.
- [[Expected Performance]] — estimated tokens per second for a 27B Q8 split
+- [[Expected Performance]] — Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on.
  across an RTX 3090 and a Strix Halo NUC over 1/2.5/10 GbE.

 ## Deployment

- [[Qwen3.8 PP Deployment]] — the real Qwen3.8-27B split: head on Dankclaw
+- [[Qwen3.8 PP Deployment]] — The real split deployment: which node is head and which is worker, the flags, and the verified gotchas.
  (Strix), worker on Dankserver (RTX 3090), b10433 on both sides, scripts in
  `scripts/`, morning-of test runbook. Verified gotchas: no `--device` with
  `--rpc`, default split is proportional to reported free memory (pin 1,1),
@@ -57,7 +53,7 @@ graph LR

 ## Sample Scripts

- [[Sample Script Set]] — a two-node llama.cpp RPC split of
+- [[Sample Script Set]] — An educational two-node RPC split script set, for reading only — never run it or deploy it.
  Qwen3.6-27B-Q8_K_P with MTP, DFlash, and ngram-mod variants, sized for
  the largest context that fits. For educational reading only. Do not run
  it in production.
@@ -68,3 +64,9 @@ graph LR
 - [[frank-zealot]]
 - [[Dankserver System Specs]]
 - [[Dankclaw System Specs]]
+
+## Pages
+
+| Page | Owns |
+|---|---|
+| [[Multi-Node Runtimes]] | How each two-machine runtime is built and launched, what it can combine with, and the mixed-vendor verdicts. |
@@ -1,9 +0,0 @@
---
-type: reference
-title: "llama.cpp RPC"
-description: "llama.cpp distributed inference via ggml-rpc-server and the --rpc flag: architecture, CLI, versions, limitations, and RPC-compatible speculative decoding."
-tags:
-  - pipeline-parallelism
-  - research
-timestamp: 2026-08-14
---
@@ -1,9 +0,0 @@
---
-type: reference
-title: "vLLM Pipeline Parallelism"
-description: "vLLM multi-node pipeline parallelism: flags, Ray/mp backends, interconnect requirements, PP bubble, and PP + speculative decoding support."
-tags:
-  - pipeline-parallelism
-  - research
-timestamp: 2026-08-14
---
@@ -1,3 +1,13 @@
+---
+type: "howto"
+title: "Quantization Production"
+description: "Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file."
+tags:
+  - llm
+  - quantization
+timestamp: 2026-10-19
+---
+
 # Quantization Production — making our own quants, per card and runtime

 **Research page (2026-10-08).** Question: how do WE produce quantizations
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: RDNA4 Container Serving
-description: Bringing a model larger than VRAM onto Dankserver's 2x R9700 (gfx1201) with the tcclaviger vLLM stack — quant-class gating, fetch verification, container GPU-init triage, memory budgeting and the expert-offload plan.
+description: "Bringing a model larger than VRAM onto the two-card box: quant gating, fetch verification, container GPU-init triage."
 tags: [rdna4, gfx1201, r9700, vllm, expert-offload, podman, rocm]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: RDNA4 MoE Expert Offload VRAM Budget
-description: The expert-offload plan arithmetic on the tcclaviger fork — which terms a deploy can move, the util law, the CUDA-graph charge, what free VRAM really is, knife-edge survivability, parallelism ceilings and the context ladder.
+description: "Expert-offload arithmetic on the RDNA4 fork: movable terms, the utilisation law, graph charge, ceilings, context ladder."
 tags: [vllm, expert-offload, vram, arena, kv-cache, ladder, tcclaviger]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series."
+description: "Serving a dense FP8 vision-language checkpoint on the Strix Halo iGPU: the load path, the flags, the format gate."
 tags:
  - llm
 timestamp: 2026-09-03
@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "Qwen3.6-35B-A3B is a Mixture-of-Experts vision-language model from the Qwen3.6 series."
+description: "Serving a mixture-of-experts FP8 checkpoint on the R9700 host: the load path, the format gate, the offload shape."
 tags:
  - llm
 timestamp: 2026-09-03
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: Tcclaviger Fork Ops
-description: Operating the tcclaviger RDNA4 vLLM fork on dankserver (shape G era, rollback target since the 2026-10-05 FreeToken cutover) — the ad-hoc sweep harness, measured knob verdicts, sweep hygiene, the large-MoE serving recipe, and compile-cache persistence.
+description: "Operating the RDNA4 vLLM fork: the sweep harness, measured knob verdicts, sweep hygiene, compile-cache persistence."
 tags: [tcclaviger, vllm, dankserver, expert-offload, sweeps, cache]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "This index contains all documentation for vLLM deployment on Dankserver and Dankclaw."
+description: "Locator for every vLLM page in this vault."
 tags:
  - llm
  - vllm
@@ -1,7 +1,7 @@
 ---
-type: research
+type: reference
 title: Dankclaw IOMMU boot line — decision record (amd_iommu=off 2026-09-11 → iommu=pt 2026-10-18)
-description: Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a measured prefill cost of nothing (+2.0% on byte-identical traffic). Records the 2026-09-11 amd_iommu=off era that bought a vendor-claimed 13-16% figure never measured here, what that cost, the AMD PerfOpt research (Linux 7.4), the tooling that owns the boot line, and the revert path.
+description: "Why Dankclaw boots with identity DMA, what the earlier off-era bought and cost, and which tooling owns the boot line."
 tags:
  - dankclaw
  - iommu
@@ -1,7 +1,7 @@
 ---
-type: research
+type: reference
 title: Frontier MoE on Dankserver — what fits, on which runtime
-description: VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and GLM-5.3-Flash against the 2x R9700 (gfx1201) / 246 GiB box, and assesses the Affinity runtime's portability off DeepSeek-V4-Flash. Records the capacity arithmetic that decides the answer before any model is loaded.
+description: "The capacity arithmetic that decides whether a frontier mixture-of-experts fits this box before any model is loaded."
 tags:
  - dankserver
  - r9700
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: dealign Ornith-1.5-35B-A3B CRACK on Dankserver
-description: dealignai/Ornith-1.5-35B-A3B-UNCENSORED (CRACK Q6_K) served by upstream llama.cpp on the RTX 3090. MoE model with ngram-mod speculative decoding; selected expert weights offloaded to CPU with an explicit --override-tensor. frank-zealot.
+description: "Retired era: a mixture-of-experts checkpoint served with CPU expert offload and n-gram drafting on a single card."
 tags:
  - llamacpp
  - cuda
@@ -9,6 +9,7 @@ tags:
  - moe
  - ornith
  - mtp
+  - retired-endpoint
 timestamp: 2026-08-24
 ---

@@ -123,4 +124,4 @@ llama-server \

 - [[deployment]] — deploy and update process
 - [[Dankserver System Specs]] — the box
- [[frank-redo-2026-08-17]] — fleet topology history
+- [[frank-redo-2026-08-17]] — fleet topology histor
@@ -1,5 +1,5 @@
 ---
-type: guide
+type: howto
 title: LLM Deployment
 description: Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot).
 tags:
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: "frank-archon on Dankserver — radiance (StillDeadcode) 1.0.13, publisher Qwen3.8-Flash-Next iq4r .rad, 2× R9700 TP=2 (ACTIVE)"
-description: "The one frank-archon page: the live radiance shape and harness contract (read off scripts/frank-archon.env), the Eras table carrying every rollback path including the re-download doors for the purged payloads, the laws this endpoint taught (pinned-RAM/TTM preflight, the KV dtype law, the one per-card VRAM budget, descriptor selection), the wire contract, and the files that own it. It replaces the nine per-era runbooks."
+description: "The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught."
 tags:
  - llm
  - frank-archon
@@ -11,6 +11,7 @@ tags:
  - r9700
  - tp2
  - expert-offload
+  - endpoint
 timestamp: 2026-10-18
 ---

@@ -139,8 +140,10 @@ marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `Sti
 | 2026-09-26 → 09-27 | unsloth Q8_0, 175.30 GiB / 6 parts + shared-Q8_0 MTP 2,786,568,256 B + `mmproj-BF16.gguf` 907,542,944 B | unsloth llama.cpp b11030-mix prebuilt (commit `6ba30d05b`), ROCm gfx120X | `-ts 1,1` layer split, routed experts **and** the PLE table computed in host RAM (`-ot exps=CPU,per_layer_token_embd.weight=CPU`), 262,144 × 1 slot, then the same-day reshape to 4 × 262,144 with V `q8_0` and hot expert blocks 5/3 | the R9V radiance-vLLM fork took the port at Frank's direction (true TP=2 plus MTP4) | trio at commit `4992e5a`+; the GGUF/MTP/mmproj set is no longer on disk → re-fetch, then redeploy |
 | 2026-09-26 (same day) | `Lucebox/DeepSeek-V4.1-Flash-ROCMFP23-GGUF`, 377.2 GiB, sha256 `2daea6ca…` | `localhost/lucebox-rocm:gfx1201`, built from `Dockerfile.rocm` + 3 local patches | ONE R9700, `--max-ctx 131072`, `--chunk 8192`, q4_0 K+V, `--ds4-prefill sparse`, routed experts on CPU from the page cache | 377 GiB fits neither 63.7 GiB of VRAM nor 246 GiB of RAM → NVMe page-cache thrash; Qwen Flash-Next Q8 gave better quality, vision and 262k at comparable speed | image still in dank's rootless storage; payload gone; the quadlet and its entrypoint were deleted by `8500600` — `git show 8500600^:dankserver/frank-primary/quadlet/frank-primary.container` (and `…:dankserver/frank-primary/frank-primary-entrypoint.sh`), stop the root unit, start the user quadlet |

-Before 2026-09-26 the port belonged to the **`frank-medium`** service: [[frank-medium-radiance-mxfp4]]
-(vllm-radiance TP=2 MXFP4, 2026-09-20 → 09-26) is that era's page and is **not** superseded here.
+Before 2026-09-26 the port belonged to the **`frank-medium`** service, and its era
+pages are **not** superseded here: [[frank-medium]] is the locator for that
+endpoint's whole shape history, including the unresolved end date of the
+vllm-radiance era (this table opens on 09-26; the era page claims 10-10).

 ## Laws learned here

@@ -438,3 +441,4 @@ Related: [[frank-dragoon]] (the abliterated family), [[frank-zealot]],
 [[Dual-R9700-Cooperation]], [[deployment]], [[swift15-campaign-worklog]],
 [[dankserver-frontier-moe-feasibility]], [[x4-pair-experiment]], [[RDNA4 Container Serving]],
 [[frank-medium-radiance-mxfp4]], [[LLM/index]].
+---
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: "frank-dragoon on Dankclaw — halogen-flash-server 0.16.2, native v2 abliterated Qwen3.8-Flash-Next (canonical)"
-description: "The one runbook for frank-dragoon, the Strix Halo iGPU endpoint (192.168.0.143:7770): current shape from the env, every retired era with its rollback path, the durable laws this endpoint paid for, the wire contract, and the files that own it."
+description: "The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it."
 tags:
  - llm
  - frank-dragoon
@@ -9,6 +9,7 @@ tags:
  - strix-halo
  - gfx1151
  - halogen
+  - endpoint
 timestamp: 2026-10-18
 ---

@@ -234,3 +235,4 @@ created it — earlier topologies in [[frank-redo-2026-08-17]].
 - Related: [[deployment]] · [[Model Swap and Rollback]] · [[frank-zealot]] · [[frank-archon]] ·
  [[dankclaw-iommu-translation-off]] · [[frank-redo-2026-08-17]] · [[Dankclaw System Specs]] ·
  [[DFlash DSpark Speculative Decoding]] · [[qwen3.8-spec-matrix]] · [[qwen3.8-strix-optimizations]].
+---
@@ -2,7 +2,7 @@
 timestamp: 2026-09-17
 type: reference
 title: frank-medium — Batch and Ubatch Sweep
-description: llama-bench sweep of --batch-size against --ubatch-size on frank-medium (2x Radeon AI PRO R9700, layer split) at the deployed context shape, and the rebalance it forced. Prefill peaks at ubatch 512 and falls on BOTH sides.
+description: "The batch-versus-ubatch sweep on the layer-split pair at its deployed shape, and the rebalance the sweep forced."
 tags:
  - benchmark
  - llamacpp
@@ -11,6 +11,7 @@ tags:
  - dankserver
  - layer-split
 ---
+  - retired-endpoint

 # frank-medium — Batch and Ubatch Sweep

@@ -222,4 +223,5 @@ re-checking that 35:30 and 37:28 still fail so the gate keeps discriminating.
  difference: with no KV to attend over, per-pass overhead dominates and a
  bigger micro-batch wins; at 64k of resident KV it does not.
 - [[Dual-R9700-Cooperation]] — why the cards are layer-split at all
- [[Dankserver System Specs]] — hardware inventory
+- [[Dankserver System Specs]] — hardware inventory---
+timestamp: 2026-09-17
@@ -1,7 +1,7 @@
 ---
-type: configuration
+type: reference
 title: frank-medium llama.cpp Runbook (DFlash2 drafter)
-description: The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on Dankserver serving orcarouter/Qwen3.8-27B-Uncensored-GGUF at Q8_0 with the z-lab DFlash2 Q4_K_M drafter (the model's own MTP head stays in the file, one SPEC_TYPE change away), as ONE layer-split process across both Radeon AI PRO R9700 cards at a balanced 32:33 split, a 327680-position KV pool (163840 per conversation over two slots), f16 KV plus a 16 GiB host-RAM prompt cache, vision via the BF16 mmproj.
+description: "Retired era: the layer-split llama.cpp shape with a drafter, its ladder measurements, and the balance it settled on."
 tags:
  - frank-medium
  - dankserver
@@ -12,6 +12,7 @@ tags:
  - gfx1201
  - r9700
  - layer-split
+  - retired-endpoint
 timestamp: 2026-09-20
 ---

@@ -730,3 +731,4 @@ fleet.
 - [[frank-medium-batch-ubatch]] — the batch/ubatch sweep that set the deployed
  geometry and moved this split from 36:29 to 35:30: the ubatch optimum at 512,
  why the logical batch is free, and the harness traps
+---
@@ -1,7 +1,7 @@
 ---
-type: runbook
+type: howto
 title: frank-medium Model Swap
-description: RETIRED-era procedure (frank-medium is a retired endpoint; `:7770` on Dankserver is [[frank-archon]] today and the live swap law is [[Model Swap and Rollback]]) — how a checkpoint swap behind frank-medium worked: the streamlining toolchain (hf-manifest.py, hf-fetch.sh, swap-frank-medium.sh), the quantization-aware entrypoint contract, the quadlet edit, the deploy, the capability battery, and the rollback path.
+description: "Retired-era swap procedure: the fetch toolchain, the entrypoint contract, the capability battery, the rollback path."
 tags:
  - frank-medium
  - dankserver
@@ -9,6 +9,7 @@ tags:
  - vllm-radiance
  - huggingface
  - runbook
+  - retired-endpoint
 timestamp: 2026-09-20
 ---

@@ -155,9 +156,10 @@ back:
 2. `deploy.sh dankserver --service frank-medium --skip-e2e`.
 3. Re-run the capability battery.

-The 2026-09-20 rollback target is the MXFP4 checkpoint at
-`/home/dank/models/Qwen3.8-27B-Uncensored-MXFP4-awq` (19.37 GiB single
-safetensors) with `SPEC_METHOD=dflash`, `SPEC=7`.
+The 2026-09-20 rollback target is the MXFP4 checkpoint with
+`SPEC_METHOD=dflash`, `SPEC=7`; [[frank-medium-radiance-mxfp4]] owns that
+payload's identity, and as of the 2026-10-19 host census it is a re-fetch, not a
+local file.

 ## 2026-09-20 swap record (MXFP4 → FP8)

@@ -184,3 +186,4 @@ safetensors) with `SPEC_METHOD=dflash`, `SPEC=7`.
  DFlash2 drafter's 2.1 GiB costs the pool vs MTP's 461,540 (1.76×); both
  smaller than MXFP4's 943,581 (3.59×) because FP8 weights are bigger (30.9
  vs 19.4 GiB).
+---
@@ -1,7 +1,7 @@
 ---
-type: configuration
+type: reference
 title: frank-medium vllm-radiance MXFP6 Runbook
-description: RETIRED era runbook (endpoint renamed frank-archon 2026-10-10; this shape is a row in [[frank-archon]]'s Eras table) — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8, DFlash2 drafter, unquantized bf16 KV) as ONE tensor-parallel process (TP=2) on Dankserver's two AMD Radeon AI PRO R9700 (gfx1201). Cut to MXFP6 on 2026-09-21 (PR #51's launch6_p kernel, image 0.13.0-e060895); the MXFP4 checkpoint (just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq, the 2026-09-21 shape) is the on-disk rollback. Model-swap process: [[frank-medium-model-swap]]. The sections below describe the 2026-09-16 two-replica era.
+description: "Retired era: the tensor-parallel radiance shape on two cards, its quant-class swap, and the on-disk rollback payload."
 tags:
  - frank-medium
  - dankserver
@@ -14,6 +14,7 @@ tags:
  - rccl
  - pcie-atomics
  - quadlet
+  - retired-endpoint
 timestamp: 2026-09-21
 ---

@@ -37,11 +38,14 @@ e4m3fn scale) with `--kv-cache-dtype auto` (unquantized bf16 KV) and
 `--gpu-memory-utilization 0.98`. The Swift variant is a finetune of
 `ukisai/Swift-Qwen3.8-27b` (a different base from `Qwen/Qwen3.8-27B`), so the
 DFlash2-FP8 drafter's hidden-state distribution is a mismatch — acceptance may
-drop from the MXFP4 era's 4.24/7, but it is still a net win over no spec
-decode. **The MXFP4 checkpoint is the on-disk rollback**: flip `SNAP` back to
-`/models/Qwen3.8-27B-Uncensored-MXFP4-awq`, and the image
-back to `localhost/radiance-vllm:0.9.3` (re-pull `docker.io/stilldeadcode/
-vllm-radiance:0.9.3` if it is no longer on disk; the MXFP4 shape auto-sized to
+drop from the 4.24/7 that the DFlash2 arm measured on the FP8 target (the A/B
+below ran both arms on FP8, not on MXFP4). **The MXFP4 checkpoint is the
+rollback**, and as of the 2026-10-19 host census it is **no longer on disk**:
+`/home/dank/models/` holds `StillDeadcode`, `rad` and `tpl.jinja` only, so a
+rollback re-fetches `/models/Qwen3.8-27B-Uncensored-MXFP4-awq` (Hub
+`just1moremodel/Qwen3.8-27B-Uncensored-MXFP4-awq`) and flips `SNAP` and the
+image back to `localhost/radiance-vllm:0.9.3` (re-pull
+`docker.io/stilldeadcode/vllm-radiance:0.9.3` if the local tag is gone; the MXFP4 shape auto-sized to
 486,464 tokens with bf16 KV and held the full 262,144 window).

 **Deployed shape since 2026-09-20: one rootless Quadlet, `--tensor-parallel-size 2`.**
@@ -528,9 +532,8 @@ soak (three concurrent loops, each re-sending a ~30k-token document every turn
 with growing history) shows what happens at that limit: turns alternate between
 **~1.1 s** when the prefix is resident and **12-22 s** when it has been evicted
 and must be prefilled again, because two families per card plus their cached
-prefixes exceed the pool. Nothing failed — ~120 requests, no errors, no failover,
-both replicas steady — and per-family pinning kept the fast turns on the replica
-that holds the prefix.
+prefixes exceed the pool. Nothing failed, and per-family pinning kept the fast
+turns on the replica that holds the prefix.

 Read that as a capacity budget, not a defect: two concurrent large-context
 sessions per card are comfortable, three or more of that size trade turns
@@ -632,3 +635,4 @@ if that person is looking. That is the honest state of "alerting" here.
  the same image and kernel 7.0.0-30 failing identically.
 - `ggz14/radiance-vllm-mxfp4` — the patch set and the reference benchmark table
  this page compares against.
+---
@@ -1,7 +1,7 @@
 ---
-type: configuration
+type: reference
 title: frank-medium ROCmFPX Runbook
-description: charlie12345/ROCmFPX llama.cpp serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF as frank-medium on Dankserver (2x AMD Radeon AI PRO R9700, gfx1201).
+description: "Retired era: the ROCmFPX fork serving an agent-tuned checkpoint on the two-card box, and the loader pin it required."
 tags:
  - frank-medium
  - dankserver
@@ -10,6 +10,7 @@ tags:
  - mtp
  - gfx1201
  - r9700
+  - retired-endpoint
 timestamp: 2026-09-15
 ---

@@ -0,0 +1,65 @@
+---
+type: reference
+title: frank-medium (retired) — the Dankserver endpoint before frank-archon
+description: "Locator for the retired Dankserver LLM endpoint: which era page owns which payload, and where the dates disagree."
+timestamp: 2026-10-19
+tags:
+  - llm
+  - dankserver
+  - frank-medium
+  - retired
+  - retired-endpoint
+---
+
+# frank-medium (retired) — the Dankserver endpoint before frank-archon
+
+`frank-medium` was the LLM service on Dankserver port 7770. It is **not** a live
+endpoint: the same port is [[frank-archon]] today, and [[frank-archon]] owns every
+shape served from 2026-09-26 onward, including the `frank-medium` → `frank-archon`
+name cutover dated 2026-10-10 in `AGENTS.md`. This page owns only the **locator** —
+which era page holds which payload's evidence — and carries no measurements: every
+number stays in the era page that measured it.
+
+## Eras
+
+Windows are quoted from each era page's own banner; no date here is derived.
+
+| Window | Runtime | Payload | Shape | Page | Rollback |
+|---|---|---|---|---|---|
+| 2026-09-15 → 2026-09-16 | llama.cpp, ROCmFPX fork pinned at commit `36e9acd` | `Qwen3.8-27B-Q8_0_ROCMFPX_AGENT.gguf` (custom tensor type 103) | single card; the fork is the only loader | [[frank-medium-rocmfpx]] | re-fetch the GGUF, rebuild the fork |
+| 2026-09-16 → 2026-09-19 | vllm-radiance on the old TUF GAMING B650-E board | ParoQuant MXFP4 | two single-GPU replicas behind an adaptive-routing proxy | [[frank-medium-radiance-mxfp4]] (sections below its deployed-shape header) | the replica pair is reconstructable from that page's flags |
+| 2026-09-16 → 2026-09-20 | llama.cpp `fb27a525`, ROCm `7.14.0a` | its own GGUF build, DFlash drafter | `-sm layer` split across the two cards, drafter pinned to one device | [[frank-medium-llamacpp-dflash2]] | the page is the named rollback target for the llama shape |
+| 2026-09-20 → disputed, see Open questions | vllm-radiance on the replacement ProArt board | ParoQuant MXFP4, then MXFP6 (`paroquant_mxfp6`) with bf16 KV | `--tensor-parallel-size 2` over x8/x8 Gen5 root ports | [[frank-medium-radiance-mxfp4]] (deployed-shape header) | re-fetch the MXFP4 checkpoint (host census 2026-10-19: not local); image `localhost/radiance-vllm:0.9.3` |
+
+`[[frank-medium-batch-ubatch]]` is not an era: it is the batch-versus-ubatch sweep run
+on the layer-split pair at its deployed shape, and its numbers belong to that
+campaign. `[[frank-medium-model-swap]]` is likewise a campaign ledger — the record of
+the 2026-09-20 swap procedure — while the procedure itself is owned by
+[[Model Swap and Rollback]].
+
+## Laws
+
+None restated. Each era page carries its own laws and links them to the technique
+page that owns the mechanism ([[PCIe Link and Atomics]],
+[[Llama Cpp Multi-Device Splits]], [[DFlash DSpark Speculative Decoding]],
+[[Llama Cpp Serving Sizing]], [[Dual-R9700-Cooperation]]).
+
+## Open questions
+
+Two date claims in the era pages do not reconcile, and both are left standing rather
+than silently corrected:
+
+- `frank-medium-rocmfpx.md` banner says the MXFP4 stack replaced that era and, in the
+  next sentence, names [[frank-medium-llamacpp-dflash2]] as the active successor. The
+  MXFP4 replica era and the llama layer-split era both open on 2026-09-16, so which
+  one directly followed the ROCmFPX fork cannot be settled from the repo.
+- `frank-medium-radiance-mxfp4.md` dates its own retired era to 2026-10-10, while
+  [[frank-archon]]'s Eras table opens on 2026-09-26. The gap is either an unrecorded
+  fortnight on that endpoint or a stale banner; nothing else in the repo covers
+  2026-09-26 → 10-10 for port 7770.
+
+## Files
+
+No repo files own this endpoint: its units, env and serve scripts were renamed to
+`frank-archon` on 2026-10-10 and are owned by [[frank-archon]]. `scripts/old/` holds
+retired serve scripts from these eras as reference only.
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: frank-npu (retired) — FLM NPU endpoint on Dankclaw
-description: The retired FastFlowLM endpoint on Dankclaw's XDNA2 NPU — what it served, why it died on 2026-09-11 with the amd_iommu=off decision, and where every dependent route went.
+description: "The retired NPU endpoint: what it served, why it died with the IOMMU decision, and where every dependent route went."
 tags:
  - flm
  - fastflowlm
@@ -10,6 +10,7 @@ tags:
  - dankclaw
  - frank-npu
  - retired
+  - retired-endpoint
 timestamp: 2026-09-11
 ---

@@ -1,9 +1,10 @@
 ---
 type: reference
-description: "This page records the full redo of the inference fleet on 2026-08-17."
+description: "The day the whole inference fleet was rebuilt: what came down, what replaced it, and why it was done that way."
 tags:
  - llm
  - frank
+  - retired-endpoint
 timestamp: 2026-09-03
 ---

@@ -223,3 +224,4 @@ repeated prompt prefixes. Measured on a 2023-token prefix (2026-08-17
 evening): 4.6s cold -> 1.1s cached (~4x TTFT), output unchanged. This is
 the main TTFT lever for pi, which re-sends the same system prompt + tool
 schema every turn. Toggled via `ENABLE_PREFIX_CACHING` in the env file.
+---
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: "frank-zealot on Dankclaw — llama.cpp (PrismML fork 10777), Ternary-Bonsai-2-9B PQ2_0-MTP, 2 × 262144, exact f16 KV"
-description: "The one frank-zealot reference: the live shape on Dankclaw's RTX 3090 (:7771) exactly as scripts/frank-zealot.env ships it, every era with its rollback path, the deduplicated laws of this card, the wire contract, and the owning files."
+description: "The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws."
 tags:
  - llm
  - frank-zealot
@@ -9,6 +9,7 @@ tags:
  - llama-cpp
  - gguf
  - rtx3090
+  - endpoint
 timestamp: 2026-10-18
 ---

@@ -1,7 +1,7 @@
 ---
-type: research
+type: reference
 title: FreeToken Benchmark Sweep — Dankserver RTX 3090
-description: KV pool vs decode speed sweep across bs1/bs3/bs4/bs6 on Dankserver with Qwen3.6-35B-A3B-FP8 via FreeToken v0.1.2.
+description: "KV pool versus decode speed for the FP8 mixture-of-experts payload on the FreeToken runtime."
 tags:
  - freetoken
  - benchmark
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: ik_llama.cpp on Dankserver (Ornith Q8_0)
-description: ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-zealot on the RTX 3090. Q8_0 model, KV K default f16 / V q6_0 with Hadamard on V only (iq5_k is not a KV-cache type in this build; q8_KV crashes on this arch), split-mode none, speculative decoding disabled, 3 parallel slots x 163840 ctx, the Qwen-Sharp template rendered with --jinja, and explicit CPU-expert offload. Includes the mandatory CUDA/IQK FA_ALL_QUANTS build flags.
+description: "Retired era: the ik_llama fork with a quantized value cache and CPU expert offload, and the build flags it required."
 tags:
  - ikllama
  - llamacpp
@@ -11,7 +11,8 @@ tags:
  - ornith
  - kv-cache
  - hadamard
-timestamp: 2026-08-26
+  - retired-endpoint
+timestamp: 2026-10-19
 ---

 # ik_llama.cpp on Dankserver (Ornith-1.5-35B-A3B CRACK)
@@ -259,4 +260,4 @@ spec / `--spec-type` flags. The model file is not changed either way.
  build flags incl. `GGML_CUDA_FA_ALL_QUANTS` and `GGML_IQK_FA_ALL_QUANTS`
 - [ik_llama.cpp serverAppConfigs-docs/parameters.md](https://github.com/ikawrakow/ik_llama.cpp/blob/main/docs/parameters.md) —
  q8_KV (PR 208), K/V Hadamard options, `-cuda graphs=0` workaround, and
-  `--split-mode graph` MoE note (PR 1080)
+  `--split-mode graph` MoE note (PR 1080)
@@ -9,37 +9,33 @@ each of those owns exactly that one thing. The incidents behind the laws are in

 | Page | Owns |
 |---|---|
-| [[frank-archon]] | The one frank-archon page: the live radiance shape and harness contract (read off scripts/frank-archon.env), the Eras… |
-| [[frank-dragoon]] | The one runbook for frank-dragoon, the Strix Halo iGPU endpoint (192.168.0.143:7770): current shape from the env,… |
-| [[frank-zealot]] | The one frank-zealot reference: the live shape on Dankclaw's RTX 3090 (:7771) exactly as scripts/frank-zealot.env… |
+| [[frank-archon]] | The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught. |
+| [[frank-dragoon]] | The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it. |
+| [[frank-zealot]] | The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws. |

 ## Retired endpoints

 | Page | Owns |
 |---|---|
-| [[frank-medium-batch-ubatch]] | llama-bench sweep of --batch-size against --ubatch-size on frank-medium (2x Radeon AI PRO R9700, layer split) at the… |
-| [[frank-medium-llamacpp-dflash2]] | The former ACTIVE frank-medium deployment since 2026-09-16 (model swapped 2026-09-19) — mainline llama.cpp on… |
-| [[frank-medium-model-swap]] | RETIRED-era swap procedure for the frank-medium endpoint (`:7770` is now [[frank-archon]]) — the… |
-| [[frank-medium-radiance-mxfp4]] | RETIRED frank-medium era runbook — vllm-radiance serving hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (ParoQuant MXFP6 W4A8,… |
-| [[frank-medium-rocmfpx]] | charlie12345/ROCmFPX llama.cpp serving kingjones777/Qwen3.8-27B-ROCmFPX-Q8_0-AGENT-GGUF as frank-medium on Dankserver… |
-| [[frank-npu]] | The retired FastFlowLM endpoint on Dankclaw's XDNA2 NPU — what it served, why it died on 2026-09-11 with the… |
-| [[frank-redo-2026-08-17]] | This page records the full redo of the inference fleet on 2026-08-17. |
-| [[ikllama-ornith-dankserver]] | ikawrakow/ik_llama.cpp replaces upstream llama.cpp for frank-zealot on the RTX 3090. Q8_0 model, KV K default f16 / V… |
-| [[kingjones-q8agent-rocmfpx]] | **Status (historical, 2026-09-13):** was to deploy on Dankclaw. |
-| [[qwen3.8-spec-matrix]] | This page records the speculative-decoding benchmark history and the |
+| [[frank-medium-batch-ubatch]] | The batch-versus-ubatch sweep on the layer-split pair at its deployed shape, and the rebalance the sweep forced. |
+| [[frank-medium-llamacpp-dflash2]] | Retired era: the layer-split llama.cpp shape with a drafter, its ladder measurements, and the balance it settled on. |
+| [[frank-medium-model-swap]] | Retired-era swap procedure: the fetch toolchain, the entrypoint contract, the capability battery, the rollback path. |
+| [[frank-medium-radiance-mxfp4]] | Retired era: the tensor-parallel radiance shape on two cards, its quant-class swap, and the on-disk rollback payload. |
+| [[frank-medium-rocmfpx]] | Retired era: the ROCmFPX fork serving an agent-tuned checkpoint on the two-card box, and the loader pin it required. |
+| [[frank-npu]] | The retired NPU endpoint: what it served, why it died with the IOMMU decision, and where every dependent route went. |
+| [[frank-redo-2026-08-17]] | The day the whole inference fleet was rebuilt: what came down, what replaced it, and why it was done that way. |
+| [[ikllama-ornith-dankserver]] | Retired era: the ik_llama fork with a quantized value cache and CPU expert offload, and the build flags it required. |
+| [[kingjones-q8agent-rocmfpx]] | Historical: a checkpoint and loader pairing staged for this host and never deployed on it. |
+| [[qwen3.8-spec-matrix]] | The speculative-decoding benchmark history for this endpoint, arm by arm, and what each arm ruled out. |

 ## Pipeline Parallelism (series)

 | Page | Owns |
 |---|---|
-| [[Ethernet Bandwidth Math]] | Bytes-per-token math for pipeline vs tensor parallelism, feasibility tables for 1/2.5/10 GbE, and real multi-node… |
-| [[Expected Performance]] | Estimated tokens per second for a 25-40B model split across two nodes over 1/2.5/10 GbE, with real benchmark anchors. |
-| [[llama.cpp RPC]] | llama.cpp distributed inference via ggml-rpc-server and the --rpc flag: architecture, CLI, versions, limitations, and… |
-| [[Mixed-Vendor Experiments]] | Real-world NVIDIA + AMD Strix Halo multi-node experiments: what works (llama.cpp RPC, Vulkan split), what breaks… |
-| [[Qwen3.6 27B Spec Methods]] | Qwen3.6-27B Q8 GGUF variants and llama.cpp speculative decoding (draft-mtp, draft-dflash, ngram-mod): flags, RPC… |
-| [[Qwen3.8 PP Deployment]] | Split Qwen3.8-27B across the Strix Halo iGPU (head) and the RTX |
-| [[Sample Script Set]] | An educational two-node llama.cpp RPC split of Qwen3.6-27B-Q8_K_P over ethernet. For reading only. Do not run or… |
-| [[vLLM Pipeline Parallelism]] | vLLM multi-node pipeline parallelism: flags, Ray/mp backends, interconnect requirements, PP bubble, and PP +… |
+| [[Ethernet Bandwidth Math]] | Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it. |
+| [[Expected Performance]] | Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on. |
+| [[Qwen3.8 PP Deployment]] | The real split deployment: which node is head and which is worker, the flags, and the verified gotchas. |
+| [[Sample Script Set]] | An educational two-node RPC split script set, for reading only — never run it or deploy it. |

 ## Techniques, hardware, trials

@@ -47,43 +43,48 @@ each of those owns exactly that one thing. The incidents behind the laws are in
 |---|---|
 | [[Dankclaw Improvements]] | This page collects concrete, sourced recommendations to improve Dankclaw |
 | [[Dankclaw System Specs]] | Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference. |
-| [[dankclaw-iommu-translation-off]] | Dankclaw boots iommu=pt since 2026-10-18 — identity DMA with the IOMMU armed, which restored the XDNA2 NPU at a… |
-| [[Dankserver System Specs]] | Dankserver hardware and OS state since the 2026-09-20 board swap (ASUS ProArt B850-Creator WiFi NEO, 2x AMD Radeon AI… |
-| [[dankserver-frontier-moe-feasibility]] | VERIFIED 2026-09-28 by measurement and source reading. Compares DeepSeek-V4.1-Flash, MiMo-V2.6-Flash-MOPD and… |
+| [[dankclaw-iommu-translation-off]] | Why Dankclaw boots with identity DMA, what the earlier off-era bought and cost, and which tooling owns the boot line. |
+| [[Dankserver System Specs]] | Dankserver hardware and OS state since the board swap: board, lane topology, cards, memory, kernel posture. |
+| [[dankserver-frontier-moe-feasibility]] | The capacity arithmetic that decides whether a frontier mixture-of-experts fits this box before any model is loaded. |
 | [[dankserver-vram-growth]] | Investigation of the recurring cuBLAS crash on frank-zealot. What grows, what does not, and what to do next. |
-| [[dealign-ornith-crack]] | dealignai/Ornith-1.5-35B-A3B-UNCENSORED (CRACK Q6_K) served by upstream llama.cpp on the RTX 3090. MoE model with… |
+| [[dealign-ornith-crack]] | Retired era: a mixture-of-experts checkpoint served with CPU expert offload and n-gram drafting on a single card. |
 | [[deployment]] | Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot). |
 | [[DFlash DSpark Speculative Decoding]] | DFlash and DSpark are the two newest speculative decoding methods. |
-| [[Dual-R9700-Cooperation]] | Why frank-medium's two Radeon AI PRO R9700s could not run tensor-parallel on the old B650-E board (PCIe atomic… |
-| [[FastFlowLM NPU Inference]] | FastFlowLM (FLM), the AMD NPU-first LLM runtime for Ryzen AI XDNA2 chips. Fit analysis for the 50 TOPS NPU on… |
-| [[FreeToken Builds]] | Building FreeToken (the official franksdank fork of FreeTok Tokenworks' freetoken vLLM-MTP runtime) on Dankclaw — the… |
-| [[freetoken-benchmark-sweep]] | KV pool vs decode speed sweep across bs1/bs3/bs4/bs6 on Dankserver with Qwen3.6-35B-A3B-FP8 via FreeToken v0.1.2. |
-| [[GGUF Compatibility]] | How to answer "will engine X load this GGUF?" from the file's header alone, why a custom tensor type forces the… |
-| [[Heterogeneous Split Triage]] | Triage before building when someone asks to split a model across unequal accelerators on one host (24 GB GDDR6X +… |
+| [[Dual-R9700-Cooperation]] | Why two R9700s could not run tensor-parallel on the old board, and what the board swap unblocked. |
+| [[FastFlowLM NPU Inference]] | The AMD NPU-first runtime: fit analysis for the XDNA2, published benchmarks, install state, and test runbook. |
+| [[FreeToken Builds]] | Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit. |
+| [[freetoken-benchmark-sweep]] | KV pool versus decode speed for the FP8 mixture-of-experts payload on the FreeToken runtime. |
+| [[GGUF Compatibility]] | Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can. |
+| [[Heterogeneous Split Triage]] | Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys. |
 | [[Hybrid CPU GPU Processing Dankserver]] | Dankserver uses NVIDIA GPU with system RAM for hybrid processing. |
-| [[ling-3.0-flash-trial]] | `bailingmoe3-support` (commit `db4480bc8`, build b10411) at `/home/dank/llama.cpp-ling`. |
-| [[ling-3.0-flash-vllm-plan]] | This page was the plan for a vLLM-ROCm deployment of `jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native` (INT4 W4A16, ~62… |
-| [[Llama Cpp Multi-Device Splits]] | What llama.cpp can and cannot express across devices, the no-collective layer-split path, split balancing by… |
-| [[Llama Cpp Serving Sizing]] | Slot-pool sizing against a headroom band, per-device VRAM accounting, prompt-cache/state sizing and the QSA… |
-| [[LLM Bench Windows and Measurement]] | Stopping-and-restoring benchmark windows on the fleet LLM hosts (llama-bench sweeps, HTTP param benches, multi-arm… |
-| [[LLM Performance Diagnosis]] | Probing why a served LLM is slow — the flat-depth probe, the expert-offload duty-cycle arithmetic, kernel-time… |
-| [[Model Swap and Rollback]] | The full model-swap discipline for the LLM fleet — fit preflight, fetch and payload verification, runtime build/flag… |
-| [[PCIe Link and Atomics]] | Link probing (idle downshift, path ceilings, CUDA/HIP bandwidth probes), the streaming-vs-compute ceiling arithmetic,… |
+| [[ling-3.0-flash-trial]] | Trial of a fork build for a new mixture-of-experts architecture: what loaded, what the harness refused, and the verdict. |
+| [[Llama Cpp Multi-Device Splits]] | What llama.cpp can express across devices: the no-collective layer-split path, balancing, and drafter KV sizing. |
+| [[Llama Cpp Serving Sizing]] | Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache. |
+| [[LLM Bench Windows and Measurement]] | Opening and restoring benchmark windows on fleet hosts, honest methodology, and the usage-counter law under drafting. |
+| [[LLM Performance Diagnosis]] | Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution. |
+| [[Model Swap and Rollback]] | The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert. |
+| [[PCIe Link and Atomics]] | Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives. |
 | [[Quantization by Hardware Architecture]] | This page maps LLM quantization formats to the GPU and CPU architectures that support them. |
-| [[Quantization Production]] | — |
-| [[qwen3.8-strix-optimizations]] | This page answers one question: **how do we make Qwen3.8-27B serve faster on |
+| [[Quantization Production]] | Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file. |
+| [[qwen3.8-strix-optimizations]] | How a served checkpoint got faster on the Strix Halo iGPU: the lever ladder and what each lever actually bought. |
 | [[RAM CPU Offloading in VLLM]] | vLLM uses CPU RAM and swap space to extend GPU memory capacity. |
-| [[RDNA4 Container Serving]] | Bringing a model larger than VRAM onto Dankserver's 2x R9700 (gfx1201) with the tcclaviger vLLM stack — quant-class… |
-| [[RDNA4 MoE Expert Offload VRAM Budget]] | The expert-offload plan arithmetic on the tcclaviger fork — which terms a deploy can move, the util law, the… |
-| [[Running Qwen3.6 27B FP8 on Dankclaw]] | Qwen3.6-27B is a dense vision-language model from the Qwen3.6 series. |
-| [[Running Qwen3.6 35B A3B FP8 on Dankserver]] | Qwen3.6-35B-A3B is a Mixture-of-Experts vision-language model from the Qwen3.6 series. |
+| [[RDNA4 Container Serving]] | Bringing a model larger than VRAM onto the two-card box: quant gating, fetch verification, container GPU-init triage. |
+| [[RDNA4 MoE Expert Offload VRAM Budget]] | Expert-offload arithmetic on the RDNA4 fork: movable terms, the utilisation law, graph charge, ceilings, context ladder. |
+| [[Running Qwen3.6 27B FP8 on Dankclaw]] | Serving a dense FP8 vision-language checkpoint on the Strix Halo iGPU: the load path, the flags, the format gate. |
+| [[Running Qwen3.6 35B A3B FP8 on Dankserver]] | Serving a mixture-of-experts FP8 checkpoint on the R9700 host: the load path, the format gate, the offload shape. |
 | [[Shared Memory Architecture Dankclaw]] | Dankclaw uses AMD Strix Halo APU with unified memory architecture. |
-| [[swift15-campaign-worklog]] | — |
-| [[Tcclaviger Fork Ops]] | Operating the tcclaviger RDNA4 vLLM fork on dankserver (shape G era, rollback target since the 2026-10-05 FreeToken… |
-| [[ubatch-benchmark-large]] | Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment. |
-| [[vLLM Boot Forensics and Config Interrogation]] | Capturing a vLLM container boot failure's real root cause from journald (the container log is already gone),… |
-| [[VLLM Index]] | This index contains all documentation for vLLM deployment on Dankserver and Dankclaw. |
+| [[swift15-campaign-worklog]] | Dated worklog of the campaign that converted and served a Swift checkpoint on the radiance endpoint, arm by arm. |
+| [[Tcclaviger Fork Ops]] | Operating the RDNA4 vLLM fork: the sweep harness, measured knob verdicts, sweep hygiene, compile-cache persistence. |
+| [[ubatch-benchmark-large]] | The wide ubatch sweep for a deployed ROCmFPX shape, run through the standardized harness. |
+| [[vLLM Boot Forensics and Config Interrogation]] | Recovering a container boot failure's real cause from journald, and interrogating a live endpoint's true config. |
+| [[VLLM Index]] | Locator for every vLLM page in this vault. |
 | [[VLLM Overview]] | vLLM is a fast and easy library for large language model inference and serving. |
-| [[vLLM Porting and Compile Triage]] | Forward-porting a model architecture to a new vLLM major (additive-only discipline, gap enumeration, shims that… |
-| [[vllm_research_findings]] | vLLM v0.26.0 (latest stable, released July 25, 2026) supports DFlash speculative decoding with the `--spec-method… |
-| [[x4-pair-experiment]] | What mainline llama.cpp can and cannot do with a 3090 + Strix Halo combination, measured 2026-09-16 on Dankclaw — the… |
+| [[vLLM Porting and Compile Triage]] | Forward-porting a model architecture: additive-only discipline, shims that survive worker spawn, compile triage. |
+| [[vllm_research_findings]] | Upstream support survey for speculative decoding: which flags, releases and pull requests carry each method. |
+| [[x4-pair-experiment]] | Measured limits of a consumer card plus APU pair: the tensor inventory, the flag that kills cross-device placement. |
+
+## Pages
+
+| Page | Owns |
+|---|---|
+| [[frank-medium]] | Locator for the retired Dankserver LLM endpoint: which era page owns which payload, and where the dates disagree. |
@@ -1,10 +1,11 @@
 ---
 type: reference
-description: "**Status (historical, 2026-09-13):** was to deploy on Dankclaw."
+description: "Historical: a checkpoint and loader pairing staged for this host and never deployed on it."
 tags:
  - llm
  - rocm
  - frank
+  - retired-endpoint
 timestamp: 2026-09-15
 ---

@@ -239,3 +240,4 @@ reads `--host/--port/--temp` as CLI flags, never as env vars.
  `gl=45`. See the `set_rows` build bug section above.
 - Preceding model: Kairic Edge Qwen3.8-27B IU4 (deployed 2026-08-25 ~05:42,
  retired the same day for the AGENT model). See [[frank-dragoon]].
+---
@@ -1,9 +1,9 @@
 ---
 type: reference
-description: "`bailingmoe3-support` (commit `db4480bc8`, build b10411) at `/home/dank/llama.cpp-ling`."
+description: "Trial of a fork build for a new mixture-of-experts architecture: what loaded, what the harness refused, and the verdict."
 tags:
  - llm
-timestamp: 2026-09-03
+timestamp: 2026-10-19
 ---

 # Ling-3.0-flash Q6_K Trial on Dankclaw
@@ -109,6 +109,13 @@ re-enable `frank-dragoon-ling` from the repo unit
 - The heap is effectively full (98.7 of 111.2 GiB). Do not raise the
  context much without checking `--list-devices` free MiB.

+## The other path: vLLM-ROCm (never shipped)
+
+A deployment plan existed for `jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native`
+(INT4 W4A16, ~62 GB) served by vLLM-ROCm on Dankclaw. It never shipped; the
+llama.cpp Q6_K fork trial above is what actually ran. The decision table the
+plan carried is in git history.
+
 ## See Also

 - [[frank-dragoon]]
@@ -1,22 +0,0 @@
---
-type: guide
-description: "This page was the plan for a vLLM-ROCm deployment of `jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native` (INT4 W4A16, ~62 GB) on"
-tags:
-  - llm
-  - vllm
-  - rocm
-timestamp: 2026-09-03
---
-
-# Ling-3.0-flash on Dankclaw — vLLM-ROCm Deployment Plan (never shipped)
-
-This page was the plan for a vLLM-ROCm deployment of
-`jcbtc/Ling-3.0-Flash-CIRU-int4-Strix-native` (INT4 W4A16, ~62 GB) on
-Dankclaw. The deployment never shipped. The actual Ling work used the
-llama.cpp Q6_K fork trial instead: [[ling-3.0-flash-trial]].
-
-The trial service (`frank-dragoon-ling`) was retired and disabled under the boot
-policy on 2026-08-17, in the same redo that renamed `frank-dragoon` to the
-Qwen3.8-27B PP head ([[frank-redo-2026-08-17]]). Two fleet generations later the
-name belongs to the halogen flash-server endpoint ([[frank-dragoon]]). See git
-history for the full decision table.
@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "This page records the speculative-decoding benchmark history and the"
+description: "The speculative-decoding benchmark history for this endpoint, arm by arm, and what each arm ruled out."
 tags:
  - llm
  - speculative
@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "This page answers one question: **how do we make Qwen3.8-27B serve faster on"
+description: "How a served checkpoint got faster on the Strix Halo iGPU: the lever ladder and what each lever actually bought."
 tags:
  - llm
 timestamp: 2026-09-03
@@ -1,3 +1,13 @@
+---
+type: "reference"
+title: "Swift1.5 Campaign Worklog"
+description: "Dated worklog of the campaign that converted and served a Swift checkpoint on the radiance endpoint, arm by arm."
+tags:
+  - llm
+  - campaign
+timestamp: 2026-10-19
+---
+
 # Swift1.5 w4dn8 Campaign — Work Log

 Campaign branch `swift15-radiance-w4dn8`. Goal: convert
@@ -1,7 +1,7 @@
 ---
-type: benchmark plan
+type: reference
 title: frank-dragoon ubatch sweep
-description: Standardized ubatch sweep for the active kingjones Qwen3.8 ROCmFPX deployment.
+description: "The wide ubatch sweep for a deployed ROCmFPX shape, run through the standardized harness."
 tags:
  - llama.cpp
  - rocm
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: vLLM Boot Forensics and Config Interrogation
-description: Capturing a vLLM container boot failure's real root cause from journald (the container log is already gone), classifying the failure, and interrogating a live endpoint's true config and client contracts.
+description: "Recovering a container boot failure's real cause from journald, and interrogating a live endpoint's true config."
 tags: [vllm, forensics, journalctl, podman, config, contracts]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -1,7 +1,7 @@
 ---
 type: reference
 title: vLLM Porting and Compile Triage
-description: Forward-porting a model architecture to a new vLLM major (additive-only discipline, gap enumeration, shims that survive worker spawn) and the torch.compile/dynamo break triage, including the EAGLE3/DFlash2 aux-hidden-state protocol port rules.
+description: "Forward-porting a model architecture: additive-only discipline, shims that survive worker spawn, compile triage."
 tags: [vllm, porting, torch-compile, dynamo, eagle3, dflash, glm]
 timestamp: 2026-10-05T00:00:00-07:00
 ---
@@ -1,6 +1,6 @@
 ---
 type: reference
-description: "vLLM v0.26.0 (latest stable, released July 25, 2026) supports DFlash speculative decoding with the `--spec-method dflash` and `--spec-model` flags."
+description: "Upstream support survey for speculative decoding: which flags, releases and pull requests carry each method."
 tags:
  - llm
  - vllm
@@ -10,7 +10,17 @@ timestamp: 2026-08-28
 # Research: vLLM Parameters for Specified Configuration

 ## Summary
-vLLM v0.26.0 (latest stable, released July 25, 2026) supports DFlash speculative decoding with the `--spec-method dflash` and `--spec-model` flags. KV cache dtype `q8_0` is **not supported** — only fp8 variants and turboquant options are available. vLLM does not have a dedicated MoE expert CPU offload flag; general CPU offloading via `--cpu-offload-gb` is the closest option. The `--max-num-seqs` flag is confirmed. Podman GPU access requires `--gpus all` with the NVIDIA container toolkit.
+vLLM v0.26.0 (released 2026-07-25, the current stable when this survey was
+gathered on 2026-08-28; PyPI shows v0.32.0 as the latest on 2026-10-19, so
+re-check this page before acting on it) supports DFlash speculative decoding.
+The preferred CLI is the `--speculative-config` JSON object; `--spec-method`
+and `--spec-model` are the per-field aliases defined in
+`vllm/engine/arg_utils.py`. KV cache dtype `q8_0` is **not supported** — only
+fp8 variants and turboquant options are available. vLLM does not have a
+dedicated MoE expert CPU offload flag; general CPU offloading via
+`--cpu-offload-gb` is the closest option. The `--max-num-seqs` flag is
+confirmed. Podman GPU access requires `--gpus all` with the NVIDIA container
+toolkit.

 ## Findings

@@ -1,7 +1,7 @@
 ---
 type: reference
 title: The x4 pair — Flash-Next across the 3090 and the Strix Halo iGPU
-description: What mainline llama.cpp can and cannot do with a 3090 + Strix Halo combination, measured 2026-09-16 on Dankclaw — the tensor inventory that decides any split, the single flag that kills per-tensor placement across devices (-ot), the two-device layer splits that DO work (pure-Vulkan and cross-vendor CUDA+Vulkan, +7 % decode at 64k depth), and the harness-parsing trap that made the first round of measurements meaningless.
+description: "Measured limits of a consumer card plus APU pair: the tensor inventory, the flag that kills cross-device placement."
 tags:
  - llm
  - experiment