Files

T

Frank a5fe045b58 docs(vault): one fact, one owner — repair the doubled pages, single-source the facts

Structural repair:
- 13 pages were the same document twice (each exactly 2x its HEAD size),
  a second frontmatter block whose closing fence had been glued onto a body
  line by a frontmatter rewrite that re-read its own output. Un-doubled by
  cutting at the second frontmatter and keeping the copy whose body is a
  superset; the orphan census reports 0 lost tokens against HEAD.
- log.md: one entry per date, bare `## YYYY-MM-DD` headings, newest first —
  66 dated headings merged into 34, titles demoted to bold leads, every
  content line verified present. The file had been descending at the head
  and ascending at the tail.

Ownership moves:
- New `LLM/frank-medium.md` is the locator for the retired Dankserver
  endpoint: era windows quoted from each era page's own banner, no numbers
  restated, and the two unresolved date conflicts recorded instead of
  silently resolved. `frank-archon.md` links to it instead of restating the
  pre-rename window.
- The four Pipeline-Parallelism stubs (frontmatter only, no body) are gone;
  `Multi-Node Runtimes.md` owns runtime construction, launch, mixed-vendor
  verdicts and cross-split spec decode, and the four links that pointed at
  the stubs were repointed at it or at the drafter page.
- Authentik's three stub pages fold into `Authentik.md`, with the process
  detail and the outpost reference split out so the parent stays under the
  length hint.

Verified corrections (host census + live probe, 2026-10-19):
- The MXFP4 checkpoint two pages called the "on-disk rollback" is gone:
  /home/dank/models/ holds StillDeadcode, rad and tpl.jinja. Stated once, on
  the era page that owns that payload; the swap page now links to it.
- frank-medium-radiance-mxfp4: the 4.24/7 accept rate belongs to the DFlash2
  arm measured on the FP8 target, not to the MXFP4 era; the 3-family soak run
  census is kept once (155 requests) instead of twice (~120 and 155).
- Fleet shapes re-confirmed against the live endpoints before touching any
  page: 3x200000 fp8 KV, 2x200704, 2x262144.

Index rows are regenerated, not hand-carved. Net -1,849 vault lines.

2026-10-19 05:00:43 -07:00

8.6 KiB

Raw Permalink Blame History

LLM fleet — index

The three live endpoint pages are the only pages that carry a shape. Everything else here is a technique, a piece of hardware, or a trial that taught a law — and each of those owns exactly that one thing. The incidents behind the laws are in log.

Live endpoints

Page	Owns
frank-archon	The radiance endpoint: live shape and harness contract, every era with its rollback path, and the laws it taught.
frank-dragoon	The iGPU endpoint: live shape, every era with its rollback path, the laws it paid for, and the files that own it.
frank-zealot	The RTX 3090 endpoint: live shape exactly as the env ships it, every era with its rollback path, and its card laws.

Retired endpoints

Page	Owns
frank-medium-batch-ubatch	The batch-versus-ubatch sweep on the layer-split pair at its deployed shape, and the rebalance the sweep forced.
frank-medium-llamacpp-dflash2	Retired era: the layer-split llama.cpp shape with a drafter, its ladder measurements, and the balance it settled on.
frank-medium-model-swap	Retired-era swap procedure: the fetch toolchain, the entrypoint contract, the capability battery, the rollback path.
frank-medium-radiance-mxfp4	Retired era: the tensor-parallel radiance shape on two cards, its quant-class swap, and the on-disk rollback payload.
frank-medium-rocmfpx	Retired era: the ROCmFPX fork serving an agent-tuned checkpoint on the two-card box, and the loader pin it required.
frank-npu	The retired NPU endpoint: what it served, why it died with the IOMMU decision, and where every dependent route went.
frank-redo-2026-08-17	The day the whole inference fleet was rebuilt: what came down, what replaced it, and why it was done that way.
ikllama-ornith-dankserver	Retired era: the ik_llama fork with a quantized value cache and CPU expert offload, and the build flags it required.
kingjones-q8agent-rocmfpx	Historical: a checkpoint and loader pairing staged for this host and never deployed on it.
qwen3.8-spec-matrix	The speculative-decoding benchmark history for this endpoint, arm by arm, and what each arm ruled out.

Pipeline Parallelism (series)

Page	Owns
Ethernet Bandwidth Math	Bytes-per-token math for pipeline versus tensor parallelism, and the link budgets that follow from it.
Expected Performance	Estimated split-model throughput over ethernet, with the benchmark anchors the estimates rest on.
Qwen3.8 PP Deployment	The real split deployment: which node is head and which is worker, the flags, and the verified gotchas.
Sample Script Set	An educational two-node RPC split script set, for reading only — never run it or deploy it.

Techniques, hardware, trials

Page	Owns
Dankclaw Improvements	This page collects concrete, sourced recommendations to improve Dankclaw
Dankclaw System Specs	Dankclaw is an AMD APU server for CIRU ROCmFPX llama.cpp inference.
dankclaw-iommu-translation-off	Why Dankclaw boots with identity DMA, what the earlier off-era bought and cost, and which tooling owns the boot line.
Dankserver System Specs	Dankserver hardware and OS state since the board swap: board, lane topology, cards, memory, kernel posture.
dankserver-frontier-moe-feasibility	The capacity arithmetic that decides whether a frontier mixture-of-experts fits this box before any model is loaded.
dankserver-vram-growth	Investigation of the recurring cuBLAS crash on frank-zealot. What grows, what does not, and what to do next.
dealign-ornith-crack	Retired era: a mixture-of-experts checkpoint served with CPU expert offload and n-gram drafting on a single card.
deployment	Scoped deployment and update process for the three inference services (frank-archon, frank-dragoon, frank-zealot).
DFlash DSpark Speculative Decoding	DFlash and DSpark are the two newest speculative decoding methods.
Dual-R9700-Cooperation	Why two R9700s could not run tensor-parallel on the old board, and what the board swap unblocked.
FastFlowLM NPU Inference	The AMD NPU-first runtime: fit analysis for the XDNA2, published benchmarks, install state, and test runbook.
FreeToken Builds	Building the FreeToken runtime here: the CUDA and ROCm builds and the dependency trap each one hit.
freetoken-benchmark-sweep	KV pool versus decode speed for the FP8 mixture-of-experts payload on the FreeToken runtime.
GGUF Compatibility	Answering whether an engine loads a GGUF from the file header alone, and pinning a fork build that can.
Heterogeneous Split Triage	Triage before splitting across unequal accelerators: link characterisation, real bottleneck, what placement never buys.
Hybrid CPU GPU Processing Dankserver	Dankserver uses NVIDIA GPU with system RAM for hybrid processing.
ling-3.0-flash-trial	Trial of a fork build for a new mixture-of-experts architecture: what loaded, what the harness refused, and the verdict.
Llama Cpp Multi-Device Splits	What llama.cpp can express across devices: the no-collective layer-split path, balancing, and drafter KV sizing.
Llama Cpp Serving Sizing	Measurement-first sizing for llama.cpp shapes: slot pools against a headroom band, per-device accounting, prompt cache.
LLM Bench Windows and Measurement	Opening and restoring benchmark windows on fleet hosts, honest methodology, and the usage-counter law under drafting.
LLM Performance Diagnosis	Probing why a served model is slow: the flat-depth probe, offload duty-cycle arithmetic, and profiler attribution.
Model Swap and Rollback	The model-swap discipline: fit preflight, fetch verification, build and flag checks, live validation, three-pin revert.
PCIe Link and Atomics	Link probing, the streaming-versus-compute ceiling arithmetic, and the atomics wall that decides multi-GPU collectives.
Quantization by Hardware Architecture	This page maps LLM quantization formats to the GPU and CPU architectures that support them.
Quantization Production	Making a quantized payload here: the conversion recipe, the scoring gate, and the campaign ladder behind a shipped file.
qwen3.8-strix-optimizations	How a served checkpoint got faster on the Strix Halo iGPU: the lever ladder and what each lever actually bought.
RAM CPU Offloading in VLLM	vLLM uses CPU RAM and swap space to extend GPU memory capacity.
RDNA4 Container Serving	Bringing a model larger than VRAM onto the two-card box: quant gating, fetch verification, container GPU-init triage.
RDNA4 MoE Expert Offload VRAM Budget	Expert-offload arithmetic on the RDNA4 fork: movable terms, the utilisation law, graph charge, ceilings, context ladder.
Running Qwen3.6 27B FP8 on Dankclaw	Serving a dense FP8 vision-language checkpoint on the Strix Halo iGPU: the load path, the flags, the format gate.
Running Qwen3.6 35B A3B FP8 on Dankserver	Serving a mixture-of-experts FP8 checkpoint on the R9700 host: the load path, the format gate, the offload shape.
Shared Memory Architecture Dankclaw	Dankclaw uses AMD Strix Halo APU with unified memory architecture.
swift15-campaign-worklog	Dated worklog of the campaign that converted and served a Swift checkpoint on the radiance endpoint, arm by arm.
Tcclaviger Fork Ops	Operating the RDNA4 vLLM fork: the sweep harness, measured knob verdicts, sweep hygiene, compile-cache persistence.
ubatch-benchmark-large	The wide ubatch sweep for a deployed ROCmFPX shape, run through the standardized harness.
vLLM Boot Forensics and Config Interrogation	Recovering a container boot failure's real cause from journald, and interrogating a live endpoint's true config.
VLLM Index	Locator for every vLLM page in this vault.
VLLM Overview	vLLM is a fast and easy library for large language model inference and serving.
vLLM Porting and Compile Triage	Forward-porting a model architecture: additive-only discipline, shims that survive worker spawn, compile triage.
vllm_research_findings	Upstream support survey for speculative decoding: which flags, releases and pull requests carry each method.
x4-pair-experiment	Measured limits of a consumer card plus APU pair: the tensor inventory, the flag that kills cross-device placement.

Pages

Page	Owns
frank-medium	Locator for the retired Dankserver LLM endpoint: which era page owns which payload, and where the dates disagree.

8.6 KiB Raw Permalink Blame History