docs: vault link integrity restored after rogue-session doc WIP landed

The parallel session's stash pop left GGUF Compatibility, Multi-Device
Splits, Serving Sizing, PCIe Link and Atomics, RDNA4 Container Serving,
vLLM Porting triage and RDNA4 MoE Budget unindexed and carrying 9
unresolved wikilinks (LLM Bench Windows / Model Swap / AGENTS targets
that never existed, a [[nodiscard]] C++ attribute parsed as a wiki link,
and two multi-line-wrapped link tokens). All repointed to real pages;
six rows added to the vault index; seam green.

serverAppConfigs-docs/LLM/Llama Cpp Serving Sizing.md

@@ -0,0 +1,163 @@
+---
+type: reference
+title: Llama Cpp Serving Sizing
+description: Slot-pool sizing against a headroom band, per-device VRAM accounting, prompt-cache/state sizing and the QSA flash-attn fix — the measurement-first procedures for llama.cpp serving shapes.
+tags: [llama.cpp, vram, kv-cache, slots, flash-attn, sizing]
+timestamp: 2026-10-05T00:00:00-07:00
+---
+
+# Llama Cpp Serving Sizing
+
+## 1. Slot-pool sizing against a headroom band
+
+Goal: choose `CTX_SIZE`/`PARALLEL`/KV types so N conversations fit a GPU while
+keeping a requested amount of VRAM free — from measurement, never arithmetic
+guesswork.
+
+1. **Fix the invariants.** Read the payload's KV geometry (attention layers
+   only, KV heads, head_dim, KV dtype bytes) and the per-slot rule
+   `n_ctx_slot = CTX_SIZE / PARALLEL`. A hybrid model with
+   `kv_unified = 'false'` gives each slot its own cache AND a fixed per-slot
+   state buffer — slot count is a memory term, not free.
+2. **Two boots on the real card** (production stopped, per the bench law),
+   each near the target; record `nvidia-smi` used/total plus the boot log's
+   `n_slots`/`n_ctx_slot`: shape A = current/cheap, shape B = new demand.
+   `non-KV = used − (tokens × KiB/token)`; the two rows separate
+   weights/buffers/graphs from the new demand (a 918 MB projector bills
+   ~1.4 GiB).
+3. **Solve the band:** `KV_tokens ≤ (card − non-KV − headroom_low) /
+   KiB/token`, round the pool DOWN to a multiple of `PARALLEL × 4096` so every
+   slot gets an equal integer window. Boot and measure; if outside the band,
+   walk one step — never re-derive from a formula.
+4. **Propagate the per-slot number — it is the client contract:**
+   `models.yml` `contextWindow` = `n_ctx_slot` (NOT the model's native window),
+   or long conversations die with `400 exceed_context_size_error` instead of
+   compacting; compaction/summary budgets sized to the old window get
+   re-checked; endpoint docs and the rollback table move too.
+5. **Prove behaviour:** boot-log slot count, `nvidia-smi` free in band, N
+   concurrent requests answered in parallel with N+1 queued, and — if a
+   projector ships — a solid-color image probe through an **`image_url`
+   content part** expecting the exact color. The legacy top-level `images`
+   field is silently ignored, so a probe that uses it measures a
+   hallucination, not vision.
+
+Gotchas: shorter slots can make decode FASTER (~5 % measured); the KV pool is
+allocated whole at boot so post-boot `memory.used` IS the ceiling; record both
+measured rows in the service `.env` comment block and mirror the non-KV price
+in the bats budget gate so a shape that busts the band fails the suite instead
+of the boot.
+
+## 2. Per-device VRAM accounting ("where did the memory go?")
+
+First decide which kind of memory is in question:
+
+| device | kind | read it with |
+|---|---|---|
+| RTX 3090 (Dankclaw) | discrete GDDR6X | `nvidia-smi --query-gpu=memory.total,memory.used,memory.free` |
+| gfx1151 iGPU (Dankclaw) | **UMA** — "VRAM" is host RAM | `/sys/class/drm/card1/device/mem_info_gtt_{total,used}` (114 GiB) vs `mem_info_vram_*` (2 GiB carve-out) |
+| R9700s (Dankserver) | discrete | `rocm-smi` figure (64 GiB, SVM-backed) |
+
+Per-process truth on NVIDIA:
+`nvidia-smi --query-compute-apps=pid,process_name,used_memory`. Never add the
+iGPU's GTT to host RAM as a separate pool; `free -g` is the arbiter for
+co-tenants.
+
+- **Engine ledger first.** Halogen prints weights pinned (+ remaining 2 MiB
+  contiguous blocks — the honest fragmentation signal: 240 blocks/12.3 GiB
+  after reboot vs 475/14.7 GiB before means the model did not come back into
+  the same hole), device budget, KV pool/working split, "left for everything
+  else". llama.cpp prints at `llama-model.cpp:1827` (per-backend model buffer)
+  and `llama-context.cpp:692` (compute buffer) — but **`llama-bench`
+  suppresses loader INFO and a systemd unit did not put them in the journal**
+  (verified `grep -c` = 0). Recipe: one launch through `llama-cli`/`llama-server`
+  keeping the first ~200 lines.
+- **Otherwise compose from tensor sizes.** Group the GGUF tensor table by
+  role (Flash-Next example: 75.0 GiB experts at 1.5625 GiB/layer, 32.8 GiB PLE
+  table, 3.2 GiB everything else), KV ≈ 2 KiB/position per f16 attention
+  layer, × derived context. Vendor delta is real: same layers+KV cost 12.80
+  GiB under Vulkan vs 14.99 GiB under CUDA (~3.3 vs ~1.1 GiB of
+  graph/compute pool).
+- **Traps:** mmap'd PLE is reclaimable page cache; `-ot` makes it pinned —
+  "move layers to the card" converts reclaimable cache into device memory and
+  frees nothing for co-tenants. A served model's footprint ≠ weight size
+  (frank-small: 22.6 of 24.1 GiB for an 11.29 GiB model). Do the capacity
+  subtraction before designing: 12 attn layers × 1.62 GiB + 200k f16 KV =
+  28.6 GiB > 24.1 GiB usable → KV-in-VRAM on that pair only reaches ~64k.
+- **Context lengths: derive, never remember.** `llama-bench` computes
+  `n_ctx = n_prompt + n_gen + n_depth` (defaults 512/128/0, type_k/v F16,
+  flash_attn AUTO, split_mode LAYER). `-p 512,16384 -n 64` = contexts
+  576/16,448; a plain `tg` row has ~zero depth — `-d` is the only thing that
+  puts KV in the picture.
+- **Label every number** measured / derived / inference, and state the context
+  each figure was taken at — 12.8 GiB at 65,568 tokens is not the same claim
+  as at 200k.
+
+## 3. Prompt cache and state sizing
+
+`--cache-ram` (MiB, default 8192) is HOST RAM for saved prompt states —
+unpinned, on demand, moves no VRAM. `--ctx-checkpoints` (default 32) is a
+different mechanism: per-slot KV snapshots; `0` disables them.
+
+**State law** (compute before trusting any cap): a state is the serialized KV
+of the whole sequence + constant recurrent state + the draft context's KV:
+`per_token = 2 × n_full_attn_layers × n_kv_heads × head_dim × bytes (K and V) +
+draft_per_token`. Measured on frank-medium (qwen3.5 hybrid, f16, MTP draft):
+16×4×256×2×2 = 64 KiB/token + 4 KiB draft = **68 KiB/token**; a full
+196,608-token slot state ≈ 12.8 GiB. Cross-check against the journal's
+`prompt state size` lines — the two must agree.
+
+**The journal is the instrument:** `exceeds cache size limit … skipping` =
+cap binding in production; `making room … removing oldest` / `cache size
+limit reached` = eviction, and the printed size proves the active cap. Do NOT
+claim a raise is proven by `prompt_tokens_cached_total` rising — on exact
+repeats that counter is 0 by construction.
+
+**Hybrid + zero-checkpoint trap:** for a hybrid (DeltaNet/SSM + attention)
+model, `seq_pos_min` sits at the END of the slot's own prompt (a recurrent
+cell stores only the last position), so with `--ctx-checkpoints 0` there is no
+checkpoint to fall back on and the server logs `forcing full prompt
+re-processing … (likely due to SWA or hybrid/recurrent memory)`. Consequences:
+intra-slot reuse only survives when the request echoes the assistant's
+previous reply verbatim (a pure user resend is always a full re-prefill even
+at f_sim 1.000); the RAM prompt cache becomes the ONLY carrier of a
+conversation state ACROSS slots, so a cap that refuses deep states costs
+minutes of re-prefill whenever slots are reassigned; state restoration only
+runs via the LRU path (or LCP with f_keep < 0.5) — a plain prefix hit never
+calls `prompt_load`.
+
+**Verify a change:** live argv carries the flag (not the deploy log); `/health`
+200 + real completion; VRAM unchanged and host RAM mostly free; no new
+`exceeds cache size limit` lines, with a `making room` entry larger than the
+old cap as direct proof. If the ceiling itself was not exercised live, say so —
+crossing it needs a >120k-token conversation and must not run beside a live
+service (bench law).
+
+## 4. QSA flash-attn assert fix (qwen4exp)
+
+`--flash-attn on` aborts graph build for qwen4exp models:
+`GGML_ASSERT(a->type == GGML_TYPE_F32)` at `ggml.c:5371` through
+`build_attn_qsa`. Root cause: `build_attn_qsa` calls `ggml_fill` on the
+F16-cast kq mask returned by `get_kq_mask()` when flash attn is on, but
+`ggml_fill` requires F32.
+
+Fix (mirrors the MLA sparse path in `llm_graph_context::build_attn`): run the
+top-k mask surgery on the F32 base mask (`inp->self_kq_mask`), cast to F16
+only at the attention call when `flash_attn` is enabled
+(`kq_mask_attn = cparams.flash_attn ? ggml_cast(ctx0, kq_mask_top_k,
+GGML_TYPE_F16) : kq_mask_top_k`). The patch lives at
+`scripts/patches/qwen4exp-qsa-f32-mask.patch`.
+
+Only the charlie12345/ROCmFPX build-qwen38 (PR #27742 qwen4exp merged) takes
+it; CIRU, upstream and stock builds do not. Validate live: load with
+`--flash-attn on`, health 200, a real completion finishing `"stop"`, journal
+clean of `GGML_ASSERT`/`cudaMalloc`/`missing tensor`, and flash-attn engaged
+in the timings. Context cap for full-model residency on that shape stayed
+32,768.
+
+## Related
+
+- [[GGUF Compatibility]] — what a payload is before it is priced
+- [[frank-small-llama-mimo-q6k-2x200704]] — the current two-boot priced shape
+- [[frank-medium-llamacpp-dflash2]] — where the 68 KiB/token datum was measured
+- [[DFlash DSpark Speculative Decoding]] — the spec A/B protocol on llama.cpp
+- [[Dual-R9700-Cooperation]] / [[x4-pair-experiment]] — the multi-device side