docs(vault): strip restated fleet facts from 17 technique pages

Enforces "one fact, one owner": technique pages keep mechanism, laws, traps
and verdicts; per-endpoint shapes, digests and era measurements are cited to
the endpoint page that owns them.

Also repairs the four pointers the pass left empty (each verified against the
cited owner before landing):
- EMULATION dequant doubles host RAM (118 GiB -> ~238 GiB) -> [[frank-archon]]
- 12.11 tok/s flat 304 -> 16,095 tokens -> [[Tcclaviger Fork Ops]]
- bundled-head acceptance ladder 0.739/0.554/0.427/0.340 -> [[Qwen3.8 PP Deployment]]
- llama.cpp PR #23897 dedup provenance -> [[qwen3.8-spec-matrix]]

Replaces the retired-shape illustration in Llama Cpp Serving Sizing with the
owned MiMo Q5_K_M row (6.40 GiB payload -> 20,530 MiB used), and unwraps a
hard-wrapped [[Dankserver System Specs]] link that failed link integrity.

Orphan census over the diff: 0 numbers lost (comma-insensitive match against
the whole vault plus AGENTS.md).
This commit is contained in:
Frank committed 2026-10-19 01:33:47 -07:00
1 parent eb21dbc705
commit 04f4e2c6d4
18 files changed
+348 -425

No files matched your search

@@ -14,54 +14,28 @@ timestamp: 2026-09-03
# Qwen3.8-27B Speculative-Decoding Matrix
This page records the speculative-decoding benchmark history and the
production tuning for the Qwen3.8-27B service (`frank-dragoon`),
now a pipeline-parallel head with a Dankserver worker.
production tuning for the then-Qwen3.8-27B service (`frank-dragoon`), at the
time a pipeline-parallel head with a Dankserver worker.
## Production state (2026-08-17: Q6_K, q8_0/f16 KV, split 0.68)
## Production state (2026-08-17)
Current production, verified against the live process and journal
(2026-08-17, load 02:06 + smoke tests). Topology detail:
[[Pipeline Parallelism/Qwen3.8 PP Deployment]].
- Devices: `RPC0` (Dankserver RTX 3090) + `ROCm0` (Strix iGPU). The head
passes no `--device` flag. Default selection puts RPC first and picks
ROCm for the iGPU (HIP registers before Vulkan). The Vulkan0 side stays
an option via `HEAD_BIN=.../build-vulkan-rpc`.
- Model: `Qwen3.8-27B-Uncensored-Q6_K.gguf` (JonathanColetti, 22.43 GB,
imatrix, 65 blocks, 866 tensors, NO bundled MTP head). The repo's
standalone draft sidecar `Qwen3.8-27B-Uncensored-draft-Q8_0.gguf`
(3.16 GB) is loaded via `--spec-draft-model`; it runs on ROCm0 only
(1718 MiB, zero worker impact).
- Spec: **MTP** (standalone draft) `--spec-draft-n-max 4` + the
`--spec-default` ngram-mod fallback (48/64/24). Measured acceptance
0.83 then 0.75, mean len 4.0-4.3, per-position ~0.86/0.71/0.71/0.71 —
stronger than the old bundled head (0.739/0.554/0.427/0.340).
- KV: K **q8_0** + V **f16** (mismatched). 3136 B/token per FA layer =
784 MiB at 262144 (K 1088 + V 2048). RPC0 8624 MiB (11 FA), ROCm0
3920 MiB (5 FA); total 12544 = 16 x 784. **Flash Attention stays
enabled with the mismatch** (`resolve_fused_ops: Flash Attention
enabled`) via the GGML_CUDA_FA_ALL_QUANTS rebuild of both binaries
at 9b05354ec — see the PP doc.
- Split: `--tensor-split 0.68,0.32` = 45 of 65 blocks (0-44) on the
3090, 69% of the layer weight. The maximum: 46 blocks -> ~0.3 GiB
margin; 48 blocks (f >= 0.7197) adds FA block 47 -> KV 9408 -> OOM by
~0.85 GiB. Boundary math: the PP doc.
- Context: pool 262144, 1 slot, `n_ctx_slot = 262144`. The pi
`models.json` declares 262144 (contextWindow = maxTokens, house
convention).
- Worker steady state: 23854/24576 MiB used on the 3090, ~273 MiB free
(tight but stable; all allocations happen at load).
- Batch 8192 / ubatch 256, `--direct-io`, `--fit off`,
`--gpu-layers all`.
The pipeline-parallel shape this matrix was tuned for — Q6_K target, standalone
Q8_0 draft on ROCm0 only, `--tensor-split 0.68,0.32` (45 of 65 blocks), K `q8_0`
/ V `f16`, pool 262144 at one slot, worker steady at 23854/24576 MiB — is
recorded once, in [[Pipeline Parallelism/Qwen3.8 PP Deployment]] §Production
state, together with the split-limit arithmetic (why 45 blocks is the maximum),
the KV-per-layer bill and the FA-all-quants rebuild the mismatch needed. That
era's endpoint identity is an [[frank-dragoon]] era. This page keeps only the
speculative-decoding measurements.
## How MTP works on Qwen3.8 (bundled-head era)
The unsloth `UD-*` GGUFs keep the MTP (Multi-Token Prediction) head inside
the model file. llama.cpp b10411+ auto-detects it and creates the draft
context from the target model. Do not pass `--spec-draft-model` for those
files. This is superseded: the production `Qwen3.8-27B-Uncensored-Q6_K.gguf`
has NO bundled head, and the serve script passes the standalone draft
sidecar with `--spec-draft-model` (see Production state above).
files. This is superseded: the production target of 2026-08-17 carried no
bundled head, and the serve script passed the standalone draft sidecar with
`--spec-draft-model` (shape: [[Pipeline Parallelism/Qwen3.8 PP Deployment]]).
The stock llama.cpp build must be b10411 or newer. The 2026-08-14 rebuild
on Dankclaw is b10433 (`9b05354ec`) with `GGML_HIP` + `gfx1151` + Vulkan.
@@ -83,14 +57,14 @@ standalone configuration only; the PP head runs ROCm0.
6/ABRT, systemd restart, ~503 during reload). DSpark reuses the DFlash
block-diffusion machinery, which shares one draft context across
parallel slots; concurrent block drafting faults in HIP stream sync.
This is the same multi-slot bug class that retired `draft-dflash`
(premature EOS). The crash shows on 16k concurrent streams, not on a
67k pair — concurrency is the trigger, not size. Fix: PARALLEL=1
(requests queue safely). Retired the same day.
This is the same multi-slot bug class that retired `draft-dflash` (premature
EOS, retired 2026-08-14). The crash shows on 16k concurrent streams, not on a
67k pair — concurrency is the trigger, not size. Fix: PARALLEL=1 (requests
queue safely). Retired the same day.
- **2026-08-15 to 08-16: `--spec-default` (ngram-mod), Vulkan0** — n_match
24 / n_min 48 / n_max 64, draftless, PARALLEL=2 (2 x 32k parallel stream
test passes). Chosen for parallel safety after DSpark; kept as the
fallback under MTP in the current config.
fallback under MTP in the 2026-08-17 shape.
## Benchmark Method
@@ -146,19 +120,19 @@ ubatch 256, single slot, on scratch port 7892.
Verdict: at very high context, Vulkan0 decodes **48% faster** (5.68 vs
3.83 t/s); ROCm0 prefills 12% faster. Generation is the interactive cost
(users stream tokens). This verdict applies to the standalone
configuration; the PP head runs ROCm0 (HIP registers before Vulkan).
(users stream tokens). This verdict applies to the standalone configuration; the PP
head runs ROCm0 — HIP registers before Vulkan and the iGPU dedup rule (the llama.cpp
PR #23897 workaround) keeps the first backend's device.
Both `tg_3s` values were rock-steady across the run (3.83 and 5.67 t/s).
High-context decode collapses on both backends versus ~25 t/s at 16k ctx;
that is the long-context attention penalty, not a spec artifact.
## Production Configuration
## Where the tunables live
The live tunables live in `scripts/frank-dragoon.env` (head) and
`scripts/frank-pp-worker.env` (worker) in this repo. The Production
state section at the top of this page records the current values: Q6_K
model + standalone MTP draft, split 0.68/0.32, 262144 ctx, PARALLEL=1,
q8_0/f16 KV.
That era's head and worker env/units are listed (and marked deleted) in
[[Pipeline Parallelism/Qwen3.8 PP Deployment]] §Files. `scripts/frank-dragoon.env`
is no longer a PP-head env at all — it is the halogen-flash-server trio of the
current era, which [[frank-dragoon]] owns.
## See Also