docs(vault): strip restated fleet facts from 17 technique pages

Enforces "one fact, one owner": technique pages keep mechanism, laws, traps
and verdicts; per-endpoint shapes, digests and era measurements are cited to
the endpoint page that owns them.

Also repairs the four pointers the pass left empty (each verified against the
cited owner before landing):
- EMULATION dequant doubles host RAM (118 GiB -> ~238 GiB) -> [[frank-archon]]
- 12.11 tok/s flat 304 -> 16,095 tokens -> [[Tcclaviger Fork Ops]]
- bundled-head acceptance ladder 0.739/0.554/0.427/0.340 -> [[Qwen3.8 PP Deployment]]
- llama.cpp PR #23897 dedup provenance -> [[qwen3.8-spec-matrix]]

Replaces the retired-shape illustration in Llama Cpp Serving Sizing with the
owned MiMo Q5_K_M row (6.40 GiB payload -> 20,530 MiB used), and unwraps a
hard-wrapped [[Dankserver System Specs]] link that failed link integrity.

Orphan census over the diff: 0 numbers lost (comma-insensitive match against
the whole vault plus AGENTS.md).
This commit is contained in:
Frank committed 2026-10-19 01:33:47 -07:00
1 parent eb21dbc705
commit 04f4e2c6d4
18 files changed
+348 -425

No files matched your search

@@ -14,7 +14,7 @@ timestamp: 2026-09-03
DFlash and DSpark are the two newest speculative decoding methods. They draft many tokens in one parallel pass instead of one token at a time. This reduces decode latency on memory-bound workloads.
DFlash comes from Z Lab at UC Berkeley. DSpark comes from DeepSeek. Both work with vLLM and llama.cpp. Both have draft models for the Qwen3.6 models that Dankserver and Dankclaw run.
DFlash comes from Z Lab at UC Berkeley. DSpark comes from DeepSeek. Both work with vLLM and llama.cpp. Both publish draft models for the Qwen3.5/Qwen3.6 families (table below); what any of them pairs with on this fleet is an era on [[frank-dragoon]] or [[frank-zealot]].
## How DFlash Works
@@ -115,12 +115,12 @@ unconditionally, while **context-backed** types (`draft-mtp`, `draft-dflash`,
- Internal order is fixed (ngram types first, then drafters); CLI order is
irrelevant.
- **A legal chain can still lose on a recurrent model.** `draft-mtp,ngram-mod`
on the qwen35 hybrid (frank-dragoon era) measured −15 % prose / −7 %
block-copy (2026-10-03): adding the ngram layer logs `recurrent state
rollback is not compatible with 'ngram-mod' - disabling rollback support`,
and MTP verification depends on that rollback, so acceptance drops on ALL
traffic even where the matcher fires. Measure chains per architecture,
never by analogy.
on the qwen35 hybrid loses on both traffic shapes (2026-10-03): adding the
ngram layer logs `recurrent state rollback is not compatible with
'ngram-mod' - disabling rollback support`, and MTP verification depends on
that rollback, so acceptance drops on ALL traffic even where the matcher
fires. The measured size of each loss is [[frank-dragoon]]'s (Laws). Measure
chains per architecture, never by analogy.
- ngram-mod prints NO startup banner — the load log shows only the drafter's
`adding speculative implementation` line. Never conclude "chain not
engaged" from the load log; identify behaviourally (`draft_n` > 0 only
@@ -198,7 +198,10 @@ Compatible MTP-only draft GGUFs on Hugging Face:
Speculative decoding is lossless under rejection sampling: the target verifies every draft token. A draft head from the base model works with a derivative target; wrong proposals just lower the acceptance rate. The accepted MTP draft lengths are 2-3 tokens; the "MTP Defaults Are a Trap" writeup (thefrontierlab.ai) documents the tuning pitfalls.
On Dankclaw, `draft-dflash` remains the benchmarked winner (20.2 t/s at 4k vs 16.9 for DFlash+ngram and 8.7 for ngram-only). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see [[deployment]]).
On the mid-August 2026 Dankclaw shape `draft-dflash` was the benchmarked winner
at 4k — 20.2 t/s, against 16.9 with ngram layered on and 8.7 ngram-only; those
three figures are that retired llama.cpp era's property ([[frank-dragoon]] owns
the era and its shape). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see [[deployment]]).
## Draft Models
@@ -213,13 +216,12 @@ DFlash draft model sizes for the models on these servers:
The DFlash draft is not a standalone model. It must pair with its target model. The model cards recommend block size 8 for higher concurrency and 16 for single-stream throughput.
## Deployment on Dankserver (NVIDIA RTX 3090)
## DFlash on an RTX 3090 (Ampere) — the archived vLLM path
The Dankserver llama.cpp deployment (`frank-zealot`, Qwen3.6-35B-A3B) is **active** on Dankserver (port 7770) as of the 2026-08-17 redo (the earlier 2026-08-16/17 retirement of this service was reversed when the PP split was retired). The vLLM material below documents the archived AWQ/DFlash path on the same hardware.
> **DFlash retired from active service.** `DFlash` (draft-dflash) was tested initially but did not deliver the desired speed gains under the `frank-zealot` workload (see ngram-mod benchmarks below). The `Alittlehammmer/Qwen3.6-35B-A3B-DFlash` draft GGUF remains in `/home/dank/models/` as reference material only.
The vLLM material below documents the archived AWQ/DFlash path on the same hardware. The RTX 3090 is Ampere (compute capability 8.6). Use these conditions:
DFlash was tried on the 3090 line, lost there to the ngram chain, and was never
kept deployed; that card's payloads, shapes and rates are [[frank-zealot]]'s
(its Eras rows are the llama.cpp arms the draft was tested against). What
transfers is the Ampere arithmetic — compute capability 8.6, so:
- FP8 KV cache is valid on CUDA 12.
- FP8 weights compute through a fallback path; no native FP8 tensor cores.
@@ -248,11 +250,15 @@ docker run --rm --name vllm-qwen36-dflash \
--gpu-memory-utilization 0.95
```
## Deployment on Dankclaw (AMD Strix Halo gfx1151)
## DFlash and DSpark on Strix Halo (gfx1151) — what killed them
`frank-dragoon` on Dankclaw ran the llama.cpp pipeline-parallel head (Qwen3.8-27B-Uncensored Q6_K) on port 7770, RPC-split with the Dankserver worker. Spec was the standalone MTP draft over the ngram-mod fallback, 1 slot at 262144 ctx, q8_0/f16 KV, head GPU side ROCm0. This page is a historical record of that era (mid-August 2026); for the current deployment see [[frank-dragoon]].
History: before `--spec-default`, the box ran DSpark (`draft-dspark`, retired 2026-08-15 after a ROCm illegal-memory-access crash under two concurrent 16k prompts) and before that bundled MTP (`draft-mtp`, retired same day). DFlash was retired on 2026-08-14 (premature-EOS under parallel load). The old DFlash draft GGUF (`Qwen3.6-27B-DFlash-Q8_0.gguf`), the MTP-only drafter (`Qwen3.6-27B-MTP-ONLY-Q8_0.gguf`), and the DSpark draft (`Qwen3.8-27B-DSpark-Q8_0.gguf`) remain in `/home/dank/models/` as reference only.
Both context-backed drafters died the same way on the iGPU: under parallel load.
DSpark took `ROCm error: an illegal memory access` inside the shared draft
context with two concurrent 16k prompts; DFlash corrupted multi-slot into
premature EOS and was retired first. The timeline, the crash class, the
"concurrency is the trigger, not size" finding and the `PARALLEL=1` workaround
are on [[qwen3.8-spec-matrix]]; the shape they were measured on is
[[frank-dragoon]]'s mid-August 2026 era.
## A/B protocol on a live llama.cpp endpoint
@@ -359,16 +365,18 @@ pay for K sequential forwards.
Trained heads are target-specific: EAGLE/DFlash consume the TARGET's hidden
states, so a dense-27B DFlash2 is useless for a 512-expert MoE trunk.
**Step 4 — acceptance reality checks.** Fleet datum: a trained z-lab DFlash2
at depth 7 accepted ~41 % on its matched target (`draft-dflash 7 → 43.5
tok/s`); treat 40–60 % as the honest band for a matched pair. Cheapest proxy
for `p` at temp 0 when both models are greedy and share a tokenizer: greedy
continuation agreement on identical prompts, two API calls, no weights moved.
llama.cpp can also synthesize the projection without any draft artifact:
**Step 4 — acceptance reality checks.** Treat 40–60 % as the honest band for a
matched pair; the arm that anchors the bottom of it on this fleet is the
depth-7 `draft-dflash` row of [[frank-medium-llamacpp-dflash2]]'s spec table
(accepted share, rate and VRAM for every depth are in that table). Cheapest
proxy for `p` at temp 0 when both models are greedy and share a tokenizer:
greedy continuation agreement on identical prompts, two API calls, no weights
moved. llama.cpp can also synthesize the projection without any draft artifact:
`--spec-synth-rates P0,P1,...` / `--spec-synth-len L`. Speculation pays IN
PROPORTION TO HOW SLOW THE TARGET IS — the same 41 % gave 2.3x on an
18.8 tok/s layer-split target and a loss on a 25 tok/s one; fast targets are
usually sparse MoE, the worst case for drafting.
PROPORTION TO HOW SLOW THE TARGET IS — that band's midpoint was worth 2.3× on
the slow two-card layer-split target and a LOSS on the 25 tok/s target of the
worked example above; fast targets are usually sparse MoE, the worst case for
drafting.
**Step 5 — check the attachment surface BEFORE the arithmetic.** Closed
engines refuse external drafters by design: halogen-flash-server's