docs(vault): destroy the dead weight — 53 pages and the change diary

The vault was 155 pages / 31,210 lines, most of it measurement ledgers for
payloads that no longer run. A throughput figure for a runtime nobody operates is
not a record: AI moves faster than the vault, so the number is wrong before
anyone reads it, and a wrong number in a runbook gets believed. Per the new
purpose test in `AGENTS.md` → *Documentation Policy*, it is destroyed.

Gone (`.AGENTTRASH/20261019-destruction/`, KILL-LIST.txt is the manifest):
- the `frank-medium` endpoint and its five era pages, `frank-npu`, FastFlowLM,
  the ikllama/kingjones/dealign/frank-redo eras, the ling/swift15/freetoken/ubatch
  campaign pages, qwen3.8 spec-matrix and strix-optimizations, the two
  "Running Qwen3.6 …" pages;
- the whole `Pipeline Parallelism/` folder (no live script passes a pipeline flag)
  and the multi-device/x4-pair/hybrid/offload experiment pages;
- every vLLM page — no vLLM endpoint exists on this fleet;
- `LLM/deployment.md` (`Architecture/Deployment Flow.md` is the test-referenced
  owner) and the external surveys (SGLang, Lemonade, llama.cpp forks/diff,
  self-hosted app census, the TTS survey) that were obsolete by construction;
- `log.md`, 2,380 lines. Git holds the chronology; a causal law belongs to the
  page that owns the thing it governs, never to a diary.

Survivors: 101 pages, 17,542 lines, 0 OKF violations, 0 unresolved links. Eras
tables keep the window, payload, runtime, reason and rollback pointer; the rate
columns are blanked. The live shapes stay verbatim — those numbers are what the
deployer and `models.yml` consume.
This commit is contained in:
Frank committed 2026-10-19 10:56:37 -07:00
1 parent 8775e64ed0
commit 327e78b950
126 files changed
+443 -14157

No files matched your search

@@ -100,7 +100,7 @@ The DFlash drafter proposes, then ngram-mod runs as an additional implementation
`--spec-type` takes a comma list, and the gate is `ctx_dft` in
`common/speculative.cpp` (`common_speculative_init_result`): **draftless**
types (the ngram family — simple, map-k, map-k4v, mod, cache) are added
unconditionally, while **context-backed** types (`draft-mtp`, `draft-dflash`,
unconditionally, while **context-backed** types (`draft-mtp`, `draft-dflash`
`draft-dspark`, `draft-eagle3`) all require a draft CONTEXT. Consequences:
- A chain may hold ONE context-backed drafter plus any number of ngram types.
@@ -201,7 +201,7 @@ Speculative decoding is lossless under rejection sampling: the target verifies e
On the mid-August 2026 Dankclaw shape `draft-dflash` was the benchmarked winner
at 4k — 20.2 t/s, against 16.9 with ngram layered on and 8.7 ngram-only; those
three figures are that retired llama.cpp era's property ([[frank-dragoon]] owns
the era and its shape). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see [[deployment]]).
the era and its shape). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see ).
## Draft Models
@@ -257,7 +257,7 @@ DSpark took `ROCm error: an illegal memory access` inside the shared draft
context with two concurrent 16k prompts; DFlash corrupted multi-slot into
premature EOS and was retired first. The timeline, the crash class, the
"concurrency is the trigger, not size" finding and the `PARALLEL=1` workaround
are on [[qwen3.8-spec-matrix]]; the shape they were measured on is
are on ; the shape they were measured on is
[[frank-dragoon]]'s mid-August 2026 era.
## A/B protocol on a live llama.cpp endpoint
@@ -303,7 +303,7 @@ Protocol:
Read the right counters — per request `timings.draft_n` / `draft_n_accepted` /
`predicted_per_second`; cumulative `/metrics`
`llamacpp:spec_decode_num_drafts_total` / `..._accepted_tokens_total`;
`llamacpp:spec_decode_num_drafts_total` / `..._accepted_tokens_total`
per-task `draft acceptance = ... (N accepted / M generated)`. `draft_n == 0`
while tokens flow = the proposer is NOT engaged, whatever the flags say. A
zero-draft config is a useful negative control: it proves the other config's
@@ -367,7 +367,7 @@ states, so a dense-27B DFlash2 is useless for a 512-expert MoE trunk.
**Step 4 — acceptance reality checks.** Treat 40–60 % as the honest band for a
matched pair; the arm that anchors the bottom of it on this fleet is the
depth-7 `draft-dflash` row of [[frank-medium-llamacpp-dflash2]]'s spec table
depth-7 `draft-dflash` row of 's spec table
(accepted share, rate and VRAM for every depth are in that table). Cheapest
proxy for `p` at temp 0 when both models are greedy and share a tokenizer:
greedy continuation agreement on identical prompts, two API calls, no weights
@@ -381,10 +381,10 @@ drafting.
**Step 5 — check the attachment surface BEFORE the arithmetic.** Closed
engines refuse external drafters by design: halogen-flash-server's
`DRAFTER_CAP` gates `dflash2` on the loaded checkpoint and an unavailable
name is a deliberate 400 (a silent downgrade would make comparisons lie);
name is a deliberate 400 (a silent downgrade would make comparisons lie)
there is no device routing in its loopback token protocol, so a drafter can
never sit on another card. Pure llama.cpp DOES route the draft:
`--spec-draft-device/-devd` (default follows `--device`), `-ngld`,
`--spec-draft-device/-devd` (default follows `--device`), `-ngld`
`-otd`, `-ctkd/-ctvd`, `--spec-draft-n-max` (default 3), `--spec-draft-p-min`.
One process sees only the devices its backends were built with — CUDA+Vulkan
is the combination that shows a CUDA card AND an AMD iGPU (HIP+CUDA cannot
@@ -411,20 +411,16 @@ measurement that would settle it.
- Speculative decoding helps low-QPS, memory-bound, latency-focused workloads. It helps less at high QPS.
- DFlash needs a non-causal attention backend; the ROCm flash_attn backend must be validated before production use.
- The old `--swap-space` flag is retired; the live CPU-KV-offload knob pair is owned by [[RAM CPU Offloading in VLLM]].
-The old `--swap-space` flag is retired; the live CPU-KV-offload knob pair is owned by .
- `--speculative-config` is the preferred CLI. Legacy `--spec-method`, `--spec-model`, and `--spec-tokens` flags still exist.
- Use `rejection_sample_method` default `strict` for lossless output.
## See Also
- [[Quantization by Hardware Architecture]]
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]]
- [[Running Qwen3.6 27B FP8 on Dankclaw]]
- [[Dankclaw System Specs]]
- [[Dankserver System Specs]]
- [[VLLM Overview]]
- [[deployment]]
- [[GGUF Compatibility]] — header parsing and custom tensor-type lock-in
- [[Llama Cpp Serving Sizing]] — slot-pool and drafter KV sizing arithmetic
- [[Llama Cpp Multi-Device Splits]] — where the drafter's KV actually lands
- where the drafter's KV actually lands
- [[frank-dragoon]] and [[frank-zealot]] — the single-GPU env drop-in A/B vehicle