docs(vault): destroy the dead weight — 53 pages and the change diary
The vault was 155 pages / 31,210 lines, most of it measurement ledgers for payloads that no longer run. A throughput figure for a runtime nobody operates is not a record: AI moves faster than the vault, so the number is wrong before anyone reads it, and a wrong number in a runbook gets believed. Per the new purpose test in `AGENTS.md` → *Documentation Policy*, it is destroyed. Gone (`.AGENTTRASH/20261019-destruction/`, KILL-LIST.txt is the manifest): - the `frank-medium` endpoint and its five era pages, `frank-npu`, FastFlowLM, the ikllama/kingjones/dealign/frank-redo eras, the ling/swift15/freetoken/ubatch campaign pages, qwen3.8 spec-matrix and strix-optimizations, the two "Running Qwen3.6 …" pages; - the whole `Pipeline Parallelism/` folder (no live script passes a pipeline flag) and the multi-device/x4-pair/hybrid/offload experiment pages; - every vLLM page — no vLLM endpoint exists on this fleet; - `LLM/deployment.md` (`Architecture/Deployment Flow.md` is the test-referenced owner) and the external surveys (SGLang, Lemonade, llama.cpp forks/diff, self-hosted app census, the TTS survey) that were obsolete by construction; - `log.md`, 2,380 lines. Git holds the chronology; a causal law belongs to the page that owns the thing it governs, never to a diary. Survivors: 101 pages, 17,542 lines, 0 OKF violations, 0 unresolved links. Eras tables keep the window, payload, runtime, reason and rollback pointer; the rate columns are blanked. The live shapes stay verbatim — those numbers are what the deployer and `models.yml` consume.
This commit is contained in:
1 parent
8775e64ed0
commit
327e78b950
126 files changed
+443
-14157
No files matched your search
@@ -100,7 +100,7 @@ The DFlash drafter proposes, then ngram-mod runs as an additional implementation
|
||||
`--spec-type` takes a comma list, and the gate is `ctx_dft` in
|
||||
`common/speculative.cpp` (`common_speculative_init_result`): **draftless**
|
||||
types (the ngram family — simple, map-k, map-k4v, mod, cache) are added
|
||||
unconditionally, while **context-backed** types (`draft-mtp`, `draft-dflash`,
|
||||
unconditionally, while **context-backed** types (`draft-mtp`, `draft-dflash`
|
||||
`draft-dspark`, `draft-eagle3`) all require a draft CONTEXT. Consequences:
|
||||
|
||||
- A chain may hold ONE context-backed drafter plus any number of ngram types.
|
||||
@@ -201,7 +201,7 @@ Speculative decoding is lossless under rejection sampling: the target verifies e
|
||||
On the mid-August 2026 Dankclaw shape `draft-dflash` was the benchmarked winner
|
||||
at 4k — 20.2 t/s, against 16.9 with ngram layered on and 8.7 ngram-only; those
|
||||
three figures are that retired llama.cpp era's property ([[frank-dragoon]] owns
|
||||
the era and its shape). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see [[deployment]]).
|
||||
the era and its shape). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see ).
|
||||
|
||||
## Draft Models
|
||||
|
||||
@@ -257,7 +257,7 @@ DSpark took `ROCm error: an illegal memory access` inside the shared draft
|
||||
context with two concurrent 16k prompts; DFlash corrupted multi-slot into
|
||||
premature EOS and was retired first. The timeline, the crash class, the
|
||||
"concurrency is the trigger, not size" finding and the `PARALLEL=1` workaround
|
||||
are on [[qwen3.8-spec-matrix]]; the shape they were measured on is
|
||||
are on ; the shape they were measured on is
|
||||
[[frank-dragoon]]'s mid-August 2026 era.
|
||||
|
||||
## A/B protocol on a live llama.cpp endpoint
|
||||
@@ -303,7 +303,7 @@ Protocol:
|
||||
|
||||
Read the right counters — per request `timings.draft_n` / `draft_n_accepted` /
|
||||
`predicted_per_second`; cumulative `/metrics`
|
||||
`llamacpp:spec_decode_num_drafts_total` / `..._accepted_tokens_total`;
|
||||
`llamacpp:spec_decode_num_drafts_total` / `..._accepted_tokens_total`
|
||||
per-task `draft acceptance = ... (N accepted / M generated)`. `draft_n == 0`
|
||||
while tokens flow = the proposer is NOT engaged, whatever the flags say. A
|
||||
zero-draft config is a useful negative control: it proves the other config's
|
||||
@@ -367,7 +367,7 @@ states, so a dense-27B DFlash2 is useless for a 512-expert MoE trunk.
|
||||
|
||||
**Step 4 — acceptance reality checks.** Treat 40–60 % as the honest band for a
|
||||
matched pair; the arm that anchors the bottom of it on this fleet is the
|
||||
depth-7 `draft-dflash` row of [[frank-medium-llamacpp-dflash2]]'s spec table
|
||||
depth-7 `draft-dflash` row of 's spec table
|
||||
(accepted share, rate and VRAM for every depth are in that table). Cheapest
|
||||
proxy for `p` at temp 0 when both models are greedy and share a tokenizer:
|
||||
greedy continuation agreement on identical prompts, two API calls, no weights
|
||||
@@ -381,10 +381,10 @@ drafting.
|
||||
**Step 5 — check the attachment surface BEFORE the arithmetic.** Closed
|
||||
engines refuse external drafters by design: halogen-flash-server's
|
||||
`DRAFTER_CAP` gates `dflash2` on the loaded checkpoint and an unavailable
|
||||
name is a deliberate 400 (a silent downgrade would make comparisons lie);
|
||||
name is a deliberate 400 (a silent downgrade would make comparisons lie)
|
||||
there is no device routing in its loopback token protocol, so a drafter can
|
||||
never sit on another card. Pure llama.cpp DOES route the draft:
|
||||
`--spec-draft-device/-devd` (default follows `--device`), `-ngld`,
|
||||
`--spec-draft-device/-devd` (default follows `--device`), `-ngld`
|
||||
`-otd`, `-ctkd/-ctvd`, `--spec-draft-n-max` (default 3), `--spec-draft-p-min`.
|
||||
One process sees only the devices its backends were built with — CUDA+Vulkan
|
||||
is the combination that shows a CUDA card AND an AMD iGPU (HIP+CUDA cannot
|
||||
@@ -411,20 +411,16 @@ measurement that would settle it.
|
||||
|
||||
- Speculative decoding helps low-QPS, memory-bound, latency-focused workloads. It helps less at high QPS.
|
||||
- DFlash needs a non-causal attention backend; the ROCm flash_attn backend must be validated before production use.
|
||||
- The old `--swap-space` flag is retired; the live CPU-KV-offload knob pair is owned by [[RAM CPU Offloading in VLLM]].
|
||||
-The old `--swap-space` flag is retired; the live CPU-KV-offload knob pair is owned by .
|
||||
- `--speculative-config` is the preferred CLI. Legacy `--spec-method`, `--spec-model`, and `--spec-tokens` flags still exist.
|
||||
- Use `rejection_sample_method` default `strict` for lossless output.
|
||||
|
||||
## See Also
|
||||
|
||||
- [[Quantization by Hardware Architecture]]
|
||||
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]]
|
||||
- [[Running Qwen3.6 27B FP8 on Dankclaw]]
|
||||
- [[Dankclaw System Specs]]
|
||||
- [[Dankserver System Specs]]
|
||||
- [[VLLM Overview]]
|
||||
- [[deployment]]
|
||||
- [[GGUF Compatibility]] — header parsing and custom tensor-type lock-in
|
||||
- [[Llama Cpp Serving Sizing]] — slot-pool and drafter KV sizing arithmetic
|
||||
- [[Llama Cpp Multi-Device Splits]] — where the drafter's KV actually lands
|
||||
- where the drafter's KV actually lands
|
||||
- [[frank-dragoon]] and [[frank-zealot]] — the single-GPU env drop-in A/B vehicle
|
||||
Reference in new issue
Block a user