docs(vault): destroy the dead weight — 53 pages and the change diary
The vault was 155 pages / 31,210 lines, most of it measurement ledgers for payloads that no longer run. A throughput figure for a runtime nobody operates is not a record: AI moves faster than the vault, so the number is wrong before anyone reads it, and a wrong number in a runbook gets believed. Per the new purpose test in `AGENTS.md` → *Documentation Policy*, it is destroyed. Gone (`.AGENTTRASH/20261019-destruction/`, KILL-LIST.txt is the manifest): - the `frank-medium` endpoint and its five era pages, `frank-npu`, FastFlowLM, the ikllama/kingjones/dealign/frank-redo eras, the ling/swift15/freetoken/ubatch campaign pages, qwen3.8 spec-matrix and strix-optimizations, the two "Running Qwen3.6 …" pages; - the whole `Pipeline Parallelism/` folder (no live script passes a pipeline flag) and the multi-device/x4-pair/hybrid/offload experiment pages; - every vLLM page — no vLLM endpoint exists on this fleet; - `LLM/deployment.md` (`Architecture/Deployment Flow.md` is the test-referenced owner) and the external surveys (SGLang, Lemonade, llama.cpp forks/diff, self-hosted app census, the TTS survey) that were obsolete by construction; - `log.md`, 2,380 lines. Git holds the chronology; a causal law belongs to the page that owns the thing it governs, never to a diary. Survivors: 101 pages, 17,542 lines, 0 OKF violations, 0 unresolved links. Eras tables keep the window, payload, runtime, reason and rollback pointer; the rate columns are blanked. The live shapes stay verbatim — those numbers are what the deployer and `models.yml` consume.
This commit is contained in:
1 parent
8775e64ed0
commit
327e78b950
126 files changed
+443
-14157
No files matched your search
@@ -93,7 +93,7 @@ of bytes), so it is tempting. Before proposing it:
|
||||
(compare GGUF `tokenizer.ggml.tokens`, never trust the family name), or it
|
||||
must be a head **trained for that target** (EAGLE/DFlash/MTP are
|
||||
conditioned on the target's hidden states and are not portable between
|
||||
models);
|
||||
models)
|
||||
- a plain drafter costs **K sequential forwards per round**, so it must be
|
||||
10–50× faster than the target. Break-even with `r = t_draft/t_target`:
|
||||
`rate_ratio = A/(K·r + 1)`, `A = (1−p^(K+1))/(1−p)`; at `r ≈ 0.4–0.5`
|
||||
@@ -107,10 +107,10 @@ of bytes), so it is tempting. Before proposing it:
|
||||
(draft gets its own device list) plus `-ngld`/`-otd`/`-ctkd -ctvd`.
|
||||
|
||||
The llama.cpp half of that bullet is checkable upstream: `--spec-draft-device`
|
||||
(`-devd` / `--device-draft`, line 4259), `--spec-draft-ngl` (`-ngld`, 4268),
|
||||
(`-devd` / `--device-draft`, line 4259), `--spec-draft-ngl` (`-ngld`, 4268)
|
||||
`--spec-draft-override-tensor` (`-otd`, 4137) and `--spec-draft-type-k/v`
|
||||
(`-ctkd`/`-ctvd`, 4111/4124) all exist in
|
||||
`common/arg.cpp` ([source](https://raw.githubusercontent.com/ggml-org/llama.cpp/master/common/arg.cpp),
|
||||
`common/arg.cpp` ([source](https://raw.githubusercontent.com/ggml-org/llama.cpp/master/common/arg.cpp)
|
||||
read 2026-10-18).
|
||||
|
||||
## 6. Expressing a split (llama.cpp, the usual vehicle)
|
||||
@@ -122,8 +122,8 @@ read 2026-10-18).
|
||||
|
||||
- buffer types match by name at load (`ggml_backend_buft_name`), so device
|
||||
names work as `-ot` values;
|
||||
- `-cmoe`/`-ncmoe` are **CPU-only** (`--cpu-moe` / `--n-cpu-moe`,
|
||||
`common/arg.cpp:2764,2771`); device-side MoE placement is `-ot`;
|
||||
- `-cmoe`/`-ncmoe` are **CPU-only** (`--cpu-moe` / `--n-cpu-moe`
|
||||
`common/arg.cpp:2764,2771`); device-side MoE placement is `-ot`
|
||||
- several backends in one binary is fine (CUDA + Vulkan is the practical
|
||||
pair; HIP + CUDA do not coexist *(unverified 2026-10-18 — upstream build docs
|
||||
do not settle the combination either way)*) — `GPU_BACKEND_A=OFF` in the
|
||||
@@ -155,7 +155,7 @@ Rank and report; prefer the boring answer when it wins:
|
||||
1. **Two independent endpoints** (zero shared traffic) — often the real
|
||||
answer to "what's a useful combination", especially over a narrow link.
|
||||
2. **One process, weights-resident, KV co-located with attention on the
|
||||
fast pool** — promising and quantifiable (+% decode at long context);
|
||||
fast pool** — promising and quantifiable (+% decode at long context)
|
||||
needs a dual-backend build and a baseline measurement.
|
||||
3. **Hot-expert cache / per-token weight streaming** — affordable only if
|
||||
the miss set per token is small (a few experts; check
|
||||
@@ -169,5 +169,5 @@ Rank and report; prefer the boring answer when it wins:
|
||||
## Related
|
||||
|
||||
- [[PCIe Link and Atomics]] — the link and collective walls underneath
|
||||
- [[Llama Cpp Multi-Device Splits]] — the deployed split shapes
|
||||
- the deployed split shapes
|
||||
- [[Llama Cpp Serving Sizing]] — KV and pool arithmetic
|
||||
Reference in new issue
Block a user