docs(vault): destroy the dead weight — 53 pages and the change diary
The vault was 155 pages / 31,210 lines, most of it measurement ledgers for payloads that no longer run. A throughput figure for a runtime nobody operates is not a record: AI moves faster than the vault, so the number is wrong before anyone reads it, and a wrong number in a runbook gets believed. Per the new purpose test in `AGENTS.md` → *Documentation Policy*, it is destroyed. Gone (`.AGENTTRASH/20261019-destruction/`, KILL-LIST.txt is the manifest): - the `frank-medium` endpoint and its five era pages, `frank-npu`, FastFlowLM, the ikllama/kingjones/dealign/frank-redo eras, the ling/swift15/freetoken/ubatch campaign pages, qwen3.8 spec-matrix and strix-optimizations, the two "Running Qwen3.6 …" pages; - the whole `Pipeline Parallelism/` folder (no live script passes a pipeline flag) and the multi-device/x4-pair/hybrid/offload experiment pages; - every vLLM page — no vLLM endpoint exists on this fleet; - `LLM/deployment.md` (`Architecture/Deployment Flow.md` is the test-referenced owner) and the external surveys (SGLang, Lemonade, llama.cpp forks/diff, self-hosted app census, the TTS survey) that were obsolete by construction; - `log.md`, 2,380 lines. Git holds the chronology; a causal law belongs to the page that owns the thing it governs, never to a diary. Survivors: 101 pages, 17,542 lines, 0 OKF violations, 0 unresolved links. Eras tables keep the window, payload, runtime, reason and rollback pointer; the rate columns are blanked. The live shapes stay verbatim — those numbers are what the deployer and `models.yml` consume.
This commit is contained in:
1 parent
8775e64ed0
commit
327e78b950
126 files changed
+443
-14157
No files matched your search
@@ -21,7 +21,7 @@ the cost; the problem is per-token expert movement and KV/attention tuning is
|
||||
ruled out entirely. A genuinely context-bound server shows a clear downward
|
||||
slope. Record cold vs warm separately — a small gap means the cache is
|
||||
working. The canonical flat run (fp8 expert-offload shape, a few hundred to
|
||||
sixteen thousand tokens of context) is in [[Tcclaviger Fork Ops]]'s profile.
|
||||
sixteen thousand tokens of context) is in 's profile.
|
||||
|
||||
## 2. Duty-cycle arithmetic (the "GPU not fully utilized" answer)
|
||||
|
||||
@@ -86,7 +86,7 @@ Worked example (GLM-5.3-Flash, 2×GPU, offload, decode 9.9 tok/s):
|
||||
|
||||
No amount of link arithmetic would have produced those two conclusions: the
|
||||
suspect was 1 %, and the mass was pointwise glue from an uncompiled trunk —
|
||||
whose own boot log names the cause (`FULL_DECODE_ONLY` override). It also
|
||||
whose own boot log names the cause (`FULL_DECODE_ONLY` override). It
|
||||
predicted a negative result in advance (spec decode cannot help a shape whose
|
||||
per-token cost scales with the token count) which matched an unexplained flat
|
||||
MTP measurement.
|
||||
@@ -117,7 +117,7 @@ per-request telemetry — it measures the real workload and costs nothing.
|
||||
- One `max_tokens=200` request is production traffic and fine; anything that
|
||||
pins the card is a bench → stop-and-mask first. Check WHICH instance you
|
||||
are reading (`systemctl status | head -12`; task ids jump across
|
||||
restarts). The engine's own ledger (`KV self size`, `n_slots`,
|
||||
restarts). The engine's own ledger (`KV self size`, `n_slots`
|
||||
`n_ctx_slot`) outranks your model of it.
|
||||
|
||||
## 5. The fork's data path (before blaming the link)
|
||||
@@ -130,7 +130,7 @@ tokens, so the step can silently cross the threshold onto the slow path as
|
||||
draft depth grows. Check the threshold's premise: a default tuned at
|
||||
12.4 MiB/expert does not transfer to 2.344 MiB/expert, where copy-in can be
|
||||
several times faster. TunableOp ghost-skip: `CLAV_TUNABLEOP_SWEEP=1` alone
|
||||
does NOTHING under expert offload (`CLAV_TUNABLEOP_WITH_OFFLOAD=1` required);
|
||||
does NOTHING under expert offload (`CLAV_TUNABLEOP_WITH_OFFLOAD=1` required)
|
||||
check for the skip line before concluding autotuning ran. Gather-geometry
|
||||
knobs read at import time → each arm is a full reload.
|
||||
|
||||
@@ -189,7 +189,7 @@ analyze; `--analyze-only`).
|
||||
Measure with a **sequential-turn client**, not only a batch benchmark — a
|
||||
batch of distinct prompts parallelises prefill across replicas and hides the
|
||||
problem. The two failure modes: round-robin loses every other cache hit
|
||||
(90k-token contexts: warm-turn TTFT 2676 ms round-robin vs 1096 ms pinned);
|
||||
(90k-token contexts: warm-turn TTFT 2676 ms round-robin vs 1096 ms pinned)
|
||||
pinning-everything loses a card (16 requests: 18.6 s spread vs 30.1 s
|
||||
pinned). **The rule that gets both: route by request shape** — a request
|
||||
carrying more than one message is a continuation (agent loop re-sending its
|
||||
@@ -209,13 +209,13 @@ is not a per-request verdict (probe directly: same prompt twice to an idle
|
||||
replica — 22.3 s cold → 1.4 s warm); salt corpora; record sample sizes;
|
||||
`vllm bench serve` needs `--tokenizer <local path>`; verify metric names
|
||||
against the live endpoint before dashboards. Test shape: drive the routing
|
||||
function directly with synthetic bodies (stdlib-only policy module);
|
||||
function directly with synthetic bodies (stdlib-only policy module)
|
||||
conversation identity is the FIRST N KiB of prompt — a shorter tested prefix
|
||||
hashes differently per turn and pinning silently never happens.
|
||||
|
||||
## Related
|
||||
|
||||
- [[LLM Bench Windows and Measurement]] — the window discipline and usage-counter law
|
||||
- the window discipline and usage-counter law
|
||||
- [[PCIe Link and Atomics]] — link probing and the streaming-vs-compute ceiling
|
||||
- [[RDNA4 MoE Expert Offload VRAM Budget]] — the plan behind duty-cycle terms
|
||||
- [[vLLM Porting and Compile Triage]] — the compile fix for the pointwise-glue bound
|
||||
- the compile fix for the pointwise-glue bound
|
||||
Reference in new issue
Block a user