docs(vault): audit tranche 1 — one-owner dedupe of the observability cluster, dead-shape numbers destroyed
Purpose-test pass over the audit findings (every verdict re-verified against the file before touching it; three scout findings were hallucinated — a nonexistent filename, my own changelog line, and sanctioned Eras rows): - Observability Provisioning: destroyed the 63-line 'Current delivered state' block — chain topology/relay/delivery restates [[Observability]] and [[Alertmanager]] (those are the owners). Its three sole-custody laws MOVED to Stale Signal Audit as 'Three rules that outlive the incident': down-beside-healthy -> read lastError; Prometheus alert state in process vs Alertmanager's --storage.path persistence; candidate-arm metrics are never fleet state. - RDNA4 MoE VRAM budget: kept the laws (registry over-prices the pool; never reuse a B/token ratio across geometries; pin pool bytes floor-sized), era-labelled the rates, destroyed the measured ladder table for the superseded shape-G/bf16 rungs. - Stale Signal Audit: new rules section + Provisioning Related link. check-okf 109/0, vault links 0 unresolved, docs seams green.
This commit is contained in:
1 parent
c6e2887f94
commit
a61308d72e
3 files changed
+30
-89
No files matched your search
@@ -70,12 +70,12 @@ back toward 52. Check `VLLM_CUDAGRAPH_*` actually present in
|
||||
|
||||
`--kv-cache-memory-bytes` does NOT appear in the arena arithmetic — the plan
|
||||
reserves the registry price, not the actual pool. Raising the pool consumes
|
||||
real free VRAM without growing the arena; the registry over-prices the pool
|
||||
(9190 B/token planned vs 7,891.8 measured at 200704 fp8 — 7,701 at 262144;
|
||||
**never reuse a B/token ratio across geometries**, and subtract ~3.87 MB
|
||||
block-alignment loss when converting a token target to bytes). The ask lands
|
||||
block-aligned: 3,166,500,000 B → 401,408 tokens; verify with
|
||||
`GPU KV cache size: N tokens, Maximum concurrency … X.Xx`.
|
||||
real free VRAM without growing the arena; the registry OVER-PRICES the pool
|
||||
(era G measured ~14 % under the planned B/token — the gap is the lesson, not
|
||||
the number; **never reuse a B/token ratio across geometries**), and there is a
|
||||
~4 MB block-alignment loss to subtract when converting a token target to
|
||||
bytes. The ask lands block-aligned: verify against the engine's own
|
||||
`GPU KV cache size: N tokens, Maximum concurrency … X.Xx` line at boot.
|
||||
|
||||
**Driver-reported free VRAM after hours of traffic is NOT allocatable
|
||||
arena.** It is the estimator gap: the registry over-prices KV and charges
|
||||
@@ -117,11 +117,13 @@ done, and read post-boot free VRAM per rank (`rocm-smi --showmeminfo vram`)
|
||||
**KV accounting — pin the pool BYTES, floor-sized:** set
|
||||
`--kv-cache-memory-bytes` PER RANK to the contract floor
|
||||
(`seqs × MAXLEN × measured B/token`), not the native window — extra pool bytes
|
||||
come out of the arena/headroom and buy nothing. Measured B/token on this
|
||||
hybrid: **7,701 fp8 / 13,839 bf16** (decomposed: attention 6,138/12,276 +
|
||||
dtype-independent mamba/side 1,563). Measured rates are shape-dependent —
|
||||
7,891.8 at the 200704×3 shape. Capacity does NOT scale with `SEQS`: the
|
||||
planner's KV reserve is for ONE max-model-len request.
|
||||
come out of the arena/headroom and buy nothing. The B/token rate is a property
|
||||
of the GEOMETRY: era G (3 × 200,704 fp8, vLLM fork) measured **7,891.8**;
|
||||
the radiance-era hybrid measured ~7,700 fp8 / ~13,800 bf16 at its own shape.
|
||||
Read your live rate from the boot ledger every time; the decomposition (a
|
||||
dtype-independent mamba/side stream rides on top of the attention KV) is why
|
||||
the ratio moves. Capacity does NOT scale with `SEQS`: the planner's KV reserve
|
||||
is for ONE max-model-len request.
|
||||
|
||||
**Parallelism ceilings you cannot flag past:**
|
||||
|
||||
@@ -179,18 +181,6 @@ Cutover order: ladder → winner passes and serves → repo env + seam + harness
|
||||
(`git stash -u`) if the tree is dirty → deploy (one restart) → verify health
|
||||
+ engine flags in the log.
|
||||
|
||||
**Measured ladder on this box (200704×3, fp8 KV, precise memory):**
|
||||
|
||||
| shape | util | graphs | arena | result |
|
||||
|---|---|---|---|---|
|
||||
| E | 0.98 | 0.55 | 17.22 GiB / 7,522 slots | boot ok |
|
||||
| F | 0.982 | 0.32 | 17.52 GiB / 7,653 slots | boot ok |
|
||||
| 1.0 | 1.00 | 0.07 | — | UNBOOTABLE (worker-init ValueError, both ranks) |
|
||||
| G (deployed) | 0.982 | 0.07 | 17.77 GiB / 7,762 slots | KV 401,408 tokens / 2.00x; 0.89 GiB real free/card held through a live 4,096-token generation at ~38–39 tok/s |
|
||||
|
||||
bf16 ceilings at util 0.98: 3 seqs → 241,664 passes / 245,760 and 262,144 OOM
|
||||
at graph capture; 6 seqs fp8 → 10.70 GiB ask with 9.18 GiB free (fails) —
|
||||
~207k per context max.
|
||||
|
||||
## 7. Arena audit procedure (when asked for "more experts")
|
||||
|
||||
|
||||
Reference in new issue
Block a user