docs(vault): audit tranche 1 — one-owner dedupe of the observability cluster, dead-shape numbers destroyed

Purpose-test pass over the audit findings (every verdict re-verified
against the file before touching it; three scout findings were hallucinated
— a nonexistent filename, my own changelog line, and sanctioned Eras rows):

- Observability Provisioning: destroyed the 63-line 'Current delivered
  state' block — chain topology/relay/delivery restates [[Observability]]
  and [[Alertmanager]] (those are the owners). Its three sole-custody laws
  MOVED to Stale Signal Audit as 'Three rules that outlive the incident':
  down-beside-healthy -> read lastError; Prometheus alert state in process
  vs Alertmanager's --storage.path persistence; candidate-arm metrics are
  never fleet state.
- RDNA4 MoE VRAM budget: kept the laws (registry over-prices the pool;
  never reuse a B/token ratio across geometries; pin pool bytes
  floor-sized), era-labelled the rates, destroyed the measured ladder
  table for the superseded shape-G/bf16 rungs.
- Stale Signal Audit: new rules section + Provisioning Related link.

check-okf 109/0, vault links 0 unresolved, docs seams green.
This commit is contained in:
Frank committed 2026-10-21 02:35:44 -07:00
1 parent c6e2887f94
commit a61308d72e
3 files changed
+30 -89

No files matched your search

@@ -70,12 +70,12 @@ back toward 52. Check `VLLM_CUDAGRAPH_*` actually present in
`--kv-cache-memory-bytes` does NOT appear in the arena arithmetic — the plan
reserves the registry price, not the actual pool. Raising the pool consumes
real free VRAM without growing the arena; the registry over-prices the pool
(9190 B/token planned vs 7,891.8 measured at 200704 fp8 — 7,701 at 262144;
**never reuse a B/token ratio across geometries**, and subtract ~3.87 MB
block-alignment loss when converting a token target to bytes). The ask lands
block-aligned: 3,166,500,000 B → 401,408 tokens; verify with
`GPU KV cache size: N tokens, Maximum concurrency … X.Xx`.
real free VRAM without growing the arena; the registry OVER-PRICES the pool
(era G measured ~14 % under the planned B/token — the gap is the lesson, not
the number; **never reuse a B/token ratio across geometries**), and there is a
~4 MB block-alignment loss to subtract when converting a token target to
bytes. The ask lands block-aligned: verify against the engine's own
`GPU KV cache size: N tokens, Maximum concurrency … X.Xx` line at boot.
**Driver-reported free VRAM after hours of traffic is NOT allocatable
arena.** It is the estimator gap: the registry over-prices KV and charges
@@ -117,11 +117,13 @@ done, and read post-boot free VRAM per rank (`rocm-smi --showmeminfo vram`)
**KV accounting — pin the pool BYTES, floor-sized:** set
`--kv-cache-memory-bytes` PER RANK to the contract floor
(`seqs × MAXLEN × measured B/token`), not the native window — extra pool bytes
come out of the arena/headroom and buy nothing. Measured B/token on this
hybrid: **7,701 fp8 / 13,839 bf16** (decomposed: attention 6,138/12,276 +
dtype-independent mamba/side 1,563). Measured rates are shape-dependent —
7,891.8 at the 200704×3 shape. Capacity does NOT scale with `SEQS`: the
planner's KV reserve is for ONE max-model-len request.
come out of the arena/headroom and buy nothing. The B/token rate is a property
of the GEOMETRY: era G (3 × 200,704 fp8, vLLM fork) measured **7,891.8**;
the radiance-era hybrid measured ~7,700 fp8 / ~13,800 bf16 at its own shape.
Read your live rate from the boot ledger every time; the decomposition (a
dtype-independent mamba/side stream rides on top of the attention KV) is why
the ratio moves. Capacity does NOT scale with `SEQS`: the planner's KV reserve
is for ONE max-model-len request.
**Parallelism ceilings you cannot flag past:**
@@ -179,18 +181,6 @@ Cutover order: ladder → winner passes and serves → repo env + seam + harness
(`git stash -u`) if the tree is dirty → deploy (one restart) → verify health
+ engine flags in the log.
**Measured ladder on this box (200704×3, fp8 KV, precise memory):**
| shape | util | graphs | arena | result |
|---|---|---|---|---|
| E | 0.98 | 0.55 | 17.22 GiB / 7,522 slots | boot ok |
| F | 0.982 | 0.32 | 17.52 GiB / 7,653 slots | boot ok |
| 1.0 | 1.00 | 0.07 | — | UNBOOTABLE (worker-init ValueError, both ranks) |
| G (deployed) | 0.982 | 0.07 | 17.77 GiB / 7,762 slots | KV 401,408 tokens / 2.00x; 0.89 GiB real free/card held through a live 4,096-token generation at ~38–39 tok/s |
bf16 ceilings at util 0.98: 3 seqs → 241,664 passes / 245,760 and 262,144 OOM
at graph capture; 6 seqs fp8 → 10.70 GiB ask with 9.18 GiB free (fails) —
~207k per context max.
## 7. Arena audit procedure (when asked for "more experts")