live: heartbeat sync 2026-10-29T06:09:49-07:00 (Refs #11)
This commit is contained in:
1 parent
f859230ff6
commit
a8e74e7266
4 files changed
+42
-5
No files matched your search
@@ -0,0 +1,37 @@
|
||||
---
|
||||
name: radiance-step-shape-ledger-audit
|
||||
description: "Price a radiance serving-shape change (MNBT / max-num-seqs / window) from the engine's own boot ledger instead of a comment, and catch a mode that carries a private step."
|
||||
---
|
||||
|
||||
Use when a radiance endpoint or campaign rig boots with a step/shape that differs from the one already proven, or refuses with `E startup stopped: out of memory` while the container image and env are unchanged.
|
||||
|
||||
## The law
|
||||
|
||||
The step's **activation arena is charged to the card**, not to the disk tier. `--max-num-batched-tokens` sizes it (see `vram_budget.cpp:117`, the engine's own fix text: "lower --max-num-batched-tokens (the arena ...)"), so a wider step is taken out of the weight tiers and the KV cache. A comment claiming a wide step is "free throughput" is not evidence. Never let one mode of a multi-mode serve script carry its own `--max-num-seqs` or step; derive both from the env file that owns them (for frank-archon: `scripts/frank-archon.env` → `FRANK_ARCHON_MAX_SEQS`, `FRANK_ARCHON_MAX_BATCHED_TOKENS`, `FRANK_ARCHON_HOST_POOL_MIB`, `FRANK_ARCHON_GPU_HEADROOM_MIB`).
|
||||
|
||||
## The audit (one ssh call, two logs)
|
||||
|
||||
Radiance writes the ledger to the file named beside the serve script (`/home/dank/logs/qwen38_bf16_<mode>.log`, `/home/dank/logs/frank_archon.log`) — **not** the journal. Extract, per boot:
|
||||
|
||||
```
|
||||
for f in <proven_mode> <failing_mode>; do L=/home/dank/logs/qwen38_bf16_$f.log
|
||||
for k in "--max-model-len" "--max-num-seqs" "--max-num-batched-tokens" \
|
||||
"--host-pool-mib" "--tp " "--kv-cache-dtype" "--placement"; do
|
||||
grep -m1 -E "^I +\\${k%}" "$L" | head -1; done
|
||||
grep -m1 "activation arena" "$L"; grep -m1 "claimable" "$L"
|
||||
grep -m1 "vram_kv" "$L"; grep -m1 "weights by tier" "$L"
|
||||
grep -m1 "full-length sessions" "$L"
|
||||
done
|
||||
```
|
||||
|
||||
Compare row by row: `activation arena` → `claimable` → `vram_kv` → `weights by tier` (VRAM vs container-on-disk). The refusal line names the shape it could not back, e.g. `kv cache 271.25 MiB ... 0% backing of the 8 x 32768 worst case: 0.0 full-length sessions`.
|
||||
|
||||
## Reading the rows
|
||||
|
||||
- Arena grows with MNBT and is subtracted before weights/KV are funded (measured: 1,024 → 504 MiB arena, 30.87 GiB claimable; 65,536 → 10.65 GiB arena, 16.85 GiB claimable, experts squeezed to 10.98 GiB, 56 GiB of weights stranded on the disk tier).
|
||||
- `vram_kv` below one full-length sequence for the requested seats ⇒ OOM refusal, even when a narrower shape at the same window booted.
|
||||
- `expert prefetch 0 B` / "no lender" means the engine already gave the expert slab everything the arena left.
|
||||
|
||||
## Then
|
||||
|
||||
Fix by inheriting the proven numbers, delete the per-mode literals (no shim, no `FRANK_*_MAX_SEQS` alias), update the mode's own comment so the false premise is destroyed rather than re-argued, and prove with `<script> <mode> --dry-run` that the argv matches the proven mode byte for byte.
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"v": 1,
|
||||
"lastConsolidatedAt": 1793275356963,
|
||||
"lastConsolidatedAt": 1793279316979,
|
||||
"lastResult": {
|
||||
"at": 1793218183375,
|
||||
"sessions": 1,
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
{
|
||||
"v": 1,
|
||||
"lastConsolidatedAt": 1793275515233
|
||||
"lastConsolidatedAt": 1793279115280
|
||||
}
|
||||
Reference in new issue
Block a user