From a8e74e726685ffac5f998c7ead71d274f252e361 Mon Sep 17 00:00:00 2001 From: Frank Date: Thu, 29 Oct 2026 06:09:49 -0700 Subject: [PATCH] live: heartbeat sync 2026-10-29T06:09:49-07:00 (Refs #11) --- .../radiance-step-shape-ledger-audit/SKILL.md | 37 +++++++++++++++++++ .../state.json | 2 +- .../state.json | 2 +- .../.local/state/noctalia/settings.toml | 6 +-- 4 files changed, 42 insertions(+), 5 deletions(-) create mode 100644 agent/managed-skills/radiance-step-shape-ledger-audit/SKILL.md diff --git a/agent/managed-skills/radiance-step-shape-ledger-audit/SKILL.md b/agent/managed-skills/radiance-step-shape-ledger-audit/SKILL.md new file mode 100644 index 0000000..61945e3 --- /dev/null +++ b/agent/managed-skills/radiance-step-shape-ledger-audit/SKILL.md @@ -0,0 +1,37 @@ +--- +name: radiance-step-shape-ledger-audit +description: "Price a radiance serving-shape change (MNBT / max-num-seqs / window) from the engine's own boot ledger instead of a comment, and catch a mode that carries a private step." +--- + +Use when a radiance endpoint or campaign rig boots with a step/shape that differs from the one already proven, or refuses with `E startup stopped: out of memory` while the container image and env are unchanged. + +## The law + +The step's **activation arena is charged to the card**, not to the disk tier. `--max-num-batched-tokens` sizes it (see `vram_budget.cpp:117`, the engine's own fix text: "lower --max-num-batched-tokens (the arena ...)"), so a wider step is taken out of the weight tiers and the KV cache. A comment claiming a wide step is "free throughput" is not evidence. Never let one mode of a multi-mode serve script carry its own `--max-num-seqs` or step; derive both from the env file that owns them (for frank-archon: `scripts/frank-archon.env` → `FRANK_ARCHON_MAX_SEQS`, `FRANK_ARCHON_MAX_BATCHED_TOKENS`, `FRANK_ARCHON_HOST_POOL_MIB`, `FRANK_ARCHON_GPU_HEADROOM_MIB`). + +## The audit (one ssh call, two logs) + +Radiance writes the ledger to the file named beside the serve script (`/home/dank/logs/qwen38_bf16_.log`, `/home/dank/logs/frank_archon.log`) — **not** the journal. Extract, per boot: + +``` +for f in ; do L=/home/dank/logs/qwen38_bf16_$f.log + for k in "--max-model-len" "--max-num-seqs" "--max-num-batched-tokens" \ + "--host-pool-mib" "--tp " "--kv-cache-dtype" "--placement"; do + grep -m1 -E "^I +\\${k%}" "$L" | head -1; done + grep -m1 "activation arena" "$L"; grep -m1 "claimable" "$L" + grep -m1 "vram_kv" "$L"; grep -m1 "weights by tier" "$L" + grep -m1 "full-length sessions" "$L" +done +``` + +Compare row by row: `activation arena` → `claimable` → `vram_kv` → `weights by tier` (VRAM vs container-on-disk). The refusal line names the shape it could not back, e.g. `kv cache 271.25 MiB ... 0% backing of the 8 x 32768 worst case: 0.0 full-length sessions`. + +## Reading the rows + +- Arena grows with MNBT and is subtracted before weights/KV are funded (measured: 1,024 → 504 MiB arena, 30.87 GiB claimable; 65,536 → 10.65 GiB arena, 16.85 GiB claimable, experts squeezed to 10.98 GiB, 56 GiB of weights stranded on the disk tier). +- `vram_kv` below one full-length sequence for the requested seats ⇒ OOM refusal, even when a narrower shape at the same window booted. +- `expert prefetch 0 B` / "no lender" means the engine already gave the expert slab everything the arena left. + +## Then + +Fix by inheriting the proven numbers, delete the per-mode literals (no shim, no `FRANK_*_MAX_SEQS` alias), update the mode's own comment so the false premise is destroyed rather than re-argued, and prove with `