command-center/agent/managed-skills/radiance-kv-cache-resize/SKILL.md

name, description

name	description
radiance-kv-cache-resize	Resize a radiance endpoint's KV cache/VRAM: the resolver's funding formula, which knob actually moves the carve, the pinned-RAM ledger to recompute, and the boot-ledger lines that referee the result.

Method

• The endpoint is sized by ONE env file (scripts/<svc>.env) plus the serve script; the boot ledger prints the split before allocating anything: VRAM budget, per card → claimable → elastic → experts / kv cache, then kv: the cache can grow to X a card -- N tokens of context, M sessions at W.
• --vram-weights-mib / --vram-kv-mib no longer exist (1.8.1). The engine REFUSES to start on them (core/config.cpp), so this skill has no direct KV knob; the budget resolver derives everything from the cache bound (--max-num-seqs x --max-model-len), --gpu-headroom-mib and --host-pool-mib.

The funding formula (radiance 1.8.1, source-cited — re-read the pinned tree before trusting it against another build)

capacity = card_total - already_held - drift - headroom - arena - step_staging - hw_queues - prefetch
elastic  = capacity - static_weights
expert_floor = min(plane+spare, max(plane + 2*unit_max - host_room, 0))   # planner.cpp expert_vram_floor
host_room  = host_pool_mib * MiB  (less the file tier's staging when disk tiering is on)
kv_cap     = max_num_seqs * max_model_len          # DEMAND, in tokens
kv         = min(kv_cap, elastic - expert_floor)   # what it actually gets
tokens     = min over paged groups of n_blocks*block_size   # the printed number

Read consequence: --max-num-seqs raises demand, --host-pool-mib raises supply, and both are the same lever — every MiB added to the host pool lowers the expert floor one-for-one and the cache takes it (up to kv_cap); the leftover returns to the experts. --max-num-seqs alone does nothing to capacity: it raises demand while the floor is unchanged, so the printed tokens stay put. The page's "funds ~91 % of demand" table rows are that cap, not a policy percentage.

• Derive the plane from a live ledger instead of guessing: plane + 2*unit ≈ printed_experts + host_pool_bytes.
• --gpu-headroom-mib lowers go to the cache but the engine's settle floor is built from it — 48 MiB measured decode-neutral, 24 untested. 48 is the first rung.

Procedure

1. Read scripts/<svc>.env for the current pool: GPU_HEADROOM_MIB (96), HOST_POOL_MIB (44,928 per rank), MAX_SEQS x MAX_MODEL_LEN (demand), HOST_POOL_MIB, PLACEMENT.
2. Compute the target as tokens: tokens x bytes/token-per-card, where bytes/token = (cache_given_bytes / tokens) from the latest ledger. If the target exceeds elastic - expert_floor, ALSO lower expert_floor via --host-pool-mib (up = more room = fewer resident experts) — a bigger host pool buys cache only while experts remain on the card; once the host absorbs the whole plane, more host RAM buys nothing (measured: +640 MiB moved vram_kv byte-identical).
3. Edit ONE knob (GPU_HEADROOM_MIB), deploy service-scoped: ./scripts/deploy.sh <host> --service <svc>. The engine reloads everything (~2-4 min); the deployer's LLM_WAIT covers it.
4. Re-read the ledger: vram_kv must be the new number, byte-exact or the knob lost to another claim; mover: N slots can be lent tells what the expert slab re-borrowed from the enlarged pool — that steal is the decode cost of the resize.
5. Re-run the probe (scripts/probe-frank-archon-kv.py): depth 1.75x tokens, then the 2x refusal. Update <svc>.md Laws with the measured pair (tokens, given, backing %).
6. Under-funded? Raise HOST_POOL_MIB in 1,024-MiB steps and redeploy; never touch demand to fake the number.

Pinned-RAM ledger to recompute BEFORE booting (this is where a resize kills the host)

total_pin = tp * HOST_POOL_MIB + PLE/n-gram table + PREFIX_CACHE_MIB + (statics + arena ~2,048) + checkpoint_store, with checkpoint_store = slots * 111.98 MiB and --checkpoint-slots auto = max_num_seqs * (1 + what the retention policy keeps) — it grows with the slot count, so a slot bump raises the pin ledger twice (state + store). The repo's bats guard recomputes this and asserts TTM_MIN_MIB >= ledger; when you change pool or slots, update TTM_MIN_MIB (and that guard) in the same commit, or the seam fails. The serve script's own gate refuses to start below TTM_MIN_MIB; ttm-preflight.sh raises the knob to TTM_TARGET_MIB, so the target must clear the new floor with slack.

Traps

• The cache is ELASTIC: the ledger's kv cache line is the ceiling, and mover: ... slots can be lent out of the activation arena (987 MiB) means the slab borrows arena first, then cache blocks. A bigger ceiling only helps if the heat engine stops re-borrowing it — check flex_bytes/flex_capacity movement after load, not just the budget line.
• A resize that shrinks the pool mid-session preempts live sequences (vllm:num_preemptions_total jumps); a resize that grows it cannot retroactively un-preempt. Deploy at low traffic.
• Do not resize the KV pool AND the expert placement in one deploy: the experts and kv cache lines move together and the attribution dies.
• The claimable arithmetic (capacity) is computed ONCE before allocation; card total differs per card (26.0 vs 26.68 GiB on dankserver), so the SLOWER card's number is the budget — read the [0] and [1] pools: lines separately, not just the shared budget table.