38 lines
4.0 KiB
Markdown
38 lines
4.0 KiB
Markdown
---
|
||
name: radiance-shape-resize-verification
|
||
description: "Verify a radiance (frank-archon) KV/slab shape change end-to-end: TTM preflight, boot ledger, boundary probe, decode ladder — with the laws that govern each axis."
|
||
---
|
||
|
||
Use when changing `FRANK_ARCHON_KV_DTYPE`, `FRANK_ARCHON_MAX_SEQS`, `FRANK_ARCHON_VRAM_WEIGHTS_MIB`, or `FRANK_ARCHON_VRAM_KV_MIB` in `scripts/frank-archon.env`, and when asked why a shape change did or did not move throughput.
|
||
|
||
## Sequence
|
||
1. Pool sizing first: `pool_mib >= seqs*maxlen*rate + 110*seqs + 448`, rate = 7.3125 KiB/token fp8, 13.8125 KiB/token bf16 (measured page sizes 26.00 vs 52.00 KiB + 3.25 KiB dtype-independent QSA key).
|
||
2. Pair ceiling: `weights + kv <= 30,969` MiB/card (card 32,624 − 3,425 engine-side fixed: 2,048 activation arena, 209.19 floor, 960 driver/kernel, 224 NCCL/IPC, 96 headroom, 600 unaccounted). Settle floor is 209.19 MiB, NOT the hand-picked 1 GiB reserve.
|
||
3. Update bats seams: pair ceiling (`FRANK_ARCHON_VRAM_CEILING_MIB`), slab floor, KV-pool contract; keep the ceiling seam owning only pair arithmetic.
|
||
4. `./tests/run.sh` (needs bats on PATH: `/home/blank/go/bin`), commit, merge `origin/wt/<branch>` ff-only into main at **/home/blank/Projects/serverAppConfig**, push main.
|
||
5. Deploy: `ssh dankserver 'cd /home/dank/serverAppConfigs && git pull --ff-only && ./scripts/deploy.sh dankserver --service frank-archon'` — boot 3–4 min; `--dry-run` first to confirm the plan selects exactly one service.
|
||
6. Read the boot ledger: `journalctl -u frank-archon | grep -E "kv cache|budgeted|expert slab|resident experts|unclaimed|settled at|sampler"` — engine prints backed tokens, slab slots, host-pool slots, settle, rows.
|
||
7. Verify contract: `ssh dankserver 'python3 /home/dank/serverAppConfigs/scripts/probe-frank-archon-kv.py'` — legs A (decode at depth), B (N×window capacity), C (prefix restore), D (N-stream decode).
|
||
|
||
## Laws (measured; trust over recalculation)
|
||
- Slab is a THRESHOLD: 22,208 → 3,397, 24,000 → 3,290, 25,600 → 3,290 t/s at 43.5k; cliff at ~26 GiB (26,071 → 4,379).
|
||
- Decode is nearly window-free: 99.0 t/s at 43.5k, 93.3 at 197k.
|
||
- Slots are paid in decode CONCURRENCY: 3 streams = 18.4 t/s aggregate, per-stream a fused-step lottery (113.9/43.3/10.9).
|
||
- Idle KV VRAM lends to the expert slab (fp8 pool 6,912 lent 6.08 GiB vs 2.69 at 3,543).
|
||
- Bigger pool can LOWER prefill: more resident experts = less pinned host-pool traffic.
|
||
- Oversized prompt is FREE and refused by name — counter `frank_archon_prompt_too_large_total` counts; refusal fires on PROMPT tokens alone ("leaves nothing to generate into"); radiance NEVER truncates.
|
||
- TTM: serve script raises `ttm.pages_limit` to `FRANK_ARCHON_TTM_TARGET_MIB` itself (reboot resets to half RAM); unit ExecStartPre hook.
|
||
- Refusal signature: `activating (auto-restart)`, exit 1 in ~50 ms, no container, no engine log.
|
||
|
||
## Probe law
|
||
- 512 generated tokens minimum (8–24-token probes report decode noise as rate); greedy; cold prompts (unique nonce) to defeat the disk prefix cache; acceptance ~0.66–0.74 on repo corpus, repetitive filler 0.93–0.97 (never compare across corpora).
|
||
- Radiance routes thinking to `reasoning_content`: set `chat_template_kwargs.enable_thinking=false`, read content + reasoning_content.
|
||
- `finish_reason: "stop"` with `completion_tokens: 1` on a capacity request = accepted, not failed.
|
||
|
||
## Traps
|
||
- Python heredoc over ssh: `str.maketrans` with quoted numeric literals collapses (`'0123456789'` → `'06789'`) — build alphabets with `chr(48+i)`/`chr(97+i)`.
|
||
- `FRANK_SUDO_PW` in `/home/dank/.profile` has a CRLF tail; strip `\r` for systemd credentials (`tr -d '\r' | sudo systemd-credential ...`).
|
||
- Deployer reports DEPLOY INCOMPLETE on a stray port-holder (`ss -ltnp | grep :7770`); never kill a root process over ssh without sudo.
|
||
- The main checkout on the dev workstation is `/home/blank/Projects/serverAppConfig`, NOT `/home/dank/serverAppConfigs` (that path is on the servers).
|
||
- `systemctl mask` fails on the relinked unit; `deploy.sh <host>` with no `--service` is a fleet-wide drain.
|