dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(frank-archon): fold the pinned host pool back to 49,152; keep the 2,048 step

Browse Source 
PR #81's pool half measured inert the day it deployed: the 66,560 boot printed the
same elastic 26.73 GiB / experts 20.72 / kv cache 5.64 / vram_weights 24.30 as the
49,152 boot and opened the pool with 0 B used, and the qwen38-w5nl64 rig moved
vram_kv 269.99 -> 271.25 MiB (window still 7,436 tokens: its 95.37 GiB n-gram table
claim decides, not the pool). The step half stays. Tests: 576 ok / 0 fail.
This commit is contained in:
Frank committed 2026-10-29 15:10:33 -07:00
1 parent ca1475543d
commit e6fba5d97e
5 files changed
+47 -39
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

serverAppConfigs-docs/LLM/frank-archon.md
+25 -20
View File
@@ -91,31 +91,36 @@ on stop. Boot budget 1800 s (the publisher's compose allows a 15-minute `start_p
| Slots / window | `MAX_SEQS=4`, `MAX_MODEL_LEN=200000` (native 262,144; longer is refused, never truncated) — this pair is the **cache bound**: the resolver sizes the KV carve against `slots × window`, and no slot count funds itself (Laws) |
| KV dtype | `KV_DTYPE=fp8` — the only dtype that funds the slot count (Laws) |
| VRAM split | **derived at boot** from `GPU_HEADROOM_MIB=96` + the cache bound + `HOST_POOL_MIB`; naming `VRAM_WEIGHTS_MIB`/`VRAM_KV_MIB` is a boot refusal (removed at 1.3.0) |
| Expert tier | `PLACEMENT=expert_tiered`, `HOST_POOL_MIB=66560` **per rank** (raised 2026-10-29 from 49152 — see *Eras*) — the pinned host pool is also the **KV lever**: the expert floor is `plane + 2 units − host room`, so every MiB pinned there lowers what the experts claim on the card one-for-one and the cache takes it (Laws) |
| Expert tier | `PLACEMENT=expert_tiered`, `HOST_POOL_MIB=49152` **per rank** (raised to 66560 on 2026-10-29, folded back the same day — the A/B under *Laws* shows the raise bought no cache) — the pinned host pool is also the **KV lever**: the expert floor is `plane + 2 units − host room`, so every MiB pinned there lowers what the experts claim on the card one-for-one and the cache takes it (Laws) |
| PLE | `NGRAM_PLACEMENT=ram` (47.7 GiB pinned; refuses boot on a short memlock) |
| Spec / batch | `SPEC_TOKENS=2` — MTP depth 2, the `.rad` carries the model's own `mtp.*` head (`mtp.fc_hidden`/`fc_embedding`/`draft_head`); `MAX_BATCHED_TOKENS=2048` (MNBT — 2560 collapses decode, Laws) |
| Prefix cache | `PREFIX_CACHE_MIB=8192`, pinned **host tier only** — the 64 GiB disk tier is gone, so an evicted prefix is re-prefilled; `STATE_DIR=/home/dank/radiance-state` now exists only to hold the rad-tune cache |
| Template | `TEMPLATE=scripts/qwen38-froggeric-v22.5.jinja` (28,234 B) via `--override-chat-template` — froggeric's fixed Qwen3.8 template (the repo root's `chat_template.jinja`, vendored); the checkpoint's own template is baked into the `.rad` by rad-convert and this overrides it, and radiance derives the reasoning and tool-call reply format from it |
| Step sharing | `BUSY_PREFILL_CHUNK=auto` — prefill-speed first: it reserves 3 tokens per waiting decoder and ⅛ of the budget per waiting request, and does **not** shrink the chunk for decoders alone · `PREFILL_INTERLEAVE=1` — the decoder dial, **armed 2026-10-20**: one decode-only step after each chunk step taken beside decoders; meter `engine_steps_held_total`, price is the prefilling prompt's TTFT |
| TTM gate | `TTM_MIN_MIB=197000`, `TTM_TARGET_MIB=224000` |
| TTM gate | `TTM_MIN_MIB=162000`, `TTM_TARGET_MIB=204800` |
| Boot | `RAD_MIN_MIB=110000` deployer floor; the listener binds only after the full load (health ~100 s, budget 1800 s) |
Boot ledger for the live shape (2026-10-29, pin 1.8.1, `MAX_SEQS=4`,
`HOST_POOL_MIB=66560`, `MAX_BATCHED_TOKENS=2048`): the resolver prints `elastic 26.73 GiB` →
**experts 20.72 GiB (78 %) / kv cache 5.64 GiB (21 %)**, `activation arena 990.36 MiB` (the
price of the 2,048-token step: `256 tokens use 155.60 MiB, 1024 tokens use 472.86 MiB of
896.20 MiB`), `prefill staging 373.65 MiB`, `vram_weights 24.30 GiB`, `host_pool 65.59 GiB`,
`unclaimed 96.89 MiB`; referee line `kv: the pool backs 93% of the worst case (4 sequences all
at 200000 tokens), 3.7 full-length sessions`; `device: the 2 cards reach each other's memory
directly (peer to peer)`; sampler **12 rows** (= `seqs × (1 + speculative depth)`), 4 hardware
queues a card (133.45 MiB), `already held 256.67 MiB`, mover `17519 slab slots, 7624 pinned
pool slots, 3.58 GiB static vram`, `load: rank 0, mover took 24.30 GiB`.
Boot ledger for the live payload. Last boot: 2026-10-29 at `MAX_SEQS=4`,
`HOST_POOL_MIB=66560`, `MAX_BATCHED_TOKENS=2048`; the live pool is `49152` again, and the
two boots are **identical above `host_pool`**, so one ledger serves both shapes:
`elastic 26.73 GiB` → **experts 20.72 GiB (78 %) / kv cache 5.64 GiB (21 %)**,
`activation arena 990.36 MiB` (the price of the 2,048-token step: `256 tokens use
155.60 MiB, 1024 tokens use 472.86 MiB of 896.20 MiB`), `prefill staging 373.65 MiB`,
`vram_weights 24.30 GiB`, `host_pool 65.59 GiB` (opened with `0 B used`; `48.59 GiB` at
the 49,152 boot), `unclaimed 96.89 MiB`; referee line `kv: the pool backs 93% of the worst
case (4 sequences all at 200000 tokens), 3.7 full-length sessions`; `device: the 2 cards
reach each other's memory directly (peer to peer)`; sampler **12 rows**
(= `seqs × (1 + speculative depth)`), 4 hardware queues a card (133.45 MiB), `already held
256.67 MiB`, mover `17519 slab slots, 7624 pinned pool slots, 3.58 GiB static vram`,
`load: rank 0, mover took 24.30 GiB`.
The 2026-10-20 boot at `HOST_POOL_MIB=49152` printed the **same** `elastic 26.73 GiB /
experts 20.72 GiB / kv cache 5.64 GiB / vram_weights 24.30 GiB`. The pool raise bought the
endpoint no cache: the engine opened that pool with `0 B used` and the VRAM split never moved
(Laws: the lever saturates at the expert floor). The era's own ledger line is superseded by
this one; the number the old era quoted is gone with it.
The A/B that ended the raise: the 2026-10-20 boot at `HOST_POOL_MIB=49152` printed the
**same** `elastic 26.73 / experts 20.72 / kv cache 5.64 / vram_weights 24.30` (Laws: the
lever saturates at the expert floor), and the campaign rig — the other payload that
inherits this env — moved `vram_kv 269.99 → 271.25 MiB`, a rounding step, because its
window is set by its own n-gram table (95.37 GiB) and expert plane rather than by the
pool. The refused calibration documents are fixed by cutting them to the window, not by
widening the pool.
The boot **ledger**, not this arithmetic, is the referee: radiance states every budget before
anything is allocated (weights vs KV, the host pool's coverage of the worst case, the PLE pin,
@@ -154,7 +159,7 @@ marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `Sti
| Window | Payload | Runtime | Shape | Replaced because | Rollback |
|---|---|---|---|---|---|
| 2026-10-20 → 10-29 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.8.1**, `sha256:d590e300…` (same digest) | 4 × 200,000 fp8 KV — resolver cache 5.64 GiB = 747,492 tokens, `prefill staging 373.65 MiB`, `vram_weights 24.30 GiB`, `host_pool 48.59 GiB`; host pool 49,152 MiB a rank, `TTM_MIN_MIB=162000` / `TTM_TARGET_MIB=204800`, `queue_cap 32`, sampler 12 rows, `--prefill-interleave 1` | the `qwen38-w5nl64` campaign rig booted this pool at the shared 1,024 step and reported `kv cache 269.99 MiB a card: 7436 tokens`, so 682 of its 1,294 calibration documents were refused; Frank ordered the pool **and** the step raised to production's widest legal step (2,048) as one shape | revert the `feat/frank-archon-pool-step` commit and redeploy `scripts/deploy.sh dankserver --service frank-archon` — trio back to `HOST_POOL_MIB=49152`, `TTM_MIN_MIB=162000`, `TTM_TARGET_MIB=204800` |
| 2026-10-20 → 10-29 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.8.1**, `sha256:d590e300…` (same digest) | 4 × 200,000 fp8 KV — resolver cache 5.64 GiB = 747,492 tokens, `prefill staging 373.65 MiB`, `vram_weights 24.30 GiB`, `host_pool 48.59 GiB`; host pool 49,152 MiB a rank, `TTM_MIN_MIB=162000` / `TTM_TARGET_MIB=204800`, `queue_cap 32`, sampler 12 rows, `--prefill-interleave 1` | Frank ordered the campaign's step (2,048) onto production; the pool half of that order (66,560) booted, came back byte-identical to this row's cache, and was folded back the same day — the calibration documents it was meant to save were cut to the window instead (`scripts/rechunk-kld-corpus.py`) | the live shape is this row's pool with `MAX_BATCHED_TOKENS=2048`: the pair is the rollback unit, not the merge commit (`scripts/frank-archon.env`) |
| 2026-10-19 → 10-20 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.8.1**, `sha256:d590e300…` (same digest) | 2 × 200,000 fp8 KV — resolver cache 2.76 GiB = 365,204 tokens (91 % backing); host pool 44,928 MiB a rank, `TTM_MIN_MIB=150000`, `queue_cap 16`, sampler 9 rows; `--prefill-interleave 1` armed on 10-20 | Frank ordered **four** concurrent conversations over a ≥ 700,000-token pool; the resolver funds that only by lowering the expert floor with more pinned host room (Laws) | revert `1bb6399` and redeploy `scripts/deploy.sh dankserver --service frank-archon` — trio back to `MAX_SEQS=2` / `HOST_POOL_MIB=44928` / `TTM_MIN_MIB=150000` |
| 2026-10-15 → 10-19 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.0.13**, `sha256:4c3f4845…` | 3 × 200,000 fp8 KV bought with the **explicit** pair `--vram-weights-mib 25600 --vram-kv-mib 5120`; host pool 44,928 → 45,568 MiB a rank; prefix cache 8 GiB host + 64 GiB disk | decode turned to sludge beside any long prompt — step contention, 18.7 ms decode-only steps against 317 ms steps carrying a prefill chunk (Laws) — and the only two knobs that fix it, `--busy-prefill-chunk` and `--prefill-interleave`, exist only after this pin | `FRANK_ARCHON_IMAGE` back to `sha256:4c3f4845…` **plus a trio rewrite**: restore the pair and the host+disk prefix knobs, delete `BUSY_PREFILL_CHUNK`/`PREFILL_INTERLEAVE`. A digest swap alone is a boot refusal (Runtime) |
| 2026-10-08 → 10-15 | in-house `qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB (Swift1.5 abliterated BF16, GPTQ w4 experts + 24 bf16 protected, KLD 0.0131 / top-1 97.92 %) | radiance 1.0.13, same digest | 3 × 200,000 fp8 KV, slab 25,600 / pool 5,120 — the shape the live file still carries | the engine author's own container is the supported path and his quality ceiling; the finetune's refusal-bending left the endpoint | `.rad` purged 2026-10-17 (trash-staged then emptied, Frank's approval). Re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` → `qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, LFS sha256 `09dd9306…`, via the iq4r fetch script's hfget/verify pattern retargeted at that repo; point `MODEL_FILE` at it, redeploy. Re-**convert** is dead: the BF16 source was `rm`'d 2026-10-15 (`kld-ref`, grams, corpora and `/home/dank/logs/swift15_*.log` survive) |
@@ -248,8 +253,8 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
| 3 × 200,000, host pool 44,928 | 600,000 tok | 4.20 GiB | 93 % |
| 3 × 200,000, host pool 45,568 | 600,000 tok | **4.20 GiB — byte-identical** | 93 % |
| 2 × 200,000, host pool 44,928 | 400,000 tok | 2.76 GiB | 91 % |
| 4 × 200,000, host pool 49,152 | 800,000 tok | 5.64 GiB = 747,492 tok | 93 % |
| 4 × 200,000, host pool 66,560 (live) | 800,000 tok | **5.64 GiB — byte-identical to 49,152** | 93 % |
| 4 × 200,000, host pool 49,152 (live) | 800,000 tok | 5.64 GiB = 747,492 tok | 93 % |
| 4 × 200,000, host pool 66,560 (2026-10-29 A/B, folded back) | 800,000 tok | **5.64 GiB — byte-identical to 49,152** | 93 % |
The 45,568 row is the ceiling case, not a mystery: at 3 slots the cache already held its whole
`ceiling`, so extra host room could not grow it and shows up as `unclaimed`. **The referee is the