Compare commits

...
Author SHA1 Message Date
Frank e6fba5d97e fix(frank-archon): fold the pinned host pool back to 49,152; keep the 2,048 step
PR #81's pool half measured inert the day it deployed: the 66,560 boot printed the
same elastic 26.73 GiB / experts 20.72 / kv cache 5.64 / vram_weights 24.30 as the
49,152 boot and opened the pool with 0 B used, and the qwen38-w5nl64 rig moved
vram_kv 269.99 -> 271.25 MiB (window still 7,436 tokens: its 95.37 GiB n-gram table
claim decides, not the pool). The step half stays. Tests: 576 ok / 0 fail.
2026-10-29 15:10:33 -07:00
Frank ca1475543d docs(frank-archon): the 2026-10-29 boot ledger - the 66,560 pool gives the same cache as 49,152
The container opened the raised pool with 0 B used and printed the same elastic 26.73 GiB / experts 20.72 GiB / kv cache 5.64 GiB / vram_weights 24.30 GiB as the 49,152 boot, so the era's superseded ledger line is destroyed and the evidence table gains the ceiling-case row beside the 45,568 one it already carried. Tests: ./tests/run.sh 576 ok / 0 fail.
2026-10-29 13:48:34 -07:00
dank f83c241668 Merge pull request 'feat(frank-archon): raise the pinned host pool to 66,560; the campaign inherits MNBT 2,048' (#81) from feat/frank-archon-pool-step into main 2026-10-29 13:14:27 -07:00
dank b4d7f6e7fe Merge pull request 'fix(qwen38): calib rides the shared step; the wide one cost the container its cache' (#80) from feat/qwen38-w5nl64 into main 2026-10-29 11:18:36 -07:00
5 changed files with 48 additions and 35 deletions

No files matched your search

+12 -10
View File
@@ -74,12 +74,14 @@ FRANK_ARCHON_PLACEMENT=expert_tiered
# residency -- ~3.2 GiB a card of experts now lives in host RAM and is read over the
# x8 link when the router picks it. The ledger's coverage line is the referee, not
# this file: if vram_kv comes back under 700,000 tokens, raise this 1024 and redeploy.
# 2026-10-29: 49,152 -> 66,560 (tp x 65 GiB, inside the 246 GiB the box has). The
# campaign rig's bf16 container at the shared step reported "kv cache 269.99 MiB a
# card: 7436 tokens", so 682 of the 1,294 calibration documents were refused at that
# pool; the pool, not the step, is what widens it (scripts/run-qwen38-bf16.sh). The
# new cache line belongs to the next boot ledger, not to this comment.
FRANK_ARCHON_HOST_POOL_MIB=66560
# 2026-10-29 A/B, measured on both payloads and folded back the same day: at 66,560 the
# endpoint printed the identical `elastic 26.73 GiB / experts 20.72 / kv cache 5.64 /
# vram_weights 24.30` it had printed at 49,152, opening the bigger pool with `0 B used`;
# the campaign rig (bf16, same step) moved `vram_kv 269.99 -> 271.25 MiB`, a rounding
# step — its window is fixed by its own n-gram table (95.37 GiB) and expert plane, not
# by this pool, so the refused calibration documents were fixed by cutting them to the
# window (scripts/rechunk-kld-corpus.py), not by widening the pool.
FRANK_ARCHON_HOST_POOL_MIB=49152
# VRAM left unclaimed per card (publisher default 96 MiB); the engine's settle
# floor is built from it.
FRANK_ARCHON_GPU_HEADROOM_MIB=96
@@ -142,10 +144,10 @@ FRANK_ARCHON_NGRAM_PLACEMENT=ram
# itself before refusing by name, so recovery is a boot, not a visit.
# Floor = 2 x HOST_POOL + 48,824 (static weights + arena) + PREFIX_CACHE + MNBT + the
# auto checkpoint store (MAX_SEQS x 10 slots of 111.98 MiB), per the bats ledger test;
# at the 66,560 pool with 4 slots that computes to 196,664 MiB (161,848 at the old
# 49,152 pool).
FRANK_ARCHON_TTM_MIN_MIB=197000
FRANK_ARCHON_TTM_TARGET_MIB=224000
# at the 49,152 pool with 4 slots that computes to 161,848 MiB (196,664 at the
# 66,560 pool the 2026-10-29 A/B rejected).
FRANK_ARCHON_TTM_MIN_MIB=162000
FRANK_ARCHON_TTM_TARGET_MIB=204800
# Escape hatch for experiments. Unset -> no extra argv; the serve script expands
# this unquoted on purpose, so keep multi-flag values as one string.
FRANK_ARCHON_EXTRA_ARGS=
+8 -7
View File
@@ -170,13 +170,14 @@ fi
# ONE step size for every mode, quoted from the file that owns it — the audit
# law for a radiance rig is to inherit the proven number, not to carry a
# campaign-private value: production's FRANK_ARCHON_MAX_BATCHED_TOKENS is 2,048,
# the widest step the quantised expert FILE tiers can stream (above it the
# engine charges a 10.65 GiB arena to the card and the load refuses), and the
# arena it charges comes out of the cache — so the corpora are cut to the window
# the boot ledger reports, never the other way round. The 1,024 default this
# script carried was PR #80's answer to a starved cache; the pool raise
# (frank-archon.env: 66,560 MiB a rank) is the fix, not a narrower step.
# campaign-private value: production's FRANK_ARCHON_MAX_BATCHED_TOKENS. The
# 1,024 default this script carried was PR #80's answer to a starved cache;
# production moved to its widest legal step (2,048) on 2026-10-29 and the rig
# follows. The pool raise that shipped with it (66,560 a rank) bought this
# container 1.26 MiB of cache (`vram_kv 269.99 -> 271.25 MiB`, window still
# 7,436 tokens — the 95.37 GiB n-gram table's claim decides the rig's cache,
# not the pool) and was folded back the same day. The corpora stay cut to the
# window the boot ledger reports, never the other way round.
MNBT="${FRANK_ARCHON_MAX_BATCHED_TOKENS:?FRANK_ARCHON_MAX_BATCHED_TOKENS must be set}"
if ! [[ "$MNBT" =~ ^[0-9]+$ ]]; then
echo "REFUSED: FRANK_ARCHON_MAX_BATCHED_TOKENS must be a token count (got: $MNBT)" >&2
+26 -16
View File
@@ -91,27 +91,36 @@ on stop. Boot budget 1800 s (the publisher's compose allows a 15-minute `start_p
| Slots / window | `MAX_SEQS=4`, `MAX_MODEL_LEN=200000` (native 262,144; longer is refused, never truncated) — this pair is the **cache bound**: the resolver sizes the KV carve against `slots × window`, and no slot count funds itself (Laws) |
| KV dtype | `KV_DTYPE=fp8` — the only dtype that funds the slot count (Laws) |
| VRAM split | **derived at boot** from `GPU_HEADROOM_MIB=96` + the cache bound + `HOST_POOL_MIB`; naming `VRAM_WEIGHTS_MIB`/`VRAM_KV_MIB` is a boot refusal (removed at 1.3.0) |
| Expert tier | `PLACEMENT=expert_tiered`, `HOST_POOL_MIB=66560` **per rank** (raised 2026-10-29 from 49152 — see *Eras*) — the pinned host pool is also the **KV lever**: the expert floor is `plane + 2 units − host room`, so every MiB pinned there lowers what the experts claim on the card one-for-one and the cache takes it (Laws) |
| Expert tier | `PLACEMENT=expert_tiered`, `HOST_POOL_MIB=49152` **per rank** (raised to 66560 on 2026-10-29, folded back the same day — the A/B under *Laws* shows the raise bought no cache) — the pinned host pool is also the **KV lever**: the expert floor is `plane + 2 units − host room`, so every MiB pinned there lowers what the experts claim on the card one-for-one and the cache takes it (Laws) |
| PLE | `NGRAM_PLACEMENT=ram` (47.7 GiB pinned; refuses boot on a short memlock) |
| Spec / batch | `SPEC_TOKENS=2` — MTP depth 2, the `.rad` carries the model's own `mtp.*` head (`mtp.fc_hidden`/`fc_embedding`/`draft_head`); `MAX_BATCHED_TOKENS=2048` (MNBT — 2560 collapses decode, Laws) |
| Prefix cache | `PREFIX_CACHE_MIB=8192`, pinned **host tier only** — the 64 GiB disk tier is gone, so an evicted prefix is re-prefilled; `STATE_DIR=/home/dank/radiance-state` now exists only to hold the rad-tune cache |
| Template | `TEMPLATE=scripts/qwen38-froggeric-v22.5.jinja` (28,234 B) via `--override-chat-template` — froggeric's fixed Qwen3.8 template (the repo root's `chat_template.jinja`, vendored); the checkpoint's own template is baked into the `.rad` by rad-convert and this overrides it, and radiance derives the reasoning and tool-call reply format from it |
| Step sharing | `BUSY_PREFILL_CHUNK=auto` — prefill-speed first: it reserves 3 tokens per waiting decoder and ⅛ of the budget per waiting request, and does **not** shrink the chunk for decoders alone · `PREFILL_INTERLEAVE=1` — the decoder dial, **armed 2026-10-20**: one decode-only step after each chunk step taken beside decoders; meter `engine_steps_held_total`, price is the prefilling prompt's TTFT |
| TTM gate | `TTM_MIN_MIB=197000`, `TTM_TARGET_MIB=224000` |
| TTM gate | `TTM_MIN_MIB=162000`, `TTM_TARGET_MIB=204800` |
| Boot | `RAD_MIN_MIB=110000` deployer floor; the listener binds only after the full load (health ~100 s, budget 1800 s) |
Boot ledger for the live shape (2026-10-20, pin 1.8.1, `MAX_SEQS=4`, `HOST_POOL_MIB=49152`):
the resolver prints `elastic 26.73 GiB` → **experts 20.72 GiB (78 %) / kv cache 5.64 GiB (21 %)**,
`prefill staging 373.65 MiB` out of the cache, `vram_weights 24.30 GiB`, `host_pool 48.59 GiB`.
The referee line: `kv: the cache can grow to 5.64 GiB a card — 747,492 tokens of context, 3.7
sessions at 200,000 — with every block it lends the experts taken back; 440.04 MiB of that is
per-sequence state, and up to 5.20 GiB is lent to the experts while no request holds it`.
`--checkpoint-slots 40` is auto (`max_num_seqs × 10` slots of 111.98 MiB = 4,479 MiB of pinned
host), `/health` reports `queue_cap 32` (`max(8, seqs × 8)`), sampler **12 rows**
(= `seqs × (1 + speculative depth)`), 4 hardware queues a card, template 28,234 B, vision live
(≤ 4,194,304 px, 16,384 patches/pass), P2P confirmed (`the 2 cards reach each other's memory
directly`). Experts living in the pinned host pool: **8.99 GiB a rank** (was 5.99) — the price of
the bigger cache, paid over PCIe when those experts are hit.
Boot ledger for the live payload. Last boot: 2026-10-29 at `MAX_SEQS=4`,
`HOST_POOL_MIB=66560`, `MAX_BATCHED_TOKENS=2048`; the live pool is `49152` again, and the
two boots are **identical above `host_pool`**, so one ledger serves both shapes:
`elastic 26.73 GiB` → **experts 20.72 GiB (78 %) / kv cache 5.64 GiB (21 %)**,
`activation arena 990.36 MiB` (the price of the 2,048-token step: `256 tokens use
155.60 MiB, 1024 tokens use 472.86 MiB of 896.20 MiB`), `prefill staging 373.65 MiB`,
`vram_weights 24.30 GiB`, `host_pool 65.59 GiB` (opened with `0 B used`; `48.59 GiB` at
the 49,152 boot), `unclaimed 96.89 MiB`; referee line `kv: the pool backs 93% of the worst
case (4 sequences all at 200000 tokens), 3.7 full-length sessions`; `device: the 2 cards
reach each other's memory directly (peer to peer)`; sampler **12 rows**
(= `seqs × (1 + speculative depth)`), 4 hardware queues a card (133.45 MiB), `already held
256.67 MiB`, mover `17519 slab slots, 7624 pinned pool slots, 3.58 GiB static vram`,
`load: rank 0, mover took 24.30 GiB`.
The A/B that ended the raise: the 2026-10-20 boot at `HOST_POOL_MIB=49152` printed the
**same** `elastic 26.73 / experts 20.72 / kv cache 5.64 / vram_weights 24.30` (Laws: the
lever saturates at the expert floor), and the campaign rig — the other payload that
inherits this env — moved `vram_kv 269.99 → 271.25 MiB`, a rounding step, because its
window is set by its own n-gram table (95.37 GiB) and expert plane rather than by the
pool. The refused calibration documents are fixed by cutting them to the window, not by
widening the pool.
The boot **ledger**, not this arithmetic, is the referee: radiance states every budget before
anything is allocated (weights vs KV, the host pool's coverage of the worst case, the PLE pin,
@@ -150,7 +159,7 @@ marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `Sti
| Window | Payload | Runtime | Shape | Replaced because | Rollback |
|---|---|---|---|---|---|
| 2026-10-20 → 10-29 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.8.1**, `sha256:d590e300…` (same digest) | 4 × 200,000 fp8 KV — resolver cache 5.64 GiB = 747,492 tokens, `prefill staging 373.65 MiB`, `vram_weights 24.30 GiB`, `host_pool 48.59 GiB`; host pool 49,152 MiB a rank, `TTM_MIN_MIB=162000` / `TTM_TARGET_MIB=204800`, `queue_cap 32`, sampler 12 rows, `--prefill-interleave 1` | the `qwen38-w5nl64` campaign rig booted this pool at the shared 1,024 step and reported `kv cache 269.99 MiB a card: 7436 tokens`, so 682 of its 1,294 calibration documents were refused; Frank ordered the pool **and** the step raised to production's widest legal step (2,048) as one shape | revert the `feat/frank-archon-pool-step` commit and redeploy `scripts/deploy.sh dankserver --service frank-archon` — trio back to `HOST_POOL_MIB=49152`, `TTM_MIN_MIB=162000`, `TTM_TARGET_MIB=204800` |
| 2026-10-20 → 10-29 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.8.1**, `sha256:d590e300…` (same digest) | 4 × 200,000 fp8 KV — resolver cache 5.64 GiB = 747,492 tokens, `prefill staging 373.65 MiB`, `vram_weights 24.30 GiB`, `host_pool 48.59 GiB`; host pool 49,152 MiB a rank, `TTM_MIN_MIB=162000` / `TTM_TARGET_MIB=204800`, `queue_cap 32`, sampler 12 rows, `--prefill-interleave 1` | Frank ordered the campaign's step (2,048) onto production; the pool half of that order (66,560) booted, came back byte-identical to this row's cache, and was folded back the same day — the calibration documents it was meant to save were cut to the window instead (`scripts/rechunk-kld-corpus.py`) | the live shape is this row's pool with `MAX_BATCHED_TOKENS=2048`: the pair is the rollback unit, not the merge commit (`scripts/frank-archon.env`) |
| 2026-10-19 → 10-20 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.8.1**, `sha256:d590e300…` (same digest) | 2 × 200,000 fp8 KV — resolver cache 2.76 GiB = 365,204 tokens (91 % backing); host pool 44,928 MiB a rank, `TTM_MIN_MIB=150000`, `queue_cap 16`, sampler 9 rows; `--prefill-interleave 1` armed on 10-20 | Frank ordered **four** concurrent conversations over a ≥ 700,000-token pool; the resolver funds that only by lowering the expert floor with more pinned host room (Laws) | revert `1bb6399` and redeploy `scripts/deploy.sh dankserver --service frank-archon` — trio back to `MAX_SEQS=2` / `HOST_POOL_MIB=44928` / `TTM_MIN_MIB=150000` |
| 2026-10-15 → 10-19 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.0.13**, `sha256:4c3f4845…` | 3 × 200,000 fp8 KV bought with the **explicit** pair `--vram-weights-mib 25600 --vram-kv-mib 5120`; host pool 44,928 → 45,568 MiB a rank; prefix cache 8 GiB host + 64 GiB disk | decode turned to sludge beside any long prompt — step contention, 18.7 ms decode-only steps against 317 ms steps carrying a prefill chunk (Laws) — and the only two knobs that fix it, `--busy-prefill-chunk` and `--prefill-interleave`, exist only after this pin | `FRANK_ARCHON_IMAGE` back to `sha256:4c3f4845…` **plus a trio rewrite**: restore the pair and the host+disk prefix knobs, delete `BUSY_PREFILL_CHUNK`/`PREFILL_INTERLEAVE`. A digest swap alone is a boot refusal (Runtime) |
| 2026-10-08 → 10-15 | in-house `qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB (Swift1.5 abliterated BF16, GPTQ w4 experts + 24 bf16 protected, KLD 0.0131 / top-1 97.92 %) | radiance 1.0.13, same digest | 3 × 200,000 fp8 KV, slab 25,600 / pool 5,120 — the shape the live file still carries | the engine author's own container is the supported path and his quality ceiling; the finetune's refusal-bending left the endpoint | `.rad` purged 2026-10-17 (trash-staged then emptied, Frank's approval). Re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` → `qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, LFS sha256 `09dd9306…`, via the iq4r fetch script's hfget/verify pattern retargeted at that repo; point `MODEL_FILE` at it, redeploy. Re-**convert** is dead: the BF16 source was `rm`'d 2026-10-15 (`kld-ref`, grams, corpora and `/home/dank/logs/swift15_*.log` survive) |
@@ -244,7 +253,8 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
| 3 × 200,000, host pool 44,928 | 600,000 tok | 4.20 GiB | 93 % |
| 3 × 200,000, host pool 45,568 | 600,000 tok | **4.20 GiB — byte-identical** | 93 % |
| 2 × 200,000, host pool 44,928 | 400,000 tok | 2.76 GiB | 91 % |
| 4 × 200,000, host pool 49,152 (live) | 800,000 tok | **5.64 GiB = 747,492 tok** | 93 % |
| 4 × 200,000, host pool 49,152 (live) | 800,000 tok | 5.64 GiB = 747,492 tok | 93 % |
| 4 × 200,000, host pool 66,560 (2026-10-29 A/B, folded back) | 800,000 tok | **5.64 GiB — byte-identical to 49,152** | 93 % |
The 45,568 row is the ceiling case, not a mystery: at 3 slots the cache already held its whole
`ceiling`, so extra host room could not grow it and shows up as `unclaimed`. **The referee is the
+1 -1
View File
@@ -158,7 +158,7 @@ nosudo_bin() {
# so the pool's token count is a boot-ledger line — verified live by the
# capacity leg of scripts/probe-frank-archon-kv.py at every cutover.
grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_HOST_POOL_MIB=66560$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_HOST_POOL_MIB=49152$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_KV_DTYPE=fp8$' "$ENV_FILE"
grep -q '^FRANK_ARCHON_NGRAM_PLACEMENT=ram$' "$ENV_FILE"
# Prefix cache: pinned host RAM only (the disk tier was removed), and the
+1 -1
View File
@@ -29,7 +29,7 @@ RUNNER="$REPO_ROOT/scripts/run-qwen38-bf16.sh"
# The reference shape, spelled exactly as run-qwen38-bf16.sh must pass it: this
# is the contract the recorded reference is measured at, so a silent edit here
# is a silent edit to every KLD number in the campaign.
SHAPE="--tp 2 --placement expert_tiered --weights-disk-tier --host-pool-mib 66560 --gpu-headroom-mib 96 --deterministic --num-speculative-tokens 0 --kv-cache-dtype fp8 --max-model-len 32768 --ngram-placement ram --override-chat-template /models/tpl.jinja"
SHAPE="--tp 2 --placement expert_tiered --weights-disk-tier --host-pool-mib 49152 --gpu-headroom-mib 96 --deterministic --num-speculative-tokens 0 --kv-cache-dtype fp8 --max-model-len 32768 --ngram-placement ram --override-chat-template /models/tpl.jinja"
# oneline — both dry runs print one shell-quoted argument a line; joining them
# lets an option and its value be asserted as the pair they are.