fix(frank-archon): fold the pinned host pool back to 49,152; keep the 2,048 step
PR #81's pool half measured inert the day it deployed: the 66,560 boot printed the same elastic 26.73 GiB / experts 20.72 / kv cache 5.64 / vram_weights 24.30 as the 49,152 boot and opened the pool with 0 B used, and the qwen38-w5nl64 rig moved vram_kv 269.99 -> 271.25 MiB (window still 7,436 tokens: its 95.37 GiB n-gram table claim decides, not the pool). The step half stays. Tests: 576 ok / 0 fail.
This commit is contained in:
1 parent
ca1475543d
commit
e6fba5d97e
5 files changed
+47
-39
No files matched your search
+12
-10
@@ -74,12 +74,14 @@ FRANK_ARCHON_PLACEMENT=expert_tiered
|
|||||||
# residency -- ~3.2 GiB a card of experts now lives in host RAM and is read over the
|
# residency -- ~3.2 GiB a card of experts now lives in host RAM and is read over the
|
||||||
# x8 link when the router picks it. The ledger's coverage line is the referee, not
|
# x8 link when the router picks it. The ledger's coverage line is the referee, not
|
||||||
# this file: if vram_kv comes back under 700,000 tokens, raise this 1024 and redeploy.
|
# this file: if vram_kv comes back under 700,000 tokens, raise this 1024 and redeploy.
|
||||||
# 2026-10-29: 49,152 -> 66,560 (tp x 65 GiB, inside the 246 GiB the box has). The
|
# 2026-10-29 A/B, measured on both payloads and folded back the same day: at 66,560 the
|
||||||
# campaign rig's bf16 container at the shared step reported "kv cache 269.99 MiB a
|
# endpoint printed the identical `elastic 26.73 GiB / experts 20.72 / kv cache 5.64 /
|
||||||
# card: 7436 tokens", so 682 of the 1,294 calibration documents were refused at that
|
# vram_weights 24.30` it had printed at 49,152, opening the bigger pool with `0 B used`;
|
||||||
# pool; the pool, not the step, is what widens it (scripts/run-qwen38-bf16.sh). The
|
# the campaign rig (bf16, same step) moved `vram_kv 269.99 -> 271.25 MiB`, a rounding
|
||||||
# new cache line belongs to the next boot ledger, not to this comment.
|
# step — its window is fixed by its own n-gram table (95.37 GiB) and expert plane, not
|
||||||
FRANK_ARCHON_HOST_POOL_MIB=66560
|
# by this pool, so the refused calibration documents were fixed by cutting them to the
|
||||||
|
# window (scripts/rechunk-kld-corpus.py), not by widening the pool.
|
||||||
|
FRANK_ARCHON_HOST_POOL_MIB=49152
|
||||||
# VRAM left unclaimed per card (publisher default 96 MiB); the engine's settle
|
# VRAM left unclaimed per card (publisher default 96 MiB); the engine's settle
|
||||||
# floor is built from it.
|
# floor is built from it.
|
||||||
FRANK_ARCHON_GPU_HEADROOM_MIB=96
|
FRANK_ARCHON_GPU_HEADROOM_MIB=96
|
||||||
@@ -142,10 +144,10 @@ FRANK_ARCHON_NGRAM_PLACEMENT=ram
|
|||||||
# itself before refusing by name, so recovery is a boot, not a visit.
|
# itself before refusing by name, so recovery is a boot, not a visit.
|
||||||
# Floor = 2 x HOST_POOL + 48,824 (static weights + arena) + PREFIX_CACHE + MNBT + the
|
# Floor = 2 x HOST_POOL + 48,824 (static weights + arena) + PREFIX_CACHE + MNBT + the
|
||||||
# auto checkpoint store (MAX_SEQS x 10 slots of 111.98 MiB), per the bats ledger test;
|
# auto checkpoint store (MAX_SEQS x 10 slots of 111.98 MiB), per the bats ledger test;
|
||||||
# at the 66,560 pool with 4 slots that computes to 196,664 MiB (161,848 at the old
|
# at the 49,152 pool with 4 slots that computes to 161,848 MiB (196,664 at the
|
||||||
# 49,152 pool).
|
# 66,560 pool the 2026-10-29 A/B rejected).
|
||||||
FRANK_ARCHON_TTM_MIN_MIB=197000
|
FRANK_ARCHON_TTM_MIN_MIB=162000
|
||||||
FRANK_ARCHON_TTM_TARGET_MIB=224000
|
FRANK_ARCHON_TTM_TARGET_MIB=204800
|
||||||
# Escape hatch for experiments. Unset -> no extra argv; the serve script expands
|
# Escape hatch for experiments. Unset -> no extra argv; the serve script expands
|
||||||
# this unquoted on purpose, so keep multi-flag values as one string.
|
# this unquoted on purpose, so keep multi-flag values as one string.
|
||||||
FRANK_ARCHON_EXTRA_ARGS=
|
FRANK_ARCHON_EXTRA_ARGS=
|
||||||
|
|||||||
@@ -170,13 +170,14 @@ fi
|
|||||||
|
|
||||||
# ONE step size for every mode, quoted from the file that owns it — the audit
|
# ONE step size for every mode, quoted from the file that owns it — the audit
|
||||||
# law for a radiance rig is to inherit the proven number, not to carry a
|
# law for a radiance rig is to inherit the proven number, not to carry a
|
||||||
# campaign-private value: production's FRANK_ARCHON_MAX_BATCHED_TOKENS is 2,048,
|
# campaign-private value: production's FRANK_ARCHON_MAX_BATCHED_TOKENS. The
|
||||||
# the widest step the quantised expert FILE tiers can stream (above it the
|
# 1,024 default this script carried was PR #80's answer to a starved cache;
|
||||||
# engine charges a 10.65 GiB arena to the card and the load refuses), and the
|
# production moved to its widest legal step (2,048) on 2026-10-29 and the rig
|
||||||
# arena it charges comes out of the cache — so the corpora are cut to the window
|
# follows. The pool raise that shipped with it (66,560 a rank) bought this
|
||||||
# the boot ledger reports, never the other way round. The 1,024 default this
|
# container 1.26 MiB of cache (`vram_kv 269.99 -> 271.25 MiB`, window still
|
||||||
# script carried was PR #80's answer to a starved cache; the pool raise
|
# 7,436 tokens — the 95.37 GiB n-gram table's claim decides the rig's cache,
|
||||||
# (frank-archon.env: 66,560 MiB a rank) is the fix, not a narrower step.
|
# not the pool) and was folded back the same day. The corpora stay cut to the
|
||||||
|
# window the boot ledger reports, never the other way round.
|
||||||
MNBT="${FRANK_ARCHON_MAX_BATCHED_TOKENS:?FRANK_ARCHON_MAX_BATCHED_TOKENS must be set}"
|
MNBT="${FRANK_ARCHON_MAX_BATCHED_TOKENS:?FRANK_ARCHON_MAX_BATCHED_TOKENS must be set}"
|
||||||
if ! [[ "$MNBT" =~ ^[0-9]+$ ]]; then
|
if ! [[ "$MNBT" =~ ^[0-9]+$ ]]; then
|
||||||
echo "REFUSED: FRANK_ARCHON_MAX_BATCHED_TOKENS must be a token count (got: $MNBT)" >&2
|
echo "REFUSED: FRANK_ARCHON_MAX_BATCHED_TOKENS must be a token count (got: $MNBT)" >&2
|
||||||
|
|||||||
@@ -91,31 +91,36 @@ on stop. Boot budget 1800 s (the publisher's compose allows a 15-minute `start_p
|
|||||||
| Slots / window | `MAX_SEQS=4`, `MAX_MODEL_LEN=200000` (native 262,144; longer is refused, never truncated) — this pair is the **cache bound**: the resolver sizes the KV carve against `slots × window`, and no slot count funds itself (Laws) |
|
| Slots / window | `MAX_SEQS=4`, `MAX_MODEL_LEN=200000` (native 262,144; longer is refused, never truncated) — this pair is the **cache bound**: the resolver sizes the KV carve against `slots × window`, and no slot count funds itself (Laws) |
|
||||||
| KV dtype | `KV_DTYPE=fp8` — the only dtype that funds the slot count (Laws) |
|
| KV dtype | `KV_DTYPE=fp8` — the only dtype that funds the slot count (Laws) |
|
||||||
| VRAM split | **derived at boot** from `GPU_HEADROOM_MIB=96` + the cache bound + `HOST_POOL_MIB`; naming `VRAM_WEIGHTS_MIB`/`VRAM_KV_MIB` is a boot refusal (removed at 1.3.0) |
|
| VRAM split | **derived at boot** from `GPU_HEADROOM_MIB=96` + the cache bound + `HOST_POOL_MIB`; naming `VRAM_WEIGHTS_MIB`/`VRAM_KV_MIB` is a boot refusal (removed at 1.3.0) |
|
||||||
| Expert tier | `PLACEMENT=expert_tiered`, `HOST_POOL_MIB=66560` **per rank** (raised 2026-10-29 from 49152 — see *Eras*) — the pinned host pool is also the **KV lever**: the expert floor is `plane + 2 units − host room`, so every MiB pinned there lowers what the experts claim on the card one-for-one and the cache takes it (Laws) |
|
| Expert tier | `PLACEMENT=expert_tiered`, `HOST_POOL_MIB=49152` **per rank** (raised to 66560 on 2026-10-29, folded back the same day — the A/B under *Laws* shows the raise bought no cache) — the pinned host pool is also the **KV lever**: the expert floor is `plane + 2 units − host room`, so every MiB pinned there lowers what the experts claim on the card one-for-one and the cache takes it (Laws) |
|
||||||
| PLE | `NGRAM_PLACEMENT=ram` (47.7 GiB pinned; refuses boot on a short memlock) |
|
| PLE | `NGRAM_PLACEMENT=ram` (47.7 GiB pinned; refuses boot on a short memlock) |
|
||||||
| Spec / batch | `SPEC_TOKENS=2` — MTP depth 2, the `.rad` carries the model's own `mtp.*` head (`mtp.fc_hidden`/`fc_embedding`/`draft_head`); `MAX_BATCHED_TOKENS=2048` (MNBT — 2560 collapses decode, Laws) |
|
| Spec / batch | `SPEC_TOKENS=2` — MTP depth 2, the `.rad` carries the model's own `mtp.*` head (`mtp.fc_hidden`/`fc_embedding`/`draft_head`); `MAX_BATCHED_TOKENS=2048` (MNBT — 2560 collapses decode, Laws) |
|
||||||
| Prefix cache | `PREFIX_CACHE_MIB=8192`, pinned **host tier only** — the 64 GiB disk tier is gone, so an evicted prefix is re-prefilled; `STATE_DIR=/home/dank/radiance-state` now exists only to hold the rad-tune cache |
|
| Prefix cache | `PREFIX_CACHE_MIB=8192`, pinned **host tier only** — the 64 GiB disk tier is gone, so an evicted prefix is re-prefilled; `STATE_DIR=/home/dank/radiance-state` now exists only to hold the rad-tune cache |
|
||||||
| Template | `TEMPLATE=scripts/qwen38-froggeric-v22.5.jinja` (28,234 B) via `--override-chat-template` — froggeric's fixed Qwen3.8 template (the repo root's `chat_template.jinja`, vendored); the checkpoint's own template is baked into the `.rad` by rad-convert and this overrides it, and radiance derives the reasoning and tool-call reply format from it |
|
| Template | `TEMPLATE=scripts/qwen38-froggeric-v22.5.jinja` (28,234 B) via `--override-chat-template` — froggeric's fixed Qwen3.8 template (the repo root's `chat_template.jinja`, vendored); the checkpoint's own template is baked into the `.rad` by rad-convert and this overrides it, and radiance derives the reasoning and tool-call reply format from it |
|
||||||
| Step sharing | `BUSY_PREFILL_CHUNK=auto` — prefill-speed first: it reserves 3 tokens per waiting decoder and ⅛ of the budget per waiting request, and does **not** shrink the chunk for decoders alone · `PREFILL_INTERLEAVE=1` — the decoder dial, **armed 2026-10-20**: one decode-only step after each chunk step taken beside decoders; meter `engine_steps_held_total`, price is the prefilling prompt's TTFT |
|
| Step sharing | `BUSY_PREFILL_CHUNK=auto` — prefill-speed first: it reserves 3 tokens per waiting decoder and ⅛ of the budget per waiting request, and does **not** shrink the chunk for decoders alone · `PREFILL_INTERLEAVE=1` — the decoder dial, **armed 2026-10-20**: one decode-only step after each chunk step taken beside decoders; meter `engine_steps_held_total`, price is the prefilling prompt's TTFT |
|
||||||
| TTM gate | `TTM_MIN_MIB=197000`, `TTM_TARGET_MIB=224000` |
|
| TTM gate | `TTM_MIN_MIB=162000`, `TTM_TARGET_MIB=204800` |
|
||||||
| Boot | `RAD_MIN_MIB=110000` deployer floor; the listener binds only after the full load (health ~100 s, budget 1800 s) |
|
| Boot | `RAD_MIN_MIB=110000` deployer floor; the listener binds only after the full load (health ~100 s, budget 1800 s) |
|
||||||
|
|
||||||
Boot ledger for the live shape (2026-10-29, pin 1.8.1, `MAX_SEQS=4`,
|
Boot ledger for the live payload. Last boot: 2026-10-29 at `MAX_SEQS=4`,
|
||||||
`HOST_POOL_MIB=66560`, `MAX_BATCHED_TOKENS=2048`): the resolver prints `elastic 26.73 GiB` →
|
`HOST_POOL_MIB=66560`, `MAX_BATCHED_TOKENS=2048`; the live pool is `49152` again, and the
|
||||||
**experts 20.72 GiB (78 %) / kv cache 5.64 GiB (21 %)**, `activation arena 990.36 MiB` (the
|
two boots are **identical above `host_pool`**, so one ledger serves both shapes:
|
||||||
price of the 2,048-token step: `256 tokens use 155.60 MiB, 1024 tokens use 472.86 MiB of
|
`elastic 26.73 GiB` → **experts 20.72 GiB (78 %) / kv cache 5.64 GiB (21 %)**,
|
||||||
896.20 MiB`), `prefill staging 373.65 MiB`, `vram_weights 24.30 GiB`, `host_pool 65.59 GiB`,
|
`activation arena 990.36 MiB` (the price of the 2,048-token step: `256 tokens use
|
||||||
`unclaimed 96.89 MiB`; referee line `kv: the pool backs 93% of the worst case (4 sequences all
|
155.60 MiB, 1024 tokens use 472.86 MiB of 896.20 MiB`), `prefill staging 373.65 MiB`,
|
||||||
at 200000 tokens), 3.7 full-length sessions`; `device: the 2 cards reach each other's memory
|
`vram_weights 24.30 GiB`, `host_pool 65.59 GiB` (opened with `0 B used`; `48.59 GiB` at
|
||||||
directly (peer to peer)`; sampler **12 rows** (= `seqs × (1 + speculative depth)`), 4 hardware
|
the 49,152 boot), `unclaimed 96.89 MiB`; referee line `kv: the pool backs 93% of the worst
|
||||||
queues a card (133.45 MiB), `already held 256.67 MiB`, mover `17519 slab slots, 7624 pinned
|
case (4 sequences all at 200000 tokens), 3.7 full-length sessions`; `device: the 2 cards
|
||||||
pool slots, 3.58 GiB static vram`, `load: rank 0, mover took 24.30 GiB`.
|
reach each other's memory directly (peer to peer)`; sampler **12 rows**
|
||||||
|
(= `seqs × (1 + speculative depth)`), 4 hardware queues a card (133.45 MiB), `already held
|
||||||
|
256.67 MiB`, mover `17519 slab slots, 7624 pinned pool slots, 3.58 GiB static vram`,
|
||||||
|
`load: rank 0, mover took 24.30 GiB`.
|
||||||
|
|
||||||
The 2026-10-20 boot at `HOST_POOL_MIB=49152` printed the **same** `elastic 26.73 GiB /
|
The A/B that ended the raise: the 2026-10-20 boot at `HOST_POOL_MIB=49152` printed the
|
||||||
experts 20.72 GiB / kv cache 5.64 GiB / vram_weights 24.30 GiB`. The pool raise bought the
|
**same** `elastic 26.73 / experts 20.72 / kv cache 5.64 / vram_weights 24.30` (Laws: the
|
||||||
endpoint no cache: the engine opened that pool with `0 B used` and the VRAM split never moved
|
lever saturates at the expert floor), and the campaign rig — the other payload that
|
||||||
(Laws: the lever saturates at the expert floor). The era's own ledger line is superseded by
|
inherits this env — moved `vram_kv 269.99 → 271.25 MiB`, a rounding step, because its
|
||||||
this one; the number the old era quoted is gone with it.
|
window is set by its own n-gram table (95.37 GiB) and expert plane rather than by the
|
||||||
|
pool. The refused calibration documents are fixed by cutting them to the window, not by
|
||||||
|
widening the pool.
|
||||||
|
|
||||||
The boot **ledger**, not this arithmetic, is the referee: radiance states every budget before
|
The boot **ledger**, not this arithmetic, is the referee: radiance states every budget before
|
||||||
anything is allocated (weights vs KV, the host pool's coverage of the worst case, the PLE pin,
|
anything is allocated (weights vs KV, the host pool's coverage of the worst case, the PLE pin,
|
||||||
@@ -154,7 +159,7 @@ marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `Sti
|
|||||||
|
|
||||||
| Window | Payload | Runtime | Shape | Replaced because | Rollback |
|
| Window | Payload | Runtime | Shape | Replaced because | Rollback |
|
||||||
|---|---|---|---|---|---|
|
|---|---|---|---|---|---|
|
||||||
| 2026-10-20 → 10-29 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.8.1**, `sha256:d590e300…` (same digest) | 4 × 200,000 fp8 KV — resolver cache 5.64 GiB = 747,492 tokens, `prefill staging 373.65 MiB`, `vram_weights 24.30 GiB`, `host_pool 48.59 GiB`; host pool 49,152 MiB a rank, `TTM_MIN_MIB=162000` / `TTM_TARGET_MIB=204800`, `queue_cap 32`, sampler 12 rows, `--prefill-interleave 1` | the `qwen38-w5nl64` campaign rig booted this pool at the shared 1,024 step and reported `kv cache 269.99 MiB a card: 7436 tokens`, so 682 of its 1,294 calibration documents were refused; Frank ordered the pool **and** the step raised to production's widest legal step (2,048) as one shape | revert the `feat/frank-archon-pool-step` commit and redeploy `scripts/deploy.sh dankserver --service frank-archon` — trio back to `HOST_POOL_MIB=49152`, `TTM_MIN_MIB=162000`, `TTM_TARGET_MIB=204800` |
|
| 2026-10-20 → 10-29 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.8.1**, `sha256:d590e300…` (same digest) | 4 × 200,000 fp8 KV — resolver cache 5.64 GiB = 747,492 tokens, `prefill staging 373.65 MiB`, `vram_weights 24.30 GiB`, `host_pool 48.59 GiB`; host pool 49,152 MiB a rank, `TTM_MIN_MIB=162000` / `TTM_TARGET_MIB=204800`, `queue_cap 32`, sampler 12 rows, `--prefill-interleave 1` | Frank ordered the campaign's step (2,048) onto production; the pool half of that order (66,560) booted, came back byte-identical to this row's cache, and was folded back the same day — the calibration documents it was meant to save were cut to the window instead (`scripts/rechunk-kld-corpus.py`) | the live shape is this row's pool with `MAX_BATCHED_TOKENS=2048`: the pair is the rollback unit, not the merge commit (`scripts/frank-archon.env`) |
|
||||||
| 2026-10-19 → 10-20 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.8.1**, `sha256:d590e300…` (same digest) | 2 × 200,000 fp8 KV — resolver cache 2.76 GiB = 365,204 tokens (91 % backing); host pool 44,928 MiB a rank, `TTM_MIN_MIB=150000`, `queue_cap 16`, sampler 9 rows; `--prefill-interleave 1` armed on 10-20 | Frank ordered **four** concurrent conversations over a ≥ 700,000-token pool; the resolver funds that only by lowering the expert floor with more pinned host room (Laws) | revert `1bb6399` and redeploy `scripts/deploy.sh dankserver --service frank-archon` — trio back to `MAX_SEQS=2` / `HOST_POOL_MIB=44928` / `TTM_MIN_MIB=150000` |
|
| 2026-10-19 → 10-20 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.8.1**, `sha256:d590e300…` (same digest) | 2 × 200,000 fp8 KV — resolver cache 2.76 GiB = 365,204 tokens (91 % backing); host pool 44,928 MiB a rank, `TTM_MIN_MIB=150000`, `queue_cap 16`, sampler 9 rows; `--prefill-interleave 1` armed on 10-20 | Frank ordered **four** concurrent conversations over a ≥ 700,000-token pool; the resolver funds that only by lowering the expert floor with more pinned host room (Laws) | revert `1bb6399` and redeploy `scripts/deploy.sh dankserver --service frank-archon` — trio back to `MAX_SEQS=2` / `HOST_POOL_MIB=44928` / `TTM_MIN_MIB=150000` |
|
||||||
| 2026-10-15 → 10-19 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.0.13**, `sha256:4c3f4845…` | 3 × 200,000 fp8 KV bought with the **explicit** pair `--vram-weights-mib 25600 --vram-kv-mib 5120`; host pool 44,928 → 45,568 MiB a rank; prefix cache 8 GiB host + 64 GiB disk | decode turned to sludge beside any long prompt — step contention, 18.7 ms decode-only steps against 317 ms steps carrying a prefill chunk (Laws) — and the only two knobs that fix it, `--busy-prefill-chunk` and `--prefill-interleave`, exist only after this pin | `FRANK_ARCHON_IMAGE` back to `sha256:4c3f4845…` **plus a trio rewrite**: restore the pair and the host+disk prefix knobs, delete `BUSY_PREFILL_CHUNK`/`PREFILL_INTERLEAVE`. A digest swap alone is a boot refusal (Runtime) |
|
| 2026-10-15 → 10-19 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.0.13**, `sha256:4c3f4845…` | 3 × 200,000 fp8 KV bought with the **explicit** pair `--vram-weights-mib 25600 --vram-kv-mib 5120`; host pool 44,928 → 45,568 MiB a rank; prefix cache 8 GiB host + 64 GiB disk | decode turned to sludge beside any long prompt — step contention, 18.7 ms decode-only steps against 317 ms steps carrying a prefill chunk (Laws) — and the only two knobs that fix it, `--busy-prefill-chunk` and `--prefill-interleave`, exist only after this pin | `FRANK_ARCHON_IMAGE` back to `sha256:4c3f4845…` **plus a trio rewrite**: restore the pair and the host+disk prefix knobs, delete `BUSY_PREFILL_CHUNK`/`PREFILL_INTERLEAVE`. A digest swap alone is a boot refusal (Runtime) |
|
||||||
| 2026-10-08 → 10-15 | in-house `qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB (Swift1.5 abliterated BF16, GPTQ w4 experts + 24 bf16 protected, KLD 0.0131 / top-1 97.92 %) | radiance 1.0.13, same digest | 3 × 200,000 fp8 KV, slab 25,600 / pool 5,120 — the shape the live file still carries | the engine author's own container is the supported path and his quality ceiling; the finetune's refusal-bending left the endpoint | `.rad` purged 2026-10-17 (trash-staged then emptied, Frank's approval). Re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` → `qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, LFS sha256 `09dd9306…`, via the iq4r fetch script's hfget/verify pattern retargeted at that repo; point `MODEL_FILE` at it, redeploy. Re-**convert** is dead: the BF16 source was `rm`'d 2026-10-15 (`kld-ref`, grams, corpora and `/home/dank/logs/swift15_*.log` survive) |
|
| 2026-10-08 → 10-15 | in-house `qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB (Swift1.5 abliterated BF16, GPTQ w4 experts + 24 bf16 protected, KLD 0.0131 / top-1 97.92 %) | radiance 1.0.13, same digest | 3 × 200,000 fp8 KV, slab 25,600 / pool 5,120 — the shape the live file still carries | the engine author's own container is the supported path and his quality ceiling; the finetune's refusal-bending left the endpoint | `.rad` purged 2026-10-17 (trash-staged then emptied, Frank's approval). Re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` → `qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, LFS sha256 `09dd9306…`, via the iq4r fetch script's hfget/verify pattern retargeted at that repo; point `MODEL_FILE` at it, redeploy. Re-**convert** is dead: the BF16 source was `rm`'d 2026-10-15 (`kld-ref`, grams, corpora and `/home/dank/logs/swift15_*.log` survive) |
|
||||||
@@ -248,8 +253,8 @@ vllm-radiance era (this table opens on 09-26; the era page claims 10-10).
|
|||||||
| 3 × 200,000, host pool 44,928 | 600,000 tok | 4.20 GiB | 93 % |
|
| 3 × 200,000, host pool 44,928 | 600,000 tok | 4.20 GiB | 93 % |
|
||||||
| 3 × 200,000, host pool 45,568 | 600,000 tok | **4.20 GiB — byte-identical** | 93 % |
|
| 3 × 200,000, host pool 45,568 | 600,000 tok | **4.20 GiB — byte-identical** | 93 % |
|
||||||
| 2 × 200,000, host pool 44,928 | 400,000 tok | 2.76 GiB | 91 % |
|
| 2 × 200,000, host pool 44,928 | 400,000 tok | 2.76 GiB | 91 % |
|
||||||
| 4 × 200,000, host pool 49,152 | 800,000 tok | 5.64 GiB = 747,492 tok | 93 % |
|
| 4 × 200,000, host pool 49,152 (live) | 800,000 tok | 5.64 GiB = 747,492 tok | 93 % |
|
||||||
| 4 × 200,000, host pool 66,560 (live) | 800,000 tok | **5.64 GiB — byte-identical to 49,152** | 93 % |
|
| 4 × 200,000, host pool 66,560 (2026-10-29 A/B, folded back) | 800,000 tok | **5.64 GiB — byte-identical to 49,152** | 93 % |
|
||||||
|
|
||||||
The 45,568 row is the ceiling case, not a mystery: at 3 slots the cache already held its whole
|
The 45,568 row is the ceiling case, not a mystery: at 3 slots the cache already held its whole
|
||||||
`ceiling`, so extra host room could not grow it and shows up as `unclaimed`. **The referee is the
|
`ceiling`, so extra host room could not grow it and shows up as `unclaimed`. **The referee is the
|
||||||
|
|||||||
@@ -158,7 +158,7 @@ nosudo_bin() {
|
|||||||
# so the pool's token count is a boot-ledger line — verified live by the
|
# so the pool's token count is a boot-ledger line — verified live by the
|
||||||
# capacity leg of scripts/probe-frank-archon-kv.py at every cutover.
|
# capacity leg of scripts/probe-frank-archon-kv.py at every cutover.
|
||||||
grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_HOST_POOL_MIB=66560$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_HOST_POOL_MIB=49152$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_KV_DTYPE=fp8$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_KV_DTYPE=fp8$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_NGRAM_PLACEMENT=ram$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_NGRAM_PLACEMENT=ram$' "$ENV_FILE"
|
||||||
# Prefix cache: pinned host RAM only (the disk tier was removed), and the
|
# Prefix cache: pinned host RAM only (the disk tier was removed), and the
|
||||||
|
|||||||
@@ -29,7 +29,7 @@ RUNNER="$REPO_ROOT/scripts/run-qwen38-bf16.sh"
|
|||||||
# The reference shape, spelled exactly as run-qwen38-bf16.sh must pass it: this
|
# The reference shape, spelled exactly as run-qwen38-bf16.sh must pass it: this
|
||||||
# is the contract the recorded reference is measured at, so a silent edit here
|
# is the contract the recorded reference is measured at, so a silent edit here
|
||||||
# is a silent edit to every KLD number in the campaign.
|
# is a silent edit to every KLD number in the campaign.
|
||||||
SHAPE="--tp 2 --placement expert_tiered --weights-disk-tier --host-pool-mib 66560 --gpu-headroom-mib 96 --deterministic --num-speculative-tokens 0 --kv-cache-dtype fp8 --max-model-len 32768 --ngram-placement ram --override-chat-template /models/tpl.jinja"
|
SHAPE="--tp 2 --placement expert_tiered --weights-disk-tier --host-pool-mib 49152 --gpu-headroom-mib 96 --deterministic --num-speculative-tokens 0 --kv-cache-dtype fp8 --max-model-len 32768 --ngram-placement ram --override-chat-template /models/tpl.jinja"
|
||||||
|
|
||||||
# oneline — both dry runs print one shell-quoted argument a line; joining them
|
# oneline — both dry runs print one shell-quoted argument a line; joining them
|
||||||
# lets an option and its value be asserted as the pair they are.
|
# lets an option and its value be asserted as the pair they are.
|
||||||
|
|||||||
Reference in new issue
Block a user