feat(frank-archon): raise the pinned host pool; the campaign inherits the step
frank-archon.env: HOST_POOL_MIB 49,152 -> 66,560 a rank (tp x 65 GiB against the 246 GiB the box has), and the TTM pair 162,000/204,800 -> 197,000/224,000 — the floor the bats ledger formula computes at the new pool. The pool is the KV lever under the budget resolver: every MiB pinned in host RAM lowers the expert floor on the card one-for-one and the cache takes it. scripts/run-qwen38-bf16.sh: the campaign's private FRANK_QWEN38_MNBT (1,024) is gone — MNBT now comes from FRANK_ARCHON_MAX_BATCHED_TOKENS, production's 2,048 and the widest step the quantised expert file tiers stream. That 1,024 was PR #80's answer to a starved cache; the pool is the fix, a narrow step was the symptom's disguise, and a per-mode literal measures a shape nothing serves. tests: frank_archon_unit greps the new pool value; qwen38_reference's SHAPE fixture carries it, and the two MNBT-override tests become the inheritance assertion (every mode's argv quotes the value the env file owns) plus the calib assertions. vault: the shape block moves to the new numbers and the Eras table gains the row for the era this replaces. The 2026-10-20 boot-ledger line stays as the old era's record until the new boot's ledger replaces it.
This commit is contained in:
1 parent
a3ff55efd8
commit
255796b3dc
5 files changed
+52
-51
No files matched your search
@@ -74,7 +74,12 @@ FRANK_ARCHON_PLACEMENT=expert_tiered
|
|||||||
# residency -- ~3.2 GiB a card of experts now lives in host RAM and is read over the
|
# residency -- ~3.2 GiB a card of experts now lives in host RAM and is read over the
|
||||||
# x8 link when the router picks it. The ledger's coverage line is the referee, not
|
# x8 link when the router picks it. The ledger's coverage line is the referee, not
|
||||||
# this file: if vram_kv comes back under 700,000 tokens, raise this 1024 and redeploy.
|
# this file: if vram_kv comes back under 700,000 tokens, raise this 1024 and redeploy.
|
||||||
FRANK_ARCHON_HOST_POOL_MIB=49152
|
# 2026-10-29: 49,152 -> 66,560 (tp x 65 GiB, inside the 246 GiB the box has). The
|
||||||
|
# campaign rig's bf16 container at the shared step reported "kv cache 269.99 MiB a
|
||||||
|
# card: 7436 tokens", so 682 of the 1,294 calibration documents were refused at that
|
||||||
|
# pool; the pool, not the step, is what widens it (scripts/run-qwen38-bf16.sh). The
|
||||||
|
# new cache line belongs to the next boot ledger, not to this comment.
|
||||||
|
FRANK_ARCHON_HOST_POOL_MIB=66560
|
||||||
# VRAM left unclaimed per card (publisher default 96 MiB); the engine's settle
|
# VRAM left unclaimed per card (publisher default 96 MiB); the engine's settle
|
||||||
# floor is built from it.
|
# floor is built from it.
|
||||||
FRANK_ARCHON_GPU_HEADROOM_MIB=96
|
FRANK_ARCHON_GPU_HEADROOM_MIB=96
|
||||||
@@ -137,9 +142,10 @@ FRANK_ARCHON_NGRAM_PLACEMENT=ram
|
|||||||
# itself before refusing by name, so recovery is a boot, not a visit.
|
# itself before refusing by name, so recovery is a boot, not a visit.
|
||||||
# Floor = 2 x HOST_POOL + 48,824 (static weights + arena) + PREFIX_CACHE + MNBT + the
|
# Floor = 2 x HOST_POOL + 48,824 (static weights + arena) + PREFIX_CACHE + MNBT + the
|
||||||
# auto checkpoint store (MAX_SEQS x 10 slots of 111.98 MiB), per the bats ledger test;
|
# auto checkpoint store (MAX_SEQS x 10 slots of 111.98 MiB), per the bats ledger test;
|
||||||
# at the 49,152 pool with 4 slots that computes to 161,848 MiB.
|
# at the 66,560 pool with 4 slots that computes to 196,664 MiB (161,848 at the old
|
||||||
FRANK_ARCHON_TTM_MIN_MIB=162000
|
# 49,152 pool).
|
||||||
FRANK_ARCHON_TTM_TARGET_MIB=204800
|
FRANK_ARCHON_TTM_MIN_MIB=197000
|
||||||
|
FRANK_ARCHON_TTM_TARGET_MIB=224000
|
||||||
# Escape hatch for experiments. Unset -> no extra argv; the serve script expands
|
# Escape hatch for experiments. Unset -> no extra argv; the serve script expands
|
||||||
# this unquoted on purpose, so keep multi-flag values as one string.
|
# this unquoted on purpose, so keep multi-flag values as one string.
|
||||||
FRANK_ARCHON_EXTRA_ARGS=
|
FRANK_ARCHON_EXTRA_ARGS=
|
||||||
|
|||||||
+27
-29
@@ -66,18 +66,14 @@
|
|||||||
# is 10,554 tokens by the engine's own count (chars/3.6 for this file, not the
|
# is 10,554 tokens by the engine's own count (chars/3.6 for this file, not the
|
||||||
# chars/4.97 this repo measured on the K3 corpus).
|
# chars/4.97 this repo measured on the K3 corpus).
|
||||||
#
|
#
|
||||||
# The step size is ONE value for the recording and every scoring pass:
|
# The step size is ONE value for the recording and every scoring pass, and it is
|
||||||
# FRANK_QWEN38_MNBT, default 1,024. The gate's whole claim is that the reference,
|
# the value frank-archon.env owns (FRANK_ARCHON_MAX_BATCHED_TOKENS, 2,048): the
|
||||||
# the served iq4r and every arm rode one shape, so the ceiling is set by the
|
# quantised expert FILE tiers (the served iq4r, the arms) stream only at
|
||||||
# TIGHTEST payload the gate boots rather than by the bf16 reference: the quantised
|
# MNBT<=2048 and refuse the widened form above it, and the arena a step costs the
|
||||||
# expert FILE tiers (the served iq4r, the arms) stream only at MNBT<=2048 and
|
# card comes straight out of the cache — 1.08 GiB at a 2,048-token step against
|
||||||
# refuse the widened form above it. 2,048 was still the wrong default on this
|
# 504.39 MiB at 1,024. The corpora are cut to the window the boot ledger reports,
|
||||||
# payload, because the arena the engine charges for a step comes straight out of
|
# which is what the pool raise is for (frank-archon.env: HOST_POOL_MIB); a narrow
|
||||||
# the cache — 1.08 GiB at a 2,048-token step, 504.39 MiB at a 1,024-token one, and
|
# step was never the fix, it was the symptom's disguise.
|
||||||
# the experts keep their share either way. At 1,024 the cache is 866 MiB: 24,300
|
|
||||||
# tokens, which scores every one of the 197 documents (longest 11,970).
|
|
||||||
# Calibration keeps the wide 65,536 — grams are not logits, nothing downstream is
|
|
||||||
# compared against a step size, and that is where a big step pays.
|
|
||||||
#
|
#
|
||||||
# usage: scripts/run-qwen38-bf16.sh kldref|calib|score RAD OUT [--dry-run] [--image REF]
|
# usage: scripts/run-qwen38-bf16.sh kldref|calib|score RAD OUT [--dry-run] [--image REF]
|
||||||
# (RAD and OUT are score's two positionals: RAD is a container under the models
|
# (RAD and OUT are score's two positionals: RAD is a container under the models
|
||||||
@@ -93,11 +89,10 @@
|
|||||||
# knobs: FRANK_QWEN38_MODELS (the dir mounted at /models), FRANK_QWEN38_RAD (the
|
# knobs: FRANK_QWEN38_MODELS (the dir mounted at /models), FRANK_QWEN38_RAD (the
|
||||||
# bf16 container, kldref only), FRANK_QWEN38_CAMPAIGN (holds kld-ref/ and
|
# bf16 container, kldref only), FRANK_QWEN38_CAMPAIGN (holds kld-ref/ and
|
||||||
# calib/), FRANK_QWEN38_KLD_CORPUS, FRANK_QWEN38_TPL (the template copy
|
# calib/), FRANK_QWEN38_KLD_CORPUS, FRANK_QWEN38_TPL (the template copy
|
||||||
# the container reads through /models), FRANK_QWEN38_LOG_DIR, and
|
# the container reads through /models), FRANK_QWEN38_LOG_DIR, and the step
|
||||||
# FRANK_QWEN38_MNBT — the step size every mode boots at (default
|
# and slot counts inherited as FRANK_ARCHON_MAX_BATCHED_TOKENS /
|
||||||
# 1,024: the file tier's ceiling is 2,048 and production runs there, but on
|
# FRANK_ARCHON_MAX_SEQS — no per-mode literals (the audit law: a mode that
|
||||||
# this payload the arena charged for a wider step is taken from the cache),
|
# carries its own step measures a shape nothing serves) —
|
||||||
# the slot count inherited as FRANK_ARCHON_MAX_SEQS — no per-mode literals —
|
|
||||||
# and FRANK_QWEN38_MAX_MODEL_LEN — the window every mode boots at (32,768;
|
# and FRANK_QWEN38_MAX_MODEL_LEN — the window every mode boots at (32,768;
|
||||||
# see the corpus census above, and raise it together with the corpus).
|
# see the corpus census above, and raise it together with the corpus).
|
||||||
# kldref refuses (exit 2) when the longest record of the corpus it is about to
|
# kldref refuses (exit 2) when the longest record of the corpus it is about to
|
||||||
@@ -173,15 +168,18 @@ if ! [[ "$MAX_SEQS" =~ ^[0-9]+$ ]]; then
|
|||||||
exit 2
|
exit 2
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# ONE step size for both logits modes. The file tier streams only at <=2,048 and
|
# ONE step size for every mode, quoted from the file that owns it — the audit
|
||||||
# the gate scores file-tier payloads (the served iq4r, the arms) against the
|
# law for a radiance rig is to inherit the proven number, not to carry a
|
||||||
# reference, so the reference is recorded at a step the payloads can meet — never
|
# campaign-private value: production's FRANK_ARCHON_MAX_BATCHED_TOKENS is 2,048,
|
||||||
# two numbers that a KL between them would have to forgive. 1,024 rather than the
|
# the widest step the quantised expert FILE tiers can stream (above it the
|
||||||
# 2,048 ceiling: on this payload the arena charged for the step is the cache's only
|
# engine charges a 10.65 GiB arena to the card and the load refuses), and the
|
||||||
# remaining supplier, and 1,024 is the rung that funds every corpus document.
|
# arena it charges comes out of the cache — so the corpora are cut to the window
|
||||||
MNBT="${FRANK_QWEN38_MNBT:-1024}"
|
# the boot ledger reports, never the other way round. The 1,024 default this
|
||||||
|
# script carried was PR #80's answer to a starved cache; the pool raise
|
||||||
|
# (frank-archon.env: 66,560 MiB a rank) is the fix, not a narrower step.
|
||||||
|
MNBT="${FRANK_ARCHON_MAX_BATCHED_TOKENS:?FRANK_ARCHON_MAX_BATCHED_TOKENS must be set}"
|
||||||
if ! [[ "$MNBT" =~ ^[0-9]+$ ]]; then
|
if ! [[ "$MNBT" =~ ^[0-9]+$ ]]; then
|
||||||
echo "REFUSED: FRANK_QWEN38_MNBT must be a token count (got: $MNBT)" >&2
|
echo "REFUSED: FRANK_ARCHON_MAX_BATCHED_TOKENS must be a token count (got: $MNBT)" >&2
|
||||||
exit 2
|
exit 2
|
||||||
fi
|
fi
|
||||||
|
|
||||||
@@ -236,10 +234,10 @@ MOUNTS=(-v "$MODELS:/models:ro")
|
|||||||
TAPENV=()
|
TAPENV=()
|
||||||
case "$MODE" in
|
case "$MODE" in
|
||||||
kldref)
|
kldref)
|
||||||
# Recording is prefill-only over four slots at the campaign's single step size
|
# Recording is prefill-only over four slots at the campaign's single step
|
||||||
# — 1,024, not production's 2,048 (frank-archon.env: FRANK_ARCHON_MAX_SEQS=4,
|
# size, which is frank-archon.env's FRANK_ARCHON_MAX_BATCHED_TOKENS (2,048) —
|
||||||
# FRANK_ARCHON_MAX_BATCHED_TOKENS=2048): the wider step costs the cache the
|
# see the header: the file tier's ceiling and the slot count both come from
|
||||||
# room the corpus needs. Corpus read-only, record dir rw.
|
# the file that owns them. Corpus read-only, record dir rw.
|
||||||
ARGS+=(--model "$MODEL_IN"
|
ARGS+=(--model "$MODEL_IN"
|
||||||
--kld-record /data/kld-ref --kld-corpus /data/kld.jsonl
|
--kld-record /data/kld-ref --kld-corpus /data/kld.jsonl
|
||||||
--max-num-seqs "$MAX_SEQS" --max-num-batched-tokens "$MNBT")
|
--max-num-seqs "$MAX_SEQS" --max-num-batched-tokens "$MNBT")
|
||||||
|
|||||||
@@ -91,13 +91,13 @@ on stop. Boot budget 1800 s (the publisher's compose allows a 15-minute `start_p
|
|||||||
| Slots / window | `MAX_SEQS=4`, `MAX_MODEL_LEN=200000` (native 262,144; longer is refused, never truncated) — this pair is the **cache bound**: the resolver sizes the KV carve against `slots × window`, and no slot count funds itself (Laws) |
|
| Slots / window | `MAX_SEQS=4`, `MAX_MODEL_LEN=200000` (native 262,144; longer is refused, never truncated) — this pair is the **cache bound**: the resolver sizes the KV carve against `slots × window`, and no slot count funds itself (Laws) |
|
||||||
| KV dtype | `KV_DTYPE=fp8` — the only dtype that funds the slot count (Laws) |
|
| KV dtype | `KV_DTYPE=fp8` — the only dtype that funds the slot count (Laws) |
|
||||||
| VRAM split | **derived at boot** from `GPU_HEADROOM_MIB=96` + the cache bound + `HOST_POOL_MIB`; naming `VRAM_WEIGHTS_MIB`/`VRAM_KV_MIB` is a boot refusal (removed at 1.3.0) |
|
| VRAM split | **derived at boot** from `GPU_HEADROOM_MIB=96` + the cache bound + `HOST_POOL_MIB`; naming `VRAM_WEIGHTS_MIB`/`VRAM_KV_MIB` is a boot refusal (removed at 1.3.0) |
|
||||||
| Expert tier | `PLACEMENT=expert_tiered`, `HOST_POOL_MIB=49152` **per rank** — the pinned host pool is also the **KV lever**: the expert floor is `plane + 2 units − host room`, so every MiB pinned there lowers what the experts claim on the card one-for-one and the cache takes it (Laws) |
|
| Expert tier | `PLACEMENT=expert_tiered`, `HOST_POOL_MIB=66560` **per rank** (raised 2026-10-29 from 49152 — see *Eras*) — the pinned host pool is also the **KV lever**: the expert floor is `plane + 2 units − host room`, so every MiB pinned there lowers what the experts claim on the card one-for-one and the cache takes it (Laws) |
|
||||||
| PLE | `NGRAM_PLACEMENT=ram` (47.7 GiB pinned; refuses boot on a short memlock) |
|
| PLE | `NGRAM_PLACEMENT=ram` (47.7 GiB pinned; refuses boot on a short memlock) |
|
||||||
| Spec / batch | `SPEC_TOKENS=2` — MTP depth 2, the `.rad` carries the model's own `mtp.*` head (`mtp.fc_hidden`/`fc_embedding`/`draft_head`); `MAX_BATCHED_TOKENS=2048` (MNBT — 2560 collapses decode, Laws) |
|
| Spec / batch | `SPEC_TOKENS=2` — MTP depth 2, the `.rad` carries the model's own `mtp.*` head (`mtp.fc_hidden`/`fc_embedding`/`draft_head`); `MAX_BATCHED_TOKENS=2048` (MNBT — 2560 collapses decode, Laws) |
|
||||||
| Prefix cache | `PREFIX_CACHE_MIB=8192`, pinned **host tier only** — the 64 GiB disk tier is gone, so an evicted prefix is re-prefilled; `STATE_DIR=/home/dank/radiance-state` now exists only to hold the rad-tune cache |
|
| Prefix cache | `PREFIX_CACHE_MIB=8192`, pinned **host tier only** — the 64 GiB disk tier is gone, so an evicted prefix is re-prefilled; `STATE_DIR=/home/dank/radiance-state` now exists only to hold the rad-tune cache |
|
||||||
| Template | `TEMPLATE=scripts/qwen38-froggeric-v22.5.jinja` (28,234 B) via `--override-chat-template` — froggeric's fixed Qwen3.8 template (the repo root's `chat_template.jinja`, vendored); the checkpoint's own template is baked into the `.rad` by rad-convert and this overrides it, and radiance derives the reasoning and tool-call reply format from it |
|
| Template | `TEMPLATE=scripts/qwen38-froggeric-v22.5.jinja` (28,234 B) via `--override-chat-template` — froggeric's fixed Qwen3.8 template (the repo root's `chat_template.jinja`, vendored); the checkpoint's own template is baked into the `.rad` by rad-convert and this overrides it, and radiance derives the reasoning and tool-call reply format from it |
|
||||||
| Step sharing | `BUSY_PREFILL_CHUNK=auto` — prefill-speed first: it reserves 3 tokens per waiting decoder and ⅛ of the budget per waiting request, and does **not** shrink the chunk for decoders alone · `PREFILL_INTERLEAVE=1` — the decoder dial, **armed 2026-10-20**: one decode-only step after each chunk step taken beside decoders; meter `engine_steps_held_total`, price is the prefilling prompt's TTFT |
|
| Step sharing | `BUSY_PREFILL_CHUNK=auto` — prefill-speed first: it reserves 3 tokens per waiting decoder and ⅛ of the budget per waiting request, and does **not** shrink the chunk for decoders alone · `PREFILL_INTERLEAVE=1` — the decoder dial, **armed 2026-10-20**: one decode-only step after each chunk step taken beside decoders; meter `engine_steps_held_total`, price is the prefilling prompt's TTFT |
|
||||||
| TTM gate | `TTM_MIN_MIB=162000`, `TTM_TARGET_MIB=204800` |
|
| TTM gate | `TTM_MIN_MIB=197000`, `TTM_TARGET_MIB=224000` |
|
||||||
| Boot | `RAD_MIN_MIB=110000` deployer floor; the listener binds only after the full load (health ~100 s, budget 1800 s) |
|
| Boot | `RAD_MIN_MIB=110000` deployer floor; the listener binds only after the full load (health ~100 s, budget 1800 s) |
|
||||||
|
|
||||||
Boot ledger for the live shape (2026-10-20, pin 1.8.1, `MAX_SEQS=4`, `HOST_POOL_MIB=49152`):
|
Boot ledger for the live shape (2026-10-20, pin 1.8.1, `MAX_SEQS=4`, `HOST_POOL_MIB=49152`):
|
||||||
@@ -150,6 +150,7 @@ marked "re-fetch" is gone from `/home/dank/models/`, which today holds only `Sti
|
|||||||
|
|
||||||
| Window | Payload | Runtime | Shape | Replaced because | Rollback |
|
| Window | Payload | Runtime | Shape | Replaced because | Rollback |
|
||||||
|---|---|---|---|---|---|
|
|---|---|---|---|---|---|
|
||||||
|
| 2026-10-20 → 10-29 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.8.1**, `sha256:d590e300…` (same digest) | 4 × 200,000 fp8 KV — resolver cache 5.64 GiB = 747,492 tokens, `prefill staging 373.65 MiB`, `vram_weights 24.30 GiB`, `host_pool 48.59 GiB`; host pool 49,152 MiB a rank, `TTM_MIN_MIB=162000` / `TTM_TARGET_MIB=204800`, `queue_cap 32`, sampler 12 rows, `--prefill-interleave 1` | the `qwen38-w5nl64` campaign rig booted this pool at the shared 1,024 step and reported `kv cache 269.99 MiB a card: 7436 tokens`, so 682 of its 1,294 calibration documents were refused; Frank ordered the pool **and** the step raised to production's widest legal step (2,048) as one shape | revert the `feat/frank-archon-pool-step` commit and redeploy `scripts/deploy.sh dankserver --service frank-archon` — trio back to `HOST_POOL_MIB=49152`, `TTM_MIN_MIB=162000`, `TTM_TARGET_MIB=204800` |
|
||||||
| 2026-10-19 → 10-20 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.8.1**, `sha256:d590e300…` (same digest) | 2 × 200,000 fp8 KV — resolver cache 2.76 GiB = 365,204 tokens (91 % backing); host pool 44,928 MiB a rank, `TTM_MIN_MIB=150000`, `queue_cap 16`, sampler 9 rows; `--prefill-interleave 1` armed on 10-20 | Frank ordered **four** concurrent conversations over a ≥ 700,000-token pool; the resolver funds that only by lowering the expert floor with more pinned host room (Laws) | revert `1bb6399` and redeploy `scripts/deploy.sh dankserver --service frank-archon` — trio back to `MAX_SEQS=2` / `HOST_POOL_MIB=44928` / `TTM_MIN_MIB=150000` |
|
| 2026-10-19 → 10-20 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.8.1**, `sha256:d590e300…` (same digest) | 2 × 200,000 fp8 KV — resolver cache 2.76 GiB = 365,204 tokens (91 % backing); host pool 44,928 MiB a rank, `TTM_MIN_MIB=150000`, `queue_cap 16`, sampler 9 rows; `--prefill-interleave 1` armed on 10-20 | Frank ordered **four** concurrent conversations over a ≥ 700,000-token pool; the resolver funds that only by lowering the expert floor with more pinned host room (Laws) | revert `1bb6399` and redeploy `scripts/deploy.sh dankserver --service frank-archon` — trio back to `MAX_SEQS=2` / `HOST_POOL_MIB=44928` / `TTM_MIN_MIB=150000` |
|
||||||
| 2026-10-15 → 10-19 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.0.13**, `sha256:4c3f4845…` | 3 × 200,000 fp8 KV bought with the **explicit** pair `--vram-weights-mib 25600 --vram-kv-mib 5120`; host pool 44,928 → 45,568 MiB a rank; prefix cache 8 GiB host + 64 GiB disk | decode turned to sludge beside any long prompt — step contention, 18.7 ms decode-only steps against 317 ms steps carrying a prefill chunk (Laws) — and the only two knobs that fix it, `--busy-prefill-chunk` and `--prefill-interleave`, exist only after this pin | `FRANK_ARCHON_IMAGE` back to `sha256:4c3f4845…` **plus a trio rewrite**: restore the pair and the host+disk prefix knobs, delete `BUSY_PREFILL_CHUNK`/`PREFILL_INTERLEAVE`. A digest swap alone is a boot refusal (Runtime) |
|
| 2026-10-15 → 10-19 | same `qwen3.8-next-flash-fp8-iq4r-moe.rad` (payload unchanged) | radiance **1.0.13**, `sha256:4c3f4845…` | 3 × 200,000 fp8 KV bought with the **explicit** pair `--vram-weights-mib 25600 --vram-kv-mib 5120`; host pool 44,928 → 45,568 MiB a rank; prefix cache 8 GiB host + 64 GiB disk | decode turned to sludge beside any long prompt — step contention, 18.7 ms decode-only steps against 317 ms steps carrying a prefill chunk (Laws) — and the only two knobs that fix it, `--busy-prefill-chunk` and `--prefill-interleave`, exist only after this pin | `FRANK_ARCHON_IMAGE` back to `sha256:4c3f4845…` **plus a trio rewrite**: restore the pair and the host+disk prefix knobs, delete `BUSY_PREFILL_CHUNK`/`PREFILL_INTERLEAVE`. A digest swap alone is a boot refusal (Runtime) |
|
||||||
| 2026-10-08 → 10-15 | in-house `qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB (Swift1.5 abliterated BF16, GPTQ w4 experts + 24 bf16 protected, KLD 0.0131 / top-1 97.92 %) | radiance 1.0.13, same digest | 3 × 200,000 fp8 KV, slab 25,600 / pool 5,120 — the shape the live file still carries | the engine author's own container is the supported path and his quality ceiling; the finetune's refusal-bending left the endpoint | `.rad` purged 2026-10-17 (trash-staged then emptied, Frank's approval). Re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` → `qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, LFS sha256 `09dd9306…`, via the iq4r fetch script's hfget/verify pattern retargeted at that repo; point `MODEL_FILE` at it, redeploy. Re-**convert** is dead: the BF16 source was `rm`'d 2026-10-15 (`kld-ref`, grams, corpora and `/home/dank/logs/swift15_*.log` survive) |
|
| 2026-10-08 → 10-15 | in-house `qwen4exp-swift15-w4dn8-i8-hc8m.rad`, 116,416 MiB (Swift1.5 abliterated BF16, GPTQ w4 experts + 24 bf16 protected, KLD 0.0131 / top-1 97.92 %) | radiance 1.0.13, same digest | 3 × 200,000 fp8 KV, slab 25,600 / pool 5,120 — the shape the live file still carries | the engine author's own container is the supported path and his quality ceiling; the finetune's refusal-bending left the endpoint | `.rad` purged 2026-10-17 (trash-staged then emptied, Frank's approval). Re-download `frank8465/Swift-1.5-Qwen3.8-Flash-Next-GPTQ-W4A8-Radiance-AMD-R9700` → `qwen4exp-w4nl64-i8-hc8m.rad`, 122,071,973,888 B, LFS sha256 `09dd9306…`, via the iq4r fetch script's hfget/verify pattern retargeted at that repo; point `MODEL_FILE` at it, redeploy. Re-**convert** is dead: the BF16 source was `rm`'d 2026-10-15 (`kld-ref`, grams, corpora and `/home/dank/logs/swift15_*.log` survive) |
|
||||||
|
|||||||
@@ -158,7 +158,7 @@ nosudo_bin() {
|
|||||||
# so the pool's token count is a boot-ledger line — verified live by the
|
# so the pool's token count is a boot-ledger line — verified live by the
|
||||||
# capacity leg of scripts/probe-frank-archon-kv.py at every cutover.
|
# capacity leg of scripts/probe-frank-archon-kv.py at every cutover.
|
||||||
grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_PLACEMENT=expert_tiered$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_HOST_POOL_MIB=49152$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_HOST_POOL_MIB=66560$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_KV_DTYPE=fp8$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_KV_DTYPE=fp8$' "$ENV_FILE"
|
||||||
grep -q '^FRANK_ARCHON_NGRAM_PLACEMENT=ram$' "$ENV_FILE"
|
grep -q '^FRANK_ARCHON_NGRAM_PLACEMENT=ram$' "$ENV_FILE"
|
||||||
# Prefix cache: pinned host RAM only (the disk tier was removed), and the
|
# Prefix cache: pinned host RAM only (the disk tier was removed), and the
|
||||||
|
|||||||
+11
-15
@@ -29,7 +29,7 @@ RUNNER="$REPO_ROOT/scripts/run-qwen38-bf16.sh"
|
|||||||
# The reference shape, spelled exactly as run-qwen38-bf16.sh must pass it: this
|
# The reference shape, spelled exactly as run-qwen38-bf16.sh must pass it: this
|
||||||
# is the contract the recorded reference is measured at, so a silent edit here
|
# is the contract the recorded reference is measured at, so a silent edit here
|
||||||
# is a silent edit to every KLD number in the campaign.
|
# is a silent edit to every KLD number in the campaign.
|
||||||
SHAPE="--tp 2 --placement expert_tiered --weights-disk-tier --host-pool-mib 49152 --gpu-headroom-mib 96 --deterministic --num-speculative-tokens 0 --kv-cache-dtype fp8 --max-model-len 32768 --ngram-placement ram --override-chat-template /models/tpl.jinja"
|
SHAPE="--tp 2 --placement expert_tiered --weights-disk-tier --host-pool-mib 66560 --gpu-headroom-mib 96 --deterministic --num-speculative-tokens 0 --kv-cache-dtype fp8 --max-model-len 32768 --ngram-placement ram --override-chat-template /models/tpl.jinja"
|
||||||
|
|
||||||
# oneline — both dry runs print one shell-quoted argument a line; joining them
|
# oneline — both dry runs print one shell-quoted argument a line; joining them
|
||||||
# lets an option and its value be asserted as the pair they are.
|
# lets an option and its value be asserted as the pair they are.
|
||||||
@@ -267,28 +267,24 @@ flag_value() {
|
|||||||
[ "$status" -eq 0 ]
|
[ "$status" -eq 0 ]
|
||||||
score_step="$(flag_value --max-num-batched-tokens <<< "$output")"
|
score_step="$(flag_value --max-num-batched-tokens <<< "$output")"
|
||||||
[ "$ref_step" = "$score_step" ]
|
[ "$ref_step" = "$score_step" ]
|
||||||
# one knob moves both, so the operator can lower it at 2 a.m. without an edit
|
# the step the argv quotes IS the number frank-archon.env owns — the campaign
|
||||||
run env FRANK_QWEN38_MODELS="$M" FRANK_QWEN38_CAMPAIGN="$C" FRANK_QWEN38_MNBT=1024 \
|
# used to carry its own FRANK_QWEN38_MNBT=1024, a per-mode literal that
|
||||||
bash "$RUNNER" kldref --dry-run
|
# measured a shape nothing serves; the alias and its override tests die with
|
||||||
[ "$(flag_value --max-num-batched-tokens <<< "$output")" = "1024" ]
|
# the shape change.
|
||||||
run env FRANK_QWEN38_MODELS="$M" FRANK_QWEN38_CAMPAIGN="$C" FRANK_QWEN38_MNBT=1024 \
|
env_step="$(awk -F= '/^FRANK_ARCHON_MAX_BATCHED_TOKENS=/{print $2}' \
|
||||||
bash "$RUNNER" score "$M/rad/arm-w5nl64.rad" "$C/scores/arm1.json" --dry-run
|
"$(dirname "$RUNNER")/frank-archon.env")"
|
||||||
[ "$(flag_value --max-num-batched-tokens <<< "$output")" = "1024" ]
|
[ "$ref_step" = "$env_step" ]
|
||||||
|
[ "$score_step" = "$env_step" ]
|
||||||
# calibration rides the shared step too. Its old exemption (8 seats at
|
# calibration rides the shared step too. Its old exemption (8 seats at
|
||||||
# MNBT 65,536, "free throughput") is what made the container refuse to
|
# MNBT 65,536, "free throughput") is what made the container refuse to
|
||||||
# boot: at that step the engine charges a 10.65 GiB activation arena to
|
# boot: at that step the engine charges a 10.65 GiB activation arena to
|
||||||
# the card, the experts fall to 10.98 GiB and the KV to 271 MiB —
|
# the card, the experts fall to 10.98 GiB and the KV to 271 MiB —
|
||||||
# "0.0 full-length sessions", "startup stopped: out of memory", measured
|
# "0.0 full-length sessions", "startup stopped: out of memory", measured
|
||||||
# 2026-10-29 while kldref on the shared step ran 16 hours without fault.
|
# 2026-10-29 while kldref on the shared step ran 16 hours without fault.
|
||||||
run env FRANK_QWEN38_MODELS="$M" FRANK_QWEN38_CAMPAIGN="$C" FRANK_QWEN38_MNBT=1024 \
|
run env FRANK_QWEN38_MODELS="$M" FRANK_QWEN38_CAMPAIGN="$C" \
|
||||||
bash "$RUNNER" calib --dry-run
|
bash "$RUNNER" calib --dry-run
|
||||||
[ "$(flag_value --max-num-batched-tokens <<< "$output")" = "1024" ]
|
[ "$(flag_value --max-num-batched-tokens <<< "$output")" = "$env_step" ]
|
||||||
[ "$(flag_value --max-num-seqs <<< "$output")" = "$ref_seats" ]
|
[ "$(flag_value --max-num-seqs <<< "$output")" = "$ref_seats" ]
|
||||||
# a step size that is not a token count is refused, never handed to the engine
|
|
||||||
run env FRANK_QWEN38_MNBT=broad bash "$RUNNER" kldref --dry-run
|
|
||||||
[ "$status" -eq 2 ]
|
|
||||||
[[ "$output" == *"FRANK_QWEN38_MNBT"* ]]
|
|
||||||
[[ "$output" == *"must be a token count"* ]]
|
|
||||||
}
|
}
|
||||||
|
|
||||||
@test "kldref refuses a corpus its window cannot hold, before the service is touched" {
|
@test "kldref refuses a corpus its window cannot hold, before the service is touched" {
|
||||||
|
|||||||
Reference in new issue
Block a user