campaign(qwen38): the tap producer is latest radiance, at a vendor-backed floor

The tap did not arm on the production pin: a 17-hour calib run on 1.8.1
wrote zero bytes while /health, /stats and the push all read healthy.
On :latest (v1.14.0, revision 4a61d7296bc5) the same boot prints
'recorded passes: off, the program accumulates calibration Grams
(gram_accum)' and a 9,129-token push sealed 24,768 gram files (42.9 GB)
in the fitting shape - 4 slots at a 2,048-token step, production's own.

The vendor's calibration shape (--max-num-batched-tokens 65536,
--max-num-seqs 64, ~930 tok/s) cannot boot two 32 GiB R9700s: the
activation arena alone is 12.83 GiB of the 27.95 GiB claimable, and
every variant stops in the VRAM resolver (short 3.19 GiB .. 86 MiB,
then a starved cache).

Floor moves 12M -> 1.5M tokens: the vendor's own measurement is 'a
pooled Gram from 10M tokens beat one from 1M by under 1%', and 1.5M is
hours at this rig's rate instead of the 29 h a 6M floor cost.

The converter moves to the same digest: rad-convert's gptq:calib=<dir>
input is version-stable, so a latest-written tap converts on latest.
This commit is contained in:
Frank committed 2026-10-30 15:02:58 -07:00
1 parent b4a1e6069e
commit 3e32cb9cd1
3 files changed
+31 -14

No files matched your search

+6 -1
View File
@@ -47,7 +47,12 @@
set -u -o pipefail
SELF="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
IMAGE=${IMAGE:-stilldeadcode/radiance@sha256:d590e300234bf6d2f942a0b8d456d183153fd808f28820e6c2fa69c9389178ed}
# THE TOOLCHAIN PIN. `:latest` on 2026-10-30 is v1.14.0 (revision 4a61d7296bc5),
# the same image the tap producer boots: a tap must be converted by the line that
# is current, and rad-convert's `gptq:calib=<dir>` input contract is version-stable
# (RADGRAM2 + gram.r<rank>.<stem>.bin), so nothing else about the recipe moves with
# it. Production stays on its own pin — this is the campaign's toolchain.
IMAGE=${IMAGE:-stilldeadcode/radiance@sha256:69add58f7de1ed2bca2b1e20a4cc75717557df3f73569a178eca5aafe8c44093}
CKPT=${CKPT:-/home/dank/models/Qwen3.8-Flash-Next}
CAMPAIGN=${CAMPAIGN:-/home/dank/campaign/qwen38-w5nl64}
RECIPE=${RECIPE:-$SELF/recipes/qwen4exp-qwen38-w5nl64-bf16trunk.recipe}
+20 -11
View File
@@ -25,13 +25,17 @@
# through the ro /models mount); OUT lands on a writable
# mount because --kld-out F also writes F.rows beside it.
# calib serve on :7770 with the calibration tap open
# (RADIANCE_CALIB_DIR + RADIANCE_CALIB_TOKENS=12000000). A
# (RADIANCE_CALIB_DIR + RADIANCE_CALIB_TOKENS=1500000). A
# separate driver — scripts/calib-push.py --port 7770
# --container qwen38-bf16 --corpus
# /home/dank/corpora/out/calib.jsonl — pushes the corpus; at
# 12M pooled positions the engine writes the Grams and the
# tap switches itself off (calib-push prints TAP_DONE). Stop
# this script after that; the trap brings frank-archon back.
# the floor the engine writes the Grams and the tap switches
# itself off (calib-push prints TAP_DONE). Stop this script
# after that; the trap brings frank-archon back. The vendor's
# own measurement is the floor's warrant — "a pooled Gram
# from 10 M tokens beat one from 1 M by under 1 %" — and at
# this rig's measured push rate a 1.5 M floor is hours, not
# the 29 h a 6 M floor cost. One seal writes ~40 GiB.
#
# Shape: the vendor's bf16 calibration lines — expert_tiered placement with the
# weights disk tier, deterministic (the resident set pinned to the loader's plan,
@@ -113,12 +117,17 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck disable=SC1091
source "$SCRIPT_DIR/frank-archon.env"
IMAGE="${FRANK_ARCHON_IMAGE:?FRANK_ARCHON_IMAGE must name the radiance image}"
# THE CALIBRATION PIN. RADIANCE_CALIB_DIR/TOKENS arms the expert tap on the 1.2.x
# line only. 1.8.1 still carries every plugin string and the seal message, and a
# 17-hour run on it wrote ZERO bytes into the tap dir while /health, /stats and
# the push all looked healthy (measured 2026-10-30) — so calib boots THIS digest
# and any other image must be named deliberately with --image.
CALIB_IMAGE="${Q38_CALIB_IMAGE:-stilldeadcode/radiance@sha256:665775ed1d90de6dbb25558e07fc60d20328f1346f96f3c6e00845eb763f01fa}"
# THE CALIBRATION PIN: the LATEST radiance, pinned by digest because a tag is a
# moving target and a tap must name the producer that wrote it. `:latest` on
# 2026-10-30 resolves to v1.14.0 (revision 4a61d7296bc5), and on that image a
# calib boot prints the arming line — "recorded passes: off, the program
# accumulates calibration Grams (gram_accum)" — where the production pin
# (1.8.1) prints nothing of the kind: a 17-hour run on 1.8.1 wrote ZERO bytes
# into the tap dir while /health, /stats and the push all looked healthy
# (measured 2026-10-30). The tap code is identical in both trees' sources, so
# the shipped 1.8.1 build is what drops it; do not spend another window on it.
# Any other image must be named deliberately with --image.
CALIB_IMAGE="${Q38_CALIB_IMAGE:-stilldeadcode/radiance@sha256:69add58f7de1ed2bca2b1e20a4cc75717557df3f73569a178eca5aafe8c44093}"
IMAGE_EXPLICIT=0
MODE=""
@@ -348,7 +357,7 @@ case "$MODE" in
}
MOUNTS+=(-v "$CALIB_DIR:/data/calib")
TAPENV=(-e RADIANCE_CALIB_DIR=/data/calib
-e "RADIANCE_CALIB_TOKENS=${FRANK_QWEN38_CALIB_TOKENS:-12000000}")
-e "RADIANCE_CALIB_TOKENS=${FRANK_QWEN38_CALIB_TOKENS:-1500000}")
# The tap's own authorization. A pusher must see this marker before it sends
# a prompt (calib-push.py --tap-marker), it names the container so a push
# aimed anywhere else refuses, and a boot that fails removes it again below.
+5 -2
View File
@@ -157,12 +157,15 @@ flag_value() {
[[ "$output" != *"--vram-kv-mib"* ]]
}
@test "the calib dry run opens the tap at 12M tokens and serves on 7770" {
@test "the calib dry run opens the tap at the 1.5M floor, on the latest pin, and serves on 7770" {
run bash "$RUNNER" calib --dry-run
[ "$status" -eq 0 ]
flat="$(oneline)"
[[ "$flat" == *"-e RADIANCE_CALIB_TOKENS=12000000"* ]]
[[ "$flat" == *"-e RADIANCE_CALIB_TOKENS=1500000"* ]]
[[ "$flat" == *"-e RADIANCE_CALIB_DIR=/data/calib"* ]]
# the tap producer is the campaign's own pin (latest radiance), not production's
[[ "$flat" == *"radiance@sha256:69add58f"* ]]
[[ "$flat" != *"radiance@sha256:d590e300"* ]]
[[ "$flat" == *"--port 7770"* ]]
[[ "$flat" == *":/data/calib "* ]]
# calib serves; it does not record