feat(qwen38): the w5nl64 recipe template and the three-arm convert driver
One template (every routed expert at five bits, E4M3 scales in 64-groups, the top level rotated through the 128-wide Hadamard chosen by GPTQ against CALIB) plus a driver that generates the per-arm expert row, so the other 60 lines can never drift between arms: control (cd=3), cd=6, and act_order=1 with NO cd= - libquant refuses the pair (lq_registry.cpp:105), so the AoA arm is its own sweep, not a fourth column. The header carries the campaign's order (fetch -> build reference -> one drained window for the calibration tap and the KLD reference -> trash the bf16 container -> imatrix -> arms -> fetch the FP8 anchor last -> gate -> trio edit), because that sequence is point-of-use for whoever runs it next, and the disk sequencing is forced: 754 GiB free does not hold checkpoint + reference + three arms. Gates that bite before 5h13m of convert time is spent: the BF16 reference still occupying the volume, a recording too thin to be a calibration (converting uncalibrated is silently worse and only whispers on stderr), an imatrix outside the mounted campaign dir, an arm that already exists, a protected derivation that cannot land its target, and a destination that cannot hold the output. Each arm runs --plan-only first so the engine's own fit check bills at plan time; the served container's argv shape is what the arms declare.
This commit is contained in:
1 parent
3d8d18a45c
commit
4e551fafd7
2 files changed
+301
No files matched your search
@@ -0,0 +1,80 @@
|
||||
# qwen4exp-qwen38-w5nl64-bf16trunk.recipe — every routed expert, BOTH projections, at five bits:
|
||||
# u5 codes into libquant's 32-level w5nl table, an E4M3 scale a group of 64 under the fixed 2^-13
|
||||
# second level, rotated by the 128-wide Hadamard, chosen by GPTQ against $CALIB and swept three
|
||||
# times by coordinate descent. This is `w5nl64a8h` in the engine's served-encoding table — the top of
|
||||
# the four-bit class — applied wider than any published radiance container of this checkpoint: the
|
||||
# author's own w5 rung (`qwen4exp-w5dn64-i8-hc8m.recipe`) moves only `down`, because that rung was
|
||||
# priced as a byte-for-error trade against prefill; this campaign buys the whole rung and pays for
|
||||
# it out of the pinned host pool (+15.3 GiB of expert plane, `--host-pool-mib 61440` per rank), not
|
||||
# out of the KV pool.
|
||||
#
|
||||
# Every other line is the served container's recipe (dump it with
|
||||
# `rad-info --recipe /home/dank/models/rad/qwen3.8-next-flash-fp8-iq4r-moe.rad`), minus the rows this
|
||||
# campaign leaves at the checkpoint:
|
||||
#
|
||||
# * the trunk's block linears, the shared expert and the hyper-connection mixers are NOT listed —
|
||||
# dropping their i8/fp8 lines leaves them bf16, which is what the checkpoint holds. Measured bill
|
||||
# +1.978 GiB of always-resident weight = ~131k tokens of KV off the pool.
|
||||
# * `ssm_inz`/`ssm_out` STAY int8: the delta-net gates are the largest trunk rows (+1.9 GiB, 126k
|
||||
# tokens) and the least sensitive per byte.
|
||||
# * `mtp.draft_head` STAYS 2-bit and the PLE n-gram table STAYS one fixed E4M3 scale: the draft
|
||||
# head's rung is cheap and its acceptance cost is a decode question this campaign does not touch,
|
||||
# and bf16 for the n-gram table would pin +47.69 GiB of host RAM for a lookup.
|
||||
#
|
||||
# Protected experts: none in this file. `scripts/apply-protected-cast.sh` appends the derived set
|
||||
# (48, two per layer so each rank's 256-expert half stays even — the four-bit GEMM takes the rest of
|
||||
# a half as two equal tables by parity) from the fresh 12M-token energy/routing census, and they must
|
||||
# sit ABOVE the expert row because the first rule that matches a weight decides it.
|
||||
#
|
||||
# CALIB=/home/dank/campaign/qwen38-w5nl64/calib \
|
||||
# rad-convert /home/dank/models/Qwen3.8-Flash-Next \
|
||||
# --recipe this.file --imatrix .../imatrix.gguf -o out.rad
|
||||
#
|
||||
# The arm driver (`scripts/convert-qwen38-w5nl64.sh`) generates the per-arm variants of the expert
|
||||
# row from this template — cd=3 (control), cd=6, and `act_order=1` with NO cd= (libquant refuses the
|
||||
# pair, lq_registry.cpp:105) — rather than keeping three near-duplicate recipes that can drift apart.
|
||||
#
|
||||
# The whole campaign, in the order that fits on one disk (every step runs on dankserver):
|
||||
#
|
||||
# 1. scripts/download-qwen38-bf16.sh --fetch the vendor BF16 checkpoint, 335.3 GiB
|
||||
# 2. scripts/build-qwen38-bf16-rad.sh the BF16 reference container, another 335.3 GiB
|
||||
# 3. one drained window, two boots:
|
||||
# scripts/run-qwen38-bf16.sh kldref the KLD reference recording (the zero of scale)
|
||||
# scripts/run-qwen38-bf16.sh calib the 12M-position calibration tap -> grams
|
||||
# 4. trash qwen38-bf16.rad the peak-disk law: the converts need its room
|
||||
# 5. scripts/grams-to-imatrix.py grams -> imatrix.gguf in the campaign dir
|
||||
# 6. scripts/convert-qwen38-w5nl64.sh the arms, planned and billed before each one runs
|
||||
# 7. fetch the published FP8-native release, then per container:
|
||||
# scripts/run-qwen38-bf16.sh score RAD OUT fp8 anchor, served iq4r, each arm
|
||||
# then scripts/kld-gate-qwen38.sh fp8.json iq4r.json cand.json
|
||||
# 8. a winner is a trio edit plus `scripts/deploy.sh dankserver --service frank-archon`; the
|
||||
# rollback is the commit before it. If the win needs the bigger pinned plane, raise
|
||||
# FRANK_ARCHON_HOST_POOL_MIB and FRANK_ARCHON_TTM_TARGET_MIB in the SAME commit: the preflight
|
||||
# floor is 2xHOST_POOL + 48,824 + prefix cache + MNBT + the store, and a pool raised without
|
||||
# the TTM target is a boot refusal that Restart=always repeats forever, invisible to deploy.sh.
|
||||
|
||||
# PROTECTED-BEGIN — scripts/apply-protected-cast.sh splices the derived cast lines in here.
|
||||
# They MUST sit above the expert rule: the first rule whose glob matches a weight decides it, so a
|
||||
# cast line below the five-bit rule would never win. The set is derived from the campaign's own
|
||||
# 12M-token recording (`derive-protected-experts.py --target-count 48` over the fresh grams), and the
|
||||
# driver splices it into a per-arm COPY so this template stays the clean, reviewable baseline.
|
||||
# PROTECTED-END
|
||||
|
||||
# Every routed expert, trunk and MTP layer alike, both projections, at five bits. The experts' scales
|
||||
# sit between 2^-15 and 2^-10.6 under w4nl's top level of 22 and half a binade lower under w5nl's 30
|
||||
# — inside E4M3's normal range under 2^-13 either way. The MTP layer has no recording and is rounded
|
||||
# to nearest, which libquant says on stderr; it is one layer of 49 and only ever changes drafts.
|
||||
blk.*.ffn_*_exps.*.weight gptq table=w5nl group=64 scale=fp8_e4m3 scale2=f32 block2=*x* scale2_value=0.0001220703125 transform=fwht128 rule=search cd=3 calib=$CALIB
|
||||
|
||||
# The delta-net's in/out gates, int8 with a bf16 scale a group of 128 searched for least error — the
|
||||
# served recipe's lines, unchanged.
|
||||
blk.*.ssm_inz.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
||||
blk.*.ssm_out.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
||||
|
||||
# The MTP head's two mixing rows, E4M3 as served. They only ever change the drafts.
|
||||
mtp.fc_hidden.weight rtn codes=fp8_e4m3 group=128 scale=f32
|
||||
mtp.fc_embedding.weight rtn codes=fp8_e4m3 group=128 scale=f32
|
||||
|
||||
# The n-gram table at one fixed E4M3 scale, and the 2-bit draft head — both the served lines.
|
||||
blk.*.ple_ngram.weight rtn codes=fp8_e4m3 block=*x* scale=bf16 rule=fixed scale_value=1.99317932128906e-4
|
||||
mtp.draft_head.weight rtn codes=u2 zero=u8 group=128 scale=f16
|
||||
Reference in new issue
Block a user