# qwen4exp-qwen38-w5nl64-bf16trunk.recipe — every routed expert, BOTH projections, at five bits: # u5 codes into libquant's 32-level w5nl table, an E4M3 scale a group of 64 under the fixed 2^-13 # second level, rotated by the 128-wide Hadamard, chosen by GPTQ against $CALIB and swept three # times by coordinate descent. This is `w5nl64a8h` in the engine's served-encoding table — the top of # the four-bit class — applied wider than any published radiance container of this checkpoint: the # author's own w5 rung (`qwen4exp-w5dn64-i8-hc8m.recipe`) moves only `down`, because that rung was # priced as a byte-for-error trade against prefill; this campaign buys the whole rung and pays for # it out of the pinned host pool (+15.3 GiB of expert plane, `--host-pool-mib 61440` per rank), not # out of the KV pool. # # Every other line is the served container's recipe (dump it with # `rad-info --recipe /home/dank/models/rad/qwen3.8-next-flash-fp8-iq4r-moe.rad`), minus the rows this # campaign leaves at the checkpoint: # # * the trunk's block linears, the shared expert and the hyper-connection mixers are NOT listed — # dropping their i8/fp8 lines leaves them bf16, which is what the checkpoint holds. Measured bill # +1.978 GiB of always-resident weight = ~131k tokens of KV off the pool. # * `ssm_inz`/`ssm_out` STAY int8: the delta-net gates are the largest trunk rows (+1.9 GiB, 126k # tokens) and the least sensitive per byte. # * `mtp.draft_head` STAYS 2-bit and the PLE n-gram table STAYS one fixed E4M3 scale: the draft # head's rung is cheap and its acceptance cost is a decode question this campaign does not touch, # and bf16 for the n-gram table would pin +47.69 GiB of host RAM for a lookup. # # Protected experts: none in this file. `scripts/apply-protected-cast.sh` appends the derived set # (48, two per layer so each rank's 256-expert half stays even — the four-bit GEMM takes the rest of # a half as two equal tables by parity) from the fresh 12M-token energy/routing census, and they must # sit ABOVE the expert row because the first rule that matches a weight decides it. # # CALIB=/home/dank/campaign/qwen38-w5nl64/calib \ # rad-convert /home/dank/models/Qwen3.8-Flash-Next \ # --recipe this.file --imatrix .../imatrix.gguf -o out.rad # # The arm driver (`scripts/convert-qwen38-w5nl64.sh`) generates the per-arm variants of the expert # row from this template — cd=3 (control), cd=6, and `act_order=1` with NO cd= (libquant refuses the # pair, lq_registry.cpp:105) — rather than keeping three near-duplicate recipes that can drift apart. # # The whole campaign, in the order that fits on one disk (every step runs on dankserver): # # 1. scripts/download-qwen38-bf16.sh --fetch the vendor BF16 checkpoint, 335.3 GiB # 2. scripts/build-qwen38-bf16-rad.sh the BF16 reference container, another 335.3 GiB # 3. one drained window, two boots: # scripts/run-qwen38-bf16.sh kldref the KLD reference recording (the zero of scale) # scripts/run-qwen38-bf16.sh calib the 12M-position calibration tap -> grams # 4. trash qwen38-bf16.rad the peak-disk law: the converts need its room # 5. scripts/grams-to-imatrix.py grams -> imatrix.gguf in the campaign dir # 6. scripts/convert-qwen38-w5nl64.sh the arms, planned and billed before each one runs # 7. fetch the published FP8-native release, then per container: # scripts/run-qwen38-bf16.sh score RAD OUT fp8 anchor, served iq4r, each arm # then scripts/kld-gate-qwen38.sh fp8.json iq4r.json cand.json # 8. a winner is a trio edit plus `scripts/deploy.sh dankserver --service frank-archon`; the # rollback is the commit before it. If the win needs the bigger pinned plane, raise # FRANK_ARCHON_HOST_POOL_MIB and FRANK_ARCHON_TTM_TARGET_MIB in the SAME commit: the preflight # floor is 2xHOST_POOL + 48,824 + prefix cache + MNBT + the store, and a pool raised without # the TTM target is a boot refusal that Restart=always repeats forever, invisible to deploy.sh. # PROTECTED-BEGIN — scripts/apply-protected-cast.sh splices the derived cast lines in here. # They MUST sit above the expert rule: the first rule whose glob matches a weight decides it, so a # cast line below the five-bit rule would never win. The set is derived from the campaign's own # 12M-token recording (`derive-protected-experts.py --target-count 48` over the fresh grams), and the # driver splices it into a per-arm COPY so this template stays the clean, reviewable baseline. # PROTECTED-END # Every routed expert, trunk and MTP layer alike, both projections, at five bits. The experts' scales # sit between 2^-15 and 2^-10.6 under w4nl's top level of 22 and half a binade lower under w5nl's 30 # — inside E4M3's normal range under 2^-13 either way. The MTP layer has no recording and is rounded # to nearest, which libquant says on stderr; it is one layer of 49 and only ever changes drafts. blk.*.ffn_*_exps.*.weight gptq table=w5nl group=64 scale=fp8_e4m3 scale2=f32 block2=*x* scale2_value=0.0001220703125 transform=fwht128 rule=search cd=3 calib=$CALIB # The delta-net's in/out gates, int8 with a bf16 scale a group of 128 searched for least error — the # served recipe's lines, unchanged. blk.*.ssm_inz.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search blk.*.ssm_out.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search # The MTP head's two mixing rows, E4M3 as served. They only ever change the drafts. mtp.fc_hidden.weight rtn codes=fp8_e4m3 group=128 scale=f32 mtp.fc_embedding.weight rtn codes=fp8_e4m3 group=128 scale=f32 # The n-gram table at one fixed E4M3 scale, and the 2-bit draft head — both the served lines. blk.*.ple_ngram.weight rtn codes=fp8_e4m3 block=*x* scale=bf16 rule=fixed scale_value=1.99317932128906e-4 mtp.draft_head.weight rtn codes=u2 zero=u8 group=128 scale=f16