dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

swift15: fetch script + w4dn8 expert recipe for the frank-primary campaign

Browse Source 
- download-frank-primary-swift15.sh: ukisai/Swift1.5-Qwen3.8-Flash-Next
  (BF16, 131 shards + index + configs, 360037355038 B = 335.3 GiB, ungated)
  via the vendored hfget.sh; verify = listed sizes + 131-shard weight-map
  count + every shard's sha256 against the Hub LFS oid. Pattern cloned
  from scripts/download-frank-primary.sh.
- qwen4exp-swift15-w4dn8-i8-hc8m.recipe: routed experts with DOWN at
  block-fp8 (rtn codes=fp8_e4m3 block=128x128 scale=bf16, the served
  FP8-native tile form; moe_gemm_fp8a8 reads it; minicpm5-2b is the
  shipped bf16->fp8a8 precedent) and gate_up at w4nl64 GPTQ cd=3;
  hc/MTP-fc fp8, trunk i8, lm_head i8, PLE fixed scale, draft_head u2
  (inert). Protected-expert cast lines carry the BASE checkpoint's names
  as a warm start; the derivation step rewrites them from Swift's own
  calibration recording before the recipe is ever run (law documented
  in the file).
This commit is contained in:
Frank
2026-10-06 19:19:40 -07:00
parent 41d2f56269
commit 9aee00f2b3
2 changed files with 204 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe
+91
View File
@@ -0,0 +1,91 @@
# qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next
# (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16, 335.3 GiB, key set identical to the official base) as
# a radiance container for frank-primary: every routed expert's DOWN projection at block-fp8 --
# the same tile form the served FP8-native container already holds (E4M3 codes, one bf16 scale a
# 128x128 block, the minicpm5-2b amax/448 rule) -- and its gate_up at four bits, the served recipe's
# w4nl64a8h encoding. The rest is qwen4exp-w5dn64-i8-hc8m.recipe verbatim: int8 trunk linears,
# block-fp8 hc mixers and MTP fc, the fixed-scale n-gram table, the 2-bit draft head.
#
# Down first because down is the larger share of an expert's output error (held-out relative error
# 0.094 against gate_up's 0.064 on the base model), and fp8 buys it a full binade for 2x the bytes
# of w4 -- while gate_up, three quarters of the expert's weights, stays at 4 bits. Expert mass:
# gate_up 150.00 GiB -> ~37.6 GiB, down 75.00 GiB -> ~37.7 GiB: ~75 GiB of experts, ~38 GiB a rank
# at TP=2, inside the 40960 MiB host pool. The published w5dn64 comparison (mean KL 0.0789 ->
# 0.0717, top-1 92.85% -> 93.34%, decode +4%, prefill -19%) is the 5-bit upper bound; this tier
# trades part of that quality back for prefill.
#
# CALIB=DIR rad-convert <Swift1.5-Qwen3.8-Flash-Next checkpoint> \
# --recipe scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe -o qwen4exp-swift15-w4dn8-i8-hc8m.rad
#
# DIR is the calibration directory the bf16 container wrote (RADIANCE_CALIB_DIR, 10M tokens of
# traffic-shaped corpus). Everything not named here is served as the checkpoint holds it (bf16:
# embeddings, the vision tower, the QSA indexer); the first rule that matches a weight decides it.
# The bf16 container is NOT passed with --reuse: the peak-disk arithmetic does not fit (checkpoint +
# container + container + output > purge-freed space), and --reuse only saves a re-read.
# The protected experts, cast to bf16 ahead of both expert rules. THE NAMES BELOW ARE THE BASE
# CHECKPOINT'S (from its 1M-token recording, qwen4exp-w5dn64-i8-hc8m.recipe) -- a warm start only.
# The Swift 1.5 finetune's routing distribution is not assumed equal: the derivation step rewrites
# this section from Swift's OWN calibration recording before this recipe is ever run, by the base's
# law -- an expert whose share of its layer's down-input energy is at least 3% and at least three
# times its share of the routing, an even count on each rank's half of the layer (the four-bit GEMM
# takes the rest as two equal tables by parity) by adding that half's next most energetic expert.
blk.34.ffn_*_exps.407.weight cast dtype=bf16
blk.34.ffn_*_exps.496.weight cast dtype=bf16
blk.44.ffn_*_exps.292.weight cast dtype=bf16
blk.44.ffn_*_exps.350.weight cast dtype=bf16
blk.46.ffn_*_exps.290.weight cast dtype=bf16
blk.46.ffn_*_exps.392.weight cast dtype=bf16
blk.47.ffn_*_exps.122.weight cast dtype=bf16
blk.47.ffn_*_exps.143.weight cast dtype=bf16
blk.47.ffn_*_exps.399.weight cast dtype=bf16
blk.47.ffn_*_exps.445.weight cast dtype=bf16
# Every other routed expert, trunk and MTP layer alike: the down projection as E4M3 codes with a
# bf16 scale a 128x128 block -- one amax per block, scale = amax/448 rounded to bf16, the exact
# tile form modelopt ships and the served FP8-native container holds, so the moe_gemm_fp8a8 kernel
# reads it unrep permuted; and the gate_up as u4 codes into the w4nl table, rotated by the 128-wide
# Hadamard, an E4M3 scale a group of 64 under the fixed 2^-13 searched for least error as stored,
# chosen by GPTQ against $CALIB and swept three times by coordinate descent. The down shapes tile
# exactly: [2560, 640] -> 20x5 blocks, [1280, 2560] -> 10x20. The MTP layer has no recording, so
# its down is rounded to nearest anyway and its gate_up quantises uncalibrated -- both said on
# stderr, as the base recipe says.
blk.*.ffn_down_exps.*.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
blk.*.ffn_*_exps.*.weight gptq table=w4nl group=64 scale=fp8_e4m3 scale2=f32 block2=*x* scale2_value=0.0001220703125 transform=fwht128 rule=search cd=3 calib=$CALIB
# Every connection's two mixing matrices -- the 96 trunk connections, the final mixer and the MTP
# head's three -- as E4M3 rows with an f32 scale a group of each row. A 128x128 block cannot cover
# them: lowrank is 320, 2.5 blocks. Down is [320, 10240] and takes a scale every 128 columns; up is
# [10240, 320] and takes four scales a row, 80 columns each.
*_hc_down.weight rtn codes=fp8_e4m3 group=128 scale=f32
*_hc_up.weight rtn codes=fp8_e4m3 group=80 scale=f32
# The MTP head's fc_hidden and fc_embedding, the same rows. They only ever change the drafts.
mtp.fc_hidden.weight rtn codes=fp8_e4m3 group=128 scale=f32
mtp.fc_embedding.weight rtn codes=fp8_e4m3 group=128 scale=f32
# The trunk's block linears as int8, a scale per 128 columns of a row searched for least error.
# The QSA indexer's projection is not one of them: its top-k is a hard choice between blocks, and
# keeping it as the checkpoint holds it costs 20 MB.
blk.*.attn_qg.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.attn_k.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.attn_v.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.attn_output.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.ssm_inz.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.ssm_out.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.ffn_gate_up_shexp.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
blk.*.ffn_down_shexp.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
# The lm_head, int8 with a bf16 scale a row per 128 columns (logits_gemm_i8): at E4M3 it costs
# ~0.6 points of top-1 on its own.
output.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
# The n-gram table at one fixed E4M3 scale; qwen4exp-w4.recipe says where the value comes from.
# The value sits on a three-octave plateau (2^-13..2^-10, rel_l2 0.02662) with 1.7x headroom over
# the base table's absmax; the prep phase verifies Swift's PLE absmax lands inside the plateau
# before this rule is trusted.
blk.*.ple_ngram.weight rtn codes=fp8_e4m3 block=*x* scale=bf16 rule=fixed scale_value=1.99317932128906e-4
# The MTP head's 2-bit draft head. INERT for this checkpoint: the HF weight map has no
# mtp.draft_head.weight (same as the base), kept for shape parity with the family.
mtp.draft_head.weight rtn codes=u2 zero=u8 group=128 scale=f16