Frank
9aee00f2b3
- download-frank-primary-swift15.sh: ukisai/Swift1.5-Qwen3.8-Flash-Next (BF16, 131 shards + index + configs, 360037355038 B = 335.3 GiB, ungated) via the vendored hfget.sh; verify = listed sizes + 131-shard weight-map count + every shard's sha256 against the Hub LFS oid. Pattern cloned from scripts/download-frank-primary.sh. - qwen4exp-swift15-w4dn8-i8-hc8m.recipe: routed experts with DOWN at block-fp8 (rtn codes=fp8_e4m3 block=128x128 scale=bf16, the served FP8-native tile form; moe_gemm_fp8a8 reads it; minicpm5-2b is the shipped bf16->fp8a8 precedent) and gate_up at w4nl64 GPTQ cd=3; hc/MTP-fc fp8, trunk i8, lm_head i8, PLE fixed scale, draft_head u2 (inert). Protected-expert cast lines carry the BASE checkpoint's names as a warm start; the derivation step rewrites them from Swift's own calibration recording before the recipe is ever run (law documented in the file).
92 lines
6.8 KiB
Plaintext
92 lines
6.8 KiB
Plaintext
# qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next
|
|
# (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16, 335.3 GiB, key set identical to the official base) as
|
|
# a radiance container for frank-primary: every routed expert's DOWN projection at block-fp8 --
|
|
# the same tile form the served FP8-native container already holds (E4M3 codes, one bf16 scale a
|
|
# 128x128 block, the minicpm5-2b amax/448 rule) -- and its gate_up at four bits, the served recipe's
|
|
# w4nl64a8h encoding. The rest is qwen4exp-w5dn64-i8-hc8m.recipe verbatim: int8 trunk linears,
|
|
# block-fp8 hc mixers and MTP fc, the fixed-scale n-gram table, the 2-bit draft head.
|
|
#
|
|
# Down first because down is the larger share of an expert's output error (held-out relative error
|
|
# 0.094 against gate_up's 0.064 on the base model), and fp8 buys it a full binade for 2x the bytes
|
|
# of w4 -- while gate_up, three quarters of the expert's weights, stays at 4 bits. Expert mass:
|
|
# gate_up 150.00 GiB -> ~37.6 GiB, down 75.00 GiB -> ~37.7 GiB: ~75 GiB of experts, ~38 GiB a rank
|
|
# at TP=2, inside the 40960 MiB host pool. The published w5dn64 comparison (mean KL 0.0789 ->
|
|
# 0.0717, top-1 92.85% -> 93.34%, decode +4%, prefill -19%) is the 5-bit upper bound; this tier
|
|
# trades part of that quality back for prefill.
|
|
#
|
|
# CALIB=DIR rad-convert <Swift1.5-Qwen3.8-Flash-Next checkpoint> \
|
|
# --recipe scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe -o qwen4exp-swift15-w4dn8-i8-hc8m.rad
|
|
#
|
|
# DIR is the calibration directory the bf16 container wrote (RADIANCE_CALIB_DIR, 10M tokens of
|
|
# traffic-shaped corpus). Everything not named here is served as the checkpoint holds it (bf16:
|
|
# embeddings, the vision tower, the QSA indexer); the first rule that matches a weight decides it.
|
|
# The bf16 container is NOT passed with --reuse: the peak-disk arithmetic does not fit (checkpoint +
|
|
# container + container + output > purge-freed space), and --reuse only saves a re-read.
|
|
|
|
# The protected experts, cast to bf16 ahead of both expert rules. THE NAMES BELOW ARE THE BASE
|
|
# CHECKPOINT'S (from its 1M-token recording, qwen4exp-w5dn64-i8-hc8m.recipe) -- a warm start only.
|
|
# The Swift 1.5 finetune's routing distribution is not assumed equal: the derivation step rewrites
|
|
# this section from Swift's OWN calibration recording before this recipe is ever run, by the base's
|
|
# law -- an expert whose share of its layer's down-input energy is at least 3% and at least three
|
|
# times its share of the routing, an even count on each rank's half of the layer (the four-bit GEMM
|
|
# takes the rest as two equal tables by parity) by adding that half's next most energetic expert.
|
|
blk.34.ffn_*_exps.407.weight cast dtype=bf16
|
|
blk.34.ffn_*_exps.496.weight cast dtype=bf16
|
|
blk.44.ffn_*_exps.292.weight cast dtype=bf16
|
|
blk.44.ffn_*_exps.350.weight cast dtype=bf16
|
|
blk.46.ffn_*_exps.290.weight cast dtype=bf16
|
|
blk.46.ffn_*_exps.392.weight cast dtype=bf16
|
|
blk.47.ffn_*_exps.122.weight cast dtype=bf16
|
|
blk.47.ffn_*_exps.143.weight cast dtype=bf16
|
|
blk.47.ffn_*_exps.399.weight cast dtype=bf16
|
|
blk.47.ffn_*_exps.445.weight cast dtype=bf16
|
|
|
|
# Every other routed expert, trunk and MTP layer alike: the down projection as E4M3 codes with a
|
|
# bf16 scale a 128x128 block -- one amax per block, scale = amax/448 rounded to bf16, the exact
|
|
# tile form modelopt ships and the served FP8-native container holds, so the moe_gemm_fp8a8 kernel
|
|
# reads it unrep permuted; and the gate_up as u4 codes into the w4nl table, rotated by the 128-wide
|
|
# Hadamard, an E4M3 scale a group of 64 under the fixed 2^-13 searched for least error as stored,
|
|
# chosen by GPTQ against $CALIB and swept three times by coordinate descent. The down shapes tile
|
|
# exactly: [2560, 640] -> 20x5 blocks, [1280, 2560] -> 10x20. The MTP layer has no recording, so
|
|
# its down is rounded to nearest anyway and its gate_up quantises uncalibrated -- both said on
|
|
# stderr, as the base recipe says.
|
|
blk.*.ffn_down_exps.*.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
|
|
blk.*.ffn_*_exps.*.weight gptq table=w4nl group=64 scale=fp8_e4m3 scale2=f32 block2=*x* scale2_value=0.0001220703125 transform=fwht128 rule=search cd=3 calib=$CALIB
|
|
|
|
# Every connection's two mixing matrices -- the 96 trunk connections, the final mixer and the MTP
|
|
# head's three -- as E4M3 rows with an f32 scale a group of each row. A 128x128 block cannot cover
|
|
# them: lowrank is 320, 2.5 blocks. Down is [320, 10240] and takes a scale every 128 columns; up is
|
|
# [10240, 320] and takes four scales a row, 80 columns each.
|
|
*_hc_down.weight rtn codes=fp8_e4m3 group=128 scale=f32
|
|
*_hc_up.weight rtn codes=fp8_e4m3 group=80 scale=f32
|
|
|
|
# The MTP head's fc_hidden and fc_embedding, the same rows. They only ever change the drafts.
|
|
mtp.fc_hidden.weight rtn codes=fp8_e4m3 group=128 scale=f32
|
|
mtp.fc_embedding.weight rtn codes=fp8_e4m3 group=128 scale=f32
|
|
|
|
# The trunk's block linears as int8, a scale per 128 columns of a row searched for least error.
|
|
# The QSA indexer's projection is not one of them: its top-k is a hard choice between blocks, and
|
|
# keeping it as the checkpoint holds it costs 20 MB.
|
|
blk.*.attn_qg.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
|
blk.*.attn_k.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
|
blk.*.attn_v.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
|
blk.*.attn_output.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
|
blk.*.ssm_inz.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
|
blk.*.ssm_out.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
|
blk.*.ffn_gate_up_shexp.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
|
blk.*.ffn_down_shexp.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
|
|
|
# The lm_head, int8 with a bf16 scale a row per 128 columns (logits_gemm_i8): at E4M3 it costs
|
|
# ~0.6 points of top-1 on its own.
|
|
output.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
|
|
|
# The n-gram table at one fixed E4M3 scale; qwen4exp-w4.recipe says where the value comes from.
|
|
# The value sits on a three-octave plateau (2^-13..2^-10, rel_l2 0.02662) with 1.7x headroom over
|
|
# the base table's absmax; the prep phase verifies Swift's PLE absmax lands inside the plateau
|
|
# before this rule is trusted.
|
|
blk.*.ple_ngram.weight rtn codes=fp8_e4m3 block=*x* scale=bf16 rule=fixed scale_value=1.99317932128906e-4
|
|
|
|
# The MTP head's 2-bit draft head. INERT for this checkpoint: the HF weight map has no
|
|
# mtp.draft_head.weight (same as the base), kept for shape parity with the family.
|
|
mtp.draft_head.weight rtn codes=u2 zero=u8 group=128 scale=f16
|