Frank
55f87fcdce
The 2026-10-09 wire aliases become the service names everywhere: systemd units, serve scripts, env files and FRANK_* variables, deploy-plan/deploy-remote service keys, container names, Prometheus job names + alert rules, Grafana dashboard queries, bats seams, runbook filenames, homepage/AGENTS lore, and historical prose. Rollback tags renamed in lockstep. Residue in the tree: zero. Retired frank-medium/small/npu names are history and stay. Known casualty: scripts/patches/*.patch context lines were rewritten; those retired-era vLLM-fork patches no longer apply byte-exact.
106 lines
7.3 KiB
Plaintext
106 lines
7.3 KiB
Plaintext
# qwen4exp-swift15-w4dn8-i8-hc8m.recipe -- the Swift 1.5 finetune of Qwen3.8-Flash-Next
|
|
# (ukisai/Swift1.5-Qwen3.8-Flash-Next, BF16 source, 335.3 GiB, key set identical to the official
|
|
# base) as a radiance container for frank-archon: EVERY routed expert -- gate_up AND down --
|
|
# through GPTQ into the w4nl64a8h table. The plugin's one-table-one-encoding law
|
|
# ("one table of experts takes one encoding, and only a whole expert kept plain bf16 is served
|
|
# beside it", qwen4exp_fp8 rad_arch_declare) forbids the original split tier (down block-fp8 +
|
|
# gate_up w4): the served kernel is w4nl64a8h for the whole expert set, bf16 experts beside it.
|
|
# The rest is qwen4exp-w5dn64-i8-hc8m.recipe verbatim: int8 trunk linears, block-fp8 hc mixers
|
|
# and MTP fc, the fixed-scale n-gram table, the 2-bit draft head.
|
|
#
|
|
# Why down is the calibrated half: the calibration tap rides the down projection, so its GPTQ
|
|
# rounding propagates against real down-input Hessians; gate_up shares the same w4 table
|
|
# uncalibrated. The published w5dn64 comparison (mean KL 0.0789 -> 0.0717, top-1 92.85% ->
|
|
# 93.34%, decode +4%, prefill -19%) is the 5-bit upper bound; this tier trades part of that
|
|
# quality back for prefill.
|
|
#
|
|
# CALIB=DIR rad-convert <Swift1.5-Qwen3.8-Flash-Next checkpoint> \
|
|
# --recipe scripts/recipes/qwen4exp-swift15-w4dn8-i8-hc8m.recipe -o qwen4exp-swift15-w4dn8-i8-hc8m.rad
|
|
#
|
|
# DIR is the calibration directory the bf16 container wrote (RADIANCE_CALIB_DIR, 10M tokens of
|
|
# traffic-shaped corpus). Everything not named here is served as the checkpoint holds it (bf16:
|
|
# embeddings, the vision tower, the QSA indexer); the first rule that matches a weight decides it.
|
|
# The bf16 container is NOT passed with --reuse: the peak-disk arithmetic does not fit (checkpoint +
|
|
# container + container + output > purge-freed space), and --reuse only saves a re-read.
|
|
|
|
# The protected experts, cast to bf16 ahead of both expert rules. THE NAMES BELOW ARE THE BASE
|
|
# CHECKPOINT'S (from its 1M-token recording, qwen4exp-w5dn64-i8-hc8m.recipe) -- a warm start only.
|
|
# The Swift 1.5 finetune's routing distribution is not assumed equal: the derivation step rewrites
|
|
# this section from Swift's OWN calibration recording before this recipe is ever run, by the base's
|
|
# law -- an expert whose share of its layer's down-input energy is at least 3% and at least three
|
|
# times its share of the routing, an even count on each rank's half of the layer (the four-bit GEMM
|
|
# takes the rest as two equal tables by parity) by adding that half's next most energetic expert.
|
|
# PROTECTED-BEGIN (rewritten from Swift's OWN grams by derive-protected-experts.py)
|
|
blk.1.ffn_*_exps.271.weight cast dtype=bf16
|
|
blk.1.ffn_*_exps.355.weight cast dtype=bf16
|
|
blk.2.ffn_*_exps.279.weight cast dtype=bf16
|
|
blk.2.ffn_*_exps.413.weight cast dtype=bf16
|
|
blk.34.ffn_*_exps.146.weight cast dtype=bf16
|
|
blk.34.ffn_*_exps.153.weight cast dtype=bf16
|
|
blk.34.ffn_*_exps.407.weight cast dtype=bf16
|
|
blk.34.ffn_*_exps.453.weight cast dtype=bf16
|
|
blk.37.ffn_*_exps.288.weight cast dtype=bf16
|
|
blk.37.ffn_*_exps.377.weight cast dtype=bf16
|
|
blk.38.ffn_*_exps.406.weight cast dtype=bf16
|
|
blk.38.ffn_*_exps.420.weight cast dtype=bf16
|
|
blk.42.ffn_*_exps.37.weight cast dtype=bf16
|
|
blk.42.ffn_*_exps.90.weight cast dtype=bf16
|
|
blk.42.ffn_*_exps.302.weight cast dtype=bf16
|
|
blk.42.ffn_*_exps.388.weight cast dtype=bf16
|
|
blk.44.ffn_*_exps.315.weight cast dtype=bf16
|
|
blk.44.ffn_*_exps.350.weight cast dtype=bf16
|
|
blk.46.ffn_*_exps.290.weight cast dtype=bf16
|
|
blk.46.ffn_*_exps.419.weight cast dtype=bf16
|
|
blk.47.ffn_*_exps.119.weight cast dtype=bf16
|
|
blk.47.ffn_*_exps.143.weight cast dtype=bf16
|
|
blk.47.ffn_*_exps.399.weight cast dtype=bf16
|
|
blk.47.ffn_*_exps.445.weight cast dtype=bf16
|
|
# PROTECTED-END
|
|
# (derive-protected-experts.py rewrote this section from Swift's own grams)
|
|
|
|
# Every other routed expert, trunk and MTP layer alike: u4 codes into the w4nl table, rotated by
|
|
# the 128-wide Hadamard, an E4M3 scale a group of 64 under the fixed 2^-13 searched for least
|
|
# error as stored, chosen by GPTQ against $CALIB and swept three times by coordinate descent.
|
|
# The calibration Grams are down-input Hessians, so down is the calibrated half of the table;
|
|
# gate_up has no recording of its own and quantises uncalibrated (said on stderr, as the base
|
|
# recipe says). The MTP layer has no recording either. Expert mass: gate_up 150.00 GiB -> ~37.6,
|
|
# down 75.00 GiB -> ~18.8: ~56 GiB of experts, ~28 GiB a rank at TP=2 -- well inside the pool.
|
|
blk.*.ffn_*_exps.*.weight gptq table=w4nl group=64 scale=fp8_e4m3 scale2=f32 block2=*x* scale2_value=0.0001220703125 transform=fwht128 rule=search cd=3 calib=$CALIB
|
|
|
|
# Every connection's two mixing matrices -- the 96 trunk connections, the final mixer and the MTP
|
|
# head's three -- as E4M3 rows with an f32 scale a group of each row. A 128x128 block cannot cover
|
|
# them: lowrank is 320, 2.5 blocks. Down is [320, 10240] and takes a scale every 128 columns; up is
|
|
# [10240, 320] and takes four scales a row, 80 columns each.
|
|
*_hc_down.weight rtn codes=fp8_e4m3 group=128 scale=f32
|
|
*_hc_up.weight rtn codes=fp8_e4m3 group=80 scale=f32
|
|
|
|
# The MTP head's fc_hidden and fc_embedding, the same rows. They only ever change the drafts.
|
|
mtp.fc_hidden.weight rtn codes=fp8_e4m3 group=128 scale=f32
|
|
mtp.fc_embedding.weight rtn codes=fp8_e4m3 group=128 scale=f32
|
|
|
|
# The trunk's block linears as int8, a scale per 128 columns of a row searched for least error.
|
|
# The QSA indexer's projection is not one of them: its top-k is a hard choice between blocks, and
|
|
# keeping it as the checkpoint holds it costs 20 MB.
|
|
blk.*.attn_qg.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
|
blk.*.attn_k.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
|
blk.*.attn_v.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
|
blk.*.attn_output.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
|
blk.*.ssm_inz.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
|
blk.*.ssm_out.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
|
blk.*.ffn_gate_up_shexp.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
|
blk.*.ffn_down_shexp.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
|
|
|
# The lm_head, int8 with a bf16 scale a row per 128 columns (logits_gemm_i8): at E4M3 it costs
|
|
# ~0.6 points of top-1 on its own.
|
|
output.weight rtn codes=i8 group=128 scale=bf16 clamp=sym rule=search
|
|
|
|
# The n-gram table at one fixed E4M3 scale; qwen4exp-w4.recipe says where the value comes from.
|
|
# The value sits on a three-octave plateau (2^-13..2^-10, rel_l2 0.02662) with 1.7x headroom over
|
|
# the base table's absmax; the prep phase verifies Swift's PLE absmax lands inside the plateau
|
|
# before this rule is trusted.
|
|
blk.*.ple_ngram.weight rtn codes=fp8_e4m3 block=*x* scale=bf16 rule=fixed scale_value=1.99317932128906e-4
|
|
|
|
# The MTP head's 2-bit draft head. INERT for this checkpoint: the HF weight map has no
|
|
# mtp.draft_head.weight (same as the base), kept for shape parity with the family.
|
|
mtp.draft_head.weight rtn codes=u2 zero=u8 group=128 scale=f16
|