frank-medium: swap to ParoQuant MXFP6 (Swift Qwen3.8-27B) + Qwen-Sharp v22.5.0

- Model: hugypufy/Swift-Qwen3.8-27B-PARO-MXFP6 (25.16 GiB, 18 shards,
  quant_method=paroquant_mxfp6, bits=6, group_size=128, krot=8, rotation on)
- Image: ggz14/vllm-radiance-mxfp4:0.13.0-e060895 (PR #51 launch6_p kernel,
  built on the published 0.9.3 base; docker->podman via save/load)
- GPU_UTIL 0.98 -> 0.99 (user request); KV_DTYPE=auto (unquantized bf16)
- Entrypoint: mxfp6 branch (17 patches + paroquant kernel compile +
  sitecustomize registration); GPU_UTIL default for set -u robustness
- Quadlet: SNAP/GPU_UTIL/Image/cache updated; new mxfp6 cache dir
- deploy-remote.sh: R4D gate + paroquant_mxfp6; new image/SNAP/cache defaults
- Chat template: Qwen-Sharp v22.5.0 (peculiar-ragdoll); v22.4.0 kept as rollback
- Test 168: dflash branch comment quant-aware (mxfp4 + paroquant_mxfp6)
- Runbook: MXFP6 active shape, MXFP4 rollback

MXFP4 checkpoint remains on disk as the rollback target.
This commit is contained in:
Frank committed 2026-09-21 22:46:44 -07:00
1 parent 600812d459
commit e9a01cd013
6 files changed
+709 -111

No files matched your search

+5 -4
View File
@@ -95,10 +95,11 @@ env_str() {
# DFlash2 drafts at most 7 tokens per step; deeper is a different
# drafter's contract and costs acceptance, not just VRAM.
[ "$spec" -eq 7 ]
# The MXFP4 decode kernel band covers MAXSEQS x (SPEC+1) rows, and the
# shipped default covers 64. Exceeding it without the entrypoint's >8
# bump is a launch failure at the worst moment (first concurrent
# request).
# The MXFP4 / ParoQuant-MXFP6 decode kernel band covers MAXSEQS x
# (SPEC+1) rows, and the shipped default covers 64 (the mxfp6 profile
# block exports the same RADIANCE_MXFP4_DECODE_MAX_M=64). Exceeding it
# without the entrypoint's >8 bump is a launch failure at the worst
# moment (first concurrent request).
[ $(( maxseqs * (spec + 1) )) -le 64 ]
grep -q 'RADIANCE_MXFP4_DECODE_MAX_M' "$ENTRY"
;;