frank-medium: swap the drafter to magnitudedev/Qwen3.8-27B-DSpark

DSpark (Q8_0, 1.27 GiB) replaces z-lab DFlash2 (1.92 GiB): same 5-layer
block-diffusion shape, Markov head instead of the candidate selector, and
block_size 7, so the draft depth drops 7 -> 6 (llama.cpp asserts
n_max <= block_size - 1).

The serve script's dspark branch never passed --spec-draft-model (only the
dflash branch did), so dspark was not deployable as written; the model file
gate now covers both types. Download script, env and the unit seam track the
new file and its exact byte count.
This commit is contained in:
Frank committed 2026-09-16 19:34:00 -07:00
1 parent 96062b627e
commit 7e11d247a8
4 files changed
+31 -33

No files matched your search

+9 -9
View File
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# Download the frank-medium payload for Dankserver (2x Radeon AI PRO R9700):
# the target GGUF and the DFlash2 draft model, into /home/dank/models.
# the target GGUF and the DSpark draft model, into /home/dank/models.
#
# bash scripts/download-frank-medium-gguf.sh
# bash scripts/download-frank-medium-gguf.sh --verify-only
@@ -17,10 +17,10 @@
# own quant of the same size, on Unsloth's importance matrix). The file
# carries the model's MTP layer (`blk.64.nextn.*`), so llama.cpp can also
# draft from the model itself with no second copy in VRAM.
# * Draft: z-lab/Qwen3.8-27B-DFlash2-GGUF at Q8_0 -- the block-diffusion
# drafter with the candidate selector (5 layers, hidden 5120). Q8_0 is
# 2.0 GiB against BF16's 3.9 GiB and the drafter's job is throughput, not
# output quality (verification is lossless either way).
# * Draft: magnitudedev/Qwen3.8-27B-DSpark-GGUF at Q8_0 -- the
# block-diffusion drafter with the Markov head (5 layers, hidden 5120,
# block_size 7). Q8_0 is 1.27 GiB against BF16's ~2.5 GiB and the
# drafter's job is throughput, not output quality (lossless verification).
set -euo pipefail
MODEL_REPO="ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF"
@@ -28,10 +28,10 @@ MODEL_DIR="/home/dank/models/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF"
MODEL_FILE="Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf"
MODEL_SIZE=31457990112
DRAFT_REPO="z-lab/Qwen3.8-27B-DFlash2-GGUF"
DRAFT_DIR="/home/dank/models/Qwen3.8-27B-DFlash2-GGUF"
DRAFT_FILE="Qwen3.8-27B-DFlash2-Q8_0.gguf"
DRAFT_SIZE=2056414816
DRAFT_REPO="magnitudedev/Qwen3.8-27B-DSpark-GGUF"
DRAFT_DIR="/home/dank/models/Qwen3.8-27B-DSpark-GGUF"
DRAFT_FILE="Qwen3.8-27B-DSpark-Q8_0.gguf"
DRAFT_SIZE=1359284737
mode="download"
[[ "${1:-}" == "--verify-only" ]] && mode="verify"