frank-medium: swap the drafter to magnitudedev/Qwen3.8-27B-DSpark
DSpark (Q8_0, 1.27 GiB) replaces z-lab DFlash2 (1.92 GiB): same 5-layer block-diffusion shape, Markov head instead of the candidate selector, and block_size 7, so the draft depth drops 7 -> 6 (llama.cpp asserts n_max <= block_size - 1). The serve script's dspark branch never passed --spec-draft-model (only the dflash branch did), so dspark was not deployable as written; the model file gate now covers both types. Download script, env and the unit seam track the new file and its exact byte count.
This commit is contained in:
1 parent
96062b627e
commit
7e11d247a8
4 files changed
+31
-33
No files matched your search
@@ -1,6 +1,6 @@
|
||||
#!/usr/bin/env bash
|
||||
# Download the frank-medium payload for Dankserver (2x Radeon AI PRO R9700):
|
||||
# the target GGUF and the DFlash2 draft model, into /home/dank/models.
|
||||
# the target GGUF and the DSpark draft model, into /home/dank/models.
|
||||
#
|
||||
# bash scripts/download-frank-medium-gguf.sh
|
||||
# bash scripts/download-frank-medium-gguf.sh --verify-only
|
||||
@@ -17,10 +17,10 @@
|
||||
# own quant of the same size, on Unsloth's importance matrix). The file
|
||||
# carries the model's MTP layer (`blk.64.nextn.*`), so llama.cpp can also
|
||||
# draft from the model itself with no second copy in VRAM.
|
||||
# * Draft: z-lab/Qwen3.8-27B-DFlash2-GGUF at Q8_0 -- the block-diffusion
|
||||
# drafter with the candidate selector (5 layers, hidden 5120). Q8_0 is
|
||||
# 2.0 GiB against BF16's 3.9 GiB and the drafter's job is throughput, not
|
||||
# output quality (verification is lossless either way).
|
||||
# * Draft: magnitudedev/Qwen3.8-27B-DSpark-GGUF at Q8_0 -- the
|
||||
# block-diffusion drafter with the Markov head (5 layers, hidden 5120,
|
||||
# block_size 7). Q8_0 is 1.27 GiB against BF16's ~2.5 GiB and the
|
||||
# drafter's job is throughput, not output quality (lossless verification).
|
||||
set -euo pipefail
|
||||
|
||||
MODEL_REPO="ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF"
|
||||
@@ -28,10 +28,10 @@ MODEL_DIR="/home/dank/models/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF"
|
||||
MODEL_FILE="Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf"
|
||||
MODEL_SIZE=31457990112
|
||||
|
||||
DRAFT_REPO="z-lab/Qwen3.8-27B-DFlash2-GGUF"
|
||||
DRAFT_DIR="/home/dank/models/Qwen3.8-27B-DFlash2-GGUF"
|
||||
DRAFT_FILE="Qwen3.8-27B-DFlash2-Q8_0.gguf"
|
||||
DRAFT_SIZE=2056414816
|
||||
DRAFT_REPO="magnitudedev/Qwen3.8-27B-DSpark-GGUF"
|
||||
DRAFT_DIR="/home/dank/models/Qwen3.8-27B-DSpark-GGUF"
|
||||
DRAFT_FILE="Qwen3.8-27B-DSpark-Q8_0.gguf"
|
||||
DRAFT_SIZE=1359284737
|
||||
|
||||
mode="download"
|
||||
[[ "${1:-}" == "--verify-only" ]] && mode="verify"
|
||||
|
||||
Reference in new issue
Block a user