frank-medium: swap the drafter to magnitudedev/Qwen3.8-27B-DSpark
DSpark (Q8_0, 1.27 GiB) replaces z-lab DFlash2 (1.92 GiB): same 5-layer block-diffusion shape, Markov head instead of the candidate selector, and block_size 7, so the draft depth drops 7 -> 6 (llama.cpp asserts n_max <= block_size - 1). The serve script's dspark branch never passed --spec-draft-model (only the dflash branch did), so dspark was not deployable as written; the model file gate now covers both types. Download script, env and the unit seam track the new file and its exact byte count.
This commit is contained in:
1 parent
96062b627e
commit
7e11d247a8
4 files changed
+31
-33
No files matched your search
+13
-15
@@ -1,5 +1,5 @@
|
||||
# Tunables for frank-medium (ajgazin/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP
|
||||
# UD-Q8_K_XL + z-lab/Qwen3.8-27B-DFlash2-GGUF Q8_0 drafter), mainline llama.cpp
|
||||
# UD-Q8_K_XL + magnitudedev/Qwen3.8-27B-DSpark-GGUF Q8_0 drafter), mainline llama.cpp
|
||||
# on Dankserver, 2x AMD Radeon AI PRO R9700 (gfx1201). Since 2026-09-16 this
|
||||
# is the SOLE LLM runtime on the box: the vllm-radiance replica pair and its
|
||||
# routing proxy are retired.
|
||||
@@ -39,9 +39,9 @@
|
||||
# carries the model's MTP head (blk.64.nextn.*), so a file-level switch to
|
||||
# --spec-type draft-mtp is available without a second copy in VRAM.
|
||||
#
|
||||
# Drafter: z-lab/Qwen3.8-27B-DFlash2-GGUF @ Q8_0, 1.92 GiB: 5 layers, hidden
|
||||
# 5120, block-diffusion proposal with the candidate selector. Decoding stays
|
||||
# lossless -- the target verifies every drafted token.
|
||||
# Drafter: magnitudedev/Qwen3.8-27B-DSpark-GGUF @ Q8_0, 1.27 GiB: 5 layers,
|
||||
# hidden 5120, block-diffusion proposal with the Markov head (dspark).
|
||||
# Decoding stays lossless -- the target verifies every drafted token.
|
||||
#
|
||||
# Architecture (GGUF arch qwen35): 64 layers as 16 x (3 x (Gated DeltaNet ->
|
||||
# FFN) + 1 x (Gated Attention -> FFN)) -- 48 linear-attention layers whose
|
||||
@@ -74,8 +74,8 @@ LLAMA_BIN=/home/dank/GitPackages/llama.cpp-dflash2/build-rdna4-rocm714/bin/llama
|
||||
# Payload, fetched with scripts/download-frank-medium-gguf.sh.
|
||||
MODEL_DIR=/home/dank/models/Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-GGUF
|
||||
MODEL_FILE=Swift-Qwen3.8-27B-Uncensored-Dynamic-MTP-UD-Q8_K_XL.gguf
|
||||
DRAFT_MODEL_DIR=/home/dank/models/Qwen3.8-27B-DFlash2-GGUF
|
||||
DRAFT_MODEL_FILE=Qwen3.8-27B-DFlash2-Q8_0.gguf
|
||||
DRAFT_MODEL_DIR=/home/dank/models/Qwen3.8-27B-DSpark-GGUF
|
||||
DRAFT_MODEL_FILE=Qwen3.8-27B-DSpark-Q8_0.gguf
|
||||
|
||||
# Fleet identity. 7770 is the fleet's dankserver endpoint; the alias is the id
|
||||
# at /v1/models and what every client asks for.
|
||||
@@ -137,15 +137,13 @@ REASONING_BUDGET=-1
|
||||
CHAT_TEMPLATE_FILE=
|
||||
CHAT_TEMPLATE_KWARGS='{"enable_thinking":true,"preserve_thinking":true}'
|
||||
|
||||
# Speculative decoding: the DFlash2 block-diffusion drafter PLUS the draftless
|
||||
# n-gram matcher. Measured against the drafter alone on 2026-09-16 (tables
|
||||
# below): the chain is byte-identical to DFlash2 by itself where n-grams cannot
|
||||
# match, and ~2.6x it where they can. DFlash2's GGUF carries block_size 8, so at
|
||||
# most 7 draft tokens per verification step (llama.cpp asserts
|
||||
# n_max <= block_size - 1); 7 is also what the drafter was published with. The
|
||||
# drafter rides ROCm1, the card that ends up holding the target's last layers.
|
||||
SPEC_TYPE=draft-dflash,ngram-mod
|
||||
SPEC_DRAFT_N_MAX=7
|
||||
# Speculative decoding: the DSpark block-diffusion drafter PLUS the draftless
|
||||
# n-gram matcher. DSpark's GGUF carries block_size 7, so at most 6 draft
|
||||
# tokens per verification step (llama.cpp asserts n_max <= block_size - 1).
|
||||
# The drafter rides ROCm1, the card that ends up holding the target's last
|
||||
# layers.
|
||||
SPEC_TYPE=draft-dspark,ngram-mod
|
||||
SPEC_DRAFT_N_MAX=6
|
||||
SPEC_DRAFT_NGL=999
|
||||
SPEC_DRAFT_DEVICE=ROCm1
|
||||
SPEC_DRAFT_TYPE_K=f16
|
||||
|
||||
Reference in new issue
Block a user