serverAppConfigs/docs/DFlash DSpark Speculative Decoding.md

DFlash and DSpark Speculative Decoding

Overview

DFlash and DSpark are the two newest speculative decoding methods. They draft many tokens in one parallel pass instead of one token at a time. This reduces decode latency on memory-bound workloads.

DFlash comes from Z Lab at UC Berkeley. DSpark comes from DeepSeek. Both work with vLLM and llama.cpp. Both have draft models for the Qwen3.6 models that Dankserver and Dankclaw run.

How DFlash Works

DFlash is a lightweight block diffusion model. It predicts a block of tokens in a single forward pass. Key design points:

• The draft model reuses the embedding and LM head from the target model.
• Only the intermediate layers are trained.
• It fuses target-model hidden features into the KV projections of every draft layer.
• A 16-token DFlash drafter has lower latency than a 1-layer EAGLE-3 drafter producing 8 tokens.

Results from the DFlash paper (arXiv:2602.06036):

• Over 6x lossless speedup on Qwen3-8B.
• Up to 2.5x higher speedup than EAGLE-3.
• Diffusion drafting cost is flat, so block size does not increase cost.

How DSpark Works

DSpark is a semi-autoregressive extension of DFlash from DeepSeek (arXiv:2607.05147). It keeps the DFlash parallel diffusion backbone and adds a lightweight sequential Markov head. This models inter-token dependencies inside each block. It also uses confidence-scheduled verification, which stops verifying tokens that have a high rejection risk.

Results from the DSpark paper:

• 60-85% faster per-user generation than MTP-1 on DeepSeek-V4 production traffic.
• Useful when parallel drafts decay in acceptance rate.

Both DFlash and DSpark are lossless under rejection sampling. Use the strict rejection sampler.

Runtime Support

Runtime	DFlash	DSpark	MTP
vLLM	yes (v0.20.1+)	yes	yes
llama.cpp	yes	yes	yes
SGLang	yes	no	yes

vLLM

vLLM v0.20.1+ has core DFlash support. Use the --speculative-config JSON object.

DFlash on vLLM:

vllm serve Qwen/Qwen3.6-35B-A3B \
    --speculative-config '{"method": "dflash", "model": "z-lab/Qwen3.6-35B-A3B-DFlash", "num_speculative_tokens": 15}' \
    --attention-backend flash_attn \
    --max-num-batched-tokens 32768

Legacy flags also work:

vllm serve Qwen/Qwen3.6-35B-A3B \
    --spec-method dflash \
    --spec-model z-lab/Qwen3.6-35B-A3B-DFlash \
    --spec-tokens 8

DFlash requires a non-causal attention backend. Set "attention_backend": "flash_attn" in the speculative config if needed. On ROCm, check that the flash_attn backend is available before using DFlash.

DSpark on vLLM:

vllm serve <target-model> \
    --speculative-config '{"method": "dspark", "model": "<dspark-draft>", "num_speculative_tokens": 15}'

llama.cpp

llama.cpp supports DFlash, DSpark, and MTP through --spec-type. When you pass --spec-draft-hf, llama.cpp auto-discovers the sidecar files and infers the type. The priority order is DSpark, DFlash, EAGLE-3, MTP.

DFlash on llama.cpp:

llama-server --hf-repo <target-GGUF> \
    --spec-draft-hf <draft-GGUF> \
    --spec-type draft-dflash \
    --spec-draft-n-max 16

DSpark on llama.cpp:

llama-server --hf-repo <target-GGUF> \
    --spec-draft-hf <draft-GGUF> \
    --spec-type draft-dspark

MTP on llama.cpp:

llama-server --hf-repo <target-with-MTP> \
    --spec-type draft-mtp \
    --spec-draft-n-max 2

Draft Models

DFlash draft model sizes for the models on these servers:

Target model	DFlash draft	Draft size
Qwen3.6-35B-A3B	z-lab/Qwen3.6-35B-A3B-DFlash	0.4B
Qwen3.6-27B	z-lab/Qwen3.6-27B-DFlash	2B
Qwen3.5-35B-A3B	z-lab/Qwen3.5-35B-A3B-DFlash	0.4B
Qwen3.5-27B	z-lab/Qwen3.5-27B-DFlash	2B

The DFlash draft is not a standalone model. It must pair with its target model. The model cards recommend block size 8 for higher concurrency and 16 for single-stream throughput.

Deployment on Dankserver (NVIDIA RTX 3090)

Dankserver runs vLLM in a Podman container with the RTX 3090. The RTX 3090 is Ampere (compute capability 8.6). Use these conditions:

• FP8 KV cache is valid on CUDA 12.
• FP8 weights compute through a fallback path; no native FP8 tensor cores.
• The DFlash draft model (0.4B) fits easily in 24 GB VRAM alongside the target.

Working DFlash flags for this server:

--spec-method dflash
--spec-model z-lab/Qwen3.6-35B-A3B-DFlash
--spec-tokens 8
--kv-cache-dtype fp8

Example DFlash + AWQ on the RTX 3090:

docker run --rm --name vllm-qwen36-dflash \
    --gpus all --ipc=host \
    -p 8000:8000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    docker.io/vllm/vllm-openai:latest \
    cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit \
    --speculative-config '{"method": "dflash", "model": "z-lab/Qwen3.6-35B-A3B-DFlash", "num_speculative_tokens": 8}' \
    --attention-backend flash_attn \
    --gpu-memory-utilization 0.95

Deployment on Dankclaw (AMD Strix Halo gfx1151)

Dankclaw runs vLLM-ROCm as its primary runtime. The active service (frank-default-large, port 7770) serves the Ling-3.0-flash CIRU INT4 (W4A16) model with native MTP speculative decoding (K1, lossless). MTP uses a NextN head preserved in the checkpoint, so no separate draft model is needed.

Ling-3.0-flash facts:

• Base model: inclusionAI/Ling-3.0-flash, 124B total / 5.1B active (hybrid linear attention: KDA + MLA).
• The MTP head is served in-band; vLLM detects it and shares the target embedding and lm_head weights with the draft head.
• The earlier ROCmFPX llama.cpp service (raulvidis ROCmFP4 MTP GGUF) was retired because it was defective. vLLM now handles concurrency and tool calling correctly.

If you want DFlash or DSpark on Dankclaw, use a vLLM speculative-config draft that pairs with the target model. DFlash needs a non-causal attention backend; the ROCm flash_attn backend must be validated before production use. MTP is the proven, lossless path on gfx1151.

Notes

• Speculative decoding helps low-QPS, memory-bound, latency-focused workloads. It helps less at high QPS.
• DFlash needs a non-causal attention backend; the ROCm flash_attn backend must be validated before production use.
• The old --swap-space flag no longer exists in current vLLM. Use --kv-offloading-size.
• --speculative-config is the preferred CLI. Legacy --spec-method, --spec-model, and --spec-tokens flags still exist.
• Use rejection_sample_method default strict for lossless output.

See Also

• Quantization by Hardware Architecture
• Running Qwen3.6 35B A3B FP8 on Dankserver
• Running Qwen3.6 27B FP8 on Dankclaw
• Dankclaw System Specs
• Dankserver System Specs
• VLLM Overview
• deployment