serverAppConfigs/docs/DFlash DSpark Speculative Decoding.md

DFlash and DSpark Speculative Decoding

Overview

DFlash and DSpark are the two newest speculative decoding methods. They draft many tokens in one parallel pass instead of one token at a time. This reduces decode latency on memory-bound workloads.

DFlash comes from Z Lab at UC Berkeley. DSpark comes from DeepSeek. Both work with vLLM and llama.cpp. Both have draft models for the Qwen3.6 models that Dankserver and Dankclaw run.

How DFlash Works

DFlash is a lightweight block diffusion model. It predicts a block of tokens in a single forward pass. Key design points:

• The draft model reuses the embedding and LM head from the target model.
• Only the intermediate layers are trained.
• It fuses target-model hidden features into the KV projections of every draft layer.
• A 16-token DFlash drafter has lower latency than a 1-layer EAGLE-3 drafter producing 8 tokens.

Results from the DFlash paper (arXiv:2602.06036):

• Over 6x lossless speedup on Qwen3-8B.
• Up to 2.5x higher speedup than EAGLE-3.
• Diffusion drafting cost is flat, so block size does not increase cost.

How DSpark Works

DSpark is a semi-autoregressive extension of DFlash from DeepSeek (arXiv:2607.05147). It keeps the DFlash parallel diffusion backbone and adds a lightweight sequential Markov head. This models inter-token dependencies inside each block. It also uses confidence-scheduled verification, which stops verifying tokens that have a high rejection risk.

Results from the DSpark paper:

• 60-85% faster per-user generation than MTP-1 on DeepSeek-V4 production traffic.
• Useful when parallel drafts decay in acceptance rate.

Both DFlash and DSpark are lossless under rejection sampling. Use the strict rejection sampler.

Runtime Support

Runtime	DFlash	DSpark	MTP
vLLM	yes (v0.20.1+)	yes	yes
llama.cpp	yes	yes	yes
SGLang	yes	no	yes

vLLM

vLLM v0.20.1+ has core DFlash support. Use the --speculative-config JSON object.

DFlash on vLLM:

vllm serve Qwen/Qwen3.6-35B-A3B \
    --speculative-config '{"method": "dflash", "model": "z-lab/Qwen3.6-35B-A3B-DFlash", "num_speculative_tokens": 15}' \
    --attention-backend flash_attn \
    --max-num-batched-tokens 32768

Legacy flags also work:

vllm serve Qwen/Qwen3.6-35B-A3B \
    --spec-method dflash \
    --spec-model z-lab/Qwen3.6-35B-A3B-DFlash \
    --spec-tokens 8

DFlash requires a non-causal attention backend. Set "attention_backend": "flash_attn" in the speculative config if needed. On ROCm, check that the flash_attn backend is available before using DFlash.

DSpark on vLLM:

vllm serve <target-model> \
    --speculative-config '{"method": "dspark", "model": "<dspark-draft>", "num_speculative_tokens": 15}'

llama.cpp

llama.cpp supports DFlash, DSpark, and MTP through --spec-type. When you pass --spec-draft-hf, llama.cpp auto-discovers the sidecar files and infers the type. The priority order is DSpark, DFlash, EAGLE-3, MTP.

DFlash on llama.cpp:

llama-server --hf-repo <target-GGUF> \
    --spec-draft-hf <draft-GGUF> \
    --spec-type draft-dflash \
    --spec-draft-n-max 16

DSpark on llama.cpp:

llama-server --hf-repo <target-GGUF> \
    --spec-draft-hf <draft-GGUF> \
    --spec-type draft-dspark

MTP on llama.cpp:

llama-server --hf-repo <target-with-MTP> \
    --spec-type draft-mtp \
    --spec-draft-n-max 2

Draft Models

DFlash draft model sizes for the models on these servers:

Target model	DFlash draft	Draft size
Qwen3.6-35B-A3B	z-lab/Qwen3.6-35B-A3B-DFlash	0.4B
Qwen3.6-27B	z-lab/Qwen3.6-27B-DFlash	2B
Qwen3.5-35B-A3B	z-lab/Qwen3.5-35B-A3B-DFlash	0.4B
Qwen3.5-27B	z-lab/Qwen3.5-27B-DFlash	2B

The DFlash draft is not a standalone model. It must pair with its target model. The model cards recommend block size 8 for higher concurrency and 16 for single-stream throughput.

Deployment on Dankserver (NVIDIA RTX 3090)

Dankserver runs vLLM in a Podman container with the RTX 3090. The RTX 3090 is Ampere (compute capability 8.6). Use these conditions:

• FP8 KV cache is valid on CUDA 12.
• FP8 weights compute through a fallback path; no native FP8 tensor cores.
• The DFlash draft model (0.4B) fits easily in 24 GB VRAM alongside the target.

Working DFlash flags for this server:

--spec-method dflash
--spec-model z-lab/Qwen3.6-35B-A3B-DFlash
--spec-tokens 8
--kv-cache-dtype fp8

Example DFlash + AWQ on the RTX 3090:

docker run --rm --name vllm-qwen36-dflash \
    --gpus all --ipc=host \
    -p 8000:8000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    docker.io/vllm/vllm-openai:latest \
    cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit \
    --speculative-config '{"method": "dflash", "model": "z-lab/Qwen3.6-35B-A3B-DFlash", "num_speculative_tokens": 8}' \
    --attention-backend flash_attn \
    --gpu-memory-utilization 0.95

Deployment on Dankclaw (AMD Strix Halo gfx1151)

Dankclaw runs the ROCmFPX llama.cpp runtime as its primary service. The active service uses the Ling-3.0-flash ROCmFP4 MTP model with --spec-type draft-mtp. This gives measured results:

• 45-50 t/s decode with MTP speculation.
• ~37 t/s plain decode.
• ~95% draft acceptance with --spec-draft-n-max 2.

ROCmFPX is a llama.cpp fork by charlie12345. It adds ROCmFP4 and ROCmFP2 GGUF weight formats with HIP/ROCm and Vulkan kernels. It is not merged into upstream llama.cpp.

Ling-3.0-flash-related facts:

• Base model: inclusionAI/Ling-3.0-flash, 124B total / 5.1B active.
• The MTP (NextN) head is preserved in the GGUF conversion.
• The raulvidis GGUF repo targets Strix Halo (gfx1151).

If you want DFlash on Dankclaw, use a DFlash GGUF draft. Community conversions exist, for example Alittlehammmer/Qwen3.6-27B-DFlash-GGUF-llama.cpp. The ROCm path for DFlash is newer than the MTP path and less proven on gfx1151.

Notes

• Speculative decoding helps low-QPS, memory-bound, latency-focused workloads. It helps less at high QPS.
• DFlash needs a non-causal attention backend; the ROCm flash_attn backend must be validated before production use.
• The old --swap-space flag no longer exists in current vLLM. Use --kv-offloading-size.
• --speculative-config is the preferred CLI. Legacy --spec-method, --spec-model, and --spec-tokens flags still exist.
• Use rejection_sample_method default strict for lossless output.

See Also

• Quantization by Hardware Architecture
• Running Qwen3.6 35B A3B FP8 on Dankserver
• Running Qwen3.6 27B FP8 on Dankclaw
• Dankclaw System Specs
• Dankserver System Specs
• VLLM Overview
• deployment