# DFlash and DSpark Speculative Decoding

## Overview

DFlash and DSpark are the two newest speculative decoding methods. They draft many tokens in one parallel pass instead of one token at a time. This reduces decode latency on memory-bound workloads.

DFlash comes from Z Lab at UC Berkeley. DSpark comes from DeepSeek. Both work with vLLM and llama.cpp. Both have draft models for the Qwen3.6 models that Dankserver and Dankclaw run.

## How DFlash Works

DFlash is a lightweight block diffusion model. It predicts a block of tokens in a single forward pass. Key design points:

- The draft model reuses the embedding and LM head from the target model.
- Only the intermediate layers are trained.
- It fuses target-model hidden features into the KV projections of every draft layer.
- A 16-token DFlash drafter has lower latency than a 1-layer EAGLE-3 drafter producing 8 tokens.

Results from the DFlash paper (arXiv:2602.06036):

- Over 6x lossless speedup on Qwen3-8B.
- Up to 2.5x higher speedup than EAGLE-3.
- Diffusion drafting cost is flat, so block size does not increase cost.

## How DSpark Works

DSpark is a semi-autoregressive extension of DFlash from DeepSeek (arXiv:2607.05147). It keeps the DFlash parallel diffusion backbone and adds a lightweight sequential Markov head. This models inter-token dependencies inside each block. It also uses confidence-scheduled verification, which stops verifying tokens that have a high rejection risk.

Results from the DSpark paper:

- 60-85% faster per-user generation than MTP-1 on DeepSeek-V4 production traffic.
- Useful when parallel drafts decay in acceptance rate.

Both DFlash and DSpark are lossless under rejection sampling. Use the strict rejection sampler.

## Runtime Support

| Runtime | DFlash | DSpark | MTP |
|---------|--------|--------|-----|
| vLLM | yes (v0.20.1+) | yes | yes |
| llama.cpp | yes | yes | yes |
| SGLang | yes | no | yes |

### vLLM

vLLM v0.20.1+ has core DFlash support. Use the `--speculative-config` JSON object.

DFlash on vLLM:

```
vllm serve Qwen/Qwen3.6-35B-A3B \
    --speculative-config '{"method": "dflash", "model": "z-lab/Qwen3.6-35B-A3B-DFlash", "num_speculative_tokens": 15}' \
    --attention-backend flash_attn \
    --max-num-batched-tokens 32768
```

Legacy flags also work:

```
vllm serve Qwen/Qwen3.6-35B-A3B \
    --spec-method dflash \
    --spec-model z-lab/Qwen3.6-35B-A3B-DFlash \
    --spec-tokens 8
```

DFlash requires a non-causal attention backend. Set `"attention_backend": "flash_attn"` in the speculative config if needed. On ROCm, check that the flash_attn backend is available before using DFlash.

DSpark on vLLM:

```
vllm serve <target-model> \
    --speculative-config '{"method": "dspark", "model": "<dspark-draft>", "num_speculative_tokens": 15}'
```

### llama.cpp

llama.cpp supports DFlash, DSpark, and MTP through `--spec-type`. When you pass `--spec-draft-hf`, llama.cpp auto-discovers the sidecar files and infers the type. The priority order is DSpark, DFlash, EAGLE-3, MTP.

DFlash on llama.cpp:

```
llama-server --hf-repo <target-GGUF> \
    --spec-draft-hf <draft-GGUF> \
    --spec-type draft-dflash \
    --spec-draft-n-max 16
```

DSpark on llama.cpp:

```
llama-server --hf-repo <target-GGUF> \
    --spec-draft-hf <draft-GGUF> \
    --spec-type draft-dspark
```

MTP on llama.cpp:

```
llama-server --hf-repo <target-with-MTP> \
    --spec-type draft-mtp \
    --spec-draft-n-max 2
```

## Draft Models

DFlash draft model sizes for the models on these servers:

| Target model | DFlash draft | Draft size |
|--------------|--------------|------------|
| Qwen3.6-35B-A3B | z-lab/Qwen3.6-35B-A3B-DFlash | 0.4B |
| Qwen3.6-27B | z-lab/Qwen3.6-27B-DFlash | 2B |
| Qwen3.5-35B-A3B | z-lab/Qwen3.5-35B-A3B-DFlash | 0.4B |
| Qwen3.5-27B | z-lab/Qwen3.5-27B-DFlash | 2B |

The DFlash draft is not a standalone model. It must pair with its target model. The model cards recommend block size 8 for higher concurrency and 16 for single-stream throughput.

## Deployment on Dankserver (NVIDIA RTX 3090)

Dankserver runs vLLM in a Podman container with the RTX 3090. The RTX 3090 is Ampere (compute capability 8.6). Use these conditions:

- FP8 KV cache is valid on CUDA 12.
- FP8 weights compute through a fallback path; no native FP8 tensor cores.
- The DFlash draft model (0.4B) fits easily in 24 GB VRAM alongside the target.

Working DFlash flags for this server:

```
--spec-method dflash
--spec-model z-lab/Qwen3.6-35B-A3B-DFlash
--spec-tokens 8
--kv-cache-dtype fp8
```

Example DFlash + AWQ on the RTX 3090:

```
docker run --rm --name vllm-qwen36-dflash \
    --gpus all --ipc=host \
    -p 8000:8000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    docker.io/vllm/vllm-openai:latest \
    cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit \
    --speculative-config '{"method": "dflash", "model": "z-lab/Qwen3.6-35B-A3B-DFlash", "num_speculative_tokens": 8}' \
    --attention-backend flash_attn \
    --gpu-memory-utilization 0.95
```

## Deployment on Dankclaw (AMD Strix Halo gfx1151)

Dankclaw runs the ROCmFPX llama.cpp runtime as its primary service. The active service uses the Ling-3.0-flash ROCmFP4 MTP model with `--spec-type draft-mtp`. This gives measured results:

- 45-50 t/s decode with MTP speculation.
- ~37 t/s plain decode.
- ~95% draft acceptance with `--spec-draft-n-max 2`.

ROCmFPX is a llama.cpp fork by charlie12345. It adds ROCmFP4 and ROCmFP2 GGUF weight formats with HIP/ROCm and Vulkan kernels. It is not merged into upstream llama.cpp.

Ling-3.0-flash-related facts:

- Base model: inclusionAI/Ling-3.0-flash, 124B total / 5.1B active.
- The MTP (NextN) head is preserved in the GGUF conversion.
- The raulvidis GGUF repo targets Strix Halo (gfx1151).

If you want DFlash on Dankclaw, use a DFlash GGUF draft. Community conversions exist, for example Alittlehammmer/Qwen3.6-27B-DFlash-GGUF-llama.cpp. The ROCm path for DFlash is newer than the MTP path and less proven on gfx1151.

## Notes

- Speculative decoding helps low-QPS, memory-bound, latency-focused workloads. It helps less at high QPS.
- DFlash needs a non-causal attention backend; the ROCm flash_attn backend must be validated before production use.
- The old `--swap-space` flag no longer exists in current vLLM. Use `--kv-offloading-size`.
- `--speculative-config` is the preferred CLI. Legacy `--spec-method`, `--spec-model`, and `--spec-tokens` flags still exist.
- Use `rejection_sample_method` default `strict` for lossless output.

## See Also

- [[Quantization by Hardware Architecture]]
- [[Running Qwen3.6 35B A3B FP8 on Dankserver]]
- [[Running Qwen3.6 27B FP8 on Dankclaw]]
- [[Dankclaw System Specs]]
- [[Dankserver System Specs]]
- [[VLLM Overview]]
- [[deployment]]