feat: frank-default-large → llama.cpp Qwen3.6-27B Q8_K_P + DFlash draft + ngram-mod (bench pending)

2026-08-11 13:47:37 -07:00
parent a74c1c5bcb
commit aa2f4bfe39
5 changed files with 268 additions and 4 deletions
@@ -75,6 +75,16 @@ vllm serve <target-model> \

 llama.cpp supports DFlash, DSpark, and MTP through `--spec-type`. When you pass `--spec-draft-hf`, llama.cpp auto-discovers the sidecar files and infers the type. The priority order is DSpark, DFlash, EAGLE-3, MTP.

+`--spec-type` accepts a **comma-separated list** of speculative implementations. `common_speculative_init` (in `common/speculative.cpp`) builds a vector of implementations and chains drafting through all of them (`common_speculative_draft_params.drafting` is "used to chain the drafts through all the available implementations"). This means you can **layer ngram-mod on top of a DFlash draft model** in one server process:
+
+```
+llama-server --hf-repo <target-GGUF> \
+    --spec-draft-hf <dflash-draft-GGUF> \
+    --spec-type draft-dflash,ngram-mod
+```
+
+The DFlash drafter proposes, then ngram-mod runs as an additional implementation in the chain. Verified against upstream `common/speculative.cpp` (b10133+).
+
 DFlash on llama.cpp:

 ```