dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: passwordless restart helpers, parallel-streams incident, MTP drafter findings

Browse Source 
- deployment.md: document the sudoers NOPASSWD restart helpers on both
  targets, the exact-argument matching quirk, and the 2026-08-13
  'premature job release' investigation (server never preempts; aborts
  were client-side; Dankserver crashed twice under researcher load).
- DFlash/DSpark doc: MTP requires MTP tensors in the target GGUF; the
  HauhauCS Qwen3.6-27B GGUF has none (verified tensor listing); MTP-only
  draft GGUFs (a4lg/lym00) pair via --spec-draft-model; flag rename
  --spec-type mtp -> draft-mtp in b9180.
- AGENTS.md: operational quirks for the helpers, journal abort-vs-release
  signature, and the Dankserver cublasSgemm crash signature.
This commit is contained in:
Frank committed 2026-08-13 20:42:57 -07:00
1 parent 6fae3a9f1a
commit e29071caa1
3 files changed
+85 -5
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/DFlash DSpark Speculative Decoding.md
+35
View File
@@ -110,6 +110,41 @@ llama-server --hf-repo <target-with-MTP> \
--spec-draft-n-max 2
```
Note: the flag was renamed from `--spec-type mtp` to `--spec-type draft-mtp` in May 2026 (PR #22673, build b9180+). An old flag silently disables speculative decoding.
### MTP drafter availability for Qwen3.6-27B
MTP is not a separate drafter model. It is a "nextn" head that must live in the target GGUF (or be loaded as a draft alongside it). llama.cpp's `draft-mtp` implementation is fully wired for the `qwen35` and `qwen35moe` architectures (PR #22673).
Local check on Dankclaw (2026-08-13), tensor listing via GGUF header parse:
| File | Architecture | Tensors | MTP tensors |
|------|--------------|---------|-------------|
| `Qwen3.6-27B-Uncensored-HauhauCS-Aggressive-Q8_K_P.gguf` | `qwen35` | 851 | 0 |
| `Qwen3.6-27B-DFlash-Q8_0.gguf` (draft) | `dflash` | 58 | 0 |
The HauhauCS uncensored GGUFs ship without the MTP head. To use MTP with them, load an MTP-only draft GGUF from the base Qwen3.6-27B model as a sidecar:
```
llama-server --model <HauhauCS-Q8_K_P.gguf> \
--spec-draft-model <MTP-only-draft.gguf> \
--spec-type draft-mtp \
--spec-draft-n-max 2
```
Compatible MTP-only draft GGUFs on Hugging Face:
| Repo | Notes |
|------|-------|
| `a4lg/Qwen3.6-27B-MTP-ONLY-GGUF` | MTP-only subset of the base model; pairs with derivative/abliterated targets via `--spec-draft-model` |
| `lym00/Qwen3.6-27B-MTP-ONLY-GGUF` | Same purpose; built from `convert_hf_to_gguf.py --mtp` |
| `ggml-org/Qwen3.6-27B-MTP-GGUF` | Full model with MTP head (replaces the target, not a sidecar) |
| `hyperhuzaifa/Qwen3.6-27B-Uncensored-MTP-GGUF` | HauhauCS Q4_K_P with the MTP head grafted in (replaces the target) |
Speculative decoding is lossless under rejection sampling: the target verifies every draft token. A draft head from the base model works with a derivative target; wrong proposals just lower the acceptance rate. The accepted MTP draft lengths are 2-3 tokens; the "MTP Defaults Are a Trap" writeup (thefrontierlab.ai) documents the tuning pitfalls.
On Dankclaw, `draft-dflash` remains the benchmarked winner (20.2 t/s at 4k vs 16.9 for DFlash+ngram and 8.7 for ngram-only). MTP is a viable alternative to test, not a bug fix: the parallel-stream releases investigated on 2026-08-13 were client-side aborts and a Dankserver crash, not speculative decoding behavior (see [[deployment]]).
## Draft Models
DFlash draft model sizes for the models on these servers: