Record no-spec ubatch benchmark

This commit is contained in:
Frank committed 2026-08-25 01:29:49 -07:00
1 parent f13124f977
commit 37fb420fcd
3 files changed
+40 -4

No files matched your search

+6 -2
View File
@@ -75,8 +75,12 @@ On Dankserver the available buffer types are `CPU` and `CUDA0`.
MoE model with a bundled MTP head -> native `draft-mtp`. The head is
auto-detected from the target (b10121 creates the MTP draft context against
the model file). ngram under MTP hurts on this family; MTP alone is the
proven best. `--spec-draft-n-max 4` is the knee.
the model file). ngram under MTP hurts on this family; MTP alone was
previously selected for the production configuration. A later 8K API
benchmark found that this MTP path produced lower generation throughput than
`--spec-type none` for that workload, so speculative decoding should be
re-evaluated before treating it as optimal. `--spec-draft-n-max 4` remains the
configured value.
```
--spec-type draft-mtp