Add ngram-mod to spec pipeline: -1.2 tok/s regression

- DFlash+ngram-mod: 17.11 tok/s vs DFlash alone: 18.34 tok/s
- ngram overhead not offset by acceptance rate at current settings
- Keeping in place for now, may tune ngram-mod params later
This commit is contained in:
Frank committed 2026-08-07 16:25:49 -07:00
1 parent 764a65080f
commit 2426ca563a
2 files changed
+2 -2

No files matched your search

+1 -1
View File
@@ -26,7 +26,7 @@ echo ""
exec "${SERVER_BIN}" \
--hf-repo "${TARGET_REPO}" \
--spec-draft-hf "${DRAFT_REPO}" \
--spec-type draft-dflash \
--spec-type draft-dflash,ngram-mod \
--spec-draft-n-max "${DRAFT_N_MAX}" \
--device ROCm0 \
--spec-draft-device ROCm0 \