dank/serverAppConfigs
1
0 Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

Make FASTGREEDY=1 compatible with non-greedy requests (argmax fastpath always off)

Browse Source 
LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=0 unconditionally. FASTGREEDY=1 now
means: reasoning off, deterministic sampler defaults, compatible with
all non-greedy requests (temp != 0, penalties, reasoning budget). This
is the card's 'compatible mode' profile.
This commit is contained in:
Frank committed 2026-08-22 13:45:57 -07:00
1 parent fe71f7516d
commit ddcd54e13f
2 files changed
+14 -17
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

scripts/frank-default-large.env
+4 -5
View File
@@ -84,11 +84,10 @@ SPEC_DRAFT_P_SPLIT=0.10
#
# FASTGREEDY mode: set to 1 for fast-greedy benchmark (card's default profile)
# FASTGREEDY=0: production chat config (reasoning on, argmax off)
# FASTGREEDY=1: reasoning off, LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=1, deterministic
# sampler. Mirrors the card's 'fast greedy (default)' benchmark config.
# 2026-08-22: flipped to 1 for real (operator decision). NOTE: the fast
# greedy path rejects non-greedy requests (temp != 0, top_k > 1, penalties,
# reasoning budgets) with 400. Chat clients must send temperature 0.
# FASTGREEDY=1: reasoning off, deterministic sampler, argmax fastpath off
# (compatible mode). Mirrors the card's 'compatible mode' profile — the
# same as production minus reasoning. Non-greedy requests work fine.
# 2026-08-22: flipped to 1 for real (operator decision).
FASTGREEDY=1
# For fast-greedy bench: set FASTGREEDY=1 above. This flips to: reasoning off,