Make FASTGREEDY=1 compatible with non-greedy requests (argmax fastpath always off)

LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=0 unconditionally. FASTGREEDY=1 now
means: reasoning off, deterministic sampler defaults, compatible with
all non-greedy requests (temp != 0, penalties, reasoning budget). This
is the card's 'compatible mode' profile.
This commit is contained in:
Frank committed 2026-08-22 13:45:57 -07:00
1 parent fe71f7516d
commit ddcd54e13f
2 files changed
+13 -16

No files matched your search

+4 -5
View File
@@ -84,11 +84,10 @@ SPEC_DRAFT_P_SPLIT=0.10
# #
# FASTGREEDY mode: set to 1 for fast-greedy benchmark (card's default profile) # FASTGREEDY mode: set to 1 for fast-greedy benchmark (card's default profile)
# FASTGREEDY=0: production chat config (reasoning on, argmax off) # FASTGREEDY=0: production chat config (reasoning on, argmax off)
# FASTGREEDY=1: reasoning off, LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=1, deterministic # FASTGREEDY=1: reasoning off, deterministic sampler, argmax fastpath off
# sampler. Mirrors the card's 'fast greedy (default)' benchmark config. # (compatible mode). Mirrors the card's 'compatible mode' profile — the
# 2026-08-22: flipped to 1 for real (operator decision). NOTE: the fast # same as production minus reasoning. Non-greedy requests work fine.
# greedy path rejects non-greedy requests (temp != 0, top_k > 1, penalties, # 2026-08-22: flipped to 1 for real (operator decision).
# reasoning budgets) with 400. Chat clients must send temperature 0.
FASTGREEDY=1 FASTGREEDY=1
# For fast-greedy bench: set FASTGREEDY=1 above. This flips to: reasoning off, # For fast-greedy bench: set FASTGREEDY=1 above. This flips to: reasoning off,
+9 -11
View File
@@ -101,16 +101,13 @@ export GGML_CUDA_GRAPH_OPT=0
# affect the Kairic Edge IU4 lane (prompt/verification route via # affect the Kairic Edge IU4 lane (prompt/verification route via
# --kairic-edge) nor the MTP draft fast path below. # --kairic-edge) nor the MTP draft fast path below.
# FASTGREEDY env var (0 = production chat, 1 = fast-greedy bench from # FASTGREEDY env var (0 = production chat, 1 = fast-greedy bench from
# card's default profile). When FASTGREEDY=1: reasoning off, argmax # card's default profile). When FASTGREEDY=1: reasoning off, deterministic
# fastpath active, deterministic sampler. Compatible mode (KAIRIC_EDGE # sampler. The argmax fastpath is always off (compatible mode) so non-greedy
# compatibility mode) only disables the target greedy argmax fast path; # requests (temp != 0, penalties, reasoning budget) work correctly.
# reasoning is disabled separately. # The card calls this "compatible mode" — it disables only the target greedy
# argmax fast path and keeps everything else (reasoning, MTP, caching).
FASTGREEDY="${FASTGREEDY:-0}" FASTGREEDY="${FASTGREEDY:-0}"
if [ "$FASTGREEDY" = "1" ]; then
export LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=1
else
export LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=0 export LLAMA_TARGET_GREEDY_ARGMAX_FASTPATH=0
fi
export LLAMA_MTP_CPU_ARGMAX_FASTPATH=1 export LLAMA_MTP_CPU_ARGMAX_FASTPATH=1
export PROMPTFORGE_TARGET_ONLY=0 export PROMPTFORGE_TARGET_ONLY=0
export PROMPTFORGE_MODE=iu4_ffn export PROMPTFORGE_MODE=iu4_ffn
@@ -136,9 +133,10 @@ export PROMPTFORGE_ENABLE_IU4_DECODE_M2_M5=0
export PROMPTFORGE_OUTPUT_K8_STRICT_GREEDY=0 export PROMPTFORGE_OUTPUT_K8_STRICT_GREEDY=0
# Sampling & reasoning defaults — production (FASTGREEDY=0) vs fast-greedy # Sampling & reasoning defaults — production (FASTGREEDY=0) vs fast-greedy
# (FASTGREEDY=1). Fast greedy uses the card's default deterministic sampler # (FASTGREEDY=1). Both use argmax fastpath off (compatible mode); the only
# with argmax fastpath active; production uses Qwen live values with # difference is reasoning: off (FASTGREEDY=1) vs on/deepseek/budget-unlimited
# reasoning budget and argmax fastpath off. # (FASTGREEDY=0). Sampling defaults are identical (temp 0, top_p 1, top_k 0,
# min_p 0) and clients override per request as the card recommends.
if [ "$FASTGREEDY" = "1" ]; then if [ "$FASTGREEDY" = "1" ]; then
TEMP=0 TEMP=0
TOP_P=1.0 TOP_P=1.0