dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

SPEC_TOKENS 3->2: decode 32.40 t/s (+2.1% over 31.72). Acceptance 0.755 (between depth-3 0.831 and depth-4 0.497) — shallower drafts verify cheaper, net positive. Depth 2 is the decode optimum. Prefill/TTFT shifts are cold-run noise (prefill is orthogonal to spec depth).

Browse Source 
Result: {"status":"keep","decode_tps":32.4031,"draft_accept":0.7549,"prefill_tps":372.9299,"ttft_ms":116984.442}
This commit is contained in:
Frank
2026-10-06 04:09:39 -07:00
parent 81e6122891
commit 95923d3a98
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/frank-primary.env
+1 -1
View File
@@ -115,7 +115,7 @@ FRANK_PRIMARY_KV_DTYPE=fp8
# Spec decode: the checkpoint's own MTP head at the publisher's depth-3 # Spec decode: the checkpoint's own MTP head at the publisher's depth-3
# operating point (probe() returns 3 when mtp_num_hidden_layers > 0; the # operating point (probe() returns 3 when mtp_num_hidden_layers > 0; the
# flag states it so the ledger records the intent). # flag states it so the ledger records the intent).
FRANK_PRIMARY_SPEC_TOKENS=3 FRANK_PRIMARY_SPEC_TOKENS=2
FRANK_PRIMARY_MAX_BATCHED_TOKENS=2048 FRANK_PRIMARY_MAX_BATCHED_TOKENS=2048
# Prefix cache tiers (publisher pattern; disk tier sized for 2 slots). # Prefix cache tiers (publisher pattern; disk tier sized for 2 slots).
FRANK_PRIMARY_PREFIX_HOST_MIB=4096 FRANK_PRIMARY_PREFIX_HOST_MIB=4096