dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

MAX_SEQS 2->3: decode 33.06 t/s (+2.0% over depth-2 32.40; +4.2% over baseline). Bigger batch headroom co-schedules concurrent decode streams (decode is weight-streaming bound, so a fuller batch amortizes the weight read). KV pool 600k tokens (3x200k), above the 400k floor. Draft acceptance unchanged (0.755).

Browse Source 
Result: {"status":"keep","decode_tps":33.0578,"draft_accept":0.7549,"prefill_tps":452.9596,"ttft_ms":96315.4258}
This commit is contained in:
Frank
2026-10-06 04:18:53 -07:00
parent 95923d3a98
commit df6b5bb6e8
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/frank-primary.env
+1 -1
View File
@@ -91,7 +91,7 @@ FRANK_PRIMARY_TP_WIRE=wht6
# start"); raise only after a real-traffic look at slot occupancy and the # start"); raise only after a real-traffic look at slot occupancy and the
# KV pool's headroom line. The recurrent-state cost scales with it (54 # KV pool's headroom line. The recurrent-state cost scales with it (54
# MiB/copy/card, 2 copies with speculation). # MiB/copy/card, 2 copies with speculation).
FRANK_PRIMARY_MAX_SEQS=2 FRANK_PRIMARY_MAX_SEQS=3
# Served window. Native is 262,144; 200,000 is the publisher's production # Served window. Native is 262,144; 200,000 is the publisher's production
# value (compose) and what models.yml's contextWindow must quote — a longer # value (compose) and what models.yml's contextWindow must quote — a longer
# prompt is refused by the engine, never truncated. # prompt is refused by the engine, never truncated.