Upstream ships dflash depth 7 as a content-dependent default measured for TP=2
on their corpus. Re-measured on one card here, two runs per setting, 16
repo-derived prompts at 512 output tokens:
SPEC=7 conc1 46.2 tok/s, TPOT 20.8 ms, 3.86 tok/update
conc4 118.7 / 127.0 tok/s
SPEC=5 conc1 41.5 tok/s, TPOT 23.1 ms, 3.41 tok/update
conc4 136.1 / 129.4 tok/s
Step time is nearly identical at both depths (~79 ms vs ~80 ms single stream):
7 wins by landing more tokens per step, 5 wins at conc 4 by ~8% because more
narrow verify steps pack better. Depth 7 stays — this fleet is interactive
agent loops where per-turn latency is what is felt — and the runbook now
records the numbers and the one-line switch for a batch-first workload.
Frank
4d37f8ca65