Frank
7daa886dd6
Probe on the live service: four concurrent 196,960-token chats, 4/4 exact marker retrieval, 787,842 tokens in 238.4 s (3,305 t/s aggregate prefill = the single-stream ceiling, so the four streams shared one pipeline and the KV pool never ran short); prefix-cache repeat answered in 1.0 s with cache_n=43008. Recorded with the caveat that 8 generated tokens do not make a decode rate.