Deployed shape verified live: KV 16,384.00 MiB (ledger-exact 32,768 B/token x
524,288), RS 100.50 MiB (2 cells), settled nvidia-smi 21,176 used / 2,949 free,
n_slots 2, n_ctx_slot 262144, DropInPaths empty.
Rates recorded as FLOORS: another consumer drove the endpoint through the whole
window (56 prompts on the other slot in three minutes, 96% GPU-util). Single
stream 134.1-141.5 (145.8/163.2 in a lighter minute) vs the 168.7 q8_0 record;
two-stream aggregate 256.6-265.1 vs ~300 across three; prefill 3,159.5 at
106,282 beats the 2,877 record while contended; decode at 106k depth 71.5 (new
baseline, no prior). Per-stream concurrency improved: ~130 each vs 102.8.
Instrument correction: the PrismML fork build 10777 answers 500 'The model
produced output that does not match the expected Content-only format' for a
greedy ignore_eos run on the raw /completion path, so the per-request timings
must be read off /v1/chat/completions. AGENTS.md Benchmarking Policy corrected.
Frank
d457985b8d