Files
Frank 1c56dea67a fix(qwen38): record at MNBT 1,024 — the step size the cache can actually pay for
The rerun after 96c5a06 refused every corpus document again ("document 196 is
8507 tokens and the KV pool holds 7436"), and the boot table shows why the
host-pool change could not have helped: vram_budget.cpp put the container in its
`no lender` branch — elastic 25.56 GiB, experts 25.29 GiB (their plan-fixed
share), kv cache 271.25 MiB — identical at 12,288 and at 49,152 MiB of host
pool. The only supplier left there is the activation arena, and the engine says
so at vram_budget.cpp:117: "lower --max-num-batched-tokens (the arena scales
with it)". The log prices both rungs directly (a 2,048-token step costs 1.08
GiB, a 1,024-token step 504.39 MiB), so the default moves to 1,024: the cache
grows to 866 MiB / 24,300 tokens and scores every one of the 197 documents
(longest 11,970). Calibration keeps its wide 65,536; it was never tied to MNBT.

This also retracts the mechanism 96c5a06 recorded in the header ("the pool buys
KV as well as residency"). Inheriting the env's 49,152 / 96 stays right as
single-source-of-truth, but on this payload the pool is not the cache's supplier.
2026-10-27 22:19:51 -07:00
..