docs: frank-large FA_TQ A/B rejected at 80k; deep roles rerouted to frank-primary

Stop-window A/B, same 80k filler prompt, 128-token essays, runtime
timings: HALOGEN_FA_TQ=256 (upstream issue #12's carve workaround) moves
decode ±0.3% (noise, draft acceptance identical) and costs 9.5% prefill
on this topology. The depth collapse sits deeper than the FA tile knob;
the default stands. The harness's slow/advisor roles moved to
frank-primary (fp8 KV 7.32 KiB/token = 8.7x less KV read at depth).
This commit is contained in:
Frank committed 2026-10-06 00:59:14 -07:00
1 parent 306ff32427
commit fadad18da1
2 files changed
+23

No files matched your search

@@ -159,6 +159,24 @@ Measured on real agent traffic (serve_api lines) the day after cutover:
measured depth — its tg rows run over the ten SHORT eval prompt
shapes — so the README's "decode barely degrades at depth" claim was
never exercised at 46-80k.
- **A/B'd the one FA lever with public precedent (2026-10-06 stop-window
probe): rejected.** Issue #12's `HALOGEN_FA_TQ=256` (the 64 GB-carve
crash workaround) vs the default tile — same 80k filler prompt,
128-token essays, the runtime's own `timings`: decode 6.35 vs 6.34 t/s
cold and 5.74 vs 5.76 warm (±0.3 % = noise; draft acceptance identical,
53/74 and 58/69), while prefill LOST 9.5 % on this topology (447.5 ->
404.8 t/s). The depth collapse is not an FA tile artifact — it sits
deeper than the knob. The engine binary advertises 40+ undocumented
`HALOGEN_*` kernel levers (FA/DN/MATMUL families, readable via
`grep -aoE "HALOGEN_[A-Z0-9_]+"` on the binary); `FA_TQ` is the only
one with public evidence, and it moved nothing that matters here.
- **Routing is the applied mitigation (2026-10-06):** the harness's
deep-conversation roles (`slow`, `advisor` in `~/.omp/agent/config.yml`)
moved to frank-primary (radiance on Dankserver: fp8 KV at
7.32 KiB/token = 8.7x less KV to read per round at depth, MTP depth 3,
200k window, prefix-cache host+disk tiers). frank-large keeps OpenWebUI
and shallow bursts, where it runs 30-50 t/s. frank-primary's
depth-decode rate is designed-for but not yet measured on record.
- **The lever that existed was the prompt cache, and it was OFF** (see
the shape bullet). The pin turns the 85-180 s re-prefill per deep
turn into a resume; decode itself is expected UNCHANGED by the pin —