docs: frank-large FA_TQ A/B rejected at 80k; deep roles rerouted to frank-primary
Stop-window A/B, same 80k filler prompt, 128-token essays, runtime timings: HALOGEN_FA_TQ=256 (upstream issue #12's carve workaround) moves decode ±0.3% (noise, draft acceptance identical) and costs 9.5% prefill on this topology. The depth collapse sits deeper than the FA tile knob; the default stands. The harness's slow/advisor roles moved to frank-primary (fp8 KV 7.32 KiB/token = 8.7x less KV read at depth).
This commit is contained in:
@@ -89,6 +89,11 @@ frank-small, port 7771]
|
||||
run`; the `-d` era wrote only the container ID). Decode-at-depth is
|
||||
structural: 3.3-3.8 t/s at 80k vs 30-50 shallow; no supported flag
|
||||
touches it.
|
||||
The FA tile lever from upstream issue #12 (`HALOGEN_FA_TQ=256`) was
|
||||
A/B'd at 80k on 2026-10-06: decode ±0.3 % (rejected), prefill −9.5 %
|
||||
on this topology — the default tile stands. The deep harness roles
|
||||
(`slow`, `advisor`) rerouted to frank-primary; frank-large keeps
|
||||
OpenWebUI and shallow callers.
|
||||
- [[frank-small-llama-swift15-iq3s-mtp-131072]] — **ACTIVE since 2026-10-05
|
||||
(evening); 2 × 131,072 no-spec since 2026-10-06** (root systemd unit,
|
||||
port 7771): llama.cpp master (CUDA sm_86) serving ukisai/Swift-1.5-
|
||||
|
||||
Reference in New Issue
Block a user