diff --git a/serverAppConfigs-docs/LLM/frank-large-halogen27b-dflash2.md b/serverAppConfigs-docs/LLM/frank-large-halogen27b-dflash2.md index 5d5124d..ae85a9a 100644 --- a/serverAppConfigs-docs/LLM/frank-large-halogen27b-dflash2.md +++ b/serverAppConfigs-docs/LLM/frank-large-halogen27b-dflash2.md @@ -159,6 +159,24 @@ Measured on real agent traffic (serve_api lines) the day after cutover: measured depth — its tg rows run over the ten SHORT eval prompt shapes — so the README's "decode barely degrades at depth" claim was never exercised at 46-80k. +- **A/B'd the one FA lever with public precedent (2026-10-06 stop-window + probe): rejected.** Issue #12's `HALOGEN_FA_TQ=256` (the 64 GB-carve + crash workaround) vs the default tile — same 80k filler prompt, + 128-token essays, the runtime's own `timings`: decode 6.35 vs 6.34 t/s + cold and 5.74 vs 5.76 warm (±0.3 % = noise; draft acceptance identical, + 53/74 and 58/69), while prefill LOST 9.5 % on this topology (447.5 -> + 404.8 t/s). The depth collapse is not an FA tile artifact — it sits + deeper than the knob. The engine binary advertises 40+ undocumented + `HALOGEN_*` kernel levers (FA/DN/MATMUL families, readable via + `grep -aoE "HALOGEN_[A-Z0-9_]+"` on the binary); `FA_TQ` is the only + one with public evidence, and it moved nothing that matters here. +- **Routing is the applied mitigation (2026-10-06):** the harness's + deep-conversation roles (`slow`, `advisor` in `~/.omp/agent/config.yml`) + moved to frank-primary (radiance on Dankserver: fp8 KV at + 7.32 KiB/token = 8.7x less KV to read per round at depth, MTP depth 3, + 200k window, prefix-cache host+disk tiers). frank-large keeps OpenWebUI + and shallow bursts, where it runs 30-50 t/s. frank-primary's + depth-decode rate is designed-for but not yet measured on record. - **The lever that existed was the prompt cache, and it was OFF** (see the shape bullet). The pin turns the 85-180 s re-prefill per deep turn into a resume; decode itself is expected UNCHANGED by the pin — diff --git a/serverAppConfigs-docs/LLM/index.md b/serverAppConfigs-docs/LLM/index.md index 428b0b5..b53b904 100644 --- a/serverAppConfigs-docs/LLM/index.md +++ b/serverAppConfigs-docs/LLM/index.md @@ -89,6 +89,11 @@ frank-small, port 7771] run`; the `-d` era wrote only the container ID). Decode-at-depth is structural: 3.3-3.8 t/s at 80k vs 30-50 shallow; no supported flag touches it. + The FA tile lever from upstream issue #12 (`HALOGEN_FA_TQ=256`) was + A/B'd at 80k on 2026-10-06: decode ±0.3 % (rejected), prefill −9.5 % + on this topology — the default tile stands. The deep harness roles + (`slow`, `advisor`) rerouted to frank-primary; frank-large keeps + OpenWebUI and shallow callers. - [[frank-small-llama-swift15-iq3s-mtp-131072]] — **ACTIVE since 2026-10-05 (evening); 2 × 131,072 no-spec since 2026-10-06** (root systemd unit, port 7771): llama.cpp master (CUDA sm_86) serving ukisai/Swift-1.5-