feat(frank-dragoon): double the KV pool to 2 x ctx for slot parity
frank-dragoon's total in-flight context was HALF frank-zealot's: zealot runs one 401,408-position pool split into two 200,704 conversations, while dragoon ran ONE 200,704-position pool behind its two slots, so a single maximum-length request filled it and two long conversations had to stay under ~100k each. HALOGEN_KV_POOL_POSITIONS 200704 -> 401408 = slots x ctx, the top of the seam's legal band (ctx <= pool <= slots * ctx). The PER-REQUEST ceiling does not move: HALOGEN_CTX stays 200704, because the checkpoint's native window is 262,144 and the engine's own 400 text refuses to exceed it without HALOGEN_ROPE_YARN. What this buys is concurrency, not a longer conversation, which is why the harness contextWindow (155648 = ceiling - 45056) is untouched. Cost: ~10.8 GiB of KV where 200,704 cost 5.5 (262,144 cost 7.2; the 2026-10-06 shape's 524,288 measured 14.6), taken from the page cache the 51.2 GB n-gram table reads through — the warm prompt-cache ratio is the number to re-measure after the boot. Engine-side total goes 74.1 -> ~79.4 GiB of 121.2. Also carries scripts/download-frank-zealot-bonsai2-ptq1.sh: the PTQ1_0 candidate payload fetcher (5,946,648,928 B, LFS sha256-verified, mmproj hardlinked), with the publisher's own throughput table in its header — the packing is smaller but the card measures it SLOWER on Ampere, so the swap is decided on our card, not on the card's A100 row.
This commit is contained in:
@@ -19,7 +19,7 @@ and Dankclaw runs two endpoints — **halogen-flash-server 0.16.2
|
||||
digest serving the NATIVE v2 conversion of the abliterated
|
||||
Qwen3.8-Flash-Next, a ~62 GiB v2 .hgn that loads in no other engine,
|
||||
MTP spec decode always-on (HALOGEN_SPEC_ADAPT=0), TWO slots over ONE
|
||||
200,704-position KV pool at ctx 200704, vision ON)** and
|
||||
401,408-position KV pool at ctx 200704, vision ON)** and
|
||||
**llama.cpp on the PrismML fork (frank-zealot, port 7771: prism-ml
|
||||
Ternary-Bonsai-2-27B at **PQ2_0** — 27B qwen35 hybrid (16 full-attention + 48
|
||||
Gated DeltaNet layers), TWO 200,704-token conversations, MTP disabled,
|
||||
@@ -54,7 +54,7 @@ gfx1151 iGPU 114 GiB GTT]
|
||||
E[flash-server container gfx1151, digest-pinned image
|
||||
frank-dragoon, port 7770]
|
||||
F[flash-server v2 native abliterated .hgn + ngram table + vision tower
|
||||
ONE 200704-position KV pool at ctx 200704, 2 slots, MTP always-on, image input]
|
||||
ONE 401408-position KV pool at ctx 200704, 2 slots, MTP always-on, image input]
|
||||
G[llama.cpp PrismML fork CUDA build
|
||||
frank-zealot, port 7771]
|
||||
H[Ternary-Bonsai-2-27B PQ2_0 GGUF + Q8_0 mmproj
|
||||
@@ -87,11 +87,14 @@ frank-zealot, port 7771]
|
||||
**hardlinked** beside it — the card publishes only the main file, and the
|
||||
engine's own tools certified the stock companions (`niah` 18/18, `ppl`
|
||||
3.968 with mean KL 0.155 against the Swift1.5 logits on the same corpus).
|
||||
`HALOGEN_CTX=200704` over `HALOGEN_KV_POOL_POSITIONS=200704`: **one pool
|
||||
behind two slots** (the card ships 262,144/262,144 = 7.2 GiB; Frank took the
|
||||
RAM the same evening — 5.5 GiB reserved, one maximum-length request filling
|
||||
the pool), and the freed RAM becomes page cache for the n-gram
|
||||
table. MTP head always on, sampler 1.0/0.95/20, coupled cap/timeout
|
||||
`HALOGEN_CTX=200704` over `HALOGEN_KV_POOL_POSITIONS=401408` (**2 × ctx**):
|
||||
**one pool behind two slots**, sized so TWO maximum-length requests coexist —
|
||||
raised from 200,704 (5.5 GiB) on 2026-10-18 for frank-zealot parity on total
|
||||
in-flight context, ~10.8 GiB reserved against the card's 262,144/262,144 =
|
||||
7.2 GiB. The RAM comes out of the n-gram table's page cache, not out of a
|
||||
longer window: the per-request ceiling stays 200,704, because 262,144 is the
|
||||
native wall and the engine refuses more without `HALOGEN_ROPE_YARN`.
|
||||
MTP head always on, sampler 1.0/0.95/20, coupled cap/timeout
|
||||
65536/7200, prompt cache ON, vision ON. Measured against the Swift1.5
|
||||
boot: cold prefill +12.6 %, decode 50.0 vs 46.8 t/s, NIAH 18/18 vs 17/18,
|
||||
PPL +2.3 %, thinking tokens ×1.9 (the finetune's compression is gone).
|
||||
|
||||
Reference in New Issue
Block a user