dank/serverAppConfigs
1
0
Fork 0
Code Issues 6 Pull Requests Actions Packages Projects Releases Wiki Activity

feat(frank-dragoon): double the KV pool to 2 x ctx for slot parity

Browse Source 
frank-dragoon's total in-flight context was HALF frank-zealot's: zealot runs
one 401,408-position pool split into two 200,704 conversations, while dragoon
ran ONE 200,704-position pool behind its two slots, so a single maximum-length
request filled it and two long conversations had to stay under ~100k each.

HALOGEN_KV_POOL_POSITIONS 200704 -> 401408 = slots x ctx, the top of the seam's
legal band (ctx <= pool <= slots * ctx). The PER-REQUEST ceiling does not move:
HALOGEN_CTX stays 200704, because the checkpoint's native window is 262,144 and
the engine's own 400 text refuses to exceed it without HALOGEN_ROPE_YARN. What
this buys is concurrency, not a longer conversation, which is why the harness
contextWindow (155648 = ceiling - 45056) is untouched.

Cost: ~10.8 GiB of KV where 200,704 cost 5.5 (262,144 cost 7.2; the
2026-10-06 shape's 524,288 measured 14.6), taken from the page cache the 51.2 GB
n-gram table reads through — the warm prompt-cache ratio is the number to
re-measure after the boot. Engine-side total goes 74.1 -> ~79.4 GiB of 121.2.

Also carries scripts/download-frank-zealot-bonsai2-ptq1.sh: the PTQ1_0 candidate
payload fetcher (5,946,648,928 B, LFS sha256-verified, mmproj hardlinked), with
the publisher's own throughput table in its header — the packing is smaller but
the card measures it SLOWER on Ampere, so the swap is decided on our card, not
on the card's A100 row.
This commit is contained in:
Frank
2026-10-18 03:05:09 -07:00
parent a4fe5f6839
commit 2b127163c8
10 changed files with 209 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files
serverAppConfigs-docs/index.md
+4 -4
View File
@@ -313,10 +313,10 @@ The LLM inference fleet on the two hosts. Full index:
- [[frank-dragoon-flash-server-v2-native]] — **ACTIVE frank-dragoon runbook
since 2026-10-17**: halogen-flash-server 0.16.2 serving the **NATIVE v2
abliterated** conversion of orcarouter's pinned BF16
(`qwen38-flash-next-v2-abliterated.hgn`, 62.1 GiB) at ctx/pool **200,704**
(`HALOGEN_KV_POOL_POSITIONS=200704`, ~5.4 GiB — one pool behind both slots,
lowered the same evening from the card's 262,144) and the stock peonist
n-gram/vision companions **hardlinked**
(`qwen38-flash-next-v2-abliterated.hgn`, 62.1 GiB) at ctx **200,704** over a
**401,408**-position pool (`HALOGEN_KV_POOL_POSITIONS=401408`, ~10.8 GiB — one
pool behind both slots, raised to 2 × ctx on 2026-10-18 so TWO maximum-length
requests coexist) and the stock peonist n-gram/vision companions **hardlinked**
in — engine-tool certified (NIAH 18/18, PPL 3.968, mean KL 0.155); MTP
always-on, vision ON, no server-side sampler change
- [[frank-dragoon-flash-server-v2-swift15]] — RETIRED 2026-10-06 → 10-17