feat(frank-dragoon): double the KV pool to 2 x ctx for slot parity
frank-dragoon's total in-flight context was HALF frank-zealot's: zealot runs one 401,408-position pool split into two 200,704 conversations, while dragoon ran ONE 200,704-position pool behind its two slots, so a single maximum-length request filled it and two long conversations had to stay under ~100k each. HALOGEN_KV_POOL_POSITIONS 200704 -> 401408 = slots x ctx, the top of the seam's legal band (ctx <= pool <= slots * ctx). The PER-REQUEST ceiling does not move: HALOGEN_CTX stays 200704, because the checkpoint's native window is 262,144 and the engine's own 400 text refuses to exceed it without HALOGEN_ROPE_YARN. What this buys is concurrency, not a longer conversation, which is why the harness contextWindow (155648 = ceiling - 45056) is untouched. Cost: ~10.8 GiB of KV where 200,704 cost 5.5 (262,144 cost 7.2; the 2026-10-06 shape's 524,288 measured 14.6), taken from the page cache the 51.2 GB n-gram table reads through — the warm prompt-cache ratio is the number to re-measure after the boot. Engine-side total goes 74.1 -> ~79.4 GiB of 121.2. Also carries scripts/download-frank-zealot-bonsai2-ptq1.sh: the PTQ1_0 candidate payload fetcher (5,946,648,928 B, LFS sha256-verified, mmproj hardlinked), with the publisher's own throughput table in its header — the packing is smaller but the card measures it SLOWER on Ampere, so the swap is decided on our card, not on the card's A100 row.
This commit is contained in:
1 parent
a4fe5f6839
commit
2b127163c8
10 files changed
+209
-62
No files matched your search
+27
-18
@@ -20,9 +20,11 @@
|
||||
# thinking tokens 1,476 vs 784 mean — the finetune compressed reasoning, the
|
||||
# base thinks ~1.9x longer per answer.
|
||||
#
|
||||
# The A/B ran at the card's ctx/pool of 262,144. Afterwards Frank lowered both
|
||||
# to 200,704 (RAM + the harness's compaction budget); the rates above are the
|
||||
# ceiling-262,144 record and the production ledger is the 200,704 one.
|
||||
# The A/B ran at the card's ctx/pool of 262,144. Afterwards Frank lowered ctx to
|
||||
# 200,704 (RAM + the harness's compaction budget) and, on 2026-10-18, raised the
|
||||
# pool to 2 x ctx = 401,408 so BOTH slots can carry a full-length request at
|
||||
# once; the rates above are the ceiling-262,144 record and the production ledger
|
||||
# is the ctx-200,704 one.
|
||||
#
|
||||
# WHY THE RUNTIME MOVED WITH THE MODEL: the payload is .hgn — per its own
|
||||
# card it "will not load in transformers, vLLM, or llama.cpp" — and the v2
|
||||
@@ -57,15 +59,21 @@
|
||||
# est-error ≈ 156.2k <= 200,704 — ~9.5× the 3 % error of margin (robust to a
|
||||
# ~25 % under-count). Compaction rides on frank-zealot: input ≈ threshold −
|
||||
# keepRecent(20k) ≈ 112k, inside its 200,704 slot.
|
||||
# HALOGEN_KV_POOL_POSITIONS=200704 — ONE pool behind HALOGEN_KV_SLOTS=2
|
||||
# (ledger form: "2 slots over ONE <pool>-position KV pool, each request up to
|
||||
# HALOGEN_CTX"). With pool == ctx a single MAXIMUM-length request fills the
|
||||
# whole pool. ~28 KiB/position (FLAGS.md) ⇒ ~5.4 GiB, where 262144 cost 7.2
|
||||
# and the 2026-10-06 shape's 524,288 measured 14.6. The engine's own preflight
|
||||
# formula bills a little more than the reservation because it adds block
|
||||
# scratch and ~115 MiB/slot of O(1) state — quote the RESERVATION for the RAM
|
||||
# ledger. FLAGS.md law: "RAM the pool takes is RAM that table loses", so every
|
||||
# position cut is page cache for the 47.68 GiB n-gram table long prompts read.
|
||||
# HALOGEN_KV_POOL_POSITIONS=401408 = SLOTS x CTX (2 x 200,704), the top of the
|
||||
# legal band (the seam enforces ctx <= pool <= slots * ctx). Still ONE pool
|
||||
# behind HALOGEN_KV_SLOTS=2 (ledger form: "2 slots over ONE <pool>-position KV
|
||||
# pool, each request up to HALOGEN_CTX"), but sized so BOTH slots can hold a
|
||||
# MAXIMUM-length request at the same time — the parity with frank-zealot's
|
||||
# 401,408-position pool that Frank ordered on 2026-10-18. The PER-REQUEST
|
||||
# ceiling does NOT move with it: HALOGEN_CTX stays 200,704, because the
|
||||
# checkpoint's native window is 262,144 and the engine refuses to go past it
|
||||
# without HALOGEN_ROPE_YARN (its own 400 text says so). ~28 KiB/position
|
||||
# (FLAGS.md) ⇒ ~10.8 GiB, where pool == ctx (200,704) cost 5.5, 262,144 cost
|
||||
# 7.2 and the 2026-10-06 shape's 524,288 measured 14.6. The engine's preflight
|
||||
# formula bills a little more than the reservation (block scratch + ~115
|
||||
# MiB/slot of O(1) state) — quote the RESERVATION. FLAGS.md law: "RAM the pool
|
||||
# takes is RAM that table loses" — the 47.68 GiB n-gram table surrenders this
|
||||
# much page cache, so the warm prompt-cache ratio is what to re-measure.
|
||||
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
|
||||
# drafter is NOT slot-locked on this engine (batching and speculation
|
||||
# coexist — DFlash-era slots>1 law does NOT carry over; the card's own
|
||||
@@ -74,13 +82,14 @@
|
||||
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is
|
||||
# mmap'd page cache beside the checkpoint, not a pinned charge — the boot
|
||||
# ledger agrees: "mapped 51.2 GB, NOT pinned (Pin::None)"):
|
||||
# weights ~62.9 GiB locked (0.8 of them the vision tower) + KV ~5.4 +
|
||||
# working ~5.7 (HALOGEN_MAX_TOK=8192) ≈ 74 GiB engine-side, with the
|
||||
# weights ~62.9 GiB locked (0.8 of them the vision tower) + KV ~10.8 +
|
||||
# working ~5.7 (HALOGEN_MAX_TOK=8192) ≈ 79.4 GiB engine-side, with the
|
||||
# prompt cache (~1.0 at this ctx; 1.3 was measured at 262144) booked
|
||||
# separately by the ledger; HALOGEN_HOST_RESERVE_GIB=20 keeps the host
|
||||
# honest; baseline co-tenancy (frank-zealot, ComfyUI, Caddy/observ)
|
||||
# ~10-15. ~30 GiB margin. The boot ledger is the referee — read it after
|
||||
# deploy and re-size if the engine prints a clamp.
|
||||
# honest. Against the 121.2 GiB box that leaves ~22 GiB over the reserve for
|
||||
# baseline co-tenancy (frank-zealot, ComfyUI, Caddy/observ ~10-15). The boot
|
||||
# ledger is the referee — read it after deploy and re-size if the engine
|
||||
# prints a clamp.
|
||||
# HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
|
||||
# turns the MTP head off for whole requests after low acceptance and
|
||||
# HALVES decode as a result. Frank accepted the head always-on trade
|
||||
@@ -174,7 +183,7 @@ FRANK_DRAGOON_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_conf
|
||||
# model_ids for the :7770 backend MUST name this value.
|
||||
HALOGEN_MODEL_ID=frank-dragoon
|
||||
HALOGEN_CTX=200704
|
||||
HALOGEN_KV_POOL_POSITIONS=200704
|
||||
HALOGEN_KV_POOL_POSITIONS=401408
|
||||
HALOGEN_KV_SLOTS=2
|
||||
HALOGEN_HOST_RESERVE_GIB=20
|
||||
# Spec decode: MTP head is INSIDE the checkpoint (do NOT set HALOGEN_MTP_HEAD);
|
||||
|
||||
Reference in new issue
Block a user