draft context (blk.32, f16, billed over the whole pool): 4,096 B/position → 2,352.00 MiB
against the 24,123 MiB the driver leaves addressable, with ~7.7 GiB of weights, projector, buffers and recurrent states
q8_0 V frees 4,410 MiB against exact V and UBATCH_SIZE=1024 hands back ~1,306 MiB of compute buffers. Those two are exactly what buys the third conversation and the drafter; exact V funds neither (18,816 MiB of cache alone at this pool). --flash-attn on stays pinned (quantized V is illegal without it) and fork build 10777 is the ≥ b10776 floor that pairs MTP with a quantized V.
Test seam
tests/frank_zealot_unit.bats (19/19) now asserts the observable contract, not the config text:
per-slot window: pool divides evenly, per-slot = 200,704 and ≤ the trained 262,144
the VRAM fit gate bills the armed drafter (whole-pool draft KV + buffers + cells) and still proves it REFUSES the two boots this shape can walk into — exact V with the drafter armed, and the retired f16/f16 pair at the 786,432 pool
the shipped MTP set reaches argv with no--spec-draft-model (that would load the trunk twice), and SPEC_TYPE=none strips the entire set (half a set has segfaulted at load)
./tests/run.sh — all 449 tests green. No benchmark was ordered, so no rates are claimed; the boot ledger is recorded on the endpoint page.
Rollback
One line each: CTX_SIZE=524288 PARALLEL=2 CACHE_TYPE_V=f16 BATCH_SIZE=4096 UBATCH_SIZE=2048 SPEC_TYPE=none and redeploy. No payload move.
## What changes
`frank-zealot` (RTX 3090, :7771) is resliced on Frank's order of 2026-10-19:
| Knob | Was | Now |
|---|---|---|
| Pool / slots | `--ctx-size 524288 --parallel 2` (262,144 a conversation) | `--ctx-size 602112 --parallel 3` (**200,704** a conversation) |
| KV cache | `f16` / `f16` (exact) | `f16` K (engine default) / **`q8_0`** V |
| Batch / ubatch | 4096 / 2048 | **6144** (1024×6) / **1024** |
| Speculation | `SPEC_TYPE=none` | **`SPEC_TYPE=draft-mtp`** (depth 2) |
Payload, runtime, alias, port, vision projector and the sampler recipe are untouched.
## Why it fits
One budget equation at 8,192 elements per token per cache (8 full-attention layers × 4 KV heads × 256 dim):
- cache: 16,384 B/token (f16 K) + 8,704 (q8_0 V) = **25,088 B/token → 14,406.00 MiB** over 602,112
- draft context (`blk.32`, f16, billed over the **whole** pool): 4,096 B/position → **2,352.00 MiB**
- against the **24,123 MiB** the driver leaves addressable, with ~7.7 GiB of weights, projector, buffers and recurrent states
`q8_0` V frees 4,410 MiB against exact V and `UBATCH_SIZE=1024` hands back ~1,306 MiB of compute buffers. Those two are exactly what buys the third conversation **and** the drafter; exact V funds neither (18,816 MiB of cache alone at this pool). `--flash-attn on` stays pinned (quantized V is illegal without it) and fork build 10777 is the ≥ b10776 floor that pairs MTP with a quantized V.
## Test seam
`tests/frank_zealot_unit.bats` (19/19) now asserts the observable contract, not the config text:
- per-slot window: pool divides evenly, per-slot = 200,704 and ≤ the trained 262,144
- the VRAM fit gate **bills the armed drafter** (whole-pool draft KV + buffers + cells) and still proves it REFUSES the two boots this shape can walk into — exact V with the drafter armed, and the retired f16/f16 pair at the 786,432 pool
- the shipped MTP set reaches argv with **no** `--spec-draft-model` (that would load the trunk twice), and `SPEC_TYPE=none` strips the **entire** set (half a set has segfaulted at load)
`./tests/run.sh` — all 449 tests green. No benchmark was ordered, so no rates are claimed; the boot ledger is recorded on the endpoint page.
## Rollback
One line each: `CTX_SIZE=524288 PARALLEL=2 CACHE_TYPE_V=f16 BATCH_SIZE=4096 UBATCH_SIZE=2048 SPEC_TYPE=none` and redeploy. No payload move.
---
cwd: /home/blank/.omp/wt/wt-20261019-020309-6d1fc1f · session: 7f352f4d0f041b0564688e79b954b10d1a536a0f52254402
Frank's order, 2026-10-19: --parallel 3 at 200,704 a conversation (pool
602,112), K at the engine default (f16), V to q8_0, batch 1024x6 / ubatch
1024, and the MTP drafter armed explicitly.
The three requests are one budget equation. At 8,192 elements per token per
cache the pair now bills 25,088 B/token = 14,406.00 MiB over the 602,112
pool, and the armed draft context adds its own f16 KV over that whole pool
(4,096 B/position = 2,352.00 MiB) plus buffers and recurrent cells. Dropping
V to the 8-bit class frees 4,410 MiB against exact V and the ubatch drop
hands back ~1,306 - that is what pays for the third slot AND the drafter,
neither of which exact V could fund.
The serve script's written-down fallbacks move with the env so a missing env
file cannot boot the retired shape, and the seam now: asserts the per-slot
contract (divides evenly, <= the trained 262,144), bills an armed drafter in
the VRAM fit gate, proves the gate still REFUSES both boots this shape can
walk into, pins the shipped MTP set (and the absence of --spec-draft-model,
which would load the trunk twice), and pins SPEC_TYPE=none as the rollback
that strips the whole set.
The endpoint page owns its shape: the flags row, the budget equation behind
it (25,088 B/token, 14,406.00 MiB of cache + 2,352.00 MiB of draft cache
inside 24,123 MiB addressable), the "MTP is ON" section replacing the "OFF by
measurement" one, the harness contract at the per-slot 200,704, and the
fallback rungs rewritten as deltas FROM the shipped shape.
The 2026-10-18 exact-KV boot (KV 16,384.00 / RS 100.50 / compute 1,408.06 +
1,088.07 / mmproj 868.32 charged after the fit check, settled 21,176/2,949)
and its contended rate floors move into that era's Eras row with the one-line
rollback, so the two pages that carried them collapse to one. Rates for the
new shape are deliberately absent - no bench was ordered, and the fit gate,
the health gate and the boot ledger are the evidence a swap needs.
AGENTS.md follows the same numbers: contextWindow 200704 is the per-slot
window and the trio's shape line.
dank
merged commit ba69ea563e into main2026-10-19 03:19:05 -07:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
What changes
frank-zealot(RTX 3090, :7771) is resliced on Frank's order of 2026-10-19:--ctx-size 524288 --parallel 2(262,144 a conversation)--ctx-size 602112 --parallel 3(200,704 a conversation)f16/f16(exact)f16K (engine default) /q8_0VSPEC_TYPE=noneSPEC_TYPE=draft-mtp(depth 2)Payload, runtime, alias, port, vision projector and the sampler recipe are untouched.
Why it fits
One budget equation at 8,192 elements per token per cache (8 full-attention layers × 4 KV heads × 256 dim):
blk.32, f16, billed over the whole pool): 4,096 B/position → 2,352.00 MiBq8_0V frees 4,410 MiB against exact V andUBATCH_SIZE=1024hands back ~1,306 MiB of compute buffers. Those two are exactly what buys the third conversation and the drafter; exact V funds neither (18,816 MiB of cache alone at this pool).--flash-attn onstays pinned (quantized V is illegal without it) and fork build 10777 is the ≥ b10776 floor that pairs MTP with a quantized V.Test seam
tests/frank_zealot_unit.bats(19/19) now asserts the observable contract, not the config text:--spec-draft-model(that would load the trunk twice), andSPEC_TYPE=nonestrips the entire set (half a set has segfaulted at load)./tests/run.sh— all 449 tests green. No benchmark was ordered, so no rates are claimed; the boot ledger is recorded on the endpoint page.Rollback
One line each:
CTX_SIZE=524288 PARALLEL=2 CACHE_TYPE_V=f16 BATCH_SIZE=4096 UBATCH_SIZE=2048 SPEC_TYPE=noneand redeploy. No payload move.cwd: /home/blank/.omp/wt/wt-20261019-020309-6d1fc1f · session: 7f352f4d0f041b0564688e79b954b10d1a536a0f52254402