MAX_SEQS=3 is a number the 1.8.1 engine cannot honour at the served window. The budget resolver sizes the KV cache from the cache bound (--max-model-len × --max-num-seqs), --gpu-headroom-mib and --host-pool-mib; the shipped CLI has no KV-size flag (--vram-weights-mib/--vram-kv-mib were removed at 1.3.0). The live boot reads:
kv cache 4.20 GiB 16% of it -- 93% backing of the 3 x 200000 worst case: 2.8 full-length sessions
kv: the cache can grow to 4.20 GiB a card -- 556348 tokens of context, 2.8 sessions at 200000
So a third full-height conversation is refused while /health answers 200 and models.yml advertises 200,000. Declare two — what the engine actually serves at the full window — and stop paying for a slot that only exists at shorter prompts. A slot costs KV, never decode (2026-10-06 sweep: seqs 2 and 3 tied inside the ±3 % floor); the fused TP all-reduce drops from 9 rows to 2 × (1 + spec) = 6, still under wht6's 25-row bit-exact bound.
HOST_POOL_MIB 45,568 → 44,928 and TTM_MIN_MIB 152,000 → 150,000. That raise bought cache the resolver never hands out. The two boot ledgers differ in exactly one line:
boot at 44,928
boot at 45,568
host_pool
44.47 GiB
45.09 GiB (env applied exactly)
vram_weights
25.81 GiB
25.81 GiB
vram_kv
4.20 GiB / 556,348 tokens
4.20 GiB / 556,348 tokens
The extra 640 MiB sat unclaimed while the TTM floor moved out from under the preflight's arithmetic. The compute-bound cap on the pool is unchanged either way (experts = min(floor × 2, 22,222 MiB) — 45,568 gives the same 22,222 as 44,928).
Frank picked this option (keep MAX_SEQS=2, keep 200,000, revert the raise) when the contract was re-asked against 1.8.1.
Tests
./tests/run.sh — 448 ok, 0 fail. The shape seam pins MAX_SEQS=2, pool 44,928 and the ledger floor (2 × 44,928 + 48,824 + 8,192 + 2,048 = 148,920 ≤ 150,000).
Deploy gate (what I will read on the boot ledger)
kv cache … -- N sessions at 200000 must read 2.0, not 2.8, and kv_attn blocks ≈ 100,000 (400,000 tokens) instead of 139,087;
experts should rise from 22.22 GiB toward ~23.3 GiB as the elastic pool keeps what the cache no longer needs — a prediction, read off the boot, not asserted here;
then one chat turn, one tool call and one solid-colour vision probe.
Documentation (vault page + AGENTS.md shape block) is deliberately not in this PR: per Frank's rule it lands after the shape has booted, written from the measured ledger rather than from arithmetic.
## Contract
`MAX_SEQS=3` is a number the 1.8.1 engine cannot honour at the served window. The budget resolver sizes the KV cache from the cache bound (`--max-model-len` × `--max-num-seqs`), `--gpu-headroom-mib` and `--host-pool-mib`; the shipped CLI has no KV-size flag (`--vram-weights-mib`/`--vram-kv-mib` were removed at 1.3.0). The live boot reads:
```
kv cache 4.20 GiB 16% of it -- 93% backing of the 3 x 200000 worst case: 2.8 full-length sessions
kv: the cache can grow to 4.20 GiB a card -- 556348 tokens of context, 2.8 sessions at 200000
```
So a **third** full-height conversation is refused while `/health` answers 200 and `models.yml` advertises 200,000. Declare two — what the engine actually serves at the full window — and stop paying for a slot that only exists at shorter prompts. A slot costs KV, never decode (2026-10-06 sweep: seqs 2 and 3 tied inside the ±3 % floor); the fused TP all-reduce drops from 9 rows to 2 × (1 + spec) = 6, still under `wht6`'s 25-row bit-exact bound.
## Revert of PR #30
`HOST_POOL_MIB` 45,568 → 44,928 and `TTM_MIN_MIB` 152,000 → 150,000. That raise bought cache the resolver never hands out. The two boot ledgers differ in exactly one line:
| | boot at 44,928 | boot at 45,568 |
|---|---|---|
| `host_pool` | 44.47 GiB | **45.09 GiB** (env applied exactly) |
| `vram_weights` | 25.81 GiB | 25.81 GiB |
| `vram_kv` | 4.20 GiB / 556,348 tokens | 4.20 GiB / 556,348 tokens |
The extra 640 MiB sat unclaimed while the TTM floor moved out from under the preflight's arithmetic. The compute-bound cap on the pool is unchanged either way (`experts = min(floor × 2, 22,222 MiB)` — 45,568 gives the same 22,222 as 44,928).
Frank picked this option (keep `MAX_SEQS=2`, keep 200,000, revert the raise) when the contract was re-asked against 1.8.1.
## Tests
`./tests/run.sh` — 448 ok, 0 fail. The shape seam pins `MAX_SEQS=2`, pool 44,928 and the ledger floor (2 × 44,928 + 48,824 + 8,192 + 2,048 = 148,920 ≤ 150,000).
## Deploy gate (what I will read on the boot ledger)
- `kv cache … -- N sessions at 200000` must read **2.0**, not 2.8, and `kv_attn` blocks ≈ 100,000 (400,000 tokens) instead of 139,087;
- `experts` should rise from 22.22 GiB toward ~23.3 GiB as the elastic pool keeps what the cache no longer needs — a prediction, read off the boot, not asserted here;
- then one chat turn, one tool call and one solid-colour vision probe.
Documentation (vault page + `AGENTS.md` shape block) is deliberately **not** in this PR: per Frank's rule it lands after the shape has booted, written from the measured ledger rather than from arithmetic.
---
cwd: /home/blank/.omp/wt/wt-20261019-020309-6d1fc1f · session: wt-20261019-020309-6d1fc1f · build worktree: /home/blank/.omp/wt/wt-archon-181c
FRANK_ARCHON_MAX_SEQS 3 -> 2. At 1.8.1 the budget resolver sizes the KV cache
from the cache bound (--max-model-len x --max-num-seqs), --gpu-headroom-mib and
--host-pool-mib, and the shipped CLI has no KV-size flag (the
--vram-weights-mib/--vram-kv-mib pair was removed at 1.3.0). The live boot
ledger reads that cache at 4.20 GiB = 556,348 tokens = 2.8 sessions at 200,000,
so a third full-height conversation is refused while /health answers green and
the harness advertises capacity the engine cannot honour. Two slots is what the
shape actually serves at the full window; a slot costs KV, never decode, and the
fused TP all-reduce drops from 9 rows to 2 x (1 + spec) = 6, still inside
wht6's 25-row bit-exact bound.
FRANK_ARCHON_HOST_POOL_MIB back to 44928 and FRANK_ARCHON_TTM_MIN_MIB back to
150000, reverting PR #30. That raise bought cache the resolver never hands out:
the two boot ledgers differ in exactly one line -- host_pool 44.47 -> 45.09 GiB
-- while vram_weights (25.81 GiB) and vram_kv (4.20 GiB / 556,348 tokens, both
ranks) are byte-identical, so the extra 640 MiB sat unclaimed in the host pool
while the TTM floor moved out from under the preflight's arithmetic. The
compute-bound cap on --host-pool-mib is unchanged either way (45,568 -> 22,222
MiB of experts either side of the 22,222 clamp).
The bats seam pins the new contract: MAX_SEQS=2, pool 44,928, and the ledger
floor that computes to 148,920 MiB at that pool against the 150,000 minimum.
dank
merged commit 88342c144b into main2026-10-19 11:09:24 -07:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Contract
MAX_SEQS=3is a number the 1.8.1 engine cannot honour at the served window. The budget resolver sizes the KV cache from the cache bound (--max-model-len×--max-num-seqs),--gpu-headroom-miband--host-pool-mib; the shipped CLI has no KV-size flag (--vram-weights-mib/--vram-kv-mibwere removed at 1.3.0). The live boot reads:So a third full-height conversation is refused while
/healthanswers 200 andmodels.ymladvertises 200,000. Declare two — what the engine actually serves at the full window — and stop paying for a slot that only exists at shorter prompts. A slot costs KV, never decode (2026-10-06 sweep: seqs 2 and 3 tied inside the ±3 % floor); the fused TP all-reduce drops from 9 rows to 2 × (1 + spec) = 6, still underwht6's 25-row bit-exact bound.Revert of PR #30
HOST_POOL_MIB45,568 → 44,928 andTTM_MIN_MIB152,000 → 150,000. That raise bought cache the resolver never hands out. The two boot ledgers differ in exactly one line:host_poolvram_weightsvram_kvThe extra 640 MiB sat unclaimed while the TTM floor moved out from under the preflight's arithmetic. The compute-bound cap on the pool is unchanged either way (
experts = min(floor × 2, 22,222 MiB)— 45,568 gives the same 22,222 as 44,928).Frank picked this option (keep
MAX_SEQS=2, keep 200,000, revert the raise) when the contract was re-asked against 1.8.1.Tests
./tests/run.sh— 448 ok, 0 fail. The shape seam pinsMAX_SEQS=2, pool 44,928 and the ledger floor (2 × 44,928 + 48,824 + 8,192 + 2,048 = 148,920 ≤ 150,000).Deploy gate (what I will read on the boot ledger)
kv cache … -- N sessions at 200000must read 2.0, not 2.8, andkv_attnblocks ≈ 100,000 (400,000 tokens) instead of 139,087;expertsshould rise from 22.22 GiB toward ~23.3 GiB as the elastic pool keeps what the cache no longer needs — a prediction, read off the boot, not asserted here;Documentation (vault page +
AGENTS.mdshape block) is deliberately not in this PR: per Frank's rule it lands after the shape has booted, written from the measured ledger rather than from arithmetic.cwd: /home/blank/.omp/wt/wt-20261019-020309-6d1fc1f · session: wt-20261019-020309-6d1fc1f · build worktree: /home/blank/.omp/wt/wt-archon-181c