frank-large: raise ctx to 262144 to close the harness 400-zone
The harness clamps max_tokens to min(45056, contextWindow - est - 4096), its chars/4 est under-counts the rendered prompt by ~3 % (measured: 152,938 -> 157,388 and 158,906 -> 165,301), and compaction fires only at 85 % -- so with window == engine ceiling 200192, every request in the est ~146k-184k zone was a guaranteed 400. HALOGEN_CTX=262144 (native) + KV pool 524288; harness window 217088 = ceiling - 45k absorbs the estimate error (worst-case request ~219k <= 262144). models.yml updated out-of-repo: contextWindow 217088, maxTokens 45056.
This commit is contained in:
Frank
@@ -26,7 +26,7 @@ merge absorbed the former `vllmScripts` repo, history preserved).
|
||||
| Target | Service | Runtime | Model | Port |
|
||||
|--------|---------|---------|-------|------|
|
||||
| Dankserver | `frank-primary` | **radiance** (StillDeadcode v1.0.7 line; image `stilldeadcode/radiance@sha256:4c3f484564…` digest-pinned INDEX digest, 287 MiB, `libr4d` gfx1201 kernels, plain `podman run`, root systemd unit) | `Qwen/Qwen3.8-Flash-Next-FP8` official (172.8 GB, 131 shards, sha256-verified) converted **no-recipe** to `qwen4exp-fp8-native.rad` (177,109 MiB measured = 173.0 GiB) — **native modelopt block-FP8, no requantisation** — on **BOTH R9700s TP=2** (`--tp-wire wht6`), experts in a **40 GiB/rank pinned host pool** + VRAM cache (`--placement expert_tiered`, `--expert-vs-cache-ratio 0.82` = the 99 %-hit point), **PLE 47.7 GiB pinned to host RAM** (`--ngram-placement ram`; root unit + `LimitMEMLOCK=infinity`), **fp8 KV 7.32 KiB/token**, **MTP spec decode depth 3**, *…
|
||||
| Dankclaw | `frank-large` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the head per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 200,192 ctx** (KV pool 400,384 positions ≈ 10.7 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold) | 7770 |
|
||||
| Dankclaw | `frank-large` | **halogen-flash-server 0.16.2** (`ghcr.io/peonist-ai/halogen-flash-server@sha256:0c61bf84…`, digest-pinned, plain `podman run` via `scripts/serve-frank-large.sh`, root systemd unit) | Quat3rnion/halogen-swift1.5-qwen3.8-flash-next-v2-abliterated **`qwen38-flash-next-v2-swift15-abliterated.hgn`** (66,687,678,464 B = compacted r2 of the card's 89,775,868,032 B original, ~62 GiB resident; the v2 `.hgn` format loads in NO other engine), **+ REQUIRED `qwen38-flash-next-ngram.hgn`** (51,200,246,144 B, peonist @5cc17cea — read beside the checkpoint) **+ vision tower** (897,916,416 B, @e053f488 — **vision ON**, `image_url` parts only, ≥256 px probes), **MTP spec decode ALWAYS-ON (`HALOGEN_SPEC_ADAPT=0` — the adaptive default disables the drafter per request and HALVES decode)**, MTP head INSIDE the checkpoint (`HALOGEN_MTP_HEAD` unset; no sidecar fetched), **2 slots × 262,144 ctx** (KV pool 524,288 positions ≈ 14.0 GiB at ~28 KiB/position; the 27B slots>1-disables-drafting law does NOT carry over — batching and speculation coexist, the card's quickstart ships 2 with the head), harness window 217,088 = ceiling − 45k estimate headroom, **server-side sampler 1.0/0.95/20** (card recipe; models.yml extraBody matches), coupled cap/timeout 65536/7200, prompt cache ON by default (warm 100k turn ≈ 2 s vs ≈ 88 s cold)|7770|
|
||||
| Dankclaw | `frank-secondary` | llama.cpp master (0.5.0-dev, commit `5fc4f3c`), CUDA sm_86 build (root systemd unit, relinked) | HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive **Q6_K** (7,359,259,008 B; dense `qwen35` hybrid: 32 layers, 8 full-attention + 24 Gated DeltaNet, 4 KV heads × 256 dim, vocab 248,320, native 262,144, **no MTP tensors in the file**) + the publisher's own **BF16 mmproj** (921,704,448 B) for **vision**, on the RTX 3090, **TWO 200,704-token slots** (`--ctx-size 401408 --parallel 2`), **exact f16 K + f16 V** under flash attention (32 KiB/token over the 8 full layers = 12.25 GiB pool), batch 4096 / **ubatch 2048** (measured CUDA optimum), `--log-verbosity 4`, **MTP OFF** (forced: no `nextn` tensors exist). Priced from the MiMo Q6_K boots (same geometry, same card, same shape): 21,276/2,849 at ubatch 1024, 21,846/2,279 at 2048. Cutover 2026-10-07 (late) from Gemma4-26B-A4B Q4_K_M 1×200,704. All three endpoints route harness **compaction** here. | 7771 |
|
||||
|
||||
Retired endpoints (`frank-npu`, and every prior runtime) are in
|
||||
@@ -367,15 +367,22 @@ endpoint).
|
||||
with evidence): `HALOGEN_SPEC_ADAPT=0` is MANDATORY — the default adaptive
|
||||
policy disables the MTP head for whole requests after low acceptance and
|
||||
HALVES decode; the head stays on and temp-1.0 traffic pays some acceptance
|
||||
for it (accepted). `HALOGEN_CTX=200192` is the PER-SLOT hard ceiling (400
|
||||
naming the limit, never truncation); `models.yml` `contextWindow: 200192`
|
||||
must match, and compaction at 70 % = 140,134 + 57,344 budget = 197,478 fits.
|
||||
for it (accepted). `HALOGEN_CTX=262144` is the PER-SLOT hard ceiling (400
|
||||
naming the limit, never truncation) — RAISED from 200192 on 2026-10-07
|
||||
under the 400-zone law: the harness clamps max_tokens to
|
||||
`min(45056, contextWindow − est − 4096)`, its chars/4 est UNDER-counts
|
||||
the rendered prompt by ~3 % (measured: est 152,938 → server 157,388,
|
||||
request 43,158 vs room 42,804), so a harness window == engine ceiling
|
||||
made every request over ~146k est a guaranteed 400. `models.yml`
|
||||
`contextWindow: 217088` leaves the 45k gap that absorbs the estimate
|
||||
error (worst-case request ≈ 219k ≤ 262144); compaction fires at 85 % of
|
||||
the harness window = 184,524 est ≈ 190k measured (Frank's 180k target).
|
||||
`HALOGEN_KV_SLOTS=2` is LEGAL here — the 27B "slots>1 disables the drafter"
|
||||
law does NOT carry over; batching and speculation coexist (the card's
|
||||
quickstart ships 2 slots with the head), so 2 is concurrent capacity.
|
||||
`HALOGEN_KV_POOL_POSITIONS=400384` (explicit 2 × ctx; ~28 KiB/position ⇒
|
||||
~10.7 GiB). `HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the
|
||||
harness always sends 57,344); `HALOGEN_MAX_TOKENS_CAP=65536` +
|
||||
`HALOGEN_KV_POOL_POSITIONS=524288` (explicit 2 × ctx; ~28 KiB/position ⇒
|
||||
14.0 GiB). `HALOGEN_MAX_TOK=8192` is only the per-request DEFAULT (the
|
||||
harness sends at most its `maxTokens` 45056); `HALOGEN_MAX_TOKENS_CAP=65536` +
|
||||
`HALOGEN_QUEUE_TIMEOUT=7200` are a coupled pair (FLAGS.md table).
|
||||
**A server-side sampler EXISTS** (`HALOGEN_TEMPERATURE/TOP_P/TOP_K` =
|
||||
1.0/0.95/20, the card's thinking recipe) — an absent temperature is no
|
||||
@@ -391,8 +398,8 @@ stays mandatory. Thinking tokens COUNT against the budget (empty content +
|
||||
"video".
|
||||
|
||||
**Memory ledger** (121.2 GiB unified; the referee is the boot ledger):
|
||||
weights ~62 + KV ~10.7 + arena ~8 (`3.8 + 8.4 × MAX_TOK/16384`) + vision
|
||||
~2 ≈ 82.7 GiB engine-side + `HALOGEN_HOST_RESERVE_GIB=20` → ~18 GiB
|
||||
weights ~62 + KV 14.0 + arena ~8 (`3.8 + 8.4 × MAX_TOK/16384`) + vision
|
||||
~2 ≈ 86 GiB engine-side + `HALOGEN_HOST_RESERVE_GIB=20` → ~15 GiB
|
||||
margin over baseline co-tenancy. The 51.2 GB n-gram table rides as mmap'd
|
||||
page cache beside the checkpoint — the card's quickstart arithmetic does
|
||||
not bill it as resident, but READ the boot ledger and re-size if the
|
||||
@@ -469,7 +476,7 @@ OpenAI `image_url` content part reaches the model.
|
||||
|
||||
**Compaction hub**: `models.yml` routes `compactionModel` for
|
||||
`frank-primary`, `frank-large` and `frank-secondary` here — 200,704 is the
|
||||
only window in the fleet that fits a worst-case compaction input (~133k
|
||||
only window in the fleet that fits a worst-case compaction input (~170k
|
||||
tokens). With `--parallel 2` a compaction request no longer serializes behind
|
||||
a live generation (the Gemma4 era's `--parallel 1` law).
|
||||
|
||||
@@ -1002,7 +1009,7 @@ Active:
|
||||
|
||||
| Service | Runbook |
|
||||
|---------|---------|
|
||||
| frank-large (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×200,192, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-flash-server-v2-swift15.md` |
|
||||
| frank-large (flash-server 0.16.2, abliterated Swift1.5 v2 .hgn + MTP always-on, 2×262,144, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-large-flash-server-v2-swift15.md` |
|
||||
| frank-secondary (llama.cpp CUDA, Qwen3.5-9B-Uncensored Q6_K, 2×200,704 exact f16 KV MTP-off, vision ON, Dankclaw) | `serverAppConfigs-docs/LLM/frank-secondary-llama-qwen35-9b-q6k-2x200704.md` |
|
||||
| frank-primary (radiance StillDeadcode, Qwen3.8-Flash-Next-FP8 native, 2×R9700 TP=2, Dankserver) | `serverAppConfigs-docs/LLM/frank-primary-radiance-qwen38-native-fp8.md` |
|
||||
| Dankclaw kernel posture (`amd_iommu=off`) | `serverAppConfigs-docs/LLM/dankclaw-iommu-translation-off.md` |
|
||||
|
||||
+24
-14
@@ -22,15 +22,25 @@
|
||||
# tool calls verified, and the native 262,144 context.
|
||||
#
|
||||
# SHAPE — the knobs that are NOT free, with the arithmetic:
|
||||
# HALOGEN_CTX=200192 is the PER-SLOT window and the per-conversation hard
|
||||
# HALOGEN_CTX=262144 is the PER-SLOT window and the per-conversation hard
|
||||
# ceiling — a prompt over it is a 400 naming the limit, never a silent
|
||||
# truncation. Harness compaction fires at 70% = 140,134, and 140,134 +
|
||||
# 57,344 (harness maxTokens) = 197,478 <= 200,192 keeps prompt + budget
|
||||
# inside the slot. models.yml contextWindow must quote 200192 or the
|
||||
# harness discovers the wall as a 400.
|
||||
# HALOGEN_KV_POOL_POSITIONS=400384 = KV_SLOTS(2) x CTX — the explicit
|
||||
# truncation. RAISED from 200192 on 2026-10-07: the harness clamps
|
||||
# max_tokens to contextWindow − est − 4096, its chars/4 est UNDER-counts
|
||||
# the rendered prompt by ~3 % on code/agent traffic, and compaction only
|
||||
# fires at 85 % — so with window == engine ceiling, every request over
|
||||
# ~146k est (65-85 % of the window) was a guaranteed 400. Measured
|
||||
# 2026-10-07: est 152,938, server 157,388, request 43,158 vs room 42,804
|
||||
# — over by 354, margin 4,096 < the 4,450 under-count.
|
||||
# Harness contract (models.yml): contextWindow 217088, maxTokens 45056.
|
||||
# Compaction fires at 85 % = 184,524 est ≈ 190k measured (Frank's 180k
|
||||
# target). Worst-case request: prompt + budget <= 217088 − 4096 + 3 %
|
||||
# est-error ≈ 219k <= 262144 — the 45k gap between harness window and
|
||||
# engine ceiling absorbs the estimate error with ~7x margin (robust to a
|
||||
# 10 % under-count). Compaction rides on frank-secondary: input ≈
|
||||
# threshold − keepRecent(20k) ≈ 170k measured, inside its 200,704 slot.
|
||||
# HALOGEN_KV_POOL_POSITIONS=524288 = KV_SLOTS(2) x CTX — the explicit
|
||||
# form of the engine's default, so the seam test can pin the product.
|
||||
# KV costs ~28 KiB/position (FLAGS.md) => 400,384 x 28 KiB = 10.7 GiB.
|
||||
# KV costs ~28 KiB/position (FLAGS.md) => 524,288 x 28 KiB = 14.0 GiB.
|
||||
# HALOGEN_KV_SLOTS=2 restores the w4b-era shape on this box: the MTP
|
||||
# drafter is NOT slot-locked on this engine (batching and speculation
|
||||
# coexist — DFlash-era slots>1 law does NOT carry over; the card's own
|
||||
@@ -39,11 +49,11 @@
|
||||
# Memory ledger against the box's 121.2 GiB unified (the n-gram table is
|
||||
# mmap'd page cache beside the checkpoint, not a pinned charge — the
|
||||
# card's own quickstart arithmetic does not bill it):
|
||||
# weights ~62 GiB resident + KV 10.7 + arena (HALOGEN_MAX_TOK=8192 =>
|
||||
# weights ~62 GiB resident + KV 14.0 + arena (HALOGEN_MAX_TOK=8192 =>
|
||||
# ~8 GiB per FLAGS.md arena formula 3.8+8.4*(MT/16384)) + vision ~2
|
||||
# = ~82.7 GiB engine-side; HALOGEN_HOST_RESERVE_GIB=20 keeps the host
|
||||
# = ~86 GiB engine-side; HALOGEN_HOST_RESERVE_GIB=20 keeps the host
|
||||
# honest; baseline co-tenancy (frank-secondary, ComfyUI, Caddy/observ)
|
||||
# ~10-15. ~18 GiB margin. The boot ledger is the referee — read it
|
||||
# ~10-15. ~15 GiB margin. The boot ledger is the referee — read it
|
||||
# after deploy and re-size if the engine prints a clamp.
|
||||
# HALOGEN_SPEC_ADAPT=0 is MANDATORY (card): the default adaptive policy
|
||||
# turns the MTP head off for whole requests after low acceptance and
|
||||
@@ -57,8 +67,8 @@
|
||||
# an absent temperature is no longer greedy. models.yml extraBody
|
||||
# carries the same values — consistent, not load-bearing.
|
||||
# * HALOGEN_MAX_TOK=8192 is only the DEFAULT for requests that omit
|
||||
# max_tokens; the harness always sends 57344. Requests over
|
||||
# HALOGEN_MAX_TOKENS_CAP (65536) are a 400 naming the cap. Cap and
|
||||
# max_tokens; the harness sends at most its maxTokens 45056. Requests
|
||||
# over HALOGEN_MAX_TOKENS_CAP (65536) are a 400 naming the cap. Cap and
|
||||
# queue timeout are COUPLED (FLAGS.md table): 65536 at ~10 t/s serial
|
||||
# worst case needs > 6550 s; 7200 honors it. Do not move one alone.
|
||||
# * max_tokens INCLUDES thinking; a reasoning reply over a small budget
|
||||
@@ -121,8 +131,8 @@ FRANK_LARGE_TOKENIZER_FILES="tokenizer.json chat_template.jinja tokenizer_config
|
||||
# --- engine flags (docs/FLAGS.md of the flash-server repo; anything unset
|
||||
# stays at the image default, which is what was qualified) ------------------
|
||||
HALOGEN_MODEL_ID=frank-large
|
||||
HALOGEN_CTX=200192
|
||||
HALOGEN_KV_POOL_POSITIONS=400384
|
||||
HALOGEN_CTX=262144
|
||||
HALOGEN_KV_POOL_POSITIONS=524288
|
||||
HALOGEN_KV_SLOTS=2
|
||||
HALOGEN_HOST_RESERVE_GIB=20
|
||||
# Spec decode: MTP head is INSIDE the checkpoint (do NOT set HALOGEN_MTP_HEAD);
|
||||
|
||||
@@ -80,21 +80,21 @@ resume law throughout; no token — the repos are public).
|
||||
|
||||
| Env | Value | Why |
|
||||
|---|---|---|
|
||||
| `HALOGEN_CTX` | 200192 | per-slot HARD ceiling (a longer prompt is a 400 naming the limit, never truncation). Harness compaction: 70 % = 140,134; 140,134 + 57,344 = 197,478 ≤ 200,192 |
|
||||
| `HALOGEN_CTX` | 262144 | per-slot HARD ceiling (the checkpoint's native window; a longer prompt is a 400 naming the limit, never truncation). **RAISED from 200192 on 2026-10-07 — the 400-zone law:** the harness clamps `max_tokens = min(45056, contextWindow − est − 4096)`, its chars/4 est UNDER-counts the rendered prompt by ~3 % (measured twice: est 152,938 → server 157,388, request over room by 354; est 158,906 → server 165,301, over by 2,300), and compaction only fires at 85 % of the harness window — so with window == engine ceiling, every request in the est ~146k–184k zone was a guaranteed 400. Harness window 217,088 = ceiling − 45k absorbs the error: worst-case request ≈ 219k ≤ 262,144 (≈10 % under-count headroom) |
|
||||
| `HALOGEN_KV_SLOTS` | 2 | the 27B engine's "slots>1 disables the drafter" law does NOT carry over — on this engine batching and speculation coexist (the card's quickstart ships 2 slots WITH the head). 2 slots = concurrent capacity |
|
||||
| `HALOGEN_KV_POOL_POSITIONS` | 400384 | explicit form of the default (2 × ctx), so the seam test pins the product; ~28 KiB/position ⇒ ~10.7 GiB KV |
|
||||
| `HALOGEN_KV_POOL_POSITIONS` | 524288 | explicit form of the default (2 × ctx), so the seam test pins the product; ~28 KiB/position ⇒ 14.0 GiB KV |
|
||||
| `HALOGEN_HOST_RESERVE_GIB` | 20 | keeps the host honest (the w4b-era lesson) |
|
||||
| `HALOGEN_SPEC_ADAPT` | **0** | MANDATORY (card): the default adaptive policy turns the MTP head off for whole requests after low acceptance and **halves decode**. The head stays on; temp-1.0 traffic pays some acceptance for it — accepted trade |
|
||||
| `HALOGEN_MAX_TOK` / `HALOGEN_MAX_TOKENS_CAP` / `HALOGEN_QUEUE_TIMEOUT` | 8192 / 65536 / 7200 | 8192 is only the per-request DEFAULT (the harness always sends 57,344); the cap/timeout pair is COUPLED per FLAGS.md (65536 at ~10 t/s serial needs > 6550 s) |
|
||||
| `HALOGEN_MAX_TOK` / `HALOGEN_MAX_TOKENS_CAP` / `HALOGEN_QUEUE_TIMEOUT` | 8192 / 65536 / 7200 | 8192 is only the per-request DEFAULT (the harness sends at most its `maxTokens` 45,056); the cap/timeout pair is COUPLED per FLAGS.md (65536 at ~10 t/s serial needs > 6550 s) |
|
||||
| `HALOGEN_TEMPERATURE` / `TOP_P` / `TOP_K` | 1.0 / 0.95 / 20 | the card's thinking recipe as SERVER-SIDE defaults — the engine HAS a sampler now, so an absent temperature is no longer greedy; models.yml extraBody matches it (consistent, not load-bearing) |
|
||||
| `HALOGEN_WEIGHTS_LOCK` | 1 | card's belt-and-braces; the engine pins weights by default on v2 (`HALOGEN_FLASH_PIN_TRUNK=0` is a w4b-only knob and is REFUSED by v2) |
|
||||
| `HALOGEN_MODEL_ID` / publish | frank-large / 7770:8731 | unchanged aliases; engine port 8730 never published |
|
||||
| MTP depth | default 2 | FLAGS.md: 2 is best all-round, 3 for code — Frank's traffic is agent/prose |
|
||||
|
||||
**Memory ledger** (121.2 GiB unified, the referee is the boot ledger):
|
||||
weights ~62 GiB resident + KV ~10.7 + arena ~8 (`3.8 + 8.4 × 8192/16384`)
|
||||
+ vision ~2 ≈ **82.7 GiB engine-side**, + 20 host reserve = ~103 with
|
||||
~18 GiB margin over the box's baseline co-tenancy. The 51.2 GB n-gram
|
||||
weights ~62 GiB resident + KV 14.0 + arena ~8 (`3.8 + 8.4 × 8192/16384`)
|
||||
+ vision ~2 ≈ **86 GiB engine-side**, + 20 host reserve = ~106 with
|
||||
~15 GiB margin over the box's baseline co-tenancy. The 51.2 GB n-gram
|
||||
table rides as mmap'd page cache beside the checkpoint — the card's own
|
||||
quickstart arithmetic does not bill it as resident, but WATCH the ledger
|
||||
at boot and re-size if the engine prints a clamp. Prompt cache is ON by
|
||||
@@ -177,11 +177,14 @@ the 27B engine ran on. Only revisit if the ledger shows SVM complaints.
|
||||
## Client contract (harness)
|
||||
|
||||
`~/.omp/agent/models.yml` — provider `llamacpp (dankclaw)` @
|
||||
`http://192.168.0.143:7770/v1`: `contextWindow: 200192`,
|
||||
`maxTokens: 57344`, `input: [text, image]`,
|
||||
`http://192.168.0.143:7770/v1`: `contextWindow: 217088` — deliberately
|
||||
BELOW the 262,144 engine ceiling: the 45k gap absorbs the harness's
|
||||
~3 % chars/4 est under-count (see the `HALOGEN_CTX` row above) —
|
||||
`maxTokens: 45056`, `input: [text, image]`,
|
||||
`reasoningContentField: "reasoning_content"`, the `high→xhigh` effort
|
||||
fold, `extraBody` temperature 1.0 / top_p 0.95. Compaction still rides
|
||||
frank-secondary (the `compactionModel` steering is unchanged).
|
||||
frank-secondary (the `compactionModel` steering is unchanged; the input
|
||||
≈ 170k fits its 200,704 slot).
|
||||
|
||||
## Rollback
|
||||
|
||||
|
||||
+28
-15
@@ -2,7 +2,7 @@
|
||||
# Seam: frank-large serves the Quat3rnion ABLITERATED Swift1.5 finetune of
|
||||
# peonist's Qwen3.8-Flash-Next v2 checkpoint (a .hgn flash-server payload)
|
||||
# on the Strix Halo iGPU through the halogen-flash-server CONTAINER (0.16.2,
|
||||
# digest-pinned), driven by the root unit's serve script — two 200,192-token
|
||||
# digest-pinned), driven by the root unit's serve script — two 262,144-token
|
||||
# slots with the MTP head always on (HALOGEN_SPEC_ADAPT=0) and the vision
|
||||
# tower mounted. 2026-10-06 cutover from the halogen 27B-dense shape.
|
||||
#
|
||||
@@ -80,8 +80,8 @@ good_env() {
|
||||
"FRANK_LARGE_VISION_BYTES=0"
|
||||
"FRANK_LARGE_TOKENIZER_FILES=tok.json"
|
||||
"HALOGEN_MODEL_ID=frank-large"
|
||||
"HALOGEN_CTX=200192"
|
||||
"HALOGEN_KV_POOL_POSITIONS=400384"
|
||||
"HALOGEN_CTX=262144"
|
||||
"HALOGEN_KV_POOL_POSITIONS=524288"
|
||||
"HALOGEN_KV_SLOTS=2"
|
||||
"HALOGEN_SPEC_ADAPT=0"
|
||||
"HALOGEN_MAX_TOK=8192"
|
||||
@@ -263,8 +263,8 @@ $(cat "$TEST_TMP/serve.log")"
|
||||
[[ "$output" == *"--name frank-large-halogen"* ]]
|
||||
[[ "$output" == *"--publish 7770:8731"* ]]
|
||||
[[ "$output" == *"-e HALOGEN_MODEL_ID=frank-large"* ]]
|
||||
[[ "$output" == *"-e HALOGEN_CTX=200192"* ]]
|
||||
[[ "$output" == *"-e HALOGEN_KV_POOL_POSITIONS=400384"* ]]
|
||||
[[ "$output" == *"-e HALOGEN_CTX=262144"* ]]
|
||||
[[ "$output" == *"-e HALOGEN_KV_POOL_POSITIONS=524288"* ]]
|
||||
[[ "$output" == *"-e HALOGEN_KV_SLOTS=2"* ]]
|
||||
[[ "$output" == *"-e HALOGEN_SPEC_ADAPT=0"* ]]
|
||||
[[ "$output" == *"-e HALOGEN_CHECKPOINT=/models/qwen.hgn"* ]]
|
||||
@@ -280,16 +280,29 @@ $(cat "$TEST_TMP/serve.log")"
|
||||
! grep -q 'PODMAN wait' "$TEST_TMP/serve.log"
|
||||
}
|
||||
|
||||
@test "the shipped env pins the fleet's window and shape" {
|
||||
# 200192 is the per-slot hard ceiling (a longer prompt is a 400
|
||||
# naming the limit, never a silent truncation), and the harness
|
||||
# compaction math must fit prompt + budget under it: OMP compacts at
|
||||
# 70% = 140134, and 140134 + 57344 = 197478 <= 200192. models.yml
|
||||
# contextWindow must quote the same number or the harness discovers
|
||||
# the wall as a 400.
|
||||
grep -qE '^HALOGEN_CTX=200192$' "$ENV_FILE"
|
||||
ctx=200192
|
||||
[ $(( ctx * 70 / 100 + 57344 )) -le "$ctx" ]
|
||||
@test "the shipped env keeps the harness fit law inside the ceiling" {
|
||||
# 262144 (the checkpoint's native window) is the per-slot hard ceiling
|
||||
# (a longer prompt is a 400 naming the limit, never a silent
|
||||
# truncation). The harness clamps max_tokens = min(45056, 217088 − est
|
||||
# − 4096), compacts at 85 % of its declared 217088 window, and its
|
||||
# chars/4 est UNDER-counts the rendered prompt by ~3 % on code/agent
|
||||
# traffic (measured 4,450 tokens at est 152,938 on 2026-10-07 — with
|
||||
# window == ceiling, every request over ~146k est was a guaranteed
|
||||
# 400 before compaction could fire at 85 %). The 45k gap between the
|
||||
# harness window and the engine ceiling must absorb that error at
|
||||
# both monotone maxima of est + budget (est ≤ 217088 − 4096):
|
||||
# full budget at the compaction threshold: 1.03*184524 + 45056
|
||||
# clamped budget at the clamp's zero point: 1.03*212992 + 0
|
||||
# and the compaction request that rides on frank-secondary (its
|
||||
# 200,704-token slot) must fit: input ≈ threshold − keepRecent(20k).
|
||||
grep -qE '^HALOGEN_CTX=262144$' "$ENV_FILE"
|
||||
ctx=262144
|
||||
hw=217088; mt=45056
|
||||
thr=$(( hw * 85 / 100 ))
|
||||
[ $(( thr * 103 / 100 + mt )) -le "$ctx" ]
|
||||
zero=$(( hw - 4096 ))
|
||||
[ $(( zero * 103 / 100 )) -le "$ctx" ]
|
||||
[ $(( (thr - 20000) * 103 / 100 )) -le 200704 ]
|
||||
}
|
||||
|
||||
@test "no HIP-sensitive bare names leak through the env file" {
|
||||
|
||||
Reference in New Issue
Block a user